Introduce a new sample that checks macro naming conventions (boostorg#97

) * Introduce a new sample that checks macro naming conventions This has a direct use in Boost's own QA checks and demonstrates the use of macro definition and include guard callbacks, as well as Boost.Filesystem.
giomasce-throwaway · Jun 15, 2020 · 8504d24 · 8504d24
1 parent 320932e
commit 8504d24
Show file tree

Hide file tree

Showing 5 changed files with 340 additions and 0 deletions.
diff --git a/doc/samples.html b/doc/samples.html
@@ -94,6 +94,25 @@ <h2>The hannibal sample </h2>
 <p>The <tt>hannibal</tt> sample shows how to base a spirit grammar on the <tt>Wave</tt> library. It was initially written and contributed to the <tt>Wave</tt> library by Danny Havenith (see his related web page <a href="http://havenith-verlinden.nl/hannibal/">here</a>). The grammar of this example uses <tt>Wave</tt> as its preprocessor. It implements around 120 of the approximately 250 grammar rules as they can be   found in <em>The C++ Programming Language, Third Edition</em>. The 120 rules allow   a C++ source file to be parsed for all type information and declarations. In   fact this grammar parses as good as anything, it parses C++ declarations, including class and template definitions, but skips function bodies. If so configured, the program will output an xml dump of the generated parse tree. </p>
 <p>It may be a good starting point for a grammar that can be used for things like   reverse engineering as some UML modelling tools do. Or whatever use you may find   for a grammar that gives you a list of all templates and classes in a file and their members. <br>
 </p>
+<h2 dir="ltr">The check_macro_naming sample</h2>
+<p>The <tt>check_macro_naming</tt> sample demonstrates the use of context hooks to understand how macros are defined within a codebase. Some projects (such as Boost itself) have conventions on the names of macros. This sample will recursively search a directory, looking for header files and searching each for macro definitions. Any that do not match the supplied convention (which defaults to <tt>^BOOST_.*</tt>) are reported, along with an annotation if they are used as an include guard. The user can also specify any number of directories to ignore in the process.</p>
+<p dir="ltr">Command line syntax:</p>
+<pre>
+Usage: check_macro_naming [options] directory:
+  -h [ --help ]            print out options
+  --match arg (=^BOOST_.*) pattern defined macros must match
+  --exclude arg            subdirectory to skip
+</pre>
+<p dir="ltr">Example usage:</p>
+<pre>
+$ check_macro_naming --exclude ./test/testwave/testfiles include
+CPP_CONTEXT_HPP_907485E2_6649_4A87_911B_7F7225F3E5B8_INCLUDED include/boost/wave/cpp_context.hpp (guard)
+WHITESPACE_HANDLING_HPP_INCLUDED include/boost/wave/whitespace_handling.hpp (guard)
+...
+TRACE_CPP_TIME_CONVERSION include/boost/wave/util/time_conversion_helper.hpp
+spirit_append_actor include/boost/wave/util/time_conversion_helper.hpp
+spirit_assign_actor include/boost/wave/util/time_conversion_helper.hpp
+</pre>
 <table border="0">
   <tr>
     <td width="10"></td>

diff --git a/samples/Jamfile.v2 b/samples/Jamfile.v2
@@ -19,3 +19,4 @@ build-project token_statistics/build ;
 build-project preprocess_pragma_output/build ;
 build-project custom_directives/build ;
 build-project emit_custom_line_directives/build ;
+build-project check_macro_naming/build ;
diff --git a/samples/check_macro_naming/build/Jamfile.v2 b/samples/check_macro_naming/build/Jamfile.v2
@@ -0,0 +1,20 @@
+# Boost.Wave: A Standard compliant C++ preprocessor library
+#
+# Boost Wave Library Sample Build Jamfile (check_macro_naming)
+#
+# http://www.boost.org/
+#
+# Copyright (c) 2001-2010 Hartmut Kaiser.
+# Copyright (c) 2020 Jeff Trull. Distributed under the Boost
+# Software License, Version 1.0. (See accompanying file
+# LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
+
+exe check_macro_naming
+    :   ../check_macro_naming.cpp
+        /boost/wave//boost_wave
+        /boost/filesystem//boost_filesystem
+        /boost/thread//boost_thread
+        /boost/regex//boost_regex
+        /boost/program_options//boost_program_options
+    ;
+
diff --git a/samples/check_macro_naming/check_macro_naming.cpp b/samples/check_macro_naming/check_macro_naming.cpp
@@ -0,0 +1,178 @@
+/*=============================================================================
+    Boost.Wave: A Standard compliant C++ preprocessor library
+    Sample demonstrating the usage of advanced preprocessor hooks.
+
+    http://www.boost.org/
+
+    Copyright (c) 2001-2010 Hartmut Kaiser.
+    Copyright (c) 2020 Jeff Trull. Distributed under the Boost
+    Software License, Version 1.0. (See accompanying file
+    LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
+=============================================================================*/
+
+#include "check_macro_naming.hpp"
+#include "libs/filesystem/include/boost/filesystem/file_status.hpp"
+
+///////////////////////////////////////////////////////////////////////////////
+//  Utilities from the rest of Boost
+#include <boost/program_options.hpp>
+#include <boost/filesystem.hpp>
+#include <boost/regex.hpp>
+#include <boost/bind/bind.hpp>
+
+///////////////////////////////////////////////////////////////////////////////
+//  Wave itself
+#include <boost/wave.hpp>
+
+///////////////////////////////////////////////////////////////////////////////
+// Include the lexer stuff
+#include <boost/wave/cpplexer/cpp_lex_token.hpp>    // token class
+#include <boost/wave/cpplexer/cpp_lex_iterator.hpp> // lexer class
+
+#include <iostream>
+#include <string>
+#include <set>
+#include <algorithm>
+
+void process_header(std::string const & filename,
+                    boost::regex const & re) {
+    using namespace boost::wave;
+
+    // data to collect
+    std::string include_guard;        // macro that protects this file, if any
+    std::set<std::string> bad_macros; // misnamed macros in this file
+
+    try {
+        // create a fake main program in memory to include our header from
+        std::string fake_main("#include \"");
+        fake_main += filename;
+        fake_main += "\"\n";
+
+        typedef cpplexer::lex_token<> token_type;
+        typedef cpplexer::lex_iterator<token_type> lex_iterator_type;
+        typedef context<std::string::iterator, lex_iterator_type,
+                        iteration_context_policies::load_file_to_string,
+                        macroname_preprocessing_hooks>
+            context_type;
+
+        context_type ctx (fake_main.begin(), fake_main.end(), "in-memory.cpp",
+                          macroname_preprocessing_hooks(re, bad_macros,
+                                                        include_guard));
+
+        // consume input, letting the hooks do the work
+        context_type::iterator_type last = ctx.end();
+        for (context_type::iterator_type it = ctx.begin(); it != last; ++it);
+
+        std::set<std::string>::const_iterator beg = bad_macros.begin();
+        std::set<std::string>::const_iterator end = bad_macros.end();
+        if (beg != end) {
+            // we have some macros that don't follow convention
+            for (std::set<std::string>::const_iterator it = beg;
+                 it != end; ++it) {
+                std::cout << *it << " " << filename;
+                if (*it == include_guard)
+                    std::cout << " (guard)\n";
+                else
+                    std::cout << "\n";
+            }
+        }
+    } catch (preprocess_exception const& e) {
+        // some preprocessing error
+        std::cerr
+            << e.file_name() << "(" << e.line_no() << "): "
+            << e.description() << std::endl;
+    } catch (cpplexer::lexing_exception const& e) {
+        std::cerr
+            << e.file_name() << "(" << e.line_no() << "): "
+            << e.description() << std::endl;
+    }
+}
+
+///////////////////////////////////////////////////////////////////////////////
+//  Main entry point
+//
+//  This sample shows how to check macros defined in header files to ensure
+//  they conform to a standard naming convention. It uses hooks for
+//  macro definition and include guard events to collect and report results.
+
+int main(int argc, char *argv[])
+{
+
+    // argument processing
+    namespace po = boost::program_options;
+    po::options_description visible("Usage: check_macro_naming [options] directory");
+    // named arguments
+    visible.add_options()
+        ("help,h", "print out options")
+        ("match", po::value<std::string>()->default_value("^BOOST_.*"),
+         "pattern defined macros must match")
+        ("exclude", po::value<std::vector<std::string> >(), "subdirectory to skip");
+
+    // positional arguments
+    po::positional_options_description p;
+    p.add("dirname", 1);
+    // this positional option should not be displayed as a named, so we separate it:
+    po::options_description hidden;
+    hidden.add_options()("dirname", po::value<std::string>());
+
+    // combine visible and hidden for parsing:
+    po::options_description desc;
+    desc.add(visible).add(hidden);
+
+    po::variables_map vm;
+    po::store(po::command_line_parser(argc, argv)
+              .options(desc)
+              .positional(p)
+              .run(),
+              vm);
+    po::notify(vm);
+
+    if (vm.count("help") || (vm.count("dirname") == 0)) {
+        std::cerr << visible << "\n";
+        std::cerr << "recursively traverse directory, reporting macro definitions ";
+        std::cerr << "that do not conform to the supplied pattern\n";
+        return 1;
+    }
+
+    // get named parameters
+    boost::regex macro_regex(vm["match"].as<std::string>());
+    std::vector<std::string> exclude_dirnames;
+    if (vm.count("exclude"))
+        exclude_dirnames = vm["exclude"].as<std::vector<std::string> >();
+
+    // get our single positional parameter - the directory to process
+    std::string dirname = vm["dirname"].as<std::string>();
+
+    // directory traversal logic
+    static const boost::regex header_regex(".*\\.(hh|hpp|h)$");
+    namespace fs = boost::filesystem;
+    std::vector<fs::path> exclude_dirs(exclude_dirnames.begin(),
+                                       exclude_dirnames.end());
+    // canonicalize exclude directories for comparison vs.
+    // search directories - either may be relative
+    typedef fs::path (*canonicalizer)(fs::path const&, fs::path const&);
+    using namespace boost::placeholders;
+    std::transform(exclude_dirs.begin(), exclude_dirs.end(),
+                   exclude_dirs.begin(),
+                   boost::bind(static_cast<canonicalizer>(&fs::canonical),
+                               _1, fs::current_path()));
+
+    fs::recursive_directory_iterator dir_end;
+    fs::recursive_directory_iterator dir_beg(dirname);
+    for (fs::recursive_directory_iterator it = dir_beg; it != dir_end; ++it) {
+        if (it->status().type() == fs::regular_file) {
+            std::string fn = it->path().native();
+            if (regex_match(fn, header_regex))
+                process_header(fn, macro_regex);
+        }
+        if ((it->status().type() == fs::directory_file) &&
+            (std::find(exclude_dirs.begin(),
+                       exclude_dirs.end(),
+                       fs::canonical(it->path())) != exclude_dirs.end())) {
+            // skip recursion here
+            it.no_push();
+        }
+    }
+
+    return 0;
+}
diff --git a/samples/check_macro_naming/check_macro_naming.hpp b/samples/check_macro_naming/check_macro_naming.hpp
@@ -0,0 +1,122 @@
+/*=============================================================================
+    Boost.Wave: A Standard compliant C++ preprocessor library
+    http://www.boost.org/
+
+    Copyright (c) 2001-2010 Hartmut Kaiser.
+    Copyrigth (c) 2020 Jeff Trull. Distributed under the Boost
+    Software License, Version 1.0. (See accompanying file
+    LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
+=============================================================================*/
+
+#if !defined(BOOST_WAVE_CHECK_MACRO_NAMING_INCLUDED)
+#define BOOST_WAVE_CHECK_MACRO_NAMING_INCLUDED
+
+#include <boost/wave/token_ids.hpp>
+#include <boost/wave/preprocessing_hooks.hpp>
+#include <boost/regex.hpp>
+
+#include <string>
+#include <set>
+
+///////////////////////////////////////////////////////////////////////////////
+//
+//  The macroname_preprocessing_hooks policy class is used to record the
+//  use of macros within a header file.
+//
+//  This policy type is used as a template parameter to the
+//  boost::wave::context<> object.
+//
+///////////////////////////////////////////////////////////////////////////////
+class macroname_preprocessing_hooks
+:   public boost::wave::context_policies::default_preprocessing_hooks
+{
+public:
+    macroname_preprocessing_hooks(boost::regex const & macro_regex,
+                                  std::set<std::string>& bad_macros,
+                                  std::string& include_guard)
+        : macro_regex_(macro_regex),
+          bad_macros_(bad_macros),
+          include_guard_(include_guard),
+          suppress_includes_(false)
+    {}
+
+    ///////////////////////////////////////////////////////////////////////////
+    //
+    //  Monitor macro definitions to verify they follow the required convention
+    //  by overriding the defined_macro hook
+    //
+    ///////////////////////////////////////////////////////////////////////////
+
+    template <typename ContextT, typename TokenT,
+              typename ParametersT, typename DefinitionT>
+    void defined_macro(ContextT const & /* ctx */, TokenT const &name,
+                       bool /* is_functionlike */, ParametersT const & /* parameters */,
+                       DefinitionT const & /* definition */, bool is_predefined)
+    {
+        using namespace boost::wave;
+        if (!is_predefined &&
+            !regex_match(name.get_value().c_str(), macro_regex_))
+            bad_macros_.insert(name.get_value().c_str());
+    }
+
+    // Wave only reports include guards in files that were actually included
+    // as a result we have to mock up the inclusion process. This means
+    // constructing a fake "includer" file in memory, and only permitting one
+    // level of includes (as we only want to analyze the header itself)
+
+    ///////////////////////////////////////////////////////////////////////////
+    //
+    //  Suppress includes of files other than the one we are analyzing
+    //  using found_include_directive
+    //
+    ///////////////////////////////////////////////////////////////////////////
+
+    template <typename ContextT>
+    bool found_include_directive(ContextT const& /* ctx */,
+                                 std::string const & filename,
+                                 bool /* include_next */)
+    {
+        return suppress_includes_;
+    }
+
+    ///////////////////////////////////////////////////////////////////////////
+    //
+    //  Suppress includes beyond the first level by setting our flag
+    //  from opened_include_file
+    //
+    ///////////////////////////////////////////////////////////////////////////
+
+    template <typename ContextT>
+    void opened_include_file(ContextT const& /* ctx */,
+                             std::string const & /* rel_filename */,
+                             std::string const & /* abs_filename */,
+                             bool /* is_system_include */)
+    {
+        suppress_includes_ = true;
+    }
+
+    // we only study one file, so no need to restore the ability to include
+
+    ///////////////////////////////////////////////////////////////////////////
+    //
+    //  Record detected include guard macros
+    //
+    ///////////////////////////////////////////////////////////////////////////
+
+    template <typename ContextT>
+    void detected_include_guard(ContextT const& /* ctx */,
+                                std::string filename,
+                                std::string const& include_guard)
+    {
+        include_guard_ = include_guard;
+    }
+
+
+private:
+    boost::regex const & macro_regex_;
+    std::set<std::string>& bad_macros_;
+    std::string& include_guard_;
+    bool suppress_includes_;
+};
+
+#endif // !defined(BOOST_WAVE_CHECK_MACRO_NAMING_INCLUDED)