Add support for CETL VariableLengthArray and polymorphic_allocator (O…

…penCyphal#316) This change modifies the C++ code-gen to create message classes that can be used with allocators. - The message class becomes an "_Impl" template that takes an allocator, which gets passed down to any variable length arrays in the messages (or in contained messages.) The normal message name is then aliased to the "_Impl" template using the allocator specified in the yaml config. In the end the message is a regular struct as it was before. - The old variable_length_array.hpp is removed from the support library. CETL is added as a submodule and the new variable_length_array.hpp from there is used. - Added a verification test to exercise a message using CETL's polymorphic_allocator
edu-rossrobotics · Sep 6, 2023 · f78e3f7 · f78e3f7
1 parent b2a82de
commit f78e3f7
Show file tree

Hide file tree

Showing 29 changed files with 643 additions and 2,849 deletions.
diff --git a/.gitmodules b/.gitmodules
@@ -10,3 +10,6 @@
 [submodule "submodules/o1heap"]
 	path = submodules/o1heap
 	url = https://github.com/pavel-kirienko/o1heap.git
+[submodule "submodules/CETL"]
+	path = submodules/CETL
+	url = https://github.com/OpenCyphal/CETL.git
diff --git a/.vscode/cmake-variants.json b/.vscode/cmake-variants.json
@@ -57,6 +57,14 @@
                     "NUNAVUT_VERIFICATION_LANG_STANDARD": "c++14"
                 }
             },
+            "CETL++": {
+                "short": "--std=cetl++14-17",
+                "long": "Compile and link using the C++14 standard and use CETL C++17 polyfill types.",
+                "settings": {
+                    "NUNAVUT_VERIFICATION_LANG": "cpp",
+                    "NUNAVUT_VERIFICATION_LANG_STANDARD": "cetl++14-17"
+                }
+            },
             "C++17": {
                 "short": "--std=c++17",
                 "long": "Compile and link using the C++17 standard.",
@@ -65,6 +73,14 @@
                     "NUNAVUT_VERIFICATION_LANG_STANDARD": "c++17"
                 }
             },
+            "C++17 PMR": {
+                "short": "--std=c++17-pmr",
+                "long": "Compile and link using the C++17 standard and use std polymorphic allocator.",
+                "settings": {
+                    "NUNAVUT_VERIFICATION_LANG": "cpp",
+                    "NUNAVUT_VERIFICATION_LANG_STANDARD": "c++17-pmr"
+                }
+            },
             "C++20": {
                 "short": "--std=c++20",
                 "long": "Compile and link using the C++20 standard.",

diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
@@ -80,8 +80,8 @@ skip the docker invocations and use ``tox -s``.
 
 To run the language verification build you'll need to use a different docker container::
 
-    docker pull ghcr.io/opencyphal/toolshed:ts20.4.1
-    docker run --rm -it -v $PWD:/workspace ghcr.io/opencyphal/toolshed:ts20.4.1
+    docker pull ghcr.io/opencyphal/toolshed:ts22.4.1
+    docker run --rm -it -v $PWD:/workspace ghcr.io/opencyphal/toolshed:ts22.4.1
     cd /workspace
     ./.github/verify.py -l c
     ./.github/verify.py -l cpp
@@ -141,7 +141,7 @@ way to ensure the example is correct especially if used in a trailing ``invisibl
         assert 'scotec_mcu_timer' == filter_to_snake_case(input)
 
 These tests are run as part of the regular pytest build. You can see the Sybil setup in the
-``conftest.py`` found under the ``src`` directory but otherwise shouldn't need to worry about
+``conftest.py`` found under the project directory but otherwise shouldn't need to worry about
 it. The simple rule is; if the docstring ends up in the rendered documentation then your
 ``code-block`` tests will be executed as unit tests.
 

diff --git a/docs/languages.rst b/docs/languages.rst
@@ -12,29 +12,55 @@ C++ (experimental)
 
 See :ref:`template-language-guide` until this section is more complete.
 
-==============================================
-Using a Different Variable-Length Array Type
-==============================================
+============================================================
+Using a Different Variable-Length Array Type and Allocator
+============================================================
 
-For now this tip is important for people using the experimental C++ support. To use :code:`std::vector` instead of the
-minimal build-in :code:`variable_length_array` type create a properties override yaml file and pass it to nnvg.
+For now this tip is important for people using the experimental C++ support.  To set which variable length array
+implementation to use, create a properties override yaml file and pass it to nnvg.  Specifying the use of an
+allocator is optional (If ctor_convention is set to "default" then the allocator_include and allocator_type
+properties don't need to be set.)
 
-vector.yaml
+Alternatively, you may specify the language standard argument as -std=c++17-pmr or -std=cetl++14-17 as short-hand for
+the following configurations shown below.  Note that "cetl++14-17" means target C++14 but use the CETL C++17 polyfill
+types.
+
+c++17-pmr.yaml
+"""""""""""""""""
+
+.. code-block :: yaml
+
+    nunavut.lang.cpp:
+      options:
+        variable_array_type_include: "<vector>"
+        variable_array_type_template: "std::vector<{TYPE}, {REBIND_ALLOCATOR}>"
+        variable_array_type_constructor_args: ""
+        allocator_include: "<memory>"
+        allocator_type: "std::pmr::polymorphic_allocator"
+        allocator_is_default_constructible: true
+        ctor_convention: "uses-trailing-allocator"
+
+cetl++14-17.yaml
 """""""""""""""""
 
 .. code-block :: yaml
 
     nunavut.lang.cpp:
       options:
-        variable_array_type_include: <vector>
-        variable_array_type_template: std::vector<{TYPE}>
+        variable_array_type_include: '"cetl/variable_length_array.hpp"'
+        variable_array_type_template: "cetl::VariableLengthArray<{TYPE}, {REBIND_ALLOCATOR}>"
+        variable_array_type_constructor_args: "{MAX_SIZE}"
+        allocator_include: '"cetl/pf17/sys/memory_resource.hpp"'
+        allocator_type: "cetl::pf17::pmr::polymorphic_allocator"
+        allocator_is_default_constructible: false
+        ctor_convention: "uses-trailing-allocator"
 
 nnvg command
 """"""""""""""""""
 
 .. code-block :: bash
 
-    nnvg --configuration=vector.yaml \
+    nnvg --configuration=c++17-pmr.yaml \  # or --configuration=cetl++14-17.yaml
          -l cpp \
         --experimental-languages \
         -I path/to/public_regulated_data_types/uavcan \

diff --git a/src/nunavut/cli/__init__.py b/src/nunavut/cli/__init__.py
@@ -484,6 +484,7 @@ def extension_type(raw_arg: str) -> str:
     ln_opt_group.add_argument(
         "--language-standard",
         "-std",
+        choices=["c11", "c++14", "cetl++14-17", "c++17", "c++17-pmr", "c++20"],
         help=textwrap.dedent(
             """
 

diff --git a/src/nunavut/cli/runners.py b/src/nunavut/cli/runners.py
@@ -155,17 +155,19 @@ def _create_language_context(self) -> LanguageContext:
 
         target_language_name = self._args.target_language
 
-        return (
-            LanguageContextBuilder(include_experimental_languages=self._args.experimental_languages)
-            .set_target_language(target_language_name)
-            .set_additional_config_files(additional_config_files)
-            .set_target_language_extension(self._args.output_extension)
-            .set_target_language_configuration_override(
-                Language.WKCV_NAMESPACE_FILE_STEM, self._args.namespace_output_stem
-            )
-            .set_target_language_configuration_override(Language.WKCV_LANGUAGE_OPTIONS, language_options)
-            .create()
+        builder: LanguageContextBuilder = LanguageContextBuilder(
+            include_experimental_languages=self._args.experimental_languages
+        )
+        builder.set_target_language(target_language_name)
+        builder.load_default_config(self._args.language_standard)
+        builder.set_additional_config_files(additional_config_files)
+        builder.validate_langauge_options()
+        builder.set_target_language_extension(self._args.output_extension)
+        builder.set_target_language_configuration_override(
+            Language.WKCV_NAMESPACE_FILE_STEM, self._args.namespace_output_stem
         )
+        builder.set_target_language_configuration_override(Language.WKCV_LANGUAGE_OPTIONS, language_options)
+        return builder.create()
 
     # +---------------------------------------------------------------------------------------------------------------+
     # | PRIVATE :: RUN METHODS

diff --git a/src/nunavut/lang/__init__.py b/src/nunavut/lang/__init__.py
@@ -220,6 +220,12 @@ def set_target_language(self, target_language: typing.Optional[str]) -> "Languag
             self._target_language_name = LanguageClassLoader.to_language_name(target_language)
         return self
 
+    def load_default_config(self, language_standard: str) -> None:
+        self._ln_loader.config.apply_defaults(language_standard)
+
+    def validate_langauge_options(self) -> None:
+        self._ln_loader.config.validate_language_options()
+
     def set_additional_config_files(
         self, additional_config_files: typing.List[pathlib.Path]
     ) -> "LanguageContextBuilder":

diff --git a/src/nunavut/lang/_config.py b/src/nunavut/lang/_config.py
@@ -10,10 +10,33 @@
 import types
 import typing
 
+from enum import auto, Enum
+
 from yaml import Loader as YamlLoader
 from yaml import load as yaml_loader
 from nunavut._utilities import deep_update
 
+NUNAVUT_LANG_CPP = "nunavut.lang.cpp"
+
+
+class ConstructorConvention(Enum):
+    Default = "default"
+    UsesLeadingAllocator = "uses-leading-allocator"
+    UsesTrailingAllocator = "uses-trailing-allocator"
+
+    @staticmethod
+    def parse_string(s: str) -> typing.Optional[typing.Any]:  # annoying mypy cheat due to returning type being defined
+        for e in ConstructorConvention:
+            if s == e.value:
+                return e
+        return None
+
+
+class SpecialMethod(Enum):
+    DefaultConstructorWithOptionalAllocator = auto()
+    CopyConstructorWithAllocator = auto()
+    MoveConstructorWithAllocator = auto()
+
 
 class LanguageConfig:
     """
@@ -595,6 +618,26 @@ def get_config_value_as_list(
 
         return default_value
 
+    def apply_defaults(self, language_standard: str) -> None:
+        defaults_key = f"{language_standard}_options"
+        if defaults_key in self.sections()[NUNAVUT_LANG_CPP]:
+            defaults_data = self.get_config_value_as_dict(NUNAVUT_LANG_CPP, defaults_key)
+            self.update_section(NUNAVUT_LANG_CPP, {"options": defaults_data})
+
+    def validate_language_options(self) -> None:
+        options = self.get_config_value_as_dict(NUNAVUT_LANG_CPP, "options")
+        ctor_convention_str: str = options["ctor_convention"]
+        ctor_convention = ConstructorConvention.parse_string(ctor_convention_str)
+        if not ctor_convention:
+            raise RuntimeError(
+                f"ctor_convention property '{ctor_convention_str}' is invalid and must be one of "
+                + (",".join([f"'{e.value}'" for e in ConstructorConvention]))
+            )
+        if ctor_convention != ConstructorConvention.Default and not options["allocator_type"]:
+            raise RuntimeError(
+                f"allocator_type property must be specified when ctor_convention is '{ctor_convention_str}'"
+            )
+
 
 # +-------------------------------------------------------------------------------------------------------------------+
 # | VersionReader

diff --git a/src/nunavut/lang/_language.py b/src/nunavut/lang/_language.py
@@ -222,6 +222,10 @@ def named_values(self) -> typing.Mapping[str, str]:
     # | METHODS
     # +-----------------------------------------------------------------------+
 
+    def _add_additional_globals(self, globals_map: typing.Dict[str, typing.Any]) -> None:
+        """Subclasses may override this method to populate additional language-specific globals"""
+        pass
+
     def get_support_module(self) -> typing.Tuple[str, typing.Tuple[int, int, int], typing.Optional["types.ModuleType"]]:
         """
         Returns the module object for the language support files.
@@ -469,6 +473,8 @@ def get_globals(self) -> typing.Mapping[str, typing.Any]:
             for key, value in self.named_values.items():
                 globals_map["valuetoken_{}".format(key)] = value
 
+            self._add_additional_globals(globals_map)
+
             self._globals = globals_map
         return self._globals