Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[P4Testgen] Extend the CompilerTarget runProgram function with data structures which can pass on more information. #4323

Merged
merged 2 commits into from
Jan 12, 2024

Conversation

fruffy
Copy link
Collaborator

@fruffy fruffy commented Jan 10, 2024

Major rewrite of the ProgramInfo and CompilerTarget infrastructure. The basic idea is to compute all necessary static analysis information in the compilation phase, then pass this information forward to ProgramInfo.

For this, we introduce a CompilerResult object. This object is produced in the compilation phase. Any P4Tools module can add arbitrary information to this class. The results object is then passed to ProgramInfo.

This PR is part 1. It introduces the CompilerResult data structure, which records information produces by the compiler which runs before P4Testgen is invoked. This includes the P4 program after the mid end for now.

@fruffy fruffy changed the title Extend the CompilerTarget runProgram function with data structures which can pass on more information. [P4Testgen] Extend the CompilerTarget runProgram function with data structures which can pass on more information. Jan 10, 2024
@fruffy fruffy added the p4tools Topics related to the P4Tools back end label Jan 10, 2024
@fruffy fruffy force-pushed the fruffy/compiler_info_1 branch from e437009 to be3c6bd Compare January 10, 2024 14:28
Copy link
Contributor

@vlstill vlstill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be really good to have more info about how this would be used and what would be the next steps. At this point, this just introduces a data structure that is not used in the end. I presume #4324 is the next step -- I'll have a look at that too as I can't really evaluate this one in isolation.

///
/// @returns std::nullopt if an error occurs during compilation.
static std::optional<const IR::P4Program *> runCompiler();
static std::optional<const CompilerResult *> runCompiler();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason to return optional of a pointer? I don't see std::nullopt meaning something different than optional with nullptr (and if it did it would be quite confusing I think). I suggest using std::optional<CompilerResult> as the struct will likely mainly contain pointers/references and it will not be copied much around so this will matter even if it become larger.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pointer is necessary here because CompilerResult will be subclassed by the target back ends, e.g., BMv2CompilerResult etc. It could also be a reference, but that would require a reference_wrapper.

I much prefer using std::optional over nullptr these days because it makes it very explicit that an error has happened. Going forward it will also be easier to replace this with std::expect or absl:StatusOr.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I did not realize that CompilerResult will be subclassed, that does make a lot of sense. I don't like that std::optional<pointer> solution much because it has the ugly state where the optional contains nullptr. You are right that optional is more explicit about having to resolve the error though :-/ (and std::optional<std::reference_wrapper<X>> looks ugly and has the additional disadvantage that auto x = opt.value() behaves weirdly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm starting to think that std::optional<std::reference_wrapper<const CompilerResult>> is the right option here if you want the added safety of optional. This is a bit uglier to use (the type is long, and one needs to explicitly call .get() in many cases). But it is unlikely the auto problem I mentioned above will cause problems exactly because one has to call get() to get the actual instance. And having std::optional<const CompilerResult *>(nullptr) would be really confusing. Too bad std does not allow std::optional<const CompilerResult &> :-/.

backends/p4tools/modules/testgen/testgen.cpp Show resolved Hide resolved
@fruffy
Copy link
Collaborator Author

fruffy commented Jan 11, 2024

It would be really good to have more info about how this would be used and what would be the next steps. At this point, this just introduces a data structure that is not used in the end. I presume #4324 is the next step -- I'll have a look at that too as I can't really evaluate this one in isolation.

The idea is that you can pass any information from the compiler mid end downstream to any P4Tools module.
https://github.com/p4lang/p4c/pull/4292/files#diff-879c9c261959c28be049f25e50b7e78ed0bf569abac22a049abe0bab107e6277R67 contains an example. We can compute most of the information we previously had to compute in the ProgramInfo class in the mid end instead. Among others, we can now interleave computation between front and mid end, which is necessary for passes such as the P4Runtime info generator.

This refactoring also ensures that we really only need to maintain one data structure for all this information, the CompilerInfo. And that info is very explicitly produce by the compiler, not somewhere in the tools framework. Previously, all the program info classes had to awkwardly apply passes in their constructor, now they all consume this constant CompilerInfo object.

@fruffy fruffy force-pushed the fruffy/compiler_info_1 branch 2 times, most recently from aa9b438 to e9a24db Compare January 12, 2024 16:18
@@ -31,7 +31,7 @@ std::optional<const P4ToolsTestCase> P4ToolsTestCase::create(
if (!compilerResults.has_value()) {
return std::nullopt;
}
return P4ToolsTestCase{compilerResults.value()->getProgram()};
return P4ToolsTestCase(compilerResults.value().get().getProgram());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you have already checked for nullopt, you can use compilerResults->get() if you want (operator-> and operator* are unchecked, unlike value()).

@fruffy fruffy force-pushed the fruffy/compiler_info_1 branch from e9a24db to 227ff4d Compare January 12, 2024 17:57
@fruffy fruffy merged commit 313faed into main Jan 12, 2024
13 checks passed
@fruffy fruffy deleted the fruffy/compiler_info_1 branch January 12, 2024 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p4tools Topics related to the P4Tools back end
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants