-
Notifications
You must be signed in to change notification settings - Fork 446
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Combo of duplication action name check and LocalizeAllActions fix #4975
base: main
Are you sure you want to change the base?
Combo of duplication action name check and LocalizeAllActions fix #4975
Conversation
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
…ce files that are the output of the midend pass of the compiler, because they can include intentional name duplication created in the LocalizeActions pass. + P4-16 source files created by translating a P4-14 source file to P4-16. Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu> Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Explicitly allow a table and action to have the same name, since there is a test case that was written explicitly to test that this is permitted. Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
and for some others, update the error messages we expect to see. Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
Signed-off-by: Andy Fingerhut <andy.fingerhut@gmail.com>
…rchical-name-check-pass
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
…om:jafingerhut/p4c into add-duplicate-hierarchical-name-check-pass
…rchical-name-check-pass
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
…rchical-name-check-pass
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
so that it prepends a "." to @name annotations of global actions, if they do not have one already. Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
…ations Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
@@ -0,0 +1,185 @@ | |||
/* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@chrispsommers @smolkaj This is one of the two new P4 test programs I would recommend you focus your review on. The comments indicate which parts of the program I expect should give errors, and you can look at the corresponding testdata/p4_16_errors_output/actions-with-duplicate-control-plane-names1.p4-stderr file here to see that it does give errors for those: https://github.com/p4lang/p4c/pull/4975/files#diff-30d824dc2073bd520ea90c54fc52935bad924ddc326b1ab76e225e9c94bacae5
@@ -0,0 +1,173 @@ | |||
/* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@chrispsommers @smolkaj This is the second of the two new P4 test programs I would recommend you focus your review on. The comments indicate which parts of the program I expect should not give errors, and they do not. You might also want to look through the contents of the expected generated P4Info file named testdata/p4_16_samples_outputs/actions-almost-duplicate-names1.p4.p4info.txtpb here: https://github.com/p4lang/p4c/pull/4975/files#diff-958b36defef5fa0260fb1c4424a06e6f725d05c1853b99070b691a48ee458c7b
…n-action-name-check-and-localizeallactions-fix
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
... the recent annotations infra changes Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jafingerhut Thanks for taking cate of this!
I have not completed the tests yet,
Please find a few comments for the C++ part below.
#include "ir/ir.h" | ||
#include "ir/visitor.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#include <vector>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added in commit 57
visitDagOnce = false; | ||
} | ||
const IR::Node *preorder(IR::P4Parser *parser) override { | ||
stack.push_back(getName(parser)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if we call prune()
to prevent the visitor from going deeper into the parser which cannot have actions?
I think, pushing the parser onto the stack can be removed as we never use parser names to build hierarchical action names.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed there should be no need to walk through sub-nodes within a parser.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in commit 57
const IR::Node *postorder(IR::P4Parser *parser) override { | ||
stack.pop_back(); | ||
return parser; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we apply the change proposed in the previous comment, this overload can be removed, I believe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in commit 57
const IR::Node *preorder(IR::P4Table *table) override { | ||
visit(table->annotations); | ||
prune(); | ||
return table; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please elaborate why we need to visit the table's annotations here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It most likely is not needed, and I will try removing it. This looks like something left over from when I started by copying-and-pasting the HierarchicalNames pass, and never noticed it could/should be removed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left the method, but removed the visit call, but left the call to prune, since there should be no need to traverse the sub-nodes of a P4 table.
class DuplicateActionControlPlaneNameCheck : public Transform { | ||
std::vector<cstring> stack; | ||
/// Used for detection of conflicting control plane names among actions. | ||
string_map<const IR::Node *> actions; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
string_map
preserves the insertion order.
This is not needed for this PR as it never iterates through the elements.
Keeping the order also consumes extra resources (std::list
+ its memory allocations).
What if we replace this with absl::flat_hash_map
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes in commit 57
} | ||
|
||
const IR::Node *DuplicateActionControlPlaneNameCheck::postorder(IR::P4Action *action) { | ||
bool topLevel = findContext<IR::P4Control>() == nullptr; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bool topLevel = !stack.empty();
has some chances to be a tiny bit more efficient, I believe.
(this requires the previously suggested change for not pushing parsers onto the stack).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to bool topLevel = stack.empty();
in commit 57
cstring name = absl::StrCat(".", action->name); | ||
checkForDuplicateName(name, action); | ||
} else { | ||
const auto *e0 = nameAnno->expr.at(0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const auto *e0 = nameAnno->expr.at(0); | |
const auto *e0 = nameAnno->getExpr(0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in commit 55, IIRC
const IR::Node *node) { | ||
bool foundDuplicate = false; | ||
auto *otherNode = node; | ||
auto [it, inserted] = actions.insert(std::pair(name, node)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
auto [it, inserted] = actions.insert(std::pair(name, node)); | |
auto [it, inserted] = actions.emplace(name, node); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in commit 57
auto [it, inserted] = actions.insert(std::pair(name, node)); | ||
if (!inserted) { | ||
foundDuplicate = true; | ||
otherNode = (*it).second; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
otherNode = (*it).second; | |
otherNode = it->second; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in commit 57
foundDuplicate = true; | ||
otherNode = (*it).second; | ||
} | ||
if (foundDuplicate) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could probably fold into !inserted
condition above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in commit 57
frontends/p4/localizeActions.cpp
Outdated
} else { | ||
cstring name = absl::StrCat(".", action->name); | ||
action->addAnnotationIfNew(IR::Annotation::nameAnnotation, new IR::StringLiteral(name), | ||
false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
false
is default, could be omitted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed in commit 57
frontends/p4/localizeActions.cpp
Outdated
// If the value of the existing name annotation does not | ||
// begin with ".", prepend "." so that the name remains | ||
// global if control plane APIs are generated later. | ||
const auto *e0 = nameAnno->expr.at(0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const auto *e0 = nameAnno->expr.at(0); | |
const auto *e0 = nameAnno->getExpr(0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in commit 55, IIRC
frontends/p4/localizeActions.cpp
Outdated
cstring name = absl::StrCat(".", action->name); | ||
action->addAnnotationIfNew(IR::Annotation::nameAnnotation, new IR::StringLiteral(name), | ||
false); | ||
auto nameAnno = action->getAnnotation(IR::Annotation::nameAnnotation); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could move declaration into if
:
if (const auto *nameAnno = ...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in commit 57
…-localizeallactions-fix
…tion Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
No worries. This is an issue I noticed way back in 2019, and it has occasionally bugged me over the years whenever I notice it again. Glad if we can get it fixed. Thanks for the comments. If it is not obvious, C++ is not something I would call myself an expert in. I believe I have addressed the current round of comments completely in commits today. While I believe these changes address the questions in the initial comment of this issue: #4651 it does not attempt to make any changes regarding Kyle's follow-up comment here: #4651 (comment) As I have tried to state elsewhere, that is caused by the LocalizeAllActions pass, and it is by design. I would recommend either: (a) realize that this is what LocalizeAllActions does, and write later passes, and your target-specific back end code, to take it into account that there can be multiple actions with the same name, but with the changes in this PR, I believe we can guarantee that all actions with the same hierarchical name were created by LocalizeAllActions, not by the P4 developer. OR (b) figure out a way in your version of p4c to avoid using the LocallizeAllActions pass. That is not as simple as just commenting out that pass, I don't think. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have reviewed only the C++ changes + the testdata/p4_16_errors/actions-with-duplicate-control-plane-names1.p4
test and related outputs. I will let others review the rest of the changes to the tests outputs. Aside from one suggestion, the parts that I have reviewed LGTM.
/// Indicates whether control plane API generation is enabled. | ||
/// @returns default to false unless a command line option was | ||
/// given explicitly enabling control plane API generation. | ||
virtual bool controlPlaneAPIGenEnabled() const { | ||
if (p4RuntimeFile.isNullOrEmpty() && p4RuntimeFiles.isNullOrEmpty() && | ||
p4RuntimeEntriesFile.isNullOrEmpty() && p4RuntimeEntriesFiles.isNullOrEmpty()) { | ||
return false; | ||
} | ||
return true; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@asl @oleg-ran-amd I think that DuplicateActionControlPlaneNameCheck
should run for our backend even when P4Info files are not generated. So we should probably override this function to unconditionally return true
.
Signed-off-by: Andy Fingerhut <andy_fingerhut@alum.wustl.edu>
…n-action-name-check-and-localizeallactions-fix
I have updated this initial comment with the status of the PR after 52 commits. The most recent changes were simply to update the C++ implementation so they worked after #4992 was merged in.
This PR is a combination of the changes in what was originally two separate PRs, but it seems to me easier to manage if they are combined and reviewed as one. The two earlier PRs were #4951 and #4970. Some of the discussion on comments in #4951 may still provide valuable context.
These changes are intended to address the problem where p4c currently allows a user to have identical
@name
annotations on multiple objects, e.g. on multiple actions. This can cause incorrect P4Info files to be generated, among other likely problems resulting from that root cause. It also addresses a minor bug in the LocalizeAllActions pass that produces wrong @name annotations for some top-level actions. See this slide deck for examples and more details:The code changes here are based upon this PR, which stalled a while ago: #3228
These changes are noticeably different from that PR, however, in that this one adds a new pass that occurs before the LocalizeAllActions pass, which is the pass where p4c by design can synthesize actions that are duplicates of user-written actions, all of which have identical
@name
annotations. Unless we change that behavior, any checks for duplicate@name
annotations must be done before that pass. Such a change is described verbally in this comment on the PR above: #3228 (comment) but to my knowledge has not been implemented before. This PR is trying to implement that approach.It passes all of the CI tests, after modifying a handful of P4 programs that had the error being checked for, so they no longer had the error. One of those was preserved to verify that p4c catches the error with the expected error messages.
The new pass is only enabled if one or more of the command line options are given to generate P4Info output files. The duplicate @name annotations checked for in the new pass only cause problems (that I am aware of) if P4Info output files are being generated. This is similar to some existing checks for duplicate names of P4 objects in the control plane serialization code for P4Info.
There have been multiple related issues filed on p4c over several years that this PR might address, at least partially if not completely.
These issues I believe are fully fixed by this PR:
p4c
should prohibit different controllable entities having the same control-plane name #4651p4c
should prohibit different controllable entities having the same control-plane name #4651's second sub-issue).If this pull request is merged, then this earlier (apparently abandoned) PR should be closed:
The issue below should remain open even if this PR is merged, as the issue is really asking questions about what names should be used for controllable P4 objects in a P4Info file. This PR does not change any names in P4Info files -- it only gives errors and prevents P4Info files from being generated when the current names would conflict:
@name
annotations when generating P4Info files? #2716I still need to re-read and think about whether the following issues are fixed by this PR, or whether there are other questions in them that are not fixed by this PR: