Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Combo of duplication action name check and LocalizeAllActions fix #4975

Open
wants to merge 62 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
e429c8a
Add pass to give error if duplicate hierarchical names are present
jafingerhut Oct 7, 2024
39538bf
Add new source files to CMakeLists.txt
jafingerhut Oct 7, 2024
98940fb
Skip new checks when compiling certain source files including: + Sour…
jafingerhut Oct 8, 2024
a2792ce
Skip new checks for P4-14 source files in run-bmv2-test.py
jafingerhut Oct 8, 2024
a553fe1
Fix lint error
jafingerhut Oct 8, 2024
26f0501
Fix lint problems.
jafingerhut Oct 8, 2024
da456fe
Do not give errors for variable declarations with identical names
jafingerhut Oct 8, 2024
b683571
Limit search for duplicate names to actions, tables, or "other objects"
jafingerhut Oct 8, 2024
ad40fe9
Update test programs that had errors in their name annotations
jafingerhut Oct 9, 2024
91166c8
Update expected output files for modified test programs
jafingerhut Oct 9, 2024
9acb7e0
Move new test program with error into p4_16_errors directory
jafingerhut Oct 9, 2024
b133b28
Add expected output files for new test programs
jafingerhut Oct 9, 2024
76bf24f
More fine-tuning to enable tests to pass
jafingerhut Oct 9, 2024
f7ec2fe
Fix lint error
jafingerhut Oct 9, 2024
1cc8224
Fix lint error
jafingerhut Oct 9, 2024
4b03eb5
Merge branch 'master' into add-duplicate-hierarchical-name-check-pass
jafingerhut Oct 9, 2024
93daba5
Address some review comments
jafingerhut Oct 9, 2024
66a0a0a
Rewording documentation of new pass for clarity.
jafingerhut Oct 9, 2024
083db34
Merge remote-tracking branch 'upstream/main' into add-duplicate-hiera…
jafingerhut Oct 12, 2024
99beb30
Enable the new pass only if P4Runtime control plane API gen is enabled
jafingerhut Oct 20, 2024
2b7c58a
Merge branch 'add-duplicate-hierarchical-name-check-pass' of github.c…
jafingerhut Oct 20, 2024
54007df
Merge remote-tracking branch 'upstream/main' into add-duplicate-hiera…
jafingerhut Oct 20, 2024
d89f214
Undo changes to Python test scripts
jafingerhut Oct 20, 2024
7424db3
Undo changes to gtestp4c source
jafingerhut Oct 20, 2024
55ad18f
Merge remote-tracking branch 'upstream/main' into add-duplicate-hiera…
jafingerhut Oct 20, 2024
799448a
Use abseil StrCat and StrAppend instead of loop
jafingerhut Oct 21, 2024
377f41a
Fix formatting with clang-format
jafingerhut Oct 21, 2024
682564b
Fix #4969 by modifying pass LocalizeActions
jafingerhut Oct 21, 2024
bb88b7d
Fix clang-format error
jafingerhut Oct 21, 2024
d265e1c
Also catch duplicates involving top-level actions without @name annot…
jafingerhut Oct 21, 2024
606300e
Merge remote-tracking branch 'upstream/main' into add-duplicate-hiera…
jafingerhut Oct 21, 2024
6bcb614
Add new test program to exercise latest code changes
jafingerhut Oct 21, 2024
55da2c3
Change pass name to reflect only checking action control plane names
jafingerhut Oct 21, 2024
d67df83
Fix lint warning, and add one more successful test program
jafingerhut Oct 22, 2024
42c1dee
Merge remote-tracking branch 'origin/fix-localizeallactions-bug' into…
jafingerhut Oct 22, 2024
95dc4e2
A few more fixes and test cases for duplicate action name checks
jafingerhut Oct 22, 2024
efd0d52
Fix code formatting to pass lint check
jafingerhut Oct 22, 2024
4540f2a
Fix lint warning for Python code
jafingerhut Oct 22, 2024
39a9251
Update old file name in CMakeLists.txt
jafingerhut Oct 23, 2024
62fe180
Merge remote-tracking branch 'upstream/main' into combo-of-duplicatio…
jafingerhut Oct 23, 2024
b614ff9
Add another case of conflicting action names between control and sub-…
jafingerhut Oct 25, 2024
968af1f
Merge remote-tracking branch 'upstream/main' into combo-of-duplicatio…
jafingerhut Oct 25, 2024
a2e913b
Add another correct case of not-conflicting @name annotations involvi…
jafingerhut Oct 25, 2024
72827dd
Fix comment, and remove unnecessary comment in P4 test program
jafingerhut Oct 30, 2024
b9a0ee9
Merge remote-tracking branch 'upstream/main' into combo-of-duplicatio…
jafingerhut Oct 30, 2024
019f3c6
Merge remote-tracking branch 'upstream/main' into combo-of-duplicatio…
jafingerhut Oct 31, 2024
bb50cc6
Put method controlPlaneAPIGenEnabled in class CompilerOptions
jafingerhut Nov 5, 2024
a840163
Fixes for incorrect previous commit.
jafingerhut Nov 5, 2024
19956fc
Fix cpplint warning from clang-format
jafingerhut Nov 5, 2024
e563b90
Merge remote-tracking branch 'upstream/main' into combo-of-duplicatio…
jafingerhut Nov 9, 2024
62ee2c1
Updates due to recent changes in annotation APIs
jafingerhut Nov 9, 2024
c3feb19
More modifications required to make the code work with ...
jafingerhut Nov 9, 2024
31f0b10
Simplify string manipulation code using latest abseil changes
jafingerhut Nov 10, 2024
bd31e6b
Fix clang-format warning
jafingerhut Nov 10, 2024
a853313
Merge branch 'master' into combo-of-duplication-action-name-check-and…
jafingerhut Dec 2, 2024
c111ab1
Updates required to work with recent changes to annotation implementa…
jafingerhut Dec 2, 2024
b542540
Address review comments
jafingerhut Dec 2, 2024
beb87b0
Removed an old line of commented-out code.
jafingerhut Dec 2, 2024
87feba6
Reword some comments for clarity.
jafingerhut Dec 2, 2024
e586849
Fix lint warning
jafingerhut Dec 2, 2024
381ec48
Address review comment
jafingerhut Dec 3, 2024
2f861cf
Merge remote-tracking branch 'upstream/main' into combo-of-duplicatio…
jafingerhut Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion control-plane/p4RuntimeSerializer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1584,7 +1584,7 @@ void P4RuntimeSerializer::serializeP4RuntimeIfRequired(const IR::P4Program *prog
std::vector<cstring> files;
std::vector<P4::P4RuntimeFormat> formats;

// only generate P4Info is required by use-provided options
// only generate P4Info if required by user-provided options
if (options.p4RuntimeFile.isNullOrEmpty() && options.p4RuntimeFiles.isNullOrEmpty() &&
options.p4RuntimeEntriesFile.isNullOrEmpty() &&
options.p4RuntimeEntriesFiles.isNullOrEmpty()) {
Expand Down
2 changes: 2 additions & 0 deletions frontends/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ set (P4_FRONTEND_SRCS
p4/frontend.cpp
p4/functionsInlining.cpp
p4/hierarchicalNames.cpp
p4/duplicateActionControlPlaneNameCheck.cpp
p4/inlining.cpp
p4/localizeActions.cpp
p4/methodInstance.cpp
Expand Down Expand Up @@ -110,6 +111,7 @@ set (P4_FRONTEND_HDRS
p4/frontend.h
p4/functionsInlining.h
p4/hierarchicalNames.h
p4/duplicateActionControlPlaneNameCheck.h
p4/inlining.h
p4/localizeActions.h
p4/methodInstance.h
Expand Down
11 changes: 11 additions & 0 deletions frontends/common/options.h
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,17 @@ class CompilerOptions : public ParserOptions {
bool optimizeSize = false; // optimize favoring size

virtual bool enable_intrinsic_metadata_fix();

/// Indicates whether control plane API generation is enabled.
/// @returns default to false unless a command line option was
/// given explicitly enabling control plane API generation.
virtual bool controlPlaneAPIGenEnabled() const {
if (p4RuntimeFile.isNullOrEmpty() && p4RuntimeFiles.isNullOrEmpty() &&
p4RuntimeEntriesFile.isNullOrEmpty() && p4RuntimeEntriesFiles.isNullOrEmpty()) {
return false;
}
return true;
}
Comment on lines +91 to +100
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asl @oleg-ran-amd I think that DuplicateActionControlPlaneNameCheck should run for our backend even when P4Info files are not generated. So we should probably override this function to unconditionally return true.

};

} // namespace P4
Expand Down
99 changes: 99 additions & 0 deletions frontends/p4/duplicateActionControlPlaneNameCheck.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
/*
Copyright 2024 Cisco Systems, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

#include "duplicateActionControlPlaneNameCheck.h"

#include "lib/log.h"

namespace P4 {

cstring DuplicateActionControlPlaneNameCheck::getName(const IR::IDeclaration *decl) {
return decl->getName();
}

void DuplicateActionControlPlaneNameCheck::checkForDuplicateName(cstring name,
const IR::Node *node) {
bool foundDuplicate = false;
auto *otherNode = node;
auto [it, inserted] = actions.insert(std::pair(name, node));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
auto [it, inserted] = actions.insert(std::pair(name, node));
auto [it, inserted] = actions.emplace(name, node);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in commit 57

if (!inserted) {
foundDuplicate = true;
otherNode = (*it).second;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
otherNode = (*it).second;
otherNode = it->second;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in commit 57

}
if (foundDuplicate) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could probably fold into !inserted condition above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in commit 57

error(ErrorType::ERR_DUPLICATE, "%1%: conflicting control plane name: %2%", node,
otherNode);
}
}

const IR::Node *DuplicateActionControlPlaneNameCheck::postorder(IR::P4Action *action) {
// Actions declared at the top level that the developer did not
// write a @name annotation for it, should be included in the
// duplicate name check. They do not yet have a @name annotation
// by the time this pass executes, so this method will handle
// such actions.
if (findContext<IR::P4Control>() == nullptr) {
auto annos = action->annotations;
bool hasNameAnnotation = false;
if (annos != nullptr) {
auto nameAnno = annos->getSingle(IR::Annotation::nameAnnotation);
if (nameAnno) {
hasNameAnnotation = true;
}
}
if (!hasNameAnnotation) {
// name is what this top-level action's @name annotation
// will be, after LocalizeAllActions pass adds one.
cstring name = "." + action->name;
checkForDuplicateName(name, action);
}
}
return action;
}

const IR::Node *DuplicateActionControlPlaneNameCheck::postorder(IR::Annotation *annotation) {
if (annotation->name != IR::Annotation::nameAnnotation) return annotation;
// The node the annotation belongs to
CHECK_NULL(getContext()->parent);
auto *annotatedNode = getContext()->parent->node;
CHECK_NULL(annotatedNode);
// We are only interested in name annotations on actions here, as
// the P4Runtime API control plane generation code already checks
// for duplicate control plane names for other kinds of objects.
if (!annotatedNode->is<IR::P4Action>()) {
return annotation;
}
cstring name = annotation->getName();
if (!name.startsWith(".")) {
// Create a fully hierarchical name beginning with ".", so we
// can compare it against any other @name annotation value
// that begins with "." and is equal.
if (stack.empty()) {
name = absl::StrCat(".", name.string_view());
} else {
name = absl::StrCat(".",
absl::StrJoin(stack, ".",
[](std::string *out, cstring s) {
absl::StrAppend(out, s.string_view());
}),
".", name.string_view());
}
}
checkForDuplicateName(name, annotatedNode);
return annotation;
}

} // namespace P4
91 changes: 91 additions & 0 deletions frontends/p4/duplicateActionControlPlaneNameCheck.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
/*
Copyright 2024 Cisco Systems, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

#ifndef FRONTENDS_P4_DUPLICATEACTIONCONTROLPLANENAMECHECK_H_
#define FRONTENDS_P4_DUPLICATEACTIONCONTROLPLANENAMECHECK_H_

#include "ir/ir.h"
#include "ir/visitor.h"
Comment on lines +22 to +23

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#include <vector>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in commit 57


namespace P4 {

/**
* This pass does not change anything in the IR. It only gives an
* error if it finds two objects with the same hierarchical name. I
* am not certain, but it might be that at this point in the front
* end, the only way such a duplicate can happen is due to @name
* annotations. @name annotations are definitely taken into account
* in this duplicate check.
*
* See also the pass HierarchicalNames, on which the implementation of
* this pass was based.
*
* This pass should be run before pass LocalizeAllActions, because
* LocalizeAllActions can create actions with duplicate names, by
* design, that were not created by the P4 developer. We should not
* issue an error if that pass creates duplicate hierarchical names.
*/
class DuplicateActionControlPlaneNameCheck : public Transform {
std::vector<cstring> stack;
/// Used for detection of conflicting control plane names among actions.
string_map<const IR::Node *> actions;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

string_map preserves the insertion order.
This is not needed for this PR as it never iterates through the elements.
Keeping the order also consumes extra resources (std::list + its memory allocations).
What if we replace this with absl::flat_hash_map?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes in commit 57


public:
cstring getName(const IR::IDeclaration *decl);

DuplicateActionControlPlaneNameCheck() {
setName("DuplicateActionControlPlaneNameCheck");
visitDagOnce = false;
}
const IR::Node *preorder(IR::P4Parser *parser) override {
stack.push_back(getName(parser));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we call prune() to prevent the visitor from going deeper into the parser which cannot have actions?

I think, pushing the parser onto the stack can be removed as we never use parser names to build hierarchical action names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed there should be no need to walk through sub-nodes within a parser.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in commit 57

return parser;
}
const IR::Node *postorder(IR::P4Parser *parser) override {
stack.pop_back();
return parser;
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we apply the change proposed in the previous comment, this overload can be removed, I believe.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in commit 57


const IR::Node *preorder(IR::P4Control *control) override {
stack.push_back(getName(control));
return control;
}
const IR::Node *postorder(IR::P4Control *control) override {
stack.pop_back();
return control;
}
kfcripps marked this conversation as resolved.
Show resolved Hide resolved

const IR::Node *preorder(IR::P4Table *table) override {
visit(table->annotations);
prune();
return table;
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please elaborate why we need to visit the table's annotations here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It most likely is not needed, and I will try removing it. This looks like something left over from when I started by copying-and-pasting the HierarchicalNames pass, and never noticed it could/should be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left the method, but removed the visit call, but left the call to prune, since there should be no need to traverse the sub-nodes of a P4 table.


void checkForDuplicateName(cstring name, const IR::Node *node);

const IR::Node *postorder(IR::P4Action *action) override;

const IR::Node *postorder(IR::Annotation *annotation) override;
// Do not change name annotations on parameters
const IR::Node *preorder(IR::Parameter *parameter) override {
prune();
return parameter;
}
};

} // namespace P4

#endif /* FRONTENDS_P4_DUPLICATEACTIONCONTROLPLANENAMECHECK_H_ */
10 changes: 10 additions & 0 deletions frontends/p4/frontend.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ limitations under the License.
#include "deprecated.h"
#include "directCalls.h"
#include "dontcareArgs.h"
#include "duplicateActionControlPlaneNameCheck.h"
#include "entryPriorities.h"
#include "evaluator/evaluator.h"
#include "frontends/common/constantFolding.h"
Expand Down Expand Up @@ -234,6 +235,15 @@ const IR::P4Program *FrontEnd::run(const CompilerOptions &options, const IR::P4P
if (policy->optimize(options)) {
passes.addPasses({
new Inline(&typeMap, *policy, options.optimizeParserInlining),
});
}
if (options.controlPlaneAPIGenEnabled()) {
passes.addPasses({
new DuplicateActionControlPlaneNameCheck(),
});
}
if (policy->optimize(options)) {
passes.addPasses({
new InlineActions(&typeMap, *policy),
new LocalizeAllActions(*policy),
new UniqueNames(),
Expand Down
21 changes: 18 additions & 3 deletions frontends/p4/localizeActions.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,24 @@ const IR::Node *TagGlobalActions::preorder(IR::P4Action *action) {
if (findContext<IR::P4Control>() == nullptr) {
auto annos = action->annotations;
if (annos == nullptr) annos = IR::Annotations::empty;
cstring name = "."_cs + action->name;
annos = annos->addAnnotationIfNew(IR::Annotation::nameAnnotation,
new IR::StringLiteral(name), false);
auto nameAnno = annos->getSingle(IR::Annotation::nameAnnotation);
if (nameAnno) {
// If the value of the existing name annotation does not
// begin with ".", prepend "." so that the name remains
// global if control plane APIs are generated later.
const auto *e0 = nameAnno->expr.at(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const auto *e0 = nameAnno->expr.at(0);
const auto *e0 = nameAnno->getExpr(0);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in commit 55, IIRC

auto nameString = e0->to<IR::StringLiteral>()->value;
if (!nameString.startsWith(".")) {
nameString = "."_cs + nameString;
auto newLit = new IR::StringLiteral(e0->srcInfo, nameString);
annos = annos->addOrReplace(IR::Annotation::nameAnnotation, newLit);
}
} else {
// Add new name annotation beginning with "."
cstring name = "."_cs + action->name;
annos = annos->addAnnotationIfNew(IR::Annotation::nameAnnotation,
new IR::StringLiteral(name), false);
}
action->annotations = annos;
}
prune();
Expand Down
Loading
Loading