Out of curiosity, when you say modernize, do you perhaps mean accepting the perhaps-unpleasant change in pip3 that prevents (or very strongly discourages) one from installing Python packages in system-wide directories? Perhaps considering using a Python virtual environment?

Out of curiosity, when you say modernize, do you perhaps mean accepting the perhaps-unpleasant change in pip3 that prevents (or very strongly discourages) one from installing Python packages in system-wide directories? Perhaps considering using a Python virtual environment?

Yes. We might have to go with the times here because the workarounds are quite aggressive/invasive.

I am exploring using poetry + virtualenv. If I work the kinks out you should be able to set up the entire Python environment in few commands:

sudo apt-get install -y python3-poetry python3-venv
cd p4c
poetry install
poetry shell

sudo applications can be run using sudo -E env PATH="$PATH" python3

FYI, my install-p4dev-v8.sh script here: https://github.com/jafingerhut/p4-guide/blob/master/bin/install-p4dev-v8.sh

can run as a normal user, with liberal sudo commands sprinkled throughout when needed, that start from unmodified Ubuntu 20.04, 22.04, or 24.04 system and install gRPC, Thrift, PI, BMv2, p4c, ptf, and mininet, all working together at least enough for the p4lang/tutorials repo exercises to work, and it uses a single Python venv for all Python packages installed.

I doubt that exactly what it does is what you need for this work, but just wanted to point it out as a working example of using a venv for all these tools together.

I believe I also needed LD_LIBRARY_PATH=$LD_LIBRARY_PATH in addition to PATH=$PATH from some sudo commands, perhaps ones like make check run in p4c/build.

Oh, and sometimes I found that instead of a command like sudo PATH=$PATH ... mn, where mn was installed in the Python venv, the best working alternative I found so far is sudo PATH=$PATH ... which mn``.

FYI, my install-p4dev-v8.sh script here: https://github.com/jafingerhut/p4-guide/blob/master/bin/install-p4dev-v8.sh

can run as a normal user, with liberal sudo commands sprinkled throughout when needed, that start from unmodified Ubuntu 20.04, 22.04, or 24.04 system and install gRPC, Thrift, PI, BMv2, p4c, ptf, and mininet, all working together at least enough for the p4lang/tutorials repo exercises to work, and it uses a single Python venv for all Python packages installed.

I doubt that exactly what it does is what you need for this work, but just wanted to point it out as a working example of using a venv for all these tools together.

I believe I also needed LD_LIBRARY_PATH=$LD_LIBRARY_PATH in addition to PATH=$PATH from some sudo commands, perhaps ones like make check run in p4c/build.

Yes, I am aware of these and have recommended it to students I work with! But for the sake of the compiler installation we can stay be much simpler. Nonetheless, the changes here should definitely integrate with your scripts. I'll explore further.

@jafingerhut Poetry works really well for managing Python dependencies. It was not difficult at all to set this up for all our distributions. All you need to do is call poetry shell in the p4c folder and you are working the correct Python environment.

Poetry also exposes a bunch of small issues we had hidden in the environment. For example, we are relying on the external p4runtime Python packages instead of the local p4runtime version from Github for testing.

One thing: It looks Poetry is not supported well Ubuntu18.04. This might be a good opportunity to finally deprecate this version.