Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assign priorities to configuration scopes #48420

Open
wants to merge 14 commits into
base: develop
Choose a base branch
from

Conversation

alalazo
Copy link
Member

@alalazo alalazo commented Jan 6, 2025

fixes #48414

Up to now configuration behaved like a stack, where scopes could be only be pushed at the top. This PR allows to assign priorities to scopes, and ensures that scopes of lower priorities are always "below" scopes of higher priorities.

When scopes have the same priority, what matters is the insertion order.

Modifications:

  • Add a mapping that iterates over keys according to priorities set when adding the key/value pair
  • Use that mapping to allow assigning priorities to configuration scopes
  • Assign different priorities for different kind of scopes, to fix a bug, and add a regression test
  • Simplify Configuration constructor f1d0371
  • Remove Configuration.pop_scope 61dd2f2

@alalazo alalazo added bugfix Something wasn't working, here's a fix v0.23.1 PRs to backport for v0.23.1 labels Jan 6, 2025
@spackbot-app spackbot-app bot added core PR affects Spack core functionality environments stand-alone-tests Stand-alone (or smoke) tests for installed packages tests General test capability(ies) utilities documentation Improvements or additions to documentation labels Jan 6, 2025
alalazo added 11 commits January 6, 2025 20:57
Up to now configuration behaved like a stack, where scopes could be
only be pushed at the top.

This commit allows to assign priorities to scopes, and ensures that
scopes of lower priorities are always "below" scopes of higher
priorities.

When scopes have the same priority, what matters is the insertion order.
The function is only used in a fixture, and it can be substituted
with already existing code.
@alalazo alalazo force-pushed the bugfix/config-priority-based-order branch from 2b17111 to 8c0cc36 Compare January 6, 2025 20:06
@alalazo
Copy link
Member Author

alalazo commented Jan 7, 2025

Failure in pipelines is unrelated, and due to system errors:

if key in self._data:
self.remove(key)

self._priorities.append((priority, self._new_id(), key))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sort/sorted is stable, so no need for _new_id().

Copy link
Member Author

@alalazo alalazo Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need that It's useful to be defensive when iterating (in case a new function scrambles the order of _priorities).

EDIT: I changed that in 87752f1 This is more similar to my first implementation, which I modified by adding _new_id() out of possibly an abundance of caution - motivated by the numerous bugs we had with config scopes order. If we have a second thought, and want to get back to having an explicit index, we can drop the commit.

psakievich
psakievich previously approved these changes Jan 10, 2025
Copy link
Contributor

@psakievich psakievich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've looked over the code and the concept seems sound to me. The idea of stack like behavior within a given priority is a more robust design. It also seems truer to our description of how configs will work with the users. I'm still mulling a bit about the defaults selected but not to the point of wanting to hold up a merge. Thanks @alalazo for taking the time to shore up this design.

lib/spack/llnl/util/lang.py Outdated Show resolved Hide resolved
lib/spack/llnl/util/lang.py Outdated Show resolved Hide resolved
lib/spack/llnl/util/lang.py Outdated Show resolved Hide resolved
lib/spack/llnl/util/lang.py Outdated Show resolved Hide resolved
@haampie
Copy link
Member

haampie commented Jan 10, 2025

I don't see any code change in bootstrap 🤔 what about

spack -c config:install_tree:root:/tmp spec zlib

bootstrapping to /tmp instead of ~/.spack/bootstrap?

@alalazo
Copy link
Member Author

alalazo commented Jan 10, 2025

@haampie There shouldn't be any change to bootstrap. The bootstrap store is specified in another place, so if you want to modify it from the command line you should:

$ spack -c bootstrap:root:/tmp spec zlib

even though the simpler way to do it is:

$ spack bootstrap root /tmp

Anyhow, I'm not aiming at changing any bootstrap semantic in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Something wasn't working, here's a fix core PR affects Spack core functionality documentation Improvements or additions to documentation environments stand-alone-tests Stand-alone (or smoke) tests for installed packages tests General test capability(ies) utilities v0.23.1 PRs to backport for v0.23.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Environment activation demotes the custom config scopes
3 participants