Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] TypeError in requisite calculation with order: last #67120

Open
2 of 9 tasks
frebib opened this issue Dec 23, 2024 · 3 comments · May be fixed by #67121
Open
2 of 9 tasks

[BUG] TypeError in requisite calculation with order: last #67120

frebib opened this issue Dec 23, 2024 · 3 comments · May be fixed by #67121
Labels
Bug broken, incorrect, or confusing behavior needs-triage

Comments

@frebib
Copy link
Contributor

frebib commented Dec 23, 2024 •
edited
Loading

Description

Passed invalid arguments: '<' not supported between instances of 'str' and 'int'.

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 216, in call
    ret["return"] = self.minion.executors[fname](
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 160, in __call__
    ret = self.loader.run(run_func, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1269, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1284, in _run_as
    return _func_or_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/executors/direct_call.py", line 10, in execute
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 160, in __call__
    ret = self.loader.run(run_func, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1269, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1284, in _run_as
    return _func_or_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/modules/state.py", line 1176, in highstate
    ret = st_.call_highstate(
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/state.py", line 4478, in call_highstate
    return self.state.call_high(high, orchestration_jid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/tracing/__init__.py", line 112, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/state.py", line 3139, in call_high
    chunks, errors = self.compile_high_data(high, orchestration_jid)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/tracing/__init__.py", line 112, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/state.py", line 1538, in compile_high_data
    chunks, errors = self.order_chunks(chunks)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/tracing/__init__.py", line 112, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/state.py", line 1433, in order_chunks
    sorted_chunks = self.dependency_dag.aggregate_and_order_chunks(cap)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/utils/requisite.py", line 365, in aggregate_and_order_chunks
    # Iterate over the nodes in topological order to get the correct
  File "/usr/lib/python3/dist-packages/networkx/algorithms/dag.py", line 426, in lexicographical_topological_sort
    zero_indegree = [create_tuple(v) for v, d in G.in_degree() if d == 0]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/networkx/algorithms/dag.py", line 426, in <listcomp>
    zero_indegree = [create_tuple(v) for v, d in G.in_degree() if d == 0]
                     ^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/networkx/algorithms/dag.py", line 422, in create_tuple
    return key(node), nodeid_map[node], node
           ^^^^^^^^^
File "/usr/lib/python3/dist-packages/salt/utils/requisite.py", line 359, in _get_order
    chunk_order = self._get_chunk_order(cap, node)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/utils/requisite.py", line 205, in _get_chunk_order
    child_min = min(child_min, child_order)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'str' and 'int'

Setup
This is ~3007.1 with #66135 applied

cc @bdrx312

  • on-prem machine
  • VM (Virtualbox, KVM, etc. please specify)
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD
  • classic packaging
  • onedir packaging
  • used bootstrap to install

Steps to Reproduce the behavior
Have a state with order: last.
Setting state_aggregate to either true/false doesn't seem to make any difference

Adding some logging yields a bit more of a clue

[TRACE   ] _get_chunk_order(): child_min=inf, child_order=10013
[TRACE   ] _get_chunk_order(): child_min=10013, child_order=10026
[TRACE   ] _get_chunk_order(): child_min=10013, child_order=10027
[TRACE   ] _get_chunk_order(): child_min=10013, child_order=10028
[TRACE   ] _get_chunk_order(): child_min=10013, child_order=10264
[TRACE   ] _get_chunk_order(): child_min=10013, child_order=10317
[TRACE   ] _get_chunk_order(): child_min=10013, child_order=10461
[TRACE   ] _get_chunk_order(): child_min=10013, child_order=10814
[TRACE   ] _get_chunk_order(): child_min=10013, child_order=10815
[TRACE   ] _get_chunk_order(): child_min=10013, child_order='last'
[WARNING ] _get_chunk_order(): child='cmd_|-$state-id_|-$the-command_|-run', stack=[('file_|-$dependency_|-$dependency_|-managed', True, inf, -inf)]

produced from a state that looks like

$state-id:
  file.managed:
    - name: $the-command
    - order: last
    - require:
      - sls: apt
  ...

where sls: apt contains the file.managed state for $dependency.

Expected behavior
TypeErrors should never happen at runtime. This is why static analysers and static typing exists

Versions Report

Python Version:
        Python: 3.11.2 (main, Sep 14 2024, 03:00:30) [GCC 12.2.0]

Dependency Versions:
          cffi: 1.15.1
      cherrypy: 18.8.0
      dateutil: 2.8.2
     docker-py: 5.0.3
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.3
      M2Crypto: 0.38.0
          Mako: Not Installed
       msgpack: 1.0.3
  msgpack-pure: Not Installed
  mysql-python: Not Installed
      networkx: 3.2.1
     packaging: 23.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.11.0
        pygit2: Not Installed
  python-gnupg: 0.4.9
        PyYAML: 6.0
         PyZMQ: 24.0.1
        relenv: Not Installed
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 6.2
           ZMQ: 4.3.4

Salt Package Information:
  Package Type: Not Installed

System Versions:
          dist: debian 2024.12.1 bookworm
        locale: utf-8
       machine: aarch64
       release: 6.6.66-cloudflare-2024.12.6
        system: Linux
       version: Debian GNU/Linux 2024.12.1 bookworm
@frebib frebib added Bug broken, incorrect, or confusing behavior needs-triage labels Dec 23, 2024
@frebib
Copy link
Contributor Author

frebib commented Dec 23, 2024

A workaround for this was to s/order: last/order: .inf/ allowed me to skip this problem and find the other show-breaking bugs in 3007 instead

@bdrx312 bdrx312 linked a pull request Dec 23, 2024 that will close this issue
Open
3 tasks
@bdrx312
Copy link
Contributor

bdrx312 commented Dec 23, 2024

A workaround for this was to s/order: last/order: .inf/ allowed me to skip this problem and find the other show-breaking bugs in 3007 instead.

The problem is that the code is not accounting for a child having a require and order: last. You can also work around it by just removing the require.

@frebib
Copy link
Contributor Author

frebib commented Dec 23, 2024 •
edited
Loading

Heh, with the size of our state DAG removing the requisite would cause chaos. I'll have a think about how this could be fixed in the new year and maybe fashion a patch to fix it

Edit: missed your patch! Thanks, will test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug broken, incorrect, or confusing behavior needs-triage
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix 67120 bdrx312/salt
2 participants
@frebib @bdrx312