Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--require-hashes mode problem #1866

Open
canassa opened this issue May 10, 2023 · 6 comments
Open

--require-hashes mode problem #1866

canassa opened this issue May 10, 2023 · 6 comments

Comments

@canassa
Copy link

canassa commented May 10, 2023

Creating a lock file with pytest==7.3.1 generates an invalid lock file

Environment Versions

  1. OS Type: OSX
  2. Python version: 3.9.16
  3. pip version: 22.0.4
  4. pip-tools version: 6.13.0

Steps to replicate

Create a lockfile with pytest==7.3.1

pytest==7.3.1

Compile it with:

pip-compile --quiet --resolver=backtracking --upgrade --generate-hashes

Now install it:

pip install -r requirements/development.lock
Looking in indexes: https://hatch:****@gitlab.com/api/v4/projects/44758594/packages/pypi/simple
Collecting iniconfig==2.0.0
  Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting packaging==23.1
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting pluggy==1.0.0
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting pytest==7.3.1
  Using cached pytest-7.3.1-py3-none-any.whl (320 kB)
Collecting tomli>=1.0.0
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    tomli>=1.0.0 from https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl#sha256=939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc (from pytest==7.3.1->-r requirements/development.lock (line 21))
WARNING: You are using pip version 22.0.4; however, version 23.1.2 is available.
You should consider upgrading via the '/Users/cesarcanassa/code/src/gitlab.com/new10/services/payment-service/.venv/bin/python -m pip install --upgrade pip' command.

Expected result

It should install correctly

Actual result

It fails while installing it
@webknjaz
Copy link
Member

Looks like a pip issue, unless you have input and output files to show with a full reproducer...

@canassa
Copy link
Author

canassa commented May 10, 2023 •
edited
Loading

I tested with the latest pip:

pip --version
pip 23.1.2

Same error:

pip install -r requirements/development.lock
Looking in indexes: https://hatch:****@gitlab.com/api/v4/projects/44758594/packages/pypi/simple
Collecting iniconfig==2.0.0 (from -r requirements/development.lock (line 9))
  Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting packaging==23.1 (from -r requirements/development.lock (line 13))
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting pluggy==1.0.0 (from -r requirements/development.lock (line 17))
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting pytest==7.3.1 (from -r requirements/development.lock (line 21))
  Using cached pytest-7.3.1-py3-none-any.whl (320 kB)
Collecting exceptiongroup>=1.0.0rc8 (from pytest==7.3.1->-r requirements/development.lock (line 21))
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    exceptiongroup>=1.0.0rc8 from https://files.pythonhosted.org/packages/61/97/17ed81b7a8d24d8f69b62c0db37abbd8c0042d4b3fc429c73dab986e7483/exceptiongroup-1.1.1-py3-none-any.whl (from pytest==7.3.1->-r requirements/development.lock (line 21))

This is the input file

pytest==7.3.1

This is the lock file:

#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
#    pip-compile --generate-hashes --output-file=requirements/development.lock --resolver=backtracking requirements/development.txt
#

iniconfig==2.0.0 \
    --hash=sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3 \
    --hash=sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374
    # via pytest
packaging==23.1 \
    --hash=sha256:994793af429502c4ea2ebf6bf664629d07c1a9fe974af92966e4b8d2df7edc61 \
    --hash=sha256:a392980d2b6cffa644431898be54b0045151319d1e7ec34f0cfed48767dd334f
    # via pytest
pluggy==1.0.0 \
    --hash=sha256:4224373bacce55f955a878bf9cfa763c1e360858e330072059e10bad68531159 \
    --hash=sha256:74134bbf457f031a36d68416e1509f34bd5ccc019f0bcc952c7b909d06b37bd3
    # via pytest
pytest==7.3.1 \
    --hash=sha256:3799fa815351fea3a5e96ac7e503a96fa51cc9942c3753cda7651b93c1cfa362 \
    --hash=sha256:434afafd78b1d78ed0addf160ad2b77a30d35d4bdf8af234fe621919d9ed15e3
    # via -r requirements/development.txt

@AndydeCleyre
Copy link
Contributor

You say you're using Python 3.9.16, and you may be for installing, but as your lockfile indicates, you've compiled with Python 3.11.

Pytest requires tomli only when using Python <3.11, so tomli does not make it into the lockfile (and so no hash for it either).

If you use the same environment (or at least the same Python version) for compiling as installing, it should work.

@webknjaz
Copy link
Member

If you use the same environment (or at least the same Python version) for compiling as installing, it should work.

Only under pre-resolvelib pip or with -r and not -c — with the new dependency resolver, there's a bug with -c+hashes that hasn't yet been solved..

@jedie
Copy link

jedie commented Jun 11, 2023 •
edited
Loading

I have the same Problem:
I add 'tomli;python_version<"3.11"' to dependencies and pip-compile add this:

...
tomli==2.0.1 ; python_version < "3.11" \
    --hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc \
    --hash=sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f
...

This looks absolutely correct, isn't it?

pip failed installing with Python 3.11:

 ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    tomli>=1.1.0 from https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl (from flynt==0.77->-r /tmp/tmpicdwstbt (line 309))

With Python 3.10 and 3.9 installing works.
Full outputs are here: https://github.com/jedie/energymeter2mqtt/actions/runs/5235346165/jobs/9452127265?pr=5#step:4:233

So it's a pip bug pypa/pip#9644 , isn't it?

jedie added a commit to jedie/energymeter2mqtt that referenced this issue Jun 11, 2023
@jedie
Update requirements
7d5ef1b 
tomli only needed for Python <3.11, but see pip bug:
pypa/pip#9644 / jazzband/pip-tools#1866
jedie added a commit to jedie/cookiecutter_templates that referenced this issue Jun 11, 2023
@jedie
update requirements and fix "tomli"
d416770 
pypa/pip#9644 / jazzband/pip-tools#1866
@jedie jedie mentioned this issue Jun 11, 2023
Merged
jedie added a commit to jedie/cookiecutter_templates that referenced this issue Jun 11, 2023
@jedie
update requirements and fix "tomli"
e14701f 
pypa/pip#9644 / jazzband/pip-tools#1866
@NfNitLoop
Copy link

It seems like the bug here is that pip-compile should resolve all dependencies for all python versions, so that the requirements.txt that it generates with --generate-hashes will work on multiple systems (& python versions). Locking down your dependencies seems less useful if you can't then get back to that state on another machine.

We ran into a build failure today because ipython specifies a conditional dependency like @jedie mentions above. As a result, pip install -r requirements.txt works on one build environment, and not on another.

Without this we'll just have to stop using --generate-hashes.

jedie added a commit to YunoHost-Apps/django_yunohost_integration that referenced this issue Aug 4, 2024
@jedie
work-a-round for: jazzband/pip-tools#1866
efc2d0e 
Indirect requirement
jedie added a commit to YunoHost-Apps/django_yunohost_integration that referenced this issue Aug 4, 2024
@jedie
Merge pull request #35 from YunoHost-Apps/dev
597fabf 
work-a-round for: jazzband/pip-tools#1866
jedie pushed a commit to jedie/cookiecutter_templates that referenced this issue Aug 25, 2024
Work-a-round for: jazzband/pip-tools#1866
97ce98c
jedie pushed a commit to jedie/cookiecutter_templates that referenced this issue Aug 25, 2024
Work-a-round for: jazzband/pip-tools#1866
e0c1322
jedie pushed a commit to jedie/cookiecutter_templates that referenced this issue Aug 25, 2024
Work-a-round for: jazzband/pip-tools#1866
7eccd69
jedie pushed a commit to jedie/cookiecutter_templates that referenced this issue Aug 25, 2024
Work-a-round for: jazzband/pip-tools#1866
6180e76 
see also: jazzband/pip-tools#994 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jedie @canassa @NfNitLoop @webknjaz @AndydeCleyre