setup.py generation for local dependencies appears to be broken. #7131
Closed as not planned
Description
- Poetry version: 1.22
- Python version: 3.10.6
- OS version and name: Ubuntu 22.04
- pyproject.toml: https://gist.github.com/jembishop/12a7fdb9b8519cd800fe647f12d408fa
- I am on the latest stable Poetry version, installed using a recommended method.
- I have searched the issues of this repo and believe that this is not a duplicate.
- I have consulted the FAQ and blog for any relevant entries or release notes.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvv
option) and have included the output below.
Issue
I am using the build.py
feature, necessary as the project is a cython project, and I also want to specify a local dependency.
(The local dependency also happens to be a cython project, but I don't think that is important to the bug)
The stack trace (edited to remove some identifying info)
➜ my-project git:(jb/fix-feed-path) ✗ poetry install -vvv
Using virtualenv: /home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10
Project environment contains an empty path in sys_path, ignoring.
Installing dependencies from lock file
Finding the necessary packages for the current system
Package operations: 0 installs, 0 updates, 0 removals, 13 skipped
• Installing my-other-project (0.1.0 /home/jem/longshot/strategy/my-other-project/python): Pending...
• Installing my-other-project (0.1.0 /home/jem/longshot/strategy/my-other-project/python): Skipped for the following reason: Already installed
• Installing urllib3 (1.26.13): Pending...
• Installing urllib3 (1.26.13): Skipped for the following reason: Already installed
Installing the current project: my-project (0.1.0)
- Building package my-project in editable mode
- Falling back on using a setup.py
Stack trace:
2 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1472 in _run
1470│ )
1471│ else:
→ 1472│ output = subprocess.check_output(
1473│ command, stderr=subprocess.STDOUT, env=env, **kwargs
1474│ )
1 /usr/lib/python3.10/subprocess.py:420 in check_output
418│ kwargs['input'] = empty
419│
→ 420│ return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
421│ **kwargs).stdout
422│
CalledProcessError
Command '['/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10', '--upgrade', '--no-deps', '-e', '/home/jem/longshot/strategy/my-project']' returned non-zero exit status 1.
at /usr/lib/python3.10/subprocess.py:524 in run
520│ # We don't call process.wait() as .__exit__ does that for us.
521│ raise
522│ retcode = process.poll()
523│ if check and retcode:
→ 524│ raise CalledProcessError(retcode, process.args,
525│ output=stdout, stderr=stderr)
526│ return CompletedProcess(process.args, retcode, stdout, stderr)
527│
528│
The following error occurred when trying to handle this error:
Stack trace:
3 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/pip.py:49 in pip_install
47│
48│ try:
→ 49│ return environment.run_pip(*args)
50│ except EnvCommandError as e:
51│ raise PoetryException(f"Failed to install {path.as_posix()}") from e
2 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1435 in run_pip
1433│ pip = self.get_pip_command()
1434│ cmd = pip + list(args)
→ 1435│ return self._run(cmd, **kwargs)
1436│
1437│ def run_python_script(self, content: str, **kwargs: Any) -> int | str:
1 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1712 in _run
1710│ def _run(self, cmd: list[str], **kwargs: Any) -> int | str:
1711│ kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
nip 1712│ return super()._run(cmd, **kwargs)
1713│
1714│ def get_temp_environ(
EnvCommandError
Command ['/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10', '--upgrade', '--no-deps', '-e', '/home/jem/longshot/strategy/my-project'] errored with the following return code 1, and output:
Obtaining file:///home/jem/longshot/strategy/my-project
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'error'
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
error in my-project setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid URL: ../my-other-project/python
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1476 in _run
1472│ output = subprocess.check_output(
1473│ command, stderr=subprocess.STDOUT, env=env, **kwargs
1474│ )
1475│ except CalledProcessError as e:
→ 1476│ raise EnvCommandError(e, input=input_)
1477│
1478│ return decode(output)
1479│
1480│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
The following error occurred when trying to handle this error:
Stack trace:
10 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:329 in run
327│
328│ try:
→ 329│ exit_code = self._run(io)
330│ except Exception as e:
331│ if not self._catch_exceptions:
9 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:185 in _run
183│ self._load_plugins(io)
184│
→ 185│ exit_code: int = super()._run(io)
186│ return exit_code
187│
8 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:423 in _run
421│ io.input.set_stream(stream)
422│
→ 423│ exit_code = self._run_command(command, io)
424│ self._running_command = None
425│
7 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
463│
464│ if error is not None:
→ 465│ raise error
466│
467│ return event.exit_code
6 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
447│
448│ if event.command_should_run():
→ 449│ exit_code = command.run(io)
450│ else:
451│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
5 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:
4 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
81│
82│ try:
→ 83│ return self.handle()
84│ except KeyboardInterrupt:
85│ return 1
3 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/install.py:177 in handle
175│ return 0
176│
→ 177│ builder.build()
178│
179│ if overwrite:
2 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/masonry/builders/editable.py:63 in build
61│ " - Falling back on using a setup.py"
62│ )
→ 63│ self._setup_build()
64│ path: Path = self._path
65│ return path
1 ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/masonry/builders/editable.py:120 in _setup_build
118│ )
119│ try:
→ 120│ pip_install(self._path, self._env, upgrade=True, editable=True)
121│ finally:
122│ shutil.move(
PoetryException
Failed to install /home/jem/longshot/strategy/my-project
at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
47│
48│ try:
49│ return environment.run_pip(*args)
50│ except EnvCommandError as e:
→ 51│ raise PoetryException(f"Failed to install {path.as_posix()}") from e
After some debugging it appears that the generated setup.py
file has an incorrect reference to the
local dependency. It generates:
install_requires = [
...
"my-other-package @ ../my-other-package",
...
]
When it should generate a file uri:
install_requires = [
...
"my-other-package @ file:///absolute/path/to/my-other-package",
...
]
to be a valid dependency.
I hacked around this by modifying sdist.py
in the convert_dependencies
function.
I simply matched against the regex (.*) @ ((?:\.|\/).*)
to pick up a local path and replace it with the file uri.
Would you be interested in me raising a pull request for this?