Skip to content

setup.py generation for local dependencies appears to be broken.  #7131

Closed as not planned
@jembishop

Description

  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

I am using the build.py feature, necessary as the project is a cython project, and I also want to specify a local dependency.
(The local dependency also happens to be a cython project, but I don't think that is important to the bug)

The stack trace (edited to remove some identifying info)

➜  my-project git:(jb/fix-feed-path) ✗ poetry install -vvv
Using virtualenv: /home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10
Project environment contains an empty path in sys_path, ignoring.
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 0 updates, 0 removals, 13 skipped

  • Installing my-other-project (0.1.0 /home/jem/longshot/strategy/my-other-project/python): Pending...
  • Installing my-other-project (0.1.0 /home/jem/longshot/strategy/my-other-project/python): Skipped for the following reason: Already installed
  • Installing urllib3 (1.26.13): Pending...
  • Installing urllib3 (1.26.13): Skipped for the following reason: Already installed

Installing the current project: my-project (0.1.0)
  - Building package my-project in editable mode
  - Falling back on using a setup.py

  Stack trace:

  2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1472 in _run
      1470│                 )
      1471│             else:
    → 1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )

  1  /usr/lib/python3.10/subprocess.py:420 in check_output
       418│         kwargs['input'] = empty
       419│ 
    →  420│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
       421│                **kwargs).stdout
       422│ 

  CalledProcessError

  Command '['/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10', '--upgrade', '--no-deps', '-e', '/home/jem/longshot/strategy/my-project']' returned non-zero exit status 1.

  at /usr/lib/python3.10/subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│ 
       528│ 

The following error occurred when trying to handle this error:


  Stack trace:

  3  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/pip.py:49 in pip_install
       47│ 
       48│     try:
    →  49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
       51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e

  2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1435 in run_pip
      1433│         pip = self.get_pip_command()
      1434│         cmd = pip + list(args)
    → 1435│         return self._run(cmd, **kwargs)
      1436│ 
      1437│     def run_python_script(self, content: str, **kwargs: Any) -> int | str:

  1  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1712 in _run
      1710│     def _run(self, cmd: list[str], **kwargs: Any) -> int | str:
      1711│         kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
    nip 1712│         return super()._run(cmd, **kwargs)
      1713│ 
      1714│     def get_temp_environ(

  EnvCommandError

  Command ['/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10', '--upgrade', '--no-deps', '-e', '/home/jem/longshot/strategy/my-project'] errored with the following return code 1, and output: 
  Obtaining file:///home/jem/longshot/strategy/my-project
    Preparing metadata (setup.py): started
    Preparing metadata (setup.py): finished with status 'error'
    error: subprocess-exited-with-error
    
    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [1 lines of output]
        error in my-project setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid URL: ../my-other-project/python
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: metadata-generation-failed
  
  × Encountered error while generating package metadata.
  ╰─> See above for output.
  
  note: This is an issue with the package mentioned above, not pip.
  hint: See above for details.
  

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1476 in _run
      1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )
      1475│         except CalledProcessError as e:
    → 1476│             raise EnvCommandError(e, input=input_)
      1477│ 
      1478│         return decode(output)
      1479│ 
      1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  Stack trace:

  10  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:329 in run
       327│ 
       328│             try:
     → 329│                 exit_code = self._run(io)
       330│             except Exception as e:
       331│                 if not self._catch_exceptions:

   9  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:185 in _run
       183│         self._load_plugins(io)
       184│ 
     → 185│         exit_code: int = super()._run(io)
       186│         return exit_code
       187│ 

   8  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:423 in _run
       421│             io.input.set_stream(stream)
       422│ 
     → 423│         exit_code = self._run_command(command, io)
       424│         self._running_command = None
       425│ 

   7  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
       463│ 
       464│         if error is not None:
     → 465│             raise error
       466│ 
       467│         return event.exit_code

   6  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
       447│ 
       448│             if event.command_should_run():
     → 449│                 exit_code = command.run(io)
       450│             else:
       451│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   5  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│ 
     → 119│         status_code = self.execute(io)
       120│ 
       121│         if status_code is None:

   4  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
        81│ 
        82│         try:
     →  83│             return self.handle()
        84│         except KeyboardInterrupt:
        85│             return 1

   3  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/install.py:177 in handle
       175│             return 0
       176│ 
     → 177│         builder.build()
       178│ 
       179│         if overwrite:

   2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/masonry/builders/editable.py:63 in build
        61│                     "  - Falling back on using a setup.py"
        62│                 )
     →  63│                 self._setup_build()
        64│                 path: Path = self._path
        65│                 return path

   1  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/masonry/builders/editable.py:120 in _setup_build
       118│                 )
       119│                 try:
     → 120│                     pip_install(self._path, self._env, upgrade=True, editable=True)
       121│                 finally:
       122│                     shutil.move(

  PoetryException

  Failed to install /home/jem/longshot/strategy/my-project

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
       47│ 
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e

After some debugging it appears that the generated setup.py file has an incorrect reference to the
local dependency. It generates:

install_requires = [
...
"my-other-package @ ../my-other-package",
...
]

When it should generate a file uri:

install_requires = [
...
"my-other-package @ file:///absolute/path/to/my-other-package",
...
]

to be a valid dependency.

I hacked around this by modifying sdist.py in the convert_dependencies function.
I simply matched against the regex (.*) @ ((?:\.|\/).*) to pick up a local path and replace it with the file uri.

Would you be interested in me raising a pull request for this?

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions