Regression: ${PROJECT_ROOT} for relative paths to local packages not working in pdm > 2.2.1 #1658
Description
- I have searched the issue tracker and believe that this is not a duplicate.
Make sure you run commands with -v
flag before pasting the output.
Steps to reproduce
-
Start with minimal pyproject.toml:
[tool.pdm] [project] requires-python = ">3.7"
-
Add local package dependency (
mypkg/
contains a minimal setup.py-based package).pdm add ./mypkg
Using pdm 2.2.1, the resulting pyproject.toml is:
[tool.pdm] [project] requires-python = ">=3.7" dependencies = [ "my-test-package @ file:///${PROJECT_ROOT}/mypkg", ] dynamic = ["dependencies"]
-
Upgrade pdm to anything > 2.2.1 and attempt to use that
pyproject.toml
.
Actual behavior
Attempts to use that pyproject.toml
with any later version of pdm (I've tried 2.4.2 and 2.3.0) fail (with FileNotFoundError: [Errno 2] No such file or directory: '/${PROJECT_ROOT}/mypkg
.
E.g.
$ pdm --version
PDM, version 2.4.2
$ pdm lock -v
STATUS: Resolving dependencies
pdm.termui: ======== Start resolving requirements ========
pdm.termui: my-test-package @ file:///${PROJECT_ROOT}/mypkg
pdm.termui: python>=3.7
pdm.termui: Adding requirement my-test-package @ file:///${PROJECT_ROOT}/mypkg
pdm.termui: Adding requirement python>=3.7
pdm.termui: ======== Starting round 0 ========
STATUS: Resolving: new pin python>=3.7
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
Traceback (most recent call last):
File ".../.local/bin/pdm", line 8, in <module>
sys.exit(main())
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 259, in main
return Core().main(args)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 191, in main
raise cast(Exception, err).with_traceback(traceback)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 186, in main
f(project, options)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/cli/commands/lock.py", line 53, in handle
actions.do_lock(
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/cli/actions.py", line 95, in do_lock
mapping, dependencies = resolve(
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/resolver/core.py", line 35, in resolve
result = resolver.resolve(requirements, max_rounds)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 521, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 402, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
for requirement in self._p.get_dependencies(candidate=candidate):
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/resolver/providers.py", line 189, in get_dependencies
deps, requires_python, _ = self.repository.get_dependencies(candidate)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/repositories.py", line 78, in get_dependencies
requirements, requires_python, summary = getter(candidate)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/repositories.py", line 35, in wrapper
result = func(self, candidate)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/repositories.py", line 228, in _get_dependencies_from_metadata
deps = prepared.get_dependencies_from_metadata()
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/candidates.py", line 539, in get_dependencies_from_metadata
self.req.project_name, self.metadata.requires or [], extras # type: ignore
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/candidates.py", line 523, in metadata
result = self.prepare_metadata()
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/candidates.py", line 442, in prepare_metadata
self.obtain(allow_all=True)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/candidates.py", line 432, in obtain
result = finder.download_and_unpack(
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/unearth/finder.py", line 358, in download_and_unpack
file = unpack_link(
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/unearth/preparer.py", line 295, in unpack_link
validator.validate_path(artifact)
File ".../.local/share/pdm/venv/lib/python3.8/site-packages/unearth/preparer.py", line 116, in validate_path
with path.open("rb") as f:
File "/usr/lib/python3.8/pathlib.py", line 1222, in open
return io.open(self, mode, buffering, encoding, errors, newline,
File "/usr/lib/python3.8/pathlib.py", line 1078, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/${PROJECT_ROOT}/mypkg'
Expected behavior
Continuing with pdm 2.2.1, things appear to work correctly, e.g.
$ pdm --version
PDM, version 2.2.1
$ pdm lock -v
pdm.termui: ======== Start resolving requirements ========
pdm.termui: my-test-package @ file:///${PROJECT_ROOT}/mypkg
pdm.termui: python>=3.7
pdm.termui: Adding requirement my-test-package @ file:///${PROJECT_ROOT}/mypkg
pdm.termui: Adding requirement python>=3.7
pdm.termui: ======== Starting round 0 ========
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui: Running PEP 517 backend to get metadata for <Link file:///[...]/mypkg (from None)>
pdm.termui: Preparing isolated env for PEP 517 build...
pdm.termui: ======== Start resolving requirements ========
pdm.termui: wheel
pdm.termui: setuptools>=40.8.0
pdm.termui: python>=3.10.9,<3.10.10
pdm.termui: Adding requirement wheel
pdm.termui: Adding requirement setuptools>=40.8.0
pdm.termui: Adding requirement python>=3.10.9,<3.10.10
pdm.termui: ======== Starting round 0 ========
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui: Pinning: setuptools 67.1.0
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
pdm.termui: Pinning: wheel 0.38.4
pdm.termui: ======== Ending round 2 ========
pdm.termui: ======== Starting round 3 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui: python None
pdm.termui: setuptools 67.1.0
pdm.termui: wheel 0.38.4
pdm.termui: Installing setuptools 67.1.0
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/c8/24/157fed9a57108b3e1b6ccd86e95823327f231ef12464b0eca13650eb33ac/setuptools-67.1.0-py3-none-any.whl (from None)>
pdm.termui: Installing wheel 0.38.4
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/bd/7c/d38a0b30ce22fc26ed7dbc087c6d00851fb3395e9d0dac40bec1f905030c/wheel-0.38.4-py3-none-any.whl (from None)>
pdm.termui: running egg_info
pdm.termui: writing my_test_package.egg-info/PKG-INFO
pdm.termui: writing dependency_links to my_test_package.egg-info/dependency_links.txt
pdm.termui: writing top-level names to my_test_package.egg-info/top_level.txt
pdm.termui: reading manifest file 'my_test_package.egg-info/SOURCES.txt'
pdm.termui: writing manifest file 'my_test_package.egg-info/SOURCES.txt'
pdm.termui: running dist_info
pdm.termui: creating /tmp/pdm-meta-uu6utr5w/my_test_package.egg-info
pdm.termui: writing /tmp/pdm-meta-uu6utr5w/my_test_package.egg-info/PKG-INFO
pdm.termui: writing dependency_links to /tmp/pdm-meta-uu6utr5w/my_test_package.egg-info/dependency_links.txt
pdm.termui: writing top-level names to /tmp/pdm-meta-uu6utr5w/my_test_package.egg-info/top_level.txt
pdm.termui: writing manifest file '/tmp/pdm-meta-uu6utr5w/my_test_package.egg-info/SOURCES.txt'
pdm.termui: reading manifest file '/tmp/pdm-meta-uu6utr5w/my_test_package.egg-info/SOURCES.txt'
pdm.termui: writing manifest file '/tmp/pdm-meta-uu6utr5w/my_test_package.egg-info/SOURCES.txt'
pdm.termui: creating '/tmp/pdm-meta-uu6utr5w/my_test_package-0.1.dist-info'
pdm.termui: Pinning: my-test-package 0.1
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui: python None
pdm.termui: my-test-package file:///${PROJECT_ROOT}/mypkg
🔒 Lock successful
Changes are written to pdm.lock.
PDM 2.2.1 is installed, while 2.4.2 is available.
Please run `pdm self update` to upgrade.
Run `pdm config check_update false` to disable the check.
Notes
If, in step 2 of "Step to Reproduce", one uses pdm 2.4.2 to add the local package (pdm add ./mypkg
), the resulting pyproject.toml
includes the absolute path to the local package, rather than using ${PROJECT_ROOT}
:
[tool.pdm]
[project]
requires-python = ">=3.7"
dependencies = [
"my-test-package @ file:///abs/path/to/project/mypkg",
]
This is less than ideal, since it eliminates the ability to relocate the project (e.g. by cloning the containing git repo to another directory.)
Environment Information
# Paste the output of `pdm info && pdm info --env` below:
$ pdm info
PDM version:
2.4.2
Python Interpreter:
/path/to/project/.venv/bin/python (3.10)
Project Root:
/path/to/project
Project Packages:
None
$ pdm info --env
{
"implementation_name": "cpython",
"implementation_version": "3.10.9",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_release": "5.14.0-1056-oem",
"platform_system": "Linux",
"platform_version": "#63-Ubuntu SMP Fri Dec 16 14:32:59 UTC 2022",
"python_full_version": "3.10.9",
"platform_python_implementation": "CPython",
"python_version": "3.10",
"sys_platform": "linux"
}
Activity
frostming commentedon Feb 2, 2023
Because the new version doesn't include a default
build-system
table anymore, and it falls back tosetuptools
. You can add it to thepyproject.toml
:dairiki commentedon Feb 2, 2023
@frostming Thank you for the quick response. That works. I figured I was missing something.
Since I am just working on a "project" rather than a "library", and never intend to build a distribution, it never occurred to me that the
build-system
setting was relevant.fix: change the fallback build backend to `pdm-pep517`