add support for __pypackages__

pypa · Feb 19, 2019 · 7c3b365 · 7c3b365
1 parent 3682721
commit 7c3b365
Show file tree

Hide file tree

Showing 6 changed files with 80 additions and 12 deletions.
diff --git a/.gitignore b/.gitignore
@@ -4,3 +4,4 @@
 build
 dist
 activate
+__pypackages__
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,8 @@
+0.12.2.0
+* Add support for PEP 582's `__pypackages__` (experimental). `pipx run BINARY` will first search in `__pypackages__` for binary, then fallback to installing from PyPI. `pipx run --pypackages BINARY` will raise an error if the binary is not found in `__pypackages__`.
+* Fix regression when installing with `--editable` flag (#93) 
+* [dev] improve unit tests
+
 0.12.1.0
 * Cache and reuse temporary Virtual Environments created with `pipx run` (#61)
 * Update binary discovery logic to find "scripts" like awscli (#91)

diff --git a/README.md b/README.md
@@ -14,7 +14,7 @@
 <a href="https://travis-ci.org/pipxproject/pipx"><img src="https://travis-ci.org/pipxproject/pipx.svg?branch=master" /></a>
 
 <a href="https://pypi.python.org/pypi/pipx/">
-<img src="https://img.shields.io/badge/pypi-0.12.1.0-blue.svg" /></a>
+<img src="https://img.shields.io/badge/pypi-0.12.2.0-blue.svg" /></a>
 <a href="https://github.com/ambv/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
 </p>
 
@@ -26,6 +26,7 @@
 * Safely install packages to isolated virtual environments, while globally exposing their CLI applications so you can run them from anywhere
 * Easily list, upgrade, and uninstall packages that were installed with pipx
 * Run the latest version of a CLI application from a package in a temporary virtual environment, leaving your system untouched after it finishes
+* Run binaries from the `__pypackages__` directory per PEP 582 as companion tool to [pythonloc](https://github.com/cs01/pythonloc)
 * Runs with regular user permissions, never calling `sudo pip install ...` (you aren't doing that, are you? 😄).
 
 pipx combines the features of JavaScript's [npx](https://medium.com/@maybekatz/introducing-npx-an-npm-package-runner-55f7d4bd282b) - which ships with npm - and Python's [pipsi](https://github.com/mitsuhiko/pipsi). pipx does not ship with pip but it is an important part of bootstrapping your system.
@@ -36,7 +37,7 @@ You can globally install a CLI application by running
 pipx install PACKAGE
 ```
 
-This automatically creates a virtual environment, installs the package, and symlinks the package's CLI binaries to a location on your `PATH`. For example, `pipx install cowsay` makes the `cowsay` command available globally, but sandboxes the cowsay package in its own virtual environment. **pipx never needs to run as sudo to do this.**
+This automatically creates a virtual environment, installs the package, and adds the package's CLI entry points to a location on your `PATH`. For example, `pipx install cowsay` makes the `cowsay` command available globally, but sandboxes the cowsay package in its own virtual environment. **pipx never needs to run as sudo to do this.**
 
 Example:
 ```
@@ -116,7 +117,7 @@ pipx {install,inject,upgrade,upgrade-all,uninstall,uninstall-all,reinstall-all,l
 You can run `pipx COMMAND --help` for details on each command.
 
 ### Install a Package
-The install command is the preferred way to globally install binaries from python packages on your system. It creates an isolated virtual environment for the package, then in a folder on your PATH creates symlinks to all the binaries provided by the installed package. It does not link to the package's dependencies.
+The install command is the preferred way to globally install binaries from python packages on your system. It creates an isolated virtual environment for the package, then ensures the package's binaries are accessible on your $PATH. (It does not link to the package's dependencies at this time, though that is under consideration.)
 
 The result: binaries you can run from anywhere, located in packages you can **cleanly** upgrade or uninstall. Guaranteed to not have dependency version conflicts or interfere with your OS's python packages. All **without** running `sudo`.
 ```
@@ -177,7 +178,7 @@ pipx inject ptpython requests pendulum
 After running the above commands, you will be able to import and use the `requests` and `pendulum` packages inside a `ptpython` repl.
 
 ### `uninstall`
-Uninstalls a package by deleting its virtual environment and any symlinks that point to its binaries.
+Uninstalls a package by deleting its virtual environment and any files that point to its binaries.
 ```
 pipx uninstall PACKAGE
 ```
@@ -204,7 +205,7 @@ pipx list
 results in something like
 ```
 venvs are in /Users/user/.local/pipx/venvs
-symlinks to binaries are in /Users/user/.local/bin
+binaries are exposed on your $PATH at /Users/user/.local/bin
    package black 18.9b0, Python 3.7.0
     - black
     - blackd
@@ -213,6 +214,9 @@ symlinks to binaries are in /Users/user/.local/bin
 ```
 ### `run`
 Run a binary from the latest version of its package in a temporary environment. The environment will be cached and re-used for up to two days. To ignore the cache, you can pass `--no-cache`.
+
+`run` will default to running a binary in the `__pypackages__` directory in support of [PEP 582](https://www.python.org/dev/peps/pep-0582/). Please note that this behavior is experimental, and is a acts as a companion tool to [pythonloc](https://github.com/cs01/pythonloc). It may be modified or removed in the future.
+
 ```
 pipx run BINARY
 pipx run [-h] [--no-cache] [--spec SPEC] [--verbose] [--python PYTHON]
@@ -311,7 +315,7 @@ When installing a package and its binaries (`pipx install package`) pipx will
 * create a virtualenv in ~/.local/pipx/venvs/PACKAGE
 * update pip to the latest version
 * install the desired package in the virtualenv
-* create symlinks in ~/.local/bin that point to new binaries in ~/.local/pipx/venvs/PACKAGE/bin (such as ~/.local/bin/black -> ~/.local/pipx/venvs/black/bin/black)
+* exposes binaries at `~/.local/bin` that point to new binaries in `~/.local/pipx/venvs/PACKAGE/bin` (such as `~/.local/bin/black` -> `~/.local/pipx/venvs/black/bin/black`)
 * As long as `~/.local/bin/` is on your PATH, you can now invoke the new binaries globally
 
 These are all things you can do yourself, but pipx automates them for you. If you are curious as to what pipx is doing behind the scenes, you can always use `pipx --verbose ...`.
@@ -382,7 +386,7 @@ First remove pipsi's directory (this is its default)
 rm -r ~/.local/pipsi
 ```
 
-There will still be symlinks in `~/.local/bin` that point to `~/.local/pipsi/venvs`. If you reinstall the same packages with `pipx`, the symlinks will be overwritten with valid symlinks that point to the new pipx directory in `~/.local/pipx/venvs`. You may also want to remove files in `~/.local/bin`, but be sure the files you delete there were created by pipsi.
+There will still be files in `~/.local/bin` that point to `~/.local/pipsi/venvs`. If you reinstall the same packages with `pipx`, the files will be overwritten with valid files that point to the new pipx directory in `~/.local/pipx/venvs`. You may also want to remove files in `~/.local/bin`, but be sure the files you delete there were created by pipsi.
 
 ## How does this compare with `pip-run`?
 [run with this](https://github.com/jaraco/pip-run) is focused on running **arbitrary Python code in ephemeral environments** while pipx is focused on running **Python binaries in ephemeral and non-ephemeral environments**.

diff --git a/pipx/commands.py b/pipx/commands.py
@@ -5,7 +5,14 @@
 from .colors import red, bold
 from .constants import LOCAL_BIN_DIR, PIPX_PACKAGE_NAME, PIPX_VENV_CACHEDIR
 from .emojies import stars, hazard, sleep
-from .util import rmdir, mkdir, PipxError, WINDOWS
+from .util import (
+    rmdir,
+    mkdir,
+    PipxError,
+    WINDOWS,
+    get_pypackage_bin_path,
+    run_pypackage_bin,
+)
 from .Venv import Venv
 from pathlib import Path
 from shutil import which
@@ -33,6 +40,7 @@ def run(
     python: str,
     pip_args: List[str],
     venv_args: List[str],
+    pypackages: bool,
     verbose: bool,
     use_cache: bool,
 ):
@@ -65,6 +73,19 @@ def run(
         binary = f"{binary}.exe"
         logging.warning(f"Assuming binary is {binary!r} (Windows only)")
 
+    pypackage_bin_path = get_pypackage_bin_path(binary)
+    if pypackage_bin_path.exists():
+        logging.info(
+            f"Using binary in local __pypackages__ directory at {str(pypackage_bin_path)}"
+        )
+        return run_pypackage_bin(pypackage_bin_path, binary_args)
+    if pypackages:
+        raise PipxError(
+            f"'--pypackages' flag was passed, but {str(pypackage_bin_path)!r} was "
+            "not found. See https://github.com/cs01/pythonloc to learn how to "
+            "install here, or omit the flag."
+        )
+
     venv_dir = _get_temporary_venv_path(package_or_url, python, pip_args, venv_args)
 
     venv = Venv(venv_dir)
@@ -459,7 +480,7 @@ def list_packages(pipx_local_venvs: Path):
         return
 
     print(f"venvs are in {bold(str(pipx_local_venvs))}")
-    print(f"symlinks to binaries are in {bold(str(LOCAL_BIN_DIR))}")
+    print(f"binaries are exposed on your $PATH at {bold(str(LOCAL_BIN_DIR))}")
     for d in dirs:
         _list_installed_package(d)
 

diff --git a/pipx/main.py b/pipx/main.py
@@ -18,7 +18,7 @@
 import textwrap
 import urllib.parse
 
-__version__ = "0.12.1.0"
+__version__ = "0.12.2.0"
 
 
 def print_version() -> None:
@@ -97,6 +97,7 @@ def run_pipx_command(args, binary_args: List[str]):
             args.python,
             pip_args,
             venv_args,
+            args.pypackages,
             verbose,
             use_cache,
         )
@@ -242,8 +243,9 @@ def get_command_parser():
 
     p = subparsers.add_parser(
         "run",
-        help="Download latest version of a package to temporary directory, "
-        "then run a binary from it. Temp dir is removed after execution is finished.",
+        help="Either download latest version of a package to temporary directory, "
+        "then run a binary from it, or invoke binary from local `__pypackages__` "
+        "directory (expiremental, see https://github.com/cs01/pythonloc)",
     )
     p.add_argument(
         "--no-cache",
@@ -257,6 +259,11 @@ def get_command_parser():
         help="arguments passed to the binary when it is invoked",
         default=[],
     )
+    p.add_argument(
+        "--pypackages",
+        action="store_true",
+        help="Require binary to be run from local __pypackages__ directory",
+    )
     p.add_argument("--spec", help=SPEC_HELP)
     p.add_argument("--verbose", action="store_true")
     p.add_argument(

diff --git a/pipx/util.py b/pipx/util.py
@@ -4,6 +4,9 @@
 from pathlib import Path
 import logging
 import shutil
+import subprocess
+import sys
+from typing import List
 
 
 class PipxError(Exception):
@@ -31,3 +34,30 @@ def mkdir(path: Path) -> None:
         return
     logging.info(f"creating directory {path}")
     path.mkdir(parents=True, exist_ok=True)
+
+
+def get_pypackage_bin_path(binary_name: str) -> Path:
+    return (
+        Path("__pypackages__")
+        / (str(sys.version_info.major) + "." + str(sys.version_info.minor))  # noqa E503
+        / "lib"  # noqa E503
+        / "bin"  # noqa E503
+        / binary_name  # noqa E503
+    )
+
+
+def run_pypackage_bin(bin_path: Path, args: List[str]) -> int:
+    def _get_env():
+        env = dict(os.environ)
+        env["PYTHONPATH"] = os.path.pathsep.join(
+            [".", str(bin_path.parent.parent)]
+            + os.getenv("PYTHONPATH", "").split(os.path.pathsep)  # noqa E503
+        )
+        return env
+
+    try:
+        return subprocess.run(
+            [str(bin_path.resolve())] + args, env=_get_env()
+        ).returncode
+    except KeyboardInterrupt:
+        return 1
-Original file line number
+Diff line change
@@ Expand Up / @@ -4,3 +4,4 @@ @@
     build
     dist
     activate
+    __pypackages__