Skip to content

Common build problems

Jump to bottom Edit New page
native-api edited this page Oct 9, 2024 · 161 revisions

First of all, make sure you have installed Python's dependencies and build tools as per https://github.com/pyenv/pyenv/wiki#suggested-build-environment , before any further troubleshooting.

Open the build log (the path to it is printed after the "BUILD FAILED" message) and look for any error messages in it (they are usually marked with the word "error"). If there are many error messages, the earliest one typically points to the root cause.

If your error message is not listed in the below table of contents (TOC), try searching this page for the message's parts (best for parts that would likely remain the same between distributions and library versions). If multiple different error messages have the same cause, we list them in the corresponding section's text rather than in the TOC.

Prerequisites

Below are some alternative packages that are not in the recommended set and should generally only be considered when there are special needs and/or problems with those in it. <!- Updates to the recommended set should go on the above link rather than here! ->

  • Ubuntu/Debian:

    • Alternative to libreadline-dev:

      sudo apt install libedit-dev

    • An inferior alternative to libncursesw5:

      sudo apt install libncurses5-dev

  • Fedora/CentOS/RHEL(aws ec2):

    • Alternative to openssl-devel:

      • If you need OpenSSL 1.0:

        sudo yum install compat-openssl10-devel --allowerasing

      • If you need OpenSSL 1.1:

        sudo yum install openssl11-devel --allowerasing

    If your yum doesn't support the --allowerasing flag, use yum swap instead, e.g.:

    sudo yum swap openssl-devel openssl11-devel

  • openSUSE:

    • Alternative to readline-devel:

      sudo zypper install libedit-devel

  • Arch and derivatives

    Installing an older version of ncurses, ncurses5, requires an AUR Helper to install. If using YAY:

    yay -S ncurses5-compat-libs

  • macOS 14+ - Apple Silicon

    Installing versions of Python >=3.12.1 on some machines may require ncurses, which seem to be missing. Install it with:

    brew install ncurses

Removing a python version manually

rm -rf ~/.pyenv/versions/"X.Y.Z"

Where "X.Y.Z" is the version that you want to remove. To list installed versions:

pyenv versions

Installing a 32 bit python on 64 bit Mac OS X (this will not work on Linux)

CONFIGURE_OPTS="--with-arch=i386" CFLAGS="-arch i386" LDFLAGS="-arch i386" python-build options

Installing a system-wide Python

If you want to install a Python interpreter that's available to all users and system scripts (no pyenv), use /usr/local/ as the install path. For example:

sudo python-build 3.3.2 /usr/local/

Build failed - bad interpreter: Permission denied

If you encounter this error while installing python and your server is a VPS, the /tmp directory where python-build download and compile the packages is probably mounted as noexec. You can check with your hosting provider if whether they provide a way to bypass this protection.

If the answer is no, just set the $TMPDIR environment variable to wherever you have a write + execution rights. For example:

export TMPDIR="$HOME/src"

Please note you'll have to do it every time you'll want to install a new version of python unless you write this command in your ~/.bashrc.

Build failed

If you've got something like that:

$ pyenv install 2.7.5
Downloading http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz...
Installing yaml-0.1.4...

BUILD FAILED

Please, make sure that "make" is installed ($ sudo apt-get install make). In Ubuntu Server, by default, it isn't.

Build failed: "ERROR: The Python zlib extension was not compiled. Missing the zlib?"

Installing Python-2.7.7...

ERROR: The Python zlib extension was not compiled. Missing the zlib?

Please consult to the Wiki page to fix the problem.
https://github.com/pyenv/pyenv/wiki/Common-build-problems

BUILD FAILED
  • On Mac OS X 10.9, 10.10, 10.11 and 10.13 you may need to set the CFLAGS environment variable when installing a new version in order for configure to find the zlib headers (XCode command line tools must be installed first):
CPPFLAGS="-I$(xcrun -show-sdk-path)/usr/include" pyenv install -v 2.7.7
  • If you installed zlib with Homebrew, you can set the CPPFLAGS environment variable:
CPPFLAGS="-I$(brew --prefix zlib)/include" pyenv install -v 3.7.0
  • Alternatively, try reinstalling XCode command line tools for your OS

If you experience both issues with openssl and zlib, you can specify both search paths as a compiler flag:

CPPFLAGS="-I$(brew --prefix openssl)/include -I$(xcrun -show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib"

If you experience issues with readline, you can also specify this as a compiler flag:

CPPFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix readline)/include -I$(xcrun -show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib"

If you are using macOS 10.14.6 with XCode 10.3, add the following:

SDKROOT=${XCODE_ROOT}/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk \
MACOSX_DEPLOYMENT_TARGET=10.14

If you are using Ubuntu/Debian, you need the following packages:

sudo apt install zlib1g zlib1g-dev libssl-dev libbz2-dev libsqlite3-dev

Build failed: Resource temporarily unavailable

If you see the string "resource temporarily unavailable" (often after "fork: "), then you may be on a shared host or other machine with low resource limits. Try running only a single job at a time to reduce resource usage:

MAKE_OPTS='-j 1' pyenv install 3.12.1

ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?

0. First, check

  • if you actually have OpenSSL and its headers installed (and for the right architecture and ecosystem if there are more than one at your machine)
    • Ubuntu: sudo apt install libssl-dev
    • Fedora: sudo dnf install openssl-devel
  • if the problem is resolved by upgrading Pyenv to the latest release and to the head version

1. OpenSSL is installed to an uncommon location.

Pass the location of its headers and libraries explicitly: (the openssl folder can be found by running openssl version -d)

CPPFLAGS="-I<openssl install prefix>/include" \
LDFLAGS="-L<openssl install prefix>/lib" \
pyenv install -v <python version>

or, alternatively, for Python 3.7+, instead of CPPFLAGS and LDFLAGS:

CONFIGURE_OPTS="--with-openssl=<openssl install prefix>"

or

LDFLAGS="-Wl,-rpath,<openssl install prefix>/lib" \
CONFIGURE_OPTS="--with-openssl=<openssl install prefix>" \
pyenv install -v <python version>

For python >= 3.10, add --with-openssl-rpath flag

LDFLAGS="-Wl,-rpath,<openssl install prefix>/lib" \
CONFIGURE_OPTS="--with-openssl=<openssl install prefix> --with-openssl-rpath=auto" \
pyenv install -v <python version>

E.g. (invocations that worked for various people):

  • RHEL6:

    CPPFLAGS=-I/usr/include/openssl \
LDFLAGS=-L/usr/lib64 \
pyenv install -v 3.4.3

  • CentOS 7 with OpenSSL 1.1.1:

    CPPFLAGS="$(pkg-config --cflags openssl11)" \
LDFLAGS="$(pkg-config --libs openssl11)" \
pyenv install -v 3.10.6

  • openSUSE:

    CPPFLAGS="$(pkg-config --cflags openssl)" \
LDFLAGS="$(pkg-config --libs openssl)" \
pyenv install -v 3.10.6

  • Arch Linux:

    LDFLAGS="-L/usr/lib/openssl-1.0" \
CPPFLAGS="-I/usr/include/openssl-1.0" \
pyenv install -v 3.4.3

  • If you installed openssl with macports or homebrew:

    CPPFLAGS="-I/opt/local/include/" \
LDFLAGS="-L/opt/local/lib/" \
pyenv install -v 3.4.3

  • On Ubuntu 14.04 on Dreamhost, an extra flag is required for Python 3.7+:

2. Your OpenSSL version is incompatible with the Python version you're trying to install

Old Python versions (for CPython, <3.5.3 and <2.7.13) require OpenSSL 1.0 while newer systems provide 1.1, and vice versa.
Note that OpenSSL 1.0 is EOL and by now practically unusable on the Internet due to using obsolete standards.

Install the right OpenSSL version, and point the build to its location as per above if needed.

E.g.:

  • On Debian stretch and Ubuntu bionic, libssl-dev is OpenSSL 1.1.x, but support for that was only added in Python 2.7.13, 3.5.3 and 3.6.0. To install earlier versions, you need to replace libssl-dev with libssl1.0-dev.

    sudo apt-get remove libssl-dev
sudo apt-get update
sudo apt-get install libssl1.0-dev

    https://github.com/pyenv/pyenv/issues/945#issuecomment-409627448 has a more complex workaround that preserves libssl-dev.

  • On FreeBSD 10-RELEASE and 11-CURRENT, you may need to recompile security/openssl without SSLv2 support. (See #464).

  • On Debian Jessie, you can use backports to install OpenSSL 1.0.2: sudo apt -t jessie-backports install openssl

python-build: definition not found

To update your python-build definitions:

If you have python-build installed as an pyenv plugin:

$ cd ~/.pyenv/plugins/python-build && git pull

macOS: "ld: symbol(s) not found for architecture x86_64" (#1245)

From (#1245).

This may be caused by an incompatible version of ar bundled with brew-distributed binutils. This is the reason why binutils is keg-only.

To fix, uninstall binutils brew remove binutils, or remove them from PATH, or execute the install command with AR=/usr/bin/ar.

Python cannot find a dependent dynamic library even though it's installed

If you're getting messages like this:

libreadline.so.7: cannot open shared object file: No such file or directory

or

ImportError: dlopen(/Users/durand/.pyenv/versions/3.10.1/lib/python3.10/site-packages/scipy/linalg/_fblas.cpython-310-darwin.so, 2): Library not loaded: /opt/homebrew/opt/gcc/lib/gcc/11/libgfortran.5.dylib
  Referenced from: /Users/durand/.pyenv/versions/3.10.1/lib/python3.10/site-packages/scipy/linalg/_fblas.cpython-310-darwin.so
  Reason: image not found

but you do have the corresponding package installed.

Check if the dynamic library's version you have installed is the same as what Python expects:

$ ls /lib/libreadline.so*
/lib/libreadline.so  /lib/libreadline.so.8  /lib/libreadline.so.8.0

Beside build time, this can also happen for an already installed version if:

  • You've installed a prebuilt version that was built for a different environment

    Many installation scripts for prebuilt versions give you a warning in such a case.

    • Replace the prebuilt version with a source one (usually, these are suffixed with -src if both a prebuilt and a source versions are provided); or
    • (not recommended, will make the system harder to maintain) Get or compile the right version of the library
      • it needs to be compiled for your system to avoid binary incompatibilies, so the best bets are either building from source or getting a binary from an official source for your distro

  • You've updated a dependent library on your system to a version with a different library filename (generally, to a new major version) since the time you had compiled Python

    • The easiest way is to rebuild all affected Python installations against the new version of the library with pyenv install <version> -force
    • If this happens for a specific 3rd-party Python package, you need to rebuild just that package.
    • There's a 3rd-party plugin that attempts to detect and do this automatically.

"python-build: definition not found" or another new feature missing even though you have a new enough Pyenv

If your Pyenv is installed with Homebrew, check if you have a second, old Pyenv installed and earlier on PATH:

zsh % whence -a pyenv
pyenv
/Users/admin/.pyenv/bin/pyenv     <-- an extraneous Pyenv in /Users/admin/.pyenv
/usr/local/bin/pyenv

bash $ which -a pyenv
/Users/admin/.pyenv/bin/pyenv
/usr/local/bin/pyenv

If yes, remove it.

"configure: error: internal configure error for the platform triplet, please file a bug report" or "configure: error: Unexpected output of 'arch' on OSX" in MacOS

checking for the platform triplet based on compiler characteristics... darwin
configure: error: internal configure error for the platform triplet, please file a bug report

or (note that the architecture is detected as arm; see another case with the same error below if the architecture is incorrect)

checking build system type... arm-apple-darwin<...>
checking host system type... arm-apple-darwin<...>
<...>
configure: error: Unexpected output of 'arch' on OSX

This means that the Python version you're installing doesn't support your MacOS and/or XCode version. In particular:

Keg-only Homebrew packages are forcibly linked / added to PATH

Some keg-only (explanation is below) Homebrew packages are known to break Pyenv builds if forcibly linked/added to PATH. Unlink/Remove them from your PATH.

Known error messages they cause:

  • Homebrew binutils on PATH (note that the architecture is detected as arm64 (aarch64 is also possible) rather than arm): 
    • checking build system type... arm64-apple-darwin<...>
checking host system type... arm64-apple-darwin<...>
configure: error: Unexpected output of 'arch' on OSX
  • Homebrew llvm on PATH:
    • warning: pointer is missing a nullability type specifier
    • llvm-ar: error: libpython3.10.a: Invalid record
    • error: unknown type name 'CF_IMPLICIT_BRIDGING_ENABLED'
  • Homebrew coreutils only causes breakage if non-prefixed executables are added to PATH.

Some Homebrew packages are installed as "keg-only" -- i.e. their executables are not linked to $(brew --prefix)/bin. This is typically done because then they would override stock MacOS software (the specific reason is mentioned in brew info output), causing breakages. The same happens if you manually add them to PATH as specified in their brew info output.

On Apple Silicon, when building for ARM64, a dependency is present in x64 Homebrew but not arm64 Homebrew

Known errors and their resolutions:

  • dyld[88714]: symbol not found in flat namespace '_libintl_bindtextdomain', __locale_textdomain in _localemodule.o ld: symbol(s) not found for architecture arm64
    • Install gettext into the arm64 Homebrew
  • unsupported hash type blake2s
  • ld: warning: ignoring file /usr/local/Cellar/<...>/libb2.dylib: found architecture 'x86_64', required architecture 'arm64'
  • [ERROR] _blake2 failed to import: dlopen(<...>/_blake2.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_blake2b_final'
    • Install libb2 into the arm64 Homebrew or uninstall it from the x64 Homebrew

Since x64's Homebrew is in /usr/local, it's always in the compiler's search path. We however prepend /opt/homebrew to the compiler's search path when compiling for arm64 (actually, when brew on PATH is pointing there).

So if a library is installed and linked in the x64 Homebrew but not in Arm64 Homebrew, the compiler still finds it, even if it's compiling for arm64. However, linking with it fails as it's for the wrong architecture.

BUILD FAILED (Linuxmint using python-build 2.4.15)

(See the "Pages" link above for a full list of wiki pages.)

Clone this wiki locally