Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: cross build some modules requiring external libraries #146

Open
lazka opened this issue Aug 16, 2023 · 2 comments
Open

CI: cross build some modules requiring external libraries #146

lazka opened this issue Aug 16, 2023 · 2 comments

Comments

@lazka
Copy link
Member

lazka commented Aug 16, 2023 •
edited
Loading

The following modules don't build in the cross CI jobs currently because external dependencies are missing:

  • Modules/pyexpat.c:5:10: fatal error: 'expat.h' file not found
  • Modules/_elementtree.c:3033:10: fatal error: 'expat.h' file not found
  • Modules/_decimal/_decimal.c:35:10: fatal error: 'mpdecimal.h' file not found
  • Modules/_ctypes/_ctypes.c:118:10: fatal error: 'ffi.h' file not found
  • _sqlite3 is disabled somehow

We could build the external dependencies, then build Python with them, dist them, and test that the modules can be imported in the "cross-gcc-x86_64-test" job.

This is motivated by #145 (comment)
@lazka lazka mentioned this issue Aug 16, 2023
Merged
@jeremyd2019
Copy link

When I was testing cross-compilation within msys2 using msys2/MINGW-packages#8762 I was able to build all the modules that were included in the MINGW-packages mingw-w64-python package. But that was with 3.10, I have not tried that with 3.11 yet.

@jeremyd2019
Copy link

jeremyd2019 commented Aug 16, 2023 •
edited
Loading

I just tried with 3.11 (before #145 was merged) and successfully got this summary from setup.py:

The necessary bits to build these optional modules were not found:
_crypt                _dbm                  _gdbm
_posixshmem           _posixsubprocess      _uuid
fcntl                 grp                   nis
ossaudiodev           readline              resource
spwd                  syslog                termios
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

Which looks correct. In case it helps, here's the script I used (needed minor tweaks for python 3.11 and the PKG_CONFIG_LIBDIR variable I just learned about)

Python cross-compile script for MSYS2 
#!/bin/bash -ex

TARGET=${TARGET:-aarch64-w64-mingw32}
PREFIX=${PREFIX:-/clangarm64}

export PKG_CONFIG_LIBDIR=${PREFIX}/lib/pkgconfig:${PREFIX}/share/pkgconfig
export CC=${TARGET}-clang
export CXX=${CC}++
pacman -S --asdeps --needed --noconfirm --noprogressbar autoconf autoconf-archive automake-wrapper

git clone --depth 1 --single-branch https://github.com/msys2-contrib/cpython-mingw.git
cd cpython-mingw
autoreconf -vfi

mkdir _build && cd _build
../configure \
        --prefix=${PREFIX} \
        --build=${MINGW_CHOST} \
        --host=${TARGET} \
        --with-build-python=python \
        --enable-shared \
        --with-system-expat \
        --with-system-ffi \
        --with-system-libmpdec \
        --without-ensurepip \
        --enable-loadable-sqlite-extensions \
        --with-tzpath=${PREFIX}/share/zoneinfo

make -j8
make -j1 install DESTDIR=python_pkgdir

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lazka @jeremyd2019