Generate/test Manylinux wheels, clean up wheel/build process #1105
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sc1f
added
enhancement
Closed
|
Thanks for the PR! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes (hopefully for the final time) our build process to build Manylinux-compliant wheels.
Due to an issue in Auditwheel (fixed and released in auditwheel 3.2.0), our internal C++ bindings were not being picked up afterauditwheel repairreplaces the RPATHs of the C++ binding.A previous iteration of this PR required a rebuild of our docker images, however I removed a hack from our Linux build where it inserted
./../../pyarrowinto the RPATH, which was originally designed as a "hint" to look for pyarrow immediately relative to the install location (usually site-packages). By removing this hack from the Linux build (but keeping it in MacOS), Auditwheel was able to properly delocate the shared libraries and repair the wheel.Testing on a clean
python:3.6-busterand apython:3.7-busterimage show that the wheels install properly frompip install, and after pulling down theperspective-pythontest suite and running it against the installed version from wheel, all tests are shown to pass.Additionally, this PR also adds
test_wheels.shwhich reduces our bloat inazure-pipelinesby externalizing the build/install in venv/test process for wheels and making it more maintainable.