Fat binaries of static libraries are not supported in the wheel #229
Description
Describe the bug
The new check to ensure the minimum macOS version of all MachO generates an error when fat binaries of static libraries are included in the wheel.
Our python wheel includes JIT compilation support using LLVM. For this we need to bundle some static libraries in the wheel that are used by the JIT linker. One of these static libraries libclang_rt.osx.a is a fat binary containing multiple architectures (e.g. arm64 and arm64e). The other "simple" static libraries are correctly ignored.
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/delocate/cmd/delocate_wheel.py", line 131, in <module>
main()
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/delocate/cmd/delocate_wheel.py", line 116, in main
copied = delocate_wheel(
^^^^^^^^^^^^^^^
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/delocate/delocating.py", line 1090, in delocate_wheel
out_wheel_fixed = _check_and_update_wheel_name(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/delocate/delocating.py", line 914, in _check_and_update_wheel_name
new_name, problematic_files = _calculate_minimum_wheel_name(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/delocate/delocating.py", line 816, in _calculate_minimum_wheel_name
for arch, version in _get_macos_min_version(lib):
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/delocate/delocating.py", line 619, in _get_macos_min_version
for header in MachO(dylib_path).headers:
^^^^^^^^^^^^^^^^^
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/macholib/MachO.py", line 121, in __init__
self.load(fp)
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/macholib/MachO.py", line 131, in load
self.load_fat(fh)
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/macholib/MachO.py", line 148, in load_fat
self.load_header(fh, arch.offset, arch.size)
File "/Users/aictx/.pyenv/versions/3.12.6/lib/python3.12/site-packages/macholib/MachO.py", line 170, in load_header
raise ValueError("Unknown Mach-O header: 0x%08x in %r" % (header, fh))
ValueError: Unknown Mach-O header: 0x213c6172 in <_io.BufferedReader name='/private/var/folders/z_/1_rkwk513p75p8gsy53mjb8r0000gn/T/tmp77ji4upg/wheel/samna/libclang_rt.osx.a'>To Reproduce
Steps used to reproduce the behavior, such as the delocate commands used.
- Add a static library as a fat binary containing multiple architectures.
- Run
python -m delocate.cmd.delocate_wheel $WHEEL_PATH -w $WHEELHOUSE_PATH
Expected behavior
Delocating the wheel is successful, as in version 0.10.7.
Proposals:
- Only check the files that were touched by delocate.
- Add a parameter for files to ignore, the current filters don't influence the check.
- Add a parameter to skip the check.
Wheels used
Unfortunately the wheel is private by my company.
Platform (please complete the following information):
- OS version: macOS 12.1
- Delocate version: 0.11.0
Additional context
My understanding of the problem is as follows. The _calculate_minimum_wheel_name looks at all the files in the wheel. It sees the static library and because it's a fat binary it contains the MachO magic number (0xCAFEBABF). Now it assumes that it's a dynamic library and looks for the magic number (0xFEEDFACF) for single binaries at the offsets given by the header. This magic number is missing because it's a static library, not a dynamic one.