-
Notifications
You must be signed in to change notification settings - Fork 36.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
depends: remove FORCE_USE_SYSTEM_CLANG
#30201
depends: remove FORCE_USE_SYSTEM_CLANG
#30201
Conversation
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code CoverageFor detailed information about the code coverage, see the test coverage report. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
depends/README.md
Outdated
@@ -45,7 +45,7 @@ The paths are automatically configured and no other options are needed. | |||
|
|||
#### For macOS cross compilation | |||
|
|||
sudo apt-get install curl bsdmainutils cmake zip | |||
sudo apt-get install clang cmake curl lld llvm make zip |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Could make sense which version is "recommended" or "tested". I presume the version used in CI? If so, could refer to that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea. Depending on when this lands, we could update this to recommend LLVM 18 (assuming we also switch to 18 in Guix), as that's likely what is going to be used in the CI, if we just apt install clang
on Ubuntu 24.04
. At a minimum, given we use -fixup_chains
, we require at least LLVM 16 or later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably best to move everything (CI + guix) at the same time, in the same pull?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea. If/when it does change, everything will continue to change in lockstep.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK
a57d897
to
4964c44
Compare
0a3631f depends: fix Qt macOS build with Clang 18 (fanquake) b018bd7 depends: qt 5.15.14 (fanquake) Pull request description: Also adds a patch to Qts internal libpng, to fix compilation using Clang 18, when targetting macOS. I'd like to get this patched, so we can continue working on removing `FORCE_USE_SYSTEM_CLANG` (#30201); otherwise builds will be broken using the default Clang (`18`) on the current Ubuntu LTS (`24.04`). With this PR, anyone using Ubuntu 24.04 should be able to `apt install clang llvm lld`, and then cross-compile for macOS using: ```bash # clang --version Ubuntu clang version 18.1.3 (1) make -C depends HOST=arm64-apple-darwin FORCE_USE_SYSTEM_CLANG=1 ./autogen.sh CONFIG_SITE=/path/to/depends/arm64-apple-darwin/share/config.site ./configure make # file src/qt/bitcoin-qt src/qt/bitcoin-qt: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE|HAS_TLV_DESCRIPTORS> ``` ACKs for top commit: TheCharlatan: ACK 0a3631f theuni: utACK 0a3631f hebasto: ACK 0a3631f, a new patch indeed fixes cross-compiling on Ubuntu 24.04 with `FORCE_USE_SYSTEM_CLANG=1`. Tree-SHA512: 711d321b1efbb1aeef802d1d7e72fff8f4e28aa2420d19df9db6f4449fc7d281e1d08ba242ce20122dfe21129e713bd59e7e6ade0b67d7271eea18b39ceb9283
4964c44
to
9af6d56
Compare
Concept ACK. |
9af6d56
to
8518b92
Compare
Includes: LLVM 18.1.x (bitcoin#30201) GCC 13.x (bitcoin#29881) Kernel Headers 6.1.80 -> 6.1.92 Commits like https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824, which should improve the bootstrap situation (bitcoin#30042).
Includes: LLVM 18.1.x (bitcoin#30201) GCC 13.x (bitcoin#29881) Kernel Headers 6.1.80 -> 6.1.92 Commits like https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824, which should improve the bootstrap situation (bitcoin#30042).
Includes: LLVM 18.1.x (bitcoin#30201) GCC 13.x (bitcoin#29881) git-minimal 2.41.0 -> 2.45.1 Kernel Headers 6.1.80 -> 6.1.92 moreutils 0.68 -> 0.69 Commits like https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824, which should improve the bootstrap situation (bitcoin#30042).
Includes: LLVM 18.1.x (bitcoin#30201) GCC 13.x (bitcoin#29881) git-minimal 2.41.0 -> 2.45.1 Kernel Headers 6.1.80 -> 6.1.92 moreutils 0.68 -> 0.69 Commits like https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824, which should improve the bootstrap situation (bitcoin#30042).
…b8c73efed126 2599655 guix: bump time-machine to f0bb724211872cd6158fce6162e0b8c73efed126 (fanquake) Pull request description: Includes: LLVM 18.1.x (#30201) GCC 13.x (#29881) git-minimal 2.41.0 -> 2.45.1 Kernel Headers 6.1.80 -> 6.1.92 moreutils 0.68 -> 0.69 Commits like https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824, which should improve the bootstrap situation (#30042). This can somewhat be visualised by comparing the (simplified) dependencies of guix itself, between the two time-machines. Master: ![master_2](https://github.com/bitcoin/bitcoin/assets/863730/714402a2-345e-43c7-974b-5112d03d44c2) PR: ![pr](https://github.com/bitcoin/bitcoin/assets/863730/7079a155-e013-4d59-9ea1-21a64d71e2d8) Note that in the case of this PR, we are better off, no-longer having to build a number of tex packages, ruby, cairo, graphics libs, openssl 1.x etc. ACKs for top commit: TheCharlatan: ACK 2599655 Tree-SHA512: 9c5675a5d563c17744c89c8a392bc7865aa1c9e71a5e3044c23f31e51458dac28c0c238d2055c86dc732df595dae60bcbc8b85266b23b7991c4c770d56f7d23a
8518b92
to
81f1516
Compare
81f1516
to
b80a846
Compare
We are going to use Clang 18, which isn't on Jammy, so bump to Noble.
Change some references to the macho ld64 to lld, which is now what is used.
b80a846
to
7cbfd7a
Compare
Added a commit to rename some references to macho ld64 to lld. |
tACK 7cbfd7a I made a Guix build, as well as a regular cross-compile on Ubuntu 24.04 for Intel macOS, and lightly tested them. Guix hashes for darwin:
|
Guix Build (aarch64): f8487a885a04a4b3c273b2d1ba3ef5e100026f16b03c08e866dbf4cd468d0802 guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/SHA256SUMS.part
b3da7604bc7302213d2864bddbccc54ede6374711377b89b2d17a989d2e9e64d guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/bitcoin-7cbfd7a7ce45-aarch64-linux-gnu-debug.tar.gz
d2340b2c084faf01a0db8ea0c077ba9f8c51ef23b680396d88001b33d126788b guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/bitcoin-7cbfd7a7ce45-aarch64-linux-gnu.tar.gz
a79f460ac8b4c893ad1cf4aba17489b0dc7834377166d4e9bc4da78bcd84a430 guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/SHA256SUMS.part
6d94b11fca39489230f00125d156c1ad36a8b8ff9c5fdd6a7d47e31c3c527230 guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/bitcoin-7cbfd7a7ce45-arm-linux-gnueabihf-debug.tar.gz
e5cf9b6bd1a8ad95a49ad523681091ad9938c02f7f995934c413bd8b8c651199 guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/bitcoin-7cbfd7a7ce45-arm-linux-gnueabihf.tar.gz
048fa3f381098477b8bea25701547f37a6680b33afb9c61ba88cb4a895aae81d guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/SHA256SUMS.part
d540c4c73b98bef810c85b0b10a6e1a004655fb0c8a4709b8f655c6924afef8b guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.tar.gz
4a0bca38e194b731a105ab791d8dfa6adc314fb0a5a963d49136d5546794919b guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.zip
993c355cdef9a246c9929dc5b9a734ae4781f6353a8c953465c81ae2739809cd guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin.tar.gz
2e58229e7f9f22448fda71b0eda9fb4f084edb1a44093e0dadf9a2ab0ab37026 guix-build-7cbfd7a7ce45/output/dist-archive/bitcoin-7cbfd7a7ce45.tar.gz
18116e5c3df56d933ce453dd9b475317fe3a9e4452fa420e57cfeb15f6dd0580 guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/SHA256SUMS.part
d23d0c154fa27210e0be3992bd34882db469dde51070b975cb5ee6b365bfc10b guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/bitcoin-7cbfd7a7ce45-powerpc64-linux-gnu-debug.tar.gz
92d17f3834ee8f8f14cea0ff7e183c8efb1a17355d4fd654ac0a96e01c880aa3 guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/bitcoin-7cbfd7a7ce45-powerpc64-linux-gnu.tar.gz
3bd64df4a6e6fd27ac72c48691e41a5b528446a4052c42caaf6e91f32419d327 guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/SHA256SUMS.part
dd23855a2f58b3334b274630b7c97be99335ff9908e8a6b0c1300a141b92f5a0 guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/bitcoin-7cbfd7a7ce45-riscv64-linux-gnu-debug.tar.gz
a40be60dd17b32b6c8c8560e579bfb02c08e2e8f082a2247b66f957b2780fee9 guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/bitcoin-7cbfd7a7ce45-riscv64-linux-gnu.tar.gz
1140b74f33745cdd0e703c7740b3adf79b233be9becca3d464b41fc508ebc3f0 guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/SHA256SUMS.part
ae7b6911d7c184e9643f8a611db02c5c5028d116937b42935052dea04cc834cf guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.tar.gz
eea2db60805e1a97ffb9cf412380e5df178dd2f7f5e42e1a06feae3185f87a79 guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.zip
90bc2c35c76cce5620fcfeb57f108e02f6be6d7d3ac36c88d9a8f991b9637b7b guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin.tar.gz
8b758e8b739b80123a132ceba172ba9fd70a476941a1b515629737b9f6020ab9 guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/SHA256SUMS.part
cd50dfb8d19c5ea62b1069655c5154965703fccfa31bad9c7169b2842a5c5048 guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/bitcoin-7cbfd7a7ce45-x86_64-linux-gnu-debug.tar.gz
1496518bd7efe42df95318666a14c97ae019a6d66c526906c61d22dd8769769d guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/bitcoin-7cbfd7a7ce45-x86_64-linux-gnu.tar.gz
56d85879d391893a174dce042ed7a4d54c4e57645629660952411052e35d793c guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/SHA256SUMS.part
633ae51f73e818edcfd7e6a03d6368abe58540a485908aa17dbd812cc790d55f guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-debug.zip
480576fb02cc43bcaea583b4975e7609c093c53a16c891e1ab1c75801cd9da18 guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-setup-unsigned.exe
ff2e48fb53aca49d3d0fcda480e64cf36f4b8250357c1adeec0abba6b66bb524 guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-unsigned.tar.gz
42ca2d89cb6703094ea22bd9a869d833f36f42cbef39ca5c3c7b8503bbe15587 guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64.zip |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested on Ubuntu 22.04 with downloaded llvm 18 binaries in PATH.
ACK 7cbfd7a
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, ACK 7cbfd7a
Guix builds (aarch64):
f8487a885a04a4b3c273b2d1ba3ef5e100026f16b03c08e866dbf4cd468d0802 guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/SHA256SUMS.part
b3da7604bc7302213d2864bddbccc54ede6374711377b89b2d17a989d2e9e64d guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/bitcoin-7cbfd7a7ce45-aarch64-linux-gnu-debug.tar.gz
d2340b2c084faf01a0db8ea0c077ba9f8c51ef23b680396d88001b33d126788b guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/bitcoin-7cbfd7a7ce45-aarch64-linux-gnu.tar.gz
a79f460ac8b4c893ad1cf4aba17489b0dc7834377166d4e9bc4da78bcd84a430 guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/SHA256SUMS.part
6d94b11fca39489230f00125d156c1ad36a8b8ff9c5fdd6a7d47e31c3c527230 guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/bitcoin-7cbfd7a7ce45-arm-linux-gnueabihf-debug.tar.gz
e5cf9b6bd1a8ad95a49ad523681091ad9938c02f7f995934c413bd8b8c651199 guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/bitcoin-7cbfd7a7ce45-arm-linux-gnueabihf.tar.gz
048fa3f381098477b8bea25701547f37a6680b33afb9c61ba88cb4a895aae81d guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/SHA256SUMS.part
d540c4c73b98bef810c85b0b10a6e1a004655fb0c8a4709b8f655c6924afef8b guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.tar.gz
4a0bca38e194b731a105ab791d8dfa6adc314fb0a5a963d49136d5546794919b guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.zip
993c355cdef9a246c9929dc5b9a734ae4781f6353a8c953465c81ae2739809cd guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin.tar.gz
2e58229e7f9f22448fda71b0eda9fb4f084edb1a44093e0dadf9a2ab0ab37026 guix-build-7cbfd7a7ce45/output/dist-archive/bitcoin-7cbfd7a7ce45.tar.gz
18116e5c3df56d933ce453dd9b475317fe3a9e4452fa420e57cfeb15f6dd0580 guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/SHA256SUMS.part
d23d0c154fa27210e0be3992bd34882db469dde51070b975cb5ee6b365bfc10b guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/bitcoin-7cbfd7a7ce45-powerpc64-linux-gnu-debug.tar.gz
92d17f3834ee8f8f14cea0ff7e183c8efb1a17355d4fd654ac0a96e01c880aa3 guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/bitcoin-7cbfd7a7ce45-powerpc64-linux-gnu.tar.gz
3bd64df4a6e6fd27ac72c48691e41a5b528446a4052c42caaf6e91f32419d327 guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/SHA256SUMS.part
dd23855a2f58b3334b274630b7c97be99335ff9908e8a6b0c1300a141b92f5a0 guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/bitcoin-7cbfd7a7ce45-riscv64-linux-gnu-debug.tar.gz
a40be60dd17b32b6c8c8560e579bfb02c08e2e8f082a2247b66f957b2780fee9 guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/bitcoin-7cbfd7a7ce45-riscv64-linux-gnu.tar.gz
1140b74f33745cdd0e703c7740b3adf79b233be9becca3d464b41fc508ebc3f0 guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/SHA256SUMS.part
ae7b6911d7c184e9643f8a611db02c5c5028d116937b42935052dea04cc834cf guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.tar.gz
eea2db60805e1a97ffb9cf412380e5df178dd2f7f5e42e1a06feae3185f87a79 guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.zip
90bc2c35c76cce5620fcfeb57f108e02f6be6d7d3ac36c88d9a8f991b9637b7b guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin.tar.gz
8b758e8b739b80123a132ceba172ba9fd70a476941a1b515629737b9f6020ab9 guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/SHA256SUMS.part
cd50dfb8d19c5ea62b1069655c5154965703fccfa31bad9c7169b2842a5c5048 guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/bitcoin-7cbfd7a7ce45-x86_64-linux-gnu-debug.tar.gz
1496518bd7efe42df95318666a14c97ae019a6d66c526906c61d22dd8769769d guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/bitcoin-7cbfd7a7ce45-x86_64-linux-gnu.tar.gz
56d85879d391893a174dce042ed7a4d54c4e57645629660952411052e35d793c guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/SHA256SUMS.part
633ae51f73e818edcfd7e6a03d6368abe58540a485908aa17dbd812cc790d55f guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-debug.zip
480576fb02cc43bcaea583b4975e7609c093c53a16c891e1ab1c75801cd9da18 guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-setup-unsigned.exe
ff2e48fb53aca49d3d0fcda480e64cf36f4b8250357c1adeec0abba6b66bb524 guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-unsigned.tar.gz
42ca2d89cb6703094ea22bd9a869d833f36f42cbef39ca5c3c7b8503bbe15587 guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64.zip
CI related stuff has to be ported to the CMake staging branch. |
Ported to the CMake-based build system in hebasto#264. |
Summary: ``` Remove FORCE_USE_SYSTEM_CLANG in favour of always using the system Clang and lld for macOS cross-compilation; rather than downloading precompiled blobs. ``` Backport of [[bitcoin/bitcoin#30201 | core#30201]] and [[bitcoin/bitcoin#30231 | core#30231]]. Depends on D17265. Note the CI update will be done in another diff. It only affects the cross osx build on CI which is currently paused. Test Plan: Run the guix osx build. Reviewers: #bitcoin_abc, PiRK Reviewed By: #bitcoin_abc, PiRK Differential Revision: https://reviews.bitcoinabc.org/D17266
Summary: ``` Remove FORCE_USE_SYSTEM_CLANG in favour of always using the system Clang and lld for macOS cross-compilation; rather than downloading precompiled blobs. ``` Backport of [[bitcoin/bitcoin#30201 | core#30201]] and [[bitcoin/bitcoin#30231 | core#30231]]. Depends on D17265. Note the CI update will be done in another diff. It only affects the cross osx build on CI which is currently paused. Test Plan: Run the guix osx build. Reviewers: #bitcoin_abc, PiRK Reviewed By: #bitcoin_abc, PiRK Differential Revision: https://reviews.bitcoinabc.org/D17266
, bitcoin#28757, bitcoin#28845, bitcoin#28932, bitcoin#28973, bitcoin#21778, bitcoin#30204, bitcoin#29765, bitcoin#30201, bitcoin#30287, bitcoin#30511 (toolchain backports: part 3) ee04e1f merge bitcoin#30511: GCC 12 consolidation (Kittywhiskers Van Gogh) cc5c30e merge bitcoin#30287: rewrite some docs & swap `mmacosx-version-min` for `mmacos-version-min` (Kittywhiskers Van Gogh) 632b8b4 merge bitcoin#30201: remove `FORCE_USE_SYSTEM_CLANG` (Kittywhiskers Van Gogh) 994e7b1 merge bitcoin#29765: Use clang-18 (Kittywhiskers Van Gogh) 7e7d7bd merge bitcoin#30204: consolidate dependency docs (Kittywhiskers Van Gogh) 061a608 merge bitcoin#21778: LLD based macOS toolchain (Kittywhiskers Van Gogh) ea891d4 fix: use `LDFLAGS` when compiling `gmp`'s `configure` test programs (Kittywhiskers Van Gogh) c2fe7c6 merge bitcoin#28973: remove `libz-dev` from macOS build deps (Kittywhiskers Van Gogh) 62e671d merge bitcoin#28932: remove `python3-setuptools` from macOS build deps (Kittywhiskers Van Gogh) 5bef1e5 merge bitcoin#28845: remove `PYTHONPATH` from config.site (Kittywhiskers Van Gogh) c501a0d merge bitcoin#28757: Zip needs to include all files and set time to SOURCE_DATE_EPOCH (Kittywhiskers Van Gogh) 122a61f merge bitcoin#28432: Produce a `.zip` for macOS distribution (Kittywhiskers Van Gogh) 407deab merge bitcoin#25612: default to using GCC tool wrappers for LTO (with GCC) (Kittywhiskers Van Gogh) f0a0040 fix: ensure that `dsymutil` available even when `FORCE_USE_SYSTEM_CLANG=1` (Kittywhiskers Van Gogh) 416e154 merge bitcoin#24866: No longer need to hack the PATH variable in `config.site` (Kittywhiskers Van Gogh) Pull request description: ## Additional Information * Depends on #6385 * Omissions related to explicitly defining the location of `DSYMUTIL` (see [dash#6384](#6384) for more information) have been remedied and are now included (completing [bitcoin#24866](bitcoin#24866) and [bitcoin#25612](bitcoin#25612) in this pull request), courtesy of a fix that defines `DSYMUTIL` even when `FORCE_USE_SYSTEM_CLANG` is defined ([commit](f0a0040)). * This fix is effectively reverted in [bitcoin#21778](bitcoin#21778) as the backport ensures that `DSYMUTIL` is defined in the `FORCE_USE_SYSTEM_CLANG` case and `cctools` is no longer used. * It still remains to allow testing and validation of commits pre-[bitcoin#21778](bitcoin#21778). * GMP runs its test programs during build configuration sans `LDFLAGS` ([source](https://github.com/gmp-mirror/gmp/blob/141ed4f98a50e4a3c1a95a1758cb599b7499ac39/acinclude.m4#L1755)), which are quite important when using an LLD-based toolchain as we rely on it to take on `ld64`'s duties ([source](https://lld.llvm.org/MachO/index.html)) using `-fuse-ld=lld` ([source](https://lld.llvm.org/#using-lld)). This has been remedied by patching `acinclude.m4` and `configure.ac` to include `LDFLAGS`. <details> <summary>configure failure:</summary> ``` checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-apple-darwin checking for a BSD-compatible install... /home/debian/.guix-profile/bin/install -c checking whether build environment is sane... yes checking for x86_64-apple-darwin-strip... no checking for strip... strip configure: WARNING: using cross tools not prefixed with host triplet checking for a thread-safe mkdir -p... /home/debian/.guix-profile/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether to enable maintainer-specific portions of Makefiles... no checking ABI=64 checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include... no checking ABI=x32 checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include... no checking ABI=32 checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include... no configure: error: could not find a working compiler, see config.log for details make: *** [funcs.mk:302: /dash/depends/x86_64-apple-darwin/.gmp_stamp_configured] Error 1 make: Leaving directory '/dash/depends' ``` </details> <details> <summary>config.log excerpt:</summary> ``` This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by GNU MP configure 6.3.0, which was generated by GNU Autoconf 2.69. Invocation command line was [...] ## --------- ## ## Platform. ## ## --------- ## hostname = [...] uname -m = x86_64 uname -r = 6.6.32[...] uname -s = Linux uname -v = #1 SMP PREEMPT_DYNAMIC Mon Jul 8 16:11:58 UTC 2024 /usr/bin/uname -p = unknown /bin/uname -X = unknown /bin/arch = unknown /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown /usr/bin/hostinfo = unknown /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown PATH: /dash/depends/x86_64-apple-darwin/native/bin PATH: /home/debian/.guix-profile/bin PATH: /home/debian/.guix-profile/sbin ## ----------- ## ## Core tests. ## ## ----------- ## [...] configure:5889: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -c conftest.c >&5 configure:5892: $? = 0 configure:5895: checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc configure:5897: result: yes configure:5911: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks 2>&1 | grep xlc >/dev/null configure:5914: $? = 1 configure:5968: checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include Test compile: configure:5982: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include conftest.c >&5 ld: unrecognized option '-dynamic' ld: use the --help option for usage information clang: error: linker command failed with exit code 1 (use -v to see invocation) configure:5985: $? = 1 failed program was: int main () { return 0; } configure:7072: result: no [...] configure:7310: error: could not find a working compiler, see config.log for details [...] ## ---------------- ## ## Cache variables. ## ## ---------------- ## [...] ac_cv_env_LDFLAGS_set=set ac_cv_env_LDFLAGS_value='-Wl,-platform_version,macos,11.0,14.0 -Wl,-no_adhoc_codesign -fuse-ld=lld -L/dash/depends/x86_64-apple-darwin/lib' [...] LD='' LDFLAGS='-Wl,-platform_version,macos,11.0,14.0 -Wl,-no_adhoc_codesign -fuse-ld=lld -L/dash/depends/x86_64-apple-darwin/lib' [...] ## ----------- ## ## confdefs.h. ## ## ----------- ## /* confdefs.h */ #define PACKAGE_NAME "GNU MP" #define PACKAGE_TARNAME "gmp" #define PACKAGE_VERSION "6.3.0" #define PACKAGE_STRING "GNU MP 6.3.0" #define PACKAGE_BUGREPORT "gmp-bugs@gmplib.org (see https://gmplib.org/manual/Reporting-Bugs.html)" #define PACKAGE_URL "http://www.gnu.org/software/gmp/" #define PACKAGE "gmp" #define VERSION "6.3.0" #define WANT_FFT 1 #define HAVE_HOST_CPU_x86_64 1 configure: exit 1 ``` **Note:** `lld` itself will never respond like this, as it will exit with a catch-all message requesting you to invoke it with the correct name ``` $ lld -dynamic lld is a generic driver. Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly) instead ``` Nor is it calling the Unix-specific variant mistakenly ``` $ ld.lld -dynamic ld.lld: error: unknown argument '-dynamic', did you mean '-Bdynamic' ``` The error displayed above is consistent with GNU `ld` ``` $ ld -dynamic ld: unrecognized option '-dynamic' ld: use the --help option for usage information ``` </details> * A similar problem is observed for Qt, which was resolved upstream by adding it to `CXXFLAGS` ([source](https://github.com/bitcoin/bitcoin/blob/e8c25e8a35e333e90514945c592557615641553f/depends/patches/qt/mac-qmake.conf#L16)) (but this comes at the downside of an unused argument warnings whenever the compiler doesn't intend to do any linking and is why we didn't just modify `darwin_{CC,CXX}` instead) <details> <summary>Warning messages when building Qt:</summary> ``` clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qplatformcursor.cpp compiling kernel/qplatformclipboard.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qplatformsurface.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] 4 warnings generated. compiling kernel/qcursor.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qguivariant.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qshortcutmap.cpp compiling kernel/qtouchdevice.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qplatformservices.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] kernel/qplatformcursor.cpp:622:17: warning: variable 'w' set but not used [-Wunused-but-set-variable] 622 | int x = -1, w = 0; | ^ compiling kernel/qplatformsessionmanager.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] ``` </details> * Clang has been bumped to 18 as [bitcoin#30201](bitcoin#30201) drops `native_llvm` (formerly known as `native_clang`) and mandates the presence of Clang 18 or higher for cross-compilation ([source](https://github.com/bitcoin/bitcoin/blob/7cbfd7a7ce45ac68d6041f42f468862f5c193d8c/depends/README.md?plain=1#L54-L57)). ## Breaking Changes None expected ## Checklist - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: PastaPastaPasta: utACK ee04e1f UdjinM6: utACK ee04e1f Tree-SHA512: 0b8be036419a53eda5be77c9abdf214d8f5baa088526f9249daba31e650c50f831f54c389845f828f8b7cf50458967f211a6d42d12b8dae8546b265d7f37a968
Remove
FORCE_USE_SYSTEM_CLANG
in favour of always using the system Clang and lld for macOS cross-compilation; rather than downloading precompiled blobs.For example, anyone using Ubuntu 24.04 should be able to
apt install clang llvm lld .. etc
, and then cross-compile for macOS using:Note that the minimum supported version of Clang we will support for macOS cross-compilation will likely be more recent than our other minimum supported version of Clang, due to compiler/linker option usage.