Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

depends: remove FORCE_USE_SYSTEM_CLANG #30201

Merged
merged 8 commits into from
Jun 11, 2024

Conversation

fanquake
Copy link
Member

@fanquake fanquake commented May 30, 2024

Remove FORCE_USE_SYSTEM_CLANG in favour of always using the system Clang and lld for macOS cross-compilation; rather than downloading precompiled blobs.

For example, anyone using Ubuntu 24.04 should be able to apt install clang llvm lld .. etc, and then cross-compile for macOS using:

# clang --version
Ubuntu clang version 18.1.3 (1)
make -C depends HOST=arm64-apple-darwin FORCE_USE_SYSTEM_CLANG=1
./autogen.sh
CONFIG_SITE=/path/to/depends/arm64-apple-darwin/share/config.site ./configure
make
# file src/qt/bitcoin-qt
src/qt/bitcoin-qt: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE|HAS_TLV_DESCRIPTORS>

Note that the minimum supported version of Clang we will support for macOS cross-compilation will likely be more recent than our other minimum supported version of Clang, due to compiler/linker option usage.

@DrahtBot
Copy link
Contributor

DrahtBot commented May 30, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK Sjors, theuni, TheCharlatan
Concept ACK hebasto

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #29881 (guix: use GCC 13 to builds releases by fanquake)
  • #25391 (guix: Use LTO to build releases by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@@ -45,7 +45,7 @@ The paths are automatically configured and no other options are needed.

#### For macOS cross compilation

sudo apt-get install curl bsdmainutils cmake zip
sudo apt-get install clang cmake curl lld llvm make zip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Could make sense which version is "recommended" or "tested". I presume the version used in CI? If so, could refer to that.

Copy link
Member Author

@fanquake fanquake May 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea. Depending on when this lands, we could update this to recommend LLVM 18 (assuming we also switch to 18 in Guix), as that's likely what is going to be used in the CI, if we just apt install clang on Ubuntu 24.04. At a minimum, given we use -fixup_chains, we require at least LLVM 16 or later.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably best to move everything (CI + guix) at the same time, in the same pull?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea. If/when it does change, everything will continue to change in lockstep.

Copy link
Member

@theuni theuni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

@fanquake fanquake force-pushed the remove_force_use_system_clang branch from a57d897 to 4964c44 Compare May 30, 2024 16:39
fanquake added a commit that referenced this pull request May 31, 2024
0a3631f depends: fix Qt macOS build with Clang 18 (fanquake)
b018bd7 depends: qt 5.15.14 (fanquake)

Pull request description:

  Also adds a patch to Qts internal libpng, to fix compilation using Clang 18, when targetting macOS. I'd like to get this patched, so we can continue working on removing `FORCE_USE_SYSTEM_CLANG` (#30201); otherwise builds will be broken using the default Clang (`18`) on the current Ubuntu LTS (`24.04`).

  With this PR, anyone using Ubuntu 24.04 should be able to `apt install clang llvm lld`, and then cross-compile for macOS using:
  ```bash
  # clang --version
  Ubuntu clang version 18.1.3 (1)
  make -C depends HOST=arm64-apple-darwin FORCE_USE_SYSTEM_CLANG=1
  ./autogen.sh
  CONFIG_SITE=/path/to/depends/arm64-apple-darwin/share/config.site ./configure
  make
  # file src/qt/bitcoin-qt
  src/qt/bitcoin-qt: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE|HAS_TLV_DESCRIPTORS>
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 0a3631f
  theuni:
    utACK 0a3631f
  hebasto:
    ACK 0a3631f, a new patch indeed fixes cross-compiling on Ubuntu 24.04 with `FORCE_USE_SYSTEM_CLANG=1`.

Tree-SHA512: 711d321b1efbb1aeef802d1d7e72fff8f4e28aa2420d19df9db6f4449fc7d281e1d08ba242ce20122dfe21129e713bd59e7e6ade0b67d7271eea18b39ceb9283
@fanquake fanquake force-pushed the remove_force_use_system_clang branch from 4964c44 to 9af6d56 Compare May 31, 2024 14:32
@hebasto
Copy link
Member

hebasto commented May 31, 2024

Concept ACK.

@fanquake fanquake force-pushed the remove_force_use_system_clang branch from 9af6d56 to 8518b92 Compare June 4, 2024 13:57
fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 5, 2024
Includes:
LLVM 18.1.x (bitcoin#30201)
GCC 13.x (bitcoin#29881)
Kernel Headers 6.1.80 -> 6.1.92
Commits like
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824,
which should improve the bootstrap situation (bitcoin#30042).
fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 5, 2024
Includes:
LLVM 18.1.x (bitcoin#30201)
GCC 13.x (bitcoin#29881)
Kernel Headers 6.1.80 -> 6.1.92
Commits like
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824,
which should improve the bootstrap situation (bitcoin#30042).
fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 5, 2024
Includes:
LLVM 18.1.x (bitcoin#30201)
GCC 13.x (bitcoin#29881)

git-minimal 2.41.0 -> 2.45.1
Kernel Headers 6.1.80 -> 6.1.92
moreutils 0.68 -> 0.69

Commits like
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824,
which should improve the bootstrap situation (bitcoin#30042).
fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 6, 2024
Includes:
LLVM 18.1.x (bitcoin#30201)
GCC 13.x (bitcoin#29881)

git-minimal 2.41.0 -> 2.45.1
Kernel Headers 6.1.80 -> 6.1.92
moreutils 0.68 -> 0.69

Commits like
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824,
which should improve the bootstrap situation (bitcoin#30042).
fanquake added a commit that referenced this pull request Jun 8, 2024
…b8c73efed126

2599655 guix: bump time-machine to f0bb724211872cd6158fce6162e0b8c73efed126 (fanquake)

Pull request description:

  Includes:
  LLVM 18.1.x (#30201)
  GCC 13.x (#29881)

  git-minimal 2.41.0 -> 2.45.1
  Kernel Headers 6.1.80 -> 6.1.92
  moreutils 0.68 -> 0.69

  Commits like https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7b0f145802f0c2c785014293d748721678fef824, which should improve the bootstrap situation (#30042). This can somewhat be visualised by comparing the (simplified) dependencies of guix itself, between the two time-machines.

  Master:
  ![master_2](https://github.com/bitcoin/bitcoin/assets/863730/714402a2-345e-43c7-974b-5112d03d44c2)

  PR:
  ![pr](https://github.com/bitcoin/bitcoin/assets/863730/7079a155-e013-4d59-9ea1-21a64d71e2d8)

  Note that in the case of this PR, we are better off, no-longer having to build a number of tex packages, ruby, cairo, graphics libs, openssl 1.x etc.

ACKs for top commit:
  TheCharlatan:
    ACK 2599655

Tree-SHA512: 9c5675a5d563c17744c89c8a392bc7865aa1c9e71a5e3044c23f31e51458dac28c0c238d2055c86dc732df595dae60bcbc8b85266b23b7991c4c770d56f7d23a
@fanquake fanquake force-pushed the remove_force_use_system_clang branch from 8518b92 to 81f1516 Compare June 8, 2024 08:35
@fanquake fanquake force-pushed the remove_force_use_system_clang branch from 81f1516 to b80a846 Compare June 10, 2024 08:08
@fanquake fanquake marked this pull request as ready for review June 10, 2024 12:08
@fanquake fanquake force-pushed the remove_force_use_system_clang branch from b80a846 to 7cbfd7a Compare June 10, 2024 12:22
@fanquake
Copy link
Member Author

Added a commit to rename some references to macho ld64 to lld.

@Sjors
Copy link
Member

Sjors commented Jun 11, 2024

tACK 7cbfd7a

I made a Guix build, as well as a regular cross-compile on Ubuntu 24.04 for Intel macOS, and lightly tested them.

Guix hashes for darwin:

048fa3f381098477b8bea25701547f37a6680b33afb9c61ba88cb4a895aae81d  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/SHA256SUMS.part
d540c4c73b98bef810c85b0b10a6e1a004655fb0c8a4709b8f655c6924afef8b  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.tar.gz
4a0bca38e194b731a105ab791d8dfa6adc314fb0a5a963d49136d5546794919b  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.zip
993c355cdef9a246c9929dc5b9a734ae4781f6353a8c953465c81ae2739809cd  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin.tar.gz
2e58229e7f9f22448fda71b0eda9fb4f084edb1a44093e0dadf9a2ab0ab37026  guix-build-7cbfd7a7ce45/output/dist-archive/bitcoin-7cbfd7a7ce45.tar.gz
1140b74f33745cdd0e703c7740b3adf79b233be9becca3d464b41fc508ebc3f0  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/SHA256SUMS.part
ae7b6911d7c184e9643f8a611db02c5c5028d116937b42935052dea04cc834cf  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.tar.gz
eea2db60805e1a97ffb9cf412380e5df178dd2f7f5e42e1a06feae3185f87a79  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.zip
90bc2c35c76cce5620fcfeb57f108e02f6be6d7d3ac36c88d9a8f991b9637b7b  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin.tar.gz

@DrahtBot DrahtBot requested review from theuni and hebasto June 11, 2024 11:23
@fanquake
Copy link
Member Author

Guix Build (aarch64):

f8487a885a04a4b3c273b2d1ba3ef5e100026f16b03c08e866dbf4cd468d0802  guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/SHA256SUMS.part
b3da7604bc7302213d2864bddbccc54ede6374711377b89b2d17a989d2e9e64d  guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/bitcoin-7cbfd7a7ce45-aarch64-linux-gnu-debug.tar.gz
d2340b2c084faf01a0db8ea0c077ba9f8c51ef23b680396d88001b33d126788b  guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/bitcoin-7cbfd7a7ce45-aarch64-linux-gnu.tar.gz
a79f460ac8b4c893ad1cf4aba17489b0dc7834377166d4e9bc4da78bcd84a430  guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/SHA256SUMS.part
6d94b11fca39489230f00125d156c1ad36a8b8ff9c5fdd6a7d47e31c3c527230  guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/bitcoin-7cbfd7a7ce45-arm-linux-gnueabihf-debug.tar.gz
e5cf9b6bd1a8ad95a49ad523681091ad9938c02f7f995934c413bd8b8c651199  guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/bitcoin-7cbfd7a7ce45-arm-linux-gnueabihf.tar.gz
048fa3f381098477b8bea25701547f37a6680b33afb9c61ba88cb4a895aae81d  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/SHA256SUMS.part
d540c4c73b98bef810c85b0b10a6e1a004655fb0c8a4709b8f655c6924afef8b  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.tar.gz
4a0bca38e194b731a105ab791d8dfa6adc314fb0a5a963d49136d5546794919b  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.zip
993c355cdef9a246c9929dc5b9a734ae4781f6353a8c953465c81ae2739809cd  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin.tar.gz
2e58229e7f9f22448fda71b0eda9fb4f084edb1a44093e0dadf9a2ab0ab37026  guix-build-7cbfd7a7ce45/output/dist-archive/bitcoin-7cbfd7a7ce45.tar.gz
18116e5c3df56d933ce453dd9b475317fe3a9e4452fa420e57cfeb15f6dd0580  guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/SHA256SUMS.part
d23d0c154fa27210e0be3992bd34882db469dde51070b975cb5ee6b365bfc10b  guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/bitcoin-7cbfd7a7ce45-powerpc64-linux-gnu-debug.tar.gz
92d17f3834ee8f8f14cea0ff7e183c8efb1a17355d4fd654ac0a96e01c880aa3  guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/bitcoin-7cbfd7a7ce45-powerpc64-linux-gnu.tar.gz
3bd64df4a6e6fd27ac72c48691e41a5b528446a4052c42caaf6e91f32419d327  guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/SHA256SUMS.part
dd23855a2f58b3334b274630b7c97be99335ff9908e8a6b0c1300a141b92f5a0  guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/bitcoin-7cbfd7a7ce45-riscv64-linux-gnu-debug.tar.gz
a40be60dd17b32b6c8c8560e579bfb02c08e2e8f082a2247b66f957b2780fee9  guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/bitcoin-7cbfd7a7ce45-riscv64-linux-gnu.tar.gz
1140b74f33745cdd0e703c7740b3adf79b233be9becca3d464b41fc508ebc3f0  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/SHA256SUMS.part
ae7b6911d7c184e9643f8a611db02c5c5028d116937b42935052dea04cc834cf  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.tar.gz
eea2db60805e1a97ffb9cf412380e5df178dd2f7f5e42e1a06feae3185f87a79  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.zip
90bc2c35c76cce5620fcfeb57f108e02f6be6d7d3ac36c88d9a8f991b9637b7b  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin.tar.gz
8b758e8b739b80123a132ceba172ba9fd70a476941a1b515629737b9f6020ab9  guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/SHA256SUMS.part
cd50dfb8d19c5ea62b1069655c5154965703fccfa31bad9c7169b2842a5c5048  guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/bitcoin-7cbfd7a7ce45-x86_64-linux-gnu-debug.tar.gz
1496518bd7efe42df95318666a14c97ae019a6d66c526906c61d22dd8769769d  guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/bitcoin-7cbfd7a7ce45-x86_64-linux-gnu.tar.gz
56d85879d391893a174dce042ed7a4d54c4e57645629660952411052e35d793c  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/SHA256SUMS.part
633ae51f73e818edcfd7e6a03d6368abe58540a485908aa17dbd812cc790d55f  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-debug.zip
480576fb02cc43bcaea583b4975e7609c093c53a16c891e1ab1c75801cd9da18  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-setup-unsigned.exe
ff2e48fb53aca49d3d0fcda480e64cf36f4b8250357c1adeec0abba6b66bb524  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-unsigned.tar.gz
42ca2d89cb6703094ea22bd9a869d833f36f42cbef39ca5c3c7b8503bbe15587  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64.zip

Copy link
Member

@theuni theuni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on Ubuntu 22.04 with downloaded llvm 18 binaries in PATH.

ACK 7cbfd7a

Copy link
Contributor

@TheCharlatan TheCharlatan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, ACK 7cbfd7a

Guix builds (aarch64):

f8487a885a04a4b3c273b2d1ba3ef5e100026f16b03c08e866dbf4cd468d0802  guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/SHA256SUMS.part
b3da7604bc7302213d2864bddbccc54ede6374711377b89b2d17a989d2e9e64d  guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/bitcoin-7cbfd7a7ce45-aarch64-linux-gnu-debug.tar.gz
d2340b2c084faf01a0db8ea0c077ba9f8c51ef23b680396d88001b33d126788b  guix-build-7cbfd7a7ce45/output/aarch64-linux-gnu/bitcoin-7cbfd7a7ce45-aarch64-linux-gnu.tar.gz
a79f460ac8b4c893ad1cf4aba17489b0dc7834377166d4e9bc4da78bcd84a430  guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/SHA256SUMS.part
6d94b11fca39489230f00125d156c1ad36a8b8ff9c5fdd6a7d47e31c3c527230  guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/bitcoin-7cbfd7a7ce45-arm-linux-gnueabihf-debug.tar.gz
e5cf9b6bd1a8ad95a49ad523681091ad9938c02f7f995934c413bd8b8c651199  guix-build-7cbfd7a7ce45/output/arm-linux-gnueabihf/bitcoin-7cbfd7a7ce45-arm-linux-gnueabihf.tar.gz
048fa3f381098477b8bea25701547f37a6680b33afb9c61ba88cb4a895aae81d  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/SHA256SUMS.part
d540c4c73b98bef810c85b0b10a6e1a004655fb0c8a4709b8f655c6924afef8b  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.tar.gz
4a0bca38e194b731a105ab791d8dfa6adc314fb0a5a963d49136d5546794919b  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin-unsigned.zip
993c355cdef9a246c9929dc5b9a734ae4781f6353a8c953465c81ae2739809cd  guix-build-7cbfd7a7ce45/output/arm64-apple-darwin/bitcoin-7cbfd7a7ce45-arm64-apple-darwin.tar.gz
2e58229e7f9f22448fda71b0eda9fb4f084edb1a44093e0dadf9a2ab0ab37026  guix-build-7cbfd7a7ce45/output/dist-archive/bitcoin-7cbfd7a7ce45.tar.gz
18116e5c3df56d933ce453dd9b475317fe3a9e4452fa420e57cfeb15f6dd0580  guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/SHA256SUMS.part
d23d0c154fa27210e0be3992bd34882db469dde51070b975cb5ee6b365bfc10b  guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/bitcoin-7cbfd7a7ce45-powerpc64-linux-gnu-debug.tar.gz
92d17f3834ee8f8f14cea0ff7e183c8efb1a17355d4fd654ac0a96e01c880aa3  guix-build-7cbfd7a7ce45/output/powerpc64-linux-gnu/bitcoin-7cbfd7a7ce45-powerpc64-linux-gnu.tar.gz
3bd64df4a6e6fd27ac72c48691e41a5b528446a4052c42caaf6e91f32419d327  guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/SHA256SUMS.part
dd23855a2f58b3334b274630b7c97be99335ff9908e8a6b0c1300a141b92f5a0  guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/bitcoin-7cbfd7a7ce45-riscv64-linux-gnu-debug.tar.gz
a40be60dd17b32b6c8c8560e579bfb02c08e2e8f082a2247b66f957b2780fee9  guix-build-7cbfd7a7ce45/output/riscv64-linux-gnu/bitcoin-7cbfd7a7ce45-riscv64-linux-gnu.tar.gz
1140b74f33745cdd0e703c7740b3adf79b233be9becca3d464b41fc508ebc3f0  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/SHA256SUMS.part
ae7b6911d7c184e9643f8a611db02c5c5028d116937b42935052dea04cc834cf  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.tar.gz
eea2db60805e1a97ffb9cf412380e5df178dd2f7f5e42e1a06feae3185f87a79  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin-unsigned.zip
90bc2c35c76cce5620fcfeb57f108e02f6be6d7d3ac36c88d9a8f991b9637b7b  guix-build-7cbfd7a7ce45/output/x86_64-apple-darwin/bitcoin-7cbfd7a7ce45-x86_64-apple-darwin.tar.gz
8b758e8b739b80123a132ceba172ba9fd70a476941a1b515629737b9f6020ab9  guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/SHA256SUMS.part
cd50dfb8d19c5ea62b1069655c5154965703fccfa31bad9c7169b2842a5c5048  guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/bitcoin-7cbfd7a7ce45-x86_64-linux-gnu-debug.tar.gz
1496518bd7efe42df95318666a14c97ae019a6d66c526906c61d22dd8769769d  guix-build-7cbfd7a7ce45/output/x86_64-linux-gnu/bitcoin-7cbfd7a7ce45-x86_64-linux-gnu.tar.gz
56d85879d391893a174dce042ed7a4d54c4e57645629660952411052e35d793c  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/SHA256SUMS.part
633ae51f73e818edcfd7e6a03d6368abe58540a485908aa17dbd812cc790d55f  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-debug.zip
480576fb02cc43bcaea583b4975e7609c093c53a16c891e1ab1c75801cd9da18  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-setup-unsigned.exe
ff2e48fb53aca49d3d0fcda480e64cf36f4b8250357c1adeec0abba6b66bb524  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64-unsigned.tar.gz
42ca2d89cb6703094ea22bd9a869d833f36f42cbef39ca5c3c7b8503bbe15587  guix-build-7cbfd7a7ce45/output/x86_64-w64-mingw32/bitcoin-7cbfd7a7ce45-win64.zip

@fanquake fanquake merged commit 337f9d4 into bitcoin:master Jun 11, 2024
16 checks passed
@fanquake fanquake deleted the remove_force_use_system_clang branch June 11, 2024 15:42
@hebasto
Copy link
Member

hebasto commented Jun 30, 2024

CI related stuff has to be ported to the CMake staging branch.

@hebasto
Copy link
Member

hebasto commented Jul 14, 2024

Ported to the CMake-based build system in hebasto#264.

Fabcien added a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Dec 2, 2024
Summary:
```
Remove FORCE_USE_SYSTEM_CLANG in favour of always using the system Clang and lld for macOS cross-compilation; rather than downloading precompiled blobs.
```

Backport of [[bitcoin/bitcoin#30201 | core#30201]] and [[bitcoin/bitcoin#30231 | core#30231]].

Depends on D17265.

Note the CI update will be done in another diff. It only affects the cross osx build on CI which is currently paused.

Test Plan: Run the guix osx build.

Reviewers: #bitcoin_abc, PiRK

Reviewed By: #bitcoin_abc, PiRK

Differential Revision: https://reviews.bitcoinabc.org/D17266
roqqit pushed a commit to doged-io/doged that referenced this pull request Dec 19, 2024
Summary:
```
Remove FORCE_USE_SYSTEM_CLANG in favour of always using the system Clang and lld for macOS cross-compilation; rather than downloading precompiled blobs.
```

Backport of [[bitcoin/bitcoin#30201 | core#30201]] and [[bitcoin/bitcoin#30231 | core#30231]].

Depends on D17265.

Note the CI update will be done in another diff. It only affects the cross osx build on CI which is currently paused.

Test Plan: Run the guix osx build.

Reviewers: #bitcoin_abc, PiRK

Reviewed By: #bitcoin_abc, PiRK

Differential Revision: https://reviews.bitcoinabc.org/D17266
kwvg added a commit to kwvg/dash that referenced this pull request Dec 31, 2024
kwvg added a commit to kwvg/dash that referenced this pull request Jan 2, 2025
kwvg added a commit to kwvg/dash that referenced this pull request Jan 3, 2025
PastaPastaPasta added a commit to dashpay/dash that referenced this pull request Jan 7, 2025
, bitcoin#28757, bitcoin#28845, bitcoin#28932, bitcoin#28973, bitcoin#21778, bitcoin#30204, bitcoin#29765, bitcoin#30201, bitcoin#30287, bitcoin#30511 (toolchain backports: part 3)

ee04e1f merge bitcoin#30511: GCC 12 consolidation (Kittywhiskers Van Gogh)
cc5c30e merge bitcoin#30287: rewrite some docs & swap `mmacosx-version-min` for `mmacos-version-min` (Kittywhiskers Van Gogh)
632b8b4 merge bitcoin#30201: remove `FORCE_USE_SYSTEM_CLANG` (Kittywhiskers Van Gogh)
994e7b1 merge bitcoin#29765: Use clang-18 (Kittywhiskers Van Gogh)
7e7d7bd merge bitcoin#30204: consolidate dependency docs (Kittywhiskers Van Gogh)
061a608 merge bitcoin#21778: LLD based macOS toolchain (Kittywhiskers Van Gogh)
ea891d4 fix: use `LDFLAGS` when compiling `gmp`'s `configure` test programs (Kittywhiskers Van Gogh)
c2fe7c6 merge bitcoin#28973: remove `libz-dev` from macOS build deps (Kittywhiskers Van Gogh)
62e671d merge bitcoin#28932: remove `python3-setuptools` from macOS build deps (Kittywhiskers Van Gogh)
5bef1e5 merge bitcoin#28845: remove `PYTHONPATH` from config.site (Kittywhiskers Van Gogh)
c501a0d merge bitcoin#28757: Zip needs to include all files and set time to SOURCE_DATE_EPOCH (Kittywhiskers Van Gogh)
122a61f merge bitcoin#28432: Produce a `.zip` for macOS distribution (Kittywhiskers Van Gogh)
407deab merge bitcoin#25612: default to using GCC tool wrappers for LTO (with GCC) (Kittywhiskers Van Gogh)
f0a0040 fix: ensure that `dsymutil` available even when `FORCE_USE_SYSTEM_CLANG=1` (Kittywhiskers Van Gogh)
416e154 merge bitcoin#24866: No longer need to hack the PATH variable in `config.site` (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Depends on #6385

  * Omissions related to explicitly defining the location of `DSYMUTIL` (see [dash#6384](#6384) for more information) have been remedied and are now included (completing [bitcoin#24866](bitcoin#24866) and [bitcoin#25612](bitcoin#25612) in this pull request), courtesy of a fix that defines `DSYMUTIL` even when `FORCE_USE_SYSTEM_CLANG` is defined ([commit](f0a0040)).
    * This fix is effectively reverted in [bitcoin#21778](bitcoin#21778) as the backport ensures that `DSYMUTIL` is defined in the `FORCE_USE_SYSTEM_CLANG` case and `cctools` is no longer used.
    * It still remains to allow testing and validation of commits pre-[bitcoin#21778](bitcoin#21778).

  * GMP runs its test programs during build configuration sans `LDFLAGS` ([source](https://github.com/gmp-mirror/gmp/blob/141ed4f98a50e4a3c1a95a1758cb599b7499ac39/acinclude.m4#L1755)), which are quite important when using an LLD-based toolchain as we rely on it to take on `ld64`'s duties ([source](https://lld.llvm.org/MachO/index.html)) using `-fuse-ld=lld` ([source](https://lld.llvm.org/#using-lld)). This has been remedied by patching `acinclude.m4` and `configure.ac` to include `LDFLAGS`.

    <details>

    <summary>configure failure:</summary>

     ```
     checking build system type... x86_64-pc-linux-gnu
     checking host system type... x86_64-apple-darwin
     checking for a BSD-compatible install... /home/debian/.guix-profile/bin/install -c
     checking whether build environment is sane... yes
     checking for x86_64-apple-darwin-strip... no
     checking for strip... strip
     configure: WARNING: using cross tools not prefixed with host triplet
     checking for a thread-safe mkdir -p... /home/debian/.guix-profile/bin/mkdir -p
     checking for gawk... gawk
     checking whether make sets $(MAKE)... yes
     checking whether make supports nested variables... yes
     checking whether to enable maintainer-specific portions of Makefiles... no
     checking ABI=64
     checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes
     checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64    -I/dash/depends/x86_64-apple-darwin/include... no
     checking ABI=x32
     checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes
     checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64    -I/dash/depends/x86_64-apple-darwin/include... no
     checking ABI=32
     checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes
     checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64    -I/dash/depends/x86_64-apple-darwin/include... no
     configure: error: could not find a working compiler, see config.log for details
     make: *** [funcs.mk:302: /dash/depends/x86_64-apple-darwin/.gmp_stamp_configured] Error 1
     make: Leaving directory '/dash/depends'
     ```

    </details>

    <details>

    <summary>config.log excerpt:</summary>

     ```
     This file contains any messages produced by compilers while
     running configure, to aid debugging if configure makes a mistake.

     It was created by GNU MP configure 6.3.0, which was
     generated by GNU Autoconf 2.69.  Invocation command line was

     [...]

     ## --------- ##
     ## Platform. ##
     ## --------- ##

     hostname = [...]
     uname -m = x86_64
     uname -r = 6.6.32[...]
     uname -s = Linux
     uname -v = #1 SMP PREEMPT_DYNAMIC Mon Jul  8 16:11:58 UTC 2024

     /usr/bin/uname -p = unknown
     /bin/uname -X     = unknown

     /bin/arch              = unknown
     /usr/bin/arch -k       = unknown
     /usr/convex/getsysinfo = unknown
     /usr/bin/hostinfo      = unknown
     /bin/machine           = unknown
     /usr/bin/oslevel       = unknown
     /bin/universe          = unknown

     PATH: /dash/depends/x86_64-apple-darwin/native/bin
     PATH: /home/debian/.guix-profile/bin
     PATH: /home/debian/.guix-profile/sbin

     ## ----------- ##
     ## Core tests. ##
     ## ----------- ##

     [...]
     configure:5889: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -c conftest.c >&5
     configure:5892: $? = 0
     configure:5895: checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc
     configure:5897: result: yes
     configure:5911: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks 2>&1 | grep xlc >/dev/null
     configure:5914: $? = 1
     configure:5968: checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64   -I/dash/depends/x86_64-apple-darwin/include
     Test compile:
     configure:5982: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64   -I/dash/depends/x86_64-apple-darwin/include conftest.c >&5
     ld: unrecognized option '-dynamic'
     ld: use the --help option for usage information
     clang: error: linker command failed with exit code 1 (use -v to see invocation)
     configure:5985: $? = 1
     failed program was:

     int main () { return 0; }
     configure:7072: result: no
     [...]
     configure:7310: error: could not find a working compiler, see config.log for details
     [...]

     ## ---------------- ##
     ## Cache variables. ##
     ## ---------------- ##

     [...]
     ac_cv_env_LDFLAGS_set=set
     ac_cv_env_LDFLAGS_value='-Wl,-platform_version,macos,11.0,14.0 -Wl,-no_adhoc_codesign -fuse-ld=lld  -L/dash/depends/x86_64-apple-darwin/lib'
     [...]
     LD=''
     LDFLAGS='-Wl,-platform_version,macos,11.0,14.0 -Wl,-no_adhoc_codesign -fuse-ld=lld  -L/dash/depends/x86_64-apple-darwin/lib'
     [...]

     ## ----------- ##
     ## confdefs.h. ##
     ## ----------- ##

     /* confdefs.h */
     #define PACKAGE_NAME "GNU MP"
     #define PACKAGE_TARNAME "gmp"
     #define PACKAGE_VERSION "6.3.0"
     #define PACKAGE_STRING "GNU MP 6.3.0"
     #define PACKAGE_BUGREPORT "gmp-bugs@gmplib.org (see https://gmplib.org/manual/Reporting-Bugs.html)"
     #define PACKAGE_URL "http://www.gnu.org/software/gmp/"
     #define PACKAGE "gmp"
     #define VERSION "6.3.0"
     #define WANT_FFT 1
     #define HAVE_HOST_CPU_x86_64 1

     configure: exit 1
     ```

     **Note:** `lld` itself will never respond like this, as it will exit with a catch-all message requesting you to invoke it with the correct name

     ```
     $ lld -dynamic
     lld is a generic driver.
     Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly) instead
     ```

     Nor is it calling the Unix-specific variant mistakenly

     ```
     $ ld.lld -dynamic
     ld.lld: error: unknown argument '-dynamic', did you mean '-Bdynamic'
     ```

     The error displayed above is consistent with GNU `ld`
     ```
     $ ld -dynamic
     ld: unrecognized option '-dynamic'
     ld: use the --help option for usage information
     ```

    </details>

    * A similar problem is observed for Qt, which was resolved upstream by adding it to `CXXFLAGS` ([source](https://github.com/bitcoin/bitcoin/blob/e8c25e8a35e333e90514945c592557615641553f/depends/patches/qt/mac-qmake.conf#L16)) (but this comes at the downside of an unused argument warnings whenever the compiler doesn't intend to do any linking and is why we didn't just modify `darwin_{CC,CXX}` instead)

      <details>

      <summary>Warning messages when building Qt:</summary>

      ```
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qplatformcursor.cpp
      compiling kernel/qplatformclipboard.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qplatformsurface.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      4 warnings generated.
      compiling kernel/qcursor.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qguivariant.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qshortcutmap.cpp
      compiling kernel/qtouchdevice.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qplatformservices.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      kernel/qplatformcursor.cpp:622:17: warning: variable 'w' set but not used [-Wunused-but-set-variable]
      622 |     int x = -1, w = 0;
          |                 ^
      compiling kernel/qplatformsessionmanager.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      ```

      </details>

  * Clang has been bumped to 18 as [bitcoin#30201](bitcoin#30201) drops `native_llvm` (formerly known as `native_clang`) and mandates the presence of Clang 18 or higher for cross-compilation ([source](https://github.com/bitcoin/bitcoin/blob/7cbfd7a7ce45ac68d6041f42f468862f5c193d8c/depends/README.md?plain=1#L54-L57)).

  ## Breaking Changes

  None expected

  ## Checklist

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK ee04e1f
  UdjinM6:
    utACK ee04e1f

Tree-SHA512: 0b8be036419a53eda5be77c9abdf214d8f5baa088526f9249daba31e650c50f831f54c389845f828f8b7cf50458967f211a6d42d12b8dae8546b265d7f37a968
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants