-
Notifications
You must be signed in to change notification settings - Fork 36.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Use clang-18 #29765
ci: Use clang-18 #29765
Conversation
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code CoverageFor detailed information about the code coverage, see the test coverage report. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
I don't think Mac cross should change until depends and guix does? |
Oh I forgot that the pre-built binary has the distro assumed:
Maybe a comment can be added that the two must match. |
They don't necessarily have to, i.e, the "Ubuntu" bins will build macOS binaries on Fedora, or Alpine, for example. Or are you only talking about the version? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK fa75220
, bitcoin#28757, bitcoin#28845, bitcoin#28932, bitcoin#28973, bitcoin#21778, bitcoin#30204, bitcoin#29765, bitcoin#30201, bitcoin#30287, bitcoin#30511 (toolchain backports: part 3) ee04e1f merge bitcoin#30511: GCC 12 consolidation (Kittywhiskers Van Gogh) cc5c30e merge bitcoin#30287: rewrite some docs & swap `mmacosx-version-min` for `mmacos-version-min` (Kittywhiskers Van Gogh) 632b8b4 merge bitcoin#30201: remove `FORCE_USE_SYSTEM_CLANG` (Kittywhiskers Van Gogh) 994e7b1 merge bitcoin#29765: Use clang-18 (Kittywhiskers Van Gogh) 7e7d7bd merge bitcoin#30204: consolidate dependency docs (Kittywhiskers Van Gogh) 061a608 merge bitcoin#21778: LLD based macOS toolchain (Kittywhiskers Van Gogh) ea891d4 fix: use `LDFLAGS` when compiling `gmp`'s `configure` test programs (Kittywhiskers Van Gogh) c2fe7c6 merge bitcoin#28973: remove `libz-dev` from macOS build deps (Kittywhiskers Van Gogh) 62e671d merge bitcoin#28932: remove `python3-setuptools` from macOS build deps (Kittywhiskers Van Gogh) 5bef1e5 merge bitcoin#28845: remove `PYTHONPATH` from config.site (Kittywhiskers Van Gogh) c501a0d merge bitcoin#28757: Zip needs to include all files and set time to SOURCE_DATE_EPOCH (Kittywhiskers Van Gogh) 122a61f merge bitcoin#28432: Produce a `.zip` for macOS distribution (Kittywhiskers Van Gogh) 407deab merge bitcoin#25612: default to using GCC tool wrappers for LTO (with GCC) (Kittywhiskers Van Gogh) f0a0040 fix: ensure that `dsymutil` available even when `FORCE_USE_SYSTEM_CLANG=1` (Kittywhiskers Van Gogh) 416e154 merge bitcoin#24866: No longer need to hack the PATH variable in `config.site` (Kittywhiskers Van Gogh) Pull request description: ## Additional Information * Depends on #6385 * Omissions related to explicitly defining the location of `DSYMUTIL` (see [dash#6384](#6384) for more information) have been remedied and are now included (completing [bitcoin#24866](bitcoin#24866) and [bitcoin#25612](bitcoin#25612) in this pull request), courtesy of a fix that defines `DSYMUTIL` even when `FORCE_USE_SYSTEM_CLANG` is defined ([commit](f0a0040)). * This fix is effectively reverted in [bitcoin#21778](bitcoin#21778) as the backport ensures that `DSYMUTIL` is defined in the `FORCE_USE_SYSTEM_CLANG` case and `cctools` is no longer used. * It still remains to allow testing and validation of commits pre-[bitcoin#21778](bitcoin#21778). * GMP runs its test programs during build configuration sans `LDFLAGS` ([source](https://github.com/gmp-mirror/gmp/blob/141ed4f98a50e4a3c1a95a1758cb599b7499ac39/acinclude.m4#L1755)), which are quite important when using an LLD-based toolchain as we rely on it to take on `ld64`'s duties ([source](https://lld.llvm.org/MachO/index.html)) using `-fuse-ld=lld` ([source](https://lld.llvm.org/#using-lld)). This has been remedied by patching `acinclude.m4` and `configure.ac` to include `LDFLAGS`. <details> <summary>configure failure:</summary> ``` checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-apple-darwin checking for a BSD-compatible install... /home/debian/.guix-profile/bin/install -c checking whether build environment is sane... yes checking for x86_64-apple-darwin-strip... no checking for strip... strip configure: WARNING: using cross tools not prefixed with host triplet checking for a thread-safe mkdir -p... /home/debian/.guix-profile/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether to enable maintainer-specific portions of Makefiles... no checking ABI=64 checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include... no checking ABI=x32 checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include... no checking ABI=32 checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include... no configure: error: could not find a working compiler, see config.log for details make: *** [funcs.mk:302: /dash/depends/x86_64-apple-darwin/.gmp_stamp_configured] Error 1 make: Leaving directory '/dash/depends' ``` </details> <details> <summary>config.log excerpt:</summary> ``` This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by GNU MP configure 6.3.0, which was generated by GNU Autoconf 2.69. Invocation command line was [...] ## --------- ## ## Platform. ## ## --------- ## hostname = [...] uname -m = x86_64 uname -r = 6.6.32[...] uname -s = Linux uname -v = #1 SMP PREEMPT_DYNAMIC Mon Jul 8 16:11:58 UTC 2024 /usr/bin/uname -p = unknown /bin/uname -X = unknown /bin/arch = unknown /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown /usr/bin/hostinfo = unknown /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown PATH: /dash/depends/x86_64-apple-darwin/native/bin PATH: /home/debian/.guix-profile/bin PATH: /home/debian/.guix-profile/sbin ## ----------- ## ## Core tests. ## ## ----------- ## [...] configure:5889: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -c conftest.c >&5 configure:5892: $? = 0 configure:5895: checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc configure:5897: result: yes configure:5911: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks 2>&1 | grep xlc >/dev/null configure:5914: $? = 1 configure:5968: checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include Test compile: configure:5982: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64 -I/dash/depends/x86_64-apple-darwin/include conftest.c >&5 ld: unrecognized option '-dynamic' ld: use the --help option for usage information clang: error: linker command failed with exit code 1 (use -v to see invocation) configure:5985: $? = 1 failed program was: int main () { return 0; } configure:7072: result: no [...] configure:7310: error: could not find a working compiler, see config.log for details [...] ## ---------------- ## ## Cache variables. ## ## ---------------- ## [...] ac_cv_env_LDFLAGS_set=set ac_cv_env_LDFLAGS_value='-Wl,-platform_version,macos,11.0,14.0 -Wl,-no_adhoc_codesign -fuse-ld=lld -L/dash/depends/x86_64-apple-darwin/lib' [...] LD='' LDFLAGS='-Wl,-platform_version,macos,11.0,14.0 -Wl,-no_adhoc_codesign -fuse-ld=lld -L/dash/depends/x86_64-apple-darwin/lib' [...] ## ----------- ## ## confdefs.h. ## ## ----------- ## /* confdefs.h */ #define PACKAGE_NAME "GNU MP" #define PACKAGE_TARNAME "gmp" #define PACKAGE_VERSION "6.3.0" #define PACKAGE_STRING "GNU MP 6.3.0" #define PACKAGE_BUGREPORT "gmp-bugs@gmplib.org (see https://gmplib.org/manual/Reporting-Bugs.html)" #define PACKAGE_URL "http://www.gnu.org/software/gmp/" #define PACKAGE "gmp" #define VERSION "6.3.0" #define WANT_FFT 1 #define HAVE_HOST_CPU_x86_64 1 configure: exit 1 ``` **Note:** `lld` itself will never respond like this, as it will exit with a catch-all message requesting you to invoke it with the correct name ``` $ lld -dynamic lld is a generic driver. Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly) instead ``` Nor is it calling the Unix-specific variant mistakenly ``` $ ld.lld -dynamic ld.lld: error: unknown argument '-dynamic', did you mean '-Bdynamic' ``` The error displayed above is consistent with GNU `ld` ``` $ ld -dynamic ld: unrecognized option '-dynamic' ld: use the --help option for usage information ``` </details> * A similar problem is observed for Qt, which was resolved upstream by adding it to `CXXFLAGS` ([source](https://github.com/bitcoin/bitcoin/blob/e8c25e8a35e333e90514945c592557615641553f/depends/patches/qt/mac-qmake.conf#L16)) (but this comes at the downside of an unused argument warnings whenever the compiler doesn't intend to do any linking and is why we didn't just modify `darwin_{CC,CXX}` instead) <details> <summary>Warning messages when building Qt:</summary> ``` clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qplatformcursor.cpp compiling kernel/qplatformclipboard.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qplatformsurface.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] 4 warnings generated. compiling kernel/qcursor.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qguivariant.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qshortcutmap.cpp compiling kernel/qtouchdevice.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] compiling kernel/qplatformservices.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] kernel/qplatformcursor.cpp:622:17: warning: variable 'w' set but not used [-Wunused-but-set-variable] 622 | int x = -1, w = 0; | ^ compiling kernel/qplatformsessionmanager.cpp clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] ``` </details> * Clang has been bumped to 18 as [bitcoin#30201](bitcoin#30201) drops `native_llvm` (formerly known as `native_clang`) and mandates the presence of Clang 18 or higher for cross-compilation ([source](https://github.com/bitcoin/bitcoin/blob/7cbfd7a7ce45ac68d6041f42f468862f5c193d8c/depends/README.md?plain=1#L54-L57)). ## Breaking Changes None expected ## Checklist - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: PastaPastaPasta: utACK ee04e1f UdjinM6: utACK ee04e1f Tree-SHA512: 0b8be036419a53eda5be77c9abdf214d8f5baa088526f9249daba31e650c50f831f54c389845f828f8b7cf50458967f211a6d42d12b8dae8546b265d7f37a968
Use clang-18, which comes with bugfixes and sanitizer upgrades.