Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Use clang-18 #29765

Merged
merged 2 commits into from
Apr 1, 2024
Merged

ci: Use clang-18 #29765

merged 2 commits into from
Apr 1, 2024

Conversation

maflcko
Copy link
Member

@maflcko maflcko commented Mar 29, 2024

Use clang-18, which comes with bugfixes and sanitizer upgrades.

@DrahtBot
Copy link
Contributor

DrahtBot commented Mar 29, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK fanquake

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #29165 (build: Bump clang minimum supported version to 15 by maflcko)
  • #28710 (Remove the legacy wallet and BDB dependency by achow101)
  • #28687 (C++20 std::views::reverse by stickies-v)
  • #25972 (build: no-longer disable WARN_CXXFLAGS when CXXFLAGS is set by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@DrahtBot DrahtBot changed the title ci: Use clang-18 ci: Use clang-18 Mar 29, 2024
@DrahtBot DrahtBot added the Tests label Mar 29, 2024
@fanquake
Copy link
Member

I don't think Mac cross should change until depends and guix does?

@maflcko
Copy link
Member Author

maflcko commented Mar 29, 2024

Oh I forgot that the pre-built binary has the distro assumed:

$(package)_file_name=clang+llvm-$($(package)_version)-x86_64-linux-gnu-ubuntu-22.04.tar.xz

Maybe a comment can be added that the two must match.

@fanquake
Copy link
Member

fanquake commented Apr 1, 2024

has the distro assumed:
Maybe a comment can be added that the two must match.

They don't necessarily have to, i.e, the "Ubuntu" bins will build macOS binaries on Fedora, or Alpine, for example. Or are you only talking about the version?

Copy link
Member

@fanquake fanquake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK fa75220

@fanquake fanquake merged commit c7247bd into bitcoin:master Apr 1, 2024
16 checks passed
@maflcko maflcko deleted the 2403-clang-18- branch April 1, 2024 14:52
kwvg added a commit to kwvg/dash that referenced this pull request Nov 27, 2024
kwvg added a commit to kwvg/dash that referenced this pull request Jan 2, 2025
kwvg added a commit to kwvg/dash that referenced this pull request Jan 3, 2025
PastaPastaPasta added a commit to dashpay/dash that referenced this pull request Jan 7, 2025
, bitcoin#28757, bitcoin#28845, bitcoin#28932, bitcoin#28973, bitcoin#21778, bitcoin#30204, bitcoin#29765, bitcoin#30201, bitcoin#30287, bitcoin#30511 (toolchain backports: part 3)

ee04e1f merge bitcoin#30511: GCC 12 consolidation (Kittywhiskers Van Gogh)
cc5c30e merge bitcoin#30287: rewrite some docs & swap `mmacosx-version-min` for `mmacos-version-min` (Kittywhiskers Van Gogh)
632b8b4 merge bitcoin#30201: remove `FORCE_USE_SYSTEM_CLANG` (Kittywhiskers Van Gogh)
994e7b1 merge bitcoin#29765: Use clang-18 (Kittywhiskers Van Gogh)
7e7d7bd merge bitcoin#30204: consolidate dependency docs (Kittywhiskers Van Gogh)
061a608 merge bitcoin#21778: LLD based macOS toolchain (Kittywhiskers Van Gogh)
ea891d4 fix: use `LDFLAGS` when compiling `gmp`'s `configure` test programs (Kittywhiskers Van Gogh)
c2fe7c6 merge bitcoin#28973: remove `libz-dev` from macOS build deps (Kittywhiskers Van Gogh)
62e671d merge bitcoin#28932: remove `python3-setuptools` from macOS build deps (Kittywhiskers Van Gogh)
5bef1e5 merge bitcoin#28845: remove `PYTHONPATH` from config.site (Kittywhiskers Van Gogh)
c501a0d merge bitcoin#28757: Zip needs to include all files and set time to SOURCE_DATE_EPOCH (Kittywhiskers Van Gogh)
122a61f merge bitcoin#28432: Produce a `.zip` for macOS distribution (Kittywhiskers Van Gogh)
407deab merge bitcoin#25612: default to using GCC tool wrappers for LTO (with GCC) (Kittywhiskers Van Gogh)
f0a0040 fix: ensure that `dsymutil` available even when `FORCE_USE_SYSTEM_CLANG=1` (Kittywhiskers Van Gogh)
416e154 merge bitcoin#24866: No longer need to hack the PATH variable in `config.site` (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Depends on #6385

  * Omissions related to explicitly defining the location of `DSYMUTIL` (see [dash#6384](#6384) for more information) have been remedied and are now included (completing [bitcoin#24866](bitcoin#24866) and [bitcoin#25612](bitcoin#25612) in this pull request), courtesy of a fix that defines `DSYMUTIL` even when `FORCE_USE_SYSTEM_CLANG` is defined ([commit](f0a0040)).
    * This fix is effectively reverted in [bitcoin#21778](bitcoin#21778) as the backport ensures that `DSYMUTIL` is defined in the `FORCE_USE_SYSTEM_CLANG` case and `cctools` is no longer used.
    * It still remains to allow testing and validation of commits pre-[bitcoin#21778](bitcoin#21778).

  * GMP runs its test programs during build configuration sans `LDFLAGS` ([source](https://github.com/gmp-mirror/gmp/blob/141ed4f98a50e4a3c1a95a1758cb599b7499ac39/acinclude.m4#L1755)), which are quite important when using an LLD-based toolchain as we rely on it to take on `ld64`'s duties ([source](https://lld.llvm.org/MachO/index.html)) using `-fuse-ld=lld` ([source](https://lld.llvm.org/#using-lld)). This has been remedied by patching `acinclude.m4` and `configure.ac` to include `LDFLAGS`.

    <details>

    <summary>configure failure:</summary>

     ```
     checking build system type... x86_64-pc-linux-gnu
     checking host system type... x86_64-apple-darwin
     checking for a BSD-compatible install... /home/debian/.guix-profile/bin/install -c
     checking whether build environment is sane... yes
     checking for x86_64-apple-darwin-strip... no
     checking for strip... strip
     configure: WARNING: using cross tools not prefixed with host triplet
     checking for a thread-safe mkdir -p... /home/debian/.guix-profile/bin/mkdir -p
     checking for gawk... gawk
     checking whether make sets $(MAKE)... yes
     checking whether make supports nested variables... yes
     checking whether to enable maintainer-specific portions of Makefiles... no
     checking ABI=64
     checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes
     checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64    -I/dash/depends/x86_64-apple-darwin/include... no
     checking ABI=x32
     checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes
     checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64    -I/dash/depends/x86_64-apple-darwin/include... no
     checking ABI=32
     checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc... yes
     checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64    -I/dash/depends/x86_64-apple-darwin/include... no
     configure: error: could not find a working compiler, see config.log for details
     make: *** [funcs.mk:302: /dash/depends/x86_64-apple-darwin/.gmp_stamp_configured] Error 1
     make: Leaving directory '/dash/depends'
     ```

    </details>

    <details>

    <summary>config.log excerpt:</summary>

     ```
     This file contains any messages produced by compilers while
     running configure, to aid debugging if configure makes a mistake.

     It was created by GNU MP configure 6.3.0, which was
     generated by GNU Autoconf 2.69.  Invocation command line was

     [...]

     ## --------- ##
     ## Platform. ##
     ## --------- ##

     hostname = [...]
     uname -m = x86_64
     uname -r = 6.6.32[...]
     uname -s = Linux
     uname -v = #1 SMP PREEMPT_DYNAMIC Mon Jul  8 16:11:58 UTC 2024

     /usr/bin/uname -p = unknown
     /bin/uname -X     = unknown

     /bin/arch              = unknown
     /usr/bin/arch -k       = unknown
     /usr/convex/getsysinfo = unknown
     /usr/bin/hostinfo      = unknown
     /bin/machine           = unknown
     /usr/bin/oslevel       = unknown
     /bin/universe          = unknown

     PATH: /dash/depends/x86_64-apple-darwin/native/bin
     PATH: /home/debian/.guix-profile/bin
     PATH: /home/debian/.guix-profile/sbin

     ## ----------- ##
     ## Core tests. ##
     ## ----------- ##

     [...]
     configure:5889: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -c conftest.c >&5
     configure:5892: $? = 0
     configure:5895: checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks is gcc
     configure:5897: result: yes
     configure:5911: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks 2>&1 | grep xlc >/dev/null
     configure:5914: $? = 1
     configure:5968: checking compiler env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64   -I/dash/depends/x86_64-apple-darwin/include
     Test compile:
     configure:5982: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/debian/.guix-profile/bin/clang --target=x86_64-apple-darwin -isysroot/dash/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers -nostdlibinc -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -pipe -std=c11 -mmacos-version-min=11.0 -mlinker-version=711 -O2 -march=x86-64   -I/dash/depends/x86_64-apple-darwin/include conftest.c >&5
     ld: unrecognized option '-dynamic'
     ld: use the --help option for usage information
     clang: error: linker command failed with exit code 1 (use -v to see invocation)
     configure:5985: $? = 1
     failed program was:

     int main () { return 0; }
     configure:7072: result: no
     [...]
     configure:7310: error: could not find a working compiler, see config.log for details
     [...]

     ## ---------------- ##
     ## Cache variables. ##
     ## ---------------- ##

     [...]
     ac_cv_env_LDFLAGS_set=set
     ac_cv_env_LDFLAGS_value='-Wl,-platform_version,macos,11.0,14.0 -Wl,-no_adhoc_codesign -fuse-ld=lld  -L/dash/depends/x86_64-apple-darwin/lib'
     [...]
     LD=''
     LDFLAGS='-Wl,-platform_version,macos,11.0,14.0 -Wl,-no_adhoc_codesign -fuse-ld=lld  -L/dash/depends/x86_64-apple-darwin/lib'
     [...]

     ## ----------- ##
     ## confdefs.h. ##
     ## ----------- ##

     /* confdefs.h */
     #define PACKAGE_NAME "GNU MP"
     #define PACKAGE_TARNAME "gmp"
     #define PACKAGE_VERSION "6.3.0"
     #define PACKAGE_STRING "GNU MP 6.3.0"
     #define PACKAGE_BUGREPORT "gmp-bugs@gmplib.org (see https://gmplib.org/manual/Reporting-Bugs.html)"
     #define PACKAGE_URL "http://www.gnu.org/software/gmp/"
     #define PACKAGE "gmp"
     #define VERSION "6.3.0"
     #define WANT_FFT 1
     #define HAVE_HOST_CPU_x86_64 1

     configure: exit 1
     ```

     **Note:** `lld` itself will never respond like this, as it will exit with a catch-all message requesting you to invoke it with the correct name

     ```
     $ lld -dynamic
     lld is a generic driver.
     Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly) instead
     ```

     Nor is it calling the Unix-specific variant mistakenly

     ```
     $ ld.lld -dynamic
     ld.lld: error: unknown argument '-dynamic', did you mean '-Bdynamic'
     ```

     The error displayed above is consistent with GNU `ld`
     ```
     $ ld -dynamic
     ld: unrecognized option '-dynamic'
     ld: use the --help option for usage information
     ```

    </details>

    * A similar problem is observed for Qt, which was resolved upstream by adding it to `CXXFLAGS` ([source](https://github.com/bitcoin/bitcoin/blob/e8c25e8a35e333e90514945c592557615641553f/depends/patches/qt/mac-qmake.conf#L16)) (but this comes at the downside of an unused argument warnings whenever the compiler doesn't intend to do any linking and is why we didn't just modify `darwin_{CC,CXX}` instead)

      <details>

      <summary>Warning messages when building Qt:</summary>

      ```
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qplatformcursor.cpp
      compiling kernel/qplatformclipboard.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qplatformsurface.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      4 warnings generated.
      compiling kernel/qcursor.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qguivariant.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qshortcutmap.cpp
      compiling kernel/qtouchdevice.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      compiling kernel/qplatformservices.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      kernel/qplatformcursor.cpp:622:17: warning: variable 'w' set but not used [-Wunused-but-set-variable]
      622 |     int x = -1, w = 0;
          |                 ^
      compiling kernel/qplatformsessionmanager.cpp
      clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
      ```

      </details>

  * Clang has been bumped to 18 as [bitcoin#30201](bitcoin#30201) drops `native_llvm` (formerly known as `native_clang`) and mandates the presence of Clang 18 or higher for cross-compilation ([source](https://github.com/bitcoin/bitcoin/blob/7cbfd7a7ce45ac68d6041f42f468862f5c193d8c/depends/README.md?plain=1#L54-L57)).

  ## Breaking Changes

  None expected

  ## Checklist

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK ee04e1f
  UdjinM6:
    utACK ee04e1f

Tree-SHA512: 0b8be036419a53eda5be77c9abdf214d8f5baa088526f9249daba31e650c50f831f54c389845f828f8b7cf50458967f211a6d42d12b8dae8546b265d7f37a968
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants