build: Support x86_64 <-> arm64 cross-compiling for macOS #22506
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fanquake
reviewed
Jul 21, 2021
depends/packages/qt.mk
Outdated
| @@ -125,6 +125,11 @@ $(package)_config_opts_darwin += -device-option MAC_SDK_VERSION=$(OSX_SDK_VERSIO | |||
| $(package)_config_opts_darwin += -device-option CROSS_COMPILE="$(host)-" | |||
| $(package)_config_opts_darwin += -device-option MAC_TARGET=$(host) | |||
| $(package)_config_opts_darwin += -device-option XCODE_VERSION=$(XCODE_VERSION) | |||
| else | |||
| ifneq ($(host_arch),$(build_arch)) | |||
There was a problem hiding this comment.
Haven't looked at this other than to say, if this is explicitly fixing the NO_QT=1 build, why is it touching Qt code.
There was a problem hiding this comment.
The reason to include this change here is to do not change build system behavior while building without NO_QT=1:
- on Intel-based macOS 11.4
- on M1-based macOS with installed Rosetta 2
There was a problem hiding this comment.
the "darwin" comes out of the blue here, it's not in a MacOS specific block, ($(host_arch),$(build_arch)) is not specifically related to it and might arise on different systems as well when cross-compiling
There was a problem hiding this comment.
the "darwin" comes out of the blue here...
These lines are placed within the else branch of the ifneq ($(build_os),darwin) statement.
|
Tested 3bf2fa5 macOS 11.4 arm64
|
|
Rebased 3bf2fa5 -> b419da7 (pr22506.01 -> pr22506.02) due to the conflict with #22526. |
|
Tested your in scope commits together -> |
|
Rebased b419da7 -> aacb8fe (pr22506.02 -> pr22506.03) due to the conflict with #22448. Also the PR description has been updated. |
Guix builds: |
|
GUIX hashes, mine match @hebasto : |
|
I think this cross-compilation configuration should be supported, but I'm not sure the changes here are correct. I wonder if it can be handled in a more general way as well, that doesn't need to special-case Darwin inside package builds. |
This PR just adds the |
|
Please un-draft and rebase this, as well as update the PR description. |
|
Rebased aacb8fe -> 46cf787 (pr22506.03 -> pr22506.04) on top of the #23583. Also the PR description has been updated.
Done 🐅 |
I've made more research, and figured out that a general approach breaks cross compiling on linux for macos due to our custom |
fanquake
reviewed
Nov 25, 2021
| @@ -10,8 +10,10 @@ build_darwin_SHA256SUM=shasum -a 256 | |||
| build_darwin_DOWNLOAD=curl --location --fail --connect-timeout $(DOWNLOAD_CONNECT_TIMEOUT) --retry $(DOWNLOAD_RETRIES) -o | |||
|
|
|||
| #darwin host on darwin builder. overrides darwin host preferences. | |||
| darwin_CC=$(shell xcrun -f clang) -mmacosx-version-min=$(OSX_MIN_VERSION) -isysroot$(shell xcrun --show-sdk-path) | |||
There was a problem hiding this comment.
Can you maintain the surrounding style until we eitherr commit to a new one, and/or change things wholesale. Otherwise depends is becoming a bit of a mis-mash, and potentially just more confusing to those who aren't familiar with it.
depends/packages/qt.mk
Outdated
| @@ -126,6 +126,15 @@ $(package)_config_opts_darwin += -device-option MAC_SDK_VERSION=$(OSX_SDK_VERSIO | |||
| $(package)_config_opts_darwin += -device-option CROSS_COMPILE="$(host)-" | |||
| $(package)_config_opts_darwin += -device-option MAC_TARGET=$(host) | |||
| $(package)_config_opts_darwin += -device-option XCODE_VERSION=$(XCODE_VERSION) | |||
| else | |||
| # Qt must be able to build its own tools for the build platform. | |||
There was a problem hiding this comment.
I'm having trouble parsing this comment. Can you describe the actual problem? What about the custom mkspec is the issue?
Guix builds: |
This change fixes cross-compiling on M1 Apple Silicon systems.
|
Closing in favor of #23603. |
fanquake
added a commit
to bitcoin-core/gui
that referenced
this pull request
Dec 9, 2021
…ing for macOS 19f3896 build: Fix x86_64 <-> arm64 cross-compiling for macOS (Hennadii Stepanov) Pull request description: Currently, on master (111c3e0), dependencies are built for the build system architecture, not the provided host. On Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: x86_64 ``` On M1-based macOS Monterey 12.0.1 (21A559) the `boost` package building fails with multiple errors like that: ``` % make -C depends boost HOST=x86_64-apple-darwin19 ... error: option 'cf-protection=return' cannot be specified on this target error: option 'cf-protection=branch' cannot be specified on this target 2 errors generated. ``` This PR allows to cross-compile as follows: - on Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: arm64 % CONFIG_SITE=$PWD/depends/arm64-apple-darwin20/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: arm64 ``` - on M1-based macOS Monterey 12.0.1 (21A559): ``` % make -C depends HOST=x86_64-apple-darwin19 % CONFIG_SITE=$PWD/depends/x86_64-apple-darwin19/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: x86_64 ``` No behavior change for other builder-host pairs. This is an alternative to bitcoin/bitcoin#22506. ACKs for top commit: fanquake: ACK 19f3896 - this is definitely cleaner than the approach in #22506. I've tested the x86_64 -> arm64 and arm64 -> x86_64 cross-compiles. Going to go-ahead and merge this given it's scoped to building for darwin targets, on darwin. Not sure if this is something we'll actually see used very often. Tree-SHA512: 67a7fed1db79bc43d41aabb5a676f90417a879e5dcffec1384c288b56ee32ba06ae3819abdc7aca85b81af6e782bcfff5c83a67a455560249fa095a807fc1187
sidhujag
pushed a commit
to syscoin/syscoin
that referenced
this pull request
Dec 10, 2021
…macOS 19f3896 build: Fix x86_64 <-> arm64 cross-compiling for macOS (Hennadii Stepanov) Pull request description: Currently, on master (111c3e0), dependencies are built for the build system architecture, not the provided host. On Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: x86_64 ``` On M1-based macOS Monterey 12.0.1 (21A559) the `boost` package building fails with multiple errors like that: ``` % make -C depends boost HOST=x86_64-apple-darwin19 ... error: option 'cf-protection=return' cannot be specified on this target error: option 'cf-protection=branch' cannot be specified on this target 2 errors generated. ``` This PR allows to cross-compile as follows: - on Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: arm64 % CONFIG_SITE=$PWD/depends/arm64-apple-darwin20/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: arm64 ``` - on M1-based macOS Monterey 12.0.1 (21A559): ``` % make -C depends HOST=x86_64-apple-darwin19 % CONFIG_SITE=$PWD/depends/x86_64-apple-darwin19/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: x86_64 ``` No behavior change for other builder-host pairs. This is an alternative to bitcoin#22506. ACKs for top commit: fanquake: ACK 19f3896 - this is definitely cleaner than the approach in bitcoin#22506. I've tested the x86_64 -> arm64 and arm64 -> x86_64 cross-compiles. Going to go-ahead and merge this given it's scoped to building for darwin targets, on darwin. Not sure if this is something we'll actually see used very often. Tree-SHA512: 67a7fed1db79bc43d41aabb5a676f90417a879e5dcffec1384c288b56ee32ba06ae3819abdc7aca85b81af6e782bcfff5c83a67a455560249fa095a807fc1187
RandyMcMillan
pushed a commit
to RandyMcMillan/mempool-tab
that referenced
this pull request
Dec 23, 2021
…ing for macOS ad46528 build: Fix x86_64 <-> arm64 cross-compiling for macOS (Hennadii Stepanov) Pull request description: Currently, on master (e475dd3), dependencies are built for the build system architecture, not the provided host. On Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: x86_64 ``` On M1-based macOS Monterey 12.0.1 (21A559) the `boost` package building fails with multiple errors like that: ``` % make -C depends boost HOST=x86_64-apple-darwin19 ... error: option 'cf-protection=return' cannot be specified on this target error: option 'cf-protection=branch' cannot be specified on this target 2 errors generated. ``` This PR allows to cross-compile as follows: - on Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: arm64 % CONFIG_SITE=$PWD/depends/arm64-apple-darwin20/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: arm64 ``` - on M1-based macOS Monterey 12.0.1 (21A559): ``` % make -C depends HOST=x86_64-apple-darwin19 % CONFIG_SITE=$PWD/depends/x86_64-apple-darwin19/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: x86_64 ``` No behavior change for other builder-host pairs. This is an alternative to bitcoin/bitcoin#22506. ACKs for top commit: fanquake: ACK ad46528 - this is definitely cleaner than the approach in #22506. I've tested the x86_64 -> arm64 and arm64 -> x86_64 cross-compiles. Going to go-ahead and merge this given it's scoped to building for darwin targets, on darwin. Not sure if this is something we'll actually see used very often. Tree-SHA512: 67a7fed1db79bc43d41aabb5a676f90417a879e5dcffec1384c288b56ee32ba06ae3819abdc7aca85b81af6e782bcfff5c83a67a455560249fa095a807fc1187
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Apr 7, 2022
…macOS 19f3896 build: Fix x86_64 <-> arm64 cross-compiling for macOS (Hennadii Stepanov) Pull request description: Currently, on master (111c3e0), dependencies are built for the build system architecture, not the provided host. On Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: x86_64 ``` On M1-based macOS Monterey 12.0.1 (21A559) the `boost` package building fails with multiple errors like that: ``` % make -C depends boost HOST=x86_64-apple-darwin19 ... error: option 'cf-protection=return' cannot be specified on this target error: option 'cf-protection=branch' cannot be specified on this target 2 errors generated. ``` This PR allows to cross-compile as follows: - on Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: arm64 % CONFIG_SITE=$PWD/depends/arm64-apple-darwin20/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: arm64 ``` - on M1-based macOS Monterey 12.0.1 (21A559): ``` % make -C depends HOST=x86_64-apple-darwin19 % CONFIG_SITE=$PWD/depends/x86_64-apple-darwin19/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: x86_64 ``` No behavior change for other builder-host pairs. This is an alternative to bitcoin#22506. ACKs for top commit: fanquake: ACK 19f3896 - this is definitely cleaner than the approach in bitcoin#22506. I've tested the x86_64 -> arm64 and arm64 -> x86_64 cross-compiles. Going to go-ahead and merge this given it's scoped to building for darwin targets, on darwin. Not sure if this is something we'll actually see used very often. Tree-SHA512: 67a7fed1db79bc43d41aabb5a676f90417a879e5dcffec1384c288b56ee32ba06ae3819abdc7aca85b81af6e782bcfff5c83a67a455560249fa095a807fc1187
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Apr 7, 2022
…macOS 19f3896 build: Fix x86_64 <-> arm64 cross-compiling for macOS (Hennadii Stepanov) Pull request description: Currently, on master (111c3e0), dependencies are built for the build system architecture, not the provided host. On Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: x86_64 ``` On M1-based macOS Monterey 12.0.1 (21A559) the `boost` package building fails with multiple errors like that: ``` % make -C depends boost HOST=x86_64-apple-darwin19 ... error: option 'cf-protection=return' cannot be specified on this target error: option 'cf-protection=branch' cannot be specified on this target 2 errors generated. ``` This PR allows to cross-compile as follows: - on Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: arm64 % CONFIG_SITE=$PWD/depends/arm64-apple-darwin20/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: arm64 ``` - on M1-based macOS Monterey 12.0.1 (21A559): ``` % make -C depends HOST=x86_64-apple-darwin19 % CONFIG_SITE=$PWD/depends/x86_64-apple-darwin19/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: x86_64 ``` No behavior change for other builder-host pairs. This is an alternative to bitcoin#22506. ACKs for top commit: fanquake: ACK 19f3896 - this is definitely cleaner than the approach in bitcoin#22506. I've tested the x86_64 -> arm64 and arm64 -> x86_64 cross-compiles. Going to go-ahead and merge this given it's scoped to building for darwin targets, on darwin. Not sure if this is something we'll actually see used very often. Tree-SHA512: 67a7fed1db79bc43d41aabb5a676f90417a879e5dcffec1384c288b56ee32ba06ae3819abdc7aca85b81af6e782bcfff5c83a67a455560249fa095a807fc1187
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Apr 7, 2022
…macOS 19f3896 build: Fix x86_64 <-> arm64 cross-compiling for macOS (Hennadii Stepanov) Pull request description: Currently, on master (111c3e0), dependencies are built for the build system architecture, not the provided host. On Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: x86_64 ``` On M1-based macOS Monterey 12.0.1 (21A559) the `boost` package building fails with multiple errors like that: ``` % make -C depends boost HOST=x86_64-apple-darwin19 ... error: option 'cf-protection=return' cannot be specified on this target error: option 'cf-protection=branch' cannot be specified on this target 2 errors generated. ``` This PR allows to cross-compile as follows: - on Intel-based macOS Big Sur 11.6.1 (20G224): ``` % make -C depends HOST=arm64-apple-darwin20 % lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: arm64 % CONFIG_SITE=$PWD/depends/arm64-apple-darwin20/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: arm64 ``` - on M1-based macOS Monterey 12.0.1 (21A559): ``` % make -C depends HOST=x86_64-apple-darwin19 % CONFIG_SITE=$PWD/depends/x86_64-apple-darwin19/share/config.site ./configure % make % lipo -info src/qt/bitcoin-qt Non-fat file: src/qt/bitcoin-qt is architecture: x86_64 ``` No behavior change for other builder-host pairs. This is an alternative to bitcoin#22506. ACKs for top commit: fanquake: ACK 19f3896 - this is definitely cleaner than the approach in bitcoin#22506. I've tested the x86_64 -> arm64 and arm64 -> x86_64 cross-compiles. Going to go-ahead and merge this given it's scoped to building for darwin targets, on darwin. Not sure if this is something we'll actually see used very often. Tree-SHA512: 67a7fed1db79bc43d41aabb5a676f90417a879e5dcffec1384c288b56ee32ba06ae3819abdc7aca85b81af6e782bcfff5c83a67a455560249fa095a807fc1187
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, on master (681b25e), dependencies are built for the build system architecture, not the provided host.
On Intel-based macOS Big Sur 11.6.1 (20G224):
On M1-based macOS Monterey 12.0.1 (21A559) the
boostpackage building fails with multiple errors like that:This PR allows to cross-compile as follows:
No behavior change for other builder-host pairs.