Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build openssl without assembly on arm #57124

Merged
merged 2 commits into from
Jul 2, 2020
Merged

Conversation

indirect
Copy link
Contributor

  • Have you followed the guidelines for contributing?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing brew install <formula>)?

Hello! Turns out openssl tries to shove in some x86 assembly when building on arm, and needs to be told to stop. I've tested this patch using --build-from-source on both x86 and arm64 machines.

@indirect indirect force-pushed the master branch 2 times, most recently from b5a52ab to 05098ef Compare June 30, 2020 05:20
@iMichka
Copy link
Member

iMichka commented Jun 30, 2020

Hi.

  • Can you provide the error message / logs of the build failure you were seeing?
  • Has a patch been submitted upstream?

@indirect
Copy link
Contributor Author

@iMichka sure:

==> perl ./Configure --prefix=/usr/local/Cellar/openssl@1.1/1.1.1g --openssldir=/usr/local/etc/openssl@1.1 no-ssl3 no-ssl3-meth
==> make
Last 15 lines from /Users/andre/Library/Logs/Homebrew/openssl@1.1/02.make:
         ^
crypto/aes/aesni-mb-x86_64.s:533:10: error: unexpected token in argument list
 movq -16(%rax),%rbp

That's the error if you don't include no-asm.

I don't know what you mean by an upstream patch, because homebrew explicitly tells openssl to optimize for x86_64 in the other config flags. See the line above my patch, where the flag is darwin64-x86_64-cc. This is simply configuring Openssl to not include x86 assembly when we aren't on x86.

@indirect
Copy link
Contributor Author

Oh. Rereading my original comment now, I now see why you asked about an upstream patch! Sorry about the misleading statement, this seems to purely be an issue with the way Homebrew is configuring OpenSSL and hardcoding x86.

@Bo98
Copy link
Member

Bo98 commented Jun 30, 2020

This makes sense.

Out of interest, is the output binary still x86_64 compiled? It doesn't really matter just now because Rosetta but is worth noting.

@indirect
Copy link
Contributor Author

@Bo98 looks like it's purely arm64 to me:

$ lipo -info /usr/local/Cellar/openssl@1.1/1.1.1g/lib/libssl.dylib
Non-fat file: /usr/local/Cellar/openssl@1.1/1.1.1g/lib/libssl.dylib is architecture: arm64

@Bo98
Copy link
Member

Bo98 commented Jun 30, 2020

Cool - thanks for checking! I was wondering since darwin64-x86_64-cc passes -arch x86_64, but I think the Homebrew superenv does some magic to filter that out

@SMillerDev
Copy link
Member

Could you verify that the other parameters here openssl/openssl#12254 are also passed?

@indirect
Copy link
Contributor Author

@SMillerDev I'm having trouble seeing what you mean. At least some of those parameters directly contradict existing Homebrew config. For example, the config you linked enables zlib, while the Homebrew formula already explicitly disables zlib. Which parameters do you think should be added, exactly, and why?

@chenrui333
Copy link
Member

chenrui333 commented Jun 30, 2020

Since there is some Mojave runner setup issue and fixed by @Bo98, going to cancel the two ongoing builds. (will trigger a new run after clearing the queued PRs)

Run log, https://github.com/Homebrew/homebrew-core/actions/runs/152330222

@SMillerDev
Copy link
Member

Which parameters do you think should be added, exactly, and why?

I mostly just wanted to make you aware of the information there and if any other flags are needed based on that information you can add them as well. I don't know enough about homebrews openssl choices to pick specific ones for you.

@indirect
Copy link
Contributor Author

indirect commented Jul 1, 2020

@SMillerDev got it, thanks! I did see that ticket, and I figure there will likely eventually be an official platform for homebrew to use when building. Until then, I believe this is the only needed flag—I tested this patch on both x86 and arm macs and it produced working native openssl on each.

Formula/openssl@1.1.rb Show resolved Hide resolved
@claui
Copy link
Contributor

claui commented Jul 1, 2020

Can confirm that this works just fine on ARM.

I was able to brew install -s wget on ARM using this fix, and use it to download a TLS-served page without any errors.
I did a similar check with python@3.8 using urllib.request.urlopen.

Co-authored-by: Claudia Pellegrino <claui@users.noreply.github.com>
@indirect indirect requested a review from claui July 1, 2020 19:47
@SMillerDev
Copy link
Member

@lionellloh @MikeMcQuaid something is off about the linter complaining about non-existing methods here.

@Bo98
Copy link
Member

Bo98 commented Jul 1, 2020

Rerunning the CI now since it's been fixed.

@lionellloh
Copy link

@lionellloh @MikeMcQuaid something is off about the linter complaining about non-existing methods here.

Hi @SMillerDev, do you have more information how I may replicate this?

@MikeMcQuaid
Copy link
Member

@lionellloh Not your fault and all addressed now. Was an issue with CI running master but your code not being in a tagged/stable release yet.

@claui
Copy link
Contributor

claui commented Jul 2, 2020

@MikeMcQuaid Is another CI run warranted here? Or shall we rather fast-track this PR?
I double-checked that the change works just fine. It also obviously has zero impact on the Intel universe.

@SMillerDev
Copy link
Member

Seeing how it's not a revision bump I'm fine with merging

@MikeMcQuaid MikeMcQuaid merged commit f329804 into Homebrew:master Jul 2, 2020
@MikeMcQuaid
Copy link
Member

Merged! No need to run again through CI for something that's going to 1) take ages and 2) not actually test the change.

@claui
Copy link
Contributor

claui commented Jul 2, 2020

Thanks for your contribution @indirect! 🍻

@claui claui added the 11 Big Sur is specifically affected label Jul 7, 2020
@mlindner

This comment was marked as abuse.

@iMichka
Copy link
Member

iMichka commented Nov 30, 2020

We are aware of that. This was accepted to help our maintainers to get things going for ARM native builds. What should have done? Stay stuck for 1 month? This was a stop-gap measure, for an unsupported configuration, so there is no real reason to complain about this.
There is a follow up PR here #65473.

Another thing: I do not really like the tone used here. Please be nice to people and maintainers, and read our code of conduct: https://github.com/Homebrew/.github/blob/HEAD/CODE_OF_CONDUCT.md#code-of-conduct

@Homebrew Homebrew locked as too heated and limited conversation to collaborators Nov 30, 2020
@claui
Copy link
Contributor

claui commented Nov 30, 2020

@mlindner Firstly: to imply we didn’t know what we’re doing seems pretty harsh, doesn’t it? We don’t appreciate such tone of voice, and I invite you to read and abide by our code of conduct.
Secondly, the upstream project just didn’t yet have any assembly for macOS/arm64 at the time this PR was merged.
Finally, it clearly says it’s a stop-gap, not a permanent solution.
Are we talking about the same thing?

(Edit: @iMichka beat me to it.)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
11 Big Sur is specifically affected
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants