Many thanks Eduardo! I'm just updating my DTK to the release of Big Sur at the moment, so I'll take a look at the state of Homebrew then. My intention is to point to this branch until some of the 4.12-specific tools are fixed up in ocaml/opam-repository#17530 (at which point we can just publish the 4.12beta into Homebrew ARM as the recommended version to use)

Naive question: if the 4.11 backport is easier (and, I suppose, less bug-prone), why not use the 4.11 backport in homebrew instead of this 4.10 backport?

Just to minimise changes with the x86 version, which is current upstream at 4.10.0. https://github.com/Homebrew/homebrew-core/blob/master/Formula/ocaml.rb#L16

Just to minimise changes with the x86 version, which is current upstream at 4.10.0

Good thing they are not stuck at 4.04.2 like some other package collection that ends in "win" :-)

I'm running this PR through CI precheck at Inria so that it gets tested on ARM64/Linux.

Well, the fan has exploded in my DTK (wasn't that 400 quid well spent!), so we have to wait for an M1 delivery. The first Mac Mini arriving that should be suitable for Inria Jenkins CI should be @MagnusS' on the 25th Sept, and mine is arriving on the 1st Dec. Anyone else got access to one sooner?

Just brew installed ocaml after applying this patch on my DTK, running macOS 11.0.1 build 20B29, against the 4.10.0 sources.

• The patch doesn’t apply cleanly (fails for the Changes file).
• The build fails at the linker stage with the following error messages (see also my full build logs):

Undefined symbols for architecture arm64:
  ".Lcaml_call_gc", referenced from:
      _caml_alloc1 in libasmrun.a(arm64.o)
      caml_alloc2 in libasmrun.a(arm64.o)
      _caml_alloc3 in libasmrun.a(arm64.o)
      caml_allocN in libasmrun.a(arm64.o)
  "_caml_alloc2", referenced from:
      _camlAttr_helper__entry in ocamlcommon.a(attr_helper.o)
      _camlBtype__entry in ocamlcommon.a(btype.o)
      _camlBytesections__entry in ocamlcommon.a(bytesections.o)
      _camlCamlinternalFormat__entry in stdlib.a(camlinternalFormat.o)
      _camlCamlinternalLazy__entry in stdlib.a(camlinternalLazy.o)
      _camlCmi_format__entry in ocamlcommon.a(cmi_format.o)
      _camlConfig__entry in ocamlcommon.a(config.o)
      ...
  "_caml_allocN", referenced from:
      _camlAst_helper__entry in ocamlcommon.a(ast_helper.o)
      _camlAst_invariants__entry in ocamlcommon.a(ast_invariants.o)
      _camlAst_iterator__entry in ocamlcommon.a(ast_iterator.o)
      _camlAst_mapper__entry in ocamlcommon.a(ast_mapper.o)
      _camlBtype__entry in ocamlcommon.a(btype.o)
      _camlBuiltin_attributes__entry in ocamlcommon.a(builtin_attributes.o)
      _camlBytegen__entry in ocamlbytecomp.a(bytegen.o)

Many thanks for checking this @claui! Does brew install ocaml --HEAD work for you instead of this backport? The trunk of OCaml (and the forthcoming 4.12.0) should work cleanly, or else we need to patch the 4.12 betas fairly urgently.

Many thanks for checking this @claui! Does brew install ocaml --HEAD work for you instead of this backport? The trunk of OCaml (and the forthcoming 4.12.0) should work cleanly, or else we need to patch the 4.12 betas fairly urgently.

Without this patch applied, brew install --HEAD ocaml builds and installs just fine, and passes brew test. ✅

Perfect, that's a relief -- our forthcoming release will work correctly. This backport PR has only run on arm64/Linux as we don't have an M1 or DTK in CI at the moment. Since @EduardoRFS doesn't have access to one either, I wonder if we should just forge ahead and put 4.12.0beta1 into Homebrew.

What we really need is a working opam when compiled with 4.12.0alpha2+ -- what's the status of that @dra27 @Octachron @kit-ty-kate? ocaml/opam-repository#17530 indicates that opam doesn't build currently on the 4.12 snaps.

I wonder if we should just forge ahead and put 4.12.0beta1 into Homebrew.

In case you decide to backport to 4.11.1 as suggested by @gasche and others:
Homebrew has been planning to update the ocaml formula to 4.11.1 anyway. If a patch is available, we’ll be happy to include it along the way.

I will try building this again on iOS, I don't have a Mac ARM but this patch was developed for iOS so that should be enough as a preliminar result

What we really need is a working opam when compiled with 4.12.0alpha2+ -- what's the status of that @dra27 @Octachron @kit-ty-kate? ocaml/opam-repository#17530 indicates that opam doesn't build currently on the 4.12 snaps.

opam itself does build, it just needs a release of extlib. However homebrew uses the vendored libraries way of building opam so ygrek/ocaml-extlib#55 (or better, the next release, cc @ygrek) needs to be vendored (cc @rjbou)

As for the rest of the ecosystem the main issue is dune (see ocaml/dune#3793 / ocaml/dune#3904). Most of the packages build fine with opam-alpha-repository but many "core packages" have to be released/fixed (dune, base, core, lwt, utop, ppx_deriving, js_of_ocaml, extlib, coq, …).

See full list here: https://github.com/kit-ty-kate/opam-alpha-repository/tree/master/packages

The error message posted by @claui is pretty informative, actually. It should be easy to fix runtime/arm64.S to get rid of those errors. No need to rush an upgrade to 4.12.

I also replicated the errors from @claui on the iPhone and I remember needing to fix those the last time I did this backport

Did some adjustments, and got this to build and tested on iPhone and I'm currently testing on a raspberry pi4.

Changes memprof.c was still required, I'm not sure on why the conflict happened. And I had forgotten a couple of details on arm64.S

@claui could you try it again?

@EduardoRFS Had to modify the patch file to remove the Changes hunk again, Otherwise, builds and runs just fine, also passes brew test. 🐫

builds and runs just fine, also passes brew test.

Excellent news! Now we're in business! Thanks to all the participants to this discussion for being so reactive.

Uploaded a snapshot to Homebrew/formula-patches#318 so we can prepare the 4.10 formula PR on Homebrew’s side.
Will update there if this PR changes.

All green on Linux ARM64

Well, the fan has exploded in my DTK (wasn't that 400 quid well spent!), so we have to wait for an M1 delivery. The first Mac Mini arriving that should be suitable for Inria Jenkins CI should be @MagnusS' on the 25th Sept, and mine is arriving on the 1st Dec. Anyone else got access to one sooner?

I have one which you can borrow, if you want to pick it up, if you're in Cambridge at the moment. Email for the address...

@xavierleroy anything preventing us from mergin this? Especially now that is already being used by homebrew

Actually I don't know what we are going to do here. We could merge in the 4.10 development branch, even though there will probably be no new 4.10 releases. But it's a big amount of code. And then it would make equal sense to backport to 4.11 and merge there, because there might be a new 4.11 release. But it is again a big amount of code. I'll let our release manager @Octachron decide.

I was able to build this with the 4.10 branch on an Apple M1 mini and xcode 12.2. One test failed with the error below, but looks like this test is skipped on 4.12 (482b7fe + 8b6241c):

$ ./configure && make -j8 && make tests
[...]
 ... testing 'driver.ml' with 1.1.1.1.1.1 (run) => failed (Running program /Users/magnus/git/ocaml/testsuite/_ocamltest/tests/unwind/driver/ocamlopt.byte/unwind_test without any argument: command
/Users/magnus/git/ocaml/testsuite/_ocamltest/tests/unwind/driver/ocamlopt.byte/unwind_test
failed with exit code 1)
[..]
List of failed tests:
    tests/unwind/'driver.ml' with 1.1.1.1.1.1 (run)

Summary:
  2588 tests passed
   37 tests skipped
    1 tests failed
  107 tests not started (parent test skipped or failed)
    0 unexpected errors
  2733 tests considered
#### Something failed. Exiting with error status.

With homebrew brew install -s ocaml and brew install opam works after adding the patch to the formula. It did have a reject on Changes as reported earlier.

@claui I was unable to use the patch from Homebrew/formula-patches#318 as I get Makefile:1165: *** The native-code compiler is not supported on this platform. Stop. -- it looks like some changes from configure may be missing

@xavierleroy , if we want to distribute the patch widely on a tier-1 platform, I think it might make sense to have an exceptional 4.10.2 release, with an understanding that any ulterior bug fixes would happen on 4.12.

Otherwise, we could merge to the dev version of 4.10, but then packagers would need to patch the VERSION file to get usable sources.

The other solution would be to merge the change in a specialized branch. But this create a new kind of release branches with a quite conditional existence.

From an exterior point of view, an exceptional "new-tier-1-platform-initial-support" version seems simpler than an exceptional source of distribution.

@Octachron : it's your choice. Personally, I'd be happy with a patched 4.10 in homebrew and perhaps macports, everyone else using 4.12 alpha/beta/final, and this PR not merged. If you choose to merge this in 4.10, watch out for config.guess and config.sub updates that may be missing from this PR.

The opam-repository PR is basically ready to go in that case (aside from a bit of fiddling with the package name). The only reason I'm not merging it immediately is that the branch is sitting in avsm/ocaml and is a bit unofficial-looking.

I do wonder if we're underestimating the amount of time it'll take (e.g.) Coq to move to 4.12 instead of 4.10 though. If @Octachron doesn't mind cutting a 4.10.2 release, I'd prefer that rather than an ad-hoc branch in opam-repository.

I also discussed in the past on having a patched version for esy + mac arm. But it would be a lot easier if we have this merged.

The config script here is already ready, but maybe a proper Changes would be appreciated

Let's go with an official exceptional release for a new tier 1 architecture then ; the maintenance cost seems quite balanced with the frequency of such events.

The changes files will need to be updated indeed.

Removed the changelog from this PR, @Octachron will do the changelog for us

Ok, I'll adjust the opam-repo PR to anticipate a 4.10.2 then.

@xavierleroy This is ready for a review / merge. Changes was fixed by @Octachron

I don't know how to dismiss a "Changes required" review other than doing another review.

I'm very sure the required changes have been made, since the patched code is working in Homebrew and in OPAM.

I'm not going to do another review of the +2422/-1820 commit 7b9612f. Squashing commits is great to have a nice history but prevents incremental reviewing.

@Octachron is welcome to merge in 4.10 on the basis that this PR is very probably equivalent to the patched code that is already working in Homebrew and OPAM.

Ok, I will take care of the merge and release on Monday.

I'm not going to do another review of the +2422/-1820 commit 7b9612f. Squashing commits is great to have a nice history but prevents incremental reviewing.

Just in case you didn't know, the github interface allows you to see the changes by clicking on the force-pushed link:

In this particular case, here is the diff it gives you: https://github.com/ocaml/ocaml/compare/8a3ea2e144ca0eee5eb95597c8c80faaaf856e09..270a7dfe467ed43911f4a58f5ff3e5f155389956

No, I did not know, and it looks very useful, so thanks @kit-ty-kate for pointing it out.

However, the link you give shows completely unrelated changes (@dra27 's fixes for mingw-64 issues), so I'm not reassured yet :-(

To be very clear: I'm expecting to see changes in runtime/arm64.S and in Changes, in response to the review comments, and no other changes.

Indeed, the diff also shows you the upstreamed changes as I'm guessing the last force-push was made to be totally up-to-date with the 4.10 branch. In this case, the only change was 4.10's current HEAD: cf8a959

However if you scroll just a bit above, 2 other force-push were made 4 and 16 days ago and they contain the changes you're looking for:

• https://github.com/ocaml/ocaml/compare/eecf8fef43cb47b9f221995ca12f563099f8eaf2..ba11dab8e2ba38a262d9f25eb9ce9b9c39d0127d
• https://github.com/ocaml/ocaml/compare/ba11dab8e2ba38a262d9f25eb9ce9b9c39d0127d..d6a77c1704838e6ab7b3aa4dcb7038fc24ef9b94

@kit-ty-kate thank you, I also didn't knew it. The last commit is just to fix the check-typo script

@xavierleroy thank you, fixed it and rewrote the commit, there is any indentation guideline for OCaml?