Description

#31522 added ARM64 wheel builds to CI. However, as discussed in the PR and in #28103, ideally users can use rayproject/ray images directly.

We should thus add a docker image build step to CI that builds the linux/aarch64 docker images.

The main open question (at least for me) is how we make sure the platform-specific images are kept in sync and added to the potentially existing images from other platforms. Concretely:

• rayproject/ray:commithash for linux/x86_64 and linux/aarch64 should be the same
• Do we allow rayproject/ray:nightly for linux/x86_64 and linux/aarch64 to point to different commits?
• Does uploading an aarch64 image to dockerhub "add to" or replace an existing x86_64 image?

Use case

No response