Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-build hyperkube and debian-iptables for ARM. Also add a flannel image #21617

Merged
merged 1 commit into from
Apr 5, 2016

Conversation

luxas
Copy link
Member

@luxas luxas commented Feb 20, 2016

We have to be able to build complex docker images too on amd64 hosts.
Right now we can't build Dockerfiles with RUN commands when building for other architectures e.g. ARM.

Resin has a tutorial about this here: https://resin.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/
But it's a bit clumsy syntax.

The other alternative would be running this command in a Makefile:

# This registers in the kernel that ARM binaries should be run by /usr/bin/qemu-{ARCH}-static
docker run --rm --privileged multiarch/qemu-user-static:register --reset

and

ADD https://github.com/multiarch/qemu-user-static/releases/download/v2.5.0/x86_64_qemu-arm-static.tar.xz /usr/bin

Then the kernel will be able to differ ARM binaries from amd64. When it finds a ARM binary, it will invoke /usr/bin/qemu-arm-static first and lets qemu translate the ARM syscalls to amd64 ones.
Some code here: https://github.com/multiarch

WDYT is the best approach? If registering binfmt_misc in the kernels of the machines is OK, then I think we should go with that.
Otherwise, we'll have to wait for resin's patch to be merged into mainline qemu before we may use the code I have here now.

@fgrzadkowski @david-mcmahon @brendandburns @zmerlynn @ixdy @ihmccreery @thockin

@k8s-github-robot
Copy link

Labelling this PR as size/L

@k8s-github-robot k8s-github-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Feb 20, 2016
@k8s-bot
Copy link

k8s-bot commented Feb 20, 2016

GCE e2e test build/test passed for commit 2d6c0ecf86c035d9f71a540d775d643049e9173b.

@k8s-github-robot
Copy link

The author of this PR is not in the whitelist for merge, can one of the admins add the 'ok-to-merge' label?

@k8s-github-robot
Copy link

Labelling this PR as size/M

@k8s-github-robot k8s-github-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 20, 2016
@k8s-bot
Copy link

k8s-bot commented Feb 20, 2016

GCE e2e test build/test passed for commit 9cef39ea0fe95741c96dd7e55db462ac14730168.

@k8s-bot
Copy link

k8s-bot commented Feb 20, 2016

GCE e2e test build/test passed for commit 3df5c7b471eb288ca31a8e3ee745f34cd5e3529f.

@k8s-github-robot
Copy link

Labelling this PR as size/L

@k8s-github-robot k8s-github-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 21, 2016
@k8s-bot
Copy link

k8s-bot commented Feb 21, 2016

GCE e2e test build/test passed for commit f83aee048d191c6c26926a545cbaaa2bea97cca4.

@lavalamp
Copy link
Member

Not sure who's best equipped, but assigning to @david-mcmahon. @thockin might be good too. Very sorry for the delay, 1.2 was consuming all of my time and I'm just now going through my stack of PRs.

@luxas
Copy link
Member Author

luxas commented Mar 17, 2016

@lavalamp Ok, what's your opinion about this? It's using qemu from @multiarch for cross-building docker images like hyperkube and debian-iptables. It also makes a flannel image that we could switch to step-by-step. On amd64, it's the official flannel image, otherwise it cross-compiles flannel.
@david-mcmahon @zmerlynn Please give your opinions too.

Before this may be merged, some of you should push the debian-iptables and the flannel images.
hyperkube will be built on every release.

It would be great to start with this, then I may create a follow-up PR that updates references to the images and so on.

@lavalamp
Copy link
Member

Unfortunately, I've never tried doing an arm cross-build, so I don't really have an opinion on this. But if you say it works I'll believe you :)

@david-mcmahon david-mcmahon removed their assignment Mar 17, 2016
@david-mcmahon
Copy link
Contributor

As I don't plan on looking at this in the next couple of weeks at least, removing myself. @luxas feel free to take this and run with it.

@luxas
Copy link
Member Author

luxas commented Mar 17, 2016

@david-mcmahon What do you mean?

@david-mcmahon
Copy link
Contributor

@luxas did you want to work on this?

@luxas
Copy link
Member Author

luxas commented Apr 4, 2016

@k8s-bot test this please github issue: #IGNORE

@luxas
Copy link
Member Author

luxas commented Apr 4, 2016

Thanks a lot @david-mcmahon!
I was already trusted for testing, so retriggered the tests now

@k8s-bot
Copy link

k8s-bot commented Apr 4, 2016

GCE e2e build/test passed for commit fab9ca549e89ba0d6fe5091b3bf8e443c600bc6b.

@david-mcmahon david-mcmahon added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed needs-ok-to-merge labels Apr 4, 2016
@david-mcmahon
Copy link
Contributor

LGTM pending passing of tests.

…r arm, arm64 and ppc64le. Build and push hyperkube for arm on every release
@luxas
Copy link
Member Author

luxas commented Apr 4, 2016

Will update. Ran hack/update-generated-docs.sh and added some READMEs

@luxas luxas force-pushed the hyperkube_for_arm branch from fab9ca5 to 560268e Compare April 4, 2016 20:35
@k8s-github-robot
Copy link

PR changed after LGTM, removing LGTM.

@k8s-github-robot k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 4, 2016
@david-mcmahon david-mcmahon added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 4, 2016
@luxas
Copy link
Member Author

luxas commented Apr 4, 2016

Yay! Was able to get k8s up and running by following the official docker guide with an arm image built from this PR!

@david-mcmahon
Copy link
Contributor

@luxas Thanks for taking the time to explain the need for bootstrapping the images.

@k8s-bot
Copy link

k8s-bot commented Apr 4, 2016

GCE e2e build/test passed for commit 560268e.

@k8s-github-robot
Copy link

The author of this PR is not in the whitelist for merge, can one of the admins add the 'ok-to-merge' label?

@k8s-github-robot
Copy link

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

@k8s-bot
Copy link

k8s-bot commented Apr 4, 2016

GCE e2e build/test passed for commit 560268e.

@k8s-github-robot
Copy link

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

@k8s-bot
Copy link

k8s-bot commented Apr 5, 2016

GCE e2e build/test passed for commit 560268e.

@k8s-github-robot
Copy link

Automatic merge from submit-queue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants