Make addon-manager cross-platform and use it with hyperkube #25631

luxas · 2016-05-15T20:33:13Z

Fixes: #22752 and some other issues as well

Big changes to the docker deployment. Added kube-addon-manager as a static pod. The addon-manager deploys kube-proxy as a DaemonSet, Dashboard and DNS automatically

Make the addon-manager cross-platform, change naming to binary-arch:version, remove deprecated kubectl command, add support for DaemonSets

@mikedanese @vishh @bgrant0607 @fgrzadkowski @dlorenc @brendandburns @thockin @roberthbailey @johndmulhausen @bryk @cheld @lavalamp

bgrant0607 · 2016-05-15T20:49:55Z

cc @Q-Lee

bprashanth · 2016-05-16T01:05:49Z

@k8s-bot unit test this github issue: #25629

bprashanth · 2016-05-16T05:00:48Z

fyi Integration test failed because of merge conflict

GitHub pull request #25631 of commit 6373bc9b55d7c3287486ecc80fa10c9d5d96d4e0, has merge conflicts.
Setting status of 6373bc9b55d7c3287486ecc80fa10c9d5d96d4e0 to PENDING with url https://console.cloud.google.com/storage/browser/kubernetes-jenkins/pr-logs/pull/25631/kubernetes-pull-test-unit-integration/25972/ and message: 'Build started sha1 is original commit.'
Using context: Jenkins unit/integration

cheld · 2016-05-16T06:14:09Z

CC @batikanu, @zreigz

bryk · 2016-05-16T06:59:37Z

cluster/images/hyperkube/addons/dashboard-rc.yaml

+    spec:
+      containers:
+      - name: kubernetes-dashboard
+        image: gcr.io/google_containers/kubernetes-dashboard-ARCH:v1.0.1


Will ARCH be replaced here? If yes, please comment how and why. If not, please fix :)

Yes, it will be replaced by a sed command in the Makefile.
Valid values are amd64, arm, arm64 and ppc64le (as you probably know already 😄)

I'll add a comment

bryk · 2016-05-16T07:03:12Z

A few comments on the dashboard UI side.

luxas · 2016-05-16T21:25:37Z

Updated.
Some patches are included in #25693, though and I'll remove them before I'm done.
Comment if there's anything other that should be done, then I'll squash and it's ready for LGTM

bryk · 2016-05-17T06:31:54Z

cluster/addons/dashboard/dashboard-controller.yaml

@@ -1,3 +1,4 @@
+# This file should be kept in sync with cluster/images/hyperkube/dashboard-rc.yaml


Line 5 also says the same about files in gce/coreos/kube-manifests/addons/dashboard. Can you merge them in one?

bryk · 2016-05-17T06:32:33Z

One small comment and LGTM for Dashboard UI side.

luxas · 2016-05-17T17:15:37Z

@mikedanese Any comments on this one?
Otherwise, I'll rebase, fix @bryk's nit and commit properly for LGTM

luxas · 2016-05-17T20:11:50Z

Updated. Let's get this in now
@mikedanese Could you kindly review this and push the v2 images after lgtm?

yifan-gu · 2016-05-18T00:50:26Z

cc @euank

luxas · 2016-05-18T04:10:18Z

@k8s-bot e2e test this please github issue: #IGNORE

cheld · 2016-05-18T06:10:55Z

CC @tamir, @kenan435

taimir · 2016-05-18T15:24:30Z

Just out of curiosity, have you thought of adding heapster to the addons in the hyperkube container?
And if so, what is the plan to pass configuration (e.g. the sink to which the metrics are sent)?

luxas · 2016-05-19T04:48:06Z

@taimir heapster may come in the future, when I've cross-compiled it for all arches.

And if so, what is the plan to pass configuration (e.g. the sink to which the metrics are sent)?

I guess the sink will also be deployed as an addon in that case (influxdb + grafana)
Maybe we'll expose /etc/kubernetes/addons to the user, but we'll have to discuss that if we want to.
The hyperkube deployment should be an easy way to get started, not being a solution that has many config options.

mikedanese · 2016-05-19T04:58:35Z

@luxas I will review this tomorrow morning, sorry

mikedanese · 2016-05-19T17:41:35Z

cluster/images/hyperkube/addons/kube-proxy.yaml

+      hostNetwork: true
+      containers:
+      - name: kube-proxy
+        image: gcr.io/google_containers/hyperkube-ARCH:VERSION


I don't like that we are introducing even more ways to templative yaml in the addons directory.

What should we do then?

More of an observation... We should redo the entire directory with a single templating langage and expansion script.

mikedanese · 2016-05-19T17:44:58Z

It seems like we are conflating hyperkube image with the docker setup. Hyperkube is generally useful, and we are adding a ton of cruft in that image that is specific to the docker setup (e.g. cert generation stuff, all addons configuration...). I know this is not introduced in this PR but I think we should work to make the hyperkube image more general and slimmer.

mikedanese · 2016-05-19T17:45:36Z

cluster/images/hyperkube/addons/dashboard-svc.yaml

@@ -1,6 +1,4 @@
-#!/bin/bash


How does github think these are the same file??

I have no idea...

mikedanese · 2016-05-19T17:53:49Z

I will push then lgtm

luxas · 2016-05-19T18:20:22Z

Yes, I know.
We should make an issue about the future of the hyperkube image.
I think that localkube (which we are developing right now in the minikube repo) will replace the current docker-based deployment in v1.4 or later.

minikube will take the same approach as hyperkube (very few deps), and make it easy to develop and test Kubernetes out. minikube (which only is a CLI) will offer two kinds of isolation for localkube (which is the kubernetes-all-in-one-server): via docker and a docker-machine VM.

To hyperkube again... There have been discussion about deprecating the individual server binaries and only having two binaries to care about: hyperkube (, maybe kubelet) and kubectl. If we go towards that direction,
the hyperkube image might be used in production deployments for hosting the control plane and the proxy instead of the kube-apiserver, ... images that we have now.

But this is outside the scope of this PR. This is about leveraging a better getting-started solution for v1.3 for people to use.

…ersion, remove deprecated kubectl command, add support for DaemonSets

… static pod. The addon-manager deploys kube-proxy as a DaemonSet as well as Dashboard and DNS automatically. SecurityContextDeny is removed from the manifests. Also, the turnup.sh and turndown.sh scripts are removed because we don't need them anymore, they're covered by the online documentation

luxas · 2016-05-20T16:41:47Z

Reapplying LGTM and retesting unit

luxas · 2016-05-20T16:41:51Z

@k8s-bot unit test this please github issue: #IGNORE

k8s-bot · 2016-05-20T17:08:44Z

GCE e2e build/test passed for commit 73947cc.

googlebot added the cla: yes label May 15, 2016

luxas added this to the v1.3 milestone May 15, 2016

luxas assigned mikedanese May 15, 2016

luxas mentioned this pull request May 15, 2016

Initial add of docker-multinode kubernetes-retired/kube-deploy#12

Merged

k8s-github-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. release-note-label-needed labels May 15, 2016

luxas force-pushed the hyperkube_cni_cross branch from 6373bc9 to a6332dd Compare May 16, 2016 06:16

luxas added the e2e-not-required label May 16, 2016

k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 16, 2016

bryk reviewed May 16, 2016
View reviewed changes

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 16, 2016

bryk reviewed May 17, 2016
View reviewed changes

luxas added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-label-needed labels May 17, 2016

luxas force-pushed the hyperkube_cni_cross branch from 8d600de to 12b8c98 Compare May 17, 2016 20:04

k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 17, 2016

mikedanese reviewed May 19, 2016
View reviewed changes

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 20, 2016

mikedanese added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 20, 2016

luxas added 2 commits May 20, 2016 19:27

Make the addon-manager cross-platform, change naming to binary-arch:v…

bff87ff

…ersion, remove deprecated kubectl command, add support for DaemonSets

luxas force-pushed the hyperkube_cni_cross branch from 12b8c98 to 73947cc Compare May 20, 2016 16:28

k8s-github-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels May 20, 2016

luxas added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 20, 2016

mikedanese merged commit bf70cfa into kubernetes:master May 20, 2016

This was referenced May 24, 2016

Deploy add-ons to docker-multinode kubernetes-retired/kube-deploy#13

Closed

Add docker-in-docker based multi-node development cluster provider #26147

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make addon-manager cross-platform and use it with hyperkube #25631

Make addon-manager cross-platform and use it with hyperkube #25631

luxas commented May 15, 2016

bgrant0607 commented May 15, 2016

bprashanth commented May 16, 2016

bprashanth commented May 16, 2016

cheld commented May 16, 2016

bryk May 16, 2016

luxas May 16, 2016

bryk commented May 16, 2016

luxas commented May 16, 2016

bryk May 17, 2016

bryk commented May 17, 2016

luxas commented May 17, 2016 •

edited

Loading

luxas commented May 17, 2016

yifan-gu commented May 18, 2016

luxas commented May 18, 2016

cheld commented May 18, 2016

taimir commented May 18, 2016

luxas commented May 19, 2016

mikedanese commented May 19, 2016

mikedanese May 19, 2016

luxas May 19, 2016

mikedanese May 19, 2016

mikedanese commented May 19, 2016

mikedanese May 19, 2016

luxas May 19, 2016

mikedanese commented May 19, 2016

luxas commented May 19, 2016

luxas commented May 20, 2016

luxas commented May 20, 2016

k8s-bot commented May 20, 2016

		@@ -1,3 +1,4 @@
		# This file should be kept in sync with cluster/images/hyperkube/dashboard-rc.yaml

Make addon-manager cross-platform and use it with hyperkube #25631

Make addon-manager cross-platform and use it with hyperkube #25631

Conversation

luxas commented May 15, 2016

bgrant0607 commented May 15, 2016

bprashanth commented May 16, 2016

bprashanth commented May 16, 2016

cheld commented May 16, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bryk commented May 16, 2016

luxas commented May 16, 2016

Choose a reason for hiding this comment

bryk commented May 17, 2016

luxas commented May 17, 2016 • edited Loading

luxas commented May 17, 2016

yifan-gu commented May 18, 2016

luxas commented May 18, 2016

cheld commented May 18, 2016

taimir commented May 18, 2016

luxas commented May 19, 2016

mikedanese commented May 19, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikedanese commented May 19, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikedanese commented May 19, 2016

luxas commented May 19, 2016

luxas commented May 20, 2016

luxas commented May 20, 2016

k8s-bot commented May 20, 2016

luxas commented May 17, 2016 •

edited

Loading