Trusty: Support hybrid cluster with nodes on ContainerVM #23079
Conversation
|
Forgot to mention why I deleted some code in function restart-kube-proxy(). Those lines were added by me to favor e2e tests, when kube-proxy was still running as binary. But now that kube-proxy is moved into a pod and test code was changed, we don't need these lines anymore. |
|
Labelling this PR as size/S |
k8s-github-robot
added
the
size/S
|
GCE e2e build/test passed for commit 7dfb71ed6b15bf4affb29c41b6330613ab40b506. |
|
The author of this PR is not in the whitelist for merge, can one of the admins add the 'ok-to-merge' label? |
|
GCE e2e build/test passed for commit 4a8647cbc79211b7ecb379fd1907880925513aba. |
|
@roberthbailey do you think we need to cherry pick this for 1.2 release branch? I am wondering if we should have a Jenkins job running hybrid clusters to test 1.2 branch, so that if there is a breakage we fix ASAP and do not miss any patch release. |
| else | ||
| ssh-to-node "$1" "sudo /etc/init.d/kube-proxy restart" | ||
| fi | ||
| ssh-to-node "$1" "sudo /etc/init.d/kube-proxy restart" |
There was a problem hiding this comment.
I wouldn't worry about changing this; see #23180
There was a problem hiding this comment.
Yes. I deleted it as it is useless after we moved kube-proxy in a pod.
There was a problem hiding this comment.
With kube-proxy in a pod, there is no longer anything in /etc/init.d/kube-proxy so what you've left isn't useful either. I'd just remove this change since it isn't related to what this PR is trying to do and is covered by my other change (and then they won't have conflicts which will make it easier to merge both and get them cherry picked).
|
@andyzheng0831 - I've added the cherrypick-candidate label. |
roberthbailey
added
the
lgtm
|
Labelling this PR as size/XS |
k8s-github-robot
added
size/XS
|
GCE e2e build/test passed for commit 2ea9ef5. |
|
The "Addon update should propagate add-on file changes" fails on the hybrid cluster. I logged an issue #23221 to request support differentiating the OS distribution in e2e test code, then I will revise the test code to fix this testing case. |
|
@k8s-bot test this [submit-queue is verifying that this PR is safe to merge] |
|
GCE e2e build/test passed for commit 2ea9ef5. |
|
Automatic merge from submit-queue |
Auto commit by PR queue bot
bgrant0607
added
the
cherry-pick-approved
Auto commit by PR queue bot (cherry picked from commit d5316c2)
bgrant0607
added
release-note
|
Commit bf5b6f0 found in the "release-1.2" branch appears to be this PR. Removing the "cherrypick-candidate" label. If this s an error find help to get your PR picked. |
Auto commit by PR queue bot (cherry picked from commit d5316c2)
Auto commit by PR queue bot (cherry picked from commit d5316c2)
Auto commit by PR queue bot (cherry picked from commit d5316c2)
Auto commit by PR queue bot (cherry picked from commit d5316c2)
@roberthbailey @dchen1107 please review it.
cc/ @wonderfly @fabioy FYI.
This change is for supporting hybrid cluster in trusty. Since our current need is to run master on trusty and nodes on ContainerVM, I intentionally did not support the reverse case, i.e., nodes on trusty and master on ContainerVM. I still put the hybrid mode under the framework of "trusty", so as to minimize impact on other distros. Therefore, users need to set "KUBE_OS_DISTRIBUTION=trusty".
When setting the following three variables, the cluster will run on trusty
To run in the hybrid mode, set additional two variables:
I have run e2e against the hybrid cluster using the following settings:
export KUBE_OS_DISTRIBUTION=trusty
export KUBE_GCE_MASTER_PROJECT=ubuntu-os-cloud
export KUBE_GCE_MASTER_IMAGE=ubuntu-1404-trusty-v20160314
export KUBE_GCE_NODE_PROJECT=google-containers
export KUBE_GCE_NODE_IMAGE=container-v1-2-v20160218
go run hack/e2e.go -v --up
go run ./hack/e2e.go -v --test '--test_args=--ginkgo.skip=[Slow]|[Serial]|[Disruptive]|[Flaky]|[Feature:.+]'
Summarizing 1 Failure:
[Fail] [k8s.io] Addon update [BeforeEach] should propagate add-on file changes
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/addon_update.go:358
Ran 159 of 266 Specs in 4323.580 seconds
FAIL! -- 158 Passed | 1 Failed | 0 Pending | 107 Skipped --- FAIL: TestE2E (4324.07s)
The failure is expected.
I also ran cluster/kube-up.sh for the hybrid mode and verified all kube-system pods running.