Labelling this PR as size/L

GCE e2e build/test passed for commit a83f11c.

• Build Log
• Test Artifacts
• Internal Jenkins Results

I simulated regional fallback by using a development GKE job that had a faked regional preference list that started with an invalid region, e.g. moon.

LGTM.

@zmerlynn I just followed your way to test it in k8s. In cluster creation, k8s uploads tarballs to each zone in the PREFERRED_REGION list. So, I put "moon" at the head of the list and hack the code to avoid creating GCS bucket in "moon" zone (otherwise, it will fail the cluster creation).

After cluster is up, I verified that the regional fallback works as cluster downloads tarballs from the second zone in the list, i.e., "us". But I see there is a bug in env variable "KUBE_ADDON_REGISTRY". The value of KUBE_ADDON_REGISTRY is calculated at https://github.com/kubernetes/kubernetes/blob/master/cluster/gce/util.sh#l164. So it is set to match the first zone in the PREFERRED_REGION list, which is "moon" in our case. Then this value is populated through kube-env. In ContainerVM-based cluster, docker images path is updated https://github.com/kubernetes/kubernetes/blob/master/cluster/saltbase/install.sh#L93-L100, because KUBE_ADDON_REGISTRY is different from "gcr.io/google_containers". In PR #22616 I followed the same logic as your PR #22571. But if the preferred zone is isolated, the image registry will not be accessible. This is what I saw in my test. Since there is no such "moon" zone, kube-addons all fail to fetch docker images from moon registry.

I think the calculation of KUBE_ADDON_REGISTRY should be in the same place as downloading the tarballs, instead of in util.sh. The bug is not related with this PR, so I think we can go ahead to let it be merged. We should fix the calculation of KUBE_ADDON_REGISTRY in another PR. What do you think?

@andyzheng0831: Yes, sorry, I meant to tell you that that method will work for the tarballs, but then add-ons will fail to load. We have no solution for fallback for the image registries right now (there are technical limitations there).

@zmerlynn Got you. Thanks for clarifying it, and I am totally fine with it as you already saw this problem.

@k8s-bot test this

Tests are more than 48 hours old. Re-running tests.

GCE e2e build/test passed for commit a83f11c.

• Build Log
• Test Artifacts
• Internal Jenkins Results

@zmerlynn @roberthbailey , is the original PR for ContainerVm merged in release-1.2 branch? If so, please mark this PR as cherry-pick candidate. You can do it at proper time to avoid interfering 1.2.1 release. Thanks!

@andyzheng0831: Yes, it's been live since 1.2 shipped.

Removing label cherrypick-candidate because no release milestone was set. This is an invalid state and thus this PR is not being considered for cherry-pick to any release branch. Please add an appropriate release milestone and then re-add the label.

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

GCE e2e build/test passed for commit a83f11c.

• Build Log
• Test Artifacts
• Internal Jenkins Results

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

GCE e2e build/test passed for commit a83f11c.

• Build Log
• Test Artifacts
• Internal Jenkins Results

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

GCE e2e build/test passed for commit a83f11c.

• Build Log
• Test Artifacts
• Internal Jenkins Results

Automatic merge from submit-queue

@andyzheng0831 According to the recently updated cherry pick process you need to create an automated cherry pick PR.

Also, all cherry picks should have a release note, so please add one to this PR.

@roberthbailey this PR was already added a "release-note-none" label. Do I still need to remove it and add a "release-note" label?

Any PR that gets cherry picked should have a release note. Please add one and re-label this PR.

From https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes I see the release note will be my PR title, so what left is to change the label from "release-note-none" to "release-note". Is my understanding correct? How to re-label it? I don't think I have permission for it.

I switched the labels (and am assuming you are happy with the title for the release note text).

Thanks! The title is good for me, as it matches the title of Zach's original PR for ContainerVM.

Commit found in the "release-1.2" branch appears to be this PR. Removing the "cherrypick-candidate" label. If this is an error find help to get your PR picked.