set default enabled admission plugins by official document #58684

hzxuzhonghu · 2018-01-23T12:16:36Z

What this PR does / why we need it:

https://kubernetes.io/docs/admin/admission-controllers/#is-there-a-recommended-set-of-admission-controllers-to-use

recommend running the following set of admission controllers

If you previously had not set the `--admission-control` flag, your cluster behavior may change (to be more standard).  See [https://kubernetes.io/docs/admin/admission-controllers/] for explanation of admission control.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Release note:

Set default enabled admission plugins `NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota`

hzxuzhonghu · 2018-01-23T12:16:50Z

/assign @sttts @deads2k

hzxuzhonghu · 2018-01-23T12:17:44Z

And after this , the document also need revise.

deads2k · 2018-01-23T13:06:59Z

pkg/kubeapiserver/options/plugins.go

-	defaultOffPlugins := sets.NewString(AllOrderedPlugins...)
-	defaultOffPlugins.Delete(lifecycle.PluginName)
+	defaultOnPlugins := sets.NewString(
+		lifecycle.PluginName,


Alias the packages to admission plugin names so we can recognize these. I think this is namespacelifecycle.

deads2k · 2018-01-23T13:07:35Z

staging/src/k8s.io/apiserver/pkg/server/options/admission.go

@@ -79,7 +79,7 @@ func NewAdmissionOptions() *AdmissionOptions {
 		// after all the mutating ones, so their relative order in this list
 		// doesn't matter.
 		RecommendedPluginOrder: []string{lifecycle.PluginName, initialization.PluginName, mutatingwebhook.PluginName, validatingwebhook.PluginName},
-		DefaultOffPlugins:      sets.NewString(initialization.PluginName, mutatingwebhook.PluginName, validatingwebhook.PluginName),


split this out, it is non-contentious

deads2k · 2018-01-23T13:10:51Z

This is a huge change. I'm strongly in favor, but we need to agree that we're willing to do this.

People who previously controlled the admission chain using --admission-control will be fine and their options are still respected.
People who specified no admission chains had a scarily exposed cluster they could not run securely, but now they'll suddenly get the admission chain and potentially breaks as things like service accounts will now be require. We've long recommended this, but we've never turned them on by default before.

@smarterclayton @bgrant0607 @kubernetes/api-approvers @lavalamp @erictune @liggitt for comment on having a default admission chain.

hzxuzhonghu · 2018-01-25T01:54:28Z

ping @smarterclayton @bgrant0607 @kubernetes/api-approvers @lavalamp @erictune @liggitt for comment on having a default admission chain.

lavalamp · 2018-01-30T17:45:33Z

I am in favor of fixing this but I'm not sure this is the right way to do it.

I think the UX is super confusing. It's not good to break naive users (e.g., I bet this breaks the local-up script). So, at a minimum I think we need users to opt-in to the new system.
Changing entries in a default-on list is always going to be a breaking change. I think the right way to do this is to have apiserver store the list, probably somewhere in etcd. If the list is unset, then we write the defaults; otherwise, we use the list. That way, we could change the defaults without totally breaking everyone on upgrades.

deads2k · 2018-01-30T17:57:42Z

I think the UX is super confusing. It's not good to break naive users (e.g., I bet this breaks the local-up script). So, at a minimum I think we need users to opt-in to the new system.

I'm ok with having a --disable-admission flag (which is what this would be). I'd like to have a timeline for having --disable-admission default to false.

Changing entries in a default-on list is always going to be a breaking change. I think the right way to do this is to have apiserver store the list, probably somewhere in etcd. If the list is unset, then we write the defaults; otherwise, we use the list. That way, we could change the defaults without totally breaking everyone on upgrades.

The idea of a "default on" list is that we think these are a good default for a safe and sane cluster, it's not a guarantee of particular plugins being on or off. Think about things like the PodToleration plugin. If you don't enable it in the same release that you added taint and toleration support, the feature presents
a risk to your cluster. Most cluster-admins probably want to have their safe cluster continue being safe.

sttts · 2018-01-30T18:25:29Z

How many (non-toy/dev) clusters are out there without --admission-control?

Compared with the big switch to RBAC-by-default, any serious cluster probably has admission plugins enabled anyway. This was very different for RBAC.

bgrant0607 · 2018-02-04T19:07:19Z

We need to think carefully about what we're trying to achieve.

If we want to make it easier for users to stay up to date with recommended admission controllers, I suggest a new flag along the lines of --enable-recommended-admission-controllers, and a warning if the existing flag is otherwise left unset.

erictune · 2018-02-06T00:03:46Z

I'm in favor of making this change, and having it be opt-out.

erictune · 2018-02-06T00:06:17Z

The release note needs to be more specific. Maybe append something like this:

If you previously had not set the `--admission-control` flag, your cluster behavior may change (to be more standard).  See [https://kubernetes.io/docs/admin/admission-controllers/] for explanation of admission control.

hzxuzhonghu · 2018-02-06T01:27:25Z

@erictune ok, will update today.

hzxuzhonghu · 2018-02-06T09:56:33Z

so many case failed with ServiceAccount plugin enabled.

deads2k · 2018-02-06T13:50:31Z

so many case failed with ServiceAccount plugin enabled.

controllers not running maybe?

deads2k · 2018-02-21T20:32:02Z

Sorry, I didn't see that this got itself sorted out.

/lgtm

squash up when you rebase

hzxuzhonghu · 2018-02-22T02:12:10Z

@deads2k @sttts squashed and rebased

deads2k · 2018-02-22T02:14:12Z

/lgtm

hzxuzhonghu · 2018-02-22T03:04:12Z

run gofmt

deads2k · 2018-02-22T12:33:49Z

/lgtm

k8s-ci-robot · 2018-02-22T12:34:34Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, hzxuzhonghu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~hack/OWNERS~~ [deads2k]
~~pkg/kubeapiserver/OWNERS~~ [deads2k]
~~staging/src/k8s.io/apiserver/OWNERS~~ [deads2k]
~~test/OWNERS~~ [deads2k]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-github-robot · 2018-02-22T12:39:29Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2018-02-22T13:24:43Z

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

lavalamp · 2018-05-23T17:52:06Z

pkg/kubeapiserver/options/plugins.go

+		setdefault.PluginName,               //DefaultStorageClass
+		defaulttolerationseconds.PluginName, //DefaultTolerationSeconds
+		mutatingwebhook.PluginName,          //MutatingAdmissionWebhook
+		validatingwebhook.PluginName,        //ValidatingAdmissionWebhook


Why are webhooks off by default?

this looks to make them on by default, not off

My bad. I was reading the function name, not the variable name. Sorry for the noise.

@hzxuzhonghu

> * Default enabled admission plugins are now `NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota`. Please note that if you previously had not set the `--admission-control` flag, your cluster behavior may change (to be more standard). ([#58684](kubernetes/kubernetes#58684), [@hzxuzhonghu](https://github.com/hzxuzhonghu)) https://github.com/kubernetes/kubernetes/blob/a6b0eaecc99c07aef4952e8fae09642cabe4df5e/CHANGELOG/CHANGELOG-1.10.md#L1542

@hzxuzhonghu

> * Default enabled admission plugins are now `NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota`. Please note that if you previously had not set the `--admission-control` flag, your cluster behavior may change (to be more standard). ([#58684](kubernetes/kubernetes#58684), [@hzxuzhonghu](https://github.com/hzxuzhonghu)) https://github.com/kubernetes/kubernetes/blob/a6b0eaecc99c07aef4952e8fae09642cabe4df5e/CHANGELOG/CHANGELOG-1.10.md#L1542

@hzxuzhonghu

> * Default enabled admission plugins are now `NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota`. Please note that if you previously had not set the `--admission-control` flag, your cluster behavior may change (to be more standard). ([#58684](kubernetes/kubernetes#58684), [@hzxuzhonghu](https://github.com/hzxuzhonghu)) https://github.com/kubernetes/kubernetes/blob/a6b0eaecc99c07aef4952e8fae09642cabe4df5e/CHANGELOG/CHANGELOG-1.10.md#L1542

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 23, 2018

k8s-ci-robot assigned deads2k and sttts Jan 23, 2018

k8s-ci-robot requested review from dims, lavalamp and smarterclayton January 23, 2018 12:16

hzxuzhonghu force-pushed the default-enabled-admission branch from 48574ab to e051ab5 Compare January 23, 2018 12:19

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 23, 2018

deads2k reviewed Jan 23, 2018

View reviewed changes

hzxuzhonghu force-pushed the default-enabled-admission branch 2 times, most recently from 4e9af03 to 21c1040 Compare January 24, 2018 08:16

hzxuzhonghu mentioned this pull request Jan 30, 2018

kube-apiserver flag --admision-control is deprecated, use the new --e… #58567

Merged

hzxuzhonghu force-pushed the default-enabled-admission branch from 21c1040 to 64cb323 Compare February 6, 2018 04:17

k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Feb 21, 2018

hzxuzhonghu force-pushed the default-enabled-admission branch from 1c81b14 to de6bb56 Compare February 22, 2018 02:11

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 22, 2018

k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 22, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 22, 2018

hzxuzhonghu mentioned this pull request Feb 22, 2018

using cobra commands for test/images #59718

Closed

set default enabled admission plugins by official document

27f3fd2

hzxuzhonghu force-pushed the default-enabled-admission branch from de6bb56 to 27f3fd2 Compare February 22, 2018 03:03

k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 22, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 22, 2018

deads2k added this to the v1.10 milestone Feb 22, 2018

k8s-github-robot merged commit 270148d into kubernetes:master Feb 22, 2018

hzxuzhonghu deleted the default-enabled-admission branch February 22, 2018 13:27

luxas mentioned this pull request May 21, 2018

kubeadm should not use --admission-controller anymore. kubernetes/kubeadm#840

Closed

lavalamp reviewed May 23, 2018

View reviewed changes

saad-ali mentioned this pull request May 25, 2018

Audit Default Admission Controllers for Storage #64295

Closed

ChiuWang mentioned this pull request Aug 8, 2018

Add default enabled admission control plugins kubernetes/website#9784

Merged

jqmichael mentioned this pull request May 21, 2020

Clarified LimitRange is enabled by default kubernetes/website#21110

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

set default enabled admission plugins by official document #58684

set default enabled admission plugins by official document #58684

hzxuzhonghu commented Jan 23, 2018 •

edited

Loading

hzxuzhonghu commented Jan 23, 2018

hzxuzhonghu commented Jan 23, 2018

deads2k Jan 23, 2018

hzxuzhonghu Jan 24, 2018

deads2k Jan 23, 2018

deads2k commented Jan 23, 2018

hzxuzhonghu commented Jan 25, 2018

lavalamp commented Jan 30, 2018

deads2k commented Jan 30, 2018

sttts commented Jan 30, 2018 •

edited

Loading

bgrant0607 commented Feb 4, 2018

erictune commented Feb 6, 2018

erictune commented Feb 6, 2018

hzxuzhonghu commented Feb 6, 2018

hzxuzhonghu commented Feb 6, 2018

deads2k commented Feb 6, 2018

deads2k commented Feb 21, 2018

hzxuzhonghu commented Feb 22, 2018

deads2k commented Feb 22, 2018

hzxuzhonghu commented Feb 22, 2018

deads2k commented Feb 22, 2018

k8s-ci-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

lavalamp May 23, 2018

liggitt May 23, 2018

lavalamp May 23, 2018

set default enabled admission plugins by official document #58684

set default enabled admission plugins by official document #58684

Conversation

hzxuzhonghu commented Jan 23, 2018 • edited Loading

hzxuzhonghu commented Jan 23, 2018

hzxuzhonghu commented Jan 23, 2018

deads2k Jan 23, 2018

Choose a reason for hiding this comment

hzxuzhonghu Jan 24, 2018

Choose a reason for hiding this comment

deads2k Jan 23, 2018

Choose a reason for hiding this comment

deads2k commented Jan 23, 2018

hzxuzhonghu commented Jan 25, 2018

lavalamp commented Jan 30, 2018

deads2k commented Jan 30, 2018

sttts commented Jan 30, 2018 • edited Loading

bgrant0607 commented Feb 4, 2018

erictune commented Feb 6, 2018

erictune commented Feb 6, 2018

hzxuzhonghu commented Feb 6, 2018

hzxuzhonghu commented Feb 6, 2018

deads2k commented Feb 6, 2018

deads2k commented Feb 21, 2018

hzxuzhonghu commented Feb 22, 2018

deads2k commented Feb 22, 2018

hzxuzhonghu commented Feb 22, 2018

deads2k commented Feb 22, 2018

k8s-ci-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

lavalamp May 23, 2018

Choose a reason for hiding this comment

liggitt May 23, 2018

Choose a reason for hiding this comment

lavalamp May 23, 2018

Choose a reason for hiding this comment

hzxuzhonghu commented Jan 23, 2018 •

edited

Loading

sttts commented Jan 30, 2018 •

edited

Loading