kubeadm: add `kubeadm phase addons` command #51171

andrewrynhard · 2017-08-23T04:05:48Z

What this PR does / why we need it:
Adds the addons phase command to kubeadm

andrewrynhard · 2017-08-23T04:06:54Z

cmd/kubeadm/app/cmd/phases/addons.go

+	var kubeConfigFile string
+	cfg := &kubeadmapiext.MasterConfiguration{}
+	cmd := &cobra.Command{
+		Use:     "addon",


I can change to plural "addons" if we want to allow multiple addons to be installed with one command.

I'd let this be addon but have an addons alias

andrewrynhard · 2017-08-23T04:07:29Z

cmd/kubeadm/app/cmd/phases/addons.go

+		Aliases: []string{},
+		Short:   "Install an addon to a Kubernetes cluster.",
+		RunE: func(cmd *cobra.Command, args []string) error {
+			if len(args) != 1 {


Do we want to allow multiple addons to be installed with one command?

See certs|kubeconfig|controlplane phases; they have an all command in addition

andrewrynhard · 2017-08-23T04:08:45Z

cmd/kubeadm/app/cmd/phases/addons.go

+			kubeadmutil.CheckErr(err)
+			addon := args[0]
+			switch addon {
+			case dnsaddon.KubeDNSServiceAccountName:


Should this be hardcoded or should we stick to using the service account name?

It might be that the ServiceAccountName constants are moved into the respective addon package eventually; so I'd hard-code kube-proxy|kube-dns here

andrewrynhard · 2017-08-23T04:09:24Z

@luxas I left a few comments. I can push updates based on your review.

andrewrynhard · 2017-08-23T06:11:32Z

cmd/kubeadm/app/cmd/phases/addons.go

+			client, err := kubeconfigutil.ClientSetFromFile(kubeConfigFile)
+			kubeadmutil.CheckErr(err)
+			addon := args[0]
+			switch addon {


If we are not happy with a switch statement, I can create a map[string]func(cfg, client), but that would depend on us sticking with the EnsureXXXAddon(cfg, client) convention.

Please try to model this as we've done for the other phases to be consistent here; a struct { } slice should do

Please provide users a way to inspect which addons could be managed with kubeadm addons; other phases are doing this by leveraging on cobra native support for --help and subcommands

luxas

Thanks @andrewrynhard
I'd like it to be a little bit more like the certs, kubeconfig and controlplane phases if that's okay with you.
Please make it possible to configure the addon creation with CLI flags.

/assign @fabriziopandini

luxas · 2017-08-23T06:46:44Z

cmd/kubeadm/app/cmd/phases/addons.go

+	var kubeConfigFile string
+	cfg := &kubeadmapiext.MasterConfiguration{}
+	cmd := &cobra.Command{
+		Use:     "addon",


I'd let this be addon but have an addons alias

luxas · 2017-08-23T06:47:27Z

cmd/kubeadm/app/cmd/phases/addons.go

+		Aliases: []string{},
+		Short:   "Install an addon to a Kubernetes cluster.",
+		RunE: func(cmd *cobra.Command, args []string) error {
+			if len(args) != 1 {


See certs|kubeconfig|controlplane phases; they have an all command in addition

luxas · 2017-08-23T06:49:04Z

cmd/kubeadm/app/cmd/phases/addons.go

+			kubeadmutil.CheckErr(err)
+			addon := args[0]
+			switch addon {
+			case dnsaddon.KubeDNSServiceAccountName:


It might be that the ServiceAccountName constants are moved into the respective addon package eventually; so I'd hard-code kube-proxy|kube-dns here

luxas · 2017-08-23T06:50:45Z

cmd/kubeadm/app/cmd/phases/addons.go

+	}
+
+	cmd.Flags().StringVar(&kubeConfigFile, "kubeconfig", "/etc/kubernetes/admin.conf", "The KubeConfig file to use for talking to the cluster")
+	return cmd


this should take --config as well and other relevant CLI args

Please add support as well for validate mixed argument logic if --config is combined with other flags. See e.g.

kubernetes/cmd/kubeadm/app/cmd/phases/util.go

Line 38 in 2f00e6d

if err := validation.ValidateMixedArguments(cmd.Flags()); err != nil {

luxas · 2017-08-23T06:51:40Z

cmd/kubeadm/app/cmd/phases/addons.go

+			client, err := kubeconfigutil.ClientSetFromFile(kubeConfigFile)
+			kubeadmutil.CheckErr(err)
+			addon := args[0]
+			switch addon {


Please try to model this as we've done for the other phases to be consistent here; a struct { } slice should do

andrewrynhard · 2017-08-25T04:23:30Z

@luxas PTAL, this is still a WIP but I believe it is a step in the direction you wanted.

fabriziopandini · 2017-08-25T06:18:35Z

cmd/kubeadm/app/cmd/phases/addons.go

+	return cmd
+}
+
+func EnsureAllAddons(cfg *kubeadmapi.MasterConfiguration, client clientset.Interface) error {


Is this function going to be replaced with https://github.com/kubernetes/kubernetes/pull/51171/files#diff-eadad99fe8edbb21ce1c0f32a362240a?

Yes, I will make that change and have it up in a bit.

fabriziopandini · 2017-08-25T06:40:25Z

@andrewrynhard 👍 from my side :-)
Apart from the comment/question above, IMO the only step missing is to add necessary CLI flags
/cc @luxas

luxas

A couple of change requests

luxas · 2017-08-25T13:11:50Z

cmd/kubeadm/app/phases/addons/addons.go

+)
+
+// EnsureAllAddons install all addons to a Kubernetes cluster.
+func EnsureAllAddons(cfg *kubeadmapi.MasterConfiguration, client clientset.Interface) error {


please move this function client-side to be consistent; i.e. in cmd/phases/addons.go

luxas · 2017-08-25T13:13:48Z

cmd/kubeadm/app/phases/addons/dns/dns.go

@@ -41,7 +41,12 @@ const (
 )

 // EnsureDNSAddon creates the kube-dns addon
-func EnsureDNSAddon(cfg *kubeadmapi.MasterConfiguration, client clientset.Interface, k8sVersion *version.Version) error {
+func EnsureDNSAddon(cfg *kubeadmapi.MasterConfiguration, client clientset.Interface) error {
+	k8sVersion, err := version.ParseSemantic(cfg.KubernetesVersion)


I'd prefer to pass k8sVersion still and do the parsing client-side in cmd/phases
Luckily, I expect to soon get rid of this in the v1.9 cycle by moving this into the internal variant of cfg or something so we can stop passing and parsing this all the time.

The only issue I had with this was that I needed all the EnsureXXXAddon functions have the same signature. Look at the EnsureAllAddons and runAddonsCmdFunc functions. I'm happy to do something different if we still prefer to pass k8sVersion in.

ok with this for now; will fix it for real later

luxas · 2017-08-25T13:18:18Z

cmd/kubeadm/app/cmd/phases/addons.go

+
+		// Add flags to the command
+		cmd.Flags().StringVar(&kubeConfigFile, "kubeconfig", "/etc/kubernetes/admin.conf", "The KubeConfig file to use for talking to the cluster")
+		cmd.Flags().StringVar(&cfgPath, "config", cfgPath, "Path to kubeadm config file (WARNING: Usage of a configuration file is experimental)")


Please add flags for these fields:
common:

KubernetesVersion

ImageRepository

proxy:

AdvertiseAddress

BindPort

PodSubnet

dns:

DNSDomain

luxas · 2017-08-25T13:19:16Z

cmd/kubeadm/app/cmd/phases/addons.go

+		},
+		{
+			use:     "kube-dns",
+			short:   "Install kube-dns addon to a Kubernetes cluster.",


nit: Install the ...

luxas · 2017-08-25T13:19:19Z

cmd/kubeadm/app/cmd/phases/addons.go

+		},
+		{
+			use:     "kube-proxy",
+			short:   "Install kube-proxy addon to a Kubernetes cluster.",


nit: Install the ...

luxas · 2017-08-25T13:19:53Z

cmd/kubeadm/app/cmd/phases/addons.go

+		api.Scheme.Convert(cfg, internalcfg, nil)
+		client, err := kubeconfigutil.ClientSetFromFile(kubeConfigFile)
+		kubeadmutil.CheckErr(err)
+		// internalcfg, err := configutil.ConfigFileAndDefaultsToInternalConfig(*cfgPath, cfg)


why commented out?
Also pass cfgPath to this func

Whoops. This was a mistake.

andrewrynhard · 2017-08-26T21:29:45Z

@fabriziopandini @luxas I believe I have everything done now.

fejta-bot · 2017-09-03T14:45:50Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

fejta-bot · 2017-09-03T20:00:50Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

fejta-bot · 2017-09-04T00:54:50Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

andrewrynhard · 2017-09-04T04:09:16Z

/retest

fejta-bot · 2017-09-04T06:51:20Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

luxas · 2017-09-04T08:10:49Z

/retest

luxas · 2017-09-04T08:11:24Z

@andrewrynhard this needs a rebase

fejta-bot · 2017-09-04T11:03:50Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

fejta-bot · 2017-09-04T13:51:49Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

fejta-bot · 2017-09-04T16:39:50Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

fejta-bot · 2017-09-04T20:09:50Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

andrewrynhard · 2017-09-05T01:34:37Z

/retest

andrewrynhard · 2017-09-07T03:57:13Z

/test pull-kubernetes-e2e-kops-aws

andrewrynhard · 2017-09-07T03:57:20Z

/test pull-kubernetes-e2e-gce-etcd3

andrewrynhard · 2017-09-07T05:26:10Z

/test pull-kubernetes-e2e-kops-aws

luxas

/lgtm

k8s-github-robot · 2017-09-07T06:08:27Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andrewrynhard, luxas

Associated issue: 418

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~cmd/kubeadm/OWNERS~~ [luxas]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

k8s-github-robot · 2017-09-07T06:09:09Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2017-09-07T07:03:15Z

Automatic merge from submit-queue

k8s-ci-robot · 2017-09-07T07:06:50Z

@andrewrynhard: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-e2e-kops-aws	`d55cea6`	link	`/test pull-kubernetes-e2e-kops-aws`
pull-kubernetes-e2e-gce-bazel	`d55cea6`	link	`/test pull-kubernetes-e2e-gce-bazel`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

fabriziopandini · 2017-09-07T09:42:53Z

👍

r2d4 · 2017-09-07T18:30:54Z

cmd/kubeadm/app/cmd/phases/addons.go

+			kubeadmutil.CheckErr(err)
+		}
+
+		var kubeConfigFile string


I'm not able to actually get any of these commands working. Because of this closure, the kubeconfig value from the flag won't ever get used.

$ _output/bin/kubeadm alpha phase addon all --kubeconfig test failed to load admin kubeconfig [open : no such file or directory]

cc @luxas @andrewrynhard

I can have a fix soon. Thanks @r2d4

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 23, 2017

k8s-github-robot assigned justinsb and timothysc Aug 23, 2017

k8s-github-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. release-note-label-needed labels Aug 23, 2017

andrewrynhard commented Aug 23, 2017

View reviewed changes

luxas reviewed Aug 23, 2017

View reviewed changes

luxas added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-label-needed labels Aug 23, 2017

luxas added this to the v1.8 milestone Aug 23, 2017

luxas assigned luxas and unassigned justinsb Aug 23, 2017

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 23, 2017

andrewrynhard force-pushed the proxy-dns-phase branch from 46b22b8 to 989d9d9 Compare August 25, 2017 04:22

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 25, 2017

k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 25, 2017

andrewrynhard force-pushed the proxy-dns-phase branch 2 times, most recently from 0a7c6b2 to 5a56adf Compare August 25, 2017 04:38

fabriziopandini reviewed Aug 25, 2017

View reviewed changes

k8s-ci-robot requested a review from luxas August 25, 2017 06:40

luxas reviewed Aug 25, 2017

View reviewed changes

andrewrynhard force-pushed the proxy-dns-phase branch from 5a56adf to 48c8423 Compare August 26, 2017 21:24

andrewrynhard force-pushed the proxy-dns-phase branch from 13b9955 to cf05141 Compare September 4, 2017 20:46

k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 4, 2017

kubeadm: add addons command

d55cea6

andrewrynhard force-pushed the proxy-dns-phase branch from cf05141 to d55cea6 Compare September 7, 2017 02:54

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 7, 2017

luxas approved these changes Sep 7, 2017

View reviewed changes

k8s-github-robot merged commit ea01771 into kubernetes:master Sep 7, 2017

andrewrynhard deleted the proxy-dns-phase branch September 7, 2017 07:04

luxas mentioned this pull request Sep 7, 2017

Apply ValidateMixedArguments to all kubeadm commands kubernetes/kubeadm#376

Closed

r2d4 reviewed Sep 7, 2017

View reviewed changes

kubeadm: add kubeadm phase addons command #51171

kubeadm: add kubeadm phase addons command #51171

Conversation

andrewrynhard commented Aug 23, 2017 • edited by luxas Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewrynhard commented Aug 23, 2017

andrewrynhard Aug 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luxas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabriziopandini Aug 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewrynhard commented Aug 25, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabriziopandini commented Aug 25, 2017

luxas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewrynhard commented Aug 26, 2017

fejta-bot commented Sep 3, 2017

fejta-bot commented Sep 3, 2017

fejta-bot commented Sep 4, 2017

andrewrynhard commented Sep 4, 2017

fejta-bot commented Sep 4, 2017

luxas commented Sep 4, 2017

luxas commented Sep 4, 2017

fejta-bot commented Sep 4, 2017

fejta-bot commented Sep 4, 2017

fejta-bot commented Sep 4, 2017

fejta-bot commented Sep 4, 2017

andrewrynhard commented Sep 5, 2017

andrewrynhard commented Sep 7, 2017

andrewrynhard commented Sep 7, 2017

andrewrynhard commented Sep 7, 2017

luxas left a comment

Choose a reason for hiding this comment

k8s-github-robot commented Sep 7, 2017

k8s-github-robot commented Sep 7, 2017

k8s-github-robot commented Sep 7, 2017

k8s-ci-robot commented Sep 7, 2017

fabriziopandini commented Sep 7, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kubeadm: add `kubeadm phase addons` command #51171

kubeadm: add `kubeadm phase addons` command #51171

andrewrynhard commented Aug 23, 2017 •

edited by luxas

Loading

andrewrynhard Aug 23, 2017 •

edited

Loading

fabriziopandini Aug 23, 2017 •

edited

Loading