Add configuration item to allow kubeadm join to use a dns name pointing to control plane #59288

stevesloka · 2018-02-02T22:05:11Z

This adds a new flag (--apiserver-advertise-dns-address) to kubeadm which is used in node kubelet.confg to point to API server allowing users to define a DNS entry instead of an IP address.

Fixes kubernetes/kubeadm#411

Adds new flag `--apiserver-advertise-dns-address` which is used in node kubelet.confg to point to API server

// @timothysc @craigtracey

timothysc · 2018-02-03T00:42:42Z

/cc @kubernetes/sig-cluster-lifecycle-pr-reviews

timothysc · 2018-02-03T00:45:40Z

cmd/kubeadm/app/cmd/phases/addons.go

@@ -145,6 +145,7 @@ func getAddonsSubCommands() []*cobra.Command {

 		if properties.use == "all" || properties.use == "kube-proxy" {
 			cmd.Flags().StringVar(&cfg.API.AdvertiseAddress, "apiserver-advertise-address", cfg.API.AdvertiseAddress, `The IP address or DNS name the API server is accessible on`)
+			cmd.Flags().StringVar(&cfg.API.AdvertiseDNSAddress, "apiserver-advertise-dns-address", cfg.API.AdvertiseDNSAddress, `The DNS name the API server is accessible on`)


I think I've turned a blind eye to some of the flag proliferation, this seems awfully redundant.

/cc @errordeveloper

part of the issue is the previous advertiseaddress was actually a listen address not an advertise.

stevesloka · 2018-02-03T01:15:26Z

Need a way to allow users to specify dns name for API server as well as the ip address in the event the API server host machine has multiple.

There are the San certs, however multiple can be passed so which one would you choose.

jamiehannaford · 2018-02-04T11:47:07Z

Any reason why we can't just use --apiserver-cert-extra-sans? We'd also need to make sure the bootstrap config files are using the DNS name provided by join cmd (rather than IP which seems to currently be the case unless I'm mistaken).

stevesloka · 2018-02-05T14:29:45Z

I started there when I began thinking through this PR to use --apiserver-cert-extra-sans, but you can pass multiple sans, so which one should I use? We could have a convention that the first one passed is the one to use, but thought that might be more confusing.

craigtracey

I think this is a pretty important change, and this will really help us close this gap. Thanks, @stevesloka!

craigtracey · 2018-02-06T01:20:08Z

cmd/kubeadm/test/util.go

@@ -57,7 +57,7 @@ func SetupMasterConfigurationFile(t *testing.T, tmpdir string, cfg *kubeadmapi.M
 		kind: MasterConfiguration
 		certificatesDir: {{.CertificatesDir}}
 		api:
-		   advertiseAddress: {{.API.AdvertiseAddress}}
+		   advertiseAddress: {{.API.AdvertiseDNSAddress}}


Is this change intentional? AdvertiseAddress should not be the same as AdvertiseDNSAddress

Nope that's a mistake, thanks for catching.

craigtracey · 2018-02-06T01:26:20Z

cmd/kubeadm/app/apis/kubeadm/v1alpha1/types.go

 	AdvertiseAddress string `json:"advertiseAddress"`
+	// AdvertiseDNSAddress sets the DNS address for the API server to advertise.
+	AdvertiseDNSAddress string `json:"advertiseDNSAddress"`


I find this naming a bit confusing as I don't think anything is being "advertised." While the kubelet uses the terminology of advertiseAddress to specify which IP the server is listening on, this new argument is being used only to indicate where a node should register itself. Perhaps a more descriptive argument would be controlPlaneEndpoint?

I think controlPlane make sense, I'll move to that, take out the args, and do a verify it works via the config.

timothysc · 2018-02-06T18:42:15Z

per sig conversation, we should leverage the config object to allow for updates and try not to proliferate cmd line flags b/c anything we do will have to be maintained post GA.

stevesloka · 2018-02-07T15:48:38Z

@timothysc @craigtracey I updated to only use the config, also rebased and pushed.

timothysc · 2018-02-09T00:52:48Z

cmd/kubeadm/app/cmd/phases/controlplane.go

@@ -143,7 +143,7 @@ func getControlPlaneSubCommands(outDir, defaultKubernetesVersion string) []*cobr
 		cmd.Flags().StringVar(&cfg.KubernetesVersion, "kubernetes-version", cfg.KubernetesVersion, `Choose a specific Kubernetes version for the control plane`)

 		if properties.use == "all" || properties.use == "apiserver" {
-			cmd.Flags().StringVar(&cfg.API.AdvertiseAddress, "apiserver-advertise-address", cfg.API.AdvertiseAddress, "The IP address or DNS name the API server is accessible on")
+			cmd.Flags().StringVar(&cfg.API.AdvertiseAddress, "apiserver-advertise-address", cfg.API.AdvertiseAddress, "The IP address of the API server is accessible on")


So I need to look through history here. I think other changes that crib'd hostIP from self-hosting affected this. Sorry it's taking a while but we want the original intent to be preserved. @kad if you have a moment could you dig the change, otherwise i'll have to search for it tomorrow.

This is actually a defect. The kube-apiserver only accepts an IP for --advertise-address.
https://kubernetes.io/docs/reference/generated/kube-apiserver/

I believe what @stevesloka has here is correct.

Just for reference: Yes this can only be an IP and #56956 was just merged to validate this.

stevesloka · 2018-02-13T00:41:21Z

Does anyone have time this week to review? Curious if the implementation is acceptable or if anyone has any additional comments.

timothysc · 2018-02-13T01:29:04Z

/approve

@kubernetes/sig-cluster-lifecycle-pr-reviews - For transparency I'd prefer someone non-heptio to lgtm

kad

small nit: may I suggest to split autogenerated changes to separate commit ?

kad · 2018-02-15T14:54:10Z

cmd/kubeadm/app/util/endpoint.go

+	if len(cfg.API.ControlPlaneEndpoint) > 0 {
+		errs := validation.IsDNS1123Subdomain(cfg.API.ControlPlaneEndpoint)
+		if len(errs) > 0 {
+			return "", fmt.Errorf("error parsing `ControlPlaneEndpoint` to valid dns subdomain")


Maybe here we want to provide to user more information about error ? e.g. what is wrong with domain provided.

stevesloka · 2018-02-15T16:24:51Z

@kad I spit the commits into 2, one for code and the other for autogen bazel deps. Also, I have the validation of the DNS address return the errors from validation method, and updated the error message to show those errors which get returned.

kad · 2018-02-15T16:50:59Z

@stevesloka Thanks. Changes in cmd/kubeadm/app/apis/kubeadm/v1alpha1/zz_generated.conversion.go seems to be still in first commit. The rest looks good for me.

…ng to control plane

stevesloka · 2018-02-15T17:07:04Z

@kad I've moved the zz_generated over to the 2nd commit

stevesloka · 2018-02-20T15:22:25Z

Does anyone have any additional comments on this PR?

timothysc

/lgtm

k8s-ci-robot · 2018-02-22T00:02:18Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: stevesloka, timothysc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~cmd/kubeadm/OWNERS~~ [timothysc]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-github-robot · 2018-02-22T00:03:09Z

/test all

Tests are more than 96 hours old. Re-running tests.

k8s-github-robot · 2018-02-22T06:11:19Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2018-02-22T07:01:07Z

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

First-time kubeadm user here: The v1.10.0 docs indicate that config file use is experimental, but judging from the comments on kubernetes/kubernetes#59288 it looks like new features are beginning to be supported only in the config file. controlPlaneEndpoint is super useful for us and while I suspect the kubeadm init docs will ultimately be re-written to emphasize the config file options rather than flags (with appropriate examples and descriptions as the flags have now), it would've saved us some time today if we'd seen controlPlaneEndpoint in the config file example.

This switches kubeadm to use the new ControlPlaneEndpoint options introduced in kubernetes/kubernetes#59288

First-time kubeadm user here: The v1.10.0 docs indicate that config file use is experimental, but judging from the comments on kubernetes/kubernetes#59288 it looks like new features are beginning to be supported only in the config file. controlPlaneEndpoint is super useful for us and while I suspect the kubeadm init docs will ultimately be re-written to emphasize the config file options rather than flags (with appropriate examples and descriptions as the flags have now), it would've saved us some time today if we'd seen controlPlaneEndpoint in the config file example.

Sebi2020 · 2020-04-21T16:11:06Z

Is this merge request included in the last kubeadm release? If I try to use this flag, kubeadm complains
unknown flag: --apiserver-advertise-dns-address

ATP-55 · 2022-11-16T10:45:54Z

Same here? How to use this flag? @stevesloka
unknown flag: --apiserver-advertise-dns-address
To see the stack trace of this error execute with --v=5 or higher

ATP-55 · 2022-11-16T10:49:22Z

Or are we allowing to pass DNS name under this flag? "--apiserver-advertise-address"

ATP-55 · 2022-11-24T11:29:41Z

Can someone update please?

neolit123 · 2022-11-24T11:39:33Z

you should use --control-plane-endpoint / ControlPlaneEndpoint (in config) which is a kubeadm construct / concept. it's the endpoint that sits in front of API servers.
the api server advertise "endpoint" does not allow using DNS names so only IPs are used.

controlPlaneEndpoint sets a stable IP address or DNS name for the control plane. It can be a valid IP address or a RFC-1123 DNS subdomain, both with optional TCP port. In case the controlPlaneEndpoint is not specified, the advertiseAddress + bindPort are used; in case the controlPlaneEndpoint is specified but without a TCP port, the bindPort is used. Possible usages are:

In a cluster with more than one control plane instances, this field should be assigned the address of the external load balancer in front of the control plane instances.
In environments with enforced node recycling, the controlPlaneEndpoint could be used for assigning a stable DNS to the control plane.

https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/

for more questions please use #kubeadm on k8s slack.

k8s-ci-robot requested review from dmmcquay and mattmoyer February 2, 2018 22:05

k8s-ci-robot added the sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. label Feb 3, 2018

k8s-ci-robot requested a review from errordeveloper February 3, 2018 00:45

timothysc reviewed Feb 3, 2018

View reviewed changes

timothysc removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Feb 3, 2018

timothysc self-assigned this Feb 3, 2018

craigtracey suggested changes Feb 6, 2018

View reviewed changes

timothysc added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 6, 2018

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 7, 2018

stevesloka force-pushed the apiServerDNS branch 2 times, most recently from e082520 to e58598b Compare February 7, 2018 15:46

k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 7, 2018

stevesloka changed the title ~~Add new flag for api-server-dns-name / use for client kubelet.conf~~ Add configuration item to allow kubeadm join to use a dns name pointing to control plane Feb 7, 2018

timothysc reviewed Feb 9, 2018

View reviewed changes

craigtracey mentioned this pull request Feb 9, 2018

WIP: Enable controlPlaneEndpoint config for kubeadm vmware-archive/wardroom#41

Closed

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 13, 2018

timothysc removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 13, 2018

kad reviewed Feb 15, 2018

View reviewed changes

stevesloka force-pushed the apiServerDNS branch from c5cc16d to fd3eca8 Compare February 15, 2018 16:21

stevesloka and others added 2 commits February 15, 2018 12:03

Add configuration item to allow kubeadm join to use a dns name pointi…

2fd29a0

…ng to control plane

Update build deps for Bazel and zz_generated

dc03cc0

stevesloka force-pushed the apiServerDNS branch from fd3eca8 to dc03cc0 Compare February 15, 2018 17:05

timothysc approved these changes Feb 22, 2018

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 22, 2018

k8s-github-robot merged commit d7cadf5 into kubernetes:master Feb 22, 2018

stevesloka deleted the apiServerDNS branch February 22, 2018 14:36

fabriziopandini mentioned this pull request Mar 9, 2018

Extend usage of ControlPlaneEndpoint for HA kubernetes/kubeadm#726

Closed

MordantWastrel mentioned this pull request Mar 29, 2018

Add controlPlaneEndpoint to config file reference kubernetes/website#7909

Merged

fabriziopandini mentioned this pull request Apr 9, 2018

kube-proxy configmap not using apiserver-advertise-address kubernetes/kubeadm#590

Closed

discordianfish added a commit to itskoko/kubecfn that referenced this pull request Apr 12, 2018

Upgrade kubeadm to v1.10.1-beta.0, drop cm-patcher

8e7e688

This switches kubeadm to use the new ControlPlaneEndpoint options introduced in kubernetes/kubernetes#59288

simonklb mentioned this pull request Jun 16, 2018

Add support for domain name as advertise address kubernetes/apiserver#33

Closed

liztio mentioned this pull request Nov 13, 2018

kubeadm alpha phase kubeconfig should take a string for --apiserver-advertise-address kubernetes/kubeadm#1057

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add configuration item to allow kubeadm join to use a dns name pointing to control plane #59288

Add configuration item to allow kubeadm join to use a dns name pointing to control plane #59288

stevesloka commented Feb 2, 2018

timothysc commented Feb 3, 2018

timothysc Feb 3, 2018

kapilt Feb 7, 2018

stevesloka commented Feb 3, 2018 •

edited

Loading

jamiehannaford commented Feb 4, 2018

stevesloka commented Feb 5, 2018

craigtracey left a comment

craigtracey Feb 6, 2018

stevesloka Feb 6, 2018

craigtracey Feb 6, 2018

stevesloka Feb 6, 2018

timothysc commented Feb 6, 2018

stevesloka commented Feb 7, 2018

timothysc Feb 9, 2018

craigtracey Feb 9, 2018

discordianfish Apr 12, 2018

stevesloka commented Feb 13, 2018

timothysc commented Feb 13, 2018

kad left a comment

kad Feb 15, 2018

stevesloka commented Feb 15, 2018

kad commented Feb 15, 2018

stevesloka commented Feb 15, 2018

stevesloka commented Feb 20, 2018

timothysc left a comment

k8s-ci-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

Sebi2020 commented Apr 21, 2020

ATP-55 commented Nov 16, 2022

ATP-55 commented Nov 16, 2022

ATP-55 commented Nov 24, 2022

neolit123 commented Nov 24, 2022

Add configuration item to allow kubeadm join to use a dns name pointing to control plane #59288

Add configuration item to allow kubeadm join to use a dns name pointing to control plane #59288

Conversation

stevesloka commented Feb 2, 2018

timothysc commented Feb 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevesloka commented Feb 3, 2018 • edited Loading

jamiehannaford commented Feb 4, 2018

stevesloka commented Feb 5, 2018

craigtracey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timothysc commented Feb 6, 2018

stevesloka commented Feb 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevesloka commented Feb 13, 2018

timothysc commented Feb 13, 2018

kad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevesloka commented Feb 15, 2018

kad commented Feb 15, 2018

stevesloka commented Feb 15, 2018

stevesloka commented Feb 20, 2018

timothysc left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

k8s-github-robot commented Feb 22, 2018

Sebi2020 commented Apr 21, 2020

ATP-55 commented Nov 16, 2022

ATP-55 commented Nov 16, 2022

ATP-55 commented Nov 24, 2022

neolit123 commented Nov 24, 2022

stevesloka commented Feb 3, 2018 •

edited

Loading