cc @davidopp @bgrant0607 @lavalamp @wojtek-t @fgrzadkowski @timothysc

I'm running some experiments to check what does not work, when --port in Kubelet is set (i.e. we don't listen on 10250). I'll update this issue with everything I find.

Proxy to cadvisor is not working. In "normal" cluster connecting to

https://kubenretes-master-ip/api/v1/proxy/nodes/node-ip:10250/stats/container

Prints cadvisor data.

If --port in Kubelet is set to something else (I used 10260)

https://kubenretes-master-ip/api/v1/proxy/nodes/node-ip:10260/stats/container

returns garbage and 10250 is not reachable.

Previous entry is not really true, as I forgot to put --kubelet-flag in the API server. When I did (i.e. I have --port=10260 in Kubelet and --kubelet-port=10260 in API server). Result is interesting:

https://kubenretes-master-ip/api/v1/proxy/nodes/node-ip:10250/stats/container

Is completely happy and prints out cadvisor data, while

https://kubenretes-master-ip/api/v1/proxy/nodes/node-ip:10260/stats/container

returns garbage.

Thanks to above behavior getContainerInfo from test/e2e/kubelet_stats.go works, as it uses the default Kubelet port.

I'm not sure if this is desired behavior, or a bug. Current setup uses a static port on a master as a proxy for Kubelet connection. This has the advantage of being simple, but makes the proxy semantics a bit strage: proxy port X "redirects" to 'real' port Y, while I think we promise mapping to the same port.

If we decide that it's a feature we probably should document this. If we think it's a bug (I'm slightly for this interpretation), we should fix this, and provide some mechanism for learning the port on which Kubelet is running on a given Node.

We can do this by adding Kubelet port to the Node API object. This shouldn't be a big API change, and is non-breaking. Other option is to make Master publish its config (including Kubelet ports) somewhere for external systems to read it.

I'm for the first option (adding this to the API), as it's slightly more general and allows having different Kubelet ports across machines, which may be needed if/when we'll be building Ubernetes (plus it'd make my life easier for writing Kubemark).

cc @bgrant0607

See my comment here https://github.com/kubernetes/kubernetes/pull/12530/files#r37931263

Is this really necessary?

This was the decision we made, plus it's probably bad to allow usage of predefined/hardcoded port only.

Can we unmake this decision? Fixing this seems like a lot of work with only marginal value.

I'd rather not. Especially that it's done and only waiting for reviews (well, actually it waits for me to have time to address comments, which will be on Friday at the earliest). @davidopp @wojtek-t

I see, it does look like it's mostly done. I don't have objections to the change in flight.

@gmarek can we close this issue?

I'll rebase PR #12919 that solves this. Other option is to remove the flag that allows setting Kubelet port, as being in state when we do allow change by flag, but it does not work is worst option.

the assumption that "communication from the API server to those two components goes over localhost" is flawed. we want to be able to split up the apiserver from the scheduler and controller-manager such that this localhost dependency no longer exists.

/cc @karlkfi

xref #13216

One of the tenants of my push for Component Registration is that the components should each be deployable in their own containers, or in their own Pods, rather than all in the same Pod. Several reasons for this:

• allow horizontal scaling of each component independently
• allow components to be independently upgraded/patched
• allow components to be packaged and distributed independently
• require components to communicate exclusively using strictly defined/versioned APIs
• require components to communicate exclusively via configurable means (localhost, socket, via proxy, via load balancer, etc)

Following that train of thought means that the components need to be independently productized and independently configurable.

#17760 and #16153 are merged, and world is still spinning, so I guess we can close this issue now.