@deads2k - I thought we were trying to get rid of kubernetes_auth files ;-)

@LiGgit - FYI

.kubeconfig files can be made portable so they could serve the same purpose. Using .kubeconfig files means that they can be manipulated and inspected using the command line tooling we have already developed. They also make it easy for users to be sure that the credentials they are going to provide to pods have the permissions they expect. The user simply creates the .kubeconfig file and runs the normal osc commands with the --kubeconfig flag.

I know there's been some difficulty understanding the merge order and the like, but a single self-container .kubeconfig file that holds all the secrets with a single set of entries isn't too bad to reason about.

Do we really need breakage? Can't we just define a different port or
service and deprecate kubernetes_ro?

On Thu, Mar 26, 2015 at 1:56 PM, David Eads notifications@github.com
wrote:

.kubeconfig files can be made portable so they could serve the same
purpose. Using .kubeconfig files means that they can be manipulated and
inspected using the command line tooling we have already developed. They
also make it easy for users to be sure that the credentials they are going
to provide to pods have the permissions they expect. The user simply
creates the .kubeconfig file and runs the normal osc commands with the
--kubeconfig flag.

I know there's been some difficulty understanding the merge order and the
like, but a single self-container .kubeconfig file that holds all the
secrets with a single set of entries isn't too bad to reason about.

—
Reply to this email directly or view it on GitHub
#5921 (comment)
.

Another alternative: Instead of creating a proxy on each node, create a proxy that we build into a docker image that can (optionally) be deployed in a pod along with a service that exposes a read-only http endpoint. This gives folks an easy transition path from http -> https but makes the default cluster installation only use secure communications to the master.

@liggitt FYI (take 2, hopefully spelled correctly this time)

@cjcullen and anyone else following along:

Make secrets at cluster startup. #5470 is in.
That creates the following user accounts, which are relevant to this Issue:

• system:logging
• system:monitoring
• system:dns

It also makes secrets at cluster startup in the default namespace with names:

• token-system-logging
• token-system-monitoring
• token-system-dns

You can see these if you start a cluster and then kubectl get secrets token-system-logging, etc.

So, dns needs to use the token-system-dns token and talk to the apiserver on port 6443, rather than using 7080. The code that sets up its client is in cluster/addons/dns/kube2sky/kube2sky.go.
So:

1. change newKubeClient in cluster/addons/dns/kube2sky/kube2sky.go to read a kubeconfig file. Follow the example in Use kubeconfig in several components #6969.
2. change the pod definition, at cluster/addons/dns/skydns-rc.yaml.in, so that the pod has a secret volume and the kube2sky container mounts that volume and has an argument to read the kubeconfig file.

Then, do the same for cluster/addons/fluentd-elasticsearch/... with the token-system-logging.

Then, decide if it is worthwhile to do:

• examples/cassandra/java/io/k8s/cassandra/KubernetesSeedProvider.java
• examples/phabricator/cloudsql-authenticator/run.sh
• examples/rethinkdb/image/run.sh
• examples/rethinkdb/image/run.sh
• contrib/for-tests/network-tester/webserver.go

If we don't change those, the demos won't work on on a system without port 7080.

We maybe don't need to use a different token for every component. We could just define a "default pod user", and make a token for that user, and use that token in all of the above examples.

#7154 updated skydns/kube2sky to use port 6443 w/ the token-system-dns.

I'm taking a look at fluentd/elasticsearch now.

It looks like fluentd/elasticsearch doesn't actually use kubernetes-ro. Am I missing something?

I can't find fluend/elasticsearch either now.

@vishh changed heapster to support kubernetes_auth files. https://github.com/GoogleCloudPlatform/heapster/pull/232/files
But then I settled on kubeconfig format instead.
So, heapster needs to change again.

@erictune: Here is the heapster logic that I was hoping would handle secrets.

@vishh You really want @erictune, not @etune. He seems a lot smarter than me.

For the examples that use KUBERNETES_RO, would it make sense to create a readonly user/token pair? We'd need to get the policy right to block that user from reading other secrets...

#7101 might make this easier: If we had service accounts, we could just have demos create a "my-demo-sa" service account, and then create a pod that used that service account.

@vishh since you made that change, we moved the bar, so another PR is needed.

@cjcullen in response to your previous comment, I filed a more detailed proposal, #7444

How it relates to #4567?

@gmarek this is a prereq for #4567.

@lavalamp @cjcullen

"Readonly" isn't actually a particularly useful protection since you can read a secret that contains a credential for accessing in a readwrite fashion.

So, I suggest that we deprecate the readonly port, and just use the per-namespace default service account (#7101) as the way to access the apiserver in all examples and contribs. We can leave it unspecified what permissions this account has (currently read-write, but evolving once the Policy feature is merged).

@erictune SGTM-- and since #7101 has merged I can get right to it.

I'm gonna say that #9233 will fix this; there may be more, but this is the last piece I'm aware of.

Looks like #9233 merged. @erictune @lavalamp to sanity check where we are and whether this is now closable?

I think this is fixed.