@uruddarraju

@liggitt

@mkumatag I've been trying the same thing, but realized that there's a bunch of options that are needed, and keystone supports authentication mechanisms other then just username/password. K2K Federation, Kerberos, Horizon, etc. So rather then add all those things to the Kubernetes client/server, I think we need to switch things around to just pass/use tokens in kubernetes and use the regular keystone client 'keystone token-get' or 'openstack token issue' to auth with all the supported auth types. kubecli can fork off the token issue call, so the user doesn't have to do anything different then their normal OpenStack rc file usage.

. myopenstackrc
kubectl get pods

Issue for keystone tokens here:
#24982

For testing purposes, I also ran into this one that might affect you:
#24984

rackspace/gophercloud module supports function call to load the options from environment which can be used to get the value from env instead of passing through CLI like opts, err := openstack.AuthOptionsFromEnv()

for #24982 :
Rightnow keystone auth code is heavily depends on external package https://github.com/rackspace/gophercloud which is currently handles only username/password based authentication at the moment.

Above usage with rc file may work fine with kubectl command but not really sure how do you handle situation in case of dashboard kind of a login.

#24984 - I got around this problem by adding keystone certificate(ca.pem) into trusted root directory. More information - link

So, if we at least moved the auth code using gophercloud from the apiserver to kubecli and passed the token through, that would get us Keystone V3 & Horizon support. I don't know what gophercloud's status is with regard to K2K federation or Kerberos, but if there was a fall-back to just allowing a raw token from an environment variable, that would work too. I'd be surprised if they didn't have that.

I've been playing with Horizon trying to see what it would take to integrate Horizon & k8s. I think the only thing preventing an Angular based Horizon k8s plugin on the OpenStack side from being developed is this patch:
https://review.openstack.org/#/c/311189/

I discussed it with the Horizon ptl and some of the cores at the summit and they are ok with the patch. It just needs some surrounding tests/docs. But as you can see, its very minimal.

Once that patch is in place, JavaScript code running in Horizon will have a scoped keystone token that it can pass to k8s's api's and can do cors calls as needed to build up an integrated UI.

Thanks for the pointer for the keystone cert thing. I'll have a look and give it a try. :)

Do you expect that there will be a single DomainID and DomainName setting for a single cluster, or multiple within a cluster? If single, flag is fine. If multiple, then this needs to be set inside each users's $HOME/.kube/config.

If you are rewriting the Keystone support, then I suggest:

• come to the SIG-Auth meeting and discuss what you want to do.
• if you just want to take a token provided in an Authorization: Bearer header and have it be checked by a third-party, then please consider using the Webhook Authenticator instead, which is going to merge soon, and is pretty generic: Webhook Token Authenticator #24902
• you might want to implement a little script that does an openstack issue token and stuffs it into the $HOME/.kube/config for the relevant cluster.

@erictune I think each user will need to be able to specify domainid/domain separately. so either $HOME/.kube/config or pulling it from the user's openstack_rc enviroronment would be better. There already is a lot of tooling around the latter.

I have a patch started... I think I might be able to post it tonight, so maybe you can have a look and let me know what you think?

When are the SIG-Auth meetings? I took a look at the wiki, https://github.com/kubernetes/kubernetes/wiki/Special-Interest-Groups-%28SIGs%29 and the time isn't there.

The webhook method looks interesting. But Keystone's validate token api requires a keystone user for the service to prevent random servers from trying to brute force keys. With the desire to have Magnum managed k8s clusters to be dynamically created and bound back to the cloud's keystone, it may be tricky to get that mechanism to work cleanly. We might need to talk about the option more though. Maybe there's a way to leverage it in a secure way.

Yeah, I was going to do the openstack issue token > $HOME/.kube/config thing as an initial pass. It will work pretty well. But to be consistant with all the other openstack cli's, it would be nice if that work flow could be integrated into kubecli directly.

@erictune, mkumatag code posted here:
#25391

It still needs some work, but is a good start I think.

I've reviewed the code and commented in #25391

Next SigAuth meeting is 5/19. They are Biweekly Wednesdays at 1100 to 1200 PT.

Can you present at that meetinggit a little bit about how OpenStack DomainNames and DomainIDs work, and what you use case is for Kube + Openstack, so that I can understand the context of this issue better? I added an item to the agenda for this:
https://docs.google.com/document/d/1woLGRoONE3EBVx-wTb4pvp4CI7tmLZ6lS26VTbosLKM/edit#

@erictune I think I can make that... but am unclear on the date/time. Wednesday is not 5/19, Thursday is.

Thanks,
Kevin

@erictune I also made a comment here:
#25391 (comment)

That covers a quick keystone 101 kind of thing and what the use cases I think are. Can you have a look and see if that's the sort of thing you would want to see at the SigAuth meeting?

@erictune Can you please update the schedule item to point to this issue:
#24982

The PR associated with it is:
#25391

And a good description of the problem I think, is here:
#25391 (comment)

@mkumatag @kfox1111 Any updates on this?

cc/ @kubernetes/sig-openstack-misc

We need to figure out how authz with openstack's going to work. I still think it should use rbac and be tenant aware, not just have policy/tenancy crammed into groups. Can't be done outside of process though.

The two approaches suggested in this issue isn't a good way to solve it IMO. The server should never get a username/password to give to keystone. only a keystone token.

We need to figure out how authz with openstack's going to work. I still think it should use rbac and be tenant aware, not just have policy/tenancy crammed into groups. Can't be done outside of process though.

Authz is another problem. I think authentication should be solved first.

Keystone's all about authz. If you want just authn, you can just plumb k8s directly into ldap and bypass Keystone. Without considering how Keystone authz works, its easy to implement something that totally doesn't work for any authz case that Keystone was built to handle without having to completely re engineer it. :/

The following PR #39587 is adding a client side auth provider for kubectl that uses Bearer Tokens based on OS_* env variable options. There's a PoC of the Webhook Keystone implementation here https://github.com/dims/k8s-keystone-auth

/close