Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Secret & ConfigMap keys to contain caps, dots, and underscores #25458

Merged
merged 1 commit into from
Aug 3, 2016

Conversation

errm
Copy link
Contributor

@errm errm commented May 11, 2016

Analytics

Re: #23722

This makes loosens the regex used in in Secrets and ConfigMap,
in order to make environment variable style keys valid

@k8s-bot
Copy link

k8s-bot commented May 11, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

2 similar comments
@k8s-bot
Copy link

k8s-bot commented May 11, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

@k8s-bot
Copy link

k8s-bot commented May 11, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

@k8s-github-robot k8s-github-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. release-note-label-needed labels May 11, 2016
@@ -2340,12 +2340,15 @@ func ValidateServiceAccountUpdate(newServiceAccount, oldServiceAccount *api.Serv
return allErrs
}

const SecretKeyFmt string = "\\.?" + validation.DNS1123LabelFmt + "(\\." + validation.DNS1123LabelFmt + ")*"
const DNSSecretKeyFmt string = "\\.?" + validation.DNS1123LabelFmt + "(\\." + validation.DNS1123LabelFmt + ")*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you defined regexes, but I don't see where you actually use them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They get combined with a | on line 2345

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohk yeah they didn't need to be const

@thockin
Copy link
Member

thockin commented May 11, 2016

Rather than allowing DNS names or env var names, can we just loosen the regex to [-A-Za-z0-9.]*[A-Za-z0-9][-A-Za-z0-9.]* or something equally loose? @pmorie? This will affect "automatic" secret file projections, but these are all valid filenames....

@errm
Copy link
Contributor Author

errm commented May 11, 2016

That probably makes sense... we would also need to allow underscores...

@errm
Copy link
Contributor Author

errm commented May 11, 2016

@thockin I have pushed something based on the suggestion, its a little more complex than you suggested as there are some tests checking that we don't allow keys with double dots through. I am not sure why that would be an issue though...

@pmorie
Copy link
Member

pmorie commented May 11, 2016

Is there a reason that you need the keys to be in a different format? We are adding control over the path and filename within the secret volume, and you have to specify an env var name if you consume a secret as an env var.

See #25285

@errm
Copy link
Contributor Author

errm commented May 12, 2016

In #23722 the usecase was that it is harder to build tooling that is targeting environment variables if everything has to have two names.

Clearly it is possible to work around this by encoding the names somehow. But I don't know of a good reason why the keys should be limited this much?

I just think it would be a good thing to make the interface simple to folks building tooling on bash and such. At the moment in my tool the we are hex encoding keys where they are env var names, but that has the downside that they get longer and are totally unreadable by someone using kubectl or whatnot.

@thockin
Copy link
Member

thockin commented May 12, 2016

I merely found it annoying an without actual benefit that I couldn't use the same names as env vars. Specifically I was taking a doc that someone had written about what env vars their app needed and putting it in a config map and I had to double-name everything. foo-bar-bat and FOO_BAR_BAT. Blech. Is there a reason? I can't think of one.

@liyimeng
Copy link
Contributor

liyimeng commented Jun 3, 2016

What is the progress of this? I cherry-pick the PR and test, underscore still not a valid key for ConfigMap.

@errm
Copy link
Contributor Author

errm commented Jun 6, 2016

Thanks for trying this @liyimeng, I will take a look at this later today...

@liyimeng
Copy link
Contributor

liyimeng commented Jun 7, 2016

Thanks @errm! Here is how I test:

[liyi@K8s-builder ~]$ kubectl create configmap ssh-config --from-file /etc/kolla/nova-ssh/id_rsa.pub
error: id_rsa.pub is not a valid key name for a configMap

@liyimeng
Copy link
Contributor

liyimeng commented Jun 7, 2016

sorry @errm, that was a fault alarm. Totally my fault, I forget to update kubectl in my test node. The validation is actually also done in kubectl.

@pmorie
Copy link
Member

pmorie commented Jun 7, 2016

@liyimeng It won't be in for 1.3.

I would really prefer to see a proposal for changing the key format, since I think it will be a little more complicated than 'allow underscores'.

@errm
Copy link
Contributor Author

errm commented Jun 8, 2016

@pmorie I can create a proposal for this... It just seemed like such a small change.

I do see the point of view that once we loosen some validation we can never tighten it up again, so I guess it should be something to have a wider discussion about...

Is there some documentation on how to write a proposal properly somewhere?

@k8s-bot
Copy link

k8s-bot commented Jun 15, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

@cgag
Copy link

cgag commented Jun 16, 2016

I'm trying to create a configmap for a Caddyfile (https://caddyserver.com/) and getting burned by the fact that the standard container images expect the name "Caddyfile". Config files starting with a capital letter seems somewhat common (I can think of Gemfile and Cargo.toml off the top of my head).

Is there any reason this supports dns names and env vars rather than linux file names, as @thockin mentioned?

@errm
Copy link
Contributor Author

errm commented Jun 17, 2016

@cgag It does support file names . . . just not keys in the configMap that are non dns. When you add the map to a container you are able to map to file names.

@errm
Copy link
Contributor Author

errm commented Jun 17, 2016

@thockin is there any way I can help to move this forward?

@cgag
Copy link

cgag commented Jun 17, 2016

@errm, Woops, you're right, I see now that I can mount the keys to custom paths, thanks. I'd gotten used to just using the keys as the filenames.

Still, it'd be nice to not have to do that, so it'd be cool to see this go forward.

@k8s-bot
Copy link

k8s-bot commented Jun 26, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

@k8s-github-robot k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 7, 2016
@errm errm force-pushed the env-var-style-config-keys branch from e77d113 to ddf3b65 Compare July 11, 2016 12:33
@k8s-github-robot k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 11, 2016
This makes environment variable style keys (uppercase with underscores) valid
in Secrets and ConfigMap.
@errm errm force-pushed the env-var-style-config-keys branch from e5f73e3 to d4969ff Compare August 1, 2016 19:56
@errm
Copy link
Contributor Author

errm commented Aug 1, 2016

@thokin nits squashed . . .

@pmorie
Copy link
Member

pmorie commented Aug 1, 2016

@thockin @errm

I have to say, I still don't understand what value this provides -- you have to specify the name of the key in the environment specification anyway.

@@ -6300,7 +6300,7 @@ func TestValidateConfigMap(t *testing.T) {
invalidName = newConfigMap("NoUppercaseOrSpecialCharsLike=Equals", "validns", nil)
emptyNs = newConfigMap("validname", "", nil)
invalidNs = newConfigMap("validname", "NoUppercaseOrSpecialCharsLike=Equals", nil)
invalidKey = newConfigMap("validname", "validns", map[string]string{"a..b": "value"})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think that two dots next to one another should still be invalid.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? What value does it provide or what does it protect against?

On Mon, Aug 1, 2016 at 2:39 PM, Paul Morie notifications@github.com wrote:

In pkg/api/validation/validation_test.go
#25458 (comment)
:

@@ -6300,7 +6300,7 @@ func TestValidateConfigMap(t *testing.T) {
invalidName = newConfigMap("NoUppercaseOrSpecialCharsLike=Equals", "validns", nil)
emptyNs = newConfigMap("validname", "", nil)
invalidNs = newConfigMap("validname", "NoUppercaseOrSpecialCharsLike=Equals", nil)

  •   invalidKey       = newConfigMap("validname", "validns", map[string]string{"a..b": "value"})
    

i think that two dots next to one another should still be invalid.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/kubernetes/pull/25458/files/d4969ff03230e61fcd18fef778debd83e896ed6a#r73057977,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFVgVFYXhORgQ3rm51RN1dSuuwcTYTlCks5qbmd8gaJpZM4IbwmI
.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I talked myself out of it, carry on

@thockin
Copy link
Member

thockin commented Aug 1, 2016

For simple auto-expansion into file projections, I can now express a much
wider array of file names without needing to write out the individual
projections.

Next, expect someone to suggest a streamlined form of env vars that don't
require you to spell the name twice. Just watch.

On Mon, Aug 1, 2016 at 2:38 PM, Paul Morie notifications@github.com wrote:

@thockin https://github.com/thockin @errm https://github.com/errm

I have to say, I still don't understand what value this provides -- you
have to specify the name of the key in the environment specification
anyway.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#25458 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFVgVMnyuZu_3HPoXoU576pkqm1ucfiSks5qbmdNgaJpZM4IbwmI
.

@errm
Copy link
Contributor Author

errm commented Aug 2, 2016

Yes ultimately I would like to be able to say project all the keys from this configMap and/or secret to environment variables.

In my usecase:
We already build a map or secret per project & environment that contains the env vars we need.

Then we need to loop over each key and build a valid config for the env section.

This at the moment involves looking at each variable we want, then mapping it's name to something valid.

At the moment if we start with an environment variable name, say FOO_BAR we then need to translate it to something like foo-bar. This is fine if we are doing this by hand (if a little bit of a pain). But if I wan't to do this automatically we can't really make a nice translation, because what happens if someone decides they want the environment variable foo-bar to be set as well. The solution we are using now is to pack the keys as hex, so the keys in our secrets look like 464f4f5f424152. This works fine, but breaks looking at things in kubectl (and being able to reed them), and just makes the whole process a little more complex and hard to use/understand.

@k8s-bot
Copy link

k8s-bot commented Aug 2, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

1 similar comment
@k8s-bot
Copy link

k8s-bot commented Aug 2, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

@pmorie
Copy link
Member

pmorie commented Aug 2, 2016

@thockin

Next, expect someone to suggest a streamlined form of env vars that don't require you to spell the name twice. Just watch.

I'm expecting it -- already thinking about how I want the API to look :)

@pmorie pmorie self-assigned this Aug 2, 2016
@pmorie pmorie added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 2, 2016
@pmorie
Copy link
Member

pmorie commented Aug 2, 2016

@errm LGTM. Sorry for the latency on this, I've been out a lot recently and have been seriously underwater / bankrupt on github notifications.

@k8s-bot
Copy link

k8s-bot commented Aug 2, 2016

GCE e2e build/test passed for commit d4969ff.

@errm errm changed the title Allow Secret & ConfigMap keys to hold caps, dots, and underscores Allow Secret & ConfigMap keys to contain caps, dots, and underscores Aug 3, 2016
@pmorie
Copy link
Member

pmorie commented Aug 3, 2016

Thanks for the patch!

@k8s-github-robot
Copy link

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

@fejta
Copy link
Contributor

fejta commented Aug 3, 2016

@k8s-bot e2e test this issue #29966

@k8s-bot
Copy link

k8s-bot commented Aug 3, 2016

GCE e2e build/test passed for commit d4969ff.

@k8s-github-robot
Copy link

Automatic merge from submit-queue

@k8s-github-robot k8s-github-robot merged commit 1edf254 into kubernetes:master Aug 3, 2016
@k8s-bot
Copy link

k8s-bot commented Aug 3, 2016

GCE e2e build/test passed for commit d4969ff.

@errm errm deleted the env-var-style-config-keys branch August 5, 2016 07:25
@kayrus
Copy link
Contributor

kayrus commented Aug 12, 2016

@pmorie could you please promote this patch into the v1.3.6 release?

@thockin
Copy link
Member

thockin commented Aug 12, 2016

This doesn't meet the bar for a patch release to 1.3, I think

On Aug 12, 2016 6:57 AM, "kayrus" notifications@github.com wrote:

@pmorie https://github.com/pmorie could you please promote this patch
into the v1.3.6 release?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#25458 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFVgVE0TM15SZS3ov9F8W8o5H76X4_NGks5qfHvDgaJpZM4IbwmI
.

@errm
Copy link
Contributor Author

errm commented Aug 15, 2016

Agreed, while this certainly shouldn't break anything currently using secrets or configMap, it is certainly a change to API and should wait for the next minor version.

openshift-publish-robot pushed a commit to openshift/kubernetes that referenced this pull request Sep 11, 2020
Bug 1874251: UPSTREAM: 94204: Add impersonated user to system:authenticated group

Origin-commit: b0068a868c4996f2a5c2a35c1bb54d700545da34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.