Unit, integration and GCE e2e build/test failed for commit 3a87238ab4a87a06d577614a7df160084bb79a16.

• Build Log
• Test Artifacts
• Internal Jenkins Results

Description, proposal, issue?

cc @kubernetes/kubectl

Relevant issues updated. I don't think this really warrants a proposal, but I can write one if you really want.

hold on the review of this, it's missing parameters, and that's really the most important part...

I was just looking for a TL;DR about WTF this was about. Discussed in person, but a brief explanation would help whomever this review gets assigned to.

Also discussing parameter syntax here: #4822 (comment)

Why not use the same syntax we use for replacement already in pod env vars? Or just use an actual template language instead of inventing one?

Like our JSONPath or golang formats

Unit, integration and GCE e2e build/test failed for commit c2f45f3b27d4933fab09c44cc370a80332bc5d9a.

• Build Log
• Test Artifacts
• Internal Jenkins Results

@smarterclayton
I don't think I'm inventing a new one. {{ foo }} is used by Angular.js amongst others.

We could use jsonpath, or golang templates but you still need some sort of delimeter (e.g. {{)

I don't think we want to use { since it will barf on JSON, so {{ seems a natural fit.

I'd be ok with $ (which is what we're using in pod env vars, right? which matches up with PHP and others.

Concretely there are two things to specify:

• Delimeters in the template that indicate that this is a value to be replaced (e.g. {{ or ${)
• The reference inside these delimeters which identifies the value to use for replacement (e.g foo)

For now, I chose {{ for a delimiter since it matches Angular, Salt/Jinja, but I could be convinced of others.

For references, I don't see the need for the complexity of anything other than flat names, but I could be convinced to use jsonpath instead.

I updated the description with a concrete example, please take another look.

Thanks
--brendan

Is that valid jsonpath if the root object is a map? Or does jsonpath require the leading .?

for jsonpath the '.' is required in theory, but we cheat:

https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/custom_column_printer.go#L50

for better usability. I can do that here as well, if you really want me to.

I'm just trying to avoid having lots of formats. Someone who gets to
this is probably familiar with go templates and jsonpath as used
elsewhere. So if we do end up making this more flexible, we need to either
have a clear answer about switching from the simple format to or the ability to tell them it's already
supported.

On Oct 2, 2015, at 3:37 PM, Brendan Burns notifications@github.com wrote:

for jsonpath the '.' is required in theory, but we cheat:

https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/custom_column_printer.go#L50

for better usability. I can do that here as well, if you really want me to.

—
Reply to this email directly or view it on GitHub
#14918 (comment)
.

I guess I think that we use this type of thing in two different ways.

we use jsonpath to select things in an object (e.g. show a column of .foo.bar values in a list of objects)

but this is really about identifying a place in a template that can be replaced, so there's no notion of selection or querying at all.

And I think this will be really familiar to people who are used to Angular or Jinja or a variety of other template systems, and it's pretty compatible with the existing jsonpath stuff too.

So hypothetically what happens if I have an arg that is an array. Or is a
map of key value pairs? Or going further, a template to a generator which
is based on another object?

On Oct 2, 2015, at 4:14 PM, Brendan Burns notifications@github.com wrote:

I guess I think that we use this type of thing in two different ways.

we use jsonpath to select things in an object (e.g. show a column of
.foo.bar values in a list of objects)

but this is really about identifying a place in a template that can be
replaced, so there's no notion of selection or querying at all.

And I think this will be really familiar to people who are used to Angular
or Jinja or a variety of other template systems, and it's pretty compatible
with the existing jsonpath stuff too.

—
Reply to this email directly or view it on GitHub
#14918 (comment)
.

Concretely, the interface to a generator is:

Generate(params map[string]interface{})

So even if you have an arg that is an array, you would still only have a single key:

...
  containers:
    - command: {{ command }}
...

but you would expect that params["command"] returns []string

--brendan

fwiw, the syntax proposed is what I would expect for templates in a data-format-used-as-config language, especially yaml. For whatever reason jinja-style substitutions have been almost solely what I've encountered over the past 3 years (specifically with salt, django and a couple of other private uses of jinja2).

I know this is a bit of a drive by, but I was following this issue as a workaround for my use of volumes requiring a 1:1 rc to pod mapping and couldn't resist providing a data point.

So map[port]port is impossible? Labels? Annotations? Selectors that
aren't strings? Trying to think of other concrete things I'd want that
would trip up a user.

Note that Openshift templates are just pure string replacement on json
string fields as well (we don't even allow arrays) and have worked fine for
us for almost a year now, but I do want to flush out the assumptions of how
this would be used. We chose ${} because of bash, but I don't think that's
any more common than mustache style.

Ultimately I just don't want to explain to users why this isn't X
templates, or go on to extend the format to something that becomes a
beautiful snowflake. As the simplest possible thing, pure name
substitution is at least comprehensible. If it can be justified in terms
of familiarity to at least one popular framework and isn't confusion
inducing with our other formats, I can live with it.

I will ask though will someone now expect -o to leverage this format?

On Oct 2, 2015, at 4:45 PM, Brendan Burns notifications@github.com wrote:

Concretely, the interface to a generator is:

Generate(params map[string]interface{})

So even if you have an arg that is an array, you would still only have a
single key:

...
containers:
- command: {{ command }}
...

but you would expect that params["command"] returns []string

--brendan

—
Reply to this email directly or view it on GitHub
#14918 (comment)
.

So these templates are also pure string substitution, and I see no reason to change that now or in the future.

Also, these templates are expected to be Kubernetes API objects, so they should print just fine. (or am I mis-interpreting your -o comment?)

And no, I violently oppose extending this syntax beyond string substituion. Take a look at Google Deployment Manager for how I think we will move to more complex templating.

--brendan

cc @vaikas @jackgr

Note that elsewhere in the API, we've settled on $() substitution syntax.

Yet another flavor of templating is being discussed here: #18016

A Template parameter example: #4210 (comment)

Yet more parameterization approaches can be seen in #17345

As mentioned elsewhere, curly braces completely break YAML parsing.

@bgrant0607 substitution happens before YAML parsing. Jinja has {{ all over various YAML files, as does deployment manager.

I would like this more if it reflected some of our problems with internal compiled-in generators:

• please let me know what parameters are required
• please let me know what each parameter means with a simple description

This gives us compatibility with what we have with the compiled in code, and makes it no worse.

• please let me store it in a server so web consoles can consume it, and not skew

I think this ends up looking more like what we have in openshift templates when you reach that point.

Have you thought about how you would represent kubectl run?

Right now, string substitution is invalid for replica count because its a number.

I guess ports have similar issue.

I think we need to allow for substitution of anything we allow in our our API conventions as base primitives, so that means base64 (needed for secrets), int (replica size), string, bool (persistent volume claims have a readonly bool, security context, etc.).. we dont need maps or arrays since i think keys and values are expressed as strings...

Is it fair to posit that if the redis example cannot be done with a generator that creates a secret to secure redis, the solution is non-viable?

@derekwaynecarr I'm not sure that's the right bar. I think the experience I want is:

kubectl create -f my-app.yaml --file=my-code.jar

and that creates a simple replicated service.

Secrets are advanced, I think we can make them work too, but its definitely not required.

@brendandburns DM input files don't contain {{}}. Only the Jinja templates contain them. DM input files are straight YAML:

imports:
- path: wordpress.jinja

resources:
- name: wordpress
  type: wordpress.jinja

+1 to @derekwaynecarr's comments. This doesn't satisfy its proposed objectives if users need to read and fully understand the templates in order to use them. There needs to be some way to declare/extract what parameters are needed and to document what they mean. With built-in commands, we have command and flag documentation. The user experience for this would be much worse without something equivalent.

Additionally, I'm concerned about the number of quirky edge cases this would add to the user experience compared to existing commands.

• YAML won't parse with curly braces (yes, I understand that you expect to perform substitution first, but it's inconvenient for validation and other preprocessing steps)
• Introduces a new substitution syntax
  • could switch to existing $() expansion syntax
• Most existing commands support multiple objects
  • should add support for this
• References to templates that can't be automatically fetched/found are problematic for http, directory, and file arguments to kubectl, and compromise the "one binary" convenience argument
  • should search input file and http paths for templates -- which also means we should use a different file extension for templates
  • or be able to fetch them from apiserver, such as via ThirdPartyResource
• This approach is not powerful enough to replace kubectl run -- no default values, no decision logic. I'm not suggesting that we add decision logic, but it shows that there will always be a place for generator code. Default values are more debatable. Openshift Templates support them.

See also the template proposal: #18215

In general, it would be a good idea to avoid baking any kind of markup, mustaches or otherwise, into Kubernetes. Who knows what better or more popular solutions may arise?

For example, what would we do if we baked in mustaches only to find that the Docker Compose markup format has become a must have feature in order to win customer business?

Before this family of PRs goes anywhere else, I suggest stepping back and writing a proposal.

@brendandburns - I think securing your database is not an advanced use case. If we end up with a template construct that allows you to define/package a large number of useful software libraries that are running unsecured or contrary to those packages best-practices, we didn't help the user, we just made it easy to write poor examples. For that reason, I think secret and config data are key concepts that should be supported in a template paradigm since those are the things I will most often parameterize in a template associated with a software library.

see: #17456

For example, what would we do if we baked in mustaches only to find that the Docker Compose markup format has become a must have feature in order to win customer business?

k8s has already adopted substitution markup in the form of "$(key)".

And if another format comes along to dominate the space, then k8s will support for that format, or conversion tools that convert from it will be written. I don't see how "something better might come along" logically leads to "therefore we should not do anything useful now"

and of course nothing prevents the development of third party client tools that consume and expand any format that might come along, again with the possibility that k8s could embrace that format via first class support if it has desirable traits.

GCE e2e build/test failed for commit e6a488f.

• Build Log
• Test Artifacts
• Internal Jenkins Results

Can we revisit this whole idea now that we have ConfigMap? That one addition makes the template concept a whole lot more possible.

Closing in favor of https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/templates.md

The feature of "kubectl run --generator=template.yaml --name=foo --image=nginx --port=8080" is still available.