Deployment generating a service seems weird - someone has to generate a
deployment, and it's trivial to generate a service for an arbitrary pod
template (via expose). The person generating the deployment is something
slightly more complicated than run. We've also found that exposing all
ports on the pod template is often non-desirable - selecting the first port
is 80% successful.

I would suggest having run --port automatically expose a service, leave
expose as is, and not have deployment tightly coupled to services

On Oct 2, 2015, at 2:18 AM, Brian Grant notifications@github.com wrote:

Forked from #1743 #1743.

A number of systems (Borg, Aurora, etc.) use a single config blob to
represent both the set of executing containers and the endpoints exposed to
communicate with them. In Kubernetes, these are currently separate
resources, such as a Service and ReplicationController:
http://kubernetes.io/v1.0/docs/user-guide/managing-deployments.html#organizing-resource-configurations
They can be specified in a single source file, but there are still multiple
concepts, a few lines of boilerplate, and a small amount of redundancy, in
the Service's label selector and port spec.

Going forward, Deployment (#1743
#1743) is the persistent
resource we expect users to create to manage their services, either
imperatively or declaratively. It already generates other resources --
ReplicationControllers. For the common case of a single release track, it
seems reasonable to autogenerate a corresponding Service by default. It
should be optional, so that multiple Deployment resources could be covered
by a single Service without creating unnecessary Service resources.

As Service becomes more complex, I wouldn't want to add all those knobs to
Deployment. I think Deployment should stay simple and it's fine to require
a separate Service for more advanced use cases (e.g., session affinity,
public IPs). We probably at least need to support external load balancers
and maybe nodePort.

cc @smarterclayton https://github.com/smarterclayton @ironcladlou
https://github.com/ironcladlou @nikhiljindal
https://github.com/nikhiljindal @brendandburns
https://github.com/brendandburns

—
Reply to this email directly or view it on GitHub
#14956.

@smarterclayton I don't disagree that it's mixing concepts.

Do you anticipate that most of your users will use kubectl expose, or write out the service YAML, or just instantiate off-the-shelf service YAML?

Alternative: a new resource type that bundles a service and list of deployments.

The bulk of our creations are a generator that is based on an image
(provide 1..N images, templates, or git source repos, and we resolve those
down to images) for which we do a simpler run. During that we generate
image -> pod template -> deployment -> service (if the image defines a port
we use it). If the image doesn't define ports, we display a message that
says "use expose to generate a service". I suspect a simpler podex
equivalent flow would also benefit Kube users similarly.

The next tier (15-20%) of creations are from run, and those users are
usually familiar with expose.

Image and template authors are expected to be familiar with all of the
above concepts and hand craft templates.

In practice, expose makes sense to users once they know it exists. Hand
crafted templates are from sophisticated users and does not seem
excessive. I think some form of template drastically reduces the incidence
of request for "I need a new API object", but does highlight where the API
has gaps (for instance, secrets are hard to templatize today because of the
base64 translation).

We have discussed, but not implemented, the "shard" command that would try
to split a deployment and service. Theoretically, that seems like a common
manual step for new users - I have deployment 1 with selector a=b under
service selector a=b, solve for creating deployment 2..N without disruption.

On Oct 2, 2015, at 8:46 AM, Brian Grant notifications@github.com wrote:

@smarterclayton https://github.com/smarterclayton I don't disagree that
it's mixing concepts.

Do you anticipate that most of your users will use kubectl expose, or write
out the service YAML, or just instantiate off-the-shelf service YAML?

—
Reply to this email directly or view it on GitHub
#14956 (comment)
.

Challenge here - the points of variation between different deployments
under a common service are likely to be subtle (images, selectors, maybe
env, volumes) and I worry the differences will be drowned by the
similarities. Sharding seems like a step you undertake on an existing
deployment, and you can craft a list of {Service, Deployments} for complex
cases as easily as define a new API object.

On Oct 2, 2015, at 9:06 AM, Brian Grant notifications@github.com wrote:

Alternative: a new resource type that bundles a service and list of
deployments.

—
Reply to this email directly or view it on GitHub
#14956 (comment)
.

@smarterclayton I'm not keen on a new resource, either.

srvexpand was pretty trivial to create:
https://github.com/bgrant0607/kubernetes/blob/b8959bf146a8d7bbbd2bbd7fa676f2e3fe711c3a/contrib/srvexpand/README.md

Maybe one of the new generator/templating solutions will help.

Is shard operation more than adding a label to a RC's selector and to its pods? I know you've mentioned this before.

Podex is completely out of date, sadly. We should rewrite it and integrate it into kubectl or server-side.

As for secrets, I'm still waiting for movement on #4822.

I don't think we should discard the idea of "a new resource type that bundles a service and list of deployments" so quickly. IMO it would be a significant usability improvement if we could give users a single object/config that generates all the others under the hood, and that they manipulate to do the common operations like updates. Not everyone is as sophisticated as OpenShift users. The current toolkit/building block approach has a lot of benefits but, even with the kubectl porcelain commands, requires learning a lot of concepts, some of which are very small deltas from one another (e.g. daemon controller, replication controller, job controller) and are more easily thought of as configurations options of a single object than as separate objects.

So OpenShift users are not expected to be sophisticated - that's probably
the exact opposite mentality for the designs we've chosen. Instead, the
generation tools gradually introduce concepts as they are relevant. If run
generated a service and explained it to users (or explained expose) that
reduces the potential for confusion. While creating a new "all-in-one
object" has appeal, it will encourage both clients and tools to ignore
complex edge cases in real use, result in an increasing focus on a single
object (at the expense of enabling composition via effective tooling), and
reduce flexibility.

My suggestion is that kubectl lacks certain niceties that provide the "easy
exposition" flows - generators, and summarizers. OpenShift uses new-app to
generate, and status to summarize (status accumulates the individual
resource types and summarizes their relationships), and templates to allow
complex setups to be delivered to new-app for easy consumption. Having
equivalents in Kube was always our intent - podex is an example of that
(taking info about an image to construct a pod), describe is type specific
summarization, and templates are being discussed.

On Oct 3, 2015, at 2:30 AM, David Oppenheimer notifications@github.com
wrote:

I don't think we should discard the idea of "a new resource type that
bundles a service and list of deployments" so quickly. IMO it would be a
significant usability improvement if we could give users a single
object/config that generates all the others under the hood, and that they
manipulate to do the common operations like updates. Not everyone is as
sophisticated as OpenShift users. The current toolkit/building block
approach has a lot of benefits but, even with the kubectl porcelain
commands, requires learning a lot of concepts, some of which are very small
deltas from one another (e.g. daemon controller, replication controller,
job controller) and are more easily thought of as configurations options of
a single object than as separate objects.

—
Reply to this email directly or view it on GitHub
#14956 (comment)
.

List of deployments is also limiting - what if if have deployments, daemon
sets, and jobs that are all logically related to each other? That's
another reason I'd prefer to build composition tooling rather than an even
higher level composition object (that itself needs composition tooling as
well).

On Oct 3, 2015, at 2:30 AM, David Oppenheimer notifications@github.com
wrote:

I don't think we should discard the idea of "a new resource type that
bundles a service and list of deployments" so quickly. IMO it would be a
significant usability improvement if we could give users a single
object/config that generates all the others under the hood, and that they
manipulate to do the common operations like updates. Not everyone is as
sophisticated as OpenShift users. The current toolkit/building block
approach has a lot of benefits but, even with the kubectl porcelain
commands, requires learning a lot of concepts, some of which are very small
deltas from one another (e.g. daemon controller, replication controller,
job controller) and are more easily thought of as configurations options of
a single object than as separate objects.

—
Reply to this email directly or view it on GitHub
#14956 (comment)
.

One more point - the biggest question we get from new users is how to
expose something to the edge (ingress / route). A new user trying this out
probably wants to see that work as quickly as possible. A generator can
easily solve that, but a combo service/deployment object will still need
that ingress generated. And ultimately, the singleton object isn't enough
to copy your app from one cluster/namespace to another because you may have
additional dependencies like secrets or ingress or persistent volume
claims. That's a job (I would argue) for exporting Templates and
Configuration that you can manually fine tune and move. Or the
Deployment Manager mentioned elsewhere.

On Oct 3, 2015, at 2:30 AM, David Oppenheimer notifications@github.com
wrote:

I don't think we should discard the idea of "a new resource type that
bundles a service and list of deployments" so quickly. IMO it would be a
significant usability improvement if we could give users a single
object/config that generates all the others under the hood, and that they
manipulate to do the common operations like updates. Not everyone is as
sophisticated as OpenShift users. The current toolkit/building block
approach has a lot of benefits but, even with the kubectl porcelain
commands, requires learning a lot of concepts, some of which are very small
deltas from one another (e.g. daemon controller, replication controller,
job controller) and are more easily thought of as configurations options of
a single object than as separate objects.

—
Reply to this email directly or view it on GitHub
#14956 (comment)
.

From past conversations, I believe that @ghodss and @kelseyhightower agree with @smarterclayton.

We do need to put more effort into tooling.

Multiple generators are in flight (e.g., #14918). And we know we need to generalize templates beyond pods (#260). They, or other tooling, may sufficiently address the issue, and I'm not keen on painting us into a corner too quickly.

Actually I have many of the same concerns about configuration as about higher-level API types. We need to try to keep the tooling and automation targeting the lower-level APIs in order to avoid ecosystem fragmentation.

Another thought: I would like to implement some of the more common operations, like expose, server-side: #12143. If we had an expose subresource on Deployment, perhaps we could somehow trigger it declaratively.

Anyway, multiple deployments per service is a more advanced use case and a list of deployments is in the direction of more complexity again, so that alternative is not compelling, IMO.

I think @smarterclayton has probably seen more of this than anyone, but I'm +1 @bgrant0607. For long running jobs, I want a single thing in which I can describe:

• An image
• A number of replicas
• A service endpoint (optional)

I think generators may get us some large percent of the way there from a front end/tooling perspective, but it's not clear to me if we need a more fundamental/first class object for this.

I'd love to see the OpenShift history - is this what you meant? https://developers.openshift.com/en/python-flask.html

cc @krmayankk

See also #18215

We've been discussing leaning more heavily on run in openshift, although we'd probably upgrade the image lookup flow to do pod-ex style things for defaulting. I think having the option / default to do something reasonable on run for exposing a service makes some sense.

I came looking for a workflow for "blue-green" deployments. This sounds similar at first glance.

cc @kow3ns

I no longer think this is something we should do, at least not in the next year, so closing.