[WIP] Add support for Config{} to kubecfg #987

mfojtik · 2014-08-21T14:47:24Z

This PR will allow users to use simple JSON file with arrays of Kubernetes objects (services, pods, replicationControllers) as a parameter for kubecfg -c config.json apply command.

The example JSON file in examples/config/guestbook.json contains all objects required for deploying the guestbook example.

mfojtik · 2014-08-21T14:48:57Z

@brendanburns @smarterclayton fyi.

smarterclayton · 2014-08-21T15:09:38Z

cmd/kubecfg/kubecfg.go

+	}
+
+	obj := api.Config{}
+	if err := gojson.Unmarshal(data, &obj); err != nil {


Should be yaml parsing to be consistent.

smarterclayton · 2014-08-21T15:35:41Z

This is to serve as a root for a conversation about how groups of resources in the Kubernetes system can be created, updated, and maintained over time. The simplest possible manifestation is "create a bunch of API objects from a static definition". Higher levels of abstraction (templatizing, generating config, and how config can be transformed in reaction to other changes in the system) ultimately must all be transformed into concrete API definitions.

smarterclayton · 2014-08-21T15:35:49Z

@bgrant0607 as well

lavalamp · 2014-08-21T17:14:53Z

pkg/api/types.go

@@ -478,3 +478,9 @@ type WatchEvent struct {
 type APIObject struct {
 	Object interface{}
 }
+
+type Config struct {
+	Services               []Service               `yaml:"services,omitempty" json:"services,omitempty"`


We have an api.XList object already, for all of these. Might make sense to use it.

bgrant0607 · 2014-08-21T17:29:10Z

Meta comment: These kinds of changes are hard to reverse engineer later. This MUST be accompanied by documentation, probably in docs/cli.md.

verbs: apply doesn't make sense to me. I'd like existing kubecfg verbs to work on the objects described in the file: create, update, even delete.

selectivity: It doesn't have to be in the initial version, but I'd like to be able to selectively apply the verbs, say to just replicationControllers (so, by type), or to just 1 object, by name.

file structure: 1 array per object type isn't going to fly long-term. It's not composable. I'd like to be able to decompose objects into 1 file per microservice and then just concatenate them together.

api endpoint specification: Related to file structure, but worth calling out. As discussed in #635, we need to think about the API surface as not being monolithic. The APIs may be served by different services eventually, and certainly we'd like to support multiple versions -- a user might copy a redis example using v1 of the API and may write their own php config using v2. Therefore, we need to be able to specify API endpoints in the file, with defaults. Again, this doesn't have to be in the initial version, but we need to keep it in mind.

creation-order dependencies: For the most part, we've tried to avoid the need for a particular creation order, due to loose coupling via label selectors. However, I imagine this may change eventually. Creation in sequential order is possible, but ultimately limiting. Anything else, however, requires object cross references. I'd recommend that we just create all objects in parallel for now.

code structure: Not specific to this PR, but I'd eventually like kubecfg to evolve into a simple command wrapper around a client library that could be used independently of the command, to facilitate programmatic use of the k8s API.

I'm going to be starting on a more comprehensive config proposal today.

lavalamp · 2014-08-21T17:39:31Z

cmd/kubecfg/kubecfg.go

+		createObject(c, pod)
+	}
+	for _, replicationController := range obj.ReplicationControllers {
+		createObject(c, replicationController)


We really need to think about ordering for something like this. Some pods may depend on some services. Can the pod handle it if it starts before its dependencies? Clearly the right answer is "yes it should", but only the human setting up the system knows this.

I suppose it's probably fine to say that this only works for pods where start order doesn't matter. The human can always sort their services/pods into batches and start each batch with this method. But we should explicitly say this somewhere.

This is also related to health-checking. How do you know that the service is 'ready'? For example the official mysql Docker image, when started it will first start the mysql to create users/init database and then it will stop and start to run the mysql daemon.

Other option is to assume that containers/images are resilient to service failure and so it will be the container responsibility to poll the services.

I think for now it is probably sufficient to start the services first (as Brian suggests below). Starting the service will cause the appropriate environment variables to get set, even if the service isn't live yet; users of the service would have to retry if it's not up yet.

smarterclayton · 2014-08-21T17:40:00Z

verbs: apply doesn't make sense to me. I'd like existing kubecfg verbs to work on the objects described in the file: create, update, even delete.

So in your config you'd say "create / update / delete" (objects described in the file)? So a config is a description of operations, rather than declarative? Or are you just saying that you'd create/update/delete a config, with that operation globally applicable, and then have filters on the CLI that would allow targeting specific types (or labels)?

smarterclayton · 2014-08-21T17:41:08Z

api endpoint specification: Related to file structure, but worth calling out. As discussed in #635, we need to think about the API surface as not being monolithic. The APIs may be served by different services eventually, and certainly we'd like to support multiple versions -- a user might copy a redis example using v1 of the API and may write their own php config using v2. Therefore, we need to be able to specify API endpoints in the file, with defaults. Again, this doesn't have to be in the initial version, but we need to keep it in mind.

We'd actually probably like to be able to do this sooner rather than later so we can contribute objects that aren't in the core api service (a short term object for us).

lavalamp · 2014-08-21T17:54:24Z

@bgrant0607:

file structure: 1 array per object type isn't going to fly long-term. It's not composable. I'd like to be able to decompose objects into 1 file per microservice and then just concatenate them together.
... Therefore, we need to be able to specify API endpoints in the file, with defaults.

Yeah, absolutely- the APIObjectList approach I suggested is probably the right direction for this. Version is encoded in the api object directly that way, so there's not a need to specify the api endpoint. (I'm thinking that plugin apis will register themselves such that apiserver acts as a proxy (or redirects) so there is a single, easily discoverable location for each api endpoint.

code structure: Not specific to this PR, but I'd eventually like kubecfg to evolve into a simple command wrapper around a client library that could be used independently of the command, to facilitate programmatic use of the k8s API.

+1; we already have a kubecfg package and a client package. As much of this PR as possible should be pushed into those packages, probably the kubecfg package. The client package is pretty nice apart from the monolithic interface (but is kinda low level), kubecfg package is not quite as nice but we can evolve it to make it more useful.

smarterclayton · 2014-08-21T17:58:15Z

(I'm thinking that plugin apis will register themselves such that apiserver acts as a proxy (or redirects) so there is a single, easily discoverable location for each api endpoint.

As a simplest step we can assume that the specification of that is outside the config and then it's either a client concern (explicitly list endpoints) or a proxy concern (the global api proxy)

bgrant0607 · 2014-08-21T18:34:05Z

@smarterclayton

you'd create/update/delete a config, with that operation globally applicable, and then have filters on the CLI that would allow targeting specific types (or labels)?

Yes, this is what I meant.

bgrant0607 · 2014-08-21T18:41:29Z

@lavalamp Re. ordering: See my comment re. dependencies. Yes, some pods will depend on services. Ideally, if they crashed due to the service environment variables not being set they could restart until they saw valid values. One problem I see with this is how the environment variables are propagated to containers. It might or might not work now, but would not work if we were to take advantage of restarts in Docker. One simple thing we could do is to create all services first, before other types of objects. We could hardcode this behavior initially, but it would be nice for the creation order to be configurable by object type. For example, if there were image builds, we'd want to do all the builds first.

lavalamp · 2014-08-21T18:50:00Z

@bgrant0607 Yeah, agree that we need to start all the services first. This may not be sufficient in the future but it's simple and will work for a while.

thockin · 2014-08-24T01:18:47Z

I'm 100% in support of the idea, but I am not sure about the details of it. This is one of those things that I think is better discussed in the abstract before quibbling over code.

smarterclayton · 2014-08-25T17:08:59Z

Discussion then in #1007

brendandburns · 2014-08-25T23:24:46Z

cmd/kubecfg/kubecfg.go

@@ -29,6 +29,8 @@ import (
 	"text/template"
 	"time"

+	goyaml "gopkg.in/v1/yaml"


no need to name this nothing else is called yaml, afaik.

lavalamp · 2014-08-26T19:40:43Z

Closing this for now. Please reopen when discussion in #1007 winds down!!

re-order the import package

add recycling for old iptables rules

mfojtik changed the title ~~Add support for Config{} to kubecfg~~ [WIP] Add support for Config{} to kubecfg Aug 21, 2014

smarterclayton reviewed Aug 21, 2014
View reviewed changes

Initial implementation of Config

a84d6dc

mfojtik force-pushed the config branch from 8665234 to a84d6dc Compare August 21, 2014 15:38

lavalamp reviewed Aug 21, 2014
View reviewed changes

smarterclayton mentioned this pull request Aug 24, 2014

Proposal: Kubernetes Build Plugin #994

Closed

brendandburns reviewed Aug 25, 2014
View reviewed changes

lavalamp closed this Aug 26, 2014

smarterclayton mentioned this pull request Oct 20, 2014

Kubectl can apply config files in bulk #1905

Closed

bgrant0607 added the area/app-lifecycle label Oct 21, 2014

mfojtik deleted the config branch October 22, 2014 12:19

vishh pushed a commit to vishh/kubernetes that referenced this pull request Apr 6, 2016

Merge pull request kubernetes#987 from carmark/import

61246be

re-order the import package

b3atlesfan pushed a commit to b3atlesfan/kubernetes that referenced this pull request Feb 5, 2021

Merge pull request kubernetes#987 from mainred/recycle_old_iptables

d043a57

add recycling for old iptables rules

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Add support for Config{} to kubecfg #987

[WIP] Add support for Config{} to kubecfg #987

mfojtik commented Aug 21, 2014

mfojtik commented Aug 21, 2014

smarterclayton Aug 21, 2014

smarterclayton commented Aug 21, 2014

smarterclayton commented Aug 21, 2014

lavalamp Aug 21, 2014

bgrant0607 commented Aug 21, 2014

lavalamp Aug 21, 2014

mfojtik Aug 22, 2014

lavalamp Aug 22, 2014

smarterclayton commented Aug 21, 2014

smarterclayton commented Aug 21, 2014

lavalamp commented Aug 21, 2014

smarterclayton commented Aug 21, 2014

bgrant0607 commented Aug 21, 2014

bgrant0607 commented Aug 21, 2014

lavalamp commented Aug 21, 2014

thockin commented Aug 24, 2014

smarterclayton commented Aug 25, 2014

brendandburns Aug 25, 2014

lavalamp commented Aug 26, 2014

[WIP] Add support for Config{} to kubecfg #987

[WIP] Add support for Config{} to kubecfg #987

Conversation

mfojtik commented Aug 21, 2014

mfojtik commented Aug 21, 2014

smarterclayton Aug 21, 2014

Choose a reason for hiding this comment

smarterclayton commented Aug 21, 2014

smarterclayton commented Aug 21, 2014

lavalamp Aug 21, 2014

Choose a reason for hiding this comment

bgrant0607 commented Aug 21, 2014

lavalamp Aug 21, 2014

Choose a reason for hiding this comment

mfojtik Aug 22, 2014

Choose a reason for hiding this comment

lavalamp Aug 22, 2014

Choose a reason for hiding this comment

smarterclayton commented Aug 21, 2014

smarterclayton commented Aug 21, 2014

lavalamp commented Aug 21, 2014

smarterclayton commented Aug 21, 2014

bgrant0607 commented Aug 21, 2014

bgrant0607 commented Aug 21, 2014

lavalamp commented Aug 21, 2014

thockin commented Aug 24, 2014

smarterclayton commented Aug 25, 2014

brendandburns Aug 25, 2014

Choose a reason for hiding this comment

lavalamp commented Aug 26, 2014