• Could this revamp include removing deprecated ports (e.g. etcd:4001)
• Does this include plans to accomodate environments that do not set $private_ipv4 ? We are currently using the @kelseyhightower 's setup-network-environment to get $DEFAULT_IPV4 but due to limitations of variable substitution the configuration cannot be the same as an environment that can use $private_ipv4
• Is there an alternate approach to DNS instead of write_files to /etc/kubernetes/manifests and then have a service to ExecStart=-/bin/kubectl create -f /etc/kubernetes/manifests?
• "Unless someone is going to use hyperkube" - Why wouldn't someone use hyperkube?
• Are containerized components a consideration or is that not the preferred implementation? Create Docker images for master components  #6326 The upgrade path of Kubernetes components is easier to manage if they are in containers.

From a starting point of a working CoreOS cluster which includes etcd and flannel, it would be a small step to run containers for kubernetes and import manifests. The current situation of cross-machine waiting scripts is fragile.

Looking forward to testing and providing feedback.

Could this revamp include removing deprecated ports (e.g. etcd:4001)

Yes.

Does this include plans to accomodate environments that do not set $private_ipv4 ?

This is a tricky one since it depends on how you provision your VM. For now, I think it's safe to keep current behavior. Do you have a better suggestion?

Is there an alternate approach to DNS?

The idea is to deploy DNS as per Kubernetes DNS add-on. Other than that, I've never used the manifests thing.

Why wouldn't someone use hyperkube?

My personal take on this is that Kubernetes is a distributed framework for distributed apps. Running it on a single machine may eventually make sense for development purposes, but not for staging or production.
Anyway, maintaining a single-node installation is at this moment a thing I'm not willing to do. Just like the AWS CloudFront template. Are you?

Are containerized components a consideration or is that not the preferred implementation?

Good question. I've never tried containerized components before but I'd say that for CoreOS it won't matter much, since everything is a systemd unit. So, updating versions is as easy with binaries as it is with containers. Even if you follow the container approach, you still need to update the units, reload and restart. Debugging may be trickier because it happens inside the container so no systemctl status -l service.

The current situation of cross-machine waiting scripts is fragile.

My question then is, how would you do it with run containers for kubernetes and import manifests?

@hobti01 final question. Would you take on this? I'm just doing it because I want to help, but right now it's become quite a burden eheh, and you seem much more interested about this than I am.

/cc @bakins

+1

+1
But I guess @errordeveloper and @squillace opinions matter more than mine for this.

@chanezon Heh. If I'm being asked to comment on the below, then +1.

1. Care if we remove single-node instructions? I don't think this makes sense anymore, unless someone is going to use hyperkube.
2. master.yaml needs instrumentation to generate service-account private key file, so it will depend on openssl. Any objections?
3. We should add DNS integration as many examples depend on it. Objections?

@chanezon @squillace @errordeveloper what I need from you guys, the maintainers of CoreOS on Azure, is to work with me or anyone else to get things tested before opening PR. I don't have an Azure account or experience with it, so that's why.

@pires I can test on azure, and I can get you a test account on azure if you would like to test as well. I'm unlikely to be able to test, however, until Monday. But I'm happy to do it.

Removing docs/getting-started-guides/coreos/coreos_single_node_cluster.md sounds good to me. There are other options for people who just want to kick the tires quickly.

Yes, add DNS integration. It is basically required.

DNS is required in pretty much every way.

On Wed, Jun 3, 2015 at 5:49 PM, Eric Tune notifications@github.com wrote:

Removing docs/getting-started-guides/coreos/coreos_single_node_cluster.md
sounds good to me. There are other options for people who just want to kick
the tires quickly.

Yes, add DNS integration. It is basically required.

—
Reply to this email directly or view it on GitHub
#9178 (comment)
.

Also, can we sync effort to get it homogeneous on all different CoreOS approaches?

@pires it's a great goal, however there should a well-defined extend to which we should go. For example, the Azure example uses this JS wrapper around the CLI which does a number of things like generating SSH config file and some other state files, as well as generating cloud-config manifests from template in a kind of nice way (I find).

I think the best we could get to would be around systemd unit names and paths of binaries, as well as other cosmetic things like etcd port that had been mentioned. After that, we could also start reducing duplication of unit definitions in some way.

I agree with @errordeveloper that there should be a common setup. Each example could complete or extend the common templates for environment specific requirements.

In order to achieve a template-based approach for any CoreOS install, is it sensible to use @crawford 's two-phase cloud-init approach? https://gist.github.com/crawford/ed7e954bfddd3394c775
This would provide an extensible place to determine the values for variables/place-holders (i.e. $private_ipv4, dns-domain, etc.) and would work on any CoreOS machine.

@pires Until our lawyers accept the CLA I cannot contribute directly, unfortunately.

@errordeveloper I'm only concerned about reproduce the following in all set-ups:

• Service-account private key file generation (openssl genrsa -out kube-serviceaccount.key 2048 2>/dev/null)
• DNS integration

Of course, how this is achieved will depend on each provider own tools (awscli, gcloud, etc.).
I'd like to have scripts that would automate this in AWS and GCE as well, but that's just too much for me right now. So, the most I can do is to provide cloud-configs that are capable or providing everything one needs - or close to that. I've got a clear idea of what we need, and am just looking at how to provide the DNS functionality.

I agree we should aim for something simpler and cleaner now and work towards a common base in the future, like launching fleet units instead of having them hard-coded into the cloud-configs.

Just want to point out that a fully automated setup is great, but there also needs to be corresponding documentation on manual setup for those with custom environments (or who do not trust third party scripts).

@bkeroackdsc while this is an official repo, a lot of the scripts you see here were/are created and maintained by people outside of Google, RedHat and all other Kubernetes official supporters. So good luck with that!

@pires we might consider a document for this on azure.com. I won't have headspace to consider what's necessary until next week, but I'm happy to respond then.

@squillace there's time before 1.0 so one week time is more than good. @AntonioMeireles will probably get on this as well.

1 and 2 are done by #10344.

Add support for root CA, #10264 (by @liggitt)

@pires Do you need help on this?

@kelseyhightower definitely! Thank you.
On Jul 14, 2015 5:05 PM, "Kelsey Hightower" notifications@github.com
wrote:

@pires https://github.com/pires Do you need help on this?

—
Reply to this email directly or view it on GitHub
#9178 (comment)
.

Can we update the config to support 1.0.1 too?

@feelobot as fair as I'm concerned it's just a matter of updating the versions on both cloud-config files, master.yaml and node.yaml.

@pires I think some of the api options that are passed no longer exist in 1.0.1 but I'm not sure.

@feelobot from 0.21.x to 1.x no flags have changed. At least, the ones used in the current CoreOS documents.

ok ill give it a go with the modified .yml files and if it works ill submit a PR

made the PR #12295

After #13151 is merged, we're only missing DNS integration and API authentication.

Note to anyone interested: some people also want the UI, since this is part of the Kubernetes official release.

DNS is a big deal since it's basically required for service discovery. I was having some issues getting it to work on CoreOS via the official docs pre-1.0. Haven't tried recently.

@bkeroackdsc after #13151 is merged, I'll add DNS integration.

@pires Please don't delete the cloudformation template, I am working on it. Please take a look at #12346.

If it's helpful, I'll have to tweak it a little, but I can provide an alternate CloudFormation intended for launching into an existing VPC, as well.

@mariomarin it has been deleted, so add it once more, if you're able to maintain it in the future.

@jcderr The CloudFormation template requires an existing VPC, that's why we are not using kube-up, because we want to use our current VPC. Please take a look at this and let me know what's the difference with the template you plan to provide.

Ok. From a prior comment, I was under the impression that your CF made a new VPC, but going back and reading the full threads in question, I see you were actually talking about kube-up.

The only non-stylistic difference in my CF is:

• I allow specification of an external etcd cluster
• Since I talk to an external etcd cluster, my master is in an autoscaling group attached to an ELB

These allow my master to be ephemeral. My nodes just point to the hostname of the ELB to access kube-apiserver etc.

@jcderr I like your approach. However, I want to match kube-up as close as possible and it seems to be using static clustering for etcd, right?

Correct. I would say leave it the way you have it, and maybe call out a more robust production suggestion for standalone etcd, similar to how the CoreOS talks about their clustering setups here.

Fixed by #14387