Initial vagrant setup and e2e testing support

duda · Jul 24, 2014 · 69ae2fe · 69ae2fe
1 parent 41eb15b
commit 69ae2fe
Show file tree

Hide file tree

Showing 43 changed files with 1,267 additions and 369 deletions.
diff --git a/.gitignore b/.gitignore
@@ -20,5 +20,8 @@
 **/.hg
 **/.hg*
 
+# Vagrant
+.vagrant
+
 # Version file we automatically make
 /pkg/version/autogenerated.go
diff --git a/README.md b/README.md
@@ -15,7 +15,8 @@ While the concepts and architecture in Kubernetes represent years of experience
 
 ### Contents
 * [Getting started on Google Compute Engine](#getting-started-on-google-compute-engine)
-* [Running a local cluster](#running-locally)
+* [Getting started with a Vagrant cluster on your host](#getting-started-with-a-vagrant-cluster-on-your-host)
+* [Running a local cluster on your host](#running-locally)
 * [Running on CoreOS](#running-on-coreos)
 * [Discussion and Community Support](#community-discussion-and-support)
 * [Hacking on Kubernetes](#development)
@@ -127,6 +128,144 @@ cd kubernetes
 cluster/kube-down.sh
 ```
 
+## Getting started with a Vagrant cluster on your host
+
+### Prerequisites
+1. Install latest version >= 1.6.2 of vagrant from http://www.vagrantup.com/downloads.html
+2. Install latest version of Virtual Box from https://www.virtualbox.org/wiki/Downloads
+3. Get the Kubernetes source:
+
+```
+git clone https://github.com/GoogleCloudPlatform/kubernetes.git
+```
+
+### Setup
+
+By default, the Vagrant setup will create a single kubernetes-master and 3 kubernetes-minions.  You can control the number of minions that are instantiated via an environment variable on your host machine.  If you plan to work with replicas, we strongly encourage you to work with enough minions to satisfy your largest intended replica size.  If you do not plan to work with replicas, you can save some system resources by running with a single minion.
+
+```
+export KUBERNETES_NUM_MINIONS=3
+```
+
+To start your local cluster, open a terminal window and run:
+
+```
+cd kubernetes
+vagrant up
+```
+
+Vagrant will provision each machine in the cluster with all the necessary components to build and run Kubernetes.  The initial setup can take a few minutes to complete on each machine.
+
+By default, each VM in the cluster is running Fedora, and all of the Kubernetes services are installed into systemd.
+
+To access the master or any minion:
+
+```
+vagrant ssh master
+vagrant ssh minion-1
+vagrant ssh minion-2
+vagrant ssh minion-3
+```
+
+To view the service status and/or logs on the kubernetes-master:
+```
+vagrant ssh master
+[vagrant@kubernetes-master ~] $ sudo systemctl status apiserver
+[vagrant@kubernetes-master ~] $ sudo journalctl -r -u apiserver
+
+[vagrant@kubernetes-master ~] $ sudo systemctl status controller-manager
+[vagrant@kubernetes-master ~] $ sudo journalctl -r -u controller-manager
+
+[vagrant@kubernetes-master ~] $ sudo systemctl status etcd
+[vagrant@kubernetes-master ~] $ sudo systemctl status nginx
+```
+
+To view the services on any of the kubernetes-minion(s):
+```
+vagrant ssh minion-1
+[vagrant@kubernetes-minion-1] $ sudo systemctl status docker
+[vagrant@kubernetes-minion-1] $ sudo journalctl -r -u docker
+[vagrant@kubernetes-minion-1] $ sudo systemctl status kubelet
+[vagrant@kubernetes-minion-1] $ sudo journalctl -r -u kubelet
+```
+
+To push updates to new Kubernetes code after making source changes:
+```
+vagrant provision
+```
+
+To shutdown and then restart the cluster:
+```
+vagrant halt
+vagrant up
+```
+
+To destroy the cluster:
+```
+vagrant destroy -f
+```
+
+You can also use the cluster/kube-*.sh scripts to interact with vagrant based providers just like any other hosting platform for kubernetes.
+
+```
+cd kubernetes
+modify cluster/kube-env.sh:
+  KUBERNETES_PROVIDER="vagrant"
+
+cluster/kube-up.sh => brings up a vagrant cluster
+cluster/kube-down.sh => destroys a vagrant cluster
+cluster/kube-push.sh => updates a vagrant cluster
+cluster/kubecfg.sh => interact with the cluster
+```
+
+
+### Running a container
+
+Your cluster is running, and you want to start running containers!
+
+You can now use any of the cluster/kube-*.sh commands to interact with your VM machines.
+```
+cluster/kubecfg.sh list /pods
+cluster/kubecfg.sh list /services
+cluster/kubecfg.sh list /replicationControllers
+cluster/kubecfg.sh -p 8080:80 run dockerfile/nginx 3 myNginx
+
+## begin wait for provision to complete, you can monitor the minions by doing
+  vagrant ssh minion-1
+  sudo docker images
+  ## you should see it pulling the dockerfile/nginx image, once the above command returns it
+  sudo docker ps
+  ## you should see your container running!
+  exit
+## end wait
+
+## back on the host, introspect kubernetes!
+cluster/kubecfg.sh list /pods
+cluster/kubecfg.sh list /services
+cluster/kubecfg.sh list /replicationControllers
+```
+
+Congratulations!
+
+### Testing
+
+The following will run all of the end-to-end testing scenarios assuming you set your environment in cluster/kube-env.sh
+
+```
+hack/e2e-test.sh
+```
+
+
+### Troubleshooting
+
+#### I just created the cluster, but I do not see my container running!
+
+If this is your first time creating the cluster, the kubelet on each minion schedules a number of docker pull requests to fetch prerequisite images.  This can take some time and as a result may delay your initial pod getting provisioned.
+
+#### I changed Kubernetes code, but its not running!
+
+Are you sure there was no build error?  After running $ vagrant provison , scroll up and ensure that each Salt state was completed successfully on each box in the cluster.  Its very likely you see a build error due to an error in your source files!
+
 ## Running locally
 In a separate tab of your terminal, run:
 

diff --git a/Vagrantfile b/Vagrantfile
@@ -0,0 +1,54 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
+VAGRANTFILE_API_VERSION = "2"
+
+# Require a recent version of vagrant otherwise some have reported errors setting host names on boxes
+Vagrant.require_version ">= 1.6.2"
+
+Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
+
+  # The number of minions to provision
+  num_minion = (ENV['KUBERNETES_NUM_MINIONS'] || 3).to_i
+
+  # ip configuration
+  master_ip = "10.245.1.2"
+  minion_ip_base = "10.245.2."
+  minion_ips = num_minion.times.collect { |n| minion_ip_base + "#{n+2}" }
+  minion_ips_str = minion_ips.join(",")
+
+  # Determine the OS platform to use
+  kube_os = ENV['KUBERNETES_OS'] || "fedora"
+
+  # OS platform to box information
+  kube_box = {
+    "fedora" => {
+      "name" => "fedora20",
+      "box_url" => "http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_fedora-20_chef-provisionerless.box"
+    }
+  }
+
+  # Kubernetes master
+  config.vm.define "master" do |config|
+    config.vm.box = kube_box[kube_os]["name"]
+    config.vm.box_url = kube_box[kube_os]["box_url"]
+    config.vm.provision "shell", inline: "/vagrant/cluster/vagrant/provision-master.sh #{master_ip} #{num_minion} #{minion_ips_str}"
+    config.vm.network "private_network", ip: "#{master_ip}"
+    config.vm.hostname = "kubernetes-master"
+  end
+
+  # Kubernetes minion
+  num_minion.times do |n|
+    config.vm.define "minion-#{n+1}" do |minion|
+      minion_index = n+1
+      minion_ip = minion_ips[n]
+      minion.vm.box = kube_box[kube_os]["name"]
+      minion.vm.box_url = kube_box[kube_os]["box_url"]
+      minion.vm.provision "shell", inline: "/vagrant/cluster/vagrant/provision-minion.sh #{master_ip} #{num_minion} #{minion_ips_str} #{minion_ip}"
+      minion.vm.network "private_network", ip: "#{minion_ip}"
+      minion.vm.hostname = "kubernetes-minion-#{minion_index}"
+    end
+  end
+
+end
diff --git a/cluster/config-default.sh → cluster/gce/config-default.sh b/cluster/config-default.sh → cluster/gce/config-default.sh
diff --git a/cluster/config-test.sh → cluster/gce/config-test.sh b/cluster/config-test.sh → cluster/gce/config-test.sh