This demo requires a Kubernetes cluster at version 1.1 or newer as it uses iptables proxy support that was released in Kubernetes 1.1.

You can see what version you are running by running:

kubectl version

Try it on a 1.0 cluster at your own risk.

To scale to one million QPS you need a number of cores. A 200 node cluster of n1-standard-4 VMs in Google Container Engine (or equivalent) (800 cores) should be sufficient.

Alternately, you can still run the demo at smaller scales, you just might not be able to hit one million requests / second.

Then you need to clone the repository:

git clone https://github.com/kubernetes/contrib
cd contrib/scale-demo

Finally, if you are using a 1.1.x cluster, you will need to activate IPTables based proxying (IPTable proxying is on by default in 1.2.x):

for node in $(kubectl get nodes -o name | cut -f2 -d/); do
    # switch to iptables mode
    kubectl annotate node $node net.beta.kubernetes.io/proxy-mode=iptables;
    # restart the proxies
    # This command is specific to GCE/GKE, other platforms may need different SSH commands
    gcloud compute ssh --zone=us-central1-b $node \
    	   --command="sudo /etc/init.d/kube-proxy restart";
done

In the scale-nginx directory, run the bootstrap script:

./bootstrap-demo.sh

This will create an initial nginx server, a single loadbot and an aggregator to aggregate data. It will also create some Services to link the pieces together.

We'll use kubectl proxy to launch the UI:

kubectl proxy --www=${PWD}/www/

Then visit http://localhost:8001/static/index.html.

First scale up nginx to handle approximately the load you expect to see. In my experiments a 1 core nginx server could handle approximately 10 thousand requests per second. So to handle one million we will scale up to 100 replicas.

kubectl scale rc nginx --replicas=100

Each loadbot applies one thousands requests per second, so to get to one million requests per second run one thousand loadbots.

kubectl scale rc vegeta --replicas=1000

Once you have reached one million requests per second, you can perform a rolling update:

kubectl rolling-update nginx --image=gcr.io/google_containers/nginx-scale:0.3

The rolling update takes quite a while to complete. If you want to run it faster (though slightly more risky, use the --update-period=1s to decrease the time between container updates to one second.

Once you are happy, tear down the load infrastructure with:

./teardown-demo.sh