Our docker 1.11.X validation stats (#23397) show overall performance improvements, but lack the performance measurement with docker pull operation. After upgrading to docker 1.11.1, the issue #25784 (comment), #25277 are not observed.

I did experiment against both docker 1.9.1 and docker 1.11.1 on docker pulling, and the overall performance on docker 1.11.1 is way better than 1.9.1, but we should include those tests to our docker performance validation test suite. The below are the steps I did:

IMAGES="gcr.io/google_containers/iperf:e2e 1>/dev/null & time for i in gcr.io/google_containers/busybox gcr.io/google_containers/busybox:1.24 gcr.io/google_containers/dnsutils:e2e gcr.io/google_containers/eptest:0.1 gcr.io/google_containers/fakegitserver:0.1 gcr.io/google_containers/hostexec:1.2 gcr.io/google_containers/iperf:e2e gcr.io/google_containers/jessie-dnsutils:e2e gcr.io/google_containers/liveness:e2e gcr.io/google_containers/mounttest:0.2 gcr.io/google_containers/mounttest:0.5 gcr.io/google_containers/mounttest:0.6 gcr.io/google_containers/mounttest-user:0.3 gcr.io/google_containers/netexec:1.4 gcr.io/google_containers/netexec:1.5 gcr.io/google_containers/nettest:1.7 gcr.io/google_containers/nettest:1.8 gcr.io/google_containers/nginx:1.7.9 gcr.io/google_containers/nginx-slim:0.5 gcr.io/google_containers/n-way-http:1.0 gcr.io/google_containers/pause:2.0 gcr.io/google_containers/pause-amd64:3.0 gcr.io/google_containers/porter:cd5cb5791ebaa8641955f0e8c2a9bed669b1eaab gcr.io/google_containers/portforwardtester:1.0 gcr.io/google_containers/redis:e2e gcr.io/google_containers/resource_consumer:beta2 gcr.io/google_containers/serve_hostname:v1.4 gcr.io/google_containers/servicelb:0.1 gcr.io/google_containers/test-webserver:e2e gcr.io/google_containers/ubuntu:14.04 gcr.io/google_containers/update-demo:kitten gcr.io/google_containers/update-demo:nautilus gcr.io/google_containers/volume-ceph:0.1 gcr.io/google_containers/volume-gluster:0.2 gcr.io/google_containers/volume-iscsi:0.1 gcr.io/google_containers/volume-nfs:0.6 gcr.io/google_containers/volume-rbd:0.1 gcr.io/google_samples/gb-redisslave:v1"

1. time the command: for i in $IMAGES; do echo $(date '+%X') pulling $i; docker pull $i 1>/dev/null; done;

2. See the impact of LIST operation to docker pull

for i in 1..1000; do docker ps -a 1>/dev/null; sleep 2; done & time for i in $IMAGES; do echo $(date '+%X') pulling $i; docker pull $i 1>/dev/null; done;

3. See the impact of CREATION operation to docker pull

for i in {1..1000}; do docker run busybox sleep 1; sleep 1; done & time for i in $IMAGES; do echo $(date '+%X') pulling $i; docker pull $i 1>/dev/null; done;

4. See the impact of DELETE operation to docker pull

for i in {1..1000}; do docker run --rm busybox sleep 1; sleep 1; done & time for i in $IMAGES; do echo $(date '+%X') pulling $i; docker pull $i 1>/dev/null; done;

5. See the impact of concurrent pulling:

IMAGES2=... ===> another set of images with some dup.

for i in $IMAGES2; do docker pull $i 1>/dev/null; done & time for i in $IMAGES; do echo $(date '+%X') pulling $i; docker pull $i 1>/dev/null; done;

@liangchenye Can we include all above tests to the docker performance validation test suite?

cc/ @kubernetes/goog-node @timothysc