Enable Kubernetes native support for AWS ECR container registries when using kube-aws #518
Description
TL;DR ECR cross-domain support is coming when k8s 1.3.0 releases and whichever version of kube-aws supports 1.3.0, provided by kube-aws patch #384 and k8s patch #24369. In the meantime you can use imagePullSecrets with a scheduled script like this one:
https://github.com/whereisaaron/kubernetes-aws-scripts/blob/master/create-ecr-imagepullsecret.sh
Kubernetes provides native support for AWS ECR container registries, including cross-region ECR support. The particular ECR features are that kubelets automatically obtains and regularly updates ECR login credentials (AWS ECR login credentials only last 12 hours), and uses those credentials when you specify an ECR repository for your image. Relevant Kurbernetes references below.
http://kubernetes.io/docs/user-guide/images/#using-aws-ec2-container-registry
kubernetes/kubernetes#19447
kubernetes/kubernetes#23298
However, this excellent feature is only activated if the kublet is started with the '--cloud-provider=aws' option and it appears the controller nodes generated by kube-aws do not have that option enabled (using kube-aws 0.7.0), CORRECTION: worker nodes do have this option.
/etc/systemd/system/kubelet.service
[Service]
Environment=KUBELET_VERSION=v1.2.3_coreos.0
Environment=KUBELET_ACI=quay.io/coreos/hyperkube
Environment="RKT_OPTS=--volume dns,kind=host,source=/etc/resolv.conf --mount volume=dns,target=/etc/resolv.conf"
ExecStart=/usr/lib/coreos/kubelet-wrapper \
--api-servers=http://localhost:8080 \
--network-plugin-dir=/etc/kubernetes/cni/net.d \
--network-plugin= \
--register-schedulable=false \
--allow-privileged=true \
--config=/etc/kubernetes/manifests \
--cluster_dns=10.3.0.10 \
--cluster_domain=cluster.local
Restart=always
RestartSec=10
The second requirement for this feature to work is the node AWS IAM roles also need ECR ReadOnly permissions added. The IAM 'AmazonEC2ContainerRegistryReadOnly' managed policy is exactly what is needed. Or the following actions could be added to the kube-aws generated role definition.
arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:BatchGetImage"
],
"Resource": "*"
}]
}