Skip to content

Commit

Permalink
Fix issue for debug containers when using custom Docker registry (lin…
Browse files Browse the repository at this point in the history
…kerd#3873)

**Subject**
Fixes bug where override of Docker registry was not being applied to debug containers (linkerd#3851)

**Problem**
Overrides for Docker registry are not being applied to debug containers and provide no means to correct the image.

**Solution**
This update expands the `data.proxy` configuration section within the Linkerd `ConfigMap` to maintain the overridden image name for debug containers at _install_-time similar to handling of the `proxy` and `proxyInit` images.

This change also enables the further override option of the registry for debug containers at _inject_-time given utilization of the `--registry` CLI option.

**Validation**
Several new unit tests have been created to confirm functionality.  In addition, the following workflows were run through:

### Standard Workflow with Custom Registry
This workflow installs Linkerd control plane based upon a custom registry, then injecting the debug sidecar into a service.

* Start with a k8s instance having no Linkerd installation
* Build all images locally using `bin/docker-build`
* Create custom tags (using same version) for generated images, e.g. `docker tag gcr.io/linkerd-io/debug:git-a4ebecb6 javaducky.com/linkerd-io/debug:git-a4ebecb6`
* Install Linkerd with registry override `bin/linkerd install --registry=javaducky.com/linkerd-io | kubectl apply -f -`
* Once Linkerd has been fully initialized, you should be able to confirm that the `linkerd-config` ConfigMap now contains the debug image name, pull policy, and version within the `data.proxy` section
* Request injection of the debug image into an available container.  I used the Emojivoto voting service as described in https://linkerd.io/2/tasks/using-the-debug-container/ as `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar - | kubectl apply -f -`
* Once the deployment creates a new pod for the service, inspection should show that the container now includes the "linkerd-debug" container name based on the applicable override image seen previously within the ConfigMap
* Debugging can also be verified by viewing debug container logs as `kubectl -n emojivoto logs deploy/voting linkerd-debug -f`
* Modifying the `config.linkerd.io/enable-debug-sidecar` annotation, setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Overriding the Custom Registry Override at Injection
This builds upon the “Standard Workflow with Custom Registry” by overriding the Docker registry utilized for the debug container at the time of injection.

* “Clean” the Emojivoto voting service by removing any Linkerd annotations from the deployment
* Request injection similar to before, except provide the `--registry` option as in `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar --registry=gcr.io/linkerd-io - | kubectl apply -f -`
* Inspection of the deployment config should now show the override annotation for `config.linkerd.io/debug-image` having the debug container from the new registry.  Viewing the running pod should show that the `linkerd-debug` container was injected and running the correct image.  Of note, the proxy and proxy-init images are still running the “original” override images.
* As before, modifying the `config.linkerd.io/enable-debug-sidecar` annotation setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Standard Workflow with Default Registry
This workflow is the typical workflow which utilizes the standard Linkerd image registry.

* Uninstall the Linkerd control plane using `bin/linkerd install --ignore-cluster | kubectl delete -f -` as described at https://linkerd.io/2/tasks/uninstall/
* Clean the Emojivoto environment using `curl -sL https://run.linkerd.io/emojivoto.yml | kubectl delete -f -` then reinstall using `curl -sL https://run.linkerd.io/emojivoto.yml | kubectl apply -f -`
* Perform standard Linkerd installation as `bin/linkerd install | kubectl apply -f -`
* Once Linkerd has been fully initialized, you should be able to confirm that the `linkerd-config` ConfigMap references the default debug image of `gcr.io/linkerd-io/debug` within the `data.proxy` section
* Request injection of the debug image into an available container as `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar - | kubectl apply -f -`
* Debugging can also be verified by viewing debug container logs as `kubectl -n emojivoto logs deploy/voting linkerd-debug -f`
* Modifying the `config.linkerd.io/enable-debug-sidecar` annotation, setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Overriding the Default Registry at Injection
This workflow builds upon the “Standard Workflow with Default Registry” by overriding the Docker registry utilized for the debug container at the time of injection.

* “Clean” the Emojivoto voting service by removing any Linkerd annotations from the deployment
* Request injection similar to before, except provide the `--registry` option as in `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar --registry=javaducky.com/linkerd-io - | kubectl apply -f -`
* Inspection of the deployment config should now show the override annotation for `config.linkerd.io/debug-image` having the debug container from the new registry.  Viewing the running pod should show that the `linkerd-debug` container was injected and running the correct image.  Of note, the proxy and proxy-init images are still running the “original” override images.
* As before, modifying the `config.linkerd.io/enable-debug-sidecar` annotation setting to “false”, should show that the pod will be recreated no longer running the debug container.

Fixes issue linkerd#3851 

Signed-off-by: Paul Balogh javaducky@gmail.com
  • Loading branch information
javaducky authored and adleong committed Jan 17, 2020
1 parent e30b9a9 commit dabee12
Show file tree
Hide file tree
Showing 47 changed files with 3,931 additions and 142 deletions.
7 changes: 5 additions & 2 deletions charts/linkerd2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,9 @@ The following table lists the configurable parameters of the Linkerd2 chart and
| `controllerReplicas` | Number of replicas for each control plane pod | `1` |
| `controllerUID` | User ID for the control plane components | `2103` |
| `dashboard.replicas` | Number of replicas of dashboard | `1` |
| `debugContainer.image.name` | Docker image for the debug container | `gcr.io/linkerd-io/debug` |
| `debugContainer.image.pullPolicy` | Pull policy for the debug container Docker image | `IfNotPresent` |
| `debugContainer.image.version` | Tag for the debug container Docker image | latest version |
| `disableHeartBeat` | Set to true to not start the heartbeat cronjob | `false` |
| `enableH2Upgrade` | Allow proxies to perform transparent HTTP/2 upgrading | `true` |
| `global.clusterDomain` | Kubernetes DNS Domain name to use | `cluster.local` |
Expand Down Expand Up @@ -122,7 +125,7 @@ The following table lists the configurable parameters of the Linkerd2 chart and
| `global.proxyInit.ignoreOutboundPorts` | Outbound ports the proxy should ignore ||
| `global.proxyInit.image.name` | Docker image for the proxy-init container | `gcr.io/linkerd-io/proxy-init` |
| `global.proxyInit.image.pullPolicy` | Pull policy for the proxy-init container Docker image | `IfNotPresent` |
| `global.proxyInit.image.version` | Tag for the proxy-init container Docker image | `v1.1.0` |
| `global.proxyInit.image.version` | Tag for the proxy-init container Docker image | latest version |
| `global.proxyInit.resources.cpu.limit` | Maximum amount of CPU units that the proxy-init container can use | `100m` |
| `global.proxyInit.resources.cpu.request` | Amount of CPU units that the proxy-init container requests | `10m` |
| `global.ProxyInit.resources.memory.limit` | Maximum amount of memory that the proxy-init container can use | `50Mi` |
Expand All @@ -148,7 +151,7 @@ The following table lists the configurable parameters of the Linkerd2 chart and
| `profileValidator.keyPEM` | Certificate key for the service profile validator. If not provided then Helm will generate one. ||
| `tap.crtPEM` | Certificate for the Tap component. If not provided then Helm will generate one. ||
| `tap.keyPEM` | Certificate key for Tap component. If not provided then Helm will generate one. ||
| `webhookFailurePolicy` | Failure policy for the proxy injector | `Ignore`
| `webhookFailurePolicy` | Failure policy for the proxy injector | `Ignore` |
| `webImage` | Docker image for the web container | `gcr.io/linkerd-io/web` |

## Get involved
Expand Down
7 changes: 6 additions & 1 deletion charts/linkerd2/templates/_config.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,12 @@
},
"disableExternalProfiles": {{not .Values.global.proxy.enableExternalProfiles}},
"proxyVersion": "{{.Values.global.proxy.image.version}}",
"proxyInitImageVersion": "{{.Values.global.proxyInit.image.version}}"
"proxyInitImageVersion": "{{.Values.global.proxyInit.image.version}}",
"debugImage":{
"imageName":"{{.Values.debugContainer.image.name}}",
"pullPolicy":"{{.Values.debugContainer.image.pullPolicy}}"
},
"debugImageVersion": "{{.Values.debugContainer.image.version}}"
}
{{- end -}}

Expand Down
7 changes: 7 additions & 0 deletions charts/linkerd2/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,13 @@ controllerUID: 2103
dashboard:
replicas: 1

# debug configuration
debugContainer:
image:
name: gcr.io/linkerd-io/debug
pullPolicy: *image_pull_policy
version: *linkerd_version

# identity configuration
identity:
issuer:
Expand Down
12 changes: 12 additions & 0 deletions cli/cmd/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,18 @@ func generateAnnotationsDocs() []annotationDoc {
Name: k8s.ProxyInitImageVersionAnnotation,
Description: "Linkerd init container image version",
},
{
Name: k8s.DebugImageAnnotation,
Description: "Linkerd debug container image name",
},
{
Name: k8s.DebugImageVersionAnnotation,
Description: "Linkerd debug container image version",
},
{
Name: k8s.DebugImagePullPolicyAnnotation,
Description: "Docker image pull policy for debug image",
},
{
Name: k8s.ProxyControlPortAnnotation,
Description: "Proxy port to use for control",
Expand Down
29 changes: 25 additions & 4 deletions cli/cmd/inject.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ const (
injectDisabledDesc = "pods are not annotated to disable injection"
unsupportedDesc = "at least one resource injected"
udpDesc = "pod specs do not include UDP ports"
slash = "/"
)

type resourceTransformerInject struct {
Expand Down Expand Up @@ -367,10 +368,13 @@ func (options *proxyConfigOptions) overrideConfigs(configs *cfg.All, overrideAnn
}

if options.dockerRegistry != "" {
configs.Proxy.ProxyImage.ImageName = registryOverride(configs.Proxy.ProxyImage.ImageName, options.dockerRegistry)
configs.Proxy.ProxyInitImage.ImageName = registryOverride(configs.Proxy.ProxyInitImage.ImageName, options.dockerRegistry)
overrideAnnotations[k8s.ProxyImageAnnotation] = configs.Proxy.ProxyImage.ImageName
overrideAnnotations[k8s.ProxyInitImageAnnotation] = configs.Proxy.ProxyInitImage.ImageName
debugImage := configs.GetProxy().GetDebugImage().GetImageName()
if debugImage == "" {
debugImage = k8s.DebugSidecarImage
}
overrideAnnotations[k8s.ProxyImageAnnotation] = overwriteRegistry(configs.GetProxy().GetProxyImage().GetImageName(), options.dockerRegistry)
overrideAnnotations[k8s.ProxyInitImageAnnotation] = overwriteRegistry(configs.GetProxy().GetProxyInitImage().GetImageName(), options.dockerRegistry)
overrideAnnotations[k8s.DebugImageAnnotation] = overwriteRegistry(debugImage, options.dockerRegistry)
}

if options.proxyImage != "" {
Expand All @@ -392,6 +396,7 @@ func (options *proxyConfigOptions) overrideConfigs(configs *cfg.All, overrideAnn
if options.imagePullPolicy != "" {
configs.Proxy.ProxyImage.PullPolicy = options.imagePullPolicy
configs.Proxy.ProxyInitImage.PullPolicy = options.imagePullPolicy
configs.Proxy.DebugImage.PullPolicy = options.imagePullPolicy
overrideAnnotations[k8s.ProxyImagePullPolicyAnnotation] = options.imagePullPolicy
}

Expand Down Expand Up @@ -479,3 +484,19 @@ func parsePortRanges(portRanges []*cfg.PortRange) string {

return strings.TrimSuffix(str, ",")
}

// overwriteRegistry replaces the registry-portion of the provided image with the provided registry.
func overwriteRegistry(image, newRegistry string) string {
if image == "" {
return image
}
registry := newRegistry
if registry != "" && !strings.HasSuffix(registry, slash) {
registry += slash
}
imageName := image
if strings.Contains(image, slash) {
imageName = image[strings.LastIndex(image, slash)+1:]
}
return registry + imageName
}
Loading

0 comments on commit dabee12

Please sign in to comment.