Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the custom injection on OpenShift #47898

Merged
merged 1 commit into from
Nov 17, 2023
Merged

Fix the custom injection on OpenShift #47898

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions pkg/kube/inject/testdata/inject/proxy-override.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ spec:
securityContext:
allowPrivilegeEscalation: true
readOnlyRootFilesystem: false
# These should not be removed or ignored, should be honored
runAsUser: 1234
runAsGroup: 1234
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why should we remove user's setting

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, this particular change in the test file was to guarantee we are not changing user values. This PR only changes it in a specific case: OpenShift.

volumes:
- name: certs
secret:
Expand Down
6 changes: 3 additions & 3 deletions pkg/kube/inject/testdata/inject/proxy-override.yaml.injected
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ spec:
prometheus.io/path: /stats/prometheus
prometheus.io/port: "15020"
prometheus.io/scrape: "true"
proxy.istio.io/overrides: '{"containers":[{"name":"istio-proxy","resources":{"limits":{"cpu":"3"},"requests":{"cpu":"123m"}},"volumeMounts":[{"name":"certs","mountPath":"/etc/certs"}],"livenessProbe":{"httpGet":{"path":"/healthz/ready","port":15021},"initialDelaySeconds":10,"timeoutSeconds":3,"periodSeconds":2,"failureThreshold":30},"lifecycle":{"preStop":{"exec":{"command":["sleep","10"]}}},"terminationMessagePath":"/foo/bar","securityContext":{"readOnlyRootFilesystem":false,"allowPrivilegeEscalation":true},"tty":true}],"initContainers":[{"name":"istio-init","image":"fake/custom-image","args":["my","custom","args"],"resources":{}}]}'
proxy.istio.io/overrides: '{"containers":[{"name":"istio-proxy","resources":{"limits":{"cpu":"3"},"requests":{"cpu":"123m"}},"volumeMounts":[{"name":"certs","mountPath":"/etc/certs"}],"livenessProbe":{"httpGet":{"path":"/healthz/ready","port":15021},"initialDelaySeconds":10,"timeoutSeconds":3,"periodSeconds":2,"failureThreshold":30},"lifecycle":{"preStop":{"exec":{"command":["sleep","10"]}}},"terminationMessagePath":"/foo/bar","securityContext":{"runAsUser":1234,"runAsGroup":1234,"readOnlyRootFilesystem":false,"allowPrivilegeEscalation":true},"tty":true}],"initContainers":[{"name":"istio-init","image":"fake/custom-image","args":["my","custom","args"],"resources":{}}]}'
sidecar.istio.io/status: '{"initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":["workload-socket","credential-socket","workload-certs","istio-envoy","istio-data","istio-podinfo","istio-token","istiod-ca-cert"],"imagePullSecrets":null,"revision":"default"}'
creationTimestamp: null
labels:
Expand Down Expand Up @@ -146,9 +146,9 @@ spec:
- ALL
privileged: false
readOnlyRootFilesystem: false
runAsGroup: 1337
runAsGroup: 1234
runAsNonRoot: true
runAsUser: 1337
runAsUser: 1234
startupProbe:
failureThreshold: 600
httpGet:
Expand Down
28 changes: 22 additions & 6 deletions pkg/kube/inject/webhook.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ import (
"istio.io/istio/pkg/slices"
"istio.io/istio/pkg/util/protomarshal"
"istio.io/istio/pkg/util/sets"
"istio.io/istio/tools/istio-iptables/pkg/constants"
)

var (
Expand Down Expand Up @@ -527,9 +528,8 @@ func reapplyOverwrittenContainers(finalPod *corev1.Pod, originalPod *corev1.Pod,
continue
}
overlay := *match.DeepCopy()
if overlay.Image == AutoImage {
overlay.Image = ""
}
resetFieldsInAutoImageContainer(&overlay, &c)

overrides.Containers = append(overrides.Containers, overlay)
newMergedPod, err := applyContainer(finalPod, overlay)
if err != nil {
Expand All @@ -549,9 +549,8 @@ func reapplyOverwrittenContainers(finalPod *corev1.Pod, originalPod *corev1.Pod,
continue
}
overlay := *match.DeepCopy()
if overlay.Image == AutoImage {
overlay.Image = ""
}
resetFieldsInAutoImageContainer(&overlay, &c)

overrides.InitContainers = append(overrides.InitContainers, overlay)
newMergedPod, err := applyInitContainer(finalPod, overlay)
if err != nil {
Expand All @@ -575,6 +574,23 @@ func reapplyOverwrittenContainers(finalPod *corev1.Pod, originalPod *corev1.Pod,
return finalPod, nil
}

func resetFieldsInAutoImageContainer(original *corev1.Container, template *corev1.Container) {
if original.Image == AutoImage {
original.Image = ""
}

// If the original pod comes with SecurityContext.RunAsUser and the template defines a value different than the default (1337),
// then ignore the original value and stick with the final (merged one)
// This is likely a scenario in OpenShift when the istio-proxy container with image: auto is parsed, if SecurityContext.RunAsUser
// does not exist, OpenShift automatically assigns a value which is based on an annotation in the namespace. Regardless if the user
// provided that value or if it was assigned by OpenShift, the correct value is the one in the template, as set by the `.ProxyUID` field.
if original.SecurityContext != nil && template.SecurityContext != nil && template.SecurityContext.RunAsUser != nil &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image: auto and securityContext are unrelated. A user can override the proxy container AND set image: some-explicit-proxy-image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's right.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@howardjohn fixed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now although the function name is maybeconfusing

*template.SecurityContext.RunAsUser != constants.DefaultProxyUIDInt {
linsun marked this conversation as resolved.
Show resolved Hide resolved
original.SecurityContext.RunAsUser = nil
original.SecurityContext.RunAsGroup = nil
}
}

// parseStatus extracts containers from injected SidecarStatus annotation
func parseStatus(status string) ParsedContainers {
parsedContainers := ParsedContainers{}
Expand Down
8 changes: 8 additions & 0 deletions releasenotes/notes/fix-custom-injection-openshift.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: release-notes/v2
kind: bug-fix
area: installation
docs:
- 'https://istio.io/latest/docs/setup/additional-setup/sidecar-injection/#customizing-injection'
releaseNotes:
- |
**Fixed** Custom injection of the `istio-proxy` container was not working on OpenShift because of the way OpenShift sets pod's `SecurityContext.RunAs` field.