Skip to content
This repository has been archived by the owner on Jul 4, 2024. It is now read-only.

Commit

Permalink
Kyma 2.4.3 (#3301)
Browse files Browse the repository at this point in the history 
* successful local install

* change cli version

* fix leftover and patches

* remove unused flag

* try to fix benchmark

* install kyma once

* reuse existing kyma

* update documentation

* add explanation message
  • Loading branch information
@ognyvrac
ognyvrac authored Sep 15, 2023
1 parent 4267c42 commit 2b858fd
Show file tree
Hide file tree
Showing 8 changed files with 63 additions and 136 deletions.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ For more information about the Compass architecture, technical details, and comp
- [Docker](https://www.docker.com/get-started)
- [k3d](https://github.com/k3d-io/k3d) v5.2.2+
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) 1.23.0+
- [Kyma CLI](https://github.com/kyma-project/cli) 2.3.0
- [Kyma CLI](https://github.com/kyma-project/cli) 2.5.0
- [helm](https://github.com/helm/helm) v3.8.0+
- [yq](https://github.com/mikefarah/yq) v4+

Expand Down
6 changes: 3 additions & 3 deletions docs/compass/04-01-installation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Therefore, `serviceAccountTokenJWKS` and `serviceAccountTokenIssuer` need to be

> **NOTE:** During the installation of Compass, the installed Kyma version (as a basis to Compass) must match to the one in the [`KYMA_VERSION`](../../installation/resources/KYMA_VERSION) file in the specific Compass commit.

If custom domains and certificates are needed, see the [Set up your custom domain TLS certificate](https://github.com/kyma-project/kyma/blob/2.3.0/docs/03-tutorials/sec-01-tls-certificates-security.md) document in the Kyma installation guide, as well as the resources in the [Certificate Management](#certificate-management) section in this document.
If custom domains and certificates are needed, see the [Set up your custom domain TLS certificate](https://github.com/kyma-project/kyma/blob/2.4.3/docs/03-tutorials/sec-01-tls-certificates-security.md) document in the Kyma installation guide, as well as the resources in the [Certificate Management](#certificate-management) section in this document.

Save the following .yaml code with installation overrides into a file (for example: additionalKymaOverrides.yaml)
```yaml
Expand Down Expand Up @@ -383,9 +383,9 @@ To install the Compass and Runtime components on a single cluster, perform the f

> **NOTE:** During the installation of Kyma, the installed version must match to the one in the [`KYMA_VERSION`](../../installation/resources/KYMA_VERSION) file in the specific Compass commit.

You must have a Kyma installation with an enabled Runtime Agent. For more information, see [Enable Kyma with Runtime Agent](https://github.com/kyma-project/kyma/blob/2.3.0/docs/04-operation-guides/operations/ra-01-enable-kyma-with-runtime-agent.md). Therefore, you must add the compass-runtime-agent module in the compass-system namespace to the list of [minimal kyma components file](../../installation/resources/kyma/kyma-components-minimal.yaml).
You must have a Kyma installation with an enabled Runtime Agent. For more information, see [Enable Kyma with Runtime Agent](https://github.com/kyma-project/kyma/blob/2.4.3/docs/04-operation-guides/operations/ra-01-enable-kyma-with-runtime-agent.md). Therefore, you must add the compass-runtime-agent module in the compass-system namespace to the list of [minimal kyma components file](../../installation/resources/kyma/kyma-components-minimal.yaml).

If custom domains and certificates are needed, see the [Set up your custom domain TLS certificate](https://github.com/kyma-project/kyma/blob/2.3.0/docs/03-tutorials/sec-01-tls-certificates-security.md) document in the Kyma installation guide, as well as the resources in the [Certificate Management](#certificate-management) section in this document.
If custom domains and certificates are needed, see the [Set up your custom domain TLS certificate](https://github.com/kyma-project/kyma/blob/2.4.3/docs/03-tutorials/sec-01-tls-certificates-security.md) document in the Kyma installation guide, as well as the resources in the [Certificate Management](#certificate-management) section in this document.

Save the following .yaml code with installation overrides to a file (for example: additionalKymaOverrides.yaml)
```yaml
Expand Down
2 changes: 1 addition & 1 deletion installation/resources/KYMA_VERSION
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.3.0
2.4.3
41 changes: 41 additions & 0 deletions installation/resources/kyma/kyma-overrides-minimal.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -124,11 +124,22 @@ istio:
monitoring:
alertmanager:
alertmanagerSpec:
podMetadata:
annotations:
sidecar.istio.io/inject: "true"
resources:
limits:
cpu: 40m
requests:
cpu: 5m
serviceMonitor:
scheme: https
# By default in Kyma, Prometheus injects the Istio certificates from a sidecar
tlsConfig:
caFile: /etc/prometheus/secrets/istio.default/root-cert.pem
certFile: /etc/prometheus/secrets/istio.default/cert-chain.pem
keyFile: /etc/prometheus/secrets/istio.default/key.pem
insecureSkipVerify: true
grafana:
env:
GF_AUTH_ANONYMOUS_ENABLED: true
Expand All @@ -144,10 +155,40 @@ monitoring:
requests:
cpu: 50m
memory: 64Mi
prometheus:
prometheusSpec:
alertingEndpoints:
- apiVersion: v2
name: monitoring-alertmanager
namespace: kyma-system
pathPrefix: /
port: http-web
scheme: https
# By default in Kyma, Prometheus injects the Istio certificates from a sidecar
tlsConfig:
caFile: /etc/prometheus/secrets/istio.default/root-cert.pem
certFile: /etc/prometheus/secrets/istio.default/cert-chain.pem
keyFile: /etc/prometheus/secrets/istio.default/key.pem
insecureSkipVerify: true
prometheus-istio:
server:
resources:
limits:
memory: 1Gi
prometheusOperator:
configReloaderMemory: 256Mi
podAnnotations:
sidecar.istio.io/inject: "true"
prometheus-node-exporter:
prometheus:
monitor:
scheme: https
# By default in Kyma, Prometheus injects the Istio certificates from a sidecar
tlsConfig:
caFile: /etc/prometheus/secrets/istio.default/root-cert.pem
certFile: /etc/prometheus/secrets/istio.default/cert-chain.pem
keyFile: /etc/prometheus/secrets/istio.default/key.pem
insecureSkipVerify: true
kube-state-metrics:
podAnnotations:
sidecar.istio.io/inject: "true"
18 changes: 7 additions & 11 deletions installation/scripts/install-kyma.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,13 @@ cp ${KYMA_OVERRIDES_MINIMAL} ${MINIMAL_OVERRIDES_TEMP}
yq -i ".istio.helmValues.pilot.jwksResolverExtraRootCA = \"$CERT\"" "${MINIMAL_OVERRIDES_TEMP}"

if [[ $(uname -m) == 'arm64' ]]; then
yq -i ".istio.global.images.istio_proxyv2.containerRegistryPath = \"europe-west1-docker.pkg.dev\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_proxyv2.directory = \"sap-cp-cmp-dev/ucl-dev\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_proxyv2.version = \"1.13.2-distroless\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_proxyv2.containerRegistryPath = \"ghcr.io\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_proxyv2.directory = \"resf/istio\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_proxyv2.version = \"1.14.1-distroless\"" "${MINIMAL_OVERRIDES_TEMP}"

yq -i ".istio.global.images.istio_pilot.containerRegistryPath = \"europe-west1-docker.pkg.dev\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_pilot.directory = \"sap-cp-cmp-dev/ucl-dev\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_pilot.version = \"1.13.2-distroless\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_pilot.containerRegistryPath = \"ghcr.io\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_pilot.directory = \"resf/istio\"" "${MINIMAL_OVERRIDES_TEMP}"
yq -i ".istio.global.images.istio_pilot.version = \"1.14.1-distroless\"" "${MINIMAL_OVERRIDES_TEMP}"
fi

trap "rm -f ${MINIMAL_OVERRIDES_TEMP}" EXIT INT TERM
Expand All @@ -42,7 +42,7 @@ KYMA_SOURCE=$(<"${ROOT_PATH}"/installation/resources/KYMA_VERSION)

echo "Using Kyma source ${KYMA_SOURCE}"

# TODO: Remove after adoption of Kyma 2.4.3 and change kyma deploy command source to --source $KYMA_SOURCE
# Reuse Kyma source, otherwise the Kyma source is fetched everytime
KYMA_WORKSPACE=${HOME}/.kyma/sources/${KYMA_SOURCE}
if [[ -d "$KYMA_WORKSPACE" ]]
then
Expand All @@ -52,9 +52,5 @@ else
git clone --single-branch --branch "${KYMA_SOURCE}" https://github.com/kyma-project/kyma.git "$KYMA_WORKSPACE"
fi

rm -rf "$KYMA_WORKSPACE"/installation/resources/crds/service-catalog || true
rm -f "$KYMA_WORKSPACE"/installation/resources/crds/service-catalog-addons/clusteraddonsconfigurations.addons.crd.yaml || true
rm -f "$KYMA_WORKSPACE"/installation/resources/crds/service-catalog-addons/addonsconfigurations.addons.crd.yaml || true

echo "Installing minimal Kyma"
kyma deploy --components-file $KYMA_COMPONENTS_MINIMAL --values-file $MINIMAL_OVERRIDES_TEMP --source=local --workspace "$KYMA_WORKSPACE"
97 changes: 1 addition & 96 deletions installation/scripts/prom-mtls-patch.sh
View file
Original file line number Diff line number Diff line change
@@ -1,103 +1,9 @@
function prometheusMTLSPatch() {
patchPrometheusForMTLS
patchAlertManagerForMTLS
enableNodeExporterMTLS
patchDeploymentsToInjectSidecar
patchKymaServiceMonitorsForMTLS
removeKymaPeerAuthsForPrometheus
}

function patchPrometheusForMTLS() {
patch=$(cat <<"EOF"
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
name: monitoring-prometheus
namespace: kyma-system
spec:
alerting:
alertmanagers:
- apiVersion: v2
name: monitoring-alertmanager
namespace: kyma-system
pathPrefix: /
port: http-web
scheme: https
tlsConfig:
caFile: /etc/prometheus/secrets/istio.default/root-cert.pem
certFile: /etc/prometheus/secrets/istio.default/cert-chain.pem
keyFile: /etc/prometheus/secrets/istio.default/key.pem
insecureSkipVerify: true
podMetadata:
annotations:
sidecar.istio.io/inject: "true"
traffic.sidecar.istio.io/includeInboundPorts: "" # do not intercept any inbound ports
traffic.sidecar.istio.io/includeOutboundIPRanges: "" # do not intercept any outbound traffic
proxy.istio.io/config: |
# configure an env variable OUTPUT_CERTS to write certificates to the given folder
proxyMetadata:
OUTPUT_CERTS: /etc/istio-output-certs
sidecar.istio.io/userVolumeMount: '[{"name": "istio-certs", "mountPath": "/etc/istio-output-certs"}]' # mount the shared volume at sidecar proxy
volumes:
- emptyDir:
medium: Memory
name: istio-certs
volumeMounts:
- mountPath: /etc/prometheus/secrets/istio.default/
name: istio-certs
EOF
)

echo "${patch}" > patch.yaml
kubectl apply -f patch.yaml
rm patch.yaml
}

function patchAlertManagerForMTLS() {
patch=$(cat <<"EOF"
apiVersion: monitoring.coreos.com/v1
kind: Alertmanager
metadata:
name: monitoring-alertmanager
namespace: kyma-system
spec:
podMetadata:
annotations:
sidecar.istio.io/inject: "true"
EOF
)

echo "${patch}" > patch.yaml
kubectl apply -f patch.yaml
rm patch.yaml
}

function patchDeploymentsToInjectSidecar() {
allDeploy=(
monitoring-kube-state-metrics
monitoring-operator
)

resource="deployment"
namespace="kyma-system"

for depl in "${allDeploy[@]}"; do
if kubectl get ${resource} -n ${namespace} "${depl}" > /dev/null; then
kubectl get ${resource} -n ${namespace} "${depl}" -o yaml > "${depl}.yaml"

if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' -e 's/sidecar.istio.io\/inject: "false"/sidecar.istio.io\/inject: "true"/g' "${depl}.yaml"
else # assume Linux otherwise
sed -i 's/sidecar.istio.io\/inject: "false"/sidecar.istio.io\/inject: "true"/g' "${depl}.yaml"
fi

kubectl apply -f "${depl}.yaml" || true

rm "${depl}.yaml"
fi
done
}

function enableNodeExporterMTLS() {
# The patches around the DaemonSet involve an addition of two init containers that together setup certificates
# for the node-exporter application to use. There are also two new mounts - a shared directory (node-certs)
Expand Down Expand Up @@ -255,11 +161,10 @@ EOF
}

function patchKymaServiceMonitorsForMTLS() {
# Some of the ServiceMonitor MTLS overrides were moved to the Kyma Helm chart overrides
kymaSvcMonitors=(
monitoring-alertmanager
monitoring-kube-state-metrics
monitoring-operator
monitoring-prometheus-node-exporter
monitoring-prometheus-pushgateway
ory-stack-oathkeeper-maester
)
Expand Down
29 changes: 6 additions & 23 deletions installation/scripts/prow/jobs/compass-gke-benchmark.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ function installHelm() {
}

function installKymaCLI() {
KYMA_CLI_VERSION="2.3.0"
KYMA_CLI_VERSION="2.5.0"
log::info "Installing Kyma CLI version: $KYMA_CLI_VERSION"

PREV_WD=$(pwd)
Expand All @@ -176,23 +176,9 @@ function installKymaCLI() {

function installKyma() {
KYMA_VERSION=$(<"${COMPASS_SOURCES_DIR}/installation/resources/KYMA_VERSION")

# TODO: Remove after adoption of Kyma 2.4.3 and change kyma deploy command source to --source="${KYMA_VERSION}"
KYMA_WORKSPACE=${HOME}/.kyma/sources/${KYMA_VERSION}
if [[ -d "$KYMA_WORKSPACE" ]]
then
echo "Kyma ${KYMA_VERSION} already exists locally."
else
echo "Pulling Kyma ${KYMA_VERSION}"
git clone --single-branch --branch "${KYMA_VERSION}" https://github.com/kyma-project/kyma.git "$KYMA_WORKSPACE"
fi

rm -rf "$KYMA_WORKSPACE"/installation/resources/crds/service-catalog || true
rm -f "$KYMA_WORKSPACE"/installation/resources/crds/service-catalog-addons/clusteraddonsconfigurations.addons.crd.yaml || true
rm -f "$KYMA_WORKSPACE"/installation/resources/crds/service-catalog-addons/addonsconfigurations.addons.crd.yaml || true

MINIMAL_KYMA="${COMPASS_SOURCES_DIR}/installation/resources/kyma/kyma-components-minimal.yaml"
kyma deploy --ci --source=local --workspace "$KYMA_WORKSPACE" --verbose -c "${MINIMAL_KYMA}" --values-file "$PWD/kyma_overrides.yaml"

kyma deploy --ci --source="${KYMA_VERSION}" --verbose -c "${MINIMAL_KYMA}" --values-file "$PWD/kyma_overrides.yaml"
}

function installOry() {
Expand Down Expand Up @@ -228,9 +214,6 @@ function installCompassOld() {
COMPASS_OVERRIDES="$PWD/compass_benchmark_overrides.yaml"
COMPASS_COMMON_OVERRIDES="$PWD/compass_common_overrides.yaml"

echo 'Installing Kyma'
installKyma

echo "Installing Ory"
installOry

Expand Down Expand Up @@ -266,9 +249,6 @@ function installCompassNew() {
COMPASS_OVERRIDES="$PWD/compass_benchmark_overrides.yaml"
COMPASS_COMMON_OVERRIDES="$PWD/compass_common_overrides.yaml"

echo 'Installing Kyma'
installKyma

echo "Installing Ory"
installOry

Expand Down Expand Up @@ -325,6 +305,9 @@ installHelm
log::info "Install Kyma CLI"
installKymaCLI

log::info "Installing Kyma"
installKyma

NEW_VERSION_COMMIT_ID=$(cd "$COMPASS_SOURCES_DIR" && git rev-parse --short HEAD)
log::info "Install Compass version from main"
installCompassOld
Expand Down
4 changes: 3 additions & 1 deletion installation/scripts/prow/jobs/provision-compass.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,10 @@ cp yq "$HOME/bin/yq" && cp yq "/usr/local/bin/yq"
log::info "Successfully installed yq version: $YQ_VERSION"

# Install Kyma to be later used in run.sh
KYMA_CLI_VERSION="2.5.0"
log::info "Installing Kyma CLI version: $KYMA_CLI_VERSION"
KYMA_CLI_VERSION=$(cat ${COMPASS_SOURCE_DIR}/installation/resources/KYMA_VERSION)
# TODO: Kyma 2.4.3 release exists, but Kyma CLI 2.4.3 does not
# KYMA_CLI_VERSION=$(cat ${COMPASS_SOURCE_DIR}/installation/resources/KYMA_VERSION)

curl -Lo kyma.tar.gz "https://github.com/kyma-project/cli/releases/download/${KYMA_CLI_VERSION}/kyma_Linux_x86_64.tar.gz" \
&& mkdir kyma-release && tar -C kyma-release -zxvf kyma.tar.gz && chmod +x kyma-release/kyma && mv kyma-release/kyma /usr/local/bin \
Expand Down

0 comments on commit 2b858fd

Please sign in to comment.