Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove ZooKeeper from packaging files #10982

Merged
merged 1 commit into from
Dec 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ After you have deployed the Cluster Operator, use a `Kafka` resource to deploy t

* A Kafka cluster that uses KRaft or ZooKeeper:
** xref:deploying-kafka-cluster-kraft-{context}[KRaft-based Kafka cluster]
** xref:deploying-kafka-cluster-zookeeper-{context}[ZooKeeper-based Kafka cluster]
* xref:deploying-the-topic-operator-using-the-cluster-operator-{context}[Topic Operator]
* xref:deploying-the-user-operator-using-the-cluster-operator-{context}[User Operator]

Expand All @@ -29,8 +28,6 @@ You can also deploy and use other Kafka components with a Kafka cluster not mana

//Deploy Kafka in KRaft mode
include::../../modules/deploying/proc-deploy-kafka-cluster-kraft.adoc[leveloffset=+1]
//Deploy ZooKeeper-based Kafka cluster
include::../../modules/deploying/proc-deploy-kafka-cluster-zookeeper.adoc[leveloffset=+1]
//Include Topic Operator in deployment
include::../../modules/deploying/proc-deploy-topic-operator-with-cluster-operator.adoc[leveloffset=+1]
//Include User Operator in deployment
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ metrics
│ ├── strimzi-kafka.json
| ├── strimzi-kraft.json
│ ├── strimzi-operators.json
│ └── strimzi-zookeeper.json
├── grafana-install
│ └── grafana.yaml <2>
├── prometheus-additional-properties
Expand Down
16 changes: 7 additions & 9 deletions documentation/modules/configuring/con-config-examples.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -34,20 +34,18 @@ examples
├── mirror-maker <4>
├── metrics <5>
├── kafka <6>
│ └── kraft <7>
├── cruise-control <8>
├── connect <9>
└── bridge <10>
├── cruise-control <7>
├── connect <8>
└── bridge <9>
--
<1> `KafkaUser` custom resource configuration, which is managed by the User Operator.
<2> `KafkaTopic` custom resource configuration, which is managed by Topic Operator.
<3> Authentication and authorization configuration for Kafka components. Includes example configuration for TLS and SCRAM-SHA-512 authentication. The Keycloak example includes `Kafka` custom resource configuration and a Keycloak realm specification. You can use the example to try Keycloak authorization services. There is also an example with enabled `oauth` authentication and `keycloak` authorization metrics.
<4> `KafkaMirrorMaker2` custom resource configurations for a deployment of MirrorMaker 2. Includes example configuration for replication policy and synchronization frequency.
<5> xref:assembly-metrics-config-files-{context}[Metrics configuration], including Prometheus installation and Grafana dashboard files.
<6> `Kafka` and `KafkaNodePool` custom resource configurations for a deployment of Kafka clusters that use ZooKeeper mode. Includes example configuration for an ephemeral or persistent single or multi-node deployment.
<7> `Kafka` and `KafkaNodePool` configurations for a deployment of Kafka clusters that use KRaft (Kafka Raft metadata) mode.
<8> `Kafka` and `KafkaRebalance` configurations for deploying and using Cruise Control to manage clusters.
<6> `Kafka` and `KafkaNodePool` custom resource configurations for a deployment of Kafka clusters that use KRaft mode. Includes example configuration for an ephemeral or persistent single or multi-node deployment.
<7> `Kafka` and `KafkaRebalance` configurations for deploying and using Cruise Control to manage clusters.
`Kafka` configuration examples enable auto-rebalancing on scaling events and set default optimization goals.
`KakaRebalance` configuration examples set proposal-specific optimization goals and generate optimization proposals in various supported modes.
<9> `KafkaConnect` and `KafkaConnector` custom resource configuration for a deployment of Kafka Connect. Includes example configurations for a single or multi-node deployment.
<10> `KafkaBridge` custom resource configuration for a deployment of Kafka Bridge.
<8> `KafkaConnect` and `KafkaConnector` custom resource configuration for a deployment of Kafka Connect. Includes example configurations for a single or multi-node deployment.
<9> `KafkaBridge` custom resource configuration for a deployment of Kafka Bridge.
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The deployment uses a YAML file to provide the specification to create a `Kafka`
Strimzi provides the following xref:config-examples-{context}[example deployment files] that you can use to create a Kafka cluster that uses node pools:

`kafka/kraft/kafka-with-dual-role-nodes.yaml`:: Deploys a Kafka cluster with one pool of nodes that share the broker and controller roles.
`kafka/kraft/kafka.yaml`:: Deploys a persistent Kafka cluster with one pool of controller nodes and one pool of broker nodes.
`kafka/kraft/kafka-persistent.yaml`:: Deploys a persistent Kafka cluster with one pool of controller nodes and one pool of broker nodes.
`kafka/kraft/kafka-ephemeral.yaml`:: Deploys an ephemeral Kafka cluster with one pool of controller nodes and one pool of broker nodes.
`kafka/kraft/kafka-single-node.yaml`:: Deploys a Kafka cluster with a single node.
`kafka/kraft/kafka-jbod.yaml`:: Deploys a Kafka cluster with multiple volumes in each broker node.
Expand Down Expand Up @@ -44,7 +44,7 @@ To change the cluster name before you deploy the cluster, edit the `Kafka.metada
To deploy a Kafka cluster with a single node pool that uses dual-role nodes:
+
[source,shell]
kubectl apply -f examples/kafka/kraft/kafka-with-dual-role-nodes.yaml
kubectl apply -f examples/kafka/kafka-with-dual-role-nodes.yaml

. Check the status of the deployment:
+
Expand Down

This file was deleted.

4 changes: 2 additions & 2 deletions documentation/modules/metrics/ref_metrics-dashboards.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ The dashboards are populated with a representative set of metrics for monitoring
|Kafka
|`strimzi-kafka.json`

|ZooKeeper
|`strimzi-zookeeper.json`
|Kafka KRaft controllers
|`strimzi-kraft.json`

|Kafka Connect
|`strimzi-kafka-connect.json`
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,50 @@
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: controller
labels:
strimzi.io/cluster: my-cluster
spec:
replicas: 3
roles:
- controller
storage:
type: jbod
volumes:
- id: 0
type: ephemeral
kraftMetadata: shared
---

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: broker
labels:
strimzi.io/cluster: my-cluster
spec:
replicas: 3
roles:
- broker
storage:
type: jbod
volumes:
- id: 0
type: ephemeral
kraftMetadata: shared
---

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: my-cluster
annotations:
strimzi.io/node-pools: enabled
strimzi.io/kraft: enabled
spec:
kafka:
version: 3.9.0
replicas: 3
metadataVersion: 3.9-IV0
listeners:
- name: plain
port: 9092
Expand All @@ -21,13 +60,6 @@ spec:
transaction.state.log.min.isr: 2
default.replication.factor: 3
min.insync.replicas: 2
inter.broker.protocol.version: "3.9"
storage:
type: ephemeral
zookeeper:
replicas: 3
storage:
type: ephemeral
entityOperator:
topicOperator: {}
userOperator: {}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,50 @@
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: controller
labels:
strimzi.io/cluster: my-cluster
spec:
replicas: 3
roles:
- controller
storage:
type: jbod
volumes:
- id: 0
type: ephemeral
kraftMetadata: shared
---

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: broker
labels:
strimzi.io/cluster: my-cluster
spec:
replicas: 3
roles:
- broker
storage:
type: jbod
volumes:
- id: 0
type: ephemeral
kraftMetadata: shared
---

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: my-cluster
annotations:
strimzi.io/node-pools: enabled
strimzi.io/kraft: enabled
spec:
kafka:
version: 3.9.0
replicas: 3
metadataVersion: 3.9-IV0
listeners:
- name: plain
port: 9092
Expand All @@ -21,13 +60,6 @@ spec:
transaction.state.log.min.isr: 2
default.replication.factor: 3
min.insync.replicas: 2
inter.broker.protocol.version: "3.9"
storage:
type: ephemeral
zookeeper:
replicas: 3
storage:
type: ephemeral
entityOperator:
topicOperator: {}
userOperator: {}
Expand Down
48 changes: 40 additions & 8 deletions packaging/examples/cruise-control/kafka-cruise-control.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,50 @@
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: controller
labels:
strimzi.io/cluster: my-cluster
spec:
replicas: 3
roles:
- controller
storage:
type: jbod
volumes:
- id: 0
type: ephemeral
kraftMetadata: shared
---

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: broker
labels:
strimzi.io/cluster: my-cluster
spec:
replicas: 3
roles:
- broker
storage:
type: jbod
volumes:
- id: 0
type: ephemeral
kraftMetadata: shared
---

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: my-cluster
annotations:
strimzi.io/node-pools: enabled
strimzi.io/kraft: enabled
spec:
kafka:
version: 3.9.0
replicas: 3
metadataVersion: 3.9-IV0
listeners:
- name: plain
port: 9092
Expand All @@ -21,13 +60,6 @@ spec:
transaction.state.log.min.isr: 2
default.replication.factor: 3
min.insync.replicas: 2
inter.broker.protocol.version: "3.9"
storage:
type: ephemeral
zookeeper:
replicas: 3
storage:
type: ephemeral
entityOperator:
topicOperator: {}
userOperator: {}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
# KRaft examples

The examples in this directory demonstrate how you can use Kraft (ZooKeeper-less Apache Kafka) with Strimzi.
* The [`kafka.yaml`](kafka.yaml) deploys a Kafka cluster with one pool of _KRaft controller_ nodes and one pool of _KRaft broker_ nodes.
* The [`kafka-persistent.yaml`](kafka-persistent.yaml) deploys a Kafka cluster with one pool of _KRaft controller_ nodes and one pool of _KRaft broker_ nodes.
* The [`kafka-ephemeral.yaml`](kafka-ephemeral.yaml) deploys a Kafka cluster with one pool of _KRaft controller_ nodes, one pool of _KRaft broker_ nodes and ephemeral storage.
* The [`kafka-with-dual-role-nodes.yaml`](kafka-with-dual-role-nodes.yaml) deploys a Kafka cluster with one pool of KRaft nodes that share the _broker_ and _controller_ roles.
* The [`kafka-single-node.yaml`](kafka-single-node.yaml) deploys a Kafka cluster with a single Kafka node that has both _broker_ and _controller_ roles.

Please note that ZooKeeper-less Apache Kafka is still under development and has some limitations.
For instance, scaling of controller node or JBOD storage (you can specify `type: jbod` storage in Strimzi custom resources, but it should contain only a single volume) are not supported.
Loading
Loading