This repository has the content to interact with Kafka using AVRO and Schema Registry.
- This should set up the Zookeeper and Kafka Broker in your local environment
docker-compose up- Run this below command
docker ps
- You should be below containers up and running in local
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fd305f78339a confluentinc/cp-schema-registry:7.1.0 "/etc/confluent/dock…" 49 seconds ago Up 48 seconds 0.0.0.0:8081->8081/tcp schema-registry
fb28f7f91b0e confluentinc/cp-server:7.1.0 "/etc/confluent/dock…" 50 seconds ago Up 49 seconds 0.0.0.0:9092->9092/tcp, 0.0.0.0:9101->9101/tcp broker
d00a0f845a45 confluentinc/cp-zookeeper:7.1.0 "/etc/confluent/dock…" 50 seconds ago Up 49 seconds 2888/tcp, 0.0.0.0:2181->2181/tcp, 3888/tcp zookeeper
- This command should take care of logging in to the Kafka container.
docker exec -it broker bash
- Command to produce messages in to the Kafka topic.
kafka-console-producer --broker-list localhost:9092 --topic test-topic
- This command should take care of logging in to the Kafka container.
docker exec -it broker bash
- Command to produce messages in to the Kafka topic.
kafka-console-consumer --bootstrap-server localhost:9092 --topic test-topic
- This command should take care of logging in to the Schema Registry container.
docker exec -it schema-registry bash
- Run the kafka-avro-console-producer with the Schema
kafka-avro-console-producer --broker-list broker:29092 --topic greetings --property value.schema='{"type": "record","name":"Greeting","fields": [{"name": "greeting","type": "string"}]}'
- Publish the Greeting message
{"greeting": "Good Morning!, AVRO"}
{"greeting": "Good Evening!, AVRO"}
{"greeting": "Good Night!, AVRO"}
- This command should take care of logging in to the Schema Registry container.
docker exec -it schema-registry bash
- Run the kafka-avro-console-consumer
kafka-avro-console-consumer --bootstrap-server broker:29092 --topic greetings --from-beginning
- To view the list of topics in the Kafka environment.
kafka-topics --bootstrap-server localhost:9092 --describe
-
Note : Schemas are stored in an internal topic named _schemas*
-
Lets view the configuration set up for _schemas topics
kafka-topics --bootstrap-server localhost:9092 --describe --topic _schemas