Kafka for Streaming

Common Services of Kafka and Streaming

Zookeeper

This service uses the confluentinc/cp-zookeeper:5.4.0 image and is responsible for running a ZooKeeper server. It exposes port 2181 and sets some environment variables related to ZooKeeper’s configuration.

Broker

This service uses the confluentinc/cp-server:5.4.0 image and represents a Kafka broker. It depends on the zookeeper service and exposes port 9092. It sets various environment variables related to Kafka’s configuration, such as the broker ID, ZooKeeper connection, advertised listeners, and metrics reporting.

Kafka-tools

This service uses the confluentinc/cp-kafka:5.4.0 image and is mainly used for administrative purposes. It runs the tail -f /dev/null command to keep the container running indefinitely. It uses the host network mode, which means it shares the network namespace with the host machine.

Schema-registry

This service uses the confluentinc/cp-schema-registry:5.4.0 image and provides a schema registry for Kafka. It depends on the zookeeper and broker services and exposes port 8081. It sets environment variables related to the schema registry’s configuration.

Control-center

This service uses the confluentinc/cp-enterprise-control-center:5.4.0 image and represents the Confluent Control Center. It depends on the zookeeper, broker, and schema-registry services. It exposes port 9021 and sets environment variables related to Control Center’s configuration.

Streaming Data and Processing Data during transitions

Schema Registry with Avro

Kafka Connect, Source and Sink

Kafka Streams

Choose Your Plan 💫