Skip to content

Files

This branch is 1145 commits behind datahub-project/datahub:master.

kafka-setup

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
May 1, 2023
Aug 23, 2024
Aug 6, 2020
Nov 28, 2023
Jul 31, 2024
Sep 17, 2024
Aug 22, 2023
Jul 31, 2024
Oct 29, 2023

Kafka, Zookeeper and Schema Registry

DataHub uses Kafka as the pub-sub message queue in the backend. Official Confluent Kafka Docker images found in Docker Hub is used without any modification.

Debugging Kafka

You can install kafkacat to consume and produce messaged to Kafka topics. For example, to consume messages on MetadataAuditEvent topic, you can run below command.

kafkacat -b localhost:9092 -t MetadataAuditEvent

However, kafkacat currently doesn't support Avro deserialization at this point, but they have an ongoing work for that.