-
Notifications
You must be signed in to change notification settings - Fork 36
References
Marco Neumann edited this page Aug 25, 2022
·
3 revisions
This contains a list of references that are useful when implementing the Apache Kafka protocol.
There are not THAT many actual client implementations, most of them are wrappers around librdkafka.
| Name | Language | Source Code |
|---|---|---|
| Apache Kafka | Java, Scala | https://github.com/apache/kafka |
| librdkafka | C | https://github.com/edenhill/librdkafka |
| Sarama | Go | https://github.com/Shopify/sarama |
| franz-go | Go | https://github.com/twmb/franz-go |
| kafka-rust | Rust | https://github.com/kafka-rust/kafka-rust |
| kafka-protocol-rs | Rust | https://github.com/tychedelia/kafka-protocol-rs |
| Marwes/kafka-protocol | Rust | https://github.com/Marwes/kafka-protocol |
| Wireshark | C | https://github.com/wireshark/wireshark |
| Name | Language | Source Code | Notes |
|---|---|---|---|
| Apache Kafka | Java, Scala | https://github.com/apache/kafka | |
| Redpanda | C++ | https://github.com/redpanda-data/redpanda | |
| Kafka-on-Pulsar | Java | https://github.com/streamnative/kop | Uses Apache Kafka protocol implementation |
| Name | Link |
|---|---|
| Apache Kafka Documentation -- Messages | https://kafka.apache.org/documentation/#messages |
| Kafka Protocol Guide | https://kafka.apache.org/protocol |
KIPs (Kafka Improvement Proposals) that are important:
- KIP-32: Add timestamps to Kafka message
- KIP-74: Add Fetch Response Size Limit in Bytes
- KIP-79: ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer
- KIP-98: Exactly Once Delivery and Transactional Messaging
- KIP-482: The Kafka Protocol should Support Optional Tagged Fields
Related information that is indirectly part of the protocol (or our interpretation of it):