Skip to content
This repository has been archived by the owner on Feb 8, 2024. It is now read-only.

kafka: check reachability + collect metrics #37

Merged
merged 3 commits into from
Oct 27, 2023
Merged

Conversation

xvello
Copy link
Contributor

@xvello xvello commented Oct 27, 2023

  • Make sure capture does not start if the Kafka brokers are unreachable. We'll want to tie that to a readiness probe later instead.
  • Collect kafka producer metrics every 10 seconds:
capture_kafka_broker_tx_errors_total{broker="1"} 0
capture_kafka_broker_tx_errors_total{broker="-1"} 0
capture_kafka_broker_rx_errors_total{broker="-1"} 0
capture_kafka_broker_rx_errors_total{broker="1"} 0
capture_kafka_producer_queue_bytes 100000
capture_kafka_producer_queue_bytes_limit 1073741824
capture_kafka_producer_queue_depth_limit 100000
capture_kafka_produce_avg_batch_size_bytes{topic="events_plugin_ingestion"} 2036
capture_kafka_callback_queue_depth 0
capture_kafka_broker_requests_pending{broker="-1"} 0
capture_kafka_broker_requests_pending{broker="1"} 0
capture_kafka_producer_queue_depth 1
capture_kafka_produce_avg_batch_size_events{topic="events_plugin_ingestion"} 1
capture_kafka_broker_responses_awaiting{broker="-1"} 0
capture_kafka_broker_responses_awaiting{broker="1"} 0

@xvello xvello marked this pull request as ready for review October 27, 2023 15:49
@xvello xvello requested a review from ellie October 27, 2023 15:49
@xvello xvello changed the title collect kafka metrics kafka: check reachability + collect metrics Oct 27, 2023
@xvello xvello merged commit fb6fa7c into main Oct 27, 2023
4 checks passed
@xvello xvello deleted the xvello/kafka-context branch October 27, 2023 16:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants