Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abstract Java Kafka client metrics via Producer/ConsumerMetrics #430

Merged
merged 9 commits into from
Jul 15, 2024

Conversation

dfakhritdinov
Copy link
Contributor

@dfakhritdinov dfakhritdinov commented Jul 11, 2024

The PR addresses issue evolution-gaming/smetrics#237 and based on @Z1kkurat PR #324, thanks for inspiration!

Goal of the change is in exposing Java Kafka client metrics in the same way as native skafka metrics.

Native metrics represented by ProducerMetrics and ConsumerMetrics traits, their implementation require com.evolutiongaming.smetrics.CollectorRegistry. This approach allows decoupling metrics and their collector implementation, which is Prometheus (at least smetrics provides Prometheus support, other implementations also possible).

Java Kafka client metrics can be exposed to Prometheus with help of com.evolutiongaming.skafka.metrics.KafkaMetricsCollector - Prometheus collector was placed in separate module with dependency on Prometheus client. This PR aimed integrate the collector with ProducerMetrics and ConsumerMetrics traits in main skafka module (one without dependency on Prometheus client). For that new methods were introduced in both traits with default empty implementation. Actual implementation is in helper objects ProducerMetricsOf and ConsumerMetricsOf respectively.

Example:

import ConsumerMetricsOf.*

val config: ConsumerConfig = ???
val registry: CollectorRegistry = ???   // Prometheus client
val metrics: ConsumerMetrics[IO] = ???

for {
  metrics   <- metrics.exposeJavaClientMetrics(registry)
  consumerOf = ConsumerOf.apply1(metrics1.some) 
  consumer  <- consumerOf(config)
} yield ???

@coveralls
Copy link

coveralls commented Jul 11, 2024

Pull Request Test Coverage Report for Build 9937634721

Details

  • 19 of 26 (73.08%) changed or added relevant lines in 5 files are covered.
  • 4 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-0.2%) to 96.88%

Changes Missing Coverage Covered Lines Changed/Added Lines %
modules/metrics/src/main/scala/com/evolutiongaming/skafka/metrics/KafkaMetricsRegistry.scala 0 7 0.0%
Files with Coverage Reduction New Missed Lines %
skafka/src/main/scala/com/evolutiongaming/skafka/consumer/ConsumerLogging.scala 2 92.12%
skafka/src/main/scala/com/evolutiongaming/skafka/producer/ProducerMetrics.scala 2 98.21%
Totals Coverage Status
Change from base Build 9220432565: -0.2%
Covered Lines: 2205
Relevant Lines: 2276

💛 - Coveralls

@dfakhritdinov dfakhritdinov requested review from Z1kkurat and mr-git July 11, 2024 14:26
@dfakhritdinov dfakhritdinov merged commit 7a7a5d3 into master Jul 15, 2024
6 of 7 checks passed
@dfakhritdinov dfakhritdinov deleted the df/expose-java-client-metrics branch July 21, 2024 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants