feat: ingest metrics #25692
Merged
feat: ingest metrics #25692
+301
−13
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added prometheus metrics to track lines written and bytes written per database. The write buffer does the tracking after validation of incoming line protocol. Tests added to verify.
hiltontj
requested review from
pauldix,
mgattozzi,
jacksonrnewhouse and
praveen-influx
pauldix
approved these changes
Dec 20, 2024
| // now do a write that will only be partially accepted to ensure that | ||
| // the metrics are only calculated for writes that get accepted: | ||
|
|
||
| // the legume will not be accepted, because it contains a new tag, |
There was a problem hiding this comment.
As a follow up it would be great to add a influxdb_write_lines_rejected_total counter.
|
@MaduMitha-Ravi once this gets in it would be great to update the perf dashboards and reports to include these figures. That way I can stop asking "what write load was this with?" 😁 |
|
Sure Paul 👍 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #25689
Added prometheus metrics to track lines written and bytes written per database. The write buffer does the tracking after validation of incoming line protocol.
Added a test to check that this works via writes made through the
write_lpmethod on the write buffer.Note: I did not track writes on a per-table basis, only per-database. Doing it per-table, as per #25689, would be possible but will require a bit of overhead tracking the table name for each line all the way through the validation process. Currently, validation converts the table name to a table ID, and carries that through to the point that the line is deemed valid, since the WAL does not need the table name for write batches.