Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

delay_serialization: implement feature #277

Merged
merged 1 commit into from
Jul 5, 2023

Conversation

pudiva
Copy link
Contributor

@pudiva pudiva commented Jun 7, 2023

Add the delay_serialization option, allowing users to delay expensive serialization until a more convenient time, such as after an HTTP request has completed. In multi-threaded mode, it causes serialization to happen inside the sender thread.

Also, support the sender_queue_size in single_thread mode, so that it can benefit from the new delay_serialization option. Messages are now queued (possibly unserialized) until sender_queue_size is reached or #flush is called. It may be set to Float::INFINITY, so that messages are indefinitely queued until an explicit #flush.

Fix #271

@pudiva pudiva requested a review from remeh as a code owner June 7, 2023 17:21
@pudiva pudiva force-pushed the delay_serialization branch 4 times, most recently from 2e51125 to d457d9e Compare June 7, 2023 18:42
@pudiva
Copy link
Contributor Author

pudiva commented Jun 7, 2023

cc @BlakeWilliams

@remeh
Copy link
Contributor

remeh commented Jun 13, 2023

Hey @pudiva, thanks a lot for the PR. I'll do my best to give it a look soon 🙇

Copy link
Contributor

@remeh remeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the quality PR. I don't have anything major to report, it's elegantly integrated to the current codebase. I just left a remark, let me know WDYT and then we'll merge this one!

lib/datadog/statsd/single_thread_sender.rb Outdated Show resolved Hide resolved
CHANGELOG.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
Add the `delay_serialization` option, allowing users to delay expensive
serialization until a more convenient time, such as after an HTTP
request has completed. In multi-threaded mode, it causes serialization
to happen inside the sender thread.

Also, support the `sender_queue_size` in `single_thread` mode, so that
it can benefit from the new `delay_serialization` option. Messages are
now queued (possibly unserialized) until `sender_queue_size` is reached
or `#flush` is called. It may be set to `Float::INFINITY`, so that
messages are indefinitely queued until an explicit `#flush`.

Fix DataDog#271

Co-Authored-By: Blake Williams <[email protected]>
@remeh remeh merged commit 0bea929 into DataDog:master Jul 5, 2023
2 checks passed
@remeh
Copy link
Contributor

remeh commented Jul 5, 2023

This will be part of next release (which should happen before the end of next week). Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Delay metric serialisation until flush
2 participants