Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce memory usage by reclaimed MetricPoint #5416

Conversation

specialforest
Copy link
Contributor

@specialforest specialforest commented Mar 6, 2024

Changes

Current MetricPoint reclaim mechanism leaves lots of unused objects in memory (histogram buckets, exemplar). With this change these objects can be garbage collected.

Tested with a sample program:

  1. Emits high cardinality (171x3x1000x2x593) metric.
  2. Maximum number of metric points allowed per metric stream is 50000.
  3. Histograms configured to use 165 fixed buckets.

Heap snapshot before the change:
image

Heap snapshot after the change:
image

Merge requirement checklist

  • CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
  • Unit tests added/updated
  • Appropriate CHANGELOG.md files updated for non-trivial changes
  • Changes in public API reviewed (if applicable)

@specialforest specialforest requested a review from a team March 6, 2024 01:07
@specialforest specialforest force-pushed the recycle-memory-used-by-reclaimed-metricpoint branch from 4dd3bf5 to 57fec34 Compare March 6, 2024 01:08
Copy link

codecov bot commented Mar 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.70%. Comparing base (6250307) to head (57fec34).
Report is 115 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #5416      +/-   ##
==========================================
+ Coverage   83.38%   84.70%   +1.32%     
==========================================
  Files         297      281      -16     
  Lines       12531    12105     -426     
==========================================
- Hits        10449    10254     -195     
+ Misses       2082     1851     -231     
Flag Coverage Δ
unittests ?
unittests-Solution-Experimental 84.48% <100.00%> (?)
unittests-Solution-Stable 84.65% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
src/OpenTelemetry/Metrics/AggregatorStore.cs 85.21% <100.00%> (+4.82%) ⬆️
src/OpenTelemetry/Metrics/MetricPoint.cs 92.33% <100.00%> (+23.86%) ⬆️

... and 55 files with indirect coverage changes

internal void Reclaim()
{
this.LookupData = null;
this.mpComponents = null;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Member

@cijothomas cijothomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️ Thank you!

@utpilla
Copy link
Contributor

utpilla commented Mar 6, 2024

@specialforest Thanks for this PR!

@utpilla utpilla merged commit bae8a6b into open-telemetry:main Mar 6, 2024
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants