Skip to content

Creating Compound Metrics and Expanding our checkpointers (2/5) #368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 30, 2025

Conversation

emersodb
Copy link
Collaborator

@emersodb emersodb commented Apr 1, 2025

PR Type

Refactor

Short Description

This PR does a few fairly small things.

  1. It slightly expands our checkpointing classes to have a class for checkpointing based on a metric. It also allows for naming the scoring function for a bit more informative logging.
  2. It moves our TransformsMetric into a new file called compound_metrics.py, which will hold metrics that wrap metrics (hence the compound). EMAMetric, which will be added in the next PR, is another example of this kind of object.

Tests Added

Most of the changes are refactors, but I added extensive testing of the TransformsMetric class, as we didn't seem to have any prior to this.

emersodb added 2 commits April 1, 2025 17:49
…d metrics file, expanding the checkpointers to make best metric available
@emersodb emersodb marked this pull request as ready for review April 1, 2025 22:01
@emersodb emersodb changed the title PR Creating Metrics Base Class and Other Small Changes Creating Metrics Base Class and Other Small Changes Apr 1, 2025
@emersodb emersodb changed the title Creating Metrics Base Class and Other Small Changes Creating Compound Metrics and Expanding our checkpointers Apr 2, 2025
Copy link
Collaborator

@scarere scarere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@emersodb emersodb changed the title Creating Compound Metrics and Expanding our checkpointers Creating Compound Metrics and Expanding our checkpointers (2/5) Apr 2, 2025
Copy link
Collaborator

@lotif lotif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Base automatically changed from dbe/creating_metrics_module to main April 30, 2025 15:59
Copy link

codecov bot commented Apr 30, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.95%. Comparing base (237b1e6) to head (7a3ec6e).
Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #368      +/-   ##
==========================================
+ Coverage   75.84%   75.95%   +0.10%     
==========================================
  Files         144      145       +1     
  Lines        8782     8809      +27     
==========================================
+ Hits         6661     6691      +30     
+ Misses       2121     2118       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@emersodb emersodb merged commit 27a910e into main Apr 30, 2025
10 checks passed
@emersodb emersodb deleted the dbe/creating_metrics_base branch April 30, 2025 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants