Add PIMO #2329

ashwinvaidya17 · 2024-09-30T13:03:46Z

📝 Description

Adds the PIMO metric created by @jpcbertoldo

✨ Changes

Select what type of change your PR is:

🐞 Bug fix (non-breaking change which fixes an issue)
🔨 Refactor (non-breaking change which refactors the code base)
🚀 New feature (non-breaking change which adds functionality)
💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
📚 Documentation update
🔒 Security update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
📚 I have made the necessary updates to the documentation (if applicable).
🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

For more information about code review checklists, see the Code Review Checklist.

* update Signed-off-by: jpcbertoldo <[email protected]> * test binclf curves numpy and numba and fixes Signed-off-by: jpcbertoldo <[email protected]> * correct som docstrings Signed-off-by: jpcbertoldo <[email protected]> * torch interface and tests Signed-off-by: jpcbertoldo <[email protected]> * torch interface and tests Signed-off-by: jpcbertoldo <[email protected]> * constants regrouped in dataclass as class vars Signed-off-by: jpcbertoldo <[email protected]> * result class was unneccesary for per_image_binclf_curve Signed-off-by: jpcbertoldo <[email protected]> * factorize function _get_threshs_minmax_linspace Signed-off-by: jpcbertoldo <[email protected]> * small docs fixes Signed-off-by: jpcbertoldo <[email protected]> * add pimo numpy version and test Signed-off-by: jpcbertoldo <[email protected]> * move validation Signed-off-by: jpcbertoldo <[email protected]> * add `shared_fpr_metric` option Signed-off-by: jpcbertoldo <[email protected]> * add pimo torch functional version and test Signed-off-by: jpcbertoldo <[email protected]> * add torchmetrics interface and test Signed-off-by: jpcbertoldo <[email protected]> * renames and put things in init Signed-off-by: jpcbertoldo <[email protected]> * validate inputs in result objects Signed-off-by: jpcbertoldo <[email protected]> * result objects to from dict and tests Signed-off-by: jpcbertoldo <[email protected]> * add save and load methods to result objects and test Signed-off-by: jpcbertoldo <[email protected]> * refactor validations and minor changes Signed-off-by: jpcbertoldo <[email protected]> * test result objects' properties Signed-off-by: jpcbertoldo <[email protected]> * minor refactors Signed-off-by: jpcbertoldo <[email protected]> * add missing docstrings Signed-off-by: jpcbertoldo <[email protected]> * minore vocabulary fix for consistency Signed-off-by: jpcbertoldo <[email protected]> * add per image scores statistics and test it Signed-off-by: jpcbertoldo <[email protected]> * refactor constants notation Signed-off-by: jpcbertoldo <[email protected]> * add stats tests and test it Signed-off-by: jpcbertoldo <[email protected]> * change the meaning of AUPIMO.num_thresh Signed-off-by: jpcbertoldo <[email protected]> * interface to format pairwise test results Signed-off-by: jpcbertoldo <[email protected]> * improve doc Signed-off-by: jpcbertoldo <[email protected]> * add optional `paths` to result objects and some minor fixes and refactors Signed-off-by: jpcbertoldo <[email protected]> * remove frozen from dataclasses and some done todos Signed-off-by: jpcbertoldo <[email protected]> * review headers Signed-off-by: jpcbertoldo <[email protected]> * doc modifs Signed-off-by: jpcbertoldo <[email protected]> * refactor `score_less_than_thresh` in `_binclf_one_curve_python` Signed-off-by: jpcbertoldo <[email protected]> * correct license comments Signed-off-by: jpcbertoldo <[email protected]> * fix doc Signed-off-by: jpcbertoldo <[email protected]> * numba as extra requirement Signed-off-by: jpcbertoldo <[email protected]> * refactor copyrights from jpcbertoldo Signed-off-by: jpcbertoldo <[email protected]> * remove from __future__ import annotations Signed-off-by: jpcbertoldo <[email protected]> * refactor validations names Signed-off-by: jpcbertoldo <[email protected]> * dedupe file path validation Signed-off-by: jpcbertoldo <[email protected]> * fix tests Signed-off-by: jpcbertoldo <[email protected]> * Add todo Signed-off-by: jpcbertoldo <[email protected]> * refactor enums Signed-off-by: jpcbertoldo <[email protected]> * only logger.warning Signed-off-by: jpcbertoldo <[email protected]> * refactor test imports Signed-off-by: jpcbertoldo <[email protected]> * refactor docs Signed-off-by: jpcbertoldo <[email protected]> * refactor some docs Signed-off-by: jpcbertoldo <[email protected]> * correct pre commit errors Signed-off-by: jpcbertoldo <[email protected]> * remove author tag Signed-off-by: jpcbertoldo <[email protected]> * add thrid party program Signed-off-by: jpcbertoldo <[email protected]> * Update src/anomalib/metrics/per_image/pimo.py * move HAS_NUMBA Signed-off-by: jpcbertoldo <[email protected]> * remove PIMOSharedFPRMetric Signed-off-by: jpcbertoldo <[email protected]> * make torchmetrics compute avg by dft Signed-off-by: jpcbertoldo <[email protected]> * pre-commit hooks corrections Signed-off-by: jpcbertoldo <[email protected]> * correct numpy.trapezoid Signed-off-by: jpcbertoldo <[email protected]> --------- Signed-off-by: jpcbertoldo <[email protected]> Co-authored-by: Samet Akcay <[email protected]>

* remove numba Signed-off-by: Ashwin Vaidya <[email protected]> * fix pre-commit checks Signed-off-by: Ashwin Vaidya <[email protected]> * add third-party-programs.txt Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]>

* remove numba Signed-off-by: Ashwin Vaidya <[email protected]> * fix pre-commit checks Signed-off-by: Ashwin Vaidya <[email protected]> * remove all unused methods Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]>

* remove numba Signed-off-by: Ashwin Vaidya <[email protected]> * fix pre-commit checks Signed-off-by: Ashwin Vaidya <[email protected]> * remove all unused methods Signed-off-by: Ashwin Vaidya <[email protected]> * replace numpy with torch Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]>

* remove numba Signed-off-by: Ashwin Vaidya <[email protected]> * fix pre-commit checks Signed-off-by: Ashwin Vaidya <[email protected]> * remove all unused methods Signed-off-by: Ashwin Vaidya <[email protected]> * replace numpy with torch Signed-off-by: Ashwin Vaidya <[email protected]> * refactor code Signed-off-by: Ashwin Vaidya <[email protected]> * refactor move functional inside update remove path from the metric * Add changes from comments Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]>

* rename to pimo Signed-off-by: Ashwin Vaidya <[email protected]> * minor refactor Signed-off-by: Ashwin Vaidya <[email protected]> * remove model to model comparison Signed-off-by: Ashwin Vaidya <[email protected]> * fix test Signed-off-by: Ashwin Vaidya <[email protected]> * PR comments Signed-off-by: Ashwin Vaidya <[email protected]> * Minor refactor Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]>

src/anomalib/metrics/__init__.py

src/anomalib/metrics/pimo/__init__.py

samet-akcay

thanks, some minor comments only.

overall, can we rename all the thresh to threshold and threshs to thresholds

src/anomalib/metrics/pimo/binary_classification_curve.py

src/anomalib/metrics/pimo/dataclasses.py

src/anomalib/metrics/pimo/functional.py

src/anomalib/metrics/pimo/pimo.py

tests/unit/metrics/pimo/test_binclf_curve.py

codecov · 2024-09-30T13:59:38Z

Codecov Report

Attention: Patch coverage is 77.34513% with 128 lines in your changes missing coverage. Please review.

Project coverage is 80.78%. Comparing base (f473df8) to head (297f0ca).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/anomalib/metrics/pimo/_validate.py	67.96%	66 Missing ⚠️
src/anomalib/metrics/pimo/dataclasses.py	72.28%	23 Missing ⚠️
src/anomalib/metrics/pimo/functional.py	81.63%	18 Missing ⚠️
...omalib/metrics/pimo/binary_classification_curve.py	86.95%	12 Missing ⚠️
src/anomalib/metrics/pimo/pimo.py	87.83%	9 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2329      +/-   ##
==========================================
- Coverage   80.93%   80.78%   -0.16%     
==========================================
  Files         247      254       +7     
  Lines       10899    11463     +564     
==========================================
+ Hits         8821     9260     +439     
- Misses       2078     2203     +125

Flag	Coverage Δ
	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Ashwin Vaidya <[email protected]>

* add aupimo basic usage tutorial notebook Signed-off-by: jpcbertoldo <[email protected]> * update scipy import Signed-off-by: jpcbertoldo <[email protected]> * add cite us Signed-off-by: jpcbertoldo <[email protected]> * minor Signed-off-by: jpcbertoldo <[email protected]> * modify texts and add illustration Signed-off-by: jpcbertoldo <[email protected]> * udpate working dir Signed-off-by: jpcbertoldo <[email protected]> --------- Signed-off-by: jpcbertoldo <[email protected]>

review-notebook-app · 2024-10-02T08:29:48Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

jpcbertoldo · 2024-10-02T13:28:14Z

Yay!

jpcbertoldo and others added 6 commits September 20, 2024 10:18

ashwinvaidya17 requested review from samet-akcay and djdameln as code owners September 30, 2024 13:03

samet-akcay reviewed Sep 30, 2024

View reviewed changes

src/anomalib/metrics/__init__.py Outdated Show resolved Hide resolved

samet-akcay reviewed Sep 30, 2024

View reviewed changes

src/anomalib/metrics/pimo/__init__.py Outdated Show resolved Hide resolved

Merge branch 'main' into feature/pimo

e5e793d

samet-akcay reviewed Sep 30, 2024

View reviewed changes

ashwinvaidya17 added 2 commits September 30, 2024 16:04

PR comments

036e1eb

Signed-off-by: Ashwin Vaidya <[email protected]>

Remove unused enums

1997751

Signed-off-by: Ashwin Vaidya <[email protected]>

jpcbertoldo mentioned this pull request Sep 30, 2024

add aupimo basic usage tutorial notebook #2330

Merged

9 tasks

ashwinvaidya17 and others added 3 commits October 1, 2024 10:17

update doc strings

49291b6

Signed-off-by: Ashwin Vaidya <[email protected]>

update param names

8ff90a8

Signed-off-by: Ashwin Vaidya <[email protected]>

samet-akcay approved these changes Oct 2, 2024

View reviewed changes

ashwinvaidya17 enabled auto-merge (squash) October 2, 2024 11:45

ashwinvaidya17 merged commit 191e21f into main Oct 2, 2024
7 of 9 checks passed

ashwinvaidya17 deleted the feature/pimo branch October 2, 2024 11:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add PIMO #2329

Add PIMO #2329

ashwinvaidya17 commented Sep 30, 2024

samet-akcay left a comment

codecov bot commented Sep 30, 2024 •

edited

Loading

review-notebook-app bot commented Oct 2, 2024

jpcbertoldo commented Oct 2, 2024

Add PIMO #2329

Add PIMO #2329

Conversation

ashwinvaidya17 commented Sep 30, 2024

📝 Description

✨ Changes

✅ Checklist

samet-akcay left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 30, 2024 • edited Loading

Codecov Report

review-notebook-app bot commented Oct 2, 2024

jpcbertoldo commented Oct 2, 2024

codecov bot commented Sep 30, 2024 •

edited

Loading