Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] Optimize OKS calculation method #5299

Open
2 of 6 tasks
unfollow7 opened this issue Dec 19, 2024 · 0 comments
Open
2 of 6 tasks

[FR] Optimize OKS calculation method #5299

unfollow7 opened this issue Dec 19, 2024 · 0 comments
Labels
feature Work on a feature request

Comments

@unfollow7
Copy link

Proposal Summary

hope the σ defined in coco OKS can be input form Dataset.evaluate_detections() or use a new default method

Motivation

method ***def _compute_object_keypoint_similarity(gtp, predp): *** in fiftyone.utils.iou.py use the default kappa(κ=1),it`s not a compatible way for many kp model like petskeleton or facelandmark,this cause the oks result is much better than the actual performance.

What areas of FiftyOne does this feature affect?

  • App: FiftyOne application
  • Core: Core fiftyone Python library
  • Server: FiftyOne server

Details

A method to get more accurate OKS result is to pass a σ,like define in coco for person.
I'd like to recommond a default method to calculate σ to replace current κ=1:measured the per-keypoint standard deviation σi with respect to object scale s in ground truth, coco defined this.
also hope users can input a custom σ list in evaluate_detections

besides,there is 2 bug in method _compute_object_keypoint_similarity:

  1. when coco calcute oks,dt is sorted by score,in COCOeval.computeOks:inds = np.argsort([-d['score'] for d in dts], kind='mergesort'),I don't find in fiftyone _compute_keypoint_similarities method
  2. scale first np.sqrt then scale**2,this can be merge to improve performance

Willingness to contribute

The FiftyOne Community welcomes contributions! Would you or another member of your organization be willing to contribute an implementation of this feature?

  • Yes. I can contribute this feature independently
  • Yes. I would be willing to contribute this feature with guidance from the FiftyOne community
  • No. I cannot contribute this feature at this time
@unfollow7 unfollow7 added the feature Work on a feature request label Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Work on a feature request
Projects
None yet
Development

No branches or pull requests

1 participant