Bug during as_coco() conversion for segmentation (+ solution)

[Youho99]

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

Ce problème fait suite au travail fait sur mon précédent issue #1052

results.as_coco(annotations_path="test_coco/", min_image_area_percentage=0.01,)

IndexError: list index out of range

The problem is due to the fact that in certain cases, the mask returned by GroundedSam is completely inconsistent (less than 3 pixels for example). In this specific case, the function approximate_mask_with_polygons() (as_coco() > detections_to_coco_annotations() > approximate_mask_with_polygons()) returns an empty array.

So, the line

list(approximate_mask_with_polygons(
             mask=mask,
             min_image_area_percentage=min_image_area_percentage,
             max_image_area_percentage=max_image_area_percentage,
             approximation_percentage=approximation_percentage,
         )[0].flatten()

causes my error, because there is no item '0' in an empty list.

I will propose a fix following the writing of this issue.

I changed the architecture of the as_coco() function to be the same as the as_yolo() and as_pascal_voc() architecture.

Environment

Supervision 0.19.0

Minimal Reproducible Example

from autodistill_grounded_sam import GroundedSAM
from autodistill.detection import CaptionOntology
from pathlib import Path
import supervision as sv
import os
import shutil

# Put the cat image on your input directory
# Put your input directory path
input_dir = "/home/ggiret/Téléchargements/chat"
output_dir = "test/"

if os.path.exists(output_dir):
    shutil.rmtree(output_dir)

classes = {"cat": "a cat",}

base_model = GroundedSAM(
    ontology=CaptionOntology(
        classes
    ),
    box_threshold = 0.005
)

results = base_model.label(
        input_folder=input_dir,
        extension=".png",
        output_folder=output_dir, 
        record_confidence=True)

results.as_coco(annotations_path="test_coco/", min_image_area_percentage=0.01,)

Image to use :

Additional

In the code that I am going to propose, as_coco() will record either the BoundingBoxes coordinates or the Segmentation coordinates, but not both at the same time.

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[0xD4rky]

hey @LinasKo, if this issue's still open then can I take this up as well as I think I may have a solution to it!

[SkalskiP]

Hi @0xD4rky 👋🏻 Honestly, I don't know if this bug still exists. Additionally, it seems that the bug is rather related to the autodistill project and not supervision.

[Youho99]

Hi @SkalskiP @0xD4rky

The Supervision library has evolved a lot since this bug was discovered.
It is indeed possible that this bug no longer exists. To test this, it would be necessary to run the bug reproduction case that I had shared in the issue ticket.

At the time, I had already proposed a fix solution for this one (in the associated PR). Since then, I have moved the PR to draft, because I do not know if the bug still exists.

However, the bug mentioned came from the Supervision library and not from Autodistill.