Questions about the IOU parameter setting for AP50 comparison in the paper

[MarineCao]

Hello,

I find that you directly load and set the YOLOv5 model in the code by

# set model
try:
    model = yolov5.load(self.model_path, device=self.device)
    model.conf = self.confidence_threshold
    self.model = model
except Exception as e:
    raise TypeError("model_path is not a valid yolov5 model path: ", e)

This means that the inference results of YOLOv5 model for each sliced image are NMS processed in the YOLOv5 code, and then the processed results are postprocessed by GREEDYNMM or other algorithms.

It seems that you only set the confidence threshold of the YOLOv5 model, while the IOU threshold is the default value of the YOLOv5 code, i.e.

class AutoShape(nn.Module):
    # YOLOv5 input-robust model wrapper for passing cv2/np/PIL/torch inputs. Includes preprocessing, inference and NMS
    conf = 0.25  # NMS confidence threshold
    iou = 0.45  # NMS IoU threshold

I'd like to ask that what the IOU parameter setting is for AP50 comparison in the paper, inluding YOLOv5 NMS IOU threshold and match_threshold in SAHI postprocess?
I have noted that the description about IOU in the paper, i.e.
The MS COCO [7] evaluation protocol has been adopted for evaluation, including overall and size-wise AP50 scores. Specifically, AP50 is computed at the single IoU threshold 0.5 over all categories and maximum number of detections is set as 500.
but which IOU does the paper mean? Only match_threshold in SAHI postprocess or both match_threshold and YOLOv5 NMS IOU threshold?

[fcakyon]

Hello @MarineCao, thanks for your interest on our paper.

In our paper we dont provide results for yolov5, we provide for FCOS, VFNet and TOOD detectors. We simply use the predict function with model_type='mmdet' and iou match threshold as 0.5, postprocess type as NMS.

More detail on the inference parameters can be found here: https://github.com/fcakyon/sahi-benchmark/issues/2#issuecomment-1094234251

All model config files can be found here: https://github.com/fcakyon/sahi-benchmark/tree/main/mmdet_configs

If you have any other questions feel free to ask 👍