Bboxes and Masks have different sizes

[FHarhammer]

Hi,
i´m using mmdeploy to convert a mask rcnn model with tensorrt. If i run the inference on this model with

result = self.model(image)

boxes_and_scores = result[0]
labels = result[1]
masks_raw = result[2]
boxes = boxes_and_scores[:,:-1].astype(np.int32)
scores = boxes_and_scores[:,-1]  

the bboxes have a different size as the masks i get. For example:
masks_raw[0].shape
returns
(55, 53)
and

boxes[0,3] - boxes[0,1]
boxes[0,2] - boxes[0,0]

returns

52
50

Any ideas why this could happen?

[lvhan028]

@grimoire

[grimoire]

What is this self.model?

[FHarhammer]

Ah sorry i forgot to add this. This is how i load the model:

from mmdeploy_python import Detector
self.model  = Detector(model_path=PATH_TO_THE_MODEL, device_name="cuda", device_id=0)

[grimoire]

@lvhan028

[FHarhammer]

Is this a known problem or does it only happen for me ? @grimoire @lvhan028

[lvhan028]

I think boxes = boxes_and_scores[:,:-1].astype(np.int32) is not precisely in your code.

Refer to

mmdeploy/csrc/mmdeploy/codebase/mmdet/instance_segmentation.cpp

Line 74 in de96f51

void ProcessMasks(Detections& result, Tensor cpu_masks, int img_w, int img_h) const {

, you can find the following codes to get the bbox [x0, y0, x1, y1] of a mask.

  auto x0 = std::max(std::floor(bbox[0]) - 1, 0.f);
  auto y0 = std::max(std::floor(bbox[1]) - 1, 0.f);
  auto x1 = std::min(std::ceil(bbox[2]) + 1, (float)img_w);
  auto y1 = std::min(std::ceil(bbox[3]) + 1, (float)img_h);
  auto width = static_cast<int>(x1 - x0);
  auto height = static_cast<int>(y1 - y0);

@lzhangzz can it be guaranteed that warped_mask has the same shape with mask

[lzhangzz]

The logic follows
https://github.com/open-mmlab/mmdetection/blob/31c84958f54287a8be2b99cbf87a6dcf12e57753/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py#L374-L380
to ensure the interpolated mask is perfectly contained in the result.

[FHarhammer]

ok thanks i see, i think i can work with that