inverse transform error: expecting ints not tensors

[jpcenteno80]

Hi, I am working with Monai version 0.5.dev2114 and I am trying to use inverse transforms to place a 3d segmentation back to its original space. I run into the following error which I think comes from monai/transforms/croppad/array.py in __call__(self, img, mode), line 151:
ValueError: self.spatial_border must contain only ints, got (tensor([72]), tensor([125]), tensor([104]), tensor([70]), tensor([95]), tensor([73])).
My transform looks like this:

val_transforms = Compose([
    LoadImaged(keys),
    Lambdad(keys, lambda x: np.moveaxis(np.asarray(x), 0, -1)),
    Lambdad(keys[1], lambda x: drop_vessels_artifacts_labels(x) if drop_v_a_labels else x),
    Lambdad(keys[1], lambda x: binarize_cac(x) if bin_cac else x),
    AddChanneld(keys),
    Spacingd(keys, pixdim=(1, 1, 1.25), mode=('bilinear', 'nearest', 'nearest')),
    CropForegroundd(keys, source_key=keys[-1], margin=10),
    DeleteItemsd(keys[-1]),
    ScaleIntensityRanged(keys[0], a_min=hu_min, a_max=hu_max,
                         b_min=0.0, b_max=1.0, clip=True),
    ToTensord(keys[:2])])

Where keys = ['image', 'label', 'mask'] and there are also some other pre-processing functions to move around label values.
My output in batch['label_transforms']:

[{'class': ['Spacingd'],
  'id': tensor([139704189840016]),
  'orig_size': [tensor([346]), tensor([346]), tensor([290])],
  'extra_info': {'meta_data_key': ['label_meta_dict'],
   'old_affine': tensor([[[   0.6758,    0.0000,    0.0000, -166.2000],
            [   0.0000,    0.6758,    0.0000, -173.0000],
            [   0.0000,    0.0000,   -1.2500,   -3.5700],
            [   0.0000,    0.0000,    0.0000,    1.0000]]], dtype=torch.float64)}},
 {'class': ['CropForegroundd'],
  'id': tensor([139704189840112]),
  'orig_size': [tensor([346]), tensor([346]), tensor([290])],
  'extra_info': {'box_start': [tensor([72]), tensor([104]), tensor([95])],
   'box_end': [tensor([221]), tensor([276]), tensor([217])]}}]

I noticed in the inverse_transforms_and_test_time_augmentations.ipynb tutorial that one of the transforms has a parameter as_tensor_output=False, and I wonder if this is a requirement for inverse transforms to work. The transform from that tutorial is below:

train_transforms = Compose(
    [
        LoadImaged(keys),
        Lambdad("label", lambda x: (x > 0).astype(
            np.float64)),  # make label binary
        RandAffined(
            keys,
            prob=1.0,
            spatial_size=(300, 300),
            rotate_range=(np.pi / 3, np.pi / 3),
            translate_range=(3, 3),
            scale_range=((0.8, 1), (0.8, 1)),
            padding_mode="zeros",
            mode=("bilinear", "nearest"),
            as_tensor_output=False,
        ),
        CropForegroundd(keys, source_key="image"),
        DivisiblePadd(keys, 16),
        ScaleIntensityd("image"),
        ToTensord(keys),
    ]
)

If as_tensor_output=False is required, are there any negative effects with respect to processing speed in the remaining transformations?

Thank you!

[jpcenteno80]

Sorry, I think the issue is that you are not supposed to invert items that have gone through a dataloader. I was trying to test the functionality by iterating through a dataloader and that is why I am getting tensors out. If I just iterate through a list of dictionaries with 'image' and 'label' in them, and pass them through the list of transforms wrapped in a Compose object, things should be ok.

[wyli]

thanks for the detailed report, a bit surprised that numpy.maximum(torch.zeros(1),1) returns tensor([1.])

MONAI/monai/transforms/croppad/array.py

Line 465 in feb3a33

pad_to_start = np.maximum(-box_start, 0)

[rijobro]

Hi @jpcenteno80.

Firstly, as_tensor_output is there because RandAffined can output the result as either numpy or torch. However, the transform that follows it (CropForegroundd) requires input as numpy. This is why we set as_tensor_output=False.

Secondly, I think it's a completely legitimate use case to want to perform the inverse of a batch of data. We have the BatchInverseTransform for that, which is also detailed in the notebook you referred to.

If using BatchInverseTransform doesn't work for you, do you think you could create a minimum working example? I imagine it would only need CropForegroundd as transform as that's the one that's creating the issue. Perhaps we just need to modify the padder (used as the inverse to the crop) so that it accepts a list of singleton torch tensors, for example.

[jpcenteno80]

Thanks for the explanation regarding as_tensor_output... I missed BatchInverseTransform while going through that tutorial. It works as expected. Thanks for the quick reply!