How to convert segmented image size to original image size?

[KarenPava24]

Hi there! I'm working on 3D Multi-organ Segmentation with UNETR (BTCV Challenge), and I already have the segmentations, I even converted them to a 3D volume to visualize them in MRIcron, but I need to extract the volumes of each organ and when I do it, those volumes are incorrect due to the transformations made. Therefore I need my segmented image to contain the size of the original image in order to have the real volumes of the organs. I convert the segmentation tensor with the transformed image header but it converts the voxels and I need the size. I have not been able to do it. Any idea what I can do?

Thank you in advance for your help

This is mi code

test_transforms = Compose(
[
LoadImaged(keys="image"),
EnsureChannelFirstd(keys="image"),
Orientationd(keys=["image"], axcodes="RAS"),
Spacingd(
keys=["image"],
pixdim=(1.5, 1.5, 2.0),
mode="bilinear",
),
ScaleIntensityRanged(keys=["image"], a_min=-175, a_max=250, b_min=0.0, b_max=1.0, clip=True),
CropForegroundd(keys=["image"], source_key="image"),
]
)

#Path for the image and pre-training model
img_name = input_path
model_fname = model_path
print(img_name)

#new path constructed for the transformed image.
img_namef = f'{output_path}/{os.path.splitext(os.path.basename(img_name))[0].rsplit(".", 1)[0]}_CT_transfda.nii.gz' 


#Image transformation
test_ds = test_transforms({"image": img_name }) 
transf_image = test_ds["image"] # Imagen Transformada
transf_image = transf_image.squeeze().detach().cpu().numpy()
nifti_transf = nib.Nifti1Image(transf_image, affine=None)
transf_path = nib.save(nifti_transf, img_namef)
transf_path = img_namef

#Segmentation with CPU
device = torch.device("cpu")

#Modelo UNETR para segmentar
model = UNETR(
    in_channels=1,
    out_channels=14,
    img_size=(96, 96, 96),
    feature_size=16,
    hidden_size=768,
    mlp_dim=3072,
    num_heads=12,
    pos_embed="perceptron",
    norm_name="instance",
    res_block=True,
    dropout_rate=0.0,
).to(device)


loss_function = DiceCELoss(to_onehot_y=True, softmax=True)

optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4, weight_decay=1e-5)

#Load the trained model
model.load_state_dict(torch.load(model_fname))
model.eval()

#Segmentation
with torch.no_grad():
  img_name = os.path.split(test_ds["image"].meta["filename_or_obj"])[1]
  img = test_ds["image"]
  test_inputs = torch.unsqueeze(img, 1).cpu()
  test_outputs = sliding_window_inference(test_inputs, (96, 96, 96), 4, model, overlap=0.8)

#Save the tensor of the segmentation
seg_vol = torch.argmax(test_outputs, dim=1).detach().cpu().numpy()
seg_vol = seg_vol[0,:,:,:]
output_tensor = seg_vol

#Path and name for the segmented image
seg_namef =f'{output_path}/{os.path.splitext(os.path.basename(img_name))[0].rsplit(".", 1)[0]}_CT_labeled.nii.gz' 

#Converting the segmentation tensor with the transformed image header
original_image = nib.load(transf_path)
original_header = original_image.header
seg_image = nib.Nifti1Image(output_tensor, original_image.affine, original_header)
nib.save(seg_image, seg_namef)

[KumoLiu]

Hi @KarenPava24, thanks for your interest here.
You can try to use Invertd to return the image to its original space.
For more details, you can refer to this tutorial.

• https://github.com/Project-MONAI/tutorials/blob/main/modules/inverse_transforms_and_test_time_augmentations.ipynb

Here are also many end-to-end pipelines using Invertd to do the evaluation on original image spacings.

• https://github.com/Project-MONAI/tutorials/blob/main/3d_segmentation/brats_segmentation_3d.ipynb
• https://github.com/Project-MONAI/tutorials/blob/main/3d_segmentation/spleen_segmentation_3d.ipynb

Hope it helps, thanks!

[KarenPava24]

Hi @KumoLiu. I'm still working on it.
I am following this tutorial https://colab.research.google.com/github/Project-MONAI/tutorials/blob/main/modules/inverse_transforms_and_test_time_augmentations.ipynb#scrollTo=6D-cU7IdCUnF and I did the inverse transform of the transformed image, from which I got the original image, but something is wrong when I try to do the same with the segmentation.
My segmentation is done in a different way than in this tutorial, therefore, I have not been able to apply the inverse transform. Any idea what I can do? I really need to apply the inverse transform to my segmentatiom. Thank u in advance for helping me

These are the transforms that I'm applying

test_transforms = Compose([
LoadImaged(keys="image"),
EnsureChannelFirstd(keys="image"),
Orientationd(keys=["image"], axcodes="RAS"),
Spacingd(
keys=["image"],
pixdim=(1.5, 1.5, 2.0),
mode="bilinear",
),
ScaleIntensityRanged(keys=["image"], a_min=-175, a_max=250, b_min=0.0, b_max=1.0, clip=True),
CropForegroundd(keys=["image"], source_key="image"),
])

These lines refer to the inverse transformation applied to the transformed image(it works)

test_ds= test_transforms({"image": img_name })
imagen_transformada = test_ds["image"]
with allow_missing_keys_mode(test_transforms):
inverted_image = test_transforms.inverse(test_ds)["image"]

These are the lines of my segmentation

loss_function = DiceCELoss(to_onehot_y=True, softmax=True)
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4, weight_decay=1e-5)
model.load_state_dict(torch.load(model_fname))
model.eval()
with torch.no_grad():
img_name = os.path.split(test_ds["image"].meta["filename_or_obj"])[1]
img_test = test_ds["image"]
test_inputs = torch.unsqueeze(img_test, 1).cpu()
test_outputs = sliding_window_inference(test_inputs, (96, 96, 96), 4, model, overlap=0.8)

and these are the lines in which I try to do the inverse transform of the segmentation but it doesn't work

seg = test_outputs[0].detach().cpu()
seg.applied_operations = test_ds["image"].applied_operations
seg_dict = {"image": seg}
with allow_missing_keys_mode(test_transforms):
inverted_seg = test_transforms.inverse(seg_dict)

This is the part of the code that I'm following

for file in np.random.choice(val_files, size=5, replace=False):
unmodified_data = minimal_transforms(file)
transformed_data = val_transforms(file)
_img = pad_list_data_collate([transformed_data["image"].to(device)])
seg = infer_seg(_img, model)[0].detach().cpu()
seg.applied_operations = transformed_data["label"].applied_operations
seg_dict = {"label": seg}
with allow_missing_keys_mode(val_transforms):
inverted_seg = val_transforms.inverse(seg_dict)
to_imshow.append(

and this is the error that I got

---

KeyError Traceback (most recent call last)
File ~/miniconda3/envs/my_monay/lib/python3.9/site-packages/monai/transforms/transform.py:90, in apply_transform(transform, data, map_items, unpack_items, log_stats)
89 return [_apply_transform(transform, item, unpack_items) for item in data]
---> 90 return _apply_transform(transform, data, unpack_items)
91 except Exception as e:

File ~/miniconda3/envs/my_monay/lib/python3.9/site-packages/monai/transforms/transform.py:54, in _apply_transform(transform, parameters, unpack_parameters)
52 return transform(*parameters)
---> 54 return transform(parameters)

File ~/miniconda3/envs/my_monay/lib/python3.9/site-packages/monai/transforms/croppad/dictionary.py:318, in Cropd.inverse(self, data)
317 for key in self.key_iterator(d):
--> 318 d[key] = self.cropper.inverse(d[key])
319 return d

File ~/miniconda3/envs/my_monay/lib/python3.9/site-packages/monai/transforms/croppad/array.py:883, in CropForeground.inverse(self, img)
882 # we moved the padding info in the forward, so put it back for the inverse
--> 883 pad_info = transform[TraceKeys.EXTRA_INFO].pop("pad_info")
884 img.applied_operations.append(pad_info)

KeyError: 'pad_info'

The above exception was the direct cause of the following exception:
...
112 else:
113 _log_stats(data=data)
--> 114 raise RuntimeError(f"applying transform {transform}") from e

RuntimeError: applying transform >
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...

[KumoLiu]

Hi @KarenPava24, from the log seems you mixing up images and labels. Could you please try:

post_transforms = Compose(
    [
        Invertd(
            keys="seg",
            transform=val_org_transforms,
            orig_keys="image",
            meta_keys="seg_meta_dict",
            orig_meta_keys="image_meta_dict",
            meta_key_postfix="meta_dict",
            nearest_interp=False,
            to_tensor=True,
            device="cpu",
        ),
        Activationsd(keys="pred", sigmoid=True),
        AsDiscreted(keys="pred", threshold=0.5),
    ]
)

with torch.no_grad():
    for val_data in val_org_loader:
        val_inputs = val_data["image"].to(device)
        val_data["seg"] = inference(val_inputs)
        val_data = [post_transforms(i) for i in decollate_batch(val_data)]

I hope it helps, thanks!