LoadImaged function

[vikashg]

Hello, I have my dataset file in the following format. I wrote my dataset in such a format because I want to use the cache dataset

{ "Train": [ { "image": "dicom_image.dcm", "label": [ "annotation_file.json" ] },
When I try to read the images using LoadImaged(keys=["image"], image_only=True) . I get the following output

{'image': NDArrayITKBase([[[0.],
                 [0.],
                 [0.],
                 ...,

                 ...,
                 [0.],
                 [0.],
                 [0.]]], dtype=float32), 'label': ['annotation_file.json']}

My question is if I use the keys="image", shouldn't the output be just the image (and not the contents of the other keys 'label'. Here is a working version of the code. I changed the filenames.

from monai.transforms import LoadImaged
train_data = {"image": 'dicom_image.dcm',
"label": ['annotations.json']}
a = LoadImaged(keys=["image"])(train_data)
print(a)

In an ideal scenario, I would like the Loading function to give the output as the images and a dictionary with the contents of the annotations.json file, but I dont think there is a support for a json file reader in LoadImaged.

Any thoughts on this issue.
Thanks
Vikash

[ebrahimebrahim]

My question is if I use the keys="image", shouldn't the output be just the image (and not the contents of the other keys 'label'.

When you apply a dictionary transform with a certain set of specified keys, the transform will be applied to the dictionary values with those, keys, and all other dictionary values are left untouched. Those other keys could perhaps be manipulated by a different transform later in your transform pipeline.

In an ideal scenario, I would like the Loading function to give the output as the images and a dictionary with the contents of the annotations.json file, but I dont think there is a support for a json file reader in LoadImaged.

You could write your own transform that loads the json file into a dictionary object, for example using the json library.

• If the transform is a simple function, you might be able to make it with monai.transforms.LambaD
• For full customization you can derive Transform base classes and make your own transform; it's done in this tutorial for example.

[vikashg]

Hi @ebrahimebrahim and @Nic-Ma Thanks for the comments, Yes keys="image" and keys=["image"] are the same. Also @ebrahimebrahim I confirmed what you said with one of my previous examples. So I have a followup question, I can write my own loader that returns the dictionary and inherits it from the Transforms. I am trying to figure out how to apply the same set of transforms to these parallel transform chains one for the image and one for the annotations file. Think of object detection as a use case where the annotations files contain the coordinates of the anchor points. If the image is rotated the labels should be rotated accordingly. Any thoughts on how can we implement this ?

[Nic-Ma]

Hi @dongyang0122 , could you please help share some ideas as you are working on the detection tasks?

Thanks in advance.

[vikashg]

@ebrahimebrahim Actually maybe neverming I see how MONAI wants to handle datasets

images = [fn["img"] for fn in filenames]
segs = [fn["seg"] for fn in filenames]

img_transform = Compose([LoadImage(image_only=True), AddChannel(), 
                         RandSpatialCrop((128, 128, 128), random_size=False), RandAdditiveNoise(), ToTensor()])
seg_transform = Compose([LoadImage(image_only=True), AddChannel(), 
                         RandSpatialCrop((128, 128, 128), random_size=False), ToTensor()])

ds = ArrayDataset(images, img_transform, segs, seg_transform)
im, seg = ds[0]
plt.imshow(np.hstack([im.numpy()[0, 48], seg.numpy()[0, 48]]))

It wants us to have a parallel transform chain for each datasets. In my case, there is some data sharing going on between different chains, the annotations are kind of dependent on the images. But I see how it can be implemented. Thanks for the comments.

[vikashg]

Thanks @Nic-Ma, I think if we can get some more insights it will be useful. Mostly if there is a mechanism for sharing some information between transform chains. For example some meta data from the image. For example, if I am rotating the image, the size and spacing information is contained in the image so it is not a problem, but the same information is missing in the annotations as they are just coordinates so I need to find a mechanism to pass that information.
Also if I am not wrong there is no support to load and manipulate geometric objects in monai yet right ?

[dongyang0122]

hi @vikashg, when you rotate the images, you may need to rotate or re-generate the boxes as well. It is not easy especially when rotating with arbitrary angles. We are currently working on a 3D medical image detection example in MONAI. Hope it can be released soon.