Randomly crop out batch images and sliding-window inference

[ELKYang]

Hello, May I ask if there is any paper or material that can support the "Medical image data volume may be too large to fit into GPU memory. A widely-used approach is to randomly draw small size data samples during training and run a ‘sliding window’ routine for inference." mentioned in "4. Randomly crop out batch images based on positive/negative ratio"

For example, if I use the RandCropByPosNegLabel transform, How can I make sure that every place containing the target area is collected? If not, does that mean I can't make the best use of my data?

[rijobro]

Hi, the text in the screenshot that you've attached is suggested two different approaches: one for training and another for validation/testing/inference.

If you have an image that is, for example 1024,1024,1024, and it's too big for your GPU, then you can perform your training using smaller sections of this image, e.g., by using RandCropByPosNegLabel. Each time you move through your training loop, you'll get a new random subsection of your image to train with.

When it comes to validation/testing/inference, you want put the whole image through your network. However, you can do this bit-by-bit by using sliding_window_inference. This will infer the whole image by breaking it up into chunks.

The 3d spleen segmentation notebook gives examples of both of these functionalities: https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/spleen_segmentation_3d.ipynb.

I hope this answers your question as to how to be sure that all of your image will be passed through the model at inference time. Let me know if you want any further clarification.

[ELKYang]

Thank you very much for your answers to my questions, and thank you for your great contributions and open source spirit in the field of medical imaging.

But I think you may have a certain misunderstanding of my question. Sliding_window_inference is a very effective method for medical image processing, but the focus of my question lies in the training process rather than the inference process.

In the training process, we use RandCropByPosNegLabel to randomly crop large medical images into small fixed patches and input them into the network, although every time I pass the training loop, I will get a new random sub-part of the image for training.

But since only a small piece is randomly taken each time, how can I ensure that RandCropByPosNegLabel makes full use of my training data? Assuming that this kind of random sampling is regarded as a kind of pre-training, can it be proved that this kind of random sampling must be better than the method of traversing the entire 3D image like GridPatchDataset? Or is there any literature or experimental support?

Thanks again for your reply。

[rijobro]

Ah, sorry I misunderstood. How about this for a citation on using random crops for training: https://arxiv.org/abs/1409.4842.

As to "how can I make sure that RandCropByPosNegLabel makes full use of my training data?", I guess you can't. If the fraction of cropped image compared to the original is big, and if there are many epochs, then you would hope that each of the voxels will have been sufficiently processed by the network. If you only have comparatively small crops and don't pass through the data many times, then I'm sure results will be sub-optimal. No citations off the top of my head for that claim, though.

As to a comparison between RandCropByPosNegLabel and GridPatchDataset, I don't have anything for that, either. Sorry!

[ELKYang]

Thank you very much for your reply. I think what you said is consistent with my idea, that is, it is impossible to ensure that all voxels are traversed. I will cut as large a voxel block as possible within the GPU memory.

At the same time, I found this article(link) proved that the method of randomly cropping small voxels is better than the method of cropping the original image into fixed blocks, that is, the comparison of RandCropByPosNegLabel and GridPatchDataset on MRI images.

Thanks again!