My augmentations feel wrong

[p-iosifidis]

Hello guys, splendid work with the monai framework, i've been enjoying writing with it for the past couple of months.
I've got a terrible predicament with data augmentations though... and i think that i'm doing something wrong but i can't spot it.

For context: i have preclinical CMR experimental data with myocardium and left ventricle annotations.
The data are are 96 3D blocks 256x256x(7-9) which i preprocess using a 1% value clip to get rid of the various maximization of a 16bit nifti image, then N4ITK bias field correction and finally crop these to 128x128.
As for the depth axis i'm zero padding in my custom dataloader to 128x128x16.
I figured i'd start by using Unet with residual blocks to get a baseline, got about an 80% mean dice.
Then i though to myself, okay time to blast it with some vision transformers and see what the clout is about.
This gave me an 87.51% mean dice which made me really happy!
Then i thought i'd start using data augmentations due to the limited data I've had. The problem is it makes the mean dice deteriorate, hard. As such:

The transforms that i'm using are these:

 transforms = Compose([
                RandFlipd(keys=['img', 'seg'],prob=.5, spatial_axis=[0]),
                RandFlipd(keys=['img', 'seg'],prob=.5, spatial_axis=[1]),
                RandFlipd(keys=['img', 'seg'],prob=.5, spatial_axis=[2]),
                RandZoomd(keys=['img', 'seg'], prob=.5, min_zoom=.9, max_zoom=1.1, mode=("bilinear", "nearest")),
                RandAffined(keys=['img', 'seg'], prob=.2, translate_range=(10, 10, 1), mode=("bilinear", "nearest"), padding_mode='zeros',
                    rotate_range=.5, scale_range=None, shear_range=None),
                RandGaussianNoised(keys=['img'], mean=.2, prob=.1),
                RandHistogramShiftd(keys=['img'], prob=.1),
                RandShiftIntensityd(keys=['img'], offsets=.1, prob=.1),
                NormalizeIntensityd(keys=['img'])])

I verified they worked by taking a good look at the output shapes and did a visual inspection on a 2D slice (got a very cool looking spider head!):

Thus my question is, am i doing aggressive over-augmentation or are my augmentation thoughts completely wrong?
Thanks in advance for any pointers to troubleshooting this.

[KumoLiu]

Hi @p-iosifidis, thanks for your interest here.
Possible issue is that your transforms are not optimal such as for RandZoomd and RandAffined, ensure that the zoom and affine transformations are not too aggressive. For example, zooming out too much can make the important structures too small, and large translations or rotations can distort the anatomical structures.
You may need a step-by-step analysis: apply one augmentation at a time and observe its impact on the validation performance. This will help identify which augmentation is causing the issue. Also try fine-tune the parameters of each transformation

Hope it helps, thanks.

[p-iosifidis]

I'm currently trying that, reducing the augmentations just to just the bare minimum to see if the model would freak out on me again:

The results were not so terrible! Although I'm afraid at the time constraints i do have and the training time I'll probably have to settle a silver lining. I'm a little bit curious however! How does one troubleshoot these problems beyond taking blocks (as in transformations and parameters) in and out and tweaking them? Is it just something that will come with experience down the line?

[KumoLiu]

I guess so, you may also need to be familiar with your dataset.

[p-iosifidis]

Understood, I'll attempt to run a 3D unet which takes about 15mins per 100 epochs to train with each one of the augmentations per run, hopefully i can see which one is malfunctioning and iterate on improving it. Thanks for the input @KumoLiu, i'll update in case i find something useful to add as a thought process.

[p-iosifidis]

Okay, i run all the models last night and you were right @KumoLiu, rotations were causing the brunt of my problems!
Last orange line is my 0.5 run, the grey one that's above it is with 0.2 and i tried one with 0.1 across all three axes.

I'm also wondering if there is any merit of instead bundling all these into one compose, or setting the compose a one of and all the augmentations at 0.5 probability and then adding a normalize on the compose end.

[KumoLiu]

Hi @p-iosifidis, I'm not sure whether I get your point. But if you find which transform causes the issue, just modify the hyperparameter or remove the transform. Then you can just include all the transforms into one Compose.
And BTW, after compose, the image may also not augment as expected, you may also need to check it. For how to check the data with the composed transforms, you can set prob=1.
Thanks.

[p-iosifidis]

Well the first thing i did, was have a compose with all the augmentations. Turned out like so (grey line):

The green line is a compose with a OneOf with all the augmentations except the normalize one (i put that in the compose).
Still noisy but i think now it just needs finetunning. Cheers.