Training loss with image augmentations

[ghost]

Hi,

I'm using UNet following the framework laid out in the Spleen tutorial.
When using the same pipeline of transformations as laid out in the Spleen tutorial, after a certain number of epochs I have a maximum validation dice score of 0.8195 with associated training loss of 0.0881 and validation loss of 0.0933 for that epoch:

To reduce overfitting and aid generalisation I have tried adding RandAdjustContrastd and Rand3DElasticd.
The equivilant scores for the metrics are: val dice score of 0.8324, training loss of 0.3154 and validation loss: 0.0875:

The validation values are similar enough between the runs, but what I'm interested in is why the training loss is now plateuing at ~0.3 rather than <0.1. I'd expect the random augmentations to slow down learning, but it doesn't look like the loss is just going down more slowly, it seems to not be improving with more epochs.

• Why has the training loss plateued at such a higher level?
• How come the dice score though is still good?
• Plus, any ideas for why only the training loss is effected and not the validation loss?

Any advice from more seasoned ML practitioners would be much apprecaited!

Thanks,

Jeff.

[rijobro]

I guess you're only applying the random transformations to the training data and not the validation? This might explain why your training loss isn't doing as well, but your dice score isn't as heavily affected. Have you visualised a selection of images after the random transformations? I've found I needed a bit of trial and error to get good magnitude and sigma values for the random elastic transformation.

[ghost]

Hi Richard, yes you're right! Correct answer, I'm only randomly transforming the training data, thanks.
Yep, I've visualised the images at different strength transformations but may need to have more of a play to ensure they're not too aggressive. I had hoped that with enough training, even for a relatively strong augmentation of the data I'd get good results but do you think this is not the case and I've selected too strong transofmrations: hence my error (with random transformations) is plateuing at a much higher level?

[rijobro]

Yeah, my guess is just that the transforms that you're applying are rather aggressive (just a guess though). You should be able to confirm this with something like this:

1. start with no extra transformations
2. add in the transformations with magnitudes of zero. check nothing changes
3. increase magnitudes of the transformations, checking that training loss evolves as expected. probably only increase magnitude of one of the transformations at any one time.
4. repeat step 3 until you're happy with training/validation metrics.

I'd recommend doing plenty of visualisations along the way to make sure you keep a handle on what you're images look like.

[ghost]

Sounds like a great plan, thanks so much for your suggestions Richard!