Auto3DSeg vs nnUNet

[FrancescoLR]

Hi,
First of all, thanks for the new features in v1.0!
Auto3DSeg in particular looks great. I am wondering if you have performed some head-to-head comparisons of Auto3DSeg and the nnUNet for different tasks? They are of course quite similar as end-to-end segmentation frameworks, but it seems that Auto3DSeg can be better customized.
Thank you!

[yujanshrestha]

I am wondering the same thing! I have used nnUnet on other projects but I am not happy with how big of a black box it is. I love how Monai is assimilating so many good ideas under one roof. Thanks a bunch team!

[kretes]

One can now check the paper from nnUnet authors: https://arxiv.org/pdf/2404.09556.pdf where they compare nnunet with auto3dseg and a lot more

[yujanshrestha]

Thank you @kretes

For convenience, here is a relevant quote from the paper.

nnU-Net is the state-of-the-art segmentation framework. We find that
none of the three methods featured in Auto3DSeg reaches the original nnU-Net
baseline ("org.") performance, indicating a substantial disadvantage due to the
underlying Auto3DSeg framework. This negative gap occurs despite significantly
lower VRAM usage and training time of the nnU-Net baseline. When comparing
the two frameworks with an identical method (SwinUNETR), nnU-Net wins in
5 out of 6 datasets. Following an official Auto3DSeg tutorial [2] we improved the
results via manual changes in configuration and further increasing its computing
budget, but failed to reach competitive performance (see Appendix Table 2).
Taken together, while Auto3DSeg can be pushed to produce state-of-the-art results, as evidenced by its recent challenge wins, its out-of-the-box capabilities do not match nnU-Net.