Is non-interactive pre-training beneficial for interactive segmentation models?

[Zrrr1997]

Dear MONAI Label team,

thank you for the great project! I have been using MONAI Label to develop my own custom interactive segmentation models, similar to DeepEdit and DeepGrow, and I was wondering about something very simple, but not much discussed in the literature.

Does non-interactive pre-training help when training interactive segmentation models? For example, if we train DeepEdit for 90 epochs without any clicks (just zero out the guidance maps) and then fine-tune for 10 epochs with 10 clicks per iteration would that give similar results as training with 10 clicks per iteration for 100 epochs? I can imagine that the model might learn the correlation between its errors and the positions of new clicks during the first epochs but have not seen any experiments regarding this.

I would be happy to hear about your experience with this since training from scratch for every new interactive configuration takes a lot of time and is not very scalable/sustainable. I saw that the original DeepEdit paper considered using 0%, 25%, and 50% click-free iterations, but I have not seen a mix of, e.g., 100% click-free for pre-training and then 0% click-free for fine-tuning.

Best,
Zdravko

[tangy5]

I think so, starting from a pre-trained weights should be better, as the model see more data, pre-training tasks such as supervised or self-supervised approaches can help the model converge faster on downstream task and achieve better performance.

But be aware of the domain gaps, if the pre-training data and fine-tuning data are unrelevent, there could be more challenges for the model.

[Zrrr1997]

Thank you! I was considering the simplest scenario where you perform both pre-training and fine-tuning on the same dataset, eliminating the challenge of domain shift. My question was whether the early iterations are critical, as they often involve learning low-level features. For example, in the case of guidance signals, the model may learn in its early iterations that a click in the signal indicates a region where it should refine its segmentation.