how do you process 50 batches before giving the loss output

[neurosynapse]

Hello,

It would be nice if you could help me to understand how its implemented that you process 50 batches at once. Here is the example of retinanet training, batch size is 4 but the code loads 50 batches in one iteration and then gives the loss output. Is the loss calculated for each batch separately and then you get average at the end? How did you implement this training process as I see it greatly improves the training speed compared to single batch process code variants. Could you explain why? Its a little bit hard to debug the code because of the register usage. Thanks a lot!

Here I print the input shape for the model, as you can see it loads up 50 batches and then gives the loss:

Best regards,
Roberts

[nijkah]

Hi @Franko9999.

Actually, mmdetection and related frameworks process each batch as one step (e.g. Model.forward(), loss.backward(), optimizer.step()).
The logger saves related information in each step, and just shows averaged metrics when the interval is met.

mmdetection/configs/_base_/default_runtime.py

Line 4 in e71b499

interval=50,

It would be nice to check Runner in mmcv if you are curious about its detailed logic.

[neurosynapse]

Hi,

Yes, I see. Thanks for the reply. Upvoted.

Best regards,
Roberts