Multi-GPU training

[feriorior]

Thanks to the author for this excellent benchmark!
When I try to change the GPU devices setting, I get the following error:

RuntimeError: It looks like your LightningModule has parameters that were not used in producing the loss returned by training_step. If this is intentional, you must enable the detection of unused parameters in DDP, either by setting the string valuestrategy='ddp_find_unused_parameters_true'or by setting the flag in the strategy withstrategy=DDPStrategy(find_unused_parameters=True).

It might be some parameter just used for forward but not backward, which might result in this error.

[mengdaxu]

Thank you for your feedback. We appreciate your interest in our benchmark. As of now, we haven't experimented with multi-GPU training. We will investigate this issue.