DeepSpeed Checkpoints

Checkpoints made with DeepSpeed ZeRO Stages 2 and 3 will be stored in deepspeed checkpoint folder As such, they will require DeepSpeed as a dependency in order to resume from or generate with. See the deespeed conversion script for details on how to convert your ZeRO stage 2/3 checkpoint to a single file. If using a single GPU, consider running with apex automatic mixed precision instead for a similar speedup to ZeRO.