amsgrad

Modified (corrected) Adam

Refencences

1. Fixing Weight Decay Regularization in Adam.
2. On the Convergence of Adam and Beyond.