Skip to content

Latest commit

 

History

History
5 lines (3 loc) · 352 Bytes

README.md

File metadata and controls

5 lines (3 loc) · 352 Bytes

An Implementation for the optimizer LAMB with Tensorflow 2.0

We use pure python code inherted from tensorflow.keras.optimizers.Adam to implement the LAMB algorithm ( Layerwise Adaptive Large Latch optimization).

This implementation uses L2 regularization for all weights excepted for layernorm and all biases.