Files

data
img
records
- 010425_SoftCap
- 011325_Fp8LmHead
- 011625_Sub3Min
- 011825_GPT2Medium
- 012625_BatchSize
- 020125_RuleTweak
- 020825_GPT2MediumWeightDecay
- 021425_GPT2MediumOptCoeffs
- 060624_AdamW
- 100924_SOAP
- 101024_Muon
  - eb5659d0-fb6a-49e5-a311-f1f89412f726.txt
  - train_gpt2.py
- 101324_llmc
- 101424_ModernArch
- 101724_DistributedMuon
- 101824_PyTorch25
- 102024_ScaleUp1B
- 102924_Optimizers
- 110324_UntieEmbed
- 110424_50Bruns
- 110624_ShortcutsTweaks
- 110824_CastBf16
- 110924_Replicateleloykun
- 111024_ScaleShortcuts
- 111024_UNetDoubleLr
- 111424_QuantizedFP4
- 111924_FlexAttention
- 112424_WindowWarmup
- 120424_ValueEmbed
- 120824_UNetValueEmbedsTweaks
- 121024_MFUTweaks
- 121724_SparsifyEmbeds
- 123124_Target350M
.gitignore
Dockerfile
LICENSE
README.md
pyproject.toml
requirements.txt
run.sh
train_gpt.py

101024_Muon

new Muon record

Oct 11, 2024

b356a1f · Oct 11, 2024

Name		Name	Last commit message	Last commit date
parent directory ..
eb5659d0-fb6a-49e5-a311-f1f89412f726.txt		eb5659d0-fb6a-49e5-a311-f1f89412f726.txt	new Muon record	Oct 11, 2024
train_gpt2.py		train_gpt2.py	new Muon record	Oct 11, 2024