Add fire finetuning#553

Draft

gkielian wants to merge 762 commits intokarpathy:masterfrom gkielian:add_fire_finetuning

+139,454-714

This pull request is big! We're only showing the most recent 250 commits

Commits on Jun 13, 2024

Add specific vizier running file for optimization
klei22
committed
Add working run_vizier modification of run_exp
klei22
committed
Convert boolean action to new format for compat
klei22
committed

Commits on Jun 15, 2024

Remove temporary test file
klei22
committed
Streamline run_vizier.py
klei22
committed
Remove leading spaces from train.py
klei22
committed
Add configuration json file to out_dir
klei22
committed
Add saving of best validation loss and iter file
klei22
committed
Copy meta.pkl to out_dir
klei22
committed
Add check for meta.pkl from out_dir to sample.py
klei22
committed
Add fast method for obtaining best validation loss
klei22
committed
Supress warnings in the run_vizier
klei22
committed
Add comments to ckpt saving and end action list
klei22
committed
Add --fast option for inspect checkpoints
klei22
committed
Add scripts compat with python-codes-25k
klei22
committed

Commits on Jun 16, 2024

Commits on Jun 17, 2024

Commits on Jun 22, 2024

Fix bug separating shuffle moveset with moveset
klei22
committed
Add trial argparse arg
klei22
committed
Upgrade progress bar
klei22
committed
Add create_datasets.sh for experiments
klei22
committed
Add latest iteration of parameter exploration
klei22
committed

Commits on Jun 23, 2024

Add quantized krmsnorm
klei22
committed
Add exploration for vizier
klei22
committed
Add option for no quantization
klei22
committed
Update ints to categories
klei22
committed

Commits on Jun 24, 2024

Commits on Jun 25, 2024

golden gen for decoder
Buck008
committed

Commits on Jun 28, 2024

update for golden gen
Buck008
committed

Commits on Jul 1, 2024

Add scripts compatible with the Newswire Dataset
klei22
committed

Commits on Jul 5, 2024

Add snac_converter.py for mp3 <-> text conversion
klei22
committed
Update prepare.py to include direct numeric tokens
klei22
committed
Add optional token boundary to sample.py
klei22
committed
Update sample.py to use write instead of append
klei22
committed
Add option for snac processing all files in a dir
klei22
committed
Add prepare.py softlink to snac folder
klei22
committed
Add example of how to create a listening sample
klei22
committed
Add helper files
klei22
committed
Update get_dataset.sh with tokenization
klei22
committed
Add sampling and training scripts
klei22
committed
Add progress bars to audio directory processing
klei22
committed
Add split_mp3s.py to help reduce mp3 size for gpu
klei22
committed
Organize imports
klei22
committed
Add sample.sh and train.sh for easy demo of vc
klei22
committed
Add README.md
klei22
committed

Commits on Jul 9, 2024

Commits on Jul 27, 2024

Add scripts compatible wtih smollm-corpus
klei22
committed

Commits on Aug 1, 2024

got rid of unnecessary linear quant
mmoffatt2
committed

Commits on Aug 4, 2024

Fix bug with MoE layer frequency
djlisbonne
committed

Commits on Aug 5, 2024

add test files for debug
xinyixuu
committed

Commits on Aug 6, 2024

Commits on Aug 8, 2024

Commits on Aug 9, 2024

Commits on Aug 11, 2024

Add implementation of Rotary Embeddings
klei22
committed

Commits on Aug 12, 2024

Commits on Aug 16, 2024

Commits on Aug 17, 2024

Commits on Aug 19, 2024

Commits on Aug 20, 2024

Commits on Aug 21, 2024

Commits on Aug 22, 2024

Commits on Aug 25, 2024

Commits on Aug 26, 2024

Commits on Aug 29, 2024

Fixed One Bug in FIRE - PR karpathy#246 v4
Mars-Cat2023
committed

Commits on Aug 31, 2024

Add MLP Expansion factor control and sweep
gkielian
committed

Commits on Sep 2, 2024

Merge pull request karpathy#251 from Mars-Cat2023/FIRE
gkielian
authored

Commits on Sep 3, 2024

Merge pull request karpathy#252 from gkielian/add_mlp_expansion_factor
klei22
authored

Commits on Sep 6, 2024

Add code for finetuning with FIRE
gkielian
committed