Skip to content

hyintell/BLOOM-fine-tuning

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

BLOOM-fine-tuning

This project is for fine-tuning BLOOM. The repo contains:

Try it on Google Colab! Build

Installation

pip install -r requirements.txt

Data: alpaca

Training

alpaca

python finetune-alpaca.py \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --num_train_epochs 2 \
    --learning_rate 2e-5 \
    --fp16 True \
    --logging_steps 10 \
    --output_dir output

Progress

  • Add Google Colab example
  • Test bloom-560m
  • Test bloom-1b7
  • Test bloom-7b1
  • Support Evaluation