BigScience Workshop

All

35 repositories

data_tooling
Public
Tools for managing datasets for governance and training.
HTML
•
Apache License 2.0
•48•79•138•3•Updated Dec 23, 2024Dec 23, 2024
biomedical
Public
Tools for curating biomedical training data for large-scale language modeling
Python
•116•462•162•16•Updated Dec 9, 2024Dec 9, 2024
xmtf
Public
Crosslingual Generalization through Multitask Finetuning
bloom language-models zero-shot-learning multitask-learning t5 multilingual-nlp large-language-models instruction-tuning bloomz mt0
Jupyter Notebook
•
Apache License 2.0
•38•518•11•0•Updated Sep 22, 2024Sep 22, 2024
petals
Public
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
nlp bloom distributed-systems machine-learning deep-learning chatbot pytorch falcon transformer neural-networks
Python
•
MIT License
•525•9.3k•87•18•Updated Sep 7, 2024Sep 7, 2024
bigscience
Public
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
nlp training machine-learning models
Shell
•
Other
•101•981•13•8•Updated Jul 29, 2024Jul 29, 2024
Megatron-DeepSpeed
Public
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Python
•
Other
•221•1.3k•74•45•Updated Mar 20, 2024Mar 20, 2024
multilingual-modeling
Public
BLOOM+1: Adapting BLOOM model to support a new unseen language
Python
•
Apache License 2.0
•16•70•13•6•Updated Mar 2, 2024Mar 2, 2024
promptsource
Public
Toolkit for creating, sharing and using natural language prompts.
nlp machine-learning natural-language-processing
Python
•
Apache License 2.0
•354•2.7k•11•32•Updated Oct 23, 2023Oct 23, 2023
massive-probing-framework
Public
Framework for BLOOM probing
Python
•10•8•0•0•Updated Oct 17, 2023Oct 17, 2023
architecture-objective
Public
Python
•
Apache License 2.0
•309•93•4•5•Updated Jul 25, 2023Jul 25, 2023
metadata
Public
Experiments on including metadata such as URLs, timestamps, website descriptions and HTML tags during pretraining.
Python
•
Apache License 2.0
•12•30•25•13•Updated Jun 12, 2023Jun 12, 2023
lm-evaluation-harness
Public
A framework for few-shot evaluation of autoregressive language models.
Python
•
MIT License
•2k•102•7•8•Updated May 9, 2023May 9, 2023
data-preparation
Public
Code used for sourcing and cleaning the BigScience ROOTS corpus
multilingual large-language-models dataset
Jupyter Notebook
•
Apache License 2.0
•42•305•9•0•Updated Mar 20, 2023Mar 20, 2023
bibliography
Public
A list of BigScience publications
research bibtex bibliography
TeX
•
Apache License 2.0
•1•3•1•0•Updated Mar 13, 2023Mar 13, 2023
multilingual-modeling-1
Public
Python
•
Apache License 2.0
•16•2•0•0•Updated Dec 5, 2022Dec 5, 2022
t-zero
Public
Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)
Python
•
Apache License 2.0
•53•459•8•2•Updated Nov 5, 2022Nov 5, 2022
carbon-footprint
Public
A repository for `codecarbon` logs.
Jupyter Notebook
•6•10•1•0•Updated Nov 3, 2022Nov 3, 2022
pii_processing
Public
PII Processing code to detect and remediate PII in BigScience datasets. Reference implementation for the PII Hackathon
Python
•
Other
•6•8•7•1•Updated Oct 6, 2022Oct 6, 2022
lam
Public
Libraries, Archives and Museums (LAM)
Apache License 2.0
•7•82•34•0•Updated Oct 4, 2022Oct 4, 2022
model_card
Public
Apache License 2.0
•4•24•0•0•Updated Jul 11, 2022Jul 11, 2022
bloom-dechonk
Public
A repo for running model shrinking experiments
Python
•4•10•0•0•Updated Jun 21, 2022Jun 21, 2022
historical_texts
Public
BigScience working group on language models for historical texts
Jupyter Notebook
•6•8•0•1•Updated May 10, 2022May 10, 2022
evaluation
Public
Code and Data for Evaluation WG
Python
•
Other
•24•41•41•9•Updated May 4, 2022May 4, 2022
catalogue_data
Public
Scripts to prepare catalogue data
Jupyter Notebook
•
Apache License 2.0
•1•8•5•3•Updated Apr 25, 2022Apr 25, 2022
training_dynamics
Public
4•5•11•2•Updated Apr 22, 2022Apr 22, 2022
evaluation-robustness-consistency
Public
Tools for evaluating model robustness and consistency
Python
•
Other
•2•2•0•2•Updated Mar 9, 2022Mar 9, 2022
interpretability-ideas
Public
1•1•10•0•Updated Feb 27, 2022Feb 27, 2022
tokenization
Public
Python
•
Apache License 2.0
•2•11•1•2•Updated Feb 16, 2022Feb 16, 2022
datasets_stats
Public
Generate statistics over datasets used in the context of BS
Makefile
•1•2•0•0•Updated Feb 1, 2022Feb 1, 2022
transformers
Public
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.
Python
•
Apache License 2.0
•27k•6•0•0•Updated Nov 22, 2021Nov 22, 2021