Skip to content

SunMarc/bitsandbytes

This branch is 15 commits ahead of, 194 commits behind bitsandbytes-foundation/bitsandbytes:main.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

b3a9bd5 · Feb 21, 2024
Feb 19, 2024
Feb 4, 2024
Feb 5, 2024
Feb 21, 2024
Feb 14, 2024
Feb 21, 2024
Feb 1, 2024
Feb 5, 2024
Feb 1, 2024
Feb 21, 2024
Oct 6, 2021
Feb 4, 2024
Feb 21, 2024
Feb 4, 2024
Feb 5, 2024
Feb 1, 2024
Jan 8, 2024
Feb 6, 2024
Oct 6, 2021
Oct 27, 2022
Oct 6, 2021
Oct 6, 2021
Feb 4, 2024
Feb 5, 2024
Jan 30, 2024
Feb 5, 2024
Feb 5, 2024
Feb 4, 2024
Feb 1, 2024
Jan 23, 2024
Feb 5, 2024
Feb 1, 2024
Feb 5, 2024
Feb 5, 2024
Feb 5, 2024

Repository files navigation

bitsandbytes

The bitsandbytes library is a lightweight Python wrapper around CUDA custom functions, in particular 8-bit optimizers, matrix multiplication (LLM.int8()), and 8 & 4-bit quantization functions.

The library includes quantization primitives for 8-bit & 4-bit operations, through bitsandbytes.nn.Linear8bitLt and bitsandbytes.nn.Linear4bit and 8-bit optimizers through bitsandbytes.optim module.

There are ongoing efforts to support further hardware backends, i.e. Intel CPU + GPU, AMD GPU, Apple Silicon. Windows support is quite far along and is on its way as well.

Please head to the official documentation page:

https://huggingface.co/docs/bitsandbytes/main

License

The majority of bitsandbytes is licensed under MIT, however small portions of the project are available under separate license terms, as the parts adapted from Pytorch are licensed under the BSD license.

We thank Fabio Cannizzo for his work on FastBinarySearch which we use for CPU quantization.

About

8-bit CUDA functions for PyTorch

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 58.4%
  • Cuda 27.3%
  • C++ 8.4%
  • C 3.6%
  • Shell 1.3%
  • Makefile 1.0%