Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Quantization] enable multi-backend bitsandbytes #10395

Open
sayakpaul opened this issue Dec 27, 2024 · 4 comments · May be fixed by #10574
Open

[Quantization] enable multi-backend bitsandbytes #10395

sayakpaul opened this issue Dec 27, 2024 · 4 comments · May be fixed by #10574

Comments

@sayakpaul
Copy link
Member

Similar to huggingface/transformers#31098

@siddhant230
Copy link

siddhant230 commented Dec 29, 2024

Hey I went through the PRs, seemingly its straightforward, but, I might be completely mistaken here (given I have very less knowledge of diffusers backend), I could pick this up. It would be helpful if someone tags along as well.

My understanding

  1. change in src/diffusers/quantizers/bitsandbytes/utils.py -> addition of multi-backend-validation
  2. Usage of validation in bnb_quantizer.py

(edit)
3. Also adding the relevant tests

Doubt:
As in bits&bytes PR here : 31098 : src/transformers/utils/init.py do we also need to account for others backends like mps, mlu, etc?

Thanks.

@sayakpaul sayakpaul changed the title enable multi-backend bitsandbytes [Quantization] enable multi-backend bitsandbytes Dec 30, 2024
@sayakpaul
Copy link
Member Author

Thanks for your interest! For this PR, we would assume you have sufficient familiarity with diffusers. The steps you outlined seem fine to me, though.

do we also need to account for others backends like mps, mlu, etc?

We don't.

@hlky hlky linked a pull request Jan 14, 2025 that will close this issue
Copy link

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@github-actions github-actions bot added the stale Issues that haven't received updates label Jan 26, 2025
@vladmandic
Copy link
Contributor

ping to remove stale...

@github-actions github-actions bot removed the stale Issues that haven't received updates label Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants