Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Dev] Improve Dequant performance on CUDA Simt #189

Merged
merged 4 commits into from
Sep 20, 2024

Conversation

LeiWang1999
Copy link
Contributor

This pull request includes several changes to improve error handling, code clarity, and functionality in the bitblas module. The most important changes include adding error handling when loading runtime modules, fixing a typo in comments, introducing alignment factors for vectorization, and adding new methods for fetching data to local memory.

Error Handling:

  • bitblas/cache/operator.py: Added error handling for loading runtime modules, logging an error message if the module fails to load.

Code Clarity:

Functionality Enhancements:

  • bitblas/gpu/matmul.py: Introduced an align_factor to handle unaligned K and BK values, and expanded the vectorization factor. [1] [2]
  • bitblas/gpu/matmul.py: Added a new method fetch_to_local to fetch data to local memory with alignment considerations.
  • bitblas/ops/operator.py: Modified update_runtime_module to handle cases where rt_mod is None and added conditional evaluation of time_evaluator.

@LeiWang1999 LeiWang1999 merged commit 6333d3b into microsoft:main Sep 20, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant