Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CPU]Implement key cache quantization by channel for SDPA #28259

Draft
wants to merge 12 commits into
base: master
Choose a base branch
from
Draft

[CPU]Implement key cache quantization by channel for SDPA #28259

wants to merge 12 commits into from
+1,477 −443

Conversation

zhangYiIntel
Copy link
Contributor

Details:

  • Implement by-channel quantization for SDPA
  • Add test case for by-channel quantization for SDPA

Tickets:

@github-actions github-actions bot added category: inference OpenVINO Runtime library - Inference category: CPU OpenVINO CPU plugin category: build OpenVINO cmake script / infra category: Python API OpenVINO Python bindings category: CPP API OpenVINO CPP API bindings labels Jan 3, 2025
@zhangYiIntel zhangYiIntel removed category: inference OpenVINO Runtime library - Inference category: build OpenVINO cmake script / infra category: Python API OpenVINO Python bindings category: CPP API OpenVINO CPP API bindings labels Jan 3, 2025
@github-actions github-actions bot added category: inference OpenVINO Runtime library - Inference category: build OpenVINO cmake script / infra category: Python API OpenVINO Python bindings category: CPP API OpenVINO CPP API bindings labels Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
category: build OpenVINO cmake script / infra category: CPP API OpenVINO CPP API bindings category: CPU OpenVINO CPU plugin category: inference OpenVINO Runtime library - Inference category: Python API OpenVINO Python bindings do not merge do_not_review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@zhangYiIntel