8 bit quantization
#514
Replies: 3 comments
-
|
Yeah, int8 quantization would be of interest for me as well! |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Any update on this? |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I just tried it with the int4 phi-3 onnx model and it worked fine if that's any help. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
Are 8 bit quantization methods planned to be supported by ONNXRuntime-genai in the future?
Thanks,
Beta Was this translation helpful? Give feedback.
All reactions