Skip to content

请问支持官网的Qwen1.5 32B awq模型吗? #2363

Answered by lvhan028
lai-serena asked this question in Q&A
Discussion options

You must be logged in to vote
  "quantization_config": {
    "bits": 4,
    "group_size": 32,
    "modules_to_not_convert": null,
    "quant_method": "awq",
    "version": "gemm",
    "zero_point": true
  },

the quant_config of qwen1.5-32b is listed above.
The group_size is 32, which is not supported by lmdeploy.
Currently, lmdeploy only supports group_size 128

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by lvhan028
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants