Skip to content

Commit 64f43f2

Browse files
committed
fix bug
Signed-off-by: Mengni Wang <[email protected]>
1 parent fbb92ee commit 64f43f2

File tree

1 file changed

+9
-7
lines changed

1 file changed

+9
-7
lines changed

docs/quantization.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,13 @@ Quantization
44
1. [Quantization Introduction](#quantization-introduction)
55
2. [Quantization Fundamentals](#quantization-fundamentals)
66
3. [Get Started](#get-started)
7+
78
3.1 [Post Training Quantization](#post-training-quantization)
9+
810
3.2 [Specify Quantization Rules](#specify-quantization-rules)
11+
912
3.3 [Specify Quantization Recipes](#specify-quantization-recipes)
13+
1014
3.4 [Specify Quantization Backend and Device](#specify-quantization-backend-and-device)
1115
4. [Examples](#examples)
1216

@@ -124,14 +128,12 @@ quantize(model, q_model_path, qconfig)
124128
Neural Compressor support specify quantization rules by operator name. Users can use `set_local` API of configs to achieve the above purpose by below code:
125129

126130
```python
127-
fp32_config = config.GPTQConfig(weight_dtype="fp32")
128-
quant_config = config.GPTQConfig(
129-
weight_bits=4,
130-
weight_dtype="int",
131-
weight_sym=False,
132-
weight_group_size=32,
131+
op_config = config.StaticQuantConfig(per_channel=False)
132+
quant_config = config.StaticQuantConfig(
133+
per_channel=True,
133134
)
134-
quant_config.set_local("/h.4/mlp/fc_out/MatMul", fp32_config)
135+
quant_config.set_local(
136+
"/h.4/mlp/fc_out/MatMul", op_config)
135137
```
136138

137139

0 commit comments

Comments
 (0)