02.srt

1
00:00:00,000 --> 00:00:04,380
字幕生成：Galaxies 字幕校对：Not_Ur_77 

2
00:00:04,380 --> 00:00:07,100
嗨大家好我是ZOMI

3
00:00:07,100 --> 00:00:08,500
卷王来了卷王来了

4
00:00:08,500 --> 00:00:11,500
那今天呢给大家带来一个新的内容

5
00:00:11,500 --> 00:00:15,260
就是推理引擎或者推理系统里面的模型压缩

6
00:00:15,260 --> 00:00:19,060
也可以叫模型小型化呀或者模型轻量化这个工作

7
00:00:19,060 --> 00:00:22,900
那今天呢主要是给大家去带来一个新的内容

8
00:00:22,900 --> 00:00:24,900
模型的量化

9
00:00:26,100 --> 00:00:28,860
回到今天课程的主角

10
00:00:28,860 --> 00:00:33,660
今天的课程呢主要是去讲讲低比特量化

11
00:00:33,660 --> 00:00:35,360
那在低比特量化里面呢

12
00:00:35,360 --> 00:00:39,660
ZOMI呢可能会分开三节课给大家介绍的

13
00:00:41,160 --> 00:00:42,360
因为内容有点多

14
00:00:42,360 --> 00:00:45,060
所以我把低比特量化呢拆分成三节课

15
00:00:45,060 --> 00:00:48,360
每节课呢尽量控制在十分钟以内

16
00:00:48,360 --> 00:00:50,460
让大家听得比较愉快

17
00:00:50,460 --> 00:00:51,960
心情比较愉悦

18
00:00:51,960 --> 00:00:55,260
那第一个呢就是最基本的量化基础啊

19
00:00:55,260 --> 00:00:56,360
Base Concept

20
00:00:56,360 --> 00:00:58,660
第二个呢就是量化的原理

21
00:00:58,660 --> 00:01:02,960
去看看具体的量化的算法是怎么实现的

22
00:01:02,960 --> 00:01:05,160
到底低比特指的是啥

23
00:01:05,160 --> 00:01:07,260
低到哪个程度

24
00:01:07,260 --> 00:01:10,360
那接着呢将会在第二节课去讲讲

25
00:01:10,360 --> 00:01:14,660
QAT Quantization Aware Training  感知量化训练 

26
00:01:14,660 --> 00:01:17,660
一般都叫QAT感知量化

27
00:01:17,660 --> 00:01:18,960
为啥叫感知呢

28
00:01:18,960 --> 00:01:21,560
是因为在训练的过程当中

29
00:01:21,560 --> 00:01:24,760
去感知到需要量化的程度

30
00:01:24,760 --> 00:01:26,960
那接着呢在最后一节课

31
00:01:27,060 --> 00:01:29,860
会去讲讲训练后量化

32
00:01:29,860 --> 00:01:33,360
PTQ Post Training Quantization 

33
00:01:33,360 --> 00:01:36,360
训练完之后再做量化

34
00:01:36,360 --> 00:01:39,760
那在第三节课还有一个很重要的内容要补充的

35
00:01:39,760 --> 00:01:42,260
就是量化的部署

36
00:01:42,260 --> 00:01:45,460
不管前面的算法多复杂多炫酷也好

37
00:01:45,460 --> 00:01:47,460
最终都离不开一个话题

38
00:01:47,460 --> 00:01:51,760
怎么把网络模型经过量化后的部署起来

39
00:01:51,760 --> 00:01:54,960
所以这个话题这个内容也是非常重要

40
00:01:55,060 --> 00:01:56,660
而我搜了很多文章

41
00:01:56,660 --> 00:01:59,760
确实很少会把这些所有的内容串起来

42
00:02:01,460 --> 00:02:04,660
现在呢开始正式的内容

43
00:02:04,660 --> 00:02:08,260
首先第一个就是量化的基础哦

44
00:02:08,260 --> 00:02:09,360
上面这段废话呢

45
00:02:09,360 --> 00:02:11,460
我就不跟大家一起去读了

46
00:02:11,460 --> 00:02:12,260
数值呢

47
00:02:12,260 --> 00:02:15,260
在计算机里面其实有很多种表示方式

48
00:02:15,260 --> 00:02:15,860
那这里面呢

49
00:02:15,860 --> 00:02:22,560
一般都会有FP32 还有FP16 INT32 INT16 还有INT8 

50
00:02:22,560 --> 00:02:24,460
一般呢都会有这些指令

51
00:02:26,760 --> 00:02:30,560
是一般用来去做训练的一个精度

52
00:02:31,460 --> 00:02:32,060
很多时候呢

53
00:02:32,060 --> 00:02:33,860
会开启混合精度

54
00:02:33,860 --> 00:02:35,060
那所谓的混合精度呢

55
00:02:35,060 --> 00:02:39,660
一般是把FP32跟FP16混合到一起去训练的

56
00:02:39,660 --> 00:02:40,560
因为FP16呢

57
00:02:40,560 --> 00:02:42,760
它这样的内存位置或者占用的地址空间呢

58
00:02:42,760 --> 00:02:44,260
确实少了很多

59
00:02:45,860 --> 00:02:48,260
而一般指的模型量化呢

60
00:02:48,260 --> 00:02:50,760
除了把FP32降成FP16

61
00:02:50,760 --> 00:02:58,460
更多的是降到INT16 INT8 甚至INT4更低比特的一种表示  

62
00:02:58,460 --> 00:03:02,060
那这种呢 叫做模型量化的技术 

63
00:03:02,060 --> 00:03:08,060
把FP32 32位的比特转换成为更低比特的方式或者技术

64
00:03:08,060 --> 00:03:10,360
叫做模型量化

65
00:03:10,360 --> 00:03:10,960
接下来呢

66
00:03:10,960 --> 00:03:13,860
看一下神经网络有什幺特点

67
00:03:13,860 --> 00:03:14,760
那右边这个图呢

68
00:03:14,760 --> 00:03:16,460
已经看了好多遍了

69
00:03:16,460 --> 00:03:16,860
其实呢

70
00:03:16,860 --> 00:03:18,460
神经网络一般来说呢

71
00:03:18,460 --> 00:03:21,560
训练的时候数据的参数量会很大

72
00:03:21,560 --> 00:03:21,960
第二个

73
00:03:21,960 --> 00:03:24,160
计算量也会很大

74
00:03:24,160 --> 00:03:27,760
第三个就是占用很大的内存空间

75
00:03:27,760 --> 00:03:30,160
而占用这幺多内存空间都这幺大

76
00:03:30,160 --> 00:03:33,860
为的就是得到一个高精度的模型

77
00:03:33,860 --> 00:03:34,460
但是呢

78
00:03:34,460 --> 00:03:36,560
要把神经网络部署起来

79
00:03:36,560 --> 00:03:38,260
所以希望这个网络模型呢

80
00:03:38,260 --> 00:03:39,360
越小越好

81
00:03:39,360 --> 00:03:39,860
于是呢

82
00:03:39,860 --> 00:03:43,060
研究者就发现了网络模型的量化的技术

83
00:03:43,060 --> 00:03:46,860
那现在首先先不要去谈谈模型量化到底做了什幺

84
00:03:46,860 --> 00:03:50,360
去看看模型量化的一些好处

85
00:03:50,460 --> 00:03:52,260
那这个呢是我总结了几个点

86
00:03:52,260 --> 00:03:55,360
第一个点就是保持相同的精度

87
00:03:55,360 --> 00:03:56,960
神经网络训练的时候呢

88
00:03:56,960 --> 00:03:57,860
一般来说啊

89
00:03:57,860 --> 00:04:00,060
对噪声不是说非常的敏感

90
00:04:00,060 --> 00:04:02,860
因为里面有非常大量的冗余的参数

91
00:04:02,860 --> 00:04:04,660
而且都是可学习的参数

92
00:04:04,660 --> 00:04:05,060
第二个呢

93
00:04:05,060 --> 00:04:07,160
就是加速运算了

94
00:04:07,160 --> 00:04:11,260
因为大家都知道像传统的卷积神经网络里面的卷积的操作

95
00:04:11,260 --> 00:04:14,460
一般都是用FP32的浮点数去运算的

96
00:04:14,460 --> 00:04:15,960
越低比特的运算呢

97
00:04:15,960 --> 00:04:18,160
它确实能够减少运算的功耗

98
00:04:18,160 --> 00:04:20,660
而且性能更高

99
00:04:20,660 --> 00:04:21,360
那第三点呢

100
00:04:21,360 --> 00:04:23,260
就是节省内存

101
00:04:23,260 --> 00:04:26,460
我从FP32存储到INT8、INT4

102
00:04:26,460 --> 00:04:27,360
越低的精度

103
00:04:27,360 --> 00:04:29,360
我所占用的地址空间和内存空间呢

104
00:04:29,360 --> 00:04:31,760
肯定是越小的

105
00:04:31,760 --> 00:04:32,260
下面呢

106
00:04:32,260 --> 00:04:36,460
看一下模型量化的五个特点

107
00:04:36,460 --> 00:04:36,960
第一个呢

108
00:04:36,960 --> 00:04:40,960
就是对神经网络模型的参数进行压缩

109
00:04:40,960 --> 00:04:42,360
把以前200兆的模型呢

110
00:04:42,360 --> 00:04:43,960
可能压缩到只有50兆

111
00:04:43,960 --> 00:04:46,460
而且可以提升运算的速度

112
00:04:46,460 --> 00:04:47,960
另外还可以降低内存

113
00:04:47,960 --> 00:04:48,960
降低功耗

114
00:04:48,960 --> 00:04:50,460
提升芯片的面积

115
00:04:50,460 --> 00:04:51,160
那这里面呢

116
00:04:51,160 --> 00:04:54,160
只是对上面进行一个简单的总结

117
00:04:54,160 --> 00:04:58,760
接下来看一下量化技术落地的三大挑战

118
00:04:58,760 --> 00:05:02,660
现在来看一下量化技术落地的三大挑战呢

119
00:05:02,660 --> 00:05:05,260
其中第一个就是精度的挑战

120
00:05:05,260 --> 00:05:05,960
首先呢

121
00:05:05,960 --> 00:05:08,860
现在的量化方法大部分都是线性的量化

122
00:05:08,860 --> 00:05:09,560
另外的话

123
00:05:09,560 --> 00:05:11,160
模型比特越低

124
00:05:11,160 --> 00:05:12,160
精度损失越大

125
00:05:12,160 --> 00:05:12,860
任务呢

126
00:05:12,860 --> 00:05:13,860
不同任务里面

127
00:05:13,860 --> 00:05:15,560
精度损失也是不一样的

128
00:05:15,560 --> 00:05:16,660
而且模型越小

129
00:05:16,660 --> 00:05:19,460
确实对损失精度影响也是很大

130
00:05:19,460 --> 00:05:22,060
所以第一大的挑战就是精度

131
00:05:22,060 --> 00:05:25,260
还是希望能够保持相同的精度

132
00:05:25,260 --> 00:05:28,760
那第二个比较大的挑战就是硬件的支持程度

133
00:05:28,760 --> 00:05:31,160
因为不同的硬件提供的比特数

134
00:05:31,160 --> 00:05:32,660
或者比特的指令集呢

135
00:05:32,660 --> 00:05:33,660
是不同的

136
00:05:33,660 --> 00:05:35,760
需要做很多的对于kernel

137
00:05:35,760 --> 00:05:39,360
或者根据指令集进行一些优化

138
00:05:39,360 --> 00:05:43,260
那最后一点就是软件算法是否真的能够加速

139
00:05:43,260 --> 00:05:45,260
这是一个悖论哦

140
00:05:45,260 --> 00:05:47,460
虽然网络模型经过压缩了

141
00:05:47,460 --> 00:05:49,760
但是要是硬件指令不支持的情况

142
00:05:49,760 --> 00:05:50,960
我在运算的时候呢

143
00:05:50,960 --> 00:05:53,360
还是要反量化量化回FP32哦

144
00:05:53,360 --> 00:05:53,960
这个时候呢

145
00:05:53,960 --> 00:05:56,460
就需要插入很多Cast算子

146
00:05:56,460 --> 00:05:58,860
那这会影响kernel的执行的性能

147
00:05:58,860 --> 00:06:03,160
所以说软件算法是否能够真正得到一个加速呢

148
00:06:03,160 --> 00:06:04,160
这是一个疑问哦

149
00:06:06,860 --> 00:06:07,460
下面呢

150
00:06:07,460 --> 00:06:09,360
我想提出几个问题

151
00:06:09,360 --> 00:06:12,060
引起大家的一个一起去思考

152
00:06:12,060 --> 00:06:13,760
大家也可以停下来几分钟

153
00:06:13,860 --> 00:06:15,360
去思考一下这些问题

154
00:06:15,360 --> 00:06:16,060
第一个就是

155
00:06:16,060 --> 00:06:20,760
为什幺模型量化技术能够对实际的部署起到真正的作用

156
00:06:20,760 --> 00:06:22,060
虽然有很多挑战

157
00:06:22,060 --> 00:06:27,060
但确实模型量化能够真正的加速部署的速度

158
00:06:27,060 --> 00:06:27,660
那第二个呢

159
00:06:27,660 --> 00:06:31,660
就是为什幺需要对网络模型进行量化压缩呢

160
00:06:31,660 --> 00:06:33,760
量化压缩到底有什幺好处

161
00:06:33,760 --> 00:06:35,060
那第三点就是

162
00:06:35,060 --> 00:06:40,460
为什么不直接训练的时候就训练低比特精度的模型呢

163
00:06:40,460 --> 00:06:41,060
哎

164
00:06:41,060 --> 00:06:42,260
这个很有意思

165
00:06:42,260 --> 00:06:43,860
对于大模型也是

166
00:06:43,860 --> 00:06:45,160
我大模型训练的时候呢

167
00:06:45,160 --> 00:06:47,860
动作就百亿级别千亿级别万亿级别的模型

168
00:06:47,860 --> 00:06:49,260
模型参数量这幺大

169
00:06:49,260 --> 00:06:52,160
为什么你不直接训练一个小模型呢

170
00:06:52,160 --> 00:06:54,160
最后一点就是

171
00:06:54,160 --> 00:06:55,460
在什幺情况下

172
00:06:55,460 --> 00:07:00,760
不应该还是应该去使用模型量化技术呢

173
00:07:00,760 --> 00:07:04,060
那这四个点我希望大家停下来去思考一下

174
00:07:13,160 --> 00:07:13,660
现在呢

175
00:07:13,660 --> 00:07:17,660
我要给大家去汇报一下量化原理了

176
00:07:17,660 --> 00:07:18,360
首先呢

177
00:07:18,360 --> 00:07:20,260
量化的方法方法呢

178
00:07:20,260 --> 00:07:23,760
现在来看的一般分为三大种

179
00:07:23,760 --> 00:07:27,660
那第一种就是量化训练QAT

180
00:07:27,660 --> 00:07:28,860
它最主要的方式呢

181
00:07:28,860 --> 00:07:33,060
就是让在网络模型训练的时候去感知量化运算

182
00:07:33,060 --> 00:07:36,260
到底对网络模型精度带来什幺样的影响

183
00:07:36,260 --> 00:07:36,960
然后呢

184
00:07:36,960 --> 00:07:41,160
通过Fine Tuning去降低整个量化的误差

185
00:07:41,160 --> 00:07:41,660
接着呢

186
00:07:41,660 --> 00:07:43,560
有PTQ

187
00:07:43,560 --> 00:07:46,160
那PTQ就是离线量化

188
00:07:46,160 --> 00:07:49,160
离线量化又分为动态和静态

189
00:07:49,160 --> 00:07:50,060
那一般来说呢

190
00:07:50,060 --> 00:07:52,160
会用静态会比较多

191
00:07:52,160 --> 00:07:52,860
动态呢

192
00:07:52,860 --> 00:07:53,960
其实比较简单

193
00:07:53,960 --> 00:07:56,560
就是直接把已经训练好的网络模型呢

194
00:07:56,560 --> 00:08:04,960
权重直接从FP32转成或者映射成INT8和FLOP16这种类型呢进行一个推理 

195
00:08:04,960 --> 00:08:11,060
而静态量化呢 可能会使用少量或者没有标签的数据进行一个校准 

196
00:08:11,460 --> 00:08:13,760
那校准的方法有非常的多

197
00:08:13,760 --> 00:08:14,360
那现在呢

198
00:08:14,360 --> 00:08:17,660
用的比较多的就是KL散度

199
00:08:17,660 --> 00:08:19,460
往下看一个图

200
00:08:19,460 --> 00:08:22,260
更好的去理解这三种具体的算法

201
00:08:22,260 --> 00:08:24,060
首先就是感知量化QAT

202
00:08:24,060 --> 00:08:26,460
有一个训练好的网络模型

203
00:08:26,460 --> 00:08:27,560
对这个网络模型呢

204
00:08:27,560 --> 00:08:30,560
进行一个转换超越一些伪量化的算子

205
00:08:30,560 --> 00:08:31,060
接着呢

206
00:08:31,060 --> 00:08:32,560
得到一个新的网络模型

207
00:08:32,560 --> 00:08:35,060
对新的网络模型进行Fine Tuning

208
00:08:35,060 --> 00:08:35,760
最后呢

209
00:08:35,760 --> 00:08:40,360
得到真正量化后或者感知量化后的一个网络模型

210
00:08:40,360 --> 00:08:40,960
接着呢

211
00:08:40,960 --> 00:08:42,260
去给部署端的

212
00:08:44,160 --> 00:08:45,360
静态离线量化呢

213
00:08:45,360 --> 00:08:47,860
首先有一个训练好的网络模型

214
00:08:47,860 --> 00:08:50,460
有一堆训练的数据

215
00:08:50,460 --> 00:08:51,860
这个是Training Data

216
00:08:51,860 --> 00:08:52,560
改错了

217
00:08:52,560 --> 00:08:55,460
有一堆训练的数据或者等待训练的数据

218
00:08:55,460 --> 00:08:56,060
然后呢

219
00:08:56,060 --> 00:08:58,760
去给网络模型进行一个校准

220
00:08:58,760 --> 00:08:59,960
那这个校准的方法呢

221
00:08:59,960 --> 00:09:02,760
有可能会用KL散度或者其他的方式

222
00:09:02,760 --> 00:09:03,460
最后呢

223
00:09:03,460 --> 00:09:07,560
就是PTQ的Dynamic动态离线量化

224
00:09:07,560 --> 00:09:09,560
输一个神级网络模型

225
00:09:09,560 --> 00:09:09,860
然后呢

226
00:09:09,860 --> 00:09:11,660
对这个神级网络模型进行转换

227
00:09:11,660 --> 00:09:14,560
最后得到转换后或者量化后的网络模型

228
00:09:14,560 --> 00:09:16,160
主要是由这三种方法

229
00:09:17,460 --> 00:09:17,960
那现在呢

230
00:09:17,960 --> 00:09:20,760
看一下这三种方法有什幺区别

231
00:09:20,760 --> 00:09:23,060
做一个简单的比较了

232
00:09:23,060 --> 00:09:24,160
量化训练呢

233
00:09:24,160 --> 00:09:27,960
确实精度损失是比较少的

234
00:09:27,960 --> 00:09:31,160
但是它的缺点就是需要使用大量带标签的数据

235
00:09:31,160 --> 00:09:33,460
进行Fine Tuning或者一个重训的

236
00:09:33,460 --> 00:09:36,760
那第二种方式就是静态离线量化PTQ

237
00:09:36,760 --> 00:09:38,160
它的好处就是

238
00:09:38,260 --> 00:09:40,460
精度损失也是比较小的

239
00:09:40,460 --> 00:09:41,660
但是不能说没有哦

240
00:09:41,660 --> 00:09:44,560
因为它缺乏了训练或者Fine Tuning的步骤

241
00:09:44,560 --> 00:09:48,860
它需要有一些少量无标签的数据进行校准的

242
00:09:48,860 --> 00:09:52,560
最后一种就是动态离线量化就是PTQ了

243
00:09:52,560 --> 00:09:54,360
它这种方式进度的损失

244
00:09:54,360 --> 00:09:56,760
一般来说不可控

245
00:09:56,760 --> 00:09:58,060
不过好处呢

246
00:09:58,060 --> 00:10:00,560
就是它没有任何使用的约束

247
00:10:00,560 --> 00:10:01,960
你想咋用就咋用

248
00:10:01,960 --> 00:10:02,360
这个呢

249
00:10:02,360 --> 00:10:03,460
也是ZOMI呢

250
00:10:03,460 --> 00:10:05,360
在MindSpore里面做的其中一个项目

251
00:10:05,360 --> 00:10:07,360
做得非常的深入了

252
00:10:07,360 --> 00:10:09,660
这可能导致我引用的额外的知识呢

253
00:10:09,660 --> 00:10:10,460
稍微有点多了

254
00:10:11,860 --> 00:10:12,360
现在呢

255
00:10:12,360 --> 00:10:15,660
正式的来到量化的一个原理

256
00:10:17,660 --> 00:10:18,160
首先呢

257
00:10:18,160 --> 00:10:20,060
来理解一下网络模型的优化

258
00:10:20,060 --> 00:10:22,960
它主要是在定点跟浮点数之间呢

259
00:10:22,960 --> 00:10:26,960
创建了一个简单的数据或者数学的映射关系

260
00:10:26,960 --> 00:10:27,960
那这个映射关系呢

261
00:10:27,960 --> 00:10:29,960
主要是线性的映射关系

262
00:10:29,960 --> 00:10:30,960
下面这个图呢

263
00:10:30,960 --> 00:10:32,860
上面是指浮点的

264
00:10:32,860 --> 00:10:35,360
或者权重的参数的数值

265
00:10:35,360 --> 00:10:37,060
它当然有最小和最大值

266
00:10:37,060 --> 00:10:38,260
当然也有个0值

267
00:10:38,260 --> 00:10:39,760
希望把这一堆数据

268
00:10:39,760 --> 00:10:41,560
映射到一个具体的范围

269
00:10:41,560 --> 00:10:43,760
-127到127之间

270
00:10:43,760 --> 00:10:44,560
那这时候呢

271
00:10:44,560 --> 00:10:45,760
-127到127

272
00:10:45,760 --> 00:10:48,760
这就是INT8所表示的范围

273
00:10:48,760 --> 00:10:50,360
那另外还有一种方式

274
00:10:50,360 --> 00:10:53,160
就是截断的方式 

275
00:10:53,160 --> 00:10:56,060
去设置一个最小值和最大值

276
00:10:56,060 --> 00:10:58,560
把最小值和最大值范围的参数呢

277
00:10:58,560 --> 00:11:01,760
去映射到-127到127之间

278
00:11:01,760 --> 00:11:02,760
而不在这个范围内

279
00:11:02,760 --> 00:11:03,460
你的数据呢

280
00:11:03,460 --> 00:11:05,560
直接把它丢弃掉

281
00:11:05,660 --> 00:11:07,260
这个呢就是简单的量化方法

282
00:11:07,260 --> 00:11:09,260
再往下看

283
00:11:09,260 --> 00:11:10,860
其实量化的类型啊

284
00:11:10,860 --> 00:11:13,960
可以分为对称量化和非对称量化

285
00:11:13,960 --> 00:11:15,560
所谓的对称很简单

286
00:11:15,560 --> 00:11:17,960
就是以0作为中心轴

287
00:11:17,960 --> 00:11:19,760
做-127到127

288
00:11:19,760 --> 00:11:21,260
就是两边的对称

289
00:11:21,260 --> 00:11:23,760
可能会用INT来表示

290
00:11:23,760 --> 00:11:25,560
另外一种非对称的量化呢

291
00:11:25,560 --> 00:11:27,060
可能就没有中心轴了

292
00:11:27,060 --> 00:11:28,860
以0作为开始

293
00:11:28,860 --> 00:11:30,560
以255作为结束

294
00:11:30,560 --> 00:11:31,260
那这种呢

295
00:11:31,260 --> 00:11:33,760
表示方式直接使用了UINT

296
00:11:33,860 --> 00:11:35,760
去进行一个表示的

297
00:11:37,760 --> 00:11:41,660
哎 下面才来真正的来到 量化的原理  

298
00:11:41,660 --> 00:11:43,260
真正看一下公式呢

299
00:11:43,260 --> 00:11:45,060
要搞清楚量化的原理

300
00:11:45,060 --> 00:11:47,860
肯定要搞清楚数据映射的关系

301
00:11:47,860 --> 00:11:51,160
就是浮点跟定点的数据的转换公式

302
00:11:51,160 --> 00:11:54,460
那现在呢 R表示数的浮点的数据 

303
00:11:54,460 --> 00:11:56,460
FP32 FP16都行

304
00:11:56,460 --> 00:11:59,760
而Q呢表示量化后的定点的一个数据 

305
00:11:59,760 --> 00:12:01,560
INT的数据

306
00:12:01,560 --> 00:12:05,960
而Z呢就是偏移值要做对称还是不对称 

307
00:12:05,960 --> 00:12:08,260
而S就是缩放因子

308
00:12:08,260 --> 00:12:11,060
这里面的很重要的就是找到缩放因子

309
00:12:11,060 --> 00:12:16,160
还有zero point 零点表示就能够求得到Q 

310
00:12:17,160 --> 00:12:19,360
上面这条公式呢 叫做量化 

311
00:12:19,360 --> 00:12:22,460
下面这条公式呢 叫做反量化 

312
00:12:22,460 --> 00:12:25,760
那终极目标还是求得到S和Z

313
00:12:25,760 --> 00:12:28,160
看看这两个是怎么去求的

314
00:12:28,160 --> 00:12:31,160
求解S和Z啊 其实有很多种方法 

315
00:12:31,160 --> 00:12:34,560
这里面呢 列举了一种线性的量化的求解方法 

316
00:12:34,560 --> 00:12:37,560
就用MinMax进行一个求解的

317
00:12:37,560 --> 00:12:39,460
S其实比较好求

318
00:12:39,460 --> 00:12:42,660
把Rmax减去Rmin除以Qmax减去Qmin

319
00:12:42,660 --> 00:12:45,360
那一般来说Qmax Qmin是知道的

320
00:12:45,360 --> 00:12:46,660
-127到117

321
00:12:46,660 --> 00:12:48,460
255到0之间

322
00:12:48,460 --> 00:12:49,760
所以通过这种公式呢

323
00:12:49,760 --> 00:12:52,060
就可以求得到一个X

324
00:12:52,060 --> 00:12:54,860
而Z呢也是比较好求的

325
00:12:54,860 --> 00:12:58,260
下面呢 来看一下一个具体的例子啊 

326
00:12:58,260 --> 00:13:00,860
刚才其实讲的有点抽象

327
00:13:00,960 --> 00:13:02,860
S就是scale offset

328
00:13:02,860 --> 00:13:04,160
就是刚才的Z

329
00:13:04,160 --> 00:13:06,160
假设输入的是一个uint的数据

330
00:13:06,160 --> 00:13:07,460
为了得到float

331
00:13:07,460 --> 00:13:08,960
可以通过下面这条公式

332
00:13:08,960 --> 00:13:10,260
当然了这是反量化

333
00:13:10,260 --> 00:13:12,460
量化的时候呢 输入float 

334
00:13:12,460 --> 00:13:16,160
然后除以scale做一个四舍五入

335
00:13:16,160 --> 00:13:19,160
减去offset 就得到了uint 

336
00:13:19,160 --> 00:13:20,660
那同样的scale和offset呢

337
00:13:20,660 --> 00:13:24,160
也是通过这个公式上面讲的 去进行计算的 

338
00:13:24,160 --> 00:13:26,560
那看这条公式或者这几条公式呢

339
00:13:26,560 --> 00:13:30,660
确实比刚才上面直接用Q啊Z啊 比较明确 

340
00:13:30,660 --> 00:13:32,660
那今天的内容呢这么多了

341
00:13:32,660 --> 00:13:34,160
可能稍微有点长了点

342
00:13:34,160 --> 00:13:36,760
欢迎大家去阅读更多新的论文

343
00:13:36,760 --> 00:13:38,260
好了 谢谢大家 

344
00:13:38,260 --> 00:13:39,360
拜了个拜

345
00:13:39,360 --> 00:13:41,060
卷的不行了卷的不行了

346
00:13:41,060 --> 00:13:42,860
记得一键三连加关注哦

347
00:13:42,860 --> 00:13:46,460
所有的内容都会开源在 下面这条链接里面 

348
00:13:46,460 --> 00:13:47,660
拜了个拜