04.srt

1
00:00:00,000 --> 00:00:04,560
字幕生成: BLACK 字幕校对: 方鸿渐

2
00:00:06,360 --> 00:00:07,480
哈喽大家好

3
00:00:07,480 --> 00:00:10,040
我是那个赚钱的时候千辛万苦

4
00:00:10,040 --> 00:00:12,800
花钱的时候稀里糊涂的ZOMI

5
00:00:12,800 --> 00:00:15,680
那今天呢来到一个比较

6
00:00:15,680 --> 00:00:17,000
比较重要的内容吧

7
00:00:17,000 --> 00:00:18,480
也不能说很重要啊

8
00:00:18,480 --> 00:00:21,520
就是整个推理系统的一个架构

9
00:00:21,520 --> 00:00:25,200
其实我觉得最重要的是推理引擎的一个架构

10
00:00:25,200 --> 00:00:26,400
就围绕着这一块

11
00:00:26,400 --> 00:00:28,520
可能是比较多的技术含量的

12
00:00:28,560 --> 00:00:32,720
那推理系统呢更多的是一些传统的一些功能点

13
00:00:32,720 --> 00:00:35,120
然后把AI的一些特性加进去

14
00:00:35,120 --> 00:00:37,680
那没关系继续来深入的去看一下

15
00:00:37,680 --> 00:00:39,840
今天呢主要是去讲一讲

16
00:00:39,840 --> 00:00:42,440
整个推理系统的具体的架构

17
00:00:42,440 --> 00:00:43,560
那在讲架构之前呢

18
00:00:43,560 --> 00:00:45,040
会来讲另外一个内容

19
00:00:45,040 --> 00:00:47,320
就是推理、部署、服务

20
00:00:47,320 --> 00:00:49,520
它三者到底有啥区别呢

21
00:00:49,520 --> 00:00:51,560
它之间是啥关系呢

22
00:00:51,560 --> 00:00:55,640
那接着去看一下推理系统的整体的架构

23
00:00:55,640 --> 00:00:56,720
如果有时间

24
00:00:56,760 --> 00:00:58,600
因为我希望能够在每个视频呢

25
00:00:58,600 --> 00:01:00,080
控制在十来分钟

26
00:01:00,080 --> 00:01:02,040
就大家看的不累我讲的不累

27
00:01:02,040 --> 00:01:04,560
然后呢如果有时间或者有剩下的时间呢

28
00:01:04,560 --> 00:01:08,480
我去想去讲讲模型的生命周期的管理

29
00:01:08,480 --> 00:01:10,360
因为模型生命周期的管理呢

30
00:01:10,360 --> 00:01:12,360
是AI属性比较强的

31
00:01:12,360 --> 00:01:14,080
在整个推理系统里面

32
00:01:14,080 --> 00:01:17,000
而推理系统更多的是一些服务的响应啊

33
00:01:17,000 --> 00:01:18,120
技术的编排啊

34
00:01:18,120 --> 00:01:19,800
系统的调度啊这些问题

35
00:01:19,800 --> 00:01:23,600
模型的生命周期呢是跟AI属性比较强相关

36
00:01:23,600 --> 00:01:25,360
所以我想给大家后面呢

37
00:01:25,400 --> 00:01:27,160
有时间去汇报一下的

38
00:01:29,120 --> 00:01:33,280
接下来呢回顾一下整个模型的生命周期

39
00:01:33,280 --> 00:01:36,800
首先呢需要去收集训练的模型

40
00:01:36,800 --> 00:01:37,680
收集完模型之后呢

41
00:01:37,680 --> 00:01:39,680
去给深度学习的这个模型呢

42
00:01:39,680 --> 00:01:41,520
去给算法呢去训练

43
00:01:41,520 --> 00:01:45,320
训练完之后呢得到一个具体的AI模型

44
00:01:45,320 --> 00:01:49,360
AI模型呢就可以真正的去用来做服务的请求和服务的响应

45
00:01:49,360 --> 00:01:51,160
而对服务的请求和服务的响应

46
00:01:51,160 --> 00:01:53,040
还有真正的推理和部署呢

47
00:01:53,080 --> 00:01:55,520
这一块就是整个AI

48
00:01:55,520 --> 00:01:59,040
AI推理系统呢需要去解决的一些问题

49
00:01:59,040 --> 00:02:00,440
或者AI推理系统呢

50
00:02:00,440 --> 00:02:03,320
它主要关注的点就在这一块了

51
00:02:05,200 --> 00:02:07,640
现在呢我想给大家提一个问题

52
00:02:07,640 --> 00:02:09,560
引起大家去思考了

53
00:02:09,560 --> 00:02:10,840
就是

54
00:02:10,840 --> 00:02:15,120
AI应用部署需要考虑哪些方面呢

55
00:02:15,120 --> 00:02:17,560
需要考虑哪些技术点呢

56
00:02:18,960 --> 00:02:20,680
诶 这个问题问得非常好

57
00:02:20,720 --> 00:02:23,120
因为只要考虑到这些点

58
00:02:23,120 --> 00:02:26,920
才能够很好的去设计整个框架

59
00:02:26,920 --> 00:02:29,000
去设计推理系统

60
00:02:29,000 --> 00:02:30,920
需要应该有哪些功能

61
00:02:32,800 --> 00:02:34,960
那下面呢去了解第一个内容

62
00:02:34,960 --> 00:02:38,640
就是推理 部署 服务化三者之间的区别

63
00:02:38,640 --> 00:02:39,880
那在了解之前呢

64
00:02:39,880 --> 00:02:42,320
我想给大家再提几个问题

65
00:02:42,320 --> 00:02:45,440
第一个呢就是什么是模型推理

66
00:02:45,440 --> 00:02:47,600
什么是叫推理服务化

67
00:02:47,600 --> 00:02:49,080
服务化到底叫啥呢

68
00:02:49,080 --> 00:02:50,280
到底有什么作用

69
00:02:50,280 --> 00:02:51,640
它是啥意思呢

70
00:02:53,120 --> 00:02:53,680
接着呢

71
00:02:53,680 --> 00:02:55,080
我想给大家提个问题

72
00:02:55,080 --> 00:02:56,640
不知道大家了解过没有

73
00:02:56,640 --> 00:03:00,920
平时常见的有哪些推理的服务的框架

74
00:03:00,920 --> 00:03:02,040
大家都叫框架

75
00:03:02,040 --> 00:03:05,840
这个推理服务框架跟AI框架不是一个概念哦

76
00:03:05,840 --> 00:03:08,440
这个Triton就是什么东西呢

77
00:03:08,440 --> 00:03:10,240
我好像见了好几个Triton

78
00:03:10,240 --> 00:03:13,200
有些Triton呢是指那个推理的服务框架

79
00:03:13,200 --> 00:03:14,800
有个Triton呢

80
00:03:14,800 --> 00:03:18,720
就好像是指用Python去写GPU的一些算子

81
00:03:18,760 --> 00:03:20,680
到底这个Triton到底是个啥玩意

82
00:03:21,760 --> 00:03:22,240
好了

83
00:03:22,240 --> 00:03:22,840
带着这个

84
00:03:24,800 --> 00:03:26,120
那带着这个疑问呢

85
00:03:26,120 --> 00:03:28,320
往下去看三个概念

86
00:03:28,320 --> 00:03:29,720
第一个呢就是Inference

87
00:03:29,720 --> 00:03:30,920
推理

88
00:03:30,920 --> 00:03:32,000
第二个就是部署

89
00:03:32,000 --> 00:03:33,080
Deployment

90
00:03:33,080 --> 00:03:34,440
第三个就是服务化

91
00:03:34,440 --> 00:03:35,360
Serving

92
00:03:35,360 --> 00:03:37,560
逐个的来去澄清一下

93
00:03:39,280 --> 00:03:41,880
首先第一个就是推理Inference的

94
00:03:41,880 --> 00:03:44,360
既然有训练肯定会有推理嘛

95
00:03:44,360 --> 00:03:46,800
推理就是一个前向的一个计算

96
00:03:46,840 --> 00:03:49,160
那接着呢看看部署

97
00:03:49,160 --> 00:03:50,840
那部署其实是训练的时候

98
00:03:50,840 --> 00:03:52,480
不是得到一个模型吗

99
00:03:52,480 --> 00:03:54,040
希望把这个模型呢

100
00:03:54,040 --> 00:03:56,400
真正部署在硬件上面

101
00:03:56,400 --> 00:03:58,000
那这个就是部署的概念

102
00:03:58,000 --> 00:03:59,600
部署在硬件上面之后呢

103
00:03:59,600 --> 00:04:01,560
才做推理的工作

104
00:04:01,560 --> 00:04:04,520
那部署呢就会涉及到包括移植压缩加速

105
00:04:04,520 --> 00:04:06,080
还有推理引擎

106
00:04:06,080 --> 00:04:08,720
这整一块的相关的工作叫做部署

107
00:04:10,840 --> 00:04:11,960
所谓的服务化呢

108
00:04:11,960 --> 00:04:15,160
其实相对应对部署的方式

109
00:04:15,200 --> 00:04:17,080
进行一个抽象的

110
00:04:17,080 --> 00:04:18,040
服务化的时候呢

111
00:04:18,040 --> 00:04:19,800
会把AI的算法呢

112
00:04:19,800 --> 00:04:23,080
会封装成一个SDK来集中在相关的APP上面

113
00:04:23,080 --> 00:04:26,320
也有可能封装成一个对应的web服务

114
00:04:26,320 --> 00:04:28,600
对外呢去暴露一些HTTP

115
00:04:28,600 --> 00:04:32,240
还有RPC等协议的一些功能

116
00:04:32,240 --> 00:04:34,800
那这个呢就是三者的区别

117
00:04:34,800 --> 00:04:36,200
有了这三者的区别呢

118
00:04:36,200 --> 00:04:39,680
看一下具体的一些服务化的一些框架

119
00:04:39,680 --> 00:04:41,400
那好像TF-Serving呢

120
00:04:41,400 --> 00:04:44,960
就是最早的推出的一个服务化的功能

121
00:04:45,000 --> 00:04:46,600
谷歌呢其实是围绕着TensorFlow

122
00:04:46,600 --> 00:04:48,520
做了非常大量相关的工作

123
00:04:48,520 --> 00:04:51,480
包括TF-Serving、TF-Lite了很多工作

124
00:04:51,480 --> 00:04:52,280
那TF-Serving呢

125
00:04:52,280 --> 00:04:54,760
就是在谷歌2016年的时候

126
00:04:54,760 --> 00:04:57,480
推出的一个服务化的框架

127
00:04:57,480 --> 00:04:58,760
通过网络请求呢

128
00:04:58,760 --> 00:05:00,680
去获取用户的一些数据

129
00:05:00,680 --> 00:05:01,560
然后进行处理

130
00:05:01,560 --> 00:05:03,520
最后呢返回相关的结果

131
00:05:03,520 --> 00:05:05,000
那现在来看看

132
00:05:05,000 --> 00:05:07,480
常见的一些服务化的框架

133
00:05:07,480 --> 00:05:08,400
那可以看到啊

134
00:05:08,400 --> 00:05:11,720
其实从那个16年到20年呢

135
00:05:11,720 --> 00:05:12,720
各大厂商呢

136
00:05:12,720 --> 00:05:16,200
推出了非常多的一些服务化的框架

137
00:05:16,200 --> 00:05:19,240
用的比较多的有Kubeflow呢

138
00:05:19,240 --> 00:05:20,600
是Kubernetes里面推出的

139
00:05:20,600 --> 00:05:23,000
还有英伟达推出的Triton呢

140
00:05:23,000 --> 00:05:25,480
现在以Triton作为一个具体的例子

141
00:05:25,480 --> 00:05:28,040
来看看常见的服务化的框架的

142
00:05:28,040 --> 00:05:29,920
一个具体的架构

143
00:05:29,920 --> 00:05:31,280
那Triton呢

144
00:05:31,280 --> 00:05:32,440
它的一个全名呢

145
00:05:32,440 --> 00:05:35,200
叫做Triton推理服务器

146
00:05:35,200 --> 00:05:38,720
NVIDIA Triton Inference Server

147
00:05:39,720 --> 00:05:42,920
给用户去提供一个部署在云端

148
00:05:42,920 --> 00:05:45,800
或者边缘侧的一个具体的解决方案

149
00:05:45,800 --> 00:05:46,800
对接的呢

150
00:05:46,800 --> 00:05:48,520
有客户的云端的服务器

151
00:05:48,520 --> 00:05:50,560
还有云端的服务的请求

152
00:05:50,560 --> 00:05:51,400
然后对应到呢

153
00:05:51,400 --> 00:05:53,520
对AI模型的一个管理

154
00:05:53,520 --> 00:05:54,520
然后这里面呢

155
00:05:54,520 --> 00:05:57,320
就是具体的一些服务化的功能了

156
00:05:58,520 --> 00:05:59,720
那下面来看一下

157
00:05:59,720 --> 00:06:01,560
它的一个具体的架构

158
00:06:01,560 --> 00:06:02,160
右边呢

159
00:06:02,160 --> 00:06:03,840
就是它的整体的一个架构

160
00:06:03,840 --> 00:06:04,480
现在呢

161
00:06:04,480 --> 00:06:05,960
逐个模块的去展开

162
00:06:05,960 --> 00:06:07,320
这里面有非常多的模块

163
00:06:07,320 --> 00:06:08,000
那第一个呢

164
00:06:08,000 --> 00:06:10,040
就是它的一个接入层

165
00:06:10,040 --> 00:06:11,480
叫做Interface

166
00:06:11,480 --> 00:06:12,400
那这个介入层呢

167
00:06:12,400 --> 00:06:14,520
主要是指这里面的这个模块

168
00:06:14,520 --> 00:06:15,960
那提供了一个HTTP

169
00:06:15,960 --> 00:06:17,720
还有GRPC

170
00:06:17,720 --> 00:06:20,280
那这个RPC就是谷歌的RPC的协议

171
00:06:20,280 --> 00:06:20,640
然后呢

172
00:06:20,640 --> 00:06:21,320
除此之外呢

173
00:06:21,320 --> 00:06:23,040
它还提供了一个C++的

174
00:06:23,040 --> 00:06:24,960
或者C的一个API的接口

175
00:06:24,960 --> 00:06:27,200
去给到一个用户

176
00:06:27,200 --> 00:06:28,880
做一个服务访问请求的

177
00:06:28,880 --> 00:06:31,040
另外它还支持一个共享内存

178
00:06:31,040 --> 00:06:33,520
Share Memory的一个IPC的通讯

179
00:06:35,200 --> 00:06:35,680
那现在呢

180
00:06:35,680 --> 00:06:36,880
看看另外

181
00:06:36,880 --> 00:06:40,160
又往右边数一数的一个功能点

182
00:06:40,160 --> 00:06:41,080
那这个功能点呢

183
00:06:41,080 --> 00:06:43,320
叫做Triton的模型仓库

184
00:06:43,320 --> 00:06:44,800
叫做Model Manager

185
00:06:45,960 --> 00:06:47,200
这个模型仓库呢

186
00:06:47,200 --> 00:06:49,560
可以接入到各种的云服务

187
00:06:49,560 --> 00:06:51,360
而且可以支持多模型呢

188
00:06:51,360 --> 00:06:53,040
也支持模型的编排

189
00:06:53,040 --> 00:06:55,360
也支持模型的持久化的存储

190
00:06:55,360 --> 00:06:56,920
那这个就是具体的功能

191
00:06:56,920 --> 00:06:59,560
下面来看一下左下角

192
00:06:59,560 --> 00:07:00,360
那左下角呢

193
00:07:00,360 --> 00:07:04,000
这里面有一个Pre Model Scheduler

194
00:07:04,040 --> 00:07:06,440
就是模型的预编排

195
00:07:06,440 --> 00:07:07,400
可以刚才呀

196
00:07:07,400 --> 00:07:09,480
其实大量的去强调

197
00:07:09,480 --> 00:07:12,040
在AI系统里面的模型的管理

198
00:07:12,040 --> 00:07:14,680
是比较特别比较内核的

199
00:07:14,680 --> 00:07:16,120
那这里面的主要的工作呢

200
00:07:16,120 --> 00:07:18,560
就是解析整个URL的请求

201
00:07:18,560 --> 00:07:20,880
就上面对于URL的请求

202
00:07:20,880 --> 00:07:21,600
请求完之后呢

203
00:07:21,600 --> 00:07:23,600
就会从模型库里面

204
00:07:23,600 --> 00:07:25,800
去做一个具体的模型的编排

205
00:07:25,800 --> 00:07:27,040
去选择哪些模型

206
00:07:27,040 --> 00:07:30,520
对于哪些任务应该用什么调度的方式

207
00:07:30,520 --> 00:07:30,880
这个呢

208
00:07:30,880 --> 00:07:33,520
就是模型预编排所做的工作

209
00:07:35,000 --> 00:07:36,320
那往下呢

210
00:07:36,320 --> 00:07:37,880
再看看下个功能

211
00:07:37,880 --> 00:07:38,440
这个呢

212
00:07:38,440 --> 00:07:40,120
叫做Backend

213
00:07:40,120 --> 00:07:42,120
在AI推理系统里面的Backend

214
00:07:42,120 --> 00:07:44,080
其实是对应的推理引擎

215
00:07:44,080 --> 00:07:45,080
那这个推理引擎呢

216
00:07:45,080 --> 00:07:46,000
在Triton里面呢

217
00:07:46,000 --> 00:07:47,000
支持非常多的

218
00:07:47,000 --> 00:07:47,520
TensorFlow

219
00:07:47,520 --> 00:07:47,920
Linux

220
00:07:47,920 --> 00:07:48,720
PyTorch

221
00:07:48,720 --> 00:07:50,880
还有自定义的也可以

222
00:07:52,160 --> 00:07:53,760
在Triton启动的时候呢

223
00:07:53,760 --> 00:07:55,160
模型仓库的模型呢

224
00:07:55,160 --> 00:07:56,440
已经被加载进来

225
00:07:56,440 --> 00:07:58,080
就是右边的这条线

226
00:07:58,080 --> 00:07:59,120
加载进来之后呢

227
00:07:59,120 --> 00:08:01,960
就在后端的服务化的推理引擎上面

228
00:08:01,960 --> 00:08:02,920
去执行的

229
00:08:05,000 --> 00:08:05,800
最后呢

230
00:08:05,800 --> 00:08:08,560
看一个最底下的功能

231
00:08:08,560 --> 00:08:09,200
那这个呢

232
00:08:09,200 --> 00:08:11,280
就分开两个把它合在一起了

233
00:08:11,880 --> 00:08:12,480
第一个呢

234
00:08:12,480 --> 00:08:14,840
就是服务的返回

235
00:08:14,840 --> 00:08:17,040
通过HTTP或者GRPC呢

236
00:08:17,040 --> 00:08:20,560
对推理引擎的一个结果进行返回

237
00:08:21,640 --> 00:08:22,280
第二个呢

238
00:08:22,280 --> 00:08:23,360
就是对state

239
00:08:23,360 --> 00:08:24,400
就训练的时候

240
00:08:24,400 --> 00:08:27,400
整个AI系统的一些状态

241
00:08:27,400 --> 00:08:28,520
健康状况呢

242
00:08:28,520 --> 00:08:29,800
进行一个监控

243
00:08:29,800 --> 00:08:31,960
然后返回给HTTP请求

244
00:08:31,960 --> 00:08:32,720
然后这个呢

245
00:08:32,720 --> 00:08:34,080
就对应HTTP

246
00:08:34,080 --> 00:08:37,120
就对应整个服务器的管理的后台

247
00:08:37,120 --> 00:08:39,640
那这个就是监控的功能了

248
00:08:39,640 --> 00:08:41,080
所以说整体的框架呢

249
00:08:41,080 --> 00:08:42,640
就分开这好几块功能

250
00:08:42,640 --> 00:08:43,720
第一块第二块

251
00:08:43,720 --> 00:08:45,520
模型的服务模型的编排

252
00:08:45,520 --> 00:08:46,520
推理引擎

253
00:08:46,520 --> 00:08:48,600
还有健康的管理

254
00:08:48,600 --> 00:08:49,600
这几大内容

255
00:08:51,480 --> 00:08:52,080
那下面呢

256
00:08:52,080 --> 00:08:55,680
就是整个Triton集成推理引擎的一个流程

257
00:08:55,680 --> 00:08:56,120
这个呢

258
00:08:56,120 --> 00:08:56,720
是知乎

259
00:08:56,720 --> 00:08:58,200
我是小baby

260
00:08:58,200 --> 00:08:58,760
哈哈

261
00:08:58,760 --> 00:09:01,920
那这位同学里面去截取的一个图

262
00:09:01,920 --> 00:09:02,560
这位同学呢

263
00:09:02,560 --> 00:09:03,800
在Triton的一个对接呢

264
00:09:03,800 --> 00:09:05,520
做了非常大量的工作

265
00:09:05,520 --> 00:09:09,600
那我引用了里面其中的一些相关的一个图

266
00:09:09,600 --> 00:09:11,240
那其实可以看到啊

267
00:09:11,240 --> 00:09:12,120
Triton呢

268
00:09:12,120 --> 00:09:16,160
主要是帮集成了很多的网络的请求和模型的编排的工作

269
00:09:16,160 --> 00:09:17,400
而服务化的功能呢

270
00:09:17,400 --> 00:09:18,520
它基本上做好了

271
00:09:18,520 --> 00:09:20,280
只需要把后端

272
00:09:20,280 --> 00:09:21,280
把模型

273
00:09:21,280 --> 00:09:23,800
把一些推理的引擎

274
00:09:23,800 --> 00:09:25,840
对接到整个Triton的后端

275
00:09:25,840 --> 00:09:27,160
就可以了

276
00:09:27,160 --> 00:09:27,560
这个呢

277
00:09:27,560 --> 00:09:30,200
是Triton给做了一个非常多的相关的工作

278
00:09:30,200 --> 00:09:31,080
而华为呢

279
00:09:31,080 --> 00:09:35,280
我理解这个对应的产品就是Mind X里面的Mind DL

280
00:09:35,280 --> 00:09:36,840
它会做很多相关的服务器

281
00:09:36,840 --> 00:09:37,960
企划服务的部署啊

282
00:09:38,960 --> 00:09:40,720
还有推理系统相关的工作

283
00:09:40,720 --> 00:09:41,280
那其实呢

284
00:09:41,280 --> 00:09:42,200
大部分的架构呢

285
00:09:42,200 --> 00:09:44,040
就是刚才所讲的

286
00:09:44,040 --> 00:09:45,600
就那几个模块

287
00:09:45,600 --> 00:09:46,520
那最后呢

288
00:09:46,520 --> 00:09:48,440
还有点时间呢

289
00:09:48,440 --> 00:09:51,320
就这个视频可能会稍微有点长

290
00:09:51,320 --> 00:09:54,120
来看看模型的生命周期管理

291
00:10:01,600 --> 00:10:02,160
首先呢

292
00:10:02,160 --> 00:10:06,680
了解一下为什么要对模型的版本进行管理

293
00:10:06,680 --> 00:10:07,280
其实呢

294
00:10:07,280 --> 00:10:09,200
在应该是我现在啊

295
00:10:09,200 --> 00:10:10,600
遇到一个比较大的挑战

296
00:10:10,600 --> 00:10:12,720
就是我PyTorch一个对应的模型

297
00:10:12,720 --> 00:10:14,040
或者MindSpore对应的模型

298
00:10:14,040 --> 00:10:15,760
MindSpore的版本升级了

299
00:10:15,760 --> 00:10:17,280
它的API接口变了

300
00:10:17,280 --> 00:10:19,000
这个模型就跑不通了

301
00:10:19,000 --> 00:10:20,560
需要重新的适配

302
00:10:20,560 --> 00:10:21,840
那这个情况呢

303
00:10:21,840 --> 00:10:23,480
会对模型呢

304
00:10:23,480 --> 00:10:24,720
重新上线

305
00:10:24,720 --> 00:10:25,400
那第二个呢

306
00:10:25,400 --> 00:10:27,920
就是我的模型的精度

307
00:10:27,920 --> 00:10:28,920
对于我同一个任务

308
00:10:28,920 --> 00:10:29,760
我是检测的

309
00:10:29,800 --> 00:10:31,880
我之前用的是YOLOv3

310
00:10:31,880 --> 00:10:32,400
这个时候呢

311
00:10:32,400 --> 00:10:33,800
模型的精度

312
00:10:33,800 --> 00:10:34,920
可能MAP呢

313
00:10:34,920 --> 00:10:36,120
只有50%多

314
00:10:36,120 --> 00:10:36,840
结果呢

315
00:10:36,840 --> 00:10:39,920
YOLOv4、V5、V6、V7都出来了

316
00:10:39,920 --> 00:10:40,600
这个时候呢

317
00:10:40,600 --> 00:10:41,320
我的MAP呢

318
00:10:41,320 --> 00:10:42,600
已经上到70%多了

319
00:10:43,640 --> 00:10:45,240
因为新的算法

320
00:10:45,240 --> 00:10:46,640
所以会对模型呢

321
00:10:46,640 --> 00:10:48,280
进行新的升级

322
00:10:48,280 --> 00:10:49,200
都是用YOLO

323
00:10:49,200 --> 00:10:50,400
都是做检测

324
00:10:50,400 --> 00:10:51,920
用户不会关心你用YOLO

325
00:10:51,920 --> 00:10:53,480
还是用YOLOv3、V2

326
00:10:53,480 --> 00:10:55,400
用户只关心我的检测模型

327
00:10:55,400 --> 00:10:57,840
我的检测算法到底准还是不准

328
00:10:57,840 --> 00:10:59,640
因为最后的模型的出口

329
00:10:59,640 --> 00:11:00,520
或者最后的任务呢

330
00:11:00,520 --> 00:11:01,200
只有一个

331
00:11:01,200 --> 00:11:02,960
那这些情况呢

332
00:11:02,960 --> 00:11:05,720
也是需要对模型进行管理的

333
00:11:05,720 --> 00:11:06,920
另外有可能

334
00:11:06,920 --> 00:11:09,040
模型出现问题呢

335
00:11:09,040 --> 00:11:10,360
模型被攻击

336
00:11:10,360 --> 00:11:11,440
或者模型

337
00:11:11,440 --> 00:11:13,640
随着时间管理的问题呢

338
00:11:13,640 --> 00:11:15,080
它有些数据丢失了

339
00:11:15,080 --> 00:11:16,240
那它的精度掉了

340
00:11:16,240 --> 00:11:16,840
这个时候呢

341
00:11:16,840 --> 00:11:19,840
也需要进行缺陷的检测

342
00:11:19,840 --> 00:11:20,960
和回顾的

343
00:11:20,960 --> 00:11:21,600
这个时候呢

344
00:11:21,600 --> 00:11:23,120
因为种种这些原因呢

345
00:11:23,120 --> 00:11:24,840
需要对模型的生命周期

346
00:11:24,840 --> 00:11:25,720
进行管理

347
00:11:25,720 --> 00:11:26,360
那这个呢

348
00:11:26,360 --> 00:11:27,560
就是TensorFlow Serving

349
00:11:27,560 --> 00:11:29,280
这篇文章里面去讲到

350
00:11:29,280 --> 00:11:30,760
它怎么去做的

351
00:11:30,760 --> 00:11:33,080
也非常欢迎大家去看看这篇文章

352
00:11:33,080 --> 00:11:34,080
所以说TF呢

353
00:11:34,080 --> 00:11:36,080
其实在AI系统里面呢

354
00:11:36,080 --> 00:11:39,480
做了非常多相关的工作和创新的

355
00:11:41,280 --> 00:11:42,640
而模型生命周期管理呢

356
00:11:42,640 --> 00:11:43,720
主要有两个

357
00:11:43,720 --> 00:11:44,280
第一个呢

358
00:11:44,280 --> 00:11:46,680
就是Canary的策略

359
00:11:46,680 --> 00:11:47,360
那第二个呢

360
00:11:47,360 --> 00:11:49,560
就是Rollback的策略

361
00:11:49,560 --> 00:11:52,120
那看一下Canary的策略呢

362
00:11:52,120 --> 00:11:54,080
它主要是有几个方面

363
00:11:55,160 --> 00:11:57,200
如果有一个新的模型的版本呢

364
00:11:57,200 --> 00:12:00,200
用户可以同时选择保留这两个版本

365
00:12:00,200 --> 00:12:01,680
就新旧版本你可以随便用

366
00:12:02,680 --> 00:12:03,080
然后呢

367
00:12:03,080 --> 00:12:04,920
会对流量的做一个AB TEST

368
00:12:04,920 --> 00:12:06,520
或者做一个分流

369
00:12:06,520 --> 00:12:08,360
分别推送到两个版本里面

370
00:12:08,360 --> 00:12:10,240
去比较对应的效果

371
00:12:10,240 --> 00:12:10,760
一旦呢

372
00:12:10,760 --> 00:12:14,120
发现最新的版本效果比较好的

373
00:12:14,120 --> 00:12:17,680
那用户就会直接切换到对应的最新的版本

374
00:12:17,680 --> 00:12:18,880
但是这种方式呢

375
00:12:18,880 --> 00:12:22,600
更多的是在互联网厂商里面用的比较多

376
00:12:22,600 --> 00:12:23,120
因为呢

377
00:12:23,120 --> 00:12:25,840
它需要非常多的一个高峰的资源

378
00:12:25,840 --> 00:12:28,640
就非常多的一个流量

379
00:12:28,640 --> 00:12:29,200
避免呢

380
00:12:29,200 --> 00:12:30,400
将大部分的用户呢

381
00:12:30,400 --> 00:12:32,320
都暴露于一些缺陷模型

382
00:12:32,320 --> 00:12:34,120
或者有问题的模型

383
00:12:34,120 --> 00:12:35,680
导致服务中断

384
00:12:36,680 --> 00:12:37,400
那接着呢

385
00:12:37,400 --> 00:12:39,400
第二个就是不管三七二十一

386
00:12:39,400 --> 00:12:41,720
都要做一个Rollback

387
00:12:41,720 --> 00:12:43,360
或者回滚的一个策略

388
00:12:44,440 --> 00:12:47,440
就假设我现在在当前的主要的服务器的版本上面呢

389
00:12:47,440 --> 00:12:49,560
去检测到我的模型是有问题的

390
00:12:49,560 --> 00:12:50,960
或者可能会有问题

391
00:12:50,960 --> 00:12:52,040
或者我升级了

392
00:12:52,040 --> 00:12:52,680
这个时候呢

393
00:12:52,680 --> 00:12:55,520
用户可以请求切换到比较低的版本

394
00:12:55,560 --> 00:12:58,480
所以会同时维护非常多不同的版本

395
00:12:58,480 --> 00:12:59,680
方便去回顾的

396
00:13:01,680 --> 00:13:02,720
然后第二个特点呢

397
00:13:02,720 --> 00:13:07,040
就是我的卸载安装了它的整体的配置的顺序

398
00:13:07,040 --> 00:13:08,440
我的卸载安装的顺序呢

399
00:13:08,440 --> 00:13:09,440
还有模型呢

400
00:13:09,440 --> 00:13:11,120
都是可以配置的

401
00:13:11,120 --> 00:13:12,600
方便解决问题之后呢

402
00:13:12,600 --> 00:13:13,680
推到新的版本

403
00:13:13,680 --> 00:13:15,640
可以结束回滚

404
00:13:15,640 --> 00:13:16,720
那这种方式呢

405
00:13:16,720 --> 00:13:19,320
主要是对版本的维护

406
00:13:19,320 --> 00:13:20,760
我现在接触到的工作呢

407
00:13:20,760 --> 00:13:23,640
更多的是采用这种回滚的策略

408
00:13:26,000 --> 00:13:28,440
好了来总结一下

409
00:13:28,440 --> 00:13:29,000
那今天呢

410
00:13:29,000 --> 00:13:31,960
看了一下推理系统的整体的架构

411
00:13:31,960 --> 00:13:34,240
希望大家能够对推理系统呢

412
00:13:34,240 --> 00:13:37,320
有个比较清晰或者全面的认识

413
00:13:37,320 --> 00:13:38,000
接着呢

414
00:13:38,000 --> 00:13:39,120
往下看一下

415
00:13:39,120 --> 00:13:40,800
今天给大家汇报了一下

416
00:13:40,800 --> 00:13:44,600
Infer、Deploy、Serving的一个相关的概念

417
00:13:44,600 --> 00:13:46,000
搞清楚这个概念之后呢

418
00:13:46,000 --> 00:13:48,840
就比较自然而然的去引入到了

419
00:13:48,840 --> 00:13:52,040
整个推理系统的整体的架构

420
00:13:52,040 --> 00:13:53,040
聊起来架构之后呢

421
00:13:53,040 --> 00:13:54,600
跟AI比较相关的

422
00:13:54,600 --> 00:13:56,760
因为这大部分都是平台性的工作

423
00:13:56,760 --> 00:13:59,720
利于Triton的服务化的工作

424
00:13:59,720 --> 00:14:01,200
对于AI比较相关的

425
00:14:01,200 --> 00:14:03,360
就是模型的生命周期的管理

426
00:14:03,360 --> 00:14:04,000
那其实呢

427
00:14:04,000 --> 00:14:05,360
这个模型生命周期管理呢

428
00:14:05,360 --> 00:14:06,840
是一个非常大的学问

429
00:14:06,840 --> 00:14:08,680
有不同的一个策略

430
00:14:08,680 --> 00:14:09,920
针对不同的业务呢

431
00:14:09,920 --> 00:14:11,440
可能会采用不同的策略

432
00:14:11,440 --> 00:14:12,760
而不是一概而论的

433
00:14:12,760 --> 00:14:13,720
那今天的内容呢

434
00:14:13,720 --> 00:14:14,600
到此为止

435
00:14:14,600 --> 00:14:15,080
好了

436
00:14:15,080 --> 00:14:16,080
谢谢各位

437
00:14:16,080 --> 00:14:16,880
卷的不行了

438
00:14:16,880 --> 00:14:17,760
卷的不行了

439
00:14:17,760 --> 00:14:19,560
记得一键三连加关注哦

440
00:14:19,560 --> 00:14:21,160
所有的内容都会开源在

441
00:14:21,160 --> 00:14:23,160
下面这条链接里面

442
00:14:23,160 --> 00:14:24,120
拜拜