03.srt

1
00:00:00,000 --> 00:00:04,500
字幕校对:米哈游天下第一

2
00:00:04,500 --> 00:00:06,000
Hello 大家好

3
00:00:06,000 --> 00:00:08,700
这又是一节没什么人来观看

4
00:00:08,700 --> 00:00:12,500
但是我依然在坚持的AI系列的课程的课里面

5
00:00:12,500 --> 00:00:16,000
今天我想给大家一起去分享一下框架之争

6
00:00:16,000 --> 00:00:19,500
就是现在有很多AI框架出现了

7
00:00:19,500 --> 00:00:22,500
不管它叫做第一代第二代第三代都好

8
00:00:22,500 --> 00:00:24,500
其实用户关心的就是

9
00:00:24,500 --> 00:00:28,500
现在我有哪些AI框架可以用

10
00:00:29,000 --> 00:00:31,000
我记得在很久之前

11
00:00:31,000 --> 00:00:35,000
应该是16年到18年我刚开始用AI框架的时候

12
00:00:35,000 --> 00:00:38,000
TensorFlow和PyTorch两个非常火

13
00:00:38,000 --> 00:00:41,000
那个时候很多人去问一个问题

14
00:00:41,000 --> 00:00:43,000
我到底是用A框架好呢

15
00:00:43,000 --> 00:00:44,500
还是用B框架好呢

16
00:00:44,500 --> 00:00:47,000
然后到18年到19年的时候

17
00:00:47,000 --> 00:00:50,000
很多人就会去争取一个标题

18
00:00:50,000 --> 00:00:52,500
PyTorch超越了TensorFlow了没有呢

19
00:00:52,500 --> 00:00:55,000
到2020年的时候或者去年

20
00:00:55,500 --> 00:00:59,500
PyTorch和TensorFlow各自又发表了一些官方的文章

21
00:00:59,500 --> 00:01:01,000
里面就去澄清

22
00:01:01,000 --> 00:01:03,500
TensorFlow依然占据着主流的位置

23
00:01:03,500 --> 00:01:06,000
PyTorch的上升趋势非常大

24
00:01:06,000 --> 00:01:08,000
框架之争这个概念

25
00:01:08,000 --> 00:01:12,000
是随着AI这个市场或者AI规模化越来越大

26
00:01:12,000 --> 00:01:16,000
所以大家都想抢夺类似于操作系统的地位

27
00:01:16,000 --> 00:01:19,000
垄断最底层的AI的核心技术

28
00:01:19,000 --> 00:01:23,000
那今天我想给大家一起去分享一下

29
00:01:23,000 --> 00:01:27,000
关于我对AI框架之争的一些看法和想法

30
00:01:27,000 --> 00:01:29,000
还有一些总结的心得体会

31
00:01:29,000 --> 00:01:31,000
那可以看到啊

32
00:01:31,000 --> 00:01:33,000
其实在2010年之前

33
00:01:33,000 --> 00:01:36,000
AI框架那时候还没有形成

34
00:01:36,000 --> 00:01:38,000
也没有东西叫做AI框架

35
00:01:38,000 --> 00:01:40,000
右边用的更多的是NumPy

36
00:01:40,000 --> 00:01:43,000
SciPy还有Metlab

37
00:01:43,000 --> 00:01:46,000
注意千万不要忽略Metlab这个里面

38
00:01:46,000 --> 00:01:49,000
会提供一些神经网络的接口

39
00:01:49,000 --> 00:01:50,000
因为呢

40
00:01:50,000 --> 00:01:52,000
最近应该是两个月前

41
00:01:52,000 --> 00:01:56,000
我去西南中医药大学去拜访的时候

42
00:01:56,000 --> 00:01:59,000
他们有一个药学智能学院

43
00:01:59,000 --> 00:02:00,000
里面的老师

44
00:02:00,000 --> 00:02:03,000
现在的教学还是用Matlab进行教学

45
00:02:03,000 --> 00:02:05,000
因为他们觉得Matlab有给他们提供

46
00:02:05,000 --> 00:02:08,000
一个比较简单的神经网络的系统

47
00:02:08,000 --> 00:02:09,000
他们的课程呢

48
00:02:09,000 --> 00:02:11,000
也有十几年没有更新过了

49
00:02:11,000 --> 00:02:14,000
里面还是用Matlab做一些心电数据的分析

50
00:02:14,000 --> 00:02:15,000
那这个呢

51
00:02:15,000 --> 00:02:17,000
分析的内容还是用神经网络

52
00:02:17,000 --> 00:02:20,000
那时候很早之前已经有神经网络了

53
00:02:20,000 --> 00:02:22,000
只是没有大规模的铺开

54
00:02:22,000 --> 00:02:24,000
那所以不要觉得Matlab很low

55
00:02:24,000 --> 00:02:26,000
其实Matlab现在还有很多人拿用的

56
00:02:26,000 --> 00:02:29,000
只是随着美国对中国的打压

57
00:02:29,000 --> 00:02:32,000
现在很多国内的高校和科研机构呢

58
00:02:32,000 --> 00:02:34,000
已经不能使用Matlab了

59
00:02:38,000 --> 00:02:39,000
在2010年之前呢

60
00:02:39,000 --> 00:02:41,000
那个时候还没有AI框架

61
00:02:41,000 --> 00:02:42,000
机器学习呢

62
00:02:42,000 --> 00:02:45,000
缺乏一些领域的专用库

63
00:02:45,000 --> 00:02:48,000
然后只能提供一些神经网络

64
00:02:48,000 --> 00:02:50,000
能够简单表示的一些接口

65
00:02:50,000 --> 00:02:52,000
那个时候这些库啊

66
00:02:52,000 --> 00:02:53,000
右边的这些库

67
00:02:53,000 --> 00:02:56,000
最重要的特点就是提供一些脚本式的编程

68
00:02:56,000 --> 00:02:57,000
简单的编程

69
00:02:57,000 --> 00:02:59,000
或者通过像Matlab

70
00:02:59,000 --> 00:03:00,000
通过一些简单的配置

71
00:03:00,000 --> 00:03:03,000
形成一些神经网络的接口

72
00:03:03,000 --> 00:03:06,000
优点就是在当时的那个情况下

73
00:03:06,000 --> 00:03:08,000
AI或者神经网络深度学习

74
00:03:08,000 --> 00:03:09,000
还没有起来的时候呢

75
00:03:09,000 --> 00:03:12,000
提供了一定程度的可编程性

76
00:03:12,000 --> 00:03:13,000
而Matlab呢

77
00:03:13,000 --> 00:03:14,000
还有那个NumPy呢

78
00:03:14,000 --> 00:03:17,000
就提供了关于CPU的计算加速

79
00:03:17,000 --> 00:03:19,000
那个时候或者那个时期

80
00:03:19,000 --> 00:03:21,000
能够有这些库出现

81
00:03:21,000 --> 00:03:22,000
还是很不容易的

82
00:03:22,000 --> 00:03:25,000
也是非常具有前瞻性的

83
00:03:25,000 --> 00:03:27,000
那其实在2010年之前

84
00:03:27,000 --> 00:03:30,000
或者在到AI正式起来之前呢

85
00:03:30,000 --> 00:03:32,000
还有另外一个过渡时期

86
00:03:32,000 --> 00:03:33,000
那个时候呢

87
00:03:33,000 --> 00:03:35,000
以caffe作为著名的代表

88
00:03:35,000 --> 00:03:36,000
里面呢就以CNN

89
00:03:36,000 --> 00:03:38,000
因为卷积神经网络

90
00:03:38,000 --> 00:03:42,000
或者视觉方面的一个AI火的非常快

91
00:03:42,000 --> 00:03:45,000
然后呢经常用一些layer去做一个组成的

92
00:03:45,000 --> 00:03:48,000
所以它比较著名的特点就是layer base

93
00:03:48,000 --> 00:03:50,000
就是基于网络层数来定义的

94
00:03:50,000 --> 00:03:51,000
一层套一层

95
00:03:51,000 --> 00:03:53,000
那右边这个图可以看到

96
00:03:53,000 --> 00:03:54,000
layer 1

97
00:03:54,000 --> 00:03:55,000
然后layer 2

98
00:03:55,000 --> 00:03:57,000
通过不断的层数的添加

99
00:03:57,000 --> 00:04:00,000
然后表示右边的这个图

100
00:04:00,000 --> 00:04:01,000
不断的一层一层

101
00:04:01,000 --> 00:04:03,000
当时应该是五年前的

102
00:04:03,000 --> 00:04:05,000
这十年前的神经网络

103
00:04:05,000 --> 00:04:06,000
还是比较简单

104
00:04:06,000 --> 00:04:08,000
只是一层一层网络模型

105
00:04:08,000 --> 00:04:10,000
不断的去堆叠下来

106
00:04:10,000 --> 00:04:12,000
那现在的神经网络

107
00:04:12,000 --> 00:04:13,000
BERT Transformer那种

108
00:04:13,000 --> 00:04:16,000
虽然大的结构上还是一层一层

109
00:04:16,000 --> 00:04:18,000
但实际上它们已经演变了非常多的

110
00:04:18,000 --> 00:04:20,000
不同灵活组合的方式

111
00:04:20,000 --> 00:04:22,000
那个时候另外一个特点就是

112
00:04:22,000 --> 00:04:25,000
支持CPU和GPU的高效的计算的

113
00:04:25,000 --> 00:04:28,000
也就是慢慢的引入了GPU

114
00:04:28,000 --> 00:04:29,000
支持GPU的加速

115
00:04:29,000 --> 00:04:31,000
那这个就是它的一个优点

116
00:04:31,000 --> 00:04:32,000
另外一个优点呢

117
00:04:32,000 --> 00:04:35,000
就提供了一定程度的可编程性

118
00:04:35,000 --> 00:04:36,000
就是像右边的这个图

119
00:04:36,000 --> 00:04:39,000
我可以一定程度的通过一些配置

120
00:04:39,000 --> 00:04:42,000
把整个神经网络运作起来

121
00:04:42,000 --> 00:04:44,000
当然这一系列的神经网络

122
00:04:44,000 --> 00:04:47,000
或者这一系列的库AI框架

123
00:04:47,000 --> 00:04:49,000
它受制于当时时代的发展

124
00:04:49,000 --> 00:04:51,000
有自身的局限性

125
00:04:51,000 --> 00:04:52,000
那第一个局限性呢

126
00:04:52,000 --> 00:04:55,000
叫做Limitation 1

127
00:04:55,000 --> 00:04:57,000
限制了整个AI框架的灵活性

128
00:04:57,000 --> 00:04:59,000
不能够很好的满足

129
00:04:59,000 --> 00:05:01,000
深度学习的快速发展

130
00:05:01,000 --> 00:05:02,000
那可以看到

131
00:05:02,000 --> 00:05:04,000
深度学习的快速发展呢

132
00:05:04,000 --> 00:05:05,000
有4点

133
00:05:05,000 --> 00:05:06,000
我所总结的4点

134
00:05:06,000 --> 00:05:09,000
第一点呢就是网络模型层出不穷

135
00:05:09,000 --> 00:05:11,000
从RNN,LSTM

136
00:05:11,000 --> 00:05:13,000
再到Transformer

137
00:05:13,000 --> 00:05:15,000
网络模型结构非常夸张

138
00:05:15,000 --> 00:05:17,000
然后新的层呢

139
00:05:17,000 --> 00:05:20,000
就会引入新的前项和后项的计算

140
00:05:20,000 --> 00:05:22,000
如果只是给提供了一个层

141
00:05:22,000 --> 00:05:24,000
让去改参数

142
00:05:24,000 --> 00:05:26,000
可能满足不了自定义的一些操作

143
00:05:26,000 --> 00:05:27,000
还有像Caffe呢

144
00:05:27,000 --> 00:05:29,000
它提供了一些非高级语言

145
00:05:29,000 --> 00:05:30,000
就是非Python

146
00:05:30,000 --> 00:05:33,000
还是集中在C++或者C这种方式

147
00:05:33,000 --> 00:05:36,000
那另外新的优化器呢

148
00:05:36,000 --> 00:05:38,000
对梯度的计算和参数的运算呢

149
00:05:38,000 --> 00:05:39,000
会更加复杂

150
00:05:39,000 --> 00:05:42,000
不仅仅是通过配置来控制网络模型

151
00:05:42,000 --> 00:05:46,000
还是希望控制整个运算的过程

152
00:05:46,000 --> 00:05:47,000
第二个约束呢

153
00:05:47,000 --> 00:05:49,000
就是现在神经网络呀

154
00:05:49,000 --> 00:05:52,000
它除了简单的LSTM Transformer

155
00:05:52,000 --> 00:05:54,000
这些网络模型结构的更新之外

156
00:05:54,000 --> 00:05:56,000
它还有新的训练的方式的更新

157
00:05:56,000 --> 00:05:59,000
例如LSTM RNN循环

158
00:05:59,000 --> 00:06:02,000
还有GAN这种左右两个网络互相博弈的

159
00:06:02,000 --> 00:06:05,000
像强化学习这种通过多轮迭代

160
00:06:05,000 --> 00:06:07,000
还有最新的Diffusion Model

161
00:06:07,000 --> 00:06:12,000
这些新的训练方式已经很难满足了

162
00:06:12,000 --> 00:06:13,000
于是呢

163
00:06:13,000 --> 00:06:17,000
后面在第二个阶段应该是2015年之后

164
00:06:17,000 --> 00:06:19,000
或者2016年之后了

165
00:06:19,000 --> 00:06:23,000
就出现了基于数据流图的计算框架的出现了

166
00:06:23,000 --> 00:06:25,000
就是叫做Base DAG

167
00:06:25,000 --> 00:06:27,000
DAG就是数据流图

168
00:06:27,000 --> 00:06:31,000
正是因为基于数据流图的AI计算框架起来

169
00:06:31,000 --> 00:06:34,000
所以会出现刚才前面所讲的那一幕

170
00:06:34,000 --> 00:06:38,000
很多人还在讨论用哪个AI框架更简单

171
00:06:38,000 --> 00:06:42,000
那来看看这个基于数据流图的计算框架

172
00:06:42,000 --> 00:06:44,000
AI框架有什么不一样

173
00:06:44,000 --> 00:06:46,000
首先第一个它是很重要的

174
00:06:46,000 --> 00:06:49,000
有一个基本的数据结构叫做张量

175
00:06:49,000 --> 00:06:51,000
Tensorflow谷歌的Tensorflow

176
00:06:51,000 --> 00:06:56,000
很重要的那个名字叫做Tensorflow张量流

177
00:06:56,000 --> 00:06:58,000
它里面的AI框架就叫做张量流

178
00:06:58,000 --> 00:07:00,000
所以张量这个数据结构呢

179
00:07:00,000 --> 00:07:02,000
是非常重要的

180
00:07:02,000 --> 00:07:04,000
里面张量可以代表非常多的

181
00:07:04,000 --> 00:07:06,000
不同的数据类型数据格式

182
00:07:06,000 --> 00:07:07,000
那第二个呢

183
00:07:07,000 --> 00:07:09,000
就是基本的运算单元

184
00:07:09,000 --> 00:07:12,000
叫做operator或者primitive operator

185
00:07:12,000 --> 00:07:14,000
最原始的计算单位

186
00:07:14,000 --> 00:07:17,000
然后它有很多代数的算子组成

187
00:07:17,000 --> 00:07:19,000
例如经常用到的

188
00:07:19,000 --> 00:07:21,000
加减乘除求开号sin cos

189
00:07:21,000 --> 00:07:24,000
这些都是基本的运算算子

190
00:07:24,000 --> 00:07:28,000
有了这些运算算子和基本的张量之后呢

191
00:07:28,000 --> 00:07:32,000
就可以把整个图构建起来了

192
00:07:32,000 --> 00:07:33,000
那这个图里面呢

193
00:07:33,000 --> 00:07:34,000
有一个非常重要的概念

194
00:07:34,000 --> 00:07:37,000
就是它是DAG有向无环图

195
00:07:37,000 --> 00:07:40,000
就是我图里面是有一个方向性的

196
00:07:40,000 --> 00:07:42,000
但是它没有回环

197
00:07:42,000 --> 00:07:44,000
就是我的图每一次都是有个方向的

198
00:07:44,000 --> 00:07:46,000
然后指向最终的输出

199
00:07:46,000 --> 00:07:48,000
如果我有回环的时候

200
00:07:48,000 --> 00:07:50,000
就会引申一个问题

201
00:07:50,000 --> 00:07:52,000
我的参数到底是如何更新的

202
00:07:52,000 --> 00:07:54,000
所以都叫做DAG图

203
00:07:54,000 --> 00:07:56,000
那DAG图里面呢

204
00:07:56,000 --> 00:07:57,000
有一个比较特殊的操作

205
00:07:57,000 --> 00:07:59,000
就是控制流

206
00:07:59,000 --> 00:08:02,000
控制流会在后面去单独的去了解一下的

207
00:08:02,000 --> 00:08:05,000
基于DAG的主要两个AI框架呢

208
00:08:05,000 --> 00:08:07,000
一个是谷歌的TensorFlow

209
00:08:07,000 --> 00:08:09,000
一个是Facebook的PyTorch

210
00:08:09,000 --> 00:08:11,000
它们代表了深度学习里面的

211
00:08:11,000 --> 00:08:13,000
两种截然不同的设计的路径

212
00:08:13,000 --> 00:08:16,000
一个TensorFlow是非常注重于性能

213
00:08:16,000 --> 00:08:18,000
比灵活性更加优先

214
00:08:18,000 --> 00:08:19,000
那PyTorch呢

215
00:08:19,000 --> 00:08:21,000
更加注重于灵活性、易用性

216
00:08:21,000 --> 00:08:23,000
比性能更加优先

217
00:08:23,000 --> 00:08:26,000
那看看下面市场或者学术界

218
00:08:26,000 --> 00:08:28,000
整个的选择

219
00:08:28,000 --> 00:08:29,000
橙色的这个呢

220
00:08:29,000 --> 00:08:31,000
就是代表TensorFlow

221
00:08:31,000 --> 00:08:34,000
然后可以看到TensorFlow的市场份额

222
00:08:34,000 --> 00:08:36,000
或者它整体的学术的研究

223
00:08:36,000 --> 00:08:38,000
使用的越来越少

224
00:08:38,000 --> 00:08:39,000
而PyTorch呢

225
00:08:39,000 --> 00:08:41,000
它从一开始比TensorFlow少的

226
00:08:41,000 --> 00:08:42,000
然后越来越多

227
00:08:42,000 --> 00:08:44,000
就是因为灵活性、易用性

228
00:08:44,000 --> 00:08:47,000
对大家来说实在是太重要了

229
00:08:47,000 --> 00:08:50,000
在不是说性能损耗的非常严重的情况下

230
00:08:50,000 --> 00:08:52,000
大家更倾向于使用一些

231
00:08:52,000 --> 00:08:55,000
能够简单易学的AI框架

232
00:08:57,000 --> 00:09:01,000
至少AI框架呢

233
00:09:01,000 --> 00:09:04,000
还是朝着未来的方向去演进的

234
00:09:04,000 --> 00:09:05,000
那这里面呢

235
00:09:05,000 --> 00:09:07,000
有一个比较重要的概念

236
00:09:07,000 --> 00:09:09,000
叫做特定领域语言

237
00:09:09,000 --> 00:09:12,000
叫做Domain Specific Language

238
00:09:12,000 --> 00:09:14,000
那面向于特定领域语言呢

239
00:09:14,000 --> 00:09:16,000
就是AI或者深度学习

240
00:09:16,000 --> 00:09:18,000
作为一种特殊的领域

241
00:09:18,000 --> 00:09:19,000
或者特殊的应用

242
00:09:19,000 --> 00:09:22,000
它又衍生了自己的领域语言

243
00:09:22,000 --> 00:09:23,000
像PyTorch JIT

244
00:09:23,000 --> 00:09:24,000
MindSpot

245
00:09:24,000 --> 00:09:25,000
JAX

246
00:09:25,000 --> 00:09:26,000
TF Eager

247
00:09:26,000 --> 00:09:27,000
还有taichi

248
00:09:27,000 --> 00:09:29,000
这种新的框架的出现

249
00:09:29,000 --> 00:09:32,000
为的就是解决某些特定领域的需求

250
00:09:32,000 --> 00:09:33,000
例如MindSpore

251
00:09:33,000 --> 00:09:34,000
在科学计算里面呢

252
00:09:34,000 --> 00:09:36,000
是非常好用的

253
00:09:36,000 --> 00:09:37,000
因为它可以灵活的

254
00:09:37,000 --> 00:09:38,000
做一些分布式并行

255
00:09:38,000 --> 00:09:40,000
高阶微分

256
00:09:40,000 --> 00:09:41,000
而JAX呢

257
00:09:41,000 --> 00:09:43,000
它利用TensorFlow的TileA

258
00:09:43,000 --> 00:09:44,000
然后加上Numpy

259
00:09:44,000 --> 00:09:46,000
可以非常灵活的

260
00:09:46,000 --> 00:09:48,000
去表达一些科学计算的问题

261
00:09:48,000 --> 00:09:49,000
像taichi

262
00:09:49,000 --> 00:09:51,000
它能够很方便的

263
00:09:51,000 --> 00:09:52,000
做一些GPU的渲染

264
00:09:52,000 --> 00:09:54,000
还有物理仿真的能力

265
00:09:54,000 --> 00:09:55,000
那这些呢

266
00:09:55,000 --> 00:09:56,000
就是特定领域语言

267
00:09:56,000 --> 00:09:58,000
所使用的框架

268
00:09:58,000 --> 00:10:00,000
也是整个学术界

269
00:10:00,000 --> 00:10:01,000
大家都叫做

270
00:10:01,000 --> 00:10:04,000
第三代AI框架的一个趋势

271
00:10:04,000 --> 00:10:05,000
那这里面的这个趋势呢

272
00:10:05,000 --> 00:10:07,000
做一个简单的总结

273
00:10:07,000 --> 00:10:08,000
或者讨论

274
00:10:08,000 --> 00:10:09,000
就是它兼顾了

275
00:10:09,000 --> 00:10:11,000
整个编程的灵活性

276
00:10:11,000 --> 00:10:13,000
还有计算的高效性

277
00:10:13,000 --> 00:10:14,000
能够非常很好的

278
00:10:14,000 --> 00:10:16,000
去表达神经网络

279
00:10:16,000 --> 00:10:17,000
同时呢

280
00:10:17,000 --> 00:10:19,000
它又通过编译优化的手段

281
00:10:19,000 --> 00:10:20,000
来改善像PyTorch

282
00:10:20,000 --> 00:10:23,000
一开始可能性能不太优的阶段

283
00:10:23,000 --> 00:10:24,000
那右边的这个图呢

284
00:10:24,000 --> 00:10:26,000
就是现在或者

285
00:10:26,000 --> 00:10:28,000
现代第三代AI框架

286
00:10:28,000 --> 00:10:29,000
基本上就是

287
00:10:29,000 --> 00:10:31,000
殊途同归的一种方式

288
00:10:38,000 --> 00:10:39,000
那可以看看

289
00:10:39,000 --> 00:10:40,000
右边的这个图

290
00:10:40,000 --> 00:10:41,000
首先往上层的

291
00:10:41,000 --> 00:10:43,000
就是前端的编程语言

292
00:10:43,000 --> 00:10:45,000
有了前端的编程语言

293
00:10:45,000 --> 00:10:46,000
或者高级语言之后呢

294
00:10:46,000 --> 00:10:48,000
就需要最核心的机制

295
00:10:48,000 --> 00:10:49,000
自动求导

296
00:10:49,000 --> 00:10:50,000
那自动求导

297
00:10:50,000 --> 00:10:51,000
也叫做自动微分

298
00:10:51,000 --> 00:10:53,000
在之前的那个系列

299
00:10:53,000 --> 00:10:54,000
已经详细的讲过了

300
00:10:54,000 --> 00:10:56,000
有了这个前端的表示之后呢

301
00:10:56,000 --> 00:10:58,000
会把所有的一切

302
00:10:58,000 --> 00:11:00,000
变成一个统一的表示

303
00:11:00,000 --> 00:11:01,000
那统一的表示呢

304
00:11:01,000 --> 00:11:02,000
在第二代AI框架里面

305
00:11:02,000 --> 00:11:04,000
最著名的就是DAG

306
00:11:04,000 --> 00:11:07,000
现在很多也会去使用DAG图

307
00:11:07,000 --> 00:11:09,000
去对计算机的一些指令流

308
00:11:09,000 --> 00:11:10,000
进行表示

309
00:11:10,000 --> 00:11:11,000
那表示完之后

310
00:11:11,000 --> 00:11:12,000
有了一个统一表示

311
00:11:12,000 --> 00:11:13,000
肯定需要一个

312
00:11:13,000 --> 00:11:15,000
对统一表示进行优化和调度

313
00:11:15,000 --> 00:11:16,000
于是呢

314
00:11:16,000 --> 00:11:19,000
就出现了图的优化和调度的编译执行

315
00:11:19,000 --> 00:11:20,000
有了这一层

316
00:11:20,000 --> 00:11:23,000
还停留在编译层的上面

317
00:11:23,000 --> 00:11:24,000
实际上代码执行

318
00:11:24,000 --> 00:11:25,000
还需要进行

319
00:11:25,000 --> 00:11:27,000
内核代码的优化和编译

320
00:11:27,000 --> 00:11:29,000
最终编译层及其指令

321
00:11:29,000 --> 00:11:30,000
让计算机的

322
00:11:30,000 --> 00:11:33,000
不同的硬件去执行

323
00:11:33,000 --> 00:11:35,000
这个呢就是现代AI框架

324
00:11:35,000 --> 00:11:37,000
基本上都会去采用的

325
00:11:37,000 --> 00:11:39,000
这种分层结构

326
00:11:39,000 --> 00:11:41,000
可能每一层结构里面的细节

327
00:11:41,000 --> 00:11:42,000
会有点不一样

328
00:11:42,000 --> 00:11:43,000
例如Mindspore

329
00:11:43,000 --> 00:11:45,000
它可能就会把自动求导

330
00:11:45,000 --> 00:11:47,000
和这一坨放在一起

331
00:11:47,000 --> 00:11:48,000
而PyTorch呢

332
00:11:48,000 --> 00:11:50,000
可能就少了这个图的优化

333
00:11:50,000 --> 00:11:51,000
和图的执行调度

334
00:11:51,000 --> 00:11:52,000
这一个模块

335
00:11:52,000 --> 00:11:54,000
或者少了编译这个模块

336
00:11:54,000 --> 00:11:57,000
或者它的编译模块非常轻量

337
00:11:57,000 --> 00:11:59,000
从左右两边可以看到

338
00:11:59,000 --> 00:12:01,000
一个呢是追求极致的性能

339
00:12:01,000 --> 00:12:04,000
一个是追求极致的应用性

340
00:12:04,000 --> 00:12:05,000
那像应用性方面呢

341
00:12:05,000 --> 00:12:08,000
可以像一个基于Python原生的语言

342
00:12:08,000 --> 00:12:11,000
有那个Numpy和SciPy这两种

343
00:12:11,000 --> 00:12:12,000
这也是所谓的

344
00:12:12,000 --> 00:12:15,000
第一代AI框架所解决的问题

345
00:12:15,000 --> 00:12:17,000
研究学者想基于AI框架

346
00:12:17,000 --> 00:12:19,000
去写一些神经网络的时候呢

347
00:12:19,000 --> 00:12:21,000
只能基于这些库来去实现

348
00:12:21,000 --> 00:12:22,000
那后来呢

349
00:12:22,000 --> 00:12:25,000
应该也是第一代跟第二代过渡之间

350
00:12:25,000 --> 00:12:27,000
有了caffe这个框架

351
00:12:27,000 --> 00:12:29,000
它呢主要是基于layer based的

352
00:12:29,000 --> 00:12:31,000
就是非常方便AI做部署

353
00:12:31,000 --> 00:12:33,000
那那个时候的AI

354
00:12:33,000 --> 00:12:35,000
其实形态没有太多样化

355
00:12:35,000 --> 00:12:38,000
在15年16年到14年之间呢

356
00:12:38,000 --> 00:12:41,000
能够满足AI的时代的发展

357
00:12:41,000 --> 00:12:43,000
然后通过简单的配置

358
00:12:43,000 --> 00:12:45,000
就可以把整个AI跑起来了

359
00:12:45,000 --> 00:12:47,000
这也是我当时候

360
00:12:47,000 --> 00:12:49,000
在研究室里面买的1080

361
00:12:49,000 --> 00:12:51,000
然后跑caffe就觉得

362
00:12:51,000 --> 00:12:53,000
哇原来搞AI这么简单

363
00:12:53,000 --> 00:12:56,000
但是后来网络模型越来越多

364
00:12:56,000 --> 00:12:57,000
越来越复杂

365
00:12:57,000 --> 00:12:58,000
出现了GAN网络

366
00:12:58,000 --> 00:13:00,000
用caffe就很难去表示了

367
00:13:00,000 --> 00:13:02,000
要自己写很多C++的代码

368
00:13:02,000 --> 00:13:04,000
这个时候Tensorflow出现了

369
00:13:04,000 --> 00:13:06,000
或者基于那个数据流图

370
00:13:06,000 --> 00:13:07,000
静态图

371
00:13:07,000 --> 00:13:08,000
那个时候还叫静态图

372
00:13:08,000 --> 00:13:11,000
的一种Tensorflow的编程方式出现了

373
00:13:11,000 --> 00:13:12,000
Tensorflow当时出现

374
00:13:12,000 --> 00:13:14,000
给我的一种感觉就是

375
00:13:14,000 --> 00:13:15,000
惊艳

376
00:13:15,000 --> 00:13:17,000
Tensorflow之上产生的keras

377
00:13:17,000 --> 00:13:20,000
我觉得真的是神一般的框架存在

378
00:13:20,000 --> 00:13:21,000
那个时候就觉得

379
00:13:21,000 --> 00:13:24,000
Tensorflow解决了开发效率的问题

380
00:13:24,000 --> 00:13:25,000
18年的时候

381
00:13:25,000 --> 00:13:28,000
我的好哥们就给我推荐了PyTorch

382
00:13:28,000 --> 00:13:31,000
他说PyTorch去写NLP的一些

383
00:13:31,000 --> 00:13:33,000
他说PyTorch这种基于命令式编程

384
00:13:33,000 --> 00:13:35,000
或者动态图的方式

385
00:13:35,000 --> 00:13:36,000
非常方便

386
00:13:36,000 --> 00:13:39,000
他去解决NLP的任务

387
00:13:39,000 --> 00:13:40,000
那这个时候呢

388
00:13:40,000 --> 00:13:41,000
我用上了PyTorch之后

389
00:13:41,000 --> 00:13:43,000
发现一个很惊艳

390
00:13:43,000 --> 00:13:44,000
更加惊艳的问题

391
00:13:44,000 --> 00:13:46,000
其实应用性跟效率

392
00:13:46,000 --> 00:13:48,000
你如果说Tensorflow很重要

393
00:13:48,000 --> 00:13:50,000
解决了用户开发效率

394
00:13:50,000 --> 00:13:51,000
那PyTorch呢

395
00:13:51,000 --> 00:13:54,000
解决了用户好不好用的问题

396
00:13:54,000 --> 00:13:58,000
面向特殊语言的一些新的任务需求

397
00:13:58,000 --> 00:13:59,000
这个时候呢

398
00:13:59,000 --> 00:14:01,000
又出现了新的AI框架

399
00:14:01,000 --> 00:14:03,000
它们融合了第二代里面数据流图

400
00:14:03,000 --> 00:14:05,000
Tensorflow和PyTorch

401
00:14:05,000 --> 00:14:06,000
之间的一些优缺点

402
00:14:06,000 --> 00:14:09,000
也结合了领域的一些特殊性

403
00:14:09,000 --> 00:14:10,000
然后做了一些优化

404
00:14:10,000 --> 00:14:12,000
生了新的AI框架

405
00:14:12,000 --> 00:14:14,000
这就是未来的发展趋势

406
00:14:14,000 --> 00:14:15,000
那下面呢

407
00:14:15,000 --> 00:14:18,000
可能从更加宏观的一个角度

408
00:14:18,000 --> 00:14:20,000
然后刚才只是讲了framework

409
00:14:20,000 --> 00:14:21,000
这一个层次

410
00:14:21,000 --> 00:14:23,000
看看hardware

411
00:14:23,000 --> 00:14:24,000
因为每一个时代啊

412
00:14:24,000 --> 00:14:27,000
它对应的一些硬件是不一样的

413
00:14:27,000 --> 00:14:28,000
随着软件的更新

414
00:14:28,000 --> 00:14:31,000
硬件也在快速的迭代

415
00:14:31,000 --> 00:14:32,000
那在第一代的时候呢

416
00:14:32,000 --> 00:14:34,000
其实那个时候并没有出现

417
00:14:34,000 --> 00:14:36,000
太多的特殊的硬件

418
00:14:36,000 --> 00:14:38,000
例如Numpy或者SciPy

419
00:14:38,000 --> 00:14:41,000
主要是解决CPU的问题

420
00:14:41,000 --> 00:14:43,000
后来出现了caffe或者theano这些框架

421
00:14:43,000 --> 00:14:47,000
把GPU的能力应用在深度学习里面

422
00:14:47,000 --> 00:14:48,000
在第二代里面呢

423
00:14:48,000 --> 00:14:50,000
出现了Tensorflow和PyTorch

424
00:14:50,000 --> 00:14:51,000
像Tensorflow呢

425
00:14:51,000 --> 00:14:52,000
谷歌就主打了一个TPU

426
00:14:52,000 --> 00:14:55,000
他们针对神经网络设计了特殊的硬件

427
00:14:55,000 --> 00:14:56,000
那在第三代啊

428
00:14:56,000 --> 00:14:59,000
就是面向于特殊的领域语言

429
00:14:59,000 --> 00:15:01,000
这个时候可能从SIMD

430
00:15:01,000 --> 00:15:03,000
MIMD的一个过渡

431
00:15:03,000 --> 00:15:04,000
或者会引入一些

432
00:15:04,000 --> 00:15:06,000
更加稀疏性的需求

433
00:15:06,000 --> 00:15:07,000
还有空字流

434
00:15:07,000 --> 00:15:09,000
在计算机体系结构里面

435
00:15:09,000 --> 00:15:12,000
做一些存算一体的特殊的优化

436
00:15:12,000 --> 00:15:14,000
所以面向硬件的发展

437
00:15:14,000 --> 00:15:17,000
也是伴随软件AI框架的发展

438
00:15:17,000 --> 00:15:19,000
不能割裂软件去看硬件

439
00:15:19,000 --> 00:15:21,000
也不能割裂硬件去看软件

440
00:15:21,000 --> 00:15:24,000
最后又到了大家喜闻乐见的环节

441
00:15:24,000 --> 00:15:26,000
那这个课程里面呢

442
00:15:26,000 --> 00:15:29,000
整体回顾了AI框架的发展趋势

443
00:15:29,000 --> 00:15:31,000
从第一代AI框架去解决了

444
00:15:31,000 --> 00:15:33,000
深度学习可编程的问题

445
00:15:33,000 --> 00:15:36,000
到第二代AI框架去加速

446
00:15:36,000 --> 00:15:38,000
整个科研和产业的落地

447
00:15:38,000 --> 00:15:39,000
那现在呢

448
00:15:39,000 --> 00:15:41,000
应该是刚迈进第三代

449
00:15:41,000 --> 00:15:43,000
AI框架的发展历程当中

450
00:15:43,000 --> 00:15:44,000
里面呢

451
00:15:44,000 --> 00:15:46,000
就结合了特定领域语言或者任务

452
00:15:46,000 --> 00:15:47,000
正在快速发展

453
00:15:47,000 --> 00:15:48,000
但是呢

454
00:15:48,000 --> 00:15:50,000
这个领域或者第三代呢

455
00:15:50,000 --> 00:15:52,000
没有完全收敛

456
00:15:52,000 --> 00:15:53,000
还一起了解了

457
00:15:53,000 --> 00:15:55,000
AI框架或者AI这个产业

458
00:15:55,000 --> 00:15:58,000
是伴随着软硬件的发展升级

459
00:15:58,000 --> 00:15:59,000
而共同发展的

460
00:15:59,000 --> 00:16:01,000
不能割裂软件而看硬件

461
00:16:01,000 --> 00:16:03,000
也不能割裂硬件而看软件

462
00:16:03,000 --> 00:16:04,000
好了

463
00:16:04,000 --> 00:16:05,000
谢谢各位

464
00:16:05,000 --> 00:16:06,000
拜了个拜

465
00:16:06,000 --> 00:16:07,000
卷的不行了

466
00:16:07,000 --> 00:16:08,000
卷的不行了

467
00:16:08,000 --> 00:16:10,000
记得一键三连加关注哦

468
00:16:10,000 --> 00:16:12,000
所有的内容都会开源在

469
00:16:12,000 --> 00:16:14,000
下面这条链接里面

470
00:16:14,000 --> 00:16:15,000
拜了个拜