You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Then, before a task is run, the node manager copies the files from the distributed filesystem to a local
disk—the cache—so the task can access the files. The files are said to be localized at this point. From the task’s point of view, the files are just there, symbolically linked from the task’s working directory.
<Hadoop: The Definitive Guide, Fourth Edition> ch9 distributed cache
hadoop用distributedcache发布数据应该是共享的。
每次task开始前会初始化,这里面会包括distributedcache下载数据。
这个操作在每个node上只能顺序进行。多个task的话只能一个一个等。
如果已经下载过了就不会重复下载。
The text was updated successfully, but these errors were encountered: