README.magpienetworkfs

Magpie Network File System Option
---------------------------------

As noted in the README file, there are some portability or "gotchas"
when using Rawnetworkfs.  In attempt to get around some of those
issues, the "Magpie Network FS" patch is available as a plugin for
Hadoop 2.0.  It is a file system plugin that is a small layer above
the Raw Local FileSystem.

The patch runs very similarly to rawnetworkfs except that it handles
the following extra cases.

1) It handles the case indicated in

https://issues.apache.org/jira/browse/MAPREDUCE-5528

2) If you specify a relative path, the Magpie Network FS option will
read/write to the base path of HADOOP_MAGPIENETWORKFS_PATH.  The
current working directory also defaults to HADOOP_MAGPIENETWORKFS_PATH
regardless of where you are in the file system tree.

For example, if the user specifies the path "mypath/file", with
rawnetworkfs it would normally be written to the current working
directory the user is running in.  The user would be required to
specify the absolute path to ensure files are written to the networked
file system (e.g. /mynetworkmount/mypath/file).

With this patch, the relative path "mypath/file" would be written to 
${HADOOP_MAGPIENETWORKFS_PATH}/mypath/file.

This can help w/ portability of code/scripts that usually assume HDFS,
which does not have a current working directory.

A patch for Magpie Network FS can be found in the patches directory.

The submission scripts do not include 'MagpieNetworkFS' fields by
default.  To include them:

cd submission-scripts/script-templates
edit Makefile and set INSERT_MAGPIENETWORKFS to 'y'
run 'make'