Fix install document

[jti-lanl]

Several things have changed:

• It's no longer necessary to build individual packages (libaws4c, libconfig, libmarfs). These are all built automatically by marfs/fuse/scripts/build_marfs, given a couple of environment variables.
• Reduce the environment variables to only those actually needed for the updated document. Have them match the ones that are now used by the build_marfs script.
• Configuration is somewhat documented in the blueprint file, and in the LANL-oriented .cfg files. But NS.alias should be explained, regarding Scality config. (The alias now maps to the scality FastCGI path, so that it is no longer necessary to reconfig Scality in order to add a MarFS namespace.)
• Scality config advice is wrong. As noted above, we don't need a new fastcgi path for every namespace.

[cbartz]

I have just tried to follow the Install document and it seems to be pretty outdated from my point of view.

• In 4.2 , the host git.lanl.gov is mentioned, which does not exist anymore. I think, the PA2X code is now on github
• 4.4.1 the build_marfs script requires the env variable LIBNE to be set, this is not mentioned anywhere
• 4.4.2 their does not exist any Makefile in $MARFS/common/configuration/src .

It would be really nice if the Install Guide would be updated soon (perhaps also in an open format, e.g. Markdown). Otherwise, I will have to guess and try to get it installed with a lot more effort and pain as I could not find any tutorial or alternative install guide anywhere else.

[wfvining]

@cbartz you are quite right that our documentation is woefully outdated. The entire build process has been overhauled in the last six months. Issue #151 should give a better idea of how to build marfs now (./configure --help should give a description of environment variables you may want/need).

The build_marfs script is no longer used and should probably be removed.

[jti-lanl]

Hi Chris, Yes, sorry, that document has gotten pretty far out of date. As far as the build goes, we’re using autotools these days. But even with autotools, we have rolled features in as options, which have then become “standard” for us, but still require selection at configure time. I’ll try to give you an off-the-cuff guide. We are also currently moving away from commodity object-stores to a custom implementation we call “Multi-Component”. You can find some presentations online. Basically, we now write erasure-coded “stripes” across an array of servers, which have local ZFS file-systems, instead of using an object-store. I’m not going to try to describe configuration for that, here. I believe the Scality-based install is still working. I’ll also assume you can download all the packages described in the old document. Let $MARFS_BUILD be some directory where you want everything installed. (Currently, libaws4c doesn’t get moved to there, but it should, eventually.) - libaws4c cd $AWS4C make clean make - libne [if you are using multi-component] cd $LIBNE git clone https://github.com/mar-file-system/erasureUtils.git autoreconf -f -i ./configure —prefix=$MARFS_BUILD LDFLAGS="-L$MARFS_BUILD/lib -L$AWS4C” make clean make make install - libmarfs [I see Will beat me to it. But here’s an outline] cd $MARFS autoreconf -f -i ./configure --enable-mc LDFLAGS=“-L$MARFS_BUILD/lib" MARFS_MNT=/campaign make sudo make mnt MARFSCONFIGRC=$MARFSCONFIGRC Note that ‘—enable-mc’ implies you are using multi-component. I think you’d leave that out, if you want to use an object-store as your backing-store. You can add '--enable-logging=stdout’ but (a) the logging output can be voluminous, and (b), it is only visible if you run fuse in “foreground” mode (with -f). Another approach if you want logging without foreground mode is to use '--enable-logging=syslog’, but, again, beware that this can generate a lot of logging output. It’s best used for debugging only, in which case you may also want ‘—enable-debug’. Logging can be helpful figuring out errors in your configuration-file, which otherwise cause somewhat-opaque failures of fuse at start-up. Thanks, Jeff

…

On May 5, 2017, at 9:56 AM, Christopher Bartz ***@***.***> wrote: I have just tried to follow the Install document and it seems to be pretty outdated from my point of view. • In 4.2 , the host git.lanl.gov is mentioned, which does not exist anymore. I think, the PA2X code is now on github • 4.4.1 the build_marfs script requires the env variable LIBNE to be set, this is not mentioned anywhere • 4.4.2 their does not exist any Makefile in $MARFS/common/configuration/src . It would be really nice if the Install Guide would be updated soon (perhaps also in an open format, e.g. Markdown). Otherwise, I will have to guess and try to get it installed with a lot more effort and pain as I could not find any tutorial or alternative install guide anywhere else. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[cbartz]

Thank both of you for your replies.

@jti-lanl You write:

We are also currently moving away from commodity object-stores to a custom implementation we call “Multi-Component”.

Moving away means, you will not support commodity object-stores in the future? Actually I am interested in using MarFS to provide POSIX for already existing object systems.

[thewacokid]

@cbartz - no, we're just moving away from that configuration in our environment. Object stores will still be supported via the DAL (data abstraction layer).