Compiling Chombo

You should checkout the adapted Chombo 3.2 version from this git repository (currently GRChombo will not compile with the standard 3.2 version of Chombo).

The installation parameters can be set by creating a file Make.defs.local in the folder

Chombo/lib/mk/

This file does not exist so you need to create it either by copying the file Make.defs.local.template:

cp Make.defs.local.template Make.defs.local

or (which may be easier) by copying one of the example Make.defs.local files for a cluster similar to your own in the InstallNotes/MakeDefsExamples folder of GRChombo. This folder contains examples for Make.defs.local files for machines we have already run GRChombo on.

As noted in Prerequisites, there are several module files that you may need to load first, depending on your cluster. For example, the intel and gcc compilers, the hdf5 parallel library, intel/mkl and PAPI (if you wish to use it for profiling the code).

GRChombo has been compiled and tested with gcc-4.9 and higher and intel icc-16.0.3 and higher. For icc note that the gcc compatibility has to be set to gcc-4.9 or higher.

GRChombo requires c++ standard 14 or higher so the compiler should be invoked with -std=c++14 (or higher).

The main things which need to be set are the direct paths to the hdf5 libraries:

USE_HDF          = TRUE
HDFINCFLAGS      = -I<path to hdf5-serial>/hdf5/include
HDFLIBFLAGS      = -L<path to hdf5-serial>/hdf5/lib -lhdf5 -lz
HDFMPIINCFLAGS   = -I<path to hdf5-parallel>/include
HDFMPILIBFLAGS   = -L<path to hdf5-parallel>lib -lhdf5 -lz

which you can usually find by displaying the details of the hdf5 module you have installed. (If you only plan to run the code in parallel, you can set the paths to the same (parallel compiled) library for serial and parallel.)

And your compilers:

MPI              = TRUE
OPENMPCC         = TRUE
CXX              = icpc -std=c++14 -openmp -mkl=sequential
FC               = ifort -openmp -mkl=sequential
MPICXX           = icpc -std=c++14 -openmp -mkl=sequential -lmpi

Once the Make.defs.local file has been created go to Chombo/lib/ and run

make all DIM=3 -j 8

This compiles in parallel on 8 ranks. This will also compile the tests which is a good thing to do once but subsequently it is faster to only compile the library by invoking

make lib DIM=3 -j 8

One can also run the tests and make and run the examples, using grep to test they run ok:

make run
make run | grep "finished with"
cd ../releasedExamples
make all
make run | grep "finished with"

All this should compile and run without errors. See Compilation FAQ for advice on common problems. The Chombo README file also provides some hints for compilation.

If you subsequently change compiler options, use the command

make clean

or

make realclean

in the lib drectory, to start the compilation from a clean slate.