Update mpich2 to use the cross compilation toolchain.

[rainwoodman]

This probably depends on
ContinuumIO/anaconda-issues#6696

After manually removed -fopenmp from the flags it compiled and I checked

mpi{cc, cxx, c++, f77, f90} --version

gave the correct compilers for cross compilation.

[rainwoodman]

The openmpi package may need an update as well, though I currently do not have a mac to test it out.

[mingwandroid]

You do not need to identify libgcc-ng as a requirements/build.

[rainwoodman]

Currently if I do not do this I see an error about not being able to find -lquadmath. conda-build doesn't automatically install libgcc-ng. I've just added a comment about this in the yaml file.

[rainwoodman]

It does automatically install libstdc++-ng and libgfortran-ng though.

[mingwandroid]

We are now forking all of our recipes from conda-forge. Is mipch 3.2 compatible with mipch 2?

If so I can fork https://github.com/conda-forge/mpich-feedstock so you can make a PR there. For -fopenmp it is tricky. conda needs to learn to run activate scripts in dependency so that we can control concatenation of FFLAGS.

Until then, this may be necessary at the top of build.sh:

re='(.*[[:space:]])\-fopenmp\=[^[:space:]]*(.*)'
if [[ "${FFLAGS}" =~ $re ]]; then
  export FFLAGS="${BASH_REMATCH[1]}${BASH_REMATCH[2]}"
fi

[rainwoodman]

I just thought FFLAGS shall not have -fopenmp by default at all If the package builder wants -fopenmp it is always easier to add than to remove -- isn't it?

Here is why libgcc-ng is needed (looked like a typo in the package spec of gcc_impl_linux-64):

gcc_impl_linux-64 7.2.0 hc5ce805_2
----------------------------------
file name   : gcc_impl_linux-64-7.2.0-hc5ce805_2.tar.bz2
name        : gcc_impl_linux-64
version     : 7.2.0
build string: hc5ce805_2
build number: 2
channel     : defaults
size        : 70.9 MB
arch        : None
license     : GPL
md5         : 757bd7369cdc8949aa68d9adc338c771
noarch      : None
platform    : None
subdir      : linux-64
timestamp   : 1507262906442
url         : https://repo.continuum.io/pkgs/main/linux-64/gcc_impl_linux-64-7.2.0-hc5ce805_2.tar.bz2
dependencies:
    binutils_impl_linux-64 2.28.1 h04c84fa_2
    libstdcxx-ng >=7.2.0

[rainwoodman]

About the mpich version. I suppose it is compatible, but I would strongly recommend be conservative and stick with the old version till the dust settles.

When the compiler toolchain has changed it is already creating a lot of headache for downstream, and updating the mpich2 package will add more uncertainty to it. Might consult @dalcinl , the author of mpi4py if there is an urgent need to update mpich2 at this point.

[mingwandroid]

I just thought FFLAGS shall not have -fopenmp by default at all If the package builder wants -fopenmp it is always easier to add than to remove -- isn't it?

Perhaps. I am undecided about this one. We do add a lot of flags by default (mostly around performance and security/hardening) but I am also wary of adding flags that can cause specific problems for a large amount of builds.

[mingwandroid]

When the compiler toolchain has changed it is already creating a lot of headache for downstream, and updating the mpich2 package will add more uncertainty to it. Might consult @dalcinl , the author of mpi4py if there is an urgent need to update mpich2 at this poin

We can probably build packages for version 2 and 3 if necessary? Let me know.

[rainwoodman]

What was the argument for not enabling it for gcc and g++?

-fopenmp silently changes the programming model of a program from single-thread to multi-thread. That's my main argument for being extra wary of it. There is implication hybriding openmp with multiprocessing, joblib, and MPI, which typically requires some careful thinking by the user.

I think therefore it is safer to leave it for the very specific packager to decide to enable openmp or not. Actually, it may be better to provide a way for packager to spit two versions of the package, openmp and non-openmp -- the openmp package is intended for a single process application model, and the non-openmp package is intended for a multi process application model.

[dalcinl]

@rainwoodman Yes, mpich2 should be kill for good, and move to the new MPICH 3.2. MPICH2 is legacy, totally unsupported code. There are no excuses to keep it around. Downstream headache is certainly not a excuse, we are talking about legacy and unsuported code. The MPI API is forward compatible, so all builds using "mpicc/mpicxx/mpif90" should succeed. Things that my fail are test scripts that used the mpdboot command from MPICH2 to lauch daemons required by mpiexec. In the new MPICH, you just mpiexec things, it does not require previous commands.

[dalcinl]

@mingwandroid @rainwoodman If your Fortran code is purely single-threaded and does not use OpenMP at all but the compiled code will end up in a library, and the library routines might be called from client code in OpenMP parallel sections, then yes, the safest thing is to compile your Fortran code with -fopenmp, otherwise bad things happens. Fortran is a language with basically no memory model. Without the -fopenmp, the Fortran compiler is allowed to perform optimizations that are not thread-safe (like allocating local array variables in non-stack memory, kind of adding the "save" attribute behind the scenes).

[mingwandroid]

What was the argument for not enabling it for gcc and g++?

-fopenmp is not a 'core' part of C or C++, it is user elective and hardly any software uses it. Fortran is very different here (thanks for the confirmation @dalcinl).

[rainwoodman]

Now I see why. My early failure was related to the fact that libgcc_ng was not pulled in by compiler('c'). Indeed adding back -fopenmp does not fail the build.

So the plan is to update to mpich3 then?

[mingwandroid]

@dalcinl, would -frecursive work here instead of -fopenmp? Would you still recommend -fopenmp regardless?

[dalcinl]

@mingwandroid Mmm.. -frecursive would help with the issue I described above. However, in general, it may not be enough. Imagine a Fortran compiler using some special memory allocator to implement allocate which is not thread-safe.

BTW, these issues also apply to C. If you don't compile your code with -pthread, you may get bad behavior from libc (like errno() not returning a thread-local value).

Folks, we have been living in a multicore world for quite a bit. If you are responsible of building a binary package, I guess your intention is for it to be usable in the most variable scenarios. For that reason you don't compile your code with fancy AVX instructions, as some (not so) old processors may not support it. Then you are effectively trading maximum performance for usability on varieties of environments. Then, regarding multi-threading, it is the same, you have to trade performance for generality. Make your binary package work in the most scenarios you can. That means, in my extremely limited knowledge, that you should compile C code with at least -pthread (and maybe even -fopenmp if available and it does not break anything else) and Fortran code with -fopenmp. And if you are building static libraries, manage to compile the code with -fPIC, otherwise they are unusable to build a third-party shared library.

[rainwoodman]

@mingwandroid Could you go ahead with a fork of mpich and openmpi -- and make sure they are built with the new cross compilation chain? I can help with some of the testing. If you will do that, we can also close this PR.

We are in the process of updating our packages to use the cross compliation chain and currently blocked because mpicc from current mpich and openmpi do not refer to the correct compiler.

[mingwandroid]

I forked them. Please make PRs to update them to use conda-build 3's new syntax for specifying compilers. I do not have any time to do this myself.

https://github.com/AnacondaRecipes/openmpi-feedstock
https://github.com/AnacondaRecipes/mpich-feedstock

[rainwoodman]

What do you say to have both mpi backends in one same repo?

[mingwandroid]

I do not know enough about it. Convince the conda-forge maintainers and I'll follow their lead (probably - if I learn if good reasons not to later then I may have reservations!)

[rainwoodman]

I'll keep them separate then. It is inconvenient in practice but there is structural simplicity to stick with one package per repo in a larger scope effort -- no need to worry about where to draw the line.