Netcdf and HDF5 ordering

[ambrad]

configure.txt
I'm creating an issue to track the problem of netcdf and hdf5 lib ordering.

For the record, let's remember that order matters only for .a libs. If we have at least libnetcdf.so, then order doesn't matter.

In what follows I summarize what seem to be the key points. I specify SEACAS, Netcdf, and HDF5 as follows:

-D Trilinos_ENABLE_SEACAS:BOOL=ON
-D TPL_ENABLE_X11:BOOL=OFF
-D TPL_ENABLE_Matio:BOOL=OFF
-D Trilinos_ENABLE_SEACASIoss:BOOL=ON
-D Trilinos_ENABLE_SEACASExodus:BOOL=ON
-D TPL_ENABLE_Netcdf:BOOL=ON
-D Netcdf_INCLUDE_DIRS:PATH="$NETCDFDIR/include"
-D Netcdf_LIBRARY_DIRS:PATH="$NETCDFDIR/lib"
-D TPL_ENABLE_HDF5:BOOL=ON
-D HDF5_INCLUDE_DIRS:PATH="$HDF5DIR/include"
-D HDF5_LIBRARY_DIRS:PATH="$HDF5DIR/lib"
-D Trilinos_DUMP_LINK_LIBS=ON \

The DUMP_LINK_LIBS flag shows that exodus has the right order:
-- exodus:LINK_LIBS='/usr/local/parallel/lib/libnetcdf.so;/usr/local/parallel/lib/libhdf5.a;/usr/local/lib/libz.so'
and indeed the corresponding link.txt file has the right lib ordering. But, for example, nem_slice does not necessarily. Its dump output is
-- nem_slice:LINK_LIBS='suplib_cpp;suplib;chaco;exodus;zoltan'
which does not include the TPLs on which exodus depends. The corresponding link.txt has HDF5 and Netcdf in the wrong order:
... /usr/local/parallel/lib/libhdf5.a /usr/local/lib/libz.so /usr/local/parallel/lib/libnetcdf.so ...
(Note that libnetcdf is shared, so this particular build goes through without a problem.)

Historically, in the case that the libs are all .a, we have hardcoded lib ordering like this:

-DTPL_ENABLE_Netcdf:STRING=ON \
-DNetcdf_INCLUDE_DIRS:PATH="${NETCDF_ROOT}/include" \
-DNetcdf_LIBRARY_DIRS:PATH="${NETCDF_ROOT}/lib" \
-DTPL_Netcdf_LIBRARIES="${NETCDF_ROOT}/lib/libnetcdf.a;${HDF5_ROOT}/lib/libhdf5_hl.a;${HDF5_ROOT}/lib/libhdf5.a;${ZLIB_ROOT}/lib/libz.a" \
-DTPL_ENABLE_HDF5:STRING=ON \
-DHDF5_INCLUDE_DIRS:PATH="${HDF5_ROOT}/include" \
-DTPL_HDF5_LIBRARIES:PATH="${HDF5_ROOT}/lib/libhdf5_hl.a;${HDF5_ROOT}/lib/libhdf5.a;${ZLIB_ROOT}/lib/libz.a" \

This has assured that all link.txt files are correct. A fix for this issue would permit the simple specification without providing full-path libs explicitly.

Attached is a text file containing the cmake script, configuration output, and selected cats of link.txt files.

[gsjaardema]

I think that part of the issue is that currently, CMake in Trilinos does not know that netcdf requires/depends on hdf5, so it does not order the libraries in dependency order. In the standalone seacas CMake build, the FindNetCDF.cmake queries the netcdf library and determines whether it was build with netcdf-4 enabled and if so, it adds hdf5 as a dependency (and calls FindHDF5.cmake).

It also determines whether NetCDF depends on PNetCDF and adds that as a dependency.

I am planning on adding this logic to the Trilinos build of SEACAS also, but haven't had a chance yet; although the priority seems to be rising...

[gsjaardema]

Just for clarification (based on some grepping of Trilinos) the PNetCDF mentioned above is not NetCDF compiled with parallel-enabled, it is a separate library from Argonne (https://trac.mcs.anl.gov/projects/parallel-netcdf) which can be used standalone, or with NetCDF to provide parallel-io capabilities for the classic NetCDF formats and HDF5 can provide parallel-io capabilities for netcdf-4 formats. There seem to be some places in the Trilinos repository that refer to pnetcdf when netcdf is compiled with parallel-enabled and that is confusing.

[nmhamster]

Greg, we seem to require both (NetCDF w/ HDF5 and NetCDF with pNetCDF) for some of the cases we are seeing. Is that meant to be the case?

[gsjaardema]

yes, definitely.

[gsjaardema]

Here is a strawman for the NetCDF config. It will definitely need some modifications and changes, but just providing something that kind of works for configuring NetCDF and detecting whether it requires HDF5 and/or PNetCDF.
0001-Strawman-for-NetCDF-smart-config.patch.txt
netcdf.txt

The netcdf.txt file shows the portion of the CMake configure output dealing with NetCDF. Note that it detemrined that the netcdf.h include file had not been correctly modified (warning now could be error) and that NetCDF required HDF5 -- it then found HDF5. It then detected that PNetCDF was also required and found it and added them both as dependencies.

[bathmatt]

In addition it would be good if HDF5 determined if it needed -ldl and
include it automatically

On Fri, Feb 26, 2016 at 1:56 PM, Greg Sjaardema [email protected]
wrote:

Here is a strawman for the NetCDF config. It will definitely need some
modifications and changes, but just providing something that kind of works
for configuring NetCDF and detecting whether it requires HDF5 and/or
PNetCDF.
0001-Strawman-for-NetCDF-smart-config.patch.txt
https://github.com/trilinos/Trilinos/files/148903/0001-Strawman-for-NetCDF-smart-config.patch.txt
netcdf.txt https://github.com/trilinos/Trilinos/files/148902/netcdf.txt

The netcdf.txt file shows the portion of the CMake configure output
dealing with NetCDF. Note that it detemrined that the netcdf.h include file
had not been correctly modified (warning now could be error) and that
NetCDF required HDF5 -- it then found HDF5. It then detected that PNetCDF
was also required and found it and added them both as dependencies.

—
Reply to this email directly or view it on GitHub
#156 (comment).

[gsjaardema]

The one in the "strawman" patch above does do the -ldl.

[bartlettroscoe]

I suspect the issue might be related to the fact that TriBITS TPLs don't know about their dependencies. That will be fixed when I TriBITSPub/TriBITS#63. For the cases where this has been a problem on a machine, I just added the libs the the downstream TPL. But this is hack. But with static libraries, getting this right on all platforms is tricky. The updated TPL system should be able to handle that (but will need some help to tell TriBITS how the TPLs depend on each other on a given system).

[bartlettroscoe]

@ambrad, specifically, what machines are you using? What dev and TPL env are you using? We can resolve issues like this using the approach in #172. I think we can address most of the common systems to allow for automatic configuration.

Trying to make all of this automatic with static libraries on all platforms is going to be a very difficult and frustrating endeavor. For shared libraries (which most developers should be using on every platform), these these problems don't exist.

[bartlettroscoe]

If I work TriBITSPub/TriBITS#115, then I suspect that this will fix this (and many other) TPL ordering problems.

[bartlettroscoe]

This is resolved with TriBITSPub/TriBITS#156 that was snapshotted to Trilinos some time ago.