Netcdf and related Find TPL modules refactoring to improve default find #156
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@gsjaardema, that for putting this PR together. Did you get a chance to test this against your SEACAS repo? Also, is there a PR for just the changes to Trilinos (i.e. part of trilinos/Trilinos#787)? Is it just the Trilinos/TPLsList.cmake file that needs to be updated as per this version? I will get to this next week after I complete trilinos/Trilinos#482. This will be a test that backward compatibility when overriding vars is maintained. |
|
I tested the PR with a Trilinos SEACAS build. I will test against the
standalone SEACAS/github build today.
You are correct that the only Trilinos change is the TPLsList.cmake.
..Greg
…On Wed, Nov 23, 2016 at 8:33 AM, Roscoe A. Bartlett < ***@***.***> wrote:
@gsjaardema <https://github.com/gsjaardema>, that for putting this PR
together. Did you get a chance to test this against your SEACAS repo? Also,
is there a PR for just the changes to Trilinos (i.e. part of
trilinos/Trilinos#787 <trilinos/Trilinos#787>)?
Is it just the Trilinos/TPLsList.cmake file that needs to be updated as per this
version
<https://github.com/gsjaardema/Trilinos/blob/e896df98cf02fdd0d2f61029ea0d1d7d318cddd4/TPLsList.cmake>
?
I will get to this next week after I complete trilinos/Trilinos#482
<trilinos/Trilinos#482>. This will be a test
that backward compatibility when overriding vars is maintained.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#156 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA2xDp6SBa3_PGDWmlDsIzETmrfOasiiks5rBFzlgaJpZM4K5mbj>
.
|
@gsjaardema, can you please do a quick PR on the github.com/trilinos/Trilinos site for the updated Trilinos/TPLsList.cmake file and reference this issue I will get to test this soon and provide comments. However, just looking over this I am seeing that some small modifications will be needed in order to address the issues described at: I can take care of those modifications and testing. But to make sure that things are still working, can you provide some Trilinos configure options on specific SNL machines to make sure that those configure options still work? |
|
@gsjaardema, also, can you please allow me to make new commits to your PR branch? See: That will all me to make some modifications without having to swap out branches. This is a fairly new GitHub feature I think. |
|
@bartlettroscoe It looks like you can make commits to my PR branch. The "Allow edits from maintainers" is enabled on this PR. |
|
This is the script I use on my Mac for building SEACAS (and some other packages enabled) |
bartlettroscoe
changed the title
bartlettroscoe
changed the title
bartlettroscoe
changed the title
|
I am a bit anxious to know when this might be committed. Any guesses? I only ask because it's quite relevant to my current project and if it is far enough away, I may try to implement a more portable PnetCDF link for my project in the meantime. Otherwise, I would be very grateful to have this find capability. |
Hopefully not too long. I can try to move this closer to the top of my personal todo list. There are some modifications that will be needed along the lines of: and then a good bit of manual testing to make sure that existing use cases still work. One of the last attempts to improve a find module ended up having to be backed out because of a break in backward compatibility (see #89). |
|
It would be good to raise the priority on this. There have been many
people having problems with this. I'm not sure how the prioritization
works, but I had thought it was a high priority already...
What is the process for getting the priority raised?
..Greg
…On Thu, Jan 12, 2017 at 4:56 PM, Roscoe A. Bartlett < ***@***.***> wrote:
I am a bit anxious to know when this might be committed. Any guesses?
Hopefully not too long. I can try to move this closer to the top of my
personal todo list. There are some modifications that will be needed along
the lines of:
- https://tribits.org/doc/TribitsDevelopersGuide.html#
how-to-use-find-package-for-a-tribits-tpl
<https://tribits.org/doc/TribitsDevelopersGuide.html#how-to-use-find-package-for-a-tribits-tpl>
and then a good bit of manual testing to make sure that existing use cases
still work. One of the last attempts to improve a find module ended up
having to be backed out because of a break in backward compatibility (see
#89 <#89>).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#156 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA2xDnbXCRPXytfH6lVyqB4Lz9-zhyCZks5rRr3JgaJpZM4K5mbj>
.
|
|
If all goes well, I should be able to start working on this next Wednesday. I have some high priority tasks (coming directly from higher up) for an important ECP-related Sandia project (the name of which we are not allowed to even mention here) that I have to attend before that. This at the top of the "Ready" column for TriBITS. |
If there are two libraries (e.g. CGNS and NetCDF) that both have a dependency on the same library (e.g. HDF5), then the previous code would only add the hdf5 dependencies to whichever of the two libraries were processed first. The current behavior fixes this by adding the dependency to both libraries, but still only adds the target for the dependent library once.
|
I pulled the branches and tested with the standard CI build for Trilinos, and unless you have netcdf in your PATH, it can't find Netcdf where before it did it does not for these updated version. The old version of the FindTPLNetcdf.cmake module works without needing to augment PATH. Therefore, I will now augment these updated I will update the DETAILED NOTES: Also see:
The Trilinos branch is: It contains the single commit: which updates just the Trilinos/TPLsList.cmake file. Then there is the TriBITS branch: Together, these two branches should give the desired update to Trilinos TPLs for CGNS, Pnetcdf, and Netcdf. I will test this with the standard SEMS-based Trilinos CI build. These updated files should work with that build with no changes. That should show some backward compatibility. I created the local Trilinos branch: I then created the TriBITS branch: Using the do-configure script: Now I can test by bulding SEACAS under Trilinos using SEMS env: That passed the configure but it showed the warning: That is a good warning about NC_MAX_DIMS and NC_MAX_VARS, but it seems a bit specific to Exodus and SEACAS. I wonder if this type of check and warning might be better to go into the SEACAS package configure itself since it is specific to SEACAS. Netcdf can be used by other packages than SEACAS that may not care if those vars are set. Note that since it has "WARNING:", this will also result in a CDash warning being displayed on the CDash dashboard. Therefore, this should likely be changed from "WARNING:" to "NOTE:". I am not sure how the libnetcdf.so file gets found. There is no indication how this gets found. Looking in the env, I see: Is FindNetcdf.cmake finding this from LD_LIBRARY_PATH or PATH? Is it finding it from SEMS_CDF_LIBRARY_PATH? I don't like magic. What if I remove the netcdf paths from PATH and LD_LIBRARY_PATH? Will it still find it? Let's try: Now configure again: That time it failed to configure with: So it seems that FindNetCDF.cmake is looking in PATH. This breaks backward compatibility because before you could specify Netcdf_INCLUDE_DIRS, Netcdf_LIBRARY_DIRS and Netcdf_LIBRARIES and it would find the Netcdf libraries. That is why the FindTPLNetcdf.cmake module must be written according to: I went back to the original definition of the FindTPLNetcdf.cmake module and it finds Netcdf just fine: This shows that the updated FindTPLNetcdf.cmake module breaks backward compatibility and will break people's existing configures who are using the TriBITS TPL system standard vars Netcdf_INCLUDE_DIRS, Netcdf_LIBRARY_DIRS and Netcdf_LIBRARIES. |
bartlettroscoe
added a commit
to bartlettroscoe/TriBITS
that referenced
this pull request
Feb 23, 2017
Now it responds to the standard TriBITS vars Netcdf_INCLUDE_DIRS, Netcdf_LIBRARY_NAMES, or Netcdf_LIBRARY_DIRS if set correctly.
|
The current NetCDF patch uses the paths NetCDF_LIBRARIES and NetCDF_INCLUDE_DIRS instead of Netcdf_INCLUDE_DIRS and Netcdf_LIBRARIES. Perhaps the Netcdf versions should emit a deprecated warning and indicate a preference for the NetCDF versions since NetCDF is a more accepted (in the NetCDF community) way of specifying the library than "Netcdf" I think this was discussed earlier, but not sure where that discussion is archived... |
|
This is unfortunate and confusing. Whoever named the TriBITS TPL with the case "Netcdf" instead of the standard CMake find module name with case "NetCDF" did a disservice. This difference is going to confuse people to no end. But it would be hard to change the Trilinos TriBITS TPL name back to "NetCDF" but perhaps we should do it? I think we might be able to preserve backward compatiblity with a warning perhaps but it would be messy. |
bartlettroscoe
added a commit
to bartlettroscoe/TriBITS
that referenced
this pull request
Feb 24, 2017
Setting this does nothing and therefore should be removed. However, it is not clear why FindTPLPnetcdf.cmake is even part of the PR since it does not impact SEACAS in any way.
bartlettroscoe
added a commit
to bartlettroscoe/TriBITS
that referenced
this pull request
Feb 24, 2017
According to the standard defined in the TriBITS Developers Guide, the FIND_PACKAGE() command optionally finds the TPL. If that fails, then the standard TriBITS find can take a crack at it. Without this, then we are techically breaking backward compatibility.
bartlettroscoe
added a commit
to bartlettroscoe/TriBITS
that referenced
this pull request
Feb 24, 2017
…vars (TriBITSPub#156) This now matches the standard behavior defined in the TriBITS Developers Guide. The FIND_PACKAGE(CGNS) call was made optional as per the standard. But that should be okay because if that fails, then if the standard TriBITS TPL find fails, then the TPL find will bomb out.
bartlettroscoe
added a commit
to bartlettroscoe/Trilinos
that referenced
this pull request
Feb 24, 2017
…rilinos#1086) I also removed other find modules already being pulled in from TriBITS. This goes with TriBITSPub/TriBITS#156.
|
I have updated the FindTPLNetcdf.cmake, FindTPLPnetcdf.cmake and FindTPLCGNS.cmake to the branch: I am ready to have the SNL customer review this. I will comment on how to do that in: DETAILED NOTES: Now to fix the updated The files that I need to update are: Just three files to update. This should not be too bad. I just pushed the commit: to the branch: I tested this with the SEMS env and it configures correctly with: But if I turn off the special include of SEMSEnv.cmake by configuring with: you get a configure failure with: So it shows that it is using But when I restore the SEMS-set PATH var and configure with: I get: That shows that the updated FindTPLNetcdf.cmake module supports both the new (better) default find operation, but still supports the old TriBITS vars that are used by many existing clients. Now to modify the other two find modules FindTPLCGNS.cmake and FindTPLPnetcdf.cmake accordingly. It does not look like the Pnetcdf.cmake module is used by SEACAS in Trilinos as shown by: Looking at the SEACAS version that exists on Greg's seacas GitHub repo, it does not seem that it uses the Pnetcdf TPL either. What is up with that? It looks like the only package that uses the "Pnetcdf" TPL in Trilinos is Trios: But Trios is an "EX" package and never gets tested in automated testing. In any case, since the SEMS env has a Pnetcdf TPL installed, I can test this module manually by configuring with -DTPL_ENABLE_Pnetcdf=ON and testing it with and without the overrides. That is what I will do. Okay, so I looked at the updated FindTPLPnetcdf.cmake file and it does not use FIND_PACKAGE(...). The only difference with what is in Trilinos/cmake/TPLs/FindTPLPnetcdf.cmake is a copyright/license and the do nothing set: Therefore, I am just going to take that out and delete the find module from Trilinos/cmake/TPLs/. I tested the FindTPLPnetcdf.cmake module with: and this passed showing the configure output: I also deleted the duplicate Now for FindTPLCGNS.cmake. I don't have that TPL, but I can see if it tries to find it after I update it. I made the similar updates and did some manual testing (the best I could without having a CGNS lib on my system). I pushed the updated code to the branch: In addition, I pushed the corresponding Trilinos changes and cleanups to the branch: |
|
SEACAS uses PNetCDF indirectly as a dependency of NetCDF. In other words,
NetCDF can be built with PnetCDF capability built in. The FindNetCDF will
determine whether this is the case and set up the link line and include
lines accordingly. So, in other words, SEACAS itself does not have a
direct dependency, but if the capability is available through NetCDF, then
it is used by IOSS and exodus libraries.
Similarly for HDF5. There is no library or application in SEACAS that
directly uses HDF5, but SEACAS does use the HDF5 capability through either
NetCDF and/or CGNS.
…On Thu, Feb 23, 2017 at 7:06 PM, Roscoe A. Bartlett < ***@***.***> wrote:
I have updated the FindTPLNetcdf.cmake, FindTPLPnetcdf.cmake and
FindTPLCGNS.cmake to the branch:
- https://github.com/bartlettroscoe/TriBITS/tree/netcdf-cgns-hdf5-156
I am ready to have the SNL customer review this. I will comment on how to
do that in:
- https://software-srn.sandia.gov/jira/browse/SPAR-168
------------------------------
*DETAILED NOTES:*
Now to fix the updated FindTPL<TPL_NAME>.cmake files according to:
- https://tribits.org/doc/TribitsDevelopersGuide.html#
how-to-use-find-package-for-a-tribits-tpl
<https://tribits.org/doc/TribitsDevelopersGuide.html#how-to-use-find-package-for-a-tribits-tpl>
The files that I need to update are:
$ git diff --name-status HEAD ^github/master -- tribits/common_tpls/*.cmake
A tribits/common_tpls/FindTPLCGNS.cmake
A tribits/common_tpls/FindTPLNetcdf.cmake
A tribits/common_tpls/FindTPLPnetcdf.cmake
Just three files to update. This should not be too bad.
I just pushed the commit:
392f435 "Update to TriBITS standard (#156)"
Author: Roscoe A. Bartlett ***@***.***>
Date: Thu Feb 23 14:51:50 2017 -0700 (9 minutes ago)
M tribits/common_tpls/FindTPLNetcdf.cmake
M tribits/common_tpls/find_modules/FindNetCDF.cmake
to the branch:
- https://github.com/bartlettroscoe/TriBITS/tree/netcdf-cgns-hdf5-156
I tested this with the SEMS env and it configures correctly with:
Processing enabled TPL: Netcdf (enabled explicitly, disable with -DTPL_ENABLE_Netcdf=OFF)
-- Searching for libs in Netcdf_LIBRARY_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib;/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib;/projects/sems/install/rhel6-x86_64/sems/tpl/zlib/1.2.8/gcc/4.8.4/base/lib'
-- Searching for a lib in the set "netcdf":
-- Searching for lib 'netcdf' ...
-- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so'
-- Searching for a lib in the set "pnetcdf":
-- Searching for lib 'pnetcdf' ...
-- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libpnetcdf.a'
-- Searching for a lib in the set "hdf5_hl":
-- Searching for lib 'hdf5_hl' ...
-- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libhdf5_hl.a'
-- Searching for a lib in the set "hdf5":
-- Searching for lib 'hdf5' ...
-- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libhdf5.a'
-- Searching for a lib in the set "z":
-- Searching for lib 'z' ...
-- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/zlib/1.2.8/gcc/4.8.4/base/lib/libz.so'
-- TPL_Netcdf_LIBRARIES='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so;/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libpnetcdf.a;/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libhdf5_hl.a;/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libhdf5.a;/projects/sems/install/rhel6-x86_64/sems/tpl/zlib/1.2.8/gcc/4.8.4/base/lib/libz.so'
-- TPL_Netcdf_INCLUDE_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
But if I turn off the special include of SEMSEnv.cmake by configuring with:
$ time ./do-configure \
-DTrilinos_TRIBITS_DIR:STRING=TriBITS/tribits \
-DTrilinos_ENABLE_SEACAS=ON \
-DTrilinos_USE_BUILD_ENV= \
&> configure.out
real 0m6.789s
user 0m3.716s
sys 0m2.055s
you get a configure failure with:
Processing enabled TPL: Netcdf (enabled explicitly, disable with -DTPL_ENABLE_Netcdf=OFF)
-- Using FIND_PACKAGE(Netcdf ...) ...
CMake Error at TriBITS/tribits/common_tpls/find_modules/FindNetCDF.cmake:163 (message):
Can not locate NetCDF include directory
Call Stack (most recent call first):
TriBITS/tribits/common_tpls/FindTPLNetcdf.cmake:66 (find_package)
TriBITS/tribits/core/package_arch/TribitsProcessEnabledTpl.cmake:89 (INCLUDE)
TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1614 (TRIBITS_PROCESS_ENABLED_TPL)
TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:202 (TRIBITS_PROCESS_ENABLED_TPLS)
TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL)
CMakeLists.txt:93 (TRIBITS_PROJECT)
CMake Error at TriBITS/tribits/common_tpls/find_modules/FindNetCDF.cmake:274 (message):
Can not locate NetCDF C library
Call Stack (most recent call first):
TriBITS/tribits/common_tpls/FindTPLNetcdf.cmake:66 (find_package)
TriBITS/tribits/core/package_arch/TribitsProcessEnabledTpl.cmake:89 (INCLUDE)
TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1614 (TRIBITS_PROCESS_ENABLED_TPL)
TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:202 (TRIBITS_PROCESS_ENABLED_TPLS)
TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL)
CMakeLists.txt:93 (TRIBITS_PROJECT)
-- NetCDF does not require HDF5
-- NetCDF does not require PNetCDF
-- Can not locate NetCDF bin directory
CMake Error at /ascldap/users/projects/sems/install/rhel6-x86_64/sems/utility/cmake/3.5.2/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
Could NOT find NetCDF (missing: NetCDF_LIBRARIES NetCDF_INCLUDE_DIRS)
Call Stack (most recent call first):
/ascldap/users/projects/sems/install/rhel6-x86_64/sems/utility/cmake/3.5.2/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
TriBITS/tribits/common_tpls/find_modules/FindNetCDF.cmake:430 (find_package_handle_standard_args)
TriBITS/tribits/common_tpls/FindTPLNetcdf.cmake:66 (find_package)
TriBITS/tribits/core/package_arch/TribitsProcessEnabledTpl.cmake:89 (INCLUDE)
TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1614 (TRIBITS_PROCESS_ENABLED_TPL)
TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:202 (TRIBITS_PROCESS_ENABLED_TPLS)
TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL)
CMakeLists.txt:93 (TRIBITS_PROJECT)
-- Configuring incomplete, errors occurred!
So it shows that it is using FIND_PACKAGE(NetCDF ...) when the vars
Netcdf_LIBRARY_DIRS, etc. are not set.
But when I restore the SEMS-set PATH var and configure with:
$ time ./do-configure \
-DTrilinos_TRIBITS_DIR:STRING=TriBITS/tribits \
-DTrilinos_ENABLE_SEACAS=ON \
-DTrilinos_USE_BUILD_ENV= \
&> configure.out
real 0m8.770s
user 0m5.845s
sys 0m2.351s
I get:
Processing enabled TPL: Netcdf (enabled explicitly, disable with -DTPL_ENABLE_Netcdf=OFF)
-- Using FIND_PACKAGE(Netcdf ...) ...
-- NetCDF include file /projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include/netcdf.h will be searched for define values
CMake Warning at TriBITS/tribits/common_tpls/find_modules/FindNetCDF.cmake:184 (message):
WARNING: The NetCDF found in does not have the correct NC_MAX_DIMS and
NC_MAX_VARS. It may not be compatible with Exodus. See NetCDF-Mapping.md
for details
Call Stack (most recent call first):
TriBITS/tribits/common_tpls/FindTPLNetcdf.cmake:66 (find_package)
TriBITS/tribits/core/package_arch/TribitsProcessEnabledTpl.cmake:89 (INCLUDE)
TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1614 (TRIBITS_PROCESS_ENABLED_TPL)
TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:202 (TRIBITS_PROCESS_ENABLED_TPLS)
TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL)
CMakeLists.txt:93 (TRIBITS_PROJECT)
-- NetCDF does not require HDF5
-- NetCDF does not require PNetCDF
-- Found NetCDF: /projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so
-- NetCDF Version:
-- NetCDF_NEEDS_HDF5 =
-- NetCDF_NEEDS_PNetCDF =
-- NetCDF_PARALLEL = False
-- NetCDF_INCLUDE_DIRS = /projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include
-- NetCDF_LIBRARIES = /projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so
-- NetCDF_BINARIES = ncdump;ncgen;nccopy
-- TPL_Netcdf_LIBRARIES='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so'
-- TPL_Netcdf_INCLUDE_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
That shows that the updated FindTPLNetcdf.cmake module supports both the
new (better) default find operation, but still supports the old TriBITS
vars that are used by many existing clients.
Now to modify the other two find modules FindTPLCGNS.cmake and
FindTPLPnetcdf.cmake accordingly.
It does not look like the Pnetcdf.cmake module is used by SEACAS in
Trilinos as shown by:
$ cd ../packages/seacas/
$ find . -name Dependencies.cmake -exec grep -nHi pnetcdf {} \;
Looking at the SEACAS version that exists on Greg's seacas GitHub repo, it
does not seem that it uses the Pnetcdf TPL either.
What is up with that?
It looks like the only package that uses the "Pnetcdf" TPL in Trilinos is
Trios:
***@***.*** packages (netcdf-cgns-hdf5-1082)]$ find . -name Dependencies.cmake -exec grep -nHi pnetcdf {} \;
./trios/cmake/Dependencies.cmake:17:SET(LIB_OPTIONAL_DEP_TPLS BGQPAMI BGPDCMF MPI Netcdf Pnetcdf Pthread CrayPortals Portals Gemini InfiniBand Pablo PAPI HPCToolkit)
./trios/services/netcdf/cmake/Dependencies.cmake:5:SET(LIB_REQUIRED_DEP_TPLS MPI Pnetcdf Netcdf)
./trios/services/netcdf/src/cmake/Dependencies.cmake:5:SET(LIB_REQUIRED_DEP_TPLS MPI Pnetcdf Netcdf)
./trios/services/netcdf/tests/cmake/Dependencies.cmake:5:SET(LIB_REQUIRED_DEP_TPLS MPI Pnetcdf Netcdf)
But Trios is an "EX" package and never gets tested in automated testing.
In any case, since the SEMS env has a Pnetcdf TPL installed, I can test
this module manually by configuring with -DTPL_ENABLE_Pnetcdf=ON and
testing it with and without the overrides. That is what I will do.
Okay, so I looked at the updated FindTPLPnetcdf.cmake file and it does
*not* use FIND_PACKAGE(...). The only difference with what is in
Trilinos/cmake/TPLs/FindTPLPnetcdf.cmake is a copyright/license and the
do nothing set:
SET(CMAKE_MODULE_PATH
"${CMAKE_MODULE_PATH}"
"${CMAKE_CURRENT_LIST_DIR}/find_modules"
"${CMAKE_CURRENT_LIST_DIR}/utils"
)
Therefore, I am just going to take that out and delete the find module
from Trilinos/cmake/TPLs/.
I tested the FindTPLPnetcdf.cmake module with:
$ ./do-configure -DTrilinos_TRIBITS_DIR:STRING=TriBITS/tribits \
-DTPL_ENABLE_Pnetcdf=ON \
-DPnetcdf_LIBRARY_DIRS=$SEMS_NETCDF_LIBRARY_PATH \
-DPnetcdf_INCLUDE_DIRS=$SEMS_NETCDF_INCLUDE_PATH \
&> configure.out
and this passed showing the configure output:
Processing enabled TPL: Pnetcdf (enabled explicitly, disable with -DTPL_ENABLE_Pnetcdf=OFF)
-- TPL_Pnetcdf_LIBRARIES='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libpnetcdf.a'
-- Searching for headers in Pnetcdf_INCLUDE_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
-- Searching for a header file in the set "pnetcdf.h":
-- Searching for header 'pnetcdf.h' ...
-- Found header '/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include/pnetcdf.h'
-- Found TPL 'Pnetcdf' include dirs '/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
-- TPL_Pnetcdf_INCLUDE_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
I also deleted the duplicate FindTPL<TPL_NAME>.cmake modules in
Trilinos/cmake/TPLs/.
Now for FindTPLCGNS.cmake. I don't have that TPL, but I can see if it
tries to find it after I update it.
I made the similar updates and did some manual testing (the best I could
without having a CGNS lib on my system).
I pushed the updated code to the branch:
To ***@***.***:bartlettroscoe/TriBITS.git
392f435..a5f7169 netcdf-cgns-hdf5-156 -> netcdf-cgns-hdf5-156
In addition, I pushed the corresponding Trilinos changes and cleanups to
the branch:
To ***@***.***:bartlettroscoe/Trilinos
* [new branch] netcdf-cgns-hdf5-1082 -> netcdf-cgns-hdf5-1082
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#156 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA2xDnkWSHgHHOl8jlUDJ4hYKAO6DzUzks5rfjtAgaJpZM4K5mbj>
.
|
|
Other issues related to PNetCDF and CGNS:
1. The PNetCDF needs to test the NC_MAX_DIMS and NC_MAX_VARS in the same
way as the test in NetCDF.
2. The CGNS library needs to check that CG_BUILD_SCOPE is enabled.
The main dependency in SEACAS is on the CGNS and NetCDF libraries.
However, both CGNS and NetCDF *may* depend on HDF5 and that dependency
needs to be automatically added if present.
NetCDF *may* also depend on PnetCDF and if so, that dependency needs to be
automatically added if present.
Here are some of the locations in which SEACAS uses pnetcdf:
s974786:build(cgns-output)> git grep -i pnetcdf
../packages/seacas/libraries/
../packages/seacas/libraries/exodus/include/exodusII.h:#define EX_PNETCDF
0x80000
../packages/seacas/libraries/exodus/include/exodusII_int.h: unsigned int
is_pnetcdf : 1; /* 1 true, 0 false */
../packages/seacas/libraries/exodus/include/exodusII_int.h:
int int64_status, int is_parallel, int is_mpiio, int is_pnetcdf);
../packages/seacas/libraries/exodus/src/ex_conv.c: int
int64_status, int is_parallel, int is_mpiio, int is_pnetcdf)
../packages/seacas/libraries/exodus/src/ex_conv.c: new_file->is_pnetcdf
= is_pnetcdf;
../packages/seacas/libraries/exodus/src/ex_create_par.c: int
is_pnetcdf = 0;
../packages/seacas/libraries/exodus/src/ex_create_par.c: /* Library does
NOT support parallel output via netcdf-4 or pnetcdf */
../packages/seacas/libraries/exodus/src/ex_create_par.c:
"pnetcdf library format, but this netcdf library does not "
../packages/seacas/libraries/exodus/src/ex_create_par.c: /* Check parallel
io mode. Valid is NC_MPIPOSIX or NC_MPIIO or NC_PNETCDF
../packages/seacas/libraries/exodus/src/ex_create_par.c: else if
(my_mode & EX_PNETCDF) {
../packages/seacas/libraries/exodus/src/ex_create_par.c: pariomode =
NC_PNETCDF;
../packages/seacas/libraries/exodus/src/ex_create_par.c: is_pnetcdf =
1;
../packages/seacas/libraries/exodus/src/ex_create_par.c:#if !NC_HAS_PNETCDF
../packages/seacas/libraries/exodus/src/ex_create_par.c:
snprintf(errmsg, MAX_ERR_LENGTH, "EXODUS: ERROR: EX_PNETCDF parallel
output requested "
../packages/seacas/libraries/exodus/src/ex_create_par.c:
"which requires PNetCDF support, but the library does "
../packages/seacas/libraries/exodus/src/ex_create_par.c: if
(ex_conv_ini(exoid, comp_ws, io_ws, 0, int64_status, 1, is_mpiio,
is_pnetcdf) != EX_NOERR) {
../packages/seacas/libraries/exodus/src/ex_open_par.c: int is_pnetcdf
= 0;
../packages/seacas/libraries/exodus/src/ex_open_par.c: /* Check parallel
io mode. Valid is NC_MPIPOSIX or NC_MPIIO or NC_PNETCDF
../packages/seacas/libraries/exodus/src/ex_open_par.c: else if (mode &
EX_PNETCDF) {
../packages/seacas/libraries/exodus/src/ex_open_par.c: pariomode =
NC_PNETCDF;
../packages/seacas/libraries/exodus/src/ex_open_par.c: is_pnetcdf = 1;
../packages/seacas/libraries/exodus/src/ex_open_par.c:#if (NC_HAS_PARALLEL
== 0) && (NC_HAS_PNETCDF == 0)
../packages/seacas/libraries/exodus/src/ex_open_par.c: if
(ex_conv_ini(exoid, comp_ws, io_ws, file_wordsize, int64_status, 1,
is_mpiio, is_pnetcdf) !=
../packages/seacas/libraries/ioss/src/exo_par/Iopx_DatabaseIO.C: static
int par_mode_default = EX_PNETCDF; // Default...
../packages/seacas/libraries/ioss/src/exo_par/Iopx_DatabaseIO.C: if
(mode_name == "pnetcdf") {
../packages/seacas/libraries/ioss/src/exo_par/Iopx_DatabaseIO.C:
par_mode = EX_PNETCDF;
../packages/seacas/libraries/ioss/src/exo_par/Iopx_DatabaseIO.C:
<< "' in the Ioss property PARALLEL_IO_MODE. Valid values are
'pnetcdf', "
../packages/seacas/libraries/ioss/src/exo_par/Iopx_DatabaseIO.C: if
((mode & EX_ALL_INT64_DB) && par_mode == EX_PNETCDF) {
../packages/seacas/libraries/ioss/src/main/shell_interface.C: "Options
are default, mpiio, mpiposix, pnetcdf",
../packages/seacas/libraries/ioss/src/main/skinner_interface.C:
"Options are default, mpiio, mpiposix, pnetcdf",
Note that there is never an "include <pnetcdf.h>" since that is handled
down at the NetCDF level and is accessed via the 'dispatch' mechanism
internal to NetCDF.
Historically, the PnetCDF was used primarily a a parallel output option,
but that is changing and it is possible to use PnetCDF in serial.
Probably more detail than you wanted, but just wanted to reinforce that
PnetCDF is in fact used by SEACAS libraries if it has been compiled into
the NetCDF library. I did have a request in to the SEMS developers to
discuss versions and build options of NetCDF, but I need to revisit this
since I think it has fallen through the cracks.
On Thu, Feb 23, 2017 at 7:54 PM, Gregory Sjaardema <[email protected]>
wrote:
… SEACAS uses PNetCDF indirectly as a dependency of NetCDF. In other words,
NetCDF can be built with PnetCDF capability built in. The FindNetCDF will
determine whether this is the case and set up the link line and include
lines accordingly. So, in other words, SEACAS itself does not have a
direct dependency, but if the capability is available through NetCDF, then
it is used by IOSS and exodus libraries.
Similarly for HDF5. There is no library or application in SEACAS that
directly uses HDF5, but SEACAS does use the HDF5 capability through either
NetCDF and/or CGNS.
On Thu, Feb 23, 2017 at 7:06 PM, Roscoe A. Bartlett <
***@***.***> wrote:
> I have updated the FindTPLNetcdf.cmake, FindTPLPnetcdf.cmake and
> FindTPLCGNS.cmake to the branch:
>
> - https://github.com/bartlettroscoe/TriBITS/tree/netcdf-cgns-hdf5-156
>
> I am ready to have the SNL customer review this. I will comment on how to
> do that in:
>
> - https://software-srn.sandia.gov/jira/browse/SPAR-168
>
> ------------------------------
>
> *DETAILED NOTES:*
>
> Now to fix the updated FindTPL<TPL_NAME>.cmake files according to:
>
> - https://tribits.org/doc/TribitsDevelopersGuide.html#how-to-
> use-find-package-for-a-tribits-tpl
> <https://tribits.org/doc/TribitsDevelopersGuide.html#how-to-use-find-package-for-a-tribits-tpl>
>
> The files that I need to update are:
>
> $ git diff --name-status HEAD ^github/master -- tribits/common_tpls/*.cmake
> A tribits/common_tpls/FindTPLCGNS.cmake
> A tribits/common_tpls/FindTPLNetcdf.cmake
> A tribits/common_tpls/FindTPLPnetcdf.cmake
>
> Just three files to update. This should not be too bad.
>
> I just pushed the commit:
>
> 392f435 "Update to TriBITS standard (#156)"
> Author: Roscoe A. Bartlett ***@***.***>
> Date: Thu Feb 23 14:51:50 2017 -0700 (9 minutes ago)
>
> M tribits/common_tpls/FindTPLNetcdf.cmake
> M tribits/common_tpls/find_modules/FindNetCDF.cmake
>
> to the branch:
>
> - https://github.com/bartlettroscoe/TriBITS/tree/netcdf-cgns-hdf5-156
>
> I tested this with the SEMS env and it configures correctly with:
>
> Processing enabled TPL: Netcdf (enabled explicitly, disable with -DTPL_ENABLE_Netcdf=OFF)
> -- Searching for libs in Netcdf_LIBRARY_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib;/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib;/projects/sems/install/rhel6-x86_64/sems/tpl/zlib/1.2.8/gcc/4.8.4/base/lib'
> -- Searching for a lib in the set "netcdf":
> -- Searching for lib 'netcdf' ...
> -- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so'
> -- Searching for a lib in the set "pnetcdf":
> -- Searching for lib 'pnetcdf' ...
> -- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libpnetcdf.a'
> -- Searching for a lib in the set "hdf5_hl":
> -- Searching for lib 'hdf5_hl' ...
> -- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libhdf5_hl.a'
> -- Searching for a lib in the set "hdf5":
> -- Searching for lib 'hdf5' ...
> -- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libhdf5.a'
> -- Searching for a lib in the set "z":
> -- Searching for lib 'z' ...
> -- Found lib '/projects/sems/install/rhel6-x86_64/sems/tpl/zlib/1.2.8/gcc/4.8.4/base/lib/libz.so'
> -- TPL_Netcdf_LIBRARIES='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so;/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libpnetcdf.a;/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libhdf5_hl.a;/projects/sems/install/rhel6-x86_64/sems/tpl/hdf5/1.8.12/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libhdf5.a;/projects/sems/install/rhel6-x86_64/sems/tpl/zlib/1.2.8/gcc/4.8.4/base/lib/libz.so'
> -- TPL_Netcdf_INCLUDE_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
>
> But if I turn off the special include of SEMSEnv.cmake by configuring
> with:
>
> $ time ./do-configure \
> -DTrilinos_TRIBITS_DIR:STRING=TriBITS/tribits \
> -DTrilinos_ENABLE_SEACAS=ON \
> -DTrilinos_USE_BUILD_ENV= \
> &> configure.out
>
> real 0m6.789s
> user 0m3.716s
> sys 0m2.055s
>
> you get a configure failure with:
>
> Processing enabled TPL: Netcdf (enabled explicitly, disable with -DTPL_ENABLE_Netcdf=OFF)
> -- Using FIND_PACKAGE(Netcdf ...) ...
> CMake Error at TriBITS/tribits/common_tpls/find_modules/FindNetCDF.cmake:163 (message):
> Can not locate NetCDF include directory
> Call Stack (most recent call first):
> TriBITS/tribits/common_tpls/FindTPLNetcdf.cmake:66 (find_package)
> TriBITS/tribits/core/package_arch/TribitsProcessEnabledTpl.cmake:89 (INCLUDE)
> TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1614 (TRIBITS_PROCESS_ENABLED_TPL)
> TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:202 (TRIBITS_PROCESS_ENABLED_TPLS)
> TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL)
> CMakeLists.txt:93 (TRIBITS_PROJECT)
>
>
> CMake Error at TriBITS/tribits/common_tpls/find_modules/FindNetCDF.cmake:274 (message):
> Can not locate NetCDF C library
> Call Stack (most recent call first):
> TriBITS/tribits/common_tpls/FindTPLNetcdf.cmake:66 (find_package)
> TriBITS/tribits/core/package_arch/TribitsProcessEnabledTpl.cmake:89 (INCLUDE)
> TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1614 (TRIBITS_PROCESS_ENABLED_TPL)
> TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:202 (TRIBITS_PROCESS_ENABLED_TPLS)
> TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL)
> CMakeLists.txt:93 (TRIBITS_PROJECT)
>
>
> -- NetCDF does not require HDF5
> -- NetCDF does not require PNetCDF
> -- Can not locate NetCDF bin directory
> CMake Error at /ascldap/users/projects/sems/install/rhel6-x86_64/sems/utility/cmake/3.5.2/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
> Could NOT find NetCDF (missing: NetCDF_LIBRARIES NetCDF_INCLUDE_DIRS)
> Call Stack (most recent call first):
> /ascldap/users/projects/sems/install/rhel6-x86_64/sems/utility/cmake/3.5.2/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
> TriBITS/tribits/common_tpls/find_modules/FindNetCDF.cmake:430 (find_package_handle_standard_args)
> TriBITS/tribits/common_tpls/FindTPLNetcdf.cmake:66 (find_package)
> TriBITS/tribits/core/package_arch/TribitsProcessEnabledTpl.cmake:89 (INCLUDE)
> TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1614 (TRIBITS_PROCESS_ENABLED_TPL)
> TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:202 (TRIBITS_PROCESS_ENABLED_TPLS)
> TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL)
> CMakeLists.txt:93 (TRIBITS_PROJECT)
>
>
> -- Configuring incomplete, errors occurred!
>
> So it shows that it is using FIND_PACKAGE(NetCDF ...) when the vars
> Netcdf_LIBRARY_DIRS, etc. are not set.
>
> But when I restore the SEMS-set PATH var and configure with:
>
> $ time ./do-configure \
> -DTrilinos_TRIBITS_DIR:STRING=TriBITS/tribits \
> -DTrilinos_ENABLE_SEACAS=ON \
> -DTrilinos_USE_BUILD_ENV= \
> &> configure.out
>
> real 0m8.770s
> user 0m5.845s
> sys 0m2.351s
>
> I get:
>
> Processing enabled TPL: Netcdf (enabled explicitly, disable with -DTPL_ENABLE_Netcdf=OFF)
> -- Using FIND_PACKAGE(Netcdf ...) ...
> -- NetCDF include file /projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include/netcdf.h will be searched for define values
> CMake Warning at TriBITS/tribits/common_tpls/find_modules/FindNetCDF.cmake:184 (message):
> WARNING: The NetCDF found in does not have the correct NC_MAX_DIMS and
> NC_MAX_VARS. It may not be compatible with Exodus. See NetCDF-Mapping.md
> for details
>
> Call Stack (most recent call first):
> TriBITS/tribits/common_tpls/FindTPLNetcdf.cmake:66 (find_package)
> TriBITS/tribits/core/package_arch/TribitsProcessEnabledTpl.cmake:89 (INCLUDE)
> TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1614 (TRIBITS_PROCESS_ENABLED_TPL)
> TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:202 (TRIBITS_PROCESS_ENABLED_TPLS)
> TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL)
> CMakeLists.txt:93 (TRIBITS_PROJECT)
>
>
> -- NetCDF does not require HDF5
> -- NetCDF does not require PNetCDF
> -- Found NetCDF: /projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so
> -- NetCDF Version:
> -- NetCDF_NEEDS_HDF5 =
> -- NetCDF_NEEDS_PNetCDF =
> -- NetCDF_PARALLEL = False
> -- NetCDF_INCLUDE_DIRS = /projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include
> -- NetCDF_LIBRARIES = /projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so
> -- NetCDF_BINARIES = ncdump;ncgen;nccopy
> -- TPL_Netcdf_LIBRARIES='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libnetcdf.so'
> -- TPL_Netcdf_INCLUDE_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
>
> That shows that the updated FindTPLNetcdf.cmake module supports both the
> new (better) default find operation, but still supports the old TriBITS
> vars that are used by many existing clients.
>
> Now to modify the other two find modules FindTPLCGNS.cmake and
> FindTPLPnetcdf.cmake accordingly.
>
> It does not look like the Pnetcdf.cmake module is used by SEACAS in
> Trilinos as shown by:
>
> $ cd ../packages/seacas/
> $ find . -name Dependencies.cmake -exec grep -nHi pnetcdf {} \;
>
> Looking at the SEACAS version that exists on Greg's seacas GitHub repo,
> it does not seem that it uses the Pnetcdf TPL either.
>
> What is up with that?
>
> It looks like the only package that uses the "Pnetcdf" TPL in Trilinos is
> Trios:
>
> ***@***.*** packages (netcdf-cgns-hdf5-1082)]$ find . -name Dependencies.cmake -exec grep -nHi pnetcdf {} \;
> ./trios/cmake/Dependencies.cmake:17:SET(LIB_OPTIONAL_DEP_TPLS BGQPAMI BGPDCMF MPI Netcdf Pnetcdf Pthread CrayPortals Portals Gemini InfiniBand Pablo PAPI HPCToolkit)
> ./trios/services/netcdf/cmake/Dependencies.cmake:5:SET(LIB_REQUIRED_DEP_TPLS MPI Pnetcdf Netcdf)
> ./trios/services/netcdf/src/cmake/Dependencies.cmake:5:SET(LIB_REQUIRED_DEP_TPLS MPI Pnetcdf Netcdf)
> ./trios/services/netcdf/tests/cmake/Dependencies.cmake:5:SET(LIB_REQUIRED_DEP_TPLS MPI Pnetcdf Netcdf)
>
> But Trios is an "EX" package and never gets tested in automated testing.
>
> In any case, since the SEMS env has a Pnetcdf TPL installed, I can test
> this module manually by configuring with -DTPL_ENABLE_Pnetcdf=ON and
> testing it with and without the overrides. That is what I will do.
>
> Okay, so I looked at the updated FindTPLPnetcdf.cmake file and it does
> *not* use FIND_PACKAGE(...). The only difference with what is in
> Trilinos/cmake/TPLs/FindTPLPnetcdf.cmake is a copyright/license and the
> do nothing set:
>
> SET(CMAKE_MODULE_PATH
> "${CMAKE_MODULE_PATH}"
> "${CMAKE_CURRENT_LIST_DIR}/find_modules"
> "${CMAKE_CURRENT_LIST_DIR}/utils"
> )
>
> Therefore, I am just going to take that out and delete the find module
> from Trilinos/cmake/TPLs/.
>
> I tested the FindTPLPnetcdf.cmake module with:
>
> $ ./do-configure -DTrilinos_TRIBITS_DIR:STRING=TriBITS/tribits \
> -DTPL_ENABLE_Pnetcdf=ON \
> -DPnetcdf_LIBRARY_DIRS=$SEMS_NETCDF_LIBRARY_PATH \
> -DPnetcdf_INCLUDE_DIRS=$SEMS_NETCDF_INCLUDE_PATH \
> &> configure.out
>
> and this passed showing the configure output:
>
> Processing enabled TPL: Pnetcdf (enabled explicitly, disable with -DTPL_ENABLE_Pnetcdf=OFF)
> -- TPL_Pnetcdf_LIBRARIES='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/lib/libpnetcdf.a'
> -- Searching for headers in Pnetcdf_INCLUDE_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
> -- Searching for a header file in the set "pnetcdf.h":
> -- Searching for header 'pnetcdf.h' ...
> -- Found header '/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include/pnetcdf.h'
> -- Found TPL 'Pnetcdf' include dirs '/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
> -- TPL_Pnetcdf_INCLUDE_DIRS='/projects/sems/install/rhel6-x86_64/sems/tpl/netcdf/4.3.2/gcc/4.8.4/openmpi/1.8.7/parallel/include'
>
> I also deleted the duplicate FindTPL<TPL_NAME>.cmake modules in
> Trilinos/cmake/TPLs/.
>
> Now for FindTPLCGNS.cmake. I don't have that TPL, but I can see if it
> tries to find it after I update it.
>
> I made the similar updates and did some manual testing (the best I could
> without having a CGNS lib on my system).
>
> I pushed the updated code to the branch:
>
> To ***@***.***:bartlettroscoe/TriBITS.git
> 392f435..a5f7169 netcdf-cgns-hdf5-156 -> netcdf-cgns-hdf5-156
>
> In addition, I pushed the corresponding Trilinos changes and cleanups to
> the branch:
>
> To ***@***.***:bartlettroscoe/Trilinos
> * [new branch] netcdf-cgns-hdf5-1082 -> netcdf-cgns-hdf5-1082
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#156 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AA2xDnkWSHgHHOl8jlUDJ4hYKAO6DzUzks5rfjtAgaJpZM4K5mbj>
> .
>
|
|
Thanks for the detail. What that means is that the pnetcdf library is lumped together with the netcdf library into the same TriBITS TPL "Netcdf" (which is found using FindNetCDF.cmake). That means that we can eliminate the FindTPLPnetcdf.cmake TPL from this PR. |
|
NO! They are separate libraries; it is just that NetCDF can depend on
PnetCDF during certain configurations. Similar to HDF5 being a dependence
of NetCDF and CGNS.
…On Fri, Feb 24, 2017 at 8:21 AM, Roscoe A. Bartlett < ***@***.***> wrote:
@gsjaardema <https://github.com/gsjaardema>,
Thanks for the detail. What that means is that the pnetcdf library is
lumped together with the netcdf library into the same TriBITS TPL "Netcdf"
(which is found using FindNetCDF.cmake). That means that we can eliminate
the FindTPLPnetcdf.cmake TPL from this PR.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#156 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA2xDkWORmF163gP9ipBf5dwxivk-rPgks5rfvV-gaJpZM4K5mbj>
.
|
@gsjaardema, I think there is some confusion here between a "TriBITS TPL" and a "library". I will call you to clear this up. |
|
I talked with @gsjaardema. He is going to look at the file FindTPLPnetcdf.cmake to call @gsjaardema agreed to review any suggested changes to any of these find modules related to HDF5, CGNS, PNetcdf, and Netcdf since I don't have a way to evaluate if they will break all use cases (but I can test the use cases that I know about and know how to reproduce). We also discussed the issue of documentation for these new customized find modules. After committing these to TriBITS and Trilinos, the default find will use the new FindHDF5.cmake, FindNetCDF.cmake, etc. modules that take a number of different variables like HDF5_ROOT, NetCDF_ROOT that users of Trilinos would have no knowledge about. They are not even defined as CMake cache vars so the user never know about them unless they looked at the custom FindHDF5.cmake and FindNetCDF.cmake files themselves. Therefore, as a follow-on to this, we will need to work out how to manage the documentation for these different custom Otherwise, after some customer-specific testing, I am ready to push this. |
|
@gsjaardema, how to you tell this new set of find modules where to find libz? I can't seem to tell it to find the version that I want. Instead, it is finding the default version as shown by: That is not the libz that I want to find. For the TriBITS "Zlib" TPL, you just point to it with Zlib_INCLUDE_DIRS and Zlib_LIBRARY_DIRS and it finds it as: Even FindHDF5.cmake is finding the right libz: What is it about this custom FindNetCDF.cmake file that is finding the wrong zlib? How can I tell it what zlib to find? |
|
The FindNetCDF.cmake should not be doing anything with libz. It only detects that it has a dependency on HDF5 and relies on the FindHDF5.cmake to take car of adding the libhdf5_hl and libhdf5 and any libraries that these depend on (libz and potentially libdylib). I think that the hdf5 configuration has the pointer to the libz that was used or found when it was built. |
It does not seem to. If you look above, you can see that FIND_PACKAGE(HDF5) found the right libz.* file when called from the FindTPLHDF.cmake module. The output from FIND_PACKAGE(NetCDF) does not give any indication that I can see how it is finding libz.* but it is the wrong file. This is not going to break any of my current use cases for Trilinos or CASL VERA because we just use the old TriBITS system and set these explicitly and will not use the default FIND_PACKAGE() calls. But let's see if the SNL customer has a problem with this or not. If not, then no reason to worry about this too much. SideNote: This is why it would be better to upgrade TriBITS so that TPLs can property depend on each other and then |
|
Thanks for working on this. In my initial testing of this in Trilinos is that it is not working how I expect. I am requiring linking of PNetCDF in the case that I am using SEACAS for my project, indirectly depending on PNetCDF as described. I am also using Spack, so I am trying to generalize the Trilinos package file in this manner: However, something regarding NetCDF or PNetCDF isn't being included on the link line and my build fails with obvious missing symbols related to them. Previously I would inject the PNetCDF libraries in a less portable fashion into the NetCDF link line with: Before I investigate this further, I just wanted to make sure that the CMake defines in my first code block are what you would expect I should be doing now that this has been merged, i.e. do I appear to be doing anything obviously wrong there? |
|
@gsjaardema, can respond in more detail but I think what you want is just: |
|
@bartlettroscoe The two lines you show should be correct. You should also add a similar line for |
|
I just tested this using your recommendation (I used _ROOT for all my TPLs) and it worked! This is great! Thanks again for doing this. Helps make the dependency on PNetCDF much more portable, and I can push this functionality into Spack. |
krcb
pushed a commit
to Tech-XCorp/Trilinos
that referenced
this pull request
Mar 29, 2017
…rilinos#1086) I also removed other find modules already being pulled in from TriBITS. This goes with TriBITSPub/TriBITS#156.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes to TPL configuration modules proposed in trilinos/Trilinos#787