Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

parmetis: use github/KarypisLab/ParMETIS as source #329102

Merged
merged 2 commits into from
Aug 21, 2024
Merged

parmetis: use github/KarypisLab/ParMETIS as source #329102

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
db2396c
parmetis: Move to pkgs/by-name/pa/parmetis
cheriimoya Jul 22, 2024
a468bfc
parmetis: Use KarypisLab GitHub mirror instead of glaros source
cheriimoya Aug 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions pkgs/by-name/pa/parmetis/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
lib,
stdenv,
fetchFromGitHub,
cmake,
metis,
mpi,
}:

stdenv.mkDerivation {
pname = "parmetis";
version = "4.0.3";

src = fetchFromGitHub {
owner = "KarypisLab";
repo = "ParMETIS";
rev = "d90a2a6cf08d1d35422e060daa28718376213659";
hash = "sha256-22YQxwC0phdMLX660wokRgmAif/9tRbUmQWwNMZ//7M=";
};

nativeBuildInputs = [ cmake ];
enableParallelBuilding = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
enableParallelBuilding = true;

already done by cmake

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure?
If i remove the line it seems like the build takes more time:

# command used: `time nix-build . -A parmetis`
# with `enableParallelBuilding = true;`
nix-build . -A parmetis  1.68s user 0.34s system 8% cpu 22.698 total
# vs without `enableParallelBuilding = true;`
nix-build . -A parmetis  1.67s user 0.35s system 3% cpu 57.441 total

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a result of running make in the configurePhase?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably want [ cmake ninja ].

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you mentioned this before:
image

#329102 (comment)

Or am i using this incorrectly? I just added it to the nativeBuildInputs

Copy link
Member

@emilazy emilazy Aug 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh, sorry, I completely forgot, I was just reacting to the parallel build stuff… it should just work with CMake builds, but I guess there’s something funky going on here. I think you’d have to drop the make stuff in configurePhase and just let the CMake hook run naturally, with any required cmakeFlags corresponding to the ones set in the upstream Makefile.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested several different approaches: Running the tar xf during preConfigure and not touching the configurePhase, commenting out make in the configurePhase, but nothing worked. How bad would it be to just set enableParallelBuilding = true;? I can also remove the line and we can settle for only building with one thread if the option is that bad...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When setting the cmakeFlags to

cmakeFlags = [
  "-DGKLIB_PATH=metis/GKlib"
  "-DMETIS_PATH=metis" 
];

The build is running in parallel as expected, but then the build fails with

parmetis> [ 55%] Building C object libparmetis/CMakeFiles/parmetis.dir/balancemylink.c.o
parmetis> [ 55%] Building C object libparmetis/CMakeFiles/parmetis.dir/akwayfm.c.o
parmetis> [ 56%] Building C object libparmetis/CMakeFiles/parmetis.dir/comm.c.o
parmetis> In file included from /build/source/libparmetis/balancemylink.c:14:
parmetis> /build/source/libparmetis/./parmetislib.h:23:10: fatal error: ../metis/libmetis/gklib_defs.h: No such file or directory
parmetis>    23 | #include "../metis/libmetis/gklib_defs.h"
parmetis>       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
parmetis> compilation terminated.
parmetis> In file included from /build/source/libparmetis/akwayfm.c:14:
parmetis> /build/source/libparmetis/./parmetislib.h:23:10: fatal error: ../metis/libmetis/gklib_defs.h: No such file or directory
parmetis>    23 | #include "../metis/libmetis/gklib_defs.h"
parmetis>       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
parmetis> make[2]: *** [libparmetis/CMakeFiles/parmetis.dir/build.make:104: libparmetis/CMakeFiles/parmetis.dir/balancemylink.c.o] Error 1
parmetis> make[2]: *** Waiting for unfinished jobs....
parmetis> compilation terminated.
parmetis> make[2]: *** [libparmetis/CMakeFiles/parmetis.dir/build.make:76: libparmetis/CMakeFiles/parmetis.dir/akwayfm.c.o] Error 1
parmetis> In file included from /build/source/libparmetis/ametis.c:15:
parmetis> /build/source/libparmetis/./parmetislib.h:23:10: fatal error: ../metis/libmetis/gklib_defs.h: No such file or directory
parmetis>    23 | #include "../metis/libmetis/gklib_defs.h"
parmetis>       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
parmetis> compilation terminated.
parmetis> make[2]: *** [libparmetis/CMakeFiles/parmetis.dir/build.make:90: libparmetis/CMakeFiles/parmetis.dir/ametis.c.o] Error 1
parmetis> In file included from /build/source/libparmetis/comm.c:11:
parmetis> /build/source/libparmetis/./parmetislib.h:23:10: fatal error: ../metis/libmetis/gklib_defs.h: No such file or directory
parmetis>    23 | #include "../metis/libmetis/gklib_defs.h"
parmetis>       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
parmetis> compilation terminated.
parmetis> make[2]: *** [libparmetis/CMakeFiles/parmetis.dir/build.make:118: libparmetis/CMakeFiles/parmetis.dir/comm.c.o] Error 1
parmetis> make[1]: *** [CMakeFiles/Makefile2:184: libparmetis/CMakeFiles/parmetis.dir/all] Error 2
parmetis> make: *** [Makefile:136: all] Error 2

I'd prefer to simply use the commits I provided.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It’s totally fine to set enableParallelBuilding = true;, in my opinion. But it is indeed the overridden configurePhase that is the issue here, because CMake’s includes this:

    if ! [[ -v enableParallelBuilding ]]; then
        enableParallelBuilding=1
        echo "cmake: enabled parallel building"
    fi

I think the issue you’re running into there might be because you’re using a relative path whereas the Makefile explicitly makes it absolute, though.

buildInputs = [ mpi ];

configurePhase = ''
tar xf ${metis.src}
mv metis-* metis
make config metis_path=metis gklib_path=metis/GKlib prefix=$out
'';

meta = with lib; {
description = "Parallel Graph Partitioning and Fill-reducing Matrix Ordering";
longDescription = ''
MPI-based parallel library that implements a variety of algorithms for
partitioning unstructured graphs, meshes, and for computing fill-reducing
orderings of sparse matrices.
The algorithms implemented in ParMETIS are based on the multilevel
recursive-bisection, multilevel k-way, and multi-constraint partitioning
schemes
'';
homepage = "http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview";
platforms = platforms.all;
license = licenses.unfree;
maintainers = [ maintainers.costrouc ];
};
}
34 changes: 0 additions & 34 deletions pkgs/development/libraries/science/math/parmetis/default.nix
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions pkgs/top-level/all-packages.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37702,8 +37702,6 @@ with pkgs;
p4est-sc = p4est-sc-dbg;
};

parmetis = callPackage ../development/libraries/science/math/parmetis { };

QuadProgpp = callPackage ../development/libraries/science/math/QuadProgpp { };

scs = callPackage ../development/libraries/science/math/scs { };
Expand Down