diff --git a/dev/configuration/index.html b/dev/configuration/index.html index b1a2f3cce..b330cc53b 100644 --- a/dev/configuration/index.html +++ b/dev/configuration/index.html @@ -12,9 +12,17 @@ abi = "OpenMPI" binary = "system" libmpi = "/software/mpi/lib/libmpi.so" -mpiexec = "/software/mpi/bin/mpiexec"
updating the contents of the [preferences.MPIPreferences]
section match those of the [MPIPreferences]
in LocalPreferences.toml
.
Append the directory containing the file to the JULIA_LOAD_PATH
environment variable, with a colon (:
) separator.
If this variable is not already set, it should be prefixed with a colon to ensure correct behavior of the Julia load path, e.g. JULIA_LOAD_PATH=":/software/mpi/julia"
. If using environment modules, this can be achieved with
append-path -d {} JULIA_LOAD_PATH :/software/mpi/julia
in the corresponding module file (preferably the module file for the MPI installation or for Julia).
The user can still provide differing MPI configurations for each Julia project that will take precedent by modifying the local Project.toml
or by providing a LocalPreferences.toml
file.
The following MPI implementations are provided as JLL packages and automatically obtained when installing MPI.jl:
MicrosoftMPI_jll
: Microsoft MPI Default for WindowsMPICH_jll
: MPICH. Default for all other systemsOpenMPI_jll
: Open MPIMPItrampoline_jll
: MPItrampoline: an MPI forwarding layer.Call MPIPreferences.use_jll_binary
, for example
julia --project -e 'using MPIPreferences; MPIPreferences.use_jll_binary("MPItrampoline_jll")'
If you omit the JLL binary name, the default is selected for the respective operating system.
The LocalPreferences.toml
must be located within the test
folder, you can either create it in place or copy it into place.
~/MPI> julia --project=test
+mpiexec = "/software/mpi/bin/mpiexec"
updating the contents of the [preferences.MPIPreferences]
section match those of the [MPIPreferences]
in LocalPreferences.toml
.
Append the directory containing the file to the JULIA_LOAD_PATH
environment variable, with a colon (:
) separator.
If this variable is not already set, it should be prefixed with a colon to ensure correct behavior of the Julia load path, e.g. JULIA_LOAD_PATH=":/software/mpi/julia"
. If using environment modules, this can be achieved with
append-path -d {} JULIA_LOAD_PATH :/software/mpi/julia
in the corresponding module file (preferably the module file for the MPI installation or for Julia).
The user can still provide differing MPI configurations for each Julia project that will take precedent by modifying the local Project.toml
or by providing a LocalPreferences.toml
file.
MPIPreferences
can load vendor-specific libraries and settings using the vendor
parameter, eg MPIPreferences.use_system_binary(mpiexec="srun", vendor="cray")
configures MPIPreferences
for use on Cray systems with srun
.
Currently vendor
only supports Cray systems.
This populates the library_names
, preloads
, preloads_env_switch
and cclibs
preferences. These are defermined by parsing cc --cray-print-opts=all
emitted from the Cray Compiler Wrappers. Therefore use_system_binary
needs to be run on the target system, with the corresponding PrgEnv
loaded.
The function of these settings are as follows:
preloads
specifies a list of libraries that are to be loaded (in order) before libmpi
.preloads_env_switch
specifies the name of an environment variable that, if set to 0
, can disable the preloads
cclibs
is a list of libraries also linked by the compiler wrappers. This is recorded mainly for debugging purposes, and the libraries listed here are not explicitly loaded by MPI.jl
.If these are set, the _format
key will be set to "1.1"
.
An example of running MPIPreferences.use_system_library(vendor="cray")
in PrgEnv-gnu
is:
[MPIPreferences]
+_format = "1.1"
+abi = "MPICH"
+binary = "system"
+cclibs = ["cupti", "cudart", "cuda", "sci_gnu_82_mpi", "sci_gnu_82", "dl", "dsmml", "xpmem"]
+libmpi = "libmpi_gnu_91.so"
+mpiexec = "mpiexec"
+preloads = ["libmpi_gtl_cuda.so"]
+preloads_env_switch = "MPICH_GPU_SUPPORT_ENABLED"
This is an example of CrayMPICH requiring libmpi_gtl_cuda.so
to be preloaded, unless MPICH_GPU_SUPPORT_ENABLED=0
(the latter allowing MPI-enabled code to run on a non-GPU enabled node without needing a seperate LocalPreferences.toml
).
The following MPI implementations are provided as JLL packages and automatically obtained when installing MPI.jl:
MicrosoftMPI_jll
: Microsoft MPI Default for WindowsMPICH_jll
: MPICH. Default for all other systemsOpenMPI_jll
: Open MPIMPItrampoline_jll
: MPItrampoline: an MPI forwarding layer.Call MPIPreferences.use_jll_binary
, for example
julia --project -e 'using MPIPreferences; MPIPreferences.use_jll_binary("MPItrampoline_jll")'
If you omit the JLL binary name, the default is selected for the respective operating system.
The LocalPreferences.toml
must be located within the test
folder, you can either create it in place or copy it into place.
~/MPI> julia --project=test
julia> using MPIPreferences
julia> MPIPreferences.use_system_binary()
~/MPI> rm test/Manifest.toml
~/MPI> julia --project
-(MPI) pkg> test
The test suite can also be modified by the following variables:
JULIA_MPI_TEST_NPROCS
: How many ranks to use within the testsJULIA_MPI_TEST_ARRAYTYPE
: Set to CuArray
or ROCArray
to test the CUDA-aware interface with CUDA.CuArray
or the ROCm-aware interface with AMDGPU.ROCArray
or buffers.JULIA_MPI_TEST_BINARY
: Check that the specified MPI binary is used for the testsJULIA_MPI_TEST_ABI
: Check that the specified MPI ABI is used for the testsFor MPI.jl v0.20, environment variables were used to configure which MPI library to use. These have been removed and no longer have any effect. The following subsections explain how to the same effects can be achieved with v0.20 or later.
Please refer to Notes to HPC cluster administrators if you want to migrate your MPI.jl preferences on a cluster with a centrally managed MPI.jl configuration.
JULIA_MPI_BINARY
Use MPIPreferences.use_system_binary
to use a system-provided MPI binary as described here. To switch back or select a different JLL-provided MPI binary, use MPIPreferences.use_jll_binary
as described here.
JULIA_MPI_PATH
Removed without replacement.
JULIA_MPI_LIBRARY
Use MPIPreferences.use_system_binary
with keyword argument library_names
to specify possible, non-standard library names. Alternatively, you can also specify the full path to the library.
JULIA_MPI_ABI
Use MPIPreferences.use_system_binary
with keyword argument abi
to specify which ABI to use. See MPIPreferences.abi
for possible values.
JULIA_MPIEXEC
Use MPIPreferences.use_system_binary
with keyword argument mpiexec
to specify the MPI launcher executable.
JULIA_MPIEXEC_ARGS
Use MPIPreferences.use_system_binary
with keyword argument mpiexec
, and pass a Cmd
object to set the MPI launcher executable and to include specific command line options.
JULIA_MPI_INCLUDE_PATH
Removed without replacement. Automatic generation of a constants file for unknown MPI ABIs is not supported anymore. See also #574.
JULIA_MPI_CFLAGS
Removed without replacement. Automatic generation of a constants file for unknown MPI ABIs is not supported anymore. See also #574.
JULIA_MPICC
Removed without replacement. Automatic generation of a constants file for unknown MPI ABIs is not supported anymore. See also #574.
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 18 July 2023. Using Julia version 1.9.2.