Example updates

docs/src/index.md

@@ -11,7 +11,7 @@ end
 
 # JutulDarcy
 
-Documentation for [JutulDarcy](https://github.com/sintefmath/JutulDarcy.jl). The documentation is currently limited to docstrings and a series of examples. The examples are sorted by complexity. We suggest you start with [Gravity segregation example](Gravity segregation example).
+Documentation for [JutulDarcy.jl](https://github.com/sintefmath/JutulDarcy.jl). The documentation is currently limited to docstrings and a series of examples. The examples are sorted by complexity. We suggest you start with [Gravity segregation example](Gravity segregation example).
 
 !!! info "Note about units"
     JutulDarcy does currently not make us of conversion factors or explicit
@@ -20,6 +20,7 @@ Documentation for [JutulDarcy](https://github.com/sintefmath/JutulDarcy.jl). The
     match that of strict SI (e.g. Pascals and cubic meters per second). These
     scaling factors are primarily used when iterative linear solvers are used.
 
+JutulDarcy builds upon the general features found in [Jutul.jl](https://github.com/sintefmath/Jutul.jl). You may also find it useful to look at the [Jutul.jl documentation](https://sintefmath.github.io/Jutul.jl/dev/).
 
 ```@index
 ```

docs/src/usage/solution.md

@@ -124,8 +124,23 @@ Once that system is solved for ``\mathbf{x}_r``, we can recover the well degrees
 
 !!! note "Efficiency of Schur complement"
     Explicitly forming the matrix ``J_{rr} - J_{rw}J_{ww}^{-1}J_{wr}`` will generally lead to a lot of fill-in in the linear system. JutulDarcy instead uses the action of ``J_{rr} - J_{rw}J_{ww}^{-1}J_{wr}`` as a linear operator from [LinearOperators.jl](https://github.com/JuliaSmoothOptimizers/LinearOperators.jl). This means that we must apply the inverse of the well system every time we need to compute the residual or action of the system matrix, but fortunately performing the action of the Schur complement is inexpensive as long as ``J_{ww}`` is small and the factorization can be stored. 
-### CSR backend and experimental features
+### Parallel support
+There are two main ways of exploiting multiple cores in Jutul/JutulDarcy: Threads are automatically used for assembly and can be used for parts of the linear solve. If you require the best performance, you have to go to MPI where the linear solvers can use a parallel [BoomerAMG preconditioner](https://hypre.readthedocs.io/en/latest/solvers-boomeramg.html) via [HYPRE.jl](https://github.com/fredrikekre/HYPRE.jl).
+#### Shared memory
 An experimental thread-parallel backend for matrices and linear algebra can be enabled by setting `backend=:csr` in the call to [`setup_reservoir_model`](@ref). This backend provides additional features such as a parallel zero-overlap ILU(0) implementation and parallel apply for AMG, but these features are still work in progress.
+
+#### MPI support for distributed memory
+It is possible to run cases using MPI. You will have to set up an environment with the following packages under Julia 1.9+:
+`PartitionedArrays`, `MPI`, `JutulDarcy` and `HYPRE`. This is generally the best performing solver setup available, even if you are working in a shared memory environment.
+
+Write your script as usual, but in your call to `setup_reservoir_simulator`, pass the optional argument `mode = :mpi`. You must then run the file using `mpiexec`. 
+
+!!! note
+   You should be familiar with the MPI programming model to use this feature. See [MPI.jl](https://juliaparallel.org/MPI.jl/stable/) and [MPIClusterManagers.jl](https://github.com/JuliaParallel/MPIClusterManagers.jl) for more details, and how MPI is handled in Julia specifically.
+
+!!! note
+   MPI consolidates results by writing files to disk. Unless you have a plan to work with the distributed states in-memory returned by the `simulate!` call, it is best to specify a `output_path` optional argument to `setup_reservoir_simulator`. After the simulation, that folder will contain output just as if you had run the case in serial.
+
 ## References
 [Wallis, J.R. "Incomplete Gaussian Elimination as a Preconditioning for Generalized Conjugate Gradient Acceleration." Paper presented at the SPE Reservoir Simulation Symposium, San Francisco, California, November 1983](https://doi.org/10.2118/12265-MS)
 

docs/src/usage/systems.md

@@ -1,5 +1,5 @@
 # Supported physical systems
-JutulDarcy supports a number of different systems. These are [`JutulSystem`](@ref) instances that describe a particular type of physics for porous media flow. We describe these in roughly the order of complexity that they can model.
+JutulDarcy supports a number of different systems. These are [`JutulSystem`](https://sintefmath.github.io/Jutul.jl/dev/usage/#Jutul.JutulSystem) instances that describe a particular type of physics for porous media flow. We describe these in roughly the order of complexity that they can model.
 
 ## Summary
 The general form of the flow systems we will discuss is a conservation law for ``N`` components on residual form:

examples/co2_brine_2d_vertical.jl

@@ -7,6 +7,10 @@
 # getting started with compositional simulation.
 # ## Set up mixture
 # We load the external flash package and define a two-component H2O-CO2 system.
+# The constructor for each species takes in molecular weight, critical pressure,
+# critical temperature, critical volume, acentric factor given as strict SI.
+# This means, for instance, that molar masses are given in kg/mole and not
+# g/mole or kg/kmol.
 using MultiComponentFlash
 h2o = MolecularProperty(0.018015268, 22.064e6, 647.096, 5.595e-05, 0.3442920843)
 co2 = MolecularProperty(0.0440098, 7.3773e6, 304.1282, 9.412e-05, 0.22394)
@@ -22,11 +26,10 @@ using Jutul, JutulDarcy, CairoMakie
 nx = 50
 ny = 1
 nz = 20
-bar = 1e5
 dims = (nx, ny, nz)
 g = CartesianMesh(dims, (100.0, 10.0, 10.0))
 nc = number_of_cells(g)
-Darcy = 9.869232667160130e-13
+Darcy, bar, kg, meter, Kelvin, day, sec = si_units(:darcy, :bar, :kilogram, :meter, :Kelvin, :day, :second)
 K = repeat([0.1, 0.1, 0.001]*Darcy, 1, nc)
 res = reservoir_domain(g, porosity = 0.3, permeability = K)
 ## Set up a vertical well in the first corner, perforated in top layer
@@ -35,25 +38,25 @@ prod = setup_well(g, K, [(nx, ny, 1)], name = :Producer)
 inj = setup_well(g, K, [(1, 1, nz)], name = :Injector)
 #-
 # ## Define system and realize on grid
-rhoLS, rhoVS = 844.23, 126.97
+rhoLS = 844.23*kg/meter^3
+rhoVS = 126.97*kg/meter^3
 rhoS = [rhoLS, rhoVS]
 L, V = LiquidPhase(), VaporPhase()
 sys = MultiPhaseCompositionalSystemLV(eos, (L, V))
 model, parameters = setup_reservoir_model(res, sys, wells = [inj, prod], reference_densities = rhoS);
 push!(model[:Reservoir].output_variables, :Saturations)
 kr = BrooksCoreyRelPerm(sys, 2.0, 0.0, 1.0)
 model = replace_variables!(model, RelativePermeabilities = kr)
-T0 = repeat([303.15], 1, nc)
+T0 = repeat([303.15*Kelvin], 1, nc)
 parameters[:Reservoir][:Temperature] = T0
 state0 = setup_reservoir_state(model, Pressure = 50*bar, OverallMoleFractions = [1.0, 0.0]);
 
 # ## Define schedule
 # 5 year (5*365.24 days) simulation period
-day = 24*3600.0
 dt0 = repeat([1]*day, 26)
 dt1 = repeat([10.0]*day, 180)
 dt = append!(dt0, dt1)
-rate_target = TotalRateTarget(9.5066e-06)
+rate_target = TotalRateTarget(9.5066e-06*meter^3/sec)
 I_ctrl = InjectorControl(rate_target, [0, 1], density = rhoVS)
 bhp_target = BottomHolePressureTarget(50*bar)
 P_ctrl = ProducerControl(bhp_target)

examples/compositional_5components.jl

@@ -3,6 +3,8 @@
 # different components. Other than that, the example is similar to the others
 # that include wells and is therefore not commented in great detail.
 using MultiComponentFlash
+Darcy, bar, kg, meter, Kelvin, day = si_units(:darcy, :bar, :kilogram, :meter, :Kelvin, :day)
+
 n2_ch4 = MolecularProperty(0.0161594, 4.58e6, 189.515, 9.9701e-05, 0.00854)
 co2 = MolecularProperty(0.04401, 7.3866e6, 304.200, 9.2634e-05, 0.228)
 c2_5 = MolecularProperty(0.0455725, 4.0955e6, 387.607, 2.1708e-04, 0.16733)
@@ -21,18 +23,20 @@ eos = GenericCubicEOS(mixture, PengRobinson())
 using Jutul, JutulDarcy, CairoMakie
 nx = ny = 20
 nz = 2
-bar = 1e5
+
 dims = (nx, ny, nz)
 g = CartesianMesh(dims, (1000.0, 1000.0, 1.0))
 nc = number_of_cells(g)
-K = repeat([5.0e-14], 1, nc)
+K = repeat([0.05*Darcy], 1, nc)
 res = reservoir_domain(g, porosity = 0.25, permeability = K)
 ## Set up a vertical well in the first corner, perforated in all layers
 prod = setup_vertical_well(g, K, nx, ny, name = :Producer)
 ## Set up an injector in the opposite corner, perforated in all layers
 inj = setup_vertical_well(g, K, 1, 1, name = :Injector)
 
-rhoLS, rhoVS = 1000.0, 100.0
+rhoLS = 1000.0*kg/meter^3
+rhoVS = 100.0*kg/meter^3
+
 rhoS = [rhoLS, rhoVS]
 L, V = LiquidPhase(), VaporPhase()
 # Define system and realize on grid
@@ -43,12 +47,10 @@ model = replace_variables!(model, RelativePermeabilities = kr)
 
 push!(model[:Reservoir].output_variables, :Saturations)
 
-T0 = repeat([387.45], 1, nc)
+T0 = repeat([387.45*Kelvin], 1, nc)
 parameters[:Reservoir][:Temperature] = T0
 state0 = setup_reservoir_state(model, Pressure = 225*bar, OverallMoleFractions = [0.463, 0.01640, 0.20520, 0.19108, 0.12432]);
 
-
-day = 24*3600.0
 dt = repeat([2.0]*day, 365)
 rate_target = TotalRateTarget(0.0015)
 I_ctrl = InjectorControl(rate_target, [0, 1, 0, 0, 0], density = rhoVS)

examples/five_spot_ensemble.jl

@@ -1,6 +1,6 @@
 # # Quarter-five-spot example
 # The quarter-five-spot is a standard test problem that simulates 1/4 of the
-# five spot well pattern by arguing for axial symmetry. The problem contains an
+# five spot well pattern by assuming axial symmetry. The problem contains an
 # injector in one corner and the producer in the opposing corner, with a
 # significant volume of fluids injected into the domain.
 using JutulDarcy, Jutul
@@ -19,19 +19,19 @@ function perm_kozeny_carman(Φ)
 end
 
 function simulate_qfs(porosity = 0.2)
-    bar = 1e5
-    day = 3600*24.0
     Dx = 1000.0
     Dz = 10.0
     Darcy = 9.869232667160130e-13
+    Darcy, bar, kg, meter, Kelvin, day, sec = si_units(:darcy, :bar, :kilogram, :meter, :Kelvin, :day, :second)
+
     mesh = CartesianMesh((nx, nx, 1), (Dx, Dx, Dz))
     K = perm_kozeny_carman.(porosity)
     domain = reservoir_domain(mesh, permeability = K, porosity = porosity)
     Inj = setup_vertical_well(domain, 1, 1, name = :Injector);
     Prod = setup_vertical_well(domain, nx, nx, name = :Producer);
     phases = (LiquidPhase(), VaporPhase())
-    rhoLS = 1000.0
-    rhoGS = 700.0
+    rhoLS = 1000.0*kg/meter^3
+    rhoGS = 700.0*kg/meter^3
     rhoS = [rhoLS, rhoGS]
     sys = ImmiscibleSystem(phases, reference_densities = rhoS)
     model, parameters = setup_reservoir_model(domain, sys, wells = [Inj, Prod])

examples/two_phase_gravity_segregation.jl

@@ -15,9 +15,11 @@ domain = get_1d_reservoir(nc, z_max = 1)
 # ## Fluid properties
 # Define two phases liquid and vapor with a 10-1 ratio reference densities and
 # set up the simulation model.
-bar = 1e5
+Darcy, bar, kg, meter, day = si_units(:darcy, :bar, :kilogram, :meter, :day)
 p0 = 100*bar
-rhoLS, rhoVS = 1000.0, 100.0
+
+rhoLS = 1000.0*kg/meter^3
+rhoVS = 100.0*kg/meter^3
 cl, cv = 1e-5/bar, 1e-4/bar
 L, V = LiquidPhase(), VaporPhase()
 sys = ImmiscibleSystem([L, V])
@@ -37,7 +39,7 @@ sL = vcat(ones(nl), zeros(nc - nl))'
 s0 = vcat(sL, 1 .- sL)
 state0 = setup_state(model, Pressure = p0, Saturations = s0)
 # Convert time-steps from days to seconds
-timesteps = repeat([0.02]*3600*24, 150)
+timesteps = repeat([0.02]*day, 150)
 #-
 ## Perform simulation
 states, report = simulate(state0, model, timesteps, info_level = -1)

examples/two_phase_unstable_gravity.jl

@@ -19,9 +19,10 @@ D = 10.0
 g = CartesianMesh((nx, 1, nz), (D, 1.0, D))
 domain = reservoir_domain(g)
 # ## Set up model and properties
-bar = 1e5
+Darcy, bar, kg, meter, day = si_units(:darcy, :bar, :kilogram, :meter, :day)
 p0 = 100*bar
-rhoLS, rhoVS = 1000.0, 500.0 # Definition of fluid phases
+rhoLS = 1000.0*kg/meter^3 # Definition of fluid phases
+rhoVS = 500.0*kg/meter^3
 cl, cv = 1e-5/bar, 1e-4/bar
 L, V = LiquidPhase(), VaporPhase()
 sys = ImmiscibleSystem([L, V])

examples/wells_intro.jl

@@ -11,9 +11,7 @@ using JutulDarcy, Jutul
 # `JutulDarcy` uses SI units internally. It is therefore convenient to define a
 # few constants at the start of the script to have more managable numbers later
 # on.
-bar = 1e5
-day = 3600*24.0
-Darcy = 9.869232667160130e-13
+Darcy, bar, kg, meter, day = si_units(:darcy, :bar, :kilogram, :meter, :day)
 
 # ## Defining a porous medium
 # We start by defining the static part of our simulation problem -- the porous medium itself.
@@ -86,7 +84,7 @@ Inj = setup_well(domain, [(nx, ny, 1)], name = :Injector);
 phases = (LiquidPhase(), VaporPhase())
 rhoLS = 1000.0
 rhoGS = 100.0
-rhoS = [rhoLS, rhoGS]
+rhoS = [rhoLS, rhoGS] .* kg/meter^3
 sys = ImmiscibleSystem(phases, reference_densities = rhoS)
 # ### Creating the model
 # The same fluid system can be used for both flow inside the wells and the