DLBFoam: Dynamic load balancing for fast reactive simulations

DLBFoam is an open-source library for OpenFOAM. It introduces dynamic load balancing and a zonal reference mapping model for fast chemistry calculation in parallel simulations.

Why do I need this?

Load imbalance in parallel reactive simulations is an issue that causes very long simulation times in OpenFOAM simulations utilizing finite-rate chemistry.

DLBFoam introduces runtime load balancing through MPI routines to minimize the load imbalance between ranks and gain speed-up. The implementation details can be found in our paper [1].

Figure: A demonstration of how load balancing model works compared to standard model.

Compilation

DLBFoam does not require any third-party dependency. After sourcing OpenFOAM-8, simply execute:

./Allwmake

Usage

Once the compilation is successful, any case running with standard OpenFOAM can be easily converted to use DLBFOAM, following these steps:

• The DLBFoam should be linked to the solver. Add the following to your system/controlDict file:

libs
(
    "libchemistryModel_DLB.so" 
);

• Select chemistry solver method as loadBalanced in constant/chemistryProperties:

chemistryType
{
    solver          ode;
    method          loadBalanced;
}

• Add the loadbalancing subdictionary to the same chemistryProperties file:

loadbalancing
{
    active true;
    log	true;
}

• (Optional) Set the refmapping as active in chemistryProperties file if you want to use the reference mapping method (you have to add an empty refmapping{} dict even if you do not use it):

refmapping
{
    active  true;
    
    mixtureFractionProperties
    {
        oxidizerMassFractions
        {
            N2       0.77;
            O2       0.23;
        }

        fuelMassFractions
        {
            NC12H26       1.0;
        }

        #include "$FOAM_CASE/constant/thermo.compressibleGasGRI"
    }
    tolerance	1e-4;  // mixture fraction tolerance
    deltaT	2; // temperature tolerance
}

Reference mapping uses mixture fraction (Z) and maps a reference solution to reference cells satisfying a condition.

The entry above sets the Z=0 and Z=1 conditions from given mass fractions. For each CFD iteration it finds a reference solution where Z<tolerance and solves the chemistry. Subsequent cells following the same condition are mapped from this reference solution.

Optional: When deltaT is explicitly set, the mapper also checks the temperature between reference solution and other reference cells and ensures: abs(T_cell-T_ref)<deltaT.

• Run the case normally with OpenFOAM's reactive solvers.

For a working example, check the tutorials given in tutorials folder.

Directory structure

├── src
└── thermophysicalModels
│    └── chemistryModel
│        ├── chemistryModel
│        │   └── loadBalancedChemistryModel
│        │       ├── LoadBalancedChemistryModel    // Main chemistry class
│        ├── loadBalancing
│        │   ├── algorithms_DLB                    // Some useful algorithms used
│        │   ├── ChemistryLoad                     // Chemistry load object
│        │   ├── ChemistryProblem                  // Chemistry problem object
│        │   ├── ChemistrySolution                 // Chemistry solution object
│        │   ├── LoadBalancerBase                  // Load balancer base class
│        │   ├── LoadBalancer                      // Load balancer implementation class
│        │   ├── RecvBuffer                        // Receive MPI buffer object
│        │   ├── runtime_assert                    // Assert functions for debugging
│        │   ├── SendBuffer                        // Send MPI buffer object
│        └── refMapping
│            ├── mixtureFraction                   // Mixture fraction implementation
│            ├── mixtureFractionRefMapper          // Reference mapper implementation class
│
├── tutorials                                      // Tutorials
└── unittests                                      // Unit tests to check if compilation is successful

Getting help and reporting bugs

Please submit a GitHub issue if you found a bug in the program. If you need help with the software or have further questions, contact [email protected].

Citation

If you use our model, please cite the publication describing its implementation [1].

References

[1] Tekgul, Peltonen, Kahila, Kaario, Vuorinen (2020). DLBFoam: An open-source dynamic load balancing model for fast reacting flow simulations in OpenFOAM