Build FANS as a library to be coupled to a macro-scale simulation via…

… preCICE and the Micro Manager (#23)

* add new features to CMakeLists.txt and adapt modern best practices

* Add Language C to project to allow HDF5 to compile its C parts

* Add autoformat file for CMake, Presets and a Doxyfile

* remove unnecessary includes and links, replace linker flag by cmake command for GNU math lib

* replace -fopenmp by modern CMake way

* move compiler options to target based locations

* Use FFTW3 find script and find_package(MPI)

* add FindFFTW3.cmake (was hidden by .git
ignore before)

* change output name of shared lib from libFANS_FANS.so to libFANS.so

* ignore .cmake folder generated by CLion

* Dockerfile for containerized build and publishable standalone selfcontained FANS container

* add another container stage for dev-env, add docker.yaml github action and update readme accordingly.

* fix typo

* fix missing CMD start command for container dev-env

* Make use of auto-determined dependencies in runtime docker image (reduces size from 495MB to 161MB). Also add dev deps to dev package explicitly. And update docker.yaml workflow

* Set up cross-compilation for arm64 using Docker buildx

* fix COMPONENT_PACKAGE_DEPENDS (needs to be uppercase), and add openmpi-bin to runtime deps

* rename packages and edit descriptions

* Add CI workflow to build and package FANS

* update version of publish artifacts action

* Add remaining dependencies to FANSConfig.cmake and include FindFFTW3.cmake in install step

* run autoformat

* Reoarganize CMake files, fix install calls, add found message

* remove COMPONENT statement in install(TARGETS ... INCLUDES) since that's not supported there and erraneously add include paths called COMPONENT

* cmake overhaul

* Add Docker images for building FANS and instructions in the README.md

* clean up comments

* Add Github Actions workflow to build and publish docker images fans-ci and fans-dev

* upgrade to latest LTS ubuntu

* rename docker github action yaml + fix dockerfile

* add workflow to build FANS

* Rename build workflow to CI and add a job for packaging FANS

* instead of putting the FANS executable in the test/ dir, just symlink it

* temporarily disable arm64 support since it takes super long to run

* add debug output

* use just one job to avoid complicated data exchange between jobs

* add fans docker image + github actions workflow

* make build variable dir in actions + streamline dockerfile of fans image

* add support for older ubuntu versions

* escape & sign in Dockerfile RUN command

* fix typo

* add support for different ubuntu LTS versions

* use matrix also for arch in docker helpers workflow

* put variables in brackets

* also use matrix.arch in fans docker image

* fix weird escape error that occurs when switching ubuntu versions

* fix comment

* remove focal as it causes some CPack error

* use correct variable substitution syntax

* variable substitution

* add ubuntu version to name of artifact

* make detection if project is top level compatible with older CMake versions

* reactivate focal again after fix

* use explicit path instead of dot

* Make IPO handling compatible with CMake 3.0

* Add max version to cmake_mininum_required() to indicate that we don't rely on old behaviour but support old versions

* make detection if project is top level compatible with older CMake versions

* fix RPATH settings

* update README.md

* fix link

* remove commas in package lists

* restructure Installation section and add link to prebuilt packages

* move docker section of README.md into docker/ dir

* add section part to link

* replace multiple docker run commands by docker create + docker exec

* fix docker exec call

* updated h5 to xdmf viz pipeline

* Fix path to filename in 3d0_MFL

* small changes

* can provide keyword for temporal dataset group names

* Adapt FANS for MM initial commit

* Adapt FANS further

* fix quotation mark

* fix quotation mark

* provide users with a cache option for enabling packaging

* Micro manager python bindings

* add symlink to test directory for convenience

* Move Micro Manager input files

* Add Micro Manager input files

* Little tweaks on the MM version

* Running Micro Manager version of FANS

* Change Class descriptions

* Clean up

* Fix building of bindings

* CMake edit temporarily

* Formatting

* restructure micromanager part of CMakeLists.txt file

* Align with current state on develop branch

* Remove unnecessary code and get homogenized stress only by access

* Remove thermal micro simulation code as it is just a copy of the mechanics code

* Remove duplicate micro.hpp from include/

* Remove more redundant code and fix types

* Streamline code in micro.hpp and micro.cpp

* A first implementation of numerically calculating the tangent stiffness matrix. NOT WORKING

* First working version with logically correct stiffness matrix C

* Bump clang-format version in pre-commit config

* Formatting

* Rename folder with content for micro library to pyfans

* Remove files related to thermal test of micro simulation library

* Replace Eigen Matrix row.begin and row.end with simpler functionality

* Remove MPI initialization

* Initial restructuring of pyfans test, and remove the numerical computation of the material tangent in the lib files

* Remove h5 file and revert inadvertant changes to input file

* Add documentation and test CI for pyFANS

* Working GitHub Action to test pyFANS

* Add CHANGELOG enty and a small comment in the pyFANS test script

* Remove stale input.json from test/

* Explicitly mention what type of FANS library is being compiled

Co-authored-by: Moritz Sigg <[email protected]>

* Incorporate review comments

* Remove unnecessary parts from pyfans/CmakeLists.txt

* Hardcode mechanical model and solver because delaying variable definitions are hacky

* Incorporate feedback regarding the test

* Remove unnecessary code

* One empty entry in macroscale_loading should be sufficient

---------

Co-authored-by: Moritz Sigg <[email protected]>
Co-authored-by: Moritz Sigg <[email protected]>
Co-authored-by: ac133718 <[email protected]>
Co-authored-by: Torben Schiz <[email protected]>
Co-authored-by: Sanath Keshav <[email protected]>

.github/workflows/test_pyfans.yaml

@@ -0,0 +1,37 @@
+name: Test PyFans
+
+on: [push, pull_request]
+
+jobs:
+  pyfans-run-test:
+    runs-on: ubuntu-latest
+    container: unistuttgartdae/fans-ci:noble
+    defaults:
+      run:
+        shell: "bash --login -eo pipefail {0}"
+    env:
+      FANS_BUILD_DIR: build
+      FANS_MPI_USER: fans
+    steps:
+
+    - name: Checkout repository
+      uses: actions/checkout@v2
+
+    - name: Generate build directory
+      run: mkdir -p ${{ env.FANS_BUILD_DIR }}
+
+    - name: Install dependencies
+      run: |
+        apt update
+        apt install -y cmake make g++ python3 python3-numpy python3.12-dev
+
+    - name: Configure
+      working-directory: ${{ env.FANS_BUILD_DIR }}
+      run: |
+        cmake .. -DFANS_LIBRARY_FOR_MICRO_MANAGER=ON
+        make
+
+    - name: Run FANS as a library via a Python script
+      run: |
+        cd test/test_pyfans
+        python3 run_fans_as_library.py

CHANGELOG.md

@@ -1,5 +1,9 @@
 # FANS Changelog
 
+## latest
+
+- Build FANS as a library to be coupled to a macro-scale simulation via preCICE and the Micro Manager https://github.com/DataAnalyticsEngineering/FANS/pull/23
+
 ## v0.3.0
 
 - Added Linear thermal and mechanical triclinic material models https://github.com/DataAnalyticsEngineering/FANS/pull/32

CMakeLists.txt

@@ -87,6 +87,17 @@ find_package(MPI REQUIRED)
 
 find_package(FFTW3 REQUIRED COMPONENTS DOUBLE MPI)
 
+option(FANS_LIBRARY_FOR_MICRO_MANAGER "Building FANS as a library to be used by the Micro Manager." OFF)
+
+if (FANS_LIBRARY_FOR_MICRO_MANAGER)
+    include(FetchContent)
+    FetchContent_Declare(
+        pybind11 GIT_REPOSITORY https://github.com/pybind/pybind11.git
+        GIT_TAG v2.12.0
+    )
+    FetchContent_MakeAvailable(pybind11)
+endif()
+
 # ##############################################################################
 # TARGETS
 # ##############################################################################
@@ -115,6 +126,10 @@ add_custom_command(
         COMMENT "Create a symlink for FANS executable to ${CMAKE_CURRENT_SOURCE_DIR}/test/"
 )
 
+if (FANS_LIBRARY_FOR_MICRO_MANAGER)
+    add_subdirectory(pyfans)
+endif ()
+
 # ##############################################################################
 # HEADERS
 # ##############################################################################

pyfans/CMakeLists.txt

@@ -0,0 +1,11 @@
+pybind11_add_module(PyFANS micro.hpp micro.cpp)
+target_link_libraries(PyFANS PRIVATE FANS::FANS)
+
+add_custom_command(
+        TARGET PyFANS
+        POST_BUILD
+        COMMAND ${CMAKE_COMMAND} -E create_symlink
+        $<TARGET_FILE:PyFANS>
+        ${CMAKE_CURRENT_SOURCE_DIR}/../test/test_pyfans/$<TARGET_FILE_NAME:PyFANS>
+        COMMENT "Create a symlink for FANS python bindings to ${CMAKE_CURRENT_SOURCE_DIR}/../test/"
+)

pyfans/README.md

@@ -0,0 +1,15 @@
+# pyFANS
+
+pyFANS is a Python-wrapped library to control FANS via the [Micro Manager](https://precice.org/tooling-micro-manager-overview.html). The main idea is to create a large number of FANS simulations, and couple them to one macro-scale simulation typically in Abaqus, CalculiX, etc. The library follows the [API of the Micro Manager](https://precice.org/tooling-micro-manager-prepare-micro-simulation.html).
+
+## Dependencies
+
+- [pybind11](https://pybind11.readthedocs.io/en/stable/index.html)
+
+## Building
+
+To build FANS as a Micro Manager compatible Python library, set the CMake variable `FANS_LIB` to `ON`. The CMake command to compile FANS would then be `cmake .. -DFANS_LIBRARY_FOR_MICRO_MANAGER=ON`.
+
+## Usage
+
+pyFANS is intended to be used with the Micro Manager and preCICE for two-scale coupled simulations. However, standalone use of the library is not restricted per se. Look at the [test_pyfans](../test/test_pyfans/) example to see how the library is used in a Python script.

pyfans/micro.cpp

@@ -0,0 +1,100 @@
+// Micro simulation for mechanical problems
+// In this file we solve a micro problem with FANS which is controlled by the Micro Manager
+// This file is compiled with pybind11 to be available as a python module
+//
+// To check if python is able to import it, run:
+// python3 -c "import micro; micro.MicroSimulation(1)"
+// from the same directory
+
+#include "micro.hpp"
+#include "setup.h"
+#include "matmodel.h"
+
+py::array_t<double> merge_arrays(py::array_t<double> array1, py::array_t<double> array2)
+{
+    // Ensure arrays are contiguous for efficient merging
+    array1 = array1.attr("copy")();
+    array2 = array2.attr("copy")();
+
+    // Get numpy concatenate function
+    py::object np          = py::module::import("numpy");
+    py::object concatenate = np.attr("concatenate");
+
+    // Concatenate the two arrays
+    py::tuple           arrays = py::make_tuple(array1, array2);
+    py::array_t<double> result = concatenate(arrays, py::int_(0)).cast<py::array_t<double>>();
+
+    return result;
+}
+
+MicroSimulation::MicroSimulation(int sim_id, char *input_file)
+{
+    MPI_Init(NULL, NULL);
+
+    // initialize fftw mpi
+    fftw_mpi_init();
+
+    // Input file name is hardcoded. TODO: Make it configurable
+    reader.ReadInputFile(input_file);
+
+    reader.ReadMS(3);
+    matmodel = createMatmodel<3>(reader);
+    solver   = createSolver<3>(reader, matmodel);
+}
+
+py::dict MicroSimulation::solve(py::dict macro_data, double dt)
+{
+    // Time step value dt is not used currently, but is available for future use
+
+    // Create a pybind style Numpy array from macro_write_data["micro_vector_data"], which is a Numpy array
+    py::array_t<double> strain1 = macro_data["strains1to3"].cast<py::array_t<double>>();
+    py::array_t<double> strain2 = macro_data["strains4to6"].cast<py::array_t<double>>();
+
+    py::array_t<double> strain = merge_arrays(strain1, strain2);
+    std::vector<double> g0     = std::vector<double>(strain.data(), strain.data() + strain.size()); // convert numpy array to std::vector.
+
+    VectorXd homogenized_stress;
+
+    matmodel->setGradient(g0);
+
+    solver->solve();
+
+    homogenized_stress = solver->get_homogenized_stress();
+
+    auto C = solver->get_homogenized_tangent(pert_param);
+
+    // Convert data to a py::dict again to send it back to the Micro Manager
+    py::dict micro_write_data;
+
+    // Add stress and stiffness matrix data to Python dict to be returned
+    std::vector<double> stress13     = {homogenized_stress[0], homogenized_stress[1], homogenized_stress[2]};
+    micro_write_data["stresses1to3"] = stress13;
+    std::vector<double> stress46     = {homogenized_stress[3], homogenized_stress[4], homogenized_stress[5]};
+    micro_write_data["stresses4to6"] = stress46;
+    std::vector<double> C_1          = {C(0, 0), C(0, 1), C(0, 2)};
+    micro_write_data["cmat1"]        = C_1;
+    std::vector<double> C_2          = {C(0, 3), C(0, 4), C(0, 5)};
+    micro_write_data["cmat2"]        = C_2;
+    std::vector<double> C_3          = {C(1, 1), C(1, 2), C(1, 3)};
+    micro_write_data["cmat3"]        = C_3;
+    std::vector<double> C_4          = {C(1, 4), C(1, 5), C(2, 2)};
+    micro_write_data["cmat4"]        = C_4;
+    std::vector<double> C_5          = {C(2, 3), C(2, 4), C(2, 5)};
+    micro_write_data["cmat5"]        = C_5;
+    std::vector<double> C_6          = {C(3, 3), C(3, 4), C(3, 5)};
+    micro_write_data["cmat6"]        = C_6;
+    std::vector<double> C_7          = {C(4, 4), C(4, 5), C(5, 5)};
+    micro_write_data["cmat7"]        = C_7;
+
+    return micro_write_data;
+}
+
+PYBIND11_MODULE(PyFANS, m)
+{
+    // optional docstring
+    m.doc() = "FANS for Micro Manager";
+
+    py::class_<MicroSimulation>(m, "MicroSimulation")
+        .def(py::init<int>())
+        .def("solve", &MicroSimulation::solve);
+}

pyfans/micro.hpp

@@ -0,0 +1,31 @@
+// This is the header file for the micro simulation class.
+// It is included in the micro_cpp_dummy.cpp file and the micro_cpp_dummy.cpp file is compiled with pybind11 to create a python module.
+// The python module is then imported in the Micro Manager.
+
+#pragma once
+#include <iostream>
+#include <vector>
+
+#include "pybind11/pybind11.h"
+#include "pybind11/numpy.h" // numpy arrays
+#include "pybind11/stl.h"   // std::vector conversion
+
+#include "general.h"
+#include "matmodel.h"
+#include "solver.h"
+
+namespace py = pybind11;
+
+class MicroSimulation {
+  public:
+    MicroSimulation(int sim_id, char *input_file = "input.json");
+    py::dict solve(py::dict macro_write_data, double dt);
+
+  private:
+    int    _sim_id;
+    Reader reader;
+    // Hardcoding mechanical models because these definitions need information from the input file.
+    Matmodel<3> *matmodel;
+    Solver<3>   *solver;
+    double       pert_param = 1e-6; // scalar strain perturbation parameter
+};

test/test_pyfans/README.md

@@ -0,0 +1,17 @@
+# Test pyFANS
+
+Test pyFANS as standalone library called from a Python script.
+
+## Build pyFANS
+
+Configure the FANS CMake build with the variable `FANS_LIB` set to `ON`.
+
+## Run the test
+
+Run
+
+```bash
+python3 run_fans_as_library.py
+```
+
+The script creates a pyFANS object and calls the `solve()` method. The script only checks if the pyFANS object is created and the solve function is callable. The result is not checked for correctness.

test/test_pyfans/input.json

@@ -0,0 +1,25 @@
+{
+    "ms_filename": "../microstructures/sphere32.h5",
+    "ms_datasetname": "/sphere/32x32x32/ms",
+    "ms_L": [1.0, 1.0, 1.0],
+
+    "problem_type": "mechanical",
+    "matmodel": "LinearElasticIsotropic",
+    "material_properties":{
+        "bulk_modulus": [62.5000, 222.222],
+        "shear_modulus": [28.8462, 166.6667]
+    },
+
+    "method": "cg",
+    "error_parameters":{
+        "measure": "Linfinity",
+        "type": "absolute",
+        "tolerance": 1e-10
+    },
+    "n_it": 100,
+    "macroscale_loading":   [
+                                [[]]
+                            ],
+
+    "results": []
+}

test/test_pyfans/run_fans_as_library.py

@@ -0,0 +1,24 @@
+"""
+Run FANS as a Python callable library.
+"""
+import PyFANS as fans
+import numpy as np
+
+# from mpi4py import MPI
+
+# MPI.Init()
+
+micro = fans.MicroSimulation(1)
+
+macro_data = dict()
+
+macro_data["strains1to3"] = np.random.rand(3)
+macro_data["strains4to6"] = np.random.rand(3)
+
+dt = 0.0001
+
+output = micro.solve(macro_data, dt)  # solve FANS for one load vector g0
+
+print(output)
+
+# MPI.Finalize()