Can't import Pygimli library after conda installation

[marex0956]

Problem description

I can't import the library pygimli after miniconda installation

Your environment

This is the error message in python:

(pg) user@user-20kf001rix:~$ python -c 'import pygimli as pg; print(pg.version())'
libcholmod.so.3: cannot open shared object file: No such file or directory
Traceback (most recent call last):
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/core.py", line 11, in
from . import pygimli # if it works: as pgcore, replace all pygimli
^^^^^^^^^^^^^^^^^^^^^^^
ImportError: cannot import name 'pygimli' from partially initialized module 'pygimli.core' (most likely due to a circular import) (/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/init.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pgcore/init.py", line 20, in
from .pygimli import *
ImportError: libcholmod.so.3: cannot open shared object file: No such file or directory
ERROR: cannot import the library 'pygimli'.
Traceback (most recent call last):
File "", line 1, in
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/init.py", line 8, in
from .core.decorators import (renamed, singleton, moduleProperty,
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/init.py", line 14, in
from .base import (isInt, isScalar, isIterable, isArray, isPos, isR3Array,
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/base.py", line 7, in
from .core import (RVector3, R3Vector, RMatrix)
ImportError: cannot import name 'RVector3' from 'pygimli.core.core' (/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/core.py)

Operating system: Debian 12 unstable branch
Python version: 3.11.9
pyGIMLi version: 1.5.1
Way of installation: Conda package

Steps to reproduce

I simply installed Miniconda latest .sh file, typed "conda create -n pg -c gimli -c conda-forge "pygimli>=1.5.0"" and "conda activate pg".

"""
(pg) user@user-20kf001rix:~$ python -c 'import pygimli as pg; print(pg.version())'
"""

Expected behavior

pygimli version ...

Actual behavior

libcholmod.so.3: cannot open shared object file: No such file or directory
Traceback (most recent call last):
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/core.py", line 11, in
from . import pygimli # if it works: as pgcore, replace all pygimli
^^^^^^^^^^^^^^^^^^^^^^^
ImportError: cannot import name 'pygimli' from partially initialized module 'pygimli.core' (most likely due to a circular import) (/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/init.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pgcore/init.py", line 20, in
from .pygimli import *
ImportError: libcholmod.so.3: cannot open shared object file: No such file or directory
ERROR: cannot import the library 'pygimli'.
Traceback (most recent call last):
File "", line 1, in
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/init.py", line 8, in
from .core.decorators import (renamed, singleton, moduleProperty,
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/init.py", line 14, in
from .base import (isInt, isScalar, isIterable, isArray, isPos, isR3Array,
File "/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/base.py", line 7, in
from .core import (RVector3, R3Vector, RMatrix)
ImportError: cannot import name 'RVector3' from 'pygimli.core.core' (/home/marco/miniconda3/envs/pg/lib/python3.11/site-packages/pygimli/core/core.py)

[robinthibaut]

Hi,

I’ve run into this issue too on Mac. The problem stems from mismatched shared library versions that pygimli expects, like libcholmod.so.3, which isn’t found because conda installs a different version (e.g., libcholmod.5).

Quick Fix: Create Symbolic Links
You can fix this by creating symbolic links, which are shortcuts that point to the actual library files installed by conda. This tells your system to use the available libcholmod.5.dylib whenever pygimli looks for libcholmod.3.dylib.

Here’s a script that I used:

#!/bin/bash
set -e

eval "$(conda shell.bash hook)"

conda remove --name gimli_env --all -y
conda create -n gimli_env -y python=3.10
conda activate gimli_env
conda install -c conda-forge boost scipy matplotlib openblas suitesparse -y
conda install -c gimli -c conda-forge "pygimli>=1.5.0" -y
conda install -c conda-forge numpy=1.25.2 "libblas=*=*openblas" -y

cd /opt/anaconda3/envs/gimli_env/lib
ln -s libumfpack.6.dylib libumfpack.5.dylib
ln -s libcholmod.5.dylib libcholmod.3.dylib
export DYLD_LIBRARY_PATH=/opt/anaconda3/envs/gimli_env/lib:$DYLD_LIBRARY_PATH

# Verify installation
if python -c "import pygimli" &> /dev/null; then
    echo "pygimli installed successfully."
else
    echo "Error: pygimli installation failed."
    exit 1
fi

This script does the following:

1. Sets up a fresh conda environment and installs the necessary packages.
2. Creates symbolic links for the expected libraries (ln -s).
3. Verifies the installation by importing pygimli.

Why This Happens
The reason for this issue is that pygimli or its dependencies might expect older versions of certain libraries. Conda, however, installs newer versions. By creating these symlinks, you make sure pygimli can find the libraries it needs.

Checking Dependencies with otool
You can use the otool -L command to check which shared libraries a binary or shared library depends on. For example:

otool -L /opt/anaconda3/envs/gimli_env/lib/libgimli.dylib

The output might look like this (this is from my terminal):

/opt/anaconda3/envs/gimli_env/lib/libgimli.dylib:
    @rpath/libgimli.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libcholmod.3.dylib (compatibility version 3.0.0, current version 3.0.14)
    @rpath/libumfpack.5.dylib (compatibility version 5.0.0, current version 5.7.9)
    @rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.61.1)

This command shows that libgimli.dylib depends on libcholmod.3.dylib and libumfpack.5.dylib, among others. If any of these libraries are missing or the versions don’t match, you’ll encounter errors like the ones you’re seeing.

@florian-wagner, @carsten-forty2, @halbmy, this issue seems to require attention, particularly for macOS users. We’ve encountered similar problems in our lab over the past few months, where installing pygimli on Macs through conda has become impossible without making these adjustments. The version mismatches between the expected and available libraries appear to be the root cause. It might be worth considering ways to ensure consistent library versions or providing clearer guidance on resolving these mismatches.

Best,
Robin

[prisae]

I guess this is a duplicate of #722

This might work:

conda create -n pg -c gimli -c conda-forge "pygimli>=1.5.0" python=3.10 suitesparse=5