Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ImportError when compiling with gcc 14.1.1 and conda #4922

Open
cphyc opened this issue Jun 10, 2024 · 6 comments
Open

ImportError when compiling with gcc 14.1.1 and conda #4922

cphyc opened this issue Jun 10, 2024 · 6 comments
Labels
bug build related to the build process

Comments

@cphyc
Copy link
Member

cphyc commented Jun 10, 2024

Bug report

Bug summary

The code compiles but fails when being imported.

Code for reproduction

python -c "import yt"

Actual outcome

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "[...]/yt/yt/__init__.py", line 14, in <module>
    from yt.data_objects.api import (
  File "[...]/yt/yt/data_objects/api.py", line 1, in <module>
    from . import construction_data_containers as __cdc, selection_objects as __sdc
  File "[...]/yt/yt/data_objects/construction_data_containers.py", line 17, in <module>
    from yt.data_objects.selection_objects.data_selection_objects import (
  File "[...]/yt/yt/data_objects/selection_objects/__init__.py", line 1, in <module>
    from .boolean_operations import (
  File "[...]/yt/yt/data_objects/selection_objects/boolean_operations.py", line 5, in <module>
    from yt.data_objects.selection_objects.data_selection_objects import (
  File "[...]/yt/yt/data_objects/selection_objects/data_selection_objects.py", line 14, in <module>
    from yt.data_objects.data_containers import YTDataContainer
  File "[...]/yt/yt/data_objects/data_containers.py", line 13, in <module>
    from yt.data_objects.profiles import create_profile
  File "[...]/yt/yt/data_objects/profiles.py", line 5, in <module>
    from yt.fields.derived_field import DerivedField
  File "[...]/yt/yt/fields/derived_field.py", line 18, in <module>
    from .field_detector import FieldDetector
  File "[...]/yt/yt/fields/field_detector.py", line 7, in <module>
    from yt.utilities.io_handler import io_registry
  File "[...]/yt/yt/utilities/io_handler.py", line 10, in <module>
    from yt.geometry.selection_routines import GridSelector
  File "yt/geometry/selection_routines.pyx", line 1, in init yt.geometry.selection_routines
    # distutils: include_dirs = LIB_DIR
ImportError: [...]/yt/yt/utilities/lib/image_samplers.cpython-312-x86_64-linux-gnu.so: undefined symbol: __cxa_call_terminate

Expected outcome

No error.

Version Information

  • Operating System: Linux
  • Python Version: 3.12.3
  • yt version: dev
  • Other Libraries (if applicable): python installed from conda environment with conda-forge channel.

Other pieces of information:

  • No issue when using the system-wide Python 3.12 installation
  • No issue if I compile with gcc-13 and g++-13 (CC=gcc-13 CXX=g++-13 python -m pip install -e yt).

The last point suggests some incompatibility between the conda-provided libstdcxx-ng (on version 13.2) install and the system-wide one (on version 14.1).

Details

Conda environment

# packages in environment at [...]/miniforge3/envs/py312:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
blosc                     1.21.5               hc2324a3_1    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
brotli-python             1.1.0           py312h30efb56_1    conda-forge
bzip2                     1.0.8                hd590300_5    conda-forge
c-ares                    1.28.1               hd590300_0    conda-forge
ca-certificates           2024.6.2             hbcca054_0    conda-forge
certifi                   2024.6.2           pyhd8ed1ab_0    conda-forge
cftime                    1.6.3           py312h085067d_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
cmyt                      2.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
colossus                  1.3.5                    pypi_0    pypi
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
contourpy                 1.2.1           py312h8572e83_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
cython                    3.0.10          py312h30efb56_0    conda-forge
debugpy                   1.8.1           py312h30efb56_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
et_xmlfile                1.1.0              pyhd8ed1ab_0    conda-forge
ewah-bool-utils           1.2.1           py312h1d6d2e6_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
fonttools                 4.53.0          py312h9a8786e_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
gmp                       6.3.0                h59595ed_1    conda-forge
gmpy2                     2.1.5           py312h1d5cde6_1    conda-forge
hdf4                      4.2.15               h2a13503_7    conda-forge
hdf5                      1.14.3          nompi_hdf9ad27_105    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
idna                      3.7                pyhd8ed1ab_0    conda-forge
importlib-metadata        7.1.0              pyha770c72_0    conda-forge
importlib_metadata        7.1.0                hd8ed1ab_0    conda-forge
ipykernel                 6.29.4             pyh3099207_0    conda-forge
ipython                   8.25.0             pyh707e725_0    conda-forge
ipywidgets                8.1.3              pyhd8ed1ab_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.2              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py312h7900ff3_0    conda-forge
jupyterlab_widgets        3.0.11             pyhd8ed1ab_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py312h8572e83_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.16                 hb7c19ff_0    conda-forge
ld_impl_linux-64          2.40                 hf3520f5_3    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.1.3                h59595ed_0    conda-forge
libblas                   3.9.0           22_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           22_linux64_openblas    conda-forge
libcurl                   8.8.0                hca28451_0    conda-forge
libdeflate                1.20                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libexpat                  2.6.2                h59595ed_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h77fa898_7    conda-forge
libgfortran-ng            13.2.0               h69a702a_7    conda-forge
libgfortran5              13.2.0               hca663fb_7    conda-forge
libgomp                   13.2.0               h77fa898_7    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           22_linux64_openblas    conda-forge
libnetcdf                 4.9.2           nompi_h135f659_114    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.27          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.43               h2797004_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.45.3               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               hc0a3c3a_7    conda-forge
libtiff                   4.6.0                h1dd3fc0_3    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.12.7               hc051c1a_1    conda-forge
libzip                    1.10.1               h2629f0a_3    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
matplotlib-base           3.8.4           py312h20ab3a6_2    conda-forge
matplotlib-inline         0.1.7              pyhd8ed1ab_0    conda-forge
matplotlib-label-lines    0.7.0                    pypi_0    pypi
more-itertools            10.2.0             pyhd8ed1ab_0    conda-forge
mpc                       1.3.1                hfe3b2da_0    conda-forge
mpfr                      4.2.1                h9458935_1    conda-forge
mpmath                    1.3.0              pyhd8ed1ab_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.5                  h59595ed_0    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
netcdf4                   1.6.5           nompi_py312h39d4375_102    conda-forge
numpy                     1.26.4          py312heda63a1_0    conda-forge
openjpeg                  2.5.2                h488ebb8_0    conda-forge
openpyxl                  3.1.2           py312h98912ed_1    conda-forge
openssl                   3.3.1                h4ab18f5_0    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pandas                    2.2.2           py312h1d6d2e6_1    conda-forge
parso                     0.8.4              pyhd8ed1ab_0    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.3.0          py312hdcec9eb_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
pooch                     1.8.2              pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.46             pyha770c72_0    conda-forge
psutil                    5.9.8           py312h98912ed_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.2              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.12.3          hab00c5b_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python_abi                3.12                    4_cp312    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyzmq                     26.0.3          py312h8fd38d8_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
scipy                     1.13.1          py312hc2bc53b_0    conda-forge
setuptools                70.0.0             pyhd8ed1ab_0    conda-forge
sh                        2.0.7                    pypi_0    pypi
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.2.0                hdb0a2a9_1    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
sympy                     1.12            pypyh9d50eac_103    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli-w                   1.0.0              pyhd8ed1ab_0    conda-forge
tornado                   6.4.1           py312h9a8786e_0    conda-forge
tqdm                      4.66.4             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
unyt                      2.9.5              pyhd8ed1ab_0    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
widgetsnbextension        4.0.11             pyhd8ed1ab_0    conda-forge
xlrd                      2.0.1              pyhd8ed1ab_3    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yt                        4.4.dev0                 pypi_0    pypi
zeromq                    4.3.5                h75354e8_4    conda-forge
zipp                      3.19.2             pyhd8ed1ab_0    conda-forge
zlib                      1.3.1                h4ab18f5_1    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge
@cphyc
Copy link
Member Author

cphyc commented Jun 10, 2024

After some digging, it seems like:

  1. numpy, cython, etc. depend on libstdcxx-ng >= 12 (the C++ standard library),
  2. conda resolves this to libstdcxx-ng == 13.2 at the time of writing (10/06/2024),
  3. with a system with GCC 14, this seems to lead to incompatibilities between the expected C++ stdlib and the one conda provides.

I'm not sure how to fix it. In the meantime, compiling with GCC-13 works :)

@cphyc
Copy link
Member Author

cphyc commented Jun 10, 2024 •
edited
Loading

Gotcha! Here's more info:

$ python -c "import yt"
# fails with `ImportError: [...]/yt/utilities/lib/image_samplers.cpython-312-x86_64-linux-gnu.so
# undefined symbol: __cxa_call_terminate`

$ ldd yt/utilities/lib/image_samplers.cpython-312-x86_64-linux-gnu.so
[...]
	libstdc++.so.6 => ~/miniforge3/envs/py312/lib/libstdc++.so.6 (0x00007fd779101000)
[...]

# If we preload the system C++ std library this now works
$ LD_PRELOAD=/usr/lib64/libstdc++.so.6.0.33 python -c 'import yt'

@cphyc cphyc added bug build related to the build process labels Jun 10, 2024
@matthewturk
Copy link
Member

Is it possible that you are compiling with the "wrong" CXX and this gets baked in? I don't see and gxx packages in your list.

@cphyc
Copy link
Member Author

cphyc commented Jun 13, 2024

The gcc/g++ version that is being used by default is the system one. Or are you saying one should install gcc/g++ with conda?

I'm not sure there's any actionable solution for this issue, so I'm also fine to close it as "stalled".

@matthewturk
Copy link
Member

Could you try installing the conda gcc/g++ and seeing if that fixes it?

@zingale
Copy link
Member

zingale commented Jul 19, 2024

probably not helpful, but I am running yt with GCC 14.1. I install from source and build with pip install ..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug build related to the build process
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@matthewturk @cphyc @zingale