"Library not loaded" after delocating Boost/ICU

[Time0o]

I'm using cibuildwheel to build a Python wheel on GitHub Actions (running OS X 10.15.7). My wheel depends on Boost Regex which in turn depends on ICU libs located under /usr/local/opt/icu4c/lib which I'm adding to the library search path during linking of a C++ extension module. cibuildwheel then internally calls:

delocate-listdeps {wheel} && delocate-wheel --require-archs x86_64 -w {dest_dir} {wheel}

So far so good. However, trying to install and run the wheel after this results in:

ImportError: dlopen(/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp8jiulamk/lib/python3.6/site-packages/mpsym/_mpsym.cpython-36m-darwin.so, 2): Library not loaded: @loader_path/libicuuc.67.dylib

So it seems like delocation has failed (?).

What can I do here? Is this a problem with delocate? I apologize if I've made a simple mistake, I usually work on Linux.

[matthew-brett]

Yes, that's strange. What output do you get for unzip -t <the-wheel-name>?

[Time0o]

Sorry, I've been busy with something else.cibuildwheel makes it sort of diffcult to debug the built wheel but here is the result of delocate-listdeps:

delocate-listdeps /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheel4_uufjsq/built_wheel/mpsym-0.5-cp36-cp36m-macosx_10_9_x86_64.whl && delocate-wheel --require-archs x86_64 -w /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheel4_uufjsq/repaired_wheel /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheel4_uufjsq/built_wheel/mpsym-0.5-cp36-cp36m-macosx_10_9_x86_64.whl
  /usr/local/Cellar/icu4c/67.1/lib/libicudata.67.1.dylib
  /usr/local/Cellar/icu4c/67.1/lib/libicui18n.67.1.dylib
  /usr/local/Cellar/icu4c/67.1/lib/libicuuc.67.1.dylib
  /usr/local/Cellar/[email protected]/5.3.6/lib/liblua.5.3.6.dylib

On Linux I have previously also had the problem that shared objects bundled into Python wheels could not be found (compare e.g. this StackOverflow question) and so I just link everything statically on Linux. But I would rather not build Boost from source on macOS so I was hoping this would work.

Here is some more context for the runtime error:

from ._mpsym import *
ImportError: dlopen(/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp7he7g3bm/lib/python3.6/site-packages/mpsym/_mpsym.cpython-36m-darwin.so, 2): Library not loaded: @loader_path/libicuuc.67.dylib
  Referenced from: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp7he7g3bm/lib/python3.6/site-packages/mpsym/.dylibs/libicui18n.67.1.dylib

[matthew-brett]

Actually, my particular interest was whether the file .dylibs/libicuuc.67.dylib exists in the installation from the wheel? I guess not...

[Time0o]

Here's what unzip -t returns:

   Archive:  /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheelrdu04dq_/repaired_wheel/mpsym-0.5-cp36-cp36m-macosx_10_9_x86_64.whl
      testing: mpsym-0.5.dist-info/RECORD   OK
      testing: mpsym-0.5.dist-info/WHEEL   OK
      testing: mpsym-0.5.dist-info/top_level.txt   OK
      testing: mpsym-0.5.dist-info/LICENSE.txt   OK
      testing: mpsym-0.5.dist-info/METADATA   OK
      testing: mpsym/_mpsym_tests.py    OK
      testing: mpsym/__init__.py        OK
      testing: mpsym/_mpsym.cpython-36m-darwin.so   OK
      testing: mpsym/.dylibs/libicudata.67.1.dylib   OK
      testing: mpsym/.dylibs/libicuuc.67.1.dylib   OK
      testing: mpsym/.dylibs/liblua.5.3.6.dylib   OK
      testing: mpsym/.dylibs/libicui18n.67.1.dylib   OK
  No errors detected in compressed data of /var/folders

[matthew-brett]

I suspect what is happening is that the run-time stuff expected a symlink between libicuuc.67.1.dylib and libicuuc.67.dylib. Can you make that symlink in .dylibs and retest your installation?

[Time0o]

Adding to symlinks into the wheel archive results in errors like:

 ImportError: dlopen(/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpmw8lg4v4/lib/python3.6/site-packages/mpsym/_mpsym.cpython-36m-darwin.so, 2): Library not loaded: @loader_path/libicuuc.67.dylib
  Referenced from: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpmw8lg4v4/lib/python3.6/site-packages/mpsym/.dylibs/libicui18n.67.1.dylib
  Reason: no suitable image found.  Did find:
	/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpmw8lg4v4/lib/python3.6/site-packages/mpsym/.dylibs/libicuuc.67.dylib: file too short

[matthew-brett]

I guess you could try copying instead.

[HexDecimal]

A lot has changed. Can you test your wheel on the latest version of delocate?