Meson doesn't work with the clang+gfortran combo on arm64 macOS

[rgommers]

An attempted build will immediately fail with:

% meson setup build
The Meson build system
Version: 0.59.99
Source dir: /Users/rgommers/code/bldscipy
Build dir: /Users/rgommers/code/bldscipy/build
Build type: native build
Project name: SciPy
Project version: 1.7.0.dev0+ababababab

meson.build:1:0: ERROR: Unable to detect GNU compiler type:
error: unknown target ABI 'lp64'
error: unknown target ABI 'lp64'

Related issue: conda-forge/gfortran_osx-64-feedstock#11. Trying that "fake cc1" workaround does not make a difference. The issue is that Meson expects that if you're using gfortran, you are also using gcc. It tries to execute gfortran -E -dm - at https://github.com/mesonbuild/meson/blob/61993f893bbdc2415155e28ee70e6ea806725e64/mesonbuild/environment.py#L668-L671. And that just does not result in a valid call to clang. Trying that with verbose output:

% $FC -E -dm - --verbose
Using built-in specs.
COLLECT_GCC=/Users/rgommers/mambaforge/envs/scipy-meson/bin/arm64-apple-darwin20.0.0-gfortran
Target: arm64-apple-darwin20.0.0
Configured with: ../configure --prefix=/Users/rgommers/mambaforge/envs/scipy-meson --build=x86_64-apple-darwin13.4.0 --host=arm64-apple-darwin20.0.0 --target=arm64-apple-darwin20.0.0 --with-libiconv-prefix=/Users/rgommers/mambaforge/envs/scipy-meson --enable-languages=fortran --disable-multilib --enable-checking=release --disable-bootstrap --disable-libssp --with-gmp=/Users/rgommers/mambaforge/envs/scipy-meson --with-mpfr=/Users/rgommers/mambaforge/envs/scipy-meson --with-mpc=/Users/rgommers/mambaforge/envs/scipy-meson --with-isl=/Users/rgommers/mambaforge/envs/scipy-meson
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.0.1 20210403 (experimental) (GCC) 
COLLECT_GCC_OPTIONS='-E' '-dm' '-v' '-mmacosx-version-min=11.4.0' '-asm_macosx_version_min=11.4' '-mlittle-endian' '-mabi=lp64'
 cc1 -E -quiet -v -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -D__DYNAMIC__ - -fPIC -mmacosx-version-min=11.4.0 -mlittle-endian -mabi=lp64 -dm -dumpbase -
clang version 11.1.0
Target: arm64-apple-darwin20.0.0
Thread model: posix
InstalledDir: /Users/rgommers/mambaforge/envs/scipy-meson/bin
clang-11: warning: argument unused during compilation: '-dm' [-Wunused-command-line-argument]
 "/Users/rgommers/mambaforge/envs/scipy-meson/bin/clang-11" -cc1 -triple arm64-apple-macosx11.4.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -fno-rounding-math -munwind-tables -target-sdk-version=11.3 -fcompatibility-qualified-id-block-type-checking -target-cpu apple-a7 -target-feature +fp-armv8 -target-feature +neon -target-feature +crypto -target-feature +zcm -target-feature +zcz -target-feature +sha2 -target-feature +aes -target-abi lp64 -fallow-half-arguments-and-returns -debugger-tuning=lldb -target-linker-version 530 -v -resource-dir /Users/rgommers/mambaforge/envs/scipy-meson/lib/clang/11.1.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -D __DYNAMIC__ -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/rgommers/mambaforge/envs/scipy-meson/lib/clang/11.1.0/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -fdebug-compilation-dir /Users/rgommers/code/bldscipy -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcolor-diagnostics -o - -x c -
error: unknown target ABI 'lp64'
 "/Users/rgommers/mambaforge/envs/scipy-meson/bin/clang-11" -cc1 -triple arm64-apple-macosx11.4.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -fno-rounding-math -munwind-tables -target-sdk-version=11.3 -fcompatibility-qualified-id-block-type-checking -target-cpu apple-a7 -target-feature +fp-armv8 -target-feature +neon -target-feature +crypto -target-feature +zcm -target-feature +zcz -target-feature +sha2 -target-feature +aes -target-abi lp64 -fallow-half-arguments-and-returns -debugger-tuning=lldb -target-linker-version 530 -v -resource-dir /Users/rgommers/mambaforge/envs/scipy-meson/lib/clang/11.1.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -D __DYNAMIC__ -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/rgommers/mambaforge/envs/scipy-meson/lib/clang/11.1.0/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -fdebug-compilation-dir /Users/rgommers/code/bldscipy -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcolor-diagnostics -o - -x c -
error: unknown target ABI 'lp64'

Looks like that needs a fix in Meson.

[matthew-brett]

I'd like to give that go - can look at it tomorrow (Wednesday 27th July). If anyone else would prefer to get a clear shot at it instead - please let me know.

[dcbaker]

that -dm look really odd, it should be -dM, right?

[rgommers]

Yes indeed, that's a typo. It reproduces the error, so I didn't notice. There's no change with -dM, still ends in error: unknown target ABI 'lp64':

% $FC -E -dM - --verbose
Using built-in specs.
COLLECT_GCC=/Users/rgommers/mambaforge/envs/scipy-meson/bin/arm64-apple-darwin20.0.0-gfortran
Target: arm64-apple-darwin20.0.0
Configured with: ../configure --prefix=/Users/rgommers/mambaforge/envs/scipy-meson --build=x86_64-apple-darwin13.4.0 --host=arm64-apple-darwin20.0.0 --target=arm64-apple-darwin20.0.0 --with-libiconv-prefix=/Users/rgommers/mambaforge/envs/scipy-meson --enable-languages=fortran --disable-multilib --enable-checking=release --disable-bootstrap --disable-libssp --with-gmp=/Users/rgommers/mambaforge/envs/scipy-meson --with-mpfr=/Users/rgommers/mambaforge/envs/scipy-meson --with-mpc=/Users/rgommers/mambaforge/envs/scipy-meson --with-isl=/Users/rgommers/mambaforge/envs/scipy-meson
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.0.1 20210403 (experimental) (GCC) 
COLLECT_GCC_OPTIONS='-E' '-dM' '-v' '-mmacosx-version-min=11.4.0' '-asm_macosx_version_min=11.4' '-mlittle-endian' '-mabi=lp64'
 cc1 -E -quiet -v -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -D__DYNAMIC__ - -fPIC -mmacosx-version-min=11.4.0 -mlittle-endian -mabi=lp64 -dM -dumpbase -
clang version 11.1.0
Target: arm64-apple-darwin20.0.0
Thread model: posix
InstalledDir: /Users/rgommers/mambaforge/envs/scipy-meson/bin
 "/Users/rgommers/mambaforge/envs/scipy-meson/bin/clang-11" -cc1 -triple arm64-apple-macosx11.4.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -fno-rounding-math -munwind-tables -target-sdk-version=11.3 -fcompatibility-qualified-id-block-type-checking -target-cpu apple-a7 -target-feature +fp-armv8 -target-feature +neon -target-feature +crypto -target-feature +zcm -target-feature +zcz -target-feature +sha2 -target-feature +aes -target-abi lp64 -fallow-half-arguments-and-returns -debugger-tuning=lldb -target-linker-version 530 -v -resource-dir /Users/rgommers/mambaforge/envs/scipy-meson/lib/clang/11.1.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -D __DYNAMIC__ -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/rgommers/mambaforge/envs/scipy-meson/lib/clang/11.1.0/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -fdebug-compilation-dir /Users/rgommers/code/scipy -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcolor-diagnostics -dM -o - -x c -
error: unknown target ABI 'lp64'
 "/Users/rgommers/mambaforge/envs/scipy-meson/bin/clang-11" -cc1 -triple arm64-apple-macosx11.4.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -fno-rounding-math -munwind-tables -target-sdk-version=11.3 -fcompatibility-qualified-id-block-type-checking -target-cpu apple-a7 -target-feature +fp-armv8 -target-feature +neon -target-feature +crypto -target-feature +zcm -target-feature +zcz -target-feature +sha2 -target-feature +aes -target-abi lp64 -fallow-half-arguments-and-returns -debugger-tuning=lldb -target-linker-version 530 -v -resource-dir /Users/rgommers/mambaforge/envs/scipy-meson/lib/clang/11.1.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -D __DYNAMIC__ -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/rgommers/mambaforge/envs/scipy-meson/lib/clang/11.1.0/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -fdebug-compilation-dir /Users/rgommers/code/scipy -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcolor-diagnostics -dM -o - -x c -
error: unknown target ABI 'lp64'

[dcbaker]

This sure looks like a bug either in gfortran, or the way gfortran is being compiled. this works perfectly fine for me with gfortran and clang on Linux (no gcc installed).

[rgommers]

Was indeed a gfortran issue, fixed in conda-forge now.

There are other issues for macOS that are due to using build flags or options that aren't portable:

[1/1311] Generating _cython_tree with a custom command
FAILED: scipy/__init__.py scipy/linalg.pxd scipy/special.pxd 
/bin/cp ../scipy/__init__.py ../scipy/linalg.pxd ../scipy/special.pxd -t scipy
cp: -t: No such file or directory

[3/1311] Compiling C object scipy/special/libcephes.a.p/cephes_btdtr.c.o
warning: unknown warning option '-Wno-unused-but-set-variable'; did you mean '-Wno-unused-const-variable'? [-Wunknown-warning-option]
1 warning generated.

Those don't really need an open issue, they simply need to be fixed by using more portable code in meson.build. So I'll close this.