3.12.1 fails to link `liblapacke`: `error: posix_spawnp: Argument list too long`

[barracuda156]

Description

I cannot build lapack 3.12.1 now, linking fails at the very end of the build for liblapacke with the error:

collect2: fatal error: posix_spawnp: Argument list too long

How could this be solved?

[martin-frbg]

Looks like you are hitting the ARG_MAX limit on your system, you could try adding -DCMAKE_Fortran_USE_RESPONSE_FILE_FOR_OBJECTS=1 to your build flags (if you are building with cmake, and if this is not already getting set somewhere in the build files).

[barracuda156]

@martin-frbg Thank you for responding!

I have tried now, and it does not seem to help (I passed it to CMake, and CMakeCache had it recorded). However, I found out that the problem occurs only when building extended index64, and without that the build succeeds.

So perhaps I need to ask first, does it make sense to build it on 32-bit platform? If yes, then we still need a solution here. I can try anything suggested.

P. S. Also, 3.12.0 has built fine from the default portfile without any manual tweaks, did something changed with regard to index64 in between?

@Dave-Allured FYI

[Dave-Allured]

@barracuda156 Is your build using full absolute paths for each object in the command line that fails? I could suggest a couple of ways to fix this.

If not, then can you please excerpt that part of your build log where this error happens? Post as file attachment. It may be a mess if posted in-line.

[barracuda156]

@Dave-Allured I just tried to update the existing portfile to the new version. I did not initially modify anything beyond version and checksums.

[Dave-Allured]

@Dave-Allured I just tried to update the existing portfile to the new version. I did not initially modify anything beyond version and checksums.

Okay. From release notes for 3.12.1, I see a lot of changes. I have not analyzed them. But if some new functions were added, that might increase the length of a library packing command line that was already extreme, pushing it over a length threshold. This is for MacPorts, right? Macports builds make this problem worse because they add that ridiculous build prefix to every object file on the command line. It is already bad enough when some build systems want to list every object file on a single library packing command line.

And yes, when extended index64 is enabled, that could also push the library pack command over the limit, by adding more objects on the same line. It depends how they are packaging their libraries, with which I am not familiar.

I may be getting ahead of myself. Can you please confirm the nature of the command line that errors? Are they putting a full absolute path on every object file in the command? Can you estimate the line length or number of objects on the same line?

[Dave-Allured]

@barracuda156 Please be aware that I just naively pushed this 3.12.1 update for MacPorts. It seems to be building fine on CI. Let's let that run through, then see if more fixes may be needed. Your feedback will be most welcome.

[barracuda156]

@Dave-Allured I would assume that the very latest macOS on 64-bit are less constrained than 10.6 on 32-bit in this context. Possibly i386 buildbot will fail, though it is not identical still, since it gonna use clang etc.

Let me re-try the build once the update is merged though, so that at least my paths are exactly identical (when I use an overlay repo, they are not).

[Dave-Allured]

Okay. Meanwhile, here is the report and fix for a similar problem that we had with GDAL on Macports. I think the same thing is going on here with lapack on older OS with smaller ARG_MAX.

OSGeo/gdal#2075

[barracuda156]

@Dave-Allured I ran the build from the default portfile, and it failed with the same issue. I have attached the log here: https://trac.macports.org/ticket/72025 (zipped, it is huge).

P. S. By default gcc-4.2 is used for non-Fortran code (which is reflected in the log), but I have tried with gcc14 earlier, it still fails with Argument list too long, so gcc version is not the problem here.