Fatal error: 'elf.h' file not found on macOS 10.15

[asbjornu]

In Homebrew/homebrew-core#60929 I've got dotnet to build for macOS 10.13 and 10.14, but on 10.15 the build fails with the following error:

/tmp/dotnet-20200912-76027-168kcki/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5/src/pal/src/exception/remote-unwind.cpp:68:10: fatal error: 'elf.h' file not found
#include <elf.h>
         ^~~~~~~

The end result reports something completely unrelated, but I suspect it's the missing elf.h file causing problem and not this:

  make: *** [all] Error 2
  Failed to build CoreCLR component.
  
  'coreclr' failed during build.
  See '/private/tmp/dotnet-20200912-4021-l77t2w/artifacts/logs/coreclr.log' for more information.
  'corefx' failed during build.
  'msbuild' failed during build.
  'known-good' failed during build.

Build FAILED.

/private/tmp/dotnet-20200912-4021-l77t2w/packages/restored/microsoft.dotnet.arcade.sdk/1.0.0-beta.19359.6/tools/ProjectLayout.props(8,3): warning MSB4011: "/private/tmp/dotnet-20200912-4021-l77t2w/packages/restored/microsoft.dotnet.arcade.sdk/1.0.0-beta.19359.6/tools/RepoLayout.props" cannot be imported again. It was already imported at "/private/tmp/dotnet-20200912-4021-l77t2w/packages/restored/microsoft.dotnet.arcade.sdk/1.0.0-beta.19359.6/tools/Build.proj (52,3)". This is most likely a build authoring error. This subsequent import will be ignored. 
/private/tmp/dotnet-20200912-4021-l77t2w/repos/Directory.Build.targets(296,5): error MSB3073: The command "/private/tmp/dotnet-20200912-4021-l77t2w/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5//build.sh x64 Release skiptests -ignoreWarnings -skipmanagedtools -skiprestore -nopgooptimize msbuildonunsupportedplatform cmakeargs -DCLR_CMAKE_USE_SYSTEM_LIBUNWIND=TRUE /bl /p:ILLinkTrimAssembly=false /p:CheckCDefines=false /p:PackagesDir=/private/tmp/dotnet-20200912-4021-l77t2w/packages/restored/ /p:ContinuousIntegrationBuild=true /p:VersionSuffix="servicing" -warnAsError:0 /v:minimal  /p:DotNetPackageVersionPropsPath=/private/tmp/dotnet-20200912-4021-l77t2w/artifacts/obj/x64/Release/PackageVersions.props /p:DotNetRestoreSourcePropsPath=/private/tmp/dotnet-20200912-4021-l77t2w/artifacts/obj/x64/Release/RestoreSources.props >> /private/tmp/dotnet-20200912-4021-l77t2w/artifacts/logs/coreclr.log 2>&1" exited with code 1. [/private/tmp/dotnet-20200912-4021-l77t2w/repos/coreclr.proj]
    1 Warning(s)
    1 Error(s)

Here's the coreclr.log file for inspection.

I don't have any good ideas how to resolve this. Since Homebrew's libelf formula does not seem to bundle elf.h, taking a dependency on libelf won't help. There's also no easy way to make dependencies conditional on the macOS version afaik.

As suggested in the same SO thread, there's a possibility to source elf.h from Apple's dtrace, but even better would be if elf.h was provided out of the box from source-build itself somehow.

[dagood]

Are you able to build dotnet/coreclr on macOS 10.15 at all? This issue should probably be raised in dotnet/runtime (since dotnet/coreclr was consolidated there), ideally without source-build infra involved so there aren't as many variables to consider.

[wfurt]

I do build runtime on 10.15 without problems. I did not try source build for a while.

[asbjornu]

I don't have a machine with 10.15 available, so I have to resort to GitHub Actions and the help from others (such as Homebrew/homebrew-core#60929 (comment)) and from everything I've seen, 10.15 fails consistently. I assume you've got elf.h installed somehow, @wfurt?

[dagood]

@asbjornu I wonder, does Homebrew not require Xcode to be installed, and would that be where it normally comes from?

I'm not particularly familiar with the Homebrew project, but my impression is that it builds an OSS ecosystem on top of macOS and other OSes, so perhaps avoiding using Xcode is something they're interested in.

dotnet/runtime (and dotnet/coreclr) list Xcode as a requirement: https://github.com/dotnet/runtime/blob/master/docs/workflow/requirements/macos-requirements.md

I'm sure this can be patched to work with different dependencies, but I have no idea how extensive a patch would be or if it would be reasonable to try to maintain it in the dotnet org repos.

[wfurt]

I did not find elf.h on my system, besides src/coreclr/src/pal/src/libunwind_mac/include/elf.h - but I don't think that is the one as added recently. And I realized I don't build old coreclr repo but runtime. The code looks similar but there may be some caveats.

cc: @janvorli as expert in this area.

[janvorli]

I have upgraded my macOS to the latest version last week (10.15.6) and also the xcode to the latest one (11.7 IIRC). I can still build the runtime repo without issues. Let me check where the elf.h comes from.

[dagood]

The exact commit being built above is https://github.com/dotnet/coreclr/tree/v3.1.8, FYI in case this is different in runtime/5.0.

[janvorli]

I was able to build the coreclr repo release/3.1 just fine too.

[MikeMcQuaid]

mdfind -name elf.h may help you figure out where it comes from.

[janvorli]

So the build doesn't include the elf.h on my mac at all. Neither in coreclr nor in runtime repos. Using build.sh in coreclr and src/coreclr/build-runtime.sh in the runtime repo with no arguments. And it is not supposed to include it. I wonder if the problem somehow arises from the non-apple libunwind being installed and used by accident. It looks like in the runtime repo, it would cause the HAVE_UNW_GET_ACCESSORS to be detected. This is never detected on the apple version of libunwind as it is missing the accessor functions. And looking at the coreclr.log above, it is really the case, the HAVE_UNW_GET_ACCESSORS is true:

-- Performing Test HAVE_UNW_GET_ACCESSORS - Success

I can see that the build is being passed -DCLR_CMAKE_USE_SYSTEM_LIBUNWIND=TRUE, so it seems that somehow a non-apple libunwind.h is found on the include paths before the apple one.

You can verify which libunwind.h the build is including e.g. this way (assuming you've ran the build.sh for debug configuration before, even though it has failed):

cd  bin/obj/OSX.x64.Debug/src/pal/src/
make exception/remote-unwind.cpp.i

Now you can open ./CMakeFiles/coreclrpal.dir/exception/remote-unwind.cpp.i and search for libunwind.h in it. In my case, it is
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/libunwind.h.

[SMillerDev]

I just checked inside and outside the homebrew sandbox. It works outside of it, but not inside. In the sandbox I get the following failure if I try and run make exception/remote-unwind.cpp.i @MikeMcQuaid any ideas?

Preprocessing CXX source to CMakeFiles/coreclrpal.dir/exception/remote-unwind.cpp.i
/tmp/dotnet-20200914-75099-1je1col/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5/src/pal/src/exception/remote-unwind.cpp:68:10: fatal error:
      'elf.h' file not found
#include <elf.h>
         ^~~~~~~
1 error generated.
make[1]: *** [src/pal/src/CMakeFiles/coreclrpal.dir/exception/remote-unwind.cpp.i] Error 1
make[1]: *** Deleting file `src/pal/src/CMakeFiles/coreclrpal.dir/exception/remote-unwind.cpp.i'
make: *** [exception/remote-unwind.cpp.i] Error 2

[janvorli]

@SMillerDev just delete the line that includes elf.h when generating the preprocessing file. Then the preprocessing should succeed and you should be able to see the path to the libunwind.h.

[SMillerDev]

Thanks, it's /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/libunwind.h within homebrew. Could it be that the check is based on the filepath?