More robust identification of the build OS in Microsoft.AspNetCore.App.Runtime.csproj

[omajid]

aspnetcore/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj

Lines 101 to 110 in a0b950f

	<!--
	Determine the crossgen2 package path property name. Special case linux-musl-arm and linux-musl-arm64 because they
	are built on an Ubuntu container with cross compilation tools. linux-musl-x64 is built in an alpine container.
	Special case the crossgen2 package reference on Windows to avoid the x86 package when building in Visual Studio.
	-->
	<BuildOsName>$(TargetOsName)</BuildOsName>
	<BuildOsName Condition="'$(TargetOsName)' == 'linux-musl' and '$(TargetArchitecture)'!='x64'">linux</BuildOsName>
	<Crossgen2BuildArchitecture Condition=" '$(BuildOsName)' == 'win' ">x64</Crossgen2BuildArchitecture>
	<Crossgen2BuildArchitecture Condition=" '$(Crossgen2BuildArchitecture)' == '' ">$(BuildArchitecture)</Crossgen2BuildArchitecture>
	<Crossgen2PackageRootVariableName>PkgMicrosoft_NETCore_App_Crossgen2_$(BuildOsName)-$(Crossgen2BuildArchitecture)</Crossgen2PackageRootVariableName>

This hardcodes some assumptions that are valid in the CI legs, but may not be in other systems.

For example, if this project is being built on Linux arm64 hardware using an OS that uses musl - in other words, linux-musl-arm64, it will try and use the linux-arm64 crossgen2 instead.

Another example, if this code is being built on Windows on arm64 (or really, any architecture that's not x64), it will use the wrong crossgen, win-x64, which will most likely fail to run on an Arm 64 machine.

See also this comment posted by @dougbu in #37374 (comment)

@dougbu

[dougbu]

@davidwrighton @rainersigwald the requirements here are very similar to the requests in dotnet/msbuild#2468 and dotnet/msbuild#5065. We should download the crossgen2 package for the current OS / processor and ignore the requested $(TargetArchitecture) in that logic.

$([MSBuild]::IsOSPlatform('FreeBSD')), $([MSBuild]::IsOSPlatform('Linux')), $([MSBuild]::IsOSPlatform('OSX')), and $([MSBuild]::IsOSPlatform('Windows')) would get us partway there.

Differentiating between arm, arm64, x64 and x86 on Windows probably just means checking $(PROCESSOR_ARCHITECTURE). May be a bit more complicated on Linux and OSX because there are a few different uname -m values for similar-enough architectures.

The special case for using x64 on Windows should be used only when $(PROCESSOR_ARCHITECTURE) is X86 and $(BuildingInsideVisualStudio) is true.

Differentiating between Linux MUSL and generic Linux probably requires

1. checking whether /lib/ld-musl-x86_64.so.1 exists (it does on Alpine, so why aren't we using a linux-musl package there❔),
2. <Exec />ing apk info musl and checking for errors, or
3. <Exec />ing ldd /bin/ls and greping for musl.

Which do you prefer❔

I'm probably missing a few things and would appreciate your guidance❕

[omajid]

<Exec />ing ldd /bin/ls and greping for musl.

FWIW, the approach runtime takes is: ldd --version 2>&1 | grep -q musl. That's the approach used in dotnet/installer#13410

[omajid]

I have been trying to build main branch on Alpine x64 (musl), and running into issues. This change to eng/build.sh let me get started:

diff --git a/eng/build.sh b/eng/build.sh
index a6b6ba99be..9a4bc957c8 100755
--- a/eng/build.sh
+++ b/eng/build.sh
@@ -37,7 +37,11 @@ runtime_source_feed_key=''
 if [ "$(uname)" = "Darwin" ]; then
     target_os_name='osx'
 else
-    target_os_name='linux'
+    if (ldd --version 2>&1 || true) | grep -q musl; then
+        target_os_name='linux-musl'
+    else
+        target_os_name='linux'
+    fi
 fi
 
 msbuild_args=()

But I am running into errors because Grpc.Tools isn't available for linux-musl-x64: grpc/grpc#18428, grpc/grpc#24188.

/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003: The specified task executable "/home/dotnet/.nuget/packages/grpc.tools/2.43.0/tools/linux_x64/protoc" could not be run. System.ComponentModel.Win32Exception (2): An error occurred trying to start process '/home/dotnet/.nuget/packages/grpc.tools/2.43.0/tools/linux_x64/protoc' with working directory '/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite'. No such file or directory [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj]
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj]  

@ayakael did you run into this? How did you work around it?

[ayakael]

I have been trying to build main branch on Alpine x64 (musl), and running into issues. This change to eng/build.sh let me get started:

diff --git a/eng/build.sh b/eng/build.sh
index a6b6ba99be..9a4bc957c8 100755
--- a/eng/build.sh
+++ b/eng/build.sh
@@ -37,7 +37,11 @@ runtime_source_feed_key=''
 if [ "$(uname)" = "Darwin" ]; then
     target_os_name='osx'
 else
-    target_os_name='linux'
+    if (ldd --version 2>&1 || true) | grep -q musl; then
+        target_os_name='linux-musl'
+    else
+        target_os_name='linux'
+    fi
 fi
 
 msbuild_args=()

But I am running into errors because Grpc.Tools isn't available for linux-musl-x64: grpc/grpc#18428, grpc/grpc#24188.

/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003: The specified task executable "/home/dotnet/.nuget/packages/grpc.tools/2.43.0/tools/linux_x64/protoc" could not be run. System.ComponentModel.Win32Exception (2): An error occurred trying to start process '/home/dotnet/.nuget/packages/grpc.tools/2.43.0/tools/linux_x64/protoc' with working directory '/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite'. No such file or directory [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj]
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj]  

@ayakael did you run into this? How did you work around it?

I have not encoutnered this issue, investigating why you have it but source-build aspnetcore doesn't

[ayakael]

@omajid Oh right, I go around this issue by building aspnetcore with --os-name=linux-musl. Thus, target_os_name is overriden. Aspnetcore should be able to detect linux-musl without source-build doing this check. See https://github.com/dotnet/installer/pull/13410/files#diff-1773f0861dc9197a3da55e87fbac6ca93eadfc48389e198085b1457422895b92