Skip to content

Commit 71db8b2

Browse files
jkoritzinskyViktorHofer
jkoritzinsky
and
ViktorHofer
authored
Build the "in build" crossgen2 into a pack that we can use within the VMR (#110676)
* Add a new crossgen2 build variant for a published crossgen2 targetting the host OS * Remove the .NET-hosted in-build crossgen2 and just use the published in-build host-targeting crossgen2. * Extract crossgen2 sfxproj changes from #109710 * Extract the properties and targets that will be shared into a separate props file. * Add a "host crossgen2" project * Add the crossgen2 host package to Subsets.props for the correct scenarios. * Fix crossgen-corelib to work with the new in-build crossgen2 * Publish the in-build crossgen2 as single file to avoid having to adjust the AOT-in-build logic to handle AOT-ing for the host. * AOT on Windows only * Don't set Platform for the shared framework projects. It's not necessary and it breaks the new crossgen2 host package * Forward ShortStack down so we don't build the host crossgen2 when runtime is the last repo. * Add os-only RID to portable RID graph in source-build to enable crossgen2 tasks in the SDK to discover the target OS from the source-build target RID * Publish the host crossgen2 as a "Vertical" visibility asset. * Cleanup crossgen2 infra, add comment about workaround, and adjust publishing to publish packages as packages. * Deduplicate artifacts * Add back the Update, somehow this got lost. * Don't forward down ShortStack * Use new "update" functionality to specify which artifacts are vertical visibility * Revert changes to Signing.props * Remove workaround now that the VMR has a bootstrap SDK with a new enough version. --------- Co-authored-by: Viktor Hofer <[email protected]>
1 parent 0df31bc commit 71db8b2

12 files changed

+106
-108
lines changed

‎eng/Publishing.props

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,15 @@
88
<PlatformName Condition="'$(TargetArchitecture)' != ''">$(TargetArchitecture)</PlatformName>
99
</PropertyGroup>
1010

11+
<!--
12+
Mark assets as Vertical visibility when building in the VMR
13+
-->
14+
<ItemGroup Condition="'$(DotNetBuildOrchestrator)' == 'true'">
15+
<Artifact Condition="'$(RuntimeFlavor)' != 'Mono' and '$(OutputRID)' != '$(NETCoreSdkRuntimeIdentifier)'"
16+
Update="$(ArtifactsPackagesDir)**\Microsoft.NETCore.App.Crossgen2.$(NETCoreSdkRuntimeIdentifier).*.nupkg"
17+
Visibility="Vertical" />
18+
</ItemGroup>
19+
1120
<!--
1221
Look through the downloaded artifacts to figure out how to ship them. Creates item groups for
1322
other types of publishing to use.
@@ -203,4 +212,4 @@
203212
<Error Text="Duplicate RuntimeNupkgFile entries for: %(RuntimeNupkgFile.Identity)" Condition="@(RuntimeNupkgFile->Count()) &gt; 1" />
204213
</Target>
205214

206-
</Project>
215+
</Project>

‎eng/Subsets.props

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -547,6 +547,13 @@
547547
<ItemGroup>
548548
<ProjectToBuild Condition="'$(NativeAotSupported)' == 'true' and '$(RuntimeFlavor)' != 'Mono' and '$(BuildOnlyPgoInstrumentedAssets)' != 'true'" Include="$(InstallerProjectRoot)\pkg\projects\nativeaot-packages.proj" Category="packs" />
549549
</ItemGroup>
550+
<ItemGroup>
551+
<!--
552+
When we're building in the VMR, we need to provide a crossgen2 that runs on the host machine for downstream repos to use to R2R their code.
553+
In non-VMR builds, downstream repos can use the crossgen2 built for the target host SDK from another build leg, but in the VMR we need to provide one to use.
554+
-->
555+
<ProjectToBuild Condition="'$(RuntimeFlavor)' != 'Mono' and '$(DotNetBuildOrchestrator)' == 'true' and '$(OutputRID)' != '$(NETCoreSdkRuntimeIdentifier)'" Include="$(InstallerProjectRoot)pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Crossgen2.Host.sfxproj" Category="packs" />
556+
</ItemGroup>
550557
<ItemGroup>
551558
<SharedFrameworkProjectToBuild Condition="'$(BuildMonoAOTCrossCompilerOnly)' != 'true'" Include="$(InstallerProjectRoot)pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj" />
552559
<SharedFrameworkProjectToBuild Condition="'$(BuildNativeAOTRuntimePack)' != 'true' and '$(RuntimeFlavor)' == '$(PrimaryRuntimeFlavor)' and '$(TargetsMobile)' != 'true'" Include="$(InstallerProjectRoot)pkg\sfx\bundle\Microsoft.NETCore.App.Bundle.bundleproj" />

‎src/coreclr/crossgen-corelib.proj

Lines changed: 4 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
<Project Sdk="Microsoft.Build.NoTargets">
22

33
<ItemGroup>
4-
<ProjectReference Include="$(CoreClrProjectRoot)/tools/aot/crossgen2/crossgen2_inbuild.csproj" OutputItemType="Crossgen2" />
4+
<ProjectReference Include="$(CoreClrProjectRoot)/tools/aot/crossgen2/crossgen2_inbuild.csproj" OutputItemType="Crossgen2Files" />
55
<ProjectReference Include="$(CoreClrProjectRoot)/tools/dotnet-pgo/dotnet-pgo.csproj" OutputItemType="DotNetPgo" Condition="'$(DotNetBuildSourceOnly)' != 'true'" />
66
<ProjectReference Include="$([MSBuild]::NormalizePath('$(CoreClrProjectRoot)', 'System.Private.CoreLib', 'System.Private.CoreLib.csproj'))" OutputItemType="CoreLib" />
77
</ItemGroup>
@@ -92,21 +92,13 @@
9292
<ItemGroup>
9393
<Crossgen2Inputs Include="@(CoreLib)" />
9494
<Crossgen2Inputs Include="$(MergedMibcPath)" />
95-
</ItemGroup>
96-
97-
<PropertyGroup>
98-
<Crossgen2OutputPath>%(Crossgen2.RootDir)/%(Crossgen2.Directory)</Crossgen2OutputPath>
99-
</PropertyGroup>
100-
<ItemGroup>
101-
<Crossgen2Files Include="$(Crossgen2OutputPath)/*.dll" />
102-
<Crossgen2Files Include="$(Crossgen2OutputPath)/*.so" />
103-
<Crossgen2Files Include="$(Crossgen2OutputPath)/*.dylib" />
95+
<Crossgen2Inputs Include="@(Crossgen2Files->Metadata('OutputPath'))" />
10496
</ItemGroup>
10597
</Target>
10698

10799
<Target Name="InvokeCrossgen"
108100
DependsOnTargets="PrepareInvokeCrossgen;CreateMergedMibcFile"
109-
Inputs="@(Crossgen2Inputs);@(Crossgen2Files)"
101+
Inputs="@(Crossgen2Inputs)"
110102
Outputs="$(CoreLibOutputPath);$(CoreLibNiPdbPath);$(CoreLibPerfMapPath)"
111103
AfterTargets="Build">
112104

@@ -117,7 +109,7 @@
117109
Text="Generating native image of System.Private.CoreLib for $(OSPlatformConfig). Logging to $(CrossGenCoreLibLog)" />
118110

119111
<PropertyGroup>
120-
<CrossGenDllCmd>$(DotNetCli) @(Crossgen2)</CrossGenDllCmd>
112+
<CrossGenDllCmd>@(Crossgen2Files->Metadata('OutputPath')->WithMetadataValue('Filename','crossgen2')->WithMetadataValue('Extension','$(ExeSuffix)'))</CrossGenDllCmd>
121113
<CrossGenDllCmd>$(CrossGenDllCmd) -o:$(CoreLibOutputPath)</CrossGenDllCmd>
122114
<CrossGenDllCmd>$(CrossGenDllCmd) -r:$([MSBuild]::NormalizePath('$(BinDir)', 'IL', '*.dll'))</CrossGenDllCmd>
123115
<CrossGenDllCmd>$(CrossGenDllCmd) --targetarch:$(TargetArchitecture)</CrossGenDllCmd>

‎src/coreclr/tools/aot/crossgen2/crossgen2_inbuild.csproj

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,20 @@
1-
<Project Sdk="Microsoft.NET.Sdk">
1+
<Project Sdk="Microsoft.NET.Sdk" DefaultTargets="Publish;PublishItemsOutputGroup">
22
<PropertyGroup>
33
<CrossHostArch Condition="'$(CrossBuild)' == 'true' or '$(TargetArchitecture)' != '$(BuildArchitecture)' or '$(HostOS)' != '$(TargetOS)' or '$(EnableNativeSanitizers)' != ''">$(BuildArchitecture)</CrossHostArch>
4-
<OutputPath>$(RuntimeBinDir)/$(BuildArchitecture)/crossgen2/</OutputPath>
5-
<TargetFramework>$(NetCoreAppToolCurrent)</TargetFramework>
6-
<UseAppHost>false</UseAppHost>
4+
<_IsPublishing>true</_IsPublishing>
5+
<RuntimeIdentifier>$(NETCoreSdkRuntimeIdentifier)</RuntimeIdentifier>
6+
<PublishDir>$(RuntimeBinDir)$(BuildArchitecture)/crossgen2/</PublishDir>
7+
<SelfContained>true</SelfContained>
8+
<PublishTrimmed>true</PublishTrimmed>
9+
<!--
10+
Publish with AOT on Windows as Single file doesn't work well when an app has to include DiaSymReader.Native.
11+
Publish single-file elsewhere so we don't need to adjust our "AOT in build" logic to handle targeting the SDK RID.
12+
-->
13+
<PublishAot Condition="$(NETCoreSdkPortableRuntimeIdentifier.StartsWith('win-'))">true</PublishAot>
14+
<PublishSingleFile Condition="'$(PublishAot)' != 'true'">true</PublishSingleFile>
15+
<PublishReadyToRun>true</PublishReadyToRun>
16+
<PublishReadyToRunComposite>true</PublishReadyToRunComposite>
17+
<TrimmerSingleWarn>false</TrimmerSingleWarn>
718
</PropertyGroup>
819
<Import Project="crossgen2.props" />
920
</Project>

‎src/installer/pkg/Directory.Build.props

Lines changed: 0 additions & 13 deletions
This file was deleted.

‎src/installer/pkg/Directory.Build.targets

Lines changed: 0 additions & 19 deletions
This file was deleted.

‎src/installer/pkg/projects/Directory.Build.props

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
<Import Project="$([MSBuild]::GetPathOfFileAbove(Directory.Build.props, $(MSBuildThisFileDirectory)..))" />
33

44
<PropertyGroup>
5+
<Platform>$(TargetArchitecture)</Platform>
56
<PackagePlatform>AnyCPU</PackagePlatform>
67

78
<SkipPackageFileCheck>true</SkipPackageFileCheck>

‎src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.Host.sfxproj

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
<Sdk Name="Microsoft.DotNet.SharedFramework.Sdk" />
3+
4+
<PropertyGroup>
5+
<CrossgenPublishProject>$(RepoRoot)src/coreclr/tools/aot/crossgen2/crossgen2_inbuild.csproj</CrossgenPublishProject>
6+
<RuntimeIdentifier>$(NETCoreSdkRuntimeIdentifier)</RuntimeIdentifier>
7+
<IsShipping>false</IsShipping>
8+
</PropertyGroup>
9+
10+
<Import Project="$(MSBuildThisFileDirectory)Microsoft.NETCore.App.Crossgen2.props" />
11+
</Project>

‎src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.props

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
<Project>
2+
<PropertyGroup>
3+
<!-- Crossgen is not used for Mono -->
4+
<SkipBuild Condition="'$(RuntimeFlavor)' == 'Mono'">true</SkipBuild>
5+
<PlatformPackageType>ToolPack</PlatformPackageType>
6+
<SharedFrameworkName>$(SharedFrameworkName).Crossgen2</SharedFrameworkName>
7+
<OverridePackageId>$(SharedFrameworkName).$(RuntimeIdentifier)</OverridePackageId>
8+
<ArchiveName>dotnet-crossgen2</ArchiveName>
9+
<SharedFrameworkHostFileNameOverride>crossgen2</SharedFrameworkHostFileNameOverride>
10+
<GenerateInstallers>false</GenerateInstallers>
11+
<PermitDllAndExeFilesLackingFileVersion>true</PermitDllAndExeFilesLackingFileVersion>
12+
<!-- Publishing as single-file or NativeAOT means we can't examine the interior DLLs -->
13+
<ShouldVerifyClosure>false</ShouldVerifyClosure>
14+
</PropertyGroup>
15+
16+
<ItemGroup>
17+
<ProjectReference
18+
Include="$(CrossgenPublishProject)"
19+
OutputItemType="_RawCrossgenPublishFiles"
20+
ReferenceOutputAssembly="false" />
21+
</ItemGroup>
22+
23+
<Target Name="PublishCrossgen"
24+
BeforeTargets="GetFilesToPackage">
25+
<ItemGroup>
26+
<_CrossgenPublishFiles Include="@(_RawCrossgenPublishFiles->'%(OutputPath)')"
27+
KeepMetadata="REMOVE_ALL" />
28+
</ItemGroup>
29+
<ItemGroup Condition="'$(NativeAotSupported)' != 'true'">
30+
<FilesToPackage Include="@(_CrossgenPublishFiles)"
31+
Exclude="*.pdb;*.h;*.lib"
32+
TargetPath="tools/" />
33+
</ItemGroup>
34+
<ItemGroup Condition="'$(NativeAotSupported)' == 'true'">
35+
<!-- Treat all native aot assets as native runtime assets -->
36+
<FilesToPackage Include="@(_CrossgenPublishFiles->Distinct())"
37+
Condition="'%(Extension)' != '.pdb'"
38+
TargetPath="tools/" />
39+
</ItemGroup>
40+
</Target>
41+
42+
<Target Name="AddCrossgen2SymbolFilesToPackage" BeforeTargets="GetFilesToPackage" DependsOnTargets="PublishCrossgen">
43+
<ItemGroup>
44+
<_Crossgen2SymbolFilesToPackage Include="@(_CrossgenPublishFiles)" Condition="'%(Extension)' == '.pdb'" />
45+
<!-- Symbol files for JIT libraries are placed in a different location for Windows builds -->
46+
<_Crossgen2SymbolFilesToPackage Include="@(NativeRuntimeAsset->'$(CoreCLRArtifactsPdbDir)%(FileName).pdb')" Condition="'$(TargetOS)' == 'windows' and '%(FileName)' != 'crossgen2'" />
47+
<_Crossgen2SymbolFilesToPackage Include="@(NativeRuntimeAsset->'$(CoreCLRArtifactsPath)%(FileName)%(Extension)$(SymbolsSuffix)')" Condition="'$(TargetOS)' != 'windows' and '%(FileName)' != 'crossgen2'" />
48+
<_Crossgen2SymbolFilesToPackage Remove="@(_Crossgen2SymbolFilesToPackage)" Condition="!Exists('%(Identity)')" />
49+
50+
<_SymbolFilesToPackage Include="@(_Crossgen2SymbolFilesToPackage)" TargetPath="tools/" />
51+
</ItemGroup>
52+
</Target>
53+
</Project>

‎src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj

Lines changed: 2 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -2,48 +2,14 @@
22
<Sdk Name="Microsoft.DotNet.SharedFramework.Sdk" />
33

44
<PropertyGroup>
5-
<!-- Crossgen is not used for Mono -->
6-
<SkipBuild Condition="'$(RuntimeFlavor)' == 'Mono'">true</SkipBuild>
7-
<PlatformPackageType>ToolPack</PlatformPackageType>
8-
<SharedFrameworkName>$(SharedFrameworkName).Crossgen2</SharedFrameworkName>
9-
<OverridePackageId>$(SharedFrameworkName).$(RuntimeIdentifier)</OverridePackageId>
10-
<ArchiveName>dotnet-crossgen2</ArchiveName>
11-
<SharedFrameworkHostFileNameOverride>crossgen2</SharedFrameworkHostFileNameOverride>
12-
<GenerateInstallers>false</GenerateInstallers>
13-
<PermitDllAndExeFilesLackingFileVersion>true</PermitDllAndExeFilesLackingFileVersion>
14-
<!-- Publishing as single-file or NativeAOT means we can't examine the interior DLLs -->
15-
<ShouldVerifyClosure>false</ShouldVerifyClosure>
5+
<CrossgenPublishProject>$(RepoRoot)src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj</CrossgenPublishProject>
166
</PropertyGroup>
177

188
<ItemGroup>
199
<PackageReference Include="Microsoft.DotNet.Build.Tasks.Archives" Version="$(MicrosoftDotNetBuildTasksArchivesVersion)" />
2010
</ItemGroup>
2111

22-
<ItemGroup>
23-
<ProjectReference
24-
Include="$(RepoRoot)src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj"
25-
OutputItemType="_RawCrossgenPublishFiles"
26-
ReferenceOutputAssembly="false" />
27-
</ItemGroup>
28-
29-
<Target Name="PublishCrossgen"
30-
BeforeTargets="GetFilesToPackage">
31-
<ItemGroup>
32-
<_CrossgenPublishFiles Include="@(_RawCrossgenPublishFiles->'%(OutputPath)')"
33-
KeepMetadata="REMOVE_ALL" />
34-
</ItemGroup>
35-
<ItemGroup Condition="'$(NativeAotSupported)' != 'true'">
36-
<FilesToPackage Include="@(_CrossgenPublishFiles)"
37-
Exclude="*.pdb;*.h;*.lib"
38-
TargetPath="tools/" />
39-
</ItemGroup>
40-
<ItemGroup Condition="'$(NativeAotSupported)' == 'true'">
41-
<!-- Treat all native aot assets as native runtime assets -->
42-
<FilesToPackage Include="@(_CrossgenPublishFiles->Distinct())"
43-
Condition="'%(Extension)' != '.pdb'"
44-
TargetPath="tools/" />
45-
</ItemGroup>
46-
</Target>
12+
<Import Project="$(MSBuildThisFileDirectory)Microsoft.NETCore.App.Crossgen2.props" />
4713

4814
<Target Name="RunPublishedCrossgen" AfterTargets="PublishCrossgen"
4915
Condition="'$(TargetOS)' == '$(HostOS)' and '$(TargetArchitecture)' == '$(BuildArchitecture)' and '$(CrossBuild)' != 'true'">
@@ -52,22 +18,4 @@
5218
<Output TaskParameter="ExitCode" PropertyName="CrossgenExitCode" />
5319
</Exec>
5420
</Target>
55-
56-
<PropertyGroup>
57-
<TargetOSComponent>unix</TargetOSComponent>
58-
<TargetOSComponent Condition="'$(TargetOS)' == 'windows'">win</TargetOSComponent>
59-
<TargetSpec>$(TargetOSComponent)-$(TargetArchitecture)</TargetSpec>
60-
</PropertyGroup>
61-
62-
<Target Name="AddCrossgen2SymbolFilesToPackage" BeforeTargets="GetFilesToPackage" DependsOnTargets="PublishCrossgen">
63-
<ItemGroup>
64-
<_Crossgen2SymbolFilesToPackage Include="@(_CrossgenPublishFiles)" Condition="'%(Extension)' == '.pdb'" />
65-
<!-- Symbol files for JIT libraries are placed in a different location for Windows builds -->
66-
<_Crossgen2SymbolFilesToPackage Include="@(NativeRuntimeAsset->'$(CoreCLRArtifactsPdbDir)%(FileName).pdb')" Condition="'$(TargetOS)' == 'windows' and '%(FileName)' != 'crossgen2'" />
67-
<_Crossgen2SymbolFilesToPackage Include="@(NativeRuntimeAsset->'$(CoreCLRArtifactsPath)%(FileName)%(Extension)$(SymbolsSuffix)')" Condition="'$(TargetOS)' != 'windows' and '%(FileName)' != 'crossgen2'" />
68-
<_Crossgen2SymbolFilesToPackage Remove="@(_Crossgen2SymbolFilesToPackage)" Condition="!Exists('%(Identity)')" />
69-
70-
<_SymbolFilesToPackage Include="@(_Crossgen2SymbolFilesToPackage)" TargetPath="tools/" />
71-
</ItemGroup>
72-
</Target>
7321
</Project>

‎src/installer/pkg/sfx/Microsoft.NETCore.App/ReadyToRun.targets

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,14 @@
22
<Target Name="ResolveReadyToRunCompilers" DependsOnTargets="ResolveRuntimeFilesFromLocalBuild">
33
<PropertyGroup>
44
<PublishReadyToRunPerfmapFormatVersion Condition="'$(PublishReadyToRunPerfmapFormatVersion)' == ''">1</PublishReadyToRunPerfmapFormatVersion>
5-
<Crossgen2Path>$([MSBuild]::NormalizePath('$(Crossgen2InBuildDir)', 'crossgen2.dll'))</Crossgen2Path>
6-
<R2RDotnetHostPath>$(DotNetTool)</R2RDotnetHostPath>
5+
<Crossgen2Path>$([MSBuild]::NormalizePath('$(Crossgen2InBuildDir)', 'crossgen2$(ExeSuffix)'))</Crossgen2Path>
76
</PropertyGroup>
87

98
<ItemGroup>
109
<Crossgen2Tool Include="$(Crossgen2Path)"
1110
TargetArch="$(TargetArchitecture)"
1211
TargetOS="$(TargetOS)"
13-
DotnetHostPath="$(R2RDotnetHostPath)"
1412
PerfmapFormatVersion="$(PublishReadyToRunPerfmapFormatVersion)"/>
1513
</ItemGroup>
1614
</Target>
17-
</Project>
15+
</Project>

‎src/installer/pkg/sfx/installers/dotnet-host.proj

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020
<MacOSScriptsTemplateDirectory>osx_scripts/host</MacOSScriptsTemplateDirectory>
2121
<MacOSPackageDescription>The .NET Shared Host.</MacOSPackageDescription>
2222
<!-- Enables stable upgrade code - do not change -->
23-
<MsiUpgradeCodeSeed>dotnet-host $(MajorVersion).$(MinorVersion) $(Platform)</MsiUpgradeCodeSeed>
23+
<MsiUpgradeCodeSeed>dotnet-host $(MajorVersion).$(MinorVersion) $(TargetArchitecture)</MsiUpgradeCodeSeed>
2424
</PropertyGroup>
2525

2626
<ItemGroup>

0 commit comments

Comments
 (0)