Skip to content

Commit 86d5d16

Browse files
jkoritzinskyViktorHofer
jkoritzinsky
and
ViktorHofer
authored
Switch to new Microsoft.DotNet.SharedFramework.Sdk and refactor Host/Installer build subsets (#38457)
* Start creating runtime, apphost, and targeting packs using the new SDK in the NewSFX subset. * Fix generation of runtime package, runtime symbols package, and host symbols package. * Correcly include hosting files and DIA in the shared framework archives and packages * Include PackageOverrides.txt in the ref pack. * Fix product brand prefix. * Add the shared host installer. * Add the hostfxr build and clean up MacOS building. * Rename host and hostfxr projects to ensure we don't collide with the nuget packaging projects. * Add bundle installer. * Update platform manifest entries to include linux/mac file variations from the runtime. * Update apphost pack to match master. * Fix typo * Include manpages and fix linux package properties. * Fix linux package properties. * Match short-description from packages in master. * Update to uploaded package. * Update metadata and SDK to generate an exact match of the productbuild xml file. * First draft moving crossgen2 to new SDK. * Get crossgen2 pack working with updated SDK. * Build archives from the installer and bundle projects. * Move the dotnet-runtime-deps installers to use the new SDK. * Rough attempt to convert the build to use the new sfx sdk. * Remove now empty subsets. * Update installer subsets for testing. * Remove non-existent packaging project from subsets. * Add missing Windows api shim dll to platform manifest. * Update sharedfx sdk. * Add both versions of the long-name dac to the platform manifest. * Skip builds for non-relevant components on Mono. * Remove installer tests by default temporarily to validate product build. * Remove pkgprojs subset from mono installer build since it doesn't build the hosts. * Skip the apphost pack on mono. * Update host test infra to use the new sfx projects. * Delete an out of date test (the alternative case this test exercises is now the only case in this repo) * Fix packaging tests and remove NETStandard packaging tests since we aren't producing that ref pack any more. * Fixes for test official build failures. Re-eneable installer tests by default. * Fix condition for building linux installers. * Exclude Mono header files from the deps.json. They don't need to be in it. * Don't include the host files on the sfxproj. * Add mono-specific files to the platform manifest. * Remove extra copy step in installer job. * Include libmonosgen-2.0.a in platform manifest. * Fix change in linux-x64 test steps. * Fix display name of linux package steps. * Build the installers based on the portable build when applicable so they can find their artifacts. * The installer build no longer consumes Microsoft.NETCore.Platforms and Microsoft.NETCore.Targets, try removing the dependency on the AllConfigurations leg from the installer build. * Set PackageBrandNameSuffix for runtime-deps packages. * Updated sharedfx sdk * Fix debian runtime-deps dependency elements. * Add new platform manifest entries. * Add icudt.dat to the platform manifest. * Invert conditions to correctly skip generating installers on non-applicable platforms. * Create extra installer subset so we can build just the installers (and avoid double-building nuget packages). Rename the newsfx subset and folder to sfx. * Fix Crossgen2 package name. * Clean up some dead MSBuild goo and update the sharedfx package so we can override the crossgen2 package name without breaking the default package id logic. * Update SharedFX SDK. * Fix discovery of the built dotnet layout for installer tests. Clean up some MSBuild along this path. * Update to use the new refactored packages. * Fix macOS build failures. * Fix linux installer build. * Upgrade to newest version of the SDK and centrally manage versions. * Update SDK to fix build skipping for the shared framework sdk. * Update SDK with full fix for skipbuild with packaging. * Another try for fixing the build-skip infra. * Upgrade SDK. * Fix package ids for mono packages. * Fix installer tests after update. * Add System.Private.Runtime.InteropServices.Javascript.dll to platform manifest * Fix Crossgen2 package to place all files in the tools folder. * Update to SDK with validation targets. * Add new ICU files to manifest. * Crossgen2 doesn't need to go through framework resolution so we can permit non-file-versioned dlls. (jitinterface.dll is currently unversioned). Signed-off-by: Jeremy Koritzinsky <[email protected]> * Put the property in a property group. * Ensure we build corehost before sfx since we no longer directly build the project from the sfx projects. * Add missing entry for additional ICU data file. * Remove step downloading all artifacts and remove MSBuild glue to hook up looking up artifacts in the full download. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Update SDK Signed-off-by: Jeremy Koritzinsky <[email protected]> * Update SDK to get MSI fix. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Don't even try building the hosting, crossgen2, or bundle installers when RuntimeFlavor is Mono. * Remove out of date comment. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Rename subsets based on PR discussion. * Update platform manifest and fix path to 3rd party notices for installer. * Remove test feed. * Update subset references I missed. * Fixed the subset arguments in other yaml files. * Add NuGet metadata to packs. * Add diagnostics-specific incremental signing in CoreCLR when the files are produced so the installer portion doesn't need to sign them. * Pass sign type to job. * Add new WASM files. * Enable static graph restore for the whole repo * Fix nits * Add back the runtime-deps packages. They were lost when we moved away from globs. * Don't disable NuGet static graph restore. * Make sure we sign the cross-arch same-os DAC * Update installers package. * Update Installers package from arcade * Update installers package. Co-authored-by: Viktor Hofer <[email protected]>
1 parent d8232a9 commit 86d5d16

File tree

195 files changed

+1315
-4411
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

195 files changed

+1315
-4411
lines changed

‎Build.proj

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -12,16 +12,6 @@
1212
<ProjectReference Include="@(ProjectToBuild)" />
1313
</ItemGroup>
1414

15-
<!--
16-
Exclude installer depproj and pkgproj from static graph restore. We restore them below.
17-
Remove when https://github.com/NuGet/Home/issues/9398 is fixed.
18-
-->
19-
<ItemGroup Condition="'$(MSBuildRestoreSessionId)' != ''">
20-
<ProjectReference Remove="@(DepprojProjectToBuild)" />
21-
<ProjectReference Remove="@(PkgprojProjectToBuild)" />
22-
<ProjectReference Remove="@(BundleProjectToBuild)" />
23-
</ItemGroup>
24-
2515
<!-- Custom arcade target which isn't available in Microsoft.Build.Traversal. -->
2616
<Target Name="Rebuild" DependsOnTargets="Clean;Build" />
2717

@@ -36,12 +26,4 @@
3626
<MSBuild Projects="$(RepoTasksDir)tasks.proj"
3727
Targets="BuildAndRestoreIncrementally"/>
3828
</Target>
39-
40-
<Target Name="RestoreWithoutStaticGraph"
41-
BeforeTargets="Restore">
42-
<MSBuild Projects="@(DepprojProjectToBuild);@(PkgprojProjectToBuild);@(BundleProjectToBuild)"
43-
Properties="MSBuildRestoreSessionId=$([System.Guid]::NewGuid());RestoreUseStaticGraphEvaluation=false"
44-
Targets="Restore" />
45-
</Target>
46-
4729
</Project>

‎eng/Signing.props

Lines changed: 29 additions & 127 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<Project>
1+
<Project InitialTargets="SetupFilesToSign">
22

33
<!-- If this file was pulled in via prepare-artifacts.proj (a non-SDK project, these files are already
44
imported. -->
@@ -12,7 +12,7 @@
1212
1313
During post build signing, there are no packages to sign during SignFinalPackages.
1414
-->
15-
<AllowEmptySignList Condition="'$(SignFinalPackages)' != 'true' or '$(PostBuildSign)' == 'true'">true</AllowEmptySignList>
15+
<AllowEmptySignList>true</AllowEmptySignList>
1616
</PropertyGroup>
1717

1818
<ItemGroup>
@@ -22,10 +22,6 @@
2222
-->
2323
<ItemsToSign Remove="@(ItemsToSign)" />
2424

25-
<!-- Find bundle artifacts, which need multiple stages to fully sign. -->
26-
<BundleInstallerEngineArtifact Include="$(ArtifactsPackagesDir)**/*engine.exe" />
27-
<BundleInstallerExeArtifact Include="$(ArtifactsPackagesDir)**/*.exe" />
28-
2925
<!-- apphost and comhost template files are not signed, by design. -->
3026
<FileSignInfo Include="apphost.exe;singlefilehost.exe;comhost.dll" CertificateName="None" />
3127

@@ -49,126 +45,32 @@
4945
<FileExtensionSignInfo Include=".deb;.rpm" CertificateName="LinuxSign" />
5046
</ItemGroup>
5147

52-
<!-- When doing post build signing, the file containers (e.g. nupkg, msi, etc.) are
53-
processed for signing (opened up, individually signed, etc.) and these individual ItemsToSign
54-
elements are unnecessary. When signing within the build, we need to individually process
55-
dll's, exes, etc. that go into msi's because these containers are not able to be processed
56-
by SignTool after they are packed up. What makes this possible for post build signing
57-
is that the build will produce a zip file containing the inputs to the Wix light linker
58-
which can be used to create the installer later, after the inputs have been signed and replaced. -->
59-
<Choose>
60-
<When Condition="'$(PostBuildSign)' != 'true'">
61-
<ItemGroup Condition="'$(SignBinaries)' == 'true'">
62-
<!-- Sign CoreCLR. -->
63-
<ItemsToSign Include="$(CoreCLRSharedFrameworkDir)*.dll" />
64-
<ItemsToSign Include="$(CoreCLRSharedFrameworkDir)*.exe" />
65-
66-
<ItemsToSign Include="$(CoreCLRArtifactsPath)System.Private.CoreLib.dll" />
67-
68-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)crossgen2.exe" />
69-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)crossgen2.dll" />
70-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)ILCompiler.DependencyAnalysisFramework.dll" />
71-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)ILCompiler.ReadyToRun.dll" />
72-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)ILCompiler.TypeSystem.ReadyToRun.dll" />
73-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)jitinterface_$(TargetArchitecture).dll" />
74-
75-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)clrjit_win_x86_$(TargetArchitecture).dll" />
76-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)clrjit_win_arm_$(TargetArchitecture).dll" />
77-
<ItemsToSign Include="$(CoreCLRCrossgen2Dir)clrjit_unix_arm_$(TargetArchitecture).dll" />
78-
<ItemsToSign Condition="'$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'x64'" Include="$(CoreCLRCrossgen2Dir)clrjit_win_x64_$(TargetArchitecture).dll" />
79-
<ItemsToSign Condition="'$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'x64'" Include="$(CoreCLRCrossgen2Dir)clrjit_win_arm64_$(TargetArchitecture).dll" />
80-
<ItemsToSign Condition="'$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'x64'" Include="$(CoreCLRCrossgen2Dir)clrjit_unix_x64_$(TargetArchitecture).dll" />
81-
<ItemsToSign Condition="'$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'x64'" Include="$(CoreCLRCrossgen2Dir)clrjit_unix_arm64_$(TargetArchitecture).dll" />
82-
83-
<ItemsToSign Include="$(CoreCLRArtifactsPath)$(CoreCLRCrossTargetComponentDirName)/sharedFramework/*.dll" Condition="'$(CoreCLRCrossTargetComponentDirName)' != ''" />
84-
<ItemsToSign Include="$(CoreCLRArtifactsPath)$(CoreCLRCrossTargetComponentDirName)/sharedFramework/*.exe" Condition="'$(CoreCLRCrossTargetComponentDirName)' != ''" />
85-
86-
<!-- Sign api-ms-win-core-xstate-l2-1-0 binary as it is only catalog signed in the current SDK. -->
87-
<ItemsToSign
88-
Condition="'$(Configuration)' == 'Release' and '$(TargetArchitecture)' == 'x86'"
89-
Include="$(CoreCLRArtifactsPath)Redist\ucrt\DLLs\$(TargetArchitecture)\api-ms-win-core-xstate-l2-1-0.dll" />
90-
91-
<!-- Sign libraries. -->
92-
<ItemsToSign Include="$(LibrariesNativeArtifactsPath)*.dll" />
93-
<ItemsToSign Include="$(LibrariesSharedFrameworkRefArtifactsPath)*.dll" />
94-
<!-- Most runtime artifacts will be crossgenned, so sign them post-crossgen. mscorlib isn't. -->
95-
<ItemsToSign Include="$(LibrariesSharedFrameworkBinArtifactsPath)mscorlib.dll" />
96-
97-
<!-- Sign the host. -->
98-
<ItemsToSign Include="$(BaseOutputRootPath)corehost/**/hostfxr.dll" />
99-
<ItemsToSign Include="$(BaseOutputRootPath)corehost/**/hostpolicy.dll" />
100-
<ItemsToSign Include="$(BaseOutputRootPath)corehost/**/dotnet.exe" />
101-
<ItemsToSign Include="$(BaseOutputRootPath)corehost/**/ijwhost.dll" />
102-
<ItemsToSign Include="$(BaseOutputRootPath)corehost/**/nethost.dll" />
103-
104-
<!-- Sign managed libraries in installer subset. -->
105-
<ItemsToSign Include="$(ArtifactsBinDir)Microsoft.NET.HostModel/**/*.dll" />
106-
</ItemGroup>
107-
108-
<!-- Sign ready-to-run binaries after crossgen is applied. -->
109-
<ItemGroup Condition="'$(SignR2RBinaries)' == 'true'">
110-
<ItemsToSign Include="$(CrossGenRootPath)**/*.dll" />
111-
</ItemGroup>
112-
113-
<ItemGroup Condition="'$(SignMsiFiles)' == 'true'">
114-
<ItemsToSign Include="$(ArtifactsPackagesDir)**/*.msi" />
115-
<ItemsToSign Include="$(ArtifactsPackagesDir)**/*.cab" />
116-
</ItemGroup>
117-
118-
<ItemGroup Condition="'$(SignBurnEngineFiles)' == 'true'">
119-
<ItemsToSign Include="@(BundleInstallerEngineArtifact)" />
120-
</ItemGroup>
121-
122-
<ItemGroup Condition="'$(SignBurnBundleFiles)' == 'true'">
123-
<!-- Sign the bundles, now that the engine is reattached. Avoid re-signing the engine. -->
124-
<ItemsToSign
125-
Include="@(BundleInstallerExeArtifact)"
126-
Exclude="@(BundleInstallerEngineArtifact)" />
127-
<!-- Note: wixstdba is internal to the engine bundle and does not get signed. -->
128-
</ItemGroup>
129-
130-
<ItemGroup Condition="'$(SignFinalPackages)' == 'true'">
131-
<DownloadedSymbolPackages Include="$(DownloadDirectory)**\*.symbols.nupkg" />
132-
<ItemsToSign Include="$(DownloadDirectory)**\*.nupkg" Exclude="@(DownloadedSymbolPackages)" />
133-
134-
<!-- The cross OS diagnostics symbol packages need to be signed as they are the only packages
135-
that have a specific version of assets that are only meant to be indexed in symbol servers.
136-
Since only *symbols.nupkg get indexed, and installer doesn't produce these, we need to glob them for signing. -->
137-
<ItemsToSign Include="$(DownloadDirectory)**\*CrossOsDiag*.symbols.nupkg" />
138-
139-
<ItemsToSign Include="$(DownloadDirectory)**\*.deb" />
140-
<ItemsToSign Include="$(DownloadDirectory)**\*.rpm" />
141-
</ItemGroup>
142-
</When>
143-
144-
<!-- When doing post build signing, we sign all artifacts we would push.
145-
Symbol packages are included too. -->
146-
<When Condition="'$(PostBuildSign)' == 'true'">
147-
<ItemGroup>
148-
<ItemsToSignWithPaths Include="$(DownloadDirectory)**/*.msi" Condition="'$(PrepareArtifacts)' == 'true'" />
149-
<ItemsToSignWithPaths Include="$(DownloadDirectory)**/*.exe" Condition="'$(PrepareArtifacts)' == 'true'" />
150-
<ItemsToSignWithPaths Include="$(DownloadDirectory)**/*.nupkg" Condition="'$(PrepareArtifacts)' == 'true'" />
151-
<ItemsToSignWithPaths Include="$(DownloadDirectory)**/*.zip" Condition="'$(PrepareArtifacts)' == 'true'" />
152-
153-
<ItemsToSignWithoutPaths Include="@(ItemsToSignWithPaths->'%(Filename)%(Extension)')" />
154-
<ItemsToSignPostBuild Include="@(ItemsToSignWithoutPaths->Distinct())" />
155-
</ItemGroup>
156-
157-
<!-- Even when doing post build signing, sign mscordaccore*.dll and mscordbi.dll -->
158-
<ItemGroup Condition="'$(SignBinaries)' == 'true'">
159-
<ItemsToSign Include="$(CoreCLRSharedFrameworkDir)mscordaccore*.dll" />
160-
<ItemsToSign Include="$(CoreCLRSharedFrameworkDir)mscordbi.dll" />
161-
<ItemsToSign Include="$(CoreCLRArtifactsPath)$(CoreCLRCrossTargetComponentDirName)/sharedFramework/mscordaccore*.dll" Condition="'$(CoreCLRCrossTargetComponentDirName)' != ''" />
162-
<ItemsToSign Include="$(CoreCLRArtifactsPath)$(CoreCLRCrossTargetComponentDirName)/sharedFramework/mscordbi.dll" Condition="'$(CoreCLRCrossTargetComponentDirName)' != ''" />
163-
</ItemGroup>
164-
165-
<ItemGroup Condition="'$(SignFinalPackages)' == 'true'">
166-
<!-- The cross OS diagnostics symbol packages need to be signed as they are the only packages
167-
that have a specific version of assets that are only meant to be indexed in symbol servers.
168-
Since only *symbols.nupkg get indexed, and installer doesn't produce these, we need to glob them for signing. -->
169-
<ItemsToSign Include="$(DownloadDirectory)**\*CrossOsDiag*.nupkg" />
170-
</ItemGroup>
171-
</When>
172-
</Choose>
48+
<Target Name="SetupFilesToSign">
49+
<!-- Ensure that we don't miss the DAC or DBI with the globbing below -->
50+
<PropertyGroup Condition="'$(SignDiagnostics)' == 'true'">
51+
<AllowEmptySignList>false</AllowEmptySignList>
52+
</PropertyGroup>
53+
54+
<ItemGroup Condition="'$(SignDiagnostics)' == 'true'">
55+
<ItemsToSign Include="$(DiagnosticsFilesRoot)/**/mscordaccore*.dll" />
56+
<ItemsToSign Include="$(DiagnosticsFilesRoot)/**/mscordbi.dll" />
57+
<!--
58+
The DAC should be signed with the SHA2 cert (both long and short name).
59+
We already add the short-name DAC above, so add the long-name DAC here.
60+
-->
61+
<DacFileSignInfo Include="@(ItemsToSign->'%(FileName)%(Extension)')"
62+
Condition="$([System.String]::new('%(FileName)').StartsWith('mscordaccore'))" />
63+
<FileSignInfo Include="@(DacFileSignInfo->ClearMetadata()->Distinct())"
64+
Exclude="mscordaccore.dll"
65+
CertificateName="MicrosoftSHA2" />
66+
</ItemGroup>
67+
68+
<ItemGroup Condition="'$(SignFinalPackages)' == 'true'">
69+
<!-- The cross OS diagnostics symbol packages need to be signed as they are the only packages
70+
that have a specific version of assets that are only meant to be indexed in symbol servers.
71+
Since only *symbols.nupkg get indexed, and installer doesn't produce these, we need to glob them for signing. -->
72+
<ItemsToSign Include="$(DownloadDirectory)**\*CrossOsDiag*.nupkg" />
73+
</ItemGroup>
74+
</Target>
17375

17476
</Project>

0 commit comments

Comments
 (0)