Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C#: Update .NET 8 models. #17666

Merged
merged 11 commits into from
Oct 25, 2024
Merged

Conversation

michaelnebel
Copy link
Contributor

@michaelnebel michaelnebel commented Oct 4, 2024

TLDR

In this PR we re-generate the .NET 8 runtime models based on the mixed model generation.

  • A minor update for the cs/information-exposure-through-exception query is needed.
  • To retain some query results we need to introduce manual models for StringWriter.
  • When we originally added the generated models for .NET Runtime we didn't see that big changes to the produced alerts (maybe this is due to our DCA suite), so it was expected that we only would see few changes to the produced alerts, when updating the .NET 8 models. In any case, that we are able to retain all alerts with only some minor manual modelling and at the same time increase the precision of the models should be considered a success.

Details

  • It turns out that the cs/information-exposure-through-exception query relies on the imprecise df-generated summaries for taint propagation from an exception to its properties. With more precise summaries this is no longer possible. Instead we encode this directly in the query sources instead.
  • The manual models for StringWriter is needed because they need to be used in conjunction with summaries for TextWriter (note that StringWriter extends TextWriter). More specifically the query results relied on the models seen below, but the content based models for StringWriter refers to the (synthetic) field containing a StringBuilder (as this is where the information is stored)
    • ["System.Text.Encodings.Web", "TextEncoder", False, "Encode", "(System.IO.TextWriter,System.String)", "", "Argument[1]", "Argument[0]", "taint", "df-generated"]
    • ["System.IO", "StringWriter", False, "ToString", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"]

DCA results

  • Peformance looks good.
  • The new results for cs/exposure-of-sensitive-information, cs/information-exposure-through-exception and cs/sensitive-data-transmission on mono are due to the manual models for System.IO.StringWriter.

@github-actions github-actions bot added the C# label Oct 4, 2024
Copy link
Contributor

github-actions bot commented Oct 4, 2024

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

csharp

Generated file changes for csharp

  • Changes to framework-coverage-csharp.rst:
-    System,"``System.*``, ``System``",47,10626,54,5
+    System,"``System.*``, ``System``",47,10313,54,5
-    Others,"``Amazon.Lambda.APIGatewayEvents``, ``Amazon.Lambda.Core``, ``Dapper``, ``ILCompiler``, ``ILLink.RoslynAnalyzer``, ``ILLink.Shared``, ``ILLink.Tasks``, ``Internal.IL``, ``Internal.Pgo``, ``Internal.TypeSystem``, ``JsonToItemsTaskFactory``, ``Microsoft.Android.Build``, ``Microsoft.Apple.Build``, ``Microsoft.ApplicationBlocks.Data``, ``Microsoft.CSharp``, ``Microsoft.Diagnostics.Tools.Pgo``, ``Microsoft.DotNet.Build.Tasks``, ``Microsoft.EntityFrameworkCore``, ``Microsoft.Extensions.Caching.Distributed``, ``Microsoft.Extensions.Caching.Memory``, ``Microsoft.Extensions.Configuration``, ``Microsoft.Extensions.DependencyInjection``, ``Microsoft.Extensions.DependencyModel``, ``Microsoft.Extensions.Diagnostics.Metrics``, ``Microsoft.Extensions.FileProviders``, ``Microsoft.Extensions.FileSystemGlobbing``, ``Microsoft.Extensions.Hosting``, ``Microsoft.Extensions.Http``, ``Microsoft.Extensions.Logging``, ``Microsoft.Extensions.Options``, ``Microsoft.Extensions.Primitives``, ``Microsoft.Interop``, ``Microsoft.NET.Build.Tasks``, ``Microsoft.NET.WebAssembly.Webcil``, ``Microsoft.VisualBasic``, ``Microsoft.WebAssembly.Build.Tasks``, ``Microsoft.Win32``, ``Mono.Linker``, ``MySql.Data.MySqlClient``, ``Newtonsoft.Json``, ``SourceGenerators``, ``Windows.Security.Cryptography.Core``",57,1821,148,
+    Others,"``Amazon.Lambda.APIGatewayEvents``, ``Amazon.Lambda.Core``, ``Dapper``, ``ILCompiler``, ``ILLink.RoslynAnalyzer``, ``ILLink.Shared``, ``ILLink.Tasks``, ``Internal.IL``, ``Internal.Pgo``, ``Internal.TypeSystem``, ``JsonToItemsTaskFactory``, ``Microsoft.Android.Build``, ``Microsoft.Apple.Build``, ``Microsoft.ApplicationBlocks.Data``, ``Microsoft.CSharp``, ``Microsoft.Diagnostics.Tools.Pgo``, ``Microsoft.DotNet.Build.Tasks``, ``Microsoft.EntityFrameworkCore``, ``Microsoft.Extensions.Caching.Distributed``, ``Microsoft.Extensions.Caching.Memory``, ``Microsoft.Extensions.Configuration``, ``Microsoft.Extensions.DependencyInjection``, ``Microsoft.Extensions.DependencyModel``, ``Microsoft.Extensions.Diagnostics.Metrics``, ``Microsoft.Extensions.FileProviders``, ``Microsoft.Extensions.FileSystemGlobbing``, ``Microsoft.Extensions.Hosting``, ``Microsoft.Extensions.Http``, ``Microsoft.Extensions.Logging``, ``Microsoft.Extensions.Options``, ``Microsoft.Extensions.Primitives``, ``Microsoft.Interop``, ``Microsoft.NET.Build.Tasks``, ``Microsoft.NET.Sdk.WebAssembly``, ``Microsoft.NET.WebAssembly.Webcil``, ``Microsoft.VisualBasic``, ``Microsoft.WebAssembly.Build.Tasks``, ``Microsoft.Win32``, ``Mono.Linker``, ``MySql.Data.MySqlClient``, ``Newtonsoft.Json``, ``SourceGenerators``, ``Windows.Security.Cryptography.Core``",57,1848,148,
-    Totals,,104,12454,396,5
+    Totals,,104,12168,396,5
  • Changes to framework-coverage-csharp.csv:
- ILCompiler,,,123,,,,,,,,,,,,,,,,,,,123,
+ ILCompiler,,,123,,,,,,,,,,,,,,,,,,,79,44
- ILLink.RoslynAnalyzer,,,145,,,,,,,,,,,,,,,,,,,145,
+ ILLink.RoslynAnalyzer,,,139,,,,,,,,,,,,,,,,,,,50,89
- ILLink.Shared,,,34,,,,,,,,,,,,,,,,,,,32,2
+ ILLink.Shared,,,31,,,,,,,,,,,,,,,,,,,11,20
- ILLink.Tasks,,,4,,,,,,,,,,,,,,,,,,,4,
+ ILLink.Tasks,,,5,,,,,,,,,,,,,,,,,,,4,1
- Internal.IL,,,46,,,,,,,,,,,,,,,,,,,44,2
+ Internal.IL,,,54,,,,,,,,,,,,,,,,,,,28,26
- Internal.Pgo,,,9,,,,,,,,,,,,,,,,,,,8,1
+ Internal.Pgo,,,9,,,,,,,,,,,,,,,,,,,2,7
- Internal.TypeSystem,,,315,,,,,,,,,,,,,,,,,,,299,16
+ Internal.TypeSystem,,,328,,,,,,,,,,,,,,,,,,,201,127
- JsonToItemsTaskFactory,,,10,,,,,,,,,,,,,,,,,,,10,
+ JsonToItemsTaskFactory,,,11,,,,,,,,,,,,,,,,,,,1,10
- Microsoft.Android.Build,,1,16,,,,,,,,,,,,,1,,,,,,16,
+ Microsoft.Android.Build,,1,14,,,,,,,,,,,,,1,,,,,,12,2
- Microsoft.Apple.Build,,,8,,,,,,,,,,,,,,,,,,,8,
+ Microsoft.Apple.Build,,,7,,,,,,,,,,,,,,,,,,,7,
- Microsoft.CSharp,,,13,,,,,,,,,,,,,,,,,,,13,
+ Microsoft.CSharp,,,2,,,,,,,,,,,,,,,,,,,2,
- Microsoft.Diagnostics.Tools.Pgo,,,12,,,,,,,,,,,,,,,,,,,12,
+ Microsoft.Diagnostics.Tools.Pgo,,,23,,,,,,,,,,,,,,,,,,,2,21
- Microsoft.DotNet.Build.Tasks,,,6,,,,,,,,,,,,,,,,,,,6,
+ Microsoft.DotNet.Build.Tasks,,,10,,,,,,,,,,,,,,,,,,,8,2
- Microsoft.Extensions.Caching.Distributed,,,10,,,,,,,,,,,,,,,,,,,10,
+ Microsoft.Extensions.Caching.Distributed,,,3,,,,,,,,,,,,,,,,,,,,3
- Microsoft.Extensions.Caching.Memory,,,39,,,,,,,,,,,,,,,,,,,38,1
+ Microsoft.Extensions.Caching.Memory,,,31,,,,,,,,,,,,,,,,,,,5,26
- Microsoft.Extensions.Configuration,,3,90,,,,,,,,,,,,,3,,,,,,89,1
+ Microsoft.Extensions.Configuration,,3,91,,,,,,,,,,,,,3,,,,,,25,66
- Microsoft.Extensions.DependencyInjection,,,134,,,,,,,,,,,,,,,,,,,133,1
+ Microsoft.Extensions.DependencyInjection,,,130,,,,,,,,,,,,,,,,,,,17,113
- Microsoft.Extensions.DependencyModel,,1,18,,,,,,,,,,,,,1,,,,,,18,
+ Microsoft.Extensions.DependencyModel,,1,16,,,,,,,,,,,,,1,,,,,,14,2
- Microsoft.Extensions.Diagnostics.Metrics,,,15,,,,,,,,,,,,,,,,,,,15,
+ Microsoft.Extensions.Diagnostics.Metrics,,,14,,,,,,,,,,,,,,,,,,,1,13
- Microsoft.Extensions.FileProviders,,,15,,,,,,,,,,,,,,,,,,,15,
+ Microsoft.Extensions.FileProviders,,,17,,,,,,,,,,,,,,,,,,,7,10
- Microsoft.Extensions.FileSystemGlobbing,,,18,,,,,,,,,,,,,,,,,,,16,2
+ Microsoft.Extensions.FileSystemGlobbing,,,22,,,,,,,,,,,,,,,,,,,11,11
- Microsoft.Extensions.Hosting,,,41,,,,,,,,,,,,,,,,,,,40,1
+ Microsoft.Extensions.Hosting,,,39,,,,,,,,,,,,,,,,,,,29,10
- Microsoft.Extensions.Http,,,9,,,,,,,,,,,,,,,,,,,9,
+ Microsoft.Extensions.Http,,,9,,,,,,,,,,,,,,,,,,,7,2
- Microsoft.Extensions.Logging,,,65,,,,,,,,,,,,,,,,,,,64,1
+ Microsoft.Extensions.Logging,,,64,,,,,,,,,,,,,,,,,,,25,39
- Microsoft.Extensions.Options,,,13,,,,,,,,,,,,,,,,,,,13,
+ Microsoft.Extensions.Options,,,14,,,,,,,,,,,,,,,,,,,14,
- Microsoft.Extensions.Primitives,,,72,,,,,,,,,,,,,,,,,,,72,
+ Microsoft.Extensions.Primitives,,,72,,,,,,,,,,,,,,,,,,,67,5
- Microsoft.Interop,,,121,,,,,,,,,,,,,,,,,,,121,
+ Microsoft.Interop,,,137,,,,,,,,,,,,,,,,,,,70,67
- Microsoft.NET.Build.Tasks,,,4,,,,,,,,,,,,,,,,,,,4,
+ Microsoft.NET.Build.Tasks,,,5,,,,,,,,,,,,,,,,,,,3,2
+ Microsoft.NET.Sdk.WebAssembly,,,2,,,,,,,,,,,,,,,,,,,1,1
- Microsoft.NET.WebAssembly.Webcil,,,8,,,,,,,,,,,,,,,,,,,8,
+ Microsoft.NET.WebAssembly.Webcil,,,6,,,,,,,,,,,,,,,,,,,6,
- Microsoft.VisualBasic,,,6,,,,,,,,,,,,,,,,,,,1,5
+ Microsoft.VisualBasic,,,13,,,,,,,,,,,,,,,,,,,1,12
- Microsoft.WebAssembly.Build.Tasks,,,4,,,,,,,,,,,,,,,,,,,4,
+ Microsoft.WebAssembly.Build.Tasks,,,9,,,,,,,,,,,,,,,,,,,8,1
- Microsoft.Win32,,4,4,,,,,,,,,,,,,,,,,,4,4,
+ Microsoft.Win32,,4,2,,,,,,,,,,,,,,,,,,4,,2
- Mono.Linker,,,285,,,,,,,,,,,,,,,,,,,285,
+ Mono.Linker,,,287,,,,,,,,,,,,,,,,,,,145,142
- SourceGenerators,,,5,,,,,,,,,,,,,,,,,,,5,
+ SourceGenerators,,,5,,,,,,,,,,,,,,,,,,,,5
- System,54,47,10626,,6,5,5,,,4,1,,33,2,,6,15,17,4,3,,8721,1905
+ System,54,47,10313,,6,5,5,,,4,1,,33,2,,6,15,17,4,3,,5351,4962

@michaelnebel michaelnebel changed the title C#: .NET 8 models C#: Update .NET 8 models. Oct 7, 2024
@michaelnebel michaelnebel force-pushed the csharp/net8models branch 3 times, most recently from caf4b25 to d6a4e71 Compare October 9, 2024 08:07
@michaelnebel michaelnebel marked this pull request as ready for review October 9, 2024 09:51
@michaelnebel michaelnebel requested a review from a team as a code owner October 9, 2024 09:51
---
category: majorAnalysis
---
* C#: The generated .NET 8 runtime models have been updated.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be wrong, but I don't think the C#: prefix is needed, as hopefully that is inferred from the fact that this change not is inside the csharp folder.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove it.

@@ -24,6 +24,7 @@ models
| 23 | Summary: System; Span<T>; false; Span; (T); ; Argument[0]; Argument[this].Element; value; manual |
| 24 | Summary: System; Span<T>; false; Span; (T[]); ; Argument[0].Element; Argument[this].Element; value; manual |
| 25 | Summary: System; Span<T>; false; ToArray; (); ; Argument[this].Element; ReturnValue.Element; value; manual |
| 26 | Summary: System.Collections.Generic; List<T>+Enumerator; false; get_Current; (); ; Argument[this].Property[System.Collections.Generic.List`1+Enumerator.Current]; ReturnValue; value; dfc-generated |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a summary we shouldn't generate: It is always the case that get_Foo induces a readstep from Argument[this] to ReturnValue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that when we discussed it last (when you made #16088) - we agreed to keep model generation for properties that wasn't read/write (in this case the property only has a getter).
Do you think we should revisit this decision?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that if we generate a summary for a getter that is saying exactly that it is a getter, then there is no point (since we already have a generic read step for that).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also not something that we generally do.
The example in question is because there are two property implementations for Current, where the concrete interface implementation returns the value of the other Current property. The other "known" example where this can happen is for recursive property definitions (which do exist - but I sure hope that this is not a general programming pattern).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks a lot for the explanation.

@michaelnebel michaelnebel requested a review from hvitved October 24, 2024 09:00
@michaelnebel michaelnebel merged commit 0b53831 into github:main Oct 25, 2024
23 checks passed
@michaelnebel michaelnebel deleted the csharp/net8models branch October 25, 2024 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants