chore: Add micro benchmarks for ZLinq/SystemLinq perf comparison #149
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR add ZLinq micro benchmarks for LINQ operations under
Benchmark.ZLinq
namespace.And adding
SystemLinqBenchmarkConfig
that compare ZLinq/SystemLinq performance.When
SystemLinq
benchmark config is used.AsValueEnumerable()
is replaced to stub implementation that returnsIEnumerable<T>
.So it can compare ZLinq/SystemLinq performance with single benchmark method.
SystemLinq benchmark results example
See: https://github.com/filzrev/ZLinq/actions/runs/14808549467
Other changes
This PR contains following changes.
Sorry for the large diffs.
1.
.github/workflows/benchmark.yml
2.
Benchmark.csproj
net10.0
to first element. (Because first TFM is used to generate benchmark lists)NuGetVersionsBenchmarkConfig
/SystemLinqBenchmarkConfig
Clean
operation executed.3.
BaseBenchmarkConfig.cs
Add
TargetFrameworkFilter
benchmark to exclude non compatible benchmarks byBenchmarkCategory
.4.
NuGetVersionsBenchmarkConfig.cs
Modify to support ZLinq versions v1.0 or later with conditional build symbols(e.g.
ZLINQ_1_4_0_OR_GREATER
).Note:
Currently multi NuGet versions benchmarks seems not works because there is problems on BenchmarkDotNet side.
It's expected to be resolved when next version is released. (
ArtifactsPath
is enabled by default)5.
SystemLinqBenchmarkConfig.cs
Add custom config to compare ZLinq/SystemLinq performance.
When SystemLinq benchmark job is used. ZLinq's
AsValueEnumerable()
is replaced to stub implementation that returnsIEnumerable<T>
**6.
TargetFrameworksBenchmarkConfig.cs
Add settings to suppress baseline related warnings that occurred.
7.
BenchmarkEventProcessor.cs
Add logics to stop benchmark when benchmark projects failed to build.
And logs outputs.
8.
ConsumerExtensions.*.cs
Add optimized version Consume methods that required for ZLinq micro benchmarks.
9.
SummariesExtensions.*.cs
Refactor codes. and add following artifact outputs.
reports.md
: Simple markdown reports.reports.json
: Contains results data that required to generate HTML report with client-side filter.These files are intended to be used to publish HTML report to
GitHub Pages
(Currently benchmark results page URL is changed every runs. So I want to publish result to fixed path)
10.
TargetFrameworkFilter
/ZLinqBenchmarkFilter.cs
Add custom benchmark filters.
These filters are required because
#if
preprocessor is not applied when running benchmark method.Because BenchmarkDotNet listup benchmarks with .NET 9 (that defined at
<TargetFrameworks>
first element).11.
ZLinq/MicroBenchmarks/*
Add simple micro benchmark classes that use enumerable.
Almost benchmarks are inherited from
EnumerableBenchmarkBase_WithBasicTypes<T>
.The following are exceptions.
EnumerableBenchmarkBase<T>
base type. BecauseINumber<T>
based type required.EnumerableBenchmarkBase<T>
base type. Because Join results generate too large data in some cases.Distinct
because value must be unique value. So actual test items count is smaller.3.1. Single/SingleOrDefault
3.2. ToDictionary
3.3. ToFrozenDictionary
3.4. ToImmutableDictionary
12.
BenchmarkResultsReport
Add model class that represent benchmark results.
It'll be used to create HTML report from artifacts.
13.
Program.cs
--verbose
extra parameter support to enable BenchmarkDotNet default progress logs.14.
ValueEnumerableExtensions.*.cs
Add extension methods that are used by
SystemLinqBenchmarks
15.
TestData/*
Add code that generate random test data for IEnumerable.