diff --git a/docs/logs/README.md b/docs/logs/README.md index 8172dd33569..77a46c847e8 100644 --- a/docs/logs/README.md +++ b/docs/logs/README.md @@ -107,6 +107,21 @@ Here is the rule of thumb: Minutes - Console Application](./getting-started-console/README.md) tutorial to learn more. +:heavy_check_mark: You should use dot-separated +[UpperCamelCase](https://en.wikipedia.org/wiki/Camel_case) as the log category +name, which makes it convenient to [filter logs](#log-filtering). A common +practice is to use fully qualified class name, and if further categorization is +desired, append a subcategory name. Refer to the [.NET official +document](https://learn.microsoft.com/dotnet/core/extensions/logging#log-category) +to learn more. + +```csharp +loggerFactory.CreateLogger(); // this is equivalent to CreateLogger("MyProduct.MyLibrary.MyClass") +loggerFactory.CreateLogger("MyProduct.MyLibrary.MyClass"); // use the fully qualified class name +loggerFactory.CreateLogger("MyProduct.MyLibrary.MyClass.DatabaseOperations"); // append a subcategory name +loggerFactory.CreateLogger("MyProduct.MyLibrary.MyClass.FileOperations"); // append another subcategory name +``` + :stop_sign: You should avoid creating loggers too frequently. Although loggers are not super expensive, they still come with CPU and memory cost, and are meant to be reused throughout the application. Refer to the [logging performance @@ -186,11 +201,6 @@ instances if they are created by you. API invocation associated with the logger factory could become no-op (i.e. no logs will be emitted). -:heavy_check_mark: You should use the fully qualified class name as the log -category name. Refer to the [.NET official -document](https://learn.microsoft.com/dotnet/core/extensions/logging#log-category) -to learn more. - ## Log Correlation In OpenTelemetry, logs are automatically correlated to diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 14a72aaf2ac..4cb57f11693 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -412,26 +412,23 @@ exemplars. #### ExemplarFilter -`ExemplarFilter` determines which measurements are eligible to become an -Exemplar. i.e. `ExemplarFilter` determines which measurements are offered to -`ExemplarReservoir`, which makes the final decision about whether the offered -measurement gets stored as an exemplar. They can be used to control the noise -and overhead associated with Exemplar collection. +`ExemplarFilter` determines which measurements are offered to the configured +`ExemplarReservoir`, which makes the final decision about whether or not the +offered measurement gets recorded as an `Exemplar`. Generally `ExemplarFilter` +is a mechanism to control the overhead associated with `Exemplar` offering. -OpenTelemetry SDK comes with the following Filters: +OpenTelemetry SDK comes with the following `ExemplarFilters` (defined on +`ExemplarFilterType`): -* `AlwaysOnExemplarFilter` - makes all measurements eligible for being an Exemplar. -* `AlwaysOffExemplarFilter` - makes no measurements eligible for being an - Exemplar. Using this is as good as turning off Exemplar feature, and is the current +* `AlwaysOff`: Makes no measurements eligible for becoming an `Exemplar`. Using + this is as good as turning off the `Exemplar` feature and is the current default. -* `TraceBasedExemplarFilter` - makes those measurements eligible for being an -Exemplar, which are recorded in the context of a sampled parent `Activity` -(span). +* `AlwaysOn`: Makes all measurements eligible for becoming an `Exemplar`. +* `TraceBased`: Makes those measurements eligible for becoming an `Exemplar` + which are recorded in the context of a sampled `Activity` (span). -`SetExemplarFilter` method on `MeterProviderBuilder` can be used to set the -desired `ExemplarFilter`. - -The snippet below shows how to set `ExemplarFilter`. +The `SetExemplarFilter` extension method on `MeterProviderBuilder` can be used +to set the desired `ExemplarFilterType` and enable `Exemplar` collection: ```csharp using OpenTelemetry; @@ -439,31 +436,14 @@ using OpenTelemetry.Metrics; using var meterProvider = Sdk.CreateMeterProviderBuilder() // rest of config not shown - .SetExemplarFilter(new TraceBasedExemplarFilter()) + .SetExemplarFilter(ExemplarFilterType.TraceBased) .Build(); ``` -> [!NOTE] -> As of today, there is no separate toggle for enable/disable Exemplar feature. -Exemplars can be disabled by setting filter as `AlwaysOffExemplarFilter`, which -is also the default (i.e Exemplar feature is disabled by default). Users can -enable the feature by setting filter to anything other than -`AlwaysOffExemplarFilter`. For example: `.SetExemplarFilter(new TraceBasedExemplarFilter())`. - -If the built-in `ExemplarFilter`s are not meeting the needs, one may author -custom `ExemplarFilter` as shown -[here](../extending-the-sdk/README.md#exemplarfilter). A custom filter, which -eliminates all un-interesting measurements from becoming Exemplar is a -recommended way to control performance overhead associated with collecting -Exemplars. See -[benchmark](../../../test/Benchmarks/Metrics/ExemplarBenchmarks.cs) to see how -much impact can `ExemplarFilter` have on performance. - #### ExemplarReservoir -`ExemplarReservoir` receives the measurements sampled in by the `ExemplarFilter` -and is responsible for storing Exemplars. `ExemplarReservoir` ultimately decides -which measurements get stored as exemplars. The following are the default +`ExemplarReservoir` receives the measurements sampled by the `ExemplarFilter` +and is responsible for recording `Exemplar`s. The following are the default reservoirs: * `AlignedHistogramBucketExemplarReservoir` is the default reservoir used for @@ -479,7 +459,7 @@ size (currently defaulting to 1) determines the maximum number of exemplars stored. > [!NOTE] -> Currently there is no ability to change or configure Reservoir. +> Currently there is no ability to change or configure `ExemplarReservoir`. ### Instrumentation diff --git a/docs/metrics/extending-the-sdk/README.md b/docs/metrics/extending-the-sdk/README.md index 28df4c467c5..c7293ac418a 100644 --- a/docs/metrics/extending-the-sdk/README.md +++ b/docs/metrics/extending-the-sdk/README.md @@ -74,44 +74,7 @@ Not supported. ## ExemplarFilter -OpenTelemetry .NET SDK has provided the following built-in `ExemplarFilter`s: - -* [AlwaysOnExemplarFilter](../../../src/OpenTelemetry/Metrics/Exemplar/AlwaysOnExemplarFilter.cs) -* [AlwaysOffExemplarFilter](../../../src/OpenTelemetry/Metrics/Exemplar/AlwaysOffExemplarFilter.cs) -* [TraceBasedExemplarFilter](../../../src/OpenTelemetry/Metrics/Exemplar/TraceBasedExemplarFilter.cs) - -Custom exemplar filters can be implemented to achieve filtering based on other criterion: - -* `ExemplarFilter` should derive from `OpenTelemetry.ExemplarFilter` (which - belongs to the [OpenTelemetry](../../../src/OpenTelemetry/README.md) package) - and implement the `ShouldSample` method. - -One example is a filter, which filters all measurements of value lower -than given threshold is given below. Such a filter prevents any measurements -below the given threshold from ever becoming a `Exemplar`. Such filters could -also incorporate the `TraceBasedExemplarFilter` condition as well, as storing -exemplars for non-sampled traces may be undesired. - -```csharp -public sealed class HighValueFilter : ExemplarFilter -{ - private readonly double maxValue; - - public HighValueFilter(double maxValue) - { - this.maxValue = maxValue; - } - public override bool ShouldSample(long value, ReadOnlySpan> tags) - { - return Activity.Current?.Recorded && value > this.maxValue; - } - - public override bool ShouldSample(double value, ReadOnlySpan> tags) - { - return Activity.Current?.Recorded && value > this.maxValue; - } -} -``` +Not supported. ## ExemplarReservoir diff --git a/examples/AspNetCore/Program.cs b/examples/AspNetCore/Program.cs index e41e068b407..18279940976 100644 --- a/examples/AspNetCore/Program.cs +++ b/examples/AspNetCore/Program.cs @@ -85,7 +85,7 @@ builder .AddMeter(Instrumentation.MeterName) #if EXPOSE_EXPERIMENTAL_FEATURES - .SetExemplarFilter(new TraceBasedExemplarFilter()) + .SetExemplarFilter(ExemplarFilterType.TraceBased) #endif .AddRuntimeInstrumentation() .AddHttpClientInstrumentation() diff --git a/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/CHANGELOG.md b/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/CHANGELOG.md index f862133e31f..d5d71920bd0 100644 --- a/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/CHANGELOG.md +++ b/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/CHANGELOG.md @@ -30,6 +30,11 @@ as it is mandated by the specification. ([#5316](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5268)) +* **Experimental (pre-release builds only):** Add support in + `OtlpMetricExporter` for emitting exemplars supplied on Counters, Gauges, and + ExponentialHistograms. + ([#5397](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5397)) + ## 1.7.0 Released 2023-Dec-08 diff --git a/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/MetricItemExtensions.cs b/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/MetricItemExtensions.cs index 53276e617e0..fb9266cb613 100644 --- a/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/MetricItemExtensions.cs +++ b/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/MetricItemExtensions.cs @@ -158,6 +158,16 @@ internal static OtlpMetrics.Metric ToOtlpMetric(this Metric metric) AddAttributes(metricPoint.Tags, dataPoint.Attributes); dataPoint.AsInt = metricPoint.GetSumLong(); + + if (metricPoint.TryGetExemplars(out var exemplars)) + { + foreach (ref readonly var exemplar in exemplars) + { + dataPoint.Exemplars.Add( + ToOtlpExemplar(exemplar.LongValue, in exemplar)); + } + } + sum.DataPoints.Add(dataPoint); } @@ -185,6 +195,16 @@ internal static OtlpMetrics.Metric ToOtlpMetric(this Metric metric) AddAttributes(metricPoint.Tags, dataPoint.Attributes); dataPoint.AsDouble = metricPoint.GetSumDouble(); + + if (metricPoint.TryGetExemplars(out var exemplars)) + { + foreach (ref readonly var exemplar in exemplars) + { + dataPoint.Exemplars.Add( + ToOtlpExemplar(exemplar.DoubleValue, in exemplar)); + } + } + sum.DataPoints.Add(dataPoint); } @@ -206,6 +226,16 @@ internal static OtlpMetrics.Metric ToOtlpMetric(this Metric metric) AddAttributes(metricPoint.Tags, dataPoint.Attributes); dataPoint.AsInt = metricPoint.GetGaugeLastValueLong(); + + if (metricPoint.TryGetExemplars(out var exemplars)) + { + foreach (ref readonly var exemplar in exemplars) + { + dataPoint.Exemplars.Add( + ToOtlpExemplar(exemplar.LongValue, in exemplar)); + } + } + gauge.DataPoints.Add(dataPoint); } @@ -227,6 +257,16 @@ internal static OtlpMetrics.Metric ToOtlpMetric(this Metric metric) AddAttributes(metricPoint.Tags, dataPoint.Attributes); dataPoint.AsDouble = metricPoint.GetGaugeLastValueDouble(); + + if (metricPoint.TryGetExemplars(out var exemplars)) + { + foreach (ref readonly var exemplar in exemplars) + { + dataPoint.Exemplars.Add( + ToOtlpExemplar(exemplar.DoubleValue, in exemplar)); + } + } + gauge.DataPoints.Add(dataPoint); } @@ -320,7 +360,14 @@ internal static OtlpMetrics.Metric ToOtlpMetric(this Metric metric) dataPoint.Positive.BucketCounts.Add((ulong)bucketCount); } - // TODO: exemplars. + if (metricPoint.TryGetExemplars(out var exemplars)) + { + foreach (ref readonly var exemplar in exemplars) + { + dataPoint.Exemplars.Add( + ToOtlpExemplar(exemplar.DoubleValue, in exemplar)); + } + } histogram.DataPoints.Add(dataPoint); } @@ -333,29 +380,7 @@ internal static OtlpMetrics.Metric ToOtlpMetric(this Metric metric) return otlpMetric; } - private static void AddAttributes(ReadOnlyTagCollection tags, RepeatedField attributes) - { - foreach (var tag in tags) - { - if (OtlpKeyValueTransformer.Instance.TryTransformTag(tag, out var result)) - { - attributes.Add(result); - } - } - } - - private static void AddScopeAttributes(IEnumerable> meterTags, RepeatedField attributes) - { - foreach (var tag in meterTags) - { - if (OtlpKeyValueTransformer.Instance.TryTransformTag(tag, out var result)) - { - attributes.Add(result); - } - } - } - - private static OtlpMetrics.Exemplar ToOtlpExemplar(T value, in Metrics.Exemplar exemplar) + internal static OtlpMetrics.Exemplar ToOtlpExemplar(T value, in Metrics.Exemplar exemplar) where T : struct { var otlpExemplar = new OtlpMetrics.Exemplar @@ -399,4 +424,26 @@ private static OtlpMetrics.Exemplar ToOtlpExemplar(T value, in Metrics.Exempl return otlpExemplar; } + + private static void AddAttributes(ReadOnlyTagCollection tags, RepeatedField attributes) + { + foreach (var tag in tags) + { + if (OtlpKeyValueTransformer.Instance.TryTransformTag(tag, out var result)) + { + attributes.Add(result); + } + } + } + + private static void AddScopeAttributes(IEnumerable> meterTags, RepeatedField attributes) + { + foreach (var tag in meterTags) + { + if (OtlpKeyValueTransformer.Instance.TryTransformTag(tag, out var result)) + { + attributes.Add(result); + } + } + } } diff --git a/src/OpenTelemetry/.publicApi/Experimental/PublicAPI.Unshipped.txt b/src/OpenTelemetry/.publicApi/Experimental/PublicAPI.Unshipped.txt index b530cdf98f6..8c1a746f041 100644 --- a/src/OpenTelemetry/.publicApi/Experimental/PublicAPI.Unshipped.txt +++ b/src/OpenTelemetry/.publicApi/Experimental/PublicAPI.Unshipped.txt @@ -5,10 +5,6 @@ OpenTelemetry.Logs.LogRecord.Severity.get -> OpenTelemetry.Logs.LogRecordSeverit OpenTelemetry.Logs.LogRecord.Severity.set -> void OpenTelemetry.Logs.LogRecord.SeverityText.get -> string? OpenTelemetry.Logs.LogRecord.SeverityText.set -> void -OpenTelemetry.Metrics.AlwaysOffExemplarFilter -OpenTelemetry.Metrics.AlwaysOffExemplarFilter.AlwaysOffExemplarFilter() -> void -OpenTelemetry.Metrics.AlwaysOnExemplarFilter -OpenTelemetry.Metrics.AlwaysOnExemplarFilter.AlwaysOnExemplarFilter() -> void OpenTelemetry.Metrics.Exemplar OpenTelemetry.Metrics.Exemplar.DoubleValue.get -> double OpenTelemetry.Metrics.Exemplar.Exemplar() -> void @@ -17,13 +13,15 @@ OpenTelemetry.Metrics.Exemplar.LongValue.get -> long OpenTelemetry.Metrics.Exemplar.SpanId.get -> System.Diagnostics.ActivitySpanId OpenTelemetry.Metrics.Exemplar.Timestamp.get -> System.DateTimeOffset OpenTelemetry.Metrics.Exemplar.TraceId.get -> System.Diagnostics.ActivityTraceId -OpenTelemetry.Metrics.ExemplarFilter -OpenTelemetry.Metrics.ExemplarFilter.ExemplarFilter() -> void +OpenTelemetry.Metrics.ExemplarFilterType +OpenTelemetry.Metrics.ExemplarFilterType.AlwaysOff = 0 -> OpenTelemetry.Metrics.ExemplarFilterType +OpenTelemetry.Metrics.ExemplarFilterType.AlwaysOn = 1 -> OpenTelemetry.Metrics.ExemplarFilterType +OpenTelemetry.Metrics.ExemplarFilterType.TraceBased = 2 -> OpenTelemetry.Metrics.ExemplarFilterType OpenTelemetry.Metrics.ExemplarMeasurement OpenTelemetry.Metrics.ExemplarMeasurement.ExemplarMeasurement() -> void OpenTelemetry.Metrics.ExemplarMeasurement.Tags.get -> System.ReadOnlySpan> OpenTelemetry.Metrics.ExemplarMeasurement.Value.get -> T -OpenTelemetry.Metrics.MetricPoint.TryGetExemplars(out OpenTelemetry.Metrics.ReadOnlyExemplarCollection? exemplars) -> bool +OpenTelemetry.Metrics.MetricPoint.TryGetExemplars(out OpenTelemetry.Metrics.ReadOnlyExemplarCollection exemplars) -> bool OpenTelemetry.Metrics.MetricStreamConfiguration.CardinalityLimit.get -> int? OpenTelemetry.Metrics.MetricStreamConfiguration.CardinalityLimit.set -> void OpenTelemetry.Metrics.ReadOnlyExemplarCollection @@ -33,8 +31,6 @@ OpenTelemetry.Metrics.ReadOnlyExemplarCollection.Enumerator.Enumerator() -> void OpenTelemetry.Metrics.ReadOnlyExemplarCollection.Enumerator.MoveNext() -> bool OpenTelemetry.Metrics.ReadOnlyExemplarCollection.GetEnumerator() -> OpenTelemetry.Metrics.ReadOnlyExemplarCollection.Enumerator OpenTelemetry.Metrics.ReadOnlyExemplarCollection.ReadOnlyExemplarCollection() -> void -OpenTelemetry.Metrics.TraceBasedExemplarFilter -OpenTelemetry.Metrics.TraceBasedExemplarFilter.TraceBasedExemplarFilter() -> void OpenTelemetry.ReadOnlyFilteredTagCollection OpenTelemetry.ReadOnlyFilteredTagCollection.Enumerator OpenTelemetry.ReadOnlyFilteredTagCollection.Enumerator.Current.get -> System.Collections.Generic.KeyValuePair @@ -51,19 +47,11 @@ static OpenTelemetry.Logs.LoggerProviderBuilderExtensions.SetResourceBuilder(thi static OpenTelemetry.Logs.LoggerProviderExtensions.AddProcessor(this OpenTelemetry.Logs.LoggerProvider! provider, OpenTelemetry.BaseProcessor! processor) -> OpenTelemetry.Logs.LoggerProvider! static OpenTelemetry.Logs.LoggerProviderExtensions.ForceFlush(this OpenTelemetry.Logs.LoggerProvider! provider, int timeoutMilliseconds = -1) -> bool static OpenTelemetry.Logs.LoggerProviderExtensions.Shutdown(this OpenTelemetry.Logs.LoggerProvider! provider, int timeoutMilliseconds = -1) -> bool -static OpenTelemetry.Metrics.MeterProviderBuilderExtensions.SetExemplarFilter(this OpenTelemetry.Metrics.MeterProviderBuilder! meterProviderBuilder, OpenTelemetry.Metrics.ExemplarFilter! exemplarFilter) -> OpenTelemetry.Metrics.MeterProviderBuilder! +static OpenTelemetry.Metrics.MeterProviderBuilderExtensions.SetExemplarFilter(this OpenTelemetry.Metrics.MeterProviderBuilder! meterProviderBuilder, OpenTelemetry.Metrics.ExemplarFilterType exemplarFilter = OpenTelemetry.Metrics.ExemplarFilterType.TraceBased) -> OpenTelemetry.Metrics.MeterProviderBuilder! static OpenTelemetry.OpenTelemetryBuilderSdkExtensions.WithLogging(this OpenTelemetry.IOpenTelemetryBuilder! builder) -> OpenTelemetry.IOpenTelemetryBuilder! static OpenTelemetry.OpenTelemetryBuilderSdkExtensions.WithLogging(this OpenTelemetry.IOpenTelemetryBuilder! builder, System.Action! configure) -> OpenTelemetry.IOpenTelemetryBuilder! static OpenTelemetry.OpenTelemetryBuilderSdkExtensions.WithLogging(this OpenTelemetry.IOpenTelemetryBuilder! builder, System.Action? configureBuilder, System.Action? configureOptions) -> OpenTelemetry.IOpenTelemetryBuilder! static OpenTelemetry.Sdk.CreateLoggerProviderBuilder() -> OpenTelemetry.Logs.LoggerProviderBuilder! -abstract OpenTelemetry.Metrics.ExemplarFilter.ShouldSample(double value, System.ReadOnlySpan> tags) -> bool -abstract OpenTelemetry.Metrics.ExemplarFilter.ShouldSample(long value, System.ReadOnlySpan> tags) -> bool -override OpenTelemetry.Metrics.AlwaysOffExemplarFilter.ShouldSample(double value, System.ReadOnlySpan> tags) -> bool -override OpenTelemetry.Metrics.AlwaysOffExemplarFilter.ShouldSample(long value, System.ReadOnlySpan> tags) -> bool -override OpenTelemetry.Metrics.AlwaysOnExemplarFilter.ShouldSample(double value, System.ReadOnlySpan> tags) -> bool -override OpenTelemetry.Metrics.AlwaysOnExemplarFilter.ShouldSample(long value, System.ReadOnlySpan> tags) -> bool -override OpenTelemetry.Metrics.TraceBasedExemplarFilter.ShouldSample(double value, System.ReadOnlySpan> tags) -> bool -override OpenTelemetry.Metrics.TraceBasedExemplarFilter.ShouldSample(long value, System.ReadOnlySpan> tags) -> bool static Microsoft.Extensions.Logging.OpenTelemetryLoggingExtensions.UseOpenTelemetry(this Microsoft.Extensions.Logging.ILoggingBuilder! builder) -> Microsoft.Extensions.Logging.ILoggingBuilder! static Microsoft.Extensions.Logging.OpenTelemetryLoggingExtensions.UseOpenTelemetry(this Microsoft.Extensions.Logging.ILoggingBuilder! builder, System.Action! configure) -> Microsoft.Extensions.Logging.ILoggingBuilder! static Microsoft.Extensions.Logging.OpenTelemetryLoggingExtensions.UseOpenTelemetry(this Microsoft.Extensions.Logging.ILoggingBuilder! builder, System.Action? configureBuilder, System.Action? configureOptions) -> Microsoft.Extensions.Logging.ILoggingBuilder! diff --git a/src/OpenTelemetry/CHANGELOG.md b/src/OpenTelemetry/CHANGELOG.md index 755d94e0b3b..2958b18487d 100644 --- a/src/OpenTelemetry/CHANGELOG.md +++ b/src/OpenTelemetry/CHANGELOG.md @@ -2,12 +2,17 @@ ## Unreleased +* Throw NotSupportedException when using `SetErrorStatusOnException` method for + Tracing in Mono Runtime and Native AOT environment because the dependent + `Marshal.GetExceptionPointers()` API is not supported on these platforms. + ([#5374](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5374)) + * Fixed an issue where `LogRecord.Attributes` (or `LogRecord.StateValues` alias) could become out of sync with `LogRecord.State` if either is set directly via the public setters. This was done to further mitigate issues introduced in 1.5.0 causing attributes added using custom processor(s) to be missing after upgrading. For details see: - [#5169](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5169) + ([#5169](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5169)) * Fixed an issue where `SimpleExemplarReservoir` was not resetting internal state for cumulative temporality. @@ -55,6 +60,16 @@ API. ([#5396](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5396)) +* **Experimental (pre-release builds only):** Removed the `ExemplarFilter`, + `AlwaysOffExemplarFilter`, `AlwaysOnExemplarFilter`, and + `TraceBasedExemplarFilter` APIs. The `MeterProviderBuilder.SetExemplarFilter` + extension method now accepts an `ExemplarFilterType` enumeration (which + contains definitions for the supported filter types `AlwaysOff`, `AlwaysOn`, + and `TraceBased`) instead of an `ExemplarFilter` instance. This was done in + response to changes made to the [OpenTelemetry Metrics SDK + Specification](https://github.com/open-telemetry/opentelemetry-specification/pull/3820). + ([#5404](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5404)) + ## 1.7.0 Released 2023-Dec-08 diff --git a/src/OpenTelemetry/Metrics/AggregatorStore.cs b/src/OpenTelemetry/Metrics/AggregatorStore.cs index 65c9c2c2df7..525d3ba7566 100644 --- a/src/OpenTelemetry/Metrics/AggregatorStore.cs +++ b/src/OpenTelemetry/Metrics/AggregatorStore.cs @@ -996,7 +996,7 @@ private void UpdateLongCustomTags(long value, ReadOnlySpan - /// Sets the to be used for this provider. - /// This is applied to all the metrics from this provider. + /// Sets the to be used for this provider + /// which controls how measurements will be offered to exemplar reservoirs. + /// Default provider configuration: . /// - /// - /// . - /// ExemplarFilter to use. - /// The supplied for chaining. + /// + /// + /// Note: Use or to enable exemplars. + /// Specification: . + /// + /// . + /// to + /// use. + /// The supplied for + /// chaining. #if NET8_0_OR_GREATER [Experimental(DiagnosticDefinitions.ExemplarExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)] #endif public #else - /// - /// Sets the to be used for this provider. - /// This is applied to all the metrics from this provider. - /// - /// . - /// ExemplarFilter to use. - /// The supplied for chaining. internal #endif - static MeterProviderBuilder SetExemplarFilter(this MeterProviderBuilder meterProviderBuilder, ExemplarFilter exemplarFilter) + static MeterProviderBuilder SetExemplarFilter( + this MeterProviderBuilder meterProviderBuilder, + ExemplarFilterType exemplarFilter = ExemplarFilterType.TraceBased) { - Guard.ThrowIfNull(exemplarFilter); - meterProviderBuilder.ConfigureBuilder((sp, builder) => { if (builder is MeterProviderBuilderSdk meterProviderBuilderSdk) { - meterProviderBuilderSdk.SetExemplarFilter(exemplarFilter); + switch (exemplarFilter) + { + case ExemplarFilterType.AlwaysOn: + meterProviderBuilderSdk.SetExemplarFilter(new AlwaysOnExemplarFilter()); + break; + case ExemplarFilterType.AlwaysOff: + meterProviderBuilderSdk.SetExemplarFilter(new AlwaysOffExemplarFilter()); + break; + case ExemplarFilterType.TraceBased: + meterProviderBuilderSdk.SetExemplarFilter(new TraceBasedExemplarFilter()); + break; + default: + throw new NotSupportedException($"SdkExemplarFilter '{exemplarFilter}' is not supported."); + } } }); diff --git a/src/OpenTelemetry/Metrics/Exemplar/AlwaysOffExemplarFilter.cs b/src/OpenTelemetry/Metrics/Exemplar/AlwaysOffExemplarFilter.cs index 6939e0b96c9..5f40db2de36 100644 --- a/src/OpenTelemetry/Metrics/Exemplar/AlwaysOffExemplarFilter.cs +++ b/src/OpenTelemetry/Metrics/Exemplar/AlwaysOffExemplarFilter.cs @@ -1,31 +1,17 @@ // Copyright The OpenTelemetry Authors // SPDX-License-Identifier: Apache-2.0 -#if EXPOSE_EXPERIMENTAL_FEATURES && NET8_0_OR_GREATER -using System.Diagnostics.CodeAnalysis; -using OpenTelemetry.Internal; -#endif - namespace OpenTelemetry.Metrics; -#if EXPOSE_EXPERIMENTAL_FEATURES /// /// An implementation which makes no measurements /// eligible for becoming an . /// /// -/// /// Specification: . /// -#if NET8_0_OR_GREATER -[Experimental(DiagnosticDefinitions.ExemplarExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)] -#endif -public -#else -internal -#endif -sealed class AlwaysOffExemplarFilter : ExemplarFilter +internal sealed class AlwaysOffExemplarFilter : ExemplarFilter { /// public override bool ShouldSample(long value, ReadOnlySpan> tags) diff --git a/src/OpenTelemetry/Metrics/Exemplar/AlwaysOnExemplarFilter.cs b/src/OpenTelemetry/Metrics/Exemplar/AlwaysOnExemplarFilter.cs index b81a144df8b..67f2e4ced5a 100644 --- a/src/OpenTelemetry/Metrics/Exemplar/AlwaysOnExemplarFilter.cs +++ b/src/OpenTelemetry/Metrics/Exemplar/AlwaysOnExemplarFilter.cs @@ -1,34 +1,17 @@ // Copyright The OpenTelemetry Authors // SPDX-License-Identifier: Apache-2.0 -#if EXPOSE_EXPERIMENTAL_FEATURES && NET8_0_OR_GREATER -using System.Diagnostics.CodeAnalysis; -using OpenTelemetry.Internal; -#endif - namespace OpenTelemetry.Metrics; -#if EXPOSE_EXPERIMENTAL_FEATURES /// /// An implementation which makes all measurements /// eligible for becoming an . /// /// -/// /// Specification: . /// -#if NET8_0_OR_GREATER -[Experimental(DiagnosticDefinitions.ExemplarExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)] -#endif -public -#else -/// -/// An ExemplarFilter which makes all measurements eligible for being an Exemplar. -/// -internal -#endif - sealed class AlwaysOnExemplarFilter : ExemplarFilter +internal sealed class AlwaysOnExemplarFilter : ExemplarFilter { /// public override bool ShouldSample(long value, ReadOnlySpan> tags) diff --git a/src/OpenTelemetry/Metrics/Exemplar/ExemplarFilter.cs b/src/OpenTelemetry/Metrics/Exemplar/ExemplarFilter.cs index b0895a52b11..20296b5540d 100644 --- a/src/OpenTelemetry/Metrics/Exemplar/ExemplarFilter.cs +++ b/src/OpenTelemetry/Metrics/Exemplar/ExemplarFilter.cs @@ -1,30 +1,16 @@ // Copyright The OpenTelemetry Authors // SPDX-License-Identifier: Apache-2.0 -#if EXPOSE_EXPERIMENTAL_FEATURES && NET8_0_OR_GREATER -using System.Diagnostics.CodeAnalysis; -using OpenTelemetry.Internal; -#endif - namespace OpenTelemetry.Metrics; -#if EXPOSE_EXPERIMENTAL_FEATURES /// /// ExemplarFilter base implementation and contract. /// /// -/// /// Specification: . /// -#if NET8_0_OR_GREATER -[Experimental(DiagnosticDefinitions.ExemplarExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)] -#endif -public -#else -internal -#endif - abstract class ExemplarFilter +internal abstract class ExemplarFilter { /// /// Determines if a given measurement is eligible for being diff --git a/src/OpenTelemetry/Metrics/Exemplar/ExemplarFilterType.cs b/src/OpenTelemetry/Metrics/Exemplar/ExemplarFilterType.cs new file mode 100644 index 00000000000..959d1f8e42f --- /dev/null +++ b/src/OpenTelemetry/Metrics/Exemplar/ExemplarFilterType.cs @@ -0,0 +1,62 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +using System.Diagnostics; +#if EXPOSE_EXPERIMENTAL_FEATURES && NET8_0_OR_GREATER +using System.Diagnostics.CodeAnalysis; +using OpenTelemetry.Internal; +#endif + +namespace OpenTelemetry.Metrics; + +#if EXPOSE_EXPERIMENTAL_FEATURES +/// +/// Defines the supported exemplar filters. +/// +/// +/// +/// Specification: . +/// +#if NET8_0_OR_GREATER +[Experimental(DiagnosticDefinitions.ExemplarExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)] +#endif +public +#else +internal +#endif + enum ExemplarFilterType +{ + /// + /// An exemplar filter which makes no measurements eligible for becoming an + /// . + /// + /// + /// Note: Setting on a meter provider + /// effectively disables exemplars. + /// Specification: . + /// + AlwaysOff, + + /// + /// An exemplar filter which makes all measurements eligible for becoming an + /// . + /// + /// + /// Specification: . + /// + AlwaysOn, + + /// + /// An exemplar filter which makes measurements recorded in the context of a + /// sampled (span) eligible for becoming an . + /// + /// + /// Specification: . + /// + TraceBased, +} diff --git a/src/OpenTelemetry/Metrics/Exemplar/ReadOnlyExemplarCollection.cs b/src/OpenTelemetry/Metrics/Exemplar/ReadOnlyExemplarCollection.cs index 5f5fdae68ad..4a43bf3ebc6 100644 --- a/src/OpenTelemetry/Metrics/Exemplar/ReadOnlyExemplarCollection.cs +++ b/src/OpenTelemetry/Metrics/Exemplar/ReadOnlyExemplarCollection.cs @@ -23,6 +23,7 @@ namespace OpenTelemetry.Metrics; #endif readonly struct ReadOnlyExemplarCollection { + internal static readonly ReadOnlyExemplarCollection Empty = new(Array.Empty()); private readonly Exemplar[] exemplars; internal ReadOnlyExemplarCollection(Exemplar[] exemplars) @@ -50,24 +51,37 @@ public Enumerator GetEnumerator() internal ReadOnlyExemplarCollection Copy() { - var exemplarCopies = new Exemplar[this.exemplars.Length]; + var maximumCount = this.MaximumCount; - int i = 0; - foreach (ref readonly var exemplar in this) + if (maximumCount > 0) { - exemplar.Copy(ref exemplarCopies[i++]); + var exemplarCopies = new Exemplar[maximumCount]; + + int i = 0; + foreach (ref readonly var exemplar in this) + { + if (exemplar.IsUpdated()) + { + exemplar.Copy(ref exemplarCopies[i++]); + } + } + + return new ReadOnlyExemplarCollection(exemplarCopies); } - return new ReadOnlyExemplarCollection(exemplarCopies); + return Empty; } internal IReadOnlyList ToReadOnlyList() { var list = new List(this.MaximumCount); - foreach (var item in this) + foreach (var exemplar in this) { - list.Add(item); + // Note: If ToReadOnlyList is ever made public it should make sure + // to take copies of exemplars or make sure the instance was first + // copied using the Copy method above. + list.Add(exemplar); } return list; diff --git a/src/OpenTelemetry/Metrics/Exemplar/TraceBasedExemplarFilter.cs b/src/OpenTelemetry/Metrics/Exemplar/TraceBasedExemplarFilter.cs index 915a050dc98..db1b16a0b15 100644 --- a/src/OpenTelemetry/Metrics/Exemplar/TraceBasedExemplarFilter.cs +++ b/src/OpenTelemetry/Metrics/Exemplar/TraceBasedExemplarFilter.cs @@ -1,34 +1,20 @@ // Copyright The OpenTelemetry Authors // SPDX-License-Identifier: Apache-2.0 -#if EXPOSE_EXPERIMENTAL_FEATURES && NET8_0_OR_GREATER -using System.Diagnostics.CodeAnalysis; -using OpenTelemetry.Internal; -#endif - using System.Diagnostics; namespace OpenTelemetry.Metrics; -#if EXPOSE_EXPERIMENTAL_FEATURES /// /// An implementation which makes measurements /// recorded in the context of a sampled (span) eligible /// for becoming an . /// /// -/// /// Specification: . /// -#if NET8_0_OR_GREATER -[Experimental(DiagnosticDefinitions.ExemplarExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)] -#endif -public -#else -internal -#endif - sealed class TraceBasedExemplarFilter : ExemplarFilter +internal sealed class TraceBasedExemplarFilter : ExemplarFilter { /// public override bool ShouldSample(long value, ReadOnlySpan> tags) diff --git a/src/OpenTelemetry/Metrics/MetricPoint.cs b/src/OpenTelemetry/Metrics/MetricPoint.cs index d9a3eef3a98..2ccba6a307a 100644 --- a/src/OpenTelemetry/Metrics/MetricPoint.cs +++ b/src/OpenTelemetry/Metrics/MetricPoint.cs @@ -2,7 +2,6 @@ // SPDX-License-Identifier: Apache-2.0 using System.Diagnostics; -using System.Diagnostics.CodeAnalysis; using System.Runtime.CompilerServices; using OpenTelemetry.Internal; @@ -373,10 +372,10 @@ public readonly bool TryGetHistogramMinMaxValues(out double min, out double max) [MethodImpl(MethodImplOptions.AggressiveInlining)] internal #endif - readonly bool TryGetExemplars([NotNullWhen(true)] out ReadOnlyExemplarCollection? exemplars) + readonly bool TryGetExemplars(out ReadOnlyExemplarCollection exemplars) { - exemplars = this.mpComponents?.Exemplars; - return exemplars.HasValue; + exemplars = this.mpComponents?.Exemplars ?? ReadOnlyExemplarCollection.Empty; + return exemplars.MaximumCount > 0; } internal readonly MetricPoint Copy() @@ -445,23 +444,7 @@ internal void Update(long number) } } - // There is a race with Snapshot: - // Update() updates the value - // Snapshot snapshots the value - // Snapshot sets status to NoCollectPending - // Update sets status to CollectPending -- this is not right as the Snapshot - // already included the updated value. - // In the absence of any new Update call until next Snapshot, - // this results in exporting an Update even though - // it had no update. - // TODO: For Delta, this can be mitigated - // by ignoring Zero points - this.MetricPointStatus = MetricPointStatus.CollectPending; - - if (this.aggregatorStore.OutputDeltaWithUnusedMetricPointReclaimEnabled) - { - Interlocked.Decrement(ref this.ReferenceCount); - } + this.CompleteUpdate(); } internal void UpdateWithExemplar(long number, ReadOnlySpan> tags, bool isSampled) @@ -479,15 +462,7 @@ internal void UpdateWithExemplar(long number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); @@ -500,15 +475,7 @@ internal void UpdateWithExemplar(long number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); @@ -521,15 +488,7 @@ internal void UpdateWithExemplar(long number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); @@ -538,58 +497,42 @@ internal void UpdateWithExemplar(long number, ReadOnlySpan> tags, bool isSampled) @@ -685,15 +612,7 @@ internal void UpdateWithExemplar(double number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); @@ -709,15 +628,7 @@ internal void UpdateWithExemplar(double number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); @@ -733,15 +644,7 @@ internal void UpdateWithExemplar(double number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); @@ -750,58 +653,42 @@ internal void UpdateWithExemplar(double number, ReadOnlySpan> tags = default, bool reportExemplar = false, bool isSampled = false) + private void UpdateHistogram(double number, ReadOnlySpan> tags = default, bool isSampled = false) { Debug.Assert(this.mpComponents?.HistogramBuckets != null, "HistogramBuckets was null"); @@ -1321,20 +1209,12 @@ private void UpdateHistogram(double number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); } - private void UpdateHistogramWithMinMax(double number, ReadOnlySpan> tags = default, bool reportExemplar = false, bool isSampled = false) + private void UpdateHistogramWithMinMax(double number, ReadOnlySpan> tags = default, bool isSampled = false) { Debug.Assert(this.mpComponents?.HistogramBuckets != null, "HistogramBuckets was null"); @@ -1350,26 +1230,18 @@ private void UpdateHistogramWithMinMax(double number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); } - private void UpdateHistogramWithBuckets(double number, ReadOnlySpan> tags = default, bool reportExemplar = false, bool isSampled = false) + private void UpdateHistogramWithBuckets(double number, ReadOnlySpan> tags = default, bool isSampled = false) { Debug.Assert(this.mpComponents?.HistogramBuckets != null, "HistogramBuckets was null"); var histogramBuckets = this.mpComponents!.HistogramBuckets; - int i = histogramBuckets!.FindBucketIndex(number); + int bucketIndex = histogramBuckets!.FindBucketIndex(number); this.mpComponents.AcquireLock(); @@ -1377,29 +1249,21 @@ private void UpdateHistogramWithBuckets(double number, ReadOnlySpan(number, tags, i)); - } + this.OfferExplicitBucketHistogramExemplarIfSampled(number, tags, bucketIndex, isSampled); this.mpComponents.ReleaseLock(); } - private void UpdateHistogramWithBucketsAndMinMax(double number, ReadOnlySpan> tags = default, bool reportExemplar = false, bool isSampled = false) + private void UpdateHistogramWithBucketsAndMinMax(double number, ReadOnlySpan> tags = default, bool isSampled = false) { Debug.Assert(this.mpComponents?.HistogramBuckets != null, "histogramBuckets was null"); var histogramBuckets = this.mpComponents!.HistogramBuckets; - int i = histogramBuckets!.FindBucketIndex(number); + int bucketIndex = histogramBuckets!.FindBucketIndex(number); this.mpComponents.AcquireLock(); @@ -1407,26 +1271,18 @@ private void UpdateHistogramWithBucketsAndMinMax(double number, ReadOnlySpan(number, tags, i)); - } + this.OfferExplicitBucketHistogramExemplarIfSampled(number, tags, bucketIndex, isSampled); this.mpComponents.ReleaseLock(); } - private void UpdateBase2ExponentialHistogram(double number, ReadOnlySpan> tags = default, bool reportExemplar = false, bool isSampled = false) + private void UpdateBase2ExponentialHistogram(double number, ReadOnlySpan> tags = default, bool isSampled = false) { if (number < 0) { @@ -1446,20 +1302,12 @@ private void UpdateBase2ExponentialHistogram(double number, ReadOnlySpan(number, tags)); - } + this.OfferExemplarIfSampled(number, tags, isSampled); this.mpComponents.ReleaseLock(); } - private void UpdateBase2ExponentialHistogramWithMinMax(double number, ReadOnlySpan> tags = default, bool reportExemplar = false, bool isSampled = false) + private void UpdateBase2ExponentialHistogramWithMinMax(double number, ReadOnlySpan> tags = default, bool isSampled = false) { if (number < 0) { @@ -1482,17 +1330,78 @@ private void UpdateBase2ExponentialHistogramWithMinMax(double number, ReadOnlySp histogram.RunningMax = Math.Max(histogram.RunningMax, number); } - if (reportExemplar && isSampled) + this.OfferExemplarIfSampled(number, tags, isSampled); + + this.mpComponents.ReleaseLock(); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private readonly void OfferExemplarIfSampled(T number, ReadOnlySpan> tags, bool isSampled) + where T : struct + { + if (isSampled) + { + Debug.Assert(this.mpComponents?.ExemplarReservoir != null, "ExemplarReservoir was null"); + + // TODO: Need to ensure that the lock is always released. + // A custom implementation of `ExemplarReservoir.Offer` might throw an exception. + if (typeof(T) == typeof(long)) + { + this.mpComponents!.ExemplarReservoir!.Offer( + new ExemplarMeasurement((long)(object)number, tags)); + } + else if (typeof(T) == typeof(double)) + { + this.mpComponents!.ExemplarReservoir!.Offer( + new ExemplarMeasurement((double)(object)number, tags)); + } + else + { + Debug.Fail("Unexpected type"); + this.mpComponents!.ExemplarReservoir!.Offer( + new ExemplarMeasurement(Convert.ToDouble(number), tags)); + } + } + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private readonly void OfferExplicitBucketHistogramExemplarIfSampled( + double number, + ReadOnlySpan> tags, + int bucketIndex, + bool isSampled) + { + if (isSampled) { - Debug.Assert(this.mpComponents.ExemplarReservoir != null, "ExemplarReservoir was null"); + Debug.Assert(this.mpComponents?.ExemplarReservoir != null, "ExemplarReservoir was null"); // TODO: Need to ensure that the lock is always released. // A custom implementation of `ExemplarReservoir.Offer` might throw an exception. - this.mpComponents.ExemplarReservoir!.Offer( - new ExemplarMeasurement(number, tags)); + this.mpComponents!.ExemplarReservoir!.Offer( + new ExemplarMeasurement(number, tags, bucketIndex)); } + } - this.mpComponents.ReleaseLock(); + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private void CompleteUpdate() + { + // There is a race with Snapshot: + // Update() updates the value + // Snapshot snapshots the value + // Snapshot sets status to NoCollectPending + // Update sets status to CollectPending -- this is not right as the Snapshot + // already included the updated value. + // In the absence of any new Update call until next Snapshot, + // this results in exporting an Update even though + // it had no update. + // TODO: For Delta, this can be mitigated + // by ignoring Zero points + this.MetricPointStatus = MetricPointStatus.CollectPending; + + if (this.aggregatorStore.OutputDeltaWithUnusedMetricPointReclaimEnabled) + { + Interlocked.Decrement(ref this.ReferenceCount); + } } [MethodImpl(MethodImplOptions.NoInlining)] diff --git a/src/OpenTelemetry/Metrics/MetricPointOptionalComponents.cs b/src/OpenTelemetry/Metrics/MetricPointOptionalComponents.cs index 84511b1b549..440a9cc36b6 100644 --- a/src/OpenTelemetry/Metrics/MetricPointOptionalComponents.cs +++ b/src/OpenTelemetry/Metrics/MetricPointOptionalComponents.cs @@ -20,7 +20,7 @@ internal sealed class MetricPointOptionalComponents public ExemplarReservoir? ExemplarReservoir; - public ReadOnlyExemplarCollection? Exemplars; + public ReadOnlyExemplarCollection Exemplars = ReadOnlyExemplarCollection.Empty; private int isCriticalSectionOccupied = 0; @@ -30,7 +30,7 @@ public MetricPointOptionalComponents Copy() { HistogramBuckets = this.HistogramBuckets?.Copy(), Base2ExponentialBucketHistogram = this.Base2ExponentialBucketHistogram?.Copy(), - Exemplars = this.Exemplars?.Copy(), + Exemplars = this.Exemplars.Copy(), }; return copy; diff --git a/src/OpenTelemetry/Trace/Builder/TracerProviderBuilderExtensions.cs b/src/OpenTelemetry/Trace/Builder/TracerProviderBuilderExtensions.cs index 76f23be65f4..a4c39becf16 100644 --- a/src/OpenTelemetry/Trace/Builder/TracerProviderBuilderExtensions.cs +++ b/src/OpenTelemetry/Trace/Builder/TracerProviderBuilderExtensions.cs @@ -24,6 +24,12 @@ public static class TracerProviderBuilderExtensions /// . /// Enabled or not. Default value is true. /// Returns for chaining. + /// + /// This method is not supported in native AOT or Mono Runtime as of .NET 8. + /// +#if NET7_0_OR_GREATER + [RequiresDynamicCode("The code for detecting exception and setting error status might not be available.")] +#endif public static TracerProviderBuilder SetErrorStatusOnException(this TracerProviderBuilder tracerProviderBuilder, bool enabled = true) { tracerProviderBuilder.ConfigureBuilder((sp, builder) => diff --git a/src/OpenTelemetry/Trace/ExceptionProcessor.cs b/src/OpenTelemetry/Trace/ExceptionProcessor.cs index 815670792d1..f0084d36cfb 100644 --- a/src/OpenTelemetry/Trace/ExceptionProcessor.cs +++ b/src/OpenTelemetry/Trace/ExceptionProcessor.cs @@ -23,19 +23,13 @@ public ExceptionProcessor() #else // When running on netstandard or similar the Marshal class is not a part of the netstandard API // but it would still most likely be available in the underlying framework, so use reflection to retrieve it. - try - { - var flags = BindingFlags.Static | BindingFlags.Public; - var method = typeof(Marshal).GetMethod("GetExceptionPointers", flags, null, Type.EmptyTypes, null) - ?? throw new InvalidOperationException("Marshal.GetExceptionPointers method could not be resolved reflectively."); - var lambda = Expression.Lambda>(Expression.Call(method)); - this.fnGetExceptionPointers = lambda.Compile(); - } - catch (Exception ex) - { - throw new NotSupportedException($"'{typeof(Marshal).FullName}.GetExceptionPointers' is not supported", ex); - } + var flags = BindingFlags.Static | BindingFlags.Public; + var method = typeof(Marshal).GetMethod("GetExceptionPointers", flags, null, Type.EmptyTypes, null) + ?? throw new InvalidOperationException("Marshal.GetExceptionPointers method could not be resolved reflectively."); + var lambda = Expression.Lambda>(Expression.Call(method)); + this.fnGetExceptionPointers = lambda.Compile(); #endif + this.fnGetExceptionPointers(); // attempt to access pointers to test for platform support } /// diff --git a/test/Benchmarks/Metrics/ExemplarBenchmarks.cs b/test/Benchmarks/Metrics/ExemplarBenchmarks.cs index 81d285ea8b4..ac1d347fba7 100644 --- a/test/Benchmarks/Metrics/ExemplarBenchmarks.cs +++ b/test/Benchmarks/Metrics/ExemplarBenchmarks.cs @@ -9,21 +9,31 @@ using OpenTelemetry.Tests; /* -BenchmarkDotNet v0.13.10, Windows 11 (10.0.23424.1000) -Intel Core i7-9700 CPU 3.00GHz, 1 CPU, 8 logical and 8 physical cores -.NET SDK 8.0.100 - [Host] : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2 - DefaultJob : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2 - - -| Method | ExemplarFilter | Mean | Error | StdDev | Allocated | -|-------------------------- |--------------- |---------:|--------:|--------:|----------:| -| HistogramNoTagReduction | AlwaysOff | 274.2 ns | 2.94 ns | 2.60 ns | - | -| HistogramWithTagReduction | AlwaysOff | 241.6 ns | 1.78 ns | 1.58 ns | - | -| HistogramNoTagReduction | AlwaysOn | 300.9 ns | 3.10 ns | 2.90 ns | - | -| HistogramWithTagReduction | AlwaysOn | 312.9 ns | 4.81 ns | 4.50 ns | - | -| HistogramNoTagReduction | HighValueOnly | 262.8 ns | 2.24 ns | 1.99 ns | - | -| HistogramWithTagReduction | HighValueOnly | 258.3 ns | 5.12 ns | 5.03 ns | - | +BenchmarkDotNet v0.13.10, Windows 11 (10.0.22631.3155/23H2/2023Update/SunValley3) +12th Gen Intel Core i9-12900HK, 1 CPU, 20 logical and 14 physical cores +.NET SDK 8.0.200 + [Host] : .NET 8.0.2 (8.0.224.6711), X64 RyuJIT AVX2 + DefaultJob : .NET 8.0.2 (8.0.224.6711), X64 RyuJIT AVX2 + + +| Method | ExemplarConfiguration | Mean | Error | StdDev | Allocated | +|-------------------------- |---------------------- |---------:|--------:|--------:|----------:| +| HistogramNoTagReduction | AlwaysOff | 174.6 ns | 1.32 ns | 1.24 ns | - | +| HistogramWithTagReduction | AlwaysOff | 161.8 ns | 2.63 ns | 2.46 ns | - | +| CounterNoTagReduction | AlwaysOff | 141.6 ns | 2.12 ns | 1.77 ns | - | +| CounterWithTagReduction | AlwaysOff | 141.7 ns | 2.11 ns | 1.87 ns | - | +| HistogramNoTagReduction | AlwaysOn | 201.1 ns | 3.05 ns | 2.86 ns | - | +| HistogramWithTagReduction | AlwaysOn | 196.5 ns | 1.91 ns | 1.78 ns | - | +| CounterNoTagReduction | AlwaysOn | 149.7 ns | 1.42 ns | 1.33 ns | - | +| CounterWithTagReduction | AlwaysOn | 143.5 ns | 2.09 ns | 1.95 ns | - | +| HistogramNoTagReduction | TraceBased | 171.9 ns | 2.33 ns | 2.18 ns | - | +| HistogramWithTagReduction | TraceBased | 164.9 ns | 2.70 ns | 2.52 ns | - | +| CounterNoTagReduction | TraceBased | 148.1 ns | 2.76 ns | 2.58 ns | - | +| CounterWithTagReduction | TraceBased | 141.2 ns | 1.43 ns | 1.34 ns | - | +| HistogramNoTagReduction | Alway(...)pling [29] | 183.9 ns | 1.49 ns | 1.39 ns | - | +| HistogramWithTagReduction | Alway(...)pling [29] | 176.1 ns | 3.35 ns | 3.29 ns | - | +| CounterNoTagReduction | Alway(...)pling [29] | 159.3 ns | 3.12 ns | 4.38 ns | - | +| CounterWithTagReduction | Alway(...)pling [29] | 158.7 ns | 3.06 ns | 3.65 ns | - | */ namespace Benchmarks.Metrics; @@ -32,51 +42,74 @@ public class ExemplarBenchmarks { private static readonly ThreadLocal ThreadLocalRandom = new(() => new Random()); private readonly string[] dimensionValues = new string[] { "DimVal1", "DimVal2", "DimVal3", "DimVal4", "DimVal5", "DimVal6", "DimVal7", "DimVal8", "DimVal9", "DimVal10" }; - private Histogram histogramWithoutTagReduction; - - private Histogram histogramWithTagReduction; - + private Histogram histogramWithoutTagReduction; + private Histogram histogramWithTagReduction; + private Counter counterWithoutTagReduction; + private Counter counterWithTagReduction; private MeterProvider meterProvider; private Meter meter; [System.Diagnostics.CodeAnalysis.SuppressMessage("StyleCop.CSharp.DocumentationRules", "SA1602:Enumeration items should be documented", Justification = "Test only.")] - public enum ExemplarFilterToUse + public enum ExemplarConfigurationType { AlwaysOff, AlwaysOn, - HighValueOnly, + TraceBased, + AlwaysOnWithHighValueSampling, } - [Params(ExemplarFilterToUse.AlwaysOn, ExemplarFilterToUse.AlwaysOff, ExemplarFilterToUse.HighValueOnly)] - public ExemplarFilterToUse ExemplarFilter { get; set; } + [Params(ExemplarConfigurationType.AlwaysOn, ExemplarConfigurationType.AlwaysOff, ExemplarConfigurationType.TraceBased, ExemplarConfigurationType.AlwaysOnWithHighValueSampling)] + public ExemplarConfigurationType ExemplarConfiguration { get; set; } [GlobalSetup] public void Setup() { this.meter = new Meter(Utils.GetCurrentMethodName()); - this.histogramWithoutTagReduction = this.meter.CreateHistogram("HistogramWithoutTagReduction"); - this.histogramWithTagReduction = this.meter.CreateHistogram("HistogramWithTagReduction"); + this.histogramWithoutTagReduction = this.meter.CreateHistogram("HistogramWithoutTagReduction"); + this.histogramWithTagReduction = this.meter.CreateHistogram("HistogramWithTagReduction"); + this.counterWithoutTagReduction = this.meter.CreateCounter("CounterWithoutTagReduction"); + this.counterWithTagReduction = this.meter.CreateCounter("CounterWithTagReduction"); var exportedItems = new List(); - ExemplarFilter exemplarFilter = new AlwaysOffExemplarFilter(); - if (this.ExemplarFilter == ExemplarFilterToUse.AlwaysOn) - { - exemplarFilter = new AlwaysOnExemplarFilter(); - } - else if (this.ExemplarFilter == ExemplarFilterToUse.HighValueOnly) - { - exemplarFilter = new HighValueExemplarFilter(); - } + var exemplarFilter = this.ExemplarConfiguration == ExemplarConfigurationType.TraceBased + ? ExemplarFilterType.TraceBased + : this.ExemplarConfiguration != ExemplarConfigurationType.AlwaysOff + ? ExemplarFilterType.AlwaysOn + : ExemplarFilterType.AlwaysOff; this.meterProvider = Sdk.CreateMeterProviderBuilder() .AddMeter(this.meter.Name) .SetExemplarFilter(exemplarFilter) - .AddView("HistogramWithTagReduction", new MetricStreamConfiguration() { TagKeys = new string[] { "DimName1", "DimName2", "DimName3" } }) + .AddView(i => + { + if (i.Name.Contains("WithTagReduction")) + { + return new MetricStreamConfiguration() + { + TagKeys = new string[] { "DimName1", "DimName2", "DimName3" }, + ExemplarReservoirFactory = CreateExemplarReservoir, + }; + } + else + { + return new MetricStreamConfiguration() + { + ExemplarReservoirFactory = CreateExemplarReservoir, + }; + } + }) .AddInMemoryExporter(exportedItems, metricReaderOptions => { metricReaderOptions.PeriodicExportingMetricReaderOptions.ExportIntervalMilliseconds = 1000; }) .Build(); + + ExemplarReservoir CreateExemplarReservoir() + { + return this.ExemplarConfiguration == ExemplarConfigurationType.AlwaysOnWithHighValueSampling + ? new HighValueExemplarReservoir(800D) + : null; + } } [GlobalCleanup] @@ -99,7 +132,7 @@ public void HistogramNoTagReduction() { "DimName5", this.dimensionValues[random.Next(0, 10)] }, }; - this.histogramWithoutTagReduction.Record(random.Next(1000), tags); + this.histogramWithoutTagReduction.Record(random.NextDouble() * 1000D, tags); } [Benchmark] @@ -115,19 +148,71 @@ public void HistogramWithTagReduction() { "DimName5", this.dimensionValues[random.Next(0, 10)] }, }; - this.histogramWithTagReduction.Record(random.Next(1000), tags); + this.histogramWithTagReduction.Record(random.NextDouble() * 1000D, tags); + } + + [Benchmark] + public void CounterNoTagReduction() + { + var random = ThreadLocalRandom.Value; + var tags = new TagList + { + { "DimName1", this.dimensionValues[random.Next(0, 2)] }, + { "DimName2", this.dimensionValues[random.Next(0, 2)] }, + { "DimName3", this.dimensionValues[random.Next(0, 5)] }, + { "DimName4", this.dimensionValues[random.Next(0, 5)] }, + { "DimName5", this.dimensionValues[random.Next(0, 10)] }, + }; + + this.counterWithoutTagReduction.Add(random.Next(1000), tags); + } + + [Benchmark] + public void CounterWithTagReduction() + { + var random = ThreadLocalRandom.Value; + var tags = new TagList + { + { "DimName1", this.dimensionValues[random.Next(0, 2)] }, + { "DimName2", this.dimensionValues[random.Next(0, 2)] }, + { "DimName3", this.dimensionValues[random.Next(0, 5)] }, + { "DimName4", this.dimensionValues[random.Next(0, 5)] }, + { "DimName5", this.dimensionValues[random.Next(0, 10)] }, + }; + + this.counterWithTagReduction.Add(random.Next(1000), tags); } - internal class HighValueExemplarFilter : ExemplarFilter + private sealed class HighValueExemplarReservoir : FixedSizeExemplarReservoir { - public override bool ShouldSample(long value, ReadOnlySpan> tags) + private readonly double threshold; + private int measurementCount; + + public HighValueExemplarReservoir(double threshold) + : base(10) + { + this.threshold = threshold; + } + + public override void Offer(in ExemplarMeasurement measurement) + { + if (measurement.Value >= this.threshold) + { + this.UpdateExemplar(this.measurementCount++ % this.Capacity, in measurement); + } + } + + public override void Offer(in ExemplarMeasurement measurement) { - return value > 800; + if (measurement.Value >= this.threshold) + { + this.UpdateExemplar(this.measurementCount++ % this.Capacity, in measurement); + } } - public override bool ShouldSample(double value, ReadOnlySpan> tags) + protected override void OnCollected() { - return value > 800; + this.measurementCount = 0; } } } diff --git a/test/OpenTelemetry.Exporter.OpenTelemetryProtocol.Tests/OtlpMetricsExporterTests.cs b/test/OpenTelemetry.Exporter.OpenTelemetryProtocol.Tests/OtlpMetricsExporterTests.cs index 85713222ea6..7322c4d90e5 100644 --- a/test/OpenTelemetry.Exporter.OpenTelemetryProtocol.Tests/OtlpMetricsExporterTests.cs +++ b/test/OpenTelemetry.Exporter.OpenTelemetryProtocol.Tests/OtlpMetricsExporterTests.cs @@ -1,8 +1,10 @@ // Copyright The OpenTelemetry Authors // SPDX-License-Identifier: Apache-2.0 +using System.Diagnostics; using System.Diagnostics.Metrics; using System.Reflection; +using Google.Protobuf; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation; @@ -18,6 +20,12 @@ namespace OpenTelemetry.Exporter.OpenTelemetryProtocol.Tests; public class OtlpMetricsExporterTests : Http2UnencryptedSupportTests { + private static readonly KeyValuePair[] KeyValues = new KeyValuePair[] + { + new KeyValuePair("key1", "value1"), + new KeyValuePair("key2", 123), + }; + [Fact] public void TestAddOtlpExporter_SetsCorrectMetricReaderDefaults() { @@ -216,14 +224,17 @@ public void ToOtlpResourceMetricsTest(bool includeServiceNameInResource) [Theory] [InlineData("test_gauge", null, null, 123L, null)] [InlineData("test_gauge", null, null, null, 123.45)] + [InlineData("test_gauge", null, null, 123L, null, true)] + [InlineData("test_gauge", null, null, null, 123.45, true)] [InlineData("test_gauge", "description", "unit", 123L, null)] - public void TestGaugeToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue) + public void TestGaugeToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, bool enableExemplars = false) { var metrics = new List(); using var meter = new Meter(Utils.GetCurrentMethodName()); using var provider = Sdk.CreateMeterProviderBuilder() .AddMeter(meter.Name) + .SetExemplarFilter(enableExemplars ? ExemplarFilterType.AlwaysOn : ExemplarFilterType.AlwaysOff) .AddInMemoryExporter(metrics) .Build(); @@ -277,29 +288,35 @@ public void TestGaugeToOtlpMetric(string name, string description, string unit, Assert.Empty(dataPoint.Attributes); - Assert.Empty(dataPoint.Exemplars); + VerifyExemplars(longValue, doubleValue, enableExemplars, d => d.Exemplars.FirstOrDefault(), dataPoint); } [Theory] [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_counter", null, null, null, 123.45, MetricReaderTemporalityPreference.Cumulative)] + [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Cumulative, false, true)] + [InlineData("test_counter", null, null, null, 123.45, MetricReaderTemporalityPreference.Cumulative, false, true)] [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Delta)] + [InlineData("test_counter", null, null, null, 123.45, MetricReaderTemporalityPreference.Delta)] + [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, false, true)] + [InlineData("test_counter", null, null, null, 123.45, MetricReaderTemporalityPreference.Delta, false, true)] [InlineData("test_counter", "description", "unit", 123L, null, MetricReaderTemporalityPreference.Cumulative)] - [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, "key1", "value1", "key2", 123)] - public void TestCounterToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, MetricReaderTemporalityPreference aggregationTemporality, params object[] keysValues) + [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, true)] + public void TestCounterToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, MetricReaderTemporalityPreference aggregationTemporality, bool enableKeyValues = false, bool enableExemplars = false) { var metrics = new List(); using var meter = new Meter(Utils.GetCurrentMethodName()); using var provider = Sdk.CreateMeterProviderBuilder() .AddMeter(meter.Name) + .SetExemplarFilter(enableExemplars ? ExemplarFilterType.AlwaysOn : ExemplarFilterType.AlwaysOff) .AddInMemoryExporter(metrics, metricReaderOptions => { metricReaderOptions.TemporalityPreference = aggregationTemporality; }) .Build(); - var attributes = ToAttributes(keysValues).ToArray(); + var attributes = enableKeyValues ? KeyValues : Array.Empty>(); if (longValue.HasValue) { var counter = meter.CreateCounter(name, unit, description); @@ -366,31 +383,37 @@ public void TestCounterToOtlpMetric(string name, string description, string unit Assert.Empty(dataPoint.Attributes); } - Assert.Empty(dataPoint.Exemplars); + VerifyExemplars(longValue, doubleValue, enableExemplars, d => d.Exemplars.FirstOrDefault(), dataPoint); } [Theory] [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_counter", null, null, null, 123.45, MetricReaderTemporalityPreference.Cumulative)] + [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Cumulative, false, true)] + [InlineData("test_counter", null, null, null, 123.45, MetricReaderTemporalityPreference.Cumulative, false, true)] [InlineData("test_counter", null, null, -123L, null, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_counter", null, null, null, -123.45, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Delta)] + [InlineData("test_counter", null, null, null, -123.45, MetricReaderTemporalityPreference.Delta)] + [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, false, true)] + [InlineData("test_counter", null, null, null, -123.45, MetricReaderTemporalityPreference.Delta, false, true)] [InlineData("test_counter", "description", "unit", 123L, null, MetricReaderTemporalityPreference.Cumulative)] - [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, "key1", "value1", "key2", 123)] - public void TestUpDownCounterToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, MetricReaderTemporalityPreference aggregationTemporality, params object[] keysValues) + [InlineData("test_counter", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, true)] + public void TestUpDownCounterToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, MetricReaderTemporalityPreference aggregationTemporality, bool enableKeyValues = false, bool enableExemplars = false) { var metrics = new List(); using var meter = new Meter(Utils.GetCurrentMethodName()); using var provider = Sdk.CreateMeterProviderBuilder() .AddMeter(meter.Name) + .SetExemplarFilter(enableExemplars ? ExemplarFilterType.AlwaysOn : ExemplarFilterType.AlwaysOff) .AddInMemoryExporter(metrics, metricReaderOptions => { metricReaderOptions.TemporalityPreference = aggregationTemporality; }) .Build(); - var attributes = ToAttributes(keysValues).ToArray(); + var attributes = enableKeyValues ? KeyValues : Array.Empty>(); if (longValue.HasValue) { var counter = meter.CreateUpDownCounter(name, unit, description); @@ -457,24 +480,30 @@ public void TestUpDownCounterToOtlpMetric(string name, string description, strin Assert.Empty(dataPoint.Attributes); } - Assert.Empty(dataPoint.Exemplars); + VerifyExemplars(longValue, doubleValue, enableExemplars, d => d.Exemplars.FirstOrDefault(), dataPoint); } [Theory] [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_histogram", null, null, null, 123.45, MetricReaderTemporalityPreference.Cumulative)] + [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Cumulative, false, true)] + [InlineData("test_histogram", null, null, null, 123.45, MetricReaderTemporalityPreference.Cumulative, false, true)] [InlineData("test_histogram", null, null, -123L, null, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_histogram", null, null, null, -123.45, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Delta)] + [InlineData("test_histogram", null, null, null, 123.45, MetricReaderTemporalityPreference.Delta)] + [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, false, true)] + [InlineData("test_histogram", null, null, null, 123.45, MetricReaderTemporalityPreference.Delta, false, true)] [InlineData("test_histogram", "description", "unit", 123L, null, MetricReaderTemporalityPreference.Cumulative)] - [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, "key1", "value1", "key2", 123)] - public void TestExponentialHistogramToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, MetricReaderTemporalityPreference aggregationTemporality, params object[] keysValues) + [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, true)] + public void TestExponentialHistogramToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, MetricReaderTemporalityPreference aggregationTemporality, bool enableKeyValues = false, bool enableExemplars = false) { var metrics = new List(); using var meter = new Meter(Utils.GetCurrentMethodName()); using var provider = Sdk.CreateMeterProviderBuilder() .AddMeter(meter.Name) + .SetExemplarFilter(enableExemplars ? ExemplarFilterType.AlwaysOn : ExemplarFilterType.AlwaysOff) .AddInMemoryExporter(metrics, metricReaderOptions => { metricReaderOptions.TemporalityPreference = aggregationTemporality; @@ -485,7 +514,7 @@ public void TestExponentialHistogramToOtlpMetric(string name, string description }) .Build(); - var attributes = ToAttributes(keysValues).ToArray(); + var attributes = enableKeyValues ? KeyValues : Array.Empty>(); if (longValue.HasValue) { var histogram = meter.CreateHistogram(name, unit, description); @@ -587,31 +616,41 @@ public void TestExponentialHistogramToOtlpMetric(string name, string description Assert.Empty(dataPoint.Attributes); } - Assert.Empty(dataPoint.Exemplars); + VerifyExemplars(null, longValue ?? doubleValue, enableExemplars, d => d.Exemplars.FirstOrDefault(), dataPoint); + if (enableExemplars) + { + VerifyExemplars(null, 0, enableExemplars, d => d.Exemplars.Skip(1).FirstOrDefault(), dataPoint); + } } [Theory] [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_histogram", null, null, null, 123.45, MetricReaderTemporalityPreference.Cumulative)] + [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Cumulative, false, true)] + [InlineData("test_histogram", null, null, null, 123.45, MetricReaderTemporalityPreference.Cumulative, false, true)] [InlineData("test_histogram", null, null, -123L, null, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_histogram", null, null, null, -123.45, MetricReaderTemporalityPreference.Cumulative)] [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Delta)] + [InlineData("test_histogram", null, null, null, 123.45, MetricReaderTemporalityPreference.Delta)] + [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, false, true)] + [InlineData("test_histogram", null, null, null, 123.45, MetricReaderTemporalityPreference.Delta, false, true)] [InlineData("test_histogram", "description", "unit", 123L, null, MetricReaderTemporalityPreference.Cumulative)] - [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, "key1", "value1", "key2", 123)] - public void TestHistogramToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, MetricReaderTemporalityPreference aggregationTemporality, params object[] keysValues) + [InlineData("test_histogram", null, null, 123L, null, MetricReaderTemporalityPreference.Delta, true)] + public void TestHistogramToOtlpMetric(string name, string description, string unit, long? longValue, double? doubleValue, MetricReaderTemporalityPreference aggregationTemporality, bool enableKeyValues = false, bool enableExemplars = false) { var metrics = new List(); using var meter = new Meter(Utils.GetCurrentMethodName()); using var provider = Sdk.CreateMeterProviderBuilder() .AddMeter(meter.Name) + .SetExemplarFilter(enableExemplars ? ExemplarFilterType.AlwaysOn : ExemplarFilterType.AlwaysOff) .AddInMemoryExporter(metrics, metricReaderOptions => { metricReaderOptions.TemporalityPreference = aggregationTemporality; }) .Build(); - var attributes = ToAttributes(keysValues).ToArray(); + var attributes = enableKeyValues ? KeyValues : Array.Empty>(); if (longValue.HasValue) { var histogram = meter.CreateHistogram(name, unit, description); @@ -690,7 +729,7 @@ public void TestHistogramToOtlpMetric(string name, string description, string un Assert.Empty(dataPoint.Attributes); } - Assert.Empty(dataPoint.Exemplars); + VerifyExemplars(null, longValue ?? doubleValue, enableExemplars, d => d.Exemplars.FirstOrDefault(), dataPoint); } [Theory] @@ -737,14 +776,155 @@ public void TestTemporalityPreferenceConfiguration(string configValue, MetricRea Assert.Equal(expectedTemporality, temporality); } - private static IEnumerable> ToAttributes(object[] keysValues) + [Theory] + [InlineData(false, false)] + [InlineData(true, false)] + [InlineData(false, true)] + [InlineData(true, true)] + public void ToOtlpExemplarTests(bool enableTagFiltering, bool enableTracing) { - var keys = keysValues?.Where((_, index) => index % 2 == 0).ToArray(); - var values = keysValues?.Where((_, index) => index % 2 != 0).ToArray(); + ActivitySource activitySource = null; + Activity activity = null; + TracerProvider tracerProvider = null; + + using var meter = new Meter(Utils.GetCurrentMethodName()); + + var exportedItems = new List(); + + using var meterProvider = Sdk.CreateMeterProviderBuilder() + .AddMeter(meter.Name) + .SetExemplarFilter(ExemplarFilterType.AlwaysOn) + .AddView(i => + { + return !enableTagFiltering + ? null + : new MetricStreamConfiguration + { + TagKeys = Array.Empty(), + }; + }) + .AddInMemoryExporter(exportedItems) + .Build(); + + if (enableTracing) + { + activitySource = new ActivitySource(Utils.GetCurrentMethodName()); + tracerProvider = Sdk.CreateTracerProviderBuilder() + .AddSource(activitySource.Name) + .SetSampler(new AlwaysOnSampler()) + .Build(); + + activity = activitySource.StartActivity("testActivity"); + } + + var counterDouble = meter.CreateCounter("testCounterDouble"); + var counterLong = meter.CreateCounter("testCounterLong"); + + counterDouble.Add(1.18D, new KeyValuePair("key1", "value1")); + counterLong.Add(18L, new KeyValuePair("key1", "value1")); + + meterProvider.ForceFlush(); + + var counterDoubleMetric = exportedItems.FirstOrDefault(m => m.Name == counterDouble.Name); + var counterLongMetric = exportedItems.FirstOrDefault(m => m.Name == counterLong.Name); - for (var i = 0; keys != null && i < keys.Length; ++i) + Assert.NotNull(counterDoubleMetric); + Assert.NotNull(counterLongMetric); + + AssertExemplars(1.18D, counterDoubleMetric); + AssertExemplars(18L, counterLongMetric); + + activity?.Dispose(); + tracerProvider?.Dispose(); + activitySource?.Dispose(); + + void AssertExemplars(T value, Metric metric) + where T : struct + { + var metricPointEnumerator = metric.GetMetricPoints().GetEnumerator(); + Assert.True(metricPointEnumerator.MoveNext()); + + ref readonly var metricPoint = ref metricPointEnumerator.Current; + + var result = metricPoint.TryGetExemplars(out var exemplars); + Assert.True(result); + + var exemplarEnumerator = exemplars.GetEnumerator(); + Assert.True(exemplarEnumerator.MoveNext()); + + ref readonly var exemplar = ref exemplarEnumerator.Current; + + var otlpExemplar = MetricItemExtensions.ToOtlpExemplar(value, in exemplar); + Assert.NotNull(otlpExemplar); + + Assert.NotEqual(default, otlpExemplar.TimeUnixNano); + if (!enableTracing) + { + Assert.Equal(ByteString.Empty, otlpExemplar.TraceId); + Assert.Equal(ByteString.Empty, otlpExemplar.SpanId); + } + else + { + byte[] traceIdBytes = new byte[16]; + activity.TraceId.CopyTo(traceIdBytes); + + byte[] spanIdBytes = new byte[8]; + activity.SpanId.CopyTo(spanIdBytes); + + Assert.Equal(ByteString.CopyFrom(traceIdBytes), otlpExemplar.TraceId); + Assert.Equal(ByteString.CopyFrom(spanIdBytes), otlpExemplar.SpanId); + } + + if (typeof(T) == typeof(long)) + { + Assert.Equal((long)(object)value, exemplar.LongValue); + } + else if (typeof(T) == typeof(double)) + { + Assert.Equal((double)(object)value, exemplar.DoubleValue); + } + else + { + Debug.Fail("Unexpected type"); + } + + if (!enableTagFiltering) + { + var tagEnumerator = exemplar.FilteredTags.GetEnumerator(); + Assert.False(tagEnumerator.MoveNext()); + } + else + { + var tagEnumerator = exemplar.FilteredTags.GetEnumerator(); + Assert.True(tagEnumerator.MoveNext()); + + var tag = tagEnumerator.Current; + Assert.Equal("key1", tag.Key); + Assert.Equal("value1", tag.Value); + } + } + } + + private static void VerifyExemplars(long? longValue, double? doubleValue, bool enableExemplars, Func getExemplarFunc, T state) + { + var exemplar = getExemplarFunc(state); + + if (enableExemplars) + { + Assert.NotNull(exemplar); + Assert.NotEqual(default, exemplar.TimeUnixNano); + if (longValue.HasValue) + { + Assert.Equal(longValue.Value, exemplar.AsInt); + } + else + { + Assert.Equal(doubleValue.Value, exemplar.AsDouble); + } + } + else { - yield return new KeyValuePair(keys[i].ToString(), values[i]); + Assert.Null(exemplar); } } } diff --git a/test/OpenTelemetry.Tests.Stress.Metrics/Program.cs b/test/OpenTelemetry.Tests.Stress.Metrics/Program.cs index f43e4d12fe8..17360444c8e 100644 --- a/test/OpenTelemetry.Tests.Stress.Metrics/Program.cs +++ b/test/OpenTelemetry.Tests.Stress.Metrics/Program.cs @@ -56,7 +56,7 @@ public MetricsStressTest(MetricsStressTestOptions options) if (options.EnableExemplars) { - builder.SetExemplarFilter(new AlwaysOnExemplarFilter()); + builder.SetExemplarFilter(ExemplarFilterType.AlwaysOn); } if (options.AddViewToFilterTags) diff --git a/test/OpenTelemetry.Tests/Metrics/MetricExemplarTests.cs b/test/OpenTelemetry.Tests/Metrics/MetricExemplarTests.cs index e1dd5effe54..c0927683f86 100644 --- a/test/OpenTelemetry.Tests/Metrics/MetricExemplarTests.cs +++ b/test/OpenTelemetry.Tests/Metrics/MetricExemplarTests.cs @@ -28,7 +28,7 @@ public void TestExemplarsCounter(MetricReaderTemporalityPreference temporality) using var container = this.BuildMeterProvider(out var meterProvider, builder => builder .AddMeter(meter.Name) - .SetExemplarFilter(new AlwaysOnExemplarFilter()) + .SetExemplarFilter(ExemplarFilterType.AlwaysOn) .AddView(i => { if (i.Name.StartsWith("testCounter")) @@ -153,7 +153,7 @@ public void TestExemplarsObservable(MetricReaderTemporalityPreference temporalit using var container = this.BuildMeterProvider(out var meterProvider, builder => builder .AddMeter(meter.Name) - .SetExemplarFilter(new AlwaysOnExemplarFilter()) + .SetExemplarFilter(ExemplarFilterType.AlwaysOn) .AddInMemoryExporter(exportedItems, metricReaderOptions => { metricReaderOptions.TemporalityPreference = temporality; @@ -237,7 +237,7 @@ public void TestExemplarsHistogramWithBuckets(MetricReaderTemporalityPreference using var container = this.BuildMeterProvider(out var meterProvider, builder => builder .AddMeter(meter.Name) - .SetExemplarFilter(new AlwaysOnExemplarFilter()) + .SetExemplarFilter(ExemplarFilterType.AlwaysOn) .AddView(i => { if (i.Name.StartsWith("histogramWithBucketsAndMinMax")) @@ -367,7 +367,7 @@ public void TestExemplarsHistogramWithoutBuckets(MetricReaderTemporalityPreferen using var container = this.BuildMeterProvider(out var meterProvider, builder => builder .AddMeter(meter.Name) - .SetExemplarFilter(new AlwaysOnExemplarFilter()) + .SetExemplarFilter(ExemplarFilterType.AlwaysOn) .AddView(i => { if (i.Name.StartsWith("histogramWithoutBucketsAndMinMax")) @@ -495,7 +495,7 @@ public void TestExemplarsExponentialHistogram(MetricReaderTemporalityPreference using var container = this.BuildMeterProvider(out var meterProvider, builder => builder .AddMeter(meter.Name) - .SetExemplarFilter(new AlwaysOnExemplarFilter()) + .SetExemplarFilter(ExemplarFilterType.AlwaysOn) .AddView(i => { if (i.Name.StartsWith("exponentialHistogramWithMinMax")) @@ -601,6 +601,53 @@ static void ValidateSecondPhase( } } + [Theory] + [InlineData(true)] + [InlineData(false)] + public void TestTraceBasedExemplarFilter(bool enableTracing) + { + var exportedItems = new List(); + + using var meter = new Meter($"{Utils.GetCurrentMethodName()}"); + + var counter = meter.CreateCounter("testCounter"); + + using var container = this.BuildMeterProvider(out var meterProvider, builder => builder + .AddMeter(meter.Name) + .SetExemplarFilter(ExemplarFilterType.TraceBased) + .AddInMemoryExporter(exportedItems)); + + if (enableTracing) + { + using var act = new Activity("test").Start(); + act.ActivityTraceFlags = ActivityTraceFlags.Recorded; + counter.Add(18); + } + else + { + counter.Add(18); + } + + meterProvider.ForceFlush(); + + Assert.Single(exportedItems); + + var metricPoint = GetFirstMetricPoint(exportedItems); + + Assert.NotNull(metricPoint); + + var exemplars = GetExemplars(metricPoint.Value); + + if (enableTracing) + { + Assert.Single(exemplars); + } + else + { + Assert.Empty(exemplars); + } + } + [Fact] public void TestExemplarsFilterTags() { @@ -612,7 +659,7 @@ public void TestExemplarsFilterTags() using var container = this.BuildMeterProvider(out var meterProvider, builder => builder .AddMeter(meter.Name) - .SetExemplarFilter(new AlwaysOnExemplarFilter()) + .SetExemplarFilter(ExemplarFilterType.AlwaysOn) .AddView(histogram.Name, new MetricStreamConfiguration() { TagKeys = new string[] { "key1" } }) .AddInMemoryExporter(exportedItems, metricReaderOptions => { diff --git a/test/OpenTelemetry.Tests/Metrics/MetricTestsBase.cs b/test/OpenTelemetry.Tests/Metrics/MetricTestsBase.cs index 11c90c8d899..7d72b773ea6 100644 --- a/test/OpenTelemetry.Tests/Metrics/MetricTestsBase.cs +++ b/test/OpenTelemetry.Tests/Metrics/MetricTestsBase.cs @@ -237,7 +237,7 @@ internal static IReadOnlyList GetExemplars(MetricPoint mp) { if (mp.TryGetExemplars(out var exemplars)) { - return exemplars.Value.ToReadOnlyList(); + return exemplars.ToReadOnlyList(); } return Array.Empty();