[BUG] Transaction constructor always throws and handles System.InvalidOperationException when WebRequestTransactionCreator invokes StartTransactionAsync in Elastic.Apm.AspNetCore when the http request has distributedTracingData

[alex-tselikovsky]

APM Agent version

1.28.6

Environment

Linux&Windows

.Net Core 6

Describe the bug

We use AspNetCoreDiagnosticSubscriber.

Invoking WebRequestTransactionCreator.StartTransactionAsync creates new Transaction for incoming request. A new System.Diagnostics activity is created in a new transaction, and the parent of new Activity is set to "Microsoft.AspNetCore.Hosting.HttpRequestIn" which is created by Asp.NetCore.

The problem occurs when the request has the distributed tracing data. In this case the APM transaction constructor tries to set for a new activity another parent from the distributed tracing. In this moment InvalidOperationException occurs, because the parent activity already exists.

Elastic.Apm.Model.Transaction.cs source code

The exception is handled but it creates noise in the .Net counter metrics and can affect the performance.

To Reproduce

Steps to reproduce the behavior:

1. Use default config.
2. Use external service which transfers distributed tracing data to our service in http request
3. See error in net counters

Expected behavior

Therea aren't any System.InvalidOpertionExeptions in .Net counters

Actual behavior

Therea are a lot of System.InvalidOpertionExeptions in .Net counters