OpenTelemetry: attributes don't show up for the span created by autoinstrumentation

[grouzen]

The bug was reported in the Discord channel:
https://discord.com/channels/629491597070827530/639825316021272602/1116089112529748000

I am using auto-instrumentation for my grpc service that also is a Kafka consumer. When I receieve a request or Kafka message I want to manually set custom attributes. I either do this by calling a java library to set custom attributes:
ZIO.serviceWithZIO[Tracing](_.scopedEffect(library.setAttributes(...))
Or I use the zio library to set custom attributes:
ZIO.serviceWithZIO[Tracing](_.setAttribute("foo", "bar")
I am seeing the autoinstrumented traces for both grpc and Kafka messages but I am not seeing any custom attributes for the traces when I look in jaeger.

Here is my Tracing layer:

    (ZLayer.fromZIO(
      ZIO.attempt(GlobalOpenTelemetry.getTracer("io.opentelemetry.api"))) ++
      ContextStorage.openTelemetryContext) >>> OtelTracing.live
  }

And I am using v3.0.0-RC14. Previosuly when I was on v2.0.3 the custom attribiutes worked for grpc but not for Kafka.
Any reason these attributes arent showing up when I view them in jaeger?

[grouzen]

@grogdotcom
I've tried to set attributes for a span created by auto-instrumentation in this PR: https://github.com/zio/zio-telemetry/pull/715/files.
It works well. Of course, in the example, the span is provided by Netty instrumentation.
Could you please elaborate more on which libraries you use for Kafka and grpc?

[Gregory-Berkman-Imprivata]

@grouzen I am using zio-grpc v0.6.0-rc5 and zio-kafka v2.3.1. Auto insrumentation is working fine for both. I am not seeing any custom attributes attached to the autoinstrumented spans when I receive the request on the server/consumer side. I am using opentelemetry version 1.18.0.
I just tried upgrading my opentelemetry version to 1.26.0 but am seeing the same behavior
The telemetry version in the jaeger trace is telemetry.sdk.version: 1.19.0

[grouzen]

The telemetry version in the jaeger trace is telemetry.sdk.version: 1.19.0

That looks suspicious to me. It should be 1.26.0 for auto-instrumented spans if you use this version in -javaagent option.

[grouzen]

@Gregory-Berkman-Imprivata Hey! Have you managed to solve it?

[Gregory-Berkman-Imprivata]

@grouzen Unfortunately no, I decided to just manually instrument these traces

[grouzen]

Okay! I hope we will solve it somehow in the future, so I'll leave the issue open for now.

[omidb]

@Gregory-Berkman-Imprivata can you elaborate a bit on this? I have the same issue: scalapb/zio-grpc#544

[omidb]

A working example here scalapb/zio-grpc#544 and @grouzen we can close it. maybe even add an example in the future.

[grouzen]

Thank you! Great stuff!
Here is the follow-up issue to add gRPC examples: #751