DSET-4002: Extract serverHost from the logs and traces

[martin-majlis-s1]

Description: Introduce server_host configuration to allow specifying server host

There is new configuration option server_host that allows specifying how to fill the field ServerHost of the Event structure.

New configuration looks like this:

    server_host:
      # If these attributes are not specified or empty,
      # use the value from the env variable SERVER_HOST
      use_host: ${env:SERVER_HOST}
      # If it's not set, use the hostname value
      use_host_name: true

This configuration is applied for logs and traces.

Since it's touching the same code, I have already merged changes from this PR - open-telemetry#23881 - into this branch, hoping that it will be merged soon. It's still not merged, so it's blocked by this.

Link to tracking Issue: open-telemetry#24252 / DSET-4002

Testing:

I have added several unit tests to make sure that it works as expected.

Documentation: There is new section in the README describing the newly added configuration options.

[tomaz-s1]

@martin-majlis-s1 An alternative to that attributes configuration option would be telling the user to configure attribute processor to do that.

So instead telling the user to configure our exporter in which attributes it should search for LogRecord server host, we should tell user that serverHost field on the OTel LogRecord should reflect server host where the event is coming from (and in case it's not provided we will default to hostname where otel collector is running).

This is similar to recommended logstash approach and it gives user the most flexibility - they can populate that attribute with OTel processor / operator / similar as they fit and they can also combine or use multiple processors / operators / etc.

In fact, that's also what we do internally with our logstash deployment (we have multiple logstash filters which set logstash event serverHost differently depending on the event and other context).

And if we go with that approach, we should include some example operator configuration on how to do that.

Also, per discussion in the past and separate ticket, we should also just try to retrieve serverHost from related LogRecord Resource - I think this should work correctly out of the box and cover most common scenarios.

WDYT?

[martin-majlis-s1]

Both approaches have some advantages and disadvantages:

• this approach - you do not have to study anything else, you do not have to modify your data
• "it has to be serverHost" approach - if you use this field for something else, then you have to move this data somewhere else and then replace it with the serverHost value

But now when I think about it, I would bet that otel maintainers would also prefer the second version => so I will change it.

[tomaz-s1]

I think it would be a good idea to document each specific scenario with config option and example event payload - just to make it more clear to the end user how the DataSet event will look like.

[tomaz-s1]

Just to confirm / clarify, extracting serverHost from the LogRecord Resource will be done in a separate PR, right?

I think that's fine, but I think it's better to combine those two PRs together into a single one when opening PR against upstream repo - this should be less work for the open telemetry maintainers since those two changes are related.

[martin-majlis-s1]

Just to confirm / clarify, extracting serverHost from the LogRecord Resource will be done in a separate PR, right?

I think that's fine, but I think it's better to combine those two PRs together into a single one when opening PR against upstream repo - this should be less work for the open telemetry maintainers since those two changes are related.

Yes. It will be in the next PR

[tomaz-s1]

@martin-majlis-s1 Thanks.

To be able to confirm that serverHost extraction from Resource works correctly, I think we should also test it with otel demo.

[martin-majlis-s1]

@martin-majlis-s1 Thanks.

To be able to confirm that serverHost extraction from Resource works correctly, I think we should also test it with otel demo.

I have added unit test to verify, that it works as expected. I have also used telemetrygen - and it works as expected:

[tomaz-s1]

@martin-majlis-s1 I think we are not exactly on the same page here.

I meant inferring serverHost from LogRecord Resource object when that value is not stored directly as part of the resource.attributes.serverHost attribute, but when it's stored as part on some other Resource attribute as part of otel conventions (likely host.name, but it needs more digging in and research - https://opentelemetry.io/docs/specs/otel/resource/semantic_conventions/host/, https://github.com/open-telemetry/semantic-conventions/blob/main/model/resource/host.yaml#L15).

Similar to what do we do with inferring service name for tracing part of the plugin.

That's why I said it's important to test it end with the actual otel demo setup. This way we known it works with real work setup and not just manual and specific code path you are exercising.

[martin-majlis-s1]

Just to confirm / clarify, extracting serverHost from the LogRecord Resource will be done in a separate PR, right?

I think that's fine, but I think it's better to combine those two PRs together into a single one when opening PR against upstream repo - this should be less work for the open telemetry maintainers since those two changes are related.

Now the serverHost is also extracted from the Resource (as well as host.name).

[martin-majlis-s1]

When I run the demo, I can see, that the host.name from the resource is correctly extracted. However it's used just for single service.

[tomaz-s1]

@martin-majlis-s1 Can you please share DataSet link for the search query above so I can quickly have a look? Thanks.

[martin-majlis-s1]

@martin-majlis-s1 Can you please share DataSet link for the search query above so I can quickly have a look? Thanks.

The thing is, that only two services are populating logs:

• https://app-qatesting.scalyr.com/y/SXRDxLUjA6D
• https://app-qatesting.scalyr.com/y/j4xov1vVCDj

[tomaz-s1]

@martin-majlis-s1 Thanks. I had a quick look and it looks good to me :)

P.S. I assume "martin" is hostname which is specified in the config which is used as a default when Resource doesn't contain host.name, right?

Did we also test a scenario where it uses collector hostname when that "martin" value is not specified in the config? Thanks.

[martin-majlis-s1]

I can see, that:

• only adservice (Java) and recommendation service (python) are producing logs
• Language of that service is from https://opentelemetry.io/docs/demo/
• Python has experimental support for logs
• Php is alpha, C++ is experimental
  • but for C++ - https://opentelemetry.io/docs/instrumentation/cpp/manual/#logs - it's not there
  • but for PHP - https://opentelemetry.io/docs/instrumentation/php/manual/#logs - it should be easy to add

[martin-majlis-s1]

@martin-majlis-s1 Thanks. I had a quick look and it looks good to me :)

P.S. I assume "martin" is hostname which is specified in the config which is used as a default when Resource doesn't contain host.name, right?

Did we also test a scenario where it uses collector hostname when that "martin" value is not specified in the config? Thanks.

Yes. We have unit tests for all scenarios and I have tested it by changing the configuration as well.

[tomaz-s1]

I didn't get a chance to look into the latest code changes in more detail, but from the end user / outcome perspective, LGTM.