OpenTelemetry通过OTLP协议直接传输Log原理和具体过程

[laziobird]

1、Log 传输原理图

https://opentelemetry.io/docs/reference/specification/logs/#direct-to-collector
通过API、SDK 封装提供各个语言通用的Appender 来实现

To facilitate both approaches described above OpenTelemetry provides an [API](https://opentelemetry.io/docs/reference/specification/logs/api/) and [SDK](https://opentelemetry.io/docs/reference/specification/logs/sdk/), 

which can be used together with existing logging libraries to automatically inject the request context in the emitted logs, 
and provide an easy way to send the logs via OTLP. Instead of modifying each logging statement, 

[Appenders](https://opentelemetry.io/docs/reference/specification/logs/api/#how-to-create-log4j-style-appender)
use the API to bridge logs from existing logging libraries to the OpenTelemetry data model, where the SDK controls how the logs are processed and exported.

2、Log4j & Logback 风格的 OpenTelemetry API、SDK 具体Appender 实现

How to Create Log4J Style Appender

2.1 SDK logRecord 实现原理

https://opentelemetry.io/docs/reference/specification/logs/sdk/#logrecordprocessor

  +-----+------------------------+   +------------------------------+   +-------------------------+
  |     |                        |   |                              |   |                         |
  |     |                        |   | Batching LogRecordProcessor  |   |    LogRecordExporter    |
  |     |                        +---> Simple LogRecordProcessor    +--->     (OtlpExporter)      |
  |     |                        |   |                              |   |                         |
  | SDK | Logger.emit(LogRecord) |   +------------------------------+   +-------------------------+
  |     |                        |
  |     |                        |
  |     |                        |
  |     |                        |
  |     |                        |
  +-----+------------------------+

2.2 SDK logrecordexporter 介绍

https://opentelemetry.io/docs/reference/specification/logs/sdk/#logrecordexporter

logrecordexporter 源代码参考

https://github.com/open-telemetry/opentelemetry-java/blob/main/exporters/otlp/logs/src/main/java/io/opentelemetry/exporter/otlp/http/logs/OtlpHttpLogRecordExporterBuilder.java

public final class OtlpHttpLogRecordExporterBuilder {
  private static final String DEFAULT_ENDPOINT = "http://localhost:4318/v1/logs";
}

https://github.com/open-telemetry/opentelemetry-java/blob/main/exporters/otlp/logs/src/main/java/io/opentelemetry/exporter/otlp/http/logs/OtlpHttpLogRecordExporter.java
https://github.com/open-telemetry/opentelemetry-java/blob/main/exporters/otlp/logs/src/main/java/io/opentelemetry/exporter/otlp/http/logs/OtlpHttpLogRecordExporterBuilder.java

3、OpenTelemetry Collector 如何Receive Telemetry Log

https://github.com/open-telemetry/opentelemetry-collector/blob/main/receiver/otlpreceiver/README.md

3.1 Writing with HTTP/JSON

The OTLP receiver can receive trace export calls via HTTP/JSON in addition to
gRPC. The HTTP/JSON address is the same as gRPC as the protocol is recognized
and processed accordingly. Note the serialization format needs to be protobuf JSON.

To write traces with HTTP/JSON, POST to [address]/v1/traces for traces,
to [address]/v1/metrics for metrics, to [address]/v1/logs for logs. The default
port is 4318.

receivers:
  otlp:
    protocols:
      http:
        endpoint: "localhost:4318"
        cors:
          allowed_origins:
            - http://test.com
            # Origins can have wildcards with *, use * by itself to match any origin.
            - https://*.example.com
          allowed_headers:
            - Example-Header
          max_age: 7200