Skip to content

Commit

Permalink
docs: add trace doc (#3165)
Browse files Browse the repository at this point in the history 
Signed-off-by: Song Gao <[email protected]>
  • Loading branch information
@Yisaer
Yisaer committed Sep 9, 2024
1 parent 04889ce commit d752170
Show file tree
Hide file tree
Showing 6 changed files with 284 additions and 0 deletions.
111 changes: 111 additions & 0 deletions docs/en_US/api/restapi/trace.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Data tracing management

eKuiper supports viewing recent tracing data of rules through API.

## View the latest Trace ID based on the rule ID

```shell
GET http://localhost:9081/trace/rule/{ruleID}"
["747743cbf1fc6d10f732d17e5626021a"]
```
## View detailed tracing data based on Trace ID
```shell
GET http://localhost:9081/trace/{id}
{
"Name": "demo",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "f560f34e0d12a0aa",
"ParentSpanID": "0000000000000000",
"Attribute": null,
"Links": null,
"StartTime": "2024-08-28T10:01:38.362706+08:00",
"EndTime": "2024-08-28T10:01:38.362745751+08:00",
"ChildSpan": [
{
"Name": "2_decoder",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "fe1cde747e6cc4ba",
"ParentSpanID": "f560f34e0d12a0aa",
"Attribute": {
"data": "{\"a\":1}"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.362842+08:00",
"EndTime": "2024-08-28T10:01:38.362865821+08:00",
"ChildSpan": [
{
"Name": "3_project",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "377ee05e98e7f00b",
"ParentSpanID": "fe1cde747e6cc4ba",
"Attribute": {
"data": "{\"a\":1,\"meta\":\"747743cbf1fc6d10f732d17e5626021a\"}"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.362926+08:00",
"EndTime": "2024-08-28T10:01:38.362977943+08:00",
"ChildSpan": [
{
"Name": "transform_op",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "7816e87f397b8ecc",
"ParentSpanID": "377ee05e98e7f00b",
"Attribute": {
"data": "{\"a\":1,\"meta\":\"747743cbf1fc6d10f732d17e5626021a\"}"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.363005+08:00",
"EndTime": "2024-08-28T10:01:38.363016309+08:00",
"ChildSpan": [
{
"Name": "transform_op_split",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "a0a6786f7905f9ba",
"ParentSpanID": "7816e87f397b8ecc",
"Attribute": null,
"Links": null,
"StartTime": "2024-08-28T10:01:38.363021+08:00",
"EndTime": "2024-08-28T10:01:38.363023415+08:00",
"ChildSpan": [
{
"Name": "log_0_1_encode",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "fecc8a2b92b72560",
"ParentSpanID": "a0a6786f7905f9ba",
"Attribute": {
"data": "[{\"a\":1,\"meta\":\"747743cbf1fc6d10f732d17e5626021a\"}]"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.363053+08:00",
"EndTime": "2024-08-28T10:01:38.363063262+08:00",
"ChildSpan": [
{
"Name": "log_0",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "c544ab89716781f6",
"ParentSpanID": "fecc8a2b92b72560",
"Attribute": {
"data": "[{\"a\":1,\"meta\":\"747743cbf1fc6d10f732d17e5626021a\"}]"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.363082+08:00",
"EndTime": "2024-08-28T10:01:38.363083833+08:00",
"ChildSpan": []
}
]
}
]
}
]
}
]
}
]
}
]
}
```
1 change: 1 addition & 0 deletions docs/en_US/guide/rules/overview.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ See the table below for a detailed explanation of each rule behavior:
| cron | string: "" | Specify the periodic trigger strategy of the rule, which is described by [cron expression](https://en.wikipedia.org/wiki/Cron) |
| duration | string: "" | Specifies the running duration of the rule, only valid when cron is specified. The duration should not exceed the time interval between two cron cycles, otherwise it will cause unexpected behavior. |
| cronDatetimeRange | lists of struct | Specify the effective time period of the Scheduled Rule, which is only valid when `cron` is specified. When this `cronDatetimeRange` is specified, the Scheduled Rule will only take effect within the time range specified. Please see [Scheduled Rule](#Scheduled Rule) for detailed configuration items |
| enableRuleTracer | bool: false | Specify whether the rule enables rule-level data tracing |

For detail about `qos` and `checkpointInterval`, please check [state and fault tolerance](./state_and_fault_tolerance.md).

Expand Down
30 changes: 30 additions & 0 deletions docs/en_US/operation/usage/trace_data.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
## Use Open Telemetry Tracing to track data

eKuiper's rule is a continuously running streaming computing task. Rules are used to process unbounded data flows. Under normal circumstances, rules will continue to run after they are started and continuously generate running status data. We can use open telemetry tracing to track the data changes of each piece of data in each operator.

## Open Telemetry Tracing configuration

You can expose Open Telemetry data to the remote Open Telemetry Collector through configuration. At the same time, eKuiper also supports built-in collection of Open Telemetry Trcing data. You can limit the maximum amount of saved data through the following configuration.

```yaml
openTelemetry:
enableRemoteCollector: false
remoteEndpoint: localhost:4318
localTraceCapacity: 2048
```
## Enable rule-level tracing
You can turn on data link tracing for the corresponding rule by setting `enableRuleTracer` in the rule `options` to true. For specific settings, please see [Rules](../../guide/rules/overview.md#rules)

## Get the Trace ID of each piece of data

You can get the latest Trace ID corresponding to the rule through the Rest API.

[View the most recent Trace ID based on rule ID](../../api/restapi/trace.md#view-the-latest-trace-id-based-on-the-rule-id)

## View traced data flow changes based on Trace ID

If you configure an Open Telemetry Tracing collector, you can query the storage and query platform behind the Open Telemetry collector through the Trace ID. At the same time, you can also obtain detailed tracing data by accessing the local Rest API.

[View detailed tracing data based on Trace ID](../../api/restapi/trace.md#view-detailed-tracing-data-based-on-trace-id)
111 changes: 111 additions & 0 deletions docs/zh_CN/api/restapi/trace.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# 数据追踪管理

eKuiper 支持通过 API 查看规则最近的追踪数据。

## 根据规则 ID 查看最近的 Trace ID

```shell
GET http://localhost:9081/trace/rule/{ruleID}"
["747743cbf1fc6d10f732d17e5626021a"]
```
## 根据 Trace ID 查看详细追踪数据
```shell
GET http://localhost:9081/trace/{id}
{
"Name": "demo",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "f560f34e0d12a0aa",
"ParentSpanID": "0000000000000000",
"Attribute": null,
"Links": null,
"StartTime": "2024-08-28T10:01:38.362706+08:00",
"EndTime": "2024-08-28T10:01:38.362745751+08:00",
"ChildSpan": [
{
"Name": "2_decoder",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "fe1cde747e6cc4ba",
"ParentSpanID": "f560f34e0d12a0aa",
"Attribute": {
"data": "{\"a\":1}"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.362842+08:00",
"EndTime": "2024-08-28T10:01:38.362865821+08:00",
"ChildSpan": [
{
"Name": "3_project",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "377ee05e98e7f00b",
"ParentSpanID": "fe1cde747e6cc4ba",
"Attribute": {
"data": "{\"a\":1,\"meta\":\"747743cbf1fc6d10f732d17e5626021a\"}"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.362926+08:00",
"EndTime": "2024-08-28T10:01:38.362977943+08:00",
"ChildSpan": [
{
"Name": "transform_op",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "7816e87f397b8ecc",
"ParentSpanID": "377ee05e98e7f00b",
"Attribute": {
"data": "{\"a\":1,\"meta\":\"747743cbf1fc6d10f732d17e5626021a\"}"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.363005+08:00",
"EndTime": "2024-08-28T10:01:38.363016309+08:00",
"ChildSpan": [
{
"Name": "transform_op_split",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "a0a6786f7905f9ba",
"ParentSpanID": "7816e87f397b8ecc",
"Attribute": null,
"Links": null,
"StartTime": "2024-08-28T10:01:38.363021+08:00",
"EndTime": "2024-08-28T10:01:38.363023415+08:00",
"ChildSpan": [
{
"Name": "log_0_1_encode",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "fecc8a2b92b72560",
"ParentSpanID": "a0a6786f7905f9ba",
"Attribute": {
"data": "[{\"a\":1,\"meta\":\"747743cbf1fc6d10f732d17e5626021a\"}]"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.363053+08:00",
"EndTime": "2024-08-28T10:01:38.363063262+08:00",
"ChildSpan": [
{
"Name": "log_0",
"TraceID": "747743cbf1fc6d10f732d17e5626021a",
"SpanID": "c544ab89716781f6",
"ParentSpanID": "fecc8a2b92b72560",
"Attribute": {
"data": "[{\"a\":1,\"meta\":\"747743cbf1fc6d10f732d17e5626021a\"}]"
},
"Links": null,
"StartTime": "2024-08-28T10:01:38.363082+08:00",
"EndTime": "2024-08-28T10:01:38.363083833+08:00",
"ChildSpan": []
}
]
}
]
}
]
}
]
}
]
}
]
}
```
1 change: 1 addition & 0 deletions docs/zh_CN/guide/rules/overview.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@ eKuiper 已经内置了丰富的 sink connector 类型，如 mqtt、rest 和 fil
| cron | string: "" | 指定规则的周期性触发策略，该周期通过 [cron 表达式](https://zh.wikipedia.org/wiki/Cron) 进行描述。 |
| duration | string: "" | 指定规则的运行持续时间，只有当指定了 cron 后才有效。duration 不应该超过两次 cron 周期之间的时间间隔，否则会引起非预期的行为。 |
| cronDatetimeRange | 结构体数组 | 指定周期性规则的生效时间段。当指定了该参数后，周期性规则只有在这个参数所制定的时间范围内才生效。请查看 [周期性规则](#周期性规则) 了解详细的配置项目 |
| enableRuleTracer | bool: false | 指定规则是否开启规则级别的数据追踪 |

有关 `qos` 和 `checkpointInterval` 的详细信息，请查看[状态和容错](./state_and_fault_tolerance.md)。

Expand Down
30 changes: 30 additions & 0 deletions docs/zh_CN/operation/usage/trace_data.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
## 使用 Open Telemetry Tracing 来追踪数据

eKuiper 的规则是一个持续运行的流式计算任务。规则用于处理无界的数据流，正常情况下，规则启动后会一直运行，不断产生运行状态数据。我们可以使用 open telemetry tracing 来追踪每一条数据在各个算子中的数据变化。

## Open Telemetry Tracing 配置

你可以通过配置将 Open Telemetry 的数据暴露给远端的 Open Telemetry Collector, 同时 eKuiper 也支持内置收集 Open Telemtry Trcing 的数据，你可以通过以下配置限制最大保存数据的数量。

```yaml
openTelemetry:
enableRemoteCollector: false
remoteEndpoint: localhost:4318
localTraceCapacity: 2048
```
## 开启规则级别的追踪
你可以通过设置规则 `options` 中的 `enableRuleTracer` 为 true，为对应规则打开数据链路追踪。具体设置请查看 [规则](../../guide/rules/overview.md#选项)

## 获取每条数据的 Trace ID

你可以通过 Rest API 获取规则对应的最近 Trace ID。

[根据规则 ID 查看最近的 Trace ID](../../api/restapi/trace.md#根据规则-id-查看最近的-trace-id)

## 根据 Trace ID 查看追踪的数据流变化

如果你配置了 Open Telemetry Tracing 收集器，你可以通过 Trace ID 向 Open Telemetry 收集器背后的存储与查询平台进行查询。 同时，你也可以通过访问本地 Rest API 的方式获取详细的追踪数据。

[根据 Trace ID 查看详细追踪数据](../../api/restapi/trace.md#根据-trace-id-查看详细追踪数据)

0 comments on commit d752170

Please sign in to comment.