You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[v2][query] Create archive reader/writer using regular factory methods (jaegertracing#6519)
## Which problem is this PR solving?
- Towards jaegertracing#6065
## Description of the changes
- This PR changes the jaegerquery extension to remove usages of
`CreateArchiveSpanReader` and `CreateArchiveSpanWriter` and replace them
with their primary storage counterparts from the v2 storage API
`CreateTraceReader` and `CreateTraceWriter`.
- 🛑 **This PR contains a breaking changes for users of jaeger-v2 that
have an ElasticSearch storage configured for `traces_archive`** 🛑
- The distinction between primary and archive storage was removed which
causes some breaking changes that can be remediated. Refer to Step 9 of
the test report below for remediation steps.
## Test Report
### 1. Establish Ground Truth on `main`
To begin, configure the storage and query settings in the
`all-in-one.yaml` file as follows:
```yaml
jaeger_storage:
backends:
some_storage:
elasticsearch:
indices:
index_prefix: "jaeger-main"
spans:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
services:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
dependencies:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
sampling:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
another_storage:
elasticsearch:
indices:
index_prefix: "jaeger-archive"
```
### 2. Spin Up Elasticsearch Using Docker
To bring up Elasticsearch, run the following commands:
```zsh
jaeger % cd docker-compose/elasticsearch/v8
v8 % docker compose up
```
### 3. Start Jaeger
Start the Jaeger service:
```zsh
jaeger % go run ./cmd/jaeger
```
### 4. Archive a Trace
From the Jaeger UI, select a trace and archive it.
**traceID**: `0dc3e460bd9b8e0dddfa29a2f751cfb9`

### 5. Update `index_prefix`
Stop Jaeger and modify the `index_prefix` in the primary configuration
to `jaeger-main-1`. This ensures the query for the same trace is no
longer found in the primary storage but will be found in the archive
storage.
### 6. Query for the Same Trace
```zsh
curl -s http://localhost:16686/api/traces/0dc3e460bd9b8e0dddfa29a2f751cfb9 | jq '.data[].spans[] | {traceID, operationName}'
{
"traceID": "0dc3e460bd9b8e0dddfa29a2f751cfb9",
"operationName": "/api/services"
}
{
"traceID": "0dc3e460bd9b8e0dddfa29a2f751cfb9",
"operationName": "GetService"
}
```
```zsh
curl -s http://localhost:16686/api/v3/traces/0dc3e460bd9b8e0dddfa29a2f751cfb9 | jq '.result.resourceSpans[].scopeSpans[].spans[] | {traceId, name}'
{
"traceId": "0dc3e460bd9b8e0dddfa29a2f751cfb9",
"name": "GetService"
}
{
"traceId": "0dc3e460bd9b8e0dddfa29a2f751cfb9",
"name": "/api/services"
}
```
### 7. Test Changes from This PR
Stop Jaeger, checkout this PR, and restart Jaeger:
```zsh
gh pr checkout 6519
go run ./cmd/jaeger
```
### 8. Query for the Same Trace After PR Changes
```zsh
curl -s http://localhost:16686/api/traces/0dc3e460bd9b8e0dddfa29a2f751cfb9 | jq .
{
"data": null,
"total": 0,
"limit": 0,
"offset": 0,
"errors": [
{
"code": 404,
"msg": "trace not found"
}
]
}
```
```zsh
curl -s http://localhost:16686/api/v3/traces/0dc3e460bd9b8e0dddfa29a2f751cfb9 | jq .
{
"error": {
"httpCode": 404,
"message": "No traces found"
}
}
```
🛑 **This is where the breaking change occurs** 🛑
### 9. Mitigation for Users
#### Set `use_aliases` for Archive Storage
To mitigate the issue, set the `use_aliases` configuration for your
archive storage to `true`. Update the configuration as follows:
```yaml
another_storage:
elasticsearch:
indices:
index_prefix: "jaeger-archive"
use_aliases: true
```
#### Add Alias from Old Index to New Index
To ensure backwards compatibility, add an alias from the old index to
the new index. You can query the current set of aliases in Elasticsearch
with:
```bash
curl -X GET "http://localhost:9200/_aliases?pretty"
```
```json
{
"jaeger-main-1-jaeger-span-2025-01-16": { "aliases": {} },
"jaeger-main-2-jaeger-span-2025-01-16": { "aliases": {} },
"jaeger-archive-jaeger-span-archive": { "aliases": {} },
"jaeger-main-1-jaeger-service-2025-01-16": { "aliases": {} }
}
```
To link the new index (`jaeger-archive-jaeger-span-read`) with the old
index (`jaeger-archive-jaeger-span-archive`), run the following:
```bash
curl -X POST "http://localhost:9200/_aliases" -H 'Content-Type: application/json' -d'
{
"actions": [
{
"add": {
"index": "jaeger-archive-jaeger-span-archive",
"alias": "jaeger-archive-jaeger-span-read"
}
}
]
}'
```
Confirm that the alias has been added:
```bash
curl -X GET "http://localhost:9200/_aliases?pretty"
```
```json
{
"jaeger-main-1-jaeger-span-2025-01-16": { "aliases": {} },
"jaeger-archive-jaeger-span-archive": { "aliases": { "jaeger-archive-jaeger-span-read": {} } },
"jaeger-main-2-jaeger-span-2025-01-16": { "aliases": {} },
"jaeger-main-2-jaeger-service-2025-01-16": { "aliases": {} },
"jaeger-main-1-jaeger-service-2025-01-16": { "aliases": {} }
}
```
Note that if you already had `use_aliases` set to true for your archive
storage before the upgrade, then Jaeger would've been using an index
name with -read suffix: `jaeger-archive-jaeger-span-archive-read`. Then
for the example above, you would create an alias
`jaeger-archive-jaeger-span-read` pointing to
`jaeger-archive-jaeger-span-archive-read`.
### 10. Restart Jaeger and Retry the Query
Finally, restart Jaeger and run the same trace queries again:
```zsh
curl -s http://localhost:16686/api/traces/0dc3e460bd9b8e0dddfa29a2f751cfb9 | jq '.data[].spans[] | {traceID, operationName}'
{
"traceID": "0dc3e460bd9b8e0dddfa29a2f751cfb9",
"operationName": "/api/services"
}
{
"traceID": "0dc3e460bd9b8e0dddfa29a2f751cfb9",
"operationName": "GetService"
}
```
```zsh
curl -s http://localhost:16686/api/v3/traces/0dc3e460bd9b8e0dddfa29a2f751cfb9 | jq '.result.resourceSpans[].scopeSpans[].spans[] | {traceId, name}'
{
"traceId": "0dc3e460bd9b8e0dddfa29a2f751cfb9",
"name": "/api/services"
}
{
"traceId": "0dc3e460bd9b8e0dddfa29a2f751cfb9",
"name": "GetService"
}
```
## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
- for `jaeger`: `make lint test`
- for `jaeger-ui`: `npm run lint` and `npm run test`
---------
Signed-off-by: Mahad Zaryab <[email protected]>
Co-authored-by: Yuri Shkuro <[email protected]>
0 commit comments