Skip to content

Commit

Permalink
dm: optimize query status doc (#15662) (#15715)
Browse files Browse the repository at this point in the history
  • Loading branch information
@ti-chi-bot
ti-chi-bot authored Dec 14, 2023
1 parent b9ce348 commit 3419db0
Showing 1 changed file with 103 additions and 68 deletions.
171 changes: 103 additions & 68 deletions dm/dm-query-status.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,26 @@ This document introduces how to use the `query-status` command to query the task

## Query result

{{< copyable "" >}}
It is recommended that you use `query-status` by the following steps:

1. Use `query-status` to check whether each on-going task is in the normal state.
2. If any error occurs in a task, use the `query-status <taskName>` command to see detailed error information. `<taskName>` in this command indicates the name of the task that encounters the error.

A successful query result is as follows:

```bash
» query-status
```

```
```json
{
"result": true, # Whether the query is successful.
"msg": "", # Describes the reason for the unsuccessful query.
"tasks": [ # Migration task list.
"result": true,
"msg": "",
"tasks": [
{
"taskName": "test", # The task name.
"taskStatus": "Running", # The status of the task.
"sources": [ # The upstream MySQL list.
"taskName": "test",
"taskStatus": "Running",
"sources": [
"mysql-replica-01",
"mysql-replica-02"
]
Expand All @@ -40,12 +45,14 @@ This document introduces how to use the `query-status` command to query the task
}
```

For detailed descriptions of `taskStatus` under the `tasks` section, refer to [Task status](#task-status).

It is recommended that you use `query-status` by the following steps:
Some fields in the query result are described as follows:

1. Use `query-status` to check whether each on-going task is in the normal state.
2. If any error occurs in a task, use the `query-status <taskName>` command to see detailed error information. `<taskName>` in this command indicates the name of the task that encounters the error.
- `result`: Whether the query is successful.
- `msg`: The error message returned when the query fails.
- `tasks`: The list of migration tasks. Each task contains the following fields:
- `taskName`: The name of the task.
- `taskStatus`: The status of the task. For detailed descriptions of `taskStatus`, refer to [Task status](#task-status).
- `sources`: The list of upstream MySQL databases.

## Task status

Expand All @@ -63,70 +70,59 @@ The status of a DM migration task depends on the status of each subtask assigned

## Detailed query result

{{< copyable "" >}}

```bash
» query-status test
```

```
» query-status
```json
{
"result": true, # Whether the query is successful.
"msg": "", # Describes the cause for the unsuccessful query.
"sources": [ # The upstream MySQL list.
"result": true,
"msg": "",
"sources": [
{
"result": true,
"msg": "",
"sourceStatus": { # The information of the upstream MySQL databases.
"sourceStatus": {
"source": "mysql-replica-01",
"worker": "worker1",
"result": null,
"relayStatus": null
},
"subTaskStatus": [ # The information of all subtasks of upstream MySQL databases.
"subTaskStatus": [
{
"name": "test", # The name of the subtask.
"stage": "Running", # The running status of the subtask, including "New", "Running", "Paused", "Stopped", and "Finished".
"unit": "Sync", # The processing unit of DM, including "Check", "Dump", "Load", and "Sync".
"result": null, # Displays the error information if a subtask fails.
"unresolvedDDLLockID": "test-`test`.`t_target`", # The sharding DDL lock ID, used for manually handling the sharding DDL
# lock in the abnormal condition.
"sync": { # The replication information of the `Sync` processing unit. This information is about the
# same component with the current processing unit.
"masterBinlog": "(bin.000001, 3234)", # The binlog position in the upstream database.
"masterBinlogGtid": "c0149e17-dff1-11e8-b6a8-0242ac110004:1-14", # The GTID information in the upstream database.
"syncerBinlog": "(bin.000001, 2525)", # The position of the binlog that has been replicated
# in the `Sync` processing unit.
"syncerBinlogGtid": "", # The binlog position replicated using GTID.
"blockingDDLs": [ # The DDL list that is blocked currently. It is not empty only when all the upstream tables of this
# DM-worker are in the "synced" status. In this case, it indicates the sharding DDL statements to be executed or that are skipped.
"name": "test",
"stage": "Running",
"unit": "Sync",
"result": null,
"unresolvedDDLLockID": "test-`test`.`t_target`",
"sync": {
"masterBinlog": "(bin.000001, 3234)",
"masterBinlogGtid": "c0149e17-dff1-11e8-b6a8-0242ac110004:1-14",
"syncerBinlog": "(bin.000001, 2525)",
"syncerBinlogGtid": "",
"blockingDDLs": [
"USE `test`; ALTER TABLE `test`.`t_target` DROP COLUMN `age`;"
],
"unresolvedGroups": [ # The sharding group that is not resolved.
"unresolvedGroups": [
{
"target": "`test`.`t_target`", # The downstream database table to be replicated.
"target": "`test`.`t_target`",
"DDLs": [
"USE `test`; ALTER TABLE `test`.`t_target` DROP COLUMN `age`;"
],
"firstPos": "(bin|000001.000001, 3130)", # The starting position of the sharding DDL statement.
"synced": [ # The upstream sharded table whose executed sharding DDL statement has been read by the `Sync` unit.
"firstPos": "(bin|000001.000001, 3130)",
"synced": [
"`test`.`t2`"
"`test`.`t3`"
"`test`.`t1`"
],
"unsynced": [ # The upstream table that has not executed this sharding DDL
# statement. If any upstream tables have not finished replication,
# `blockingDDLs` is empty.
"unsynced": [
]
}
],
"synced": false # Whether the incremental replication catches up with the upstream and has the same binlog position as that in the
# upstream. The save point is not refreshed in real time in the `Sync` background, so `false` of `synced`
# does not always mean a replication delay exits.
"totalRows": "12", # The total number of rows that are replicated in this subtask.
"totalRps": "1", # The number of rows that are replicated in this subtask per second.
"recentRps": "1" # The number of rows that are replicated in this subtask in the last second.
"synced": false,
"totalRows": "12",
"totalRps": "1",
"recentRps": "1"
}
}
]
Expand All @@ -147,11 +143,11 @@ The status of a DM migration task depends on the status of each subtask assigned
"unit": "Load",
"result": null,
"unresolvedDDLLockID": "",
"load": { # The replication information of the `Load` processing unit.
"finishedBytes": "115", # The number of bytes that have been loaded.
"totalBytes": "452", # The total number of bytes that need to be loaded.
"progress": "25.44 %", # The progress of the loading process.
"bps": "2734" # The speed of the full loading.
"load": {
"finishedBytes": "115",
"totalBytes": "452",
"progress": "25.44 %",
"bps": "2734"
}
}
]
Expand All @@ -169,7 +165,7 @@ The status of a DM migration task depends on the status of each subtask assigned
"name": "test",
"stage": "Paused",
"unit": "Load",
"result": { # The error example.
"result": {
"isCanceled": false,
"errors": [
{
Expand Down Expand Up @@ -205,26 +201,65 @@ The status of a DM migration task depends on the status of each subtask assigned
"unit": "Dump",
"result": null,
"unresolvedDDLLockID": "",
"dump": { # The replication information of the `Dump` processing unit.
"totalTables": "10", # The number of tables to be dumped.
"completedTables": "3", # The number of tables that have been dumped.
"finishedBytes": "2542", # The number of bytes that have been dumped.
"finishedRows": "32", # The number of rows that have been dumped.
"estimateTotalRows": "563", # The estimated number of rows to be dumped.
"progress": "30.52 %", # The progress of the dumping process.
"bps": "445" # The dumping speed.
"dump": {
"totalTables": "10",
"completedTables": "3",
"finishedBytes": "2542",
"finishedRows": "32",
"estimateTotalRows": "563",
"progress": "30.52 %",
"bps": "445"
}
}
]
},
]
}
```

For the status description and status switch relationship of "stage" of "subTaskStatus" of "sources", see the [subtask status](#subtask-status).

For operation details of "unresolvedDDLLockID" of "subTaskStatus" of "sources", see [Handle Sharding DDL Locks Manually](/dm/manually-handling-sharding-ddl-locks.md).
Some fields in the returned result are described as follows:

- `result`: Whether the query is successful.
- `msg`: The error message returned when the query fails.
- `sources`: The list of upstream MySQL instances. Each source contains the following fields:
- `result`
- `msg`
- `sourceStatus`: The information of the upstream MySQL databases.
- `subTaskStatus`: The information of all subtasks of upstream MySQL databases. Each subtask might contain the following fields:
- `name`: The name of the subtask.
- `stage`: The status of the subtask. For the status description and status switch relationship of "stage" of "subTaskStatus" of "sources", see the [subtask status](#subtask-status).
- `unit`: The processing unit of DM, including "Check", "Dump", "Load", and "Sync".
- `result`: Displays the error information if a subtask fails.
- `unresolvedDDLLockID`: The sharding DDL lock ID, used for manually handling the sharding DDL lock in the abnormal condition. For operation details of "unresolvedDDLLockID" of "subTaskStatus" of "sources", see [Handle Sharding DDL Locks Manually](/dm/manually-handling-sharding-ddl-locks.md).
- `sync`: The replication information of the `Sync` processing unit. This information is about the same component with the current processing unit.
- `masterBinlog`: The binlog position in the upstream database.
- `masterBinlogGtid`: The GTID information in the upstream database.
- `syncerBinlog`: The position of the binlog that has been replicated in the `Sync` processing unit.
- `syncerBinlogGtid`: The binlog position replicated using GTID.
- `blockingDDLs`: The DDL list that is blocked currently. It is not empty only when all the upstream tables of this DM-worker are in the "synced" status. In this case, it indicates the sharding DDL statements to be executed or that are skipped.
- `unresolvedGroups`: The sharding group that is not resolved. Each group contains the following fields:
- `target`: The downstream database table to be replicated.
- `DDLs`: A list of DDL statements.
- `firstPos`: The starting position of the sharding DDL statement.
- `synced`: The upstream sharded table whose executed sharding DDL statement has been read by the `Sync` unit.
- `unsynced`: The upstream table that has not executed this sharding DDL statement. If any upstream tables have not finished replication, `blockingDDLs` is empty.
- `synced`: Whether the incremental replication catches up with the upstream and has the same binlog position as that in the upstream. The save point is not refreshed in real time in the `Sync` background, so `false` of `synced` does not always mean a replication delay exits.
- `totalRows`: The total number of rows that are replicated in this subtask.
- `totalRps`: The number of rows that are replicated in this subtask per second.
- `recentRps`: The number of rows that are replicated in this subtask in the last second.
- `load`: The replication information of the `Load` processing unit.
- `finishedBytes`: The number of bytes that have been loaded.
- `totalBytes`: The total number of bytes that need to be loaded.
- `progress`: The progress of the loading process.
- `bps`: The speed of the full loading.
- `dump`: The replication information of the `Dump` processing unit.
- `totalTables`: The number of tables to be dumped.
- `completedTables`: The number of tables that have been dumped.
- `finishedBytes`: The number of bytes that have been dumped.
- `finishedRows`: The number of rows that have been dumped.
- `estimateTotalRows`: The estimated number of rows to be dumped.
- `progress`: The progress of the dumping process.
- `bps`: The dumping speed in bytes/second.

## Subtask status

Expand Down

0 comments on commit 3419db0

Please sign in to comment.