You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: ticdc/ticdc-split-update-behavior.md
+9-5Lines changed: 9 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -7,12 +7,16 @@ summary: Introduce the behavior changes about whether TiCDC splits `UPDATE` even
7
7
8
8
## Split `UPDATE` events for MySQL sinks
9
9
10
-
In v8.1.0, when using the MySQL sink, TiCDC fetches the current timestamp `thresholdTS` from PD at startup and decides whether to split `UPDATE` events based on the value of this timestamp:
10
+
Starting from v6.5.10, v7.5.2, and v8.1.1, when using the MySQL sink, any TiCDC node that receives a request for replicating a table will fetch the current timestamp `thresholdTS` from PD before starting the replication to the downstream. Based on the value of this timestamp, TiCDC decides whether to split `UPDATE` events:
11
11
12
12
- For transactions containing one or multiple `UPDATE` changes, if the transaction `commitTS` is less than `thresholdTS`, TiCDC splits the `UPDATE` event into a `DELETE` event and an `INSERT` event before writing them to the Sorter module.
13
13
- For `UPDATE` events with the transaction `commitTS` greater than or equal to `thresholdTS`, TiCDC does not split them. For more information, see GitHub issue [#10918](https://github.com/pingcap/tiflow/issues/10918).
14
14
15
-
This behavior change addresses the issue of downstream data inconsistencies caused by the potentially incorrect order of `UPDATE` events received by TiCDC, which can lead to an incorrect order of split `DELETE` and `INSERT` events.
15
+
> **Note:**
16
+
>
17
+
> In v8.1.0, when using MySQL Sink, TiCDC also decides whether to split `UPDATE` events based on the value of `thresholdTS`, but `thresholdTS` is obtained differently. Specifically, in v8.1.0, `thresholdTS` is the current timestamp fetched from PD at TiCDC startup, but this way might cause data inconsistency issues in multi-node scenarios. For more information, see GitHub issue [#11219](https://github.com/pingcap/tiflow/issues/11219).
18
+
19
+
This behavior change (that is, deciding whether to split `UPDATE` events based on `thresholdTS`) addresses the issue of downstream data inconsistencies caused by the potentially incorrect order of `UPDATE` events received by TiCDC, which can lead to an incorrect order of split `DELETE` and `INSERT` events.
16
20
17
21
Take the following SQL statements as an example:
18
22
@@ -49,7 +53,7 @@ UPDATE t SET a = 3 WHERE a = 2;
49
53
50
54
After the downstream executes the transaction, the records in the database are `(3, 2)`, which are different from the records in the upstream database (`(2, 1)`and`(3, 2)`), indicating a data inconsistency issue.
51
55
52
-
- After this behavior change, if the transaction `commitTS` is less than the `thresholdTS`obtained by TiCDC at startup, TiCDC splits these `UPDATE` events into `DELETE`and`INSERT` events before writing them to the Sorter module. After the sorting by the Sorter module, the actual execution order of these events in the downstream is as follows:
56
+
- After this behavior change, if the transaction `commitTS` is less than the `thresholdTS`fetched from PD when TiCDC starts replicating the corresponding table to the downstream, TiCDC splits these `UPDATE` events into `DELETE`and`INSERT` events before writing them to the Sorter module. After the sorting by the Sorter module, the actual execution order of these events in the downstream is as follows:
53
57
54
58
```sql
55
59
BEGIN;
@@ -145,11 +149,11 @@ Starting from v6.5.10, v7.1.6, v7.5.3, and v8.1.1, when using a non-MySQL sink,
145
149
| Version | Protocol | Split UK/PK `UPDATE` events | Not split UK/PK `UPDATE` events | Comments |
146
150
| -- | -- | -- | -- | -- |
147
151
| <=v7.5.2 | ALL | ✓ | ✗ |
148
-
| \>=v7.5.3(not released yet) | ALL | ✓ (Default value:`output-raw-change-event = false`) | ✓ (Optional: `output-raw-change-event = true`) | |
0 commit comments