From 8fd6013bfd9448ea39fc9c9785628aa47c1d5aff Mon Sep 17 00:00:00 2001 From: Tetsuya Hasegawa Date: Sun, 28 Jan 2024 21:21:31 +0900 Subject: [PATCH 1/7] Note that GC disable may be skipped for Dumpling --- migrate-from-tidb-to-mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migrate-from-tidb-to-mysql.md b/migrate-from-tidb-to-mysql.md index 9181c70fa1147..cea0af4e16838 100644 --- a/migrate-from-tidb-to-mysql.md +++ b/migrate-from-tidb-to-mysql.md @@ -56,7 +56,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) 1. Disable Garbage Collection (GC). - To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. + To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. (This step can be skipped if the TiDB version is later than or equal to v4.0.0 and certain conditions are met. For more details, please refer [here](/dumpling-overview#manually-set-the-tidb-gc-time)) Run the following command to disable GC: From d1e0c6d5c550b2af11fccebc4619f93406d40170 Mon Sep 17 00:00:00 2001 From: Tetsuya Hasegawa Date: Sun, 28 Jan 2024 22:01:56 +0900 Subject: [PATCH 2/7] Fix link format mistake --- migrate-from-tidb-to-mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migrate-from-tidb-to-mysql.md b/migrate-from-tidb-to-mysql.md index cea0af4e16838..fd1090abf01bf 100644 --- a/migrate-from-tidb-to-mysql.md +++ b/migrate-from-tidb-to-mysql.md @@ -56,7 +56,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) 1. Disable Garbage Collection (GC). - To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. (This step can be skipped if the TiDB version is later than or equal to v4.0.0 and certain conditions are met. For more details, please refer [here](/dumpling-overview#manually-set-the-tidb-gc-time)) + To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. (This step can be skipped if the TiDB version is later than or equal to v4.0.0 and certain conditions are met. For more details, please refer [here](/dumpling-overview.md#manually-set-the-tidb-gc-time)) Run the following command to disable GC: From c398f2ea07ad726fbd4b0635981452b9f60d1fa5 Mon Sep 17 00:00:00 2001 From: Tetsuya Hasegawa Date: Wed, 7 Feb 2024 22:38:58 +0900 Subject: [PATCH 3/7] Update migrate-from-tidb-to-mysql.md Co-authored-by: Aolin --- migrate-from-tidb-to-mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migrate-from-tidb-to-mysql.md b/migrate-from-tidb-to-mysql.md index fd1090abf01bf..86fa077c8cda9 100644 --- a/migrate-from-tidb-to-mysql.md +++ b/migrate-from-tidb-to-mysql.md @@ -56,7 +56,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) 1. Disable Garbage Collection (GC). - To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. (This step can be skipped if the TiDB version is later than or equal to v4.0.0 and certain conditions are met. For more details, please refer [here](/dumpling-overview.md#manually-set-the-tidb-gc-time)) + To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. For TiDB v4.0.0 and later versions, you can skip this step under specific conditions. For more details, see [Manually set the TiDB GC time](/dumpling-overview.md#manually-set-the-tidb-gc-time). Run the following command to disable GC: From 7d770a64b93370ce0103e959988d70aa0ed4c4ec Mon Sep 17 00:00:00 2001 From: Tetsuya Hasegawa Date: Wed, 7 Feb 2024 22:54:12 +0900 Subject: [PATCH 4/7] Update migrate-from-tidb-to-mysql.md --- migrate-from-tidb-to-mysql.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/migrate-from-tidb-to-mysql.md b/migrate-from-tidb-to-mysql.md index 86fa077c8cda9..5acc8719c41c2 100644 --- a/migrate-from-tidb-to-mysql.md +++ b/migrate-from-tidb-to-mysql.md @@ -56,7 +56,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) 1. Disable Garbage Collection (GC). - To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. For TiDB v4.0.0 and later versions, you can skip this step under specific conditions. For more details, see [Manually set the TiDB GC time](/dumpling-overview.md#manually-set-the-tidb-gc-time). + To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. This is a must step since even though Dumpling may [auto-extend the GC](/dumpling-overview.md#manually-set-the-tidb-gc-time) for TiDB v4.0.0 and later versions, after the Dumpling exits, GC may kick in, leading to failure in migrating the incremental change. Run the following command to disable GC: @@ -83,7 +83,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) 1 row in set (0.00 sec) ``` -2. Back up data. +3. Back up data. 1. Export data in SQL format using Dumpling: @@ -106,7 +106,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) Finished dump at: 2022-06-28 17:49:57 ``` -3. Restore data. +4. Restore data. Use MyLoader (an open-source tool) to import data to the downstream MySQL instance. For details about how to install and use MyLoader, see [MyDumpler/MyLoader](https://github.com/mydumper/mydumper). Note that you need to use MyLoader v0.10 or earlier versions. Higher versions cannot process metadata files exported by Dumpling. @@ -116,7 +116,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) myloader -h 127.0.0.1 -P 3306 -d ./dumpling_output/ ``` -4. (Optional) Validate data. +5. (Optional) Validate data. You can use [sync-diff-inspector](/sync-diff-inspector/sync-diff-inspector-overview.md) to check data consistency between upstream and downstream at a certain time. From d4e49203222e9d314ac02e1b73ee5cfab52c1ac9 Mon Sep 17 00:00:00 2001 From: Tetsuya Hasegawa Date: Wed, 7 Feb 2024 22:55:35 +0900 Subject: [PATCH 5/7] Update migrate-from-tidb-to-mysql.md --- migrate-from-tidb-to-mysql.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/migrate-from-tidb-to-mysql.md b/migrate-from-tidb-to-mysql.md index 5acc8719c41c2..1f5dc2381eacf 100644 --- a/migrate-from-tidb-to-mysql.md +++ b/migrate-from-tidb-to-mysql.md @@ -83,7 +83,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) 1 row in set (0.00 sec) ``` -3. Back up data. +2. Back up data. 1. Export data in SQL format using Dumpling: @@ -106,7 +106,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) Finished dump at: 2022-06-28 17:49:57 ``` -4. Restore data. +3. Restore data. Use MyLoader (an open-source tool) to import data to the downstream MySQL instance. For details about how to install and use MyLoader, see [MyDumpler/MyLoader](https://github.com/mydumper/mydumper). Note that you need to use MyLoader v0.10 or earlier versions. Higher versions cannot process metadata files exported by Dumpling. @@ -116,7 +116,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) myloader -h 127.0.0.1 -P 3306 -d ./dumpling_output/ ``` -5. (Optional) Validate data. +4. (Optional) Validate data. You can use [sync-diff-inspector](/sync-diff-inspector/sync-diff-inspector-overview.md) to check data consistency between upstream and downstream at a certain time. From bfac2301674bc1087f7a288312b6b4048a38b196 Mon Sep 17 00:00:00 2001 From: Tetsuya Hasegawa Date: Thu, 8 Feb 2024 12:51:35 +0900 Subject: [PATCH 6/7] Update migrate-from-tidb-to-mysql.md Co-authored-by: Aolin --- migrate-from-tidb-to-mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migrate-from-tidb-to-mysql.md b/migrate-from-tidb-to-mysql.md index 1f5dc2381eacf..7b903e38a6445 100644 --- a/migrate-from-tidb-to-mysql.md +++ b/migrate-from-tidb-to-mysql.md @@ -56,7 +56,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) 1. Disable Garbage Collection (GC). - To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. This is a must step since even though Dumpling may [auto-extend the GC](/dumpling-overview.md#manually-set-the-tidb-gc-time) for TiDB v4.0.0 and later versions, after the Dumpling exits, GC may kick in, leading to failure in migrating the incremental change. + To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. For TiDB v4.0.0 and later versions, Dumpling might [automatically extend the GC time](/dumpling-overview.md#manually-set-the-tidb-gc-time). Nevertheless, manually disable GC is still necessary because the GC process might begin after Dumpling exits, leading to the failure of incremental changes migration. Run the following command to disable GC: From f767868932adfe68097cf54e7e11954d06ea959b Mon Sep 17 00:00:00 2001 From: Tetsuya Hasegawa Date: Thu, 8 Feb 2024 14:57:19 +0900 Subject: [PATCH 7/7] Update migrate-from-tidb-to-mysql.md Co-authored-by: Ran --- migrate-from-tidb-to-mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migrate-from-tidb-to-mysql.md b/migrate-from-tidb-to-mysql.md index 7b903e38a6445..716b167ae18d1 100644 --- a/migrate-from-tidb-to-mysql.md +++ b/migrate-from-tidb-to-mysql.md @@ -56,7 +56,7 @@ After setting up the environment, you can use [Dumpling](/dumpling-overview.md) 1. Disable Garbage Collection (GC). - To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. For TiDB v4.0.0 and later versions, Dumpling might [automatically extend the GC time](/dumpling-overview.md#manually-set-the-tidb-gc-time). Nevertheless, manually disable GC is still necessary because the GC process might begin after Dumpling exits, leading to the failure of incremental changes migration. + To ensure that newly written data is not deleted during incremental migration, you should disable GC for the upstream cluster before exporting full data. In this way, history data is not deleted. For TiDB v4.0.0 and later versions, Dumpling might [automatically extend the GC time](/dumpling-overview.md#manually-set-the-tidb-gc-time). Nevertheless, manually disabling GC is still necessary because the GC process might begin after Dumpling exits, leading to the failure of incremental changes migration. Run the following command to disable GC: