From 0d78805fa0152f1198c7edb2112b46573a241d45 Mon Sep 17 00:00:00 2001 From: houfaxin Date: Wed, 7 Aug 2024 08:46:21 +0800 Subject: [PATCH 1/5] initial trans --- accelerated-table-creation.md | 20 ++------------------ upgrade-tidb-using-tiup.md | 1 + 2 files changed, 3 insertions(+), 18 deletions(-) diff --git a/accelerated-table-creation.md b/accelerated-table-creation.md index f97eec9d5cfd0..751aab58c86cc 100644 --- a/accelerated-table-creation.md +++ b/accelerated-table-creation.md @@ -8,9 +8,9 @@ aliases: ['/tidb/dev/ddl-v2/'] TiDB v7.6.0 introduces the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_enable_fast_create_table-new-in-v800) to support accelerating table creation, which improves the efficiency of bulk table creation. Starting from v8.0.0, this system variable is renamed to [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). -TiDB uses the online asynchronous schema change algorithm to change the metadata. All DDL jobs are submitted to the `mysql.tidb_ddl_job` table, and the owner node pulls the DDL job to execute. After executing each phase of the online DDL algorithm, the DDL job is marked as completed and moved to the `mysql.tidb_ddl_history` table. Therefore, DDL statements can only be executed on the owner node and cannot be linearly extended. +When accelerated table creation is enabled, table creation statements for the same schema submitted to the same TiDB node at the same time will be merged into a batch table creation statement to improve table creation performance. Therefore, to improve the table creation performance, you need to connect to the same TiDB node to create tables under the same schema concurrently and increase the concurrency appropriately. -However, for some DDL statements, it is not necessary to strictly follow the online DDL algorithm. For example, the `CREATE TABLE` statement only has two states for the job: `none` and `public`. Therefore, TiDB can simplify the execution process of DDL, and executes the `CREATE TABLE` statement on a non-owner node to accelerate table creation. +The merged bulk table creation statements are executed within the same transaction, so if one of them fails, all of them will fail. > **Warning:** > @@ -39,19 +39,3 @@ To disable performance optimization for creating tables, set the value of this v ```sql SET GLOBAL tidb_enable_fast_create_table = OFF; ``` - -## Implementation principle - -The detailed implementation principle of performance optimization for table creation is as follows: - -1. Create a `CREATE TABLE` Job. - - The corresponding DDL Job is generated by parsing the `CREATE TABLE` statement. - -2. Execute the `CREATE TABLE` job. - - The TiDB node that receives the `CREATE TABLE` statement executes it directly, and then persists the table structure to TiKV. At the same time, the `CREATE TABLE` job is marked as completed and inserted into the `mysql.tidb_ddl_history` table. - -3. Synchronize the table information. - - TiDB notifies other nodes to synchronize the newly created table structure. diff --git a/upgrade-tidb-using-tiup.md b/upgrade-tidb-using-tiup.md index 66e8a904ea159..9f8866bc8854f 100644 --- a/upgrade-tidb-using-tiup.md +++ b/upgrade-tidb-using-tiup.md @@ -21,6 +21,7 @@ This document is targeted for the following upgrade paths: > 3. **DO NOT** upgrade a TiDB cluster when a DDL statement is being executed in the cluster (usually for the time-consuming DDL statements such as `ADD INDEX` and the column type changes). Before the upgrade, it is recommended to use the [`ADMIN SHOW DDL`](/sql-statements/sql-statement-admin-show-ddl.md) command to check whether the TiDB cluster has an ongoing DDL job. If the cluster has a DDL job, to upgrade the cluster, wait until the DDL execution is finished or use the [`ADMIN CANCEL DDL`](/sql-statements/sql-statement-admin-cancel-ddl.md) command to cancel the DDL job before you upgrade the cluster. > 4. If the TiDB version before upgrade is 7.1.0 or later, you can ignore the preceding warnings 2 and 3. For more information, see [limitations on using TiDB smooth upgrade](/smooth-upgrade-tidb.md#limitations). > 5. Be sure to read [limitations on user operations](/smooth-upgrade-tidb.md#limitations-on-user-operations) before upgrading your TiDB cluster using TiUP. +> 6. If the current cluster version is from TiDB v7.6 to v8.2, the upgrade target version is v8.3.0 or later, and [accelerated table creation](/accelerated-table-creation.md) is enabled, you need to disable this feature first, and then enable it as needed after the upgrade is completed. Otherwise, some metadata KVs added by this feature will remain in the cluster. Starting from v8.3.0, this feature has been optimized. Upgrading to a higher version will no longer generate and retain this type of metadata KVs. > **Note:** > From 34eb2849a9721bfb3a15de9386a8345e62400970 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 7 Aug 2024 09:06:21 +0800 Subject: [PATCH 2/5] Update upgrade-tidb-using-tiup.md --- upgrade-tidb-using-tiup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/upgrade-tidb-using-tiup.md b/upgrade-tidb-using-tiup.md index 9f8866bc8854f..d9e1a2ed7b336 100644 --- a/upgrade-tidb-using-tiup.md +++ b/upgrade-tidb-using-tiup.md @@ -21,7 +21,7 @@ This document is targeted for the following upgrade paths: > 3. **DO NOT** upgrade a TiDB cluster when a DDL statement is being executed in the cluster (usually for the time-consuming DDL statements such as `ADD INDEX` and the column type changes). Before the upgrade, it is recommended to use the [`ADMIN SHOW DDL`](/sql-statements/sql-statement-admin-show-ddl.md) command to check whether the TiDB cluster has an ongoing DDL job. If the cluster has a DDL job, to upgrade the cluster, wait until the DDL execution is finished or use the [`ADMIN CANCEL DDL`](/sql-statements/sql-statement-admin-cancel-ddl.md) command to cancel the DDL job before you upgrade the cluster. > 4. If the TiDB version before upgrade is 7.1.0 or later, you can ignore the preceding warnings 2 and 3. For more information, see [limitations on using TiDB smooth upgrade](/smooth-upgrade-tidb.md#limitations). > 5. Be sure to read [limitations on user operations](/smooth-upgrade-tidb.md#limitations-on-user-operations) before upgrading your TiDB cluster using TiUP. -> 6. If the current cluster version is from TiDB v7.6 to v8.2, the upgrade target version is v8.3.0 or later, and [accelerated table creation](/accelerated-table-creation.md) is enabled, you need to disable this feature first, and then enable it as needed after the upgrade is completed. Otherwise, some metadata KVs added by this feature will remain in the cluster. Starting from v8.3.0, this feature has been optimized. Upgrading to a higher version will no longer generate and retain this type of metadata KVs. +> 6. If the current cluster version is from TiDB v7.6.0 to v8.2.0, the upgrade target version is v8.3.0 or later, and [accelerated table creation](/accelerated-table-creation.md) is enabled, you need to disable this feature first, and then enable it as needed after the upgrade is completed. Otherwise, some metadata KVs added by this feature will remain in the cluster. Starting from v8.3.0, this feature has been optimized. Upgrading to a higher version will no longer generate and retain this type of metadata KVs. > **Note:** > From bc5adf025527b5c627b3bd3c5af044b7c1ab37fe Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 7 Aug 2024 14:58:19 +0800 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Frank945946 <108602632+Frank945946@users.noreply.github.com> --- accelerated-table-creation.md | 2 +- upgrade-tidb-using-tiup.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/accelerated-table-creation.md b/accelerated-table-creation.md index 751aab58c86cc..de387cead49ee 100644 --- a/accelerated-table-creation.md +++ b/accelerated-table-creation.md @@ -8,7 +8,7 @@ aliases: ['/tidb/dev/ddl-v2/'] TiDB v7.6.0 introduces the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_enable_fast_create_table-new-in-v800) to support accelerating table creation, which improves the efficiency of bulk table creation. Starting from v8.0.0, this system variable is renamed to [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). -When accelerated table creation is enabled, table creation statements for the same schema submitted to the same TiDB node at the same time will be merged into a batch table creation statement to improve table creation performance. Therefore, to improve the table creation performance, you need to connect to the same TiDB node to create tables under the same schema concurrently and increase the concurrency appropriately. +When [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800) is enabled, table creation statements for the same schema submitted to the same TiDB node at the same time will be merged into a batch table creation statement to improve table creation performance. Therefore, to improve the table creation performance, you need several sessions to connect to the same TiDB node to create tables under the same schema concurrently and increase the concurrency appropriately. The merged bulk table creation statements are executed within the same transaction, so if one of them fails, all of them will fail. diff --git a/upgrade-tidb-using-tiup.md b/upgrade-tidb-using-tiup.md index d9e1a2ed7b336..f10f4fd8ced7d 100644 --- a/upgrade-tidb-using-tiup.md +++ b/upgrade-tidb-using-tiup.md @@ -21,7 +21,7 @@ This document is targeted for the following upgrade paths: > 3. **DO NOT** upgrade a TiDB cluster when a DDL statement is being executed in the cluster (usually for the time-consuming DDL statements such as `ADD INDEX` and the column type changes). Before the upgrade, it is recommended to use the [`ADMIN SHOW DDL`](/sql-statements/sql-statement-admin-show-ddl.md) command to check whether the TiDB cluster has an ongoing DDL job. If the cluster has a DDL job, to upgrade the cluster, wait until the DDL execution is finished or use the [`ADMIN CANCEL DDL`](/sql-statements/sql-statement-admin-cancel-ddl.md) command to cancel the DDL job before you upgrade the cluster. > 4. If the TiDB version before upgrade is 7.1.0 or later, you can ignore the preceding warnings 2 and 3. For more information, see [limitations on using TiDB smooth upgrade](/smooth-upgrade-tidb.md#limitations). > 5. Be sure to read [limitations on user operations](/smooth-upgrade-tidb.md#limitations-on-user-operations) before upgrading your TiDB cluster using TiUP. -> 6. If the current cluster version is from TiDB v7.6.0 to v8.2.0, the upgrade target version is v8.3.0 or later, and [accelerated table creation](/accelerated-table-creation.md) is enabled, you need to disable this feature first, and then enable it as needed after the upgrade is completed. Otherwise, some metadata KVs added by this feature will remain in the cluster. Starting from v8.3.0, this feature has been optimized. Upgrading to a higher version will no longer generate and retain this type of metadata KVs. +> 6. If the current cluster version is from TiDB v7.6.0 to v8.2.0, the upgrade target version is v8.3.0 or later, and [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800) is enabled, you need to disable this feature first, and then enable it as needed after the upgrade is completed. Otherwise, some metadata KVs added by this feature will remain in the cluster. Starting from v8.3.0, this feature has been optimized. Upgrading to a higher version will no longer generate and retain this type of metadata KVs. > **Note:** > From 0b1344b17e82636b83b89aeced64fa9915e1f7b6 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 12 Aug 2024 15:29:39 +0800 Subject: [PATCH 4/5] Apply suggestions from code review Co-authored-by: Lilian Lee --- accelerated-table-creation.md | 4 ++-- upgrade-tidb-using-tiup.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/accelerated-table-creation.md b/accelerated-table-creation.md index de387cead49ee..6338b9c80c754 100644 --- a/accelerated-table-creation.md +++ b/accelerated-table-creation.md @@ -8,9 +8,9 @@ aliases: ['/tidb/dev/ddl-v2/'] TiDB v7.6.0 introduces the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_enable_fast_create_table-new-in-v800) to support accelerating table creation, which improves the efficiency of bulk table creation. Starting from v8.0.0, this system variable is renamed to [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). -When [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800) is enabled, table creation statements for the same schema submitted to the same TiDB node at the same time will be merged into a batch table creation statement to improve table creation performance. Therefore, to improve the table creation performance, you need several sessions to connect to the same TiDB node to create tables under the same schema concurrently and increase the concurrency appropriately. +When you enable accelerated table creation via [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800), table creation statements with the same schema committed to the same TiDB node at the same time are merged into batch table creation statements to improve table creation performance. Therefore, to improve the table creation performance, try to connect to the same TiDB node, create tables with the same schema concurrently, and increase the concurrency appropriately. -The merged bulk table creation statements are executed within the same transaction, so if one of them fails, all of them will fail. +The merged batch table creation statements are executed within the same transaction, so if one statement of them fails, all of them will fail. > **Warning:** > diff --git a/upgrade-tidb-using-tiup.md b/upgrade-tidb-using-tiup.md index f10f4fd8ced7d..b3069dc80abd0 100644 --- a/upgrade-tidb-using-tiup.md +++ b/upgrade-tidb-using-tiup.md @@ -21,7 +21,7 @@ This document is targeted for the following upgrade paths: > 3. **DO NOT** upgrade a TiDB cluster when a DDL statement is being executed in the cluster (usually for the time-consuming DDL statements such as `ADD INDEX` and the column type changes). Before the upgrade, it is recommended to use the [`ADMIN SHOW DDL`](/sql-statements/sql-statement-admin-show-ddl.md) command to check whether the TiDB cluster has an ongoing DDL job. If the cluster has a DDL job, to upgrade the cluster, wait until the DDL execution is finished or use the [`ADMIN CANCEL DDL`](/sql-statements/sql-statement-admin-cancel-ddl.md) command to cancel the DDL job before you upgrade the cluster. > 4. If the TiDB version before upgrade is 7.1.0 or later, you can ignore the preceding warnings 2 and 3. For more information, see [limitations on using TiDB smooth upgrade](/smooth-upgrade-tidb.md#limitations). > 5. Be sure to read [limitations on user operations](/smooth-upgrade-tidb.md#limitations-on-user-operations) before upgrading your TiDB cluster using TiUP. -> 6. If the current cluster version is from TiDB v7.6.0 to v8.2.0, the upgrade target version is v8.3.0 or later, and [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800) is enabled, you need to disable this feature first, and then enable it as needed after the upgrade is completed. Otherwise, some metadata KVs added by this feature will remain in the cluster. Starting from v8.3.0, this feature has been optimized. Upgrading to a higher version will no longer generate and retain this type of metadata KVs. +> 6. If the version of your current TiDB cluster is TiDB v7.6.0 to v8.2.0, the upgrade target version is v8.3.0 or later, and accelerated table creation is enabled via [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800), you need to first disable the accelerated table creation feature, and then enable it as needed after the upgrade is completed. Otherwise, some metadata KVs added by this feature remain in the cluster. Starting from v8.3.0, this feature is optimized. Upgrading to a later TiDB version no longer generates and retains this type of metadata KVs. > **Note:** > From b5cdaf79a05f054234d1408b9443adbb937cd42f Mon Sep 17 00:00:00 2001 From: lilin90 Date: Mon, 12 Aug 2024 15:33:25 +0800 Subject: [PATCH 5/5] Update wording --- accelerated-table-creation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accelerated-table-creation.md b/accelerated-table-creation.md index 6338b9c80c754..ec3ee720f825c 100644 --- a/accelerated-table-creation.md +++ b/accelerated-table-creation.md @@ -8,7 +8,7 @@ aliases: ['/tidb/dev/ddl-v2/'] TiDB v7.6.0 introduces the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_enable_fast_create_table-new-in-v800) to support accelerating table creation, which improves the efficiency of bulk table creation. Starting from v8.0.0, this system variable is renamed to [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). -When you enable accelerated table creation via [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800), table creation statements with the same schema committed to the same TiDB node at the same time are merged into batch table creation statements to improve table creation performance. Therefore, to improve the table creation performance, try to connect to the same TiDB node, create tables with the same schema concurrently, and increase the concurrency appropriately. +When accelerated table creation is enabled via [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800), table creation statements with the same schema committed to the same TiDB node at the same time are merged into batch table creation statements to improve table creation performance. Therefore, to improve the table creation performance, try to connect to the same TiDB node, create tables with the same schema concurrently, and increase the concurrency appropriately. The merged batch table creation statements are executed within the same transaction, so if one statement of them fails, all of them will fail.