Skip to content

Commit

Permalink
Update tidb-distributed-execution-framework.md (#15606)
Browse files Browse the repository at this point in the history
  • Loading branch information
hfxsd authored Dec 12, 2023
1 parent d2f02a4 commit 8286f4b
Show file tree
Hide file tree
Showing 14 changed files with 90 additions and 77 deletions.
5 changes: 3 additions & 2 deletions TOC-tidb-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,9 @@
- [TiDB Serverless Limitations](/tidb-cloud/serverless-limitations.md)
- [Limited SQL Features on TiDB Cloud](/tidb-cloud/limited-sql-features.md)
- [TiDB Limitations](/tidb-limitations.md)
- TiDB Distributed eXecution Framework (DXF)
- [Introduction](/tidb-distributed-execution-framework.md)
- [TiDB Global Sort](/tidb-global-sort.md)
- Benchmarks
- [TPC-C Performance Test Report](/tidb-cloud/v7.1.0-performance-benchmarking-with-tpcc.md)
- [Sysbench Performance Test Report](/tidb-cloud/v7.1.0-performance-benchmarking-with-sysbench.md)
Expand Down Expand Up @@ -621,8 +624,6 @@
- [update](/tidb-cloud/ticloud-update.md)
- [Table Filter](/table-filter.md)
- [Resource Control](/tidb-resource-control.md)
- [TiDB Backend Task Distributed Execution Framework](/tidb-distributed-execution-framework.md)
- [TiDB Global Sort](/tidb-global-sort.md)
- [DDL Execution Principles and Best Practices](/ddl-introduction.md)
- [Troubleshoot Inconsistency Between Data and Indexes](/troubleshoot-data-inconsistency-errors.md)
- [Support](/tidb-cloud/tidb-cloud-support.md)
Expand Down
6 changes: 3 additions & 3 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -644,6 +644,9 @@
- [Data Validation](/tiflash/tiflash-data-validation.md)
- [Compatibility](/tiflash/tiflash-compatibility.md)
- [Pipeline Execution Model](/tiflash/tiflash-pipeline-model.md)
- TiDB Distributed eXecution Framework (DXF)
- [Introduction](/tidb-distributed-execution-framework.md)
- [TiDB Global Sort](/tidb-global-sort.md)
- [System Variables](/system-variables.md)
- [Server Status Variables](/status-variables.md)
- Configuration File Parameters
Expand Down Expand Up @@ -1000,9 +1003,6 @@
- [Table Filter](/table-filter.md)
- [Schedule Replicas by Topology Labels](/schedule-replicas-by-topology-labels.md)
- [URI Formats of External Storage Services](/external-storage-uri.md)
- Internal Components
- [TiDB Backend Task Distributed Execution Framework](/tidb-distributed-execution-framework.md)
- [TiDB Global Sort](/tidb-global-sort.md)
- FAQs
- [FAQ Summary](/faq/faq-overview.md)
- [TiDB FAQs](/faq/tidb-faq.md)
Expand Down
2 changes: 1 addition & 1 deletion basic-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ You can try out TiDB features on [TiDB Playground](https://play.tidbcloud.com/?u
| [Runaway Queries management](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries) | E | N | N | N | N | N | N | N | N | N |
| [Background tasks management](/tidb-resource-control.md#manage-background-tasks) | E | N | N | N | N | N | N | N | N | N |
| [TiFlash Disaggregated Storage and Compute Architecture and S3 Support](/tiflash/tiflash-disaggregated-and-s3.md) | Y | E | N | N | N | N | N | N | N | N |
| [Selecting TiDB nodes for the distributed framework tasks](/system-variables.md#tidb_service_scope-new-in-v740) | Y | N | N | N | N | N | N | N | N | N |
| [Selecting TiDB nodes for the Distributed eXecution Framework (DXF) tasks](/system-variables.md#tidb_service_scope-new-in-v740) | Y | N | N | N | N | N | N | N | N | N |

[^1]: TiDB incorrectly treats latin1 as a subset of utf8. See [TiDB #18955](https://github.com/pingcap/tidb/issues/18955) for more details.

Expand Down
2 changes: 1 addition & 1 deletion external-storage-uri.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ s3://external/testfolder?access-key=${access-key}&secret-access-key=${secret-acc
The following is an example of an Amazon S3 URI for [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md). In this example, you need to specify a specific filename `test.csv`.

```shell
s3://external/test.csv?access-key=${access-key}&secret-access-key=${secret-access-key}"
s3://external/test.csv?access-key=${access-key}&secret-access-key=${secret-access-key}
```

## GCS URI format
Expand Down
8 changes: 4 additions & 4 deletions releases/release-7.1.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,11 +128,11 @@ Compared with the previous LTS 6.5.0, 7.1.0 not only includes new features, impr

For more information, see [documentation](/sql-non-prepared-plan-cache.md).

* Support the DDL distributed parallel execution framework (experimental) [#41495](https://github.com/pingcap/tidb/issues/41495) @[benjamin2037](https://github.com/benjamin2037)
* Support the TiDB Distributed eXecution Framework (DXF) (experimental) [#41495](https://github.com/pingcap/tidb/issues/41495) @[benjamin2037](https://github.com/benjamin2037)

Before TiDB v7.1.0, only one TiDB node can serve as the DDL owner and execute DDL tasks at the same time. Starting from TiDB v7.1.0, in the new distributed parallel execution framework, multiple TiDB nodes can execute the same DDL task in parallel, thus better utilizing the resources of the TiDB cluster and significantly improving the performance of DDL. In addition, you can linearly improve the performance of DDL by adding more TiDB nodes. Note that this feature is currently experimental and only supports `ADD INDEX` operations.
Before TiDB v7.1.0, only one TiDB node can serve as the DDL owner and execute DDL tasks at the same time. Starting from TiDB v7.1.0, in the new DXF, multiple TiDB nodes can execute the same DDL task in parallel, thus better utilizing the resources of the TiDB cluster and significantly improving the performance of DDL. In addition, you can linearly improve the performance of DDL by adding more TiDB nodes. Note that this feature is currently experimental and only supports `ADD INDEX` operations.

To use the distributed framework, set the value of [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) to `ON`:
To use the DXF, set the value of [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) to `ON`:

```sql
SET GLOBAL tidb_enable_dist_task = ON;
Expand Down Expand Up @@ -332,7 +332,7 @@ Compared with the previous LTS 6.5.0, 7.1.0 not only includes new features, impr
| [`authentication_ldap_simple_server_host`](/system-variables.md#authentication_ldap_simple_server_host-new-in-v710) | Newly added | Specifies the LDAP server host in LDAP simple authentication. |
| [`authentication_ldap_simple_server_port`](/system-variables.md#authentication_ldap_simple_server_port-new-in-v710) | Newly added | Specifies the LDAP server TCP/IP port number in LDAP simple authentication. |
| [`authentication_ldap_simple_tls`](/system-variables.md#authentication_ldap_simple_tls-new-in-v710) | Newly added | Specifies whether connections by the plugin to the LDAP server are protected with StartTLS in LDAP simple authentication. |
| [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) | Newly added | Controls whether to enable the distributed execution framework. After enabling distributed execution, DDL, import, and other supported backend tasks will be jointly completed by multiple TiDB nodes in the cluster. This variable was renamed from `tidb_ddl_distribute_reorg`. |
| [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) | Newly added | Controls whether to enable the Distributed eXecution Framework (DXF). After enabling the DXF, DDL, import, and other supported DXF tasks will be jointly completed by multiple TiDB nodes in the cluster. This variable was renamed from `tidb_ddl_distribute_reorg`. |
| [`tidb_enable_non_prepared_plan_cache_for_dml`](/system-variables.md#tidb_enable_non_prepared_plan_cache_for_dml-new-in-v710) | Newly added | Controls whether to enable the [Non-prepared plan cache](/sql-non-prepared-plan-cache.md) feature for DML statements. |
| [`tidb_enable_row_level_checksum`](/system-variables.md#tidb_enable_row_level_checksum-new-in-v710) | Newly added | Controls whether to enable the TiCDC data integrity validation for single-row data feature.|
| [`tidb_opt_fix_control`](/system-variables.md#tidb_opt_fix_control-new-in-v710) | Newly added | This variable provides more fine-grained control over the optimizer and helps to prevent performance regression after upgrading caused by behavior changes in the optimizer. |
Expand Down
2 changes: 1 addition & 1 deletion releases/release-7.1.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.1/quick-start-with-
- Fix the issue that the cluster upgrade fails when there are paused DDL operations before the upgrade [#44225](https://github.com/pingcap/tidb/issues/44225) @[zimulala](https://github.com/zimulala)
- Fix the `duplicate entry` error that occurs when restoring a table with `AUTO_ID_CACHE=1` using BR [#44716](https://github.com/pingcap/tidb/issues/44716) @[tiancaiamao](https://github.com/tiancaiamao)
- Fix the data index inconsistency issue triggered by multiple switches of DDL owner [#44619](https://github.com/pingcap/tidb/issues/44619) @[tangenta](https://github.com/tangenta)
- Fix the issue that canceling an `ADD INDEX` DDL task in the `none` status might cause memory leak because this task is not removed from the backend task queue [#44205](https://github.com/pingcap/tidb/issues/44205) @[tangenta](https://github.com/tangenta)
- Fix the issue that canceling an `ADD INDEX` DDL task in the `none` status might cause memory leak because this task is not removed from the Distributed eXecution Framework (DXF) task queue [#44205](https://github.com/pingcap/tidb/issues/44205) @[tangenta](https://github.com/tangenta)
- Fix the issue that the proxy protocol reports the `Header read timeout` error when processing certain erroneous data [#43205](https://github.com/pingcap/tidb/issues/43205) @[blacktear23](https://github.com/blacktear23)
- Fix the issue that PD isolation might block the running DDL [#44267](https://github.com/pingcap/tidb/issues/44267) @[wjhuang2016](https://github.com/wjhuang2016)
- Fix the issue that the query result of the `SELECT CAST(n AS CHAR)` statement is incorrect when `n` in the statement is a negative number [#44786](https://github.com/pingcap/tidb/issues/44786) @[xhebox](https://github.com/xhebox)
Expand Down
2 changes: 1 addition & 1 deletion releases/release-7.2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.2/quick-start-with-

The `IMPORT INTO` statement integrates the [Physical Import Mode](/tidb-lightning/tidb-lightning-physical-import-mode.md) capability of TiDB Lightning. With this statement, you can quickly import data in formats such as CSV, SQL, and PARQUET into an empty table in TiDB. This import method eliminates the need for a separate deployment and management of TiDB Lightning, thereby reducing the complexity of data import and greatly improving import efficiency.

For data files stored in Amazon S3 or GCS, when the [Backend task distributed execution framework](/tidb-distributed-execution-framework.md) is enabled, `IMPORT INTO` also supports splitting a data import job into multiple sub-jobs and scheduling them to multiple TiDB nodes for parallel import, which further enhances import performance.
For data files stored in Amazon S3 or GCS, when the [TiDB Distributed eXecution Framework (DXF)](/tidb-distributed-execution-framework.md) is enabled, `IMPORT INTO` also supports splitting a data import job into multiple sub-jobs and scheduling them to multiple TiDB nodes for parallel import, which further enhances import performance.

For more information, see [documentation](/sql-statements/sql-statement-import-into.md).

Expand Down
Loading

0 comments on commit 8286f4b

Please sign in to comment.