From 88ee4ee7e58aa81f29a63463f198b9f6186ce71a Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 15 Apr 2024 14:02:01 +0800 Subject: [PATCH 01/55] releases: add release notes for v8.1.0 --- releases/release-8.1.0.md | 136 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 136 insertions(+) create mode 100644 releases/release-8.1.0.md diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md new file mode 100644 index 0000000000000..0e0dbab178c39 --- /dev/null +++ b/releases/release-8.1.0.md @@ -0,0 +1,136 @@ +--- +title: TiDB 8.1.0 Release Notes +summary: Learn about the new features, compatibility changes, improvements, and bug fixes in TiDB 8.1.0. +--- + +# TiDB 8.1.0 Release Notes + +Release date: xx xx, 2024 + +TiDB version: 8.1.0 + +Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.1/quick-start-with-tidb) | [Production deployment](https://docs.pingcap.com/tidb/v8.1/production-deployment-using-tiup) | [Installation packages](https://www.pingcap.com/download/?version=v8.1.0#version-list) + +TiDB 8.1.0 is a Long-Term Support Release (LTS). + +Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, and bug fixes released in [7.6.0-DMR](/releases/release-7.6.0.md) and [8.0.0-DMR](/releases/release-8.0.0.md). When you upgrade from 7.5.x to 8.1.0, you can download the [TiDB Release Notes PDF](https://download.pingcap.org/tidb-v7.6-to-v8.1-en-release-notes.pdf) to view all release notes between the two LTS versions. The following table lists some highlights from 7.6.0 to 8.1.0: + +## Feature details + +### Scalability + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +### Performance + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +### Reliability + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +### Availability + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +### SQL + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +### DB operations + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +### Observability + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +### Security + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +### Data migration + +* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) + + Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + + For more information, see [documentation](doc-link). + +## Compatibility changes + +> **Note:** +> +> This section provides compatibility changes you need to know when you upgrade from v8.0.0 to the current version (v8.1.0). If you are upgrading from v7.6.0 or earlier versions to the current version, you might also need to check the compatibility changes introduced in intermediate versions. + +### Behavior changes + +### MySQL compatibility + +### System variables + +| Variable name | Change type | Description | +|--------|------------------------------|------| +| | | | +| | | | +| | | | +| | | | + +### Configuration file parameters + +| Configuration file | Configuration parameter | Change type | Description | +| -------- | -------- | -------- | -------- | +| | | | | +| | | | | +| | | | | +| | | | | + +### Others + +## Offline package changes + +## Deprecated features + +## Improvements + +## Bug fixes + +## Performance test + +To learn about the performance of TiDB v8.1.0, you can refer to the [TPC-C performance test report](https://docs.pingcap.com/tidbcloud/v8.1.0-performance-benchmarking-with-tpcc) and [Sysbench performance test report](https://docs.pingcap.com/tidbcloud/v8.1.0-performance-benchmarking-with-sysbench) of the TiDB Dedicated cluster. + +## Contributors + +We would like to thank the following contributors from the TiDB community: From ad52b2b741cd94f77005337f3fc93abd007848ec Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 22 Apr 2024 17:38:15 +0800 Subject: [PATCH 02/55] add highlights and new features --- releases/release-8.1.0.md | 115 +++++++++++++++++++++++++++++++++----- 1 file changed, 102 insertions(+), 13 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 0e0dbab178c39..e3441ded39622 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -15,6 +15,79 @@ TiDB 8.1.0 is a Long-Term Support Release (LTS). Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, and bug fixes released in [7.6.0-DMR](/releases/release-7.6.0.md) and [8.0.0-DMR](/releases/release-8.0.0.md). When you upgrade from 7.5.x to 8.1.0, you can download the [TiDB Release Notes PDF](https://download.pingcap.org/tidb-v7.6-to-v8.1-en-release-notes.pdf) to view all release notes between the two LTS versions. The following table lists some highlights from 7.6.0 to 8.1.0: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CategoryFeature/EnhancementDescription
Scalability and PerformanceAcceleration of cluster snapshot restore speed (GA in v8.0.0)With this feature, BR can fully leverage the scale advantage of a cluster, enabling all TiKV nodes in the cluster to participate in the preparation step of data restores. This feature can significantly improve the restore speed of large datasets in large-scale clusters. Real-world tests show that this feature can saturate the download bandwidth, with the download speed improving by 8 to 10 times, and the end-to-end restore speed improving by approximately 1.5 to 3 times.
Achieve up to 10 times faster for creating tables in batch (experimental, introduced in v7.6.0)With the implementation of the new DDL architecture in v7.6.0, the performance of batch table creation has witnessed a remarkable improvement, up to 10 times faster. This substantial enhancement drastically reduces the time needed for creating numerous tables. This acceleration is particularly noteworthy in SaaS scenarios, where the prevalence of high volumes of tables, ranging from tens to hundreds of thousands, is a common challenge.
Use Active PD Followers to enhance PD's Region information query service (experimental, introduced in v7.6.0)TiDB v7.6.0 introduces an experimental feature "Active PD Follower", which allows PD followers to provide Region information query services. This feature improves the capability of the PD cluster to handle GetRegion and ScanRegions requests in clusters with a large number of TiDB nodes and Regions, thereby reducing the CPU pressure on the PD leader.
Bulk DML for much larger transactions (experimental, introduced in v8.0.0)Large batch DML jobs, such as extensive cleanup jobs, joins, or aggregations, can consume a significant amount of memory and have previously been limited at very large scales. Bulk DML (tidb_dml_type = "bulk") is a new DML type for handling large batch DML tasks more efficiently while providing transaction guarantees and mitigating OOM issues. This feature differs from import, load, and restore operations when used for data loading.
Enhance the stability of caching the schema information when there is a massive number of tables (experimental, , introduced in v8.0.0)SaaS companies using TiDB as the system of record for their multi-tenant applications often need to store a substantial number of tables. In previous versions, handling table counts in the order of a million or more was feasible, but it had the potential to degrade the overall user experience. TiDB v8.0.0 improves the situation with the following enhancements: +
    +
  • Introduce a new schema information caching system, incorporating a lazy-loading Least Recently Used (LRU) cache for table metadata and more efficiently managing schema version changes.
    • +
  • Implement a priority queue for auto analyze, making the process less rigid and enhancing stability across a wider array of tables.
    • +
+
Reliability and availability 建表性能提升 10 倍（实验特性，从 v7.6.0 开始引入）全局排序功能旨在提高 IMPORT INTO 和 CREATE INDEX 的稳定性与效率。通过将任务需要处理的数据进行全局排序，可以提高数据写入 TiKV 的稳定性、可控性和可扩展性，从而提供更好的数据导入与 DDL 任务的用户体验及更高质量的服务。目前已经支持 40 TiB 的数据进行导入或者添加索引。
Cross-database SQL binding (introduced in v7.6.0)When managing hundreds of databases with the same schema, it is often necessary to apply SQL bindings across these databases. For example, in SaaS or PaaS data platforms, each user typically operates separate databases with the same schema and runs similar SQL queries on them. In this case, it is impractical to bind SQL for each database one by one. TiDB v7.6.0 introduces cross-database SQL bindings that enable matching bindings across all schema-equivalent databases.
Support TiProxy (GA in v8.0.0)Full support for the TiProxy service, easily deployable via deployment tooling, to manage and maintain connections to TiDB so that they live through rolling restarts, upgrades, or scaling events.
Data Migration (DM) officially supports MySQL 8.0 (GA in v7.6.0)Previously, using DM to migrate data from MySQL 8.0 is an experimental feature and is not available for production environments. TiDB v7.6.0 enhances the stability and compatibility of this feature to help you smoothly and quickly migrate data from MySQL 8.0 to TiDB in production environments. In v7.6.0, this feature becomes generally available (GA).
资源管控支持管理资源消耗超出预期的查询 (GA)通过资源组的规则，TiDB 能够自动识别出运行超出预期的查询，并对该查询进行限流或取消处理。 即使没有被规则识别，用户仍旧可以手工添加查询特征以及对应的手段措施，从而降低突发的查询性能对整个数据库的影响。
DB Operations and ObservabilitySupport monitoring index usage statistics (introduced in v8.0.0)Proper index design is a crucial prerequisite to maintaining database performance. TiDB v8.0.0 introduces the INFORMATION_SCHEMA.TIDB_INDEX_USAGE table and the sys.schema_unused_indexes view to provide usage statistics of indexes. This feature helps you assess the efficiency of indexes in the database and optimize the index design.
Data MigrationTiCDC adds support for the Simple protocol (introduced in v8.0.0)TiCDC introduces a new protocol, the Simple protocol. This protocol provides in-band schema tracking capabilities by embedding table schema information in DDL and BOOTSTRAP events.
TiCDC adds support for the Debezium format protocol(introduced in v8.0.0)TiCDC introduces a new protocol, the Debezium protocol. TiCDC can now publish data change events to a Kafka sink using a protocol that generates Debezium style messages.
+ ## Feature details ### Scalability @@ -35,11 +108,17 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### Reliability -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) +* 管理资源消耗超出预期的查询（GA）[#43691](https://github.com/pingcap/tidb/issues/43691) @[nolouch](https://github.com/nolouch) **tw@lilin90** - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) - - For more information, see [documentation](doc-link). + 突发的 SQL 性能问题引发数据库整体性能下降，是数据库稳定性最常见的挑战。造成 SQL 性能问题的原因有很多，例如未经充分测试的新 SQL、数据量剧烈变化、执行计划突变等，这些问题很难从源头上完全规避。在过去的版本中，TiDB 增加了对资源超出预期的查询的管理能力，用以快速降低 SQL 性能造成的影响范围，这个功能在 v8.1.0 成为正式功能。 + + 你可以针对某个资源组 (Resource Group) 设置查询的最长执行时间。当查询的执行时间超过设置值时，自动降低查询的优先级或者取消查询。你还可以设置在一段时间内通过文本或者执行计划立即匹配已经识别出的查询，从而避免问题查询的并发度太高时，在识别阶段就造成大量资源消耗的情况。 + + TiDB 同时支持手动标记查询的功能。利用命令 [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md)，你可以根据 SQL 的文本、SQL Digest 或执行计划标记查询，命中的查询可以被降级或取消，达到添加 SQL 黑名单的目的。 + + 对资源超出预期查询的自动管理能力，为用户提供了有效的手段，在根本原因被定位之前，该功能可以快速缓解其对整体性能的影响，从而提升数据库的稳定性。 + + 更多信息，请参考[用户文档](/tidb-resource-control.md#管理资源消耗超出预期的查询-runaway-queries)。 ### Availability @@ -51,11 +130,11 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### SQL -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) +* 支持在 TiDB 建表时使用更多的表达式设置列的默认值的功能成为正式功能（GA）[#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala)**tw@hfxsd** - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + 在 v8.0.0 之前，建表时指定列的默认值仅限于固定的字符串、数字和日期。从 v8.0.0 开始，TiDB 支持使用部分表达式作为列的默认值，例如将列的默认值设置为 UUID()，从而满足多样化的业务需求。同时在 8.1 版本，支持添加列时，也支持使用表达式作为列的默认值。 - For more information, see [documentation](doc-link). + 更多信息，请参考[用户文档](https://docs.pingcap.com/zh/tidb/dev/data-type-default-values#%E8%A1%A8%E8%BE%BE%E5%BC%8F%E9%BB%98%E8%AE%A4%E5%80%BC)。 ### DB operations @@ -75,19 +154,29 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### Security -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) +* 增强 TiDB 日志脱敏（GA） [#52364](https://github.com/pingcap/tidb/issues/52364) @[xhebox](https://github.com/xhebox) **tw@hfxsd** - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + TiDB 日志脱敏增强是通过对日志文件中的 SQL 文本信息进行标记，支持在查看时安全展示敏感数据。你可以控制是否对日志信息进行脱敏，以实现在不同场景下安全使用 TiDB 日志，提升了使用日志脱敏能力的安全性和灵活性。要使用此功能，可以将系统变量 `tidb_redact_log` 的值设置为 `MARKER`，此时 TiDB 的运行日志中的 SQL 文本会被标记，查看时将基于标记进行数据的安全展示，从而保护日志信息。这个功能在 v8.1.0 成为正式功能。 - For more information, see [documentation](doc-link). + 更多信息，请参考[用户文档](/system-variables.md#tidb_redact_log)。 ### Data migration -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) +* IMPORT INTO ... FROM SELECT 语法成为正式功能（GA），丰富了 IMPORT INTO 功能场景 [#49883](https://github.com/pingcap/tidb/issues/49883) @[D3Hunter](https://github.com/D3Hunter) **tw@qiancai** - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) + 在之前的 TiDB 版本中，将查询结果导入目标表只能通过 INSERT INTO ... SELECT 语句，但该语句在一些大数据量的场景中的导入效率较低。从 v8.0.0 开始，TiDB 新增支持通过 IMPORT INTO ... FROM SELECT 将 SELECT 的查询结果导入到一张空的 TiDB 目标表中，其性能最高可达 INSERT INTO ... SELECT 的 8 倍，可以大幅缩短导入所需的时间。 - For more information, see [documentation](doc-link). + 此外，你还可以通过 IMPORT INTO ... FROM SELECT 导入使用 [AS OF TIMESTAMP](https://docs.pingcap.com/zh/tidb/dev/as-of-timestamp) 查询的历史数据。 + + 更多信息，请参考[用户文档](https://docs.pingcap.com/zh/tidb/dev/sql-statement-import-into)。 + +* TiDB Lightning 简化冲突处理策略，同时支持以 replace 方式处理冲突数据的功能成为正式功能（GA）[#51036](https://github.com/pingcap/tidb/issues/51036) @[lyzx2001](https://github.com/lyzx2001)**tw@qiancai** + + 在之前的版本中，TiDB Lightning 逻辑导入模式有[一套数据冲突处理策略](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-logical-import-mode-usage#%E5%86%B2%E7%AA%81%E6%95%B0%E6%8D%AE%E6%A3%80%E6%B5%8B)，而物理导入模式有[两套数据冲突处理策略](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-physical-import-mode-usage#%E5%86%B2%E7%AA%81%E6%95%B0%E6%8D%AE%E6%A3%80%E6%B5%8B)，不易理解和配置。 + + 从 v8.0.0 开始，TiDB Lightning 废弃了物理导入模式下的[旧版冲突检测](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-physical-import-mode-usage#%E6%97%A7%E7%89%88%E5%86%B2%E7%AA%81%E6%A3%80%E6%B5%8B%E4%BB%8E-v800-%E5%BC%80%E5%A7%8B%E5%B7%B2%E8%A2%AB%E5%BA%9F%E5%BC%83)策略，支持通过 [conflict.strategy](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-configuration) 参数统一控制逻辑导入和物理导入模式的冲突检测策略，并简化了该参数的配置。此外，在物理导入模式下，当导入遇到主键或唯一键冲突的数据时，replace 策略支持保留最新的数据、覆盖旧的数据。 + + 更多信息，请参考[用户文档](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-configuration)。 ## Compatibility changes From 2b16653e1d54993ab3ecb0f74d63f9058a9a2820 Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 22 Apr 2024 18:14:50 +0800 Subject: [PATCH 03/55] fix build error and format --- releases/release-8.1.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index e3441ded39622..1add4f35dd6d5 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -26,7 +26,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Scalability and Performance - Acceleration of cluster snapshot restore speed (GA in v8.0.0) + Acceleration of cluster snapshot restore speed (GA in v8.0.0) With this feature, BR can fully leverage the scale advantage of a cluster, enabling all TiKV nodes in the cluster to participate in the preparation step of data restores. This feature can significantly improve the restore speed of large datasets in large-scale clusters. Real-world tests show that this feature can saturate the download bandwidth, with the download speed improving by 8 to 10 times, and the end-to-end restore speed improving by approximately 1.5 to 3 times. @@ -38,7 +38,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, TiDB v7.6.0 introduces an experimental feature "Active PD Follower", which allows PD followers to provide Region information query services. This feature improves the capability of the PD cluster to handle GetRegion and ScanRegions requests in clusters with a large number of TiDB nodes and Regions, thereby reducing the CPU pressure on the PD leader. - Bulk DML for much larger transactions (experimental, introduced in v8.0.0) + Bulk DML for much larger transactions (experimental, introduced in v8.0.0) Large batch DML jobs, such as extensive cleanup jobs, joins, or aggregations, can consume a significant amount of memory and have previously been limited at very large scales. Bulk DML (tidb_dml_type = "bulk") is a new DML type for handling large batch DML tasks more efficiently while providing transaction guarantees and mitigating OOM issues. This feature differs from import, load, and restore operations when used for data loading. @@ -52,7 +52,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Reliability and availability - 建表性能提升 10 倍（实验特性，从 v7.6.0 开始引入） + 全局排序成为正式功能（从 v8.0.0 开始 GA）**tw@qiancai** 全局排序功能旨在提高 IMPORT INTO 和 CREATE INDEX 的稳定性与效率。通过将任务需要处理的数据进行全局排序，可以提高数据写入 TiKV 的稳定性、可控性和可扩展性，从而提供更好的数据导入与 DDL 任务的用户体验及更高质量的服务。目前已经支持 40 TiB 的数据进行导入或者添加索引。 @@ -60,15 +60,15 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, When managing hundreds of databases with the same schema, it is often necessary to apply SQL bindings across these databases. For example, in SaaS or PaaS data platforms, each user typically operates separate databases with the same schema and runs similar SQL queries on them. In this case, it is impractical to bind SQL for each database one by one. TiDB v7.6.0 introduces cross-database SQL bindings that enable matching bindings across all schema-equivalent databases. - Support TiProxy (GA in v8.0.0) + Support TiProxy (GA in v8.0.0) Full support for the TiProxy service, easily deployable via deployment tooling, to manage and maintain connections to TiDB so that they live through rolling restarts, upgrades, or scaling events. - Data Migration (DM) officially supports MySQL 8.0 (GA in v7.6.0) + Data Migration (DM) officially supports MySQL 8.0 (GA in v7.6.0) Previously, using DM to migrate data from MySQL 8.0 is an experimental feature and is not available for production environments. TiDB v7.6.0 enhances the stability and compatibility of this feature to help you smoothly and quickly migrate data from MySQL 8.0 to TiDB in production environments. In v7.6.0, this feature becomes generally available (GA). - 资源管控支持管理资源消耗超出预期的查询 (GA) + 资源管控支持管理资源消耗超出预期的查询 (GA) **tw@lilin90** 通过资源组的规则，TiDB 能够自动识别出运行超出预期的查询，并对该查询进行限流或取消处理。 即使没有被规则识别，用户仍旧可以手工添加查询特征以及对应的手段措施，从而降低突发的查询性能对整个数据库的影响。 @@ -82,7 +82,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, TiCDC introduces a new protocol, the Simple protocol. This protocol provides in-band schema tracking capabilities by embedding table schema information in DDL and BOOTSTRAP events. - TiCDC adds support for the Debezium format protocol(introduced in v8.0.0) + TiCDC adds support for the Debezium format protocol (introduced in v8.0.0) TiCDC introduces a new protocol, the Debezium protocol. TiCDC can now publish data change events to a Kafka sink using a protocol that generates Debezium style messages. From 69769c437f03b92583284b3269dbae681a77d8bd Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 24 Apr 2024 11:30:59 +0800 Subject: [PATCH 04/55] Apply suggestions from code review --- releases/release-8.1.0.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 1add4f35dd6d5..f9eefdeac9826 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -130,11 +130,13 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### SQL -* 支持在 TiDB 建表时使用更多的表达式设置列的默认值的功能成为正式功能（GA）[#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala)**tw@hfxsd** +* Support using some expressions to set default column values when creating a table (GA) [#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala) - 在 v8.0.0 之前，建表时指定列的默认值仅限于固定的字符串、数字和日期。从 v8.0.0 开始，TiDB 支持使用部分表达式作为列的默认值，例如将列的默认值设置为 UUID()，从而满足多样化的业务需求。同时在 8.1 版本，支持添加列时，也支持使用表达式作为列的默认值。 + Before v8.0.0, when you create a table, the default value of a column is limited to strings, numbers, dates, and certain expressions. Starting from v8.0.0, you can use more expressions as the default column values. For example, you can set the default value of a column to `DATE_FORMAT`. This feature helps you meet more diverse requirements. In v8.1.0, this feature becomes GA. + + Starting from v8.1.0, you can use expressions as default values when adding columns by `ADD COLUMN`. - 更多信息，请参考[用户文档](https://docs.pingcap.com/zh/tidb/dev/data-type-default-values#%E8%A1%A8%E8%BE%BE%E5%BC%8F%E9%BB%98%E8%AE%A4%E5%80%BC)。 + For more information, see [documentation](/data-type-default-values.md#specify-expressions-as-default-values). ### DB operations @@ -154,11 +156,11 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### Security -* 增强 TiDB 日志脱敏（GA） [#52364](https://github.com/pingcap/tidb/issues/52364) @[xhebox](https://github.com/xhebox) **tw@hfxsd** +* Enhance TiDB log desensitization (GA) [#52364](https://github.com/pingcap/tidb/issues/52364) @[xhebox](https://github.com/xhebox) **tw@hfxsd** - TiDB 日志脱敏增强是通过对日志文件中的 SQL 文本信息进行标记，支持在查看时安全展示敏感数据。你可以控制是否对日志信息进行脱敏，以实现在不同场景下安全使用 TiDB 日志，提升了使用日志脱敏能力的安全性和灵活性。要使用此功能，可以将系统变量 `tidb_redact_log` 的值设置为 `MARKER`，此时 TiDB 的运行日志中的 SQL 文本会被标记，查看时将基于标记进行数据的安全展示，从而保护日志信息。这个功能在 v8.1.0 成为正式功能。 + The enhancement of TiDB log desensitization is based on marking SQL text information in log files, facilitating deleting of sensitive data when users view the logs. You can control whether to mark log information to enable secure use of TiDB logs in different scenarios, enhancing the security and flexibility of using log desensitization. To use this feature, set the system variable `tidb_redact_log` to `MARKER`. This marks the SQL text in TiDB logs. In addition, you can use the `collect-log` subcommand on the TiDB server to remove marked sensitive data from the logs and display the logs in a data-safe manner. You can also remove all markers and get the normal logs. This feature became generally available in v8.1.0. - 更多信息，请参考[用户文档](/system-variables.md#tidb_redact_log)。 + For more information, see [documentation](/system-variables.md#tidb_redact_log). ### Data migration From 4cb7dce4d9b0a3e8693ec653a76585c55a23b0f4 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 24 Apr 2024 16:39:24 +0800 Subject: [PATCH 05/55] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index f9eefdeac9826..4d49574afe7b6 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -9,7 +9,7 @@ Release date: xx xx, 2024 TiDB version: 8.1.0 -Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.1/quick-start-with-tidb) | [Production deployment](https://docs.pingcap.com/tidb/v8.1/production-deployment-using-tiup) | [Installation packages](https://www.pingcap.com/download/?version=v8.1.0#version-list) +Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.1/quick-start-with-tidb) | [Production deployment](https://docs.pingcap.com/tidb/v8.1/production-deployment-using-tiup) TiDB 8.1.0 is a Long-Term Support Release (LTS). From d06b70b9e5a938f80e2c01036fc15590bc1aabe3 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Thu, 25 Apr 2024 11:55:37 +0800 Subject: [PATCH 06/55] Fix punctuation --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 4d49574afe7b6..896f89e1c7805 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -42,7 +42,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Large batch DML jobs, such as extensive cleanup jobs, joins, or aggregations, can consume a significant amount of memory and have previously been limited at very large scales. Bulk DML (tidb_dml_type = "bulk") is a new DML type for handling large batch DML tasks more efficiently while providing transaction guarantees and mitigating OOM issues. This feature differs from import, load, and restore operations when used for data loading. - Enhance the stability of caching the schema information when there is a massive number of tables (experimental, , introduced in v8.0.0) + Enhance the stability of caching the schema information when there is a massive number of tables (experimental, introduced in v8.0.0) SaaS companies using TiDB as the system of record for their multi-tenant applications often need to store a substantial number of tables. In previous versions, handling table counts in the order of a million or more was feasible, but it had the potential to degrade the overall user experience. TiDB v8.0.0 improves the situation with the following enhancements:
  • Introduce a new schema information caching system, incorporating a lazy-loading Least Recently Used (LRU) cache for table metadata and more efficiently managing schema version changes.
    • From db3b67f845049eb54a1e4d4ba394694293cca2d3 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Thu, 25 Apr 2024 14:51:07 +0800 Subject: [PATCH 07/55] Add translation for managing queries that consume more resources than expected --- releases/release-8.1.0.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 896f89e1c7805..87392bf791f74 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -68,8 +68,8 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Previously, using DM to migrate data from MySQL 8.0 is an experimental feature and is not available for production environments. TiDB v7.6.0 enhances the stability and compatibility of this feature to help you smoothly and quickly migrate data from MySQL 8.0 to TiDB in production environments. In v7.6.0, this feature becomes generally available (GA). - 资源管控支持管理资源消耗超出预期的查询 (GA) **tw@lilin90** - 通过资源组的规则，TiDB 能够自动识别出运行超出预期的查询，并对该查询进行限流或取消处理。 即使没有被规则识别，用户仍旧可以手工添加查询特征以及对应的手段措施，从而降低突发的查询性能对整个数据库的影响。 + TiDB resource control supports managing queries that consume more resources than expected (GA) + Through the rules of resource groups, TiDB can automatically identify queries that consume more resources than expected, and then limit or cancel these queries. Even if the queries are not identified by the rules, you can still manually add query characteristics and take corresponding measures to reduce the sudden query performance problem's impact on the entire database. DB Operations and Observability @@ -108,17 +108,17 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### Reliability -* 管理资源消耗超出预期的查询（GA）[#43691](https://github.com/pingcap/tidb/issues/43691) @[nolouch](https://github.com/nolouch) **tw@lilin90** - - 突发的 SQL 性能问题引发数据库整体性能下降，是数据库稳定性最常见的挑战。造成 SQL 性能问题的原因有很多，例如未经充分测试的新 SQL、数据量剧烈变化、执行计划突变等，这些问题很难从源头上完全规避。在过去的版本中，TiDB 增加了对资源超出预期的查询的管理能力，用以快速降低 SQL 性能造成的影响范围，这个功能在 v8.1.0 成为正式功能。 - - 你可以针对某个资源组 (Resource Group) 设置查询的最长执行时间。当查询的执行时间超过设置值时，自动降低查询的优先级或者取消查询。你还可以设置在一段时间内通过文本或者执行计划立即匹配已经识别出的查询，从而避免问题查询的并发度太高时，在识别阶段就造成大量资源消耗的情况。 - - TiDB 同时支持手动标记查询的功能。利用命令 [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md)，你可以根据 SQL 的文本、SQL Digest 或执行计划标记查询，命中的查询可以被降级或取消，达到添加 SQL 黑名单的目的。 - - 对资源超出预期查询的自动管理能力，为用户提供了有效的手段，在根本原因被定位之前，该功能可以快速缓解其对整体性能的影响，从而提升数据库的稳定性。 - - 更多信息，请参考[用户文档](/tidb-resource-control.md#管理资源消耗超出预期的查询-runaway-queries)。 +* Support managing queries that consume more resources than expected (GA) [#43691](https://github.com/pingcap/tidb/issues/43691) @[nolouch](https://github.com/nolouch) **tw@lilin90** + + Sudden SQL query performance problems can cause a decline in overall database performance, which is the most common challenge to database stability. The reasons for these problems are diverse, such as untested new SQL statements, drastic changes in data volume, and sudden changes in execution plans. These problems are difficult to avoid completely at the source. In earlier versions, TiDB has added the capability to manage queries that consume more resources than expected to quickly reduce the impact of sudden query performance problems. This feature becomes generally available in v8.1.0. + + You can set the maximum execution time for a query in a resource group. When the execution time of a query exceeds the set value, the priority of the query is automatically reduced or the query is canceled. You can also set immediately matching identified queries through text or execution plans within a period of time, to avoid excessive resource consumption during the identification phase when the concurrency of problematic queries is too high. + + TiDB also supports the manual marking of queries. By using the [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) command, you can mark queries based on the SQL text, SQL Digest, or execution plan. The queries that match the mark can be downgraded or canceled, achieving the purpose of adding a SQL blocklist. + + The automatic management capability of queries that consume more resources than expected provides users with an effective means to quickly mitigate the impact of query problems on overall performance before the root cause is identified, thereby improving the stability of the database. + + For more information, see [documentation](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). ### Availability From 33f4b669f1c207cb3073127a47cb6cd3917b373a Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Thu, 25 Apr 2024 14:59:11 +0800 Subject: [PATCH 08/55] Add tw mark --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 87392bf791f74..d21a27395982b 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -68,7 +68,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Previously, using DM to migrate data from MySQL 8.0 is an experimental feature and is not available for production environments. TiDB v7.6.0 enhances the stability and compatibility of this feature to help you smoothly and quickly migrate data from MySQL 8.0 to TiDB in production environments. In v7.6.0, this feature becomes generally available (GA). - TiDB resource control supports managing queries that consume more resources than expected (GA) + TiDB resource control supports managing queries that consume more resources than expected (GA) **tw@lilin90** Through the rules of resource groups, TiDB can automatically identify queries that consume more resources than expected, and then limit or cancel these queries. Even if the queries are not identified by the rules, you can still manually add query characteristics and take corresponding measures to reduce the sudden query performance problem's impact on the entire database. From 7d2208337e2dcdd3889da10a432fbb24de9379a6 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 25 Apr 2024 15:55:32 +0800 Subject: [PATCH 09/55] add translation for two features --- releases/release-8.1.0.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index d21a27395982b..d15ecbc4836e1 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -51,9 +51,10 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Reliability and availability - 全局排序成为正式功能（从 v8.0.0 开始 GA）**tw@qiancai** - 全局排序功能旨在提高 IMPORT INTO 和 CREATE INDEX 的稳定性与效率。通过将任务需要处理的数据进行全局排序，可以提高数据写入 TiKV 的稳定性、可控性和可扩展性，从而提供更好的数据导入与 DDL 任务的用户体验及更高质量的服务。目前已经支持 40 TiB 的数据进行导入或者添加索引。 + Reliability and availability + Global Sort (GA in v8.0.0) **tw@qiancai** + The Global Sort feature aims to improve the stability and efficiency of `IMPORT INTO` and `CREATE INDEX`. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, 40 TiB of data for importing into or creating indexes is now supported. + Cross-database SQL binding (introduced in v7.6.0) @@ -164,21 +165,21 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### Data migration -* IMPORT INTO ... FROM SELECT 语法成为正式功能（GA），丰富了 IMPORT INTO 功能场景 [#49883](https://github.com/pingcap/tidb/issues/49883) @[D3Hunter](https://github.com/D3Hunter) **tw@qiancai** +* Support the `IMPORT INTO ... FROM SELECT` syntax (GA) [#49883](https://github.com/pingcap/tidb/issues/49883) @[D3Hunter](https://github.com/D3Hunter) **tw@qiancai** - 在之前的 TiDB 版本中，将查询结果导入目标表只能通过 INSERT INTO ... SELECT 语句，但该语句在一些大数据量的场景中的导入效率较低。从 v8.0.0 开始，TiDB 新增支持通过 IMPORT INTO ... FROM SELECT 将 SELECT 的查询结果导入到一张空的 TiDB 目标表中，其性能最高可达 INSERT INTO ... SELECT 的 8 倍，可以大幅缩短导入所需的时间。 + Before v8.0.0, importing query results into a target table could only be done using the `INSERT INTO ... SELECT` statement, which is relatively inefficient in some large dataset scenarios. In v8.0.0, TiDB introduces `IMPORT INTO ... FROM SELECT` as an experimental feature, which enables you to import the results of a `SELECT` query into an empty TiDB target table. It achieves up to 8 times the performance of `INSERT INTO ... SELECT` and significantly reduces the import time. In addition, you can use `IMPORT INTO ... FROM SELECT` to import historical data queried with [`AS OF TIMESTAMP`](/as-of-timestamp.md). - 此外，你还可以通过 IMPORT INTO ... FROM SELECT 导入使用 [AS OF TIMESTAMP](https://docs.pingcap.com/zh/tidb/dev/as-of-timestamp) 查询的历史数据。 + In v8.1.0, the `IMPORT INTO ... FROM SELECT` syntax becomes generally available (GA), enriching the functionality scenarios of the `IMPORT INTO` statement. - 更多信息，请参考[用户文档](https://docs.pingcap.com/zh/tidb/dev/sql-statement-import-into)。 + For more information, see [documentation](/sql-statements/sql-statement-import-into.md). -* TiDB Lightning 简化冲突处理策略，同时支持以 replace 方式处理冲突数据的功能成为正式功能（GA）[#51036](https://github.com/pingcap/tidb/issues/51036) @[lyzx2001](https://github.com/lyzx2001)**tw@qiancai** +* TiDB Lightning simplifies conflict resolution strategies and supports handling conflicting data using the `replace` strategy (GA) [#51036](https://github.com/pingcap/tidb/issues/51036) @[lyzx2001](https://github.com/lyzx2001) **tw@qiancai** - 在之前的版本中，TiDB Lightning 逻辑导入模式有[一套数据冲突处理策略](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-logical-import-mode-usage#%E5%86%B2%E7%AA%81%E6%95%B0%E6%8D%AE%E6%A3%80%E6%B5%8B)，而物理导入模式有[两套数据冲突处理策略](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-physical-import-mode-usage#%E5%86%B2%E7%AA%81%E6%95%B0%E6%8D%AE%E6%A3%80%E6%B5%8B)，不易理解和配置。 + Before v8.0.0, TiDB Lightning has [one data conflict resolution strategy](/tidb-lightning/tidb-lightning-logical-import-mode-usage.md#conflict-detection) for the logical import mode and [two data conflict resolution strategies](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#conflict-detection) for the physical import mode, which are not easy to understand and configure. - 从 v8.0.0 开始，TiDB Lightning 废弃了物理导入模式下的[旧版冲突检测](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-physical-import-mode-usage#%E6%97%A7%E7%89%88%E5%86%B2%E7%AA%81%E6%A3%80%E6%B5%8B%E4%BB%8E-v800-%E5%BC%80%E5%A7%8B%E5%B7%B2%E8%A2%AB%E5%BA%9F%E5%BC%83)策略，支持通过 [conflict.strategy](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-configuration) 参数统一控制逻辑导入和物理导入模式的冲突检测策略，并简化了该参数的配置。此外，在物理导入模式下，当导入遇到主键或唯一键冲突的数据时，replace 策略支持保留最新的数据、覆盖旧的数据。 + In v8.0.0, TiDB Lightning deprecates the [old version of conflict detection](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#the-old-version-of-conflict-detection-deprecated-in-v800) strategy for the physical import mode, enables you to control the conflict detection strategy for both logical and physical import modes via the [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md) parameter (experimental), and simplifies the configuration of this parameter. In addition, in the physical import mode, the `replace` strategy supports retaining the latest data and overwriting the old data when the import detects data with primary key or unique key conflicts. In v8.1.0, the capability to handle conflicting data with the `replace` strategy becomes generally available (GA). - 更多信息，请参考[用户文档](https://docs.pingcap.com/zh/tidb/dev/tidb-lightning-configuration)。 + For more information, see [documentation](/tidb-lightning/tidb-lightning-configuration.md). ## Compatibility changes From 07370f0677915782f7a736f0913416f822703027 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 26 Apr 2024 09:20:00 +0800 Subject: [PATCH 10/55] fix build failure --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index d15ecbc4836e1..857f431fbd1e6 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -53,7 +53,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Reliability and availability Global Sort (GA in v8.0.0) **tw@qiancai** - The Global Sort feature aims to improve the stability and efficiency of `IMPORT INTO` and `CREATE INDEX`. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, 40 TiB of data for importing into or creating indexes is now supported. + The Global Sort feature aims to improve the stability and efficiency of `IMPORT INTO` and `CREATE INDEX`. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, 40 TiB of data for importing into or creating indexes is now supported. From 402104d21e3d83cf3c0815ce686bcfcaa03f2f53 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Fri, 26 Apr 2024 16:41:53 +0800 Subject: [PATCH 11/55] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 857f431fbd1e6..2bf32acff9c85 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -70,7 +70,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, TiDB resource control supports managing queries that consume more resources than expected (GA) **tw@lilin90** - Through the rules of resource groups, TiDB can automatically identify queries that consume more resources than expected, and then limit or cancel these queries. Even if the queries are not identified by the rules, you can still manually add query characteristics and take corresponding measures to reduce the sudden query performance problem's impact on the entire database. + Through the rules of resource groups, TiDB can automatically identify queries that consume more resources than expected, and then limit or cancel these queries. Even if the queries are not identified by the rules, you can still manually add query characteristics and take corresponding measures to reduce the impact of the sudden query performance problem on the entire database. DB Operations and Observability From b87cea632b56f02dfd382d8e0c9f0036a50f31a5 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 26 Apr 2024 16:59:33 +0800 Subject: [PATCH 12/55] Apply suggestions from code review Co-authored-by: Lilian Lee --- releases/release-8.1.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 2bf32acff9c85..af93f191a36dd 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -131,7 +131,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### SQL -* Support using some expressions to set default column values when creating a table (GA) [#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala) +* Support using more expressions to set default column values when creating a table (GA) [#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala) Before v8.0.0, when you create a table, the default value of a column is limited to strings, numbers, dates, and certain expressions. Starting from v8.0.0, you can use more expressions as the default column values. For example, you can set the default value of a column to `DATE_FORMAT`. This feature helps you meet more diverse requirements. In v8.1.0, this feature becomes GA. @@ -159,7 +159,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, * Enhance TiDB log desensitization (GA) [#52364](https://github.com/pingcap/tidb/issues/52364) @[xhebox](https://github.com/xhebox) **tw@hfxsd** - The enhancement of TiDB log desensitization is based on marking SQL text information in log files, facilitating deleting of sensitive data when users view the logs. You can control whether to mark log information to enable secure use of TiDB logs in different scenarios, enhancing the security and flexibility of using log desensitization. To use this feature, set the system variable `tidb_redact_log` to `MARKER`. This marks the SQL text in TiDB logs. In addition, you can use the `collect-log` subcommand on the TiDB server to remove marked sensitive data from the logs and display the logs in a data-safe manner. You can also remove all markers and get the normal logs. This feature became generally available in v8.1.0. + The enhanced TiDB log desensitization supports removing sensitive data when users view logs, implemented by marking SQL text information in log files. You can control whether to mark log information to enable secure use of TiDB logs in different scenarios, enhancing the security and flexibility of using log desensitization. To use this feature, set the system variable `tidb_redact_log` to `MARKER`, and then the SQL text in TiDB's runtime logs are marked. In addition, you can use the `collect-log` subcommand on the TiDB server to remove marked sensitive data from the logs and display the logs in a secure manner. You can also remove all markers and get the normal logs. This feature became generally available in v8.1.0. For more information, see [documentation](/system-variables.md#tidb_redact_log). From 0753cfb5a689f778ea4d77c6c6b8f65edd17f270 Mon Sep 17 00:00:00 2001 From: Ben Meadowcroft Date: Mon, 29 Apr 2024 09:52:43 -0700 Subject: [PATCH 13/55] Update release-8.1.0.md Added TiCDC related release notes for client authentication --- releases/release-8.1.0.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index af93f191a36dd..17c1e92c6a3c5 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -78,7 +78,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Proper index design is a crucial prerequisite to maintaining database performance. TiDB v8.0.0 introduces the INFORMATION_SCHEMA.TIDB_INDEX_USAGE table and the sys.schema_unused_indexes view to provide usage statistics of indexes. This feature helps you assess the efficiency of indexes in the database and optimize the index design. - Data Migration + Data Migration TiCDC adds support for the Simple protocol (introduced in v8.0.0) TiCDC introduces a new protocol, the Simple protocol. This protocol provides in-band schema tracking capabilities by embedding table schema information in DDL and BOOTSTRAP events. @@ -86,6 +86,10 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, TiCDC adds support for the Debezium format protocol (introduced in v8.0.0) TiCDC introduces a new protocol, the Debezium protocol. TiCDC can now publish data change events to a Kafka sink using a protocol that generates Debezium style messages. + + TiCDC adds new Client authentication capabilities + TiCDC introduces support for both certificate based client authentication, and authentication using TiDB credentials. Theses features enable CLI or API clients to authenticate their connections to TiCDC. + From 7b82e3c2c0bfc3508ebfea80be162e8a5dbb79eb Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 30 Apr 2024 15:20:08 +0800 Subject: [PATCH 14/55] highlight: update TiCDC client authentication --- releases/release-8.1.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 17c1e92c6a3c5..2809ee7173e8f 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -87,8 +87,8 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, TiCDC introduces a new protocol, the Debezium protocol. TiCDC can now publish data change events to a Kafka sink using a protocol that generates Debezium style messages. - TiCDC adds new Client authentication capabilities - TiCDC introduces support for both certificate based client authentication, and authentication using TiDB credentials. Theses features enable CLI or API clients to authenticate their connections to TiCDC. + TiCDC adds support for client authentication (introduced in v8.1.0) **tw@Oreoxmt** + TiCDC supports client authentication using Mutual Transport Layer Security (mTLS) or TiDB username and password. This feature enable CLI or OpenAPI clients to authenticate their connections to TiCDC. From 719b606f314362c38a74ef09fe57683c8d9126ca Mon Sep 17 00:00:00 2001 From: Ben Meadowcroft Date: Tue, 30 Apr 2024 12:20:04 -0700 Subject: [PATCH 15/55] Update release-8.1.0.md Added feature description for TiCDC Client authentication --- releases/release-8.1.0.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 2809ee7173e8f..ef329555dbde8 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -185,6 +185,12 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, For more information, see [documentation](/tidb-lightning/tidb-lightning-configuration.md). +* TiCDC adds support for client authentication [#10636](https://github.com/pingcap/tiflow/issues/10636) @[CharlesCheung96](https://github.com/CharlesCheung96) + + In v8.1.0. TiCDC adds support for configuring the TiCDC service to require client authentication when using the TiCDC CLI or API. This provides the option for you to configure the use of client certificates with TiCDC in order to establish mutual TLS. This improvement also provides the option for configuring the use of username and password based authentication. + + For more information, see [documentation](/ticdc/ticdc-client-authentication.md). + ## Compatibility changes > **Note:** From b163516a84a88c9943c4f41fc417a4b4c8fc245e Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 6 May 2024 10:29:10 +0800 Subject: [PATCH 16/55] Update releases/release-8.1.0.md --- releases/release-8.1.0.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index ef329555dbde8..c1d1e108ebf22 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -150,6 +150,13 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) For more information, see [documentation](doc-link). +* Enable the TiDB Distributed eXecution Framework (DXF) by default to enhance the performance and stability of `ADD INDEX` or `IMPORT INTO` tasks in parallel execution [#52441](https://github.com/pingcap/tidb/issues/52441) @[D3Hunter](https://github.com/D3Hunter) **tw@qiancai** + + The DXF becomes generally available (GA) in v7.5.0, but it is disabled by default. This means that an `ADD INDEX` or `IMPORT INTO` task is executed only by one TiDB node by default. + + Starting from v8.1.0, TiDB enables this feature by default ([`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-introduced-since-v710) defaults to `ON`). When enabled, the distributed execution framework can schedule multiple TiDB nodes to execute the same `ADD INDEX` or `IMPORT INTO` task in parallel, fully utilizing the resources of the TiDB cluster and greatly improving the performance of these tasks. In addition, you can linearly improve the performance of `ADD INDEX` and `IMPORT INTO` tasks by adding TiDB nodes and configuring [`tidb_service_scope`](/system-variables.md#tidb_service_scope-introduced-since-v740) for the newly added nodes. + + For more information, see [documentation](/tidb-distributed-execution-framework.md). ### Observability From 3e62c4108ca9b2ec797158980d6f9f1c4306855f Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 6 May 2024 10:29:59 +0800 Subject: [PATCH 17/55] format udpates --- releases/release-8.1.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index c1d1e108ebf22..1a414915af879 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -150,6 +150,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) For more information, see [documentation](doc-link). + * Enable the TiDB Distributed eXecution Framework (DXF) by default to enhance the performance and stability of `ADD INDEX` or `IMPORT INTO` tasks in parallel execution [#52441](https://github.com/pingcap/tidb/issues/52441) @[D3Hunter](https://github.com/D3Hunter) **tw@qiancai** The DXF becomes generally available (GA) in v7.5.0, but it is disabled by default. This means that an `ADD INDEX` or `IMPORT INTO` task is executed only by one TiDB node by default. From b438b70a0eaac45bed89fa4bd4a9bd493c431f3c Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 6 May 2024 10:32:49 +0800 Subject: [PATCH 18/55] fix broken links --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 1a414915af879..13ba5a82c9111 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -155,7 +155,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, The DXF becomes generally available (GA) in v7.5.0, but it is disabled by default. This means that an `ADD INDEX` or `IMPORT INTO` task is executed only by one TiDB node by default. - Starting from v8.1.0, TiDB enables this feature by default ([`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-introduced-since-v710) defaults to `ON`). When enabled, the distributed execution framework can schedule multiple TiDB nodes to execute the same `ADD INDEX` or `IMPORT INTO` task in parallel, fully utilizing the resources of the TiDB cluster and greatly improving the performance of these tasks. In addition, you can linearly improve the performance of `ADD INDEX` and `IMPORT INTO` tasks by adding TiDB nodes and configuring [`tidb_service_scope`](/system-variables.md#tidb_service_scope-introduced-since-v740) for the newly added nodes. + Starting from v8.1.0, TiDB enables this feature by default ([`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) defaults to `ON`). When enabled, the distributed execution framework can schedule multiple TiDB nodes to execute the same `ADD INDEX` or `IMPORT INTO` task in parallel, fully utilizing the resources of the TiDB cluster and greatly improving the performance of these tasks. In addition, you can linearly improve the performance of `ADD INDEX` and `IMPORT INTO` tasks by adding TiDB nodes and configuring [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) for the newly added nodes. For more information, see [documentation](/tidb-distributed-execution-framework.md). From 71a2b745c1129a3477b415e07be3ef1f843f7a17 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 6 May 2024 10:33:59 +0800 Subject: [PATCH 19/55] minor wording updates --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 13ba5a82c9111..3d7ae88df7096 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -155,7 +155,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, The DXF becomes generally available (GA) in v7.5.0, but it is disabled by default. This means that an `ADD INDEX` or `IMPORT INTO` task is executed only by one TiDB node by default. - Starting from v8.1.0, TiDB enables this feature by default ([`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) defaults to `ON`). When enabled, the distributed execution framework can schedule multiple TiDB nodes to execute the same `ADD INDEX` or `IMPORT INTO` task in parallel, fully utilizing the resources of the TiDB cluster and greatly improving the performance of these tasks. In addition, you can linearly improve the performance of `ADD INDEX` and `IMPORT INTO` tasks by adding TiDB nodes and configuring [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) for the newly added nodes. + Starting from v8.1.0, TiDB enables this feature by default ([`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) defaults to `ON`). When enabled, the DXF can schedule multiple TiDB nodes to execute the same `ADD INDEX` or `IMPORT INTO` task in parallel, fully utilizing the resources of the TiDB cluster and greatly improving the performance of these tasks. In addition, you can linearly improve the performance of `ADD INDEX` and `IMPORT INTO` tasks by adding TiDB nodes and configuring [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) for the newly added nodes. For more information, see [documentation](/tidb-distributed-execution-framework.md). From 4e41f23036e4bb07691d39d29e101b7f1e7f8b94 Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 6 May 2024 11:24:57 +0800 Subject: [PATCH 20/55] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-8.1.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 3d7ae88df7096..4486861d8faa5 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -88,7 +88,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, TiCDC adds support for client authentication (introduced in v8.1.0) **tw@Oreoxmt** - TiCDC supports client authentication using Mutual Transport Layer Security (mTLS) or TiDB username and password. This feature enable CLI or OpenAPI clients to authenticate their connections to TiCDC. + TiCDC supports client authentication using mutual Transport Layer Security (mTLS) or TiDB username and password. This feature enables CLI or OpenAPI clients to authenticate their connections to TiCDC. @@ -193,10 +193,10 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, For more information, see [documentation](/tidb-lightning/tidb-lightning-configuration.md). -* TiCDC adds support for client authentication [#10636](https://github.com/pingcap/tiflow/issues/10636) @[CharlesCheung96](https://github.com/CharlesCheung96) +* TiCDC adds support for client authentication [#10636](https://github.com/pingcap/tiflow/issues/10636) @[CharlesCheung96](https://github.com/CharlesCheung96) **tw@Oreoxmt** + + In v8.1.0, TiCDC supports client authentication when using the TiCDC CLI or OpenAPI. This feature enables you to configure TiCDC to require client authentication using client certificates, thereby establishing mutual Transport Layer Security (mTLS). Additionally, you can configure authentication based on TiDB username and password. - In v8.1.0. TiCDC adds support for configuring the TiCDC service to require client authentication when using the TiCDC CLI or API. This provides the option for you to configure the use of client certificates with TiCDC in order to establish mutual TLS. This improvement also provides the option for configuring the use of username and password based authentication. - For more information, see [documentation](/ticdc/ticdc-client-authentication.md). ## Compatibility changes From a7348dc8651035fa77cdc501451eb8b2fe5c775e Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 6 May 2024 13:11:30 +0800 Subject: [PATCH 21/55] Update releases/release-8.1.0.md --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 4486861d8faa5..7b02487180c4e 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -53,7 +53,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Reliability and availability Global Sort (GA in v8.0.0) **tw@qiancai** - The Global Sort feature aims to improve the stability and efficiency of `IMPORT INTO` and `CREATE INDEX`. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, 40 TiB of data for importing into or creating indexes is now supported. + The Global Sort feature aims to improve the stability and efficiency of `IMPORT INTO` and `CREATE INDEX`. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, each `IMPORT INTO` or `CREATE INDEX` statement now supports importing or adding indexes for up to 40 TiB of data. From ec0682ec8e2677d83e263289b79d01cba3d0985f Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 6 May 2024 16:14:25 +0800 Subject: [PATCH 22/55] Update releases/release-8.1.0.md Co-authored-by: Aolin --- releases/release-8.1.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 7b02487180c4e..bc52849e25839 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -52,8 +52,8 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Reliability and availability - Global Sort (GA in v8.0.0) **tw@qiancai** - The Global Sort feature aims to improve the stability and efficiency of `IMPORT INTO` and `CREATE INDEX`. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, each `IMPORT INTO` or `CREATE INDEX` statement now supports importing or adding indexes for up to 40 TiB of data. + Global Sort (GA in v8.0.0) **tw@qiancai** + The Global Sort feature aims to improve the stability and efficiency of IMPORT INTO and CREATE INDEX. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, each IMPORT INTO or CREATE INDEX statement now supports importing or adding indexes for up to 40 TiB of data. From 22e32601328f4098950b77b5f01042caeb080ba0 Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 6 May 2024 16:15:38 +0800 Subject: [PATCH 23/55] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-8.1.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index bc52849e25839..4c40fb79f8a11 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -79,15 +79,15 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Data Migration - TiCDC adds support for the Simple protocol (introduced in v8.0.0) + TiCDC supports the Simple protocol (introduced in v8.0.0) TiCDC introduces a new protocol, the Simple protocol. This protocol provides in-band schema tracking capabilities by embedding table schema information in DDL and BOOTSTRAP events. - TiCDC adds support for the Debezium format protocol (introduced in v8.0.0) + TiCDC supports the Debezium format protocol (introduced in v8.0.0) TiCDC introduces a new protocol, the Debezium protocol. TiCDC can now publish data change events to a Kafka sink using a protocol that generates Debezium style messages. - TiCDC adds support for client authentication (introduced in v8.1.0) **tw@Oreoxmt** + TiCDC supports client authentication (introduced in v8.1.0) **tw@Oreoxmt** TiCDC supports client authentication using mutual Transport Layer Security (mTLS) or TiDB username and password. This feature enables CLI or OpenAPI clients to authenticate their connections to TiCDC. @@ -193,7 +193,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, For more information, see [documentation](/tidb-lightning/tidb-lightning-configuration.md). -* TiCDC adds support for client authentication [#10636](https://github.com/pingcap/tiflow/issues/10636) @[CharlesCheung96](https://github.com/CharlesCheung96) **tw@Oreoxmt** +* TiCDC supports client authentication [#10636](https://github.com/pingcap/tiflow/issues/10636) @[CharlesCheung96](https://github.com/CharlesCheung96) **tw@Oreoxmt** In v8.1.0, TiCDC supports client authentication when using the TiCDC CLI or OpenAPI. This feature enables you to configure TiCDC to require client authentication using client certificates, thereby establishing mutual Transport Layer Security (mTLS). Additionally, you can configure authentication based on TiDB username and password. From 31abc4a2ca17fac0bb9249a75599615f32d6cbbe Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 6 May 2024 16:32:26 +0800 Subject: [PATCH 24/55] add deprecation info for duplicate-resolution --- releases/release-8.1.0.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 4c40fb79f8a11..d959aadf35feb 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -233,6 +233,8 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ## Deprecated features +* Starting from v8.0.0, TiDB Lightning deprecates the [old version of conflict detection](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#the-old-version-of-conflict-detection-deprecated-in-v800) strategy for the physical import mode, and enables you to control the conflict detection strategy for both logical and physical import modes via the [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md) parameter. The [`duplicate-resolution`](/tidb-lightning/tidb-lightning-configuration.md) parameter for the old version of conflict detection will be removed in a future release. + ## Improvements ## Bug fixes From 7f1554141d764199370d78a02c432f2d557b1260 Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 6 May 2024 17:05:11 +0800 Subject: [PATCH 25/55] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index d959aadf35feb..e5b41e5f6bb91 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -195,7 +195,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, * TiCDC supports client authentication [#10636](https://github.com/pingcap/tiflow/issues/10636) @[CharlesCheung96](https://github.com/CharlesCheung96) **tw@Oreoxmt** - In v8.1.0, TiCDC supports client authentication when using the TiCDC CLI or OpenAPI. This feature enables you to configure TiCDC to require client authentication using client certificates, thereby establishing mutual Transport Layer Security (mTLS). Additionally, you can configure authentication based on TiDB username and password. + In v8.1.0, TiCDC supports client authentication when you are using the TiCDC CLI or OpenAPI. This feature enables you to configure TiCDC to require client authentication using client certificates, thereby establishing mutual Transport Layer Security (mTLS). Additionally, you can configure authentication based on TiDB username and password. For more information, see [documentation](/ticdc/ticdc-client-authentication.md). From 6f597b16d45a7c238138e4a45d39d70579346b48 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 6 May 2024 17:54:39 +0800 Subject: [PATCH 26/55] add compatibility changes --- releases/release-8.1.0.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index e5b41e5f6bb91..7613ea786d0dc 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -214,7 +214,8 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, | Variable name | Change type | Description | |--------|------------------------------|------| | | | | -| | | | +| [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) | Modified | Changes the default value from `OFF` to `ON`. This means that Distributed eXecution Framework (DXF) is enabled by default, which fully utilizes the resources of the TiDB cluster and greatly improves the performance of `ADD INDEX` and `IMPORT INTO` tasks. If your cluster was upgraded from an earlier version, the variable retains its value before the upgrade. | +| [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Modified | Changes the optional value from "" or `background` to a string with a length of up to 64 characters, which enables you to control the service scope of each TiDB node more flexibly. Available legal characters include digits `0-9`, letters `a-zA-Z`, underscore `_`, and hyphen `-`. The Distributed eXecution Framework (DXF) determines which TiDB nodes can be scheduled to execute distributed tasks based on the value of this variable. For specific rules, see [Task scheduling](/tidb-distributed-execution-framework.md#task-scheduling). | | | | | | | | | @@ -222,7 +223,8 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, | Configuration file | Configuration parameter | Change type | Description | | -------- | -------- | -------- | -------- | -| | | | | +| TiDB Lightning | [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) | Modified | Changes the default value from `9223372036854775807` to `10000` to quickly interrupt abnormal tasks so that you can make corresponding adjustments as soon as possible. This saves time and computational resources by avoiding the scenario where a large amount of conflicting data is discovered after the import, caused by abnormal data sources or incorrect table schema definitions. | +| TiDB Lightning | [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-configuration) | Modified | Starting from v8.1.0, there is no need to configure `conflict.max-record-rows` manually, because TiDB Lightning automatically assigns the value of `conflict.max-record-rows` with the value of `conflict.threshold`, regardless of the user input. `conflict.max-record-rows` will be deprecated in a future release. | | | | | | | | | | | | | | | | From f9fb895795a2df523d35758e5cc6a17d9ee0b0fe Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 May 2024 11:40:06 +0800 Subject: [PATCH 27/55] Apply suggestions from code review --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 7613ea786d0dc..67a354c364ddb 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -214,7 +214,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, | Variable name | Change type | Description | |--------|------------------------------|------| | | | | -| [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) | Modified | Changes the default value from `OFF` to `ON`. This means that Distributed eXecution Framework (DXF) is enabled by default, which fully utilizes the resources of the TiDB cluster and greatly improves the performance of `ADD INDEX` and `IMPORT INTO` tasks. If your cluster was upgraded from an earlier version, the variable retains its value before the upgrade. | +| [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) | Modified | Changes the default value from `OFF` to `ON`. This means that Distributed eXecution Framework (DXF) is enabled by default, which fully utilizes the resources of the TiDB cluster and greatly improves the performance of `ADD INDEX` and `IMPORT INTO` tasks. If you want to upgrade a cluster with the DXF enabled to v8.1.0 or later, disable the DXF (by setting `tidb_enable_dist_task` to `OFF`) before the upgrade, which avoids `ADD INDEX` operations during the upgrade causing data index inconsistency. After the upgrade, you can manually enable the DXF. | | [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Modified | Changes the optional value from "" or `background` to a string with a length of up to 64 characters, which enables you to control the service scope of each TiDB node more flexibly. Available legal characters include digits `0-9`, letters `a-zA-Z`, underscore `_`, and hyphen `-`. The Distributed eXecution Framework (DXF) determines which TiDB nodes can be scheduled to execute distributed tasks based on the value of this variable. For specific rules, see [Task scheduling](/tidb-distributed-execution-framework.md#task-scheduling). | | | | | | | | | From 8005223a527741a8c59dbd006a827a03880db427 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 8 May 2024 11:22:57 +0800 Subject: [PATCH 28/55] update the description for tidb_service_scope --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 67a354c364ddb..873dab8f7d8eb 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -215,7 +215,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, |--------|------------------------------|------| | | | | | [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) | Modified | Changes the default value from `OFF` to `ON`. This means that Distributed eXecution Framework (DXF) is enabled by default, which fully utilizes the resources of the TiDB cluster and greatly improves the performance of `ADD INDEX` and `IMPORT INTO` tasks. If you want to upgrade a cluster with the DXF enabled to v8.1.0 or later, disable the DXF (by setting `tidb_enable_dist_task` to `OFF`) before the upgrade, which avoids `ADD INDEX` operations during the upgrade causing data index inconsistency. After the upgrade, you can manually enable the DXF. | -| [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Modified | Changes the optional value from "" or `background` to a string with a length of up to 64 characters, which enables you to control the service scope of each TiDB node more flexibly. Available legal characters include digits `0-9`, letters `a-zA-Z`, underscore `_`, and hyphen `-`. The Distributed eXecution Framework (DXF) determines which TiDB nodes can be scheduled to execute distributed tasks based on the value of this variable. For specific rules, see [Task scheduling](/tidb-distributed-execution-framework.md#task-scheduling). | +| [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Modified | Changes the optional value from "" or `background` to a string with a length of up to 64 characters, which enables you to control the service scope of each TiDB node more flexibly. Valid characters include digits `0-9`, letters `a-zA-Z`, underscores `_`, and hyphens `-`. The Distributed eXecution Framework (DXF) determines which TiDB nodes can be scheduled to execute distributed tasks based on the value of this variable. For specific rules, see [Task scheduling](/tidb-distributed-execution-framework.md#task-scheduling). | | | | | | | | | From 82da5f5035b823bd2a4660ab3c32bf9c281ff8d4 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 8 May 2024 15:32:05 +0800 Subject: [PATCH 29/55] fix format and links, remove release notes template, add deprecated features,improvements, bug fixes, and contributors --- releases/release-8.1.0.md | 250 +++++++++++++++++++++++++++++++------- 1 file changed, 203 insertions(+), 47 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 873dab8f7d8eb..9e94e0abc2f73 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -26,26 +26,26 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Scalability and Performance - Acceleration of cluster snapshot restore speed (GA in v8.0.0) + Acceleration of cluster snapshot restore speed (GA in v8.0.0) With this feature, BR can fully leverage the scale advantage of a cluster, enabling all TiKV nodes in the cluster to participate in the preparation step of data restores. This feature can significantly improve the restore speed of large datasets in large-scale clusters. Real-world tests show that this feature can saturate the download bandwidth, with the download speed improving by 8 to 10 times, and the end-to-end restore speed improving by approximately 1.5 to 3 times. - Achieve up to 10 times faster for creating tables in batch (experimental, introduced in v7.6.0) + Achieve up to 10 times faster for creating tables in batch (experimental, introduced in v7.6.0) With the implementation of the new DDL architecture in v7.6.0, the performance of batch table creation has witnessed a remarkable improvement, up to 10 times faster. This substantial enhancement drastically reduces the time needed for creating numerous tables. This acceleration is particularly noteworthy in SaaS scenarios, where the prevalence of high volumes of tables, ranging from tens to hundreds of thousands, is a common challenge. - Use Active PD Followers to enhance PD's Region information query service (experimental, introduced in v7.6.0) + Use Active PD Followers to enhance PD's Region information query service (experimental, introduced in v7.6.0) TiDB v7.6.0 introduces an experimental feature "Active PD Follower", which allows PD followers to provide Region information query services. This feature improves the capability of the PD cluster to handle GetRegion and ScanRegions requests in clusters with a large number of TiDB nodes and Regions, thereby reducing the CPU pressure on the PD leader. - Bulk DML for much larger transactions (experimental, introduced in v8.0.0) + Bulk DML for much larger transactions (experimental, introduced in v8.0.0) Large batch DML jobs, such as extensive cleanup jobs, joins, or aggregations, can consume a significant amount of memory and have previously been limited at very large scales. Bulk DML (tidb_dml_type = "bulk") is a new DML type for handling large batch DML tasks more efficiently while providing transaction guarantees and mitigating OOM issues. This feature differs from import, load, and restore operations when used for data loading. Enhance the stability of caching the schema information when there is a massive number of tables (experimental, introduced in v8.0.0) SaaS companies using TiDB as the system of record for their multi-tenant applications often need to store a substantial number of tables. In previous versions, handling table counts in the order of a million or more was feasible, but it had the potential to degrade the overall user experience. TiDB v8.0.0 improves the situation with the following enhancements:
      -
    • Introduce a new schema information caching system, incorporating a lazy-loading Least Recently Used (LRU) cache for table metadata and more efficiently managing schema version changes.
      • +
    • Introduce a new schema information caching system, which provides a cache strategy based on the Least Recently Used (LRU) algorithm for table metadata. It prioritizes storing the metadata of frequently accessed tables in the cache, thereby reducing memory usage in scenarios with a large number of tables.
    • Implement a priority queue for auto analyze, making the process less rigid and enhancing stability across a wider array of tables.
    @@ -57,15 +57,15 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Cross-database SQL binding (introduced in v7.6.0) + Cross-database SQL binding (introduced in v7.6.0) When managing hundreds of databases with the same schema, it is often necessary to apply SQL bindings across these databases. For example, in SaaS or PaaS data platforms, each user typically operates separate databases with the same schema and runs similar SQL queries on them. In this case, it is impractical to bind SQL for each database one by one. TiDB v7.6.0 introduces cross-database SQL bindings that enable matching bindings across all schema-equivalent databases. - Support TiProxy (GA in v8.0.0) + Support TiProxy (GA in v8.0.0) Full support for the TiProxy service, easily deployable via deployment tooling, to manage and maintain connections to TiDB so that they live through rolling restarts, upgrades, or scaling events. - Data Migration (DM) officially supports MySQL 8.0 (GA in v7.6.0) + Data Migration (DM) officially supports MySQL 8.0 (GA in v7.6.0) Previously, using DM to migrate data from MySQL 8.0 is an experimental feature and is not available for production environments. TiDB v7.6.0 enhances the stability and compatibility of this feature to help you smoothly and quickly migrate data from MySQL 8.0 to TiDB in production environments. In v7.6.0, this feature becomes generally available (GA). @@ -95,22 +95,6 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ## Feature details -### Scalability - -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) - - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) - - For more information, see [documentation](doc-link). - -### Performance - -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) - - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) - - For more information, see [documentation](doc-link). - ### Reliability * Support managing queries that consume more resources than expected (GA) [#43691](https://github.com/pingcap/tidb/issues/43691) @[nolouch](https://github.com/nolouch) **tw@lilin90** @@ -124,18 +108,11 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, The automatic management capability of queries that consume more resources than expected provides users with an effective means to quickly mitigate the impact of query problems on overall performance before the root cause is identified, thereby improving the stability of the database. For more information, see [documentation](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). - -### Availability - -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) - - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) - - For more information, see [documentation](doc-link). +tion, see [documentation](doc-link). ### SQL -* Support using more expressions to set default column values when creating a table (GA) [#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala) +* Support using more expressions to set default column values when creating a table (GA) [#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala) **tw@hfxsd** Before v8.0.0, when you create a table, the default value of a column is limited to strings, numbers, dates, and certain expressions. Starting from v8.0.0, you can use more expressions as the default column values. For example, you can set the default value of a column to `DATE_FORMAT`. This feature helps you meet more diverse requirements. In v8.1.0, this feature becomes GA. @@ -145,12 +122,6 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### DB operations -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) - - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) - - For more information, see [documentation](doc-link). - * Enable the TiDB Distributed eXecution Framework (DXF) by default to enhance the performance and stability of `ADD INDEX` or `IMPORT INTO` tasks in parallel execution [#52441](https://github.com/pingcap/tidb/issues/52441) @[D3Hunter](https://github.com/D3Hunter) **tw@qiancai** The DXF becomes generally available (GA) in v7.5.0, but it is disabled by default. This means that an `ADD INDEX` or `IMPORT INTO` task is executed only by one TiDB node by default. @@ -159,14 +130,6 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, For more information, see [documentation](/tidb-distributed-execution-framework.md). -### Observability - -* Feature summary [#issue-number](issue-link) @[pr-auorthor-id](author-link) - - Feature descriptions (including what the feature is, why it is valuable for users, and how to use this feature generally) - - For more information, see [documentation](doc-link). - ### Security * Enhance TiDB log desensitization (GA) [#52364](https://github.com/pingcap/tidb/issues/52364) @[xhebox](https://github.com/xhebox) **tw@hfxsd** @@ -235,12 +198,195 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ## Deprecated features +* It is planned to redesign [the auto-evolution of execution plan bindings](/sql-plan-management.md#baseline-evolution) in subsequent releases, and the related variables and behavior will change. +* The TiDB Lightning parameter `conflict.max-record-rows` is scheduled for deprecation in a future release and will be subsequently removed. This parameter will be replaced by `conflict.threshold`, which means that the maximum number of conflicting records is consistent with the maximum number of conflicting records that can be tolerated in a single import task. * Starting from v8.0.0, TiDB Lightning deprecates the [old version of conflict detection](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#the-old-version-of-conflict-detection-deprecated-in-v800) strategy for the physical import mode, and enables you to control the conflict detection strategy for both logical and physical import modes via the [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md) parameter. The [`duplicate-resolution`](/tidb-lightning/tidb-lightning-configuration.md) parameter for the old version of conflict detection will be removed in a future release. ## Improvements ++ TiDB + + - 提升了外键在 SHOW CREATE TABLE 结果中显示的 MySQL 兼容性 [#51837](https://github.com/pingcap/tidb/issues/51837) @[negachov](https://github.com/negachov) + - 提升了表达式默认值在 SHOW CREATE TABLE 结果中显示的 MySQL 兼容性 [#52939](https://github.com/pingcap/tidb/issues/52939) @[CbcWestwolf](https://github.com/CbcWestwolf) + - 允许使用 ingest 模式并发添加多个索引 [#52596](https://github.com/pingcap/tidb/issues/52596) @[lance6716](https://github.com/lance6716) + - 允许将系统变量 `tidb_service_scope` 设置为不同的值，从而更好地利用分布式框架功能 [#52441](https://github.com/pingcap/tidb/issues/52441) @[ywqzzy](https://github.com/ywqzzy) + - 增强 TiDB 对总是 false 的 DNF 的处理，直接忽略这种过滤条件 [#40997](https://github.com/pingcap/tidb/issues/40997) @[hi-rustin](https://github.com/hi-rustin) + - 不允许将 tidb_auto_analyze_ratio 参数设置为 0 [#51582](https://github.com/pingcap/tidb/issues/51582) @[hi-rustin](https://github.com/hi-rustin) + - 如果查询有除了全表扫描以外的单索引扫描方式可以选择，优化器不会自动选择索引合并。现在可以通过 Optimizer Fix Controls 机制解除这个限制 [#52869](https://github.com/pingcap/tidb/issues/52869) @[time-and-fate](https://github.com/time-and-fate) + - (dup): release-7.1.5.md > Improvements> TiDB - Add a timeout mechanism for LDAP authentication to avoid the issue of resource lock (RLock) not being released in time [#51883](https://github.com/pingcap/tidb/issues/51883) @[YangKeao](https://github.com/YangKeao) + ++ TiKV + + - (dup): release-6.5.9.md > Improvements> TiKV - Avoid performing IO operations on snapshot files in raftstore threads to improve TiKV stability [#16564](https://github.com/tikv/tikv/issues/16564) @[Connor1996](https://github.com/Connor1996) + ++ PD + + - 优化调度 operator 相关逻辑竞争锁的开销 [#7897](https://github.com/tikv/pd/issues/7897) @[nolouch](https://github.com/nolouch) + ++ TiFlash + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ Tools + + + Backup & Restore (BR) + + - (dup): release-7.1.5.md > Improvements> Tools> Backup & Restore (BR) - Add PITR integration test cases to cover compatibility testing for log backup and adding index acceleration [#51987](https://github.com/pingcap/tidb/issues/51987) @[Leavrth](https://github.com/Leavrth) + - (dup): release-7.1.5.md > Improvements> Tools> Backup & Restore (BR) - Remove the invalid verification for active DDL jobs when log backup starts [#52733](https://github.com/pingcap/tidb/issues/52733) @[Leavrth](https://github.com/Leavrth) + - 增加测试用例测试 PITR 和添加索引加速兼容性 [#51988](https://github.com/pingcap/tidb/issues/51988) @[Leavrth](https://github.com/Leavrth) + - BR 恢复过程中清理空 sst 文件 [#16005](https://github.com/tikv/tikv/issues/16005) @[Leavrth](https://github.com/Leavrth) + + + TiCDC + + - 提升了使用 redo log 恢复数据过程中的内存稳定性，减少了 OOM 的概率 [#10900](https://github.com/pingcap/tiflow/issues/10900) @[CharlesCheung96](https://github.com/CharlesCheung96) + - 显著提升了事务冲突场景的数据同步的稳定性，性能最高提升 10 倍 [#10896](https://github.com/pingcap/tiflow/issues/10896) @[CharlesCheung96](https://github.com/CharlesCheung96) + + + TiDB Data Migration (DM) + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiDB Lightning + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + Dumpling + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiUP + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiDB Binlog + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ## Bug fixes ++ TiDB + + - 修复当 SQL 涉及包含多值索引的表时，执行可能报错“Can't find a proper physical plan for this query”的问题 [#49438](https://github.com/pingcap/tidb/issues/49438) @[qw4990](https://github.com/qw4990) + - 修复自动分析在 OOM 后卡住的问题 [#51993](https://github.com/pingcap/tidb/issues/51993) @[hi-rustin](https://github.com/hi-rustin) + - 修复使用 BR 恢复一张表之后，即使这张表没有统计信息，统计信息健康度也显示 100 的问题 [#29769](https://github.com/pingcap/tidb/issues/29769) @[winoros](https://github.com/winoros) + - 修复了 TiDB 在升级过程中会为系统表创建统计信息的问题 [#52040](https://github.com/pingcap/tidb/issues/52040) @[hi-rustin](https://github.com/hi-rustin) + - 修复 TiDB 在统计信息初始化完成前就进行自动分析的问题 [#52346](https://github.com/pingcap/tidb/issues/52346) @[hi-rustin](https://github.com/hi-rustin) + - 修复启用 `tidb_mem_quota_analyze` 时，自动分析使用的内存超过限制可能导致 TiDB crash 的问题 [#52601](https://github.com/pingcap/tidb/issues/52601) @[hawkingrei](https://github.com/hawkingrei) + - 修复 TiDB 统计信息同步加载机制无限重试加载空统计信息并打印“fail to get stats version for this histogram”日志的问题 [#52657](https://github.com/pingcap/tidb/issues/52657) @[hawkingrei](https://github.com/hawkingrei) + - 修复在关闭新排序规则框架的时候，涉及不同排序规则的表达式可能导致查询 panic 的问题 [#52772](https://github.com/pingcap/tidb/issues/52772) @[wjhuang2016](https://github.com/wjhuang2016) + - 修复 `CPS by type` 监控项显示错误的问题 [#52605](https://github.com/pingcap/tidb/issues/52605) @[nolouch](https://github.com/nolouch) + - 修复查询 `INFORMATION_SCHEMA.TIKV_REGION_STATUS` 出现空指针的问题 [#52013](https://github.com/pingcap/tidb/issues/52013) @[JmPotato](https://github.com/JmPotato) + - 修正了指定非法列默认值时的错误提示信息 [#51592](https://github.com/pingcap/tidb/issues/51592) @[danqixu](https://github.com/danqixu) + - 修复了 ingest 模式添加索引时，在边角案例下导致数据索引不一致的问题 [#51954](https://github.com/pingcap/tidb/issues/51954) @[lance6716](https://github.com/lance6716) + + - 修复了恢复含有外键的表时，DDL 卡住的问题 [#51838](https://github.com/pingcap/tidb/issues/51838) @[YangKeao](https://github.com/YangKeao) + - 修复了加索引期间 TiDB 网络隔离导致加索引失败的问题 [#51846](https://github.com/pingcap/tidb/issues/51846) @[ywqzzy](https://github.com/ywqzzy) + - 修复了重命名索引后再添加同名索引时的报错问题 [#51431](https://github.com/pingcap/tidb/issues/51431) @[lance6716](https://github.com/lance6716) + - 修复了添加索引期间升级集群导致的数据索引不一致问题 [#52411](https://github.com/pingcap/tidb/issues/52411) @[tangenta](https://github.com/tangenta) + - 修复了开启分布式框架后为大数据量表添加索引不成功的问题 [#52640](https://github.com/pingcap/tidb/issues/52640) @[tangenta](https://github.com/tangenta) + - 修复了并发添加索引时报 “文件不存在” 错误的问题 [#52475](https://github.com/pingcap/tidb/issues/52475) @[tangenta](https://github.com/tangenta) + - 修复了添加索引失败后清理不掉临时数据的问题 [#52639](https://github.com/pingcap/tidb/issues/52639) @[lance6716](https://github.com/lance6716) + - 修复了元数据锁在 plan cache 场景下未能阻止 DDL 推进的问题 [#51407](https://github.com/pingcap/tidb/issues/51407) @[wjhuang2016](https://github.com/wjhuang2016) + - 修复了执行 IMPORT INTO 大数据量任务时卡住的问题 [#52884](https://github.com/pingcap/tidb/issues/52884) @[lance6716](https://github.com/lance6716) + - 修复了记录 GRPC 错误时 TIDB 意外重启的问题 [#52179](https://github.com/pingcap/tidb/issues/52179) @[guo-shaoge](https://github.com/guo-shaoge) + - 修复了 IndexHashJoin 在执行 Anti Left Outer Semi Join 计算时输出冗余数据的问题 [#52923](https://github.com/pingcap/tidb/issues/52923) @[yibin87](https://github.com/yibin87) + - 修复了 HashJoin Probe 执行时间统计不精确的问题 [#52227](https://github.com/pingcap/tidb/issues/52227) @[windtalker](https://github.com/windtalker) + - 修复了在分区裁剪模式为 static 情况下，使用 tablesample 返回错误结果的问题 [#52282](https://github.com/pingcap/tidb/issues/52282) @[tangenta](https://github.com/tangenta) + - 修复 TTL 功能在非默认时区下的错误 [#51675](https://github.com/pingcap/tidb/issues/51675) @[lcwangchao](https://github.com/lcwangchao) + - 修复监控面板中链接数的计算和显示错误 [#51889](https://github.com/pingcap/tidb/issues/51889) @[YangKeao](https://github.com/YangKeao) + - 修复回滚改写分区 DDL 任务时，状态卡住的问题 [#51090](https://github.com/pingcap/tidb/issues/51090) @[lcwangchao](https://github.com/lcwangchao) + - 修复 lighting 导入后 TiDB 意外重启 [#52827](https://github.com/pingcap/tidb/issues/52827) @[hawkingrei](https://github.com/hawkingrei) + - 修复 TiDB `explain analyze` 结果中 `max_remote_stream` 显示不正确的问题 [#52646](https://github.com/pingcap/tidb/issues/52646) @[JaySon-Huang](https://github.com/JaySon-Huang) + - (dup): release-6.5.9.md > Bug fixes> TiDB - Fix the issue that querying the `TIDB_HOT_REGIONS` table might incorrectly return `INFORMATION_SCHEMA` tables [#50810](https://github.com/pingcap/tidb/issues/50810) @[Defined2014](https://github.com/Defined2014) + - (dup): release-7.1.5.md > Bug fixes> TiDB - Fix the issue that the `EXPLAIN` statement might display incorrect column IDs in the result when statistics for certain columns are not fully loaded [#52207](https://github.com/pingcap/tidb/issues/52207) @[time-and-fate](https://github.com/time-and-fate) + - (dup): release-6.5.9.md > Bug fixes> TiDB - Fix the issue that the type returned by the `IFNULL` function is inconsistent with MySQL [#51765](https://github.com/pingcap/tidb/issues/51765) @[YangKeao](https://github.com/YangKeao) + - (dup): release-7.1.5.md > Bug fixes> TiDB - Fix the issue that adding a unique index might cause TiDB to panic [#52312](https://github.com/pingcap/tidb/issues/52312) @[wjhuang2016](https://github.com/wjhuang2016) + ++ TiKV + + - (dup): release-7.1.5.md > Bug fixes> TiKV - Fix the issue that resolve-ts is blocked when a stale Region peer ignores the GC message [#16504](https://github.com/tikv/tikv/issues/16504) @[crazycs520](https://github.com/crazycs520) + - (dup): release-6.5.9.md > Bug fixes> TiKV - Fix the issue that inactive Write Ahead Logs (WALs) in RocksDB might corrupt data [#16705](https://github.com/tikv/tikv/issues/16705) @[Connor1996](https://github.com/Connor1996) + ++ PD + + - 修复切换 PD 模式时 TSO 可能卡住的问题 [#7849](https://github.com/tikv/pd/issues/7849) @[JmPotato](https://github.com/JmPotato) + - 修复 DR Auto Sync 监控状态显示的问题 [#7974](https://github.com/tikv/pd/issues/7974) @[lhy1024](https://github.com/lhy1024) + - 修复检查 binary 版本时可能导致的 panic 问题 [#7978](https://github.com/tikv/pd/issues/7978) @[JmPotato](https://github.com/JmPotato) + - 修复 TTL 参数类型转换错误的问题 [#7980](https://github.com/tikv/pd/issues/7980) @[HuSharp](https://github.com/HuSharp) + - 修复两中心部署切换时 leader 无法迁移的问题 [#7992](https://github.com/tikv/pd/issues/7992) @[TonsnakeLin](https://github.com/TonsnakeLin) + - 修复 pd-ctl 标准错误重定向的问题 [#8022](https://github.com/tikv/pd/issues/8022) @[HuSharp](https://github.com/HuSharp) + - 修复在生成 merge 调度时可能出现的 panic 问题 [#8049](https://github.com/tikv/pd/issues/8049) @[nolouch](https://github.com/nolouch) + - 修复 `GetAdditionalInfo` 导致的 panic 问题 [#8079](https://github.com/tikv/pd/issues/8079) @[HuSharp](https://github.com/HuSharp) + - 修复 scatter range 调度器监控显示的问题 [#8125](https://github.com/tikv/pd/issues/8125) @[HuSharp](https://github.com/HuSharp) + - (dup): release-6.5.9.md > Bug fixes> PD - Fix the issue that the query result of `SHOW CONFIG` includes the deprecated configuration item `trace-region-flow` [#7917](https://github.com/tikv/pd/issues/7917) @[rleungx](https://github.com/rleungx) + - (dup): release-6.5.9.md > Bug fixes> PD - Fix the issue that the scaling progress is not correctly displayed [#7726](https://github.com/tikv/pd/issues/7726) @[CabinfeverB](https://github.com/CabinfeverB) + ++ TiFlash + + - (dup): release-6.5.9.md > Bug fixes> TiFlash - Fix the issue that TiFlash might panic when you insert data to columns with invalid default values in non-strict `sql_mode` [#8803](https://github.com/pingcap/tiflash/issues/8803) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + - 修复 TiFlash 在高并发读的情况下，可能返回瞬时不正确的结果的问题 [#8845](https://github.com/pingcap/tiflash/issues/8845) @[JaySon-Huang](https://github.com/JaySon-Huang) + - 修复存算分离架构下，计算节点的本地缓存容量配置项的值修改后，上报 PD 的硬盘使用量不正确的问题 [#8920](https://github.com/pingcap/tiflash/issues/8920) @[JinheLin](https://github.com/JinheLin) + ++ Tools + + + Backup & Restore (BR) + + - (dup): release-6.5.9.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that BR could not back up the `AUTO_RANDOM` ID allocation progress in a union clustered index that contains an `AUTO_RANDOM` column [#52255](https://github.com/pingcap/tidb/issues/52255) @[Leavrth](https://github.com/Leavrth) + - (dup): release-6.5.9.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that removing a log backup task after it is paused does not immediately restore the GC safepoint [#52082](https://github.com/pingcap/tidb/issues/52082) @[3pointer](https://github.com/3pointer) + - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix a rare issue that special event timing might cause the data loss in log backup [#16739](https://github.com/tikv/tikv/issues/16739) @[YuJuncen](https://github.com/YuJuncen) + - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that the global checkpoint of log backup is advanced ahead of the actual backup file write point due to TiKV restart, which might cause a small amount of backup data loss [#16809](https://github.com/tikv/tikv/issues/16809) @[YuJuncen](https://github.com/YuJuncen) + - 去除全量备份日志输出无效的 concurrency 属性 [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) + - 修复在 PD leader 丢失情况下, BR 恢复或者是 lightning 物理模式下失败的问题 [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) + - 修复日志备份在暂停, 停止,再重建任务操作后, 任务状态显示正常, 但 checkpoint 不推进的问题 [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) + - 修复不稳定测试用例 `TestClearCache` [#51671](https://github.com/pingcap/tidb/issues/51671) @[zxc111](https://github.com/zxc111) + - 修复不稳定测试用例 `TestGetMergeRegionSizeAndCount` [#52095](https://github.com/pingcap/tidb/issues/52095) @[3pointer](https://github.com/3pointer) + - 修复不稳定集成测试 `br_tikv_outage` [#52673](https://github.com/pingcap/tidb/issues/52673) @[Leavrth](https://github.com/Leavrth) + - 控制测试用例 `TestGetTSWithRetry` 执行时间 [#52547](https://github.com/pingcap/tidb/issues/52547) @[Leavrth](https://github.com/Leavrth) + + + TiCDC + + - 修复调用驱逐 TiCDC owner 节点（/api/v2/owner/resign）的命令后意外导致 CDC 任务重启的问题 [#10781](https://github.com/pingcap/tiflow/issues/10781) @[sdojjy](https://github.com/sdojjy) + - 修复当下游 pulsar 下线后移除 changefeed 会导致 CDC 正常流程卡住从而引起其他 changefeed 进度卡住的问题 [#10629](https://github.com/pingcap/tiflow/issues/10629) @[asddongmen](https://github.com/asddongmen) + - 修复 grafana 监控中 Owner 面板显示不稳定的问题 [#10796](https://github.com/pingcap/tiflow/issues/10796) @[hongyunyan](https://github.com/hongyunyan) + - 修复重启 PD 可能导致 TiCDC 节点报错重启的问题 [#10799](https://github.com/pingcap/tiflow/issues/10799) @[3AceShowHand](https://github.com/3AceShowHand) + - 修复当 PD 磁盘 IO 延迟较大会导致显著的 CDC 同步延迟的问题 [#9054](https://github.com/pingcap/tiflow/issues/9054) @[asddongmen](https://github.com/asddongmen) + - 修复没有按照正确的时区设置默认值的问题 [#10931](https://github.com/pingcap/tiflow/issues/10931) @[3AceShowHand](https://github.com/3AceShowHand) + - 修复 drop primary/unique key 没有没正确同步的问题 [#10890](https://github.com/pingcap/tiflow/issues/10890) @[asddongmen](https://github.com/asddongmen) + - (dup): release-7.1.5.md > Bug fixes> Tools> TiCDC - Fix the issue that TiCDC fails to execute the `Exchange Partition ... With Validation` DDL downstream after it is written upstream, causing the changefeed to get stuck [#10859](https://github.com/pingcap/tiflow/issues/10859) @[hongyunyan](https://github.com/hongyunyan) + + + TiDB Data Migration (DM) + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiDB Lightning + + - 修复了 lightning 导入时，因源文件存在不兼容的 SQL 而报 “no database selected” 的问题 [#51800](https://github.com/pingcap/tidb/issues/51800) @[lance6716](https://github.com/lance6716) + - 修复了 lightning 在 server 模式下可能会打印敏感信息到日志的问题 [#36374](https://github.com/pingcap/tidb/issues/36374) @[kennytm](https://github.com/kennytm) + - 修复了 lightning 导入期间报 “invalid store ID 0” 错误的问题 [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) + - 修复了 lighting 使用 replace 作为去重算法时报 “Unknown column in where clause” 错误的问题 [#52886](https://github.com/pingcap/tidb/issues/52886) @[lyzx2001](https://github.com/lyzx2001) + - (dup): release-7.1.5.md > Bug fixes> Tools> TiDB Lightning - Fix the issue that TiDB Lightning panics when importing an empty table of Parquet format [#52518](https://github.com/pingcap/tidb/issues/52518) @[kennytm](https://github.com/kennytm) + + + Dumpling + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiUP + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiDB Binlog + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ## Performance test To learn about the performance of TiDB v8.1.0, you can refer to the [TPC-C performance test report](https://docs.pingcap.com/tidbcloud/v8.1.0-performance-benchmarking-with-tpcc) and [Sysbench performance test report](https://docs.pingcap.com/tidbcloud/v8.1.0-performance-benchmarking-with-sysbench) of the TiDB Dedicated cluster. @@ -248,3 +394,13 @@ To learn about the performance of TiDB v8.1.0, you can refer to the [TPC-C perfo ## Contributors We would like to thank the following contributors from the TiDB community: + +- [arturmelanchyk](https://github.com/arturmelanchyk) (First-time contributor) +- [CabinfeverB](https://github.com/CabinfeverB) +- [danqixu](https://github.com/danqixu) (First-time contributor) +- [imalasong](https://github.com/imalasong) (First-time contributor) +- [jiyfhust](https://github.com/jiyfhust) +- [negachov](https://github.com/negachov) (First-time contributor) +- [testwill](https://github.com/testwill) +- [yzhan1](https://github.com/yzhan1) (First-time contributor) +- [zxc111](https://github.com/zxc111) (First-time contributor) \ No newline at end of file From 536d5b0f1d393c0cb13394510ac5dd6b5f1c5448 Mon Sep 17 00:00:00 2001 From: Aolin Date: Thu, 9 May 2024 12:09:36 +0800 Subject: [PATCH 30/55] update TiDB improvements and bug fixes --- releases/release-8.1.0.md | 40 +++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 9e94e0abc2f73..fe9692e753ae5 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -108,7 +108,6 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, The automatic management capability of queries that consume more resources than expected provides users with an effective means to quickly mitigate the impact of query problems on overall performance before the root cause is identified, thereby improving the stability of the database. For more information, see [documentation](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). -tion, see [documentation](doc-link). ### SQL @@ -206,13 +205,14 @@ tion, see [documentation](doc-link). + TiDB - - 提升了外键在 SHOW CREATE TABLE 结果中显示的 MySQL 兼容性 [#51837](https://github.com/pingcap/tidb/issues/51837) @[negachov](https://github.com/negachov) - - 提升了表达式默认值在 SHOW CREATE TABLE 结果中显示的 MySQL 兼容性 [#52939](https://github.com/pingcap/tidb/issues/52939) @[CbcWestwolf](https://github.com/CbcWestwolf) - - 允许使用 ingest 模式并发添加多个索引 [#52596](https://github.com/pingcap/tidb/issues/52596) @[lance6716](https://github.com/lance6716) - - 允许将系统变量 `tidb_service_scope` 设置为不同的值，从而更好地利用分布式框架功能 [#52441](https://github.com/pingcap/tidb/issues/52441) @[ywqzzy](https://github.com/ywqzzy) - - 增强 TiDB 对总是 false 的 DNF 的处理，直接忽略这种过滤条件 [#40997](https://github.com/pingcap/tidb/issues/40997) @[hi-rustin](https://github.com/hi-rustin) - - 不允许将 tidb_auto_analyze_ratio 参数设置为 0 [#51582](https://github.com/pingcap/tidb/issues/51582) @[hi-rustin](https://github.com/hi-rustin) - - 如果查询有除了全表扫描以外的单索引扫描方式可以选择，优化器不会自动选择索引合并。现在可以通过 Optimizer Fix Controls 机制解除这个限制 [#52869](https://github.com/pingcap/tidb/issues/52869) @[time-and-fate](https://github.com/time-and-fate) + - Improve the MySQL compatibility of foreign keys displayed in the output of `SHOW CREATE TABLE` [#51837](https://github.com/pingcap/tidb/issues/51837) @[negachov](https://github.com/negachov) + - Improve the MySQL compatibility of expression default values displayed in the output of `SHOW CREATE TABLE` [#52939](https://github.com/pingcap/tidb/issues/52939) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Support adding multiple indexes concurrently in the ingest mode [#52596](https://github.com/pingcap/tidb/issues/52596) @[lance6716](https://github.com/lance6716) + - Support configuring the system variable `tidb_service_scope` with various values, enhancing the utilization of the Distributed eXecution Framework [#52441](https://github.com/pingcap/tidb/issues/52441) @[ywqzzy](https://github.com/ywqzzy) + - Enhance the handling of DNFs that are always `false` by directly ignoring such filter conditions [#40997](https://github.com/pingcap/tidb/issues/40997) @[hi-rustin](https://github.com/hi-rustin) + - Do not support setting the system variable `tidb_auto_analyze_ratio` to `0` [#51582](https://github.com/pingcap/tidb/issues/51582) @[hi-rustin](https://github.com/hi-rustin) + - Support using Optimizer Fix Controls to remove the limitation that the optimizer does not automatically select Index Merge when the query can select a single index scan method other than a full table scan [#52869](https://github.com/pingcap/tidb/issues/52869) @[time-and-fate](https://github.com/time-and-fate) + - Add the `total_kv_read_wall_time` metric to the column `execution info` of Coprocessor operators [#28937](https://github.com/pingcap/tidb/issues/28937) @[cfzjywxk](https://github.com/cfzjywxk) - (dup): release-7.1.5.md > Improvements> TiDB - Add a timeout mechanism for LDAP authentication to avoid the issue of resource lock (RLock) not being released in time [#51883](https://github.com/pingcap/tidb/issues/51883) @[YangKeao](https://github.com/YangKeao) + TiKV @@ -271,18 +271,18 @@ tion, see [documentation](doc-link). + TiDB - - 修复当 SQL 涉及包含多值索引的表时，执行可能报错“Can't find a proper physical plan for this query”的问题 [#49438](https://github.com/pingcap/tidb/issues/49438) @[qw4990](https://github.com/qw4990) - - 修复自动分析在 OOM 后卡住的问题 [#51993](https://github.com/pingcap/tidb/issues/51993) @[hi-rustin](https://github.com/hi-rustin) - - 修复使用 BR 恢复一张表之后，即使这张表没有统计信息，统计信息健康度也显示 100 的问题 [#29769](https://github.com/pingcap/tidb/issues/29769) @[winoros](https://github.com/winoros) - - 修复了 TiDB 在升级过程中会为系统表创建统计信息的问题 [#52040](https://github.com/pingcap/tidb/issues/52040) @[hi-rustin](https://github.com/hi-rustin) - - 修复 TiDB 在统计信息初始化完成前就进行自动分析的问题 [#52346](https://github.com/pingcap/tidb/issues/52346) @[hi-rustin](https://github.com/hi-rustin) - - 修复启用 `tidb_mem_quota_analyze` 时，自动分析使用的内存超过限制可能导致 TiDB crash 的问题 [#52601](https://github.com/pingcap/tidb/issues/52601) @[hawkingrei](https://github.com/hawkingrei) - - 修复 TiDB 统计信息同步加载机制无限重试加载空统计信息并打印“fail to get stats version for this histogram”日志的问题 [#52657](https://github.com/pingcap/tidb/issues/52657) @[hawkingrei](https://github.com/hawkingrei) - - 修复在关闭新排序规则框架的时候，涉及不同排序规则的表达式可能导致查询 panic 的问题 [#52772](https://github.com/pingcap/tidb/issues/52772) @[wjhuang2016](https://github.com/wjhuang2016) - - 修复 `CPS by type` 监控项显示错误的问题 [#52605](https://github.com/pingcap/tidb/issues/52605) @[nolouch](https://github.com/nolouch) - - 修复查询 `INFORMATION_SCHEMA.TIKV_REGION_STATUS` 出现空指针的问题 [#52013](https://github.com/pingcap/tidb/issues/52013) @[JmPotato](https://github.com/JmPotato) - - 修正了指定非法列默认值时的错误提示信息 [#51592](https://github.com/pingcap/tidb/issues/51592) @[danqixu](https://github.com/danqixu) - - 修复了 ingest 模式添加索引时，在边角案例下导致数据索引不一致的问题 [#51954](https://github.com/pingcap/tidb/issues/51954) @[lance6716](https://github.com/lance6716) + - Fix the issue that executing SQL statements containing tables with multi-valued indexes might return the `Can't find a proper physical plan for this query` error [#49438](https://github.com/pingcap/tidb/issues/49438) @[qw4990](https://github.com/qw4990) + - Fix the issue that auto analyze gets stuck after an OOM error occurs [#51993](https://github.com/pingcap/tidb/issues/51993) @[hi-rustin](https://github.com/hi-rustin) + - Fix the issue that after using BR to restore a table that has no statistics, the statistics health is still 100% [#29769](https://github.com/pingcap/tidb/issues/29769) @[winoros](https://github.com/winoros) + - Fix the issue that TiDB creates statistics for system tables during upgrade [#52040](https://github.com/pingcap/tidb/issues/52040) @[hi-rustin](https://github.com/hi-rustin) + - Fix the issue that auto analyze is triggered before the initialization of statistics finishes [#52346](https://github.com/pingcap/tidb/issues/52346) @[hi-rustin](https://github.com/hi-rustin) + - Fix the issue that TiDB might crash when `tidb_mem_quota_analyze` is enabled and the memory used by auto analyze exceeds the limit [#52601](https://github.com/pingcap/tidb/issues/52601) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the TiDB synchronously loading statistics mechanism retries to load empty statistics indefinitely and prints the `fail to get stats version for this histogram` log [#52657](https://github.com/pingcap/tidb/issues/52657) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that expressions containing different collations might cause the query to panic when the new framework for collations is disabled [#52772](https://github.com/pingcap/tidb/issues/52772) @[wjhuang2016](https://github.com/wjhuang2016) + - Fix the issue that the `CPS by type` metric displays incorrect values [#52605](https://github.com/pingcap/tidb/issues/52605) @[nolouch](https://github.com/nolouch) + - Fix the issue that the null pointer error occurs when querying `INFORMATION_SCHEMA.TIKV_REGION_STATUS` [#52013](https://github.com/pingcap/tidb/issues/52013) @[JmPotato](https://github.com/JmPotato) + - Fix the incorrect error message displayed when an invalid default value is specified for a column [#51592](https://github.com/pingcap/tidb/issues/51592) @[danqixu](https://github.com/danqixu) + - Fix the issue that adding indexes in the ingest mode might cause inconsistent data index in some corner cases [#51954](https://github.com/pingcap/tidb/issues/51954) @[lance6716](https://github.com/lance6716) - 修复了恢复含有外键的表时，DDL 卡住的问题 [#51838](https://github.com/pingcap/tidb/issues/51838) @[YangKeao](https://github.com/YangKeao) - 修复了加索引期间 TiDB 网络隔离导致加索引失败的问题 [#51846](https://github.com/pingcap/tidb/issues/51846) @[ywqzzy](https://github.com/ywqzzy) From 21ad845e41fb79a0dab12a2981d956b805fdadf0 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 10 May 2024 09:55:29 +0800 Subject: [PATCH 31/55] Apply suggestions from code review --- releases/release-8.1.0.md | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index fe9692e753ae5..7b69b30cd5f36 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -234,13 +234,13 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - (dup): release-7.1.5.md > Improvements> Tools> Backup & Restore (BR) - Add PITR integration test cases to cover compatibility testing for log backup and adding index acceleration [#51987](https://github.com/pingcap/tidb/issues/51987) @[Leavrth](https://github.com/Leavrth) - (dup): release-7.1.5.md > Improvements> Tools> Backup & Restore (BR) - Remove the invalid verification for active DDL jobs when log backup starts [#52733](https://github.com/pingcap/tidb/issues/52733) @[Leavrth](https://github.com/Leavrth) - - 增加测试用例测试 PITR 和添加索引加速兼容性 [#51988](https://github.com/pingcap/tidb/issues/51988) @[Leavrth](https://github.com/Leavrth) - - BR 恢复过程中清理空 sst 文件 [#16005](https://github.com/tikv/tikv/issues/16005) @[Leavrth](https://github.com/Leavrth) + - Add test cases to test compatibility between PITR and the acceleration of adding indexes feature [#51988](https://github.com/pingcap/tidb/issues/51988) @[Leavrth](https://github.com/Leavrth) + - BR cleans up empty SST files during data recovery [#16005](https://github.com/tikv/tikv/issues/16005) @[Leavrth](https://github.com/Leavrth) + TiCDC - - 提升了使用 redo log 恢复数据过程中的内存稳定性，减少了 OOM 的概率 [#10900](https://github.com/pingcap/tiflow/issues/10900) @[CharlesCheung96](https://github.com/CharlesCheung96) - - 显著提升了事务冲突场景的数据同步的稳定性，性能最高提升 10 倍 [#10896](https://github.com/pingcap/tiflow/issues/10896) @[CharlesCheung96](https://github.com/CharlesCheung96) + - Improve memory stability during data recovery using redo log to reduce the probability of OOM [#10900](https://github.com/pingcap/tiflow/issues/10900) @[CharlesCheung96](https://github.com/CharlesCheung96) + - Significantly improve the stability of data replication in transaction conflict scenarios, with up to 10 times performance improvement [#10896](https://github.com/pingcap/tiflow/issues/10896) @[CharlesCheung96](https://github.com/CharlesCheung96) + TiDB Data Migration (DM) @@ -340,23 +340,23 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - (dup): release-6.5.9.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that removing a log backup task after it is paused does not immediately restore the GC safepoint [#52082](https://github.com/pingcap/tidb/issues/52082) @[3pointer](https://github.com/3pointer) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix a rare issue that special event timing might cause the data loss in log backup [#16739](https://github.com/tikv/tikv/issues/16739) @[YuJuncen](https://github.com/YuJuncen) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that the global checkpoint of log backup is advanced ahead of the actual backup file write point due to TiKV restart, which might cause a small amount of backup data loss [#16809](https://github.com/tikv/tikv/issues/16809) @[YuJuncen](https://github.com/YuJuncen) - - 去除全量备份日志输出无效的 concurrency 属性 [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) - - 修复在 PD leader 丢失情况下, BR 恢复或者是 lightning 物理模式下失败的问题 [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - - 修复日志备份在暂停, 停止,再重建任务操作后, 任务状态显示正常, 但 checkpoint 不推进的问题 [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) - - 修复不稳定测试用例 `TestClearCache` [#51671](https://github.com/pingcap/tidb/issues/51671) @[zxc111](https://github.com/zxc111) - - 修复不稳定测试用例 `TestGetMergeRegionSizeAndCount` [#52095](https://github.com/pingcap/tidb/issues/52095) @[3pointer](https://github.com/3pointer) - - 修复不稳定集成测试 `br_tikv_outage` [#52673](https://github.com/pingcap/tidb/issues/52673) @[Leavrth](https://github.com/Leavrth) - - 控制测试用例 `TestGetTSWithRetry` 执行时间 [#52547](https://github.com/pingcap/tidb/issues/52547) @[Leavrth](https://github.com/Leavrth) + - Fix the issue that redundant concurrency related information appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) + - Fix the issue that the Region fetched from PD does not have a Leader when restoring data by BR or importing data by TiDB Lightning in physical import mode [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) + - Fix the issue that after pausing, stopping, and rebuilding the log backup task, the task status is normal, but the checkpoint does not advance [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) + - Fix the unstable test case `TestClearCache` [#51671](https://github.com/pingcap/tidb/issues/51671) @[zxc111](https://github.com/zxc111) + - Fix the unstable test case `TestGetMergeRegionSizeAndCount` [#52095](https://github.com/pingcap/tidb/issues/52095) @[3pointer](https://github.com/3pointer) + - Fix the unstable integration test `br_tikv_outage` [#52673](https://github.com/pingcap/tidb/issues/52673) @[Leavrth](https://github.com/Leavrth) + - Fix the issue that the test case `TestGetTSWithRetry` takes too long to execute [#52547](https://github.com/pingcap/tidb/issues/52547) @[Leavrth](https://github.com/Leavrth) + TiCDC - - 修复调用驱逐 TiCDC owner 节点（/api/v2/owner/resign）的命令后意外导致 CDC 任务重启的问题 [#10781](https://github.com/pingcap/tiflow/issues/10781) @[sdojjy](https://github.com/sdojjy) - - 修复当下游 pulsar 下线后移除 changefeed 会导致 CDC 正常流程卡住从而引起其他 changefeed 进度卡住的问题 [#10629](https://github.com/pingcap/tiflow/issues/10629) @[asddongmen](https://github.com/asddongmen) - - 修复 grafana 监控中 Owner 面板显示不稳定的问题 [#10796](https://github.com/pingcap/tiflow/issues/10796) @[hongyunyan](https://github.com/hongyunyan) - - 修复重启 PD 可能导致 TiCDC 节点报错重启的问题 [#10799](https://github.com/pingcap/tiflow/issues/10799) @[3AceShowHand](https://github.com/3AceShowHand) - - 修复当 PD 磁盘 IO 延迟较大会导致显著的 CDC 同步延迟的问题 [#9054](https://github.com/pingcap/tiflow/issues/9054) @[asddongmen](https://github.com/asddongmen) - - 修复没有按照正确的时区设置默认值的问题 [#10931](https://github.com/pingcap/tiflow/issues/10931) @[3AceShowHand](https://github.com/3AceShowHand) - - 修复 drop primary/unique key 没有没正确同步的问题 [#10890](https://github.com/pingcap/tiflow/issues/10890) @[asddongmen](https://github.com/asddongmen) + - Fix the issue that calling the API (/api/v2/owner/resign) that evicts the TiCDC owner node causes the TiCDC task to restart unexpectedly [#10781](https://github.com/pingcap/tiflow/issues/10781) @[sdojjy](https://github.com/sdojjy) + - Fix the issue that when the downstream Pulsar is stopped, removing the changefeed causes the normal TiCDC process to get stuck, which causes other changefeed processes to get stuck [#10629](https://github.com/pingcap/tiflow/issues/10629) @[asddongmen](https://github.com/asddongmen) + - Fix the issue that the **Ownership history** panel in Grafana is unstable [#10796](https://github.com/pingcap/tiflow/issues/10796) @[hongyunyan](https://github.com/hongyunyan) + - Fix the issue that restarting PD might cause the TiCDC node to restart with an error [#10799](https://github.com/pingcap/tiflow/issues/10799) @[3AceShowHand](https://github.com/3AceShowHand) + - Fix the issue that long latency in the PD disk IO causes significant latency in data replication [#9054](https://github.com/pingcap/tiflow/issues/9054) @[asddongmen](https://github.com/asddongmen) + - Fix the issue that the default value is not set according to the correct time zone [#10931](https://github.com/pingcap/tiflow/issues/10931) @[3AceShowHand](https://github.com/3AceShowHand) + - Fix the issue that `DROP PRIMARY KEY` and `DROP UNIQUE KEY` statements are not replicated correctly [#10890](https://github.com/pingcap/tiflow/issues/10890) @[asddongmen](https://github.com/asddongmen) - (dup): release-7.1.5.md > Bug fixes> Tools> TiCDC - Fix the issue that TiCDC fails to execute the `Exchange Partition ... With Validation` DDL downstream after it is written upstream, causing the changefeed to get stuck [#10859](https://github.com/pingcap/tiflow/issues/10859) @[hongyunyan](https://github.com/hongyunyan) + TiDB Data Migration (DM) From 34a0bea5db2dceb2bb358350bcacb9e46f3dd721 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 10 May 2024 11:18:58 +0800 Subject: [PATCH 32/55] add compatibility changes > system variables: tidb_auto_analyze_ratio --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 7b69b30cd5f36..32c5bdfd500c3 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -175,7 +175,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, | Variable name | Change type | Description | |--------|------------------------------|------| -| | | | +| [`tidb_auto_analyze_ratio`](/system-variables.md#tidb_auto_analyze_ratio) | Modified | Changes the value range from `[0, 18446744073709551615]` to `(0, 1]`. | | [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) | Modified | Changes the default value from `OFF` to `ON`. This means that Distributed eXecution Framework (DXF) is enabled by default, which fully utilizes the resources of the TiDB cluster and greatly improves the performance of `ADD INDEX` and `IMPORT INTO` tasks. If you want to upgrade a cluster with the DXF enabled to v8.1.0 or later, disable the DXF (by setting `tidb_enable_dist_task` to `OFF`) before the upgrade, which avoids `ADD INDEX` operations during the upgrade causing data index inconsistency. After the upgrade, you can manually enable the DXF. | | [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Modified | Changes the optional value from "" or `background` to a string with a length of up to 64 characters, which enables you to control the service scope of each TiDB node more flexibly. Valid characters include digits `0-9`, letters `a-zA-Z`, underscores `_`, and hyphens `-`. The Distributed eXecution Framework (DXF) determines which TiDB nodes can be scheduled to execute distributed tasks based on the value of this variable. For specific rules, see [Task scheduling](/tidb-distributed-execution-framework.md#task-scheduling). | | | | | From d583a53787c931e551b7fb0acd0fd64e6cf1f02d Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 10 May 2024 11:28:59 +0800 Subject: [PATCH 33/55] update highlights --- releases/release-8.1.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 32c5bdfd500c3..0924f92b0c136 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -44,15 +44,15 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Enhance the stability of caching the schema information when there is a massive number of tables (experimental, introduced in v8.0.0) SaaS companies using TiDB as the system of record for their multi-tenant applications often need to store a substantial number of tables. In previous versions, handling table counts in the order of a million or more was feasible, but it had the potential to degrade the overall user experience. TiDB v8.0.0 improves the situation with the following enhancements: -
      -
    • Introduce a new schema information caching system, which provides a cache strategy based on the Least Recently Used (LRU) algorithm for table metadata. It prioritizes storing the metadata of frequently accessed tables in the cache, thereby reducing memory usage in scenarios with a large number of tables.
      • -
    • Implement a priority queue for auto analyze, making the process less rigid and enhancing stability across a wider array of tables.
      • -
    +
      +
    • Introduce a new schema information caching system, which provides a cache strategy based on the Least Recently Used (LRU) algorithm for table metadata. It prioritizes storing the metadata of frequently accessed tables in the cache, thereby reducing memory usage in scenarios with a large number of tables.
      • +
    • Implement a priority queue for auto analyze, making the process less rigid and enhancing stability across a wider array of tables.
      • +
    Reliability and availability - Global Sort (GA in v8.0.0) **tw@qiancai** + Global Sort (GA in v8.0.0) The Global Sort feature aims to improve the stability and efficiency of IMPORT INTO and CREATE INDEX. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, each IMPORT INTO or CREATE INDEX statement now supports importing or adding indexes for up to 40 TiB of data. @@ -69,7 +69,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Previously, using DM to migrate data from MySQL 8.0 is an experimental feature and is not available for production environments. TiDB v7.6.0 enhances the stability and compatibility of this feature to help you smoothly and quickly migrate data from MySQL 8.0 to TiDB in production environments. In v7.6.0, this feature becomes generally available (GA). - TiDB resource control supports managing queries that consume more resources than expected (GA) **tw@lilin90** + TiDB resource control supports managing queries that consume more resources than expected (GA in v8.1.0) Through the rules of resource groups, TiDB can automatically identify queries that consume more resources than expected, and then limit or cancel these queries. Even if the queries are not identified by the rules, you can still manually add query characteristics and take corresponding measures to reduce the impact of the sudden query performance problem on the entire database. @@ -87,7 +87,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, TiCDC introduces a new protocol, the Debezium protocol. TiCDC can now publish data change events to a Kafka sink using a protocol that generates Debezium style messages. - TiCDC supports client authentication (introduced in v8.1.0) **tw@Oreoxmt** + TiCDC supports client authentication (introduced in v8.1.0) TiCDC supports client authentication using mutual Transport Layer Security (mTLS) or TiDB username and password. This feature enables CLI or OpenAPI clients to authenticate their connections to TiCDC. From a374ced7dda4c26b0f66760704ca8878dbe4cd71 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 10 May 2024 11:51:54 +0800 Subject: [PATCH 34/55] update compatibility changes --- releases/release-8.1.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 0924f92b0c136..8579509caf442 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -177,19 +177,19 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, |--------|------------------------------|------| | [`tidb_auto_analyze_ratio`](/system-variables.md#tidb_auto_analyze_ratio) | Modified | Changes the value range from `[0, 18446744073709551615]` to `(0, 1]`. | | [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-new-in-v710) | Modified | Changes the default value from `OFF` to `ON`. This means that Distributed eXecution Framework (DXF) is enabled by default, which fully utilizes the resources of the TiDB cluster and greatly improves the performance of `ADD INDEX` and `IMPORT INTO` tasks. If you want to upgrade a cluster with the DXF enabled to v8.1.0 or later, disable the DXF (by setting `tidb_enable_dist_task` to `OFF`) before the upgrade, which avoids `ADD INDEX` operations during the upgrade causing data index inconsistency. After the upgrade, you can manually enable the DXF. | -| [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Modified | Changes the optional value from "" or `background` to a string with a length of up to 64 characters, which enables you to control the service scope of each TiDB node more flexibly. Valid characters include digits `0-9`, letters `a-zA-Z`, underscores `_`, and hyphens `-`. The Distributed eXecution Framework (DXF) determines which TiDB nodes can be scheduled to execute distributed tasks based on the value of this variable. For specific rules, see [Task scheduling](/tidb-distributed-execution-framework.md#task-scheduling). | -| | | | -| | | | +| [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Modified | Changes the optional value from `""` or `background` to a string with a length of up to 64 characters, which enables you to control the service scope of each TiDB node more flexibly. Valid characters include digits `0-9`, letters `a-zA-Z`, underscores `_`, and hyphens `-`. The Distributed eXecution Framework (DXF) determines which TiDB nodes can be scheduled to execute distributed tasks based on the value of this variable. For specific rules, see [Task scheduling](/tidb-distributed-execution-framework.md#task-scheduling). | ### Configuration file parameters | Configuration file | Configuration parameter | Change type | Description | | -------- | -------- | -------- | -------- | -| TiDB Lightning | [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) | Modified | Changes the default value from `9223372036854775807` to `10000` to quickly interrupt abnormal tasks so that you can make corresponding adjustments as soon as possible. This saves time and computational resources by avoiding the scenario where a large amount of conflicting data is discovered after the import, caused by abnormal data sources or incorrect table schema definitions. | | TiDB Lightning | [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-configuration) | Modified | Starting from v8.1.0, there is no need to configure `conflict.max-record-rows` manually, because TiDB Lightning automatically assigns the value of `conflict.max-record-rows` with the value of `conflict.threshold`, regardless of the user input. `conflict.max-record-rows` will be deprecated in a future release. | -| | | | | -| | | | | -| | | | | +| TiDB Lightning | [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) | Modified | Changes the default value from `9223372036854775807` to `10000` to quickly interrupt abnormal tasks so that you can make corresponding adjustments as soon as possible. This saves time and computational resources by avoiding the scenario where a large amount of conflicting data is discovered after the import, caused by abnormal data sources or incorrect table schema definitions. | +| TiCDC | [`security.client-allowed-user`](/ticdc/ticdc-server-config.md#cdc-server-configuration-file-parameters) | Newly added | Lists the usernames that are allowed for client authentication. Authentication requests with usernames not in this list will be rejected. The default value is null. | +| TiCDC | [`security.client-user-required`](/ticdc/ticdc-server-config.md#cdc-server-configuration-file-parameters) | Newly added | Controls whether to use username and password for client authentication. The default value is `false`. | +| TiCDC | [`security.mtls`](/ticdc/ticdc-server-config.md#cdc-server-configuration-file-parameters) | Newly added | Controls whether to enable the TLS client authentication. The default value is `false`. | +| TiCDC | [`sink.debezium.output-old-value`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | Newly added | Controls whether to output the value before the row data changes. The default value is true. When it is disabled, the UPDATE event does not output the "before" field. | +| TiCDC | [`sink.open.output-old-value`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | Newly added | Controls whether to output the value before the row data changes. The default value is true. When it is disabled, the UPDATE event does not output the "p" field. | ### Others From 649f2348d1f210b3da985e1ab7fedbd7b70e4c5a Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 10 May 2024 15:04:17 +0800 Subject: [PATCH 35/55] add translation for PD/TiFlash/TiDB Lightning bug fixes --- releases/release-8.1.0.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 8579509caf442..eda19b066f414 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -221,7 +221,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + PD - - 优化调度 operator 相关逻辑竞争锁的开销 [#7897](https://github.com/tikv/pd/issues/7897) @[nolouch](https://github.com/nolouch) + - Optimize the logic for `OperatorController` to reduce the overhead of competition locks [#7897](https://github.com/tikv/pd/issues/7897) @[nolouch](https://github.com/nolouch) + TiFlash @@ -314,23 +314,23 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + PD - - 修复切换 PD 模式时 TSO 可能卡住的问题 [#7849](https://github.com/tikv/pd/issues/7849) @[JmPotato](https://github.com/JmPotato) - - 修复 DR Auto Sync 监控状态显示的问题 [#7974](https://github.com/tikv/pd/issues/7974) @[lhy1024](https://github.com/lhy1024) - - 修复检查 binary 版本时可能导致的 panic 问题 [#7978](https://github.com/tikv/pd/issues/7978) @[JmPotato](https://github.com/JmPotato) - - 修复 TTL 参数类型转换错误的问题 [#7980](https://github.com/tikv/pd/issues/7980) @[HuSharp](https://github.com/HuSharp) - - 修复两中心部署切换时 leader 无法迁移的问题 [#7992](https://github.com/tikv/pd/issues/7992) @[TonsnakeLin](https://github.com/TonsnakeLin) - - 修复 pd-ctl 标准错误重定向的问题 [#8022](https://github.com/tikv/pd/issues/8022) @[HuSharp](https://github.com/HuSharp) - - 修复在生成 merge 调度时可能出现的 panic 问题 [#8049](https://github.com/tikv/pd/issues/8049) @[nolouch](https://github.com/nolouch) - - 修复 `GetAdditionalInfo` 导致的 panic 问题 [#8079](https://github.com/tikv/pd/issues/8079) @[HuSharp](https://github.com/HuSharp) - - 修复 scatter range 调度器监控显示的问题 [#8125](https://github.com/tikv/pd/issues/8125) @[HuSharp](https://github.com/HuSharp) + - Fix the issue that TSO might get stuck when toggling PD microservice mode on and off [#7849](https://github.com/tikv/pd/issues/7849) @[JmPotato](https://github.com/JmPotato) + - Fix the issue that the `State` monitoring metric for DR Auto-Sync does not show any data [#7974](https://github.com/tikv/pd/issues/7974) @[lhy1024](https://github.com/lhy1024) + - Fix the issue that checking binary versions might cause PD panic [#7978](https://github.com/tikv/pd/issues/7978) @[JmPotato](https://github.com/JmPotato) + - Fix the type conversion error that occurs when TTL parameters are parsed [#7980](https://github.com/tikv/pd/issues/7980) @[HuSharp](https://github.com/HuSharp) + - Fix the issue that the Leader fails to transfer when you switch it between two deployed data centers [#7992](https://github.com/tikv/pd/issues/7992) @[TonsnakeLin](https://github.com/TonsnakeLin) + - Fix the issue that `PrintErrln` in pd-ctl fails to output error messages to stderr [#8022](https://github.com/tikv/pd/issues/8022) @[HuSharp](https://github.com/HuSharp) + - Fix the issue that PD might panic when generating `Merge` schedules [#8049](https://github.com/tikv/pd/issues/8049) @[nolouch](https://github.com/nolouch) + - Fix the panic issue caused by `GetAdditionalInfo` [#8079](https://github.com/tikv/pd/issues/8079) @[HuSharp](https://github.com/HuSharp) + - Fix the issue that the `Filter target` monitoring metric for PD does not provide scatter range information [#8125](https://github.com/tikv/pd/issues/8125) @[HuSharp](https://github.com/HuSharp) - (dup): release-6.5.9.md > Bug fixes> PD - Fix the issue that the query result of `SHOW CONFIG` includes the deprecated configuration item `trace-region-flow` [#7917](https://github.com/tikv/pd/issues/7917) @[rleungx](https://github.com/rleungx) - (dup): release-6.5.9.md > Bug fixes> PD - Fix the issue that the scaling progress is not correctly displayed [#7726](https://github.com/tikv/pd/issues/7726) @[CabinfeverB](https://github.com/CabinfeverB) + TiFlash - (dup): release-6.5.9.md > Bug fixes> TiFlash - Fix the issue that TiFlash might panic when you insert data to columns with invalid default values in non-strict `sql_mode` [#8803](https://github.com/pingcap/tiflash/issues/8803) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) - - 修复 TiFlash 在高并发读的情况下，可能返回瞬时不正确的结果的问题 [#8845](https://github.com/pingcap/tiflash/issues/8845) @[JaySon-Huang](https://github.com/JaySon-Huang) - - 修复存算分离架构下，计算节点的本地缓存容量配置项的值修改后，上报 PD 的硬盘使用量不正确的问题 [#8920](https://github.com/pingcap/tiflash/issues/8920) @[JinheLin](https://github.com/JinheLin) + - Fix the issue that TiFlash might return transiently incorrect results in high-concurrency read scenarios [#8845](https://github.com/pingcap/tiflash/issues/8845) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the issue that in the disaggregated storage and compute architecture, the disk `used_size` metric displayed in Grafana is incorrect after you modify the value of the `storage.remote.cache.capacity` configuration item for TiFlash compute nodes [#8920](https://github.com/pingcap/tiflash/issues/8920) @[JinheLin](https://github.com/JinheLin) + Tools @@ -366,10 +366,10 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + TiDB Lightning - - 修复了 lightning 导入时，因源文件存在不兼容的 SQL 而报 “no database selected” 的问题 [#51800](https://github.com/pingcap/tidb/issues/51800) @[lance6716](https://github.com/lance6716) - - 修复了 lightning 在 server 模式下可能会打印敏感信息到日志的问题 [#36374](https://github.com/pingcap/tidb/issues/36374) @[kennytm](https://github.com/kennytm) - - 修复了 lightning 导入期间报 “invalid store ID 0” 错误的问题 [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - - 修复了 lighting 使用 replace 作为去重算法时报 “Unknown column in where clause” 错误的问题 [#52886](https://github.com/pingcap/tidb/issues/52886) @[lyzx2001](https://github.com/lyzx2001) + - Fix the issue that TiDB Lightning reports `no database selected` during data import due to incompatible SQL statements in the source files [#51800](https://github.com/pingcap/tidb/issues/51800) @[lance6716](https://github.com/lance6716) + - Fix the issue where TiDB Lightning might print sensitive information to logs in server mode [#36374](https://github.com/pingcap/tidb/issues/36374) @[kennytm](https://github.com/kennytm) + - Fix the issue that killing the PD Leader causes TiDB Lightning to report the `invalid store ID 0` error during data import [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) + - Fix the issue that TiDB Lightning reports the `Unknown column in where clause` error when processing conflicting data using the `replace` method [#52886](https://github.com/pingcap/tidb/issues/52886) @[lyzx2001](https://github.com/lyzx2001) - (dup): release-7.1.5.md > Bug fixes> Tools> TiDB Lightning - Fix the issue that TiDB Lightning panics when importing an empty table of Parquet format [#52518](https://github.com/pingcap/tidb/issues/52518) @[kennytm](https://github.com/kennytm) + Dumpling From e87378699624a4cdf9f660e9d23f4a352148b3a5 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 10 May 2024 15:10:42 +0800 Subject: [PATCH 36/55] Update release-8.1.0.md --- releases/release-8.1.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index eda19b066f414..11c153486b183 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -314,7 +314,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + PD - - Fix the issue that TSO might get stuck when toggling PD microservice mode on and off [#7849](https://github.com/tikv/pd/issues/7849) @[JmPotato](https://github.com/JmPotato) + - Fix the issue that TSO might get stuck when toggling the PD microservice mode on and off [#7849](https://github.com/tikv/pd/issues/7849) @[JmPotato](https://github.com/JmPotato) - Fix the issue that the `State` monitoring metric for DR Auto-Sync does not show any data [#7974](https://github.com/tikv/pd/issues/7974) @[lhy1024](https://github.com/lhy1024) - Fix the issue that checking binary versions might cause PD panic [#7978](https://github.com/tikv/pd/issues/7978) @[JmPotato](https://github.com/JmPotato) - Fix the type conversion error that occurs when TTL parameters are parsed [#7980](https://github.com/tikv/pd/issues/7980) @[HuSharp](https://github.com/HuSharp) @@ -367,7 +367,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + TiDB Lightning - Fix the issue that TiDB Lightning reports `no database selected` during data import due to incompatible SQL statements in the source files [#51800](https://github.com/pingcap/tidb/issues/51800) @[lance6716](https://github.com/lance6716) - - Fix the issue where TiDB Lightning might print sensitive information to logs in server mode [#36374](https://github.com/pingcap/tidb/issues/36374) @[kennytm](https://github.com/kennytm) + - Fix the issue that TiDB Lightning might print sensitive information to logs in server mode [#36374](https://github.com/pingcap/tidb/issues/36374) @[kennytm](https://github.com/kennytm) - Fix the issue that killing the PD Leader causes TiDB Lightning to report the `invalid store ID 0` error during data import [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - Fix the issue that TiDB Lightning reports the `Unknown column in where clause` error when processing conflicting data using the `replace` method [#52886](https://github.com/pingcap/tidb/issues/52886) @[lyzx2001](https://github.com/lyzx2001) - (dup): release-7.1.5.md > Bug fixes> Tools> TiDB Lightning - Fix the issue that TiDB Lightning panics when importing an empty table of Parquet format [#52518](https://github.com/pingcap/tidb/issues/52518) @[kennytm](https://github.com/kennytm) From 74d6e7e85ccf993922dda87df523ba79268aeede Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 10 May 2024 17:36:34 +0800 Subject: [PATCH 37/55] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-8.1.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 11c153486b183..bdff8ed488d13 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -209,9 +209,9 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Improve the MySQL compatibility of expression default values displayed in the output of `SHOW CREATE TABLE` [#52939](https://github.com/pingcap/tidb/issues/52939) @[CbcWestwolf](https://github.com/CbcWestwolf) - Support adding multiple indexes concurrently in the ingest mode [#52596](https://github.com/pingcap/tidb/issues/52596) @[lance6716](https://github.com/lance6716) - Support configuring the system variable `tidb_service_scope` with various values, enhancing the utilization of the Distributed eXecution Framework [#52441](https://github.com/pingcap/tidb/issues/52441) @[ywqzzy](https://github.com/ywqzzy) - - Enhance the handling of DNFs that are always `false` by directly ignoring such filter conditions [#40997](https://github.com/pingcap/tidb/issues/40997) @[hi-rustin](https://github.com/hi-rustin) + - Enhance the handling of DNF items that are always `false` by directly ignoring such filter conditions, thus avoiding unnecessary full table scans [#40997](https://github.com/pingcap/tidb/issues/40997) @[hi-rustin](https://github.com/hi-rustin) - Do not support setting the system variable `tidb_auto_analyze_ratio` to `0` [#51582](https://github.com/pingcap/tidb/issues/51582) @[hi-rustin](https://github.com/hi-rustin) - - Support using Optimizer Fix Controls to remove the limitation that the optimizer does not automatically select Index Merge when the query can select a single index scan method other than a full table scan [#52869](https://github.com/pingcap/tidb/issues/52869) @[time-and-fate](https://github.com/time-and-fate) + - Support using Optimizer Fix Controls to remove the limitation that the optimizer does not automatically choose Index Merge for a query when the optimizer can choose the single index scan method (other than full table scan) for the query [#52869](https://github.com/pingcap/tidb/issues/52869) @[time-and-fate](https://github.com/time-and-fate) - Add the `total_kv_read_wall_time` metric to the column `execution info` of Coprocessor operators [#28937](https://github.com/pingcap/tidb/issues/28937) @[cfzjywxk](https://github.com/cfzjywxk) - (dup): release-7.1.5.md > Improvements> TiDB - Add a timeout mechanism for LDAP authentication to avoid the issue of resource lock (RLock) not being released in time [#51883](https://github.com/pingcap/tidb/issues/51883) @[YangKeao](https://github.com/YangKeao) @@ -272,15 +272,15 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + TiDB - Fix the issue that executing SQL statements containing tables with multi-valued indexes might return the `Can't find a proper physical plan for this query` error [#49438](https://github.com/pingcap/tidb/issues/49438) @[qw4990](https://github.com/qw4990) - - Fix the issue that auto analyze gets stuck after an OOM error occurs [#51993](https://github.com/pingcap/tidb/issues/51993) @[hi-rustin](https://github.com/hi-rustin) - - Fix the issue that after using BR to restore a table that has no statistics, the statistics health is still 100% [#29769](https://github.com/pingcap/tidb/issues/29769) @[winoros](https://github.com/winoros) + - Fix the issue that automatic statistics collection gets stuck after an OOM error occurs [#51993](https://github.com/pingcap/tidb/issues/51993) @[hi-rustin](https://github.com/hi-rustin) + - Fix the issue that after using BR to restore a table that has no statistics, the statistics health of that table is still 100% [#29769](https://github.com/pingcap/tidb/issues/29769) @[winoros](https://github.com/winoros) - Fix the issue that TiDB creates statistics for system tables during upgrade [#52040](https://github.com/pingcap/tidb/issues/52040) @[hi-rustin](https://github.com/hi-rustin) - - Fix the issue that auto analyze is triggered before the initialization of statistics finishes [#52346](https://github.com/pingcap/tidb/issues/52346) @[hi-rustin](https://github.com/hi-rustin) - - Fix the issue that TiDB might crash when `tidb_mem_quota_analyze` is enabled and the memory used by auto analyze exceeds the limit [#52601](https://github.com/pingcap/tidb/issues/52601) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that automatic statistics collection is triggered before the initialization of statistics finishes [#52346](https://github.com/pingcap/tidb/issues/52346) @[hi-rustin](https://github.com/hi-rustin) + - Fix the issue that TiDB might crash when `tidb_mem_quota_analyze` is enabled and the memory used by updating statistics exceeds the limit [#52601](https://github.com/pingcap/tidb/issues/52601) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that the TiDB synchronously loading statistics mechanism retries to load empty statistics indefinitely and prints the `fail to get stats version for this histogram` log [#52657](https://github.com/pingcap/tidb/issues/52657) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that expressions containing different collations might cause the query to panic when the new framework for collations is disabled [#52772](https://github.com/pingcap/tidb/issues/52772) @[wjhuang2016](https://github.com/wjhuang2016) - Fix the issue that the `CPS by type` metric displays incorrect values [#52605](https://github.com/pingcap/tidb/issues/52605) @[nolouch](https://github.com/nolouch) - - Fix the issue that the null pointer error occurs when querying `INFORMATION_SCHEMA.TIKV_REGION_STATUS` [#52013](https://github.com/pingcap/tidb/issues/52013) @[JmPotato](https://github.com/JmPotato) + - Fix the issue that the nil pointer error occurs when you query `INFORMATION_SCHEMA.TIKV_REGION_STATUS` [#52013](https://github.com/pingcap/tidb/issues/52013) @[JmPotato](https://github.com/JmPotato) - Fix the incorrect error message displayed when an invalid default value is specified for a column [#51592](https://github.com/pingcap/tidb/issues/51592) @[danqixu](https://github.com/danqixu) - Fix the issue that adding indexes in the ingest mode might cause inconsistent data index in some corner cases [#51954](https://github.com/pingcap/tidb/issues/51954) @[lance6716](https://github.com/lance6716) From 81860044ae528e2ec7da47d69a153fb27cf1ced5 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 10 May 2024 17:42:38 +0800 Subject: [PATCH 38/55] Apply suggestions from code review --- releases/release-8.1.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index bdff8ed488d13..23097ae7318b6 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -295,6 +295,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - 修复了执行 IMPORT INTO 大数据量任务时卡住的问题 [#52884](https://github.com/pingcap/tidb/issues/52884) @[lance6716](https://github.com/lance6716) - 修复了记录 GRPC 错误时 TIDB 意外重启的问题 [#52179](https://github.com/pingcap/tidb/issues/52179) @[guo-shaoge](https://github.com/guo-shaoge) - 修复了 IndexHashJoin 在执行 Anti Left Outer Semi Join 计算时输出冗余数据的问题 [#52923](https://github.com/pingcap/tidb/issues/52923) @[yibin87](https://github.com/yibin87) + - 修复关联子查询中 TopN 算子结果不正确的问题 [#52777](https://github.com/pingcap/tidb/issues/52777) @[yibin87](https://github.com/yibin87) - 修复了 HashJoin Probe 执行时间统计不精确的问题 [#52227](https://github.com/pingcap/tidb/issues/52227) @[windtalker](https://github.com/windtalker) - 修复了在分区裁剪模式为 static 情况下，使用 tablesample 返回错误结果的问题 [#52282](https://github.com/pingcap/tidb/issues/52282) @[tangenta](https://github.com/tangenta) - 修复 TTL 功能在非默认时区下的错误 [#51675](https://github.com/pingcap/tidb/issues/51675) @[lcwangchao](https://github.com/lcwangchao) From d89c71c40596703beadcec15b2e6edb4b371a199 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Fri, 10 May 2024 18:14:34 +0800 Subject: [PATCH 39/55] Add part of the translation for TiDB bug fixes --- releases/release-8.1.0.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 11c153486b183..5195af962fe7c 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -284,17 +284,18 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the incorrect error message displayed when an invalid default value is specified for a column [#51592](https://github.com/pingcap/tidb/issues/51592) @[danqixu](https://github.com/danqixu) - Fix the issue that adding indexes in the ingest mode might cause inconsistent data index in some corner cases [#51954](https://github.com/pingcap/tidb/issues/51954) @[lance6716](https://github.com/lance6716) - - 修复了恢复含有外键的表时，DDL 卡住的问题 [#51838](https://github.com/pingcap/tidb/issues/51838) @[YangKeao](https://github.com/YangKeao) - - 修复了加索引期间 TiDB 网络隔离导致加索引失败的问题 [#51846](https://github.com/pingcap/tidb/issues/51846) @[ywqzzy](https://github.com/ywqzzy) - - 修复了重命名索引后再添加同名索引时的报错问题 [#51431](https://github.com/pingcap/tidb/issues/51431) @[lance6716](https://github.com/lance6716) - - 修复了添加索引期间升级集群导致的数据索引不一致问题 [#52411](https://github.com/pingcap/tidb/issues/52411) @[tangenta](https://github.com/tangenta) - - 修复了开启分布式框架后为大数据量表添加索引不成功的问题 [#52640](https://github.com/pingcap/tidb/issues/52640) @[tangenta](https://github.com/tangenta) - - 修复了并发添加索引时报 “文件不存在” 错误的问题 [#52475](https://github.com/pingcap/tidb/issues/52475) @[tangenta](https://github.com/tangenta) - - 修复了添加索引失败后清理不掉临时数据的问题 [#52639](https://github.com/pingcap/tidb/issues/52639) @[lance6716](https://github.com/lance6716) - - 修复了元数据锁在 plan cache 场景下未能阻止 DDL 推进的问题 [#51407](https://github.com/pingcap/tidb/issues/51407) @[wjhuang2016](https://github.com/wjhuang2016) - - 修复了执行 IMPORT INTO 大数据量任务时卡住的问题 [#52884](https://github.com/pingcap/tidb/issues/52884) @[lance6716](https://github.com/lance6716) - - 修复了记录 GRPC 错误时 TIDB 意外重启的问题 [#52179](https://github.com/pingcap/tidb/issues/52179) @[guo-shaoge](https://github.com/guo-shaoge) - - 修复了 IndexHashJoin 在执行 Anti Left Outer Semi Join 计算时输出冗余数据的问题 [#52923](https://github.com/pingcap/tidb/issues/52923) @[yibin87](https://github.com/yibin87) + - Fix the issue that DDL operations get stuck when restoring a table with the foreign key [#51838](https://github.com/pingcap/tidb/issues/51838) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that adding indexes fails when the TiDB network is isolated [#51846](https://github.com/pingcap/tidb/issues/51846) @[ywqzzy](https://github.com/ywqzzy) + - Fix the issue that adding an index with the same name after renaming an index causes an error [#51431](https://github.com/pingcap/tidb/issues/51431) @[lance6716](https://github.com/lance6716) + - Fix the issue of inconsistent data indexes caused by cluster upgrade during adding indexes [#52411](https://github.com/pingcap/tidb/issues/52411) @[tangenta](https://github.com/tangenta) + - Fix the issue that adding indexes to large tables fails after enabling the distributed framework [#52640](https://github.com/pingcap/tidb/issues/52640) @[tangenta](https://github.com/tangenta) + - Fix the issue that adding indexes concurrently reports the error `no such file or directory` [#52475](https://github.com/pingcap/tidb/issues/52475) @[tangenta](https://github.com/tangenta) + - Fix the issue that the temporary data cannot be cleaned up after adding indexes fails [#52639](https://github.com/pingcap/tidb/issues/52639) @[lance6716](https://github.com/lance6716) + - Fix the issue that the metadata lock fails to prevent DDL operations from executing in the plan cache scenario [#51407](https://github.com/pingcap/tidb/issues/51407) @[wjhuang2016](https://github.com/wjhuang2016) + - Fix the issue that the `IMPORT INTO` operation gets stuck when processing importing tasks of a large amount of data [#52884](https://github.com/pingcap/tidb/issues/52884) @[lance6716](https://github.com/lance6716) + - Fix the issue that TiDB unexpectedly restarts when logging gRPC errors [#52179](https://github.com/pingcap/tidb/issues/52179) @[guo-shaoge](https://github.com/guo-shaoge) + - Fix the issue that IndexHashJoin outputs redundant data when calculating Anti Left Outer Semi Join [#52923](https://github.com/pingcap/tidb/issues/52923) @[yibin87](https://github.com/yibin87) + - Fix the incorrect result of the TopN operator in correlated subqueries [#52777](https://github.com/pingcap/tidb/issues/52777) @[yibin87](https://github.com/yibin87) - 修复了 HashJoin Probe 执行时间统计不精确的问题 [#52227](https://github.com/pingcap/tidb/issues/52227) @[windtalker](https://github.com/windtalker) - 修复了在分区裁剪模式为 static 情况下，使用 tablesample 返回错误结果的问题 [#52282](https://github.com/pingcap/tidb/issues/52282) @[tangenta](https://github.com/tangenta) - 修复 TTL 功能在非默认时区下的错误 [#51675](https://github.com/pingcap/tidb/issues/51675) @[lcwangchao](https://github.com/lcwangchao) From f78ee05baf868e7414a60c28919d2acc6fe4e25b Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 10 May 2024 18:38:08 +0800 Subject: [PATCH 40/55] Update releases/release-8.1.0.md Co-authored-by: Aolin --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 643606ba84f27..a7a462aa58a1a 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -370,7 +370,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that TiDB Lightning reports `no database selected` during data import due to incompatible SQL statements in the source files [#51800](https://github.com/pingcap/tidb/issues/51800) @[lance6716](https://github.com/lance6716) - Fix the issue that TiDB Lightning might print sensitive information to logs in server mode [#36374](https://github.com/pingcap/tidb/issues/36374) @[kennytm](https://github.com/kennytm) - Fix the issue that killing the PD Leader causes TiDB Lightning to report the `invalid store ID 0` error during data import [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - - Fix the issue that TiDB Lightning reports the `Unknown column in where clause` error when processing conflicting data using the `replace` method [#52886](https://github.com/pingcap/tidb/issues/52886) @[lyzx2001](https://github.com/lyzx2001) + - Fix the issue that TiDB Lightning reports the `Unknown column in where clause` error when processing conflicting data using the `replace` strategy [#52886](https://github.com/pingcap/tidb/issues/52886) @[lyzx2001](https://github.com/lyzx2001) - (dup): release-7.1.5.md > Bug fixes> Tools> TiDB Lightning - Fix the issue that TiDB Lightning panics when importing an empty table of Parquet format [#52518](https://github.com/pingcap/tidb/issues/52518) @[kennytm](https://github.com/kennytm) + Dumpling From 995d3c75ad8162f4df72843e8da29851277a7d10 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Sat, 11 May 2024 08:49:59 +0800 Subject: [PATCH 41/55] Update releases/release-8.1.0.md --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index a7a462aa58a1a..b90d89a23389c 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -341,7 +341,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - (dup): release-6.5.9.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that removing a log backup task after it is paused does not immediately restore the GC safepoint [#52082](https://github.com/pingcap/tidb/issues/52082) @[3pointer](https://github.com/3pointer) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix a rare issue that special event timing might cause the data loss in log backup [#16739](https://github.com/tikv/tikv/issues/16739) @[YuJuncen](https://github.com/YuJuncen) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that the global checkpoint of log backup is advanced ahead of the actual backup file write point due to TiKV restart, which might cause a small amount of backup data loss [#16809](https://github.com/tikv/tikv/issues/16809) @[YuJuncen](https://github.com/YuJuncen) - - Fix the issue that redundant concurrency related information appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) + - Fix the issue that redundant `--concurrency` related information appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) - Fix the issue that the Region fetched from PD does not have a Leader when restoring data by BR or importing data by TiDB Lightning in physical import mode [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - Fix the issue that after pausing, stopping, and rebuilding the log backup task, the task status is normal, but the checkpoint does not advance [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) - Fix the unstable test case `TestClearCache` [#51671](https://github.com/pingcap/tidb/issues/51671) @[zxc111](https://github.com/zxc111) From b58a0308225701e933e591835d85a697dfa6d8c3 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Sat, 11 May 2024 08:57:32 +0800 Subject: [PATCH 42/55] Update releases/release-8.1.0.md --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index b90d89a23389c..087c3686924c9 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -239,7 +239,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + TiCDC - - Improve memory stability during data recovery using redo log to reduce the probability of OOM [#10900](https://github.com/pingcap/tiflow/issues/10900) @[CharlesCheung96](https://github.com/CharlesCheung96) + - Improve memory stability during data recovery using redo logs to reduce the probability of OOM [#10900](https://github.com/pingcap/tiflow/issues/10900) @[CharlesCheung96](https://github.com/CharlesCheung96) - Significantly improve the stability of data replication in transaction conflict scenarios, with up to 10 times performance improvement [#10896](https://github.com/pingcap/tiflow/issues/10896) @[CharlesCheung96](https://github.com/CharlesCheung96) + TiDB Data Migration (DM) From 4a9f73d88af706fa77c4c1acf45e74ac9a8ec809 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Sat, 11 May 2024 09:02:09 +0800 Subject: [PATCH 43/55] Update releases/release-8.1.0.md --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 087c3686924c9..82b3c1374deea 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -341,7 +341,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - (dup): release-6.5.9.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that removing a log backup task after it is paused does not immediately restore the GC safepoint [#52082](https://github.com/pingcap/tidb/issues/52082) @[3pointer](https://github.com/3pointer) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix a rare issue that special event timing might cause the data loss in log backup [#16739](https://github.com/tikv/tikv/issues/16739) @[YuJuncen](https://github.com/YuJuncen) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that the global checkpoint of log backup is advanced ahead of the actual backup file write point due to TiKV restart, which might cause a small amount of backup data loss [#16809](https://github.com/tikv/tikv/issues/16809) @[YuJuncen](https://github.com/YuJuncen) - - Fix the issue that redundant `--concurrency` related information appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) + - Fix the issue that the redundant information related to `--concurrency` appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) - Fix the issue that the Region fetched from PD does not have a Leader when restoring data by BR or importing data by TiDB Lightning in physical import mode [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - Fix the issue that after pausing, stopping, and rebuilding the log backup task, the task status is normal, but the checkpoint does not advance [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) - Fix the unstable test case `TestClearCache` [#51671](https://github.com/pingcap/tidb/issues/51671) @[zxc111](https://github.com/zxc111) From 3d498670623e0e78aba64b1ad544cb378244f344 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Sat, 11 May 2024 10:02:51 +0800 Subject: [PATCH 44/55] Update releases/release-8.1.0.md --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 82b3c1374deea..17681d582a2f3 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -356,7 +356,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that the **Ownership history** panel in Grafana is unstable [#10796](https://github.com/pingcap/tiflow/issues/10796) @[hongyunyan](https://github.com/hongyunyan) - Fix the issue that restarting PD might cause the TiCDC node to restart with an error [#10799](https://github.com/pingcap/tiflow/issues/10799) @[3AceShowHand](https://github.com/3AceShowHand) - Fix the issue that long latency in the PD disk IO causes significant latency in data replication [#9054](https://github.com/pingcap/tiflow/issues/9054) @[asddongmen](https://github.com/asddongmen) - - Fix the issue that the default value is not set according to the correct time zone [#10931](https://github.com/pingcap/tiflow/issues/10931) @[3AceShowHand](https://github.com/3AceShowHand) + - Fix the issue that the default value of `TIMEZONE` type is not set according to the correct time zone [#10931](https://github.com/pingcap/tiflow/issues/10931) @[3AceShowHand](https://github.com/3AceShowHand) - Fix the issue that `DROP PRIMARY KEY` and `DROP UNIQUE KEY` statements are not replicated correctly [#10890](https://github.com/pingcap/tiflow/issues/10890) @[asddongmen](https://github.com/asddongmen) - (dup): release-7.1.5.md > Bug fixes> Tools> TiCDC - Fix the issue that TiCDC fails to execute the `Exchange Partition ... With Validation` DDL downstream after it is written upstream, causing the changefeed to get stuck [#10859](https://github.com/pingcap/tiflow/issues/10859) @[hongyunyan](https://github.com/hongyunyan) From b70f8651a9bb99b82a470196bb833168a25041d2 Mon Sep 17 00:00:00 2001 From: Aolin Date: Sat, 11 May 2024 10:37:09 +0800 Subject: [PATCH 45/55] Apply suggestions from code review --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 17681d582a2f3..dac4ade584106 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -210,9 +210,9 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Support adding multiple indexes concurrently in the ingest mode [#52596](https://github.com/pingcap/tidb/issues/52596) @[lance6716](https://github.com/lance6716) - Support configuring the system variable `tidb_service_scope` with various values, enhancing the utilization of the Distributed eXecution Framework [#52441](https://github.com/pingcap/tidb/issues/52441) @[ywqzzy](https://github.com/ywqzzy) - Enhance the handling of DNF items that are always `false` by directly ignoring such filter conditions, thus avoiding unnecessary full table scans [#40997](https://github.com/pingcap/tidb/issues/40997) @[hi-rustin](https://github.com/hi-rustin) - - Do not support setting the system variable `tidb_auto_analyze_ratio` to `0` [#51582](https://github.com/pingcap/tidb/issues/51582) @[hi-rustin](https://github.com/hi-rustin) - Support using Optimizer Fix Controls to remove the limitation that the optimizer does not automatically choose Index Merge for a query when the optimizer can choose the single index scan method (other than full table scan) for the query [#52869](https://github.com/pingcap/tidb/issues/52869) @[time-and-fate](https://github.com/time-and-fate) - Add the `total_kv_read_wall_time` metric to the column `execution info` of Coprocessor operators [#28937](https://github.com/pingcap/tidb/issues/28937) @[cfzjywxk](https://github.com/cfzjywxk) + - Add the `RU (max)` metric on the Resource Control dashboard[#49318](https://github.com/pingcap/tidb/issues/49318) @[nolouch](https://github.com/nolouch) - (dup): release-7.1.5.md > Improvements> TiDB - Add a timeout mechanism for LDAP authentication to avoid the issue of resource lock (RLock) not being released in time [#51883](https://github.com/pingcap/tidb/issues/51883) @[YangKeao](https://github.com/YangKeao) + TiKV From 121b9185e8c47c21e485f64d812ffcaad2cd3183 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Sat, 11 May 2024 14:21:23 +0800 Subject: [PATCH 46/55] Add translation for 7 tidb bug fixes --- releases/release-8.1.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index dac4ade584106..f8d4c0d2ac541 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -296,13 +296,13 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that TiDB unexpectedly restarts when logging gRPC errors [#52179](https://github.com/pingcap/tidb/issues/52179) @[guo-shaoge](https://github.com/guo-shaoge) - Fix the issue that IndexHashJoin outputs redundant data when calculating Anti Left Outer Semi Join [#52923](https://github.com/pingcap/tidb/issues/52923) @[yibin87](https://github.com/yibin87) - Fix the incorrect result of the TopN operator in correlated subqueries [#52777](https://github.com/pingcap/tidb/issues/52777) @[yibin87](https://github.com/yibin87) - - 修复了 HashJoin Probe 执行时间统计不精确的问题 [#52227](https://github.com/pingcap/tidb/issues/52227) @[windtalker](https://github.com/windtalker) - - 修复了在分区裁剪模式为 static 情况下，使用 tablesample 返回错误结果的问题 [#52282](https://github.com/pingcap/tidb/issues/52282) @[tangenta](https://github.com/tangenta) - - 修复 TTL 功能在非默认时区下的错误 [#51675](https://github.com/pingcap/tidb/issues/51675) @[lcwangchao](https://github.com/lcwangchao) - - 修复监控面板中链接数的计算和显示错误 [#51889](https://github.com/pingcap/tidb/issues/51889) @[YangKeao](https://github.com/YangKeao) - - 修复回滚改写分区 DDL 任务时，状态卡住的问题 [#51090](https://github.com/pingcap/tidb/issues/51090) @[lcwangchao](https://github.com/lcwangchao) - - 修复 lighting 导入后 TiDB 意外重启 [#52827](https://github.com/pingcap/tidb/issues/52827) @[hawkingrei](https://github.com/hawkingrei) - - 修复 TiDB `explain analyze` 结果中 `max_remote_stream` 显示不正确的问题 [#52646](https://github.com/pingcap/tidb/issues/52646) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the inaccurate execution time statistics of HashJoin Probe [#52227](https://github.com/pingcap/tidb/issues/52227) @[windtalker](https://github.com/windtalker) + - Fix the issue that using `TABLESAMPLE` returns incorrect results in static partition pruning mode [#52282](https://github.com/pingcap/tidb/issues/52282) @[tangenta](https://github.com/tangenta) + - Fix the error of TTL in non-default time zone [#51675](https://github.com/pingcap/tidb/issues/51675) @[lcwangchao](https://github.com/lcwangchao) + - Fix the incorrect calculation and display of the number of connections in the monitoring panel [#51889](https://github.com/pingcap/tidb/issues/51889) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the status gets stuck when rolling back the partition DDL tasks [#51090](https://github.com/pingcap/tidb/issues/51090) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that TiDB unexpectedly restarts after importing data using TiDB Lightning [#52827](https://github.com/pingcap/tidb/issues/52827) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue of incorrect `max_remote_stream` in the `explain analyze` result [#52646](https://github.com/pingcap/tidb/issues/52646) @[JaySon-Huang](https://github.com/JaySon-Huang) - (dup): release-6.5.9.md > Bug fixes> TiDB - Fix the issue that querying the `TIDB_HOT_REGIONS` table might incorrectly return `INFORMATION_SCHEMA` tables [#50810](https://github.com/pingcap/tidb/issues/50810) @[Defined2014](https://github.com/Defined2014) - (dup): release-7.1.5.md > Bug fixes> TiDB - Fix the issue that the `EXPLAIN` statement might display incorrect column IDs in the result when statistics for certain columns are not fully loaded [#52207](https://github.com/pingcap/tidb/issues/52207) @[time-and-fate](https://github.com/time-and-fate) - (dup): release-6.5.9.md > Bug fixes> TiDB - Fix the issue that the type returned by the `IFNULL` function is inconsistent with MySQL [#51765](https://github.com/pingcap/tidb/issues/51765) @[YangKeao](https://github.com/YangKeao) From f83f1b3dfdc1a5b56a20fe2786ad3eb7649472f8 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Sat, 11 May 2024 14:24:14 +0800 Subject: [PATCH 47/55] Remove extra spaces --- releases/release-8.1.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index f8d4c0d2ac541..eef89c42a524a 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -342,8 +342,8 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix a rare issue that special event timing might cause the data loss in log backup [#16739](https://github.com/tikv/tikv/issues/16739) @[YuJuncen](https://github.com/YuJuncen) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that the global checkpoint of log backup is advanced ahead of the actual backup file write point due to TiKV restart, which might cause a small amount of backup data loss [#16809](https://github.com/tikv/tikv/issues/16809) @[YuJuncen](https://github.com/YuJuncen) - Fix the issue that the redundant information related to `--concurrency` appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) - - Fix the issue that the Region fetched from PD does not have a Leader when restoring data by BR or importing data by TiDB Lightning in physical import mode [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - - Fix the issue that after pausing, stopping, and rebuilding the log backup task, the task status is normal, but the checkpoint does not advance [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) + - Fix the issue that the Region fetched from PD does not have a Leader when restoring data by BR or importing data by TiDB Lightning in physical import mode [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) + - Fix the issue that after pausing, stopping, and rebuilding the log backup task, the task status is normal, but the checkpoint does not advance [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) - Fix the unstable test case `TestClearCache` [#51671](https://github.com/pingcap/tidb/issues/51671) @[zxc111](https://github.com/zxc111) - Fix the unstable test case `TestGetMergeRegionSizeAndCount` [#52095](https://github.com/pingcap/tidb/issues/52095) @[3pointer](https://github.com/3pointer) - Fix the unstable integration test `br_tikv_outage` [#52673](https://github.com/pingcap/tidb/issues/52673) @[Leavrth](https://github.com/Leavrth) From 72ce3f33e8a32ad4e5c7fd1202d542080f62925b Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Sat, 11 May 2024 14:27:05 +0800 Subject: [PATCH 48/55] Replace a PR link with its issue link --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index eef89c42a524a..4c14ad52899d0 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -293,7 +293,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that the temporary data cannot be cleaned up after adding indexes fails [#52639](https://github.com/pingcap/tidb/issues/52639) @[lance6716](https://github.com/lance6716) - Fix the issue that the metadata lock fails to prevent DDL operations from executing in the plan cache scenario [#51407](https://github.com/pingcap/tidb/issues/51407) @[wjhuang2016](https://github.com/wjhuang2016) - Fix the issue that the `IMPORT INTO` operation gets stuck when processing importing tasks of a large amount of data [#52884](https://github.com/pingcap/tidb/issues/52884) @[lance6716](https://github.com/lance6716) - - Fix the issue that TiDB unexpectedly restarts when logging gRPC errors [#52179](https://github.com/pingcap/tidb/issues/52179) @[guo-shaoge](https://github.com/guo-shaoge) + - Fix the issue that TiDB unexpectedly restarts when logging gRPC errors [#51301](https://github.com/pingcap/tidb/issues/51301) @[guo-shaoge](https://github.com/guo-shaoge) - Fix the issue that IndexHashJoin outputs redundant data when calculating Anti Left Outer Semi Join [#52923](https://github.com/pingcap/tidb/issues/52923) @[yibin87](https://github.com/yibin87) - Fix the incorrect result of the TopN operator in correlated subqueries [#52777](https://github.com/pingcap/tidb/issues/52777) @[yibin87](https://github.com/yibin87) - Fix the inaccurate execution time statistics of HashJoin Probe [#52227](https://github.com/pingcap/tidb/issues/52227) @[windtalker](https://github.com/windtalker) From 6094c7470ca7e0f0af0cfa267924400f8f6bf6a3 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Sat, 11 May 2024 14:54:37 +0800 Subject: [PATCH 49/55] Update format and issue link --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index eef89c42a524a..0c1f66b6a93d5 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -296,7 +296,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that TiDB unexpectedly restarts when logging gRPC errors [#52179](https://github.com/pingcap/tidb/issues/52179) @[guo-shaoge](https://github.com/guo-shaoge) - Fix the issue that IndexHashJoin outputs redundant data when calculating Anti Left Outer Semi Join [#52923](https://github.com/pingcap/tidb/issues/52923) @[yibin87](https://github.com/yibin87) - Fix the incorrect result of the TopN operator in correlated subqueries [#52777](https://github.com/pingcap/tidb/issues/52777) @[yibin87](https://github.com/yibin87) - - Fix the inaccurate execution time statistics of HashJoin Probe [#52227](https://github.com/pingcap/tidb/issues/52227) @[windtalker](https://github.com/windtalker) + - Fix the inaccurate execution time statistics of HashJoin probe [#52222](https://github.com/pingcap/tidb/issues/52222) @[windtalker](https://github.com/windtalker) - Fix the issue that using `TABLESAMPLE` returns incorrect results in static partition pruning mode [#52282](https://github.com/pingcap/tidb/issues/52282) @[tangenta](https://github.com/tangenta) - Fix the error of TTL in non-default time zone [#51675](https://github.com/pingcap/tidb/issues/51675) @[lcwangchao](https://github.com/lcwangchao) - Fix the incorrect calculation and display of the number of connections in the monitoring panel [#51889](https://github.com/pingcap/tidb/issues/51889) @[YangKeao](https://github.com/YangKeao) From c733ff86458aed65af8c6382c808854ab74359cd Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Mon, 13 May 2024 11:54:03 +0800 Subject: [PATCH 50/55] Apply suggestions from hfxsd's review Co-authored-by: xixirangrang --- releases/release-8.1.0.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 06e82c3395bda..0d95e14be6db9 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -288,21 +288,20 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that adding indexes fails when the TiDB network is isolated [#51846](https://github.com/pingcap/tidb/issues/51846) @[ywqzzy](https://github.com/ywqzzy) - Fix the issue that adding an index with the same name after renaming an index causes an error [#51431](https://github.com/pingcap/tidb/issues/51431) @[lance6716](https://github.com/lance6716) - Fix the issue of inconsistent data indexes caused by cluster upgrade during adding indexes [#52411](https://github.com/pingcap/tidb/issues/52411) @[tangenta](https://github.com/tangenta) - - Fix the issue that adding indexes to large tables fails after enabling the distributed framework [#52640](https://github.com/pingcap/tidb/issues/52640) @[tangenta](https://github.com/tangenta) + - Fix the issue that adding indexes to large tables fails after enabling the Distributed eXecution Framework (DXF) [#52640](https://github.com/pingcap/tidb/issues/52640) @[tangenta](https://github.com/tangenta) - Fix the issue that adding indexes concurrently reports the error `no such file or directory` [#52475](https://github.com/pingcap/tidb/issues/52475) @[tangenta](https://github.com/tangenta) - Fix the issue that the temporary data cannot be cleaned up after adding indexes fails [#52639](https://github.com/pingcap/tidb/issues/52639) @[lance6716](https://github.com/lance6716) - Fix the issue that the metadata lock fails to prevent DDL operations from executing in the plan cache scenario [#51407](https://github.com/pingcap/tidb/issues/51407) @[wjhuang2016](https://github.com/wjhuang2016) - - Fix the issue that the `IMPORT INTO` operation gets stuck when processing importing tasks of a large amount of data [#52884](https://github.com/pingcap/tidb/issues/52884) @[lance6716](https://github.com/lance6716) + - Fix the issue that the `IMPORT INTO` operation gets stuck when importing a large amount of data [#52884](https://github.com/pingcap/tidb/issues/52884) @[lance6716](https://github.com/lance6716) - Fix the issue that TiDB unexpectedly restarts when logging gRPC errors [#51301](https://github.com/pingcap/tidb/issues/51301) @[guo-shaoge](https://github.com/guo-shaoge) - Fix the issue that IndexHashJoin outputs redundant data when calculating Anti Left Outer Semi Join [#52923](https://github.com/pingcap/tidb/issues/52923) @[yibin87](https://github.com/yibin87) - Fix the incorrect result of the TopN operator in correlated subqueries [#52777](https://github.com/pingcap/tidb/issues/52777) @[yibin87](https://github.com/yibin87) - Fix the inaccurate execution time statistics of HashJoin probe [#52222](https://github.com/pingcap/tidb/issues/52222) @[windtalker](https://github.com/windtalker) - - Fix the issue that using `TABLESAMPLE` returns incorrect results in static partition pruning mode [#52282](https://github.com/pingcap/tidb/issues/52282) @[tangenta](https://github.com/tangenta) - - Fix the error of TTL in non-default time zone [#51675](https://github.com/pingcap/tidb/issues/51675) @[lcwangchao](https://github.com/lcwangchao) - - Fix the incorrect calculation and display of the number of connections in the monitoring panel [#51889](https://github.com/pingcap/tidb/issues/51889) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the status gets stuck when rolling back the partition DDL tasks [#51090](https://github.com/pingcap/tidb/issues/51090) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that TiDB unexpectedly restarts after importing data using TiDB Lightning [#52827](https://github.com/pingcap/tidb/issues/52827) @[hawkingrei](https://github.com/hawkingrei) - - Fix the issue of incorrect `max_remote_stream` in the `explain analyze` result [#52646](https://github.com/pingcap/tidb/issues/52646) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the issue that using `TABLESAMPLE` returns incorrect results in static partition pruning mode (`tidb_partition_prune_mode='static'`) [#52282](https://github.com/pingcap/tidb/issues/52282) @[tangenta](https://github.com/tangenta) + - Fix the issue that TTL is deviated by 1 hour in daylight saving time [#51675](https://github.com/pingcap/tidb/issues/51675) @[lcwangchao](https://github.com/lcwangchao) + - Fix the incorrect calculation and display of the number of connections (Connection Count) on the TiDB Dashboard Monitoring page [#51889](https://github.com/pingcap/tidb/issues/51889) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the status gets stuck when rolling back the partition DDL tasks [#51090](https://github.com/pingcap/tidb/issues/51090) @[jiyfhust](https://github.com/jiyfhust) + - Fix the issue that the value of `max_remote_stream` is incorrect when executing `EXPLAIN ANALYZE` [#52646](https://github.com/pingcap/tidb/issues/52646) @[JaySon-Huang](https://github.com/JaySon-Huang) - (dup): release-6.5.9.md > Bug fixes> TiDB - Fix the issue that querying the `TIDB_HOT_REGIONS` table might incorrectly return `INFORMATION_SCHEMA` tables [#50810](https://github.com/pingcap/tidb/issues/50810) @[Defined2014](https://github.com/Defined2014) - (dup): release-7.1.5.md > Bug fixes> TiDB - Fix the issue that the `EXPLAIN` statement might display incorrect column IDs in the result when statistics for certain columns are not fully loaded [#52207](https://github.com/pingcap/tidb/issues/52207) @[time-and-fate](https://github.com/time-and-fate) - (dup): release-6.5.9.md > Bug fixes> TiDB - Fix the issue that the type returned by the `IFNULL` function is inconsistent with MySQL [#51765](https://github.com/pingcap/tidb/issues/51765) @[YangKeao](https://github.com/YangKeao) From 7aa8f2f33fb1526f79c33169bd38b9f570aff66d Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Mon, 13 May 2024 11:59:09 +0800 Subject: [PATCH 51/55] Make description more specific Co-authored-by: xixirangrang --- releases/release-8.1.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 0d95e14be6db9..2f12185503e34 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -99,7 +99,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, * Support managing queries that consume more resources than expected (GA) [#43691](https://github.com/pingcap/tidb/issues/43691) @[nolouch](https://github.com/nolouch) **tw@lilin90** - Sudden SQL query performance problems can cause a decline in overall database performance, which is the most common challenge to database stability. The reasons for these problems are diverse, such as untested new SQL statements, drastic changes in data volume, and sudden changes in execution plans. These problems are difficult to avoid completely at the source. In earlier versions, TiDB has added the capability to manage queries that consume more resources than expected to quickly reduce the impact of sudden query performance problems. This feature becomes generally available in v8.1.0. + Sudden SQL query performance problems can cause a decline in overall database performance, which is the most common challenge to database stability. The reasons for these problems are diverse, such as untested new SQL statements, drastic changes in data volume, and sudden changes in execution plans. These problems are difficult to avoid completely at the source. TiDB v7.2.0 has introduced the capability to manage queries that consume more resources than expected to quickly reduce the impact of sudden query performance problems. This feature becomes generally available in v8.1.0. You can set the maximum execution time for a query in a resource group. When the execution time of a query exceeds the set value, the priority of the query is automatically reduced or the query is canceled. You can also set immediately matching identified queries through text or execution plans within a period of time, to avoid excessive resource consumption during the identification phase when the concurrency of problematic queries is too high. From 43c112c06581e7adbb7abe321b981775853758ed Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 13 May 2024 16:48:00 +0800 Subject: [PATCH 52/55] Apply suggestions from code review Co-authored-by: Lilian Lee --- releases/release-8.1.0.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 2f12185503e34..04cda5a5f97c2 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -190,6 +190,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, | TiCDC | [`security.mtls`](/ticdc/ticdc-server-config.md#cdc-server-configuration-file-parameters) | Newly added | Controls whether to enable the TLS client authentication. The default value is `false`. | | TiCDC | [`sink.debezium.output-old-value`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | Newly added | Controls whether to output the value before the row data changes. The default value is true. When it is disabled, the UPDATE event does not output the "before" field. | | TiCDC | [`sink.open.output-old-value`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | Newly added | Controls whether to output the value before the row data changes. The default value is true. When it is disabled, the UPDATE event does not output the "p" field. | +| TiDB| [`concurrently-init-stats`](/tidb-configuration-file.md#concurrently-init-stats-new-in-v810-and-v752) | Newly added | Controls whether to initialize statistics concurrently during TiDB startup. The default value is `false`. | ### Others @@ -340,8 +341,8 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - (dup): release-6.5.9.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that removing a log backup task after it is paused does not immediately restore the GC safepoint [#52082](https://github.com/pingcap/tidb/issues/52082) @[3pointer](https://github.com/3pointer) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix a rare issue that special event timing might cause the data loss in log backup [#16739](https://github.com/tikv/tikv/issues/16739) @[YuJuncen](https://github.com/YuJuncen) - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that the global checkpoint of log backup is advanced ahead of the actual backup file write point due to TiKV restart, which might cause a small amount of backup data loss [#16809](https://github.com/tikv/tikv/issues/16809) @[YuJuncen](https://github.com/YuJuncen) - - Fix the issue that the redundant information related to `--concurrency` appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) - - Fix the issue that the Region fetched from PD does not have a Leader when restoring data by BR or importing data by TiDB Lightning in physical import mode [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) + - Fix the issue that the confusing information related to `--concurrency` appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) + - Fix the issue fetched from PD that the Region does not have a Leader when restoring data using BR or importing data using TiDB Lightning in physical import mode [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - Fix the issue that after pausing, stopping, and rebuilding the log backup task, the task status is normal, but the checkpoint does not advance [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) - Fix the unstable test case `TestClearCache` [#51671](https://github.com/pingcap/tidb/issues/51671) @[zxc111](https://github.com/zxc111) - Fix the unstable test case `TestGetMergeRegionSizeAndCount` [#52095](https://github.com/pingcap/tidb/issues/52095) @[3pointer](https://github.com/3pointer) @@ -350,11 +351,11 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + TiCDC - - Fix the issue that calling the API (/api/v2/owner/resign) that evicts the TiCDC owner node causes the TiCDC task to restart unexpectedly [#10781](https://github.com/pingcap/tiflow/issues/10781) @[sdojjy](https://github.com/sdojjy) + - Fix the issue that calling the API (`/api/v2/owner/resign`) that evicts the TiCDC owner node causes the TiCDC task to restart unexpectedly [#10781](https://github.com/pingcap/tiflow/issues/10781) @[sdojjy](https://github.com/sdojjy) - Fix the issue that when the downstream Pulsar is stopped, removing the changefeed causes the normal TiCDC process to get stuck, which causes other changefeed processes to get stuck [#10629](https://github.com/pingcap/tiflow/issues/10629) @[asddongmen](https://github.com/asddongmen) - Fix the issue that the **Ownership history** panel in Grafana is unstable [#10796](https://github.com/pingcap/tiflow/issues/10796) @[hongyunyan](https://github.com/hongyunyan) - Fix the issue that restarting PD might cause the TiCDC node to restart with an error [#10799](https://github.com/pingcap/tiflow/issues/10799) @[3AceShowHand](https://github.com/3AceShowHand) - - Fix the issue that long latency in the PD disk IO causes significant latency in data replication [#9054](https://github.com/pingcap/tiflow/issues/9054) @[asddongmen](https://github.com/asddongmen) + - Fix the issue that high latency in the PD disk I/O causes severe latency in data replication [#9054](https://github.com/pingcap/tiflow/issues/9054) @[asddongmen](https://github.com/asddongmen) - Fix the issue that the default value of `TIMEZONE` type is not set according to the correct time zone [#10931](https://github.com/pingcap/tiflow/issues/10931) @[3AceShowHand](https://github.com/3AceShowHand) - Fix the issue that `DROP PRIMARY KEY` and `DROP UNIQUE KEY` statements are not replicated correctly [#10890](https://github.com/pingcap/tiflow/issues/10890) @[asddongmen](https://github.com/asddongmen) - (dup): release-7.1.5.md > Bug fixes> Tools> TiCDC - Fix the issue that TiCDC fails to execute the `Exchange Partition ... With Validation` DDL downstream after it is written upstream, causing the changefeed to get stuck [#10859](https://github.com/pingcap/tiflow/issues/10859) @[hongyunyan](https://github.com/hongyunyan) From 33acf36050061d2ce7aa15a376ab5821b70d8b2a Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 14 May 2024 14:16:49 +0800 Subject: [PATCH 53/55] Apply suggestions from code review --- releases/release-8.1.0.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 04cda5a5f97c2..fd6c2b3ea61e6 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -218,7 +218,10 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + TiKV + - (dup): release-6.5.9.md > Improvements> TiKV - Avoid performing IO operations on snapshot files in raftstore threads to improve TiKV stability [#16564](https://github.com/tikv/tikv/issues/16564) @[Connor1996](https://github.com/Connor1996) + - Accelerate the shutdown speed of TiKV [#16680](https://github.com/tikv/tikv/issues/16680) @[LykxSassinator](https://github.com/LykxSassinator) + - Add metrics for memory usage per thread [#15927](https://github.com/tikv/tikv/issues/15927) @[Connor1996](https://github.com/Connor1996) + PD From 69f2ead908f90cba9ffbbaf66f56caa06a7d782e Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 14 May 2024 15:41:37 +0800 Subject: [PATCH 54/55] remove GitHub ID --- releases/release-8.1.0.md | 139 +++++++++++--------------------------- 1 file changed, 39 insertions(+), 100 deletions(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index fd6c2b3ea61e6..43f37491a0509 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -53,8 +53,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, Reliability and availability Global Sort (GA in v8.0.0) - The Global Sort feature aims to improve the stability and efficiency of IMPORT INTO and CREATE INDEX. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, each IMPORT INTO or CREATE INDEX statement now supports importing or adding indexes for up to 40 TiB of data. - + The Global Sort feature aims to improve the stability and efficiency of IMPORT INTO and CREATE INDEX. By globally sorting the data to be processed, this feature improves the stability, controllability, and scalability of data writing to TiKV, consequently enhancing the user experience and service quality of data import and index creation. With global sorting enabled, each IMPORT INTO or CREATE INDEX statement now supports importing or adding indexes for up to 40 TiB of data. Cross-database SQL binding (introduced in v7.6.0) @@ -97,7 +96,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### Reliability -* Support managing queries that consume more resources than expected (GA) [#43691](https://github.com/pingcap/tidb/issues/43691) @[nolouch](https://github.com/nolouch) **tw@lilin90** +* Support managing queries that consume more resources than expected (GA) [#43691](https://github.com/pingcap/tidb/issues/43691) @[nolouch](https://github.com/nolouch) Sudden SQL query performance problems can cause a decline in overall database performance, which is the most common challenge to database stability. The reasons for these problems are diverse, such as untested new SQL statements, drastic changes in data volume, and sudden changes in execution plans. These problems are difficult to avoid completely at the source. TiDB v7.2.0 has introduced the capability to manage queries that consume more resources than expected to quickly reduce the impact of sudden query performance problems. This feature becomes generally available in v8.1.0. @@ -111,7 +110,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### SQL -* Support using more expressions to set default column values when creating a table (GA) [#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala) **tw@hfxsd** +* Support using more expressions to set default column values when creating a table (GA) [#50936](https://github.com/pingcap/tidb/issues/50936) @[zimulala](https://github.com/zimulala) Before v8.0.0, when you create a table, the default value of a column is limited to strings, numbers, dates, and certain expressions. Starting from v8.0.0, you can use more expressions as the default column values. For example, you can set the default value of a column to `DATE_FORMAT`. This feature helps you meet more diverse requirements. In v8.1.0, this feature becomes GA. @@ -121,7 +120,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### DB operations -* Enable the TiDB Distributed eXecution Framework (DXF) by default to enhance the performance and stability of `ADD INDEX` or `IMPORT INTO` tasks in parallel execution [#52441](https://github.com/pingcap/tidb/issues/52441) @[D3Hunter](https://github.com/D3Hunter) **tw@qiancai** +* Enable the TiDB Distributed eXecution Framework (DXF) by default to enhance the performance and stability of `ADD INDEX` or `IMPORT INTO` tasks in parallel execution [#52441](https://github.com/pingcap/tidb/issues/52441) @[D3Hunter](https://github.com/D3Hunter) The DXF becomes generally available (GA) in v7.5.0, but it is disabled by default. This means that an `ADD INDEX` or `IMPORT INTO` task is executed only by one TiDB node by default. @@ -131,7 +130,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### Security -* Enhance TiDB log desensitization (GA) [#52364](https://github.com/pingcap/tidb/issues/52364) @[xhebox](https://github.com/xhebox) **tw@hfxsd** +* Enhance TiDB log desensitization (GA) [#52364](https://github.com/pingcap/tidb/issues/52364) @[xhebox](https://github.com/xhebox) The enhanced TiDB log desensitization supports removing sensitive data when users view logs, implemented by marking SQL text information in log files. You can control whether to mark log information to enable secure use of TiDB logs in different scenarios, enhancing the security and flexibility of using log desensitization. To use this feature, set the system variable `tidb_redact_log` to `MARKER`, and then the SQL text in TiDB's runtime logs are marked. In addition, you can use the `collect-log` subcommand on the TiDB server to remove marked sensitive data from the logs and display the logs in a secure manner. You can also remove all markers and get the normal logs. This feature became generally available in v8.1.0. @@ -139,7 +138,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ### Data migration -* Support the `IMPORT INTO ... FROM SELECT` syntax (GA) [#49883](https://github.com/pingcap/tidb/issues/49883) @[D3Hunter](https://github.com/D3Hunter) **tw@qiancai** +* Support the `IMPORT INTO ... FROM SELECT` syntax (GA) [#49883](https://github.com/pingcap/tidb/issues/49883) @[D3Hunter](https://github.com/D3Hunter) Before v8.0.0, importing query results into a target table could only be done using the `INSERT INTO ... SELECT` statement, which is relatively inefficient in some large dataset scenarios. In v8.0.0, TiDB introduces `IMPORT INTO ... FROM SELECT` as an experimental feature, which enables you to import the results of a `SELECT` query into an empty TiDB target table. It achieves up to 8 times the performance of `INSERT INTO ... SELECT` and significantly reduces the import time. In addition, you can use `IMPORT INTO ... FROM SELECT` to import historical data queried with [`AS OF TIMESTAMP`](/as-of-timestamp.md). @@ -147,7 +146,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, For more information, see [documentation](/sql-statements/sql-statement-import-into.md). -* TiDB Lightning simplifies conflict resolution strategies and supports handling conflicting data using the `replace` strategy (GA) [#51036](https://github.com/pingcap/tidb/issues/51036) @[lyzx2001](https://github.com/lyzx2001) **tw@qiancai** +* TiDB Lightning simplifies conflict resolution strategies and supports handling conflicting data using the `replace` strategy (GA) [#51036](https://github.com/pingcap/tidb/issues/51036) @[lyzx2001](https://github.com/lyzx2001) Before v8.0.0, TiDB Lightning has [one data conflict resolution strategy](/tidb-lightning/tidb-lightning-logical-import-mode-usage.md#conflict-detection) for the logical import mode and [two data conflict resolution strategies](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#conflict-detection) for the physical import mode, which are not easy to understand and configure. @@ -155,7 +154,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, For more information, see [documentation](/tidb-lightning/tidb-lightning-configuration.md). -* TiCDC supports client authentication [#10636](https://github.com/pingcap/tiflow/issues/10636) @[CharlesCheung96](https://github.com/CharlesCheung96) **tw@Oreoxmt** +* TiCDC supports client authentication [#10636](https://github.com/pingcap/tiflow/issues/10636) @[CharlesCheung96](https://github.com/CharlesCheung96) In v8.1.0, TiCDC supports client authentication when you are using the TiCDC CLI or OpenAPI. This feature enables you to configure TiCDC to require client authentication using client certificates, thereby establishing mutual Transport Layer Security (mTLS). Additionally, you can configure authentication based on TiDB username and password. @@ -167,10 +166,6 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, > > This section provides compatibility changes you need to know when you upgrade from v8.0.0 to the current version (v8.1.0). If you are upgrading from v7.6.0 or earlier versions to the current version, you might also need to check the compatibility changes introduced in intermediate versions. -### Behavior changes - -### MySQL compatibility - ### System variables | Variable name | Change type | Description | @@ -183,6 +178,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, | Configuration file | Configuration parameter | Change type | Description | | -------- | -------- | -------- | -------- | +| TiDB| [`concurrently-init-stats`](/tidb-configuration-file.md#concurrently-init-stats-new-in-v810-and-v752) | Newly added | Controls whether to initialize statistics concurrently during TiDB startup. The default value is `false`. | | TiDB Lightning | [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-configuration) | Modified | Starting from v8.1.0, there is no need to configure `conflict.max-record-rows` manually, because TiDB Lightning automatically assigns the value of `conflict.max-record-rows` with the value of `conflict.threshold`, regardless of the user input. `conflict.max-record-rows` will be deprecated in a future release. | | TiDB Lightning | [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) | Modified | Changes the default value from `9223372036854775807` to `10000` to quickly interrupt abnormal tasks so that you can make corresponding adjustments as soon as possible. This saves time and computational resources by avoiding the scenario where a large amount of conflicting data is discovered after the import, caused by abnormal data sources or incorrect table schema definitions. | | TiCDC | [`security.client-allowed-user`](/ticdc/ticdc-server-config.md#cdc-server-configuration-file-parameters) | Newly added | Lists the usernames that are allowed for client authentication. Authentication requests with usernames not in this list will be rejected. The default value is null. | @@ -190,11 +186,6 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, | TiCDC | [`security.mtls`](/ticdc/ticdc-server-config.md#cdc-server-configuration-file-parameters) | Newly added | Controls whether to enable the TLS client authentication. The default value is `false`. | | TiCDC | [`sink.debezium.output-old-value`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | Newly added | Controls whether to output the value before the row data changes. The default value is true. When it is disabled, the UPDATE event does not output the "before" field. | | TiCDC | [`sink.open.output-old-value`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | Newly added | Controls whether to output the value before the row data changes. The default value is true. When it is disabled, the UPDATE event does not output the "p" field. | -| TiDB| [`concurrently-init-stats`](/tidb-configuration-file.md#concurrently-init-stats-new-in-v810-and-v752) | Newly added | Controls whether to initialize statistics concurrently during TiDB startup. The default value is `false`. | - -### Others - -## Offline package changes ## Deprecated features @@ -205,7 +196,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, ## Improvements + TiDB - + - Improve the MySQL compatibility of foreign keys displayed in the output of `SHOW CREATE TABLE` [#51837](https://github.com/pingcap/tidb/issues/51837) @[negachov](https://github.com/negachov) - Improve the MySQL compatibility of expression default values displayed in the output of `SHOW CREATE TABLE` [#52939](https://github.com/pingcap/tidb/issues/52939) @[CbcWestwolf](https://github.com/CbcWestwolf) - Support adding multiple indexes concurrently in the ingest mode [#52596](https://github.com/pingcap/tidb/issues/52596) @[lance6716](https://github.com/lance6716) @@ -214,67 +205,36 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Support using Optimizer Fix Controls to remove the limitation that the optimizer does not automatically choose Index Merge for a query when the optimizer can choose the single index scan method (other than full table scan) for the query [#52869](https://github.com/pingcap/tidb/issues/52869) @[time-and-fate](https://github.com/time-and-fate) - Add the `total_kv_read_wall_time` metric to the column `execution info` of Coprocessor operators [#28937](https://github.com/pingcap/tidb/issues/28937) @[cfzjywxk](https://github.com/cfzjywxk) - Add the `RU (max)` metric on the Resource Control dashboard[#49318](https://github.com/pingcap/tidb/issues/49318) @[nolouch](https://github.com/nolouch) - - (dup): release-7.1.5.md > Improvements> TiDB - Add a timeout mechanism for LDAP authentication to avoid the issue of resource lock (RLock) not being released in time [#51883](https://github.com/pingcap/tidb/issues/51883) @[YangKeao](https://github.com/YangKeao) + - Add a timeout mechanism for LDAP authentication to avoid the issue of resource lock (RLock) not being released in time [#51883](https://github.com/pingcap/tidb/issues/51883) @[YangKeao](https://github.com/YangKeao) + TiKV - - - (dup): release-6.5.9.md > Improvements> TiKV - Avoid performing IO operations on snapshot files in raftstore threads to improve TiKV stability [#16564](https://github.com/tikv/tikv/issues/16564) @[Connor1996](https://github.com/Connor1996) + - Avoid performing IO operations on snapshot files in raftstore threads to improve TiKV stability [#16564](https://github.com/tikv/tikv/issues/16564) @[Connor1996](https://github.com/Connor1996) - Accelerate the shutdown speed of TiKV [#16680](https://github.com/tikv/tikv/issues/16680) @[LykxSassinator](https://github.com/LykxSassinator) - Add metrics for memory usage per thread [#15927](https://github.com/tikv/tikv/issues/15927) @[Connor1996](https://github.com/Connor1996) + PD - - - Optimize the logic for `OperatorController` to reduce the overhead of competition locks [#7897](https://github.com/tikv/pd/issues/7897) @[nolouch](https://github.com/nolouch) - -+ TiFlash - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + - Optimize the logic for `OperatorController` to reduce the overhead of competition locks [#7897](https://github.com/tikv/pd/issues/7897) @[nolouch](https://github.com/nolouch) + Tools + Backup & Restore (BR) - - - (dup): release-7.1.5.md > Improvements> Tools> Backup & Restore (BR) - Add PITR integration test cases to cover compatibility testing for log backup and adding index acceleration [#51987](https://github.com/pingcap/tidb/issues/51987) @[Leavrth](https://github.com/Leavrth) - - (dup): release-7.1.5.md > Improvements> Tools> Backup & Restore (BR) - Remove the invalid verification for active DDL jobs when log backup starts [#52733](https://github.com/pingcap/tidb/issues/52733) @[Leavrth](https://github.com/Leavrth) + + - Add PITR integration test cases to cover compatibility testing for log backup and adding index acceleration [#51987](https://github.com/pingcap/tidb/issues/51987) @[Leavrth](https://github.com/Leavrth) + - Remove the invalid verification for active DDL jobs when log backup starts [#52733](https://github.com/pingcap/tidb/issues/52733) @[Leavrth](https://github.com/Leavrth) - Add test cases to test compatibility between PITR and the acceleration of adding indexes feature [#51988](https://github.com/pingcap/tidb/issues/51988) @[Leavrth](https://github.com/Leavrth) - BR cleans up empty SST files during data recovery [#16005](https://github.com/tikv/tikv/issues/16005) @[Leavrth](https://github.com/Leavrth) + TiCDC - + - Improve memory stability during data recovery using redo logs to reduce the probability of OOM [#10900](https://github.com/pingcap/tiflow/issues/10900) @[CharlesCheung96](https://github.com/CharlesCheung96) - Significantly improve the stability of data replication in transaction conflict scenarios, with up to 10 times performance improvement [#10896](https://github.com/pingcap/tiflow/issues/10896) @[CharlesCheung96](https://github.com/CharlesCheung96) - + TiDB Data Migration (DM) - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + TiDB Lightning - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + Dumpling - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + TiUP - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + TiDB Binlog - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - ## Bug fixes + TiDB - + - Fix the issue that executing SQL statements containing tables with multi-valued indexes might return the `Can't find a proper physical plan for this query` error [#49438](https://github.com/pingcap/tidb/issues/49438) @[qw4990](https://github.com/qw4990) - Fix the issue that automatic statistics collection gets stuck after an OOM error occurs [#51993](https://github.com/pingcap/tidb/issues/51993) @[hi-rustin](https://github.com/hi-rustin) - Fix the issue that after using BR to restore a table that has no statistics, the statistics health of that table is still 100% [#29769](https://github.com/pingcap/tidb/issues/29769) @[winoros](https://github.com/winoros) @@ -287,7 +247,6 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that the nil pointer error occurs when you query `INFORMATION_SCHEMA.TIKV_REGION_STATUS` [#52013](https://github.com/pingcap/tidb/issues/52013) @[JmPotato](https://github.com/JmPotato) - Fix the incorrect error message displayed when an invalid default value is specified for a column [#51592](https://github.com/pingcap/tidb/issues/51592) @[danqixu](https://github.com/danqixu) - Fix the issue that adding indexes in the ingest mode might cause inconsistent data index in some corner cases [#51954](https://github.com/pingcap/tidb/issues/51954) @[lance6716](https://github.com/lance6716) - - Fix the issue that DDL operations get stuck when restoring a table with the foreign key [#51838](https://github.com/pingcap/tidb/issues/51838) @[YangKeao](https://github.com/YangKeao) - Fix the issue that adding indexes fails when the TiDB network is isolated [#51846](https://github.com/pingcap/tidb/issues/51846) @[ywqzzy](https://github.com/ywqzzy) - Fix the issue that adding an index with the same name after renaming an index causes an error [#51431](https://github.com/pingcap/tidb/issues/51431) @[lance6716](https://github.com/lance6716) @@ -306,18 +265,18 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the incorrect calculation and display of the number of connections (Connection Count) on the TiDB Dashboard Monitoring page [#51889](https://github.com/pingcap/tidb/issues/51889) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the status gets stuck when rolling back the partition DDL tasks [#51090](https://github.com/pingcap/tidb/issues/51090) @[jiyfhust](https://github.com/jiyfhust) - Fix the issue that the value of `max_remote_stream` is incorrect when executing `EXPLAIN ANALYZE` [#52646](https://github.com/pingcap/tidb/issues/52646) @[JaySon-Huang](https://github.com/JaySon-Huang) - - (dup): release-6.5.9.md > Bug fixes> TiDB - Fix the issue that querying the `TIDB_HOT_REGIONS` table might incorrectly return `INFORMATION_SCHEMA` tables [#50810](https://github.com/pingcap/tidb/issues/50810) @[Defined2014](https://github.com/Defined2014) - - (dup): release-7.1.5.md > Bug fixes> TiDB - Fix the issue that the `EXPLAIN` statement might display incorrect column IDs in the result when statistics for certain columns are not fully loaded [#52207](https://github.com/pingcap/tidb/issues/52207) @[time-and-fate](https://github.com/time-and-fate) - - (dup): release-6.5.9.md > Bug fixes> TiDB - Fix the issue that the type returned by the `IFNULL` function is inconsistent with MySQL [#51765](https://github.com/pingcap/tidb/issues/51765) @[YangKeao](https://github.com/YangKeao) - - (dup): release-7.1.5.md > Bug fixes> TiDB - Fix the issue that adding a unique index might cause TiDB to panic [#52312](https://github.com/pingcap/tidb/issues/52312) @[wjhuang2016](https://github.com/wjhuang2016) + - Fix the issue that querying the `TIDB_HOT_REGIONS` table might incorrectly return `INFORMATION_SCHEMA` tables [#50810](https://github.com/pingcap/tidb/issues/50810) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that the `EXPLAIN` statement might display incorrect column IDs in the result when statistics for certain columns are not fully loaded [#52207](https://github.com/pingcap/tidb/issues/52207) @[time-and-fate](https://github.com/time-and-fate) + - Fix the issue that the type returned by the `IFNULL` function is inconsistent with MySQL [#51765](https://github.com/pingcap/tidb/issues/51765) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that adding a unique index might cause TiDB to panic [#52312](https://github.com/pingcap/tidb/issues/52312) @[wjhuang2016](https://github.com/wjhuang2016) + TiKV - - (dup): release-7.1.5.md > Bug fixes> TiKV - Fix the issue that resolve-ts is blocked when a stale Region peer ignores the GC message [#16504](https://github.com/tikv/tikv/issues/16504) @[crazycs520](https://github.com/crazycs520) - - (dup): release-6.5.9.md > Bug fixes> TiKV - Fix the issue that inactive Write Ahead Logs (WALs) in RocksDB might corrupt data [#16705](https://github.com/tikv/tikv/issues/16705) @[Connor1996](https://github.com/Connor1996) + - Fix the issue that resolve-ts is blocked when a stale Region peer ignores the GC message [#16504](https://github.com/tikv/tikv/issues/16504) @[crazycs520](https://github.com/crazycs520) + - Fix the issue that inactive Write Ahead Logs (WALs) in RocksDB might corrupt data [#16705](https://github.com/tikv/tikv/issues/16705) @[Connor1996](https://github.com/Connor1996) + PD - + - Fix the issue that TSO might get stuck when toggling the PD microservice mode on and off [#7849](https://github.com/tikv/pd/issues/7849) @[JmPotato](https://github.com/JmPotato) - Fix the issue that the `State` monitoring metric for DR Auto-Sync does not show any data [#7974](https://github.com/tikv/pd/issues/7974) @[lhy1024](https://github.com/lhy1024) - Fix the issue that checking binary versions might cause PD panic [#7978](https://github.com/tikv/pd/issues/7978) @[JmPotato](https://github.com/JmPotato) @@ -327,23 +286,23 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that PD might panic when generating `Merge` schedules [#8049](https://github.com/tikv/pd/issues/8049) @[nolouch](https://github.com/nolouch) - Fix the panic issue caused by `GetAdditionalInfo` [#8079](https://github.com/tikv/pd/issues/8079) @[HuSharp](https://github.com/HuSharp) - Fix the issue that the `Filter target` monitoring metric for PD does not provide scatter range information [#8125](https://github.com/tikv/pd/issues/8125) @[HuSharp](https://github.com/HuSharp) - - (dup): release-6.5.9.md > Bug fixes> PD - Fix the issue that the query result of `SHOW CONFIG` includes the deprecated configuration item `trace-region-flow` [#7917](https://github.com/tikv/pd/issues/7917) @[rleungx](https://github.com/rleungx) - - (dup): release-6.5.9.md > Bug fixes> PD - Fix the issue that the scaling progress is not correctly displayed [#7726](https://github.com/tikv/pd/issues/7726) @[CabinfeverB](https://github.com/CabinfeverB) + - Fix the issue that the query result of `SHOW CONFIG` includes the deprecated configuration item `trace-region-flow` [#7917](https://github.com/tikv/pd/issues/7917) @[rleungx](https://github.com/rleungx) + - Fix the issue that the scaling progress is not correctly displayed [#7726](https://github.com/tikv/pd/issues/7726) @[CabinfeverB](https://github.com/CabinfeverB) + TiFlash - - - (dup): release-6.5.9.md > Bug fixes> TiFlash - Fix the issue that TiFlash might panic when you insert data to columns with invalid default values in non-strict `sql_mode` [#8803](https://github.com/pingcap/tiflash/issues/8803) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + + - Fix the issue that TiFlash might panic when you insert data to columns with invalid default values in non-strict `sql_mode` [#8803](https://github.com/pingcap/tiflash/issues/8803) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) - Fix the issue that TiFlash might return transiently incorrect results in high-concurrency read scenarios [#8845](https://github.com/pingcap/tiflash/issues/8845) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix the issue that in the disaggregated storage and compute architecture, the disk `used_size` metric displayed in Grafana is incorrect after you modify the value of the `storage.remote.cache.capacity` configuration item for TiFlash compute nodes [#8920](https://github.com/pingcap/tiflash/issues/8920) @[JinheLin](https://github.com/JinheLin) + Tools + Backup & Restore (BR) - - - (dup): release-6.5.9.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that BR could not back up the `AUTO_RANDOM` ID allocation progress in a union clustered index that contains an `AUTO_RANDOM` column [#52255](https://github.com/pingcap/tidb/issues/52255) @[Leavrth](https://github.com/Leavrth) - - (dup): release-6.5.9.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that removing a log backup task after it is paused does not immediately restore the GC safepoint [#52082](https://github.com/pingcap/tidb/issues/52082) @[3pointer](https://github.com/3pointer) - - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix a rare issue that special event timing might cause the data loss in log backup [#16739](https://github.com/tikv/tikv/issues/16739) @[YuJuncen](https://github.com/YuJuncen) - - (dup): release-7.1.5.md > Bug fixes> Tools> Backup & Restore (BR) - Fix the issue that the global checkpoint of log backup is advanced ahead of the actual backup file write point due to TiKV restart, which might cause a small amount of backup data loss [#16809](https://github.com/tikv/tikv/issues/16809) @[YuJuncen](https://github.com/YuJuncen) + + - Fix the issue that BR could not back up the `AUTO_RANDOM` ID allocation progress in a union clustered index that contains an `AUTO_RANDOM` column [#52255](https://github.com/pingcap/tidb/issues/52255) @[Leavrth](https://github.com/Leavrth) + - Fix the issue that removing a log backup task after it is paused does not immediately restore the GC safepoint [#52082](https://github.com/pingcap/tidb/issues/52082) @[3pointer](https://github.com/3pointer) + - Fix a rare issue that special event timing might cause the data loss in log backup [#16739](https://github.com/tikv/tikv/issues/16739) @[YuJuncen](https://github.com/YuJuncen) + - Fix the issue that the global checkpoint of log backup is advanced ahead of the actual backup file write point due to TiKV restart, which might cause a small amount of backup data loss [#16809](https://github.com/tikv/tikv/issues/16809) @[YuJuncen](https://github.com/YuJuncen) - Fix the issue that the confusing information related to `--concurrency` appears in the log during a full backup [#50837](https://github.com/pingcap/tidb/issues/50837) @[BornChanger](https://github.com/BornChanger) - Fix the issue fetched from PD that the Region does not have a Leader when restoring data using BR or importing data using TiDB Lightning in physical import mode [#51124](https://github.com/pingcap/tidb/issues/51124) [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - Fix the issue that after pausing, stopping, and rebuilding the log backup task, the task status is normal, but the checkpoint does not advance [#53047](https://github.com/pingcap/tidb/issues/53047) @[RidRisR](https://github.com/RidRisR) @@ -353,7 +312,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that the test case `TestGetTSWithRetry` takes too long to execute [#52547](https://github.com/pingcap/tidb/issues/52547) @[Leavrth](https://github.com/Leavrth) + TiCDC - + - Fix the issue that calling the API (`/api/v2/owner/resign`) that evicts the TiCDC owner node causes the TiCDC task to restart unexpectedly [#10781](https://github.com/pingcap/tiflow/issues/10781) @[sdojjy](https://github.com/sdojjy) - Fix the issue that when the downstream Pulsar is stopped, removing the changefeed causes the normal TiCDC process to get stuck, which causes other changefeed processes to get stuck [#10629](https://github.com/pingcap/tiflow/issues/10629) @[asddongmen](https://github.com/asddongmen) - Fix the issue that the **Ownership history** panel in Grafana is unstable [#10796](https://github.com/pingcap/tiflow/issues/10796) @[hongyunyan](https://github.com/hongyunyan) @@ -361,35 +320,15 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, - Fix the issue that high latency in the PD disk I/O causes severe latency in data replication [#9054](https://github.com/pingcap/tiflow/issues/9054) @[asddongmen](https://github.com/asddongmen) - Fix the issue that the default value of `TIMEZONE` type is not set according to the correct time zone [#10931](https://github.com/pingcap/tiflow/issues/10931) @[3AceShowHand](https://github.com/3AceShowHand) - Fix the issue that `DROP PRIMARY KEY` and `DROP UNIQUE KEY` statements are not replicated correctly [#10890](https://github.com/pingcap/tiflow/issues/10890) @[asddongmen](https://github.com/asddongmen) - - (dup): release-7.1.5.md > Bug fixes> Tools> TiCDC - Fix the issue that TiCDC fails to execute the `Exchange Partition ... With Validation` DDL downstream after it is written upstream, causing the changefeed to get stuck [#10859](https://github.com/pingcap/tiflow/issues/10859) @[hongyunyan](https://github.com/hongyunyan) - - + TiDB Data Migration (DM) - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + - Fix the issue that TiCDC fails to execute the `Exchange Partition ... With Validation` DDL downstream after it is written upstream, causing the changefeed to get stuck [#10859](https://github.com/pingcap/tiflow/issues/10859) @[hongyunyan](https://github.com/hongyunyan) + TiDB Lightning - + - Fix the issue that TiDB Lightning reports `no database selected` during data import due to incompatible SQL statements in the source files [#51800](https://github.com/pingcap/tidb/issues/51800) @[lance6716](https://github.com/lance6716) - Fix the issue that TiDB Lightning might print sensitive information to logs in server mode [#36374](https://github.com/pingcap/tidb/issues/36374) @[kennytm](https://github.com/kennytm) - Fix the issue that killing the PD Leader causes TiDB Lightning to report the `invalid store ID 0` error during data import [#50501](https://github.com/pingcap/tidb/issues/50501) @[Leavrth](https://github.com/Leavrth) - Fix the issue that TiDB Lightning reports the `Unknown column in where clause` error when processing conflicting data using the `replace` strategy [#52886](https://github.com/pingcap/tidb/issues/52886) @[lyzx2001](https://github.com/lyzx2001) - - (dup): release-7.1.5.md > Bug fixes> Tools> TiDB Lightning - Fix the issue that TiDB Lightning panics when importing an empty table of Parquet format [#52518](https://github.com/pingcap/tidb/issues/52518) @[kennytm](https://github.com/kennytm) - - + Dumpling - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + TiUP - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + TiDB Binlog - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + - Fix the issue that TiDB Lightning panics when importing an empty table of Parquet format [#52518](https://github.com/pingcap/tidb/issues/52518) @[kennytm](https://github.com/kennytm) ## Performance test @@ -407,4 +346,4 @@ We would like to thank the following contributors from the TiDB community: - [negachov](https://github.com/negachov) (First-time contributor) - [testwill](https://github.com/testwill) - [yzhan1](https://github.com/yzhan1) (First-time contributor) -- [zxc111](https://github.com/zxc111) (First-time contributor) \ No newline at end of file +- [zxc111](https://github.com/zxc111) (First-time contributor) From c347efecea439f766e6d8990213f00127e645c90 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 15 May 2024 14:31:00 +0800 Subject: [PATCH 55/55] Apply suggestions from code review --- releases/release-8.1.0.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 43f37491a0509..897569cc2965a 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -292,8 +292,9 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, + TiFlash - Fix the issue that TiFlash might panic when you insert data to columns with invalid default values in non-strict `sql_mode` [#8803](https://github.com/pingcap/tiflash/issues/8803) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) - - Fix the issue that TiFlash might return transiently incorrect results in high-concurrency read scenarios [#8845](https://github.com/pingcap/tiflash/issues/8845) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the issue that TiFlash might return transiently incorrect results in high-concurrency read scenarios [#8845](https://github.com/pingcap/tiflash/issues/8845) @[JinheLin](https://github.com/JinheLin) - Fix the issue that in the disaggregated storage and compute architecture, the disk `used_size` metric displayed in Grafana is incorrect after you modify the value of the `storage.remote.cache.capacity` configuration item for TiFlash compute nodes [#8920](https://github.com/pingcap/tiflash/issues/8920) @[JinheLin](https://github.com/JinheLin) + - Fix the issue that TiFlash metadata might become corrupted and cause the process to panic when upgrading a cluster from a version earlier than v6.5.0 to v6.5.0 or later [#9039](https://github.com/pingcap/tiflash/issues/9039) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools