From 7d7b98cc5c9911b067e528e7dc68ca07b5dbe923 Mon Sep 17 00:00:00 2001 From: Frank945946 Date: Mon, 11 Nov 2024 16:39:06 +0800 Subject: [PATCH] Update tidb-roadmap.md --- tidb-roadmap.md | 387 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 349 insertions(+), 38 deletions(-) diff --git a/tidb-roadmap.md b/tidb-roadmap.md index 490d074a0c386..bc01727bf59d7 100644 --- a/tidb-roadmap.md +++ b/tidb-roadmap.md @@ -10,49 +10,360 @@ This roadmap provides a look into the proposed future. This will be continually In the course of development, this roadmap is subject to change based on user needs and feedback. **DO NOT** schedule your release plans according to the content of the roadmap. If you have a feature request or want to prioritize a feature, please file an issue on [GitHub](https://github.com/pingcap/tidb/issues). > **Note:** +> > If not marked as GA, these features are experimental. ## Rolling roadmap highlights - - - - - - - +
CategoryEnd of CY24 releaseMid of CY25 releaseFuture releases
+ + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+
Category
+		 +
End of CY24 release
+		 +
Mid of CY25 release
+		 +
Future releases
+
Unmatched Scalability and Peak Performance
Deliver massive scalability and faster performance to support larger workloads, optimize resource utilization, and ensure superior responsiveness
In-memory caching in TiKV
TiKV maintains recent versions of data in memory to reduce redundant MVCC scans, thus improving performance.
Adaptive Parallelism for Stats Collection (GA)
Dynamically adjusts parallelism and concurrency based on hardware and node count, accelerating statistics collection.
Faster Database Restores
Reduces recovery time for full database and point-in-time restores (PITR).
Unlimited Transaction Size
Removes memory limits on uncommitted transactions, improving batch task success rates.
Load-Based Traffic Rebalancing in TiProxy(GA)
Routes traffic based on TiDB node workloads to optimize resource utilization.
PD Microservice Router Service
Enables independent deployment, stateless(no Strong Leader) and easy scaling of Router Service(region meta query/updates) in PD, preventing PD from becoming a bottleneck for cluster resources.
Reduced I/O for Statistics Collection (GA)
Allows users to scan only a portion of data samples on TiKV, reducing time and resource consumption for statistics collection.
Limit Operator Pushdown
Removes limitations on pushing down the Limit operator from TiDB to TiKV, enabling more efficient query processing directly at the storage layer.
Cascades Optimizer Framework
Introduces a mature, advanced optimizer framework, expanding the capabilities of the existing optimizer.
Scalable DDL Execution Framework
Provides a parallel DDL execution framework to improve the performance and stability of DDL operations.
Increased TiCDC Scalability
Updated TiCDC architecture delivers increased scalability and performance for change data capture use cases.
Table-Level Load Balancing
Optimizes data scheduling in PD based on workload distribution across Regions for each table.
System Table Performance Optimization
Enhances query speed and reduces overhead for system tables with large data volumes.
Enhance the Scability of Region Meta Storage
Migrate region meta storage from PD to TiKV, the storage layer can scale infinitely ans easily.
Advanced SQL Features and Flexibility
Cutting-edge SQL capabilities, improving compatibility, flexibility, and ease of use for complex queries and modern applications
Vector Search Support
Enables vector data types, indexing, and high-performance vector search, with support for mixed queries involving vector and relational data.
Foreign Keys(GA)
Providing robust relational integrity.
Global indexing on partitioned tables(GA)
Removes the unique key restriction on partition keys, boosting query performance for non-partitioned columns.		Modifiable Column Types in Partitioned Tables
Allows users to change column data types in partitioned tables, even if the column is a partitioning key.
Materialized Views Support
Enables materialized views to improve pre-computation, boost computational efficiency, and enhance data analysis performance.
Unbreakable Reliability and Always-On Availability
Near-zero downtime and enhanced fault tolerance to maintain uninterrupted operations and deliver a rock-solid user experience
Limit Memory for Backups
Limit Memory for Statistics Collection (GA)
Enhanced SQL Binding Management (GA)
Simplifies creating and managing large numbers of execution plans to stabilize performance.
Improved Resource Group Control for Complex SQL (GA)
Monitors RU usage of complex queries mid-execution to minimize system impact.
Automatic Resource Group Switching for Runaway Queries (GA)
Detects runaway queries and redirects them to designated resource groups with set limits.
Limit Memory Usage for Schema Metadata（GA）
Enhances stability in large-scale clusters by capping memory consumption for schema metadata.
Robust and Resilient Backup
Reduces memory-related issues during backup processes, ensuring dependable data protection and availability.
Optimized Memory Management with Disk Spilling
Allows operators like HashAgg, Sort, and Join to spill to disk, reducing memory load and preventing out-of-memory (OOM) issues.
Sharing Plan Cache across Sessions (GA)
Shares execution plan cache across sessions in the same TiDB instance, optimizing memory usage.
Resource Group Quota Management (GA)
Dynamically adjusts resource limits for Burstable resource groups, fully utilizing available resources without impacting other quotas.
Adaptive Resource Group
Automatically adjusts Request Unit (RU) settings in resource groups based on past execution patterns.
Enhanced Memory Protection
Monitors memory usage across all components to prevent operations that could impact system stability.
Automatic SQL Binding
Analyzes SQL performance metrics to automatically create bindings, stabilizing execution plans for transactional processing.
Multi-Versioned Statistics
Allows users to view and restore previous statistics versions after updates.
Distributed Statistics Collection
Enables parallel statistics collection across multiple TiDB nodes to boost efficiency.
Effortless Operations and Intelligent Observability
Simplify management with proactive monitoring and insights to optimize performance and ensure smooth operations
Reliable Query Termination（GA）
Instantly terminates running SQL statements and frees resources in TiDB and TiKV.
Permissioned Resource Group Switching (GA)
Restricts resource group switching to authorized users, preventing resource misuse.
CPU Time Observation for TiDB and TiKV (GA)
Adds CPU time metrics to logs, enabling quick identification of statements causing CPU spikes.
Customizable Statistics Collection (GA)
Allows tailored statistics strategies for specific tables, adjusting parameters like health and parallelism.
Workload Repository (GA)
Stores workload stats and real-time data for improved troubleshooting and analysis.
Automated Index Advisor (GA)
Automatically analyzes SQL statements to recommend index optimizations, including suggestions for creating or dropping indexes.
Standardized Time Model (GA)
Establishes a unified SQL execution time model to help identify database load sources through logs and cluster metrics, pinpointing problematic nodes and statements.
TiFlash CPU Time Monitoring (GA)
Adds TiFlash CPU time metrics to logs, enabling quick identification of statements that cause CPU spikes in TiFlash.		Workload Analysis
Analyzes historical data from the Workload Repository to provide optimization recommendations, including SQL tuning and statistics collection.
End-to-End SQL Monitoring
Tracks the entire lifecycle of SQL statements, measuring time spent across TiDB, TiKV, PD, and TiFlash for detailed performance insights.
Comprehensive Data Security and Privacy
Robust security measures to safeguard sensitive data, ensuring top-tier protection, encryption, and compliance with evolving privacy regulations		Google Cloud KMS(GA)
General availability for encryption-at-rest key management with Google Cloud KMS.
Azure Key Vault
Enhanced encryption-at-rest key management with Azure Key Vault integration.
Marker-Based Log Desensitization
Marks and selectively desensitizes sensitive data in cluster logs based on use case.
Column-Level Permission Management(GA)
Adds MySQL-compatible permissions at the column level for fine-grained access control.		AWS IAM Authentication
Supports AWS IAM third-party ARN integration for secure access control in TiDB.
Kerberos Authentication(GA)
Enables authentication using Kerberos for added security.
Multi-Factor Authentication (MFA)
Adds support for multi-factor authentication to enhance user verification the multi-factor authentication mechanism.
Enhanced TLS Security(GA)
Ensures encrypted connections between all components within the TiDB cluster.
Refined Dynamic Privileges
Improves dynamic privilege management, including limitations on Super privilege.
FIPS Compliance(GA)
Ensures encryption methods comply with FIPS standards for secure data handling.
Label-Based Access Control
Enables data access control through configurable labels.
Enhanced Client-Side Encryption
Supports encryption of key fields on the client side to strengthen data security.
Dynamic Data Desensitization
Allows data desensitization based on application scenarios, protecting sensitive business fields.
+ + +
Unmatched Scalability and Peak Performance Deliver massive scalability and faster performance to support larger workloads, optimize resource utilization, and ensure superior responsiveness
+
 
+ + + +
 
+ + + +
 
+ + + +
 
+ + + + +
Advanced SQL Features and Flexibility
+
Cutting-edge SQL capabilities, improving compatibility, flexibility, and ease of use for complex queries and modern applications
+
 
+ + + + + + +
 
+ + + +
 
+ + + + +
Unbreakable Reliability and Always-On Availability Near-zero downtime and enhanced fault tolerance to maintain uninterrupted operations and deliver a rock-solid user experience
+
 
+ + + +
 
+ + + +
 
+ + + +
 
+ + + + +
Effortless Operations and Intelligent Observability Simplify management with proactive monitoring and insights to optimize performance and ensure smooth operations
+
 
+ + + +
 
+ + + + + + +
 
+ + + + +
Comprehensive Data Security and Privacy Robust security measures to safeguard sensitive data, ensuring top-tier protection, encryption, and compliance with evolving privacy regulations
+ + + +
 
+ + + +
 
+ + + +
 
+ + + + > **Note:** >