partition: add doc for global index, also with GLOBAL IndexOption/ColumnOption

partitioned-table.md

@@ -1662,6 +1662,101 @@ CREATE TABLE t (a varchar(20), b blob,
 ERROR 1503 (HY000): A UNIQUE INDEX must include all columns in the table's partitioning function
 ```
 
+#### Global indexes
+
+Before the introduction of global indexes, TiDB created a local index for each partition. Therefore, there was [a limitation](#partitioning-keys-primary-keys-and-unique-keys) that primary keys and unique keys had to include the partition key to ensure uniqueness of the data. Additionally, when querying data that spans multiple partitions, the system needed to scan the data of each partition to return results.
+
+To address these issues, TiDB introduces the global indexes feature in v8.3.0. Global indexes cover the data of the entire table with a single index, allowing primary keys and unique keys to maintain global uniqueness without including the partition key. At the same time, global indexes can access data spanning multiple partitions in a single operation, significantly improving query performance for non-partitioned keys.
+
+If you need to create unique indexes that **do not include all the columns used in the partition expressions**, you can achieve this by enabling the [`tidb_enable_global_index`](/system-variables.md#tidb_enable_global_index-new-in-v760) system variable. After enabling this variable, any unique index that does not meet the preceding constraint will need the `GLOBAL` attribute to be created as a global index.
+
+Note that global indexes affect partition management. `DROP`, `TRUNCATE`, and `REORGANIZE PARTITION` also require managing table-level global indexes, meaning that these DDL operations will only return results after the global indexes have been fully updated.
+
+```sql
+SET tidb_enable_global_index = ON;
+
+CREATE TABLE t1 (
+    col1 INT NOT NULL,
+    col2 DATE NOT NULL,
+    col3 INT NOT NULL,
+    col4 INT NOT NULL,
+    UNIQUE KEY uidx12(col1, col2) GLOBAL,
+    UNIQUE KEY uidx3(col3)
+)
+PARTITION BY HASH(col3)
+PARTITIONS 4;
+```
+
+In this example, the unique index `uidx12` will be a global index, but `uidx3` remains a regular unique index.
+
+Note that a **clustered index** cannot be a global index, as shown in the following example:
+
+```sql
+SET tidb_enable_global_index = ON;
+
+CREATE TABLE t2 (
+    col1 INT NOT NULL,
+    col2 DATE NOT NULL,
+    PRIMARY KEY (col2) CLUSTERED GLOBAL
+) PARTITION BY HASH(col1) PARTITIONS 5;
+```
+
+```
+ERROR 1503 (HY000): A CLUSTERED INDEX must include all columns in the table's partitioning function
+```
+
+The reason is that if the clustered index is a global index, the table will no longer be partitioned. This is because the key of the clustered index is also the record key, which means it should be on partition level. But the global index is on table level, which causes a conflict. If you need to set the primary key as a global index, you must explicitly define it as a non-clustered index, for example, `PRIMARY KEY(col1, col2) NONCLUSTERED GLOBAL`.
+
+You can identify a global index by the GLOBAL IndexOption in the [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md) output.
+
+```sql
+SHOW CREATE TABLE t1\G
+```
+
+```
+       Table: t1
+Create Table: CREATE TABLE `t1` (
+  `col1` int(11) NOT NULL,
+  `col2` date NOT NULL,
+  `col3` int(11) NOT NULL,
+  `col4` int(11) NOT NULL,
+  UNIQUE KEY `uidx12` (`col1`,`col2`) /*T![global_index] GLOBAL */,
+  UNIQUE KEY `uidx3` (`col3`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
+PARTITION BY HASH (`col3`) PARTITIONS 4
+1 row in set (0.00 sec)
+```
+
+Or querying the [`information_schema.tidb_indexes`](/information-schema/information-schema-tidb-indexes.md) table and checking the table structure.
+
+```sql
+SELECT * FROM information_schema.tidb_indexes WHERE table_name='t1';
+```
+
+```
++--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
+| TABLE_SCHEMA | TABLE_NAME | NON_UNIQUE | KEY_NAME | SEQ_IN_INDEX | COLUMN_NAME | SUB_PART | INDEX_COMMENT | Expression | INDEX_ID | IS_VISIBLE | CLUSTERED | IS_GLOBAL |
++--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
+| test         | t1         |          0 | uidx12   |            1 | col1        |     NULL |               | NULL       |        1 | YES        | NO        |         1 |
+| test         | t1         |          0 | uidx12   |            2 | col2        |     NULL |               | NULL       |        1 | YES        | NO        |         1 |
+| test         | t1         |          0 | uidx3    |            1 | col3        |     NULL |               | NULL       |        2 | YES        | NO        |         0 |
++--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
+3 rows in set (0.00 sec)
+```
+
+When partitioning a non-partitioned table or re-partitioning an already partitioned table it is possible to update the indexes to be [global index](#global-indexes) or default local indexes.
+
+```sql
+ALTER TABLE t1 PARTITION BY HASH (col1) PARTITIONS 3 UPDATE INDEXES (uidx12 LOCAL, uidx3 GLOBAL);
+```
+
+##### Limitations of global indexes
+
+- If the `GLOBAL` keyword is not explicitly specified, a local index will be created by default.
+- The `GLOBAL` and `LOCAL` keywords only apply to partitioned tables and have no effect on non-partitioned tables. In other words, there is no difference between a global index and a local index in non-partitioned tables.
+- DDL operations such as `ADD PARTITION`, `DROP PARTITION`, `TRUNCATE PARTITION`, `REORGANIZE PARTITION`, `SPLIT PARTITION`, and `EXCHANGE PARTITION` will trigger updates to global indexes. The results of these DDL operations will only be returned after the global index updates are complete. This impacts scenarios that require quick DDL completion, for example, data archiving operations such as `EXCHANGE PARTITION`, `TRUNCATE PARTITION`, and `DROP PARTITION`. In contrast, when global indexes are not involved, these DDL operations can be completed immediately.
+- By default, the primary key of a partitioned table is a clustered index and must include the partition key. If you require the primary key to exclude the partition key, you can explicitly specify the primary key as a non-clustered global index when creating the table, for example, `PRIMARY KEY(col1, col2) NONCLUSTERED GLOBAL`.
+
 ### Partitioning limitations relating to functions
 
 Only the functions shown in the following list are allowed in partitioning expressions:

placement-rules-in-sql.md

@@ -297,32 +297,35 @@ ALTER TABLE t PLACEMENT POLICY=default; -- Removes the placement policy 'five_re
 You can also specify a placement policy for a partitioned table or a partition. For example:
 
 ```sql
-CREATE PLACEMENT POLICY storageforhisotrydata CONSTRAINTS="[+node=history]";
+CREATE PLACEMENT POLICY storageforhistorydata CONSTRAINTS="[+node=history]";
 CREATE PLACEMENT POLICY storagefornewdata CONSTRAINTS="[+node=new]";
 CREATE PLACEMENT POLICY companystandardpolicy CONSTRAINTS="";
 
-CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE)
+SET tidb_enable_global_index = ON;
+
+CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE, UNIQUE INDEX idx(id) GLOBAL)
 PLACEMENT POLICY=companystandardpolicy
 PARTITION BY RANGE( YEAR(purchased) ) (
-  PARTITION p0 VALUES LESS THAN (2000) PLACEMENT POLICY=storageforhisotrydata,
+  PARTITION p0 VALUES LESS THAN (2000) PLACEMENT POLICY=storageforhistorydata,
   PARTITION p1 VALUES LESS THAN (2005),
   PARTITION p2 VALUES LESS THAN (2010),
   PARTITION p3 VALUES LESS THAN (2015),
   PARTITION p4 VALUES LESS THAN MAXVALUE PLACEMENT POLICY=storagefornewdata
 );
 ```
 
-If no placement policy is specified for a partition in a table, the partition attempts to inherit the policy (if any) from the table. In the preceding example:
+If no placement policy is specified for a partition in a table, the partition attempts to inherit the policy (if any) from the table. If the table has a [global index](/partitioned-table.md#global-indexes), the index will apply the same placement policy as the table. In the preceding example:
 
-- The `p0` partition will apply the `storageforhisotrydata` policy.
+- The `p0` partition will apply the `storageforhistorydata` policy.
 - The `p4` partition will apply the `storagefornewdata` policy.
 - The `p1`, `p2`, and `p3` partitions will apply the `companystandardpolicy` placement policy inherited from the table `t1`.
-- If no placement policy is specified for the table `t1`, the `p1`, `p2`, and `p3` partitions will inherit the database default policy or the global default policy.
+- The global index `idx` will apply the same `companystandardpolicy` placement policy as the table `t1`.
+- If no placement policy is specified for the table `t1`, then the `p1`, `p2`, and `p3` partitions and the global index `idx` will inherit the database default policy or the global default policy.
 
 After placement policies are attached to these partitions, you can change the placement policy for a specific partition as in the following example:
 
 ```sql
-ALTER TABLE t1 PARTITION p1 PLACEMENT POLICY=storageforhisotrydata;
+ALTER TABLE t1 PARTITION p1 PLACEMENT POLICY=storageforhistorydata;
 ```
 
 ## High availability examples
@@ -479,4 +482,4 @@ After executing the statements in the example, TiDB will place the `app_order` d
 | TiDB Lightning | Not compatible yet | An error is reported when TiDB Lightning imports backup data that contains placement policies  |
 | TiCDC | 6.0 | Ignores placement policies, and does not replicate the policies to the downstream |
 
-</CustomContent>
+</CustomContent>

sql-statements/sql-statement-add-column.md

@@ -29,8 +29,8 @@ ColumnType
 ColumnOption
          ::= 'NOT'? 'NULL'
            | 'AUTO_INCREMENT'
-           | 'PRIMARY'? 'KEY' ( 'CLUSTERED' | 'NONCLUSTERED' )?
-           | 'UNIQUE' 'KEY'?
+           | 'PRIMARY'? 'KEY' ( 'CLUSTERED' | 'NONCLUSTERED' )? ( 'GLOBAL' | 'LOCAL' )?
+           | 'UNIQUE' 'KEY'? ( 'GLOBAL' | 'LOCAL' )?
            | 'DEFAULT' ( NowSymOptionFraction | SignedLiteral | NextValueForSequence )
            | 'SERIAL' 'DEFAULT' 'VALUE'
            | 'ON' 'UPDATE' NowSymOptionFraction
@@ -89,6 +89,7 @@ mysql> SELECT * FROM t1;
 * Adding a new column and setting it to the `PRIMARY KEY` is not supported.
 * Adding a new column and setting it to `AUTO_INCREMENT` is not supported.
 * There are limitations on adding generated columns, refer to: [generated column limitations](/generated-columns.md#limitations).
+* Setting a global index by setting `PRIMARY KEY` or `UNIQUE INDEX` to `GLOBAL` when you add a new column is a TiDB extension for [global indexes](/partitioned-table.md#global-indexes) and is not compatible with MySQL.
 
 ## See also
 

sql-statements/sql-statement-add-index.md

@@ -43,6 +43,8 @@ IndexOption
            | 'COMMENT' stringLit
            | 'VISIBLE'
            | 'INVISIBLE'
+           | 'GLOBAL'
+           | 'LOCAL'
 
 IndexType
          ::= 'BTREE'
@@ -90,6 +92,7 @@ mysql> EXPLAIN SELECT * FROM t1 WHERE c1 = 3;
 * TiDB supports parsing the `FULLTEXT` syntax but does not support using the `FULLTEXT` indexes.
 * Descending indexes are not supported (similar to MySQL 5.7).
 * Adding the primary key of the `CLUSTERED` type to a table is not supported. For more details about the primary key of the `CLUSTERED` type, refer to [clustered index](/clustered-indexes.md).
+* Setting a `PRIMARY KEY` or `UNIQUE INDEX` as a global index with `GLOBAL` IndexOption is a TiDB extension for [global indexes](/partitioned-table.md#global-indexes) and is not compatible with MySQL.
 
 ## See also
 

sql-statements/sql-statement-create-index.md

@@ -44,6 +44,7 @@ IndexOption ::=
 |   'WITH' 'PARSER' Identifier
 |   'COMMENT' stringLit
 |   ("VISIBLE" | "INVISIBLE")
+|   ("GLOBAL" | "LOCAL")
 
 IndexTypeName ::=
     'BTREE'
@@ -356,6 +357,7 @@ The system variables associated with the `CREATE INDEX` statement are `tidb_ddl_
 * Expression indexes are incompatible with views. When a query is executed using a view, the expression index cannot be used at the same time.
 * Expression indexes have compatibility issues with bindings. When the expression of an expression index has a constant, the binding created for the corresponding query expands its scope. For example, suppose that the expression in the expression index is `a+1`, and the corresponding query condition is `a+1 > 2`. In this case, the created binding is `a+? > ?`, which means that the query with the condition such as `a+2 > 2` is also forced to use the expression index and results in a poor execution plan. In addition, this also affects the baseline capturing and baseline evolution in SQL Plan Management (SPM).
 * The data written with multi-valued indexes must exactly match the defined data type. Otherwise, data writes fail. For details, see [create multi-valued indexes](/sql-statements/sql-statement-create-index.md#create-multi-valued-indexes).
+* Setting a `UNIQUE KEY` as a global index with `GLOBAL` IndexOption is a TiDB extension for [global indexes](/partitioned-table.md#global-indexes) and is not compatible with MySQL.
 
 ## See also
 

sql-statements/sql-statement-create-table.md

@@ -45,8 +45,8 @@ ColumnOptionList ::=
 ColumnOption ::=
     'NOT'? 'NULL'
 |   'AUTO_INCREMENT'
-|   PrimaryOpt 'KEY'
-|   'UNIQUE' 'KEY'?
+|   PrimaryOpt 'KEY' ( 'GLOBAL' | 'LOCAL' )?
+|   'UNIQUE' 'KEY'? ( 'GLOBAL' | 'LOCAL' )?
 |   'DEFAULT' DefaultValueExpr
 |   'SERIAL' 'DEFAULT' 'VALUE'
 |   'ON' 'UPDATE' NowSymOptionFraction
@@ -77,6 +77,7 @@ IndexOption ::=
     'COMMENT' String
 |   ( 'VISIBLE' | 'INVISIBLE' )
 |   ('USING' | 'TYPE') ('BTREE' | 'RTREE' | 'HASH')
+|   ( 'GLOBAL' | 'LOCAL' )
 
 ForeignKeyDef
          ::= ( 'CONSTRAINT' Identifier )? 'FOREIGN' 'KEY'
@@ -242,6 +243,7 @@ mysql> DESC t1;
 * All of the data types except spatial types are supported.
 * TiDB accepts index types such as `HASH`, `BTREE` and `RTREE` in syntax for compatibility with MySQL, but ignores them.
 * TiDB supports parsing the `FULLTEXT` syntax but does not support using the `FULLTEXT` indexes.
+* Setting a `PRIMARY KEY` or `UNIQUE INDEX` as a global index with `GLOBAL` IndexOption is a TiDB extension for [global indexes](/partitioned-table.md#global-indexes) and is not compatible with MySQL.
 
 <CustomContent platform="tidb">
 

system-variables.md

@@ -2128,13 +2128,17 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1;
 
 ### tidb_enable_global_index <span class="version-mark">New in v7.6.0</span>
 
+> **Warning:**
+>
+> The feature controlled by this variable is an experimental feature. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub.
+
 - Scope: SESSION | GLOBAL
 - Persists to cluster: Yes
 - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
 - Type: Boolean
 - Default value: `OFF`
 - Possible values: `OFF`, `ON`
-- This variable controls whether to support creating `Global indexes` for partitioned tables. `Global index` is currently in the development stage. **It is not recommended to modify the value of this system variable**.
+- This variable controls whether to support creating [global indexes](/partitioned-table.md#global-indexes) for partitioned tables. When this variable is enabled, TiDB allows to create unique indexes that do not contain all columns in the partitioning expressions by using the `GLOBAL` IndexOption or ColumnOption.
 
 ### tidb_enable_non_prepared_plan_cache