Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partition: add doc for global index #18217

Merged
merged 15 commits into from
Aug 19, 2024
105 changes: 103 additions & 2 deletions partitioned-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -1461,6 +1461,10 @@ SELECT store_id, COUNT(department_id) AS c

本节讨论分区键,主键和唯一键之间的关系。一句话总结它们之间的关系要满足的规则:**分区表的每个唯一键,必须包含分区表达式中用到的所有列**。

> **注意:**
>
> 该规则仅适用于 [`tidb_enable_global_index`](/system-variables.md#tidb_enable_global_index-new-in-v760) 未开启的场景。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

这里所指的唯一也包含了主键,因为根据主键的定义,主键必须是唯一的。例如,下面这些建表语句就是无效的:

{{< copyable "sql" >}}
Expand Down Expand Up @@ -1637,8 +1641,6 @@ PARTITIONS 4;

DDL 变更时,添加唯一索引也需要考虑到这个限制。比如创建了这样一个表:

{{< copyable "sql" >}}

```sql
CREATE TABLE t_no_pk (c1 INT, c2 INT)
PARTITION BY RANGE(c1) (
Expand Down Expand Up @@ -1672,6 +1674,105 @@ CREATE TABLE t (a varchar(20), b blob,
ERROR 1503 (HY000): A UNIQUE INDEX must include all columns in the table's partitioning function
```

#### 全局索引

hfxsd marked this conversation as resolved.
Show resolved Hide resolved
在引入全局索引 (Global Index) 之前,TiDB 为每个分区创建一个局部索引 (Local Index),即一个分区对应一个局部索引。因此有一个[限制](#分区键主键和唯一键),主键和唯一键必须包含分区键,以确保数据的全局唯一性。此外,当查询的数据跨越多个分区时,系统需要扫描每个分区的数据才能返回结果。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

为解决这些问题,TiDB 从 v8.3.0 开始引入全局索引。全局索引通过一个单一的索引覆盖整个表的数据,使得主键和唯一键在不包含分区键的情况下仍能保持全局唯一性。同时,全局索引可以一次性访问跨多个分区的数据,显著提升了针对非分区键的查询性能。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

> **警告:**
>
> 该变量控制的功能为实验特性,不建议在生产环境中使用。该功能可能会在未事先通知的情况下发生变化或删除。如果发现 bug,请在 GitHub 上提 [issue](https://github.com/pingcap/tidb/issues) 反馈。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

如果你需要创建的唯一索引**不包含分区表达式中使用的所有列**,可以通过启用 [`tidb_enable_global_index`](/system-variables.md#tidb_enable_global_index-从-v760-版本开始引入) 系统变量来实现。启用此变量后,任何不符合前述约束条件的唯一索引都需要使用 `GLOBAL` 属性来将其创建为全局索引。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

注意,全局索引对分区管理有影响,`DROP`、`TRUNCATE` 和 `REORGANIZE PARTITION` 也需要管理表级全局索引,即这些 DDL 需要更新完全局索引后才会返回结果。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

```sql
SET tidb_enable_global_index = ON;

CREATE TABLE t1 (
col1 INT NOT NULL,
col2 DATE NOT NULL,
col3 INT NOT NULL,
col4 INT NOT NULL,
UNIQUE KEY uidx12(col1, col2) GLOBAL,
UNIQUE KEY uidx3(col3)
)
PARTITION BY HASH(col3)
PARTITIONS 4;
```

在上面示例中,唯一索引 `uidx12` 将成为全局索引,但 `uidx3` 仍是常规的唯一索引。

请注意,**聚簇索引**不能成为全局索引,如下例所示:

```sql
SET tidb_enable_global_index = ON;

CREATE TABLE t2 (
col1 INT NOT NULL,
col2 DATE NOT NULL,
PRIMARY KEY (col2) CLUSTERED GLOBAL
) PARTITION BY HASH(col1) PARTITIONS 5;
```

```
ERROR 1503 (HY000): A CLUSTERED INDEX must include all columns in the table's partitioning function
```

聚簇索引不能成为全局索引,是因为如果聚簇索引是全局索引,则表将不再分区。这是因为聚簇索引的键也是行数据的键,而行数据应该是分区级别的,但全局索引是表级别,这就造成了冲突。如果需要将主键设置为全局索引,则需要显式设置该主键为非聚簇索引,如 `PRIMARY KEY(col1, col2) NONCLUSTERED GLOBAL`。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

你可以通过 [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md) 输出中的 `GLOBAL IndexOption` 来识别全局索引。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

```sql
SHOW CREATE TABLE t1\G
```

```
Table: t1
Create Table: CREATE TABLE `t1` (
`col1` int(11) NOT NULL,
`col2` date NOT NULL,
`col3` int(11) NOT NULL,
`col4` int(11) NOT NULL,
UNIQUE KEY `uidx12` (`col1`,`col2`) /*T![global_index] GLOBAL */,
UNIQUE KEY `uidx3` (`col3`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
PARTITION BY HASH (`col3`) PARTITIONS 4
1 row in set (0.00 sec)
```

或查询 [`information_schema.tidb_indexes`](/information-schema/information-schema-tidb-indexes.md) 表并检查表结构来识别全局索引。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

```sql
SELECT * FROM information_schema.tidb_indexes WHERE table_name='t1';
```

```
+--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
| TABLE_SCHEMA | TABLE_NAME | NON_UNIQUE | KEY_NAME | SEQ_IN_INDEX | COLUMN_NAME | SUB_PART | INDEX_COMMENT | Expression | INDEX_ID | IS_VISIBLE | CLUSTERED | IS_GLOBAL |
+--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
| test | t1 | 0 | uidx12 | 1 | col1 | NULL | | NULL | 1 | YES | NO | 1 |
| test | t1 | 0 | uidx12 | 2 | col2 | NULL | | NULL | 1 | YES | NO | 1 |
| test | t1 | 0 | uidx3 | 1 | col3 | NULL | | NULL | 2 | YES | NO | 0 |
+--------------+------------+------------+----------+--------------+-------------+----------+---------------+------------+----------+------------+-----------+-----------+
3 rows in set (0.00 sec)
```

在分区非分区表、或重新分区已经分区的表时,可以将索引更新为全局索引或默认本地索引。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

```sql
ALTER TABLE t1 PARTITION BY HASH (col1) PARTITIONS 3 UPDATE INDEXES (uidx12 LOCAL, uidx3 GLOBAL);
```

hfxsd marked this conversation as resolved.
Show resolved Hide resolved
##### 全局索引的限制
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

- 如果未显式指定 `GLOBAL` 关键字,将默认创建局部索引 (local Index)。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
- `GLOBAL` 和 `LOCAL` 关键字仅适用于分区表,对非分区表没有影响。即在非分区表中,全局索引和局部索引之间没有区别。
- DDL 操作如 `ADD PARTITION`、`DROP PARTITION`、`TRUNCATE PARTITION`、`REORGANIZE PARTITION`、`SPLIT PARTITION` 和 `EXCHANGE PARTITION` 等 DDL 会触发全局索引的更新,DDL 执行结果将在全局索引更新完成后才会返回。而如果没有全局索引,在一些场景,如数据归档,`EXCHANGE PARTITION`、`TRUNCATE PARTITION` 和 `DROP PARTITION` 这些 DDL 可以立即执行完成。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
- 默认情况下,分区表的主键为聚簇索引,必须包含分区键。如果要求主键不包含分区建,可以在建表时显式指定主键为非聚簇的全局索引,例如:`PRIMARY KEY(col1, col2) NONCLUSTERED GLOBAL`。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

### 关于函数的分区限制

只有以下函数可以用于分区表达式:
Expand Down
15 changes: 9 additions & 6 deletions placement-rules-in-sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,11 +280,13 @@ ALTER TABLE t PLACEMENT POLICY=default; -- 删除表 t 已绑定的放置策略
你还可以给表分区指定放置策略。示例如下:

```sql
CREATE PLACEMENT POLICY storageforhisotrydata CONSTRAINTS="[+node=history]";
CREATE PLACEMENT POLICY storageforhistorydata CONSTRAINTS="[+node=history]";
CREATE PLACEMENT POLICY storagefornewdata CONSTRAINTS="[+node=new]";
CREATE PLACEMENT POLICY companystandardpolicy CONSTRAINTS="";

CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE)
SET tidb_enable_global_index = ON;

CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE, UNIQUE INDEX idx(id) GLOBAL)
PLACEMENT POLICY=companystandardpolicy
PARTITION BY RANGE( YEAR(purchased) ) (
PARTITION p0 VALUES LESS THAN (2000) PLACEMENT POLICY=storageforhisotrydata,
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -295,17 +297,18 @@ PARTITION BY RANGE( YEAR(purchased) ) (
);
```

如果没有为表中的某个分区指定任何放置策略,该分区将尝试继承表上可能存在的策略。在上面示例中:
如果没有为表中的某个分区指定任何放置策略,该分区将尝试继承表上可能存在的策略。如果该表有[全局索引](/partitioned-table.md#全局索引),索引将应用与该表相同的放置策略。在上面示例中:

- `p0` 分区将会应用 `storageforhisotrydata` 策略
- `p0` 分区将会应用 `storageforhistorydata` 策略
- `p4` 分区将会应用 `storagefornewdata` 策略
- `p1`、`p2`、`p3` 分区将会应用表 `t1` 的放置策略 `companystandardpolicy`
- 如果 `t1` 没有绑定任何策略,`p1`、`p2`、`p3` 会继承数据库或全局的默认策略
- 全局索引 `idx` 将应用与表 `t1` 相同的 `companystandardpolicy` 放置策略
- 如果没有为表 `t1` 指定放置策略,`p1`、`p2` 和 `p3` 分区以及全局索引 `idx` 将继承数据库默认策略或全局默认策略

给分区绑定放置策略后,你可以更改指定分区的放置策略。示例如下:

```sql
ALTER TABLE t1 PARTITION p1 PLACEMENT POLICY=storageforhisotrydata;
ALTER TABLE t1 PARTITION p1 PLACEMENT POLICY=storageforhistorydata;
```

## 高可用场景示例
Expand Down
5 changes: 3 additions & 2 deletions sql-statements/sql-statement-add-column.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ ColumnType
ColumnOption
::= 'NOT'? 'NULL'
| 'AUTO_INCREMENT'
| 'PRIMARY'? 'KEY' ( 'CLUSTERED' | 'NONCLUSTERED' )?
| 'UNIQUE' 'KEY'?
| 'PRIMARY'? 'KEY' ( 'CLUSTERED' | 'NONCLUSTERED' )? ( 'GLOBAL' | 'LOCAL' )?
| 'UNIQUE' 'KEY'? ( 'GLOBAL' | 'LOCAL' )?
| 'DEFAULT' ( NowSymOptionFraction | SignedLiteral | NextValueForSequence )
| 'SERIAL' 'DEFAULT' 'VALUE'
| 'ON' 'UPDATE' NowSymOptionFraction
Expand Down Expand Up @@ -136,6 +136,7 @@ SELECT * FROM t1;
* 不支持将新添加的列设为 `PRIMARY KEY`。
* 不支持将新添加的列设为 `AUTO_INCREMENT`。
* 对添加生成列有局限性,具体可参考:[生成列局限性](/generated-columns.md#生成列的局限性)。
* TiDB 对[全局索引](/partitioned-table.md#全局索引)进行了扩展,你可以在添加新列时通过将 `PRIMARY KEY` 或 `UNIQUE INDEX` 设置为 `GLOBAL` 来设置全局索引。该扩展与 MySQL 不兼容。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

## 另请参阅

Expand Down
3 changes: 3 additions & 0 deletions sql-statements/sql-statement-add-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ IndexOption
| 'COMMENT' stringLit
| 'VISIBLE'
| 'INVISIBLE'
| 'GLOBAL'
| 'LOCAL'

IndexType
::= 'BTREE'
Expand Down Expand Up @@ -118,6 +120,7 @@ EXPLAIN SELECT * FROM t1 WHERE c1 = 3;
* TiDB 支持解析 `FULLTEXT` 语法,但不支持使用 `FULLTEXT` 索引。
* 不支持降序索引(类似于 MySQL 5.7)。
* 无法向表中添加 `CLUSTERED` 类型的 `PRIMARY KEY`。要了解关于 `CLUSTERED` 主键的详细信息,请参考[聚簇索引](/clustered-indexes.md)。
* TiDB 对[全局索引](/partitioned-table.md#全局索引)进行了扩展。你可以使用 `GLOBAL` IndexOption 将 `PRIMARY KEY` 或 `UNIQUE INDEX` 设置为全局索引。该扩展与 MySQL 不兼容。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

## 另请参阅

Expand Down
2 changes: 2 additions & 0 deletions sql-statements/sql-statement-create-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ IndexOption ::=
| 'WITH' 'PARSER' Identifier
| 'COMMENT' stringLit
| ("VISIBLE" | "INVISIBLE")
| ("GLOBAL" | "LOCAL")

IndexTypeName ::=
'BTREE'
Expand Down Expand Up @@ -407,6 +408,7 @@ CREATE UNIQUE INDEX c1 ON t1 (c1) INVISIBLE;
* 表达式索引与视图存在兼容性问题。通过视图进行查询时,无法使用上表达式索引。
* 表达式索引与 Binding 存在兼容性问题。当表达式索引中的表达式存在常量时,对应查询所建的 Binding 会扩大范围。假设表达式索引中的表达式为 `a+1`,对应的查询条件为 `a+1 > 2`。则建立的 Binding 为 `a+? > ?`,这会导致像 `a+2 > 2` 这样的查询也会强制使用表达式索引,得到一个较差的执行计划。这同样影响 SQL Plan Management (SPM) 中的捕获和演进功能。
* 多值索引写入的数据必须与定义类型完全匹配,否则数据写入失败。详见[创建多值索引](/sql-statements/sql-statement-create-index.md#创建多值索引)。
* TiDB 对[全局索引](/partitioned-table.md#全局索引)进行了扩展。你可以使用 `GLOBAL` IndexOption 将 `PRIMARY KEY` 或 `UNIQUE INDEX` 设置为全局索引。该扩展与 MySQL 不兼容。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

## 另请参阅

Expand Down
6 changes: 4 additions & 2 deletions sql-statements/sql-statement-create-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ ColumnOptionList ::=
ColumnOption ::=
'NOT'? 'NULL'
| 'AUTO_INCREMENT'
| PrimaryOpt 'KEY'
| 'UNIQUE' 'KEY'?
| PrimaryOpt 'KEY' ( 'GLOBAL' | 'LOCAL' )?
| 'UNIQUE' 'KEY'? ( 'GLOBAL' | 'LOCAL' )?
| 'DEFAULT' DefaultValueExpr
| 'SERIAL' 'DEFAULT' 'VALUE'
| 'ON' 'UPDATE' NowSymOptionFraction
Expand Down Expand Up @@ -77,6 +77,7 @@ IndexOption ::=
'COMMENT' String
| ( 'VISIBLE' | 'INVISIBLE' )
| ('USING' | 'TYPE') ('BTREE' | 'RTREE' | 'HASH')
| ( 'GLOBAL' | 'LOCAL' )

ForeignKeyDef
::= ( 'CONSTRAINT' Identifier )? 'FOREIGN' 'KEY'
Expand Down Expand Up @@ -234,6 +235,7 @@ mysql> DESC t1;
* `COMMENT` 属性不支持 `WITH PARSER` 选项。
* TiDB 在单个表中默认支持 1017 列,最大可支持 4096 列。InnoDB 中相应的数量限制为 1017 列,MySQL 中的硬限制为 4096 列。详情参阅 [TiDB 使用限制](/tidb-limitations.md)。
* 分区表支持 `HASH`、`RANGE`、`LIST` 和 `KEY` [分区类型](/partitioned-table.md#分区类型)。对于不支持的分区类型,TiDB 会报 `Warning: Unsupported partition type %s, treat as normal table` 错误,其中 `%s` 为不支持的具体分区类型。
* TiDB 对[全局索引](/partitioned-table.md#全局索引)进行了扩展。你可以使用 `GLOBAL` IndexOption 将 `PRIMARY KEY` 或 `UNIQUE INDEX` 设置为全局索引。该扩展与 MySQL 不兼容。
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

## 另请参阅

Expand Down
6 changes: 5 additions & 1 deletion system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -1919,13 +1919,17 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1;

### `tidb_enable_global_index` <span class="version-mark">从 v7.6.0 版本开始引入</span>

> **警告:**
>
> 该变量控制的功能为实验特性,不建议在生产环境中使用。该功能可能会在未事先通知的情况下发生变化或删除。如果发现 bug,请在 GitHub 上提 [issue](https://github.com/pingcap/tidb/issues) 反馈。

- 作用域:SESSION | GLOBAL
- 是否持久化到集群:是
- 是否受 Hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value) 控制:否
- 类型:布尔型
- 默认值:`OFF`
- 可选值:`OFF`,`ON`
- 这个变量用于控制是否支持对分区表创建 `Global index`。`Global index` 当前正处于开发阶段,**不推荐修改该变量值**
- 该变量控制是否支持为分区表创建[全局索引](/partitioned-table.md#全局索引)。启用此变量后,你可以使用 `GLOBAL` IndexOption 或 ColumnOption 创建不包含分区表达式中所有列的唯一索引
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

### `tidb_enable_non_prepared_plan_cache`

Expand Down
Loading