Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documents for information_schema.tidb_index_usage and sys.schema_unused_index. #16446

Merged
merged 7 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions TOC-tidb-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -585,6 +585,7 @@
- [`TABLE_STORAGE_STATS`](/information-schema/information-schema-table-storage-stats.md)
- [`TIDB_HOT_REGIONS_HISTORY`](/information-schema/information-schema-tidb-hot-regions-history.md)
- [`TIDB_INDEXES`](/information-schema/information-schema-tidb-indexes.md)
- [`TIDB_INDEX_USAGE`](/information-schema/information-schema-tidb-index-usage.md)
- [`TIDB_SERVERS_INFO`](/information-schema/information-schema-tidb-servers-info.md)
- [`TIDB_TRX`](/information-schema/information-schema-tidb-trx.md)
- [`TIFLASH_REPLICA`](/information-schema/information-schema-tiflash-replica.md)
Expand Down
2 changes: 2 additions & 0 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -960,6 +960,7 @@
- [`TIDB_HOT_REGIONS`](/information-schema/information-schema-tidb-hot-regions.md)
- [`TIDB_HOT_REGIONS_HISTORY`](/information-schema/information-schema-tidb-hot-regions-history.md)
- [`TIDB_INDEXES`](/information-schema/information-schema-tidb-indexes.md)
- [`TIDB_INDEX_USAGE`](/information-schema/information-schema-tidb-index-usage.md)
- [`TIDB_SERVERS_INFO`](/information-schema/information-schema-tidb-servers-info.md)
- [`TIDB_TRX`](/information-schema/information-schema-tidb-trx.md)
- [`TIFLASH_REPLICA`](/information-schema/information-schema-tiflash-replica.md)
Expand All @@ -976,6 +977,7 @@
- PERFORMANCE_SCHEMA
- [Overview](/performance-schema/performance-schema.md)
- [`SESSION_CONNECT_ATTRS`](/performance-schema/performance-schema-session-connect-attrs.md)
- [`SYS`](/sys-schema.md)
- [Metadata Lock](/metadata-lock.md)
- [TiDB DDL V2](/ddl-v2.md)
- UI
Expand Down
103 changes: 103 additions & 0 deletions information-schema/information-schema-tidb-index-usage.md
YangKeao marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: TIDB_INDEX_USAGE
summary: Learn the `TIDB_INDEX_USAGE` information_schema table.
Oreoxmt marked this conversation as resolved.
Show resolved Hide resolved
---

# TIDB_INDEX_USAGE

<CustomContent platform="tidb">

Starting from v8.0.0, TiDB provides the `TIDB_INDEX_USAGE` table. You can use `TIDB_INDEX_USAGE` to get the usage statistics of all indexes on the current TiDB node. By default, TiDB collects these index usage statistics during SQL statement execution. You can disable this feature by turning off the [`instance.tidb_enable_collect_execution_info`](/tidb-configuration-file.md#tidb_enable_collect_execution_info) configuration item or the [`tidb_enable_collect_execution_info`](/system-variables.md#tidb_enable_collect_execution_info) system variable.

</CustomContent>

<CustomContent platform="tidb-cloud">

Starting from v8.0.0, TiDB provides the `TIDB_INDEX_USAGE` table. You can use `TIDB_INDEX_USAGE` to get the usage statistics of all indexes on the current TiDB node. By default, TiDB collects these index usage statistics during SQL statement execution. You can disable this feature by turning off the [`instance.tidb_enable_collect_execution_info`](https://docs.pingcap.com/tidb/v8.0/tidb-configuration-file#tidb_enable_collect_execution_info) configuration item or the [`tidb_enable_collect_execution_info`](/system-variables.md#tidb_enable_collect_execution_info) system variable.

</CustomContent>

```sql
USE INFORMATION_SCHEMA;
DESC TIDB_INDEX_USAGE;
```

```sql
+--------------------------+-------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------------------+-------------+------+------+---------+-------+
| TABLE_SCHEMA | varchar(64) | YES | | NULL | |
| TABLE_NAME | varchar(64) | YES | | NULL | |
| INDEX_NAME | varchar(64) | YES | | NULL | |
| QUERY_TOTAL | bigint(21) | YES | | NULL | |
| KV_REQ_TOTAL | bigint(21) | YES | | NULL | |
| ROWS_ACCESS_TOTAL | bigint(21) | YES | | NULL | |
| PERCENTAGE_ACCESS_0 | bigint(21) | YES | | NULL | |
| PERCENTAGE_ACCESS_0_1 | bigint(21) | YES | | NULL | |
| PERCENTAGE_ACCESS_1_10 | bigint(21) | YES | | NULL | |
| PERCENTAGE_ACCESS_10_20 | bigint(21) | YES | | NULL | |
| PERCENTAGE_ACCESS_20_50 | bigint(21) | YES | | NULL | |
| PERCENTAGE_ACCESS_50_100 | bigint(21) | YES | | NULL | |
| PERCENTAGE_ACCESS_100 | bigint(21) | YES | | NULL | |
| LAST_ACCESS_TIME | datetime | YES | | NULL | |
+--------------------------+-------------+------+------+---------+-------+
14 rows in set (0.00 sec)
```

The columns in the `TIDB_INDEX_USAGE` table are as follows:

* `TABLE_SCHEMA`: The name of the database to which the table containing the index belongs.
* `TABLE_NAME`: The name of the table containing the index.
* `INDEX_NAME`: The name of the index.
* `QUERY_TOTAL`: The total number of statements accessing the index.
* `KV_REQ_TOTAL`: The total number of KV requests generated when accessing the index.
* `ROWS_ACCESS_TOTAL`: The total number of rows scanned when accessing the index.
* `PERCENTAGE_ACCESS_0`: The number of times the row access ratio (the percentage of accessed rows out of the total number of rows in the table) is 0.
* `PERCENTAGE_ACCESS_0_1`: The number of times the row access ratio is between 0% and 1%.
* `PERCENTAGE_ACCESS_1_10`: The number of times the row access ratio is between 1% and 10%.
* `PERCENTAGE_ACCESS_10_20`: The number of times the row access ratio is between 10% and 20%.
* `PERCENTAGE_ACCESS_20_50`: The number of times the row access ratio is between 20% and 50%.
* `PERCENTAGE_ACCESS_50_100`: The number of times the row access ratio is between 50% and 100%.
* `PERCENTAGE_ACCESS_100`: The number of times the row access ratio is 100%.
* `LAST_ACCESS_TIME`: The time of the most recent access to the index.

## CLUSTER_TIDB_INDEX_USAGE

The `TIDB_INDEX_USAGE` table only provides usage statistics of all indexes on a single TiDB node. To obtain the index usage statistics on all TiDB nodes in the cluster, you need to query the `CLUSTER_TIDB_INDEX_USAGE` table.

Compared with the `TIDB_INDEX_USAGE` table, the query result of the `CLUSTER_TIDB_INDEX_USAGE` table includes an additional `INSTANCE` field. This field displays the IP address and port of each node in the cluster, which helps you distinguish the statistics across different nodes.

```sql
USE INFORMATION_SCHEMA;
DESC CLUSTER_TIDB_INDEX_USAGE;
```

The output is as follows:

```sql
+-------------------------+-----------------------------------------------------------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------------+-----------------------------------------------------------------+------+------+---------+-------+
| INSTANCE | varchar(64) | YES | | NULL | |
| ID | bigint(21) unsigned | NO | PRI | NULL | |
| START_TIME | timestamp(6) | YES | | NULL | |
| CURRENT_SQL_DIGEST | varchar(64) | YES | | NULL | |
| CURRENT_SQL_DIGEST_TEXT | text | YES | | NULL | |
| STATE | enum('Idle','Running','LockWaiting','Committing','RollingBack') | YES | | NULL | |
| WAITING_START_TIME | timestamp(6) | YES | | NULL | |
| MEM_BUFFER_KEYS | bigint(64) | YES | | NULL | |
| MEM_BUFFER_BYTES | bigint(64) | YES | | NULL | |
| SESSION_ID | bigint(21) unsigned | YES | | NULL | |
| USER | varchar(16) | YES | | NULL | |
| DB | varchar(64) | YES | | NULL | |
| ALL_SQL_DIGESTS | text | YES | | NULL | |
| RELATED_TABLE_IDS | text | YES | | NULL | |
| WAITING_TIME | double | YES | | NULL | |
+-------------------------+-----------------------------------------------------------------+------+------+---------+-------+
15 rows in set (0.00 sec)
```

## Limitations

- The data in the `TIDB_INDEX_USAGE` table might be delayed by up to 5 minutes.
- After TiDB restarts, the data in the `TIDB_INDEX_USAGE` table is cleared.
3 changes: 3 additions & 0 deletions information-schema/information-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi
| `CLUSTER_STATEMENTS_SUMMARY` | Provides a cluster-level view of the `STATEMENTS_SUMMARY` table. This table is not available on [TiDB Serverless](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-serverless) clusters. |
| `CLUSTER_STATEMENTS_SUMMARY_HISTORY` | Provides a cluster-level view of the `STATEMENTS_SUMMARY_HISTORY` table. This table is not available on [TiDB Serverless](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-serverless) clusters. |
| `CLUSTER_TIDB_TRX` | Provides a cluster-level view of the `TIDB_TRX` table. |
| `CLUSTER_TIDB_INDEX_USAGE` | Provides a cluster-level view of the `TIDB_INDEX_USAGE` table. |
Oreoxmt marked this conversation as resolved.
Show resolved Hide resolved
| [`CLUSTER_SYSTEMINFO`](/information-schema/information-schema-cluster-systeminfo.md) | Provides details about kernel parameter configuration for servers in the cluster. This table is not applicable to TiDB Cloud. |
| [`DATA_LOCK_WAITS`](/information-schema/information-schema-data-lock-waits.md) | Provides the lock-waiting information on the TiKV server. |
| [`DDL_JOBS`](/information-schema/information-schema-ddl-jobs.md) | Provides similar output to `ADMIN SHOW DDL JOBS` |
Expand All @@ -147,6 +148,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi
| [`TIDB_INDEXES`](/information-schema/information-schema-tidb-indexes.md) | Provides index information about TiDB tables. |
| [`TIDB_SERVERS_INFO`](/information-schema/information-schema-tidb-servers-info.md) | Provides a list of TiDB servers (namely, tidb-server component) |
| [`TIDB_TRX`](/information-schema/information-schema-tidb-trx.md) | Provides the information of the transactions that are being executed on the TiDB node. |
| [`TIDB_INDEX_USAGE`](/information-schema/information-schema-tidb-index-usage.md) | Provides the information of the index usage statistics on the TiDB node. |
Oreoxmt marked this conversation as resolved.
Show resolved Hide resolved
| [`TIFLASH_REPLICA`](/information-schema/information-schema-tiflash-replica.md) | Provides details about TiFlash replicas. |
| [`TIKV_REGION_PEERS`](/information-schema/information-schema-tikv-region-peers.md) | Provides details about where regions are stored. |
| [`TIKV_REGION_STATUS`](/information-schema/information-schema-tikv-region-status.md) | Provides statistics about regions. |
Expand Down Expand Up @@ -198,6 +200,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi
| [`TIDB_INDEXES`](/information-schema/information-schema-tidb-indexes.md) | Provides index information about TiDB tables. |
| [`TIDB_SERVERS_INFO`](/information-schema/information-schema-tidb-servers-info.md) | Provides a list of TiDB servers (namely, tidb-server component) |
| [`TIDB_TRX`](/information-schema/information-schema-tidb-trx.md) | Provides the information of the transactions that are being executed on the TiDB node. |
| [`TIDB_INDEX_USAGE`](/information-schema/information-schema-tidb-index-usage.md) | Provides the information of the index usage statistics on the TiDB node. |
Oreoxmt marked this conversation as resolved.
Show resolved Hide resolved
| [`TIFLASH_REPLICA`](/information-schema/information-schema-tiflash-replica.md) | Provides details about TiFlash replicas. |
| [`TIKV_REGION_PEERS`](/information-schema/information-schema-tikv-region-peers.md) | Provides details about where regions are stored. |
| [`TIKV_REGION_STATUS`](/information-schema/information-schema-tikv-region-status.md) | Provides statistics about regions. |
Expand Down
54 changes: 54 additions & 0 deletions sys-schema.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
title: sys Schema
summary: Learn about the system tables in the `sys` schema.
---

# `sys` Schema

Starting from v8.0.0, TiDB provides the `sys` Schema. You can use the views in `sys` schema to understand the data in the system tables, [`INFORMATION_SCHEMA`](/information-schema/information-schema.md), and [`PERFORMANCE SCHEMA`](/performance-schema/performance-schema.md) of TiDB.
Oreoxmt marked this conversation as resolved.
Show resolved Hide resolved

## Manually create the `sys` schema and views

For clusters upgraded from versions earlier than v8.0.0, the `sys` schema and the views in it are not created automatically. You can manually create them using the following SQL statements:

```sql
CREATE DATABASE IF NOT EXISTS sys;
CREATE OR REPLACE VIEW sys.schema_unused_indexes AS
SELECT
table_schema as object_schema,
table_name as object_name,
index_name
FROM information_schema.cluster_tidb_index_usage
WHERE
table_schema not in ('sys', 'mysql', 'INFORMATION_SCHEMA', 'PERFORMANCE_SCHEMA') and
index_name != 'PRIMARY'
GROUP BY table_schema, table_name, index_name
HAVING
sum(last_access_time) is null;
```

## `schema_unused_index`

`schema_unused_index` records indexes that have not been used since the last start of TiDB. It includes the following columns:

- `OBJECT_SCHEMA`: The name of the database to which the table containing the index belongs.
- `OBJECT_NAME`: The name of the table containing the index.
- `INDEX_NAME`: The name of the index.

```sql
USE SYS;
DESC SCHEMA_UNUSED_INDEXES;
```

The output is as follows:

```sql
+---------------+-------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+-------------+------+------+---------+-------+
| object_schema | varchar(64) | YES | | NULL | |
| object_name | varchar(64) | YES | | NULL | |
| index_name | varchar(64) | YES | | NULL | |
+---------------+-------------+------+------+---------+-------+
3 rows in set (0.00 sec)
```
2 changes: 1 addition & 1 deletion system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -1951,7 +1951,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1;
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Type: Boolean
- Default value: `ON`
- This variable controls whether to record the execution information of each operator in the slow query log.
- This variable controls whether to record the execution information of each operator in the slow query log and whether to record the [usage statistics of indexes](/information-schema/information-schema-tidb-index-usage.md).

### tidb_enable_column_tracking <span class="version-mark">New in v5.4.0</span>

Expand Down
2 changes: 1 addition & 1 deletion tidb-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -843,7 +843,7 @@ Configuration items related to read isolation.

### `tidb_enable_collect_execution_info`

- This configuration controls whether to record the execution information of each operator in the slow query log.
- This configuration controls whether to record the execution information of each operator in the slow query log and whether to record the [usage statistics of indexes](/information-schema/information-schema-tidb-index-usage.md).
- Default value: `true`
- Before v6.1.0, this configuration is set by `enable-collect-execution-info`.

Expand Down
Loading