Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update system-variables tidb_ddl_reorg_batch_size (#18616) #18706

Merged
merged 7 commits into from
Aug 26, 2024
Merged

Update system-variables tidb_ddl_reorg_batch_size (#18616) #18706

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
6ac155b
Update system-variables `tidb_ddl_reorg_batch_size`.md
Frank945946 Aug 19, 2024
5a06cc9
Update system-variables.md
Frank945946 Aug 19, 2024
ea62bea
Update system-variables.md
hfxsd Aug 20, 2024
8e05bab
Update system-variables.md
hfxsd Aug 21, 2024
8d23f85
Apply suggestions from code review
hfxsd Aug 21, 2024
9334b14
Apply suggestions from code review
hfxsd Aug 26, 2024
987d88e
Update system-variables.md
hfxsd Aug 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions system-variables.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1640,8 +1640,8 @@
- Range: `[32, 10240]`
- Unit: Rows
- This variable is used to set the batch size during the `re-organize` phase of the DDL operation. For example, when TiDB executes the `ADD INDEX` operation, the index data needs to backfilled by `tidb_ddl_reorg_worker_cnt` (the number) concurrent workers. Each worker backfills the index data in batches.
- If many updating operations such as `UPDATE` and `REPLACE` exist during the `ADD INDEX` operation, a larger batch size indicates a larger probability of transaction conflicts. In this case, you need to adjust the batch size to a smaller value. The minimum value is 32.
- If the transaction conflict does not exist, you can set the batch size to a large value (consider the worker count. See [Interaction Test on Online Workloads and `ADD INDEX` Operations](https://docs.pingcap.com/tidb/stable/online-workloads-and-add-index-operations) for reference). This can increase the speed of the backfilling data, but the write pressure on TiKV also becomes higher.
- If `tidb_ddl_enable_fast_reorg` is set to `OFF`, `ADD INDEX` is executed as a transaction. If there are many update operations such as `UPDATE` and `REPLACE` in the target columns during the `ADD INDEX` execution, a larger batch size indicates a larger probability of transaction conflicts. In this case, it is recommended that you set the batch size to a smaller value. The minimum value is 32.

Check warning on line 1643 in system-variables.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[PingCAP.Ambiguous] Consider using a clearer word than 'many' because it may cause confusion.

Raw Output:
{"message": "[PingCAP.Ambiguous] Consider using a clearer word than 'many' because it may cause confusion.", "location": {"path": "system-variables.md", "range": {"start": {"line": 1643, "column": 111}}}, "severity": "INFO"}
- If the transaction conflict does not exist, or if `tidb_ddl_enable_fast_reorg` is set to `ON`, you can set the batch size to a large value. This makes data backfilling faster but also increases the write pressure on TiKV. For a proper batch size, you also need to refer to the value of `tidb_ddl_reorg_worker_cnt`. See [Interaction Test on Online Workloads and `ADD INDEX` Operations](https://docs.pingcap.com/tidb/dev/online-workloads-and-add-index-operations) for reference.

### tidb_ddl_reorg_priority

Expand Down
Loading