Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update system-variables tidb_ddl_reorg_batch_size #18616

Merged
merged 7 commits into from
Aug 26, 2024
Merged

Update system-variables tidb_ddl_reorg_batch_size #18616

Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5063033
Update system-variables `tidb_ddl_reorg_batch_size`.md
Frank945946 Aug 19, 2024
975c693
Update system-variables.md
Frank945946 Aug 19, 2024
8c55b87
Update system-variables.md
hfxsd Aug 20, 2024
f86fc1b
Update system-variables.md
hfxsd Aug 21, 2024
ee8fc45
Apply suggestions from code review
hfxsd Aug 21, 2024
e236140
Apply suggestions from code review
hfxsd Aug 26, 2024
0f552f6
Update system-variables.md
hfxsd Aug 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions system-variables.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1694,8 +1694,8 @@
- Range: `[32, 10240]`
- Unit: Rows
- This variable is used to set the batch size during the `re-organize` phase of the DDL operation. For example, when TiDB executes the `ADD INDEX` operation, the index data needs to backfilled by `tidb_ddl_reorg_worker_cnt` (the number) concurrent workers. Each worker backfills the index data in batches.
- If many updating operations such as `UPDATE` and `REPLACE` exist during the `ADD INDEX` operation, a larger batch size indicates a larger probability of transaction conflicts. In this case, you need to adjust the batch size to a smaller value. The minimum value is 32.
- If the transaction conflict does not exist, you can set the batch size to a large value (consider the worker count. See [Interaction Test on Online Workloads and `ADD INDEX` Operations](https://docs.pingcap.com/tidb/stable/online-workloads-and-add-index-operations) for reference). This can increase the speed of the backfilling data, but the write pressure on TiKV also becomes higher.
- If `tidb_ddl_enable_fast_reorg` is set to `Off`, `ADD INDEX` is executed as a transactional. If there are many updating operations such as `UPDATE` and `REPLACE` exist during the `ADD INDEX` operation, a larger batch size indicates a larger probability of transaction conflicts. In this case, you need to adjust the batch size to a smaller value. The minimum value is 32.

Check warning on line 1697 in system-variables.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[PingCAP.Ambiguous] Consider using a clearer word than 'many' because it may cause confusion.

Raw Output:
{"message": "[PingCAP.Ambiguous] Consider using a clearer word than 'many' because it may cause confusion.", "location": {"path": "system-variables.md", "range": {"start": {"line": 1697, "column": 113}}}, "severity": "INFO"}
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
- If the transaction conflict does not exist, or if `tidb_ddl_enable_fast_reorg` is set to `ON`, you can set the batch size to a large value. In this case, consider the worker count. See [Interaction Test on Online Workloads and `ADD INDEX` Operations](/benchmark/online-workloads-and-add-index-operations.md) for reference). This can increase the speed of the backfilling data, but the write pressure on TiKV also becomes higher.
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

### tidb_ddl_reorg_priority

Expand Down
Loading