Skip to content

Commit

Permalink
tikv config: add in memory pessismitic lock size limit config (pingca…
Browse files Browse the repository at this point in the history
  • Loading branch information
cfzjywxk authored Oct 18, 2024
1 parent 7050e53 commit 50d6236
Show file tree
Hide file tree
Showing 3 changed files with 44 additions and 1 deletion.
2 changes: 2 additions & 0 deletions dynamic-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,8 @@ The following TiKV configuration items can be modified dynamically:
| `pessimistic-txn.wake-up-delay-duration` | The duration after which a pessimistic transaction is woken up |
| `pessimistic-txn.pipelined` | Determines whether to enable the pipelined pessimistic locking process |
| `pessimistic-txn.in-memory` | Determines whether to enable the in-memory pessimistic lock |
| `pessimistic-txn.in-memory-peer-size-limit` | Controls the memory usage limit for in-memory pessimistic locks in a Region |
| `pessimistic-txn.in-memory-instance-size-limit` | Controls the memory usage limit for in-memory pessimistic locks in a TiKV instance |
| `quota.foreground-cpu-time` | The soft limit on the CPU resources used by TiKV foreground to process read and write requests |
| `quota.foreground-write-bandwidth` | The soft limit on the bandwidth with which foreground transactions write data |
| `quota.foreground-read-bandwidth` | The soft limit on the bandwidth with which foreground transactions and the Coprocessor read data |
Expand Down
31 changes: 30 additions & 1 deletion pessimistic-transaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,17 @@ If the application logic relies on the locking or lock waiting mechanisms, or if
In v6.0.0, TiKV introduces the feature of in-memory pessimistic lock. When this feature is enabled, pessimistic locks are usually stored in the memory of the Region leader only, and are not persisted to disk or replicated through Raft to other replicas. This feature can greatly reduce the overhead of acquiring pessimistic locks and improve the throughput of pessimistic transactions.
When the memory usage of in-memory pessimistic locks exceeds the memory threshold of the Region or the TiKV node, the acquisition of pessimistic locks turns to the [pipelined locking process](#pipelined-locking-process). When the Region is merged or the leader is transferred, to avoid the loss of the pessimistic lock, TiKV writes the in-memory pessimistic lock to disk and replicates it to other replicas.
<CustomContent platform="tidb">
When the memory usage of in-memory pessimistic locks exceeds the memory threshold of the [Region](/tikv-configuration-file.md#in-memory-peer-size-limit-new-in-v840) or the [TiKV node](/tikv-configuration-file.md#in-memory-instance-size-limit-new-in-v840), the acquisition of pessimistic locks turns to the [pipelined locking process](#pipelined-locking-process). When the Region is merged or the leader is transferred, to avoid the loss of the pessimistic lock, TiKV writes the in-memory pessimistic lock to disk and replicates it to other replicas.
</CustomContent>
<CustomContent platform="tidb-cloud">
When the memory usage of in-memory pessimistic locks exceeds the memory threshold of the [Region](https://docs.pingcap.com/tidb/dev/tikv-configuration-file#in-memory-peer-size-limit-new-in-v840) or the [TiKV node](https://docs.pingcap.com/tidb/dev/tikv-configuration-file#in-memory-instance-size-limit-new-in-v840), the acquisition of pessimistic locks turns to the [pipelined locking process](#pipelined-locking-process). When the Region is merged or the leader is transferred, to avoid the loss of the pessimistic lock, TiKV writes the in-memory pessimistic lock to disk and replicates it to other replicas.
</CustomContent>
The in-memory pessimistic lock performs similarly to the pipelined locking process, which does not affect the lock acquisition when the cluster is healthy. However, when network isolation occurs in TiKV or a TiKV node is down, the acquired pessimistic lock might be lost.
Expand All @@ -220,3 +230,22 @@ To dynamically disable this feature, modify the TiKV configuration dynamically:
```sql
set config tikv pessimistic-txn.in-memory='false';
```
<CustomContent platform="tidb">
Starting from v8.4.0, you can configure the memory usage limit for in-memory pessimistic locks in a Region or a TiKV instance using [`pessimistic-txn.in-memory-peer-size-limit`](/tikv-configuration-file.md#in-memory-peer-size-limit-new-in-v840) or [`pessimistic-txn.in-memory-instance-size-limit`](/tikv-configuration-file.md#in-memory-instance-size-limit-new-in-v840):
```toml
[pessimistic-txn]
in-memory-peer-size-limit = "512KiB"
in-memory-instance-size-limit = "100MiB"
```
To dynamically modify these limits, [modify TiKV configuration dynamically](/dynamic-config.md#modify-tikv-configuration-dynamically) as follows:
```sql
SET CONFIG tikv `pessimistic-txn.in-memory-peer-size-limit`="512KiB";
SET CONFIG tikv `pessimistic-txn.in-memory-instance-size-limit`="100MiB";
```
</CustomContent>
12 changes: 12 additions & 0 deletions tikv-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -2321,6 +2321,18 @@ For pessimistic transaction usage, refer to [TiDB Pessimistic Transaction Mode](
+ Default value: `true`
+ Note that `in-memory` takes effect only when the value of `pipelined` is `true`.

### `in-memory-peer-size-limit` <span class="version-mark">New in v8.4.0</span>

+ Controls the memory usage limit for [in-memory pessimistic locks](/pessimistic-transaction.md#in-memory-pessimistic-lock) in a Region. When this limit is exceeded, TiKV writes pessimistic locks persistently.
+ Default value: `512KiB`
+ Unit: KiB|MiB|GiB

### `in-memory-instance-size-limit` <span class="version-mark">New in v8.4.0</span>

+ Controls the memory usage limit for [in-memory pessimistic locks](/pessimistic-transaction.md#in-memory-pessimistic-lock) in a TiKV instance. When this limit is exceeded, TiKV writes pessimistic locks persistently.
+ Default value: `100MiB`
+ Unit: KiB|MiB|GiB

## quota

Configuration items related to Quota Limiter.
Expand Down

0 comments on commit 50d6236

Please sign in to comment.