Skip to content

Commit

Permalink
update titan doc to provide guidline on min-blob-size (#16312)
Browse files Browse the repository at this point in the history
  • Loading branch information
hfxsd authored Jan 25, 2024
1 parent 533c83e commit 6c458e3
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 3 deletions.
2 changes: 1 addition & 1 deletion storage-engine/titan-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ By properly configuring Titan parameters, you can effectively improve database p
### `min-blob-size`
You can use [`min-blob-size`](/tikv-configuration-file.md#min-blob-size) to set the threshold for the value size to determine which data is stored in RocksDB and which in Titan's blob files. According to the test, `32KB` is a appropriate threshold that has better write throughput without scan throughput regression compared with RocksDB. If you want further improve write performance and accept scan performance regression, you can change the value to `1KB`.
You can use [`min-blob-size`](/tikv-configuration-file.md#min-blob-size) to set the threshold for the value size to determine which data is stored in RocksDB and which in Titan's blob files. According to the test, `32KB` is an appropriate threshold. It ensures that Titan's performance does not regress compared with RocksDB. However, in many scenarios, this value is not optimal. It is recommended that you refer to [Impact of `min-blob-size` on performance](/storage-engine/titan-overview.md#impact-of-min-blob-size-on-performance) to choose an appropriate value. If you want to further improve write performance and can tolerate scan performance regression, you can set it to the minimum value `1KB`.
### `blob-file-compression` and `zstd-dict-size`
Expand Down
25 changes: 23 additions & 2 deletions storage-engine/titan-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ A blob file mainly consists of blob records, meta blocks, a meta index block, an
>
> + The Key-Value pairs in the blob file are stored in order, so that when the Iterator is implemented, the sequential reading performance can be improved via prefetching.
> + Each blob record keeps a copy of the user key corresponding to the value. This way, when Titan performs Garbage Collection (GC), it can query the user key and identify whether the corresponding value is outdated. However, this process introduces some write amplification.
> + BlobFile supports compression at the blob record level. Titan supports multiple compression algorithms, such as [Snappy](https://github.com/google/snappy), [LZ4](https://github.com/lz4/lz4), and [Zstd](https://github.com/facebook/zstd). Currently, the default compression algorithm Titan uses is LZ4.
> + The Snappy compressed file must be in the [official Snappy format](https://github.com/google/snappy). Other variants of Snappy compression are not supported.
> + BlobFile supports compression at the blob record level. Titan supports several compression algorithms, such as [Snappy](https://github.com/google/snappy), [`lz4`](https://github.com/lz4/lz4), and [`zstd`](https://github.com/facebook/zstd). In versions prior to v7.6.0, the default compression algorithm is `lz4`. Starting from v7.6.0, the default compression algorithm is `zstd`.
> + The Snappy compressed file must be in the [official Snappy format](https://github.com/google/snappy). Other variants are not supported.
### TitanTableBuilder

Expand Down Expand Up @@ -130,3 +130,24 @@ Therefore, the Range Merge operation is needed to keep the number of sorted runs
### Scale out and scale in

For backward compatibility, the TiKV snapshots are still in the RocksDB format during scaling. Because the scaled nodes are all from RocksDB at the beginning, they carry the characteristics of RocksDB, such as higher compression rate than the old TiKV nodes, smaller store size, and relatively larger write amplification in compaction. These SST files in RocksDB format will be gradually converted to Titan format after compaction.

### Impact of `min-blob-size` on performance

[`min-blob-size`](/tikv-configuration-file.md#min-blob-size) determines whether a value is stored in Titan. If the value is greater than or equal to `min-blob-size`, it is stored in Titan. Otherwise, it is stored in the native RocksDB format. If `min-blob-size` is too small or too large, the performance will be affected.

The following table lists the QPS comparison of the YCSB workload based on different `min-blob-size` values. In each round of testing, the row width of the test data is equal to `min-blob-size`, so that the data is stored in Titan when Titan is enabled.

| Row width (Bytes) | `Point_Get` | `Point_Get` (Titan)| scan100 | scan100 (Titan)| scan10000 | scan10000 (Titan)| `UPDATE` | `UPDATE` (Titan) |
| ---------------- | ---------| -------------- | --------| ------------- | --------- | --------------- | ------ | ------------ |
| 1KB | 139255 | 140486 | 25171 | 21854 | 533 | 175 | 17913 | 30767 |
| 2KB | 114201 |124075 | 12466 |11552 |249 |131 |10369 | 27188 |
| 4KB | 92385 | 103811 | 7918 | 5937 | 131 | 87 | 5327 | 22653 |
| 8KB |104380 | 130647 | 7365 | 5402 | 86.6 | 68 | 3180 | 16745 |
| 16KB | 54234 | 54600 | 4937 | 5174 | 55.4 | 58.9 |1753 | 10120 |
| 32KB | 31035 |31052 | 2705 | 3422 | 38 | 45.3 | 984 | 5844 |

> **Note:**
>
> `scan100` means scanning 100 records, and `scan10000` means scanning 10000 records.
From the table, you can see that when the row width is `16KB`, Titan outperforms RocksDB in all YCSB workloads. However, in some extreme scenarios with heavy scan loads, such as running Dumpling, Titan's performance with a row width of `16KB` decreases by 10%. Therefore, if the workload is mainly write and point read, it is recommended that you set `min-blob-size` to `1KB`. If the workload contains a large number of scans, it is recommended that you set `min-blob-size` to at least `16KB`.

0 comments on commit 6c458e3

Please sign in to comment.