add v7.5.0 dbaas test report for sysbench/tpcc (#15362)

TOC-tidb-cloud.md

@@ -315,8 +315,12 @@
   - [Limited SQL Features on TiDB Cloud](/tidb-cloud/limited-sql-features.md)
   - [TiDB Limitations](/tidb-limitations.md)
   - Benchmarks
-    - [TPC-C Performance Test Report](/tidb-cloud/v7.1.0-performance-benchmarking-with-tpcc.md)
-    - [Sysbench Performance Test Report](/tidb-cloud/v7.1.0-performance-benchmarking-with-sysbench.md)
+    - TiDB v7.5
+      - [TPC-C Performance Test Report](/tidb-cloud/v7.5.0-performance-benchmarking-with-tpcc.md)
+      - [Sysbench Performance Test Report](/tidb-cloud/v7.5.0-performance-benchmarking-with-sysbench.md)
+    - TiDB v7.1
+      - [TPC-C Performance Test Report](/tidb-cloud/v7.1.0-performance-benchmarking-with-tpcc.md)
+      - [Sysbench Performance Test Report](/tidb-cloud/v7.1.0-performance-benchmarking-with-sysbench.md)
   - SQL
     - [Explore SQL with TiDB](/basic-sql-operations.md)
     - SQL Language Structure and Syntax

tidb-cloud/v7.1.0-performance-benchmarking-with-sysbench.md

@@ -1,11 +1,11 @@
 ---
-title: TiDB Cloud Sysbench Performance Test Report
-summary: Introduce the Sysbench performance test results for TiDB Cloud.
+title: TiDB Cloud Sysbench Performance Test Report for TiDB v7.1.0
+summary: Introduce the Sysbench performance test results for a TiDB Dedicated cluster with the TiDB version of v7.1.0.
 ---
 
-# TiDB Cloud Sysbench Performance Test Report
+# TiDB Cloud Sysbench Performance Test Report for TiDB v7.1.0
 
-This document provides the Sysbench performance test steps and results for a TiDB Dedicated cluster with the TiDB version of v7.1.0. This report can also be used as a reference for the performance of TiDB v7.1.0 clusters.
+This document provides the Sysbench performance test steps and results for a TiDB Dedicated cluster with the TiDB version of v7.1.0. This report can also be used as a reference for the performance of TiDB Self-Hosted v7.1.0 clusters.
 
 ## Test overview
 

tidb-cloud/v7.1.0-performance-benchmarking-with-tpcc.md

@@ -1,9 +1,9 @@
 ---
-title: TiDB Cloud TPC-C Performance Test Report
-summary: Introduce the TPC-C performance test results for TiDB Cloud.
+title: TiDB Cloud TPC-C Performance Test Report for TiDB v7.1.0
+summary: Introduce the TPC-C performance test results for a TiDB Dedicated cluster with the TiDB version of v7.1.0.
 ---
 
-# TiDB Cloud TPC-C Performance Test Report
+# TiDB Cloud TPC-C Performance Test Report for TiDB v7.1.0
 
 This document provides the TPC-C performance test steps and results for a TiDB Dedicated cluster with the TiDB version of v7.1.0. This report can also be used as a reference for the performance of TiDB Self-Hosted v7.1.0 clusters.
 

tidb-cloud/v7.5.0-performance-benchmarking-with-sysbench.md

@@ -0,0 +1,175 @@
+---
+title: TiDB Cloud Sysbench Performance Test Report for TiDB v7.5.0
+summary: Introduce the Sysbench performance test results for a TiDB Dedicated cluster with the TiDB version of v7.5.0.
+---
+
+# TiDB Cloud Sysbench Performance Test Report for TiDB v7.5.0
+
+This document provides the Sysbench performance test steps and results for a TiDB Dedicated cluster with the TiDB version of v7.5.0. This report can also be used as a reference for the performance of TiDB Self-Hosted v7.5.0 clusters.
+
+## Test overview
+
+This test aims at showing the Sysbench performance of TiDB v7.5.0 in the Online Transactional Processing (OLTP) scenario.
+
+## Test environment
+
+### TiDB cluster
+
+The test is conducted on a TiDB cluster with the following settings:
+
+- Cluster type: [TiDB Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-dedicated)
+- Cluster version: v7.5.0
+- Cloud provider: AWS (us-west-2)
+- Cluster configuration:
+
+   | Node type | Node size | Node quantity | Node storage |
+   |:----------|:----------|:----------|:----------|
+   | TiDB      | 16 vCPU, 32 GiB | 2 | N/A |
+   | TiKV      | 16 vCPU, 64 GiB | 3 | 1000 GiB |
+
+### Parameter configuration
+
+> **Note:**
+>
+> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help.
+
+The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests based on different workloads with the following `prefill-for-recycle` configuration:
+
+- For the `oltp_point_select` workload, use the default value of the [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) parameter:
+
+    ```yaml
+    raft-engine.prefill-for-recycle = false
+    ```
+
+- For `oltp_insert`, `oltp_read_write` , `oltp_update_index`, and `oltp_update_non_index` workloads, enable the [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) parameter:
+
+    ```yaml
+    raft-engine.prefill-for-recycle = true
+    ```
+
+### Benchmark executor
+
+The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows:
+
+- Machine type: Amazon EC2 (us-west-2)
+- Instance type: c6a.2xlarge
+- Sysbench version: sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)
+
+## Test steps
+
+This section introduces how to perform the Sysbench performance test step by step.
+
+1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Dedicated cluster that meets the [test environment](#tidb-cluster) requirements.
+
+   For more information, see [Create a TiDB Dedicated cluster](/tidb-cloud/create-tidb-cluster.md).
+
+2. On the benchmark executor, connect to the newly created cluster and create a database named `sbtest`.
+
+   To connect to the cluster, see [Connect to TiDB Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md).
+
+   To create the `sbtest` database, execute the following SQL statement:
+
+   ```sql
+   CREATE DATABASE sbtest;
+   ```
+
+3. Load Sysbench data to the `sbtest` database.
+
+   1. The test in this document is implemented based on [sysbench](https://github.com/akopytov/sysbench). To install sysbench, see [Building and installing from source](https://github.com/akopytov/sysbench#building-and-installing-from-source).
+
+   2. Run the following `sysbench prepare` command to import 32 tables and 10,000,000 rows to the `sbtest` database. Replace `${HOST}`, `${PORT}`, `${THREAD}`, and `${PASSWORD}` with your actual values.
+
+      ```shell
+      sysbench oltp_common \
+         --threads=${THREAD} \
+         --db-driver=mysql \
+         --mysql-db=sbtest \
+         --mysql-host=${HOST} \
+         --mysql-port=${PORT} \
+         --mysql-user=root \
+         --mysql-password=${PASSWORD} \
+         prepare --tables=32 --table-size=10000000
+      ```
+
+4. Run the following `sysbench run` command to conduct Sysbench performance tests on different workloads. This document conducts tests on five workloads: `oltp_point_select`, `oltp_read_write`, `oltp_update_non_index`, `oltp_update_index`, and `oltp_insert`. For each workload, this document conducts three tests with the `${THREAD}` value of `100`, `200`, and `400`. For each concurrency, the test takes 20 minutes.
+
+   ```shell
+   sysbench ${WORKLOAD} run \
+      --mysql-host=${HOST} \
+      --mysql-port=${PORT} \
+      --mysql-user=root \
+      --db-driver=mysql \
+      --mysql-db=sbtest \
+      --threads=${THREAD} \
+      --time=1200 \
+      --report-interval=10 \
+      --tables=32 \
+      --table-size=10000000 \
+      --mysql-ignore-errors=1062,2013,8028,9007 \
+      --auto-inc=false \
+      --mysql-password=${PASSWORD}
+   ```
+
+## Test results
+
+This section introduces the Sysbench performance of v7.5.0 in the [test environment](#test-environment).
+
+### Point select performance
+
+The performance on the `oltp_point_select` workload is as follows:
+
+| Threads |  TPS | 95% latency (ms)|
+|:--------|:----------|:----------|
+| 100 | 64,810  | 2.03 |
+| 200 | 118,651 | 2.22 |
+| 400 | 153,580 | 3.96 |
+
+![Sysbench point select performance](/media/tidb-cloud/v7.5.0_oltp_point_select.png)
+
+### Read write performance
+
+The performance on the `oltp_read_write` workload is as follows:
+
+| Threads |  TPS | 95% latency (ms)|
+|:--------|:----------|:----------|
+| 100 | 2,134 | 54.8 |
+| 200 | 3,020 | 99.3 |
+| 400 | 3,251 | 193  |
+
+![Sysbench read write performance](/media/tidb-cloud/v7.5.0_oltp_read_write.png)
+
+### Update non-index performance
+
+The performance on the `oltp_update_non_index` workload is as follows:
+
+| Threads |  TPS | 95% latency (ms)|
+|:--------|:----------|:----------|
+| 100 | 10,567 | 11.7 |
+| 200 | 20,223 | 13.0 |
+| 400 | 34,011 | 14.7 |
+
+![Sysbench update non-index performance](/media/tidb-cloud/v7.5.0_oltp_update_non_index.png)
+
+### Update index performance
+
+The performance on the `oltp_update_index` workload is as follows:
+
+| Threads |  TPS | 95% latency (ms)|
+|:--------|:----------|:----------|
+| 100 | 8,896  | 14.7 |
+| 200 | 1,3718 | 19.0 |
+| 400 | 2,0377 | 26.9 |
+
+![Sysbench update index performance](/media/tidb-cloud/v7.5.0_oltp_update_index.png)
+
+### Insert performance
+
+The performance on the `oltp_insert` workload is as follows:
+
+| Threads |  TPS | 95% latency (ms)|
+|:--------|:----------|:----------|
+| 100 | 15,132 | 8.58 |
+| 200 | 24,756 | 10.8 |
+| 400 | 37,247 | 16.4 |
+
+![Sysbench insert performance](/media/tidb-cloud/v7.5.0_oltp_insert.png)

tidb-cloud/v7.5.0-performance-benchmarking-with-tpcc.md

@@ -0,0 +1,111 @@
+---
+title: TiDB Cloud TPC-C Performance Test Report for TiDB v7.5.0
+summary: Introduce the TPC-C performance test results for a TiDB Dedicated cluster with the TiDB version of v7.5.0.
+---
+
+# TiDB Cloud TPC-C Performance Test Report for TiDB v7.5.0
+
+This document provides the TPC-C performance test steps and results for a TiDB Dedicated cluster with the TiDB version of v7.5.0. This report can also be used as a reference for the performance of TiDB Self-Hosted v7.5.0 clusters.
+
+## Test overview
+
+This test aims at showing the TPC-C performance of TiDB v7.5.0 in the Online Transactional Processing (OLTP) scenario.
+
+## Test environment
+
+### TiDB cluster
+
+The test is conducted on a TiDB cluster with the following settings:
+
+- Cluster type: [TiDB Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-dedicated)
+- Cluster version: v7.5.0
+- Cloud provider: AWS (us-west-2)
+- Cluster configuration:
+
+   | Node type | Node size | Node quantity | Node storage |
+   |:----------|:----------|:----------|:----------|
+   | TiDB      | 16 vCPU, 32 GiB | 2 | N/A |
+   | TiKV      | 16 vCPU, 64 GiB | 3 | 1000 GiB |
+
+### Benchmark executor
+
+The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows:
+
+- Machine type: Amazon EC2 (us-west-2)
+- Instance type: c6a.2xlarge
+
+## Test steps
+
+This section introduces how to perform the TPC-C performance test step by step.
+
+1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Dedicated cluster that meets the [test environment](#tidb-cluster) requirements.
+
+   For more information, see [Create a TiDB Dedicated cluster](/tidb-cloud/create-tidb-cluster.md).
+
+2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`.
+
+   To connect to the cluster, see [Connect to TiDB Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md).
+
+   To create the `tpcc` database, execute the following SQL statement:
+
+   ```sql
+   CREATE DATABASE tpcc;
+   ```
+
+3. Load TPC-C data to the `tpcc` database.
+
+   1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command:
+
+      ```shell
+      curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh
+      ```
+
+   2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`.
+
+      ```shell
+      go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error
+      ```
+
+4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test:
+
+   ```sql
+   ANALYZE TABLE customer;
+   ANALYZE TABLE district;
+   ANALYZE TABLE history;
+   ANALYZE TABLE item;
+   ANALYZE TABLE new_order;
+   ANALYZE TABLE order_line;
+   ANALYZE TABLE orders;
+   ANALYZE TABLE stock;
+   ANALYZE TABLE warehouse;
+   ```
+
+   To accelerate the collection of statistics, execute the following SQL statements before collecting:
+
+   ```sql
+   SET tidb_build_stats_concurrency=16;
+   SET tidb_distsql_scan_concurrency=16;
+   SET tidb_index_serial_scan_concurrency=16;
+   ```
+
+5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Dedicated cluster. For each concurrency, the test takes two hours.
+
+   ```shell
+   go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error
+   ```
+
+6. Extract the tpmC data of `NEW_ORDER` from the result.
+
+   TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute.
+
+## Test results
+
+The TPC-C performance of v7.5.0 in the [test environment](#test-environment) is as follows:
+
+| Threads |  v7.5.0 tpmC |
+|:--------|:----------|
+| 50  | 37,443 |
+| 100 | 67,899 |
+| 200 | 93,038 |
+
+![TPC-C](/media/tidb-cloud/v7.5.0_tpcc.png)