From 08ff48f766488457e201673c1087a64caed2adc6 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 15:31:18 +0800 Subject: [PATCH 01/19] Update tiup-cluster-topology-reference.md --- tiup/tiup-cluster-topology-reference.md | 44 +++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index c62bb83df8f1b..7a6bc3e893449 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -463,6 +463,50 @@ tiproxy_servers: - host: 10.0.1.22 ``` +### `kvcdc_servers` + +`kvcdc_servers` specifies the machines to which the TiKV-CDC services are deployed. It also specifies the service configuration on each machine. `kvcdc_servers` is an array. Each array element contains the following fields: + +- `addr`: The listening address of TiKV-CDC, the HTTP API address, and the Prometheus address of the TiKV-CDC service. The default value is 127.0.0.1:8600. + +- `advertise-addr`: The advertised address via which clients access TiKV-CDC. If unspecified, the value is the same as addr. + +- `pd`: A comma-separated list of PD endpoints. + +- `config`: The address of the configuration file that TiKV-CDC uses (optional). + +- `data-dir`: Specifies the directory that TiKV-CDC uses to store temporary files for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). + +- `gc-ttl`: The TTL (Time To Live, in seconds) of the service level GC safepoint in PD set by TiKV-CDC (optional). It’s the duration that replication tasks can suspend, defaults to 86400, i.e. 24 hours. Note that suspending of replication task will affect the progress of TiKV garbage collection. The longer of gc-ttl, the longer a changefeed can be paused, but at the same time more obsolete data will be kept and larger space will be occupied. Vice versa. + +- `log-file`: The path to which logs are output when the TiKV-CDC process is running (optional). If this parameter is not specified, logs are written to the standard output (stdout). + +- `log-level`: The log level when the TiKV-CDC process is running (optional). The default value is “info”. + +- `ca`: The path of the CA certificate file in PEM format for TLS connection (optional). + +- `cert`: The path of the certificate file in PEM format for TLS connection (optional). + +- `key`: The path of the private key file in PEM format for TLS connection (optional). + +- `cert-allowed-cn`: Specifies to verify caller’s identity (certificate Common Name, optional). Use comma to separate multiple CN. + +Among the above fields, you cannot modify the following configured fields after the deployment: + +- `host` +- `port` +- `deploy_dir` +- `data_dir` +- `log_dir` + +A `kvcdc_servers` configuration example is as follows: + +```yaml +kvcdc_servers: + - host: 10.0.1.21 + - host: 10.0.1.22 +``` + ### `cdc_servers` `cdc_servers` specifies the machines to which the TiCDC services are deployed. It also specifies the service configuration on each machine. `cdc_servers` is an array. Each array element contains the following fields: From c200fa2ce69d17209f76a0c94327ad2fc315534c Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 15:48:42 +0800 Subject: [PATCH 02/19] Update tiup-cluster-topology-reference.md --- tiup/tiup-cluster-topology-reference.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index 7a6bc3e893449..33e74c25e3153 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -477,11 +477,11 @@ tiproxy_servers: - `data-dir`: Specifies the directory that TiKV-CDC uses to store temporary files for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). -- `gc-ttl`: The TTL (Time To Live, in seconds) of the service level GC safepoint in PD set by TiKV-CDC (optional). It’s the duration that replication tasks can suspend, defaults to 86400, i.e. 24 hours. Note that suspending of replication task will affect the progress of TiKV garbage collection. The longer of gc-ttl, the longer a changefeed can be paused, but at the same time more obsolete data will be kept and larger space will be occupied. Vice versa. +- `gc-ttl`: The TTL (Time To Live, in seconds) of the service level GC safepoint in PD set by TiKV-CDC (optional). It's the duration that replication tasks can suspend, defaults to 86400, that is 24 hours. Note that suspending of replication task will affect the progress of TiKV garbage collection. The longer of gc-ttl, the longer a changefeed can be paused, but at the same time more obsolete data will be kept and larger space will be occupied. Vice versa. - `log-file`: The path to which logs are output when the TiKV-CDC process is running (optional). If this parameter is not specified, logs are written to the standard output (stdout). -- `log-level`: The log level when the TiKV-CDC process is running (optional). The default value is “info”. +- `log-level`: The log level when the TiKV-CDC process is running (optional). The default value is "info". - `ca`: The path of the CA certificate file in PEM format for TLS connection (optional). @@ -489,7 +489,7 @@ tiproxy_servers: - `key`: The path of the private key file in PEM format for TLS connection (optional). -- `cert-allowed-cn`: Specifies to verify caller’s identity (certificate Common Name, optional). Use comma to separate multiple CN. +- `cert-allowed-cn`: Specifies to verify caller's identity (certificate Common Name, optional). Use comma to separate multiple CN. Among the above fields, you cannot modify the following configured fields after the deployment: From b4f5a6b85542b42407e8e36122a1db788b70e6ad Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 16:56:16 +0800 Subject: [PATCH 03/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index 33e74c25e3153..aa6d947bca1bb 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -465,7 +465,7 @@ tiproxy_servers: ### `kvcdc_servers` -`kvcdc_servers` specifies the machines to which the TiKV-CDC services are deployed. It also specifies the service configuration on each machine. `kvcdc_servers` is an array. Each array element contains the following fields: +`kvcdc_servers` specifies the machines to which the [TiKV-CDC](https://tikv.org/docs/7.1/concepts/explore-tikv-features/cdc/cdc/) services are deployed. It also specifies the service configuration on each machine. `kvcdc_servers` is an array. Each array element contains the following fields: - `addr`: The listening address of TiKV-CDC, the HTTP API address, and the Prometheus address of the TiKV-CDC service. The default value is 127.0.0.1:8600. From 3768e540d4eac24ae67fb0fe7d0f9801cfed8b13 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 16:56:33 +0800 Subject: [PATCH 04/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index aa6d947bca1bb..d063b6daaba0a 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -467,7 +467,7 @@ tiproxy_servers: `kvcdc_servers` specifies the machines to which the [TiKV-CDC](https://tikv.org/docs/7.1/concepts/explore-tikv-features/cdc/cdc/) services are deployed. It also specifies the service configuration on each machine. `kvcdc_servers` is an array. Each array element contains the following fields: -- `addr`: The listening address of TiKV-CDC, the HTTP API address, and the Prometheus address of the TiKV-CDC service. The default value is 127.0.0.1:8600. +- `addr`: The listening address of TiKV-CDC, the HTTP API address, and the Prometheus address of the TiKV-CDC service. The default value is `127.0.0.1:8600`. - `advertise-addr`: The advertised address via which clients access TiKV-CDC. If unspecified, the value is the same as addr. From e03caaae89dcc5c9de952a447bfae61ca6a1e3b5 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 16:56:55 +0800 Subject: [PATCH 05/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index d063b6daaba0a..f06714790daff 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -469,7 +469,7 @@ tiproxy_servers: - `addr`: The listening address of TiKV-CDC, the HTTP API address, and the Prometheus address of the TiKV-CDC service. The default value is `127.0.0.1:8600`. -- `advertise-addr`: The advertised address via which clients access TiKV-CDC. If unspecified, the value is the same as addr. +- `advertise-addr`: The advertised address via which clients access TiKV-CDC. If not specified, the value is the same as that of `addr`. - `pd`: A comma-separated list of PD endpoints. From eb4fffb56c58eed2af153551966b79d2df54cdc4 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 16:57:09 +0800 Subject: [PATCH 06/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index f06714790daff..42013e94a475c 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -475,7 +475,7 @@ tiproxy_servers: - `config`: The address of the configuration file that TiKV-CDC uses (optional). -- `data-dir`: Specifies the directory that TiKV-CDC uses to store temporary files for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). +- `data-dir`: The directory that TiKV-CDC uses to store temporary files for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). - `gc-ttl`: The TTL (Time To Live, in seconds) of the service level GC safepoint in PD set by TiKV-CDC (optional). It's the duration that replication tasks can suspend, defaults to 86400, that is 24 hours. Note that suspending of replication task will affect the progress of TiKV garbage collection. The longer of gc-ttl, the longer a changefeed can be paused, but at the same time more obsolete data will be kept and larger space will be occupied. Vice versa. From 13ba4f87619d29a5141785435775ce1dc49bd7dd Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 16:58:33 +0800 Subject: [PATCH 07/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index 42013e94a475c..f07e86c108e86 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -477,7 +477,7 @@ tiproxy_servers: - `data-dir`: The directory that TiKV-CDC uses to store temporary files for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). -- `gc-ttl`: The TTL (Time To Live, in seconds) of the service level GC safepoint in PD set by TiKV-CDC (optional). It's the duration that replication tasks can suspend, defaults to 86400, that is 24 hours. Note that suspending of replication task will affect the progress of TiKV garbage collection. The longer of gc-ttl, the longer a changefeed can be paused, but at the same time more obsolete data will be kept and larger space will be occupied. Vice versa. +- `gc-ttl`: The TTL (Time to Live, in seconds) of the service-level GC safepoint in PD set by TiKV-CDC (optional). It is the duration that replication tasks can be suspended, defaulting to `86400`, which is 24 hours. Note that suspending replication tasks affects the progress of TiKV garbage collection. The longer the `gc-ttl`, the longer changefeeds can be paused, but at the same time, more obsolete data will be kept and occupy more space. Vice versa. - `log-file`: The path to which logs are output when the TiKV-CDC process is running (optional). If this parameter is not specified, logs are written to the standard output (stdout). From 555c97ad78d69fb7fdd36cfd9b05be15494dc559 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 16:58:59 +0800 Subject: [PATCH 08/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index f07e86c108e86..f2041f39b7f11 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -479,7 +479,7 @@ tiproxy_servers: - `gc-ttl`: The TTL (Time to Live, in seconds) of the service-level GC safepoint in PD set by TiKV-CDC (optional). It is the duration that replication tasks can be suspended, defaulting to `86400`, which is 24 hours. Note that suspending replication tasks affects the progress of TiKV garbage collection. The longer the `gc-ttl`, the longer changefeeds can be paused, but at the same time, more obsolete data will be kept and occupy more space. Vice versa. -- `log-file`: The path to which logs are output when the TiKV-CDC process is running (optional). If this parameter is not specified, logs are written to the standard output (stdout). +- `log-file`: The path to which logs are output when the TiKV-CDC process is running (optional). If not specified, logs are written to the standard output (stdout). - `log-level`: The log level when the TiKV-CDC process is running (optional). The default value is "info". From 1e332ff69272a32aea6efd9918c953196b7d9eaf Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 16:59:43 +0800 Subject: [PATCH 09/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index f2041f39b7f11..2acc5bda63696 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -489,7 +489,7 @@ tiproxy_servers: - `key`: The path of the private key file in PEM format for TLS connection (optional). -- `cert-allowed-cn`: Specifies to verify caller's identity (certificate Common Name, optional). Use comma to separate multiple CN. +- `cert-allowed-cn`: Controls whether to verify the caller's identity (certificate Common Name, optional). Use commas to separate multiple CNs. Among the above fields, you cannot modify the following configured fields after the deployment: From 11130d324b6914277e4fa0d2dfcc59e21202cf8c Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 17:01:29 +0800 Subject: [PATCH 10/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index 2acc5bda63696..c96a0881eb148 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -491,7 +491,7 @@ tiproxy_servers: - `cert-allowed-cn`: Controls whether to verify the caller's identity (certificate Common Name, optional). Use commas to separate multiple CNs. -Among the above fields, you cannot modify the following configured fields after the deployment: +For the above fields, you cannot modify these configured fields after the deployment: - `host` - `port` From fd14ffccd8d1f1da7e2984a4250afaf2d3b51ea2 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 29 Oct 2024 17:02:18 +0800 Subject: [PATCH 11/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index c96a0881eb148..3c6c0b8a08253 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -481,7 +481,7 @@ tiproxy_servers: - `log-file`: The path to which logs are output when the TiKV-CDC process is running (optional). If not specified, logs are written to the standard output (stdout). -- `log-level`: The log level when the TiKV-CDC process is running (optional). The default value is "info". +- `log-level`: The log level when the TiKV-CDC process is running (optional). The default value is `"info"`. - `ca`: The path of the CA certificate file in PEM format for TLS connection (optional). From 8c254839ecdc68e54ab8058ee7ff93b1ee940df3 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 29 Oct 2024 17:17:11 +0800 Subject: [PATCH 12/19] Update tiup-cluster-topology-reference.md --- tiup/tiup-cluster-topology-reference.md | 1 + 1 file changed, 1 insertion(+) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index 3c6c0b8a08253..02b134771f531 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -24,6 +24,7 @@ A topology configuration file for TiDB deployment using TiUP might contain the f - [tikv_servers](#tikv_servers): The configuration of the TiKV instance. This configuration specifies the machines to which the TiKV component is deployed. - [tiflash_servers](#tiflash_servers): The configuration of the TiFlash instance. This configuration specifies the machines to which the TiFlash component is deployed. - [tiproxy_servers](#tiproxy_servers): The configuration of the TiProxy instance. This configuration specifies the machines to which the TiProxy component is deployed. +- [kvcdc_servers](#kvcdc_servers): The configuration of the [TiKV-CDC](https://tikv.org/docs/7.1/concepts/explore-tikv-features/cdc/cdc/) instance. This configuration specifies the machines to which the TiKV-CDC component is deployed. - [cdc_servers](#cdc_servers): The configuration of the TiCDC instance. This configuration specifies the machines to which the TiCDC component is deployed. - [tispark_masters](#tispark_masters): The configuration of the TiSpark master instance. This configuration specifies the machines to which the TiSpark master component is deployed. Only one node of TiSpark master can be deployed. - [tispark_workers](#tispark_workers): The configuration of the TiSpark worker instance. This configuration specifies the machines to which the TiSpark worker component is deployed. From 8ab98b06ed978ecfae85a87af66442f0ca263ef0 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Wed, 30 Oct 2024 10:57:55 +0800 Subject: [PATCH 13/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index 02b134771f531..3f2c491c32263 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -490,7 +490,7 @@ tiproxy_servers: - `key`: The path of the private key file in PEM format for TLS connection (optional). -- `cert-allowed-cn`: Controls whether to verify the caller's identity (certificate Common Name, optional). Use commas to separate multiple CNs. +- `cert-allowed-cn`: The allowed caller identities (certificate Common Name, optional). Use commas to separate multiple CNs. For the above fields, you cannot modify these configured fields after the deployment: From c0f35587d1208f9f8216b0802bdccbba37cd730e Mon Sep 17 00:00:00 2001 From: nhsmw Date: Wed, 30 Oct 2024 14:00:09 +0800 Subject: [PATCH 14/19] Update tiup/tiup-cluster-topology-reference.md Co-authored-by: Grace Cai --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index 3f2c491c32263..cfc23792c7c05 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -468,7 +468,7 @@ tiproxy_servers: `kvcdc_servers` specifies the machines to which the [TiKV-CDC](https://tikv.org/docs/7.1/concepts/explore-tikv-features/cdc/cdc/) services are deployed. It also specifies the service configuration on each machine. `kvcdc_servers` is an array. Each array element contains the following fields: -- `addr`: The listening address of TiKV-CDC, the HTTP API address, and the Prometheus address of the TiKV-CDC service. The default value is `127.0.0.1:8600`. +- `addr`: The listening address of TiKV-CDC, used to provide HTTP API and Prometheus queries. The default value is `127.0.0.1:8600`. - `advertise-addr`: The advertised address via which clients access TiKV-CDC. If not specified, the value is the same as that of `addr`. From 23464bf5c33b96bdea1c2828357de0582ec6829c Mon Sep 17 00:00:00 2001 From: nhsmw Date: Wed, 30 Oct 2024 18:33:46 +0800 Subject: [PATCH 15/19] Apply suggestions from code review Co-authored-by: xixirangrang --- tiup/tiup-cluster-topology-reference.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index cfc23792c7c05..cfcf5b9b2fce0 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -476,9 +476,9 @@ tiproxy_servers: - `config`: The address of the configuration file that TiKV-CDC uses (optional). -- `data-dir`: The directory that TiKV-CDC uses to store temporary files for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). +- `data-dir`: The directory that TiKV-CDC uses to store temporary files primarily for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). -- `gc-ttl`: The TTL (Time to Live, in seconds) of the service-level GC safepoint in PD set by TiKV-CDC (optional). It is the duration that replication tasks can be suspended, defaulting to `86400`, which is 24 hours. Note that suspending replication tasks affects the progress of TiKV garbage collection. The longer the `gc-ttl`, the longer changefeeds can be paused, but at the same time, more obsolete data will be kept and occupy more space. Vice versa. +- `gc-ttl`: The TTL (Time to Live, in seconds) of the service-level GC safepoint in PD set by TiKV-CDC (optional). It is the duration that replication tasks can be suspended, defaulting to `86400`, which is 24 hours. Note that suspending replication tasks affect the progress of TiKV garbage collection safepoint. The longer the `gc-ttl`, the longer changefeeds can be suspended, but at the same time, more obsolete data will be kept and occupy more space. Vice versa. - `log-file`: The path to which logs are output when the TiKV-CDC process is running (optional). If not specified, logs are written to the standard output (stdout). From 3714c4678533c4654c7fdd77c47dfa4d654b8c7e Mon Sep 17 00:00:00 2001 From: nhsmw Date: Thu, 31 Oct 2024 17:43:16 +0800 Subject: [PATCH 16/19] Update tiup-cluster-topology-reference.md --- tiup/tiup-cluster-topology-reference.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index cfcf5b9b2fce0..7043ec565ff1f 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -468,29 +468,29 @@ tiproxy_servers: `kvcdc_servers` specifies the machines to which the [TiKV-CDC](https://tikv.org/docs/7.1/concepts/explore-tikv-features/cdc/cdc/) services are deployed. It also specifies the service configuration on each machine. `kvcdc_servers` is an array. Each array element contains the following fields: -- `addr`: The listening address of TiKV-CDC, used to provide HTTP API and Prometheus queries. The default value is `127.0.0.1:8600`. +- `host`: Specifies the machine to which the TiKV-CDC services are deployed. The field value is an IP address and is mandatory. -- `advertise-addr`: The advertised address via which clients access TiKV-CDC. If not specified, the value is the same as that of `addr`. +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. -- `pd`: A comma-separated list of PD endpoints. +- `port`: The listening port of the TiKV-CDC services. The default value is `8600`. -- `config`: The address of the configuration file that TiKV-CDC uses (optional). +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. - `data-dir`: The directory that TiKV-CDC uses to store temporary files primarily for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). - `gc-ttl`: The TTL (Time to Live, in seconds) of the service-level GC safepoint in PD set by TiKV-CDC (optional). It is the duration that replication tasks can be suspended, defaulting to `86400`, which is 24 hours. Note that suspending replication tasks affect the progress of TiKV garbage collection safepoint. The longer the `gc-ttl`, the longer changefeeds can be suspended, but at the same time, more obsolete data will be kept and occupy more space. Vice versa. -- `log-file`: The path to which logs are output when the TiKV-CDC process is running (optional). If not specified, logs are written to the standard output (stdout). +- `tz`: The time zone that the TiKV-CDC services use. TiKV-CDC uses this time zone when internally converting time data types such as timestamp and when replicating data to the downstream. The default value is the local time zone where the process runs. -- `log-level`: The log level when the TiKV-CDC process is running (optional). The default value is `"info"`. +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". -- `ca`: The path of the CA certificate file in PEM format for TLS connection (optional). +- `config`: The address of the configuration file that TiKV-CDC uses (optional). -- `cert`: The path of the certificate file in PEM format for TLS connection (optional). +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. -- `key`: The path of the private key file in PEM format for TLS connection (optional). +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. -- `cert-allowed-cn`: The allowed caller identities (certificate Common Name, optional). Use commas to separate multiple CNs. +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. For the above fields, you cannot modify these configured fields after the deployment: @@ -499,6 +499,8 @@ For the above fields, you cannot modify these configured fields after the deploy - `deploy_dir` - `data_dir` - `log_dir` +- `arch` +- `os` A `kvcdc_servers` configuration example is as follows: From 57eeffb2aa834a40b1f6e30090a88eded36db662 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Tue, 5 Nov 2024 12:07:22 +0800 Subject: [PATCH 17/19] Update tiup-cluster-topology-reference.md --- tiup/tiup-cluster-topology-reference.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index 7043ec565ff1f..be955d958407e 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -478,6 +478,8 @@ tiproxy_servers: - `data-dir`: The directory that TiKV-CDC uses to store temporary files primarily for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + - `gc-ttl`: The TTL (Time to Live, in seconds) of the service-level GC safepoint in PD set by TiKV-CDC (optional). It is the duration that replication tasks can be suspended, defaulting to `86400`, which is 24 hours. Note that suspending replication tasks affect the progress of TiKV garbage collection safepoint. The longer the `gc-ttl`, the longer changefeeds can be suspended, but at the same time, more obsolete data will be kept and occupy more space. Vice versa. - `tz`: The time zone that the TiKV-CDC services use. TiKV-CDC uses this time zone when internally converting time data types such as timestamp and when replicating data to the downstream. The default value is the local time zone where the process runs. From 4ae9c4e1ef19b39607a67c770ee1d77d7e351352 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 6 Nov 2024 10:18:24 +0800 Subject: [PATCH 18/19] update wording --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index be955d958407e..c678f45530fa7 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -476,7 +476,7 @@ tiproxy_servers: - `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. -- `data-dir`: The directory that TiKV-CDC uses to store temporary files primarily for sorting. It is recommended to ensure that the free disk space for this directory is greater than or equal to 500 GiB (optional). +- `data-dir`: Specifies the directory that TiKV-CDC uses to store temporary files primarily for sorting (optional). The free disk space for this directory is recommended to be greater than or equal to 500 GiB. - `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. From d769ccad003167dcc314d8e472fa592c27aa4bbd Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 6 Nov 2024 11:45:25 +0800 Subject: [PATCH 19/19] fix grammar --- tiup/tiup-cluster-topology-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md index c678f45530fa7..924a38bb63aed 100644 --- a/tiup/tiup-cluster-topology-reference.md +++ b/tiup/tiup-cluster-topology-reference.md @@ -480,7 +480,7 @@ tiproxy_servers: - `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. -- `gc-ttl`: The TTL (Time to Live, in seconds) of the service-level GC safepoint in PD set by TiKV-CDC (optional). It is the duration that replication tasks can be suspended, defaulting to `86400`, which is 24 hours. Note that suspending replication tasks affect the progress of TiKV garbage collection safepoint. The longer the `gc-ttl`, the longer changefeeds can be suspended, but at the same time, more obsolete data will be kept and occupy more space. Vice versa. +- `gc-ttl`: The TTL (Time to Live, in seconds) of the service-level GC safepoint in PD set by TiKV-CDC (optional). It is the duration that replication tasks can be suspended, defaulting to `86400`, which is 24 hours. Note that suspending replication tasks affects the progress of TiKV garbage collection safepoint. The longer the `gc-ttl`, the longer changefeeds can be suspended, but at the same time, more obsolete data will be kept and occupy more space. Vice versa. - `tz`: The time zone that the TiKV-CDC services use. TiKV-CDC uses this time zone when internally converting time data types such as timestamp and when replicating data to the downstream. The default value is the local time zone where the process runs.