From 21b3e93f450ecbd85c660ce293a14b76612e2f85 Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Thu, 29 Feb 2024 17:47:44 +0800 Subject: [PATCH 1/7] update hardare requirement for disaggregated tiflash Signed-off-by: guo-shaoge --- hardware-and-software-requirements.md | 31 +++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/hardware-and-software-requirements.md b/hardware-and-software-requirements.md index 8896807aa8663..5f97e4372fcae 100644 --- a/hardware-and-software-requirements.md +++ b/hardware-and-software-requirements.md @@ -267,3 +267,34 @@ As an open-source distributed SQL database, TiDB requires the following network ## Web browser requirements TiDB relies on [Grafana](https://grafana.com/) to provide visualization of database metrics. A recent version of Microsoft Edge, Safari, Chrome or Firefox with Javascript enabled is sufficient. + +## Hardware and Software Requirements for TiFlash Disaggregated Storage and Compute Architecture +All the descriptions above are for the coupled TiFlash architecture. Starting from v7.0.0, TiFlash introduces support for the [disaggregated of storage and compute](tiflash/tiflash-disaggregated-and-s3.md) architecture. In this architecture, TiFlash is divided into storage nodes and compute nodes. The corresponding hardware and software requirements are as follows: +1. **Software:** + - The operating system and platform requirements are same with the coupled storage and compute architecture. + +2. **Network Ports:** + - Network port requirements are same with the coupled storage and compute mode. + +3. **Disk Space Requirements:** + - Storage Nodes: Same with the coupled storage and compute mode, but an S3-compatible object storage is required. + - Compute Nodes: Recommended 100GB. The storage on compute nodes is mainly used to cache data read from storage nodes to accelerate performance. Therefore, it is normal to see the cache on compute nodes being fully utilized. + +4. **CPU and Other Memory Requirements - Refer to the Next Section** + +### Development and Testing Environment +| **Component** | **CPU** | **Memory** | **Local Storage** | **Network** | **Instance Quantity (Minimum Requirement)** | +| --- | --- | --- | --- | --- | --- | +| TiFlash Storage Node | 16 cores+ | 32 GB+ | SSD, 200 GB+ | Gigabit Ethernet | 1 | +| TiFlash Compute Node | 16 cores+ | 32 GB+ | SSD, 100 GB+ | Gigabit Ethernet | 0 | + +### Production Environment +| **Component** | **CPU** | **Memory** | **Disk Type** | **Network** | **Instance Quantity (Minimum Requirement)** | +| --- | --- | --- | --- | --- | --- | +| TiFlash Storage Node | 32 cores+ | 64 GB+ | 1 or more SSDs | 10 Gigabit Ethernet (2 recommended) | 1 | +| TiFlash Compute Node | 32 cores+ | 64 GB+ | 1 or more SSDs | 10 Gigabit Ethernet (2 recommended) | 0 | + +> **Note:** +> +> - TiFlash Compute Nodes can be quickly scaled in or out using deployment tools such as TiUP, with a scaling range of [0, inf+]. + From 362fd235d0af639faad8a64f7396455ab48c9c29 Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Thu, 29 Feb 2024 17:50:32 +0800 Subject: [PATCH 2/7] fix link Signed-off-by: guo-shaoge --- hardware-and-software-requirements.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hardware-and-software-requirements.md b/hardware-and-software-requirements.md index 5f97e4372fcae..461866ca15c5a 100644 --- a/hardware-and-software-requirements.md +++ b/hardware-and-software-requirements.md @@ -269,7 +269,7 @@ As an open-source distributed SQL database, TiDB requires the following network TiDB relies on [Grafana](https://grafana.com/) to provide visualization of database metrics. A recent version of Microsoft Edge, Safari, Chrome or Firefox with Javascript enabled is sufficient. ## Hardware and Software Requirements for TiFlash Disaggregated Storage and Compute Architecture -All the descriptions above are for the coupled TiFlash architecture. Starting from v7.0.0, TiFlash introduces support for the [disaggregated of storage and compute](tiflash/tiflash-disaggregated-and-s3.md) architecture. In this architecture, TiFlash is divided into storage nodes and compute nodes. The corresponding hardware and software requirements are as follows: +All the descriptions above are for the coupled TiFlash architecture. Starting from v7.0.0, TiFlash introduces support for the [disaggregated of storage and compute](/tiflash/tiflash-disaggregated-and-s3.md) architecture. In this architecture, TiFlash is divided into storage nodes and compute nodes. The corresponding hardware and software requirements are as follows: 1. **Software:** - The operating system and platform requirements are same with the coupled storage and compute architecture. From 60f6f0acfb766b5dc8537f1b0ada37df0957b38e Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Thu, 29 Feb 2024 18:12:17 +0800 Subject: [PATCH 3/7] fix blank Signed-off-by: guo-shaoge --- hardware-and-software-requirements.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/hardware-and-software-requirements.md b/hardware-and-software-requirements.md index 461866ca15c5a..bb8ee212c4c29 100644 --- a/hardware-and-software-requirements.md +++ b/hardware-and-software-requirements.md @@ -269,6 +269,7 @@ As an open-source distributed SQL database, TiDB requires the following network TiDB relies on [Grafana](https://grafana.com/) to provide visualization of database metrics. A recent version of Microsoft Edge, Safari, Chrome or Firefox with Javascript enabled is sufficient. ## Hardware and Software Requirements for TiFlash Disaggregated Storage and Compute Architecture + All the descriptions above are for the coupled TiFlash architecture. Starting from v7.0.0, TiFlash introduces support for the [disaggregated of storage and compute](/tiflash/tiflash-disaggregated-and-s3.md) architecture. In this architecture, TiFlash is divided into storage nodes and compute nodes. The corresponding hardware and software requirements are as follows: 1. **Software:** - The operating system and platform requirements are same with the coupled storage and compute architecture. @@ -283,12 +284,14 @@ All the descriptions above are for the coupled TiFlash architecture. Starting fr 4. **CPU and Other Memory Requirements - Refer to the Next Section** ### Development and Testing Environment + | **Component** | **CPU** | **Memory** | **Local Storage** | **Network** | **Instance Quantity (Minimum Requirement)** | | --- | --- | --- | --- | --- | --- | | TiFlash Storage Node | 16 cores+ | 32 GB+ | SSD, 200 GB+ | Gigabit Ethernet | 1 | | TiFlash Compute Node | 16 cores+ | 32 GB+ | SSD, 100 GB+ | Gigabit Ethernet | 0 | ### Production Environment + | **Component** | **CPU** | **Memory** | **Disk Type** | **Network** | **Instance Quantity (Minimum Requirement)** | | --- | --- | --- | --- | --- | --- | | TiFlash Storage Node | 32 cores+ | 64 GB+ | 1 or more SSDs | 10 Gigabit Ethernet (2 recommended) | 1 | From fe59f9d54db8d040addc6c9b850786bee93a9375 Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Thu, 29 Feb 2024 19:19:40 +0800 Subject: [PATCH 4/7] fix blank Signed-off-by: guo-shaoge --- hardware-and-software-requirements.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/hardware-and-software-requirements.md b/hardware-and-software-requirements.md index bb8ee212c4c29..d080e5db9e9bf 100644 --- a/hardware-and-software-requirements.md +++ b/hardware-and-software-requirements.md @@ -271,17 +271,13 @@ TiDB relies on [Grafana](https://grafana.com/) to provide visualization of datab ## Hardware and Software Requirements for TiFlash Disaggregated Storage and Compute Architecture All the descriptions above are for the coupled TiFlash architecture. Starting from v7.0.0, TiFlash introduces support for the [disaggregated of storage and compute](/tiflash/tiflash-disaggregated-and-s3.md) architecture. In this architecture, TiFlash is divided into storage nodes and compute nodes. The corresponding hardware and software requirements are as follows: -1. **Software:** - - The operating system and platform requirements are same with the coupled storage and compute architecture. -2. **Network Ports:** - - Network port requirements are same with the coupled storage and compute mode. - -3. **Disk Space Requirements:** - - Storage Nodes: Same with the coupled storage and compute mode, but an S3-compatible object storage is required. - - Compute Nodes: Recommended 100GB. The storage on compute nodes is mainly used to cache data read from storage nodes to accelerate performance. Therefore, it is normal to see the cache on compute nodes being fully utilized. - -4. **CPU and Other Memory Requirements - Refer to the Next Section** +1. Software: The operating system and platform requirements are same with the coupled storage and compute architecture. +2. Network Ports: Network port requirements are same with the coupled storage and compute mode. +3. Disk Space Requirements: + 1. Storage Nodes: Same with the coupled storage and compute mode, but an S3-compatible object storage is required. + 2. Compute Nodes: Recommended 100GB. The storage on compute nodes is mainly used to cache data read from storage nodes to accelerate performance. Therefore, it is normal to see the cache on compute nodes being fully utilized. +4. CPU and Other Memory Requirements - Refer to the Next Section ### Development and Testing Environment From 70faf92608468440f9eb7cc9b44a67fbbff1cefe Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Fri, 1 Mar 2024 11:42:35 +0800 Subject: [PATCH 5/7] Update hardware-and-software-requirements.md Co-authored-by: Aolin --- hardware-and-software-requirements.md | 1 - 1 file changed, 1 deletion(-) diff --git a/hardware-and-software-requirements.md b/hardware-and-software-requirements.md index d080e5db9e9bf..9044630e32bb1 100644 --- a/hardware-and-software-requirements.md +++ b/hardware-and-software-requirements.md @@ -296,4 +296,3 @@ All the descriptions above are for the coupled TiFlash architecture. Starting fr > **Note:** > > - TiFlash Compute Nodes can be quickly scaled in or out using deployment tools such as TiUP, with a scaling range of [0, inf+]. - From aa1d9e0c79bd963e3f33e3ab8d98d23c86835567 Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 12 Mar 2024 13:37:51 +0800 Subject: [PATCH 6/7] refine wording --- hardware-and-software-requirements.md | 34 +++++++++++++-------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/hardware-and-software-requirements.md b/hardware-and-software-requirements.md index 9044630e32bb1..a6a7eb1f5a4ae 100644 --- a/hardware-and-software-requirements.md +++ b/hardware-and-software-requirements.md @@ -268,31 +268,31 @@ As an open-source distributed SQL database, TiDB requires the following network TiDB relies on [Grafana](https://grafana.com/) to provide visualization of database metrics. A recent version of Microsoft Edge, Safari, Chrome or Firefox with Javascript enabled is sufficient. -## Hardware and Software Requirements for TiFlash Disaggregated Storage and Compute Architecture +## Hardware and software requirements for TiFlash disaggregated storage and compute architecture -All the descriptions above are for the coupled TiFlash architecture. Starting from v7.0.0, TiFlash introduces support for the [disaggregated of storage and compute](/tiflash/tiflash-disaggregated-and-s3.md) architecture. In this architecture, TiFlash is divided into storage nodes and compute nodes. The corresponding hardware and software requirements are as follows: +The preceding TiFlash software and hardware requirements are for the coupled storage and compute architecture. Starting from v7.0.0, TiFlash supports the [disaggregated storage and compute architecture](/tiflash/tiflash-disaggregated-and-s3.md). In this architecture, TiFlash is divided into two types of nodes: the Write Node and the Compute Node. The requirements for these nodes are as follows: -1. Software: The operating system and platform requirements are same with the coupled storage and compute architecture. -2. Network Ports: Network port requirements are same with the coupled storage and compute mode. -3. Disk Space Requirements: - 1. Storage Nodes: Same with the coupled storage and compute mode, but an S3-compatible object storage is required. - 2. Compute Nodes: Recommended 100GB. The storage on compute nodes is mainly used to cache data read from storage nodes to accelerate performance. Therefore, it is normal to see the cache on compute nodes being fully utilized. -4. CPU and Other Memory Requirements - Refer to the Next Section +- Software: remain the same as the coupled storage and compute architecture, see [OS and platform requirements](#os-and-platform-requirements). +- Network port: remain the same as the coupled storage and compute architecture, see [Network](#network-requirements). +- Disk space: + - TiFlash Write Node: it is recommended to configure 200 GB+ of disk space, which is used as a local buffer when adding TiFlash replicas and migrating Region replicas before uploading data to Amazon S3. In addition, an object storage compatible with Amazon S3 is required. + - TiFlash Compute Node: it is recommended to configure 100 GB+ of disk space, which is mainly used to cache the data read from the Write Node to improve performance. The cache of the compute node may be fully used, which is normal. +- CPU and memory requirements are as follows. -### Development and Testing Environment +### Development and test environments -| **Component** | **CPU** | **Memory** | **Local Storage** | **Network** | **Instance Quantity (Minimum Requirement)** | +| Component | CPU | Memory | Local Storage | Network | Number of Instances (Minimum Requirement) | | --- | --- | --- | --- | --- | --- | -| TiFlash Storage Node | 16 cores+ | 32 GB+ | SSD, 200 GB+ | Gigabit Ethernet | 1 | -| TiFlash Compute Node | 16 cores+ | 32 GB+ | SSD, 100 GB+ | Gigabit Ethernet | 0 | +| TiFlash Write Node | 16 cores+ | 32 GB+ | SSD, 200 GB+ | Gigabit Ethernet | 1 | +| TiFlash Compute Node | 16 cores+ | 32 GB+ | SSD, 100 GB+ | Gigabit Ethernet | 0 (see the following note) | -### Production Environment +### Production environment -| **Component** | **CPU** | **Memory** | **Disk Type** | **Network** | **Instance Quantity (Minimum Requirement)** | +| Component | CPU | Memory | Disk Type | Network | Number of Instances (Minimum Requirement) | | --- | --- | --- | --- | --- | --- | -| TiFlash Storage Node | 32 cores+ | 64 GB+ | 1 or more SSDs | 10 Gigabit Ethernet (2 recommended) | 1 | -| TiFlash Compute Node | 32 cores+ | 64 GB+ | 1 or more SSDs | 10 Gigabit Ethernet (2 recommended) | 0 | +| TiFlash Write Node | 32 cores+ | 64 GB+ | 1 or more SSDs | 10 Gigabit Ethernet (2 recommended) | 1 | +| TiFlash Compute Node | 32 cores+ | 64 GB+ | 1 or more SSDs | 10 Gigabit Ethernet (2 recommended) | 0 (see the following note) | > **Note:** > -> - TiFlash Compute Nodes can be quickly scaled in or out using deployment tools such as TiUP, with a scaling range of [0, inf+]. +> You can use deployment tools such as TiUP to quickly scale in or out the TiFlash Compute Node, within the range of `[0, +inf]`. From d2ff71fc5167901300d45467f3575a287f23ba4c Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 12 Mar 2024 14:32:35 +0800 Subject: [PATCH 7/7] Apply suggestions from code review Co-authored-by: xixirangrang --- hardware-and-software-requirements.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hardware-and-software-requirements.md b/hardware-and-software-requirements.md index a6a7eb1f5a4ae..b345b4fbadf3d 100644 --- a/hardware-and-software-requirements.md +++ b/hardware-and-software-requirements.md @@ -275,9 +275,9 @@ The preceding TiFlash software and hardware requirements are for the coupled sto - Software: remain the same as the coupled storage and compute architecture, see [OS and platform requirements](#os-and-platform-requirements). - Network port: remain the same as the coupled storage and compute architecture, see [Network](#network-requirements). - Disk space: - - TiFlash Write Node: it is recommended to configure 200 GB+ of disk space, which is used as a local buffer when adding TiFlash replicas and migrating Region replicas before uploading data to Amazon S3. In addition, an object storage compatible with Amazon S3 is required. - - TiFlash Compute Node: it is recommended to configure 100 GB+ of disk space, which is mainly used to cache the data read from the Write Node to improve performance. The cache of the compute node may be fully used, which is normal. -- CPU and memory requirements are as follows. + - TiFlash Write Node: it is recommended to configure at least 200 GB of disk space, which is used as a local buffer when adding TiFlash replicas and migrating Region replicas before uploading data to Amazon S3. In addition, an object storage compatible with Amazon S3 is required. + - TiFlash Compute Node: it is recommended to configure at least 100 GB of disk space, which is mainly used to cache the data read from the Write Node to improve performance. The cache of the Compute Node might be fully used, which is normal. +- CPU and memory requirements are described in the following sections. ### Development and test environments