-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: DanRoscigno <[email protected]> (cherry picked from commit 0be4658) # Conflicts: # docs/docusaurus/sidebars.json # docs/en/_assets/commonMarkdown/loadMethodIntro.md # docs/en/administration/management/resource_management/be_label.md # docs/en/data_source/catalog/iceberg_catalog.md # docs/en/introduction/Architecture.md # docs/en/loading/BrokerLoad.md # docs/en/loading/Etl_in_loading.md # docs/en/loading/Flink_cdc_load.md # docs/en/loading/InsertInto.md # docs/en/loading/Load_to_Primary_Key_tables.md # docs/en/loading/RoutineLoad.md # docs/en/loading/StreamLoad.md # docs/en/sql-reference/information_schema/loads.md # docs/en/sql-reference/sql-functions/table-functions/files.md # docs/en/sql-reference/sql-statements/Resource/CREATE_RESOURCE.md # docs/en/sql-reference/sql-statements/loading_unloading/INSERT.md # docs/en/sql-reference/sql-statements/loading_unloading/SHOW_LOAD.md # docs/en/sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md # docs/en/sql-reference/sql-statements/loading_unloading/routine_load/CREATE_ROUTINE_LOAD.md # docs/en/sql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md # docs/en/sql-reference/sql-statements/table_bucket_part_index/CREATE_TABLE.md # docs/en/sql-reference/sql-statements/table_bucket_part_index/SHOW_CREATE_TABLE.md # docs/en/table_design/StarRocks_table_design.md # docs/en/table_design/data_distribution/Data_distribution.md # docs/en/table_design/hybrid_table.md # docs/en/table_design/indexes/Ngram_Bloom_Filter_Index.md # docs/en/table_design/indexes/Prefix_index_sort_key.md # docs/en/table_design/indexes/inverted_index.md # docs/en/table_design/table_design.md # docs/en/table_design/table_types/aggregate_table.md # docs/en/table_design/table_types/duplicate_key_table.md # docs/en/table_design/table_types/primary_key_table.md # docs/en/table_design/table_types/unique_key_table.md # docs/en/using_starrocks/_category_.yml # docs/en/using_starrocks/indexes.md # docs/zh/loading/BrokerLoad.md # docs/zh/loading/Etl_in_loading.md # docs/zh/loading/Flink_cdc_load.md # docs/zh/loading/InsertInto.md # docs/zh/loading/Load_to_Primary_Key_tables.md # docs/zh/loading/RoutineLoad.md # docs/zh/loading/StreamLoad.md # docs/zh/sql-reference/sql-statements/loading_unloading/INSERT.md # docs/zh/sql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md # docs/zh/sql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.md # docs/zh/sql-reference/sql-statements/table_bucket_part_index/CREATE_TABLE.md # docs/zh/sql-reference/sql-statements/table_bucket_part_index/SHOW_CREATE_TABLE.md # docs/zh/table_design/StarRocks_table_design.md # docs/zh/table_design/Table_design.md # docs/zh/table_design/data_distribution/Data_distribution.md # docs/zh/table_design/data_distribution/Temporary_partition.md # docs/zh/table_design/data_distribution/expression_partitioning.md # docs/zh/table_design/data_distribution/list_partitioning.md # docs/zh/table_design/hybrid_table.md # docs/zh/table_design/indexes/Ngram_Bloom_Filter_Index.md # docs/zh/table_design/indexes/Prefix_index_sort_key.md # docs/zh/table_design/indexes/inverted_index.md # docs/zh/table_design/table_design.md # docs/zh/table_design/table_types/aggregate_table.md # docs/zh/table_design/table_types/duplicate_key_table.md # docs/zh/table_design/table_types/primary_key_table.md # docs/zh/table_design/table_types/unique_key_table.md # docs/zh/using_starrocks/_category_.yml # docs/zh/using_starrocks/indexes.md
- Loading branch information
1 parent
4453e15
commit 610aff6
Showing
120 changed files
with
4,220 additions
and
2,497 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
For the best practices of multi-service access control, see [Multi-service access control](../../administration/user_privs/User_privilege.md#multi-service-access-control). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
|
||
:::tip | ||
This example uses the Local Climatological Data(LCD) dataset featured in the [StarRocks Basics](../../quick_start/shared-nothing.md) Quick Start. You can load the data and try the example yourself. | ||
::: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
## Learn by doing | ||
|
||
Try the [Quick Starts](../../quick_start/quick_start.mdx) to get an overview of using StarRocks with realistic scenarios. |
5 changes: 5 additions & 0 deletions
5
docs/en/_assets/commonMarkdown/quickstart-routine-load-tip.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
|
||
:::tip | ||
Try Routine Load out in this [Quick Start](../../quick_start/routine-load.md) | ||
::: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
|
||
:::tip | ||
Give [shared-data](../../quick_start/shared-data.md) a try using MinIO for object storage. | ||
::: | ||
|
5 changes: 5 additions & 0 deletions
5
docs/en/_assets/commonMarkdown/quickstart-shared-nothing-tip.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
|
||
:::tip | ||
This example uses the Local Climatological Data(LCD) dataset featured in the [StarRocks Basics](../../quick_start/shared-nothing.md) Quick Start. You can load the data and try the example yourself. | ||
::: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1 change: 1 addition & 0 deletions
1
docs/en/administration/management/resource_management/Blacklist.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 80 | ||
--- | ||
|
||
# Blacklist Management | ||
|
1 change: 1 addition & 0 deletions
1
docs/en/administration/management/resource_management/Load_balance.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 60 | ||
--- | ||
|
||
# Load Balancing | ||
|
1 change: 1 addition & 0 deletions
1
docs/en/administration/management/resource_management/Memory_management.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 40 | ||
--- | ||
|
||
# Memory Management | ||
|
1 change: 1 addition & 0 deletions
1
docs/en/administration/management/resource_management/Query_management.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 30 | ||
--- | ||
|
||
# Query Management | ||
|
1 change: 1 addition & 0 deletions
1
docs/en/administration/management/resource_management/Replica.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 70 | ||
--- | ||
|
||
# Replica management | ||
|
128 changes: 128 additions & 0 deletions
128
docs/en/administration/management/resource_management/be_label.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 80 | ||
--- | ||
|
||
# Add labels on BEs | ||
|
||
Since v3.2.8, StarRocks supports adding labels on BEs. When creating a table or an asynchronous materialized view, you can specify the label of a certain group of BE nodes. This ensures that data replicas are distributed only on the BE nodes associated with that label. Data replicas will be evenly distributed among nodes with the same label, enhancing data high availability and resource isolation. | ||
|
||
## Usage | ||
|
||
### Add labels on BEs | ||
|
||
Suppose that one StarRocks cluster includes six BEs which are distributed evenly across three racks. You can add labels on BEs based on the racks where the BEs are located. | ||
|
||
```SQL | ||
ALTER SYSTEM MODIFY BACKEND "172.xx.xx.46:9050" SET ("labels.location" = "rack:rack1"); | ||
ALTER SYSTEM MODIFY BACKEND "172.xx.xx.47:9050" SET ("labels.location" = "rack:rack1"); | ||
ALTER SYSTEM MODIFY BACKEND "172.xx.xx.48:9050" SET ("labels.location" = "rack:rack2"); | ||
ALTER SYSTEM MODIFY BACKEND "172.xx.xx.49:9050" SET ("labels.location" = "rack:rack2"); | ||
ALTER SYSTEM MODIFY BACKEND "172.xx.xx.50:9050" SET ("labels.location" = "rack:rack3"); | ||
ALTER SYSTEM MODIFY BACKEND "172.xx.xx.51:9050" SET ("labels.location" = "rack:rack3"); | ||
``` | ||
|
||
After adding labels, you can execute `SHOW BACKENDS;` and view the labels of BEs in the `Location` field of the returned result. | ||
|
||
If you need to modify the labels of BEs, you can execute `ALTER SYSTEM MODIFY BACKEND "172.xx.xx.48:9050" SET ("labels.location" = "rack:xxx");`. | ||
|
||
### Use labels to specify table data distribution on BE nodes | ||
|
||
If you need to specify the locations to which a table's data is distributed, for example, distributing a table's data across two racks, rack1 and rack2, you can add labels to the table. | ||
|
||
After labels are added, all the replicas of the same tablet in the table are distributed across labels in a Round-Robin approach. Moreover, if multiple replicas of the same tablet exist within the same label, these replicas will be distributed as evenly as possible across different BEs in that label. | ||
|
||
:::note | ||
|
||
- The total number of BE nodes associated with the labels must be greater than the number of replicas. Otherwise, an error `Table replication num should be less than or equal to the number of available BE nodes` will occur. | ||
- The label to be associated with a table must already exist. Otherwise, an error `Getting analyzing error. Detail message: Cannot find any backend with location: rack:xxx` will occur. | ||
|
||
::: | ||
|
||
#### At table creation | ||
|
||
You can use the property `"labels.location"` to distribute the table's data across rack 1 and rack 2 at table creation: | ||
|
||
```SQL | ||
CREATE TABLE example_table ( | ||
order_id bigint NOT NULL, | ||
dt date NOT NULL, | ||
user_id INT NOT NULL, | ||
good_id INT NOT NULL, | ||
cnt int NOT NULL, | ||
revenue int NOT NULL | ||
) | ||
PROPERTIES | ||
("labels.location" = "rack:rack1,rack:rack2"); | ||
``` | ||
|
||
For newly created tables, the default value of the table property `labels.location` is `*`, indicating that replicas are evenly distributed across all labels. If the data distribution of a newly created table does not need to be aware of the geographical locations of servers in the cluster, you can manually set the table property `"labels.location" = ""`. | ||
|
||
#### After table creation | ||
|
||
If you need to modify the data distribution location of the table after table creation, for example, modify the location to rack 1, rack 2, and rack 3, you can execute the following statement: | ||
|
||
```SQL | ||
ALTER TABLE example_table | ||
SET ("labels.location" = "rack:rack1,rack:rack2,rack:rack3"); | ||
``` | ||
|
||
:::note | ||
|
||
If you have upgraded StarRocks to version 3.2.8 or later, for historical tables created before the upgrade, data is not distributed based on labels by default. If you need to distribute data of a historical table based on labels, you can execute the following statement to add labels to the historical table: | ||
|
||
```SQL | ||
ALTER TABLE example_table1 | ||
SET ("labels.location" = "rack:rack1,rack:rack2"); | ||
``` | ||
|
||
::: | ||
|
||
### Use labels to specify materialized view data distribution on BE nodes | ||
|
||
If you need to specify the locations to which an asynchronous materialized view's data is distributed, for example, distributing data across two racks, rack1 and rack2, you can add labels to the materialized view. | ||
|
||
After labels are added, all the replicas of the same tablet in the materialized view are distributed across labels in a Round-Robin approach. Moreover, if multiple replicas of the same tablet exist within the same label, these replicas will be distributed as evenly as possible across different BEs in that label. | ||
|
||
:::note | ||
|
||
- The total number of BE nodes associated with the labels must be greater than the number of replicas. Otherwise, an error `Table replication num should be less than or equal to the number of available BE nodes` will occur. | ||
- The labels to be associated with the materialized view must already exist. Otherwise, an error `Getting analyzing error. Detail message: Cannot find any backend with location: rack:xxx` will occur. | ||
|
||
::: | ||
|
||
#### At materialized view creation | ||
|
||
If you want to distribute the materialized view's data across rack 1 and rack 2 while creating it, you can execute the following statement: | ||
|
||
```SQL | ||
CREATE MATERIALIZED VIEW mv_example_mv | ||
DISTRIBUTED BY RANDOM | ||
PROPERTIES ( | ||
"labels.location" = "rack:rack1,rack:rack2") | ||
as | ||
select order_id, dt from example_table; | ||
``` | ||
|
||
For newly created materialized view, the default value of the property `labels.location` is `*`, indicating that replicas are evenly distributed across all labels. If the data distribution of a newly created materialized view does not need to be aware of the geographical locations of servers in the cluster, you can manually set the property `"labels.location" = ""`. | ||
|
||
#### After materialized view creation | ||
|
||
If you need to modify the data distribution location of the materialized view after it is created, for example, modify the location to rack 1, rack 2, and rack 3, you can execute the following statement: | ||
|
||
```SQL | ||
ALTER MATERIALIZED VIEW mv_example_mv | ||
SET ("labels.location" = "rack:rack1,rack:rack2,rack:rack3"); | ||
``` | ||
|
||
:::note | ||
|
||
If you have upgraded StarRocks to version 3.2.8 or later, for existing materialized views created before the upgrade, data is not distributed based on labels by default. If you need to distribute data of an existing based on labels, you can execute the following statement to add labels to the materialized view: | ||
|
||
```SQL | ||
ALTER TABLE example_mv1 | ||
SET ("labels.location" = "rack:rack1,rack:rack2"); | ||
``` | ||
|
||
::: | ||
|
1 change: 1 addition & 0 deletions
1
docs/en/administration/management/resource_management/filemanager.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 90 | ||
--- | ||
|
||
# File manager | ||
|
1 change: 1 addition & 0 deletions
1
docs/en/administration/management/resource_management/query_queues.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 20 | ||
--- | ||
|
||
# Query queues | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9 changes: 0 additions & 9 deletions
9
docs/en/administration/management/resource_management/resource_management.mdx
This file was deleted.
Oops, something went wrong.
1 change: 1 addition & 0 deletions
1
docs/en/administration/management/resource_management/spill_to_disk.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
displayed_sidebar: docs | ||
sidebar_position: 50 | ||
--- | ||
|
||
# Spill to disk | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.