Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[reconfigurator] initialize clickhouse cluster db schema #7306

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

[reconfigurator] initialize clickhouse cluster db schema #7306

wants to merge 6 commits into from
+135 −34

Conversation

karencfv
Copy link
Contributor

@karencfv karencfv commented Jan 6, 2025 •
edited
Loading

This commit allows reconfigurator to initialise the oximeter DB schema on a replicated cluster via clickhouse admin.

Tested by deploying Omicron locally and setting/enabling the clickhouse policy to "both"

$ zoneadm list
global
sidecar_softnpu
oxz_switch
oxz_internal_dns_e79062f1-0562-4866-84cc-30741f4b44f3
oxz_internal_dns_125f2a03-9206-4f6c-97fe-56d62626d1be
oxz_internal_dns_a7007297-7150-40cd-83e4-0fec9a02e051
oxz_ntp_576a6dc7-7f33-4ffc-9f77-e998eeac41a5
oxz_cockroachdb_d3caaaa3-4115-4789-8f05-af09f494c8c8
oxz_cockroachdb_f5840cbe-0218-45c4-a259-8eb455814fc7
oxz_cockroachdb_48d7a4e6-0a38-4a86-888e-4349412a5933
oxz_cockroachdb_58b2c1f3-b6f5-4391-88cd-6e976bdbd00b
oxz_cockroachdb_a627c4bb-9681-46dd-bdc7-219bcffe5d33
oxz_crucible_53530a3a-165a-4722-bd22-029ee961f5ff
oxz_crucible_2308e266-39f5-4d40-b9f2-d47b214cd8c3
oxz_crucible_382a33a1-258e-4793-bf31-01c5a07720e2
oxz_crucible_10d3cb3e-bbb7-45ac-80b0-a7e47415123c
oxz_crucible_058adb53-ce33-4fe6-92cb-6d6759af7643
oxz_crucible_6d44a11d-be4d-4c48-8744-dc978fa1263a
oxz_crucible_pantry_69e56321-1d5f-42ed-8285-3ff50bdf7f4c
oxz_external_dns_28ddebab-352c-48be-a26f-403218c11639
oxz_crucible_2fc57911-8177-4957-9920-4717e847a300
oxz_nexus_c31317d6-e072-40f7-a7ba-7c3bab3b30ee
oxz_crucible_02bca09d-5886-4a1d-91b6-9ee86b6a2c4b
oxz_crucible_pantry_1b24234a-aed6-446a-94ba-38573f097d2d
oxz_crucible_a3ccf208-d678-4e1d-a24c-a8bacf69266a
oxz_crucible_pantry_7dbc74c7-a8a0-4292-bc45-9ba0d0b70a7d
oxz_oximeter_85d30f98-950a-4c2f-979a-10a4de75b16d
oxz_external_dns_23ed7c7c-4244-4fe3-90bd-74e212f33626
oxz_nexus_02670c10-c872-4ab8-950a-f1c8dda67328
oxz_nexus_bdf0c647-82b8-45c0-a9ed-df408f21316d
oxz_clickhouse_2a3e6b87-80a9-484f-b4ce-5359cc6be7ff
oxz_clickhouse_keeper_84a8be56-3354-4afa-852d-c85dab34f8f9
oxz_clickhouse_keeper_a44cee3f-9917-4ba6-a45f-f5c03e04c782
oxz_clickhouse_keeper_5f3e655e-4927-43b9-b45d-bf8f5d957aa7
oxz_clickhouse_keeper_bdf37cf4-c043-4845-9fd4-3e64cacd3f13
oxz_clickhouse_server_3847cf44-55a0-41e1-bfbd-0c954637423a
oxz_clickhouse_server_607aa891-0808-4a78-ba9c-7d53b7d99af1
oxz_clickhouse_keeper_b7d86a07-695a-4d6c-95d1-228b668020de
oxz_clickhouse_server_23012e68-1fab-4a2d-beee-acba56e0eb7f
root@oxz_clickhouse_server_3847cf44:~# /opt/oxide/clickhouse_server/clickhouse client --host fd00:1122:3344:101::27
ClickHouse client version 23.8.7.1.
Connecting to fd00:1122:3344:101::27:9000 as user default.
Connected to ClickHouse server version 23.8.7 revision 54465.

oximeter_cluster_2 :) SHOW DATABASES

SHOW DATABASES

Query id: 10b0eb8f-9fa9-411b-be66-70d68940551c

┌─name───────────────┐
│ INFORMATION_SCHEMA │
│ default            │
│ information_schema │
│ oximeter           │
│ system             │
└────────────────────┘

5 rows in set. Elapsed: 0.003 sec. 

oximeter_cluster_2 :) SHOW TABLES in oximeter

SHOW TABLES FROM oximeter

Query id: cbc8c338-6a04-4577-9255-140287d3ba1e

┌─name─────────────────────────────┐
│ fields_bool                      │
│ fields_bool_local                │
│ fields_i16                       │
│ fields_i16_local                 │
│ fields_i32                       │
│ fields_i32_local                 │
│ fields_i64                       │
│ fields_i64_local                 │
│ fields_i8                        │
│ fields_i8_local                  │
│ fields_ipaddr                    │
│ fields_ipaddr_local              │
│ fields_string                    │
│ fields_string_local              │
│ fields_u16                       │
│ fields_u16_local                 │
│ fields_u32                       │
│ fields_u32_local                 │
│ fields_u64                       │
│ fields_u64_local                 │
│ fields_u8                        │
│ fields_u8_local                  │
│ fields_uuid                      │
│ fields_uuid_local                │
│ measurements_bool                │
│ measurements_bool_local          │
│ measurements_bytes               │
│ measurements_bytes_local         │
│ measurements_cumulativef32       │
│ measurements_cumulativef32_local │
│ measurements_cumulativef64       │
│ measurements_cumulativef64_local │
│ measurements_cumulativei64       │
│ measurements_cumulativei64_local │
│ measurements_cumulativeu64       │
│ measurements_cumulativeu64_local │
│ measurements_f32                 │
│ measurements_f32_local           │
│ measurements_f64                 │
│ measurements_f64_local           │
│ measurements_histogramf32        │
│ measurements_histogramf32_local  │
│ measurements_histogramf64        │
│ measurements_histogramf64_local  │
│ measurements_histogrami16        │
│ measurements_histogrami16_local  │
│ measurements_histogrami32        │
│ measurements_histogrami32_local  │
│ measurements_histogrami64        │
│ measurements_histogrami64_local  │
│ measurements_histogrami8         │
│ measurements_histogrami8_local   │
│ measurements_histogramu16        │
│ measurements_histogramu16_local  │
│ measurements_histogramu32        │
│ measurements_histogramu32_local  │
│ measurements_histogramu64        │
│ measurements_histogramu64_local  │
│ measurements_histogramu8         │
│ measurements_histogramu8_local   │
│ measurements_i16                 │
│ measurements_i16_local           │
│ measurements_i32                 │
│ measurements_i32_local           │
│ measurements_i64                 │
│ measurements_i64_local           │
│ measurements_i8                  │
│ measurements_i8_local            │
│ measurements_string              │
│ measurements_string_local        │
│ measurements_u16                 │
│ measurements_u16_local           │
│ measurements_u32                 │
│ measurements_u32_local           │
│ measurements_u64                 │
│ measurements_u64_local           │
│ measurements_u8                  │
│ measurements_u8_local            │
│ timeseries_schema                │
│ timeseries_schema_local          │
│ version                          │
└──────────────────────────────────┘

81 rows in set. Elapsed: 0.005 sec. 

oximeter_cluster_2 :) exit
Bye.
root@oxz_clickhouse_server_607aa891:~# /opt/oxide/clickhouse_server/clickhouse client --host fd00:1122:3344:101::26
ClickHouse client version 23.8.7.1.
Connecting to fd00:1122:3344:101::26:9000 as user default.
Connected to ClickHouse server version 23.8.7 revision 54465.

oximeter_cluster_3 :) SHOW DATABASES

SHOW DATABASES

Query id: bf3f7d69-e829-415c-8737-cef2f68e1f46

┌─name───────────────┐
│ INFORMATION_SCHEMA │
│ default            │
│ information_schema │
│ oximeter           │
│ system             │
└────────────────────┘

5 rows in set. Elapsed: 0.004 sec. 

oximeter_cluster_3 :) SHOW TABLES in oximeter

SHOW TABLES FROM oximeter

Query id: 46358626-5f25-437b-a1d7-c89b0aa34807

┌─name─────────────────────────────┐
│ fields_bool                      │
│ fields_bool_local                │
│ fields_i16                       │
│ fields_i16_local                 │
│ fields_i32                       │
│ fields_i32_local                 │
│ fields_i64                       │
│ fields_i64_local                 │
│ fields_i8                        │
│ fields_i8_local                  │
│ fields_ipaddr                    │
│ fields_ipaddr_local              │
│ fields_string                    │
│ fields_string_local              │
│ fields_u16                       │
│ fields_u16_local                 │
│ fields_u32                       │
│ fields_u32_local                 │
│ fields_u64                       │
│ fields_u64_local                 │
│ fields_u8                        │
│ fields_u8_local                  │
│ fields_uuid                      │
│ fields_uuid_local                │
│ measurements_bool                │
│ measurements_bool_local          │
│ measurements_bytes               │
│ measurements_bytes_local         │
│ measurements_cumulativef32       │
│ measurements_cumulativef32_local │
│ measurements_cumulativef64       │
│ measurements_cumulativef64_local │
│ measurements_cumulativei64       │
│ measurements_cumulativei64_local │
│ measurements_cumulativeu64       │
│ measurements_cumulativeu64_local │
│ measurements_f32                 │
│ measurements_f32_local           │
│ measurements_f64                 │
│ measurements_f64_local           │
│ measurements_histogramf32        │
│ measurements_histogramf32_local  │
│ measurements_histogramf64        │
│ measurements_histogramf64_local  │
│ measurements_histogrami16        │
│ measurements_histogrami16_local  │
│ measurements_histogrami32        │
│ measurements_histogrami32_local  │
│ measurements_histogrami64        │
│ measurements_histogrami64_local  │
│ measurements_histogrami8         │
│ measurements_histogrami8_local   │
│ measurements_histogramu16        │
│ measurements_histogramu16_local  │
│ measurements_histogramu32        │
│ measurements_histogramu32_local  │
│ measurements_histogramu64        │
│ measurements_histogramu64_local  │
│ measurements_histogramu8         │
│ measurements_histogramu8_local   │
│ measurements_i16                 │
│ measurements_i16_local           │
│ measurements_i32                 │
│ measurements_i32_local           │
│ measurements_i64                 │
│ measurements_i64_local           │
│ measurements_i8                  │
│ measurements_i8_local            │
│ measurements_string              │
│ measurements_string_local        │
│ measurements_u16                 │
│ measurements_u16_local           │
│ measurements_u32                 │
│ measurements_u32_local           │
│ measurements_u64                 │
│ measurements_u64_local           │
│ measurements_u8                  │
│ measurements_u8_local            │
│ timeseries_schema                │
│ timeseries_schema_local          │
│ version                          │
└──────────────────────────────────┘

81 rows in set. Elapsed: 0.006 sec. 

oximeter_cluster_3 :) exit
Bye.
root@oxz_clickhouse_server_23012e68:~# /opt/oxide/clickhouse_server/clickhouse client --host fd00:1122:3344:101::28
ClickHouse client version 23.8.7.1.
Connecting to fd00:1122:3344:101::28:9000 as user default.
Connected to ClickHouse server version 23.8.7 revision 54465.

oximeter_cluster_1 :) SHOW DATABASES

SHOW DATABASES

Query id: f5408b05-4bb9-4b8c-b27c-820cc50eaa56

┌─name───────────────┐
│ INFORMATION_SCHEMA │
│ default            │
│ information_schema │
│ oximeter           │
│ system             │
└────────────────────┘

5 rows in set. Elapsed: 0.005 sec. 

oximeter_cluster_1 :) SHOW TABLES IN oximeter

SHOW TABLES FROM oximeter

Query id: fb3533c3-2309-4ad8-8b8a-5cc9d4e2c4e7

┌─name─────────────────────────────┐
│ fields_bool                      │
│ fields_bool_local                │
│ fields_i16                       │
│ fields_i16_local                 │
│ fields_i32                       │
│ fields_i32_local                 │
│ fields_i64                       │
│ fields_i64_local                 │
│ fields_i8                        │
│ fields_i8_local                  │
│ fields_ipaddr                    │
│ fields_ipaddr_local              │
│ fields_string                    │
│ fields_string_local              │
│ fields_u16                       │
│ fields_u16_local                 │
│ fields_u32                       │
│ fields_u32_local                 │
│ fields_u64                       │
│ fields_u64_local                 │
│ fields_u8                        │
│ fields_u8_local                  │
│ fields_uuid                      │
│ fields_uuid_local                │
│ measurements_bool                │
│ measurements_bool_local          │
│ measurements_bytes               │
│ measurements_bytes_local         │
│ measurements_cumulativef32       │
│ measurements_cumulativef32_local │
│ measurements_cumulativef64       │
│ measurements_cumulativef64_local │
│ measurements_cumulativei64       │
│ measurements_cumulativei64_local │
│ measurements_cumulativeu64       │
│ measurements_cumulativeu64_local │
│ measurements_f32                 │
│ measurements_f32_local           │
│ measurements_f64                 │
│ measurements_f64_local           │
│ measurements_histogramf32        │
│ measurements_histogramf32_local  │
│ measurements_histogramf64        │
│ measurements_histogramf64_local  │
│ measurements_histogrami16        │
│ measurements_histogrami16_local  │
│ measurements_histogrami32        │
│ measurements_histogrami32_local  │
│ measurements_histogrami64        │
│ measurements_histogrami64_local  │
│ measurements_histogrami8         │
│ measurements_histogrami8_local   │
│ measurements_histogramu16        │
│ measurements_histogramu16_local  │
│ measurements_histogramu32        │
│ measurements_histogramu32_local  │
│ measurements_histogramu64        │
│ measurements_histogramu64_local  │
│ measurements_histogramu8         │
│ measurements_histogramu8_local   │
│ measurements_i16                 │
│ measurements_i16_local           │
│ measurements_i32                 │
│ measurements_i32_local           │
│ measurements_i64                 │
│ measurements_i64_local           │
│ measurements_i8                  │
│ measurements_i8_local            │
│ measurements_string              │
│ measurements_string_local        │
│ measurements_u16                 │
│ measurements_u16_local           │
│ measurements_u32                 │
│ measurements_u32_local           │
│ measurements_u64                 │
│ measurements_u64_local           │
│ measurements_u8                  │
│ measurements_u8_local            │
│ timeseries_schema                │
│ timeseries_schema_local          │
│ version                          │
└──────────────────────────────────┘

81 rows in set. Elapsed: 0.006 sec. 

oximeter_cluster_1 :) exit
Bye.

Closes: #7060

@karencfv karencfv marked this pull request as ready for review January 8, 2025 04:13
@karencfv karencfv requested a review from andrewjstone January 8, 2025 04:13
plotnick
plotnick reviewed Jan 9, 2025
View reviewed changes
nexus/reconfigurator/execution/src/clickhouse.rs Outdated
));
let admin_url = format!("http://{admin_addr}");
let log = opctx.log.new(slog::o!("admin_url" => admin_url.clone()));
let client = ClickhouseSingleClient::new(&admin_url, log.clone());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be ClickhouseServerClient instead of ClickhouseSingleClient? Right now they happen to have compatible APIs for this one method (.init_db), but we probably don't want to depend on that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops! good catch. Thanks!

andrewjstone
andrewjstone reviewed Jan 9, 2025
View reviewed changes
Copy link
Contributor

@andrewjstone andrewjstone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great @karencfv! Thank you so much for doing this.

Can you please test adding a node to see what happens with the schema? I don't believe it will work without initializing it specifically.

nexus/reconfigurator/execution/src/clickhouse.rs Outdated
@@ -160,6 +160,38 @@ pub(crate) async fn deploy_nodes(
"Successfully deployed all clickhouse server and keeper configs"
);

// We only need to initialise the database schema into one of the ClickHouse replica
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is only true for the initial cluster. If you add a new node, it will not get initialized automatically. Since the operation is idempotent I would go ahead and initialize all the servers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you add a new node, it will not get initialized automatically.

ugh, that's a bummer. yep! will change then

clickhouse-admin/src/http_entrypoints.rs
}

enum ClickhouseAdminKeeperImpl {}

impl ClickhouseAdminKeeperApi for ClickhouseAdminKeeperImpl {
type Context = Arc<ServerContext>;
type Context = Arc<KeeperServerContext>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call on the rename here ;)

clickhouse-admin/src/http_entrypoints.rs Outdated
"initializing replicated ClickHouse cluster to version {OXIMETER_VERSION}"
);
ctx.oximeter_client()
.initialize_db_with_version(true, OXIMETER_VERSION)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: APIs with bool parameters make it hard to figure out what is being passed in at the callsite. I usually create a temp variable to name the flag then pass that in. I'd suggest that here with something like:

let replicated = true;
ctx.oximeter_client().initialize_db_with_version(replicated, OXIMETER_VERSION);

karencfv
karencfv commented Jan 13, 2025
View reviewed changes
Copy link
Contributor Author

@karencfv karencfv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review both! I've updated the PR with your suggestions.

Additionally, I tested adding a node to an already running cluster:

root@oxz_switch:~# omdb nexus clickhouse-policy get
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::a]:12221
Clickhouse Policy: 
    version: 1
    creation time: 2025-01-13 06:56:25.883275 UTC
    mode: both single-node and cluster
    target-servers: 3
    target-keepers: 5
root@oxz_switch:~# omdb nexus clickhouse-policy set both --target-servers 4 -w
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::a]:12221
Successfully inserted new policy at version 2
root@oxz_switch:~# omdb nexus clickhouse-policy get
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::a]:12221
Clickhouse Policy: 
    version: 2
    creation time: 2025-01-13 07:08:12.959900 UTC
    mode: both single-node and cluster
    target-servers: 4
    target-keepers: 5
root@oxz_switch:~# omdb nexus blueprints regenerate -w
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::a]:12221
generated new blueprint c61ecd89-2bb7-445b-a339-138ec59bb525
root@oxz_switch:~# omdb nexus blueprints diff target c61ecd89-2bb7-445b-a339-138ec59bb525
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::a]:12221
from: blueprint 9ab9b68d-6032-41ce-a763-7035b1d3e4e7
to:   blueprint c61ecd89-2bb7-445b-a339-138ec59bb525

 MODIFIED SLEDS:

  sled 8d86b923-8ccd-4003-8c83-531ab1b5533d (active):

    physical disks at generation 2:
    ----------------------------------------------------------
    vendor             model                serial            
    ----------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-4
    synthetic-vendor   synthetic-model-U2   synthetic-serial-5
    synthetic-vendor   synthetic-model-U2   synthetic-serial-6
    synthetic-vendor   synthetic-model-U2   synthetic-serial-7
    synthetic-vendor   synthetic-model-U2   synthetic-serial-8


    datasets generation 2 -> 3:
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                     dataset uuid                           quota     reservation   compression
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/cockroachdb                                                       d1a3f670-11f0-4890-b690-fc4fb002913c   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/cockroachdb                                                       213f9856-33f3-4e10-98b6-9da74a0198cd   none      none          off        
    oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/cockroachdb                                                       c32de361-ee45-422c-a060-ae2cb9d8e9bd   none      none          off        
    oxp_be5cff87-fc4b-4935-b7be-8e1e3f62638c/crypt/cockroachdb                                                       ca3c87da-4d1c-4383-8c84-b60f3172cfa3   none      none          off        
    oxp_cf753d90-7ebd-4562-9cd4-7bb6b0fb7217/crypt/cockroachdb                                                       a20cceca-a8e7-4f8a-ab90-c0fefdc99490   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crucible                                                                25fbdb48-c9b6-4c0f-8ae4-6d3db8c6649e   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crucible                                                                bfc3cf17-7012-4a43-b100-b6b33a2a4403   none      none          off        
    oxp_7dade47b-2ef5-446e-8c5f-cdc59790c2a4/crucible                                                                b44aac16-0cea-4a00-a7b3-36295c42a9c5   none      none          off        
    oxp_a3654b74-626c-42fa-acb7-7799c367e545/crucible                                                                70ede0f9-30dc-4907-be35-102867c00fcd   none      none          off        
    oxp_a8cb5a24-553c-430b-ad06-cc3236c55170/crucible                                                                ba88fa0e-f5a6-40ca-af6d-d98931421096   none      none          off        
    oxp_b0d1c952-1c4a-47f8-ad7d-db0e3e5bb9f4/crucible                                                                e99f301f-ced5-44f0-8f7e-281fb272d6a7   none      none          off        
    oxp_be5cff87-fc4b-4935-b7be-8e1e3f62638c/crucible                                                                b3f6ca77-28b1-4736-8125-29b464b6ee68   none      none          off        
    oxp_cf753d90-7ebd-4562-9cd4-7bb6b0fb7217/crucible                                                                ea1b73c1-629a-4d46-aa11-c797f21fec4a   none      none          off        
    oxp_d8e41e8a-48fe-4499-b974-1af69a372047/crucible                                                                29237a38-c765-41c8-9a5d-140518de089c   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/clickhouse                                                        058a9349-cfa1-4dd6-8396-28fab7aa1899   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/clickhouse_keeper                                                 3d47ea44-2ea4-425e-b26d-857f598aa01b   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/clickhouse_keeper                                                 12781612-0b73-49d4-ada1-787a93f3eab7   none      none          off        
    oxp_7dade47b-2ef5-446e-8c5f-cdc59790c2a4/crypt/clickhouse_keeper                                                 63eb425c-9aba-4c89-942f-e6aa5878b8a3   none      none          off        
    oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/clickhouse_keeper                                                 dbe58737-945a-4fc5-894e-76cad7a08fb4   none      none          off        
    oxp_a8cb5a24-553c-430b-ad06-cc3236c55170/crypt/clickhouse_keeper                                                 0d24b5dd-dbbf-4096-94e7-532a43802dad   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/clickhouse_server                                                 bb6f65f7-f336-46f2-96a5-638eea2652c4   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/clickhouse_server                                                 8880ac02-b680-4b36-882f-8452ad0926f0   none      none          off        
    oxp_7dade47b-2ef5-446e-8c5f-cdc59790c2a4/crypt/clickhouse_server                                                 b6ae3ffa-81f0-47dd-99e7-7cf5875e4593   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/external_dns                                                      aa6022e7-7a49-430a-8daf-9c2e36d418d0   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/external_dns                                                      ed505dc0-c420-45ae-9311-cda770a5bc92   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/internal_dns                                                      030ce58b-1e43-4bb2-b88d-d35e12ed8e29   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/internal_dns                                                      c376feec-238b-4198-bb91-e1c82304fa36   none      none          off        
    oxp_be5cff87-fc4b-4935-b7be-8e1e3f62638c/crypt/internal_dns                                                      bd0c5076-bcbf-4a2e-9662-aff228fefe44   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/zone                                                              0116855a-a42a-42d9-8868-bb25e209e716   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone                                                              b478f29c-e2ff-40f8-bcfc-6596ac6df7ab   none      none          off        
    oxp_7dade47b-2ef5-446e-8c5f-cdc59790c2a4/crypt/zone                                                              8ded3124-5e74-48cb-a474-cb0e7051ef64   none      none          off        
    oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/zone                                                              4d289176-3f42-4e3c-a6ac-738d5dc7f9b0   none      none          off        
    oxp_a8cb5a24-553c-430b-ad06-cc3236c55170/crypt/zone                                                              d5ac31d2-91d9-47f3-8b8c-a92fc14dc213   none      none          off        
    oxp_b0d1c952-1c4a-47f8-ad7d-db0e3e5bb9f4/crypt/zone                                                              4ba65d16-a540-411b-8a7c-3a4070501fe9   none      none          off        
    oxp_be5cff87-fc4b-4935-b7be-8e1e3f62638c/crypt/zone                                                              b5de2b33-edea-4907-a845-f376709cbaab   none      none          off        
    oxp_cf753d90-7ebd-4562-9cd4-7bb6b0fb7217/crypt/zone                                                              fd821588-27fc-4260-a1f8-bab9c48368f6   none      none          off        
    oxp_d8e41e8a-48fe-4499-b974-1af69a372047/crypt/zone                                                              e2bff34c-efe7-413a-9754-9dd5a0c53f4d   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_clickhouse_9ae49761-6c16-49c7-9c31-b171c8037bfa          32d64891-4304-45ed-b420-414a23d9e79e   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/zone/oxz_clickhouse_keeper_3fc49953-6bc8-46e2-a92b-1b3807525505   bbfbfb65-3986-4f81-989e-f3232d1aaec2   none      none          off        
    oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/zone/oxz_clickhouse_keeper_401030db-456f-4776-8ff2-b00cbb30d389   7d4ac858-4df4-43d7-ae0b-ba356e77248c   none      none          off        
    oxp_a8cb5a24-553c-430b-ad06-cc3236c55170/crypt/zone/oxz_clickhouse_keeper_5e1c994d-c002-4c2e-8aa0-0214ece2b91d   896552aa-3609-4862-83bb-ba26e9f9749b   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_clickhouse_keeper_89cb500e-5d7d-455e-8dfe-1a8cc098632b   e6e5b7ea-6ea1-4acb-ac00-31dd1b9b31dc   none      none          off        
    oxp_7dade47b-2ef5-446e-8c5f-cdc59790c2a4/crypt/zone/oxz_clickhouse_keeper_8ab22a99-17cf-4b14-9ad8-1916085ae0c7   2bd8bdf5-86f3-4b2f-bd18-17c4b79ebc13   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_clickhouse_server_467c46fe-5293-43d9-acc0-64177cd5ae8d   74b3cec2-9328-4022-b0fc-e5f63b930e0d   none      none          off        
    oxp_7dade47b-2ef5-446e-8c5f-cdc59790c2a4/crypt/zone/oxz_clickhouse_server_762a98d6-4d37-4c70-8fd4-8e9933f77104   93026d1f-d52a-4c32-8094-04cba2f0d047   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/zone/oxz_clickhouse_server_ea0a1e2c-c24b-4683-980c-8dd419e6f62a   9da75e64-b44f-4321-9d89-fb06e8734c00   none      none          off        
    oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/zone/oxz_cockroachdb_07d33598-8a36-4bc6-a576-9b7f1678c8c9         82117a99-4655-4816-8c83-b3b06914ccc4   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_cockroachdb_1bf24121-468f-4212-b2d4-6f03cfad65e2         4d94fac1-a681-4e65-b6a4-fc79c1ba2d92   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/zone/oxz_cockroachdb_265014ac-6a92-4520-bd3c-30ab13925c04         e68e4494-9d14-4873-a3b0-75e9b343bf23   none      none          off        
    oxp_cf753d90-7ebd-4562-9cd4-7bb6b0fb7217/crypt/zone/oxz_cockroachdb_a555bbc9-95a7-471f-a31d-1a312b55a854         6fe6e162-fbd0-4011-944b-7e59af70c440   none      none          off        
    oxp_be5cff87-fc4b-4935-b7be-8e1e3f62638c/crypt/zone/oxz_cockroachdb_af1da743-fa76-40a6-9cc0-0249d9c5ab1a         77efd4ec-769e-4b5c-8d90-c6900897fc43   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_crucible_0aec808f-3082-4130-abaf-85a8f2a0bcba            87a81497-70c2-42b0-b17b-79490289b702   none      none          off        
    oxp_a8cb5a24-553c-430b-ad06-cc3236c55170/crypt/zone/oxz_crucible_4f64ecab-7734-48cd-ac33-5e7435fddba5            ea2d0c3f-fbb6-4c51-859c-85aab7fe0c8e   none      none          off        
    oxp_cf753d90-7ebd-4562-9cd4-7bb6b0fb7217/crypt/zone/oxz_crucible_56bfb9d8-8fb4-4534-b98f-8bb33a26553c            2f04ca5d-a129-4997-957c-de00771ce219   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/zone/oxz_crucible_68473cf6-4ada-40e1-896a-a4ba514387a6            c5eac2a9-5706-420b-8628-6b122dc22776   none      none          off        
    oxp_be5cff87-fc4b-4935-b7be-8e1e3f62638c/crypt/zone/oxz_crucible_6c1c23ea-a224-4fa1-86ee-fd971f1d3ac2            2bd81360-1d89-4645-97c2-1aa128157e94   none      none          off        
    oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/zone/oxz_crucible_6ecf7b43-3310-407c-836c-8342012d61fa            caf90788-f264-4d1f-b207-c64f242fc717   none      none          off        
    oxp_7dade47b-2ef5-446e-8c5f-cdc59790c2a4/crypt/zone/oxz_crucible_8aba67ce-c290-4ac6-8293-b1b1bafbdfb8            f3254881-3f78-43c2-8397-2b442b665db8   none      none          off        
    oxp_d8e41e8a-48fe-4499-b974-1af69a372047/crypt/zone/oxz_crucible_b7634935-a8de-48fd-bb49-8f20f54929cc            e5459335-e4ea-421a-af6d-b8de2897b4a9   none      none          off        
    oxp_b0d1c952-1c4a-47f8-ad7d-db0e3e5bb9f4/crypt/zone/oxz_crucible_ecec2f83-7a69-4593-b5bf-18ba8b24ff98            fd0a996c-3bdc-45cd-85ea-0af04d050192   none      none          off        
    oxp_b0d1c952-1c4a-47f8-ad7d-db0e3e5bb9f4/crypt/zone/oxz_crucible_pantry_10e2a6e9-8598-44d3-b956-f2b28a605f23     286d9087-3021-41da-9c3e-452de8333b68   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_crucible_pantry_3683344c-47a3-4519-8f59-36ed6854e6c2     72c760ec-b73c-42d9-b470-1c9019beb585   none      none          off        
    oxp_b0d1c952-1c4a-47f8-ad7d-db0e3e5bb9f4/crypt/zone/oxz_crucible_pantry_eb697684-55c0-463f-a540-6e3a258eb8e6     7fb7199a-091c-4ea8-aa64-46302be7ee4e   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_external_dns_3ab274f6-31fc-4fb6-9cc7-9c25d12f113b        9aea31ad-a547-4b51-97de-c75936fc1e23   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/zone/oxz_external_dns_fb9c962f-89d9-4195-8e47-97ffd8eef3e1        98d129d5-da9b-41eb-bd7c-dcc1e3740e01   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_internal_dns_6e0cf81d-e3a1-4582-a159-98094b81347e        268cc4c8-6ac4-41a1-85bc-a7cf7cda5bec   none      none          off        
    oxp_be5cff87-fc4b-4935-b7be-8e1e3f62638c/crypt/zone/oxz_internal_dns_dfaf9c9b-042d-45b9-940a-404077ed216d        9cc97f70-6846-4e6e-89e3-d2cab2cf53fa   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/zone/oxz_internal_dns_f630320c-8d92-4bcb-ae6b-d0f7cf307d44        ff0db1f4-0bec-4033-b2f5-5de2d2c01800   none      none          off        
    oxp_b0d1c952-1c4a-47f8-ad7d-db0e3e5bb9f4/crypt/zone/oxz_nexus_66806605-6817-4895-82d8-1fba57a7e6a2               82fc313a-d35f-4dab-aae3-cbb6612d3af5   none      none          off        
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/zone/oxz_nexus_6fe5d6f6-1554-479f-9f11-f63c2e606a88               81a3ffa8-7171-4c0a-9630-6bd861069d88   none      none          off        
    oxp_cf753d90-7ebd-4562-9cd4-7bb6b0fb7217/crypt/zone/oxz_nexus_d81680d1-9c44-4a77-87fd-941d99d08931               066f1765-44e1-4382-a4af-8390d230747f   none      none          off        
    oxp_b0d1c952-1c4a-47f8-ad7d-db0e3e5bb9f4/crypt/zone/oxz_ntp_4db5df5e-22ac-4860-bd11-0e4f7fc1202b                 31795a3e-aeab-4401-98b2-7cfd3779f32b   none      none          off        
    oxp_cf753d90-7ebd-4562-9cd4-7bb6b0fb7217/crypt/zone/oxz_oximeter_ed375bbe-d549-41b4-a59e-9c68a20e1f0c            ac8dae2d-abcc-4534-a277-12a266bf4425   none      none          off        
    oxp_08f35583-0026-4c81-ab9d-c2b1a6771f02/crypt/debug                                                             12bed11f-5a77-41a0-b502-8a6e92550a8b   100 GiB   none          gzip-9     
    oxp_097a0b0c-8509-4c76-8b6a-eed70d597f2c/crypt/debug                                                             3c3b8c06-3d24-47aa-8a48-590538460946   100 GiB   none          gzip-9     
    oxp_7dade47b-2ef5-446e-8c5f-cdc59790c2a4/crypt/debug                                                             cedb9334-229a-4b2e-be1b-7abf2d318868   100 GiB   none          gzip-9     
    oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/debug                                                             b737c9c4-5a7b-4509-bc62-e0dbe0768b36   100 GiB   none          gzip-9     
    oxp_a8cb5a24-553c-430b-ad06-cc3236c55170/crypt/debug                                                             b5781fa9-5351-40bb-8142-21b69f0d5f4d   100 GiB   none          gzip-9     
    oxp_b0d1c952-1c4a-47f8-ad7d-db0e3e5bb9f4/crypt/debug                                                             69c41951-eb11-4e5d-8d38-cb5462e7d9e5   100 GiB   none          gzip-9     
    oxp_be5cff87-fc4b-4935-b7be-8e1e3f62638c/crypt/debug                                                             f374e133-307f-4152-9e75-b0de8e7a9e1c   100 GiB   none          gzip-9     
    oxp_cf753d90-7ebd-4562-9cd4-7bb6b0fb7217/crypt/debug                                                             f1ebb99f-25a4-45a7-9513-14d3583cffd9   100 GiB   none          gzip-9     
    oxp_d8e41e8a-48fe-4499-b974-1af69a372047/crypt/debug                                                             59da6493-bd96-4ad1-955a-fa51f0e2fdf5   100 GiB   none          gzip-9     
+   oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/clickhouse_server                                                 b36fd635-bc68-4915-8e6a-24540372c819   none      none          off        
+   oxp_a3654b74-626c-42fa-acb7-7799c367e545/crypt/zone/oxz_clickhouse_server_fb40d6fe-6c7f-4005-bb97-684bb74b6054   b6ea1041-0b4a-44ad-ae39-446af04303ae   none      none          off        


    omicron zones generation 6 -> 7:
    -----------------------------------------------------------------------------------------------
    zone type           zone id                                disposition   underlay IP           
    -----------------------------------------------------------------------------------------------
    boundary_ntp        4db5df5e-22ac-4860-bd11-0e4f7fc1202b   in service    fd00:1122:3344:101::1b
    clickhouse          9ae49761-6c16-49c7-9c31-b171c8037bfa   in service    fd00:1122:3344:101::e 
    clickhouse_keeper   3fc49953-6bc8-46e2-a92b-1b3807525505   in service    fd00:1122:3344:101::21
    clickhouse_keeper   401030db-456f-4776-8ff2-b00cbb30d389   in service    fd00:1122:3344:101::24
    clickhouse_keeper   5e1c994d-c002-4c2e-8aa0-0214ece2b91d   in service    fd00:1122:3344:101::25
    clickhouse_keeper   89cb500e-5d7d-455e-8dfe-1a8cc098632b   in service    fd00:1122:3344:101::22
    clickhouse_keeper   8ab22a99-17cf-4b14-9ad8-1916085ae0c7   in service    fd00:1122:3344:101::23
    clickhouse_server   467c46fe-5293-43d9-acc0-64177cd5ae8d   in service    fd00:1122:3344:101::27
    clickhouse_server   762a98d6-4d37-4c70-8fd4-8e9933f77104   in service    fd00:1122:3344:101::28
    clickhouse_server   ea0a1e2c-c24b-4683-980c-8dd419e6f62a   in service    fd00:1122:3344:101::26
    cockroach_db        07d33598-8a36-4bc6-a576-9b7f1678c8c9   in service    fd00:1122:3344:101::6 
    cockroach_db        1bf24121-468f-4212-b2d4-6f03cfad65e2   in service    fd00:1122:3344:101::3 
    cockroach_db        265014ac-6a92-4520-bd3c-30ab13925c04   in service    fd00:1122:3344:101::4 
    cockroach_db        a555bbc9-95a7-471f-a31d-1a312b55a854   in service    fd00:1122:3344:101::7 
    cockroach_db        af1da743-fa76-40a6-9cc0-0249d9c5ab1a   in service    fd00:1122:3344:101::5 
    crucible            0aec808f-3082-4130-abaf-85a8f2a0bcba   in service    fd00:1122:3344:101::12
    crucible            4f64ecab-7734-48cd-ac33-5e7435fddba5   in service    fd00:1122:3344:101::18
    crucible            56bfb9d8-8fb4-4534-b98f-8bb33a26553c   in service    fd00:1122:3344:101::16
    crucible            68473cf6-4ada-40e1-896a-a4ba514387a6   in service    fd00:1122:3344:101::13
    crucible            6c1c23ea-a224-4fa1-86ee-fd971f1d3ac2   in service    fd00:1122:3344:101::14
    crucible            6ecf7b43-3310-407c-836c-8342012d61fa   in service    fd00:1122:3344:101::15
    crucible            8aba67ce-c290-4ac6-8293-b1b1bafbdfb8   in service    fd00:1122:3344:101::19
    crucible            b7634935-a8de-48fd-bb49-8f20f54929cc   in service    fd00:1122:3344:101::17
    crucible            ecec2f83-7a69-4593-b5bf-18ba8b24ff98   in service    fd00:1122:3344:101::1a
    crucible_pantry     10e2a6e9-8598-44d3-b956-f2b28a605f23   in service    fd00:1122:3344:101::f 
    crucible_pantry     3683344c-47a3-4519-8f59-36ed6854e6c2   in service    fd00:1122:3344:101::10
    crucible_pantry     eb697684-55c0-463f-a540-6e3a258eb8e6   in service    fd00:1122:3344:101::11
    external_dns        3ab274f6-31fc-4fb6-9cc7-9c25d12f113b   in service    fd00:1122:3344:101::8 
    external_dns        fb9c962f-89d9-4195-8e47-97ffd8eef3e1   in service    fd00:1122:3344:101::9 
    internal_dns        6e0cf81d-e3a1-4582-a159-98094b81347e   in service    fd00:1122:3344:1::1   
    internal_dns        dfaf9c9b-042d-45b9-940a-404077ed216d   in service    fd00:1122:3344:3::1   
    internal_dns        f630320c-8d92-4bcb-ae6b-d0f7cf307d44   in service    fd00:1122:3344:2::1   
    nexus               66806605-6817-4895-82d8-1fba57a7e6a2   in service    fd00:1122:3344:101::a 
    nexus               6fe5d6f6-1554-479f-9f11-f63c2e606a88   in service    fd00:1122:3344:101::b 
    nexus               d81680d1-9c44-4a77-87fd-941d99d08931   in service    fd00:1122:3344:101::c 
    oximeter            ed375bbe-d549-41b4-a59e-9c68a20e1f0c   in service    fd00:1122:3344:101::d 
+   clickhouse_server   fb40d6fe-6c7f-4005-bb97-684bb74b6054   in service    fd00:1122:3344:101::29


 COCKROACHDB SETTINGS:
    state fingerprint:::::::::::::::::   d4d87aa2ad877a4cc2fddd0573952362739110de (unchanged)
    cluster.preserve_downgrade_option:   "22.1" (unchanged)

 METADATA:
*   internal DNS version:   1 -> 2
    external DNS version:   2 (unchanged)

 CLICKHOUSE CLUSTER CONFIG:
*   generation:::::::::::::::::::::::::::::::::::::   2 -> 3
*   max used server id:::::::::::::::::::::::::::::   3 -> 4
    max used keeper id:::::::::::::::::::::::::::::   5 (unchanged)
    cluster name:::::::::::::::::::::::::::::::::::   oximeter_cluster (unchanged)
    cluster secret:::::::::::::::::::::::::::::::::   b2cd6a29-e360-4afd-8e10-6814ebb8587c (unchanged)
*   highest seen keeper leader committed log index:   0 -> 18117

    clickhouse keepers generation 2 -> 3:
    ------------------------------------------------
    zone id                                keeper id
    ------------------------------------------------
    3fc49953-6bc8-46e2-a92b-1b3807525505   1        
    401030db-456f-4776-8ff2-b00cbb30d389   2        
    5e1c994d-c002-4c2e-8aa0-0214ece2b91d   3        
    89cb500e-5d7d-455e-8dfe-1a8cc098632b   4        
    8ab22a99-17cf-4b14-9ad8-1916085ae0c7   5        

    clickhouse servers generation 2 -> 3:
    ------------------------------------------------
    zone id                                server id
    ------------------------------------------------
    467c46fe-5293-43d9-acc0-64177cd5ae8d   1        
    762a98d6-4d37-4c70-8fd4-8e9933f77104   2        
    ea0a1e2c-c24b-4683-980c-8dd419e6f62a   3        
+   fb40d6fe-6c7f-4005-bb97-684bb74b6054   4        


root@oxz_switch:~# omdb nexus blueprints target set c61ecd89-2bb7-445b-a339-138ec59bb525 enabled -w
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::a]:12221
set target blueprint to c61ecd89-2bb7-445b-a339-138ec59bb525
root@oxz_clickhouse_server_fb40d6fe:~# /opt/oxide/clickhouse_server/clickhouse client --host fd00:1122:3344:101::29
ClickHouse client version 23.8.7.1.
Connecting to fd00:1122:3344:101::29:9000 as user default.
Connected to ClickHouse server version 23.8.7 revision 54465.

oximeter_cluster_4 :) SHOW DATABASES

SHOW DATABASES

Query id: 73f4f6ab-d084-44e9-8d41-1520262883e7

┌─name───────────────┐
│ INFORMATION_SCHEMA │
│ default            │
│ information_schema │
│ oximeter           │
│ system             │
└────────────────────┘

5 rows in set. Elapsed: 0.001 sec. 

oximeter_cluster_4 :) SHOW TABLES IN oximeter

SHOW TABLES FROM oximeter

Query id: f9da32b2-be21-4bad-90d0-11c3d6d401a7

┌─name─────────────────────────────┐
│ fields_bool                      │
│ fields_bool_local                │
│ fields_i16                       │
│ fields_i16_local                 │
│ fields_i32                       │
│ fields_i32_local                 │
│ fields_i64                       │
│ fields_i64_local                 │
│ fields_i8                        │
│ fields_i8_local                  │
│ fields_ipaddr                    │
│ fields_ipaddr_local              │
│ fields_string                    │
│ fields_string_local              │
│ fields_u16                       │
│ fields_u16_local                 │
│ fields_u32                       │
│ fields_u32_local                 │
│ fields_u64                       │
│ fields_u64_local                 │
│ fields_u8                        │
│ fields_u8_local                  │
│ fields_uuid                      │
│ fields_uuid_local                │
│ measurements_bool                │
│ measurements_bool_local          │
│ measurements_bytes               │
│ measurements_bytes_local         │
│ measurements_cumulativef32       │
│ measurements_cumulativef32_local │
│ measurements_cumulativef64       │
│ measurements_cumulativef64_local │
│ measurements_cumulativei64       │
│ measurements_cumulativei64_local │
│ measurements_cumulativeu64       │
│ measurements_cumulativeu64_local │
│ measurements_f32                 │
│ measurements_f32_local           │
│ measurements_f64                 │
│ measurements_f64_local           │
│ measurements_histogramf32        │
│ measurements_histogramf32_local  │
│ measurements_histogramf64        │
│ measurements_histogramf64_local  │
│ measurements_histogrami16        │
│ measurements_histogrami16_local  │
│ measurements_histogrami32        │
│ measurements_histogrami32_local  │
│ measurements_histogrami64        │
│ measurements_histogrami64_local  │
│ measurements_histogrami8         │
│ measurements_histogrami8_local   │
│ measurements_histogramu16        │
│ measurements_histogramu16_local  │
│ measurements_histogramu32        │
│ measurements_histogramu32_local  │
│ measurements_histogramu64        │
│ measurements_histogramu64_local  │
│ measurements_histogramu8         │
│ measurements_histogramu8_local   │
│ measurements_i16                 │
│ measurements_i16_local           │
│ measurements_i32                 │
│ measurements_i32_local           │
│ measurements_i64                 │
│ measurements_i64_local           │
│ measurements_i8                  │
│ measurements_i8_local            │
│ measurements_string              │
│ measurements_string_local        │
│ measurements_u16                 │
│ measurements_u16_local           │
│ measurements_u32                 │
│ measurements_u32_local           │
│ measurements_u64                 │
│ measurements_u64_local           │
│ measurements_u8                  │
│ measurements_u8_local            │
│ timeseries_schema                │
│ timeseries_schema_local          │
│ version                          │
└──────────────────────────────────┘

81 rows in set. Elapsed: 0.002 sec. 

oximeter_cluster_4 :) exit
Bye.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@plotnick plotnick plotnick left review comments

@andrewjstone andrewjstone andrewjstone left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Initialize clickhouse cluster schema via clickhouse-admin
3 participants
@karencfv @andrewjstone @plotnick