Skip to content

Commit

Permalink
Release ticloud cli v1.0.0-beta.2 (#18938)
Browse files Browse the repository at this point in the history
  • Loading branch information
@shiyuhang0
shiyuhang0 authored Sep 25, 2024
1 parent 314300e commit f3b4e3e
Show file tree
Hide file tree
Showing 21 changed files with 569 additions and 87 deletions.
8 changes: 7 additions & 1 deletion TOC-tidb-cloud.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -675,6 +675,7 @@
- auth
- [login](/tidb-cloud/ticloud-auth-login.md)
- [logout](/tidb-cloud/ticloud-auth-logout.md)
- [whoami](/tidb-cloud/ticloud-auth-whoami.md)
- serverless
- [create](/tidb-cloud/ticloud-cluster-create.md)
- [delete](/tidb-cloud/ticloud-cluster-delete.md)
Expand All @@ -701,6 +702,11 @@
- [list](/tidb-cloud/ticloud-serverless-export-list.md)
- [cancel](/tidb-cloud/ticloud-serverless-export-cancel.md)
- [download](/tidb-cloud/ticloud-serverless-export-download.md)
- sql-user
- [create](/tidb-cloud/ticloud-serverless-sql-user-create.md)
- [delete](/tidb-cloud/ticloud-serverless-sql-user-delete.md)
- [list](/tidb-cloud/ticloud-serverless-sql-user-list.md)
- [update](/tidb-cloud/ticloud-serverless-sql-user-update.md)
- [ai](/tidb-cloud/ticloud-ai.md)
- [completion](/tidb-cloud/ticloud-completion.md)
- config
Expand All @@ -713,7 +719,7 @@
- [use](/tidb-cloud/ticloud-config-use.md)
- project
- [list](/tidb-cloud/ticloud-project-list.md)
- [update](/tidb-cloud/ticloud-update.md)
- [upgrade](/tidb-cloud/ticloud-upgrade.md)
- [help](/tidb-cloud/ticloud-help.md)
- [Table Filter](/table-filter.md)
- [Resource Control](/tidb-resource-control.md)
Expand Down
Binary file added media/tidb-cloud/serverless-external-storage/azure-sas-create.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/serverless-external-storage/azure-sas-position.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/serverless-external-storage/gcs-service-account-key.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/serverless-external-storage/gcs-service-account.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 14 additions & 13 deletions tidb-cloud/cli-reference.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,20 @@ The following table lists the commands available for the TiDB Cloud CLI.

To use the `ticloud` CLI in your terminal, run `ticloud [command] [subcommand]`. If you are using [TiUP](https://docs.pingcap.com/tidb/stable/tiup-overview), use `tiup cloud [command] [subcommand]` instead.

| Command | Subcommand | Description |
|-------------------|--------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|
| auth | login, logout | Login and logout |
| serverless (alias: s) | create, delete, describe, list, update, spending-limit, region, shell | Manage TiDB Cloud Serverless clusters |
| serverless branch | create, delete, describe, list, shell | Manage TiDB Cloud Serverless branches |
| serverless import | cancel, describe, list, start | Manage TiDB Cloud Serverless import tasks |
| serverless export | create, describe, list, cancel, download | Manage TiDB Cloud Serverless export tasks |
| ai | - | Chat with TiDB Bot |
| completion | bash, fish, powershell, zsh | Generate completion script for specified shell |
| config | create, delete, describe, edit, list, set, use | Configure user profiles |
| project | list | Manage projects |
| update | - | Update the CLI to the latest version |
| help | cluster, completion, config, help, import, project, update | View help for any command |
| Command | Subcommand | Description |
|-----------------------|-----------------------------------------------------------------------|------------------------------------------------|
| auth | login, logout, whoami | Login and logout |
| serverless (alias: s) | create, delete, describe, list, update, spending-limit, region, shell | Manage TiDB Cloud Serverless clusters |
| serverless branch | create, delete, describe, list, shell | Manage TiDB Cloud Serverless branches |
| serverless import | cancel, describe, list, start | Manage TiDB Cloud Serverless import tasks |
| serverless export | create, describe, list, cancel, download | Manage TiDB Cloud Serverless export tasks |
| serverless sql-user | create, list, delete, update | Manage TiDB Cloud Serverless SQL users |
| ai | - | Chat with TiDB Bot |
| completion | bash, fish, powershell, zsh | Generate completion script for specified shell |
| config | create, delete, describe, edit, list, set, use | Configure user profiles |
| project | list | Manage projects |
| upgrade | - | Update the CLI to the latest version |
| help | auth, config, serverless, ai, project, upgrade, help, completion | View help for any command |

## Command modes

Expand Down
2 changes: 1 addition & 1 deletion tidb-cloud/release-notes-2023.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -863,7 +863,7 @@ This page lists the release notes of [TiDB Cloud](https://www.pingcap.com/tidb-c

- Add a new command `ticloud cluster connect-info` for [TiDB Cloud CLI](/tidb-cloud/cli-reference.md).

`ticloud cluster connect-info` is a command that allows you to get the connection string of a cluster. To use this command, [update `ticloud`](/tidb-cloud/ticloud-update.md) to v0.3.2 or a later version.
`ticloud cluster connect-info` is a command that allows you to get the connection string of a cluster. To use this command, [update `ticloud`](/tidb-cloud/ticloud-upgrade.md) to v0.3.2 or a later version.

## February 21, 2023

Expand Down
126 changes: 117 additions & 9 deletions tidb-cloud/serverless-export.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,14 @@ While you can also export data using tools such as [mysqldump](https://dev.mysql

## Export locations

You can export data to a local file or [Amazon S3](https://aws.amazon.com/s3/).
You can export data to the following locations:

- A local file
- An external storage, including:

- [Amazon S3](https://aws.amazon.com/s3/)
- [Google Cloud Storage](https://cloud.google.com/storage)
- [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/)

> **Note:**
>
Expand All @@ -42,11 +49,38 @@ To export data to Amazon S3, you need to provide the following information:

For more information, see [Configure External Storage Access for TiDB Cloud Serverless](/tidb-cloud/serverless-external-storage.md#configure-amazon-s3-access).

### Google Cloud Storage

To export data to Google Cloud Storage, you need to provide the following information:

- URI: `gs://<bucket-name>/<file-path>`
- Access credential: a **base64 encoded** [service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) for your bucket. Make sure the service account key has the `storage.objects.create` permission.

For more information, see [Configure External Storage Access for TiDB Serverless](/tidb-cloud/serverless-external-storage.md#configure-gcs-access).

> **Note:**
>
> Currently, you can only export to Google Cloud Storage using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md).
### Azure Blob Storage

To export data to Azure Blob Storage, you need to provide the following information:

- URI: `azure://<account-name>.blob.core.windows.net/<container-name>/<file-path>`
- Access credential: a [shared access signature (SAS) token](https://docs.microsoft.com/en-us/azure/storage/common/storage-sas-overview) for your Azure Blob Storage container. Make sure the SAS token has the `Read` and `Write` permissions on the `Container` and `Object` resources.

For more information, see [Configure External Storage Access for TiDB Serverless](/tidb-cloud/serverless-external-storage.md#configure-azure-blob-storage-access).

> **Note:**
>
> Currently, you can only export to Azure Blob Storage using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md).
## Export options

### Data filtering

- TiDB Cloud console supports exporting data with the selected databases and tables.
- TiDB Cloud CLI supports exporting data with SQL statements and [table filters](/table-filter.md).

### Data formats

Expand All @@ -58,14 +92,15 @@ You can export data in the following formats:
- `separator`: specify the character used to separate fields in the exported data. The default separator is `,`.
- `header`: specify whether to include a header row in the exported data. The default value is `true`.
- `null-value`: specify the string that represents a NULL value in the exported data. The default value is `\N`.
- `Parquet`: export data in Parquet format. Currently, it is only supported in TiDB Cloud CLI.

The schema and data are exported according to the following naming conventions:

| Item | Not compressed | Compressed |
|-----------------|------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
| Database schema | {database}-schema-create.sql | {database}-schema-create.sql.{compression-type} |
| Table schema | {database}.{table}-schema.sql | {database}.{table}-schema.sql.{compression-type} |
| Data | {database}.{table}.{0001}.{csv&#124;sql} | {database}.{table}.{0001}.{csv&#124;sql}.{compression-type} |
| Item | Not compressed | Compressed |
|-----------------|-------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
| Database schema | {database}-schema-create.sql | {database}-schema-create.sql.{compression-type} |
| Table schema | {database}.{table}-schema.sql | {database}.{table}-schema.sql.{compression-type} |
| Data | {database}.{table}.{0001}.{csv&#124;parquet&#124;sql} | {database}.{table}.{0001}.{csv&#124;sql}.{compression-type}<br/>{database}.{table}.{0001}.{compression-type}.parquet |

### Data compression

Expand All @@ -76,7 +111,58 @@ You can compress the exported CSV and SQL data using the following algorithms:
- `zstd`: compress the exported data with `zstd`.
- `none`: do not compress the exported `data`.

## Steps
You can compress the exported Parquet data using the following algorithms:

- `zstd` (default): compress the Parquet file with `zstd`.
- `gzip`: compress the Parquet file with `gzip`.
- `snappy`: compress the Parquet file with `snappy`.
- `none`: do not compress the Parquet file.

### Data conversion

When exporting data to the Parquet format, the data conversion between TiDB Cloud Serverless and Parquet is as follows:

| TiDB Cloud Serverless Type | Parquest primitive type | Parquet logical type |
|----------------------------|-------------------------|----------------------------------------------|
| VARCHAR | BYTE_ARRAY | String(UTF8) |
| TIME | BYTE_ARRAY | String(UTF8) |
| TINYTEXT | BYTE_ARRAY | String(UTF8) |
| MEDIUMTEXT | BYTE_ARRAY | String(UTF8) |
| TEXT | BYTE_ARRAY | String(UTF8) |
| LONGTEXT | BYTE_ARRAY | String(UTF8) |
| SET | BYTE_ARRAY | String(UTF8) |
| JSON | BYTE_ARRAY | String(UTF8) |
| DATE | BYTE_ARRAY | String(UTF8) |
| CHAR | BYTE_ARRAY | String(UTF8) |
| VECTOR | BYTE_ARRAY | String(UTF8) |
| DECIMAL(1<=p<=9) | INT32 | DECIMAL(p,s) |
| DECIMAL(10<=p<=18) | INT64 | DECIMAL(p,s) |
| DECIMAL(p>=19) | BYTE_ARRAY | String(UTF8) |
| ENUM | BYTE_ARRAY | String(UTF8) |
| TIMESTAMP | INT64 | TIMESTAMP(unit=MICROS,isAdjustedToUTC=false) |
| DATETIME | INT64 | TIMESTAMP(unit=MICROS,isAdjustedToUTC=false) |
| YEAR | INT32 | / |
| TINYINT | INT32 | / |
| UNSIGNED TINYINT | INT32 | / |
| SMALLINT | INT32 | / |
| UNSIGNED SMALLINT | INT32 | / |
| MEDIUMINT | INT32 | / |
| UNSIGNED MEDIUMINT | INT32 | / |
| INT | INT32 | / |
| UNSIGNED INT | FIXED_LEN_BYTE_ARRAY(9) | DECIMAL(20,0) |
| BIGINT | FIXED_LEN_BYTE_ARRAY(9) | DECIMAL(20,0) |
| UNSIGNED BIGINT | BYTE_ARRAY | String(UTF8) |
| FLOAT | FLOAT | / |
| DOUBLE | DOUBLE | / |
| BLOB | BYTE_ARRAY | / |
| TINYBLOB | BYTE_ARRAY | / |
| MEDIUMBLOB | BYTE_ARRAY | / |
| LONGBLOB | BYTE_ARRAY | / |
| BINARY | BYTE_ARRAY | / |
| VARBINARY | BYTE_ARRAY | / |
| BIT | BYTE_ARRAY | / |

## Examples

### Export data to a local file

Expand Down Expand Up @@ -160,16 +246,38 @@ You can compress the exported CSV and SQL data using the following algorithms:
<div label="CLI">

```shell
ticloud serverless export create -c <cluster-id> --s3.bucket-uri <uri> --s3.access-key-id <access-key-id> --s3.secret-access-key <secret-access-key>
ticloud serverless export create -c <cluster-id> --s3.uri <uri> --s3.access-key-id <access-key-id> --s3.secret-access-key <secret-access-key> --filter "database.table"
```

- `s3.bucket-uri`: the Amazon S3 URI with the `s3://<bucket-name>/<file-path>` format.
- `s3.uri`: the Amazon S3 URI with the `s3://<bucket-name>/<file-path>` format.
- `s3.access-key-id`: the access key ID of the user who has the permission to access the bucket.
- `s3.secret-access-key`: the access key secret of the user who has the permission to access the bucket.

</div>
</SimpleTab>

### Export data to Google Cloud Storage

Currently, you can only export data to Google Cloud Storage using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md).

```shell
ticloud serverless export create -c <cluster-id> --gcs.uri <uri> --gcs.service-account-key <service-account-key> --filter "database.table"
```

- `gcs.uri`: the URI of the Google Cloud Storage bucket in the `gs://<bucket-name>/<file-path>` format.
- `gcs.service-account-key`: the base64 encoded service account key.

### Export data to Azure Blob Storage

Currently, you can only export data to Azure Blob Storage using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md).

```shell
ticloud serverless export create -c <cluster-id> --azblob.uri <uri> --azblob.sas-token <sas-token> --filter "database.table"
```

- `azblob.uri`: the URI of the Azure Blob Storage in the `azure://<account-name>.blob.core.windows.net/<container-name>/<file-path>` format.
- `azblob.sas-token`: the account SAS token of the Azure Blob Storage.

### Cancel an export task

To cancel an ongoing export task, take the following steps:
Expand Down
Loading

0 comments on commit f3b4e3e

Please sign in to comment.