-
Notifications
You must be signed in to change notification settings - Fork 688
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
export #16776
Closed
Closed
export #16776
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
d04f909
export
shiyuhang0 12d7681
fix lint
shiyuhang0 2264f3e
polish
shiyuhang0 1fe8248
Update tidb-cloud/serverless-export.md
shiyuhang0 283c15d
fix
shiyuhang0 fbd1be4
toc
shiyuhang0 5bb07d5
Update tidb-cloud/serverless-export.md
shiyuhang0 5a58626
Apply suggestions from code review
shiyuhang0 095a307
refactor
shiyuhang0 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,120 @@ | ||
--- | ||
title: TiDB Serverless Export (Beta) | ||
summary: Learn how to export data from TiDB Serverless clusters. | ||
--- | ||
|
||
# TiDB Serverless Export (Beta) | ||
|
||
TiDB Serverless Export (Beta) is a service that enables you to export data from a TiDB Serverless cluster to local storage or an external storage service. You can use the exported data for backup, migration, data analysis, or other purposes. | ||
|
||
While you can also export data using `mysqldump`, TiDB Dumpling, or other tools, TiDB Serverless Export offers a more convenient and efficient way to export data from a TiDB Serverless cluster. It brings the following benefits: | ||
|
||
- Convenience: the export service provides a simple and easy-to-use way to export data from a TiDB Serverless cluster, eliminating the need for additional tools or resources. | ||
- Isolation: the export service uses separate computing resources, ensuring isolation from the resources used by your online services. | ||
- Consistency: the export service ensures the consistency of the exported data without causing locks, which does not affect your online services. | ||
|
||
## Features | ||
|
||
### Location of files | ||
|
||
You can export data to the local storage or an external storage service. | ||
|
||
**Local storage** | ||
|
||
There are some limitations when you export data to local storage: | ||
|
||
1. You are not allowed to export multiple databases at the same time. | ||
2. The exported data will be expired after two days, please download the data in time. | ||
3. The exported data will be saved in the stashing area, which offers 250 GB storage space for each organization per region. If the storage space is full, you will not be able to export data to local. | ||
|
||
**[Amazon S3](https://aws.amazon.com/s3/)** | ||
|
||
You need to provide the credentials of the S3 bucket. Supported credentials include: | ||
|
||
- [Access Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html): The long-term credentials for an IAM user or the AWS account root user. Please make sure the access key has the necessary permissions to your S3 bucket, we recommend you create a new bucket with full s3 access. | ||
|
||
> **Note:** | ||
> | ||
> We recommend you export to the external storage service such as S3 when you want to export a large amount of data. | ||
|
||
### Data Filtering | ||
|
||
You can filter data by specifying the database and table you want to export. If you do not specify the table, we will export all tables in the specified database. If you do not specify the database, we will export all databases in the cluster. | ||
|
||
> **Note:** | ||
> | ||
> You must specify the database when you export data to local storage. | ||
|
||
### Data Formats | ||
|
||
shiyuhang0 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
You can export data in the following formats: | ||
|
||
- SQL(default): export data in SQL format. | ||
- CSV: export data in CSV format. | ||
|
||
### Data Compression | ||
|
||
You can compress the exported data in the following algorithms: | ||
|
||
- gzip(default): compress the exported data with gzip | ||
- snappy: compress the exported data with snappy. | ||
- zstd: compress the exported data with zstd. | ||
- none: do not compress the exported data. | ||
|
||
### Cancel Export | ||
|
||
You can cancel an export job that is in running state. | ||
|
||
## Examples | ||
|
||
Now, you can manage exports with [TiDB Cloud CLI](/tidb-cloud/cli-reference.md). | ||
|
||
### Export to local | ||
|
||
First, create an export job which specifies the database and table you want to export. It will output the export ID. | ||
|
||
```sh | ||
ticloud serverless export create -c <cluster-id> --database <database> --table <table> | ||
``` | ||
|
||
Then, download the exported data after the export is succeeded. | ||
|
||
```sh | ||
ticloud serverless export download -c <cluster-id> -e <export-id> | ||
``` | ||
shiyuhang0 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Export to S3 | ||
|
||
```sh | ||
ticloud serverless export create -c <cluster-id> --bucket-uri <bucket-uri> --access-key-id <access-key-id> --secret-access-key <secret-access-key> | ||
``` | ||
|
||
shiyuhang0 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
### Export with CSV format | ||
|
||
```sh | ||
ticloud serverless export create -c <cluster-id> --file-type CSV | ||
``` | ||
|
||
### Export the whole database | ||
|
||
```sh | ||
ticloud serverless export create -c <cluster-id> --database <database> | ||
``` | ||
|
||
### Export with snappy compression. | ||
|
||
```sh | ||
ticloud serverless export create -c <cluster-id> --compress snappy | ||
``` | ||
|
||
### Cancel an export job | ||
|
||
```sh | ||
ticloud serverless export cancel -c <cluster-id> -e <export-id> | ||
``` | ||
|
||
## Pricing | ||
|
||
You will only be charged for a successful or canceled export. | ||
shiyuhang0 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The export service is free during the beta period. You only need to pay for the [Request Units (RUs)](/tidb-cloud/tidb-cloud-glossary.md#request-unit) generated during the export process. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it mean the exported data will be removed from the stashing area when it has been downloaded for two days?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it will be removed from the stashing area after three days, but it will not be allowed to download after two days.