Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dumpling: merge userguide into docs #19890

Merged
merged 6 commits into from
Jan 20, 2025
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions dumpling-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -411,3 +411,38 @@
| `--tidb-mem-quota-query` | The memory limit of exporting SQL statements by a single line of Dumpling command, and the unit is byte. For v4.0.10 or later versions, if you do not set this parameter, TiDB uses the value of the `mem-quota-query` configuration item as the memory limit value by default. For versions earlier than v4.0.10, the parameter value defaults to 32 GB. | 34359738368 |
| `--params` | Specifies the session variable for the connection of the database to be exported. The required format is `"character_set_client=latin1,character_set_connection=latin1"` |
| `-c` or `--compress` | Compresses the CSV and SQL data and table structure files exported by Dumpling. It supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. | "" |

## Output filename template
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

The `--output-filename-template` argument specifies how all files are named, before including the file extensions. It accepts strings in the [Go `text/template` syntax](https://golang.org/pkg/text/template/).

The following fields are available to the template:

* `.DB`: the database name
* `.Table`: the table name or the object name
* `.Index`: when a table is split into multiple files, this is the 0-based sequence number indicating which part you are dumping

The database and table names might contain special characters such as `/` that is not acceptable in the file system. Thus, Dumpling also provides a function `fn` to percent-escape these special characters:

* U+0000 to U+001F (control characters)
* `/`, `\`, `<`, `>`, `:`, `"`, `*`, `?` (invalid Windows path characters)
* `.` (database/table name separator)
* `-`, if used as part of `-schema`
dveeden marked this conversation as resolved.
Show resolved Hide resolved

For instance, using `--output-filename-template '{{fn .Table}}.{{printf "%09d" .Index}}'`, Dumpling will write the table `"db"."tbl:normal"` into files named such as `tbl%3Anormal.000000000.sql`, `tbl%3Anormal.000000001.sql`, and so on.

Check failure on line 432 in dumpling-overview.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [PingCAP.Latin] Use 'such as' instead of 'and so on.'. Raw Output: {"message": "[PingCAP.Latin] Use 'such as' instead of 'and so on.'.", "location": {"path": "dumpling-overview.md", "range": {"start": {"line": 432, "column": 227}}}, "severity": "ERROR"}
dveeden marked this conversation as resolved.
Show resolved Hide resolved

Besides the data files, you can also define named templates to replace the file name of the schema files. The following table shows the default configurations.
dveeden marked this conversation as resolved.
Show resolved Hide resolved

| Name | Content |
|------|---------|
| data | `{{fn .DB}}.{{fn .Table}}.{{.Index}}` |
| schema | `{{fn .DB}}-schema-create` |
| table | `{{fn .DB}}.{{fn .Table}}-schema` |
| event | `{{fn .DB}}.{{fn .Table}}-schema-post` |
| function | `{{fn .DB}}.{{fn .Table}}-schema-post` |
| procedure | `{{fn .DB}}.{{fn .Table}}-schema-post` |
| sequence | `{{fn .DB}}.{{fn .Table}}-schema-sequence` |
| trigger | `{{fn .DB}}.{{fn .Table}}-schema-triggers` |
| view | `{{fn .DB}}.{{fn .Table}}-schema-view` |

For instance, using `--output-filename-template '{{define "table"}}{{fn .Table}}.$schema{{end}}{{define "data"}}{{fn .Table}}.{{printf "%09d" .Index}}{{end}}'`, Dumpling will write the schema of the table `"db"."tbl:normal"` into the file `tbl%3Anormal.$schema.sql`, and data into the files like `tbl%3Anormal.000000000.sql`.
dveeden marked this conversation as resolved.
Show resolved Hide resolved
Loading