Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dumpling: merge userguide into docs #19890

Merged
merged 6 commits into from
Jan 20, 2025
Merged
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions dumpling-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -411,3 +411,38 @@
| `--tidb-mem-quota-query` | The memory limit of exporting SQL statements by a single line of Dumpling command, and the unit is byte. For v4.0.10 or later versions, if you do not set this parameter, TiDB uses the value of the `mem-quota-query` configuration item as the memory limit value by default. For versions earlier than v4.0.10, the parameter value defaults to 32 GB. | 34359738368 |
| `--params` | Specifies the session variable for the connection of the database to be exported. The required format is `"character_set_client=latin1,character_set_connection=latin1"` |
| `-c` or `--compress` | Compresses the CSV and SQL data and table structure files exported by Dumpling. It supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. | "" |

## Output filename template
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

The `--output-filename-template` argument defines the naming convention for all files, excluding the file extensions. It accepts strings in the [Go `text/template` syntax](https://golang.org/pkg/text/template/).
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

The following fields are available for the template:

* `.DB`: the database name
* `.Table`: the table name or the object name
* `.Index`: the 0-based sequence number of the file when a table is split into multiple files, indicating which part is being dumped
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

Database and table names might contain special characters, such as /, that are not allowed in file systems. To handle this, Dumpling provides the `fn` function to percent-encode these special characters:
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

* U+0000 to U+001F (control characters)
* `/`, `\`, `<`, `>`, `:`, `"`, `*`, `?` (invalid Windows path characters)
* `.` (database or table name separator)
* `-`, if used as part of `-schema`

For example, using `--output-filename-template '{{fn .Table}}.{{printf "%09d" .Index}}'`, Dumpling will write the table `"db"."tbl:normal"` into files named `tbl%3Anormal.000000000.sql`, `tbl%3Anormal.000000001.sql`, and so on.

Check failure on line 432 in dumpling-overview.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [PingCAP.Latin] Use 'such as' instead of 'and so on.'. Raw Output: {"message": "[PingCAP.Latin] Use 'such as' instead of 'and so on.'.", "location": {"path": "dumpling-overview.md", "range": {"start": {"line": 432, "column": 218}}}, "severity": "ERROR"}
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

In addition to data files, you can define named templates to replace the file name of the schema files. The following table shows the default configurations.
hfxsd marked this conversation as resolved.
Show resolved Hide resolved

| Name | Content |
|------|---------|
| data | `{{fn .DB}}.{{fn .Table}}.{{.Index}}` |
| schema | `{{fn .DB}}-schema-create` |
| table | `{{fn .DB}}.{{fn .Table}}-schema` |
| event | `{{fn .DB}}.{{fn .Table}}-schema-post` |
| function | `{{fn .DB}}.{{fn .Table}}-schema-post` |
| procedure | `{{fn .DB}}.{{fn .Table}}-schema-post` |
| sequence | `{{fn .DB}}.{{fn .Table}}-schema-sequence` |
| trigger | `{{fn .DB}}.{{fn .Table}}-schema-triggers` |
| view | `{{fn .DB}}.{{fn .Table}}-schema-view` |

For example, using `--output-filename-template '{{define "table"}}{{fn .Table}}.$schema{{end}}{{define "data"}}{{fn .Table}}.{{printf "%09d" .Index}}{{end}}'`, Dumpling will write the schema of the table `"db"."tbl:normal"` into a file named `tbl%3Anormal.$schema.sql`, and the data into files `tbl%3Anormal.000000000.sql`, `tbl%3Anormal.000000001.sql`, and so on.

Check failure on line 448 in dumpling-overview.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [PingCAP.Latin] Use 'such as' instead of 'and so on.'. Raw Output: {"message": "[PingCAP.Latin] Use 'such as' instead of 'and so on.'.", "location": {"path": "dumpling-overview.md", "range": {"start": {"line": 448, "column": 355}}}, "severity": "ERROR"}
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
Loading