Skip to content

Commit

Permalink
remove redundant lightning command line flags doc (pingcap#17678)
Browse files Browse the repository at this point in the history
  • Loading branch information
zhangyangyu authored May 31, 2024
1 parent b480789 commit de3ea07
Show file tree
Hide file tree
Showing 3 changed files with 2 additions and 53 deletions.
51 changes: 0 additions & 51 deletions tidb-lightning/tidb-lightning-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -463,54 +463,3 @@ log-progress = "5m"
# The default value is 60 seconds.
# check-disk-quota = "60s"
```

## Command line parameters

### Usage of `tidb-lightning`

| Parameter | Explanation | Corresponding setting |
|:----|:----|:----|
| --config *file* | Reads global configuration from *file*. If not specified, the default configuration would be used. | |
| -V | Prints program version | |
| -d *directory* | Directory or [external storage URI](/external-storage-uri.md) of the data dump to read from | `mydumper.data-source-dir` |
| -L *level* | Log level: debug, info, warn, error, fatal (default = info) | `lightning.log-level` |
| -f *rule* | [Table filter rules](/table-filter.md) (can be specified multiple times) | `mydumper.filter` |
| --backend *[backend](/tidb-lightning/tidb-lightning-overview.md)* | Select an import mode. `local` refers to the physical import mode; `tidb` refers to the logical import mode. | `local` |
| --log-file *file* | Log file path. By default, it is `/tmp/lightning.log.{timestamp}`. If set to '-', it means that the log files will be output to stdout. | `lightning.log-file` |
| --status-addr *ip:port* | Listening address of the TiDB Lightning server | `lightning.status-port` |
| --pd-urls *host:port* | PD endpoint address | `tidb.pd-addr` |
| --tidb-host *host* | TiDB server host | `tidb.host` |
| --tidb-port *port* | TiDB server port (default = 4000) | `tidb.port` |
| --tidb-status *port* | TiDB status port (default = 10080) | `tidb.status-port` |
| --tidb-user *user* | User name to connect to TiDB | `tidb.user` |
| --tidb-password *password* | Password to connect to TiDB. The password can either be plaintext or Base64 encoded. | `tidb.password` |
| --enable-checkpoint *bool* | Whether to enable checkpoints (default = true) | `checkpoint.enable` |
| --analyze *level* | Analyze tables after importing. Available values are "required", "optional" (default value), and "off" | `post-restore.analyze` |
| --checksum *level* | Compare checksum after importing. Available values are "required" (default value), "optional", and "off" | `post-restore.checksum` |
| --check-requirements *bool* | Check cluster version compatibility before starting the task, and check whether TiKV has more than 10% free space left during running time. (default = true) | `lightning.check-requirements` |
| --ca *file* | CA certificate path for TLS connection | `security.ca-path` |
| --cert *file* | Certificate path for TLS connection | `security.cert-path` |
| --key *file* | Private key path for TLS connection | `security.key-path` |
| --server-mode | Start TiDB Lightning in server mode | `lightning.server-mode` |

If a command line parameter and the corresponding setting in the configuration file are both provided, the command line parameter will be used. For example, running `tiup tidb-lightning -L debug --config cfg.toml` would always set the log level to "debug" regardless of the content of `cfg.toml`.

## Usage of `tidb-lightning-ctl`

This tool can execute various actions given one of the following parameters:

| Parameter | Explanation |
|:----|:----|
| --compact | Performs a full compaction |
| --switch-mode *mode* | Switches every TiKV store to the given mode: normal, import |
| --fetch-mode | Prints the current mode of every TiKV store |
| --import-engine *uuid* | Imports the closed engine file from TiKV Importer into the TiKV cluster |
| --cleanup-engine *uuid* | Deletes the engine file from TiKV Importer |
| --checkpoint-dump *folder* | Dumps current checkpoint as CSVs into the folder |
| --checkpoint-error-destroy *tablename* | Removes the checkpoint and drops the table if it caused error |
| --checkpoint-error-ignore *tablename* | Ignores any error recorded in the checkpoint involving the given table |
| --checkpoint-remove *tablename* | Unconditionally removes the checkpoint of the table |

The *tablename* must either be a qualified table name in the form `` `db`.`tbl` `` (including the backquotes), or the keyword "all".

Additionally, all parameters of `tidb-lightning` described in the section above are valid in `tidb-lightning-ctl`.
2 changes: 1 addition & 1 deletion tidb-lightning/tidb-lightning-physical-import-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The backend for the physical import mode is `local`. You can modify it in `tidb-

1. Before importing data, TiDB Lightning automatically switches the TiKV nodes to "import mode", which improves write performance and stops auto-compaction. TiDB Lightning determines whether to pause global scheduling according to the TiDB Lightning version.

- Starting from v7.1.0, you can you can control the scope of pausing scheduling by using the TiDB Lightning parameter [`pause-pd-scheduler-scope`](/tidb-lightning/tidb-lightning-configuration.md).
- Starting from v7.1.0, you can control the scope of pausing scheduling by using the TiDB Lightning parameter [`pause-pd-scheduler-scope`](/tidb-lightning/tidb-lightning-configuration.md).
- For TiDB Lightning versions between v6.2.0 and v7.0.0, the behavior of pausing global scheduling depends on the TiDB cluster version. When the TiDB cluster >= v6.1.0, TiDB Lightning pauses scheduling for the Region that stores the target table data. After the import is completed, TiDB Lightning recovers scheduling. For other versions, TiDB Lightning pauses global scheduling.
- When TiDB Lightning < v6.2.0, TiDB Lightning pauses global scheduling.

Expand Down
2 changes: 1 addition & 1 deletion tidb-lightning/tidb-lightning-prechecks.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The following table describes each check item and detailed explanation.
| Cluster version and status| >= 5.3.0 | Check whether the cluster can be connected in the configuration, and whether the TiKV/PD/TiFlash version supports the physical import mode. |
| Permissions | >= 5.3.0 | When the data source is cloud storage (Amazon S3), check whether TiDB Lightning has the necessary permissions and make sure that the import will not fail due to lack of permissions. |
| Disk space | >= 5.3.0 | Check whether there is enough space on the local disk and on the TiKV cluster for importing data. TiDB Lightning samples the data sources and estimates the percentage of the index size from the sample result. Because indexes are included in the estimation, there might be cases where the size of the source data is less than the available space on the local disk, but still, the check fails. In the physical import mode, TiDB Lightning also checks whether the local storage is sufficient because external sorting needs to be done locally. For more details about the TiKV cluster space and local storage space (controlled by `sort-kv-dir`), see [Downstream storage space requirements](/tidb-lightning/tidb-lightning-requirements.md#storage-space-of-the-target-database) and [Resource requirements](/tidb-lightning/tidb-lightning-physical-import-mode.md#environment-requirements). |
| Region distribution status | >= 5.3.0 | Check whether the Regions in the TiKV cluster are distributed evenly and whether there are too many empty Regions. If the number of empty Regions exceeds max(1000, number of tables * 3), i.e. greater than the bigger one of "1000" or "3 times the number of tables ", then the import cannot be executed. |
| Region distribution status | >= 5.3.0 | Check whether the Regions in the TiKV cluster are distributed evenly and whether there are too many empty Regions. If the number of empty Regions exceeds max(1000, number of tables * 3), i.e. greater than the bigger one of "1000" or "3 times the number of tables", then the import cannot be executed. |
| Exceedingly Large CSV files in the data file | >= 5.3.0 | When there are CSV files larger than 10 GiB in the backup file and auto-slicing is not enabled (StrictFormat=false), it will impact the import performance. The purpose of this check is to remind you to ensure the data is in the right format and to enable auto-slicing. |
| Recovery from breakpoints | >= 5.3.0 | This check ensures that no changes are made to the source file or schema in the database during the breakpoint recovery process that would result in importing the wrong data. |
| Import into an existing table | >= 5.3.0 | When importing into an already created table, it checks, as much as possible, whether the source file matches the existing table. Check if the number of columns matches. If the source file has column names, check if the column names match. When there are default columns in the source file, it checks if the default columns have Default Value, and if they have, the check passes. |
Expand Down

0 comments on commit de3ea07

Please sign in to comment.