Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/master'
Browse files Browse the repository at this point in the history
  • Loading branch information
qiancai committed Oct 31, 2024
2 parents a6a9c25 + 0eb1cd5 commit d9fe7b3
Show file tree
Hide file tree
Showing 7 changed files with 44 additions and 2 deletions.
2 changes: 2 additions & 0 deletions migrate-from-csv-files-to-tidb.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ Put all the CSV files in the same directory. If you need TiDB Lightning to recog
- If a CSV file contains the data for an entire table, name the file `${db_name}.${table_name}.csv`.
- If the data of one table is separated into multiple CSV files, append a numeric suffix to these CSV files. For example, `${db_name}.${table_name}.003.csv`. The numeric suffixes can be inconsecutive but must be in ascending order. You also need to add extra zeros before the number to ensure all the suffixes are in the same length.

TiDB Lightning recursively searches for all `.csv` files in this directory and its subdirectories.

## Step 2. Create the target table schema

Because CSV files do not contain schema information, before importing data from CSV files into TiDB, you need to create the target table schema. You can create the target table schema by either of the following two methods:
Expand Down
1 change: 1 addition & 0 deletions releases/release-6.5.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -351,6 +351,7 @@ Compared with TiDB [6.4.0-DMR](/releases/release-6.4.0.md), TiDB 6.5.0 introduce
| TiDB | [`server-memory-quota`](/tidb-configuration-file.md#server-memory-quota-new-in-v409) | Deprecated | Starting from v6.5.0, this configuration item is deprecated. Instead, use the system variable [`tidb_server_memory_limit`](/system-variables.md#tidb_server_memory_limit-new-in-v640) to manage memory globally. |
| TiDB | [`disconnect-on-expired-password`](/tidb-configuration-file.md#disconnect-on-expired-password-new-in-v650) | Newly added | Determines whether TiDB disconnects the client connection when the password is expired. The default value is `true`, which means the client connection is disconnected when the password is expired. |
| TiKV | `raw-min-ts-outlier-threshold` | Deleted | This configuration item was deprecated in v6.4.0 and deleted in v6.5.0. |
| TiKV | [`raft-engine.bytes-per-sync`](/tikv-configuration-file.md#bytes-per-sync-2) | Deprecated | Starting from v6.5.0, Raft Engine writes logs to disk directly without buffering. Therefore, this configuration item is deprecated and no longer functional. |
| TiKV | [`cdc.min-ts-interval`](/tikv-configuration-file.md#min-ts-interval) | Modified | To reduce CDC latency, the default value is changed from `"1s"` to `"200ms"`. |
| TiKV | [`memory-use-ratio`](/tikv-configuration-file.md#memory-use-ratio-new-in-v650) | Newly added | Indicates the ratio of available memory to total system memory in PITR log recovery. |
| TiCDC | [`sink.terminator`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | Newly added | Indicates the row terminator, which is used for separating two data change events. The value is empty by default, which means `\r\n` is used. |
Expand Down
9 changes: 9 additions & 0 deletions system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -2662,6 +2662,15 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1;
- Controls whether to enable the temporary storage for some operators when a single SQL statement exceeds the memory quota specified by the system variable [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query).
- Before v6.3.0, you can enable or disable this feature by using the TiDB configuration item `oom-use-tmp-storage`. After upgrading the cluster to v6.3.0 or a later version, the TiDB cluster will initialize this variable using the value of `oom-use-tmp-storage` automatically. After that, changing the value of `oom-use-tmp-storage` **does not** take effect anymore.
### tidb_enable_stats_owner <span class="version-mark">New in v8.4.0</span>
- Scope: GLOBAL
- Persists to cluster: No, only applicable to the current TiDB instance that you are connecting to.
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Default value: `ON`
- Possible values: `OFF`, `ON`
- This variable controls whether the corresponding TiDB instance can run [automatic statistics update](/statistics.md#automatic-update) tasks. If there is only one TiDB instance in the current TiDB cluster, you cannot disable automatic statistics update on this instance, which means you cannot set this variable to `OFF`.
### tidb_enable_stmt_summary <span class="version-mark">New in v3.0.4</span>
> **Note:**
Expand Down
14 changes: 14 additions & 0 deletions ticdc/ticdc-bidirectional-replication.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,20 @@ In short, in BDR mode, TiCDC only replicates replicable DDLs in the PRIMARY clus
### Replication scenarios of replicable DDLs

1. Choose a TiDB cluster and execute `ADMIN SET BDR ROLE PRIMARY` to set it as the primary cluster.

```sql
ADMIN SET BDR ROLE PRIMARY;
Query OK, 0 rows affected
Time: 0.003s

ADMIN SHOW BDR ROLE;
+----------+
| BDR_ROLE |
+----------+
| primary |
+----------+
```

2. On other TiDB clusters, execute `ADMIN SET BDR ROLE SECONDARY` to set them as the secondary clusters.
3. Execute **replicable DDLs** on the primary cluster. The successfully executed DDLs will be replicated to the secondary clusters by TiCDC.

Expand Down
7 changes: 6 additions & 1 deletion ticdc/ticdc-changefeed-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,11 @@ case-sensitive = false
# The format of this parameter is "h m s", for example, "1h30m30s".
changefeed-error-stuck-duration = "30m"
# The default value is false, indicating that bi-directional replication (BDR) mode is not enabled.
# To set up BDR clusters using TiCDC, modify this parameter to `true` and set the TiDB clusters to BDR mode.
# For more information, see https://docs.pingcap.com/tidb/stable/ticdc-bidirectional-replication.
# bdr-mode = false
[mounter]
# The number of threads with which the mounter decodes KV data. The default value is 16.
# worker-num = 16
Expand All @@ -100,7 +105,7 @@ rules = ['*.*', '!test.*']
# The second event filter rule.
# matcher = ["test.fruit"] # matcher is an allow list, which means this rule only applies to the fruit table in the test database.
# ignore-event = ["drop table", "delete"] # Ignore the `drop table` DDL events and the `delete` DML events.
# ignore-event = ["drop table", "delete"] # Ignore the `drop table` DDL events and the `delete` DML events. Note that when a value in the clustered index column is updated in TiDB, TiCDC splits an `UPDATE` event into `DELETE` and `INSERT` events. TiCDC cannot identify such events as `UPDATE` events and thus cannot correctly filter out such events.
# ignore-sql = ["^drop table", "alter table"] # Ignore DDL statements that start with `drop table` or contain `alter table`.
# ignore-insert-value-expr = "price > 1000 and origin = 'no where'" # Ignore insert DMLs that contain the conditions "price > 1000" and "origin = 'no where'".
Expand Down
7 changes: 7 additions & 0 deletions tidb-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -955,6 +955,13 @@ Configuration items related to read isolation.
- The value of this configuration will initialize the value of the system variable [`tidb_enable_ddl`](/system-variables.md#tidb_enable_ddl-new-in-v630)
- Before v6.3.0, this configuration is set by `run-ddl`.

### `tidb_enable_stats_owner` <span class="version-mark">New in v8.4.0</span>

- This configuration controls whether the corresponding TiDB instance can run [automatic statistics update](/statistics.md#automatic-update) tasks.
- Default value: `true`
- Possible values: `true`, `false`
- The value of this configuration will initialize the value of the system variable [`tidb_enable_stats_owner`](/system-variables.md#tidb_enable_stats_owner-new-in-v840).

### `tidb_stmt_summary_enable_persistent` <span class="version-mark">New in v6.6.0</span>

> **Warning:**
Expand Down
6 changes: 5 additions & 1 deletion tikv-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -1954,9 +1954,13 @@ Configuration items related to Raft Engine.

### `bytes-per-sync`

> **Warning:**
>
> Starting from v6.5.0, Raft Engine writes logs to disk directly without buffering. Therefore, this configuration item is deprecated and no longer functional.
+ Specifies the maximum accumulative size of buffered writes. When this configuration value is exceeded, buffered writes are flushed to the disk.
+ If you set this configuration item to `0`, incremental sync is disabled.
+ Default value: `"4MiB"`
+ Before v6.5.0, the default value is `"4MiB"`.

### `target-file-size`

Expand Down

0 comments on commit d9fe7b3

Please sign in to comment.