Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ticdc canal json doc (#14943) #15090

Merged
merged 5 commits into from
Oct 17, 2023
Merged

Update ticdc canal json doc (#14943) #15090

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
30dd6e7
Add temp.md
sdojjy Sep 28, 2023
b50c16e
Delete temp.md
sdojjy Sep 28, 2023
5be01f9
update changefeed configurations
sdojjy Sep 28, 2023
a540dd0
Apply suggestions from code review
sdojjy Oct 16, 2023
55d6a68
Update ticdc/ticdc-open-api-v2.md
qiancai Oct 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions ticdc/ticdc-canal-json.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -301,10 +301,10 @@ For example, the following 16 bytes `[5 7 10 15 36 50 43 99 120 60 38 255 254 45

The way that TiCDC implements the Canal-JSON data format, including the `Update` Event and the `mysqlType` field, differs from the official Canal. The following table shows the main differences.

| Item | TiCDC Canal-JSON | Canal |
|:----------------|:-------------------------|:-------------------------------------|
| Event of `Update` Type | The `old` field contains all the column data | The `old` field contains only the modified column data |
| `mysqlType` field | For types with parameters, it does not contain the information of the type parameter | For types with parameters, it contains the full information of the type parameter |
| Item | TiCDC Canal-JSON | Canal |
|:----------------|:---------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------|
| Event of `Update` Type | By default, the `old` field contains all the column data. When `only_output_updated_columns` is `true`, the `old` field contains only the modified column data. | The `old` field contains only the modified column data |
| `mysqlType` field | For types with parameters, it does not contain the information of the type parameter | For types with parameters, it contains the full information of the type parameter |

### Event of `Update` Type

Expand Down
23 changes: 12 additions & 11 deletions ticdc/ticdc-open-api-v2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -320,17 +320,18 @@ The `mounter` parameter is described as follows:

The `sink` parameters are described as follows:

| Parameter name | Description |
|:-----------------|:---------------------------------------|
| `column_selectors` | The column selector configuration. (Optional) |
| `csv` | The CSV configuration. (Optional) |
| `date_separator` | `STRING` type. Indicates the date separator type of the file directory. Value options are `none`, `year`, `month`, and `day`. `none` is the default value and means that the date is not separated. (Optional) |
| `dispatchers` | An configuration array for event dispatching. (Optional) |
| `encoder_concurrency` | `INT` type. The number of encoder threads in the MQ sink. The default value is `16`. (Optional) |
| `protocol` | `STRING` type. For MQ sinks, you can specify the protocol format of the message. The following protocols are currently supported: `canal-json`, `open-protocol`, `canal`, `avro`, and `maxwell`. |
| `schema_registry` | `STRING` type. The schema registry address. (Optional) |
| `terminator` | `STRING` type. The terminator is used to separate two data change events. The default value is null, which means `"\r\n"` is used as the terminator. (Optional) |
| `transaction_atomicity` | `STRING` type. The atomicity level of the transaction. (Optional) |
| Parameter name | Description |
|:-----------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `column_selectors` | The column selector configuration. (Optional) |
| `csv` | The CSV configuration. (Optional) |
| `date_separator` | `STRING` type. Indicates the date separator type of the file directory. Value options are `none`, `year`, `month`, and `day`. `none` is the default value and means that the date is not separated. (Optional) |
| `dispatchers` | An configuration array for event dispatching. (Optional) |
| `encoder_concurrency` | `INT` type. The number of encoder threads in the MQ sink. The default value is `16`. (Optional) |
| `protocol` | `STRING` type. For MQ sinks, you can specify the protocol format of the message. The following protocols are currently supported: `canal-json`, `open-protocol`, `canal`, `avro`, and `maxwell`. |
| `schema_registry` | `STRING` type. The schema registry address. (Optional) |
| `terminator` | `STRING` type. The terminator is used to separate two data change events. The default value is null, which means `"\r\n"` is used as the terminator. (Optional) |
| `transaction_atomicity` | `STRING` type. The atomicity level of the transaction. (Optional) |
| `only_output_updated_columns` | `BOOLEAN` type. For MQ sinks using the `canal-json` or `open-protocol` protocol, you can specify whether only output the modified columns. The default value is `false`. |

`sink.column_selectors` is an array. The parameters are described as follows:

Expand Down
Loading