Skip to content

Commit

Permalink
ticdc: remove canal protocol document (#16773) (#16797)
Browse files Browse the repository at this point in the history
  • Loading branch information
ti-chi-bot authored Mar 21, 2024
1 parent a48fef9 commit beca2f5
Show file tree
Hide file tree
Showing 4 changed files with 9 additions and 9 deletions.
6 changes: 3 additions & 3 deletions ticdc/manage-ticdc.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ The following are descriptions of parameters and parameter values that can be co
| `max-message-bytes` | The maximum size of data that is sent to Kafka broker each time (optional, `10MB` by default). From v5.0.6 and v4.0.6, the default value has changed from 64MB and 256MB to 10MB. |
| `replication-factor` | The number of Kafka message replicas that can be saved (optional, `1` by default) |
| `compression` | The compression algorithm used when sending messages (value options are `none`, `lz4`, `gzip`, `snappy`, and `zstd`; `none` by default). |
| `protocol` | The protocol with which messages are output to Kafka. The value options are `canal-json`, `open-protocol`, `canal`, `avro` and `maxwell`. |
| `protocol` | The protocol with which messages are output to Kafka. The value options are `canal-json`, `open-protocol`, `avro` and `maxwell`. |
| `auto-create-topic` | Determines whether TiCDC creates the topic automatically when the `topic-name` passed in does not exist in the Kafka cluster (optional, `true` by default) |
| `enable-tidb-extension` | Optional. `false` by default. When the output protocol is `canal-json`, if the value is `true`, TiCDC sends Resolved events and adds the TiDB extension field to the Kafka message. From v6.1.0, this parameter is also applicable to the `avro` protocol. If the value is `true`, TiCDC adds three TiDB extension fields to the Kafka message. |
| `max-batch-size` | New in v4.0.9. If the message protocol supports outputting multiple data changes to one Kafka message, this parameter specifies the maximum number of data changes in one Kafka message. It currently takes effect only when Kafka's `protocol` is `open-protocol`. (optional, `16` by default) |
Expand Down Expand Up @@ -341,7 +341,7 @@ The following are descriptions of parameters that can be configured for the sink
| `auth.tls` | Uses the TLS mode to verify the downstream Pulsar (optional). For example, `auth=tls&auth.tlsCertFile=/path/to/cert&auth.tlsKeyFile=/path/to/key`. |
| `auth.token` | Uses the token mode to verify the downstream Pulsar (optional). For example, `auth=token&auth.token=secret-token` or `auth=token&auth.file=path/to/secret-token-file`. |
| `name` | The name of Pulsar producer in TiCDC (optional) |
| `protocol` | The protocol with which messages are output to Pulsar. The value options are `canal-json`, `open-protocol`, `canal`, `avro`, and `maxwell`. |
| `protocol` | The protocol with which messages are output to Pulsar. The value options are `canal-json`, `open-protocol`, `avro`, and `maxwell`. |
| `maxPendingMessages` | Sets the maximum size of the pending message queue, which is optional and defaults to 1000. For example, pending for the confirmation message from Pulsar. |
| `disableBatching` | Disables automatically sending messages in batches (optional) |
| `batchingMaxPublishDelay` | Sets the duration within which the messages sent are batched (default: 10ms) |
Expand Down Expand Up @@ -650,7 +650,7 @@ worker-num = 16
# {matcher = ['test6.*'], partition = "ts"}
# ]
# For the sink of MQ type, you can specify the protocol format of the message.
# Currently the following protocols are supported: canal-json, open-protocol, canal, avro, and maxwell.
# Currently the following protocols are supported: canal-json, open-protocol, avro, and maxwell.
# protocol = "canal-json"
```
Expand Down
8 changes: 4 additions & 4 deletions ticdc/ticdc-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,20 +106,20 @@ If you use the `cdc cli changefeed create` command without specifying the `-conf
- Enables the Old Value feature
- Skips replicating tables that do not contain [valid indexes](/ticdc/ticdc-overview.md#restrictions)

## Does TiCDC support outputting data changes in the Canal format?
## Does TiCDC support outputting data changes in the Canal protocol?

Yes. To enable Canal output, specify the protocol as `canal` in the `--sink-uri` parameter. For example:
Yes. Note that for the Canal protocol, TiCDC only supports the JSON output format, while the protobuf format is not officially supported yet. To enable Canal output, specify `protocol` as `canal-json` in the `--sink-uri` configuration. For example:

{{< copyable "shell-regular" >}}

```shell
cdc cli changefeed create --pd=http://10.0.10.25:2379 --sink-uri="kafka://127.0.0.1:9092/cdc-test?kafka-version=2.4.0&protocol=canal" --config changefeed.toml
cdc cli changefeed create --pd=http://10.0.10.25:2379 --sink-uri="kafka://127.0.0.1:9092/cdc-test?kafka-version=2.4.0&protocol=canal-json" --config changefeed.toml
```

> **Note:**
>
> * This feature is introduced in TiCDC 4.0.2.
> * TiCDC currently supports outputting data changes in the Canal format only to MQ sinks such as Kafka and Pulsar.
> * TiCDC currently supports outputting data changes in the Canal-JSON format only to MQ sinks such as Kafka and Pulsar.
For more information, refer to [Create a replication task](/ticdc/manage-ticdc.md#create-a-replication-task).

Expand Down
2 changes: 1 addition & 1 deletion ticdc/ticdc-open-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ The configuration parameters of sink are as follows:

`matcher`: The matching syntax of matcher is the same as the filter rule syntax.

`protocol`: For the sink of MQ type, you can specify the protocol format of the message. Currently the following protocols are supported: `canal-json`, `open-protocol`, `canal`, `avro`, and `maxwell`.
`protocol`: For the sink of MQ type, you can specify the protocol format of the message. Currently the following protocols are supported: `canal-json`, `open-protocol`, `avro`, and `maxwell`.

### Example

Expand Down
2 changes: 1 addition & 1 deletion ticdc/troubleshoot-ticdc.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ Since v4.0.9, you can try to enable the unified sorter feature in your replicati

## After I upgrade the TiCDC cluster to v4.0.8, the `[CDC:ErrKafkaInvalidConfig]Canal requires old value to be enabled` error is reported when I execute a changefeed. What should I do?

Since v4.0.8, if the `canal-json`, `canal` or `maxwell` protocol is used for output in a changefeed, TiCDC enables the old value feature automatically. However, if you have upgraded TiCDC from an earlier version to v4.0.8 or later, when the changefeed uses the `canal-json`, `canal` or `maxwell` protocol and the old value feature is disabled, this error is reported.
Since v4.0.8, if the `canal-json` or `maxwell` protocol is used for output in a changefeed, TiCDC enables the old value feature automatically. However, if you have upgraded TiCDC from an earlier version to v4.0.8 or later, when the changefeed uses the `canal-json` or `maxwell` protocol and the old value feature is disabled, this error is reported.

To fix the error, take the following steps:

Expand Down

0 comments on commit beca2f5

Please sign in to comment.