Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v7.5.1: ticdc: fix pulsar sink documentation error (#16557) #16562

Merged
50 changes: 41 additions & 9 deletions ticdc/ticdc-sink-to-pulsar.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,12 +99,16 @@ token-from-file="/data/pulsar/token-file.txt"
basic-user-name="root"
# Pulsar uses the basic account and password to authenticate the identity. Specify the password.
basic-password="password"
# The certificate path for Pulsar TLS encrypted authentication.
# The certificate path on the client, which is required when Pulsar enables the mTLS authentication.
auth-tls-certificate-path="/data/pulsar/certificate"
# The private key path for Pulsar TLS encrypted authentication.
# The private key path on the client, which is required when Pulsar enables the mTLS authentication.
auth-tls-private-key-path="/data/pulsar/certificate.key"
# Path to trusted certificate file of the Pulsar TLS encrypted authentication.
# The path to the trusted certificate file of the Pulsar TLS authentication, which is required when Pulsar enables the mTLS authentication or TLS encrypted transmission.
tls-trust-certs-file-path="/data/pulsar/tls-trust-certs-file"
# The path to the encrypted private key on the client, which is required when Pulsar enables TLS encrypted transmission.
tls-key-file-path="/data/pulsar/tls-key-file"
# The path to the encrypted certificate file on the client, which is required when Pulsar enables TLS encrypted transmission.
tls-certificate-file="/data/pulsar/tls-certificate-file"
# Pulsar oauth2 issuer-url. For more information, see the Pulsar website: https://pulsar.apache.org/docs/2.10.x/client-libraries-go/#tls-encryption-and-authentication
oauth2.oauth2-issuer-url="https://xxxx.auth0.com"
# Pulsar oauth2 audience
Expand Down Expand Up @@ -136,6 +140,32 @@ send-timeout=30
* You need to specify the `protocol` parameter when creating a changefeed. Currently, only the `canal-json` protocol is supported for replicating data to Pulsar.
* The `pulsar-producer-cache-size` parameter indicates the number of producers cached in the Pulsar client. Because each producer in Pulsar can only correspond to one topic, TiCDC adopts the LRU method to cache producers, and the default limit is 10240. If the number of topics you need to replicate is larger than the default value, you need to increase the number.

### TLS encrypted transmission

Starting from v7.5.1 and v8.0.0, TiCDC supports TLS encrypted transmission for Pulsar. The configuration example is as follows:
qiancai marked this conversation as resolved.
Show resolved Hide resolved

Sink URI:

```shell
--sink-uri="pulsar+ssl://127.0.0.1:6651/persistent://public/default/yktest?protocol=canal-json"
```

Configuration:

```toml
[sink.pulsar-config]
tls-trust-certs-file-path="/data/pulsar/tls-trust-certs-file"
```

If the `tlsRequireTrustedClientCertOnConnect=true` parameter is configured for your Pulsar server, you also need to configure the `tls-key-file-path` and `tls-certificate-file` parameters in the changefeed configuration file. For example:

```toml
[sink.pulsar-config]
tls-trust-certs-file-path="/data/pulsar/tls-trust-certs-file"
tls-certificate-file="/data/pulsar/tls-certificate-file"
tls-key-file-path="/data/pulsar/tls-key-file"
```

### TiCDC authentication and authorization for Pulsar

The following is a sample configuration when you use token authentication with Pulsar:
Expand Down Expand Up @@ -171,32 +201,34 @@ The following is a sample configuration when you use token authentication with P
token-from-file="/data/pulsar/token-file.txt"
```

- TLS encrypted authentication
- mTLS authentication

Sink URI:

```shell
--sink-uri="pulsar+ssl://127.0.0.1:6650/persistent://public/default/yktest?protocol=canal-json"
--sink-uri="pulsar+ssl://127.0.0.1:6651/persistent://public/default/yktest?protocol=canal-json"
```

Config parameters:

```toml
[sink.pulsar-config]
# Certificate path of the Pulsar TLS encrypted authentication
# Certificate path of the Pulsar mTLS authentication
auth-tls-certificate-path="/data/pulsar/certificate"
# Private key path of the Pulsar TLS encrypted authentication
# Private key path of the Pulsar mTLS authentication
auth-tls-private-key-path="/data/pulsar/certificate.key"
# Path to trusted certificate file of the Pulsar TLS encrypted authentication
# Path to the trusted certificate file of the Pulsar mTLS authentication
tls-trust-certs-file-path="/data/pulsar/tls-trust-certs-file"
```

- OAuth2 authentication

Starting from v7.5.1 and v8.0.0, TiCDC supports the OAuth2 authentication for Pulsar.
qiancai marked this conversation as resolved.
Show resolved Hide resolved

Sink URI:

```shell
--sink-uri="pulsar+ssl://127.0.0.1:6650/persistent://public/default/yktest?protocol=canal-json"
--sink-uri="pulsar://127.0.0.1:6650/persistent://public/default/yktest?protocol=canal-json"
```

Config parameters:
Expand Down
Loading