Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connect to Kafka PL/PSC service from TiDB Cloud #19323

Merged
merged 66 commits into from
Jan 2, 2025
Merged
Show file tree
Hide file tree
Changes from 63 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
5e94493
changefeed: add private link access steps
grovecai Oct 17, 2024
45491b0
changefeed: setup Kafka private link service in AWS
grovecai Oct 28, 2024
35d80d6
restruct to make it simple for reconfigure exsting Kafka cluster
grovecai Nov 5, 2024
359866d
add FAQ for AWS part
grovecai Nov 5, 2024
9629427
add output values to network section
grovecai Nov 6, 2024
14247e6
rafactor doc since we rename/remove fields
grovecai Nov 6, 2024
da07007
Add setup self hosted Kafka PSC for GCP
grovecai Nov 7, 2024
c9c3d91
Fix typo and images
grovecai Nov 11, 2024
d06e84c
fix terms
grovecai Nov 11, 2024
5b29bae
Apply suggestions from code review
hfxsd Nov 27, 2024
c0049dc
Update setup-self-hosted-kafka-pls.md
hfxsd Nov 27, 2024
c909031
Update setup-self-hosted-kafka-psc.md
hfxsd Nov 28, 2024
ecfb6df
refined wording and format
hfxsd Dec 5, 2024
f2d7676
Update TOC-tidb-cloud.md
hfxsd Dec 5, 2024
9de77a7
refined - ongoing
hfxsd Dec 5, 2024
408b9ad
nearly finished
hfxsd Dec 5, 2024
ec2533b
fix formats
hfxsd Dec 6, 2024
b7ff092
Update changefeed-sink-to-apache-kafka.md
hfxsd Dec 6, 2024
c7ed9bc
removed level5 and level6 headings
hfxsd Dec 6, 2024
acfb472
fix formats
hfxsd Dec 6, 2024
d936516
Update setup-self-hosted-kafka-psc.md
hfxsd Dec 9, 2024
dc11dbf
fix term and remove uneccessary steps
grovecai Dec 9, 2024
3504c09
Apply suggestions from code review
hfxsd Dec 10, 2024
1939d7c
Apply suggestions from code review
hfxsd Dec 10, 2024
b1d58a8
refined
hfxsd Dec 10, 2024
52bfa77
Apply suggestions from code review
hfxsd Dec 10, 2024
65a5b18
Update tidb-cloud/changefeed-sink-to-apache-kafka.md
hfxsd Dec 19, 2024
9b44054
fix spelling errors
hfxsd Dec 19, 2024
d47a249
Use simpletab in Step 2
hfxsd Dec 19, 2024
aa0f3d7
refine wording
hfxsd Dec 19, 2024
db140d2
refine lists in tables
hfxsd Dec 19, 2024
063aef0
Apply suggestions from code review
hfxsd Dec 24, 2024
84c2dc3
changefeed: support more partition dispatchers and more data formats …
grovecai Dec 26, 2024
d526581
Apply suggestions from code review
hfxsd Dec 26, 2024
bead4c1
Update tidb-cloud/changefeed-sink-to-apache-kafka.md
grovecai Dec 26, 2024
ae30ab7
Apply suggestions from code review
hfxsd Dec 26, 2024
a3a747c
add version restrictions
grovecai Dec 27, 2024
b329918
Apply suggestions from code review
hfxsd Dec 31, 2024
a0e89ac
Update tidb-cloud/tidb-cloud-billing-ticdc-rcu.md
grovecai Jan 2, 2025
aca9214
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
32e79e1
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
24b2664
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
2e9e05d
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
b91ddec
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
102b916
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
9cd4466
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
57c81a3
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
d26b381
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
80a1346
Update tidb-cloud/setup-self-hosted-kafka-pls.md
grovecai Jan 2, 2025
1c626b7
Update tidb-cloud/setup-self-hosted-kafka-psc.md
grovecai Jan 2, 2025
39ce3d2
Apply suggestions from code review
grovecai Jan 2, 2025
031db20
tidb-cloud: use keywords in file name
lilin90 Jan 2, 2025
d7f765f
Merge branch 'changefeed-pl' of https://github.com/grovecai/docs into…
lilin90 Jan 2, 2025
3751006
Apply suggestions from code review
lilin90 Jan 2, 2025
2244cf0
change images
grovecai Jan 2, 2025
26ca6d2
Apply suggestions from code review
grovecai Jan 2, 2025
ad04e26
Revert "tidb-cloud: use keywords in file name"
grovecai Jan 2, 2025
a1974bb
Revert "Revert "tidb-cloud: use keywords in file name""
grovecai Jan 2, 2025
13f438a
Update tidb-cloud/setup-self-hosted-kafka-private-link-service.md
grovecai Jan 2, 2025
d86cd20
Update tidb-cloud/setup-self-hosted-kafka-private-link-service.md
grovecai Jan 2, 2025
6c53953
Update tidb-cloud/setup-self-hosted-kafka-private-service-connect.md
grovecai Jan 2, 2025
f0d1cb2
Update tidb-cloud/setup-self-hosted-kafka-private-service-connect.md
grovecai Jan 2, 2025
4130552
Update tidb-cloud/setup-self-hosted-kafka-private-service-connect.md
grovecai Jan 2, 2025
a39f316
fix link
hfxsd Jan 2, 2025
89efb57
Update tidb-cloud/setup-self-hosted-kafka-private-link-service.md
hfxsd Jan 2, 2025
3614d23
Apply suggestions from code review
hfxsd Jan 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions TOC-tidb-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,9 @@
- [To Kafka Sink](/tidb-cloud/changefeed-sink-to-apache-kafka.md)
- [To TiDB Cloud Sink](/tidb-cloud/changefeed-sink-to-tidb-cloud.md)
- [To Cloud Storage](/tidb-cloud/changefeed-sink-to-cloud-storage.md)
- Reference
- [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-self-hosted-kafka-private-link-service.md)
- [Set Up Self-Hosted Kafka Private Service Connect in Google Cloud](/tidb-cloud/setup-self-hosted-kafka-private-service-connect.md)
- Disaster Recovery
- [Recovery Group Overview](/tidb-cloud/recovery-group-overview.md)
- [Get Started](/tidb-cloud/recovery-group-get-started.md)
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
145 changes: 126 additions & 19 deletions tidb-cloud/changefeed-sink-to-apache-kafka.md

Large diffs are not rendered by default.

761 changes: 761 additions & 0 deletions tidb-cloud/setup-self-hosted-kafka-private-link-service.md

Large diffs are not rendered by default.

704 changes: 704 additions & 0 deletions tidb-cloud/setup-self-hosted-kafka-private-service-connect.md

Large diffs are not rendered by default.

12 changes: 10 additions & 2 deletions tidb-cloud/tidb-cloud-billing-ticdc-rcu.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@ aliases: ['/tidbcloud/tidb-cloud-billing-tcu']

# Changefeed Billing

## RCU cost

TiDB Cloud measures the capacity of [changefeeds](/tidb-cloud/changefeed-overview.md) in TiCDC Replication Capacity Units (RCUs). When you [create a changefeed](/tidb-cloud/changefeed-overview.md#create-a-changefeed) for a cluster, you can select an appropriate specification. The higher the RCU, the better the replication performance. You will be charged for these TiCDC changefeed RCUs.

## Number of TiCDC RCUs
### Number of TiCDC RCUs

The following table lists the specifications and corresponding replication performances for changefeeds:

Expand All @@ -26,6 +28,12 @@ The following table lists the specifications and corresponding replication perfo
>
> The preceding performance data is for reference only and might vary in different scenarios. It is strongly recommended that you conduct a real workload test before using the changefeed feature in a production environment. For further assistance, contact [TiDB Cloud support](/tidb-cloud/tidb-cloud-support.md#get-support-for-a-cluster).

## Price
### Price

To learn about the supported regions and the price of TiDB Cloud for each TiCDC RCU, see [Changefeed Cost](https://www.pingcap.com/tidb-cloud-pricing-details/#changefeed-cost).

## Private Data Link cost

If you choose the **Private Link** or **Private Service Connect** network connectivity method, additional **Private Data Link** costs will be incurred. These charges fall under the [Data Transfer Cost](https://www.pingcap.com/tidb-dedicated-pricing-details/#data-transfer-cost) category.

The price of **Private Data Link** is **$0.01/GiB**, the same as **Data Processed** of [AWS Interface Endpoint pricing](https://aws.amazon.com/privatelink/pricing/#Interface_Endpoint_pricing) and **Consumer data processing** of [Google Cloud Private Service Connect pricing](https://cloud.google.com/vpc/pricing#psc-forwarding-rules).
Loading