tidb2dw as a cloud service

[zhangjinpeng87]

Background

Right now the tidb2dw is more like a standalone tool that can connect stream data in blob storage to big data cloud platform like snowflake and redshift. When it comes to TiDB Cloud, when we want to connect a tenant's data to big cloud platform, there would be a dedicated tidb2dw component to sync the data changes. This makes the deployment and maintenance more complicate.

Proposal

Evolute tidb2dw as a service when deploy on the tidb cloud, so we don't need to deploy a new tidb2dw component when there is a new sync to big data cloud platform.

Task breakdowns

• tidb2dw should only has dependency on shared storage #101
• Remove the dependency on upstream CDC cluster  #103