Destination Clickhouse - Add Typing and Deduping #35339
Replies: 28 comments 10 replies
-
How can I revert to pre 1.0.0 upgrade of the Clickhouse destination? |
Beta Was this translation helpful? Give feedback.
-
Definitely we need normalization for clickhouse! Is there any way to stay on old version of clickhouse connection until normalization will be implemented? |
Beta Was this translation helpful? Give feedback.
-
airbyte should bring typing and deduping for clickhouse 🥹 |
Beta Was this translation helpful? Give feedback.
-
Strong +1 on this, we would benefit greatly from typing and deduping on ClickHouse as a destination. |
Beta Was this translation helpful? Give feedback.
-
In support to have this feature implemented. |
Beta Was this translation helpful? Give feedback.
-
FYI: The airbyte team published the connector docker images on dockerhub. You can 'revert' to an older connector version (with normalization for example) by uploading it's older image as a custom connector. See clickhouse connector images here cc @asimumba @kvosha |
Beta Was this translation helpful? Give feedback.
-
I've spent hours debugging why the final tables were not created before finding this discussion. |
Beta Was this translation helpful? Give feedback.
-
I can't understand why this feature is removed. Please bring this back in destination v2. For anyone wants this feature, you can go back to version 0.2.5 of clickhouse destination. Note that you'll be switched back to destination v1. |
Beta Was this translation helpful? Give feedback.
-
+1 need this |
Beta Was this translation helpful? Give feedback.
-
Holy hell, please bring normalization back +1000. |
Beta Was this translation helpful? Give feedback.
-
Pretty please bring normalisation back :) 🍰 |
Beta Was this translation helpful? Give feedback.
-
We need the normalization for our workflows. Please keep supporting this in Clickhouse! |
Beta Was this translation helpful? Give feedback.
-
Please bring it back! |
Beta Was this translation helpful? Give feedback.
-
A big heartfelt YES! Please bring Typing and Deduping to ClickHouse! |
Beta Was this translation helpful? Give feedback.
-
Please bring it back, I've spent countless hours trying to figure out what is wrong with my setup! We really need this for ClickHouse. |
Beta Was this translation helpful? Give feedback.
-
Just for the history - as of now - this is the number one issue by popularity in https://github.com/airbytehq/airbyte/discussions/categories/ideas-and-features |
Beta Was this translation helpful? Give feedback.
-
+1 - this would be extremely handy |
Beta Was this translation helpful? Give feedback.
-
+1 - Definitely will need this feature for Clickhouse to reduce the operational overhead. |
Beta Was this translation helpful? Give feedback.
-
Please bring normalization back for Clickhouse +1 |
Beta Was this translation helpful? Give feedback.
-
+1 - We are waiting for this for too long, it would be very good for our company! |
Beta Was this translation helpful? Give feedback.
-
Please bring normalization back for Clickhouse +1 |
Beta Was this translation helpful? Give feedback.
-
+1 - Please, please, we want it 😎 |
Beta Was this translation helpful? Give feedback.
-
+1 We would need this feature back. |
Beta Was this translation helpful? Give feedback.
-
+1 like many others I've spent hours trying to figure out what's going on, please bring this back! |
Beta Was this translation helpful? Give feedback.
-
I just don't get it. Without trying to sound rude but Airbyte as a whole is completely useless for our purposes without normalization. Why take the T out of the ETL only to ask if you should put it back in? This downgrade has completely messed up our pipelines and cost us multiple days to fix. Please, please put it back in asap. |
Beta Was this translation helpful? Give feedback.
-
An update from the Airbyte engineering team:
|
Beta Was this translation helpful? Give feedback.
-
Hello, and thank you for your work on Airbyte, especially for making it open-source and available for enthusiasts and low-budget builders. What is the current status of the Clickhouse destination connector? From the documentation page it seems that all the features are currently supported: four "yes" features and zero "No". However, I cannot make it support normalization and transformation of data. Please note that Clickhouse is also an open-source product and thus is cherished by enthusiasts and makers, the crowd you are addressing by making Airbyte OSS. |
Beta Was this translation helpful? Give feedback.
-
Yes Please, bring typing and deduping for clickhouse. |
Beta Was this translation helpful? Give feedback.
-
We removed (PR )"normalization" from Destination Clickhouse. Normalization was a process which would extract data from the raw json format and cast the data into tables with the appropriate types. For other destinations, we implemented Typing and Deduping to replace normalization and we will bring typing and deduplication to Clickhouse if there is sufficient interest
The purpose of this discussion is to gauge interest in bringing this functionality back to Clickhouse.
Without typing and deduping, you would need to integrate another tool, such as dbt to accomplish the same result. While integrating another tool gives you greater customization over the data format, it comes with additional operational overhead.
Should Airbyte bring Typing and Deduping to Clickhouse?
Beta Was this translation helpful? Give feedback.
All reactions