Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] Quarantine transformation #35

Open
mikita-sakalouski opened this issue May 30, 2024 · 0 comments
Open

[FEATURE] Quarantine transformation #35

mikita-sakalouski opened this issue May 30, 2024 · 0 comments
Assignees
@mikita-sakalouski
Labels
enhancement New feature or request

Comments

@mikita-sakalouski
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Quarantine transformation can be used to split your incoming data based on custom logic and write "bad" part to different table, while propagating good part to the next koheesio Step.

Describe the solution you'd like

Should be pretty simple, but customizable pyspark transformation with splitting logic.

Describe alternatives you've considered

We should Delta Live Tables feature for the same issue, which is providing more functionality (metrics, stats and etc.)
@mikita-sakalouski mikita-sakalouski added the enhancement New feature or request label May 30, 2024
@mikita-sakalouski mikita-sakalouski self-assigned this May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mikita-sakalouski mikita-sakalouski

Labels
enhancement New feature or request
Projects
Koheesio
Status: No status
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mikita-sakalouski