Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplest way of marking record according to which filter it failed on? #1963

Open
meredigr opened this issue May 15, 2024 · 0 comments
Open

Comments

@meredigr
Copy link

Good day - I am evaluating DataCleaner as a potential means of identifying data quality issues in source data, prior to a data migration. I have added various filters, each of which is checking a certain aspect of the input data. For each record that 'fails' a filter (and, therefore, violates a Data Quality rule), I am writing the record to a staging table. There are several such filters in my job, each of which tests a different rule, but each violated record gets written to the same staging table. Is there any relatively easy way to add a field to the output from each filter that defines which filter (i.e. which rule) the record failed on?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant