Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor VCE RARE asset checks #3926

Open
3 of 4 tasks
Tracked by #3914
jdangerx opened this issue Oct 23, 2024 · 1 comment
Open
3 of 4 tasks
Tracked by #3914

Refactor VCE RARE asset checks #3926

jdangerx opened this issue Oct 23, 2024 · 1 comment
Labels
dagster Issues related to our use of the Dagster orchestrator data-validation Issues related to checking whether data meets our quality expectations. testing Writing tests, creating test data, automating testing, etc. vcerare Pertaining to Vibrant Clean Energy's Resource Adequacy Renewable Energy Power datasets

Comments

@jdangerx
Copy link
Member

jdangerx commented Oct 23, 2024

Tasks

Preview Give feedback
@jdangerx jdangerx moved this from New to Backlog in Catalyst Megaproject Oct 23, 2024
@zaneselvans zaneselvans added dagster Issues related to our use of the Dagster orchestrator data-validation Issues related to checking whether data meets our quality expectations. vcerare Pertaining to Vibrant Clean Energy's Resource Adequacy Renewable Energy Power datasets testing Writing tests, creating test data, automating testing, etc. labels Oct 23, 2024
@zaneselvans zaneselvans changed the title Refactor VCE asset check to accumulate an error report and return after all checks Refactor VCE RARE asset checks Oct 23, 2024
@zaneselvans
Copy link
Member

Most of these issues were addressed in #3959 but the row-count check still only passes if all data is processed, meaning the fast_etl will always fail if asset checks are enabled, which they should be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dagster Issues related to our use of the Dagster orchestrator data-validation Issues related to checking whether data meets our quality expectations. testing Writing tests, creating test data, automating testing, etc. vcerare Pertaining to Vibrant Clean Energy's Resource Adequacy Renewable Energy Power datasets
Projects
Status: Backlog
Development

No branches or pull requests

2 participants