Skip to content
This repository has been archived by the owner on May 17, 2024. It is now read-only.

Evolve dbt data diff output #857

Merged
merged 47 commits into from
Feb 6, 2024
Merged

Evolve dbt data diff output #857

merged 47 commits into from
Feb 6, 2024

Conversation

sungchun12
Copy link
Contributor

@sungchun12 sungchun12 commented Jan 24, 2024

resolves #409

Cooking up progress

note: may be different from initial design in referenced issue as development progresses

data-diff --dbt
image

data-diff --dbt --cloud

Note: Columns added [+1]: {'NEW_COLUMN'} does NOT appear in this output because the API results payload doesn't include that column at all in the diff. This is intended behavior.
image

Adding a --show sample diff command is NOT worth implementing until the Datafold REST API is updated to return a limited a result set. If implemented today, it would slow down data-diff significantly and consume more memory given 1,000,000 results can be returned by the API with no pre-filtering possible.

Datafold API Needs

@sungchun12 sungchun12 self-assigned this Jan 24, 2024
@sungchun12 sungchun12 merged commit cac665b into master Feb 6, 2024
6 checks passed
@sungchun12 sungchun12 deleted the evolve-dbt-output branch February 6, 2024 19:03
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make all format options available when using --dbt
2 participants