Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Changing Incremental Model Partitions on Bigquery #1350

Open
2 tasks done
jennlw opened this issue Sep 20, 2024 · 0 comments
Open
2 tasks done

[Bug] Changing Incremental Model Partitions on Bigquery #1350

jennlw opened this issue Sep 20, 2024 · 0 comments
Labels
bug Something isn't working help_wanted Extra attention is needed incremental

Comments

@jennlw
Copy link

jennlw commented Sep 20, 2024

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Recently ran into an issue with Bigquery and incremental model partitions.

When the partition of an existing incremental model needs to be changed, the dbt full refresh command runs a create or replace statement against the model, which fails with the error:

Incompatible table partitioning specification. Expected partitioning specification interval(type:,field:<field_1>), but input partitioning specification is interval(type:,field:<field_2>)

This indicates that despite the replace statement, Bigquery expects the same partitioning specification as the original table.
We found that running a drop statement followed by a create statement succeeds with no errors.

Ideally dbt would be able to generate these statements on full refresh.

Please let us know your thoughts!

Expected Behavior

Full refresh should be able to generate a drop/create on full refresh.

Steps To Reproduce

Have incremental model partitions
When the partition of this model needs to be changed, get error when dbt full refresh command runs create/replace against the model.

Relevant log output

This is reported by the customer. I will ask them to subscribe to this Bug so that they can add comments if additional questions/input are needed.

Environment

- OS:
- Python:
- dbt-core:
- dbt-bigquery:

Additional Context

No response

@jennlw jennlw added bug Something isn't working triage labels Sep 20, 2024
@amychen1776 amychen1776 added incremental help_wanted Extra attention is needed and removed triage labels Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help_wanted Extra attention is needed incremental
Projects
None yet
Development

No branches or pull requests

2 participants