Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Temporary Tables not being cleaned for MERGE and on_schema_change != 'ignore' #1389

Open
2 tasks done
joshuao11 opened this issue Oct 29, 2024 · 0 comments
Open
2 tasks done
Labels
bug Something isn't working

Comments

@joshuao11
Copy link

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

If you had some example model:

{{
    config(
        materialized='incremental',
        schema='some_example_data_set',
        alias='some_example_model',
        incremental_strategy = 'merge',
        unique_key='example_unique_key',
        on_schema_change= 'sync_all_columns',
    )
}}

When we run the model we create a temporary table (as we have specified to have it check the schema some_example_model__dbt_tmp)

Which is then used for the MERGE:

{% set build_sql = get_merge_sql(target_relation, source_sql, unique_key, dest_columns, predicates) %}

However we do not clean the temporary table

Expected Behavior

The temporary table is removed as part of the run, like it currently is for insert_overwrite:

Would I be able to raise a PR against main to fix this? it's not the biggest issue in the world but can make user facing datasets unnecessarily messy for 12 hours until the tables self delete

Steps To Reproduce

  1. Run model
  2. Observe not deleted temp tables

Relevant log output

No response

Environment

- OS:
- Python:
- dbt-core:
- dbt-bigquery: 1.8

Additional Context

No response

@joshuao11 joshuao11 added bug Something isn't working triage labels Oct 29, 2024
@amychen1776 amychen1776 removed the triage label Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants