Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of delete+insert incremental strategy #151

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Commits on Apr 10, 2024

  1. Fix incremental delete+insert SQL

    resolves dbt-labs#150
    
    Problem
    The delete query for the 'delete+insert' incremental_strategy with 2+ unique_key columns is VERY inefficient. In many cases, it will hang and never return for deleting small amounts of data (<100K rows).
    
    Solution
    Improve the query by switching to a much more efficient delete strategy:
    
    ```
    delete from table1
    where (col1, col2) in (
        select distinct col1, col2 from table1_tmp
    )
    ```
    ataft committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    773e8d3 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Configuration menu
    Copy the full SHA
    b8b8a7f View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. Configuration menu
    Copy the full SHA
    66facbc View commit details
    Browse the repository at this point in the history