Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Display supported incremental strategies in a tabular format for legibility #4634

Closed
1 task done
dbeatty10 opened this issue Dec 13, 2023 · 0 comments · Fixed by #4649
Closed
1 task done

Display supported incremental strategies in a tabular format for legibility #4634

dbeatty10 opened this issue Dec 13, 2023 · 0 comments · Fixed by #4649
Labels
content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear

Comments

@dbeatty10
Copy link
Contributor

dbeatty10 commented Dec 13, 2023 •
edited
Loading

Contributions

  • I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates

https://docs.getdbt.com/docs/build/incremental-models#supported-incremental-strategies-by-adapter

What part(s) of the page would you like to see updated?

It could be more clear which strategy is the default per adapter and which strategies are supporter per adapter.

The following idea was originally included as part of #2895 on 2023-02-20.

Goal

Be able to easily answer the questions:

  • which adapters support the delete+insert strategy?
  • what are the names of the standard incremental strategies that are supported in at least one adapter?

Idea 1

The following is probably not even close to accurate, but is an example of the data in a tabular form instead:

Adapter Default strategy append merge insert_overwrite delete+insert
dbt-postgres append ✔️
dbt-redshift append ✔️
dbt-bigquery merge ✔️ ✔️
dbt-spark append ✔️ ✔️ ✔️
dbt-snowflake merge ✔️ ✔️ ✔️

Idea 2

And if that is too busy, then here's a two-table variant instead:

Supported strategies

Adapter append merge insert_overwrite delete+insert
dbt-postgres ✔️
dbt-redshift ✔️
dbt-bigquery ✔️ ✔️
dbt-spark ✔️ ✔️ ✔️
dbt-snowflake ✔️ ✔️ ✔️

Default strategy

The default strategy is used when incremental_strategy is not configured.

Adapter Default strategy
dbt-postgres append
dbt-redshift append
dbt-bigquery merge
dbt-spark append
dbt-snowflake merge

Additional information

I seem to recall that @mirnawong1 may have already taken a look at refactoring this table and decided against this proposal in favor of a different format. If that is the case, then this issue can be closed as "not planned".
@dbeatty10 dbeatty10 added content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear labels Dec 13, 2023
This was referenced Dec 13, 2023
Open
Merged
dbeatty10 added a commit that referenced this issue Dec 14, 2023
@dbeatty10 @matthewshaver
Display supported incremental strategies in a tabular format (#4649)
3ad19b4 
[Preview](https://docs-getdbt-com-git-dbeatty-tabular-incremental-326e95-dbt-labs.vercel.app/docs/build/incremental-models#supported-incremental-strategies-by-adapter)

resolves #4634

## What are you changing in this pull request and why?

The main aim is to be able to easily answer each of these questions:
- which adapters support the `delete+insert` strategy?
- what are the names of the standard incremental strategies that are
supported in at least one adapter?
- what are the names of the standard incremental strategies that are
supported most frequently?
- what are the names of the standard incremental strategies that are
supported least frequently?

A tabular representation with one column per strategy is the easiest way
to answer all of the above.

## Design decisions to be made

1. Style of the icons
1. Number of tables

### Style of the icons

Any of these could work:
✅ - Green check mark
✔️ - Heavy check mark
🟢 - Green circle

We probably wouldn't choose any of these:
☑️ - Ballot box with check
🆗 - OK button
👍 - Thumbs up
etc.

### Number of tables

We have several options:

1. Keep the status quo
1. Split into two tables: one for the available strategies, and another
for the defaults
1. Keep a single table, keeping the "default strategy" column, but
replacing the "additional supported strategies" with one column per
strategy
1. Defer the default strategy information to each adapter-specific
config page instead (see
[dbt-bigquery](https://docs.getdbt.com/reference/resource-configs/bigquery-configs#merge-behavior-incremental-models)
or
[dbt-redshift](https://docs.getdbt.com/reference/resource-configs/redshift-configs#incremental-materialization-strategies)
for examples)

### Trade-offs

I strongly prefer not staying with the status quo, or I wouldn't have
opened this issue 🤣

Splitting into two separate tables would solve the key problem, but at
the cost of taking up more vertical space on the page.

Keeping a single table but refactoring it would work for me.

Completely moving the default strategy information into the
adapter-specific pages might be the best of all worlds. It would solve
the problem at hand while preserving documentation about the default
strategy that is relevant for a particular adapter. dbt-postgres and
dbt-redshift in particular have caveats related to the default -- it
will change [depending if a `unique_key` is defined
](#4651 not.

## 🎩 


[Preview](https://docs-getdbt-com-git-dbeatty-tabular-incremental-326e95-dbt-labs.vercel.app/docs/build/incremental-models#supported-incremental-strategies-by-adapter)

### v3 (final)

<img width="550" alt="image"
src="https://github.com/dbt-labs/docs.getdbt.com/assets/44704949/66689bca-c733-487e-b8c3-0251da01948f">

<img width="550" alt="image"
src="https://github.com/dbt-labs/docs.getdbt.com/assets/44704949/2c87ce14-746c-436e-a7d9-bd4c15ffc163">

### v2

<img width="500" alt="image"
src="https://github.com/dbt-labs/docs.getdbt.com/assets/44704949/2c4bb14f-4c60-4262-aec1-d10351333bb7">

### v1

**Note:** I've since removed the 2nd table that has the defaults, but
here's what it looked like with them:

<img width="447" alt="image"
src="https://github.com/dbt-labs/docs.getdbt.com/assets/44704949/6fda6a40-2acb-44a4-b4eb-65d379020ece">

<img width="442" alt="image"
src="https://github.com/dbt-labs/docs.getdbt.com/assets/44704949/a1e33dbc-f3a4-4dd8-997b-36e77269085d">

## Checklist
- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.

---------

Co-authored-by: Matt Shaver <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Display supported incremental strategies in a tabular format dbt-labs/docs.getdbt.com
1 participant
@dbeatty10