Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CT-2819] default__alter_relation_add_remove_columns macro does not use quoting with case sensitive Snowflake relation #1277

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

[CT-2819] default__alter_relation_add_remove_columns macro does not use quoting with case sensitive Snowflake relation #1277

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
92431f7
create bigquery version of test as it has different default behavior …
McKnight-42 Jul 16, 2024
6ef9eda
update dev-requirements
McKnight-42 Jul 17, 2024
435b2af
add changelog
McKnight-42 Jul 17, 2024
2ecd49e
typo
McKnight-42 Jul 17, 2024
99347fd
use universal version of test, remove bigquery version
McKnight-42 Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion dev-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@
# TODO: how to automate switching from develop to version branches?
git+https://github.com/dbt-labs/dbt-core.git#egg=dbt-core&subdirectory=core
git+https://github.com/dbt-labs/dbt-common.git
git+https://github.com/dbt-labs/dbt-adapters.git
git+https://github.com/dbt-labs/dbt-adapters.git@mcknight/ct-2819
# -e /Users/matthewmcknight/git/dbt-adapters
git+https://github.com/dbt-labs/dbt-adapters.git#subdirectory=dbt-tests-adapter
McKnight-42 marked this conversation as resolved.
Show resolved Hide resolved
# -e /Users/matthewmcknight/git/dbt-adapters/dbt-tests-adapter
# if version 1.x or greater -> pin to major version
# if version 0.x -> pin to minor
black>=24.3
Expand Down
58 changes: 57 additions & 1 deletion tests/functional/adapter/incremental/test_incremental_on_schema_change.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
BaseIncrementalOnSchemaChangeSetup,
BaseIncrementalOnSchemaChange,
)

from dbt.tests.util import run_dbt, run_dbt_and_capture
from dbt.tests.adapter.incremental.fixtures import (
_MODELS__A,
_MODELS__INCREMENTAL_SYNC_ALL_COLUMNS_TARGET,
Expand Down Expand Up @@ -251,3 +251,59 @@ def test_run_incremental_sync_all_columns_time_ingestion_partitioning(self, proj
compare_source = "incremental_sync_all_columns_time_ingestion_partitioning"
compare_target = "incremental_sync_all_columns_time_ingestion_partitioning_target"
self.run_twice_and_assert(select, compare_source, compare_target, project)


_MODELS__SRC_ARTISTS = """
{{
config(
materialized='table',
)
}}
{% if var("version", 0) == 0 %}
select {{ dbt.current_timestamp() }} as inserted_at, 'john' as name
{% else %}
-- add a non-zero version to the end of the command to get a different version:
-- --vars "{'version': 1}"
select {{ dbt.current_timestamp() }} as inserted_at, 'engineer' as Job,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Name purposely skipped here? I noticed you had it in the dbt-tests-adapter version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah for bigquery when ran it seemed to count the column as already existing and would fail. removing and running seems fine even if I changed the casing of name to Name it still failed. for column already existing happy to look into it further.

my thought was that since bigquery checks column casing natively it sees them as the same thing so nothing new to add and that seems to borq the query

{% endif %}
"""

_MODELS__DIM_ARTISTS = """
{{
config(
materialized='incremental',
on_schema_change='append_new_columns',
)
}}
select * from {{ ref("src_artists") }}
"""


class BaseBigQueryIncrementalCaseSenstivityOnSchemaChange:
@pytest.fixture(scope="class")
def models(self):
return {
"src_artists.sql": _MODELS__SRC_ARTISTS,
"dim_artists.sql": _MODELS__DIM_ARTISTS,
}

def test_incremental_case_sensitivity(self, project):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could be missing it, but this looks like the same test as in the base. Can we inherit from the new test case and just overwrite models? Also, this should just be a test class. We don't need the base class and the test class since this is in the concrete adapter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had pulled the dbt-adapter ones into its own base case and test class so that the location I had them originally as part of BaseIncrementalOnSchemaChangeSetup but didn't want to always pull it in since bigquery requires its own default version

select = "src_artists dim_artists"
run_dbt(["run", "--models", select, "--full-refresh"])
run_dbt(["show", "--inline", "select * from {{ ref('dim_artists') }}"])
res, logs = run_dbt_and_capture(
["show", "--inline", "select * from {{ ref('dim_artists') }}"]
)
assert "Job" not in logs
run_dbt(["run", "--vars", "{'version': 1}"])
run_dbt(["show", "--inline", "select * from {{ ref('dim_artists') }}"])
res, logs = run_dbt_and_capture(
["show", "--inline", "select * from {{ ref('dim_artists') }}"]
)
assert "Job" in logs


class TestBigQueryIncrementalCaseSenstivityOnSchemaChange(
BaseBigQueryIncrementalCaseSenstivityOnSchemaChange
):
pass
Loading