Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADAP-894: Support test results as views #889

Merged
merged 20 commits into from
Oct 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
7b35a43
implement tests for persist test results functionality
mikealfare Sep 21, 2023
f931543
updated project config for tests
mikealfare Sep 22, 2023
62dda03
revert config changes
mikealfare Sep 22, 2023
1ea46bc
skip Spark, which needs different delete sql
mikealfare Sep 22, 2023
b9f9209
add Spark config
mikealfare Sep 22, 2023
d41cb38
add a skip marker indicating the issue that needs to be resolved
mikealfare Sep 22, 2023
b7bf980
pull the test case in from core to show that everything passes except…
mikealfare Sep 22, 2023
8802cb8
update changelog and test names to reflect renamed parameter
mikealfare Sep 29, 2023
5c0fff3
Merge branch 'main' into feature/materialized-tests/adap-894
mikealfare Sep 29, 2023
6806364
correct fixture names in test
mikealfare Sep 29, 2023
724b3d3
updated the name of the overridden test so that it actually overrides…
mikealfare Sep 29, 2023
72239b3
update the row count method so support spark requirements
mikealfare Sep 29, 2023
8dd19be
added --store-failures parameter to the dbt invocation
mikealfare Sep 29, 2023
9d619d3
Merge branch 'main' into feature/materialized-tests/adap-894
mikealfare Oct 2, 2023
c16095e
implement store-failures-as tests
mikealfare Oct 3, 2023
8c70314
Merge remote-tracking branch 'origin/feature/materialized-tests/adap-…
mikealfare Oct 3, 2023
88be78f
skip spark-session from store-failures-as tests
mikealfare Oct 3, 2023
7496c9e
revert dev requirements to point back to main on dbt-core
mikealfare Oct 10, 2023
156d876
revert dev requirements to point back to main on dbt-core
mikealfare Oct 10, 2023
824db25
update signature of get_catalog to match dbt-core and appease mypy
mikealfare Oct 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/Features-20230921-180958.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Features
body: Support storing test failures as views
time: 2023-09-21T18:09:58.174136-04:00
custom:
Author: mikealfare
Issue: "6914"
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ dev: ## Installs adapter in develop mode along with development dependencies
dev-uninstall: ## Uninstalls all packages while maintaining the virtual environment
## Useful when updating versions, or if you accidentally installed into the system interpreter
pip freeze | grep -v "^-e" | cut -d "@" -f1 | xargs pip uninstall -y
pip uninstall -y dbt-spark

.PHONY: mypy
mypy: ## Runs mypy against staged changes for static type checking.
Expand Down
4 changes: 3 additions & 1 deletion dbt/adapters/spark/impl.py
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,9 @@ def _get_columns_for_catalog(self, relation: BaseRelation) -> Iterable[Dict[str,
as_dict["table_database"] = None
yield as_dict

def get_catalog(self, manifest: Manifest) -> Tuple[agate.Table, List[Exception]]:
def get_catalog(
self, manifest: Manifest, selected_nodes: Optional[Set] = None
) -> Tuple[agate.Table, List[Exception]]:
schema_map = self._get_catalog_schemas(manifest)
if len(schema_map) > 1:
raise dbt.exceptions.CompilationError(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import pytest

from dbt.tests.adapter.store_test_failures_tests import basic
from dbt.tests.adapter.store_test_failures_tests.test_store_test_failures import (
StoreTestFailuresBase,
TEST_AUDIT_SCHEMA_SUFFIX,
Expand Down Expand Up @@ -42,3 +43,33 @@ def project_config_update(self):
def test_store_and_assert_failure_with_delta(self, project):
self.run_tests_store_one_failure(project)
self.run_tests_store_failures_and_assert(project)


@pytest.mark.skip_profile("spark_session")
class TestStoreTestFailuresAsInteractions(basic.StoreTestFailuresAsInteractions):
pass


@pytest.mark.skip_profile("spark_session")
class TestStoreTestFailuresAsProjectLevelOff(basic.StoreTestFailuresAsProjectLevelOff):
pass


@pytest.mark.skip_profile("spark_session")
class TestStoreTestFailuresAsProjectLevelView(basic.StoreTestFailuresAsProjectLevelView):
pass


@pytest.mark.skip_profile("spark_session")
class TestStoreTestFailuresAsGeneric(basic.StoreTestFailuresAsGeneric):
pass


@pytest.mark.skip_profile("spark_session")
class TestStoreTestFailuresAsProjectLevelEphemeral(basic.StoreTestFailuresAsProjectLevelEphemeral):
pass


@pytest.mark.skip_profile("spark_session")
class TestStoreTestFailuresAsExceptions(basic.StoreTestFailuresAsExceptions):
pass