Alerts[Teams] - test_results_sample doesn't output anything

[christopherekfeldt]

Describe the bug
Since the launch of Elementary alerting for Teams we have been using it, I've added different alert_fields and config for different tests. Now after the upgrade to dbt 1.8 and latest elementary version, the alerts test_result_sample stopped to reveal. We use it both to visualize and show model cost spikes and our longest running models this way. We have limit the output to 5 rows since that is the maximum of test_results_sample. But now it doesn't appear at all.

Expected behavior
I expect the test_results_sample to output the 5 samples in the result if an alert appears.

Screenshots

Environment (please complete the following information):

• Elementary CLI (edr) version: 0.16.1
• Elementary dbt package version: 0.16.0
• dbt core: 1.8.3
• dbt-bigquery: 1.8.2
• BigQuery
• prod

Additional context
My first initial thought was that this could be connected to that dbt changed from tests -> data_tests in dbt 1.8, which may cause the issue of the config not attaching. But I'm not sure..

Would you be willing to contribute a fix for this issue?
No, thanks!

[timhiebenthal]

same issues,
elementary-dbt: 0.16.0
elementary-python: 0.16.1
dbt core: 1.8.6
dbt-bigquery: 1.8.0
BigQuery as database
Slack as messenger

the elementary_test_results model has an empty field result_rows in case that helps.
so I bet the issue is rather on the dbt-package side of things

[ofek1weiss]

Hey @christopherekfeldt , @timhiebenthal, I assume you both started experiencing this issue after upgrading dbt to version 1.8, is that right?
If so, you should see the following warning message when running any dbt command:

IMPORTANT - Starting from dbt 1.8, users must explicitly allow packages to override materializations. 
Elementary requires this ability to support collection of samples and failed row count for dbt tests.
Please add the following flag to dbt_project.yml to allow it:

flags:
  require_explicit_package_overrides_for_builtin_materializations: false

Notes - 
* This is a temporary measure that will result in a deprecation warning, please ignore it for now. Elementary is working with the dbt-core team on a more permanent solution.
* This message can be muted by setting the 'mute_ensure_materialization_override' var to true.

make sure to follow these instructions, it should resolve the issue.

In case my assumption is incorrect and you do not use version 1.8, or the solution does not work, please let me know

[timhiebenthal]

thanks @ofek1weiss
but to my understanding we did exactly that, right?

and yes, we use dbt 1.8

[ofek1weiss]

Ok, so lets try to debug this, i need to know a few things:

1. did samples in the alert work for you previously, or did it stop working due to an upgrade?
2. if it was caused by an upgrade, it would really help to know what you upgraded, from which version to which version (eg: dbt from 1.7.2 to 1.8.6, elementary dbt package from 0.15.2 to 0.16
3. i see that you have alert_fields configured, how exactly are they configured (it is cut off in the screenshot)

[timhiebenthal]

We have a new implementation of Elementary so pointing it to a certain version upgrade is tough regarding question 1 and 2.

we implemented the newest versions right away (dbt 1.8, elementary 0.16.1) and then I played a bit around with the elementary-dbt package in the hope that a downgrade would solve it

full snippet of the definition

data_tests:
  elmo:
    +meta:
      subscribers: ["@tim.hiebenthal"]
      alert_fields: [ "description", "owner", "subscribers", "result_message", "test_results_sample", "test_query" ]
      alert_suppression_interval: 24
      channel: elmo-dev

[timhiebenthal]

Update:
I achieved the expected result with the following versions:

Elementary version 0.15.1.

dbt

Core:
  - installed: 1.8.6
  - latest:    1.8.6 - Up to date!

Plugins:
  - bigquery: 1.8.0 - Update available!

dbt deps

15:09:57  Running with dbt=1.8.6
15:09:57  Updating lock file in file path: /app/dbt/package-lock.yml
15:09:57  Installing elementary-data/elementary
15:09:58  Installed from version 0.15.2
15:09:58  Updated version available: 0.16.1
15:09:58  Installing dbt-labs/dbt_utils
15:09:58  Installed from version 1.3.0
15:09:58  Up to date!

will now upgrade and keep you in the loop

[timhiebenthal]

Seems to also work with dbt-package 0.16.1

How about you @christopherekfeldt ?

[ica-christopher-ekfeldt]

I'm not following, what resolved the issue for you @timhiebenthal , was it to downgrade and upgrade again?

[timhiebenthal]

try the dbt package 0.16.1

[christopherekfeldt]

No I still have the same issue..