Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORE-364] Add group info to RunResultError, RunResultFailure, RunResultWarning log lines #10535

Merged
merged 10 commits into from
Aug 19, 2024

Conversation

aranke
Copy link
Member

@aranke aranke commented Aug 6, 2024

Resolves https://dbtlabs.atlassian.net/browse/CORE-364

Problem

We need to add group info to RunResultError, RunResultFailure, RunResultWarning log lines for alerting.

Solution

  1. Add a new Group proto message.
  2. Plumb metadata from the manifest through to RunResultError, RunResultFailure, RunResultWarning.
  3. Add tests for the same.

Checklist

  • I have read the contributing guide and understand what's expected of me.
  • I have run this code in development, and it appears to resolve the stated issue.
  • This PR includes tests, or tests are not required or relevant for this PR.
  • This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
  • This PR includes type annotations for new and modified functions.

@cla-bot cla-bot bot added the cla:yes label Aug 6, 2024
Copy link
Contributor

github-actions bot commented Aug 6, 2024

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

Copy link

codecov bot commented Aug 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.84%. Comparing base (9ca1bc5) to head (94fb7f7).
Report is 14 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10535      +/-   ##
==========================================
- Coverage   88.93%   88.84%   -0.10%     
==========================================
  Files         180      180              
  Lines       22735    22755      +20     
==========================================
- Hits        20220    20217       -3     
- Misses       2515     2538      +23     
Flag Coverage Δ
integration 86.08% <100.00%> (-0.17%) ⬇️
unit 62.34% <34.78%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Unit Tests 62.34% <34.78%> (-0.04%) ⬇️
Integration Tests 86.08% <100.00%> (-0.17%) ⬇️

@aranke aranke changed the title Add group info to RunResultError log line Add group info to RunResultError, RunResultFailure, RunResultError log line Aug 7, 2024
@aranke aranke changed the title Add group info to RunResultError, RunResultFailure, RunResultError log line Add group info to RunResultError, RunResultFailure, RunResultError log lines Aug 7, 2024
@aranke aranke marked this pull request as ready for review August 7, 2024 20:47
@aranke aranke requested a review from a team as a code owner August 7, 2024 20:47
@aranke aranke changed the title Add group info to RunResultError, RunResultFailure, RunResultError log lines [CORE-364] Add group info to RunResultError, RunResultFailure, RunResultError log lines Aug 7, 2024
@aranke aranke changed the title [CORE-364] Add group info to RunResultError, RunResultFailure, RunResultError log lines [CORE-364] Add group info to RunResultError, RunResultFailure, RunResultWarning log lines Aug 7, 2024
@aranke aranke requested a review from gshank August 7, 2024 20:56
@@ -144,9 +157,11 @@ def print_run_end_messages(results, keyboard_interrupt: bool = False) -> None:
)

for error in errors:
print_run_result_error(error, is_warning=False)
group = groups.get(error.node.unique_id) if groups and hasattr(error, "node") else None

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is neat, iiuc this is printing the summary of all model errors with the group info for those models

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these summary logs have a different code (like how individual model failures have "Z002")?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes:

  1. RunResultWarning – Z021
  2. RunResultFailure – Z022
  3. RunResultError – Z024

Copy link
Contributor

@gshank gshank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good, but I'd simplify the protobuf Group a bit.

core/dbt/events/core_types.proto Show resolved Hide resolved
@aranke aranke requested a review from gshank August 19, 2024 17:24
@aranke aranke merged commit 3695698 into main Aug 19, 2024
64 of 66 checks passed
@aranke aranke deleted the fix_core_364 branch August 19, 2024 18:26
Copy link
Contributor

The backport to 1.7.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.7.latest 1.7.latest
# Navigate to the new working tree
cd .worktrees/backport-1.7.latest
# Create a new branch
git switch --create backport-10535-to-1.7.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 3695698e220843a1c41d9184c6fb4cbbf52eb1fc
# Push it to GitHub
git push --set-upstream origin backport-10535-to-1.7.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.7.latest

Then, create a pull request where the base branch is 1.7.latest and the compare/head branch is backport-10535-to-1.7.latest.

Copy link
Contributor

The backport to 1.8.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.8.latest 1.8.latest
# Navigate to the new working tree
cd .worktrees/backport-1.8.latest
# Create a new branch
git switch --create backport-10535-to-1.8.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 3695698e220843a1c41d9184c6fb4cbbf52eb1fc
# Push it to GitHub
git push --set-upstream origin backport-10535-to-1.8.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.8.latest

Then, create a pull request where the base branch is 1.8.latest and the compare/head branch is backport-10535-to-1.8.latest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants