Allow dbt show and dbt compile to output JSON without extra logs

[dbeatty10]

resolves #9840

Problem

As described in #9840:

I would like the output of dbt show to be valid JSON, containing just the data and no logs.

dbt list uses the --quiet flag to isolate the desired output from the log output, allowing the results to be piped or redirected.

But currently, dbt show and dbt compile do not work similarly. Rather, the --quiet flag suppresses all output.

Latest Solution

The latest solution uses dbt-labs/dbt-common#216 so that CompiledNode and ShowNode keep their same event names in the JSON logs, but also allow it to be emitted without timestamps even when --quiet.

When --quiet, also skips any extraneous output like:

• Previewing node 'my_model':
• Previewing inline node:
• Compiled node 'my_model' is:
• Compiled inline node is:

Initial Solution

The initial solution adopted the same exact approach as dbt list here, and basically copy-pasted from there.

Similar to how #10131 stopped using ListCmdOut in favor of PrintEvent, this PR stopped using CompiledNode and ShowNode in favor of PrintEvent.

👉 So any consumers relying on CompiledNode or ShowNode existing with JSON logs wouldn't see those anymore but would see only PrintEvent instead. So I switched to the latest solution, to avoid any unintentional breakage for anyone creating and parsing JSON logs for dbt show / dbt compile.

Before vs. after

Scenarios:

• show vs. compile
• --select vs. --inline
• --quiet vs. --no-quiet
• --output text vs. json
• --log-format text vs. json

Example for initial solution

dbt show --select my_model --log-format json --quiet

logs/dbt.log before:

{"data": {"is_inline": false, "node_name": "my_model", "output_format": "text", "preview": "| event_id |   date_day |\n| -------- | ---------- |\n|        1 | 2002-02-02 |\n", "unique_id": "model.my_project.my_model"}, "info": {"category": "", "code": "Q041", "extra": {}, "invocation_id": "b295d589-99ba-450f-b261-0d60c5bcc195", "level": "info", "msg": "Previewing node 'my_model':\n| event_id |   date_day |\n| -------- | ---------- |\n|        1 | 2002-02-02 |\n", "name": "ShowNode", "pid": 96009, "thread": "MainThread", "ts": "2024-11-04T19:40:52.452943Z"}}

logs/dbt.log after:

{"data": {"msg": "| event_id |   date_day |\n| -------- | ---------- |\n|        1 | 2002-02-02 |\n"}, "info": {"category": "", "code": "Z052", "extra": {}, "invocation_id": "d0857f0c-b58b-4374-893c-c454aba291b5", "level": "info", "msg": "| event_id |   date_day |\n| -------- | ---------- |\n|        1 | 2002-02-02 |\n", "name": "PrintEvent", "pid": 96415, "thread": "MainThread", "ts": "2024-11-04T19:41:43.897976Z"}}

Checklist

• I have read the contributing guide and understand what's expected of me
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests
• This PR has already received feedback and approval from Product or DX
• This PR includes type annotations for new and modified functions

[ChenyuLInx]

We can use the new PrintEvent being added to dbt-common.

[dbeatty10]

Awesome! I'll check out dbt-labs/dbt-common#130 and give PrintEvent a try.

[dbeatty10]

Note to self: see #10131 for an example how to use PrintEvent.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.09%. Comparing base (bdf28d7) to head (df7aaee).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9958      +/-   ##
==========================================
- Coverage   89.14%   89.09%   -0.06%     
==========================================
  Files         183      183              
  Lines       23553    23559       +6     
==========================================
- Hits        20996    20989       -7     
- Misses       2557     2570      +13     

Flag	Coverage Δ
integration	86.39% <100.00%> (-0.13%)	⬇️
unit	62.75% <62.50%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Unit Tests	62.75% <62.50%> (-0.01%)	⬇️
Integration Tests	86.39% <100.00%> (-0.13%)	⬇️

[dbeatty10]

@b-per FYI this and dbt-labs/dbt-common#216 are meant to solve your request in #9840