Only try to create log directory if file logging is enabled

[ericfreese]

Resolves #10948

Problem

I ran into this while deploying DBT core in an environment where dbt's current working directory is not writeable. I had set DBT_LOG_PATH to a separate writeable directory, but then realized recently that we actually want to entirely disable DBT's file logging because we store the stdout/stderr output separately.

After some digging through the source (because I couldn't find documentation of how to disable the file logging), I found this line:

dbt-core/core/dbt/events/logging.py

Line 95 in dd77210

if flags.LOG_LEVEL_FILE != "none":

A bit more digging and I found that I could set a DBT_LOG_LEVEL_FILE=none environment variable to effectively disable the file logging. I figured that after disabling file logging, I would no longer need to set DBT_LOG_PATH so I removed that environment variable.

Upon testing, I was surprised to see that dbt debug now provided no output and only exited with error code 2. I'd run into this issue before when it couldn't open the log file for writing, so I added DBT_LOG_PATH=/tmp back to the environment, and it ran successfully.

It seems to me that if file logging is disabled, dbt should not try to make the logging directory.

Solution

As mentioned above, I can work around the problem for now by setting DBT_LOG_PATH=/tmp so that it will just create an empty directory in /tmp but that feels kind of silly.

I think it would be best to only try to make the logging directory if file logging is enabled, as the changes in my PR implement.

Checklist

• I have read the contributing guide and understand what's expected of me.
• I have run this code in development, and it appears to resolve the stated issue.
• This PR includes tests, or tests are not required or relevant for this PR.
• This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
• This PR includes type annotations for new and modified functions.

[cla-bot]

Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA.

In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR.

CLA has not been signed by users: @ericfreese

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.12%. Comparing base (dd77210) to head (968689f).
Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10949      +/-   ##
==========================================
- Coverage   89.19%   89.12%   -0.08%     
==========================================
  Files         183      183              
  Lines       23496    23496              
==========================================
- Hits        20958    20941      -17     
- Misses       2538     2555      +17     

Flag	Coverage Δ
integration	86.44% <100.00%> (-0.15%)	⬇️
unit	62.08% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Unit Tests	62.08% <100.00%> (ø)
Integration Tests	86.44% <100.00%> (-0.15%)	⬇️

[dbeatty10]

Thanks for opening this PR @ericfreese !

We haven't finished evaluating #10948 as a feature request yet (i.e. whether we want to commit to this behavior or not).

If we do choose to adopt it, then we'd need to add applicable testing to this PR.

[dbeatty10]

We're considering this a feature rather than a fix, so could you make this update?

Suggested change

	kind: Fixes
	kind: Features