[Feature] Skip creating a log directory when file logging is disabled

[ericfreese]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

I ran into this while deploying DBT core in an environment where dbt's current working directory is not writeable. I had set DBT_LOG_PATH to a separate writeable directory, but then realized recently that we actually want to entirely disable DBT's file logging because we store the stdout/stderr output separately.

After some digging through the source (because I couldn't find documentation of how to disable the file logging), I found this line:

dbt-core/core/dbt/events/logging.py

Line 95 in dd77210

if flags.LOG_LEVEL_FILE != "none":

A bit more digging and I found that I could set a DBT_LOG_LEVEL_FILE=none environment variable to effectively disable the file logging. I figured that after disabling file logging, I would no longer need to set DBT_LOG_PATH so I removed that environment variable.

Upon testing, I was surprised to see that dbt debug now provided no output and only exited with error code 2. I'd run into this issue before when it couldn't open the log file for writing, so I added DBT_LOG_PATH=/tmp back to the environment, and it ran successfully.

Expected Behavior

It seems to me that if file logging is disabled, dbt should not try to make the logging directory.

Steps To Reproduce

1. In an environment where DBT cannot make a directory in its current working directory (you may also be able to make the mkdir fail by creating logs as a regular file rather than a directory e.g. touch logs)
2. With env vars:

   DBT_TARGET_PATH=/tmp # or some writeable directory
   DBT_PACKAGES_DIR=/tmp # or some writeable directory
   DBT_LOG_LEVEL_FILE=none

3. Run dbt debug
4. See no output and exit code 2 (via echo $?)

Instead, I would expect dbt debug to succeed and print its output. Nothing should be logged to the file system.

Relevant log output

No response

Environment

- OS: Amazon Linux 2023
- Python: 3.11.6
- dbt: 1.8.7

Which database adapter are you using with dbt?

postgres

Additional Context

I dug back through the git history a bit and I think the make_log_dir_if_missing call should have been moved under the if statement when the none option was added in commit a2213ab

[dbeatty10]

Thanks for reporting this and digging into the git history @ericfreese 🤩

At the very least, there should be documentation that describes the LOG_LEVEL_FILE config, so I've opened this docs issue: dbt-labs/docs.getdbt.com#6404

I don't think we ever intentionally designed or documented dbt to not create a logging directory in this scenario, so I'm going to switch this to a feature request for further consideration for your PR: #10949