Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LazyFormat to reduce logging overhead on performance #1419

Merged
merged 1 commit into from
Sep 24, 2024

Conversation

plypaul
Copy link
Contributor

@plypaul plypaul commented Sep 23, 2024

For larger semantic manifests / queries, the overhead of formatting objects for logging / writing log lines can be significant. This adds a LazyFormat class can be used with the existing logger so that log messages at the debug level don't get formatted nor written when the configured log level would exclude it. Using this, a quick way to migrate the code base would be a replacement like the following:

    Example:
        logger.debug(f"Result is: {expensive_function()}")
        ->
        logger.debug(LazyFormat(lambda: f"Result is: {expensive_function()}")

@cla-bot cla-bot bot added the cla:yes label Sep 23, 2024
@plypaul plypaul changed the title Add LazyFormat to reduce logging performance overheads Add LazyFormat to reduce logging overhead on performance Sep 23, 2024
@plypaul plypaul marked this pull request as ready for review September 23, 2024 22:05
Copy link
Collaborator

@marcodamore marcodamore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall lgtm!

Base automatically changed from p--short-term-perf--01 to main September 24, 2024 02:36
@plypaul plypaul merged commit 9674fe7 into main Sep 24, 2024
15 checks passed
@plypaul plypaul deleted the p--short-term-perf--02 branch September 24, 2024 02:42
plypaul added a commit that referenced this pull request Sep 24, 2024
As per #1419, this PR updates
log calls to lazily evaluate the arguments, and changes most of them to
the `DEBUG` level. This way, performance in production will be improved
if the log level is set to `INFO` or higher.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants