Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the volume of logs produced by the airflow scheduler #200

Open
MattTriano opened this issue Sep 22, 2024 · 0 comments
Open

Reduce the volume of logs produced by the airflow scheduler #200

MattTriano opened this issue Sep 22, 2024 · 0 comments

Comments

@MattTriano
Copy link
Owner

The airflow scheduler currently outputs over 200Mb of logs every day, and the majority of them are for DAGs that haven't run. It would be good to reduce this.

I could manage this by automating deletion of logs, but I'd rather explore configuring airflow to output fewer logs first.

To achieve this, I'll want to add the airflow.cfg file to version control, which will involve mounting the file to the host and confirming that no sensitive information would be leaked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant