Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add django management command to purge app logs #439

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

rohan-chaturvedi
Copy link
Member

🔍 Overview

Secret access logs, particularly reads, can get to many thousands of entries quite quickly, especially when frequently polling for secrets.

💡 Proposed Changes

Add a django management command to safely purge logs older than a specified date. This util will delete all log events older than the spcified date (30 days be default), except for create events. The command can be run in the backend container with:

python manage.py purge_app_logs <organisation_name>

This will purge logs for all Apps in the given organisation that are older than 30 days.

Optionally, the number of days and a specific app_id can be supplied as arguments.

Purge logs for all apps older than 60 days:

python manage.py purge_app_logs <organisation_name> --days 60

Purge all logs for a specific app only

python manage.py purge_app_logs <organisation_name> --app_id <app_id>

Purge logs older than 60 days for a specific app

python manage.py purge_app_logs <organisation_name> --days 60 --app_id <app_id>

🖼️ Screenshots or Demo

/app $ python manage.py purge_app_logs testorg
Deleting all logs older than 2024-12-26 08:14:07.049733+00:00 for organisation 'testorg'.
Deleted 35 logs for app GroundRelay
Deleted 0 logs for app FleetMonitor
Deleted 256 logs for app SatCommand
Deleted 0 logs for app CollisionAvoidance
Deleted 106 logs for app SkyMesh
Deleted 0 logs for app OrbitalControl
Deleted 18 logs for app LaunchTracker
Deleted 0 logs for app BridgeNet
Deleted 10 logs for app test
Deleted 0 logs for app test
Successfully cleared logs older than 30 days.
/app $ python manage.py purge_app_logs testorg --days 1
Deleting all logs older than 2025-01-24 08:14:28.523618+00:00 for organisation 'testorg'.
Deleted 3 logs for app GroundRelay
Deleted 0 logs for app FleetMonitor
Deleted 25 logs for app SatCommand
Deleted 0 logs for app CollisionAvoidance
Deleted 10 logs for app SkyMesh
Deleted 0 logs for app OrbitalControl
Deleted 0 logs for app LaunchTracker
Deleted 0 logs for app BridgeNet
Deleted 10 logs for app test
Deleted 0 logs for app test
Successfully cleared logs older than 1 days.
/app $ python manage.py purge_app_logs testorg --days 2 --app_id 61673ec9-d70a-4910-943d-29a0b992d8fa
Deleting logs older than 2025-01-23 08:24:58.530282+00:00 for organisation 'testorg'.
Deleted 0 logs for app 'SatCommand'.
Log deletion completed successfully

📝 Release Notes

Added a django management command to purge secret logs.

🎯 Reviewer Focus

  • Test the util with various combinations of arguments
  • Test the util with a large (>10,000) number of log entries

✨ How to Test the Changes Locally

Shell into your backend container with:

docker exec -it <container_id> /bin/sh

And run the command

💚 Did You...

  • Ensure linting passes (code style checks)?
    - [ ] Update dependencies and lockfiles (if required)
    - [ ] Regenerate graphql schema and types (if required)
  • Verify the app builds locally?
  • Manually test the changes on different browsers/devices?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant