Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement storage- and relationship-aware cleanup #973

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

Swatinem
Copy link
Contributor

This implements a generic mechanism to resolve the model relationship graph based on Django metadata. That graph is then sorted in topological order, and all models are being deleted. Special manual deletion code is written for all the models that have some files in storage, in order to make sure that files are also being cleaned up properly.


This is work towards codecov/engineering-team#1127

@Swatinem Swatinem self-assigned this Dec 19, 2024
@codecov-notifications
Copy link

codecov-notifications bot commented Dec 19, 2024

Codecov Report

Attention: Patch coverage is 30.00000% with 98 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
services/cleanup/models.py 0.00% 68 Missing ⚠️
services/cleanup/cleanup.py 0.00% 17 Missing ⚠️
services/cleanup/utils.py 0.00% 8 Missing ⚠️
services/cleanup/relations.py 87.50% 5 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link

✅ All tests successful. No failed tests were found.

📣 Thoughts on this report? Let Codecov know! | Powered by Codecov

Copy link

codecov bot commented Dec 19, 2024

Codecov Report

Attention: Patch coverage is 30.00000% with 98 lines in your changes missing coverage. Please review.

Project coverage is 97.70%. Comparing base (dc51116) to head (389d49e).
Report is 2 commits behind head on main.

Current head 389d49e differs from pull request most recent head 73f9f21

Please upload reports for the commit 73f9f21 to get more accurate results.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
services/cleanup/models.py 0.00% 68 Missing ⚠️
services/cleanup/cleanup.py 0.00% 17 Missing ⚠️
services/cleanup/utils.py 0.00% 8 Missing ⚠️
services/cleanup/relations.py 87.50% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #973      +/-   ##
==========================================
- Coverage   97.96%   97.70%   -0.27%     
==========================================
  Files         446      451       +5     
  Lines       36045    36185     +140     
==========================================
+ Hits        35311    35353      +42     
- Misses        734      832      +98     
Flag Coverage Δ
integration 41.88% <5.00%> (-0.15%) ⬇️
unit 90.40% <30.00%> (-0.24%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

⚠️ Impact Analysis from Codecov is deprecated and will be sunset on Jan 31 2025. See more

This implements a generic mechanism to resolve the model relationship graph based on Django metadata.
That graph is then sorted in topological order, and all models are being deleted.
Special manual deletion code is written for all the models that have some files in storage, in order to make sure that files are also being cleaned up properly.
@Swatinem Swatinem force-pushed the swatinem/bulk-cleanups branch from 2dc9ed9 to 389d49e Compare January 7, 2025 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant