Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory leak viewing all events in admin timeline #8369

Open
gbp opened this issue Aug 29, 2024 · 0 comments · May be fixed by #8458
Open

Memory leak viewing all events in admin timeline #8369

gbp opened this issue Aug 29, 2024 · 0 comments · May be fixed by #8458
Labels
bug Breaks expected functionality f:admin x:uk

Comments

@gbp
Copy link
Member

gbp commented Aug 29, 2024 

/admin/timeline?event_type=info_request_event&start_date=1970-01-01T00%3A00%3A00Z
CONSUME MEMORY: 5547600 KB

/admin/timeline?start_date=1970-01-01T00%3A00%3A00Z
CONSUME MEMORY: 5615192 KB

/admin/timeline?event_type=info_request_event&start_date=1970-01-01T00%3A00%3A00Z
CONSUME MEMORY: 5616836 KB

/admin/timeline?start_date=1970-01-01T00%3A00%3A00Z
CONSUME MEMORY: 5637360 KB
@gbp gbp added bug Breaks expected functionality f:admin labels Aug 29, 2024
@garethrees garethrees changed the title Memory leak viewing admin timeline Memory leak viewing all events in admin timeline Aug 29, 2024
garethrees added a commit that referenced this issue Aug 29, 2024
@garethrees
Remove "All time" filter from admin timeline
0c9f549 
On WhatDoTheyKnow this loads 17 million+ events, which attempts to use
over 5GB of memory.

In all but the smallest installs pagination will be required for this
page to be performant, so this commit removes the filter entirely.

Fixes #8369.
@garethrees garethrees mentioned this issue Aug 29, 2024
Closed
gbp added a commit that referenced this issue Nov 19, 2024
@gbp
Refactor timeline method in AdminGeneralController
d9116aa 
- Introduce get_timeline_events method for improved timeline event
  fetching
- Improve pagination handling for better performance limiting the amount
  of data loaded into memory
- Optimize SQL queries using UNION ALL for combined event types
- Separate logic for authority changes and info request events
- Remove redundant code and simplify overall structure

Fixes #8369
@gbp gbp linked a pull request Nov 19, 2024 that will close this issue
Open
gbp added a commit that referenced this issue Nov 19, 2024
@gbp
Refactor timeline method in AdminGeneralController
c71395e 
- Introduce get_timeline_events method for improved timeline event
  fetching
- Improve pagination handling for better performance limiting the amount
  of data loaded into memory
- Optimize SQL queries using UNION ALL for combined event types
- Separate logic for authority changes and info request events
- Remove redundant code and simplify overall structure

Fixes #8369
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Breaks expected functionality f:admin x:uk
Projects
None yet
Milestone
No milestone
2 participants
@gbp @garethrees