Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

respect channel layer capacity in RedisChannelLayer.receive_buffer #219

Merged
merged 1 commit into from
Sep 3, 2020
Merged

respect channel layer capacity in RedisChannelLayer.receive_buffer #219

merged 1 commit into from
Sep 3, 2020
+42 −2

Conversation

ryanpetrello
Copy link
Contributor

@ryanpetrello ryanpetrello commented Aug 20, 2020 •
edited
Loading

cc @carltongibson

respect the capacity setting so that the receive_buffer does not grow
without bounds (which can lead to a memory leak)

see: #212

based on the idea outlined here:

#384

I'd argue that these per-channel Queue objects should probably be bound in size. There's already a capacity argument; maybe the per-channel buffer should respect that, and only buffer up to that many objects before dropping old ones?

This was referenced Aug 20, 2020
Closed
Merged
@carltongibson
Copy link
Member

Hey @ryanpetrello — super stuff. I'm on 🏝 — give me a week or so to respond properly. Thanks!

@ryanpetrello
Copy link
Contributor Author

Hey @carltongibson,

I'm jealous 😄 enjoy!

ryanpetrello added a commit to ryanpetrello/awx that referenced this pull request Aug 21, 2020
@ryanpetrello
work around a memory leak in channels_redis
aa077bf 
see: django/channels_redis#212
see: django/channels_redis#219
ryanpetrello added a commit to ryanpetrello/awx that referenced this pull request Aug 25, 2020
@ryanpetrello
work around a memory leak in channels_redis
e25da21 
see: django/channels_redis#212
see: django/channels_redis#219
chrismeyersfsu pushed a commit to chrismeyersfsu/awx that referenced this pull request Aug 27, 2020
@ryanpetrello @chrismeyersfsu
work around a memory leak in channels_redis
087a189 
see: django/channels_redis#212
see: django/channels_redis#219
mabashian pushed a commit to mabashian/awx that referenced this pull request Aug 31, 2020
@ryanpetrello @mabashian
work around a memory leak in channels_redis
4256ee3 
see: django/channels_redis#212
see: django/channels_redis#219
@ryanpetrello
Copy link
Contributor Author

ryanpetrello commented Sep 1, 2020 •
edited
Loading

Hey @carltongibson,

Any chance you've had some time to look over this PR? I have a number of Ansible AWX users that are using a version of this patch, and it looks promising (none have yet reported back the memory leak after applying this patch; I've got at least one user who was seeing memory spikes every 48 hours and was restarting Daphne, and has now not reported any issues for nearly 3 weeks).

elyezer pushed a commit to ryanpetrello/awx that referenced this pull request Sep 1, 2020
@ryanpetrello @elyezer
work around a memory leak in channels_redis
e7fd2d9 
see: django/channels_redis#212
see: django/channels_redis#219
@carltongibson
Copy link
Member

Hi @ryanpetrello. I'm just back from holiday today (and had three versions of Django out this morning).

I'll try and give it a look this week. Thanks for the hustle! 😀

@ryanpetrello
Copy link
Contributor Author

Awesome - thanks!

carltongibson
carltongibson reviewed Sep 2, 2020
View reviewed changes
Copy link
Member

@carltongibson carltongibson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ryanpetrello — I know this is a slow leak but do you think you can add some basic regression tests of the the new queue?

@ryanpetrello
Copy link
Contributor Author

Hey @carltongibson - what sort tests are you looking for? Something that verifies that the queue won't grow beyond the configured capacity?

@carltongibson
Copy link
Member

Yeah, that sounds fine. "Did the basic behaviour break?" is enough.

@ryanpetrello
Copy link
Contributor Author

@carltongibson test added - let me know if this is what you're looking for.

@ryanpetrello ryanpetrello force-pushed the receive_buffer_capacity branch 2 times, most recently from 0037e2a to b6afb2a Compare September 2, 2020 14:09
@carltongibson
Copy link
Member

@ryanpetrello Quick glance, looks about right. Thank you.

I'll have one more look over the issues and then we'll bundle this up for a release. 👍

Thanks for your input here! 🥇

ryanpetrello
ryanpetrello commented Sep 2, 2020
View reviewed changes
channels_redis/core.py
@@ -544,7 +560,11 @@ async def new_channel(self, prefix="specific"):
Returns a new channel name that can be used by something in our
process as a specific channel.
"""
return "%s.%s!%s" % (prefix, self.client_prefix, uuid.uuid4().hex,)
return "%s.%s!%s" % (
Copy link
Contributor Author

@ryanpetrello ryanpetrello Sep 2, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@carltongibson I changed this because black was complaining about it style-wise

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem. Whatever black says. 😀

@ryanpetrello @carltongibson
Respect channel layer capacity in RedisChannelLayer.receive_buffer
64b258f 
respect the capacity setting so that the receive_buffer does not grow
without bounds

see: django#212
@carltongibson carltongibson merged commit 1d8db66 into django:master Sep 3, 2020
@ryanpetrello ryanpetrello mentioned this pull request Sep 8, 2020
Merged
AlanCoding pushed a commit to AlanCoding/awx that referenced this pull request Sep 10, 2020
@ryanpetrello @AlanCoding
work around a memory leak in channels_redis
02b955b 
see: django/channels_redis#212
see: django/channels_redis#219
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@carltongibson carltongibson carltongibson left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ryanpetrello @carltongibson