Skip to content

Commit

Permalink
Fix memory leak
Browse files Browse the repository at this point in the history
Creating a new session each time S3Storage is instantiated creates a memory leak. It seems that S3Storage can get created a bunch of times (I'm seeing it get created again and again as my app runs) and a boto3's Session takes loads of memory (see boto/boto3#1670) so my app eventually runs out of memory. This should fix the issue while still avoiding using the same session across different threads.
  • Loading branch information
almost committed Oct 11, 2024
1 parent 53af304 commit 9cae97f
Showing 1 changed file with 9 additions and 3 deletions.
12 changes: 9 additions & 3 deletions django_s3_storage/storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,16 @@ def open(self, mode="rb"):
self.file = self._storage.open(self.name, mode).file
return super().open(mode)


class _LocalSession(local):
def __init__(self):
self.session = boto3.session.Session()


local_session = _LocalSession()

class _Local(local):

class _Local(local):
"""
Thread-local connection manager.
Expand All @@ -117,8 +124,7 @@ def __init__(self, storage):
connection_kwargs["aws_session_token"] = storage.settings.AWS_SESSION_TOKEN
if storage.settings.AWS_S3_ENDPOINT_URL:
connection_kwargs["endpoint_url"] = storage.settings.AWS_S3_ENDPOINT_URL
self.session = boto3.session.Session()
self.s3_connection = self.session.client(
self.s3_connection = local_session.client(
"s3",
config=Config(
s3={"addressing_style": storage.settings.AWS_S3_ADDRESSING_STYLE},
Expand Down

0 comments on commit 9cae97f

Please sign in to comment.