Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stderr backtrace caused by redis.client.Redis.__del__ on process exit. #3014

Open
byron-lambda opened this issue Oct 17, 2023 · 3 comments · May be fixed by #3397
Open

Stderr backtrace caused by redis.client.Redis.__del__ on process exit. #3014

byron-lambda opened this issue Oct 17, 2023 · 3 comments · May be fixed by #3397

Comments

@byron-lambda
Copy link

byron-lambda commented Oct 17, 2023

Version: 5.0.1

Platform: Python 3.8.18 on Ubuntu 20.04

Description: Backtrace to stderr on process exit due to redis.client.Redis.__del__()

Here is the trace:

Exception ignored in: <function Redis.__del__ at 0x7fcf29e50ca0>
Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/redis/client.py", line 491, in __del__
  File "/venv/lib/python3.8/site-packages/redis/client.py", line 506, in close
  File "/venv/lib/python3.8/site-packages/redis/connection.py", line 1155, in disconnect
  File "/venv/lib/python3.8/site-packages/redis/connection.py", line 1062, in _checkpid
AttributeError: 'NoneType' object has no attribute 'getpid'

Occurs due to unloaded globals being referenced in methods called in destructors on process exit. See warning in Python docs here

This can be fixed in a couple of ways:

  1. Add an exception handler to redis.client.Redis.__del__() as it appears has been done in many other destructors (
    def __del__(self):
    for example)
  2. Add references to required globals to the redis.connection.ConnectionPool class and call via the references. The globals that require references in are os.getpid and chain (from itertools) afaict.

NOTE: It is quite hard to come up with a small piece of code to reproduce this issue as it is caused by a race in the unload order at exit. I'll try to come up with something reliable if req'd.

NOTE: The issue is triggered by

self.connection_pool.disconnect()
which was recently introduced in 012f7cf

@diogosilva30
Copy link

Can confirm this behaviour with version 5.0.1 with Django. I get this error when the development server restarts

@Alucido
Copy link

Alucido commented Aug 23, 2024

Can confirm this behaviour with version 5.0.8, Python 3.9, when running pytest test suite opening a Redis Client in a subclass of the beam.DoFn class, as part of its setup method. The error is reproduced when the test suite ends and the client is closed.

noirbee added a commit to noirbee/redis-py that referenced this issue Oct 1, 2024
Client.close() may call ConnectionPool.release() or
ConnectionPool.disconnect(); both methods may end up calling
os.getpid() (through ConnectionPool._checkpid() or threading.Lock()
(through ConnectionPool.reset()). As mentioned in the Python
documentation [1], at interpreter shutdown, module globals (in this
case, the os and threading module references) may be deleted or set to
None before __del__() methods are called. This causes an
AttributeError to be raised when trying to run e.g. os.getpid(); while
the error is ignored by the interpreter, the traceback is still
printed out to stderr.

Closes redis#3014

[1] https://docs.python.org/3/reference/datamodel.html#object.__del__
@noirbee noirbee linked a pull request Oct 1, 2024 that will close this issue
6 tasks
@noirbee
Copy link

noirbee commented Oct 1, 2024

I've opened #3397 after encountering this issue following a ddtrace update (probably due to the module loading / patching shenanigans it performs for libraries it wants to monkey-patch).

Thanks @byron-lambda for opening this issue in the first place, and pointing me in the right direction !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants