Uvicorn workers don't shutdown on Ctrl+C when running as a gunicorn worker inside Docker and no TTY #2497
Unanswered
kajic
asked this question in
Potential Issue
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have been investigating an issue where Uvicorn workers don't shutdown when you issue a Ctrl+C, if you run them as Gunicorn workers inside Docker without a TTY. This StackOverflow issue which I created today describes the situation in some detail:
https://stackoverflow.com/questions/79139077/how-do-you-make-gunicorn-forward-sigint-to-uvicorn-when-running-inside-docker
I've been reading through Gunicorn's
arbiter.py
,workers/base.py
and Uvicorn'sworkers.py
, and I'm getting closer to understanding what's going on, if not why the implementation is what it is.First, when running without a TTY:
When gunicorn receives a SIGINT, it sends a SIGQUIT to its workers.
(
gunicorn/arbiter.py:handle_int
callsstop(False)
, andstop()
sends SIGQUIT whengraceful=False
.)In
uvicorn/workers.py:_install_sigquit_handler
, Uvicorn register's as callbackgunicorn/workers/base.py:handle_exit
for SIGQUIT.Interestingly, these two issues are referenced in _install_sigquit_handler.
- #1116
- benoitc/gunicorn#2604
Here I found it strange that Gunicorn doesn't forward the SIGINT, but forwards a SIGQUIT instead.
Anyway, when uvicorn receives SIGQUIT, the registered
handle_exit()
callback just setsself.alive=False
.I'm not sure how
self.alive
affects the lifecycle of a worker, but from here on nothing much happens until Gunicorn's--graceful-timeout
has elapsed, and then it sends a SIGKILL to the Uvicorn workers, and prints an error like "Worker (pid:8) was sent SIGKILL! Perhaps out of memory?".When instead you run Gunicorn+Uvicorn with a TTY, using
docker run -it ...
, and press Ctrl+C, mostly everything happens as before, but now (before Gunicorn receives SIGINT)uvicorn/server.py:handle_exit
receives SIGINT first, and it is able to shutdown successfully.I still haven't been able to figure out why Uvicorn is able to receive SIGINT when there is a TTY, but not otherwise, or why it is even able to receive the SIGINT before Gunicorn does. I'd be curious to know, but assuming this discrepancy is expected, I also don't understand why
_install_sigquit_handler
registershandle_exit
as the callback, which doesn't seem to do much more than setself.alive=False
, instead of, perhaps, usinghandle_quit
as the callback, which (I tried) does properly shutdown the workers.Sorry for the long post, but I thought I'd share my investigation in case someone with a better understanding of signal handling in Uvicorn can elaborate on why Uvicorn is able to receive SIGINT with a TTY, but not without, and why
_install_sigquit_handler
useshandle_exit
instead ofhandle_quit
to handle SIGQUIT.Thanks for your time!
Beta Was this translation helpful? Give feedback.
All reactions