Skip to content
This repository has been archived by the owner on Oct 16, 2023. It is now read-only.

Install graceful shutdown on server #18

Merged
merged 1 commit into from
Sep 18, 2023

Conversation

bltavares
Copy link
Contributor

This commit enables gracefull shutdown on the server to handle SIGTERM signals from the cluster and improve service restarts

Copy link
Collaborator

@awoimbee awoimbee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand much but it looks a lot like the example implem so it's OK.

Thanks for the PR, sorry I didn't see it sooner, we are in the process of moving to https://github.com/Trow-Registry/trow

@@ -226,6 +226,10 @@ impl TrowBuilder {
// Start GRPC Backend task
tokio::task::spawn(init_trow_server(self.config.clone())?);

// Listen for termination signal
let handle = axum_server::Handle::new();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer handle to be called shutdown_handle, but it's OK

This commit enables gracefull shutdown on the server to handle SIGTERM signals from the cluster and improve service restarts
@awoimbee awoimbee merged commit ab1c68a into Extrality:main Sep 18, 2023
4 of 6 checks passed
@bltavares
Copy link
Contributor Author

Thanks for merging it. I'm glad the transfer worked out.
This is a really nifty project that was exactly what I needed for my homelab.

I guess I could have done a better job explaining the need for this change.

While I was tuning resource utilization and healthchecks timeout, I've noticed that trow would ignore SIGTERM calls from the scheduler.

It would hang for up-to 30s the graceful time expired and get SIGKILL-ed. This caused a larger delay as the scheduler has to wait for the process to stop to start another instance (in my case).

I believe this was being caused by the GRPC + Axum spawning tasks that would not receive the termination signal. With this change, calling ctrl-c, for example, immediatelly asks the reactor and axum to terminate connections and tasks, so it can finish faster and be re-scheduled.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants