Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shutdown server while client is still connected #57

Open
flixr opened this issue Nov 30, 2020 · 3 comments
Open

shutdown server while client is still connected #57

flixr opened this issue Nov 30, 2020 · 3 comments

Comments

@flixr
Copy link

flixr commented Nov 30, 2020

Is there a way to cleanly shutdown the server while there are still clients connected (i.e. finish the connections)?
I could not find any method/callback for this.. Am I missing something or is this not implemented?

When I try to shutdown the server and a RpcHandler is still running (because the client didn't disconnect yet), it will just hang forever.
If I then shutdown the client I get a failure with RPCs still in flight:

I1130 16:54:18.130954     7 server.cc:215] Shutting down server.

-> hangs here until I shutdown the client and then fails:

I1130 16:56:32.376948     7 completion_queue_thread.cc:39] Shutting down completion queue 0x557085fd3a40
I1130 16:56:32.377357     7 completion_queue_thread.cc:39] Shutting down completion queue 0x557085fc8f20
I1130 16:56:32.377645     7 event_queue_thread.cc:38] Shutting down event queue 0x557085fcc400
I1130 16:56:32.377735     7 event_queue_thread.cc:38] Shutting down event queue 0x557085fcbb00
I1130 16:56:32.377812     7 server.cc:237] Shutdown complete.
F1130 16:56:32.383496     7 rpc.cc:355] RPCs still in flight!

It would be nice to have a OnShutdown callback in the RpcHandler (where the handler could then finish the write and hence shutdown the connection to the client).
Or should this be done e.g. by registering a custom callback via ExecutionContext?

@TobyEalden
Copy link

I came across this too recently, I ended up adding a Rpc::TryCancel method and calling it from Service::StopServing.

Something along the lines of:

void Rpc::TryCancel() {
  if (IsRpcEventPending(Rpc::Event::READ) ||
      IsRpcEventPending(Rpc::Event::WRITE) ||
      IsRpcEventPending(Rpc::Event::FINISH)) {
    LOG(TRACE) << "Rpc::TryCancel for " << (void*)this;
    server_context_.TryCancel();
  }
}

which is called from a new method on ActiveRpcs:

void ActiveRpcs::TryCancel() {
  common::MutexLocker locker(&lock_);
  for (auto it : rpcs_) {
    it.second->TryCancel();
  }
}

which in turn is called from Service!

void Service::StopServing() {
  shutting_down_ = true;
  active_rpcs_.TryCancel();
}

@flixr
Copy link
Author

flixr commented Dec 1, 2020

Hm... interesting..
I'm not very familiar with gRPC (yet), but it says

it is generally preferred to explicitly finish an RPC by returning Status::CANCELLED rather than using TryCancel

so I added a OnServerShutdown callback where Finish(Status::CANCELLED) can be called. See #58
Feedback would be very much appreciated ;-)

@flixr
Copy link
Author

flixr commented Dec 2, 2020

Neither the TryCancel nor the OnServerShutdown (at least as done in #58) works.

I have a working solution through custom callback via execution context, but it would be nice to have this supported by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants