Process ends unexpectedly with exit code 0

[darkling]

We're running a website built with next.js, running on node. This is hosted within kubernetes, with four k8s pods running the service.

Every so often (a couple of times a day), one or more of the pods halts, and is restarted by k8s, and we don't know why. In one recent incident, all four pods restarted at the same time, leading to a brief service outage.

The key thing here is that it's exiting with an exit code of 0, so it appears to be a normal clean exit from the process.

There's nothing on stdout or stderr to indicate why the pod stops -- all we get is a message from npm saying that there's an updated version of npm available (but I think that's normal for npm exiting). k8s says that the pods are failing their readiness and liveness probes, but it's not killing them itself (because then we'd see an exit code >128 from the kill signal). The pod failures and restarts do not appear to be correlated with high workload on either the site itself or the k8s nodes hosting them.

We're at a loss as to how to pin this down any further -- we don't think this code should be stopping at all, let alone exiting with a code of 0. Is there anything we could do in node itself to enable additional debugging or tracing to work out why this is happening? Unfortunately, our only reproducer is to let it run for a few days and wait for it to restart.

[RedYetiDev]

Can you reproduce without dependencies? This might be an issue with one of them.

[darkling]

Not really, it's a big and complex site, and if we started pruning dependencies, we'd very rapidly end up with no site (and the customer would get unhappy about us taking away their UAT platform for days or weeks at a time while we wait for it to happen again).

How would "a dependency" cause this behaviour in the first place? Simply something calling exit(0)? (Which seems like a Bad Thing for any code to be doing, but at least we could audit for it).

What I'd like to do is put in some hook that gets called on exit, possibly to dump a stack trace or print some logging so that we can see how it's getting to the point of this exit... but we don't know how to do things at that kind of low level (if it's even possible).

[RedYetiDev]

https://nodejs.org/api/process.html#event-exit ?

---

Without a minimal reproduction, there's not much that can be done on this end.

[RedYetiDev]

Hey, I've closed this discussion a there's no minimal reproduction, but I'm happy to reopen if that changes