Add way to change log level without restart

[awlayton]

Have trace on always is too much information.

[awlayton]

So there is a way to change the log level of pino, we would just need a way to signal the code to do so. My first thought was maybe having code watch some file?

[aultac]

The traditional way is set an env var and re-up container. It always seems a bit heavy to me though just to turn debugging messages on/off, and it doesn’t help you much if the thing is in some state where re-building causes the problem you’re debugging to go away. We can also pass signals, so in theory we could pick a standard signal and write a standard signal handler for Pino to catch and toggle through debug levels. Or, just have it watch a file is also fine and probably quickest, just kinda non-standard. Aaron

…

On Mar 12, 2021, at 11:07 AM, Alex Layton ***@***.***> wrote: So there is a way to change the log level of pino, we would just need a way to signal the code to do so. My first thought was maybe having code watch some file? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2KEUDG43A5D77CFHH7ZJDTDIU3DANCNFSM4KOKF4ZQ>.

[awlayton]

Re upping the container is restarting so that does not really apply here.

[awlayton]

With a file we could in effect update the DEBUG variable, not just pick between info, trace etc.

[aultac]

That is a good idea. I think watching the file is the simplest solution, but we need to figure out how to reconcile between DEBUG in the deployment environment and what’s in the file. How about on startup the entry point writes the current value of the env var to the file before starting node. Then node uses that file as authoritative (fallback to env), and sets watch on it. Then you can change DEBUG live while container is running, but when you restart or re-up container it falls back to the DEBUG you’ve specified in the environment in your overrides. That could get annoying though if the container is restarting and the reason you’re trying to change debug is to figure out (w/o rebuilding container) why its restarting. What we really need is some way to set the contents of the file to the env var when the container is built (i.e. on the first run of the entrypoint from the base image), but then any subsequent changes to the file just stay in place. That would allow you to actually override it, and then it would fall back to your env when you decide to rebuild container. I think that would be pretty easy to achieve by placing a file in the container when the base image that acts as a flag for “first run”, and it is set to false. Then entrapping changes that file to “true” the first time it runs. This wouldn’t be necessary if docker would just let you alter the environment of a running container from outside. Still thinking on it…. Aaron

…

On Mar 12, 2021, at 11:54 PM, Alex Layton ***@***.***> wrote: With a file we could in effect update the DEBUG variable, not just pick between info, trace etc. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2KEUEBPAHQI2NQRHMWOR3TDLOYJANCNFSM4KOKF4ZQ>.

[abalmos]

This wouldn’t be necessary if docker would just let you alter the environment of a running container from outside.

Even if you could change the env var, you would still have to restart the process for it to see the new value. You can't change the environment of a running process from outside of the process. At that point, why not just recreate the container? It should be fast now that it's based on an image.

Seems like a config file is the only real way forward. It would be pretty "standard" to change a config and then send the process a SIGHUP to re-read it. The crazy config setup could implement this for all services. I suppose the config lib would need to offer some sort of call back interface for things like pino to register with.

You could also set debug's $DEBUG value at runtime (say as an overridable default) from the config as well. Would just need to be sure to load the config before debug.

[awlayton]

Yes simply changing the environment variables at runtime will not do anything.

We need actual code that is triggered either on file change or a signal or something. Then that code can make the appropriate calls to pino and/or debug to update the log settings.

[aultac]

Sounds like the file is the preferred method as long as we architect it so that the env var takes precedence on container rebuild. Aaron

…

On Mar 14, 2021, at 6:06 PM, Alex Layton ***@***.***> wrote: Yes simply changing the environment variables at runtime will not do anything. We need actual code that is triggered either on file change or a signal or something. Then that code can make the appropriate calls to pino and/or debug to update the log settings. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2KEUBJUE45T6YDGHTYIBDTDUXNDANCNFSM4KOKF4ZQ>.