Buffer JSON slog messages #62
Conversation
|
The non-flushing behavior is probably unexpected and might make debugging confusing but I don't think is a problem in prod. Would it make sense to only turn on the bufwriter behavior with a release profile? |
I think it's fine if we enable https://docs.rs/slog-json/latest/slog_json/struct.JsonBuilder.html#method.set_flush |
Nice catch, thanks! I'll increase the buffer size as well. |
| @@ -70,6 +71,9 @@ pub(crate) fn init(service_info: &ServiceInfo, settings: &LoggingSettings) -> Bo | |||
| // NOTE: OXY-178, default is 128 (https://docs.rs/slog-async/2.7.0/src/slog_async/lib.rs.html#251) | |||
| const CHANNEL_SIZE: usize = 1024; | |||
|
|
|||
| // buffer json log lines up to 500 characters | |||
| const BUF_SIZE: usize = 500; | |||
There was a problem hiding this comment.
With lots of keys in json it's easy to exceed this. I'd go with something like 4k.
There was a problem hiding this comment.
Sorry, didn't see this last night and it didn't go to my cloudflare email for some reason even though I added it to this account. Thanks again!
serde_json will happily do a `write` syscall for every token it serializes. Add a std::io::BufWriter to prevent excessive syscalls.
serde_json will happily do a
writesyscall for every token it serializes. Add a small 100-character BufWriter buffer to prevent excessive syscalls.Note that serde_json does not appear to call
Write::flushso short lines will not be printed until the buffer is full.