You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've found what I think it is some inconsistency about the possible data loss on the tail input plugin in respect of buffering.
In the "backpressure" documentation, it is stated that "The tail plugin can handle pauses without data
ingloss (sic), storing its current file offset and resuming reading later":
(for example, TCP input plugin). The tail plugin can handle pauses without data
In the other hand, the "Buffering and Storage" documentation says the opposite, "For some inputs, such as TCP and tail, pausing the input will almost certainly lead to log loss":
In a high load environment with backpressure the risks of having high memory usage is the chance of getting killed by the Kernel \(OOM Killer\). A workaround for this backpressure scenario is to limit the amount of memory in records that an input plugin can register, this configuration property is called `mem_buf_limit`. If a plugin has enqueued more than the `mem_buf_limit`, it won't be able to ingest more until that data can be delivered or flushed properly. In this scenario the input plugin in question is paused. When the input is paused, records will not be ingested until it is resumed. For some inputs, such as TCP and tail, pausing the input will almost certainly lead to log loss. For the tail input, Fluent Bit can save its current offset in the current file it is reading, and pick back up when the input is resumed.
I'd like to know, then, what is the behavior expected in this realm?
The text was updated successfully, but these errors were encountered:
Hi!
I've found what I think it is some inconsistency about the possible data loss on the
tail
input plugin in respect of buffering.In the "backpressure" documentation, it is stated that "The tail plugin can handle pauses without data
ingloss (sic), storing its current file offset and resuming reading later":
fluent-bit-docs/administration/backpressure.md
Line 27 in 20bf74f
In the other hand, the "Buffering and Storage" documentation says the opposite, "For some inputs, such as TCP and tail, pausing the input will almost certainly lead to log loss":
fluent-bit-docs/administration/buffering-and-storage.md
Line 35 in 20bf74f
I'd like to know, then, what is the behavior expected in this realm?
The text was updated successfully, but these errors were encountered: