BatchWriter is missing functionality for limiting the queue size

[Zero3]

As far as I can see, the BatchWriter feature of this library has no functionality for limiting the queue size. So if the remote database becomes unavailable, there is nothing to prevent the application from running out of memory if points keep being added to the BatchWriter.

My particular use case, for what it is worth: I'm logging a significant number of events/metrics to an InfluxDB database every second. If my InfluxDB database goes down, my application should keep running. But if what I wrote above is true, the BatchWriter would quickly kill my application by making it run out of memory because of the unbounded queue.

I thus suggest new functionality for putting an upper bound on the number of enqueued points. It would be nice with the following two options for when the limit is reached:

1. Block the add operation until there is room in the queue (not a good idea for my use case, but is likely useable in other use cases)

2. Discard new points (what I would like in my use case).

It should be possible to add an event handler that gets notified in either case. In my use case, I would log the issue to our error management system.

[tihomir-kit]

Hi,

I agree, this is a good idea. I would be in favour of the option 2). And perhaps also option 3) which would start dropping the oldest points but keep the latest. And then once the DB becomes ready, flush.

However, at the moment I unfortunately don't have enough free time to work on this. If someone makes a PR, I'll gladly accept it.