RD_KAFKA_RESP_ERR_REBALANCE_IN_PROGRESS - “Group rebalance in progress“ breaks consumption of a topic

[sash]

Describe the bug
When consuming from time to time I get RD_KAFKA_RESP_ERR_REBALANCE_IN_PROGRESS from the broker. Those are short lived and look like something that has to be retried.

To Reproduce
I don’t have a way to cause rebalancing on queue to reproduce - this is something I observe in the runtime logs.

Expected behavior
I would think that the RD_KAFKA_RESP_ERR_REBALANCE_IN_PROGRESS should be one of the Retriable Errors in RetryableCommitter to have a retry auto applied when consuming.

Suggested fix (not tested)
See https://github.com/mateusjunges/laravel-kafka/blob/master/src/Commit/RetryableCommitter.php and add RD_KAFKA_RESP_ERR_REBALANCE_IN_PROGRESS to RETRYABLE_ERRORS

[sash]

Hey @mateusjunges any thoughts on this? I keep having short bursts of this error every day or two when I’m unable to consume because of rebalancing, and I think that if the retry is applied that would be solved.

[mateusjunges]

@sash I just released a new version as an attempt to fix this. If you see this problem again, please lmk and I'll investigate further