You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Whenever rebalance event is happening, each workers is revoked and then assigned the list of partitions each. As a result, consumer might get new partitions or remove existing. In sake of simplicity and safety in both cases the consumer state is reset and it start consuming messages from previous commit/offset.
However, in theory if the partitions is only added to the consumers and all of existing are kept with a consumer it makes sense to not reset the state for performance reasons.
Whenever rebalance event is happening, each workers is revoked and then assigned the list of partitions each. As a result, consumer might get new partitions or remove existing. In sake of simplicity and safety in both cases the consumer state is reset and it start consuming messages from previous commit/offset.
However, in theory if the partitions is only added to the consumers and all of existing are kept with a consumer it makes sense to not reset the state for performance reasons.
See https://github.com/delta-io/kafka-delta-ingest/pull/27/files/8e598e35406f5d2978306134ddde9d9d54b4dccc#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759R420 for inspiration
The text was updated successfully, but these errors were encountered: