Flush on terminate event #9
Labels
enhancement
New feature or request
Comments
|
@doranych it could even be that the kafka message is sent via an async message, like first SQS and after that Kafka so a retry mechanism is in place. This will also prevent the wait in the interface for 10 seconds. So then the current implementation is just fine. So it would be nice if this enhancement is been made, it should be customizable when the flush occures. |
|
чт, 7 мая 2020 г., 14:43 Rudy den Otter <[email protected]>:
… @doranych <https://github.com/doranych> it could even be that the kafka
message is sent via an async message, like first SQS and after that Kafka
so a retry mechanism is in place. This will also prevent the wait in the
interface for 10 seconds. So then the current implementation is just fine.
So it would be nice if this enhancement is been made, it should be
customizable when the flush occures.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#9 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEZEDLQBDLYJKKWD326TJS3RQKNHHANCNFSM4LFFGVAA>
.
|
|
@KonstantinCodes @rdotter |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I think, that we could wait too much, if kafka cluster is lagging.
Imagine, you have to sent 10 events to kafka. Each one will wait max 10s.
In case of max waiting per message, you will get 100 sec await per process.
Sounds crazy.
What do you think about subscription on
kernel.teminateevent to flush?We can call flush in cycle, to prevent message loss. In worst case process will wait 10 seconds, because of rdkafka batch collection.
The text was updated successfully, but these errors were encountered: