Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kafka concurrent access #1282

Merged

Conversation

bbortt
Copy link
Collaborator

@bbortt bbortt commented Dec 17, 2024

closes #1281.

@bbortt bbortt self-assigned this Dec 17, 2024
@bbortt bbortt force-pushed the fix/#1281-kafka-concurrent-access branch from 5c41054 to 1c3db3d Compare December 17, 2024 18:31
@bbortt bbortt requested review from tschlat and christophd and removed request for tschlat December 17, 2024 18:34
@bbortt
Copy link
Collaborator Author

bbortt commented Dec 17, 2024

@bbortt bbortt requested review from tschlat and christophd and removed request for tschlat

ah yea, that makes sense 💃🏼 my clicking is way too fast for GitHub.

@bbortt bbortt force-pushed the fix/#1281-kafka-concurrent-access branch from 1c3db3d to de66a59 Compare December 17, 2024 18:38
@bbortt bbortt added State: Review If pull-request has been opened an is ready/in review and removed State: WIP labels Dec 17, 2024
@bbortt bbortt force-pushed the fix/#1281-kafka-concurrent-access branch from de66a59 to bf69def Compare December 18, 2024 08:32
@bbortt bbortt force-pushed the fix/#1281-kafka-concurrent-access branch 3 times, most recently from 39f687b to aa1566a Compare January 8, 2025 10:36
@bbortt
Copy link
Collaborator Author

bbortt commented Jan 8, 2025

@tschlat I've now added a flag useThreadSafeConsumer... that makes it backwards compatible.

@bbortt bbortt requested a review from tschlat January 8, 2025 12:17
bbortt added 2 commits January 9, 2025 07:39
…a consumer

added a reproducer for citrusframework#1281.
indeed the test which is part of the `KafkaEndpointJavaIT` throws:

```shell
java.util.ConcurrentModificationException: KafkaConsumer is not safe for multi-threaded access. currentThread(name: Thread-1, id: 124) otherThread(id: 125)
```
…tion

The `org.apache.kafka.clients.consumer.KafkaConsumer<K,V>` is **not thread-safe**.
When using selective message consumption, it is recommended to configure `useThreadSafeConsumer` respectively `thread-safe-consumer` for the Kafka endpoint.
Otherwise, you will experience errors when executing tests in parallel.
@bbortt bbortt force-pushed the fix/#1281-kafka-concurrent-access branch from aa1566a to 8744ddd Compare January 9, 2025 06:41
@bbortt bbortt merged commit 8e2a3f7 into citrusframework:main Jan 9, 2025
1 check passed
@bbortt bbortt deleted the fix/#1281-kafka-concurrent-access branch January 9, 2025 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Prio: High State: Review If pull-request has been opened an is ready/in review State: WIP Type: Bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

KafkaMessageFilteringConsumer#findMessageWithTimeout concurrent access
3 participants