Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possibility to Pass PartitioningKey Dynamically in Kafka Sensor #3358

Open
MrHassanMurtaza opened this issue Nov 12, 2024 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@MrHassanMurtaza
Copy link

Is your feature request related to a problem? Please describe.
We have usecase where we want to specify the partitioning key for the kafka dynamically in the Sensor "Kafka Trigger"

Describe the solution you'd like
In the Kafka Trigger Sensor, we want to have the possibility to specify the partitioning key dynamically.

For Example:
If we're listening to Argo WF events using K8s resource event source, we would like possibility to pass WorkflowName as a partition key to make sure all the events are processed in order for the specific workflow.

Describe alternatives you've considered
Alternative could using Lambda functions as a trigger and parsing the message in the function to eventually add a partitioning key.

Another alternative could be to use a static partition key, but that wouldn't very scalable across multiple consumers.

Additional context
N/A

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

@MrHassanMurtaza MrHassanMurtaza added the enhancement New feature or request label Nov 12, 2024
@Thomasr-02
Copy link
Contributor

Hey @MrHassanMurtaza , see this, #2360.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants