SQS Trigger never executes

[mmmoli]

Describe the issue

Take this flow:

id: sqs
namespace: demo

tasks:
  - id: log_trigger
    type: io.kestra.plugin.core.log.Log
    message: "{{trigger.data}}"

triggers:
  - id: "realtime_trigger"
    type: io.kestra.plugin.aws.sqs.RealtimeTrigger
    accessKeyId: "{{ envs.aws_sqs_access_key_id }}"
    secretKeyId: "{{ envs.aws_sqs_secret_access_key }}"
    region: "{{ envs.aws_sqs_region }}"
    queueUrl: "{{ envs.aws_sqs_url_instruct_kestra }}"
    serdeType: JSON
    waitTime: "20.0000"
    maxNumberOfMessage: 10

• Send a message.
• No flow executes.
• Head over the Administration > Trigger.
• Hit "Unlock trigger" Button (don't know what this does).
• Check back at executions
• No flow executes

Logs show 243 entries of:

2024-12-18 14:20:31.881demosqsrealtime_trigger
[namespace: [demo](http://localhost:8080/ui/flows?namespace=demo)] [flow: [sqs](http://localhost:8080/ui/flows/edit/demo/sqs)] [trigger: realtime_trigger] Type io.kestra.plugin.aws.sqs.RealtimeTrigger started
2024-12-18 14:20:29.966demosqsrealtime_trigger
Unable to execute HTTP request: Acquire operation took longer than the configured maximum time. This indicates that a request cannot get a connection from the pool within the specified maximum time. This can be due to high request rate.
 Consider taking any of the following actions to mitigate the issue: increase max connections, increase acquire timeout, or slowing the request rate.
 Increasing the max connections can increase client throughput (unless the network interface is already fully utilized), but can eventually start to hit operation system limitations on the number of file descriptors used by the process. If you already are fully utilizing your network interface or cannot further increase your connection count, increasing the acquire timeout gives extra time for requests to acquire a connection before timing out. If the connections doesn't free up, the subsequent requests will still timeout.
 If the above mechanisms are not able to fix the issue, try smoothing out your requests so that large traffic bursts cannot overload the client, being more efficient with the number of times you need to call AWS, or by increasing the number of hosts sending requests.
 Acquire operation took longer than the configured maximum time. This indicates that a request cannot get a connection from the pool within the specified maximum time. This can be due to high request rate.
 Consider taking any of the following actions to mitigate the issue: increase max connections, increase acquire timeout, or slowing the request rate.
 Increasing the max connections can increase client throughput (unless the network interface is already fully utilized), but can eventually start to hit operation system limitations on the number of file descriptors used by the process. If you already are fully utilizing your network interface or cannot further increase your connection count, increasing the acquire timeout gives extra time for requests to acquire a connection before timing out. If the connections doesn't free up, the subsequent requests will still timeout.
 If the above mechanisms are not able to fix the issue, try smoothing out your requests so that large traffic bursts cannot overload the client, being more efficient with the number of times you need to call AWS, or by increasing the number of hosts sending requests.
 Acquire operation took longer than 5000 milliseconds.
2024-12-18 14:20:29.966[demo](http://localhost:8080/ui/flows?namespace=demo)sqsrealtime_trigger
[namespace: demo] [flow: [sqs](http://localhost:8080/ui/flows/edit/demo/sqs)] [trigger: realtime_trigger] [date: 2024-12-18T07:15:55.031Z] Trigger evaluation failed in the worker with error: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Acquire operation took longer than the configured maximum time. This indicates that a request cannot get a connection from the pool within the specified maximum time. This can be due to high request rate.
 Consider taking any of the following actions to mitigate the issue: increase max connections, increase acquire timeout, or slowing the request rate.
 Increasing the max connections can increase client throughput (unless the network interface is already fully utilized), but can eventually start to hit operation system limitations on the number of file descriptors used by the process. If you already are fully utilizing your network interface or cannot further increase your connection count, increasing the acquire timeout gives extra time for requests to acquire a connection before timing out. If the connections doesn't free up, the subsequent requests will still timeout.
 If the above mechanisms are not able to fix the issue, try smoothing out your requests so that large traffic bursts cannot overload the client, being more efficient with the number of times you need to call AWS, or by increasing the number of hosts sending requests.

Environment

Tried with kestra:v0.20.6 and kestra:develop

[anna-geller]

Could you try the same with maxNumberOfMessage: 1? I believe it might have sth to do with your trigger waiting for 10 messages, so with 1, it will be easier to test

[anna-geller]

related #568

[mmmoli]

Could you try the same with maxNumberOfMessage: 1? I believe it might have sth to do with your trigger waiting for 10 messages, so with 1, it will be easier to test

Thanks for coming back. Same issue.

[mmmoli]

Suspect it's more to do with how SQS is polled a while loop. I attempted to push a fix.

[anna-geller]

gotcha, do you want to contribute a PR from your fork?

[mmmoli]

gotcha, do you want to contribute a PR from your fork?

Happy to but it needs a review from someone who knows what they're doing.

I'll make it so we can review

• I don't have the build tools installed so I don't know if my PR compiles
• I don't know how to mount my fork into Kestra so I don't even know if it fixes the problem

[mmmoli]

fixed by #571