Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Idempotent decorator for asynchronous functions #4483

Open
1 of 2 tasks
esthermcmahon opened this issue Jun 10, 2024 · 2 comments
Open
1 of 2 tasks
Labels
feature-request feature request help wanted Could use a second pair of eyes/hands idempotency Idempotency utility need-customer-feedback Requires more customers feedback before making or revisiting a decision

Comments

@esthermcmahon
Copy link

Use case

Would love to be able to use the AsyncBatchProcessor and async_process_partial_response function along with the idempotency utility.

Solution/User Experience

def async_idempotent_function():
    def wrapper(func):
        @functools.wraps(func)
        async def wrapped(*args):
            return await func(*args)
        return wrapped
    return wrapper

Alternative solutions

No response

Acknowledgment

@esthermcmahon esthermcmahon added feature-request feature request triage Pending triage from maintainers labels Jun 10, 2024
Copy link

boring-cyborg bot commented Jun 10, 2024

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@heitorlessa heitorlessa removed the triage Pending triage from maintainers label Jun 12, 2024
@heitorlessa
Copy link
Contributor

hey @esthermcmahon, thank you so much for taking the time to open this feature request. Are you using DynamoDB or Redis as a persistence storage for Idempotency?

The reason we haven't added support for async in idempotency yet is that the AWS SDK is synchronous. This means it'll be blocking and you wouldn't fully benefit from async I/O - e.g., calls to store/retrieve data from Amazon DynamoDB / Redis would block the event loop for a few milliseconds.


for maintainers and anyone looking to help implementing it :)

Off the top of my head, a new async decorator along with these two changes could make it possible (needs to be tested):

  • For DynamoDB. We'd need a new AsyncDynamoDBPersistenceLayer requiring aioboto3 as a dependency.
  • For Redis. We'd need a new AsyncRedisCachePersistenceLayer; Redis client supports asyncio so only syntax changes

PS: Adding help wanted to also signal we need to track customer demand. Personally, this is gonna become a common need in Data Engineering and large data pipelines.

@heitorlessa heitorlessa added help wanted Could use a second pair of eyes/hands need-customer-feedback Requires more customers feedback before making or revisiting a decision idempotency Idempotency utility labels Jun 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request feature request help wanted Could use a second pair of eyes/hands idempotency Idempotency utility need-customer-feedback Requires more customers feedback before making or revisiting a decision
Projects
Development

No branches or pull requests

2 participants