You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
All of our examples currently are using the multi-threaded, work stealing flavor of tokio's runtime. This probably isn't the right choice for the generic use case - after all, lambdas only process one request at a time. So the multithreaded runtime is just added overhead, hurts cold starts, etc.
Probably we should cut over to making them explicitly current threaded, and just leave a note on why they might want to cut to multi-threaded (heavy use of async concurrency, background tasks that need cpu cycles, etc).
The main downside is that the current thread runtime is very vulnerable to blocking, so it might make it easier for people to shoot themselves in the foot. We should at least leave a breadcrumb, maybe a link to Alice's blog, if we cut over?
If this isn't a change we want to make, we should probably at least explicitly depend on the multithreaded runtime feature in our examples. Today it is implicitly enabled by the dependency in lambda_runtime (and sometimes lambda_extension). So the examples would break if we stripped that out per #984 . Better to be explicit in either case.
The text was updated successfully, but these errors were encountered:
If we did want to keep the multithreaded runtime usage, we probably should explicitly spawn the service function future to a tokio task.
Since, it is inefficient to run a loop delegating hot work, on the block_on thread. It is not a worker thread. This has a couple consequences:
It can result in sockets bouncing around between the worker thread and the external (block_on) threads which is inefficient
Any tasks the block_on thread spawns go to the global queue rather than a worker queue (requiring workers to synchronize before popping the queue and other overhead
It might even be worth documenting this best practice explicitly in crate docs. I think we probably should add a 'selecting a runtime flavor' section, which could mention this.
jlizen
changed the title
Consider shifting examples to use current thread tokio runtime
Add selecting a tokio runtime flavor section to docs, consider shifting examples to use current thread tokio runtime
May 6, 2025
All of our examples currently are using the multi-threaded, work stealing flavor of tokio's runtime. This probably isn't the right choice for the generic use case - after all, lambdas only process one request at a time. So the multithreaded runtime is just added overhead, hurts cold starts, etc.
Probably we should cut over to making them explicitly current threaded, and just leave a note on why they might want to cut to multi-threaded (heavy use of async concurrency, background tasks that need cpu cycles, etc).
The main downside is that the current thread runtime is very vulnerable to blocking, so it might make it easier for people to shoot themselves in the foot. We should at least leave a breadcrumb, maybe a link to Alice's blog, if we cut over?
If this isn't a change we want to make, we should probably at least explicitly depend on the multithreaded runtime feature in our examples. Today it is implicitly enabled by the dependency in
lambda_runtime
(and sometimeslambda_extension
). So the examples would break if we stripped that out per #984 . Better to be explicit in either case.The text was updated successfully, but these errors were encountered: