Skip to content

Add selecting a tokio runtime flavor section to docs, consider shifting examples to use current thread tokio runtime #985

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jlizen opened this issue May 6, 2025 · 1 comment

Comments

@jlizen
Copy link

jlizen commented May 6, 2025

All of our examples currently are using the multi-threaded, work stealing flavor of tokio's runtime. This probably isn't the right choice for the generic use case - after all, lambdas only process one request at a time. So the multithreaded runtime is just added overhead, hurts cold starts, etc.

Probably we should cut over to making them explicitly current threaded, and just leave a note on why they might want to cut to multi-threaded (heavy use of async concurrency, background tasks that need cpu cycles, etc).

The main downside is that the current thread runtime is very vulnerable to blocking, so it might make it easier for people to shoot themselves in the foot. We should at least leave a breadcrumb, maybe a link to Alice's blog, if we cut over?

If this isn't a change we want to make, we should probably at least explicitly depend on the multithreaded runtime feature in our examples. Today it is implicitly enabled by the dependency in lambda_runtime (and sometimes lambda_extension). So the examples would break if we stripped that out per #984 . Better to be explicit in either case.

@jlizen
Copy link
Author

jlizen commented May 6, 2025

If we did want to keep the multithreaded runtime usage, we probably should explicitly spawn the service function future to a tokio task.

Since, it is inefficient to run a loop delegating hot work, on the block_on thread. It is not a worker thread. This has a couple consequences:

  • It can result in sockets bouncing around between the worker thread and the external (block_on) threads which is inefficient
  • Any tasks the block_on thread spawns go to the global queue rather than a worker queue (requiring workers to synchronize before popping the queue and other overhead

It might even be worth documenting this best practice explicitly in crate docs. I think we probably should add a 'selecting a runtime flavor' section, which could mention this.

@jlizen jlizen changed the title Consider shifting examples to use current thread tokio runtime Add selecting a tokio runtime flavor section to docs, consider shifting examples to use current thread tokio runtime May 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant