Add `selecting a tokio runtime flavor` section to docs, consider shifting examples to use current thread tokio runtime

[jlizen]

All of our examples currently are using the multi-threaded, work stealing flavor of tokio's runtime. This probably isn't the right choice for the generic use case - after all, lambdas only process one request at a time. So the multithreaded runtime is just added overhead, hurts cold starts, etc.

Probably we should cut over to making them explicitly current threaded, and just leave a note on why they might want to cut to multi-threaded (heavy use of async concurrency, background tasks that need cpu cycles, etc).

The main downside is that the current thread runtime is very vulnerable to blocking, so it might make it easier for people to shoot themselves in the foot. We should at least leave a breadcrumb, maybe a link to Alice's blog, if we cut over?

If this isn't a change we want to make, we should probably at least explicitly depend on the multithreaded runtime feature in our examples. Today it is implicitly enabled by the dependency in lambda_runtime (and sometimes lambda_extension). So the examples would break if we stripped that out per #984 . Better to be explicit in either case.

[jlizen]

If we did want to keep the multithreaded runtime usage, we probably should explicitly spawn the service function future to a tokio task.

Since, it is inefficient to run a loop delegating hot work, on the block_on thread. It is not a worker thread. This has a couple consequences:

• It can result in sockets bouncing around between the worker thread and the external (block_on) threads which is inefficient
• Any tasks the block_on thread spawns go to the global queue rather than a worker queue (requiring workers to synchronize before popping the queue and other overhead

It might even be worth documenting this best practice explicitly in crate docs. I think we probably should add a 'selecting a runtime flavor' section, which could mention this.

[jlizen]

I discussed this with some of my colleagues, and we are aligned that it's probably a bad idea to point people to current thread runtime in canonical docs or examples. While it technically might eke out some minor performance benefits for smaller lambdas running on 1vcpu, it just has so many footguns - for instance if any libraries you call use block_in_place(), it will panic.

So probably implicitly guiding people towards multi-threaded is the right choice, and anyway expert users can make their own decisions.

I'll still cut a PR to cut our examples over to spawning the handler future to a worker thread just since it is a general best practice.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.