Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the 'static bound from everywhere #100

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

alecmocatta
Copy link
Member

@alecmocatta alecmocatta commented Jul 30, 2020

This enables big performance improvements by not having to clone elements for ParallelStreams.

The complication is that, akin to crossbeam::scope, we need to block before returning to avoid the user mem::forgetting the handle and mutating the data while other threads still have a reference to it. If we were sync (like for example rayon) this wouldn't be much of an issue. As we're async and have to return whenever I/O blocks, this is harder to deal with.

The current compromise in this PR is using tokio to block the current task (when running under the multithreaded scheduler) or current thread (when running under the basic scheduler). This is unfortunately mildly surprising from a user POV. WIP.

TODO:

  • wait for cancelled ThreadPool::spawned futures (e.g. during .all()) to ack that they've been cancelled, to avoid returning while a task may be running on another thread that uses referenced data
    • sync block - necessary for safety
    • async - optimisation to avoid blocking

See also:
tokio-rs/tokio#1879 tokio-rs/tokio#2576 tokio-rs/tokio#2596 rust-lang/rfcs#2958 rmanoka/async-scoped

There are two issues:

  • It's possible to forget a borrow without running an associated destructor RFC: Alter mem::forget to be safe rust-lang/rfcs#1066. Result: we can't async yield to the user until tasks have been cancelled.
  • Cancellation of tasks on other threads is async. Result: in JoinHandle::drop we must block on this async cancellation. Prefer some sort of async drop as an optimisation to avoid the blocking.

And a third:

  • Cancellation points. Almost no code has them. What to do about that‽

@alecmocatta alecmocatta force-pushed the relax-static branch 2 times, most recently from f8e5880 to 5361648 Compare August 18, 2020 17:30
@alecmocatta alecmocatta changed the title WIP: Remove the 'static bound from everywhere Remove the 'static bound from everywhere Aug 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant