Tracking Issue for externally implementable items

[traviscross]

This is a tracking issue for the lang experiment on externally implementable items. This currently covers these proposals:

• [RFC] externally implementable functions rfcs#3632
• [RFC] externally definable statics rfcs#3635
• Externally implementable traits rfcs#3645
• RFC: Existential types with external definition rfcs#2492

The feature gate for the issue is #![feature(extern_item_impls)].

About tracking issues

Tracking issues are used to record the overall progress of implementation. They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions. A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Get alternate proposals.
  • Original proposal from @m-ou-se. [RFC] externally implementable functions rfcs#3632
  • Alternative proposal from @m-ou-se. [RFC] externally definable statics rfcs#3635
  • Alternative proposal from @Amanieu. Externally implementable traits rfcs#3645
  • Alternative proposal from @tmandry. [RFC] externally implementable functions rfcs#3632 (comment)
• Pick a design for the experiment.
  • Using attributes rather than new syntax: Tracking Issue for externally implementable items #125418 (comment)
• Submit project goal: Add project goal: Externally Implemtable Items (EII) rust-project-goals#198
  • Approval
• Implement the experiment.
  • First fix attributes so we can do name resolution in them: Attribute handling reworks compiler-team#796
  • Implement externally implementable items using attributes. (In progress!)
• Verify proposals are consistent with:
  • Unsafe Extern Blocks rfcs#3484
• Write/update RFC based on experiment.
• Accept an RFC.
• Add documentation to the dev guide.
  • See the instructions.
• Add documentation to the reference.
  • See the instructions.
• Add formatting for new syntax to the style guide.
  • See the nightly style procedure.

Unresolved Questions

• Which exact alternative or alternatives should we adopt?

Related

• [RFC] externally implementable functions rfcs#3632
• [RFC] externally definable statics rfcs#3635
• Externally implementable traits rfcs#3645
• RFC: Existential types with external definition rfcs#2492

Implementation history

TODO.

cc @m-ou-se @Amanieu @tmandry @joshtriplett

[m-ou-se]

@jdonszelmann and I are currently implementing this, with a different syntax. Instead of using new syntax (e.g. extern impl fn ...) as in the RFC draft, we're using attributes instead:

#[externally_implementable(thing_handler)]
fn do_thing();

#[thing_handler]
fn my_thing_handler() {
    ..;
}

This has a number of advantages. #[thing_handler] could be replaced by a proc_macro in semver-compatible versions of the crate, so we don't need to worry about providing conversion/upgrade mechanisms. It also makes it look identical to #[panic_handler] and #[global_allocator] attributes. It clarifies the naming and visibility issue: do_thing is the function to call, and #[thing_handler] is the attribute to override it, so they each have their own name and visibility. And it allows us to place unsafe in the attribute itself, to separate "unsafe to implement" and "unsafe to call".

More updates soon!

[Amanieu]

Does this introduce a new global namespace for thing_handler? I would prefer if this was a proper path to an item within a crate since that properly takes care of things like multiple instances of the same crate with different major versions.

[m-ou-se]

Does this introduce a new global namespace for thing_handler?

Nope! It behaves the same as any other attribute (macro), as an item with a path.

[bstrie]

It would be nice to have a more-or-less complete list of externally-definable items that the stdlib would seek to provide. RFC 3632 mentions panic_handler and global_allocator, and implies that there are others that could use this, but I'm drawing a blank.

[bjorn3]

The rng used by DefaultHasher (so we can move HashMap to alloc), and probably things like the Mutex implementation, and other well defined OS interfaces.

[LegNeato]

Why not...everything? Some use-cases I can come up with off the top of my head:

• Filesystem so you can swap out for an in-memory FS during testing and fuzzing. Or so you can scope a binary to a particular FS sandbox and your deps are blissfully unaware. Or for FS support on embedded with custom impls for your particular board + storage + OS combo.
• Channels so you can use them to communicate between workgroups when run on the GPU via rust-gpu
• Network so you can simulation test code in a made up network or proxy real services in production. You could also add observability at the network api level so every dependency using networking use doesn't need to do so.
• Threads, so I can put a watchdog or heartbeat on all threads created.

Give me a rust api, and I can come up with multiple situations for why I'd want to control it from user code. Almost all of them involve testing and observability, but there are real capability use-cases as well.

IMO std should be a bunch of traits with a default concrete implementation. That's too big of an ask, but this feature has the potential to get 80% of the way there and unlock use-cases and quality we can't imagine today.

[Enselic]

[...] implies that there are others that could use this, but I'm drawing a blank.

I intend to try to solve the SIGPIPE problem with an externally implementable item, as suggested here.

libstd would somehow (tbd) define a "pre-main" hook that by default sets SIG_IGN on SIGPIPE on unix. Then a simple [dependencies] sigpipe-default = "1.0.0" in Cargo.toml would override that function in libstd and instead set SIG_DFL on SIGPIPE. Or leave it untouched (sigpipe-inherit).

[programmerjake]

[...] implies that there are others that could use this, but I'm drawing a blank.

I intend to try to solve the SIGPIPE problem with an externally implementable item, as suggested here.

libstd would somehow (tbd) define a "pre-main" hook that by default sets SIG_IGN on SIGPIPE on unix. Then a simple [dependencies] sigpipe-default = "1.0.0" in Cargo.toml would override that function in libstd and instead set SIG_DFL on SIGPIPE. Or leave it untouched (sigpipe-inherit).

maybe a better approach would be to have an externally implemented item that handles EPIPE from the print! family of macros, by default it would panic, but it could be overridden to exit without any messages (exiting without messages might be a better default anyway...maybe a silent panic so cleanup still runs?). that way your process wouldn't abort if a network connection was dropped, but would still behave like a standard unix application and terminate if stdout wasn't fully read in a pipeline.

[bjorn3]

The intent of not changing SIGPIPE is that if the program inherited the default signal disposition for SIGPIPE, the program will kill the process on any read/write of a broken pipe, not just for those in the print! family. However if the program inherited SIGIGN for SIGPIPE it would behave as libstd currently does and returning EPIPE. The inheriting the signal disposition is the important part. If not for that part everyone could just change the signal disposition for SIGPIPE back to the default.