feat: create Bytes from std::sync::Arc

[scottlamb]

...via infallible (Bytes::from_arc_projection) and fallible (Bytes::try_from_arc_projection) constructors.

#359 (comment) #437 (comment)

[Darksonn]

I'm not sold on this idea.

You can do something pretty similar with the existing from_owner method by wrapping the Arc<T> in a custom struct that runs the closure in the AsRef impl.

[scottlamb]

You can do something pretty similar with the existing from_owner method by wrapping the Arc in a custom struct that runs the closure in the AsRef impl.

That would work but it's a lot of otherwise-unneeded allocations I'd prefer to avoid.

[scottlamb]

Expanding a bit...

I'm trying to use this to replace my use of reffers::ARefss, reducing my dependency count and gaining compatibility with e.g. axum. In my current usage, I have a variety of calls to ARefss::map and ARefss::try_map with closures referencing the outer scope. A couple examples:

https://github.com/scottlamb/moonfire-nvr/blob/284a3dd5a9640c0fb854e9076a1e96da715c91e1/server/src/mp4.rs#L718

https://github.com/scottlamb/moonfire-nvr/blob/284a3dd5a9640c0fb854e9076a1e96da715c91e1/server/src/mp4.rs#L750

If I did this via an owner + Deref, I would have to make a 'static version of those lambdas, so put that state into the object that stays around. I'd either end up with a Box<dyn Fn> (in addition to the boxing that Bytes::from_owner) does or some ugly templatized thing to specialize on the function and state that it keeps (for as long as the Bytes exists, even though it's only necessary to get the slice at the beginning).

[scottlamb]

Fixed no-std.

[scottlamb]

...or not quite, on thumbv6m-none-eabi it still says this:

  1 | use alloc::sync::Arc;
    |            ^^^^ could not find `sync` in `alloc`

Hmm, would the best thing be to just gate this feature with something like #[cfg(not(feature = "extra-platforms"))]? It looks like there's also a portable_atomic_util::Arc, but that's a whole different type, so I'm not sure it makes sense to replace the method with that on those platforms. And I'm not sure if you want the portable_atomic stuff to be in the public API surface anyway.

[Darksonn]

I'm sorry, but I don't think we want to add this feature at this time. I recommend using Bytes::from_owner instead.

If I did this via an owner + Deref, I would have to make a 'static version of those lambdas, so put that state into the object that stays around. I'd either end up with a Box<dyn Fn> (in addition to the boxing that Bytes::from_owner) does or some ugly templatized thing to specialize on the function and state that it keeps (for as long as the Bytes exists, even though it's only necessary to get the slice at the beginning).

You can write your own custom struct that stores an Arc<T> and an *const [u8] for the output of the projection, then have AsRef<[u8]> dereference the raw pointer.

[scottlamb]

You can write your own custom struct that stores an Arc and an *const [u8] for the output of the projection, then have AsRef<[u8]> dereference the raw pointer.

Thanks, I hadn't considered layering an unsafe on top of a safe interface like that.

It does work. The extra allocations and deallocations are about 2% of my program's user+system CPU time under load in CPU profiles. 2% might not sound like much, but I'd love to eliminate it if you'd ever be willing to reconsider either this approach or exposing the vtable in some fashion. If there's anything I can do to make either of those acceptable to you, I'm all ears.

(I'm trying to mux .mp4 video files to HTTP clients on-the-fly on $15 SBCs while also saving several streams of incoming video to disk. It's not easy to quickly produce all the indexes at the beginning of a .mp4 so the web interface feels responsive, so I chase down all the 2% wins I can, and I'm hesitating at the regression.)

[bschoenmaeckers]

Would an implementation for From<Arc<[u8]>> for Bytes using Bytes::from_owner be accepted?

[Darksonn]

Please open a new issue for feature requests, instead of commenting on old closed issues.