Integration with the bytes crates

[dignifiedquire]

A lot of my networking code uses bytes to handle efficient storing of incoming and outgoing packets. I would like to transition more of that code to use zerocopy, but for that some integration would be required.

The following example is for the parsing side of what I would like to do, but this fails as ByteSlice is not implemented for bytes::Bytes or bytes::BytesMut.

I can't implement this in my own crate, as both the type and the trait are in 3rd party crates.

use zerocopy::{AsBytes, ByteSlice, FromBytes, FromZeroes, Ref, Unaligned};
use bytes::Bytes;

#[derive(FromZeroes, FromBytes, AsBytes, Unaligned)]
#[repr(C)]
struct UdpHeader {
    src_port: [u8; 2],
    dst_port: [u8; 2],
    length: [u8; 2],
    checksum: [u8; 2],
}

struct UdpPacket {
    header: Ref<Bytes, UdpHeader>,
    body: Bytes,
}

impl UdpPacket {
    fn parse(bytes: Bytes) -> Option<UdpPacket> {
        let (header, body) = Ref::new_from_prefix(bytes)?;
        Some(UdpPacket { header, body })
    }
}

[joshlf]

We'll have a chance to look at this issue in more detail later, but briefly off the top of my head: check out how we handle this in 0.8. The ByteSlice trait has been split up, and we've made the resulting traits unsealed so that external crates can implement them.

[dignifiedquire]

and we've made the resulting traits unsealed so that external crates can implement them.

Unfortunately that doesn't help, as the orphan rule prevents us from implementing the trait for an external type :/

[matheus23]

It may be possible to workaround this via a newtype to experiment with this as a solution.

[somethingelseentirely]

The prevalence of bytes in the Rust ecosystem is extremely unfortunate, because the set of byte sources that it can operate over is essentially closed. The extensible approach used by minibytes from facebooks Sapling CMS and ownedbytes from the Tantivy fulltext search engine, is much better imho.

I've forked the somewhat dead minibytes into a new crate called anybytes.
It's still in early development but you might find the Packed types interesting. They provide read-only zerocopy views for scalars, slices and string over arbitrary byte sources (including bytes::bytes, ownedbytes::Bytes, Vec<T> where T: AsBytes, Box<T> where T: AsBytes, and memmap2::Mmap). Especially the latter are nice, because it allows you to not only read some (e.g.) memory mapped file as a PackedSlice<T>, but you can also just turn any existing Vec<T> into a Bytes/PackedSlice<T>.

[joshlf]

Filed upstream: tokio-rs/bytes#738

[dignifiedquire]

Looks like the last missing piece was published in 1.8: tokio-rs/bytes#741