BytesMut should implicitly grow the internal storage in its BufMut implementation.

[carllerche]

This will bring the implementation in line with that of Vec. This is a breaking change.

Relates to #131, #77.

[cramertj]

Can you explain why this is a breaking change? Are users relying on the fact that the buffer does not grow implicitly?

[carllerche]

Yes, users (well, I am at the very least) are relying on the fact that the buffer does not grow implicitly.

[stbuehler]

I'm very much in favor of this; or in providing another type (BytesExt) or a newtype wrapper (see below) that defaults to extend the allocation by default.

Even with #199 fixed there are still some places where not extending is imho neither expected nor useful:

• Extend: the name already suggests it should allocate (Is it safe to use size_hint() in Extend implementation? #159 discussed whether it is "safe"; yes, but...)
• std::fmt::Write: estimating the required space is hard with utf8 Strings and so on

Maybe ButMut could also solve this by adding another trait BufMutReserve that provides an abstraction to reserve more space if the BufMut implementation for the type doesn't allocate itself (i.e. the implementation for Vec<u8> would for now be a noop), and a newtype wrapper for any BufMut+BufMutReserve, for which the BufMut implementation would automatically reserve the needed space.

[carllerche]

If this is functionality that you require immediately, could you define a wrapper type for now?

[BusyJay]

I'm in favor with the reserve function, which leave more flexibility for users to control how the capacity grows.

[mzabaluev]

It looks as though there may be a need for two traits serving the extending and non-extending use cases. If there is really just one use case, the preferred behavior should be fixed in the contract of BufMut and all implementations should be made conformant.

I'm in favor with the reserve function, which leave more flexibility for users to control how the capacity grows.

If slices, Cursor, and other non-extendable types are to keep implementing BufMut, such reserve method needs to be fallible, right?

[mzabaluev]

For the implicitly allocating case, what I would like to have is a type that (to largely reuse the current code) wraps BytesMut, implements std::fmt::Write and all the put* facilities of BufMut but in a grow-as-needed fashion, internally forming a sequence of Bytes chunks with take() from the underlying buffer as it gets filled up, in order to avoid copying reallocations. The chunks can then be handed off in a Buf for iovec output or iterated over otherwise.

This could allow simple implementations of non-reallocating structured/framed/serde sinks in a way that current Tokio interfaces do not provide.

[BusyJay]

Any updates on this?

[carllerche]

Still scheduled for 0.5 :)

[mzabaluev]

For the implicitly allocating case, what I would like to have is a type that (to largely reuse the current code) wraps BytesMut, implements std::fmt::Write and all the put* facilities of BufMut but in a grow-as-needed fashion, internally forming a sequence of Bytes chunks with take() from the underlying buffer as it gets filled up, in order to avoid copying reallocations. The chunks can then be handed off in a Buf for iovec output or iterated over otherwise.

To follow up on this, here is ChunkedBytes that I wrote as part of a larger project: https://github.com/mzabaluev/tokio-text/blob/master/src/chunked_bytes.rs

I think something like this would be more efficient than serializing into a contiguous growing BytesMut.