Length-delimited framing actually emitting big-endian length, not little-endian

[blind-oracle]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

The docs for framing in sinks say:

length_delimited

Event data is prefixed with its length in bytes.
The prefix is a 32-bit unsigned integer, little endian.

Where in fact this framing emits Big Endian-encoded integer.
Documentation needs to be fixed.

See here, the tokio_util::codec::LengthDelimitedCodec codec is used:

vector/lib/codecs/src/encoding/framing/length_delimited.rs

Line 25 in 7cf2f00

pub struct LengthDelimitedEncoder(LengthDelimitedCodec);

And this codec emits big-endian: https://docs.rs/tokio-util/latest/tokio_util/codec/length_delimited/#getting-started

The returned transport implements Sink + Stream for BytesMut. It encodes the frame with a big-endian u32 header denoting the frame payload length:

I've confirmed it when I was writing a server to receive Vector events.

Configuration

socket:
    type: socket
    inputs:
    - parse_logs
    
    mode: unix
    path: /tmp/foo.socket

    encoding:
      codec: text
    
    framing:
      method: length_delimited
    
    healthcheck:
      enabled: false

Version

0.35

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response