fix(codecs): Ensure that batches using newline delimited framing end in a newline

[jszwedko]

Fixes: #21086

Signed-off-by: Jesse Szwedko [email protected]

[datadog-vectordotdev]

Datadog Report

Branch report: jszwedko/add-batch-trailing-newline
Commit report: aa6e9ef
Test service: vector

❌ 6 Failed (0 Known Flaky), 334 Passed, 0 Skipped, 4m 18.73s Total Time

❌ Failed Tests (6)

This report shows up to 5 failed tests.

• sinks::nats::integration_tests::nats_tls_client_cert_valid - vector - Details

  Expand for error

   Test failed, but output was not captured
  

• sinks::nats::integration_tests::nats_tls_jwt_auth_valid - vector - Details

  Expand for error

   Test failed, but output was not captured
  

• sinks::nats::integration_tests::nats_tls_valid - vector - Details

  Expand for error

   Test failed, but output was not captured
  

• sources::nats::integration_tests::nats_tls_client_cert_valid - vector - Details

  Expand for error

   Test failed, but output was not captured
  

• sources::nats::integration_tests::nats_tls_jwt_auth_valid - vector - Details

  Expand for error

   Test failed, but output was not captured
  

[neuronull]

It does feel a bit strange that this is only coming up now 🤔

[jszwedko]

It does feel a bit strange that this is only coming up now 🤔

Agreed. I think most consumers would handle newline or EOF as the delimiter though so needing the trailing newline may not be a common need. I think it is more "correct" though and less likely to cause weird issues if EOFs aren't handled as delimiters.

[jszwedko]

I think ideally the encoding, for newline delimited framing, would append the newline while encoding each element (including the last) rather than adding it when finalizing the batch, but that was a bit more refactoring than I was willing to take on at the moment.

[neuronull]

LGTM