Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vector disk buffer pre-maturely ack to source #19599

Open
anil-db opened this issue Jan 11, 2024 · 2 comments
Open

Vector disk buffer pre-maturely ack to source #19599

anil-db opened this issue Jan 11, 2024 · 2 comments
Labels
domain: buffers Anything related to Vector's memory/disk buffers type: bug A code related bug.

Comments

@anil-db
Copy link
Contributor

anil-db commented Jan 11, 2024

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

When using disk buffer, record.encode in writer.ts moves the record and drop it causing acknowledgement to happen to source.
but record is not written to disk so such ack are premature.

record should not dropped here and same record should be returned incase of any error happened rather than decoded one.

Configuration

No response

Version

35.0

Debug Output

No response

Example Data

No response

Additional Context

To repro we need to create the situation where disk write fail after encoding of record, which is probably easier to in a unit test.

References

No response

@anil-db anil-db added the type: bug A code related bug. label Jan 11, 2024
@anil-db anil-db changed the title Vector disk buffer pre-maturely ack to event Vector disk buffer pre-maturely ack to source Jan 11, 2024
@jszwedko jszwedko added the domain: buffers Anything related to Vector's memory/disk buffers label Jan 12, 2024
@jszwedko
Copy link
Member

Thanks for filing this. I would like to see a test (agreed a unit test would be good) that shows the failure.

@anil-db
Copy link
Contributor Author

anil-db commented Jan 18, 2024

Thanks for filing this. I would like to see a test (agreed a unit test would be good) that shows the failure.

I created this PR with unit test #19650 which shows the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: buffers Anything related to Vector's memory/disk buffers type: bug A code related bug.
Projects
None yet
Development

No branches or pull requests

2 participants