Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ParseStream "eating" bytes after json body #2294

Open
diehard2 opened this issue Jul 26, 2024 · 0 comments
Open

ParseStream "eating" bytes after json body #2294

diehard2 opened this issue Jul 26, 2024 · 0 comments

Comments

@diehard2
Copy link

diehard2 commented Jul 26, 2024

I recently updated from an old version of RapidJSON and I'm seeing some unexpected behavior. I've attached the minimal reproducer below. ParseStream is consuming some bytes of the stream that I don't want it to consume. Since this is a non-seekable stream, I can't move the stream back a byte or two.

Minimal reproducer

For this payload

[{"nodes":[{"node-path":"foo","content":1}]}]
--test

and this code, which reads a json header while ideally not advancing the buffer

#include <rapidjson/document.h>
#include <rapidjson/istreamwrapper.h>
#include <rapidjson/reader.h>
#include <rapidjson/stringbuffer.h>
#include <rapidjson/writer.h>

int main()
{
  std::ifstream file("mini.txt");
  rapidjson::IStreamWrapper isw(file);
  rapidjson::Document doc;
  doc.ParseStream<rapidjson::kParseStopWhenDoneFlag>(isw);

  // Create a JSON string buffer
  rapidjson::StringBuffer jsonBuffer;
  rapidjson::Writer<rapidjson::StringBuffer> writer(jsonBuffer);
  doc.Accept(writer);

  // Print the JSON string
  std::cout << jsonBuffer.GetString() << std::endl;

  print_istream(file);
  std::cout << "\n";

  return 0;
}

I see different behavior after June 2020

Before June 2020 (expected)

[{"nodes":[{"node-path":"foobar","content":1}]}]

--test

After June 2020

[{"nodes":[{"node-path":"foobar","content":1}]}]
test

interestingly, the number consumed appears dependent on the size of the json

pre 2020

[{"nodes":[{"node-path":"foo","content":1}]}]

--test

post 2020

[{"nodes":[{"node-path":"foo","content":1}]}]
-test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant