Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Json parsing is extremely slow #2

Open
pradeepg26 opened this issue May 10, 2014 · 0 comments · May be fixed by #3
Open

Json parsing is extremely slow #2

pradeepg26 opened this issue May 10, 2014 · 0 comments · May be fixed by #3

Comments

@pradeepg26
Copy link

In using the JsonSerde, we noticed that it was extremely slow at parsing our log files. We were spending about 40-45 min to parse about 5GB of uncompressed data (140MB xz -6 compressed).
This was especially odd since we could read a file fully (with JsonPath parsing) using custom code in less than 2 min on a single machine using 4 threads.

This obviously was not usable for us since we have more than 1 PB of uncompressed data.

Primary problem is that this library is using a very old version of JsonPath.

@pradeepg26 pradeepg26 linked a pull request May 10, 2014 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant