-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
handling huge data #1
Comments
Thank you for the full problem description.
That is ugly, we will add proper restore on error.
You could try to use
In jsreport v3, we solve the sandboxing and cross process communication differently. |
Thank you for you response
👍
Thx for you response ! I forgot that server strategy implies serialization. I have read again https://jsreport.net/blog/running-scripts-and-helpers-in-main-process The gain of performance with in-process strategy is appealing, but we are reluctant to do rendering in the main process : we have some reports wich are kind of user-controlled and even with developers reports, we are never sure of the size of the data processed.
Great news ! |
What nodejs version you have? I see the string size limit was increased in v9 to 1GB.
I see... The You can fork or monkey patch this and experiment with chunks serialization. We need to concentrate on v3 now so we aren't going to invest into this direction. |
You're right, it's definitely not reasonable to have >1Gb data in a string...
Yes I thought about it. Another idea would be to deploy 2 different servers :
ok, I understand |
Hi and thx for the module jsreport, it's very useful
We rely on jsreport in production heavily and we often stumble on this error :
which originates from
serializator/index.js
Line 8 in 8344812
which is caused by a huge amont of data to render, which probably exceed the maximum length allowed in nodejs (which itself is limited by V8, presumably here https://github.com/nodejs/node/blob/eb24573987b27ad516b5bc86bfb320660540e2bf/deps/v8/include/v8.h#L3004-L3007)
Consequences :
How do you think things could be improved ?
Of course we have tried to change business logic, to reduce the size of the data being rendered, but we have reached a limit now.
Do you think it could be possible to stringify by chunks ?
eg semthing like https://dev.to/madhunimmo/json-stringify-rangeerror-invalid-string-length-3977 ?
Also try/catch line 29 to be sure to restore prototype method in all cases ?
Thx by advance for your ideas
The text was updated successfully, but these errors were encountered: