Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partial parsing with JsonValue #157

Merged
merged 3 commits into from
Oct 31, 2024
Merged

partial parsing with JsonValue #157

merged 3 commits into from
Oct 31, 2024

Conversation

samuelcolvin
Copy link
Member

@davidhewitt please review this when you have a chance.

The weird recursive thing we do in value.rs makes this very ugly, but it seems to be working.

Copy link

codecov bot commented Oct 30, 2024

Codecov Report

Attention: Patch coverage is 88.94231% with 23 lines in your changes missing coverage. Please review.

Project coverage is 91.90%. Comparing base (0496e89) to head (a6d08f4).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
crates/jiter/src/value.rs 89.05% 6 Missing and 16 partials ⚠️
crates/jiter/src/errors.rs 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #157      +/-   ##
==========================================
+ Coverage   90.94%   91.90%   +0.96%     
==========================================
  Files          12       12              
  Lines        2042     2138      +96     
  Branches     2042     2138      +96     
==========================================
+ Hits         1857     1965     +108     
+ Misses        112      109       -3     
+ Partials       73       64       -9     
Files with missing lines Coverage Δ
crates/jiter/src/jiter.rs 92.85% <100.00%> (+0.06%) ⬆️
crates/jiter/src/python.rs 97.85% <100.00%> (+0.41%) ⬆️
crates/jiter/src/errors.rs 88.13% <75.00%> (-0.47%) ⬇️
crates/jiter/src/value.rs 87.90% <89.05%> (+6.19%) ⬆️

... and 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0496e89...a6d08f4. Read the comment docs.

Copy link

codspeed-hq bot commented Oct 30, 2024

CodSpeed Performance Report

Merging #157 will not alter performance

Comparing JsonValue-partial (a6d08f4) with main (0496e89)

Summary

✅ 73 untouched benchmarks

Copy link
Collaborator

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation... seems correct, but I think it could have been simpler.

Yeah, the recursion loop is a bit weird and complex (could probably be refactored better) but it does avoid recursive function calls at the benefit of bounded stack space (i.e. no segfaults from stack overflow) and higher performance 😂

Comment on lines +146 to +147
| JsonErrorType::ExpectedListCommaOrEnd
| JsonErrorType::ExpectedObjectCommaOrEnd
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder, how does this influence cases of invalid JSON like

[1, 2, 3,,, 4, 5,  // do we get [1, 2, 3] ?

{"a": 1,, "b": 2. // do we get {"a": 1} ?

peek = next_peek;
continue;
}
Err(e) if !(allow_partial && e.allowed_if_partial()) => return Err(e),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than having every possible ? replaced in this way, I feel like there might have been a solution where the whole loop body goes into a sub-function, and then can match once on a possible error and if it's a partial end, wind up the recursion stack to build the partial datastructure.

But I'm a bit tired to think in more detail or push a useful commit to that effect tonight.

Copy link
Collaborator

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to proceed, also ok if you want to merge this and leave a TODO somewhere for me to follow up another time.

@samuelcolvin samuelcolvin enabled auto-merge (squash) October 31, 2024 17:22
@samuelcolvin samuelcolvin merged commit e1d4c9f into main Oct 31, 2024
23 checks passed
@samuelcolvin samuelcolvin deleted the JsonValue-partial branch October 31, 2024 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants