Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: parse JSON with UTF-8 BOMs #87

Merged
merged 2 commits into from
Mar 14, 2024
Merged

fix: parse JSON with UTF-8 BOMs #87

merged 2 commits into from
Mar 14, 2024

Conversation

mistydemeo
Copy link
Contributor

While many JSON parsers can read UTF-8 data that begins with a byte order mark, json-serde can't. This caused an error for us when reading data that originated from PowerShell on Windows, which always writes a byte order mark at the beginning of the file even when using UTF-8.

I've added a test for this and confirmed the new test fails without the new changes.

See: serde-rs/json#1115

@mistydemeo mistydemeo requested a review from Gankra March 14, 2024 21:14
src/source.rs Outdated Show resolved Hide resolved
While many JSON parsers can read UTF-8 data that begins with a byte
order mark, json-serde can't. This caused an error for us when reading
data that originated from PowerShell on Windows, which always writes
a byte order mark at the beginning of the file even when using UTF-8.
@mistydemeo mistydemeo merged commit 34f3306 into main Mar 14, 2024
14 checks passed
@mistydemeo mistydemeo deleted the strip_json_bom branch March 14, 2024 21:50
@mistydemeo mistydemeo mentioned this pull request Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants