Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly handle partial JSON at the end of a chunk #104

Merged
merged 2 commits into from
Nov 14, 2023

Conversation

PallavAg
Copy link
Contributor

What

Streaming a response occasionally encounters the error "The given data was not valid JSON."
Related Issues: #97 #103

Why

The end of a chunk can have an incomplete JSON.
The rest of the JSON will come through at the beginning of the next chunk.

Affected Areas

StreamingSession.swift

More Info

This change stores the last JSON from the chunk and appends it to the beginning of the next chunk.

A likely partial JSON is stored when:

  1. ResultType decoding fails
  2. APIErrorResponse decoding fails
  3. The JSON is last in the chunk

Having an incomplete JSON at the end of a chunk is seemingly random - likely based on OpenAI server load etc.
A request that surfaces this issue more often than not is: "Write 1 to 40 using bullet points"

// Sample incomplete chunk
{"id":"chatcmpl-893wTs9tI87Yt0t6moRbyhXT2xn93","object":"chat.completion.chunk","created":1697170797,"model":"gpt-4-0613","choices":[{"index":0,"delta":{"content":"."},"fi

Kudos to @jcmourey for suggesting the fix in #97

@PSchmiedmayer
Copy link

Nice, great addition @PallavAg, we run experience the same issue as noted in #103, this looks like a nice and useful fix, makes sense that it is not guaranteed that each streaming package contains a fully contained JSON.

Would be amazing if this can be merged and added in a next release.

nathanborror added a commit to nathanborror/swift-openai that referenced this pull request Oct 31, 2023
@ingvarus-bc
Copy link
Contributor

Neat! 🔥
Amazing addition to the project. What do you think about moving all json handling outside of urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) to a separate method for the sake of testability and more obvious look?

@ingvarus-bc ingvarus-bc self-requested a review November 9, 2023 16:43
Copy link

sonarcloud bot commented Nov 13, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

No Coverage information No Coverage information
0.0% 0.0% Duplication

@ingvarus-bc
Copy link
Contributor

@PallavAg, hi, I just moved json processing out, what do you think?

@ingvarus-bc ingvarus-bc merged commit d9192df into MacPaw:main Nov 14, 2023
5 checks passed
@ingvarus-bc ingvarus-bc mentioned this pull request Nov 14, 2023
@prateekshac1
Copy link

prateekshac1 commented Jan 31, 2024

It looks like this issue has started to happen again. Have noticed the intensity of this go up in the past 2-3 days. I'm using the latest version of OpenAI 0.2.5 which has this fix. But yet many of my chatsStream(query: query) queries are erroring out with this error

"The data couldn’t be read because it isn’t in the correct format. dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "Unable to parse empty data." UserInfo={NSDebugDescription=Unable to parse empty data.})))"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants