Skip to content

Commit

Permalink
fix(temporary): update LLM request content type to multipart/form-data (
Browse files Browse the repository at this point in the history
#665)

This commit applies a patch that temporarily updates LLM request type to `multipart/form-data` to make the API spec work. It also add warning and note regarding streaming responses.
  • Loading branch information
ad-astra-video authored Oct 2, 2024
1 parent f717793 commit 6c3f78e
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 1 deletion.
2 changes: 1 addition & 1 deletion ai/api-reference/gateway.openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -767,7 +767,7 @@ paths:
operationId: genLLM
requestBody:
content:
application/x-www-form-urlencoded:
multipart/form-data:
schema:
$ref: '#/components/schemas/Body_genLLM'
required: true
Expand Down
17 changes: 17 additions & 0 deletions ai/api-reference/llm.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,24 @@
---
openapi: post /llm
---
<Warning>
We are currently deploying the Large Language Model (LLM) pipeline to our gateway infrastructure.
This warning will be removed once all listed gateways have successfully transitioned to serving the LLM pipeline, ensuring a seamless and enhanced user experience.
</Warning>
<Note>
The LLM pipeline supports streaming response by setting `stream=true` in the request. The response is then streamed with Server Sent Events (SSE)
in chunks as the tokens are generated.

Each streaming response chunk will have the following format:

`data: {"chunk": "word "}`

The final chunk of the response will be indicated by the following format:

`data: {"chunk": "[DONE]", "tokens_used": 256, "done": true}`

The Response type below is for non-streaming responses that will return all of the response in one
</Note>
<Info>
The default Gateway used in this guide is the public
[Livepeer.cloud](https://www.livepeer.cloud/) Gateway. It is free to use but
Expand Down

0 comments on commit 6c3f78e

Please sign in to comment.