Skip to content

Support streaming API to help avoid timeouts #3

Open
@simonw

Description

@simonw

I'm seeing a lot of timeouts where the openai_davinci() SQL function takes too long to load.

The streaming API - add "stream": true to the API JSON call - could be a neat way to avoid this... but it's obviously not compatible with running inside a SQL execution.

So how about a separate mechanism where you can define a SQL query that returns a prompt and Datasette then gives you a separate UI which executes that prompt in a streaming manner and streams it to your browser?

This might also provide a neat way to add features like rate-limiting, and to hide the API key while still allowing users to use it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions