Open
Description
I'm seeing a lot of timeouts where the openai_davinci()
SQL function takes too long to load.
The streaming API - add "stream": true
to the API JSON call - could be a neat way to avoid this... but it's obviously not compatible with running inside a SQL execution.
So how about a separate mechanism where you can define a SQL query that returns a prompt and Datasette then gives you a separate UI which executes that prompt in a streaming manner and streams it to your browser?
This might also provide a neat way to add features like rate-limiting, and to hide the API key while still allowing users to use it.