Add wait mode for cli

[blackrez]

I don't know how (and if) it's possible, but it would be interesting to have a kind of wait mode. My use case is use duckdb with gcp cloud run, a serverless docker service. I imagine something like
duckdb :memory: "LOAD httpserver; SELECT httpserve_start('0.0.0.0', 9999, wait=true); And duckdb wait until a signal kill the process.

[lmangani]

Hey @blackrez the server process gets killed if the parent exists, which should apply find to the DuckDB CLI terminating.
Could you explain in more detail what the wait mode should provide and/or show the current limitation? Let's make it work!

[blackrez]

When I launched my container, the duckdb process will stop immediately after the function was call.
What I want is something that wait until a signal is send.

[lmangani]

Currently the serve function runs in a separate thread and returns to the DuckDB prompt as if a query succeeded with results, which I suppose causes your serverless execution to terminate. To have a blocking approach the server should run in the main thread and that's possible, but not sure desirable or free from side effects.... if you want you can try this locally: #8

Note: https://github.com/metrico/quackpipe does this job better at the moment if you need a serverless execution

[blackrez]

This is exactly what I need (and a lot of people I think).

[lmangani]

Are you referring to Quackpipe or the PR? If you're referring to the PR, we can add this in. The foreground mode works, but before merging we really need to work on proper thread termination to avoid data loss, corruption, etc.

[blackrez]

Alright we can add this in. The foreground mode works, but before merging we really need to work on proper thread termination to avoid data loss, corruption, etc.Message ID: ***@***.***>

Stupid question, but is duckdb_cli already handle all that situations ?

[lmangani]

DuckDB CLI is not DuckDB and our code runs in a separate thread inside an extension which means we have to make sure the states are propagated, etc. Which I'm no expert of but with this said, I've added the missing code to the branch and it seems to behave as expected if you wanna try it locally:

> git clone https://github.com/lmangani/duckdb-extension-httpserver
> cd duckdb-extension-httpserver
> git submodule update --init --recursive
> git checkout thread-options

# build the branch and set the foreground ENV
> GEN=ninja make
> export DUCKDB_HTTPSERVER_FOREGROUND=1

# test it
> ./build/release/duckdb

[lmangani]

@blackrez does the implementation match your usecase?

[blackrez]

@lmangani it matches with my usecase.

[lmangani]

merged, will be part of next release (no WIN32 support)