Quack node #25
Quack node #25
Conversation
src/quack_planner.cpp
Outdated
| } | ||
| } | ||
|
|
||
| quackNode->custom_private = list_make2(db, preparedQuery.release()); |
There was a problem hiding this comment.
Should be in Quack_EndCustomScan
There was a problem hiding this comment.
But we release the unique_ptr here, so we are now responsible for freeing it
We place this in custom_private, which is never passed along to anything else or has free called on it.
There was a problem hiding this comment.
pointers from custom_private will be stored inside QuackScanState (Quack_CreateCustomScanState callback). Delete will be called to this pointers.
There was a problem hiding this comment.
End callback is called at the end of query execution.
|
Correct me if I'm wrong, but this is what I understand from this PR: We no longer do the ExecutorRunHook, so Postgres is now in charge of execution. Inside this node we run DuckDB with the given query, outputting to the What happens when our execution fails here? Can we fall back to Postgres or not? |
* Hook into postgres planner rather than on execution. Idea is to split
DuckDB execution into prepare/execute phase. If DuckDB prepare is not
valid will we fall back to normal postgres execution.
* Another benefit of having planner hook is that custom RETURN TABLE
FUNCTIONS can be used in PG syntax. This custom function should only
be created to pass parsing phase. Wee can now use `SELECT * FROM
read_parquet('...')` that will read parquet files through DuckDB.
* Created quack node which will be used for duckdb exection. This quack
nodes is defined as `Custom Scan Node`. EXPLAIN will work out of box
with this approach - we'll output just explain plan from DuckDB
execution.
* Added `httpfs` extension to be build together with parquer extension.
* Memory allocated columns needs to be released for each result tuple * Filter for TIMESTAMP
|
@Tishj You are right. We are now hooking immediately after parsing is done (in this way we can use read_parquet for example - which is declared as dummy PLSQL function). Execution is split between Logic is based on |
That is correct for the most part, but some InvalidInputExceptions or NotImplementedExceptions could still be reached during execution |
After |
No, to document what we discussed during the meeting, exceptions caused by the data can not be caught by preparing alone, and will arise only during execution. I was merely bringing up the fact that we can still hit an exception even if prepare succeeds, and thus can't fall back later to postgres execution/planning when that happens Checking |
Hook into postgres planner rather than on execution. Idea is to split DuckDB execution into prepare/execute phase. If DuckDB prepare is not valid will we fall back to normal postgres execution.
Another benefit of having planner hook is that custom RETURN TABLE FUNCTIONS can be used in PG syntax. This custom function should only be created to pass parsing phase. Wee can now use
SELECT * FROM read_parquet('...')that will read parquet files through DuckDB.Created quack node which will be used for duckdb exection. This quack nodes is defined as
Custom Scan Node. EXPLAIN will work out of box with this approach - we'll output just explain plan from DuckDB execution.Added
httpfsextension to be build together with parquer extension.Filter for DATE/TIMESTAMP types