Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RD-15410: DAS function support #25

Merged
merged 1 commit into from
Mar 28, 2025
Merged

Conversation

bgaidioz
Copy link

@bgaidioz bgaidioz commented Mar 14, 2025

(Goes with raw-labs/multicorn-das#28)

This PR adds DAS function support to multicorn by introducing:

  • A new C function entry point multicorn_function_execute, which can be invoked
    from SQL with a set of parameters that includes FDW-like options, named
    argument strings, and the actual function parameters.
  • A bridging helper function das_function_execute that:
    • Converts options and arguments into Python dictionaries,
    • Dynamically imports and calls the Python module/class (multicorn_das.DASFunction.execute_static) for DAS function execution,
    • Then converts the Python result back into a PostgreSQL Datum.
  • An auxiliary function build_dummy_attinmeta(Oid retType) for creating a
    single-column AttInMetadata structure, used when converting the returned
    Python object to a Datum of the indicated retType. If the return type is an
    array, it sets attndims=1 to handle array‐like results.

A fix:

  • INTERVAL was read from a dict representation, that isn't anything standard (it is the Python object generated by multicorn-das), to turn it into an ISO string which Postgres can parse and import. If multicorn-das generates that a Python ISO string instead of that dictionary, then importing an interval is simplified: we can pass the string to Postgres.
  • That implementation also helps with extracting fields from a JSONB returned by a DAS function: if the JSONB contains the JSON representation of the original Python dict, a user wouldn't easily translate it into a Postgres INTERVAL, while if it comes as an ISO string, value::interval works.

multicorn_function_execute

Runs the logic to execute a DAS function statically.

  • It receives at least two arguments:
    • a text[] of options, which permits to identify which DAS and function to execute,
    • and a text[] of parameter names, which is used eventually to build the named arguments as a Python dictionary,
    • plus N further actual arguments for the DAS function.
  • The argument/return types are retrieved from pg_proc, in order to calls foreign_function_execute, that bridges to Python.

foreign_function_execute

It bridges the SQL function to Python:

  • Converts FDW options to a Python dict.
  • Converts named function arguments to another dict.
  • Imports multicorn_das.DASFunction and calls execute_static(...).
  • Uses pyobjectToDatum(...) to transform the Python result to a Postgres Datum.

build_dummy_attinmeta

Converting the function result to Datum reuses the existing Python conversion
code that runs for tables. Because that code takes its conversion information
from column descriptions, we add build_dummy_attinmeta, a function that
returns a single‐attribute AttInMetadata with the given type for the required
text-based conversion that are already in place for column values.

@bgaidioz bgaidioz force-pushed the RD-15410-das-functions-support branch from 92c4bf1 to baff6f4 Compare March 28, 2025 09:22
@bgaidioz bgaidioz merged commit d526aef into main Mar 28, 2025
@bgaidioz bgaidioz deleted the RD-15410-das-functions-support branch March 28, 2025 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants