Canonical PRQL go library

[max-sixty]

Hi there, from the PRQL project! This looks great, thanks for putting it together.

I don't have much context on the best approach for PRQL & Go, having not written much Go. Our current version is not easy to use — you can't just install a go dependency like folks can with python or JS.

So to the extent this is a better approach, we're happy to pursue that as the canonical approach for folks using go; let us know if you have thoughts.

Thanks!

[pims]

Hi @max-sixty,

⚠️ Disclaimer: I don't have much more context on the best approach to PRQL & Go either.

It is my understanding that the Go community is, in general, a bit averse to using CGO when there are alternatives. https://dave.cheney.net/2016/01/18/cgo-is-not-go is a good summary of the complexity added by using CGO.

That said, there are quite a few successful Go libraries that use CGO, and it wouldn't take much to take what's already there in the prql-lib crate and make it a standalone Go library.

An alternative, is to use WebAssembly as some kind of FFI.
Wazero is the only Go Wasm runtime that doesn't use CGO afaik, which plays to all of Go's strength (static binary file, easy cross compilation, etc).

For the initial version, I went with using Wasi as the guest/host interface, as it is straightforward to implement on the guest side (Rust), especially for a Rust beginner like me.

With a larger API surface (Compile, ToJSON, ToYAML, etc.) it makes more sense to not use WASI and default to standard Wasm.

It would look very similar to what's already in prql-lib, but with the wasm32 target_arch.

#[cfg_attr(all(target_arch = "wasm32"), export_name = "compile")]
#[no_mangle]
pub unsafe extern "C" fn _compile(ptr: u32, len: u32) {
    compile(&ptr_to_string(ptr, len));
}

• the required alloc/dealloc methods.

In my early experiments, the dealloc function was very slow compared to the simpler Wasi interface. Being very new to Rust, I was probably doing something wrong, so it's worth revisiting.

Regardless of the approach chosen, I don't have a good sense as to how the wasm binary should be published/integrated in the Go package. I went the manual route here, but for anything serious, a more sustainable approach is probably required.

As a next step, I'll make another attempt at using a standard Wasm binary (non Wasi) and tag you on the relevant PRs.
I'll try to allocate some time over the next week to this if that sounds like a good plan.

Thanks for reaching out!

[max-sixty]

Awesome @pims !

Interesting tradeoff re cgo vs WASM.

Not specific to this project, but I had been hoping that WASM would be good at this sort of thing — allowing easy™ interop with a small-n performance hit. So great if prql-go can be an example of that.

Lmk anything you need from our side. If you do get this done, happy to update docs & the website to point here for the Go bindings.