extend rustfft to singlestore DB #49
Conversation
examples/rust/fft/bin/main.rs
Outdated
| @@ -0,0 +1,53 @@ | |||
| use rustfft::{FftPlanner}; | |||
There was a problem hiding this comment.
Can you add a comment here that just says that this file is a test driver for the module, and is intended to be compiled and run on the native platform?
There was a problem hiding this comment.
I was going to ask you more about this. It's seems a bit inefficient to compile this with the module. This can be extended further to generate Makefile test (seems to be simple enough if we are going with the make test approach, I'm not sure). However, the main point is this can be kept separately, maybe a generate_tests library to generate tests for this (and potentially other modules we will be writing) but I don't know if it's a good idea or the best way to do this.
There was a problem hiding this comment.
One way we can deal with this is to use the pattern here:
https://github.com/singlestore-labs/singlestoredb-extension-bloom-filters/blob/main/src/lib.rs
Using this approach, we can use Rust's built-in test framework and not require an additional file.
Note the conditional compilation of the interface at the top. For this purpose, you can just ignore the "handle" stuff.
There was a problem hiding this comment.
Either way, I'm not sure it's possible to completely remove the cognitive overhead of having native tests (other than not to include them).
examples/rust/fft/README.md
Outdated
| @@ -0,0 +1,36 @@ | |||
| # Prerequisite | |||
There was a problem hiding this comment.
Let's flesh this file out a little more. Here's a good template to follow:
https://github.com/singlestore-labs/singlestoredb-extension-bloom-filters
| ## Contents | ||
| This library provides the following database objects. | ||
|
|
||
| ### `st_planner_forward(len: u8, buffer: [Complex<f64>])` |
There was a problem hiding this comment.
Let's use the database types here. So, for this case it would be:
st_planner_forward(len TINYINT UNSIGNED NOT NULL, buffer RECORD(re DOUBLE NOT NULL, im DOUBLE NOT NULL)This is a bit long for the heading line, so maybe just put st_planner_forward in the heading and then put the UDF signature in the description.
| This is a TVF that will create a a new FFT algorthim instance for computing forward FFT of size `len`. Divides the `buffer` into chunks of size `len`, and computes FFT forward on each chunk. | ||
| This method will panic (on Rust side) if: | ||
| ``` | ||
| buffer.len() % len > 0 |
There was a problem hiding this comment.
Using database operators, this would be:
LENGTH(buffer) MOD len > 0| buffer.len() < len | ||
| ``` | ||
|
|
||
| ### `st_planner_inverse(len: u8, buffer: [Complex<f64>])` |
There was a problem hiding this comment.
Same comments as above in favor of using database syntax.
| ``` | ||
|
|
||
| ## Usage | ||
| The following is a simple example that creates two tables with a columns of strings. The first table's column is used to generate a Bloom Filter, which we store in a User Defined Variable. We then run the Bloom Filter on the strings in the second table. |
There was a problem hiding this comment.
I think you forgot to delete this paragraph :-)
|
|
||
| ## Usage | ||
| The following is a simple example that creates two tables with a columns of strings. The first table's column is used to generate a Bloom Filter, which we store in a User Defined Variable. We then run the Bloom Filter on the strings in the second table. | ||
| The following is a simple example that performs forward FFT on a vector `buffer` of two complex numbers `{"re": 1.0, "im": 2.5}` and `{"re": 2.0, "im": 2.5}`. This will divides the vector `buffer` into chunks of size `1` and computes a FFT on each chunk. |
There was a problem hiding this comment.
divides -> divide
computes -> compute
| The following is a simple example that creates two tables with a columns of strings. The first table's column is used to generate a Bloom Filter, which we store in a User Defined Variable. We then run the Bloom Filter on the strings in the second table. | ||
| The following is a simple example that performs forward FFT on a vector `buffer` of two complex numbers `{"re": 1.0, "im": 2.5}` and `{"re": 2.0, "im": 2.5}`. This will divides the vector `buffer` into chunks of size `1` and computes a FFT on each chunk. | ||
| ```sql | ||
| SELECT * FROM (st_process_forward(1, [ROW(1.0, 2.5), ROW(2.0, 2.5)])); |
There was a problem hiding this comment.
The outer parens aren't needed here.
No description provided.