Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync UDFs: Unexpected Requirement for Parameter #699

Closed
hazelnut-99 opened this issue Jul 31, 2024 · 0 comments · Fixed by #753
Closed

Sync UDFs: Unexpected Requirement for Parameter #699

hazelnut-99 opened this issue Jul 31, 2024 · 0 comments · Fixed by #753

Comments

@hazelnut-99
Copy link

UDF Definition

/*
[dependencies]
chrono = "0.4"
*/
 
use arroyo_udf_plugin::udf;
use chrono::{Utc, Duration};
 
#[udf]
fn date_id_with_delta(delta: i64) -> String {
    let now = Utc::now();
    let result_date = now + Duration::days(delta);
    result_date.format("%Y%m%d").to_string()
}

Query (table is a Kafka source)

SELECT a, b, date_id_with_delta(-1) AS date_str
FROM table

An error is thrown after the first row is processed:

ERROR arroyo_server_common: panicked at crates/arroyo-worker/src/arrow/mod.rs:67:31:
should be able to compute batch: ArrowError(InvalidArgumentError("all columns in a record batch must have the same length"), None) panic.file="crates/arroyo-worker/src/arrow/mod.rs" panic.line=67 panic.column=31

When changing the UDF to include an extra parameter that isn’t used:

/*
[dependencies]
chrono = "0.4"
*/
 
use arroyo_udf_plugin::udf;
use chrono::{Utc, Duration};
 
#[udf]
fn date_id_with_delta(x: f64, delta: i64) -> String {
    let now = Utc::now();
    let result_date = now + Duration::days(delta);
    result_date.format("%Y%m%d").to_string()
}

And adjust the query to:

SELECT a, b, date_id_with_delta(x, -1) AS date_str
FROM table

The query runs without any issues.

@hazelnut-99 hazelnut-99 changed the title Sync UDFs: Unexpected Requirement for Input Table Parameter Sync UDFs: Unexpected Requirement for Parameter Jul 31, 2024
@mwylde mwylde closed this as completed in 1d95e13 Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant