Open
Description
from https://jira.apache.org/jira/browse/SPARK-23908
SELECT transform(ARRAY [], x -> x + 1); -- []
SELECT transform(ARRAY [5, 6], x -> x + 1); -- [6, 7]
SELECT transform(ARRAY [5, NULL, 6], x -> COALESCE(x, 0) + 1); -- [6, 1, 7]
SELECT transform(ARRAY ['x', 'abc', 'z'], x -> x || '0'); -- ['x0', 'abc0', 'z0']
SELECT transform(ARRAY [ARRAY [1, NULL, 2], ARRAY[3, NULL]], a -> filter(a, x -> x IS NOT NULL)); -- [[1, 2], [3]]
I'm thinking something like
df %>%
transform(nested_col, y = x -> x + 1)
This might need revision though. Not sure what the most intuitive way to specify the output column name will be. Also undecided w.r.t. NSE or not