Weekly Plan: Andrew Lamb 2025-05-12

[alamb]

This is my plan this week for reviews, etc. I am putting it here to make it visible and keep myself organized

• Arrow Filter Performance: Complete ClickBench benchmark: Add arrow_reader_clickbench benchmark arrow-rs#7470
• Arrow Filter Performance: review / start merging improvements to clickbench derived benchmark with @zhuqi-lucas
• Arrow Variant: Create parquet-variant create skeleton PR : [Variant] Add (empty) parquet-variant crate, update parquet-testing pin arrow-rs#7485
• Arrow Variant: work with @PinkCrow007 to merge parts of Add API for Creating Variant Values arrow-rs#7452 from @PinkCrow007
• DataFusion: Metadata Handling / extension types from @timsaucer: feat: metadata handling for aggregates and window functions #15911
• DataFusion Feature: async user defined functions with @goldmedal Introduce Async User Defined Functions  #14837
• Arrow Bug with concat'ing dictionaries from @davidhewitt: support merging primitive dictionaries in interleave and concat arrow-rs#7468
• DataFusion perf script from @logan-keede : Shell script to collect benchmarks for multiple versions #15144
• DataFusion perf script draft: feat(benchmark): collect benchmarks for last 5 versions in line protocol format #15846
• DataFusion PR about pruning ordering: pipe column orderings into pruning predicate creation #15821
• DataFusion Sorting: test: add fuzz test for doing aggregation with larger than memory groups and sorting with limited memory #15727
• Arrow Dictionary ID next steps from @brancz: Remove dict_id from schema and make it an IPC concern only arrow-rs#7467
• Arrow Variant: Expose tape decoder Add custom decoder in arrow-json arrow-rs#7442

Nice to have (really would be great to have someone help review):

• DataFusion: Aggregate UDFs in FFI: feat: Add Aggregate UDF to FFI crate #14775
• Arrow: Avro cleanup: Avro codec enhancements arrow-rs#6965
• Arrow: Avro Utf8View: Support Utf8View for Avro arrow-rs#7434