Blog post with DataFusion July - Sep 2024

[alamb]

Is your feature request related to a problem or challenge?

We have had good luck writing up quarterly updates for DataFusion, most recently:
https://datafusion.apache.org/blog/2024/07/24/datafusion-40.0.0/

See #9602

Describe the solution you'd like

Blog post

Describe alternatives you've considered

No response

Additional context

No response

[alamb]

Here is my wishlist for things to write about in the next blog:

• Charts showing the speedup from [Epic] Implement support for StringView in DataFusion #10918 (from @XiangpengHao @Weijun-H @PsiACE and others)
• Charts showing improvements realted to aggregation improvements (e.g. Skipping partial aggregation when it is not helping for high cardinality aggregates #11627 etc with @jayzhan211 and @korowa )
• Something about Substrait (thanks to @Blizzara @dharanad and others) -- is there any big milestone we can claim?
• Maybe something about better MAP type support that @goldmedal and others have been working on

Also, of course, I would love to have more help writing a blog (maybe someone else could draft it 🤔 🎣 )

[dharanad]

@alamb Thank you for considering me, but I think there may be some confusion - I wasn't involved in the work on Substrait. However, I'd be happy to contribute to a blog post on MAP once I've completed adding support for Arrays in #11436

[alamb]

@alamb Thank you for considering me, but I think there may be some confusion

Yes I was probably confused -- sorry about that

[Blizzara]

@alamb for Substrait - maybe the work @Lordworms has been doing to add the TPC-H tests would be good at least? From my side, I don't know if there's any precise milestone as such - but maybe something around supporting VirtualTables, more literals and types, better interoperability with other substrait producers. (I do hope to write a separate blog post from our perspective if/when I've proven the whole setup I'm working on works and is faster, but we're not there yet unfortunately.)

[alamb]

Blog with #11627 performance high cardinality aggs / partial skipping

[alamb]

It would also be cool to discuss efforts for chunked emission #11943 for (more) aggregage performance

[alamb]

My plan for this is that we will finish up enabling string view and then make that performance improvement be the headline for this post