Skip to content

Can we use compression on Parquet writing? #6061

Closed Answered by tustvold
metesynnada asked this question in Q&A
Discussion options

You must be logged in to vote

Parquet natively supports block level compression - see https://github.com/apache/parquet-format/blob/master/Compression.md

Compressing the file as a whole would lose the ability to do meaningful IO pushdown, and would likely yield minimal size savings, so I'm not aware of anyone doing this

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by metesynnada
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants