Use of sorting_columns
parquet metadata
#18785
Labels
bug
Something isn't working
needs triage
Awaiting prioritization by a maintainer
python
Related to Python Polars
Checks
Reproducible example
Then checking the parquet metadata using pyarrow:
Log output
No response
Issue description
It seems that when sinking/writing frames to parquet, polars isn't able to save the sorting columns metadata, whereas e.g. pyarrow can.
As a side-note, I have trouble figuring out if/when polars would use this information (wether directly from the parquet metadata or by the use of
set_sorted()
?Expected behavior
Efficiently write/read the
sorting_columns
parquet metadata to speed-up further processingInstalled versions
The text was updated successfully, but these errors were encountered: