You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've merge three dask dataframes and tried to save the resulting dataframe back to parquet.
Our writing approach uses pyarrow. I run into the following issue: ArrowInvalid: offset overflow while concatenating arrays
I was able to avoid the issue by decreasing the size of the dataset or using fastparquet.
Requirements:
pyarrow==13.0.0
The text was updated successfully, but these errors were encountered:
I've merge three dask dataframes and tried to save the resulting dataframe back to parquet.
Our writing approach uses
pyarrow
. I run into the following issue:ArrowInvalid: offset overflow while concatenating arrays
I was able to avoid the issue by decreasing the size of the dataset or using
fastparquet
.Requirements:
pyarrow==13.0.0
The text was updated successfully, but these errors were encountered: