Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adds check to ensure consolidated=True is not passed to open_dataset #381

Closed
wants to merge 2 commits into from
Closed

Adds check to ensure consolidated=True is not passed to open_dataset #381

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7c982c6
added check to ensure consolidated=True is not passed to open_dataset
norlandrhagen Oct 19, 2023
a60d5f7
removed duplicate
norlandrhagen Oct 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion kerchunk/xarray_backend.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,12 @@ def open_reference_dataset(
if open_dataset_options is None:
open_dataset_options = {}

if open_dataset_options.get("consolidated"):
raise ValueError(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A warning would do? Without making this check, it only results in a warning from the zarr engine anyway I think. Note that parquet references do work as consolidated or not, it doesn't matter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah good to know! Happy to close the PR.

"consolidated = True is not supported when opening Kerchunk reference files through Xarray."
)
open_dataset_options["consolidated"] = False

m = fsspec.get_mapper("reference://", fo=filename_or_obj, **storage_options)

return xr.open_dataset(m, engine="zarr", consolidated=False, **open_dataset_options)
return xr.open_dataset(m, engine="zarr", **open_dataset_options)
Loading