Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zarr to Zarr capabilites? #501

Closed
kbren opened this issue Mar 14, 2023 · 3 comments
Closed

Zarr to Zarr capabilites? #501

kbren opened this issue Mar 14, 2023 · 3 comments

Comments

@kbren
Copy link

kbren commented Mar 14, 2023

I'm interested in using pangeo-forge-recipes to do pre-processing and re-chunking of various large Zarr datasets that are already on GCP. In the documentation for the XarrayZarrRecipe class (https://pangeo-forge.readthedocs.io/en/latest/pangeo_forge_recipes/recipe_user_guide/recipes.html) it says the inputs can be "any file format Xarray can read, including NetCDF, OPeNDAP, GRIP, Zarr." However, when I input Zarr files to FilePattern() with the file_type='zarr' keyword argument and then build and try to run a recipe I get an error:

OSError: [Errno -36] NetCDF: Invalid argument

I also note that FileType.Zarr is set to be an unopenable file type:

non_openable_file_types = [FileType.opendap, FileType.zarr]

Is this an error in the documentation or are there plans to make Zarr to Zarr recipes possible? OR are they already possible and I'm messing up keyword arguments?

Thanks!

@rabernat
Copy link
Contributor

rabernat commented Mar 14, 2023

Hi Katie! This will definitely be supported in the next release of Pangeo Forge (see #256).

@kbren
Copy link
Author

kbren commented Mar 14, 2023

Awesome thanks for the quick reply Ryan! Any estimates on that release date?

@rabernat
Copy link
Contributor

Hopefully by the end of the week!

@kbren kbren closed this as completed Mar 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants