0.9.3

• Bugfix to allow opening of zarr files. This fix allows using Zarr stores as an input source for recipes. #462
• Add netcdf3 support for opening source files with kerchunk. Resolves a long-standing issue wherein netcdf3 source files could not be loaded lazily, which effectively blocked the use of large netcdf3 files as recipe sources. #383
• Fix zarr reference bug #455
• Add dataset_type class attribute for recipe classes #437

0.9.2

What's Changed

• Add SciPy video and slides to what_is_pangeo_forge.md by @rabernat in #416
• Document how to access data with Globus by @rabernat in #408
• Remove pre-commit action in favor of pre-commit.ci by @andersy005 in #423
• Cancel Redundant CI jobs by @andersy005 in #424
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #381
• Fix CI by @andersy005 in #430
• Avoid serialization blues by computing + caching the hash. by @alxmrs in #429
• Bump peter-evans/slash-command-dispatch from 2 to 3 by @dependabot in #434
• Bump codecov/codecov-action from 2.0.2 to 3.1.1 by @dependabot in #431
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #426
• Bump actions/setup-python from 3 to 4 by @dependabot in #433
• Bump LouisBrunner/checks-action from 1.2.0 to 1.5.0 by @dependabot in #432
• Update release notes in preparation for new release by @andersy005 in #436

New Contributors

• @andersy005 made their first contribution in #423
• @pre-commit-ci made their first contribution in #381
• @dependabot made their first contribution in #434

Full Changelog: 0.9.1...0.9.2

0.9.1

• Persist Pangeo Forge execution context metadata in target datasets. This information, which includes the pangeo-forge-recipes version as well as recipe and input hashes, attaches execution provenance to the dataset itself. #359
• File pattern support for .tiffs. #393
• Improved HDFReferenceRecipe by passing target_options to the MultiZarrToZarr class.
• Typo fixes, documentation updates, and project health improvements. #364, #365, #366, #388, #396, #394, #398, #407.
• Fixed XarrayZarrRecipe's finalize_target() stage by using bulk delete APIs for consolidating Zarr coordinates.

0.9.0

• Breaking changes: Deprecated XarrayZarrRecipe manual stage methods. Also deprecated FilePattern(..., is_opendap=True) kwarg, which is superseded by FilePattern(..., file_type="opendap"). #362
• Added serialization module along with BaseRecipe.sha256 and FilePattern.sha256 methods. Collectively, this provides for generation of deterministic hashes for both recipe and file pattern instances. Checking these hashes against those from a prior version of the recipe can be used to determine whether or not a particular recipe instance in a Python module (which may contain arbitrary numbers of recipe instances) has changed since the last time the instances in that module were executed. The file pattern hashes are based on a merkle tree built cumulatively from all of the index:filepath pairs yielded by the pattern's self.items() method. As such, in cases where a new pattern is intended to append to an existing dataset which was built from a prior version of that pattern, the pattern hash can be used to determine the index from which to begin appending. This is demonstrated in the tests. #349
• Created new Prefect executor which wraps the Dask executor in a single Task. This should mitigate problems related to large numbers of Prefect Tasks (#347).
• Implemented feature to cap cached filename lengths at 255 bytes on local filesystems, to accommodate the POSIX filename length limit. Cached filename lengths are not truncated on any other filesystem. #353

0.8.3

• Added .file_type attribute to pangeo_forge_recipes.patterns.FilePattern. This attribute will eventually supercede .is_opendap, which will be deprecated in 0.9.0. Until then, FilePattern(..., is_opendap=True) is supported as equivalent to FilePattern(..., file_type="opendap"). #322

0.8.2

• Removed click from dependencies and removed cli entrypoint.

0.8.1

• Fixed dependency issue with pip installation.
• Fixed bug where recipes would fail if the target chunks exceeded the full array length. #279

0.8.0

• Breaking change: Replace recipe classes' storage attibutes with .storage_config of type pangeo_forge_recipes.storage.StorageConfig. #288
• Add setup_logging convenience function. #287

0.7.0

• Apache Beam executor added. #169. By Alex Merose. ❤️
• Dask executor updates. #260, #261
• Index type update. #257
• Fix incompatibility with fsspec>=2021.11.1. #247

0.6.1

• Major internal refactor of executors. #219.
• Began deprecation cycle for recipe methods (e.g. recipe.prepare_target()) in favor of module functions.
• Addition of open_input_with_fsspec_reference option on pangeo_forge_recipes.recipes.XarrayZarrRecipe, permitting the bypassing of h5py when opening inputs. #218.