NCZarr Support Part I: Local Datasets #884
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In response to #672, I've added in the logic to handle Zarr datasets by passing them through to netCDF-C's NCZarr protocol. protocols/zarr.py takes a Zarr dataset specified in any of the following formats and returns a valid NCZarr URI (specified here), to be recognized by netCDF-C:
Note that so far, this will only work on LOCAL datasets, with the default libnetcdf build installed when compliance-checker is set up with conda. NCZarr is also only fully supported in Linux at the moment, I added an OS check to pass this caveat through to a user trying to run on a Zarr from another OS.
Getting S3 support down for netCDF-C is an ongoing effort. Once it's solid, the S3 test that is currently commented out in test_cli.py should pass and it should work on S3 Zarr datasets.
Update: it looks like this is on the home stretch, AWSome!
While I was in
test_protocols.py
, I also refactored it to use Pytest, continuing the upgrade to Pytest.When run they look like: