-
I have a Zarr file on Amazon blob storage which I want to read without downloading. Zarr has the ABSStore class to do this, but xarray doesn't seem to be able to open it. I can't find any examples of people adopting this workflow and I am starting to wonder if this is because it can't be done this way. It is equally likely that I have got the wrong end of the stick somewhere, so any advice either way would be very welcome. store = zarr.ABSStore(container='my_container_name', prefix='monthly/', account_name=my_account_name, account_key=my_cred)
dataset = xarray.open_dataset(store) however, this fails with: ---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
----> 5 dataset = xarray.open_dataset(store)
python3.7/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, group, decode_cf, mask_and_scale, decode_times, autoclose, concat_characters, decode_coords, engine, chunks, lock, cache, drop_variables, backend_kwargs, use_cftime, decode_timedelta)
534 "with engine='scipy' or 'h5netcdf'"
535 )
--> 536 engine = _get_engine_from_magic_number(filename_or_obj)
537 if engine == "scipy":
538 store = backends.ScipyDataStore(filename_or_obj, **backend_kwargs)
python3.7/site-packages/xarray/backends/api.py in _get_engine_from_magic_number(filename_or_obj)
113 magic_number = filename_or_obj[:8]
114 else:
--> 115 if filename_or_obj.tell() != 0:
116 raise ValueError(
117 "file-like object read/write pointer not at zero "
AttributeError: 'ABSStore' object has no attribute 'tell' For info, versions of key packages are: |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
thanks to some help from my colleagues, the key mistake here is that I need to use: import zarr
import xarray
xarray.set_options(display_style='text') # Work around for AML bug that won't display HTML output.
store = zarr.ABSStore(container='my_container_name', prefix='monthly/', account_name="my_account_name", blob_service_kwargs={"sas_token":SAS})
xr = xarray.open_zarr(store)
xr (without the display_style option then if you just put xr in a jupyter notebook for example, then it doesn't show anything, although print does work) |
Beta Was this translation helpful? Give feedback.
thanks to some help from my colleagues, the key mistake here is that I need to use:
open_zarr not open_data
Also I had not set the creditials correctly.
(without the display_style option then if you just put xr in a jupyter notebook for example, then it doesn't show anything, although print does work)