Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LazilyOuterIndexedArray doesn't support slicing with slice objects #3022

Open
mrocklin opened this issue Jun 14, 2019 · 2 comments
Open

LazilyOuterIndexedArray doesn't support slicing with slice objects #3022

mrocklin opened this issue Jun 14, 2019 · 2 comments

Comments

@mrocklin
Copy link
Contributor

Code Sample, a copy-pastable example if possible

from xarray.core.indexing import LazilyOuterIndexedArray
import numpy as np
x = LazilyOuterIndexedArray(np.ones((5, 5)))
x[:3]
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-42bee9beb30a> in <module>
----> 1 x[:3]

~/workspace/xarray/xarray/core/indexing.py in __getitem__(self, indexer)
    518             array = LazilyVectorizedIndexedArray(self.array, self.key)
    519             return array[indexer]
--> 520         return type(self)(self.array, self._updated_key(indexer))
    521
    522     def __setitem__(self, key, value):

~/workspace/xarray/xarray/core/indexing.py in _updated_key(self, new_key)
    483
    484     def _updated_key(self, new_key):
--> 485         iter_new_key = iter(expanded_indexer(new_key.tuple, self.ndim))
    486         full_key = []
    487         for size, k in zip(self.array.shape, self.key.tuple):

AttributeError: 'slice' object has no attribute 'tuple'

Problem description

Dask array meta computations like to run x[:0, :0] on input arrays. This breaks with this class.

This is on master

@shoyer
Copy link
Member

shoyer commented Jun 14, 2019

How does this end up getting called with dask? I don't think we pass LazilyOuterIndexedArray into dask.array.from_array() directly. For example, in xarray.Variable.chunk(), we always wrap it with ImplicitToExplicitIndexingAdapter, which should be the only object seen directly by dask:

data = indexing.ImplicitToExplicitIndexingAdapter(
data, indexing.OuterIndexer)
data = da.from_array(data, chunks, name=name, lock=lock)

@mrocklin
Copy link
Contributor Author

@requires_cftime
@requires_dask
def test_cftime_datetime_mean_dask_error():
times = cftime_range('2000', periods=4)
da = DataArray(times, dims=['time']).chunk()
with pytest.raises(NotImplementedError):
da.mean()

(Pdb) pp times
CFTimeIndex([2000-01-01 00:00:00, 2000-01-02 00:00:00, 2000-01-03 00:00:00,
             2000-01-04 00:00:00],
            dtype='object')
(Pdb) pp da
<xarray.DataArray (time: 4)>
dask.array<shape=(4,), dtype=object, chunksize=(4,)>
Coordinates:
  * time     (time) object 2000-01-01 00:00:00 ... 2000-01-04 00:00:00
(Pdb) pp da.data
dask.array<xarray-<this-array>, shape=(4,), dtype=object, chunksize=(4,)>
(Pdb) pp da.data._meta
PandasIndexAdapter(array=CFTimeIndex([], dtype='object'), dtype=dtype('O'))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants