Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make compatible with CFTime #34

Open
willirath opened this issue Dec 9, 2019 · 6 comments
Open

Make compatible with CFTime #34

willirath opened this issue Dec 9, 2019 · 6 comments

Comments

@willirath
Copy link
Owner

https://github.com/willirath/xorca/blob/1cab7220aa2cea8698db5d86b3e0d75449959539/xorca/lib.py#L67_L68 tries to find time dimensions by ckecking their data types. This fails if cftime objects are used on the time axis.

Road to fully support cftime:

  1. We pass use_cftime kwarg to all the open_dataset calls.

  2. Fix

    def _is_time_dim(ds, dim):
        return (dim in orca_names.t_dims and
                np.issubdtype(ds[dim].dtype,
                              np.datetime64))

to sth like

    def _is_time_dim(ds, dim):
        return (dim in orca_names.t_dims and
                np.issubdtype(ds[dim].dtype,
                              'object'))

(But note that it would be better to actively check for valid time types. this is likely also done somewhere in xarray's plotting backend.)

@willirath
Copy link
Owner Author

(... and (np.issubdtype(ds[dim].dtype, np.datetime64) or <check for cftime>))

@janharlass
Copy link

janharlass commented Jan 7, 2020

Apparently it does work with files that have a monthly resolution in time or better - more than one time step. There is no issue with the time axis for times after year 2262 or so.

@janharlass
Copy link

But it still fails with some '<' not supported between instances of 'NoneType' and 'NoneType' error, when trying to read yearly data, meaning only 1 time step in a file. Could it be related to the possible chunks? Or sorting?

@willirath
Copy link
Owner Author

Could it be related to the possible chunks? Or sorting?

No. It's because the type of the time dim is not recognised. (See above). XORCA makes sure that time dims are not squeezed away.

@janharlass
Copy link

XORCA checks for datetime64, which the yearly time values before year 26?? clearly are. I double checked with:
d1 = xr.concat( (xr.open_dataset(file.format(year=y), use_cftime=False).rename({'time_counter': 'time'}) for y in (2000, 2001, 2002, 2003)), dim="time")
If use_cftime=True and/or year is after 26??, they become objects and are not recognized by XORCA anymore.
Still, if I remove the time dimension check and simply allow all time axis to be a time axis, no matter what their type might be and they will not be squeezed, it still fails.

@janharlass
Copy link

I think I found a solution. Probably improvable in coding style, but it works. All variables from the aux files are correctly transferred to the new data set and time axis are carfeuly checked. All complies to cftime. See #35.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants