Skip to content

Rasterio missing _FillValue in DataArray #1736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
snowman2 opened this issue Nov 22, 2017 · 5 comments
Closed

Rasterio missing _FillValue in DataArray #1736

snowman2 opened this issue Nov 22, 2017 · 5 comments

Comments

@snowman2
Copy link
Contributor

snowman2 commented Nov 22, 2017

Problem description

When xarray opens a dataset, it stores the the _FillValue in the encoding and replaces values with NaN values. However, using open_rasterio, this behavior does not occur and the _FillValue value is missing. It only has the transform, crs, is_tiled, and res attributes. Also, the encoding is empty.

Expected Output

It would be nice to have the _FillValue as an attribute or in the encoding.

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.3.final.0 python-bits: 64 OS: Linux OS-release: 4.10.0-40-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

xarray: 0.10.0rc2
pandas: 0.21.0
numpy: 1.13.2
scipy: 1.0.0
netCDF4: 1.3.1
h5netcdf: 0.5.0
Nio: dev_20170921-05806a2
bottleneck: 1.2.1
cyordereddict: None
dask: 0.15.4
matplotlib: 2.1.0
cartopy: None
seaborn: None
setuptools: 36.6.0
pip: 9.0.1
conda: None
pytest: 3.2.5
IPython: 6.2.1
sphinx: None

@fmaussion
Copy link
Member

I agree that it would be nice to have. It seems that the corresponding rasterio attribute is nodatavals (https://github.com/mapbox/rasterio/blob/master/docs/topics/masks.rst). It shouldn't be too difficult to add to the existing attributes: https://github.com/pydata/xarray/blob/master/xarray/backends/rasterio_.py#L142

@fmaussion
Copy link
Member

What I'm not sure, though, is if we should rename the attribute to _FillValue or keep its name as it is. I am also not sure if we should do the automatic conversion as it is done with the netCDF backends. Quite often geotiff files store iteger types, and filling with NaNs require floats.

Right now I would tend to just parse the nodatavals attribute and leave everything else to the user.

@snowman2
Copy link
Contributor Author

snowman2 commented Nov 23, 2017

One thought I had was that there could be a similar parameter to mask_and_scale added that could be just called mask so open_rasterio behaves similarly to the open_dataset api. The mask parameter can allow the user to decide if they want to convert to float to get a mask or leave it alone if they want to preserve the integer data.

I would vote for adding the _FillValue attribute as it conforms with the CF conventions and it will make the dataset work better with other software looking for this value if you write it out with to_netcdf().

Also, on a similar vein, it would be nice to break out the bands into separate variables if it is a multi-band raster. This way, each band can store it's own nodata value and metadata separately in the dataset.

fmaussion pushed a commit that referenced this issue Jan 19, 2018
* Added nodatavals attribute

Connected with issue #1736

* Replace None nodatavals with np.nan

Fixes the type error with serialization: https://travis-ci.org/pydata/xarray/jobs/306107679

* Fix typo

* Added nodatavals open_rasterio information.

* added issue and author information

* added tests for nodatavals attr

* moved back to nan

* added separate test for missing nodataval
@jsignell
Copy link
Contributor

Is there still interest in implementing the mask option as outlined in this issue?

@snowman2
Copy link
Contributor Author

It would definitely be a fantastic feature to have. Some discussion happened in #2308 to discuss standardizing methods for storing data that would definitely assist with this, however it did not seem like a priority at the time. Definitely feel free to share your opinion and use cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants