Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"GRIB_PARAM" attributes are not saving to netcdf #596

Open
pp-mo opened this issue Jan 7, 2025 · 2 comments
Open

"GRIB_PARAM" attributes are not saving to netcdf #596

pp-mo opened this issue Jan 7, 2025 · 2 comments

Comments

@pp-mo
Copy link
Member

pp-mo commented Jan 7, 2025

User reported locally that this appeared to have changed with new versions of iris-grib,
and I originally thought was due to extension of the "GRIB_PARAM" attribute scheme to GRIB2 data in the latest v0.20 release.

But actually it doesn't work with GRIB2 data either,
so I now think this is actually a bug in newer versions of Iris, i.e. (probably) v3.11
-- TBC

Example:

>>> import iris
>>> from iris.tests import get_data_path
>>> 
>>> pth2 = itst.get_data_path(["GRIB", "time_processed", "time_bound.grib2"])
>>> 
>>> cube2 = iris.load_cube(pth2)
>>> print(cube2)
air_temperature / (K)               (latitude: 73; longitude: 96)
    Dimension coordinates:
        latitude                             x              -
        longitude                            -              x
    Scalar coordinates:
        forecast_period             -11055.0 hours, bound=(-28587.0, 6477.0) hours
        forecast_reference_time     1998-03-06 03:00:00
        pressure                    100000.0 Pa
        time                        1996-11-30 12:00:00, bound=(1994-12-01 00:00:00, 1998-12-01 00:00:00)
    Cell methods:
        0                           time: mean
    Attributes:
        GRIB_PARAM                  GRIB2:d000c000n000
>>> 
>>> iris.save(cube2, "tmp2.nc")
/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py:2668: IrisDeprecation: Saving to netcdf with legacy-style attribute handling for backwards compatibility.
This mode is deprecated since Iris 3.8, and will eventually be removed.
Please consider enabling the new split-attributes handling mode, by setting 'iris.FUTURE.save_split_attrs = True'.
  warn_deprecated(message)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/io/__init__.py", line 477, in save
    result = saver(source, target, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py", line 2750, in save
    sman.write(
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py", line 647, in write
    self.update_global_attributes(global_attributes)
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py", line 712, in update_global_attributes
    _setncattr(self._dataset, attr_name, attributes[attr_name])
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py", line 271, in _setncattr
    return variable.setncattr(name, attribute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/netCDF4/_netCDF4.pyx", line 3087, in netCDF4._netCDF4.Dataset.setncattr
  File "src/netCDF4/_netCDF4.pyx", line 1888, in netCDF4._netCDF4._set_att
TypeError: illegal data type for attribute b'GRIB_PARAM', must be one of dict_keys(['S1', 'i1', 'u1', 'i2', 'u2', 'i4', 'u4', 'i8', 'u8', 'f4', 'f8']), got O
>>> 
@pp-mo
Copy link
Member Author

pp-mo commented Jan 7, 2025

Actually I think no arbitrary objects save to netcdf.

>>> import iris
>>> print(iris.__file__)
/home/h05/itpp/git/iris/iris_main/lib/iris/__init__.py
>>> print(iris.__version__)
3.11.1
>>> if hasattr(iris.FUTURE, "save_split_attrs"):
...     iris.FUTURE.save_split_attrs = True
... 
>>> from iris.cube import Cube
>>> class MyObj:
...   x: int = 3
... 
>>> cube = Cube(
...   [1],
...   attributes={
...     'x': MyObj()
...   }
... )
>>> iris.save(cube, "tmp.nc")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/io/__init__.py", line 477, in save
    result = saver(source, target, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py", line 2750, in save
    sman.write(
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py", line 589, in write
    cf_var_cube = self._create_cf_data_variable(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py", line 2249, in _create_cf_data_variable
    _setncattr(cf_var, attr_name, value)
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/saver.py", line 271, in _setncattr
    return variable.setncattr(name, attribute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/h05/itpp/git/iris/iris_main/lib/iris/fileformats/netcdf/_thread_safe_nc.py", line 122, in setncattr
    return self._contained_instance.setncattr(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/netCDF4/_netCDF4.pyx", line 4604, in netCDF4._netCDF4.Variable.setncattr
  File "src/netCDF4/_netCDF4.pyx", line 1762, in netCDF4._netCDF4._set_att
TypeError: illegal data type for attribute b'x', must be one of dict_keys(['S1', 'i1', 'u1', 'i2', 'u2', 'i4', 'u4', 'i8', 'u8', 'f4', 'f8']), got O
>>>   

@pp-mo
Copy link
Member Author

pp-mo commented Jan 7, 2025

There is special support for STASH objects : here
But not for anything else

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

1 participant