Loading file with scalar coordinate with `valid_min`/`valid_max` fails if coordinate point is outside valid range

[schlunma]

🐛 Bug Report

A cube with a scalar coordinate that has a point outside its valid_min/valid_max cannot be read with Iris. Loading it fails with a TypeError.

How To Reproduce

Steps to reproduce the behaviour:

import iris
from iris.cube import Cube
from iris.coords import AuxCoord
from pathlib import Path

iris.FUTURE.save_split_attrs = True

path = Path.home() / "test.nc"

coord = AuxCoord(-1.0, var_name="lon", attributes={"valid_min": 0, "valid_max": 360})
cube = Cube(0.0, var_name="tas", aux_coords_and_dims=[(coord, ())])

iris.save(cube, path)

Reading this file with iris.load(path) fails with a

TypeError: unhashable type: 'MaskedConstant'

Full traceback

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[2], line 1
----> 1 iris.load(path)

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/loading.py:169, in load(uris, constraints, callback)
    145 def load(uris, constraints=None, callback=None):
    146     """Load any number of Cubes for each constraint.
    147 
    148     For a full description of the arguments, please see the module
   (...)    167 
    168     """
--> 169     cubes = _load_collection(uris, constraints, callback).combined().cubes()
    170     return cubes

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/loading.py:120, in _CubeFilterCollection.combined(self)
    114 def combined(self):
    115     """Return a new :class:`_CubeFilterCollection` by combining all the cube lists of this collection.
    116 
    117     Combines each list of cubes using :func:`~iris._combine_load_cubes`.
    118 
    119     """
--> 120     return _CubeFilterCollection([pair.combined() for pair in self.pairs])

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/loading.py:78, in _CubeFilter.combined(self)
     69 """Return a new :class:`_CubeFilter` by combining the list of cubes.
     70 
     71 Combines the list of cubes with :func:`~iris._combine_load_cubes`.
     72 
     73 """
     74 from iris._combine import _combine_load_cubes
     76 return _CubeFilter(
     77     self.constraint,
---> 78     _combine_load_cubes(self.cubes),
     79 )

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/_combine.py:421, in _combine_load_cubes(cubes)
    418     if _MULTIREF_DETECTION.found_multiple_refs:
    419         options["merge_concat_sequence"] += "c"
--> 421 return _combine_cubes(cubes, options)

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/_combine.py:393, in _combine_cubes(cubes, options)
    386     cubelist = cubelist.concatenate()
    387 if "m" in sequence:
    388     # merge if requested.
    389     # NOTE: the 'unique' arg is configurable in the combine options.
    390     # All CombineOptions settings have "unique=False", as that is needed for
    391     #  "iris.load_xxx()" functions to work correctly.  However, the default
    392     #  for CubeList.merge() is "unique=True".
--> 393     cubelist = cubelist.merge(unique=merge_unique)
    394 if sequence[-1] == "c":
    395     # concat if it comes last
    396     cubelist = cubelist.concatenate()

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/cube.py:435, in CubeList.merge(self, unique)
    433 for name in sorted(proto_cubes_by_name, key=_none_sort):
    434     for proto_cube in proto_cubes_by_name[name]:
--> 435         merged_cubes.extend(proto_cube.merge(unique=unique))
    437 return merged_cubes

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/_merge.py:1206, in ProtoCube.merge(self, unique)
   1189 """Return the list of cubes resulting from merging the registered source-cubes.
   1190 
   1191 Parameters
   (...)   1200 
   1201 """
   1202 positions = [
   1203     {i: v for i, v in enumerate(skeleton.scalar_values)}
   1204     for skeleton in self._skeletons
   1205 ]
-> 1206 indexes = build_indexes(positions)
   1207 relation_matrix = derive_relation_matrix(indexes)
   1208 groups = derive_groups(relation_matrix)

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/_merge.py:581, in build_indexes(positions)
    578 for name, value in position.items():
    579     name_index_by_scalar = scalar_index_by_name[name]
--> 581     if value in name_index_by_scalar:
    582         value_index_by_name = name_index_by_scalar[value]
    583         for other_name in names:

File ~/micromamba/envs/esm/lib/python3.12/site-packages/iris/coords.py:1281, in Cell.__hash__(self)
   1279 print(self.bound)
   1280 if self.bound is None:
-> 1281     return hash(self.point)
   1282 bound = self.bound
   1283 rbound = bound[::-1]

TypeError: unhashable type: 'MaskedConstant'

Expected behaviour

No error.

Environment

• OS & Version: Linux
• Iris Version: 3.12.0

Additional context

It looks like the resulting coordinate point is set to masked, which causes the __hash__ function to fail.

Doing the same with a 1D coordinate that contains an invalid value works just fine:

coord = AuxCoord([-1.0, 1.0], var_name="lon", attributes={"valid_min": 0, "valid_max": 360})
# ...
print(cube.coord("lon"))  # --> <AuxCoord: lon / (unknown)  [-1., 1.]  shape(2,)>

Interestingly enough, the first value is not masked now.

Repeating this with invalid data also works:

cube = Cube(-1.0, var_name="tas", aux_coords_and_dims=[(coord, ())], attributes={"valid_min": 0, "valid_max": 360})
print(cube.data)  # --> -1.0

Again, the data is not masked at all.

[stephenworsley]

I believe this may have been solved by #6468.

[schlunma]

Yes, with the current main branch this works as expected now (also the cases listed under "Additional context"). Thanks so much for fixing this!