"not all values found in index 'lat'"

[BenjaminFTurner]

Hello. I'm getting the following error when attempting to run MetSim (installed yesterday with "pip install metsim"):

$ ms west_fork.conf -v -n 15
Traceback (most recent call last):
  File "/home/bent/anaconda3/bin/ms", line 11, in <module>
    sys.exit(main())
  File "/home/bent/anaconda3/lib/python3.7/site-packages/metsim/cli/ms.py", line 98, in main
    ms = MetSim(setup)
  File "/home/bent/anaconda3/lib/python3.7/site-packages/metsim/metsim.py", line 193, in __init__
    self._times = self._get_output_times(freq=self.params['out_freq'])
  File "/home/bent/anaconda3/lib/python3.7/site-packages/metsim/metsim.py", line 451, in _get_output_times
    prototype = self.met_data
  File "/home/bent/anaconda3/lib/python3.7/site-packages/metsim/metsim.py", line 244, in met_data
    self._met_data = io.read_met_data(self.params, self._domain)
  File "/home/bent/anaconda3/lib/python3.7/site-packages/metsim/io.py", line 46, in read_met_data
    return process_funcs[params['forcing_fmt']](params, domain)
  File "/home/bent/anaconda3/lib/python3.7/site-packages/metsim/io.py", line 74, in process_nc
    calendar=params['calendar'], var_dict=params.get('forcing_vars', None))
  File "/home/bent/anaconda3/lib/python3.7/site-packages/metsim/io.py", line 139, in read_netcdf
    for k in list(domain.dims.keys())
  File "/home/bent/anaconda3/lib/python3.7/site-packages/xarray/core/dataset.py", line 1621, in sel
    self, indexers=indexers, method=method, tolerance=tolerance)
  File "/home/bent/anaconda3/lib/python3.7/site-packages/xarray/core/coordinates.py", line 355, in remap_label_indexers
    obj, v_indexers, method=method, tolerance=tolerance
  File "/home/bent/anaconda3/lib/python3.7/site-packages/xarray/core/indexing.py", line 250, in remap_label_indexers
    dim, method, tolerance)
  File "/home/bent/anaconda3/lib/python3.7/site-packages/xarray/core/indexing.py", line 189, in convert_label_indexer
    % index_name)
KeyError: "not all values found in index 'lat'"

From the looks of it, it seems that there is a problem reading the forcings file. The data for this file look like this:

File metsim_forcings.nc (NC_FORMAT_NETCDF4):

     4 variables (excluding dimension variables):
        float Prec[lon,lat,time]   (Contiguous storage)  
            units: mm
            _FillValue: 9.96920996838687e+36
            long_name: Precipitation
        float Tmin[lon,lat,time]   (Contiguous storage)  
            units: C
            _FillValue: 9.96920996838687e+36
            long_name: Daily minimum temperature
        float Tmax[lon,lat,time]   (Contiguous storage)  
            units: C
            _FillValue: 9.96920996838687e+36
            long_name: Daily maximum temperature
        float wind[lon,lat,time]   (Contiguous storage)  
            units: m/s
            _FillValue: 9.96920996838687e+36
            long_name: mean daily wind speed

     3 dimensions:
        lon  Size:18
            units: degrees_east
            long_name: lon
        lat  Size:22
            units: degrees_north
            long_name: lat
        time  Size:10
            units: days since 1900-01-01 00:00:00
            long_name: time
            calendar: gregorian

I'm not sure if it matters, but I made sure that the lengths of the lat and lon vectors match those in the domain file, and they are all increasing in value. The dimensions of the variables all seem to check out, and I've run out of ideas for other things to check.

Please let me know if you have any advice.

Ben
[Edit: adding files: metsim.zip]

[arbennett]

Hi @BenjaminFTurner - I took a look at your setup and it appears that there is a slight difference in the latitudes between the domain and forcing data. Note the difference starting around the 5th decimal place:

Domain lats - [ 38.72952781  38.77119381  38.81285981  38.85452582  38.89619182
  38.93785782  38.97952382  39.02118982  39.06285582  39.10452182
  39.14618783  39.18785383  39.22951983  39.27118583  39.31285183
  39.35451783  39.39618384  39.43784984  39.47951584  39.52118184
  39.56284784  39.60451384]

Forcing lats - [ 38.72952652  38.77119446  38.81285858  38.85452652  38.89619064
  38.93785858  38.97952271  39.02119064  39.06285477  39.10451889
  39.14618683  39.18785095  39.22951889  39.27118301  39.31285095
  39.35451508  39.39618301  39.43784714  39.47951508  39.5211792
  39.56284714  39.60451126]

Currently, MetSim requires these to be exact, so I would suggest simply rounding these values so that they match.

[BenjaminFTurner]

Thank you very much, Andrew, for taking the time to look at this and reply with a suggestion. My domain and forcings data were based on the same grid, but the domain data had been processed slightly with my R script which introduced some rounding error. To make them the same, I just used the domain coordinates, and I added some code in my script to verify that the forcings coordinates were the same within a tolerable error. So this issue is fixed for me.

Because it may not be obvious that the coordinates need to match exactly, however, I suggest that either the docs be updated to reflect this or something in the code check this to provide a more human-understandable error message.

Thanks for the free software! - Ben