[5pt] CatFIM - issues with new docker image

[RobHanna-NOAA]

During the development of the major update to CatFIM recently, it was detected that some new problems were created when running CatFIM code against the latest Docker image, fim_4:dev_20240823_6321468. Addressing that issue was deferred until now:

These are related to python package updates.

---

There is at least one issue but possibly two;

1. Confirmed: Running flow based:

ERROR 12040101 : fcwt2 : action - Reformatted inundation map - Exception
ERROR Traceback (most recent call last):
File "/foss_fim/tools/generate_categorical_fim_mapping.py", line 879, in reformat_inundation_maps
extent_poly_diss.to_file(
File "/usr/local/lib/python3.10/dist-packages/geopandas/geodataframe.py", line 1536, in to_file
_to_file(self, filename, driver, schema, index, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/geopandas/io/file.py", line 686, in _to_file
_to_file_pyogrio(df, filename, driver, schema, crs, mode, metadata, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/geopandas/io/file.py", line 748, in _to_file_pyogrio
pyogrio.write_dataframe(df, filename, driver=driver, metadata=metadata, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pyogrio/geopandas.py", line 622, in write_dataframe
write_arrow(
File "/usr/local/lib/python3.10/dist-packages/pyogrio/raw.py", line 857, in write_arrow
ogr_write_arrow(
File "pyogrio/_io.pyx", line 2594, in pyogrio._io.ogr_write_arrow
File "pyogrio/_io.pyx", line 2734, in pyogrio._io.create_fields_from_arrow_schema
pyogrio.errors.FieldError: Error while creating field from Arrow for field 5 with name 'interval_stage' and type n (Type 'n' for field interval_stage is not supported.).

---

2. Another one seen earlier was:
   ERROR An ind reformat map error occured for 07130008 - cdli2 - magnitude action
   ERROR Traceback (most recent call last):
   File "/foss_fim/tools/generate_categorical_fim_mapping.py", line 831, in reformat_inundation_maps
   extent_poly = gpd.GeoDataFrame.from_features(list(results), crs=PREP_PROJECTION) # Previous code
   File "/usr/local/lib/python3.10/dist-packages/geopandas/geodataframe.py", line 745, in from_features
   return cls(rows, columns=columns, crs=crs)
   File "/usr/local/lib/python3.10/dist-packages/geopandas/geodataframe.py", line 212, in init
   raise ValueError(
   ValueError: Assigning CRS to a GeoDataFrame without a geometry column is not supported. Supply geometry using the 'geometry=' keyword argument, or by providing a DataFrame with column name 'geometry'

Some quick research at that time, shows a block of code that should be redone.

        # Convert raster to shapes
        with rasterio.open(tif_to_process) as src:
            image = src.read(1)
            mask = image > 0
        # Aggregate shapes
        results = (
            {'properties': {'extent': 1}, 'geometry': s}
            for i, (s, v) in enumerate(shapes(image, mask=mask, transform=src.transform))
        )
        # Convert list of shapes to polygon
        # lots of polys
        extent_poly = gpd.GeoDataFrame.from_features(list(results), crs=PREP_PROJECTION)  # Previous code

We might be able to follow other FIM code that is more verbose for creating shapes from polys, and might make it easier to debug.

I believe using 07130008 can replicate this error, not sure if it was flow or stage though.

[mluck]

The first problem should be solved by adding engine='fiona' to the .to_file() call on line 879. See #1267 for more details.

[RobHanna-NOAA]

Doh... good find. Thank you. I will double check that. Sometime the obvious can go right over my head :)