Failed saving animation when using napari.utils.DirectLabelColormap

[manerotoni]

Hello,
with napari-animation and I encountered a reproducible error when using DirectLabelColormap. Such a map is needed as I do class-labelling and it allows easy configuration and consistency in the colors. If I modify the standard CyclicLabelColormap the animate command works properly.

I add two small code snipptes. The first save the data correctly. In the second example the animate command fails.

# Succesful example
from napari import Viewer
from napari_animation import Animation
import numpy as np
from napari.utils import DirectLabelColormap

viewer = Viewer()
viewer.add_labels(test_lbl)
animation = Animation(viewer)
out_name_mov = './test.mov'
viewer.add_labels(test_lbl)
for idx in range(0,10):
    viewer.dims.current_step = (idx,4, 4)
    animation.capture_keyframe()
animation.animate(out_name_mov, canvas_only=True, quality = 1)

# Failed example when using a colormap DirectLabelColormap()
cmap = DirectLabelColormap()
cmap.color_dict = {None: None, 0: None, 1:'blue', 2:'red'}

viewer = Viewer()
viewer.add_labels(test_lbl, colormap=cmap)

animation = Animation(viewer)
out_name_mov = './test.mov'
viewer.add_labels(test_lbl)
for idx in range(0,10):
    viewer.dims.current_step = (idx,4, 4)
    animation.capture_keyframe()

# This command fails with TypeError: cannot pickle '_nrt_python._MemInfo' object
animation.animate(out_name_mov, canvas_only=True, quality = 1)

[manerotoni]

Here is the error (sorry it is really long)

0%|                                                                                          | 0/136 [00:00<?, ?it/s]IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (2258, 1275) to (2272, 1280) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
  1%|▌                                                                                 | 1/136 [00:00<00:13,  9.75it/s]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[36], line 9
      7     viewer.dims.current_step = (idx,4, 4)
      8     animation.capture_keyframe()
----> 9 animation.animate(out_name_mov, canvas_only=True, quality = 1)

File ~\Miniconda3\envs\btrack_env\lib\site-packages\napari_animation\animation.py:237, in Animation.animate(self, filename, fps, quality, file_format, canvas_only, scale_factor)
    235 sleep(0.05)
    236 with tqdm(total=n_frames) as pbar:
--> 237     for frame_index, image in enumerate(frame_generator):
    238         if save_as_folder is True:
    239             frame_filename = (
    240                 folder_path / f"{file_path.stem}_{frame_index:06d}.png"
    241             )

File ~\Miniconda3\envs\btrack_env\lib\site-packages\napari_animation\frame_sequence.py:152, in FrameSequence.iter_frames(self, viewer, canvas_only, scale_factor)
    145 def iter_frames(
    146     self,
    147     viewer: napari.viewer.Viewer,
    148     canvas_only: bool = True,
    149     scale_factor: float = None,
    150 ) -> Iterator[np.ndarray]:
    151     """Iterate over interpolated viewer states, and yield rendered frames."""
--> 152     for _i, state in enumerate(self):
    153         frame = state.render(viewer, canvas_only=canvas_only)
    154         if scale_factor not in (None, 1):

File ~\Miniconda3\envs\btrack_env\lib\_collections_abc.py:1043, in Sequence.__iter__(self)
   1041 try:
   1042     while True:
-> 1043         v = self[i]
   1044         yield v
   1045         i += 1

File ~\Miniconda3\envs\btrack_env\lib\site-packages\napari_animation\frame_sequence.py:136, in FrameSequence.__getitem__(self, key)
    134         self._cache[key] = kf0.viewer_state
    135     else:
--> 136         self._cache[key] = interpolate_viewer_state(
    137             kf0.viewer_state,
    138             kf1.viewer_state,
    139             frac,
    140             self.state_interpolation_map,
    141         )
    143 return self._cache[key]

File ~\Miniconda3\envs\btrack_env\lib\site-packages\napari_animation\interpolation\viewer_state_interpolation.py:39, in interpolate_viewer_state(initial_state, final_state, fraction, interpolation_map)
     16 """Interpolate a state between two states
     17 
     18 Parameters
   (...)
     34     Description of viewer state.
     35 """
     37 viewer_state_data = {}
---> 39 initial_state = asdict(initial_state)
     40 final_state = asdict(final_state)
     42 for keys in keys_to_list(initial_state):

File ~\Miniconda3\envs\btrack_env\lib\dataclasses.py:1238, in asdict(obj, dict_factory)
   1236 if not _is_dataclass_instance(obj):
   1237     raise TypeError("asdict() should be called on dataclass instances")
-> 1238 return _asdict_inner(obj, dict_factory)

File ~\Miniconda3\envs\btrack_env\lib\dataclasses.py:1245, in _asdict_inner(obj, dict_factory)
   1243 result = []
   1244 for f in fields(obj):
-> 1245     value = _asdict_inner(getattr(obj, f.name), dict_factory)
   1246     result.append((f.name, value))
   1247 return dict_factory(result)

File ~\Miniconda3\envs\btrack_env\lib\dataclasses.py:1275, in _asdict_inner(obj, dict_factory)
   1273     return type(obj)(_asdict_inner(v, dict_factory) for v in obj)
   1274 elif isinstance(obj, dict):
-> 1275     return type(obj)((_asdict_inner(k, dict_factory),
   1276                       _asdict_inner(v, dict_factory))
   1277                      for k, v in obj.items())
   1278 else:
   1279     return copy.deepcopy(obj)

File ~\Miniconda3\envs\btrack_env\lib\dataclasses.py:1276, in <genexpr>(.0)
   1273     return type(obj)(_asdict_inner(v, dict_factory) for v in obj)
   1274 elif isinstance(obj, dict):
   1275     return type(obj)((_asdict_inner(k, dict_factory),
-> 1276                       _asdict_inner(v, dict_factory))
   1277                      for k, v in obj.items())
   1278 else:
   1279     return copy.deepcopy(obj)

File ~\Miniconda3\envs\btrack_env\lib\dataclasses.py:1275, in _asdict_inner(obj, dict_factory)
   1273     return type(obj)(_asdict_inner(v, dict_factory) for v in obj)
   1274 elif isinstance(obj, dict):
-> 1275     return type(obj)((_asdict_inner(k, dict_factory),
   1276                       _asdict_inner(v, dict_factory))
   1277                      for k, v in obj.items())
   1278 else:
   1279     return copy.deepcopy(obj)

File ~\Miniconda3\envs\btrack_env\lib\dataclasses.py:1276, in <genexpr>(.0)
   1273     return type(obj)(_asdict_inner(v, dict_factory) for v in obj)
   1274 elif isinstance(obj, dict):
   1275     return type(obj)((_asdict_inner(k, dict_factory),
-> 1276                       _asdict_inner(v, dict_factory))
   1277                      for k, v in obj.items())
   1278 else:
   1279     return copy.deepcopy(obj)

File ~\Miniconda3\envs\btrack_env\lib\dataclasses.py:1279, in _asdict_inner(obj, dict_factory)
   1275     return type(obj)((_asdict_inner(k, dict_factory),
   1276                       _asdict_inner(v, dict_factory))
   1277                      for k, v in obj.items())
   1278 else:
-> 1279     return copy.deepcopy(obj)

File ~\Miniconda3\envs\btrack_env\lib\copy.py:172, in deepcopy(x, memo, _nil)
    170                 y = x
    171             else:
--> 172                 y = _reconstruct(x, memo, *rv)
    174 # If is its own copy, don't memoize.
    175 if y is not x:

File ~\Miniconda3\envs\btrack_env\lib\copy.py:271, in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    269 if state is not None:
    270     if deep:
--> 271         state = deepcopy(state, memo)
    272     if hasattr(y, '__setstate__'):
    273         y.__setstate__(state)

File ~\Miniconda3\envs\btrack_env\lib\copy.py:146, in deepcopy(x, memo, _nil)
    144 copier = _deepcopy_dispatch.get(cls)
    145 if copier is not None:
--> 146     y = copier(x, memo)
    147 else:
    148     if issubclass(cls, type):

File ~\Miniconda3\envs\btrack_env\lib\copy.py:231, in _deepcopy_dict(x, memo, deepcopy)
    229 memo[id(x)] = y
    230 for key, value in x.items():
--> 231     y[deepcopy(key, memo)] = deepcopy(value, memo)
    232 return y

File ~\Miniconda3\envs\btrack_env\lib\copy.py:146, in deepcopy(x, memo, _nil)
    144 copier = _deepcopy_dispatch.get(cls)
    145 if copier is not None:
--> 146     y = copier(x, memo)
    147 else:
    148     if issubclass(cls, type):

File ~\Miniconda3\envs\btrack_env\lib\copy.py:231, in _deepcopy_dict(x, memo, deepcopy)
    229 memo[id(x)] = y
    230 for key, value in x.items():
--> 231     y[deepcopy(key, memo)] = deepcopy(value, memo)
    232 return y

File ~\Miniconda3\envs\btrack_env\lib\copy.py:146, in deepcopy(x, memo, _nil)
    144 copier = _deepcopy_dispatch.get(cls)
    145 if copier is not None:
--> 146     y = copier(x, memo)
    147 else:
    148     if issubclass(cls, type):

File ~\Miniconda3\envs\btrack_env\lib\copy.py:231, in _deepcopy_dict(x, memo, deepcopy)
    229 memo[id(x)] = y
    230 for key, value in x.items():
--> 231     y[deepcopy(key, memo)] = deepcopy(value, memo)
    232 return y

File ~\Miniconda3\envs\btrack_env\lib\copy.py:172, in deepcopy(x, memo, _nil)
    170                 y = x
    171             else:
--> 172                 y = _reconstruct(x, memo, *rv)
    174 # If is its own copy, don't memoize.
    175 if y is not x:

File ~\Miniconda3\envs\btrack_env\lib\copy.py:271, in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    269 if state is not None:
    270     if deep:
--> 271         state = deepcopy(state, memo)
    272     if hasattr(y, '__setstate__'):
    273         y.__setstate__(state)

File ~\Miniconda3\envs\btrack_env\lib\copy.py:146, in deepcopy(x, memo, _nil)
    144 copier = _deepcopy_dispatch.get(cls)
    145 if copier is not None:
--> 146     y = copier(x, memo)
    147 else:
    148     if issubclass(cls, type):

File ~\Miniconda3\envs\btrack_env\lib\copy.py:231, in _deepcopy_dict(x, memo, deepcopy)
    229 memo[id(x)] = y
    230 for key, value in x.items():
--> 231     y[deepcopy(key, memo)] = deepcopy(value, memo)
    232 return y

File ~\Miniconda3\envs\btrack_env\lib\copy.py:161, in deepcopy(x, memo, _nil)
    159 reductor = getattr(x, "__reduce_ex__", None)
    160 if reductor is not None:
--> 161     rv = reductor(4)
    162 else:
    163     reductor = getattr(x, "__reduce__", None)

TypeError: cannot pickle '_nrt_python._MemInfo' object

[brisvag]

Might be caused by the recent napari/napari#7025...

[jni]

Might be caused by the recent

no, based on this SO question it's because there is a numba typed dict in the class, so the class cannot be pickled. I think the solution would be to add a __pickle__ or __copy__ method (I'm not sure what the right magic dunder methods are here) to DirectLabelColormap that will swap out the numba dict for a regular dict.

There might be a user-space solution here (check if it's a DirectLabelColormap, replace it with a dict — thanks to napari/napari#7025, actually. 😂), so I'll leave this issue here and make a more targeted one in napari. Thanks @manerotoni for the report! 🙏

[psobolewskiPhD]

I think this came up with the PRs fixing some of the state/properties comparisons.
Does interpolation of colors of labels even make sense -- particularly for a user provided direct colormap?

[jni]

You might not want to interpolate the colors, (in fact, in the case of colormapped values, you certainly don't want to interpolate in color space), but that can be done by setting interpolation=None. I can see it being useful to change the colormap in the middle of an animation, so checkpointing a colormap is certainly a necessary feature.

[Czaki]

no, based on this SO question it's because there is a numba typed dict in the class, so the class cannot be pickled. I think the solution would be to add a __pickle__ or __copy__ method (I'm not sure what the right magic dunder methods are here) to DirectLabelColormap that will swap out the numba dict for a regular dict.

I think that __copy__ is enough.