Issues with installing/importing CaImAn using only conda-forge on a windows

[isaacracine]

Overview

Setup:

1. Operating System (Linux, MacOS, Windows): windows
2. Hardware type (x86, ARM..) and RAM: x86-64 hardware, 384GB
3. Python Version (e.g. 3.9): 3.7.12
4. Caiman version (e.g. 1.9.12): 1.9.8
5. How you installed Caiman (pure conda, conda + compile, colab, ..): conda via miniforge

I am experiencing issues being able to build a working CaImAn environment via miniforge. My institute is no longer able to use ananconda or the defaults channel, therefore we have been primarily using miniforge instead. The issue seems to be stemming from the conda-forge channel only having older versions of tensorflow available for windows. This issue was touched on #1332 (here), however the work around involved downloading tensorflow from the defaults channel which is not an option for me. However, if I use a yml file with strict specifications of acceptable package versions I can build a CaImAn environment. Unfortunately, this issue persists as when I attempt to import caiman an error occurs. I have tried trouble shooting the error but have had no success. If anyone has some advice that would be great, otherwise I'd like to draw attention to this short coming.

Creation from yml:

mamba env create -f environment.yml

#environment.yml
name: caiman
channels:
  - conda-forge
dependencies:
  - python<=3.10
  - pip
  - caiman
  - tensorflow<=1.14

Import error:
import caiman as cm

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\AppData\Local\Temp\6\ipykernel_15368\4048091256.py in <module>
----> 1 import caiman as cm

c:\Users\general\AppData\Local\miniforge3\envs\caiman\lib\site-packages\caiman\__init__.py in <module>
      2 
      3 import pkg_resources
----> 4 from .base.movies import movie, load, load_movie_chain
      5 from .base.timeseries import concatenate
      6 from .cluster import start_server, stop_server

c:\Users\general\AppData\Local\miniforge3\envs\caiman\lib\site-packages\caiman\base\movies.py in <module>
     55 
     56 from ..mmapping import load_memmap
---> 57 from ..utils import visualization
     58 from .. import summary_images as si
     59 from ..motion_correction import apply_shift_online, motion_correct_online

c:\Users\general\AppData\Local\miniforge3\envs\caiman\lib\site-packages\caiman\utils\visualization.py in <module>
     31 from typing import Dict
     32 from warnings import warn
---> 33 import holoviews as hv
     34 import functools as fct
     35 

c:\Users\general\AppData\Local\miniforge3\envs\caiman\lib\site-packages\holoviews\__init__.py in <module>
     82                                         reponame="holoviews"))
     83 
---> 84 from . import util                                       # noqa (API import)
     85 from .core import archive, config                        # noqa (API import)
     86 from .core.boundingregion import BoundingBox             # noqa (API import)

c:\Users\general\AppData\Local\miniforge3\envs\caiman\lib\site-packages\holoviews\util\__init__.py in <module>
     12 from pyviz_comms import extension as _pyviz_extension
     13 
---> 14 from ..core import (
     15     Dataset, DynamicMap, HoloMap, Dimensioned, ViewableElement,
     16     StoreOptions, Store

c:\Users\general\AppData\Local\miniforge3\envs\caiman\lib\site-packages\holoviews\core\__init__.py in <module>
      4 
      5 from .boundingregion import *
----> 6 from .data import *
      7 from .dimension import *
      8 from .element import *

c:\Users\general\AppData\Local\miniforge3\envs\caiman\lib\site-packages\holoviews\core\data\__init__.py in <module>
     25 from .dictionary import DictInterface         # noqa (API import)
     26 from .grid import GridInterface               # noqa (API import)
---> 27 from .ibis import IbisInterface               # noqa (API import)
     28 from .interface import Interface, iloc, ndloc
     29 from .multipath import MultiInterface         # noqa (API import)

c:\Users\general\AppData\Local\miniforge3\envs\caiman\lib\site-packages\holoviews\core\data\ibis.py in <module>
     13 
     14 
---> 15 @lru_cache
     16 def ibis_version():
     17     import ibis

c:\Users\general\AppData\Local\miniforge3\envs\caiman_new\lib\functools.py in lru_cache(maxsize, typed)
    488             maxsize = 0
    489     elif maxsize is not None:
--> 490         raise TypeError('Expected maxsize to be an integer or None')
    491 
    492     def decorating_function(user_function):

TypeError: Expected maxsize to be an integer or None

[pgunn]

We know about this, and unfortunately right now there's not a great solution; we've been holding off on new releases because we can't make them work across all three platforms (forcing older versions of tensorflow also doesn't work, as you've discovered).

We've been hoping for things to be a bit easier with pytorch (and have some almost-ready-to-commit work done to switch over), but we also need pytorch on windows to exist - it's currently blocked:

conda-forge/pytorch-cpu-feedstock#32

Right now the only real way I see to make this work is to start like you've done and then use pip to replace tensorflow (this may require some other version pins). This is pretty suboptimal though. Hopefully pytorch on windows will come to conda-forge soon, and we'll be able to land our switchover diffs to get rid of our tensorflow dependency shortly afterwards.

[pgunn]

(I should note that the place I work has also blocked access to the defaults channel, for the same reasons probably that your institution did so - it's a shame that ContinuumIO did this)