Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mcorr fails if input video has less than 1000 frames #180

Open
kushalkolar opened this issue Mar 1, 2023 · 1 comment
Open

mcorr fails if input video has less than 1000 frames #180

kushalkolar opened this issue Mar 1, 2023 · 1 comment

Comments

@kushalkolar
Copy link
Collaborator

traceback, this input video had 300 frames:

---------------------------------------------------------------------------
BatchItemUnsuccessfulError                Traceback (most recent call last)
Cell In [24], line 2
      1 df = df.caiman.reload_from_disk()
----> 2 df.iloc[-1].mcorr.get_output()

File ~/repos/mesmerize-core/mesmerize_core/caiman_extensions/_utils.py:24, in validate.<locals>.dec.<locals>.wrapper(self, *args, **kwargs)
     22 if not self._series["outputs"]["success"]:
     23     tb = self._series["outputs"]["traceback"]
---> 24     raise BatchItemUnsuccessfulError(f"Batch item was unsuccessful, traceback from subprocess:\n{tb}")
     25 return func(self, *args, **kwargs)

BatchItemUnsuccessfulError: Batch item was unsuccessful, traceback from subprocess:
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/kushalk/repos/CaImAn/caiman/summary_images.py", line 784, in local_correlations_movie_parallel
    mv = cm.load(mv_name, subindices=idx, in_memory=True)
  File "/home/kushalk/repos/CaImAn/caiman/base/movies.py", line 1447, in load
    return load_movie_chain(file_name,
  File "/home/kushalk/repos/CaImAn/caiman/base/movies.py", line 1774, in load_movie_chain
    m = load(f,
  File "/home/kushalk/repos/CaImAn/caiman/base/movies.py", line 1706, in load
    images = images[subindices]
  File "/home/kushalk/python-venvs/mescore/lib/python3.10/site-packages/numpy/core/memmap.py", line 334, in __getitem__
    res = super().__getitem__(index)
IndexError: index -700 is out of bounds for axis 0 with size 300
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/kushalk/repos/mesmerize-core/mesmerize_core/algorithms/mcorr.py", line 95, in run_algo
    Cns = local_correlations_movie_offline(
  File "/home/kushalk/repos/CaImAn/caiman/summary_images.py", line 773, in local_correlations_movie_offline
    parallel_result = dview.map_async(local_correlations_movie_parallel, params).get(4294967)
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 771, in get
    raise self._value
IndexError: index -700 is out of bounds for axis 0 with size 300

The issue actually comes from computing the local correlations, we set the default window_size to 1000: https://github.com/nel-lab/mesmerize-core/blob/master/mesmerize_core/algorithms/mcorr.py#L98

We should probably allow a parameter outside of "main" where these things can be set.

@kushalkolar
Copy link
Collaborator Author

this is basically done in #183

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant