IndexError in update_temporal_components when running CNMF on images

[DrDaheeJang]

Operating System (Linux, MacOS, Windows): Linux
Hardware type (x86, ARM..) and RAM: x86, 376 GB RAM
Python Version (e.g. 3.9): 3.10.15
Caiman version (e.g. 1.9.12): 1.11.3
How you installed Caiman (pure conda, conda + compile, colab, ..): conda

Issue Description
I encountered an IndexError while running the CNMF pipeline on imaging data with CaImAn v1.11.3. The error occurs during the update_temporal step inside the update_temporal_components function. This seems to happen when the code attempts to delete an invalid index from an array, causing the process to terminate unexpectedly.

The error is specifically triggered when using the parameters:

alpha_snmf=0.5
L1_ratio=0.1

These settings are important as they provide a clear representation of dendritic structures with minimal background noise. The same issue does not occur with other parameter values (e.g., alpha_snmf=100, L1_ratio=1.0), where the pipeline runs successfully.

Error Log
Traceback (most recent call last):
File "RunCaimanDendritic.py", line 397, in
main()
File "RunCaimanDendritic.py", line 379, in main
cnm2, dims, param_string = run_cnmf(images, dims, dview, opts)
File "RunCaimanDendritic.py", line 231, in run_cnmf
cnm = cnm.fit(images)
File "/usr/local/bin/anaconda3/envs/caiman-1-11-3/lib/python3.10/site-packages/caiman/source_extraction/cnmf/cnmf.py", line 632, in fit
self.update_temporal(Yr, use_init=False)
File "/usr/local/bin/anaconda3/envs/caiman-1-11-3/lib/python3.10/site-packages/caiman/source_extraction/cnmf/cnmf.py", line 882, in update_temporal
self.estimates.g, self.estimates.YrA, self.estimates.lam = update_temporal_components(
File "/usr/local/bin/anaconda3/envs/caiman-1-11-3/lib/python3.10/site-packages/caiman/source_extraction/cnmf/temporal.py", line 229, in update_temporal_components
S = np.delete(S, list(ff), 0)
File "/usr/local/bin/anaconda3/envs/caiman-1-11-3/lib/python3.10/site-packages/numpy/lib/function_base.py", line 5330, in delete
raise IndexError(
IndexError: index 7 is out of bounds for axis 0 with size 5

Do you have any insights into what might be causing this issue? Is there a possible workaround or fix for this issue?

[pgunn]

What's the name of the CNMF Parameter you're using that you're calling L1_ratio here?

[DrDaheeJang]

@pgunn Thank you for your question!

The L1_ratio parameter in the code corresponds to the snmf_l1_ratio parameter within the CNMF options in CaImAn. It controls the regularization strength for the sparse NMF initialization, influencing the balance between L1 (sparsity) and L2 (smoothness) penalties.

In this script, I’ve set it to 0.1, which gives more weight to sparsity in the initialization process.

Let me know if you need further clarification!

[pgunn]

I'm not finding this within the current codebase or a sampling of older versions of the codebase.

Can you include a code snippet showing how you're setting this? It's possible that that parameter is not being consumed? If you'd like to just pastebin your RunCaimanDendridic that'd work.

[DrDaheeJang]

Our RunCaimanDendritic.py is just an adaptation of the demo. We modified params.py and initialization.py so we can just set l1_ratio like we do with alpha_snmf:
#1428 (comment)

We could upload the movie, the altered code, and the script somewhere if you'd like to try it.

[pgunn]

Oh! I had not realised you were the same person who had asked for the parameter to be threaded through to sklearn's NMF function (had actually forgotten about that request as well).

That changes things ; let me think about this further.

[pgunn]

Also, on that other request, we may eventually want to change a lot of how this works, but for now I think it probably makes sense to thread this parameter through and I'll look to do that soon.