L1_ratio in sparseNMF

[clee162]

Operating System (Linux, MacOS, Windows): Linux
Hardware type (x86, ARM..) and RAM: x86, 376 GB RAM
Python Version (e.g. 3.9): 3.10.15
Caiman version (e.g. 1.9.12): 1.11.3
How you installed Caiman (pure conda, conda + compile, colab, ..): conda

Hello,

I'm using the sparse NMF method in v1.11.3 to segment two photon images of dendrites. In the past, our lab has used 1.8.3. We have now upgraded to 1.11.3 and noticed that while using the same parameters, our spatial components contained more noise. If I understand correctly, in v1.8.3, the L1_ratio was set to 1 but in 1.11.3 L1_ratio = 0. Does this mean the default sparseNMF has switched from using L1 loss to using L2 loss?

With our data, we noticed we were getting cleaner spatial components with less background noise when we set the L1 ratio to 0.1 instead of 0. We are considering making some modifications so that L1_ratio can be set in the parameter dictionary in the demo file and we were wondering if you had any recommendations or considerations for changing the L1 ratio?

I have included an example of the same component using v1.11.3 with L1_ratio=0 on the left and L1_ratio=1 on the right. Thank you very much for your help!

[pgunn]

I don't remember when/why we made this change; will dig through the commit history to figure out why. I hope it wasn't just a mistake - perhaps it was.

[kushalkolar]

It was changed in this PR: #1078

The value might need to be tuned per dataset, you can go ahead and use whichever value works best for you.

By the way I'm working on a new algorithm for spatial initialization, if you've got a few movies to share that could help us.

[ifittakesallnight]

Reading #1078, I see last year you were already discussing steering people to graphNMF over sparseNMF. There is no documentation of graphNMF on readthedocs. Could you point us to something? Or a paper?

[kushalkolar]

It's a matter of try and see what works best for your data, there are no hard rules.

[clee162]

Thank you for your help!

[ifittakesallnight]

Would it be worth adding l1_ratio to the init group of the parameters dictionary in the release version of caiman? So users could easily play with both l1_ratio and alpha_snmf? We've tweaked params.py and initialization.py of our installation to do exactly this.

[kushalkolar]

@clee162 how exactly did you produce the images? Are these the seeds from running sparse nmf, or is this the spatial component after CNMF?