Support other layer types

[lazigu]

Todo list:

• add documentation
• add example data (or add recommendation to use the plugin with nppas that already has example data)
• implement support for points layer

[codecov]

Codecov Report

Attention: 70 lines in your changes are missing coverage. Please review.

Comparison is base (96b8ad9) 75.88% compared to head (9c7fb83) 76.52%.
Report is 11 commits behind head on main.

Files	Patch %	Lines
napari_clusters_plotter/_plotter.py	51.04%	47 Missing ⚠️
napari_clusters_plotter/_utilities.py	63.88%	13 Missing ⚠️
...pari_clusters_plotter/_dimensionality_reduction.py	52.63%	9 Missing ⚠️
napari_clusters_plotter/_clustering.py	88.88%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #229      +/-   ##
==========================================
+ Coverage   75.88%   76.52%   +0.63%     
==========================================
  Files          14       16       +2     
  Lines        1667     1853     +186     
==========================================
+ Hits         1265     1418     +153     
- Misses        402      435      +33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lazigu]

Hi Robert @haesleinhuepf,

I will hand over this PR to you now. This is our hackathon's branch with all merge conflicts fixed, and up to date with the main branch (it is not up to date only with histogram1d branch, which has an open PR now).

I have also added a couple of tests for surfaces. Now support for the points layer could be added or just documentation regarding surfaces, as you see fit :)

The name of this branch is suppor-other-layer-types-new (excuse the typo)
Our hackathon's branch was support-other-layer-types2 (now very outdated)
And the branch support-other-layer-types was your initial trial at support for surfaces implementation.
These old branches are not deleted yet just in case, so adding this here for clarification.

[jo-mueller]

Hi @zoccoler @thawn @Cryaaa @haesleinhuepf @stefanhahmann @marabuuu , sorry for spaming - would any of you have some time to give this not a deeper look, but at least try whether it works on your end/for your usecase?

I am afraid that the clusters plotter is falling into a bit of disrepair at the moment. With more features coming in (e.g. PR #257 ) and this one having rather extensive changes, I think it would be helpful for other contributors to move their contributions forward to see this through so that others can adapt.

Thanks!

[Cryaaa]

Hey @jo-mueller,
firstly thanks for taking care of this huge PR and making it work. I definitely agree that this should be merged before anything else. I can test it with some of the previous functionality checks I still have but do you maybe have some toy datasets for other layer types that I can just check on my end? Maybe you have something, which you used which is easy for me to download and try?

[jo-mueller]

@Cryaaa
A few samples I could imagine to work well:

• 2D timelapse: https://github.com/BiAPoL/PoL-BioImage-Analysis-TS-Early-Career-Track/blob/main/data/CalibZAPWfixed_000154_max.tif
  For this I think you'll first have to convert it into a 2D+t stack (there should be something in the Utilities in the Tools menu for this) and then do a measurement for all timepoints (I also think there's a specific measurement function for this in napari-skimage regionprops)
• 3D single timepoint: cells3d?
• 2D single timepoint: human mitosis?

[Cryaaa]

@jo-mueller
Actually I meant some points or Surface data examples if we have any. I have quite some test cases for the original labels layer and will try those tomorrow!

[stefanhahmann]

Resolves #103 and #118

Test procedure:

• Load testing data into clusters plotter
• Make a free hand clustering
• See if a new layer with the clustered data appears

Test cases:

• "Manual" Test Points Layer (new)
• "Manual" Test Surface Layer (new)
• "Manual" Test 4D pseudo Labels - "old" tracking data (existing)
• "Manual" Test Vectors (new)
• Test 2D Labels, e.g. test with built-in Napari stuff (Human Mytosis)

[jo-mueller]

Code snippets for test data

For points

import numpy as np

n_points = 100
points = np.random.rand(n_points, 2)
feature1 = np.random.normal(size=n_points)
feature2 = np.random.normal(size=n_points, loc=1)
annotations = 1 * (np.random.uniform(size=100) > 0.5)

viewer.add_points(points, properties={"feature1": feature1, "feature2": feature2})

For points timelapse:

import numpy as np
import pandas as pd

n_points = 100
points = np.random.rand(n_points, 4)
time_frame = np.array([0,1]).repeat(50)
points[:, 0] = time_frame
feature1 = np.random.normal(size=n_points)
feature2 = np.random.normal(size=n_points, loc=1)
df = pd.DataFrame({'frame': time_frame, 'feature1': feature1, 'feature2': feature2})

viewer.add_points(points, features=df, size=0.1)

For surfaces

import numpy as np
import pandas as pd

# Create a random mesh
vertices = np.random.rand(100, 3)
faces = np.random.randint(0, 100, (100, 3))

feature1 = np.random.normal(size=100)
feature2 = np.random.normal(size=100, loc=1)

layer = viewer.add_surface((vertices, faces))
layer.features = pd.DataFrame({"feature1": feature1, "feature2": feature2})

For 4D surfaces

import numpy as np
import pandas as pd

vertices = np.random.rand(100, 4)
time_frame = np.array([0, 1]).repeat(50)
vertices[:, 0] = time_frame

faces1 = np.random.randint(0, 50, (50, 3))
faces2 = np.random.randint(0, 50, (50, 3))  + 50
faces = np.concatenate([faces1, faces2])

feature1 = np.random.normal(size=100)
feature2 = np.random.normal(size=100, loc=1)

layer = viewer.add_surface((vertices, faces))
layer.features = pd.DataFrame({"feature1": feature1, "feature2": feature2, 'frame': time_frame})

[Cryaaa]

A few notes on some of my tests:

• The original points and surface layers are not hidden when the cluster image is generated, while it IS hidden for original labels layers.
• For the point timelapse, current frame points are not highlighted in the plot
• For surface layers when I switch from 3D visualisation to 2D the whole notebook crashed without an error message after I made some manual clusters
• For the surface layer the visualisation is strange and does not reflect the manual clusters present (will show in person during meeting)
• For surface data dimension reduction did not work and returned empty feature data

Otherwise everything I tested seemed to work (although I still need to test 3D images and time lapses)

[jo-mueller]

Hi @Cryaaa ,

thanks for taking a look at this. I added a little change (8b5fa1d) that essentially just sets the current_frame and frame_id for pointsdata in the 3D PointsData example and it gives me this:

Which I suppose it is supposed to look like 💪

[jo-mueller]

And 8d47668 does the same for 4D surface data:

[jo-mueller]

For the surface layer the visualisation is strange and does not reflect the manual clusters present (will show in person during meeting)

Maybe one side note: The peculiar thing here is that the features to color the mesh are stored per vertex, but the color is shown for a triangle. So it can happen that a face of the mesh is neighbor to three vertices with a different value to visualize on each, which creates the odd color gradient on the surface above.

[Cryaaa]

Hey @jo-mueller

Which I suppose it is supposed to look like 💪

Yep that looks good now!

And 8d47668 does the same for 4D surface data:

Perfect!

For the surface layer the visualisation is strange and does not reflect the manual clusters present (will show in person during meeting)

Maybe one side note: The peculiar thing here is that the features to color the mesh are stored per vertex, but the color is shown for a triangle. So it can happen that a face of the mesh is neighbor to three vertices with a different value to visualize on each, which creates the odd color gradient on the surface above.

Its not exactly that. As you select more and more clusters with the shift button, even the clusters which you did not change (basically the orange ones) change colors in the cluster image (see video for demonstration). No idea what might be going on there though and didn't have enough brain power to investigate today :D

demonstraction.surface.bug.mp4

[jo-mueller]

Ouf ok that looks really weird 😅 it's probably something in the function that creates the colormap for surfaces, will investigate.

[Cryaaa]

@jo-mueller, @zoccoler, @stefanhahmann
I will merge this PR if all of you agree, so that we can continue with our other changes on Thursday!

[haesleinhuepf]

Great to see this merged everyone. Fantastic work! 😍