Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: Draw motion vectors #469

Merged
merged 41 commits into from
Nov 20, 2024
Merged

feature: Draw motion vectors #469

merged 41 commits into from
Nov 20, 2024

Conversation

ShrimpCryptid
Copy link
Contributor

@ShrimpCryptid ShrimpCryptid commented Nov 5, 2024

Problem

Closes #42, "visualize XY movement delta." This PR gets a functional implementation of vector lines into TFE! They can now be toggled on and off, and parameters like smoothing and scaling can be edited from a new settings area.

There are a number of spin-off issues that I've created that will be follow-ups to this PR.

Solution

This is a big one, but I think it's split into relatively understandable chunks! I'll be sending a meeting invite to whoever's assigned to this PR so I can walk through it.

  • Added math logic for calculating the motion delta of all objects, with smoothing and thresholding applied.
  • Added a new async worker task for calculation of motion deltas.
  • Added a React hook for managing/syncing motion delta state.
  • Added a settings area for configuring vector visualization.
  • Added the new VectorField class, which manages a Three.JS line used for rendering the vector arrows.
    • VectorField calculates the vertices for ALL objects across ALL timepoints into a big buffer, then only renders slices of it based on the current frame.

Type of change

  • New feature (non-breaking change which adds functionality)

Steps to Verify:

  1. Open the PR preview link here: https://allen-cell-animated.github.io/timelapse-colorizer/pr-preview/pr-469
  2. Go to the settings tab and turn on vector arrows
  3. Try changing frames, playing through time, and adjusting vector settings.
  4. The vector arrows should stay constant size with zooming also.

Screenshots (optional):

Small-volume.25.mp4

Keyfiles (delete if not relevant):

SUGGESTED REVIEW ORDER:

  1. math_utils.ts -> Deltas are calculated here!
  2. types.ts -> Vector configuration
  3. SharedWorkerPool.ts + worker.ts -> Delta calculations can be triggered asynchronously on workers.
  4. Viewer + react_utils.ts -> Manages synchronizing react state and delta calculation
  5. ColorizeCanvas + VectorField -> actually renders the vectors to the screen
  6. VectoFieldSettings -> new settings area

@ShrimpCryptid ShrimpCryptid added the new feature New feature or request label Nov 5, 2024
@ShrimpCryptid ShrimpCryptid self-assigned this Nov 5, 2024
Copy link

github-actions bot commented Nov 5, 2024

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 75% 5226 / 6968
🔵 Statements 75% 5226 / 6968
🔵 Functions 57.69% 135 / 234
🔵 Branches 80.74% 436 / 540
File Coverage
File Stmts % Branch % Funcs % Lines Uncovered Lines
Changed Files
src/colorizer/Dataset.ts 78.74% 74.35% 72.22% 78.74% 189, 225-226, 244-245, 257-258, 285-288, 310-311, 315-317, 321-322, 334-335, 341-342, 345-360, 388-389, 393-394, 410-419, 434-435, 454-456, 466-472, 490-493, 510-516, 520-521, 525-526, 532-539, 542-546, 549-573, 580-587
src/colorizer/Track.ts 0% 100% 0% 0% 1-62
src/colorizer/types.ts 97.65% 100% 66.66% 97.65% 131-132, 228-231
src/colorizer/utils/math_utils.ts 65.35% 95.23% 53.84% 65.35% 15-16, 27-28, 157-162, 165-166, 178-197, 203-223, 237-280
src/colorizer/workers/SharedWorkerPool.ts 14.28% 100% 0% 14.28% 11-70
src/colorizer/workers/worker.ts 100% 100% 100% 100%
src/components/Export.tsx 62.27% 42.3% 28.57% 62.27% 167-171, 185-187, 200-201, 212-222, 230-248, 255-256, 260-264, 273-344, 365-381, 384-412, 416, 418-419, 427-437, 517-518, 537-578, 586-605, 623-624, 634-635, 643-644
src/components/SettingsContainer.tsx 96.27% 50% 100% 96.27% 29-30, 144-147
Generated in workflow #1235

Copy link

github-actions bot commented Nov 5, 2024

PR Preview Action v1.4.8
Preview removed because the pull request was closed.
2024-11-20 01:48 UTC

@ShrimpCryptid ShrimpCryptid force-pushed the feature/motion-vectors branch from 238398b to cca9dba Compare November 8, 2024 20:54
@ShrimpCryptid ShrimpCryptid marked this pull request as ready for review November 12, 2024 00:07
@ShrimpCryptid ShrimpCryptid requested a review from a team as a code owner November 12, 2024 00:07
@ShrimpCryptid ShrimpCryptid requested review from meganrm and rugeli and removed request for a team November 12, 2024 00:07
Comment on lines +148 to +157
export const VECTOR_KEY_MOTION_DELTA = "_motion_";

export const getDefaultVectorConfig = (): VectorConfig => ({
visible: false,
key: VECTOR_KEY_MOTION_DELTA,
timeIntervals: 5,
color: new Color(0x000000),
scaleFactor: 4,
});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move to constants?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to leave this for a follow-up PR because otherwise it'll make the code diff messier

@ShrimpCryptid ShrimpCryptid marked this pull request as draft November 14, 2024 18:14
@ShrimpCryptid ShrimpCryptid marked this pull request as ready for review November 18, 2024 19:18
Comment on lines +127 to +132
// o <- normVectorEnd
// /|\
// / | \ <- arrow head maintains onscreen pixel length with zoom
// |
// |
// o <- normVectorStart
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is such a great comment!

Copy link

@rugeli rugeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm, great work!

Copy link
Contributor

@meganrm meganrm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, I made a couple suggestions just to shorten your functions but you can take it or leave it

@ShrimpCryptid ShrimpCryptid merged commit 5cffd70 into main Nov 20, 2024
2 checks passed
@ShrimpCryptid ShrimpCryptid deleted the feature/motion-vectors branch November 20, 2024 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Visualize XY movement delta
3 participants