-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a bonsai guide page to the SLEAP website #1785
base: develop
Are you sure you want to change the base?
Changes from all commits
b2930ae
d5d2b09
9c13e12
d2ec2db
2a1c587
4f8e111
182bb4c
4b798ce
5d8de25
f332499
c54bac3
331144a
fd6ddea
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,77 @@ | ||||||||
(bonsai)= | ||||||||
|
||||||||
# Using Bonsai with SLEAP | ||||||||
|
||||||||
Bonsai is a visual language for reactive programming and currently supports SLEAP models. | ||||||||
|
||||||||
:::{note} | ||||||||
Currently Bonsai supports only single instance, top-down and top-down-id SLEAP models. | ||||||||
::: | ||||||||
|
||||||||
### Exporting a SLEAP trained model | ||||||||
|
||||||||
Before we can import a trained model into Bonsai, we need to use the {code}`sleap-export` command to convert the model to a format supported by Bonsai. For example, to export a top-down-id model, the command is as follows: | ||||||||
|
||||||||
```bash | ||||||||
sleap-export -m centroid/model/folder/path -m top_down_id/model/folder/path -e exported/model/path | ||||||||
``` | ||||||||
|
||||||||
Please refer to the {ref}`sleap-export` docs for more details on using the command. | ||||||||
|
||||||||
This will generate the necessary `.pb` file and other information files required by Bonsai. In this example, these files were saved to the specified `exported/model/path` folder. | ||||||||
|
||||||||
The `exported/model/path` folder will have a structure like the following: | ||||||||
|
||||||||
``` | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Specify the language for fenced code blocks. - ```
+ ```plaintext ToolsMarkdownlint
|
||||||||
exported/model/path | ||||||||
├── centroid_config.json | ||||||||
├── confmap_config.json | ||||||||
├── frozen_graph.pb | ||||||||
└── info.json | ||||||||
``` | ||||||||
|
||||||||
### Installing Bonsai and necessary packages | ||||||||
|
||||||||
1. Install Bonsai. See the [Bonsai installation instructions](https://bonsai-rx.org/docs/articles/installation.html). | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Remove trailing spaces. - 1. Install Bonsai. See the [Bonsai installation instructions](https://bonsai-rx.org/docs/articles/installation.html).
+ 1. Install Bonsai. See the [Bonsai installation instructions](https://bonsai-rx.org/docs/articles/installation.html).
- Right click on the `FileCapture` module and select **Create Connection**. Now click on the required SLEAP module to complete the connection.
+ Right click on the `FileCapture` module and select **Create Connection**. Now click on the required SLEAP module to complete the connection. Also applies to: 67-67
ToolsMarkdownlint
|
||||||||
|
||||||||
2. Download and add the necessary packages for Bonsai to run with SLEAP. See the official [Bonsai SLEAP documentation](https://github.com/bonsai-rx/sleap?tab=readme-ov-file#bonsai---sleap) for more information. | ||||||||
|
||||||||
### Using Bonsai SLEAP modules | ||||||||
|
||||||||
Once you have Bonsai installed with the required packages, you should be able to open the Bonsai application. | ||||||||
|
||||||||
The workflow must have a source module `FileCapture` which can be found in the toolbox search in the workflow editor. Provide the path to the video that was used to train the SLEAP model in the `FileName` field of the module. | ||||||||
|
||||||||
![Bonsai FileCapture module](../_static/bonsai-filecapture.jpg) | ||||||||
|
||||||||
#### Top-down model | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ensure headings are surrounded by blank lines. - The top-down model requires both the `PredictCentroids` and the `PredictPoses` modules.
+
+ The top-down model requires both the `PredictCentroids` and the `PredictPoses` modules.
+
- The `PredictPoseIdentities` module will predict the instances with identities.
+
+ The `PredictPoseIdentities` module will predict the instances with identities.
+
- The `PredictSinglePose` module will predict the poses for single instance models.
+
+ The `PredictSinglePose` module will predict the poses for single instance models.
+
- Right click on the `FileCapture` module and select **Create Connection**. Now click on the required SLEAP module to complete the connection.
+
+ Right click on the `FileCapture` module and select **Create Connection**. Now click on the required SLEAP module to complete the connection.
+ Also applies to: 58-58, 63-63, 66-66
ToolsMarkdownlint
|
||||||||
The top-down model requires both the `PredictCentroids` and the `PredictPoses` modules. | ||||||||
|
||||||||
The `PredictCentroids` module will predict the centroids of detections. There are two fields inside the `PredictCentroids` module: the `ModelFileName` field and the `TrainingConfig` field. The `TrainingConfig` field expects the path to the training config JSON file for the centroid model. The `ModelFileName` field expects the path to the `frozen_graph.pb` file in the `exported/model/path` folder. | ||||||||
|
||||||||
![Bonsai PredictCentroids module](../_static/bonsai-predictcentroids.jpg) | ||||||||
|
||||||||
The `PredictPoses` module will predict the instances of detections. Similar to the `PredictCentroid` module, there are two fields inside the `PredictPoses` module: the `ModelFileName` field and the `TrainingConfig` field. The `TrainingConfig` field expects the path to the training config JSON file for the centered instance model. The `ModelFileName` field expects the path to the `frozen_graph.pb` file in the `exported/model/path` folder. | ||||||||
|
||||||||
![Bonsai PredictPoses module](../_static/bonsai-predictposes.jpg) | ||||||||
|
||||||||
#### Top-Down-ID model | ||||||||
The `PredictPoseIdentities` module will predict the instances with identities. This module has two fields: the `ModelFileName` field and the `TrainingConfig` field. The `TrainingConfig` field expects the path to the training config JSON file for the top-down-id model. The `ModelFileName` field expects the path to the `frozen_graph.pb` file in the `exported/model/path` folder. | ||||||||
|
||||||||
![Bonsai PredictPoseIdentities module](../_static/bonsai-predictposeidentities.jpg) | ||||||||
|
||||||||
#### Single instance model | ||||||||
The `PredictSinglePose` module will predict the poses for single instance models. This module also has two fields: the `ModelFileName` field and the `TrainingConfig` field. The `TrainingConfig` field expects the path to the training config JSON file for the single instance model. The `ModelFileName` field expects the path to the `frozen_graph.pb` file in the `exported/model/path` folder. | ||||||||
|
||||||||
### Connecting the modules | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct the spelling of "Right-click". - Right click on the `FileCapture` module and select **Create Connection**.
+ Right-click on the `FileCapture` module and select **Create Connection**. Committable suggestion
Suggested change
ToolsLanguageTool
Markdownlint
|
||||||||
Right click on the `FileCapture` module and select **Create Connection**. Now click on the required SLEAP module to complete the connection. | ||||||||
|
||||||||
![Bonsai module connection ](../_static/bonsai-connection.jpg) | ||||||||
|
||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Avoid multiple consecutive blank lines. -
ToolsMarkdownlint
|
||||||||
Once it is done, the workflow in Bonsai will look something like the following: | ||||||||
|
||||||||
![Bonsai.SLEAP workflow](../_static/bonsai-workflow.jpg) | ||||||||
|
||||||||
Now you can click the green start button to run the workflow and you can add more modules to analyze and visualize the results in Bonsai. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Use a comma before 'and' in a compound sentence. - Now you can click the green start button to run the workflow and you can add more modules to analyze and visualize the results in Bonsai.
+ Now you can click the green start button to run the workflow, and you can add more modules to analyze and visualize the results in Bonsai. Committable suggestion
Suggested change
ToolsLanguageTool
|
||||||||
|
||||||||
For more documentation on various modules and workflows, please refer to the [official Bonsai docs](https://bonsai-rx.org/docs/articles/editor.html). | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ensure the file ends with a single newline character. + Committable suggestion
Suggested change
ToolsMarkdownlint
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ensure heading levels increment by one level at a time.
Committable suggestion
Tools
Markdownlint