Napari-VASCilia.json

{
  "name": "Napari-VASCilia",
  "display_name": "VASCilia",
  "visibility": "public",
  "icon": "",
  "categories": [
    "Segmentation",
    "Annotation",
    "Measurement",
    "Image Processing",
    "Visualization"
  ],
  "schema_version": "0.2.1",
  "on_activate": null,
  "on_deactivate": null,
  "contributions": {
    "commands": [
      {
        "id": "Napari-VASCilia.initialize_ui",
        "title": "Initialize UI",
        "python_name": "napari_vascilia.Napari_VASCilia_v1_1_0:initialize_vascilia_ui",
        "short_title": null,
        "category": null,
        "icon": null,
        "enablement": null
      }
    ],
    "readers": null,
    "writers": null,
    "widgets": [
      {
        "command": "Napari-VASCilia.initialize_ui",
        "display_name": "VASCilia UI",
        "autogenerate": false
      }
    ],
    "sample_data": null,
    "themes": null,
    "menus": {},
    "submenus": null,
    "keybindings": null,
    "configuration": []
  },
  "package_metadata": {
    "metadata_version": "2.1",
    "name": "Napari-VASCilia",
    "version": "1.1.0",
    "dynamic": null,
    "platform": null,
    "supported_platform": null,
    "summary": "VASCilia (Vision Analysis StereoCilia): A Napari Plugin for Deep Learning-Based 3D Analysis of Cochlear Hair Cell Stereocilia Bundles",
    "description": "# VASCilia (Vision Analysis StereoCilia): A Napari Plugin for Deep Learning-Based 3D Analysis of Cochlear Hair Cell Stereocilia Bundles \n\n<p align=\"left\">\n  <img src=\"images/logo_3d.png\" alt=\"VASCilia Logo\" width=\"170\">\n</p>\n\nExplore the complexities of the cochlea with VASCilia, a Napari plugin created to aid in the 3D segmentation and quantification of stereocilia bundles. Equipped with a range of thoughtful features, VASCilia stands for (Vision Analysis StereoCilia) and it provides a supportive tool for auditory research, including:  \n1. Slice Selection: Easily navigate through 3D stacks to find the slices that matter most for your research.\n2. Stack Rotation: Adjust the orientation of your stack to facilitate better analysis.\n3. 3D Instance Segmentation: Identify and assess individual bundles with clear separation using deep learning.\n4. Bundle Deletion: Remove unwanted bundles to streamline your dataset.\n5. Regional Classification: identify whether the region is from BASE, MIDDLE, or APEX in the cochlea using deep learning.\n6. Hair Cell Differentiation: Distinguish between Inner Hair Cells and Outer Hair Cells with confidence using deep learning.\n7. Measurement Analysis: Calculate various measurements such as volume, centroid location, and surface area.\n8. Fluorescence Intensity Analysis: Assess the intensity of signal or protein with detailed precision.\n9. 3D Bundle Height Calculation: Measure the 3D distance from the peak to the base of each bundle, according to your sample's resolution.\n10. Bundle orientation: Determine bundle orientation for all hair cells based on two strategies: Height-only and Height&Distance.\n\nVASCilia &#x2764;&#xfe0f; is a valuable resource for the ear research community &#128066;, simplifying the complexity of measurement and analysis. It comes with a suite of pre-trained models to facilitate 3D segmentation, cell type identification and regional classification.\n\nFurthermore, we are committed to supporting research growth with a comprehensive training section for those looking to explore different staining techniques or develop new segmentation models through annotation and refinement.\n\nVASCilia is here to support researchers in their quest for deeper understanding and innovation in the study of cochlear structures.  \n*[click the image to see a highlights reel of the plugin](https://youtu.be/MwMOxJQ_elo)*  \n\n[![Watch the video](images/VASCilia_pipeline.png)](https://youtu.be/MwMOxJQ_elo)\n\n*[Click me to see a video demo of the entire workflow](https://youtu.be/mNPJ1g0vEW8)*  \n\n## How to install : \nSTEP1[Install WSL]:  \n1. Open the Command Prompt and install the Ubuntu 20.04 Distribution by simply copy paste this command  \nwsl --install -d Ubuntu-20.04\n2. After the setup successfully completes, reboot your computer. Open Ubuntu by typing \"Ubuntu\" in the search bar. A pop-up window for Ubuntu will appear. To check if CUDA and the GPU are correctly installed and available, type nvidia-smi in the terminal  \n\nSTEP2[Download the deep learning trained models]:\n1. Download the VASCilia_trained_models from https://www.dropbox.com/scl/fo/jsvldda8yvma3omfijxxn/ALeDfYUbiOuj69Flbc728rs?rlkey=mtilfz33qiizpul7uyisud5st&st=41kjlbw0&dl=0 \nnow you should have a folder called 'models'\n\n- \ud83d\udcc1 **models** `[Trained models]`\n    - \ud83d\udcc1 **cell_type_identification_model** `[has weights for cell type identification IHC vs OHC]`\n    - \ud83d\udcc1 **new_seg_model** `[incase you fine tune the existing model, the new model will be stored here]`\n    - \ud83d\udcc1 **region_prediction** `[has weights for region prediction]`\n    - \ud83d\udcc1 **seg_model**  `[has the weights for the 3D instance segmentation model]`\n    - \ud83d\udcc1 **Train_predict_stereocilia_exe** `[executible needed by the plugin to segment and retrain the model using WSL]`  \n    - \ud83d\udcc1 **ZFT_trim_model** `[deep learning model weights for z focus tracker algorithm]`  \n    - \ud83d\udcc1 **rotation_correction_model** `[deep learning model weights for correcting the orientation of the stack]`  \n \nSTEP3[download one dataset to test VASCilia]:  \ndownload one sample from our datasets to try in this link https://www.dropbox.com/scl/fo/pg3i39xaf3vtjydh663n9/h?rlkey=agtnxau73vrv3ism0h55eauek&dl=0  \ncreate a folder, called raw_data folder and put the downloaded dataset inside the raw_data folder\n\n  - \ud83d\udcc1 **raw_data** `[raw data (stacks) is placed here]`\n    - \ud83d\udcc4 Litter 12 Mouse 1 MIDDLE - delBUNdelCAP_Airyscan Processing.czi\n   \nAlso create another folder called processed_data in which the plugin will use to store the results of the analysis\n  \n  - \ud83d\udcc1 **processed_data** `[processed data will be stored here]`\n\n## Instructions for Cloning the Repository [You can do either Option A or Option B]:\n## Option A: Cloning the Repository:  \n```sh\ngit clone https://github.com/ucsdmanorlab/Napari-VASCilia.git\ncd Napari-VASCilia\nconda create -y -n napari-VASCilia -c conda-forge python=3.10    \nconda activate napari-VASCilia    \npip install -r requirements.txt\npip install -e .\nnapari  \n```\n## Option B: Installing via PyPI:\n```sh\nconda create -y -n napari-VASCilia -c conda-forge python=3.10    \nconda activate napari-VASCilia \n# Download the requirements.txt file from this repository and ensure you have it in your working directory. \npip install -r requirements.txt\npip install Napari-VASCilia\nnapari  \n```\nPost-installation:  \n1. Activate the plugin through Plugins -> VASCilia UI (Napari-VASCilia).\n2. This will generate the config.json file at C:/Users/Username/.napari-vascilia/config.json. Update the paths in config.json as needed.\nconfig.json will be generated upon running the plugin for the first time.\n\n- \ud83d\udcc1 C:/Users/Username/   [your home folder]\n  - \ud83d\udcc1 **.napari-vascilia** `[Folder_path]`\n    - \ud83d\udcc4 **config.json**\n\nPlease update the /.../ portion according to your paths:\n\n```sh\n{\n\"rootfolder\": \"C:/Users/.../processed_data/\",\n\"wsl_executable\": \"C:/Users/.../models/Train_predict_stereocilia_exe/Train_Predict_stereocilia_exe_v2\",\n\"model\": \"C:/Users/.../models/seg_model/stereocilia_v7/\",\n\"model_output_path\": \"C:/Users/.../models/new_seg_model/stereocilia_v8/\",\n\"model_region_prediction\": \"C:/Users/.../models/region_prediction/resnet50_best_checkpoint_resnet50_balancedclass.pth\",\n\"model_celltype_identification\": \"C:/Users/.../models/cell_type_identification_model/\",\n\"model_ZFT_prediction\": str('C:/Users/..../models/ZFT_trim_model/'),\n\"model_rotation_prediction\": str('C:/Users/..../models/rotation_correction_model/'),\n\"flag_to_resize\": false,\n\"flag_to_pad\": false,\n\"resize_dimension\": 1200,\n\"pad_dimension\": 1500,\n\"button_width\": 60,\n\"button_height\": 22\n}\n```\n\nCongratulations :) &#127881;, now you can enjoy working with the plugin. \n\n## Unique about VASCilia :  \nVASCilia saves all the intermediate results and the variables inside a pickle file while the user is using it in a very effiecint way. That allows a super fast uploading for the analysis if the user or their supervisor wants to keep working or review the analysis steps.  \n*[Click me to learn how to upload a z-stack](https://youtu.be/Sxm_fsjoWL0)*  \n\n## How to use VASCilia :  \n*[Click me to see a video demo of the entire workflow](https://youtu.be/mNPJ1g0vEW8)*  \n\nThere are several buttons inside the blugin in the right hand side of Napari:\n\n1. 'Open CZI Cochlea Files and Preprocess' button: read the CZI file.\n2. 'Upload Processed CZI Stack' button: Incase you already have processed the stack, then just uplead your Analysis_state.pkl that usually has all the variables needed to upload your analysis\n3. 'Trim Full Stack' button: this button allows you to choose only the slices of interest (has been automated in v_1_1_0)\n4. \"Rotate' buttom: this button allows to rotate the stack to have proper analysis (has been automated in v_1_1_0)  \n5. Segment with 3DBundleSeg: it is a two steps algorithm (2D detection + multi-object assignment algorithm across all slices) to produce robust 3D detection. 3DBundleSeg is the first instance segmentation model for stereocilia bundles in the literature. It is trained on P5 and P21 3D stacks (thousands of 2D instances) and it produces highly acccurate boundary delineation even in the most challenging datasets. Here are some examples:  \n\n<p align=\"center\">\n  <strong>3DBundleSeg can tackle challenged cases</strong>  \n  <br>\n  <img src=\"images/challenged_cases_gray.png\" width=\"100%\">\n</p>\n\n<p align=\"center\">\n  <strong>Multi-object assignment algorithm to produce robust 3D detection</strong>  \n  <br>\n  <img src=\"images/3DBundleSeg.png\" width=\"100%\">\n</p>\n\n\n6. Delete Label 'button': delete the unwanted detection if it is near the boundary or for any other reason.\n7. Calculate measurments 'button': calculate different measurments from the detected bundles and store them in csv file\n8. Calculate Bundle Height 'button': compute the 3D distance from the highest point in the 3D detection of each bundle to it's base. This calculation will consider the sample resolution.\n9. Perform Cell Clustering 'button': find the IHC, OHC1, OHC2, and OHC3 using either GMM, Kmeans or Deep Learning. Those layers will be added to the plugin to be used during the analysis. \n10. Compute Fluorescence Intensity 'button': produce plots and CSV files that has the accumelated intensity and mean intensity for the fluorescence signal.\n11. Predict Region 'button': Predict whether the region is from the BASE, MIDDLE, or APEX region using a RESNET50 trained model. \n12. Compute Orientation: It computes the orientation using two strategies.\n\n<p align=\"center\">\n  <strong>Bundle Height with top and bottom adjustable points in red and green, orientation with two points in magenta, and bundle ID in green</strong>  \n  <br>\n  <img src=\"images/Bundles.png\" width=\"50%\">\n</p>\n\n<p align=\"center\">\n  <strong>Cell type identification (IHC1 in yellow, OHC1 in cyan, OHC2 in green, and OHC3 in magenta)</strong>  \n  <br>\n  <img src=\"images/clustering.png\" width=\"50%\">\n</p>\n\n13. Training Section.\n\n<p align=\"center\">\n  <strong>Training section</strong>  \n  <br>\n  <img src=\"images/Training_section2.png\" width=\"40%\">\n</p>\n\nThe training section is for the research ear community incase their datasets are little different than ours then they can easily create their cround truth, train a new model and use it in the plugin\n1. Create/Save Ground Truth 'button': this button will create a new layer to draw new ground truth and save them as variables inside the plugin\n2. Generate Ground Truth Mask 'button': this button will save all the generated masks after finish annotating to a new folder. \n3. Display Stored Ground Truth 'button': this button will display the stored masks in the plugin.\n4. Copy Segmentation Masks to Ground Truth 'button': this button helps in speeding up the annotation process by copying what our trained model is producing sothat the annotator will only correct the wrong part.\n5. Move Ground Truth to Training Folder 'button': this button will move all the annotated ground truth to the training folder to start the training process. \n6. Check Training Data 'button': this button checks the training data whether they follow the format needed by the architecture. It checks whether there are training and valiation folders and it reads every single file to make sure it doesn't have redundant or no information. It gives warning messages incase it finds an issue.\n7. Train New Model for 3DBundleSeg 'button': this button will start the training.\n\nVASCilia also equipped with two more buttons for resetting (to facilitate transitions between analyzing several stacks) and also exit VASCilia.  \nWe are still working on the documentation, so this gihub will be continiuosly updated.\n\n## Testing Other Lab Data  \nLiberman Data *[Click me to see a video demo of the entire workflow](https://youtu.be/PIG3q7G6Xr0)*  \nArtur Indzhykulian Data *[Click me to see a video demo of the entire workflow](https://youtu.be/WseYK4Zn-3o)*  \n\n## Paper and Citation\n\nThis work will be submitted very soon. If you want to read or cite the paper &#128522;, you can find it [here](https://doi.org/10.1101/2024.06.17.599381).  \n\nKassim, Y. M., Rosenberg, D. B., Renero, A., Das, S., Rahman, S., Al Shammaa, I., Salim, S., Huang, Z., Huang, K., Ninoyu, Y., Friedman, R. A., Indzhykulian, A. A., & Manor, U. (2024). VASCilia (Vision Analysis StereoCilia): A Napari Plugin for Deep Learning-Based 3D Analysis of Cochlear Hair Cell Stereocilia Bundles. bioRxiv. https://doi.org/10.1101/2024.06.17.599381\n\n## Project Authors and Contacts\n\n**Python Implementation of this repository:** Dr. Yasmin M. Kassim    \n**Contact:** ykassim@ucsd.edu, ymkgz8@mail.missouri.edu  \nYasmin Kassim was responsible for the plugin design, fully implemented all functions in Python, wrote the manuscript,\nproofread the ground truth data, created all figures, and established the GitHub repository and codebase.\n\n**Stacks used in this study imaged by:** Dr. David Rosenberg   \n\n**Height bundle ground truth analyses**: Samprita Das and Alma Renero.  \n\n**StereoCilia Bundles Ground Truth**: 55 (P5 and P21) 3D stacks were manually annotated by Yasmin Kassim and five undergraduate students using the CVAT annotation tool. This is an extremely challenging process, as each 3D stack might have up to 60 bundles in a 3D setting, which could translate to around 1000 bundles in a 2D setting across all frames. The students involved in this effort are:  \n**Samia Rahman, Ibraheem Al Shammaa, Samer Salim, Zhuoling Huang, and Kevin Huang**.\n\nThis dataset will be the first annotated dataset in the literature to 3D segment the stereocilia bundles and it will be published and available for the ear research community with the publication of this paper.\n\n**Other Lab Support**:   \nYuzuru Ninoyu assisted with some of the imaging data, with Rick Friedman\u2019s supervision and support.   \nArtur Indzhykulian provided additional imaging data for testing.  \n\n**Lab Supervisor:** Dr. Uri Manor   \nThe Principal Investigator, conceived and supervised the project, and provided critical\nrevisions and updates to the manuscript.  \n\n**Contact:** u1manor@UCSD.EDU  \n**Department:** Cell and Development Biology Department/ UCSD  \n**Lab Website:** https://manorlab.ucsd.edu/\n\n\n\n\n",
    "description_content_type": "text/markdown",
    "keywords": null,
    "home_page": "https://github.com/ucsdmanorlab/Napari-VASCilia",
    "download_url": null,
    "author": "Yasmin Kassim",
    "author_email": "ymkgz8@mail.missouri.edu",
    "maintainer": null,
    "maintainer_email": null,
    "license": "\nCopyright (c) 2024, Yasmin Kassim\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are met:\n\n* Redistributions of source code must retain the above copyright notice, this\n  list of conditions and the following disclaimer.\n\n* Redistributions in binary form must reproduce the above copyright notice,\n  this list of conditions and the following disclaimer in the documentation\n  and/or other materials provided with the distribution.\n\n* Neither the name of copyright holder nor the names of its\n  contributors may be used to endorse or promote products derived from\n  this software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE\nFOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\nDAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\nSERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\nCAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\nOR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n",
    "classifier": [
      "Development Status :: 2 - Pre-Alpha",
      "Framework :: napari",
      "Intended Audience :: Developers",
      "License :: OSI Approved :: BSD License",
      "Operating System :: OS Independent",
      "Programming Language :: Python",
      "Programming Language :: Python :: 3",
      "Programming Language :: Python :: 3 :: Only",
      "Programming Language :: Python :: 3.9",
      "Programming Language :: Python :: 3.10",
      "Programming Language :: Python :: 3.11",
      "Programming Language :: Python :: 3.12",
      "Topic :: Scientific/Engineering :: Image Processing"
    ],
    "requires_dist": [
      "numpy",
      "tox ; extra == 'testing'",
      "pytest ; extra == 'testing'",
      "pytest-cov ; extra == 'testing'"
    ],
    "requires_python": ">=3.9",
    "requires_external": null,
    "project_url": null,
    "provides_extra": [
      "testing"
    ],
    "provides_dist": null,
    "obsoletes_dist": null
  },
  "npe1_shim": false
}