WandB/Colab + docs (#48)

* WANDB integration for WNet * Colab training script * Update colab_training.py * Update colab_training.py * Update colab_training.py * Update colab_training.py * Update colab_training.py * Added minimal Colab notebook * ZeroCostDL4Mic-style notebook * Added WANDB to Colab notebook * Added softmax param * Fix reset of paths in training UI * WNet visualization * Disable intensity transforms in WNet * Fixed supervised loss saving * Update view_wnet.ipynb * Update view_wnet.ipynb * Fixed filepaths * Added more checks for csv + fix eval batch size * Utils+training fixes * Colab training improvements (#49) * Update colab_wnet_training.ipynb * Update colab_wnet_training.ipynb --------- Co-authored-by: Cyril Achard <[email protected]> * Reverted include_background=True in Dice * Reintroduced best Dice channel seeking + refacto * Improve filepath messages * Fix unsup image loading when not validating * Fixed deletion of Qt imports in interface * Added auto-check for which channel has foreground Preventing background to be shown first when using WNet inference, for better UX * Fixed eval_batch_size assignment * Fix training tests * Experimental auto-discard * Improved auto-discard utility * Create experimental sliding window Voronoi-Otsu for large volumes * Disable resize error handler * Docs update & jupyter-books port (#50) * Moved docs to source folder * Updated to jupyter-books * Update GH Action to jupyter-book * Updated guides * Update reqs for jupyter-books - Fixed typo * Fix typo and file duplicate * Added auto API docs * Update welcome.rst * Update interface docstrings * Update ruff to enforce docstring style * Updated docstrings in interface/config * Add additional docstring rule exceptions * Update utils docstring * Update plugins.py * Update dev_scripts * Updated code_plugin folder docstrings * Update code_models docstrings * Update tests docstrings * Update docstring rules to enforce docs * Fix warnings when building docs * Restore docs images * Remove old API docs * Added module docstring for all public modules * Update pyproject.toml * Update folder desc * Update inference and training guides * Update inference_module_guide.rst * Guides updates * Change indexing, guides updates * Fix unindent warning * Update _toc.yml * Fixed model list inconsistencies * Update model list * Trying to fully use autosummary (see #51) * Docs update * Fix missing refactor in CRF * Update cropping_module_guide.rst * Updated docs ref for code * Figs, refs, utils guide update * Update images to figures where relevant * Updated refs to source code * Improved utils guide layout and contents * Review update + crop/utils * Review update * Small inference tweaks for pipeline * Config changes * Update worker_inference.py * Small instance seg refactor * Added properties for instance seg params * Utils cleanup * Small refactor of unclear train/val split terminology * Fix log parameters incorrectly printing def dir * Enable sliding window for all instance seg methods * Update instance_segmentation.py Refactor sliding window to not shadow builtin "function" * Fixed mistake in auto-check for WNet results * Fix for restarting train worker after force stop * Docs & images update * mv welcome image * Improved inference docs * Update inference_module_guide.rst * Attempt to fix M1 yaml * Config docstring update * Welcome page update * Docs update * Add WandB for supervised models * Added auto contrast reset in cropping * Added checks for instance labels in training * Check for custom weights before locking window inference * Improve review plots makers * Documentation update (Timokleia) (#53) * Update cropping_module_guide.rst changes in the cropping module guide * Update cropping_module_guide.rst add `` * Update inference_module_guide.rst additions to inference module * Update metrics_module_guide.rst adjusting the metrics module * Update review_module_guide.rst review module updates * Update training_module_guide.rst changes in the training module * Update training_wnet.rst additions in the wnet module * Update utils_module_guide.rst utilities updates * review parameters image * Update cropping_module_guide.rst emoji * Update metrics_module_guide.rst emoji * Update training_module_guide.rst emoji * Update utils_module_guide.rst emoji * Update cropping_module_guide.rst mild style changes * Update review_module_guide.rst adjusting image code * Update welcome.rst changes to welcome * Add files via upload * Review and training tweaks - Fixed a few links, indents, typos - Replaced some emojis and added new ones for other pages * Welcome and inference tweaks - Fixed minor formatting, tables, broken links issues, etc * RM unused model wrappers attributes * Inference edits * Update WNet and inference * Images update * Update plots_train.png * Restore walkthrough * Update detailed_walkthrough.rst * Update detailed_walkthrough.rst editions * Minor edits --------- Co-authored-by: C-Achard <[email protected]> * Added floats in small remove * Try to fix issue with float/long * Add except handling for layer removal in train * Add first version of dedicated install page - Add install page - Refacto/clean up welcome page - Add more useful plugins - Update TODO * Improve training guide * Move M1 install * Added label stats utils * Added label stats doc * Added check for dtype in stats labels * Update utils_module_guide.rst * Experimental classifier to reject bad preds on empty * Experimental classifier to reject bad preds on empty * Update classifier_test.ipynb * WIP Trying to fix some QoL issues - Better exception handling in inference - Swapaxes at mode output - Remove small in Voronoi-Otsu * Fix exception handling * Locating csv issue * Enable ruff again * Handle empty labels in csv stats * Fix call to get_dict when stats is None * Ensure consistent rotation * Fix rotation checks + auto-channel select * Fix test for review * Update installation_guide.rst (#54) refining * Minor edits to install guide * [WIP] GUI-less training example script (#52) * Add remote train scripts * mv dockerfile * Paths for remote train * Change device * Update remote_training.py * Change results path to include more info * Update colab_training.py * Delete Dockerfile.cellseg3d * Make all wandb inits point to same project * Remove deprecated call to AddChannel * Fix tests issue with MONAI 1.3.0 * Create whole_brain_utils.py Add segmentation correction scripts * Update whole_brain_utils.py * Update whole_brain_utils.py * Update whole_brain_utils.py * Update whole_brain_utils.py * Update _config.yml --------- Co-authored-by: Timokleia <[email protected]> Co-authored-by: Mackenzie Mathis <[email protected]> --------- Co-authored-by: Timokleia <[email protected]> Co-authored-by: Mackenzie Mathis <[email protected]>
AdaptiveMotorControlLab · Nov 15, 2023 · eef7eac · eef7eac
1 parent 0c68c31
commit eef7eac
Show file tree

Hide file tree

Showing 127 changed files with 10,220 additions and 3,524 deletions.
diff --git a/.github/workflows/build_docs.yml b/.github/workflows/build_docs.yml
@@ -1,19 +1,38 @@
-name: Deploy Sphinx documentation to Pages
+name: deploy
 
 on:
   push:
     branches: # branch to trigger deployment
       - main
 
+# This job installs dependencies, build the book, and pushes it to `gh-pages`
 jobs:
-  pages:
-    runs-on: ubuntu-20.04
+  build-and-deploy-book:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest]
+        python-version: [3.8]
     steps:
-    - id: deployment
-      uses: sphinx-notes/pages@v3
+    - uses: actions/checkout@v2
+
+    # Install dependencies
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v1
       with:
-        publish: false
-    - uses: peaceiris/actions-gh-pages@v3
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        pip install jupyter-book
+
+    # Build the book
+    - name: Build the book
+      run: |
+        jupyter-book build docs/
+
+    # Deploy the book's HTML to gh-pages branch
+    - name: GitHub Pages action
+      uses: peaceiris/[email protected]
       with:
         github_token: ${{ secrets.GITHUB_TOKEN }}
-        publish_dir: ${{ steps.deployment.outputs.artifact }}
+        publish_dir: docs/_build/html
diff --git a/.gitignore b/.gitignore
@@ -93,14 +93,17 @@ venv/
 # written by setuptools_scm
 **/_version.py
 
+# WANDB
+/wandb/
 
 
 ########
 #project specific
 #dataset, weights, old logos, requirements
 /napari_cellseg3d/code_models/models/dataset/
 /napari_cellseg3d/code_models/models/saved_weights/
-/docs/res/logo/old_logo/
+/docs/source/logo/old_logo/
+/docs/source/code/_autosummary/
 /reqs/
 /loss_plots/
 /wandb/
@@ -120,3 +123,8 @@ notebooks/instance_test.ipynb
 !napari_cellseg3d/_tests/res/wnet_test/lab/*.tif
 !napari_cellseg3d/_tests/res/wnet_test/vol/*.tif
 cov.syspath.txt
+
+#include docs images
+!docs/source/logo/*
+!docs/source/images/*
+napari_cellseg3d/dev_scripts/wandb
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -7,6 +7,7 @@ repos:
       - id: trailing-whitespace
       - id: check-yaml
       - id: check-added-large-files
+        args: [--maxkb=5000]
       - id: check-toml
 #  - repo: https://github.com/pycqa/isort
 #    rev: 5.12.0

diff --git a/conda/napari_cellseg3d_m1.yml b/conda/napari_cellseg3d_m1.yml
@@ -16,7 +16,7 @@ dependencies:
         - tifffile>=2022.2.9
         - imageio-ffmpeg>=0.4.5
         - torch>=1.11
-        - monai[nibabel,einops]>=0.9.0
+        - monai>=0.9.0
         - tqdm
         - nibabel
         - scikit-image

diff --git a/docs/Makefile b/docs/Makefile
diff --git a/docs/TODO.md b/docs/TODO.md
@@ -0,0 +1,8 @@
+[//]: # (
+TODO:
+- [ ] Add a way to get the current version of the library
+- [x] Update all modules
+- [x] Better WNet tutorial
+- [x] Setup GH Actions
+- [ ] Add a bibliography
+)
diff --git a/docs/_config.yml b/docs/_config.yml
@@ -0,0 +1,51 @@
+# Book settings
+# Learn more at https://jupyterbook.org/customize/config.html
+
+title: napari-cellseg3d Documentation
+author: Cyril Achard, Maxime Vidal, Timokleia Kousi, Mackenzie Mathis | Mathis Laboratory
+logo: source/logo/logo_alpha.png
+
+# Force re-execution of notebooks on each build.
+# See https://jupyterbook.org/content/execute.html
+execute:
+  execute_notebooks: force
+
+# Define the name of the latex output file for PDF builds
+latex:
+  latex_documents:
+    targetname: book.tex
+
+# Add a bibtex file so that we can create citations
+bibtex_bibfiles:
+  - references.bib
+
+# Information about where the book exists on the web
+repository:
+  url: https://github.com/AdaptiveMotorControlLab/CellSeg3d  # Online location of your book
+  path_to_book: docs  # Optional path to your book, relative to the repository root
+  branch: main  # Which branch of the repository should be used when creating links (optional)
+
+# Add GitHub buttons to your book
+# See https://jupyterbook.org/customize/config.html#add-a-link-to-your-repository
+html:
+  use_issues_button: true
+  use_repository_button: true
+
+# Add auto-generated API docs
+sphinx:
+  extra_extensions:
+  - 'sphinx.ext.napoleon'
+  - 'sphinx.ext.autodoc'
+  - 'sphinx.ext.autosummary'
+  - 'sphinx.ext.viewcode'
+  - 'sphinx.ext.autosectionlabel'
+  config:
+    add_module_names: False
+    autosectionlabel_prefix_document: True
+    autosummary_generate: True
+    autoclass_content: "both"
+    # templates_path: ['_templates']
+    exclude_patterns:
+      - '_build'
+      - '_templates'
+    #   - 'napari_cellseg3d/__pycache__'
diff --git a/docs/_templates/custom-class-template.rst b/docs/_templates/custom-class-template.rst
@@ -0,0 +1,32 @@
+{{ fullname | escape | underline}}
+
+.. currentmodule:: {{ module }}
+
+.. autoclass:: {{ objname }}
+   :members:
+   :show-inheritance:
+
+   {% block methods %}
+   {% if methods %}
+   .. rubric:: {{ _('Methods') }}
+
+   .. autosummary::
+      :nosignatures:
+   {% for item in methods %}
+      {%- if not item.startswith('_') %}
+      ~{{ name }}.{{ item }}
+      {%- endif -%}
+   {%- endfor %}
+   {% endif %}
+   {% endblock %}
+
+   {% block attributes %}
+   {% if attributes %}
+   .. rubric:: {{ _('Attributes') }}
+
+   .. autosummary::
+   {% for item in attributes %}
+      ~{{ name }}.{{ item }}
+   {%- endfor %}
+   {% endif %}
+   {% endblock %}