Updating demo notebook for training (#100)

* first pass on VSCyto2D demo

* updating notebooks

* draft modifications updating 2DUnet to UNeXT2

* editing solution

* modifying to relative paths based on setup.sh download paths.

* small edits

* fixing video and adding todo comment for students to switch the top_dir.

* updating the readme and setup.sh

* addressing some typos

* html to md (#106)

* typographical

* nits

* -adding alias for UNeXt2_2D
-fixing bug fluor-phase
-addingtensorboard instructions to open on browser
-fixing the markdowns to render properly on vscode jupyter

* ruff and black

* edit tensorboard instructions

* clean up the order for Part 2: fluor2phase, training call follows model construction

* restructured fluor2phase sections

* Text edits

* markdown and html fixes

* merge import blocks

* add line break

* tweak typesetting

* use the actual pi
and more line breaks

* rotate 'around'

* addaptive figure size

* remove division

* FOVs -> samples

* line breaks

* remove duplicate function and more line breaks

* send data to device

* fix multisamping in example data for graphing

* fix definition location

* fix device and markdown block

* reordering variable that prevented fluor2phase not to run if previous cells were not run

* fixing vscode and juypter readability

* remove old debug script

* fix syntax

* fix hyperlink

* fix batch size story

* remove emojis

* making 3D_UNeXt2 example work

* pointing to config files from 0.1.0 release

---------

Co-authored-by: Shalin Mehta <[email protected]>
Co-authored-by: Ziwen Liu <[email protected]>
Co-authored-by: Ziwen Liu <[email protected]>

examples/demo_dlmbl/README.md

@@ -1,6 +1,6 @@
-# Exercise 4: Image translation
+# Exercise 6: Image translation - Part 1
 
-This demo script was developed for the DL@MBL 2023 course by Ziwen Liu and Shalin Mehta, with many inputs and bugfixes by [Morgan Schwartz](https://github.com/msschwartz21), [Caroline Malin-Mayor](https://github.com/cmalinmayor), and [Peter Park](https://github.com/peterhpark).
+This demo script was developed for the DL@MBL 2024 course by Eduardo Hirata-Miyasaki, Ziwen Liu and Shalin Mehta, with many inputs and bugfixes by [Morgan Schwartz](https://github.com/msschwartz21), [Caroline Malin-Mayor](https://github.com/cmalinmayor), and [Peter Park](https://github.com/peterhpark).  
 
 
 
@@ -9,10 +9,10 @@ This demo script was developed for the DL@MBL 2023 course by Ziwen Liu and Shali
 
 Make sure that you are inside of the `image_translation` folder by using the `cd` command to change directories if needed.
 
-Make sure that you can use mamba to switch environments.
+Make sure that you can use conda to switch environments.
 
 ```bash
-mamba init
+conda init
 ```
 
 **Close your shell, and login again.** 
@@ -23,7 +23,7 @@ sh setup.sh
 ```
 Activate your environment
 ```bash
-mamba activate 04_image_translation
+conda activate 06_image_translation
 ```
 
 ## Use vscode
@@ -42,7 +42,8 @@ jupyter notebook
 
 ...and continue with the instructions in the notebook.
 
-If 04_image_translation is not available as a kernel in jupyter, run
+If `06_image_translation` is not available as a kernel in jupyter, run:
+
 ```
-python -m ipykernel install --user --name=04_image_translation
+python -m ipykernel install --user --name=06_image_translation
 ```

examples/demo_dlmbl/convert-solution.py

@@ -1,15 +1,15 @@
 import argparse
-from traitlets.config import Config
-import nbformat as nbf
-from nbconvert.preprocessors import TagRemovePreprocessor, ClearOutputPreprocessor
+
 from nbconvert.exporters import NotebookExporter
+from nbconvert.preprocessors import ClearOutputPreprocessor, TagRemovePreprocessor
+from traitlets.config import Config
 
 
 def get_arg_parser():
     parser = argparse.ArgumentParser()
 
-    parser.add_argument('input_file')
-    parser.add_argument('output_file')
+    parser.add_argument("input_file")
+    parser.add_argument("output_file")
 
     return parser
 
@@ -21,15 +21,15 @@ def convert(input_file, output_file):
     c.ClearOutputPreprocesser.enabled = True
     c.NotebookExporter.preprocessors = [
         "nbconvert.preprocessors.TagRemovePreprocessor",
-        "nbconvert.preprocessors.ClearOutputPreprocessor"
+        "nbconvert.preprocessors.ClearOutputPreprocessor",
     ]
 
     exporter = NotebookExporter(config=c)
     exporter.register_preprocessor(TagRemovePreprocessor(config=c), True)
     exporter.register_preprocessor(ClearOutputPreprocessor(), True)
 
     output = NotebookExporter(config=c).from_filename(input_file)
-    with open(output_file, 'w') as f:
+    with open(output_file, "w") as f:
         f.write(output[0])
 
 
@@ -38,4 +38,4 @@ def convert(input_file, output_file):
     args = parser.parse_args()
 
     convert(args.input_file, args.output_file)
-    print(f'Converted {args.input_file} to {args.output_file}')
+    print(f"Converted {args.input_file} to {args.output_file}")

examples/demo_dlmbl/setup.sh

@@ -2,31 +2,36 @@
 
 START_DIR=$(pwd)
 
-# Create mamba environment
-mamba create -y --name 04_image_translation python=3.10
+# Create conda environment
+conda create -y --name 06_image_translation python=3.10
 
 # Install ipykernel in the environment.
-mamba install -y ipykernel nbformat nbconvert black jupytext ipywidgets --name 04_image_translation
+conda install -y ipykernel nbformat nbconvert black jupytext ipywidgets --name 06_image_translation
 # Specifying the environment explicitly.
-# mamba activate sometimes doesn't work from within shell scripts.
+# conda activate sometimes doesn't work from within shell scripts.
 
 # install viscy and its dependencies`s in the environment using pip.
 mkdir -p ~/code/
 cd ~/code/
 git clone https://github.com/mehta-lab/viscy.git
 cd viscy
-git checkout 7c5e4c1d68e70163cf514d22c475da8ea7dc3a88 # Exercise is tested with this commit of viscy
-# Find path to the environment - mamba activate doesn't work from within shell scripts.
-ENV_PATH=$(conda info --envs | grep 04_image_translation | awk '{print $NF}')
+git checkout main #FIXME: change after merging this PR   # Exercise is tested with this commit of viscy
+
+# Find path to the environment - conda activate doesn't work from within shell scripts.
+ENV_PATH=$(conda info --envs | grep 06_image_translation | awk '{print $NF}')
 $ENV_PATH/bin/pip install ".[metrics]"
 
-# Create data directory
-mkdir -p ~/data/04_image_translation
-cd ~/data/04_image_translation
-wget https://dl-at-mbl-2023-data.s3.us-east-2.amazonaws.com/DLMBL2023_image_translation_data_pyramid.tar.gz
-wget https://dl-at-mbl-2023-data.s3.us-east-2.amazonaws.com/DLMBL2023_image_translation_test.tar.gz
-tar -xzf DLMBL2023_image_translation_data_pyramid.tar.gz
-tar -xzf DLMBL2023_image_translation_test.tar.gz
+# Create the directory structure
+mkdir -p ~/data/06_image_translation/training
+mkdir -p ~/data/06_image_translation/test
+
+# Change to the target directory
+cd ~/data/06_image_translation/training
+
+# Download the OME-Zarr dataset recursively
+wget -m -np -nH --cut-dirs=4 -R "index.html*" "https://public.czbiohub.org/comp.micro/viscy/VSCyto2D/training/a549_hoechst_cellmask_train_val.zarr/"
+cd ~/data/06_image_translation/test
+wget -m -np -nH --cut-dirs=4 -R "index.html*" "https://public.czbiohub.org/comp.micro/viscy/VSCyto2D/test/a549_hoechst_cellmask_test.zarr/"
 
 # Change back to the starting directory
 cd $START_DIR