diff --git a/.github/workflows/check-jupyter.yml b/.github/workflows/check-jupyter.yml new file mode 100644 index 0000000..c47958e --- /dev/null +++ b/.github/workflows/check-jupyter.yml @@ -0,0 +1,30 @@ +name: Test Notebook + +on: + push: + branches: + - "*" + pull_request: + branches: + - "*" + +jobs: + test-notebook: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v2 + + - name: Set up Python + uses: actions/setup-python@v2 + with: + python-version: 3.8 + + - name: Install dependencies + run: | + pip install jupyter + + - name: Test notebook + run: | + jupyter nbconvert --to notebook --execute /tutorials/notebooks/LifeSciencesAPI/nextflow/*.ipynb diff --git a/.github/workflows/check_links.yml b/.github/workflows/check_links.yml new file mode 100644 index 0000000..5b3d487 --- /dev/null +++ b/.github/workflows/check_links.yml @@ -0,0 +1,28 @@ +name: Check Links + +on: + push: + branches: + - '*' + pull_request: + branches: + - '*' +jobs: + check-links: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v2 + + - name: Set up Node.js + uses: actions/setup-node@v3 + with: + node-version: 16 + + - name: Install dependencies + run: | + npm install -g markdown-link-check + + - name: Check links in Markdown files + run: find . -name '*.md' -print0 | xargs -0 -n1 markdown-link-check -q -c .markdown-link-check.json diff --git a/.markdown-link-check.json b/.markdown-link-check.json new file mode 100644 index 0000000..d8132d2 --- /dev/null +++ b/.markdown-link-check.json @@ -0,0 +1,43 @@ +{ + "ignorePatterns": [ + { + "pattern": "^#" + }, + { + "pattern": "^https://iam.nih.gov" + }, + { + "pattern": "^https://github.com/conda-forge/miniforge/releases/" + }, + { + "pattern": "^https://github.com/david-thrower-nih/DL-gwas-gcp-example" + } + ], + "replacementPatterns": [ + { + "pattern": "^/docs", + "replacement": "https://github.com/STRIDES/NIHCloudLabGCP/tree/main/docs" + }, + { + "pattern": "^/tutorials", + "replacement": "https://github.com/STRIDES/NIHCloudLabGCP/tree/main/tutorials" + }, + { + "pattern": "^/images", + "replacement": "https://github.com/STRIDES/NIHCloudLabGCP/tree/main/images" + }, + { + "pattern": "^/issues", + "replacement": "https://github.com/STRIDES/NIHCloudLabGCP/issues" + }, + { + "pattern": "^/assets", + "replacement": "https://github.com/STRIDES/NIHCloudLabGCP/tree/main/tutorials/notebooks/DL-gwas-gcp-example/assets" + } + ], + "timeout": "20s", + "retryOn429": true, + "retryCount": 5, + "fallbackRetryDelay": "30s", + "aliveStatusCodes": [200, 206] +} \ No newline at end of file diff --git a/tutorials/README.md b/tutorials/README.md index ff6929c..b2f966e 100644 --- a/tutorials/README.md +++ b/tutorials/README.md @@ -31,14 +31,14 @@ There are a lot of ways to run workflows on GCP. Here we list a few possibilitie - The simplest method is probably to spin up a Compute Engine instance, and run your command interactively, or using `screen` or, as a [startup script](https://cloud.google.com/compute/docs/instances/startup-scripts/linux) attached as metadata. - You could also run your pipeline via a Vertex AI notebook, either by splitting out each command as a different block, or by running a workflow manager (Nextflow etc.). [Schedule notebooks](https://codelabs.developers.google.com/vertex_notebook_executor#0) to let them run longer. You can find a nice tutorial for using managed notebooks [here](https://codelabs.developers.google.com/vertex_notebook_executor#0). Note that there is now a difference between `managed notebooks` and `user managed notebooks`. The `managed notebooks` have more features and can be scheduled, but give you less control about conda environments/install. -- You can interact with [Google Batch](https://cloud.google.com/batch/docs/get-started), or the [Google Life Sciences API](https://cloud.google.com/life-sciences/docs/reference/rest) using a workflow manager like [Nextflow](https://cloud.google.com/life-sciences/docs/tutorials/nextflow), [Snakemake](https://snakemake.readthedocs.io/en/stable/executing/cloud.html), or [Cromwell](https://github.com/GoogleCloudPlatform/rad-lab/tree/main/modules/genomics_cromwell). We currently have example notebooks for both [Nextflow and Snakemake that use the Life Sciences API](/tutorials/notebooks/LifeSciencesAPI/), as well as [Google Batch with Nextflow](/tutorials/notebooks/GooogleBatch/nextflow) as well as a [local version of Snakemake run via Pangolin](/tutorials/notebooks/pangolin). +- You can interact with [Google Batch](https://cloud.google.com/batch/docs/get-started), or the [Google Life Sciences API](https://cloud.google.com/life-sciences/docs/reference/rest) using a workflow manager like [Nextflow](https://cloud.google.com/life-sciences/docs/tutorials/nextflow), [Snakemake](https://snakemake.readthedocs.io/en/stable/executing/cloud.html), or [Cromwell](https://github.com/GoogleCloudPlatform/rad-lab/tree/main/modules/genomics_cromwell). We currently have example notebooks for both [Nextflow and Snakemake that use the Life Sciences API](/tutorials/notebooks/LifeSciencesAPI/), as well as [Google Batch with Nextflow](/tutorials/notebooks/GoogleBatch/nextflow) as well as a [local version of Snakemake run via Pangolin](/tutorials/notebooks/pangolin). - You may find other APIs better suite your needs such as the [Google Cloud Healthcare Data Engine](https://cloud.google.com/healthcare). - Most of the notebooks below require just a few CPUs. Start small (maybe 4 CPUs), then scale up as needed. Likewise, when you need a GPU, start with a smaller or older generation GPU (e.g. T4) for testing, then switch to a newer GPU (A100/V100) once you know things will work or you need more horsepower. ## **Artificial Intelligence and Machine Learning** Machine learning is a subfield of artificial intelligence that focuses on the development of algorithms and models that enable computers to learn from and make predictions or decisions based on data, without being explicitly programmed. Machine learning on GCP generally occurs within VertexAI. You can learn more about machine learning on GCP at this [Google Crash Course](https://developers.google.com/machine-learning/crash-course). For hands-on examples, try out [this module](https://github.com/NIGMS/COVIDMachineLearningSFSU) developed by San Francisco State University or [this one from the University of Arkasas](https://github.com/NIGMS/MachineLearningUA) developed for the NIGMS Sandbox Project. -Now that the age of **Generative AI** (Gen AI) has arrived, Google has released a host of Gen AI offerings within the Vertex AI suite. Some examples of what generative AI models are capabile of are extracting wanted information from text, transforming speech into text, generating images from describtions and vice versa, and much more. Vertex AI's [Generative AI Studio](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/generative-ai-studio) console allows the user to rapidly create, test, and train generative AI models on the cloud in a safe and secure setting. See our overview in [this tutorial](/tutorials/notebooks/GenAI/GenAIStudioGCP.ipynb). The studio also has ready-to-use models all contained with in the [Model Garden](https://cloud.google.com/vertex-ai/docs/start/explore-models). These models range from foundation models, fine-tunable models, and task-specific solutions. You are also able to use these models within a jupyter notebook and Google provides many generative AI tutorials hosted on [GitHub](https://github.com/GoogleCloudPlatform/generative-ai/tree/main). Some example they provide are [document summarization](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/language/examples/document-summarization/summarization_with_documentai.ipynb), and [Q&A](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gen-app-builder/retrieval-augmented-generation/examples/question_answering.ipynb). +Now that the age of **Generative AI** (Gen AI) has arrived, Google has released a host of Gen AI offerings within the Vertex AI suite. Some examples of what generative AI models are capabile of are extracting wanted information from text, transforming speech into text, generating images from describtions and vice versa, and much more. Vertex AI's [Generative AI Studio](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/generative-ai-studio) console allows the user to rapidly create, test, and train generative AI models on the cloud in a safe and secure setting. See our overview in [this tutorial](/tutorials/notebooks/GenAI/GenAIStudioGCP.ipynb). The studio also has ready-to-use models all contained with in the [Model Garden](https://cloud.google.com/vertex-ai/docs/start/explore-models). These models range from foundation models, fine-tunable models, and task-specific solutions. You are also able to use these models within a jupyter notebook and Google provides many generative AI tutorials hosted on [GitHub](https://github.com/GoogleCloudPlatform/generative-ai/tree/main). Some example they provide are under [language here](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/language). We created [this tutorial](/tutorials/notebooks/GenAI/langchain_on_vertex.ipynb) that uses [langchain with Vertex AI](https://python.langchain.com/docs/integrations/llms/google_vertex_ai_palm) to walk you through various Gen AI use cases, including programmatically querying a LLM, creating a custom chatbox that queries a scientific article about menopause in :monkey: (sorry no emoji for chimps), and generates code according to a user-specified prompt. @@ -111,7 +111,7 @@ NCBI BLAST (Basic Local Alignment Search Tool) is a widely used bioinformatics p - We also rewrote [this ElastBLAST tutorial](https://blast.ncbi.nlm.nih.gov/doc/elastic-blast/quickstart-gcp.html) as a [notebook](/tutorials/notebooks/elasticBLAST) that will work in VertexAI. ## **Long Read Sequence Analysis** -Long read DNA sequence analysis involves analyzing sequencing reads typically longer than 10 thousand base pairs (bp) in length, compared with short read sequencing where reads are about 150 bp in length. Oxford Nanopore has a pretty complete offering of notebook tutorials for handling long read data to do a variety of things including variant calling, RNAseq, Sars-Cov-2 analysis and much more. You can find a list and description of notebooks [here](https://labs.epi2me.io/nbindex/), or clone the [GitHub repo](https://github.com/epi2me-labs/tutorials/tree/master/tutorials). Note that these notebooks expect you are running locally and accessing the epi2me notebook server. To run them in Cloud Lab, skip the first cell that connects to the server and then the rest of the notebook should run correctly, with a few tweaks. If you are just looking to try out notebooks, don't start with these. If you are interested in long read sequence analysis, then some troubleshooting may be needed to adapt these to the Cloud Lab environment. You may even need to rewrite them in a fresh notebook by adapting the commands. +Long read DNA sequence analysis involves analyzing sequencing reads typically longer than 10 thousand base pairs (bp) in length, compared with short read sequencing where reads are about 150 bp in length. Oxford Nanopore has a pretty complete offering of notebook tutorials for handling long read data to do a variety of things including variant calling, RNAseq, Sars-Cov-2 analysis and much more. You can find a list and description of notebooks [here](https://labs.epi2me.io/nbindex/), or clone the [GitHub repo](https://github.com/epi2me-labs). Note that these notebooks expect you are running locally and accessing the epi2me notebook server. To run them in Cloud Lab, skip the first cell that connects to the server and then the rest of the notebook should run correctly, with a few tweaks. If you are just looking to try out notebooks, don't start with these. If you are interested in long read sequence analysis, then some troubleshooting may be needed to adapt these to the Cloud Lab environment. You may even need to rewrite them in a fresh notebook by adapting the commands. ## **Drug Discovery** The [Accelerating Therapeutics for Opportunities in Medicine (ATOM) Consortium](https://atomscience.org/) created a series of [Jupyter notebooks](https://github.com/ATOMScience-org/AMPL/tree/master/atomsci/ddm/examples/tutorials) that walk you through the ATOM approach to Drug Discovery. diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/00-create-new-notebook1.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/00-create-new-notebook1.png new file mode 100644 index 0000000..894f63e Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/00-create-new-notebook1.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/000-enable-apis.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/000-enable-apis.png new file mode 100644 index 0000000..4c05358 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/000-enable-apis.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/001-marketplace.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/001-marketplace.png new file mode 100644 index 0000000..fbb045a Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/001-marketplace.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/002-name-minikf.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/002-name-minikf.png new file mode 100644 index 0000000..8068370 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/002-name-minikf.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/003-pick-machine.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/003-pick-machine.png new file mode 100644 index 0000000..8395ba6 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/003-pick-machine.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/004-notebook.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/004-notebook.png new file mode 100644 index 0000000..2995a72 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/004-notebook.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/01-create-new-notebook2.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/01-create-new-notebook2.png new file mode 100644 index 0000000..5b8ae51 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/01-create-new-notebook2.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/01-r2-create-new-notebook2.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/01-r2-create-new-notebook2.png new file mode 100644 index 0000000..db6f110 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/01-r2-create-new-notebook2.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/01-r3-create-new-notebook2.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/01-r3-create-new-notebook2.png new file mode 100644 index 0000000..e812cc8 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/01-r3-create-new-notebook2.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/02-create-new-notebook3.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/02-create-new-notebook3.png new file mode 100644 index 0000000..06f5dd5 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/02-create-new-notebook3.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/03-open-notebook.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/03-open-notebook.png new file mode 100644 index 0000000..2ab5d98 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/03-open-notebook.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/04-upload-notebook-and-data.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/04-upload-notebook-and-data.png new file mode 100644 index 0000000..880532e Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/04-upload-notebook-and-data.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/05-enable-kale.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/05-enable-kale.png new file mode 100644 index 0000000..29b141f Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/05-enable-kale.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/06-pipeline-parameters.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/06-pipeline-parameters.png new file mode 100644 index 0000000..5ee06bd Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/06-pipeline-parameters.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/07-pipeline-parameters-katib.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/07-pipeline-parameters-katib.png new file mode 100644 index 0000000..a2b8dba Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/07-pipeline-parameters-katib.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/08-pipeline-step.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/08-pipeline-step.png new file mode 100644 index 0000000..441d296 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/08-pipeline-step.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/09-pipeline-metrics.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/09-pipeline-metrics.png new file mode 100644 index 0000000..22882b8 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/09-pipeline-metrics.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/10-setup-katib.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/10-setup-katib.png new file mode 100644 index 0000000..7439df1 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/10-setup-katib.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/11-r2-setup-job.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/11-r2-setup-job.png new file mode 100644 index 0000000..779bd62 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/11-r2-setup-job.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/11-setup-job.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/11-setup-job.png new file mode 100644 index 0000000..0741436 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/11-setup-job.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/12-compare.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/12-compare.png new file mode 100644 index 0000000..6cf1500 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/12-compare.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/13-parallel-coords.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/13-parallel-coords.png new file mode 100644 index 0000000..7c36105 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/13-parallel-coords.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/14-successful-katib-run.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/14-successful-katib-run.png new file mode 100644 index 0000000..be7b4ed Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/14-successful-katib-run.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/click-set-cell-kind.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/click-set-cell-kind.png new file mode 100644 index 0000000..5e5add7 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/click-set-cell-kind.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/dl-gwas-headline-1.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/dl-gwas-headline-1.png new file mode 100644 index 0000000..3c6592e Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/dl-gwas-headline-1.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/dl-gwas-headline-2.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/dl-gwas-headline-2.png new file mode 100644 index 0000000..c3a300f Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/dl-gwas-headline-2.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/enable-compute-engine.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/enable-compute-engine.png new file mode 100644 index 0000000..193ab59 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/enable-compute-engine.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/enable-service-management.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/enable-service-management.png new file mode 100644 index 0000000..4985b25 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/enable-service-management.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/n2-standard-16.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/n2-standard-16.png new file mode 100644 index 0000000..2776a65 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/n2-standard-16.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/old-11-r2-setup-job.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/old-11-r2-setup-job.png new file mode 100644 index 0000000..120d88b Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/old-11-r2-setup-job.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/old-x002-final-results-page.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/old-x002-final-results-page.png new file mode 100644 index 0000000..a21da27 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/old-x002-final-results-page.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/run-minikf-startup.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/run-minikf-startup.png new file mode 100644 index 0000000..018c4d2 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/run-minikf-startup.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/service-management-api.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/service-management-api.png new file mode 100644 index 0000000..e4d0ef5 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/service-management-api.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/service-usage-api.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/service-usage-api.png new file mode 100644 index 0000000..d07c49c Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/service-usage-api.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/ssh-link.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/ssh-link.png new file mode 100644 index 0000000..f51a3a3 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/ssh-link.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/stop-instance.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/stop-instance.png new file mode 100644 index 0000000..e039784 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/stop-instance.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/updated-pipeline-params.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/updated-pipeline-params.png new file mode 100644 index 0000000..f4453bd Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/updated-pipeline-params.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/x001-new-notebook.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/x001-new-notebook.png new file mode 100644 index 0000000..951d6d5 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/x001-new-notebook.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/x002-final-results-page.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/x002-final-results-page.png new file mode 100644 index 0000000..b1fe14d Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/x002-final-results-page.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/x003-git-clone.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/x003-git-clone.png new file mode 100644 index 0000000..6f5ea89 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/x003-git-clone.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/x004-launch-terminal.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/x004-launch-terminal.png new file mode 100644 index 0000000..17d4e95 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/x004-launch-terminal.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/x006-skip.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/x006-skip.png new file mode 100644 index 0000000..015afdb Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/x006-skip.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/x007-wait.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/x007-wait.png new file mode 100644 index 0000000..0c62041 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/x007-wait.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/x008-restart-ok.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/x008-restart-ok.png new file mode 100644 index 0000000..2faeff7 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/x008-restart-ok.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/x009-preprocessing.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/x009-preprocessing.png new file mode 100644 index 0000000..49a8839 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/x009-preprocessing.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/xx-0002-pick-a-run.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx-0002-pick-a-run.png new file mode 100644 index 0000000..888224b Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx-0002-pick-a-run.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0001-navigate-to-experiment.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0001-navigate-to-experiment.png new file mode 100644 index 0000000..540af7e Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0001-navigate-to-experiment.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0003-pick-pipeline-step.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0003-pick-pipeline-step.png new file mode 100644 index 0000000..5560c63 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0003-pick-pipeline-step.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0006-step-logs.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0006-step-logs.png new file mode 100644 index 0000000..e5fb1d2 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0006-step-logs.png differ diff --git a/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0007-step-io.png b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0007-step-io.png new file mode 100644 index 0000000..42f02b7 Binary files /dev/null and b/tutorials/notebooks/DL-gwas-gcp-example/assets/xx0007-step-io.png differ diff --git a/tutorials/notebooks/SpleenLiverSegmentation/README.md b/tutorials/notebooks/SpleenLiverSegmentation/README.md index 0e1c4a3..976552f 100644 --- a/tutorials/notebooks/SpleenLiverSegmentation/README.md +++ b/tutorials/notebooks/SpleenLiverSegmentation/README.md @@ -1,14 +1,16 @@ # Spleen Segmentation with Liver Example using NVIDIA Models and MONAI _We have put together a training example that segments the Spleen in 3D CT Images. At the end is an example of combining both the Spleen model and the Liver model._ +*Nvidia has changed some of the models used in this tutorial and it may crash, if you have issues, try commenting out the liver model, we are working on a patch* + ## Introduction Two pre-trained models from NVIDIA are used in this training, a Spleen model and Liver. The Spleen model is additionally retrained on the medical decathlon spleen dataset: [http://medicaldecathlon.com/](http://medicaldecathlon.com/) Data is not necessary to be downloaded to run the notebook. The notebook downloads the data during it's run. The notebook uses the Python package [MONAI](https://monai.io/), the Medical Open Network for Artificial Intelligence. -- Spleen Model - [clara_pt_spleen_ct_segmentation_V2](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/med/models/clara_pt_spleen_ct_segmentation) -- Liver Model - [clara_pt_liver_and_tumor_ct_segmentation_V1](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/med/models/clara_pt_liver_and_tumor_ct_segmentation) +- Spleen Model - [clara_pt_spleen_ct_segmentation_V2](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/monaitoolkit/models/monai_spleen_ct_segmentation) +- Liver Model - [clara_pt_liver_and_tumor_ct_segmentation_V1]() ## Outcomes After following along with this notebook the user will be familiar with: