Deployed 4cf19e1 with MkDocs version: 1.5.3

uc-cdis · May 1, 2024 · 79f3a03 · 79f3a03
1 parent a5d45d5
commit 79f3a03
Show file tree

Hide file tree

Showing 7 changed files with 57 additions and 39 deletions.
diff --git a/nextflow-create-docker/index.html b/nextflow-create-docker/index.html
@@ -1201,16 +1201,16 @@ <h2 id="start-with-a-security-validated-base-image"><strong>Start with a securit
 <h3 id="how-to-choose-your-base-image"><strong>How to choose your base image</strong><a class="headerlink" href="#how-to-choose-your-base-image" title="Permanent link">¶</a></h3>
 <p><strong>GPU vs. CPU</strong></p>
 <p><em>Not sure what these are? <a href="https://blogs.nvidia.com/blog/whats-the-difference-between-a-cpu-and-a-gpu/">Here's a nice overview.</a></em></p>
-<p>If your workflow requires GPU (e.g., deep learning or other AI/ML models), please use the GPU instance; otherwise, use CPU.</p>
+<p>In our BRH Workspace, we offer workspace images for CPU and GPU tools. You can read more about this on our <a href="../nextflow-getting-started/#gpu-vs-cpu-nextflow-workspace-images">Getting Started page</a>. Choose the appropriate workspace image (CPU or GPU) for your Docker image and tools.</p>
 <p><strong>GPU images</strong></p>
-<p>We have 3 images in our current selection that offer <a href="https://www.turing.com/kb/understanding-nvidia-cuda">CUDA</a> support for running on GPUs -- these have "cuda" in the image name, followed by the CUDA version. When possible, please choose the latest version of CUDA compatible with your tools.</p>
+<p>We have 3 base images in our current selection that offer <a href="https://www.turing.com/kb/understanding-nvidia-cuda">CUDA</a> support for running on GPUs -- these have "cuda" in the image name, followed by the CUDA version. When possible, please choose the latest version of CUDA compatible with your tools.</p>
 <blockquote>
 <p>gen3-cuda-12.3-ubuntu22.04-openssl <em>(preferred)</em></p>
 <p>gen3-cuda-12.3-torch2.2-ubuntu22.04-openssl <em>(also preferred)</em></p>
 <p>gen3-cuda-11.8-ubuntu22.04-openssl <em>(only use if your tools require a lower version of CUDA)</em></p>
 </blockquote>
 <p><strong>CPU images</strong></p>
-<p>We have one image that is available for running workflows on CPUs.</p>
+<p>We have one base image that is available for running workflows on CPUs.</p>
 <blockquote>
 <p>amazonlinux-base</p>
 </blockquote>
@@ -1220,7 +1220,8 @@ <h3 id="test-pulling-the-docker-image"><strong>Test pulling the Docker image</st
 <p>Next, open your terminal. Run <code>docker pull &lt;image URL&gt;</code>, where the image URL is the full line as displayed in the <a href="https://github.com/uc-cdis/containers/blob/eec9789a57c5bb196a91f035e4cb069cfaa5abcd/nextflow-base-images/allowed_base_images.txt">file of security-validated base images</a>. If it's working, you will see language that it is pulling (see below). When it's complete (and successfully pulled), there will be a line that says <code>Status: Downloaded &lt;image&gt;</code> (see yellow highlight below). If you see this, you know that all the steps necessary to pull your image work. If you don't see this, reach out to us on Slack.</p>
 <p><img alt="Test docker pull command in terminal" src="../img/test-docker-pull.png"></p>
 <h3 id="test-using-docker-scout-to-evaluate-image-vulnerabilities"><strong>Test using Docker Scout to evaluate image vulnerabilities</strong><a class="headerlink" href="#test-using-docker-scout-to-evaluate-image-vulnerabilities" title="Permanent link">¶</a></h3>
-<p>At the end of your test fetch, Docker offers a suggestion to use Docker Scout to examine your image for vulnerabilities (see red box above). We have already evaluated the security compliance for our image, so it's not necessary here. However, since you will want to use Docker Scout to evaluate your custom build later, now is a convenient time to test this tool and make sure you are fully set up to run Docker Scout.</p>
+<p>At the end of your test fetch, Docker offers a suggestion to use Docker Scout to examine your image for vulnerabilities (see red box above). We have already evaluated the security compliance for our image, so it's not necessary for security here. However, since you will want to use Docker Scout to evaluate your custom build later, now is a convenient time to test this tool and make sure you are fully set up to run Docker Scout.  </p>
+<p><em>Note: If you don't seem to have access to Docker Scout, check whether you're using the <a href="#prerequisites">latest Docker version</a>.</em></p>
 <h4 id="run-docker-scout"><strong>Run Docker Scout</strong><a class="headerlink" href="#run-docker-scout" title="Permanent link">¶</a></h4>
 <p>To run Docker Scout, you must:</p>
 <ul>

diff --git a/nextflow-getting-started/index.html b/nextflow-getting-started/index.html
@@ -866,6 +866,13 @@
     Automated shutdown for idle workspaces
   </a>
 
+</li>
+
+          <li class="md-nav__item">
+  <a href="#gpu-vs-cpu-nextflow-workspace-images" class="md-nav__link">
+    GPU vs CPU Nextflow workspace images
+  </a>
+
 </li>
 
       </ul>
@@ -1180,6 +1187,13 @@
     Automated shutdown for idle workspaces
   </a>
 
+</li>
+
+          <li class="md-nav__item">
+  <a href="#gpu-vs-cpu-nextflow-workspace-images" class="md-nav__link">
+    GPU vs CPU Nextflow workspace images
+  </a>
+
 </li>
 
       </ul>
@@ -1262,6 +1276,8 @@ <h4 id="store-all-data-in-the-persistent-directory-pd">Store all data in the per
 <p><img alt="Screenshot of /pd and data folders" src="../img/workspace_pd_folder_080422.png" style="height:400px"></p>
 <h4 id="automated-shutdown-for-idle-workspaces">Automated shutdown for idle workspaces<a class="headerlink" href="#automated-shutdown-for-idle-workspaces" title="Permanent link">¶</a></h4>
 <p>Workspaces will automatically be shut down (and all workflows terminated) after <strong>90 minutes of idle time</strong>.</p>
+<h4 id="gpu-vs-cpu-nextflow-workspace-images">GPU vs CPU Nextflow workspace images<a class="headerlink" href="#gpu-vs-cpu-nextflow-workspace-images" title="Permanent link">¶</a></h4>
+<p>As you can see <a href="#3-launch-a-workspace-with-the-persistent-paymodel">in the screenshot above</a>, there are 2 Nextflow workspace images: A CPU image and a GPU image. If your workflow requires GPU (e.g., deep learning or other AI/ML models), please use the GPU instance; otherwise, use CPU. You can <a href="../nextflow-create-docker/#how-to-choose-your-base-image">read more about CPU and GPU options in Gen3 Nextflow here</a>.  </p>
 <p><a href="../nextflow-overview-containers/"><em>Continue to Overview of Containers in Gen3</em></a></p></div>
 
 

diff --git a/nextflow-overview-containers/index.html b/nextflow-overview-containers/index.html
@@ -945,15 +945,15 @@
 <div><p><a href="https://www.nextflow.io/"><img alt="Nextflow logo" src="../img/nextflow.svg" style="height:75px"></a></p>
 <h1 id="overview-developing-and-deploying-containers-in-gen3"><strong>Overview: Developing and Deploying Containers in Gen3</strong><a class="headerlink" href="#overview-developing-and-deploying-containers-in-gen3" title="Permanent link">¶</a></h1>
 <p><img alt="Overview of steps in developing a container and making it available for use in workflows" src="../img/container-development.png"></p>
-<p><strong>Locally build and test container:</strong>
-Gen3 provides several <a href="https://github.com/uc-cdis/containers/blob/eec9789a57c5bb196a91f035e4cb069cfaa5abcd/nextflow-base-images/allowed_base_images.txt">FedRAMP security-compliant base images</a> that users can pull and customize.</p>
-<p><strong>Request credentials and push container to Gen3 staging:</strong>
-Users can email Gen3 to <a href="../nextflow-request-creds/">request short-term credentials</a> that permit them to authenticate Docker in their terminal to upload the local Docker image to a Gen3 staging repo for security review.</p>
-<p><strong>Container is security-scanned; Gen3 sends approved container URI:</strong>
-Gen3 completes the security scan within minutes. If it is compliant, the image is moved to an ECR repo ("approved") from where the container can be run, and Gen3 staff will send a container URI to the user.</p>
+<p><strong>Locally build and test container:</strong></p>
+<p>Gen3 provides several <a href="https://github.com/uc-cdis/containers/blob/eec9789a57c5bb196a91f035e4cb069cfaa5abcd/nextflow-base-images/allowed_base_images.txt">FedRAMP security-compliant base images</a> that users can pull and customize.</p>
+<p><strong>Request credentials and push container to Gen3 staging:</strong></p>
+<p>Users can email Gen3 to <a href="../nextflow-request-creds/">request short-term credentials</a> that permit them to authenticate Docker in their terminal to upload the local Docker image to a Gen3 staging repo for security review.</p>
+<p><strong>Container is security-scanned; Gen3 sends approved container URI:</strong></p>
+<p>Gen3 completes the security scan of the container. Typically, the scanning completes within a couple hours; however it takes longer for larger images with more layers. If it is security-compliant, the image is moved to an ECR repo ("approved") from where the container can be run, and Gen3 staff will send a container URI to the user for use in the Nextflow workflows.</p>
 <p>If there are problems that make the image non-compliant with security requirements, a report of the vulnerabilities is provided to the user for remediation and resubmission. Users are responsible for resolving image vulnerabilities and resubmitting for scanning.  </p>
-<p><strong>Run workflow using approved container URI:</strong>
-In the BRH workspace, use a Nextflow Jupyter notebook to run Nextflow workflows in the approved container using the approved container URI. Some example notebooks can be found <a href="https://github.com/uc-cdis/bio-nextflow/tree/master/nextflow_notebooks">here</a>, and specific examples that use an approved image URI can be found <a href="https://github.com/uc-cdis/bio-nextflow/blob/master/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test/torch_cuda_batch_template.ipynb">here</a> and <a href="https://github.com/uc-cdis/bio-nextflow/blob/master/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test/README.md">here</a></p>
+<p><strong>Run workflow using approved container URI:</strong></p>
+<p>In the BRH workspace, use a Nextflow Jupyter notebook to run Nextflow workflows in the approved container using the approved container URI. Some example notebooks can be found <a href="https://github.com/uc-cdis/bio-nextflow/tree/master/nextflow_notebooks">here</a>, and specific examples that use an approved image URI can be found <a href="https://github.com/uc-cdis/bio-nextflow/blob/master/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test/torch_cuda_batch_template.ipynb">here</a> and <a href="https://github.com/uc-cdis/bio-nextflow/blob/master/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test/README.md">here</a></p>
 <hr>
 <p><a href="../nextflow-create-docker/"><em>Continue to Create Dockerfile</em></a></p></div>
 

diff --git a/nextflow-upload-docker/index.html b/nextflow-upload-docker/index.html
@@ -1280,9 +1280,10 @@ <h2 id="push-the-docker-image-to-the-ecr"><strong>Push the Docker image to the E
 <h3 id="completion"><strong>Completion</strong><a class="headerlink" href="#completion" title="Permanent link">¶</a></h3>
 <p>Once the push completes, your Docker image will be available in the ECR repository (although you will not be able to see it). It will be scanned, and if passes the security scanning, CTDS will move it to the nextflow-approved repo. When it's available in nextflow-approved, User Services will share a docker URI that looks something like this:<br>
 <code>143731057154.dkr.ecr.us-east-1.amazonaws.com/nextflow-approved/&lt; your username &gt;:&lt; image-tag &gt;</code><br>
-You can then use this new URI to run Nextflow workflows with your container in the BRH workspace.</p>
+You can then use this new URI to run Nextflow workflows with your container in the BRH workspace. (Note that you need to copy the whole URI into the container field of the nextflow notebook, as described <a href="#how-to-use-an-approved-docker-uri">in the next section</a>.)</p>
 <h2 id="how-to-use-an-approved-docker-uri"><strong>How to use an approved Docker URI</strong><a class="headerlink" href="#how-to-use-an-approved-docker-uri" title="Permanent link">¶</a></h2>
-<p>Once you have your Docker URI, you are ready to run your Nextflow workflow! You can take the Docker URI and make it the value for the "container" field(s) in your Nextflow notebook. For example, in the <code>torch_cuda_batch</code> <a href="https://github.com/uc-cdis/bio-nextflow/blob/master/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test/torch_cuda_batch_template.ipynb">Nextflow notebook</a>, you would go to the <code>nextflow.config</code> section and replace the <code>placeholder</code> value for <code>container</code> with the approved Docker URI.  </p>
+<p>Once you have your Docker URI, you are ready to run your Nextflow workflow! You can take the Docker URI (copy the entire line) and make it the value for the "container" field(s) in your Nextflow notebook. For example, in the <code>torch_cuda_batch</code> <a href="https://github.com/uc-cdis/bio-nextflow/blob/master/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test/torch_cuda_batch_template.ipynb">Nextflow notebook</a>, you would go to the <code>nextflow.config</code> section and replace the <code>placeholder</code> value for <code>container</code> with the approved Docker URI.  </p>
+<p>Please note that you will need to replace all <code>placeholder</code> values in the <code>nextflow.config</code> with values specific to your workspace. Please see the section <a href="../nextflow-tutorial-workflows/#get-and-replace-placeholder-values-from-the-nextflow-config">"Get and replace placeholder values from the Nextflow config"</a> on the Tutorials page for more information.</p>
 <p><img alt="Screenshot of nextflow.config, showing where you put the Docker URI" src="../img/nextflow-config.png"></p>
 <h2 id="support"><strong>Support</strong><a class="headerlink" href="#support" title="Permanent link">¶</a></h2>
 <p>If you encounter any issues or require assistance, please reach out to the User Services team that provided you with the temporary credentials, or <a href="mailto:[email protected]">[email protected]</a>, or reach out on Slack. (Slack will result in the quickest reply.)</p>

diff --git a/search/search_index.json b/search/search_index.json
diff --git a/sitemap.xml b/sitemap.xml
@@ -2,122 +2,122 @@
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
          <loc>https://brh.data-commons.org/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/01-home/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/02-types_of_shared_data/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/03-data_and_repos/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/04-BRH_overview/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/05-workspace_registration/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/06-loginoverview/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/07-how_to_check_request_access/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/08-discovery_page/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/09-workspace_page/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/10-profile_page/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/11-downloading_data_files/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/12-contact/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/13-workspace_accounts/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/14-lauch_with_persistent_paymodel/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/15-monitor_compute_usage/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/16-usage_exceeds_funding/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/17-workspace_faq/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/nextflow-create-docker/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/nextflow-getting-started/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/nextflow-overview-containers/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/nextflow-request-creds/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/nextflow-tutorial-workflows/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://brh.data-commons.org/nextflow-upload-docker/</loc>
-         <lastmod>2024-04-30</lastmod>
+         <lastmod>2024-05-01</lastmod>
          <changefreq>daily</changefreq>
     </url>
 </urlset>
diff --git a/sitemap.xml.gz b/sitemap.xml.gz