Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nextflow-create-docker suggestions #31

Closed
wants to merge 1 commit into from
Closed

nextflow-create-docker suggestions #31

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions docs/nextflow-create-docker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ This guide is for users who want to build Docker containers for use in Gen3 work

Gen3 offers a collection of FedRAMP security-compliant base images. Building on these base images makes it easier for your customized Docker image to pass the security scanning.

You can access these images on on Quay.io, a repository site for Docker images:
You can access these images on Quay.io, a repository site for Docker images:

[https://quay.io/repository/cdis/containers?tab=tags&tag=latest](https://quay.io/repository/cdis/containers?tab=tags&tag=latest)
[https://quay.io/repository/cdis/containers?tab=tags](https://quay.io/repository/cdis/containers?tab=tags)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I accepted this change but in a new PR


### How to choose your base image

Expand All @@ -32,6 +32,7 @@ Some tools you may be using in your workflow can take advantage of GPU capacity
We have 2 images in our current selection that offer [CUDA](https://www.turing.com/kb/understanding-nvidia-cuda) support for running on GPUs -- these have "cuda" in the image name, followed by the CUDA version. When possible, please choose the latest version of CUDA compatible with your tools.

> gen3-cuda-12.3-ubuntu22.04-openssl *(preferred)*
>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i accepted this change, but in a new PR

> gen3-cuda-11.8-ubuntu22.04-openssl *(only use if your tools require a lower version of CUDA)*

**CPU images**
Expand All @@ -56,7 +57,7 @@ Before you proceed with this command in your Dockerfile, you want to make sure y

First, open your Docker Desktop application (just to be sure Docker is running).

Next, open your terminal. Paste the fetch tag command you copied from Quay. If it's working, you will see language that it is pulling (see below). When it's complete (and successfully pulled), there will be a line that says `Status: Downloaded <image>` (see yellow highlight below). If you see this, you know that all the steps necessary to pull your image work. If you don't see this, reach out to us on Slack.
Next, open your terminal. Paste the fetch tag command you copied from Quay. If it's working, you will see language that it is pulling (see below). When it's complete (and successfully pulled), there will be a line that says `Status: Downloaded <image>` (see yellow highlight below). If you see this, you know that all the steps necessary to pull your image work. If you don't see this, reach out to support at [email protected] or on [Slack](https://docs.google.com/forms/d/e/1FAIpQLSczyhhOXeCK9FdVtpQpelOHYnRj1EAq1rwwnm9q6cPAe5a7ug/viewform).
smvgarcia marked this conversation as resolved.
Show resolved Hide resolved

![Test fetch tag command in terminal](img/test-fetch-tag.png)

Expand All @@ -78,7 +79,7 @@ Once you are signed in to Docker, you can run the command they suggest after pul

You can run the next suggested command (shown in red box above, `docker scout cves...`) to see the full list of vulnerabilities.

Images will be able to pass Gen3 security scanning if there are no Critical or High vulnerabilities, and **[add something about CVSS?]**
Images should be able to pass Gen3 security scanning if there are no Critical vulnerabilities.
smvgarcia marked this conversation as resolved.
Show resolved Hide resolved

*Want to know more about Docker Scout? [Check out the documentation](https://docs.docker.com/scout/quickstart/).*

Expand All @@ -98,7 +99,17 @@ In our example here, we will have you build your image using a `requirements.txt

Our example will use the files in the [torch_cuda_test directory](https://github.com/uc-cdis/bio-nextflow/tree/master/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test) of the bio-nextflow repository. You can review the `readme` file in this directory for more information. It is a simple example that will build up from our base image by adding PyTorch. The Nextflow script will ultimately use a python script that checks the version of CUDA in the GPU instance and checks whether it is compatible with the version of PyTorch and CUDA available in the container.

First, in the terminal, navigate to the directory where the downloaded Dockerfile and requirements.txt are located.
First, download the contents of the [torch_cuda_test directory](https://github.com/uc-cdis/bio-nextflow/tree/master/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test). If you have `git`, the easiest way is to clone the repository:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this -- I put in the prerequisites that you should have already cloned the repo, so I would rather not put them here. I can add these steps up there, though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha, i still think "navigate to the directory where the downloaded Dockerfile and requirements.txt are located" is a bit out of the blue here, the prerequisite are a lot of text ago

maybe "from the directory where you cloned or downloaded the bio-nextflow repository as a prerequisite, navigate to the torch_cuda_test directory" and keep the cd command i added

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated this info to reflect your suggestions in a new PR

```
git clone [email protected]:uc-cdis/bio-nextflow.git
or
git clone https://github.com/uc-cdis/bio-nextflow.git
```

Then, in the terminal, navigate to the directory where the downloaded Dockerfile and requirements.txt are located.
```
cd bio-nextflow/nextflow_notebooks/containerized_gpu_workflows/torch_cuda_test
```

> Note that the first line of the Dockerfile references the fetch tag for one of our GPU base images. This is always how you will reference a base image -- with `FROM` and the Dockertag.
smvgarcia marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
Loading