From b9fba01480d3b2b328c54c88e4fef32f77ed1817 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Thu, 10 Oct 2024 18:06:35 +0200 Subject: [PATCH 01/13] Add wave docs Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 61 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/docs/wave.md b/docs/wave.md index 6be2711557..aaa9728ddb 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -145,6 +145,67 @@ The first repository is used to store the built container images. The second one The repository access keys must be provided as Seqera Platform credentials (see [Authenticate private repositories](#authenticate-private-repositories) above). +### Mirroring containers + +Wave allows mirroring i.e. copying containers used by your pipeline into a container registry of your choice, so that +containers are pulled by the pipeline from the target registry instead of the original registry. + +This is useful to create on-demand cache of containers images that are co-located in the same region where the pipeline +is executed, and therefore optimising cost and network efficiency. + +To enable this capability include those settings in your Nextflow configuration: + +```groovy +wave.enabled = true +wave.mirror = true +wave.build.repository = '' +tower.accessToken = '' +``` + +In the above snippet replace `` with a container registry of your choice e.g. `quay.io` (no prefix or suffix is needed) +The container will be copied with the same name, tag and checksum in the specified registry. For example, if the source +container is `quay.io/biocontainers/bwa:0.7.13--1` and the build repository setting is `foo.com`, the resulting container +name will be `foo.com/biocontainers/bwa:0.7.13--1`. + +:::{tip} +When using a path prefix in target registry name, it will be pre-pended to the resulting container name. For example +having `quay.io/biocontainers/bwa:0.7.13--1` as source container and `foo.com/bar` as build repository, the resulting +container will be named `foo.com/bar/biocontainers/bwa:0.7.13--1`. +::: + +The credentials to allow the push of containers in the target repository needs to be provided via Seqera Platform +credentials manager, whose account is specified in the `tower.accessToken` in the above configuration. + +### Container security scanning + +Wave allows scanning containers used in your pipelines for security vulnerabilities, and report an execution error +when one or more security issues are found. + +To enable this capability adds the following settings in your Nextflow configuration file: + +```groovy +wave.enabled = true +wave.scan.mode = 'required' +tower.accessToken = '' +``` + +When using these setting will only allows the use of the container in your Nextflow pipeline only if has no security +vulnerabilities. You can define the level of accepted vulnerabilities by using the `wave.scan.levels`. For example: + +``` +wave.scan.levels = 'low,medium' +``` + +The above setting will allow the use of containers having vulnerabilities with *low* and *medium* +levels [common vulnerabilities scoring system](https://en.wikipedia.org/wiki/Common_Vulnerability_Scoring_System). +Accepted values are `low`, `medium`, `high` and `critical`. + +:::{note} +Wave security scanning applies to any container used in your pipeline, irrespective it was built by Wave or accessed +through it. The container scan automatically expires after one week, if containers is accessed again after 7 days or +more the security scan is executed one more time. +::: + ### Run pipelines using Fusion file system Wave containers allows you to run your containerised workflow with the {ref}`fusion-page`. From 567f13bdecec39d116dc09d2c618314a4f6146f5 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:41:19 +0200 Subject: [PATCH 02/13] Update docs/wave.md [ci fast] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/wave.md b/docs/wave.md index aaa9728ddb..98cad48017 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -147,8 +147,7 @@ The repository access keys must be provided as Seqera Platform credentials (see ### Mirroring containers -Wave allows mirroring i.e. copying containers used by your pipeline into a container registry of your choice, so that -containers are pulled by the pipeline from the target registry instead of the original registry. +Wave allows mirroring, i.e., copying containers used by your pipeline into a container registry of your choice. This allows the pipeline to pull containers from the target registry rather than the original registry. This is useful to create on-demand cache of containers images that are co-located in the same region where the pipeline is executed, and therefore optimising cost and network efficiency. From babdae0943f2a3d43bd941cdb700425c2beb917a Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:41:58 +0200 Subject: [PATCH 03/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/wave.md b/docs/wave.md index 98cad48017..38bd31eb5a 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -152,7 +152,7 @@ Wave allows mirroring, i.e., copying containers used by your pipeline into a con This is useful to create on-demand cache of containers images that are co-located in the same region where the pipeline is executed, and therefore optimising cost and network efficiency. -To enable this capability include those settings in your Nextflow configuration: +Include the following settings in your Nextflow configuration to enable this capability: ```groovy wave.enabled = true From bf010214729f29370a347d51b1b39a1269907d44 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:42:08 +0200 Subject: [PATCH 04/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/wave.md b/docs/wave.md index 38bd31eb5a..bb23f472c5 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -149,7 +149,7 @@ The repository access keys must be provided as Seqera Platform credentials (see Wave allows mirroring, i.e., copying containers used by your pipeline into a container registry of your choice. This allows the pipeline to pull containers from the target registry rather than the original registry. -This is useful to create on-demand cache of containers images that are co-located in the same region where the pipeline +Mirroring is useful to create an on-demand cache of container images that are co-located in the same region where the pipeline is executed, and therefore optimising cost and network efficiency. Include the following settings in your Nextflow configuration to enable this capability: From 9257f36392be50f8b8c40fa83e16567040a2e743 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:42:36 +0200 Subject: [PATCH 05/13] Update docs/wave.md [ci skp] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/wave.md b/docs/wave.md index bb23f472c5..e732f90a86 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -161,7 +161,7 @@ wave.build.repository = '' tower.accessToken = '' ``` -In the above snippet replace `` with a container registry of your choice e.g. `quay.io` (no prefix or suffix is needed) +In the above snippet, replace `` with a container registry of your choice. For example, `quay.io` (no prefix or suffix is needed). The container will be copied with the same name, tag and checksum in the specified registry. For example, if the source container is `quay.io/biocontainers/bwa:0.7.13--1` and the build repository setting is `foo.com`, the resulting container name will be `foo.com/biocontainers/bwa:0.7.13--1`. From c2c1d3ef7a1710393ac01b13cbb64cfbfc4f0f5f Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:42:44 +0200 Subject: [PATCH 06/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/wave.md b/docs/wave.md index e732f90a86..7eb72c8a88 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -162,7 +162,7 @@ tower.accessToken = '' ``` In the above snippet, replace `` with a container registry of your choice. For example, `quay.io` (no prefix or suffix is needed). -The container will be copied with the same name, tag and checksum in the specified registry. For example, if the source +The container will be copied with the same name, tag, and checksum in the specified registry. For example, if the source container is `quay.io/biocontainers/bwa:0.7.13--1` and the build repository setting is `foo.com`, the resulting container name will be `foo.com/biocontainers/bwa:0.7.13--1`. From e7c521ac3e8e876ce087821417a2ef88d9c37487 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:42:58 +0200 Subject: [PATCH 07/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/wave.md b/docs/wave.md index 7eb72c8a88..9c7cc93981 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -172,8 +172,8 @@ having `quay.io/biocontainers/bwa:0.7.13--1` as source container and `foo.com/ba container will be named `foo.com/bar/biocontainers/bwa:0.7.13--1`. ::: -The credentials to allow the push of containers in the target repository needs to be provided via Seqera Platform -credentials manager, whose account is specified in the `tower.accessToken` in the above configuration. +The credentials to allow the push of containers in the target repository need to be provided via the Seqera Platform +credentials manager. The account used for this is specified by the `tower.accessToken` in the configuration above. ### Container security scanning From 4d1bea8f3bda1e50eb367814cd594f9e54bb2334 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:43:12 +0200 Subject: [PATCH 08/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/wave.md b/docs/wave.md index 9c7cc93981..272aacd94a 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -177,10 +177,10 @@ credentials manager. The account used for this is specified by the `tower.access ### Container security scanning -Wave allows scanning containers used in your pipelines for security vulnerabilities, and report an execution error -when one or more security issues are found. +Wave enables the scanning of containers used in your pipelines for security vulnerabilities. +If any issues are detected, it will trigger an execution error and provide a report. -To enable this capability adds the following settings in your Nextflow configuration file: +To enable this capability add the following settings to your Nextflow configuration file: ```groovy wave.enabled = true From d9ac1ad57f014a97672d0db2bbb860ef75f42864 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:43:26 +0200 Subject: [PATCH 09/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/wave.md b/docs/wave.md index 272aacd94a..7d48ce7560 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -188,8 +188,8 @@ wave.scan.mode = 'required' tower.accessToken = '' ``` -When using these setting will only allows the use of the container in your Nextflow pipeline only if has no security -vulnerabilities. You can define the level of accepted vulnerabilities by using the `wave.scan.levels`. For example: +Nextflow will only allow the use of containers with no security +vulnerabilities when using these settings. You can define the level of accepted vulnerabilities using `wave.scan.levels`. For example: ``` wave.scan.levels = 'low,medium' From d5f650b13fd004bd739157c93085e1f9ca211432 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:43:41 +0200 Subject: [PATCH 10/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/wave.md b/docs/wave.md index 7d48ce7560..651a67353d 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -195,9 +195,7 @@ vulnerabilities when using these settings. You can define the level of accepted wave.scan.levels = 'low,medium' ``` -The above setting will allow the use of containers having vulnerabilities with *low* and *medium* -levels [common vulnerabilities scoring system](https://en.wikipedia.org/wiki/Common_Vulnerability_Scoring_System). -Accepted values are `low`, `medium`, `high` and `critical`. +The above setting will allow the use of containers with *low* and *medium* vulnerabilities. Accepted values are `low`, `medium`, `high`, and `critical`. See [common vulnerabilities scoring system](https://en.wikipedia.org/wiki/Common_Vulnerability_Scoring_System) for more information about these levels. :::{note} Wave security scanning applies to any container used in your pipeline, irrespective it was built by Wave or accessed From 0cd5251bfd1029f1083b7955d09dbf6d8bd2ca4c Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:44:01 +0200 Subject: [PATCH 11/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/wave.md b/docs/wave.md index 651a67353d..b0689ff602 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -198,9 +198,7 @@ wave.scan.levels = 'low,medium' The above setting will allow the use of containers with *low* and *medium* vulnerabilities. Accepted values are `low`, `medium`, `high`, and `critical`. See [common vulnerabilities scoring system](https://en.wikipedia.org/wiki/Common_Vulnerability_Scoring_System) for more information about these levels. :::{note} -Wave security scanning applies to any container used in your pipeline, irrespective it was built by Wave or accessed -through it. The container scan automatically expires after one week, if containers is accessed again after 7 days or -more the security scan is executed one more time. +Wave's security scanning applies to any container used in your pipeline, whether it was built by Wave or simply accessed through it. The security scan automatically expires after one week. If a container is accessed again after 7 days or more, the scan will be re-executed. ::: ### Run pipelines using Fusion file system From 4fc2dd49fc26be636ca6d42f60c6d65812a41118 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:44:25 +0200 Subject: [PATCH 12/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/wave.md b/docs/wave.md index b0689ff602..5f992d3139 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -164,7 +164,7 @@ tower.accessToken = '' In the above snippet, replace `` with a container registry of your choice. For example, `quay.io` (no prefix or suffix is needed). The container will be copied with the same name, tag, and checksum in the specified registry. For example, if the source container is `quay.io/biocontainers/bwa:0.7.13--1` and the build repository setting is `foo.com`, the resulting container -name will be `foo.com/biocontainers/bwa:0.7.13--1`. +name is `foo.com/biocontainers/bwa:0.7.13--1`. :::{tip} When using a path prefix in target registry name, it will be pre-pended to the resulting container name. For example From 960fb585d0df8eeb9207813accc484cfdb2c9ff4 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Fri, 11 Oct 2024 12:44:42 +0200 Subject: [PATCH 13/13] Update docs/wave.md [ci skip] Co-authored-by: Christopher Hakkaart Signed-off-by: Paolo Di Tommaso --- docs/wave.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/wave.md b/docs/wave.md index 5f992d3139..2c3655962d 100644 --- a/docs/wave.md +++ b/docs/wave.md @@ -167,7 +167,7 @@ container is `quay.io/biocontainers/bwa:0.7.13--1` and the build repository sett name is `foo.com/biocontainers/bwa:0.7.13--1`. :::{tip} -When using a path prefix in target registry name, it will be pre-pended to the resulting container name. For example +When using a path prefix in the target registry name, it will be prepended to the resulting container name. For example, having `quay.io/biocontainers/bwa:0.7.13--1` as source container and `foo.com/bar` as build repository, the resulting container will be named `foo.com/bar/biocontainers/bwa:0.7.13--1`. :::