Skip to content

Commit

Permalink
bump docker dependencies for v1.2.3
Browse files Browse the repository at this point in the history
  • Loading branch information
mlin committed Sep 28, 2021
1 parent e86b33e commit 79d53c3
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 8 deletions.
6 changes: 3 additions & 3 deletions WDL/runtime/config_templates/default.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ disable_patterns = ["*.php", "*.aspx"]

[download_aria2c]
# see: https://github.com/chanzuckerberg/miniwdl/tree/main/tools_image
docker = ghcr.io/chanzuckerberg/miniwdl_tools:Id_sha256_ad9e362d2de002f2128f52cf5961831dbeed944637f347df69df814dc8cfa349
docker = ghcr.io/miniwdl-ext/miniwdl-tools:Id_sha256_03e517dc32d6a35643726397030300780f15d78f214ce5ab427098da2e5b2d12


[download_awscli]
Expand All @@ -137,12 +137,12 @@ docker = ghcr.io/chanzuckerberg/miniwdl_tools:Id_sha256_ad9e362d2de002f2128f52cf
# Failing all of the above, public S3 URIs can always be used.
host_credentials = false
# see: https://github.com/chanzuckerberg/miniwdl/tree/main/tools_image
docker = ghcr.io/chanzuckerberg/miniwdl_tools:Id_sha256_ad9e362d2de002f2128f52cf5961831dbeed944637f347df69df814dc8cfa349
docker = ghcr.io/miniwdl-ext/miniwdl-tools:Id_sha256_03e517dc32d6a35643726397030300780f15d78f214ce5ab427098da2e5b2d12


[download_gsutil]
# current version from https://github.com/GoogleCloudPlatform/cloud-sdk-docker/releases
docker = gcr.io/google.com/cloudsdktool/cloud-sdk:350.0.0-slim
docker = gcr.io/google.com/cloudsdktool/cloud-sdk:358.0.0-slim


[call_cache]
Expand Down
4 changes: 4 additions & 0 deletions WDL/runtime/download.py
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,8 @@ def awscli_downloader(
# credentials are available or (ii) the IAM policy restricts accessible S3
# buckets regardless of whether the desired object is public.
rm -f __out/*
>&2 echo 'Retrying with --no-sign-request in case the object is public.' \
' If the overall operation fails, the real error may precede this message.'
aws s3 cp --no-sign-request "~{uri}" __out/
fi
>>>
Expand Down Expand Up @@ -284,6 +286,8 @@ def awscli_directory_downloader(
# credentials are available or (ii) the IAM policy restricts accessible S3
# buckets regardless of whether the desired object is public.
rm -f "__out/~{dnm}/*"
>&2 echo 'Retrying with --no-sign-request in case the folder is public.' \
' If the overall operation fails, the real error may precede this message.'
aws s3 cp --recursive --no-sign-request "~{uri}" "__out/~{dnm}/"
fi
>>>
Expand Down
10 changes: 5 additions & 5 deletions tools_image/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ This subdirectory is the recipe for a Docker image bundling tools that miniwdl u

For example, the image bundles [aria2c](https://aria2.github.io/), which miniwdl uses to download large input files supplied as URLs (without requiring end-users to install extra OS packages). Miniwdl synthesizes a WDL task with this image, which inputs the URL and outputs the desired file. The image is served publicly from GitHub Container Registry, referenced in the miniwdl configuration defaults (where it can be overridden if necessary).

This image doesn't change often, so we build it manually. First, authenticate your local `docker` CLI to GitHub Container Registry ([instructions](https://docs.github.com/en/packages/guides/pushing-and-pulling-docker-images#authenticating-to-github-container-registry)) using a Personal Access Token *with SSO enabled* for the `chanzuckerberg` organization. [Enable docker build -squash](https://stackoverflow.com/a/44346323/13393076) and,
This image doesn't change often, so we build it manually. First, authenticate your local `docker` CLI to GitHub Container Registry ([instructions](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-to-the-container-registry)) using a Personal Access Token with appropriate scope for the [miniwdl-ext organization](https://github.com/miniwdl-ext/). [Enable docker build --squash](https://stackoverflow.com/a/44346323/13393076) and,

```
docker pull ubuntu:20.04
docker build --no-cache --squash -t miniwdl_tools:latest tools_image/
TAG=$(docker inspect miniwdl_tools:latest | jq -r .[0].Id | tr ':' '_' \
| xargs printf 'ghcr.io/chanzuckerberg/miniwdl_tools:Id_%s')
docker tag miniwdl_tools:latest $TAG
docker build --no-cache --squash -t miniwdl-tools:latest tools_image/
TAG=$(docker inspect miniwdl-tools:latest | jq -r .[0].Id | tr ':' '_' \
| xargs printf 'ghcr.io/miniwdl-ext/miniwdl-tools:Id_%s')
docker tag miniwdl-tools:latest $TAG
docker push $TAG
echo $TAG
```
Expand Down

0 comments on commit 79d53c3

Please sign in to comment.