-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
85 changed files
with
5,114 additions
and
79 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| name: Build and push AmazonLinux jupyter-scipy | ||
|
|
||
| on: | ||
| push: | ||
| paths: | ||
| - azlinux-jupyter-scipy/** | ||
| - .github/workflows/build_azlinux_jupyter_scipy_image.yml | ||
|
|
||
| jobs: | ||
| jupyter-nextflow: | ||
| name: Build and Push AmazonLinux jupyter-scipy | ||
| uses: uc-cdis/.github/.github/workflows/image_build_push.yaml@master | ||
| with: | ||
| DOCKERFILE_LOCATION: "./azlinux-jupyter-scipy/Dockerfile" | ||
| DOCKERFILE_BUILD_CONTEXT: "./azlinux-jupyter-scipy" | ||
| OVERRIDE_REPO_NAME: "jupyter-notebook" | ||
| OVERRIDE_TAG_NAME: "azlinux-scipy-$(echo ${GITHUB_REF#refs/*/} | tr / _)" | ||
| secrets: | ||
| ECR_AWS_ACCESS_KEY_ID: ${{ secrets.ECR_AWS_ACCESS_KEY_ID }} | ||
| ECR_AWS_SECRET_ACCESS_KEY: ${{ secrets.ECR_AWS_SECRET_ACCESS_KEY }} | ||
| QUAY_USERNAME: ${{ secrets.QUAY_USERNAME }} | ||
| QUAY_ROBOT_TOKEN: ${{ secrets.QUAY_ROBOT_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| name: Build and Push Jupyter-Nextflow image | ||
|
|
||
| on: | ||
| push: | ||
| paths: | ||
| - jupyter-nextflow/** | ||
| - .github/workflows/build_jupyter_nextflow_image.yml | ||
|
|
||
| jobs: | ||
| jupyter-nextflow: | ||
| name: Build and Push jupyter-nextflow image | ||
| uses: uc-cdis/.github/.github/workflows/image_build_push.yaml@master | ||
| with: | ||
| DOCKERFILE_LOCATION: "./jupyter-nextflow/Dockerfile" | ||
| DOCKERFILE_BUILD_CONTEXT: "./jupyter-nextflow" | ||
| OVERRIDE_REPO_NAME: "jupyter-notebook" | ||
| OVERRIDE_TAG_NAME: "nextflow-$(echo ${GITHUB_REF#refs/*/} | tr / _)" | ||
| secrets: | ||
| ECR_AWS_ACCESS_KEY_ID: ${{ secrets.ECR_AWS_ACCESS_KEY_ID }} | ||
| ECR_AWS_SECRET_ACCESS_KEY: ${{ secrets.ECR_AWS_SECRET_ACCESS_KEY }} | ||
| QUAY_USERNAME: ${{ secrets.QUAY_USERNAME }} | ||
| QUAY_ROBOT_TOKEN: ${{ secrets.QUAY_ROBOT_TOKEN }} |
22 changes: 22 additions & 0 deletions
22
.github/workflows/build_jupyter_restricted_download_image.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| name: Build and Push jupyter-restricted-download | ||
|
|
||
| on: | ||
| push: | ||
| paths: | ||
| - jupyter-restricted-download/** | ||
| - .github/workflows/build_jupyter_restricted_download_image.yml | ||
|
|
||
| jobs: | ||
| jupyter-restricted-download: | ||
| name: Build and Push jupyter-restricted-download | ||
| uses: uc-cdis/.github/.github/workflows/image_build_push.yaml@master | ||
| with: | ||
| DOCKERFILE_LOCATION: "./jupyter-restricted-download/Dockerfile" | ||
| DOCKERFILE_BUILD_CONTEXT: "./jupyter-restricted-download" | ||
| OVERRIDE_REPO_NAME: "jupyter-notebook" | ||
| OVERRIDE_TAG_NAME: "restricted-download-$(echo ${GITHUB_REF#refs/*/} | tr / _)" | ||
| secrets: | ||
| ECR_AWS_ACCESS_KEY_ID: ${{ secrets.ECR_AWS_ACCESS_KEY_ID }} | ||
| ECR_AWS_SECRET_ACCESS_KEY: ${{ secrets.ECR_AWS_SECRET_ACCESS_KEY }} | ||
| QUAY_USERNAME: ${{ secrets.QUAY_USERNAME }} | ||
| QUAY_ROBOT_TOKEN: ${{ secrets.QUAY_ROBOT_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| name: Push Stata Gen3-licensed Image to quay | ||
|
|
||
| on: | ||
| push: | ||
| paths: | ||
| - jupyter-pystata-gen3-licensed/** | ||
| - .github/workflows/build_push_stata_gen3_licensed.yml | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Maximize build space | ||
| uses: easimon/maximize-build-space@master | ||
| with: | ||
| root-reserve-mb: 30000 | ||
| swap-size-mb: 1024 | ||
| remove-dotnet: 'true' | ||
| remove-android: 'true' | ||
| remove-haskell: 'true' | ||
| - uses: actions/checkout@v2 | ||
| - uses: prewk/[email protected] | ||
| env: | ||
| AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| SOURCE: 's3://ctds-stata/Stata17Linux64.tar.gz' | ||
| DEST: './jupyter-pystata-gen3-licensed/resources/' | ||
|
|
||
| - name: Extract branch name | ||
| shell: bash | ||
| run: echo "branch=$(echo $(echo ${GITHUB_REF#refs/*/} | tr / _))" >> $GITHUB_OUTPUT | ||
| id: extract_branch | ||
|
|
||
| - name: Determine image to build | ||
| id: parse_image | ||
| shell: python | ||
| run: | | ||
| import os | ||
| build_target = "jupyter-pystata-licensed" | ||
| print(f"Will trigger build for: {build_target}") | ||
| with open(os.environ['GITHUB_OUTPUT'], 'a') as fh: | ||
| print(f'build_target={build_target}', file=fh) | ||
| - if: ${{ steps.parse_image.outputs.build_target }} | ||
| name: Sanitize image name | ||
| id: sanitize_name | ||
| run: | | ||
| IMAGE_NAME=$( sed 's/[^[:alnum:]]/_/g' <<< ${{ steps.parse_image.outputs.build_target }} ); | ||
| echo "image_name=$IMAGE_NAME" >> $GITHUB_OUTPUT | ||
| - name: Build Image | ||
| id: build-image | ||
| uses: redhat-actions/buildah-build@v2 | ||
| with: | ||
| image: jupyter-pystata-gen3-licensed | ||
| tags: ${{ steps.extract_branch.outputs.branch }} | ||
| dockerfiles: ./jupyter-pystata-gen3-licensed/Dockerfile | ||
|
|
||
| - name: Push To quay.io | ||
| id: push-to-quay | ||
| uses: redhat-actions/push-to-registry@v2 | ||
| with: | ||
| image: ${{ steps.build-image.outputs.image }} | ||
| tags: ${{ steps.build-image.outputs.tags }} | ||
| registry: quay.io/cdis | ||
| username: ${{ secrets.QUAY_SERVICE_ACCOUNT_USER }} | ||
| password: ${{ secrets.QUAY_SERVICE_ACCOUNT_PASSWORD }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,115 @@ | ||
| name: Build and push VADC notebook image | ||
|
|
||
| on: | ||
| push: | ||
| paths: | ||
| - jupyter-vadc/** | ||
| - .github/workflows/build_vadc_notebook_image.yml | ||
|
|
||
| jobs: | ||
| push-image: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Maximize build space | ||
| uses: easimon/maximize-build-space@master | ||
| with: | ||
| root-reserve-mb: 30000 | ||
| swap-size-mb: 1024 | ||
| remove-dotnet: 'true' | ||
| remove-android: 'true' | ||
| remove-haskell: 'true' | ||
|
|
||
| - name: clean | ||
| run: sudo apt clean | ||
|
|
||
| - name: Extract branch name | ||
| shell: bash | ||
| run: echo "branch=$(echo $(echo ${GITHUB_REF#refs/heads/} | tr / _))" >> $GITHUB_OUTPUT | ||
| id: extract_branch | ||
|
|
||
| - name: Checkout repo | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Get changed files | ||
| id: get_changed_files | ||
| uses: jitterbit/get-changed-files@v1 | ||
| with: | ||
| format: 'json' | ||
|
|
||
| - name: Determine image to build | ||
| id: parse_image | ||
| shell: python | ||
| env: | ||
| CHANGES: ${{ steps.get_changed_files.outputs.added_modified }} | ||
| run: | | ||
| import os, json, re | ||
| notebook_dir = "jupyter-vadc" | ||
| changed_vadc_notebook_files = json.loads(os.environ['CHANGES']) | ||
| print(f"Changed files {changed_vadc_notebook_files}") | ||
| # build an image for every subdir of jupyter-vadc with a changed file | ||
| subdirs = list( | ||
| filter( | ||
| lambda d: os.path.isdir(f"{os.environ['GITHUB_WORKSPACE']}/{notebook_dir}/{d}"), | ||
| os.listdir(notebook_dir) | ||
| ) | ||
| ) | ||
| print(f"Scanning subdirs for changed files: {subdirs}") | ||
| buildable_images = list( | ||
| filter( | ||
| lambda subdir: any( | ||
| changed_file.startswith(f"{notebook_dir}/{subdir}") | ||
| for changed_file in changed_vadc_notebook_files | ||
| ), | ||
| subdirs | ||
| ) | ||
| ) | ||
| if not len(buildable_images): | ||
| print(f"None of {changed_vadc_notebook_files} triggers a build for any of {subdirs}. Done.") | ||
| exit(0) | ||
| elif len(buildable_images) > 1: | ||
| print("Found multiple notebook directories with changes: {buildable_images}") | ||
| print("Only one image can be built at a time. Exiting.") | ||
| exit(1) | ||
| build_target = buildable_images[0] | ||
| print(f"Will trigger build for: {build_target}") | ||
| with open(os.environ['GITHUB_OUTPUT'], 'a') as fh: | ||
| print(f'build_target={build_target}', file=fh) | ||
| - if: ${{ steps.parse_image.outputs.build_target }} | ||
| name: Sanitize image name | ||
| id: sanitize_name | ||
| run: | | ||
| IMAGE_NAME=$( sed 's/[^[:alnum:]]/_/g' <<< ${{ steps.parse_image.outputs.build_target }} ); | ||
| echo "image_name=$IMAGE_NAME" >> $GITHUB_OUTPUT | ||
| - if: ${{ steps.parse_image.outputs.build_target }} | ||
| name: Build image | ||
| id: build-image | ||
| uses: redhat-actions/buildah-build@v2 | ||
| with: | ||
| image: jupyter-vadc | ||
| context: ./jupyter-vadc/ | ||
| tags: | ||
| ${{ steps.sanitize_name.outputs.image_name }}__${{ steps.extract_branch.outputs.branch }} | ||
| ${{ steps.sanitize_name.outputs.image_name }}__${{ github.sha }} | ||
| ${{ steps.sanitize_name.outputs.image_name }}__latest | ||
| dockerfiles: ./jupyter-vadc/Dockerfile | ||
| build-args: | ||
| NOTEBOOK_DIR=jupyter-vadc/${{ steps.parse_image.outputs.build_target }} | ||
|
|
||
| - if: ${{ steps.parse_image.outputs.build_target }} | ||
| name: Push To quay.io | ||
| id: push-to-quay | ||
| uses: redhat-actions/push-to-registry@v2 | ||
| with: | ||
| image: ${{ steps.build-image.outputs.image }} | ||
| tags: ${{ steps.build-image.outputs.tags }} | ||
| registry: quay.io/cdis | ||
| username: ${{ secrets.QUAY_SERVICE_ACCOUNT_USER }} | ||
| password: ${{ secrets.QUAY_SERVICE_ACCOUNT_PASSWORD }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| name: Build VLMD Image | ||
|
|
||
| on: | ||
| push: | ||
| paths: | ||
| - vlmd-submission-tools/** | ||
| - .github/workflows/build_vlmd_submission_python_image.yml | ||
|
|
||
| jobs: | ||
| ci: | ||
| name: Build VLMD Image | ||
| uses: uc-cdis/.github/.github/workflows/image_build_push.yaml@master | ||
| with: | ||
| DOCKERFILE_LOCATION: "./vlmd-submission-tools/Dockerfile" | ||
| DOCKERFILE_BUILD_CONTEXT: "./vlmd-submission-tools" | ||
| OVERRIDE_REPO_NAME: "vlmd-submission-tools" | ||
| OVERRIDE_TAG_NAME: "vlmd-submission-tools-$(echo ${GITHUB_REF#refs/*/} | tr / _)" | ||
| secrets: | ||
| ECR_AWS_ACCESS_KEY_ID: ${{ secrets.ECR_AWS_ACCESS_KEY_ID }} | ||
| ECR_AWS_SECRET_ACCESS_KEY: ${{ secrets.ECR_AWS_SECRET_ACCESS_KEY }} | ||
| QUAY_USERNAME: ${{ secrets.QUAY_USERNAME }} | ||
| QUAY_ROBOT_TOKEN: ${{ secrets.QUAY_ROBOT_TOKEN }} |
Oops, something went wrong.