Skip to content

Latest commit

 

History

History
52 lines (38 loc) · 3.17 KB

README.md

File metadata and controls

52 lines (38 loc) · 3.17 KB
Raw

Automated Argo workflows

If you'd like to automate your Jupyter notebooks using Argo, please use these kustomize manifests. If you follow the steps below, your application is fully set and ready to be deployed via Argo CD.

For a detailed guide on how to adjust your notebooks etc, please consult documentation

  1. Replace all <VARIABLE> mentions with your project name, respective url or any fitting value

  2. Define your automation run structure in the templates section of cron-workflow.yaml

  3. Set up sops:

    1. Install go from your distribution repository

    2. Setup GOPATH

      echo 'export GOPATH="$HOME/.go"' >> ~/.bashrc
echo 'export PATH="${GOPATH//://bin:}/bin:$PATH"' >> ~/.bashrc
source  ~/.bashrc

    3. Install sops from your distribution repository if possible or use sops GitHub release binaries

    4. Import AICoE-SRE's public key EFDB9AFBD18936D9AB6B2EECBD2C73FF891FBC7E:

      gpg --keyserver keyserver.ubuntu.com --recv EFDB9AFBD18936D9AB6B2EECBD2C73FF891FBC7E

    5. Import tcoufal's (A76372D361282028A99F9A47590B857E0288997C) and mhild's 04DAFCD9470A962A2F272984E5EB0DA32F3372AC keys (so they can help)

      gpg --keyserver keyserver.ubuntu.com --recv A76372D361282028A99F9A47590B857E0288997C  # tcoufal
gpg --keyserver keyserver.ubuntu.com --recv 04DAFCD9470A962A2F272984E5EB0DA32F3372AC  # mhild

    6. If you'd like to be able to build the manifest on your own as well, please list your GPG key in the .sops.yaml file, pgp section (add to the comma separated list). With your key present there, you can later generate the full manifests using kustomize yourself (ksops has to be installed, please follow ksops guide.

  4. Create a secret and encrypt it with sops:

    # If you're not already in the `manifest` folder, cd here
cd manifests
# Mind that `SECRET_NAME` must match the `SECRET_NAME` used in `cron-workflow.yaml`
oc create secret generic <SECRET_NAME> \
  --from-literal=path=<BASE_PATH_WITHIN_CEPH_BUCKET> \
  --from-literal=bucket=<BUCKET> \
  --from-literal=access-key-id=<AWS_ACCESS_KEY_ID> \
  --from-literal=secret-access-key=<AWS_SECRET_ACCESS_KEY> \
  --dry-run -o yaml |
sops --input-type=yaml --output-type=yaml -e /dev/stdin > ceph-creds.yaml

Note: You can use the S2I image, that was built by s2i-custom-notebook for this automation. This image is expected to be used by default, therefore the workingDir is adjusted to /opt/app-root/backup. Please change or remove this settings in case you plan on using different image.