Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(dask): Dask integration (#821) #821

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat(dask): Dask integration (#821) #821

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions helm/reana/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,9 @@ dependencies:
version: 0.47.7
repository: https://fluent.github.io/helm-charts
condition: fluent-bit.enabled
- name: dask-kubernetes-operator
version: 2024.8.0
repository: https://helm.dask.org
condition: dask.enabled
tags:
- dask
5 changes: 5 additions & 0 deletions helm/reana/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,11 @@ This Helm automatically prefixes all names using the release name to avoid colli
| `components.reana_workflow_engine_snakemake.environment` | [REANA-Workflow-Engine-Snakemake](https://github.com/reanahub/reana-workflow-engine-snakemake) environment variables | `{}` |
| `components.reana_workflow_engine_snakemake.image` | [REANA-Workflow-Engine-Snakemake image](https://hub.docker.com/r/reanahub/reana-workflow-engine-snakemake) to use | `docker.io/reanahub/reana-workflow-engine-snakemake:<chart-release-version>` |
| `compute_backends` | List of supported compute backends (kubernetes, htcondorcern, slurmcern) | "kubernetes" |
| `dask.enabled` | Install dask-kubernetes-operator custom resources in the cluster to support Dask workflows | false |
| `dask.cluster_max_memory_limit` | Max memory limit for Dask clusters | "16Gi" |
| `dask.cluster_default_number_of_workers` | Number of workers in Dask clusters by default | 2 |
| `dask.cluster_default_single_worker_memory` | Amount of memory used by a single Dask worker by default | "2Gi" |
| `dask.cluster_max_single_worker_memory` | Maximum amount of memory that can be used by a single Dask worker | "8Gi" |
| `db_env_config.REANA_DB_HOST` | Environment variable to connect to external databases | `<chart-release-name>-db` |
| `db_env_config.REANA_DB_NAME` | Environment variable to connect to external databases | reana |
| `db_env_config.REANA_DB_PORT` | Environment variable to connect to external databases | "5432" |
Expand Down
12 changes: 12 additions & 0 deletions helm/reana/templates/reana-server.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,18 @@ spec:
value: !!str {{ .Values.kubernetes_jobs_timeout_limit | default 604800 }} # 604800 seconds = 7 days
- name: REANA_KUBERNETES_JOBS_MEMORY_LIMIT
value: {{ .Values.kubernetes_jobs_memory_limit | default "4Gi" }}
- name: DASK_ENABLED
value: {{ .Values.dask.enabled | quote }}
{{- if .Values.dask.enabled }}
- name: REANA_DASK_CLUSTER_MAX_MEMORY_LIMIT
value: {{ .Values.dask.cluster_max_memory_limit | default "16Gi" }}
- name: REANA_DASK_CLUSTER_DEFAULT_NUMBER_OF_WORKERS
value: !!str {{ .Values.dask.cluster_default_number_of_workers | default 2 }}
- name: REANA_DASK_CLUSTER_DEFAULT_SINGLE_WORKER_MEMORY
value: {{ .Values.dask.cluster_default_single_worker_memory | default "2Gi" }}
- name: REANA_DASK_CLUSTER_MAX_SINGLE_WORKER_MEMORY
value: {{ .Values.dask.cluster_max_single_worker_memory | default "8Gi" }}
{{- end }}
- name: WORKSPACE_PATHS
value: {{ .Values.workspaces.paths | toJson | quote }}
- name: WORKSPACE_RETENTION_PERIOD
Expand Down
12 changes: 12 additions & 0 deletions helm/reana/templates/reana-workflow-controller.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,18 @@ spec:
value: {{ .Values.components.reana_job_controller.environment | toJson | quote }}
- name: REANA_INTERACTIVE_SESSIONS_ENVIRONMENTS
value: {{ .Values.interactive_sessions.environments | toJson | quote }}
- name: DASK_ENABLED
value: {{ .Values.dask.enabled | quote }}
{{- if .Values.dask.enabled }}
- name: REANA_DASK_CLUSTER_MAX_MEMORY_LIMIT
value: {{ .Values.dask.cluster_max_memory_limit | default "16Gi" }}
- name: REANA_DASK_CLUSTER_DEFAULT_NUMBER_OF_WORKERS
value: !!str {{ .Values.dask.cluster_default_number_of_workers | default 2 }}
- name: REANA_DASK_CLUSTER_DEFAULT_SINGLE_WORKER_MEMORY
value: {{ .Values.dask.cluster_default_single_worker_memory | default "2Gi" }}
- name: REANA_DASK_CLUSTER_MAX_SINGLE_WORKER_MEMORY
value: {{ .Values.dask.cluster_max_single_worker_memory | default "8Gi" }}
{{- end }}
{{- if .Values.reana_hostname }}
- name: REANA_HOSTNAME
value: {{ .Values.reana_hostname }}
Expand Down
12 changes: 12 additions & 0 deletions helm/reana/templates/roles.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,18 @@ rules:
- apiGroups: ["metrics.k8s.io"]
resources: ["pods", "nodes"]
verbs: ["get", "list", "watch"]
# Custom dask kubernetes resources
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to wrap the Dask stuff into a conditional variable such as dask.enabled -- similarly as we have traefik.enabled -- because some REANA deployments may not want to (or may not be allowed to) support Dask operators in their deployments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, when dask.enabled is true, it would be good to deploy Dask operator automatically, in quite the same way as we do for traefik; see the dependencies in Chart.yaml.

{{- if .Values.dask.enabled }}
- apiGroups: ["kubernetes.dask.org"]
resources: ["daskclusters", "daskautoscalers"]
verbs: ["create", "get", "list", "watch", "update", "patch", "delete"]
{{- end }}
# Traefik Middleware permissions
{{- if and .Values.traefik.enabled .Values.dask.enabled }}
- apiGroups: ["traefik.io"]
resources: ["middlewares"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
{{- end }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
Expand Down
7 changes: 7 additions & 0 deletions helm/reana/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,13 @@ traefik:
enabled: true
isDefaultClass: true

dask:
enabled: false
cluster_max_memory_limit: "16Gi"
cluster_default_number_of_workers: 2
cluster_default_single_worker_memory: "2Gi"
cluster_max_single_worker_memory: "4Gi"

pgbouncer:
enabled: false
image: docker.io/bitnami/pgbouncer:1.23.1
Expand Down