feat(dask): Dask integration (#821)

[Alputer]

This pull request contains the ongoing work of dask integration into REANA.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 31.08%. Comparing base (467be7a) to head (c6faf71).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #821   +/-   ##
=======================================
  Coverage   31.08%   31.08%           
=======================================
  Files          26       26           
  Lines        2487     2487           
=======================================
  Hits          773      773           
  Misses       1714     1714           

[tiborsimko]

It would be good to wrap the Dask stuff into a conditional variable such as dask.enabled -- similarly as we have traefik.enabled -- because some REANA deployments may not want to (or may not be allowed to) support Dask operators in their deployments.

[tiborsimko]

Also, when dask.enabled is true, it would be good to deploy Dask operator automatically, in quite the same way as we do for traefik; see the dependencies in Chart.yaml.

[tiborsimko]

Suggested change

	| `dask.enabled` | Install dask-kubernetes-operator custom resources in the cluster to enable dask workflows | true |
	| `dask.enabled` | Install dask-kubernetes-operator custom resources in the cluster to support Dask workflows? | false |

• minor verbiage change;
• dunno if we want to keep it on true yet; better to let it on false until the feature is stable.

[tiborsimko]

Cosmetics: s/dask/Dask/g here and elsewhere, the term is usually written with capital D.

[tiborsimko]

In order to better distinguish between (individual) worker-based specs and (overall) cluster-based specs, what about renaming the values to something like:

• dask.cluster_cores
• dask.cluster_memory
• dask.worker_cores
• dask.worker_memory

The similar terms would be closer in the naming, and the fact that some of these are "defaults" we can express in the documentation verbiage.

(Dunno about the word "limit" that we could perhaps leave out or perhaps include, e.g. worker_cores_limit.)

And, if we expose all these for the user to possibly modify in their reana.yaml files, then we'd need the user limits for all these:

• dask.cluster_cores_max_user_limit
• dask.cluster_memory_max_user_limit
• dask.worker_cores_max_user_limit
• dask.worker_memory_max_user_limit

[tiborsimko]

Example: a user asks for:

  resources:
    dask:
      image: coffeateam/coffea-dask-almalinux8
      number_of_workers: 3
      single_worker_memory: "3Gi"

We would need just two max_user_limit values for these, whilst the admin will set desired default values in values.yaml.

(And if we don't need to specify cores, and Dask uses "auto" detect or some such, then we don't even have to specify them.)