Support alternative compute clusters (DASK, Fugue, etc)

[jmakov]

Missing functionality

Currently only Spark is supported.

Proposed feature

There are a few alternatives already like Dask and ray.io. Would be interesting to implement it based on https://github.com/fugue-project/fugue.

Alternatives considered

No response

Additional context

No response

[fabclmnt]

Hi @jmakov ,

thank you for your request. We will keep it in mind but as mentioned in past requests from the community, given the metrics of usage of other engines (like spark) there is no reason for prioritizing the support for engines like DASK, fugue, etc.

In case you want to contribute or if there is a high demand from the community to integrate a new engine, its prioritization will be considered.