Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dask: refactor hard-coded values and rename dask cluster components #841

Open
Alputer opened this issue Nov 4, 2024 · 0 comments · May be fixed by reanahub/reana-workflow-controller#613 or reanahub/reana-commons#478
Assignees

Comments

@Alputer
Copy link
Member

Alputer commented Nov 4, 2024

Currently, name of our Dask components Dask cluster, Dask autoscaler, dashboard ingress and are hardcoded in the code. For example the code snippet at the end of this paragraph is taken from reana_workflow_controller.consumer.py. We should probably have a function (e.g def get_dask_cluster_name(worklow_id)) in reana-commons to get the name for these components and make it configurable to cluster admins. Then we can use it where necessary. (reana-server, reana-workflow-controller etc.). Besides, we may consider changing the names of Dask components since some of them are quite verbose and not easy to read. (e.g an ingress component dask-dashboard-ingress-reana-run-dask-acd35ec0-3b81-4eda-b252-dd0d6b180869)

def _delete_dask_cluster(workflow: Workflow) -> None:
    """Delete the Dask cluster resources."""
    current_k8s_custom_objects_api_client.delete_namespaced_custom_object(
        group="kubernetes.dask.org",
        version="v1",
        plural="daskclusters",
        namespace="default",
        name=f"reana-run-dask-{workflow.id_}",
    )

    if DASK_AUTOSCALER_ENABLED:
        current_k8s_custom_objects_api_client.delete_namespaced_custom_object(
            group="kubernetes.dask.org",
            version="v1",
            plural="daskautoscalers",
            namespace="default",
            name=f"dask-autoscaler-reana-run-dask-{workflow.id_}",
        )

    delete_dask_dashboard_ingress(
        f"dask-dashboard-ingress-reana-run-dask-{workflow.id_}", workflow.id_
    )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In review
1 participant