From 44dc7e7e9752f3215cb11e841e1ca5b7f2c161d0 Mon Sep 17 00:00:00 2001 From: Shuying Liang Date: Tue, 21 Jan 2025 12:25:10 -0800 Subject: [PATCH 1/3] Add Flyte K8d data service agent docs Signed-off-by: Shuying Liang --- docs/deployment/agents/index.md | 3 + docs/deployment/agents/k8sservice.rst | 178 ++++++++++++++++++++++++++ 2 files changed, 181 insertions(+) create mode 100644 docs/deployment/agents/k8sservice.rst diff --git a/docs/deployment/agents/index.md b/docs/deployment/agents/index.md index 11ce607788..6914ce90bf 100644 --- a/docs/deployment/agents/index.md +++ b/docs/deployment/agents/index.md @@ -33,6 +33,8 @@ If you are using a managed deployment of Flyte, you will need to contact your de - Configuring your Flyte deployment for the SnowFlake agent. * - {ref}`OpenAI Batch ` - Submit requests to OpenAI GPT models for asynchronous batch processing. +* - {ref}`LinkedIn K8s Service Batch ` + - Configuring your Flyte deployment for the K8s service agent. ``` ```{toctree} @@ -49,4 +51,5 @@ sagemaker_inference sensor snowflake openai_batch +k8sservice ``` diff --git a/docs/deployment/agents/k8sservice.rst b/docs/deployment/agents/k8sservice.rst new file mode 100644 index 0000000000..3d71c8153a --- /dev/null +++ b/docs/deployment/agents/k8sservice.rst @@ -0,0 +1,178 @@ +.. _deployment-agent-setup-k8sservice: + +Kubernetes (K8s) Service Agent +================== + +The Kubernetes (K8s) Service Agent enables machine learning (ML) users to efficiently handle non-training tasks—such as data loading, caching, and processing—concurrently with training jobs in Kubernetes clusters. +This capability is particularly valuable in deep learning applications, such as those in Graph Neural Networks (GNNs). + +This guide offers a comprehensive overview of setting up the K8s Service Agent within your Flyte deployment. + +Spin up a cluster +----------------- + +.. tabs:: + + .. group-tab:: Flyte binary + + You can spin up a demo cluster using the following command: + + .. code-block:: bash + + flytectl demo start + + Or install Flyte using the :ref:`flyte-binary helm chart `. + + .. group-tab:: Flyte core + + If you've installed Flyte using the + `flyte-core helm chart `__, please ensure: + + * You have the correct kubeconfig and have selected the correct Kubernetes context. + * You have configured the correct flytectl settings in ``~/.flyte/config.yaml``. + +.. note:: + + Add the Flyte chart repo to Helm if you're installing via the Helm charts. + + .. code-block:: bash + + helm repo add flyteorg https://flyteorg.github.io/flyte + +Specify agent configuration +---------------------------- + +Enable the K8s service agent by adding the following config to the relevant YAML file(s): + +.. code-block:: yaml + + tasks: + task-plugins: + enabled-plugins: + - agent-service + default-for-task-types: + - dataservicetask: agent-service + +.. code-block:: yaml + + plugins: + agent-service: + agents: + k8sservice-agent: + endpoint: + insecure: true + agentForTaskTypes: + - dataservicetask: k8sservice-agent + - sensor: k8sservice-agent + +Substitute ```` with the endpoint of your MMCloud agent. + + +Setup the RBAC +---------------------- + +The k8s agent will create statefulset and expose the service end point for the statefulset pods. +RBAC needs to be set up to allow the K8s agent to CRUD statefulset and service. + +The role `flyte-flyteagent-role` set up: +.. code-block:: yaml + + # Example of the role/binding set up for the data service to create/update/delete resources in the sandbox flyte namespace + apiVersion: rbac.authorization.k8s.io/v1 + kind: Role + metadata: + name: flyte-flyteagent-role + namespace: flyte + labels: + app.kubernetes.io/name: flyteagent + app.kubernetes.io/instance: flyte + rules: + - apiGroups: + - apps + resources: + - statefulsets + - statefulsets/status + - statefulsets/scale + - statefulsets/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - apiGroups: + - "" + resources: + - pods + - configmaps + - serviceaccounts + - secrets + - pods/exec + - pods/log + - pods/status + - services + verbs: + - '*' + + +The binding `flyte-flyteagent-rolebinding` for the role `flyte-flyteagent-role` + .. code-block:: yaml + # Example of the role/binding set up for the data service to create/update/delete resources in the sandbox flyte namespace + apiVersion: rbac.authorization.k8s.io/v1 + kind: RoleBinding + metadata: + name: flyte-flyteagent-rolebinding + namespace: flyte + labels: + app.kubernetes.io/name: flyteagent + app.kubernetes.io/instance: flyte + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: flyte-flyteagent-role + subjects: + - kind: ServiceAccount + name: flyteagent + namespace: flyte + +Upgrade the deployment +---------------------- + +.. tabs:: + + .. group-tab:: Flyte binary + + .. tabs:: + + .. group-tab:: Demo cluster + + .. code-block:: bash + + kubectl rollout restart deployment flyte-sandbox -n flyte + + .. group-tab:: Helm chart + + .. code-block:: bash + + helm upgrade flyteorg/flyte-binary -n --values + + Replace ```` with the name of your release (e.g., ``flyte-backend``), + ```` with the name of your namespace (e.g., ``flyte``), + and ```` with the name of your YAML file. + + .. group-tab:: Flyte core + + .. code-block:: + + helm upgrade flyte/flyte-core -n --values values-override.yaml + + Replace ```` with the name of your release (e.g., ``flyte``) + and ```` with the name of your namespace (e.g., ``flyte``). + +Wait for the upgrade to complete. You can check the status of the deployment pods by running the following command: + +.. code-block:: + + kubectl get pods -n flyte From 65290304509e556237b4edd497c4b223838b34de Mon Sep 17 00:00:00 2001 From: Shuying Liang Date: Wed, 22 Jan 2025 15:10:51 -0800 Subject: [PATCH 2/3] tweaks Signed-off-by: Shuying Liang --- docs/deployment/agents/k8sservice.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/deployment/agents/k8sservice.rst b/docs/deployment/agents/k8sservice.rst index 3d71c8153a..c247e9dfe1 100644 --- a/docs/deployment/agents/k8sservice.rst +++ b/docs/deployment/agents/k8sservice.rst @@ -1,12 +1,12 @@ .. _deployment-agent-setup-k8sservice: Kubernetes (K8s) Service Agent -================== +============================== -The Kubernetes (K8s) Service Agent enables machine learning (ML) users to efficiently handle non-training tasks—such as data loading, caching, and processing—concurrently with training jobs in Kubernetes clusters. +The Kubernetes (K8s) Data Service Agent enables machine learning (ML) users to efficiently handle non-training tasks—such as data loading, caching, and processing—concurrently with training jobs in Kubernetes clusters. This capability is particularly valuable in deep learning applications, such as those in Graph Neural Networks (GNNs). -This guide offers a comprehensive overview of setting up the K8s Service Agent within your Flyte deployment. +This guide offers a comprehensive overview of setting up the K8s Data Service Agent within your Flyte deployment. Spin up a cluster ----------------- @@ -71,8 +71,8 @@ Substitute ```` with the endpoint of your MMCloud agent. Setup the RBAC ---------------------- -The k8s agent will create statefulset and expose the service end point for the statefulset pods. -RBAC needs to be set up to allow the K8s agent to CRUD statefulset and service. +The K8s Data Service Agent will create a StatefulSet and expose the Service endpoint for the StatefulSet pods. +RBAC needs to be set up to allow the K8s Data Service Agent to perform CRUD operations on the StatefulSet and Service. The role `flyte-flyteagent-role` set up: .. code-block:: yaml From 5aacc74c126e4055c8934d3bebba40d48a8fbd39 Mon Sep 17 00:00:00 2001 From: Shuying Liang Date: Fri, 24 Jan 2025 09:21:59 -0800 Subject: [PATCH 3/3] formatting Signed-off-by: Shuying Liang --- docs/deployment/agents/k8sservice.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/deployment/agents/k8sservice.rst b/docs/deployment/agents/k8sservice.rst index c247e9dfe1..eb1246614a 100644 --- a/docs/deployment/agents/k8sservice.rst +++ b/docs/deployment/agents/k8sservice.rst @@ -67,14 +67,14 @@ Enable the K8s service agent by adding the following config to the relevant YAML Substitute ```` with the endpoint of your MMCloud agent. - Setup the RBAC ----------------------- +-------------- The K8s Data Service Agent will create a StatefulSet and expose the Service endpoint for the StatefulSet pods. RBAC needs to be set up to allow the K8s Data Service Agent to perform CRUD operations on the StatefulSet and Service. The role `flyte-flyteagent-role` set up: + .. code-block:: yaml # Example of the role/binding set up for the data service to create/update/delete resources in the sandbox flyte namespace @@ -116,9 +116,10 @@ The role `flyte-flyteagent-role` set up: verbs: - '*' - The binding `flyte-flyteagent-rolebinding` for the role `flyte-flyteagent-role` + .. code-block:: yaml + # Example of the role/binding set up for the data service to create/update/delete resources in the sandbox flyte namespace apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding