From f0227bce0aeb95e10e8dbf32c0556248258f3d3f Mon Sep 17 00:00:00 2001 From: Yaron Date: Thu, 31 Oct 2024 09:47:23 +0200 Subject: [PATCH] Merge pull request #1206 from run-ai/support-matrix-integration Support matrix integration --- .../integrations/integration-overview.md | 54 +++++++++---------- mkdocs.yml | 40 +++++++------- 2 files changed, 47 insertions(+), 47 deletions(-) diff --git a/docs/platform-admin/integrations/integration-overview.md b/docs/platform-admin/integrations/integration-overview.md index d57d9a0b48..f19d29a6bc 100644 --- a/docs/platform-admin/integrations/integration-overview.md +++ b/docs/platform-admin/integrations/integration-overview.md @@ -11,33 +11,33 @@ The Run:ai community portal is password protected and access is provided to cust ## Integrations -| Tool | Category | Run:ai support details | Additional Links| -| ------------------ | ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Spark | Orchestration |
It is possible to schedule Spark workflows with the Run:ai scheduler. For details, please contact Run:ai customer support.
| Run:ai customer success community portal: [https://runai.my.site.com/community/s/article/How-to-Run-Spark-jobs-with-Run-AI](https://runai.my.site.com/community/s/article/How-to-Run-Spark-jobs-with-Run-AI){target=_blank} | -| Kubeflow Pipelines | Orchestration | It is possible to schedule kubeflow pipelines with the Run:ai scheduler. For details please contact Run:ai customer support| Run:ai customer success community portal
[https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Kubeflow](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Kubeflow){target=_blank} | -| Apache Airflow | Orchestration | It is possible to schedule Airflow workflows with the Run:ai scheduler. For details, please contact Run:ai customer support. | Run:ai customer success community portal: [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Apache-Airflow](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Apache-Airflow){target=_blank} | -| Argo workflows | Orchestration | It is possible to schedule Argo workflows with the Run:ai scheduler. For details, please contact Run:ai customer support. | Run:ai customer success community portal [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Argo-Workflows](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Argo-Workflows){target=_blank} | -| SeldonX | Orchestration | It is possible to schedule Seldon Core workloads with the Run:ai scheduler. For details, please contact Run:ai customer success. | Run:ai customer success community portal: [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Seldon-Core](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Apache-Airflow){target=_blank} | -| Triton | Orchestration | Usage via docker base image | Quickstart inference [example](../../Researcher/Walkthroughs/quickstart-inference.md) | -| Jupyter Notebook | Development | Run:ai provides integrated support with Jupyter Notebooks | Quickstart example: [https://docs.run.ai/latest/Researcher/Walkthroughs/quickstart-jupyter/](https://docs.run.ai/latest/Researcher/Walkthroughs/quickstart-jupyter/) | -| Jupyter Hub | Development | It is possible to submit Run:ai workloads via JupyterHub. For more information please contact Run:ai customer support | | -| PyCharm | Development | Containers created by Run:ai can be accessed via PyCharm | PyCharm [example](../../Researcher/tools/dev-pycharm.md) | -| VScode | Development | - Containers created by Run:ai can be accessed via Visual Studio Code.
- You can automatically launch Visual Studio code web from the Run:ai console | VSCode [example](../../Researcher/tools/dev-vscode.md) and VSCode quickstart [example](../../Researcher/Walkthroughs/quickstart-vscode.md). | -| Kubeflow notebooks | Development | It is possible to launch a kubeflow notebook with the Run:ai scheduler. For details please contact Run:ai customer support | Run:ai customer success community portal:[https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Kubeflow
](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Kubeflow){target=_blank} | -| Ray | training, inference, data processing. | It is possible to schedule Ray jobs with the Run:ai scheduler. | Run:ai customer success community portal [https://runai.my.site.com/community/s/article/How-to-Integrate-Run-ai-with-Ray](https://runai.my.site.com/community/s/article/How-to-Integrate-Run-ai-with-Ray){target=_blank} | -| TensorBoard | Experiment tracking | Run:ai comes with a preset Tensorboard [Environment](../workloads/assets/environments.md) asset. | TensorBoard [example](../../Researcher/tools/dev-tensorboard.md).
Additional [sample](https://github.com/run-ai/use-cases/tree/master/runai_tensorboard_demo_with_resnet){target=_blank} | -| Weights & Biases | Experiment tracking | It is possible to schedule W&B workloads with the Run:ai scheduler. For details, please contact Run:ai customer success. | Run:ai Customer success community portal: [https://runai.my.site.com/community/s/article/How-to-integrate-with-Weights-and-Biases](https://runai.my.site.com/community/s/article/How-to-integrate-with-Weights-and-Biases){target=_blank}
Additional samples [here](https://github.com/run-ai/use-cases/tree/master/runai_wandb){target=_blank} | -| ClearML | Experiment tracking | It is possible to schedule ClearML workloads with the Run:ai scheduler. For details, please contact Run:ai customer success. | [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-ClearML](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-ClearML){target=_blank} | -| MLFlow | Model Serving | It is possible to use ML Flow together with the Run:ai scheduler. For details, please contact Run:ai customer support. | Run:ai customer success community portal: [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-MLflow](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-MLflow){target=_blank}
Additional MLFlow [sample](https://github.com/run-ai/use-cases/tree/master/runai_mlflow_demo){target=_blank} | -| Hugging Face | Repositories | Run:ai provides an out of the box integration with Hugging Face | | -| Docker Registry | Repositories | Run:ai allows using a docker registry as a Credentials asset | Credentials [documentation](../workloads/assets/credentials.md) | -| S3 | Storage| Run:ai communicates with S3 by defining it as a data source asset| Data source [documentation](../workloads/assets/datasources.md) | -| Github | Storage| Run:ai communicates with GitHub by defining it as a data source asset | Data source [documentation](../workloads/assets/datasources.md) | -| Tensorflow | Training | Run:ai provides out of the box support for submitting TensorFlow workloads | TensorFlow [via API](../../Researcher/cli-reference/new-cli/runai_tensorflow.md) or use Workflow submission [via user interface](../../Researcher/workloads/trainings.md) | -| Pytorch | Training | Run:ai provides out of the box support for submitting PyTorch workloads | PyTorch [via API](../../Researcher/cli-reference/new-cli/runai_pytorch.md) or use Workflow submission [via user interface](../../Researcher/workloads/trainings.md) | -| [Kubeflow MPI](https://www.kubeflow.org/docs/components/training/user-guides/mpi/){target=_blank} | Training | Run:ai provides out of the box support for submitting MPI workloads | MPI [via API](../../Researcher/cli-reference/new-cli/runai_mpi.md) or use Workflow submission [via user interface](../../Researcher/workloads/trainings.md) | -| [XGBoost](https://xgboost.readthedocs.io/en/stable/){target=_blank} | Training | Run:ai provides out of the box support for submitting XGBoost workloads | XGBoost [via API](../../Researcher/cli-reference/new-cli/runai_xgboost.md) or use Workflow submission [via user interface](../../Researcher/workloads/trainings.md) | -| [Karpenter](https://karpenter.sh){target=_blank} | Cost Optimization | Run:ai provides out of the box support for Karpenter to save cloud costs | Integration notes with Karpenter can be found [here](karpenter.md) | +| Tool | Category | Run:ai support details | Additional Information| +| ------------------ | ----------------| --------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Triton | Orchestration | Supported | Usage via docker base image. Quickstart inference [example](../../Researcher/Walkthroughs/quickstart-inference.md) | +| Spark | Orchestration | |
It is possible to schedule Spark workflows with the Run:ai scheduler. For details, please contact Run:ai customer support.
Sample code can be found in the Run:ai customer success community portal: [https://runai.my.site.com/community/s/article/How-to-Run-Spark-jobs-with-Run-AI](https://runai.my.site.com/community/s/article/How-to-Run-Spark-jobs-with-Run-AI){target=_blank} | +| Kubeflow Pipelines | Orchestration | | It is possible to schedule kubeflow pipelines with the Run:ai scheduler. For details please contact Run:ai customer support. Sample code can be found in the Run:ai customer success community portal
[https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Kubeflow](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Kubeflow){target=_blank} | +| Apache Airflow | Orchestration | | It is possible to schedule Airflow workflows with the Run:ai scheduler. For details, please contact Run:ai customer support. Sample code can be found in the Run:ai customer success community portal: [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Apache-Airflow](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Apache-Airflow){target=_blank} | +| Argo workflows | Orchestration | | It is possible to schedule Argo workflows with the Run:ai scheduler. For details, please contact Run:ai customer support. Sample code can be found in the Run:ai customer success community portal [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Argo-Workflows](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Argo-Workflows){target=_blank} | +| SeldonX | Orchestration | | It is possible to schedule Seldon Core workloads with the Run:ai scheduler. For details, please contact Run:ai customer success. Sample code can be found in the Run:ai customer success community portal: [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Seldon-Core](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Apache-Airflow){target=_blank} | +| Jupyter Notebook | Development | Supported | Run:ai provides integrated support with Jupyter Notebooks. Quickstart example: [https://docs.run.ai/latest/Researcher/Walkthroughs/quickstart-jupyter/](https://docs.run.ai/latest/Researcher/Walkthroughs/quickstart-jupyter/) | +| Jupyter Hub | Development | | It is possible to submit Run:ai workloads via JupyterHub. For more information please contact Run:ai customer support | | +| PyCharm | Development | Supported | Containers created by Run:ai can be accessed via PyCharm. PyCharm [example](../../Researcher/tools/dev-pycharm.md) | +| VScode | Development | Supported | - Containers created by Run:ai can be accessed via Visual Studio Code. [example](../../Researcher/tools/dev-vscode.md)
- You can automatically launch Visual Studio code web from the Run:ai console. [example](../../Researcher/Walkthroughs/quickstart-vscode.md). | +| Kubeflow notebooks | Development | | It is possible to launch a kubeflow notebook with the Run:ai scheduler. For details please contact Run:ai customer support Sample code can be found in the Run:ai customer success community portal:[https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Kubeflow
](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-Kubeflow){target=_blank} | +| Ray | training, inference, data processing. | | It is possible to schedule Ray jobs with the Run:ai scheduler. Sample code can be found in the Run:ai customer success community portal [https://runai.my.site.com/community/s/article/How-to-Integrate-Run-ai-with-Ray](https://runai.my.site.com/community/s/article/How-to-Integrate-Run-ai-with-Ray){target=_blank} | +| TensorBoard | Experiment tracking | Supported | Run:ai comes with a preset Tensorboard [Environment](../workloads/assets/environments.md) asset. TensorBoard [example](../../Researcher/tools/dev-tensorboard.md).
Additional [sample](https://github.com/run-ai/use-cases/tree/master/runai_tensorboard_demo_with_resnet){target=_blank} | +| Weights & Biases | Experiment tracking | | It is possible to schedule W&B workloads with the Run:ai scheduler. For details, please contact Run:ai customer success. | Run:ai Customer success community portal: [https://runai.my.site.com/community/s/article/How-to-integrate-with-Weights-and-Biases](https://runai.my.site.com/community/s/article/How-to-integrate-with-Weights-and-Biases){target=_blank}
Additional samples [here](https://github.com/run-ai/use-cases/tree/master/runai_wandb){target=_blank} | +| ClearML | Experiment tracking | | It is possible to schedule ClearML workloads with the Run:ai scheduler. For details, please contact Run:ai customer success. | [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-ClearML](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-ClearML){target=_blank} | +| MLFlow | Model Serving | | It is possible to use ML Flow together with the Run:ai scheduler. For details, please contact Run:ai customer support. Sample code can be found in the Run:ai customer success community portal: [https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-MLflow](https://runai.my.site.com/community/s/article/How-to-integrate-Run-ai-with-MLflow){target=_blank}
Additional MLFlow [sample](https://github.com/run-ai/use-cases/tree/master/runai_mlflow_demo){target=_blank} | +| Hugging Face | Repositories | Supported | Run:ai provides an out of the box integration with Hugging Face | +| Docker Registry | Repositories | Supported | Run:ai allows using a docker registry as a [Credentials](../workloads/assets/credentials.md) asset. | +| S3 | Storage | Supported | Run:ai communicates with S3 by defining a [data source](../workloads/assets/datasources.md) asset. | +| Github | Storage | Supported | Run:ai communicates with GitHub by defining it as a [data source](../workloads/assets/datasources.md) asset | +| Tensorflow | Training | Supported | Run:ai provides out of the box support for submitting TensorFlow workloads [via API](../../Researcher/cli-reference/new-cli/runai_tensorflow.md) or by submitting workloads [via user interface](../../Researcher/workloads/trainings.md). | +| Pytorch | Training | Supported | Run:ai provides out of the box support for submitting PyTorch workloads [via API](../../Researcher/cli-reference/new-cli/runai_pytorch.md) or by submitting workloads [via user interface](../../Researcher/workloads/trainings.md). | +| [Kubeflow MPI](https://www.kubeflow.org/docs/components/training/user-guides/mpi/){target=_blank} | Training | Supported |Run:ai provides out of the box support for submitting MPI workloads [via API](../../Researcher/cli-reference/new-cli/runai_mpi.md) or by submitting workloads [via user interface](../../Researcher/workloads/trainings.md) | +| [XGBoost](https://xgboost.readthedocs.io/en/stable/){target=_blank} | Training | Supported | Run:ai provides out of the box support for submitting XGBoost workloads [via API](../../Researcher/cli-reference/new-cli/runai_xgboost.md) or by submitting workloads [via user interface](../../Researcher/workloads/trainings.md) | +| [Karpenter](https://karpenter.sh){target=_blank} | Cost Optimization | Supported | Run:ai provides out of the box support for Karpenter to save cloud costs. Integration notes with Karpenter can be found [here](karpenter.md) | ## Kubernetes Workloads Integration diff --git a/mkdocs.yml b/mkdocs.yml index 2f8bff919f..561d3ec661 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -154,26 +154,26 @@ plugins: 'Researcher/workloads/statuses.md' : 'Researcher/workloads/overviews/managing-workloads.md' 'platform-admin/workloads/submitting-workloads.md' : 'platform-admin/workloads/overviews/managing-workloads.md' 'Researcher/workloads/workload-support.md' : 'Researcher/workloads/overviews/workload-support.md' - 'admin/integration/kubeflow.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/jupyterhub.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/airflow.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/mlflow.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/seldon.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/clearml.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/argo-workflows.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/weights-and-biases.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/spark.md' : 'platform-admin/workloads/integrations.md' - 'admin/integration/ray.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/kubeflow.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/jupyterhub.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/airflow.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/mlflow.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/seldon.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/clearml.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/argo-workflows.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/weights-and-biases.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/spark.md' : 'platform-admin/workloads/integrations.md' - 'Administrator/integration/ray.md' : 'platform-admin/workloads/integrations.md' + 'admin/integration/kubeflow.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/jupyterhub.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/airflow.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/mlflow.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/seldon.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/clearml.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/argo-workflows.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/weights-and-biases.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/spark.md' : 'platform-admin/integrations/integration-overview.md' + 'admin/integration/ray.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/kubeflow.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/jupyterhub.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/airflow.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/mlflow.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/seldon.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/clearml.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/argo-workflows.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/weights-and-biases.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/spark.md' : 'platform-admin/integrations/integration-overview.md' + 'Administrator/integration/ray.md' : 'platform-admin/integrations/integration-overview.md' 'platform-admin/workloads/assets/secrets.md' : 'Researcher/best-practices/secrets-as-env-var-in-cli.md' 'admin/runai-setup/access-control/rbac.md' : 'admin/authentication/roles.md' 'platform-admin/workloads/assets/existing-PVC.md' : 'platform-admin/workloads/assets/datasources.md'