|
| 1 | +Seldon |
| 2 | +========== |
| 3 | + |
| 4 | +<p align="left";> |
| 5 | + <a href="https://www.elastic.co" alt="elk"> |
| 6 | + <img src="./images/logos/seldon_logo.jpg" align="center" alt="ELK logo" width="200px" /> |
| 7 | + </a> |
| 8 | +</p> |
| 9 | + |
| 10 | +[Seldon Core](https://www.seldon.io/tech/products/core/https://www.seldon.io/tech/products/core/) is an open source platform for deploying machine learning models on a Kubernetes cluster. It extends Kubernetes with **its own custom resource SeldonDeployment** where you can define your runtime inference graph made up of models and other components that Seldon will manage. |
| 11 | + |
| 12 | + |
| 13 | +##Install Seldon Core |
| 14 | + |
| 15 | +#### 1. Deploy Seldon core |
| 16 | + |
| 17 | +Set seldon-core-operator.enabled to `true` in the `values.yaml`. |
| 18 | + |
| 19 | + |
| 20 | +## Deploy your model |
| 21 | + |
| 22 | +#### 1. Wrap your model ( Dockerize it ) |
| 23 | + |
| 24 | +To allow your component (model, router etc.) to be managed by seldon-core it needs to be built into a **Docker container** and to expose the appropriate [service microservice APIs over REST or gRPC](https://docs.seldon.io/projects/seldon-core/en/latest/reference/apis/internal-api.html). |
| 25 | + |
| 26 | +To wrap your model follow these [instructions](https://docs.seldon.io/projects/seldon-core/en/latest/wrappers/README.html). |
| 27 | + |
| 28 | +#### 2. Create your inference graph |
| 29 | + |
| 30 | +Seldon Core extends Kubernetes with its own custom resource **SeldonDeployment** where you can define your runtime [inference graph](https://docs.seldon.io/projects/seldon-core/en/latest/graph/inference-graph.html) made up of models and other components that Seldon will manage. |
| 31 | + |
| 32 | +A **SeldonDeployment** is a JSON or YAML file that allows you to define your graph of component images and the resources each of those images will need to run (using a Kubernetes PodTemplateSpec). The following is minimal example for a single model, in YAML: |
| 33 | + |
| 34 | +``` |
| 35 | +apiVersion: machinelearning.seldon.io/v1alpha2 |
| 36 | +kind: SeldonDeployment |
| 37 | +metadata: |
| 38 | + name: seldon-model |
| 39 | +spec: |
| 40 | + name: test-deployment |
| 41 | + predictors: |
| 42 | + - componentSpecs: |
| 43 | + - spec: |
| 44 | + containers: |
| 45 | + - name: classifier |
| 46 | + image: seldonio/mock_classifier:1.0 |
| 47 | + graph: |
| 48 | + children: [] |
| 49 | + endpoint: |
| 50 | + type: REST |
| 51 | + name: classifier |
| 52 | + type: MODEL |
| 53 | + name: example |
| 54 | + replicas: 1 |
| 55 | +``` |
| 56 | +The key components are: |
| 57 | + |
| 58 | +* A list of Predictors, each with a specification for the number of replicas. |
| 59 | + * Each defines a graph and its set of deployments. Multiple predictors is useful when you want to split traffic between a main graph and a canary or for other production rollout scenarios. |
| 60 | +* For each predictor a list of componentSpecs. Each componentSpec is a Kubernetes PodTemplateSpec which Seldon will build into a Kubernetes Deployment. Place here the images from your graph and their requirements, e.g. Volumes, ImagePullSecrets, Resources Requests etc. |
| 61 | +* A graph specification that describes how your components are joined together. |
| 62 | + |
| 63 | +To understand the inference graph definition in detail see [here](https://docs.seldon.io/projects/seldon-core/en/latest/reference/seldon-deployment.html) |
| 64 | + |
| 65 | +#### 3. Deploy your model |
| 66 | + |
| 67 | + Once you have created your inference graph as a JSON or YAML Seldon Deployment resource ( the previous step ) you can deploy to Kubernetes with kubectl: |
| 68 | + |
| 69 | +``` |
| 70 | +kubectl apply -f my_deployment.yaml |
| 71 | +``` |
| 72 | +To delete ( or manage ) your **SeldonDeployment** you can use kubectl for the custom resource SeldonDeployment, for example to see if there are any models deployed: |
| 73 | + |
| 74 | +``` |
| 75 | +kubectl get seldondeployment |
| 76 | +``` |
| 77 | +to delete the model `seldon-model`: |
| 78 | + |
| 79 | +``` |
| 80 | +kubectl delete seldondeployment seldon-model |
| 81 | +``` |
0 commit comments