Skip to content

Commit 2b9189a

Browse files
xanwalshohei1029
xanwal
and
shohei1029
authored
CLI Example Registered Model (Azure#1800)
* Init * readme index * Add readme * Readme edit * Fix name * Update cli/deploy-moe-minimal-single-model-registered.sh Co-authored-by: Shohei Nagata <[email protected]> Co-authored-by: Shohei Nagata <[email protected]>
1 parent d379da7 commit 2b9189a

File tree

7 files changed

+150
-0
lines changed

7 files changed

+150
-0
lines changed

‎.github/workflows/cli-scripts-deploy-moe-minimal-single-model-registered.yml

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: cli-scripts-deploy-moe-minimal-single-model-registered
2+
on:
3+
workflow_dispatch:
4+
schedule:
5+
- cron: "0 0/4 * * *"
6+
pull_request:
7+
branches:
8+
- main
9+
- sdk-preview
10+
paths:
11+
- cli/deploy-moe-minimal-single-model-registered.sh
12+
- .github/workflows/cli-scripts-deploy-moe-minimal-single-model-registered.yml
13+
- cli/setup.sh
14+
jobs:
15+
build:
16+
runs-on: ubuntu-latest
17+
steps:
18+
- name: check out repo
19+
uses: actions/checkout@v2
20+
- name: azure login
21+
uses: azure/login@v1
22+
with:
23+
creds: ${{secrets.AZ_CREDS}}
24+
- name: setup
25+
run: bash setup.sh
26+
working-directory: cli
27+
continue-on-error: true
28+
- name: test script script
29+
run: set -e; bash -x deploy-moe-minimal-single-model-registered.sh
30+
working-directory: cli

‎cli/README.md

+1
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ path|status|
5757
[deploy-mlcompute-update-to-system-identity.sh](deploy-mlcompute-update-to-system-identity.sh)|[![deploy-mlcompute-update-to-system-identity](https://github.com/Azure/azureml-examples/workflows/cli-scripts-deploy-mlcompute-update-to-system-identity/badge.svg?branch=main)](https://github.com/Azure/azureml-examples/actions/workflows/cli-scripts-deploy-mlcompute-update-to-system-identity.yml)
5858
[deploy-mlcompute-update-to-user-identity.sh](deploy-mlcompute-update-to-user-identity.sh)|[![deploy-mlcompute-update-to-user-identity](https://github.com/Azure/azureml-examples/workflows/cli-scripts-deploy-mlcompute-update-to-user-identity/badge.svg?branch=main)](https://github.com/Azure/azureml-examples/actions/workflows/cli-scripts-deploy-mlcompute-update-to-user-identity.yml)
5959
[deploy-moe-autoscale.sh](deploy-moe-autoscale.sh)|[![deploy-moe-autoscale](https://github.com/Azure/azureml-examples/workflows/cli-scripts-deploy-moe-autoscale/badge.svg?branch=main)](https://github.com/Azure/azureml-examples/actions/workflows/cli-scripts-deploy-moe-autoscale.yml)
60+
[deploy-moe-minimal-single-model-registered.sh](deploy-moe-minimal-single-model-registered.sh)|[![deploy-moe-minimal-single-model-registered](https://github.com/Azure/azureml-examples/workflows/cli-scripts-deploy-moe-minimal-single-model-registered/badge.svg?branch=main)](https://github.com/Azure/azureml-examples/actions/workflows/cli-scripts-deploy-moe-minimal-single-model-registered.yml)
6061
[deploy-moe-vnet-mlflow.sh](deploy-moe-vnet-mlflow.sh)|[![deploy-moe-vnet-mlflow](https://github.com/Azure/azureml-examples/workflows/cli-scripts-deploy-moe-vnet-mlflow/badge.svg?branch=main)](https://github.com/Azure/azureml-examples/actions/workflows/cli-scripts-deploy-moe-vnet-mlflow.yml)
6162
[deploy-moe-vnet.sh](deploy-moe-vnet.sh)|[![deploy-moe-vnet](https://github.com/Azure/azureml-examples/workflows/cli-scripts-deploy-moe-vnet/badge.svg?branch=main)](https://github.com/Azure/azureml-examples/actions/workflows/cli-scripts-deploy-moe-vnet.yml)
6263
[deploy-rest.sh](deploy-rest.sh)|[![deploy-rest](https://github.com/Azure/azureml-examples/workflows/cli-scripts-deploy-rest/badge.svg?branch=main)](https://github.com/Azure/azureml-examples/actions/workflows/cli-scripts-deploy-rest.yml)

‎cli/deploy-moe-minimal-single-model-registered.sh

+80
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
#!/bin/bash
2+
set -e
3+
4+
# <set_variables>
5+
RAND=`echo $RANDOM`
6+
ENDPOINT_NAME="endpt-moe-$RAND"
7+
MODEL_VERSION=$RAND
8+
ASSET_PATH=endpoints/online/model-1
9+
# </set_variables>
10+
11+
BASE_PATH=endpoints/online/managed/minimal/single-model-registered
12+
13+
# Helper function to change parameters in yaml files
14+
change_vars() {
15+
for FILE in "$@"; do
16+
TMP="${FILE}_"
17+
cp $FILE $TMP
18+
readarray -t VARS < <(cat $TMP | grep -oP '{{.*?}}' | sed -e 's/[}{]//g');
19+
for VAR in "${VARS[@]}"; do
20+
sed -i "s/{{${VAR}}}/${!VAR}/g" $TMP
21+
done
22+
done
23+
}
24+
25+
# <create_endpoint>
26+
az ml online-endpoint create -n $ENDPOINT_NAME
27+
# </create_endpoint>
28+
29+
# Check if endpoint was successful
30+
endpoint_status=`az ml online-endpoint show --name $ENDPOINT_NAME --query "provisioning_state" -o tsv `
31+
echo $endpoint_status
32+
if [[ $endpoint_status == "Succeeded" ]]
33+
then
34+
echo "Endpoint created successfully"
35+
else
36+
echo "Endpoint creation failed"
37+
exit 1
38+
fi
39+
40+
# <register_model>
41+
change_vars $BASE_PATH/model.yml
42+
az ml model create -f $BASE_PATH/model.yml_
43+
# </register_model>
44+
45+
rm $BASE_PATH/model.yml_
46+
47+
# <create_deployment>
48+
change_vars $BASE_PATH/deployment.yml
49+
az ml online-deployment create -f $BASE_PATH/deployment.yml_ --all-traffic
50+
# </create-deployment>
51+
52+
rm $BASE_PATH/deployment.yml_
53+
54+
# Check if deployment was successful
55+
deploy_status=`az ml online-deployment show --name smr --endpoint $ENDPOINT_NAME --query "provisioning_state" -o tsv `
56+
echo $deploy_status
57+
if [[ $deploy_status == "Succeeded" ]]
58+
then
59+
echo "Deployment completed successfully"
60+
else
61+
echo "Deployment failed"
62+
exit 1
63+
fi
64+
65+
# Get key
66+
echo "Getting access key..."
67+
KEY=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME --query primaryKey -o tsv )
68+
69+
# Get scoring url
70+
echo "Getting scoring url..."
71+
SCORING_URL=$(az ml online-endpoint show -n $ENDPOINT_NAME --query scoring_uri -o tsv )
72+
echo "Scoring url is $SCORING_URL"
73+
74+
# <test_deployment_conda_in_dockerfile>
75+
curl -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d @$ASSET_PATH/sample-request.json $SCORING_URL
76+
# </test_deployment_conda_in_dockerfile>
77+
78+
# <delete_online_endpoint>
79+
az ml online-endpoint delete -y -n $ENDPOINT_NAME --no-wait
80+
# </delete_online_endpoint>

‎cli/endpoints/online/managed/minimal/single-model-registered/README.md

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Deploy an already-registered model to a Managed Online Example
2+
In this example, a model is registered before deploying an endpoint rather than defining the local model inline. The model is then included in the deployment YAML as a reference to the registered model.
3+
4+
To run this example, please execute the [script](../../../../../deploy-moe-minimal-single-model-registered.sh) in the CLI folder.
5+
6+
# Model
7+
The model is registered using the `az ml model create -f` command and passing the `model.yml` file. This example uses a model with a local path, thereby uploading the model assets to Azure at registration time. However, a path to a datastore may also be used to register a model that already exists in Azure Storage.
8+
9+
# Deployment
10+
In an inline-model deployment, the `model` block is multiple lines and may contain sub-fields such as name, version, and path. Since the model is already registered ahead of deployment creation, the model block is replaced with a one-line reference to the registered model in the format `azureml:<MODEL_NAME>:<MODEL_VERSION>`.

‎cli/endpoints/online/managed/minimal/single-model-registered/deployment.yml

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
2+
name: smr
3+
endpoint_name: {{ENDPOINT_NAME}}
4+
model: azureml:minimalregistered:{{MODEL_VERSION}}
5+
code_configuration:
6+
code: ../../../model-1/onlinescoring
7+
scoring_script: score.py
8+
environment:
9+
image: mcr.microsoft.com/azureml/minimal-ubuntu20.04-py38-cpu-inference:latest
10+
conda_file: env.yml
11+
instance_type: Standard_DS2_v2
12+
instance_count: 1

‎cli/endpoints/online/managed/minimal/single-model-registered/env.yml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
name: model-env
2+
channels:
3+
- conda-forge
4+
dependencies:
5+
- python=3.7
6+
- numpy=1.21.2
7+
- pip=21.2.4
8+
- scikit-learn=0.24.2
9+
- scipy=1.7.1
10+
- pip:
11+
- azureml-defaults==1.38.0
12+
- joblib==1.0.1
13+
- azureml-inference-server-http

‎cli/endpoints/online/managed/minimal/single-model-registered/model.yml

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
$schema: https://azuremlschemas.azureedge.net/latest/model.schema.json
2+
name: minimalregistered
3+
version: {{MODEL_VERSION}}
4+
path: ../../../model-1/model

0 commit comments

Comments
 (0)