seed_code/01-ml-deploy/notebooks/01-Deploy-SageMaker-Endpoint-Custom-Container.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Deploy ML Model - Custom Contaienr\n",
    "\n",
    "**SageMaker Studio Kernel**: Data Science\n",
    "\n",
    "In this exercise you will do:\n",
    " - Get latest approved model from the Amazon SageMaker Model Registry\n",
    " - Deploy an Amazon SageMaker Real-Time Endpoint using the latest approved model and a Custom Container (BYOC)\n",
    " - Test the endpoint by performing a prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Install Requirements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "! pip install sagemaker-studio-image-build"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Build Container\n",
    "\n",
    "The Dockerfile defined is created starting from the public tensorflow 2.5.0 gpu image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "! pygmentize ./../algorithms/processing/docker/custom-container/Dockerfile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "%%sh\n",
    "\n",
    "cd ./../algorithms/inference-custom-container\n",
    "\n",
    "sm-docker build . --file docker/custom-container/Dockerfile --repository sm-end-to-end-inference-mlops-custom:latest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Part 1/4 - Setup\n",
    "Here we'll import some libraries and define some variables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Import required modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "from botocore.exceptions import ClientError\n",
    "from datetime import datetime\n",
    "import logging\n",
    "import sagemaker.session\n",
    "from sagemaker.model import Model\n",
    "import traceback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "s3_client = boto3.client(\"s3\")\n",
    "sagemaker_client = boto3.client(\"sagemaker\")\n",
    "\n",
    "region = boto3.session.Session().region_name\n",
    "role_name = \"mlops-sagemaker-execution-role\"\n",
    "role = \"arn:aws:iam::{}:role/{}\".format(boto3.client('sts').get_caller_identity().get('Account'), role_name)\n",
    "\n",
    "kms_account_id = boto3.client('sts').get_caller_identity().get('Account')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "logging.basicConfig(level=logging.INFO)\n",
    "LOGGER = logging.getLogger(__name__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Part 2/4 - Model Package Definition\n",
    "During this steps, we are retrieving model informations from the Amazon SageMaker Model Registry"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Get Approved Model Packages\n",
    "\n",
    "This method can be used for returning the last approved model from the specified model package group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model_package_group = \"ml-end-to-end-group\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    # Get the latest approved model package\n",
    "    response = sagemaker_client.list_model_packages(\n",
    "        ModelPackageGroupName=model_package_group,\n",
    "        ModelApprovalStatus=\"Approved\",\n",
    "        SortBy=\"CreationTime\",\n",
    "        SortOrder=\"Descending\",\n",
    "        MaxResults=1,\n",
    "    )\n",
    "    approved_packages = response[\"ModelPackageSummaryList\"]\n",
    "\n",
    "    # Return error if no packages found\n",
    "    if len(approved_packages) == 0:\n",
    "        error_message = (\"No approved ModelPackage found for ModelPackageGroup: {}\".format(model_package_group))\n",
    "        LOGGER.error(\"{}\".format(error_message))\n",
    "\n",
    "        raise Exception(error_message)\n",
    "\n",
    "    model_package = approved_packages[0]\n",
    "    LOGGER.info(\"Identified the latest approved model package: {}\".format(model_package))\n",
    "except ClientError as e:\n",
    "    stacktrace = traceback.format_exc()\n",
    "    error_message = e.response[\"Error\"][\"Message\"]\n",
    "    LOGGER.error(\"{}\".format(stacktrace))\n",
    "\n",
    "    raise Exception(error_message)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### List Model Packages\n",
    "\n",
    "This method can be used for listing all the registered models in a Model Package Group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model_package_arn = model_package[\"ModelPackageArn\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    model_package = sagemaker_client.describe_model_package(\n",
    "        ModelPackageName=model_package_arn\n",
    "    )\n",
    "\n",
    "    LOGGER.info(\"{}\".format(model_package))\n",
    "\n",
    "    if len(model_package) == 0:\n",
    "        error_message = (\"No ModelPackage found for: {}\".format(model_package_arn))\n",
    "        LOGGER.error(\"{}\".format(error_message))\n",
    "\n",
    "        raise Exception(error_message)\n",
    "except ClientError as e:\n",
    "    stacktrace = traceback.format_exc()\n",
    "    error_message = e.response[\"Error\"][\"Message\"]\n",
    "    LOGGER.error(\"{}\".format(stacktrace))\n",
    "\n",
    "    raise Exception(error_message)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Part 3/4 - Deploy an Amazon SageMaker Endpoint\n",
    "Here we are deploying an Amazon SageMaker Endpoint by using the ML model taken from the Model Registry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "ecr_image_name = \"sm-end-to-end-inference-mlops-custom\"\n",
    "ecr_image_version = \"latest\"\n",
    "\n",
    "kms_alias = \"ml-kms\"\n",
    "\n",
    "bucket_inference = \"\"\n",
    "\n",
    "inference_instance_count = 1\n",
    "inference_instance_type = \"ml.m5.xlarge\"\n",
    "\n",
    "model_package_group = \"ml-end-to-end-group\"\n",
    "\n",
    "\n",
    "monitoring_output_path = \"data/monitoring/captured\"\n",
    "\n",
    "training_framework_version = 2.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "kms_key = \"arn:aws:kms:{}:{}:alias/{}\".format(region, kms_account_id, kms_alias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "boto_session = boto3.Session(region_name=region)\n",
    "\n",
    "sagemaker_client = boto_session.client(\"sagemaker\")\n",
    "runtime_client = boto_session.client(\"sagemaker-runtime\")\n",
    "\n",
    "sagemaker_session = sagemaker.session.Session(\n",
    "    boto_session=boto_session,\n",
    "    sagemaker_client=sagemaker_client,\n",
    "    sagemaker_runtime_client=runtime_client,\n",
    "    default_bucket=bucket_inference\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Create SageMaker model\n",
    "\n",
    "This method can be used for creating a SageMaker model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    model = Model(\n",
    "        image_uri=\"{}.dkr.ecr.{}.amazonaws.com/{}:{}\".format(\n",
    "            boto3.client('sts').get_caller_identity().get('Account'),\n",
    "            boto3.session.Session().region_name,\n",
    "            ecr_image_name,\n",
    "            ecr_image_version),\n",
    "        model_data=model_package[\"InferenceSpecification\"][\"Containers\"][0][\"ModelDataUrl\"],\n",
    "        model_kms_key=kms_key,\n",
    "        role=role,\n",
    "        sagemaker_session=sagemaker_session\n",
    "    )\n",
    "except Exception as e:\n",
    "    stacktrace = traceback.format_exc()\n",
    "    LOGGER.error(\"{}\".format(stacktrace))\n",
    "\n",
    "    raise e"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Deploy a SageMaker Endpoint\n",
    "\n",
    "Lets deploy the endpoint. If we want to update an existing endpoint, we have to create a new endpoint configuration defined in the method below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def get_deployed_model():\n",
    "    try:\n",
    "        response = sagemaker_client.list_models(\n",
    "            SortBy=\"CreationTime\",\n",
    "            SortOrder=\"Descending\",\n",
    "            MaxResults=1\n",
    "        )\n",
    "\n",
    "        model_name = None\n",
    "\n",
    "        if \"Models\" in response and len(response[\"Models\"]) > 0:\n",
    "            model_name = response[\"Models\"][0][\"ModelName\"]\n",
    "\n",
    "        return model_name\n",
    "    except Exception as e:\n",
    "        stacktrace = traceback.format_exc()\n",
    "        LOGGER.error(\"{}\".format(stacktrace))\n",
    "\n",
    "        raise e"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def update_model(\n",
    "        bucket_inference,\n",
    "        model_name,\n",
    "        model_package_group_name,\n",
    "        env,\n",
    "        inference_instance_count,\n",
    "        inference_instance_type,\n",
    "        kms_key,\n",
    "        monitoring_output_path):\n",
    "    try:\n",
    "        config_name = \"{}-{}-{}\".format(model_package_group_name, env, datetime.today().strftime('%Y-%m-%d-%H-%M-%S'))\n",
    "\n",
    "        LOGGER.info(\"Creating endpoint configuration {}\".format(config_name))\n",
    "\n",
    "        response_endpoint_config = sagemaker_client.create_endpoint_config(\n",
    "            EndpointConfigName=config_name,\n",
    "            ProductionVariants=[\n",
    "                {\n",
    "                    \"VariantName\": \"AllTraffic\",\n",
    "                    \"ModelName\": model_name,\n",
    "                    \"InitialInstanceCount\": inference_instance_count,\n",
    "                    \"InstanceType\": inference_instance_type,\n",
    "                    \"InitialVariantWeight\": 1.0\n",
    "                }\n",
    "            ],\n",
    "            DataCaptureConfig={\n",
    "                'EnableCapture': False\n",
    "            }\n",
    "        )\n",
    "\n",
    "        LOGGER.info(response_endpoint_config)\n",
    "\n",
    "        response = sagemaker_client.update_endpoint(\n",
    "            EndpointName=model_package_group_name + \"-\" + env,\n",
    "            EndpointConfigName=config_name\n",
    "        )\n",
    "\n",
    "        LOGGER.info(\"Update endpoint {}-{}\".format(model_package_group_name, env))\n",
    "        LOGGER.info(response)\n",
    "\n",
    "    except Exception as e:\n",
    "        stacktrace = traceback.format_exc()\n",
    "        LOGGER.info(\"{}\".format(stacktrace))\n",
    "\n",
    "        raise e"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model_package_group = \"ml-end-to-end-group\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    model.deploy(\n",
    "        endpoint_name=model_package_group + \"-custom-dev\",\n",
    "        initial_instance_count=inference_instance_count,\n",
    "        instance_type=inference_instance_type,\n",
    "        update_endpoint=True\n",
    "    )\n",
    "except ClientError as e:\n",
    "    stacktrace = traceback.format_exc()\n",
    "    LOGGER.info(\"{}\".format(stacktrace))\n",
    "\n",
    "    model_name = get_deployed_model()\n",
    "\n",
    "    update_model(\n",
    "            bucket_inference,\n",
    "            model_name,\n",
    "            model_package_group,\n",
    "            \"dev\",\n",
    "            inference_instance_count,\n",
    "            inference_instance_type,\n",
    "            kms_key,\n",
    "            monitoring_output_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Test the SageMaker Endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "* Negative - 0\n",
    "* Neutral - 1\n",
    "* Positive - 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "region = boto3.session.Session().region_name\n",
    "\n",
    "bucket_inference = \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "boto_session = boto3.Session(region_name=region)\n",
    "\n",
    "sagemaker_client = boto_session.client(\"sagemaker\")\n",
    "runtime_client = boto_session.client(\"sagemaker-runtime\")\n",
    "\n",
    "sagemaker_session = sagemaker.session.Session(\n",
    "    boto_session=boto_session,\n",
    "    sagemaker_client=sagemaker_client,\n",
    "    sagemaker_runtime_client=runtime_client,\n",
    "    default_bucket=bucket_inference\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model_package_group = \"ml-end-to-end-group\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.deserializers import CSVDeserializer\n",
    "from sagemaker.serializers import CSVSerializer\n",
    "from sagemaker.predictor import Predictor\n",
    "\n",
    "predictor = Predictor(\n",
    "    endpoint_name=model_package_group + \"-custom-dev\",\n",
    "    accept_type=\"text/csv\",\n",
    "    serializer=CSVSerializer(),\n",
    "    deserializer=CSVDeserializer(),\n",
    "    sagemaker_session=sagemaker_session\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "inputs = [\"ti imploro di guardare questo documentario. molto spaventoso e informativo. uno dei motivi esatti che sto eliminando fb entir\"]\n",
    "\n",
    "result = predictor.predict(inputs)\n",
    "\n",
    "LOGGER.info(\"{}\".format(result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Delete Endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "predictor.delete_model()\n",
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We have just seen how to deploy a ML models by using Amazon SageMaker Hosting Services and real-time endpoints. \n",
    "For creating monitoring jobs for checking the quality of the deployed model, we can execute the following lab (Optional).\n",
    "\n",
    " > [Model-Monitor](./03-Model-Monitor.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "If you want to analyze which is the importance of the features, and in particular for NLP which sections of text are most important for the predictions made by your model your text, you can execute the followint lab (Optional)\n",
    "\n",
    " > [Model-Monitor](./02-Model-Explainability.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now we are ready to execute our end to end workflow using a Pipeline.\n",
    "\n",
    " > [Pipeline](./04-Pipeline-Deployment.ipynb)"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "interpreter": {
   "hash": "7830b2e0dcc405ab83456d8c26dd7c2db32ddf1a7b2e64ef505b215ebac66515"
  },
  "kernelspec": {
   "name": "python3",
   "language": "python",
   "display_name": "Python 3 (ipykernel)"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}