4_grpc_requests_multi_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f73046ff",
   "metadata": {},
   "source": [
    "# Using the model server to do an inference using gRPC\n",
    "\n",
    "In this notebook we are going to use the gRPC inference endpoint to query our model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "443e7e73-24cb-4f03-9491-a6edcc24f0cc",
   "metadata": {
    "tags": []
   },
   "source": [
    "## First, replace the placeholder with the grpcUrl you got at the previous step from the Model Serving configuration\n",
    "\n",
    "The port, the model name and the classes definition file are already filled in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db9df000-a171-4652-8160-272f81e49612",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install grpcio grpcio-tools\n",
    "!pip install --upgrade pip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d17b252-7827-4cae-adb0-f98c9d80bcd7",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = 'fraud'\n",
    "grpc_host = 'change_me' # e.g. 'modelmesh-serving.<your-namespace>'\n",
    "grpc_port = 8033"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4269da9e-5683-4531-9a3f-a1cdad42e3af",
   "metadata": {},
   "source": [
    "## Inspect the gRPC Endpoint\n",
    "\n",
    "Check the gRPC endpoint's model metadata:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "545aa5f4-356f-4e70-b7e6-cd352a68927a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append('./utils')\n",
    "\n",
    "# grpc_predict_v2_pb2 and grpc_predict_v2_pb2_grpc were created from grpc_predict_v2.proto using protoc\n",
    "import grpc\n",
    "import utils.grpc_predict_v2_pb2 as grpc_predict_v2_pb2\n",
    "import utils.grpc_predict_v2_pb2_grpc as grpc_predict_v2_pb2_grpc\n",
    "\n",
    "\n",
    "channel = grpc.insecure_channel(f\"{grpc_host}:{grpc_port}\")\n",
    "stub = grpc_predict_v2_pb2_grpc.GRPCInferenceServiceStub(channel)\n",
    "\n",
    "request = grpc_predict_v2_pb2.ModelMetadataRequest(name=model_name)\n",
    "response = stub.ModelMetadata(request)\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd5affbf-36c3-4e17-9788-5fc0904de143",
   "metadata": {},
   "source": [
    "### Request Function\n",
    "\n",
    "Build and submit the gRPC request. \n",
    "\n",
    "Note: You submit the data in the same format that you used for an ONNX inference.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67c1d001-ff99-414a-95d4-5729d5849298",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def grpc_request(data):\n",
    "    # request content building\n",
    "    inputs = []\n",
    "    inputs.append(grpc_predict_v2_pb2.ModelInferRequest().InferInputTensor())\n",
    "    inputs[0].name = \"dense_input\"\n",
    "    inputs[0].datatype = \"FP32\"\n",
    "    inputs[0].shape.extend([1, 5])\n",
    "    inputs[0].contents.fp32_contents.extend(data)\n",
    "\n",
    "    # request building\n",
    "    request = grpc_predict_v2_pb2.ModelInferRequest()\n",
    "    request.model_name = model_name\n",
    "    request.inputs.extend(inputs)\n",
    "\n",
    "    response = stub.ModelInfer(request)\n",
    "    result_arr = np.frombuffer(response.raw_output_contents[0], dtype=np.float32)\n",
    "    return result_arr"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "911b1015-28b0-4d60-bc17-7b30326b97bc",
   "metadata": {},
   "source": [
    "### Run the Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fc549f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Load the scaler\n",
    "import pickle\n",
    "with open('artifact/scaler.pkl', 'rb') as handle:\n",
    "    scaler = pickle.load(handle)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12947866-e0f5-4c72-ba9a-04229b1af990",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [0.3111400080477545, 1.9459399775518593, 1.0, 0.0, 0.0]\n",
    "prediction = grpc_request(scaler.transform([data]).tolist()[0])\n",
    "prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "946f9f1d-b24a-4aa6-b839-f0e8013ef84d",
   "metadata": {},
   "outputs": [],
   "source": [
    "threshhold = 0.95\n",
    "\n",
    "if (prediction[0] > threshhold):\n",
    "    print('fraud')\n",
    "else:\n",
    "    print('not fraud')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d7f6b51",
   "metadata": {},
   "source": [
    "## Example 1: user buys a coffee\n",
    "\n",
    "In this example, the user is buying a coffee. The parameters given to the model are:\n",
    "* same location as the last transaction (distance=0)\n",
    "* same median price as the last transaction (ratio_to_median=1)\n",
    "* using a pin number (pin=1)\n",
    "* using the credit card chip (chip=1)\n",
    "* not an online transaction (online=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0a68b67-b109-4a2f-b097-092f4a4d25ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [0.0, 1.0, 1.0, 1.0, 0.0]\n",
    "prediction = grpc_request(scaler.transform([data]).tolist()[0])\n",
    "threshhold = 0.95\n",
    "\n",
    "if (prediction[0] > threshhold):\n",
    "    print('The model predicts that this is fraud')\n",
    "else:\n",
    "    print('The model predicts that this is not fraud')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1dd27d88",
   "metadata": {},
   "source": [
    "## Example 2: fraudulent transaction\n",
    "\n",
    "In this example, someone stole the user's credit card and is buying something online. The parameters given to the model are:\n",
    "* very far away from the last transaction (distance=100)\n",
    "* median price similar to the last transaction (ratio_to_median=1.2)\n",
    "* not using a pin number (pin=0)\n",
    "* not using the credit card chip (chip=0)\n",
    "* is an online transaction (online=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a736a21",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = [100, 1.2, 0.0, 0.0, 1.0]\n",
    "prediction = grpc_request(scaler.transform([data]).tolist()[0])\n",
    "threshhold = 0.95\n",
    "\n",
    "if (prediction[0] > threshhold):\n",
    "    print('The model predicts that this is fraud')\n",
    "else:\n",
    "    print('The model predicts that this is not fraud')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}