22. Tensorflow使用tf-serving实现在线预估服务.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zwBCE43Cv3PH"
   },
   "source": [
    "## Tensorflow使用tf-serving实现在线预估服务"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "背景：\n",
    "* tensorflow提供了tf-serving功能，用于高性能的线上tf预估服务\n",
    "* 使用tf-serving的最佳方式，是使用docker进行模型加载启动服务\n",
    "\n",
    "演示步骤：\n",
    "1. 将Keras模型导出成tf-serving需要的格式\n",
    "2. 使用docker拉取tf-serving镜像\n",
    "3. 使用docker命令加载模型启动服务\n",
    "4. 使用HTTP/GRPC即可访问服务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 1. 准备和训练一个模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5IoRbCA2n0_V"
   },
   "source": [
    "#### 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UEfJ8TcMpe-2"
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"./datas/heart/heart.csv\")\n",
    "\n",
    "# 把thal列变成数字编码\n",
    "df['thal'] = pd.Categorical(df['thal'])\n",
    "df['thal'] = df['thal'].cat.codes\n",
    "\n",
    "# 要预测的目标，这是个二分类问题\n",
    "target = df.pop('target')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LmCl5R5C2IKo"
   },
   "source": [
    "#### 构建dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "W6Yc-D3aqyBb"
   },
   "outputs": [],
   "source": [
    "# 构建dataset，其实是把pandas数据转换成numpy数组进行转换的\n",
    "dataset = tf.data.Dataset.from_tensor_slices((df.values, target.values))\n",
    "# Shuffle and batch the dataset.\n",
    "train_dataset = dataset.shuffle(len(df)).batch(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 44.    1.    2.  120.  220.    0.    0.  170.    0.    0.    1.    0.\n",
      "    3. ]\n",
      " [ 60.    1.    4.  145.  282.    0.    2.  142.    1.    2.8   2.    2.\n",
      "    4. ]\n",
      " [ 51.    1.    3.  110.  175.    0.    0.  123.    0.    0.6   1.    0.\n",
      "    3. ]\n",
      " [ 59.    1.    0.  164.  176.    1.    0.   90.    0.    1.    1.    2.\n",
      "    0. ]]\n"
     ]
    }
   ],
   "source": [
    "# 用于一会的测试\n",
    "for x, y in train_dataset.take(1):\n",
    "    input_data = x.numpy()\n",
    "    print(input_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "R3dQ-83Ztsgl"
   },
   "source": [
    "#### 搭建训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 39.1831 - accuracy: 0.6733\n",
      "Epoch 2/10\n",
      "76/76 [==============================] - 0s 2ms/step - loss: 9.4004 - accuracy: 0.5446\n",
      "Epoch 3/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 7.1186 - accuracy: 0.5776\n",
      "Epoch 4/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 5.1598 - accuracy: 0.6040\n",
      "Epoch 5/10\n",
      "76/76 [==============================] - 0s 2ms/step - loss: 3.4348 - accuracy: 0.5941\n",
      "Epoch 6/10\n",
      "76/76 [==============================] - 0s 865us/step - loss: 2.1056 - accuracy: 0.6469\n",
      "Epoch 7/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 1.3750 - accuracy: 0.6865\n",
      "Epoch 8/10\n",
      "76/76 [==============================] - 0s 937us/step - loss: 0.8724 - accuracy: 0.7261\n",
      "Epoch 9/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 0.6434 - accuracy: 0.7525\n",
      "Epoch 10/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 0.6946 - accuracy: 0.8086\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fedd49b96d0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = tf.keras.Sequential([\n",
    "    tf.keras.layers.Dense(10, input_shape=(df.shape[1],)),\n",
    "    tf.keras.layers.Dense(10, activation='relu'),\n",
    "    tf.keras.layers.Dense(1, activation=\"sigmoid\")\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "            loss=tf.keras.losses.BinaryCrossentropy(),\n",
    "            metrics=['accuracy'])\n",
    "\n",
    "model.fit(train_dataset, epochs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 2. 以tf-serving格式保存模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 44. ,   1. ,   2. , 120. , 220. ,   0. ,   0. , 170. ,   0. ,\n",
       "          0. ,   1. ,   0. ,   3. ],\n",
       "       [ 60. ,   1. ,   4. , 145. , 282. ,   0. ,   2. , 142. ,   1. ,\n",
       "          2.8,   2. ,   2. ,   4. ],\n",
       "       [ 51. ,   1. ,   3. , 110. , 175. ,   0. ,   0. , 123. ,   0. ,\n",
       "          0.6,   1. ,   0. ,   3. ],\n",
       "       [ 59. ,   1. ,   0. , 164. , 176. ,   1. ,   0. ,  90. ,   0. ,\n",
       "          1. ,   1. ,   2. ,   0. ]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'[[44.0,1.0,2.0,120.0,220.0,0.0,0.0,170.0,0.0,0.0,1.0,0.0,3.0],[60.0,1.0,4.0,145.0,282.0,0.0,2.0,142.0,1.0,2.8,2.0,2.0,4.0],[51.0,1.0,3.0,110.0,175.0,0.0,0.0,123.0,0.0,0.6,1.0,0.0,3.0],[59.0,1.0,0.0,164.0,176.0,1.0,0.0,90.0,0.0,1.0,1.0,2.0,0.0]]'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.DataFrame(input_data).to_json(orient='values')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /home/pss/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "INFO:tensorflow:Assets written to: ./tf_serving_model/heart_model/1/assets\n"
     ]
    }
   ],
   "source": [
    "tf.keras.models.save_model(\n",
    "    model,\n",
    "    \"./tf_serving_model/heart_model/1\",\n",
    "    overwrite=True,\n",
    "    include_optimizer=True,\n",
    "    save_format=None,\n",
    "    signatures=None,\n",
    "    options=None\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 以下操作均为shell命令"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 下载docker镜像的命令：\n",
    "```\n",
    "docker pull tensorflow/serving\n",
    "docker images\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 启动服务的shell命令\n",
    "```\n",
    "model_dir=\"/home/pss/workbench/ant-learn-tensorflow/tf_serving_model/heart_model\"\n",
    "sudo docker run -p 8501:8501 \\\n",
    "  --mount type=bind,source=${model_dir},target=/models/heart_model \\\n",
    "  -e MODEL_NAME=heart_model -t tensorflow/serving\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 访问接口的命令：\n",
    "```\n",
    "curl -d '{\"instances\": [[62.0,0.0,3.0,130.0,263.0,0.0,0.0,97.0,0.0,1.2,2.0,1.0,4.0],[43.0,1.0,4.0,150.0,247.0,0.0,0.0,171.0,0.0,1.5,1.0,0.0,3.0],[76.0,0.0,3.0,140.0,197.0,0.0,1.0,116.0,0.0,1.1,2.0,0.0,3.0]]}' -X POST http://192.168.0.119:8501/v1/models/heart_model:predict\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "pandas_dataframe.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}