l07c01_saving_and_loading_models.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "l07c01_saving_and_loading_models.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W_tvPdyfA-BL"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "0O_LFhwSBCjm"
      },
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9-3Pry4jh1-E"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/examples/blob/master/courses/udacity_intro_to_tensorflow_for_deep_learning/l07c01_saving_and_loading_models.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/examples/blob/master/courses/udacity_intro_to_tensorflow_for_deep_learning/l07c01_saving_and_loading_models.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NxjpzKTvg_dd"
      },
      "source": [
        "# Saving and Loading Models\n",
        "\n",
        "In this tutorial we will learn how we can take a trained model, save it, and then load it back to keep training it or use it to perform inference. In particular, we will use transfer learning to train a classifier to classify images of cats and dogs, just like we did in the previous lesson. We will then take our trained model and save it as an HDF5 file, which is the format used by Keras. We will then load this model, use it to perform predictions, and then continue to train the model. Finally, we will save our trained model as a TensorFlow SavedModel and then we will download it to a local disk, so that it can later be used for deployment in different platforms."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "crU-iluJIEzw"
      },
      "source": [
        "## Concepts that will be covered in this Colab\n",
        "\n",
        "1. Saving models in HDF5 format for Keras\n",
        "2. Saving models in the TensorFlow SavedModel format\n",
        "3. Loading models\n",
        "4. Download models to Local Disk\n",
        "\n",
        "Before starting this Colab, you should reset the Colab environment by selecting `Runtime -> Reset all runtimes...` from menu above."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7RVsYZLEpEWs"
      },
      "source": [
        "# Imports\n",
        "\n",
        "In this Colab we will use the TensorFlow 2.0 Beta version. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e3BXzUGabcI9",
        "outputId": "80a3381d-27d8-4339-e597-7ebb3c3f7d3d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "!pip install -U tensorflow_hub\n",
        "!pip install -U tensorflow_datasets"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: tensorflow_hub in /usr/local/lib/python3.7/dist-packages (0.12.0)\n",
            "Requirement already satisfied: numpy>=1.12.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow_hub) (1.19.5)\n",
            "Requirement already satisfied: protobuf>=3.8.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow_hub) (3.17.3)\n",
            "Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.7/dist-packages (from protobuf>=3.8.0->tensorflow_hub) (1.15.0)\n",
            "Requirement already satisfied: tensorflow_datasets in /usr/local/lib/python3.7/dist-packages (4.0.1)\n",
            "Collecting tensorflow_datasets\n",
            "  Downloading tensorflow_datasets-4.4.0-py3-none-any.whl (4.0 MB)\n",
            "\u001b[K     |████████████████████████████████| 4.0 MB 5.3 MB/s \n",
            "\u001b[?25hRequirement already satisfied: protobuf>=3.12.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (3.17.3)\n",
            "Requirement already satisfied: absl-py in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (0.12.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (3.10.0.2)\n",
            "Requirement already satisfied: dill in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (0.3.4)\n",
            "Requirement already satisfied: promise in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (2.3)\n",
            "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (2.23.0)\n",
            "Requirement already satisfied: tensorflow-metadata in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.4.0)\n",
            "Requirement already satisfied: termcolor in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.1.0)\n",
            "Requirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (5.4.0)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (0.16.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (4.62.3)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.15.0)\n",
            "Requirement already satisfied: attrs>=18.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (21.2.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.19.5)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19.0->tensorflow_datasets) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19.0->tensorflow_datasets) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19.0->tensorflow_datasets) (2021.10.8)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19.0->tensorflow_datasets) (3.0.4)\n",
            "Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources->tensorflow_datasets) (3.6.0)\n",
            "Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-metadata->tensorflow_datasets) (1.53.0)\n",
            "Installing collected packages: tensorflow-datasets\n",
            "  Attempting uninstall: tensorflow-datasets\n",
            "    Found existing installation: tensorflow-datasets 4.0.1\n",
            "    Uninstalling tensorflow-datasets-4.0.1:\n",
            "      Successfully uninstalled tensorflow-datasets-4.0.1\n",
            "Successfully installed tensorflow-datasets-4.4.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a28UtPNlizGI"
      },
      "source": [
        "Some normal imports we've seen before. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OGNpmn43C0O6"
      },
      "source": [
        "import time\n",
        "import numpy as np\n",
        "import matplotlib.pylab as plt\n",
        "\n",
        "import tensorflow as tf\n",
        "import tensorflow_hub as hub\n",
        "import tensorflow_datasets as tfds\n",
        "tfds.disable_progress_bar()\n",
        "\n",
        "from tensorflow.keras import layers"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "amfzqn1Oo7Om"
      },
      "source": [
        "# Part 1: Load the Cats vs. Dogs Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z93vvAdGxDMD"
      },
      "source": [
        "We will use TensorFlow Datasets to load the Dogs vs Cats dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DrIUV3V0xDL_",
        "outputId": "2fa6a737-ba9d-4006-b9f8-dc1765a46177",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "(train_examples, validation_examples), info = tfds.load(\n",
        "    'cats_vs_dogs',\n",
        "    split=['train[:80%]', 'train[80%:]'],\n",
        "    with_info=True,\n",
        "    as_supervised=True,\n",
        ")"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[1mDownloading and preparing dataset 786.68 MiB (download: 786.68 MiB, generated: Unknown size, total: 786.68 MiB) to /root/tensorflow_datasets/cats_vs_dogs/4.0.0...\u001b[0m\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:absl:1738 images were corrupted and were skipped\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[1mDataset cats_vs_dogs downloaded and prepared to /root/tensorflow_datasets/cats_vs_dogs/4.0.0. Subsequent calls will reuse this data.\u001b[0m\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mbgpD3E6gM2P"
      },
      "source": [
        "The images in the Dogs vs. Cats dataset are not all the same size. So, we need to reformat all images to the resolution expected by MobileNet (224, 224)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "we_ftzQxNf7e"
      },
      "source": [
        "def format_image(image, label):\n",
        "  # `hub` image modules exepct their data normalized to the [0,1] range.\n",
        "  image = tf.image.resize(image, (IMAGE_RES, IMAGE_RES))/255.0\n",
        "  return  image, label\n",
        "\n",
        "num_examples = info.splits['train'].num_examples\n",
        "\n",
        "BATCH_SIZE = 32\n",
        "IMAGE_RES = 224\n",
        "\n",
        "train_batches      = train_examples.cache().shuffle(num_examples//4).map(format_image).batch(BATCH_SIZE).prefetch(1)\n",
        "validation_batches = validation_examples.cache().map(format_image).batch(BATCH_SIZE).prefetch(1)"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JzV457OXreQP"
      },
      "source": [
        "# Part 2: Transfer Learning with TensorFlow Hub\n",
        "\n",
        "We will now use TensorFlow Hub to do Transfer Learning."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5wB030nezBwI"
      },
      "source": [
        "URL = \"https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4\"\n",
        "feature_extractor = hub.KerasLayer(URL,\n",
        "                                   input_shape=(IMAGE_RES, IMAGE_RES,3))"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CtFmF7A5E4tk"
      },
      "source": [
        "Freeze the variables in the feature extractor layer, so that the training only modifies the final classifier layer."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Jg5ar6rcE4H-"
      },
      "source": [
        "feature_extractor.trainable = False"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RPVeouTksO9q"
      },
      "source": [
        "## Attach a classification head\n",
        "\n",
        "Now wrap the hub layer in a `tf.keras.Sequential` model, and add a new classification layer."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mGcY27fY1q3Q",
        "outputId": "1fa6604b-43ca-4268-b421-acd687ec1625",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "model = tf.keras.Sequential([\n",
        "  feature_extractor,\n",
        "  layers.Dense(2)\n",
        "])\n",
        "\n",
        "model.summary()"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " keras_layer (KerasLayer)    (None, 1280)              2257984   \n",
            "                                                                 \n",
            " dense (Dense)               (None, 2)                 2562      \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 2,260,546\n",
            "Trainable params: 2,562\n",
            "Non-trainable params: 2,257,984\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OHbXQqIquFxQ"
      },
      "source": [
        "## Train the model\n",
        "\n",
        "We now train this model like any other, by first calling `compile` followed by `fit`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3n0Wb9ylKd8R",
        "outputId": "bca1e6ac-0c5b-4957-88cc-725f172d2ff1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "model.compile(\n",
        "  optimizer='adam', \n",
        "  loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
        "  metrics=['accuracy'])\n",
        "\n",
        "EPOCHS = 1\n",
        "history = model.fit(train_batches,\n",
        "                    epochs=EPOCHS,\n",
        "                    validation_data=validation_batches)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "582/582 [==============================] - ETA: 0s - loss: 0.0540 - accuracy: 0.9819"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kb__ZN8uFn-D"
      },
      "source": [
        "## Check the predictions\n",
        "\n",
        "Get the ordered list of class names."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W_Zvg2i0fzJu",
        "outputId": "0f2b94ef-2d7a-4d8c-9aab-05104f94a9e5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "class_names = np.array(info.features['label'].names)\n",
        "class_names"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['cat', 'dog'], dtype='<U3')"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Olg6MsNGJTL"
      },
      "source": [
        "Run an image batch through the model and convert the indices to class names."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e4ONBQvy6dfF"
      },
      "source": [
        "image_batch, label_batch = next(iter(train_batches.take(1)))\n",
        "image_batch = image_batch.numpy()\n",
        "label_batch = label_batch.numpy()\n",
        "\n",
        "predicted_batch = model.predict(image_batch)\n",
        "predicted_batch = tf.squeeze(predicted_batch).numpy()\n",
        "predicted_ids = np.argmax(predicted_batch, axis=-1)\n",
        "predicted_class_names = class_names[predicted_ids]\n",
        "predicted_class_names"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CkGbZxl9GZs-"
      },
      "source": [
        "Let's look at the true labels and predicted ones."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nL9IhOmGI5dJ"
      },
      "source": [
        "print(\"Labels: \", label_batch)\n",
        "print(\"Predicted labels: \", predicted_ids)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wC_AYRJU9NQe"
      },
      "source": [
        "plt.figure(figsize=(10,9))\n",
        "for n in range(30):\n",
        "  plt.subplot(6,5,n+1)\n",
        "  plt.imshow(image_batch[n])\n",
        "  color = \"blue\" if predicted_ids[n] == label_batch[n] else \"red\"\n",
        "  plt.title(predicted_class_names[n].title(), color=color)\n",
        "  plt.axis('off')\n",
        "_ = plt.suptitle(\"Model predictions (blue: correct, red: incorrect)\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mmPQYQLx3cYq"
      },
      "source": [
        "# Part 3: Save as Keras `.h5` model\n",
        "\n",
        "Now that we've trained the model,  we can save it as an HDF5 file, which is the format used by Keras. Our HDF5 file will have the extension '.h5', and it's name will correpond to the current time stamp."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tCnNWTkZ3Ckz"
      },
      "source": [
        "t = time.time()\n",
        "\n",
        "export_path_keras = \"./{}.h5\".format(int(t))\n",
        "print(export_path_keras)\n",
        "\n",
        "model.save(export_path_keras)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9tdJWVHmnKxJ"
      },
      "source": [
        "!ls"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JgqmH1WVUKli"
      },
      "source": [
        " You can later recreate the same model from this file, even if you no longer have access to the code that created the model.\n",
        "\n",
        "This file includes:\n",
        "\n",
        "- The model's architecture\n",
        "- The model's weight values (which were learned during training)\n",
        "- The model's training config (what you passed to `compile`), if any\n",
        "- The optimizer and its state, if any (this enables you to restart training where you left off)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yXole25Z799G"
      },
      "source": [
        "# Part 4:  Load the Keras `.h5` Model\n",
        "\n",
        "We will now load the model we just saved into a new model called `reloaded`. We will need to provide the file path and the `custom_objects` parameter. This parameter tells keras how to load the `hub.KerasLayer` from the `feature_extractor` we used for transfer learning."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rx-z3Qwx5RnB"
      },
      "source": [
        "reloaded = tf.keras.models.load_model(\n",
        "  export_path_keras, \n",
        "  # `custom_objects` tells keras how to load a `hub.KerasLayer`\n",
        "  custom_objects={'KerasLayer': hub.KerasLayer})\n",
        "\n",
        "reloaded.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XxDl2vPkf2ST"
      },
      "source": [
        "We can check that the reloaded model and the previous model give the same result"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MFljA-Hd85Tu"
      },
      "source": [
        "result_batch = model.predict(image_batch)\n",
        "reloaded_result_batch = reloaded.predict(image_batch)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YeCZUUJK9Svv"
      },
      "source": [
        "The difference in output should be zero:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S3p5-uD39PC1"
      },
      "source": [
        "(abs(result_batch - reloaded_result_batch)).max()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KqU79kKFo7S2"
      },
      "source": [
        "As we can see, the reult is 0.0, which indicates that both models made the same predictions on the same batch of images."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nKunO4soA4Dm"
      },
      "source": [
        "# Keep Training\n",
        "\n",
        "Besides making predictions, we can also take our `reloaded` model and keep training it. To do this, you can just train the `reloaded` as usual, using the `.fit` method."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NEv-fnHEAplx"
      },
      "source": [
        "EPOCHS = 3\n",
        "history = reloaded.fit(train_batches,\n",
        "                    epochs=EPOCHS,\n",
        "                    validation_data=validation_batches)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5OKoZaAHFH_s"
      },
      "source": [
        "# Part 5: Export as SavedModel\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3V47IwQbFTYT"
      },
      "source": [
        "You can also export a whole model to the TensorFlow SavedModel format. SavedModel is a standalone serialization format for Tensorflow objects, supported by TensorFlow serving as well as TensorFlow implementations other than Python. A SavedModel contains a complete TensorFlow program, including weights and computation. It does not require the original model building code to run, which makes it useful for sharing or deploying (with TFLite, TensorFlow.js, TensorFlow Serving, or TFHub).\n",
        "\n",
        "The SavedModel files that were created contain:\n",
        "\n",
        "* A TensorFlow checkpoint containing the model weights.\n",
        "* A SavedModel proto containing the underlying Tensorflow graph. Separate graphs are saved for prediction (serving), train, and evaluation. If the model wasn't compiled before, then only the inference graph gets exported.\n",
        "* The model's architecture config, if available.\n",
        "\n",
        "\n",
        "Let's save our original `model` as a TensorFlow SavedModel. To do this we will use the `tf.saved_model.save()` function. This functions takes in the model we want to save and the path to the folder where we want to save our model. \n",
        "\n",
        "This function will create a folder where you will find an `assets` folder, a `variables` folder, and the `saved_model.pb` file. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LtpeKMfoGXrj"
      },
      "source": [
        "t = time.time()\n",
        "\n",
        "export_path_sm = \"./{}\".format(int(t))\n",
        "print(export_path_sm)\n",
        "\n",
        "tf.saved_model.save(model, export_path_sm)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5h6B1wITlu-9"
      },
      "source": [
        "!ls {export_path_sm}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ktpsqHxJPIQW"
      },
      "source": [
        "# Part 6: Load SavedModel"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v0FDcCn9ncDb"
      },
      "source": [
        "Now, let's load our SavedModel and use it to make predictions. We use the `tf.saved_model.load()` function to load our SavedModels. The object returned by `tf.saved_model.load` is 100% independent of the code that created it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c7_PM7lofG2V"
      },
      "source": [
        "reloaded_sm = tf.saved_model.load(export_path_sm)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "esEODdtM0kHB"
      },
      "source": [
        "Now, let's use the `reloaded_sm` (reloaded SavedModel) to make predictions on a batch of images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lpQldy_bm3Ty"
      },
      "source": [
        "reload_sm_result_batch = reloaded_sm(image_batch, training=False).numpy()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "65upCyVN0u3Q"
      },
      "source": [
        "We can check that the reloaded SavedModel and the previous model give the same result."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hoCwmkGznR_0"
      },
      "source": [
        "(abs(result_batch - reload_sm_result_batch)).max()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wxuETjjs01pL"
      },
      "source": [
        "As we can see, the result is 0.0, which indicates that both models made the same predictions on the same batch of images."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7Nom-ka0yuqB"
      },
      "source": [
        "# Part 7: Loading the SavedModel as a Keras Model\n",
        "\n",
        "The object returned by `tf.saved_model.load` is not a Keras object (i.e. doesn't have `.fit`, `.predict`, `.summary`, etc. methods). Therefore, you can't simply take your `reloaded_sm` model and keep training it by running `.fit`. To be able to get back a full keras model from the Tensorflow SavedModel format we must use the `tf.keras.models.load_model` function. This function will work the same as before, except now we pass the path to the folder containing our SavedModel."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vi1jaqh8yvt6"
      },
      "source": [
        "t = time.time()\n",
        "\n",
        "export_path_sm = \"./{}\".format(int(t))\n",
        "print(export_path_sm)\n",
        "tf.saved_model.save(model, export_path_sm)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1VPE2_QQGmAP"
      },
      "source": [
        "reload_sm_keras = tf.keras.models.load_model(\n",
        "  export_path_sm,\n",
        "  custom_objects={'KerasLayer': hub.KerasLayer})\n",
        "\n",
        "reload_sm_keras.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "thTbiHE72GL4"
      },
      "source": [
        "Now, let's use the `reloaded_sm)keras` (reloaded Keras model from our SavedModel) to make predictions on a batch of images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-0oCJrNLKdKj"
      },
      "source": [
        "result_batch = model.predict(image_batch)\n",
        "reload_sm_keras_result_batch = reload_sm_keras.predict(image_batch)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jUQaxzFj2Q8k"
      },
      "source": [
        "We can check that the reloaded Keras model and the previous model give the same result."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DJCD9JJxKg9F"
      },
      "source": [
        "(abs(result_batch - reload_sm_keras_result_batch)).max()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NFa6jNc4PW9F"
      },
      "source": [
        "# Part 8:  Download your model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2W_y1qMVQeCm"
      },
      "source": [
        "You can download the SavedModel to your local disk by creating a zip file. We wil use the `-r` (recursice) option to zip all subfolders. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WPRFoU1xPCGF"
      },
      "source": [
        "!zip -r model.zip {export_path_sm}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MBZL85RsQBTj"
      },
      "source": [
        "The zip file is saved in the current working directory. You can see what the current working directory is by running:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ALP-DfwSQRL8"
      },
      "source": [
        "!ls"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IR89aU-SRmsL"
      },
      "source": [
        "Once the file is zipped, you can download  it to your local disk. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lOXYrlDkNjKQ"
      },
      "source": [
        "try:\n",
        "  from google.colab import files\n",
        "  files.download('./model.zip')\n",
        "except ImportError:\n",
        "  pass"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RzLutxq_SeLQ"
      },
      "source": [
        "The `files.download` command will  search for files in your current working directory. If the file you want to download is in a directory other than the current working directory, you have to include the path to the directory where the file is located."
      ]
    }
  ]
}