DevFestIkorodu19_TensorFlow_Lite.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "DevFestIkorodu19 TensorFlow Lite.ipynb",
      "provenance": [],
      "private_outputs": true,
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Za8-Nr5k11fh"
      },
      "source": [
        "# TensorFlow Lite Demo with Fashio MNIST"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_O2_dLCBVbJS",
        "colab_type": "text"
      },
      "source": [
        "So as mentioned earlier, TensorFlow Lite is a set of tools to help developers run TensorFlow models on mobile, embedded, and IoT devices. \n",
        "\n",
        "\n",
        "<img src = \"tensoflow_lite_logo.png\" width=600px >"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V2lIXiMFU6mV",
        "colab_type": "text"
      },
      "source": [
        "### In this demo we would\n",
        "- Download datasets\n",
        "- Preprocess them\n",
        "- Build our model\n",
        "- Train the model\n",
        "- Test the  model\n",
        "- Convert model to a tflite file\n",
        "- Test the tflite file with an interpreter"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qP1Vz6xKU6mX",
        "colab_type": "text"
      },
      "source": [
        "Ladies and gentlemen shall we begin\n",
        "(https://www.google.com/url?sa=i&source=images&cd=&ved=2ahUKEwjYvIKM8-7lAhVFqxoKHf4nDFQQjRx6BAgBEAQ&url=https%3A%2F%2Ftenor.com%2Fview%2Fthe-originals-the-cw-shall-we-joseph-morgan-elijah-mikaelson-gif-9488740&psig=AOvVaw0ijz30tqq0wCb3TXuTjTax&ust=1573999468310156)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "06ndLauQxiQm"
      },
      "source": [
        "## The Fashion MNIST Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Ka96-ajYzxVU"
      },
      "source": [
        "\n",
        "\n",
        "The [Fashion MNIST](https://github.com/zalandoresearch/fashion-mnist) dataset contains 70,000 grayscale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels), as seen here:\n",
        "\n",
        "<table>\n",
        "  <tr><td>\n",
        "    <img src=\"https://tensorflow.org/images/fashion-mnist-sprite.png\"\n",
        "         alt=\"Fashion MNIST sprite\"  width=\"600\">\n",
        "  </td></tr>\n",
        "  <tr><td align=\"center\">\n",
        "    <b>Figure 1.</b> <a href=\"https://github.com/zalandoresearch/fashion-mnist\">Fashion-MNIST samples</a> (by Zalando, MIT License).<br/>&nbsp;\n",
        "  </td></tr>\n",
        "</table>\n",
        "\n",
        "Fashion MNIST is intended as a drop-in replacement for the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset—often used as the \"Hello, World\" of machine learning programs for computer vision. The MNIST dataset contains images of handwritten digits (0, 1, 2, etc.) in a format identical to that of the articles of clothing we'll use here.\n",
        "\n",
        "This uses Fashion MNIST for variety, and because it's a slightly more challenging problem than regular MNIST. Both datasets are relatively small and are used to verify that an algorithm works as expected. They're good starting points to test and debug code.\n",
        "\n",
        "We will use 60,000 images to train the network and 10,000 images to evaluate how accurately the network learned to classify images. You can access the Fashion MNIST directly from TensorFlow. Import and load the Fashion MNIST data directly from TensorFlow:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "rjOAfhgd__Sp"
      },
      "source": [
        "### As usual, we call in the boss"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Bp_nvHnh_tDo",
        "colab": {}
      },
      "source": [
        "try:\n",
        "    %tensorflow_version 2.x\n",
        "except:\n",
        "    pass"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "pfyZKowNAQ4j",
        "colab": {}
      },
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "# TensorFlow and tf.keras\n",
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "\n",
        "# Helper libraries\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import pathlib\n",
        "\n",
        "\n",
        "print(tf.__version__)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "tadPBTEiAprt"
      },
      "source": [
        "# Download Fashion MNIST Dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "jmSkLCyRKqKB",
        "colab": {}
      },
      "source": [
        "import tensorflow_datasets as tfds\n",
        "tfds.disable_progress_bar()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "XcNwi6nFKneZ",
        "colab": {}
      },
      "source": [
        "splits = tfds.Split.ALL.subsplit(weighted=(80, 10, 10))\n",
        "\n",
        "splits, info = tfds.load('fashion_mnist', with_info=True, as_supervised=True, split=splits)\n",
        "\n",
        "(train_examples, validation_examples, test_examples) = splits\n",
        "\n",
        "num_examples = info.splits['train'].num_examples\n",
        "num_classes = info.features['label'].num_classes"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "-eAv71FRm4JE",
        "colab": {}
      },
      "source": [
        "class_names = ['T-shirt_top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
        "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "hXe6jNokqX3_",
        "colab": {}
      },
      "source": [
        "with open('labels.txt', 'w') as f:\n",
        "  f.write('\\n'.join(class_names))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "iubWCThbdN8K",
        "colab": {}
      },
      "source": [
        "IMG_SIZE = 28"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ZAkuq0V0Aw2X"
      },
      "source": [
        "# Preprocessing data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "_5SIivkunKCC"
      },
      "source": [
        "## Preprocess"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "BwyhsyGydHDl",
        "colab": {}
      },
      "source": [
        "def format_example(image, label):\n",
        "    image = tf.cast(image, tf.float32)\n",
        "    image = tf.image.resize(image, (IMG_SIZE, IMG_SIZE))\n",
        "    image = image / 255.0\n",
        "    return image, label"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "HAlBlXOUMwqe",
        "colab": {}
      },
      "source": [
        "BATCH_SIZE = 32"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "JM4HfIJtnNEk"
      },
      "source": [
        "## Create a Dataset from images and labels"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "uxe2I3oxLDhq",
        "colab": {}
      },
      "source": [
        "train_batches = train_examples.cache().shuffle(num_examples//4).batch(BATCH_SIZE).map(format_example).prefetch(1)\n",
        "validation_batches = validation_examples.cache().batch(BATCH_SIZE).map(format_example).prefetch(1)\n",
        "test_batches = test_examples.cache().batch(1).map(format_example)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "M-topQaOm_LM"
      },
      "source": [
        "# Building the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "kDqcwksFB1bh",
        "colab": {}
      },
      "source": [
        "model = tf.keras.Sequential([\n",
        "  tf.keras.layers.Conv2D(16, 3, activation='relu', input_shape=(IMG_SIZE, IMG_SIZE, 1)),\n",
        "  tf.keras.layers.MaxPooling2D(),\n",
        "  tf.keras.layers.Conv2D(32, 3, activation='relu'),\n",
        "  tf.keras.layers.Flatten(),\n",
        "  tf.keras.layers.Dense(64, activation='relu'),\n",
        "  tf.keras.layers.Dense(10, activation='softmax')\n",
        "])\n",
        "\n",
        "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "zEMOz-LDnxgD"
      },
      "source": [
        "## Train"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "1fk2faPsjqfU",
        "colab": {}
      },
      "source": [
        "validation_batches"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "DGJe_CNvjnhT",
        "colab": {}
      },
      "source": [
        "train_batches, "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "JGlNoRtzCP4_",
        "colab": {}
      },
      "source": [
        "model.fit(train_batches, \n",
        "          epochs=10,\n",
        "          validation_data=validation_batches)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "TZT9-7w9n4YO"
      },
      "source": [
        "# Exporting to TFLite"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "9dq78KBkCV2_",
        "colab": {}
      },
      "source": [
        "export_dir = 'saved_model/1'\n",
        "tf.saved_model.save(model, export_dir)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "EDGiYrBdE6fl",
        "colab": {}
      },
      "source": [
        "#@title Select mode of optimization\n",
        "mode = \"Speed\" #@param [\"Default\", \"Storage\", \"Speed\"]\n",
        "\n",
        "if mode == 'Storage':\n",
        "    optimization = tf.lite.Optimize.OPTIMIZE_FOR_SIZE\n",
        "elif mode == 'Speed':\n",
        "    optimization = tf.lite.Optimize.OPTIMIZE_FOR_LATENCY\n",
        "else:\n",
        "    optimization = tf.lite.Optimize.DEFAULT"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "RbcS9C00CzGe",
        "colab": {}
      },
      "source": [
        "# Convert the model.\n",
        "converter = tf.lite.TFLiteConverter.from_saved_model(export_dir)\n",
        "converter.optimizations = [optimization]\n",
        "tflite_model = converter.convert()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "q5PWCDsTC3El",
        "colab": {}
      },
      "source": [
        "tflite_model_file = 'model.tflite'\n",
        "\n",
        "with open(tflite_model_file, \"wb\") as f:\n",
        "    f.write(tflite_model)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "SR6wFcQ1Fglm"
      },
      "source": [
        "# Test the model with TFLite interpreter "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "rKcToCBEC-Bu",
        "colab": {}
      },
      "source": [
        "# Load TFLite model and allocate tensors.\n",
        "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n",
        "interpreter.allocate_tensors()\n",
        "\n",
        "input_index = interpreter.get_input_details()[0][\"index\"]\n",
        "output_index = interpreter.get_output_details()[0][\"index\"]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "E8EpFpIBFkq8",
        "colab": {}
      },
      "source": [
        "# Gather results for the randomly sampled test images\n",
        "predictions = []\n",
        "test_labels = []\n",
        "test_images = []\n",
        "\n",
        "for img, label in test_batches.take(50):\n",
        "    interpreter.set_tensor(input_index, img)\n",
        "    interpreter.invoke()\n",
        "    predictions.append(interpreter.get_tensor(output_index))\n",
        "    test_labels.append(label[0])\n",
        "    test_images.append(np.array(img))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "both",
        "colab_type": "code",
        "id": "kSjTmi05Tyod",
        "colab": {}
      },
      "source": [
        "#@title Utility functions for plotting\n",
        "# Utilities for plotting\n",
        "\n",
        "def plot_image(i, predictions_array, true_label, img):\n",
        "    predictions_array, true_label, img = predictions_array[i], true_label[i], img[i]\n",
        "    plt.grid(False)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "  \n",
        "    img = np.squeeze(img)\n",
        "\n",
        "    plt.imshow(img, cmap=plt.cm.binary)\n",
        "\n",
        "    predicted_label = np.argmax(predictions_array)\n",
        "    if predicted_label == true_label.numpy():\n",
        "      color = 'green'\n",
        "    else:\n",
        "      color = 'red'\n",
        "    \n",
        "    plt.xlabel(\"{} {:2.0f}% ({})\".format(class_names[predicted_label],\n",
        "                                100*np.max(predictions_array),\n",
        "                                class_names[true_label]),\n",
        "                                color=color)\n",
        "\n",
        "def plot_value_array(i, predictions_array, true_label):\n",
        "    predictions_array, true_label = predictions_array[i], true_label[i]\n",
        "    plt.grid(False)\n",
        "    plt.xticks(list(range(10)), class_names, rotation='vertical')\n",
        "    plt.yticks([])\n",
        "    thisplot = plt.bar(range(10), predictions_array[0], color=\"#777777\")\n",
        "    plt.ylim([0, 1])\n",
        "    predicted_label = np.argmax(predictions_array[0])\n",
        "\n",
        "    thisplot[predicted_label].set_color('red')\n",
        "    thisplot[true_label].set_color('green')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "ZZwg0wFaVXhZ",
        "colab": {}
      },
      "source": [
        "#@title Visualize the outputs { run: \"auto\" }\n",
        "index = 12 #@param {type:\"slider\", min:1, max:50, step:1}\n",
        "plt.figure(figsize=(6,3))\n",
        "plt.subplot(1,2,1)\n",
        "plot_image(index, predictions, test_labels, test_images)\n",
        "plt.show()\n",
        "plot_value_array(index, predictions, test_labels)\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "076bo3FMpRDb"
      },
      "source": [
        "# Download TFLite model and assets\n",
        "\n",
        "**NOTE: You might have to run to the cell below twice**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "XsPXqPlgZPjE",
        "colab": {}
      },
      "source": [
        "try:\n",
        "    from google.colab import files\n",
        "\n",
        "    files.download(tflite_model_file)\n",
        "    files.download('labels.txt')\n",
        "except:\n",
        "    pass"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "H8t7_jRiz9Vw"
      },
      "source": [
        "# Prepare the test images for download (Optional)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Fi09nIps0gBu",
        "colab": {}
      },
      "source": [
        "!mkdir -p test_images"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "sF7EZ63J0hZs",
        "colab": {}
      },
      "source": [
        "from PIL import Image\n",
        "\n",
        "for index, (image, label) in enumerate(test_batches.take(50)):\n",
        "  image = tf.cast(image * 255.0, tf.uint8)\n",
        "  image = tf.squeeze(image).numpy()\n",
        "  pil_image = Image.fromarray(image)\n",
        "  pil_image.save('test_images/{}_{}.jpg'.format(class_names[label[0]].lower(), index))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "uM35O-uv0iWS",
        "colab": {}
      },
      "source": [
        "!ls test_images"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "aR20r4qW0jVm",
        "colab": {}
      },
      "source": [
        "!zip -qq fmnist_test_images.zip -r test_images/"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "tjk4537X0kWN",
        "colab": {}
      },
      "source": [
        "try:\n",
        "  files.download('fmnist_test_images.zip')\n",
        "except:\n",
        "  pass"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "aK9Y9v6xi9IL",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}