RPS_Quantization_Aware_Training.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "RPS Quantization Aware Training.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wGXdqVmi5fEN"
      },
      "source": [
        "# [Quantization Aware Training](https://www.tensorflow.org/model_optimization/guide/quantization/training)\n",
        "\n",
        "Implementing quantisation aware training technique on RPS Dataset.\n",
        "\n",
        "\n",
        "Author: [Sayan Nath](http://github.com/sayannath)<br>\n",
        "Last Updated: 2021/03/29\n",
        "<br>\n",
        "<br>\n",
        "<a href=\"https://colab.research.google.com/drive/1Wdso2N_76E8Xxniqd4C6T1sV5BuhKN1o?usp=sharing\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X3lj4RLb56TM"
      },
      "source": [
        "## Intital Setup"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JDveHwKM5AbD",
        "outputId": "0dcf7e86-b995-4466-a8f6-c1fc4ac6945b"
      },
      "source": [
        "!nvidia-smi"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mon Mar 29 12:09:25 2021       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 460.56       Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   45C    P8    10W /  70W |      0MiB / 15109MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qcHfZDBQ9008",
        "outputId": "80cbb0f4-7279-467d-9db3-000c8e5a1f1a"
      },
      "source": [
        "!pip uninstall -y tensorflow\n",
        "!pip install -q tf-nightly\n",
        "!pip install -qq tensorflow-model-optimization"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Uninstalling tensorflow-2.4.1:\n",
            "  Successfully uninstalled tensorflow-2.4.1\n",
            "\u001b[K     |████████████████████████████████| 454.3MB 38kB/s \n",
            "\u001b[K     |████████████████████████████████| 1.2MB 48.0MB/s \n",
            "\u001b[K     |████████████████████████████████| 6.0MB 47.4MB/s \n",
            "\u001b[K     |████████████████████████████████| 4.0MB 42.9MB/s \n",
            "\u001b[K     |████████████████████████████████| 471kB 53.8MB/s \n",
            "\u001b[K     |████████████████████████████████| 4.0MB 51.4MB/s \n",
            "\u001b[K     |████████████████████████████████| 3.9MB 54.0MB/s \n",
            "\u001b[31mERROR: fancyimpute 0.4.3 requires tensorflow, which is not installed.\u001b[0m\n",
            "\u001b[K     |████████████████████████████████| 174kB 16.8MB/s \n",
            "\u001b[?25h"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2cQmIwPm5-iq"
      },
      "source": [
        "## Data Gathering"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lWCc30Da5-EZ",
        "outputId": "a66b5813-2d30-4c0e-c8e8-9b6edc7384ae"
      },
      "source": [
        "!wget https://storage.googleapis.com/laurencemoroney-blog.appspot.com/rps.zip \n",
        "!wget https://storage.googleapis.com/laurencemoroney-blog.appspot.com/rps-test-set.zip"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2021-03-29 12:12:14--  https://storage.googleapis.com/laurencemoroney-blog.appspot.com/rps.zip\n",
            "Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.164.144, 142.250.73.240, 172.217.8.16, ...\n",
            "Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.164.144|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 200682221 (191M) [application/zip]\n",
            "Saving to: ‘rps.zip’\n",
            "\n",
            "rps.zip             100%[===================>] 191.38M   145MB/s    in 1.3s    \n",
            "\n",
            "2021-03-29 12:12:15 (145 MB/s) - ‘rps.zip’ saved [200682221/200682221]\n",
            "\n",
            "--2021-03-29 12:12:15--  https://storage.googleapis.com/laurencemoroney-blog.appspot.com/rps-test-set.zip\n",
            "Resolving storage.googleapis.com (storage.googleapis.com)... 172.253.122.128, 172.217.2.112, 142.250.31.128, ...\n",
            "Connecting to storage.googleapis.com (storage.googleapis.com)|172.253.122.128|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 29516758 (28M) [application/zip]\n",
            "Saving to: ‘rps-test-set.zip’\n",
            "\n",
            "rps-test-set.zip    100%[===================>]  28.15M  --.-KB/s    in 0.1s    \n",
            "\n",
            "2021-03-29 12:12:15 (206 MB/s) - ‘rps-test-set.zip’ saved [29516758/29516758]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jR7V2APv7RY-"
      },
      "source": [
        "!unzip -q rps.zip\n",
        "!unzip -qq rps-test-set.zip"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W-56s3T46KrP"
      },
      "source": [
        "## Import necessary modules"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H91gzQc-6QF_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "06d9a300-be15-402c-dbdb-11137e14f006"
      },
      "source": [
        "from imutils import paths\n",
        "from pprint import pprint\n",
        "from collections import Counter\n",
        "import tensorflow as tf\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import *\n",
        "from tensorflow.keras.utils import plot_model\n",
        "import tensorflow_model_optimization as tfmot\n",
        "import re \n",
        "\n",
        "import os\n",
        "from sys import getsizeof\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import matplotlib.image as mpimg\n",
        "import seaborn as sns\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sys import getsizeof\n",
        "\n",
        "import numpy as np\n",
        "np.random.seed(42)\n",
        "\n",
        "\n",
        "print(tf.__version__)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Enabling eager execution\n",
            "INFO:tensorflow:Enabling v2 tensorshape\n",
            "INFO:tensorflow:Enabling resource variables\n",
            "INFO:tensorflow:Enabling tensor equality\n",
            "INFO:tensorflow:Enabling control flow v2\n",
            "2.6.0-dev20210329\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IPLJsYCxPukw"
      },
      "source": [
        "def get_file_size(file_path):\n",
        "    size = os.path.getsize(file_path)\n",
        "    return size\n",
        "\n",
        "def convert_bytes(size, unit=None):\n",
        "    if unit == \"KB\":\n",
        "        return print('File Size: ' + str(round(size/1024, 3)) + 'Kilobytes')\n",
        "    elif unit == 'MB':\n",
        "        return print('File Size: ' + str(round(size/(1024*1024), 3)) + 'Megabytes')\n",
        "    else:\n",
        "        return print('File Size: ' + str(size) + 'bytes')"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z6m7IS8j7p1f"
      },
      "source": [
        "## Determine the dimension of the images"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qs9sW74b7uxj"
      },
      "source": [
        "train_dir='rps/'\n",
        "test_dir='rps-test-set/'"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 529
        },
        "id": "UzTUR5OZ7i8o",
        "outputId": "43da78e5-f0e0-4416-f0d6-9ab538cee66d"
      },
      "source": [
        "dim1 = []\n",
        "dim2 = []\n",
        "for image_filename in os.listdir(train_dir+'paper'):\n",
        "    \n",
        "    img = mpimg.imread(train_dir+'paper'+'/'+image_filename)\n",
        "    d1,d2,colors = img.shape\n",
        "    dim1.append(d1)\n",
        "    dim2.append(d2)\n",
        "\n",
        "sns.jointplot(dim1,dim2)\n",
        "\n",
        "print(\"Height: \",np.mean(dim1))\n",
        "print(\"Width: \",np.mean(dim2))"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
            "  FutureWarning\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Height:  300.0\n",
            "Width:  300.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbUAAAGoCAYAAADB4nuYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAXNUlEQVR4nO3cf6zddX348ecLbksRLM3gotL2u9ZoZxonZVSCskWtwXR1ARMlc4uORDYMMsMPv264r18Ttu8f6vYV/WMJIxRCpps4ZRMJm3ahagyu5QJtoRRZYW794eyFLxVdbKH0tT/Ou3C833N7z23v7Wlf5/lITvicz+d9Puf9trfn2c85xxuZiSRJFZw06AlIkjRTjJokqQyjJkkqw6hJksowapKkMkYGPYFp8quakoZZDHoCxzuv1CRJZRg1SVIZJ9rbj0ds4eL/we6dOwY9DUlD7pxFi9m14z8GPY2y4gT7jSJHPNmI4Lf/6v6ZnIskTdudH34rR/G662dqU/DtR0lSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllRGYOeg59i4h/As4a9Dym4Szg6UFP4hhwnfUMy1pPtHU+nZmrBz2J49kJFbUTTUSMZebKQc9jtrnOeoZlrcOyzmHi24+SpDKMmiSpDKM2u24Z9ASOEddZz7CsdVjWOTT8TE2SVIZXapKkMoyaJKkMoyZJKsOoSZLKMGqSpDJOqKitXr06AW/evHkb1lvfir9eTuqEitrTT59Iv6JNkgZnWF8vT6ioSZJ0OEZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGSODnoAkaeZt2bKFiBj0NGbUOYsWs2vHfxx2jFGTpIJeeOEFfvuv7h/0NGbUnR9+65RjfPtRklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGVNGLSLmRcTGiNgcEVsj4sa2f2lEbIiI7RFxZ0TMbftPafe3t+NLDnPukyPi4Yi4Z6YWJEkaXv1cqe0HVmXmucAKYHVEXAh8BrgpM18HPAtc0cZfATzb9t/Uxk3mGmDbkU5ekqRuU0YtO37W7s5ptwRWAV9t++8A3tO2L233acffGREx8bwRsQh4N3DrEc9ekqQufX2m1t4m3ATsAdYBTwJ7M/NAG7ITWNi2FwI7ANrxnwBn9jjt54E/Ag5O8dxXRsRYRIyNj4/3M11JGkrdr5eDnsug9BW1zHwxM1cAi4ALgDcczZNGxG8BezLzwT6e+5bMXJmZK0dHR4/maSWptO7Xy0HPZVCm9e3HzNwLrAfeAiyIiJF2aBGwq23vAhYDtONnAM9MONVFwCUR8UPgy8CqiPjikSxAkqRD+vn242hELGjbpwIX0/lyx3rgfW3Y5cDX2/bd7T7t+H2Zmd3nzMxPZOaizFwCvL+N+cBRrkWSNOT6uVJ7DbA+IrYADwDrMvMe4I+B6yNiO53PzNa28WuBM9v+64EbACLinIi4d6YXIEnSISNTDcjMLcB5PfY/RefztYn79wGX9di/G1jTY/+3gW/3NVtJkg7D3ygiSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSypgyahExLyI2RsTmiNgaETe2/UsjYkNEbI+IOyNibtt/Sru/vR1f0uOciyNifUQ81s55zUwvTJI0fPq5UtsPrMrMc4EVwOqIuBD4DHBTZr4OeBa4oo2/Ani27b+pjZvoAPCxzFwOXAhcHRHLj24pkqRhN2XUsuNn7e6cdktgFfDVtv8O4D1t+9J2n3b8nRERE875o8x8qG3/FNgGLDyKdUiS1N9nahFxckRsAvYA64Angb2ZeaAN2cnLUVoI7ABox38CnHmYcy8BzgM2TH/6kiS9rK+oZeaLmbkCWARcALxhJp48Ik4HvgZcm5nPTTLmyogYi4ix8fHxmXhaSSqp+/Vy0HMZlGl9+zEz9wLrgbcACyJipB1aBOxq27uAxQDt+BnAMxPPFRFz6ATtS5l512Ge85bMXJmZK0dHR6czXUkaKt2vl4Oey6D08+3H0YhY0LZPBS6m8xnYeuB9bdjlwNfb9t3tPu34fZmZE84ZwFpgW2Z+7mgXIUkS9Hel9hpgfURsAR4A1mXmPcAfA9dHxHY6n5mtbePXAme2/dcDNwBExDkRcW8bcxHwQWBVRGxqtzUztipJ0lAamWpAZm6h80WOifufovP52sT9+4DLeuzfDaxp298DYuIYSZKOhr9RRJJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZRg1SVIZRk2SVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEllGDVJUhlGTZJUhlGTJJVh1CRJZUwZtYiYFxEbI2JzRGyNiBvb/qURsSEitkfEnRExt+0/pd3f3o4vmeS8qyPiB23cDTO5KGkm7f35Pjb+2zN8Y/NuNv7bM+z9+b5BT0nSJPq5UtsPrMrMc4EVwOqIuBD4DHBTZr4OeBa4oo2/Ani27b+pjfsFEXEy8JfAbwLLgd+JiOVHuxhppu39+T6+9eg4v3fbRj76tw/ze7dt5FuPjhs26Tg1ZdSy42ft7px2S2AV8NW2/w7gPW370nafdvydERETTnsBsD0zn8rM54Evt8dJx5Un/vO/+NTdj7LvhYMA7HvhIJ+6+1Ge+M//GvDMJPXS12dqEXFyRGwC9gDrgCeBvZl5oA3ZCSxs2wuBHQDt+E+AMyec8qUxPR4/8bmvjIixiBgbHx/vZ7rSjPnxc/tfCtoh+144yI+f2z+gGUmT6369HPRcBqWvqGXmi5m5AlhE5yrrDbM6q1987lsyc2VmrhwdHT1WTysB8Kr5pzBvzi/+NZk35yReNf+UAc1Imlz36+Wg5zIo0/r2Y2buBdYDbwEWRMRIO7QI2NW2dwGLAdrxM4BnJpzqpTE9Hi8dN5a9+jT+9JI3vhS2eXNO4k8veSPLXn3agGcmqZeRqQZExCjwQmbujYhTgYvpfPljPfA+Op+HXQ58vT3k7nb/++34fZmZE077APD6iFhKJ2bvB3736JcjzawFp87jXW8cZclZF/Dj5/bzqvmnsOzVp7Hg1HmDnpqkHqaMGvAa4I72jcWTgK9k5j0R8Rjw5Yj4P8DDwNo2fi3w1xGxHfh/dIJFRJwD3JqZazLzQET8IfBN4GTgtszcOqMrk2bIglPnccFSIyadCKaMWmZuAc7rsf8pOp+vTdy/D7isx/7dwJqu+/cC905zvpIkTcrfKCJJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKmDJqEbE4ItZHxGMRsTUirmn7z42I70fEIxHxjYiY3/bPjYjb2/7NEfH2Sc67IiL+JSI2RcRYRFwwoyuTJA2dfq7UDgAfy8zlwIXA1RGxHLgVuCEzfxX4e+DjbfwfALT9FwP/NyJ6Pc9ngRszcwXwqXZfkqQjNmXUMvNHmflQ2/4psA1YCCwDvtuGrQPe27aXA/e18XuAvcDKXqcG5rftM4DdR7YESZI6pvWZWkQsAc4DNgBbgUvbocuAxW17M3BJRIxExFLg/K5j3a4F/jwidgB/AXxikue8sr09OTY+Pj6d6UrSUOl+vRz0XAal76hFxOnA14BrM/M54EPARyLiQeCVwPNt6G3ATmAM+DxwP/Bij1NeBVyXmYuB64C1vZ43M2/JzJWZuXJ0dLTf6UrS0Ol+vRz0XAZlpJ9BETGHTtC+lJl3AWTm48C72vFlwLvb/gN0InXosfcDT/Q47eXANW377+h8RidJ0hHr59uPQecqaltmfq5r/9ntvycBnwRubvdfERGnte2LgQOZ+ViPU+8G3ta2VwH/ehTrkCSpryu1i4APAo9ExKa270+A10fE1e3+XcDtbfts4JsRcRDY1R4LQETcCtycmWN0viX5hYgYAfYBVx7tYiRJw23KqGXm94CY5PAXeoz/IfArk5zr9yec9/y+ZilJUh/8jSKSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpjCmjFhGLI2J9RDwWEVsj4pq2/9yI+H5EPBIR34iI+W3/3Ii4ve3fHBFvP8y5PxoRj7fzfnbGViVJGkojfYw5AHwsMx+KiFcCD0bEOuBW4H9m5nci4kPAx4H/DfwBQGb+akScDfxjRLw5Mw92nzQi3gFcCpybmfvbWEmSjtiUV2qZ+aPMfKht/xTYBiwElgHfbcPWAe9t28uB+9r4PcBeYGWPU18FfDoz93eNlSTpiE3rM7WIWAKcB2wAttK50gK4DFjctjcDl0TESEQsBc7vOtZtGfAbEbEhIr4TEW+e5DmvjIixiBgbHx+fznQlaah0v14Oei6D0nfUIuJ04GvAtZn5HPAh4CMR8SDwSuD5NvQ2YCcwBnweuB94sccpR4BfAi6k89blVyIiJg7KzFsyc2VmrhwdHe17YZI0bLpfLwc9l0Hp5zM1ImIOnaB9KTPvAsjMx4F3tePLgHe3/QeA67oeez/wRI/T7gTuyswENkbEQeAswMsxSdIR6efbjwGsBbZl5ue69p/d/nsS8Eng5nb/FRFxWtu+GDiQmY/1OPU/AO9o45YBc4Gnj2o1kqSh1s+V2kXAB4FHImJT2/cnwOsj4up2/y7g9rZ9NvDNduW1qz0WgIi4Fbg5M8fovE15W0Q8Suety8vbVZskSUdkyqhl5veA/++zruYLPcb/EPiVSc71+13bzwMf6GuWkiT1wd8oIkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpDKMmSSrDqEmSyjBqkqQyjJokqYyRQU9AkjTz5syZw50ffuugpzGjzlm0eMoxRk2SCnrTm97E2NjYoKdxzPn2oySpDKMmSSrDqEmSyjBqkqQyjJokqQyjJkkqw6hJksowapKkMoyaJKkMoyZJKsOoSZLKMGqSpDKMmiSpjMjMQc+hbxExDvz7oOcxDWcBTw96EseA66xnWNZ6oq3z6cxc3c/AiPinfsdWckJF7UQTEWOZuXLQ85htrrOeYVnrsKxzmPj2oySpDKMmSSrDqM2uWwY9gWPEddYzLGsdlnUODT9TkySV4ZWaJKkMoyZJKsOo9SEiFkfE+oh4LCK2RsQ1bf+5EfH9iHgkIr4REfPb/rkRcXvbvzki3n6Yc380Ih5v5/3sMVrSpGZrrRGxIiL+JSI2RcRYRFxwDJfVaz7zImJjm/PWiLix7V8aERsiYntE3BkRc9v+U9r97e34kknOuzoiftDG3XDsVtTbbKxzsp+RQZutP9M29uSIeDgi7jk2q9ERy0xvU9yA1wC/1rZfCTwBLAceAN7W9n8I+LO2fTVwe9s+G3gQOKnHed8B/DNwyqGxhdf6LeA32/Ya4NsDXmcAp7ftOcAG4ELgK8D72/6bgava9keAm9v2+4E7e5zzZOBJ4LXAXGAzsLzgOnv+jBwHP7szvtauc18P/A1wz6DX6e3wN6/U+pCZP8rMh9r2T4FtwEJgGfDdNmwd8N62vRy4r43fA+wFev0fPK8CPp2Z+7vGDtQsrjWB+W37DGD3bMy/X9nxs3Z3TrslsAr4att/B/Cetn1pu087/s6IiAmnvQDYnplPZebzwJfb4wZmNtZ5mJ+RgZqlP1MiYhHwbuDWWZq6ZpBRm6b2FsV5dP4VuJWXX7QuAxa37c3AJRExEhFLgfO7jnVbBvxGe+vjOxHx5tmc+3TN8FqvBf48InYAfwF8YvZm3p/2ltImYA+dUD8J7M3MA23ITl5+sV4I7ABox38CnDnhlC+N6fH4gZmFdXafewkv/4wM3Cyt9fPAHwEHZ3HqmiFGbRoi4nTga8C1mfkcnbfhPhIRD9J5G+b5NvQ2On95xuj8hbgfeLHHKUeAX6LzFsnHga/0+pfiIMzCWq8CrsvMxcB1wNrZXcHUMvPFzFwBLKJzlfWGAU9pVszWOnv8jAzcTK81In4L2JOZD87E/DT7RgY9gRNFRMyh8xf4S5l5F0BmPg68qx1fRuctikP/6ruu67H30/ncYaKdwF2ZmcDGiDhI5xesjs/iUqY0S2u9HDj0hYK/4zh6Kycz90bEeuAtwIKIGGnrWgTsasN20bkC3RkRI3TeQn1mwqkOjTmk+/EDN4Pr7PkzcjyZwbVeROediDXAPGB+RHwxMz9wTBaiafNKrQ/t6mktsC0zP9e1/+z235OAT9L5EJqIeEVEnNa2LwYOZOZjPU79D3S+LHIoFHMZ8G8Mn8W17gbe1rZXAf86a4voQ0SMRsSCtn0qcDGdz4bWA+9rwy4Hvt627273acfva/8Y6fYA8Pr2bbu5dL58cPfsrWJqs7HOyX5GBm021pqZn8jMRZm5hM6f530G7Tg36G+qnAg34NfpfOC8BdjUbmvoXHk80W6f5uXf0LIE+AGdv1D/DPxy17luBVa27bnAF4FHgYeAVYXX+ut0vhm5mc7nL+cPeJ1vAh5u63wU+FTb/1pgI7CdzhXloW+mzmv3t7fjr237zwHu7Trvmva/0ZPA/zoO/jxnfJ2T/YxUXOuE878dv/143N/8NVmSpDJ8+1GSVIZRkySVYdQkSWUYNUlSGUZNklSGUZMklWHUJEll/DeGxsDvgxGCHAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x432 with 3 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c5EOzdop8Fzo"
      },
      "source": [
        "`Height` is 300 and `Width` is 300"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3RnaiNNH8CU6",
        "outputId": "64802ab8-1174-4dcc-d04a-b7bfdd9dd3f4"
      },
      "source": [
        "image_paths = list(paths.list_images(\"rps\"))\n",
        "np.random.shuffle(image_paths)\n",
        "image_paths[:5]"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['rps/rock/rock04-115.png',\n",
              " 'rps/scissors/scissors04-068.png',\n",
              " 'rps/scissors/scissors04-047.png',\n",
              " 'rps/rock/rock07-k03-044.png',\n",
              " 'rps/paper/paper07-117.png']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CUSXaM_F8Y-y"
      },
      "source": [
        "## Couting the number of Images per classes"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AOW02S1K8UPj",
        "outputId": "95c68313-3303-47d4-84cc-e3f690983d7b"
      },
      "source": [
        "labels = []\n",
        "for image_path in image_paths:\n",
        "    label = image_path.split(\"/\")[1]\n",
        "    labels.append(label)\n",
        "class_count = Counter(labels) \n",
        "pprint(class_count)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Counter({'rock': 840, 'scissors': 840, 'paper': 840})\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SrrFJ-x58cpq"
      },
      "source": [
        "Wow! Balanced Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qPlOPCsM8obx"
      },
      "source": [
        "## Define the Hyperparameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MDw1ldXT8baT"
      },
      "source": [
        "BATCH_SIZE = 128\n",
        "EPOCHS = 20\n",
        "IMG_SIZE = 150\n",
        "NUM_CLASSES=3"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6GJEbFhGMBHv",
        "outputId": "3d1a9459-37b0-4d66-fd08-aaa5f84c8a90"
      },
      "source": [
        "training_datagen = tf.keras.preprocessing.image.ImageDataGenerator(\n",
        "      rescale = 1./255,\n",
        "\t    rotation_range=40,\n",
        "      width_shift_range=0.2,\n",
        "      height_shift_range=0.2,\n",
        "      shear_range=0.2,\n",
        "      zoom_range=0.2,\n",
        "      horizontal_flip=True,\n",
        "      vertical_flip=True,\n",
        "      fill_mode='nearest')\n",
        "\n",
        "\n",
        "validation_datagen = tf.keras.preprocessing.image.ImageDataGenerator(rescale = 1./255)\n",
        "\n",
        "train_generator = training_datagen.flow_from_directory(\n",
        "\ttrain_dir,\n",
        "\ttarget_size=(IMG_SIZE,IMG_SIZE),\n",
        "\tclass_mode='categorical',\n",
        "    batch_size=BATCH_SIZE\n",
        ")\n",
        "\n",
        "validation_generator = validation_datagen.flow_from_directory(\n",
        "\ttest_dir,\n",
        "\ttarget_size=(IMG_SIZE,IMG_SIZE),\n",
        "\tclass_mode='categorical',\n",
        "    batch_size=BATCH_SIZE\n",
        ")"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 2520 images belonging to 3 classes.\n",
            "Found 372 images belonging to 3 classes.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vAo0InP-Phot",
        "outputId": "4a52201b-415c-4144-bd1f-35b7efb08834"
      },
      "source": [
        "train_generator.class_indices"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'paper': 0, 'rock': 1, 'scissors': 2}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VJsgdjo_CVbi"
      },
      "source": [
        "## Define the Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KbduXXbg_WbL"
      },
      "source": [
        "def training_model():\n",
        "    model = Sequential([\n",
        "        tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
        "        tf.keras.layers.MaxPooling2D(2, 2),\n",
        "        # The second convolution\n",
        "        tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
        "        tf.keras.layers.MaxPooling2D(2,2),\n",
        "        # The third convolution\n",
        "        tf.keras.layers.Conv2D(128, (3,3), activation='relu'),\n",
        "        tf.keras.layers.MaxPooling2D(2,2),\n",
        "        # The fourth convolution\n",
        "        tf.keras.layers.Conv2D(128, (3,3), activation='relu'),\n",
        "        tf.keras.layers.MaxPooling2D(2,2),\n",
        "        # Flatten the results to feed into a DNN\n",
        "        tf.keras.layers.Flatten(),\n",
        "        tf.keras.layers.Dropout(0.5),\n",
        "        # 512 neuron hidden layer\n",
        "        tf.keras.layers.Dense(512, activation='relu'),\n",
        "        tf.keras.layers.Dense(NUM_CLASSES, activation='softmax')                       \n",
        "    ])\n",
        "    return model"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yuo9je7lCrJ_"
      },
      "source": [
        "## Summary of the Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WddTQKKGCc_n",
        "outputId": "d7f099e2-eb97-447a-f5d4-1db49acc8775"
      },
      "source": [
        "model = training_model()\n",
        "model.summary()"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv2d (Conv2D)              (None, 148, 148, 64)      1792      \n",
            "_________________________________________________________________\n",
            "max_pooling2d (MaxPooling2D) (None, 74, 74, 64)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_1 (Conv2D)            (None, 72, 72, 64)        36928     \n",
            "_________________________________________________________________\n",
            "max_pooling2d_1 (MaxPooling2 (None, 36, 36, 64)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_2 (Conv2D)            (None, 34, 34, 128)       73856     \n",
            "_________________________________________________________________\n",
            "max_pooling2d_2 (MaxPooling2 (None, 17, 17, 128)       0         \n",
            "_________________________________________________________________\n",
            "conv2d_3 (Conv2D)            (None, 15, 15, 128)       147584    \n",
            "_________________________________________________________________\n",
            "max_pooling2d_3 (MaxPooling2 (None, 7, 7, 128)         0         \n",
            "_________________________________________________________________\n",
            "flatten (Flatten)            (None, 6272)              0         \n",
            "_________________________________________________________________\n",
            "dropout (Dropout)            (None, 6272)              0         \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 512)               3211776   \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 3)                 1539      \n",
            "=================================================================\n",
            "Total params: 3,473,475\n",
            "Trainable params: 3,473,475\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZjP44DVHC1y3"
      },
      "source": [
        "## Compile and Train the Baseline Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "S1CJjfzfCvQG",
        "outputId": "472567c7-5c5a-4e6e-dfe6-882459f6c762"
      },
      "source": [
        "#Compile the Model\n",
        "model.compile(loss = 'categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])\n",
        "\n",
        "#Train the Model\n",
        "history = model.fit(train_generator, epochs=EPOCHS, steps_per_epoch=20, validation_data = validation_generator, validation_steps=3, batch_size=BATCH_SIZE)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "20/20 [==============================] - 153s 8s/step - loss: 1.6691 - accuracy: 0.3288 - val_loss: 1.0963 - val_accuracy: 0.3333\n",
            "Epoch 2/20\n",
            "20/20 [==============================] - 150s 7s/step - loss: 1.1090 - accuracy: 0.3786 - val_loss: 1.0841 - val_accuracy: 0.3925\n",
            "Epoch 3/20\n",
            "20/20 [==============================] - 151s 8s/step - loss: 1.0853 - accuracy: 0.3995 - val_loss: 1.0907 - val_accuracy: 0.3360\n",
            "Epoch 4/20\n",
            "20/20 [==============================] - 153s 8s/step - loss: 1.0781 - accuracy: 0.4319 - val_loss: 1.0076 - val_accuracy: 0.4919\n",
            "Epoch 5/20\n",
            "20/20 [==============================] - 152s 8s/step - loss: 1.0135 - accuracy: 0.4760 - val_loss: 0.9914 - val_accuracy: 0.6855\n",
            "Epoch 6/20\n",
            "20/20 [==============================] - 152s 8s/step - loss: 0.9675 - accuracy: 0.5599 - val_loss: 0.7068 - val_accuracy: 0.7527\n",
            "Epoch 7/20\n",
            "20/20 [==============================] - 153s 8s/step - loss: 0.8461 - accuracy: 0.5912 - val_loss: 0.6123 - val_accuracy: 0.7231\n",
            "Epoch 8/20\n",
            "20/20 [==============================] - 152s 8s/step - loss: 0.8882 - accuracy: 0.5715 - val_loss: 0.6119 - val_accuracy: 0.7742\n",
            "Epoch 9/20\n",
            "20/20 [==============================] - 152s 8s/step - loss: 0.6629 - accuracy: 0.7259 - val_loss: 0.3910 - val_accuracy: 0.8333\n",
            "Epoch 10/20\n",
            "20/20 [==============================] - 151s 8s/step - loss: 0.6706 - accuracy: 0.6997 - val_loss: 0.3811 - val_accuracy: 0.8952\n",
            "Epoch 11/20\n",
            "20/20 [==============================] - 152s 8s/step - loss: 0.6148 - accuracy: 0.7508 - val_loss: 0.6685 - val_accuracy: 0.5887\n",
            "Epoch 12/20\n",
            "20/20 [==============================] - 151s 8s/step - loss: 0.5901 - accuracy: 0.7504 - val_loss: 0.2708 - val_accuracy: 0.9382\n",
            "Epoch 13/20\n",
            "20/20 [==============================] - 151s 8s/step - loss: 0.4371 - accuracy: 0.8319 - val_loss: 0.3410 - val_accuracy: 0.7796\n",
            "Epoch 14/20\n",
            "20/20 [==============================] - 150s 7s/step - loss: 0.4914 - accuracy: 0.7939 - val_loss: 0.3374 - val_accuracy: 0.8790\n",
            "Epoch 15/20\n",
            "20/20 [==============================] - 152s 8s/step - loss: 0.3918 - accuracy: 0.8519 - val_loss: 0.4638 - val_accuracy: 0.7097\n",
            "Epoch 16/20\n",
            "20/20 [==============================] - 153s 8s/step - loss: 0.4647 - accuracy: 0.8049 - val_loss: 0.3544 - val_accuracy: 0.8387\n",
            "Epoch 17/20\n",
            "20/20 [==============================] - 153s 8s/step - loss: 0.3336 - accuracy: 0.8631 - val_loss: 0.3203 - val_accuracy: 0.8333\n",
            "Epoch 18/20\n",
            "20/20 [==============================] - 158s 8s/step - loss: 0.2608 - accuracy: 0.9066 - val_loss: 0.3127 - val_accuracy: 0.8763\n",
            "Epoch 19/20\n",
            "20/20 [==============================] - 152s 8s/step - loss: 0.3312 - accuracy: 0.8861 - val_loss: 0.1272 - val_accuracy: 0.9704\n",
            "Epoch 20/20\n",
            "20/20 [==============================] - 152s 8s/step - loss: 0.2326 - accuracy: 0.9215 - val_loss: 0.3334 - val_accuracy: 0.8575\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GN14Ym8zW2fw"
      },
      "source": [
        "## Evaluate our Baseline Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bMK5SRHSPezP",
        "outputId": "36ce64b5-8463-4c0b-9d4f-85e1b5e299a1"
      },
      "source": [
        "test_loss, test_acc = model.evaluate(validation_generator, verbose=1)\n",
        "print('Test Accuracy:', test_acc)\n",
        "print('Test Loss:', test_loss)"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3/3 [==============================] - 6s 2s/step - loss: 0.3334 - accuracy: 0.8575\n",
            "Test Accuracy: 0.8575268983840942\n",
            "Test Loss: 0.3334425091743469\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FC6bAfYCW6pS"
      },
      "source": [
        "## Plot the Acc and Loss Curve"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8hr9p57tDZJd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 573
        },
        "outputId": "ed9dd7c0-0f11-4efc-d792-4d4b93afa4cc"
      },
      "source": [
        "plt.plot(history.history['accuracy'])\n",
        "plt.plot(history.history['val_accuracy'])\n",
        "plt.legend(['train', 'test'], loc='lower right')\n",
        "plt.title('accuracy plot - train vs test')\n",
        "plt.xlabel('epoch')\n",
        "plt.ylabel('accuracy')\n",
        "\n",
        "plt.show()\n",
        "\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.legend(['training loss', 'validation loss'], loc = 'upper right')\n",
        "plt.title('loss plot - training vs validation')\n",
        "plt.xlabel('epoch')\n",
        "plt.ylabel('loss')\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hUVfrA8e9JT+ik0AKhB0KHUBQUFFGKgiyCYkHsXVZdV/1ZV9dVd9eyKq7KCnZAiggKgkgoSk3ogQBJBJLQUiCQnknO748zgSGkTJKZZJK8n+fJk8y9d+59J4F5557yHqW1RgghRP3lVtMBCCGEqFmSCIQQop6TRCCEEPWcJAIhhKjnJBEIIUQ9J4lACCHqOUkEQgBKqfZKKa2U8qjpWOyllLpNKbWqpuMQtZ8kAiEqSCn1ilLq6yqe43Ol1N+rcg6t9Tda62urco6qcGTydMTvQ1SeJAJRo5Qh/w6LqU13JqIO0FrLVz3/Ap4F4oBzwD5gYrH99wH7bfb3t25vCywGkoFU4EPr9leAr22e3x7QgIf18VrgdeB3IBvoDNxlc4144IFiMUwAdgJnrbGOBiYDUcWOexL4oZTXuRZ4A9hqPc8PQPNSYmwNLAXSgFjgPuv20UAekA9kALsq8fu+3/r8POs5llm3HwaeAXYDuYBHWX8bYDrwm81jDTwIHALOADMBVcL1W1t/781ttvUDUgBP699jHZBu3Ta/lNdx1HrNDOvXZdbtd1v/lqeBlUCIdbsC3gVOWX//e4Cepf0+5Ksa3wNqOgD5qvkv6xtqa8wd4s1AJtDKZl8SMND6H7kzEAK4A7us/7EbAD7AMOtzXqH8RHAU6GF9s/MExgGdrNcYDmRxIeEMsr4pjbLG2AboBnhb36i721xrBzCplNe51vpaelpjXlQUZwkxrgc+sr6uvphkd3VJr6+Sv/PPgb8X23YYk+zaAr52/G2mc2ki+BFoCrSzxjy6lOuvwZrcrI//BXxs/Xku8Lz1muf/riWc46LfmXXbBEzi7G79274AbLTuuw6IssanrMe0Ku33IV/V9yW35AKt9QKt9TGtdaHWej7mE+Ug6+57gX9qrbdpI1ZrfcS6vzXwtNY6U2udo7X+rQKX/VxrHa21tmit87XWP2mt46zXWAesAq6wHnsPMFtr/Ys1xiStdYzWOheYD9wOoJTqgXlz+rGM636ltd6rtc4EXgSmKKXcbQ9QSrUFhgLPWF/XTuB/wLQKvL7Kel9rnaC1zoZy/zYleVNrfUZrfRSIwCSxknwLTAXTPAfcYt0G5tN5CNC6En/XB4E3tNb7tdYW4B9AX6VUiPW8jTBJXFmPOV6BcwsnkUQgUEpNU0rtVEqdUUqdwXxiDrDubotpmiiuLXDE+p+9MhKKxTBGKbVZKZVmjWGsHTEAfAHcan0zuwP4zpog7LnuEczdSECxY1oDaVrrc8WObVPWC7J5LbcppTKsXyvseU4p8ZX3tynJCZufs4CGpRy3CLhMKdUKuBIoBDZY9/0V84l9q1IqWil1dwXiDwH+YxNvmvVcbbTWa4APMU1Wp5RSnyqlGlfg3MJJJBHUc9ZParOARwF/rXVTYC/mPy+YN6ZOJTw1AWhXSqdmJuBn87hlCcecL3urlPLGvDH9G2hhjWG5HTGgtd6MaVu+ArgV+Kqk42y0tfm5HeZTakqxY44BzZVSjYodm1Q89lJi+kZr3dD6Naa0w8rbbsffptK01qcxd103Y35v87S2ti9pfUJrfZ/WujXwAPCRUqqzna8hAdO/09Tmy1drvdF67ve11gOAMKAr8HQZ5xLVRBKBaID5T5gMoJS6C/Ops8j/gL8opQZYR/h0tr5BbQWOA28qpRoopXyUUkOtz9kJXKmUaqeUagI8V04MXpj2/mTAopQaA9gOi/wMuEspNVIp5aaUaqOU6maz/0vMJ818O5oxbldKhSml/IBXgYVa6wLbA7TWCcBG4A3r6+qNaZ4qGjJ6EmhfxdFOJ4GO5RxT3t+mqr7FNHfdxIVmIZRSk5VSwdaHp60xFJbw/GTrdtvX8THwnLWZDqVUE6XUZOvPA5VSg5VSnpgPCzk257Xn9yGcRBJBPae13ge8DWzC/GfshRnNU7R/AWaEz7eYkStLMKNNCoAbMJ3HR4FEzKdLtNa/YNrud2M6B8tqs8faBPM48B3mjedWzIidov1bMaOK3sV0Gq/DNEEU+QrzBmnP2P6vMB2TJzAdoY+XctxUTH/DMeB74GWt9WrrvgXW76lKqe12XLMknwFh1iaUJSUdUN7fxgGWAl2AE1rrXTbbBwJblFIZ1mNmaK3jS4gvC+voL+vrGKK1/h54C5inlDqLuYMpuitqjLnDOY1pakvFdFKDHb8P4TzKejcoRK2llPLFDEnsr7U+VMZxazGjff5XXbEJURvIHYGoCx4CtpWVBIQQpZPZi6JWU0odxnSe3ljDoQhRazmtaUgpNRu4Hjiltb6kg8s63O8/mGGCWcB0rXVl21uFEEJUkjObhj7HTMcvzRhMR1UXzBTz/zoxFiGEEKVwWtOQ1nq9Uqp9GYdMAL60jl3erJRqqpRqVd5Mw4CAAN2+fVmnFUIIUVxUVFSK1jqwpH012UfQhotnUSZat12SCJRS92PuGmjXrh2RkZHVEqAQQtQVSqkjpe2rFaOGtNafaq3DtdbhgYElJjQhhBCVVJOJIImLp/sHc2EKvxBCiGpSk4lgKTDNWrZgCJAulQiFEKL6Oa2PQCk1FxgBBCilEoGXMZUe0Vp/jCkqNhZTuzwLU0JACCFENXPmqKGp5ezXwCPOur4QQgj71IrOYiGEEM4jiUAIIeo5SQRCCOFshYUQ9TlkpdV0JCWSRCCEEM62ez4smwERr9d0JCWSRCCEEM6Unw1rXjM/7/gaMouvjFrzJBEIIYQzbf4IzibB9e+CJQe2flrTEV1CEoEQQjhLRjJseBdCx0L43RA6ziSCvMyajuwikgiEEMJZ1r0F+Vlwzd/M46EzIPu0aSJyIZIIhBDCGVIOQeRsGDAdAruabe0GQ9shsPFDKLDUaHi2JBEIIYQz/PIyePrBiOcu3j50BqQfhX1LaiauEkgiEEIIRzv8Oxz4CYb9GRoWK53fdTQEdIXf3wMnLRVcUZIIhBDCkQoLYdUL0Kg1DHn40v1ubnD543BiD8RHVH98JZBEIERtkJ8DG952udEmogTRi+HYdhj5Inj5lXxM7ynQqBX89l71xlYKSQRC1AZxv8Kvr8Lu72o6ElGW/BxY/Tdo2Qt631z6cR7eMOQh+GMdHNtRffGVQhKBELVBaqz5vu+Hmo1DlG3rp6Yj+Nq/g5t72ccOmA7ejeH396sltLJIIhCiNihKBH+sd9nCZfVeVhqs/zd0HgUdR5R/vE8TCL/LjB5K+8PZ0ZVJEoEQtUFqHPj5gy6AmJ9qOhpRknX/hLxzMOpV+58z+CFQ7rBppvPisoMkAiFqg9RY6DoGmoZI85ArSo2Dbf+DfndAizD7n9e4FfS5ucaL0UkiEMLV5ZyFjJMQ0BnCJkD8WlOmQLiOX/8G7l5w1f9V/LmXPw6WbNg6y/Fx2UkSgRCuLi3OfPfvDD1uhMJ8OLCiZmMSFxzdYu7Shj4OjVpW/PmBoaYoXQ0Wo5NEIISrS7F2FPt3gdb9oUlbaR5yFVqbyWMNW8Llj1X+PENnQHYa7PimxN3H07N5bO4OYk6crfw1yiCJQAhXlxoLKGjeAZQyzUNxayAnvaYjE/t+gMStcPXz4NWg8udpNwTaDoZNH1xUjC7PUsgn6+IY+fY6VkWfYP/xWpgIlFKjlVIHlFKxSqlnS9gfopT6VSm1Wym1VikV7Mx4hKiVUmOhaTszCQlMIijIg4Mrazau+s6SB6tfgaAw6Htb1c83dAacuVCMbmNsCmPf38AbK2K4vFMAq58czsR+znmL9HDKWQGllDswExgFJALblFJLtdb7bA77N/Cl1voLpdTVwBvAHc6KSYhaKTXW9A8UaRNu6tjs+8GUKhA1I/IzOP0H3Lao/Mlj9ug6Bvy7kL/hPZ7c05Flu4/Ttrkvn90ZzsjuLap+/jI4845gEBCrtY7XWucB84AJxY4JA9ZYf44oYb8Q9ZvWZmiibSJwc4Ow8XDoF8g9V3Ox1WfZp82iMx2vgs4jHXLKfA3rg6bieWoP5/avZsbILvzyxHCnJwFwbiJoAyTYPE60brO1C/iT9eeJQCOllH/xEyml7ldKRSqlIpOTk50SrBBlKsiH+HXVXzY445SZpGSbCMDaPJQLh1ZVbzz1VHpWPl9uOkzEgVPk5BeYAoDZZ+Da10y/TRVtjk9l3PsbuHdHJ067N+e/7TfwxKiu+Hg64E7DDk5rGrLTX4APlVLTgfVAElBQ/CCt9afApwDh4eGuUcBb1B9aw49PwI6v4O5VZpWp6lJUWsK/08Xb2w6Ghi1M81DPSdUXTz2Tayngq01H+GBNLOnZ+QB08kjhZ4+P+aPVDbi5daCT1qhKJoNTZ3P4x/L9LNl5jDZNfZk57TKaps1ArX4Zju2E1n0d+XJK5cxEkAS0tXkcbN12ntb6GNY7AqVUQ2CS1vqME2MSouI2fWiSAMDJPdWcCA6Z7wFdLt7u5g7db4Cd35qx51UZsSIuUVioWbb7GP9aeYDE09kM7xrIk6O6ciY7n+YrHqLgtGLaH6M48c46gpv5MrxrIMO7BnJ55wAaepf/tmopKOSLTUd495eD5FkKefzqzjw0ojO+Xu6Qc5epWbTxfbhpdjW8Wucmgm1AF6VUB0wCuAW41fYApVQAkKa1LgSeA6rnVQthrwMrYNWL1iGbEXBqf/VePzUW3L2hcQmjRcImmLIGsavNz8IhNsal8MbyGPYkpRPWqjFf39ObYV0CzM7EKDj9C1z5NAv63sT6Q8msPZDMkh1JfLPlKJ7uigEhzRgRGsTwroF0a9nokruFrX+k8dIPe4k5cY7hXQN5ZXwPOgTYJPKiYnSbPoSRL0Gz9k5/zU5LBFpri1LqUWAl4A7M1lpHK6VeBSK11kuBEcAbSimNaRp6xFnxCFFhJ/bAwnvM7fmNH8OXE+BUTPXGkBpnmoXcSujOa3c5+AWY5iFJBFV28OQ53lwRw5qYU7Ru4sM7U/pwY982uLlZ38iLJo81CIShM2jr7cdtg0O4bXAIeZZCoo6cZt3BZNYdTObNFTG8uSKGoEbe5m4hNJDurRozMyKWxduTaNPUl0/uGMC1YS1KblYa8hBs/q8pRjf2X05/7U7tI9BaLweWF9v2ks3PC4GFzoxBiEo5dxK+vcV8OrtlrllpKqg77F9m3hAc0EFol9RYU4KgJO4e0P162L0A8rPB09d5cRQWQuwv0GmkuW4dcvJsDu+sOsiCqAQaeHvw7JhuTL+8/aUdtQeWw9GNMO4d8G500S4vDzcu6+TPZZ38eXZMN06ezTmfFFZGn2BBVCIAnu6KR67qxCNXdcbPq4zfY+PWZmGb7V/B8GehwSVjaByqbv1FhXCE/GyYN9VM+b/7Z1MhEkwi2P4FZCZDwyDnx1FgMXXqu40r/ZiwCRD1OcT+apKCs+z6Fn54BCZ+An1ucd51qlFGroVP1sUxa0M8BYWa6Zd34LGrO9OsgdelB+eeg19eMovO97+z3HO3aOzDlPC2TAlvi6WgkF2J6ew4epqrugXRKbChfQFe/hjs/Bq2zYIRl8zHdShJBELYKiyEJQ9B0na4+Wto1efCvqDu5vupfdWTCNKPmgJzxYeO2mp/Bfg2M81DzkoEllxY+6b5OebHWp8I8gsKmbf1KO+tPkRqZh439GnN09eG0s6/lPWFTx+GuVMhLR5uW1DhOyIPdzcGhDRjQEizigUa1M1MMtvyialQWtr6xw4giUAIW+vehOjv4Zq/XfrGGliUCGLsW4GqqlKLqo52Kf0Yd09zxxD9g3nDLipD4UiRcyA9wazDG7vGrMvr6eP46ziZ1pqV0Sf5588xxKdkMqhDcz4b252+bZuW/qTDv8N3d0ChBW5fBJ2urr6AwZSdmDMadn4Dg+5z2mUkEQhRZPcCM1u07+3mP2BxDYPAt7m5I6gOKdaho2XdEQCE3WgWNomLgNDRjo0hNwM2/Bs6XAmXz4BvJpkF17te59jrVFB+QSGZuRbO5VjIzLOQkWPhXK6FzFzzc0au9cv687lcC4dTMok+dpbOQQ3537RwRnYPKnv8f9Tn8NNTZtTO1PlmPYjq1m4IBA+CjR/AgLuc1j8jiUAIgIStpg08ZChc/27JncFKmeah5GoaOZQaCz5Nwa952cd1GG46tff94PhEsOVj0ydy9Vxo1Ru8GppO02pOBKv3neTtXw6SfC6HczkWci2Fdj3Pz8udht4eNPTxoImvJ/+Y2Isp4cF4uJdRVKHAAqueN6+900gzlt+3jLsGZ1LKfCiZfxvsd97kQUkEQpw5CvNuNSM1pnwFHiV0FhYJ6g67v6uekUNFxebKu46HF4SOgwM/mYqYZcVfEdmn4ff3zaIpbQeabZ1HmrkV494teUirg53JyuPVZftYvCOJLkENua5HS/PGbn1zb+DtQSNv872hz8U/N/DywN2tgn+j7NOw4C6Ij4AhD8Oo12p+lFToWPPv4Pf/QI8/OeXfnSQCUb/lnIVvbzZvoNO/K3+YXlB3yD0LZ49Bk+KlsxwsNQ7aD7Pv2LAJZmTPH+uhyzWOuf7v75vXetXzF7aFjjN3Hse2Q3C4Y65TilXRJ3h+yV5OZ+bx+MguPHpVZ7w8nJh8Ug6ZfwtnjsL4D6G/ixRCdnMzncXLHjfNch1HOP4SDj+jELVFYQEsuheSD8CULyCwa/nPOd9h7OQZxnlZcDax/P6BIp2uAq9G52vZV9m5k6ZppNdN0LLnhe1dRoFyh5ifHHOdEpzOzGPGvB3c/1UUAQ29WfLIUJ4c1dW5SSB2NcwaaRb7uXOZ6ySBIn1uMRMILblOOb0kAlF/rXoRDq00Mzc7XWXfc4qGkCY7ORGkxZvvxYvNlcbDG0LHmOGdBflVv/6Gt82bzojnLt7u1xxCLnfamsk/7z3OqHfX8dPu4zxxTVd+eGQoPds0ccq1ANPEt/m/8M1kaNoW7o+AkMucd73K8vCGu1c4rW9GEoGonyJnw+aZMPhBGHiP/c/za26qfjr7jqC0YnNlCZtg2rgP/1a1a585an4//e8oORGFjjWJsChZOUBqRi6PfrudB7/eTovGPix9dBgzruni3LsASx4sfQx+fta8prtXmpXg6iFJBKL+iV8LP/0FOo+Ca1+v+PODuldDIrCWn27e0f7ndB4Jng2qvrD92rdAucGVfy15f7ex5nvM8pL3V9DyPce59t31rIw+wVOjurLkkaGEtW7skHOXKiMZvhxvqspe+bQZJOBt54zfOkgSgahfUg7Bd9NMqYCbZlduREigdQhpoX1DGCslNQ4at6lYeWlPX9N0sH/ZRQugV0jyQdPpPOi+0jvDm7WHoB5Vbh5KycjlkW+28/A322nd1Jdljw3jsZFd8CxraKcjnNgLs66GYztg0mdw9QvVMgLKlcmoIVF/ZKXBt1PAzRNunQ8+lfzUGdQd8rPgzBFo3sGxMRZJjbW/f8BW2ASIXmyKo3W4suLPj3gdPP1g2BNlH9dtrOlHyEorf55DMVprftx9nJeXRpORY+Hp60J54MqOZY/tP30EZl0Fbh7QqKVZs7lRSzPkt/hj32alD7Hc/yMsvt/87e9aAW36Vyj2ukoSgagfLHkw/w5IT4Q7f4RmIZU/1/kO4xjnJoIeEyv+vC6jwMPXNA9VNBEc22lGHQ1/BhoElH1s6BhY/y84uBL6TrX7EsnncnlxyV5+jj5Bn7ZN+ddNvenaolH5T4yaY5aG7DvVjGhKT4TErZCVeumx7l4lJ4vMFLPYS5sBcPM3F4oJCkkEoh7Q2nQKHvkNJn5a9RXGAruZ76f2mTdER8tKM52+9g4dteXVwCSD/ctgzD/NSmb2WvN382n6MjuWBWnVDxq1MpPY7EgEhYWapbuO8cqyaLLyCnh2TDfuHdah7LuAIpY8U0Kj62iYMLPYvlw4d8L6dfzC11nr95N74dAvkJ9pju81Bca/79yS3bWQJAJR9635O+yeZyZG9bm56ufzaWxWDHPWIjXn1ymuZG2bsAmwfykkbDFDPe1xZKNZb2DUq6ZcRXnc3EwS3DW/zCJ06Vn5LIhK4OvNRzicmkW/dk3510196BxUgY7ZmB9NmYvwuy7d5+Ft7u7Ku8PLOWsmxzVuU31rSdQikghE3RY5xxRN6z/NjA5xFGeOHLK32Fxpul5nlrfc98P5RBCfnMHff9pP7+AmjO3Vii5BDS8UXNMafn0VGraEgRWocBk6zgwz/WM9dL32ol37jp3lq82HWbLjGNn5BQwIacYTo7pyfe/WFS/7EDXHDOusSuVPn8aV7xOqByQRiLrr4EpTPbLzKLOqlCM/CQZ1M2+ABRbH16JJjTWdok0r2Y/h3Qg6XwP7lsJ1b3AqI49ps7eSkpFLxIFTvLf6EJ0CGzC2VyvG9GxF98wtqKObYNzbFat53+EKaxG6n6DrteRZCvk5+gRfbjxM5JHT+Hi6cWPfNtxxWQg9WldyUlhKrPk9X/1ixZq5RIVIIhB1U9J2WDDdlEeY/Lmp2+9IQWFQkAun/6jYpC97pMZCsw5VSzBhE+DAT2T+sZlpyyykZebx3QOX0bKJDyujT7Jiz3FmRsTy4ZqDrPR7iSDv1iQEjaen1mWXZrbl4Q2dR1Kwfzn/8XqQb7clkZKRS4i/Hy+M687kAW1p4lfF33vUHJMU+7lYyYc6RhKBqHvS/jDDRBsEwK0LnDNR6HyH8X4nJIK4yjcLFQkdjXbzJGLxLGJPT2b29IH0DjallO8YEsIdQ0JIzchl/69f0XVHPE9lPsyi/26jTVNfxvZqyZheregb3PTCwu3FaK3ZHJ/GgdQwpmf9wPp1q+jd9XKmXRbClV0CS31eheTnwM5vzcI7jVpU/XyiVJIIRN2SlQbf3GTq7Uxf7rw3kMBQQJlEEDbecectLIS0OPtrH5WiwKsx0T4D6Juxnrcnv8aVXQMvOcbf151hCZ9AYDdemPYSgw+k8PPeE3y+8TCzNvxBqyY+jO7ZkrG9WjGgXTPc3BQZuRa+35HEV5sOc/BkBu18uzANd764PIUm1w+sUsyX2L/UrBs9oIROYuFQkghE3ZGfDXNvgTMJMO0H+6qJVpZXAzNSxdHF584mgSWnSncEWmte+mEvuel9+LfnZoKDTgLBlx64ez6kHISbv6ZZI9/zi62nZ+ezJuYky/ec4JstR5nz+2GCGnkT3r4Z6w+mkJFroWebxvzzpt6M79Mat2/m0OTIL8BrlY65RJFzTBNZh+GOPa+4hFMTgVJqNPAfwB34n9b6zWL72wFfAE2txzyrtXZMARNRvxQWmBmjCVth8pzqqSAZFOb4kUNVHToKvP9rLN9sOcqMoZNgx2dm9FCbARcfVLQgfet+0O3itZmb+HoysV8wE/sFk5FrYU3MKVbsOc7WP9IYFdaCOy4LoV/bphf6EkLHwsrnTBG6itRGKsupGDM7etSr9b78Q3VwWiJQSrkDM4FRQCKwTSm1VGttu+DrC8B3Wuv/KqXCgOVAe2fFJOqwVS+YpoRrX6/cjNzKCOwGh1Y5dlWwokRQyX6Hb7Yc4d3VB5nUP5g/X98b0keYRHDN3y4eNRX1BaQfhfH/KXM0VUNvD8b3ac34Pq1Lv2joGJMIDqywbzKaPaLmmBnCfW9zzPlEmZyZagcBsVrreK11HjAPmFDsGA0UDe5tAhxzYjyirto0EzZ/BIMfctwbkT2CwqDQcuHN2xFSY82QzIYV79v4ee8JXlyyl6tCA3lzUi/ziT1sApw+DMd3XTgwL9OUh2h/BXSsWl8EYMpsBIU5rBop+dmway50v6H8UhfCIZyZCNoACTaPE63bbL0C3K6USsTcDTxW0omUUvcrpSKVUpHJycnOiFXUVtFLYOXz5k3juterd9ZokHXkkCP7CYqKzVXwdWyJT+XxeTvoHdyUmbf1v1DBM3ScWVHMtjT1lk8g8xSMfMlxv6/QsaYpJyut6ueK/t6sFBZ+d9XPJexS041vU4HPtdbBwFjgK6XUJTFprT/VWodrrcMDAy8d/SDqqSObTL9A20Hwp1nVP+HIv4t5k3VkP0HRgvUVEHPiLPd+GUnbZr7MmT4QPy+bFt8G/mbi174lZgZx9hn4/T3oOsb83hyl21jQhaaprKoiZ5sy4SFDq34uYRdnJoIkoK3N42DrNlv3AN8BaK03AT6A3AuK8iUfNCOEmraFqfNqpoiYp4/pHHVUIrDkmtXBKpAIEtKymPbZVhp4efDlPYNp1qCEvoqwCaYj92Q0bPzAfNq++gXHxFykqAhdVdcyPrEXErfBgOlSE6gaOTMRbAO6KKU6KKW8gFuApcWOOQqMBFBKdcckAmn7EWU7dxK+mWRmC9+2sML18B3KkTWHTh82n6rtTARpmXncOXsrOfkFfHH3INo0LSUZdrverDi2bZZZn7dnsQXpHcHNzVQHjf3VTASrrKg5pk5SH/tLW4uqc1oi0FpbgEeBlcB+zOigaKXUq0qpohk4TwH3KaV2AXOB6Vpr7ayYRB2QmwHfTja15W+d77z1AOwV1N2UmcjPrvq5zg8dLX9Bmqw8C3d9vo2kM9l8Nn0goS3LqOnfMMg0s0R9buYoXPV/VY+1JN3GmXLPf6yv3PNzM0w10x4Taza510NOnUdgnROwvNi2l2x+3gdIQ6CwT4HF1A86sQdumXvp2PiaENTdfIpPOQit+lTtXHZWHc0vKOShr7ezJ/EMH98+gIHt7XjTDJsAhzdAv9srt/KZPTpcaS1Ct/ySaqR22bsI8s6VXG5aOFVNdxYLYR+t4acnTM38cW9D6OiajsgItK5W5oi1CVJjoUFQmesBFBZqnlm4m3UHk/nHxF5c26OlfefuNdm0u1/1fNXjLI21CB0HVlRuPeeoOWYYatsqLhwkKkwSgXB9WsMvL8L2L+GKp1xrWKF/J7MG8ql95R9bHjuKzb31cwyLdyTx1Kiu3DKonf3n9m0KN/zH+cXbQsdCxgmzMHxFHNthvgbcJZ3ENUASgXBtWsPPz5nRLgPvNXXpXYm7p5kFnOygO4JSmm201obyaqQAACAASURBVMxaH88n6+OZdlkIj15dxeqkztLlWjOk9kAFRw9FzjFrLfee4py4RJmk6JxwXYWFsOKvZrTL4Idg9Buu+WkxqLsZ8lgVOemQeYozfiFEx6ZwJDWLI2mZHEnJ4khaFkdTM8nMK2Bcr1a8fEMP+9cMqG5+zc2qaAdWmAlr9sg5C3sWQq9J5s5FVDtJBMI1FRaaPoGoz+Hyx2DUa66ZBMD0E+xdZEa9lLP2QU5+AQlpWRxJzeJwaiZHrT/7JO/iE+CvEZms+nULAF7ubgQ39yWkuR+DOzSnS4uG3DQguOJLPVa30DGw8v/MuhD2jOra850ZbTTAhZr86hlJBML1FBbAssdhx9cw7EnHlkJwhiBrh3HyAQgueSTTqugTvPbTPhLSLh5m2sjbg3b+fkxqfBqy4cZrRjA9pCch/g1o2djH9d/0SxI61iSCA8vLr/2kNUR+Di17QZv+1RKeuJQkAuFaCgtgycOwex4MfwZGPOfaSQBsEsH+SxJBYaHm/TWHeG/1IXq0bsyTo9oS4u9Hu+Z+hPg3oJmfp2nmidgAp9wYe+VlZvRNbVZUhM6eaqSJkXByj+PXlBYVIolA2Ccj2dzC977F1K9xhgILfP8A7F1ohjkO/6tzruNozdqDh88lM4wzci08OX8nq/ad5E/92/CPib3w8SylHlJqLDRtV/uTQJHQsfDbu6YIXVmTw6LmmLkH0klco2TUkChfQT7Mv93c7r/fDzZ+aGrwO/oai+4xSWDky7UnCYApdhfQ9aJEcDglk4kzf+fXmFO8dH0Yb0/uU3oSgEoVm3NpoWNBF5RdhC77NOxdDL1uAu8yZkYLp5NEIMr3y8uQsNksbtJ2EKx6Hj4aDPt/NG28VWXJMzOG9y2Ba/8OVzxZ9XNWN5vVytYdTGb8h7+RnJHLl3cP4u5hHcoe5aO1YxasdyWt+0HDlmUXods1HyzZsiaxC5BEIMq2dzFsngmDH4Rhf4bbF8Jti8wkqvm3wRc3wPHdlT+/JRe+mwYxP8Lot8wIodooqBucO8ac1Tu5a85WWjf1Zekjwxja2Y5iuhknIS+jbiUCNzczeqi0InRam2ah1v2hdd/qj09cRBKBKF3yAVj6GAQPMsM3i3S5Bh7aCGP/bUobf3KlOe7cyYqdPz/HNDkdXGHKRgx50LHxV6Pc5maRmh9/XcOYnq1Y/PDltPP3s+/JFSg2V6sUFaE7vOHSfUc3m0l4UlfIJUgiECXLzYD5d5hO0ClfXLomr7sHDLoPHt9uRobsnAsf9IcNb9tXhjg/G+ZNhUO/mNIHA+91zuuoBklnsnlwZSYAj/a08OGt/S5eHKY854vNVW6dYpfV/grwbFBy81DkbPBuDD0nVX9c4hKSCMSltDaf8FMPwU2zoXEZC5f7NjNLRD6yBToMh19fhZkDTZNSaf0HeZnw7RSIi4AJM00xtFpqS3wq4z/4jagzDbB4NOCqZikVn/WbGmsSbuPiK7nWcp4+JRehy0ozS2f2vhm8GtRcfOI8SQTiUls+gejFpq5Px+H2Pce/E0z9FqYtBe8msPAumD0akqIuPi73HHwzGQ7/BhM/gX63OT7+aqC15stNh7ntf1to4ufJ949egUeLbpUrPpcaB807mXb1uqbbOFOE7rhNEbqd30JBrjQLuZA6+C9PVMnRzWZUUOg4GPZExZ/fcTg8sA5ueN8sjzjralj8AKQnmZoyX08y1/jTLOhzs+Pjrwa5lgKeXbSHl36IZnjXQJY8MpROgQ3NxLLKFJ8ro9hcrVdUhC7GuixJUSdx8CBo0aNmYxPnSSIQF2ScMsM4m7SFGz+q/ExPN3cYcKfpPxj2JER/Dx8MMEkhKco0N/W6yaGhl6ewULMlPpU9iekknckmJ7+gUuc5dTaHqZ9uZn5kAo9e1ZlZ08Jp7ONpdgaFQWayWT3NXgUWs8JZXRoxZMuvObS7zJSbANNxnBrrWqXEhcwsFlYFFlh4N2SfgXt/cUwVSO9GcM3Lpg9g9ctw4GeY/AV0v77q566gJTuTePK7XRdta+Dljn9Db5o38CKgoRfNG3jh39Ab/wZe+Df0onkD25+92HfsLA98FUVGroWPbuvP2F6tLr5IoBk5xKn90OEK+wI7cwQKLXU3EQB0G3uhCF3kHPBpCj1urOmohA1JBMJY85r5tHbjx6YAmCM1C4HJn5tk414z/+Tmb0sgxN+P58d2Jy0zj9TMPFIz8kjNzCUtM4+kMznsSUonNSMPS2Hpk+TaNvfly3sup1vLxpfuDAoz3yuSCFLjzPe6nAiKqpFu/xL2LzMjxDx9azoqYUMSgTAzhH9/z8zw7DvVedepoSRwJDWTLX+k8fR1oeUu7ai15myOhdSM3IsSRlpmLpZCzZ2XtadZA6+Sn9yopVlmMnl/yftLkmodOhpQx4aO2mre0ZTq/v09s76zdBK7HLv+ZyqlFgOfASu01pVYjFS4rNQ4WPKQKQkw5q2ajsYpFkYl4qZgUv/gco9VStHE15Mmvp50DKzghZS6qNSEXVJjzRDcsgqz1QXdxsKG/RAyFAJDazoaUYy9ncUfAbcCh5RSbyql7PpLKqVGK6UOKKVilVLPlrD/XaXUTuvXQaXUmQrELqoqL8tMGnNzhylf1p3KlzYKCjULoxK5smsgLZv4OP+Cgd1MIrC3BlNdKzZXmrAJgIJB99d0JKIEdiUCrfVqrfVtQH/gMLBaKbVRKXWXUsqzpOcopdyBmcAYIAyYqpQKK3beJ7TWfbXWfYEPgMWVfymiQrSGH58w494n/c+UQK6Dfo9N4Xh6DpMHtK2eCwaFQc4ZOHfCvuPrWrG50rTqA0/ul05iF2X38FGllD8wHbgX2AH8B5MYfinlKYOAWK11vNY6D5gHTCjjElOBufbGI6oocrZZ/GXEc9D5mpqOxmkWRCXS1M+Ta8KCqueCQdaRQ/b0E+RlwtmkujuHoLjGrco/RtQIuxKBUup7YAPgB9ygtR6vtZ6vtX4MKG2R1jZAgs3jROu2ks4fAnQA1pSy/36lVKRSKjI5OdmekEVZEqPg52eh8yi48umajsZp0rPyWRl9ghv7tsHbo4y1ABzJduRQedLizff6cEcgXJq9wzje11pHlLRDax3ugDhuARZqrUuc5aO1/hT4FCA8PNwBBfDrscxUWHCnqRX/p0/rZlkDq6W7ksizFHLTgPI7iR2mQQD4BdiXCM5XHZVEIGqWve8CYUqp8zOMlFLNlFIPl/OcJMC2YTbYuq0ktyDNQs5XWACL7zX172/+ss6PVPkuMpHurRrTs02T6r1wUHf7EkGKNRE0rydNQ8Jl2ZsI7tNanx/Ro7U+DdxXznO2AV2UUh2UUl6YN/ulxQ9SSnUDmgGb7IxFVNa6tyBuDYz9lxkuWoftP36WPUnpTAmvxruBIkU1h8obOZQaC42DwcvOdQuEcBJ7E4G7sqmtax0RVMqsGkNrbQEeBVYC+4HvtNbRSqlXlVLjbQ69BZintSPWPBSlOrrZJIK+t0H/O2s6GqdbEJmIp7tiQt8aKO0c1N2sOJaeUPZxdbnYnKhV7O0j+BmYr5T6xPr4Aeu2MmmtlwPLi217qdjjV+yMQVTF3kXg6WdWFatsMblaIs9SyJKdSYwKa0Hz0mYBO1Ngd/P9VEzpw3K1NrOKe1Zv8T0hSmLvHcEzQATwkPXrV+CvzgpKOEFchJnVWQ+aIdbEnCItM6/65g4UVzSEtKy1CbLSICddOoqFS7DrjsBaVuK/1i9R26Qnmk+ftXglsIpYEJlAUCNvruhix8LxzuDbDBq1KnttAhkxJFyIvfMIuiilFiql9iml4ou+nB2ccJA468jfTlfVbBzV4NTZHNYeTGbSgGA83GtwaGxQ97LvCOrqgvWiVrL3f8oczN2ABbgK+BL42llBCQeLj4CGLS5MdqrDFu9IoqBQM7k65w6UJLA7JB80Q3ZLknoI3DyhaUj1xiVECexNBL5a618BpbU+Yu3gHee8sITDFBZC/FroOKLOdxJrrVkQmcCAkGZ0DCxtwns1CeoOlmw4fbjk/amx0LxDjZXmFsKWvYkgVynlhqk++qhSaiKll5YQruTkHshKhY51v1loR8IZ4pIza2buQHFB1pFDpfUT1Jdic6JWsDcRzMDUGXocGADcDtT9weh1QVH/QMcRNRlFtVgQmYCvpzvjereu6VAu1NwvqZ+gsNCaCKR/QLiGcu9LrZPHbtZa/wXIAGR5odokfq1pr67jlR+z8wpYtus4Y3q1pKG3CzS3eDeCJu3MXILiziZCQa7cEQiXUe4dgbUQ3LBqiEU4Wn4OHN1UL0YLrdh7nIxcC1PCa2juQElKqzkkQ0eFi7H3o9MOpdRSYAGQWbRRay0Lybiyo5vAklNPmoUSadfcj8EdXKiQXlA3M2KrIB/cbdZvOr9gfR1ep1jUKvYmAh8gFbjaZptGVhRzbfERZohiyNCajsSpEtKy2BSfylOjuqJcaWRUUBgU5Jl1B2zX6U05BF6NoGE1LZYjRDnsnVks/QK1UVwEtB0E3nV7gNeCqESUgkk1PXeguMCiUhP7L04ERcXmXClpiXrNrkSglJqDuQO4iNb6bodHJBwjMwVO7IarXqjpSJyqsFCzKCqRYZ0DaN3Ut6bDuVhgKKBMIrBdqzc1FoIH1lhYQhRn7/DRH4GfrF+/Ao0xI4iEq4pfa77X8Y7ijXGpJJ3JZrIrdRIX8fQ1k8Zs1y+25MKZo9JRLFyKvU1Di2wfK6XmAr85JSLhGPER4NOkzi9AsyAqgcY+Hlwb1qKmQylZUNjFI4fS/gC0JALhUipblasLID1drkpriFsLHa4Et2patL0GpGfn8/PeE4zv2xofTxd9nYHdzCghS655LMXmhAuyt/roOaXU2aIvYBlmjQLhilLjzKSlOl5WYtmuY+RaCl1r7kBxQd1BF5iRQiBzCIRLsrdpqJGzAxEOFF8/ykosiEoktEUjelX34vQVYVtzqGVPU3W0YQvwaVyzcQlhw947golKqSY2j5sqpW4s6zmiBsVFmCUSm3es6Uic5uDJc+xKOMPk8GDXmjtQnH8XcPO4UHNIis0JF2RvH8HLWuv0ogda6zPAy84JSVRJgQUObzDNQq78BllFCyIT8HBTTOxXA4vTV4SHFzTvdKHDWBasFy7I3kRQ0nEuUNlLXCIpCnLPutyw0bTMPOZuPUp6Vn6Vz5VfUMj3O5K4ulsQ/g29HRCdkxXVHMo+A5nJckcgXI69iSBSKfWOUqqT9esdIMqZgYlKio8AFHQYXtORnBd9LJ0bPviN5xbvYdg/1/DBr4fIyLVU+nwRMadIychz7U5iW0HdzQI1J/eax5IIhIuxNxE8BuQB84F5QA7wSHlPUkqNVkodUErFKqWeLeWYKda1kKOVUt/aG7goRVwEtO4Lfq5RfG3prmNM+u9GCrXmw1v7MaSjP2//cpAr/xnBrPXx5OSXspRjGRZEJRLQ0JsRoYFOiNgJgroDGmKWm8eSCISLsXfUUCZQ4ht5aazrGMwERgGJwDal1FKt9T6bY7oAzwFDtdanlVIyN6Eqcs5C4jYYOqOmI6GgUPPPlTF8si6ege2b8dFtAwhs5M31vVuzK+EM/151gNeX72fWhngeu7ozNw9sh5dH+Z9Lks/lsibmFPcM61Czi9NXRKB15ND+ZaDcoFmHmo1HiGLsHTX0i1Kqqc3jZkqpleU8bRAQq7WO11rnYe4kJhQ75j5gptb6NIDW+pT9oYtLHPndjFmv4f6B9Kx87v58G5+si+e2we345t4hBDa60Jbfp21TvrpnMPPvH0KIvx8v/hDN1W+vZUFkApaCwjLPvcRVFqeviOYdwd0L0o+axeo9vGo6IiEuYu9HqgDrSCEArG/c5X16bwMk2DxOtG6z1RXoqpT6XSm1WSk1uqQTKaXuV0pFKqUik5OT7Qy5HoqLAA9faDu4xkI4dPIcE2b+xsa4FP4xsRevT+xV6if9wR39+e6By/ji7kE0b+DF0wt3c+1761m26xiFhZfUOERrzXeRCfRt25QuLWrR1BZ3Dwjoan6WZiHhguxNBIVKqXZFD5RS7SmhGmkleGDKVYwApgKzbO88imitP9Vah2utwwMDa0m7cE2Ij4CQy8GjZkbSrIo+wY0zfycjt4C59w3h1sHtyn2OUorhXQP54ZGhfHLHADzd3Hhs7g7Gvr+BX/adROsL/8x2JaZz6FRG7ekktlU0sUwSgXBB9g4BfR74TSm1DlDAFcD95TwnCbD9Hxts3WYrEdiitc4H/lBKHcQkhm12xiWKpCdBykHoP63aL11YqHl/zSHeW32IPsFN+PiOAbRqUrGS0EopruvRkmu6t+DH3cd4b/Uh7vsykj5tm/KXa7syrHMACyIT8PF04/o+tXD95fOJQOYQCNdjb2fxz0qpcMyb/w5gCZBdztO2AV2UUh0wCeAW4NZixyzB3AnMUUoFYJqK4u0PX5x3vqxE9fYPZORaeHL+TlbtO8mf+rfhHxN7VakAnLubYkLfNozr1YpF2xN5/9dY7vhsK4M7NGff8bOM7tGSxj6e5Z/I1bToZb7bLlAjhIuwd2Gae4EZmE/1O4EhwCYuXrryIlpri1LqUWAl4A7M1lpHK6VeBSK11kut+65VSu0DCoCntdapVXlB9VZcBDQIghY9qu2Sh1Myue/LSOJTMnnp+jDuGtreYeUePNzduHlgO27s14Z5WxP4MCKWczkWpgyshc1CAJ2vgVsXQPsrajoSIS6hbNtgSz1IqT3AQGCz1rqvUqob8A+t9Z+cHWBx4eHhOjIysrov69oKC+HfXaDT1TBpVrVcct3BZB77djtuboqZt/ZnaOcAp14vO6+A/SfO0r9dM6deR4i6SikVpbUOL2mfvX0EOVrrHKUUSilvrXWMUkrucV3FqWjISqmWYaNaaz5dH89bP8fQtUUjZk0Lp21zP6df19fLXZKAEE5ibyJItI7mWQL8opQ6DRxxXliiQuKqp+x0dl4BzyzazdJdxxjXqxX/mtwbPy8pOSVEbWdvZ/FE64+vKKUigCbAz06LSlRMfAQEhELj1k67RPK5XKbP2cq+42d5+rpQHh7RybXLPwsh7Fbhj3Na63XOCERUUn4OHNkIA6Y79TIvLNlD7KkMPrsznKu7uej6wEKISqklxVpEqRI2gyXHqcNGV0WfYGX0SWZc00WSgBB1kCSC2i4uwqyA1X6oU06fkWvh5aXRdGvZiPuuqLsrnglRn0lPX20XHwHBg8DbObV33l51gBNnc/jw1v541pZqn0KICpH/2bVZZioc3+20YaO7E8/wxcbD3Da4HQNCZOimEHWVJILa7I91gHZK/4CloJDnFu8hoKE3fx3dzeHnF0K4Dmkaqs3iI8C7CbTu5/BTf77xMNHHzvLRbf1rZ20fIYTd5I6gttIa4tZChytMvXsHSjydxdurDjKyWxBjerZ06LmFEK5HEkFtlRZvVrzqOMKhp9Va89IP0QD8bUIPmTQmRD0giaC2iltjvncqtQBspazYe4I1Mad46tquBDdzfg0hIUTNk0RQW8WvhSbtzHq4DnI2J59XlkbTs01jpl/e3mHnFUK4NkkEtVGBBf5YD51GgAObbv75cwwpGbm8MbE3HjJnQIh6Q/6310bHtkPuWYcOG406cppvthxl+uUd6BXcxGHnFUK4PkkEtVH8WkA5rKM4v6CQ/1u8h5aNfXjy2q4OOacQovaQeQS1UVwEtOoDfs0dcrpZG+I5cPIcs6aF09Bb/kkIUd/IHUFtk3sOErc67G7gSGom/1l9iNE9WjIqTCqLClEfSSKobQ7/DoUWh9QX0lrzwpK9eLq78cr46lv0XgjhWiQR1DbxEeDhA22HVPlUS3cdY8OhFJ6+LpSWTXwcEJwQojaSRFDbxEVAyOXgWbU37jNZeby6bB992zbl9iEhDgpOCFEbOTURKKVGK6UOKKVilVLPlrB/ulIqWSm10/p1rzPjqfXSkyDlgEOGjb6xPIYz2fm88adeuLtJGQkh6jOnDRFRSrkDM4FRQCKwTSm1VGu9r9ih87XWjzorjjolfq35XsX+gS3xqcyPTOCB4R3p3qpx1eMSQtRqzrwjGATEaq3jtdZ5wDxgghOvV/fFr4UGgRBU+Y7dXEsB//f9HoKb+fLnkTJnQAjh3ETQBkiweZxo3VbcJKXUbqXUQqVU25JOpJS6XykVqZSKTE5Odkasrk9rkwg6jgC3yv/ZPl4bT1xyJn+/sSe+Xu6Oik4IUYvVdGfxMqC91ro38AvwRUkHaa0/1VqHa63DAwMDqzVAl3EyGjJPVWn+QFxyBjMjYrmhT2tGhAY5LDQhRO3mzESQBNh+wg+2bjtPa52qtc61PvwfMMCJ8dRuh1aZ75XsKNZa8/z3e/DxdOOl68McGJgQorZzZiLYBnRRSnVQSnkBtwBLbQ9QSrWyeTge2O/EeGq3fUugTTg0Kal1rWyWgkL+/tN+Nsen8dzY7gQ28nZCgEKI2sppo4a01hal1KPASsAdmK21jlZKvQpEaq2XAo8rpcYDFiANmO6seGq1tHg4vguu/XuFn5qakctjc3ewMS6VOy8L4ebwErthhBD1mFMrjGmtlwPLi217yebn54DnnBlDnRC9xHwPq9igqz2J6Tz4dRQpGbn8e3IfbhoQ7ITghBC1nZSarA2ivzfNQk3b2f2UBZEJPL9kL4ENvVn00OX0bCNrDAghSiaJwNWlxsGJ3XDt63Ydnmcp5LUf9/HV5iMM7ezPB1P707yBl5ODFELUZpIIXN0++5uFTp3N4eFvthN55DT3X9mRv14XKktOCiHKJYnA1UUvgeCB0LTsTt6oI2k89PV2zuVY+GBqP27o07qaAhRC1HbycdGVFTULhd1Y6iFaa77adJhbPt2Mr5c7Sx4ZKklACFEhckfgysppFsrJL+CFJXtZGJXIVaGBvHdLP5r4elZjgEKIukASgSuL/r7UZqGkM9k8+FUUe5LSeXxkF/48sgtuUk5aCFEJkghcVWocnNgD1/3jkl0bY1N4dO4O8i2FzJoWLmsNCyGqRBKBq4r+3ny3aRbSWjNrQzxvroihY2BDPrljAJ0CG9ZQgEKIukISgavatwSCB0ETMxs4K8/CXxfu5sfdxxnTsyX/mtyHht7y5xNCVJ28k7ii881CbwBwLiefu+ZsI+roaf46OpSHhndCKekPEEI4hiQCV2TTLHQmK49ps7ey79hZPpzan3G9W5X9XCGEqCBJBK4oegm0HUyKewC3f7qZ+ORMPr59ANdIp7AQwglkQpmrSYmFk3s423EcN3+yicOpmXw2PVySgBDCaeSOwNXsM81Cd21pzYmsHL68ezCDOjSv4aCEEHWZJAIXk7trMQdUKIdyGvPNfYPp27ZpTYckhKjjpGnIhfwRsxPv1H2s5HLm3j9EkoAQolpIInARe5PSWT7/vwDcdPtD9GgtC8kIIaqHJAIXsP3oaabO2sy1bCan1UA6dAqt6ZCEEPWI9BHUsM3xqdzz+Tb6NUihS/Zh6PNgTYckRJ2Un59PYmIiOTk5NR2KU/n4+BAcHIynp/2ViCUR1KB1B5N54KtIgpv58XHPo7CRCi9QL4SwT2JiIo0aNaJ9+/Z1dma+1prU1FQSExPp0KGD3c+TpqEa8su+k9z3RSQdAhoy//4hNIz9EdoOgcayqIwQzpCTk4O/v3+dTQIASin8/f0rfNfj1ESglBqtlDqglIpVSj1bxnGTlFJaKRXuzHhcxbJdx3jo6yi6t27MvPuG4J99BE5FQ4+JNR2aEHVaXU4CRSrzGp2WCJRS7sBMYAwQBkxVSoWVcFwjYAawxVmxuJKFUYnMmLeD/u2a8fU9g2ji52mzEtn4mg1OCFEvOfOOYBAQq7WO11rnAfOAkhrAXwPeAup2Dw7w9eYj/GXBLoZ2DuCLuwfRyMfamRO9BNpdJs1CQtRhZ86c4aOPPqrw88aOHcuZM2ecENEFzkwEbYAEm8eJ1m3nKaX6A2211j+VdSKl1P1KqUilVGRycrLjI3USrTWxp84xa308Uz/dzAtL9nJN9yBmTQvH18vdHJR80DQLlbFAvRCi9istEVgsljKft3z5cpo2de7k0hobNaSUcgPeAaaXd6zW+lPgU4Dw8HDt3MhKcWQj7JoL174OPo1LPSw7r4BN8SlExCQTceAUiaezAejWshF/vqYLD4/ojJeHTf7dtwRQ0iwkRDX627Jo9h0769BzhrVuzMs39Ch1/7PPPktcXBx9+/bF09MTHx8fmjVrRkxMDAcPHuTGG28kISGBnJwcZsyYwf333w9A+/btiYyMJCMjgzFjxjBs2DA2btxImzZt+OGHH/D19a1y7M5MBEmA7arrwdZtRRoBPYG11s6NlsBSpdR4rXWkE+OquJPR8O3NkHsWUuPh9kXg6XN+95HUTCJiThFxIJlN8ankWQrx83JnaOcAHh7RmRGhgbRuWsofK/p7aCejhYSo695880327t3Lzp07Wbt2LePGjWPv3r3nh3nOnj2b5s2bk52dzcCBA5k0aRL+/v4XnePQoUPMnTuXWbNmMWXKFBYtWsTtt99e5dicmQi2AV2UUh0wCeAW4NainVrrdCCg6LFSai3wF5dLAulJ8PVN4NUAhj8Dq56nYMFdbBrwLhGH0oiIOUV8SiYAHQMacPvgEK7uFsTADs3w9nAv+9zJB+DUPhjzz2p4IUKIImV9cq8ugwYNumis//vvv8/335vqwwkJCRw6dOiSRNChQwf69u0LwIABAzh8+LBDYnFaItBaW5RSjwIrAXdgttY6Win1KhCptV7qrGs7TE46fHMT5J4jZ9pyFic1Bf8T3HrwA47vy+ArHuKyjgFMuyyEEaFBtA9oULHzR1ubhbpLs5AQ9U2DBhfeL9auXcvq1avZtGkTfn5+jBgxosS5AN7e3ud/dnd3Jzs72yGxOLWPQGu9HFhebNtLpRw7wpmxVJglD+bfDikHybn5O+74MYNthxNo0/Rq5+8IVgAADXVJREFUWrTMZPKJ2dw4uBeeo1+Hyo5N3lc0WkiWnxSirmvUqBHnzp0rcV96ejrNmjXDz8+PmJgYNm/eXK2xSYmJkmgNSx+FP9aTd8NH3L2+AVFHUvnPLX0Z36c1iqtghQeeW2ZCwwC44smKX0OahYSoV/z9/Rk6dCg9e/bE19eXFi0urDo4evRoPv74Y7p3705oaChDhgyp1tgkEZRkzWuwez6WEc9z764ubIpP5u3JfZjQ12b06+i3ICsNfv0b+DWHAdMrdg1pFhKi3vn2229L3O7t7c2KFStK3FfUDxAQEMDevXvPb//LX/7isLgkERQXORs2vE1Bvzt58PAI1h9M5q1JvfhT/+CLj3Nzgxv/a/oRfnwCfJpCjwrMBYj+HkIul2YhIUSNk6Jztg78DD89RWHnUTx+9nZWxyTz2oQe3DywXcnHe3jBlC8heCAsvg/iIuy7zqkYSN4vk8iEEC5BEkGRpChYeBe6ZW+eVk/wU3QyL14fxh2XtS/7eV5+cOt88O8C826DxKjyryWTyIQQLkQSAUDaH/DtzegGgbzS6BUW7TnDM6O7cc8wO+t5+zaDOxZDgwD4ZpLpCC5L9BLTLNSoZdVjF0KIKpJEkJkKX09CF1p4J+gffLE7iyeu6cpDIzpV7DyNWsK0JeDmCV/eCGeOlnxcUbOQlJwWQriI+p0I8rNh7i3o9EQ+C/4HH+xWPDyiE4+P7Fy58zXvaO4M8jLhq4mQmXLpMftktJAQwrXU30RQWACL7kUnbmNR+5f5+54m3DusA09fF1q1xSta9jJ9BumJ8PWfIKdYYavo7yFkKDRqUfLzhRB1UmXLUAO89957ZGVlOTiiC+pnItAaVv4fxPxIRPsZ/CW6PXdeFsLz47o7ZgWjkMvMaKKT0TDvVsi3ThU/tR+SYyo2zFQIUSe4ciKon/MINs2ELR+zvfWt3B0ziKmD2vLyDT0cu4xd1+vMPIPF98Gie2DyFzKJTAhXseJZOLHHseds2QvGvFnqbtsy1KNGjSIoKIjvvvuO3NxcJk6cyN/+9jcyMzOZMmUKiYmJFBQU8OKLL3Ly5EmOHTvGVVddRUBAABERdg5Tr4D6lwj2LoZVzxMXeA2T4scyqX8wr9/YCzc3J6xl2nuKmX388zOwbAYkRUqzkBD1lG0Z6lWrVrFw4UK2bt2K1prx48ezfv16kpOTad26NT/9ZNbqSk9Pp0mTJrzzzjtEREQQEBBQzlUqp34lgiMb4fsHONGkH2MTbueGPsH886bezkkCRYY8CNlpsO4t83jsv513LSGEfcr45F4dVq1axapVq+jXrx8AGRkZHDp0iCuuuIKnnnqKZ555huuvv54rrriiWuKpP4kg+QDMnUq6T2uuO/kQV/dsxztT+uDuzCRQZMRzkH0Gds+TZiEhBFprnnvuOR544IFL9m3fvv3/27v7GCnqO47j7w9wekUMcIVa8HwAaxo0sUCvVHyKKZYnG7GNbRGxVJsQUkhK2l61kVLrX9WmNtoQ0NpLsdBKtGKJOaNIG1r/QB4ugA9YOChND5Czh4Fen8Vv/5jf4bC3e7fc3cwsO99XsrnZmd/sfPe3M/u7+c3s90dzczPLli1j2rRpLF9eNGHzgMrPxeLWTfzLBnPzsaV8asJ4Hpk7iSGDU3r7Esx+CL6117uFnMupeBrqGTNm0NTURGdnJwCHDh2ivb2dw4cPM3ToUObPn09jYyMtLS3d1k1Cbs4Inqudww9O1HHV5eNZccfk08cNTktseEvnXL7E01DPmjWLefPmMXXqVACGDRvGmjVraG1tpbGxkUGDBlFTU8PKlSsBWLhwITNnzmTs2LGJXCyWWTZjwfdVQ0ODbd9+5qNZbv3zMZ744wEevX0StTW9DCHpnKs6e/bsYcKECVmHkYpi71XSDjNrKFY+N2cEU8bVMWVcXdZhOOdcxcnPNQLnnHNFeUPgnMuNs60rvC/68h69IXDO5UJtbS0dHR1V3RiYGR0dHdTWntmNKYleI5A0E3gEGAw8YWY/LFi+CFgMnAQ6gYVm9maSMTnn8qm+vp62tjbeeeedrENJVG1tLfX19b0XjEmsIZA0GFgBfBZoA7ZJ2lDwRf8rM1sVyt8CPAzMTCom51x+1dTUMG5cmYNN5UySXUNTgFYzO2Bm/wWeAubEC5hZPEfzeUD1nrM551yFSrJr6ELgr7HnbcCnCwtJWgx8EzgH+EyxF5K0EFgIcPHFJQaSd8451yeZXyw2sxVmdhlwD7CsRJnHzazBzBpGjx6dboDOOVflkjwjOARcFHteH+aV8hSwsrcX3bFjx98k/aWPMY0CiowfWTE8vv7x+Pqv0mP0+PruklILkmwItgGXSxpH1ADMBebFC0i63Mz2hac3A/vohZn1+ZRA0vZSP7GuBB5f/3h8/VfpMXp8yUisITCz9yQtAV4kun20yczekPQAsN3MNgBLJN0E/A94F1iQVDzOOeeKS/R3BGbWDDQXzFsem/5Gktt3zjnXu8wvFqfs8awD6IXH1z8eX/9VeoweXwLOujTUzjnnBlbezgicc84V8IbAOedyriobAkkzJf1JUquke4ssP1fSurD8VUmXphjbRZJ+L+lNSW9I6nbBXNKNko5L2hkeyY9effr2D0p6LWy723Bwijwa6m+3pMkpxvbxWL3slHRC0tKCMqnXn6QmSe2SXo/Nq5O0UdK+8HdkiXUXhDL7JA34nXMlYvuRpLfC57de0ogS6/a4LyQc4/2SDsU+x9kl1u3xeE8wvnWx2A5K2lli3VTqsF/MrKoeRLeq7gfGE6Wt2AVcUVDm68CqMD0XWJdifGOAyWH6fGBvkfhuBJ7PsA4PAqN6WD4beAEQcDXwaoaf9dvAJVnXH3ADMBl4PTbvIeDeMH0v8GCR9eqAA+HvyDA9MoXYpgNDwvSDxWIrZ19IOMb7gW+XsQ/0eLwnFV/B8h8Dy7Osw/48qvGMoNdkd+H56jD9DDBNktIIzsyOmFlLmP47sIcoL9PZZA7wpEW2ACMkjckgjmnAfjPr6y/NB4yZ/QE4VjA7vp+tBm4tsuoMYKOZHTOzd4GNDHAG3mKxmdlLZvZeeLqF6Jf/mSlRf+Uo53jvt57iC98dXwJ+PdDbTUs1NgTFkt0VftGeKhMOhuPAh1OJLiZ0SU0CXi2yeKqkXZJekHRlqoFFWWBfkrQjJPwrVE4dp2EupQ++LOuvywVmdiRMvw1cUKRMJdTl3URneMX0ti8kbUnovmoq0bVWCfV3PXDUPsiSUCjrOuxVNTYEZwVJw4DfAEvt9HTcAC1E3R2fAH4KPJdyeNeZ2WRgFrBY0g0pb79Xks4BbgGeLrI46/rrxqI+goq7V1vSfcB7wNoSRbLcF1YClwETgSNE3S+V6HZ6Phuo+OOpGhuCcpLdnSojaQgwHOhIJbpomzVEjcBaM3u2cLmZnTCzzjDdDNRIGpVWfGZ2KPxtB9YTnX7HnWlCwSTMAlrM7GjhgqzrL+ZoV5dZ+NtepExmdSnpq8DngDtCQ9VNGftCYszsqJmdNLP3gZ+V2Ham+2L4/vgCsK5UmSzrsFzV2BCcSnYX/mucC2woKLOBD/Ia3Qb8rtSBMNBCf+LPgT1m9nCJMh/tumYhaQrR55RKQyXpPEnnd00TXVR8vaDYBuAr4e6hq4HjsS6QtJT8LyzL+isQ388WAL8tUuZFYLqkkaHrY3qYlyhFw8h+B7jFzP5Zokw5+0KSMcavO32+xLbLOd6TdBPwlpm1FVuYdR2WLeur1Uk8iO5q2Ut0N8F9Yd4DRDs9QC1Rl0IrsBUYn2Js1xF1EewGdobHbGARsCiUWQK8QXQHxBbgmhTjGx+2uyvE0FV/8fhENAzpfuA1oCHlz/c8oi/24bF5mdYfUaN0hCiBYhvwNaLrTpuIsuq+DNSFsg1EY3h3rXt32BdbgbtSiq2VqG+9ax/suotuLNDc076QYv39Muxfu4m+3McUxhiedzve04gvzP9F134XK5tJHfbn4SkmnHMu56qxa8g559wZ8IbAOedyzhsC55zLOW8InHMu57whcM65nPOGwLkUhcyoz2cdh3Nx3hA451zOeUPgXBGS5kvaGnLIPyZpsKROST9RNI7EJkmjQ9mJkrbEcvuPDPM/JunlkPyuRdJl4eWHSXomjAewNq3Mt86V4g2BcwUkTQC+DFxrZhOBk8AdRL9o3m5mVwKbge+HVZ4E7jGzq4h+Cds1fy2wwqLkd9cQ/TIVooyzS4EriH55em3ib8q5HgzJOgDnKtA04JPAtvDP+oeIEsa9zwfJxdYAz0oaDowws81h/mrg6ZBf5kIzWw9gZv8GCK+31UJumjCq1aXAK8m/LeeK84bAue4ErDaz7542U/peQbm+5mf5T2z6JH4cuox515Bz3W0CbpP0ETg19vAlRMfLbaHMPOAVMzsOvCvp+jD/TmCzRaPPtUm6NbzGuZKGpvounCuT/yfiXAEze1PSMqJRpQYRZZxcDPwDmBKWtRNdR4AoxfSq8EV/ALgrzL8TeEzSA+E1vpji23CubJ591LkySeo0s2FZx+HcQPOuIeecyzk/I3DOuZzzMwLnnMs5bwiccy7nvCFwzrmc84bAOedyzhsC55zLuf8DVw9/VltnPSQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hU1dbA4d9KJyGEQEIEUukkoYcakK4IggJKUVQELFx7u2K51k+v91ouFrBjQQERQUBQFATp0nvvhFBCIIEQAiTZ3x9nwIDpmckkzHqfZ57MzNlnz5pkMuucfXYRYwxKKaVcl5uzA1BKKeVcmgiUUsrFaSJQSikXp4lAKaVcnCYCpZRycZoIlFLKxWkiuEqJyD4R6ebsOABExIhIHWfHUVgi0kFEttu7bHkjIgtEZITt/u0i8mthyhbjdcJFJE1E3IsbqyoZTQSqzBCRoSKyuIR1vCQi35SkDmPMImNMfXuXLc+MMd8aY66zR11XHqQYYw4YYyoaY7LsUb8qOk0EyqWIRT/3SuWg/xAuQES8RWS0iCTabqNFxNu2LUhEfhKRFBE5ISKLLn5RisjTInJIRE6LyHYR6ZpH/V+KyEci8put7B8iEpFH2QAR+VpEkkRkv4g8LyJuItIQ+Ahoa2smSCnG++wBPAsMtNWx3vb8AhF5TUSWAOlALRG5W0S22uLdIyL35aink4gk5Hi8T0SeFJENIpIqIt+JiE9Ry9q2/1NEDtv+DiPyajYTkYEisuqK5x4TkRm2+z1FZIst/kMi8mQudXjb/q6xOZ4LFpGzIlJNRAJtf/skETlpux+ax+/2srM1EekuItts7/EDQHJsqy0iv4tIsogcF5FvRaSybdt4IByYafsb/VNEIm2/Bw9bmRoiMsP2edwlIvfkqPslEZls+wydFpHNIhKXW8yqCIwxersKb8A+oJvt/ivAcqAaEAwsBV61bfs31hewp+3WAeufuj5wEKhhKxcJ1M7jtb4ETgPXAt7Au8DiHNsNUMd2/2tgOuBvq3MHMNy2bWjO/Yr5vl8CvrniuQXAASAG8LC9z15Abdt77YiVIJrbyncCEq74Xa4AagBVgK3A/cUo2wM4YovDF/gm5+/miph9bb/TujmeWwkMst0/DHSw3Q+8GHsu9YwDXsvx+AHgF9v9qkB/22v5A98DP17xextx5d8GCLLFdovtd/kYkJmjbB2gu+2zEAwsBEbn9tnM8dkygIft8UJgLOADNAWSgC45/r4ZQE/AHevzu9zZ/2/l/aZnBK7hduAVY8wxY0wS8DJwh23bBaA6EGGMuWCsNm8DZGH9I0eLiKcxZp8xZnc+rzHLGLPQGHMOeA7ryD4sZwGxLgYOAp4xxpw2xuwD3s4RiyN9aYzZbIzJtL3PWcaY3cbyB/ArVhLMy3vGmERjzAlgJtYXVFHLDgC+sMWRjvWllivb9unAYAARqQs0AGbYilzA+ttUMsacNMasyaOqCVi/84tusz2HMSbZGPODMSbdGHMaeA0rKRakJ7DZGDPFGHMBGI2V4C7GvssY85sx5pzt8/ZOIevF9pmJB542xmQYY9YBnwF35ii22Bgz21jXFMYDTQpTt8qbJgLXUAPYn+PxfttzAG8Cu4BfbU0ko8D6ZwYexfqyOiYik0SkBnk7ePGOMSYNOJHjNS4KwjqCvDKWmoV5E2L10Emz3TYXZp/c4rPVdYOILLc1P6RgfbkF5bP/kRz304GKxShb44o4LospFxOwJQKsL/AfbQkCrCP5nsB+W1Nc2zzqmA/4ikhrEYnESkrTAETEV0Q+tjXRncI6Eq8sBffeuex92A4cLj0WkRDb5+WQrd5vyP93e2XdJ2yJ6aIrPyNX/n59LjYrqeLRROAaEoGcbfbhtuewHZk/YYypBfQBHhfbtQBjzARjTHvbvgb4Tz6vcenoX0QqYjWLJF5R5jjWkeyVsRyy3c93Klzb2UpF2y0mr2IFPS/W9ZEfgLeAEGNMZWA2Odq5HeQwkLMNPiyvgja/AcEi0hQrIUy4uMEYs9IYcxNWc9+PwOTcKrAdNU+27T8Y+CnHl+wTWE2ArY0xlbCa9qDg38NhLv97yxXv5XWs33cjW71Drqgzv79zIlBFRPxzPJfzM6IcQBOBa5gIPG+7UBgEvIB1lIaI3CgidWz/zKlYTULZIlJfRLrYvjQzgLNAdj6v0VNE2ouIF/AqVrvtZUe8Ob6UXhMRf7EuKD9+MRbgKBBqq6O4jgKRkn/PIC+sZq8kIFNEbgDs0jWyAJOBu0WkoYj4Av/Kr7Ct2eV7rLO2KliJARHxEqtff4CtzCny/9tMAAZiNRFOyPG8P9bfNUVEqgAvFvJ9zAJiRKSf7Uj8YeCaK+pNA1JFpCbw1BX7HwVq5Vax7TOzFPi3iPiISGNgOH99RpQDaCJwDf8HrAI2ABuBNbbnAOoCc7H+cZcBY40x87G+KN/AOoo/gnXk+Uw+rzEB64vkBNAC6ygwNw8BZ4A9wGLbfuNs234HNgNHROR4Ud+kzfe2n8kikmu7ue2I+GGsL+aTWM0uM3Ira0/GmJ+B97Caa3ZhXcAHOJfPbhOAbsD3xpjMHM/fAeyzNb3cj/Uln9fr/on1O68B/Jxj02igAtbfeDnwSyHfx3HgVqzPRzLWZ2hJjiIvA82xDixmAVOvqOLfWAcmKbn1dsI6c4nEOjuYBrxojJlbmNhU8YjVvKdU8YnIl1g9Z553dizliVhdZjcB3ld8yStVqvSMQKlSJCJ9bf37A7GuuczUJKCcTROBUqXrPuAYsBvresxI54ajlDYNKaWUy9MzAqWUcnHlbhBGUFCQiYyMdHYYSilVrqxevfq4MSY4t23lLhFERkayatWqggsqpZS6RET257VNm4aUUsrFaSJQSikXp4lAKaVcXLm7RqCUKn0XLlwgISGBjIwMZ4eiCuDj40NoaCienp6F3kcTgVKqQAkJCfj7+xMZGYk1P6Eqi4wxJCcnk5CQQFRUVKH306YhpVSBMjIyqFq1qiaBMk5EqFq1apHP3DQRKKUKRZNA+VCcv5PLJILdSWm8PHMzF7Lym7ZdKaVcj8skgv3JZ/hiyT5mbzzs7FCUUkWUkpLC2LFji7Vvz549SUlJybfMCy+8wNy59lnyIDIykuPHi7uchnO4TCLoVK8atYL8GLd4LzrRnlLlS36JIDMz/1m8Z8+eTeXKlfMt88orr9CtW7dix1feuUwicHMT7o6PZH1CKmsOnHR2OEqpIhg1ahS7d++madOmPPXUUyxYsIAOHTrQp08foqOjAbj55ptp0aIFMTExfPLJJ5f2vXiEvm/fPho2bMg999xDTEwM1113HWfPngVg6NChTJky5VL5F198kebNm9OoUSO2bdsGQFJSEt27dycmJoYRI0YQERFR4JH/O++8Q2xsLLGxsYwePRqAM2fO0KtXL5o0aUJsbCzffffdpfcYHR1N48aNefLJ3BZucxyX6j7ar3kob87ZzrjF+2gRUcXZ4ShVLr08czNbEk/Ztc7oGpV4sXdMntvfeOMNNm3axLp16wBYsGABa9asYdOmTZe6SY4bN44qVapw9uxZWrZsSf/+/alatepl9ezcuZOJEyfy6aefMmDAAH744QeGDPn7qqpBQUGsWbOGsWPH8tZbb/HZZ5/x8ssv06VLF5555hl++eUXPv/883zf0+rVq/niiy/4888/McbQunVrOnbsyJ49e6hRowazZs0CIDU1leTkZKZNm8a2bdsQkQKbsuzNZc4IAPy8PRjcKpyfNx0m4WS6s8NRSpVAq1atLusr/95779GkSRPatGnDwYMH2blz59/2iYqKomnTpgC0aNGCffv25Vp3v379/lZm8eLFDBo0CIAePXoQGBiYb3yLFy+mb9+++Pn5UbFiRfr168eiRYto1KgRv/32G08//TSLFi0iICCAgIAAfHx8GD58OFOnTsXX17eov44ScakzAoA720Xy2eK9jF+2n2d6NnR2OEqVO/kduZcmPz+/S/cXLFjA3LlzWbZsGb6+vnTq1CnXvvTe3t6X7ru7u19qGsqrnLu7e4HXIIqqXr16rFmzhtmzZ/P888/TtWtXXnjhBVasWMG8efOYMmUKH3zwAb///rtdXzc/LnVGAFCzcgV6xFzDxBUHOHNOl4pVqjzw9/fn9OnTeW5PTU0lMDAQX19ftm3bxvLly+0eQ3x8PJMnTwbg119/5eTJ/K81dujQgR9//JH09HTOnDnDtGnT6NChA4mJifj6+jJkyBCeeuop1qxZQ1paGqmpqfTs2ZP//e9/rF+/3u7x58flzggAhrWPZNbGw0xdk8AdbSOdHY5SqgBVq1YlPj6e2NhYbrjhBnr16nXZ9h49evDRRx/RsGFD6tevT5s2bewew4svvsjgwYMZP348bdu25ZprrsHf3z/P8s2bN2fo0KG0atUKgBEjRtCsWTPmzJnDU089hZubG56ennz44YecPn2am266iYyMDIwxvPPOO3aPPz/lbs3iuLg4U9KFaYwx3Dx2KafOXmDe4x1xc9MRk0rlZ+vWrTRs6NpNqefOncPd3R0PDw+WLVvGyJEjL128Lmty+3uJyGpjTFxu5V3yjEBEGBYfySOT1rFgxzG6NAhxdkhKqTLuwIEDDBgwgOzsbLy8vPj000+dHZLduGQiAOjZqDqvz97KuMX7NBEopQpUt25d1q5d6+wwHMLlLhZf5Onuxp1tI1m86zjbj+R9EUoppa52LpsIAG5rFY63hxtfLNnr7FCUUsppXDoRBPp50a95KFPXHiI57Zyzw1FKKadw6UQAMCw+kvOZ2UxcccDZoSillFO4fCKoG+LPtfWC+XrZfs5n6loFSl0tKlasCEBiYiK33HJLrmU6depEQd3RR48eTXr6X1PSFGZa68J46aWXeOutt0pcjz24fCIA66zg2OlzzNqY6OxQlFJ2VqNGjUszixbHlYmgMNNalzeaCIBr6wZTO9iPz3WtAqXKpFGjRjFmzJhLjy8eTaelpdG1a9dLU0ZPnz79b/vu27eP2NhYAM6ePcugQYNo2LAhffv2vWyuoZEjRxIXF0dMTAwvvvgiYE1kl5iYSOfOnencuTNw+cIzuU0znd9013lZt24dbdq0oXHjxvTt2/fS9BXvvffepampL05498cff9C0aVOaNm1Ks2bN8p16o7AcNo5ARMYBNwLHjDGxuWy/HXgaEOA0MNIYU7oTbNhYaxVE8fyPm1i1/yQtI3WKaqXy9PMoOLLRvnVe0whueCPPzQMHDuTRRx/lgQceAGDy5MnMmTMHHx8fpk2bRqVKlTh+/Dht2rShT58+ea7b++GHH+Lr68vWrVvZsGEDzZs3v7Tttddeo0qVKmRlZdG1a1c2bNjAww8/zDvvvMP8+fMJCgq6rK68ppkODAws9HTXF9155528//77dOzYkRdeeIGXX36Z0aNH88Ybb7B37168vb0vNUe99dZbjBkzhvj4eNLS0vDx8Sn0rzkvjjwj+BLokc/2vUBHY0wj4FXgk3zKOly/5jUJqODJuMXalVSpsqZZs2YcO3aMxMRE1q9fT2BgIGFhYRhjePbZZ2ncuDHdunXj0KFDHD16NM96Fi5ceOkLuXHjxjRu3PjStsmTJ9O8eXOaNWvG5s2b2bJlS74x5TXNNBR+umuwJsxLSUmhY8eOANx1110sXLjwUoy3334733zzDR4e1nF7fHw8jz/+OO+99x4pKSmXni8Jh50RGGMWikhkPtuX5ni4HAh1VCyF4etlrVXwycLdHDyRTliV0p0PXKlyI58jd0e69dZbmTJlCkeOHGHgwIEAfPvttyQlJbF69Wo8PT2JjIzMdfrpguzdu5e33nqLlStXEhgYyNChQ4tVz0WFne66ILNmzWLhwoXMnDmT1157jY0bNzJq1Ch69erF7NmziY+PZ86cOTRo0KDYsULZuUYwHPg5r40icq+IrBKRVUlJSQ4L4q52EYgIXy/b57DXUEoVz8CBA5k0aRJTpkzh1ltvBayj6WrVquHp6cn8+fPZv39/vnVce+21TJgwAYBNmzaxYcMGAE6dOoWfnx8BAQEcPXqUn3/+6+sorymw85pmuqgCAgIIDAy8dDYxfvx4OnbsSHZ2NgcPHqRz58785z//ITU1lbS0NHbv3k2jRo14+umnadmy5aWlNEvC6XMNiUhnrETQPq8yxphPsDUdxcXFOexqbvWACvRsVJ1JKw7ySLd6VPR2+q9HKWUTExPD6dOnqVmzJtWrVwfg9ttvp3fv3jRq1Ii4uLgCj4xHjhzJ3XffTcOGDWnYsCEtWrQAoEmTJjRr1owGDRoQFhZGfHz8pX3uvfdeevToQY0aNZg/f/6l5/OaZjq/ZqC8fPXVV9x///2kp6dTq1YtvvjiC7KyshgyZAipqakYY3j44YepXLky//rXv5g/fz5ubm7ExMRwww03FPn1ruTQaahtTUM/5Xax2La9MTANuMEYs6MwddpjGur8rD1wkr5jl/JS72iGxkcVvINSLkCnoS5fijoNtdOahkQkHJgK3FHYJFAamoUH0iy8Ml8s3Ud2tnYlVUpd/RyWCERkIrAMqC8iCSIyXETuF5H7bUVeAKoCY0VknYg47jC/iIbFR7E/OZ3ftx1zdihKKeVwjuw1NLiA7SOAEY56/ZLoEXsN1QN8GLdkL92ida0CpcBa2S+v/vmq7ChOc39Z6TVUpni6u3FXu0iW7k5m6+FTzg5HKafz8fEhOTlZR96XccYYkpOTizzITLvF5GFQyzDenbuTcYv38uatTZwdjlJOFRoaSkJCAo7svq3sw8fHh9DQog3L0kSQh8q+XvRvUZPJKxN4+oYGBFX0Lngnpa5Snp6eREVpL7qrlTYN5WNouyjOZ2Xz7XJdq0ApdfXSRJCPOtUq0ql+MOOX7+dcZpazw1FKKYfQRFCAYfFRHE87x0/rDzs7FKWUcghNBAXoUDeIutUqMm6JrlWglLo6aSIogIgwrH0UmxNPsWLvCWeHo5RSdqeJoBD6NqtJoK8nn+taBUqpq5B2Hy0EH093bmsdztgFu7lpzBLqVqtInWoVqRNs/Qyr4ou7m464VEqVT66TCM4kw4k9UDkcKlaDIg6Vv79jbc5nZrM58RQLdyQxZXXCpW1eHm7UCvKzkkOOW1SQH94e7vZ+J0opZVeukwj2/gFT7rbue/hAQJiVFC67ReSZKPx9PHmuV/Slx6lnL7A7KY1dR9PYlZTGrmNpbEhIZdbGw1jXlA1+co7oQENMYBZ1K2URVsWXBq2up1pAhdJ730opVQDXSQSRHeC2yZByAFL2234egMPrID358rJ5JYqKIXD+DGSkEHD2JM3PptA8IwUyToJnClQ5SXaFk2Slp+CecRI3kwnpWLdDVtXTF7bj62pP0aFhKN0ahhBTo5JO5KWUciqHLkzjCA5ZmOZcGqQe/Cs55EwUKQf+nihy8q4EFSqDT2XrZ4XAXO4HkuUdQPK2RVRb9TZbPaMZkvYwyaYSIZW86dKgGl0bhBBfJ4gKXtqUpJSyv/wWptFEUBjnTkPKQUg7+tcXf4VA6757EU+qNk+DafeT5RfC3GYfMP2QHwt3HCftXCbeHm60q12Vrg1D6NKgGjUqaxOSUso+NBGUNQmrYOIgyDoPA8ZzPrwDK/aeYN62o8zbeowDJ9IBiK5eia4Nq9GlQTWahFbGTXsmKaWKSRNBWXRyP0wYCMk74cbR0PwOwJpPfHdSGvO2HmPetmOs2neCbANBFb3o0qAat7WOoGlYZScHr5QqbzQRlFUZqfD9UNj9O8Q/Cl1fBLfLx/ilpJ/njx1JzNt6jN+3HSPtXCbNwyszvH0tro8JwcNdxwQqpQqmiaAsy8qEn5+CVeOgYR/o+zF4+eZaNO1cJlNWHeSLpfvYn5xOjQAf7moXyaBW4QRU8CzlwJVS5YkmgrLOGFg+FuY8BzWbw6CJ4J/3WslZ2YZ5W48ybslelu85ga+XO7e0COXu+CiigvxKMXClVHmhiaC82DYLfhgBvlWtMQ8h0QXusulQKl8s2cfM9YlcyM6mS/1qDGsfRbvaVXV8glLqEk0E5UniOqtH0bk0GPAl1OlWqN2Onc7gm+UH+Hb5fpLPnKfBNf4Mi4+iT9Ma+Hjq2ASlXJ0mgvIm9ZDVo+jYFuj5X2g5otC7ZlzIYsa6RMYt2cu2I6ep6ufF7W0iuKNNBMH+uu6yUq5KE0F5dO40TBkOO+dAmwfgulfBrfBH9sYYlu5OZtzivczbdgwvdzf6NK3Bk9fV55oAHwcGrpQqi/JLBA7reygi40TkmIhsymO7iMh7IrJLRDaISHNHxVIuefvD4InQ+n5YPga+G2I1FxWSiBBfJ4jPh7bk9yc6MqhVGDPXJ9L9nT+YuOKArramlLrEYWcEInItkAZ8bYyJzWV7T+AhoCfQGnjXGNO6oHpd5owgpz8/gV+ehuCGENsPqjWE4AYQGFmks4R9x88wauoGlu85QdtaVXmjfyMiqmovI6VcgdOahkQkEvgpj0TwMbDAGDPR9ng70MkYk+8q8S6ZCAB2/GqNNzi576/nPHwgqK6VFIIbFCpBZGcbJq08yL9nb+VCdjZPXlefu+OjdGEdpa5y+SUCZ05DXRM4mONxgu25vyUCEbkXuBcgPDy8VIIrc+pdZ93OnYak7ZC0DY5ttX4eWA4bv/+r7KUE0RCC61+WINzcrNXWOjcI5vlpm/i/WVuZueEwb97SmHoh/s57f0opp3HmGcFPwBvGmMW2x/OAp40x+R7uu+wZQUFySxBJ263ptS/yrQrDfoWgOoB1QXnG+kRenrmF0xkXeLBzXUZ2qo2Xh05bodTVpqyeERwCwnI8DuXS8i2qyLz9ITTOuuWUcQqO74Cjm+Gnx2DteOj+MmBdUL6paU3a1wnipZlb+N/cHfy86TD/6d+YJjqxnVIuw5mHfjOAO229h9oAqQVdH1DF4FPJSg4t7oLanWHzVLjiLLBqRW/eH9yMT++M42T6efqOXcK/Z2/l7PksJwWtlCpNjuw+OhFYBtQXkQQRGS4i94vI/bYis4E9wC7gU+AfjopF2cT2t1ZcO7Q6183do0P49bGODGwZxscL93DDuwtZvief1dmUUlcFHVDmSjJS4c060PIe6PF6vkWX7jrOqKkbOXAindtbhzPqhgb4++gMp0qVV04ZUKbKIJ8AqNPdah7Kzs63aLs6QfzyaAeGt49iwooDXPe/hczfdqyUAlVKlSZNBK4mth+cPgwHlhVY1NfLg3/dGM0PI9tR0duDu79cyfR1ej1fqauNJgJXU68HeFSwzgoKqXl4ID893J64iECem7aJA8npDgxQKVXaNBG4Gu+KUL8HbP7RWh2tsLt5uDN6UFNE4OFJa7mQlX/TklKq/NBE4Ipi+kH6cdi3sEi7hQb68u9+jVh3MIXRc3c4KDilVGnTROCK6nYHL3/YVPjmoYtubFyDAXGhjF2wm6W7jzsgOKVUadNE4Io8K0CDXrB1BmSeL/LuL/WJIaqqH49/t56TZ4q+v1KqbNFE4Kpi+1njCvbML/Kuvl4evDe4GclnzvH0Dxt0bQOlyjlNBK6qVmfwqQybfijW7rE1A3i6RwN+3XKUb/88YOfglFKlSROBq/Lwgug+sG0WXDhbrCqGxUdxbb1gXv1pCzuOnrZzgEqp0qKJwJXF9IPzabDzt2Lt7uYmvH1rE/x9PHh44loyLugkdUqVR5oIXFlkB/ALLnbzEECwvzdv3dqEbUdO8+/ZW+0YnFKqtGgicGXuHhB9M+yYA+fSil1Np/rVGN4+iq+W7WfulqN2DFApVRo0Ebi62H6QeRZ2/FKiav7Zoz7R1Svx1JT1HD2VYafglFKlQROBqwtrA/41StQ8BNYUFO8NbkbGhWwen7yO7GztUqpUeaGJwNW5uUFMX9g1F86mlKiqOtUq8mLvaJbsSuaTRXvsFKBSytE0EShr5bKs81ZX0hIa2DKMno2u4a0521l/sGSJJacz5zJZvPM4WXqmoZTdaSJQULM5VI4ocfMQgIjw776NCankw8OT1pJ2rvAznOZmc2Iqz/+4kdavz2PI53/y+WI901DK3jQRKBCxLhrvWQBnSr5GcYCvJ6MHNeXgiXRemL6pyPufOZfJpBUHuOmDxfR6bzHfr0rgupgQWkQEMnbBbk5lXChxjEqpv2giUJbY/mCyYOt0u1TXMrIKD3Wpy9Q1hwq9qtmmQ6k8O20jrV6by6ipGzl7IYsXe0fz57NdeWdAU17qHUNK+gU+W6hnBUrZk4ezA1BlREgsBNWzpqaOG2aXKh/qUoclu47z3LRNNAsLJLyq79/KpJ3LZOb6RCauOMCGhFS8Pdy4sXENbmsdRvPwQETkUtlGoQH0alSdzxbv5Y62kQT7e9slTqVcnZ4RKIuINeXEvsVw+ohdqvRwd8tzVbOLR/+tX5vLM1M3cu5CNi/1jmbFs914e0ATWkRUuSwJXPT4dfU4l5nNmPm77BKjUkrPCFROsf3gjzesZSzb3G+XKkMDfXmjX2MemLCG//6yjaigikxccYCNh1Lx8bSO/ge3Cqd5eOVcv/ivVDu4IgPiQvn2z/0Mbx9FWJW/n2UopYpGE4H6S3B9CGlk9R6yUyIA6NW4Ogt3hPHpor0A1A/x5+U+MdzcrCYBFTyLXN/DXevyw5pDjJ67k7cHNLFbnEq5KocmAhHpAbwLuAOfGWPeuGJ7OPAVUNlWZpQxZrYjY1IFiO0L816BlANQOdxu1b7YJ5rIID9a16pCs7DCHf3npXpABYa2i+TTRXu4r2Mt6oX42y1OpVyRw64RiIg7MAa4AYgGBotI9BXFngcmG2OaAYOAsY6KRxVSTD/r5+Zpdq3W18uDkZ1q/+0CcHGN7Fibil4evDVnux2iU8q1OfJicStglzFmjzHmPDAJuOmKMgaoZLsfACQ6MB5VGFWioEbzYi1sX5oC/by499pa/LrlKGsOnHR2OEqVa45MBDWBgzkeJ9iey+klYIiIJACzgYdyq0hE7hWRVSKyKikpyRGxqpxi+8PhdZC829mR5GtY+yiCKnrx31+26brJSpWAs7uPDga+NMaEAj2B8SLyt5iMMZ8YY+KMMXHBwcGlHqTLielr/SzjZwV+3h482LkOy/ecYNHO484OR6lyy5GJ4BAQloQTe30AACAASURBVONxqO25nIYDkwGMMcsAHyDIgTGpwgioCeFtYXPZTgQAg1uHU7NyBd6cs12nvlaqmByZCFYCdUUkSkS8sC4Gz7iizAGgK4CINMRKBNr2UxbE9odjW+DoFmdHki9vD3ce716PjYdS+XmTfQbCKeVqHJYIjDGZwIPAHGArVu+gzSLyioj0sRV7ArhHRNYDE4GhRht7y4bom0DcysVZwc3NalIvpCJv/7qdzByjl5VShePQawTGmNnGmHrGmNrGmNdsz71gjJlhu7/FGBNvjGlijGlqjPnVkfGoIqhYzVrcftNUKOO52d1NePK6+uw5foYpqxOcHY5S5U6hEoGIPCIilcTyuYisEZHrHB2ccrLY/nBiNxxe7+xICtQ9OoRm4ZUZPXcnGReynB2OUuVKYc8IhhljTgHXAYHAHcAb+e+iyr2GvcHNo1w0D4kI/7y+AUdOZTB+2X5nh6NUuVLYRHBxKGhPYLwxZnOO59TVyrcK1O5SLpqHANrWrsq19YIZs2CXLl6jVBEUNhGsFpFfsRLBHBHxB/SqnCuI7Q+pByFhpbMjKZR/Xl9fF69RqogKmwiGA6OAlsaYdMATuNthUamyo35PcPcu84PLLoqtGUCvxtbiNUmnzzk7HKXKhcImgrbAdmNMiogMwZosLtVxYakyw6cS1O1uTUKXXT4uwj7RXRevUaooCpsIPgTSRaQJVt//3cDXDotKlS2x/SHtCOxf6uxICqVWjsVrDp5Id3Y4SpV5hU0EmbaBXjcBHxhjxgA6CbyrqHc9ePqWi95DFz3ctS5uIoyeu9PZoShV5hU2EZwWkWewuo3Osk0MV/SlpVT55OUH9W+ALdMhK9PZ0RRK9YAK3NUukqlrE9hx9LSzw1GqTCtsIhgInMMaT3AEawK5Nx0WlSp7Ym+B9GRrTeNy0JUUdPEapQqrUInA9uX/LRAgIjcCGcYYvUbgSur1gGZDYOGbMPelcpEMdPEapQqnsFNMDABWALcCA4A/ReQWRwamyhg3N+j9PsQNgyWj4ZdnykUy0MVrlCpYYRevfw5rDMExABEJBuYCUxwVmCqD3Nyg1zvWuII/P4Ss89DzLev5Muri4jUvzdzCop3HubaeLmyk1JUK+x/sdjEJ2CQXYV91NRGBHv+G+Edg1ecw86EyP75gcOtwQgMr8N8523TxGqVyUdgv819EZI6IDBWRocAsrDWGlSsSgW4vQ8enYe03MO3+Mt2byNvDnce61WPToVPc+P5iPlm4myOpGc4OS6kyQwrbbioi/YF428NFxphpDosqH3FxcWbVqlXOeGmVm4Vvwe+vQvTN0P8zcC+bvYqzsw3frjjAlFUHWZ+Qigi0jqrCzU1rckNsdQJ8HRz32ZMw+U64/nW4ppFjX0upXIjIamNMXK7bytsFNE0EZdDS9+HX56F+L7j1C/DwdnZE+dp7/AzT1x1i+rpE9h4/g5e7G53qB3NT05p0bVgNH093+7/o6i9h5iMQ/yh0f9n+9StVgPwSQb4Xi0XkNJBbphDAGGMq2SE+Vd61ewg8fGD2kzDpdhg4HjwrODuqPEUF+fFot3o80rUuGw+lMn1dIjPXJ/LrlqNU9Pbg+phruLlZDdrVDsLdzU6zrV+ctG//EvvUp5Qd6RmBsp/VX8LMR6FWRxg0wRqRXE5kZRuW70nmx7WH+GXTEU6fyySooje9m1Tn5qY1aRwagEgxk0JaErxdDzz9IPMsjDpQrn436uqgTUOq9KybANMfgPC2cNt34F3+pqTKuJDF/G3HmL4ukd+3HeN8VjZRQX70aVKD/s1DCa/qW7QKV34Gs56A7q/Aby/AHT9C7c6OCV6pPGgiUKVr4xSYei/UbAFDpoBPgLMjKrbUsxeYs+kIP647xLI9yRgD8XWqMrBlONdFhxTuesIXveBMEoyYC/+JgA5PQJfnHR+8UjkU+xqBUsXS6BZw94Ipw+Drm2DIVGvZy3IooIInA1qGMaBlGIkpZ5myOoHvVh7k4YlrqezrSd9mNRnYMowG1+RxuezUYeu6QKdR1toO1ZvAPr1OoMoWHRSmHCO6Dwz8Bo5uhq/6wJnjzo6oxGpUrsDDXeuy6J+dGT+8Fe3rBPHt8gP0GL2Im8csYdKKA6Sdu2I8xdYZgIGYvtbjiHg4tAou6DgGVXZoIlCOU78HDJ4EyTvhy15w+qizI7ILNzehQ91gPritOcuf7cq/bowm/Xwmo6ZupNVrc/nnlPWs3n/Smtto01SoFgPB9a2dI+KtqTkOafOmKjsc2jQkIj2AdwF34DNjzBu5lBkAvITVTXW9MeY2R8akSlmdrnD79zBhIIy7HiLbWyOTARDb/bx+XlHG08fqh1+hsjPeSa6q+HkxvH0Uw+IjWXswhckrDzJjfSKTVyXQLugsE9KWk97+GS5dXo5oC4jVPBTZ3omRK/UXhyUCEXEHxgDdgQRgpYjMMMZsyVGmLvAMEG+MOSki1RwVj3KiqGut6wQzH4Fdc3PMWmps93P8hL8/d7H4uVRw84Quz5X2OyiQiNA8PJDm4YE8f2M0szYkkv7HuwD0mR9M/aNrGNgyjA51g5CQWB1PoMoUR54RtAJ2GWP2AIjIJKylLrfkKHMPMMYYcxLgiont1NUkoi08uKJkdUwYZE101+HxMj1graK3BwNbhsO6dWT4xHJtRFumrk1g1sbDtIqqwsfV4gjcOgkyz4OHl7PDVcqh1whqAgdzPE6wPZdTPaCeiCwRkeW2pqS/EZF7RWSViKxKSkpyULiqzGv7D2uVtI3fOzuSgp3cD4dW4dP0Fl7oHc2fz3bl9b6N2HH0NM+vCYDMs2Qc0OsEqmxw9sViD6Au0AkYDHwqIn9rADbGfGKMiTPGxAUH63zyLiuyA4Q0gmVjy/6iOJttczLaegt5e7hzW+tw5j3ekeDYLgB8OeFbfttydVxAV+WbIxPBISAsx+NQ23M5JQAzjDEXjDF7gR1YiUGpvxOxzgqStsKe+c6OJn+bp1oD6gIjL3u6akVvXhrckbOV69KCLdzz9Sru+XoVh1LOOidOpXBsIlgJ1BWRKBHxAgYBM64o8yPW2QAiEoTVVLTHgTGp8i62P/hVs84Kyqrk3XB4/V9jB3JRoU4H4tx28Oz1dVi88zjd3v6Dj/7YzYWs7FIMVCmLwxKBMSYTeBCYA2wFJhtjNovIKyLSx1ZsDpAsIluA+cBTxphkR8WkrgIe3tByBOz6DZK2Ozua3F3RLJSriHjkfBr31ktj7hMd6VA3iDd+3kav9xaxYu+J0olTKRuHXiMwxsw2xtQzxtQ2xrxme+4FY8wM231jjHncGBNtjGlkjJnkyHjUVSJumLVu8vIPnR1J7jZPg7DWEBCad5kI2xpP+5dSs3IFPrkzjs/ujOPMuSwGfLyMJ79fT3LaudKJV7k8Z18sVqroKgZD4wGwfhKkl7Gj56QdcHQTxPTLv1yl6lCl1mXzDnWLDuG3x69lZKfa/Lj2EF3f+YNJKw7oOsvK4TQRqPKpzT+suf1Xf+HsSC63eSogEH1TwWUj4uHAUsjOuvSUr5cHT/dowM+PdKB+iD+jpm7klo+WsiXxlONiVi5PE4Eqn0KioVZnWPGpNTCrrNg8DSLaWUf8BYlsDxmpcGzL3zbVDfFn0r1tePvWJuxPTqf3B4t59actnD2flUtFSpWMJgJVfrV9AE4fhi0/OjsSy9EtkLQt/4vEOUW0s37mMS21iNC/RSjznujIwJZhjFuyl+FfrSTjgiYDZV+aCFT5VbsrVK0Ly8aUjQFmm6eCuBWuWQigcjgEhMP+xfkX8/Xi9b6NeOuWJizbk8x941dzLlOTgbIfTQSq/HJzgzYj4fA6OLDMubFcnHI6sj1ULMLciZHxsH9poRJZ/xahvNGvEX/sSOKBb9dwPlPHHCj70ESgyrcmg6FCICx38gCzIxvhxO6CewtdKaKdNX9SIcdEDGwZzqs3xTB36zEembSWTB2ApuxAE4Eq37x8ocXdsG0WnNznvDg2TwVxh4Z9Ci6b06XxBPk3D+V0R9tI/nVjND9vOsJjk9eTpd1LVQlpIlDlX6t7rLb5Pz92zutfbBaq1Qn8qhZt3yq1wL+61TxUBMPbR/F0jwbMXJ/IP6ds0LEGqkQ0Eajyr1INq0lmzXjIcEJ/+8Q1kLK/8L2FchKxmof2LSnyBe+RnWrzWLd6/LAmgWenbdRkoIpNE4G6OrQZCedPw9rxpf/am6ZaK6c1vLF4+0fEQ9oROFH0+RYf7lqHBzvXYdLKg7w4Y7O1TrJSRaSJQF0dajaH8Lbw50eXjdR1OGNg849Qu4t10bo4Lq5dvK/w1wkuEhGeuK4e915bi/HL9/N/s7ZqMlBFpolAXT3a/ANSDlgXjktLwko4lQCxRewtlFNQPfANKvJ1gotEhGduaMDQdpF8vngv/52zXZOBKhJHrlmsVOlq0AsqR1hdSaOL2HunuDZNBXcvqH9D8eu4eJ2gBAvaiwgv9o7mfFY2Hy7YjZe7G491r1f8mJRL0TMCdfVwc4fW91uDyw6tcfzrZWdb01vU6Q4+ASWrK7I9pB601jouJhHh/26K5dYWobw7bydj5u8qWUzKZWgiUFeXZkPAy790BpgdXG7NdVSSZqGLcqxPUBJubsIb/Rtzc9MavDlnO58tKtmCf2fOZbJoZxJv/7qdu8at4H+/7Sj+sprpJ8rGVCDqb7RpSF1dfCpB8zthxcfQ/RWra6mjbJoKHj5Qr0fJ66oWDT6VrYFlTQeXqCp3N+GtW5twIcvwf7O24unuxl3tIgu1b2r6BVbuO8GKfSf4c+8JNh1KJSvb4O4mRFb1ZeHOJN77fSed6gUzqFU4XRpUw9O9EMeT+5fBV72h0S1w84dWc5gqMzQRqKtP63vhzw9hxSfQ7SXHvEZ2FmyZDnWvA++KJa/Pze2v8QR24OHuxuhBTbmQlc2LMzbj6e7Gba3D/1Yu6fQ564t/7wmW70lm+9HTGANe7m40CQvg/o61aBVVlRYRgVT09uDgiXQmrzrI5FUHuW/8aoL9vRkQF8rAuHDCq/rmHszpo/D9UPCsAOsnWgPour1ol/ep7EMTgbr6BEZaF45XfQHX/tOahsLe9i2GM8fs0yx0UUQ8bJ8NpxLtcibj6e7G+7c14/7xq3l22kY83YV2dYJYsTeZFXutI/49SWcAqODpTouIQHo2qk6rqCo0DauMj6f73+oMq+LLE9fV55GudZm/PYlJKw7w4YLdjJm/m/Z1ghjUKozroq/By8N2lpCVCT8Mt9ZduGeetX7E4nes99fqnhK/R2UfmgjU1anNA7B1pnUE2nK4/evfPA08/aDu9far8+L6BPuXWk0oduDt4c6HQ1pwz9ereGrKhkvP+/t40CqyCgPjwmgVVYXYmgGFa+Kx8XB3o3t0CN2jQzicepbvVyXw3cqDPDhhLVX9vOjfIpRBLcOotf4t2LcIbv4IQmKg51uQdhRmPwX+10DD3nZ5n6pkpLz1N46LizOrVq1ydhiqrDMGPu0M59LggRVW04u9ZGXC2/WsuYVuGWffev8TaSWB3qPtVy9w9nwWny7ag7+PB62jqlL/Gn/c3ezbTp+VbVi0M4lJKw4yd+tROrGKz7zeZm/kAKrf/tFfZxjn0+HrPnB4A9w5HSLa2jUOlTsRWW2Micttm/YaUlcnEeusIHkn7Jpr37r3/mFNHV2cuYXy4+4B4W1K3HMoNxW83Hm4a13ujo8iukYluycBsC5Sd6pfjY/uaMGfI2sxpsLHbHOrTY9tPWn9+jxenrmZY6czrKa6wd9B5TCYOKjQU3Arx9FEoK5eMTdbFyaXj7FvvZunWl1U63S3b71gNQ8d3w5pSfavu7RcOEvVWSPw9vSg3oNT+WJEB66tF8w3y/fT+/3FrDuYYs3SOuQHazDeN/3h1GFnR+3SNBGoq5e7p3VBcs8COLrZPnVmnoetP0GDnuDpY586c7o471AJRhk73ewnrYV6+n2CW5VI2tUJ4v3BzZjxYHs83d0Y8PEyflidYF3Uv/17OHsSvr3FuqCsnMKhiUBEeojIdhHZJSKj8inXX0SMiOTafqVUsbW4GzwqwPIP7VPfngWQkVL0lcgKq3pT8PR1SPNQqVgzHtZ+A9c+BfUuv5DesHolZjzYnriIQJ74fj2v/rSFzJDGMOBrSNoG3w2xEq0qdQ5LBCLiDowBbgCigcEiEp1LOX/gEeBPR8WiXJhvFWuA1obJ9mlu2TwVvAOs2UYdwcMLQluWzzOCw+uts4FanaDTM7kWqeLnxdfDWnF3vDVB3tAvVpJSowPcNAb2LoTp/7Cm7lClypFnBK2AXcaYPcaY88Ak4KZcyr0K/AfIcGAsypW1+QdknYNVJezhcyHDmtm04Y3WF7ajRLa3mrLSTzjuNeztbApMvhMqVIH+n1vzPuXBw92NF3vH8N9bGrNi7wn6fLCE7SG9oOsLsPF7mKuDzUqbIxNBTeBgjscJtucuEZHmQJgxJt95g0XkXhFZJSKrkpLK8UU05RxBda0RwCs/hRN7IfNc8erZPQ/OnXJcs9BFEfGAsSbPKw+ys+HHkZCaAAO+Ar+gQu02IC6MSfe1IeNCFn3HLuGXyoOh5QhY+p79mvJUoThtQJmIuAHvAEMLKmuM+QT4BKxxBI6NTF2V2j5o9V1/r6n1uEIVq0eR/zU5btX/+lkxxLrlPPLfPM3ar1ZHx8ZaswW4e1vXCRr0cuxr2cPSd60R0T3+A2GtirRr8/BAZj7UnvvGr+b+b9fyaJcRPFL/MPLLM9bfwt5ddFWuHJkIDgFhOR6H2p67yB+IBRaINQHVNcAMEeljjNERY8q+anWEEfPg2BZr7pvTh+H0Eevnsa3WaFeTy8pmvkF/JYj9S6DRrVZvJEfy9IHQuGKtWFbq9i6Cea9YX9it7ytWFSGVfJh0bxue/3ETo3/fw84GD/BezSTcp94LfsF/9aRSDuPIRLASqCsiUVgJYBBw28WNxphU4NI5pIgsAJ7UJKAcJjTOuuUmOwvOHLfWDr6YIC77eQQqVoMWQ0sn1oh4WPSW1aWypGsdOMqpwzDlbqhaB/q8X6IZRX083XnzlsbE1KjE/83ayoCqDzOx0kt4TbwNhv0CIX/rZ6LsyGGJwBiTKSIPAnMAd2CcMWaziLwCrDLGzHDUaytVZG7u4B9i3ao3cXY01sCyhdlwcAXUdcDAtZLKumAlgfNn4K6Z4O1f4ipFhLvjo6gf4s8/Jqyhd/ZjzKjwMt7f3gLDf4OAmgVXoorFoeMIjDGzjTH1jDG1jTGv2Z57IbckYIzppGcDStmEtQI3j7LbPDT3Jetidu/3oFpDu1bdrk4QMx5oj1QOp1/qo5w/k4L59harZ5JyCB1ZrFRZ5OUHNZqXzfEEW6bDsg+g5T3Q+FaHvER4VV9+GNmO8Og2DD37CFnHdpA98TZrwjpld5oIlCqrIuMhca3V/FJWHN8FPz5g9Wy6/jWHvpSftwdjb29O2679eOL8vbgdWELmm3U5+/19sPt3a7ZWZReaCJQqqyLiITvTuk5QFpw/A5PvsHpN3foVeHg7/CVFhIe61uXGIY8y1LzEtLPNydw0Hcb3JfX1Oqz75D5WL5lD8mkdj1oSujCNUmVVWGsQN6t5qHbnktW16Qf4+WlrfQZPH2s+Iw8fa/lIzwpX3K+Qe5l9S6yutkN+sKaQLkXdo0OIf+4hNifexQ/7j5K9fQ61j/1Cm0M/4J04if1zqvGTZwf21ehJUK2mNA4NoFHNACr7OnAE+FVEF6ZRqiz7pJP1hXz37OLtfy4Nfv4nrPvWas6JaGdNlZF5Fi6cveL+WcjMgAvptudt97NzNMF0+Rdc+6Rd3po9nE5J5tiKKXhvm0aNE3/iRjZbs8OZkdWOmdltcQuMoFFoAI1rBtAoNIC4iCp/LaNZio6kZvDdyoN0rB9M07DKRa/AGBh/szVeo5hdmPNbmEYTgVJl2ZznrHV+Rx0o+rTXh9bADyPgxB7ry7vj08UbDJeVaSWL7CyoUIwvsdKSdgw2TyNz/fd4JK4EYJdPLDOy2vLt6eYkE0A1f2/uahfJ7a3DS+VsYU9SGh//sYepaxO4kGXw8XRjzG3N6dowpGgV7fzNmqr7pjHQbEixYtFEoFR5tW02TBoMQ2cVfoRtdjYsex/mvWoNguv3ieuNzj25z2oO2zgFjm3BiDvJ1dqyPL0GG064k+YeQEydKLo0a0j1GqHWLLXeAXZb0nTToVTGLtjFz5uO4OXuxoC4MG5pEcrzP25iy+FTvNGvEbfGFaF5bdwNkLIfHl5X7AkP80sEeo1AqbIsoi0g1rxDhfkyP30Ept1nrZvQsLfVz9+3iqOjLHsCI6HDE9bt6GZk4xSCtkznxvQV3OhpW/Ngj+1mY8Qd8a0CvlVtt5z3q4JfNWvmWc8Kub6kMYble04wdsEuFu08jr+3ByM71ubu+CiC/a0L6xPvbcP941fz1JQNJJ85z33X1kIKGpF9YDkcWAo93nDYrLd6RqBUWfdhvPVFdFcBg/G3/wzTH7D62t/wBjS/q0TTPlyVjLF6P6Unc+L4Yeav2caabbvwPp9C/UrnaRViiPDJwO3sCWtd6ou3i/NQNR5onWHlkJ1tmLv1KGMX7GbdwRSCKnozvH0Ut7cJp5LP35vizmdm88T365m5PpER7aN4tmdD3PJbQ/rbAZCwEh7bZI0vKSY9I1CqPIuIhzVfW6t35XZEeOEs/PYCrPgErmkE/cdBcL3Sj7M8EAHviuBdkSqBEfSv24ae57OYujaBjxft5eltZ6gR4MPQ+EgGtgwnoIKnlTwyUuGP/1jTY8c/CiHRXMjKZsa6RD76Yzc7j6URVqUCr94cy60tQvHxzHs9Bi8PN94d2JSqfl58tngvx9PO8d9bmuR+EfvIJtg5Bzo/V6IkUOCvRc8IlCrjtky3Fn0Z9iuEt75827GtMGWYNatqmweg24ul0r//apSdbZi//RifLdrLsj3J+Hm5M6BlGMPiowir4mstFPRuU7Ii2jM+4jU+XbSXQylnaXCNPyM71aZXo+p4uBf+GoMxhrELdvPmnO1cWy+YD29vjp/3FcfmU4bDjl+ss4EKgSV6f3pGoFR5Ft7O+rl/yV+JwBhY9bnVq8jbH27/Aep2c16MVwE3N6FrwxC6Ngxh06FUPl+8l/HL9vPV0n30iL2GIa0jkOq30XbHh/y4sTXVw1vy6s0xdK5freB2/lyICA90rkNQRS+embqR2z77ky+GtqSKn+2s78Qea2nUtg+UOAkUGIueEShVDnzQyhrENeQHOJMMMx60FoOp0w1u/tDqHaTs7khqBl8u3ceEP/dzKiMTXzJY7vs4JiSGgPuKObYjF79uPsJDE9dSM7ACXw9rRWigL8x81Br/8cgGqFS9xK+h3UeVKu9+egw2TIYBX1sXhNOTodvL0Pp+u3V5VHk7cy6TuVuPUreaP9EHvoVfRsGd06FWJ7u9xoq9Jxj+1Up8vdz5dmAkdSa0haa3Qe937VJ/folAP0FKlQcR8XA+Db7pZzUFjZgHbf+hSaCU+Hl7cFPTmkTXqARxw6BSqLUymx0PpFtFVeH7+9tiDCwc/zImOxPiH7Fb/fnRT5FS5UFUR6sfe/O74N4FUL2xsyNyXR7e0GkUHFoN22bZteoG11Ri6rAYBvIbs7LaMPeI43oK5aSJQKnyoGIwPLkD+rzn0G6EqpCaDIaqdeH3/7Om3rCj0J3f4MdZfqsymPu+Wc3klQftWn9uNBEoVV7o4LCyw90DujwHSVth4/f2q/d8ujVWoU53Xh95G+1qV+WfP2xgzPxdOPJ6riYCpZQqjoY3wTWNYf7r1mA/e1g73uoI0OFx/Lw9+PyultzUtAZvztnOKz9tITvbMclAE4FSShWHmxt0fdGaDG7t1yWvL+sCLH0fwtpY04VjjUL+34CmDIuP4osl+3h99taSv04udECZUkoVV52u1oC/P96EJreBl2/x69r4PaQehF5vX/a0m5vwrxsbUjOwAh3rBZcw4NzpGYFSShWXCHR9AdKOWHM9FVd2NiweDSGxUPe6XF5GGN4+ijrVKpYg2LxpIlBKqZKIaGt9eS/+H5xNKV4d22fB8e3Q/jGndArQRKCUUiXV5XnISIFlHxR9X2Ng0TvWGgrRN9s9tMJwaCIQkR4isl1EdonIqFy2Py4iW0Rkg4jME5EIR8ajlFIOUb0JxPSDZWOtJTOLYu8fkLjGGkXs7pzLtg5LBCLiDowBbgCigcEiEn1FsbVAnDGmMTAF+K+j4lFKKYfq/BxkZlhH90Wx6B2oGGJdbHYSR54RtAJ2GWP2GGPOA5OAm3IWMMbMN8ak2x4uB0IdGI9SSjlOUB1odrs1PXjKgcLtc2i1dUbQ9gHw9HFsfPlwZCKoCeQcG51gey4vw4Gfc9sgIveKyCoRWZWUlGTHEJVSyo46Pm39/OM/hSu/6B3wCbAmsnOiMnGxWESGAHHAm7ltN8Z8YoyJM8bEBQc7ph+tUkqVWEAotBwB6ybA8Z35l03aDtt+glb3WjPKOpEjE8EhICzH41Dbc5cRkW7Ac0AfY8w5B8ajlFKO1+EJ8PS1JqTLz+LR4FHBWlPCyRyZCFYCdUUkSkS8gEHAjJwFRKQZ8DFWEijipXallCqD/IKgzT9gy4+QuC73MikHYeNkaH6nVd7JHJYIjDGZwIPAHGArMNkYs1lEXhGRPrZibwIVge9FZJ2IzMijOqWUKj/aPWitM/z7q7lvvzjeoN1DpRdTPhzaadUYMxuYfcVzL+S4r6ttK6WuPj4B1ijh316AfUsgMv6vbWeOw+qvoNEAax3qMqBMXCxWSqmrTst7oOI1f1/S8s+PrPEG7R91XmxX0ESglFKO4OULHf8JB5fDzt+s5zJOWZPTNegFwfWdG18OmgiUUspRmt1hzSE07xVrhtHVX0BGKnR43NmRXUYTgVJKOYqHlzX1xNGNsGESLBsDUR2hZgtnR3YZTQRKKeVITiN6WAAAB2dJREFUsf2hWjTMfATSjpa5swHQRKCUUo7l5m5NU511Hmo0t84IyhhdqlIppRytfk9rHqK61ztl4ZmCaCJQSilHE4HOzzo7ijxp05BSSrk4TQRKKeXiNBEopZSL00SglFIuThOBUkq5OE0ESinl4jQRKKWUi9NEoJRSLk5MznmyywERSQL2F3P3IOC4HcOxt7IeH5T9GDW+ktH4SqYsxxdhjAnObUO5SwQlISKrjDFxzo4jL2U9Pij7MWp8JaPxlUxZjy8v2jSklFIuThOBUkq5OFdLBJ84O4AClPX4oOzHqPGVjMZXMmU9vly51DUCpZRSf+dqZwRKKaWuoIlAKaVc3FWZCESkh4hsF5FdIjIql+3eIvKdbfufIhJZirGFich8EdkiIptF5JFcynQSkdT/b+9cQ62oojj++6f2UtFrlplGpUWkUGYi5ovAMJVQCyvLzDQIST/4IcqwTPxmUUEhKZWkJSWaloiSj8Dwg6/EVym+EFJ8QIpm0UNdfdj72HTumetR75m53bN+MMyevdfMXrNm71kza2b2SNoap6lZ6RfrPyhpR6x7c4lySXo/2m+7pO4Z6nZ3wi5bJZ2WNKlIJnP7SZoj6biknYm8NpJWSdob5zUp646JMnsljclQv7cl7Y7HcImk1inr1tkeKqjfNEmHE8dxSMq6dfb3Cuq3IKHbQUlbU9atuP2uGDNrVBPQBNgPdAKuBrYBXYpkXgJmxfRIYEGG+rUHusd0S2BPCf0eApblaMODQNs6yocAKwABvYANOR7ro4QPZXK1H9Af6A7sTOS9BUyO6cnAjBLrtQEOxHlNTNdkpN9AoGlMzyilXzntoYL6TQNeLqMN1NnfK6VfUfk7wNS87HelU2O8I+gJ7DOzA2b2F/AlMKxIZhgwN6YXAQOkbH4kamZHzGxLTP8K7AI6ZFF3PTIMmGeB9UBrSe1z0GMAsN/MLvdL83rDzL4HThRlJ9vZXGB4iVUfAVaZ2QkzOwmsAgZloZ+ZrTSzs3FxPdCxvustlxT7lUM5/f2KqUu/eO54EviivuvNisboCDoAPyeWD1H7RHtBJnaEU8ANmWiXIIak7gc2lCh+UNI2SSskdc1UMTBgpaQfJL1YorwcG2fBSNI7X572K9DOzI7E9FGgXQmZhmLLcYS7vFJcrD1UkokxdDUnJbTWEOzXDzhmZntTyvO0X1k0Rkfwv0BSC+ArYJKZnS4q3kIId9wHfAB8nbF6fc2sOzAYmCCpf8b1XxRJVwNDgYUlivO2Xy0sxAga5LvakqYAZ4H5KSJ5tYcPgc5AN+AIIfzSEHmauu8GGnx/aoyO4DBwa2K5Y8wrKSOpKdAK+CUT7UKdzQhOYL6ZLS4uN7PTZnYmppcDzSS1zUo/Mzsc58eBJYTb7yTl2LjSDAa2mNmx4oK87ZfgWCFkFufHS8jkaktJzwOPAqOis6pFGe2hIpjZMTM7Z2bngY9S6s3bfk2Bx4EFaTJ52e9SaIyOYBNwl6Q74lXjSGBpkcxSoPB2xgjgu7ROUN/EeOInwC4zezdF5ubCMwtJPQnHKRNHJam5pJaFNOGB4s4isaXAc/HtoV7AqUQIJCtSr8LytF8RyXY2BvimhMy3wEBJNTH0MTDmVRxJg4BXgKFm9nuKTDntoVL6JZ87PZZSbzn9vZI8DOw2s0OlCvO03yWR99PqSkyEt1r2EN4mmBLzphMaPMC1hJDCPmAj0ClD3foSQgTbga1xGgKMB8ZHmYnAj4Q3INYDvTPUr1Osd1vUoWC/pH4CZkb77gB6ZHx8mxNO7K0Sebnaj+CUjgB/E+LULxCeO60B9gKrgTZRtgfwcWLdcbEt7gPGZqjfPkJ8vdAOC2/S3QIsr6s9ZKTfZ7F9bSec3NsX6xeXa/X3LPSL+Z8W2l1CNnP7XenkQ0w4juNUOY0xNOQ4juNcAu4IHMdxqhx3BI7jOFWOOwLHcZwqxx2B4zhOleOOwHEyJI6MuixvPRwniTsCx3GcKscdgeOUQNKzkjbGMeRnS2oi6Yyk9xT+I7FG0o1Rtpuk9Ylx/Wti/p2SVsfB77ZI6hw330LSovgvgPlZjXzrOGm4I3CcIiTdAzwF9DGzbsA5YBThi+bNZtYVWAu8GVeZB7xqZvcSvoQt5M8HZloY/K434ctUCCPOTgK6EL487VPxnXKcOmiatwKO0wAZADwAbIoX69cRBow7z7+Di30OLJbUCmhtZmtj/lxgYRxfpoOZLQEwsz8A4vY2WhybJv7V6nZgXeV3y3FK447AcWojYK6ZvfafTOmNIrnLHZ/lz0T6HN4PnZzx0JDj1GYNMELSTXDh38O3EfrLiCjzDLDOzE4BJyX1i/mjgbUW/j53SNLwuI1rJF2f6V44Tpn4lYjjFGFmP0l6nfBXqasII05OAH4Desay44TnCBCGmJ4VT/QHgLExfzQwW9L0uI0nMtwNxykbH33UccpE0hkza5G3Ho5T33hoyHEcp8rxOwLHcZwqx+8IHMdxqhx3BI7jOFWOOwLHcZwqxx2B4zhOleOOwHEcp8r5B2M+AJPM8bmQAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-HyvAMbVXG8L"
      },
      "source": [
        "## Saving our Baseline Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FAaYyrzlQbLJ",
        "outputId": "9ac2f8f2-1de2-449d-d380-5e86e2bd6d89"
      },
      "source": [
        "model.save('rps_model')\n",
        "model.save('rps.h5')"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: rps_model/assets\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2fOWErhkQ-kO",
        "outputId": "8d17ea6c-523f-44d1-9f0e-cc94baa8ebde"
      },
      "source": [
        "!du -lh rps_model"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "27M\trps_model/variables\n",
            "4.0K\trps_model/assets\n",
            "27M\trps_model\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yjdkDugfRMuc",
        "outputId": "3a69a5ce-6edd-4965-df40-0b62716d9aea"
      },
      "source": [
        "convert_bytes(get_file_size('rps.h5'), \"MB\")"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "File Size: 26.557Megabytes\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oxZPUH3wRAI-",
        "outputId": "4a1c8296-57d3-47fa-a465-7b504462e123"
      },
      "source": [
        "!tar cvf rps.tar.gz rps_model rps.h5"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "rps_model/\n",
            "rps_model/keras_metadata.pb\n",
            "rps_model/saved_model.pb\n",
            "rps_model/variables/\n",
            "rps_model/variables/variables.index\n",
            "rps_model/variables/variables.data-00000-of-00001\n",
            "rps_model/assets/\n",
            "rps.h5\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xQDBdfKRXK2p"
      },
      "source": [
        "## Quantising our Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WRh4RUo7XQ5O",
        "outputId": "24430a6d-e433-4e9a-8363-e460b972595c"
      },
      "source": [
        "quantize_model = tfmot.quantization.keras.quantize_model\n",
        "\n",
        "# q_aware stands for for quantization aware.\n",
        "q_aware_model = quantize_model(model)\n",
        "\n",
        "# `quantize_model` requires a recompile.\n",
        "q_aware_model.compile(optimizer='rmsprop',\n",
        "              loss='categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "q_aware_model.summary()"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "quantize_layer (QuantizeLaye (None, 150, 150, 3)       3         \n",
            "_________________________________________________________________\n",
            "quant_conv2d (QuantizeWrappe (None, 148, 148, 64)      1923      \n",
            "_________________________________________________________________\n",
            "quant_max_pooling2d (Quantiz (None, 74, 74, 64)        1         \n",
            "_________________________________________________________________\n",
            "quant_conv2d_1 (QuantizeWrap (None, 72, 72, 64)        37059     \n",
            "_________________________________________________________________\n",
            "quant_max_pooling2d_1 (Quant (None, 36, 36, 64)        1         \n",
            "_________________________________________________________________\n",
            "quant_conv2d_2 (QuantizeWrap (None, 34, 34, 128)       74115     \n",
            "_________________________________________________________________\n",
            "quant_max_pooling2d_2 (Quant (None, 17, 17, 128)       1         \n",
            "_________________________________________________________________\n",
            "quant_conv2d_3 (QuantizeWrap (None, 15, 15, 128)       147843    \n",
            "_________________________________________________________________\n",
            "quant_max_pooling2d_3 (Quant (None, 7, 7, 128)         1         \n",
            "_________________________________________________________________\n",
            "quant_flatten (QuantizeWrapp (None, 6272)              1         \n",
            "_________________________________________________________________\n",
            "quant_dropout (QuantizeWrapp (None, 6272)              1         \n",
            "_________________________________________________________________\n",
            "quant_dense (QuantizeWrapper (None, 512)               3211781   \n",
            "_________________________________________________________________\n",
            "quant_dense_1 (QuantizeWrapp (None, 3)                 1544      \n",
            "=================================================================\n",
            "Total params: 3,474,274\n",
            "Trainable params: 3,473,475\n",
            "Non-trainable params: 799\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SJ79lNsPXO0E"
      },
      "source": [
        "## Train the Quantise Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mtowFk9IXp_F",
        "outputId": "954becbc-176b-46d9-bd77-f23099a2d62e"
      },
      "source": [
        "q_aware_model.fit(train_generator, epochs=20, validation_data = validation_generator, batch_size=256)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "20/20 [==============================] - 177s 9s/step - loss: 0.9902 - accuracy: 0.7936 - val_loss: 0.1707 - val_accuracy: 0.8978\n",
            "Epoch 2/20\n",
            "20/20 [==============================] - 177s 9s/step - loss: 0.4041 - accuracy: 0.8493 - val_loss: 0.1008 - val_accuracy: 0.9704\n",
            "Epoch 3/20\n",
            "20/20 [==============================] - 175s 9s/step - loss: 0.2003 - accuracy: 0.9246 - val_loss: 0.2222 - val_accuracy: 0.8978\n",
            "Epoch 4/20\n",
            "20/20 [==============================] - 173s 9s/step - loss: 0.1648 - accuracy: 0.9408 - val_loss: 0.4998 - val_accuracy: 0.7581\n",
            "Epoch 5/20\n",
            "20/20 [==============================] - 171s 9s/step - loss: 0.3253 - accuracy: 0.8878 - val_loss: 0.0466 - val_accuracy: 0.9812\n",
            "Epoch 6/20\n",
            "20/20 [==============================] - 173s 9s/step - loss: 0.1835 - accuracy: 0.9324 - val_loss: 0.0570 - val_accuracy: 0.9758\n",
            "Epoch 7/20\n",
            "20/20 [==============================] - 172s 9s/step - loss: 0.1908 - accuracy: 0.9304 - val_loss: 0.2010 - val_accuracy: 0.9140\n",
            "Epoch 8/20\n",
            "20/20 [==============================] - 172s 9s/step - loss: 0.2084 - accuracy: 0.9306 - val_loss: 0.0991 - val_accuracy: 0.9731\n",
            "Epoch 9/20\n",
            "20/20 [==============================] - 172s 9s/step - loss: 0.1430 - accuracy: 0.9511 - val_loss: 0.1567 - val_accuracy: 0.9516\n",
            "Epoch 10/20\n",
            "20/20 [==============================] - 172s 9s/step - loss: 0.1159 - accuracy: 0.9577 - val_loss: 0.0688 - val_accuracy: 0.9704\n",
            "Epoch 11/20\n",
            "20/20 [==============================] - 172s 9s/step - loss: 0.1244 - accuracy: 0.9578 - val_loss: 0.0868 - val_accuracy: 0.9704\n",
            "Epoch 12/20\n",
            "20/20 [==============================] - 171s 9s/step - loss: 0.1370 - accuracy: 0.9478 - val_loss: 0.0866 - val_accuracy: 0.9704\n",
            "Epoch 13/20\n",
            "20/20 [==============================] - 172s 9s/step - loss: 0.0979 - accuracy: 0.9677 - val_loss: 0.2620 - val_accuracy: 0.9167\n",
            "Epoch 14/20\n",
            "20/20 [==============================] - 177s 9s/step - loss: 0.1594 - accuracy: 0.9446 - val_loss: 0.2123 - val_accuracy: 0.9167\n",
            "Epoch 15/20\n",
            "20/20 [==============================] - 178s 9s/step - loss: 0.1143 - accuracy: 0.9640 - val_loss: 0.0628 - val_accuracy: 0.9731\n",
            "Epoch 16/20\n",
            "20/20 [==============================] - 179s 9s/step - loss: 0.1030 - accuracy: 0.9627 - val_loss: 0.0811 - val_accuracy: 0.9731\n",
            "Epoch 17/20\n",
            "20/20 [==============================] - 179s 9s/step - loss: 0.1237 - accuracy: 0.9561 - val_loss: 0.1925 - val_accuracy: 0.9355\n",
            "Epoch 18/20\n",
            "20/20 [==============================] - 180s 9s/step - loss: 0.1547 - accuracy: 0.9478 - val_loss: 0.0321 - val_accuracy: 1.0000\n",
            "Epoch 19/20\n",
            "20/20 [==============================] - 180s 9s/step - loss: 0.0786 - accuracy: 0.9691 - val_loss: 0.0523 - val_accuracy: 0.9785\n",
            "Epoch 20/20\n",
            "20/20 [==============================] - 180s 9s/step - loss: 0.1128 - accuracy: 0.9650 - val_loss: 0.0561 - val_accuracy: 0.9785\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7fe82de83950>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GtanDpIkXSWI"
      },
      "source": [
        "## Evaluate the Quantise Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FuwyMi1vYtWU",
        "outputId": "fa9710fb-cf01-45e9-ccfa-7956cef2ebbf"
      },
      "source": [
        "q_aware_model_loss, q_aware_model_accuracy = q_aware_model.evaluate(validation_generator, verbose=1)\n",
        "print('Quant Test Accuracy:', q_aware_model_accuracy)\n",
        "print('Quant Test Loss:', q_aware_model_loss)"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3/3 [==============================] - 7s 2s/step - loss: 0.0561 - accuracy: 0.9785\n",
            "Quant Test Accuracy: 0.9784946441650391\n",
            "Quant Test Loss: 0.05613008141517639\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rZawE0lfXvMM"
      },
      "source": [
        "## Convert the Baseline Model to TF-Lite"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vaI-0o4lY4p3",
        "outputId": "f8432d18-4597-49b1-e878-fcd1baa7aecf"
      },
      "source": [
        "TF_LITE_MODEL_FILE_NAME = 'tf_lite_model_rps.tflite'\n",
        "\n",
        "tf_lite_converter = tf.lite.TFLiteConverter.from_keras_model(model)\n",
        "tflite_model = tf_lite_converter.convert()\n",
        "\n",
        "tflite_model_name = TF_LITE_MODEL_FILE_NAME\n",
        "open(tflite_model_name, \"wb\").write(tflite_model)"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tmp/tmpagum708k/assets\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tmp/tmpagum708k/assets\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "13898360"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CHKjs1KOX5Py"
      },
      "source": [
        "## Convert the Quantise Model to TF-Lite"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dWWY9Mqtvtfw",
        "outputId": "f10ecf81-11c8-4c26-9eab-36c097e81063"
      },
      "source": [
        "TF_LITE_QUANT_MODEL_FILE_NAME = 'tf_lite_quant_model_rps.tflite'\n",
        "\n",
        "tf_lite_converter = tf.lite.TFLiteConverter.from_keras_model(q_aware_model)\n",
        "tf_lite_converter.optimizations = [tf.lite.Optimize.DEFAULT]\n",
        "tflite_model = tf_lite_converter.convert()\n",
        "\n",
        "tflite_model_name = TF_LITE_QUANT_MODEL_FILE_NAME\n",
        "open(tflite_model_name, \"wb\").write(tflite_model)"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:absl:Found untraced functions such as conv2d_layer_call_and_return_conditional_losses, conv2d_layer_call_fn, conv2d_1_layer_call_and_return_conditional_losses, conv2d_1_layer_call_fn, conv2d_2_layer_call_and_return_conditional_losses while saving (showing 5 of 40). These functions will not be directly callable after loading.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tmp/tmpi7qy6wk8/assets\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tmp/tmpi7qy6wk8/assets\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3491848"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0FY2g-T0X7X7"
      },
      "source": [
        "## Calculating the file size"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tUvid56XwA0j",
        "outputId": "9478adb4-3d52-4e10-a7f3-7bf69cdf382d"
      },
      "source": [
        "convert_bytes(get_file_size('tf_lite_model_rps.tflite'), \"MB\")\n",
        "convert_bytes(get_file_size('tf_lite_quant_model_rps.tflite'), \"MB\")"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "File Size: 13.255Megabytes\n",
            "File Size: 3.33Megabytes\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "afGOdqf9UTvZ"
      },
      "source": [
        "images, labels = (next(iter(validation_generator)))"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nycRGIg6UdIT",
        "outputId": "633cfd45-e9c9-4ac6-9a51-5eeb44e28bd5"
      },
      "source": [
        "print(images.shape)\n",
        "print(images.dtype)"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(128, 150, 150, 3)\n",
            "float32\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F1St2NuzYBC0"
      },
      "source": [
        "## Check the Tensors Shape - Baseline Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nA498GSBWK6U",
        "outputId": "aa828331-6e4f-438b-805e-8e22625026d1"
      },
      "source": [
        "interpreter = tf.lite.Interpreter(model_path = TF_LITE_MODEL_FILE_NAME)\n",
        "input_details = interpreter.get_input_details()\n",
        "output_details = interpreter.get_output_details()\n",
        "\n",
        "print(\"Input Shape:\", input_details[0]['shape'])\n",
        "print(\"Input Type:\", input_details[0]['dtype'])\n",
        "print(\"Output Shape:\", output_details[0]['shape'])\n",
        "print(\"Output Type:\", output_details[0]['dtype'])"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Input Shape: [  1 150 150   3]\n",
            "Input Type: <class 'numpy.float32'>\n",
            "Output Shape: [1 3]\n",
            "Output Type: <class 'numpy.float32'>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U7paq-MjYUgY"
      },
      "source": [
        "## Resize the Tensor Shape - Baseline Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3iC0oisvWa8J",
        "outputId": "dcedd070-f620-44f9-ec25-84aa3d1cd762"
      },
      "source": [
        "interpreter.resize_tensor_input(input_details[0]['index'], (128, 150, 150, 3))\n",
        "interpreter.resize_tensor_input(output_details[0]['index'], (128, 3))\n",
        "interpreter.allocate_tensors()\n",
        "\n",
        "input_details = interpreter.get_input_details()\n",
        "output_details = interpreter.get_output_details()\n",
        "\n",
        "print(\"Input Shape:\", input_details[0]['shape'])\n",
        "print(\"Input Type:\", input_details[0]['dtype'])\n",
        "print(\"Output Shape:\", output_details[0]['shape'])\n",
        "print(\"Output Type:\", output_details[0]['dtype'])"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Input Shape: [128 150 150   3]\n",
            "Input Type: <class 'numpy.float32'>\n",
            "Output Shape: [128   3]\n",
            "Output Type: <class 'numpy.float32'>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0SH-N7njYzqw"
      },
      "source": [
        "## Making Predictions on Baseline TF-Lite Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9I3ugQJXWgsm",
        "outputId": "8681f22c-7e6d-47ea-d270-9c44dc640ad2"
      },
      "source": [
        "interpreter.set_tensor(input_details[0]['index'], images)\n",
        "interpreter.invoke()\n",
        "\n",
        "tflite_model_predictions = interpreter.get_tensor(output_details[0]['index'])\n",
        "\n",
        "print(\"Prediction results shape:\", tflite_model_predictions.shape)\n",
        "prediction_classes = np.argmax(tflite_model_predictions, axis=1)\n",
        "expected_classes = np.argmax(labels, axis=1)\n",
        "\n",
        "acc = accuracy_score(prediction_classes, expected_classes)\n",
        "print('Test accuracy TFLITE model :', acc)"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Prediction results shape: (128, 3)\n",
            "Test accuracy TFLITE model : 0.8125\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fD4MZ-pfYMmf"
      },
      "source": [
        "## Check the Tensors Shape - Quantise Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w7ElpD22T1Rz",
        "outputId": "a0e1c700-bbfa-4f23-9a3c-18a501742e71"
      },
      "source": [
        "interpreter = tf.lite.Interpreter(model_path = TF_LITE_QUANT_MODEL_FILE_NAME)\n",
        "input_details = interpreter.get_input_details()\n",
        "output_details = interpreter.get_output_details()\n",
        "\n",
        "print(\"Input Shape:\", input_details[0]['shape'])\n",
        "print(\"Input Type:\", input_details[0]['dtype'])\n",
        "print(\"Output Shape:\", output_details[0]['shape'])\n",
        "print(\"Output Type:\", output_details[0]['dtype'])"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Input Shape: [  1 150 150   3]\n",
            "Input Type: <class 'numpy.float32'>\n",
            "Output Shape: [1 3]\n",
            "Output Type: <class 'numpy.float32'>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eB0uCIT4Ymji"
      },
      "source": [
        "## Resize the Tensor Shape - Quantise Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BHaLzGO2UhYO",
        "outputId": "5af988a5-e714-4ad9-c5d1-4fc481f9a721"
      },
      "source": [
        "interpreter.resize_tensor_input(input_details[0]['index'], (128, 150, 150, 3))\n",
        "interpreter.resize_tensor_input(output_details[0]['index'], (128, 3))\n",
        "interpreter.allocate_tensors()\n",
        "\n",
        "input_details = interpreter.get_input_details()\n",
        "output_details = interpreter.get_output_details()\n",
        "\n",
        "print(\"Input Shape:\", input_details[0]['shape'])\n",
        "print(\"Input Type:\", input_details[0]['dtype'])\n",
        "print(\"Output Shape:\", output_details[0]['shape'])\n",
        "print(\"Output Type:\", output_details[0]['dtype'])"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Input Shape: [128 150 150   3]\n",
            "Input Type: <class 'numpy.float32'>\n",
            "Output Shape: [128   3]\n",
            "Output Type: <class 'numpy.float32'>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W1Na44SaZ8WT"
      },
      "source": [
        "## Making Predictions on Quantise TF-Lite Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ksKlPdlXUkoA",
        "outputId": "ecb4842c-bd5d-4675-a919-9baa41d54bd4"
      },
      "source": [
        "interpreter.set_tensor(input_details[0]['index'], images)\n",
        "interpreter.invoke()\n",
        "\n",
        "tflite_model_predictions = interpreter.get_tensor(output_details[0]['index'])\n",
        "\n",
        "print(\"Prediction results shape:\", tflite_model_predictions.shape)\n",
        "prediction_classes = np.argmax(tflite_model_predictions, axis=1)\n",
        "expected_classes = np.argmax(labels, axis=1)\n",
        "\n",
        "acc = accuracy_score(prediction_classes, expected_classes)\n",
        "print('Test accuracy TFLITE model :', acc)"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Prediction results shape: (128, 3)\n",
            "Test accuracy TFLITE model : 0.9609375\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ygUCpNvVcdPB"
      },
      "source": [
        "# Result Summary\n",
        "\n",
        "\n",
        "| Configuration          | Baseline Model | Quantised Model |\n",
        "|------------------------|----------------|-----------------|\n",
        "| EPOCHS                 | 20             | 20              |\n",
        "| Model Accuracy         | 85.75%         | 97.84%          |\n",
        "| TF-Lite File Size      | 13.255 MB      | 3.33 MB         |\n",
        "| TF-Lite Model Accuracy | 81.25%         | 96.09%          |"
      ]
    }
  ]
}