Simple_RNN_PyTorch.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Simple RNN PyTorch.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/paulgureghian/Google_Colab_Notebooks/blob/master/Simple_RNN_PyTorch.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "299kFn972QIO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Created by Paul A. Gureghian on 12/16/18.** "
      ]
    },
    {
      "metadata": {
        "id": "c53TGGEamACF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Simple RNN in PyTorch.**\n",
        "\n",
        "**In ths notebook, I will train a simple RNN to do time-series prediction.** \n",
        "\n",
        "**Given some set of input data, it should be able to generate a prediction for the next time step.**\n",
        "\n",
        "\n",
        "* **First, I will create the data.**\n",
        "* **Then, define an RNN in PyTorch.**\n",
        "* **Finally, I will train my network and see how it performs.**"
      ]
    },
    {
      "metadata": {
        "id": "YWowVGZO2eQf",
        "colab_type": "code",
        "outputId": "49e1d211-ac49-4c42-f80a-f1e42717f3e7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "### Mount Google drive\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Y8TRsqz02imn",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Install PyTorch\n",
        "from os.path import exists\n",
        "from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag\n",
        "platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())\n",
        "cuda_output = !ldconfig -p|grep cudart.so|sed -e 's/.*\\.\\([0-9]*\\)\\.\\([0-9]*\\)$/cu\\1\\2/'\n",
        "accelerator = cuda_output[0] if exists('/dev/nvidia0') else 'cpu'\n",
        "\n",
        "!pip install -q http://download.pytorch.org/whl/{accelerator}/torch-0.4.1-{platform}-linux_x86_64.whl torchvision"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "CZd0NUIymACH",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Import resources and create data "
      ]
    },
    {
      "metadata": {
        "id": "JIiI7axomACJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Import packages\n",
        "import torch\n",
        "import numpy as np\n",
        "from torch import nn\n",
        "import matplotlib.pyplot as plt\n",
        "import matplotlib.image as mpimg\n",
        "%matplotlib inline"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ok3D3Euu5did",
        "colab_type": "code",
        "outputId": "ffd90ff9-1670-4ece-8a9e-27a5afcdff16",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 264
        }
      },
      "cell_type": "code",
      "source": [
        "### Time prediction image\n",
        "time_prediction = mpimg.imread(\"/content/drive/My Drive/Images/time_prediction.png\") \n",
        "plt.imshow(time_prediction)\n",
        "plt.grid(False)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD3CAYAAADi8sSvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8FdXdx/HP3IvsKKiVIKaA2B4l\ngloElEXjLi7YR1BUxAVbtVXUqmBbl4prxVZbl9bS4o4WtFRBUWQVhcKDGwK1R/FhU0BQhIIiJDPz\n/DE3N/eS7Sa5Wybf9+vFi5kzZ2Z+JOSXc39zZsbxfR8REQmHSK4DEBGR9FFSFxEJESV1EZEQUVIX\nEQkRJXURkRBRUhcRCZEm6T6gMeZB4CjAB6611i5O9zlERKRyaR2pG2OOBX5grT0auAx4KJ3HFxGR\n6qW7/HIC8BKAtfYjoJ0xZs80n0NERKqQ7qReAGxKWN8UaxMRkSxIe019N04N2/WMAhGR2qsyt6Z7\npL6O5JH5/sD6NJ9DRESqkO6k/gYwBMAY8yNgnbV2W5rPISIiVXDS/ZRGY8xvgWMAD7jKWrukmu4q\nv4iI1F6V5Ze0J/VaUlIXEam9rNXURUQkh5TURURCREldRCRElNRFREJESV1EJESU1EVEQkRJXUQk\nRJTURURCREldRCRElNRFREJESV1EJESU1EVEQkRJXUQkRJTURURCREldRCRElNRFREJESV1EJESU\n1EVEQkRJXUQkRJTURURCREldRCRElNRFREJESb0KjuPkOgQRkVprFEl9zJgx7L///syYMaPCtrlz\n57LffvsxYMCAeNvo0aOT/gaYN28e++23H/379898wCIideT4vp/L82f85DfffDP9+/dn4MCB3HLL\nLaxcuZIJEyYAcP755xOJRJgwYQKe59G1a1dWrlwJBCP1sq/N8OHDKSkp4e9//zue59GlSxdWr16d\n6dBFRKpSZSkh9Em9ZcuWfPvtt/H1qVOncuaZZwKwfPlyioqK4tsSE3nicnX9RERyoMqkHuryy6JF\nizj++OOT2soSOkBRURETJ06kffv21dbQi4qKmDRpEgUFBaq1i0heC3VSLyoqYunSpUlt1tr4cps2\nbRg6dChffPFFtSPv1q1bc+6557Jhw4aURuiO4yj5i8T89Z8eAIedV8q6TZn7hHvBr92MHr+hCHVS\nb926NWvWrElq69OnT3y5d+/eKR2nZ8+eaY1LpDGa97coHfatud+Q0W7mgwmxUCd1AN/3ad68OY7j\nMGjQILZs2RLfNnTo0Pio+oEHHqBHjx7xEfY111wTX77ooovi/e6//36OOOIIjcQlVCbPDkbTd493\nOf6KUp551YtvW7TM5/3/+Ix9yuOh54N23/c57opSBo+qmICfmupx1MWlPDXVS2qfNMMn8YPu+Jc8\njrywlF8+VH6M2x5z+WSNz22Plbc99g+PIy4o5Y/PJx8P4I/Pe5xwZSlT51Xc1mj5vp/LP6FEcAE4\n12GIpOzKu0v9dsUl/rpNnu/7vn/6taX+hi+D5XGTXf/eJ1x/wmuuv6vE87f81/PbHlsS3zdxecVa\nz7/6vlLf933/f5d5/oGDSvxxk13f933/xJ+V+CUlwTHf/sDzn5wStD/7qpt0jMGjSuPLb77r+s++\nGvQb/5Lrdzi5vF+74hJ/wrTybadfU+J/vtFL01ck71WZV5ukkviNMYcCLwMPWmsfMcYUAs8AUWA9\nMNxau9MYMwy4DvCAcdba8Rn5TSQiabd5Tnk6eOUPUdoVl/L13KDtzXc9Xns4WO58Znk7wKpXovz4\nBpeXfh/lyAvd+LZeRQ4rXoryt5cq1rnPuLa837DTIhxxcOWffM+63ov3G3FWhJbNy7e9OS5K9x84\n8W03PKjROlBzUjfGtAIeBmYlNN8BPGqtfcEYcw8wwhjzNHAb0BvYBSw2xvzTWrs5A3GLSBYN7Jtc\nqT3ywtKk9c82Vr5fUKZMTuo7dvp8vyC5X7cDKyb1rdt9mjdNPldJKZx3SoQt2/x4Qi/zo4Nr+Ec0\nEqmM1HcCpwE3JbQVA1fGlqcCNwIWWGyt3QpgjJkP9IttF5EGbN+2yevvPJvSh/xKtWjmsGZDctu6\nTT77fy85Se/V2qHUrfxcbds47Njp06JZ+T5LPqlzSKFS44VSa22ptXbHbs2trLU7Y8sbgQ5AAbAp\noU9Zu4g0AIkXRwH+57jKSyLfawdr1pePvm96yOXRScG+bdsk9731z6nNZBk4svJ+pbs19xxWPmov\n/mnyRleTZoDURuo1qWoaiKaHiDQg55/iYM4uZeNmOOyHMHdc5enh4382YeIbHoedH2TRDydGKWwf\n/LivnNqEQwaXsuErOGA/WDqpSXyeeqKv5zahy5mlbNkGrVrAyqnR+LZ57/nxev7Xc5vQ+YxStm4P\n+q14qbzfrMeiHDCwlG92wFHdYWBfpRyoe1LfboxpERvBdwTWxf4kVso6AgvrGZ+IZEmTJg52csWU\n8NP/qfiBfujJEYaeXPkH/Y/+kXyMsv1n/Cm5feXUytPPxpnJ7ateqbxf65YOn72WjnFpuNR1nvpM\nYHBseTDwOrAI6GWMaWuMaU1QT3+r/iGKiEiqanyglzGmJ/B7oDNQAnwODAOeBJoDq4FLrbUlxpgh\nwCiCy90PW2sn1HD+UN7TW3ZjUk1fW5F8sXmrz957qXzRgDTepzTmgpK6iGRY43xKo4hIY6OkLiIS\nIkrqIiIhoqQuIhIiSuoiIiGipC4iDdrnGzXLLJGSuog0WN/t9Dn0XJeVnyuxl9E89QzQPHWR7Ch7\nLgyQ9Iz3RkDz1EUkfMqeC7NuerSGno2HRuoZoJG6SGZ4ns8+x7tEIvDV7EY1Mt+dRuoi0vAVXx48\n7tfzoLRUg6bKaKSeARqpi6TXNzt8WrUIfq7aFZfy6cvRxv4AMj3QK5uU1EXSp+xlGo3sQmhNVH4R\nkYZn5y6fLduC5d8+offVpUIj9QzQSF0kfdqfVErrlvDpyxqpJ1D5JZuU1EXqrsfQUtZ+oXJLDVR+\nEZGGYe0Xwd8dTy3NbSANlJK6iOSVeX+N0qIZfP66Rup1ofJLBqj8IpK6b7/z6Xiqy5ezokSjjXqa\nYm2opp5NSuoiqWtXXF5mUR09Zaqpi0h+Wjk1eG7Lhjf0/JZ0UFIXkax77jWPTV8Hn2TbtnFYNTVK\ns6YqvaSDyi8ZoPKLSNUSyy0PXB/h0kEaW9aBaurZpKQuUjXf99n7OJeuB8A7z6qGXkeqqYuEib9o\nQa5DqJUZizzM2cEI3XEcvp7bRAk9Q5TURRoYf9kSvBFDcbt3ynUoKbnudy7n3uSxcTP87B49vyXT\nVH7JAJVfJFP8NavwBg5IaotMmY3T9Qc5iig1I8a4zF/iYydrdJ4mqqlnk5K6ZIK/6Qu84iMr3eb8\nagyRC0dkOaLq7XtCKV/OUhLPECX1bFJSl3Tzv9mO1/uQ+Hp0+VoA3MMPhJKSoPH7nYm+9lYuwkuy\nZZtPlzPLyyy6oSgjdKFUpKHyd+2sNKEDRD/4v/KOa1blRZ29bRuHe0cGqeWXlyrFZJtG6hmgkbqk\ni+95eAmJOvLhKpxoxTsvvckT8W+9sbzfK3NxunTNSoxl2hWXMum+CCf1USLPgvqVX4wxY4EBQBPg\nXmAx8AwQBdYDw621O40xw4DrAA8YZ60dX8OhQ5n1lNQlXdyiwvhy5P0VOE2bVd130Anw6cfx9cQR\nfaZ1PqOUrduDZZVbsqLu5RdjzHHAodbao4FTgT8AdwCPWmsHACuAEcaYVsBtwIlAMfALY8ze9Y9d\npHFKSuj/WlZtQgeITpmVlMjdokLc3WbKZMqqV4JEfsmZutU/11L5nDQPOCe2vAVoRZC0p8TaphIk\n8j7AYmvtVmvtDmA+0C+t0Yo0EkkJfe67OHvulfK+kdffLl9Zswrv+afSGVrc4y973Pt4+QXRueOi\nPHiDHsqVazUmdWuta639JrZ6GTANaGWt3Rlr2wh0AAqATQm7lrWLSC0kJfRX5uJ8b79a7e8UdgpG\n7E2C0bN/1y1Jx0yHEWNcbnjQY+zTPl//NygzHvZDjdLzQcpXNIwxZxEk9at321TVd1LfYZFaco/8\nYXw58sK0el3sjC5ZCQk3Jbk9OtcntCSP/6Z8RN5uT/2o55OUkrox5hTgZmCgtXYrsN0Y0yK2uSOw\nLvanIGG3snaRvOV/sz3XIcS5RYWwYwcAkb88g9Ote72PGZ0yG+f2+2IncHGLCvFXr6zTsV59y6vw\nQgtdFM0/qVwo3Qu4HzjDWrs51jwTGBxbHgy8DiwCehlj2hpjWhPU03N/J4RIFbzRI5Pmf+eSe+JR\n8WXnd4/i9C9O27Ej51xAJOGmJO+0Y/AmPlOrY3z7nc+Ft3oALPlYs7ryWY1TGo0xlwO3Ax8nNF8M\n/A1oDqwGLrXWlhhjhgCjCKYqPmytnVDD+UP5v0NTGvPf7jXmbE7/211iLM6oW4lccnnmznVYFyiN\njba7dCX6ytyU9+14ainffqcpi3lCjwnIJiX1/ObPm433s4uT2iJz38H5Xvusx+IOHwzv/S8Azk+v\nJnLdTZk/Z+IvtGiU6Ierqux73BWlzPlLeRLfVeLTdA/V0POAkno2KannL/eYI+CrL4OVNnvCN9vB\nC8oKkedexjnsR9mLJXGEPmgwkXv/kLVzexOfxb/jV/H1yLR5OJ26JPUpq5/v0QQ2ztToPM/o2S8i\n/tYt5Qk9GiW6cDmRJeUXDb0LzspaLN5vRpevFJ+Y1YQOEBl6IXQ+sDye046p0Oeg2O+cX49QmmhI\nNFLPAI3U08MtKkzbI2W9G36O//rU+PruNfT4qNkcQnTyG/U+X3WSyh89jiD6/JSqO2dBYjznFkzi\n+dePipdYXNcnGlW5JQ9ppC4NS9nt7f69vwludz/7ZPwVts7HS0zokWVrKnZo3jz4236EP2dGnc9T\nk6S7O7v+MOcJHYLSC0CXfZYy2+1F+5PK7xJVQm94NFLPAI3U68e3H+GdfXK1fZzzLsL51RicJtXX\nev3Zb+CNvCy+Xt0sF/fEo2D95zX2qyu3R2dwYwlzvwKicxan/Rz18d/zzqXThue4crDDvSN1u3+e\n04XSbFJSr5/EckB0+Vq8P9yH/9dHKu/cqjWRux/AOWlgjceKTJ+Pc8D3Uzt3h45EZy6sXeDV8Bct\nwBsxNFjZqy3RBUvTdux0evt9n/5HaHTeACipZ5OSet3tntB3529Yj3fTNfBOFQm3Tz8i9z0Ee+yB\n169HtceqjD93Jt5VlwYrzZsTffeT1IOvgtu3O2zdEqw0a070vfofUxo91dQl/3mTJ8aXI1PnVNrH\nKehA9KkXiC5fi3PPgxU7LJqPV9wzKaE7t96dcgxO8YlwcLdg5bvv8O68OeV9K+OvXV2e0KNRJXTJ\nOI3UM0Aj9drzbrkB/5+TgpVa3ukI4JeU4N91M/6Lzye1R5atiX8/aiOpbLN0NU6k9uMfd9Dx8Gl5\nEs/lXasSOiq/ZJOSeu3VVHapDf/D9/HOHwQ9+xB9+sW6HWPJe+Xz1iMRoktX127/Kl4ULZImSurZ\npKReO+lM6Onkb/oCr/jI+Hqqsbk/vwTenFXr/URqQTV1yU/uuafHl51TzshhJBU532uPM7x8OqS/\n87sa9/E9LymhR6p5ropIJmikngEaqafGX/cZ3klHA8HFzMh5F+U4osol1ddnLsLpsH+l/bwH78X/\n25/i6xqhSwap/JJNSuqpydeyy+787dvw+nSLr1cVa4UXRdfivaIitaTyi+SXhpLQAZzWbZIeLeCO\n/EnSdm/ayxX+PUrokitK6pJ1/lvlc9AjE17KYSSpS5oWOXs6/scfxVf9UeWv7Y3UciqmSLqp/JIB\nKr9ULanu3L6A6Oz8ev5JTdxeBr79FoDIpFfxEi705vsnDgkV1dSzSUm9ag2p7FKV3V+FBxB57Gmc\nAcflIBpppFRTl9xzDy1/mFZDTegAzo3Jjw6ILl+rhC55Q0ldssIdMRTKPrn06ZfbYOopcumVsM++\nADg33pLjaESSqfySASq/JPM3f4U34HAAnJ//gshV1+c4IpEGTzX1bFJSTxaGOrpInlFNXXLDPaF3\nfNm5fGQOIxFpHKp/F5hIPfgfvg8b1gMQ+dOTOMeekOOIRMJP5ZcMUPkloLKLSMao/CLZpYQukhtK\n6g2EW1SI99qUXIeREm/CE/HlyPT5OYxEpPFR+SUD0l1+SbqDsWkzou+vSMtxM8G78Sr8sl8+hxxK\n9MXXchuQSDhpSmM2pTOpJz5rJFG+ljRUdhHJCtXUGyLv/jvLHx418RUi0+bFt7lFhbj9D8tVaJVS\nQhfJPSX1POV/tBz/yXHxdefQw3A6dSHyaHm9mq83J70OLpfcQcfHl50h5+cwEpHGrcbyizGmJfAk\n0B5oDtwJLAGeAaLAemC4tXanMWYYcB3gAeOsteNrOL/KL1WIj3qbNyf67icVt5/QOz4HHCDyj+k4\nB3er0C8b/P9bgXdm8EAr5+4HiPz4nJzEIdKI1L2mbowZCnSy1o41xnQCZgDzgWnW2heMMfcAa4Gn\ngfeA3sAuYDFwjLV2czWHV1KvhHt4VyjZBVRfxvCeexL/7lvLG447megjNf0eTT+VXUSyru41dWvt\nRGvt2NhqIfAZUAyUza+bCpwI9AEWW2u3Wmt3ECT+hv04vhxwiwrjCT0yZXa1fSMXXBIk0ZatgoY5\nb+AWFeJ/sb7a/dJJCV0kv6RcUzfGLACeIyivtLLW7oxt2gh0AAqATQm7lLVLivz3yt8C5Iy8Eafr\nD1LaL7r4P0nr3vG98XftSmtslfGnvxJfjvxjesbPJyI1SzmpW2v7AoOAZ0ke+lf1MaDKjwdSkffr\nX+ANPztYabMnkSuvrdX+0eVriSxdXX68I7pW+oaedPHuuQ3v+p8FK4WdclbPF5FkNSZ1Y0xPY0wh\ngLX2A4KHgG0zxrSIdekIrIv9KUjYtaxdUuC//GJ8ObpweZ2O4UQiRP61LKnNPfzAesVVFT/hrtHo\n629n5BwiUnupjNSPAW4AMMa0B1oDM4HBse2DgdeBRUAvY0xbY0xrgnr6W2mPOITSWZd29twreL1a\n2bTCkpLgEQOP/L5ex02kOrpI/kpl9ksLYDzBRdIWwBjgHYLZLs2B1cCl1toSY8wQYBTBrJaHrbUT\najh/o5/94h76/fhr3pxfjSFy4Yi0xeF/tgbvlIRr1Xu1Jbpgab2O6Q77MXzwbrBy3ElEH3m8XscT\nkTrRYwKyKdWk7l7zU5j1OgCRWf+LU5CZ68ruT4fBgvK7UZ07f0fk7KG1Po7/xXq844OXXjg33hy8\nqzOD/H8vxenWPaPnEGmg9JiAfOPPfiOe0IGMJXSA6F8nQMfykol/6434y5bU+jhlCR3IeEIHYO2a\nzJ9DJGQ0Us+Amkbqvufhde8UrOxXQHTO4kr7ZcLuM2JSrYkn7hdZtib+bxSRnNBIPZ/EEzpkNaFD\nLIm32TO+7hYV4q/7rNp9/EUL4suRJyalNaH7rhs8nGzEUNwenfF37cTtGczP999dFPy9vuIkKvew\nLsn7XzE8+Le4btD+8Ud4N1+Pe/iBuD+/JLh2IdII6B2lWZY04n279iWQdIguXI6/8G28y4IZMt5J\nR8NBhujLMyv09R77I/7DvwtW9tkXp/fRaY3F69E56dOCe+m5Ffo4HfbH7dON6KJ/A+B/8w2R94Jn\nynu9Dk7ev6gwvu6/9EKV20TCSiP1LPKmTo4vO2Mfxmm3d85icY7qT2RZQs16ha30ZqV4Qgei895P\nfyBFPZJWIw/+pfJ+27fFF70rL8KJRgFwRt+a3O/YE/C3/TdYPvm0tIUp0lAoqWeJe9EQ/F/G7hIt\n7ETk9B/nNiCC2n90+VqcO8sTt1tUGMyWAdwenePtGRvhtmiZvF72HJvdROZ/iHtqf3zPgxW2fEPz\nFkn9on96EidWXnKqOJZImCmpZ0usPgz5dwdm5OyhOOdfXN6wYB7uwAEQq0/zo16ZO/k7C5NW/T/8\nttJuTtt2sHY1/k3XEHl8Ynn/p/+a1M/9yQX4lbwpSqSxUFLPMH/XzgZxB2bklruC2Mougq5ZBYBz\nyRVEn5lc9Y71Pe/0+cEFzjdn4d11C86Pz4FItPK+0+bhz5qOc0hRvM05Zxhu90747yzE+/UviJxz\nAU7LlpXuL9IYaEpjBiROaUy6MPqnJ3GOPSFXYaXM/fkl8OYsIHu/hLypk3FOPRO2fI036iqiT75Q\nu/3/OQnnqP44HfbPUIQieUV3lGZTWVIvHXwqfBQ8YCuy2DaoEaT/zTewxx44TZtm9DxuUWHS3bSa\noSKSkiqTuqY0Zsj1+7SOJ3QikQaV0AGcVtm5yBhdvhb3vDNh6QcARJaszMp5RcJKI/UM+KBrAd2b\n7xGsHHIo0Rdfy21AIhI2Kr9kU0O4MCoiDZoeE5AN/vZtyRdG31+Rw2hEpDFSUk8jr0/5K936r9yI\n07RZDqMRkcZIF0rTxD21f3y5+b8/ozSHsYhI46WRehp4998Fa2MvfW7aTAldRHJGSb2e/K834z8Z\newhVn35EVUcXkRzS7Jd6qmymS23eUSoiUgea/ZIJSTNdlq7OYSQiIgEl9Trybr8pvhyZMhsnoi+l\niOSeZr/UgXvMEfDVl8HKCafidP1BbgMSEYlRTb0O4mWXNnsSXbi8wnbV1EUkw/SYgHRxe3SOvzyi\nqkcAKKmLSIbpKY3pkHRh9I0FOYxERKRyurqXIv+tOfFl51djcDpWfEmziEiuqfySAnfkT2D29GBl\nn32Jznu/2v4qv4hIhqmmXh+1fZSukrqIZJhuPqqPyAN/BvRsdBHJfxqpp8j3vJRvMNJIXUQyTCP1\n+tIdoyLSEKQ0pdEY0wJYBtwJzAKeAaLAemC4tXanMWYYcB3gAeOsteMzE7KIiFQl1eHnLcDm2PId\nwKPW2gHACmCEMaYVcBtwIlAM/MIYs3eaYxURkRrUmNSNMQcD3YBXY03FwJTY8lSCRN4HWGyt3Wqt\n3QHMB/qlPVoREalWKiP13wPXJ6y3stbujC1vBDoABcCmhD5l7SIikkXVJnVjzEXAv6y1K6voUtUV\n2CqvzIqISObUdKH0dOBAY8wZwAHATmC7MaZFrMzSEVgX+1OQsF9HYGEG4hURkWqkPE/dGHM7sAro\nC8yz1j5rjHkI+BCYACwFjgRKgfeAXtbarTUcNpQTuTVPXUQyLK3z1H8DXGyMeQvYG3gqNmr/JTAd\nmAmMSSGhi4hImumO0gzQSF1EMkx3lIqINAZK6iIiIaKkLiISIkrqIiIhoqQuIhIiSuoiIiGipC4i\nEiJK6iIiIaKkLiISIkrqIiIhoqQuIhIiSuoiIiGipC4iEiJK6iIidTB58uRch1ApPXo3A/ToXZFw\nchwnX36u9ehdEZH6WLRoEQCjR48GYNWqVfFt48ePZ/369XTo0IELL7wQgBkzZlBYWMjYsWOTjjNn\nzhwKCgq49tprMxKnRuoZoJG6SDgljtTnz59Pv3794u3fffcdzZo1q9BvwYIF9O3bN95eUlJCkybB\n66Gj0Siu69YplKo21PTiaalGWfKu63YRyZ5MDrL23nvveEIHuOKKK+LLffv2xfd9HMfhvPPOiyd0\ngIMOOijtsSip10NV/0k0UhdpXHr16pW03qdPn6T1HTt20LJlSzZs2MAFF1wQb+/ZsydLly6le/fu\naYtFSV1EJEsGDBjAHXfckdFz6EKpiEiW3HnnncycOTO+3qpVq7SfQyN1EZEUjRw5sl7TGn3f5/DD\nD2fJkiXx9XTT7JcMUE1dRDJM89RFRBoDJXURkRBRUhcRCRFdKM0A1dJFJFc0UhcRCREldRGREFFS\nFxEJESV1EZEQqfFCqTGmGHgBWB5rWgqMBZ4BosB6YLi1dqcxZhhwHeAB46y14zMRtIiIVK7GO0pj\nSf1qa+2QhLYngGnW2heMMfcAa4GngfeA3sAuYDFwjLV2czWH1zQREZHaS/sdpcXAlNjyVOBEoA+w\n2Fq71Vq7A5gP9Kvj8UVEpA5SnafezRgzBdgbGAO0stbujG3bCHQACoBNCfuUtYuISJakktQ/IUjk\nk4ADgTm77VfVxwC99kdEJMtqTOrW2s+BibHVT40xG4BexpgWsTJLR2Bd7E9Bwq4dgYVpjldERKpR\nY03dGDPMGHNjbLkAaA88AQyOdRkMvA4sIkj2bY0xrQnq6W9lJGoREalUKrNf2gDPAW2BpgSlmPcJ\nZrs0B1YDl1prS4wxQ4BRBLNaHrbWTqjh/Jr9IiJSe1WWt/WSDBGRhkcvyRARaQyU1EVEQkRJXUQk\nRJTURURCREldRCRElNRFREJESV1EJERy/eJpPR9GRCSNNFIXEQkRJXURkRBRUhcRCREldRGREFFS\nFxEJESV1EZEQydmURmPMg8BRBI/fvdZauziHsRwKvAw8aK19xBhTCDwDRIH1wHBr7U5jzDDgOsAD\nxllrx2chtrHAAILv1b3A4jyKrSXwJMGLU5oDdwJL8iW+hDhbAMti8c3Kp/iMMcXAC8DyWNNSYGye\nxTgMGA2UArcBH+ZLfMaYy4DhCU1HEryg588EueVDa+3PYn1HAefE2sdYa6dlIb7WBO+eaAc0I3gf\nxYZMxpeT56kbY44FRllrzzDGHAI8bq09OuuBBLG0Al4heBfrh7Gk/gQwzVr7gjHmHmAtwTfmPaA3\nsIsguR5jrd2cwdiOI/g6nWaM2Yfg5SSz8iG2WHxDgU7W2rHGmE7ADGB+vsSXEOfdwMnAo8Cx+RRf\nLKlfba0dktCWF///YrHsA/wL6Am0JkhKe+RLfLvFeixwLtANGG2tXWyMeY7gF9B/gBeBo4G9CN7K\nVmStdTMc09VAR2vtr4wx+wOzCX4RZiy+XJVfTgBeArDWfgS0M8bsmaNYdgKnEbxjtUwxMCW2PBU4\nEegDLLbWbo29m3U+wYggk+YR/OYG2AK0yqPYsNZOtNaOja0WAp/lU3wAxpiDCX7IX4015VV8VSgm\nf2I8EZhprd1mrV1vrb08z+JLdBtwH9Al4ZN/WXzHAa9Za3dZazcRvLGtWxZi+hLYJ7bcDtic6fhy\nVX4pAN5NWN8Ua/tvtgOx1pYCpcaYxOZW1tqdseWNQAeC+DYl9Clrz2RsLvBNbPUyYBpwSj7ElsgY\nswA4ADiDIAHkU3y/B64GLo7LfgbIAAACu0lEQVSt58X3djfdjDFTgL0JRsL5FGNnoGUsvnbA7XkW\nHwDGmF4EnxhKga8rieOrKuJbmsm4rLV/N8ZcYoxZQfD1O5PgE2PG4suXC6X5/LiAqmLLWszGmLMI\nkvrVKcaQ1a+ntbYvMAh4drdz5zQ+Y8xFwL+stStrGUc2v36fECTyswh+8YwnebCV6xgdgpHm2cAl\nBC+dz5vvcYKfEFzfSTWObP0fvBBYY609CDie4GcklTjqHF+ukvo6gt/sZfYnqDPli+2xi2sAHQni\n3T3msvaMMsacAtwMDLTWbs2z2HrGLipjrf2AIBlty5f4gNOBs4wxCwl+6G8lj75+ANbaz2NlLN9a\n+ynBRbR2eRTjF8ACa21pLL5t5Nf3uEwxsIBgtLtPQnuu4+sHTAew1i4BWgD7ZjK+XCX1N4AhAMaY\nHwHrrLXbchRLZWYCg2PLg4HXgUVAL2NM29gV7X4EFzMyxhizF3A/cEbCBae8iC3mGOCGWKztCS6k\n5U181tqh1tpe1tqjgL8RzH7Jm/ggmFlijLkxtlxAMJPoiTyK8Q3geGNMJHbRNK++xwCxC5DbY/Xo\nEuA/xpj+sc1nx+KbDZxujGka698R+HcWwltBcL2B2GSCbcBHmYwvJ7NfAIwxvyVICh5wVey3WC7i\n6ElQd+0MlACfA8MIPso1J7hgcam1tsQYMwQYRTDl6GFr7YQMx3Y5QQ3z44TmiwkSVE5ji8XXgqBc\nUEgwAhkDvEMwEyLn8e0W6+3AKoJRU97EZ4xpAzwHtAWaEnwN38+zGK8gKP8B3EUwsyWf4usJ3GWt\nHRhb7wb8hWDQushae32sfSTBz7YP3GKtnZWF2FoDjxP8sm5C8GlxQybjy1lSFxGR9MuXC6UiIpIG\nSuoiIiGipC4iEiJK6iIiIaKkLiISIkrqIiIhoqQuIhIiSuoiIiHy/875eD5RdyzsAAAAAElFTkSu\nQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f5e779324a8>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "0B02h7GqmACO",
        "colab_type": "code",
        "outputId": "cc265451-47c9-4b57-ad9e-31acf71b6b10",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 320
        }
      },
      "cell_type": "code",
      "source": [
        "### Create and plot the data\n",
        "plt.figure(figsize=(8,5))\n",
        "\n",
        "### How many time steps/data pts are in one batch of data\n",
        "seq_length = 20\n",
        "\n",
        "### Generate evenly spaced data pts\n",
        "time_steps = np.linspace(0, np.pi, seq_length + 1)\n",
        "data = np.sin(time_steps)\n",
        "data.resize((seq_length + 1, 1)) # size becomes (seq_length+1, 1), adds an input_size dimension\n",
        "\n",
        "x = data[:-1] # all but the last piece of data\n",
        "y = data[1:] # all but the first\n",
        "\n",
        "### Display the data\n",
        "plt.plot(time_steps[1:], x, 'r.', label='input, x') # x\n",
        "plt.plot(time_steps[1:], y, 'b.', label='target, y') # y\n",
        "\n",
        "plt.legend(loc='best')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAAEvCAYAAAByqHnKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHQhJREFUeJzt3X+UXGWd5/F3ugsHOpUMDdsBgvye\nymMIukoTCKIQN45tkFERnOOclVkwOu4Ie1BXBxTl4DKMIw5Ggy7iHNTZ2YPHWQVGR9ZmBjcOmlFi\no6Ch5+mS8EMNkoa0kCL8qu7eP6pau5rudJG6XU9X1/t1DodbdW8/9fS3n7qfus+t3LtofHwcSZLU\nfB2pOyBJUrsyhCVJSsQQliQpEUNYkqREDGFJkhIxhCVJSiTX7BccHt7tv4mq6u7uYmRkT+puzBvW\no5b1qGU9almPWvO9Hj09SxZN97xHwgnlcp2puzCvWI9a1qOW9ahlPWq1aj0MYUmSEjGEJUlKxBCW\nJCkRQ1iSpEQMYUmSEjGEJUlKxBCWJCmRukI4hHBCCOG+EMJF06x7bQjhzhDCv4UQPpp9FyVJqvjB\nD7Zw881fy6StX//619x7788yaWtfzRrCIYTFwLXA7TNssgk4BzgNeF0I4fjsuidpb0olGBjooFTK\nttHcwFaybVTKxpo1r+Tss8/NpK277trK4OC2TNraV/VctvIZ4EzgkqkrQgjHArtijL+oPr4VWAfc\nm2UnJT1fqQR9fV0Ui50UCqP09+8hn2+80e6+teSKQ5QLKxjp30zjjUrZufXWb7J9+32cc84fc9VV\nV7B8+eH8/OdFXvayE3jvey/lqquu4IADDuDBBx/k8cd/w4c/fDlLlizlIx+5hBtu+HsANmw4jw98\n4FK++MUvkMvlOOSQQ3nVq8543mt9+9vf4t57f8b7338J/f23cs89P+GDH/xwpr/PrCEcYywD5RDC\ndKsPBYYnPd4JHLe39rq7u1r28mJzoadnSeouzCvWo9be6rF9OxSLleVisZOdO5dwzDENvuD2e6E4\nBECuOETPzofgmFMabDQ7jo9aLVOPUgm2bYNVqxr+ULdkyf50db2Igw5azNDQv/PZz27i4IMP5vTT\nT+eSSy5h//33Y7/9Orjxxr/nO9/5Djfe+GU+9KEPkct1/LZeuVwHxx77Ys499xy6u7s5++yzpn2t\n8857GxdccAE7dz7ETTd9lS9/+cssXZptzbO+gcO0F6iebD5fYLvZenqWMDy8O3U35o2FXI9SCWLs\nIISxuvdBs9Vj2TIoHHcAxftyFI4rs2zZUwwPz7h5fZYdyX7HvZx434sIxz3Lc8uOhCz+JqUSuThI\nOazc553wQh4f+6Jl6pHx7Mru3U+zZ8+z7Nr1JMuXvxjYn8cee5Jly5bx4IO/5umnn+PEE09heHg3\nRxxR4Oc/v49du56kXB77bb3K5TF27XqSJ598hv32e3qvdXzPe97HO97xDi666H0888yifa75TB+Y\nGv129A4qR8MTDq8+J6lqYtp4/frF9PV1ZXaqNU+JO1nNDziFO1lNnsYbLpHnZLayhh9yMlspkcFU\ndHUn3L1+Hd19az3X3GZycZDcpNmVXBzMrO3OztpZ1fHxyk36xsbGJz1exKJFtceH5XK57td4/PHH\nOeCAxQwP72ysszNoKIRjjA8AS0MIR4cQcsBZwG1ZdExaKGLsoFis7CyKxU5izOZfBubiIAfe9xNO\n4U4OvO8nmezcYuygeF9lgqx4Xy6Tvs7lTljzXzmspFxYUVkurKjMhsyxe+75MQDbtt3D0UcfQ1fX\nYkZGdjE+Ps5jjz3Kjh2/BKCjo4PR0dGZ+14uc911m/jc577AHXd8l4cfzv4Yc9bp6BBCL3ANcDTw\nXAjhXOAbwP0xxpuBPwe+Ut38qzHGocx7KbWwEMYoFEZ/+wWqEMYyaXdi5zYxzZfFzm0u+joX/VQL\nyecZ6d/c8OmIF+LZZ5/lL/7ivTzyyCNcfvmVLF26lJNOOpl3vvNP+YM/KFAoVL7jdMIJL+Uv//IK\nDjywm3K5zOLFec444zW/becf/uFGTjvt1SxbdgjvfveFbNx4NVdf/elM+7po4vC9WYaHdzf3Beex\nljmn0yQLuR5zcU54ouGsd2770td6GvWccLasR62Jelx11RWsXbuO00579Qv6+fvv387g4DbOPPOP\n5qp/035nKusvZkmaRj4Pvb3ZHAFPbbjcuzrrJrPv6xz0U8rS008/xZo1r2z66xrCkqQF47LLrtin\nn1u5clW2HamT146WJCkRQ1iSpEQMYWkKr8ecvTmpabXhdq6rWp/nhKVJvB5z9uakptWG27muWhg8\nEpYmmYsLa7T7xSrm8mIl7VzXdrZ580w39WtMilsbGsLSJBMXqwAyv1gFNO+KQfPJXNQUrGu7evjh\nHfzLv/TPSdspbm3odLQ0ST4P/f17sr1YRYIrBs0nc1LTasPtXNd29alPfYLBwW186Ut/yxve8Eau\nvPJyABYtGueSSy7n8MNfzNvedjYrVryEk08+hUMOOYxNm67hoIP+A0ceeRQHHnggGza8m+uv/xz3\n3PMTxsZGectb/piTTjpl1lsbXn75h3jjG8/mpJNO5tlnn+Xtb38rN974dXK5fY9Sj4SlKSYuVpHp\nPn3iYhVtGhRzUtNqw+1c11aR5Rfz/uRPzuPlLz+RCy54F4899igXXPAurr32es455xxuuun/ALBj\nx684//x3ctZZb+a6667lox/9H3zqU9dSLEYA7r77xzzyyK/53Of+ls985vP83d99ka6uA1i//ize\n+ta3TRvAAH19Z3L77f8MwMDAnaxZ88qGAhg8EpYkzaE5+2IecNBBB/PpT/8NN9xwPU899STHHVc5\nPbH//gdw7LGVW9s/8sjDrFjxEgDWrHklo6Oj/PSnd7Nt20+56KI/A2B8fIxHH3101tc75ZRTue66\nTZTLZe6447uZXOLSEJYkzZnpvpiX1WVRb7jhek45ZQ1vfvO5DAx8n29/u3KUut9+00fbxC0N99tv\nP846602cd94FL+j1crkcq1ev4Uc/upP779/OCSe8rLFfAKejJUlzKOsv5k2+/eBvfvMbDj/8xYyP\nj3P77bfz3HPPPW/7gw46mAcffIDR0VG2bv0hAMcffwLf//4djI2N8cwzz7Bx49XPa3smfX1ncsMN\nn+cVr+ht6PeY4JGwJGnOZP3FvKOOOoYY/51Nm67hTW96Cxs3fpJDD13Ohg3nc9llH+HOO39Qs/27\n3vUeLrvsgxx22HKOOupoOjs7eelL/yOveEUv7373BcA4Z5/9VmD2WxsCvOQlK3niiSf4wz98fWO/\nSJW3MkzIW5HVsh61rEct61HLetSaqR533vkDjjjiSA47bDlXX30VL395L6973ewBOtOtDR966EGu\nueYTfOYz//OF9s9bGUqS2sv4+Dgf/vAH6OpaTHf3QbzmNevq+rnpbm14yy1f4xvfuJnLLvtYZv3z\nSDghP8nWsh61rEct61HLetSa7/WY6UjYL2ZJkpSIISxJUiKGsFqatx1sb/791er8YpZalrcdbG/+\n/bUQeCSsluVtB9ubf38tBIawWpa3HWxv/v21EDgdrZblbQfbm39/LQSGsFraxC3ysm603Ls62zY1\nJ/z7q9U5HS1JUiKGsCRJiRjCkiQlYghLkpSIISxJUiKGsCRJiRjCkiQlYghLkpSIISxJUiKGsCRJ\niRjCkiQlYghLkpSIISxJUiKGsCRJiRjCkiQlUtf9hEMIG4E1wDhwcYxx66R1FwJvB0aBH8UY3zsX\nHVVrK5XI9ubrkxr2BuzKkmNVzTTrkXAI4QygEGM8FdgAbJq0binwQeDVMcZXAceHENbMVWfVmkol\n6OvrYv36xfT1dVEqZddwd99autevo7tvLdk1rHblWFWz1TMdvQ64BSDGOAh0V8MX4Nnqf/kQQg7o\nAnbNRUfVumLsoFjsBKBY7CTGbM6C5OIgueJQZbk4RC4OZtKu2pdjVc1Wzwg7FBie9Hi4+hwxxqeB\njwHbgQeBH8YYh7LupFpbCGMUCqMAFAqjhDCWSbvlsJJyYUVlubCiMs0nNcCxqmZbND4+vtcNQghf\nAL4VY/zH6uPvAe+IMQ5Vj4j/DTgDeAL4DnBhjPHumdorl0fHc7nOrPqvFlEqwbZtsGpVxqfD5qxh\ntSvHqubIoumerOeLWTuoHvlWLQceri6vBLbHGB8FCCHcAfQCM4bwyMieejrbFnp6ljA8vDt1N5rm\n2GPhqacq/01nn+tx7PHw1Dg8tbBq2W7jYzbNrMdsY3XfG85urDo+as33evT0LJn2+Xqmo28DzgUI\nIZwI7IgxTvymDwArQwgHVB+fBBQb6qkkSW1i1iPhGOOWEMJACGELMAZcGEI4H3g8xnhzCOGTwP8L\nIZSBLTHGO+a2y5IkLQx1/TvhGOOlU566e9K664Hrs+yUJEntwCtmSZKUiCEsSVIihrAkSYkYwpIk\nJWIIS5KUiCEsSVIihrAkSYkYwpIkJWIIS5KUiCEsSVIihrAkSYkYwpIkJWIIS5KUiCEsSVIihrAk\nSYkYwpIkJWII63lKJRgY6KBUyrbR3MBWsm1Uah2+rzSdXOoOaH4plaCvr4tisZNCYZT+/j3k8403\n2t23llxxiHJhBSP9m2m8Ual1+L7STDwSVo0YOygWOwEoFjuJsfEhkouD5IpDleXiELk42HCbUiuZ\ni/cV27b5vloADGHVCGGMQmEUgEJhlBDGGm6zHFZSLqyoLBdWUA4rG25TaiVz8b5i1SrfVwuA09Gq\nkc9Df/8eYuwghLFsZrfyeUb6N5OLg5UdhVNmajO+rzQTQ1jPk89Db28Gn9SnNFruXZ1tm1IL8X2l\n6TgdLUlSIoawJEmJGMKSJCViCEuSlIghLElSIoawJEmJGMKSJCViCEuSlIghLElSIoawJEmJGMKS\nJCViCEuSlIghLElSIoawJEmJGMKSJCViCEuSlIghLElSIoawJEmJ5OrZKISwEVgDjAMXxxi3Tlp3\nBPAV4EXAXTHG/zoXHZUkaaGZ9Ug4hHAGUIgxngpsADZN2eQa4JoY48nAaAjhyOy7KUnSwlPPdPQ6\n4BaAGOMg0B1CWAoQQugAXg18o7r+whjjQ3PUV0mSFpR6QvhQYHjS4+HqcwA9wG5gYwjheyGEj2fc\nP0mSFqy6zglPsWjK8uHAZ4AHgG+FEN4QY/zWTD/c3d1FLte5Dy+7MPX0LEndhXnFetSyHrWsRy3r\nUasV61FPCO/gd0e+AMuBh6vLjwIPxhjvAwgh3A6sAmYM4ZGRPfvW0wWop2cJw8O7G2qjVIIYOwhh\njHw+o46VSuTiIOWwkuwanV0W9VhIrEct61Grp2cJ99+/e8G8/xs138fHTB8Q6pmOvg04FyCEcCKw\nI8a4GyDGWAa2hxAK1W17gdhwb1WXUgn6+rpYv34xfX1dlErZNNrdt5bu9evo7ltLNo1Kyprv/4Vh\n1hCOMW4BBkIIW6h8M/rCEML5IYSzq5u8F/hSdf3jwDfnrLeqEWMHxWJlar9Y7CTGxv/Zdy4OkisO\nVZaLQ+TiYMNtSsretm34/l8A6jonHGO8dMpTd09a93PgVVl2SvUJYYxCYZRisZNCYZQQxhpusxxW\nUi6sIFccolxYUZmSkjTvrFqF7/8FYF++mKV5Ip+H/v492Z4TyucZ6d/ckueEpHbi+39hMIRbXD4P\nvb2NfwKe2mi5d3W2bUrKnO//1ue1oyVJSsQQliQpEUNYkqREDGFJkhIxhCVJSsQQliQpEUNYkqRE\nDGFJkhIxhCVJSsQQliQpEUNYkqREDGFJkhIxhCVJSsQQliQpEUNYkqREDGFJkhIxhCVJSsQQliQp\nEUNYkqREDGFJkhIxhCVJSsQQliQpEUNYkqREDGFJkhIxhCVJSsQQliQpEUNYkqREDOEmKZVgYKCD\nUin7hnMDW8m+YUntyH1Vc+VSd6AdlErQ19dFsdhJoTBKf/8e8vlsGu7uW0uuOES5sIKR/s1k07Ck\nduS+qvk8Em6CGDsoFjsBKBY7iTGbsufiILniUGW5OEQuDmbSrqT25L6q+QzhJghhjEJhFIBCYZQQ\nxjJptxxWUi6sqCwXVlAOKzNpV1J7cl/VfE5HN0E+D/39e4ixgxDGspuFyecZ6d9MLg5WBrXTO5Ia\n4L6q+QzhJsnnobc3m0+VUxsu967Ovl1Jbcl9VXM5HS1JUiKGsCRJiRjCkiQlYghLkpSIISxJUiKG\nsCRJidT1T5RCCBuBNcA4cHGMces023wcODXGuDbTHkqStEDNeiQcQjgDKMQYTwU2AJum2eZ44PTs\nuydJ0sJVz3T0OuAWgBjjINAdQlg6ZZtrgMsy7pskSQtaPSF8KDA86fFw9TkAQgjnA98FHsiyY5Ik\nLXT7ctnKRRMLIYSDgAuA1wKH1/PD3d1d5HKd+/CyC1NPz5LUXZhXrEct61HLetSyHrVasR71hPAO\nJh35AsuBh6vL/wnoAe4Afg84LoSwMcb4vpkaGxnZs49dXXh6epYwPLw7dTfmDetRy3rUsh61rEet\n+V6PmT4g1DMdfRtwLkAI4URgR4xxN0CM8WsxxuNjjGuAs4G79hbAkiTpd2YN4RjjFmAghLCFyjej\nLwwhnB9COHvOeydJ0gJW1znhGOOlU566e5ptHgDWNt4lSZLag1fMkiQpEUNYkqREDGFJkhIxhCVJ\nSsQQliQpEUNYkqREDGFJkhIxhCVJSsQQliQpEUNYkqREDGFJkhIxhCVJSsQQliQpEUNYkqREDGFJ\nkhIxhKdRKsHAQAelUraN5ga2km2jktQa3K9OL5e6A/NNqQR9fV0Ui50UCqP09+8hn2+80e6+teSK\nQ5QLKxjp30zjjUpSa2jGfpW7BjLpa7N5JDxFjB0Ui50AFIudxNh4iXJxkFxxqLJcHCIXBxtuU5Ja\nRTP2q2zb1nCbKRjCU4QwRqEwCkChMEoIYw23WQ4rK5/UgHJhBeWwsuE2JalVNGO/yqpVDbeZgtPR\nU+Tz0N+/hxg7CGEsm1njfJ6R/s3k4mAlgJ2KltRGmrFf7cnn4andGTTcXIbwNPJ56O1t/JPa1EbL\nvauzbVOSWoT71ek5HS1JUiKGsCRJiRjCkiQlYghLkpSIISxJUiKGsCRJiRjCkiQlYghLkpSIISxJ\nUiKGsCRJiRjCkiQlYghLkpSIISxJUiKGsCRJiRjCkiQlYghLkpSIISxJUiKGsCRJiRjCkiQlkqtn\noxDCRmANMA5cHGPcOmnda4CPA6NABN4ZYxybg75KkrSgzHokHEI4AyjEGE8FNgCbpmzyBeDcGONp\nwBLg9Zn3UpKkBaie6eh1wC0AMcZBoDuEsHTS+t4Y4y+ry8PAwdl2UZKkhameED6USrhOGK4+B0CM\n8QmAEMJhwOuAW7PsoCRJC1Vd54SnWDT1iRDCMuCbwHtijI/t7Ye7u7vI5Tr34WUXpp6eJam7MK9Y\nj1rWo5b1qGU9arViPeoJ4R1MOvIFlgMPTzyoTk3/X+CyGONtszU2MrLnhfZxwerpWcLw8O7U3Zg3\nrEct61HLetSyHrXmez1m+oBQz3T0bcC5ACGEE4EdMcbJv+k1wMYY47cb7aQkSe1k1iPhGOOWEMJA\nCGELMAZcGEI4H3gc6Af+FCiEEN5Z/ZEbY4xfmKsOS5K0UNR1TjjGeOmUp+6etPx72XVHkqT24RWz\nJElKxBCWJCkRQ1iSpEQMYUmSEmnpEC6VYGCgg1Ip+4ZzA1vJvmFJUlbmJAOavP/flytmzQulEvT1\ndVEsdlIojNLfv4d8PpuGu/vWkisOUS6sYKR/M9k0LEnKytQMuOuubBpt9v6/ZY+EY+ygWKxc/rJY\n7CTGbH6VXBwkVxyqLBeHyMXBTNqVJGVnagZs29Z4myn2/y0bwiGMUSiMAlAojBJCNrcwLoeVlAsr\nKsuFFZTDykzalSRlZ2oGrFrVeJsp9v8tOx2dz0N//x5i7CCEsexmDPJ5Rvo3k4uDlT+AU9GSNO88\nPwOW8NRTjTfa7P1/y4YwVOrT25vNEfDUhsu9q7NvV5KUmTnJgCbv/1t2OlqSpFZnCEuSlIghLElS\nIoawJEmJGMKSJCViCEuSlIghLElSIoawJEmJGMKSJCViCEuSlIghLElSIoawJEmJGMKSJCViCEuS\nlIghLElSIoawJEmJGMKSJCViCEuSlIghLElSIoawJEmJGMKSJCViCEuSlIghLElSIoawJEmJGMKS\nJCViCEuSlIghLElSIoawJEmJGMKSJCViCEuSlIghLElSIrl6NgohbATWAOPAxTHGrZPWvRb4K2AU\nuDXGeOVcdFSSpIVm1iPhEMIZQCHGeCqwAdg0ZZNNwDnAacDrQgjHZ97LmZRK5Aa2QqnUtJeUJC1c\npRIMDHQ0LVbqmY5eB9wCEGMcBLpDCEsBQgjHArtijL+IMY4Bt1a3n3ulEt19a+lev47uvrUGsSSp\nIaUS9PV1sX79Yvr6upoSK/VMRx8KDEx6PFx97onq/4cnrdsJHLe3xrq7u8jlOl9gN6ex/V4oDgGQ\nKw7Rs/MhOOaUxtttsp6eJam7MK9Yj1rWo5b1qGU9ajVaj+3boVisLBeLnezcuYRjjsmgY3tR1znh\nKRbt4zoARkb27MNLTmPZkXQXVpArDlEurGBk2ZEwvDubtpukp2cJwy3W57lkPWpZj1rWo5b1qJVF\nPZYtg0Khi2Kxk0JhlGXL9jA8PPvP1du/6dQTwjuoHPFOWA48PMO6w6vPzb18npH+zeTiIOWwEvL5\nprysJGlhyuehv38PMXYQwlhTYqWec8K3AecChBBOBHbEGHcDxBgfAJaGEI4OIeSAs6rbN0c+T7l3\ntQEsScpEPg+9vc0JYKjjSDjGuCWEMBBC2AKMAReGEM4HHo8x3gz8OfCV6uZfjTEOzVlvJUlaQOo6\nJxxjvHTKU3dPWvevwKlZdkqSpHbgFbMkSUrEEJYkKRFDWJKkRAxhSZISMYQlSUrEEJYkKRFDWJKk\nRBaNj4+n7oMkSW3JI2FJkhIxhCVJSsQQliQpEUNYkqREDGFJkhIxhCVJSqSuWxmqMSGEjcAaYBy4\nOMa4ddK6B4BfAKPVp/5zjPFXze5js4UQTgD+EdgYY/zslHWvBf6KSk1ujTFemaCLTTVLPR6gzcZI\nCOFq4NVU9lEfjzHeNGldO46PvdXjAdpkfIQQuoAvA4cA+wNXxhj/adL6lhsbhvAcCyGcARRijKeG\nEFYCX+T5919eH2MsNb93aYQQFgPXArfPsMkmoA/4FfDdEMLXY4z3Nqt/zVZHPaCNxkgI4TXACdX3\nzMHAj4GbJm3SbuNjtnpA+4yPPwJ+FGO8OoRwFPDPwD9NWt9yY8Pp6Lm3DrgFIMY4CHSHEJam7VJy\nzwBnAjumrgghHAvsijH+IsY4BtxKpYYL2Yz1aFP/Cry1uvwbYHEIoRPadnzMWI92E2P8aozx6urD\nI4BfTqxr1bHhkfDcOxQYmPR4uPrcE5Oe+3wI4Wjge8CHYowL+jJmMcYyUA4hTLf6UCo1mrATOK4Z\n/UpllnpMaJsxEmMcBZ6sPtxAZVpxYqq1HcfH3uoxoW3GB0AIYQvwYuCsSU+35NjwSLj5Fk15fDnw\nfmAtcAJwTrM7NM9NrVc7assxEkJ4E5XQuWgvm7XN+NhLPdpufMQYXwm8EfjfIYSZxkBLjA2PhOfe\nDiqf0CYsBx6eeBBj/F8TyyGEW4GXAl9rWu/mn6n1Opw2n6ZtxzESQugDLgNeH2N8fNKqthwfe6lH\nW42PEEIvsLM65fyTEEIO6KFy1NuSY8Mj4bl3G3AuQAjhRGBHjHF39fHvhxD6Qwgvqm57BvCzNN2c\nH2KMDwBLQwhHV99gZ1GpYVtqxzESQvh94JPAWTHGXZPXteP42Fs92nB8nA78d4AQwiFAHngUWnds\neBelJggh/DWVwTMGXAi8Ang8xnhzCOFi4L8AT1H51uN/a4PzOb3ANcDRwHNUvsn4DeD+ak1OBz5R\n3fzrMca/SdLRJqmjHm01RkIIfwZcAQxNevo7wE/bdHzMVo+2GR8hhAOAG6h8KesA4GPAwfxuf9py\nY8MQliQpEaejJUlKxBCWJCkRQ1iSpEQMYUmSEjGEJUlKxBCWJCkRQ1iSpEQMYUmSEvn/yQtrkrok\nJOIAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f5e778c57b8>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "ApYZ83IPmACS",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "---\n",
        "**Define the RNN.**\n",
        "\n",
        "**Next, I will define an RNN in PyTorch. I will use `nn.RNN` to create an RNN layer, then I will add a last, fully-connected layer to get the output size that I want.** \n",
        "\n",
        "**An RNN takes in a number of parameters:**\n",
        "\n",
        "* **input_size** - the size of the input\n",
        "* **hidden_dim** - the number of features in the RNN output and in the hidden state\n",
        "* **n_layers** - the number of layers that make up the RNN, typically 1-3; greater than 1 means that you'll create a stacked RNN\n",
        "* **batch_first** - whether or not the input/output of the RNN will have the batch_size as the first dimension (batch_size, seq_length, hidden_dim)"
      ]
    },
    {
      "metadata": {
        "id": "L5JrgtVimACU",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Define an RNN class with PyTorch\n",
        "class RNN(nn.Module):\n",
        "    def __init__(self, input_size, output_size, hidden_dim, n_layers):\n",
        "        super(RNN, self).__init__()\n",
        "        \n",
        "        self.hidden_dim=hidden_dim\n",
        "\n",
        "        ### Define RNN parameters\n",
        "        ### Batch_first means that the first dim of the input and output will be the batch_size\n",
        "        self.rnn = nn.RNN(input_size, hidden_dim, n_layers, batch_first=True)\n",
        "        \n",
        "        ### Last, fully-connected layer\n",
        "        self.fc = nn.Linear(hidden_dim, output_size)\n",
        "\n",
        "    def forward(self, x, hidden):\n",
        "        ### x (batch_size, seq_length, input_size)\n",
        "        ### Hidden (n_layers, batch_size, hidden_dim)\n",
        "        ### r_out (batch_size, time_step, hidden_size)\n",
        "        batch_size = x.size(0)\n",
        "        \n",
        "        ### Get RNN outputs\n",
        "        r_out, hidden = self.rnn(x, hidden)\n",
        "        \n",
        "        ### Shape output to be (batch_size*seq_length, hidden_dim)\n",
        "        r_out = r_out.view(-1, self.hidden_dim)  \n",
        "        \n",
        "        ### Get final output \n",
        "        output = self.fc(r_out)\n",
        "        \n",
        "        return output, hidden"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NFort7kCmACX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Check the input and output dimensions.**\n",
        "\n",
        "**As a check that my model is working as expected, test out how it responds to input data.**"
      ]
    },
    {
      "metadata": {
        "id": "AGjv4eyjmACZ",
        "colab_type": "code",
        "outputId": "6779f7de-66ce-4d65-dfa4-027c5d2d1c3e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "### Test that dimensions are as expected\n",
        "test_rnn = RNN(input_size=1, output_size=1, hidden_dim=10, n_layers=2)\n",
        "\n",
        "### Generate evenly spaced, test data pts\n",
        "time_steps = np.linspace(0, np.pi, seq_length)\n",
        "data = np.sin(time_steps)\n",
        "data.resize((seq_length, 1))\n",
        "\n",
        "test_input = torch.Tensor(data).unsqueeze(0) # give it a batch_size of 1 as first dimension\n",
        "print('Input size: ', test_input.size())\n",
        "\n",
        "### Test out rnn sizes\n",
        "test_out, test_h = test_rnn(test_input, None)\n",
        "print('Output size: ', test_out.size())\n",
        "print('Hidden state size: ', test_h.size())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Input size:  torch.Size([1, 20, 1])\n",
            "Output size:  torch.Size([20, 1])\n",
            "Hidden state size:  torch.Size([2, 1, 10])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "IXzkcIgnmACd",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Training the RNN.**\n",
        "\n",
        "**Next, I will instantiate an RNN with some specified hyperparameters. Then train it over a series of steps, and see how it performs.**"
      ]
    },
    {
      "metadata": {
        "id": "10qDO5IUmACe",
        "colab_type": "code",
        "outputId": "04ac9f60-dfcb-48cf-9a69-119af1019f9e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "### Set the hyperparameters\n",
        "input_size=1 \n",
        "output_size=1\n",
        "hidden_dim=32\n",
        "n_layers=1\n",
        "\n",
        "### Instantiate an RNN\n",
        "rnn = RNN(input_size, output_size, hidden_dim, n_layers)\n",
        "print(rnn)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RNN(\n",
            "  (rnn): RNN(1, 32, batch_first=True)\n",
            "  (fc): Linear(in_features=32, out_features=1, bias=True)\n",
            ")\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "5av0j9G2mACi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Loss and Optimization.**\n",
        "\n",
        "**This is a regression problem: can I train an RNN to accurately predict the next data point, given a current data point?**\n",
        "\n",
        ">*  **The data points are coordinate values, so to compare a predicted and ground_truth point, I will use a regression loss: the mean squared error.**\n",
        "* **It's typical to use an Adam optimizer for recurrent models.**"
      ]
    },
    {
      "metadata": {
        "id": "TEBP1vMYmACj",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Define a loss function\n",
        "criterion = nn.MSELoss()\n",
        "\n",
        "### Define an optimizer\n",
        "optimizer = torch.optim.Adam(rnn.parameters(), lr=0.01) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Dx8NQZ4kmACn",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Defining the training function.**\n",
        "\n",
        "**This function takes in an rnn, a number of steps to train for, and returns a trained rnn.** \n",
        "\n",
        "**This function is also responsible for displaying the loss and the predictions, every so often.**\n",
        "\n",
        "**Hidden State.**\n",
        "\n",
        "**Pay close attention to the hidden state, here:**\n",
        "* **Before looping over a batch of training data, the hidden state is initialized.**\n",
        "* **After a new hidden state is generated by the rnn, I get the latest hidden state, and use that as input to the rnn for the following steps.**"
      ]
    },
    {
      "metadata": {
        "id": "58bmoxpTmACo",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Training function\n",
        "def train(rnn, n_steps, print_every):\n",
        "    \n",
        "    ### Initialize the hidden state\n",
        "    hidden = None      \n",
        "    \n",
        "    for batch_i, step in enumerate(range(n_steps)):\n",
        "        ### Defining the training data \n",
        "        time_steps = np.linspace(step * np.pi, (step+1)*np.pi, seq_length + 1)\n",
        "        data = np.sin(time_steps)\n",
        "        data.resize((seq_length + 1, 1)) # input_size=1\n",
        "\n",
        "        x = data[:-1]\n",
        "        y = data[1:]\n",
        "        \n",
        "        ### Convert data into Tensors\n",
        "        x_tensor = torch.Tensor(x).unsqueeze(0) # unsqueeze gives a 1, batch_size dimension\n",
        "        y_tensor = torch.Tensor(y)\n",
        "\n",
        "        ### Outputs from the rnn\n",
        "        prediction, hidden = rnn(x_tensor, hidden)\n",
        "\n",
        "        ### Representing Memory ##\n",
        "        ### Make a new variable for hidden and detach the hidden state from its history\n",
        "        ### This way, I don't backpropagate through the entire history\n",
        "        hidden = hidden.data\n",
        "\n",
        "        ### Calculate the loss\n",
        "        loss = criterion(prediction, y_tensor)\n",
        "        \n",
        "        ### Zero the gradients\n",
        "        optimizer.zero_grad()\n",
        "        \n",
        "        ### Perform backprop and update weights\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "\n",
        "        ### Display the loss and predictions\n",
        "        if batch_i%print_every == 0:        \n",
        "            print('Loss: ', loss.item())\n",
        "            plt.plot(time_steps[1:], x, 'r.') # input\n",
        "            plt.plot(time_steps[1:], prediction.data.numpy().flatten(), 'b.') # predictions\n",
        "            plt.show()\n",
        "    \n",
        "    return rnn"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kTskr34LmACz",
        "colab_type": "code",
        "outputId": "21c78047-83eb-4b65-e36d-2908a101b21a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1342
        }
      },
      "cell_type": "code",
      "source": [
        "### Train the RNN model and monitor results\n",
        "n_steps = 75\n",
        "print_every = 15\n",
        "\n",
        "trained_rnn = train(rnn, n_steps, print_every)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Loss:  0.7177762985229492\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAD4CAYAAADmWv3KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFGNJREFUeJzt3X+sZOdd3/H3eqfQrOcumkZjtjYY\nO9Hdr9ebCNVbJ97GiZeum2XBtEIxqkQLdeSoIhhkWoRkGrUqjZpAqLuq4Y+CoIqqitQSYBNg3d3W\nwUnQ/rPcFIs6m+/e4iwxrCXf4Iu9k3VD7u72j5lJhsveuztnZu4589z3S7I8c37N95nnnM+cfe6c\nMzuuXLmCJKk8N9RdgCRpNgx4SSqUAS9JhTLgJalQBrwkFapVdwFDKysXivg6T6ezi9XVi3WXMRW2\npZlsSzPV1ZZud2HHRvM8g5+yVmtn3SVMjW1pJtvSTE1siwEvSYUy4CWpUAa8JBXKgJekQhnwklSo\nib4mGRFvA34LOJaZv7hu3v3AR4BLwPHM/PAkryVJGk/lM/iIuBH4BeDZDRZ5Angf8C7gvRFxZ9XX\nkhqr16O1dBp6vborkf6aSYZovgp8D3B+/YyIeAvwama+lJmXgePA4QleS2qeXo/OkUN0jh6mc+SQ\nIa/GqTxEk5lrwFpEXG32HmBl5PkrwFs3216ns6uRFwpU0e0u1F3C1BTfll4PXngB9u+Hdnu8Db74\neVg+C0Br+SzdV74Et79z/MIq1FB8v8ypprVlq25VsOGltEOlXK7c7S6wsnKh7jKmovi2DM7AW8tn\nWVvcy+qJ58YL+ZtupbO49xvr33QrjPt+Vaih+H6ZU3W1ZbMPlVl9i+Y8/bP4oVu4ylCOVKdWnqE1\ncgbeyjPjbaDdZvXEc6w+8+z4Hw7TqkHaxEwCPjPPAbsj4raIaAEPACdn8VpSVWuxj7XFvf3Hi3tZ\ni33jb6TdZu3A3ZXCfWo1SBuoPEQTEQeAx4HbgK9FxIPAJ4EvZuZTwAeBTwwWfzIzz05YqzRdgzPw\nVp7pB2vFkJ77GlSsSf7IugQc2mT+Z4CDVbcvbYnhGfh2r0FF8kpWzTe/h+57oA0Z8Jpffg/d90Cb\nMuA1t/wGiu+BNmfAa275DRTfA22uMb/JKo3Nb6D4HmhTBrzmm99A8T3QhhyikaRCGfCSVCgDXpIK\nZcBLUqEMeNXLqzDrZx8Uy4BXfbwKs372QdEMeNXGqzDrZx+UzYBXbbwKs372Qdm80En18SrM+tkH\nRTPgVS+vwqyffVAsh2gkqVAGvCQVyoCXpEIZ8JJUKANekgplwEtSoQx4SSrURN+Dj4hjwD3AFeDR\nzDw9Mu8R4J8Cl4A/yMyfmOS11FC9nhfJbHfuA41V+Qw+Iu4DFjPzIPAw8MTIvN3ATwHvzsx7gTsj\n4p5Ji1XDeKMquQ802iRDNIeBpwEy8wzQGQQ7wF8O/mtHRAvYBbw6SaFqHm9UJfeBZptkiGYPsDTy\nfGUw7fXM/H8R8TPAi8AbwH/PzLObbazT2UWrtXOCcpqj212ou4Sp2bQt974D7rgDvvAFuOMOOve+\no9H/RN82/bKVprAPNKYtU9C0tkzzXjQ7hg8GZ/L/CtgLvA58KiK+MzOf32jl1dWLUyylPt3uAisr\nF+ouYyquqy3HP/WN8dc3rsAbzWz7tuuXrTTBPtC4tkygrrZs9qEyScCfp3/GPnQz8PLg8T7gxcz8\nMkBEfBY4AGwY8JpT3qhK7gONNckY/EngQYCIuAs4n5nDj69zwL6IeNPg+d8Flid4LUnSmCqfwWfm\nqYhYiohTwGXgkYh4CHgtM5+KiJ8Hfi8i1oBTmfnZ6ZQsSboeE43BZ+Zj6yY9PzLvl4BfmmT7kqTq\nvJJVkgplwEtSoQx4SSqUAS9JhTLgJalQBrwkFcqAl6RCGfDbXa9Ha+m0t3lVfdwHZ8aA3868l7fq\n5j44Uwb8Nua9vFW7F15wH5whA34bW4t9rC3u7T9e3Nu/3au0lfbvdx+coWneD17zpt1m9cRz/p6m\n6uM+OFMG/HbnvbxVN/fBmXGIRpIKZcBLUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQhnwklQoA16S\nCmXAS1KhKt+qICKOAfcAV4BHM/P0yLxvBz4BfBPwucz8kUkLlSSNp9IZfETcByxm5kHgYeCJdYs8\nDjyeme8ALkXErZOVKUkaV9UhmsPA0wCZeQboRMRugIi4AXg38MnB/Ecy80tTqFWSNIaqQzR7gKWR\n5yuDaa8DXeACcCwi7gI+m5k/fa0Ndjq7aLV2ViynWbrdha17sV4PXngB9u+fya1Wt7QtM2Zbmmni\ntsz4GBhH0/plWrcL3rHu8S3AfwLOAb8bEd+bmb+72QZWVy9OqZR6dbsLrKxc2JoXG/zcWWv5LGuL\ne1k98dxUd/AtbcuM2ZZmmrgtMz4GxlFXv2z2oVJ1iOY8/TP2oZuBlwePvwz8SWb+cWZeAp4F9ld8\nHW3Cn9zTducxsLmqAX8SeBBgMAxzPjMvAGTmGvBiRCwOlj0A5KSF6q/zJ/e03XkMbK7SEE1mnoqI\npYg4BVwGHomIh4DXMvMp4CeAjw/+4PpHwG9Pq2CN8OfOtN15DGyq8hh8Zj62btLzI/P+L3Bv1W1r\nDP7cmbY7j4ENeSWrJBXKgJekQhnwklQoA16SCmXAS1KhDHhJKpQBL0mFMuAlqVAGvCQVyoCXpEIZ\n8JJUKANekgplwEtSoQx4SSqUAS9JhTLg69br0Vo63f/hYElbr+Bj0ICv0+AHgztHD9M5cqjIHUxq\ntMKPQQO+Rv5gsFSv0o9BA75G/mCwVK/Sj8HKv8mqKfAHg6V6FX4MGvB18weDpXoVfAw6RCNJhTLg\nJalQlYdoIuIYcA9wBXg0M09fZZmPAgcz81DlCiVJlVQ6g4+I+4DFzDwIPAw8cZVl7gTeM1l5kqSq\nqg7RHAaeBsjMM0AnInavW+Zx4EMT1CZJmkDVIZo9wNLI85XBtNcBIuIh4NPAuevdYKezi1ZrZ8Vy\nmqXbXai7hKmxLc1kW5qpaW2Z1tckdwwfRMTfAt4P3A/ccr0bWF29OKVS6tXtLrCycqHuMqbCtjST\nbWmmutqy2YdK1SGa8/TP2IduBl4ePP77QBf4LPAUcNfgD7KSpC1UNeBPAg8CRMRdwPnMvACQmb+e\nmXdm5j3A9wOfy8x/MZVqJUnXrVLAZ+YpYCkiTtH/Bs0jEfFQRHz/VKuTJFVWeQw+Mx9bN+n5qyxz\nDjhU9TUkSdV5JaskFcqAl6RCGfCSVCgDXpIKZcBLUqEMeEkqlAEvSYUy4CfV69FaOg29Xt2VSKpD\ngzPAgJ9Er0fnyCE6Rw/TOXKokR0saYZGMoC7725cBhjwE2jlGVrLZ/uPl8/SyjM1VyRpK41mAF/4\nQuMywICfwFrsY21xb//x4l7WYl/NFUnaSqMZwB13NC4DpnU/+O2p3Wb1xHO08ky/Y9vtuiuStJVG\nMqBz7zvgjSt1V/RXGPCTardZO3B33VVIqsswA9pteKNZP17iEI0kFcqAl6RCGfCSVCgDXpIKZcBL\nUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQlW+VUFEHAPuAa4Aj2bm6ZF53wV8FLgEJPCBzLw8Ya2S\npDFUOoOPiPuAxcw8CDwMPLFukV8GHszMdwELwHdPVKUkaWxVh2gOA08DZOYZoBMRu0fmH8jMPx08\nXgHeXL1ESVIVVYdo9gBLI89XBtNeB8jM1wEi4m8D7wX+9bU22OnsotXaWbGcZul2F+ouYWpsSzPZ\nlmZqWlumdbvgHesnRMRNwG8DP5qZf36tDayuXpxSKfXqdhdYWWnWLUOrsi3NZFuaqa62bPahUjXg\nz9M/Yx+6GXh5+GQwXPMM8KHMPFnxNSRJE6g6Bn8SeBAgIu4Czmfm6EfX48CxzPwfE9YnSaqo0hl8\nZp6KiKWIOAVcBh6JiIeA14ATwA8DixHxgcEqv5aZvzyNgiVJ16fyGHxmPrZu0vMjj7+56na3XK/n\nb6pKqs8MM2h7X8na69E5cojO0cN0jhyCXq/uiiRtJzPOoG0d8K08Q2v5bP/x8llaeabmiiRtJ7PO\noG0d8Guxj7XFvf3Hi3v7/0SSpC0y6wya1vfg51O7zeqJ5xyDl1SPGWfQ9g54gHabtQN3112FpO1q\nhhm0rYdoJKlkBrwkFcqAl6RCGfCSVCgDXpIKZcBLUqEMeEkqlAEvSYUy4CWpUAa8JBXKgJekQhnw\nklQoA16SCmXAS1KhDHhJKpQBL0mFMuAlqVAGvCQVqvJP9kXEMeAe4ArwaGaeHpl3P/AR4BJwPDM/\nPGmhkqTxVDqDj4j7gMXMPAg8DDyxbpEngPcB7wLeGxF3TlTlZno9Wkunodeb2UtI0jyqOkRzGHga\nIDPPAJ2I2A0QEW8BXs3MlzLzMnB8sPz09Xp0jhyic/QwnSOHDHlJGlF1iGYPsDTyfGUw7fXB/1dG\n5r0CvPVaG+x0dtFq7Ryvihc/D8tnAWgtn6X7ypfg9neOt40Z6HYX6i5hamxLM9mWZmpaWyqPwa+z\no+K8r1tdvTj+q950K53FvbSWz7K2uJfVm26FlQvjb2eKut0FVmquYVpsSzPZlmaqqy2bfahUDfjz\n9M/Uh24GXt5g3i2DadPXbrN64jlaeYa12Aft9kxeRpLmUdUx+JPAgwARcRdwPjMvAGTmOWB3RNwW\nES3ggcHys9Fus3bgbsNdktapdAafmaciYikiTgGXgUci4iHgtcx8Cvgg8InB4k9m5tmpVCtJum6V\nx+Az87F1k54fmfcZ4GDVbUuSJueVrJJUKANekgplwEtSoQx4SSqUAS9JhTLgJalQBrwkFcqAl6RC\nGfCSVCgDXpIKZcBLUqEMeEkqlAEvSYUy4CWpUAa8JBXKgNe21uvB0tIN9Hr1rN+EGupeX7NjwKtW\ndYZLrwdHjuzi6NEbOXJk19jbmHT9JtRQ9/rDbfgBMxsG/IS2+845zwGbeQPLyzsBWF7eSeZ4h8Ok\n6zehhrrXr3sfGG5jno/BzRjwE2jCzjmpOgO67nCJuMzi4iUAFhcvEXF5S9dvQg11r//CC8z1B8xw\nG039gKj8m6y6+s51++2TrX/gwHgHSK/X307EZdrtsVb9+s69vLyTxcVLnDhxcaxtTFr/MByGr181\nXKqu327DiRMXK79/k67fhBrqXn//fmrdBybdh0ePoTvugOPHqXQcTrIPbWbHlStXprvFilZWLtRS\nyLQD8vbbF1hZuVB5/XFqmHT9paUbOHr0xq8/f+aZr/yVnbvb3bwtk77+cBuT7NzXu/612jJPSmvL\nF794YUv2gY3WneUxNOvXB+h2F3ZsNG9bB/wsAmrcg2+SnXPWO9f1tGWWZx/TVFoo2pbpmdYHRP8M\n/sKWfkDA5gG/rYdopjFE0m4z9jrTWr/uIYrhNiZpv1S3Sfbh0WPo3ntv5I03xlt/0mP4WioFfET8\nDeDjwHcAl4D3Z+aL65b5x8BPApeBZzPzQ5OVOn2zfnNnzYCW6jc8htptxg74aRzDm6n6LZofBP4i\nM+8F/j3w0dGZEbEL+DngMHAQuD8i7pyk0FkYvrnPPPOVSsMzTTC6c0maP7M8hqsG/GHgqcHj/wW8\na3RmZl4E3p6ZFzLzCvDnwJsrVzlDBqSkUlX6I2tEnAR+KjOfHzx/CXhrZv7lVZZ9O/Ak8J2Z+bWN\ntrm2dulKq7Vz7FokaZur/kfWiPgA8IF1k995PS8QEYvArwE/uFm4A6yuXrxWKXOhCd8KmBbb0ky2\npZnqaku3u7DhvGsGfGb+CvAro9Mi4uPAHuD5wR9cd6w/e4+IbwOeBn4oM/9w/LIlSZOoOgZ/EviB\nwePvA37vKsv8KvDBzPxcxdeQJE2g6vfgnwT+QUT8PvBV4CGAiHgM+DT9P6q+G/h3ETFc5z9m5icn\nqvYq5uVCG0naapUCPjMvAe+/yvSfHXm6q2pR12saV6JKUqnm+m6S07hdqySVaq4TcRq3a5WkUs31\nvWhmfZmvJM2zuQ548F4qkrSRuR6ikSRtzICXpEIZ8JJUKANekgplwEtSoQx4SSpUY350W5I0XZ7B\nS1KhDHhJKpQBL0mFMuAlqVAGvCQVyoCXpEIZ8JJUqLm/XXCdIuIYcA9wBXg0M0+PzDsHvARcGkz6\nJ5n5Z1td4/WKiLcBvwUcy8xfXDfvfuAj9NtyPDM/XEOJ1+0abTnHfPXLx+j/vnEL+Ghm/ubIvHnr\nl83aco456ZeI2AV8HPhW4G8CH87M3xmZ35h+MeArioj7gMXMPBgR+4D/Ahxct9jRzOxtfXXjiYgb\ngV8Ant1gkSeAI8CfAZ+OiN/IzM9vVX3juI62wPz0y3cBbxvsY28G/jfwmyOLzFO/XKstMCf9Anwf\n8AeZ+bGI+A7gfwK/MzK/Mf3iEE11h4GnATLzDNCJiN31llTZV4HvAc6vnxERbwFezcyXMvMycJx+\n25tqw7bMoc8APzB4/BfAjRGxE+ayXzZsy7zJzCcz82ODp98O/OlwXtP6xTP46vYASyPPVwbTXh+Z\n9p8j4jbg94GfzsxG3hciM9eAtYi42uw99Ns29Arw1q2oq4prtGVoXvrlEvCVwdOH6f9zfziEMW/9\nsllbhuaiX4Yi4hTwbcADI5Mb1S+ewU/PjnXP/w3wL4FDwNuA9211QTOyvp3zZu76JSL+Ef1Q/LFN\nFpuLftmkLXPXL5n594B/CPy3iNjo/a+1XzyDr+48/U/roZuBl4dPMvO/Dh9HxHHg7cCvb1l107O+\nnbcwx8Mf89YvEXEE+BDw3Zn52sisueuXTdoyV/0SEQeAVwbDMH8YES2gS/9svVH94hl8dSeBBwEi\n4i7gfGZeGDz/log4ERHfNFj2PuD/1FPmZDLzHLA7Im4b7MgP0G/73Jm3fomIbwF+HnggM18dnTdv\n/bJZW+atX4D3AD8JEBHfCrSBL0Pz+sXbBU8gIn6WfmdfBh4B/g7wWmY+FRGPAv8MeIP+NwZ+vKlj\nioMzkseB24Cv0f/r/yeBLw7a8h7g5waL/0Zm/odaCr0O19GWeeqXfw78W+DsyORPAX80h/1yrbbM\nU7+8CfhV+n9gfRPwM8Cb+cax35h+MeAlqVAO0UhSoQx4SSqUAS9JhTLgJalQBrwkFcqAl6RCGfCS\nVKj/D+RbpEEH0+zuAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f5e77843438>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Loss:  0.00943851564079523\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAD4CAYAAADmWv3KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAF5lJREFUeJzt3X9wHOddx/G3pHPS2qdOVeZclxpC\nIk7fyG6npsZxPEmJwHFF2magLWmn6UDj1lMoHegPKJMODD9aQqD86rTDMBQXDMykeDpD2gKh56TE\noW4SY5SJKUL6aitTGieFHK1cvJET+2Txx52CctFJvtXd7d5zn9c/3rvdu/s+3tNHj57dfbZvcXER\nEREJT3/aBYiISHso4EVEAqWAFxEJlAJeRCRQCngRkUDl0i5gSbl8NpOn8wwNbWRubj7tMlpCbckm\ntSWbuqUthcJgX6N16sGvIZcbSLuEllFbskltyaYQ2qKAFxEJlAJeRCRQCngRkUAp4EVEAqWAFxEJ\nVOLTJM3sD4FrgUXgfe5+Ytm6G4HfAhaAe9z9o+stVEREmpOoB29mNwBFd98DvAv4RN0mnwDeDFwH\nvNbMtq2rytXEMbmJExDHbfsIEZFulHSIZi/wOQB3nwKGzOxFAGZ2FfBtd3/M3S8C99S2b704Zmh8\njKGb9jI0PqaQFxFZJukQzRZgYtnjcu25/639W1627klgeK03HBra2PyFBaf+HaIZAHLRDIUnvwFX\n7m7uPS5BoTDY8vdMi9qSTWpLNnV7W1o1VUHDS2XXWPesRJcEb/5eNgzvwGcvw4bPc2Hz90L5bPPv\ns4pCYZByi98zLWpLNqkt2dQtbVntl1DSgH+Cak99yXcD32yw7uW151ouJs84J4jIUaRCiXPk2/FB\nIiJdKOkY/BHgJwDM7NXAE+5+FsDdvw68yMy+z8xywBtq27ecez/RbPV3VDSbw11nfYqILEmUiO7+\nIDBhZg9SPWPmvWZ2m5m9sbbJe4DPAF8GDrv7TEuqrWN2kWJxAYBicQGzi+34GBGRrpR4DN7db697\n6uSydf8E7En63pcqn4dSaR73fswuktf4jIjIszIzH3xS+Tzs3Kmeu4hIPQ1ai4gESgEvIhIoBbyI\nSKAU8CIigVLAi4gESgEvIhIoBbyISKAU8CIigVLAi4ikKI5hYqK/Lbez6PorWUVEulUcw/j4RqJo\ngGJxgVJpvqVTrqgHLyKSEvd+oqh6o6MoGmj5jLgKeBGRlLR7RlwN0YiIpKTdM+Iq4EVEUtTOGXE1\nRCMiEigFvIhIoBTwIiKBUsCLiARKAS8iEigFvIhIoBTwIiKBUsCLiARKAR/H5CZO0Jap3ERE1tLG\nDOrtgI9jhsbHGLppL0PjYwp5EemsNmdQTwd8zqd4Onqc41zD09Hj5Hwq7ZJEpIfkfIpcNFNdjmZa\nnkE9HfBntm5j14aTXMtxdm04yZmt29IuSUR6SMVGqRRHqsvFESo22tL37+nJxvz0INMXtgAwfWEY\nP/0UO1/ankl/RESeJ59nrnSUnE9Vw73F00n2dA++3XMxi4isKZ+nsnNXy8MderwH3+65mEVE0pQo\n4M1sA3AIuAJYAPa7+6m6bd4K/AJwEfiSu//y+kptj3bOxSwikqakQzS3Amfc/XrgDuDO5SvNbCPw\nO8BeYA9wo5npCKaISAclDfi9wN215fuA65avdPd54JXuftbdF4FvAd+VuEoREWla3+LiYtMvMrMj\nwIfc/WTt8WPAsLufX2HbVwKHgVe5+4VG71mpLCzmcgNN1yIi0uP6Gq1YcwzezA4AB+qe3n0pH2Bm\nReAu4NbVwh1gbm5+rVJSUSgMUi6fTbuMllBbskltyaZuaUuhMNhw3ZoB7+4HgYPLnzOzQ8AW4GTt\ngGtffe/dzLYCnwN+0t0fbb5sERFZj6Rj8EeAW2rLNwP3r7DNp4H3uPsjCT9DRETWIel58IeBfWZ2\nDHgGuA3AzG4HHqB6UPU1wEfMbOk1f+DuX1hXtSIicskSBby7LwD7V3j+t5c93Ji0KBERWb+enqpA\nRCRkCngRkUAp4EVEAqWAFxEJlAJeRCRQCngRkXWIY5iY6M/kLZ17ej54EZH1iGMYH99IFA1QLC5Q\nKs1n6r4S6sGLiCTk3k8UVSdJjKIB3LMVqdmqRkSki2T9tp8aohERSSjrt/1UwIuIrEOWb/upIRoR\nkUAp4EVEAqWAFxEJlAJ+veKY3MQJMnmVg4i0X4YzQAdZ1yOO2bDvdfjsZdjweS7cew+ZO4wuIu0T\nxwyNj5GLZqgUR5grHc1UBqgHvw5PPzrDntm7uJbj7Jm9i6cfnUm7JBHpoJxPkYuqP/e5aIacT6Vc\n0XMp4Ndhku1MMwrANKNMsj3likSkkyo2SqU4Ul0ujlCx0ZQrei4N0ayD7bic4nCFaDZHcbiC7bg8\n7ZJEpJPyeeZKR8n5VDXcMzQ8Awr4dcnnoXTvucxexSYiHZDPU9m5K+0qVqSAX6csX8UmIr1NY/Ai\nIoFSwIuIBEoBLyISKAW8iEigFPAiIoFSwIuIBEoBLyISKAW8iEigFPAiIoFKdCWrmW0ADgFXAAvA\nfnc/1WDbzwDPuPttCWsUEZEEkvbgbwXOuPv1wB3AnSttZGb7gOGEnyEiIuuQNOD3AnfXlu8Drqvf\nwMwuB34F+M2EnyEiIuuQNOC3AGUAd78ILJrZZXXbfBj4Y+B/k5cnIiJJrTkGb2YHgAN1T++ue9xX\n95oi8IPu/utmNnYphQwNbSSXG7iUTTuuUBhMu4SWUVuySW3Jpm5vS9/i4mLTLzKzQ8Bn3L1UO+D6\ndXd/+bL17wfeCcwDLwIKwO+6+8cavWe5fLb5QjqgUBikXD6bdhktobZkk9qSTd3SlkJhsK/RuqTz\nwR8BbgFKwM3A/ctXuvvHgY8D1Hrwt60W7iIi0npJx+APAwNmdgx4L9XxdszsdjPb06riREQkuUQ9\neHdfAPav8Pxvr/DcUeBoks8REWm3OCbY227qln0i0rPiGMbHNxJFAxSLC5RK80GFvKYqEJGe5d5P\nFFXP3ouiAdzDisSwWiMi0gSzixSLCwAUiwuYXUy5otbSEE3a4picT1GxUYL621CkC+TzUPqbMl+7\n7zTff+NW8vlNaZfUUgr4NMUxQ+Nj5KIZKsUR5kpHFfIinRTHfM+bxrgy0J9BDdGkKOdT5KKZ6nI0\nQ86nUq5IpLeE/jOogE9RxUY5M7yD41zDmeEd1WEaEemYio1SKY5Ul4sjwf0MaogmRTF5xjlBRI4i\nFUqcI5w/DkW6QD7PXOlosMfB1INPkXs/0Wz1d2w0mwvuFC2RrpDPU9m5K7hwBwV8qkI/RUtE0qUh\nmhTl81AqzQd7mbSIpEsBn7J8HnbuVM9dRFpPQzQiIoFSwIuIBEoBLyISKAW8iEigFPAiIoFSwIuI\nBEoBLyISKAW8iEigFPAiIoFSwIuIBEoBLyISKAW8iEigFPAiIoFSwIuIBEoBLyISKAW8iEigFPAi\n0tXiGCYm+onjtCvJHgV8t4tjchMn0LdbelEcw/i+F3LTTZsY3/dC/RjUUcB3szhmaHyMoZv2MjQ+\nppCXnuOPPkM0W73zaDSbwx99JuWKsiXRPVnNbANwCLgCWAD2u/upum1eBXy69vDz7v7RddQpK8j5\nFLloproczZDzKSo7d6VclUjnbGeSq9nMNKNczRTbeRJ4ddplZUbSHvytwBl3vx64A7hzhW0+Bbwb\nuAbYZmYbE36WNFCxUSrFkepycYSKjaZckUhnvWDHCA8N38rD7Oah4Vt5wY6RtEvKlEQ9eGAv8Je1\n5fuAP1u+0sxeCuTd/ZHaU29L+DmymnyeudLRas/dRiGfT7sikc7K57lw7z2M+BQX9DPwPH2Li4tN\nv8jMjgAfcveTtcePAcPufr72eDfwe8AsUAQ+6+4fX+09K5WFxVxuoOlael0cw+QkbN+u77ZIj+pr\ntGLNHryZHQAO1D29e40P6AOuBH4cOAc8ZGb3uvtko8+Zm5tfq5RUFAqDlMtn0y5jRXEM4+MbiaIB\nisUFSqX5VUM+y21pltqSTWpL5xUKgw3XrRnw7n4QOLj8OTM7BGwBTtYOuPYt9d5r/huYdPdv1bY/\nBmwHGga8NM+9nyiq/tUTRQO497Nz58WUqxKRrEh6kPUIcEtt+Wbg/uUr3f0/gEEze4mZ9QM7AE9c\npazI7CLF4gIAxeICZgp3Efl/SQ+yHgb21XrmzwC3AZjZ7cAD7v4Q8AHgH4BF4ItL4/XSOvk8lErz\nuPdjdlFj8CLyHIkOsrZDuXw2G4XU6ZZxuEuhtmST2pJN3dKWQmGw4UFWXckqIhIoBbyISKAU8CIi\ngVLAi4gESgEvIhIoBbyISKAU8CIigVLAi4gESgEvIhIoBbyISKAU8CIigVLAi4gESgEvIhIoBXyv\ni2NyEyeqt4cSSYO+g22TdD54CUEcMzQ+Ri6aoVIcYa50VDd2lc6KYzbsex0+exk2fJ4L996j72AL\nqQffw3I+RS6aqS5HM+R8KuWKpNc8/egMe2bv4lqOs2f2Lp5+dCbtkoKigO9hFRulUhypLhdHqNho\nyhVJr5lkO9NUv3fTjDLJ9pQrCouGaHpZPs9c6Sg5n6qGu/40lg6zHZdTHK4QzeYoDlewHZenXVJQ\nFPC9Lp+nsnNX2lVIj8rnoXTvOd1XuE00RNPj4hgmJvp1AoOkJp+HnTsV7u2gHnwPi2MYH99IFA1Q\nLC5QKs3rh0wkIOrB9zD3fqJoAIAoGsBdXweRkOgnuoeZXaRYXACgWFzA7GLKFYlIK2mIpofl81Aq\nzesAl0igFPA9bukAl4iER0M0IiKBUsCLiARKAS8iEigFvIhIoBTwIiKBSnQWjZltAA4BVwALwH53\nP1W3zR3AGNVfIne7+8fWVamIiDQlaQ/+VuCMu18P3AHcuXylmb0C+GF3vw64DthvZlvWVamIiDQl\nacDvBe6uLd9HNcSX+w7wAjO7HHgBcBGYT/hZIiKSQN/i4mLTLzKzI8CH3P1k7fFjwLC7n1+2zYeB\n9wEDwEfc/ZOrvWelsrCYyw00XYukLI5hchK2b9d88r1K34G09TVaseYYvJkdAA7UPb17tQ8ws6uA\nNwJXARuAB83ssLs/2ehz5uay2cEvFAYpl8+mXUZLtLwtKd7TVfslI+q+A7lHJiifa77TmEXdsl8K\nhcGG69YMeHc/CBxc/pyZHQK2ACdrB1z7lvfegV3AcXefr23/r8ArgH9stnjJrpXu6aqbh/SWnE/x\ndPQ4k1zD9miS/OQkXLUt7bKkJukY/BHgltryzcD9deu/BvygmfXXfgG8EjiFBEX3dJUzW7exa8NJ\nruU4uzacJL5C91TNkqSTjR0G9pnZMeAZ4DYAM7sdeMDdH6qN0x+rbX/Q3b++zlola3RP157npweZ\nvlA9QW76wjCT/wlXXZX9YY1ekeggazuUy2ezUUidbhmHuxTtaEsck8p0w9ov2VB/V7BHHhng3Lnu\nbEu9btkvhcJg8oOsIo3oln/y/HsKDHLuXNpVyRJNVSCJ6ZZ/ArppdpbpJ1IS0y3/RLJNQzSSmG75\nJ5JtCnhZF93yTyS7NEQjIhIoBbyISKAU8CIigVLAi4gESgEvIhIoBbykK47JTZyoXhYr6dA+CJYC\nXtJTm0t86Ka9DI2PKWDSoH0QNAW8pGal+eSls5bmcz/ONTwdPa59EBgFvKRG88mnr34+9zNbdbOO\nkOhKVklPPs9jf/MAX7vvNN9/41by+U1pV9Rz6udz99NPsfOlujI5FAp4SU0cw/ibCkTRFk03nJKl\nCeOWpnzWhHFhUcBLalaabljz2nSWJowLm8bgJTWabjgbNJ97uNSDl9So9yjSXgp4SZWmGxZpHw3R\niIgESgEvIhIoBbx0N82jov8DaUgBL91L86jo/0BWpYCXrqW5bDSXjKxOZ9FI16rYKGeGd+Czl2HD\n53tyLpszW7fxug0nmb4wzNUbZrln6yZ0tqksUQ9eulZMnms4wbUc5xpOEPdgtFXnkhkGluaSGUy5\nIskSBbx0Lfd+otnqH6HRbA733vs662pgWY2GaKRraaIsXQ0sq1PAS9dqSbjFMTmfqo7fp5WOtdMc\nk9agq4GlkcQBb2Y3AJ8F3unuf7fC+rcD7wcuAp9y908nrlKkgXWFW+0Uw1w0Q6U4wlzpaOdDPo7h\nh36Eoenp9GqQYCUatDSzYeCDwFcarN8E/CpwIzAGfMDMXpKwRpG2aMkphuu8yCjnU8TTj+k0R2mL\npEelvgm8CfhOg/W7gRPu/h13P0f1F8F1CT9LpC3Wfbu6FlxkdGbrNnZdplvmSXskGqJx93kAM2u0\nyRagvOzxk8DLVnvPoaGN5HIDScppu0IhnFPP1Jb/d+rUINMXqsvTF4Z58im4stDMG/w7cfQ4k1zD\n9miSwpPfgCt3N1/D+XXUkEH6jmXHmgFvZgeAA3VP/5q7l5r4nL61Npibm2/i7TqnUBikXD6bdhkt\nobY81+bNUCxufPYsnM2b5ymX137dknjTFc+9yGjTJvJN1rR5M1x99SDT0ySqIWv0Heu81X4JrRnw\n7n4QONjkZz5BtRe/5OXAw02+h0hbrfcsnFbcsDqfhxMn4Nixp3Sao7Rcu64MOQ7sMrMXm1me6vj7\nl9v0WSKJred2da26yEi3zJN2STQGb2avBz4EXA3sNLOfd/fXmtntwAPu/lBtuQQsAr/h7o0OyIp0\nJV1kJFnXt7i4mHYNAJTLZ7NRSJ1uGYe7FGpLNqkt2dQtbSkUBhse4+y9yTtERHqEAl5EJFAKeBGR\nQCngRUQCpYAXEQmUAl5EJFCZOU1SRERaSz14EZFAKeBFRAKlgBcRCZQCXkQkUAp4EZFAKeBFRAKl\ngBcRCVSi+eBDYmYvBP4N+CjwemDpjpgvAR5293cv2/a22naztafudfc7Olft6uracgr4LeAC8BTw\nk+4+t2zbDcAh4ApgAdjv7qc6XXMjTbblNrpnvzwMfIrqfRJmgPe4e6Vu+z8Erq1t8z53P9HZihtr\npi1mNgZ8FpisPfVVd/+5jha8gpXqAj4G/BUwAHyT6nfsmbrXZXa/NKIePPwK8G0Ad7/F3cfcfQz4\nF1a+VeHhpW2yFCI1z7YF+APgXe7+w8CDwE/XbXsrcMbdrwfuAO7sWJWXppm2QPfsl98B7nT3G4Bv\nAG9ZvqGZ3QAU3X0P8C7gE50s9BJccltqHli2X1IP92Xq6/oI8Efu/hrga8A7l2/cBftlRT0d8GZ2\nNbAN+Pu65w14sbv/cyqFJbBCW/4H+K7a8lDt8XJ7gbtry/dRva1iJiRoS2at0JYisPS9KgGvrXvJ\nXuBzAO4+BQyZ2Ys6UOqaErSlm4wBX6gt/y1wY936zO6X1fR0wAO/D3xwheffB3yywWtuMLMvmtmX\nzOwH2lda0+rb8gHgc2bmwGuoDscstwUoA7j7RWDRzC7rQJ2Xotm2QPfsl69SHQoEGAdeWrf9s/ul\npsxzb2CfpmbbArDNzL5gZsfMbF+7C2xCfV2blg3JPAm8rG77LO+Xhno24M3sp4CH3P0/6p6/DLje\n3e9f4WUPA7/u7j9K9U/Vv2x/pWtr0JZPAm90dwOOAT+7xts0vO1XJyVsSzftl18E3mJm/0j152+t\n//cs75e12hIBvwH8GPAO4NMZ6UQ8ry6eezzyUv7PM7Ff1tLLB1lfD1xlZm8AtgLPmNlpqjtuxaEZ\nd58GpmvLD5lZwcwG3H2hU0U38Ly2AEPu/pXa+nuBt9e95gmqPZCTtQOufe5+vlMFr6LptnTZfjnt\n7m8AMLNxnt9TXNovS76b6kG/tDXdFnd/HDhcezhrZv8FvBx4Tqeq0xrUtcvMXuju56jW+ETdy7K6\nX1bVsz14d3+ru+9y92upHkz9qLvfB+wCTq70GjP7JTN7W235FUA5AyGyYluA02a2rbbJLqq9luWO\nALfUlm8GVvqLpeOStKXL9strzGxpWGM/1fHe5Y4APwFgZq8GnnD31O/8nKQtZvZ2M/vF2vIWqkM4\nj3ew7BU1qOvPgTfXNnkz8MW6l2Vyv6ylZwN+FS+jOgb3LDP7fG3xLuDdZvYA8CdUj6Zn1c8Af2pm\nR4FXUzumsKwth4EBMzsGvBf4cBpFXqK12tJN++Uu4NfM7ATVkPh7ADP761oP8kFgwswepHqmxntT\nrHUtq7aF6kHLG8zsy8DnqZ5GmYW/Ep9XF/DLwDtqz70E+Avo2v3yLM0HLyISKPXgRUQCpYAXEQmU\nAl5EJFAKeBGRQCngRUQCpYAXEQmUAl5EJFD/B7CjJcujZgzqAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f5e777ad4a8>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Loss:  0.001342865522019565\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAD4CAYAAADmWv3KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFv5JREFUeJzt3X+MHGd9x/G37zahPc+lLLBpSCgk\nDntfX4wUmsON3QTiYsjGNAhFuKoKVTEKQtC0Cq1oG35UAqHCHyiySFu1oJaWtiREVEoIjcOZAg6x\nXIg5l4g65+9tnYRfDs0BV+KJHcjeuX/sbnxd7m5n53ZndmY+L8ny3M7MzvPcs/u5Z5+deWbDmTNn\nEBGR/BlJuwAiIjIYCngRkZxSwIuI5JQCXkQkpxTwIiI5VUr6gPPzJwd+2k65PMbCwqlBHyYRqstw\nUl2GU57rUqmMb+j1OXLZgy+VRtMuQt+oLsNJdRlOqsv/l8uAFxERBbyISG4p4EVEckoBLyKSUwp4\nEZGcUsCLiOSUAl4KIwxhZmaEMEy7JCLJSPxCJ5E0hCHUamPU66NUq4tMT58iCNIulchgqQcvheA+\nQr3evHCkXh/FvfeXvj4BSNaoBy+FYLZEtbr4bA/ebKmn/Ts/ARw5MqCCivSRAl4yIwybPXGzpZ6H\nV4IApqdPxd6/8xPA0aOwaVNvzyGSNA3RSCa0e9C7dm2kVhuLNUwSBDA11Xu4w9lPAADV6iJbtvT+\nHCJJi9SDN7OXAZ8D9rr7X3Wsew3wYWAR2OfuH+p7KaXwVhpDn5rqbZhlPX7+E8A4p08ndniRWLr2\n4M1sI/CXwJdW2eQ24I3AVcC1ZnZZ/4on0tTZg+51DL0fAkKu5OsE6FtWyYYoPfifAq8D/qxzhZlt\nAn7s7t9t/bwP2Ak83M9Ciqx3DH3dwpBybQel+hyN6gQcmYnzFOmVXwqpa8C7ewNomNlKqy8A5pf9\n/ARw6VrPVy6PJTJnc6UyPvBjJEV1ae8Ll1zSx8L04pGHoT4HQKk+B0ePUrnyysi7hyG86lVw7Bhs\n3gyHDzNUIa/X2HBab136fRZN1zuOJHG3lUplnPn5kwM/ThJUlz4KQ0o+S8Mme0/X819MuTrxbA++\ntGVLT3WZmRnh2LGNQDPkDx58KtHvENaServ0UZ7rEifs1xvwJ2j24tsuaj0mMlw6hlgWpg/0FvJB\nwML0gWf/QFSCAE5HDxJ70Uk2n/MDjj1zKZvPOY69aCOwsedqiPRiXadJuvtjwHlmdrGZlYDrgf39\nKJhIP5V8tjm0QnOIpeSzvT9JENCY2hprbOW533uYw89czte4ksPPXM5zv6evqWTwuvbgzWwKuBW4\nGHjGzHYD9wCPuvtdwDuBO1qb3+nucwMqq0hsDZtsDq20evANm0z8+OPVi7iy/iCN6gSnEz6+FFOU\nL1lngB1rrP8qsL2PZZKcas/lkspZJB1DLIkXIO3jSyFpqgJJxNmzSDamN5tje4glLWkfXwpHUxVI\nItxHOHasuRx3NkcR6Y3eZZIIsyU2b24ux74SNQwpzRymsPP1Fr3+0jMN0UgigqB5cc/Bg0/FG4Nf\n72mOWReGnPPa1+HHz8Uu/RnPfHFfseovsagHL4lZz2yOfTnNMcOe/uYc24/fzja+zvbjt/P0N3Wy\nmnSngJdMaJ/mCKRymmPajrKFYzTrfIxJjqL5iqU7DdFINhT8NEN7+XOoXtqgfrxE9dIG9vLnpF0k\nyQAFvGRHgU8zDAKY/uJpzUYpPVHAi2RE+zsMkag0Bi8iklMKeBGRnFLAi4jklAJeRCSnFPASWXs2\nyNhXyutS+3Tp9184CniJJAyhVhtj166N1GpjvWdEGMLWrZR37aRc26GQSVprqgf9/otFAS+RuI9Q\nrzdvlh5nNsiSz9KeTrKIUw2krehTPRSVAl4iMVuiWl0E4s0G2bBJ2tNJFnGqgbQVfaqHotKFThJJ\nEMD09Kn4V1K2ppNcOPhgIacaSF3Bp3ooKgW8RLbuKykLPNXAMAgJcK7EWELxXgwKeJECaH9JXq+P\npnfLREmcxuBFCmC9X5JLNqmVRQpgvV+SSzZpiEakANb9JblkkgJepCA03XDxaIhGRCSnFPAiIjml\ngBcRySkFvIhITingRURyKtJZNGa2F9gGnAFudvfDy9bdBPwusAh8w93fNYiCyvqFIes7TS4MNZdJ\nkan9M6drD97MrgGq7r4duBG4bdm684A/AV7p7lcDl5nZtkEVVuLrx3zumk+8wNT+mRRliGYncDeA\nu88C5VawA/ys9S8wsxIwBvx4EAWV9enHfO6aT7y41P7ZFOVdfgEwv+zn+dZjuPvTwAeBR4BvA193\n97l+F1LWrx/zuWs+8eJS+2dTnCtZN7QXWj359wITwJPAl83scnd/aLWdy+UxSqXRGIftTaUyPvBj\nJKUfdalU4MgROHoUtmwZJQh6fM7KOByZgaNHKW3ZQiXmGKzaZTh1rUuf2j8JhWqXLqIE/AlaPfaW\nC4HHW8uTwCPu/kMAM3sAmAJWDfiFhVPxStqDSmWc+fmTAz9OEvpdl02b4PTp5r94T3AZnD4Dp3sv\nk9plOPVUl3W0fxLy3C5xwj7KEM1+YDeAmV0BnHD39lEfAybN7BdbP78CqPdcChER6buuAe/uh4AZ\nMztE8wyam8xsj5nd4O7/A3wU+IqZHQT+090fGGyRRSQNYQgzMyM6gSZDIo3Bu/stHQ89tGzdx4GP\n97NQIjJcdEeobNKVrCLSle4IlU1qJRHpSneEyibd8ENEutIdobJJAS8ikeiOUNmjIRoRkZxSwIuI\n5JQCXkQkpxTwIiI5pYDPCF1FKCK90lk0GaCrCEUkDvXgM6BvVxGGIaWZw7obj6RDr7/EKeAzoC9X\nEeqWa5Imvf5SoYDPgPZVhPfd91Ts4Rndck3SpNdfOhTwGdG+ijDu2LtuuSZp0usvHfqStSiCgIXp\nA5R8tvnm0re0kiS9/lKhgC+SIKAxtTXtUkhR6fWXOA3RiIjklAJeRCSnFPAiIjmlgBcRySkFvIhI\nTingRURySgEvIonQjKjJ03nwIjJwmhE1HerBi8jA9W1GVOmJfssiMnB9mRFVeqYhGhEZuPaMqO4j\nmMWfNE96o4AXkUS0Z0SV5GiIJiE6g0BEkhapB29me4FtwBngZnc/vGzdrwB3AOcCR9z9HYMoaJbp\nDAIRSUPXHryZXQNU3X07cCNwW8cmtwK3uvuvAYtm9uL+FzPbdAaBiKQhStLsBO4GcPdZoGxm5wGY\n2QjwSuCe1vqb3P07AyprZukMAhFJQ5QhmguAmWU/z7ceexKoACeBvWZ2BfCAu7+n76XMOJ1BICJp\niHMWzYaO5YuAjwGPAfea2W+6+72r7Vwuj1EqjcY4bG8qlfGBH6MXlQpcckncfcebA/lHj8KWLZm+\n3dmwtct6qC4Ji/geyERdIlpvXaIE/AmaPfa2C4HHW8s/BL7t7scBzOxLwBZg1YBfWDgVr6Q9qFTG\nmZ8/OfDjJKFSGWf+0ccp13ZQqs/RqE6wMH0gkyGfu3ZRXZIThpHeA5moS0SddYkT9lHG4PcDuwFa\nwzAn3P0kgLs3gEfMrNradgrwnkshayr5LKX6XHO5PkfJZ1MukUiy9B6Ip2vAu/shYMbMDtE8g+Ym\nM9tjZje0NnkX8A+t9T8BPj+w0hZUwyZpVCeay9WJ5l3pRQpE74F4Io3Bu/stHQ89tGzdfwNX97NQ\n0iEIWJg+QMlnmy/sDA7PiKyL3gOxaKqCrAgCGlNb0y6FSHr0HuiZrrgREckpBbyISE4p4EVEckoB\nLyKSUwp4EZGcUsCLiOSUAl5EJKcU8CIiOaWAFxHJKQW8iEhOKeBFJBN04/reaS4aERl6unF9POrB\ni8jQ043r49FvKSJ9PBRJj25cH4+GaCLQx0ORdOnG9fGoBx+BPh6KpC8IYGpK4d4LJVUE+ngoIlmk\nIZoI9PFQRLJIAR9R++OhiEhWaIgmKWFIaeYwOg1HJCUFfA8q4JMQhpRrOyjv2km5tqNQLzCRoVDQ\n96ACPgEln6VUn2su1+co+WzKJRIplqK+BxXwCWjYJI3qRHO5OkHDJlMukUixFPU9qC9ZkxAELEwf\noOSzzReWTsMRSVZB34MK+KQEAY2prWmXQqS4Cvge1BCNiEhOKeBFRHJKAS8iklMKeBGRnIr0JauZ\n7QW2AWeAm9398ArbfATY7u47+lpCERGJpWsP3syuAaruvh24EbhthW0uA17V/+KJiEhcUYZodgJ3\nA7j7LFA2s/M6trkVeF+fyyYiIusQZYjmAmBm2c/zrceeBDCzPcD9wGNRDlguj1EqjfZUyDgqlfGB\nHyMpqstwUl2Gk+pyVpwLnTa0F8zsecBbgdcAF0XZeWHhVIxD9qZSGWd+/uTAj5ME1WU4qS7DKc91\niRP2UYZoTtDssbddCDzeWn41UAEeAO4Crmh9ISsiIimLEvD7gd0AZnYFcMLdTwK4+7+6+2Xuvg24\nATji7n80sNKKiEhkXQPe3Q8BM2Z2iOYZNDeZ2R4zu2HgpRMRkdgijcG7+y0dDz20wjaPATvWXyQR\nEekHXckqIpJTCngRkZxSwIuI5JQCXkQKIQxhZmakKPfbBgoU8EVsXBFpCkOo1cbYtWsjtdpYYXKg\nEAFf1MYVkSb3Eer15hQp9foo7oWIvmIEfFEbV0SazJaoVhcBqFYXMVtKuUTJKMRNt9uNW6+PFqpx\nRaQpCGB6+hTuI5gtEQRplygZhQj4ojauiJwVBDA1VazOXWHGKtqNGzvcw5DSzGE0gC9SUBnMgMIE\n/LqEIeXaDsq7dlKu7chUA4tIH2Q0AxTwEZR8llJ9rrlcn6PksymXSESSlNUMUMBH0LBJGtWJ5nJ1\ngoZNplwiEUlSVjOgEF+yrlsQsDB9gJLPNhtW39KKFEtGM0ABH1UQ0JjamnYpRCQtGcwADdGIiOSU\nAl5EJKcU8CIiOaWAFxHJKQW8iEhOKeBFRHJKAS8iklMKeBGRnFLAi4jklAJeRCSnFPAiIjmlgBcR\nySkFvIhITingRURyKtJ0wWa2F9gGnAFudvfDy9b9BvARYBFw4G3uXqw724qIDKGuPXgzuwaouvt2\n4Ebgto5NPgHsdvergHHgur6XUkREehZliGYncDeAu88CZTM7b9n6KXf/Xmt5Hnh+f4soIiJxRBmi\nuQCYWfbzfOuxJwHc/UkAM3shcC3w52s9Wbk8Rqk0GquwvahUxgd+jKSoLsNJdRlOqstZcW7Zt6Hz\nATM7H/g88Pvu/qO1dl5YOBXjkL2pVMaZnz858OMkQXUZTqrLcMpzXeKEfZSAP0Gzx952IfB4+4fW\ncM19wPvcfX/PJRARkYGIMga/H9gNYGZXACfcffmfyFuBve7+hQGUT0RkKIQhzMyMEIZplyS6rj14\ndz9kZjNmdghYAm4ysz3AT4Bp4PeAqpm9rbXL7e7+iUEVWEQkaWEItdoY9foo1eoi09OnCIK0S9Vd\npDF4d7+l46GHli0/p3/FWV0YgvsIZkuZ+MWKSH64j1CvN08OqddHcR9hamr4L/fJxJWs7b+eu3Zt\npFYbi/cRKQwpzRwmU5+vRGQomC1RrS4CUK0uYhYj3FPIoDhn0SRu3X89w5BybQel+hyN6gQL0wfQ\nxwARiSoIYHr6VPxRhJQyKBM9+PX+9Sz5LKX6XHO5PkfJZ/teRhHJtyCAqal4Q8RpZVAmAr791/O+\n+56K9eVGwyZpVCeay9UJGjY5gFKKiKwsrQzKxBANnP3rGXfnhekDlHy2+YvV8IyIJCmlDMpMwK9b\nENCY2pp2KUSkqFLIoEwM0YiISO8U8CIiOaWAFxHJKQW8iEhOKeBFRHJKAS8iklMKeBGRnFLAi4jk\nlAJeRCSnFPAiIjmlgBcRySkFvIhITingRURySgEvIpJTCngRkZxSwIuI5JQCXkQkpxTwIiI5pYAX\nEckpBbyISE4p4EVEckoBLyKSUwp4EZGcKkXZyMz2AtuAM8DN7n542brXAB8GFoF97v6hQRRURCTL\nwhDcRzBbIgiSOWbXHryZXQNU3X07cCNwW8cmtwFvBK4CrjWzy/peSoAwpDRzuPlbEhHJkDCEWm2M\nXbs2UquNJRZjUYZodgJ3A7j7LFA2s/MAzGwT8GN3/667LwH7Wtv3VxhSru2gvGsn5doOhbyIZIr7\nCPX6KAD1+ijuyYyORxmiuQCYWfbzfOuxJ1v/zy9b9wRw6VpPVi6PUSqN9lbKRx6G+hwApfoclSe+\nA5dcueYulcp4b8cYYqrLcFJdhtMw1uXqq2HzZjh2rPn/1VdvjDRMs966RBqD77Ah5joAFhZO9X7E\n819MuTpBqT5HozrBwvkvhvmTq25eqYwzv8b6LFFdhpPqMpyGuS779p0dgz99Gk6fXnv7zrrECfso\nAX+CZk+97ULg8VXWXdR6rL+CgIXpA5R8loZNktg3FCIifRIEMDW1lOgxowwE7Qd2A5jZFcAJdz8J\n4O6PAeeZ2cVmVgKub23ff0FAY2qrwl1EJKKuPXh3P2RmM2Z2CFgCbjKzPcBP3P0u4J3AHa3N73T3\nuYGVVkREIos0Bu/ut3Q89NCydV8FtvezUCIisn66klVEJKcU8CIiOaWAFxHJKQW8iEhObThz5kza\nZRARkQFQD15EJKcU8CIiOaWAFxHJKQW8iEhOKeBFRHJKAS8iklMKeBGRnIpzw49UmdkI8LfAy4Cf\nAe9w92OtdTXgC+6+oWOfHcBngaOth77l7n+YWKFXsVJdgFuAKeBHrc0+6u73duy36k3Q0xKnLhlr\nl+PAp4CXAieB3e6+0LFfVtplzbpkrF0+BFRamzwP+Jq7v33ZPucA/wi8BFgE3urujyRY7BXFrMue\n1jbHWw990d3/Yq3jZC7ggTcAv+Tuv25mlwIfA643s18A3sPZm5F0ut/ddydVyIhWqssPgfe4+7+t\ntMPym6Cb2STwSYZjNs+e69KSlXbZB8y7+5vM7O3AK4F72jtkrF3WrEtLJtrF3a9vrzSzTwJ/17HP\nm4D/dfc3m9m1wEeA306sxKuLUxdoTsn+7qgHyeIQTRV4EMDdjwMvMbNR4L3AX9P8a5gVP1cXoNsN\na1e9CXrK4tRlWK1Ul9cDn2499gl37wzELLVLt7oMq9Xe+5iZAc919wc79tkJ3NVa/nfgqoTK2k2c\nuvQsiwH/LaBmZqOtX8Qm4BXA5e7+2TX2u8zM7jGzg2b22kRK2t1KdXkB8Adm9mUz+4yZvaBjn84b\nnbdvgp62OHWB7LTLxcAuMzvQqsvzOvbJUrtczNp1gey0S/s1dTPwlyvs82y7uPsScMbMzk2isF3E\nqQvANWb2BTP7kpn9areDZC7g3f0+mn/5vgq8C5gFPgz88Rq71YEP0vxY9Bbg74ehkVepy78At7j7\nq4FvAh/o8jRdb3SehJh1yVK7bGiu8h3Af9EcDlzLMLdLt7pkql1aZbva3b8S4WmGul261OVrwAfc\n/Trg/cA/dTtOFsfgcff3t5fN7FHg+cCnm38IeaGZ3e/u1yzb/vvAna0fj5vZD2jeIPzR5Eq9so66\nHAfuaPU0oDku+jcdu6x1E/RU9VqXjLXLCeD+1kPTNANwuSy1y5p1yVi7PEFzGGa14Yx2uzzU+sJ1\ng7sPxTBur3VpnUxyrLX8H2ZWMbNRd19c7RiZ68Gb2eWtLyAws+uAb7j7Je6+zd23AY8vD/fWdm82\ns3e3li8Afhn4ftJl77RCXY4AnzWzTa1NdtDsYS236k3Q0xSnLhlrl3uB61qbTAHesVuW2mXNumSp\nXVodiK0su41oh/3Ab7WWXw9E6eUPXJy6mNmfmtnvtJZfRvOL8lXDHbLZg/8WMGJmDwJPA29ebUMz\n+wzwVpq9x9vN7A3AucA7h+Sv+Ep1eSlwp5mdAkKa5X+2LivdBD2dov+cnutCttrlR8CnzOxGmnV5\nC2S2XdasC9lqF4AXcvbUQQDM7HPu/gaan0Rea2YHgZ8Ce5Ir7pri1OV24J/N7B00s/vGbgfRfPAi\nIjmVuSEaERGJRgEvIpJTCngRkZxSwIuI5JQCXkQkpxTwIiI5pYAXEcmp/wMXMft8dnnXTwAAAABJ\nRU5ErkJggg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f5e777674a8>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Loss:  0.0007128288852982223\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAD4CAYAAADmWv3KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFqNJREFUeJzt3X+MHGd9x/H33W1IuOwhDnqpQyCJ\nY+1945iIiIsxVoK41kmvJo1UfkSoqRA2tVohKvFLQU5BQANpAlUh5UdTIadyoSVY0AYoxKxJiFMF\nQmRdFFd1j+8tSdOgguoTuQRPznKyd9c/di/ZrG/vbmd2b2ae/bykKLszs7PP42f2c88+MztP3+Li\nIiIiEp7+tAsgIiLdoYAXEQmUAl5EJFAKeBGRQCngRUQCVUi7AEtmZk60dTnP8PAgs7Nz3SrOulJd\nskl1ySbV5cVGRob6Wq3LbQ++UBhIuwgdo7pkk+qSTarL2uU24EVEZGUKeBGRQCngRUQCpYAXEQmU\nAl5EJFAKeBGRQCngRURSFEUwOdlPFHV+35n5oZOISK+JIpiYGKRSGaBUmqdcnqNY7Nz+1YMXEUnJ\nsWNQqdR+7FSpDODe2UiO3YM3s88DbwQWgfe7+5GGdVcBfwXMA3e7+6eSFlREJDRbtkCpNP98D95s\noaP7j/XnwszeDJTcfTvwJ8AXmjb5AvB24Arg98zskkSlXEkUUZg8QlcGsEREuqhYhHJ5joMHn+n4\n8AzEH6LZAXwbwN2ngGEzexmAmV0EPOnuv3D3BeDu+vadF0UMT4wzvHMHwxPjCnkRyZ1iEcbGFjoe\n7hB/iGYDMNnwfKa+7Df1/880rDsObFpth8PDg23feGfk+BNQmQagUJmuPd+4ra19ZMXIyFDaRegY\n1SWbVJds6mZdOnUVTcvbVa6y7nnt3jJzZGSImXPOZ7g0SqEyTbU0yuw558PMibb2kwUjI0PM5LDc\ny1Fdskl1yaZO1GWlPxBxA/6X1HrqS14F/KrFuvPqyzqvWGS2fJiCT1G1zXTlO46ISE7FHYM/BLwD\nwMxeD/zS3U8AuPvjwMvM7EIzKwB/UN++O4pFqmNbFe4iIk1i9eDd/SdmNmlmPwEWgPeZ2S7gaXe/\nC3gvcGd98wPuPt2R0oqIyJrFHoN3971Ni442rPt3YHvcfYuISHL6JauISKAU8CIigVLAi4gESgEv\nIhIoBbyISKAU8CIigVLAi4gESgEvIhIoBbyISKAU8CIigVLAi4gESgEvIhIoBbyISKAU8CIigVLA\ni4gESgEvIhIoBbyISKAU8CIigVLAi4gkEEUwOdlPFKVdktPFnpNVRKTXRRFMTAxSqQxQKs1TLs9R\nLKZdqheoBy8iEpN7P5XKAACVygDu2YrUbJVGRCRHzBYoleYBKJXmMVtIuUQvpiEaEZGYikUol+dw\n78dsIVPDM6AePEQRhckjZPIMiYhkXrEIY2MJwr2LGdTbAR9FDE+MM7xzB8MT4wp5EVlfXc6gng74\ngk9RqEzXHlemKfhUyiUSkZ5y7FhXMyj3AZ/kGtSqbaZaGq09Lo1Stc0dLp2IyAq2bOlqBuX6JGvi\na1CLRWbLhyn4VO0fNmtnSEQkbF3OoFz34DtyDWqxSHVsq8JdRNLRxQzKdcBn/RpUEZE0xRqiMbMz\ngP3ABcA8sNvdH2va5p3Ah4EF4F53/2iyop4u69egioikKW4P/nrgKXe/ErgZuKVxpZkNAp8BdgDb\ngavM7JIkBW0l8TWoIiKBihvwO4C76o/vAa5oXOnuc8Cl7n7C3ReBXwOvjF1KERFpW9/i4mLbLzKz\nQ8AN7n60/vwXwCZ3f3aZbS8FDgCvc/fnWu2zWp1fLBQG2i6LiEiP62u1YtUxeDPbA+xpWrxtLW9g\nZiXg68D1K4U7wOzs3GpFeZGRkSFmZk609ZqsUl2ySXXJJtXl9H20smrAu/s+YF/jMjPbD2wAjtZP\nuPY1997N7NXAt4F3ufsj7RdbRESSiDsGfwi4rv74WuC+Zba5A3ivuz8c8z1ERCSBuL9kPQBcbWYP\nAKeAXQBmthe4n9pJ1TcBN5nZ0ms+5+7fTVRaERFZs1gB7+7zwO5llt/a8HQwbqFERCS5XP+SVURE\nWlPAi4gESgEvIhIoBbyISKAU8CIigVLAi4gESgEvIhIoBbyISKAU8CIigVLAi0hPiyKYnOwnitIu\nSefFvReNiEjuRRFMTAxSqQxQKs1TLs8FNTucevAi0rPc+6lUahMNVSoDuIcViWHVRkSkDWYLlErz\nAJRK85gtpFyizur5gE88/hZFFCaPEOQAnkjgikUol+c4ePCZ+MMzGc6Anh6DTzz+FkUMT4xTqExT\nLY0yWz5MUAN4Ij2gWISxsZg994xnQE/34JOOvxV8ikJluva4Mk3BpzpeRhHJrqxnQE8HfNLxt6pt\nploarT0ujVK1zR0vo4hkV9YzoKeHaJbG39z7MVto/5tVschs+TAFn6o1bIa+monIOsh4BvR0wEPC\n8bf6DqpjWztXIBHJlwxnQE8P0YiIhEwBLyISKAW8iEigFPAiIoFSwIuIBEoBLyISKAW8iEigFPAi\nIoFSwIuIBEoBLyISKAW8iEigYt2LxszOAPYDFwDzwG53f6zFtncCp9x9V8wyiohIDHF78NcDT7n7\nlcDNwC3LbWRmVwObYr6HiIgkEDfgdwB31R/fA1zRvIGZnQl8DPh0zPcQEZEE4t4ueAMwA+DuC2a2\naGYvcfdnG7a5Ebgd+M1adjg8PEihMNBWIUZGhtraPstUl2xSXbJJdVmbVQPezPYAe5oWb2t63tf0\nmhJwubt/0szG11KQ2dm5tWz2vJGRIWZmTrT1mqxSXbJJdckm1eX0fbSyasC7+z5gX+MyM9tPrRd/\ntH7Cta+p934NcL6Z/RR4GTBiZh9x98+2X3wREYkj7hDNIeA6oAxcC9zXuNLdbwNuA6j34Hcp3EVE\n1lfck6wHgAEzewB4H7Xxdsxsr5lt71ThRERWE0UwOdlPFKVdkuyJ1YN393lg9zLLb11m2WHgcJz3\nyYMoIv6k3SKSSBTBxMQglcoApdI85fKcPocN9EvWBJYOrp07z2ZiYlA9CJF15t5PpVK7+q5SGcBd\nkdZI/xoJ6OASSZfZAqXSPACl0jxmCymXKFuUSAl05OCKInjoIdT9F2lfsQjl8hwHDz4Tf3gmiihM\nHgnyMxj3KhrhhYMr9hh8FDE8MQ6VaYZLo8yWD6MBRJH2FIswNhaz517/DBYq01QD/AyqB5/Q0sEV\n55go+BSFynTtcWWagk91uHQispLQP4MK+BRVbTPV0mjtcWmUqm1OuUQivSX0z6CGaNJULDJbPszI\n8SeYPef8oL4aiuRC/TNY8KlauAf2GVTAp61YhI3bIJB7a4jkTrFIdWxr2qXoCg3RiIgESgEvIhIo\nBbyISKAU8CIigVLAi4gESgEvIhIoBbyISKAU8CIigVLAi4gESgEvIhIoBbyISKAU8CIigVLAi4gE\nSgEvIhIoBbyISKAU8CnTnNsi0i0K+BRFEUxMDPLGN9b+r5CXXhRFMDnZr+O/CxTwKXLvp1IZAKBS\nGcBdzSG9ZamTs3Pn2erkdIESJUVmC5RK8wCUSvOYLaRcIpH1pU5Od2lO1hQVi1Auz3H8+BDnnDMX\n2ny/Iqta6uRUKgPq5HSBAj5lxSJs3AgzMzF3EEXBzggv4Vvq5Lj3Y7YQ7xDWZ6ClWAFvZmcA+4EL\ngHlgt7s/1rTN64A76k+/4+6fSlBOWU4UMTwxTqEyTbU0ymz5sA5wyZ1iEcbGYvbc9RlYUdwBr+uB\np9z9SuBm4JZltvkK8KfAG4BLzGww5ntJCwWfolCZrj2uTFPwqZRLJLK+9BlYWdwhmh3AV+uP7wH+\noXGlmf02UHT3h+uL/ijm+8gKqraZamn0+d5L1TanXSSRdaXPwMriBvwGYAbA3RfMbNHMXuLuz9bX\nXwg8aWb7gRLwTXe/LWlhpUmxyGz5sMYfpXfpM7CiVQPezPYAe5oWb2t63rfM843AHwIngQfN7Ifu\nfqzV+wwPD1IoDKxe4gYjI0NtbZ9lsesyMgQbz+1sYRJSu2RTsHXJ4GegHd1sl1UD3t33Afsal9V7\n5huAo/UTrn0NvXeA/wOOufuv69s/AGwBWgb87OxcWwUfGRliZuZEW6/JKtUlm1SXbFJdTt9HK3FP\nsh4Crqs/vha4r3Glu/83MGRmrzCzfuAywGO+l4iIxBB3DP4AcHW9Z34K2AVgZnuB+939QeCDwEFg\nEfiBux9NXlwREVmrWAHv7vPA7mWW39rw+CFOH6sXEZF1ohs/iIgESgEvIhIoBbyISKAU8CIigVLA\ni4gESgEvIhIoBbyISKAU8CIigVLA55xmpBeRVhTwOaYZ6SUL1MnILgV8jmlGekmbOhnZpkTIsaUZ\n6QHNSC+pUCcj2+LeTVIyoCMz0osksNTJqFQG1MnIIAV8ziWakR4gijTdmcTWkU6GjsGuUcD3sihi\neGL8+QmLZ8uH9QGTtiXqZOgY7CoNmPWwgk9RqEzXHlemKfhUyiWSXqNjsLsU8D2sapuplkZrj0uj\nta/IIutIx2B3aYimlxWLzJYPa/xT0qNjsKsU8L2uWKQ6tjXtUkgv0zHYNRqiEREJlAJeRCRQCngR\nkUAp4EVEAqWAFxEJlAJeRCRQCngRkUAp4EVEAqWA73GajUckXPolaw9bmo1n6V7e5fKcfikuEhD1\n4HuYZuMRCVusHryZnQHsBy4A5oHd7v5Y0zY3A+PU/ojc5e6fTVRS6TjNxiNQ+yanWcHCFLfLdj3w\nlLtfCdwM3NK40sxeC/yOu18BXAHsNrMNiUoqHbc0G8/Bg89oeKZHadLssMUN+B3AXfXH91AL8UZP\nA2eZ2ZnAWcACMBfzvaSLlmbjUbj3Jg3Tha1vcXGx7ReZ2SHgBnc/Wn/+C2CTuz/bsM2NwPuBAeAm\nd//iSvusVucXC4WBtssiKYsiOHYMtmzRvbxzKIpg61b42c/g4ovhyJEYzahjIG19rVasOgZvZnuA\nPU2Lt630BmZ2EfBW4CLgDOAnZnbA3Y+3ep/Z2fY6+CMjQ8zMnGjrNVmV27osM5/myMZz81mXZeS2\nXZaxUl3uvvuFMfiTJ+HkyTZ2nMKcqr3SLu3so5VVA97d9wH7GpeZ2X5gA3C0fsK1r7H3DmwFHnL3\nufr2/wG8FvhRu4WX7Fp2Ps2N56ZcKmlXkkmzlzsGNHlHdsQdcDsEXFd/fC1wX9P6nwOXm1l//Q/A\npcBjSFA0n6boGMi2uD90OgBcbWYPAKeAXQBmthe4390frI/TP1Dffp+7P56wrJI1mk9TdAxkWqyT\nrN0wM3OirYJoHC6bVJdsUl2yqUNj8C1PsuqaKBGRQCngRUQCpYAXEQmUAl4S0e2GRbJLAS+x6T4m\nItmmgJfYdB8TkWzTJ1JiW7rdMKDbDYtkkGZ0ktiWbjese4mnS/dzl1YU8JJIkvuYSHKadlFWoiEa\nkRzTeRBZiY4GSVcUUZg8gi7Biacj50HUBsHSEI2kJ4V7iYcm8XkQtUHQ1IOX1Cx7P3lpW5JpF9UG\nYVPAS2qqtpmnNl3GQ7yBpzZdpnuJp0D3cw+bhmgkNRFFJjhChQIlqpQ5iQYH1pnu5x409eAlNe79\nVB6t9TEqjxZ0BUhaisXaNHsK9+DoEyWp0S9hRbpLQzSSGv0SVqS71IOXVCW5AiQUuuWydIt68CIp\n0q0GpJvUg5dcy3vvV7cakG7S0SS5FUUwcfVLaxOOXP3SXIa8bjUg3aQhGsktf+QUlUeHgPpllo+c\nYuzKM1MuVXt0qwHpJvXgJbe2cIyLqf20/mKm2MKxtveRdIinE0NEutWAdIt68JJbZ102yoOb3oI/\n+hJs07M8d9ndbb0+6QnOLJwgXbrVwFIPXrcakEYKeMmvYpHnfng3oz7FczF+Zr/cCc52Ji9J+vqO\n0K0GZAUaopF8S/Az+6QnODPzS1zdakBaUA9eelbSE5z6Ja5knQJeelrSOWU1J61kmYZopLdl4Rry\nLJRBghS7B29mbwa+CbzH3b+3zPo/Bj4ALABfcfc7YpdSpBuycA15FsogwYrVgzezTcCHgB+3WH82\n8HHgKmAc+KCZvSJmGUW6oiPXkCfsfes6dummuEM0vwLeBjzdYv024Ii7P+3uJ6n9Ibgi5nuJdEXi\n6erqve/hnTsYnhiPFfKaMk+6KdYQjbvPAZhZq002ADMNz48D5660z+HhQQqFgbbKMTIy1Nb2Waa6\npGBkCB6ehGPHKGzZwsgyQyMr1uWx/4KG3vfI8Sdg47aOl6FTctMua6C6rM2qAW9me4A9TYs/4e7l\nNt6nb7UNZmfn2thd7R9lZuZEW6/JKtUlZRddAicX4eSLy71qXc45n+GGX5HOnnM+xK17izJ0Si7b\npQXV5fR9tLJqwLv7PmBfm+/5S2q9+CXnAT9tcx8i2aZfkUrGdes6+IeAfWb2cqBKbfz9A116L5H0\nLP2KVCSD4l5Fc42ZHQZ+H7jFzA7Vl+81s+31E6t7gTJwD/CX7t7qhKyIiHRB3JOs3we+v8zyWxse\nfwv4VvyiiYhIEvolq4hIoBTwIiKBUsCLiARKAS8iEqi+xcXFtMsgIiJdoB68iEigFPAiIoFSwIuI\nBEoBLyISKAW8iEigFPAiIoFSwIuIBKpbtwuOxcxeC3wH+Ly7f6lh+QTwA3fvqz8fBu4EInd/xzL7\n2Q+MAb+uL/rr+g3S1k0bdXkn8GFqk5Pf6+4fbdrPa4CvAQPUpkp8l7ufWp9aPF+GTtVlP/lpl48D\nO6lNVvM9d/90037y1C6r1WU/OWmXhuV3AqfcfVfT8ty0S8PyVnXZT8J2yUwPvj5R9xeBe5uWnwXc\nSK2xlvw98MAqu7zR3cfr/633wbqmupjZIPAZYAewHbjKzC5p2t1NwJfd/U3Az4H3dLf0L9bhukA+\n2uVC4FJ3305tLoN3m9mrmnaXl3a5kNXrAjlol4blVwObWuwuF+3SsHylukDCdslMwAOngLdQmw2q\n0V8AXwaebVi2h9UDPk1rqkt9bttL3f2Euy9S+0v9yqbXjAPfrT/+N+CqLpW5lU7WJW1rrcvj7n5d\nfd0wtW8kv2l6zTj5aJe11CVta/7sm9mZwMeAT7O8cXLQLrCmuiSWmYB392p9opDnmdko8Dp3/2bT\ntmuZxPDPzexHZvYNM/utTpZ1NXHqYmaXAhdy+tSGZzd8xVx18vJO63BdICftUl/3t8Ax4FPuHjWt\nzk271NetVBfIT7vcCNxO6z9SeWqX1eoCCdslMwHfwueBD8V43deAve7+u8AjwCc7WaiYWtbFzErA\n14Hr3f25Ffax6uTl6yRuXXLVLu7+fuBi4AYz27jCPjLfLqvUJRftUj+2Lnf3b6xxH5ltlzXWJXG7\nZDbgzew8agfkP5vZT4Fzzez+tbzW3e9190fqT78LXNqlYq7JSnUxs1cD3wbe3VDmRpGZvbT++DxO\n/+q3rpLUJS/tYmavMbPLAdx9Fvgx0Dzxai7aZS11yUu7ANcA59eX/R1wjZl9pOnluWgX1lCXTrRL\npq6iaeTu/0vDyQcze9zd37yW15rZvwA3uPtj1Mbk/rMrhVyjVepyB/Bed3+4xcvvAd4O/FP9/z/o\nZllXk6QueWkXM3s9cLuZbQcWqV3J8JWml+eiXdZSl7y0S/3pbfVl48Aud/9s08tz0S71pyvWpRPt\nkpmAN7Mx4G+ojd0+Z2bvAN7m7k82bTdA7Qz1y4HzrDb5903Ak8Bb3f0TwJeAA2Y2B0TA7vWqR72M\na63LKPAm4CYzW1r8OeAJXqjLJ4CvmtmfAf8D/OO6VOKFMnayLrloF3d/2Mz+lVpvtw/4vrs/YmaX\nkbN2WWNdctEuK7w+d+2ywus72i66H7yISKAyOwYvIiLJKOBFRAKlgBcRCZQCXkQkUAp4EZFAKeBF\nRAKlgBcRCdT/AwomD5y8wfSsAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f5e777a1198>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Loss:  0.00012177378812339157\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAF75JREFUeJzt3X+QJGV9x/H37Y4a7npPVh08QC1+\nZPbLcSQSNidccXCLh6yL/qNcmT+MCQZTCV5SqIklasVErUKNRS4CKYWqVFFa8UeVCQjlnYNgTjkv\nymaNRM+9747gKXJYLLJy19xGmd3NHz0jw7A/en5uP7OfV9XV9fbM9Dzf6Z7PPPN0T/e6hYUFREQk\nXH2r3QAREWmNglxEJHAKchGRwCnIRUQCpyAXEQlcrttPWC7PLczMnOj203bE4OB6VEv2qJZsUi2t\nyecH1i11W9d75Llcf7efsmNUSzaplmxSLZ2joRURkcApyEVEAqcgFxEJnIJcRCRwCnIRkcClOvzQ\nzM4DvgLscfdb6m67HLgBmAP2uvtH295KERFZ0oo9cjPbANwM3LfEXW4CrgIuBq4ws3Pb1zyR9olj\nmJjoI46bX0BuYpzmFyDSGWmGVn4NXAkcrb/BzM4CnnT3R9x9HtgL7GxvE0VaF8cwOrqesbENjI6u\nbzyL45gXvO5Kpsbezwted6XCXDJlxaEVdy8DZTNb7OZNwHTN348DZ6+0zHx+IG37Mk+1ZFN9LQ8/\nDKVSMl0q9fP44wOceWb65cU/PMzWhz7PYTZzzkOTjB/5OdFlW9vY4qX18noJWZZqafdP9Jf8CWmt\n6enjbX7a1ZHPD6iWDFqsllNOgcLZJ1F6KEfh7DKnnDLL9PQSC1jExK/O5DAvA+AwmznwqycYbvT1\nimNyPknZNkMUpXpIr6+XUK1GLct9cLR61MpRkl551eksMgQj0g6tjHFHxDzAVr7DhTzAViIaW4id\n/yIKZ5cBKJxdxs5/UWMNiGMGR0cYHNvJ4OiIhmakrVoKcnc/Amw0szPMLAe8EbinHQ0TqdXqGHfO\nJzn5oe9zIQ9w8kPfJ+eTDT0+iqD49Vn27Xua4tdn03aon/P8udJUMl2aavj5RZaz4tCKmQ0DNwJn\nAM+Y2S7gLuAn7n4HcC3whcrdv+TuUx1qq6xh7n2USsmJikqlftz7GB6eT/34sm2mXBgiV5qiXBhK\nhjcaFEU09Jztfn6RpaTZ2TkBjCxz+7eAbW1sk8jzmM1TKMxRKvVTKMxh1mCgRhEzxf0Nj1G3TRTx\nyH98kx/f+3N+9/JXEEUbuvv80tO6fj5ykWZEERSLJ3Dvw2y+uRyOIsrD3TnSpF4cw+ib85RKmygU\n5igWT3T9s0R6l36iL8GoDm2EGICLDQ2JtIu2JglHwL+srA4NAc0NDUHQ9UtnaWhFuiaOaX5opHL4\nXnVn4Uxxf/fHuVvQ8tBQ4PVLZ6lHLl3RjsMHQz98r5WhoV6oXzpHQS5d0eoYcfXwPWBNHr631uuX\n5WloRboi+MMHV9tar1+WpSCXrgj98MFMWOv1y5IU5NI1rfwyUkSWpjFyEZHAKchFAtHyFY6kZ2lo\nRSQA1cM3qzuL9RN/qaUeuUgA9BN/WY62BpEAtOUn/tKzNLQi3dPEpc4k0ZbDN/X69yz1yCW1lna2\n6VJnLWvp7I96/XuaglxS0blSwqbXv7cpyCUVnSslbHr9e5vGyCUVnSslcHr9e5qCXFLRuVJ6gF7/\nnqUgl9R0rhSRbNIYuYhI4BTkIiKBU5CLiAROQS4iEjgFuYhI4BTkImuEzmfeu3T4ocgaoPOZ9zb1\nyEXWAJ3PvLdpbYqsATqfeW/T0Iqkp/NZB0vnM+9tqYLczPYAFwELwHXuPl5z227gj4E54L/d/V2d\naKi0Lo5p/o1cOZ91rjRFuTDETHG/3syBaekUC1r/mbbi0IqZ7QAK7r4NuAa4qea2jcB7gUvcfTtw\nrpld1KnGSvN0PnFphdZ/tqUZI98J3Ang7pPAYCXAAX5T+ReZWQ5YDzzZiYZKa3Q+cWmF1n+2pRla\n2QRM1Pw9XZl3zN3/z8w+DDwMzAJfdPeplRaYzw8009ZMCqWW7dvhnHPg8OHk/+3bNzzvm/GyteQH\n4HsTcOgQuS1byGf8a3Uo6yWNTNTSpvWfiVraJEu1NLOzc111otIz/wAwBBwDvmFmr3b3B5dbwPT0\n8SaeNnvy+YGgatm799kx8tlZmJ199rbUtZx1LswuwGx26w5tvSwnc7W0sP4zV0sLVqOW5T440ny/\nPkrSA686DXisMr0ZeNjdn3D33wD3A8NNtlM6rKWL94pIZqUJ8nuAXQBmdgFw1N2rH0VHgM1mdlLl\n7z8ESu1upIiILG3FoRV3P2hmE2Z2EJgHdpvZ1cBT7n6HmX0S+E8zKwMH3f3+zjZZRERqpRojd/fr\n62Y9WHPbrcCt7WyUiIikp5/oi4gETkEuIhI4BbmISOAU5CIigVOQi4gETkEuIhI4BbmIpKJrfmaX\nLiwhIivSNT+zTT1yEVmRrvmZbVobIrIiXfMz2zS0spbomovSpLZd83NiXNtfB6hHHoiWdzRVrrk4\nOLaTwdERtMdKGtXSaZDjGLZu1fbXIQryALR6vU3QNRdldeV8Mrk8Fdr+OkFBHoB27GjSNRdlNZVt\nc3KNQbT9dYLGyANQ3dFUPfSrqR1NUcRMcb/GyGV1RBGMjzNz4AFtfx2gIA9AW3Y0VRZUHt7a1raJ\npKbtr2MU5IGo7mgSEamnMXIRkcApyEVEAqcgFxEJnIJcRCRwCnIRkcApyEVEAqcgFxEJnIJcRCRw\nCnIRkcApyEVEAqcgFxEJnIJcRCRwCnIRkcClOvuhme0BLgIWgOvcfbzmtlcCXwBeCHzP3f+yEw0V\nkbBVL1fY0qmYZVEr9sjNbAdQcPdtwDXATXV3uRG40d1fA8yZ2ava30wRCVnlkp0tXa5QlpZmaGUn\ncCeAu08Cg2a2EcDM+oBLgLsqt+929591qK0iEij3vuolO5u+XKEsLc3QyiZgoubv6cq8Y0AeOA7s\nMbMLgPvd/f0rLTCfH2iiqdmUtpY4hkOHYMuWJq9y1fICVrYW10sIeqGW7duTS3YePpz8v337hsY3\n4y68BxqRpfXSzBWC1tVNnw58CjgCfNXM3uDuX11uAdPTx5t42uzJ5wdS1RLHMDq6/rfX3CwWTzS2\nHcYxg6Mj5EpTlAtDzBT3t31DTltLCFRLNo2PD3DgwNOYzTM7C7OzDTy4C++BRqzGelnugyPN95uj\nJD3wqtOAxyrTTwA/dfeH3H0OuA/Y0mQ7e5Z7H6VSP9Dc18qcT5IrTSXTpSlyPtn2Nop0WvVyhc3k\nr94Dy0uTKPcAuwAqwydH3f04gLuXgYfNrFC57zDgnWhoyMzmKRTmACgU5jBr7NqbZdtMuTCUTBeG\nkquQi6wheg8sb8WhFXc/aGYTZnYQmAd2m9nVwFPufgfwLuD2yo7PHwB3d7LBIYoiKBZP4N7koVdR\nxExxPzmfTDbgDIwPinSV3gPLWrewsNDt51zolTG/Xhq/VC3ZpFqyaZXGyNctdZuOARIRCZyCXEQk\ncApyEZHAKchFRAKnIBcRCZyCXEQkcApyEZHAKchFRAKnIBcRCZyCXEQkcApyEZHAKchFRAKnIBcR\nCZyCXEQkcApyEZHAKchFRAKnIBcRCZyCXEQkcApyEZHAKchFJAhxDBMTfcTxarckexTkKWkjElk9\ncQyjo+sZG9vA6Oh6vQ/rKMhTaMtGFMfkJsbRFijSOPc+SqV+AEqlftybiK4efg8qyFNoeSOKYwZH\nRxgc28ng6EhPbkginWQ2T6EwB0ChMIfZfGML6PH3oII8hVY3opxPkitNJdOlKXI+2fY2ivSyKIJi\n8QT79j1NsXiCKGrs8b3+HsytdgNCUN2I3Pswm294IyrbZsqFIXKlKcqFIcq2uTMNFelhUQTDww32\nxCt6/T2oIE+plY2IKGKmuJ+cTyYbUKOfBCLSmh5/DyrIuyWKKA9vXe1WiKxdPfwe1Bi5iEjgFOQi\nIoFTkIuIBC7VGLmZ7QEuAhaA69x9fJH7fAzY5u4jbW2hiIgsa8UeuZntAAruvg24BrhpkfucC1za\n/uaJiMhK0gyt7ATuBHD3SWDQzDbW3edG4INtbpuIiKSQZmhlEzBR8/d0Zd4xADO7GvgmcCTtk+bz\nA6kbmHWqJZtUSzapls5o5jjyddUJM3sJ8HbgcuD0tAuYnj7exNNmTz4/oFoySLVkk2pp/TmXkmZo\n5ShJD7zqNOCxyvRrgTxwP3AHcEFlx6iIiHRJmiC/B9gFYGYXAEfd/TiAu3/Z3c9194uANwHfc/d3\nd6y1IiLyPCsGubsfBCbM7CDJESu7zexqM3tTx1snIiIrSjVG7u7X1816cJH7HAFGWm+SiIg0Qr/s\nFBEJnIJcRCRwCnIRkcApyEVEAqcgFxEJnIJcRCRwCnIRkcApyEVEArdmgjyOYWKijzhe7ZaIiLRX\nM2c/DE4cw+joekqlfgqFOYrFE0TRardKRKQ91kSP3L2PUqkfgFKpH/cmyo5jchPjqEsvEqaWv5Vn\nOAPWRJCbzVMozAFQKMxhNt/YAuKYwdERBsd2Mjg6kskVKSJLq34rHxvbwOjo+sbfwhnPgDUR5FEE\nxeIJ9u17uqlhlZxPkitNJdOlKXI+2YFWikintPqtvD4DOHSo7W1sxZoIckjCfHh4vqmx8bJtplwY\nSqYLQ5Rtc5tbJyKd1Oq38voMYMuWtrexFWtiZ2fLooiZ4n5yPpmEuPaUigSl+q3cvQ+zJjp0dRmQ\njyKYzc5l6xTkaUUR5eGtq90KEWlS9Vt5KwvIagasmaEVEZFepSAXEQmcglxEJHAKchGRwCnIRUQC\npyAXEQmcglxEJHAKchGRwCnIRUQCpyAXEQmcglxEJHAKchGRwCnIRUQCpyAXEQlcqtPYmtke4CJg\nAbjO3cdrbrsM+BgwBzjwDndv4VyRIiLSiBV75Ga2Ayi4+zbgGuCmurvcBuxy94uBAeD1bW+liIgs\nKc3Qyk7gTgB3nwQGzWxjze3D7v7zyvQ08NL2NlFERJaTZmhlEzBR8/d0Zd4xAHc/BmBmpwJXAH+3\n0gLz+YGGG5pVqiWbVEs2qZbOaOZSb+vqZ5jZKcDdwDvd/ZcrLWB6OjvXumtFPj+gWjJItWSTamn9\nOZeSJsiPkvTAq04DHqv+URlm2Qd80N3vabKNIiLSpDRj5PcAuwDM7ALgqLvXfhTdCOxx9691oH0i\nIrKCFXvk7n7QzCbM7CAwD+w2s6uBp4Ai8CdAwczeUXnI5939tnY3NI7BvQ+zeaKo3UsXEQlXqjFy\nd7++btaDNdMval9zFhfHMDq6nlKpn0JhjmLxhMJcRKQiiF92uvdRKvUDUCr14x5Es0VEuiKIRDSb\np1CYA6BQmMOsiR+OxjG5ifGkey8i0m0dzKBmDj/suiiCYvFE82Pkcczg6Ai50hTlwhAzxf1obEZE\nGlG7ny6fb/zBncygIHrkkNQ8PNzcjs6cT5IrTSXTpSlyPtnm1olIL6vupxsb28Do6PqGO9WdzqBg\ngrwVZdtMuTCUTBeGKNvmVW6RiISkfj/doUONPb7TGRTE0ErLooiZ4n5yPpm8gBpWEZEGVPfTVY+c\n27Kln9nZBhbQ4QxaG0EOEEWUh7euditEJEDP30830FiQVxbSqQxaO0EuItKC6n66LFoTY+QiIr1M\nQS4iEjgFuYhI4BTkIiKBU5CLiAROQS4iEjgFuYhI4BTkIiKBU5CLiAROQS4iEjgFuYhI4BTkIiKB\nU5CLiAROQS4iEjgFuYhI4BTkIiKBU5CLiAROQS4iEjgFuYhI4BTkIiKBU5CLiAROQS4iErhcmjuZ\n2R7gImABuM7dx2tuuxy4AZgD9rr7RzvRUBERWdyKPXIz2wEU3H0bcA1wU91dbgKuAi4GrjCzc9ve\nShERWVKaoZWdwJ0A7j4JDJrZRgAzOwt40t0fcfd5YG/l/u0Xx+QmxiGOO7J4EZFQpQnyTcB0zd/T\nlXmL3fY4cGp7mlYjjhkcHWFwbCeDoyMKcxEJThzDxERfR+Ir1Rh5nXVN3vZb+fxAY8/48I+gNAVA\nrjRF/vGfwZkXNraMDmm4lgxTLdmkWrKpkVriGC69FA4fhnPOgfFxiKL2tSVNkB/l2R44wGnAY0vc\ndnpl3rKmp4+nbV/ilFcxWBgiV5qiXBhi5pRXQaPL6IB8fqDxWjJKtWSTasmmRmuZmOjj8OENQBLm\nBw48zfDwfMPPuZQ0Qyv3ALsAzOwC4Ki7Hwdw9yPARjM7w8xywBsr92+vKGKmuJ+ZffcxU9zf3o8y\nEZEOM5unUJgDoFCYw6yxEF/Jij1ydz9oZhNmdhCYB3ab2dXAU+5+B3At8IXK3b/k7lNtbWFVFFEe\n3tqRRYuIdFIUQbF4Avc+zObb3hdNNUbu7tfXzXqw5rZvAdva2SgRkV4TRTQ8nJKWftkpIhI4BbmI\nSOAU5CIigVOQi4gETkEuIhI4BbmISODWLSwsrHYbRESkBeqRi4gETkEuIhI4BbmISOAU5CIigVOQ\ni4gETkEuIhI4BbmISOCaudTb85jZecBXgD3ufouZXQrcADwDPA28DTgG3AoMAS8E/sXdP1e3nCPA\nI8BcZdZb3f3RdrQxrTS1uPuMmX0YeD1QBt7n7gfqlvNq4NPAAvC/7n5tN+uotKFdtewHNlQeA/A3\n7j7RpTKqbaiv5RzgNpLXdwq41t3LZvZW4F0k586/zd3/tW45rwQ+B/STXOnqbe7+6y6W0s5abgeG\ngV9WZn3S3b/apTKqbUhbyyDJdQtid9+1yHJCWi8r1XI7XV4vLffIzWwDcDNwX83sfwKucffLgIPA\nXwBjwAZ3vxS4DPiEmS32/GPuPlL51+0QT1WLmf0B8DqS87C/EfjEIov7Z+A6d78YeLGZjXW08XXa\nXAvA22vWS7dDfLFaPgF8zN13AD8D3lK534eAy4ER4N1m9pK6xX2EpBNxCfBj4M863PznaHMtAO+v\nWS/dDvFUtVTmfwY4wNKCWC+V+SvVAl1eL+0YWvk1cCXPvVbnE8BLK9ODlb+fAE6uhHcEHHf3zpxl\nvXlpaykAE+4+7+4zwFNmdkb1AWb2QuBMdx+vzLqb5A3ZTW2pJSMWq6UAPFCZLgJXABcC4+7+lLvP\nAt8GLq5b1ghwV2U6K+ul2VpWW9paAN7B8uE3QhjrBVaupetaDnJ3L1c2tFrvBu40MwcuAW539++Q\nfKr9hORrSv1Vh6o+Y2YHzOzjZrau1fY1Im0twA+BHWa23sxeDpwPvLzmMS8DZmr+fhw4tWMNX0Qb\na6n6iJl9y8xuNbOTOtn2ekvU8gPgDZXpUZI2bwKma+6z2Ou+oeYre1bWS7O1APyVmX3DzL5oZi9r\ne4OX0UAtVK/zu4xQ1kuaWqDL66VTOztvBt7k7kbyyfVOM7sEeCVwNnAe8PFKz7XWh4D3kHw6nwdc\n1aH2NeJ5tbj7j0jGzu4FbiS59N1yHzpd/UBaRrO1fAp4b2VYbB7Y3b0mL+lvSYYgvkGyHS/2Gq/0\numdlvTRby+eA6939tcD3gX/oWAvTS1PLSkJaL4vp+nppy87ORfy+u3+7Mv114K3AScB97l4GHjWz\nJ4FXAA9XH+Tun61Om9le4PeAL3eojWktVgvufgtwC4CZ/RdwpOYx0zw7hAFwOs/9yrZamqmFykW2\nq+4G/qjjLV2Buz9CMqaPmY2S9OCOkvRkq04HvlP30NjMTqr0vjKxXpqtxd1rx3PvItm5vqqWqCWN\nUNZLmsd1fb10qkf+CzM7tzK9FSiR7MB4DYCZbSRZWY9VH2BmLzazYk0vfQfJ1/7V9rxazCxvZnvN\nbJ2ZbQH63P0X1Qe4+zPAYTPbXpn1ZuBr3W32ohqupTL/XjM7uTJrhAysFzP7sJlVv/a+neQD5rvA\nVjM72cwikjHl++seei/PftO7igysl2ZrMbN/N7OzKn+OkN31kkYo6yXN47q+Xlo+ja2ZDZN8JT+D\n5LC2R4EPAJ+s/P0kyR7oYySfTFtIDjH6lLt/0cyuBp5y9zvM7DrgT4FZ4H+Av3b3rp1nN20t7v4r\nM7uBZNxsDvhzd3/QzF5PspPz05XAvJXkw/K77v6ebtXRgVreAryP5PDDR0mOfDmxyrW8j2SoaB1w\nf/X1NbNdwHtJDhm72d3/zczOJxlS+nszOxX4LPA7wE9JjsZ5JtBaLgP+ETgBxJVaHs9aLWbWT3I0\nyMkkHbhDJEepPElg6yVlLV1fLzofuYhI4PTLThGRwCnIRUQCpyAXEQmcglxEJHAKchGRwCnIRUQC\npyAXEQnc/wNn27Lr11EbgwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f5e7770ddd8>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "UsGi8EXXmAC3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Time-Series Prediction\n",
        "\n",
        "Time-series prediction can be applied to many tasks. Think about weather forecasting or predicting the ebb and flow of stock market prices. You can even try to generate predictions much further in the future than just one time step!"
      ]
    },
    {
      "metadata": {
        "id": "x24r-me2mAC5",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}