nlp_constructing_text_generation_model.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "l10c03_nlp_constructing_text_generation_model.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3.8.5 64-bit"
    },
    "interpreter": {
      "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ph5eir3Pf-3z"
      },
      "source": [
        "# Constructing a Text Generation Model\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7GbGfr_oLCat"
      },
      "source": [
        "Using most of the techniques you've already learned, it's now possible to generate new text by predicting the next word that follows a given seed word. To practice this method, we'll use the [Kaggle Song Lyrics Dataset](https://www.kaggle.com/mousehead/songlyrics)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4aHK2CYygXom"
      },
      "source": [
        "## Import TensorFlow and related functions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2LmLTREBf5ng"
      },
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
        "\n",
        "# Other imports for processing data\n",
        "import string\n",
        "import numpy as np\n",
        "import pandas as pd"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GmLTO_dpgge9"
      },
      "source": [
        "## Get the Dataset\n",
        "\n",
        "As noted above, we'll utilize the [Song Lyrics dataset](https://www.kaggle.com/mousehead/songlyrics) on Kaggle."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4Bf5FVHfganK"
      },
      "source": [
        "!wget --no-check-certificate \\\n",
        "    \"https://drive.google.com/uc?id=1LiJFZd41ofrWoBtW-pMYsfz1w8Ny0Bj8\" \\\n",
        "    -O /tmp/songdata.csv"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2021-06-29 19:45:21--  https://drive.google.com/uc?id=1LiJFZd41ofrWoBtW-pMYsfz1w8Ny0Bj8\n",
            "Resolving drive.google.com (drive.google.com)... 172.217.27.206, 2404:6800:4009:800::200e\n",
            "Connecting to drive.google.com (drive.google.com)|172.217.27.206|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Moved Temporarily\n",
            "Location: https://doc-04-ak-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/bluuv2n88gutef2it2egvhvn305im880/1624976100000/11118900490791463723/*/1LiJFZd41ofrWoBtW-pMYsfz1w8Ny0Bj8 [following]\n",
            "Warning: wildcards not supported in HTTP.\n",
            "--2021-06-29 19:45:24--  https://doc-04-ak-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/bluuv2n88gutef2it2egvhvn305im880/1624976100000/11118900490791463723/*/1LiJFZd41ofrWoBtW-pMYsfz1w8Ny0Bj8\n",
            "Resolving doc-04-ak-docs.googleusercontent.com (doc-04-ak-docs.googleusercontent.com)... 142.250.192.65, 2404:6800:4009:829::2001\n",
            "Connecting to doc-04-ak-docs.googleusercontent.com (doc-04-ak-docs.googleusercontent.com)|142.250.192.65|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [text/csv]\n",
            "Saving to: ‘/tmp/songdata.csv’\n",
            "\n",
            "/tmp/songdata.csv       [   <=>              ]  69.08M  3.44MB/s    in 29s     \n",
            "\n",
            "2021-06-29 19:45:55 (2.36 MB/s) - ‘/tmp/songdata.csv’ saved [72436445]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gu1BTzMIS1oy"
      },
      "source": [
        "## **First 10 Songs**\n",
        "\n",
        "Let's first look at just 10 songs from the dataset, and see how things perform."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fmb9rGaAUDO-"
      },
      "source": [
        "### Preprocessing\n",
        "\n",
        "Let's perform some basic preprocessing to get rid of punctuation and make everything lowercase. We'll then split the lyrics up by line and tokenize the lyrics."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2AVAvyF_Vuh5"
      },
      "source": [
        "def tokenize_corpus(corpus, num_words=-1):\n",
        "  # Fit a Tokenizer on the corpus\n",
        "  if num_words > -1:\n",
        "    tokenizer = Tokenizer(num_words=num_words)\n",
        "  else:\n",
        "    tokenizer = Tokenizer()\n",
        "  tokenizer.fit_on_texts(corpus)\n",
        "  return tokenizer\n",
        "\n",
        "def create_lyrics_corpus(dataset, field):\n",
        "  # Remove all other punctuation\n",
        "  dataset[field] = dataset[field].str.replace('[{}]'.format(string.punctuation), '')\n",
        "  # Make it lowercase\n",
        "  dataset[field] = dataset[field].str.lower()\n",
        "  # Make it one long string to split by line\n",
        "  lyrics = dataset[field].str.cat()\n",
        "  corpus = lyrics.split('\\n')\n",
        "  # Remove any trailing whitespace\n",
        "  for l in range(len(corpus)):\n",
        "    corpus[l] = corpus[l].rstrip()\n",
        "  # Remove any empty lines\n",
        "  corpus = [l for l in corpus if l != '']\n",
        "\n",
        "  return corpus"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "apcEXp7WhVBs"
      },
      "source": [
        "# Read the dataset from csv - just first 10 songs for now\n",
        "dataset = pd.read_csv('/tmp/songdata.csv', dtype=str)[:10]\n",
        "# Create the corpus using the 'text' column containing lyrics\n",
        "corpus = create_lyrics_corpus(dataset, 'text')\n",
        "# Tokenize the corpus\n",
        "tokenizer = tokenize_corpus(corpus)\n",
        "\n",
        "total_words = len(tokenizer.word_index) + 1\n",
        "\n",
        "print(tokenizer.word_index)\n",
        "print(total_words)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'you': 1, 'i': 2, 'and': 3, 'a': 4, 'me': 5, 'the': 6, 'is': 7, 'my': 8, 'to': 9, 'ma': 10, 'it': 11, 'of': 12, 'im': 13, 'your': 14, 'love': 15, 'so': 16, 'as': 17, 'that': 18, 'in': 19, 'andante': 20, 'boomaboomerang': 21, 'make': 22, 'on': 23, 'oh': 24, 'for': 25, 'but': 26, 'new': 27, 'bang': 28, 'its': 29, 'be': 30, 'like': 31, 'know': 32, 'now': 33, 'how': 34, 'could': 35, 'youre': 36, 'sing': 37, 'never': 38, 'no': 39, 'chiquitita': 40, 'can': 41, 'we': 42, 'song': 43, 'had': 44, 'good': 45, 'youll': 46, 'she': 47, 'just': 48, 'girl': 49, 'again': 50, 'will': 51, 'take': 52, 'please': 53, 'let': 54, 'am': 55, 'eyes': 56, 'was': 57, 'always': 58, 'cassandra': 59, 'blue': 60, 'time': 61, 'dont': 62, 'were': 63, 'return': 64, 'once': 65, 'then': 66, 'sorry': 67, 'cryin': 68, 'over': 69, 'feel': 70, 'ever': 71, 'believe': 72, 'what': 73, 'do': 74, 'go': 75, 'all': 76, 'out': 77, 'think': 78, 'every': 79, 'leave': 80, 'look': 81, 'at': 82, 'way': 83, 'one': 84, 'music': 85, 'down': 86, 'our': 87, 'give': 88, 'learn': 89, 'more': 90, 'us': 91, 'would': 92, 'there': 93, 'before': 94, 'when': 95, 'with': 96, 'feeling': 97, 'play': 98, 'cause': 99, 'away': 100, 'here': 101, 'have': 102, 'yes': 103, 'baby': 104, 'get': 105, 'didnt': 106, 'see': 107, 'did': 108, 'closed': 109, 'realized': 110, 'crazy': 111, 'world': 112, 'lord': 113, 'shes': 114, 'kind': 115, 'without': 116, 'if': 117, 'touch': 118, 'strong': 119, 'making': 120, 'such': 121, 'found': 122, 'true': 123, 'stay': 124, 'together': 125, 'thought': 126, 'come': 127, 'they': 128, 'sweet': 129, 'tender': 130, 'sender': 131, 'tune': 132, 'humdehumhum': 133, 'gonna': 134, 'last': 135, 'leaving': 136, 'sleep': 137, 'only': 138, 'saw': 139, 'tell': 140, 'hes': 141, 'her': 142, 'sound': 143, 'tread': 144, 'lightly': 145, 'ground': 146, 'ill': 147, 'show': 148, 'life': 149, 'too': 150, 'used': 151, 'darling': 152, 'meant': 153, 'break': 154, 'end': 155, 'yourself': 156, 'little': 157, 'dumbedumdum': 158, 'bedumbedumdum': 159, 'youve': 160, 'dumbbedumbdumb': 161, 'bedumbbedumbdumb': 162, 'by': 163, 'theyre': 164, 'alone': 165, 'misunderstood': 166, 'day': 167, 'dawning': 168, 'some': 169, 'wanted': 170, 'none': 171, 'listen': 172, 'words': 173, 'warning': 174, 'darkest': 175, 'nights': 176, 'nobody': 177, 'knew': 178, 'fight': 179, 'caught': 180, 'really': 181, 'power': 182, 'dreams': 183, 'weave': 184, 'until': 185, 'final': 186, 'hour': 187, 'morning': 188, 'ship': 189, 'gone': 190, 'grieving': 191, 'still': 192, 'pain': 193, 'cry': 194, 'sun': 195, 'try': 196, 'face': 197, 'something': 198, 'sees': 199, 'makes': 200, 'fine': 201, 'who': 202, 'mine': 203, 'leaves': 204, 'walk': 205, 'hand': 206, 'well': 207, 'about': 208, 'things': 209, 'slow': 210, 'theres': 211, 'talk': 212, 'why': 213, 'up': 214, 'lousy': 215, 'packing': 216, 'ive': 217, 'gotta': 218, 'near': 219, 'keeping': 220, 'intention': 221, 'growing': 222, 'taking': 223, 'dimension': 224, 'even': 225, 'better': 226, 'thank': 227, 'god': 228, 'not': 229, 'somebody': 230, 'happy': 231, 'question': 232, 'smile': 233, 'mean': 234, 'much': 235, 'kisses': 236, 'around': 237, 'anywhere': 238, 'advice': 239, 'care': 240, 'use': 241, 'selfish': 242, 'tool': 243, 'fool': 244, 'showing': 245, 'boomerang': 246, 'throwing': 247, 'warm': 248, 'kiss': 249, 'surrender': 250, 'giving': 251, 'been': 252, 'door': 253, 'burning': 254, 'bridges': 255, 'being': 256, 'moving': 257, 'though': 258, 'behind': 259, 'are': 260, 'must': 261, 'sure': 262, 'stood': 263, 'hope': 264, 'this': 265, 'deny': 266, 'sad': 267, 'quiet': 268, 'truth': 269, 'heartaches': 270, 'scars': 271, 'dancing': 272, 'sky': 273, 'shining': 274, 'above': 275, 'hear': 276, 'came': 277, 'couldnt': 278, 'everything': 279, 'back': 280, 'long': 281, 'waitin': 282, 'cold': 283, 'chills': 284, 'bone': 285, 'youd': 286, 'wonderful': 287, 'means': 288, 'special': 289, 'smiles': 290, 'lucky': 291, 'fellow': 292, 'park': 293, 'holds': 294, 'squeezes': 295, 'walking': 296, 'hours': 297, 'talking': 298, 'plan': 299, 'easy': 300, 'gently': 301, 'summer': 302, 'evening': 303, 'breeze': 304, 'grow': 305, 'fingers': 306, 'soft': 307, 'light': 308, 'body': 309, 'velvet': 310, 'night': 311, 'soul': 312, 'slowly': 313, 'shimmer': 314, 'thousand': 315, 'butterflies': 316, 'float': 317, 'put': 318, 'rotten': 319, 'boy': 320, 'tough': 321, 'stuff': 322, 'saying': 323, 'need': 324, 'anymore': 325, 'enough': 326, 'standing': 327, 'creep': 328, 'felt': 329, 'cheap': 330, 'notion': 331, 'deep': 332, 'dumb': 333, 'mistake': 334, 'entitled': 335, 'another': 336, 'beg': 337, 'forgive': 338, 'an': 339, 'feels': 340, 'hoot': 341, 'holler': 342, 'mad': 343, 'under': 344, 'heel': 345, 'holy': 346, 'christ': 347, 'deal': 348, 'sick': 349, 'tired': 350, 'tedious': 351, 'ways': 352, 'aint': 353, 'walkin': 354, 'cutting': 355, 'tie': 356, 'wanna': 357, 'into': 358, 'eye': 359, 'myself': 360, 'counting': 361, 'pride': 362, 'unright': 363, 'neighbours': 364, 'ride': 365, 'burying': 366, 'past': 367, 'peace': 368, 'free': 369, 'sucker': 370, 'street': 371, 'singing': 372, 'shouting': 373, 'staying': 374, 'alive': 375, 'city': 376, 'dead': 377, 'hiding': 378, 'their': 379, 'shame': 380, 'hollow': 381, 'laughter': 382, 'while': 383, 'crying': 384, 'bed': 385, 'pity': 386, 'believed': 387, 'lost': 388, 'from': 389, 'start': 390, 'suffer': 391, 'sell': 392, 'secrets': 393, 'bargain': 394, 'playing': 395, 'smart': 396, 'aching': 397, 'hearts': 398, 'sailing': 399, 'father': 400, 'sister': 401, 'reason': 402, 'linger': 403, 'deeply': 404, 'future': 405, 'casting': 406, 'shadow': 407, 'else': 408, 'fate': 409, 'bags': 410, 'thorough': 411, 'knowing': 412, 'late': 413, 'wait': 414, 'watched': 415, 'harbor': 416, 'sunrise': 417, 'sails': 418, 'almost': 419, 'slack': 420, 'cool': 421, 'rain': 422, 'deck': 423, 'tiny': 424, 'figure': 425, 'rigid': 426, 'restrained': 427, 'filled': 428, 'whats': 429, 'wrong': 430, 'enchained': 431, 'own': 432, 'sorrow': 433, 'tomorrow': 434, 'hate': 435, 'shoulder': 436, 'best': 437, 'friend': 438, 'rely': 439, 'broken': 440, 'feather': 441, 'patch': 442, 'walls': 443, 'tumbling': 444, 'loves': 445, 'blown': 446, 'candle': 447, 'seems': 448, 'hard': 449, 'handle': 450, 'id': 451, 'thinking': 452, 'went': 453, 'house': 454, 'hardly': 455, 'guy': 456, 'closing': 457, 'front': 458, 'emptiness': 459, 'he': 460, 'disapeared': 461, 'his': 462, 'car': 463, 'stunned': 464, 'dreamed': 465, 'lifes': 466, 'part': 467, 'move': 468, 'feet': 469, 'pavement': 470, 'acted': 471, 'told': 472, 'lies': 473, 'meet': 474, 'other': 475, 'guys': 476, 'stupid': 477, 'blind': 478, 'smiled': 479, 'took': 480, 'said': 481, 'may': 482, 'couple': 483, 'men': 484, 'them': 485, 'brother': 486, 'joe': 487, 'seeing': 488, 'lot': 489, 'him': 490, 'nice': 491, 'sitting': 492, 'sittin': 493, 'memories': 494}\n",
            "495\n",
            "<ipython-input-3-fbdddccf8583>:12: FutureWarning: The default value of regex will change from True to False in a future version.\n",
            "  dataset[field] = dataset[field].str.replace('[{}]'.format(string.punctuation), '')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v9x68iN_X6FK"
      },
      "source": [
        "### Create Sequences and Labels\n",
        "\n",
        "After preprocessing, we next need to create sequences and labels. Creating the sequences themselves is similar to before with `texts_to_sequences`, but also including the use of [N-Grams](https://towardsdatascience.com/introduction-to-language-models-n-gram-e323081503d9); creating the labels will now utilize those sequences as well as utilize one-hot encoding over all potential output words."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QmlTsUqfikVO"
      },
      "source": [
        "sequences = []\n",
        "for line in corpus:\n",
        "\ttoken_list = tokenizer.texts_to_sequences([line])[0]\n",
        "\tfor i in range(1, len(token_list)):\n",
        "\t\tn_gram_sequence = token_list[:i+1]\n",
        "\t\tsequences.append(n_gram_sequence)\n",
        "\n",
        "# Pad sequences for equal input length \n",
        "max_sequence_len = max([len(seq) for seq in sequences])\n",
        "sequences = np.array(pad_sequences(sequences, maxlen=max_sequence_len, padding='pre'))\n",
        "\n",
        "# Split sequences between the \"input\" sequence and \"output\" predicted word\n",
        "input_sequences, labels = sequences[:,:-1], sequences[:,-1]\n",
        "# One-hot encode the labels\n",
        "one_hot_labels = tf.keras.utils.to_categorical(labels, num_classes=total_words)"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zsmu3aEId49i"
      },
      "source": [
        "# Check out how some of our data is being stored\n",
        "# The Tokenizer has just a single index per word\n",
        "print(tokenizer.word_index['know'])\n",
        "print(tokenizer.word_index['feeling'])\n",
        "# Input sequences will have multiple indexes\n",
        "print(input_sequences[5])\n",
        "print(input_sequences[6])\n",
        "# And the one hot labels will be as long as the full spread of tokenized words\n",
        "print(one_hot_labels[5])\n",
        "print(one_hot_labels[6])"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "32\n97\n[  0   0   0   0   0   0   0   0   0   0   0   0   0  81  82 142 197  29\n   4]\n[  0   0   0   0   0   0   0   0   0   0   0   0  81  82 142 197  29   4\n 287]\n[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-1TAJMlmfO8r"
      },
      "source": [
        "### Train a Text Generation Model\n",
        "\n",
        "Building an RNN to train our text generation model will be very similar to the sentiment models you've built previously. The only real change necessary is to make sure to use Categorical instead of Binary Cross Entropy as the loss function - we could use Binary before since the sentiment was only 0 or 1, but now there are hundreds of categories.\n",
        "\n",
        "From there, we should also consider using *more* epochs than before, as text generation can take a little longer to converge than sentiment analysis, *and* we aren't working with all that much data yet. I'll set it at 200 epochs here since we're only use part of the dataset, and training will tail off quite a bit over that many epochs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G1YXuxIqfygN",
        "tags": [
          "outputPrepend"
        ]
      },
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Embedding(total_words, 64, input_length=max_sequence_len-1))\n",
        "model.add(Bidirectional(LSTM(20)))\n",
        "model.add(Dense(total_words, activation='softmax'))\n",
        "model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
        "history = model.fit(input_sequences, one_hot_labels, epochs=200, verbose=1)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " 5.3733 - accuracy: 0.0399\n",
            "Epoch 4/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 5.3227 - accuracy: 0.0338\n",
            "Epoch 5/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 5.2562 - accuracy: 0.0399\n",
            "Epoch 6/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 5.1851 - accuracy: 0.0409\n",
            "Epoch 7/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 5.1222 - accuracy: 0.0404\n",
            "Epoch 8/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 5.0489 - accuracy: 0.0434\n",
            "Epoch 9/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 4.9635 - accuracy: 0.0565\n",
            "Epoch 10/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 4.8749 - accuracy: 0.0802\n",
            "Epoch 11/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 4.7769 - accuracy: 0.0903\n",
            "Epoch 12/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 4.6839 - accuracy: 0.1065\n",
            "Epoch 13/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 4.5874 - accuracy: 0.1135\n",
            "Epoch 14/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 4.4893 - accuracy: 0.1312\n",
            "Epoch 15/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 4.3921 - accuracy: 0.1362\n",
            "Epoch 16/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 4.2963 - accuracy: 0.1509\n",
            "Epoch 17/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 4.2103 - accuracy: 0.1609\n",
            "Epoch 18/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 4.1084 - accuracy: 0.1715\n",
            "Epoch 19/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 4.0416 - accuracy: 0.1761\n",
            "Epoch 20/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 3.9606 - accuracy: 0.1842\n",
            "Epoch 21/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 3.8569 - accuracy: 0.2048\n",
            "Epoch 22/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 3.7699 - accuracy: 0.2190\n",
            "Epoch 23/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 3.6987 - accuracy: 0.2356\n",
            "Epoch 24/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 3.6402 - accuracy: 0.2447\n",
            "Epoch 25/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 3.5722 - accuracy: 0.2508\n",
            "Epoch 26/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 3.5108 - accuracy: 0.2785\n",
            "Epoch 27/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 3.4213 - accuracy: 0.2906\n",
            "Epoch 28/200\n",
            "62/62 [==============================] - 1s 15ms/step - loss: 3.3410 - accuracy: 0.3138\n",
            "Epoch 29/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 3.2899 - accuracy: 0.3214\n",
            "Epoch 30/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 3.2495 - accuracy: 0.3330\n",
            "Epoch 31/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 3.1989 - accuracy: 0.3375\n",
            "Epoch 32/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 3.1413 - accuracy: 0.3547\n",
            "Epoch 33/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 3.0757 - accuracy: 0.3688\n",
            "Epoch 34/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 2.9963 - accuracy: 0.3829\n",
            "Epoch 35/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 2.9472 - accuracy: 0.4051\n",
            "Epoch 36/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 2.8957 - accuracy: 0.4021\n",
            "Epoch 37/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 2.8400 - accuracy: 0.4122\n",
            "Epoch 38/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 2.7905 - accuracy: 0.4208\n",
            "Epoch 39/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 2.7409 - accuracy: 0.4299\n",
            "Epoch 40/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 2.6774 - accuracy: 0.4490\n",
            "Epoch 41/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 2.6247 - accuracy: 0.4632\n",
            "Epoch 42/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 2.5775 - accuracy: 0.4743\n",
            "Epoch 43/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 2.5308 - accuracy: 0.4773\n",
            "Epoch 44/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 2.4900 - accuracy: 0.4854\n",
            "Epoch 45/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 2.4522 - accuracy: 0.4899\n",
            "Epoch 46/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 2.3985 - accuracy: 0.5005\n",
            "Epoch 47/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 2.3651 - accuracy: 0.5101\n",
            "Epoch 48/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 2.3187 - accuracy: 0.5121\n",
            "Epoch 49/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 2.2898 - accuracy: 0.5207\n",
            "Epoch 50/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 2.2485 - accuracy: 0.5348\n",
            "Epoch 51/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 2.2093 - accuracy: 0.5328\n",
            "Epoch 52/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 2.1702 - accuracy: 0.5439\n",
            "Epoch 53/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 2.1525 - accuracy: 0.5535\n",
            "Epoch 54/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 2.1364 - accuracy: 0.5570\n",
            "Epoch 55/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 2.1152 - accuracy: 0.5560\n",
            "Epoch 56/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 2.0449 - accuracy: 0.5777\n",
            "Epoch 57/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.9986 - accuracy: 0.5853\n",
            "Epoch 58/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.9883 - accuracy: 0.5843\n",
            "Epoch 59/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.9490 - accuracy: 0.6019\n",
            "Epoch 60/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.9139 - accuracy: 0.5999\n",
            "Epoch 61/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.8859 - accuracy: 0.6201\n",
            "Epoch 62/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.8431 - accuracy: 0.6201\n",
            "Epoch 63/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.8036 - accuracy: 0.6342\n",
            "Epoch 64/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.7701 - accuracy: 0.6413\n",
            "Epoch 65/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 1.7374 - accuracy: 0.6473\n",
            "Epoch 66/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.7177 - accuracy: 0.6463\n",
            "Epoch 67/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 1.6882 - accuracy: 0.6544\n",
            "Epoch 68/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.6532 - accuracy: 0.6640\n",
            "Epoch 69/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.6206 - accuracy: 0.6680\n",
            "Epoch 70/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.5927 - accuracy: 0.6741\n",
            "Epoch 71/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.5766 - accuracy: 0.6776\n",
            "Epoch 72/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.5611 - accuracy: 0.6887\n",
            "Epoch 73/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.5281 - accuracy: 0.6887\n",
            "Epoch 74/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 1.5009 - accuracy: 0.6927\n",
            "Epoch 75/200\n",
            "62/62 [==============================] - 1s 15ms/step - loss: 1.4727 - accuracy: 0.7013\n",
            "Epoch 76/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.4592 - accuracy: 0.7038\n",
            "Epoch 77/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.4430 - accuracy: 0.7033\n",
            "Epoch 78/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 1.4302 - accuracy: 0.7089\n",
            "Epoch 79/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 1.3992 - accuracy: 0.7159\n",
            "Epoch 80/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.3866 - accuracy: 0.7200\n",
            "Epoch 81/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.3878 - accuracy: 0.7089\n",
            "Epoch 82/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.4130 - accuracy: 0.6988\n",
            "Epoch 83/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.3521 - accuracy: 0.7205\n",
            "Epoch 84/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.3423 - accuracy: 0.7225\n",
            "Epoch 85/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.3187 - accuracy: 0.7265\n",
            "Epoch 86/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 1.2843 - accuracy: 0.7301\n",
            "Epoch 87/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.2623 - accuracy: 0.7412\n",
            "Epoch 88/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.2914 - accuracy: 0.7281\n",
            "Epoch 89/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.4321 - accuracy: 0.6852\n",
            "Epoch 90/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.3546 - accuracy: 0.7069\n",
            "Epoch 91/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 1.2572 - accuracy: 0.7326\n",
            "Epoch 92/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.2272 - accuracy: 0.7321\n",
            "Epoch 93/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.1936 - accuracy: 0.7472\n",
            "Epoch 94/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.1631 - accuracy: 0.7508\n",
            "Epoch 95/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.1346 - accuracy: 0.7654\n",
            "Epoch 96/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.1144 - accuracy: 0.7629\n",
            "Epoch 97/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0950 - accuracy: 0.7725\n",
            "Epoch 98/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0800 - accuracy: 0.7760\n",
            "Epoch 99/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0610 - accuracy: 0.7805\n",
            "Epoch 100/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0498 - accuracy: 0.7800\n",
            "Epoch 101/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0306 - accuracy: 0.7820\n",
            "Epoch 102/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0189 - accuracy: 0.7810\n",
            "Epoch 103/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0035 - accuracy: 0.7866\n",
            "Epoch 104/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0052 - accuracy: 0.7881\n",
            "Epoch 105/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0353 - accuracy: 0.7770\n",
            "Epoch 106/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0049 - accuracy: 0.7790\n",
            "Epoch 107/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.9883 - accuracy: 0.7871\n",
            "Epoch 108/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.9818 - accuracy: 0.7901\n",
            "Epoch 109/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.9592 - accuracy: 0.7921\n",
            "Epoch 110/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 1.0035 - accuracy: 0.7820\n",
            "Epoch 111/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.9733 - accuracy: 0.7841\n",
            "Epoch 112/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.9361 - accuracy: 0.7931\n",
            "Epoch 113/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.9103 - accuracy: 0.7967\n",
            "Epoch 114/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.8895 - accuracy: 0.8047\n",
            "Epoch 115/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.8751 - accuracy: 0.8103\n",
            "Epoch 116/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.8609 - accuracy: 0.8088\n",
            "Epoch 117/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.8464 - accuracy: 0.8098\n",
            "Epoch 118/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.8371 - accuracy: 0.8133\n",
            "Epoch 119/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.8386 - accuracy: 0.8088\n",
            "Epoch 120/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.8374 - accuracy: 0.8078\n",
            "Epoch 121/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.8271 - accuracy: 0.8078\n",
            "Epoch 122/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.8083 - accuracy: 0.8138\n",
            "Epoch 123/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.8004 - accuracy: 0.8179\n",
            "Epoch 124/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.8210 - accuracy: 0.8113\n",
            "Epoch 125/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.8151 - accuracy: 0.8179\n",
            "Epoch 126/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.8528 - accuracy: 0.8037\n",
            "Epoch 127/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.8889 - accuracy: 0.7972\n",
            "Epoch 128/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.8133 - accuracy: 0.8184\n",
            "Epoch 129/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.7821 - accuracy: 0.8295\n",
            "Epoch 130/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.7724 - accuracy: 0.8320\n",
            "Epoch 131/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.7550 - accuracy: 0.8340\n",
            "Epoch 132/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.7395 - accuracy: 0.8365\n",
            "Epoch 133/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.7254 - accuracy: 0.8375\n",
            "Epoch 134/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.7139 - accuracy: 0.8370\n",
            "Epoch 135/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.7065 - accuracy: 0.8375\n",
            "Epoch 136/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.6943 - accuracy: 0.8426\n",
            "Epoch 137/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.6837 - accuracy: 0.8436\n",
            "Epoch 138/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.6995 - accuracy: 0.8335\n",
            "Epoch 139/200\n",
            "62/62 [==============================] - 3s 51ms/step - loss: 0.6838 - accuracy: 0.8401\n",
            "Epoch 140/200\n",
            "62/62 [==============================] - 1s 17ms/step - loss: 0.6688 - accuracy: 0.8416\n",
            "Epoch 141/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.6611 - accuracy: 0.8466\n",
            "Epoch 142/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.6558 - accuracy: 0.8421\n",
            "Epoch 143/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.6414 - accuracy: 0.8491\n",
            "Epoch 144/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.6807 - accuracy: 0.8406\n",
            "Epoch 145/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.6953 - accuracy: 0.8380\n",
            "Epoch 146/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.6873 - accuracy: 0.8401\n",
            "Epoch 147/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.6372 - accuracy: 0.8507\n",
            "Epoch 148/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.6218 - accuracy: 0.8537\n",
            "Epoch 149/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.6134 - accuracy: 0.8587\n",
            "Epoch 150/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.6076 - accuracy: 0.8522\n",
            "Epoch 151/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.5953 - accuracy: 0.8557\n",
            "Epoch 152/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.5886 - accuracy: 0.8613\n",
            "Epoch 153/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.6238 - accuracy: 0.8491\n",
            "Epoch 154/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.6040 - accuracy: 0.8507\n",
            "Epoch 155/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.5822 - accuracy: 0.8633\n",
            "Epoch 156/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.5730 - accuracy: 0.8658\n",
            "Epoch 157/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.5650 - accuracy: 0.8607\n",
            "Epoch 158/200\n",
            "62/62 [==============================] - 1s 15ms/step - loss: 0.5602 - accuracy: 0.8638\n",
            "Epoch 159/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.5534 - accuracy: 0.8648\n",
            "Epoch 160/200\n",
            "62/62 [==============================] - 1s 12ms/step - loss: 0.5529 - accuracy: 0.8648\n",
            "Epoch 161/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.5451 - accuracy: 0.8658\n",
            "Epoch 162/200\n",
            "62/62 [==============================] - 3s 51ms/step - loss: 0.5352 - accuracy: 0.8658\n",
            "Epoch 163/200\n",
            "62/62 [==============================] - 1s 15ms/step - loss: 0.5278 - accuracy: 0.8698\n",
            "Epoch 164/200\n",
            "62/62 [==============================] - 1s 15ms/step - loss: 0.5185 - accuracy: 0.8703\n",
            "Epoch 165/200\n",
            "62/62 [==============================] - 1s 13ms/step - loss: 0.5139 - accuracy: 0.8724\n",
            "Epoch 166/200\n",
            "62/62 [==============================] - 1s 16ms/step - loss: 0.5068 - accuracy: 0.8764\n",
            "Epoch 167/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.5022 - accuracy: 0.8754\n",
            "Epoch 168/200\n",
            "62/62 [==============================] - 1s 15ms/step - loss: 0.4963 - accuracy: 0.8799\n",
            "Epoch 169/200\n",
            "62/62 [==============================] - 1s 14ms/step - loss: 0.4929 - accuracy: 0.8799\n",
            "Epoch 170/200\n",
            "62/62 [==============================] - 1s 19ms/step - loss: 0.4864 - accuracy: 0.8794\n",
            "Epoch 171/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4847 - accuracy: 0.8769\n",
            "Epoch 172/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4828 - accuracy: 0.8769\n",
            "Epoch 173/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4946 - accuracy: 0.8718\n",
            "Epoch 174/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.4848 - accuracy: 0.8734\n",
            "Epoch 175/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4741 - accuracy: 0.8819\n",
            "Epoch 176/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4710 - accuracy: 0.8794\n",
            "Epoch 177/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4623 - accuracy: 0.8835\n",
            "Epoch 178/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4598 - accuracy: 0.8870\n",
            "Epoch 179/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.4539 - accuracy: 0.8855\n",
            "Epoch 180/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4525 - accuracy: 0.8824\n",
            "Epoch 181/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4477 - accuracy: 0.8890\n",
            "Epoch 182/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4412 - accuracy: 0.8835\n",
            "Epoch 183/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4386 - accuracy: 0.8915\n",
            "Epoch 184/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.4351 - accuracy: 0.8885\n",
            "Epoch 185/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4390 - accuracy: 0.8900\n",
            "Epoch 186/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4592 - accuracy: 0.8824\n",
            "Epoch 187/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4486 - accuracy: 0.8814\n",
            "Epoch 188/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4417 - accuracy: 0.8829\n",
            "Epoch 189/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4294 - accuracy: 0.8900\n",
            "Epoch 190/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4386 - accuracy: 0.8910\n",
            "Epoch 191/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4364 - accuracy: 0.8880\n",
            "Epoch 192/200\n",
            "62/62 [==============================] - 1s 9ms/step - loss: 0.4417 - accuracy: 0.8829\n",
            "Epoch 193/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.4311 - accuracy: 0.8915\n",
            "Epoch 194/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.4135 - accuracy: 0.8905\n",
            "Epoch 195/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 0.4292 - accuracy: 0.8915\n",
            "Epoch 196/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 0.4179 - accuracy: 0.8865\n",
            "Epoch 197/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.4706 - accuracy: 0.8804\n",
            "Epoch 198/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.4526 - accuracy: 0.8840\n",
            "Epoch 199/200\n",
            "62/62 [==============================] - 1s 11ms/step - loss: 0.4352 - accuracy: 0.8855\n",
            "Epoch 200/200\n",
            "62/62 [==============================] - 1s 10ms/step - loss: 0.4126 - accuracy: 0.8910\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AXVFpoREhV6Y"
      },
      "source": [
        "### View the Training Graph"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aeSNfS7uhch0"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "def plot_graphs(history, string):\n",
        "  plt.plot(history.history[string])\n",
        "  plt.xlabel(\"Epochs\")\n",
        "  plt.ylabel(string)\n",
        "  plt.show()\n",
        "\n",
        "plot_graphs(history, 'accuracy')"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "<Figure size 432x288 with 1 Axes>",
            "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (https://matplotlib.org/) -->\n<svg height=\"262.19625pt\" version=\"1.1\" viewBox=\"0 0 385.78125 262.19625\" width=\"385.78125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <metadata>\n  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2021-06-29T20:04:28.202106</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.3.4, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 262.19625 \nL 385.78125 262.19625 \nL 385.78125 0 \nL 0 0 \nz\n\" style=\"fill:none;\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 43.78125 224.64 \nL 378.58125 224.64 \nL 378.58125 7.2 \nL 43.78125 7.2 \nz\n\" style=\"fill:#ffffff;\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <defs>\n       <path d=\"M 0 0 \nL 0 3.5 \n\" id=\"mb6297117cf\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n      </defs>\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.999432\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- 0 -->\n      <g transform=\"translate(55.818182 239.238437)scale(0.1 -0.1)\">\n       <defs>\n        <path d=\"M 31.78125 66.40625 \nQ 24.171875 66.40625 20.328125 58.90625 \nQ 16.5 51.421875 16.5 36.375 \nQ 16.5 21.390625 20.328125 13.890625 \nQ 24.171875 6.390625 31.78125 6.390625 \nQ 39.453125 6.390625 43.28125 13.890625 \nQ 47.125 21.390625 47.125 36.375 \nQ 47.125 51.421875 43.28125 58.90625 \nQ 39.453125 66.40625 31.78125 66.40625 \nz\nM 31.78125 74.21875 \nQ 44.046875 74.21875 50.515625 64.515625 \nQ 56.984375 54.828125 56.984375 36.375 \nQ 56.984375 17.96875 50.515625 8.265625 \nQ 44.046875 -1.421875 31.78125 -1.421875 \nQ 19.53125 -1.421875 13.0625 8.265625 \nQ 6.59375 17.96875 6.59375 36.375 \nQ 6.59375 54.828125 13.0625 64.515625 \nQ 19.53125 74.21875 31.78125 74.21875 \nz\n\" id=\"DejaVuSans-48\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_2\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"97.23607\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- 25 -->\n      <g transform=\"translate(90.87357 239.238437)scale(0.1 -0.1)\">\n       <defs>\n        <path d=\"M 19.1875 8.296875 \nL 53.609375 8.296875 \nL 53.609375 0 \nL 7.328125 0 \nL 7.328125 8.296875 \nQ 12.9375 14.109375 22.625 23.890625 \nQ 32.328125 33.6875 34.8125 36.53125 \nQ 39.546875 41.84375 41.421875 45.53125 \nQ 43.3125 49.21875 43.3125 52.78125 \nQ 43.3125 58.59375 39.234375 62.25 \nQ 35.15625 65.921875 28.609375 65.921875 \nQ 23.96875 65.921875 18.8125 64.3125 \nQ 13.671875 62.703125 7.8125 59.421875 \nL 7.8125 69.390625 \nQ 13.765625 71.78125 18.9375 73 \nQ 24.125 74.21875 28.421875 74.21875 \nQ 39.75 74.21875 46.484375 68.546875 \nQ 53.21875 62.890625 53.21875 53.421875 \nQ 53.21875 48.921875 51.53125 44.890625 \nQ 49.859375 40.875 45.40625 35.40625 \nQ 44.1875 33.984375 37.640625 27.21875 \nQ 31.109375 20.453125 19.1875 8.296875 \nz\n\" id=\"DejaVuSans-50\"/>\n        <path d=\"M 10.796875 72.90625 \nL 49.515625 72.90625 \nL 49.515625 64.59375 \nL 19.828125 64.59375 \nL 19.828125 46.734375 \nQ 21.96875 47.46875 24.109375 47.828125 \nQ 26.265625 48.1875 28.421875 48.1875 \nQ 40.625 48.1875 47.75 41.5 \nQ 54.890625 34.8125 54.890625 23.390625 \nQ 54.890625 11.625 47.5625 5.09375 \nQ 40.234375 -1.421875 26.90625 -1.421875 \nQ 22.3125 -1.421875 17.546875 -0.640625 \nQ 12.796875 0.140625 7.71875 1.703125 \nL 7.71875 11.625 \nQ 12.109375 9.234375 16.796875 8.0625 \nQ 21.484375 6.890625 26.703125 6.890625 \nQ 35.15625 6.890625 40.078125 11.328125 \nQ 45.015625 15.765625 45.015625 23.390625 \nQ 45.015625 31 40.078125 35.4375 \nQ 35.15625 39.890625 26.703125 39.890625 \nQ 22.75 39.890625 18.8125 39.015625 \nQ 14.890625 38.140625 10.796875 36.28125 \nz\n\" id=\"DejaVuSans-53\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-50\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_3\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"135.472707\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 50 -->\n      <g transform=\"translate(129.110207 239.238437)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-53\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_4\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"173.709345\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 75 -->\n      <g transform=\"translate(167.346845 239.238437)scale(0.1 -0.1)\">\n       <defs>\n        <path d=\"M 8.203125 72.90625 \nL 55.078125 72.90625 \nL 55.078125 68.703125 \nL 28.609375 0 \nL 18.3125 0 \nL 43.21875 64.59375 \nL 8.203125 64.59375 \nz\n\" id=\"DejaVuSans-55\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-55\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_5\">\n     <g id=\"line2d_5\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"211.945983\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_5\">\n      <!-- 100 -->\n      <g transform=\"translate(202.402233 239.238437)scale(0.1 -0.1)\">\n       <defs>\n        <path d=\"M 12.40625 8.296875 \nL 28.515625 8.296875 \nL 28.515625 63.921875 \nL 10.984375 60.40625 \nL 10.984375 69.390625 \nL 28.421875 72.90625 \nL 38.28125 72.90625 \nL 38.28125 8.296875 \nL 54.390625 8.296875 \nL 54.390625 0 \nL 12.40625 0 \nz\n\" id=\"DejaVuSans-49\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-49\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_6\">\n     <g id=\"line2d_6\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"250.18262\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 125 -->\n      <g transform=\"translate(240.63887 239.238437)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-49\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-50\"/>\n       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_7\">\n     <g id=\"line2d_7\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"288.419258\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_7\">\n      <!-- 150 -->\n      <g transform=\"translate(278.875508 239.238437)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-49\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-53\"/>\n       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_8\">\n     <g id=\"line2d_8\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"326.655896\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 175 -->\n      <g transform=\"translate(317.112146 239.238437)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-49\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-55\"/>\n       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_9\">\n     <g id=\"line2d_9\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"364.892534\" xlink:href=\"#mb6297117cf\" y=\"224.64\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 200 -->\n      <g transform=\"translate(355.348784 239.238437)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-50\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_10\">\n     <!-- Epochs -->\n     <g transform=\"translate(193.265625 252.916562)scale(0.1 -0.1)\">\n      <defs>\n       <path d=\"M 9.8125 72.90625 \nL 55.90625 72.90625 \nL 55.90625 64.59375 \nL 19.671875 64.59375 \nL 19.671875 43.015625 \nL 54.390625 43.015625 \nL 54.390625 34.71875 \nL 19.671875 34.71875 \nL 19.671875 8.296875 \nL 56.78125 8.296875 \nL 56.78125 0 \nL 9.8125 0 \nz\n\" id=\"DejaVuSans-69\"/>\n       <path d=\"M 18.109375 8.203125 \nL 18.109375 -20.796875 \nL 9.078125 -20.796875 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.390625 \nQ 20.953125 51.265625 25.265625 53.625 \nQ 29.59375 56 35.59375 56 \nQ 45.5625 56 51.78125 48.09375 \nQ 58.015625 40.1875 58.015625 27.296875 \nQ 58.015625 14.40625 51.78125 6.484375 \nQ 45.5625 -1.421875 35.59375 -1.421875 \nQ 29.59375 -1.421875 25.265625 0.953125 \nQ 20.953125 3.328125 18.109375 8.203125 \nz\nM 48.6875 27.296875 \nQ 48.6875 37.203125 44.609375 42.84375 \nQ 40.53125 48.484375 33.40625 48.484375 \nQ 26.265625 48.484375 22.1875 42.84375 \nQ 18.109375 37.203125 18.109375 27.296875 \nQ 18.109375 17.390625 22.1875 11.75 \nQ 26.265625 6.109375 33.40625 6.109375 \nQ 40.53125 6.109375 44.609375 11.75 \nQ 48.6875 17.390625 48.6875 27.296875 \nz\n\" id=\"DejaVuSans-112\"/>\n       <path d=\"M 30.609375 48.390625 \nQ 23.390625 48.390625 19.1875 42.75 \nQ 14.984375 37.109375 14.984375 27.296875 \nQ 14.984375 17.484375 19.15625 11.84375 \nQ 23.34375 6.203125 30.609375 6.203125 \nQ 37.796875 6.203125 41.984375 11.859375 \nQ 46.1875 17.53125 46.1875 27.296875 \nQ 46.1875 37.015625 41.984375 42.703125 \nQ 37.796875 48.390625 30.609375 48.390625 \nz\nM 30.609375 56 \nQ 42.328125 56 49.015625 48.375 \nQ 55.71875 40.765625 55.71875 27.296875 \nQ 55.71875 13.875 49.015625 6.21875 \nQ 42.328125 -1.421875 30.609375 -1.421875 \nQ 18.84375 -1.421875 12.171875 6.21875 \nQ 5.515625 13.875 5.515625 27.296875 \nQ 5.515625 40.765625 12.171875 48.375 \nQ 18.84375 56 30.609375 56 \nz\n\" id=\"DejaVuSans-111\"/>\n       <path d=\"M 48.78125 52.59375 \nL 48.78125 44.1875 \nQ 44.96875 46.296875 41.140625 47.34375 \nQ 37.3125 48.390625 33.40625 48.390625 \nQ 24.65625 48.390625 19.8125 42.84375 \nQ 14.984375 37.3125 14.984375 27.296875 \nQ 14.984375 17.28125 19.8125 11.734375 \nQ 24.65625 6.203125 33.40625 6.203125 \nQ 37.3125 6.203125 41.140625 7.25 \nQ 44.96875 8.296875 48.78125 10.40625 \nL 48.78125 2.09375 \nQ 45.015625 0.34375 40.984375 -0.53125 \nQ 36.96875 -1.421875 32.421875 -1.421875 \nQ 20.0625 -1.421875 12.78125 6.34375 \nQ 5.515625 14.109375 5.515625 27.296875 \nQ 5.515625 40.671875 12.859375 48.328125 \nQ 20.21875 56 33.015625 56 \nQ 37.15625 56 41.109375 55.140625 \nQ 45.0625 54.296875 48.78125 52.59375 \nz\n\" id=\"DejaVuSans-99\"/>\n       <path d=\"M 54.890625 33.015625 \nL 54.890625 0 \nL 45.90625 0 \nL 45.90625 32.71875 \nQ 45.90625 40.484375 42.875 44.328125 \nQ 39.84375 48.1875 33.796875 48.1875 \nQ 26.515625 48.1875 22.3125 43.546875 \nQ 18.109375 38.921875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 75.984375 \nL 18.109375 75.984375 \nL 18.109375 46.1875 \nQ 21.34375 51.125 25.703125 53.5625 \nQ 30.078125 56 35.796875 56 \nQ 45.21875 56 50.046875 50.171875 \nQ 54.890625 44.34375 54.890625 33.015625 \nz\n\" id=\"DejaVuSans-104\"/>\n       <path d=\"M 44.28125 53.078125 \nL 44.28125 44.578125 \nQ 40.484375 46.53125 36.375 47.5 \nQ 32.28125 48.484375 27.875 48.484375 \nQ 21.1875 48.484375 17.84375 46.4375 \nQ 14.5 44.390625 14.5 40.28125 \nQ 14.5 37.15625 16.890625 35.375 \nQ 19.28125 33.59375 26.515625 31.984375 \nL 29.59375 31.296875 \nQ 39.15625 29.25 43.1875 25.515625 \nQ 47.21875 21.78125 47.21875 15.09375 \nQ 47.21875 7.46875 41.1875 3.015625 \nQ 35.15625 -1.421875 24.609375 -1.421875 \nQ 20.21875 -1.421875 15.453125 -0.5625 \nQ 10.6875 0.296875 5.421875 2 \nL 5.421875 11.28125 \nQ 10.40625 8.6875 15.234375 7.390625 \nQ 20.0625 6.109375 24.8125 6.109375 \nQ 31.15625 6.109375 34.5625 8.28125 \nQ 37.984375 10.453125 37.984375 14.40625 \nQ 37.984375 18.0625 35.515625 20.015625 \nQ 33.0625 21.96875 24.703125 23.78125 \nL 21.578125 24.515625 \nQ 13.234375 26.265625 9.515625 29.90625 \nQ 5.8125 33.546875 5.8125 39.890625 \nQ 5.8125 47.609375 11.28125 51.796875 \nQ 16.75 56 26.8125 56 \nQ 31.78125 56 36.171875 55.265625 \nQ 40.578125 54.546875 44.28125 53.078125 \nz\n\" id=\"DejaVuSans-115\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-69\"/>\n      <use x=\"63.183594\" xlink:href=\"#DejaVuSans-112\"/>\n      <use x=\"126.660156\" xlink:href=\"#DejaVuSans-111\"/>\n      <use x=\"187.841797\" xlink:href=\"#DejaVuSans-99\"/>\n      <use x=\"242.822266\" xlink:href=\"#DejaVuSans-104\"/>\n      <use x=\"306.201172\" xlink:href=\"#DejaVuSans-115\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_10\">\n      <defs>\n       <path d=\"M 0 0 \nL -3.5 0 \n\" id=\"mad94285763\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n      </defs>\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mad94285763\" y=\"218.984046\"/>\n      </g>\n     </g>\n     <g id=\"text_11\">\n      <!-- 0.0 -->\n      <g transform=\"translate(20.878125 222.783265)scale(0.1 -0.1)\">\n       <defs>\n        <path d=\"M 10.6875 12.40625 \nL 21 12.40625 \nL 21 0 \nL 10.6875 0 \nz\n\" id=\"DejaVuSans-46\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_11\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mad94285763\" y=\"173.690712\"/>\n      </g>\n     </g>\n     <g id=\"text_12\">\n      <!-- 0.2 -->\n      <g transform=\"translate(20.878125 177.489931)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-50\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_12\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mad94285763\" y=\"128.397377\"/>\n      </g>\n     </g>\n     <g id=\"text_13\">\n      <!-- 0.4 -->\n      <g transform=\"translate(20.878125 132.196596)scale(0.1 -0.1)\">\n       <defs>\n        <path d=\"M 37.796875 64.3125 \nL 12.890625 25.390625 \nL 37.796875 25.390625 \nz\nM 35.203125 72.90625 \nL 47.609375 72.90625 \nL 47.609375 25.390625 \nL 58.015625 25.390625 \nL 58.015625 17.1875 \nL 47.609375 17.1875 \nL 47.609375 0 \nL 37.796875 0 \nL 37.796875 17.1875 \nL 4.890625 17.1875 \nL 4.890625 26.703125 \nz\n\" id=\"DejaVuSans-52\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-52\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_4\">\n     <g id=\"line2d_13\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mad94285763\" y=\"83.104043\"/>\n      </g>\n     </g>\n     <g id=\"text_14\">\n      <!-- 0.6 -->\n      <g transform=\"translate(20.878125 86.903262)scale(0.1 -0.1)\">\n       <defs>\n        <path d=\"M 33.015625 40.375 \nQ 26.375 40.375 22.484375 35.828125 \nQ 18.609375 31.296875 18.609375 23.390625 \nQ 18.609375 15.53125 22.484375 10.953125 \nQ 26.375 6.390625 33.015625 6.390625 \nQ 39.65625 6.390625 43.53125 10.953125 \nQ 47.40625 15.53125 47.40625 23.390625 \nQ 47.40625 31.296875 43.53125 35.828125 \nQ 39.65625 40.375 33.015625 40.375 \nz\nM 52.59375 71.296875 \nL 52.59375 62.3125 \nQ 48.875 64.0625 45.09375 64.984375 \nQ 41.3125 65.921875 37.59375 65.921875 \nQ 27.828125 65.921875 22.671875 59.328125 \nQ 17.53125 52.734375 16.796875 39.40625 \nQ 19.671875 43.65625 24.015625 45.921875 \nQ 28.375 48.1875 33.59375 48.1875 \nQ 44.578125 48.1875 50.953125 41.515625 \nQ 57.328125 34.859375 57.328125 23.390625 \nQ 57.328125 12.15625 50.6875 5.359375 \nQ 44.046875 -1.421875 33.015625 -1.421875 \nQ 20.359375 -1.421875 13.671875 8.265625 \nQ 6.984375 17.96875 6.984375 36.375 \nQ 6.984375 53.65625 15.1875 63.9375 \nQ 23.390625 74.21875 37.203125 74.21875 \nQ 40.921875 74.21875 44.703125 73.484375 \nQ 48.484375 72.75 52.59375 71.296875 \nz\n\" id=\"DejaVuSans-54\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_5\">\n     <g id=\"line2d_14\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mad94285763\" y=\"37.810708\"/>\n      </g>\n     </g>\n     <g id=\"text_15\">\n      <!-- 0.8 -->\n      <g transform=\"translate(20.878125 41.609927)scale(0.1 -0.1)\">\n       <defs>\n        <path d=\"M 31.78125 34.625 \nQ 24.75 34.625 20.71875 30.859375 \nQ 16.703125 27.09375 16.703125 20.515625 \nQ 16.703125 13.921875 20.71875 10.15625 \nQ 24.75 6.390625 31.78125 6.390625 \nQ 38.8125 6.390625 42.859375 10.171875 \nQ 46.921875 13.96875 46.921875 20.515625 \nQ 46.921875 27.09375 42.890625 30.859375 \nQ 38.875 34.625 31.78125 34.625 \nz\nM 21.921875 38.8125 \nQ 15.578125 40.375 12.03125 44.71875 \nQ 8.5 49.078125 8.5 55.328125 \nQ 8.5 64.0625 14.71875 69.140625 \nQ 20.953125 74.21875 31.78125 74.21875 \nQ 42.671875 74.21875 48.875 69.140625 \nQ 55.078125 64.0625 55.078125 55.328125 \nQ 55.078125 49.078125 51.53125 44.71875 \nQ 48 40.375 41.703125 38.8125 \nQ 48.828125 37.15625 52.796875 32.3125 \nQ 56.78125 27.484375 56.78125 20.515625 \nQ 56.78125 9.90625 50.3125 4.234375 \nQ 43.84375 -1.421875 31.78125 -1.421875 \nQ 19.734375 -1.421875 13.25 4.234375 \nQ 6.78125 9.90625 6.78125 20.515625 \nQ 6.78125 27.484375 10.78125 32.3125 \nQ 14.796875 37.15625 21.921875 38.8125 \nz\nM 18.3125 54.390625 \nQ 18.3125 48.734375 21.84375 45.5625 \nQ 25.390625 42.390625 31.78125 42.390625 \nQ 38.140625 42.390625 41.71875 45.5625 \nQ 45.3125 48.734375 45.3125 54.390625 \nQ 45.3125 60.0625 41.71875 63.234375 \nQ 38.140625 66.40625 31.78125 66.40625 \nQ 25.390625 66.40625 21.84375 63.234375 \nQ 18.3125 60.0625 18.3125 54.390625 \nz\n\" id=\"DejaVuSans-56\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-56\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_16\">\n     <!-- accuracy -->\n     <g transform=\"translate(14.798438 138.479375)rotate(-90)scale(0.1 -0.1)\">\n      <defs>\n       <path d=\"M 34.28125 27.484375 \nQ 23.390625 27.484375 19.1875 25 \nQ 14.984375 22.515625 14.984375 16.5 \nQ 14.984375 11.71875 18.140625 8.90625 \nQ 21.296875 6.109375 26.703125 6.109375 \nQ 34.1875 6.109375 38.703125 11.40625 \nQ 43.21875 16.703125 43.21875 25.484375 \nL 43.21875 27.484375 \nz\nM 52.203125 31.203125 \nL 52.203125 0 \nL 43.21875 0 \nL 43.21875 8.296875 \nQ 40.140625 3.328125 35.546875 0.953125 \nQ 30.953125 -1.421875 24.3125 -1.421875 \nQ 15.921875 -1.421875 10.953125 3.296875 \nQ 6 8.015625 6 15.921875 \nQ 6 25.140625 12.171875 29.828125 \nQ 18.359375 34.515625 30.609375 34.515625 \nL 43.21875 34.515625 \nL 43.21875 35.40625 \nQ 43.21875 41.609375 39.140625 45 \nQ 35.0625 48.390625 27.6875 48.390625 \nQ 23 48.390625 18.546875 47.265625 \nQ 14.109375 46.140625 10.015625 43.890625 \nL 10.015625 52.203125 \nQ 14.9375 54.109375 19.578125 55.046875 \nQ 24.21875 56 28.609375 56 \nQ 40.484375 56 46.34375 49.84375 \nQ 52.203125 43.703125 52.203125 31.203125 \nz\n\" id=\"DejaVuSans-97\"/>\n       <path d=\"M 8.5 21.578125 \nL 8.5 54.6875 \nL 17.484375 54.6875 \nL 17.484375 21.921875 \nQ 17.484375 14.15625 20.5 10.265625 \nQ 23.53125 6.390625 29.59375 6.390625 \nQ 36.859375 6.390625 41.078125 11.03125 \nQ 45.3125 15.671875 45.3125 23.6875 \nL 45.3125 54.6875 \nL 54.296875 54.6875 \nL 54.296875 0 \nL 45.3125 0 \nL 45.3125 8.40625 \nQ 42.046875 3.421875 37.71875 1 \nQ 33.40625 -1.421875 27.6875 -1.421875 \nQ 18.265625 -1.421875 13.375 4.4375 \nQ 8.5 10.296875 8.5 21.578125 \nz\nM 31.109375 56 \nz\n\" id=\"DejaVuSans-117\"/>\n       <path d=\"M 41.109375 46.296875 \nQ 39.59375 47.171875 37.8125 47.578125 \nQ 36.03125 48 33.890625 48 \nQ 26.265625 48 22.1875 43.046875 \nQ 18.109375 38.09375 18.109375 28.8125 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 20.953125 51.171875 25.484375 53.578125 \nQ 30.03125 56 36.53125 56 \nQ 37.453125 56 38.578125 55.875 \nQ 39.703125 55.765625 41.0625 55.515625 \nz\n\" id=\"DejaVuSans-114\"/>\n       <path d=\"M 32.171875 -5.078125 \nQ 28.375 -14.84375 24.75 -17.8125 \nQ 21.140625 -20.796875 15.09375 -20.796875 \nL 7.90625 -20.796875 \nL 7.90625 -13.28125 \nL 13.1875 -13.28125 \nQ 16.890625 -13.28125 18.9375 -11.515625 \nQ 21 -9.765625 23.484375 -3.21875 \nL 25.09375 0.875 \nL 2.984375 54.6875 \nL 12.5 54.6875 \nL 29.59375 11.921875 \nL 46.6875 54.6875 \nL 56.203125 54.6875 \nz\n\" id=\"DejaVuSans-121\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-97\"/>\n      <use x=\"61.279297\" xlink:href=\"#DejaVuSans-99\"/>\n      <use x=\"116.259766\" xlink:href=\"#DejaVuSans-99\"/>\n      <use x=\"171.240234\" xlink:href=\"#DejaVuSans-117\"/>\n      <use x=\"234.619141\" xlink:href=\"#DejaVuSans-114\"/>\n      <use x=\"275.732422\" xlink:href=\"#DejaVuSans-97\"/>\n      <use x=\"337.011719\" xlink:href=\"#DejaVuSans-99\"/>\n      <use x=\"391.992188\" xlink:href=\"#DejaVuSans-121\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_15\">\n    <path clip-path=\"url(#p326614cd88)\" d=\"M 58.999432 214.756364 \nL 60.528897 209.957373 \nL 62.058363 209.957373 \nL 63.587828 211.328513 \nL 65.117294 209.957373 \nL 66.646759 209.72885 \nL 68.176225 209.843111 \nL 69.70569 209.157541 \nL 71.235156 206.186737 \nL 72.764621 200.816438 \nL 74.294087 198.531203 \nL 75.823552 194.87483 \nL 77.353018 193.275166 \nL 78.882483 189.276006 \nL 80.411949 188.133391 \nL 81.941414 184.819801 \nL 85.000345 180.135071 \nL 86.529811 179.106717 \nL 88.059276 177.278529 \nL 89.588742 172.593799 \nL 91.118208 169.394474 \nL 92.647673 165.623836 \nL 94.177139 163.567125 \nL 95.706604 162.195988 \nL 97.23607 155.911595 \nL 98.765535 153.169315 \nL 100.295001 147.913275 \nL 101.824466 146.199348 \nL 103.353932 143.571332 \nL 104.883397 142.542972 \nL 106.412863 138.658079 \nL 109.471794 132.259422 \nL 111.001259 127.231905 \nL 112.530725 127.917479 \nL 114.06019 125.632242 \nL 117.119121 121.633086 \nL 118.648587 117.291142 \nL 120.178052 114.091817 \nL 121.707518 111.578058 \nL 123.236983 110.892485 \nL 124.766449 109.0643 \nL 126.295914 108.035946 \nL 127.82538 105.636446 \nL 129.354845 103.465474 \nL 130.884311 103.008429 \nL 132.413776 101.065987 \nL 133.943242 97.866648 \nL 135.472707 98.323706 \nL 137.002173 95.809941 \nL 138.531638 93.638969 \nL 140.061104 92.839144 \nL 141.590569 93.06766 \nL 143.120035 88.154407 \nL 144.6495 86.44048 \nL 146.178966 86.669009 \nL 147.708431 82.669846 \nL 149.237897 83.12689 \nL 150.767362 78.556431 \nL 152.296828 78.556431 \nL 153.826293 75.357106 \nL 155.355759 73.75743 \nL 156.885224 72.386296 \nL 158.41469 72.614825 \nL 159.944155 70.786634 \nL 161.473621 68.615662 \nL 163.003086 67.701573 \nL 164.532552 66.330426 \nL 166.062017 65.530601 \nL 167.591483 63.016836 \nL 169.120948 63.016836 \nL 170.650414 62.102747 \nL 172.17988 60.160291 \nL 173.709345 59.588982 \nL 175.238811 59.703246 \nL 176.768276 58.446363 \nL 178.297742 56.846701 \nL 179.827207 55.932612 \nL 181.356673 58.446363 \nL 182.886138 60.7316 \nL 184.415604 55.818347 \nL 185.945069 55.361303 \nL 189.004 53.647375 \nL 190.533466 51.133624 \nL 192.062931 54.10442 \nL 193.592397 63.816674 \nL 195.121862 58.903422 \nL 196.651328 53.076066 \nL 198.180793 53.190331 \nL 199.710259 49.762477 \nL 201.239724 48.962652 \nL 202.76919 45.649062 \nL 204.298655 46.220371 \nL 205.828121 44.0494 \nL 207.357586 43.249561 \nL 208.887052 42.221208 \nL 210.416517 42.335472 \nL 211.945983 41.878428 \nL 213.475448 42.106943 \nL 215.004914 40.850074 \nL 216.534379 40.507281 \nL 218.063845 43.021046 \nL 219.59331 42.564001 \nL 221.122776 40.73581 \nL 222.652241 40.050236 \nL 224.181707 39.593192 \nL 225.711172 41.878428 \nL 227.240638 41.421383 \nL 228.770103 39.364663 \nL 230.299569 38.564838 \nL 231.829034 36.736646 \nL 233.3585 35.479764 \nL 234.887965 35.822557 \nL 236.417431 35.594028 \nL 237.946896 34.794204 \nL 239.476362 35.822557 \nL 241.005827 36.051073 \nL 242.535293 36.051073 \nL 244.064758 34.679939 \nL 245.594224 33.76585 \nL 247.123689 35.251248 \nL 248.653155 33.76585 \nL 250.18262 36.965175 \nL 251.712086 38.450573 \nL 253.241552 33.651586 \nL 254.771017 31.13782 \nL 259.359414 29.538158 \nL 260.888879 29.309642 \nL 262.418345 29.423907 \nL 263.94781 29.309642 \nL 265.477276 28.167024 \nL 267.006741 27.938495 \nL 268.536207 30.223731 \nL 270.065672 28.738333 \nL 271.595138 28.395539 \nL 273.124603 27.252935 \nL 274.654069 28.281288 \nL 276.183534 26.681626 \nL 277.713 28.624069 \nL 279.242465 29.195378 \nL 280.771931 28.738333 \nL 282.301396 26.338832 \nL 283.830862 25.653259 \nL 285.360327 24.510654 \nL 286.889793 25.996052 \nL 288.419258 25.196214 \nL 289.948724 23.939345 \nL 291.478189 26.681626 \nL 293.007655 26.338832 \nL 294.53712 23.482287 \nL 296.066586 22.910978 \nL 297.596051 24.053596 \nL 299.125517 23.368036 \nL 300.654982 23.139507 \nL 302.184448 23.139507 \nL 303.713913 22.910978 \nL 305.243379 22.910978 \nL 306.772844 21.996889 \nL 308.30231 21.882624 \nL 309.831775 21.42558 \nL 311.361241 20.511491 \nL 312.890706 20.740006 \nL 314.420172 19.711653 \nL 315.949637 19.711653 \nL 317.479103 19.825917 \nL 319.008568 20.397226 \nL 320.538034 20.397226 \nL 322.067499 21.539844 \nL 323.596965 21.197064 \nL 325.12643 19.254608 \nL 326.655896 19.825917 \nL 329.714827 18.11199 \nL 331.244292 18.454783 \nL 332.773758 19.140344 \nL 334.303224 17.654945 \nL 335.832689 18.911828 \nL 337.362155 17.083636 \nL 338.89162 17.76921 \nL 340.421086 17.426416 \nL 341.950551 19.140344 \nL 343.480017 19.368873 \nL 345.009482 19.026093 \nL 346.538948 17.426416 \nL 348.068413 17.197901 \nL 349.597879 17.883474 \nL 351.127344 19.026093 \nL 352.65681 17.083636 \nL 354.186275 17.312165 \nL 355.715741 17.083636 \nL 357.245206 18.226254 \nL 358.774672 19.597388 \nL 360.304137 18.797564 \nL 361.833603 18.454783 \nL 363.363068 17.197901 \nL 363.363068 17.197901 \n\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n   </g>\n   <g id=\"patch_3\">\n    <path d=\"M 43.78125 224.64 \nL 43.78125 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 378.58125 224.64 \nL 378.58125 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 43.78125 224.64 \nL 378.58125 224.64 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 43.78125 7.2 \nL 378.58125 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"p326614cd88\">\n   <rect height=\"217.44\" width=\"334.8\" x=\"43.78125\" y=\"7.2\"/>\n  </clipPath>\n </defs>\n</svg>\n",
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAptUlEQVR4nO3deXiU1d3/8fc3IRsECFkRCFvYERGMiEstiAuu1NYNrbVW5VEfbW1rW5/a2j62T1u7/mpr3RW1VtC6UXeliAuCBGTfCYEQAgkBEhKyzpzfHzPQgAkMmJl7kvm8risXM2fuDJ+5Z3J/517OOeacQ0REYlec1wFERMRbKgQiIjFOhUBEJMapEIiIxDgVAhGRGNfJ6wBHKzMz0/Xv39/rGCIi7cqiRYt2OueyWnqs3RWC/v37U1BQ4HUMEZF2xcw2t/aYDg2JiMQ4FQIRkRinQiAiEuNUCEREYpwKgYhIjFMhEBGJcSoEIiIxrt31IxARiRY+v2PGwi2M6t2dUb27s6umgfQuiZjZQcvtrmlgeUklNfVNTBiaTUpiPAALCitYua2Knt2T2banlsE5XfnykM/3+XLOMX1eEZeO6U1a58Q2fx0qBCIirXDOMXddOTuq6jipXw8GZXdl0ebd/OjFZVx3aj+27q7l4Q8KAeiW3Imquia+OqY3v798NHFxgWKwoqSSqx+dT1VdEwA9Oifw7UmDOb53d655bAFN/oPnhLl2fD9+ctFwkjoFikV1fRM/+ucyXl9eSpPPcdOZA9v8daoQiIgENfr8LCjcxdh+aRSW1/DL11cxv3DXgcdH56axYcdemvyOn766EoCp4/qSl9WF9TuqAZhZUExCfByTR/Vk6+5a/vTuOromJ/Dg108C4KG5G/nff62iU5zRp0cKT31rHHvrmsjulsSjHxTy6Ieb2FhezdRxfXny400s21qJzzn+5/xh3PilAWF53SoEItIh1Tf5SIiL47Pi3dz72mp27q2nU7yR1jmRUwakc+6IHFIS4xmUnXrg2/dPX1nBjIXFdE6Mp7bRR4/OifxiykhOzcvkw/XlTJ9XRO/gxnvGp8Vs21PLvVNGkhAfON3qnCMlMZ7p84qYWVAMQP+Mzky/fhz9M7sAcFpeBk98XMTMhVu4f+oY+mV0OZD57gtHMKxnN3704jLmbaxgYGYXpp05kLNH5DC2b4+wrStrb1NV5ufnO401JCKtqa5v4qH3N/LIB4XExxl1TT56p6UwfmAGjT4/ZVX1LCzadeCQTF5WF3516SgKNu/md2+v5aqTczEzMrokMu3LA+mWnHDguZ1zOMeBwz6tKdtbx+aKfeR0TSY3PeVz5wyO5NNNuyitrOXCUcfRKb5trukxs0XOufyWHtMegYi0a36/4+XPSvh44062V9ZRsHk3DU1+Lh7di+yuSXROjOfmL+fRJek/m7uK6no+27KH3fsauO+ttVz5yHwAJgzN4v8uHUV8Kxt6MyOUbXp212SyuyYf82saNyD9mH/3WKgQiEjUqaxtZNbSbRSWV3Pzl/PI6JLI4i172FheTXVdEw0+P5sraqisbaR4Vy2rSqvI6ZZETrdkrh3fj4tH9+LE3LRWnz8jNYmzR+QAMGFoNnPWljE4O5UT+qS1WgQ6MhUCEYmoRp//wDF1gI837GRJ8R5unZCHmVFT38QFf/6Qkj21xBnMWrKNlMR4tu6uPeh5MlOTyOiSSFJCHH+4fDRfHdv7qA/BAGR1TeKK/Nwv/LraMxUCEYkI5xzPLtjCL19fxbXj+/HDycN44qNN3PfWGvwOBmZ24fxRx/HER5so2VPLk988mdz0FH780grM4K7zhzG6Txo9uiQSb3bgWnz54lQIRCTsPlxfzt/mbOSTwgr6pnfm0Q838eLiEnbVNDB5ZE8Kd1bzm7fWMLRnVx75oJBzRuQwcVg2AM/ffKrH6Ts+DTEhIsesrtHHvI078fsdH63fycV/+YgVJZUHLTN79Q6uffxTCndW87OLR/D+nRO4beIgctM78/C1J/Hg18fy4wuGs7liH2f9YS61jT5+eN5Qj15RbNLloyLSKuccb6/cweItuwH44XlD6RQfh3OOtTv28t2ZS1ldWsWJuWms2V5FXaOf3mkpvHb7GfToksjeukbO/dMHdEtOYNbtpx+4Xr8lj3+0CeccZw7JYkhO10i9xJihy0dF5Ij2NTRRWF5DVW0j3VISqGv08eS8Il5fVkpifBwNPj9pnRMYkNGFn81aSdneetI6J3D7WYOYPq+IXmkp/PTCEfzX3xdx6d8+5oJRxzFvYwXbq+r42zVjD1sEAG44Izy9ZuXItEcgEsNq6pt4b/UOZi3Zxgfry2n0Hbw9iI8z7jx3KNPOHMjtzy3m3VU78Pkdo3p357L8XM4bkUN2t2QqaxtJiDc6J3bi/bVl/OXfG1i0eTcDM7vwrTMG8PXx/Tx6hbKf9ghE5CCrtlXx4NyNvLdqB7WNPnp2S+a6U/sztl8PenROpLK2kZTEeAZmdiE3vTMAv5hyPIs27yYvK5XHrsunc+J/Nh/dU/7T+3bC0GwmDM2mpr7poE5cEr30LonEmOr6Jr41fSG1jT6+OrY3l4zuxcn90484bEJGahJzfzCRpE5xIV2vryLQfuidEokxf35vHdur6njp1tOOeiCz5ARdu98R6fJRkQ6ksLyaW/6+iPtnr6ehyc89r67giY82HXh8c0UNT3xcxNRxuWEdzVLaF+0RiES5ukYfifFxxMUZVXWNPPbhJjZX1HDH2UMYkPmfIYz/sWALP5u1AoA3V2znn4u2smXXPpI6xXHx6F5kdU3itWWl+PyO288a7NXLkSgU1kJgZpOBPwPxwGPOud8c8nhf4CkgLbjMXc65N8KZSaQ9mblwCz99ZSVJCXHkdEtmy659NDT56ZwYz9srtzNxaDa901KoqGng5c9KmDA0i99edgIPvr+R6fOKuGVCHg/P3cgTH2/iR5OH8fbK7ZyYm0avtBSvX5pEkbAVAjOLBx4AzgG2AgvNbJZzblWzxX4CPO+ce9DMRgBvAP3DlUmkPfnju+u4f/Z6Th+UQd/0LuysruesYdlcEvx2f9+ba1hSvIf315YDcM0pffnfS0bSKT6On108kjsmDaF75wSKd+3jmU82M3lkT5ZtreRHk4d5/Mok2oRzj2AcsME5VwhgZjOAKUDzQuCAbsHb3YFtYcwj0m68uGgr989ez+Un9eHXXx3V4uQkf7zyxMM+R/fOgUs6vz1pMLNXl3H5w58AcN7InDbPK+1bOE8W9waKm93fGmxr7ufA181sK4G9gdtbeiIzm2ZmBWZWUF5eHo6sIlFhd00Dv3htFXe9tIzT8jL4VStF4GgMyenK9OtPJiHOGNazKwOzUtsorXQUXp8sngpMd879wcxOBZ4xs+Odc/7mCznnHgEegUDPYg9yihyT+qbAid7m193va2jipcUlvL6slDXbq3j8mycztm8P6hp9TH10Put27OWrY/vw0wtHHDRu/xdxysAM3rrjzJBm15LYE85CUAI0n+2hT7CtuRuAyQDOuU/MLBnIBMrCmEsk7N5ZuZ0/z17P6tIqrjmlH7/4yvEAfLKxglueXcSefY0Mzk6lU3wcd76wlNdv/xL3vraSNdv38uT1JzNxaHabZ9rfQ1jkUOEsBAuBwWY2gEABuAq4+pBltgCTgOlmNhxIBnTsR9qN0spaCop2k5IQz6ItuynaWUNKQjwvfVbC0JyunJaXyTPzN3Pp2N706JzIzX9fRGZqIo99I5+T+vXg4w0VfP3xBeT/8l1qGnzcMiEvLEVA5HDCVgicc01mdhvwNoFLQ59wzq00s3uBAufcLOD7wKNm9l0CJ46/6drbKHgSk3x+x4yFW/jV66upafAB0CnO6NMjhe1VdVx2Uh9++ZXj8fkdk/4wl5ufWURto49OccaT3xxH34zAt/MzBmdy57lDWFVaxdnDc5hy4qGn0UTCT6OPihylRZt3cdeLy1lfVs1peRn8cPIwfH7HoOxUuqck4Jw76JzAv9fs4OezVjFuQDrfOn0AI3p1O8yzi4SHRh8VaSNle+u46elFdE6M569Xj+GC44/73GBthw7IdtawHM4apks2JXqpEIiE4DdvrmHuunIS4o2a+iZmThvPYM2iJR2ECoHIEZRW1vL4R4V0T0lgZ3UD904ZqSIgHYoKgcgRPPlxEX4HL996Ot1SEg6ahEWkI1AhEDmMFSWV/GPBFi464Thdhy8dlgqBSCv+8M5a/vLvDaR1TuC2iYO8jiMSNpqYRmLSJxsr2F5Z1+rjO6rqeGjuRi4Y1ZO5P5iocwLSoakQSMxZUVLJNY/N59ZnF9FaP5qn5hXh8zvumjxc5wSkw1MhkJji9zvufmUFZsbiLf8Zy3+/D9eX88CcDTy7YAvnjex5oAewSEemQiAxo6yqjjtfWMrS4j38+tJR9E3vzO/fWXtgr8A5x/efX8rv3l7L3rpGpp050OPEIpGhk8XS4VXWNvL0vCIenLuRRp+fWybkcXl+H+qbfPz01ZUU7qwhLyuVDWXVlO2t5xdTRjJlTG+6JeuQkMQGFQLpEHx+x8yFxdQ3+bj+9AEH2r494zPeWrEdn99x/vE9uev8YfTLCEz4PqpPGgAby6rJy0rlow07AZgwNFtFQGKKCoG0e9sr67jx6YWsKKkCYFjPbpyal0FB0S5eX1bKFfl9uHZ8f0b16X7Q7w3MChSEjeU1AHy8oYK+6Z3VX0Bijs4RSLu2s7qeax6bT9HOffzpytHkpqdwz6sraGjy886qHSTGx3HPxSM/VwQAuiUnkN01iQ1l1TT5/CworOD0QRkevAoRb2mPQNqtwvJqbny6gG17annq+nGcMjCDbskJ3PBUAQ/P3ci7q3Zw+qAMUpNa/5gPyk5lY3k1y0sq2VvfxGl5mRF8BSLRQYVA2o26Rh+PflDIPxdvpbSyDhykJnfi6W+dwrgB6QBMGp7DRSccx5/eW4ffwS0T8g77nHlZqbyypIR/rykjzuD0QSoEEntUCKRdcM7x45eX89LiEk4flMHkkT1p9DmuP73/547p3zvleOYXVlBR08Ck4Yef9jEvqwt765p4vqCY/P7ppHdJDOfLEIlKKgQS9ZxzPPphIS8tLuE7kwbz3XOGHHb59C6J/O2ak1heUkl21+TDLjsoOzB0xI6qem76kvoNSGxSIZCo9dqybSzctItVpVUsLNrN2cNz+M6kwSH97rgB6QcOFx1OXnaXA7fPHdHzmLOKtGcqBBKVlm3dw+3PfUbnhHjSOifyq0tHceXJuZ+bFvKL6tktmS6J8eSmd9ZwEhKzVAgk6vj8jp+8soLM1CRmf//LYe3cZWbccfYQ+qkISAxTIZCosnJbJb95cw3Ltlby56tOjEgP35s0ppDEOBUCiRr/WrqN7z+/lJTEeH5y4XAuGd3L60giMUGFQDxXXd/E/bPX88gHhYzrn87D155ED13GKRIxKgTiqT37Grjkrx+zZdc+po7L5WcXjyQ5Id7rWCIxRYVAPLO/k9i2PbU8d9N4Ts3TOD8iXtCgc+KZFwq28sby7Xzv3CEqAiIe0h6BRFSjz0/xrn1sr6zj7leWc8agTP7rzMOPByQi4aVCIBFTtreO659cyMptgXkDBmR24YGrxxLfxp3EROToqBBIRJTsqWXqI/Mp31vPTy4cTpPfcfHoXnTvrJnARLymQiBhV1ZVxzWPzmf3vgaemzaeE3PTvI4kIs3oZLGE3f+8tJyyvfVMv36cioBIFFIhkLBaua2S2WvKuHVCHif16+F1HBFpgQ4NSVg456hv8vPAnA10TerEtaf29zqSiLRChUDC4scvr+C5T7cAcOuEPLqn6KSwSLRSIZA2V9vg45XPSjh1YAaThmdz1bi+XkcSkcMI6zkCM5tsZmvNbIOZ3dXKMleY2SozW2lm/whnHomMOWvLqG30cfukQdz4pYGkJun7hkg0C9tfqJnFAw8A5wBbgYVmNss5t6rZMoOB/wFOd87tNrPDzzQu7cLry0rJTE3klAEaNkKkPQjnV7VxwAbnXCGAmc0ApgCrmi1zE/CAc243gHOuLIx5JMzu/dcqPi2qYP2Oaq7Iz1WPYZF2IpyHhnoDxc3ubw22NTcEGGJmH5vZfDOb3NITmdk0Mysws4Ly8vIwxZUvoqK6nqc/KaJ8bz1mcNlJfbyOJCIh8vrgbSdgMDAB6AN8YGajnHN7mi/knHsEeAQgPz/fRTijhODlz0po8jueueEUBmenYqa9AZH2Ipx7BCVAbrP7fYJtzW0FZjnnGp1zm4B1BAqDtBMNTX521zTwQsFWTsxNY0hOVxUBkXYmnHsEC4HBZjaAQAG4Crj6kGVeAaYCT5pZJoFDRYVhzCRtaEdVHZc9NI/iXbUA/OrSUR4nEpFjEbZC4JxrMrPbgLeBeOAJ59xKM7sXKHDOzQo+dq6ZrQJ8wA+ccxXhyiRtp7q+iW8+uZCK6gZ+cN5Q6hp9XDrm0FNAItIehPUcgXPuDeCNQ9ruaXbbAd8L/kg78kJBMatLq5h+/clMGKqrfkXaMw06J8fknZU7GJKTqiIg0gGoEMhR213TwKdFuzhnRI7XUUSkDagQyFH795oyfH7HuSN6eh1FRNqA1/0IpJ1o9Pl5vqCY2avLKNpZQ89uyYzq3d3rWCLSBlQI5Ij2NTRx2YOfsKq0in4Zndlb18R1p/YnTkNIiHQIKgRyRP/3+mpWb6/iL1PHcNEJx6nDmEgHo0IghzVnTRnPLtjCtDMHcvHoXl7HEZEwCOlksZm9ZGYXmplOLseQ2gYfP311BYOzU/n+uUO8jiMiYRLqHsHfgOuB+83sBeBJ59za8MUSL80vrOC5T7fQ6POzdXctM6aNJ6lTvNexRCRMQioEzrn3gPfMrDuBsYHeM7Ni4FHg7865xjBmlAjy+x0/e3Ul68r24hx8dUxvxg/UBDMiHVnI5wjMLAP4OnAt8BnwLHAGcB2BYaSlA/j3mjLW7tjLHy4fzejcNHLTU7yOJCJhFlIhMLOXgaHAM8DFzrnS4EMzzawgXOEkspxz/HXOBvr0SGHKib3oFK9TQiKxINQ9gvudc3NaesA5l9+GecRDD80tZEnxHu772igVAZEYEupf+wgzS9t/x8x6mNmt4YkkkVZaWcujHxTyu7fXcNEJx3FFfu6Rf0lEOoxQ9whucs49sP+Oc263md1E4GoiaceWbd3DZQ99QkOTn5P69eC+r52gDmMiMSbUQhBvZhacPwAziwcSwxdLImFfQxN3zFhCRpdEnrlhHIOyu3odSUQ8EGoheIvAieGHg/f/K9gm7dhv3lzDpooanr3xFBUBkRgWaiH4EYGN/y3B++8Cj4UlkUREQdEunv5kM9ef3p/T8jK9jiMiHgq1Q5kfeDD4I+3c7poG7nppOb3TUrjz3KFexxERj4Xaj2Aw8GtgBJC8v905NzBMuSRMPlxfzndnLqWytoHHrzuZLkkad1Ak1oV6+eiTBPYGmoCJwNPA38MVSsJjRUklNz1dQHqXBF797zM4c0iW15FEJAqEWghSnHOzAXPObXbO/Ry4MHyxpC35/I63V27nxqcKSO+cyLM3jmdEr25exxKRKBHqcYH64BDU683sNqAESA1fLGkrzjmmPV3A7DVl9OmRwiPX5pPVNcnrWCISRUItBN8BOgPfBn5B4PDQdeEKJW3nX8tKmb2mjO+dM4RbJ+Rp6AgR+ZwjFoJg57ErnXN3AtUE5iWQdmBvXSO/fG0Vo3p3578nDiJecwyLSAuO+PXQOecjMNy0tDNPzSuibG89904ZqSIgIq0K9dDQZ2Y2C3gBqNnf6Jx7KSyp5Avb19DEEx8XMXFoFmP69vA6johEsVALQTJQAZzVrM0BKgRR6rlPi9lV08BtZw3yOoqIRLlQexbrvEA74pxj+rxNjOufzkn90r2OIyJRLtSexU8S2AM4iHPuW22eSL6wxVv2ULyrljsmDfE6ioi0A6EeGnqt2e1k4FJgW9vHkbYwa0kJSZ3iOHdkjtdRRKQdCPXQ0IvN75vZc8BHYUkkX0iTz89ry0o5e3gOXZMTvI4jIu3AsfYuGgxkt2UQaRuvLy+loqaBS07s5XUUEWknQj1HsJeDzxFsJzBHgUSRzRU1/OTlFZzQpztnDVOdFpHQhHpoSNNXRbm6Rh+3/H0xcXHGA1ePJUFDSYhIiELaWpjZpWbWvdn9NDP7SthSyVH7+ayVrCqt4k9XjiY3vbPXcUSkHQn1a+PPnHOV++845/YAPzvSL5nZZDNba2YbzOyuwyz3NTNzZpYfYh5p5tUlJcxYWMytE/I4a5iuFBKRoxNqIWhpucMeVgoOVvcAcD6Bmc2mmtmIFpbrSmB00wUhZhECQ0h8vGEn2/bUcs+rKxnTN43vnaN+AyJy9ELtR1BgZn8ksGEH+G9g0RF+ZxywwTlXCGBmM4ApwKpDlvsFcB/wgxCzCPC3ORv565wNJMQbcWb8/vLRGmJaRI5JqFuO24EGYCYwA6gjUAwOpzdQ3Oz+1mDbAWY2Fsh1zr0eYg4JemfVdobkpHL6oEx+8ZXjycvSPEEicmxCvWqoBmj1GP+xCM549kfgmyEsOw2YBtC3b9+2jNEubanYx7od1fz0ohHccMYAr+OISDsX6lVD75pZWrP7Pczs7SP8WgmQ2+x+n2Dbfl2B44H3zawIGA/MaumEsXPuEedcvnMuPytLE66/t3oHAGcPV18BEfniQj00lBm8UggA59xujtyzeCEw2MwGmFkicBUwq9lzVDrnMp1z/Z1z/YH5wCXOuYKjeQGx6L3VOxiSk0q/jC5eRxGRDiDUQuA3swPHZMysPy2MRtqcc64JuA14G1gNPO+cW2lm95rZJceYN+YV79rHgk27OGeELhMVkbYR6lVDdwMfmdlcwIAvETxmfzjOuTeANw5pu6eVZSeEmCWmPfJBIXEG147v73UUEekgQj1Z/Fbw2P004DPgFaA2jLmkBWVVdcwsKOayk/rQs3uy13FEpIMIddC5Gwl0+uoDLCFwYvcTDp66UsLs8Y820eTzc/OX87yOIiIdSKjnCL4DnAxsds5NBMYAe8IVSj5vz74G/j5/MxeP7qWTxCLSpkItBHXOuToAM0tyzq0BhoYvlhxq+rwiahp83DpBk9GLSNsK9WTx1mA/gleAd81sN7A5XKHkYLUNPqbPK+KcETkM7akRwUWkbYV6svjS4M2fm9kcoDvwVthSyUFeX17Knn2N6kUsImER6h7BAc65ueEIIq2buXALAzO7cMqAdK+jiEgHpOEqo9yGsr0sLNrNlSfnYmZexxGRDkiFIIoV79rHXS8up1Oc8bWT+ngdR0Q6qKM+NCSRUbxrHxf8+UMc8PvLR5OZmuR1JBHpoFQIotSv3lhNo9/Pm985kwGZ6jcgIuGjQ0NRaN7Gnby5Yju3ThikIiAiYadCEGW27t7HHTOWkJuewrQzB3odR0RigApBFKlt8HHdE59S1+jjsW+cTHJCvNeRRCQG6BxBFJm7royN5TU8+o189SAWkYjRHkEUmbOmnK7JnZgwVNNxikjkqBBECeccc9aWcebgLBLi9baISORoixMlVm6romxvPROHaUJ6EYksFYIoMWdNGQBfHqLDQiISWSoEUaCu0cfMgmLG9k0jq6t6EItIZKkQRIGn5hWxdXct3ztHc/2ISOSpEHhsz74G/jpnAxOHZnHG4Eyv44hIDFIh8NjcdeXsrWvi9kmDvY4iIjFKhcBjCzbtIjWpE6P7pHkdRURilAqBxz7dtIv8/j2Ij9OkMyLiDRUCD+2srmdDWTWnDMjwOoqIxDAVAg8VFO0CYJzmIhYRD6kQeGh+4S6SE+IY1bu711FEJIapEHjkrRWl/OPTLZyel0liJ70NIuIdbYE88N6qHdzy7GKO79WN318+2us4IhLjNB9BhJXsqeX7LyxlxHHdePbG8aQkavIZEfGW9ggi7K4Xl+HzOx64eqyKgIhEBRWCCFq/Yy8frt/JrRPz6K9J6UUkSqgQRNCzC7aQEG9cmZ/rdRQRkQNUCCKktsHHi4u3cv7xx5GRqqGmRSR6qBBEyMuflbC3rolrTunrdRQRkYOoEERAk8/PQ3M3MrpPd/UiFpGoE9ZCYGaTzWytmW0ws7taePx7ZrbKzJaZ2Wwz6xfOPF55fXkpW3bt49aJgzDT4HIiEl3CVgjMLB54ADgfGAFMNbMRhyz2GZDvnDsB+Cfw23Dl8dLDcwsZkpPKOcNzvI4iIvI54dwjGAdscM4VOucagBnAlOYLOOfmOOf2Be/OB/qEMY8nNpRVs6q0iqnj+hKnoaZFJAqFsxD0Boqb3d8abGvNDcCbLT1gZtPMrMDMCsrLy9swYvi9taIUgMnH9/Q4iYhIy6LiZLGZfR3IB37X0uPOuUecc/nOufysrKzIhvuC3li+nbF90ziue4rXUUREWhTOQlACNO851SfYdhAzOxu4G7jEOVcfxjwRt7mihlWlVZx//HFeRxERaVU4C8FCYLCZDTCzROAqYFbzBcxsDPAwgSJQFsYsEef3O379xhriTIeFRCS6ha0QOOeagNuAt4HVwPPOuZVmdq+ZXRJc7HdAKvCCmS0xs1mtPF274vc77ntrDW+t3M6PLxhObnpnryOJiLQqrMNQO+feAN44pO2eZrfPDuf/74Wyqjq+PeMz5hfu4upT+nLDGQO8jiQicliaj6ANOef47vNLWFpcyX1fG8UV+bnqQCYiUS8qrhrqKGYuLObjDRX85KLhXHlyXxUBEWkXVAjayPbKOv7v9dWMH5jO1JM1sJyItB8qBG3AOcfdLy+n0e/nvq+doB7EItKuqBC0gdeXlzJ7TRl3njuUfhmaeUxE2hcVgi/I73fcP3s9Q3O6cv3pukJIRNofFYIv6P11ZazbUc3NEwYSr0NCItIOqRB8AT6/48H3N9KrezIXndDL6zgiIsdEheAY7ayu59rHF7CwaDe3TBxEQrxWpYi0T+pQdozu/dcqFm3ezW+/dgKX53e4aRREJIaoEByDPfsaeGvldq4e15crTs498i+IiEQxHc84BrOWbqOhya89ARHpEFQIjsHzBcWM7NWNkb26ex1FROQLUyE4Sg/P3ciKkiqu0iEhEekgVAiOwj8XbeXXb67h4tG9uPqUfl7HERFpEyoEIWry+fnjO2sZ2zeNP14xWp3HRKTDUCEI0ew1ZWyrrGPamXnqMyAiHYq2aCF65pPN9OqezNnDs72OIiLSplQIQrC0eA8fbdjJ1af0pZP2BkSkg9FW7QjqGn3c+cJSenZL5tpT+3sdR0Skzaln8RH87f2NrC+rZvr1J9M9JcHrOCIibU57BIfhnOPFRVuZODSLCUN1bkBEOiYVgsPYWF5NyZ5azh6R43UUEZGwUSE4jPfXlgNob0BEOjQVgsN4f205g7NT6Z2W4nUUEZGwUSFoxb6GJj7dtIsvD8nyOoqISFipELTi1SXbaPD5mTRc5wdEpGNTIWhBXaOP+2evZ0zfNMYPTPc6johIWKkQHMI5x2MfFlJaWced5w7FTIPLiUjHFnMdyh77sJBV26rISE3k25MGU7a3nvtnr2dDWTVNPkddk4/NFfs4c0gWp+VleB1XRCTsYqoQlFXV8X9vrCa9cyJ7ahv5YN1OyvbW4fM7xvTtQVKnOBp9fr4zaTAXndBLewMiEhNiqhC8s2oHzsFz08azbU8ttz67mOyuSTz1rXH0y+jidTwREU/EVCF4e+V2BmR2YXB2KkNyujLnzgmkJnWiS1JMrQYRkYPEzMniyn2NfLKxgvNG9jxwyCenW7KKgIjEvJgpBLPX7KDJ7zhvpPoFiIg0FzOFoGtyAueMyGF0nzSvo4iIRJWwFgIzm2xma81sg5nd1cLjSWY2M/j4AjPrH64s54zI4dFv5BOnSedFRA4StkJgZvHAA8D5wAhgqpmNOGSxG4DdzrlBwJ+A+8KVR0REWhbOPYJxwAbnXKFzrgGYAUw5ZJkpwFPB2/8EJpku3hcRiahwFoLeQHGz+1uDbS0u45xrAiqBz3XnNbNpZlZgZgXl5eVhiisiEpvaxcli59wjzrl851x+VpaGhRYRaUvhLAQlQG6z+32CbS0uY2adgO5ARRgziYjIIcJZCBYCg81sgJklAlcBsw5ZZhZwXfD2ZcC/nXMujJlEROQQYetW65xrMrPbgLeBeOAJ59xKM7sXKHDOzQIeB54xsw3ALgLFQkREIiis4ys4594A3jik7Z5mt+uAy8OZQUREDs/a25EYMysHNh/jr2cCO9swTluK1mzKdXSU6+hFa7aOlqufc67Fq23aXSH4IsyswDmX73WOlkRrNuU6Osp19KI1WyzlaheXj4qISPioEIiIxLhYKwSPeB3gMKI1m3IdHeU6etGaLWZyxdQ5AhER+bxY2yMQEZFDqBCIiMS4mCkER5okJ4I5cs1sjpmtMrOVZvadYPvPzazEzJYEfy7wIFuRmS0P/v8FwbZ0M3vXzNYH/+0R4UxDm62TJWZWZWZ3eLW+zOwJMyszsxXN2lpcRxZwf/Azt8zMxkY41+/MbE3w/37ZzNKC7f3NrLbZunsowrlafe/M7H+C62utmZ0XrlyHyTazWa4iM1sSbI/IOjvM9iG8nzHnXIf/ITDExUZgIJAILAVGeJTlOGBs8HZXYB2BiXt+Dtzp8XoqAjIPafstcFfw9l3AfR6/j9uBfl6tL+BMYCyw4kjrCLgAeBMwYDywIMK5zgU6BW/f1yxX/+bLebC+Wnzvgn8HS4EkYEDwbzY+ktkOefwPwD2RXGeH2T6E9TMWK3sEoUySExHOuVLn3OLg7b3Aaj4/T0M0aT550FPAV7yLwiRgo3PuWHuWf2HOuQ8IjIvVXGvraArwtAuYD6SZ2XGRyuWce8cF5vkAmE9gBOCIamV9tWYKMMM5V++c2wRsIPC3G/FsZmbAFcBz4fr/W8nU2vYhrJ+xWCkEoUySE3EWmKN5DLAg2HRbcPfuiUgfgglywDtmtsjMpgXbcpxzpcHb24EcD3LtdxUH/2F6vb72a20dRdPn7lsEvjnuN8DMPjOzuWb2JQ/ytPTeRdP6+hKwwzm3vllbRNfZIduHsH7GYqUQRB0zSwVeBO5wzlUBDwJ5wIlAKYHd0kg7wzk3lsA80/9tZmc2f9AF9kU9ud7YAkOZXwK8EGyKhvX1OV6uo9aY2d1AE/BssKkU6OucGwN8D/iHmXWLYKSofO8OMZWDv3REdJ21sH04IByfsVgpBKFMkhMxZpZA4E1+1jn3EoBzbodzzuec8wOPEsZd4tY450qC/5YBLwcz7Ni/qxn8tyzSuYLOBxY753YEM3q+vpppbR15/rkzs28CFwHXBDcgBA+9VARvLyJwLH5IpDId5r3zfH3BgUmyvgrM3N8WyXXW0vaBMH/GYqUQhDJJTkQEjz0+Dqx2zv2xWXvz43qXAisO/d0w5+piZl333yZwonEFB08edB3waiRzNXPQNzSv19chWltHs4BvBK/sGA9UNtu9Dzszmwz8ELjEObevWXuWmcUHbw8EBgOFEczV2ns3C7jKzJLMbEAw16eRytXM2cAa59zW/Q2RWmetbR8I92cs3GfBo+WHwNn1dQQq+d0e5jiDwG7dMmBJ8OcC4BlgebB9FnBchHMNJHDFxlJg5f51BGQAs4H1wHtAugfrrAuBKUy7N2vzZH0RKEalQCOB47E3tLaOCFzJ8UDwM7ccyI9wrg0Ejh/v/5w9FFz2a8H3eAmwGLg4wrlafe+Au4Pray1wfqTfy2D7dODmQ5aNyDo7zPYhrJ8xDTEhIhLjYuXQkIiItEKFQEQkxqkQiIjEOBUCEZEYp0IgIhLjVAhEgszMZwePdNpmo9QGR6/0sq+DSKs6eR1AJIrUOudO9DqESKRpj0DkCILj0v/WAnM1fGpmg4Lt/c3s38HB02abWd9ge44Fxv9fGvw5LfhU8Wb2aHCc+XfMLCW4/LeD488vM7MZHr1MiWEqBCL/kXLIoaErmz1W6ZwbBfwV+H/Btr8ATznnTiAwoNv9wfb7gbnOudEExrtfGWwfDDzgnBsJ7CHQWxUC48uPCT7PzeF5aSKtU89ikSAzq3bOpbbQXgSc5ZwrDA4Itt05l2FmOwkMj9AYbC91zmWaWTnQxzlX3+w5+gPvOucGB+//CEhwzv3SzN4CqoFXgFecc9VhfqkiB9EegUhoXCu3j0Z9s9s+/nOO7kIC48WMBRYGR78UiRgVApHQXNns30+Ct+cRGMkW4Brgw+Dt2cAtAGYWb2bdW3tSM4sDcp1zc4AfAd2Bz+2ViISTvnmI/EeKBScrD3rLObf/EtIeZraMwLf6qcG224EnzewHQDlwfbD9O8AjZnYDgW/+txAY5bIl8cDfg8XCgPudc3va6PWIhETnCESOIHiOIN85t9PrLCLhoENDIiIxTnsEIiIxTnsEIiIxToVARCTGqRCIiMQ4FQIRkRinQiAiEuP+P/aRF7rW+q3CAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1rAgRpxYhjpB"
      },
      "source": [
        "### Generate new lyrics!\n",
        "\n",
        "It's finally time to generate some new lyrics from the trained model, and see what we get. To do so, we'll provide some \"seed text\", or an input sequence for the model to start with. We'll also decide just how long of an output sequence we want - this could essentially be infinite, as the input plus the previous output will be continuously fed in for a new output word (at least up to our max sequence length)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DC7zfcgviDTp"
      },
      "source": [
        "seed_text = \"im feeling chills\"\n",
        "next_words = 100\n",
        "  \n",
        "for _ in range(next_words):\n",
        "\ttoken_list = tokenizer.texts_to_sequences([seed_text])[0]\n",
        "\ttoken_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n",
        "\tpredicted = np.argmax(model.predict(token_list), axis=-1)\n",
        "\toutput_word = \"\"\n",
        "\tfor word, index in tokenizer.word_index.items():\n",
        "\t\tif index == predicted:\n",
        "\t\t\toutput_word = word\n",
        "\t\t\tbreak\n",
        "\tseed_text += \" \" + output_word\n",
        "print(seed_text)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "im feeling chills me to could to on ma ma ma ma ma ma ma ma ma ma ma ma ma ma ma ma on here my life is here and good mean blue wait quiet brother chiquitita ground ground ground ground eyes deal park shame butterflies slack always advice how eyes ground ground ground ground eyes own pavement eyes own intention intention intention intention sweet touch is disapeared good figure found give yourself a break break break rotten eyes stay tedious ways touch have music how to how do world eyes bridges car tedious break shame neighbours pavement eyes advice good song\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": []
    }
  ]
}