Artificial Neural Networks - Convolutional Neural Networks Hour.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setting up our Notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Things to import\n",
    "\n",
    "# Standard data, plotting, and mathematical tools\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import seaborn as sns\n",
    "\n",
    "# PCA\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# CNN\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from tensorflow import keras\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from scikeras.wrappers import KerasClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Importing the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importing the data\n",
    "\n",
    "dfs=['Non-Scaled TA Features 1H for BTC.csv']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolutional Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Description"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convolution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A convolution is an operation which passes filters over the image and transforms it into a Feature Map. They can be thout of as detecting increasingly complex patterns in the pixel data.\n",
    "\n",
    "Graphically, this can be represented as:\n",
    "\n",
    "<img src=\"Graphics/Convolution.png\" width=\"600\" height=\"600\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convolution, Pooling, and Fully-Connected Layers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After a round of convolutions, given that one might have many filters, the feature map will be quite large. A Pooling Layer follows which takes sections of the feature map and finds either the Average or the Max among the values and creates a second, pooled feature map. After several iterations of this, the final feature map is flattened and fed to a fully-connected layer, or just a regular DNN/MLP.\n",
    "\n",
    "Graphically, this can be represented as:\n",
    "\n",
    "<img src=\"Graphics/CNN.png\" width=\"600\" height=\"600\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting the train/test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Defining a function to create two PCA matrices\n",
    "\n",
    "def PCA_creation_train_test(no_components, X_train, X_test):\n",
    "    \n",
    "    # Scaling the data with our X_train matrix\n",
    "    scaler=StandardScaler()\n",
    "    X_train=scaler.fit_transform(X_train)\n",
    "    \n",
    "    # Fitting the PCA to our X_train matrix\n",
    "    pca=PCA(n_components=no_components)\n",
    "    X_train=pca.fit_transform(X_train)\n",
    "\n",
    "    # Scaling X_test \n",
    "    X_test=scaler.transform(X_test)\n",
    "\n",
    "    # Transforming X_test\n",
    "    X_test=pca.transform(X_test)\n",
    "    \n",
    "    return X_train, X_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confirming we have all the columns we want: \n",
      " Index(['BB_upper 10', 'BB_middle 10', 'BB_lower 10', 'Aroon_down 10',\n",
      "       'Aroon_up 10', 'Aroonosc 10', 'RSI 10', 'NATR 10', 'BB_upper 12',\n",
      "       'BB_middle 12',\n",
      "       ...\n",
      "       'MACD Signal 18', 'MACD Hist18', 'Med Price', 'Chaikin A/D Line',\n",
      "       'HT Dom Phase', 'HT Dom Period', 'High', 'Low', 'Close', 'Volume'],\n",
      "      dtype='object', length=101)\n"
     ]
    }
   ],
   "source": [
    "# Train and test splitting and scaling\n",
    "X=pd.read_csv(dfs[0])\n",
    "\n",
    "X['Date']=pd.to_datetime(X['Date']) # Setting the index as the Date\n",
    "X=X.set_index('Date').asfreq('H')\n",
    "\n",
    "X=X.dropna()\n",
    "y=X['Label']\n",
    "X=X.drop('Label', axis=1)\n",
    "X=X.drop('Percent Change', axis=1)\n",
    "\n",
    "\n",
    "# Removing the testing data\n",
    "X_test=X[-4380:] # Getting the last 6 months as the testing data\n",
    "print('Confirming we have all the columns we want:','\\n', X_test.columns)\n",
    "y_test=y[-4380:].values\n",
    "\n",
    "X=X[:-4380] # Getting everything but the first 6 months as the training data\n",
    "y=y[:-4380]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating \"images\" of our sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Image creation function\n",
    "\n",
    "def image_creation(n_steps, X, y):\n",
    "    X_images=[]\n",
    "    y_values=[]\n",
    "    \n",
    "    for i in range(n_steps, len(X)):\n",
    "        img=X[i-n_steps,:]\n",
    "        img=np.vstack((img, X[i-n_steps+1,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+2,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+3,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+4,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+5,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+6,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+7,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+8,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+9,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+10,:]))\n",
    "        img=np.vstack((img, X[i-n_steps+11,:]))\n",
    "        X_images.append(img)\n",
    "        y_values.append(int(y[i-n_steps+11]))\n",
    "        \n",
    "    return X_images, np.array(y_values) # Note, this will not include the very last one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Second to last X and y test after PCA: \n",
      " [12.00381295 10.53103332 -1.88724162 -1.1895646  -2.92464478  3.88806865\n",
      " -0.08486031 -1.60395372 -2.61335648 -0.14934652 -0.73597246  1.54475008\n",
      "  2.1361784  -1.16336855 -0.96483312 -0.52009332 -0.53966392  0.15052979\n",
      " -0.07019397  0.08884808] 0.0\n",
      "Final X and y test after PCA: \n",
      " [ 1.19564840e+01  1.00366560e+01 -6.45265865e-01 -1.32061430e+00\n",
      " -3.30942941e+00  3.79137389e+00  4.11759151e-01 -7.04865916e-01\n",
      " -1.77866589e+00 -4.68654783e-01 -8.73414523e-01  1.15515299e+00\n",
      "  1.72232772e+00 -9.64270883e-01 -1.74905022e+00 -3.91583857e-01\n",
      " -4.62367228e-01 -5.02975770e-03 -4.92897987e-02 -2.06024229e-01] 0.0\n",
      "X_train shape: (21011, 12, 20, 1)\n",
      "21011 train samples\n",
      "5253 val samples\n",
      "4368 test samples\n",
      "Final X and y test after PCA: \n",
      " [[12.00381295]\n",
      " [10.53103332]\n",
      " [-1.88724162]\n",
      " [-1.1895646 ]\n",
      " [-2.92464478]\n",
      " [ 3.88806865]\n",
      " [-0.08486031]\n",
      " [-1.60395372]\n",
      " [-2.61335648]\n",
      " [-0.14934652]\n",
      " [-0.73597246]\n",
      " [ 1.54475008]\n",
      " [ 2.1361784 ]\n",
      " [-1.16336855]\n",
      " [-0.96483312]\n",
      " [-0.52009332]\n",
      " [-0.53966392]\n",
      " [ 0.15052979]\n",
      " [-0.07019397]\n",
      " [ 0.08884808]] 0\n"
     ]
    }
   ],
   "source": [
    "n_steps=12\n",
    "\n",
    "# First, we create our PCA matrices from the X and X_test\n",
    "X, X_test=PCA_creation_train_test(20, X, X_test)\n",
    "print('Second to last X and y test after PCA:', '\\n',X_test[-2], y_test[-2])\n",
    "print('Final X and y test after PCA:', '\\n',X_test[-1], y_test[-1])\n",
    "    \n",
    "# Getting the X and X_test matrices' images\n",
    "X_set, y_set=image_creation(n_steps, X ,y)\n",
    "X_test, y_test=image_creation(n_steps, X_test, y_test)\n",
    "\n",
    "\n",
    "# Split X into training and validation for our neural net\n",
    "X_train, X_val, y_train, y_val=train_test_split(X_set, y_set, test_size=0.2, random_state=100, shuffle=True)\n",
    "# Note, we have already made our images from sequential time slices, so we can shuffle our sets of images\n",
    "\n",
    "y_train=np.array(y_train)\n",
    "y_val=np.array(y_val)\n",
    "y_test=np.array(y_test)\n",
    "\n",
    "# Finalizing the shapes\n",
    "num_classes = 2\n",
    "input_shape = (n_steps, len(X[0]), 1)\n",
    "\n",
    "# Add a dimension at the end to make sure images have shape (n_steps, n_features, 1)\n",
    "X_train = np.expand_dims(X_train, -1)\n",
    "X_val = np.expand_dims(X_val, -1)\n",
    "X_test = np.expand_dims(X_test, -1)\n",
    "\n",
    "print(\"X_train shape:\", X_train.shape)\n",
    "print(X_train.shape[0], \"train samples\")\n",
    "print(X_val.shape[0], \"val samples\")\n",
    "print(X_test.shape[0], \"test samples\")\n",
    "\n",
    "print('Final X and y test after PCA:', '\\n',X_test[-1][-1], y_test[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This confirms that our \"image\" creation function does not include the final datum."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grid Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to create the model for Keras wrapper to scikit learn\n",
    "\n",
    "# Now, we will run grid search on the optimizer.\n",
    "\n",
    "def create_cnn_model(pool_type='max', conv_activation='sigmoid', activ_function='relu',\n",
    "                     dropout_rate=0.1, filters=16, kern_size=2, neurons=256,\n",
    "                     mid_layers=1, optimizer='adam', learning_rate=1e-3):\n",
    "    # Create model\n",
    "    model = keras.Sequential()\n",
    "    \n",
    "    # First layer: convolution\n",
    "    model.add(keras.layers.Conv2D(filters, kernel_size=(kern_size, kern_size), activation=conv_activation,\n",
    "                                  input_shape=(n_steps,20,1))) \n",
    "        \n",
    "    # First layer\n",
    "    if mid_layers==1:\n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2)))\n",
    "    \n",
    "    # Second layer  \n",
    "    if mid_layers==2:  \n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2))) \n",
    "        model.add(keras.layers.Conv2D(2*filters, kernel_size=(kern_size, kern_size), activation=conv_activation, padding='same'))\n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2))) \n",
    "        \n",
    "    # Third layer\n",
    "    if mid_layers==3:   \n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2)))\n",
    "        model.add(keras.layers.Conv2D(2*filters, kernel_size=(kern_size, kern_size), activation=conv_activation, padding='same'))\n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2)))\n",
    "        model.add(keras.layers.Conv2D(4*filters, kernel_size=(kern_size, kern_size), activation=conv_activation, padding='same'))\n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2)))\n",
    "        \n",
    "    # Fourth layer   \n",
    "    if mid_layers==4:   \n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2)))\n",
    "        model.add(keras.layers.Conv2D(2*filters, kernel_size=(kern_size, kern_size), activation=conv_activation, padding='same'))\n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2)))\n",
    "        model.add(keras.layers.Conv2D(4*filters, kernel_size=(kern_size, kern_size), activation=conv_activation, padding='same'))\n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2)))\n",
    "        model.add(keras.layers.Conv2D(2*filters, kernel_size=(kern_size, kern_size), activation=conv_activation, padding='same'))\n",
    "        if pool_type == 'max':\n",
    "            model.add(keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
    "        if pool_type == 'average':\n",
    "            model.add(keras.layers.AveragePooling2D(pool_size=(2, 2)))\n",
    "        \n",
    "    # Penultimate layer with Flatten, Dense, and Dropout from the MLP grid search\n",
    "    model.add(keras.layers.Flatten())         \n",
    "    # First Fully Connected Layer\n",
    "    model.add(keras.layers.Dense(neurons, activation=activ_function))\n",
    "    model.add(keras.layers.Dropout(dropout_rate))\n",
    "    # Second Fully Connected layer\n",
    "    model.add(keras.layers.Dense(neurons, activation=activ_function))\n",
    "    model.add(keras.layers.Dropout(dropout_rate))\n",
    "    \n",
    "    # Final layer with the sigmoid for classification\n",
    "    model.add(keras.layers.Dense(1, activation='sigmoid'))\n",
    "    \n",
    "    # Running through the optimizers\n",
    "    if optimizer=='adam':\n",
    "        optimizer = keras.optimizers.Adam(learning_rate=learning_rate)\n",
    "        \n",
    "    if optimizer=='RMSprop':\n",
    "        optimizer = keras.optimizers.RMSProp(learning_rate=learning_rate)\n",
    "        \n",
    "    if optimizer=='SGD':\n",
    "        optimizer = keras.optimizers.SGD(learning_rate=learning_rate)\n",
    "        \n",
    "    # Compile model\n",
    "    model.compile( \n",
    "        optimizer=optimizer,\n",
    "        loss='binary_crossentropy',\n",
    "        metrics=['accuracy'],\n",
    "        )    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, the grid search section for this script, as well as the MLP and LSTM, take heavy influence from the following Kaggle notebook: MNIST: CNN, grid search, data augmentation\n",
    "\n",
    "This can be found at:\n",
    "\n",
    "https://www.kaggle.com/code/cedricb/mnist-cnn-grid-search-data-augmentation/notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " conv2d (Conv2D)             (None, 11, 19, 16)        80        \n",
      "                                                                 \n",
      " max_pooling2d (MaxPooling2D  (None, 5, 9, 16)         0         \n",
      " )                                                               \n",
      "                                                                 \n",
      " flatten (Flatten)           (None, 720)               0         \n",
      "                                                                 \n",
      " dense (Dense)               (None, 256)               184576    \n",
      "                                                                 \n",
      " dropout (Dropout)           (None, 256)               0         \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 256)               65792     \n",
      "                                                                 \n",
      " dropout_1 (Dropout)         (None, 256)               0         \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 1)                 257       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 250,705\n",
      "Trainable params: 250,705\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Creating and compiling the model with a summary\n",
    "\n",
    "cnn = create_cnn_model()\n",
    "cnn.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Optimize the model with grid search\n",
    "\n",
    "\n",
    "# Grid search parameters\n",
    "n_epochs_cv = 30 # Number of epochs for our grid search\n",
    "n_cv = 3 # Number of cross validations\n",
    "\n",
    "# Create model to feed to our parameter grid search\n",
    "model = KerasClassifier(model=create_cnn_model, verbose=1)\n",
    "\n",
    "# Define parameters and values for grid search to check in our model\n",
    "param_grid = {\n",
    "    # The features to range over in our model\n",
    "    'model__pool_type': ['max', 'average'],\n",
    "    'model__conv_activation': ['relu', 'sigmoid', 'tanh'],\n",
    "    'model__activ_function':['tanh'],\n",
    "    'model__filters': [16, 32, 64, 128],\n",
    "    'model__neurons':[512],\n",
    "    'model__dropout_rate':[0.6],\n",
    "    'model__optimizer':['adam'],\n",
    "    'model__kern_size':[2, 3],\n",
    "    'model__mid_layers':[1, 2],\n",
    "     # The features to range over for our optimizer\n",
    "    'model__learning_rate':[1e-5],\n",
    "    # The number of epochs for each model\n",
    "    'epochs': [n_epochs_cv],\n",
    "}\n",
    "\n",
    "# Creating the grid\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=1, cv=n_cv)\n",
    "\n",
    "# Fitting the grid\n",
    "grid_result = grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'epochs': 1, 'model__activ_function': 'tanh', 'model__conv_activation': 'relu', 'model__dropout_rate': 0.6, 'model__filters': 128, 'model__kern_size': 3, 'model__learning_rate': 1e-05, 'model__mid_layers': 1, 'model__neurons': 512, 'model__optimizer': 'adam', 'model__pool_type': 'max'}\n"
     ]
    }
   ],
   "source": [
    "print(grid_result.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CNN with Optimized Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating and compiling a model with the best parameters from the grid search\n",
    "\n",
    "cnn_model = create_cnn_model(pool_type = grid_result.best_params_['pool_type'],\n",
    "                             conv_activation = grid_result.best_params_['conv_activation'],\n",
    "                             activ_function=grid_result.best_params_['activ_function'],\n",
    "                            neurons=grid_result.best_params_['neurons'],\n",
    "                             filters=grid_result.best_params_['filters'],\n",
    "                             mid_layers=grid_result.best_params_['mid_layers'],\n",
    "                            dropout_rate=grid_result.best_params_['dropout_rate'],\n",
    "                            optimizer=grid_result.best_params_['optimizer'],\n",
    "                            kern_size=grid_result.best_params_['kern_size'],\n",
    "                             learning_rate=grid_result.best_params_['learning_rate'])\n",
    "# Fitting the model\n",
    "history = cnn_model.fit(X_train, y_train, epochs=100, validation_data=(X_val,y_val), verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/35\n",
      "657/657 [==============================] - 25s 37ms/step - loss: 0.7354 - accuracy: 0.5048 - val_loss: 0.6918 - val_accuracy: 0.5207\n",
      "Epoch 2/35\n",
      "657/657 [==============================] - 24s 36ms/step - loss: 0.7310 - accuracy: 0.5099 - val_loss: 0.6912 - val_accuracy: 0.5357\n",
      "Epoch 3/35\n",
      "657/657 [==============================] - 24s 36ms/step - loss: 0.7238 - accuracy: 0.5086 - val_loss: 0.6922 - val_accuracy: 0.5195\n",
      "Epoch 4/35\n",
      "657/657 [==============================] - 24s 37ms/step - loss: 0.7203 - accuracy: 0.5122 - val_loss: 0.6928 - val_accuracy: 0.5197\n",
      "Epoch 5/35\n",
      "657/657 [==============================] - 28s 42ms/step - loss: 0.7173 - accuracy: 0.5121 - val_loss: 0.6902 - val_accuracy: 0.5300\n",
      "Epoch 6/35\n",
      "657/657 [==============================] - 21s 32ms/step - loss: 0.7130 - accuracy: 0.5226 - val_loss: 0.6901 - val_accuracy: 0.5369\n",
      "Epoch 7/35\n",
      "657/657 [==============================] - 20s 30ms/step - loss: 0.7072 - accuracy: 0.5249 - val_loss: 0.6905 - val_accuracy: 0.5313\n",
      "Epoch 8/35\n",
      "657/657 [==============================] - 20s 31ms/step - loss: 0.7075 - accuracy: 0.5244 - val_loss: 0.6895 - val_accuracy: 0.5378\n",
      "Epoch 9/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.7073 - accuracy: 0.5256 - val_loss: 0.6895 - val_accuracy: 0.5346\n",
      "Epoch 10/35\n",
      "657/657 [==============================] - 21s 31ms/step - loss: 0.7053 - accuracy: 0.5247 - val_loss: 0.6895 - val_accuracy: 0.5369\n",
      "Epoch 11/35\n",
      "657/657 [==============================] - 20s 30ms/step - loss: 0.7027 - accuracy: 0.5267 - val_loss: 0.6899 - val_accuracy: 0.5327\n",
      "Epoch 12/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.7016 - accuracy: 0.5301 - val_loss: 0.6897 - val_accuracy: 0.5369\n",
      "Epoch 13/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6993 - accuracy: 0.5343 - val_loss: 0.6899 - val_accuracy: 0.5382\n",
      "Epoch 14/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.7007 - accuracy: 0.5285 - val_loss: 0.6895 - val_accuracy: 0.5319\n",
      "Epoch 15/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6976 - accuracy: 0.5405 - val_loss: 0.6889 - val_accuracy: 0.5348\n",
      "Epoch 16/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6998 - accuracy: 0.5308 - val_loss: 0.6896 - val_accuracy: 0.5348\n",
      "Epoch 17/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6996 - accuracy: 0.5285 - val_loss: 0.6892 - val_accuracy: 0.5340\n",
      "Epoch 18/35\n",
      "657/657 [==============================] - 20s 30ms/step - loss: 0.6964 - accuracy: 0.5351 - val_loss: 0.6887 - val_accuracy: 0.5414\n",
      "Epoch 19/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6979 - accuracy: 0.5298 - val_loss: 0.6892 - val_accuracy: 0.5370\n",
      "Epoch 20/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6937 - accuracy: 0.5447 - val_loss: 0.6888 - val_accuracy: 0.5397\n",
      "Epoch 21/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6942 - accuracy: 0.5412 - val_loss: 0.6893 - val_accuracy: 0.5391\n",
      "Epoch 22/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6936 - accuracy: 0.5397 - val_loss: 0.6895 - val_accuracy: 0.5351\n",
      "Epoch 23/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6937 - accuracy: 0.5438 - val_loss: 0.6896 - val_accuracy: 0.5363\n",
      "Epoch 24/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6965 - accuracy: 0.5394 - val_loss: 0.6894 - val_accuracy: 0.5363\n",
      "Epoch 25/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6919 - accuracy: 0.5403 - val_loss: 0.6896 - val_accuracy: 0.5351\n",
      "Epoch 26/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6938 - accuracy: 0.5404 - val_loss: 0.6893 - val_accuracy: 0.5426\n",
      "Epoch 27/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6910 - accuracy: 0.5472 - val_loss: 0.6893 - val_accuracy: 0.5376\n",
      "Epoch 28/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6911 - accuracy: 0.5432 - val_loss: 0.6900 - val_accuracy: 0.5441\n",
      "Epoch 29/35\n",
      "657/657 [==============================] - 19s 29ms/step - loss: 0.6920 - accuracy: 0.5428 - val_loss: 0.6896 - val_accuracy: 0.5363\n",
      "Epoch 30/35\n",
      "657/657 [==============================] - 19s 30ms/step - loss: 0.6924 - accuracy: 0.5465 - val_loss: 0.6896 - val_accuracy: 0.5340\n",
      "Epoch 31/35\n",
      "657/657 [==============================] - 19s 28ms/step - loss: 0.6910 - accuracy: 0.5472 - val_loss: 0.6900 - val_accuracy: 0.5372\n",
      "Epoch 32/35\n",
      "657/657 [==============================] - 19s 28ms/step - loss: 0.6896 - accuracy: 0.5500 - val_loss: 0.6898 - val_accuracy: 0.5376\n",
      "Epoch 33/35\n",
      "657/657 [==============================] - 19s 28ms/step - loss: 0.6910 - accuracy: 0.5472 - val_loss: 0.6891 - val_accuracy: 0.5422\n",
      "Epoch 34/35\n",
      "657/657 [==============================] - 18s 28ms/step - loss: 0.6911 - accuracy: 0.5431 - val_loss: 0.6892 - val_accuracy: 0.5435\n",
      "Epoch 35/35\n",
      "657/657 [==============================] - 18s 28ms/step - loss: 0.6906 - accuracy: 0.5426 - val_loss: 0.6893 - val_accuracy: 0.5422\n"
     ]
    }
   ],
   "source": [
    "# Creating and compiling a model with the best parameters from the cluster\n",
    "\n",
    "cnn_model = create_cnn_model(pool_type = 'average', conv_activation = 'tanh', activ_function='tanh',\n",
    "                            neurons=512, filters=64, mid_layers=1, dropout_rate=0.6, optimizer='adam',\n",
    "                            kern_size=3, learning_rate=1e-5)\n",
    "# Fitting the model\n",
    "history = cnn_model.fit(X_train, y_train, epochs=35, validation_data=(X_val,y_val), verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABIEAAAHWCAYAAADghZkpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABezElEQVR4nO3deZwcZ2Hn/+9TVX3NPZrRZUm2ZWNbtiTLxuYORja72GQBL8SACXGMN8CPJYEEXmEJZEm8gWSzEMhvs7Cw3ixXYtb4Z8KGTQhsCBaCrDlsfEjyIYywrdE9oxnN2VfV8/ujqrure3oOzbQ0M6rP++V2VT1V1fN0T02p69vP85Sx1goAAAAAAADnNmepKwAAAAAAAIAzjxAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACAB5gyBjDGfN8YcN8bsnWG9Mcb8hTHmaWPMY8aY57e+mgAAAAAAAFiM+bQE+qKkm2ZZ/2pJl0SPd0r67OKrBQAAAAAAgFaaMwSy1u6WdHKWTW6W9GUb+qGkHmPM+lZVEAAAAAAAAIvXijGBNkg6GFseiMoAAAAAAACwTHgteA7TpMw23dCYdyrsMqZcLnfNpk2bWvDjl14QBHIcxthG63BModU4pnAmcFyh1Tim0GocU2g1jim02pk4pvbv3z9orV3dbF0rQqABSfE0Z6Okw802tNbeJekuSbr22mvtgw8+2IIfv/R27dqlnTt3LnU1cA7hmEKrcUzhTOC4QqtxTKHVOKbQahxTaLUzcUwZY56daV0r4qZvSPr16C5hL5Z0ylp7pAXPCwAAAAAAgBaZsyWQMeZ/Stopqd8YMyDpDyWlJMla+zlJ35T0y5KeljQp6Y4zVVkAAAAAAAAszJwhkLX2LXOst5J+s2U1AgAAAAAAQMu1YkwgAAAAAABwjiuVShoYGFA+n1/qqpwzuru79cQTTyxo32w2q40bNyqVSs17H0IgAAAAAAAwp4GBAXV2durCCy+UMc1uFI7TNTY2ps7OztPez1qroaEhDQwMaPPmzfPej3vbAQAAAACAOeXzefX19REALQPGGPX19Z12qyxCIAAAAAAAMC8EQMvHQn4XhEAAAAAAAGBF6OjoWOoqrGiEQAAAAAAAAAlACAQAAAAAAFYUa60+8IEPaNu2bdq+fbu++tWvSpKOHDmi6667TldddZW2bdum73//+/J9X29729uq2/75n//5Etd+6XB3MAAAAAAAcFr+w//ep8cPj7b0Oa84r0t/+Nqt89r2b/7mb/TII4/o0Ucf1eDgoF7wghfouuuu01e+8hXdeOON+v3f/335vq/JyUk98sgjOnTokPbu3StJGhkZaWm9VxJaAgEAAAAAgBXlBz/4gd7ylrfIdV2tXbtWr3jFK/STn/xEL3jBC/SFL3xBd955p/bs2aPOzk5ddNFFOnDggN7znvfoW9/6lrq6upa6+kuGlkAAAAAAAOC0zLfFzplirW1aft1112n37t36+7//e9122236wAc+oF//9V/Xo48+qm9/+9v6zGc+o3vvvVef//znz3KNlwdaAgEAAAAAgBXluuuu01e/+lX5vq8TJ05o9+7deuELX6hnn31Wa9as0Tve8Q79xm/8hn76059qcHBQQRDoV37lV/TRj35UP/3pT5e6+kuGlkAAAAAAAGBFef3rX68HHnhAO3bskDFGH//4x7Vu3Tp96Utf0ic+8QmlUil1dHToy1/+sg4dOqQ77rhDQRBIkv7jf/yPS1z7pUMIBAAAAAAAVoTx8XFJkjFGn/jEJ/SJT3yibv3tt9+u22+/fdp+SW79E0d3MAAAAAAAgAQgBAIAAAAAAEgAQiAAAAAAAIAEIAQCAAAAAABIAEIgAAAAAACABCAEAgAAAAAASABCIAAAAAAAgAQgBAIAAAAAAIgpl8tLXYUzghAIAAAAAACsGP/6X/9rXXPNNdq6davuuusuSdK3vvUtPf/5z9eOHTv0yle+UpI0Pj6uO+64Q9u3b9eVV16pr33ta5Kkjo6O6nPdd999etvb3iZJetvb3qb3v//9uv766/XBD35QP/7xj/XSl75UV199tV760pfqqaeekiT5vq/f/d3frT7vf/kv/0X/9E//pNe//vXV5/3Hf/xHveENbzgbb8dp8Za6AgAAAAAAYIX5h9+Tju5p7XOu2y69+k/n3Ozzn/+8Vq1apampKb3gBS/QzTffrHe84x3avXu3Nm/erJMnT0qSPvrRj6q7u1t79oT1HB4envO59+/fr+985ztyXVejo6PavXu3PM/Td77zHX34wx/W1772Nd111136xS9+oYcfflie5+nkyZPq7e3Vb/7mb+rEiRNavXq1vvCFL+iOO+5Y3PtxBhACAQAAAACAFeMv/uIv9PWvf12SdPDgQd1111267rrrtHnzZknSqlWrJEnf+c53dM8991T36+3tnfO53/jGN8p1XUnSqVOndPvtt+tnP/uZjDEqlUrV533Xu94lz/Pqft5tt92mv/7rv9Ydd9yhBx54QF/+8pdb9IpbhxAIAAAAAACcnnm02DkTdu3ape985zt64IEH1NbWpp07d2rHjh3Vrlpx1loZY6aVx8vy+Xzduvb29ur8Rz7yEV1//fX6+te/rmeeeUY7d+6c9XnvuOMOvfa1r1U2m9Ub3/jGaki0nDAmEAAAAAAAWBFOnTql3t5etbW16cknn9QPf/hDFQoFfe9739MvfvELSap2B3vVq16lT3/609V9K93B1q5dqyeeeEJBEFRbFM30szZs2CBJ+uIXv1gtf9WrXqXPfe5z1cGjKz/vvPPO03nnnaePfexj1XGGlhtCIAAAAAAAsCLcdNNNKpfLuvLKK/WRj3xEL37xi7V69WrdddddesMb3qAdO3bozW9+syTp3//7f6/h4WFt27ZNO3bs0P333y9J+tM//VO95jWv0Q033KD169fP+LP+3b/7d/rQhz6kl73sZfJ9v1r+9re/Xeeff76uvPJK7dixQ1/5yleq69761rdq06ZNuuKKK87QO7A4y69tEgAAAAAAQBOZTEb/8A//0HTdq1/96rrljo4OfelLX5q23S233KJbbrllWnm8tY8kveQlL9H+/furyx/96EclSZ7n6VOf+pQ+9alPTXuOH/zgB3rHO94x5+tYKoRAAAAAAAAAi3TNNdeovb1dn/zkJ5e6KjMiBAIAAAAAAFikhx56aKmrMCfGBAIAAAAAAEgAQiAAAAAAAIAEIAQCAAAAAABIAEIgAAAAAACABCAEAgAAAAAASABCIAAAAAAAcM7p6OiYcd0zzzyjbdu2ncXaLA+EQAAAAAAAAAngLXUFAAAAAADAyvKffvyf9OTJJ1v6nFtWbdEHX/jBGdd/8IMf1AUXXKB3v/vdkqQ777xTxhjt3r1bw8PDKpVK+tjHPqabb775tH5uPp/Xv/23/1YPPvigPM/Tpz71KV1//fXat2+f7rjjDhWLRQVBoK997Ws677zz9KY3vUkDAwPyfV8f+chH9OY3v3lRr/tsIgQCAAAAAADL3q233qrf+Z3fqYZA9957r771rW/pfe97n7q6ujQ4OKgXv/jFet3rXidjzLyf9zOf+Ywkac+ePXryySf1qle9Svv379fnPvc5/fZv/7be+ta3qlgsyvd9ffOb39R5552nv//7v5cknTp1qvUv9AwiBAIAAAAAAKdlthY7Z8rVV1+t48eP6/Dhwzpx4oR6e3u1fv16ve9979Pu3bvlOI4OHTqkY8eOad26dfN+3h/84Ad6z3veI0nasmWLLrjgAu3fv18veclL9Md//McaGBjQG97wBl1yySXavn27fvd3f1cf/OAH9ZrXvEYvf/nLz9TLPSMYEwgAAAAAAKwIt9xyi+677z599atf1a233qq7775bJ06c0EMPPaRHHnlEa9euVT6fP63ntNY2Lf/VX/1VfeMb31Aul9ONN96o7373u7r00kv10EMPafv27frQhz6kP/qjP2rFyzpraAkEAAAAAABWhFtvvVXveMc7NDg4qO9973u69957tWbNGqVSKd1///169tlnT/s5r7vuOt1999264YYbtH//fj333HO67LLLdODAAV100UV673vfqwMHDuixxx7Tli1btGrVKv3ar/2aOjo69MUvfrH1L/IMIgQCAAAAAAArwtatWzU2NqYNGzZo/fr1eutb36rXvva1uvbaa3XVVVdpy5Ytp/2c7373u/Wud71L27dvl+d5+uIXv6hMJqOvfvWr+uu//mulUimtW7dOf/AHf6Cf/OQn+sAHPiDHcZRKpfTZz372DLzKM4cQCAAAAAAArBh79uypzvf39+uBBx5out34+PiMz3HhhRdq7969kqRsNtu0Rc+HPvQhfehDH6oru/HGG3XjjTcuoNbLA2MCAQAAAAAAJAAtgQAAAAAAwDlpz549uu222+rKMpmMfvSjHy1RjZYWIRAAAAAAADgnbd++XY888shSV2PZoDsYAAAAAABAAhACAQAAAAAAJAAhEAAAAAAAQAIQAgEAAAAAACQAIRAAAAAAADjndHR0LHUVlh1CIAAAAAAAgDOkXC4vdRWquEU8AAAAAAA4LUf/5E9UeOLJlj5n5vItWvfhD8+4/oMf/KAuuOACvfvd75Yk3XnnnTLGaPfu3RoeHlapVNLHPvYx3XzzzXP+rPHxcd18881N9/vyl7+sP/uzP5MxRldeeaX+6q/+SseOHdO73vUuHThwQJL02c9+Vuedd55e85rXaO/evZKkP/uzP9P4+LjuvPNO7dy5Uy996Uv1z//8z3rd616nSy+9VB/72MdULBbV19enu+++W2vXrtX4+Lje+9736sEHH5QxRn/4h3+okZER7d27V3/+538uSfrv//2/64knntCnPvWpRb2/EiEQAAAAAABYAW699Vb9zu/8TjUEuvfee/Wtb31L73vf+9TV1aXBwUG9+MUv1ute9zoZY2Z9rmw2q69//evT9nv88cf1x3/8x/rnf/5n9ff36+TJk5Kk9773vXrFK16hr3/96/J9X+Pj4xoeHp71Z4yMjOh73/ueJGl4eFg//OEPZYzRX/7lX+rjH/+4PvnJT+rjH/+4uru7tWfPnup26XRaV155pT7+8Y8rlUrpC1/4gv7bf/tvi337JBECAQAAAACA0zRbi50z5eqrr9bx48d1+PBhnThxQr29vVq/fr3e9773affu3XIcR4cOHdKxY8e0bt26WZ/LWqsPf/jD0/b77ne/q1tuuUX9/f2SpFWrVkmSvvvd7+rLX/6yJMl1XXV3d88ZAr35zW+uzg8MDOjNb36zjhw5omKxqM2bN0uSdu3apXvvvbe6XW9vryTphhtu0N/93d/p8ssvV6lU0vbt20/z3WqOEAgAAAAAAKwIt9xyi+677z4dPXpUt956q+6++26dOHFCDz30kFKplC688ELl8/k5n2em/ay1c7YiqvA8T0EQVJcbf257e3t1/j3veY/e//7363Wve5127dqlO++8U5Jm/Hlvf/vb9Sd/8ifasmWL7rjjjnnVZz4YGBoAAAAAAKwIt956q+655x7dd999uuWWW3Tq1CmtWbNGqVRK999/v5599tl5Pc9M+73yla/Uvffeq6GhIUmqdgd75Stfqc9+9rOSJN/3NTo6qrVr1+r48eMaGhpSoVDQ3/3d38368zZs2CBJ+tKXvlQtv+GGG/TpT3+6ulxpXfSiF71IBw8e1Fe+8hW95S1vme/bMydCIAAAAAAAsCJs3bpVY2Nj2rBhg9avX6+3vvWtevDBB3Xttdfq7rvv1pYtW+b1PDPtt3XrVv3+7/++XvGKV2jHjh16//vfL0n6z//5P+v+++/X9u3bdc0112jfvn1KpVL6gz/4A73oRS/Sa17zmll/9p133qk3vvGNevnLX17taiZJH/jABzQ8PKxt27Zpx44duv/++6vr3vSmN+llL3tZtYtYK9AdDAAAAAAArBiVQZQlqb+/Xw888EDT7cbHx2d8jtn2u/3223X77bfXla1du1Z/+7d/O23b9773vXrve987rXzXrl11yzfffHPTu5Z1dHTUtQyK+8EPfqD3ve99M72EBaElEAAAAAAAwDIxMjKiSy+9VLlcTq985Stb+ty0BAIAAAAAAOekPXv26Lbbbqsry2Qy+tGPfrRENZpbT0+P9u/ff0aemxAIAAAAAACck7Zv365HHnlkqauxbNAdDAAAAAAAzIu1dqmrgMhCfheEQAAAAAAAYE7ZbFZDQ0MEQcuAtVZDQ0PKZrOntR/dwQAAAAAAwJw2btyogYEBnThxYqmrcs7I5/OnHeRUZLNZbdy48bT2IQQCAAAAAABzSqVS2rx581JX45yya9cuXX311Wft59EdDAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAeYVAhljbjLGPGWMedoY83tN1ncbY/63MeZRY8w+Y8wdra8qAAAAAAAAFmrOEMgY40r6jKRXS7pC0luMMVc0bPabkh631u6QtFPSJ40x6RbXFQAAAAAAAAs0n5ZAL5T0tLX2gLW2KOkeSTc3bGMldRpjjKQOSScllVtaUwAAAAAAACzYfEKgDZIOxpYHorK4T0u6XNJhSXsk/ba1NmhJDQEAAAAAALBoxlo7+wbGvFHSjdbat0fLt0l6obX2PbFtbpH0Mknvl3SxpH+UtMNaO9rwXO+U9E5JWrt27TX33HNPC1/K0hkfH1dHR8dSVwPnEI4ptBrHFM4Ejiu0GscUWo1jCq3GMYVWOxPH1PXXX/+QtfbaZuu8eew/IGlTbHmjwhY/cXdI+lMbJkpPG2N+IWmLpB/HN7LW3iXpLkm69tpr7c6dO+f1Apa7Xbt26Vx5LVgeOKbQahxTOBM4rtBqHFNoNY4ptBrHFFrtbB9T8+kO9hNJlxhjNkeDPd8q6RsN2zwn6ZWSZIxZK+kySQdaWVEAAAAAAAAs3Jwtgay1ZWPMb0n6tiRX0uettfuMMe+K1n9O0kclfdEYs0eSkfRBa+3gGaw3AAAAAAAATsN8uoPJWvtNSd9sKPtcbP6wpFe1tmoAAAAAAABolfl0BwMAAAAAAMAKRwgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAA3lJXYKV7990P6dDRvB4p79eOjT26cmO3+joyS10tAAAAAACAOoRAi7S6I6NHpgL953/6mawNyzb25qqB0I5NPdq2oVsdGd5qAAAAAACwdEgmFuk/3LxN13cP6tqX/JL2HjqlRw+O6LGBU3p0YER/v+eIJMkY6XmrO7RjU492bOzWlRt7tGV9pzKeu8S1BwAAAAAASUEI1CIdGU8vvqhPL76or1o2NF6oBkKPDZzS/U8e130PDUiS0q6jy9d36sqoxdBVm3p00eoOuY5ZqpcAAAAAAADOYYRAZ1BfR0bXb1mj67eskSRZa3VoZCoMhg6O6NGBEX394UP6qx8+K0lqT7vatiEMhCrh0MbenIwhGAIAAAAAAItDCHQWGWO0sbdNG3vb9Mvb10uSgsDqwOC4Hjl4So8NjOjRgVP6wj8/o6IfSJL62tO6MupCtmNTOO1n4GkAAAAAAHCaCIGWmOMYPW9Np563plO3XLNRklQsB3ry6KgeHTilx6IWQ7v2n6gOPL2hJ6cdm7p18eoObejJaWNvmzb05nReT5ZxhgAAAAAAQFOEQMtQ2nOi7mA90osvkCRNFMrhwNNRa6E9A6f0rb1HFdj6fVd3ZrSxN6cNPTlt6M1pYywk2tCTUzt3KQMAAAAAIJFIBFaI9oynF13UpxfFBp4u+YGOnsrr0MiUDg1PaWB4SodGJnVoZEp7D53S/9l3rNqtrKKnLRW1HsppQ08tHNrYGz66cynGIAIAAAAA4BxECLSCpVxHm1a1adOqtqbrg8DqxHghCocqQVEYEh04MaHv/2xQk0W/bp/2tBu2IOptq7YmqoZGvTmt7sgQEgEAAAAAsAIRAp3DHMdobVdWa7uyuuaC3mnrrbUanizpUNSCaKDamigMjB56dlinpkp1+6RdR+0ZV9mUq1zKVSblKptylEvFy5zqfDblKOu5yqXDbRvLsilHmeq8q6znhPOeK8eZf9hkrVU5sPIDq5IfyA/C5bJvVQ6CaFqb94PGcis/CFTy659DkrqyKXW3pcJpLqWunKdcyiUMAwAAAACsKIRACWaM0ar2tFa1p7V9Y3fTbcbypWoodGhkSodH8poolJUv+cqXA00VfRXKvqaKvkbzJeVLtbJ8KdBUya+GKacr7TnKemGglEk5CgJVw5m6oCcKf86mlGtioVD46M6l1JX1qmXd1bJaeNSdS6kzm5J7GgEXAAAAAACtQAiEWXVmU9qyLqUt67oW/BwlPwyD8iVfhVJtPl8336wsqJYXyoFcx8hzjDzXyHMceY6R60ZlseWU48h1jFKukRuVe66JypzY80Trquudac8fWKuxfFmnpkoazZfC6VTjckmnJot6bmhCo9G2c4VSnRkvFh55dYFSdy6lE4dKyu89otWdWa3pzGh1Z0bZFHd+AwAAAAAsHCEQzriU6yjlOurKppa6KmeFtVaTRb8WFE2WquFQNTSK1o1GodKzQ5PVUKkyTtNfPf7TuuftzHhaHQVCdY+OjNZ0ZbW6I1xe1Z6mpREAAAAAYBpCIKDFjDFqz3hqz3g6T7nT3r/kB/r7f9ylS668RsfHCjoRf4wXdGK0oH2HR3VirKDxQnna/q5j1NeerguJVndmohZF2boAqSPDKQAAAAAAkoIrQGCZSbmOerKOtp7Xra1zbDtZLGtwrKjjY/laSBQFRpUA6ckjYxocL6jcpItaW9qtC4ra0p6srBRtahW2bKrNNy+vbW9r29hwuTZfK1esXAqDq7TnKO05yniO0q5TXU67bm3ec5SpW+fUrUu70f7N1rsOg3kDAAAASDRCIGAFa0t7Or/P0/l9bbNuFwRWI1OlWlg0Nj0s2n9sTPlSIEkyJnxIkpGJzasapJjq/6aXN9tXM2wjSX5gVSwHKpQDFf1AxXL0iN2lrRUaQ6GUF44hlXKdcCwo11E6GhMq5TlKReNIea6JujVWtgnHjKps40VdHlNuk+2d6evS1e0dpT1TnU+5sXpFP4PgCgAAAECrEAIBCeA4tTvBbVm31LU5PZWAqFgOVPD9uoCoWA5U8qPwqKE8Pl9oti7atxRYlf1o3g/vPFf2raamSioHgUplq1IQVMsr25WjadEPzujrT1cCJC8WErmx4MgLg6vUrMGS0ZHDBe0ee3zOn2c1v9DNnkY25zpGjgmPQ9cYOcbE5qPyyjbRetcJt3GMwu2cSnnDNtFz1LavPa+ZY+qYMKQ0MnKqzytJ9XWphKIzLit6LkfV1miEdwAAAFiOCIEALGuuY5RLu8qlXUnLb3Bxa638wKochIFQLSiqhUXx8sZtir5VKRZIVecr6/ygWlase+5AxXL98kTRr9u/ElKV/ED5Qlmpowfn96LmmV/MZ7NKN0I/sPKtrb5fLWzgtew4Jmyll0u7ak+7yqU9taVdtaVd5VLRNF6WdtWWctWWiW/XsD7anoAJAAAAi0EIBACLYIyJupJJ2ZS71NWZ0a5du7Rz586lrkaVteH4UX4UCtXP14KioDpvFQThNuF8GCoFQcM21sqPyqwNf04QjU9Veb7Kz55pOYiaOQWx56+Mg1XbZ/pyYK0K5UCTxbImi76mir4mo8dUqazxQlknxgrVssliWVMl/7RaVTlGyqXCEKk9UwuVwm6ObrVVWNqrbwlWt+zVWpSlPTeaOrVuil64LtN0n8pzhz+nFIRBZKVLaKWr59kOquJhbLMAthyEoWk5qG/xVwrCkLVxv+p8bPsgsHKdsEunG7Vkc53w798xRp7TuOyEy1F5Zd6JlXmOI8dRdVu3bl24bdpz1JZy5bnOWX1PAQDAuYkQCABw1pmoW5Ujo2WcnZ1x1lrlS7HgqBQLiCoBUtHXREOwNFUq1wKmYthNcnSqVOvmGIUYhXjLsijsaLn/8w9Ni8OudrFueAoLqt3nYusb551YoFTpnlfpuiepaZhzOmHaSpRyTRT6edUWYmEg6EatxrxqWV0Ls7SnbHW+1rIsvm8u5cpxWhPcVQK5SqjrB7UAt7LsV4LcKLiLB7mHxgMNDE9W65f1Wle3s8nasCXmZMHXeCH8e50oljVZqPw9l1UoBerIeurKptSdS6krl1JX1lNnNqW0R+gHADgzCIEAAFgixtS6O/adhZ8XBGFgEoZFthoOVboNlspWRd+vdjUsVrsiRt0L67orBvr5zw9o8+bNYWsrW7tDoK22nqqVVVpkVW5AGATTt4nvF8TmpTBIqIxZFR+gvTqge2zg9drA7U0Gaq8b0L02fpbnRgPFe5UB3U1dueMYBVFoEYYXgYIgDKQqwUbZrw9Ayn4YbIT7BPJj25cDW/d8fjwUCWr7FMpBGP6VwsCvNh8GC2P5so6PFqoB4lSxrMnTbGEmSdmUUw2ZcmlXjlG1RV7ltcYDnEo9/ViAUw5sa8K4H9w/Y92yKScKuMJwK5dyovL6ACybqgVc2Vh53XIUNGU8R4VyoIlYWDNR8KPlcH6yWNZE0ddkIZrGtqmEtNX9o+liQtdcylVXLgyIKuFQOE1Vy2vBUapu286sp1QLWo6VohCrElpNFOqDrOr7Un1/4u9b47ZlBVbV97zptDIfCzPjv9dsbLvquijIPBthYaX1ZzzM9P3obz92PqhMnVhrwUrLv8bWfi43gACwBAiBFqnkl1QMipoqT0X/OARRt4NwsNhmy03XWSlQUC2zUReFQEH0odhOW+c4jjzjKeWmlDIpeY4nz/GUcurnXSfBX7MDAKocxyjjuMp4rfl3YZcGtHPnJS15rpXAcYzS1QvN5ftvq426JjYGRpWWY3WBUVSeL9V3XwwCVbuoVQZhd51YWWyAd9etdY+rXPhO305yXSfaLmztVek6V9nOGKNH9+zV5ksuq9ZnKla3qVL0iMpHp0o6dqq+fLJYPqNjjhkjtUetsdqjcbza055Wtae1aVWb2qOWVu2ZaJoOx/tqT3tqy4TbtkfTlOdoolDW6FRJp6ZKGs2XNDoVLlfn8+H84HhRBwYnonXlOe+c2ZZ2p4VDXVlP3bmwlVHld10JbMabBF7F8vxvfJDxHLVnaq+tLe2qI+NpTWem+todY6q/y3ypchyWNTheiC2H60r+6f8SM55TC/5irdxGT03p00/832qrs3g4Oy2ItfWhbGP5mdCsK6jrOHVdTBu7oNatb9gvbE1Zu0OrY2qtMGstM+OtMM20fepbaMZbZzbZJ9aCs+6GD7Pd1KHx5hCmdoOG+PkgfG/CnxE/x1RuCGGMom684Rcc8TEXy7EuvWW/0u036vIbVFqS1rr+hvvG5+NdhSutTsP9x8entOrxf67edTYTu/Ns7S60bnU+M21dbN5zlHEdZVL1+8S3q+wvKbzJSfxLmthNUuJf1jTeRGXG7eJfBsW+FArLrIplX1LtdxP+Lioteut/t/Ebepi631PsJh6m9jt0Yr/Pxv0dJ2z92p6JusA3nltj51i6S58+QqBFetPfvUlPjzwt3b3UNZmZkZkWDlWWK2XN1nmOVw2ZKvOe48k1bvVbC8c44Z11jDN92UiOnGiciMoJwKnOV7aLLxtjmu5TYaOvGKthWPStcN18k3WN5ZVvo6vl8XUNX2O6jivXuNXX7hpXruPKM970dQ3l05ajstn28RxPk/6kTuZPyg98+dZXOSjLt351uTJftuWm2zSWBzZo+jyN+xiZ039tc7wfc71W17jhcTDH8bGUrLXyrV99HyvvX3y5Oh89gqC23Lhf5Tmrx2JsWXb6MVm3vnHfJttX7/Bla8f94xOPK/+LvBzjVB+Vv+XK76BxXWPZTPu4xq0eO45x5MiR40RT48wajsdD7sZ1lXC8LhBvCMcry7LTQ/eZ3rv4usayptvFzhXV7Zo8jxSeb13jynGcaX/zlffMc7xwarx5bRc/57Za5cuL6vtsg7r3sfI7qn5J0fBlx8nySR2bOFb3N1w5h1eOj8rfcnUb1ebRGo2/RysrawJl0lapVKDONitrjax1FcgosI6s9RTYVN2XTPHfsaTav8WV83LDsqRp5+nGf8sbn6fyb3r83/n4NHXC6LrnR7extPV/d5XX2my5UlbphpWvhF2lcjRf1lQpCKdFX/lSuRoeFUuBsilPHZmU2tNpdWRS6shkomlaHZlagJNNLf2xa63VZNHXaD4Kj+qCozAkagySjo/l9fTxcD5f8htCKU+dWU/rurLVsraMq460FwVYbm2a9urej7aMe0bGrCr5gaZKfvX3WA2I4iFmKQqTio3Bkl/dd7IY/pubSTnKmfoxutwoNHGNwqlTm3qOUxdUNo7n1VhWXRcLQa1qrQNrLQjDFn7lIGxF1LS8suw3L29skRi+V+Fyya/9+1Qdvy6ar/xzFh8rz1ZacUYtL+v38WVtIGt8WetH5xdfMr5sNG9NoEDl6N/n8Nzi+0ayriRHso6snHDZhstS5bG0Kq1DK61FPafW+jM+n3LDVqVpz1GbG7YkHSxNqCPjqVAONJYva2iGu9NW5pcr1zHhuIDxsf+ajAFojFHZD8Kuu9UxEcMxGGvztjq+Y3xMxaAyfmJQv134XPXrFpqzpj1nWkjUNmMgX38+i4dK7VFL2J5c6pwPlgiBFunXLv81PfTEQ3rexc+bdtHaLBiZKfxYyLrKhXs5KKsUlFQO6ucbp43zs+1XKBfC5crz+7X5xgu3ykXStAuzeMul+LbLWPx9r6hctJ91Xz27P87ILOvfT+XiMh4u1v29VdbFLkKnBYsNIWMlnIkHOPHwprKuckG04u1e6gpgoSphUCUsqoZIUbmk+qBmhtCmcX1L3Le41zVTaBQPGxv/LVzMRXj8/L4QMwaKTcLCpttVnqcxzJjleRt/n9Xgs5W/x+Xkr5a6AvUqx2H87y9eFl/XGJA3W1fdxnHqAuyZPk/NNo2H3vGQXJICL5DttLIdVkZSpw3U0eTf+anocbLytxHUCpv9rc33b2imfSv7N66PB4l104bPZbNu6xnJqy07Rsp0jcl0dclKKkWPpvWd6XVZSX70mOF1zef54q+jLjQNP9hUGxc2vkfV5zHhJp6MMjM9jzTti8Bpyw1fAMaXrfVVCkrh/CyfCY1qdwhdSJvIyhfUjol/+eFG/665sfLacvgFgisnKjPRFwpSeHzL1KbVE7Kx0afbSlnl/Fr/xdNMvS6srMo2UFn1XzTlU3kVstm61+RFj/Ymx1Hlnaxk1vFu0LXlWrmsqYV0qt1Mo3Y9WP/lerVVTuwLGTe2jRuFmk60jRMtu9H+8c/Wzb6cb/wSbNoXkY1fpjV8QRnft+nzRG9S+P7Gz3OqdjcPKvNBbT4MkGo38vCtNBlIY5XlkuQXba07c/Wfy+h3ZE112cbKP3HDh/TaK66a/SBe4QiBFulXLv0V9R3u085tO5e6KitG0xNtZbn67cX0lgONH/ynfUOpJuti24Qr1HSf2f5Br3zonq21zbTWNzO00KlrhTPL/vt/tl9bLt3StFXN6bZEquwzW+ucygfSynt9Oq2MZmtZNJ/XWg0WY8dA4/Ex03z8wrbxuGnWwqRx3/gH9eqH84bWGI0f3Bs/5De25qh+wHfqL9LjFw7NvjWPzzd++Ksrrx7KzY/jeHllOxnpxz/+sV7wghfIt361ZVO8hVPlGG8siz/86JvASlBmZasfFCsBWuWiNP5oDMebhXMzBeCS6sKB+YTldX/38feryUXDtIuQZtvNcAHS7EKlsdVXNWSMt+KLtR6Lt9Rrtl9l/VzbNb6n82mFM2erndj7OtO2Tz75pC657JL5hU9qHkTN1Bqp6X6xv+mFaGzpeVr7yk77N2Om5YrZLmrrtpvjeast7Bq/FGryu4n/rc2nfKa/z/j7FT//xpfjH/KnLTeZVp4zHnJUyiq/22d+8Yw2b97c9Bw52/s123s70/taKavUcaYWntO+MIi1Bp1t3UzPVwyK1XNmpeV05TU0tpRyHbd6LotfpDUu1537Yl9Exs8N1Z/T5DNPs7+NpiFA06L57Rv/nTf+3PixULes5q0xZ9w2VsfK77Voimrz2qZXfI76Lma7ZnWedgHc0IpNami52uy1N67X9NZw8c+DnuMpYzLhctSqvzIf/yzYdBr7TDlt2rC9a9zqv0lzBU3V5YbPjk2nTT4zhutL1bL6lofNP0PUfVk4j88e0/6eYl80GhkdO3ZM69asm/fxMWP5fP/uwhUznmNn+1K+ev6OvjyIb18OyjOes+PPE/+3babPpo2frZp9NnUcp3a+brgmq+wbL6+83uprbxZANdS78h42C5nCc3ali2dQnQaVqQ1b2q3uXL7dvVuFEAhnXePJebmrfBvtylXaTZ+Vn7nr2C7t3LLzrPysRo5x5LiOUkotyc/HmfFc6jld3HPxUlcD55hdh3Zp56U7l7oaOIfsGt6lnTt2LnU1cA7ZtWuXdu7cudTVwDlk165d2vnynUtdDWDBVsZVOAAAAAAAABaFEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASYF4hkDHmJmPMU8aYp40xvzfDNjuNMY8YY/YZY77X2moCAAAAAABgMby5NjDGuJI+I+lfShqQ9BNjzDestY/HtumR9F8l3WStfc4Ys+YM1RcAAAAAAAALMJ+WQC+U9LS19oC1tijpHkk3N2zzq5L+xlr7nCRZa4+3tpoAAAAAAABYjPmEQBskHYwtD0RlcZdK6jXG7DLGPGSM+fVWVRAAAAAAAACLZ6y1s29gzBsl3WitfXu0fJukF1pr3xPb5tOSrpX0Skk5SQ9I+lfW2v0Nz/VOSe+UpLVr115zzz33tPClLJ3x8XF1dHQsdTVwDuGYQqtxTOFM4LhCq3FModU4ptBqHFNotTNxTF1//fUPWWuvbbZuzjGBFLb82RRb3ijpcJNtBq21E5ImjDG7Je2QVBcCWWvvknSXJF177bV2586d83oBy92uXbt0rrwWLA8cU2g1jimcCRxXaDWOKbQaxxRajWMKrXa2j6n5dAf7iaRLjDGbjTFpSbdK+kbDNn8r6eXGGM8Y0ybpRZKeaG1VAQAAAAAAsFBztgSy1paNMb8l6duSXEmft9buM8a8K1r/OWvtE8aYb0l6TFIg6S+ttXvPZMUBAAAAAAAwf/PpDiZr7TclfbOh7HMNy5+Q9InWVQ0AAAAAAACtMp/uYAAAAAAAAFjhCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEsBb6goAAAAAAIAEs1aaHJLGjkjjx8Jl40iOJzluODWu5Dix+Uq507BNtE98/2o57WAIgQAAAIBzkV+STjwlHdsrHd0jHX1MmhiSei+UVm2W+i6WVl0krbpY6trAxRGwTASFgkoDAyo++6yKzz6n4nPPyuYLMtmMnExWJpcNp9mMnGxWJpOVk8vKZGLL2YxMNhdNs3Iy4dS47tl9MdZKhTFp7GgY8FQfR6XRw1H5UWn8qOQXz06dZguR3nKPtOH5Z6ceS4QQCAAAAFjppkZiYc/eMPA58WTtosrLSmsul3o2SScPSE9/R/ILtf3dTBgMrbo4FhBFIdEiAiIbBLJTU7K+H158plLh1HFkjFn868a5wdrwWC1NSeWCVM7HHoXm5YEvZTqlTFc07ZSy0Xy6U3KX96VukM+r+NxzKj33XBT0PBeGPs89q/KRo+F7EnG6uuS0tckWCgoKBdl8XgqCBf1ck0qFYVAUKIXhUW3ZZLNhkFRZ35aTyeXktLXLyeXktOXktLWFZWlPjibllEfl+KfklE7KFAZlJo7WQp/RI1JpYnpFMl1S5zqpc710wUtr813rpY61YUhj/fD3HJSj+SA2Xw7X2SA2HyuvLp/m/rnehf5KV4zl/ZcBAAAAoMZaaeS5KOzZEwU/j4VlFW390vorpYveJa27Ulq3Xep7Xv1FcRBIo4dkB5+WPfqUgqNPKzj+CwU/2y87tEtBsaSgbGR9o8CmFaT6FaRWyaa6FTjtCtQmq7SCslEwlZedmlJQeeSnZCfDeVsoTH8NFZ4n43lhKBRNjedJnivjRsspT3Ir27gyXirazg3LPa8237i/56rj+Amd2LNXTlubnPZ2Oe1t4XzDw0TrTSpFOFUR+FJpUipOSsXx2nxpIiqbCOdLU6cX3sxU3mqptukhUWU526Sscbts1/zDpEqIVZwI36fSlFScUDA2HAY8AwMqHjyi0pHjKh4dUvHYiMrDk3VP4eZcpXo9tXUZpTemle4oKd1WUKptUp5zONzISUluWtZNy5qMrFIKbFrWphTYlGzgyQauAhtNfVOb+k6Yf1TyjrJVUAqi6aTs5Jj8EV+2VFZQLMsWywoKRdlCUbZUPq233niSk3bkZFJyspvk5HIy7e1y2jvldPbI6eqVk+mS8dvkFNrkuG1ybE5OORcuT7hyurrk9vTI7e6Wk82e1s/H7AiBsCIEU1NycrmlrsYZZ61V6eBBpQ4cUOH88+V2dcnt6pJJp5e6agCAM8SWy/LHxhSMjdVPR8ckY+St7pfX1ye3v1/eqlXhRS7mpzghjR+XJk6E0/FjtfmJ4+F0aji8+GtfLbX3R9PV05fb+s5qywJrrZSfUDDwmIKDj8oe2qfgyJOyR59WMDURXtSVHQW5tbJtGxVkn68g3S/r9SgoObL78woezSvIf0926tsK8nnZfL4a1FRCm+YhTXeTsrykw5IOy7iBHM/Kca1MSnIyGTm5nNz2DqX6u+V0rZHTvVqmu19Ork1OW05yXcn3Zcu+rF+Wyr5suVyb933Zcqm2Tbks+eXq9rZYlMqlcJtSXsFkKSwvl2Z4Ll/yfbUXihr8znfm/8Y7Rk7GDS9g046clIk9wmtwx7NyUjacer4cNwjfE7ccPXwZx5F10pJb2Skt66TCZVOb2sqy40pOStbxwu1NSnI9WeNF3VXCclsZ1yRWbk1tvBO3PSOvIy233ZWbMTJ+fnpoMy3IiU9jgc9CghkvJ3mZsOVZKhtOvUxUng1bWVTWe7H1qdwc5bm6MmsdBVMFyc/LBFMy5QmZ8kTY7agwJuVHpcJobbkwFi5PnKjNF8bCViBzqYZJYVh09fiE9ESq+p75k1MqnSyoOGpUHPdUHHNVGvdUHPNUztd3vXIzvtIdvtq7y0ptktK9ntK9GaX7cnK72qVUu5RuC39muj18/ZV5mTBo8osy5YJMNO+WC9VyVebLhbA7qF+olflFqVwMy06z25UNpMD3FGTWKMj0y6b6FKR6FbhdCpxOBSanwObCMKpkFUzlFUxNKpicDM81k1MKJidVGplScPgXCqb2hUHx5GRdi6eZmGxWbnd3NRSqm/bMUN7dzTXUDPgUgWXHlkrKP7VfUz/9qaYeeViTDz+i8pEj8tatU3brVmWvuDyaXqHUmjVLXd0Fs9aqfPiwpvbuU37vXuX37dXU3n0KRke1StKBj3+iuq3J5aqBkNPdJberO1zu7gpT8q7u8ATYFS13dycyQLK+L390VP7wiPyRJo/h4dr82Jjc7m6l1q2Vt2atvHVrlVq3Tt6atUqtWyu3r0+GsRGAUCkftjIYfiZ8jDwrjR7SZUNjUvGfwg/1uV6pbVVtPhfOWzcjWyyGF6DFYngRWijIRo8gX5AtFqLyomwhXysvFBQU8uG3kIVwP5XLMulMQ7P12LgHTcZBqIyREDZvj42V0IIwxVorm8/XwpvRUQXj4+F0bFz+2KiC0TH542P107HK+jHZycm5f1CFMXJ7e+X19clb3S+3rz823yevf7W8/j55/f1ye3vP/tgPc7BBoGBiIgq6xhWMR4HX+ISC8XFln31WE+l0+Lr6++T29Ex/DcXJ5mHOtPkT4cVsM7lVUseaMNxZc3l4MTg6IB15JNwvmOFb79yq5gFRs/Ao2y3r+yoPnVR58ITKJ+of/slhBVOTslP58O9jclzBxJiCyQnZQlFB0ZeaXhvlokf1DZH0TPSQlErVjvdcLppm5WRzcvv7w7+ZqEtHuH76vJPLhX9DzeZTKZnxI2GXspM/D6dDlfl99V3MilmpfbPUfnF4fqhcgJajC9LqfHTBWi40rI8uZoPSnMfWbMILWBO2bCqH06DkKPBTCgI3vLj13fBRdhT4QXghW7ZhL5FS2LAjKFkFRStbsgqKzcIDN3rE+dHjDLR0mYuxctOBvGwgNxPIy/pys5LX7sntSIdhUWdOXneH3J5uOV0bZDIdUejQJqVj85VwIh0PKmKBRSonuWlpAa2obKkk/9Qp+cPDKg8Ph5/jhkfkDw9Fn93i5eFnuWC8yd+244THZyoVthCL5pWqzHfIeD3127gmfBgr40hyAhnjy5hARr6MStGjLGOLMrao8RFfE8WUisNpFYes/HFXUke1Gm5Pu9Lr+tS+fY3SG9Yrff4mpS64QOkLN8vtXR29X21L213N2lhIFAuGmv2NZrtlOtfLbV8t12ntvynW2vDf+qkpBROTslOT0fyE/FOj4XExMhJOT43IHwmnhQM/j9adkkoznx9MW1t4jdTd02Q6c5C03P7tbDVCICw5f2REU48+qsmfPqyphx/W1J49slNTkiRv/Xq1XX210rf8iorPPKv8vn0a/+53q4mxu7pfuSu2Krv1imow5K1bt+ya8VprVT52TPm9ezW1d6/ye/cpv2+f/OHhcAPPU/bSS9V1003Kbt2qJ44f19YLL5Q/ekrB6Gh4EhwdDZdPjap06JDyTzyh4NSpMEGfxekESKby4S6Vlkl5Tf8hrczrDDeXDorF6WFOPMRpVjY6OvO3CZ5Xd5JPrV8v/9QpTT74kErHj0//B8TzlFqzRt7aKCBau07e2jAg8tauU2rtGnlr1oTvBZaNYHJS5cFBGdc9J38/1vdVPnpUxYMDKh58TqWDAyodOiQ7ywegeT5zeKFVrHxDPFn/bXGp4eLFcSU3q3KxpGfLj4ZNyX0jGyi8wArCLiTWN7LB4s4TJhrI0kmnqwNaVkOlKFCS7y/syVOp6kCZ1WkUFDUuy6gW6tSFO+OzfgCVFJ5/OjvldHXK7QinmdUXy+nskNvZVZt2dYbbdVamXVLgqzw4pPLgCflDQyqfGFR5aEjloUH5JwZVfO5hlQcHw/EhGjmO3O5OeT2d8rrb5HXl5LZ78jo8eTkjNxteFHrpvFx3SqYY+8bcBuEYMV46nLphK4bAT8kve+EFdMmRX3YUFI38ohQUrYKClZ/3FRR8+QVfQb6sYKoof6qkYLKgIF+c9VvfbknPfeELsQNActvC+nrZstxUUV66ENY944cXt5XX0dMj0x0FOxuuqYU8HWtj89HUneXcYK2UH5EmBmNB04na8sQJBaeOq/yzPSoPDat8alLlvKPylKvylKNy3g3n8678gtM0yHE7cnK72+W4gRwV5NoJOXZKJmPltFs5uTaZ7tVyetfL9G2Ss/pCOf0bZbJt4fgd2Vw0jQKdSuCTyZyd817PpvBx0Svqy6MuZtMCoqGnpUOnwpDATYetOSpTLx22sPAy4e+l4birznvRvnXrU9F+zdaHjwd+/JBe8kvXyXU8uY4Xa5njLiiwqLBBEJ6DJifrHxMTCiYmw78h44Q/wzHhZybHkWTC/5xoXbSNcUy0HJUpkAl8yZaivjul2rJfisY0KclEUwVhuT/ly58oqTxelD+WV3l0Uv6pcZVHRpUfHlH5yLCCsbHoVRSjxylJ4fnW7Vslry9sdej2WXl9KbmrPHl97XJX9cnrWyW3s09eb0/TLxprX8gNVx91AU7lc9vwsMojYXkwOjrj++y0tcnt7a0+0psvlBfNO21t1VZjtlQM/y0sl2VLpegRny9F21XmS2G3xVJJmnW7slSeHgp7azqUPv8idVx7vtIXXKj0+ecrfcH5Sm06X25H+4KPq7PGmPDvxEtLmaWshqn+W6ve0x+Lx1orOzlZC4oap8P1y4WjT0WB0qkZPztccPdfq+2aaxb70pY1QiCcVdZaFX/xjKYefliTD/9UUw8/ouLPfx6udF1lL79cPbfcorbnX63cVVcptX79tOcIJiaUf/JJ5fc9rvy+fco//rjGv//96uBo7qpVyl5RC4WyW7cqteG8sxoMlY4fD+sWtfKZ2rdP/uBg9XVmnvc8ddxwvXLbtim7bZsyl14qJ1M7Axd37VL3zp3z+lm2VJI/Nib/VBQYVUOjeIBUmy8dPqz8k08oGJk7QJpTk3Bo5m9fUmE//mnbeAomJqeFO7PVzeRyUaDTI6+3R6kN51WX3Z6e8INCw7LT3j7jMWCDQP7JkyodPaby8WMqHT2q8tFjKh87qtKx4yo88aTGd32vGk7WKmLk9vfVAqK1a+WtiwKiteuiwGhtIroynknB1JTKQ0PyBwdVHhwML4iHwnl/cCgsGwqndS0qHEfe6tVKrV+v1Hnr5a1fr9T685Q6b71S69fLW7cu/LZnmYXG/tiYSgcPqnhwQKWBgyoePKjSwQEVBw6qdOhw/YdRz1Nq/Xo52Xl8gguC8MIhKEbf/pWib/miaeOVaqXbgtsuub3VcQjCCyhPMtK4X1TXun45qZTclBP2TnCsjGtlnLIcU5ZRSY6KMjYvY6fkBFMywYRMeUyOKSrsxRB1K4kejmvD8Ke9V6YtF2td1Ctlu6fdLtYGRkHZhmFUOfqWvuRHy4FsKQjHPSj5YXmxMi2H00JJQSka/6BYki1Myp88FbZAKpYUFMLgwulol9vRLre9Tem158tpb5Pb0Sano01uezRty8npyIXL7eHUZFIyUtTtwEYhyGzTCUnjUv6w5BeVdkelnlEpNyatrgQ1RalgpEJGNt+hYCyQPzKu8uiUypO+ynlHft5VeWpM5YKj8nOuivkwoGgazBmFAVFnRl7nOsmYWniTz8vPT8iW5u42YZyom0wqkJsK5KSs0ulATqeV0xfIjdY5KVtd76aDqGtNoKDsRHUPp2W/Q345pXIxJT+fUXE0p/JYUbbYvKWO0yV5fYG8vqLcvil5fRNy+0/J60vJ6zPhRW2/L6+vT05bW92+1loF4+NhS53jDS13BgfrlmsXrdnoIckxUeDWrtTqtHLtjtyclZcpyvMmlXLH5GlIbmpS1S/TjSP1Xyqte1k4bs+67dLa7VLH6jnf62XJcWYOiJZIIftsGP61mHGccByhhuNoJQiKxTBUHjop/2TDdGhI5ZMnVT5xQvmnnpI/NDTjFw1Od7e8VavkdHaGnzOHh2f9Qs7kcnJ7o89vPb3KbdwYhTvhZzUvFvZUPsPFPx8vFRsEYTBULEnlkr7/ox9p5403LnW1oChEam+X096u1IYN896vcr4PrztOxYKiEaUvvPDMVXiZIATCGRVMTSm/d68mH34k6t71iPyREUnhPxxtV12l7te+VrnnX63ctm3z+ofUaW9X2zXX1CW0wdSUCk89pakoFMrve1xD/+N/VC+WnO5uZa+4XLl4MLRpU0u6+5RPnqxv4bN3r8rHj0eVdZS5+CJ1/NIvKbttm3LbtiqzZUtLBzczqZS8VavkrVp12vs2BkjBVL767UjlG5DT+UYl/DalYbvKNoWigvGJ+m9rou2qHwpW9ytzyfNmDXPOxAcC4zjy+vvl9fdL2tr8vbJWwdhYGBAdi4KiY8fDoOjoMZUOHtTkgw8qOHVq2r5Od3cUEK2Vt3p1eHeFSreVXKxrSuNyXfeVTG3+HBi0MigUwlAnCm8qD39wqFpWCX2CiSZ3lJDCJrvR7y23bVtd1xgb+CofOaLSkaMqHTmiqX37VP7H70z7IGtyuTAkigdF69bXBUWtPt5qrXlqAU8Y9gyodPBg9RxZfZ09PUpt2qTc1q3quvEmpTZtVHrTJqU2blJq3dpalya/HH4DP/JsrdvW8DPScLQ8OVhfkXSntOpCqeeC8HbRvRdKvZul3guk7k3hWA5z2LVrl66cZ2A9/Y2wYf+KqWFp6mQ0jR6TDctTw2GLgqlhKX+q/i4ekoymd8CYl1T0WMw1XCBpNHqcDY4XG7w0nJqu8+SuvkxuplPp6rgV3dMHOM10yaY7FJSMyiMT8oeiv8ETg/XB6tCQJMnt65DX0Smno0NuZ4ecjs6o1VKnnI7OsKyzsj5swVT9ewmCJt0L4mNUNOl+4Jf02P5ndeVLXjlni51gcrJ2rjh5shoO+4PhBaw/OKjC/v2aGBqasZWByeXCsZa6u+WPjqp84kTTFlUmk5G3erW8/n5lLrpI7S96kbw1q8OyyqO/X+6qVXN3IbA2bHE3cSKc9l0cdqMBziInnZYT/ds3l+rFciUcGhqSP3RS5ZO1aTA6pvTGDdHntViQ09tTC3Z6elbsl2LGccJWT5WWT8sgmMLiGGPkRq1utWnTUlfnrCMEQkuVjh0Lu3Q9/LAmf/qw8k88UQ1i0hddpI5X3qC2q69W7vnPV/rCC1s25oqTyyl31VXKXXVVtSwoFFTYvz9sMfR42Gro5Je+XL0IdDo6wkCo0mpo6xVz1skfGQmDpmoLn70qHz4SrjRG6c2b1faiFym3bauy27Ypu2WLnPbl2yR0MQFS0hhjqt3qdOmlM24XTE6qdOxYXUBUPnYsLDt6VIUnnwoH4sznF96Fx3Fm776SydS6BUTjpXQcPabjP3241vTdqDofBkomti5sql6ZNya2bobtatvE1xnZQr52cXaiFvrUmqI3vLSurnB8k/5+Za64XO39q5uPd7Jq1WmPd1Vt8XXkiEpHjoQh0eEj1eX8U0/VWuzFuP39taBo/bppLYrcVaumnTdOuzXPhvOU3rhJ2ZturAY86fM3KbV+nVyvJE0OhV1RJgej+QPSY0PSD6PuKSMHpVMH68cxMW74bXzvhdKWfxULeqJHrndRXSEWzZhofIk2qXv+397VsbYy2Efslq/x28LGbxnbeJvYmW4fWw4DjGn7B7WuGlrI1In+PE5zXzcdtn6qhDledlG/t0pg5q6SdNHmBT/PnBxHcnKnHXCcHNolnXfV3E/f1qZ0W5vS8/jwbovF8OJ1cChs7dAYGI2MKL15czXMaQx4nM7O1gXvxkiZjvABrADxi+UktJAAkoAQCAtmy2Xln3pKUw8/Uu3eVQlETDar3Pbt6vs3/0a5q8NwxltAP8/FcDIZ5bZvV2779lqdi0UVnn5a+ccfr7YaGv6f/7N6ZwynrU2Zyy+PgqEr5PWvVuHJJ8LBm/ftU+ngwepzpS44X21XXa3sr92m7LawhZHbsQI/1E0NS8efkI4/Hn5D27FW6lxXm6aXb4i1HDltbcps3qzM5rkvrqzvRwPgRmOb5Avh4LdT+XCajwbIjS/nC2Ef9nw0YG5sOdymoPLoaN14KTafV1s+r6HKRYy1tUdl+Qxycml53e1yu9uU6W9X+/MuCZd7O+V1d8jrDccscbs75WTSsTESGh8FyRyWxo9IE83WRw+ZcLBFNxONNxHebcS4aXk9YcgUPy/EBcWiykePxsKhw9WwqPDzn2v8+9+f1i3QpNPy1q9Tat36MAScrTXP5Zepa+dLle7vVGpVRuluV162KJMfDgOeif3S5APS/kHp4cFwXJKZZLvDuxW19UvnXS1tfX0s5LlA6tq4tINOng3GhGGX40pKziD4OD0mnVZq3Tql1q1b6qoAALDkzvFPh5iNtba+q0+5oatPtUtPsbpdMDmpqX37wuDnsceq4294a9Yo9/znq+1tb1Pu6quV3bJlWQ7IatLpauufnltukRR2iSoc+EV1fKH8vn0aue8+2b+qXeilNm5Udts29bzpjeE4PldcIbe72e1Tl7HSlHTiyTDwObavFvyMHZl9v3Sn1LlW6lg3+zTbs7StChajcoeEyi1A/Mo0Gmgx8GvzlXVBubZPdV20bXU+ti4Ib3Ur64ff7KfaZFJZGS8nJxV9W+5lpY6c1JuTvNXRrVVj6xbRcm7Xrl3aOVu3HWul4rjsZNQVJz8iTY5E05NhF5ypEdmpkVqXnKmRcJo/FYVK0VPFntYYK2emf2nGosfAgl/WwnnZhpAoHFjU8TJKRw+5GakvI63NSNdkJO9SWXebgqJR6VRZpVNFlUbyKo9MqXRyQqWTh+WmXWV3rFO6e4NSHb7SuYJSmVG5/og0sVsqT0kFSYeiR4Xj1QKd9r5wXJC2/rCsPZo2zs82sC0AAADQBCHQMlG9Pd74eHhLvGgajE9EdxloKJ8Ibx9aC2/qx1hpNrJ94/ycdzOZiesqe9ll6nn965V7/tVqu/pqeevXr9gxSkwqpexllyp72aXSG14vKWyhUXzmGZVPDCpz2aVnvRXTovjlcOyM4/vqA5+TB1S9PHcz0urLpM2vkNZeIa25IrxFrpeTxo9KY0fD2+82Tg/9NJyWmgza7GXDMRzqwqHoES9r61tYmOGXo7sVVR7jTeYnG8obtqvbf6I2KK5d4J2FzjY3EwZDqbbw/a6EQ6m2KDCKylK5KDyqhUgbDz4jffcH0fgqI2G4Uw10RqrjrMz4V2xcKdcjk+sNA7+ufmnN86LBenvCslxPNHBvT60s3RZreRTM8agMijvHNnM9TxBEoVs0Bkm5Mu5IPrYcWzdtOV8bq6Yc7Rftb8pFueW83KBUGQpW6oweF8Ter3RneDvk9n6p7TypbXsY7lSCnsZAJ9u9ckNUAAAArBiEQIsUFAoyo6MqPvvs/MKbSnkl7JkYVzAxqWB8fH63uDVGTjQCuoluA9p4NyanvT2cT6ca7t7U5M5M6YZbfntedHvwaD7dsH0mo8xFFy3rcW5awbiuMhdfrMzFFy91VWZmrXRqIGrRUwl8HpcGn4ru9KOwa8yqi6S1W6Xtb6wFPr2bZ+4m0t4Xbj/bzy2MTQ+Jxo7U5k88JR3YLRWmD5Isx5Pa19S3IpKiAKcx3Ikt+4X5vzeOF3ZjS3dE02i+c31tOdUWu9NRdLcj16vdEcnxorJofWXdtO2i28xWt2vc16v/GY4bBg2lyVrYUM6Ht+CeVjYZlpenYtOp2PrY/OTJhrJon+h9e54kHXDCsCEe2PScHwtxGgKdeFm6g5AiLghqwVI1YCqGoVdu1bwGVQYAAADONkKgRfrFG35Fa37+c/18to2MkdPWJqejIwxwOjrktLcpvbo/vEtQQ7lbt9wupz2cuh3tMrlcywZTPi2VC//8Kck/JU3ka10oHI+LwzNt8mSsC1dl+oRUiN3xpGtD2Jrn4utrLXtWX9b6u44YE95lJtsl9V8y+7alqZlbFY0dDQeyHfhJNEBsLLTJdkld6+tDnFR7faDTdD56uOnlfUymsmcvJAgCqTyl7+/epZff8OpFdSlDjONIzln8PQIAAAAtQAi0SH2/8Rt66rFHddlVV80Y3jhtSxTczKRcDG/HO3kyNh1uKBuevi5+15k6JuyG4qVnHGejti4987ZzrXNT0QCgUWuKSmuLpsvxsti6Mx0MVO4wUx0LJrYclOaxPrwTzboj35e+9X/CMXuOPx6GJhXZnrClzpVvisKeK6Q1W8IWG8tNKiet2hw+sDQcR0q3y/faCYAAAACAhCMEWqSeN7xeU6t61TPbgKtnShCE3W0qQU0lwInPTwt6hsPuNTNxM+E4FrlV4XT1ZWG4UCnL9YRjbkwbSyPqCjHTOBvFybAOfrH5GBzBAscnOl3GmSM8ipYrYZNxYiFNk4ffsFw3JO7CbZHCsVxWXyY971+ErXrWXC6t2RqOsbOcW7kAAAAAAJYlQqDlzi9Jw89KQz+TBn8mDT1de0ycCAOZZowTthhpWxWGOJ3rw9YjuVVSW28U6PTWBz65VWHLjaUIGIKgPkyqjrMRC4yqYYvf0IKmPL9lu5B9guktjNzGFketX/+jR/frRTe9KbrtMQAAAAAAi0cItBxYGwY6gz8Lw56hp6XBp8P54Wfqu2G19YfjsFzyL8NgJx7gVAKfyt15VlLXD8eRnFzrx69ZoaZ+NkYABAAAAABoKUKgs6k4KZ38eRT2/DzWuufn9XdR8rLSqovDsV6uuFnquyQMfvouXp7jvgAAAAAAgGWPEKjVgiC849FQFO5UWvcMPi2NDtRv27VR6n9eOMBvJeTpu0Tq3rSyWvEAAAAAAIBljxBosfZ+TZsPfFM69j/CblwnD4Tj2VRkuqS+50kXviyc9j0vDHxWXSyl25au3gAAAAAAIFEIgRbrB3+u84/uk/ouCgOei2+IWvVcEi53rOFOTgAAAAAAYMkRAi3WW7+m3T/Zo1fc8C+WuiYAAAAAAAAzYuCZxepcK+uQpQEAAAAAgOWNEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACAB5hUCGWNuMsY8ZYx52hjze7Ns9wJjjG+MuaV1VQQAAAAAAMBizRkCGWNcSZ+R9GpJV0h6izHmihm2+0+Svt3qSgIAAAAAAGBx5tMS6IWSnrbWHrDWFiXdI+nmJtu9R9LXJB1vYf0AAAAAAADQAvMJgTZIOhhbHojKqowxGyS9XtLnWlc1AAAAAAAAtIqx1s6+gTFvlHSjtfbt0fJtkl5orX1PbJv/T9InrbU/NMZ8UdLfWWvva/Jc75T0Tklau3btNffcc0/LXshSGh8fV0dHx1JXA+cQjim0GscUzgSOK7QaxxRajWMKrcYxhVY7E8fU9ddf/5C19tpm67x57D8gaVNseaOkww3bXCvpHmOMJPVL+mVjTNla+7/iG1lr75J0lyRde+21dufOnfOp/7K3a9cunSuvBcsDxxRajWMKZwLHFVqNYwqtxjGFVuOYQqud7WNqPiHQTyRdYozZLOmQpFsl/Wp8A2vt5sp8rCXQ/2pdNQEAAAAAALAYc4ZA1tqyMea3FN71y5X0eWvtPmPMu6L1jAMEAAAAAACwzM2nJZCstd+U9M2Gsqbhj7X2bYuvFgAAAAAAAFppPncHAwAAAAAAwApHCAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkADzCoGMMTcZY54yxjxtjPm9Juvfaox5LHr8X2PMjtZXFQAAAAAAAAs1ZwhkjHElfUbSqyVdIektxpgrGjb7haRXWGuvlPRRSXe1uqIAAAAAAABYuPm0BHqhpKettQestUVJ90i6Ob6Btfb/WmuHo8UfStrY2moCAAAAAABgMeYTAm2QdDC2PBCVzeQ3JP3DYioFAAAAAACA1jLW2tk3MOaNkm601r49Wr5N0gutte9psu31kv6rpF+y1g41Wf9OSe+UpLVr115zzz33LP4VLAPj4+Pq6OhY6mrgHMIxhVbjmMKZwHGFVuOYQqtxTKHVOKbQamfimLr++usfstZe22ydN4/9ByRtii1vlHS4cSNjzJWS/lLSq5sFQJJkrb1L0XhB1157rd25c+c8fvzyt2vXLp0rrwXLA8cUWo1jCmcCxxVajWMKrcYxhVbjmEKrne1jaj7dwX4i6RJjzGZjTFrSrZK+Ed/AGHO+pL+RdJu1dn/rqwkAAAAAAIDFmLMlkLW2bIz5LUnfluRK+ry1dp8x5l3R+s9J+gNJfZL+qzFGksozNT0CAAAAAADA2Tef7mCy1n5T0jcbyj4Xm3+7pLe3tmoAAAAAAABolfl0BwMAAAAAAMAKRwgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAAhEAAAAAAAAAJQAgEAAAAAACQAIRAAAAAAAAACUAIBAAAAAAAkACEQAAAAAAAAAlACAQAAAAAAJAA8wqBjDE3GWOeMsY8bYz5vSbrjTHmL6L1jxljnt/6qgIAAAAAAGCh5gyBjDGupM9IerWkKyS9xRhzRcNmr5Z0SfR4p6TPtrieAAAAAAAAWIT5tAR6oaSnrbUHrLVFSfdIurlhm5slfdmGfiipxxizvsV1BQAAAAAAwALNJwTaIOlgbHkgKjvdbQAAAAAAALBEvHlsY5qU2QVsI2PMOxV2F5OkcWPMU/P4+StBv6TBpa4EzikcU2g1jimcCRxXaDWOKbQaxxRajWMKrXYmjqkLZloxnxBoQNKm2PJGSYcXsI2stXdJumseP3NFMcY8aK29dqnrgXMHxxRajWMKZwLHFVqNYwqtxjGFVuOYQqud7WNqPt3BfiLpEmPMZmNMWtKtkr7RsM03JP16dJewF0s6Za090uK6AgAAAAAAYIHmbAlkrS0bY35L0rcluZI+b63dZ4x5V7T+c5K+KemXJT0taVLSHWeuygAAAAAAADhd8+kOJmvtNxUGPfGyz8XmraTfbG3VVpRzrosblhzHFFqNYwpnAscVWo1jCq3GMYVW45hCq53VY8qE+Q0AAAAAAADOZfMZEwgAAAAAAAArHCHQIhhjbjLGPGWMedoY83tLXR+cG4wxzxhj9hhjHjHGPLjU9cHKY4z5vDHmuDFmb6xslTHmH40xP4umvUtZR6wsMxxTdxpjDkXnqkeMMb+8lHXEymKM2WSMud8Y84QxZp8x5rejcs5VWJBZjinOVVgQY0zWGPNjY8yj0TH1H6JyzlNYsFmOq7N2rqI72AIZY1xJ+yX9S0kDCu+i9hZr7eNLWjGseMaYZyRda60dXOq6YGUyxlwnaVzSl62126Kyj0s6aa390yi07rXWfnAp64mVY4Zj6k5J49baP1vKumFlMsasl7TeWvtTY0ynpIck/WtJbxPnKizALMfUm8S5CgtgjDGS2q2148aYlKQfSPptSW8Q5yks0CzH1U06S+cqWgIt3AslPW2tPWCtLUq6R9LNS1wnAJC1drekkw3FN0v6UjT/JYUfjIF5meGYAhbMWnvEWvvTaH5M0hOSNohzFRZolmMKWBAbGo8WU9HDivMUFmGW4+qsIQRauA2SDsaWB8Q/NGgNK+n/GGMeMsa8c6krg3PGWmvtESn8oCxpzRLXB+eG3zLGPBZ1F6M5PBbEGHOhpKsl/Uicq9ACDceUxLkKC2SMcY0xj0g6LukfrbWcp7BoMxxX0lk6VxECLZxpUkbfOrTCy6y1z5f0akm/GXXDAIDl5rOSLpZ0laQjkj65pLXBimSM6ZD0NUm/Y60dXer6YOVrckxxrsKCWWt9a+1VkjZKeqExZtsSVwnngBmOq7N2riIEWrgBSZtiyxslHV6iuuAcYq09HE2PS/q6wq6HwGIdi8ZLqIybcHyJ64MVzlp7LPoQE0j67+JchdMUjYXwNUl3W2v/JirmXIUFa3ZMca5CK1hrRyTtUjhuC+cptET8uDqb5ypCoIX7iaRLjDGbjTFpSbdK+sYS1wkrnDGmPRrMUMaYdkmvkrR39r2AefmGpNuj+dsl/e0S1gXngMoH4MjrxbkKpyEaGPN/SHrCWvup2CrOVViQmY4pzlVYKGPMamNMTzSfk/QvJD0pzlNYhJmOq7N5ruLuYIsQ3bbt/5XkSvq8tfaPl7ZGWOmMMRcpbP0jSZ6kr3Bc4XQZY/6npJ2S+iUdk/SHkv6XpHslnS/pOUlvtNYy0C/mZYZjaqfCJstW0jOS/p/KGAnAXIwxvyTp+5L2SAqi4g8rHMOFcxVO2yzH1FvEuQoLYIy5UuHAz67CxhP3Wmv/yBjTJ85TWKBZjqu/0lk6VxECAQAAAAAAJADdwQAAAAAAABKAEAgAAAAAACABCIEAAAAAAAASgBAIAAAAAAAgAQiBAAAAAAAAEoAQCAAAAAAAIAEIgQAAAAAAABKAEAgAAAAAACAB/n+yMeqj0N+rkwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1440x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Training Performance\n",
    "\n",
    "history.history.keys()\n",
    "pd.DataFrame(history.history).plot(figsize=(20, 8))\n",
    "plt.grid(True)\n",
    "plt.gca().set_ylim(0, 1) # set the vertical range to [0-1]\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEWCAYAAACZnQc8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjsUlEQVR4nO3dd5xU1R338c8XUFhRQIpKs7dgx4ZGFCUqdo0NJXaDGozm0ZjE8qgxmvIkMfaCBUUFxdhjrKhBE4kiRcVKRKQpgqKCKOzu7/nj3sVhXXZnl52d4e737eu+9s655fxmWX9z5txzz1VEYGZm2dCi2AGYmVnjcVI3M8sQJ3UzswxxUjczyxAndTOzDHFSNzPLECd1W2GSyiQ9JukLSfevwHkGSXq6MWMrBklPSDqh2HFY8+Sk3oxIOlbSOEkLJM1Ok89ujXDqI4C1gU4RcWRDTxIR90TEPo0QzzIk9ZMUkh6sVr5NWv5Cnue5VNLdde0XEftFxJ0NDNdshTipNxOSzgGuAn5PkoDXBW4ADmmE068HvBcR5Y1wrkL5FNhVUqecshOA9xqrAiX8/5QVlf8AmwFJ7YHLgCER8WBELIyIJRHxWEScl+7TWtJVkmaly1WSWqfb+kmaIelcSXPSVv5J6bbfAhcDR6ffAE6p3qKVtH7aIm6Vvj5R0geSvpI0VdKgnPKXco7bVdKrabfOq5J2zdn2gqTfSfp3ep6nJXWu5dewGHgYGJge3xI4Crin2u/qaknTJX0p6TVJfdPyAcAFOe9zUk4cV0j6N/A1sGFadmq6/UZJf885/58kjZakfP/9zOrDSb152AVoAzxUyz4XAn2AbYFtgJ2Ai3K2rwO0B7oDpwDXS1ozIi4haf3fFxGrR8RttQUiqS1wDbBfRKwB7ApMrGG/jsDj6b6dgCuBx6u1tI8FTgLWAlYFfllb3cBw4Ph0fV9gMjCr2j6vkvwOOgIjgPsltYmIJ6u9z21yjjkOGAysAUyrdr5zga3TD6y+JL+7E8Lzc1iBOKk3D52AuXV0jwwCLouIORHxKfBbkmRVZUm6fUlE/BNYAGzWwHgqgS0llUXE7IiYXMM+BwDvR8RdEVEeESOBd4CDcvYZFhHvRcQiYBRJMl6uiPgP0FHSZiTJfXgN+9wdEfPSOv8KtKbu93lHRExOj1lS7XxfAz8h+VC6G/h5RMyo43xmDeak3jzMAzpXdX8sRzeWbWVOS8uWnqPah8LXwOr1DSQiFgJHA6cDsyU9LmnzPOKpiql7zuuPGxDPXcCZwJ7U8M0l7WJ6O+3ymU/y7aS2bh2A6bVtjIhXgA8AkXz4mBWMk3rz8DLwDXBoLfvMIrngWWVdvt81ka+FwGo5r9fJ3RgRT0XE3kBXktb3LXnEUxXTzAbGVOUu4GfAP9NW9FJp98ivSfra14yIDsAXJMkYYHldJrV2pUgaQtLinwX8qsGRm+XBSb0ZiIgvSC5mXi/pUEmrSVpF0n6S/l+620jgIkld0guOF5N0FzTERGB3SeumF2nPr9ogaW1JB6d969+SdONU1HCOfwKbpsMwW0k6GugF/KOBMQEQEVOBPUiuIVS3BlBOMlKmlaSLgXY52z8B1q/PCBdJmwKXk3TBHAf8StK2DYverG5O6s1ERFwJnENy8fNTki6DM0lGhECSeMYBrwNvAOPTsobU9QxwX3qu11g2EbcguXg4C/iMJMH+rIZzzAMOTPedR9LCPTAi5jYkpmrnfikiavoW8hTwBMkwx2kk325yu1aqbqyaJ2l8XfWk3V13A3+KiEkR8T7JCJq7qkYWmTU2+SK8mVl2uKVuZpYhTupmZhnipG5mliFO6mZmGVLbzShFtWTuB76Ca99T1q1vsUOwElS+eOYKz6VTn5yzSucNS3buHrfUzcwypGRb6mZmTaqypnvgVj5O6mZmABWl/DiA/Dmpm5kBEZXFDqFROKmbmQFUOqmbmWWHW+pmZhniC6VmZhnilrqZWXaER7+YmWWIL5SamWWIu1/MzDLEF0rNzDLELXUzswzxhVIzswzJyIVST71rZgZEVOS91EXS7ZLmSHozp+x3kl6XNFHS05K65Ww7X9IUSe9K2jenfHtJb6TbrpFU5zzuTupmZpD0qee71O0OYEC1sj9HxNYRsS3wD+BiAEm9gIHAFukxN0hqmR5zIzAY2CRdqp/ze5zUzcwg6X7Jd6lDRIwBPqtW9mXOy7ZA1ZOWDgHujYhvI2IqMAXYSVJXoF1EvBwRAQwHDq2rbvepm5lBvUa/SBpM0oKuMjQihuZx3BXA8cAXwJ5pcXdgbM5uM9KyJel69fJauaVuZgZQsSTvJSKGRsQOOUudCR0gIi6MiJ7APcCZaXFN/eRRS3mtnNTNzKBRu1/yMAI4PF2fAfTM2dYDmJWW96ihvFZO6mZm0NgXSr9H0iY5Lw8G3knXHwUGSmotaQOSC6KvRMRs4CtJfdJRL8cDj9RVj/vUzcygUcepSxoJ9AM6S5oBXALsL2kzoBKYBpwOEBGTJY0C3gLKgSHx3bjJM0hG0pQBT6RL7XUnF1VLz5K5H5RmYFZUZd36FjsEK0Hli2fWOX67Lt+8eFfeOadN3+NWuL5CcUvdzAyIiiXFDqFROKmbmYEn9DIzy5SMzP3ipG5mBm6pm5llilvqZmYZ4pa6mVmGlPshGWZm2eGWuplZhrhP3cwsQ9xSNzPLELfUzcwyxC11M7MM8egXM7MMKdEZa+vLSd3MDNynbmaWKU7qZmYZ4gulZmYZUlFR9z4rASd1MzNw94uZWaY4qZuZZYj71M3MsiMqPU7dzCw73P1iZpYhHv1iZpYhbqmbmWVIRpJ6i2IH0Fxd9Psr2f2AgRz6k9OXll07dDiHHX8Gh58whJ/+4gLmfDpvmWNmfzyHHX90GMNG/H1p2ZIlS7j0T1dzwMBTOeiYn/LM8y812Xuwwjr7rJ8yaeJzTJwwmrvvup7WrVtz+OEHMmnicyz+Zjrb99566b4/6t+X/459ggnjn+W/Y59gz34/LGLkK6mI/JcS5qReJIfuvzc3XXn5MmUnDTqch4bfyAN3Xs8eP9yZG4eNWGb7n64ZSt8+OyxTdvOd99JxzQ48fu+tPHLPzeyw3VYFj90Kr1u3dThzyMns3Gd/tt2uPy1btuToow5h8uR3OPKon/Lii2OX2X/uvM849LAT2a73jzj5lF9wx7CrixT5SqyyMv+lhLn7pUh22HYrZs7+ZJmy1du2Xbq+aNE3SN9tGz3mP/Totg5lZW2WOeahx5/msRG3ANCiRQvW7NC+cEFbk2rVqhVlZW1YsmQJq5WVMXv2x7zzzpQa9504cfLS9cmT36VNmzasuuqqLF68uKnCXfl5SOPySepd2/aIGF+IerPg6pvv4NEnR7NG27bcfu0fAfh60Tfcfvf93HLV7xk28oGl+3751QIArrtlOK9OeJ2e3btywTk/o3PHNYsSuzWeWbM+5sq/3cTU/73CokXf8Myz/+KZZ8fkdeyPf3wAEye+6YReXxkZ/VKo7pe/1rL8ZXkHSRosaZykcbcOH1mg0Erb2aedyOiH7uKAffZkxAOPAXD9bXdx3NGHsdpqZcvsW1FRwSdz5rLdVr24f9h1bLPlD/jLdbcWI2xrZB06tOfgg/Zl40370HO93rRtuxrHHvvjOo/r1WtT/nDFBZwx5NdNEGW2RGVl3kspK0hLPSL2bOBxQ4GhAEvmfpCN70INdMA+/fjZLy/hzFOP443J7/LM8y9x5Q238dWChUii9aqrcszhB1HWpjX999gVgH327MuDjz1V5MitMfTv35epH37E3LmfAfDQw0+wS58dGDHiweUe0717V/5+/22cdPLZfPDBtKYKNTvc/bJ8kmptUkTE8v8ym7Fp02eyXs/uADz/4lg2WK8HAMNv/O7LzfW33c1qZW049oiDAdjjhzvz6oTX2Xn7bfnvuIlstMG6TR+4NbrpH81k5517U1bWhkWLvmGvPXfjtdcmLXf/9u3b8egjw7nwoj/wn5fHNWGkGdKIc79Iuh04EJgTEVumZX8GDgIWA/8DToqI+em284FTgArgrIh4Ki3fHrgDKAP+CZwdUfvwG9WxvaFvaFgtmyMiTq7rHFlvqZ93yR95dcLrzJ//JZ06duBnpxzHiy+/yocfzUAtRLd11uLi837O2l06L3NcVVI/6dgjAJj18Secf9lf+HLBAjp2aM/lF5xD13XWKsZbahJl3foWO4Qmc8nF53LkkQdTXl7OxImTGXzaL9lvv724+m+X06VLR+bP/5JJkyaz/4GDuOD8s/n1r87k/SlTlx6/3/7H8Gm1YbFZVb54pureq3YLLxuUd85pe/E9tdYnaXdgATA8J6nvAzwXEeWS/gQQEb+W1AsYCewEdAOeBTaNiApJrwBnA2NJkvo1EfFErXUXIqk3hqwndWuY5pTULX+NktQvHph/Ur/s3jrrk7Q+8I+qpF5t22HAERExKG2lExF/SLc9BVwKfAg8HxGbp+XHAP0i4rTa6i3oOHVJa0u6TdIT6etekk4pZJ1mZg0SlXkvuYM60mVwPWs7GahqcXcHpudsm5GWdU/Xq5fXqtDj1O8AhgEXpq/fA+4DbitwvWZm9VOPC6W5gzrqS9KFQDlwT1VRTVXUUl6rQt9R2jkiRgGVABFRTnIhwMyspDTFkEZJJ5BcQB2Uc8FzBtAzZ7cewKy0vEcN5bUqdFJfKKkT6aeLpD7AFwWu08ys/ioj/6UBJA0Afg0cHBFf52x6FBgoqbWkDYBNgFciYjbwlaQ+kgQcDzxSVz2F7n45hyTgjST9G+gCHFHgOs3M6q8Rx6lLGgn0AzpLmgFcApwPtAaeSXI0YyPi9IiYLGkU8BZJt8yQiKjq0TiD74Y0PsF3/fDLVahx6jsC0yNivKQ9gNOAw4GnWbbj38ysNDTiNAERcUwNxcu9lhgRVwBX1FA+Dvje6JnaFKr75WaSAfYAu5JcKL0e+JwGXlwwMyukqIy8l1JWqO6XlhHxWbp+NDA0Ih4AHpA0sUB1mpk1XIkn63wVqqXeUlLVB0Z/4LmcbZ7u18xKj+dTr9VI4F+S5gKLgBcBJG2MR7+YWSnKSEu9ULM0XiFpNNAVeDpnPGYL4OeFqNPMbIU4qdcuIsbWUPZeoeozM1sRUVHa3Sr5cv+2mRm4pW5mliWlPlQxX07qZmbglrqZWaZko0vdSd3MDCDKs5HVndTNzMAtdTOzLPGFUjOzLHFL3cwsO9xSNzPLErfUzcyyI8qLHUHjcFI3MwMiIy31es2nLmlNSVsXKhgzs6KprMdSwupM6pJekNROUkdgEjBM0pWFD83MrOlEZf5LKcunpd4+Ir4EfgwMi4jtgR8VNiwzs6aVlaSeT596K0ldgaNIHiBtZpY5UaFih9Ao8knqlwFPAS9FxKuSNgTeL2xYZmZNq9Rb4PmqM6lHxP3A/TmvPwAOL2RQZmZNLSoz3lKXdC2w3FusIuKsgkRkZlYEzaGlPq7JojAzK7KIjLfUI+LO3NeS2kbEwsKHZGbW9LLSUs9nnPoukt4C3k5fbyPphoJHZmbWhCorlPdSyvIZp34VsC8wDyAiJgG7FzAmM7MmF5XKeyllec39EhHTpWXeSEVhwjEzK45ST9b5yiepT5e0KxCSVgXOIu2KMTPLisjGdOp5db+cDgwBugMzgW3T12ZmmdGY3S+Sbpc0R9KbOWVHSposqVLSDtX2P1/SFEnvSto3p3x7SW+k265RtS6TmtSZ1CNibkQMioi1I6JLRPwkIubV+a7MzFYiEcp7ycMdwIBqZW+SzKE1JrdQUi9gILBFeswNklqmm28EBgObpEv1c35PPqNfNpT0mKRP00+eR9KpAszMMqOiQnkvdYmIMcBn1crejoh3a9j9EODeiPg2IqYCU4Cd0jm32kXEyxERwHDg0Lrqzqf7ZQQwCugKdCOZMmBkHseZma00GrmlXh/dgek5r2ekZd3T9erltconqSsi7oqI8nS5m1qmDzAzWxnVp09d0mBJ43KWwStQdU2fElFLea1qm/ulY7r6vKTfAPemJzwaeLzuOM3MVh71Gf0SEUOBoY1U9QygZ87rHsCstLxHDeW1qm1I42ss+2lxWs62AH6XR7BmZiuFIo5TfxQYkT5RrhvJBdFXIqJC0leS+gD/BY4Hrq3rZLXN/bJBIwVsZlbyKirr9cjmWkkaCfQDOkuaAVxCcuH0WqAL8LikiRGxb0RMljQKeAsoB4ZERNUNnmeQjKQpA55Il9rrjjy+c0jaEugFtKkqi4jh+b7Bhlgy9wP329v3lHXrW+wQrASVL565ws3s19c/KO+cs/WHj5Xs7ad13lEq6RKST5xewD+B/YCXSIbXmJllQmVGpt7N5/vGEUB/4OOIOAnYBmhd0KjMzJpYEYc0Nqp85n5ZFBGVksoltQPmAL75yMwyJStzv+ST1MdJ6gDcQjIiZgHwSiGDAigf+3Chq7CV0OFddyx2CJZRWel+yefB0z9LV2+S9CTJbauvFzYsM7Om1ZijX4qptpuPete2LSLGFyYkM7Oml5Hel1pb6n+tZVsAezVyLGZmRZP57peI2LMpAzEzK6ZSH9WSr7weZ2dmlnWVxQ6gkTipm5kBUeOkiCsfJ3UzM6A8I90v+Tz5SJJ+Iuni9PW6knYqfGhmZk0nUN5LKctnYOYNwC7AMenrr4DrCxaRmVkRVNZjKWX5dL/sHBG9JU0AiIjPJa1a4LjMzJpUqbfA85VPUl+SPtk6ACR1ofQ/rMzM6iUrSS2fpH4N8BCwlqQrSGZtvKigUZmZNbGK5tJSj4h7JL1GMv2ugEMj4u2CR2Zm1oSK9zS7xpXPQzLWBb4GHssti4iPChmYmVlTqmwuLXXgcb57AHUbYAPgXWCLAsZlZtakmsOEXgBExFa5r9PZG08rWERmZkXQnC6ULiMixkvykwrMLFMq1Uy6XySdk/OyBdAb+LRgEZmZFUFFsQNoJPm01NfIWS8n6WN/oDDhmJkVR7MY/ZLedLR6RJzXRPGYmRVF5ke/SGoVEeW1PdbOzCwrmsPol1dI+s8nSnoUuB9YWLUxIh4scGxmZk2mWXS/pDoC80ieSVo1Xj0AJ3Uzy4zmMKRxrXTky5t8l8yrZOWbipkZABXNoKXeElgdarx64KRuZpnSHFrqsyPisiaLxMysiJpDUs/IlxEzs7pl5BGltSb1/k0WhZlZkWWlpb7cZ5RGxGdNGYiZWTFV1GOpi6TbJc2R9GZOWUdJz0h6P/25Zs628yVNkfSupH1zyreX9Ea67Rqp7glq8nnwtJlZ5lUq/yUPdwADqpX9BhgdEZsAo9PXSOoFDCSZznwAcEN6Nz/AjcBgYJN0qX7O73FSNzMj6X7Jd6lLRIwBqvd2HALcma7fCRyaU35vRHwbEVOBKcBOkroC7SLi5YgIYHjOMcvlpG5mRv2SuqTBksblLIPzqGLtiJgNkP5cKy3vDkzP2W9GWtY9Xa9eXqt6z6duZpZF9bn5JiKGAkMbqerl3QvUoHuEnNTNzGiSuV8+kdQ1ImanXStz0vIZQM+c/XoAs9LyHjWU18rdL2ZmNO7ol+V4FDghXT8BeCSnfKCk1pI2ILkg+kraRfOVpD7pqJfjc45ZLrfUzcyAykac/UTSSKAf0FnSDOAS4I/AKEmnAB8BRwJExGRJo4C3SB5ENCQiqj47ziAZSVMGPJEutXJSNzOjcW8+iohjlrOpxps6I+IK4IoayscBW9anbid1MzOyM0uhk7qZGdmZJsBJ3cwMKFc22upO6mZmuPvFzCxT3P1iZpYhjTmksZic1M3McPeLmVmmuPvFzCxDKjLSVndSNzPDLXUzs0wJt9TNzLLDLXVbIZfc+wJj3p5Gx9XLeOC8owC48rGXGTP5I1Zp1YIendrx24H9aFfWGoD3Zs3j8r+PYcE3S2ghcc8vDqO8opKTrn906TnnzF/I/ttvzK8O/WFR3pM1rv1OOpD+x+wNEs+NfIZ/3v4YR517LDvsvRNRGXwx7wtuPPdqPp/zOat3WINzbvoVG229MS/8/TmGXXxLscNf6XhIo62Qg3fclIG7bcFFI59fWtZn0x6ctf/OtGrZgqv+MZbbR0/gFwf2obyikgtHPMflx+7FZt06MX/hN7Rq2YLWq7Ri1LlHLD3+mL89QP+tNijG27FG1nPTdel/zN5ccPB5lC8p54LhlzD+uXE8dvNDjPrrCAAGnHgAh599NLdeeBNLvl3MfX8ZQc/N1qXnZusWOfqVUzZSuh+SUTTbb9SNdqu1WaZs18160qpl8k+y9Xpr88n8hQC8/N4MNunakc26dQKgQ9s2tGyx7D/dtE+/4LOvFtF7w65NEL0VWveNe/D+hPdY/M1iKisqeeu/k9lp3z4sWrBo6T5tVmtD8jxi+HbRt7w77m2WfLukWCGv9MqJvJdSVtCkLukBSQdI8odHPT38yjvs9oPkCVfTPp2PJM64+XEGXvkAw56b+L39n5wwhX233YjkASm2spv+3kdsvlMvVu+wBqu2WZXt9uxNp26dATj6vEFc//Kt7Hbo7oy6cmSRI82OqMd/pazQyfZG4FjgfUl/lLR5bTvnPqH7tidfLnBopeuWZ8fTskUL9u+9CQAVFcGEqR/z+0F7MezMg3n+zan8970Zyxzz1MQpDNhu42KEawUwc8oMHr3pIS6651IuGH4J0976kIry5GE49/35HobsciovPTyGASfsX+RIs6OyHkspK2hSj4hnI2IQ0Bv4EHhG0n8knSRplRr2HxoRO0TEDqcM2KWQoZWsR199lxffmsbvB+21tNW9doe2bL9hV9ZcvYyyVVdhtx+sy9sz5y495t1Z8yivCHr17FKssK0Anr/vWX5zwLlcetSFLJi/gI8/nL3M9pceGcPO+zXP/08KwS31PEnqBJwInApMAK4mSfLPFLrulc2/3/mIO56fyFUnD6Bs1e8+83bdrCfvz/6MRYuXUF5RyWv/m82Ga6+5dPuT46cwYLuNihGyFVC7Tu0B6NStMzsN6MO/HxnDOut/d81kh713Yub/ZhYrvMzJSku9oKNfJD0IbA7cBRyUPh0b4D5J4wpZd6n7zV3PMu5/s5m/8Bv2uexuzth3B24fPYHF5RWcfvPjAGy93lpcdMTutFutNcftsRWDrnoICXbbfF1277Xe0nM9Pel/XHfqfsV6K1Yg59z0a9ZYcw0qlpRz+8VDWfjlQk77f2fSbcNuVFYGc2d+yi0X3Lh0/2tfGspqa5TRapVW7LjPzlxx3KXMfH9GLTVYrooo7RZ4vhQFfCOS9oqI5xpy7KJ/XJmN37A1qhOHjCl2CFaC7pv28AqPEDh2vcPyzjkjpj1UsiMSCj1OvYek46sXRsTwAtdrZlYvpd5Xnq9CJ/Udc9bbAP2B8YCTupmVlFLvK89XQZN6RPw897Wk9iT962ZmJcXTBDTM18AmTVynmVmd3P2SB0mP8d2UCi2AXsCoQtZpZtYQWRn9UuiW+l9y1suBaRHhMVZmVnLc/ZKHiPhX1bqkzsC8QtZnZtZQWblQWpA7SiX1kfSCpAclbSfpTeBN4BNJAwpRp5nZisjKNAGFaqlfB1wAtAeeA/aLiLHphF4jgScLVK+ZWYO4+6WO80bE0wCSLouIsQAR8Y6nhjWzUlTIu+ubUqGSem731KJq27LxmzOzTKnISGoq1CyN20j6UtJXwNbpetXrrQpUp5lZg1USeS91kXS2pDclTZb0i7Sso6RnJL2f/lwzZ//zJU2R9K6kfVfkfRQkqUdEy4hoFxFrRESrdL3q9ffmUTczK7aIyHupjaQtgZ8COwHbAAdK2gT4DTA6IjYBRqevkdQLGAhsAQwAbpDUsqHvw4+ZMzOjUVvqPwDGRsTXEVEO/As4DDgEuDPd507g0HT9EODeiPg2IqYCU0g+EBrESd3MjPoNacx99Ga6DM451ZvA7pI6SVoN2B/oCaxd9UyJ9Oda6f7dgek5x89Iyxqkqed+MTMrSfWZJiAihgJDl7PtbUl/Inm62wJgEskd9ctT05DABl+1dUvdzIzGvVAaEbdFRO+I2B34DHif5ObLrgDpzznp7jNIWvJVegCzGvo+nNTNzGj00S9rpT/XBX5MctPlo8AJ6S4nAI+k648CAyW1lrQByUy2rzT0fbj7xcyMRr/56AFJnYAlwJCI+FzSH4FRkk4BPgKOTOudLGkU8BZJN82QiKhoaMVO6mZmNO40ARHRt4ayeSRPf6tp/yuAKxqjbid1MzP8kAwzs0ypiGxMvuukbmaGJ/QyM8sUT71rZpYh7lM3M8uQSne/mJllh1vqZmYZ4tEvZmYZ4u4XM7MMcfeLmVmGuKVuZpYhbqmbmWVIRcMnRiwpTupmZniaADOzTPE0AWZmGeKWuplZhnj0i5lZhnj0i5lZhniaADOzDHGfuplZhrhP3cwsQ9xSNzPLEI9TNzPLELfUzcwyxKNfzMwyxBdKzcwyxN0vZmYZ4jtKzcwyxC11M7MMyUqfurLy6ZRlkgZHxNBix2GlxX8XVpMWxQ7A8jK42AFYSfLfhX2Pk7qZWYY4qZuZZYiT+srB/aZWE/9d2Pf4QqmZWYa4pW5mliFO6mZmGeKkXiIkXShpsqTXJU2UtHMt+94h6Yh0/QVJOzRdpNZUJFWkfwuTJI2XtGuxY7LS5ztKS4CkXYADgd4R8a2kzsCqRQ7Lim9RRGwLIGlf4A/AHkWNyEqeW+qloSswNyK+BYiIuRExS9L2kv4l6TVJT0nqWuQ4rXjaAZ8DSOon6R9VGyRdJ+lESf0lPZRTvrekB4sQqxWRk3ppeBroKek9STdI2kPSKsC1wBERsT1wO3BFUaO0plaWdr+8A9wK/K6O/Z8DfiCpS/r6JGBYIQO00uPulxIQEQskbQ/0BfYE7gMuB7YEnpEE0BKYXbQgrRhyu192AYZL2nJ5O0dESLoL+ImkYcAuwPFNEqmVDCf1EhERFcALwAuS3gCGAJMjYpeiBmYlISJeTq+1dAHKWfZbdpuc9WHAY8A3wP0RUd50UVopcPdLCZC0maRNcoq2Bd4GuqQtNCStImmLYsRnxSdpc5Jva/OAaUAvSa0ltQf6V+0XEbOAWcBFwB1FCNWKzC310rA6cK2kDiStsCkkM/ANBa5J/8dtBVwFTC5SjNb0yiRNTNcFnJB+o5suaRTwOvA+MKHacfcAXSLirSaL1EqGpwkwyxhJ1wETIuK2YsdiTc9J3SxDJL0GLAT2rhoia82Lk7qZWYb4QqmZWYY4qZuZZYiTuplZhjip2/fkzA74pqT7Ja22AufKnVHyVkm9atm3X0NmIpT0YXpjTl7l1fZZUM+6LpX0y/rGaNZUnNStJosiYtuI2BJYDJyeu1FSy4acNCJOrWPsdD/A08uarQAndavLi8DGaSv6eUkjgDcktZT0Z0mvpnPAnwagxHWS3pL0OLBW1Yly536XNCCdI3ySpNGS1if58Pg/6beEvpK6SHogreNVST9Mj+0k6WlJEyTdTHJjTq0kPZzOdjlZ0uBq2/6axjK6ajIsSRtJejI95sX0js7q5zwrfZ+vS7q3gb9fs0blO0ptuSS1AvYDnkyLdgK2jIipaWL8IiJ2lNQa+Lekp4HtgM2ArYC1gbdIZpjMPW8X4BZg9/RcHSPiM0k3AQsi4i/pfiOAv0XES5LWBZ4CfgBcArwUEZdJOoDk7tu6nJzWUQa8KumBiJgHtAXGR8S5ki5Oz30myd28p0fE+0oeWHIDsFe1c/4G2CCdA79DPr9Ts0JzUrea5N6e/iJwG0m3yCsRMTUt3wfYuqq/HGgPbALsDoxMb2efJem5Gs7fBxhTda6I+Gw5cfyIZI6TqtftJK2R1vHj9NjHJX2ex3s6S9Jh6XrPNNZ5QCXJrJgAdwMPSlo9fb/359TduoZzvg7cI+lh4OE8YjArOCd1q8nSKV+rpMltYW4R8POIeKrafvsDdd3Rpjz2gaR7cJeIWFRDLHnfNSepH8kHxC4R8bWkF1h2ZsNckdY7v/rvoAYHkHzAHAz8X0lbeFZEKzb3qVtDPQWckT7MA0mbSmoLjAEGpn3uXUnmh6/uZWAPSRukx3ZMy78C1sjZ72mSrhDS/bZNV8cAg9Ky/YA164i1PfB5mtA3J/mmUKUFUPVt41iSbp0vgamSjkzrkKRtck8oqQXQMyKeB34FdCCZmM2sqNxSt4a6FVgfGK+k6fwpcCjwEEnf8xvAe8C/qh8YEZ+mffIPpslxDrA3yTzgf5d0CPBz4Czgekmvk/ytjiG5mPpbYKSk8en5P6oj1ieB09PzvAuMzdm2ENginTPlC+DotHwQcKOki4BVgHuBSTnHtQTuVjKDpkj6/ufXEYdZwXnuFzOzDHH3i5lZhjipm5lliJO6mVmGOKmbmWWIk7qZWYY4qZuZZYiTuplZhvx/DBvch8KR5WoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "\n",
    "# # Performance\n",
    "y_pred=cnn_model.predict(X_test) # Returns a list where every element is an array with the continuous\n",
    "# value prediction\n",
    "y_pred=[x[0] for x in y_pred] # Getting all the continuous value predictions from the arrays\n",
    "for i in range(len(y_pred)): # Transforming the continuous value into binary labels\n",
    "    if y_pred[i] <= 0.5:\n",
    "        y_pred[i] = 0\n",
    "    if y_pred[i] > 0.5:\n",
    "        y_pred[i] = 1\n",
    "\n",
    "# Plotting Confusion Matrix\n",
    "cm=confusion_matrix(y_test, y_pred)\n",
    "ax= plt.subplot()\n",
    "sns.heatmap(cm, annot=True, fmt='g', ax=ax);  #annot=True to annotate cells, ftm='g' to disable scientific notation\n",
    "\n",
    "# labels, title and ticks\n",
    "ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels'); \n",
    "ax.set_title('Confusion Matrix'); \n",
    "ax.xaxis.set_ticklabels(['Sell', 'Buy']); ax.yaxis.set_ticklabels(['Sell', 'Buy']);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Saving and Reloading the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Saving the model\n",
    "\n",
    "cnn_model.save('Models/CNN 24_4 BTC.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reloading the model\n",
    "\n",
    "cnn_model_BTC=model = keras.models.load_model('Models/CNN 1H BTC.h5')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.11"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "179.594px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}