Deep_learning_with_keras.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyOqnBHfDClQElc5GzLpT7IT",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Shaghayegh-Aflatounian/Churn-Prediction/blob/main/Deep_learning_with_keras.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5pe52pVZposV",
        "outputId": "c25c4a1c-f9c4-4993-acc3-d399ee00f355"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense (Dense)               (None, 10)                30        \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 1)                 11        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 41 (164.00 Byte)\n",
            "Trainable params: 41 (164.00 Byte)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ],
      "source": [
        "# Import the Sequential model and Dense layer\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "\n",
        "# Create a Sequential model\n",
        "model = Sequential()\n",
        "\n",
        "# Add an input layer and a hidden layer with 10 neurons\n",
        "model.add(Dense(10, input_shape=(2,), activation=\"relu\"))\n",
        "\n",
        "# Add a 1-neuron output layer\n",
        "model.add(Dense(1))\n",
        "\n",
        "# Summarise your model\n",
        "model.summary()"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "## we have 3 layers with 50 neurons?do we have 3 layers with 50 neurons\n",
        "\n",
        "# Instantiate a Sequential model\n",
        "model = Sequential()\n",
        "\n",
        "# Add a Dense layer with 50 neurons and an input of 1 neuron\n",
        "model.add(Dense(50, input_shape=(1,), activation='relu'))\n",
        "\n",
        "# Add two Dense layers with 50 neurons and relu activation\n",
        "model.add(Dense(50, activation='relu'))\n",
        "model.add(Dense(50, activation='relu'))\n",
        "\n",
        "# End your model with a Dense layer and no activation\n",
        "model.add(Dense(1))"
      ],
      "metadata": {
        "id": "wzQjV_g4q2KO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Compile your model\n",
        "model.compile(optimizer = 'adam', loss = 'mse')\n",
        "\n",
        "print(\"Training started..., this can take a while:\")\n",
        "\n",
        "# Fit your model on your data for 30 epochs\n",
        "model.fit(time_steps, y_positions, epochs = 30)\n",
        "\n",
        "# Evaluate your model\n",
        "print(\"Final loss value:\",model.evaluate(time_steps, y_positions))"
      ],
      "metadata": {
        "id": "bQdsb_BMprQe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Remember np.arange(x,y) produces a range of values from x to y-1. That is the [x, y) interval."
      ],
      "metadata": {
        "id": "mlSZteA5s_ms"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Predict the twenty minutes orbit\n",
        "twenty_min_orbit = model.predict(np.arange(-10, 11))\n",
        "# Plot the twenty minute orbit\n",
        "plot_orbit(twenty_min_orbit)\n"
      ],
      "metadata": {
        "id": "gNJFMcZys8Ww"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Different classification models**"
      ],
      "metadata": {
        "id": "g3t9ceIrDWJC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "you will know how to solve binary, multi-class, and multi-label problems with neural networks. All of this by solving problems like detecting fake dollar bills, deciding who threw which dart at a board, and building an intelligent system to water your farm."
      ],
      "metadata": {
        "id": "BgJHjgmBDVVI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Binary Classification**"
      ],
      "metadata": {
        "id": "G6T6uRLC4cfl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "**Banknote detection**"
      ],
      "metadata": {
        "id": "aELDxvcSyUpw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "You will practice building classification models in Keras with the Banknote Authentication dataset.\n",
        "\n",
        "Your goal is to distinguish between real and fake dollar bills. In order to do this, the dataset comes with 4 features: variance,skewness,kurtosis and entropy. These features are calculated by applying mathematical operations over the dollar bill images. The labels are found in the dataframe's class column.You will perform binary classification by using a single neuron as an output. The input layer will have 4 neurons since we have 4 features in our dataset. The model's output will be a value constrained between 0 and 1."
      ],
      "metadata": {
        "id": "mMN14rfeC18P"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "\n",
        "# Load the dataset\n",
        "banknotes = pd.read_csv('your_dataset.csv')\n",
        "\n",
        "# Plot the pairplot\n",
        "sns.pairplot(banknotes, hue='class')\n",
        "plt.show()\n",
        "\n",
        "# Describe the data\n",
        "print('Dataset stats: \\n', banknotes.describe())\n",
        "\n",
        "# Count the number of observations per class\n",
        "print('Observations per class: \\n', banknotes['class'].value_counts())\n",
        "\n",
        "# Prepare your data for training\n",
        "X = banknotes.drop('class', axis=1)\n",
        "y = banknotes['class']\n",
        "\n",
        "# Create a sequential model\n",
        "model = Sequential()\n",
        "\n",
        "# Add a dense layer\n",
        "model.add(Dense(1, input_shape=(4,), activation='sigmoid'))\n",
        "\n",
        "# Compile your model\n",
        "model.compile(loss='binary_crossentropy', optimizer='sgd', metrics=['accuracy'])\n",
        "\n",
        "# Fit the model on your data\n",
        "model.fit(X, y, epochs=10, batch_size=32)\n",
        "\n",
        "# Evaluate the model\n",
        "loss, accuracy = model.evaluate(X, y)\n",
        "print('Loss:', loss)\n",
        "print('Accuracy:', accuracy)"
      ],
      "metadata": {
        "id": "RNMQsU1f3Nfn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Train your model for 20 epochs\n",
        "model.fit(X_train, y_train, epochs = 20)\n",
        "\n",
        "# Evaluate your model accuracy on the test set\n",
        "accuracy = model.evaluate(X_test, y_test)[1]\n",
        "\n",
        "# Print accuracy\n",
        "print('Accuracy', accuracy)"
      ],
      "metadata": {
        "id": "aTMzyDZF5L7n"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Multi classification**"
      ],
      "metadata": {
        "id": "O9x2CVbJ4YWG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "we're going to build a model that predicts who threw which dart only based on where that dart landed! (That is the dart's x and y coordinates on the board.)\n",
        "\n",
        "This problem is **a multi-class classification problem since each dart can only be thrown by one of 4 competitors.** So classes/labels are mutually exclusive, and therefore we can build a neuron with as many output as competitors and use the **softmax activation function to achieve a total sum of probabilities of 1 over all competitors.**\n",
        "\n",
        "The Sequential model and Dense layers are already imported for you to use."
      ],
      "metadata": {
        "id": "QiMLSSSo_YwS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "why ouer 2 input? because we are importing the x and y location of each dart\n",
        "why we have 4 output? because we have 4competitors\n"
      ],
      "metadata": {
        "id": "mz53Cnr1GKHs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Instantiate a sequential model\n",
        "model = Sequential()\n",
        "# Add 3 dense layers of 128, 64 and 32 neurons each\n",
        "model.add(Dense(128, input_shape=(2,), activation='relu'))\n",
        "model.add(Dense(64, activation='relu'))\n",
        "model.add(Dense(32, activation='relu'))\n",
        "\n",
        "# Add a dense layer with as many neurons as competitors\n",
        "model.add(Dense(4, activation='softmax'))\n",
        "# Compile your model using categorical_crossentropy loss\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer='adam',\n",
        "              metrics=['accuracy'])"
      ],
      "metadata": {
        "id": "PrSKFx924hF7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Categorical to numerical**"
      ],
      "metadata": {
        "id": "9q1BfC3NAbM-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In the console you can check** that your labels, darts.competitor are not yet in a format to be understood by your network.** They contain the names of the competitors as strings. You will first turn these competitors into unique numbers,then use the to_categorical() function from keras.utils to turn these numbers into their one-hot encoded representation."
      ],
      "metadata": {
        "id": "gqFIZBF46wjP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Transform into a categorical variable\n",
        "darts.competitor = pd.Categorical(darts.competitor)\n",
        "\n",
        "# Assign a number to each category (label encoding)\n",
        "darts.competitor = darts.competitor.cat.codes\n",
        "\n",
        "# Import to_categorical from keras utils module\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "\n",
        "coordinates = darts.drop(['competitor'], axis=1)\n",
        "# Use to_categorical on your labels\n",
        "competitors = to_categorical(darts.competitor)\n",
        "# Now print the one-hot encoded labels\n",
        "print('One-hot encoded competitors: \\n',competitors)"
      ],
      "metadata": {
        "id": "euahryTrKJSK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Each competitor is now a vector of length 4, full of zeroes except for the position representing her or himself."
      ],
      "metadata": {
        "id": "ChDnPS4w8jiS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The pd.Categorical() function is applied to the “competitor” column.\n",
        "By doing this, the values in the “competitor” column are treated as distinct categories rather than continuous data.\n",
        "This transformation is useful when dealing with non-numeric data, such as names or labels."
      ],
      "metadata": {
        "id": "ZP20Knfy8TK_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Fit your model to the training data for 200 epochs\n",
        "model.fit(coord_train,competitors_train,epochs=200)\n",
        "\n",
        "# Evaluate your model accuracy on the test data\n",
        "accuracy = model.evaluate(coord_test, competitors_test)[1]\n",
        "\n",
        "# Print accuracy\n",
        "print('Accuracy:', accuracy)"
      ],
      "metadata": {
        "id": "2ra7muG19JYJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        " As you've seen you can easily interpret the softmax output. This can also help you spot those observations where your network is less certain on which class to predict, since you can see the probability distribution among classes per prediction. Let's learn how to solve new problems with neural networks!*italicized text*"
      ],
      "metadata": {
        "id": "H1sTUhJyhsbD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Predict on coords_small_test\n",
        "preds = model.predict(coords_small_test)\n",
        "\n",
        "# Print preds vs true values\n",
        "print(\"{:45} | {}\".format('Raw Model Predictions','True labels'))\n",
        "for i,pred in enumerate(preds):\n",
        "  print(\"{} | {}\".format(pred,competitors_small_test[i]))\n",
        "\n",
        "# Extract the position of highest probability from each pred vector\n",
        "preds_chosen = [np.argmax(pred) for pred in preds]\n",
        "# Print preds vs true values\n",
        "print(\"{:10} | {}\".format('Rounded Model Predictions','True labels'))\n",
        "for i,pred in enumerate(preds_chosen):\n",
        "  print(\"{:25} | {}\".format(pred,competitors_small_test[i]))"
      ],
      "metadata": {
        "id": "dO7UgwhSgLVn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Multi-label**\n"
      ],
      "metadata": {
        "id": "y16yREUn6enq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "You're going to automate the watering of farm parcels by making an intelligent irrigation machine. Multi-label classification problems differ from multi-class problems in that each observation can be labeled with zero or more classes. So classes/labels are not mutually exclusive, you could water all, none or any combination of farm parcels based on the inputs.\n",
        "\n",
        "To account for this behavior what we do is have an output layer with as many neurons as classes but this time, unlike in multi-class problems, each output neuron has a sigmoid activation function. This makes each neuron in the output layer able to output a number between 0 and 1 independently."
      ],
      "metadata": {
        "id": "YqihC_Xa6def"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Instantiate a Sequential model\n",
        "model = Sequential()\n",
        "\n",
        "# Add a hidden layer of 64 neurons and a 20 neuron's input\n",
        "model.add(Dense(64, input_shape=(20,), activation=\"relu\"))\n",
        "\n",
        "\n",
        "# Add an output layer of 3 neurons with sigmoid activation\n",
        "model.add(Dense(3, activation='sigmoid'))\n",
        "# Compile your model with binary crossentropy loss\n",
        "model.compile(optimizer='adam',\n",
        "           loss = 'binary_crossentropy',\n",
        "           metrics=['accuracy'])\n",
        "\n",
        "model.summary()"
      ],
      "metadata": {
        "id": "sSFpSp-YgLN6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Train for 100 epochs using a validation split of 0.2\n",
        "model.fit(sensors_train, parcels_train, epochs = 100, validation_split = 0.2)\n",
        "\n",
        "# Predict on sensors_test and round up the predictions\n",
        "preds = model.predict(sensors_test)\n",
        "preds_rounded = np.round(preds)\n",
        "\n",
        "# Print rounded preds\n",
        "print('Rounded Predictions: \\n', preds_rounded)\n",
        "\n",
        "# Evaluate your model's accuracy on the test data\n",
        "accuracy = model.evaluate(sensors_test,parcels_test)[1]\n",
        "\n",
        "# Print accuracy\n",
        "print('Accuracy:', accuracy)"
      ],
      "metadata": {
        "id": "zmfZZfdsAon3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**FULL CODE FOR SETTING THE RATIO OF TRAINING AND TEST SIZE**"
      ],
      "metadata": {
        "id": "5QFC9CHObpFp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "from sklearn.model_selection import train_test_split\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras.callbacks import EarlyStopping\n",
        "\n",
        "# Define the train sizes\n",
        "train_sizes = [0.6, 0.7, 0.8]\n",
        "\n",
        "# Load your dataset (X_train, y_train) and perform any necessary preprocessing\n",
        "\n",
        "# Define your Keras model\n",
        "model = keras.models.Sequential()\n",
        "# Add your layers\n",
        "\n",
        "# Get the initial weights of the model\n",
        "initial_weights = model.get_weights()\n",
        "\n",
        "# Lists for storing accuracies\n",
        "train_accs = []\n",
        "test_accs = []\n",
        "\n",
        "# Iterate over train sizes\n",
        "for train_size in train_sizes:\n",
        "    # Split the data into training and test sets\n",
        "    X_train_frac, X_test, y_train_frac, y_test = train_test_split(X_train, y_train, train_size=train_size)\n",
        "\n",
        "    # Set the initial weights for the model\n",
        "    model.set_weights(initial_weights)\n",
        "\n",
        "    # Fit the model on the training set fraction\n",
        "    model.fit(X_train_frac, y_train_frac, epochs=100, verbose=0, callbacks=[EarlyStopping(monitor='loss', patience=1)])\n",
        "\n",
        "    # Get the accuracy for this training set fraction\n",
        "    train_acc = model.evaluate(X_train_frac, y_train_frac, verbose=0)[1]\n",
        "    train_accs.append(train_acc)\n",
        "\n",
        "    # Get the accuracy on the whole test set\n",
        "    test_acc = model.evaluate(X_test, y_test, verbose=0)[1]\n",
        "    test_accs.append(test_acc)\n",
        "\n",
        "    print(\"Done with size:\", train_size)\n",
        "\n",
        "# Print the accuracies for each train size\n",
        "print(\"Train accuracies:\", train_accs)\n",
        "print(\"Test accuracies:\", test_accs)"
      ],
      "metadata": {
        "id": "qpKtW-7-boA0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        " **Call Back in *keras**"
      ],
      "metadata": {
        "id": "UIVs7cLiBX7j"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        " These graphs are really useful for detecting overfitting and to know if your neural network would benefit from more training data."
      ],
      "metadata": {
        "id": "Z6EgYdWcVN39"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Train your model and save its history\n",
        "h_callback = model.fit(X_train, y_train, epochs = 25,\n",
        "               validation_data=(X_test, y_test))\n",
        "\n",
        "plot_loss(h_callback.history['loss'], h_callback.history['val_loss'])\n",
        "\n",
        "plot_accuracy(h_callback.history['accuracy'], h_callback.history['val_accuracy'])"
      ],
      "metadata": {
        "id": "g1yqAR7GBgwF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now you won't ever fall short of epochs! Your model will stop early if the quantity monitored doesn't improve for the given amount of epochs."
      ],
      "metadata": {
        "id": "j7B_rVayd_6V"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import the early stopping callback\n",
        "from tensorflow.keras.callbacks import EarlyStopping\n",
        "\n",
        "# Define a callback to monitor val_accuracy\n",
        "monitor_val_acc = EarlyStopping(monitor='val_accuracy',\n",
        "                       patience=5)\n",
        "\n",
        "# Train your model using the early stopping callback\n",
        "model.fit(X_train, y_train,\n",
        "           epochs=1000, validation_data=(X_test, y_test),\n",
        "           callbacks=[monitor_val_acc])"
      ],
      "metadata": {
        "id": "nExSmY1NVOnW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Deep learning models can take a long time to train, especially when you move to deeper architectures and bigger datasets. Saving your model every time it improves as well as stopping it when it no longer does allows you to worry less about choosing the number of epochs to train for. You can also restore a saved model anytime and resume training where you left it."
      ],
      "metadata": {
        "id": "NoxNCyKChYc0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import the EarlyStopping and ModelCheckpoint callbacks\n",
        "from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint\n",
        "\n",
        "# Early stop on validation accuracy\n",
        "monitor_val_acc = EarlyStopping(monitor = 'val_accuracy', patience = 3)\n",
        "\n",
        "# Save the best model as best_banknote_model.hdf5\n",
        "model_checkpoint = ModelCheckpoint('best_banknote_model.hdf5', save_best_only = True)\n",
        "\n",
        "# Fit your model for a stupid amount of epochs\n",
        "h_callback = model.fit(X_train, y_train,\n",
        "                    epochs = 1000000000000,\n",
        "                    callbacks = [monitor_val_acc, model_checkpoint],\n",
        "                    validation_data = (X_test, y_test))"
      ],
      "metadata": {
        "id": "GwzS1vsGi5k8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Learning curves**"
      ],
      "metadata": {
        "id": "9BAK_QySoLOA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Plotting the learning curve helps visualize the training progress, including how the loss changes over epochs for both the training and validation sets. It can provide insights into whether the model is overfitting, underfitting, or converging well."
      ],
      "metadata": {
        "id": "M35faMX_jhSJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Instantiate a Sequential model\n",
        "model = Sequential()\n",
        "\n",
        "# Input and hidden layer with input_shape, 16 neurons, and relu\n",
        "model.add(Dense(16, input_shape = (64,), activation = 'relu'))\n",
        "model.add(Dense(10, activation = 'softmax'))\n",
        "\n",
        "# Compile your model\n",
        "model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])\n",
        "\n",
        "# Test if your model is well assembled by predicting before training\n",
        "print(model.predict(X_train))"
      ],
      "metadata": {
        "id": "2__bNO-MoK8-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Train your model for 60 epochs, using X_test and y_test as validation data\n",
        "h_callback = model.fit(X_train, y_train, epochs=60, validation_data=(X_test, y_test), verbose=0)\n",
        "\n",
        "# Extract from the h_callback object loss and val_loss to plot the learning curve\n",
        "plot_loss(h_callback.history['loss'], h_callback.history['val_loss'])"
      ],
      "metadata": {
        "id": "xZnWgal5h1Cz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "verbose parameter in fit:\n",
        "The verbose parameter controls the verbosity mode during training. It determines whether or not to show the progress of the training process on the console output.\n",
        "\n",
        "If verbose is set to 0, no output will be displayed on the console during the training.\n",
        "If verbose is set to 1, progress bars will be shown for each epoch.\n",
        "If verbose is set to 2, a simple one-line progress bar will be displayed for each epoch."
      ],
      "metadata": {
        "id": "yyuXTF_AhxQP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "When we want to make our model we first initilize by setting the initial weight\n"
      ],
      "metadata": {
        "id": "BrQ6whXSiWQ7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "from tensorflow import keras\n",
        "\n",
        "# Define a simple model\n",
        "model = keras.models.Sequential()\n",
        "model.add(keras.layers.Dense(10, activation='relu', input_shape=(100,)))  # Input layer with 100 features\n",
        "model.add(keras.layers.Dense(10, activation='softmax'))  # Output layer with 10 classes\n",
        "\n",
        "# Get the current weights of the model\n",
        "current_weights = model.get_weights()\n",
        "\n",
        "# Create your own initial weights\n",
        "initial_weights = []\n",
        "for weight in current_weights:\n",
        "    shape = weight.shape\n",
        "    initial_weight = np.random.rand(*shape)  # Set initial weights randomly\n",
        "    initial_weights.append(initial_weight)\n",
        "\n",
        "# Set the initial weights for the model\n",
        "model.set_weights(initial_weights)"
      ],
      "metadata": {
        "id": "7S0fC0XQQW4y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Yes, you are correct. When you create** a Keras model, the initial weights are automatically set for you**. The specific initialization method depends on the type of layer used in the model. For example, dense layers typically use a Glorot uniform initializer by default.\n",
        "\n",
        "However, **if you want to set your own initial weights, you can do so by providing them to the model during its creation or by using the model.set_weights() function.**"
      ],
      "metadata": {
        "id": "4cMUrjQva_5O"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for size in training_sizes:\n",
        "  \t# Get a fraction of training data (we only care about the training data)\n",
        "    X_train_frac, y_train_frac = X_train[:size], y_train[:size]\n",
        "\n",
        "    # Reset the model to the initial weights and train it on the new training data fraction\n",
        "    model.set_weights(initial_weights)\n",
        "    model.fit(X_train_frac, y_train_frac, epochs = 50, callbacks = [early_stop])\n",
        "\n",
        "    # Evaluate and store both: the training data fraction and the complete test set results\n",
        "    train_accs.append(model.evaluate(X_train, y_train)[1])\n",
        "    test_accs.append(model.evaluate(X_test, y_test)[1])\n",
        "\n",
        "# Plot train vs test accuracies\n",
        "plot_results(train_accs,test_accs)"
      ],
      "metadata": {
        "id": "7epk8JEdh8gf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        " The results shows that your model would not benefit a lot from more training data, since the test set accuracy is already starting to flatten. It's time to look at activation funtions!"
      ],
      "metadata": {
        "id": "_0h5WO1OomNS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#enumerate function"
      ],
      "metadata": {
        "id": "lGd4Gbk1ol5e"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = ('apple', 'banana', 'cherry')\n",
        "y = enumerate(x)\n",
        "\n",
        "print(list(y))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4JA37f4RRA7l",
        "outputId": "7a7430fb-4dc0-4309-b71e-e820de4144f0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[(0, 'apple'), (1, 'banana'), (2, 'cherry')]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Predict on coords_small_test\n",
        "preds = model.predict(coords_small_test)\n",
        "# Print preds vs true values\n",
        "print(\"{:45} | {}\".format('Raw Model Predictions','True labels'))\n",
        "for i,pred in enumerate(preds):\n",
        "  print(\"{} | {}\".format(pred,competitors_small_test[i]))\n",
        "\n",
        "# Extract the position of the highest probability from each pred vector\n",
        "preds_chosen = [np.argmax(pred) for pred in preds]\n"
      ],
      "metadata": {
        "id": "Nvoco-BkRHo5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Comparing activation functions**\n"
      ],
      "metadata": {
        "id": "Kj4YatLdTILg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "np.random.seed(1)\n",
        "\n",
        "def get_model(act_function) :\n",
        "  model = Sequential()\n",
        "  model.add(Dense(4, input_shape=(2,), activation=activation_function))\n",
        "  model.add(Dense(50, activation='sigmoid'))\n",
        "  return model"
      ],
      "metadata": {
        "id": "c_Yh0Ua5TL-e"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "activations = ['relu', 'leaky_relu', 'sigmoid', 'tanh']\n",
        "\n",
        "for act in activations:\n",
        "  # Get a new model with the current activation\n",
        "  model = get_model(act)\n",
        "  # Fit the model and store the history results\n",
        "  h_callback = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=20, verbose=0)\n",
        "  activation_result[act]=history\n",
        "\n"
      ],
      "metadata": {
        "id": "mDfpp3j5qPU_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "\n",
        "val_loss_per_function={k:v.history['val_loss'] for k, v , in activation_result.items()}\n",
        "\n",
        "val_acc_per_function={k:v.history['val_acc'] for k, v , in activation_result.items()}"
      ],
      "metadata": {
        "id": "-a0qRj7NvSA2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a dataframe from val_loss_per_function\n",
        "val_loss = pd.DataFrame(val_loss_per_function)# Call plot on the dataframe\n",
        "val_loss.plot()\n",
        "plt.show()\n",
        "\n",
        "# Create a dataframe from val_acc_per_function\n",
        "val_acc = pd.DataFrame(val_acc_per_function)\n",
        "\n",
        "# Call plot on the dataframe\n",
        "val_acc.plot()\n",
        "plt.show()\n"
      ],
      "metadata": {
        "id": "EMZEqxKHr9E2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "\n",
        "np.random.seed(1)\n",
        "\n",
        "def get_model(activation_function):\n",
        "    model = Sequential()\n",
        "    model.add(Dense(4, input_shape=(2,), activation=activation_function))\n",
        "    model.add(Dense(50, activation='sigmoid'))\n",
        "    return model\n",
        "\n",
        "activations = ['relu', 'leaky_relu', 'sigmoid', 'tanh']\n",
        "activation_result = {}  # Dictionary to store activation function results\n",
        "\n",
        "for act in activations:\n",
        "    # Get a new model with the current activation\n",
        "    model = get_model(act)\n",
        "    # Fit the model and store the history results\n",
        "    history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=20, verbose=0)\n",
        "    activation_result[act] = history\n",
        "\n",
        "# Create dictionaries for validation loss and validation accuracy per activation function\n",
        "val_loss_per_function = {k: v.history['val_loss'] for k, v in activation_result.items()}\n",
        "val_acc_per_function = {k: v.history['val_acc'] for k, v in activation_result.items()}\n",
        "\n",
        "# Create dataframes from the dictionaries\n",
        "val_loss = pd.DataFrame(val_loss_per_function)\n",
        "val_acc = pd.DataFrame(val_acc_per_function)\n",
        "\n",
        "# Plot validation loss\n",
        "val_loss.plot()\n",
        "plt.show()\n",
        "\n",
        "# Plot validation accuracy\n",
        "val_acc.plot()\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "2pZsOFrz4oXs"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Certainly! Let's break down the code snippet `k: v.history['val_loss'] for k`:\n",
        "\n",
        "In the dictionary comprehension `k: v.history['val_loss'] for k`, the `k:` part represents the key of the resulting dictionary, and the `v.history['val_loss']` part represents the value.\n",
        "\n",
        "Here's a step-by-step explanation:\n",
        "\n",
        "1. `k:`: This defines the key of each key-value pair in the resulting dictionary. In this case, `k` represents the activation function name.\n",
        "\n",
        "2. `v.history['val_loss']`: This specifies the value associated with each key in the resulting dictionary. Here, `v` represents the history object corresponding to the activation function, and `v.history['val_loss']` retrieves the validation loss values from the history object.\n",
        "\n",
        "3. `for k`: This indicates that the dictionary comprehension iterates over each activation function name (`k`). It assigns the activation function name to the key (`k:`) and retrieves the corresponding validation loss values (`v.history['val_loss']`) for that activation function.\n",
        "\n",
        "In summary, the dictionary comprehension `k: v.history['val_loss'] for k` creates a dictionary where the keys are the activation function names, and the values are the corresponding validation loss values for each activation function."
      ],
      "metadata": {
        "id": "x2h9p8hd4ppi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Batch size and batch normalization**\n"
      ],
      "metadata": {
        "id": "8ovydmim5fh7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "You've seen models are usually trained in batches of a fixed size. The smaller a batch size, the more weight updates per epoch, but at a cost of a more unstable gradient descent. Specially if the batch size is too small and it's not representative of the entire training set.\n",
        "\n",
        "Let's see how different batch sizes affect the accuracy of a simple binary classification model that separates red from blue dots.\n",
        "\n",
        "You'll use a batch size of one, updating the weights once per sample in your training set for each epoch. Then you will use the entire dataset, updating the weights only once per epoch."
      ],
      "metadata": {
        "id": "4DrYdwNGurfn"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can see that accuracy is lower when using a batch size equal to the training set size. This is not because the network had more trouble learning the optimization function: Even though the same number of epochs were used for both batch sizes the number of resulting weight updates was very different!. With a batch of size the training set and 5 epochs we only get 5 updates total, each update computes and averaged gradient descent with all the training set observations. To obtain similar results with this batch size we should increase the number of epochs so that more weight updates take place."
      ],
      "metadata": {
        "id": "iPcDjgT15oUM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Get a fresh new model with get_model\n",
        "## the get model is defined about in the function\n",
        "model = get_model()\n",
        "\n",
        "# Train your model for 5 epochs with a batch size of 1\n",
        "model.fit(X_train, y_train, epochs=5, batch_size=1)\n",
        "print(\"\\n The accuracy when using a batch of size 1 is: \",\n",
        "      model.evaluate(X_test, y_test)[1])\n",
        "\n"
      ],
      "metadata": {
        "id": "2QVlXIIS5gpU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model = get_model()\n",
        "\n",
        "# Fit your model for 5 epochs with a batch of size the training set\n",
        "model.fit(X_train, y_train, epochs=5, batch_size=700)\n",
        "print(\"\\n The accuracy when using the whole training set as batch-size was: \",\n",
        "      model.evaluate(X_test, y_test)[1])"
      ],
      "metadata": {
        "id": "beX3mJjUx8mS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "No, stochastic gradient descent (SGD) is not only for a batch size of 1. In fact, SGD refers to the optimization algorithm where the model updates its weights based on a randomly selected mini-batch of samples from the training set, which can have a batch size greater than 1.\n",
        "\n",
        "SGD can be used with various batch sizes, including 1 (which is often referred to as true SGD) or larger sizes like 16, 32, or even more. The choice of batch size depends on several factors, including:\n",
        "\n",
        "1. Computational Resources: Larger batch sizes require more memory to store the gradients and may need more computational power for processing. If you have limited resources, you may choose a smaller batch size.\n",
        "\n",
        "2. Generalization: Smaller batch sizes, such as 1 or a few samples, can introduce more randomness into the weight updates and act as a form of regularization. This can help the model generalize better to unseen data. However, larger batch sizes may provide more stable updates and can be useful for certain types of problems.\n",
        "\n",
        "3. Convergence Speed: Smaller batch sizes can lead to faster convergence since the model updates its weights more frequently. On the other hand, larger batch sizes may require more epochs to achieve similar convergence, but each epoch can be faster due to parallelization and more efficient computation.\n",
        "\n",
        "Regarding your example with 5 epochs and a data size of 700, let's consider two scenarios:\n",
        "\n",
        "1. Batch Size of 1:\n",
        "   - With a batch size of 1, the model updates its weights after processing each individual sample.\n",
        "   - In 5 epochs, the model goes through the entire dataset 5 times, resulting in a total of 5 * 700 = 3500 weight updates.\n",
        "\n",
        "2. Batch Size of 700:\n",
        "   - With a batch size equal to the data size, the model updates its weights once after processing the entire dataset.\n",
        "   - In 5 epochs, the model goes through the entire dataset 5 times, resulting in a total of 5 weight updates.\n",
        "\n",
        "As you can see, the number of weight updates differs between the two scenarios. With a batch size of 1, the model updates its weights more frequently, whereas with a batch size equal to the data size, the weight update happens only once after processing the entire dataset.\n",
        "\n",
        "The choice of batch size depends on the specific problem, computational resources, and the trade-off between convergence speed and solution quality. It is often a hyperparameter that needs to be tuned through experimentation to find the optimal balance."
      ],
      "metadata": {
        "id": "Ix_rVVgOx9JG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        " SGD randomly selects mini-batches from the training set during each iteration, and the batch size determines the number of samples in each mini-batch. The batch size is typically a fixed hyperparameter, but it can be experimented with to find the optimal value for a particular problem."
      ],
      "metadata": {
        "id": "gjdrphcw5kqE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import batch normalization from keras layers\n",
        "from tensorflow.keras.layers import BatchNormalization\n",
        "\n",
        "# Build your deep network\n",
        "batchnorm_model = Sequential()\n",
        "batchnorm_model.add(Dense(50, input_shape=(64,), activation='relu', kernel_initializer='normal'))\n",
        "batchnorm_model.add(BatchNormalization())\n",
        "\n",
        "batchnorm_model.add(Dense(50, activation='relu', kernel_initializer='normal'))\n",
        "batchnorm_model.add(BatchNormalization())\n",
        "\n",
        "batchnorm_model.add(Dense(50, activation='relu', kernel_initializer='normal'))\n",
        "batchnorm_model.add(BatchNormalization())\n",
        "\n",
        "batchnorm_model.add(Dense(10, activation='softmax', kernel_initializer='normal'))\n",
        "\n",
        "# Compile your model with sgd\n",
        "batchnorm_model.compile(optimizer='sgd', loss='categorical_crossentropy', metrics=['accuracy'])"
      ],
      "metadata": {
        "id": "day0F2r26wdr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this case, kernel_initializer='normal' indicates that the weights of the Dense layers will be initialized using a normal distribution. The normal distribution, also known as the Gaussian distribution, is a common choice for weight initialization. It generates random values centered around zero, following a bell-shaped curve.\n",
        "\n",
        "By using kernel_initializer='normal', the weights of each Dense layer in the batchnorm_model will be randomly initialized from a normal distribution. The random initialization helps in breaking the symmetry in the network and allows the model to learn distinct representations."
      ],
      "metadata": {
        "id": "v2_QA_6785uv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Train your standard model, storing its history callback\n",
        "h1_callback = standard_model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, verbose=0)\n",
        "\n",
        "# Train the batch normalized model you recently built, store its history callback\n",
        "h2_callback = batchnorm_model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, verbose=0)\n",
        "\n",
        "# Call compare_histories_acc passing in both model histories\n",
        "compare_histories_acc(h1_callback,h2_callback)"
      ],
      "metadata": {
        "id": "PrQkDCer87RA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Batch normalization tends to increase the learning speed of our models and make their learning curves more stable. Let's see how two identical models with and without batch normalization compare."
      ],
      "metadata": {
        "id": "gJHAO7cB-hdG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can see that for this deep model batch normalization proved to be useful, helping the model obtain high accuracy values just over the first 10 training epochs."
      ],
      "metadata": {
        "id": "D_zKK7WX-h-4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# ***hyperparamethers tuning***"
      ],
      "metadata": {
        "id": "08zKbvi_LR0T"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "what can we change to optimize the performance of our model?"
      ],
      "metadata": {
        "id": "JsIhB4afNRrd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "![hyperparamethers.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAzQAAAI4CAYAAAC8x6y4AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAFiUAABYlAUlSJPAAAKAOSURBVHhe7d0FmFTVG8fx18IOQMVu+SsKFoKCIAqCiIB0Sac0gkEJKo1KKyFIKCnSSAiiCCoGdicIBo2YCPuf39k7y+zsnQ3YXebC9/M88+zMjZl7z4097z11xJ9/7k4wAAAAAAigI/74418CGgAAAACBdMTO3/8moAEAAAAQSEckhHjvAQAAACBQjvT+AgAAAEDgENAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAOqgBzV9//2OtHnrccpxTIMWrRMX69uP6jd6SGRfruzVN8xDfeg8aleLY3V27hW3Ztt1bAoh/8Xwec40BAA4VcVtCs/qdtfb0c1Psn3//9aYAAAAAQHJxXeXs2edftNfffNf7BCA9vvr2BytYqlqKp+9TXlrgLQEAQHBQooy0xHVA8/uuP2zg8HG28ZffvCkAAAAAsE/cdwqwYtUaGzl+qv333x5vCgAAAAAkivuARiZMm2Mr33rP+wQAAAAAiQIR0Pz86yYbMnqi/fLbZm8KAAAAAAQkoJGFr7xu4ya/RNUzAAAAAEniMqA58/RcdvJJJ3qf9hk1YRpVzwAAAAAkicuApnyZ26xTq0bep31U9Wz42Bfs101bvClZb+/evfbpF9+4LgPVReBZ+Yq57gJz573Z7qze1J54+jn74uvv3HLpcaBd6mZ0/bS6OtQ4P6+8/qY16dDdri5WIdlyGgsolnC69Bww3IqVvzdpnUKlq9v93fu7dSNL07R9kd+tl/ZD+3MwpGfgVW2/9kP7E5k22l/tt/Y/vcc9lt82b7Xnprxk9973YLLfCKfja6vfSXMgWG1jeD299D0fffalN3ef+q07J1su/NI58sO6DXbL3XVSzBs8aqIlJCR43xBbrPNyzKQZ3hKpi7V+en5faTh11sIUaaj3Oq+nz1nklkmv9Fxj6zf87K593QN0Lwgvk1kD9+qe4nc89GrTuZfrATKrKd2VFn0Gj05xjStddd9Izzhh6bnWUqN7le5Z0evrvPWTnuOn31362mq3H9qf8DLha/v9jz474GtbMuP6jpRZ9/N4SaOdv+9y12ftFg/YZYXudN+v/7GV6rdxwzZEXrcZPQ+i74vhVzgddH7rOov+H6ZXrO8UradBv6Ov//D9RmkWeUzT2o5oaR3jyP+9kcclfE7pe1OryaI0VdqWr3Nf0rYr7XUMdCz++PMvb8n9E94+7Ye2e3/yTEq/6HvGowNHeHP3WbJitZ19VfFky+mlc1vneFp0/s2ctyTFOR4+lqoZpGXSKyvOOf2+tiN6G9N7vA8ncRnQHHnkkVa7cjm7u3QJb8o+8xa/ahOnz8nyA6iLTT2sVbi3pV13e2V3Meni2bp9h5uvDMXyN962Lr0GWYFb77EaTTva2o8/TzPzFU++/OZ7qxna7rtqNreJ0+ak6wYgupnr4lK6KLPz9nsfeXPMPvjkCxd0lqhY36o1bh+4NBEd+/c+/NTdjLUf2p/ItNH+ar+1/0qHnzb+4s1JP6Vh+6597coi5ax5x57uH0nkb4TT8Y6qja1oudo2bfbLWTrI7NlnnWGFrivgfdrnrfc+DN1Q0848f/blN75B1Op3PkhX5lvpHb3+2XnOsKKFrrcjjjjCm5Lcpi3b7JF+Q61A8YpWr9XDKdJQ73VeKzOpZbSs1jkQOgbPPDfVbryjurv2dQ/I7OBCD2y69hlia97/2Juyjx72dLv/Pt8S7Myk+1y30DbcXLaW++cbfY0rXXXfKFOtqTt2QbnGtZ1vvvuBlazc0MrVauH2Q/sTFr62b7qzplVp2G6/718H4/re3/t5tOxII+2rHkIUuau2uz5fnLvY1v200c3Tubdg6WvW8sHH3HU7ZPSkA85kR9M126P/sFDa10nxPyw1un/ouri+ZJUU13/4fqM0U9otW/lWqpn2/aH/NZH/eyOPS/icCv/v1fkQSUGC0lJpqrRd/OqqpG1X2usY6FjonHz1jTUZPqZByjPpOCr98hYua7Wad0pxjoeP5T31WoeOddVMOQczes5FXiPajuhtTOt4H47itg3NmWfktoa1KsWoejY9KeLNCjrx9E9cN2tdkOkx5+VlVrZGM3t+xrxARMuff/WdNW7fzf3jSC/deHQTLBfK6Gs/06LvVprMCN0oD8ZNa3/oprx4+RvuZqubb1qUDjWbdXRPXdJD3//S/KUuDZ9+bkrSP5TUfPL511a35UP2QM+BSf8cMtuxOXJY8Ztv8D7t82YoIPn2h3XeJ3//7t5tq9b4X4/K7K7fkHrApxv3O2s/8T7tU+j6AnbZJRd4n/bRuaRjo3Or39Bn05UmWkbLah3dO/bnfNR1PXTM89aua58sOw76p9l70Ej34CZakRuvs77dOlie0L0xK/362xZr2uERGzhiXJrnp9JS14ruC/F+jWv7dC+qVK+NvftByvMtWvj+pWAjvRnTg3V978/93E92pJH2UfuqhxBpBV3hZZWJ13mZGXb98Zd17TPY3Q/Sc3zClPmu1qh9uq4LpV31xh1s0MiJmZYf0DHW/5r0/u/V+RD+v6T7ykOPPZGuc0zbXrVROxs/dVa6t13pEYQ8k85vBesV67Zy25ue603BntKtQZsuSUF3RmX0nNPxUuCXnmtEdLzrtHjQPYA8nMV1pwClbi1idatV8D7to5Nq2LMvHPDTVj86wR989IkM3+xE6+omMmjUhLgOataFMph6Wu33BDg1ar+kYuCMPPlTmtz3wKP2zPip3pT4pn3s0L1fhm5cSkc9UU+rKqTOiQnTZlvT+x/JUBqGjRw/zWU0s6rK5bVXXxkKIvJ7nxKpmqeCktT8Elom1rmkf6hvv5/6TfbX3zb7/kbRQtfZaaec7H1KFM5w1W7WybdEKC1ap16rzu6faUYz4C/OW2z9hozxPmU+nR/KBOs4R9NxGflED8t76UXelKzxSyjTqH+8fgFVLLpWdM/85IuvvSnxacWqd6xL78HpysSEaVnd81TCktb5crCu7/29n/vJ6jRSRk0ZSb9zPDWqFqTM3RffZDxdoz09bnKGf1/3MVX1zMiDVOUfOvd6yrr3HepN2X/ab5X4ZeQYa9nu/YZ5vcROytA+a9s79Rhor65K+6Gejn8Q8kw6N/XgpW7Lh9MVrEfT/wxVy0vvw8tIGTnnwv8HlMfNCP1v69RjgH3zfeoPIA9lcR3QHH/csdaiQQ0rkO9/3pR9dHKpCG7Pnswr0g3fbMe+MNObsn+U6VHmJ6MZpuyiC/KNt9/3PqWP1unSe9B+PaHQTS69RfoHmzIi+geQUcoApvYPXdN1TuifREZv+pH0O3qCr3M1s8WqdpZWtbGvvvvRPfWK5d0PPk21ncBHn32VYv1Y1c0UcGY0wxVN53Cnnk+478qIl5etPKBjl5rw+eEXMF1w3jnWp2sHu+LyS7wpWUf/FPfnKb/uD89NnnXA1aay0vips/b7/tUtlClN7Xw5mNf3/tzPYxmfhWmUWsCeHjo392fboqk9QkakVgU0PVK7N6aX9nt/HuAon9Smc2978unnvCnpp2Oq6rWptT8MUp5J52brh3sd0DmkcyA9Dy+jZeSc+/DTL1wtpP2h7Zs4bXaWB4fxKq4DGvnfZRfbfQ1rep+S01P/tJ7+ZoT+kb/w4nzvU0rXXn2FC7B6d23v/uqzH90IVLfx2x/We1Pim6r13X5LYevcrqk99nBbq17xzmRPgpUZ1dOC1G7o4bTR+lXKl7Zcp53qzQm24jcXdOmil9IotbYLC0PnT6yxknQu6JyIldlRprVqhTIu/dL6LZ2jkZlOVUX6d+NHSa9PVs71fQgwYXjfZMuFX107NHfzY1U7S63amB4ovLbqHe+TPwWzakTvR+v7BbsF8uW1iy88z/uUSO0SHuk3LOY/JKVXmduKWo8HWqWZhvqOp54Zf0BjW+kaqVejorsfdGzZwG4pHLu9T2r0T1xVdvSUO/r80Pb36tzWit2U8rhkB13X2jelqdI2tfN/0fKV+1UycbBEXnOp3c9F54v+32zbsdObklxWXt8Zpe9N7X6eEZmZRsrYD0/lqbP+Z5S741Z3rumcS+23MlP4f1f4/3rhG/Y91NH9aXwoyEutxDLyPlC3egWXZlkt8v+S3qdm7qLlyc5LpXOdquXTtb3zl6wI/d+P/TAyq/JMepg9on/3ZP+ndF5EK12iiP386evJltPr3VdmJDvnVZtHeZhY/zsizz299D5WHkbngqrJHcjD9NTOOT2c8NvOGveUtc9Wzbe/f/rAvfRe06LNfnl5oO7Dmelo72/cUgahWuiGumrN+ynqjuqg6yS9Mu+llvPUU7yp+2fjL7+5nj/8/iFVLFvSeoZO8nz/uzRZhkUZETXMUiNB9WwSSZn/F16c5xrwHnVUfMaN+sfXon4Na9+ivp2RO6c3NZH2Lfy0RP+I1DjNzx23Fgn9s2tj1+W/0nXmEKYnN0rPvoNHH9DT9INF+9WrSzt344k85t//+JM9+NiT7slXNDUC/fTLb1zpQiQ9LdFTE7+AUDfd3qHfubNkMRdQRNJNWE/WlPGOpHP0uckvuUxu9G8dqHC1s8ht1RNgPTXS+R/tt81b0iy+15NFBUV+mSqt71fv9+ZQkJY7575/KOGMhV+Vj9TO43B1i+g0FD01U/W11o1rZygQKXjt1fZ4KMi4rWihZOe8qB1BRoMaPTnU022/e8/DoQxL1fJlMvydB0oBcb9H7ncZ48h9VHr2HTLa9ym7qjq89e6Hlv/KvN6U+KSMSuf2zazJvVXtxBOO96YmHju1k+j11EjfgELVniqUuc1qVS7nTUkUL9d3eu/n6ZHZaaT/B6qO51f6re3u2qG5+61TTj7Jm7rv/+tjTzx9QAFeLMq0PtKpZYp7vIQzq2poHSvDrmukb/cOoftAYTv66KO8qYltAmcteMWlUWZnLPWbg3t3tiI3Xpt0XeqYqA1ixx79fc/BSA+2bmwdWzVMll/SsdGQGOply+8epAdOZUsWT5GPCUqeSb+poM7vf7bOc+VfFOBFnueSWrroXC4buqb9/iemJq1z7u9//vFts1oi9L+m/yMd7ZyzzvSmmF128QU2sOcDbtsiS4BS+399qMu8syYL6SbXIXST9qtyoRuoAp2M3Kz9LH71DddDR7Tm9avbuCG97KorLktxAuqzMvLjhvZ2vQ9F0wUb68n0waZ/IsP6dnUZs+h/fqJ90w1TN+f5i1f43rS0z9r3G665KkXGTjeHtk3vtSmjn8yWJ1aZSTcPtVfQsY0+5io1GN6vm+/xFgU80fRPTU9NoilwmDbmKXfzj87siI5Lr87t3FOcaAqe3s6CBoCxqp3pqZFftbHPQzfPNWtT/ycq+ofrVx1J66v710g6N0sUvTFZ2itjMXP+Uu/TPmmdx8oQxkpD0T+5jPRSp+P+4rjBVrLYTSnOedG06HMmNfrnE6sqp57ctWxYK1lmKTso0/T8M/2tVPGbU+yj0rNvt/vdtvnRua5OIuKV7kXjh/d196boDIz2VfeysaH7feM6VbypyanBc/S9MB6u7/Tez9MjK9Losy+/taU+jcW13XoSr//vkcGMaJt1D544op/7P5yZ9H36Xr97vChDrTzFkhWr3DUaTcdW14gefEVfnzrWNSvd5Y69lsss4d9USXDksdR7PeF/dtDjqf5ezwdbu1f0w9/E/9V13cMTP19//6Pt+uNP79M+QckzqRRePbhF03mu/EnTutVSnOeiaTovdX7qPI2kc0LnRkbynek75/YF05Euv+RCy+lTYnTWmae7ICnadz75kMNB+u5wceDqKy63ds3qep+S0xPY6ExRRujm+/qbKev+6h97myb3pjiZo6nXofbN66V4mqYnDplRfzYrVCx7u+sWO61/crEabCtt9OQxtR6XdNEqY6pi8SBp2bCmnX/u2d6nlLTPrRrX9j0vdPOMvsmpdNHvn2KrRrXdTT81+mdZv8Y9vl2YL33tzVTbpuwP/TP2q3amJz46FyJpP/VEPjrz0u3+FnZzwWu9T4nWrP3Ifv4l+dPZWOvr6WPeSy/2PiVS1VL/NKxl1SuWTfU8VhoqMPDLhL/+5ruhfUtf3XQdb31H5FOyA5Fa3XxlFpX58PtHm9VUxVdVfWPRNrWoX9P3AZOOZTzX31amQoGaX4YiTE9tH27b1FXljKYg44f1G7xPieLh+k7v/Tw9siKNVLLqVzrTsWXDNEsgdd11btfMPWjKDDpvm9Wrnub/dXVX71d6rP/zjz/cLs02bTr2qr6U1u+kV8NalVO9LjVPy/hRoKOhMGI9HEntPNR9e8fO5NUIg5RnUq0JPSCI1rRuVVddL7VzT/P0QEIBarRlr79pW7alr/ZJes+5447N4YKUaCol+zHqmgq7t1oFV+0u8hUrr3yoC0xAoxOr8t13mNpnRNPTTQ3WFKvublp0o/34s6+8T/vcdkshu+iCc71PqVMR4u3FbvI+7aMnUwdS1zKrqFpA9BMxPz/+9LPvDaZaxTJ2+SVp18vWcburVHH3RDsIbi1yoxW+4RrvU2yqVlPoupRPwxTQ/P3PvpIIZUhUxB5NGX714pUeZ56eyxVvR9OT4e37ec6nxq+3M/2jie7FSjfzN6OqgKkYvMxtt9jVV17uTUnkt/72nb+76ivRoqubqRjebzn9k6hZKfY/6UjKhNetVjHFP1DRP4v0XKO6vlXdLDP8+Wfs7pn15FLBjDKN2U3HT8c+tX/yovuiAs9oKu366++/vU/xRftVLZR5Ts/5csF5Z7v2BdFUrS4yeImX6zu99/O0ZEUa/fnX3/b5V996n/ZR5rdq6P95en5LDxFqVU6ZqdwfCtb8qr9G++W3Te4YRFOm/6aCaf+PEAV8lcqV8j7tv/Rcl5qnZbRsNJ2P55yd+oMYlez53d9U9Sryf5oEJc+kh2Z+D4z0f0Cla+mp2qb2PH7HUO19fv71N+9T6tJ7zukYqgQnmqptV23U3j2817UVWQqu7cud87Rkr5NOPMGbe3gJTEAjKirt1LKhKyqMpqpn6v8/I0WAYZu3bvPtQUQnz0kX3pBsNNdYL42Cq/qf0X76+dfQDT3ze6Q6ELrhXepz0/OjhtjR9JRB9bvTW8/1zNNzZ1pGMKvlznWanXB82k/FdcO45KLzvU+x6dj7PVlRkBgeGTs9LzUYj7YhdG7p3M1ssaqdaVTzyBupqteph7JIBa76nws0Cl57lTdlH7/1oxuc6tyKrm6mf6jf+XSwoXMwuuOA1Fxx+cW+/0BjVamIdsVlF9spJx14plGBXN8hY2L29lSr0l2plnxmpXPPzmOn50pZZSma/olq2SBJT6YuLLXMYWR1jni4vjNyP09LVqSRrl+//yOqJpVaSXg0/Q/x+62MujLvJb5VAKOpdy+NERRNVb50/qeHHqQUvj7lvTSjzsidy05NR8CqZbRstNNCeaccxxzjffKn43lxOoORoOSZFIjpOoqmgEwDw/ptl99Lg9VGU+Ce3t7O0nvOicZf8yspU3Ct8XA0KOnZVxW38nXuc+Oi6WHf4dqrWbRABTRyfYF8MYvT1MXg/oyW6nezzSybNm+JuzrlyrCn9+L60adu/8UXnJeuTE+YAp+gNFDT040cOVK/8WfElq3bTWN7ZAXdUKOra2WGWNXO9KRLY86E+VUj0VPXU04+0TfzoSfZSo8wNYaPXt+vulmsNNRT2/RmLETBkl+GaJeqSe35z/sU23Gh30pvEJ8apaMa08YyfOzkLL0npebY0LmfnvNfmZ/MqN6UnU7PnTPNTF0knV+X+ATMkZmHeLi+M3I/T0tWpNG27Tt8x4zL6PUb67cyKr1Pr2O1rbvw/Iy1CU2r2mF6nHzSCXZiOrZby2jZrBaUPFOsBw6ZZVMqXVpHykiJiUps1XYntRId3Rc0po7GnSlcpoYVKlPdJk2f6x4eHM4CF9DoH+m91cr7Vj3TE4PnpsT3WAjx4Kwzc7uSiP11oOsj/uX732UuOImkp84ac0Z27NyVoipiZOnKReefm6LqnkpoVJ9ZdEP2raoTVd0sNZmRucmI7Po9BYqDRk7I9PZRh7sLMlAakBr17hRPxyYz78fZmUbZff2GZTQgiaR7YkYe5gGS0XNOtQ9eGDnAt8G/H5UkaoDSe+970D0IOVwFLqCR1Kqe6cmnGvoi6+iJZOSTdhx6VBUksm/8sFVvv+/qN6/bsNE+impMr9JTld6JghvVkY6m0glVC/Wrg+1X3Sw1h3JPLpNmzHU9sO1PFVr4W5dJvSdltGQhSLIzjYJ4/eqhaVZU8wWiXXPVFTZ1zJM274VnrMKdt7v/j2lR9+YN2nQ+bIOaQAY0EqvqmZ78jp44wxVbples6HnckN4pBmzK6Gv+5JGuGlNmUV3/TVvSV8yZGS70CRq/X/dThm7qygCrod/hSE9O9QQ1WqPalW37t2t8z5mMvPx6GcoMyoyorng0jTmjzjf0RCj6pnnTDde49lJhfuNoKEOg3oPUc150HWy/6mYSKw0z+qRc9wa/G/1JoX8URx91cIbkUgcA/brfn+KflbZVA83tTxVa+Nu8ZVuGqrJo+Y1RVSIlshF7UK/vWLIijdTdrF9X0hm9frV8dgZB551zlvcuuYxWt/r0i8RS6UNJUPJMqo554fkp2wWpk6INH7/m+/sZeUWPt5TZVJVUAxprqICfPlphby2aak88+mCqgxzroaHGxToc29UENqBJreqZnmzOWrjMDTiVHrp4ontlEnUFnV3V15TRSs/TWA1SGd3uICv53biU2VL7h/T2RJKewRcPVccfd5zvP0b1+PVbOhsUHiwabyK62pk6Afj6ux98u0m/teiNydqYXHrRBXZzVE9Yb77zgcuk+wW4saqbqWGtXwcMOgf9xv2JRV3J+nVBfvnFFx6UXmGUWe3brYPd17CW1a2Wsrco/WN6+rkph2y9aPVOmZ59y6x2KqoiufHn9PVKJKr659coPLKqVJCvbz9ZkUa6fv3+j6h3wYyMOaL/Idn55FltGfzyBbHG5PKj8/vtVEbaD6p4yDOlR6xukD/76lv7YV3Wta3JCnrIqAf5Gh9KpTbrPlhu44f18W1ro7GgfvwpWPuXGQIb0EhqVc80wmt0Hf9YzjrzDN+TYu7iV31v1n70z1kjPg8cMS7ZS+0GIsV6WrV6zdrQP+3k43xE041i4dJ9I8JmhwvPO9v1fBNtxpzFLmObFgVpGsXWrx/4w8EJxx9nV+ZN2SGCMqvzlqxIVxCrf57jp85KcW6pEWBGnnBmlF+1MwXTy15/K0X7F50jeS+50PuU6NRTTkpx7mh9Dcj2wSfJu2FOrbrZccce69uVpbqvnDprQbqeROna0QC8fuOFaB8zo7F/Rqh3KA3eqvuO/lGpEajf03j1hBb0qmc6pH7puzoU3K79OPXxw7TfClz9elTKKF1zGpw1PQ9i9BR+wtQ53qd91KlE5PgjQb6+/WRnGumYvjhvSbquX5XOTHlpofcpe8TKF8wPHVeNn5UeCvhmLXjF+3ToyM4804HQ/xO//x36P/TivMXpCr50LagH3ejtVCdU+ztUSCxKE+1/9GuzTxV/PSioXeVuV2ITTYFlentgO5QEOqARjZlxX4Oa3qd99CQnvdWclPHSAEvRdHL1GTw6zRNDT2E0Ds7Djz9lXXsPTnr1GzLGjjkmeVUW9SBz6qkne5/2UYb/+Rfnxby56x/jomUrbcK02d6U7JHnzNPdk/po+mekQQFTSxtt84pV77guag9nyqj7Bd3q4lKZtdQoDZWh7fjIgGTnll7rN/7inkCll7ohzYhY1c5Gjp+aooRGGfSzfMZ4USY9utqZHjYoMxspVnWzsMLXX5MskxQ2YtwUmz7n5VRLY3VNTZ4537dnMV3311yVvBQqq+lc6NO1Q7L90RPsrvc3961G8NQzEwJd9UwBqV8XzyrpVcCW2j1E+/3clJe8TwfumdC5q9HIUws0VAreb+gYlxmNpo4u1OFFpHi5vjNLVqSR331Angz931TGMrXf0nnSd8ho31Hps5J6a1Q12mjKDHfvN8T34UgkVTV7dOAIt/2HmuzMM6Xm911/2h9pdLmv+7vftur/gYLN1M490cC5Dz72ZLLt1EulsLq3ZSaN+3dH1cYpXm+8FbtduLra9istOxwFPqDRk796NSq60VwPxF0li7tMWTQNfNeobdfQSb3WN9Okp1QtOvX0HVNCY15EP5lS1RZVcfGji7n/sGdt5++7vCmJ9BRhyksLrGOPgdl+c1QdzrvLlPDNaIXTRtV4otNGNyz1kV6rWUd3kzucKeOqge+iKV0atevmns76Vb3RNPV2pTEqoo97eGAw/xKNHO7pTbQZcxel+Ces45baUya/amcKjKK3RwOS+nX3etklGhAueSmP3/rq5lklrrFoFOwqd9/hfdpH39Omc2/r3neob9ewynx06zvEmnfs6Xvt6L4Rq658Vrnisot8u3K9rWhha9WolvdpHz08GDB8bIr7QpBc9T//rmvD9xCNpRCZsdB5qao99973kCs1yCy65hq07uzuTdHXnH5T97LGoWty7AszvanJlS5RJMW9MLuv76yWFWmkbvvvCE2Ppv3W/isdos9vnQ86L+q1ethGTZjuTc0+SvvSJYr6PkjROalzU4Ff9ENI/b+eOmuh1Wh6f6aeu/Emu/JMYX7VglULZ9qcRSmOgc7bcOmm7u9+47qEzz0FVlu2pSwB0XHUgzBdw355mDtuvdk99MtMGn8s+v+tTJoxL2aQqIeL0aViuv6OOjLtAWsPNYEPaERVuDq0qOf7lCy99IS0Ya3K3qfkdNO67Z76bkCjJh26u+LGHv2H2Z3Vm9r1JavYtNkve0vuoxOqXvUKKTJpCsBuK1bYN0DQBaYnOnkLl7VazTtZv6HPWtsuve2GklWtQZsuvhdVdlC1oZqV/EdpVtrcXLaWG6RK26ptrt3iAbumRCU3CJSe5B3uFBQ2rF3J9x+jjmmz+3vYpQVLW6X6baz3oFHJ0lBPsPwy4joeBa7K631K7tRTTnGDY0bTP9cSFetbw9A/G53DOpd1Tj89boq3REqxejuLpH9qV1/h/4TotFNOTnPEdF0LxW4umGq1L81rULOSb7UspY/254qb73KDjSkN9dL7q4tVcNUa/NxVqrhVq1DmoGQa/aghdctGtd12RVN1uQnT5qT5NDFeFciX17fqqugeorEU9CRS91W9ileoa7dXapApVc2i6Z6ke5OuL11nut4693rK3cN0L1NPQX7UXvNOnxH9s/v6zg6ZnUZ6wFK/xj2+pTTaf6WD/u+F00i/pXNA50Ws38oOepBSp+rd3qfkdG6Wq9XCrr2tUlK+QH/1/1pBmAZCPJRlV54p7NIYA1mrtKTEPfWS7h36jZKVGya1z9L9vXrFO61E0ULucySde1pf557W0/qRxzFWvkvn+a0+33eglKZ+pUkKEhUEvrP2k6ROO/QgcvzUWda93zD3OZLa2lxwXuZ0wR4kh0RAI6qS4lf1LL100uufSIsGNbwpKekGNTGUqdAFoGpUy9942/efkegm6DcquahOZ+nbinqfUtI/k5nzltgj/Ya6pxgH+8aopxBtmtTxfRoTpjYV2lZt84tzFx+04CteKcPfpX0z30BWdMz1j1sBbVppqOPQsFalmIPpqaqE31Me0e9odGadwzqX0zq3YlU7i6TuJVU10Y+uK21vrP2WQtfltyt9MoPRdLN/7OE2MR9c6FrUYGNKQ730Ptb1qe+4/74Gvg1GDyZtj7bLbx9VhcmvM4YgUGBcqVwp75M/dbev+6pe2fFkW9eXrjNdb08+Pd53XKQwHQ/9f4mV2crO6zs7ZWYaKaBtHfo/EktkGum31GnAwRZ+kKLeCGOJzBek5556qMjOPJOozU6s/2u6X4TvHfqN6BIbdSPe7f4Wqf7v0HpaP63jmNZ5fiB0zes+6Rf469ooWq62nXThDZbjnAKW58pb3MMSv/uI/mdH9jh6uDhkAprMqHqmp0hdO7Qwv57TMkI3P32PX7Uf0YWgCyLWxZUa9bySWuYwq+jpo+r97882a3vTesp/qNPNv2r5MqFMj387ifRS+ke3v4im31K3jvtzrPz4VTuLpJtnakXvahujNjKx3FQweXfPqVFX0H26trdcp6Vv8E0/SpcnenZy3xWPVArVvH5179M++selJ+V+VeviXTjz41f6lBade5lxLrdvXi/VhzKx6Hrt1bltqudLdl7fWSkr08iVQDaslWoGODWZdR5klKoB9e7Sbr/SRWKVTB4KsivPJHqgVbJ47IAnLZnxvyM95/mBSivwT4v+fygIT63Gw6HqkNrjzKh6ppvX6CcftQdbN/amZIwu7GF9u7rvSY0uCF0YGfnnpzrVI5/omTR4YXbTNo/o3923d5NYtH9PPPqANaubMoN2uNE/dJ2fOr/25xxVuo8d/Hi6bqYqNfHLFO+P1KqdKZPh12lEJHXFrC6Z/ej8SKu6WSRlHFVNbPLoJ1INsmLROhNH9HUPPvRd8Ujnif4h+T2cUQNyPT1Mb5fp8USlTx1bNszQ/UP3vOH9u7l2RwdKJePqESgj540yP7rnqcpKWudLdl7fWSWr00gZ1p4Pts5wUKNAePRTj2bKebA/FGAO69vNt8prarp3vM8eeaCl9+nQlF15Jle1s1al/Q4sw/87XpowdL/+d2TkPD8Q4cD/4bZNvCnpp7TRNir4OxwdciHcgVY9E2WyVLVlyYxnXQPH9NDJPrDnA25gKRVvpkUXRI17ytqL44a4BtFp0T+A8cP7ZigzkNm0zXryv2DySDcGUFq0X9o/Zc50kSJ0wR15pLuBL5s5zt200hvQKr0XTRvtGt+n52aq9FZ/9UN6dzmgJ1KSWrWza/Nf4duDVSRtr0ph/PY1vdXNIun7br+lsL0cSg/d9NOzf1pGy2odZUrSk4YHkx7OaHv9MsbqhUo97wSRMusvjBxg5e641ZviT+fKA60a2aRn+tvlUd2BHwj1WjV74rB03b+0jYtnjHElS7pu0yO7ru+slNVpFP5fOXFEvzT/n4XPg3FDex/0TJqCvRnjBrvtSeu4ar8mPd3fOrdrlukNx+OR0iOr80yiwHLs4F5p3j9i0bWl+7/+DyiwTu//xv05zw+EAv9HOrVy51B68nxK/8Z1qrj8ll+HM4eLQy6g0ZNe3YhTq/OaHjpp1Yhs7vNP20evzXZPrdRTRmQG49qrr3DV3KaMesK+evtla9esboZuXrq4brulkC176TlbMGWk+67Ik1dPxXXRfbJyrsuY+o1fczDoH8uzgx63tctfcvXGI5/eK01aN65jS18ca6/OHu/2L94zjweD0nBw7872+eoF9vwzA9yxV9qF6TzT+abz7uu3F7mbeEZ749ITrfsa1rR3lk63Pt06uCAg/I9Y55meND0XyijoqVdaYlU7U4P/1KoJhCloUfASLSPVzaLpenjs4bb20etzktIw8vrR+/D1+fmbC9yy8XINpYcaduqeEk1Vz9Se5tc0ukaNVyo9nDrmSVs4dVSyY6ZzU+eoztUPXn3Jeh9g9ZBYdB3Fun/pvaZpRO6Zzw1x27o/suP6zkpZnUa6NylzuHrhZJc+VSuUSfrfqmOuDKRKRD4OXdtZdR7sD90/enVpZ+8vm+l7T9Vxnj1xuL29eJp7YHk4PcjLjjyTqKOGWPcPPXjTuTmsX9dUA2AdRy2nbdDx0gPjyHNc55sCsx4PtHLXwIHcC/aXzh2dQzqXtI2x7iG6Tj5cMcueHvBI3LUJzW5HJAS12xwAwCFPbYbqtXwoxeC8ymx07dDc+7SPGvPWbv5Aih7SJgzva7Uql/M+Hd5IIwCHmkOuhAYAEB807lF6R76O5ZffNtmPPj35qIMUAACEgAYAkCU0Un70qNd6pTbydSRVIFCXrN98v86bss/FFx6czlEAAPGHgAYAkCViBR0a3VsDw6Xl2x/W23NTXvI+7aOuTfNmYmcBAIBgI6ABAGSJq/53mZX0GSxv5rwlNvzZF+yvv//xpqSktjMaNd5vkM3bixW2c8+On0b0AICDi4AGAJAl1OvOXTG6WH38yWesWqN2tmj5G7Zj5y5vqrmSG423U7FuK3vhxXne1H3CvfscjgPHAQD88R8BAJAl1GV7lbvvcN25+lmyYrVVuLelnXFFEctxTgH3ynPlLVatcQd794NPvKWSa1S7crZ3oQoAiG8ENACALKNB87rd3yLZeBT7S+OLaQA5BuoFAEQioAEAZKliN91gT/TsdEBBjYKZYX27Wp4z9m8gVgDAoYuBNQEA2eLDT7+wLr0G29LXVntT0qYRwFvUr2EdWzWMmxHjAQDxhYAGAJBt1LPZwqWv2Yhxk+2Nt9/3pqakQKZi2dutffN6lv/KvK49DgAAfghoAADZbu/evfbrpi320Wdf2adffG17Qp/l5BNPtOvyX2lXXH6JnXrKSW4aAACpIaABAAAAEFh0CgAAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAdlgFN70GjLMc5BazArffYx59/5U1N3Vff/mAFS1VzL70Pki3bttvdtVu4fZ7y0gJv6uHtw0+/sFrNO1nuvDfbWfmKuffpPa5//f2PtXrocZeeOpcAAABw8BzWJTRffP2d9Rwwwn7dtMWbgsPBex9+alUatreZ85bY77v+sK3bd7j3m7du85YAAABAUBz2Vc7mLX7VJk6fY//9t8ebgkPZnj17bfbCZbbup41WIN//7PlnBtjSF8fa7InD7ewzz/CWAgAAQFDQhiak35Ax9uqqt71POJT9u3u3K5GR5vWrW/WKd9qtRW60u0oVt4svPM9NBwAAQHAc1gFN++b1rHGdKq7a0aMDR9g336/z5uBwcNKJJ3jvAAAAEFSHdUBz6iknW7tmda3Q9fltzfsf29Axk+yPP//y5qZfuJMBNRRXg3E/qXUqsPqdtW59NdzftGWbrVi1xjVSV2P1cIN1Tdu7d69b/p9//7VZC16x8nXuS2rU3rTDI/bRZ19aQkKCWyY1+p7I39B36Lv0nfru1Gjdt9770Jp37GmXFbrTbffVxSpYzwHD7cf1G72lkotsRK9OCX7a+Is90HNg0vr701GBjtPUWQuT0kDfU6pKI3v6uSmuE4Ro4fQ/9ZIbbcykGW5a/dad3Xrhl45DZkkrjSOPefT2hs+n1DocCK+fWicV+l6lh9JFy2obqjZqb3NeXhbzOEdul9qWzV74it1ZvWmKbdV5pjZokcfxlrvr2KgJ0/frGgIAANhfcRnQrP34c5fxe/3Nd70pWed/l11srRvXsZNPOtFGjp/mMnvpCQqywp9//W0Dhj1rVRq2c43UVTUq3GBd08ZMetEFPO279rUaTe+3xa+uSmrUPmHabCtdtYnNmLs41e1XW6FBIycm+w19h75L39mi06PuN/woo6rApXj5uvbclJdcOxRRhrrP4NFWuEwNmzR9bqrtkXRsazbraENGT0paP6P0HeVqtbB6rR5OSgPR+aK0KVOtqQu6Dhalaftufa10tSax03jzVm/prKH9VzooPcLXkbZh7qLlVq1xB7v3vodSTX+di/2HPmvVm9xvy99IXh1T55fOsxIV6yc7jnoo0KZzL2vSoTsdbQAAgGwTVwGNutK9vVIDlzFWxk9Pli8pWNqmzX7ZWyLzHXHEEVaxbEmrW62C+/xI/+H2/kefuffZ7Y2333cZRLXnUCN1NVYf2PMBy3vpRS4zOnDEOGvesYeNfWGm1binbIpllHHu9dRI+/yr77xvTGn0pOmh4GOUFb+5oE0d/WRSg/h7q5V3Qd0LL86zx598OsVTdj3Rf+yJp61fKJOr5Vo2rGULp45KWl/bE87IvzgvdlA1eNRE++a7dfZIp5a2aPoY91Lj/PRSqYAyzSpJuOC8c6zHA63cd0Smg0qqlGGPLHE5O88ZNqRPF1swZaRVuPN2N617x/vceuFX3ksvdtMPhEqjHhv4tAuOJfI4zRg7yKqUL+3SuG2XPm5+VlAvbtp/pUPBa6+2sYN7Je1jOI0UuHfo3j9m4KFzcfjYF6x0iSJJ58n9LRvasTlyuJI4lWbqeKv9UXi+vlvHREHc+KmzXAcMAAAAWS6U8YwLG37+NeGCa29POObs/L6v2Qtf8ZY8cKFMv/tO/Q37+ddNCRXrtnLTKzdom/DLb5u9OYm+/Ob7hBtKVnUvvY8U/r6WDz6W8Odff3tTk0tt/VVr3k/azwd6DkzxHR9/9pVbL7zMgOFjE3bv/s+bmyhymVDQ4E1NtHnrtoRytZonrd9v6JiEv//5x5ubaM+ePQmhjHZCrstvcsuMmzzTm5NoyYpVbt6lN5ZJeHnZyoS9e/d6cxJpe4Y9+7xb99YK9RJ+WLfBm5Pg9kdpo3laf/nKt1Osnx6R31O0XO2ETz7/2puzz4/rNyQdx5rNOiaEMt3enESR3zF55nxvasZEfkfkOSThdNJLaRh9nJTuSn+tq5eOi45PJL/zM1r4nIk+n3bs/D2hYdsubl6LTj0Ttmzb7s3Z56tvf0i4o2pjt8zQMc8nOxaR52Lrhx9P2Pn7Lm/OPrF+W2YtWOrmlarSKGHjL795UwEAALJO3JTQDBg21kJBhfcppV5PZe0Ahmedebrdf18D94T5YHXlfMXll1iDmpXs+OOO9aYkuuqKy1wJiqi9T7XyZezoo49yn8OuzHuplS1ZzL1XFbBYbXlKFrvJ6lWv6J60RzryyCOtesWy1rFlQ/d54SsrbduOne69vktdHauUqHblu6xU8ZtdyVYkbY++V9upkpHlb7zlzUmuQpnbrEiha1Osnx6hAMbmL1nhSlsef7idS5doOn4qedHfJa+uctXTsktkOtWsdFfoVS7FcVK6N6tX3ZXUZIVPvvja5ry83J1LLRvVslynnerN2efySy60B1o3ciVtC5e+ZqHg3Zuzj9K4XvV73DKxHHdcDstxzDHep0QlQ+fGq7MnWM8HW9PpAgAAyBZxE9CofUFqVB0tVqPzzFLsphtcJwGirpxfXva6e59dLjj3LMtzZm7v0z7K/Oc5I3H6aaecbCefnDKTedRRR1q+/13qfYqt2M0FXfDmR5nvO2+/xWVmPw1ljNV4X7aHAptww/M8oXVXrXnfXlv9ToqXgodwBlrv//4nZVB1fYF8KYKp9Prgk89d0Ht7KCgreO1V3tSU8v3vMitzW1EXWKx6+/1saxO1afMWF3RJpXKlUgSmYTlPPcXKlbrV+5S5Pv3iG7ffl150fiitNvseJ7127txlF55/jq1Z+7Gt++lnb+198oUC5IsuONf7lNxlF19od5cu4drMdO831N5Z+4nt+uNPN08BUNFC17lXasEQAABAZombgOb3Xbu8d7GFM01ZRYFDo9qVrUWDGi5T2H/Ys67NxqHk4lAmNbXSEQUzamOhLqyVBqK/27YnltZ06NbP7qjaOOZr6Jjn3XLf/bDet7crBV776zevIf0lF56X6tN/BRLnnHWme6/Sh7//Sb3ntsyi39E+X33l5Xb+OWd5U/1dclHWjHkTTqMFS19zvZL5HSO91Puagi8d2+9+XO/WiaTjdOSR/ufJmafnsr7dOli5O2517duKlqttuS6/Kc3e7gAAALJC3AQ0aTUMV9UW9UiW1U484Xhr23RfV87q9tYvYw4cznQtzhg72D5cMcsG9XrYVaHbvGVbUm930+csyraSMQAAcHiLm4BGJSOpUalJdHuErHLZxRe4NhoqrQh35Zya9GyXMnd7Ew5+r0/fr9uQakZTVbpUvUxpEK4ydNyxOVygp88r5kywfzd+lOZr/uSRljvnaW79zKKSAfnux59SLa1TW5aNv/zm3qt6nbY/O4TTSSUf673qerGoalhaNJZNRoOCcMlV07rVbMd37/gem+hXrcrl3DoZpfNebbdaNaptU0Y9Yd++u8SG9O5iu//7z7WJ+/q7H70lAQAAsk7cBDRVK5Sxh9s28T4lp6otTzz6oPcpe6g9Tesmddz7p56ZkGoG9IJzz3Z/NR5HrNKcz7/6Nql9xcGkwR5jdb6gThA0kKLmX3XF5XaeV23qjNNzu2pUqp60aNnKmJ0laLq6vFYbjayoqnft1Ve6IHP5yrfs3Q8+9aam9M33P9rKt95LbM9R+Pr96oBgf4TTSTSOUqyOGdTZQvTYLpHCAfJHn31l23f+7t5HUpDz8Wf+59KN113t9lv7r3SIRd016zitWrM2qWpheuze/Z/rulvjAGngz3937/bmJJZuVr/nTru54DWuy+hwuysAAICsFDcBjTz2cFubP/kZ11PWTTdcY+XL3Gajn3rUZk0Y5i2RfZSpbHJvVVeVRpmzvkNG26Yt/oMhqvqNMtrq2EClOdEZfpUWTJoxz/t0cCmgmTRjboqR4lUaMH3OyzZi3BT3uXK5Uq7xuqhNyj13lXQZ5ckvLbT5S15NUXKgz9r3SvXbWOUGbW1jKj3W7S8FC2qMroDriRHjXDufaErrngNGuICq9G1F7br8V3pzsl5kOk2dtTD0WpDiXFC6j5443Y3VEsuN1+V336H0nL94hTs2kT778tvQMZzjfUru6lAgWrHs7W7/FYj7DZKqaR0f6e/a0miAVPVwl17HHHO05c55qi19bbXrNENtpSL9/vsfvkEYAABAVomrgEZKlyhq44b0ttfnTbKZzw1x3RgfLMrQ9+jU0rWn+eCTL2KWbKjajTLa8uCjT1inHgPsldffdE/ANXJ+1UbtXVfQ8UCBV/e+Q61KKOh4af5St40LX3ndje7epnNv97Re1fvCg0+GaSBOBXgqhWrcPnFZraf1ta/qLKBZxx4ujepUvdsKX1/AWzPzKGBo06SOOx7KUN9Zo5krJVBph7ZDg5JqdH6ltbpt1rLhoCy73FL4BjdIq9Kxecee1rBtl6R0UnrXbNrRpb+OQywKwhSMiQYqvb97/6TzSfuoQWfVvsvPKSef5KqAaf81gGfFuq1s1ITpbl29dD5qmtq4aBl1ta2SlYzQNVrkxuvcNjRq181mzF2UtH+tH37cTdf8/Ffm9dYAAADIOnEX0MQblb60blzHPTGPRRltdfesTJwysupI4K6azd0T8Mbtu7mep9S2ICMj4meVrh2au6p9S1astprNOrptvKdea3t+xjy37XWqlrfuHVumyOSqq2WN7t+lfTP3WaUMWk/ra1+1z+FgSGOQZDSTnF4aX2VY324urRVcPTpwhN1Zvanbjgd6DnTVnJTOzz/T3y2T3XQuPN65rQsqRL2AhdNJ6a3ex5TGQ/t0cfP9RAbS0eeT9vHCUCAy6sme3tIp3XDNVfbCMwOs4LVX27sffBIKPnu5dfXS+ahp6sluRP/urmplRl1w3tn2cLumLiDSd9Vp8WDS/um80vT2zeu65QAAALIaAU0a1P6iavkySQNOxqIM4oxxg61Xl3Z27dVXJE1r37yeLZs5zkoWv8lNO9gUmCngWDLjWVN1Oo0bo2kat2XamKds5BM97IzcOb2lk1OQoqDmldD+KMjT/okysGoDpe8c3Kuz72COmUklGAumjLSJI/q57Q4HmypFGty7sy2eMcZVWTxYtD1PPvZg6mnsdXAQiwK3WROGu/MpnM7aPwVz08cOSrPHv8I3FLBF00a7wCcyjTRd36nBLzV9f9oXaR2NV6TzWoG8jr9Enu/33FUq29ouAQCAw9sRCRntRgnAQadG/yUrN7TSJYrYhFBgl9k9ygEAAAQFJTRAHNFgrjnOKWDl69xnm7du96Ymp2cQ732Y2MubeqI7/rjj3HsAAIDDEQENEEfCPZytfucDW/vxZ97U5L785nt7cd5i915VyE44noAGAAAcvghogDhS8JqrXLfL6gxAPcdpXKDw2EYa80VVzTo+MsD1JHZXqeJWtmRxNw8AAOBwRRsaIM6op7Zm9/ew1e+s9aakpJ7chvfvdlA7PwAAAIgHBDRAHNLgl2MmzbChoyfZ1u07vKmJPahpnJtOrRq69jMAAACHOwIaII6pmtmOnbts/YafLc8ZuS1XztPcWDcAAABIREADAAAAILDoFAAAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwDsuApvegUZbjnAJW4NZ77OPPv/Kmpu6rb3+wgqWquZfeB8mWbdvt7tot3D5PeWmBN/Xw9uGnX1it5p0sd96b7ax8xdz7oB1XHJ50Deta1jWtaxsAgMPdYV1C88XX31nPASPs101bvCk4HLz34adWpWF7mzlvif2+6w/bun2He7956zZvCQAAAATFYV/lbN7iV23i9Dn23397vCk4lO3Zs9dmL1xm637aaAXy/c+ef2aALX1xrM2eONzOPvMMbykAAAAEBW1oQvoNGWOvrnrb+4RD2b+7d7sSGWlev7pVr3in3VrkRrurVHG7+MLz3HQAAAAEx2Ed0LRvXs8a16niqh09OnCEffP9Om8ODgcnnXiC9w4AAABBdVgHNKeecrK1a1bXCl2f39a8/7ENHTPJ/vjzL29u+oU7GWj10OP219//eFOTS61TgdXvrE1q5LtpyzZbsWqNa6SuxurhBuuatnfvXrf8P//+a7MWvGLl69yX1Ki9aYdH7KPPvrSEhAS3TGr0PZG/oe/Qd+k79d2p0bpvvfehNe/Y0y4rdKfb7quLVbCeA4bbj+s3ekslpzRR2oQ7Jfhp4y/2QM+BSevvT0cFOk5TZy1MSgN9T6kqjezp56b4NpQOp/+pl9xoYybNcNPqt+7s1gu/dBzSI3p/tC2jJkx3v69p2i+lT3qOx/oNP1ufwaOtUOnqbl0dj4Ztu9rrb76bdLwjRf62zrtYIs+pyPTI6LHQ9qut2YOPPuGOs5YJ75/OA79tlMjf37x1u0uLyHNGaaU0S+1603breN5yd52k39W2xjrPUhOdbtqv/dmmsIweN1Haallth0oJR0+cnrRvmqZtzCy6jwwYPjZp+3TsdAyVdtHnQKTIbYy1PeFrScvFumbi5T4BADg8xF1Ao8Di/u79rWLdVtakQ3eXyc5K/7vsYmvduI6dfNKJNnL8NJvz8rJ0BQVZ4c+//rYBw561Kg3buUbqyvSEG6xr2phJL7qMSvuufa1G0/tt8aurkhq1T5g220pXbWIz5i5OdfvVVmjQyInJfkPfoe/Sd7bo9Kj7DT/K6ClDUrx8XXtuykuuHYoog6PMXeEyNWzS9Lmptkda+/HnVrNZRxsyelLS+hml7yhXq4XVa/VwUhqIMpNKmzLVmrrMVHb46edf3Xa06dzL/b5ov5Q+qR0PTZu98BW7rVJDl6YffPKFm67j8cKL81zmun23vu5zVknrWCiD2XfIGCtaro4NHjUxKRAP75/OA217agGA2ixpf267p0Gyc0ZppTTTNe7XKYd+q0qDdu546p4gWlfbWi4UJK186z03bX/s3r3bxk+dFXObqjfuELO0NjOOm+bpHtf64V5J+5ZZtH26Jm67p7516zMkafuUnjqGJUPb99L8JTGDrswQL/cJAMDhI64Cmv6hzLyeWA4f+4K9vGylTZw2x2WylWHMKkcccYRVLFvS6lar4D4/0n+4vf/RZ+59dnvj7ffdP3C151AjdTVWH9jzAct76UUu0z5wxDhr3rGHjX1hptW4p2yKZZRR6vXUSPv8q++8b0xp9KTpoUzFKCt+c0GbOvrJpAbx91Yr74I6Zcoef/LpFJlUldw89sTT1m/os265lg1r2cKpo5LW1/bo95WZe3Fe7KBKmapvvltnj3RqaYumj3EvNc5PL5UWKNOpJ8MXnHeO9XiglfuOyHTQk/d773so2dPjs/OcYUP6dLEFU0ZahTtvd9O6d7zPrRd+5b30Yjc9I7r2HmyrQsctvD/6nl5d2lmu00516TFg2Fj7+rsfvaX3mb9khTVu391l1kqXKJJ0LPQd+i6trwA7rYDhQKR2LJTZ1HWo39e5V+6OW5OdLzreovNBaRDraf6ylW+5p+w3F7wmaf2Xxg913ycKqseHggsFPmE617Tv4WOs46r1tL7aOyljPGjkBPtt81ZvjYyZv+Q16z1otHuYMXZwL/fdOpfbNKnj0n3pa6vtocee9A20MuO4aZ8nz5yf7BrW+6OOOvDb8ZvvfuBKOZRGuhbCaaft6921vR13bA5r26WP6wwlK8TLfQIAcJgJ/UOJC6F/sAnHnJ0/5qtbnyHekgculOl336m/YT//uimhYt1WbnrlBm0Tfvltszcn0ZfffJ9wQ8mq7qX3kcLf1/LBxxL+/Otvb2pyqa2/as37SfsZyvyl+I6PP/vKrRdeZsDwsQm7d//nzU0UuUwoM+BNTbR567aEcrWaJ63fb+iYhL//+cebm2jPnj0JoWAmIdflN7llxk2e6c1JtGTFKjfv0hvLJISCzYS9e/d6cxJpe4Y9+7xb99YK9RJ+WLfBm5Pg9kdpo3laf/nKt1Osnx6R31O0XO2ETz7/2puzz4/rNyQdx5rNOiaEMk/enESR3xHKVHpTMyY9+xPKWCbkL17RLRN9PDb8/GvCHVUbu3mhzF+K4y2hzLxbX2k+d9Fyb2ry3448f6OFzykddx3/sPRsu7z+5rsJF1x7u1vO73zR8dY5ou2L3kaJPKdbP/x4ws7fd3lzEmk7dK5rfvHydd1xC4s8X6OP0fc//uSOveaFAg9vatoi9zvWNikdlB5KFy0TCkoSQoGdN/fAjptoX8K/75em6RX+nuhju2Pn7wkN23Zx83QNRKZpmK6ZcPrpFZ2+4e9WWvntn4TvZVpOxzlSPNwnAACHn7gpoQm3a4hlxLjJ3ruscdaZp9v99zVwT4QPVlfOV1x+iTWoWcmOP+5Yb0qiq664zJWgiNr7VCtfxo4++ij3OezKvJda2ZLF3Hs9nY31xLxksZusXvWKdmyOHN6UREceeaRVr1jWOrZs6D4vfGWlbdux073Xd6mrYz2pr135LitV/GZXshVJ26Pv1XbqyfryN97y5iRXocxtVqTQtSnWT49QZsw9IVdpy+MPt3PpEk3HTyUv+rvk1VWu6kpWql6hjBW76YYU+1P4+gJWs9Jd7v23P6yzUObVvZfXVr/j2jCVKFrIdUoRfbzlphuuce27lOZzFi3PklKaWMci3EYrFOS77VOVzOjzRce7dpW7rcm9VVPdxssuviD0HVXd0/pI2metq6fun3zxdei3NntzkotOm/PPPdtG9O/unvjnveRCb2rG6DprVq96im1SOpQoeqN1btfUfVZJjdrKhGXWcdM1XCN0rUWn6YFSOs55ebm7PnQd6xqIpmvmoTZNUux7ZoiX+wQA4PATNwFNWtW8dv3xp3325bfep6yhjKkyI6KunF9e9rp7n10uOPcsy3Nmbu/TPvqnnueMxOmnnXKynXxyysyIqqvk+9+l3qfYit1c0AVvfpTZuPP2W1yG6NNQ5kiNcmV7KLBRkCR5QuuuWvO+y9xFvxQ8qMqN6H1kJj7s+gL59jsj98Enn7tM9u2hoKzgtVd5U1PK97/LrMxtRV3GStXBQoG7Nyfz5c+XN0VwKTpmF19wrnuv6lThTdD78Hl80fnnuupofmmp9hxHHnmEnXl6LhfI/fLbJrdOZop1LLZu22Eff/6Ve1+lfGk78YTj3ftoWrdi2dtd5ljXb/h8iXTJheeFgpCzvE/J5c51WuhczO2O0569+x4enHbKKe48lV5PjXJtVtSua+/eve48v+aqK1zVM78Me3roOr/sYv9gSMftjltvtpsLXuvat3wTCkYlM4/b1Vdcbmd613NmemftJy4tdX1ce/UV3tSUbgrtW6Hr8nufMk+83CcAAIefuAlojj/+OO9dbKr/nZWUmWlUu7K1aFDDZQzUpkdtNg4lymSn9tRTwYzq3qtRtNJA9Hfb9sTSmg7d+tkdVRvHfA0d87xb7rsf1vs+nT6QdgLhNhPKJKfW5bKenJ9z1pnu/S+/bQ5lmFLvuS07RY6DM37qLN80DL/UQ5T2WcGCX3uOAxXrWISP99VXXm7nn+MfjISdnitn6Jw6z2Xe/XqX2x/aLrW9eLhtE/t+3U9Wvcn9dm7+W+2cq29Nsyex9NC54Ve6EpYzlNm+5KLz3ftN3jmXmcftqKOOSvUa3F966CNpXR+aF96/zBQv9wkAwOEnbv5rqHpCalRNJCv+CUfT0+i2Tfd15axuY/3+4QLIOir1efShNvbpG/Ps+WcGuIcMp+fOmdSTmDoa4LoEAAASNwFNx5YN7JSTT/I+pdSlfTPvXdZTvX+10VBpRbgr59T4VTmKpmpPexOyrqvU9Pp+3YZUq2CpSpeqjSgNwvXsVTKmQE+fV8yZYP9u/CjN1/zJIy13ztPc+plF1Xjkux9/Snoa7Ud1+Tf+8pt7r+p1WV2ylxF68nyCVxrZ88HW9s+GD33TL/pV5Mbr3Dp6sJ+ep9eRvYZllI5zztNOcaUu632qkUXavHWbK0VRaU5mH2+VYuj4Va94pw3t09U+em22azujhw3Dnn3B5i5a7i2ZMTo3YrUxk23bd7iSAznDO+cO9Lhlh3CpTFrXR2TVsFj27NmT4aqa8XKfAAAcfuImoLn0ogtszqQRKbrmPPWUk+zpAY8kNbDOLqpn37pJHff+qWcm2KdffOPe+7ng3LPdX3XlGuup8edffesyiAebGjUraPGjThDUXkHzr7ricjvPq250xum5XYZVVUoWLVsZs7METVc1G9WTz4qqetdefaULMpevfMve/eBTb2pK33z/oxunRBmrooWvz5LqPfsrxzHHuPYBktqxkO9DGVOl5dvvfZQ04Olxxx5rec5IbAMVK2OujOh7H8ZOn7Tkynmq5b8yr3uvLoZjndM63jofdF5on8Lny4FSV88aZ0gDn0a2y1HHFWo7U7ZkcfdZ55qqgmWUzg2lrR+lnbpPV/fHCpwuC92X5ECPW3a48bqr3Tmv6yM8/oyfdz/4xKWdn/DDGXUw8FuMao4KmNQ1erR4uU8AAA4/cVVRuWih6+zdV2bYkhljbdSTPW3mc0Ns/Yevut6Qspv+set31Sha/7z7Dhltm7b4j3uh8SyU0daAdirNif5HroznpBnzvE8HlzJjk2bMTZHRUpuE6XNethHjprjPlcuVspynnuLeq73BPXeVdJmlyS8ttPlLXk3x9Fafte+V6rexyg3a2sZUMnz7S5mlu0uXcJnJJ0aM8x38UGndc8AIl1EqfVtRuy7/ld6c+KFMuXrK0rEYOX6qb8CgRufNO/V07Q0WLV9pRx91tDfH7JqrEoN+9fj26htvpzgWaryuMT72lxpjVwodf53T0+cscoO2Rp/TkeeLzouKd94es/OAjFJJnEp9tH8KLiL3T+ftlq0H1lZH58Yz46e4jHck/c6KVe+4saikfJnbXK9qYQd63LKaOhtQJw26Pp58+jnfASn1YEZtA6P3Peyq/13uqvequu0LM+enuE/o+orVI2W83CcAAIefuApowtR1asNalV2GQk+kDxZl6Ht0aume1OqJZ6ynsuoyWRltefDRJ6xTjwH2yutvuieQGhG7aqP2WTaQXUYpk9q971CrEspMvDR/qdvGha+87kZsb9O5t8voqL1CePDJMA3EqQBPmSQNLKhltZ7W176qEXCzjj1cGtWperfrtjizKcOkwQ91PNSl7p01mlnvQaNseShTr+3QoKSlqzVxaa0esLRsOCiLJ2qUrm1Txk8DENZo0sG1DdE+hPdDGT5lnLWv0YMuFgqlbTiwa9S2a4o00GC0BzoCvXr5UgmlzgeN1l+tcftk54tKTxq06eLma1DaUrcW8dY8cOqUQtd++Lc1oKx+V+fZQ48+6QIspZ3OSZWcZJQCpskzF1jFuq3c9Rn+bl23tZp1dOe4fl9dqEem+4Eet6ymKru6RnXu6xjdVauF2yZtm84PnScV67V2A++Gq29Gu/Ti85O6f3904AhrErrWw9d5eu5l8XCfAAAchhIOQ34Da8aigd2mvLTADRandfwGxhRN00BxWib6pUHinh43xa3rt36sQRAjxRpML1KsQfEiByocNWFaQve+Q9x7v1cok5rw2+at3prJ7frjz4Qe/YclpYXfq03nXglbtm331kgUOWBe9EB+++P9jz6LmdZ6KY01sKWfzNiW9H5HrOMhOq9mL3wl4apbyidtd/RL+6h99aOBL2Otq4ETdZz1Pvp8ycj+a9lQJjjV461zSedFtPSc05HnZfQAjZEDpPq9Yv1uLJH7rXM4PCho9PfqdVfN5glff/ejt2ZyB3LcUjsfMiL8PX5pq+0LBVQxt0/3oudnzE1o0amn++x3DmhQYQ0uHL2uXrq2xr4w0/3V5+jjJvFynwAAHD7isoQmnqj9RdXyZZIGnIxFT5VnjBtsvbq0SxoDQtPaN69ny2aOs5LFb3LTDjY9XVaj5iUznnXV6TQehKZp3JZpY56ykU/0sDNy5/SWTk5Vih7p1NJeCe2PBlvU/omeCFetUMZ95+BenZPGmMgqqka2YMpImziin9tubb/o6fDg3p1t8YwxboDDeKbzqmLZkq6Re/9HOrptF+3L7bcUtmF9u7l9jFVlTm28Fkwe6c6v8HHQeafzb9aE4a563oFSiZgGmVy14IVkv6PjrRLU1+dNcudSZlU1i6TfeGHkQJcOKu0QnVc6Z3WeHcjvHnPMMa70Rfulcaf0W6JjoN+bPnaQ6xTDz4Eet6ym7VPVuFdnT4h5L6p8d2nXHikWjXmlaysy7cPn1svTRtstha9302KJl/sEAODwcYSiGu89AByS1HmCqpSp/UePB1pZ1w7NvTnIqC+/+d5qNe9kRx5xpE0eNTApYAEA4GChhAYAYDPmLrIc5xSwgqWqpdqtc7jHRpXkahBSAAAONgIaAEBSD2fq1VFdW/sV3v+6aUtSj403FbyGamMAgLhAQAMAcFXH7imb2Lth196DXfW8bTt2us/qtlulMqq2p17ONF5YdvfiBgBALPw3AgC4sbdaNa7jOl7Yun2HtX64l+W58hZXDe2EC66z60tWsWmzX3aN+7vd39yNvwUAQDygUwAAQBINGDpx2hwbMW5yirY0KpV5qE2TTOlFDwCAzEJAAwBIQdXMfv/jDzdI5qknn2yn585pJ514gjcXAID4QUADAAAAILBoQwMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwDosA5reg0ZZjnMKuL8Irr/+/sdaPfS4O5ZTXlrgTQUAAMDhhBIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgyaC9e/failVrrFbzTnZWvmKWO+/NVr7OfTZrwSv2z7//2up31rpG6nfXbmFbtm331kqUns4IwusXLFXNvvr2B29qcvrep5+bYqWqNHLLahuqNmpvc15e5rbBT+R2/bppi81e+IrdWb1pzG1NS0JCgn3x9Xf24KNP2NXFKrjvuazQnda8Y097670PXTr5yeh2bNqyzQYMH2uFSld3y+i39FnT00Pboe3Rdmn7wt/Rc8Bw+3H9Rm+p5KI7G/hp4y/2QM+BSevTAQEAAED8iNuARpnd3bv/8z7Fh63bd1j7bn2tdLUmNnPeEvf5911/2OJXV1mNpvdbi06P2qbNW72ls4Yy52WqNbX2Xfva62++66ZpG+YuWm7VGnewe+97yNb95J9Rlz//+tv6D33Wqje535a/8bY3NWOU4e87ZIwVLVfHBo+amBR46Xefm/KSFS9f1wUMf/z5l5vuJ63tUMD02up37LZ76lu3PkPsg0++cNP1W/qs6eH9j0W/r+3Q9mi7wumi7+gzeLQVLlPDJk2fa//9t8dN97P248+tZrOONmT0pFTTFQAAAAdHXAU0epquJ+FnXlHUzr/mNjvxwutdoKCSgINNmfjHBj5tI8dPc59r3FPWZk8cbktfHGszxg6yKuVL2wsvzrO2Xfq4+VnhvQ8/dQHLR599aQWvvdrGDu7lfl+vgT0fsLyXXuRKaTp07+8CQj9vvP2+DR/7gpUuUcSmjn7SrXt/y4Z2bI4c3hKpU+Zf6ytQUCBV7o5bk75H6aF0kX6hYKVr78Eu3fyktR3vf/SZNW7f3QUf2q/eXdvbouljkvZVOoSCy1Vr3nfvo6mk6rEnnnbbcfJJJ1rLhrVs4dRRybYzHKC+OG+xC6D8KGD75rt19kinlu739SqQ73/eXAAAABx0oYxc3LijauOEY87On+KV58pbEj769EtvqQPX66mR7nv1N72WrFiVkOvym9xr3OSZCbt3/+fNSfT3P/8k9Bs6Jmmby9VqnrB56zZvbqL0/G4og+6WuaFk1YQvv/nem5qQsGPn7wkN23Zx81p06pmwZdt2b84+ocx/UhoOHfN8QihA9Obs+169Wj/8eMLO33d5czLm9TffTbjg2tvd92h/td+RlC5Kn3BazV203JuTKD3bEbmvFeu2Svhx/QZvzj4bfv41oXKDtknfNXnmfG9OovDxuvTGMgkvL1uZLC1E2zns2efdurdWqJfww7p9v/HnX38ntHzwMTdP6y9f+XaK9QEAABAf4qaERlWY1DbFj56kd+0z2PuU/VTKMHvhMlciUbPSXaFXOTv66KO8uYlUstCsXnVXUpMVPvnia5vz8nK74vJLrGWjWpbrtFO9OftcfsmF9kDrRq5EYuHS1+yX3zZ7c/Y5O88ZVq/6PW6ZjFKph9oK/fzrJmtcp4q1blwnRcmO0qV2lbutyb1VXXrNWbTct+pZatsR3lct07FlQ7vgvHO8Ofucc9aZ1qJBDd/1I49X7cp3WaniN9sRRxzhzU2k7axXvaLdW628a9ez/I23vDnJVShzmxUpdG2K9QEAABAf4iageWn+Uu+dv0XL37Dfsrh9SiybNm+xTz7/2r2vVK6UHX/cse59tJynnmLlSt3qfcpcn37xjcugX3rR+aGAYrNrX+L32rlzl114/jm2Zu3Htu6nn72198mX91K76IJzvU8Zs3XbDvv486/cewVuJ55wvHsfTUFOxbK3u2BDVcfUqD5aatsR3tfbi91k1159hTc1pevy57MiN17rfdpn+46dSe168px5uquW5pdWah8TDgz1/u9/UlaPu75AvnRXxwMAAED2i5uAZsPPKTO90X71KXHIDn//868rZbj6ysvt/HPO8qb6u+Si87x3mSsczC1Y+prrDeyOqo19X+p9TcGXAoLvflzv1ol01FFH2pFH7l9pg75z2/ad6UqH03PltIsvOM9ti18PaqltR3hfL7nwPDvpxBPcez8KqPxKb8LbKR269fNNp/Br6Jjn3XLf/bDetyRJ2wkAAID4FTe5tYtCmd+0nHtOHu8dAAAAAMRRQBPuHSsWVfXyazeSHY47NocrDVBpw3qf6lORVF0qLerNLSFGr1qxhEsqmtatZju+e8f+3fhRmq9alcu5dTKLqpDlPO2UdKXD5q3b7Pt1P7nSnNw5T/Omps+Zp+dyf7/78Sfb9cef7r0fzVPJSrTw8dL2rpgzwTdtol/zJ4/M8HYCAADg4IubgKZds7pW4c7bvU/JXXzheda/e0fvU/Y74/TcLmMuahQfqyvibTt2pjq2S7gjgY8++8q27/zdvY+kIOfjzxLb6kS78bqrXQZ95Vvv2Tff/+hNTUndNat9yKo1a13Vq8yUK+eplv/KvO69xuGJNc6MunZetGyl+321QTkvjepp0a664jK3r8tXvmXvfvCpNzWlDz753LUVihY+Xvp9bUescWY0XW18lF7x0DU4AAAAMi6uGgi8OG6wPfpQG7vs4gvcZz2pV4nEa3Mm7ndD9sygTgDuuauky2RPnbUw9FqQIpOsHsBGT5zuMvqx3HhdfvcdGitm/uIVKUbT/+zLb23SjDnep+SuvuJy19BeGe+nnpngO1K+pnV8pL9rG6KBJI88MnMPrxrHq6RMvY9Nn7PIJkybnSIdtE/T57xsI8ZNcftaMRSkxuo8IJbwvqo3tSdGjLMvv/nem7OPBrkcMXayb9AWebwmv7TQ5i95NUWJmD7rOFSq38YqN2hrG0O/BQAAgOA5qmeI9z4uFLvpBmvVuLZ17dDcddmrgRtTaxi+P1TKoafyCpyU6f1x/caYrx07f7czcueyc8/OY5s2b3U9Zs1fssK++X6dHXfssbbxl9/s/Q8/s0f6DbNRE6a7zL6qQqk3MmX+Tzj+OO9XzU479RT7IpQ5//yrb+3VN962X37dbEcfc7St3/CL66a4c69BrvRG8pxxuutJLHeuxGpQxx6bw5V0LH3tTXsjtP0aJV/BxB9//um2c+Wb71nHHgNC81e7hvIaCFLdOIepitj4qbN9tysjzj0rj+1N2GsvL1vpep7TCP45jjnGNeRXQPb4k89Y70Gj7N9/d1uTOlVdV9bHHH20t3b6tiNyX9d+/JktWPq6C1z+27PH9dymtOr4yAD79Mtv7LRTTnYlRZXuKpVUeiTnnZMndOx22eLQNmo7w1XkdLx07IaNed5tpzosaFS7stWvcY/bD1G6LlmxypXeRH8vAAAA4ssRCRltzHEIUEb20YEjvE+pUwnRE48+6J76K1OtwGXEuMne3OTqVC1v95S93ao17uBGwJ8wol+KdhkqYWnSobuteT9lVak7bi1iVSuUtuYde7rR6CePGuhGyY/09nsfWYfu/ezdDz7xpiSn5Z987CH3+5Fjp2islRIV68fcroxQlbtBIyfYk08/51tCIg+3bWIPtW2aonQmvduh01JBW6uHHk/qgjmSgrZHH2xlb7z9vo19YaZNGN43RZshBToDh4+1Yc++EHM7NZZNzwdbJ2ufpf3rFAoOx0ya4fu9AAAAiB9xVeUs3qk058nHHrQlM551pSfKBGtamduK2rQxT9nIJ3rYGV6D9lg0MOasCcOtV5d2ScFK8ZsL2rC+3Wz62EH2v8sudtNiKXxDAVs0bbSNerKn+139fni6vvPV2RPc9KwcCFLBXed2TW3VghesffN6SfuhIKNhrcr2+rxJLkjIaFWzSNr+W4vc6PZH+xUej0a/pd9cNnOcVb67dKrV6vT7Kql6JbSsBgGN3M6qFcq44zi4V+eD1tkEAAAADtxhWUITz1QdrmTlhplSkgIAAAAc6iihySb9hz1rOc4pYOXr3Gebt6YcaFIUW773YWKvXmpDcvxx+9fOBQAAADhcENBkk3APZ6vf+cA1dPej3rxenLfYvVcVsv1tuA8AAAAcLghosknBa65yXRGrcXqHbv1s9sJXksZx+Xf3blfVTD13qbOAu0oVt7Ili7t5AAAAAGKjDU02Um9dze7v4Xr6ikW9mw3v381uuuEabwoAAACAWAhospkGv1R3wENHT7Kt23d4UxN7UKtbrYJ1atUwwyPrAwAAAIcrApqDRNXMNPDj+g0/W54zcluunKe57pABAAAApB8BDQAAAIDAolMAAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwDosA5reg0ZZjnMKuL849CQkJNiLcxfbZYXutGb397BNW7Z5c4Lh101brH7rznZ1sQo25+Vlbn8AAADgjxIaBE44IG310OP219//eFP3Wb/hZxv27Au27qeNNn7qLFux6u3EGXHgq29/sIKlqrntX/3OWm9qclNnLbQpLy1wyz43ZZZt3/m7NwcAAADRCGhwyDnmmGMsR45jvE9m//23x3sXDCccf5z3zmzPnj2h117vEwAAAKIR0OCQc9aZp1vzetXtgvPOsQY1K1mpW4t4c4Khwp23W63K5SzvpRdZk3urWu6cp3pzAAAAEI2ABoecI444wqpWKGPfrFlko5961M7IndObEwx5zshtE4b3tU9WzrWKZUu6/QEAAIA/ApoM2rt3r61YtcZqNe9kZ+UrZrnz3mzl69xnsxa8Yv/8+69rF6H2EXfXbmFbtm331kqUns4IwuurnYXaUPjR9z793BQrVaWRW1bbULVRe9eAXNvgJ3K71Oh89sJX7M7qTWNua1oOJB3C/vjzL9deROtpfS2vfdK+Ra8T2fbk0YEj3LQxk2bYqZfc6KbppXYnYXqvaeF2Nqp29ki/oW5akw7d3W/Hou2/v3t/t6z+RqepOhkYMHysFSpd3S2jxvsPPvqE/bh+o/st/Wb09oSPvZb96LMv3bQSFeu7aXpFppP+6rOmx2pno44Cvvj6O/e7+k4tq04QmnfsaW+996E7Pn4i00VpkNox9BP+3Qd6DnS/p++65e46NmrC9FTTFAAAIKvEXUCjjNiMOYus/7BnbewLM23Dz796cw6+rdt3WPtufa10tSY2c94S9/n3XX/Y4ldXWY2m91uLTo/aps1bvaWzhjKrZao1tfZd+9rrb77rpmkb5i5abtUad7B773vINYaP5c+//rb+Q5+16k3ut+Vv7F9j+cxIhy+/+d6qh7a3XquH3XpaX7RP2jfto/Y1sxx99FFW4pbCdvJJJ9rqNWvtux/Xe3NS+vb79fbK62+6Ze8uU8KOzZHDTVdmXtt62z31rVufIfbBJ1+46Qq2Bo+aaCVDwdhL85fEDCYyi4KmvkPGWNFyddzvhgNfHffnprxkxcvXtZ4DhqcZtA0IXWOxjmGXXoNSrK/9nzF3sQvEhoyelHSerXn/Y2vTuZcLFBUsAwAAZKe4CmgWLH3NLr2xjNW570Hr3neo3ffAo3bxDXfYwBHjvCUOHmUiHxv4tI0cP819rnFPWZs9cbgtfXGszRg7yKqUL20vvDjP2nbp4+Znhfc+/NQFLHrCX/Daq23s4F7u9/Ua2PMB1+ZCpTQduvePmbF84+33bfjYF6x0iSI2dfSTbt37WzZMyrSnJTPSQU/4G7fvZktfW+3aufR4oJUtmj4m2X5oHzv1GOCWlbPznGFD+nRxyzSuU8VNU1uTBVNGuml6FS10nZsey3X5r7Tbi91k33y/zpa9/pZvd8iatmTFKve7WlbrhL357geuZEMBhLZR26rf1bb37trejjs2h9vveYtf9dbYp8rdd7hlVZXssosvcNOG9Q2lgbft3Tu2tJNOPMFNT41KmnT8FLAoACl3x61Jx1HHQcdD+oWC1q69B7vj5WfitDkuKIo8fi+MHODOK1EvcQqSI6kEauiYSS74ubXIjUm/q3TQcVRgNH7qLDoxAAAA2SuUgYsLH376RcIxZ+eP+Ro9cbq35IHr9dRI9536m16hTG5Crstvcq9xk2cm7N79nzcn0d///JPQb+iYpO0tV6t5wuat27y5idLzu6vWvO+WuaFk1YQvv/nem5qQsGPn7wkN23Zx81p06pmwZdt2b84+oYx2wh1VG7tlho55PmHv3r3enH3fq1frhx9P2Pn7Lm9OxhxoOvz5198JLR98zM0rWq52wieff+3N2efH9RsSKtZt5ZbRslonUjgd/eaFTZ4533cZbbOm31m9acLGX37zpu6zacu2hLtrt3DLaNmwyPTXtmkbo2lftE9aRi9tQzQdUx1bzdcx8aP0Urr5LfP6m+8mXHDt7W6e0lnpHUnHQ9sdPkahoMSbkyicLrHW/23z1oQ6LR5w82s0vT9h+47fvTmxz02ZtWCpm1eqSiPfdAUAAMgqcVNCo6ozqTmYpTR6yj174TL3RLxmpbtCr3KuClMklXA0q1fdlVBkhU+++NrmvLzcrrj8EmvZqJblOi1lz1eXX3KhPdC6kasqtXDpa/bLb5u9OfuopKNe9XvcMhmVGekQyvTb/CUr3HY8/nA7u+qKy7w5++hpv77jjluLWCiQsN8ysRrTbUULW6Hr89uylW/Z2z5V2lQytPqdD9wyWjYsnP7a7o4tG7ptjKZ9eahNk/1K2/RQNTG1b/n5102ulKp14zopStZ0PGpXudv1jqbjNGfRct+qZ7cUvt7urVo+xfrqQEHnl/bhh3UbbPPWlFUHjzsuh+U4Zl+32FKy+M326uwJ1vPB1ukqaQIAAMgscRPQvLY6sT1ILMpcqarQwbBp8xaXEZdK5UrZ8ccd695Hy3nqKVau1K3ep8z16RffuAzqpRedH8rQbg6l1zu+r507d9mF559ja9Z+bOt++tlbe598eS+1iy441/uUMZmRDh988rnLkKs6V8Frr/KmpnRXqeKuOtnzzwxw+5NZzj/3bBcoyfxQ0BeZ2VfAMH/xCpfOCma0bNg7az9x07Xd1159hTc1pZsKXmuFrsvvfcpcW7ftsI8//8q9V8B44gnHu/fRFKRULHu7C0re/+gz+2njL96cfa4MnQe5cp7mfUru9Fw57eILznPV2yKrj1128YV2d+kSrs1M935DXZrs+uNPN0+/pSp/emVVQAcAAOAnbgKa3f/9572L7b90LJMV/v7nX5fxvfrKy+38c87ypvq75KLzvHeZ6zevkb3aGakHrDuqNvZ9qccqBR3KfPs1fD/qqCPtyCP3rxvgzEiH8H5ccuF5B+VJvvb/rlCwpZIWZczVLiQssjOA24oVdsuGhTPuaW235l0SCjqzgo7ptu0705X+4aBE50KsXuYy6szTc1nfbh1cu51ps1+2ouVqW67Lb3K9rKlNT2RaAgAAZJe4CWgKX1/Ae+cvd87TXHUr4EDl+9+ldkeJIq7hvzoASPA6B1AnBX6dAWCf/112sc0YO9g+XDHLBvV62JUUbd6yzfoMHm2Fy9Sw6XMW+Xa2AAAAkFXiJqC5r2FN752/ts3qeu+yn3qvUvUePe1e71N9J5KqhqVF3fpmNNMXLhVoWrea7fjuHft340dpvjTafGbKjHTQU3757sefkko9spv24e47EqvELV7+hmtrtG3HTtcDnNxVqpirNhcpnP5pbff20PeEu1HObCo5ynnaKelK/81bt9n3635ypTl6GJCZ1E5HVdZaNaptU0Y9Yd++u8SG9O7iSlkHDBtrX3/3o7ckAABA1oubgOb2Wwq7bmz9KNjp3K6p9yn7nXF6bpcxFDXKjtUVrjLFqY3tEm5A/9FnX9n2nb+795EU5Hz8WWIblWg3Xne1y9CufOs9++b72BlGddestjSr1qx1VZQyU2akw7VXX+mqey1f+Za9+8Gn3tSUFr7yupWr1cIatOliG3/5zZuaeQrfcI2VLHaT6xxA3WGv/fhzt03RnQGEhdNfy4THn/Hz7gefuHYrWSFXzlMt/5V53Xt1kezX2F/U9mXRspXu+F9fIJ+dl0b1tPTYvfs/1xW0jokGCf13925vTmKAWP2eO+3mgte4ThWyKqADAADwEzcBjTSvX90+Xz3fHn2ojdWrUdE6tmxgr8wc557+Hkxq/H7PXSVdhlYj20+dtcBlGiOpQfnoidNdRjOWG6/L775DY8Wo8Xn0AIyfffmtTZoxx/uU3NVXXO4aeqtK1FPPTHCj1UfTtI6P9HdtaTTA4pFHZu7hzYx0UECkhuXqGKB7vyG+JTnaxz6DR7kqYBdfcK7lOeN0b07mOTMUnN0UyoDL4lffsAVLEjsDUIcBkZ0BhIXTX9v95NPP+Q5eqn3RgLCZHUiGqbG/OmNQQKiqXROmzU6R/jqnps952UaMm+KOU8U7b4/ZeUBGHHPM0ZY7FFDpmLy87HX77ofk7bN+//0P3yAdAAAgqx3VM8R7HxfUHXGxm25wgyaWKn5zpvZwFaZSDpViaIBDZfrUmDnWS90Gn5E7l517dh43+v2qNe+7bofV49pxxx7rSg/e//Aze6TfMBs1YbrLbKpKknojU+bzhOOP837V7LRTT7EvvvnePv/qW3v1jbftl18329GhjOL6Db+4LoE79xrkSm9EmXi1T8idK7G60LHH5nBP2pe+9qa9Edp+jaivzOwff/7ptnPlm+9Zxx4DkgarfKRTS9eNc5iqKI2fOtt3uzLinLPOPKB0OOboo9209z761DXKnxHKmKuzgf/27HG9sikdHn78KZcOKi3RfpzupUHYhp9/dd1Hb9663S4+/1zXW5eqi/2wfoNdcmFig3xVy9IyN1xzlZW+raj73UjqGOHEE05wmfMVq95x26Jt7tqhRSid83hL7aP0P+P0XLb8jTWuFGbR8jdcFat/QtuuamiTZy6wh0LbrWN52iknu9KTSneVSipRCVNFw2Wvv2nfuoDgCNeeZ9v2HS4AVGcCChr/+vvv0La/4pZpUKtSsgDr3LPy2N6EvaHtXum2QaVF6kJZnS0oIH78yWcSS1D+3W1N6lR13V9H7nta6SLqRGDm/KXufeQ5qI4GVJr11nsf2juhNDjl5BNd+xkd996DRobOyfesyI3XWfP6NULn+sluHQAAgCyXcBgKD8yYnlfkwIwajLJ9176+y+nVoE2XhFBG1L33G1hTQsFMssEXI1931WyeNPCj3+CF8ta7HybcXLZWinXDr6tuKZ8QyugmG1RTwoMixtqujMiMdPji6+/c/kavG37dWqFewqdffOMtndwP6za4+dHrRA5kGWtgzUihgCuhcftuSevrvabFojRdsWqNS+PwOpGvS28sk/D8jLlu4FN99htYU9+hQU+j141MJ/2NNbCmaH9CQYsbODP6e8Kv7n2H+O5LetIlPPhn9DmobQ8FUm4/o39PL03XAJvR5x4AAEBWiqsqZ/FOpTlPPvagLZnxrHtyrdIkTStzW1GbNuYpG/lED/cUPzXqqW3WhOHWq0s7y3vpRW5a8ZsLuvZD08cOcr1IpabwDQVs0bTRNurJnu539fvh6fpODW6o6UccsX9dM6dHZqSD9lP7O3FEv2T7EU4LjUGj0gs/KrWbMLyvG1wy/Nsq0ctoL3iRnQOIxlBJrXqW0vTWIje6NFZah8ej0XFs37yeLZs5zirfXTrVqn76jmb1qtmkp/u7EijR+hqY8oTj01c1TKU4alO2asEL7nfD55FK5hrWqmyvz5vkBrjMjKpmkbTtd95+i9vPds3qJg0uGrn/99xVKkvPPQAAgGhHKKrx3iMOqDpcycoNrXSJIjYhlNnP7B6qAAAAgEMJJTTZRI3Fc5xTwMrXuc+1/fCj2FJtFETtZY4/bv/auQAAAACHCwKabBLu4Wz1Ox/Y2o/9u/X98pvv7cV5i917VSHb34b7AAAAwOGCgCabFLzmKtftr7r07dCtn+vFKjyOiMb0UFWzjo8McL1t3VWquJUtWdzNAwAAABAbbWiykQYcbHZ/D1v9zlpvSkoF8v3PhvfvZjfdkDhGCgAAAIDYCGiymQa/HDNphg0dPcm2bt/hTU3sOaxutQrWqVXDTBnZHQAAADgcENAcJKpmtmPnLlu/4WfLc0ZuNzikuuMFAAAAkH4ENAAAAAACi04BAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAuuwDGh6DxplOc4pEPOVO+/NVrVRe1v86ir7599/vbWC56+//7FWDz3u9kn7HA9+3bTF6rfubFcXq2BzXl5mCQkJ3hwAAAAg4yih8fH7rj9s7qLlVr7OffZAz4G2dfsOb07m2rJtu91du4ULOKa8tMCbemibOmuh29evvv3Bnpsyy7bv/N2bAwAAAGTcYR3QNK5TxZa+ODbFa9H0MXb/fQ3s5JNOtJHjp1n/oc8GuqQmnpxw/HHeO7M9e/aEXnu9TwAAAEDGHdYBzXnnnGW3Frkxxev2Wwpb324dbET/7i6oUanCp198462FA1HhztutVuVylvfSi6zJvVUtd85TvTkAAABAxlHlLIYjjjjCSt1axIrceK39/Osm+/Kb7705OBB5zshtE4b3tU9WzrWKZUu6dAYAAAD2FwFNKo7NcYydnjuX98nf3r17bcWqNVareSc7K18x16GA2t7MWvCKq6a2+p21ro2M2sqozYyoDYmmnX1VcVuyYrWbpobymqZXwVLVXBuTaFr/6eemWKkqjdxy4d9SCdIff/7lLRWbtkfbpXW0rra3aYdHbM37H+9343zt/1vvfei+R9+n7bqzetOk/Y8W2W5IaRMWTqf0vCLTMtL6DT9bn8GjrVDp6m45bU/Dtl3t9TffddsZi9JOaajtDq+n/fnosy/ptAAAACDOxV1AM232y1axbiu7ssjdVqJifXvi6ee8Odnvh/Ub7L0PP7UrLr/ErrnqCm/qPuosoH23vla6WhObOW+J+6wOBdQ7Wo2m91uLTo/aps1bvaX3nzLVyvCXqdbU2nft6zLoEv6teq0etuqNO9g3369z0/3s2Pm76+BA26V1tK62d8K02Va2ZjMbP3WW/fffnsSF00nLDxo50crVauG+J9x5wvI33na/06XXoHQFWgdK6TN74St2W6WG1nPAcPvgky/cdG3PCy/OcwGgjlN4+yKp17UmHbq7NNR2SzhdSldtYjPmLiaoAQAAiGNHhDJrcZNba9ult2uEH+2Wwtfbgimj7PjjjvWmHBh1YfzowBHW44FW1rVDc2/qPnqav/bjz+2RfsNs6WurrXfX9taheX07+uijvCUSu0Tu/PhTrsREatxT1rUNOfGE4237jp02NRSYKcg5O88Zrspa6RJFbMKIfpY752m27qeN9v26Dbbrjz+t75DRroSke8f7rPjNBd13HX300VYgX17XfkcUwDRq182td/WVl1vDWpXd/D2h7Vy9Zq0Nf/YFlwkvX+Y2e3rAI65al2gbO/UYYGMmzXCf9X11q1Wwu8uUsGNCv6GMv+apNOiC886xaWOetBuuucotmx7vrP3EqjZq5/avSvnS7ruPCx2jlaHtHTrmeRc0qXqZ0iVMJSv1Q8GDSqZWzJlgRW68zk3fvHW7ffrF1+69H22jjtlvoQCxZcNa1rf7/Unnw7zFr1qDNl3c7ymdG9WuYrlznZYifVo0qGF9u93vjpHo1B8yepI9+OgTLm3aNr3XioWOwd+hdJs0Y647fgXy/c8mjxro2vwAAAAgDimgiQcTps5OOObs/DFfLTr19JY8cL2eGun7G9GvPFfekvD0uCkJf//zj7fmPktWrErIdflN7jVu8syE3bv/8+Yk0jr9ho5J+q5ytZonbN66zZubSJ81XfMnz5zvTU3u101bEirVb+OWqdygbcKGn3/15uzz3oefJhQtV9sto9/87789bvqff/2d0PLBx9x0beeUlxYkhII1Ny/s86++TVq3W58hSeumh7ZZ691du0XCpi379k2/MWjkBDcvFGgk7Px9lzcn+T6vWvO+NzV1X3/3Y8JdNRPXaf3w48m+T+lxR9XGbt5jTzzt9jna6nfWJuQvXtGlwdxFy72pydMnFOQmS5tQAJRQtVF7N2/0xOneVAAAAMSbuKlyNnH6HO+dv7EvzHQlDtnt519/S1FtStsxe+EyVyJQs9JdoVe5ZKU3cmyOHNasXnVXcnEgVq953+YvWeFKCnp3aWfnnHWmN2ef6/JfafVr3GN33FrE1v30syv5iVapXClXghPdCF/V6bSufPz5V7Zz1y73PiOOO+44O+qofaeSfuPeahVcF9gt6tewHDmO8eZknKqEPfTYk66kTNvf7f77kkqu5LXV77g2TCWKFnLdcPuV4t10wzXWrlldd7zmLFruWw0usjtpyXnqKfb4w23cPhS76QZvKgAAAOJN3AQ0n3/1rfcutm9TaSOyP2KNQ6PX7InD7bZiha3f0GddA3FlrMM2bd5in3yeWD1KgUKsqnDKFJcrdav3KeM0Rst7H37m3pctWcwuv8S/2pMCiKZ1q9mCKSNdV9OnnnKSN2efiy84L6mqVTQFNfLff/+l2ng+mqqnKdDSiP99Bo1yXVuHg87Tc53musAufEMBF9ztD6V5ywcfc1XKVDVNXWmHq9OJ0uezLxPPm4vOP9e+/u5HF+BEv1Rl78gjj7AzT8/ljtsvv21y6+i4qTqjDB872cZNfsk2/PxrUlsipYv2IZw+AAAAiD9xE9Dkynma9y6200492XuXOWKNQ6PXXaWK27ODHnftLpShHjl+atIgkH//8697yq/2LOeHviM1l1x0nvcu4/7dvTupIfv/Lrs4WSlIZtrf7738kgttWL+uVvDaq11blOtur2ynXnKj3XJ3HRs0coJt2rLNWzLjlL69B410aV/o+vw28okeKdqxRKbP+Kmz7I6qjWO+mnfs6drfvP/RZ8mC08p332FDenexXbv+sPseeNQuvuEOy5PvFqvaqL0L1BhQFQAAIL7FTUBT7o7i3jt/ha8v4AKQ7KQSjbrVKrqG/ctXvm0bfv7FmwNRydDNBa91jftXLZhsfbp1sDK3FbXPv/rOVRO77Z76tvKt97yl00/BTP+hY1wHEeqsoE/XDllWSqLSo/sa1rTP31zgSuU6tmxgl150vs1dtNyqNe7gOhuIDIAAAAAQX+ImoOncrnmqPUn1eLCV9y57qfrWGblz2SdffG0//7rZTTvu2Bwu2FH1pfUbUw9yVA1rf+U45hjLdVriSPoa2DNcQhRvtJ03Xne1dWrZ0Oa98Ix99fbLoePZ1PVMppIalYykl6p7qec4VfVTMDPmqcditmFRyVK47UvPB1vbPxs+tH83fpTmK9yzWiRVD1SpnHpBe3vxNFu7/CX3WT2djZ86K27THgAA4HAXNwGNAoeFU0a5ht+Rrr36CpdJLlX8Zm9K9tqxc5dt2rLVTjrxBDvmmKPdtDNOz+2qm4kGkIzVWcG2HTuTxjbZH8qw33BNPvf+5WUr7evvUg62KQkJCa77ZY0Hc3/3/q7xe1bTb+i39JtqexJJXVPXqVreta/ROD4/pRH0han9zvQ5L1u/IWNcw/9endtaiaI3pujIIEyB1PUFEtNHHQOo++hYvv/xJ9ee5u33PkqqRqZgs3KDtnZPvdauQ4Qw/d5VV1zmuuIWdeHt19ECAAAADr64CWjkgvPOtpnPDbHv31tqy2eNtw9XzLI1S6a7akwHg6o+TZoxx2WUNe6LSgxEjcnvuauky3RrhPmpsxakGJRSmebRE6e7J/wHokih6+3u0iXcqPVd+wyxjb/85s3ZR9W6+g8b63oCy5/v8mS9gGWVE44/3pVS6TcXLH0tRbWscCCYXgrKps9ZZG0693afn3j0AatavkzMYCZM7Z3Uw5kCGrVz8uvBTJ0FNO/U07WlWbR8pR19VGJgmivnqS4wW/jK66HpbyQ7htoeqpoBAADEv6N6hnjv48YpJ5/kghtV9coKCgD0tP6yiy9wmf8f129M8VLvWb2eGmmTps91y3Tp0NxVqwpT98mbNm+1VV63yhql/7hjj3UBx/sffuYG5Rw1Ybprf6On+2qXoR7RIrsHVmZdg2qqofq//+62K/NeYnv27LGXFiy1nKee6jpBUNCg71gR2t531n7sMt+7//vP/vnnX/vux5/shRfn2wM9B9ovv222imVLWocWDZJ6M1MGfcmKVe77lekPD9wZTdXmxk+d7buNsajXMJWqLX3tTbddql6mUiwFfyvffM8ef+oZN63M7bfYvdXKJ/UE99fff7tR/b/9Yb01qFXJzj/3bDf9zXc/CAUzfVyaVrjzdrvtlsKuxzG/Y6PXSSee6LZTx0bppBIstXNaG9pXBSPbtu90y815ebkbOFMlRepcoEv7Zq63M9H27vh9ly15dZXbhx07f7cjjzrSDXo6eeZ8e+qZ8a70rcm91Vy1t7SCKwAAAGS/I0KZvwTv/WGj96BRbtT59FCG+fHOba1Z3eopxprR030FLiPGTfamJKdqV/eUvd01LtcI9hNG9HPVsSJFjnIfFj06vQ6RMvxtQxl+ldT40Xg3Tz76YLJxapQZ79RjgKuO1uOBVtY1FJT5Wf3OWitRsX7MbYxFAZNKsDr1GOhbzU37Mbx/NzcOTNiWbdutfquHQ4HWateZQLg9y5SXFlj91p3d+/SIXFfpo0b8KsFSEOVHyw7q9bAbsyeSeknr2nuwG+fIj47hgB6dQsF1Tm8KAAAA4klcVTmLJ6pe1rBWZXt19ni7r0HNFMGMKNh58rEHbcmMZ11AoQb8mqYqctPGPOW6Gj7DKw2Ipdwdt9r0sYPs9lsKu8/6XQUWuXPtCypUMqAM+eIZY2xw785JJS2RvzV+WB/fQTezktKkQc1KLo00OGe4AwO1e+rVpZ29PG10smAmqyh9VDql8YP6P9IxWfooXYf17ebG6IkOZkTbrDRVGoaPQYpjSDADAAAQtw7LEhoAAAAAhwZKaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAOiwDmt6DRlmOcwq4v0ju101brH7rznZ1sQo25+VllpCQ4M0BAAAA4g8lNEhm6qyFNuWlBfbVtz/Yc1Nm2fadv3tzEC0cGLd66HH76+9/vKkAAADITgQ0SOaE44/z3pnt2bMn9NrrfQIAAADiDwENkqlw5+1Wq3I5y3vpRdbk3qqWO+ep3hwAAAAg/hDQIJk8Z+S2CcP72icr51rFsiXtiCOO8OYAAAAA8YeAJoP27t1rb733oTXv2NMuK3Sna0OhBvQ9Bwy3H9dv9JaK7Z9//7VZC16x8nXus9x5b7az8hWzWs072YpVa9x3q/2KX7sMvdc0zdMyP238xR7oOTBpGzTNb7k//vzLRk2YbqWqNHLTtLy2/aPPvvRt8L9l23a7u3YLt+zqd9Z6UxPps6Zr/uat2913RKaDfkO/pd9MjfZT+6v91v4rHZQear+jddV+p2Cpau6l9+mV3jQKi7UdOj46TpH7q3SR8LZp+qMDR7hpYybNsFMvudFN8/sdWb/hZ+szeLQVKl3dLaPfa9i2q73+5rtuO2JReihd7qzeNGm9ph0eiXn8AAAADjdxGdBs+PlXe+Pt9+3zr771psQHZS4VuBQvX9eem/KSrfspMYBRJleZ1cJlatik6XPtv//2uOnRtPy99z1kNZreb4tfXWW/7/rDtm7fYTPnLbHS1Zq479a0tKz9+HOr2ayjDRk9KWkb/PwUSsd6rR62Np17uYyzaHlte+mqTWzG3MX7lSlWu5oXXpxnt93TIFk66Df0W006dHe9pfkJp6H2V/ut/dc+Kz20rXVaPGDfr9vgLb3/0koj/W77bn19t0PHp0WnR23T5q3e0vtP6Tt74St2W6WGbr8/+OQLN12/pzRUEKjt0OdoSkOlpdJl+Rtvu2labsK02Qd0/AAAAA4lcRXQrPvpZ6vSsJ1dfMMddnulBnZNiUruibYymQebntg/9sTT1m/os3bySSday4a1bOHUUbb0xbE2e+Jwq3FP2aRM8ovzUmY0N23ZZh17DHRdIUev/9zQ3la6RBH33enpSnrwqIn2zXfr7JFOLW3R9DHuVSDf/7y5+3TtPdhWhQLD8HL6rV5d2lmu00512zpg2Fj7+rsfvaXTb9nKt1zJx80Fr7Gpo5903/vS+KFW7o5b3XwFCOOnzkrRoYACvaefm+L2U7R8eH2lhdJk5VvvWcdH+ofS68CCidTSSCU5jw182kaOn+Y+69jpGGo7ZowdZFXKl3bBRtsufdz8SGfnOcOG9Onilm1cp4qbpnZHC6aMdNP0KlroOjdd5i9ZYY3bd3dBlY5xeH+1Pdo2HQtth4KdyJItnT8q6VFa6nzp2qG5W0fbqe07kOMHAABwSAllnOLC9h2/J1x1S/mEY87O7/ta+tpqb8kD1+upke479Te9lqxYlZDr8psSLr2xTMLLy1Ym7N2715uTaPfu/xKGPfu8+95bK9RL+GHdBm9Oglv22edfdPNirb/z910JrR9+PGl/Wz74WMKff/3tzU1w7zUt/B3LV76d4jskPcu9+e4HCfmLV3TLhDL+3tREm7duSyhXq7mbt2rN+97URPqs6XppW7XNkfTboUDHzS9evm7Cj+v3pYG8+8Enbps0v9/QMQl///OPNyeRtjOUiXfprGVuKFk14ctvvvfmpi29aRQ+lnqNmzzTHbtI2i5tn75HL6WH0iVa+DyKPlZhG37+NeGOqo3dMqFg2HeZ1e+sdcdC2zJ30XJvavJ9CQW5yfYjFMwkVG3U3s0bPXG6NxUAAODwFDclNH2HjEq1vcSjAxLbKxwMeqI/e+EyVyWpduW7rFTxm1M0lj/66KOsXvWKdm+18q7txfI33vLmqF3KDlftSJrWrWp33Fokxfp6Ct+pZSMrcuO+p/uxVChzmxUpdG2aDfarVyhjxW66IcVyha8vYDUr3eXef/vDOgtl4N379Lrs4guscZ2qbpsjHX/csa5nNJWEfPLF1/bzr5u9OYnV1FRaoZIKpdF9DWrasTlyeHMTaTsrlStl9Wvc403Zf7HSKPJYKg1qVirnjl0kbVezetVdSciBeG31O66NTomihVxpjtIn2k03XGPtmtV12zNn0fJkpTRhkV1pS85TT7HHH27jSnp0fAEAAA5ncRPQLFj6uvfO39vvf+QaeR8M23fsTAq28px5uq1a877LrEa/1G5DVYhE78OBgjLxH332lQsE7ipV3I46yj/ZLzjvbCtZ/CbvU2zXF8iXIhjwkz9f3hSZdVEm/+ILznXvFWhktBnGJReeZ+efe5b3KbncuU6zs87M7TLoe/bua0u0c9culyaigO6Uk09y76Npv24vVtj7tP9ipdGmzVvsk8+/du8VPPkFGaKgoVypxCp0+0Pp+tmXiW3ALjr/XFc1zO+cUbujI488ws48PZfbrl9+2+TW0XbdUvh693742Mk2bvJLrm1ZuH3WFZdfYrcWudH9BQAAOJzFTUCz1etFKjXbdxycUeuVOd+2fad736FbP7ujauOYr6FjnnfLfffD+qSn7Qpsfv51k1143jmhzP4ZbpofBRoKetISKyCKZ2pg/9PGX937C88/x/2N5YzcuVKU/mRUrDT6+59/3XG5+srL7fxz/IOysEsuOs97l3H/7t6d1NB//NRZvudK+KWe4n4Lpc/7H32WrDOFynffYUN6d7FdofPvvgcedW3L8uS7xao2au/aYqldFwAAwOEubnLGV+a91HsX26XpyOwDhwqVMN3XsKZ9/uYC1xlAx5YN7NKLzre5i5ZbtcYdrEGbLjF7kwMAADhcxE1Ao/YnqYnVBiE7HHdsDjvxhONdqcGKORPs340fpfmaP3mk5c55mrf+sa53rB9/2phUpchPZDWlQ80Zp+ey887J496nNV6PejhTqVhWCB9LVe9an0YVxk+/+MZ7l3EqIQq3fen5YGv7Z8OHvudJ9MuvDZWqv6mqYt9u99vbi6fZ2uUvuc+xepMDAAA4nMRPQFOjorVoUMP7lJzaEjz1+MPep+x3xum5XRUlZbIXLVsZc5wZTVe1IbWN+OLr77ypahtzjhXIl9e++X6dLXzl9ZgZULURUrfFh6JTTjrJ8l+Z171f+tpq2/n7Lvc+mqpRLV+ZOOZKVggfS9EAmuokwM+2HTuTxn7ZHzmOOca14xF1DKAqh7F8/+NP7px5+72PkqqRKZiq3KCt3VOvtX38+Vdumqha4lVXXOa6mha1S9r1x5/uPQAAwOEorhpjDO3T1SY93d/Klixml150gXta3adbB1s+a/xBK50R/fY9d5V0JTSTX1po85e8mmKcGX1Wu4ZK9du4jOjGiAxs7pynhtYv5d6PmfSiy9BHr69g6Ymnx6UYnf9QoRKL0rcVdSVVz8+YZ8+Mn5qiDYjSREGGBo7MKpHHUiPwT521IEWAqu0aPXG6KwE5EGq0rx7OFNCMDO2vXw9m6iygeaeeri3NouUr7eijjnbTc4XOGZ0TCoAXLX8j2TYqnahqBgAAkCiuAhrRk+c5k0bY56vnu+pdnVo29OZkPpWIRPY45ff68NMvXIlK8ZsLui6J1WOZBkps07m3y2xqmVdef9N1FtCsYw/3JL5O1btd18hheqquwRcrli3p1tdo+Fpe62n9SdPnWsW6rWzUhOkuw3+ourngtda6SR33vnvfoVazaUd7af7SZGnY6qHHXTuRrEyHWwrfYHWrVXABgxrkN2zbJelYanu0Xdq+tLZBvb2JBn59ObS+ggwFH9oXOeesM61NaH8VPGkw0RpNOrgBO/U7eg0ZPckFwAp4Cl2f35374c4MzjrzdHfOSL8hY6xb3yGuxEgvDb6qaVK00PV2yskH1oECAABAoCUchsIDIqbnFTlo4q4//kzo0X9Y0sCPfq82nXslbNm23S0fTQMt1mzW0Xc9vbr3HZIwasI09z56sMbIgRYnz5zvTU0pvctpnt/vpGdgzVgDTUpq64vSUPup+X6vBm26JKx86z03qOaBDKyZ2r6LBgVt37Vvit8Pv7Qdsxe+4t7H2l8NnqpBVKPXjfxtDYip70lt0Fh9x/sffeatsY/Ooxadevquo5e28bfNW72lAQAADk9xV0ITz9SY/JFOLe2VmeOsdeM6lvfSi9x0tZGpWqGMLZnxrA3u1TlpLJpoemI/flgfmzbmKStzW1H35F7LagBHravG45p2KFMaaj+1vyqBiEyDeS88YyOf6GGnnnKS/flXyupZmUm/++RjD7rt0G9rGzRNx0XHR9uhjgxSo+6nJwzv6zqsCK+vfYocG0alcyqZ0yCY/R/p6Er6RMvefkthG9a3my2YMtKuy3+lmx5J3zm4d2e3PVpWUmxj7pxuOgAAwOHqCEU13nsgLqjK1l01m7sqapOe7ucCRgAAAMAPJTTIFuq9Lcc5BezC60raO2s/8aampMbvr69+x72/5KLzLedp/qVdAAAAgBDQIFtcdvEFVrLYTa7ThEkz5vj2+CXq5U09yUnxm29wVawAAACAWAhokC3Ua1f9mve4AGXk+GnWolNPN7hluDtijfsyfuosa9Sum+sJTu1awl1dAwAAALHQhgbZRsGLuiruM3iU6zI5ljtuLWJD+nRxpToAAABAaghokK327t1rr7/5rgtsFix9zZuaSI3/NW6LxvtRb2gAAABAWghocNDs+uNP27HzdzcgpYKZ00452Y4++ihvLgAAAJA2AhoAAAAAgUWnAAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQBPy3397bPU7a+3+7v2tUOnqluOcAu6l92279Lalr622v/7+x1v64Pt10xar37qzXV2sgs15eZklJCR4c7LewfxtAAAAINphHdAoM/7qG2usdLXGVqJifRs+9gX74JMvvLnm3o8cP83K1WphJSs3tGUr37K9e/d6c7PGV9/+YAVLVXMBlYIsP1NnLbQpLy1wyz43ZZZt3/m7NyfrHczfBgAAAKIdtgGNSmWeGT/VqjZqZ2+8/b6dfNKJVuOesjZj7CBb+uJY95o9cbjd17Cm5TrtVHv3g0+sbI1m1nvQqINeWnPC8cd578z27NkTemVtkBXpYP42AAAAEO2oniHe+8OGMuEqjek5YIT9vusPu7daeZsy6glrWLuyXXH5JXbR+ee61+WXXGhlSxaz+jUrufXefPcDe/3Nd13wU/j6AnbkkZkfD27Ztt1mzl/qqnY1qFXJzj/3bG/OPuedc5b98ttm+/ff3fZAq0Z2Xf4r7YgjjvDmZq2D+dsAAABAtCMSDsNGEKrKVa9VZ1v300br3vE+6xTKmB9/3LHeXH8q0Rk9abp17zvUfR4/rI+VL3Obe5+ZVJWrdvMH7KPPvrQVcyZYkRuv8+YAAAAAiHbYVTn748+/bNzkl1wwU6V8aWvdpE6awYwcffRR1rBWZatbrYIr1Zk4fa5t27HTm5tIbUvU9uXu2i1cScumLdtswPCxSR0NqCH9g48+YT+u3+itsY+qsoWXUTAjatejaZHfKfqrz5oe3c5Gn8PL6/dXrFpjtZp3srPyFXMvvde0cFugf/7912YteMXK17nPcue92S3TtMMjbhv8Yt1Yvx3+3fS8Ivcl0voNP1ufwaOT0kvb0rBtV1cqFqvtUuT+qlRr9sJX7M7qTVP8jvbli6+/swd6DrTLCt3p5t9ydx0bNWG6OycAAAAQTHEX0Hz7wzrrO2SMNenQ3Tr3esplZjPTl998b0tXrHbVxprWrWY5Tz3Fm5M2BT6N763iqqUtX/mWrf34c29OSu9+8KlVqt/auvUZktTRgEpfBo+aaCWrNHIZ76wsHPvzr79twLBnrUrDdjZz3hLbun2He+m9po2Z9KILeNp37Ws1mt5vi19d5QI1LTNh2mwrXbWJzZi7OFt6MdNvKD1uq9TQeg4YnpRe2pYXXpxnpULp1b5bX/c5Fu1v/6HPWvUm99vyN972pibS92tfFCAOGT3JBbOy5v2PrU3nXu5cUzAEAACA4ImrKmd6Wq4MZjQ1zB/Su4v36cCMmTTDWj30uJW5rag9N6yvnZ7rNG9O+qhDgE49BrjvebxzW3uoTRNvTmIJjbo0vuC8c+y0U062nzb+4kqAihS6zo468kj76LOv7LkpL9knn3/tlhk3pJcVv7mgW1elB8pUb/zlN3v8yWfsm+/X2bC+3ULB08Vu/nHHHmvX5r/Cjs2Rw5U61G/1sC0JBWbR1dJUYqGMe5g6OqhVuZydeMLxLlDQdiuw0u8XyJfX5i9ZEXMZBW5TRz9p+f53qfdtiSU0fr+9eet2+/SLr917P/q+RweOsN82b7WWDWtZ3+73J5WMzVv8qjVo08UFVKVLFLFGtatY7tBx2bN3r61es9aGP/uCC2ZaNKhhfbvd77YzLHp/o9cvdF1+2xwK3O5t+aALYG4tcqPd16Cmm699HRb6bgU4OpadWjayo446rDv+AwAACB4FNPFg2cq3Eo45O3/MV5/Bo70lD0yvp0a67wsFTgl//f23NzVjwt/R8sHHEv78a993TJ45P2l7i5arnRAKXLw5+2z4+deEyg3aumXurN40IRTAeHMSffnN9wk3lKzq5q9a8743NbnNW7cllKvV3HcZfQ5vwwM9BybbPvn4s6+Svl+vAcPHJuze/Z83N1HkMoNHTfSmJkrtt2P5+rsfE+6qmbhO64cfT9j5+y5vTmJ63FG1sZv32BNPp9heCQUtCfmLV0zIdflNCXMXLfemJorc3+jvDgsvo31S+kaatWCpm1eqSqMUxwIAAADxL24eRz/z3FTvnb+hoyd57/afejf755/ELpfznHG6K/XYH5dceJ73Lja1t4ks2Qg756wzreeDrVzph8a1efu9D705mUvf36BmpRTtg6664jLXq5sUuj6/VStfxrUPinRl3ktd726ikpUD6aZapU4PPfakG5xUnSh0u/8+V90v7LXV77g2PSWKFrLGdar4tme66YZrrF2zuq4EZ86i5b5tXs7Oc4bVq35Psu+OdtxxOSzHMcd4nxKVLH6zvTp7QuiYtLaTTjzBmwoAAICgiJuA5u33P/Le+VNVJ1XLOhCqTnSsF8T8ummz/e0FNxn13Y8/ee/8Fcj3Pyt20w0xuzP+32UXuypv8t6Hn2XJWC4XnHuW5Tkzt/dpH21TnjMSp6ta3MknpwwAlE5+wVhGKZhp+eBjrkqZqqb17dYh6bdF+/3Zl9+69+om++vvfnQBTvRL7aiOPPIIO/P0XK663i+/bXLrRMoXCsIuuuBc71Nyl118od1duoSrcta931B7Z+0ntuuPP908BUBFC13nXqkFQwAAAIhPcRPQHHP00d672I5OxzJpUaZYvg8FJbv+yHjvViqtUDsXueC8s31LFM4KBRJqoxGLSgnyXnqRe6+2If/u3u3eH0pUitJ70EgXzKgkaOQTPZL2OUz7HW7oP37qLLujauOYr+Yde7r2N+9/9JlvA34FYQp6/OiYK5gqd8etNm32y1a0XG3LdflNrkc5dULg1+scAAAAgiFuAppbiyQ2jo9FT98vu/gC79P+u+Gaq1z1pNXvfJDUPXJGfPP9j7byrffc0/wbr8vvTUUkBTP9h46xkeOnuc4H+nTt4KrAHUwqFZsxdrB9uGKWDer1sOuyW50FqJvowmVq2PQ5i7KlRzcAAABkrrgJaNo3r+e986dR6TODMrZ3lCji2mOoN6/osWRSo9KZsc/PdFXfbi92kxsl388vv22xLVtTjrMSppIJtU2RXKedmqJdR5BpANKnn5ti/YY+64KZMU895qrf+VGpygnHH+feqw3LPxs+tH83fpTma38HG1VbIbUPatWotk0Z9YR9++4S13ve7v/+swHDxroqbwAAAAiWuAlo1O5k1oRhdu7Zebwp+/Tu2t6NGZMZ1OVvo9qVXWZbY7KoS+D0NHpXRl1dLk+aMdeVztSrXiHmGDYq+Vm15n3vU0rf/bDeXn1jjXt/wzX5DpmugjX45fQ5L1u/IWNcGvXq3NZKFL0xZlsiBXLXF8jn3qtjgJ9/Tdk2JkxVBNWe5u33PnKDgabX7t3/ueCqXK0WbvDSyOp9Oheq33On3VzwGnfMwkEmAAAAgiOuctJq4/DtO4vthWcGuHFBnhnYw75/b2mmlc6E3VzwWuvaoZnLdGvMl1YPPZZqOwoNQNmt7xA3CKVKdh5u19TKlizuzfU39oWZ9ukX33if9lH7m659hrgMtBqqFyl0vTcn2FRdS9W22nTu7T4/8egDVrV8mZjBTJjGhVEPZwpoRo6f6tuDmUpOmnfq6drSLFq+0o4+Kv1tqY455mjLnfNU18vay8ted8FkpN9//8O27/zd+wQAAICgibuigSOPPNKqVbzTDVipbnz9SmwOlDLZdatVdEGTgprnZ8yz60tWsbotH7I5Ly9L6l1r4SuvW7uufSx/sQr21DPj3brdO95nrRvXSdHVcSR95w/rNljFeq1dqYBGrtf3aRDHu+vc5xrKq4RI1ezCnRSE5cp5mp2d53T3fvyU2fbtD+vcK6PV47Lbm+9+EAr6hrqAr8ztt7j9UylVOC2jXxqIU9SNdZsmdVyaqSSlRpMO9sKL85KW08j+leq3cQGPOhfQIKAZLdEqXaKoq6amXs4atetmM+Yuct/90vyl1vrhx910zc9/ZV5vDQAAAATFoVHXaT8oINGI8S+OG2K3FL7eZcTVA1a1xh2Seta6JxSQaHwc9cRV8Nqr7eVpo61rh+a+PZtFUhWmsUN6u97ONDr+ndWbuu/r+Eh/1+2wMvtP9Ozk27ZEpQmlb7vFvR8/dZZdWeRu91KgpSpd8UolXBpxX16cu9jK1miWlI5+r6++/d4tKyqpGjekl+sFbcmK1dawbdek5R7oOdBVBVPAMaxvt/3qXEC90alUTen+7gefWJ0WD7rvrtmso/u9xOCyrlsOAAAAwXLYBjSikprbbilkS2aMtRVzJriSl2uvvsKba+59iwY1bMGUkbbspeesZLGbXAlSehS89iqbM2mEDe7d2ZUsiDLsKpVZNnOc3XNXKd/qWJrWrF41m/R0/2TraQDIE44/3n0+1GifK5YtaUtfHGv9H+loxW9O7PFOpTa331LYBTI6BrE6YUiLvv/O229x6a4BOhXASHqOBwAAAOLbEQn0VZtppry0wOq37mylSxSxCSP6We6csceiAQAAAHDgDusSGgAAAADBRkADAAAAILAIaAAAAAAEFgENAAAAgMCiUwAAAAAAgUUJDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAituA5q9e/d67wAAAADAX9wFNH2HjLF8Re+248671s4rUMJaPfS4/fLbZm8uAAAAAOxzREKI9/6gq9qovc1dtNz7tM/FF55ni6eNsYsuONebAgAAAABxVEIzZPQk32BGvv/xJ3vo8Se9TwAAAACQKG4CmmmzX/be+Zu14BXbun2H9wkAAAAA4iig+WHdT9672DZs/NV7BwAAAABxFNCce/ZZ3rvY8px5uvcOAAAAAOIooKl89x3eO3933n6LnXl6Lu8TAAAAAMRZL2elqzWxFavWeJ/2yXXaqbb0xbGWP19ebwoAAAAAxFEJjSyaNtraNatrp51ysjfFrFK5UrZizgSCGQAAAAApxFVAc+SRR9rAng/Yb1+ssvUfvmp//Pi+TRvzlF1x+SXeEgdu05Zt1uz+HnZWvmI2eNRE+++/Pd4cs2++X2eV6rexywrdaS/OXWzhwiv9nfPyMru6WAW7s3pT++rbH9x0AAAAAAdXXAU0kfKckduOOeZo71PmWbHqbRs/dZbrAnr0xOn2+dffJs4ImTl/iS1Y+pqt+2mjjQrN++W3zW769p2/23NTZrlAZvkbb9uk6XNtz569bh4AAACAgyduAxoAAAAASMtRPUO894eFM0/PbVu37bB1P/1sD7drZqVLFHVV3eSsM0+3H9ZvsCOOOMI6t2tqN1xzlXt/3LE57ITjj7MPP/3Sbip4jT3QurGdnus0tw4AAACAgyeuejkDAAAAgIygyhkAAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAACBRUADAAAAILAIaAAAAAAEFgENAAAAgMAioAEAAAAQWAQ0AAAAAAKLgAYAAABAYBHQAAAAAAgsAhoAAAAAgUVAAwAAACCwCGgAAAAABBYBDQAAAIDAIqABAAAAEFgENAAAAAACi4AGAAAAQGAR0AAAAAAILAIaAAAAAIFFQAMAAAAgsAhoAAAAAAQWAQ0AAACAwCKgAQAAABBYBDQAAAAAAouABgAAAEBgEdAAAAAACCwCGgAAAAABZfZ/9NfVzerKx+MAAAAASUVORK5CYII=)"
      ],
      "metadata": {
        "id": "zUDJm_O-NJME"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's tune the hyperparameters of a binary classification model that does well classifying the breast cancer dataset.\n",
        "\n",
        "we've seen that the first step to turn a model into a sklearn estimator is to build a function that creates it. The definition of this function is important since hyperparameter tuning is carried out by varying the arguments your function receives.\n",
        "\n",
        "Build a simple create_model() function that receives both a learning rate and an activation function as arguments. The Adam optimizer has been imported as an object from tensorflow.keras.optimizers so that you can also change its learning rate parameter."
      ],
      "metadata": {
        "id": "tlsoKe7iM0uN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "a sample of hyperpaeamether optimization in sklearn"
      ],
      "metadata": {
        "id": "lYK5ozzvPKLp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "# Instantiate your classifier\n",
        "tree = DecisionTreeClassifier()\n",
        "# Define a series of parameters to look over\n",
        "params = {'max_depth':[3,None],\"max_features\":range(1,4),'min_samples_leaf': range(1,4)}\n",
        "# Perform random search with cross validation\n",
        "tree_cv = RandomizedSearchCV(tree, params, cv=5)\n",
        "tree_cv.fit(X,y)\n",
        "# Print the best parameters\n",
        "print(tree_cv.best_params_)"
      ],
      "metadata": {
        "id": "sbmNLXufNsrA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "to perform randomized search on a keras model we need some paramethers to try\n",
        "for example:"
      ],
      "metadata": {
        "id": "Qv9qi3vtQNpw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Random search on Keras models\n",
        "# Define a series of parameters\n",
        "params = dict(optimizer=['sgd','adam'], epochs=3,batch_size=[5, 10, 20], activation=['relu','tanh'])\n",
        "# Create a random search cv object and fit it to the data\n",
        "random_search = RandomizedSearchCV(model, params_dist=params, cv=3)\n",
        "random_search_results = random_search.fit(X, y)\n",
        "# Print results\n",
        "print(\"Best: %f using %s\".format(random_search_results.best_score_,random_search_results.best_params_))"
      ],
      "metadata": {
        "id": "H435Rk1ePzM0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "how to optimize the number of layers and number of neurons per layer"
      ],
      "metadata": {
        "id": "jCBfbq8dYac3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "params = dict(nl=[1, 2, 9], nn=[128,256,1000])"
      ],
      "metadata": {
        "id": "Sxogf3x4Ys4g"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def create\n",
        "_\n",
        "model(nl=1,nn=256):\n",
        "model = Sequential()\n",
        "model.add(Dense(16, input\n",
        "_\n",
        "shape=(2,), activation='relu'))\n",
        "# Add as many hidden layers as specified in nl\n",
        "for i in range(nl):\n",
        "# Layers have nn neurons\n",
        "model.add(Dense(nn, activation='relu'))\n",
        "# End defining and compiling y"
      ],
      "metadata": {
        "id": "ylBRSdHkYfks"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "we can optimize our model by hyperparamthers in sklearn but first we need to change keras model to  sklearn estimators"
      ],
      "metadata": {
        "id": "Fpuy0ZLbNgk8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Integrating Keras Models with sklearn: By defining a function that creates a model and using the KerasClassifier wrapper from TensorFlow's keras scikit-learn wrappers, you transformed Keras models into sklearn estimators. This allows for leveraging sklearn's cross-validation and hyperparameter search techniques on Keras models."
      ],
      "metadata": {
        "id": "HpMzUvycYi1v"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.layers import Dense\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.optimizers import Adam\n",
        "# Creates a model given an activation and learning rate\n",
        "def create_model(learning_rate, activation):\n",
        "\n",
        "  \t# Create an Adam optimizer with the given learning rate\n",
        "  \topt = Adam(lr = learning_rate)\n",
        "\n",
        "  \t# Create your binary classification model\n",
        "  \tmodel = Sequential()\n",
        "  \tmodel.add(Dense(128, input_shape = (30,), activation = activation))\n",
        "  \tmodel.add(Dense(256, activation = activation))\n",
        "  \tmodel.add(Dense(1, activation = 'sigmoid'))\n",
        "\n",
        "  \t# Compile your model with your optimizer, loss, and metrics\n",
        "  \tmodel.compile(optimizer = Adam, loss = validation, metrics = ['accuracy'])\n",
        "  \treturn model\n",
        "\n",
        "  # Import KerasClassifier from tensorflow.keras scikit learn wrappers\n",
        "from tensorflow.keras.wrappers.scikit_learn import KerasClassifier\n",
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "\n",
        "# Create a KerasClassifier\n",
        "model = KerasClassifier(build_fn = create_model)\n",
        "\n",
        "# Define the parameters to try out\n",
        "params = {'activation': ['relu', 'tanh'], 'batch_size': [ 32, 128,  256],\n",
        "          'epochs': [50, 100, 200], 'learning_rate': [0.1, 0.01,0.001]}\n",
        "\n",
        "# Create a randomize search cv object passing in the parameters to try\n",
        "random_search = RandomizedSearchCV(model, param_distributions = params, cv = KFold(3))\n",
        "\n",
        "# Running random_search.fit(X,y) would start the search,but it takes too long!\n",
        "show_results()\n",
        "\n",
        "# Import KerasClassifier from tensorflow.keras wrappers\n",
        "from tensorflow.keras.wrappers.scikit_learn import KerasClassifier\n",
        "\n",
        "# Create a KerasClassifier\n",
        "model = KerasClassifier(build_fn = create_model(learning_rate = 0.001, activation = 'relu'), epochs = 50,\n",
        "             batch_size = 128, verbose = 0)\n",
        "\n",
        "# Calculate the accuracy score for each fold\n",
        "kfolds = cross_val_score(X, y, model, cv = 3)\n",
        "\n",
        "# Print the mean accuracy\n",
        "print('The mean accuracy was:', kfolds.mean())\n",
        "\n",
        "# Print the accuracy standard deviation\n",
        "print('With a standard deviation of:', kfolds.std())"
      ],
      "metadata": {
        "id": "kWmFeUfGZaki"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Function that creates our Keras model\n",
        "def create_model(optimizer='adam', activation='relu'):\n",
        "model = Sequential()\n",
        "model.add(Dense(16, input_shape=(2,), activation=activation))\n",
        "model.add(Dense(1, activation='sigmoid'))\n",
        "model.compile(optimizer=optimizer, loss='binary_crossentropy')\n",
        "return model\n",
        "# Import sklearn wrapper from keras\n",
        "from tensorflow.keras.wrappers.scikit_learn import KerasClassifier\n",
        "# Create a model as a sklearn estimator\n",
        "model = KerasClassifier(build_fn=create_model, epochs=6, batch_size=16)"
      ],
      "metadata": {
        "id": "jITHQgbgLSe_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# for example using crossvalidation\n",
        "from sklearn.model_selection import cross_val_score\n",
        "# Check how your keras model performs with 5 fold crossvalidation\n",
        "kfold = cross_val_score(model, X, y, cv=5)\n",
        "# Print the mean accuracy per fold\n",
        "kfold.mean()\n",
        "0.913333\n",
        "# Print the standard deviation per fold\n",
        "kfold.std()"
      ],
      "metadata": {
        "id": "hyIlFH7zPiLQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Tensors, layers and autoencoders**"
      ],
      "metadata": {
        "id": "IAoX6MWV7UPb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "With this you can go through our layers. This is a useful tool when we want to obtain the output of a network at an intermediate layer.\n",
        "\n",
        "For instance, if you get the input and output from the first layer of a network, you can build an inp_to_out function that returns the result of carrying out forward propagation through only the first layer for a given input tensor."
      ],
      "metadata": {
        "id": "vf5LsqtUfFJJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**It can be used to extract intermediate layer outputs for visualization, debugging, or other custom operations.**"
      ],
      "metadata": {
        "id": "0ID81ssxgmAa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import tensorflow.keras backend\n",
        "import tensorflow.keras.backend as K\n",
        "\n",
        "# Input tensor from the 1st layer of the model\n",
        "inp = model.layers[0].input\n",
        "\n",
        "# Output tensor from the 1st layer of the model\n",
        "out = model.layers[0].output\n",
        "# Define a function from inputs to outputs\n",
        "inp_to_out = K.function([inp], [out])\n",
        "\n",
        "# Print the results of passing X_test through the 1st layer\n",
        "print(inp_to_out([X_test]))"
      ],
      "metadata": {
        "id": "7FB2m4Hp7UwS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for i in range(0, 21):\n",
        "  \t# Train model for 1 epoch\n",
        "    h = model.fit(X_train, y_train, batch_size = 16, epochs = 1, verbose = 0)\n",
        "    if i%4==0:\n",
        "      # Get the output of the first layer\n",
        "      layer_output = inp_to_out([X_test])[0]\n",
        "      test_accuracy = model.evaluate(X_test, y_test)[1]\n",
        "\n",
        "      # Plot 1st vs 2nd neuron output\n",
        "      plot()"
      ],
      "metadata": {
        "id": "22aiiDTkvAPB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**autoencoder**"
      ],
      "metadata": {
        "id": "JZSu0tYYxLqR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Autoencoders have several interesting applications like anomaly detection or image denoising. They aim at producing an output identical to its inputs. The input will be compressed into a lower dimensional space, encoded. The model then learns to decode it back to its original form. It is a method for dimentionality reduction, De-noising our data and help with anomaly detection."
      ],
      "metadata": {
        "id": "bvKnwYzFxPOp"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Building an autoencoder\n",
        "Autoencoders have several interesting applications like anomaly detection or image denoising. They aim at producing an output identical to its inputs. The input will be compressed into a lower dimensional space, encoded. The model then learns to decode it back to its original form.\n",
        "\n",
        "You will encode and decode the MNIST dataset of handwritten digits, the hidden layer will encode a 32-dimensional representation of the image, which originally consists of 784 pixels (28 x 28). The autoencoder will essentially learn to turn the 784 pixels original image into a compressed 32 pixels image and learn how to use that encoded representation to bring back the original 784 pixels image."
      ],
      "metadata": {
        "id": "9j472BhSfv4F"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Start with a sequential model\n",
        "autoencoder = Sequential()\n",
        "# Add a dense layer with input the original image pixels and neurons the encoded representation\n",
        "autoencoder.add(Dense(32, input_shape=(784, ), activation=\"relu\"))\n",
        "\n",
        "# Add an output layer with as many neurons as the orginal image pixels\n",
        "autoencoder.add(Dense(784, activation = \"sigmoid\"))\n",
        "\n",
        "# Compile your model with adadelta\n",
        "#AdaDelta is a stochastic optimization technique that allows for per-dimension learning rate method for SGD.\n",
        "autoencoder.compile(optimizer = 'adadelta', loss = 'binary_crossentropy')\n",
        "\n",
        "# Summarize your model structure\n",
        "autoencoder.summary()"
      ],
      "metadata": {
        "id": "97XH2MATxN9k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Build your encoder by using the first layer of your autoencoder\n",
        "encoder = Sequential()\n",
        "encoder.add(autoencoder.layers[0])\n",
        "\n",
        "# Encode the noisy images and show the encodings for your favorite number [0-9]\n",
        "encodings = encoder.predict(X_test_noise)\n",
        "show_encodings(encodings, number = 1)\n",
        "\n",
        "decoded_imgs = autoencoder.predict(X_test_noise)\n",
        "\n",
        "# Plot noisy vs decoded images\n",
        "compare_plot(X_test_noise, decoded_imgs)"
      ],
      "metadata": {
        "id": "BlYkQXqb2mBW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**CNN**"
      ],
      "metadata": {
        "id": "ER9DjqRoc7cQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Add a first convolutional layer with 32 filters of size 3x3 and the corresponding 3D tuple as input_shape.\n",
        "Add a second convolutional layer with 16 filters of size 3x3 with relu activation.\n",
        "Flatten the previous layer output to create a one-dimensional vector.\n",
        "we flatten our layer to convert it to unidimensional layer tp pass it to fully conected layer that carryout our classification\n",
        "You're going to build a shallow convolutional model that classifies the MNIST digits dataset. The same one you de-noised with your autoencoder! The images are 28 x 28 pixels and just have one channel, since they are black and white pictures."
      ],
      "metadata": {
        "id": "81ap-97-iDPH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import the Conv2D and Flatten layers and instantiate model\n",
        "from tensorflow.keras.layers import Conv2D,Flatten\n",
        "model = Sequential()\n",
        "\n",
        "# Add a convolutional layer of 32 filters of size 3x3\n",
        "#28*28 pixel balck and white picture> 1\n",
        "model.add(Conv2D(32, kernel_size = 3, input_shape = (28, 28, 1), activation = 'relu'))\n",
        "\n",
        "# Add a convolutional layer of 16 filters of size 3x3\n",
        "model.add(Conv2D(16,kernel_size = 3, activation = 'relu'))\n",
        "\n",
        "# Flatten the previous layer output\n",
        "model.add(Flatten())\n",
        "\n",
        "# Add as many outputs as classes with softmax activation\n",
        "model.add(Dense(10, activation = 'softmax'))"
      ],
      "metadata": {
        "id": "fmwyLsa3c5Lq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "Pre-processing images for ResNet50\n",
        "# Import image from keras preprocessing\n",
        "from tensorflow.keras.preprocessing import image\n",
        "# Import preprocess_input from tensorflow keras applications resnet50\n",
        "from tensorflow.keras.applications.resnet50 import preprocess_input\n",
        "# Load the image with the right target size for your model\n",
        "img = image.load_img(img_path, target_size=(224, 224))\n",
        "# Turn it into an array\n",
        "img = image.img_to_array(img)\n",
        "# Expand the dimensions so that it's understood by our network:\n",
        "# img.shape turns from (224, 224, 3) into (1, 224, 224, 3)\n",
        "img = np.expand_dims(img, axis=0)\n",
        "# Pre-process the img in the same way training images were\n",
        "img = preprocess_input(img)"
      ],
      "metadata": {
        "id": "P2W4dti5iDz3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "Using the ResNet50 model in Keras\n",
        "Predicted: [('n07697313','cheeseburger', 0.9868016)]\n",
        "# Import ResNet50 and decode_predictions\n",
        "from tensorflow.keras.applications.resnet50from tensorflow.keras.applications.resnet50 import ResNet50, decode_predictions\n",
        "# Instantiate a ResNet50 model with imagenet weights\n",
        "model = ResNet50(weights='imagenet')\n",
        "# Predict with ResNet50 on our img\n",
        "preds = model.predict(img)\n",
        "# Decode predictions and print it\n",
        "print('Predicted:', decode_predictions(preds, top=1)[0])"
      ],
      "metadata": {
        "id": "AvbJuINjjRw0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Preparing your input image**\n"
      ],
      "metadata": {
        "id": "A-H5VFLzk0El"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The original ResNet50 model was trained with images of size 224 x 224 pixels and a number of preprocessing operations; like the subtraction of the mean pixel value in the training set for all training images. You need to pre-process the images you want to predict on in the same way.\n",
        "\n",
        "When predicting on a single image you need it to fit the model's input shape, which in this case looks like this: (batch-size, width, height, channels),np.expand_dims with parameter axis = 0 adds the batch-size dimension, representing that a single image will be passed to predict. This batch-size dimension value is 1, since we are only predicting on one image."
      ],
      "metadata": {
        "id": "pt1tDgQzkx9N"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import image and preprocess_input\n",
        "from tensorflow.keras.preprocessing import image\n",
        "from tensorflow.keras.applications.resnet50 import preprocess_input\n",
        "\n",
        "# Load the image with the right target size for your model\n",
        "img = image.load_img(img_path, target_size=(224, 224))\n",
        "\n",
        "# Turn it into an array\n",
        "img_array = image.img_to_array(img)\n",
        "\n",
        "# Expand the dimensions of the image, this is so that it fits the expected model input format\n",
        "img_expanded = np.expand_dims(img_array, axis = 0)\n",
        "\n",
        "# Pre-process the img in the same way original images were\n",
        "img_ready = preprocess_input(img_expanded)"
      ],
      "metadata": {
        "id": "ms61xLtQkyjc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        " Ivy is now ready for ResNet50. Do you know this dog's breed? Let's see what this model thinks it is!"
      ],
      "metadata": {
        "id": "GHRio5ZflE2f"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Instantiate a ResNet50 model with 'imagenet' weights\n",
        "model = ResNet50(weights='imagenet')\n",
        "\n",
        "# Predict with ResNet50 on your already processed img\n",
        "preds = model.predict(img_ready)\n",
        "\n",
        "# Decode the first 3 predictions\n",
        "print('Predicted:', decode_predictions(preds, top=3)[0])"
      ],
      "metadata": {
        "id": "CET-Pj6AlHFf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "ResNet50 is a model trained on the Imagenet dataset that is able to distinguish between 1000 different labeled objects. ResNet50 is a deep model with 50 layers, you can check it in 3D here."
      ],
      "metadata": {
        "id": "zu5Ao-LNlONg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**LSTM**"
      ],
      "metadata": {
        "id": "a--eKNhDE2Tk"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "At each iteration, the loop grabs a slice of the words list using the words[i-4:i] expression:\n",
        "i-4 means \"the index 4 positions before the current index i\".\n",
        "i means \"the current index\".\n",
        "So, words[i-4:i] selects a slice of 4 words, starting from 4 words before the current index and ending at the current index.\n",
        "The ' '.join() function then joins these 4 words into a single string, separated by spaces, and appends it to the sentences list.\n",
        "Let's look at a simple example to make this clearer: Suppose the words list is ['the', 'quick', 'brown', 'fox', 'jumps', 'over', 'the', 'lazy', 'dog'].\n",
        "When i is 4, words[i-4:i] will be ['the', 'quick', 'brown', 'fox'], and this sentence will be added to the sentences list.\n",
        "When i is 5, words[i-4:i] will be ['quick', 'brown', 'fox', 'jumps'], and this sentence will be added to the sentences list.\n",
        "And so on, moving one word at a time and creating sentences of 4 words each."
      ],
      "metadata": {
        "id": "OM7ATJeSLd4R"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Split text into an array of words\n",
        "words = text.split()\n",
        "\n",
        "# Make sentences of 4 words each, moving one word at a time\n",
        "sentences = []\n",
        "for i in range(4, len(words)):\n",
        "  sentences.append(' '.join(words[i-4:i]))\n",
        "\n",
        "# Instantiate a Tokenizer, then fit it on the sentences\n",
        "tokenizer = Tokenizer()\n",
        "tokenizer.fit_on_texts(sentences)\n",
        "sequences = tokenizer.texts_to_sequences(sentences)\n",
        "print(\"Sentences: \\n {} \\n Sequences: \\n {}\".format(sentences[:5],sequences[:5]))"
      ],
      "metadata": {
        "id": "5J81WuS2E4NI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "# Instantiate Tokenizer\n",
        "tokenizer = Tokenizer()\n",
        "# Fit it on the previous lines\n",
        "tokenizer.fit_on_texts(lines)\n",
        "# Turn the lines into numeric sequences\n",
        "sequences = tokenizer.texts_to_sequences(lines)"
      ],
      "metadata": {
        "id": "S0CBXhubpqGK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "to decode our output we can use this dictionary"
      ],
      "metadata": {
        "id": "atd9sneLpvub"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(tokenizer.index_word)"
      ],
      "metadata": {
        "id": "UMBViDzupyx2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "your sequences had 4 words each, your model will be **trained on the first three** words of each sequence, **predicting the 4th on**e. You are going to use an **Embedding layer** that will essentially** learn to turn words into meaningful vectors.** These vectors will then be passed to a simple LSTM layer. Our output is a** Dense layer **with as many neurons as words in the vocabulary and **softmax activation**. This is because we want to** obtain the highest probable next word out of all possible words**"
      ],
      "metadata": {
        "id": "Poxyb6CXTp55"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import the Embedding, LSTM and Dense layer\n",
        "from tensorflow.keras.layers import Embedding, LSTM , Dense\n",
        "\n",
        "model = Sequential()\n",
        "\n",
        "# Add an Embedding layer with the right parameters\n",
        "model.add(Embedding(input_dim = vocab_size, input_length = 3, output_dim = 8))\n",
        "\n",
        "\n",
        "# Add a 32 unit LSTM layer\n",
        "model.add(LSTM(32))\n",
        "\n",
        "# Add a hidden Dense layer of 32 units and an output layer of vocab_size with softmax\n",
        "model.add(Dense(32, activation='relu'))\n",
        "model.add(Dense(vocab_size, activation='softmax'))\n",
        "model.summary()"
      ],
      "metadata": {
        "id": "EwIKQhOuQIgA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Input dimension **is the number of different toys (or words) you have\n",
        "**Output dimension** is the number of features used to describe each toy (or word)"
      ],
      "metadata": {
        "id": "KGtm7MSCTSqK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def predict_text(test_text, model = model):\n",
        "  if len(test_text.split()) != 3:\n",
        "    print('Text input should be 3 words!')\n",
        "    return False\n",
        "\n",
        "  # Turn the test_text into a sequence of numbers\n",
        "  test_seq = tokenizer.texts_to_sequences([test_text])\n",
        "  pred = model.predict(test_seq).argmax(axis = 1)[0]\n",
        "\n",
        "  # Use the model passed as a parameter to predict the next word\n",
        "  pred = model.predict(test_seq).argmax(axis = 1)[0]\n",
        "\n",
        "  # Return the word that maps to the prediction\n",
        "  return tokenizer.index_word[pred]\n"
      ],
      "metadata": {
        "id": "8Gzi-N_vq2qs"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The resulting sequence is then converted into a NumPy array. This is necessary because **the model expects the input to be in the form of a NumPy array.** ( to make prediction and also we can apply the argmax function)"
      ],
      "metadata": {
        "id": "jH53nMiETmyz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**The argmax(axis=1)[0]** function is used to find the index of the word with the highest probability. a**rgmax(axis=1) returns the indices of the maximum values along the specified axis (in this case, axis 1)**. **The [0] is used to get the first (and only) element of the resulting array**, which is the index of the predicted word."
      ],
      "metadata": {
        "id": "mT894YqoLdIp"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The **index_word** dictionary of the tokenizer is used to **map the predicted index back to the corresponding word.** This dictionary was created when the tokenizer was fitted on the text data."
      ],
      "metadata": {
        "id": "_dRPdr6OUDyZ"
      }
    }
  ]
}