Notebook_1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a9bf2227",
   "metadata": {},
   "source": [
    "# Introduction: Machine Learning Project Part 1: Naive Bayes Classification for Content Moderation\n",
    "\n",
    "![Naive Bayes Classification](naive.png)\n",
    "\n",
    "In this segment of our comprehensive machine learning project, we are delving into the crucial task of content moderation in digital communication platforms. Our objective is to develop a model capable of identifying and flagging potentially harmful content, a necessity in today's era of expansive digital interactions. This project represents a supervised classification challenge in machine learning.\n",
    "\n",
    "In the initial phase of our project, we have embarked on several critical steps in the machine learning pipeline, tailored to address the nuances of content moderation:\n",
    "\n",
    "1. Data Cleaning and Preprocessing: Preparing the Jigsaw Toxic Comment Classification Dataset for analysis, including text normalization and removal of non-essential elements.\n",
    "\n",
    "2. Exploratory Data Analysis: Understanding the characteristics of the dataset, including the distribution of different classes of toxicity.\n",
    "\n",
    "3. Feature Extraction: Implementing techniques like TF-IDF to convert textual data into a format suitable for machine learning algorithms.\n",
    "\n",
    "4. Model Building: Focusing on the Naive Bayes classifier to establish a baseline for our content moderation model.\n",
    "\n",
    "5. Model Training and Evaluation: Assessing the performance of our model using appropriate metrics, such as accuracy and F1-score, to gauge its effectiveness in classifying toxic comments.\n",
    "\n",
    "6. Interpreting Results: Analyzing the outcomes of our model to understand its decision-making process.\n",
    "\n",
    "In this notebook, we will concentrate on the implementation and evaluation of the Naive Bayes classifier. This model, known for its simplicity and effectiveness in text classification tasks, serves as our starting point in the quest to tackle online toxicity. We aim to explore its strengths and limitations in handling the complexities of content moderation.\n",
    "\n",
    "The subsequent parts of the project will expand upon more sophisticated models and techniques, enhancing our ability to moderate digital content effectively. As we progress, we will document our findings and methodologies, culminating in a comprehensive and insightful report."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3918ce80",
   "metadata": {},
   "source": [
    "# Imports\n",
    "We will use a familiar stack of data science and machine learning libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "903608ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pandas for data manipulation\n",
    "import pandas as pd\n",
    "\n",
    "# Regular expressions for text processing\n",
    "import re\n",
    "\n",
    "# Sklearn model_selection for splitting data into training and test sets\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# TfidfVectorizer for converting text data into numerical form\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "# NLTK for natural language processing tasks\n",
    "import nltk\n",
    "\n",
    "# nltk.download('stopwords')\n",
    "# nltk.download('punkt')\n",
    "\n",
    "# Stopwords and tokenization functions from NLTK\n",
    "from nltk.corpus import stopwords\n",
    "from nltk.tokenize import word_tokenize\n",
    "\n",
    "# MultinomialNB for Naive Bayes classification\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "\n",
    "# LinearSVC for Support Vector Machine classification\n",
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "# LogisticRegression for logistic regression modeling\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "# Sklearn metrics for model evaluation\n",
    "from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, classification_report, multilabel_confusion_matrix\n",
    "\n",
    "# Matplotlib for plotting and data visualization\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Numpy for numerical and array operations\n",
    "import numpy as np\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32f5277a",
   "metadata": {},
   "source": [
    "# Data Preprocessing\n",
    "\n",
    "Data preprocessing is a critical step in our machine learning pipeline. For text data, especially in content moderation, it's essential to normalize the text to ensure consistency. This process includes:\n",
    "\n",
    "- Converting text to lowercase to maintain uniformity.\n",
    "- Removing URLs as they often don't contribute to text classification.\n",
    "- Eliminating non-alphanumeric characters which might introduce noise in the data.\n",
    "- Stripping extra white spaces for clean, concise text data.\n",
    "\n",
    "These steps help in creating a standardized dataset, ready for feature extraction and model training.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c558a923",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to remove stopwords\n",
    "def remove_stopwords(text):\n",
    "    stop_words = set(stopwords.words('english'))\n",
    "    words = word_tokenize(text)\n",
    "    cleaned_text = ' '.join([word for word in words if word not in stop_words])\n",
    "    return cleaned_text.strip()  # Strip spaces after processing\n",
    "\n",
    "def preprocess_text(text: str):\n",
    "    text = text.lower()\n",
    "    text = re.sub(r'http\\S+|www\\S+|https\\S+', '', text).strip()  # Remove URLs\n",
    "    text = re.sub(r'[^a-zA-Z0-9\\s]', '', text).strip()  # Remove non-alphanumeric characters\n",
    "    text = re.sub(r'\\s+', ' ', text).strip()  # Remove extra white spaces\n",
    "    text = remove_stopwords(text) # Remove stop_words from the NLTK library\n",
    "    \n",
    "    text = text.strip() # remove extra spaces\n",
    "    return text\n",
    "\n",
    "def preprocess_data(series: pd.Series):\n",
    "    series = series.apply(preprocess_text)\n",
    "    return series\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7f2313b",
   "metadata": {},
   "source": [
    "# Loading and Preprocessing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ca46ba10",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>comment_text</th>\n",
       "      <th>toxic</th>\n",
       "      <th>severe_toxic</th>\n",
       "      <th>obscene</th>\n",
       "      <th>threat</th>\n",
       "      <th>insult</th>\n",
       "      <th>identity_hate</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0000997932d777bf</td>\n",
       "      <td>explanation edits made username hardcore metal...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>000103f0d9cfb60f</td>\n",
       "      <td>daww matches background colour im seemingly st...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>000113f07ec002fd</td>\n",
       "      <td>hey man im really trying edit war guy constant...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0001b41b1c6bb37e</td>\n",
       "      <td>cant make real suggestions improvement wondere...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0001d958c54c6e35</td>\n",
       "      <td>sir hero chance remember page thats</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159566</th>\n",
       "      <td>ffe987279560d7ff</td>\n",
       "      <td>second time asking view completely contradicts...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159567</th>\n",
       "      <td>ffea4adeee384e90</td>\n",
       "      <td>ashamed horrible thing put talk page 128611993</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159568</th>\n",
       "      <td>ffee36eab5c267c9</td>\n",
       "      <td>spitzer umm theres actual article prostitution...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159569</th>\n",
       "      <td>fff125370e4aaaf3</td>\n",
       "      <td>looks like actually put speedy first version d...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159570</th>\n",
       "      <td>fff46fc426af1f9a</td>\n",
       "      <td>really dont think understand came idea bad rig...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>159571 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      id                                       comment_text   \n",
       "0       0000997932d777bf  explanation edits made username hardcore metal...  \\\n",
       "1       000103f0d9cfb60f  daww matches background colour im seemingly st...   \n",
       "2       000113f07ec002fd  hey man im really trying edit war guy constant...   \n",
       "3       0001b41b1c6bb37e  cant make real suggestions improvement wondere...   \n",
       "4       0001d958c54c6e35                sir hero chance remember page thats   \n",
       "...                  ...                                                ...   \n",
       "159566  ffe987279560d7ff  second time asking view completely contradicts...   \n",
       "159567  ffea4adeee384e90     ashamed horrible thing put talk page 128611993   \n",
       "159568  ffee36eab5c267c9  spitzer umm theres actual article prostitution...   \n",
       "159569  fff125370e4aaaf3  looks like actually put speedy first version d...   \n",
       "159570  fff46fc426af1f9a  really dont think understand came idea bad rig...   \n",
       "\n",
       "        toxic  severe_toxic  obscene  threat  insult  identity_hate  \n",
       "0           0             0        0       0       0              0  \n",
       "1           0             0        0       0       0              0  \n",
       "2           0             0        0       0       0              0  \n",
       "3           0             0        0       0       0              0  \n",
       "4           0             0        0       0       0              0  \n",
       "...       ...           ...      ...     ...     ...            ...  \n",
       "159566      0             0        0       0       0              0  \n",
       "159567      0             0        0       0       0              0  \n",
       "159568      0             0        0       0       0              0  \n",
       "159569      0             0        0       0       0              0  \n",
       "159570      0             0        0       0       0              0  \n",
       "\n",
       "[159571 rows x 8 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_file_path = 'train.csv'\n",
    "train_data = pd.read_csv(train_file_path)\n",
    "\n",
    "train_data['comment_text'] = preprocess_data(train_data['comment_text'])\n",
    "train_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a0476b9e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>comment_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>00001cee341fdb12</td>\n",
       "      <td>yo bitch ja rule succesful youll ever whats ha...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0000247867823ef7</td>\n",
       "      <td>rfc title fine imo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>00013b17ad220c46</td>\n",
       "      <td>sources zawe ashton lapland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>00017563c3f7919a</td>\n",
       "      <td>look back source information updated correct f...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>00017695ad8997eb</td>\n",
       "      <td>dont anonymously edit articles</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153159</th>\n",
       "      <td>fffcd0960ee309b5</td>\n",
       "      <td>totally agree stuff nothing toolongcrap</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153160</th>\n",
       "      <td>fffd7a9a6eb32c16</td>\n",
       "      <td>throw field home plate get faster throwing cut...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153161</th>\n",
       "      <td>fffda9e8d6fafa9e</td>\n",
       "      <td>okinotorishima categories see changes agree co...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153162</th>\n",
       "      <td>fffe8f1340a79fc2</td>\n",
       "      <td>one founding nations eu germany law return qui...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153163</th>\n",
       "      <td>ffffce3fb183ee80</td>\n",
       "      <td>stop already bullshit welcome im fool think ki...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>153164 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      id                                       comment_text\n",
       "0       00001cee341fdb12  yo bitch ja rule succesful youll ever whats ha...\n",
       "1       0000247867823ef7                                 rfc title fine imo\n",
       "2       00013b17ad220c46                        sources zawe ashton lapland\n",
       "3       00017563c3f7919a  look back source information updated correct f...\n",
       "4       00017695ad8997eb                     dont anonymously edit articles\n",
       "...                  ...                                                ...\n",
       "153159  fffcd0960ee309b5            totally agree stuff nothing toolongcrap\n",
       "153160  fffd7a9a6eb32c16  throw field home plate get faster throwing cut...\n",
       "153161  fffda9e8d6fafa9e  okinotorishima categories see changes agree co...\n",
       "153162  fffe8f1340a79fc2  one founding nations eu germany law return qui...\n",
       "153163  ffffce3fb183ee80  stop already bullshit welcome im fool think ki...\n",
       "\n",
       "[153164 rows x 2 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_file_path = 'data/test.csv'  # Replace with the actual path to your test.csv file\n",
    "test_label_path = 'data/test_labels.csv'\n",
    "test_data = pd.read_csv(test_file_path)\n",
    "test_labels = pd.read_csv(test_label_path)\n",
    "\n",
    "test_data['comment_text'] = preprocess_data(test_data['comment_text'])\n",
    "test_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b977320d",
   "metadata": {},
   "source": [
    "# Feature Extraction: TF-IDF Vectorization\n",
    "\n",
    "Feature extraction is crucial for converting our textual data into a numerical format that our machine learning models can understand. We use TF-IDF (Term Frequency-Inverse Document Frequency) vectorization, which reflects the importance of a word in a corpus. TF-IDF increases proportionally to the number of times a word appears in a document but is offset by the frequency of the word in the corpus. This helps in handling the most common problem in text data - the variance in word frequency.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "48414509",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape: (63978, 2)\n",
      "x_test:\n",
      "                  id                                       comment_text\n",
      "5   0001ea8717f6de06  thank understanding think highly would revert ...\n",
      "7   000247e83dcc1211                             dear god site horrible\n",
      "11  0002f87b16116a7f  somebody invariably try add religion really me...\n",
      "13  0003e1cccfd5a40a  says right type type institution needed case t...\n",
      "14  00059ace3e3e9a53  adding new product list make sure relevant add...\n",
      "\n",
      "y_test:\n",
      "[[0 0 0 0 0 0]\n",
      " [0 0 0 0 0 0]\n",
      " [0 0 0 0 0 0]\n",
      " ...\n",
      " [0 0 0 0 0 0]\n",
      " [1 0 1 0 1 0]\n",
      " [0 0 0 0 0 0]]\n"
     ]
    }
   ],
   "source": [
    "# Merge/join the dataframes based on the 'id' column\n",
    "merged_data = pd.merge(test_data, test_labels, on='id')\n",
    "\n",
    "# there are alot of comments that are not givien any labels so just filtering those out\n",
    "filtered_data = merged_data[merged_data.iloc[:, 2:].sum(axis=1)>=0] \n",
    "\n",
    "# Separate the merged data into x_test and y_test\n",
    "x_test = filtered_data[['id', 'comment_text']]\n",
    "y_test = filtered_data[['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']].to_numpy()\n",
    "\n",
    "X_test_tfidf = tfidf_vectorizer.transform(x_test['comment_text'])\n",
    "\n",
    "\n",
    "# Print x_test and y_test to verify the structure\n",
    "print(f\"Shape: {x_test.shape}\")\n",
    "print(\"x_test:\")\n",
    "print(x_test.head())\n",
    "print(\"\\ny_test:\")\n",
    "print(y_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50828ac8",
   "metadata": {},
   "source": [
    "# Splitting Data for Analysis\n",
    "\n",
    "Before building our models, it's important to understand the distribution of data across different categories. This helps us identify any imbalances which can significantly impact model performance. We'll visualize the number of comments per category and analyze the label distribution. Understanding these aspects is crucial for making informed decisions in the subsequent stages of model training and evaluation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c92c7c41",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'category')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsoAAAIjCAYAAAAa3T0KAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB6VUlEQVR4nO3deVgVZf8/8PdhOyzBkUU2Q8SNRFxwRzN3cEFcKp+iUNPUMiVELX3MRM3dXNIeNTM1l6w0y1wI3DDDXXHPUkE0QUwQRBAOcP/+8Md8PZ5BQTnMAd6v6+LKc899Zj4zw8Sbmzn3qIQQAkREREREpMNE6QKIiIiIiIwRgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRERERCSDQZmIiIiISAaDMlElt2bNGqhUKlhaWuLatWt6yzt27AhfX18FKgP2798PlUqFzZs3K7L90kpMTESvXr3g4OAAlUqF8PBwpUuqsOLi4hAZGYm7d+8qXYrR4jEiUh6DMlEVkZubi08++UTpMiq0MWPG4MiRI/jmm29w6NAhjBkzRumSKqy4uDhMnTqVIfAJeIyIlMegTFRFdO/eHRs3bsTp06eVLqXc5eTkQAjx3Os5d+4cWrVqhb59+6JNmzbw9PQsg+qoIhFCICcnR+kyjAqPCVVmDMpEVcRHH30ER0dHfPzxx0/sl5iYCJVKhTVr1ugtU6lUiIyMlF5HRkZCpVLhzJkzeP3116HRaODg4ICIiAjk5+fj0qVL6N69O2xtbVGrVi3MnTtXdpsPHjxAREQEXF1dYWVlhQ4dOuDUqVN6/Y4fP47g4GA4ODjA0tISfn5++OGHH3T6FN1qEh0djSFDhqB69eqwtrZGbm5usfuclJSEt99+G87OzlCr1WjQoAE+//xzFBYWAvi/W0QuX76MXbt2QaVSQaVSITExsdh1FhYWYsmSJWjatCmsrKxQrVo1tGnTBtu2bdPpM3fuXLz00ktQq9VwdnbGwIEDcePGDZ11Fd0ec+jQIbRt2xZWVlaoVasWVq9eDQDYsWMHmjVrBmtrazRq1AhRUVE67y+L85SZmYlx48bBy8sLFhYWqFGjBsLDw3H//n2dfiqVCqNGjcK6devQoEEDWFtbo0mTJti+fbtOPePHjwcAeHl5Scdz//79AIC9e/eiY8eOcHR0hJWVFWrWrIlXX30V2dnZxR5vAKhVqxaCgoKwdetWNG7cGJaWlqhduza++OKL596f5cuXo0GDBlCr1Vi7du0T69i4cSP8/f3xwgsv4IUXXkDTpk2xatUqaXlMTAz69OmDF198EZaWlqhbty5GjBiBf//9t8THCAC+//57+Pv7w8bGBi+88AICAwNlr5uVK1eifv36UKvV8PHxwcaNGzF48GDUqlVLp19aWhpGjhyJGjVqwMLCArVr18akSZP0rh25Y7JmzRrUq1cPgYGBetvPysqCRqPBBx988MTjRmSUBBFVaqtXrxYAxLFjx8TixYsFALFnzx5peYcOHUTDhg2l1wkJCQKAWL16td66AIgpU6ZIr6dMmSIACG9vbzF9+nQRExMjPvroIwFAjBo1Srz00kviiy++EDExMeKdd94RAMSWLVuk9+/bt08AEB4eHqJPnz7i119/FevXrxd169YVdnZ24sqVK1LfvXv3CgsLC9G+fXvx/fffi6ioKDF48GC9Wov2t0aNGmL48OFi165dYvPmzSI/P1/2+KSmpooaNWqI6tWri+XLl4uoqCgxatQoAUC8//77QgghMjIyxKFDh4Srq6to166dOHTokDh06JB48OBBscc9NDRUqFQq8e6774pffvlF7Nq1S8yYMUMsXrxY6jN8+HDpWEVFRYnly5eL6tWrCw8PD3H79m2dc+To6Ci8vb3FqlWrxG+//SaCgoIEADF16lTRqFEj8d1334mdO3eKNm3aCLVaLf75558yO0/3798XTZs2FU5OTmLBggVi9+7dYvHixUKj0YjOnTuLwsJCne+RWrVqiVatWokffvhB7Ny5U3Ts2FGYmZlJ5/P69eti9OjRAoD46aefpOOZkZEhEhIShKWlpejWrZv4+eefxf79+8WGDRtEaGioSE9PL/Z4CyGEp6enqFGjhqhZs6b45ptvxM6dO8Vbb70lAIh58+Y98/7UqFFDNG7cWGzcuFHs3btXnDt3rtgaJk+eLACI/v37ix9//FFER0eLBQsWiMmTJ0t9li1bJmbNmiW2bdsmYmNjxdq1a0WTJk2Et7e3yMvLe+oxEkKIGTNmCJVKJYYMGSK2b98ufvrpJ+Hv7y9sbGzE+fPnpW2tWLFCABCvvvqq2L59u9iwYYOoX7++8PT0FJ6enlK/nJwc0bhxY2FjYyPmz58voqOjxeTJk4WZmZno2bOnzj4Wd0wWL14sVCqV+Ouvv3T6f/nllwKATl1EFQWDMlEl92hQzs3NFbVr1xYtWrSQwkBZBOXPP/9cp1/Tpk2lH/BFtFqtqF69uujfv7/UVhSUmzVrphNOEhMThbm5uXj33Xeltpdeekn4+fkJrVars62goCDh5uYmCgoKdPZ34MCBJTo+EyZMEADEkSNHdNrff/99oVKpxKVLl6Q2T09P0atXr6eu88CBAwKAmDRpUrF9Ll68KACIkSNH6rQfOXJEABD//e9/pbYOHToIAOL48eNS2507d4SpqamwsrLSCcXx8fECgPjiiy+ktuc9T7NmzRImJibi2LFjOu/fvHmzACB27twptQEQLi4uIjMzU2pLSUkRJiYmYtasWVLbvHnzBACRkJAgu874+Hj5A/cEnp6eQqVS6b23W7duws7OTty/f/+Z9kej0Yi0tLSnbv/q1avC1NRUvPXWWyWuubCwUGi1WnHt2jUBQPzyyy/SsuKOUVJSkjAzMxOjR4/Wab93755wdXUVAwYMEEIIUVBQIFxdXUXr1q11+l27dk2Ym5vrBOXly5cLAOKHH37Q6TtnzhwBQERHR0ttxR2TzMxMYWtrKz788EOddh8fH9GpU6cSHQ8iY8NbL4iqEAsLC3z22Wc4fvy43i0LzyMoKEjndYMGDaBSqdCjRw+pzczMDHXr1pWdeSMkJAQqlUp67enpibZt22Lfvn0AgMuXL+PPP//EW2+9BQDIz8+Xvnr27Ink5GRcunRJZ52vvvpqiWrfu3cvfHx80KpVK532wYMHQwiBvXv3lmg9j9q1axcAPPFPzUX7NnjwYJ32Vq1aoUGDBtizZ49Ou5ubG5o3by69dnBwgLOzM5o2bQp3d3epvUGDBgAge5yf9Txt374dvr6+aNq0qc6xDwwM1LsdAAA6deoEW1tb6bWLiwucnZ1la3pc06ZNYWFhgeHDh2Pt2rW4evXqU9/zqIYNG6JJkyY6bSEhIcjMzMTJkyefaX86d+4Me3v7p247JiYGBQUFT73FIDU1Fe+99x48PDxgZmYGc3Nz6X73ixcvPnU7v/32G/Lz8zFw4ECd+i0tLdGhQwep/kuXLiElJQUDBgzQeX/NmjXRrl07nba9e/fCxsYGr732mk570ffn49+PcsfE1tYW77zzDtasWSPdwrJ3715cuHABo0aNeup+ERkjBmWiKuaNN95As2bNMGnSJGi12jJZp4ODg85rCwsLWFtbw9LSUq/9wYMHeu93dXWVbbtz5w4A4NatWwCAcePGwdzcXOdr5MiRAKBzfyfwMFiWxJ07d2T7FoXPohpK4/bt2zA1NZXdr0e3W1yd7u7uett9/BgDD4+n3LEHIHucn/U83bp1C2fOnNE79ra2thBC6B17R0dHvW2r1eoSfeCrTp062L17N5ydnfHBBx+gTp06qFOnDhYvXvzU9wLFfy8B0Pl+Ks3+lPR76fbt2wCAF198sdg+hYWFCAgIwE8//YSPPvoIe/bswdGjR3H48GEAKNExKroeWrZsqbcP33//vVR/0f66uLjorePxtjt37sDV1VXnF1YAcHZ2hpmZmd73Y3HHZPTo0bh37x42bNgAAFi6dClefPFF9OnT56n7RWSMzJQugIjKl0qlwpw5c9CtWzd89dVXesuLQtPjH+B5lsBYUikpKbJtRYHLyckJADBx4kT0799fdh3e3t46rx//gV8cR0dHJCcn67XfvHlTZ9ulUb16dRQUFCAlJaXYQFG0b8nJyXrB6ubNm8+0XUNxcnKClZUVvvnmm2KXl6X27dujffv2KCgowPHjx7FkyRKEh4fDxcUFb7zxxhPfW9z3EgCd76fS7E9Jv5eqV68OALhx4wY8PDxk+5w7dw6nT5/GmjVrMGjQIKn98uXLJdrGo/Vt3rz5iTOvFO1vUbB+1OPHydHREUeOHIEQQmd/U1NTkZ+fX+JjUrduXfTo0QNffvklevTogW3btmHq1KkwNTUt2c4RGRmOKBNVQV27dkW3bt0wbdo0ZGVl6SxzcXGBpaUlzpw5o9P+yy+/GKye7777Tmf6tmvXriEuLg4dO3YE8DAE16tXD6dPn0aLFi1kvx79U39pdOnSBRcuXJD+LF/k22+/hUqlQqdOnUq9zqJbGZYtW1Zsn86dOwMA1q9fr9N+7NgxXLx4EV26dCn1dg0lKCgIV65cgaOjo+yxf3z2hJJQq9UAnjyCampqitatW+PLL78EAL1zJOf8+fN6UyBu3LgRtra2aNasmcH2BwACAgJgamr6xPNeFDCL9r/IihUr9PoWd4wCAwNhZmaGK1euFHs9AA+vG1dXV73brJKSkhAXF6fT1qVLF2RlZeHnn3/Waf/222+l5SX14Ycf4syZMxg0aBBMTU0xbNiwEr+XyNhwRJmoipozZw6aN2+O1NRUNGzYUGpXqVR4++238c0336BOnTpo0qQJjh49io0bNxqsltTUVPTr1w/Dhg1DRkYGpkyZAktLS0ycOFHqs2LFCvTo0QOBgYEYPHgwatSogbS0NFy8eBEnT57Ejz/++EzbHjNmDL799lv06tUL06ZNg6enJ3bs2IH//e9/eP/991G/fv1Sr7N9+/YIDQ3FZ599hlu3biEoKAhqtRqnTp2CtbU1Ro8eDW9vbwwfPhxLliyBiYkJevTogcTEREyePBkeHh5G9TCT8PBwbNmyBa+88grGjBmDxo0bo7CwEElJSYiOjsbYsWPRunXrUq2zUaNGAIDFixdj0KBBMDc3h7e3NzZs2IC9e/eiV69eqFmzJh48eCCN/Hbt2vWp63V3d0dwcDAiIyPh5uaG9evXIyYmBnPmzIG1tbXB9gd4OD3df//7X0yfPh05OTl48803odFocOHCBfz777+YOnUqXnrpJdSpUwcTJkyAEAIODg749ddfERMTU+JjVKtWLUybNg2TJk3C1atX0b17d9jb2+PWrVs4evQobGxsMHXqVJiYmGDq1KkYMWIEXnvtNQwZMgR3797F1KlT4ebmBhOT/xsrGzhwIL788ksMGjQIiYmJaNSoEQ4ePIiZM2eiZ8+eJTr2Rbp16wYfHx/s27dPmnaRqMJS8pOERGR4j8568biQkBABQGfWCyEeTof27rvvChcXF2FjYyN69+4tEhMTi5314tGpzIQQYtCgQcLGxkZve4/PsFE068W6detEWFiYqF69ulCr1aJ9+/Y6MzwUOX36tBgwYIBwdnYW5ubmwtXVVXTu3FksX768RPtbnGvXromQkBDh6OgozM3Nhbe3t5g3b540k0aRks56IcTDGQcWLlwofH19hYWFhdBoNMLf31/8+uuvOn3mzJkj6tevL8zNzYWTk5N4++23xfXr13XW9fhxe1o9AMQHH3wgvX7e8ySEEFlZWeKTTz4R3t7e0v40atRIjBkzRqSkpBS77UdrHTRokE7bxIkThbu7uzAxMREAxL59+8ShQ4dEv379hKenp1Cr1cLR0VF06NBBbNu2TW+dxR2PzZs3i4YNGwoLCwtRq1YtsWDBAr2+z7s/T/Ltt9+Kli1bCktLS/HCCy8IPz8/nVlkLly4ILp16yZsbW2Fvb29eP3110VSUpLe9VXcMSry888/i06dOgk7OzuhVquFp6eneO2118Tu3bt11vHVV1+JunXrCgsLC1G/fn3xzTffiD59+gg/Pz+dfnfu3BHvvfeecHNzE2ZmZsLT01NMnDhRbxrEkhyTyMhIAUAcPny45AeOyAiphCiDx1UREREprFatWvD19dV5uAnpu3v3LurXr4++ffvKfk6hLLRo0QIqlQrHjh0zyPqJygtvvSAiIqqkUlJSMGPGDHTq1AmOjo64du0aFi5ciHv37uHDDz8s021lZmbi3Llz2L59O06cOIGtW7eW6fqJlMCgTEREVEmp1WokJiZi5MiRSEtLg7W1Ndq0aYPly5frfDahLJw8eVIK5FOmTEHfvn3LdP1ESuCtF0REREREMjg9HBERERGRDMWD8oEDB9C7d2+4u7tDpVLpzeEIPHykZ3BwMDQaDWxtbdGmTRskJSVJy3NzczF69Gg4OTnBxsYGwcHBuHHjhs460tPTERoaCo1GA41Gg9DQUNy9e1enT1JSEnr37g0bGxs4OTkhLCwMeXl5hthtIiIiIjJyigfl+/fvo0mTJli6dKns8itXruDll1/GSy+9hP379+P06dOYPHmyziNXw8PDsXXrVmzatAkHDx5EVlYWgoKCUFBQIPUJCQlBfHw8oqKiEBUVhfj4eISGhkrLCwoK0KtXL9y/fx8HDx7Epk2bsGXLFowdO9ZwO09ERERERsuo7lFWqVTYunWrzgcA3njjDZibm2PdunWy78nIyED16tWxbt06/Oc//wHw8PGvHh4e2LlzJwIDA3Hx4kX4+Pjg8OHD0iTyhw8fhr+/P/788094e3tj165dCAoKwvXr1+Hu7g4A2LRpEwYPHozU1FTY2dmVaB8KCwtx8+ZN2Nralvixp0RERERUfoQQuHfvHtzd3XUeviPX0WgAEFu3bpVeFxQUiBdeeEFMmzZNBAQEiOrVq4tWrVrp9NmzZ48AINLS0nTW1bhxY/Hpp58KIYRYtWqV0Gg0etvTaDTim2++EUIIMXnyZNG4cWOd5WlpaQKA2Lt3b4n34fr16wIAv/jFL37xi1/84he/jPzr8Qc8Pc6op4dLTU1FVlYWZs+ejc8++wxz5sxBVFQU+vfvj3379qFDhw5ISUmBhYUF7O3tdd7r4uKClJQUAA/nkZR7hKazs7NOHxcXF53l9vb2sLCwkPrIyc3NRW5urvRa/P8B+oSEBNja2j7bjhsJrVaLffv2oVOnTjA3N1e6HHoMz4/x4rkxXjw3xo3nx3hVtnNz7949eHl5PTWrGXVQLiwsBAD06dMHY8aMAQA0bdoUcXFxWL58OTp06FDse4UQOrc+yN0G8Sx9Hjdr1ixMnTpVr/3QoUOwtrYu9n0VhbW1NY4cOaJ0GVQMnh/jxXNjvHhujBvPj/GqTOcmOzsbgHz2e5RRB2UnJyeYmZnBx8dHp71BgwY4ePAgAMDV1RV5eXlIT0/XGVVOTU1F27ZtpT63bt3SW//t27elUWRXV1e9k5+eng6tVqs30vyoiRMnIiIiQnqdmZkJDw8PBAQElPi+ZmOl1WoRExODbt26VYrfHisbnh/jxXNjvHhujBvPj/GqbOcmMzOzRP2MOihbWFigZcuWuHTpkk77X3/9BU9PTwBA8+bNYW5ujpiYGAwYMAAAkJycjHPnzmHu3LkAAH9/f2RkZODo0aNo1aoVAODIkSPIyMiQwrS/vz9mzJiB5ORkuLm5AQCio6OhVqvRvHnzYmtUq9VQq9V67ebm5pXiGwmoXPtSGfH8GC+eG+PFc2PceH6MV2U5NyXdB8WDclZWFi5fviy9TkhIQHx8PBwcHFCzZk2MHz8e//nPf/DKK6+gU6dOiIqKwq+//or9+/cDADQaDYYOHYqxY8fC0dERDg4OGDduHBo1aoSuXbsCeDgC3b17dwwbNgwrVqwAAAwfPhxBQUHw9vYGAAQEBMDHxwehoaGYN28e0tLSMG7cOAwbNqzCjwwTERERUekpPo/y8ePH4efnBz8/PwBAREQE/Pz88OmnnwIA+vXrh+XLl2Pu3Llo1KgRvv76a2zZsgUvv/yytI6FCxeib9++GDBgANq1awdra2v8+uuvMDU1lfps2LABjRo1QkBAAAICAtC4cWOdKedMTU2xY8cOWFpaol27dhgwYAD69u2L+fPnl9ORICIiIiJjoviIcseOHaWZIoozZMgQDBkypNjllpaWWLJkCZYsWVJsHwcHB6xfv/6J26lZsya2b9/+5IKJiIiIqEpQfESZiIiIiMgYMSgTEREREclgUCYiIiIiksGgTEREREQkg0GZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSYaZ0AfRsak3YYfBtqE0F5rYCfCN/Q26ByqDbSpzdy6DrJyIiIiotjigTEREREclgUCYiIiIiksGgTEREREQkg0GZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRERERCSDQZmIiIiISAaDMhERERGRDAZlIiIiIiIZDMpERERERDIYlImIiIiIZDAoExERERHJYFAmIiIiIpLBoExEREREJINBmYiIiIhIBoMyEREREZEMBmUiIiIiIhkMykREREREMhiUiYiIiIhkMCgTEREREclQPCgfOHAAvXv3hru7O1QqFX7++edi+44YMQIqlQqLFi3Sac/NzcXo0aPh5OQEGxsbBAcH48aNGzp90tPTERoaCo1GA41Gg9DQUNy9e1enT1JSEnr37g0bGxs4OTkhLCwMeXl5ZbSnRERERFSRKB6U79+/jyZNmmDp0qVP7Pfzzz/jyJEjcHd311sWHh6OrVu3YtOmTTh48CCysrIQFBSEgoICqU9ISAji4+MRFRWFqKgoxMfHIzQ0VFpeUFCAXr164f79+zh48CA2bdqELVu2YOzYsWW3s0RERERUYZgpXUCPHj3Qo0ePJ/b5559/MGrUKPz222/o1auXzrKMjAysWrUK69atQ9euXQEA69evh4eHB3bv3o3AwEBcvHgRUVFROHz4MFq3bg0AWLlyJfz9/XHp0iV4e3sjOjoaFy5cwPXr16Uw/vnnn2Pw4MGYMWMG7OzsDLD3RERERGSsFB9RfprCwkKEhoZi/PjxaNiwod7yEydOQKvVIiAgQGpzd3eHr68v4uLiAACHDh2CRqORQjIAtGnTBhqNRqePr6+vzoh1YGAgcnNzceLECUPtHhEREREZKcVHlJ9mzpw5MDMzQ1hYmOzylJQUWFhYwN7eXqfdxcUFKSkpUh9nZ2e99zo7O+v0cXFx0Vlub28PCwsLqY+c3Nxc5ObmSq8zMzMBAFqtFlqttgR7+GzUpsJg65a2YSJ0/mtIhjxWlVXRMeOxMz48N8aL58a48fwYr8p2bkq6H0YdlE+cOIHFixfj5MmTUKlUpXqvEELnPXLvf5Y+j5s1axamTp2q1x4dHQ1ra+tS1Vwac1sZbNV6prcoNPg2du7cafBtVFYxMTFKl0DF4LkxXjw3xo3nx3hVlnOTnZ1don5GHZR///13pKamombNmlJbQUEBxo4di0WLFiExMRGurq7Iy8tDenq6zqhyamoq2rZtCwBwdXXFrVu39NZ/+/ZtaRTZ1dUVR44c0Vmenp4OrVarN9L8qIkTJyIiIkJ6nZmZCQ8PDwQEBBj0vmbfyN8Mtu4iahOB6S0KMfm4CXILS/eLSmmdiww06PorI61Wi5iYGHTr1g3m5uZKl0OP4LkxXjw3xo3nx3hVtnNTdAfA0xh1UA4NDZU+oFckMDAQoaGheOeddwAAzZs3h7m5OWJiYjBgwAAAQHJyMs6dO4e5c+cCAPz9/ZGRkYGjR4+iVauHQ7FHjhxBRkaGFKb9/f0xY8YMJCcnw83NDcDDUWG1Wo3mzZsXW6NarYZardZrNzc3N+g3Um6BYYOrzrYKVQbfXmW46JRi6O81enY8N8aL58a48fwYr8pybkq6D4oH5aysLFy+fFl6nZCQgPj4eDg4OKBmzZpwdHTU6W9ubg5XV1d4e3sDADQaDYYOHYqxY8fC0dERDg4OGDduHBo1aiSF7AYNGqB79+4YNmwYVqxYAQAYPnw4goKCpPUEBATAx8cHoaGhmDdvHtLS0jBu3DgMGzaMM14QERERVUGKz3px/Phx+Pn5wc/PDwAQEREBPz8/fPrppyVex8KFC9G3b18MGDAA7dq1g7W1NX799VeYmppKfTZs2IBGjRohICAAAQEBaNy4MdatWyctNzU1xY4dO2BpaYl27dphwIAB6Nu3L+bPn192O0tEREREFYbiI8odO3aEECWfVSExMVGvzdLSEkuWLMGSJUuKfZ+DgwPWr1//xHXXrFkT27dvL3EtRERERFR5KT6iTERERERkjBiUiYiIiIhkMCgTEREREclgUCYiIiIiksGgTEREREQkg0GZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRERERCSDQZmIiIiISAaDMhERERGRDAZlIiIiIiIZDMpERERERDIYlImIiIiIZDAoExERERHJYFAmIiIiIpLBoExEREREJINBmYiIiIhIBoMyEREREZEMBmUiIiIiIhkMykREREREMhiUiYiIiIhkMCgTEREREclgUCYiIiIiksGgTEREREQkg0GZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRERERCRD8aB84MAB9O7dG+7u7lCpVPj555+lZVqtFh9//DEaNWoEGxsbuLu7Y+DAgbh586bOOnJzczF69Gg4OTnBxsYGwcHBuHHjhk6f9PR0hIaGQqPRQKPRIDQ0FHfv3tXpk5SUhN69e8PGxgZOTk4ICwtDXl6eoXadiIiIiIyY4kH5/v37aNKkCZYuXaq3LDs7GydPnsTkyZNx8uRJ/PTTT/jrr78QHBys0y88PBxbt27Fpk2bcPDgQWRlZSEoKAgFBQVSn5CQEMTHxyMqKgpRUVGIj49HaGiotLygoAC9evXC/fv3cfDgQWzatAlbtmzB2LFjDbfzRERERGS0zJQuoEePHujRo4fsMo1Gg5iYGJ22JUuWoFWrVkhKSkLNmjWRkZGBVatWYd26dejatSsAYP369fDw8MDu3bsRGBiIixcvIioqCocPH0br1q0BACtXroS/vz8uXboEb29vREdH48KFC7h+/Trc3d0BAJ9//jkGDx6MGTNmwM7OzoBHgYiIiIiMjeIjyqWVkZEBlUqFatWqAQBOnDgBrVaLgIAAqY+7uzt8fX0RFxcHADh06BA0Go0UkgGgTZs20Gg0On18fX2lkAwAgYGByM3NxYkTJ8phz4iIiIjImCg+olwaDx48wIQJExASEiKN8KakpMDCwgL29vY6fV1cXJCSkiL1cXZ21lufs7OzTh8XFxed5fb29rCwsJD6yMnNzUVubq70OjMzE8DD+6u1Wu0z7GXJqE2FwdYtbcNE6PzXkAx5rCqromPGY2d8eG6MF8+NceP5MV6V7dyUdD8qTFDWarV44403UFhYiP/9739P7S+EgEqlkl4/+u/n6fO4WbNmYerUqXrt0dHRsLa2fmqdz2puK4OtWs/0FoUG38bOnTsNvo3K6vHbk8h48NwYL54b48bzY7wqy7nJzs4uUb8KEZS1Wi0GDBiAhIQE7N27V+d+YVdXV+Tl5SE9PV1nVDk1NRVt27aV+ty6dUtvvbdv35ZGkV1dXXHkyBGd5enp6dBqtXojzY+aOHEiIiIipNeZmZnw8PBAQECAQe9r9o38zWDrLqI2EZjeohCTj5sgt7D4XxbKwrnIQIOuvzLSarWIiYlBt27dYG5urnQ59AieG+PFc2PceH6MV2U7N0V3ADyN0QflopD8999/Y9++fXB0dNRZ3rx5c5ibmyMmJgYDBgwAACQnJ+PcuXOYO3cuAMDf3x8ZGRk4evQoWrV6OBR75MgRZGRkSGHa398fM2bMQHJyMtzc3AA8HBVWq9Vo3rx5sfWp1Wqo1Wq9dnNzc4N+I+UWGDa46myrUGXw7VWGi04phv5eo2fHc2O8eG6MG8+P8aos56ak+6B4UM7KysLly5el1wkJCYiPj4eDgwPc3d3x2muv4eTJk9i+fTsKCgqk+4UdHBxgYWEBjUaDoUOHYuzYsXB0dISDgwPGjRuHRo0aSbNgNGjQAN27d8ewYcOwYsUKAMDw4cMRFBQEb29vAEBAQAB8fHwQGhqKefPmIS0tDePGjcOwYcM44wURERFRFaR4UD5+/Dg6deokvS66jWHQoEGIjIzEtm3bAABNmzbVed++ffvQsWNHAMDChQthZmaGAQMGICcnB126dMGaNWtgamoq9d+wYQPCwsKk2TGCg4N15m42NTXFjh07MHLkSLRr1w5WVlYICQnB/PnzDbHbRERERGTkFA/KHTt2hBDFz6rwpGVFLC0tsWTJEixZsqTYPg4ODli/fv0T11OzZk1s3779qdsjIiIiosqvws2jTERERERUHhiUiYiIiIhkMCgTEREREclgUCYiIiIiksGgTEREREQkg0GZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRERERCSDQZmIiIiISAaDMhERERGRDAZlIiIiIiIZDMpERERERDIYlImIiIiIZJRJUL5+/TqioqJw586dslgdEREREZHiSh2UP/nkE4wZM0Z6vXv3btSvXx+9evVC/fr1cf78+TItkIiIiIhICaUOylu2bIGPj4/0+pNPPkHjxo2xdetWeHp64rPPPivTAomIiIiIlGBW2jf8888/qFu3LgDgzp07OHbsGHbu3InAwEA8ePAAY8eOLfMiiYiIiIjKW6lHlIUQKCwsBAD88ccfMDU1xSuvvAIAcHNzw7///lu2FRIRERERKaDUQblOnTrYvn07AGDTpk1o1aoVrKysAADJycmwt7cv2wqJiIiIiBRQ6lsvRowYgQ8++ADffvst7t69i2+++UZa9scff+jcv0xEREREVFGVOii///77sLe3R1xcHFq1aoW3335bWpaTk4PBgweXZX1ERERERIoodVAGgDfeeANvvPGGXvtXX3313AURERERERmDZwrKAPDnn38iNjYW//77L4YOHQpXV1fcvHkT9vb20j3LREREREQVVamDckFBAYYPH441a9ZACAGVSoUePXrA1dUVI0aMgJ+fH6ZNm2aIWomIiIiIyk2pZ72YMWMGNm7ciHnz5uHcuXMQQkjLevTogaioqDItkIiIiIhICaUeUV6zZg0mT56MiIgIFBQU6Czz8vJCQkJCmRVHRERERKSUUo8o//PPP/D395ddZmlpiXv37j13UURERERESit1UHZ2dsbVq1dll126dAkvvvjicxdFRERERKS0Ugflnj17YsaMGfjnn3+kNpVKhYyMDHzxxRfo3bt3mRZIRERERKSEUgfladOmIT8/Hz4+Pnj11VehUqnw3//+F76+vnjw4AEmT55siDqJiIiIiMpVqYOyi4sLjh07hjfffBMnTpyAqakpTp8+jR49eiAuLg4ODg6GqJOIiIiIqFw90wNHXFxcsHz58rKuhYiIiIjIaJR6RFmr1eL+/fuyy+7fvw+tVluq9R04cAC9e/eGu7s7VCoVfv75Z53lQghERkbC3d0dVlZW6NixI86fP6/TJzc3F6NHj4aTkxNsbGwQHByMGzdu6PRJT09HaGgoNBoNNBoNQkNDcffuXZ0+SUlJ6N27N2xsbODk5ISwsDDk5eWVan+IiIiIqHIodVAeNmwY3n33Xdllw4cPx/vvv1+q9d2/fx9NmjTB0qVLZZfPnTsXCxYswNKlS3Hs2DG4urqiW7duOtPQhYeHY+vWrdi0aRMOHjyIrKwsBAUF6czzHBISgvj4eERFRSEqKgrx8fEIDQ2VlhcUFKBXr164f/8+Dh48iE2bNmHLli0YO3ZsqfaHiIiIiCqHUt96sW/fPsyePVt2We/evTFx4sRSra9Hjx7o0aOH7DIhBBYtWoRJkyahf//+AIC1a9fCxcUFGzduxIgRI5CRkYFVq1Zh3bp16Nq1KwBg/fr18PDwwO7duxEYGIiLFy8iKioKhw8fRuvWrQEAK1euhL+/Py5dugRvb29ER0fjwoULuH79Otzd3QEAn3/+OQYPHowZM2bAzs6uVPtFRERERBVbqYPyrVu34ObmJrvM1dUVKSkpz11UkYSEBKSkpCAgIEBqU6vV6NChA+Li4jBixAicOHECWq1Wp4+7uzt8fX0RFxeHwMBAHDp0CBqNRgrJANCmTRtoNBrExcXB29sbhw4dgq+vrxSSASAwMBC5ubk4ceIEOnXqJFtjbm4ucnNzpdeZmZkAHt6iUtrbUEpDbSqe3ul5t2EidP5rSIY8VpVV0THjsTM+PDfGi+fGuPH8GK/Kdm5Kuh+lDsrVqlXD5cuX0bFjR71lly9fhq2tbWlXWayi0O3i4qLT7uLigmvXrkl9LCwsYG9vr9en6P0pKSlwdnbWW7+zs7NOn8e3Y29vDwsLiyeG/1mzZmHq1Kl67dHR0bC2tn7aLj6zua0Mtmo901sUGnwbO3fuNPg2KquYmBilS6Bi8NwYL54b48bzY7wqy7nJzs4uUb9SB+VOnTph1qxZ6N+/v85UcGlpaZg9ezY6d+5c2lU+lUql0nkthNBre9zjfeT6P0ufx02cOBERERHS68zMTHh4eCAgIMCgt2v4Rv5msHUXUZsITG9RiMnHTZBb+OTj/bzORQYadP2VkVarRUxMDLp16wZzc3Oly6FH8NwYL54b48bzY7wq27kpugPgaUodlCMjI9GyZUvUq1cP//nPf1CjRg3cuHEDP/74I7Rarezo6rNydXUF8HC099HbPVJTU6XRX1dXV+Tl5SE9PV1nVDk1NRVt27aV+ty6dUtv/bdv39ZZz5EjR3SWp6enQ6vV6o00P0qtVkOtVuu1m5ubG/QbKbfAsMFVZ1uFKoNvrzJcdEox9PcaPTueG+PFc2PceH6MV2U5NyXdh1LPeuHt7Y3ff/8dTZs2xcqVKzF58mR8/fXXaNq0KX7//Xd4e3uXutjieHl5wdXVVWeYPy8vD7GxsVIIbt68OczNzXX6JCcn49y5c1Iff39/ZGRk4OjRo1KfI0eOICMjQ6fPuXPnkJycLPWJjo6GWq1G8+bNy2yfiIiIiKhieKYHjjRp0gR79uxBTk4O0tPT4eDgAEtLy2cqICsrC5cvX5ZeJyQkID4+Hg4ODqhZsybCw8Mxc+ZM1KtXD/Xq1cPMmTNhbW2NkJAQAIBGo8HQoUMxduxYODo6wsHBAePGjUOjRo2kWTAaNGiA7t27Y9iwYVixYgWAh1PZBQUFScE+ICAAPj4+CA0Nxbx585CWloZx48Zh2LBhnPGCiIiIqAp6pqBcxMrKClZWVs9VwPHjx3VmlCi633fQoEFYs2YNPvroI+Tk5GDkyJFIT09H69atER0drfOhwYULF8LMzAwDBgxATk4OunTpgjVr1sDU1FTqs2HDBoSFhUmzYwQHB+vM3WxqaoodO3Zg5MiRaNeuHaysrBASEoL58+c/1/4RERERUcX0TEE5MTERP/zwA65du4acnBydZSqVCqtWrSrxujp27Aghip9+TKVSITIyEpGRkcX2sbS0xJIlS7BkyZJi+zg4OGD9+vVPrKVmzZrYvn37U2smIiIiosqv1EF5x44d6N+/PwoKCuDs7Kz3QbanzUZBRERERFQRlDooT5o0Ce3atcOmTZtk5yYmIiIiIqoMSh2U//77b/z0008MyURERERUqZV6ejhPT09kZWUZohYiIiIiIqNR6qD83//+F/Pnzy/xo/+IiIiIiCqiUt96cfToUaSmpqJu3bro1KkTHB0ddZarVCosXry4zAokIiIiIlJCqYPyo3MPf/fdd3rLGZSJiIiIqDIodVAuLCw0RB1EREREREal1PcoExERERFVBc8clH/77TdMnDgRw4YNQ1JSEgDg2LFjuH37dpkVR0RERESklFLfepGdnY0+ffpgz5490lP43n//fdSsWRPz58+Hh4cH5s+fX+aFEhERERGVp1KPKE+aNAnHjx/Hli1bkJGRASGEtCwgIAC7d+8u0wKJiIiIiJRQ6hHlH3/8EdOnT0e/fv1QUFCgs6xmzZrSbRhERERERBVZqUeUb9++jYYNG8qvzMQEOTk5z10UEREREZHSSh2Ua9SogbNnz8ouO3PmDLy8vJ67KCIiIiIipZU6KPfv3x8zZszAqVOnpDaVSoVr165h4cKFeP3118u0QCIiIiIiJZQ6KE+ZMgXu7u5o1aoVWrRoAZVKhXfeeQe+vr5wdnbGhAkTDFEnEREREVG5KnVQtrW1RVxcHKZPn44XXngBderUgbW1NSZOnIgDBw7AysrKEHUSEREREZWrUs16kZOTg6FDh2LkyJGYMGECR4+JiIiIqNIq1YiylZUVfvnlFxQWFhqqHiIiIiIio1DqWy+aNm2Kc+fOGaIWIiIiIiKjUeqgPHv2bMydOxexsbGGqIeIiIiIyCiU+sl8I0eORFZWFjp37gx7e3u4ublBpVJJy1UqFU6fPl2mRRIRERERlbdSB2VHR0c4OTkZohYiIiIiIqNR6qAcHR0NMzMzmJiU+q4NIiIiIqIKo1Rp98GDB9LMF0RERERElVmpgrKlpSUcHR1hY2NjqHqIiIiIiIxCqe+f6N27N7Zu3WqIWoiIiIiIjEap71F+4403MHToUAwZMgT9+/fXm/UCAJo1a1ZmBRIRERERKaHUQTkwMBAAsGbNGqxdu1ZnmRACKpUKBQUFZVMdEREREZFCSh2UV69ebYg6iIiIiIiMSqmD8qBBgwxRBxFRuag1YYfBt6E2FZjbCvCN/A25Baqnv+E5JM7uZdD1ExFVZZwMmYiIiIhIRqlHlIcMGfLE5SqVCqtWrXrmgoiIiIiIjEGpg/LevXv1Zrm4c+cOsrKyUK1aNVSrVq2saiMiIiIiUkypg3JiYqJs+969ezFy5Ej8+OOPz1sTEREREZHiyuwe5c6dO2PUqFH48MMPy2qVRERERESKKdMP8/n4+ODo0aNluUrk5+fjk08+gZeXF6ysrFC7dm1MmzYNhYWFUh8hBCIjI+Hu7g4rKyt07NgR58+f11lPbm4uRo8eDScnJ9jY2CA4OBg3btzQ6ZOeno7Q0FBoNBpoNBqEhobi7t27Zbo/RERERFQxlGlQjo2NhZOTU1muEnPmzMHy5cuxdOlSXLx4EXPnzsW8efOwZMkSqc/cuXOxYMECLF26FMeOHYOrqyu6deuGe/fuSX3Cw8OxdetWbNq0CQcPHkRWVhaCgoJ0Ho4SEhKC+Ph4REVFISoqCvHx8QgNDS3T/SEiIiKiiqHU9yhPmzZNry03NxdnzpzBrl27MH78+DIprMihQ4fQp08f9Or1cK7QWrVq4bvvvsPx48cBPBxNXrRoESZNmoT+/fsDANauXQsXFxds3LgRI0aMQEZGBlatWoV169aha9euAID169fDw8MDu3fvRmBgIC5evIioqCgcPnwYrVu3BgCsXLkS/v7+uHTpEry9vct0v4iIiIjIuJU6KEdGRuq1qdVq1KpVC9OmTSvzoPzyyy9j+fLl+Ouvv1C/fn2cPn0aBw8exKJFiwAACQkJSElJQUBAgE49HTp0QFxcHEaMGIETJ05Aq9Xq9HF3d4evry/i4uIQGBiIQ4cOQaPRSCEZANq0aQONRoO4uLhig3Jubi5yc3Ol15mZmQAArVYLrVZblodCh9pUGGzd0jZMhM5/DcmQx6qyKjpmPHalw2unauN1Y9x4foxXZTs3Jd2PUgflR+8NLg8ff/wxMjIy8NJLL8HU1BQFBQWYMWMG3nzzTQBASkoKAMDFxUXnfS4uLrh27ZrUx8LCAvb29np9it6fkpICZ2dnve07OztLfeTMmjULU6dO1WuPjo6GtbV1Kfa0dOa2Mtiq9UxvYfhzvnPnToNvo7KKiYlRuoQKhdcOAbxujB3Pj/GqLOcmOzu7RP1KHZTL2/fff4/169dj48aNaNiwIeLj4xEeHg53d3edx2k/PrezEEKv7XGP95Hr/7T1TJw4EREREdLrzMxMeHh4ICAgAHZ2dk/dv2flG/mbwdZdRG0iML1FISYfN0FuoWEfw3suMtCg66+MtFotYmJi0K1bN5ibmytdToXBa6dq43Vj3Hh+jFdlOzdFdwA8TamD8vbt25GYmIhRo0bpLfvyyy/h5eWFnj17lna1xRo/fjwmTJiAN954AwDQqFEjXLt2DbNmzcKgQYPg6uoK4OGIsJubm/S+1NRUaZTZ1dUVeXl5SE9P1xlVTk1NRdu2baU+t27d0tv+7du39UarH6VWq6FWq/Xazc3NDfqNlFtg2B++OtsqVBl8e5XholOKob/XKhteOwTwujF2PD/Gq7Kcm5LuQ6mD8owZM9CnTx/ZZffv38fMmTPLNChnZ2fDxER3cg5TU1PpFhAvLy+4uroiJiYGfn5+AIC8vDzExsZizpw5AIDmzZvD3NwcMTExGDBgAAAgOTkZ586dw9y5cwEA/v7+yMjIwNGjR9Gq1cO/zR45cgQZGRlSmCYiIqqqak3YYfBtqE0F5rZ6+JcfQ/6SmTi7l8HWTZVLqaeH+/PPP9GsWTPZZX5+frhw4cJzF/Wo3r17Y8aMGdixYwcSExOxdetWLFiwAP369QPw8HaJ8PBwzJw5E1u3bsW5c+cwePBgWFtbIyQkBACg0WgwdOhQjB07Fnv27MGpU6fw9ttvo1GjRtIsGA0aNED37t0xbNgwHD58GIcPH8awYcMQFBTEGS+IiIiIqqBSjyjn5uYiLy+v2GU5OTnPXdSjlixZgsmTJ2PkyJFITU2Fu7s7RowYgU8//VTq89FHHyEnJwcjR45Eeno6WrdujejoaNja2kp9Fi5cCDMzMwwYMAA5OTno0qUL1qxZA1NTU6nPhg0bEBYWJs2OERwcjKVLl5bp/hARERFRxVDqoOzt7Y3t27cjKChIb9n27dtRv379MimsiK2tLRYtWiRNBydHpVIhMjJSduq6IpaWlliyZInOg0oe5+DggPXr1z9HtURERERUWZT61oshQ4bg66+/xpQpU6QPv926dQuRkZH4+uuvMXTo0DIvkoiIiIiovJV6RHnUqFE4duwYpk+fjs8++0ya21gIgdDQUISFhRmiTiIiIiKiclXqoKxSqfDtt99i2LBhiIqKwu3bt1G9enX06NEDL7/8siFqJCIiIiIqd8/8wJH27dujffv2ZVkLEREREZHRKPU9yocPH8YPP/wgu+yHH37AkSNHnrsoIiIiIiKllToo//e//8XZs2dll124cAGffPLJcxdFRERERKS0UgflM2fOoE2bNrLLWrdujdOnTz93UURERERESit1UL5//z7MzORvbTYxMcG9e/eeuygiIiIiIqWVOih7eXlh3759ssv27dsHT0/P5y6KiIiIiEhppQ7Kb7zxBhYuXIjVq1frtK9ZswaLFi3Cm2++WWbFEREREREppdRBecKECWjXrh2GDh0KGxsb1KtXDzY2Nhg6dCjatWuHiRMnGqJOIiIiIqJyVep5lC0sLBATE4ONGzdi165d+Pfff9GqVSv06NEDb775JkxNTQ1RJxERERFRuXqmB46YmpoiNDQUoaGhZV0PEREREZFRKHVQTk1Nxf79+5GUlASVSoWaNWuiQ4cOcHZ2NkR9RERERESKKHFQfvDgASIiIrBq1Srk5+dDCCEtMzc3x7BhwzB//nxYWloapFAiIiIiovJUoqCcn5+Pnj17Yv/+/QgICEDfvn3h5eUFIQQSExOxdetW/O9//8PFixcRHR3N+5SJiIiIqMIrUVD+6quv8Pvvv+P777/H66+/rrf8vffew/fff4/Q0FCsXLkS7733XpkXSkRERERUnko0Pdy6deswYsQI2ZBc5D//+Q+GDRuGtWvXlllxRERERERKKVFQvnDhAoKCgp7aLzg4GBcuXHjuooiIiIiIlFaioJyfnw8rK6un9lOr1cjPz3/uooiIiIiIlFaioFyrVi0cOnToqf0OHTqEWrVqPW9NRERERESKK1FQDg4Oxvz583HlypVi+/z9999YsGAB+vbtW1a1EREREREppkRBefz48bC2tkbLli0xb948/PXXX9BqtdBqtfjrr78wd+5ctG7dGtbW1hg3bpyhayYiIiIiMrgSTQ/n4OCA6OhovPrqq/j4448xYcIEneVCCPj4+GDLli2wt7c3SKFEREREROWpxE/me+mll3D27Fn89NNP2L17N5KSkgAANWvWRLdu3dCvXz+YmJRogJqIiIiIyOiVOCgDgImJCV577TW89tprhqqHiIiIiMgocAiYiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyShSUv/jiC6Smphq6FiIiIiIio1GioDxmzBgkJiYCAExNTXH06FFD1kREREREpLgSBWVbW1vcvXsXwMOHixARERERVXYlmke5adOmCAsLQ9euXQEAn3/+OVxcXGT7qlQqLF68uOwqJCIiIiJSQImC8qJFi/DOO+/gq6++gkqlwo8//lhsXwZlIiIiIqoMSnTrRdOmTXHq1Cnk5eVBCIHDhw+jsLBQ9qugoMDQNRMRERERGVypp4dbvXo16tSpY4haiIiIiIiMRqmD8qBBg+Do6Ih79+4hOjoa3333HWJiYnDv3j1D1AcA+Oeff/D222/D0dER1tbWaNq0KU6cOCEtF0IgMjIS7u7usLKyQseOHXH+/HmddeTm5mL06NFwcnKCjY0NgoODcePGDZ0+6enpCA0NhUajgUajQWhoqPQhRiIiIiKqWp7pgSPz58+Hu7s7evTogbfeegvdu3eHu7s7FixYUNb1IT09He3atYO5uTl27dqFCxcu4PPPP0e1atWkPnPnzsWCBQuwdOlSHDt2DK6urujWrZtOeA8PD8fWrVuxadMmHDx4EFlZWQgKCtK5VSQkJATx8fGIiopCVFQU4uPjERoaWub7RERERETGr0Qf5nvUt99+i48++gg9evTA4MGD4e7ujps3b2Lt2rUYP348qlevXqbhcs6cOfDw8MDq1aultlq1akn/FkJg0aJFmDRpEvr37w8AWLt2LVxcXLBx40aMGDECGRkZWLVqFdatWyfN3LF+/Xp4eHhg9+7dCAwMxMWLFxEVFYXDhw+jdevWAICVK1fC398fly5dgre3d5ntExEREREZv1IH5YULFyIkJATr16/XaX/99dfx9ttvY+HChWUalLdt24bAwEC8/vrriI2NRY0aNTBy5EgMGzYMAJCQkICUlBQEBARI71Gr1ejQoQPi4uIwYsQInDhxAlqtVqePu7s7fH19ERcXh8DAQBw6dAgajUYKyQDQpk0baDQaxMXFFRuUc3NzkZubK73OzMwEAGi1Wmi12jI7Do9Tmxp+Pmu1idD5ryEZ8lhVVkXHjMeudHjtVG28bp5dZbp2eP5Lr7JdOyXdj1IH5T///BOzZs2SXfb222+jX79+pV3lE129ehXLli1DREQE/vvf/+Lo0aMICwuDWq3GwIEDkZKSAgB68zq7uLjg2rVrAICUlBRYWFjA3t5er0/R+1NSUuDs7Ky3fWdnZ6mPnFmzZmHq1Kl67dHR0bC2ti7dzpbC3FYGW7We6S0KDb6NnTt3GnwblVVMTIzSJVQovHYI4HXzLCrTtcPr5tlVlmsnOzu7RP1KHZStrKyQlpYmuywtLQ1WVlalXeUTFRYWokWLFpg5cyYAwM/PD+fPn8eyZcswcOBAqZ9KpdJ5nxBCr+1xj/eR6/+09UycOBERERHS68zMTHh4eCAgIAB2dnZP3rnn4Bv5m8HWXURtIjC9RSEmHzdBbuGTj+XzOhcZaND1V0ZarRYxMTHo1q0bzM3NlS6nwuC1U7Xxunl2lena4XVTepXt2im6A+BpSh2U27dvj8jISHTs2BHu7u5Se0pKCqZNm4ZXXnmltKt8Ijc3N/j4+Oi0NWjQAFu2bAEAuLq6Stt3c3OT+qSmpkqjzK6ursjLy0N6errOqHJqairatm0r9bl165be9m/fvl3sUwiBh7d5qNVqvXZzc3ODfiPlFhj2h6/OtgpVBt9eZbjolGLo77XKhtcOAbxunkVlunZ47p9dZbl2SroPpZ71YubMmbh16xbq1q2L3r17Y/jw4ejduzfq1KmDlJQUaeS3rLRr1w6XLl3Safvrr7/g6ekJAPDy8oKrq6vOnwLy8vIQGxsrheDmzZvD3Nxcp09ycjLOnTsn9fH390dGRgaOHj0q9Tly5AgyMjKkPkRERERUdZR6RLlhw4Y4duwYpkyZgn379uHOnTtwdHRE3759MWXKFNSvX79MCxwzZgzatm2LmTNnYsCAATh69Ci++uorfPXVVwAe3i4RHh6OmTNnol69eqhXrx5mzpwJa2trhISEAAA0Gg2GDh2KsWPHwtHREQ4ODhg3bhwaNWokzYLRoEEDdO/eHcOGDcOKFSsAAMOHD0dQUBBnvCAiIiKqgkodlAGgfv36+O6778q6FlktW7bE1q1bMXHiREybNg1eXl5YtGgR3nrrLanPRx99hJycHIwcORLp6elo3bo1oqOjYWtrK/VZuHAhzMzMMGDAAOTk5KBLly5Ys2YNTE1NpT4bNmxAWFiYNDtGcHAwli5dWi77SURERETG5ZmCcnkLCgpCUFBQsctVKhUiIyMRGRlZbB9LS0ssWbIES5YsKbaPg4OD3rR3RERERFQ1PdOT+YiIiIiIKjsGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyXimoDxkyBDcuHGjrGshIiIiIjIazxSU165di7S0NABAYWEhOnfujL///rtMCyMiIiIiUlKJ5lH28vJC69at0bp1a7Rs2RJCCKhUD5/BLoTA/v37ce/ePYMWSkRERERUnkoUlMPCwnDkyBEsXrwYSUlJUKlU+OCDDxAQEIC2bdsCgBSciYiIiIgqgxLdejFmzBhs2rQJiYmJuHnzJoQQcHJywqZNm9CtWzeoVCpERERg+fLlvHeZiIiIiCqFEgXlR+8/dnV1BQBERkbi3LlzSE5OhhACdnZ2mDlzJjw9PQ1TKRERERFROSrRrRfe3t6wt7dHq1at0KpVK6hUKhQUFAAAHB0dAQBTpkxBs2bNEB8fb7BiiYiIiIjKS4mCcnx8PI4dO4Zjx45h+/btEEKgbdu2aNmyJV5++WWoVCoUFhYCAJo2bWrIeomIiIiIykWJgnLjxo3RuHFjDB06FABgYmKC6dOn4+bNm/jtt98ghEDHjh3RpUsXBAcHS/2IiIiIiCqqZ34yX0BAABYsWIAjR44AAD755BM4Ojpi0qRJZVYcEREREZFSSjSi/DhPT09YWFgA+L9p4QICAjBhwgQIIcquOiIiIiIihTxTUE5ISJD+rVKp0KFDB9ja2kqviYiIiIgqumcKyo8yMTHBvn37yqIWIiIiIiKj8cz3KBMRERERVWYMykREREREMhiUiYiIiIhkMCgTEREREclgUCYiIiIiksGgTEREREQkg0GZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRERERCSDQZmIiIiISAaDMhERERGRDAZlIiIiIiIZDMpERERERDIYlImIiIiIZFS4oDxr1iyoVCqEh4dLbUIIREZGwt3dHVZWVujYsSPOnz+v877c3FyMHj0aTk5OsLGxQXBwMG7cuKHTJz09HaGhodBoNNBoNAgNDcXdu3fLYa+IiIiIyNhUqKB87NgxfPXVV2jcuLFO+9y5c7FgwQIsXboUx44dg6urK7p164Z79+5JfcLDw7F161Zs2rQJBw8eRFZWFoKCglBQUCD1CQkJQXx8PKKiohAVFYX4+HiEhoaW2/4RERERkfGoMEE5KysLb731FlauXAl7e3upXQiBRYsWYdKkSejfvz98fX2xdu1aZGdnY+PGjQCAjIwMrFq1Cp9//jm6du0KPz8/rF+/HmfPnsXu3bsBABcvXkRUVBS+/vpr+Pv7w9/fHytXrsT27dtx6dIlRfaZiIiIiJRjpnQBJfXBBx+gV69e6Nq1Kz777DOpPSEhASkpKQgICJDa1Go1OnTogLi4OIwYMQInTpyAVqvV6ePu7g5fX1/ExcUhMDAQhw4dgkajQevWraU+bdq0gUajQVxcHLy9vWXrys3NRW5urvQ6MzMTAKDVaqHVasts/x+nNhUGW7e0DROh819DMuSxqqyKjhmPXenw2qnaeN08u8p07fD8l15lu3ZKuh8VIihv2rQJJ0+exLFjx/SWpaSkAABcXFx02l1cXHDt2jWpj4WFhc5IdFGfovenpKTA2dlZb/3Ozs5SHzmzZs3C1KlT9dqjo6NhbW39lD17dnNbGWzVeqa3KDT4Nnbu3GnwbVRWMTExSpdQofDaIYDXzbOoTNcOr5tnV1munezs7BL1M/qgfP36dXz44YeIjo6GpaVlsf1UKpXOayGEXtvjHu8j1/9p65k4cSIiIiKk15mZmfDw8EBAQADs7OyeuP3n4Rv5m8HWXURtIjC9RSEmHzdBbuGTj+XzOhcZaND1V0ZarRYxMTHo1q0bzM3NlS6nwuC1U7Xxunl2lena4XVTepXt2im6A+BpjD4onzhxAqmpqWjevLnUVlBQgAMHDmDp0qXS/cMpKSlwc3OT+qSmpkqjzK6ursjLy0N6errOqHJqairatm0r9bl165be9m/fvq03Wv0otVoNtVqt125ubm7Qb6TcAsP+8NXZVqHK4NurDBedUgz9vVbZ8NohgNfNs6hM1w7P/bOrLNdOSffB6D/M16VLF5w9exbx8fHSV4sWLfDWW28hPj4etWvXhqurq86fAvLy8hAbGyuF4ObNm8Pc3FynT3JyMs6dOyf18ff3R0ZGBo4ePSr1OXLkCDIyMqQ+RERERFR1GP2Isq2tLXx9fXXabGxs4OjoKLWHh4dj5syZqFevHurVq4eZM2fC2toaISEhAACNRoOhQ4di7NixcHR0hIODA8aNG4dGjRqha9euAIAGDRqge/fuGDZsGFasWAEAGD58OIKCgor9IB8RERERVV5GH5RL4qOPPkJOTg5GjhyJ9PR0tG7dGtHR0bC1tZX6LFy4EGZmZhgwYABycnLQpUsXrFmzBqamplKfDRs2ICwsTJodIzg4GEuXLi33/SEiIiIi5VXIoLx//36d1yqVCpGRkYiMjCz2PZaWlliyZAmWLFlSbB8HBwesX7++jKokIiIioorM6O9RJiIiIiJSAoMyEREREZEMBmUiIiIiIhkMykREREREMhiUiYiIiIhkMCgTEREREclgUCYiIiIiksGgTEREREQkg0GZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRERERCSDQZmIiIiISAaDMhERERGRDAZlIiIiIiIZDMpERERERDIYlImIiIiIZDAoExERERHJYFAmIiIiIpLBoExEREREJINBmYiIiIhIBoMyEREREZEMBmUiIiIiIhkMykREREREMhiUiYiIiIhkMCgTEREREclgUCYiIiIiksGgTEREREQkg0GZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQYfVCeNWsWWrZsCVtbWzg7O6Nv3764dOmSTh8hBCIjI+Hu7g4rKyt07NgR58+f1+mTm5uL0aNHw8nJCTY2NggODsaNGzd0+qSnpyM0NBQajQYajQahoaG4e/euoXeRiIiIiIyQ0Qfl2NhYfPDBBzh8+DBiYmKQn5+PgIAA3L9/X+ozd+5cLFiwAEuXLsWxY8fg6uqKbt264d69e1Kf8PBwbN26FZs2bcLBgweRlZWFoKAgFBQUSH1CQkIQHx+PqKgoREVFIT4+HqGhoeW6v0RERERkHMyULuBpoqKidF6vXr0azs7OOHHiBF555RUIIbBo0SJMmjQJ/fv3BwCsXbsWLi4u2LhxI0aMGIGMjAysWrUK69atQ9euXQEA69evh4eHB3bv3o3AwEBcvHgRUVFROHz4MFq3bg0AWLlyJfz9/XHp0iV4e3uX744TERERkaKMPig/LiMjAwDg4OAAAEhISEBKSgoCAgKkPmq1Gh06dEBcXBxGjBiBEydOQKvV6vRxd3eHr68v4uLiEBgYiEOHDkGj0UghGQDatGkDjUaDuLi4YoNybm4ucnNzpdeZmZkAAK1WC61WW3Y7/hi1qTDYuqVtmAid/xqSIY9VZVV0zHjsSofXTtXG6+bZVaZrh+e/9CrbtVPS/ahQQVkIgYiICLz88svw9fUFAKSkpAAAXFxcdPq6uLjg2rVrUh8LCwvY29vr9Sl6f0pKCpydnfW26ezsLPWRM2vWLEydOlWvPTo6GtbW1qXYu9KZ28pgq9YzvUWhwbexc+dOg2+jsoqJiVG6hAqF1w4BvG6eRWW6dnjdPLvKcu1kZ2eXqF+FCsqjRo3CmTNncPDgQb1lKpVK57UQQq/tcY/3kev/tPVMnDgRERER0uvMzEx4eHggICAAdnZ2T9z+8/CN/M1g6y6iNhGY3qIQk4+bILfwycfyeZ2LDDTo+isjrVaLmJgYdOvWDebm5kqXU2Hw2qnaeN08u8p07fC6Kb3Kdu0U3QHwNBUmKI8ePRrbtm3DgQMH8OKLL0rtrq6uAB6OCLu5uUntqamp0iizq6sr8vLykJ6erjOqnJqairZt20p9bt26pbfd27dv641WP0qtVkOtVuu1m5ubG/QbKbfAsD98dbZVqDL49irDRacUQ3+vVTa8dgjgdfMsKtO1w3P/7CrLtVPSfTD6WS+EEBg1ahR++ukn7N27F15eXjrLvby84OrqqvOngLy8PMTGxkohuHnz5jA3N9fpk5ycjHPnzkl9/P39kZGRgaNHj0p9jhw5goyMDKkPEREREVUdRj+i/MEHH2Djxo345ZdfYGtrK90vrNFoYGVlBZVKhfDwcMycORP16tVDvXr1MHPmTFhbWyMkJETqO3ToUIwdOxaOjo5wcHDAuHHj0KhRI2kWjAYNGqB79+4YNmwYVqxYAQAYPnw4goKCOOMFERERURVk9EF52bJlAICOHTvqtK9evRqDBw8GAHz00UfIycnByJEjkZ6ejtatWyM6Ohq2trZS/4ULF8LMzAwDBgxATk4OunTpgjVr1sDU1FTqs2HDBoSFhUmzYwQHB2Pp0qWG3UEiIiIiMkpGH5SFePoUMSqVCpGRkYiMjCy2j6WlJZYsWYIlS5YU28fBwQHr169/ljKJiIiIqJIx+nuUiYiIiIiUwKBMRERERCSDQZmIiIiISIbR36NMVBHVmrDD4NtQmwrMbfXwIQCGnG80cXYvg62biIjImHFEmYiIiIhIBoMyEREREZEMBmUiIiIiIhkMykREREREMvhhPiIiIqIKrDJ9gBwwrg+Rc0SZiIiIiEgGgzIRERERkQwGZSIiIiIiGQzKREREREQyGJSJiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRERERCSDQZmIiIiISAaDMhERERGRDAZlIiIiIiIZDMpERERERDIYlImIiIiIZDAoExERERHJYFAmIiIiIpLBoExEREREJINBmYiIiIhIBoMyEREREZEMBmUiIiIiIhkMykREREREMhiUiYiIiIhkmCldABEREQDUmrDD4NtQmwrMbQX4Rv6G3AKVQbeVOLuXQddPRIbHEWUiIiIiIhkMykREREREMhiUiYiIiIhkMCgTEREREclgUCYiIiIiksGgTEREREQkg0FZxv/+9z94eXnB0tISzZs3x++//650SURERERUzhiUH/P9998jPDwckyZNwqlTp9C+fXv06NEDSUlJSpdGREREROWIQfkxCxYswNChQ/Huu++iQYMGWLRoETw8PLBs2TKlSyMiIiKicsQn8z0iLy8PJ06cwIQJE3TaAwICEBcXJ/ue3Nxc5ObmSq8zMjIAAGlpadBqtQar1Sz/vsHWLW2jUCA7uxBmWhMUFBr2CVZ37twx6PrLW2U6Pzw3z7ANXjvPhOfGuFWm88Nz8wzbqGTXzr179wAAQogndxQk+eeffwQA8ccff+i0z5gxQ9SvX1/2PVOmTBEA+MUvfvGLX/ziF7/4VcG+rl+//sRsyBFlGSqV7m9KQgi9tiITJ05ERESE9LqwsBBpaWlwdHQs9j0VRWZmJjw8PHD9+nXY2dkpXQ49hufHePHcGC+eG+PG82O8Ktu5EULg3r17cHd3f2I/BuVHODk5wdTUFCkpKTrtqampcHFxkX2PWq2GWq3WaatWrZqhSlSEnZ1dpbgoKiueH+PFc2O8eG6MG8+P8apM50aj0Ty1Dz/M9wgLCws0b94cMTExOu0xMTFo27atQlURERERkRI4ovyYiIgIhIaGokWLFvD398dXX32FpKQkvPfee0qXRkRERETliEH5Mf/5z39w584dTJs2DcnJyfD19cXOnTvh6empdGnlTq1WY8qUKXq3lpBx4PkxXjw3xovnxrjx/BivqnpuVEI8bV4MIiIiIqKqh/coExERERHJYFAmIiIiIpLBoExEREREJINBmYiIiIhIBoMyEREREZEMBmXSsXPnTvz222967b/99ht27dqlQEVEFUteXh4uXbqE/Px8pUuh/8/U1BSpqal67Xfu3IGpqakCFRFRRcGgTDomTJiAgoICvXYhBCZMmKBARVQkLCwMX3zxhV770qVLER4eXv4FkY7s7GwMHToU1tbWaNiwIZKSkgA8PG+zZ89WuLqqrbhZUHNzc2FhYVHO1dDjpk2bhuzsbL32nJwcTJs2TYGKiP4P51EmHVZWVrh48SJq1aql056YmIiGDRvi/v37yhRGqFGjBrZt24bmzZvrtJ88eRLBwcG4ceOGQpURAHz44Yf4448/sGjRInTv3h1nzpxB7dq1sW3bNkyZMgWnTp1SusQqp+gXyzFjxmD69Ol44YUXpGUFBQU4cOAAEhMTeW4UZmpqiuTkZDg7O+u037lzB87OzrKDN1R+fv/9d6xYsQJXrlzB5s2bUaNGDaxbtw5eXl54+eWXlS7P4PhkPtKh0Whw9epVvaB8+fJl2NjYKFMUAXj4Q0Oj0ei129nZ4d9//1WgInrUzz//jO+//x5t2rSBSqWS2n18fHDlyhUFK6u6Fi5cCODhiPLy5ct1brOwsLBArVq1sHz5cqXKo/9PCKFzzRQ5ffo0HBwcFKiIimzZsgWhoaF46623cOrUKeTm5gIA7t27h5kzZ2Lnzp0KV2h4DMqkIzg4GOHh4di6dSvq1KkD4GFIHjt2LIKDgxWurmqrW7cuoqKiMGrUKJ32Xbt2oXbt2gpVRUVu376tNyIGAPfv35cNAWR4CQkJAIBOnTrhp59+gr29vcIV0aPs7e2hUqmgUqlQv359neukoKAAWVlZeO+99xSskD777DMsX74cAwcOxKZNm6T2tm3bVpnbYhiUSce8efPQvXt3vPTSS3jxxRcBADdu3ED79u0xf/58haur2iIiIjBq1Cjcvn0bnTt3BgDs2bMHn3/+ORYtWqRscYSWLVtix44dGD16NABIP/RXrlwJf39/JUur8vbt26d0CSRj0aJFEEJgyJAhmDp1qs5fzIpG/HntKOvSpUt45ZVX9Nrt7Oxw9+7d8i9IAQzKpEOj0SAuLg4xMTE4ffo0rKys0LhxY9kLhcrXkCFDkJubixkzZmD69OkAgFq1amHZsmUYOHCgwtXRrFmz0L17d1y4cAH5+flYvHgxzp8/j0OHDiE2Nlbp8qq8GzduYNu2bUhKSkJeXp7OsgULFihUVdU2aNAgAICXlxfatm0Lc3NzhSuix7m5ueHy5ct6t2MePHiwyvwlkx/mI6qAbt++DSsrK50PJ5Hyzp49i/nz5+PEiRMoLCxEs2bN8PHHH6NRo0ZKl1al7dmzB8HBwfDy8sKlS5fg6+uLxMRECCHQrFkz7N27V+kSq5zMzMwS97WzszNgJfQkc+fOxdq1a/HNN9+gW7du2LlzJ65du4YxY8bg008/1bsVsDJiUCZ88cUXGD58OCwtLWWnH3tUWFhYOVVFRFQ2WrVqhe7du2PatGmwtbXF6dOn4ezsjLfeegvdu3fH+++/r3SJVY6JiclT790v+pAfZ71Q1qRJk7Bw4UI8ePAAAKBWqzFu3DjpL5uVHYMywcvLC8ePH4ejoyO8vLyK7adSqXD16tVyrIyaNWuGPXv2wN7eHn5+fk/8wXLy5MlyrIzkFBYW4vLly0hNTUVhYaHOMt6+pBxbW1vEx8ejTp06sLe3x8GDB9GwYUOcPn0affr0QWJiotIlVjmluR2pQ4cOBqyESiI7OxsXLlxAYWEhfHx8qtRfM3mPMkmfDH/836S8Pn36QK1WAwD69u2rbDH0RIcPH0ZISAiuXbum94ALjoopy8bGRprWyt3dHVeuXEHDhg0BgFMrKoTht2IYMmQIFi9eDFtbW7Ro0UJqv3//PkaPHo1vvvlGwerKB0eUSUdOTg6srKxklyUnJ8PNza2cKyKqGJo2bYr69etj6tSpcHNz0xv9l5sDm8pH37590atXLwwbNgwfffQRtm7disGDB0tTxu3evVvpEqu0AwcOPHE5/xqjnOIeBvPvv//C1dUV+fn5ClVWfjiiTDr8/PywceNGNGvWTKd98+bNeP/993H79m2FKqPdu3eja9eusstWrFiBESNGlHNF9Ki///4bmzdvRt26dZUuhR6zYMECZGVlAQAiIyORlZWF77//HnXr1pUeSkLK6dixo17b43MqU/nKzMyEEAJCCNy7dw+WlpbSsoKCAuzcuVN23vjKiEGZdHTr1g1t27ZFZGQkPv74Y9y/fx+jRo3Cjz/+iNmzZytdXpXWq1cvjBo1CrNmzYKFhQWAh7NfDBkyBH/88QeDssJat26Ny5cvMygboUensbK2tsb//vc/Bauhx6Wnp+u81mq1OHXqFCZPnowZM2YoVFXVVq1aNZ2HwTxOpVJh6tSpClRW/njrBemJiorCO++8g7p16+LmzZuws7PDhg0b4OPjo3RpVdqRI0cQGhoKKysrbNy4EYmJiRgyZAh8fHzw7bffwsPDQ+kSq7StW7fik08+wfjx49GoUSO9OWEbN26sUGUEAHfv3sXmzZtx5coVjB8/Hg4ODjh58iRcXFxQo0YNpcsjGQcOHMCYMWNw4sQJpUupcmJjYyGEQOfOnbFlyxadR4lbWFjA09MT7u7uClZYfhiUSU9hYSFGjx6NZcuWwczMDL/++isCAwOVLovw8AMU7733Hn788UcUFhbis88+w/jx4/mIZCNgYmKi16ZSqTjFlRE4c+YMunbtCo1Gg8TERFy6dAm1a9fG5MmTce3aNXz77bdKl0gyLl68iJYtW0q3zVD5u3btGjw8PGT//1ZV8NYL0nHlyhWEhIQgJSUFv/32G2JjY9GnTx+EhYVhxowZfHKSwi5duoRjx47hxRdfxM2bN/Hnn38iOzsbNjY2SpdW5XHGGOMVERGBwYMHY+7cubC1tZXae/TogZCQEAUrI+DhLzKPEkIgOTkZs2fPRpMmTRSqigDA09MTwMPp4eSealkV/lLGEWXSYWtri169emH58uWoVq0aACAuLg4DBw6Era0tTp06pWyBVdjs2bMxZcoUDB8+HPPmzcOVK1fw9ttvIzMzE+vXr4e/v7/SJRIZJY1Gg5MnT6JOnTrSA0dq166Na9euwdvbW3qQAimj6OEjj8eRNm3a4JtvvsFLL72kUGV0+/ZtvPPOO9i1a5fs8qrwl7KqO5ZOsv73v/9h06ZNUkgGgLZt2+LUqVN6M2FQ+Vq8eDF+/vlnLFmyBJaWlmjYsCGOHj2K/v37y35qnMrfunXr0K5dO7i7u+PatWsAgEWLFuGXX35RuLKqzdLSUvaRyZcuXUL16tUVqIgelZCQgKtXryIhIQEJCQm4du0asrOzERcXx5CssPDwcKSnp+Pw4cOwsrJCVFQU1q5di3r16mHbtm1Kl1cuOKJMxbpx4wZUKhU/6GIk/v33Xzg5Ockui42N5QT+Clu2bBk+/fRThIeHY8aMGTh37hxq166NNWvWYO3atdi3b5/SJVZZw4cPx+3bt/HDDz/AwcEBZ86cgampKfr27YtXXnkFixYtUrpEeszdu3d1BmxIGW5ubvjll1/QqlUr2NnZ4fjx46hfvz62bduGuXPn4uDBg0qXaHAcUSYdhYWFmDZtGjQaDTw9PVGzZk1Uq1YN06dP13skL5UvJycnFBQUYMuWLfjss88wY8YM/PTTTygoKGBINgJLlizBypUrMWnSJJiamkrtLVq0wNmzZxWsjObPn4/bt2/D2dkZOTk56NChA+rWrQtbW1tOP2YE5syZg++//156PWDAADg4OKBGjRo4ffq0gpXR/fv3pfmSHRwcpGcpNGrUCCdPnlSytHLDD/ORjkmTJmHVqlWYPXs22rVrByEE/vjjD0RGRuLBgwf8oaKgy5cvo2fPnvjnn3/g7e0NIQT++usveHh4YMeOHahTp47SJVZpCQkJ8PPz02tXq9W4f/++AhVRETs7Oxw8eBB79+7FyZMnUVhYiGbNmhX7AB8qXytWrMD69esBADExMYiJiUFUVBR++OEHjB8/HtHR0QpXWHV5e3vj0qVLqFWrFpo2bYoVK1agVq1aWL58eZV5Ui+DMulYu3Ytvv76awQHB0ttTZo0QY0aNTBy5EgGZQWFhYWhTp06OHz4sDSn5Z07d/D2228jLCwMO3bsULjCqs3Lywvx8fHSp8SL7Nq1i3OQKyg/Px+WlpaIj49H586d0blzZ6VLosckJydL88Bv374dAwYMQEBAAGrVqoXWrVsrXF3VFh4ejuTkZADAlClTEBgYiA0bNsDCwgJr1qxRtrhywqBMOtLS0mQ/PPHSSy8hLS1NgYqoSGxsrE5IBgBHR0dp9J+UNX78eHzwwQd48OABhBA4evQovvvuO8yaNQtff/210uVVWWZmZvD09KwSn86vqOzt7XH9+nV4eHggKioKn332GYCH08TxvCnrrbfekv7t5+eHxMRE/Pnnn6hZs2axn5mpbBiUSUeTJk2wdOlSfPHFFzrtS5cu5XyWClOr1bh3755ee1ZWlvRIa1LOO++8g/z8fHz00UfIzs5GSEgIatSogcWLF+ONN95Qurwq7ZNPPsHEiROxfv16nV80yTj0798fISEhqFevHu7cuYMePXoAAOLj4/lIeCNjbW1d5WbA4qwXpCM2Nha9evVCzZo14e/vD5VKhbi4OFy/fh07d+5E+/btlS6xyho4cCBOnjyJVatWoVWrVgAePtZ62LBhaN68eZX5M1hF8O+//6KwsFD6EAwpy8/PD5cvX4ZWq4Wnp6feA3qqyoeSjJVWq8XixYtx/fp1DB48WLrXf9GiRXjhhRfw7rvvKlxh1VVQUIA1a9Zgz549SE1N1ftQ/969exWqrPwwKJOOpKQkmJmZ4csvv8Sff/4JIQR8fHwwcuRI5Ofno2bNmkqXWGXdvXsXgwYNwq+//io9ITE/Px/BwcFYvXo1p1JSWEJCAvLz81GvXj2d9r///hvm5uaoVauWMoURpk6d+sTlU6ZMKadKiCqWUaNGYc2aNejVqxfc3NygUql0li9cuFChysoPgzLpMDU1RXJyst5I2J07d+Ds7Mz7xYzA5cuXcfHiRemXGP5p0jh06NABQ4YMwaBBg3Ta169fj6+//hr79+9XpjCiCuCvv/7C/v37ZUctP/30U4WqIicnJ3z77bfo2bOn0qUohkGZdJiYmCAlJUUvKF+7dg0+Pj6c5kpB06ZNw7hx42Btba3TnpOTg3nz5vGHicLs7Oxw8uRJvV9cLl++jBYtWuDu3bvKFEaSvLw82SDGv5Qpa+XKlXj//ffh5OQEV1dXnVFLlUrFW2MU5O7ujv3796N+/fpKl6IYBmUCAERERAB4+JjkYcOG6YSxgoICHDlyBKampvjjjz+UKrHK42i/cdNoNNi/f7/eXMonTpxAx44dZT+ISeXjr7/+wtChQxEXF6fTLoSASqXitaMwT09PjBw5Eh9//LHSpdBjPv/8c1y9ehVLly7Vu+2iquCsFwQAOHXqFICHPzjOnj2rM4uChYUFmjRpgnHjxilVHuH/fqg/7vTp0/wkvxFo3749Zs2ahe+++056Ml9BQQFmzZqFl19+WeHqqrZ33nkHZmZm2L59u+x9lqSs9PR0vP7660qXQf9f//79dV7v3bsXu3btQsOGDaXPxxT56aefyrM0RTAoEwBg3759AB7+QFm8eDHs7OwUroiK2NvbQ6VSQaVSoX79+jo/5AsKCpCVlYX33ntPwQoJAObOnYtXXnkF3t7e0uwwv//+OzIzM6vEJ8ONWXx8PE6cOCE7Rzwp7/XXX0d0dDT/P2YkNBqNzut+/fopVIlx4K0XREZu7dq1EEJgyJAhWLRokc7/xCwsLFCrVi34+/srWCEVuXnzJpYuXYrTp0/DysoKjRs3xqhRozjir7CWLVti4cKFHNk3UrNmzcKCBQvQq1cvNGrUSG/UMiwsTKHKqKT++OMPtGjRAmq1WulSyhyDMlEFERsbi3bt2sHM7Ml/CJo9ezbee+89ThdHVVpmZqb07+PHj+OTTz7BzJkzZYMY/4KmLC8vr2KXqVQqXL16tRyroWdhZ2eH+Ph41K5dW+lSyhyDMlElU5n/h2XMoqKi8MILL0ijll9++SVWrlwJHx8ffPnll7C3t1e4wqrFxMRE5zYluXv8+WE+orJha2uL06dPV8qfO7xHmaiS4e++yhg/fjzmzJkDADh79iwiIiIwduxY7N27FxEREVi9erXCFVYtRZ+7AIDExER4eHhIH7IsUlhYiKSkpPIujfBwpqXp06fDxsZGmnVJjkqlwueff16OlRHpYlAmIioDCQkJ8PHxAQBs2bIFvXv3xsyZM3Hy5MkqPVm/Ujp06CD9u3PnzsVOrdi1a1e9h8SQ4Z06dQparVb6d3E4QwkpjUGZiKgMWFhYIDs7GwCwe/duDBw4EADg4OCgc78slb/iplbMysqCpaWlAhXRoyP+j/6byNgwKBMRlYGXX34ZERERaNeuHY4ePYrvv/8ewMOHXbz44osKV1c1Ff1JX6VSYfLkybIPUmratKlC1RFVHpV55J9BmYioDCxduhQjR47E5s2bsWzZMtSoUQMAsGvXLnTv3l3h6qomPkiJqHxU5s/GcNYLokqmZ8+eWLVqFdzc3JQuhcgo8EFKRM8mMjIS77zzDjw9PZUuRTEMykQVyJUrV7B69WpcuXIFixcvhrOzM6KiouDh4YGGDRsqXV6VV1BQgK1bt+LixYtQqVR46aWX0Ldv36fOfU1EZIyaN2+O06dPo0OHDhg6dCj69+9f5e7rN1G6ACIqmdjYWDRq1AhHjhzBTz/9hKysLADAmTNnMGXKFIWro3PnzqFevXoYNGgQtm7dip9++gmDBw9GvXr1cPbsWaXLIyIqtRMnTuDkyZNo3LgxxowZAzc3N7z//vs4duyY0qWVG44oE1UQ/v7+eP311xEREaEzufuxY8fQt29f/PPPP0qXWKW1adMGzs7OWLt2rfRwkfT0dAwePBipqak4dOiQwhUSET27/Px8/Prrr1i9ejWioqLg7e2Nd999F4MHD4ZGo1G6PIPhiDJRBXH27Fn069dPr7169eq4c+eOAhXRo06fPo1Zs2bpPIHP3t4eM2bMQHx8vHKFERGVgcLCQuTl5SE3NxdCCDg4OGDZsmXw8PCQZvmpjBiUiSqIatWqITk5Wa/91KlT0gwLpBxvb2/cunVLrz01NRV169ZVoCIioud34sQJjBo1Cm5ubhgzZgz8/Pxw8eJFxMbG4s8//8SUKVMQFhamdJkGw6BMVEGEhITg448/RkpKClQqFQoLC/HHH39g3Lhx0sMtqHxlZmZKXzNnzkRYWBg2b96MGzdu4MaNG9i8eTPCw8OlR1sTEVUkjRs3Rps2bZCQkIBVq1bh+vXrmD17ts4v/wMHDsTt27cVrNKweI8yUQWh1WoxePBgbNq0CUIImJmZoaCgACEhIVizZg1MTU2VLrHKMTEx0Zlov+h/p0Vtj74uKCgo/wKJiJ7D9OnTMWTIkCr9V0vOWURUAQghcPPmTaxcuRLTp0/HyZMnUVhYCD8/P9SrV0/p8qosPnqXiCozIYTO5y6K5OTkYN68efj0008VqKp8cUSZqAIoLCyEpaUlzp8/z2BsxO7evYtVq1ZJ8yg3aNAAQ4cOrdSfCCeiysvU1BTJyclwdnbWab9z5w6cnZ2rxF/KeI8yUQVgYmKCevXqcXYLI3b8+HHUrVsXCxcuRFpaGv79918sXLgQderUwcmTJ5Uuj4io1IQQOreXFTl9+jQcHBwUqKj8cUSZqILYsWMHZs+ejWXLlsHX11fpcugx7du3R926dbFy5UrpSXz5+fl49913cfXqVRw4cEDhComISsbe3h4qlQoZGRmws7PTCcsFBQXIysrCe++9hy+//FLBKssHgzJRBWFvb4/s7Gzk5+fDwsICVlZWOsvT0tIUqowAwMrKCqdOncJLL72k037hwgW0aNEC2dnZClVGRFQ6a9euhRACQ4YMwaJFi3RuH7OwsECtWrXg7++vYIXlhx/mI6ogFi1apHQJ9AR2dnZISkrSC8rXr1+Hra2tQlUREZXeoEGDAABeXl5o27YtzM3NFa5IORxRJiIqA2FhYdi6dSvmz5+Ptm3bQqVS4eDBgxg/fjxeffVV/qJDRBVCZmYm7OzspH8/SVG/yoxBmagCuXLlClavXo0rV65g8eLFcHZ2RlRUFDw8PNCwYUOly6vS8vLyMH78eCxfvhz5+fkAAHNzc7z//vuYPXs21Gq1whUSET3dozNdPD5XfJGiD/lVhVkvGJSJKojY2Fj06NED7dq1w4EDB3Dx4kXUrl0bc+fOxdGjR7F582alSyQA2dnZuHLlCoQQqFu3LqytrZUuiYioxGJjY9GuXTuYmZkhNjb2iX07dOhQTlUph0GZqILw9/fH66+/joiICNja2uL06dOoXbs2jh07hr59++Kff/5RukQiIqpEkpKS4OHhoTeqLITA9evXUbNmTYUqKz+cR5mogjh79iz69eun1169enXOr0xERGXOy8sLt2/f1mtPS0uDl5eXAhWVPwZlogqiWrVqSE5O1ms/deoUatSooUBFRERUmRX3wJGsrCxYWloqUFH54/RwRBVESEgIPv74Y/z4449QqVQoLCzEH3/8gXHjxmHgwIFKl0dERJVEREQEAEClUmHy5Mk6n7UoKCjAkSNH0LRpU4WqK1+8R5mogtBqtRg8eDA2bdoEIQTMzMxQUFCAkJAQrFmzBqampkqXSERElUCnTp0APPxgn7+/PywsLKRlRQ8cGTduHOrVq6dUieWGQZmogrly5QpOnTqFwsJC+Pn5VYn/URERUfl75513sHjx4ioxX3JxGJSJKojY2NgqMRUPERGRsWBQJqogLCws4OrqipCQELz99tvw9fVVuiQiIqrE7t+/j9mzZ2PPnj1ITU1FYWGhzvKrV68qVFn54Yf5iCqImzdvYtOmTfjuu+8wd+5c+Pr64u2330ZISAhefPFFpcsjIqJK5t1330VsbCxCQ0Ph5uYmOwNGZccRZaIKKCEhARs3bsR3332HP//8E6+88gr27t2rdFlERFSJVKtWDTt27EC7du2ULkUxDMpEFVRBQQF27dqFyZMn48yZMygoKFC6JCIiqkS8vLywc+dONGjQQOlSFMMHjhBVMH/88QdGjhwJNzc3hISEoGHDhti+fbvSZRERUSUzffp0fPrpp8jOzla6FMVwRJmogvjvf/+L7777Djdv3kTXrl3x1ltvoW/fvjoTwRMREZUVPz8/XLlyBUII1KpVC+bm5jrLT548qVBl5Ycf5iOqIPbv349x48bhP//5D5ycnJQuh4iIKrm+ffsqXYLiOKJMRERERCSD9ygTVSDr1q1Du3bt4O7ujmvXrgEAFi1ahF9++UXhyoiIqDK6e/cuvv76a0ycOBFpaWkAHt5y8c8//yhcWflgUCaqIJYtW4aIiAj07NkTd+/elWa5qFatGhYtWqRscUREVOmcOXMG9evXx5w5czB//nzcvXsXALB161ZMnDhR2eLKCYMyUQWxZMkSrFy5EpMmTYKpqanU3qJFC5w9e1bByoiIqDKKiIjA4MGD8ffff8PS0lJq79GjBw4cOKBgZeWHQZmogkhISICfn59eu1qtxv379xWoiIiIKrNjx45hxIgReu01atRASkqKAhWVPwZlogrCy8sL8fHxeu27du2Cj49P+RdERESVmqWlJTIzM/XaL126hOrVqytQUfnj9HBEFcT48ePxwQcf4MGDBxBC4OjRo/juu+8wa9YsfP3110qXR0RElUyfPn0wbdo0/PDDDwAAlUqFpKQkTJgwAa+++qrC1ZUPTg9HVIGsXLkSn332Ga5fvw4AePHFFzFlyhQMHTpU4cqIiKiyyczMRM+ePXH+/Hncu3cP7u7uSElJgb+/P3bu3AkbGxulSzQ4BmWiCiInJwdCCFhbW+Pff//F1atX8ccff8DHxweBgYFKl0dERJXU3r17cfLkSRQWFqJZs2bo2rWr0iWVGwZlogoiICAA/fv3x3vvvYe7d+/ipZdegrm5Of79918sWLAA77//vtIlEhERVSq8R5mogjh58iQWLlwIANi8eTNcXFxw6tQpbNmyBZ9++imDMhERPbcvvviixH3DwsIMWIlx4IgyUQVhbW2NP//8EzVr1sSAAQPQsGFDTJkyBdevX4e3tzeys7OVLpGIiCo4Ly8vnde3b99GdnY2qlWrBuDhk/qsra3h7OyMq1evKlBh+eL0cEQVRN26dfHzzz/j+vXr+O233xAQEAAASE1NhZ2dncLVERFRZZCQkCB9zZgxA02bNsXFixeRlpaGtLQ0XLx4Ec2aNcP06dOVLrVccESZqILYvHkzQkJCUFBQgC5duiA6OhoAMGvWLBw4cAC7du1SuEIiIqpM6tSpg82bN+s97OrEiRN47bXXkJCQoFBl5Yf3KBNVEK+99hpefvllJCcno0mTJlJ7ly5d0K9fPwUrIyKiyig5ORlarVavvaCgALdu3VKgovLHEWUiIiIi0tO7d28kJSVh1apVaN68OVQqFY4fP45hw4bBw8MD27ZtU7pEg+M9ykRERESk55tvvkGNGjXQqlUrWFpaQq1Wo1WrVnBzc8PKlSuVLq9ccESZiIiIiIr1999/4+LFixBCoEGDBqhfv77SJZUbBmUiIiIiAgBERERg+vTpsLGxQURExBP7LliwoJyqUg4/zEdEREREAIBTp05JH+A7depUsf1UKlV5laQojigTEREREcngh/mIiIiIiGQwKBMRERERyWBQJiIiIiKSwaBMRFTJZWdnIzIyEvv371e6FCKiCoVBmYioksvOzsbUqVMZlImISolBmYiIjEZ2drbSJRARSRiUiYiM2J9//ok333wTLi4uUKvVqFmzJgYOHIjc3Fzcvn0bI0eOhI+PD1544QU4Ozujc+fO+P3336X3JyYmonr16gCAqVOnQqVSQaVSYfDgwVKfv//+GyEhIXB2doZarUaDBg3w5Zdf6tVy/vx5BAQEwNraGtWrV8cHH3yAHTt2QKVS6Y1Wf/PNN2jSpAksLS3h4OCAfv364eLFizp9Bg8ejBdeeAFnz55FQEAAbG1t0aVLF0yfPh1mZma4fv26Xg1DhgyBo6MjHjx48BxHlYioZBiUiYiM1OnTp9GyZUscPnwY06ZNw65duzBr1izk5uYiLy8PaWlpAIApU6Zgx44dWL16NWrXro2OHTtKwdXNzQ1RUVEAgKFDh+LQoUM4dOgQJk+eDAC4cOECWrZsiXPnzuHzzz/H9u3b0atXL4SFhWHq1KlSLcnJyejQoQMuXbqEZcuW4dtvv8W9e/cwatQovbpnzZqFoUOHomHDhvjpp5+wePFinDlzBv7+/vj77791+ubl5SE4OBidO3fGL7/8gqlTp2LEiBEwMzPDihUrdPqmpaVh06ZNGDp0KCwtLcvsOBMRFUsQEZFR6ty5s6hWrZpITU0tUf/8/Hyh1WpFly5dRL9+/aT227dvCwBiypQpeu8JDAwUL774osjIyNBpHzVqlLC0tBRpaWlCCCHGjx8vVCqVOH/+vN77AYh9+/YJIYRIT08XVlZWomfPnjr9kpKShFqtFiEhIVLboEGDBADxzTff6NU1aNAg4ezsLHJzc6W2OXPmCBMTE5GQkFCi40FE9Lw4okxEZISys7MRGxuLAQMGSLdOyFm+fDmaNWsGS0tLmJmZwdzcHHv27NG7zUHOgwcPsGfPHvTr1w/W1tbIz8+Xvnr27IkHDx7g8OHDAIDY2Fj4+vrCx8dHZx1vvvmmzutDhw4hJydH59YOAPDw8EDnzp2xZ88evTpeffVVvbYPP/wQqamp+PHHHwEAhYWFWLZsGXr16oVatWo9dd+IiMoCgzIRkRFKT09HQUEBXnzxxWL7LFiwAO+//z5at26NLVu24PDhwzh27Bi6d++OnJycp27jzp07yM/Px5IlS2Bubq7z1bNnTwDAv//+K/V1cXHRW8fjbXfu3AHw8JaPx7m7u0vLi1hbW8POzk6vr5+fH9q3by/dK719+3YkJibK3upBRGQoZkoXQERE+hwcHGBqaoobN24U22f9+vXo2LEjli1bptN+7969Em3D3t4epqamCA0NxQcffCDbx8vLCwDg6OiIW7du6S1PSUnRee3o6Ajg4T3Nj7t58yacnJx02lQqVbH1hYWF4fXXX8fJkyexdOlS1K9fH926dXvyThERlSGOKBMRGSErKyt06NABP/74ozSq+ziVSgW1Wq3TdubMGRw6dEinrajP46PM1tbW6NSpE06dOoXGjRujRYsWel9FwbdDhw44d+4cLly4oLOOTZs26bz29/eHlZUV1q9fr9N+48YN7N27F126dCnhEQD69euHmjVrYuzYsdi9ezdGjhz5xGBNRFTWGJSJiIzUggULoNVq0bp1a6xcuRL79u3Dpk2bEBISgnv37iEoKAjR0dGYMmUK9u7di2XLliEwMFAaBS5ia2sLT09P/PLLL4iOjsbx48eRmJgIAFi8eDGSkpLQvn17rFmzBvv378evv/6KhQsXonPnztI6wsPD4eDggB49emDt2rWIiorCwIED8eeffwIATEwe/jipVq0aJk+ejG3btmHgwIHYtWsX1q9fj06dOsHS0hJTpkwp8f6bmprigw8+wP79+2Ftba133zMRkcEp/WlCIiIq3oULF8Trr78uHB0dhYWFhahZs6YYPHiwePDggcjNzRXjxo0TNWrUEJaWlqJZs2bi559/FoMGDRKenp4669m9e7fw8/MTarVaABCDBg2SliUkJIghQ4aIGjVqCHNzc1G9enXRtm1b8dlnn+ms49y5c6Jr167C0tJSODg4iKFDh4q1a9cKAOL06dM6fb/++mvRuHFjYWFhITQajejTp4/ejBmDBg0SNjY2T9z/xMREAUC89957pT94RETPSSWEEApndSIiqqCGDx+O7777Dnfu3IGFhUWZr3/JkiUICwvDuXPn0LBhwzJfPxHRk/DDfEREVCLTpk2Du7s7ateujaysLGzfvh1ff/01PvnkkzIPyadOnUJCQgKmTZuGPn36MCQTkSIYlImIqETMzc0xb9483LhxA/n5+ahXrx4WLFiADz/8sMy31a9fP6SkpKB9+/ZYvnx5ma+fiKgkeOsFEREREZEMznpBRERERCSDQZmIiIiISAaDMhERERGRDAZlIiIiIiIZDMpERERERDIYlImIiIiIZDAoExERERHJYFAmIiIiIpLBoExEREREJOP/ATmISeQkGrcIAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 800x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# reference from https://towardsdatascience.com/multi-label-text-classification-with-scikit-learn-30714b7819c5\n",
    "# to visualize the number of comments per category\n",
    "df_stats = train_data[['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']].sum()\n",
    "df_stats.plot(x='category', y='number_of_comments', kind='bar', legend=False, grid=True, figsize=(8, 5))\n",
    "plt.title(\"Number of comments per category\")\n",
    "plt.ylabel('# of Occurrences', fontsize=12)\n",
    "plt.xlabel('category', fontsize=12)\n",
    "# this show that it is very imablannced dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "26b6745e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    143346\n",
       "1      6360\n",
       "3      4209\n",
       "2      3480\n",
       "4      1760\n",
       "5       385\n",
       "6        31\n",
       "dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# After analyzing the above setup, it hit us that they don't add up to the total number of comments\n",
    "# It means some comments are untagged/unlabeled\n",
    "labels_per_comment = train_data[['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']].sum(axis=1)\n",
    "labels_per_comment.value_counts()\n",
    "# This shows that most of the comments have 0 labels\n",
    "# This can be explained by the fact that most of the comments are not labeled means they are not toxic in anyway\n",
    "# can be considered as a negative class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "58fc3a40",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((135635, 8), (23936, 8))"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Splitting the training data into training and validation sets (85% - 15%)\n",
    "train_data, val_data = train_test_split(train_data, test_size=0.15, random_state=42)\n",
    "train_data.shape, val_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9c7de37",
   "metadata": {},
   "source": [
    "# Initializing TF-IDF Vectorizer\n",
    "\n",
    "After splitting the data, we initialize the TF-IDF Vectorizer. This vectorizer will transform the text comments into a format that's suitable for model input. TF-IDF helps in reflecting the importance of words in relation to the document and the entire corpus, which is vital for effective text classification.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0831ff42",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize the TF-IDF Vectorizer for the training data\n",
    "tfidf_vectorizer = TfidfVectorizer()\n",
    "X_train_tfidf = tfidf_vectorizer.fit_transform(train_data['comment_text'])\n",
    "X_val_tfidf = tfidf_vectorizer.transform(val_data['comment_text'])\n",
    "y_train = train_data[['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']].to_numpy()\n",
    "y_val = val_data[['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']].to_numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c83d5e6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.multiclass import OneVsRestClassifier\n",
    "from sklearn.svm import LinearSVC, SVC\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import f1_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "0827ccc8",
   "metadata": {},
   "outputs": [],
   "source": [
    "def eval(clf,X, Y, categories):\n",
    "    eval_metrics = {\"acc\": {}, \"f1\": {}}\n",
    "    predictions = clf.predict(X)\n",
    "    for i, category in enumerate(categories):\n",
    "        eval_metrics[\"acc\"][category] = accuracy_score(Y[:, i], predictions[:, i])\n",
    "        eval_metrics[\"f1\"][category] = f1_score(Y[:, i], predictions[:, i])\n",
    "\n",
    "    return pd.DataFrame(eval_metrics)\n",
    "\n",
    "def fit_eval(clf, categories, X_train, y_train, X_val, y_val):\n",
    "    clf.fit(X_train, y_train)\n",
    "    return eval(clf, X_val, y_val, categories)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ed6b2cb",
   "metadata": {},
   "source": [
    "# Model Fitting and Evaluation\n",
    "\n",
    "Now, we will fit our models to the training data and evaluate them using the validation set. We are employing a OneVsRestClassifier with different underlying algorithms: Naive Bayes, Logistic Regression, and LinearSVC. This approach is suitable for multi-label classification tasks like ours. After training, we will evaluate each model's performance using accuracy and F1-score, focusing on the latter due to the imbalanced nature of our dataset.\n",
    "\n",
    "Naive Bayes:\n",
    "- Here F1-scores are low because of the imbalanced dataset\n",
    "- This show how the imbalance in the data affects our naive bayes model's performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9cb0feec",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>acc</th>\n",
       "      <th>f1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>toxic</th>\n",
       "      <td>0.920914</td>\n",
       "      <td>0.280502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>severe_toxic</th>\n",
       "      <td>0.990600</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>obscene</th>\n",
       "      <td>0.952081</td>\n",
       "      <td>0.188252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>threat</th>\n",
       "      <td>0.997786</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>insult</th>\n",
       "      <td>0.952081</td>\n",
       "      <td>0.072757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>identity_hate</th>\n",
       "      <td>0.990516</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    acc        f1\n",
       "toxic          0.920914  0.280502\n",
       "severe_toxic   0.990600  0.000000\n",
       "obscene        0.952081  0.188252\n",
       "threat         0.997786  0.000000\n",
       "insult         0.952081  0.072757\n",
       "identity_hate  0.990516  0.000000"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define a pipeline combining a text feature extractor with multi lable classifier\n",
    "clf_naive =  OneVsRestClassifier(MultinomialNB(fit_prior=True, class_prior=None))\n",
    "categories = ['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']\n",
    "\n",
    "eval_metrics = fit_eval(clf_naive, categories, X_train_tfidf, y_train, X_val_tfidf, y_val)\n",
    "eval_metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69556f85",
   "metadata": {},
   "source": [
    "Logistic Regression:\n",
    "- Here F1-scores are also low because of the imbalanced dataset\n",
    "- This show how the imbalance in the data affects our logistic regression model's performance\n",
    "- But it is still much better than Naive Bayes Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7bd79a34",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/linear_model/_logistic.py:460: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>acc</th>\n",
       "      <th>f1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>toxic</th>\n",
       "      <td>0.957595</td>\n",
       "      <td>0.729261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>severe_toxic</th>\n",
       "      <td>0.991018</td>\n",
       "      <td>0.354354</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>obscene</th>\n",
       "      <td>0.976270</td>\n",
       "      <td>0.733583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>threat</th>\n",
       "      <td>0.997911</td>\n",
       "      <td>0.242424</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>insult</th>\n",
       "      <td>0.970296</td>\n",
       "      <td>0.630265</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>identity_hate</th>\n",
       "      <td>0.991352</td>\n",
       "      <td>0.241758</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    acc        f1\n",
       "toxic          0.957595  0.729261\n",
       "severe_toxic   0.991018  0.354354\n",
       "obscene        0.976270  0.733583\n",
       "threat         0.997911  0.242424\n",
       "insult         0.970296  0.630265\n",
       "identity_hate  0.991352  0.241758"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define a pipeline combining a text feature extractor with multi lable classifier\n",
    "clf_log =  OneVsRestClassifier(LogisticRegression())\n",
    "categories = ['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']\n",
    "\n",
    "eval_metrics = fit_eval(clf_log, categories, X_train_tfidf, y_train, X_val_tfidf, y_val)\n",
    "eval_metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71731990",
   "metadata": {},
   "source": [
    "SVC:\n",
    "- Here F1-scores are also low because of the imbalanced dataset\n",
    "- But SVC is still much better than Naive Bayes Classifier and logistic model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "56136bb4",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/svm/_classes.py:32: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
      "  warnings.warn(\n",
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/svm/_classes.py:32: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
      "  warnings.warn(\n",
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/svm/_classes.py:32: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
      "  warnings.warn(\n",
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/svm/_classes.py:32: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
      "  warnings.warn(\n",
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/svm/_classes.py:32: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
      "  warnings.warn(\n",
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/svm/_classes.py:32: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>acc</th>\n",
       "      <th>f1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>toxic</th>\n",
       "      <td>0.960812</td>\n",
       "      <td>0.768852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>severe_toxic</th>\n",
       "      <td>0.990642</td>\n",
       "      <td>0.370787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>obscene</th>\n",
       "      <td>0.979403</td>\n",
       "      <td>0.785745</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>threat</th>\n",
       "      <td>0.997953</td>\n",
       "      <td>0.363636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>insult</th>\n",
       "      <td>0.972468</td>\n",
       "      <td>0.681796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>identity_hate</th>\n",
       "      <td>0.991895</td>\n",
       "      <td>0.370130</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    acc        f1\n",
       "toxic          0.960812  0.768852\n",
       "severe_toxic   0.990642  0.370787\n",
       "obscene        0.979403  0.785745\n",
       "threat         0.997953  0.363636\n",
       "insult         0.972468  0.681796\n",
       "identity_hate  0.991895  0.370130"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define a pipeline combining a text feature extractor with multi lable classifier\n",
    "clf_svc =  OneVsRestClassifier(LinearSVC())\n",
    "categories = ['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']\n",
    "\n",
    "eval_metrics = fit_eval(clf_svc, categories, X_train_tfidf, y_train, X_val_tfidf, y_val)\n",
    "eval_metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2653e544",
   "metadata": {},
   "source": [
    "- From the above methods, Linear SVC performed the best\n",
    "- Let's run it on the test data\n",
    "- In the below cell we use a sample hate sentence for see the output of the model\n",
    "- Used all the bad words we know for the sake of testing only XD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "9c176829",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 0, 0, 0, 1, 1]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#text = \"FUCK YOU BLACK NIGGA I WILL STAB YOU TO DEATH DIE CUNT\"\n",
    "text = \"You are all niggers that should be burned at the stake and I hope you do.\"\n",
    "\n",
    "demo = tfidf_vectorizer.transform([preprocess_text(text)])\n",
    "pred = clf_svc.predict(demo)\n",
    "pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "72c8ce2d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>acc</th>\n",
       "      <th>f1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>toxic</th>\n",
       "      <td>0.927397</td>\n",
       "      <td>0.671337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>severe_toxic</th>\n",
       "      <td>0.992732</td>\n",
       "      <td>0.351464</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>obscene</th>\n",
       "      <td>0.963472</td>\n",
       "      <td>0.681998</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>threat</th>\n",
       "      <td>0.996639</td>\n",
       "      <td>0.380403</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>insult</th>\n",
       "      <td>0.961831</td>\n",
       "      <td>0.611765</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>identity_hate</th>\n",
       "      <td>0.990372</td>\n",
       "      <td>0.406551</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    acc        f1\n",
       "toxic          0.927397  0.671337\n",
       "severe_toxic   0.992732  0.351464\n",
       "obscene        0.963472  0.681998\n",
       "threat         0.996639  0.380403\n",
       "insult         0.961831  0.611765\n",
       "identity_hate  0.990372  0.406551"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eval(clf_svc, X_test_tfidf, y_test, categories)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc692d58",
   "metadata": {},
   "source": [
    "# Prediction and Evaluation on Test Data\n",
    "\n",
    "After training and evaluating our models on the validation set, we will use them to predict labels on the test set. We use the **SVC Classifier** because it gave us the best result out of all. This step is crucial for understanding how our models perform on unseen data. We will use classification reports and confusion matrices to assess the performance in a detailed manner, providing insights into each category's prediction accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "2b59e05e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "               precision    recall  f1-score   support\n",
      "\n",
      "        toxic       0.59      0.78      0.67      6090\n",
      " severe_toxic       0.36      0.34      0.35       367\n",
      "      obscene       0.69      0.68      0.68      3691\n",
      "       threat       0.49      0.31      0.38       211\n",
      "       insult       0.67      0.56      0.61      3427\n",
      "identity_hate       0.65      0.30      0.41       712\n",
      "\n",
      "    micro avg       0.62      0.66      0.64     14498\n",
      "    macro avg       0.57      0.50      0.52     14498\n",
      " weighted avg       0.63      0.66      0.63     14498\n",
      "  samples avg       0.07      0.06      0.06     14498\n",
      "\n",
      "[[[54589  3299]\n",
      "  [ 1346  4744]]\n",
      "\n",
      " [[63387   224]\n",
      "  [  241   126]]\n",
      "\n",
      " [[59135  1152]\n",
      "  [ 1185  2506]]\n",
      "\n",
      " [[63697    70]\n",
      "  [  145    66]]\n",
      "\n",
      " [[59612   939]\n",
      "  [ 1503  1924]]\n",
      "\n",
      " [[63151   115]\n",
      "  [  501   211]]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/metrics/_classification.py:1471: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in samples with no predicted labels. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n",
      "/home/junaid/anaconda3/envs/pytorch/lib/python3.11/site-packages/sklearn/metrics/_classification.py:1471: UndefinedMetricWarning: Recall and F-score are ill-defined and being set to 0.0 in samples with no true labels. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n"
     ]
    }
   ],
   "source": [
    "preds = clf_svc.predict(X_test_tfidf)\n",
    "print(classification_report(y_test, preds, target_names=categories))\n",
    "print(multilabel_confusion_matrix(y_test, preds))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.11.5 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  },
  "vscode": {
   "interpreter": {
    "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}