examples/02_model_hybrid/lightfm_deep_dive.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<i>Copyright (c) Microsoft Corporation. All rights reserved.</i>\n",
    "\n",
    "<i>Licensed under the MIT License.</i>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LightFM -  hybrid matrix factorisation on MovieLens (Python, CPU)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook explains the concept of a hybrid matrix factorisation based model for recommendation, it also outlines the steps to construct a pure matrix factorisation and a hybrid models using the [LightFM](https://github.com/lyst/lightfm) package. It also demonstrates how to extract both user and item affinity from a fitted hybrid model.\n",
    "\n",
    "## 1. Hybrid matrix factorisation model\n",
    "\n",
    "### 1.1 Background\n",
    "\n",
    "In general, most recommendation models can be divided into two categories:\n",
    "- Content based model,\n",
    "- Collaborative filtering model.\n",
    "\n",
    "The content-based model recommends based on similarity of the items and/or users using their description/metadata/profile. On the other hand, collaborative filtering model (discussion is limited to matrix factorisation approach in this notebook) computes the latent factors of the users and items. It works based on the assumption that if a group of people expressed similar opinions on an item, these peole would tend to have similar opinions on other items. For further background and detailed explanation between these two approaches, the reader can refer to machine learning literatures [3, 4].\n",
    "\n",
    "The choice between the two models is largely based on the data availability. For example, the collaborative filtering model is usually adopted and effective when sufficient ratings/feedbacks have been recorded for a group of users and items.\n",
    "\n",
    "However, if there is a lack of ratings, content based model can be used provided that the metadata of the users and items are available. This is also the common approach to address the cold-start issues, where there are insufficient historical collaborative interactions available to model new users and/or items.\n",
    "\n",
    "<!-- In addition, most collaborative filtering models only consume explicit ratings e.g. movie \n",
    "\n",
    "**NOTE** add stuff about implicit and explicit ratings -->\n",
    "\n",
    "### 1.2 Hybrid matrix factorisation algorithm\n",
    "\n",
    "In view of the above problems, there have been a number of proposals to address the cold-start issues by combining both content-based and collaborative filtering approaches. The hybrid matrix factorisation model is among one of the solutions proposed [1].  \n",
    "\n",
    "In general, most hybrid approaches proposed different ways of assessing and/or combining the feature data in conjunction with the collaborative information.\n",
    "\n",
    "### 1.3 LightFM package \n",
    "\n",
    "LightFM is a Python implementation of a hybrid recommendation algorithms for both implicit and explicit feedbacks [1].\n",
    "\n",
    "It is a hybrid content-collaborative model which represents users and items as linear combinations of their content features’ latent factors. The model learns **embeddings or latent representations of the users and items in such a way that it encodes user preferences over items**. These representations produce scores for every item for a given user; items scored highly are more likely to be interesting to the user.\n",
    "\n",
    "The user and item embeddings are estimated for every feature, and these features are then added together to be the final representations for users and items. \n",
    "\n",
    "For example, for user i, the model retrieves the i-th row of the feature matrix to find the features with non-zero weights. The embeddings for these features will then be added together to become the user representation e.g. if user 10 has weight 1 in the 5th column of the user feature matrix, and weight 3 in the 20th column, the user 10’s representation is the sum of embedding for the 5th and the 20th features multiplying their corresponding weights. The representation for each items is computed in the same approach. \n",
    "\n",
    "#### 1.3.1 Modelling approach\n",
    "\n",
    "Let $U$ be the set of users and $I$ be the set of items, and each user can be described by a set of user features $f_{u} \\subset F^{U}$ whilst each items can be described by item features $f_{i} \\subset F^{I}$. Both $F^{U}$ and $F^{I}$ are all the features which fully describe all users and items. \n",
    "\n",
    "The LightFM model operates based binary feedbacks, the ratings will be normalised into two groups. The user-item interaction pairs $(u,i) \\in U\\times I$ are the union of positive (favourable reviews) $S^+$ and negative interactions (negative reviews) $S^-$ for explicit ratings. For implicit feedbacks, these can be the observed and not observed interactions respectively.\n",
    "\n",
    "For each user and item feature, their embeddings are $e_{f}^{U}$ and $e_{f}^{I}$ respectively. Furthermore, each feature is also has a scalar bias term ($b_U^f$ for user and $b_I^f$ for item features). The embedding (latent representation) of user $u$ and item $i$ are the sum of its respective features’ latent vectors:\n",
    "\n",
    "$$ \n",
    "q_{u} = \\sum_{j \\in f_{u}} e_{j}^{U}\n",
    "$$\n",
    "\n",
    "$$\n",
    "p_{i} = \\sum_{j \\in f_{i}} e_{j}^{I}\n",
    "$$\n",
    "\n",
    "Similarly the biases for user $u$ and item $i$ are the sum of its respective bias vectors. These variables capture the variation in behaviour across users and items:\n",
    "\n",
    "$$\n",
    "b_{u} = \\sum_{j \\in f_{u}} b_{j}^{U}\n",
    "$$\n",
    "\n",
    "$$\n",
    "b_{i} = \\sum_{j \\in f_{i}} b_{j}^{I}\n",
    "$$\n",
    "\n",
    "In LightFM, the representation for each user/item is a linear weighted sum of its feature vectors.\n",
    "\n",
    "The prediction for user $u$ and item $i$ can be modelled as sigmoid of the dot product of user and item vectors, adjusted by its feature biases as follows:\n",
    "\n",
    "$$\n",
    "\\hat{r}_{ui} = \\sigma (q_{u} \\cdot p_{i} + b_{u} + b_{i})\n",
    "$$\n",
    "\n",
    "As the LightFM is constructed to predict binary outcomes e.g. $S^+$ and $S^-$, the function $\\sigma()$ is based on the [sigmoid function](https://mathworld.wolfram.com/SigmoidFunction.html). \n",
    "\n",
    "The LightFM algorithm estimates interaction latent vectors and bias for features. For model fitting, the cost function of the model consists of maximising the likelihood of data conditional on the parameters described above using stochastic gradient descent. The likelihood can be expressed as follows:\n",
    "\n",
    "$$\n",
    "L = \\prod_{(u,i) \\in S+}\\hat{r}_{ui} \\times \\prod_{(u,i) \\in S-}1 - \\hat{r}_{ui}\n",
    "$$\n",
    "\n",
    "Note that if the feature latent vectors are not available, the algorithm will behaves like a [logistic matrix factorisation model](http://stanford.edu/~rezab/nips2014workshop/submits/logmat.pdf)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Movie recommender with LightFM using only explicit feedbacks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "System version: 3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54) \n",
      "[GCC 7.3.0]\n",
      "LightFM version: 1.15\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "sys.path.append(\"../../\")\n",
    "import os\n",
    "\n",
    "import itertools\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "import lightfm\n",
    "from lightfm import LightFM\n",
    "from lightfm.data import Dataset\n",
    "from lightfm import cross_validation\n",
    "\n",
    "# Import LightFM's evaluation metrics\n",
    "from lightfm.evaluation import precision_at_k as lightfm_prec_at_k\n",
    "from lightfm.evaluation import recall_at_k as lightfm_recall_at_k\n",
    "\n",
    "# Import repo's evaluation metrics\n",
    "from reco_utils.evaluation.python_evaluation import (\n",
    "    precision_at_k, recall_at_k)\n",
    "\n",
    "from reco_utils.common.timer import Timer\n",
    "from reco_utils.dataset import movielens\n",
    "from reco_utils.recommender.lightfm.lightfm_utils import (\n",
    "    track_model_metrics, prepare_test_df, prepare_all_predictions,\n",
    "    compare_metric, similar_users, similar_items)\n",
    "\n",
    "print(\"System version: {}\".format(sys.version))\n",
    "print(\"LightFM version: {}\".format(lightfm.__version__))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Defining variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "# Select MovieLens data size\n",
    "MOVIELENS_DATA_SIZE = '100k'\n",
    "\n",
    "# default number of recommendations\n",
    "K = 10\n",
    "# percentage of data used for testing\n",
    "TEST_PERCENTAGE = 0.25\n",
    "# model learning rate\n",
    "LEARNING_RATE = 0.25\n",
    "# no of latent factors\n",
    "NO_COMPONENTS = 20\n",
    "# no of epochs to fit model\n",
    "NO_EPOCHS = 20\n",
    "# no of threads to fit model\n",
    "NO_THREADS = 32\n",
    "# regularisation for both user and item features\n",
    "ITEM_ALPHA=1e-6\n",
    "USER_ALPHA=1e-6\n",
    "\n",
    "# seed for pseudonumber generations\n",
    "SEEDNO = 42"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Retrieve data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4.81k/4.81k [00:06<00:00, 794KB/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>itemID</th>\n",
       "      <th>rating</th>\n",
       "      <th>genre</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>86814</th>\n",
       "      <td>854</td>\n",
       "      <td>544</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Crime|Drama|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7007</th>\n",
       "      <td>625</td>\n",
       "      <td>4</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Action|Comedy|Drama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40359</th>\n",
       "      <td>526</td>\n",
       "      <td>271</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Action|Adventure|Sci-Fi|War</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21909</th>\n",
       "      <td>910</td>\n",
       "      <td>405</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Action|Adventure|Mystery</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33526</th>\n",
       "      <td>58</td>\n",
       "      <td>56</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Crime|Drama</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       userID  itemID  rating                        genre\n",
       "86814     854     544     3.0          Crime|Drama|Romance\n",
       "7007      625       4     4.0          Action|Comedy|Drama\n",
       "40359     526     271     3.0  Action|Adventure|Sci-Fi|War\n",
       "21909     910     405     4.0     Action|Adventure|Mystery\n",
       "33526      58      56     5.0                  Crime|Drama"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = movielens.load_pandas_df(\n",
    "    size=MOVIELENS_DATA_SIZE,\n",
    "    genres_col='genre',\n",
    "    header=[\"userID\", \"itemID\", \"rating\"]\n",
    ")\n",
    "# quick look at the data\n",
    "data.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Prepare data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before fitting the LightFM model, we need to create an instance of `Dataset` which holds the interaction matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = Dataset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `fit` method creates the user/item id mappings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num users: 943, num_topics: 1682.\n"
     ]
    }
   ],
   "source": [
    "dataset.fit(users=data['userID'], \n",
    "            items=data['itemID'])\n",
    "\n",
    "# quick check to determine the number of unique users and items in the data\n",
    "num_users, num_topics = dataset.interactions_shape()\n",
    "print(f'Num users: {num_users}, num_topics: {num_topics}.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next is to build the interaction matrix. The `build_interactions` method returns 2 COO sparse matrices, namely the `interactions` and `weights` matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "(interactions, weights) = dataset.build_interactions(data.iloc[:, 0:3].values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LightLM works slightly differently compared to other packages as it expects the train and test sets to have same dimension. Therefore the conventional train test split will not work.\n",
    "\n",
    "The package has included the `cross_validation.random_train_test_split` method to split the interaction data and splits it into two disjoint training and test sets. \n",
    "\n",
    "However, note that **it does not validate the interactions in the test set to guarantee all items and users have historical interactions in the training set**. Therefore this may result into a partial cold-start problem in the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_interactions, test_interactions = cross_validation.random_train_test_split(\n",
    "    interactions, test_percentage=TEST_PERCENTAGE,\n",
    "    random_state=np.random.RandomState(SEEDNO))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double check the size of both the train and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of train interactions: (943, 1682)\n",
      "Shape of test interactions: (943, 1682)\n"
     ]
    }
   ],
   "source": [
    "print(f\"Shape of train interactions: {train_interactions.shape}\")\n",
    "print(f\"Shape of test interactions: {test_interactions.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Fit the LightFM model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, the LightFM model will be using the weighted Approximate-Rank Pairwise (WARP) as the loss. Further explanation on the topic can be found [here](https://making.lyst.com/lightfm/docs/examples/warp_loss.html#learning-to-rank-using-the-warp-loss).\n",
    "\n",
    "\n",
    "In general, it maximises the rank of positive examples by repeatedly sampling negative examples until a rank violation has been located. This approach is recommended when only positive interactions are present."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model1 = LightFM(loss='warp', no_components=NO_COMPONENTS, \n",
    "                 learning_rate=LEARNING_RATE,                 \n",
    "                 random_state=np.random.RandomState(SEEDNO))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The LightFM model can be fitted with the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "model1.fit(interactions=train_interactions,\n",
    "          epochs=NO_EPOCHS);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 Prepare model evaluation data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we can evaluate the fitted model and to get the data into a format which is compatible with the existing evaluation methods within this repo, the data needs to be massaged slightly.\n",
    "\n",
    "First the train/test indices need to be extracted from the `lightfm.cross_validation` method as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "uids, iids, interaction_data = cross_validation._shuffle(\n",
    "    interactions.row, interactions.col, interactions.data, \n",
    "    random_state=np.random.RandomState(SEEDNO))\n",
    "\n",
    "cutoff = int((1.0 - TEST_PERCENTAGE) * len(uids))\n",
    "test_idx = slice(cutoff, None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then the the mapping between internal and external representation of the user and item are extracted as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "uid_map, ufeature_map, iid_map, ifeature_map = dataset.mapping()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the train/test indices and mapping are ready, the test dataframe can be constructed as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Took 2.0 seconds for prepare and predict test data.\n"
     ]
    }
   ],
   "source": [
    "with Timer() as test_time:\n",
    "    test_df = prepare_test_df(test_idx, uids, iids, uid_map, iid_map, weights)\n",
    "print(f\"Took {test_time.interval:.1f} seconds for prepare and predict test data.\")  \n",
    "time_reco1 = test_time.interval"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And samples of the test dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>itemID</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>20675</th>\n",
       "      <td>749</td>\n",
       "      <td>731</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22917</th>\n",
       "      <td>224</td>\n",
       "      <td>470</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11688</th>\n",
       "      <td>682</td>\n",
       "      <td>280</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6005</th>\n",
       "      <td>838</td>\n",
       "      <td>271</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4884</th>\n",
       "      <td>92</td>\n",
       "      <td>452</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       userID  itemID  rating\n",
       "20675     749     731     3.0\n",
       "22917     224     470     4.0\n",
       "11688     682     280     3.0\n",
       "6005      838     271     4.0\n",
       "4884       92     452     2.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition, the predictions of all unseen user-item pairs (e.g. removing those seen in the training data) can be prepared as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Took 763.3 seconds for prepare and predict all data.\n"
     ]
    }
   ],
   "source": [
    "with Timer() as test_time:\n",
    "    all_predictions = prepare_all_predictions(data, uid_map, iid_map, \n",
    "                                              interactions=train_interactions,\n",
    "                                              model=model1, \n",
    "                                              num_threads=NO_THREADS)\n",
    "print(f\"Took {test_time.interval:.1f} seconds for prepare and predict all data.\")\n",
    "time_reco2 = test_time.interval"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Samples of the `all_predictions` dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>itemID</th>\n",
       "      <th>prediction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1143776</th>\n",
       "      <td>759</td>\n",
       "      <td>678</td>\n",
       "      <td>115.532867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>879478</th>\n",
       "      <td>128</td>\n",
       "      <td>1158</td>\n",
       "      <td>-24.874775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887889</th>\n",
       "      <td>37</td>\n",
       "      <td>61</td>\n",
       "      <td>12.243621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17054</th>\n",
       "      <td>35</td>\n",
       "      <td>1188</td>\n",
       "      <td>-30.835514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1075633</th>\n",
       "      <td>337</td>\n",
       "      <td>1062</td>\n",
       "      <td>-45.256855</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         userID  itemID  prediction\n",
       "1143776     759     678  115.532867\n",
       "879478      128    1158  -24.874775\n",
       "887889       37      61   12.243621\n",
       "17054        35    1188  -30.835514\n",
       "1075633     337    1062  -45.256855"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_predictions.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the **raw prediction values from the LightFM model are for ranking purposes only**, they should not be used directly. The magnitude and sign of these values do not have any specific interpretation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6 Model evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the evaluation data are ready, they can be passed into to the repo's evaluation methods as follows. The performance of the model will be tracked using both Precision@K and Recall@K.\n",
    "\n",
    "In addition, the results have also being compared with those computed from LightFM's own evaluation methods to ensure accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------ Using Repo's evaluation methods ------\n",
      "Precision@K:\t0.131601\n",
      "Recall@K:\t0.038056\n",
      "\n",
      "------ Using LightFM evaluation methods ------\n",
      "Precision@K:\t0.131601\n",
      "Recall@K:\t0.038056\n"
     ]
    }
   ],
   "source": [
    "with Timer() as test_time:\n",
    "    eval_precision = precision_at_k(rating_true=test_df, \n",
    "                                rating_pred=all_predictions, k=K)\n",
    "    eval_recall = recall_at_k(test_df, all_predictions, k=K)\n",
    "time_reco3 = test_time.interval\n",
    "\n",
    "with Timer() as test_time:\n",
    "    eval_precision_lfm = lightfm_prec_at_k(model1, test_interactions, \n",
    "                                           train_interactions, k=K).mean()\n",
    "    eval_recall_lfm = lightfm_recall_at_k(model1, test_interactions, \n",
    "                                          train_interactions, k=K).mean()\n",
    "time_lfm = test_time.interval\n",
    "    \n",
    "print(\n",
    "    \"------ Using Repo's evaluation methods ------\",\n",
    "    f\"Precision@K:\\t{eval_precision:.6f}\",\n",
    "    f\"Recall@K:\\t{eval_recall:.6f}\",\n",
    "    \"\\n------ Using LightFM evaluation methods ------\",\n",
    "    f\"Precision@K:\\t{eval_precision_lfm:.6f}\",\n",
    "    f\"Recall@K:\\t{eval_recall_lfm:.6f}\", \n",
    "    sep='\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Movie recommender with LightFM using explicit feedbacks and additional item and user features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the LightFM was designed to incorporates both user and item metadata, the model can be extended to include additional features such as movie genres and user occupations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Extract and prepare movie genres"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, the movie's genres will be used as the item metadata. As the genres have already been loaded during the initial data import, it can be processed directly as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# split the genre based on the separator\n",
    "movie_genre = [x.split('|') for x in data['genre']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Action',\n",
       " 'Adventure',\n",
       " 'Animation',\n",
       " \"Children's\",\n",
       " 'Comedy',\n",
       " 'Crime',\n",
       " 'Documentary',\n",
       " 'Drama',\n",
       " 'Fantasy',\n",
       " 'Film-Noir',\n",
       " 'Horror',\n",
       " 'Musical',\n",
       " 'Mystery',\n",
       " 'Romance',\n",
       " 'Sci-Fi',\n",
       " 'Thriller',\n",
       " 'War',\n",
       " 'Western',\n",
       " 'unknown']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retrieve the all the unique genres in the data\n",
    "all_movie_genre = sorted(list(set(itertools.chain.from_iterable(movie_genre))))\n",
    "# quick look at the all the genres within the data\n",
    "all_movie_genre"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Retrieve and prepare movie genres"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Further user features can be included as part of the model fitting process. In this notebook, **only the occupation of each user will be included** but the feature list can be extended easily.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1 Retrieve and merge data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The user features can be retrieved directly from the grouplens website and merged with the existing data as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>itemID</th>\n",
       "      <th>rating</th>\n",
       "      <th>genre</th>\n",
       "      <th>occupation</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6917</th>\n",
       "      <td>624</td>\n",
       "      <td>678</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Drama|Thriller</td>\n",
       "      <td>student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24294</th>\n",
       "      <td>446</td>\n",
       "      <td>748</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Action|Romance|Thriller</td>\n",
       "      <td>educator</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22108</th>\n",
       "      <td>104</td>\n",
       "      <td>346</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Crime|Drama</td>\n",
       "      <td>student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93983</th>\n",
       "      <td>763</td>\n",
       "      <td>588</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Animation|Children's|Musical</td>\n",
       "      <td>scientist</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58278</th>\n",
       "      <td>290</td>\n",
       "      <td>825</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Action|Sci-Fi|Thriller</td>\n",
       "      <td>engineer</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       userID  itemID  rating                         genre occupation\n",
       "6917      624     678     3.0                Drama|Thriller    student\n",
       "24294     446     748     2.0       Action|Romance|Thriller   educator\n",
       "22108     104     346     3.0                   Crime|Drama    student\n",
       "93983     763     588     4.0  Animation|Children's|Musical  scientist\n",
       "58278     290     825     3.0        Action|Sci-Fi|Thriller   engineer"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_feature_URL = 'http://files.grouplens.org/datasets/movielens/ml-100k/u.user'\n",
    "user_data = pd.read_table(user_feature_URL, \n",
    "              sep='|', header=None)\n",
    "user_data.columns = ['userID','age','gender','occupation','zipcode']\n",
    "\n",
    "# merging user feature with existing data\n",
    "new_data = data.merge(user_data[['userID','occupation']], left_on='userID', right_on='userID')\n",
    "# quick look at the merged data\n",
    "new_data.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.2 Extract and prepare user occupations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# retrieve all the unique occupations in the data\n",
    "all_occupations = sorted(list(set(new_data['occupation'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Prepare data and features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to the previous model, the data is required to be converted into a `Dataset` instance and then create a user/item id mapping with the `fit` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset2 = Dataset()\n",
    "dataset2.fit(data['userID'], \n",
    "            data['itemID'], \n",
    "            item_features=all_movie_genre,\n",
    "            user_features=all_occupations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The movie genres are then converted into a item feature matrix using the `build_item_features` method as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "item_features = dataset2.build_item_features(\n",
    "    (x, y) for x,y in zip(data.itemID, movie_genre))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The user occupations are then converted into an user feature matrix using the `build_user_features` method as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_features = dataset2.build_user_features(\n",
    "    (x, [y]) for x,y in zip(new_data.userID, new_data['occupation']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the item and user features matrices have been completed, the next steps are similar as before, which is to build the interaction matrix and split the interactions into train and test sets as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "(interactions2, weights2) = dataset2.build_interactions(data.iloc[:, 0:3].values)\n",
    "\n",
    "train_interactions2, test_interactions2 = cross_validation.random_train_test_split(\n",
    "    interactions2, test_percentage=TEST_PERCENTAGE,\n",
    "    random_state=np.random.RandomState(SEEDNO))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Fit the LightFM model with additional user and item features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The parameters of the second model will be similar to the first model to facilitates comparison.\n",
    "\n",
    "The model performance at each epoch is also tracked by the same metrics as before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "model2 = LightFM(loss='warp', no_components=NO_COMPONENTS, \n",
    "                 learning_rate=LEARNING_RATE, \n",
    "                 item_alpha=ITEM_ALPHA,\n",
    "                 user_alpha=USER_ALPHA,\n",
    "                 random_state=np.random.RandomState(SEEDNO))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The LightFM model can then be fitted:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "model2.fit(interactions=train_interactions2,\n",
    "           user_features=user_features,\n",
    "           item_features=item_features,\n",
    "           epochs=NO_EPOCHS);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Prepare model evaluation data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to the previous model, the evaluation data needs to be prepared in order to get them into a format consumable with this repo's evaluation methods.\n",
    "\n",
    "Firstly the train/test indices and id mappings are extracted using the new interations matrix as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "uids, iids, interaction_data = cross_validation._shuffle(\n",
    "    interactions2.row, interactions2.col, interactions2.data, \n",
    "    random_state=np.random.RandomState(SEEDNO))\n",
    "\n",
    "uid_map, ufeature_map, iid_map, ifeature_map = dataset2.mapping()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The test dataframe is then constructed as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Took 2.0 seconds for prepare and predict test data.\n"
     ]
    }
   ],
   "source": [
    "with Timer() as test_time:\n",
    "    test_df2 = prepare_test_df(test_idx, uids, iids, uid_map, iid_map, weights2)\n",
    "print(f\"Took {test_time.interval:.1f} seconds for prepare and predict test data.\")  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The predictions of all unseen user-item pairs can be prepared as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Took 274.7 seconds for prepare and predict all data.\n"
     ]
    }
   ],
   "source": [
    "with Timer() as test_time:\n",
    "    all_predictions2 = prepare_all_predictions(data, uid_map, iid_map, \n",
    "                                              interactions=train_interactions2,\n",
    "                                               user_features=user_features,\n",
    "                                               item_features=item_features,\n",
    "                                               model=model2,\n",
    "                                               num_threads=NO_THREADS)\n",
    "\n",
    "print(f\"Took {test_time.interval:.1f} seconds for prepare and predict all data.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5 Model evaluation and comparison"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The predictive performance of the new model can be computed and compared with the previous model (which used only the explicit rating) as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------ Using only explicit ratings ------\n",
      "Precision@K:\t0.131601\n",
      "Recall@K:\t0.038056\n",
      "\n",
      "------ Using both implicit and explicit ratings ------\n",
      "Precision@K:\t0.147826\n",
      "Recall@K:\t0.053450\n"
     ]
    }
   ],
   "source": [
    "eval_precision2 = precision_at_k(rating_true=test_df2, \n",
    "                                rating_pred=all_predictions2, k=K)\n",
    "eval_recall2 = recall_at_k(test_df2, all_predictions2, k=K)\n",
    "\n",
    "print(\n",
    "    \"------ Using only explicit ratings ------\",\n",
    "    f\"Precision@K:\\t{eval_precision:.6f}\",\n",
    "    f\"Recall@K:\\t{eval_recall:.6f}\",\n",
    "    \"\\n------ Using both implicit and explicit ratings ------\",\n",
    "    f\"Precision@K:\\t{eval_precision2:.6f}\",\n",
    "    f\"Recall@K:\\t{eval_recall2:.6f}\",\n",
    "    sep='\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The new model which used both implicit and explicit data performed consistently better than the previous model which used only the explicit data, thus highlighting the benefits of including such additional features to the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.6 Evaluation metrics comparison"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the evaluation approaches here are solely for demonstration purposes only.\n",
    "\n",
    "If the reader were using the LightFM package and/or its models, the LightFM's built-in evaluation methods are much more efficient and are the recommended approach for production usage as they are designed and optimised to work with the package.\n",
    "\n",
    "As a comparison, the times recorded to compute Precision@K and Recall@K for model1 are shown as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------ Using Repo's evaluation methods ------\n",
      "Time [sec]:\t770.3\n",
      "\n",
      "------ Using LightFM evaluation methods ------\n",
      "Time [sec]:\t0.3\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    \"------ Using Repo's evaluation methods ------\",\n",
    "    f\"Time [sec]:\\t{(time_reco1+time_reco2+time_reco3):.1f}\",\n",
    "    \"\\n------ Using LightFM evaluation methods ------\",\n",
    "    f\"Time [sec]:\\t{time_lfm:.1f}\",\n",
    "    sep='\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Evaluate model fitting process"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to the inclusion of both implicit and explicit data, the model fitting process can also be monitored in order to determine whether the model is being trained properly. \n",
    "\n",
    "This notebook also includes a `track_model_metrics` method which plots the model's metrics e.g. Precision@K and Recall@K as model fitting progresses.\n",
    "\n",
    "For the first model (using only explicit data), the model fitting progress is shown as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeIAAADQCAYAAADbLGKxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5gcVZ3/8fcnmU1AQBIg8GAgJki8BNEgA4oowg9kA+uPywoSEMUVzSIgsuqu+FMRcfcRvKGuoERluQuICzvrRkEBWQ2oGe4kLBguQoAlCQlIABMm8/39UdWh0pnurnR3Td8+r+eZZ7rqnKo63UnNt8+lzlFEYGZmZq0xptUFMDMz62UOxGZmZi3kQGxmZtZCDsRmZmYt5EBsZmbWQg7EZmZmLeRA3IYkzZR0cJX0fknfKeja+0p6VtIdku6T9MUmnfcESR+skn6IpNOacS2zrDa6n/5H0tcLuMaFko5IX/9aUn+zr2HF6mt1AWxEM4F+YF55gqS+iBgEBgu8/m8i4j2SNgPulPSziLitrAxDG3PCiPh+jfQBYKC+4ppV1S7306bAHZKuiYj5BV7POoxrxAWQNDX99vtDSfdKukzSAZLmS/qjpD3TfJtJukDSgvQb86GSxgFnAkdJulPSUZLOkDRX0vXAxem37J+l59hc0r9JukfS3ZLe26z3ERHPA7cBr5H0IUk/kfSfwPXptf8xLfvdkr6Uef8fTPfdJemSdN8Zkj6dvj5F0qI0zxXpvg9J+m76+tWSbkjTb5A0Jd1/oaTvSLpF0kOlWoB1ty66n14E7gQmVypvun+spK9nyvDxdP/pad570/KrWWWz1nKNuDg7A0cCc4AFwDHAO4BDgP8HHAZ8DrgxIj4saQLwB+BXwOlAf0ScDEkQA3YH3hERL0raN3OdLwDPRsSuad6J5QWRdA6w3whlvCIizqr0BiRtDbwN+DKwB7AX8KaIWCHpQGA6sCcgYEDSPsDT6fvaOyKWS9pqhFOfBkyLiNXp+y73XeDiiLhI0oeB76SfF8D2JJ/j60lq0FdXKr91lW64nyaS3DP/ne7aoLySfgV8EJgG7BYRQ5l76LsRcWZ6rkuA9wD/Wel61jkciIvzcETcAyBpIXBDRISke4CpaZ4DgUNKNUVgE2BKhfMNpN+oyx0AzC5tRMTK8gwR8Q8bWfZ3SroDGAbOioiFkvYAfhkRKzJlPxC4I93enOSPzJuBqyNieXrtFWzobuAySdcC146Qvhfwt+nrS4CvZtKujYhhYJGk7TbyfVnn6vT76W7gdST30//WKO8BwPdL3T+Ze2g/Sf8EvALYCliIA3FXcCAuzurM6+HM9jAvf+4C3hsR92cPlPTWEc73fIXrCKg6YXgd3+B/ExHvqVEGAV+JiPPLrnVKrfIAfwPsQ1Kb+YKkXWrkz54v+7m6aa53dPz9JOm1wG+V9BHfWaW8G5RB0ibAeSQ1+8fSWv0m1cppncN9xK11HfDxUl+PpN3S/c8BW+Q8x/XAyaWNkZrSIuIfImLmCD8Vm9Fylv3DkjZPrztZ0rbADcD70mZtypumJY0BdoyIm4B/AiaQ1KazbuHlWsn7gd82UE7rHW19P0XEA8BXgM/UKO/1wAmS+tL9W/Fy0F2e3nMeH9FFHIhb68vAXwF3S7o33Qa4CZhRGlxS4xz/DExMB3Dcxcjf1JsuIq4HLgduTZsHrwa2iIiFwL8AN6fl+WbZoWOBS9Nj7gDOiYhnyvKcAvxd2pz3AeATBb4V6x6dcD99H9hH0rQq5f0h8Gi6/y7gmPQe+QFwD0l3zoIml8taSF4G0czMrHVcIzYzM2shB2IzM7MWciA2MzNrIQdiMzOzFuqaQDxr1qwgefbOP/5p1U9H8T3jnzb4MbooEC9fvrzVRTDrKL5nzNpD1wRiMzOzTuRAbGZm1kIOxGZmZi3kQGxmZtZChQZiSbMk3S9psaTTRkg/IV38+k5Jv5U0I90/VdKL6f47JX2/yHKamZm1SmGBWNJY4FzgIGAGcHQp0GZcHhG7RsRMkjVnswsEPJhZ1eSEospp1k5yfHkdL+nKNP33kqam+9+f+eJ6p6RhSTNHu/xmtvGKrBHvCSyOiIciYg1wBXBoNkNE/DmzuRl+rsx6WM4vr8cDKyNiZ+Ac4GyAiLis9MWVZMWqR9I1b82szRUZiCcDj2W2l6T71iPpJEkPktSIT8kkTZN0h6SbJb1zpAtImiNpUNLgsmXLmll2s1ao+eU13b4ofX01sH9pPduMo4EfF1pSM2uaIgNx+R8HGKHGGxHnRsRrSBbL/ny6+0lgSkTsBnwSuFzSK0c4dm5E9EdE/6RJk5pYdLOWyPPldV2eiBgCngW2LstzFA7EZh2jyEC8BNgxs70D8ESV/FcAhwFExOqIeDp9fRvwIPDagspp1i7yfHmtmkfSW4EXIuLeES/gViSztlNkIF4ATJc0TdI4YDYwkM0gaXpm82+AP6b7J6X9ZUjaCZgOPFRgWc3aQZ4vr+vySOoDtgRWZNJnU6U27FYks/bTV9SJI2JI0snAdcBY4IKIWCjpTGAwIgaAkyUdALwErASOSw/fBzhT0hCwFjghIlZseBWz4gwPB08/v4Y1Q2sZ1zeWrTcbx5gxI1VIm2bdl1fgcZKgekxZngGS++RW4AjgxogIAEljgCNJ7h8z6xCFBWKAiJgHzCvbd3rm9ScqHPdT4KdFls2smuHh4P6nnuOjFw+yZOWL7DBxU37wwX5et90WhQXjnF9efwRcImkxSU14duYU+wBLIsKtR2YdROmX6Y7X398fg4ODrS6GdYllz63m8PPms2Tli+v27TBxU645cW8mbTG+0mGFVpebzfeMtYGOumeKUmiN2KzVajUvV0pfM7R2vSAMsGTli6wZWjvab8HMupwDsXW8SsG0VvNytfRxfWPZYeKmG9SIx/WNbeE7NbNu5EUfrKOVgunh581n77Nv4vDz5nP/U8+tC86lIAtJjfajFw/y9PNrAKqmb73ZOH7wwX52mLgpwLogvfVm41rzRs2sa7lGbB2hUq23UjC95sS9azYvV0sfM0a8brst1p1nlEZNm1kPciC2tletCblaMK3VvFwrfcwYVRuYZWbWFG6atrYwPBwse241j698gWXPrWZ4+OXR/NWakEvBNKsUTGs1L7v52czagWvE1jT1jlCuNaiqWq13+y2TvOXHls5drXnZzc9m1g4ciG2jFDFCuVo/76QtxldtQs4TbKs1L7v52cxazU3TlltRI5RrDaqq1YRcCqaTJ76CSVuMd43WzDqKa8S2gdEeoZxn0JSbkM2sW7lGbOupVuvNE0yzRhqhPFJ6nkFTrvWaWbdyIO5B7TZCOVvjnf+Z/bjmxL0LXVzBzKyduGm6x7TrCGUPmjKzXlVoIJY0C/g2yZJuP4yIs8rSTwBOIllzeBUwJyIWpWmfBY5P006JiOuKLGuv8AhlM7P2UljTtKSxwLnAQcAM4GhJM8qyXR4Ru0bETOCrwDfTY2eQrLO6CzALOC89n+VUqfnZI5TNzNpLkTXiPYHFpUXKJV0BHAosKmWIiD9n8m8GlDorDwWuiIjVwMPpIuh7ArcWWN6u0ciqQh6h3Fo5WpHGAxcDuwNPA0dFxCNp2puA84FXAsPAHhHxl9ErvZnVo8jBWpOBxzLbS9J965F0kqQHSWrEp2zksXMkDUoaXLZsWdMK3gnqHXDlEcrtK2cr0vHAyojYGTgHODs9tg+4FDghInYB9gVeGqWim1kDiqwRj/TXOzbYEXEucK6kY4DPA8dtxLFzgbkA/f39G6R3q0YGXLnG29ZqtiKl22ekr68GvitJwIHA3RFxF0BEPD1ahTazxhRZI14C7JjZ3gF4okr+K4DD6jy2p9SaxarWM72u8batPC1B6/JExBDwLLA18FogJF0n6XZJ/zQK5TWzJigyEC8ApkuaJmkcyeCrgWwGSdMzm38D/DF9PQDMljRe0jRgOvCHAsvaMtWamIsacGVtK09LUKU8fcA7gPenvw+XtP8GF+jh7hyzdlVY03REDEk6GbiOZODJBRGxUNKZwGBEDAAnSzqApC9rJUmzNGm+q0ia5IaAkyJibVFlbZVqTcyAB1z1njwtQaU8S9J+4S2BFen+myNiOYCkecBbgBuyB/dqd45ZO1NEd9yL/f39MTg42OpibJRlz63m8PPmbxBQrzlxb4CKaVtvNq5qH7G1TEMffhpYHwD2Bx4naVU6JiIWZvKcBOwaESdImg38bUS8T9JEkqD7DmAN8AvgnIj4r0rX68R7xrqO/2DhmbUKV22N3lpNzB5w1VtytiL9CLgkfaRvBUmXDxGxUtI3SYJ3APOqBWEzax8OxE1Q7xq9tZqYazU/e5aq7hMR84B5ZftOz7z+C3BkhWMvJXmEycw6iBd9aFAja/RWG1TlAVdmZr3BNeIGNbJGb60mZjc/m5l1PwfiBjWy4D1Ub2J287OZWfdz03QO1Z71bXTBezMz622uEddQa8BVKdjWs0avmZmZA3Gq0sjnWuv3esF7MzNrhAMx1Wu9tQZcgYOtmZnVz33EVF9EodYCCmZmZo3oqUBczyIKHnBlZmZF6pmm6WrNz9UeM/KAKzMzK1LP1IirNT/XqvV6/V4zs9qqPepplfVMjbha87NrvWZmjan1qKdVVmiNWNIsSfdLWizptBHSPylpkaS7Jd0g6dWZtLWS7kx/BhotS61BV671mplVV63GW2tufaussEAsaSxwLnAQMAM4WtKMsmx3AP0R8SbgauCrmbQXI2Jm+nNIo+XxoCszs9oqBdtqC9xA7WVdrbIim6b3BBZHxEMAkq4ADgUWlTJExE2Z/L8Dji2qMG5+NjOrrlrzcq3JjfLMrW8jK7JpejLwWGZ7SbqvkuOBn2e2N5E0KOl3kg4b6QBJc9I8g8uWLatZIDc/W7vL0Z0zXtKVafrvJU1N90+V9GKmO+f7o1126xyVar3Vmpdr1Xjd6li/ImvEI0W5EYfQSToW6Afeldk9JSKekLQTcKOkeyLiwfVOFjEXmAvQ39/v4XnW0TLdOe8m+eK6QNJARCzKZDseWBkRO0uaDZwNHJWmPRgRM0e10NZx6p1JsFaN162O9SuyRrwE2DGzvQPwRHkmSQcAnwMOiYjVpf0R8UT6+yHg18BuBZbVrB2s686JiDVAqTsn61DgovT11cD+kvyXztZT76CqRleTc6tjfYqsES8ApkuaBjwOzAaOyWaQtBtwPjArIpZm9k8EXoiI1ZK2AfZm/YFcZt1opO6ct1bKExFDkp4Ftk7Tpkm6A/gz8PmI+E3B5bU2VOsxomq13u233NSrybVAYYE4/SNxMnAdMBa4ICIWSjoTGIyIAeBrwObAT9Iv9Y+mI6TfAJwvaZik1n5WWfOcWTfK051TKc+TJN05T0vaHbhW0i4R8ef1DpbmAHMApkyZ0oQiW7tpZFCVV5NrjUIn9IiIecC8sn2nZ14fUOG4W4BdiyybWRvK051TyrNEUh+wJbAiIgJYDRARt0l6EHgtMJg92OMqukelpVvzDqoaqdYLDrat0DMza5l1gJrdOcAAcBxwK3AEcGNEhKRJJAF5bTrAcTrw0OgV3UZTvXPngwdVtaOemWvarN1FxBBQ6s65D7iq1J0jqTSpzY+ArSUtBj4JlB5x2ge4W9JdJIO4ToiIFaP7DqyZ6h1w5UFVncc1YrM2kqM75y/AkSMc91Pgp4UX0JqqUvNyIwOuXOPtPK4Rm5m1QLUpI2vN2+y58/ORdKqkV7S6HLXUDMSStpP0I0k/T7dnSDq++KKZmXU+z2LVUqcCbR+I8zRNXwj8G8mkGwAPAFeS9FWZmVkFnsVq9EjaDLiK5GmDscBPgFcBN0laHhH7SfoesAewKXB1RHwxPfZg4JvAcuB2YKeIeE96zn8leYqnDzgjIv6j2WXP0zS9TURcBQzDugElXk7DzAzPYtVGZgFPRMSbI+KNwLdIHv/bLyL2S/N8LiL6gTcB75L0JkmbkEwsdVBEvAOYlDnn50ieTNgD2A/4WhqcmypPjfh5SVuTTiwg6W3As80uiJlZp/EsVm3lHuDrks4GfhYRvxlh9tf3pZPa9AHbkyzROwZ4KCIeTvP8mHTSG+BA4BBJn063NwGmkDzV0DR5AvEnSZ5dfI2k+STfFo5oZiHMzNpZpdHNnsWqfUTEA+mscgcDX5F0fTY9fT7/08AeEbFS0oUkgbXatxsB742I+wsqNpCjaToibidZFentwN8Du0TE3UUWysysXVQb3dzooCo3LzePpFeRrFFwKfB14C3Ac8AWaZZXAs8Dz0raDjgo3f8/wE6lJUV5eTUzSJ7p/3hpYZV0fYSmq1kjlvTBsl1vkUREXFxEgczM2km1Wq8HVbWVXUn6cIeBl4CPAXsBP5f0ZDpY6w5gIcmsc/MBIuJFSScCv5C0HPhD5pxfJulrvjsNxo8A72l2wfM0Te+Reb0JsD/JqDIHYjPrCpWanoG6+3lL3MQ8OiLiOpIabNYgyajnUp4PVTj8poh4fRpsz02PIyJeJGkJLlTNQBwRH89uS9oSuKSwEpmZjaJaA64a6ee1jvFRSccB44A7SEZRj5p6ZtZ6gWRCeTOzjlHPxBrgft5eEBHnRMTMiJgREe+PiBdG8/p5+oj/k5fXRB1DMtz7qjwnlzQL+DbJw9U/jIizytI/CXwEGAKWAR+OiD+laccBn0+z/nNEXJTnmmZm5eqdWAPcz2vFy9NH/PXM6yHgTxGxpNZBksaStLW/m2QN1QWSBiJiUSbbHUB/RLwg6WPAV4GjJG0FfBHoJ/kScFt67Mpc78rMLKORAVfgfl4rVp7Hl27O/MzPE4RTewKLI+KhiFgDXAEcWnbumzJNAL8jmZoM4K+BX0bEijT4/pJk1hQzs41WrdbreZut1SrWiCU9x8tN0uslARERr6xx7snAY5ntJcBbq+Q/Hvh5lWMnj1DGOaQzoEyZMqVGccys21Ua/ewBV9bOKtaII2KLiHjlCD9b5AjCMPJsJSMFdiQdS9IM/bWNOTYi5kZEf0T0T5o0aYRDzDqLpFmS7pe0WNJpI6SPl3Rlmv77zCQEpfQpklZlpuTrKtXmda428YYHXFk1kiakzxJv7HHzJE1o9Pp5+ohLF9yW5DliACLi0RqHLAF2zGzvQDIBd/l5DyCZWPtdEbE6c+y+Zcf+Om9ZzTpRznEVxwMrI2JnSbOBs1l/JqBzeLllqavUesyo1nSTrvVaFROAE4HzsjsljY2IioscRcTBzbh4nvWID5H0R+Bh4GaSmUXy3OgLgOmSpkkaB8wmmbM6e+7dSJ7XOiQilmaSrgMOlDRR0kSSibfLH9Q26zY1x1Wk26UnCK4G9s9Mv3cYyYxBC0epvKOq1mNGeUY/u9bbHVYPrd3r8ZUv3vKnp59/+PGVL96yemjtXg2e8iyS9RTulLRA0k2SLidZSAJJ10q6TdLCtEuUdP8jkraRNFXSfZJ+kOa5XtKmlS5WLs9zxF8G3gY8EBHTSGbWml/roHS5xJNJAuh9wFURsVDSmZIOSbN9Ddgc+En6AQykx65Ir7sg/Tkz3WfWzfKMjViXJ73HngW2Tpdm+wzwpVEoZ6EqNT/XCrTVlhW07rF6aO1eDzy1auCoubfu9a6v/XrqUXNv3euBp1YNNBiMTwMejIiZwD+SfCn+XETMSNM/HBG7k3ShnpKuSFhuOnBuROwCPAO8N+/F8wTilyLiaWCMpDERcRMwM8/JI2JeRLw2Il4TEf+S7js9IkoB94CI2C59kHpmRBySOfaCiNg5/fm3vG/IrIPlGRtRKc+XgHMiYlXVC0hzJA1KGly2bFmdxSxOtX7eWoHWo597w/Ln1nzjY5fetk22ZeRjl962zfLn1nyjiZf5Q2ZZREiC710kT/fsyMiTWj0cEXemr28Dpua9WJ4+4mckbQ78BrhM0lKS54nNrLnyjKso5VkiqQ/YElhB8kTCEZK+StLfNSzpLxHx3ezBETEXmAvQ398/4uDJVqrWz1sKtJXmdfbo594wNDy8/UgtI0PDw9s38TLPl15I2hc4ANgrnfPi12TGS2WszrxeC+Rums4TiP+b5Mb+BHAsyY1/Zt4LmFlu68ZVAI+TjKs4pizPAHAccCvJuuA3RkQA7yxlkHQGsKo8CLeLehdYyBNoPfFG9+sbM+bJHSZuOrX8UbS+MWOebOC02eUSy21JMkDyBUmvJ+mqbao8TdMi6ef9NUl/7pVpU7WZNVHOcRU/IukTXgx8kqRvq2NUa3qG2v28HnBl22wx7lPfO3b35dkuiO8du/vybbYY96l6z5nGtPmS7uXlx2hLfgH0SbqbZOzS7+q9TiVKvkznyCi9ieQxifcCSyLigGYXphH9/f0xODjY6mJYb+uoqNCKe2bZc6s5/Lz5G0ysUXrEqNYjStZ16vpHXT20dq/lz635xtDw8PZ9Y8Y8uc0W4z41vm/src0u3GjJ/RwxsBT4X+BpYNtiimNm3cwLLFgzjO8be+vkiZu+vdXlaJY8qy99jKQmPInkucWPlk0wYGa2nnqmmixxP6/1mjw14lcDp2aGZZuZVVStebnWyGezXpS7j7jduY/Y2kBHtZ8Wdc/k6QeuNGraeo7/4dm4PmIzs3UqBdS8U02aWcKB2Mw2WrXm5zz9wGb2sjzPEZtZD6q25GC1BRg81aR1mnqXQUyPPVXSKxq5vmvEZraBWs/zNjoDlnWp4WF4YRkMrYG+cfCKSTCmI+p7Iy6DmNOpwKXAC/VevCM+ITMbXbWWHPQMWLaB4WFYugh+eAB8643J76WLkv3NNrR6L5557BZWPPwwzzx2C0Orm7kM4tck/WO6HOLdkr4EIGkzSf8l6S5J90o6StIpwKuAmyTdVO/FHYjNbAO1Bly5+dk28MIyuOJoeObRZPuZR5PtF5q8ytfQ6r1Yet8AFx68F9+ZOZULD062GwvG2WUQf0myutKeJCsN7i5pH2AW8EREvDki3gj8IiK+Q7Iwy34RsV+9Fy80EEuaJel+SYslbTAnrqR9JN0uaUjSEWVpa9NvJ+vWKTaz0ZGnxltqfp7/mf245sS9PQ1lLxgehlVPwTOPJb+ztd2hNS8H4ZJnHk32N9Oqpd/gqg9ss17Av+oD27BqabOWQTww/bkDuB14PUlgvgc4QNLZkt4ZEc826XrF9RFLGgucC7ybZOm2BZIGymblehT4EPDpEU7xYvrtxMxGWZ6JN/wYUo8pNT2Xar0TpsDsH8O2M5J+4L5xyb5sMJ4wJdnf1HIMbT9iwB8eatYyiAK+EhHnb5Ag7Q4cDHxF0vUR0ZSVCIscrLUnsDgiHgKQdAVwKLAuEEfEI2laAZ0IZlYvD7jqYZUGXFVqev7Ir2Dz7ZJ8s3+8YaB+xaTmlm9M35NMmDJ1g4A/pq9ZyyBeB3xZ0mURsUrSZOAlkni5IiIulbSKpBKZPXZ5vRcvMhBPBh7LbC8hWbw8r00kDQJDwFkRcW15BklzgDkAU6ZMaaCoZlbONd4eVK3WW6vpecyYJN9HflXsqOnNt/0U77tkYF3z9IQp8L5LlrP5tg0tgyiptAziz4HLgVslAawCjgV2Br6WVhxfAj6WHj4X+LmkJ+vtJy4yEI/01Xlj5tOcEhFPSNoJuFHSPRHx4Honi5hL8iHQ39/fHXN1mpnl0cijQvXUevM0PY8Zk9SOi9Q3/la2fcMhfGjeNxge2p4xfU+y+bafom98Q8sgRsQxZbu+Xbb9IEltufy4fwX+tZFrFzlYawmwY2Z7B5LRZblExBPp74eAXwO7NbNwZu0oxwDH8ZKuTNN/L2lqun/PzODGuyQdPtplt1HUyKNC1Y6tVustNT1PSFsfi2p6zqNv/K1M2PHtbDVtGhN2fHujQbjVigzEC4DpkqZJGgfMBnKNfpY0UdL49PU2wN5k+pbNulFmgONBwAzgaEkzyrIdD6yMiJ2Bc4Cz0/33Av3pAMdZwPmSPGFPt6r1qFC10c3Vji3VerNKtd5s0/Op9ya/SwO1rCGFfYIRMQScTFKVvw+4KiIWSjpT0iEAkvaQtAQ4kuQPx8L08DcAg5LuAm4i6SN2ILZut26AY0SsAUoDHLMOBS5KX18N7C9JEfFCes8BbMLGdQNZp6lWc61VW26k1ltqep6wY/LbQbgpCv3GHBHzgHll+07PvF5A0mRdftwtwK5Fls2sDeUZ4LguT0QMSXoW2BpYLumtwAUka4h/IBOYrZWq9eXWm1atv7bW6OZqx47WgCtbjz9ds/aRZ4BjxTwR8fuI2AXYA/ispE02uIA0R9KgpMFly5o845FtqFrttN40qF5zrTW62bXetuM+JLP2kWeAYynPkrQPeEtgRTZDRNwn6XngjcBgWZqfNBhN1WqnUF9aKThWqrnWGt3sWm/b8Sdv1j7yDHAcAI5LXx8B3BgRkR7TByDp1cDrgEdGp9g9oNrgp3qnfaw3raRSzTXP6GbXetuKa8RmbSLt8y0NcBwLXFAa4AgMRsQA8CPgEkmLSWrCs9PD3wGcJuklYBg4MSLqnumn59Tqq600yQU0Nu1jvWnVuMbbcRTRHa1T/f39MTg4WDujWXE6av5H3zOpWnMor3oq6aMtD4qlJuRKaZtvV38Qr5bWXQG1o+6ZorhGbGa9od45lGs1Ezcy7WO9adZVHIjNrPs1ModyI83LUH3ax3rTrKv465WZdY9KA6fqnU0Kqg9+aqdpH61juUZsZt2h3lrvKydXX76vkeZlsxwciM2sc1Qb3VzvykF5Rhm7CdkK5K9tZtYZPIeydSnXiM2svdQ7utlzKFuHciA2s+ZrZKGDekc3l2q91fp63YRsbciB2Myaq5GZqOrt5wXXeq1jFfo/VNIsSfdLWizptBHS95F0u6QhSUeUpR0n6Y/pz3Hlx5pZm6r2qFCtBe0b6ecF9/VaRyqsRixpLHAu8G6SFWMWSBqIiEWZbI8CHwI+XXbsVsAXgX6SJd5uS49dWVR5zaxJGpmJyv281oOK/B+8J7A4Ih6KiDXAFcCh2QwR8UhE3E0ySX3WXwO/jIgVafD9JTCrwLKa2caqNHlGtQkyGpk8A1zjta5UZB/xZOCxzPYS4K0NHDu5PJOkOcAcgClTppQnm1kj6h1UVWvQVCOTZ5h1oSID8UirauRd6inXsV7k3AZposIAAAmYSURBVKwgtVYkqvUoUSMzUXl0s/WYIr9mLgF2zGzvADwxCseaWaMaGVQF1ZuQ3bxstp4i74AFwHRJ0ySNI1nAfCDnsdcBB0qaKGkicGC6z6yr5XjSYLykK9P030uamu5/t6TbJN2T/v4/DRUk74pEWXkXrjez9RQWiCNiCDiZJIDeB1wVEQslnSnpEABJe0haAhwJnC9pYXrsCuDLJMF8AXBmus+sa2WeNDgImAEcLWlGWbbjgZURsTNwDnB2un858H8jYlfgOOCShgrT6KAqM8tNEd3Rtdrf3x+Dg4OtLob1tpHGNuQ/WNoLOCMi/jrd/ixARHwlk+e6NM+tkvqA/wUmReZGliSSwPyqiFhd6XpV75lafcSlPJUGc5nl09A90y08s5ZZ+8jzpMG6PBExJOlZYGuSwFvyXuCOakG4pkZXJDKz3ByIzdpHnqcFquaRtAtJc/WBI15gYx75c6A1GxVuRzJrH3meFliXJ22a3hJYkW7vAFwDfDAiHhzpAhExNyL6I6J/0iT355q1Awdi622VZoeqlVaMPE8aDJAMxgI4ArgxIkLSBOC/gM9GxPyiC2pmzeOmaWsPRQ38qXd2KKg9WKnJ0j7f0pMGY4ELSk8aAIMRMQD8CLhE0mKSmvDs9PCTgZ2BL0j6QrrvwIhYWkhhzaxpPGraNtTIWrLVgmml9EZH6NZ73lVPwQ8P2HCBgY/8KnldKa1yv2lHjQD1PWNtoKPumaK4RtytiliYHeqvQVY7b63pEustU63zNrJKkJlZk7iPuNXq7aOslbZ0UVKj+9Ybk99LFyX7q6VB/WvJ1poSsVp6rYBY73UbmR3KM0eZ2ShxIM6j1qCd0Q6YRQVTqB686k2rdd5aQa/e6zYyO5RnjjKzUdJbgbiegFkr6LVj7bORgFlvLbFW0KuWXivo1XvdPGvbliatOPXe5HepKb1amplZE/XOX5V6A2YjTa6tqn0WtTB7IzXIaum1gl69180TTL1KkJm1WO8M1qo2cAcqpzXS5Fp6XSmtFBTLR+aWgmK9aUUuzF5vWq3zVpvFqZEyeXYoM2tzvROI6w2YtYJlI8G0kYBZVDAtHV8tKNaTlie9mkaua2bWxnonENcbMGsFy3asfZbO7cBlZtb2Cp3QQ9Is4NskswT9MCLOKksfD1wM7A48DRwVEY+ki53fB9yfZv1dRJxQ7Vo1JydoZBaleieqqJVm3aajJifwhB7WBjrqnilKYTXizCLn7yaZqH6BpIGIWJTJtm6Rc0mzSVaNOSpNezAiZjatQEX2M7r2aWZmdSqyaXpPYHFEPAQg6QrgUCAbiA8FzkhfXw18N13UvBgOmGZm1maKbCMdaZHzyZXyRMQQUFrkHGCapDsk3SzpnSNdQNIcSYOSBpctW9bc0puZmY2CIgNxI4ucPwlMiYjdgE8Cl0t65QYZvbaqmZl1uCIDcd2LnEfE6oh4GiAibgMeBF5bYFnNzMxaorBR02lgfQDYH3icZNHzYyJiYSbPScCuEXFCOljrbyPifZImkQTktZJ2An6T5ltR5XrLgD/lLN42wPK63lhxXKZ82rlMyyNiVqsLk9dG3DPt/Jm3E5cpn2yZOuqeKUphg7UaXOR8H+BMSUPAWuCEakE4vV7utmlJgxHRv/HvqjguUz4uU/PkvWfa8f25TPm4TJ2h0Ak9ImIeMK9s3+mZ138BjhzhuJ8CPy2ybGZmZu3AM0uYmZm1UK8G4rmtLsAIXKZ8XKbR147vz2XKx2XqAIVOcWlmZmbV9WqN2MzMrC04EJuZmbVQTwViSbMk3S9psaTTWl0eAEmPSLpH0p2SWrIUjqQLJC2VdG9m31aSfinpj+nviW1QpjMkPZ5+VndKOniUy7SjpJsk3SdpoaRPpPtb+lkVyfdMxTL4nslXpp67Z+rRM4E4sxrUQcAM4GhJM1pbqnX2i4iZLXy27kKg/KH604AbImI6cEO63eoyAZyTflYz08fjRtMQ8KmIeAPwNuCk9P9Qqz+rQvieqepCfM/k0VP3TL16JhCTWQ0qItYApdWgel5E/DfJhCpZhwIXpa8vAg5rgzK1VEQ8GRG3p6+fI1kzezIt/qwK5HumAt8z+fTgPVOXXgrEeVaDaoUArpd0m6Q5rS5MxnYR8SQkNxOwbYvLU3KypLvTZriWNWdJmgrsBvye9v2sGuV7ZuO06/8D3zNtrpcCcZ7VoFph74h4C0nz30mS9ml1gdrY94DXADNJVuj6RisKIWlzkpnfTo2IP7eiDKPE90zn8z3TAXopEOdZDWrURcQT6e+lwDUkzYHt4ClJ2wOkv5e2uDxExFMRsTYihoEf0ILPStJfkfxBuSwi/j3d3XafVZP4ntk4bff/wPdMZ+ilQLwAmC5pmqRxJAtMDLSyQJI2k7RF6TVwIHBv9aNGzQBwXPr6OOA/WlgWYN0NW3I4o/xZSRLJQiX3RcQ3M0lt91k1ie+ZjdN2/w98z3SGnppZKx26/y1eXg3qX1pcnp1IvtFDsgDH5a0ok6QfA/uSLE/2FPBF4FrgKmAK8ChwZK0VsEahTPuSNLEF8Ajw96V+plEq0ztIluS8BxhOd/8/kj6vln1WRfI9U7Ecvmfylann7pl69FQgNjMzaze91DRtZmbWdhyIzczMWsiB2MzMrIUciM3MzFrIgdjMzKyFHIgtN0n7SvpZq8th1il8z1geDsRmZmYt5EDchSQdK+kP6fqj50saK2mVpG9Iul3SDZImpXlnSvpdOin8NaVJ4SXtLOlXku5Kj3lNevrNJV0t6X8kXZbOnGPW0XzPWCs5EHcZSW8AjiKZGH8msBZ4P7AZcHs6Wf7NJLPuAFwMfCYi3kQy+01p/2XAuRHxZuDtJBPGQ7J6yqkk69PuBOxd+JsyK5DvGWu1vlYXwJpuf2B3YEH6xXtTkgnVh4Er0zyXAv8uaUtgQkTcnO6/CPhJOpfv5Ii4BiAi/gKQnu8PEbEk3b4TmAr8tvi3ZVYY3zPWUg7E3UfARRHx2fV2Sl8oy1dtbtNqTWerM6/X4v9D1vl8z1hLuWm6+9wAHCFpWwBJW0l6Ncm/9RFpnmOA30bEs8BKSe9M938AuDldL3SJpMPSc4yX9IpRfRdmo8f3jLWUv5l1mYhYJOnzwPWSxgAvAScBzwO7SLoNeJakTwySJci+n/7ReAj4u3T/B4DzJZ2ZnuPIUXwbZqPG94y1mldf6hGSVkXE5q0uh1mn8D1jo8VN02ZmZi3kGrGZmVkLuUZsZmbWQg7EZmZmLeRAbGZm1kIOxGZmZi3kQGxmZtZC/x8Vtp9erBUe3QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 491.375x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "output1, _ = track_model_metrics(model=model1, train_interactions=train_interactions, \n",
    "                              test_interactions=test_interactions, k=K,\n",
    "                              no_epochs=NO_EPOCHS, no_threads=NO_THREADS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second model (with both implicit and explicit data) fitting progress:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeIAAADQCAYAAADbLGKxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5wcVZn/8c93ZpyQmySECQsJARRUgiLIgLJRhJ/ABsUgK1dXideoiMqqu+K6Ios32BWRVVCiolwERHbR6OJy19WIkuEOQTQiQgRzI8HcTJj08/ujqpNOp7u60z013TPzfb9e85ruqjpVpzqpefqcOvUcRQRmZmbWGh2troCZmdlI5kBsZmbWQg7EZmZmLeRAbGZm1kIOxGZmZi3kQGxmZtZCDsRtSNIBkl6fsb5X0n/mdOzDJT0r6V5Jj0j69ADt932STstYP0vSWQNxLLNSbXQ9/UbSF3M4xncknZC+/qmk3oE+huWrq9UVsIoOAHqBG8tXSOqKiD6gL8fj/zwijpU0FrhP0o8j4u6yOvRvzw4j4us11s8D5jVWXbNM7XI9jQbulXRDRMzP8Xg2xLhFnANJe6bffr8p6SFJ35V0pKT5kn4n6ZB0u7GSLpO0IP3GfJykbuBc4GRJ90k6WdI5kuZKuhm4Iv2W/eN0H+MkfVvSg5IekPTmgTqPiFgL3A28UNLbJX1f0o+Am9Nj/1Na9wck/VvJ+Z+WLrtf0pXpsnMkfSx9/SFJC9Ntrk2XvV3SV9PXe0i6LV1/m6Rp6fLvSPpPSb+U9FixFWDD2zC6ntYD9wFTqtU3Xd4p6YsldfhguvzsdNuH0vproOpmreUWcX72Bk4E5gALgLcArwZmAf8CvAn4JHB7RLxT0gTgLuBW4GygNyLOgCSIAQcBr46I9ZIOLznOp4BnI+Jl6bYTyysi6ULgiAp1vDYizqt2ApImAa8CPgMcDBwK7B8Rz0g6GtgHOAQQME/SYcCK9LxmRMRySTtV2PVZwF4RsSE973JfBa6IiMslvRP4z/TzAtiV5HN8CUkL+vpq9bdhZThcTxNJrpn/SxdtU19JtwKnAXsBB0ZEf8k19NWIODfd15XAscCPqh3Phg4H4vz8ISIeBJD0MHBbRISkB4E9022OBmYVW4rADsC0Kvubl36jLnckcErxTUSsLN8gIv5xO+v+Gkn3AgXgvIh4WNLBwC0R8UxJ3Y8G7k3fjyP5I/Ny4PqIWJ4e+xm29QDwXUk/AH5QYf2hwN+nr68E/r1k3Q8iogAslLTLdp6XDV1D/Xp6AHgxyfX05xr1PRL4evH2T8k1dISkfwbGADsBD+NAPCw4EOdnQ8nrQsn7Als+dwFvjohHSwtKemWF/a2tchwBmQnDG/gG//OIOLZGHQR8ISIuLTvWh2rVB3gDcBhJa+ZTkvarsX3p/ko/V3fNjRxD/nqS9CLgF0ruEd+XUd9t6iBpB+ASkpb9k2mrfoesetrQ4XvErXUT8MHivR5JB6bLVwPj69zHzcAZxTeVutIi4h8j4oAKP1W70eqs+zsljUuPO0XSZOA24KS0W5vyrmlJHcDuEXEH8M/ABJLWdKlfsqVV8g/AL5qop40cbX09RcRvgS8AH69R35uB90nqSpfvxJaguzy95jw+YhhxIG6tzwDPAx6Q9FD6HuAOYHpxcEmNfXwWmJgO4Lifyt/UB1xE3AxcDdyZdg9eD4yPiIeBzwE/S+vzpbKincBVaZl7gQsjYlXZNh8C3pF2570N+HCOp2LDx1C4nr4OHCZpr4z6fhN4Il1+P/CW9Br5BvAgye2cBQNcL2sheRpEMzOz1nGL2MzMrIUciM3MzFrIgdjMzKyFHIjNzMxaaNgE4pkzZwbJs3f+8U+rfoYUXzP+aYMfYxgF4uXLl7e6CmZDiq8Zs/YwbAKxmZnZUORAbGZm1kIOxGZmZi3kQGxmZtZCDsRmZmYt5EBsZmbWQg7EZmZmLeRAbGZm1kIOxGZmZi3kQGxmZtZCDsRmZmYt5EBsZmbWQg7EZmZmLZRrIJY0U9KjkhZJOqvC+vdJelDSfZJ+IWl6ybpPpOUelfR3edbThq9CIVi2egN/WrmOZas3UCh45jUzay9dee1YUidwMXAUsBhYIGleRCws2ezqiPh6uv0s4EvAzDQgnwLsB+wG3CrpRRGxKa/62vBTKASPLlnNe67oY/HK9UydOJpvnNbLi3cZT0eHWl09MzMg3xbxIcCiiHgsIjYC1wLHlW4QEX8peTuWLRNFHwdcGxEbIuIPwKJ0fzZCNdKyXbF24+YgDLB45Xrec0UfK9ZuzLu6ZmZ1y61FDEwBnix5vxh4ZflGkj4AfAToBv5fSdlflZWdUqHsHGAOwLRp0wak0tZ+Gm3ZbuzftDkIFy1euZ6N/e5YMbP2kWeLuNJfyG2aMRFxcUS8EPg48K/bWXZuRPRGRG9PT09TlbXm5XU/ttGWbXdXJ1Mnjt5q2dSJo+nu6qzruL6/bGaDIc9AvBjYveT9VOCpjO2vBd7UYFlrsWKr9fhL5jPj/Ds4/pL5PLpk9YAEr0ZbtpPGdvON03o3B+NiS3rS2O6ax8zzfMzMSuUZiBcA+0jaS1I3yeCreaUbSNqn5O0bgN+lr+cBp0gaJWkvYB/grhzrak2qp9XaaAuz0ZZtR4d48S7jueH0Gcz/+BHccPqMbbqzq9XJ95fNbLDkdo84IvolnQHcBHQCl0XEw5LOBfoiYh5whqQjgeeAlcDstOzDkq4DFgL9wAc8Yrq91Wq1NjOCudiyLS9bT8u2o0P0jB9VcV1WnXx/2cwGS56DtYiIG4Eby5adXfL6wxllPwd8Lr/aWSMKhWDF2o1s7N9Ed1cnk8Z209Ghza3W0uBV2mqt1sK84fQZ9IwfVXW/sHXLttL6rLJZsupU63zMzAZKroHYmtNogMmzPtVakLVarVktzHpay9Vats20tLPqtOuOoxtuhZuZbQ8H4jaVZzKKPFqQPeNHZbZas1qYtfbbTJ2yZNWpVis8L5JmAheR3M75ZkScV7Z+FHAFcBCwAjg5Ih5P1+0PXAo8HygAB0fEX3OtsJk1zbmm21Szg4WqDUJqZjRwrfumxVbrlIlj6Bk/aquglTWCuZn7sc2UrTWqOut88lCSje4YYDpwamna19S7gJURsTdwIXB+WrYLuAp4X0TsBxxOMvbCzNqcW8RtqpkAk9WarqcF2eh94CxZLcxm9ptXnVpkczY6AEnFbHSlaWGPA85JX18PfFWSgKOBByLifoCIWDFYlTaz5rhF3KaaSUaR1Zqud3RzpRZzM8/lQvUWZjP7zatOLVIpG115RrnN20REP/AsMAl4ERCSbpJ0j6R/rnQASXMk9UnqW7Zs2YCfgJltP7eIU+02MKqZR3aygm2zo5vzaEE20zJtw1ZtM+rJKFdtmy7g1cDBwDrgNkl3R8RtW20YMReYC9Db2+vsJGZtwIGY9pylp5kAkxVsmxndXKxXrUFQjWhmv3nVqQXqyShX3GZxel94R+CZdPnPImI5gKQbgVcAt2Fmbc1d0+Q3MKpZWd2mWcfM6q6tlW2q2fzM1pSa2ejS97PT1ycAt0dEkCTO2V/SmDRAv5at7y2bWZtyi5j8Bkbl1ZqudcxaremsFmQzXeLWnDqz0X0LuFLSIpKW8Clp2ZWSvkQSzAO4MSL+pyUnYmbbRcmX6aGvt7c3+vr6Giq7bPUGjr9k/jZdufU8i9pM2Ublfcx2u18+hAypD6mZa8ZsgAypayYv7pqmuZG39YxCzuq2bqRbO+88yG02ktjMbFgbUV3T1Vp6eQ2MqtWF3Gi3tvMgm5kNHyOmRVwro1QeA6NqDQJrdJBYs8/OmplZ+xgxLeJGcxI3MzCqVhdyo13Mw+zZWTOzEW3EtIgbDXr1tFqrtaZrPQrUzKNCvo9rZjY8jJhA3GjQy3NSAXcxm5nZiOmabvT52DwnFXAXs5mZjZhA3GjQazbBRa30i8MoPaOZmTVgxARiaCzoudVqZmZ5yvUesaSZkh6VtEjSWRXWf0TSQkkPSLpN0h4l6zZJui/9Kc+3O6g8MMrMzPKSW4tYUidwMXAUycwwCyTNi4jSRPT3Ar0RsU7S+4F/B05O162PiAPyqp+ZmVk7yLNFfAiwKCIei4iNwLXAcaUbRMQdEbEuffsrkmnfzMzMRow8A/EU4MmS94vTZdW8C/hJyfsdJPVJ+pWkN1UqIGlOuk3fsmXLmq+xmZnZIMtzsFalG6kVZzSQ9Fagl2QO1aJpEfGUpBcAt0t6MCJ+v9XOIuYCcyGZSWZgqm1mZjZ48mwRLwZ2L3k/FXiqfCNJRwKfBGZFxIbi8oh4Kv39GPBT4MAc62pmZtYSeQbiBcA+kvaS1E0ygflWo58lHQhcShKEl5YsnyhpVPp6Z2AGUDrIy8zMbFjIrWs6IvolnQHcBHQCl0XEw5LOBfoiYh7wH8A44PuSAJ6IiFnAvsClkgokXxbOKxttbWZmNizkmtAjIm4EbixbdnbJ6yOrlPsl8LI862ZmZtYORsykD2ZmZu3IgdjMzKyFHIjN2kgdaWFHSfpeuv7XkvZMl+8paX1JWtivD3bdzawxI2rSB7N2Vmda2HcBKyNib0mnAOezJS3s750W1ppWKMC6ZdC/Ebq6YUwPdLjNlid/umbto2Za2PT95enr64HXKX3kwEaYQgHWLIFVTya/C4WB2efShfDNI+HLL01+L104MPu2qhyIzdpHPWlhN28TEf3As8CkdN1eku6V9DNJr6l0AKeFHUKyAm2tgNlokF63DK49FVY9kbxf9UTyfp3/r+TJgdisfdSTFrbaNk+TpIU9EPgIcLWk52+zYcTciOiNiN6enp6mK9x28mgltkKtQJsVMJtp1fZv3LLPolVPJMstNw7EZu2jnrSwm7eR1AXsCDwTERsiYgVARNwN/B54Ue41bidDsVu12heHWi3TrIDZTKu2qxsmTNt62YRpyfJmzscyORCbtY+aaWHT97PT1ycAt0dESOpJB3uRTpSyD/DYINW7PQy1btWsLw61WqZZAbOJVm1h9M70n3T1ln1PmEb/SVdTGL1zc+djmRyIzdpEes+3mBb2EeC6YlpYSbPSzb4FTJK0iKQLuviI02HAA5LuJxnE9b6IeGZwz6DFhlq3atYXh1ot0zE9cMo1WwVMTrkmWd5Eq3bFun4+cOt67j7q+zw5+y7uPur7fODW9axY19/c+VgmP75k1kbqSAv7V+DECuX+C/iv3CvYzooBqDQYb0+3aqMafdwn64vD86ckgbUY2EoDLST7nzwd3n3rNsctjN6ZwklX03XdWzaX7T/pajpG77yl5VWlzhv7N3HTwmXctHDr4Hn2Gzc1dz6WyYHYzIaHYiuxWvDKQ7E7tvyYk6fXDsZZXxwyAu1mHR0wbpdtdrtiXT//eut65hz1fSaPEUvXBXNv/QufPb6fnvGdmXXu7upk6sTRLF65fvP+pk4cTXdXZ+3PolVfhIYBRZQPyhyaent7o6+vr9XVsJFtSD3POyyvmcFORrFmSXIvtDz4vPvWikGyvK6xdCEqCYhxyjWoniCe4U8r1zHj/Du2WT7/40cwZeKYzDoXxkzm0SWrec8VfSxeuZ6pE0fzjdN6efEu4+noyP7vXdi0icKShdu2xHeZTkdn1UCe6zUj6UxgbkSsy/M4zarZIpa0C/B5YLeIOEbSdODQiPhW7rUzM9seVVqJNdUK4NXWNzMwCvHHzj1Yc8x/M6G7wKqNHYzr/Bv2QHQAhUKwYu1GNvZvorurk0lju2sGQ6B2qzajzh0d4sW7jOeG02dUPG5WnWq2xFvjTOAqYGgHYuA7wLeBT6bvfwt8j2TQiJnZ0FarezlrfRPdsSvWbuRtly0oC5hPcsPpM5g0trvhlumksd1847TebcpOGpvWqUadOzpEz/hRFT6myKxTU/eXB4CkscB1JI/9dQLfB3YD7pC0PCKOkPQ14GBgNHB9RHw6Lft64EvAcuAe4AURcWy6z6+QTMvbBZwTET8c6LrX0/+xc0RcBxRg88jOwflkzczyVmu0b9b6rNHLNWzs37RVEAZYvHI9G/s3sWLtxs0Br7j8PVf0sWLtlpZ2oRAsW72BP61cx7LVGygUktuMpa3a+R8/ghtOn7F1AB/TQ5TVOeqo84q1G/nyLb/homN34+fv3ZuLjt2NL9/ym811KrbES9V9f3lgzASeioiXR8RLgS+TPId/REQckW7zyYjoBfYHXitpf0k7AJcCx0TEq4HSD+KTJI8IHgwcAfxHGpwHVD0t4rWSJpFm+JH0KpK0emZmbaWh7txa3ctZ6+sYVFXYtIlNa5ahTRuIzlF0juuho7Mzsws5K0gXzzOrdVqtVQu1u8SricImPj+ji0k/OhFWPcHuE6bx+TdezqZCUqeaLfH8PQh8UdL5wI8j4ucV0rCfJGkOSezbFZhO0iB9LCL+kG5zDTAnfX00MEvSx9L3OwDTSB4vHDD1BOKPkCQReKGk+STfFk4YyEqYmW3W4ICrWsGp6n5rdS/XWp9xX7o4gOl5ZQOY2GV6ZuBasXZj5n3eai3mG06fUTUAF2V1iWeVnchfeN6PZm/VMzDpR7N57h23AGNq3l/OW0T8VtJBwOuBL0i6uXS9pL2AjwEHR8RKSd8hCaxZFRTw5oh4NKdqA3V0TUfEPcBrgb8F3gvsFxEP5FkpMxuhmsjOlNmdm7XfWt3LTXQ/b1qzbMsoYoBVT9B13VvYtGZZZhdyMUgXu3rLW5e1WsxZGi3bFc9V7Bnoiuc2vy22xKdMHEPP+FGDFoQBJO0GrIuIq4AvAq8AVgPj002eD6wFnk0HIR+TLv8N8ILi3N5smVYUkuQ6HyzOcCbpwDzqXs+o6dPKFr1CEhFxRR4VMrMRrNr92DoeB8oMMOtWZe83q3u5nmd6q9CmDRWDlzZtTHdduQu5Vuuymed9Gy2rKj0Dap/nhF9Gcg+3ADwHvB84FPiJpKfTwVr3Ag+TpH+dDxAR6yWdDvyvpOXAXSX7/AzJveYH0mD8OHDsQFe8nsFaB5f8vAY4B5iVVaBI0kxJj0paJOmsCus/ImmhpAck3SZpj5J1syX9Lv2ZXV7WzFosjwT/TTwOlDlYqNZ+i93LE3ZPfpcF2QJiWUzgTzGJZTGBQp2Pv0bnqIrpJqOzdvDKal3WajFnabhsEz0DgyEiboqI/SPigIg4OCL6IuIrEfGS4mCtiHh7ROwbEW+IiL+PiO+kxe+IiJeQxLgdgL50+/UR8d6IeFlEvDQiBjwIQx0t4oj4YOl7STsCV9Yqlyagvxg4imTGmAWS5kXEwpLN7gV6I2KdpPcD/w6cLGkn4NNAL8kgsbvTsivrPC8zy1MzGaWyNPE4UOZgoXXZ+80a5FXr3nNW2c5xPfRXSDfZOa654NXM/diGyzbRMzAEvCdt8HWTxKVLB/Pg251ZS9LzgAciYt8a2x1K8szV36XvPwEQEV+osv2BwFcjYoakU4HDI+K96bpLgZ9GxDXVjjcsswTZUDNyMmvVyiiVNeCq1romAnzVoJix3wLKDLTLVm/g+Evmb9OVW+/zvltGTW8kOrs3j5o2YIhdM3mp5x7xj9gyOXkHyXDv6+rY9xTgyZL3i4FXZmz/LuAnGWWnVKjbHNJh5tOmTStfbWZ5yerqzQqmkB1om2x1VX1sJ2O/K1ZvyByB3MjzvqUjkDs6O+nY8W+248O1kaaex5e+WPK6H/hjRCyuo1ylbzoVm9+S3krSDf3a7SkbEXOBuZB8u6+jTmY2ELK6kLMGXEHtwViNpqmspcp+a40ibuZ5X7N61PP40s9KfubXGYQhacXuXvJ+KkmWk61IOpIke8msiNiwPWXNrEWyBu5ktZbbcKq8WhmhsgY3tUE2KRsGqraIJa2mcgtWQETE82vsewGwT/oQ9Z+AU4C3lB3jQJKb4jMjYmnJqpuAz0uamL4/GvhEjeOZ2WDJ6kKuNeCqianyGp0IIUutjFBZg5vaIJuUDQO5ToOYJtL+MkkC7ssi4nOSzgX6ImKepFtJnv16Oi3yRETMSsu+E/iXdPnnIuLbWcfyYC1rA0Nq4EnNa6bRKQVr3CNudOq/mpmzmtBMgM/jy8EI0hYflKQJwFsi4pLtLHdjWm5VU8evNxBLmkzyfBUAEfFExuaDzoHY2kBb/FGpV+Y1U8fo5cxHfqqMFC4Ugj+uWMOaZ/68Jc/xTn/DHpPG1QxeWaOXa6V1tLbVFtdMmlXrx+lkEaXLOyMi9xv+9YyangVcQDKd1FJgD5KE1/vlWzUza5kaGa6yWqcAjy5dy3uuWLjNukbzHEMyqKpn3PO46NjdNs93+9mfLvPAqBFoQ/+mQ5ev3nhBf6Gwa1dHx9M7j+/+6Kiuzjub2OV5JPMp3EeSlWsNSU/tAcB0ST8gGbe0A3BROlAYSY+TDDQeR/LUzy9I0kH/CTguItZTh3qeCfgM8CrgtxGxF/A60tRgZjaw6shGN0rS99L1vy7Jj1tcP03SmpLZYhpTY1BVVl7nrHX1jDKuNr3fmO4Ovv36sRx0y4nsfvkhHHTLiXz79WMZ013fo03V9mtDy4b+TYf+dsmaeSfPvfPQ1/7HT/c8ee6dh/52yZp5G/o3HdrEbs8Cfh8RBwD/BBxCMmVi+swd74yIg0iC7ofSGQnL7QNcHBH7AauAN9d78Hr+Bz8XESuADkkdEXEHybcEMxtAJdnojiF5Xv9USdPLNnsXsDIi9gYuBM4vW38hW57Hb1xxwFWpkkFVWQE1a12tUcbFlvbxl8xnxvl3cPwl83l0yWoKhWBC4Vkm/HDr2X8m/HA2Ewq1Z2XN2q8NLctXb7zg/VfdvXPpF733X3X3zstXb7xgAA9zV8m0iJAE3/uBX5G0jPepUOYPEXFf+vpuYM96D1ZPIF4laRzwc+C7ki4ieZ7YzAbWIcCiiHgsIjYC1wLHlW1zHHB5+vp64HUlM8O8iSSZ/cNN16RGXuGsgJq1rlae46zWtDZVbqUXJ1DIkjkzkw0p/YXCrpW+6PUXCrsO4GHWFl9IOhw4Ejg0Il5OkgJzhwplNpS83kR9eTqgzg3/D5gAfBh4K7AjcG69BzCzutWTjW7zNhHRL+lZYJKk9cDHSXK7V+2WrjsbXUcHhZ592fSOW7YecJUO1Kr12E61dbXyHGd2XT+v8TzUTrwxfHR1dDw9deLoPcsH7XV1dDydUayW0ukSy+1I0gu1TtJLSG7VDqh6ArFInut9huQb+vfSrmozG1j1ZJSrts2/ARdGxJq0gVxRvdnoCoWoOuCqo0M1A2rWuqppKKkxRV+xlV4+kruO2X+amTbQ2svO47s/+rW3HjSv2D09deJovvbWg5bvPL77o43uMyJWSJov6SFgPbCkZPX/Au+T9ADwKEn39IDanseX9ieZMPnNwOKIOHKgK9MMP75kbaCpRzHqmShF0k3pNndK6gL+DPSQ9FwVs9FNAArA2RHx1WrHy7pmWvWoUM1nhRt8tjnPZ5CtKQ19+DmMmm6puvuwSR5d+jOwApicT3XMRrSa2eiAecBs4E7gBOD2SL5Nv6a4gaRzgDVZQbiWVnXl1pyir8E81M1MG2jtZ1RX551TJo7+21bXY6DU8xzx+0lawj0kg0PeUzansJkNgPSe7xkkt4KK2egeLs1GB3wLuFLSIpLbRafkUZdWduVmdV23437NmlVPi3gP4MySYdlmlpOIuBG4sWzZ2SWv/wqcWGMf5zRbD+dQNhs8NQNxRGyTVMDMhjd35ZoNnu25R2xmI4i7cs0GR3254czMzCwXDsRmZjaiSZog6fQGy54paUwzx3cgNjOzkW4C0FAgBs4EmgrEvkdsZmZDS/+GQ1mz9AIK/bvS0fU04yZ/lK5RAzUN4i0keTNOAkYBN0TEpyWNBa4DppI8XvgZYBeSKYLvkLQ8Io5o5OAOxGZmNnT0bziUpY/M47q37ZymOt2Tk66cx+R9ZzURjM8CXhoRB0g6miRZziEkmb/mSTqMJJfGUxHxBgBJO0bEs5I+AhwREcsbPSV3TZuZ2dCxZukFm4MwJHnHr3vbzqxZOlDTIB6d/twL3AO8hGTawweBIyWdL+k1EVF7/s06uUVsZmZDR6F/10rTYVLoH6hpEAV8ISIu3WaFdBDweuALkm6OiAGZiTDXFrGkmZIelbRI0jaJQSQdJukeSf2STihbt0nSfenPvDzraWZmQ0RH19Ob58kumjAtWd640mkQbwLeKWkcgKQpkiZL2g1YFxFXAV8EXlGhbENyC8SSOoGLgWOA6cCpkqaXbfYE8Hbg6gq7WB8RB6Q/s/Kqp5mZDSHjJn+Uk65cvjkYT5gGJ125nHGTm5oGEShOg3gUSUy6U9KDJHMsjAdeBtyVDuj6JPDZtPhc4CeS7mj0+Hl2TR8CLIqIxwAkXQscB2yeMCIiHk/XFXKsh5mZDRddo+5k8r6zePuNAzlqmogon+nsorL3vydpLZeX+wrwlWaOnWcgngI8WfJ+MfDK7Si/g6Q+oB84LyJ+MJCVMzOzIapr1J1M2H3kTIPYhErZ4WM7yk+LiKckvQC4XdKDEfH7rQ4gzQHmAEybNq3SPszMzNpanoO1FgO7l7yfCjxVb+GIeCr9/RjwU+DACtvMjYjeiOjt6elprrZmZmYtkGcgXgDsI2kvSd0kE5jXNfpZ0kRJo9LXOwMzKLm3bGZmNlzkFogjoh84g+Tm9iPAdRHxsKRzJc0CkHSwpMUkE51fKunhtPi+QJ+k+4E7SO4ROxCbmdmwk2tCj4i4EbixbNnZJa8XkHRZl5f7JclQcTMzs2HNKS7NzMxayIHYzMyshRyIzdpIHWlhR0n6Xrr+15L2TJcfUpIS9n5Jxw923c2sMQ7EZm2izrSw7wJWRsTewIXA+enyh4DeiDgAmEky+NGTupgNAQ7EZu1jc1rYiNgIFNPCljoOuDx9fT3wOkmKiHXpkwoAO7B9yXPMrIUciM3aR6W0sFOqbZMG3meBSQCSXpk+Avgg8L6SwLyZpDmS+iT1LVu2LIdTMLPt5UBs1j7qSQtbdZuI+HVE7AccDHxC0g7bbOhsdGZtx4HYrH3UkxZ28zbpPeAdgWdKN4iIR4C1wEtzq6mZDRgHYrP2UU9a2HnA7PT1CcDtERFpmS4ASXsALwYeH5xqm1kzPKrSrBLO+fcAAAqHSURBVE1ERL+kYlrYTuCyYlpYoC8i5gHfAq6UtIikJXxKWvzVwFmSngMKwOkRsXzwz8LMtpcDsVkbqSMt7F9JcrOXl7sSuDL3CprZgHPXtJmZWQs5EJuZmbWQA7GZmVkLORCbmZm1kAOxmZlZCzkQm5mZtZADsZmZWQs5EJuZmbWQA7GZmVkL5RqIJc2U9KikRZLOqrD+MEn3SOqXdELZutmSfpf+zC4va2ZmNhzkluJSUidwMXAUyYwxCyTNi4iFJZs9Abwd+FhZ2Z2ATwO9JFO83Z2WXZlXfW0QFAqwbhn0b4SubhjTAx11fhdstGwrjmlmth3yzDV9CLAoIh4DkHQtcBywORBHxOPpukJZ2b8DbomIZ9L1twAzgWtyrK81KytwFQqwdCFceyqsegImTINTroHJ02sHt0bLtuKYZmbbKc+/KFOAJ0veL06XDVhZSXMk9UnqW7ZsWcMVtQFQDFzfPBK+/NLk99KFyXJIAnQxqEHy+9pTk+XF8muWwKonk9+Fku9mtcpW02i5ZsuamW2HPAOxKiyLgSwbEXMjojcient6erarckNCVnBqpmyt/TZy3FqBq3/jlnVFq55IltcK4llls+pbq1yWZsqamW2HPAPxYmD3kvdTgacGoWxjmgl6edUnKzjVCrTVytaz30aOWytwdXUn3bulJkxLltcK4llls+qbVa70s6p0PvWUNTMbAHkG4gXAPpL2ktRNMoH5vDrL3gQcLWmipInA0emyfNQKPs3uu5EAnxWcmukGrhX0Gj1urcA1pie5x1rcpnjPdUxP7SCeVTarvlnliv821c6nVlkzswGS22CtiOiXdAZJAO0ELouIhyWdC/RFxDxJBwM3ABOBN0r6t4jYLyKekfQZkmAOcG5x4FZTqg0mqvbH/N23wrhdmjteowN+soJTrfrWCmxZ6xo9bjFwlZ9rMXB1dCTn/e5bt/38i0G89LilQTyrbFZ9s8pB7c8xq6yZ2QDJc9Q0EXEjcGPZsrNLXi8g6XauVPYy4LIBq0xWUGz2fmAeAT4rONXbDVwtsGWta/S4tYIeJK8rnXetIJ5Vtp4gXu2zrvU5ZpU1MxsgI+frfVYXZq1u1UbvxzYT4LO6RpvpBq7V5drMcYuBa8Luye96W4+lQfzMh5Lf9T4m1EwXsu8Dm1kbUES9A5nbW29vb/T19VXfYNWTSaAsd+ZD8Pwp1VvLkN29vGZJEnzLW2TvvjV5XW1dsaVV69nbSuvq6fJuZL9FzRy3FZpJ9jGw51NptH/bqnnNmOVvSF0zeRk5gTgrYI7bpfof81rlGg3wzQa2VmV9Gm7Zpgb2fJr+oyJpJnARybiKb0bEeWXrRwFXAAcBK4CTI+JxSUcB5wHdwEbgnyLi9qxjORBbG3AgJud7xG2lnsFEle4HNnM/ttnBQlladf9yuN03baPzqTMt7LuAlRGxt6RTgPOBk4HlwBsj4ilJLyUZJFlvAh0za6GRE4jrGUxUSa3BQI0GeHDSCCtXMy1s+v6c9PX1wFclKSLuLdnmYWAHSaMiYkP+1TazZoycQAyNtX6aeSynllpB3kaaSqldX1ltm/QRwWeBSSQt4qI3A/dWCsKS5gBzAKZNm1a+2sxaYGQF4kY081hOLfU8tmMjST2pXTO3kbQfSXf10ZUOEBFzgbmQ3CNurJpmNpAciOuR133EZlrTNhzVk9q1uM1iSV3AjkBxlrKpJAlyTouI3+dfXTMbCP6L32qNPntrw1E9aWHnAbPT1ycAt0dESJoA/A/wiYiYP2g1NrOm+a++WZuIiH6gmBb2EeC6YlpYSbPSzb4FTJK0CPgIcFa6/Axgb+BTku5LfyYP8imYWQPcNW3WRupIC/tX4MQK5T4LfDb3CprZgHOL2MzMrIWGTWYtScuAP9a5+c5s/bhHO3Cd6tPOdVoeETNbXZl6bcc1086feTtxnepTWqchdc3kZdgE4u0hqS8ieltdj1KuU31cp8HXjufnOtXHdRoa3DVtZmbWQg7EZmZmLTRSA/HcVlegAtepPq7T4GvH83Od6uM6DQEj8h6xmZlZuxipLWIzM7O24EBsZmbWQiMqEEuaKelRSYsknVW7RP4kPS7pwTQlYV+L6nCZpKWSHipZtpOkWyT9Lv09sQ3qdI6kP5WkcHz9INdpd0l3SHpE0sOSPpwub+lnlSdfM1Xr4GumvjqNuGumESMmEEvqBC4GjgGmA6dKmt7aWm12REQc0MJn674DlD9UfxZwW0TsA9zGlpzGrawTwIXpZ3VAmg5yMPUDH42IfYFXAR9I/w+1+rPKha+ZTN/B10w9RtQ106gRE4iBQ4BFEfFYRGwErgWOa3Gd2kJE/B/pVHoljgMuT19fDrypDerUUhHxdETck75eTTIxwxRa/FnlyNdMFb5m6jMCr5mGjKRAPAV4suT94nRZqwVws6S7Jc1pdWVK7BIRT0NyMQHtMpPPGZIeSLvhWtadJWlP4EDg17TvZ9UsXzPbp13/H/iaaXMjKRCrwrJ2eHZrRkS8gqT77wOSDmt1hdrY14AXAgcATwMXtKISksYB/wWcGRF/aUUdBomvmaHP18wQMJIC8WJg95L3U4GnWlSXzSLiqfT3UuAGku7AdrBE0q4A6e+lLa4PEbEkIjZFRAH4Bi34rCQ9j+QPyncj4r/TxW33WQ0QXzPbp+3+H/iaGRpGUiBeAOwjaS9J3cApwLxWVkjSWEnji6+Bo4GHsksNmnnA7PT1bOCHLawLsPmCLTqeQf6sJAn4FvBIRHypZFXbfVYDxNfM9mm7/we+ZoaGEZVZKx26/2WgE7gsIj7X4vq8gOQbPUAXcHUr6iTpGuBwkunJlgCfBn4AXAdMA54AToyIQRsIUqVOh5N0sQXwOPDe4n2mQarTq4GfAw8ChXTxv5Dc82rZZ5UnXzNV6+Frpr46jbhrphEjKhCbmZm1m5HUNW1mZtZ2HIjNzMxayIHYzMyshRyIzczMWsiB2MzMrIUciK1ukg6X9ONW18NsqPA1Y/VwIDYzM2shB+JhSNJbJd2Vzj96qaROSWskXSDpHkm3SepJtz1A0q/SpPA3FJPCS9pb0q2S7k/LvDDd/ThJ10v6jaTvpplzzIY0XzPWSg7Ew4ykfYGTSRLjHwBsAv4BGAvckybL/xlJ1h2AK4CPR8T+JNlvisu/C1wcES8H/pYkYTwks6ecSTI/7QuAGbmflFmOfM1Yq3W1ugI24F4HHAQsSL94jyZJqF4AvpducxXw35J2BCZExM/S5ZcD309z+U6JiBsAIuKvAOn+7oqIxen7+4A9gV/kf1pmufE1Yy3lQDz8CLg8Ij6x1ULpU2XbZeU2zeo621DyehP+P2RDn68Zayl3TQ8/twEnSJoMIGknSXuQ/FufkG7zFuAXEfEssFLSa9LlbwN+ls4XuljSm9J9jJI0ZlDPwmzw+JqxlvI3s2EmIhZK+lfgZkkdwHPAB4C1wH6S7gaeJbknBskUZF9P/2g8BrwjXf424FJJ56b7OHEQT8Ns0PiasVbz7EsjhKQ1ETGu1fUwGyp8zdhgcde0mZlZC7lFbGZm1kJuEZuZmbWQA7GZmVkLORCbmZm1kAOxmZlZCzkQm5mZtdD/B97FjZ1ZIZHoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 491.375x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "output2, _ = track_model_metrics(model=model2, train_interactions=train_interactions2, \n",
    "                              test_interactions=test_interactions2, k=K,\n",
    "                              no_epochs=NO_EPOCHS, no_threads=NO_THREADS, \n",
    "                              item_features=item_features,\n",
    "                              user_features=user_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These show slightly different behaviour with the two approaches, the reader can then tune the hyperparameters to improve the model fitting process.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Performance comparison"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition, the model's performance metrics (based on the test dataset) can be plotted together to facilitate easier comparison as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZwcdZn48c/T3dNzH8lkch+Ti5AEQo4hB0IEYiAgJMghQeRQVtwDV5efq+i6AgFBRVfdXXYVBUVAQRExsigEEMJNJpCEnOQmkzszyWQmc3c/vz+qJvTM1CTVM9Pdczzv16tf0/2tb1U9XT1dT9f3W/UtUVWMMcaY1gKpDsAYY0z3ZAnCGGOMJ0sQxhhjPFmCMMYY48kShDHGGE+WIIwxxniyBGF8EZF1InLuSeqMFJFqEQkmKaxuTUSuFZHnUx2HH34+X9P3iF0H0bOJyA5gEBABjgHPAl9S1epUxmUMgIjcCPydqp7dBcva4S7rhc4uq9Vy7wDGqepnu3K5vYEdQfQOl6pqDjAdOBP4VusK4rDPO0lEJJTqGIzpNFW1Rw9+ADuAT8S8vg94xn3+MvAd4HWgFhgH5AMPAnuB3cDdQDBm/i8AG4AqYD0wvfV6gJlAKXAU2A/8h1teDCgQcl8PBZYCFcAW4Asx67kD+B3wa3dd64CSE7zPycAyd1n7gW+65enAj4E97uPHQLo77VygDPgacMB9z5cBFwMfuMv6ZquYngSecGN6FzgjZvptwNaYbfOpmGk3utv5R+5y73bLXnOnizvtAFAJrAFOc6flu9vhILATJ8EHYpb7GvAD4DCwHbjoBNtJcX4NN7/+FXC3+3wA8AxwxI3x1Zj1xH6+J/xscH6IvOdO+727ve72iGUiUIdzdFsNHIn5zH4AfOh+lj8FMk8UI/AIEMX5P64GvuaxvhO9v6HAH9xtvB34Z7d8AdAANLrLXZ3q73R3eqQ8AHt08gNs+cUe4X6Z73Jfv+x+CScDISANeBr4GZANDATeAb7o1r8KJ2mc6e7QxgGjPNbzJnCd+zwHmO0+L6ZlgngF+B8gA5jqfjnnudPucHceFwNB4F7grXbeYy7Ozv3/ucvKBWa505YAb7nvpQh4I+b9nws0Ad923/sX3Bh+4y5jshvDmJiYGoEr3fpfdXcmaTHbZ6i7w7oap0lviDvtRnddX3K3dSYtE8SFwEqgwN22E2Pm/TXwJzemYpzkdVPMchvd2IPAP+AkQmlnW50oQdyLszNOcx/n8FEzc+zn2+5nA4RxktiX3WVcjrODbZMgYuJ/rVXZj3F+OPR33/OfgXvjibGddXnO635eK93/gzAwBtgGXBjzfh9N9Xe5Oz5SHoA9OvkBOl+aapxfTTtxdsjNv8ZeBpbE1B0E1DdPd8uuAf7mPn8O+PIJ1tO8A1kO3AkMaFWn2N1BhXCSVQTIjZl+L/Ar9/kdwAsx0yYBte2s+xrgvXambQUujnl9IbDDfX4uzi/OoPs6141vVkz9lcBlMTG9FTMtgJOYzmln3auARe7zG4EPW00/vnMEzsfZ8c/G/VXrlgfdz2RSTNkXgZdjlrElZlqW+x4GtxPTiRLEEpxENM5jvtjPt93PBpiL8yNCYqa/hs8EgbPDPgaMjSmbA2yPJ8Z21uU5LzDL47P5BvDLmPdrCcLjYW3SvcNlqlqgqqNU9R9VtTZm2q6Y56NwflntFZEjInIE52hioDt9BM4O92RuAk4BNorIChG5xKPOUKBCVatiynYCw2Je74t5XgNktNN2f6K4hrrLjV3H0JjX5aoacZ83b5f9MdNrcY6Cmh3fXqoaxWmiGgogIteLyKqYbXcaTrNGm3lbU9WXgP8G7gf2i8gDIpLnzt/8qzz2PXhuJ1WtcZ/GxuzXfThNfc+LyDYRue0Eddv7bIYCu9Xds7rafd8einCS3MqY7fhXtzzeGFtrb95RwNDm9bnr/CbODyZzApYger/WX+R6nF/+Be4jT1Unx0wfe9IFqm5W1WtwEsv3gCdFJLtVtT1AfxHJjSkbifPrM14nimsPzg4gdh17OrCOZiOan7id+sOBPSIyCvg5cAtQqKoFwFqcX8TNTnhKoKr+p6rOwGnaOgX4V+AQThNS6/fQke0Ezs48K+b14Jj1V6nq/1PVMcClwK0iMi/O5e8FholI7Pse0V5l2m6TQzhJeXLM/2C+OidZnCzGk23f9ubdhXOEUhDzyFXVi/0sty+zBNGHqOpe4HnghyKSJyIBERkrIh93q/wC+KqIzHDPehrn7hhbEJHPikiR+wv7iFscia2jqrtw+gPuFZEMEZmCc+TxWAdCfwYYLCJfEZF0EckVkVnutN8C3xKRIhEZgNPO/GgH1tFshohc7v5a/gpOQn0Lp89GcfowEJHP4RxB+CIiZ4rILBFJw2liqQMi7tHN74DvuO9rFHBrJ97DKuAzIhIUkQVA82eLiFzifqaCc4JBhFafmw9vuvPcIiIhEVmEc9JCe/YDw0UkDMePyn4O/EhEBrpxDRORC33EuB+n/8DTCeZ9BzgqIl8XkUx325wmImfGLLfYzvJryzZI33M9TpPGepyzYp4EhgCo6u9xznr6Dc4ZKk/jdCS2tgBYJyLVwE+Axapa51HvGpx+iT3AH4HbVXVZvAG7zVTzcX4V7gM2A+e5k+/GOaNqDfA+zplHd8e7jhh/wumAPgxcB1yuqo2quh74Ic4Ocj9wOs5ZS37l4ewYD+M0IZXjnMkDTsf2MZyO09dwtv9DHYz/yzjb6QhwLc5n2Gw88AJOn9WbwP+o6svxLFxVG3A6pm9y1/FZnARe384sL+GcOLFPRA65ZV/HaQp6S0SOujFN8BHjvTg/Bo6IyFc91uU5r5uEL8U5UWI7zlHML3DOHgPnTCyAchF51+em6BPsQjljXHbBVMeIyNvAT1X1l6mOxXQtO4IwxsRFRD4uIoPdJqYbgCk4Hc2ml7GrPY0x8ZqA02+Sg3N22ZVu/5bpZayJyRhjjCdrYjLGGOOp1zQxDRgwQIuLi1MdhjHG9CgrV648pKpFXtN6TYIoLi6mtLQ01WEYY0yPIiI725tmTUzGGGM8WYIwxhjjyRKEMcYYT72mD8IY07c1NjZSVlZGXZ3XqC8mIyOD4cOHk5aW5nseSxDGmF6hrKyM3NxciouLaTnYrFFVysvLKSsrY/To0b7nsyYmY0yvUFdXR2FhoSUHDyJCYWFh3EdXliCMMb2GJYf2dWTbWIIwxhjjyRKEMcYkwB133MEPfvCDdqc//fTTrF+/PokRxc8ShDHGpIAlCGOM6UO+853vMGHCBD7xiU+wadMmAH7+859z5plncsYZZ3DFFVdQU1PDG2+8wdKlS/nXf/1Xpk6dytatWz3rpZolCGOM6QIrV67k8ccf57333uOpp55ixYoVAFx++eWsWLGC1atXM3HiRB588EHOOussFi5cyH333ceqVasYO3asZ71Us+sgjDGmC7z66qt86lOfIisrC4CFCxcCsHbtWr71rW9x5MgRqqurufDCCz3n91svmSxBGGNMF/E6lfTGG2/k6aef5owzzuBXv/oVL7/8sue8fuslkzUxGWNMF5g7dy5//OMfqa2tpaqqij//+c8AVFVVMWTIEBobG3nssceO18/NzaWqqur46/bqpZIlCGOM6QLTp0/n6quvZurUqVxxxRWcc845ANx1113MmjWL+fPnc+qppx6vv3jxYu677z6mTZvG1q1b262XSgm9J7WILAB+AgSBX6jqd1tNnwv8GJgCLFbVJ2Om/RWYDbymqpecbF0lJSVqNwwypu/asGEDEydOTHUY3ZrXNhKRlapa4lU/YUcQIhIE7gcuAiYB14jIpFbVPgRuBH7jsYj7gOsSFZ8xxpgTS2QT00xgi6puU9UG4HFgUWwFVd2hqmuAaOuZVfVFoKp1uTHGmORIZIIYBuyKeV3mlnUZEblZREpFpPTgwYNduWhjjOnzEpkgvIYO7NIOD1V9QFVLVLWkqKioKxdtjDF9XiITRBkwIub1cGBPAtdnjDGmCyUyQawAxovIaBEJA4uBpQlcnzHGmC6UsAShqk3ALcBzwAbgd6q6TkSWiMhCABE5U0TKgKuAn4nIuub5ReRV4PfAPBEpE5HUX3dujDFJUlxczKFDh3zX+fznP8/AgQM57bTTuiyGhA61oarPAs+2Kvt2zPMVOE1PXvOek8jYjDGmN7nxxhu55ZZbuP7667tsmXYltTGmT3r7wHa+8c7TfPHV3/CNd57m7QPbO73MHTt2cOqpp/J3f/d3nHbaaVx77bW88MILfOxjH2P8+PG88847VFRUcNlllzFlyhRmz57NmjVrACgvL+eCCy5g2rRpfPGLXyT2IuZHH32UmTNnMnXqVL74xS8SiUTarHvu3Ln079+/0+8hliUIY0yf8/aB7Ty6+R0q6p17LlTU1/Do5ne6JEls2bKFL3/5y6xZs4aNGzfym9/8htdee40f/OAH3HPPPdx+++1MmzaNNWvWcM899xz/xX/nnXdy9tln895777Fw4UI+/PBDwLn6+YknnuD1119n1apVBIPBpI3VZKO5GmP6nKd3rKYh2vJXeEM0wtM7VjNr4OhOLXv06NGcfvrpAEyePJl58+YhIpx++uns2LGDnTt38oc//AGA888/n/LyciorK1m+fDlPPfUUAJ/85Cfp168fAC+++CIrV67kzDPPBKC2tpaBAwd2Kka/LEEYY/qc5iMHv+XxSE9PP/48EAgcfx0IBGhqaiIUarvbbR4m3Gu4cFXlhhtu4N577+10bPGyJiZjTJ/TPz0rrvKuNHfu3ONNRC+//DIDBgwgLy+vRflf/vIXDh8+DMC8efN48sknOXDgAAAVFRXs3Lkz4XGCJQhjTB90WfEZhAPBFmXhQJDLis9I+LrvuOMOSktLmTJlCrfddhsPP/wwALfffjvLly9n+vTpPP/884wcORKASZMmcffdd3PBBRcwZcoU5s+fz969e9ss95prrmHOnDls2rSJ4cOHd8ktSxM63Hcy2XDfxvRt8Q73/faB7Ty9YzUV9TX0T8/isuIzOt3/0N3FO9y39UEYY/qkWQNH9/qE0FnWxGSMMcaTJQhjjDGeLEEYY4zxZAnCGGOMJ0sQxhhjPFmCMMaYbiie4b537drFeeedx8SJE5k8eTI/+clPuiQGO83VGGN6uFAoxA9/+EOmT59OVVUVM2bMYP78+UyaNKlTy7UjCGNMnxTd8CaRn3+NyH/cROTnXyO64c1OLzNVw30PGTKE6dOnA5Cbm8vEiRPZvXt3p9+PJQhjTJ8T3fAmuuzXUFXuFFSVo8t+3SVJItXDfe/YsYP33nuPWbNmdfq9WBOTMabP0df+CE0NLQubGpzyiXM6texUDvddXV3NFVdcwY9//GPy8vI69T7AEoQxpi9qPnLwWx6HVA333djYyBVXXMG1117L5Zdf3pm3cJw1MRlj+p7cwvjKu1AihvtWVW666SYmTpzIrbfe2mWxJjRBiMgCEdkkIltE5DaP6XNF5F0RaRKRK1tNu0FENruPGxIZpzGmb5GzPwWhcMvCUNgpT7BEDPf9+uuv88gjj/DSSy8xdepUpk6dyrPPPtvpWBM23LeIBIEPgPlAGbACuEZV18fUKQbygK8CS1X1Sbe8P1AKlAAKrARmqOrh9tZnw30b07fFO9x3dMObTp9DVTnkFiJnf4pAJ/sfurvuNNz3TGCLqm5zg3gcWAQcTxCqusOdFm0174XAMlWtcKcvAxYAv01gvMaYPiQwcU6nO6R7u0Q2MQ0DdsW8LnPLumxeEblZREpFpPTgwYMdDtQYY0xbiUwQbbvjneaiLptXVR9Q1RJVLSkqKoorOGNM19D6GrSxPtVhANBb7pCZCB3ZNolMEGXAiJjXw4E9SZjXGJMEWldDdMdaos/8FF32MHp4H9rqCt9kysjIoLy83JKEB1WlvLycjIyMuOZLZB/ECmC8iIwGdgOLgc/4nPc54B4R6ee+vgD4RteHaIzpKN2/HX3qR85zQLeuInDjdyC334ln7Kr1NzVCzVF022rIzGHY4LHsrjhKUpubo1Gcdy9Ou4ck+cqBONafkZHB8OHD41p8whKEqjaJyC04O/sg8JCqrhORJUCpqi4VkTOBPwL9gEtF5E5VnayqFSJyF06SAVjS3GFtjEk9ra9FVz7fsrCxHt21EZmUpI7fo4eIPnonNDUCECwYRPHVX0ey85Oyej1WSfRP/wX7tgOCTD0PmbMQycxNzfrPOBc56zIkM6fL1pHQK6lV9Vng2VZl3455vgKn+chr3oeAhxIZnzGmgwJBJDO3bcdgVpJ2jo316JtLjycHAI7sR/fvQMackfj1R5rQVS+5O2cARVe9hEycA0lIEBqNoKv/1nL9q/+GTDoLujBB2JXUxvRxWhd/J7OkhZHZl0I4pk17wHCkaGQXR9cOVbShrm15fW1y1t/UgO7d2qZYD36YnPU3NqB7PNZ/YKdH5Y6zsZiM6aO07hi6Zwu6chlkZhM46zLIL0KCPncLeQMI3Hg3WrYZMnOQAcOR7M4PEOeHhDMIzLyY6PY1HxWmZyIjJiRl/YQzkHEz0A83tIxreLLWn46Mn45+uL5FcVev3xKEMX2U7t2GPv2fx19Ht60h8Ll7fHcySzAIOf2QU2cmKsQTGzCcwDXfJFr6PJKVi5QsgKwkJSgJwCklUL4HXbscwpnIxz8NSer/EAnAeHf977vrn/tpyC7o2vX0llPCbKgNY/zT+hqiz/wUdq5rUS4XfYHAxNkpiqpjtKkBJOD/yKcr191YD81NXRk5TtLsYetP1VAbxpjuKhBCsvJS1sncTBvqnB2cCGRkd2gnL60H3UsiSUuHtPSTV+yh67cEYUwfJGlhmLMQ3boKGtyO3YEjkQEjTjxjF9KaKvS1p9D1rztt+nM/DeOmIxlZSYvBnJglCGP6qtxCp5N5zxbn13vhsKR1Mms0im54y2m/B6g7hj7/S2ToWLAE0W1YgjCmj3I6mQuQUzybnxOrsQ7dtqpNse7ejPQfkvx4jCdLEMakiDbWQ1UFumY55OQjp85Gcrr2LJRuKxRGho5Dd21sUSwDk3QdBc3b/7BzFlBWHjJxNmTne972s6+yBGFMqlTsI/qbu0Gd26Hou8sIXPvtpA0VkUoSDMHU853rCPZuBRFk+gWQNyB5QRzeT/Sxu1pt/3+HvpKkfbAEYfosjUagpsoZniEjG/oNQuI8j17rjkFjvTNIWnqW0/nrZ76GOqJvLj2+cwKg+gi6dxsyblpcMfRUkp1PYNGXnO0XCDoXf6Unp/9BG+uJvvVMy+1/7Ai6Z0tqmty6KUsQJqU0GoXaandsn+zkrvxoOdHHlkB9rXO65+AxBC77ku8koTVHiT73S9i+xmkyOWsRnHaOk2z8LcGjqHdcl+SXZOUCyT219oT62PY/GRuLyaSM1laja14m+vvvE136X+i+7c5FT8lYd2M9+uafWo7ds28bemi3v/mjEXT1K05yAGdsnuW/h+ojvuaXcAaBOQud8/+b5RQ4Z/GYhJO0dAKzL2k5PHZ2PjL8lNQF1Q3ZEYRJCVVFt7+PvvTY8bLoE991hnrIK/S/nGOVcOwopIWdUzX9jmQZjaDVlW3La476m7+hDv1wXZti3b8dGeDzzrr9BhO4/i50zcvOzmnSnLj7H1pcSZuZgwSSeyVvj9ZvEIEblqBrXnE6qSedlbShOnoKSxAmNepr0LWvtiyLNDltwD4ThB6tIPrEvVDl3ipk/AwC865zmy1OTNKzkGnz0F0xg62Fwsiw8f7iD2cgIyejuze3XO6g0f7mxzmKoHAIct41vueJpTVV6NvPOGfhZGQh514DoyYj6ZkdWl5fI2np0H8Icu7iVIfSbVkTk0mNUBqS3/Y+4pLrMzk0NaDvPPNRcgDYvBIqD/gOQYafglzyDzBsPIyd5pzB4vMXpASCyBkfh+Z7D4TCyMevTtoZMKpRdNM76HsvQFOD08H9zP/6PwIyKafRCFp9hOjmd4nu2oh2w8/OjiA6SSMRqDsGgUCX3smpt5NQGOZcim5f89FOrfg06DfI3wKaGtGKvW2K9fABZIi/dnzJyEZOKUFHnuqMTRSO7369kpVHYMFNHTqLqdPq69DNK9sU6+7NiN9taFKrqoLoo0ugvgYALRpB4PJbk3Y1ux+WIDpBa6vR9W+gq1+GrFwC5y6GAcNSOnhYj5JbSOCzdzi/+tOznHZgv4PFpWchk85Cyz74qEwCyHCfTUQxJKPjiV0yssH3WUtdKBRGBhWjZZtaxjMgvnsOm9TQpgb07WeOJwcADu5CD+xERp+eusBasSamDlJVdOsq9JUn4Mh+2LOF6BPfhZrqVIfWY4gIkpOPDBuPDBjmPzk0zzt2KnL2FZDbH4pGErjqq0m53WN3IKEQMuMCKGoeXE+QM86DfP8d/CaFolHnBIvWvMpSyI4gOqr+mHcn694tSF6KbqDSx0hmLsy4AJn8MefoIclDVaea5BQQuOJWaKiHYBDSMmwk1B5CwhnI9Pno9vc/KgyFkVGTUheUh4QeQYjIAhHZJCJbROQ2j+npIvKEO/1tESl2y8Mi8ksReV9EVovIuYmMs0OCaUhB27ZeSeZQAQYJhpDs/D6XHJpJVh5SUITk9rfk0MPIoGLksn+GERNg3PS4TpJIloQdQYhIELgfmA+UAStEZKmqxt5E9SbgsKqOE5HFwPeAq4EvAKjq6SIyEPiLiJypGntdfGpJWjqctRDd8f5HnaxjpoLHmTndmUYj7pXMAecXuTEmKSQjGxlzBjp0nDOSQJwnSSRDIpuYZgJbVHUbgIg8DiwCYhPEIuAO9/mTwH+LM5TiJOBFAFU9ICJHgBLgnQTGG7/cQgLX3QFHyyGcAVm5PWon63Syv46u+htk5hA4dzEUjUzemTjGmDiGZkm+RDYxDQN2xbwuc8s866hqE1AJFAKrgUUiEhKR0cAMIHm3uvJJRJzmjSFjkMKhPSs5HO9k/x1UHoR924n+7vtQW5Xq0Iwx3UQijyC8BlVvPRJWe3UeAiYCpcBO4A2gqc0KRG4GbgYYOTJ548j3CnU16LrXW5ZFI3FdyWyM6d0SeQRRRstf/cOBPe3VEZEQkA9UqGqTqv6Lqk5V1UVAAbC51byo6gOqWqKqJUVFHWv719oq9Gg5Wn3YuYF6XxFK87ygyuvqZmNM35TII4gVwHi3iWg3sBj4TKs6S4EbgDeBK4GXVFVFJAsQVT0mIvOBplad211Cj1USfeZ/YfdmCIaQ2QvhjHO7dZtgV5G0MMy+1Olkbx6BdGzP62Q3xiROwhKEqjaJyC3Ac0AQeEhV14nIEqBUVZcCDwKPiMgWoAIniQAMBJ4TkShOcrmuy+NrakRXPu8kB3CuYXj9KedmLX0gQQCQ25/Atd+GqsMQTndGA+1B/SjGmMRK6IVyqvos8Gyrsm/HPK8DrvKYbwcwIZGx0ViP7v6gTbEe3IUUDk3oqruS1tc6YwEBZOY6N6L3SUQgO995GGNMK333SupwBlJ8Grp3W4tiGVScknA6QmuOosufRDe+BemZznDPY6Yk7baNxpjerc+OxSTBkDN2zfgS565e6VnIBZ+DHnJFrkYi6NpX0fWvg3uxm/7l51Bjp6kaY7pG3z2CwB2u+YIb4LzFgEBGDhLqIZukoRbdtqZNse7bbsM9G2O6RJ89gmgm6VlITj8kp6DnJAeAtHTP+x7YcM/GmK7S5xNETyWhNKTkAhjs3uJSAsjMT0Juv9QGZozpNXrQT+beRyMRqD2K7tvh3I2u3yAkjtEcJbuAwGVfds5iCgSdjne7H7ExpotYgkilo4eIPrYEGuqcMUiGjCWw6Jb4kkRWLtAzOtaNMT2LNTGliDbWo288DbHDe+zdChX7UheUMcbEsASRKpEmz1sOak33uuWgMabvsgSRIpKRjUw9v2VhWjoyZFxqAjLGmFasDyKFZORE+OTfo6tehMxcAmdf3mMu1DPG9H6WIFJIMrKRCWeioyZCINQtbzlojOm7LEF0A5KRk+oQjDGmDUsQplP0WCVUH4a0DHe4cEt2xvQWJ00QIjIIuAcYqqoXicgkYI6qPpjw6Ey3pkfLiT5+z0c3HBo3ncAnrnevzTDG9HR+zmL6Fc5Nf5pvkvAB8JVEBWR6Bm1sQN965qPkALDlXag8kLqgjDFdyk+CGKCqvwOi4NwpDogkNCrT/UUa0SNtL+rTIwdTEIwxJhH8JIhjIlIIzmgQIjIbsKu5+rr0LGTSx1qWBYLIsPGpiccY0+X8dFLfCiwFxorI60ARcGVCozLdnojA2Kkw9yp09d8gI4fAudeA3dPamF7jpAlCVd8VkY/j3CNagE2q2pjwyEy3J5k5MH0+MnGOM9y4dU4b06v4OYvp+lZF00UEVf11gmIyPYgEgpCdn+owjDEJ4KcP4syYxznAHcBCPwsXkQUisklEtojIbR7T00XkCXf62yJS7JanicjDIvK+iGwQkW/4fD/GGGO6iJ8mpi/FvhaRfOCRk80nIkHgfmA+UAasEJGlqro+ptpNwGFVHScii4HvAVcDVwHpqnq6iGQB60Xkt6q6w+f7MsYY00kdGc21BvBzqspMYIuqblPVBuBxYFGrOouAh93nTwLzRERwzpjKFpEQkAk0AEc7EKsxxpgO8tMH8WfcU1xxEsok4Hc+lj0M2BXzugyY1V4dVW0SkUqgECdZLAL2AlnAv6hqhUdsNwM3A4wcOdJHSMYYY/zyc5rrD2KeNwE7VbXMx3ziUaY+68zEuRhvKNAPeFVEXlDVbS0qqj4APABQUlLSetnGGGM6wU8fxCsdXHYZMCLm9XBgTzt1ytzmpHygAvgM8Ff3dNoD7vUXJcA2jDHGJEW7fRAiUiUiRz0eVSLipz9gBTBeREaLSBhYjHPBXaylwA3u8yuBl1RVgQ+B88WRDcwGNsb75owxxnRcu0cQqtqpq57cPoVbcAb6CwIPqeo6EVkClKrqUuBB4BER2YJz5LDYnf1+4JfAWpxmqF+q6prOxGOMMSY+4vxg91FRZCBw/JZnqvphooLqiJKSEi0tLU11GMYY06OIyEpVLfGadtLTXEVkoYhsBrYDrwA7gL90aYTGGGO6HT/XQdyF0wfwgaqOBuYBryc0KmOMMZvCYycAABgUSURBVCnnJ0E0qmo5EBCRgKr+DZia4LiMMcakmJ/rII6ISA7wKvCYiBzAuR7CGGNML+bnCGI5UAB8GfgrsBW4NJFBGWOMST0/CUJwTlV9GcgBnnCbnIwxxvRiJ00Qqnqnqk4G/gln6ItXROSFhEdmjDEmpfz0QTQ7AOwDyoGBiQnHGGOMX5UNtVTW15IWDJIbSicnnHHymeLgZzTXf8C5R0MRziirX2h1TwdjjDFJdri+hu+vfp6K+hoAJvcbwudOmUNuFyYJP0cQo4CvqOqqLlurMcYYjjXWUx9pIiBCRjCNjFCar/maohGWlW04nhwA1h3ey56aSiYkM0GoaptbhRpjjOmcow21/HLTW6w/spegBFgwfBLnD5tATlr6SedtjEbYV9t2zNT9tUeZUDCoy2LsyB3ljDHGdEIkGuW1fVtZf2Sv81qj/N+utVTUHfM1f2YozFmDxrQoCyBMLBjSpXFagjDGpEx9U6PT0dpQSyQaTXU4SVMfbWJT5f425dur/V9BcGrBYK4eM4OijBxG5vTjX04/nzwfRx/xiOcsJmOM6TJVjXX8acca3ty/jYxQGleNns6UwmFkhcKpDs23mqYG6iNNCEJWKEw4GPQ1X0YwxGn9hrLxSMskMS6vyPe6c9LS+fiQ8cwoGklAhNy0rj2DCSxBGGNSIKpRVhzYyav7tgBQ3VjPLz94kztnXNJjEkRVQx2/3bqCdw/tIi0QZOGoKZw1aAzZPn7FByTA7IGj2VldQenBDwkHgywaNYWCcFZcMQQDAfLDmR19CydlCcIYk3R1kSZWlbe9tf3mygMMzspLQUTxiWiUNw9sY+WhXQA0RCM8uf09JvUb4itBAOSGM7h23JlcOXoaIGSnpZEW6F67ZOuDMMZ0WEMkQmVDLUcbaon6vPkYQDgQYkzegDblI3L6dWV4J9Xoxl/ZUEtU/feB1EeaWHd4b5vyrUcPxrX+zFCYgvQsCtIzu11yADuCMMZ0UHVjHc+XbWT53s1khcJ8euwMJuQPItPHufyhQIDzhp7ChiP72FFVjiCcP/QUBmRkxx1HfaSRgARIC/hr//8o/nqWlW3gFTf+q8ZM59SCwb7iTw+EODV/cJs+hOLcwrhi6O4sQRhj4hZV5d1Du3iuzBlUoTbSyE/XL2dJyaW+drAA+eFMbpn0ceqjTQQlQEYwRGYc/Q81TQ3sPnaEZWUbyElL56IRp9E/PYtg4OQNI6rKe4d28deY+H+24VXuLLnEV/zBQICPDR7LlqMHWXt4DyEJcNGIyfRPjz/BdWeWIIzpw6KqHGusJyDiu+0coC7SSOnBlrelV+CDygMMzMz1vZzccAb+a7dUVn2YH77/4vHXKw99yJ0zLqEg/eQdvbWRRkoP7WxRpsAHR/YzKNNfH0heOIPPT5hDfTRCAKe5KD3Yu3apCe2DEJEFIrJJRLaISJsrskUkXUSecKe/LSLFbvm1IrIq5hEVEbuLnTExGqJNHKqr5vld63lz/zYqG2rjmv9YYz1v7N/Kj95/if/d8Co7qsppiPi7F1g4EGSkR3/BsKz8uGLoqLqmRp4r29CyLNLEpiNtry3wEg6EGJXdv035sOz4+kCy09Lpn55FQXpWr0sOkMAEISJB4H7gImAScI2ITGpV7SbgsKqOA34EfA9AVR9T1amqOhW4DthhY0EZ09LB2mpuL32GP+xYxa8+eIvvr14WV5LYeGQfj2x+h901R9hceYDvr15GVWO9r3lDgSCfGD6RITEJ4axBYxiQmRP3++iIgIjn6bB+T5ENBQKcP+xUhsbEP2fgaIoykhN/T5HIlDcT2KKq2wBE5HFgERA7Euwi4A73+ZPAf4uIqLY4HeIa4LcJjNOYlKprcjpZ/V5kBU7H7J93vk9TzJk3h+qq2VlVwZTCYSedv7apgeXuNQjNIhrlg8r9zMkY085cLRWEM7n19HnURRoJSYD0YCiuZqrOCAdDXDLyNFaV76IhGgFgcGYeo3LbHhW0pyA9k385fR71kUaCSY6/p0hkghgG7Ip5XQbMaq+OqjaJSCVQCByKqXM1TiJpQ0RuBm4GGDlyZNdEbUyS1DQ18GF1BcvKNpKXlsHFI51OTn+drM6Aba01Rv01EYUkSFFGLhtp2SRTGGcna144gzy6/gpePwozsllScgnrDu8jJy2dMbmF5MV50VheOANSFH9PkMg+CPEoa32i9AnriMgsoEZV13qtQFUfUNUSVS0pKvJ/iboxXSWiUY421FHts2km1o6qcn70/kusPbyHNw5s4+73/kJVY52veTNCaVw0YnKLsuxQOmN9DtWQFgxy0YjJLYZnmJA/qEWTUXcXCgTpl57N2YPHMrVweNzJwZxcIo8gyoARMa+HA3vaqVMmIiEgH6iImb4Ya14y3VR1Yz2lB3fy8t4Pjp9HPyyrgLCPzsrapgaWlW1sUVYXaWJz5QHOHFjsa/3Dsgv45tQFvLh7E/nhDM4bdkpcO8n+6Vn8+/SLOFhbRUYwjfxwZpfebMb0fIlMECuA8SIyGtiNs7P/TKs6S4EbgDeBK4GXmvsfRCQAXAXMTWCMpg+rjzRSXlfD8r2byQ9nMGfQGPLDmYh4Hdi2te7wXn67tfT46/tWv8BdJZdS6CNBBCRAlsf59vGMQ5QZSmNUbn+uP2UmARECEl+DgIiQH85M6Fg+pmdLWIJw+xRuAZ4DgsBDqrpORJYApaq6FHgQeEREtuAcOSyOWcRcoKy5k9uYrra3ppLvrlqGuq2af9u7mX+btsDXDrOmqYHXPDp5N1ceoDBj9EnnTw+GuHTUFNZU7D7eyTo0K58ROf47WZuF4ryC2Bi/Enrirqo+CzzbquzbMc/rcI4SvOZ9GZidyPhM31XX1MgzH649nhzAuQH89qpyphYOP+n8IQkyMDOXDyoPtCiPZ6iIARnZLCm5lA1H9pGbls6onEK309SY7qH3XdlhepSqhroOD7XQ2fnF4xwJf41LEA4GuXjEaawp381Rt2N5UsFg31fhQnMna1abO4MZ011YgjApU9lQy/3rXmFndQWCMH/YqVw4YpKve/J6zf+JYRNYMGKyr/kzQmlcMvJ01lbsIeoeRfQLZ8U12Fr/9Cy+Nf0iDtVVkxFMIy+ckZCbthiTKpYgTEo0RSMsK9vAzmrnpDVFeX73BmYOLPa1g2+KRnihbGOL+Zft3sisgaN9J5jBmXncPuOTvL5/KwVpmZQMHBVXh6118prezhKESYn6SBPbq9ref3f3sSO+7glQH2liW9WhNuVlxw77vqdAeijE4FAeV4ye5qu+MX2N3TDIpERGKM2zM9jrJjKJmN8Yc3J2BGE65WhDHQ2RJoKB+DqJg+49effVHOXNA9vJCqXx6TEzyPXZPNTe/HnWB2BMlxGN4zaB3VlJSYmWlpaevKLpMkfqa/nvdS+z69hhBGHesAlc5LOTuFldUyP17vhBOaEwwTjP6e/s/Mb0dSKyUlVLvKbZEUSK1TU1Uh9pAoGcULqvgdq6g6ZohGW717Pr2GHA6SR+YfdGZsfRSQxOU1EG/u5Aloj5jTHtswSRQlUNdTy1fRVvHdxOdiidq8fO4LR+Q33fshHgWGMDDdEmBMhOC8d94/MW84fCpPm86YnTyVzRptxvJ7MxpvuzBJEikWiU1/dv5Y0DzkgiVY11/GLj69wVxz19jzbU8ejmt1lTsZtwMMTlxVOZObDY93g+RxvqeGzzO6yuKCMcDPGp4qnM8jl/RiiNaYXD2Xr0YIty6yQ2pvfoGe0ZvVBdpJHV5bvblG/3OHXTS1M0yit7N7O6YjeK84v+t1tLOdrgb7jopmiU5Xs3s6qi7Pj8j8cxf3Mn8TmDxxGSAHlpGXzh1LPtQjFjehE7gkiRcDDEmLzCNufyD8su8DV/XaSR9Yf3tinfWV3O4KyTD/dQH2lk/ZG28++o8jc/ODecv2rMNC4ddToAOWnpBOMcUdQY033ZtzlF0gJBLhg+6fiN3wMIC4ZPpF84y9f8GcE0JhQMalM+wuNG7F7Sg2mckt92/pFxjiaa7t5HID+cacnBmF7GjiBSKD+cyT9PPq9Dg82FAgHOH3oKO6oOseHIftICQRaOmuJ72Ifm+XdWlbP+yD5n/pGn27ARxpjj7DqIHq66sZ6GSBMBETJDYdJ9noXkPX8a6UE7ZdSYvsSug2hHQ6SJyoY6Vh7aSUE4i4n9Bve4X9A5aekQx3UHXT2/Mab36tMJYl/tUe5d9RxR9yhqUGYuX53yibju61vZUEt1Yz3hQJDMUDiui8SMMaY767MJorapgaU71hxPDgD7a6vYfeyI7wRRUXeM769ZxuH6GgBmFo3i6jEzyLG7ghljeoE+e9qJAo0aaVPeGG1b5qUh0sT/fbj2eHIAeOfgTg7VH+uqEI0xJqX6bILICoW5aMTkFmW5aRmM8nlHscZohL21lW3KD9ZWdUl8xhiTan22iQlgVE5/bpt6IX/bvYmC9CzOG3qK7+GiM0NhZhWNZuvRjy50C4gwJq8oUeEaY0xSJTRBiMgC4CdAEPiFqn631fR04NfADKAcuFpVd7jTpgA/A/KAKHCmqvobB8KnzFCY0bmFjDhlFgERAnFc6BUQYUbRCKob61m+bzM5aelcPbbE9/0MjDGmu0tYghCRIHA/MB8oA1aIyFJVXR9T7SbgsKqOE5HFwPeAq0UkBDwKXKeqq0WkEGhMVKyhDt5DICctgwtHTOTsIWMJIORa57QxphdJZB/ETGCLqm5T1QbgcWBRqzqLgIfd508C80REgAuANaq6GkBVy1U9epS7gVAgSH4405KDMabXSWSCGAbsinld5pZ51lHVJqASKAROAVREnhORd0Xka14rEJGbRaRUREoPHjzoVcUYY0wHJTJBiEdZ63E92qsTAs4GrnX/fkpE5rWpqPqAqpaoaklRkXUOG2NMV0pkgigDRsS8Hg7saa+O2++QD1S45a+o6iFVrQGeBaYnMFZjjDGtJDJBrADGi8hoEQkDi4GlreosBW5wn18JvKTO6IHPAVNEJMtNHB8H1mOMMSZpEnYWk6o2icgtODv7IPCQqq4TkSVAqaouBR4EHhGRLThHDovdeQ+LyH/gJBkFnlXV/0tUrMYYY9qy4b6NMaYPO9Fw3312qA1jjDEnZgnCGGOMJ0sQxhhjPFmCMMYY48kShDHGGE+WIIwxxniyBGGMMcaTJQhjjDGeLEEYY4zxZAnCGGOMJ0sQxhhjPFmCMMYY48kShDHGGE+WIIwxxniyBGGMMcaTJQhjjDGeLEEYY4zxZAnCGGOMJ0sQxhhjPFmCMMYY4ymhCUJEFojIJhHZIiK3eUxPF5En3Olvi0ixW14sIrUissp9/DSRcRpjjGkrlKgFi0gQuB+YD5QBK0Rkqaquj6l2E3BYVceJyGLge8DV7rStqjo1UfEZY4w5sUQeQcwEtqjqNlVtAB4HFrWqswh42H3+JDBPRCSBMRljjPEpkQliGLAr5nWZW+ZZR1WbgEqg0J02WkTeE5FXROQcrxWIyM0iUioipQcPHuza6I0xpo9LZILwOhJQn3X2AiNVdRpwK/AbEclrU1H1AVUtUdWSoqKiTgdsjDHmI4lMEGXAiJjXw4E97dURkRCQD1Soar2qlgOo6kpgK3BKAmM1xhjTSiITxApgvIiMFpEwsBhY2qrOUuAG9/mVwEuqqiJS5HZyIyJjgPHAtgTGaowxppWEncWkqk0icgvwHBAEHlLVdSKyBChV1aXAg8AjIrIFqMBJIgBzgSUi0gREgL9X1YpExWqMMaYtUW3dLdAzlZSUaGlpaarDMMaYHkVEVqpqidc0u5LaGGOMJ0sQxhhjPFmCMMYY48kShDHGGE+WIIwxxniyBGGMMcaTJQhjjDGeLEEYY4zxZAnCGGOMJ0sQxhhjPFmCMMYY48kShDHGGE+WIIwxxniyBGGMMcaTJQhjjDGees39IETkILCzE4sYABzqonASweLrHIuvcyy+zunO8Y1S1SKvCb0mQXSWiJS2d9OM7sDi6xyLr3Msvs7p7vG1x5qYjDHGeLIEYYwxxpMliI88kOoATsLi6xyLr3Msvs7p7vF5sj4IY4wxnuwIwhhjjCdLEMYYYzz1uQQhIgtEZJOIbBGR2zymp4vIE+70t0WkOImxjRCRv4nIBhFZJyJf9qhzrohUisgq9/HtZMXnrn+HiLzvrrvUY7qIyH+622+NiExPYmwTYrbLKhE5KiJfaVUnqdtPRB4SkQMisjamrL+ILBORze7ffu3Me4NbZ7OI3JDE+O4TkY3u5/dHESloZ94T/i8kML47RGR3zGd4cTvznvC7nsD4noiJbYeIrGpn3oRvv05T1T7zAILAVmAMEAZWA5Na1flH4Kfu88XAE0mMbwgw3X2eC3zgEd+5wDMp3IY7gAEnmH4x8BdAgNnA2yn8rPfhXASUsu0HzAWmA2tjyr4P3OY+vw34nsd8/YFt7t9+7vN+SYrvAiDkPv+eV3x+/hcSGN8dwFd9fP4n/K4nKr5W038IfDtV26+zj752BDET2KKq21S1AXgcWNSqziLgYff5k8A8EZFkBKeqe1X1Xfd5FbABGJaMdXehRcCv1fEWUCAiQ1IQxzxgq6p25ur6TlPV5UBFq+LY/7GHgcs8Zr0QWKaqFap6GFgGLEhGfKr6vKo2uS/fAoZ39Xr9amf7+eHnu95pJ4rP3W98GvhtV683WfpaghgG7Ip5XUbbHfDxOu6XpBIoTEp0MdymrWnA2x6T54jIahH5i4hMTmpgoMDzIrJSRG72mO5nGyfDYtr/YqZy+wEMUtW94PwoAAZ61Oku2/HzOEeEXk72v5BIt7hNYA+100TXHbbfOcB+Vd3czvRUbj9f+lqC8DoSaH2er586CSUiOcAfgK+o6tFWk9/FaTY5A/gv4OlkxgZ8TFWnAxcB/yQic1tN7w7bLwwsBH7vMTnV28+v7rAd/w1oAh5rp8rJ/hcS5X+BscBUYC9OM05rKd9+wDWc+OghVdvPt76WIMqAETGvhwN72qsjIiEgn44d4naIiKThJIfHVPWp1tNV9aiqVrvPnwXSRGRAsuJT1T3u3wPAH3EO5WP52caJdhHwrqrubz0h1dvPtb+52c39e8CjTkq3o9spfglwrboN5q35+F9ICFXdr6oRVY0CP29nvanefiHgcuCJ9uqkavvFo68liBXAeBEZ7f7KXAwsbVVnKdB8xsiVwEvtfUG6mttm+SCwQVX/o506g5v7RERkJs5nWJ6k+LJFJLf5OU5n5tpW1ZYC17tnM80GKpubU5Ko3V9uqdx+MWL/x24A/uRR5zngAhHp5zahXOCWJZyILAC+DixU1Zp26vj5X0hUfLF9Wp9qZ71+vuuJ9Algo6qWeU1M5faLS6p7yZP9wDnL5gOcMxz+zS1bgvNlAMjAaZrYArwDjElibGfjHAavAVa5j4uBvwf+3q1zC7AO56yMt4CzkhjfGHe9q90YmrdfbHwC3O9u3/eBkiR/vlk4O/z8mLKUbT+cRLUXaMT5VXsTTp/Wi8Bm929/t24J8IuYeT/v/h9uAT6XxPi24LTfN/8PNp/VNxR49kT/C0mK7xH3f2sNzk5/SOv43NdtvuvJiM8t/1Xz/1xM3aRvv84+bKgNY4wxnvpaE5MxxhifLEEYY4zxZAnCGGOMJ0sQxhhjPFmCMMYY48kShDHdgDvK7DOpjsOYWJYgjDHGeLIEYUwcROSzIvKOO4b/z0QkKCLVIvJDEXlXRF4UkSK37lQReSvmvgr93PJxIvKCO2DguyIy1l18jog86d6L4bFkjSJsTHssQRjjk4hMBK7GGWRtKhABrgWyccZ+mg68AtzuzvJr4OuqOgXnyt/m8seA+9UZMPAsnCtxwRm99yvAJJwrbT+W8DdlzAmEUh2AMT3IPGAGsML9cZ+JM9BelI8GZXsUeEpE8oECVX3FLX8Y+L07/s4wVf0jgKrWAbjLe0fdsXvcu5AVA68l/m0Z480ShDH+CfCwqn6jRaHIv7eqd6Lxa07UbFQf8zyCfT9NilkTkzH+vQhcKSID4fi9pUfhfI+udOt8BnhNVSuBwyJyjlt+HfCKOvf3KBORy9xlpItIVlLfhTE+2S8UY3xS1fUi8i2cu4AFcEbw/CfgGDBZRFbi3IHwaneWG4CfuglgG/A5t/w64GcissRdxlVJfBvG+GajuRrTSSJSrao5qY7DmK5mTUzGGGM82RGEMcYYT3YEYYwxxpMlCGOMMZ4sQRhjjPFkCcIYY4wnSxDGGGM8/X8Z+GNJ1NaStgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXQc1ZX48e/tbrX2xZYXjBdkjPEGxjbCZt8MZktswCQYmOAQMpCZYSaZrOSXBQKBhIGEZCZMEggc9hgGAnEIhM3si1dsgzEGecPybtnat5b6/v6oktySS3a1pFZL6vs5R0fdr1513S616na9V/WeqCrGGGNMe4FkB2CMMaZ3sgRhjDHGkyUIY4wxnixBGGOM8WQJwhhjjCdLEMYYYzxZgjDdTkTOFJHSmOebROScZMbU3USkWkSOTHYchyIi/09E/pTsOEzfZAmin3MPznXuAW2HiDwoIjnJjquvU9UcVd2Q7DgORVVvV9WvJ+K1RURF5KhueJ2bReTR7oip3esWuTGGuvu1U4UliNTwRVXNAaYAU4EfJjmePssONiaVWIJIIaq6A3gRJ1EAICLpInKXiHwuIjtF5A8ikhmzfI6IrBSRShFZLyLnu+XXiMhaEakSkQ0icn1nYhKRTBH5lYhsFpEKEXm7ZfsiMltE1ohIuYi8LiITYtbbJCLfE5HVIlIjIveLyFARecGN6RURGeDWbfkmeZ2IbBOR7SLynZjXmi4i77nb2S4ivxORcMxyFZF/E5HPgM9iyo5yH18oIh+7290qIt+NWfefRaRERPaKyEIRObzd635DRD4TkX0ico+ISAf76UER+XnM8/bNeD9wt10lIutEZKZb3vrtPGY/zHf/3ntE5Eft/hYPubGsFZHvx26jXTxvug9XuWenl7vlX3A/L+Ui8q6ITD5YjO7n6f8Bl7uvs6qD7XX0/gIicqP72SwTkSdFZKC7WkuM5e5rn+T12uYgVNV++vEPsAk4x308AvgQ+G3M8t8AC4GBQC7wN+AX7rLpQAVwLs6XieHAeHfZRcAYQIAzgFpgmrvsTKDUKwaP+O4BXndfOwicDKQDRwM17rbTgO8DJUA45jXfB4a66+4CVuCcIaUDi4Cb3LpFgAJ/BrKBY4HdMfvleOBEIOTWXQt8KyZGBV5291FmTNlR7uPtwGnu4wEx++FsYA8wzY3pf4A3273uc0ABMMqN6fwO9tODwM9jnrfuY2AcsAU4POb9jnEf3ww82m4/3AdkAscBDcAEd/kvgTfc9zACWB37d/SIqXUfuM+nuX+HGe7fcr77d0r3G2MH2znYut9yPwcj3O38Efhzu/cbSvb/YV/9SXoA9pPgP7DzD1oNVLn/LK8CBe4ywTkIj4mpfxKw0X38R+Bun9t5Fvim+7j14BUTwwEJAifp1AHHeSz7CfBku7pbgTNjXvOqmOVPA7+Pef7vwLPu45YDxfiY5f8F3N/Be/kW8EzMcwXOblcnNkF8DlwP5LWrcz/wXzHPc4AIUBTzGqfGLH8SuLGDmB6k4wRxFM6B+Rwgrd16rQffmP0wImb5EmCe+3gDcF7Msq8TX4L4PXBruzrrcL5A+Iqxg+0cbN21wMyY58PcfdyS7C1BdOHHmphSw8WqmotzUBkPDHLLBwNZwHK3SaAc+IdbDjASWO/1giJygYi87zadlAMXxryuX4OAjA62cTiwueWJqkZxvkUOj6mzM+Zxncfz9p3xW2Ieb3a3gYgcLSLPidOJXwnczoHvZQsdm4vz/jeLyBsxTRnt30M1UNbuPeyIeVzrEfMhqWoJTlK7GdglIgtim7I8dLTNw2n7Pg/2nr0cAXyn5bPkfi5G4nzzjzfGVodY9wjgmZjtrQWacc4sTRdZgkghqvoGzjfRu9yiPTgH0kmqWuD+5KvToQ3OAWJM+9cRkXScb+x3AUNVtQB4HueMJB57gHqvbQDbcP75W7YpOAebrXFuI9bImMej3G2A8833E2CsqubhtIm3fy8dDnusqktVdQ4wBOdM6skO3kM2UNjJ91CDk8xbHNYuhsdV9VR3ewrc0YltbMdpqmkxsqOKHdgC3BbzWSpQ1SxV/fMhYjzkkNIHWXcLcEG7bWao6lY/r2sOzhJE6vkNcK6ITHG/ld8H3C0iQwBEZLiInOfWvR+4xu1MDLjLxgNhnPbe3UCTiFwAzIo3EHf7DwC/FpHDRSQoIie5CehJ4CJ322nAd3Day9/twnv/iYhkicgk4BrgCbc8F6gEqt339y9+X1BEwiJylYjkq2rEfZ1md/HjOPtvivuebgcWq+qmTsS+ErhQRAaKyGE436hbYhgnIme726jHSfrNHbzOwTwJ/FBEBojIcOCGQ9TfCcTeC3If8A0RmSGObBG5SERyDxHjTqBIRDyPR4dY9w/AbSJyhFt3sIjMcZftBqLtYjRxsASRYlR1N/AwThs/wA9wOn/fd5tXXsHpFERVl+AcSO/G6ax+AzhCVauA/8A5oOwDrsTp6O6M7+J0nC8F9uJ8Mwyo6jrgn3A6dvcAX8S5XLexk9vBjb8Epx/mLlV9KSaGK3H6ae5jf+Lw6yvAJnf/fcONG1V9FWc/P43z7XwMMK+TsT8CrMLpe3mpXYzpOB3Me3Caj4bgnAXF6xagFNiI8zl4Cicpd+Rm4CG3eefLqroM+GfgdzifixLgqz5i/D/3d5mIrPDYzsHW/S3OZ+8lEanC6bCeAaCqtcBtwDtujCf62gumlbgdO8b0WyJShHPQS1PVpuRG03eIyL/gdGCfkexYTHLYGYQxBgARGSYip7jNieNwmvWeSXZcJnnsrlBjTIswzqXNo4FyYAHwv0mNyCSVNTEZY4zxZE1MxhhjPPWbJqZBgwZpUVFRssMwxpg+Zfny5XtUdbDXsn6TIIqKili2bFmywzDGmD5FRDZ3tMyamIwxxniyBGGMMcaTJQhjjDGe+k0fhDEmtUUiEUpLS6mvr092KL1SRkYGI0aMIC0tzfc6liCMMf1CaWkpubm5FBUVId4T86UsVaWsrIzS0lJGjx7tez1rYjLG9Av19fUUFhZacvAgIhQWFsZ9dmUJwhjTb1hy6Fhn9o0lCGOMMZ4sQRhjTALcfPPN3HXXXR0uf/bZZ/n44497MKL4WYIwxpgksARhjDEp5LbbbmPcuHGcc845rFu3DoD77ruPE044geOOO465c+dSW1vLu+++y8KFC/ne977HlClTWL9+vWe9ZLMEYYwx3WD58uUsWLCADz74gL/85S8sXboUgEsvvZSlS5eyatUqJkyYwP3338/JJ5/M7NmzufPOO1m5ciVjxozxrJdsdh+EMcZ0g7feeotLLrmErKwsAGbPng3ARx99xI9//GPKy8uprq7mvPPO81zfb72eZAnCGGO6idelpF/96ld59tlnOe6443jwwQd5/fXXPdf1W68nWROTMcZ0g9NPP51nnnmGuro6qqqq+Nvf/gZAVVUVw4YNIxKJ8Nhjj7XWz83NpaqqqvV5R/WSyRKEMcZ0g2nTpnH55ZczZcoU5s6dy2mnnQbArbfeyowZMzj33HMZP358a/158+Zx5513MnXqVNavX99hvWTqN3NSFxcXq00YZEzqWrt2LRMmTEh2GL2a1z4SkeWqWuxV384gjDHGeLIEYYwxxpMlCGOMMZ4sQRhjjPGU0AQhIueLyDoRKRGRGz2Wp4vIE+7yxSJSFLNssoi8JyJrRORDEclIZKzGGGPaSliCEJEgcA9wATARuEJEJrardi2wT1WPAu4G7nDXDQGPAt9Q1UnAmUAkUbEaY4w5UCLPIKYDJaq6QVUbgQXAnHZ15gAPuY+fAmaKcyviLGC1qq4CUNUyVW1OYKzGGNOrFBUVsWfPHt91vva1rzFkyBCOOeaYboshkQliOLAl5nmpW+ZZR1WbgAqgEDgaUBF5UURWiMj3vTYgIteJyDIRWbZ79+5ufwPGGNNXfPWrX+Uf//hHt75mIhOE1/x27e/K66hOCDgVuMr9fYmIzDygouq9qlqsqsWDBw/uarzGmBSyeNdGfrjkWa5/63F+uORZFu/a2OXX3LRpE+PHj+frX/86xxxzDFdddRWvvPIKp5xyCmPHjmXJkiXs3buXiy++mMmTJ3PiiSeyevVqAMrKypg1axZTp07l+uuvJ/Ym5kcffZTp06czZcoUrr/+epqbD2xQOf300xk4cGCX30OsRCaIUmBkzPMRwLaO6rj9DvnAXrf8DVXdo6q1wPPAtATGaoxJIYt3beTRz5awt8GZc2FvQy2PfrakW5JESUkJ3/zmN1m9ejWffPIJjz/+OG+//TZ33XUXt99+OzfddBNTp05l9erV3H777Vx99dUA/OxnP+PUU0/lgw8+YPbs2Xz++eeAc/fzE088wTvvvMPKlSsJBoM9NlZTIkdzXQqMFZHRwFZgHnBluzoLgfnAe8BlwCJVVRF5Efi+iGQBjcAZOJ3YxhjTZc9uWkVjtO238MZoM89uWsWMIaO79NqjR4/m2GOPBWDSpEnMnDkTEeHYY49l06ZNbN68maeffhqAs88+m7KyMioqKnjzzTf5y1/+AsBFF13EgAEDAHj11VdZvnw5J5xwAgB1dXUMGTKkSzH6lbAEoapNInID8CIQBB5Q1TUicguwTFUXAvcDj4hICc6Zwzx33X0i8mucJKPA86r690TFaoxJLS1nDn7L45Gent76OBAItD4PBAI0NTURCh142G0ZJtxruHBVZf78+fziF7/ocmzxSuh9EKr6vKoerapjVPU2t+ynbnJAVetV9UuqepSqTlfVDTHrPqqqk1T1GFX17KQ2xpjOGJieFVd5dzr99NNbm4hef/11Bg0aRF5eXpvyF154gX379gEwc+ZMnnrqKXbt2gXA3r172bx5c8LjBLuT2hiTgi4uOo5wINimLBwIcnHRcQnf9s0338yyZcuYPHkyN954Iw895Fzpf9NNN/Hmm28ybdo0XnrpJUaNGgXAxIkT+fnPf86sWbOYPHky5557Ltu3bz/gda+44gpOOukk1q1bx4gRI7plylIb7tsY0y/EO9z34l0beXbTKvY21DIwPYuLi47rcv9DbxfvcN825agxJiXNGDK63yeErrImJmOMMZ4sQRhjjPFkCcIYY4wnSxDGGGM8WYIwxhjjyRKEMcb0QvEM971lyxbOOussJkyYwKRJk/jtb3/bLTHYZa7GGNPHhUIhfvWrXzFt2jSqqqo4/vjjOffcc5k4sf0cbfGxMwhjTEqKrn2P5vu+T/Ovr6X5vu8TXftel18zWcN9Dxs2jGnTnAGvc3NzmTBhAlu3bu3y+7EEYYxJOdG176EvPwxVZU5BVRn68sPdkiSSPdz3pk2b+OCDD5gxY0aX34s1MRljUo6+/Qw0NbYtbGp0yiec1KXXTuZw39XV1cydO5ff/OY35OXldel9gCUIY0wqajlz8Fseh2QN9x2JRJg7dy5XXXUVl156aVfeQitrYjLGpJ7cwvjKu1EihvtWVa699lomTJjAt7/97W6L1RKEMSblyKmXQCjctjAUdsoTLBHDfb/zzjs88sgjLFq0iClTpjBlyhSef/75Lsdqw30bY/qFeIf7jq59z+lzqCqD3ELk1EsIdLH/obez4b6NMcaHwISTutwh3d9ZE5MxxhhPliCMMf1Gf2kyT4TO7BtLEMaYfiEjI4OysjJLEh5UlbKyMjIyMuJaz/ogjDH9wogRIygtLWX37t3JDqVXysjIYMSIEXGtYwnCGNMvpKWlMXq0zTHdnayJyRhjjCdLEMYYYzwlNEGIyPkisk5ESkTkRo/l6SLyhLt8sYgUueVFIlInIivdnz8kMk5jjDEHSlgfhIgEgXuAc4FSYKmILFTVj2OqXQvsU9WjRGQecAdwubtsvapOSVR8xhjTHbS+BoJpSFr40JX7mESeQUwHSlR1g6o2AguAOe3qzAEech8/BcwUr+EMjTH9kmoUratGIw3JDiVuWldNdO17RBfegy56DK3c0+8usU3kVUzDgS0xz0uB9jNYtNZR1SYRqQBahlMcLSIfAJXAj1X1rfYbEJHrgOuA1oGtjDE9Q6NRqKuE2ipIz4JwJpKR5X/9umq0ZAX60dtIwWA4+RLIG4iI/++tGmmExjoIBJDM3M68jU7RaDO6bgm6yBl9VUvXoRtXE/inmyEnv8fiSLREJgivM4H26bWjOtuBUapaJiLHA8+KyCRVrWxTUfVe4F5wBuvrhphND9O6amiKQCAAGTlIMJjskIxfFbuILvgl1FUBIMXnwfSLkIzsQ66qzc3ox++ibzzhPN++Ht20hsDVP4NsfwdYra1Cl/wdXbcEcgcSmPkVGDQcCfbA1ft1NejKRW3LaiuhYne/ShCJbGIqBUbGPB8BbOuojoiEgHxgr6o2qGoZgKouB9YDRycwVpMEWl1O9Lk/EL3vu0QfuRn9fE2fbGpIRVpfS/S1P7cmBwBd9iLU1/h7gfoadPXrbcvqqpwDrJ/tN0XQFS+jK16GmgrYsZHoE7+Eumqf76CLAgHIzDmwPD2zZ7bfQxKZIJYCY0VktIiEgXnAwnZ1FgLz3ceXAYtUVUVksNvJjYgcCYwFNiQwVtPDtLEBffsp2LLWKaitRP/6O2ioTW5gKUQ1itZUoHu2oZV70Xj2fXME9u08sLy63N/6gQBkeUyJme6ziaqhFv10aduypka0fJe/9btIMnMInHE5BGLOeIuO8X5PCaLNzc6XrC2foGXb0Jhk3V0Sdi7m9incALwIBIEHVHWNiNwCLFPVhcD9wCMiUgLsxUkiAKcDt4hIE9AMfENV9yYqVpMEkXr080/alkWbobIMcgYkJ6ZUU77b+dZdWwkIMv1CKD7PVxMR6VnI0cXo0hf2l4XCUOA9V3J7kplD4Mx5RBf8ApqbnMIjp0CWz36EYJqzrXYJQXw2T3WLQcMJfO0X6NbPkLxCGHAY4jf+7lC+k+ifb4PGehSQ8SfCWVcgXmc2nZTQxjpVfR54vl3ZT2Me1wNf8ljvaeDpRMZmkiwUhsNGQ8m+/WUikDMweTGlkNYmotqWbj1Fl/wdOeY08JEgJJQGx8+CpojTB5BX6PQBxHNwKnQPsNs3OAfYvEG+O5olI4vAWVc4CcZtVpJp58a3/S6SUBjyCp3Ye5jz91sAjfX7yz55H5lxUbfuAxuLySSFpGcSOHMe0fKdsGerM93jzKv6XRtur9Ucgb3bDyyvKYeCwb5eQrLy4NS5yAkXQDAY91VEEkqD3IFIbie/FOQPIfCVnzlJLqPlKiofZz8xtL4WmhqdLyeZOUigj1wk0RyB6gMbVbS2CunGfGUJwiSN5BUSuOy7EGmEYMhptuiHNxt1RJsaobYK3VaC5A+G/ME910TR0kS07B/7y0JhyPeXHFpIWhiS9DeTQAByCpyfTtCaCqKvPAzrV0F2PjJrPowYh6Sld3OkCZCZg0w8GX07pqElnIkMGNqtm7EEYZJKerBTr9fZXer0AUSbneu/j5xC4LxrurUNuSMSSoPi8yDSsL+J6JyrfTUv9QcaaUTfWwjrVzoFNeXoX3+HXPtL6AMJQgJBOOY0EEHXvOP8/c64HLr5XhBLEMYkgdZVEX19gdMx32LDSudSzx5qR5esPDj9S8iML0Ag2LMdrMnWWIduXtO2LNqMlu/qfJNXD5OsXDh+FjLxZAilIX6vAIuDJQiT0rShzmmDBsjKI96RXrSpCRpqIBiKr/07Gm1zD0Grhrq4tt9VkpbeJ74xd7u0dBhadMB9F5I3KDnxdJIEgr5vLOwMSxAmZWlNBfr6AvSz5ZA3iMB516BDi5zmFz/r11ahK19FP1nsnOKfdSUMGOqvozMzBznurNY7iQHnHz0JV8SkIglnEDjjy0TLtkLZNifBn/5lp7PbtJL+MrhUcXGxLlu2LNlhmD5CI43oG0+0vZs3GCJw7S8RH/dhaFMEXfwcuvi5/YXpmQTm/xzx2WmqddXo+pXomreRgqHISbOdq3psvMoeozWV0NTgXCQRzkLCPXs2pY0NzqWqAUlaf5yILFfVYq9ldgZhUlNjHbpxdduy5iao8HmjXkMtuvb9dmV17o1+/hKEZObApFOQMVOdNuQUuoKrt5Ds5F0kobWV6HsL0U/eh1z3IoEho3yfwfYEm1EuxWlNBVq+C63el1rjIIXSYJDHBO65Pi+ZDIa8m4Pi7OgVESQzu1PJQRvq0J2bib72ONHVb6A1FXG/hkkObYqgy19CV73mfLHYU0r0qTuhvofGkvLJziBSmFbuIfr0r50xdUJpyJlXwLjpSB+5WU1bOnprq5wb7NIzfV/JIelZBM66kuierVBV5lzFc+qlvscCkoxsAmdfRXTB7a13s8rkM/2PJdRFqopuWYsuvGd/2cpFBOZ+J6nfio1PDbVO31espgi6b6evJs6eYgkiRWlDLdFFj+8fcK0pgr76CFJ0TN+5m7l8lzuCpzvc9IwvwPHn+Z+TIH8QgSt/5BzgQ2EIZ8SXHAccRuCrtzlXwmTlQUZ2j9zDAEBdFfruX9uW7Sl17oS2BNH7hcJQePiBY0n1ouQA1sSUupoisGtz2zJV5wDTB2h9DdFFj7Udbnrxc84lpz6JCJKdjwwYiuQOiPvMSYJBJKcAGT7WeY0eHAfI9G2Snunc2NbaXyXOkCW97DNkZxBdpLWV+4eKyMhyBvDqC8LpyKiJ6Mfv7i8LhqCP3CTU4XDTNZVxDxfRJ2XmIifNRv/2v/vLCodDdueGnTBJkD+YwFU/dYa4T0t3z2B712W2liC6QCvLiD7zWyjbCmnpyMyvwFFTkXBGskM7JEnLgNMuc2Z02/gh5A4gMOuavjPUQnoWMnaaM2FMi7T0lLmPQERg1ATkyh+jH73tzKQ29njrf+hDRMS596UnhyiPkyWITtKGOqKvPe4kB3DGtHnxAWTkHeAzQWi02elgra10DsxxzunbVZKdT+CCrzvNTSLOt9JA32h1lFAYpl8IzU3ouqVOf8I5V/e6U/REkvQsOGw0ctjoZIdi+ilLEJ3V1Ag7NrYt06gzo5bfZpp9O51OVneaxng7WbW+FvbtcJqJBo9AxkyNe8KUeIdH7k1SeiwhY3qAJYjOSktHRo53hlloEQhCrr+rELS+huirj7aZw1cXP+dO2HLoBKEaRTd9hD7/x/1lq94gMPc/U2qE1JQdS8iYHtA32hN6IQlnOGO3jBznFGTnIxf/O6T7/Ebe3HTAJW6A0+TkR20V+n67Kb53fx4zQ5gxxnSNnUF0geQUEPjCvzpt+IFAfDNShTORscejH7zSpszvGchBouri+sYY47AziC6SzBznGvrs/LimK5S0MDLjIuS4s52brIaNIXD5D/xP+JGVi5w0p23ZkFFxD/VgjDEdSfnRXLW+xp2TNpCUq3g00uhcB92JOX21vhbKdzqd1INGIGOmxN1JbYxJbTaaawe0upzoC3+CLWshpwA572tw+NgeHVWzK3P6SoZd5miMSZyUbWLSxnr0zSed5ABQXY4++99xDdVg3P1YU+HcUW6M6VdS9wyisR7d8knbsuYm5z6GXjZgVm+lNRXom0/un5Ft1jW9bjx7Y0znpewZBGlhGHpE2zIJ9Orb3nsTjTSi7//NmTSnKQJ7txN96q4293UYY/q2hCYIETlfRNaJSImI3OixPF1EnnCXLxaRonbLR4lItYh8t9tjS88icPZVMHCYUxAKI7Ou6bHx/Pu8xjp0/cq2ZU2NztwKxph+IWFNTCISBO4BzgVKgaUislBVP46pdi2wT1WPEpF5wB3A5THL7wZeSFiMeYMIfPn7+0djTc+yaR/9CqbBoOFQvS+mUOwMzJh+JJFnENOBElXdoKqNwAKg3YX7zAEech8/BcwUd8Z2EbkY2ACsSWCMSFYekj/IGdffkoNvkuGegbX010gAOW0uhO0MzJj+IpGd1MOBLTHPS4EZHdVR1SYRqQAKRaQO+AHO2UeHzUsich1wHcCoUaO6L3LjT/5gAlf9xJ2RLc2Z8jPcR2ajM8YcUiLPILzGfGh/V15HdX4G3K2qB53BW1XvVdViVS0ePDgFJonpZdrOyDbQkoMx/UwizyBKgZExz0cA2zqoUyoiISAf2ItzpnGZiPwXUABERaReVX+XwHiNMcbESGSCWAqMFZHRwFZgHnBluzoLgfnAe8BlwCJ1xv44raWCiNwMVFtyMMaYnpWwBOH2KdwAvAgEgQdUdY2I3AIsU9WFwP3AIyJSgnPmMC9R8RhjjInPIQfrE5GhwO3A4ap6gYhMBE5S1ft7IkC/OjtYnzHGpLKDDdbnp5P6QZyzgMPd558C3+qe0IwxxvRWfhLEIFV9EoiC03QENCc0KmOMMUnnJ0HUiEgh7iWqInIiUJHQqIwxxiSdn07qb+NcbTRGRN4BBuNccWSMMaYfO2SCUNUVInIGMA7nxrZ1qhpJeGTGGGOS6pAJQkSublc0TURQ1YcTFJMxxphewE8T0wkxjzOAmcAKwBKEMcb0Y36amP499rmI5AOPJCwiY4wxvUJnBuurBcZ2dyDGGGN6Fz99EH9j/yisAWAi8GQigzLGGJN8fvog7op53ARsVtXSBMVjjDGml/DTB/FGTwRijDGmd+kwQYhIFQdO8APOvRCqqnkJi8oYY4wvdU2NhAJB0gLBbn/tDhOEquZ2+9aMMcZ0i5pIA5+U7+StHSUMzsjhgpGTGJCehYjXRJ2d43s+CBEZgnMfBACq+nm3RWGMMca3qCoflG3hkc+WALAWWFlWyo+nXUB+N079e8jLXEVktoh8BmwE3gA2AS90WwTGGGPiUh2p59Wt69qUVUbq2V1X1a3b8XMfxK3AicCnqjoa507qd7o1CmOMMb4FJUBWKHxAeUYorVu34ydBRFS1DAiISEBVXwOmdGsUxhiTYmojjWyu2stjJUtYtHUdFY11vtfNTkvnstFTCcr+Q/i4/KHkp3Vf8xL464MoF5Ec4C3gMRHZhXM/hDHGmE76rHIX//vxm63P39j+Gd+ZfA554YyDrLXf8OwCbi3+AuvKd1KYkcOwrHxyfa7rl58E8SZQAHwT+CcgH7ilW6MwxpgUUtVYz3Off9imbEddJXsbanwniHAwRGEwh5MPy0lEiIC/JibBmZP6dSAHeMJtcjLGmJRWHWlgb0MN5Q21NEW7PhNz912g2j0OmbztfToAABaxSURBVCBU9WeqOgn4N+Bw4A0ReSXhkRljTC9W3lDL/378Jj9c8lduWv53VuzZQn2Tv7nUcsMZzD5icpuyYVn5DEjPSkSoneb7PghgF7ADKAOGJCYcY0wqqWqspzHaTFCErLQw4UA8h6Tu0Zk7kRuam3h20yrWV+4GoL45wgPr3uW2E+b4vpLoqLzB/Gjq+by3cyOHZeUxpXAEed14D0N38DOa678Al+PMRf0U8M+q+nGiAzPG9G/7Gmr5/cdvsrl6L+nBEPPGFDOtcGS3X6rZkZY7kd/esZ4hmTmcN2Ki7zuR65sjfOYmhxYK7KmvojAj29f2M0NhRuUMZFTOwM6E3yP8pOsjgG+p6spEB2OMSQ31TRGe2biSzdV7Aecb+cOfvs+4E4b2SIKIapQVe7bwaIlzJ/LH5fBBWSk/mnq+rzuRM4JpjM0bzJ766tYyAQZl9K8Rivz0QdzY2eQgIueLyDoRKRGRGz2Wp4vIE+7yxSJS5JZPF5GV7s8qEbmkM9s3xvRODdEmSjy+gZc1VHuv4KE52kx5Qy3ryneyvbaC6kiD73WrI428uq3tncgVjXVtDvgHkx4McXHRcYzJGwQ4CeOacSeR7XHzWl+WsAY/EQkC9wDnAqXAUhFZ2K556lpgn6oeJSLzgDtwmrM+AopVtUlEhgGrRORvqmr3XxjTD2QE0zg6fwjv7drYWiYIgzL8X7K5s66KX656iYZm57Bw4pDRfOnIaeSkpR9y3YAImcEDz1QyPMo6UpCexb9OPIPG5iYCIuSkpRNKwIiqydSZKUf9mg6UqOoGVW0EFgBz2tWZAzzkPn4KmCkioqq1MckgA+9hx40xSdbQHKGioY7KOO4ChpZv4FM4Ot+53iU7FOa68ad4Dh/hpSbSwBMblrcmB4D3d230fRaRk5bOZUdOJRDT3zA+fyh5afHdaJaTls7AjGwK0rP6XXKABJ5BAMOBLTHPS4EZHdVxzxYqgEJgj4jMAB7A6QP5itfZg4hcB1wHMGrUqG5/A8b0dk3RZmqaGglJgGwf35xjNTQ3URWpZ+2+HQzOzGF4VkFcd+JWNtazcPNqlu7exMD0bL4ydgYjsweQFvR3oCxIz+T6CacRiTYTQMhOC/s+yDZrlL0NtQeUV0fqAX9T1YzMHsCtxV/kk/KdDMrIYXgC7kTu6xKZILwuBWh/JtBhHVVdDEwSkQnAQyLygqrWt6moei9wL0BxcbGdZZiUUhWpZ9HWdSzetYnCjGyuGFPM0Mw8ggF/DQNbqvfxq9WvEHX/LSfkH8a1E04m18e36EhzMy+WfsxbO0oA2FZbwa8/fJWfF3+RgqD/a/n9NAd5yQqlc9KQ0fx18+rWssxgGoMz/XcSh4MhBgVzODWBdyL3dYlsYioFRsY8HwFs66iOiIRwhvHYG1tBVdcCNcAxCYvUmD6mKdrMoq3reH7LGsoaavi0Yhd3rHqZKp9NLNWRBp7auKI1OQCsrdhBVWP9Qdbar7a5kZVlW9qURaLN7PbZydtVoUCA0w47iouLjuOwzDwmDRjGD6bM6nTCMd4SeQaxFBgrIqOBrcA84Mp2dRYC84H3gMuARaqq7jpb3GanI4BxOPNQGGOAmqZGluze1KasvjlCWUM1BemHvkwzqlFqPe76bfA5XEQ4EGRYZgF76mvalBeEe+5O4NxwBrOGT+CUoWMIBbyHvzZdk7AzCLfP4AaccZzWAk+q6hoRuUVEZrvV7gcKRaQE+DbQcinsqThXLq0EngH+VVX3JCpWYzqrJtJAeUMtFY11NGs0rnVVlYrGOnbWVbKvodb3MA0AIQl4XvHjp3kInOGiZw4f16asIJzJQJ9DPWSGwlw+ZhoF7j0DAnxh1LE9fplnMBAgL5xhySFBRLV/NN0XFxfrsmXLkh2G6YOiqm2uZvGrorGOB9e9x8flO8hLy+Dqo2dwdP5Q0oP+Tsx311Xzqw9fYV9DLQGE2UWTOWPYWN8Hu+21Fdyx8iXqmp3Ecsawscw5YrLvzuqWO4nf2bmeIRm5zBo5gYHp/u4CBifBVUbqqWuKkB4MkR4M2YG6DxKR5apa7LnMEoRJVbWRBrbXVvLOzvUMzyqgeMgRvufzbWiOsKBkOe/u2tBaFhDh9hPm+Bpwra6pkfs/eY8P921tU37bCbN93wvQHI1S3dRAWX0NOWnpZIfCcV/J1BJLWiDYLy/TNId2sATR8yNjGdMLRFVZs28Hf1q3f/bct3au59vHzvQ1Hn99cxNrK3Yc8Jpl9TW+EkRjtJltteUHlJc31PlOEMFAgPxwZpcnqc+0b/2mA4m8ismYXqs60sDft3zUpmx7bYXvaR/DgSCjcwvblAn4b8MPpjF54PA2ZWmBIIN8DvRmTE+wMwjTZ9VGGtlRV8m7O9YzPKeAaYNGxfVtOuBxG47fvojMUJgvHTmNHbWVbKutIC0QZN6Y431/Gw8HQ1w4ahK1zRFW7PmcQvdGs+yQXaZpeg/rgzB9UlSVFXs+575P9jcRHZ6Vz3/6bCJSVVaVbeX3a/fPCTwyewD/ccxZvqd8BOdu4sZoEyFxLrMM++ygblHfFKEh2oQgcW3XmO5ifRCm36mO1PPc522biLbVVlDZWOfrQCsijCsYwo+nXsDiXZsYnp3PpAHD4j5Id/WgnhFKI4Oemf/AmHhZgjBJ09Acobyxjo/2bmNoZh6jcgb6PuAK4jlOi5/JXlpkhsKMzAkzMmeA73WMSSWWIEzSbKgs47cfvYa6wz2Myx/KP084xdfNXjlp6cw+YjJ/WPtWa9nI7AG+bxQzxhyaJQjTJZWNddQ3N5EWCJIVSiPd53j6VY31PL3xg9bkALCuYifVkQZfB3kRYXzBUH409Xze37mR4dkFHDvwcGvHN6YbWYIwnbanvpq7P3yVPfU1BCXAZaOnctLQ0b6u5FGUhuiB8z9FfI4FBH1jTl9j+jK7D8J0Sm1TIwtKlrcO1tasUZ7csNxzADgv2WnpnDN8fJuyQRnZXb7pyxjTfewMwlDX1EgoECQtjqEWItFmttS0GZkdxRmfqNDHzV5BCVA8aBQF4Uze3rGeYVl5nH34OEsQxvQiliD6gcZm5zr8gM+JYlpURxr4aO823tu1gcMy8zhv5ETfg7VlBNM4ZsDhvL1zfWtZSAK+7yQG5yziuMIRjC84jJAEfE90Y4zpGZYgkkhVqYrUU9/cRDgQJDOOTl5wDvAbK/fw9s71HJ6VzxmHH906/PKhNEejvL9zI/+3cQUAn5TvZNXerfxwynm+vsWnB0PMLppMTVMDK8u2UpiRxfyxJ5KVFv+4Pn5HPzXG9Cz7z0yiPfU13P3hq5Q1xN/J26xRlu/5nMdLlgKwsqyUZbs/53vHnUOejwN8dVMDr23/tE3ZPndeA7/NPPnhTK4++kSuiDYjOHMRxHMfgjGmd7Nz+iSpbWrkz+uXUtbQtpO3zmcnb3WkgVdKP2lTtqu+igqfU0YGRDwndwkH4vvOkBUKkx/OJC+cacnBmH7GEkSSRKLNbK1pO9yzAhURf6OJCkI4eGCncshnO35uWgZfOnJamwHrjh04nJxONBEZY/ona2LqBrXuhCvxXAWUEUxj0oDDeSemkzctEGSAzzl988IZXDp6Kv/z0Wutt5qNyx8a16Tto3IGcusJX+TT8p0MycxlaGYeOXYnsjHGZQmiC6ojDazZu413d21gWGY+542c6GuyGHA6ZucUTaa2qZFVZaUUZmQz/+gT4xrueUzuIH52/BdYtbeUYVn5FOUUxjXUhDNNZA6DDvM3QY0xJrWk9HDfzRqlOtJAZWM9WaEwmaE033PqNkejLNq2jqc2ftBaVpiezY1TZvnqJG5R19RIo3XyGmOSxIb77sCuuiruXPUyNU2NAFw08hjOGTHeV5KobmrgtW1trwIqa6ihorE+rgSRGQpjt4YZY3qjlO2krok08FjJ0tbkAPD3LR9RF/P8YAKIZyLx6jg2xpi+KGUTRJNG2VVXdUB5daTB1/q54QOvApo8cLhNGWmM6TdStokpM5jGtMKRbW4WywimUeDzKiKAopxCbj3hi6xzrwI6LDMvrquIjDGmN0vZBOFMGn8MirJ8z+cMzsjlqrEnxHUfQHooRHrIrgIyxvRPCU0QInI+8FsgCPxJVX/Zbnk68DBwPFAGXK6qm0TkXOCXQBhoBL6nqou6O768cAZzR0/lwlHHEJSAffs3xpgYCUsQIhIE7gHOBUqBpSKyUFU/jql2LbBPVY8SkXnAHcDlwB7gi6q6TUSOAV4EhiciznAwRNgGizPGmAMkspN6OlCiqhtUtRFYAMxpV2cO8JD7+ClgpoiIqn6gqtvc8jVAhnu2YYwxpockMkEMB7bEPC/lwLOA1jqq2gRUAIXt6swFPlDVAy4vEpHrRGSZiCzbvXt3twVujDEmsQnC65bg9rdtH7SOiEzCaXa63msDqnqvqharavHgwYM7HagxxpgDJTJBlAIjY56PALZ1VEdEQkA+sNd9PgJ4BrhaVddjjDGmRyUyQSwFxorIaBEJA/OAhe3qLATmu48vAxapqopIAfB34Ieq+k4CYzTGGNOBhCUIt0/hBpwrkNYCT6rqGhG5RURmu9XuBwpFpAT4NnCjW34DcBTwExFZ6f4MSVSsxhhjDpTSo7kaY0yqO9horik7FpMxxpiDswRhjDHGkyUIY4wxnixBGGOM8WQJwhhjjCdLEMYYYzxZgjDGGOPJEoQxxhhPliCMMcZ4sgRhjDHGkyUIY4wxnixBGGOM8WQJwhhjjCdLEMYYYzxZgjDGGOPJEoQxxhhPliCMMcZ4sgRhjDHGkyUIY4wxnixBGGOM8WQJwhhjjCdLEMYYYzxZgjDGGOPJEoQxxhhPliCMMcZ4SmiCEJHzRWSdiJSIyI0ey9NF5Al3+WIRKXLLC0XkNRGpFpHfJTJGY4wx3hKWIEQkCNwDXABMBK4QkYntql0L7FPVo4C7gTvc8nrgJ8B3ExWfMcaYg0vkGcR0oERVN6hqI7AAmNOuzhzgIffxU8BMERFVrVHVt3EShTHGmCRIZIIYDmyJeV7qlnnWUdUmoAIo9LsBEblORJaJyLLdu3d3MVxjjDGxEpkgxKNMO1GnQ6p6r6oWq2rx4MGD4wrOGGPMwSUyQZQCI2OejwC2dVRHREJAPrA3gTEZY4zxKZEJYikwVkRGi0gYmAcsbFdnITDffXwZsEhVfZ9BGGOMSZxQol5YVZtE5AbgRSAIPKCqa0TkFmCZqi4E7gceEZESnDOHeS3ri8gmIA8Ii8jFwCxV/ThR8RpjjGkrYQkCQFWfB55vV/bTmMf1wJc6WLcokbEZY4w5OLuT2hhjjCdLEMYYYzxZgjDGGOPJEoQxxhhPliCMMcZ4sgRhjDHGkyUIY4wxnixBGGOM8WQJwhhjjCdLEMYYYzxZgjDGGOPJEoQxxhhPliCMMcZ4sgRhjDHGkyUIY4wxnixBGGOM8ST9ZYZPEdkNbO7CSwwC9nRTOIlg8XWNxdc1Fl/X9Ob4jlDVwV4L+k2C6CoRWaaqxcmOoyMWX9dYfF1j8XVNb4+vI9bEZIwxxpMlCGOMMZ4sQex3b7IDOASLr2ssvq6x+Lqmt8fnyfogjDHGeLIzCGOMMZ4sQRhjjPGUUglCRM4XkXUiUiIiN3osTxeRJ9zli0WkqAdjGykir4nIWhFZIyLf9KhzpohUiMhK9+enPRVfTAybRORDd/vLPJaLiPy3uw9Xi8i0HoxtXMy+WSkilSLyrXZ1enQfisgDIrJLRD6KKRsoIi+LyGfu7wEdrDvfrfOZiMzvwfjuFJFP3L/fMyJS0MG6B/0sJDC+m0Vka8zf8MIO1j3o/3sC43siJrZNIrKyg3UTvv+6TFVT4gcIAuuBI4EwsAqY2K7OvwJ/cB/PA57owfiGAdPcx7nApx7xnQk8l+T9uAkYdJDlFwIvAAKcCCxO4t97B85NQEnbh8DpwDTgo5iy/wJudB/fCNzhsd5AYIP7e4D7eEAPxTcLCLmP7/CKz89nIYHx3Qx818ff/6D/74mKr93yXwE/Tdb+6+pPKp1BTAdKVHWDqjYCC4A57erMAR5yHz8FzBQR6YngVHW7qq5wH1cBa4HhPbHtbjYHeFgd7wMFIjIsCXHMBNaralfuru8yVX0T2NuuOPZz9hBwsceq5wEvq+peVd0HvAyc3xPxqepLqtrkPn0fGNHd2/Wrg/3nh5//9y47WHzusePLwJ+7e7s9JZUSxHBgS8zzUg48ALfWcf9BKoDCHokuhtu0NRVY7LH4JBFZJSIviMikHg3MocBLIrJcRK7zWO5nP/eEeXT8j5nsfThUVbeD88UAGOJRp7fsx6/hnBF6OdRnIZFucJvAHuigia437L/TgJ2q+lkHy5O5/3xJpQThdSbQ/hpfP3USSkRygKeBb6lqZbvFK3CaTI4D/gd4tidjc52iqtOAC4B/E5HT2y3vDfswDMwG/s9jcW/Yh370hv34I6AJeKyDKof6LCTK74ExwBRgO04zTntJ33/AFRz87CFZ+8+3VEoQpcDImOcjgG0d1RGREJBP505vO0VE0nCSw2Oq+pf2y1W1UlWr3cfPA2kiMqin4nO3u839vQt4BudUPpaf/ZxoFwArVHVn+wW9YR8CO1ua3dzfuzzqJHU/up3iXwCuUrfBvD0fn4WEUNWdqtqsqlHgvg62m+z9FwIuBZ7oqE6y9l88UilBLAXGisho9xvmPGBhuzoLgZarRS4DFnX0z9Hd3PbK+4G1qvrrDuoc1tInIiLTcf5+ZT0Rn7vNbBHJbXmM05n5UbtqC4Gr3auZTgQqWppTelCH39ySvQ9dsZ+z+cBfPeq8CMwSkQFuE8ostyzhROR84AfAbFWt7aCOn89CouKL7dO6pIPt+vl/T6RzgE9UtdRrYTL3X1yS3Uvekz84V9h8inN1w4/csltw/hEAMnCaJUqAJcCRPRjbqTinwKuBle7PhcA3gG+4dW4A1uBckfE+cHIP778j3W2vcuNo2YexMQpwj7uPPwSKezjGLJwDfn5MWdL2IU6i2g5EcL7VXovTr/Uq8Jn7e6Bbtxj4U8y6X3M/iyXANT0YXwlO+33L57Dlyr7DgecP9lnoofgecT9bq3EO+sPax+c+P+D/vSfic8sfbPnMxdTt8f3X1R8basMYY4ynVGpiMsYYEwdLEMYYYzxZgjDGGOPJEoQxxhhPliCMMcZ4sgRhTC/gjjL7XLLjMCaWJQhjjDGeLEEYEwcR+ScRWeKO4f9HEQmKSLWI/EpEVojIqyIy2K07RUTej5lXYYBbfpSIvOIOGLhCRMa4L58jIk+5czE81lMjCRvTEUsQxvgkIhOAy3EGWZsCNANXAdk4Yz9NA94AbnJXeRj4gapOxrnzt6X8MeAedQYMPBnnTlxwRvD9FjAR507bUxL+pow5iFCyAzCmD5kJHA8sdb/cZ+IMtBdl/6BsjwJ/EZF8oEBV33DLHwL+zx1/Z7iqPgOgqvUA7ustUXfsHncWsiLg7cS/LWO8WYIwxj8BHlLVH7YpFPlJu3oHG7/mYM1GDTGPm7H/T5Nk1sRkjH+vApeJyBBonVv6CJz/o8vcOlcCb6tqBbBPRE5zy78CvKHOHB+lInKx+xrpIpLVo+/CGJ/sG4oxPqnqxyLyY5xZwAI4I3j+G1ADTBKR5TizEF7urjIf+IObADYA17jlXwH+KCK3uK/xpR58G8b4ZqO5GtNFIlKtqjnJjsOY7mZNTMYYYzzZGYQxxhhPdgZhjDHGkyUIY4wxnixBGGOM8WQJwhhjjCdLEMYYYzz9f4HlnBWrjeRoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i in ['Precision', 'Recall']:\n",
    "    sns.set_palette(\"Set2\")\n",
    "    plt.figure()\n",
    "    sns.scatterplot(x=\"epoch\", y=\"value\", hue='data',\n",
    "                data=compare_metric(df_list = [output1, output2], metric=i)\n",
    "               ).set_title(f'{i} comparison using test set');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Referring to the figures above, it is rather obvious that the number of epochs is too low as the model's performances have not stabilised. Reader can decide on the number of epochs and other hyperparameters to adjust suit the application.\n",
    "\n",
    "As stated previously, it is interesting to see model2 (using both implicit and explicit data) performed consistently better than model1 (using only explicit ratings). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Similar users and items"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the LightFM package operates based on latent embeddings, these can be retrieved once the model has been fitted to assess user-user and/or item-item affinity."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 User affinity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The user-user affinity can be retrieved with the `get_user_representations` method from the fitted model as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.6208545 , -2.0801244 ,  0.29000854, ...,  1.9506047 ,\n",
       "        -2.9648001 ,  3.3220475 ],\n",
       "       [ 2.532207  , -0.17591256,  1.4384304 , ...,  2.552038  ,\n",
       "        -4.923464  ,  3.7328699 ],\n",
       "       [ 2.7307076 ,  1.2757684 ,  0.40170258, ...,  0.53212243,\n",
       "        -5.6057224 ,  4.1384106 ],\n",
       "       ...,\n",
       "       [ 3.2812703 , -2.0771208 ,  1.802339  , ...,  1.3999628 ,\n",
       "        -2.4171133 ,  2.7771792 ],\n",
       "       [ 2.4532616 , -0.15192671,  1.4297578 , ...,  2.5044203 ,\n",
       "        -4.7720947 ,  3.591335  ],\n",
       "       [ 1.8169118 ,  0.9889456 ,  0.6572489 , ...,  2.2373357 ,\n",
       "        -4.187849  ,  4.355454  ]], dtype=float32)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_, user_embeddings = model2.get_user_representations(features=user_features)\n",
    "user_embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to retrieve the top N similar users, we can use the `similar_users` from `reco_utils`. For example, if we want to choose top 10 users most similar to the user 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userID</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>555</td>\n",
       "      <td>0.999998</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>54</td>\n",
       "      <td>0.999996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>314</td>\n",
       "      <td>0.999992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>465</td>\n",
       "      <td>0.999990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>395</td>\n",
       "      <td>0.999990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>282</td>\n",
       "      <td>0.999989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>411</td>\n",
       "      <td>0.999989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>527</td>\n",
       "      <td>0.999988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>481</td>\n",
       "      <td>0.999982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>881</td>\n",
       "      <td>0.999980</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userID     score\n",
       "0     555  0.999998\n",
       "1      54  0.999996\n",
       "2     314  0.999992\n",
       "3     465  0.999990\n",
       "4     395  0.999990\n",
       "5     282  0.999989\n",
       "6     411  0.999989\n",
       "7     527  0.999988\n",
       "8     481  0.999982\n",
       "9     881  0.999980"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar_users(user_id=1, user_features=user_features, \n",
    "            model=model2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 Item affinity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to the user affinity, the item-item affinity can be retrieved with the `get_item_representations` method using the fitted model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.11479151,  0.93140936,  1.5966691 , ...,  1.8705213 ,\n",
       "        -2.4184883 ,  2.2852988 ],\n",
       "       [ 0.6439721 ,  0.85031474,  1.6993418 , ...,  1.8807421 ,\n",
       "        -2.1288579 ,  2.0951574 ],\n",
       "       [ 0.3511901 ,  0.98630357,  1.5790751 , ...,  1.7143724 ,\n",
       "        -2.2341046 ,  2.0473025 ],\n",
       "       ...,\n",
       "       [-0.5794922 ,  0.45613813,  1.2313437 , ...,  0.8687049 ,\n",
       "        -0.91763794,  0.9281359 ],\n",
       "       [-1.5155623 , -0.10658365,  1.5712368 , ...,  1.788055  ,\n",
       "        -1.261617  ,  1.6786036 ],\n",
       "       [ 0.12454936, -0.19605719,  1.2740307 , ...,  1.2044019 ,\n",
       "        -1.3142295 ,  1.3720794 ]], dtype=float32)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_, item_embeddings = model2.get_item_representations(features=item_features)\n",
    "item_embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function to retrieve the top N similar items is similar to similar_users() above. For example, if we want to choose top 10 items most similar to the item 10:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>itemID</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>146</td>\n",
       "      <td>0.999761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14</td>\n",
       "      <td>0.999761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>44</td>\n",
       "      <td>0.999708</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>499</td>\n",
       "      <td>0.999683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1101</td>\n",
       "      <td>0.999626</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>613</td>\n",
       "      <td>0.999604</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>219</td>\n",
       "      <td>0.999599</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>373</td>\n",
       "      <td>0.999597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>706</td>\n",
       "      <td>0.999579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>305</td>\n",
       "      <td>0.999577</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   itemID     score\n",
       "0     146  0.999761\n",
       "1      14  0.999761\n",
       "2      44  0.999708\n",
       "3     499  0.999683\n",
       "4    1101  0.999626\n",
       "5     613  0.999604\n",
       "6     219  0.999599\n",
       "7     373  0.999597\n",
       "8     706  0.999579\n",
       "9     305  0.999577"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar_items(item_id=10, item_features=item_features, \n",
    "            model=model2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, the background of hybrid matrix factorisation model has been explained together with a detailed example of LightFM's implementation. \n",
    "\n",
    "The process of incorporating additional user and item metadata has also been demonstrated with performance comparison. Furthermore, the calculation of both user and item affinity scores have also been demonstrated and extracted from the fitted model.\n",
    "\n",
    "This notebook remains a fairly simple treatment on the subject and hopefully could serve as a good foundation for the reader."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [[1](https://arxiv.org/abs/1507.08439)]. Maciej Kula - Metadata Embeddings for User and Item Cold-start Recommendations, 2015. arXiv:1507.08439\n",
    "- [[2](https://making.lyst.com/lightfm/docs/home.html)]. LightFM documentation,\n",
    "- [3]. Charu C. Aggarwal - Recommender Systems: The Textbook, Springer, April 2016. ISBN 978-3-319-29659-3\n",
    "- [4]. Deepak K. Agarwal, Bee-Chung Chen - Statistical Methods for Recommender Systems, 2016. ISBN: 9781107036079 \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (reco_base)",
   "language": "python",
   "name": "reco_base"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}