examples/interfaces/automatic-differentiation-using-autograd.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using autograd to calculate the gradient of a log-likelihood\n",
    "\n",
    "It is straightforward to use the automatic differentiation library [autograd](https://github.com/HIPS/autograd) to take the derivative of log-likelihoods defined in pints. Below is an example of how to do this.\n",
    "\n",
    "WARNING: We currently find this method of caculating model sensitivities to be quite slow for most time-series models, and so do not recommended it for use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import warnings\n",
    "\n",
    "from timeit import repeat\n",
    "\n",
    "import pints\n",
    "import pints.toy as toy\n",
    "\n",
    "import autograd.numpy as np\n",
    "from autograd.scipy.integrate import odeint\n",
    "from autograd.builtins import tuple\n",
    "from autograd import grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We begin be defining a model, identical to the [Fitzhugh Nagumo](https://pints.readthedocs.io/en/latest/toy/fitzhugh_nagumo_model.html) toy model implemented in pints. The corresponding toy model in pints has its `evaluateS1()` method defined, so we can compare the results using automatic differentiation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "class AutoGradFitzhughNagumoModel(pints.ForwardModel):\n",
    "    def simulate(self, parameters, times):\n",
    "        y0 = np.array([-1, 1], dtype=float)\n",
    "        \n",
    "        def rhs(y, t, p):\n",
    "            V, R = y\n",
    "            a, b, c = p\n",
    "            dV_dt = (V - V**3 / 3 + R) * c\n",
    "            dR_dt = (V - a + b * R) / -c\n",
    "            return np.array([dV_dt, dR_dt])\n",
    "        \n",
    "        return odeint(rhs, y0, times, tuple((parameters,)))\n",
    "    \n",
    "    def n_parameters(self):\n",
    "        return 3\n",
    "    \n",
    "    def n_outputs(self):\n",
    "        return 2\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we wrap an existing pints likelihood class, and use the `autograd.grad` function to calculate the gradient of the given log-likelihood"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class AutoGradLogLikelihood(pints.ProblemLogLikelihood):\n",
    "    def __init__(self, likelihood):\n",
    "        self.likelihood = likelihood\n",
    "        f = lambda x: self.likelihood(x)\n",
    "        self.likelihood_grad = grad(f)\n",
    "    def __call__(self, x):\n",
    "        return self.likelihood(x)\n",
    "    def evaluateS1(self, x):\n",
    "        values = self.likelihood(x)\n",
    "        gradient = self.likelihood_grad(x)\n",
    "        return values, gradient\n",
    "    def n_parameters(self):\n",
    "        return self.likelihood.n_parameters()\n",
    "    \n",
    "autograd_model = AutoGradFitzhughNagumoModel()\n",
    "pints_model = pints.toy.FitzhughNagumoModel()\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now create some toy data and ensure that the new model gives the same output as the toy model in pints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABaNUlEQVR4nO2dd5gTVduH7zNJNrvZlu0FlgVpCioIWFEBO6BiQQEbVuy9vHaxYuVVwQJWVBCxgqJS7LyKUqRK79trNrubLUnmfH8k64e4PWWS3bmvay+SmZPz/JiceXLmlOcRUkp0dHR0dDo+itYCdHR0dHSCg+7wdXR0dDoJusPX0dHR6SToDl9HR0enk6A7fB0dHZ1OglFrAc2RnJwsu3fvrrUMHR0dnbBh1apVJVLKlMbOhbTD7969OytXrtRaho6Ojk7YIITY09Q5fUhHR0dHp5OgO3wdHR2dToLu8HV0dHQ6CbrD19HR0ekk6A5fR0dHp5Pgs8MXQmQJIX4QQmwSQmwUQtzaSBkhhHhZCLFdCLFOCDHIV7s6Ojo6Om3DH8syXcCdUsrVQohYYJUQYomU8q/9yowEenv/jgZe8/6ro6OjoxMkfHb4Usp8IN/7ulIIsQnoAuzv8McA70lPLOblQgirECLD+9mQpqLWwWM/ziI2Ipqz+g5lcJeeWkvS0dHRaRd+3XglhOgOHAH8fsCpLsC+/d7neI/9y+ELISYBkwC6devmT3lt5uddG7n1+ztwGfMA+GTvCxxkPpnpI++jm7XRjWw6Ojo6IYvfHL4QIgb4FLhNSmk/8HQjH2k084qUciYwE2DIkCGaZme576cncIlyJvV5gt5JWby66gN21n3HmZ+u4L4hTzBhwDAt5emEGKqqMv6Th9hWuRIVJ4dbR/DwsKvpnZyhtTSdEMLhrOPWr19mZckSVFz0jTuWR4ddzyGpXQNu2y+rdIQQJjzOfraU8rNGiuQAWfu97wrk+cN2oNhSnEcFGzks7gxuPnYMZ/QZxIIJU3niqBkIDDz55y1cPf8ZXG631lJ1QoTbvnmFTTULMCtxRCsprKmax3kLxvDQ0ndRVVVreTohwj2LXme57T1MIppoQxp/VX/FhQvP5c5vXwt4O/HHKh0BvAVsklJObaLYAuAy72qdY4CKUB+/n/7HJwghufqIsf84fk6/Y/jmgs9IVYbwu+0DRrw/kYLKco1U6oQKS7ev5fuit4mXh/PrxI/49YqPeOn42USRyRe5L3D67OspdVRqLVNHY0odlfxUNJdo9WD+uPJzfr38Q2aMmEsMPVhc+Conf3AVefaygNn3Rw9/KHApcJIQYo33b5QQ4johxHXeMl8DO4HtwBvADX6wG1B+K1yCydWVk3oe/q9zmXGJLL3kDU5KuZZy1nPGvAv5dc9mDVTqhAoz/5wHQvLOmc+jKJ7b6qSeh/PrxE8ZEjeBfPdvnPLh+fy8a6PGSnW05N4lr4OhitsG3/z3saHZh/DLZR9yXMJEitXVjPz4fL7ctCIg9n12+FLKZVJKIaU8XEo50Pv3tZTydSnl694yUkp5o5Syp5TyMCllSIfA3FCwlzrDboaknNRkGUVReGnUTdxx6HO4hZ1rv7uMN1YsCqJKnVBiZ+V6LGr3f43XGw0G3jn3fm7p/wwuYeeGHycydVljo546nYGVJUuwqH0Yf/iJ/zhuNBiYcfZdPHDEy0jh5P7fbqK46sCpUN/Rd9o2wq/7PL2woV1b3h925ZDTePOUDzDKBF7aeDc3ffWSPl7bySh1VFKr7KFn7L+fBhuYdORI3h85F7NM5+3tk7l6/jN6O+lklDuqcBoK6B3XdDuZMGAYn4/5hFsOe4yUmDi/a9AdfiNsLtkNwODM3q0qf3S33nw7bh6J4gh+Kn2TUXNuprKuJoAKdUKJL/76FSFUTuh2VLPlBmZ057sJH5FmOJrfbR9w+uwbqKh1BEmljtZ8t2MtQkgGpPZvtlzPpHSuOfL0gGjQHX4j7LHvQaom+rVhmVRaTDzfX/I2A2LGkuv+mZNnT2BHaUEAVeqECj/u+R0pBef1G9piWWtUNEsunsGQuAkUqP/j5Dnj2VycEwSVOlrza846AEb0GKiZBt3hN0JRbS4mNeXvybfWYjQY+OD8RxiffR8OsYfzvhivT9J1Arba12JWs0iPTWhVeUVReOfc+7nsoIepFTlc+OWEgE3S6YQOW8o3gTuKQZkHaaZBd/iNUOkqIN7Y/s0yDwy/iIcGv4wq6rjxhyt4d9VSP6rTCSVUVaWaXWRZ+rX5s3efcAHPHDcTJNy3/FpeWPZpABTqhAoFtTuJFtlt7kj6E93hH0C9y4VLKSbV0sWnesYdfgJvnvIeBhnP8+vv4sGl7/hJoU4osdtWjFCcdI1p3y7J0X2HMO/sD4mUXXln+6Pc9NVLflaoEwo4nHXUiVy6RvfSVIfu8A9gfeEehOKmR1x3n+s6ultvvhr7EbGyD/Nzp3LRJ5P1nbkdjI2FewHIjs9sdx0Hp3Rlyfg5JHkn/c+dew/1Lpe/JOqEAL/s/guhuDgsue1Pgv5Ed/gHsDpvOwCHJPfwS31d4xP57pL36WoczvrqTzlt9rX6yowOxPYyz4Rrz0Tf4qAkWGJYcvFb9DaPYnvdN5w6+yrKHVX+kKgTAqzO2wrAkV10hx9SbC7dCcCgLv579LKYzCyc8BLHJlxGsfydU+boK3g6CrsrcgHon5rtc10RRiOfjX+G4cnXUCr/5NS5F7GtJKQjkOi0ktxKz/3eN9m3oWJf0R3+Aeyx7/UsyUzJarlwG1AUhZln382lPR6iRuzlvC8uYkXOdr/a0Ak++dX5SNVAz8Q0v9U5bfQtXNHrEWpFDmPnT9BXenUAimuKkdJAtsZh1XWHfwAlNfkY1SSMBkNA6r/nxAt5YNCLqKKKKxdfxsItIR1lQqcFSmuLMKhWv7eXO48/n0ePmu5d6XUlH6z5wa/16wSX8rpSFHdswPxKa9Ed/gHUqHbMiv+3NO/PhAHDeGnYmwgM/OfX63h75eKA2tMJHJXOEiKVpIDUfX7/47wrvWJ4+s/b9WWbYUyVs4wIYdVahu7wD6ROrcRiCKzDB08kxQ9GvY9JTWTq+nt46qcPA25Tx//UyjLijIF7TD+6W28WnD+XSNmNd7Y/yv1L3gqYLZ3AUaPasBhatzEvkOgO/wDcoopoY3xQbB2e3p0F53+IRfZgzq4p3Lzw5aDY1fEP9S4XqsFGcmRqQO10s6awaPxs4mQ/vsx7kWsXPB9Qezr+xyVsxJkStZahO/z9cbndSKUaqzl4v8RZ1iQWT/iARHEEP5a8wcWfTtajKIYJ20rzEUIlIybwKQyTLLEsuWgWKeJofi2fxYUfP6C3kzChsq4GDA6SIrXPg607/P3IqShFCElCZHAfvaxR0Sy9+C26GoezrupTzp93r34zhwEbi/YA0MPa/k1XbSHabObbi14n23QymxwLGP3hrdQ664NiW6f9bC3xZHNNi+4gDl8I8bYQokgIsaGJ88OFEBX7ZcR62B92/c3eimIAUizBf/SKMBpZOOGlvzfejP7wVn23ZYizo8yzBr93kn+X8DZHhNHIgvFTGRAzlhzXj5wxZ5IeijvE2VHm2UuRFee/pbvtxV89/HeBM1oo88t+GbEe85Ndv5JrLwEgLVqbsTZFUfjkwil/38wjP7weh7NOEy06LbPP7um5+WPTVVtQFIUPzn+EYUlXU8oqRn54lb57O4TZY/NsuupuDfzQX0v4xeFLKX8GApd5N0jkVXocfmasdo9eDTfzcQkTKVKXc/qcq/UeXIhSWluKlAa6xGmz+mL6mbdyRvpNVIj1jJx7Fbaaak106DRPflUhAL2SgjP01xzBHMM/VgixVgjxjRCiyZQvQohJQoiVQoiVxcXFQZQHRdWe36wsjXfDAcw4+y5OTb0BG2sYNXcS1XV6Tz/UqKyvQKgWTcPdPnf6tZyVcRt2NjJy7hV6/J0QpNBRjJSCnonpWksJmsNfDWRLKQcA04AvmioopZwppRwipRySkhJcx1ta63H42fHJQbXbFFNHXs9paR6nP3LuNbrTDzGqXXaMMkZrGTx12lWc2/UOKsVmzvhoYkCSX+u0n7LaUoQaQ4TRqLWU4Dh8KaVdSlnlff01YBJChIZX3Q9bnQ2pmkiwaH8TN/DCGddzcup1lPMno+ZO0sf0Q4hatYoIJTTayuOnXM6F2fdQLbZx5sd6Tz+UqHSWYsKqtQwgSA5fCJEuhBDe10d57ZYGw3ZbsNfbUNRYrWX8ixdH3siI5EmUsZpRc67TnX6IUK9WEamETnt5eMQl/+/05+nDgKGCQ7VhUaxaywD8tyzzQ+A3oK8QIkcIcZUQ4johxHXeImOBDUKItcDLwHgppfSHbX/icFVgEqHRYzuQl0ffzIlJV1HKSs6ae5OeSCUEcIkqLMbAh+FoCw+PuIRRGTdhF+s586Pr9HX6IYBLVhJttGotA/DfKp0JUsoMKaVJStlVSvmWlPJ1KeXr3vPTpZT9pZQDpJTHSCl/9Yddf1OrVgY8cJovvHLmbRxtvYQidTljP9Y3Z2mJqqpIUU1sROi1l2dPn8QJiVdSIlcy5qPb9M6BxqhKDdHG0HgS1Hfa7odTVhIdhMBpvjDzrLvpEzmaHXXfcs2CZ7WW02kpdVQhFDdWs1VrKY3y6lm3c0TsOPLcv3D+vHv0zoFGOJx1CKWOOHNo+BXd4e+HW6ki1mTVWkazKIrCR2OfJF0Zyh8Vs7ln0UytJXVKGnZlJ0ZatRXSDO+ecz99Ikezs34xV3wxRWs5nZLcCs/Kv3hzcAIytoTu8L1U1tUglDqsIXwDN2A0GJg//iXi5KF8nT+d5375RGtJnY48u+dG1iIMR2tRFIWPL3iKDMPxrK6cy4NL39FaUqcj1+5Zm5IUZdVWiBfd4XvZa/Pssk0McuC09mIxmVlw4Uwi1e7M2v4kc9f9rLWkTkVBlcfhp8WErsMHj9P//IIXiVYP5oucF3lr5SKtJXUqCqvKAUiK0nv4IcU+7yN6agj32A4kyRLLJ+e8iUG18uTK/7Aqd4fWkjoNRdWenlt6iDt88ETZ/Oz8GZjc6by4/kGWbFujtaROQ7HDBmgXn+tAdIfvpaDS+4geHR49/Aa6J6by0ohpgIurv71R32UZJEprPD23rvGBSW/obzLjEnln1AyQJu76+Tb22oIbtqSzUuLwtJO0mNDwK7rD99LwS5xssWqqoz0MP+hQbjr0UZyGPM779EZ9GV4QKKu1AZAVp33cpdYyMKM7Dw55BrfBxvjPb9bX6AeBspoKALqESMdAd/heyms9X0xGbGj8EreVa48axYiUq7GxhqsXPK21nA5PRV0FUjUTbTZrLaVNjDv8BEZn3EilspFLP5+stZwOj63O41cyQ8Sv6A7fi63WMxSSHhsaY23t4aWRN5GuDGVlxUf65FyAqXbaUdRorWW0i2dOv4YeEaeyueZLnvxxjtZyOjSV9XZQI4k0RWgtBdAd/t/Y6+1IKUiPCY3Z9PagKApzzn0WozuNF9dNZmPhPq0ldVgcbjsmEZ4OH2Du2ClEug9i7q7/8vvebVrL6bBUO+0I1aK1jL/RHb6Xyno7Qo3EaDBoLcUnUmLimDr8BaSo44qFes7TQFGrVmEWobFdvj1YTGZeO+0FkHDj0jv1gHwBosZdhTGEOga6w/ficFWjyND5JfaFk3oeztjs26gxbOPaL5/TWk6HxCmriAqxwGltZUjXXlzU8y7qDLu44ovHtZbTIalVq4jQHX7oUeOuxCg6hsMHmHzSZaQqx7DKPo+FW1ZqLafDoYoqYsLc4QPcN2wc3Uwj+MsxX5/3CQBOWU1kiORMAN3h/029Wh3WY7KN8dboJxFqNA8ue1CPje5H6l0upFIbkpEy28MH5zyFwZXKS2ufotRRqbWcDoVbVBNtCp2hP93hewm1X2J/0D0xlasOvgeXMZdJX+nBs/xFQVU5QsiQCYjlKwmWGO4a/CDSWMakL5/UWk6HwRNC26E7/FDELWqIMnYshw9w23HnkGk4gbWVnzP/r9+1ltMhaAiclhDZMRw+wCUDR9A94hS21HzNpxtDMl1F2FFR60AobuIjrFpL+Rt/Zbx6WwhRJITY0MR5IYR4WQixXQixTggxyB92W4PqqKRy9lQKrhnF3pGD2X3KQPaNOZqyx6/Dteuv/y8nHMSE0C+xP3nrrCcQajSPLX+cepdLazlhT8HfAbGs2grxMzNGP4SixvHE8kf1VTt+YJ/dE5DRGhk6Q3/+6uG/C5zRzPmRQG/v3yTgNT/ZbRJ3SR4l91zM9uOOJOfxN7D9thO3ox6hCOoLKimc/RM7zj6P8ik3Ya+pRihOYiM6psPvGp/IBT1uoN6wh/uWvqG1nLCnuNqzezIl2qqtED+TGZfIFX3vxGXM45avX9RaTtiTXxl6HQN/pTj8GShrpsgY4D3pYTlgFUJk+MP2v7SoKhWvPsTO006ieMFqorLiyHp4En1XrKLHT+vJXvwnPX//i4Pen0ZU12gKZn1H0a0XgJRYQyQrTSB4cNhFRKt9WZz3DvtsIZc/Pqwo+TsCYmhsl/cntw89lwQGsbzsYzYV5WgtJ6z5OyBjCMXnCtYYfhdg/22fOd5j/0IIMUkIsVIIsbK4uO0R/dSSXApnfIwx1kT36Y+S9eUfxFx0OyLqnytwzEeeQtbCFSQMzcb58y7O+kNi7UBjsgeiKAqTh96PVGq5ffHzWssJaxoCp2WEcRiO5nh6+H0gXNyxRI/J5AsNHYPUEAmNDMFz+KKRY7KxglLKmVLKIVLKISkpbY9EaEjNIvutV+m+dBVRp1zYvChFIW3GV9T1jWHCjyq99+1us71w4ow+g8gyncjm6m/07fQtIFUVtaoCtariX+ca4i51iQudG9mfHJd9MIdYRrHP+TOLtv2ptZyQR9ZUo3qHb/an1OvwQykgY7Acfg6Qtd/7rkBeoIyZh5yEaGWwImE0svLa63CY4dC35yI7+KTm86f8B1B46OepWksJOZw71lM6+Rr2nD6IrQP7sWXIMWwZcgzbjzyE/ImnUr/es3rFXl+BlAbiIzvORr0DeeG02xHSzFO/vqi1lJDDlbONsieuZ++oIWw5/GA2HzGELUcex7bBh5B/2cnU/m8hAOXeSJmhlDPBGCQ7C4CbhBBzgaOBCillfpBst0helJnVJyvc/JWbilcfxHpLx32U7Z+WxcHRp7PZsZBfdv3FCT36aS1Jc+rX/kLxlAexry0EKTAnGYgblIUpIx3cKrXbd1Gxch+2C68kdexxVPWNQahRKErHXdXczZrCoPhzWF05l/l//c6YfkdrLUlz6v/6ndKnH6BiZQ5SFUQkKMQN7IopIw0k1O3aQ8WqXGxX3UnKmDlUDD4IKRVSo0NnqNgvDl8I8SEwHEgWQuQAjwAmACnl68DXwChgO+AArvCHXX9hq63gl0MFty5XKJ2zgPjrH2v1E0I4MuWkWzj3y0U8tuxllvR4XWs5mqHaSii+70rKftqKEJBwQk8SrrsH86Bh/yrr3LWRwjuvoejj37hgcBQvjrAGX3CQeebUGzj14wU8v2J6p3b4alUFpQ9dQ+midQDEH92NhGtuI/K4Uf8q687fTcFtl1I8fzVjd/zF+pGRIdUx8NcqnQlSygwppUlK2VVK+ZaU8nWvs8e7OudGKWVPKeVhUsqQCu5SUVcJQpB46VjqbRL7W09oLSmg9E7O4JDoM8h3/8qKnO1ay9EEx6I57Dz1BMp+2Eb84C70/OpT0mcubNTZA5h69KfLvJ9JOL47R6+qYdSGqiArDj7psQkMsp6NjTUs3b5WazmaUPPjp+w69ThKvllP7GEp9PxiLhnvLG7U2QMYMrqT+eFPJJ3ej0M21HLeytDazxA6Pz0aYq+3I1UTyVc9RESiQun7nyFVVWtZAeWhEycBgqeWzdRaSlCRqkrZE9ez57bHAMh+7h4y3/8OU4/+LX5WGI2kvTaf7VmC8Uvt1K1YGmi5mvPo8GuQqonnls/QWkrQsU29iz03PIBar5L16A10+egXTH0Gtvg5oSikvPARm3oZuPDHWmp+/DzwYluJ7vCBamclQkYhjEaSzj+DulI3NQtnaS0roBye3p0Mw7Fsq/menIrmtlB0HFRbCXnjTqTwgx+J6R1Pj6++xXJW20YXhSmCqWcm4DZC4UP/6fAdg+6JqfSKOplc1//YULBXazlBQXVUkn/5qeTPXIglO5oeX35NzLib21SHMBp5aVRXaqIEhY9NDpl2ojt8wOGqwuiNhR939f0oEZKyWW9qrCrw3DLkKoRSx5M/v6u1lIDjzt/N3nNPwr6hhJRzh9D1s/9hSMtuV10lsU5+Py6F6t0Oqj9+xc9KQ497j7saIVSe+/V9raUEHHdJHvvGnIhteQ5JpxxM1oLfMGb2aFddNnMdS09MpiavHvsrD/hZafvQHT6eJAUNWWmU+CSsx/WmcmMpzl0bNVYWWM465Eii3D35rWghaoj0QAKBc8d69owdTW1hPV3umUjylPcRxvatV3C53Uilll9PHIYpDopentHhl/Ie060vsWp//rR926EzqLlydrD3vNNx5NSQectYUqd/7tPiDVU4WH7UECJTjRS/9wWyrtaPatuH7vDxhEY2K/+/Ezfh+ntACmyvdPwsQKdmnY3bWMScdT9pLSUg1K//lT3jL8RZ4SbrqTuJu+I+n+orqq5ACElMdBIpE8+nrtRN1ZyOv6fh3F5jkQYbr/2xUGspAcG5Yz17LjybulInXR+cRPwNvt37no5BDdHmeJKvugRnJdjf1H4xiO7wAZd0EGX4/9DIEQNOILpnNLbv1yBrHRoqCzx3DL0A1Eje2zBPayl+x7l1DXuvvAq1XtLt5SeIPucan+tsCI1sjYwj7uoHMcVA6buzfa431Ln5mLMR7ng+2/ap1lL8jmvPZvZeNA6X3U3WU3cRe/EdPtdZUOXpGMRHxBNz8Z2YkxRKZn+u+dOg7vDxPHpZDoiFnzB+PC6HoPL9jh13JskSS7Z5KHnO3ylsJIxAuOLK2cbeiRfhrpVkvfQkUSeN9Uu9+0dAFOZIEs8ZTk1ePY5vOvb4dqQpgr4xJ1LOenaXFWktx2+4i3PZe8lYnFUqWc89QPSYq/1Sb26FJ0BhQmS8ZzHIxedRX6ZSNfclv9TfXjq9w28Yk405IDRyzPhbMMVA+bzQWVIVKCb0PwehOJmx4kutpfgFd3Eueyeci9OukjXlXqKGn++3uourGxy+Z/ek9abHMZglZTM6/uTtZYedixAq0//4TGspfkGtLGffhNHUlbjo+tANWEZe6re6C6o8T4LJFk8cnbgr78MYDeUfzPGbjfbQ6R1+w5hsfMQ/tz8LUwTW04/Csa+Wuj+WaKQuOIw79ASEO56le8P//ylrHeRcehZ1pd6b+MzL/Vr/3xEQYzw3smJNxnpiPyq32Kjv4BnFRvcdjMGVxi/5HaCdOOvJvWQkNTm1ZN56ITHjb/Fr/YXV3tDI3pwJItJCwulHUr3bQe1v3/rVVlvo9A7/7zHZRmLhW69/EKFIymd27Ek5o8FAT8txlKnrKGgk6l+4IFWVguvPxbG7hoxJZ/v9JgYorfEMe+0fGjnh5gcBKJ/esfPBKorCwMQRVIttrCvYrbUcnyi+fRxVWypIu2QE8dc95vf6GzoG6TH/306sNz2KMEjKX9NumLjTO/zCahsACVH/DnBk7Nqb2MNSqfh9F2p5xxm3bIxx/c5EKC5mrPhKayntpvypm7D9tpekU/pive3ZgNiweSMgZu4XGtnUZxCx/ZKwLduKaisJiN1QYeKAsxFCMmvNN1pLaTcV0+6jdOlmrMdmkfhgYJLvldbYgH92DIyZPYgfkkXFqhxcOTsCYrclOr3DL/LmJ20qK03C5degOgUVb3Ts3tvY/kPBHcdPOT9qLaVdVH3yCoWzvyembxwp/w3ciiN7nR0pFVIs/5zzSbz6OtR6QcXrjwbMdigwrHt/FFcSfxT8T2sp7aJm0Yfkv/Y5lm6RpL/2RcDsNJUzIfGGu5Buge2VyQGz3Ryd3uGXOLz5SWOsjZ6POv1izMkGyr/8LmS2RwcCo8FAl4iBFLvWh12ic+fW1eQ9Pg1zooEu734Z0EinlU47QrX8KwJi1OkXE5lqpGz+9x26nSiKQo/oIymXGyl3hFcAOdfereTc9yjGGEGXdz9DBDCfgb2+AqkasR6Yae/o04nuYaF88UpNlnx3eodf7h2TTY9pPCuNUBQSzhxBXbGbmiVzgykt6JyYdTwYavhi03KtpbQaWesg97orkG7oMm06SkJqQO1VOSswyJh/HReKQsLY0dSXq1R/FphhglBh5EEjEIqLD9Z+p7WUViNdLvKunYC7Drr+9/l2h0toLVXOSoRs/Acl4ZKLcVWD/d3g593QHX5dwyRc02nI4iY9gGKS2N7t2BEDLxlwClIKvtz6g9ZSWk3RHeOpyasn4+bxmAeNCLi9GtVOhPi3wweIu/oBDJGS8lkdO/DeRQNGINUIFu0Kn3ZS+sBEqnc5SJt4OpFDRwfcXo2rEqOMbvRczLhbiIgXlH30RcB1HEind/j2ejtSimaz0hgS04k/Mhv72kLNJluCQTdrCpFqdzZVrNBaSquofP95yr7fhvXYbsRdOzkoNuvVKiINsY2eUyyxJJw0gKptldSv+TkoerQg1hxFgjiEnJr1WktpFY4v36F4wSriDk/Getd/g2KzRq3EJBp3+MJoJOGs4dTmO6lZ9GFQ9DTgF4cvhDhDCLFFCLFdCHFvI+eHCyEqhBBrvH8P+8OuP6iqr0SokRgNhmbLWa+5HakKKmZoHw8jkPSzHkWtspt9tlKtpTSLa89m8l94E3OKgbRpwQsL4RJVRBub7hxYb34YFEnZKx03TSZA/6QjcBuL2Fyco7WUZnEX7iF38rNExCmkv/4xIkjZp5zSgdnQ+JMggPXGySgmSdmbwd2w5/P/XghhAF4BRgL9gAlCiMYSpf4ipRzo/fP/wtd24nBVoTQx1rY/kceeQVRXM+WLlmseDyOQnJR9DEJIFmz+VWspTSJVlfybJ6LWQ5cX/osSE5ycoaqqIkU1sRH/3rPRgKlHf+IOS6Vi+U7cJXlB0aUFp3Q/BoAvNoX2ap2CWy7B5ZBkPvM4hsT0oNl1ySosTTwJAigJqViH9sa+oQTn1jVB0+WPn7ujgO1Syp1SynpgLjDGD/UGhRp3JUbRutn6hLFn47TToWOgn3XI0Uip8L+cVVpLaZKK/95N1VY7qeNOxHzUqUGzW+qoQihurGZrs+USJ93kWcr7Wsj0a/zOqIOHIFUTy/NCKlvpP7C/+Tj2tSUkjx7o1/AarUFVqomNaL4jknCzJ0Z++fTgtRN/OPwuwL793ud4jx3IsUKItUKIb4QQLeeTCxJ1qqPJSbgDiZv4HwxRkrIPPgiwKu1IssRiVruys3KD1lIapX79rxS+sxBLdhQJD7waVNt7bJ7Nd8mRic2Wizr5QqIyTJR99XOHfRq0mMzE0ou9jtBsJ649mymYNpvIVCPJT7wdVNuljkqE4iTB3Hw7ieh/DLF9rdh++gu1IjhDqP5w+KKRY/KA96uBbCnlAGAa8EWTlQkxSQixUgixsri42A/ymufAWPjNIaKiSTjlCKp3VFG3OnxWKLSVLEs/qtgZcskupKqSf+eNICDzpTfancSkveTaPTdlQ0Cs5kgYdw7OCknVRy8HWpZm9IkfQL2SG3IpMqWqkn/L5ahOyHx+akDX2zfGbu+u/KQoa4tlE668BnedoGJmcHJv+MPh5wBZ+73vCvxj8FJKaZdSVnlffw2YhBDJjVUmpZwppRwipRySkpLiB3nN424kNHJzJNz8qCe+zvSOOyl3ZPoRCMXJku1rtJbyDyr+ezeOvbWkThyF6eDBQbdfUOVx+GmxzffcAOIuvxejRVI+W9voiIFkePZRITnfU/nm41RtqSBl7NCgDvk1sK/C01FNi27Uxf0Dy5lXYE4xUP75kqBs2POHw18B9BZC9BBCRADjgQX7FxBCpAshhPf1UV67IbEMRAoHMaamJ+EOxNitD3FHZGL7fQ/uwj0BVKYdo/seB8CSnaET/dG16y8KZy3EkhUZsDg5LdEQAbFLbMs3soi0kHDqYKp3VlO3YmmgpWnC6b09P7qr8kMnFai7aB8Fr31IZKqRxIe02QCXV+mJp5QRm9RiWaEoJJ4/iroyFcdngd/n47PDl1K6gJuARcAmYJ6UcqMQ4johxHXeYmOBDUKItcDLwHgp5YHDPkGnotaBUFzERjQ9m94Yidff7omHMX1yYIRpzMCM7gh3PJvKQmd8tvDuSUgXpD8zNehDOQ2UODxxl7LiW3b4ANabJyMUSdmr2vxABZrMuEQUVxI77Vu0lvI3RfdcibsW0h97LKAhNpqj0PskmBXfuhGKhg17ZbPeCaQswE/r8KWUX0sp+0gpe0opn/Qee11K+br39XQpZX8p5QAp5TFSypB4Bsyv9PTY4s1tW9YXefxZWLpFUvb18pBITBwI4pRulNSHxhNM5eyp2DeUknzm4KDspm2K8lob8O+AWE1h7NqbuCMyqfhjb4d9Gkww9qDMuUtrGQA4Fs7CtjyHxGG9iRp+rmY6ih0ev5JtTWtVeSUm3rthzx7wDXudeqdtQ7o6a2TbevgAiZdMwFUNlbOe8beskKBrdE+chgKq6+o01aFWllP44htEJCokPTpTUy32ehu4o4hsQ88x8brbvE+DHTOK5kHxfVCNJX/nldAKWVNN/pPPYoqBlClvaaqlvLYcKRUyY62t/kzCzY+AAmUBnhvs1A6/qMoGQHIToZGbI2bCbZjiBaWzP+2Q0RH7JfdFCDfL9v6lqY7SR6/HWQnp/7kdEdW61VSBosplR2kkcFpzRJ5wNpasSMq/+Q0ZYque/MGg9EMB+G7HGk11lD19K/VlKmm3XR3wAHotYXfaEGr0vyKqNoexRz/iB6R5NuwV5wZMW6d2+IXevJNpMa17RN8fYYog6bxTqC104vgyuOt8g8FxWYcBsHyfduP49X/9Tuk3a4g9NNFvyaV9ocZlx9TKPRv7kzDhApxVUPnBCwFQpS0nH3QEAL/nahdXx7VnMyWfLSO6Vwyxl9ypmY4GKp0VGGXbRw0Srr0F1RXYWPmd2uEXVHlm07Pj29cjiL/hUQyRktKZr/tTVkhwXPbBSNXAxtLNmmkoeuA2ANKemqaZhv2pk5VEKm13+LEX34EpBspmfxQAVdpySGpXcMeyzabdxG3RAzeguiHtMe1SB+5PrbsCs9J2hx81/HxP+JaFywL2NNipHX5Rjcfh90hs3eTKgSixCSSeMYTqHdXULvvSn9I0x2IyE6FmkFutzYRc9SevUbnJRvKZgzH1GaSJhgNxyioszQROawphjiTx7BOpyanD8c37AVCmLbFKNsX1uzWxXfP9J1SszCNxeB/Mg4ZpouFA6mUlFkP74jslThiLsxKqZgcmj3andvjlteVI1USSpe2/xg0k3P4kilFSOu05PyoLDZIjsqlU9wbdrqyrpXDqdExxkPhQcMMnNIeqVBNrat+NbL3lSQxmSemrofG04k9SI7tSLwpRgzyXJVWVwicexxAFyY+FzlO2W1QR0852EnvJXRhjoGxOYJ4GO7XDt9fbUNT2O3sAQ1o21uN7Y19XRP1fobNRyR8cZO2NNFSwuyy4Cdwrpj9AXZlK6rUXBy0SZkt49mzUt3kJbwOKNZmE0wdRta2S2v8t9LM6beke3x2h1LMpgJONjVH5xmPU5NWTeumZGJIzg2q7KWqd9UilhviIlsNvNIYwR5I4eihKVARqVYWf1XVyh1/triBCtH6XbVMk3uGJg1E2dbLPdYUS/ZN7AfB7TvDGZ1VbCcWzFxKVGUHsFfcHzW5L7CwrBCApqu0T/A0k3jnF8zT4UscKy9Ev+SAAVuZuDZpNWVNN0VsfYU4yEH/zU0Gz2xL7KkoRQpIY1T6HD5D4yEyy5v8ekM5Op3b4tWoFkYrvDt/UZyDxgzOx/boL567Q2WbuK4el9QRgU0nwxvHLnroVl0OQevddQUtW0Rp2lHrCQ3WJbd98D3ifBk/sg319MfXrQjuOfFsYnNkHgI3FwcsGV/7CXTjtkHrTNZrtqG2Mhjg6qZb2dwwC2e5D547SABd2YoxWv9SVfM/jSAllT//HL/WFAoO79ERKwU5bcMbxXXu3Uvr1KmIPjscy8tKg2GwteyoKAOhu9S2JRuKdTyIElL4w2Q+qQoMB6d2RqpFdFbuDYs9dkkfJJz9g6RZJ9Libg2KztTQ4/PSYluPoaEGndfiqqqKKKuJaSGbRWiIOH0r8wHTKl23HtTd4j7aBJNYcheKOp6A6OGnsSh69BdUNKQ9PCYq9tpBb6ZnH6JXo21ixqedhxB/dDduKfTg3h26SmbZgNBgwqSkU1OxrubAfKH3yVty1gtR7Hwipp0D4/zg6XeICH+m3PYTW1QoiBVU2hOImKdJ/v8RJdz+CVKHs2Xv8VqfWRCmplDsLAm6n/q/fKf91N9ajszSNl9MURY4ipBT0SvI9TV7yfzw/aCVP/yv9c9gSZ8yg0hX4duLa9Rdli9cTd2gSUSeNDbi9tlLoaFvgtGDTaR1+Q5KCVIv/HL550AjiDk+h7MfNuHKCN54ZSBIjMqmlMOB2Sp66DyEg+cHQ2DxzIGW1JQg1uk1xdJrCdPBgrEdnY/tjH85NK/ygTnvSLVm4DMXUBzjDV8mUu5EqpDwUek+BAIXVno5Bz3bu7Qk0ndbh763wOPw0P4+1Jd/5MNIFZc/e7dd6tSIzuisYqigMwBKxBurX/kLFqjysx/fE1GtAwOz4gt1Zhgmr3+pLvm8KAih5OnRWIvlCj/hshHDzZ17gJvidO9ZjW7aD+MGZRAw4IWB2fKG0thihxvqlYxAIOq3Dz7N7dtlmxfk30JL5qFOJOzSZsu//wrVHu7AE/qJnQjcAVuVuD5iN4ikPIhRIvi90N6/VqOVEKVa/1WfqMwjrMZ5efkfYv3FoqmcJ76r8wC3hLZ1yD1JC8n2hswzzQCqcpUTQ/iWZgabTOvyCaq/Dt/o/sl7y/U8i3VDypPaBnHylf4pnjfXGop0Bqb9uxVLsawpJGN4XY49+AbHhD+qpINbY/qV2jZF07zOeFTsdoJd/aFoPAHbZArP5yrl1NbZfd2E9KouI/scExIY/qHGXEW3wbzvxJ53W4ZfUeCZXegQglKp50DDih3TBtmxH2K/EGNylNwDbywOTwKP4mUdQjJB0X2Bih/iDepcLqVSR6McJfvDs37AedxC2FbnUrf7Jr3UHm4OTM5FSkFeVH5D6S6bciwSS7wvtTWsuUY41onUZ0bTALw5fCHGGEGKLEGK7EOJfSw+Eh5e959cJITSPhlVeWw5qJPEBymif8oAnrV3JU+G9YqdrfCK4LeRU+X/JXd2KpVRuKCXh5P4Yu/b0e/3+Yld5EUKopFgC8DT40FSEAYqfDO9efqQpAkWNo6TG/2E4nJtXYft9L9ajszVJXt9abDXVYKghJUrbePzN4bPDF0IYgFeAkUA/YIIQ4sBn85FAb+/fJECb7ML7YXeWo6htD3XbWkwHD8Z6fM8O0XuLIAlbvf9v5NKpTyAMkHh3aOd83V7m3WUb4/8b2Zh9MEmnDaByYxk133/i9/qDSQSJ2J0lfq+39PkHAc9EdyjzV5GnU5QRE5ordMA/PfyjgO1Syp1SynpgLjDmgDJjgPekh+WAVQiR4Qfb7cbh8k8cneZIfuAFT+/tqfDuvVkMSTjc/k1hV79xORVrCkg4vldI9+4Bdpd7him6+bjLtikS7/8vhkhJ0bOhOxnZGmKNSdSopX6t07VnM7bfdhE/uEvIhMluim2lnvmL7PjQCOTWGP5w+F2A/Z/3c7zH2loGACHEJCHESiHEyuLiYj/Ia5xa1UaUwRqw+mG/3tuGMmoWfRhQW4Ek3pSES9j8Wmfp8w8jBCTe/aRf6w0EuXbP001PH3fZNoUhOZPk84bh2F1D1SevBMRGMEg0p+FSyv0aJrns+fuRKiTf+Yjf6gwUeyo8HYNeSR3b4YtGjsl2lPEclHKmlHKIlHJISkrgdqu5lHISzYEfa0t6aBqGKEnhM0+Hbe7b5KgUMDioqHX4pT7nro1U/L6X+COzQnbd/f4UODwOv08AQ/Ba73oeUywUvfQaMsCblwJFmiUNoTjJsfunl+8u2kf5j38R1z+ZiIEn+qXOQJJb6dlpfEhKV42VNI0/HH4OkLXf+65AXjvKBI2CynJQakmzBH6sTUlIJeXiUdTk1VP5duj3ZhsjM9ozlLG1xD9fWdmz9yElJN31qF/qCzSlNSXgjgrYBD+AYokl5coLqCt2UzE9PIcAs+I8P4gNY9m+Uj71PlSnIOnW8Fj4UOQoQqpm0mM79jr8FUBvIUQPIUQEMB5YcECZBcBl3tU6xwAVUsrArN9qBQ0NsktsYMZkD8R669OYkxSKZs5BdVQGxaY/ybJ6plu2lvoeRM1duIfyX7YSd3gqEYcd53N9waC0rgATgY9+GHftZKIyTBTNWoC7LPBxafxNz0TPKO22Ut8dvlpRStm3K4nuFUPkCWf7XF8wsNUXY1RDI2FPU/js8KWULuAmYBGwCZgnpdwohLhOCHGdt9jXwE5gO/AGcIOvdn1hi9dx9UwMzqOXMEWQest1OO1Q/sztQbHpTw5K8Dj83eW+9/DLX3oI6RIk3Rw+oSeq3UXEGAI//CcUhbT778ddIyh99MaA2/M3/VI8u7J3V/jeTmyvPIy7VpB8/U0+1xUsqlxlRCqhu+kK/LQOX0r5tZSyj5Syp5TySe+x16WUr3tfSynljd7zh0kpV/rDbnvZXe6ZTT84OauFkv4jZtzNRB8UTclny8Iu5ELDmGRupW9B1GRNNeXf/kF0DwuRx5/lD2kBR1VVnEopKZHBWVQWdep44gamUrZkY9iFXOiTnImUCvlVPrYTZz1l838gKsOEZfREP6kLPHWynDhjaMbBb6BT7rTNqypASsEhqcGdXEl74jlUNxTdf31Q7fpK17gkpGqkyMdNNRUzH8PlECRefrl/hAWBraV5CMVJ19jgtZXUx14CAUUP3RE0m/4gwmhEccdRUuubw6+aPRVnhSTxotALf9wUtc56VEMFySG86Qo6qcMvqS1EqLFYTOag2jUPGkHSSQdTsaoAx6I5QbXtC4qiYFDjKa9r/6YaqaqUzVuIOUkh+oLwGa5Ym++J/tgrIXhPg6Y+A0kadQSVG8uo/kTzPYptwiySfN58VfrBXEyxEHtZeEzWAqzN34MQKt3jsrWW0iyd0uFX1BdjRpuxtuTHZ2KMhsInpyCd9ZpoaA9mkUiVq/3L7ao/e426UjeJF4wOuSxFzbGl1BNDqH9qj6DaTXrkdUxxUPD8tLCa6I81JlGrtn+TXs1386jJqSPxzBMQ5kg/Kgssawq2AXBISnDbSVsJnzvPjzjUMmIM2oy1KQmppE0aR22RC9vz4fPIHmNMpFYtb/fny95+B0OUJG7SQ35UFXh2efP5HpFxUFDtKjHxpN95A/U2Sdnk61r+QIgQF5GIW7T/B6rs9ZdRTJL4Gyf7T1QQ2FLqeRIclNlbYyXN0+kcvqqquJRyEoKw6aopYq95GEu3SIo+XIpr11+a6WgLCeZk3EpFu3ZR1q38nuqd1SSeNhjFEhsAdYGjwJEH7lgSLIGLu9QUMeNuJrZ/IiULV1G//teg228PCeYEMNRQXVfX5s86N6/CvqEE6wl9MARwk1sg2GPfh1RNHJLSaACBkKHTOfy8ShtCqSM9Ojhr8BtDKArpT09FuqDgrms009EWUi2pCMVJrr3tvfzymS+AIrHeGPrb4w+kvL6ASLTLT5r2zKsIAQX33hoWO7WTvSlDd5W3feK2fPoTACTeEn7tpKg2F5OaghLiw5WhrS4AbCryPKJ3DdKmq6YwDxpB8tlDqNxYRuWsZzTV0hoaNqltLm7bphrVVkLFbzuIOzQFY7c+gZAWUBxqMfEm7aIfmnoNIOWCE6neUYX99dB3hOnRnljwbXX4alUFtmWbiO1rDekQyE1R5Sog1qitT2kNnc7hb/XuAgzWpqvmSJo8E3OSgYKX3sFdHJhMQf6iS6ynl5tjb1tAO/tbU1CdgoSJVwdCVkBxOOtQDeWkWbQdXki4fzpRGSYKZ3wc8ns4usR520ll29pJ5TtP464VJFx2eQBUBRaX241TKSEtKrSHc6ATOvwtpbsBGBjkSbjGEJEWMh59GFcNFN1zhdZymiUzztNzy69q25K78vmLMScpRI28NBCyAsq6/N0IIcmODd6SzMYQpggynp2K6oSCO67UVEtLdIv3zI0VtNHhl322kAirwHLOpEDICigbivYhFBfZcd20ltIinc7h763ci1TN9AmREKZRp1xI4om9sP22L6RD42ZbPT23Ikfrl9zVfDeP2iIX1tHDw2opZgO/7fNMqA/M6KuxEjAfeQrJY46kcmM59jcf11pOkxyU6Bn+KnK0fglvzY+fUpvvJGH0iWHZTv7M8yzJ7JsU2ksyoRM6/KLaHMwyNaQmV1Kef5+IBEH+lOm4CwOTO9ZXunkdfllN6ydty99+DWGUxE96IFCyAsqG4i0AnNi9v8ZKPCQ9MoPIVCMF02fjytmmtZxGSY2OR6pGSmtb7/DL33zF006uC68luw1sKvEsyTwiI7QT+UAndPjV7nysptAaa1NiE8h88jFcDknBLaE59GExmcEdRUVd6xy+O3839jX5xB/ZLeyW2DWwp3InwpUQMuFuRaSFjKefQa2H/BsvCclVO4qioKgxbWsnf+YRPyQLQ4gvaWyKHbadSNXI4endtZbSIp3K4VfW1eA2lJFh0X7C9kCiThpL8uiB2NcWY3/jMa3lNIoiY6h02lpV1jbjSaRbkHD1rYEVFUBK6/cSZwitthJ53ChSLjyBqi12bM/dprWcRjGJOKpdtlaVtb32uKedXHNbQDUFkjzHTsyyCxFGo9ZSWqRTOfyVudsRQtIrITTH2pKfeJvINBMF0+fg3Lpaazn/IkLE4XDbWywnXS5s3/yPqMwIIoeODoIy/1PrrMepFJBpCb22kvjga0T3sFD43mLqVn6vtZx/EanEU6O2rp2Uf/srUV3NYdtOVFWlSu4l1dxdaymtolM5/DX52wE4NCU0x9pEpIXM/05DuiH3+iuRfkop6C+iDHHUteJGdnzxBvUVkoTzwyMEcmMs37cVobjpmxh6W+WF0UjG9HdRjJB3x63ImmqtJf2DGKMVp2y5nVR/8ipOOySMDY8EJ42xqTgXDNX0sobHHpNO5fC3lO0E4Kis0P1yzIOGkX7DBdTk1lF89yVay/kHMcZ4XK2Ik1I++z0MkZLYK/4TBFWB4Y8czwqdo7r001hJ45h6HkbG7ZdTW+Si8ObQCiMcH5GAqlS2GIajfPYHnvhKE8O3nfy0ey0AQzJCY2K/JXxy+EKIRCHEEiHENu+/jc5uCSF2CyHWCyHWCCE0S36yr3IvuC1/rzgJVeJveBzrUZmULtlE1UfTtJbzN3ERVqRS3eyN7Ny+lsrN5ViPPzjs4ubsz4aSrUgpOD47NB0+QOzl95I4rCfly3ZT8frDWsv5m8SoRITioriZKJ/OTSuo2mbHOvwwRFR0ENX5lz8LNgFw0kEDtRXSSnzt4d8LfCel7A18533fFCOklAOllEN8tNluSupyiCT0tz8DpE2bhzlJIW/KKzg3r9JaDgCJkYkIoZJXaWuyjO21p0CC9drwiWXeGHsrt2NwJ2sSNK0tpL40j6guZvKnz6NuxVKt5QCQEuWNp9NMXt7yGc+AgIRrm3MZoc9O+1aEy0qWNbQzXTXgq8MfA8zyvp4FnONjfQHFIfNJjAiPpV9KfBJdpk1HumHfNRNRK9ofi95fpFg8OQT22BqPkyJrHdh+WEt0z9iwSVDeFGXu7SRHhOZcz/6ISAtdZsxCMULOrbeilvuWlcwfZMR6dmXvsTWuRdY6sP20gZhecWEZN2d/yur3EGcI/R22Dfjq8NOklPkA3n+bijksgcVCiFVCiGb3TgshJgkhVgohVhYXt217dnNsLs4BQyW9rdrvmmwt5kEjyLx3EnXFLvKvOUfzdddpMZ5eTE5F4+EVKue8iMshSBg/Lpiy/M66gt1IQwX9Eg/TWkqrMPUaQJeHb6e+3E3u5WdpnlgnK87jBnKbiLtUOXsq7hqBdfz4YMryO+WOKpyGQrrFhH7HoIEWHb4QYqkQYkMjf2PaYGeolHIQMBK4UQhxYlMFpZQzpZRDpJRDUlL8N9b+w651QPhMrjQQe/EdpJx3FPZ1JZQ9pG0clUxvzy2vsnGHXz7vE4wxEDP+lmDK8jvfbv0DgGHdwqf3GX3etaRdejJVW+wU3XaBplq6xnvu24LqxtuJ7ePPPe3kwpuCKcvvLNj8O0KoHJU5SGspraZFhy+lPEVKeWgjf/OBQiFEBoD330af4aSUed5/i4DPgaP8919oHavzNwJwcs8jgm3aZ5KeeJe4Q5Mo+nQ59ref0kxHlvdGLmzkRq5b/ROO3TUknDIYYYoItjS/srJgLVI1cHqf8LmRARLvf4WE47pR9t1WbP+9WzMd3ayejkF5bcW/ztWv/YXq3Q4SThoU9u3k570rADjn4PAZvvR1SGcBMNH7eiIw/8ACQohoIURsw2vgNGCDj3bbjGdyJTFsJlf2RygKGe98SVSmmbzn38OxcFbLHwoAByV4AmOV1tj+dc424zlPkpPrHwyyKv+zu2oTkbIbseYoraW0mbRXPye6u4X8N76k+rPXNdGQEBmNVI1U1Nv+da585vMgJPFhPlkLsMW2HoMrje6J2mXPayu+OvyngVOFENuAU73vEUJkCiG+9pZJA5YJIdYCfwALpZTf+mi3zZTW7ybeGD6TKweixCbQ9YMvMMUp7LtviiYrMhIsMUjVRHndPyNmqhWl2H7bTly/ZIzZBwddlz+pddbjELvpagmfuZ79EZEWurz3BeYEAzkPv0jN958EXYOiKAhpobL+n5uvVEclFcu2EHtwAqae4TE/0hQut5sKdRsZkYdoLaVN+OTwpZSlUsqTpZS9vf+WeY/nSSlHeV/vlFIO8P71l1I+6Q/hbcFWU43LUEj3mNDbNdkWjJk9yHrrbYQB9l5/sybhFxQ1FvsBPTf720+j1gsSJl4VdD3+ZtG2PxGKk0FpA7SW0m4MqVlkvTcXQ5Rg3x0PahJ+wShjcLj+6fAr330Wd53AenFobShsDz/v3giGGo5IHai1lDbRKXbafr9zHUJIDk8Nr1/jxojofwzdXnoatU6y55KLcW5fG1T7JhFDteufY7PlXyzCnKgQNXpiE58KH77a9jMAF/Qfrq0QHzH1PIxub84EYO+1N+LctCKo9iOUaGrVf268sn3+FaY4iD7n2qBqCQTfbl8OwMhex2qspG10Cof/6z6PUzwxe6C2QvxE5Inn0G3qZNw1kr0XT8C5a2PQbJtFLHVq1d/va76bR22hE+uoYWGZvOJANpStwOjK5JDU0IqS2R4iBpxAt2nPojoley69jPr1vwbNdqQSS736/zF+6v5YgmNfLQmnHo0Ig6iSLbG6aCW4YxkaZkOY4X+HtoL1JevAHcORXcNnvWxLRJ06nqxnH8RVrbL3oguC1tOPMsbikv9/I5fNmIZiksTfEDpb+9uLraaaSrbRI3qg1lL8RuQJZ9Pt5Skep3/5VdSv+Tkodi3GONzi/9tJ+YwXQJHEX3t/UOwHknqXi0LnWjIiBoRUIqXWEF5q20lB3WYSDL3D7stpCcvIS8h6+l5cVSq7x48PylhttDH27xvZuXUN9vXFWIf2xpAYHiErmuPjDcsQiouTuh+vtRS/EjX8XLq9+gLSLdl9xSRqfwv8molYUxyq8MRdcuXswLZ8N/FHZGDsFrqBC1vLV1tWgMHB8V2Gai2lzXQsD9gIO0oLUI3FHJwQ3qsCmsJy5uV0mzYF6ZLsueoGapbMDai92Ig4pOLA5XZT/vKjACTcEv5LMQGW7v4FKRXGHTpMayl+J3LoaLJnTkMI2DPptoAv2Yw3WxGKi4paB+UvPYh0C5JuCf+lmAALtn6PlIJLBpyqtZQ20+Ed/pdbPJMrw7KP1FhJ4Igafi7dZ72BYhbsuW0y9reeCJit+Ih4hJAUbl9D+Y+biO2fRES/owNmL5hsta8gWvYiJSZOaykBwXzUqXSfMxtTnIG9D7yI7cXABbhLiIwHYO++rZQv+ZPoXjGYjz49YPaCycbyP4hUs/9O2B5OdHiHvzx3FVIqjO7TcR0+eCboun/0KebUCHKfm03RLecFJKZKYpQVAPtz96O6IeXex/1uQwv+t2cTLmMeR6Y0GfWjQ2A6eDDZXywmOjua/Ne/pPD6swOSaCfF4omU7p4+GXetIPnG8A630cDOskJqlF0cHK9Z0F+f6PAOf1fVRiLVbljDOOZ2azH26Ef2wv9hPborpYs3sW/McbhytvnVRrIlgSS7RP62h/gj0jEPOcmv9WvFrLVfAXDVoPDNvtRaDCldyPriFxKGZlP2wzb2jDzG7yG4UywJxFdLIn/cSuzB8VhGXurX+rVixooFCCEZ12+U1lLaRYd2+BW1DqrZRXZ0eAVM8wXFEkvGrCWkXz0Sx+4qdp55ll/j76RFxTPpGxUEJD/4jN/q1ZrVJb8Q4c7miMzQy2EbCESkhfS3vqXLHeOpK6ln17iLsc981G8RWTPikrjkexVckPJwx2knv+QtRXElM7pv+ATW258O7fA/9a66GJYVXpsj/EHCXVPp/taLmOJM5D77PrnjTsC1d6vP9R684H2O2CnZcVqfDjN2v6FgL3WGXQxMPEFrKUEnbtIj9PjgDUxWE7lT55Izxj+9/e5fvMuwDZIdQ9MxD+oYk+A7ywqxi830izshbFf8hafqVrJ09zKkVLjwsI49LtsUkceeQffFv5N81kDs64rZMfpsSv5zcbuTqVR98gqRH/3K+mzBDyef5me12jHtj3kAXHlExx/OaYyIASfQffEKUscfT/VOOzvGXkzJPRfjLslrV301iz5EzlrEX1mwYGT4rWRpitdXzEcIlYsOPUtrKe2mQzv8rfY1RKndSY9tNNVup0BEWkh57kMOev9VLD2tFM9fzY4RQyl96EpcOTtaXY/9jcfIeWQaEYlGpp6rYHO2nMw8HFBVlT+KFxPpPoih2eEfeqO9CHMkSZPf4KCPZxHd00rxgtVsP+kkj+Mv2tfqeqrmvszeOx/FaBG8fKaZcndVyx8KE37M/RbFlRK2wznQgR1+nr2MWmU3vePCL/59IDAPOYmsL5aT/cK9RCRFUfTxb2w/bTS5406k8v3nG02NJ1WVmqXzyDn/OHJf+BBzSgTZcz6jyhxJRd2/Y52HIws2/4HLmM/wLuE5CedvIvodTdb85XR//Qks3eMpXrCabSNOJXfcidjffgp3ce6/PiNVldqfvyB3/DD2TX4NU5yB7A/nUR4bS9UBETPDlV92/UWNYRtHJp8etsM5AOEf1KIJPlr/E0JITu0RfrvhAoll9ESyR0+k7vdFlL/xEhW/78S+9i146k0i4hVMiRaUCCPu2nrqChy4awWKSZJ85kCSn3gbEWlBUS1UOTvGjfzO2o+RqpFbj9E2S1SoETX8fLKGn0/tLwsof/sV7Kv2YF/7Pjz7PhHxAlNCFCLShKxzUlfkwFUNQpEknXEoyY+/gRKbgOHHaKrdHaOdTF85BykV7jz2Yq2l+ESHdfg/7P0FqZo4r7/u8BvDfPTppB99OqmOSmq+nYPjf99Tt2MPzgoHLlsNSqSRmEPSiRo8iLjL78KQnPn3Z41E43CH/5BOcZWdnbU/k2YcQtf4RK3lhCSRJ5xNxglnk15Xi2PRbGqWfU/t9l04y6uQVXUIo0J0n1SiBhxO3MQ7MWR0//uzZiWGug4wpONw1vFX5fckGA4P+6B6HdLhq6rKbscqEoz9iY+0aC0npFEssUSfdy3R57U+ZG1EB7mRp/wyG5RarhnQMdaIBxJhjiT67KuIPrv1OQ8ilVgqXDkBVBUc/vu/T8FQyQW9x2otxWd8GowSQlwghNgohFCFEE1uPRNCnCGE2CKE2C6ECHhAjZ92b0Qayzg6Te/dB4IoQwxOGd4OX1VVfsj7jAh3Fhce2rGCpYUK0aY4XPtFzAxXPt85F8WVwg1Hn6m1FJ/xdfZhA3Ae0GTMVSGEAXgFGAn0AyYIIfr5aLdZPtq4GICLD+8YsTtCDYsx/G/kt1YtwWUs4IyssWE9CRfKxJo8gfZUP23m0oJ565dRZ9jF8IzzMBoMWsvxGV9THG6SUm5podhRwHZvqsN6YC4wxhe7LbGm5DeMrsxOs2sy2MSa4pAivG/ktza8Ce447jlhvNZSOizWSCtCuCl2hO98z2t/vgXuSB488TKtpfiFYHRtugD7L+TN8R5rFCHEJCHESiHEyuLi4jYbK3dUUcUO+sR17GBpWhJnjkMobsprw7OXP2/9MqqVzZyQMlaf4wkgiWYrADkVJdoKaSff71hHsbqKw+JGd5gIqi1O2gohlgKNZbd4QEo5vxU2RCPHZFOFpZQzgZkAQ4YMabJcUyRYYvj63MXUOJ1t/ahOK0nw3sj7KkpJssRqK6YdTFs9A9wWHh0R/knXQ5lkb8TMPHspg7uEX7a5p36dDtLEUyfdoLUUv9Giw5dSnuKjjRwga7/3XYH27dluJd2sKYGsvtOTbLECnht54H7L8MKBeeuXYWMNRyZe1GF6baFKaozH4RdUlWuspO38uHMDBe7l9Is5i+6JqVrL8RvBGNJZAfQWQvQQQkQA44EFQbCrEyAaYp0XVLYvJo9WqKrK1JUvgjuGZ065UWs5HZ7M2CQAiqrDq50APPLLcyAjmHLSzVpL8Su+Lss8VwiRAxwLLBRCLPIezxRCfA0gpXQBNwGLgE3APCnlRt9k62hJl7hkAAqrw6vnNmPlt1QrWxieNkHv3QeBzDjPZrbSmvAKw/HRul8oYzVHWs+jZ1L452reH582XkkpPwc+b+R4HjBqv/dfA1/7YksndOga73H4xY4yjZW0nuq6Omasn4pCMk+dco3WcjoFWXGeodXy2vDpGLjcbp5f+TwQy/Ond6zePXTg4Gk6gaNbfPjdyHcuno7bWMg1/e8g1hyltZxOQbTZjFTN2MMogNrD371LrWEnY7pdE5YLElpCd/g6babhRrbV2bSW0io2Fu5jWcmHWBnITceEbyzzcCScAu3tLiviy5yZRLl78dhJE7WWExA6ZCwdncBjUGPC5ka+dfHjgJvnRjyotZROh1HE4HCFRzu58dsnkUotjw99uMPuvu6Y/yudgGMUMTjcoT8ZN2v1dxSqvzEw7hyO6dZXazmdDrOIoVYN/bhLn278lT31P9A3aiSn9+64OTR0h6/TLiKVWOrU0N4yX1xlZ+qfj6O4kpg28k6t5XRKogyxOGVot5PqujqeXP44Qo3llVH/0VpOQNEdvk67sBjiQj5i5pVfPoLbUMbdgx8hwRKjtZxOicUYi1s4tJbRLNd8+SROYw5X9Lmzw6dD1R2+TruIMcWjKqEbS2fmim/YXb+UvlGjuGTgCK3ldFriIqwhHTHzo3W/sK7qC7oYTuSO48/TWk7A0R2+TruIN8eDUkt1XZ3WUv5Fnr2M6eufwuBK462zH9JaTqfGao5HCJWCKpvWUv5FqaOSp1Y8guJO4N0xT2gtJyjoDl+nXSRGenZR7rO3PaJpIFFVlUvn/wdVqeDBox7FGhWttaROTWKUFYB9ttALr3D5/IdxG0q484iHO/xQTgO6w9dpFw0B1HIqQutGvmfxTIrU5RxtvYixh+kZz7SmIWJmflVotZMpP33E7vqlHGwZzcRBJ2stJ2jo6/B12kV6tCcwVq49dGKdL9yykm/zZxBHf2acdbfWcnSA9BjPk2B+VeiE4fhl11/M3vkcUfIg3jt3stZygorew9dpF+l/R0IMjRu5oLKc+5fdgyKjmXXWfztEOrqOQIPDLw6RdmKrqeaW729HSANvjHwJi8mstaSgovfwddpFQzydohAIoOZyuxn/2e24DSX8Z8CL9E7O0FqSjpcsb6C90hqbtkLwzO+M//QenIZ8bjjkqbDL5eAP9B6+TrvIsnpu5LIQuJGvnD+FUlZxQtLlXHrESVrL0dmPbGsKUioUO7Qf+rv1m+nkun9mYOz53HD0mVrL0QTd4eu0i/hIC1KNwFanbcTM5375hD8rPyJDGcoro2/TVIvOvzEaDChqLOV12jr8mSu+4YfiN0ngCN49p/PGVNIdvk67UdRoTQOoLdr2J7O2T8Hs7s7HF0ztsAGvwh0T8did2g39/W/PJqatfwSTO4OPz5/Wqed3fM14dYEQYqMQQhVCDGmm3G4hxHohxBohxEpfbOqEDiYRQ7VLmwBqGwr2cvfPtyBkJLNGv0p8pEUTHTotY1ESqHFr8yS411bMjUtvAhRmnv4KaTHxmugIFXydtN0AnAfMaEXZEVJK7QfydPyGWcRRqwa/h19QWc6lCyehKjVMOeY1+qdlBV2DTuuJi0jEVrsz6HZtNdWM/ewaXEop9x7xX47s2ivoGkINn3r4UspNUsot/hKjE17EmZKoJ7g9t+q6Os795Fqchnxu7v8YZx1yZFDt67SdRHMyUqmi1lkfNJv1Lhdj5t2AQ9nJxQfdp8dT8hKsQU8JLBZCrBJCTAqSTZ0AkxyViqrYg3Yjq6rKOfNupUrZxDldb+Pao0a1/CEdzUmNTkEIyY6ywqDYU1WVsR/fQxmrGZFyNfcNGxcUu+FAiw5fCLFUCLGhkb8xbbAzVEo5CBgJ3CiEOLEZe5OEECuFECuLi0MrTovOP8mMyUAIyeaSvKDYu/jTyRSo/2NI3ASeOOWKoNjU8Z0uMWkAbCvNDYq9SV8+x676JRwSdTbTRt8SFJvhQotj+FLKU3w1IqXM8/5bJIT4HDgK+LmJsjOBmQBDhgyRvtrWCRzZ8RlQAJuL9wZ8E8uNX73IBsfndDOdxFtj7g2oLR3/kp2QDsBuW0HAbT2w5G1+t31AmnIsc85/LOD2wo2AD+kIIaKFELENr4HT8Ez26oQ5fZI8k6U7ynICaufuRTP4ufQtkhjCpxc8py+/DDN6JmYCkFsZ2CGdp376kPm5LxKr9mP+hZ17+WVT+Los81whRA5wLLBQCLHIezxTCPG1t1gasEwIsRb4A1gopfzWF7s6oUH/tG4A5FQGruf26Pfv803+K8TJw1g4/nUiTREBs6UTGHonekJdFFYHboj2xV+/YM6up7GoPfly3JtEmztXjJzW4tOyTCnl58DnjRzPA0Z5X+8EBvhiRyc0SY+xIlUzBdWBcfjP/fIJH+99nmjZm4Xj39Bv4jAl2mwGdzRltYFZlT1zxTe8uWUykTKLBRe8TZIlNiB2OgL6s7FOu1EUBaOaQHldkd/rfu2Pr5m14wki1WwWXPC2nsgkzDHKeCqc/o+J/8GaH3h5wwOY1HQ+O++dTpPIpL3o0TJ1fMKiJFHl9u+N/OrvX/HqXw8RITP4/Lx3Ov3uyI5AVAB2285d9zNP/3k3RjWReWPeoZs1xa/1d0T0Hr6OT8RHJFOH/+KkTPttPq/+9SARajqfnPMuWdYkv9Wtox2xpkTq8V8Yjvf//J4nVt2OUY3ng9Fv6yGxW4nu8HV8IjkyDalU+iWZ+dRlnzFjyyOYZRe+OP89DkpM84NCnVCgYZNeZV2Nz3W9tXIRz6y5C6OayIdnzeLQ9G5+UNg50B2+jk90iU33br7ybWnmc798zNvbHyVSzWL++e/rj+cdjH5JfRBC5efdG32q57U/vua/6+/FpKbw0dmzOCS1q58Udg50h6/jE9nWLgBsLtnX7jqe+ulDZu14gii1OwvGzqJrfKK/5OmECEO7HQ7Ab/vWt7uOab/N55WN9xOhpvPpue/RNyXTX/I6DbrD1/GJAWkHAbCuYFu7Pv/YDx8wZ9cULOpBfHXhe2TG6c6+I3JMVh+kamRTaftiLerDff5BX6Wj4xPHZPUBt4V1JWvb/Nk7v32NRQWvES1789WF75ASExcAhTqhQKQpArPMJM/R9jDJj/3wAfP2PEekms0X57+rPwH6gO7wdXxCURSsSm/yaze3+jOqqnL1gmdYUTEHK4fz5bgZJFhiAqhSJxRIiehObv3qNn3m1q+n813RTKJlb+Zf8Ka+zt5H9CEdHZ/pYz0Ut7GQvbaWt8673G4u+Ph+VlTMIV0ZyuKL39adfSehl7UPGKrYXNzyBL+qqlz66WN8XzyDRDGQxRPe1529H9Advo7PHJ81GIAvNy9vtpzDWceoD29ia+1CeprP4JuLXsFi0sMldBYGZ/QD4Kdd65otV+9yMeajO1lT9TFdjcNYfNGbegpLP6E7fB2fObPv0Ugp+DV3VZNl8uxlnDr7cvLdyxgcN57PLnxGj2bYyRjRwxNSa3numibLlDuqOH3OJHbXL+UQy9ksnPCyHjDPj+gOX8dnUmLiiFC7st3e+JK7FTnbGf3xOCr4i5HpN/PuuQ/oIY47Id0TU4ly92ZV2aJGs6StK9jNKXPHUayu5PjEK5h3wZN6O/Ez+tXU8QuHWo/DoWxl1urv/nH83VVLuXLxJbhEBXcc+hzPnq5nuOzMTOh7KdJYxgv/++Qfx+etX8bFX19EnShkUp/Hee2sOzRS2LERUoZuUqkhQ4bIlStXai1DpxWUO6oY9uFoFCJYNO4TiqsqeOjH6Wyr/RajO4VpJ03jhB79tJapozEut5sh756GwMQHZ75BpNHE/d9PZ2P1lxjcCbww7CVO6aVHU/cFIcQqKeWQRs/pDl/HX0z7bT4ztz6IlAIQgKSH+WTeHfOYHqNc52+e+ulD5ux6Gk8bURFC0tU4nJmjJ+vB8vxAcw5fX4ev4zduPnYMCVFxLNrxCyoqNx85gWO69dValk6Icf+wCZzcYwjPLX+baGMMEw4dyRl9Bmktq1PgUw9fCPEccBZQD+wArpBS2hopdwbwEmAA3pRSPt2a+vUevo6Ojk7baK6H7+uk7RLgUCnl4cBW4L5GjBuAV4CRQD9gghBCH8zV0dHRCTI+OXwp5WIppcv7djnQWKzSo4DtUsqdUsp6YC4wxhe7Ojo6Ojptx5/LMq8EvmnkeBdg/9i5Od5jjSKEmCSEWCmEWFlcHLgs9zo6OjqdjRYnbYUQS4H0Rk49IKWc7y3zAOACZjdWRSPHmpw4kFLOBGaCZwy/JX06Ojo6Oq2jRYcvpTylufNCiInAmcDJsvEZ4Bwga7/3XYG8tojU0dHR0fEdn4Z0vKtv/gOcLaV0NFFsBdBbCNFDCBEBjAcW+GJXR0dHR6ft+DqGPx2IBZYIIdYIIV4HEEJkCiG+BvBO6t4ELAI2AfOklL4lttTR0dHRaTM+bbySUvZq4ngeMGq/918DX/tiS0dHR0fHN0I6tIIQohjY086PJwMlfpTjL3RdbUPX1TZ0XW2jI+rKllKmNHYipB2+LwghVja120xLdF1tQ9fVNnRdbaOz6dLDI+vo6Oh0EnSHr6Ojo9NJ6MgOf6bWAppA19U2dF1tQ9fVNjqVrg47hq+jo6Oj8086cg9fR0dHR2c/dIevo6Oj00kIa4cvhDhDCLFFCLFdCHFvI+eFEOJl7/l1QoigpNURQmQJIX4QQmwSQmwUQtzaSJnhQogK7w7lNUKIh4OkbbcQYr3X5r+yy2hxzYQQffe7DmuEEHYhxG0HlAnK9RJCvC2EKBJCbNjvWKIQYokQYpv334QmPttsewyArueEEJu939PnQghrE59t9jsPgK7JQojc/b6rUU18NtjX66P9NO0WQqxp4rOBvF6N+oagtTEpZVj+4cmetQM4CIgA1gL9DigzCk/IZgEcA/weJG0ZwCDv61g8yWEO1DYc+EqD67YbSG7mvCbX7IDvtQDP5pGgXy/gRGAQsGG/Y88C93pf3ws80572GABdpwFG7+tnGtPVmu88ALomA3e14nsO6vU64PwLwMMaXK9GfUOw2lg49/Bbk1hlDPCe9LAcsAohMgItTEqZL6Vc7X1diSeGUJM5AEIMTa7ZfpwM7JBStneHtU9IKX8Gyg44PAaY5X09CzinkY8GNNFPY7pk6xIQBZQmrldrCPr1akAIIYALgQ/9Za+1NOMbgtLGwtnhtyaxSpuSrwQCIUR34Ajg90ZOHyuEWCuE+EYI0T9IkiSwWAixSggxqZHzWl+z8TR9I2pxvQDSpJT54LlhgdRGymh93ZpKQAQtf+eB4CbvUNPbTQxPaHm9TgAKpZTbmjgflOt1gG8IShsLZ4ffmsQqbUq+4m+EEDHAp8BtUkr7AadX4xm2GABMA74IkqyhUspBeHIM3yiEOPGA85pdM+EJn3028HEjp7W6Xq1Fy+vWXAIiaPk79zevAT2BgUA+nuGTA9Hy3pxA8737gF+vFnxDkx9r5Fibrlk4O/zWJFbRLPmKEMKE5wudLaX87MDzUkq7lLLK+/prwCSESA60LumJZIqUsgj4HM9j4v5ombBmJLBaSll44AmtrpeXwoZhLe+/RY2U0eS6if9PQHSx9A70HkgrvnO/IqUslFK6pZQq8EYT9rS6XkbgPOCjpsoE+no14RuC0sbC2eG3JrHKAuAy78qTY4CKhsemQOIdI3wL2CSlnNpEmXRvOYQQR+H5LkoDrCtaCBHb8BrPpN+GA4ppcs28NNnz0uJ67ccCYKL39URgfiNlgp7oR7QiAVErv3N/69p/zufcJuxplRjpFGCzlDKnsZOBvl7N+IbgtLFAzEQH6w/PipKteGauH/Aeuw64zvtaAK94z68HhgRJ1/F4HrXWAWu8f6MO0HYTsBHPTPty4Lgg6DrIa2+t13YoXTMLHgcev9+xoF8vPD84+YATT4/qKiAJ+A7Y5v030Vs2E/i6ufYYYF3b8YzpNrSx1w/U1dR3HmBd73vbzjo8DikjFK6X9/i7DW1qv7LBvF5N+YagtDE9tIKOjo5OJyGch3R0dHR0dNqA7vB1dHR0Ogm6w9fR0dHpJOgOX0dHR6eToDt8HR0dnU6C7vB1dHR0Ogm6w9fR0dHpJPwfLi85BKvGx58AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create some toy data\n",
    "real_parameters = np.array(pints_model.suggested_parameters(), dtype='float64')\n",
    "times = pints_model.suggested_times()\n",
    "pints_values = pints_model.simulate(real_parameters, times)\n",
    "autograd_values = autograd_model.simulate(real_parameters, times)\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(times, autograd_values)\n",
    "plt.plot(times, pints_values)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add some noise to the values, and then create log-likelihoods using both the new model, and the pints model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "noise = 0.1\n",
    "values = pints_values + np.random.normal(0, noise, pints_values.shape)\n",
    "\n",
    "# Create an object with links to the model and time series\n",
    "autograd_problem = pints.MultiOutputProblem(autograd_model, times, values)\n",
    "pints_problem = pints.MultiOutputProblem(pints_model, times, values)\n",
    "\n",
    "# Create a log-likelihood function\n",
    "autograd_log_likelihood = pints.GaussianKnownSigmaLogLikelihood(autograd_problem, noise)\n",
    "autograd_likelihood = AutoGradLogLikelihood(autograd_log_likelihood)\n",
    "pints_log_likelihood = pints.GaussianKnownSigmaLogLikelihood(pints_problem, noise)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can calculate the gradients of both likelihood functions at the given parameters to make sure that they are the same"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(369.5127431705977, array([1278.50348645, 1043.41582511,  908.49197428]))"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "autograd_likelihood.evaluateS1(real_parameters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(369.51292685246295, array([1278.47284308, 1043.38969978,  908.46353422]))"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pints_log_likelihood.evaluateS1(real_parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll time both functions. You can see that the function using `autgrad` is significantly slower than the in-built `evaluateS1` function for the PINTS model, which calculates the sensitivities analytically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Elapsed time: 5439 ms'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "statement = 'autograd_likelihood.evaluateS1(real_parameters)'\n",
    "setup = 'from __main__ import autograd_likelihood, real_parameters'\n",
    "\n",
    "time_taken = min(repeat(stmt=statement, setup=setup, number=1, repeat=5))\n",
    "\n",
    "'Elapsed time: {:.0f} ms'.format(1000. * time_taken)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Elapsed time: 45 ms'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "statement = 'pints_log_likelihood.evaluateS1(real_parameters)'\n",
    "setup = 'from __main__ import pints_log_likelihood, real_parameters'\n",
    "\n",
    "time_taken = min(repeat(stmt=statement, setup=setup, number=1, repeat=5))\n",
    "\n",
    "'Elapsed time: {:.0f} ms'.format(1000. * time_taken)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}