unit03_mult_lin_reg/demo1_glucose.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Demo:  Predicting Glucose Levels using Mulitple Linear Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this demo, you will learn how to:\n",
    "* Fit multiple linear regression models using python's `sklearn` pachage.  \n",
    "* Split data into training and test.\n",
    "* Manipulate and visualize multivariable arrays.\n",
    "\n",
    "We first load the packages as usual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Diabetes Data Example\n",
    "To illustrate the concepts, we load the well-known diabetes data set.  This dataset is included in the `sklearn.datasets` module and can be loaded as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets, linear_model\n",
    "\n",
    "# Load the diabetes dataset\n",
    "diabetes = datasets.load_diabetes()\n",
    "X = diabetes.data\n",
    "y = diabetes.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The target values `y` represent the blood glucose levels that we wish to predict.  The number of attributes and samples are computed from the shape:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "num samples=442  num attributes=10\n"
     ]
    }
   ],
   "source": [
    "nsamp, natt = X.shape\n",
    "print(\"num samples={0:d}  num attributes={1:d}\".format(nsamp,natt))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Simple Linear Regression for Each Feature Individually\n",
    "\n",
    "As a first attempt to predict the glucouse level, we could try *one attribute at a time*.  That is, for each attribute $x_k$, we could attempt to fit a [simple linear regression](../simp_lin_reg/readme.md) model:\n",
    "$$ \\hat{y} = \\beta_{0,k} + \\beta_{1,k}x_k$$\n",
    "where $\\beta_{0,k}$ and $\\beta_{1,k}$ are the coefficients in the simple linear regression model using only the attribute $x_k$.\n",
    "\n",
    "Now, we saw in class, that the goodness of fit in a simple linear regression model is given by the coefficient of determination.  Let $R_k^2$ be the coefficient for predicting $y$ from the $k$-th predictor $x_k$:\n",
    "$$R^2_k = \\frac{|s_{x_k,y}|^2}{s_{x_k}^2 s_y^2}.$$\n",
    "The following code computes $R^2_k$ for each variable $k$ as well as the coefficients in the linear model, $\\beta_{0,k}$ and $\\beta_{1,k}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 0 Rsq=0.035302 beta0=152.133484 beta1=304.183075\n",
      " 1 Rsq=0.001854 beta0=152.133484 beta1=69.715356\n",
      " 2 Rsq=0.343924 beta0=152.133484 beta1=949.435260\n",
      " 3 Rsq=0.194908 beta0=152.133484 beta1=714.741644\n",
      " 4 Rsq=0.044954 beta0=152.133484 beta1=343.254452\n",
      " 5 Rsq=0.030295 beta0=152.133484 beta1=281.784593\n",
      " 6 Rsq=0.155859 beta0=152.133484 beta1=-639.145279\n",
      " 7 Rsq=0.185290 beta0=152.133484 beta1=696.883030\n",
      " 8 Rsq=0.320224 beta0=152.133484 beta1=916.138723\n",
      " 9 Rsq=0.146294 beta0=152.133484 beta1=619.222821\n"
     ]
    }
   ],
   "source": [
    "ym = np.mean(y)\n",
    "syy = np.mean((y-ym)**2)\n",
    "Rsq = np.zeros(natt)\n",
    "beta0 = np.zeros(natt)\n",
    "beta1 = np.zeros(natt)\n",
    "for k in range(natt):\n",
    "    xm = np.mean(X[:,k])\n",
    "    sxy = np.mean((X[:,k]-xm)*(y-ym))\n",
    "    sxx = np.mean((X[:,k]-xm)**2)\n",
    "    beta1[k] = sxy/sxx\n",
    "    beta0[k] = ym - beta1[k]*xm\n",
    "    Rsq[k] = (sxy)**2/sxx/syy\n",
    "    \n",
    "    print(\"{0:2d} Rsq={1:f} beta0={2:f} beta1={3:f}\".format(k,Rsq[k],beta0[k],beta1[k]))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the best $R^2_k = 0.34$.  That is, only about 34% of the variance is explained by a linear model with any one variable.  We can see this somewhat poor fit in the scatter plot as well where there is a significat variation from the regression line.\n",
    "Also notice that beta0 are all the same no matter which attribute is used to predict the target!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXt4VNW5/z8rwwQmBJmAGCGiaGuh\nUo6gqVrtrwfohd7EiFa0tl5qtT1eKqjYeKmCl8KReqm1tbWtra1WsF4iSi1aJadHWrRgQMTKEUWF\nQVEhQUKGZDJZvz9mdrJnZq99mdlzyWR9nidPkn1da++Z7373u971vkJKiUaj0WjKl4piN0Cj0Wg0\n+UULvUaj0ZQ5Wug1Go2mzNFCr9FoNGWOFnqNRqMpc7TQazQaTZmjhV6j0WjKHC30Go1GU+Zooddo\nNJoyZ1CxGwCw//77y3Hjxlmu27t3L0OHDi1sgwrMQOgjDIx+6j6WB/2lj2vXrv1QSjnKabuSEPpx\n48axZs0ay3XNzc1MnTq1sA0qMAOhjzAw+qn7WB70lz4KId52s5123Wg0Gk2Zo4Veo9Foyhwt9BqN\nRlPmaKHXaDSaMsdR6IUQQ4QQLwoh1gshNgohFiSX/14IsUUIsS75Mzm5XAgh7hRCbBZCvCyEOCrf\nndBoNBqNGjdRN53AdClluxAiCDwvhHgquW6elPLhtO2/Ahye/DkWuDv5W6PRaDRFwNGilwnak/8G\nkz92ZalOAv6Q3G81EBZCjM69qRqNRqPJBlc+eiFEQAixDngfeEZK+UJy1c1J98ztQojByWV1wFbT\n7tuSyzQajUZTBISXmrFCiDDwGHAJsBN4D6gE7gHekFLeIIRYDiyUUj6f3OdZ4Eop5dq0Y10AXABQ\nW1t79JIlSyzP2d7eTnV1tdd+9SsGQh9hYPRT97E88NLHtmiMHbv30RXvoTJQQe3wIYRDwTy3MMG0\nadPWSinrnbbzNDNWStkmhGgGviyl/ElycacQ4nfAFcn/twFjTbsdBGy3ONY9JB4Q1NfXS9UstP4y\nQy0XBkIfYWD0U/exPHDbx6aWCFc9u4ForALDQRIKxlk46wgappSOI8NN1M2opCWPECIEfAF4zfC7\nCyEE0AC8ktxlGXBWMvrmOGC3lPLdvLReo9FoisjiFZuIxuIpy6KxOItXbCpSi6xxY9GPBu4TQgRI\nPBgeklI+KYR4TggxChDAOuD7ye3/AnwV2Ax0AOf632yNRjMQaGqJsHjFJra3RRkTDjFvxviSspS3\nt0U9LS8WjkIvpXwZmGKxfLpiewlclHvTNBrNQKapJcJVj27otZgjbVGuenQDQMmI/ZhwiIiFqI8J\nh4rQGjV6ZqxGoylJ+oNbZN6M8YSCgZRloWCAeTPGF6lF1pREmmKNRlOe5OJ66Q9uEaMvpexeAi30\nGo0mT+TqeukvbpGGKXUlJ+zpaNeNRtPPaWqJcMKi5zi0cTknLHqOppZIsZsE5O566S9ukf6Atug1\nmn5MKQ9Y5up66S9ukf6AFnqNph9jZzUXWxD9cL30B7dIf0C7bjSafkwpD1hq10vpoIVeo+nHqKzj\nUhiwbJhSx8JZk6gLhxBAXTjEwlmTtIVeBLTrRqPpx8ybMT7FRw+lZTVr10tpoIVeo+nH6AFLjRu0\n0Gs0/ZxiW82lno9Go4Veo9HkQCmHd2r60EKv0WSJlSUbLnajCkwph3dq+tBRNxpNFhiWbKQtiqTP\nkm2LxordtIJSyuGdmj600Gs0WaCyZHfs3lekFhWHUg7vLFmkhG3bCnpKLfQaTRaoLNaueE+BW1Jc\n9KQoD0gJy5fDMcfA8cdDZ2fBTq199JqiUarRGm7apZreXxnItJ1KsZ9+tUmHd7pASvjLX2D+fFiz\npm/5734H3/++cjc/0UKvKQqlGq3htl2qiUq1wyuzOl4h8btNhQrvLMUHpi0qgQcYPBh27SpYU7Tr\nRlMUSrV6kNt2qab3h0PBrI5XSEqxTU6oBr9LJSVzClLCk08mXDRf/3qqyA8eDD/4Abz5Jlx9dcGa\npC16TVEo1WgNL+2ysmSbm1/P+nj5xGwNS8U2xb72dvSLME7DBz9/Pqxdm7pu8OCEm+bKK2HMmII3\nTVv0mqJQqtEafrerFPqZbg2rKPa1t6NUHpiWGBb8pz8NJ56YKvKDB8OllyYs+DvuKIrIgxZ6TZEo\nZLSGlwpMfrdr2oRRnpbnAytrOJ1Sj5QphQdmBlLCE09YC/yQISUh8AaOQi+EGCKEeFEIsV4IsVEI\nsSC5/FAhxAtCiNeFEEuFEJXJ5YOT/29Orh+X3y5o+iOFSmHr1bfrd7tWvvaBp+X5wM7q7S/pg0sq\njNMQ+Pp6mDkzU+DnzCkZgTdw46PvBKZLKduFEEHgeSHEU8BlwO1SyiVCiF8C5wF3J3+3Sik/LoQ4\nHfhvYHae2q/pxxQiWiMb366f7SoFl4MqFLQuHGJV4/SCtSMXSiKM0xD4BQvgpZdS1w0Z0ueDHz26\ncG1yiaPQSykl0J78N5j8kcB04JvJ5fcB80kI/UnJvwEeBu4SQojkcTQaz+QSVldsofWjnF6ulHrO\nercULUunIfDz50NLS+q6IUPgv/4L5s0rSYE3EG70VwgRANYCHwd+DiwGVkspP55cPxZ4Skr5KSHE\nK8CXpZTbkuveAI6VUn6YdswLgAsAamtrj16yZInludvb26murs6ye/2DgdBHyK6fbdEYkdYoPabP\naYUQ1NWEMkIZrdj03h7L2aoCgURSGaigdvgQV8dyQ3ofc22/+Tg7du+jK96TVZtz3d/MQPi8tre3\nUz10KCP/8Q/G3Xcfw15PjaaKV1ayfeZMtp5xBl0jRhSplTBt2rS1Usp6p+1chVdKKePAZCFEGHgM\n+KTVZsnfwmad+Zj3APcA1NfXy6lTp1qeu7m5GdW6cmEg9BGy6+cJi54j0hbIWF4XDrCq0flYbWmT\ng6wIBeMsnHWEL9aiuY9NLREWr95EpC1AQAjiUlKXhcuhqSXCVc9uIBqrwBhW87PNXin7z6uUbLj5\nZiY9+qjSgg9ceSVjDzyQscVpoWc8xdFLKduEEM3AcUBYCDFIStkNHARsT262DRgLbBNCDAKGA4Wb\nAqbJC9m6T8z7NU7uoa0l4kmccnW9pPt2K5KCayYf8djps0/jUva6S7yep1/EkJcDUsKyZbBgAZPS\nBT4U6nPRHHhgcdqXA26ibkYlLXmEECHgC8C/gZXAqcnNzgYeT/69LPk/yfXPaf98/ybbWYnp+3XF\nezzPZvQjrK5hSh2rGqezZdHXUlwoZvz22fs5+7TY4wxlj5TQ1ARHHQUNDalWfCgEl12WiKK59dZ+\nKfLgLo5+NLBSCPEy8C/gGSnlk8APgcuEEJuBkcBvk9v/FhiZXH4Z0Oh/szWFJFvR8kPs/A6rK1Q8\ntp/iHK6y9qWX8gSnfoFZ4E8+Gdat610VHzwYLr8ctmzp1wJv4Cbq5mVgisXyN4FjLJbvA77hS+s0\nJUG2ouWH2PkdVleoCBS/om2aWiK07+vOWB4MiH4XNVMySAmPP54IkzSJO5Cw4C+8kNXHH88Js2YV\np315QOe60TiSrWj5JXZ+htUVKh7brwfK4hWbiPVkupuGVg7S/nmv9PT0Cfz69anrkgLPvHlQW0us\nubkoTcwXWug1jrgRLavB2mLHb6sGkAsRj+3XA0X19rN7gJUszAkngb/oIrjiCqitLU77CoAWeo0j\nTqKlym++cNYkFs6a1LtfZaCiYFPtSyEPvB8PlFKYcNVv6elJ+OAXLICXX05dN0AE3kAL/QDHbdik\nnWjZDbquapzeu19zczNT85DLxqr9hQ5JNNpx+tg9XLPoOVfWu5trX+y3omKRU5ERO4GvquoT+AMO\n8L/hJYoW+gGMX1ZvscL/7NpfyDaltGOsu+vo9trna0yhlKs1Zf251AKvRKcpHsD4FeudS8iilxTC\n6di1v5BpbbO5jm73yYcgl3q1Js/Xs6cHHnkEJk+GU05JFfmqqsQA65YtcMstA1LkQVv0Axq/rN5s\n3Qsqy23N27tY+doHjuJm1/7bZ08umMvDy3U0hNvK756+j931Wf7yu7R2JAZkw6Eg82dOdP0A8OLW\nsnrQhF2dJXtcX8+eHnjssYQFv2FD6rqqKrj44kQs/AAVdzNa6AcwfoY/gnf3gkpwHlj9Tm9yJNVr\ne1NLxDKdgdH+Qqa1dXsd04VbdSwD1fW5f/U7KcvaojHm/TkRTeKmf26F9NqmDZb3YuHxmbmH/MTx\nevb0wKOPJgT+lVdSNzIE/oorYFThiruUOlroBzB+DvRlE2GiEpx06U63Ng3BtBJ5c/vzFUaZbuVO\nmzCKR9ZGlNfRyYq32ge8vVnFeqTrgWY3D6amlkiKyBtEY3F27M5vaKfyc/nFw+Hhh60FfujQPgte\nC3wGWugHMMUu5qASHCvMoqcqjRcQwlP4Zjb+byt3yiNrI5xydF2yatSelAyVbqx4wDKrpZfrA+4f\nDG4e8ItXbFLWl01P++z3OEL657Juv8HcFnyDY86ZpwU+S7TQD3CKVswBa8ERWOS0JiF6TpZxj5Se\nRD6byA6VO2Xlax+wqnE6zc3NXHLmVNvt0wmHgpaVnrxcH3DvcnPzgLd7aFQG+mI48jVfoWFKHQ1H\njk4Mst7QqAU+R7TQa4qGleCo3CDTJozy5N92Its4e68D2G6s7L1d3TRZpG9WXZ+l/9pKLJ4q98EK\nb7lvnB7wqrcJAdQOH9L7f17mKxhRNAsWwMaNqeuGDoVLLkkI/P77Z3f8AYgWek1RsRKc+kNGZFib\nTpZxMCCYNmEUkxc8TVsyPUBNVZDrT7SORslWmL0OYLtxv8Tiav+66voseGJj1lE3blC9TZx53MGE\nQzt7l/k6X0ELfN7QQq8pOazEbe7SdYqtE8R7JH9a/Q5m73FrR4x5D1tHo2QbceR1ANtqeyu8ZvTM\n14Qp8/JwVZDBgyrYHY2lbNNsSvjlS+RWT09ikPWGGzIFvro6IfCXXaYFPge00Gt8Id8zLZ0sY4sE\nj4DaWs424sjrAHb69mDtYx/uU81aN9jF55vdZq0dMULBALfPnqzsX06RW1rgC4YWek3OFCKBmFvL\n2AorazmXiCOvFrV5+yk3PN3rcjEjrCot+4j5Qawqp/jgC1s9l1nM6jrG430C/+qrqeu0wOcFLfSa\nnHEakGtqiTB/2UbO+3iUcxqX2/rOVZgFxUvIIajdCLm6QLKZNdpmIfKq5X69JVnVr7VCtdzJraS6\njhnt/8LHaXjjn2qB/8EPEgI/cqRtX0o1R08po4VekxXmL5wq3G97W5Smlgjz/rw+pXiGne/cDkNQ\nmloizHHw2RvkqxKTMjWzw6zRbGfRun1LshJCNyGekJiHoJpp7BVz+yt64kz55wo+deu58GHqrF63\nAp9+TChO6un+ik5qpvFMelIsFWPCIWWFJMN3ng0NU+oIK3zaZhdITVWQxacemRcRUL3F7Ni9z3Y/\ntzVws0mUpkpW5uYNKBQMcMaxY32rz7t4xSY6O7s48dX/YcW9F3PXslv4uFnkq6vhmmvgrbfg5psd\nRd44pl8F1wca2qLvhxT79dWNhWgIhF20TC4pg+fPnGg5CFiowiaqtqfPGk3HrU87m7BFlRCqLPWA\nEPRImdIGq9BWz9czHueof/yVH/xjCYfv3Jqyqr0yRPW8y2DuXFfibqZY6bDLAS30eSJfYpzL66tf\nbbL7YgnIKACisihzSRlcqukbzLNGVbgZG1AdXwInKAqbqO5LXEpCwYCrh2JO4xbxODz0ENxwAz97\n7bWUVXsqQ/z+6Jn85fOzeerGk7M6vK62lT2On0ohxFghxEohxL+FEBuFEJcml88XQkSEEOuSP181\n7XOVEGKzEGKTEGJGPjtQiuQz33e2r69+tkn1xaoLh9iy6GspVaXmzRhPsCIzpMQP33nDlDpWNU7P\nOGchULlgzLNG/T6+gere2d2XhbMmURcOIUz/+3a94nF48EH41Kfgm98Ek8jvqQxx52dm89nv38sv\nPn8O35t1TNancev20mTixqLvBi6XUr4khBgGrBVCPJNcd7uU8ifmjYUQRwCnAxOBMcDfhBCfkFJ6\nj4vrp+SzjF22r69+tkkV6thhMZXf+Hv+so0kPkp9M1YhYZ1mY5H7/cbk9XiqN4rw7tezboPq+FZW\nrNW9s4tpz0tOI5MFT5oFz7BhbJr9HS4dPZVNXcG8JDvTUTfucRR6KeW7wLvJv/cIIf4N2F3Zk4Al\nUspOYIsQYjNwDPBPH9rbL8inLzHb11c/22QWbyPdACSiaVTl8IwZlW8lE37l6oLyM/oi2+NZiWdz\nsz9Cbz7+oY3LLQe90+9dwYQwHoc//QluvNFS4JkzB+bMYfyIEfzV3zMXNQlff0ZIReys5cZCjAP+\nDnwKuAw4B/gIWEPC6m8VQtwFrJZS3p/c57fAU1LKh9OOdQFwAUBtbe3RS5YssTxne3s71dXVnjpV\nbDa9t8dyUK4yUMH4A4dlLPfSx7ZojEhrlB7TfasQgrqakDISJZs2ucHrMds+2sOOjsSApUAgLeTL\nTXtU5xUIDhphfx28HM9oT+3wIa6P6fZetkVj7Ni9j654j+M58nHvsiIe54CVKzn497+nOpLqNuqu\nqmLbKaew7dRT6d5vv8K1KU/0F92ZNm3aWillvdN2rgdjhRDVwCPAHCnlR0KIu4EbSYwP3QjcCnyH\nxHhcOhnfaCnlPcA9APX19XLq1KmW521ubka1rlRps8hBbgx+TbWwRrz2MRu3hdc2ueHcxuVIi2Ee\nAWxZNDWjzZHta1n8cgC7oSGrfd2eFyAUjLNw1hHuJvAkr5vd8ZyOmY6be9nUEmHe0+uJ9VRgXItg\nRYzF37A+Rz7unRea1rzD+sW/5Ft/+yMf27UtdeV++8GllzJozhzGjRjBuLy3pjD0R92xw5XQCyGC\nJET+ASnlowBSyh2m9b8Gnkz+uw0Ya9r9IGC7L63tJ+T7FTqb19d8tMmLG2nxik2cPtb57VHlgnKa\nwm9gV/tU5Z5xyqPj1/iKwfxlGzPmFsR6JFc9+rIygyUUwTcdj7Nm0S/4j9tvoWFnqsDHqocRvGxu\nwk1TU5PfdpQRxQqNdhR6IYQAfgv8W0p5m2n56KT/HuBkwKgMsAz4kxDiNhKDsYcDL/ra6n5AKfoS\n7dqUzQfQS0Kr7W3R1Me/Bap93U7hTzlXGnaD0W7y6PgZq20e10htT49lXnoo8OcpHoclS+DGG6nf\nlBrN9VFlFa997USum3QGf11wUmHaUyYUc2avG4v+BODbwAYhhDH75WrgDCHEZBJumbeA7wFIKTcK\nIR4CXiURZnHRQIq46Y/kMhgJ7izNhKW+J2O51aSddNxO4U89Vyp2g9Fu8ugUKlbbzzcHz3R39wo8\n//d/Kas+qqzi3k+fxL31J3H+p4ewaYOeguOVfEbjOeEm6uZ5rP3uf7HZ52bg5hzapSkguXwA3Vqa\n82aMJ/LvtSnL3M5kdZqgZbbvVW8FTm4mcx4dvwqmq6ipClpmsIQizfK0Efj2wUP5Tf1M7q0/iY+G\nGIOT3Vk/+Io9q7uYFHNmr34sawryAWyYUkfTe69SFw7YfsnTC19Iqa6Rmi7yAjjlaOsHj8rNNG3C\nqIxY/oWzJuVVjK4/caIyKVtBZ3l2dycmOt14I7yeFhY6fDjMmcP/TPsGv/rb1pTrViGym+w20JOS\nFXNmrxZ6TcE+gIki2FOV69OFQGX1gnWRbAmsfO0Dy+2t3Ezp9Wl7M1DOmmRZrNsvGqbUsebtXTyw\n+h1XbyNu8GQpOwn83Llw6aUQDvM1ILZfOOXYdTXxrIS5kK6LUnxzyKlIS45oodcwbcIo7l/9juXy\nQuLWF19nEyUTaYtyaONyyy93upvphEXPZSU8fojITQ2T/EkghgdL2YPAm0m/buZSgl4olOuiVN8c\nijmzVwu9RmkFq5aryFUA3XzhBbCqcTonLHpOKfbmXD6g/nJnIzxOItIWjblO6+BXJI2jpdzd3TeT\ndfPm1J1tBN5vCvXmWMxBTyeKFY2n89FrfLG0nJKmNbVE2PTeHg5tXM4Ji56zTKbm5gtvbGOX9MvA\nKdmb6nx27bATkaaWCJHWaF6S2dmhuk87drXDH/4An/wknH12qsgPHw4LFiTywV9/fd5FHgqXlEyn\nM85EC70mK8FLx0kAr3p0A13xHlsBdBJvsyg0TKlLyciowu7LnY3w2InI4hWbUlJTQGEKY6Tfp0BP\nnFM2PMvKey/MFPhwuE/gr7uuIAJvkH7PfM+imcSPz3O5oV03Gl8GiZwE0M2rdLoP04i62R2N9Q6e\nLl6xiblL1/W6RYxBU5UrJ1wVVLpSsvGZ2rkfVJPC8m1JGvevq7OLho3NXPKPJYxrezd1o3A4Ua7v\nBz9IWPNFohCui2IOepYqWug1vgwSOQqgBVbLVULg5Bu3+nIHA4L2fd290TtWfnuvwmMnIgnLPXNS\nWL4tyYZJtYx94s/U/mIxB+1KyzZSIgJfSHQ640y00GuA3C0tJwHMdRDO6a3A6su9t7M7I91ANBZn\nztJ1vakP/M4ZZDUpLG+WZHc33H8/3HQTR7/xRuq6ASjwZkoxBUkx0UKv8QUnAUxY0t2923sRwKaW\niDLCxvxWkP7lPrRxufKYdlE5TtFDViJi7HP6WNlbo7XOx4IokbZo73EPHlbJHd2vcNQffw5WAn/5\n5XDJJQNS4DXWaKHX9JJLeKTdvsbvHZteyqgp6+a4hiBbYfdWkE1WymxisFP2GdtXo9UPkU95S4p3\nc+rGlVz8j6WZPviamoQFrwVeY4EWeg3gf8WnuUvXMWfpuhSrtnn364655tOxm0Tl9FaQTVbKbGKw\n/YjbtnpQGscdFO/m5I0rufifSzmk7b3UHWtq+iz4Mij4ockPWug1gHPRcTtL32pfI8jQ/MDIJpDP\nLmLFKTQvm6yU2cRgZzvxyhxd1L6vuzdHvXHNYvs6+YZC4NuGVPObT5/MFU/epQVe44gW+iwpxVwa\nubRJJUqG6NhZ+k7hg8YD4+bjvE/bULlf6sIhV33zmpUym9mbXvdxyukzKN7N119+mkv+sZSDd+9I\nWdc2pJpff/pk7jv6RIbXjuQKLfIaF+gJU1ngNAu0P7YpXGVdr7RCYGvpg7vomWxjyVWTqDq6uj1d\nb7eTdbKZROV1H5U7alC8m2+8/DTP/fp7LH7qzhSRbx0yjFs+dxaf/f69/Pz42cSrhw3ouHCNN7RF\nnwWlmEsj1zapijb1KJabhduNL9xtKKXVW8nCWZOYv2xjSqhka0fMc6IqNyF32cRgm/eBPY7RNukP\nvYQP/jlLC75reJjfHTuLnx3xZaJDhvoWzaMZWGihz4JSzKVh1yY3Lp3divJ2KszCne4LVxYD2Z2W\nMTEN1YDwwlmTGDp4kGVMfD4ertnEYBv7NDc3c8mZU223NVw9g+LdzHrlOS7+p4WLJjSMyHkXMfHH\nV/O9YcMS5ds0mizRQp8FxSwgoELVpuGhoKtoGtX+4VCQzu4eR9+2WRxVD5bmZnuht3sr8fvhWswx\nliunH8a/bvgp33v+QcamCXzrkGEs/c/TOOjaeXz9s9o1o/EHLfRZUOxcGlYipWqTsPGxm4VNtf/8\nmROBPms9IESKj95KHLOdlWgn5n4+XIuWrzwWg/vu46Sbb+akt95KWdU1PEzlD6+k5uKL+f6wYflr\ng2ZAogdjs6BQWfisUA26ApZtanNZm9SuT0YumVAwQFymhgD6OQBtl3XQzxS3TqGkvtPVBb/5DXzi\nE3D++YnMkQYjR8LChVRufQeuugq0yGvygLbos6RYuTTsRGpV4/SMNnnJM2PXp/nLNrp6M8jFJWL3\npuRHoipzOgEr3LqBXPexqyuRD/7mm1PFHRICf8UVcNFFWtw1ecdR6IUQY4E/AAcCPcA9UsqfCiFG\nAEuBccBbwGlSylYhhAB+CnwV6ADOkVK+lJ/mlxduBMQu3r2pJWIZLmiV1XFvZ7ey5J5Vu9IHQq3a\n48UlYtdXq+W5+tSt4ujTceMGctXHri64776EwL/9duoBRo6EefPgwgtLTuBLcW6Ixh/cWPTdwOVS\nypeEEMOAtUKIZ4BzgGellIuEEI1AI/BD4CvA4cmfY4G7k781NrgVSbv8LVbbp0fEVAiIxWWvcLvx\nT9u5NIaHginbubX67fpq9VDI1afuVI/WrRvIto8TRzH6ySfhnHPUAn/RRVBd7arNhaRU66xq/MHR\nRy+lfNewyKWUe4B/A3XAScB9yc3uAxqSf58E/EEmWA2EhRCjfW95meHWb2xXhUnlZzb72K3i4p38\n03YuDaNGalNLxHVkjFcfuVP1qhMWPWdbotCpDwBDgtZfhfTjWz1kg/EY/9n8GBx+OONvvTVV5EeO\nhEWLEq6bH/6wJEUeijBuoSkoQqpmylhtLMQ44O/Ap4B3pJRh07pWKWWNEOJJYJGU8vnk8meBH0op\n16Qd6wLgAoDa2tqjlyxZYnnO9vZ2qkv0y+EX7e3tbNmttjYn1aVmI2yLxti6q8P19gCb3ttDV7zH\nth1W+7ndt0IkCvqll9IDCFQIxoRDdO/r4F11s5Vt2BDZbXte8zkrhKCuJkQ4FKQtGmPH7n10xXsQ\nCCT2n3XzvpC4zpHWqGWfACq6Y3zy789R//jD7LcztZB61/DhbJ09m+0NDcRDpV/Czu4ap9+TgfKd\n7A99nDZt2lopZb3Tdq4HY4UQ1cAjwBwp5UdCKCt1Wq3I+KZIKe8B7gGor6+XU6dOtTxYc3MzqnXl\nQnNzM0te6VHmdLGagKOyLlXbn9u4HGnzAqfaD6CtJcLcpescZDKRLsHqjWFoZYAeGefCCXDrBvVH\nTtWGaxR9NfKzZx4nwLwZh3PVsxuIxirwElxWFw6wqjHRhsQ1znx7CsZjnLrhWS7651IO+ihT4Cuv\nvprKCy/kY9XVfMz1mYuL6hpb3ZOB8p0spz66+gYIIYIkRP4BKeWjycU7DJdM8vf7yeXbSK2ceRCQ\nVt9Mk47X8EGv29sNNDr5pxum1HHmcQfbFuEGdbqEvV1xW/+4UxtUfbUSeVDXqYXEw8FtMfF0d08w\nHuOb655i5T0XsHDFXakiv//+cMstvPDgg3DllVBd7dqtVAr4Gb6qKT0chT4ZRfNb4N9SyttMq5YB\nZyf/Pht43LT8LJHgOGC3lDKAaOriAAAgAElEQVStSoImHa+x+ebtgZSJTFaCovLth0NBV3MAbmqY\nxJnHHUxA/SZnu86OgBCccrR9uOrgQX0f1ZqqYErf07GrU9sjJVsWfc123/S/zQL/4xU/txR4tmyB\nefN63TSlmPjOjmLODdHkHzeumxOAbwMbhBDrksuuBhYBDwkhzgPeAb6RXPcXEqGVm0mEV57ra4vL\niL7yc3u4ZtFzzJsxnlWN013vby7T5xQtkWscelNLhEfWRpRWdCgY4JSj63hkbSQjDn7woApleCYk\nKjI9sjZC/SEjHCNuAPbFEuMFudSpdTO7+YfTxvHSgts5//ml1O1JddHsDO3Hkqmnc9Gfb4OhQzPO\nU4qJ75zQdVbLF0ehTw6qqky1z1tsL4GLcmxX2ZNefi7bcDYvgpLLF9kuPNGcTbH+kBEZDxPIrBmb\njqrNThPEjG1UdWpVQm774OvshN/9jpk//jEzt25NOfeHVcP51TGn8MgxJ3Ld6Z+2FHkozcR3moGL\nnhlbJLwItN1EllxnebpFdTwBKW8h5kIfi1dsYu7SdYwJhzjl6Doq972VkdnS6RxOgql6eLl5g8nY\nt7MTfvlL+PGPIU3gO2tG8uvjTuUXE75IzQE1XOfwNhSuCmYUFDGWg56cpCksWuiLhFuLz24iC6AU\nTr8zaXpJKmbV5kfWRlh4/BC2LPqiMmLI6li5JDNz/QbT2cm6Bbcx+he3U7s71UXDAQfAlVcy+Pvf\n5+KhQ7nY+WiAOr+/lHpykqbw6KRmeUYVeWGXwMuMneW/eMUmS5EX4Hu0hCoqY9qEURn9U7V5x+59\nymMB7O3MrBqVTTSI62iXzk64+246DjmUyQuvThH5D4eGeWXuj+DNNxPFtxUuGhWq/P67ozE9OUlT\ncLRFn0dUltuat3extzPTX20lYNn4eiX+W4YNU+pY8/YuHnxhK3EpCQjBUQcPTxl8Ta8vm44x6cpo\n24InNqa4N9qimVWjvA4iu7KWOzvht7+FhQth2zaqTPt/UBXmV8fO4oHJX2Xw8GEM/dkLWblX7N5E\n8u2/124hTTpa6POIynJ7YPU7GZZ4TVWQ60+cmPGFdHJdqCa5GPj1pU+PuolLyao3dmVsF43FlROZ\nKgN9L5ANU+pYvGJThh/bapzCyyCy7djHEfunCLyZD6rC/PLYU3hgylfYFxyS2C8a85QTyEwuEUG5\noN1CGiu00OcRlYVm5W6pqhxkmaVx2oRRLP3XVmLxvr2CAZESzaKKLPHzS++UFMxMXEpCwUBGu2qH\nV6Zsl6tla75W4aogUmIZxlnZHWP6c8vh52dnCDwHHMCd9bP4xYQv9Aq8Ci/hkXZvImve3pXxsPdr\nclJ/DOvU5B8t9HnELtNkOkZt13RhXvri1kzrOPmvk1vDzy+9F7eCEW6Z3q5wWs3YXAZa06+VVYTL\n4O4uTnv5aS78558Z3b4zdWVtbSLJ2Pe+x8GbWhGPbgAXDzK3NXhBnYnzkbWRFJEX4DhhzC06rFNj\nhRb6PGL1+m4XJWMlzDGLvAKxHtkr1nZuDT+/9G4fWsZAsFW70mvGqq5PpC3KCckJZHapk1VvGG4F\nnqqEd75hSlXvMQ3x7ujqtnx4WNXgnbt0HWve3sVNDZOU18Wu3RJY+doH1jt4pBTrGWuKjxb6PGJl\ncU+bMCpj9miFSLhi5i5dpzpUBnZibVicqnj1bL70VqJshZeB4PRc+eaHoFPREisxcxL4DWd+n0uH\nH8OWHZIxd65OeZCkP5isZuSqavBK4IHV71B/yAjCpJJu/ed73kOx6xlrShMt9Dng5hXeyrJNnz1a\nVxPvHZx06+pRibVTJSUni1nVJ1WkTDqqHDIqjGNbxdbbFS0xM7i7i9nrV3Dh6j9zYHvaAHFtLRu+\n9V+cP3gK78UD0JFa89Zog1W7INMtpnoYy+S2Nx/XN+Bs5YrL97wHP0ouasoPLfRZkstAZ7r4Nzc3\nA4qyfxUCklWhDOwsNDuXhpPF7KbykzFgPH/ZxoyBz1wsx2yKltgJ/AfVNbz3vUt5a9aZXPnUZstr\n4jReYfWQdq45OzRlWyvrP13s/ba4dc4aTTp6wlSW5GPSi1UGwcXfOJLFpx7pKqugyqVhkG5JprfX\nS5+GDk7YCEbGylyzHbqdQLa9Lcrg7i7OXvsE//Or73LD336VIvLvD63hjq9fyD+ffpFJP7meRX9/\nx9bdZDUT2W6y1bwZ45WJn6zaaoUEnSVSU1C0RZ8l+YpucMrdosLKpeEGc3tVDwnz8nSr3wilzNU9\n4Mq3vG8fP3j1r5yx8k8ZFvyHw0aw/43XccAFFzDHVNHJ6X6YxdlugtvK1z7odYUc/7ER/OONXdZW\nuSmySOWTrwuHPGUp1WhyRQt9lpRadIOdy8YuVbC5vaqJTuY88yqrf8ETG3PyC9v6lvftg1//GhYu\nZO67qaUN3h9aw29OOI1Pzb+CmZ/5uGX/VA+w9AeJmwlukbYou/Z2ceZxB6eIv9HWpqde5YRFz7G9\nLcrwUJBgQLh2u2k0+UILfZY4WaCFnoZuZ7kunJUI+7MapO3oSuSXaZhSp8w1b16uOk9rR6x3kDbb\niVkZbzPRKNx5Z6K4dprAfzhsJD8/5hSa/7OBS7/+H8xUnEcVLWQ1E9ntBLdoLM7K1z7IsMqbWiJE\nWqO95QeNB+vQygAdXXE9MKopGlros8TOAi3GNHQ7y3Xxik3MmzGehbMmZQyitnb05Zeps3E1OIVs\nppPTbMxoNGHBWwg8o0dDYyP7n38+14dCXO9wKC9RKF4nuKWTKCKTeYU6uuLcPnuyFnhN0dBCnwMq\nf3oxpqHbxbkbD5qFsyYxdPCgDBeO0TbVW8q0CaNcxdCn43m8IhqFe+5JCPx776WuGz0arroKvvtd\nCGUXwumE1wlu6Wxvi6ZWS05ihF5qodcUCy30LvDqhinGNPT0yUfpGGJu1zaV9eslz40Z1+MVbgT+\n/PNhiH0umlxxO8FN5WdP9HeP5bF1CgJNMdFC70A2bpjhoaDlwOfwUDB/DaXPcj20cbmlFWo3O7NC\niF5ffXq/vMzYNXA16FgiAm/G3H9zbn1joLrO5kE/b8Z4tr66xvK4OgWBppjoOHoHsomXF4pAa9Vy\nv1E9UAwL3aroR1xKrnp0g2WRDq8iVVMVtI8Nj0bhjjvgsMNgzpxUkR8zBn72s0TBj0suKajImzEe\n8MZD0U0YacOUOkYOrcyIs9eRNppio4XeAdUrt5FGwGpiTZsiRYBquRvcVk1qaomwtyuzqEkFiQib\nuUvXMSRYYfnQUT3ArB4OwYD6qWWkXM5oQ2dnn8DPnWst8G+8ARdfXDSBN8h2QtyYcIjbZ0/WE6I0\nJYWj60YIcS/wdeB9KeWnksvmA+cDRsq9q6WUf0muuwo4D4gDP5BSrshDuwuGytVh5IyBTHeO3zH2\nXtxHi1dsSonbNuihL5WvXa4a4wEWaYumuCtOObouI258jsKlk/Fw7OiAX/2K4268EVpbU9fV1SVc\nNOedB0OGcG3ThpQqVmccO9ZVVshssBt7yWWcxe3gr64EpSkUbiz63wNftlh+u5RycvLHEPkjgNOB\nicl9fiGEyPQT9COsrFmrSAyztZdNnVM7vFiXuQ76mR9gRvy8Udx73ozxbFn0NVY1TqdhSp0ygVnv\nA62jA26/PWHBX3YZlWaRr6uDu+6CzZvhoot6Rf7+1e+kVLG6f/U7XNvkfcavE2bXjKTv4em1pm++\nzq/R+Imj0Esp/w5k1oyz5iRgiZSyU0q5BdgMHJND+4qOVf4ZVSy5IbJW++Ty+u7FusxViFR9s3qw\nqB5ojZ87OEXg2bGjbwMLgTd48IWtludWLc8Fp4fnvBnjEwnlTAQrhG++dl0gXFNIhFTMhkzZSIhx\nwJNprptzgI+ANcDlUspWIcRdwGop5f3J7X4LPCWlfNjimBcAFwDU1tYevWTJEstzt7e3U11d7bVf\neWXTe3t6C12bqQxUMP7AYZ6P59RH1fkGVQgqhKAr3kNloILa4QnRjLRG6XFxX7NhUt3wlP/bojF2\n7N5HV7yHqu4Yxzz/DOMf+XOq9Q507r8//3fqqbSefDI9laklBQ02RHa7Pq8XzG00rtPWXR2252qL\nxtjWGsX8/RBCcFBNiLBN9JTbz2u++loISvE76Tf9pY/Tpk1bK6Wsd9ou2/DKu4EbSRiANwK3At8B\ny8R+loojpbwHuAegvr5eTp061fJEzc3NqNYVizZFUYqFsyYxNQur3amPVucLBgRIowJVRbIN8cSb\nxIHOseDZUBcOccmZFu3s6IBf/hL++7/h/ffTdqqDq69m8HnnsfOf/+ztp5V/+o4V65VpGOq29mTl\nw25qiXDVsxuIxiowX6fBg0KWIbBGHxPjFJlex7pwgFWNUy3Pk5gZG2fJK85tvcYi/775/KVMKX4n\n/abc+piV0Espe9/FhRC/Bp5M/ruN1LmBBwHbs25dkXAaJMumuEMuA29W59vb2a2c4Wr40M2Yi50M\nCVYQjWW+IdhhOcawd29C4G+5JVPgDzoIrr4avvMdGDyYppYIO97bw7mNywlXBWnf191bJtHwTx93\nWA2r3rD2EkbaosxZuo4FT2zMyFFjh8pFMiRYYVnA3OijF3dZymD5WHXWS/M915WgNIUkK6EXQoyW\nUhpJSE4GXkn+vQz4kxDiNmAMcDjwYs6tLCBuI1y8FHfwI/dN+vkObVxuuV0kWbzartKVVTUnOzIm\nCTkI/LpvX8icqim8/XacMbev6n2juHBCD5IKy6ifaCzOWzujfOu4g3ujbqww5+Zxc+1Ugt3WEeP2\n2ZOVD19V5FS4KtibndJu5rBV1sv0Ii7g3lgoRISOjgIqX9yEVz4ITAX2F0JsA64HpgohJpNwy7wF\nfA9ASrlRCPEQ8CrQDVwkpczNX1Bg8pGnJh/HtEvA5SSEbiJzDFdUyjFcWPDLjprB5Y+/Rqy976F2\n/+p3XPQo0a6bGiZxU8MkxikeZJC4dpc/tJ65S9c5CpJdqKvdw9qy2ldA0L6vOyNLp8olporMUtWp\nVVGIJHnFSMSnKRyOQi+lPMNi8W9ttr8ZuDmXRhWSQhRvdusG8GJR2SUxc3qIqPoZEIIeKTPPvXcv\n3H13QuA/+CB1p7FjEy6ac8+FwYO5bsHTvS4Zr3hJEWEO/bQTpGxdJOm5gwJCWM5PMKdHcEM2n6NC\nJMkrRiI+TeEY0LluClW8WZX7xk11I1AXr17z9i6ltWwnKCrxs7TgXQq8gVU/3bLXlBu/pipoO7HL\njJ0g5VIs29jGKXOnkR7BTdbL4aFM149TWwqRJK8Yifg0hWNAC72qeHM6uQySqVISpMdke7Womloi\nPLJWPbnG7sHkKH5OAn/NNXDOOSkCb7yN5EIsLnv7e/2JE5n38HpLK9oKO0HKpVi2m8yddSZfPeyh\nThHpFKwQ7O3qG0R36x4pRDWzUquYpvGXAS30duIQDgXZHY3lPCilSklQPSQ1H4xXi8qpdKAb10RG\nn/buhV/8AhYvzhD4jgPHUDX/ugyBh8y3ESuGVgaoDPQgsB9fME86g1TXiZEWwcpNki9BcrJozYnO\nGqbU0dzc3BseaY50GhMO0dHVnfGWYow3gFrsCxGho6OAypsBLfR2gjN08CDWXf+lnM9hF/Xhpi0q\nAXMqHeglIuhnT6zjCysf4fv/eoyavW0p6yPDRvHz40/jyaNmcMMxR9FgYcU7RfAEA4KbT55EePfr\nbFk0FVBH/pj99FYPI6uHSj4Fye4zYpeyGNxHShmZQ419rI4DqQ898yxaP3zoubi4NKXPgM5eaScO\nfvkm3eZM8ZofRzVwWZeMJnHDE6v+j81XXMdDi77JVc2/SxH5bfuN4qoZFzP1e/fwp8lf4aOeihTX\nTHoaXxV14RCLTz0ypU1NLRE6LNxZ0OenV+F3egknVLmOssHurcMp/UHDlLretqQPRPuVH6dhSh2r\nGqen5DPSlAcDWuiNQT8r/HIFOAl4U0uETe/t6U0fHA4FHQXs2qYNloOernOxtLfDLbfw2RnHcMVz\n9zIy+lHvKkPgp11wDw9O/jKxQN/1MT/83Pqu0wXDeECoBlpjccmcpetsUzEXEvODBVIHWb0KraoW\ngIGTcZHv/DhuU2Fr+h8D2nUDcP2JE/PqCrB7JTZEzzyRKBQM2BaSbmqJKCNtup3CGtvb+3zwH35I\njWnVtv0O4K7PnMYjkz6fIu5mzA8/J1ESwLQJozKWuy1LqBqoLEa8t+GCsXI3eQlBNLa5/CHrVA9O\nxkU+I2N0HH15M+CFvhC+SVXURzaRNsbAnRUSxWSp9nb4+c/hJz+BDz9M2cdK4MOhIJ3dPRlt29vZ\nF/5o57s22vLI2gj1h4xwNehshdW1KGa8tx9CqwrZdGNc5DMyRsfRlzcDXught/C7XMgmn4rTxJyU\nL6ch8IsXw86dqRsecggt376Qs+Of5KOePg9eKBhg/syJACx4YmOKi6Ut2pd+wG7ClmVbkvtXeJhc\nBJnXws01U008y3WKv19Cm61xkc/IGB1HX95ooS8iXoTDrcsDoG3HzkTB7Z/8xFLgufZaOOssplRW\ncoON+C1esckyHNBInGZssz1ZPMMKQyiaWiJEWqPEZaaPWjW5CDKvhdM1u7Zpg2WOmTVv70qJa8/G\nNeGn0LoxLqweTAtnTcrL26eOoy9vtNAXEUM4EmmBEqiEw41lNbSzg7NalnP+i4+BaYAVgHHjEhOd\nzjoLTPng7QTHycpzkyjNEIpECt9MOQ8Iwa2nHQm4c2fYiW1TSyRF5A2isbhlojSvrolChiCqfOYL\nZ03qfcj6iY6jL2+00BcRQyB2bHqpdyKRSjhsY/5NAj8iTeC3Dq/l/s9/iyN+eDEyGGTxbc+7Fikv\nVp6TUGxvi6YmsE7SI2WGD97J5WJVv9YYLFW9GajcRV5dE4Vy8xXaZ67j6MubAS/0xU7N2jCljmbT\nRCIVVkI6sqeT2S88zvkvPkbNvj0p228dXsvPPjObRz81ne7AIIKPv2oqVOLOdeHFyjNy75gLe59y\ndJ8oJh4OezL2Mz803EyQMurXWoWe2ol2oWfU5koxfObFGqvS5J8BLfR+hZT5+bBQHctsce3esZOL\n/72C76x+hMrdqTNZ0wXeQJV50c5C9GLlGbl3zIW9jVw8K1/7IPFmkGbRu3ENeLFsVW8gAjjj2LEZ\nuWdK2TWhfeYaPxnQQu/H67Gf8cdOx2r42DAaelbDH26FXamVmN4ZXsvPjp/NYxNTBd4JJwvRrZWn\nupaqmH+n9AFO7Yu0RTOyQFq9gQjgzOMO5qaGSRm5Z0rZNaF95ho/GdBC78frsR8Pi7ZoTDmYGY3F\n+fnjL9Hw1H1wq1rgmyZOJxYYRJ2izKAKw0LM9a3EyzULh4K2A4rmtqjCMQX0Xi/zQKVdVEp/ck1o\nn7nGTwa00PvxepxLXLexLtIatSxEXd3ZwVkvPZmIoknzwb8dPpC7PjObxyZO67XgjZQDVom/UouJ\n99HR1c21TRtyDj10mkBlxvwQSr826el9VSKvqt7Un3K0WPU9fZA5HxE2moHHgBZ6P16PnR4WTu4Y\nq7DD6s4Ozl77BOf/6zHC+9pTD3zYYcwbf2KKwBuYwx7TB0Znf3os9YeMYP6yjSlC29oRs3SveH0r\ncTOBykxTSySjLZG2qGV4JKRWv8pHFbBCY/W5MN8HnYJA4ycDPqlZrpkQnZKWOSWiMotTdWcHF/1j\nKc//8jvM+98/por8YYfBvffCa6/xj8/NtPTDmx8uqoHRoYP989+bSU/+ZcfQygBXPWqdmE0VHtkj\nZW9WRdU5+tNApZsJcH4mLNMMbMrKos/Gz5yr39bJl+rk2hkTDhHs2MHF//gL3/1XU4YFv/egQxh6\n43w480wIJnLROE0askqaZYiGF/GWwLjG5QSE4Ixjx3JTwyTHa9EwpY5DG5crBTsYEAQDFZ5LDppF\nvBwGKt3eh/70lqIpXcpG6IuZfc/uYWHr2vnoI3699a8cdtPdDNmbKvDvjBjNncfN5sXjv8LnwqNZ\neev/ZkyFN7s+hgQreqf5200O8uJLN4hL2etWcBJ7uz4LBItPPZK5S9fZ7p/ug08X8WIMVPo938Lt\nfehPbyma0sVR6IUQ9wJfB96XUn4quWwEsBQYB7wFnCalbBVCCOCnwFeBDuAcKeVL+Wl6Kk4ukvQv\nqdWyfAiFlfU5Kr6PX2/9Hxh3Kke0tqZs/07NaO464XQe+eRU4hUB2BOz9N2ecnQdnd09vctbO2JK\n/7aB0U8vvnQzD76w1ZXQqyzug0ZU9o5LqEQuFAwoZ76aKWQETT6MCDf3ob+9pWhKFzcW/e+Bu4A/\nmJY1As9KKRcJIRqT//8Q+ApwePLnWODu5O+8Yxdvnf4lnffn9SD6JhH5Zf3bWX2LV2xiz44PueSV\npzj7hceo3LM7deePfQx+9CO+tW0M7+zpsj2PKneLncgLyGhPpC1qm1AsnfTzuZncZV4X3v06oBa5\nmqog1584seQGH1VGxOUPrWfu0nVZGQpW18gq6qbUroWmf+Io9FLKvwshxqUtPgmYmvz7PqCZhNCf\nBPxBSimB1UKIsBBitJTyXb8arEL1KmzU1zSTHmIIuecRsbX6DqumIbYKfn8btKXOZOVjH+Pfp57K\nJ2+6CQYNYquirmg6XlL9Qp+YmycZ3TF7MoCthZ2OUXXIKmImZXKXhcXd3Px673rjvP1B1FRGRHpJ\nP4Cwh+P2p7h+Tf8mWx99rSHeUsp3hRAHJJfXAVtN221LLsu70KvcBV5cFF4GvppaIin52q0s40Ht\nH/HhldfAmsczBf7jH4cf/Qi++U02PbOS7/7k77YThHIlHAoqsyF6cefMe3i9ZTw+eK+2VKw0E15x\n4083+n7zcQM6kE1TogjpQlSSFv2TJh99m5QybFrfKqWsEUIsBxZKKZ9PLn8WuFJKudbimBcAFwDU\n1tYevWTJEstzt7e3U11dbdu+tmiMHbv30RXvQSCQSCoDFdQOH9K73A2VgQrGHzjMcbu2aIxtrVFU\n166yYy9HrniSyU8tY0jH3pR1HXV1vP3tb/P+F76ADARoi8aIRTt4z4fgCoFAiEQoopkKIagQ1qUG\nKwMJYXJ7jdwwqW645XI399KKtmiMSGs0pV8VQlBXEyKsKJLuJ1bnV3Ho8EBWfTTOY3xejc9vIfrn\nlWzvY3+iv/Rx2rRpa6WU9U7bZWvR7zBcMkKI0cD7yeXbSE1ddRCw3eoAUsp7gHsA6uvr5dSpUy1P\n1NzcjGodJF0mz24gGqvAmBYQCgZ64+EtZ4lWiBQfvXmfqS6sxES6gsyZrPvta+ectU9w1r+aGN6Z\nKvAcfjj86EdUnXEGnxw0iE+ajnX6WLh1g/WtUGVdTF8fDgXZ29WdkbwsHAoyf+ZE5i5d5+CL99ES\n3bDXMpeN071UobredeEAqxq9Hy8b3KRlqAuHuLmuIqs+Wn+O4yycdUTJuXeyvY/9iXLrY7ZCvww4\nG1iU/P24afnFQoglJAZhd+fbP++Ua0blD7Za5vYLle7i2W9fO+euWcZ5ax5nvzSBbz/4UKpvWgBn\nnAGDMi+3Kk+7QVxK2wHTuJSEggGEsM5QuWdfoqhJNmGVueBneGsplLkzf5asjIfeCJnkgLMdVm4o\nXbNVk0/chFc+SGLgdX8hxDbgehIC/5AQ4jzgHeAbyc3/QiK0cjOJ8Mpz89DmFJxEwM63m+0XyBBN\nO4HfUjOGtWddxKk/udJS4M3HssrTbsbJYRCNxZX+9biUveGY6Wl67TAeLl6icqza5YdQhauCGSUN\nwf8Yc7fjAHaDycaAs905rMZKVPdFT5jS+IGbqJszFKs+b7GtBC7KtVFesJuQlK9JVFcffyCbr/0x\n577YlCHwb9aM4c4TzuCJT36OnooAV1y7QpmSt6klwt7ObvKNkS64pioISKIxe3+8WdxzHRbO9S2i\nqSVC+77MaxQMCF9jzL1+VrKNmFFZ7v2tMIqmf9HvQwTscs04TaLyTFsbzJ/P12Yez6X/+0CKyL85\noo45X7+cL373bpomTiNeEcgoUG2EJkKfsHhNBaAiHAomMlTa0NoRY5+NyNeFQ9SFQ57E3emcAlL6\n7ZXFKzZZRvgMrRzkq0vD98+KArtQTbucSRpNLvR7obdLTOZUtMK1ACUFnnHjYMEC2G2a7PSJT8D9\n9/PF837RK/BWpIuGm6RWbgkFA8yfOTExyOyAnYhPmzDKk6ugpirI4lOPtE1kJqG330be/UMbl7u+\n/qr27PbpAdnUElHWAoDEZyWXB1U6Kgvd+NzmkmBPo1FRFrluVK/RdgOQrtw4ra1wxx2Jn49Si27z\niU+w5qyLmSsmsG1DFxUBAQ7hd+aqSF5dIuFQkM7unoyHgxFVA9Dh4JJx4pG1EYaHgq7eMkLBQO8s\nVuP6jVNM9tqeFEtz3n23brR8ltSzGlS1ws+cSdMmjLJMCz1twig9gUqTN/q9RW+HlVvHjPLVvLUV\nrr8+YcHfcEOqyCct+KYHnuHb+w5n60ddSNzNVDWqItltGQ4FLV/h58+cmGHx3TF7Muuu/1Jv/phc\nicbiCIHtNTP6cdTBw5m/bCPjGpczrnE5U254WhnzPSYcYvGKTRlx6G5cI05poHPB7VuVny6cla99\n4Gm5RuMHZWHRq0jP62JFimvAzoIfPx6uu46mT3yWxX/bTGTDBs/tcXoUGIJutNlLpJBf0RltHTHO\nPO5gy1w6BhJY9UZqScPWjhiBCkGwQqT41A1Rnrt0nWUYqZuatZCfdAlerplf17cUQkU1A4+yFnro\nc+uo/LBjwqGEwN9+O/z0pxkC/8aIg/jjF89i8hXfg0CAeX9ebzk4aCY99YKbEMU6F/H9duF/dm4q\nI6KjLhyio6vbMlTRoKoy4JgFU0W8R7JfVZCqykEZbUxYxJlhpG5cMPlyaXiZW+BX9IsbV1Qx0z1o\nypOyF3oDq3wuB3Z38Ns3n4FxJ2cK/Mix/PT403lywmfpqQgQevxVQDqKvCHYVhNirL7gdeEQk+oq\nuOTMqbYhfoBt+N+8GYLh1QAAABB1SURBVOMtH0LBQCIHvN1kH/O2e7tyGyBu64jRct2XMpbPmzGe\nyL9TM2EUO6rE6jOhmjXtVzudiqYUs66CpnwZMEJvdgHsffd95mxYzpkvNhFMK/jBhAlcf+Qp/PHg\nY+gxRdC48eUaX1iVBZr+BTd89pve66EtacVZhfgteGIjH0W7lVWjjPOZk6wZxOKyd5sm0zkMK99s\n7e/t7HY1EGv3hqKyfBum1NH03qvUhQMlY6nmY9Z0tuc0L9czZDV+M2CEHqDhkBCffGMZB/3x1wzt\n7EhdOWECXHcdnHYaf7jmr55dF2bXizkVsFWO9vQ88F3xHtvoDztXi9m326bYzoh6MZ/DiNs2h/Ad\napMi2Vyce9qEUSx9cavl24Od5RsOBVnVOLX3gTN36ToWr9hUVMFXPZTz2R47V5T24WvywcAQ+l27\n4Pbbid1+B+PTLPjN+4+l9fJGPj0v4YMH73lhaqqCrGqc7vjabTdekG1MvdmCtvP/qizFOUvXcflD\n6znj2LE2JQDh1tOOTBGn+kNGpOSkd1s0RLsm7MlnOKlm4FLeQr9rF9x2G9x5J+zZgzn47/WRY7nz\n+NNZPuGzjKaaVYE+N42X/OzBgOD6E/siZdy8dvtpnZktaFW7nQZgjZqwJ3xsBLv2dmW4l8487mDf\nSvn54ZrI12BlKQyClkPhc03pUZ5Cv3NnIoomKfBmXh85lp+ecAZ/GX9Crw8+XXjT/ah2bhzzQKfd\n5KxDG5f3ioefmSTNQmT8nV79yU7kzax+s5VbTzsyJ7FzEku72cqTFzzN/Jn2bwX5eiPI53G9XM/+\nVn1L0z8oL6HfubPPgm9PG2Q94giuO3IW9x/06ZRBVrB+LTZbrKrQzLpwKOULaJc7XpJa2Ds9k2Qu\nWSLT2z1/2cas9o1LmVMooxuxtHvItUVjiXq+qMU1X4OV+Thutg8PPUNW4zflMTN250645prETNYf\n/zhV5I84ApYsgZdf5qh532fw4MqUXd28FtvNzjRypRzauNzV7NhoLM7K1z5ImeVaGajgzOMOdtvb\nXqqCqbevqSXC5AVPZ50oLSCcc+XY4SYxmNNs5ViPtJ2Fmq/Bynwct1CJ0jQaJ8pD6B94wFrgly6F\nDRtg9mwIBGwToDkxxCSq4VCQhbMmAYmQSae0BulE2qI0TKljVeN0tiz6GuMPHMZNDZOSaYTdE4vL\n3oRbbrJhOsn4cYfVeDp/OnZuGaOdxj3I5jigHpTMdbAyH8fVETSaUqE8hP788+HAAxN/mwX+tNOg\nIrWLZoFd1TjddZSI2c/d2Z1IHpZtBkpV6t7rT5zomGfGjNn6ddMWp7eGt3a6EyDzW4w5C6WdKJof\nQg1T6mwzXtodJ1+5b/Jx3Hw9lDQar5SH0IdCCd+8jcBni11YYrYDqhIs/ejpbxxuMKxDJyuxpirI\nTQ3ZW9IGxoPPeIuJtEWZs3QdZ/76n0ybMEq5XzQWZ8fufb3/q7YNVtjH4ufyVmZHPo6bz4RsGo0X\nymcw9gxVISzvmCMl/BggtaItGqOpJWIbtmiXJ93AsA7tBjmNlMKQEHxVFI4bS1P15rDqjV20vNNm\nu29XPPEm1NQSYemLWy23mX3MWEdxzddgpd/H1RE0mlKhPCx6H0m3WL3ipviHQTYpes2YrUPVtjVV\nwRTL9Gv/MdryWAEHS9rAzup3yodfGUh83FRVo6D80vV6dRVqNPlAC72JppYIlz+0PqfKT4u/cSTf\nOu5gV64XNyl6ze6Emqog4VDQ0rXQMKWOU46uy4icqapMfWlTCemwwe5K82XrXw4FA9QOHwLY91sP\nVFqjGhfRaNxQPq6bHDEseTchknYYuVvqDxnR+8ouBFgZsH6k6L22aYNt7vj02G23pflUE33mzRjP\nnKXrlO1Jz0cPfVWwwrtfB+zdTH4OVPox07UUZsvqtBGaXMnJohdCvCWE2CCEWCeEWJNcNkII8YwQ\n4vXk79xi9nzEzipyilpxG2Nu/hIar+y3nTY5L4Ny1zZt4P7V7zg+nMyx2yohrRAiI1TTPOBqFDdv\nmFLHCR8boT6ZIOWtw6iCBbDpvT0c2ricjq5uyw+eU1I0L9j1oZDH8AMdj6/JFT9cN9OklJOllPXJ\n/xuBZ6WUhwPPJv8vOk5fWjuXgcBdqUCD9C9hviJFHrCoParC6J/Klx+XknkPr09JZWzG3KcHzv8M\n31KEasbikqGDB6X4pJtaIsz783q64j1IkikZBIRMcxOMQuN+Wah+iGOpCKyOx9fkSj5cNycBU5N/\n3wc0Az/Mw3k84TTF3c6dkI0zxyp/jhsRc+sqaGqJeGqXYckbx7rsoXUZ7qRYXNq6Zcx9uqlhkrIS\nVXrf5y/bmOHO6ZGwL9aDSLbNb5eIH+JYKgKrM1pqciVXi14CTwsh1gohLkguq5VSvguQ/H1Ajufw\nBacv7bwZ4z1FzEDC/aKazZrNl9CLq8CLVSlIzXLZMKXOcszAifQ+uZ0QpJqtKyFvLhE/JiuVyoQn\nHY+vyRUhcxh8FEKMkVJuF0IcADwDXAIsk1KGTdu0Sikz/PTJB8MFALW1tUcvWbLE8hzt7e1UV1dn\n3UaDTe/t6Y3jNlMZqGD8gcMA+Pe7H9HtUgEFgoNGJL7w23ZFkSbb1lgXDrlLaWD00U0bDTZEdrs6\nNsDIoZUZ4uRlf0j48OtqUvvUFo0RaY3SY/oMGdsB7Ni9L6U/tSHYYWMMW/UzW+za5va+ZHMMvz6v\nVm0xrmdloILa4UNc98Nv8tXHUqK/9HHatGlrTW5zJTm5bqSU25O/3xdCPAYcA+wQQoyWUr4rhBgN\nvK/Y9x7gHoD6+no5depUy3M0NzejWueFNotaqUaFpalJl8G5jctdu0MEsGXRF2lqifDT59en1BhN\n1Gk9ove4YO+SMfqYOH/mS5YAbp99eMr+HV0hy4lPgwdVEBB9Me1CwJnHHsQ3p6bOip3jMvmZk2vF\nql9glE2swPzSePmkbm7doP7IJa7pVMc2uaUYUTd+fV5LGd3H/kfWQi+EGApUSCn3JP/+EnADsAw4\nG1iU/P24Hw3NFTezFL3kiTcs5MUrNqWIPKTWaQX34XGq84erghn7BysEwYDIKGJ91MHDWfXGrt5l\nUsL9yUFbcwqE+TMnWhYTN1MXDrGqcbrtdbAaezhh0XNZzUXw2yXix0xXnTJYUw7k4qOvBZ4XQqwH\nXgSWSyn/SkLgvyiEeB34YvL/ksBplqKVLzRg4bc3+0fdDNgteGKjq+gNlS9WysxSg7EeSSwue8M+\njUie1W+2WrbnwRdSUw40TKlj8TeO7E0ult5LNz5gVbiqUwRTTVUwYzxE+5w1mvyRtUUvpXwTONJi\n+U7g87k0qliorH6rZca2YUXuGMM6bWqJKHPLOFW2Ms411yYSxijybbRJFTVjFR5qtla9uijs3lJU\nbyaVgQq2LPpaVufTaDTZo2fGpqF6VVf5p9v3dWcsN0/8sYuOcapsZbB4xSZbl5I5TFRV5cppwpdX\nF4VduKqq7mnt8L6iL9olotEUDp3rJgdUybmGVvbljbFzY7h1VTglNzOf54xjx1quVy3PFjuXlWqC\nWLGiRDSagY626HPATd4Y5QBrKOjaojW7dJxyxBgDrkb+m4AQnHHsWMdc9F5xmsRjZbE3N7/uaxs0\nGo07tNDngJsZiyo3xvyZEz2dyxDOdN+4cTzz28FNDZN8F/Z05s0YnxG141Q0RKPRFAftuskBNzMW\n/c5zk6+8OVmR7vbPrba4RqPJE9qizwG3FYTyUbmo2AOZbuYPaDSa0kALfY6UgugWg1JJ+KXRaJzR\nrhuNZ5paIlQowjV1RkWNpvTQQq/xhF0lLj27VaMpTbTQazyhqsQVEKJ4g8IajcYWLfQaT6h88D1S\napHXaEoULfQaT5RKMQ6NRuMeLfQaT+hqRxpN/0OHV2o84XbugEajKR200Gs8M1DnDmg0/RXtutFo\nNJoyRwu9RqPRlDla6DUajabM0UKv0Wg0ZY4Weo1GoylztNBrNBpNmaOFXqPRaMocIS2yEBa8EUJ8\nALytWL0/8GEBm1MMBkIfYWD0U/exPOgvfTxESjnKaaOSEHo7hBBrpJT1xW5HPhkIfYSB0U/dx/Kg\n3PqoXTcajUZT5mih12g0mjKnPwj9PcVuQAEYCH2EgdFP3cfyoKz6WPI+eo1Go9HkRn+w6DUajUaT\nAyUh9EKIEUKIZ4QQryd/1yi2+6sQok0I8WTa8kOFEC8k918qhKgsTMvd46GPZye3eV0IcbZpebMQ\nYpMQYl3y54DCtd4eIcSXk23bLIRotFg/OHlfNifv0zjTuquSyzcJIWYUst1eyLaPQohxQoio6b79\nstBt94KLfn5OCPGSEKJbCHFq2jrLz26pkWMf46Z7uaxwrc4RKWXRf4BbgMbk343Afyu2+zxwIvBk\n2vKHgNOTf/8S+K9i9ymbPgIjgDeTv2uSf9ck1zUD9cXuh0WbA8AbwGFAJbAeOCJtmwuBXyb/Ph1Y\nmvz7iOT2g4FDk8cJFLtPPvdxHPBKsfvgYz/HAf8B/AE41bRc+dktpZ9c+phc117sPmTzUxIWPXAS\ncF/y7/uABquNpJTPAnvMy4QQApgOPOy0f5Fx08cZwDNSyl1SylbgGeDLBWpfthwDbJZSviml7AKW\nkOirGXPfHwY+n7xvJwFLpJSdUsotwObk8UqNXPrYn3Dsp5TyLSnly0BP2r795bObSx/7LaUi9LVS\nyncBkr+9uCVGAm1Syu7k/9uAUix/5KaPdcBW0//pffld8pXxRyUkIk5tTtkmeZ92k7hvbvYtBXLp\nI8ChQogWIcT/CCH+X74bmwO53I9yupd2DBFCrBFCrBZClKJBaUnBSgkKIf4GHGix6ppcD22xrCih\nRD700a4vZ0opI0KIYcAjwLdJvFoWGzfXX7VNydw7B3Lp47vAwVLKnUKIo4EmIcREKeVHfjfSB3K5\nH+V0L+04WEq5XQhxGPCcEGKDlPINn9qWNwom9FLKL6jWCSF2CCFGSynfFUKMBt73cOgPgbAQYlDS\nkjoI2J5jc7PChz5uA6aa/j+IhG8eKWUk+XuPEOJPJF5BS0HotwFjTf9bXX9jm21CiEHAcGCXy31L\ngaz7KBOO3U4AKeVaIcQbwCeANXlvtXdyuR/Kz26JkdNnTkq5Pfn7TSFEMzCFhM+/pCkV180ywBil\nPxt43O2OyS/SSsAYHfe0fwFx08cVwJeEEDXJqJwvASuEEIOEEPsDCCGCwNeBVwrQZjf8Czg8GflU\nSWIgMj0awdz3U4HnkvdtGXB6MmLlUOBw4MUCtdsLWfdRCDFKCBEASFqBh5MYqCxF3PRTheVnN0/t\nzIWs+5js2+Dk3/sDJwCv5q2lflLs0eDkSPZI4Fng9eTvEcnl9cBvTNv9L/ABECXxZJ6RXH4YCYHY\nDPwZGFzsPuXQx+8k+7EZODe5bCiwFngZ2Aj8lBKKTgG+CvwfCcvmmuSyG4CZyb+HJO/L5uR9Osy0\n7zXJ/TYBXyl2X/zuI3BK8p6tB14CTix2X3Ls56eT3729wE5go91ntxR/su0jcDywIXkvNwDnFbsv\nbn/0zFiNRqMpc0rFdaPRaDSaPKGFXqPRaMocLfQajUZT5mih12g0mjJHC71Go9GUOVroNRqNpszR\nQq/RaDRljhZ6jUajKXP+P/9XagXF11oMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a122ee278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Find the index of the single variable with the best R^2\n",
    "imax = np.argmax(Rsq)\n",
    "\n",
    "# Regression line over the range of x values\n",
    "xmin = np.min(X[:,imax])\n",
    "xmax = np.max(X[:,imax])\n",
    "ymin = beta0[imax] + beta1[imax]*xmin\n",
    "ymax = beta0[imax] + beta1[imax]*xmax\n",
    "plt.plot([xmin,xmax], [ymin,ymax], 'r-', linewidth=3)\n",
    "\n",
    "# Scatter plot of points\n",
    "plt.scatter(X[:,imax],y)\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the calculations above could have been done without a for-loop using Python broadcasting:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove the means\n",
    "ym = np.mean(y) \n",
    "y1 = y-ym  # a column vecotor each minus mean\n",
    "Xm = np.mean(X,axis=0) # averaging over column, resulting a row vector of dimension natt\n",
    "X1 = X - Xm[None,:] # minus the same mean in each column\n",
    "\n",
    "# Compute the correlations per features\n",
    "syy = np.mean(y1**2)\n",
    "Sxx = np.mean(X1**2,axis=0) #a row vector with each element indicating the variance of one attribute\n",
    "Sxy = np.mean(X1*y1[:,None],axis=0) #a row vector with each element indicating the covarance on one attribute to the targer\n",
    "\n",
    "# Compute the coefficients and R^2 value per feature\n",
    "beta1 = Sxy/Sxx # element wise division, resulting a row vector containing  beta1 for each attribute\n",
    "beta0 = ym - beta1*Xm # element wise multiplication, resulting a row vector containing beta0 for each attribute\n",
    "Rsq = Sxy**2/Sxx/syy #a row vector containing Rsq for each attribute"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us see whether we get the same result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 0 Rsq=0.035302 beta0=152.133484 beta1=304.183075\n",
      " 1 Rsq=0.001854 beta0=152.133484 beta1=69.715356\n",
      " 2 Rsq=0.343924 beta0=152.133484 beta1=949.435260\n",
      " 3 Rsq=0.194908 beta0=152.133484 beta1=714.741644\n",
      " 4 Rsq=0.044954 beta0=152.133484 beta1=343.254452\n",
      " 5 Rsq=0.030295 beta0=152.133484 beta1=281.784593\n",
      " 6 Rsq=0.155859 beta0=152.133484 beta1=-639.145279\n",
      " 7 Rsq=0.185290 beta0=152.133484 beta1=696.883030\n",
      " 8 Rsq=0.320224 beta0=152.133484 beta1=916.138723\n",
      " 9 Rsq=0.146294 beta0=152.133484 beta1=619.222821\n"
     ]
    }
   ],
   "source": [
    "for k in range(natt):\n",
    "    print(\"{0:2d} Rsq={1:f} beta0={2:f} beta1={3:f}\".format(k,Rsq[k],beta0[k],beta1[k]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improvements with a Multiple Variable Linear Model\n",
    "\n",
    "One possible way to try to improve the fit is to use multiple variables at the same time.  We can fit the multipe linear model using the `sklearn` package.  \n",
    "\n",
    "For reasons that we will explain in the next demo, we need to split the data into two parts:  one part for training the model and a second part for testing the fit.  In this example, we will use `ns_test=300` samples for training and the remaining `ns_test=442-300=142` for testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "ns_train = 300\n",
    "ns_test = nsamp - ns_train\n",
    "X_tr = X[:ns_train,:]     # Gets the first ns_train rows of X\n",
    "y_tr = y[:ns_train]       # Gets the correspoinding rows of y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To fit the linear model, we first create a regression object and then fit the data with regression object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "regr.fit(X_tr,y_tr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "152.34786451820108"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ -16.57607993, -254.66532396,  560.98630022,  278.91811152,\n",
       "       -393.41357305,   97.05460405,  -19.0023093 ,  169.46450327,\n",
       "        632.95050374,  114.21638941])"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We next compute the RSS (per sample) and the R^2 on the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RSS per sample = 0.485281\n",
      "R^2 =            0.514719\n"
     ]
    }
   ],
   "source": [
    "y_tr_pred = regr.predict(X_tr)\n",
    "RSS_tr = np.mean((y_tr_pred-y_tr)**2)/(np.std(y_tr)**2)\n",
    "Rsq_tr = 1-RSS_tr\n",
    "print(\"RSS per sample = {0:f}\".format(RSS_tr))\n",
    "print(\"R^2 =            {0:f}\".format(Rsq_tr))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see R^2 is higher than the best single variable model -- more than 51% of the target variance is explained by the model instead of just 34% for the best single variable model.  \n",
    "\n",
    "We also create a scatter plot of predicted vs. actual"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXl4VOXVwH8nYYAAlkDFiAHFKkVB\nFITigrWAWlCLpri3/bQuxb1KKRq0FagLUVSq1qVW61oFt0YUFBGICxatGBAREVpRCYhaCLIMMEne\n7497J0xm7r1zZzIrOb/nyZOZO3c5c2fmPe97VjHGoCiKoijRFGRbAEVRFCU3UQWhKIqiOKIKQlEU\nRXFEFYSiKIriiCoIRVEUxRFVEIqiKIojqiAURVEUR1RBKIqiKI6oglAURVEcaZVtAZrDnnvuaXr0\n6JHUsVu3bqV9+/apFSiN5JO8+SQr5Je8+SQr5Je8+SQrNE/eRYsWfWuM6RJ3R2NM3v4NGDDAJMv8\n+fOTPjYb5JO8+SSrMfklbz7Jakx+yZtPshrTPHmB942PMVZNTIqiKIojqiAURVEUR1RBKIqiKI6o\nglAURVEcUQWhKIqiOJI2BSEibUXkPRFZIiLLRGSSvf1REflMRBbbf/3s7SIid4vIKhH5UEQOT5ds\niqIoSnzSmQexAxhmjNkiIgHgbRF5xX5tnDHmuaj9TwR62n9HAPfb/xVFUZQskLYVhB1uu8V+GrD/\nvPqbngo8bh+3ECgWka7pkk9RFCUvCYWgooI9Pvkk7ZcSk8ae1CJSCCwCDgTuNcZcKyKPAkdhrTDm\nAuXGmB0i8jJQYYx52z52LnCtMeb9qHOOBkYDlJSUDJg2bVpSsm3ZsoUOHTok98ayQD7Jm0+yQn7J\nm0+yQn7Jmw+ydli5kl5TprDHypWsOu001lxxRVLnGTp06CJjzMC4O/rJpmvuH1AMzAcOAboCArQB\nHgNusPeZCRwTccxcYIDXeTWTOjfJJ1mNyS9580lWY/JL3pyWNRg05rrrjCksNKakxJjnntt9MqmN\nMbVAFTDCGLPOlnEH8AgwyN5tDdA94rBuwNpMyKcoipKzLFgA/frBLbfAuefC8uVw2mkZuXQ6o5i6\niEix/bgIOB74JOxXEBEByoCP7ENmAOfa0UxHApuMMevSJZ+iKEpOs3kzXHkl/PjHsH07zJ4Nf/87\ndOqUMRHSGcXUFXjM9kMUAM8YY14WkXki0gXLzLQYuMTefxZwErAK2Aacn0bZFEVRcpfZs2H0aPjy\nS0tJ3HwzZME/kjYFYYz5EOjvsH2Yy/4GuDxd8iiKouQ8GzbA734Hjz0GBx0Eb70FgwdnTRzNpFYU\nRckFnn8eeveGJ5+E66+H6uqsKgfI84ZBiqIoec+6dXDFFfDCC3D44fDqq5ZTOgfQFYSiKEo2MAYe\necRaNcycCRUV8O67OaMcQFcQiqIomWf1assJPWeOFaX00EPwwx9mW6oYdAWhKIqSKerr4e674ZBD\n4F//gnvvhaqqnFQOoCsIRVGUzLB8OVx0EbzzDowYAX/9K+y7b7al8kRXEIqiKOkkFLLyGPr1g08+\ngSeegFmzcl45gK4gFEVR0seiRXDhhbBkCZx5JtxzD+y1V7al8o2uIBRFUVJNMAjl5XDEEfD11/DP\nf8L06XmlHEBXEIqiKKnlzTctX8PKldbq4fbbobg421Ilha4gFEVRUsF338Hll8NPfgJ1dfD661b4\nap4qB1AFoSiK0nxeecUKXb3/frj6ali6FI47LttSNRs1MSmKoiTL//4HY8ZYkUm9e1shrEcemW2p\nUoauIBRFURLFGHjmGTj4YHj6abjhBvjgg91KOYCuIBRFURJj7Vq47DJ48UUYONDyNRx6aLalSgu6\nglAURfGDMfDww5YpafZsmDLFKpexmyoH0BWEoihKfP77X/jNb2DePCtK6aGH4MADsy1V2lEFoShK\nRqmsrmHK7BWsrQ2yT3ER44b3oqx/abbFcqa+3sp+vv56KCyEBx6wFEVByzC+qIJQFCVjVFbXMP6F\npQRD9QDU1AYZ/8JSgNxTEsuWWYlu774LJ59sKYdu3bItVUZJmxoUkbYi8p6ILBGRZSIyyd6+v4i8\nKyIrRWS6iLS2t7exn6+yX++RLtkURckOU2avaFQOYYKheqbMXpEliRzYuRP+9Cfo3x/+8x946il4\n6aUWpxwgvU7qHcAwY8xhQD9ghIgcCdwKTDXG9AQ2Ahfa+18IbDTGHAhMtfdTFGU3Ym1tMKHtGeff\n/7YikyZMgNNPh48/hnPOAZFsS5YV0qYgjMUW+2nA/jPAMOA5e/tjQJn9+FT7Ofbrx4m00E9FUXZT\n9ikuSmh7xti2jR888ICVx7BhA8yYYa0cunTJrlxZRowx6Tu5SCGwCDgQuBeYAiy0VwmISHfgFWPM\nISLyETDCGLPGfu0/wBHGmG+jzjkaGA1QUlIyYNq0aUnJtmXLFjp06JDcG8sC+SRvPskK+SVvPskK\nsfLWBkPUbAzSEDHuFIhQ2qmI4qJANkSkePFifnj77bSrqWHtz37Gfy6+mPo8uMfN+S4MHTp0kTFm\nYNwdjTFp/wOKgfnAj4FVEdu7A0vtx8uAbhGv/Qf4vtd5BwwYYJJl/vz5SR+bDfJJ3nyS1Zj8kjef\nZDXGWd5/frDGHD15rulx7cvm6MlzzT8/WJN5wYwxprbWmIsvNgaMOeAAU33nndmRI0ma810A3jc+\nxu6MRDEZY2pFpAo4EigWkVbGmDqgG7DW3m2NrTDWiEgroCOwIRPyKYqSOcr6l2Y/Yunll+GSS2Dd\nOvj972HSJGrfey+7MuUg6Yxi6iIixfbjIuB4YDnWSuJ0e7fzgBftxzPs59ivz7M1naIoSmr45hv4\nxS9g5Ejo1MnKhJ4yBdq1y7ZkOUk6VxBdgcdsP0QB8Iwx5mUR+RiYJiI3AdXAw/b+DwNPiMgqrJXD\n2WmUTVGUloQxMG0a/Pa3sGkTTJpkdXxr3TrbkuU0aVMQxpgPgf4O2/8LDHLYvh04I13yKIrSQlmz\nBi691DIrDRpk1VM65JBsS5UXtIx8cUVRWh4NDfDgg9CnD8ydC3feafVrUOXgGy21oSjK7seqVVbN\npKoqGDbMUhQHHJBtqfIOXUEoirL7UFcHt98OfftaDXz+9jerX4Mqh6TQFYSiKLsHS5daxfX+/W84\n5RS47z4ozbECgHmGKghFUZKmsrqGSS8tY+O2EADFRQEmntIns3kOO3bALbdYf506WdFKZ57ZYusn\npRJVEIqiJEVldQ3jnltCqH5XulJtMMS4Z5cAGSrf/e671qph2TL41a9g6lTYc8/0XzeDZLN/hvog\nFEVJiimzVzRRDmFCDSb95bu3boXf/Q6OOsrKa3j5ZXjiid1SOYx/YSk1tUEMu/pnVFbXZOT6qiAU\npQVRWV3D4Ip57F8+k8EV85o10HiV6E5r+e5586w+0FOnWuUyli2zGvrshmS7f4aamBSlhZDqbm77\nFBdR46II0lK+u7YWxo2z+kH37AlvvAHHHpv66ziQLTOPd/+M9mm/vioIRWkheM1Gkxnsxg3vFeOD\nAAgUCOOG92qWrDG8+KKVDb1+PVxzDUycCEWpV0JOigDIWptUNyWcqf4ZamJSlBZCqru5lfUvZcrp\nh9Gp3a4+DsVFAaaccVjqBs6vv4azz4ayMqt5z7vvwq23pkU51AZDjvb+SS8ty5qZZ9zwXhQFCpts\nKwoUpl4Bu6ArCEVpIaRjNpqK0t2O5pt++8A//gFXXQVbtsCNN8K110IgfU2F1m/aTjDUdM4cDNXH\nKIcwmWiTGr63TuatqqqVab++KghFaSGMG96riakEMjsbdcLJL3LXI/M4svpR9n57ntUC9OGHoXfv\ntMuys76BRIwqmTLzZLN/hioIRWkheM1Gs0WkX0RMA79c/CrlVY9QaAzcdRdcfjkUFsY5S2poXeis\nHESsauGRZFuxZgpVEIrSgsiJbm4RhM00+2+ooeKVuzlizTLe2q8f1424grd+e2FGZSnp2JaiQKxJ\nKVo5pDtbPJuJcdGoglAUJWt036M1J77+FGPefoodhQHGnXgVz/Y9ntJOme/wVlwUYPKo3o2Dc4EI\n9Q5NLdu3aZXSATtSIRS3C7Blex2hBuu6mYyYckIVhKIo2WHJEmY8OZbiT5by6g+P4o8nXMo3HTpn\n1XwTucLav3ym4z6pdE5H+2DCNa0iaU4ocnNRBaEoSmbZsQNuugkqKiju3Jn3bnuAG+sO5NtN2ylN\nwqSSLpNMJnIQnHJTnMhExJQTqiAURckc77wDF10Ey5fDeefBHXcw6PvfZ0GSp0t1dngkmYj68jvw\nZypiKpq0JcqJSHcRmS8iy0VkmYhcZW+fKCI1IrLY/jsp4pjxIrJKRFaIyPB0yaYoSobZssXKaTjm\nGKvQ3quvwqOPwve/36zTprNWUVn/UiaP6ktpcREClBYXMXlU35SaevwM/Nk0uaVzBVEHjDXGfCAi\newCLRGSO/dpUY8ztkTuLSG/gbKAPsA/wuoj80BgTf/2lKEruMmcOjB4Nq1fDFVdYfRv22CMlp051\ndng06Y76clqlBAqF9q1bsSkY2n2jmIwx64B19uPNIrIc8HqXpwLTjDE7gM9EZBUwCPhXumRUlN2Z\nrIdLbtxIr1tvtVYLvXrBW29ZK4gUku1aRc0lF3NTIsmID0JEegD9gXeBwcAVInIu8D7WKmMjlvJY\nGHHYGrwViqIoLqTTNu+Lf/4TLruMvb/+GsaPhxtugLZtkzpVWNHV1AYptENPw87sXMwOT5Rcy02J\nRIxDnG9KLyDSAXgDuNkY84KIlADfAga4EehqjLlARO4F/mWMedI+7mFgljHm+ajzjQZGA5SUlAyY\nNm1aUnJt2bKFDh06JPu2Mk4+yZtPskJ+yetX1hVfbbZLRzSldWEBvfZOjXnHidYbNnDg3Xez1xtv\nsPnAA6m+/HIa+vVL+ny1wRA1G4M0OIxTBSKUdrJWCus3bWdnfQOtCwso6diW4qLEazbl0/cAmifv\n0KFDFxljBsbbL60rCBEJAM8D/zDGvABgjFkf8frfgJftp2uA7hGHdwPWRp/TGPMg8CDAwIEDzZAh\nQ5KSraqqimSPzQb5JG8+yQr5Ja9fWc8vn4lxiEER4LOK+McnjDHw+OMwZgxs2wa33MIev/89DQsW\n+JLXzRw2uGIeNbXupTZKiwtZUD4sJW8hn74HkBl506YgRESAh4Hlxpg7I7Z3tf0TAD8HPrIfzwCe\nEpE7sZzUPYH30iWfouzOZNQ2//nncPHFMHs2DB5sNfQ56CDAWgEMrpgXM/BHKoSORQG27qxr7CsR\naQ6L52zOVn5ASyGdK4jBwP8BS0Vksb3tOuAcEemHZWJaDVwMYIxZJiLPAB9jRUBdrhFMipIcGbHN\nNzTAffdBebn1/J574LLLoMBauVRW11CzMdi4AggP/O9/voHnF9U0ylYbdM8e9upaB5bCy7ozfjcm\nnVFMb2OtaKOZ5XHMzcDN6ZJJUZxwm+XmM2mPjlmxAi68EBYsgOHDmX3lJP60dCtrr3ul8VpTZq/g\n7O5NfQfBUD1Pv/ulY42jaNbWBpl6Vr8YRRemKFDI0IO6ZNUZv7srJ82kVlo0brNcyE5xtFSSluiY\nUAhuvx0mTYJ27eDRR6nsexzj//lRzCAdDNU39Sra+FEOYK0OIhWdUxRTqtuoJkLWI8UygLYcVVJG\nZXUNgyvmsbRmE4Mr5lFZXZO2a+xfPjMl15gye0VMhEym2klmipTds+pqGDQIrrsORo6Ejz+G885j\nymufOg7SheJkQHA2K0QTXh0MrpjHmOmWhfrPZ/XjjjMPo7S4iLW1wUal4UQmfBPpzOLOFXQFoaSE\nJrOp7umZTaVjxra2Nug4y8228zNVpotk71nk9Xu0L+Svn73MDx9/wOoL/fzzMGpU475u96reGAqi\nlESgQGgA6huaKuUCoGO7ALXbQhS3C7AjVM+TC79ofL2mNsi4Z5eA0MSZ7UYmEuXSncWdC+gKQkkJ\nmZhNpeMabgNJNjNxw4N6TW0Qw65B3W3m77VCSOaeRV5/wJplPPTni/jhI/fy+cmnW6uGCOUA7veq\ntLiI0k5FTWoZdWjbKkY5gKUcqm/4KVPP6sf2UAPbQrE5HKEG06gcvAgUiKczPlUrqlz87qQaXUEo\nKSETs6l0XGPc8F7ULF/UZFu2M3G9BvWbj2w6p4u3Qoh3z5xWKlNmr6Bgy2YmvfkY530wky87lvCr\nM2/ks/5Hs6BTp5hzeUVMFW9ayYLyIY3b3Xos1Np9EPyWv/aiQ1v3hj5e96s4wevsDlnc8dAVhJIS\nMjGbSsc1yvqXxsxyU12xM1ESUYTxVghe98xtpXLAB28z+++X838fzOKRASMZfsFfeHv//q5yJVL1\nNN5nmIoJRVjZOJHKVWgmqr1mG11BKCkhE7OpdF2juCjQZJabbRJJcounTLzuWfRg2TG4mRtm/o3T\nPprHqs7dOP2Xt/FBt4M9rx/Gb8RUvM/QK+8hUCBNfBBueMnpfb/ax5E+llyuo5QKdAWhpITI2RSk\nZzaV7hlbqiOkkmXc8F4UBZqWl3BThPFm5F73rHGwNIYTP3mb1x+6lFM+foN7jjqL00bf20Q5pErZ\nx/sMnd47WEp8yhmHMeX0wxqP7dQuYCmNCOLJ2RL8BqlEVxBKygjPpqqqqrjyl0PSeo1Uk0sx7V5J\nblVVK5vs62dV5XbP9ikuYueaGm6ccz8jPv0XS0sO4Lwz/8SmXn2YZK8w0pEA5vUZ+knwi3ycaLSX\n5/3atNL1uFQmxOVTcp0qCEXB2zadjR+vX0WYdMa0MdyzfTEHPDyBNnU7mTzk1zz0o5/Tuk1rJtvH\nJxtW20SWwxJ3OPu9djIDbfj1iTOWNZb4aBvwNqSkcvKQSxMRP6iCUBQScwwnOjClcsboVhYkofN9\n9hmMHs3hr7/Ot/2P4OKhl/JuYM+UzIyjB7+ajfVUVtfEnLO596S5A+2Oul1htBu3hRj/wlImH+1c\nNTaVk4dcm4jEQxWEouDfMZzowJTq2WezyoLU18O991oNfAoK4L772PPii5lWkBpXpNPg12BMzOAX\n7574UR7NGWjdjl2/yTn6KZXh1fmWXKdOakXBv2M40TDJVIZVNqssyMcfw49/DFddBT/5CSxbBpde\n2lh5NRX4Hfy87onfJMHmDLRu+zg1WILUOrbzzUmuCkJR8B8hlejAlPXZZygEN90E/fvDp5/CE0/A\nzJmw774JXz8efgc/r/fhV6E2Z6B126d1ofNwmEhUWTxSea5MoCamPGF3LEmda/ix5SfaiCeVjXus\nYzb7P9eiRXDBBfDhh3DWWXD33bDXXglf1y9OEUIFElv2wuue+M38rqkNIlhNZcL4HWjdIplKOrZ2\n3D+VpdPTXoY9xaiCyAN255LU+UaiyXqpTO7zXRYkGISJE62y3CUlUFkJp56a8PUSxWnwK+1UH/Md\ndbsnQw/q4torIjLzO3ycgUYlUeow0Lr5MtwG6WKPMNdUhlfnU3Kdp4IQkd95vR7ZSlRJH26NV3I1\n8iFfCA8gZ3ffzPUV8xIKk/Q7A0z17LPyq48pLS50P9ebb8JFF8HKldb/KVOgONEqQ8kTPfhVVVU5\n7gNN78nQg7rw/KIaR+XglvkNu5RDdF/qeI7waBmmzF6RVEju7k68FcQe9v9ewI+w+kYDjATeTJdQ\nSlNytSR1PlNZXcO4Z5cQajCN5cmvnr6Y9z/fwE1lfT2PTXQGmMoZo2tZkO++s1p/3n8/7L8/vP46\nHHdcSq6ZDqLvyeCKeY5F+gpFGn1B4b4Q0SRaoyocKeU3JLcl4+mkNsZMMsZMAvYEDjfGjDXGjAUG\nAN0yIaCSf5EP+cDEGcss5RDFkwu/yFqJjaSZNQv69IEHHoAxY2Dp0pxWDk641V+qN6ZxwE7kdxDP\nl+EVkqvswm8U077AzojnO4EeXgeISHcRmS8iy0VkmYhcZW/vLCJzRGSl/b+TvV1E5G4RWSUiH4rI\n4Um8n92SccN7xTReyeXIh3wgnEXrRN4MEt9+C7/6FZx8Mnzve/DOO3DnndA+8aJz2cat+1zk9lTW\nqMq3fIRs4VdBPAG8JyITRWQC8C7weJxj6oCxxpiDgSOBy0WkN1AOzDXG9ATm2s8BTgR62n+jgfsT\neie7MblYknp3JucHCWNg+nTo3dv6P2ECfPABHHlkSi+TTPHC6GO8FHEkbn2qI7cnUqwxnjLRVbk/\nfEUxGWNuFpFXgB/bm843xlTHOWYdsM5+vFlElgOlwKnAEHu3x4Aq4Fp7++PGGAMsFJFiEelqn6fF\nk2slqfOdTu0CbHTpG5DsIJFI+YhkS020/vZbKCuDGTNg4ECYOxf6evtMkiGZbGcgabt+qUvoa2nU\nZ5GqGlV+Q3JbOomEubYDvjPGPCIiXURkf2PMZ34OFJEeQH+slUdJeNA3xqwTkXBgdinwZcRha+xt\nqiCUGJpby2fCyD6MfXZJTPvLeO0qveTxW1IjqfIbxsDDDzPo6qutkhm3325lRbdKT6R6vIQ1J/nb\nBgp8ldpwIh29PhKtGusUktvSEeOytGuyk2VWGgj0Msb8UET2AZ41xgz2cWwH4A3gZmPMCyJSa4wp\njnh9ozGmk4jMBCYbY962t88FrjHGLIo632gsExQlJSUDpk2b5vvNRrJlyxY6dOiQ1LHZIJ/kTbes\ntcEQNRuDTcpOFIhQ2smaba7ftJ2d9Q20LiygpGNbiosCrudZVxvk+20M64NQWCDsU1zkur8XK77a\n7FiqoXVhAb323iPpfQHa1tTQ64476FRdzf8OOYRV5eUES9M7kC2t2eT6WuvCAteyFNGUFMH6IPQt\n7Rh339pgyPdnlyh+zp1PvzFonrxDhw5dZIwZGG8/v9OPn2OtAD4AMMasFZHYb3IUIhIAngf+YYx5\nwd68Pmw6EpGuwNf29jU0DebsBqyNPqcx5kHgQYCBAweaIUOG+HwLTamqqiLZY7NBsvJmo/Z8uu/t\n4Ip5jUmDkRQXGXbUNRAMFRB2rxUF6pk8qrfne66qquJMF3n93r/zy2diHFx6AnxWMSS5fevr4a67\n4A9/sFYKf/0rSw88kCHDhsUcmwh+3tP1FfNcTT5ra4OO8jsxtm8d077cI239QfxQWV3DuNeWEGrY\n9b0IFISYckbT70VLGRMSwa+TeqftGzAAIhI3TEJEBHgYWB6VUDcDOM9+fB7wYsT2c+1opiOBTS3Z\n/5Cssy/6HH4Kn+Ubbk7k2mAoZYXxwPn+jZm+mB4OTttEnJ6+9v3oIzj6aBg71gpZ/fhjGD262cX1\n4n0nwt+7cCmLSMImHzf5i4sCMY7hXLDrO4U0hxoME2csy5JE+YPfFcQzIvJXoFhEfgNcADwU55jB\nwP8BS0UknOFyHVBhn+9C4AvgDPu1WcBJwCpgG3C+73exm5GqJJ58qz3vF6++xU4kG5XklrkLuwbW\n9z/fwPxPvnGsDRQoELbtrGP/8plNZuqe9vadO2HyZLj5ZujYEZ56Cs4+G+xwz+bW5ErEt+BVysJJ\n/omn9Gm8Rjbs+m4rI7fJVTKTrpaG3yim20XkBOA7rKzqG4wxc+Ic8zbETELCxGTx2CuUy/3Is7vj\nt65+PHbXWG+3AbZtoMAxMinZqKR49ykYqucfC79oVAqRA2pxUYCtO+sa5XFyRMcMZnVrYcAIa/Xw\ni1/An/8MXbo0Xi8VNbkSraTqVMoiXoRQvFIbkaTKBNokMx7r3ox7dknC51Ga4ktBiMitxphrgTkO\n25QUk6qBvdgllNPPgJnsD7eyuob1X23m/KhZcypxG6AgdmYbKBS27oidxfvBz0olOsQjPKBC7Aw1\ncvXWJMJm2za44QaYOhW6drVCWEeOjLlWKmpyNaeSaiSpKB+SymZKXmYkt5DmTu1S4wDfnfFr0DzB\nYduJqRRE2UUqkngqq2vYsr0uZnugML5N2M1O/YfKpZ6JU+HjdtY3pNXn4VWlMzKRqlO7ABhroE5G\nHqdkKz+srQ36H2znz4dDD4U77oDf/MZq5OOgHByPjbPdCa8Eskwnj6WymZKXGWnCyD4ECpsaMwKF\nwoSRfRK+TksjXjXXS4HLgANE5MOIl/YA3kmnYC2ZVCTxTJm9wrHWUPvWrZJuyRhpTnEzmaTb5+Gn\nSmf4WoMr5sXMHIOhesY+s6SJ3G6EX5/00jLXpDonwgOqZx+ITZvgmmvgwQfhgANg3jwYOtT1nJXV\nNTHlVmLO6YN45qFU5yJEE6nc3QLsU20CzbceDLlEPBPTU8ArwGR2lcQA2GyM2ZA2qVo4qUjicfuR\nbfLhmHM7NvoHHQzVM3HGska5MuHzSEQJuV233hjfpoyy/qVMmb3CUUEUBQoAcR1QXQfbl16CSy6B\nr76C3/8eJk2Cdu1cZQgrRa9S2IngZh5K90AardzdSGbFEs+MlE89GHIJTwVhjNkEbBKRu4ANxpjN\nACKyh4gcYYx5NxNCtkT81NX3ojmdzBKJEqoNhhqjq1LZPS2ayE5iTjgpA6/3kcjKxk3RbA81MPWs\nfp4DauRr1w/ak5OmjIOnn7bKY1RWwo9+FPf6TkoRmpbCThXpHEjd3kckya5YJozsw7jnlhCq36VE\n1YzUfPyGud4PRFZX3eqwTckhmlO6wOnY6BDOSMIDbfg4q05jYtf0ws/M00kJOb2PSPyubLwUX7xy\nDmX9S60yGU8/DWeeafVtmDTJ6t3Q2rnFpV85GyJKYecDXvdbwPeKxSuAQs1IqcWvghATUZPDGNMg\nItquNIdpzg/GrePXkwu/cNw//MMPH7d+xQdNfvBAymP3I/FSQm1axdYHCuN3ZZOIso0evP7Ybw9G\n3PsnePllOOIIePhhq3dDAqRzZZZJ3N6HU0c4N/x2ilNSg98opv+KyG9FJGD/XQX8N52CKc0jcqAq\nbhdg6446xkxf7Ltsc1n/UhaUD+OzipNZUD6Mm8r6uoYFRg5UZf1L6bX3Ho3HAc3O5vaaebqVfA4P\nJF7JUEMP6uL6WiR+y0xHRn9hGvhJ1Qscc+pPqHv9datPw4IFCSsHSKwPQi6TiveRysgnJT5+VwGX\nAHcDf8CyNMzFLpin5B7Rs6xI511zYs0njOyTsNmquZFN4egdJwet18zTj717/iffxL1+GD+z0/A1\ne2yooeLVezjyy49YsN+h/Pmig1yBAAAgAElEQVTMcTw75le+r+V07fD5YXNMVnO+kAoz0O6a/Jmr\n+M2k/ho4O82yKCki3uCYbOhpMj/wZH/QldU1TJyxzHMFsHVHnWv5ET8DRqoHlfUbtvCbf7/I2Lef\nZGdhgGtG/JZnDj0BcQlPTYSwgqqqqmpW4btsFG+MpLlmoN3F3JYvxMuDuMYYc5uI3IODj9IY89u0\nSaYkTXMHR69BJNEfeDI/aL/hkLXBkOtqyE8kVkoHlQ8/5KWnruHgmhW81vNI/nDCpXy9x/dTf51m\nkGzmcraVSiTp6BuhuBPPB7Hc/v8+sMjhT8lB/IayOpHqCrDJ2J39mIfCuNmfxw3vRaDAe+Y+bngv\nKqtrWPHV5oTaajZhxw6r5eeAAfxg27eMGTWe0T+/vlE5RL/XZNp4epHI+ZKx3+daReBE2o4qzSde\nHsRL9v/HMiOOkgrihXeGBy2nmWGqs6FTaZZKeH8P/RB2uI9/YSmXHdSAoSBx/8zChXDhhfDxx7za\n73jGH3sBpvP36SRQuy0U815TWXsomfO53aea2qBrrapcrAis0UqZI56J6SXcw98xxpyScomUZhM9\nKBe3C2CMlUXt1T84FTkDbvKkwizlRnFEdFW8hDqwFOSEkX2SH/y2brWa+Nx1F9v22psxZ09i9n4D\nwiegKFDI1LP6Uda/tHGGv7Y26OhsD4bqmfTSMsfrOSnw4ojXE5Xf675Grg5g13eouU7hRMqT+zFl\n5ZK5qyUQz0l9u/1/FLA38KT9/BxgdZpkUlJAvEF5cMU8x8Gl0CViKFV2dD8/8HHDe3H19MUuZ4hl\ny/a6RpOHH99F2CThdg1P5TR3rlVU77PP4LLLOLXLCFZub2qpdeuv4HRfwYoyi3a2u60OJh+9y1yX\n6OAdb2UZKXtYluY4hRMpT+5nNZTqFZgSH08fhDHmDWPMG0B/Y8xZxpiX7L9fAMdkRkQlHXjVKfLj\nM0jGlu7Xnl3WvzShUsyhBqtXhh/fRamd/QxWqQonHLfX1sJFF8Hxx1vtP994A+69l1XbnX9Cbv0V\n3Ij2A7itDtZv2t74PNHqq9H2ezcivxvNyV2YMntFk77h4O7z8OMf0RyIzOM3Ua6LiPwg/ERE9gf8\nZRkpOYnbIBJ2+nk5AZN1XCbyA58wsk9Cpba9SmyHEVvWsEJzm9HXG0O/Sa/tej8vvgi9e8Ojj8K1\n18KSJXDssYD3IJ2IWS56X7djd9Y3ND5OZvCOTIAs9aFgmuMUTmSF42dfzYHIPH4T5cYAVSISzp7u\nAVycFomUjOAVLhjtwwgP4PHKekfb0qPNSYkU2ouWwdURZuNVYhua1pIKKzS3CqBg2c4rHq1i4LXT\n6TbnJTjsMKsK64ABTfbzuo/xfCFO8ofpWBRwzQEZXDEvJfWH/IaMJusUtt7TZsft0d8NP82tNAci\n8/hNlHtVRHoCB9mbPjHG7EifWEq68RpcnGy9Y6Yv5v3PN3BTWV/XGVvYll6Ms73YreCfl0kksreD\n22Ab7v28cVso5hpFgULatCpw7O7WplWBPQOPaqxkDD9fNp8b5v6N9qHtcNNNVu+GQKzZK9H+CoFC\nob7e0BBxjkBBbK8Pr9y6mtog457b1dMi0kY/ZfYKxkxf7EtZJKpgvPxHTq+NG96LmuVNo+GLAoUM\nPahLzHcjUCAECqVJNdZoZZXuHIjm9vveHfHbcrQd8DtgP2PMb0Skp4j0Msa8nF7xlFTjx0ns1pv4\nHwu/YOB+nT1XA2OfWcLVh4T48+wlMSacyJ7NYSJ/4F6yuTlYiwIF1DWYxtln5DXCJSnGuDijNwVD\nTD2rH19+/H7jtn2++5qbZ9/L0P8uYtE+B3HNiVcx9/pLHI8Pk0h/haEHdWH6v7+kIWIgdHII1Lqs\nbMKE6k2TFVuyDly/qwOv84NzRNzkUX0p7VREaXFh3FDqUIOhuChA+zatXL+b6azYmop+37sjfk1M\nj2Alxh1lP18DPAu4KggR+TvwM+BrY8wh9raJwG+AcBGc64wxs+zXxgMXAvXAb40xsxN6J0pc/A4i\nXg2Dpsxe4RllFFYKbvb98MDtZ9XiVKXTKW8jWllFKocps1e4mqfC5br/uuIDxDTwy+pXKH/jUQpM\nAxOPG83jh5+MKSh0Lefhh+gBeHDFvCazZLAG++jQVD+hvpEmmXTnK8TzH7m9Nu6w2HN5NbNaPOGn\nnnKkKwciFf2+d0f8KogDjDFnicg5AMaYoMQvMPMo8Bfg8ajtU40xt0duEJHeWLWe+gD7AK+LyA+N\nMf5CQBRf+B1E3OzBYP24y/qXxq2T5IZbgT0/sjkNDm6rg3h5HZErlx9s/pZnnr6DH325jDd79Oe6\nEVewpmNJE9niDRJ+4/P9Olr9hKQmc95kSeb8NbVBajbWx8zK/fgbMs3a2iB0d9negvEbxbRTRIqw\nrQMicgDg6YMwxrwJ+G1LeiowzRizwxjzGbAKGOTzWMUnfn/kLpN/6zWgR/lMRIhbyiIaL3txsgOc\n26BSKOI6uDZG4vQtgVtv5bjLL+aw2jX8/qSrOffMPzVRDn5k8BvVlUhf6bL+pZw2wFspiX1Op+PD\nFIikpKyHV7SW12fgFOZqDDlXvjzRkOGWghiv0SC8k8gJWKW+ewOvAYOBXxtjquIc1wN4OcrE9Gvg\nO6z6TmONMRtF5C/AQmPMk/Z+DwOvGGOeczjnaOxS4yUlJQOmTZvm423GsmXLFjp06JDUsU7UBkOs\n37SdnfUNtC4soKRjW4qL/MfyxyMV8q74anOTMMkwrQsL6LX3Ho3Pl9Zs8nU+EUEgZhAoKYL1UWNq\nYYFY0Sou98SvbNHUBkPUbAw2kaHAYWCKpHvndnSr+ZwDb72V4lWrWHPEkcw/7xK2FHeiriH2uHgy\n+JHdSc5IWhUIXaPuj9N5o+9tgQilnewILo/zR+6bzPfS7T67XTv8GTh9F8D6DNL5e0mU2mCIUHAb\nXznc22zK5UVzxoShQ4cuMsYMjLdfXBOTbUr6BCub+kisictVxphvk5DrfuBGrInojcAdwAU4V81x\n/KYbYx4EHgQYOHCgGTJkSBJiWD2ekz02msrqGsbPXUowVEB4UVYUqGfyqN4ps1+mQt5ahyqpRYFC\nJo/qyxDbD2DZ9P03C3TKvB7bt447ljY9h3Wd3gxxuR/xZPPCraaUkw2/Td1Orlv0KKe8NZ3/tf0e\nl5SNp8/pR3DH0lYEvhQQYiJp4slwfvlMjMNiXIDPKoYA4Sgs77yO6O+M03md7m1pcSELyoc1uQ9u\nPTQKJUSD2ZmUgzfRKCbLrr/ZQd4iFvzSXwe5TFL5yhymfVqYN1FMqRzD3Ig7EhhjjIhUGmMGADOb\nczFjzPrwYxH5G7uc3GtoagHsBqxtzrUySbYLmvm1fycS2uoXN2d0NPHuR3MiVNwcl9GO9MPXLOe2\nV+7iwA1rmHX4Txl/zPlsKtqDPnaYq59IGif8xOf7sWUHQ/VcPX1xYyCA35pU4X0i78P+5c4/1fDn\nlUyUjpeD2O01pzDXXC3NXVwUYEH5kGyLkVP4nSouFJEfGWP+3ZyLiUhXY8w6++nPgY/sxzOAp0Tk\nTiwndU/gveZcK5NkM8Mz0fBGtx9yImUhInGr3eSE0/2orK5h0kvLGp2WxUWBxkJ3zaGsf2njedvt\nDDLuzcc5b9HLrP3enow5v4LKvQ5xXKL6iaSJxk98vpfjP5rwZ3jagFKeX1QT93NxKg3iR7mkexJT\n1r+Uyq8+jglzzeVZudIUvwpiKHCJiKwGtmKHmhtjDnU7QESeBoYAe4rIGmACMERE+mGZj1ZjZ2Mb\nY5aJyDPAx1hZS5fnUwRTNjM8U7V68VJmRYFC10GqTSshVG/NvuMRfT8qq2sY99ySJiad2mCIcc/u\nSgJrDhNG9uGlKY8yaebd7PPdNzx++Mncc9z5/PGcI3jPxQSVzGfmZ/XjU4c2EgzVM/+Tb5g8qm/c\nbHInBe03Cirdkxidlec3fhXEiYme2BhzjsPmhz32vxm4OdHr5ALpzPAMm4/O7r6Z66NKLEDqVi9u\nSi6cTxA5y49kW6iBQKFQXBRoLCf+/fY7KApI3PsxZfaKmJwA2FV8r1kKYuNGyu7+A2VPP8rnXbpz\n1i8rWHvIQP7okeXcnM8sXnz+piRCgsMhxZHZ5E6lK5xqKkUrLTefhJPS1nLaSph4/SDaApcABwJL\ngYeNMXVex7RE0pXh2cR81N3ZfJSq1Uu82kxTZq9wNZGE6g3t27RqNM1UVVUxeVTPuPfDS4k1a2b7\nwgtw+eXwzTcwfjz73XADz7Zt22SXyM8MNjcqwnQNhon2uAgfE4lb6Qo3pRZdhiOeQtRy2ko08VYQ\njwEh4C2sVURv4Kp0C5WPpCPD04/5aOhBXfjHwi9cy1f4JZ6SizdgR7/u5354DZpJmee++gquuAKe\nfx769YNZs6B/f9fdwzJWVVVx5S+HJH49/M+43RTw5FF9AX+rmebY9P1MYrIdbKHkHvEURG9jTF9o\nzE3IG8fx7kA881FldQ3PL6ppohwEOG2A9+DsNqiF/yKLvvmNqElmQB83vFeMDwKci9d5Ygw89hj8\n7newbRtMngxjxzoW10slicy4/Q7Q8Qb+5tj04yltLaetRBNPQTTaFIwxdfGrayipJJ75yK2o3vxP\nvok5Jky8Qc3tda+IGreGQvEGvPDz6Cimiaf08V3aQj5fzR1z7+eIle/DMcfAQw9Br8yEUSY6404m\nTLQ5JOpP0HLaSjTxFMRhIvKd/ViAIvt5OIrpe2mVroUTz/mdzIwv3qDm9npkRE1NbbAxvNXJdl8b\nDNmJg/5m1okOjJXVNVz3/BLOeHcG17zxGEaEG0dcRt+byinr5VBQx+M8XgEA8fCzwsuWwzcZf0K6\ny2kr+YengjDG+G/ppSSNl8kH3B2pycz44g1qXq/7HczXb9puZ5XvIpW27OlPzOGxZ6bwo5qPeWP/\nw7lu+BXUdNyL0jkrKRvgT0H4CQCI3Nfp8/G6/9l2+CbjT0hXsIWSv/ivqaCkBT9lrt0cqcnM+OIp\nlVSYGaz6QbGlJ5ptyw6FYMoUHr17AsFAW3538hhe6DOssbtOTW2Q/ctn+hrY/A6gXp+PV4BAMgN0\nKlccyfoT0lVOW8lPVEFkmeZEjpT1L+X9zzfw9LtfUm8MhSIxDuroQWfoQV1ifAmRSmXc8F6Me3ZJ\nk8S3eE7j6Gv8aj9/FUsT4oMP4MILYfFiFhxyLNcM+Q3ftu8Us1tkNVXA9V64JZ1FD6Be7VW3hxpc\nAwTc+mW4OfpTveJQf4KSCvyW+1bSRHMiR8JRTJFNev6x8Av+ULm08fXoMtTPL6rhtAGljU3oO7UL\n0KZVAWOmL2ZwxTze/3wD0XVJY+uUNpUh+hr1DVZrzUiStmUHgzB+PAwaxPY1axn/q0lccPI1/M9B\nOTQ5LKKZjZOcbkQPoF7tVb0CBJzKX3htj9eQJ1HGDe+VcyW1lfxDVxBZxs9Mr7K6hvVfbeb8KPOJ\nWxTTkwu/4MmFXzjWSQo7nMPVP6NnrU8u/CJGlvqGXe0to1cLW3fUOchgaN+6lWvRO9+mlLfftlYN\nn37K56eexZkHnsb6Vu0a36dbj+swkYO7n1pTTgNox6JAQo2Rwtd0q0/ltj3VIabqT1BSgSqILBPP\njxAexC87qAFDATW1Qa6evthXR7d4g1EiBfo2bgvRb9JrbN1Z15i34JUX4Vb0zpcpZfNma9Vw773Q\nowfMmcMv3i9gfXRjI3aVmWhuNdVwg6HwjD2sDLfujC0cECgQ2rdp5Xj/ww5qN+XlVBYjfFyqTULq\nT1Cai5qYskxZ/1Krubtt8mnsdhYxA3QaxJNp9xnGAH+oXJrw7LQ2GHKsneSE28AW15TyyivQpw/c\ndx9cdRUsXQrHH+85w/ZjTnGTp1O7QJM6RZHd4NxqRXVo24qJp/RxvaZbH2wBVxNPvPdQWV3D4Ip5\nLK3Z1OzucIriF11B5ABeM710ZbE+ufAL2rcuZOvO1BfNLRB3p7bb+9m2bj2cey488QQcfDAsWABH\nHdX4utcM2485xW2lZkxsR7ywwnKTtXZbyPOabn2yvVSr1/kSCclVlFSiCiLHSabIm1+27az3LOWd\nDGHHt9vAFfN+jOGkFQu4ae4DsH0L/OEP1l+bNk2Oi2eKi2dOcRuA3QbztbVB1x4Oxe0Cntf0+sxS\n1atDayQpmUAVRI4THhjBvYhuoFB8m34iMVhhmdGx/IFCoX1rZxu7n3PWBkMMrpjXJLR2/iffNA66\nBViRUV22bOCm1+5j+MqFbDz4UHj6cTjsMMfzpsLp6jQAh5MQowk74B3fY5xb7dWLIZW9OrRGkpJu\nVEHkOOGBZM3H7zu+3qldgAkj+zD2mSW+O7uFKRRh/iffxJg+wuW7J57SJ7k2pA2mcQYdHRm1cVsI\njOGMpXP447yHaV0f4tZhF3BQxQ2ceth+nudNh9PVq4S22+oiXm+HsIxuuRCp6tWhOQ1KulEndR5Q\n1r+Ubp3bxeQWBAqFCSOtwnbRdnQ/nHNE97ilNSaP6usau1+QRO3GbrVf8cT0PzLllbtZvtf+nHj+\nPdz/o1HcNvc/vs8RdtjuXz4zrsM23r5l/Usp7VTkGCTgNgD7GZjL+pd6RiwlguY0KNlCVxB5QnFR\ngCmn93Y1sbjNMouLArRv06rJa4UinHNEd24q68v8T77xnJ2Gz+9UlhsDhQVCvY92owUN9Zz3wcuM\ne/Nx6qWA6396GU/1G4ERa47id1adSMax333dSmj7KWXildORquJ3mW5upChhVEHkEV4mFje7twie\ng4mfQaysf6lj3kUDID6Uw4HffsFtr9zF4WtXMP8HA7hu+BWs+16XJvv4nVUn4rBtrnM3nt/DTx0t\nr+MTIRXNjRQlUdKmIETk78DPgK+NMYfY2zoD04EewGrgTGPMRrEaTdwFnARsA35tjPkgXbJlilSX\ne/Y6X/h/9EC+cVsobuQM0KSMd3TCGLjb3b3UQ6A+xCULn+OKf01na+t2XPWzsbzYe0hjcb0wYYXk\n534l4rBNhXPXSyn7UUCarKbkM+lcQTwK/AV4PGJbOTDXGFMhIuX282ux2pn2tP+OAO63/+ctqS6+\n5rfHwubtsZE3kQO+V1lxL3ndTFhO5TwA+q5byW2v3MXB36xmxsHHMum40fyvfbHje3Nquxl5/Ui5\nC1yuF16BRCqZePs2F40uUnZ30qYgjDFvikiPqM2nAkPsx48BVVgK4lTgcWOMARaKSLGIdDXGrEuX\nfOkmFbHrkYPd2L71BENNHZWR5wsrJLdIpvCA66YA3OQd+8wSwNkUJVjlPCLLSrQJ7eDop5/gslkz\n+KZ9MReN+iOv93TX9aV2otvginmO1584Yxk76hoaX3N6f5ErkEgZvfZNBRpdpOzuiEki+sX3yS0F\n8XKEianWGFMc8fpGY0wnEXkZqDDGvG1vnwtca4yJie0UkdHAaICSkpIB06ZNS0q2LVu20KFDh6SO\n9cPSmk2ur/Ut7eh5bG0wxLraIHUR9v2SIljvMjHtW9qRFV9ttvswOCMIxsEg1LqwgF577+Epb4EI\npZ2sQW/9pu2u19ln+Ucc99C9FK9fx0dDT2DB2eexs733Pe7euR3FRQHP63vRqkDoWlxEcVHA9R6E\n33vrwgJKOraluKhpr2qn70JtMNT4Xt2Oqw2GqNkYbBJBFr5X0fuminR/b1NNPsmbT7JC8+QdOnTo\nImPMwHj75YqT2ilg0lFzGWMeBB4EGDhwoBkyZEhSF6yqqiLZY/1wfcU8x9llaXGRp5OxsrrGNiU1\nXS2M7VvHHUtjP67w+c4vn4lxiVr2ypYW4LOKIa7y7rpOIQvKhwEwOGrfDju2UV71CKctfoUvivdm\n/vg/MabhcPivdf6jD+jMB19sill9/PLIfbnyRMu8FO/6bhQFCpk8qjdD+pe63gPrPZ7seo7o78Ku\nz6CAcCR4UaCeyaN6++42ly7S/b1NNfkkbz7JCpmRN9N5EOtFpCuA/f9re/saILJXZDdgbYZlSynJ\nxq4nUmE18nxuZo0CgTat3D/m8HFO8kZSUxtszCOItLEP/c+/ee3hyzhnyWz+9qMyhp//F9b0ObTx\ndQMsW7s5piDh1LP6MXC/zo05Ctt21hEoiHVed2rnPROP9K80J28hkkR6M5T1L2VB+TA+qziZBeXD\n1CGt7FZkWkHMAM6zH58HvBix/VyxOBLYlM/+B4hfpdUNP2Wpnc7nNMAHCoVCEc+SGVt31FFZXRM3\nKQ52+S2K2wXotG0TU1+6nUeem8TmNu047VdTuHnYRQRbt405Lnz9yIEUaNLAZ+O2EIiVkxD5/iaM\njK2aGk34nqUqoUydz4pikc4w16exHNJ7isgaYAJQATwjIhcCXwBn2LvPwgpxXYUV5np+uuTKJMmE\nOHoVerPMKbFKJmzmCIbqG6OKSu1aQvHqKdUGY8NgvcprBHfWcfySN5n4+l/ZY8c2/jz4HO496kxa\ntW2LhOpdw16jnfNOs/RwIp6TOSgchutEdFJfc00+6nxWFIt0RjGd4/LScQ77GuDydMmS60TasTsW\nBVyL70X3mw4fGx25I3g384kmMhoqOi8ikpLN33LTa/dzwqp3Wdy1J9eeeBUruvSgU7sAJx/a1bEb\nXZjo2bdrKe1gqHFFEyYsV/R7BeekvuaaeVKVAa0o+U6uOKlbLNGDXm0wZHUsc+jV8PyiGgbu1znu\nTDyZuLTIATs8yDY6o43h7CWzuW7+3wk01HPT0Av4+8BTaSiwzDnGWLJ5ET379lopuYUCZ6qNZjqu\nk2lntqKkAlUQWSBeMleowdAQig3XdMqjSJVd3Ml8Mm54L+57aDYTX76Lo7/4kH/t25fyEVfyead9\nmuwXz4wVKIhtIDRueK+kqp1mKjM5lddJRdJk+DtzdvfNXF8xTxWMkhG0mmuGCQ8WYedsos3ta2qD\nTSqTpsouvm1nXdNKp/X1lM2fzqy/X86h6//DdcOv4Bdn3xyjHOJRXBRgyhmHxQxmZf1LXSOUdjdb\nfyJRUU5EfmegaVtURUknqiAyjN8wVq9oIoM1SIyZvph2rQsck0gSJVyzqbK6Bj76CI4+GsaOpdXx\nx9Nh1Sfc8uo9jZVX/VJaXMTiCT91nek6RSjtjrb+5kZFNVfBKEqyqIkpw/gZFIoChRy+b0fe+c8G\nT3+CAVZ+vdXzXAK089l7um77dr4ddz28+RR07AhPPw1nndVYXK80gfanfgb6ZGz9+WjLb25UlIbd\nKtlCFUSG8Sp612BMY4vO5xfVJOVsjsYAO+sa4rYlPWztCm595W4O+vZzXjvsOOrunMpJw/o2vl5Z\nXcO2ne5tTyMRxFfOByRm6091AcRM0dyoKA27VbKFmpgyjFsy1x1nHtaYRDb/k28SbvPpRajB0L51\nqyZJe0UB66NvG9rO9fMe4oUnx9Fx+xYuOO0GRo8Yw9j5NY027vDAvHFbU2d0u0CBY/Zzt85FaRmw\n89XUkmzSZBjtKKdkC11BZBg/ZpV0mA42BUMsnvDTxuf9//Qa/VYtpuLVu9mv9iv+0W8EFUPOZ3Ob\n9kDTiCk3v0mn9m0YN7xXzHsp3rQy5fKDP1NLrpqgmhMVpR3llGyhCiILxBssvHIEkqWJOWLTJsa9\ncCe/WDKb1cVdOfucW1i476Exx4QH3nh9q6PfS1VVehSE233pWBRozNmILD2eLyYoP2hHOSUbqIkp\nS1RW1zQWqguHrIaJVzgvHtFRTU3MES+9RLBnL876cA4PDBrFiAvucVQOYA20gyvmUZwj4aiO9aYK\nhK076xoVR7SXJVkTlNfnoygtBV1BZIHK6hrGPbuEkN3voaY2yLhnrcY8br2Mu3euZ3XFCU1MKG4u\nZ4Nl525iZunWmjUjyug2+0VWd+nBNf93B0u79owra01tkECBxDi5s2EDd7ov23bWxfhGoglXok0k\nUiofneGKkmpUQWSBiTOWNSqHMKEGw8QZy1x7GVe+MqfJIDf1rH6uBexKi4saK6ZiDDz1FDuGXcle\n323mjmN+yQNHnk6o0H9Dm1CDobgoQPs2rbJu24++L/uXz4x7TGRtKj+DfSq6ASrK7oAqiCzgVprC\nbXtldQ1rNgSpqbXMK+EVx1mDuvP8ohr38Mkvv4RLL4WZM1nV/WCuHnUTK7vs5ypXONTWaWUS7eSO\nlC3GSe16hdQTz18T6ZMIE2+w17wDRbFQH0QeMHHGsph2oaEGw8tL1jmHTx7WFR54APr0gfnzYepU\nTjm7wlM5gFXew82v0NGhhWZ02ZDw7DxebaZU4uSXCPtgSouLXM1wXoN9qhoPKUq+owoiC7jVIHLb\n7rXiiOlo1mEbDBtmrRwGDYKlS+Hqq9m7c/zeteHwyejcBoCt0bWacDfFfLlhW8Ycu045BlPP6sdq\n+36UJjHYa96BoliogsgCE0b2IVDYdBAOFAoTRvZJ/qR1dTBlChx6KCxeDA8/DHPmwA9+AMSPjAoP\ngGX9S+nQNtbyGKo3MdFAXrPwTBaU82r7mcxg39zENkXZXVAfRBZItAaRtbKILXPRuOJYsgQuvBAW\nLYJTT4X77oN9dlVddeo416ldAGMs30L09WtdooKiFUI8+38uOHaT7e2QqbLiipLLqILIEokMQBNG\n9mHN8kVNtgUKhUnDD4Q//hEqKqBzZ3jmGTj9dCoXr2XK4/MaO9Rt3VnXGKJabwxFgUImjOzjen2/\ntX+cagxFkwuOXR3sFSU51MSUB5T1L6Vbp6ImJo+HetZxyvk/g5tugnPOgY8/hjPOoHLx2iaO49pg\nKKZIX2TymFNCmF+zTKQpxg117CpK/pKVFYSIrAY2A/VAnTFmoIh0BqYDPYDVwJnGmI3ZkC/bOIaO\nFgVYUD4Etm6F66+H6+6Gbt1g1iw48cTGY/32m1hbG3RNCJs8qi+TR/X1ZZZJpF+0oij5RTZNTEON\nMd9GPC8H5hpjKkSk3H5+bXZEyx6ug/bRhfD66/Cb38Dq1XDZZTB5Mnzve02O92vS2ae4yDMhLNrZ\nGw8tKKcoux+5ZGI6FemxwX8AAAzRSURBVHjMfvwYUJZFWbKG06Ad2LyJg267DU44AQIBePNNuPfe\nGOUA/kw64Zl9qhPCwtFEfUs7JqxgFEXJPbKlIAzwmogsEpHR9rYSY8w6APv/XlmSLatED84//fRf\nzHn4Mn745jwoL7ciln78Y9fj3QradWoXiAnZzJWEMC2Mpyi5iRiTir5lCV5UZB9jzFoR2QuYA1wJ\nzDDGFEfss9EY08nh2NHAaICSkpIB06ZNS0qGLVu20KFD/OSxTLPiq83srG+gaFMtP3nsQXq+9w7f\n7NuDJVdeQauBA3ydozYYYv2m7eysb6B1YQElHdtS7JAJXRsMUbMxSEPEd6BAhNJORY77+yV8b/3I\nkS4ZkpE3H8gnWSG/5M0nWaF58g4dOnSRMWZgvP2y4oMwxqy1/38tIv8EBgHrRaSrMWadiHQFvnY5\n9kHgQYCBAweaIUOGJCVDVVUVyR6bTmo/WMPCSX+m/LW/UhTazm3HnssTg8/gxj5tKEuDvOlosFNV\nVUVtx56Mn7uUYKiA8EK1KFDP5FG9m5zf6uMQm8BXWlxoOeUzQK5+F5zIJ1khv+TNJ1khM/JmXEGI\nSHugwBiz2X78U+BPwAzgPKDC/v9ipmXLOl98Qdn1F1P26qss3bc3Y064kuCBP+TGiC5tqR7Q05Uj\n4LciqhbGU5TcJRsriBLgnyISvv5TxphXReTfwDMiciHwBXBGFmTLDg0NcP/9UF5OXX0D94y8gr8c\ndAJ7d+7QqACqqlbmVZ8CvwO/36Q8RVEyT8YVhDHmv8BhDtv/BxyXaXmyzooVcNFF8PbbfH3ksZwz\n8AL+035PoKkCKCa/+hQ0Jxtb8ycUJTfIpTDXlkUoZJXIOOww+OgjeOQRfn7KDY3KIUxk1nM+mWOS\nycbWwniKkltoLaZsUF1tFderroZRo6ychr33Zq1LdzRLAbTPK3NMIkXytFaSouQmqiAyyfbtcOON\ncOutsOee8NxzcNppjS/HUwD5Zo7RgV9R8hs1MWWKBQugXz+45Rb41a+s4noRygHim2XUHKMoSibR\nFUS62bIFrrsO/vIX2HdfePVVGD7ccVcvs0xV1crGfVQhKIqSCVRBpJPZs+Hii+GLL+CKK6zVQ5zM\nR1UAiqLkCmpiSgcbNsCvfw0jRkDbtvDWW3D33XGVg6IoSi6hCiLVPP889O4NTz5pmZYWL4bBg7Mt\nlaIoSsKoiSlVrFtnmZFeeAH697d8Df36ZVsqRVGUpNEVRHMxBh591Fo1zJxpJb+9+64qB0VR8h5d\nQTSH1ath9GiYMweOOQYeegh65WZOgqIoSqLoCiIZ6uvhnnvgkEPgX/+yMqHfeEOVg6IouxW6gkiU\n5cut4nrvvGNFKT3wAOy3X7alUhRFSTm6gvBLKAQ332z5Fj75BB5/HGbNUuWgKMpui64g/PDBB3DB\nBVY/6DPOsMxLJSXZlkpRFCWt6ArCi2AQysth0CBYv94KYX3mGVUOiqK0CHQF4cZbb1m+hk8/tUpz\nT5kCnTplWypFUZSMoSuIaL77Di6/HI49FnbutEJYH3pIlYOiKC0OVRCRvPKKFbp6//1w9dVWp7fj\nj8+2VIqiKFkh5xSEiIwQkRUiskpEyjNy0f/9D849F046ySqot2ABTJ0K7dtn5PKKoii5SE4pCBEp\nBO4FTgR6A+eISO+0XdAYy+l88MHw9NPwxz9abUCPOiptl1QURckXcs1JPQhYZYz5L4CITANOBT5O\n9YVaf/ut1Q+6shIGDIDXX4dDD031ZRRFUfKWXFMQpcCXEc/XAEek/CqzZjHo17+2SmbcdhuMGQOt\ncu1WKIqiZBcxxmRbhkZE5AxguDHmIvv5/wGDjDFXRuwzGhgNUFJSMmDatGkJX6eopob977yTz8aM\nIditW2qETzNbtmyhQ540HMonWSG/5M0nWSG/5M0nWaF58g4dOnSRMWZg3B2NMTnzBxwFzI54Ph4Y\n77b/gAEDTLLMnz8/6WOzQT7Jm0+yGpNf8uaTrMbkl7z5JKsxzZMXeN/4GJNzykkN/BvoKSL7i0hr\n4GxgRpZlUhRFaZHklOHdGFMnIlcAs4FC4O/GmGVZFktRFKVFklMKAsAYMwuYlW05FEVRWjq5ZmJS\nFEVRcgRVEIqiKIojqiAURVEUR1RBKIqiKI6oglAURVEcyalM6kQRkW+Az5M8fE/g2xSKk27ySd58\nkhXyS958khXyS958khWaJ+9+xpgu8XbKawXRHETkfeMn1TxHyCd580lWyC9580lWyC9580lWyIy8\namJSFEVRHFEFoSiKojjSkhXEg9kWIEHySd58khXyS958khXyS958khUyIG+L9UEoiqIo3rTkFYSi\nKIriQYtUECIyQkRWiMgqESnPtjzRiMhqEVkqIotF5H17W2cRmSMiK+3/nbIo399F5GsR+Shim6N8\nYnG3fa8/FJHDc0DWiSJSY9/fxSJyUsRr421ZV4jI8EzKal+/u4jMF5HlIrJMRK6yt+fc/fWQNSfv\nr4i0FZH3RGSJLe8ke/v+IvKufW+n260GEJE29vNV9us9ckDWR0Xks4h728/enp7vgZ+mEbvTH1YZ\n8f8APwBaA0uA3tmWK0rG1cCeUdtuA8rtx+XArVmU71jgcOCjePIBJwGvAAIcCbybA7JOBH7vsG9v\n+/vQBtjf/p4UZljersDh9uM9gE9tuXLu/nrImpP3175HHezHAeBd+549A5xtb38AuNR+fBnwgP34\nbGB6Dsj6KHC6w/5p+R60xBXEIGCVMea/xpidwDTg1CzL5IdTgcfsx48BZdkSxBjzJrAharObfKcC\njxuLhUCxiHTNjKSusrpxKjDNGLPDGPMZsArr+5IxjDHrjDEf2I83A8uxerXn3P31kNWNrN5f+x5t\nsZ8G7D8DDAOes7dH39vwPX8OOE5EJMuyupGW70FLVBClwJcRz9fg/aXOBgZ4TUQW2T24AUqMMevA\n+mECe2VNOmfc5MvV+32FvRT/e4S5LqdktU0a/bFmjzl9f6NkhRy9vyJSKCKLga+BOVirmFpjTJ2D\nTI3y2q9vAr6fLVmNMeF7e7N9b6eKSJtoWW1Scm9booJwmgHkWijXYGPM4cCJwOUicmy2BWoGuXi/\n7wcOAPoB64A77O05I6uIdACeB642xnzntavDtozK7CBrzt5fY0y9MaYf0A1r9XKwh0xZlTdaVhE5\nBBgPHAT8COgMXGvvnhZZW6KCWAN0j3jeDVibJVkcMcastf9/DfwT64u8PrxktP9/nT0JHXGTL+fu\ntzFmvf3jawD+xi4zR07IKiIBrAH3H8aYF+zNOXl/nWTN9fsLYIypBaqw7PXFIhLurhkpU6O89usd\n8W+uTBkRso6wzXrGGLMDeIQ039uWqCD+DfS0IxdaYzmfZmRZpkZEpL2I7BF+DPwU+AhLxvPs3c4D\nXsyOhK64yTcDONeOsjgS2BQ2lWSLKNvsz7HuL1iynm1Hr+wP9ATey7BsAjwMLDfG3BnxUs7dXzdZ\nc/X+ikgXESm2HxcBx2P5TeYDp9u7Rd/b8D0/HZhnbI9wlmT9JGKSIFi+ksh7m/rvQaa88rn0h+Xx\n/xTL/nh9tuWJku0HWJEeS4BlYfmwbJ9zgZX2/85ZlPFpLNNBCGvmcqGbfFhL33vte70UGJgDsj5h\ny/Kh/cPqGrH/9basK4ATs3Bvj8EyDXwILLb/TsrF++sha07eX+BQoNqW6yPgBnv7D7AU1SrgWaCN\nvb2t/XyV/foPckDWefa9/Qh4kl2RTmn5HmgmtaIoiuJISzQxKYqiKD5QBaEoiqI4ogpCURRFcUQV\nhKIoiuKIKghFURTFEVUQihIHEfm5iBgROSjOfr8WkX2acZ0hIvJysscrSqpRBaEo8TkHeBsrqdKL\nXwNJKwhFyTVUQSiKB3adocFYCXZnR2y/RqyeHUtEpEJETgcGAv+w6/QXidXXY097/4EiUmU/HiQi\n74hItf2/V+bfmaLEp1X8XRSlRVMGvGqM+VRENtiNWErs7UcYY7aJSGdjzAYRuQKrD0K4yZPbOT8B\njjXG1InI8cAtwGnpfyuKkhiqIBTFm3OAP9uPp9nPC4BHjDHbAIwxiRZw6wg8JiI9sUpVBFIkq6Kk\nFFUQiuKCiHwfq5nMISJisLoRGqzqpX5q1NSxy4zbNmL7jcB8Y8zP7T4KVSkSWVFSivogFMWd07G6\ndO1njOlhjOkOfIZV8vkCEWkHVr9oe//NWK03w6wGBtiPI01IHYEa+/Gv0yO6ojQfVRCK4s45WP04\nInkeK1JpBvC+3fHr9/ZrjwIPhJ3UwCTgLhF5C6iPOMdtwGQRWYC1KlGUnESruSqKoiiO6ApCURRF\ncUQVhKIoiuKIKghFURTFEVUQiqIoiiOqIBRFURRHVEEoiqIojqiCUBRFURxRBaEoiqI48v/juk6/\n4AHiTQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a122dbcc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(y_tr,y_tr_pred)\n",
    "plt.plot([0,350],[0,350],'r')\n",
    "plt.xlabel('Actual')\n",
    "plt.ylabel('Predicted')\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validate on Test Data\n",
    "\n",
    "As we will explain in the next lab, one should not evaluate the performance of a model on the data on which it is trained.  It is important to see how the model works on *new* data that is independent of the training data set.  For this reason, we will evaluate the model on the *test* samples that were not used in training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RSS per sample = 0.492801\n",
      "R^2 =            0.507199\n"
     ]
    }
   ],
   "source": [
    "X_test = X[ns_train:,:]\n",
    "y_test = y[ns_train:]\n",
    "y_test_pred = regr.predict(X_test)\n",
    "RSS_test = np.mean((y_test_pred-y_test)**2)/(np.std(y_test)**2)\n",
    "Rsq_test = 1-RSS_test\n",
    "print(\"RSS per sample = {0:f}\".format(RSS_test))\n",
    "print(\"R^2 =            {0:f}\".format(Rsq_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the model predicts new samples almost as well as it did the training samples.  We can also plot the fit on the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEKCAYAAAAcgp5RAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXl4FFXWh9+TGCAQJKgYMTCCGwo6\ngmRwQR2CCy6jMO4637ihqMCojKLBZYRxi6I4ooyI4r4AIxpREEQgLgioGHZBUFwICCoEiQQIyf3+\nqOrY6fS+VnXO+zx50l3rr253nzp17rnnijEGRVEUJX3JSLUARVEUJbGooVcURUlz1NAriqKkOWro\nFUVR0hw19IqiKGmOGnpFUZQ0Rw29oihKmqOGXlEUJc1RQ68oipLm7JFqAQD77LOP6dChQ1T7/vbb\nb7Ro0SK+ghKIm/S6SSu4S6+btIK79LpJK8Smd+HChT8bY9qE3NAYk/K/7t27m2iZM2dO1PumAjfp\ndZNWY9yl101ajXGXXjdpNSY2vcDnJgwbq6EbRVGUNEcNvaIoSpqjhl5RFCXNUUOvKIqS5qihVxRF\nSXNCGnoRaSYin4rIYhFZLiIj7OXPi8haEVlk/3W1l4uIjBaRNSKyRESOTvRFKIqiKIEJJ49+J9Db\nGFMpIlnAxyLyrr1uqDHmdZ/tzwAOsf+OAZ60/yuKoigpIKRHb6drVtpvs+y/YPMP9gVetPebD+SK\nSNvYpSqKoqQR1dVQXEzLlSsTfioxYcwZKyKZwELgYGCMMeY2EXkeOA7L458FFBljdorIO0CxMeZj\ne99ZwG3GmM99jjkAGACQl5fXfcKECVFdQGVlJTk5OVHtmwrcpNdNWsFdet2kFdyl1w1ac1avptPI\nkbRcvZo1553HusGDozpOYWHhQmNMQcgNwxlV5fkDcoE5wBFAW0CApsALwL/sbaYCJ3jtMwvoHuy4\nOjLWmbhJqzHu0usmrca4S6+jtVZVGXP77cZkZhqTl2fM6687b2SsMaYCKAVON8ZssM+1E3gO6GFv\ntg5o77VbO2B9JOdRFEVJO+bOha5d4f774bLL4Msv4bzzknLqcLJu2ohIrv06GzgFWOmJu4uIAP2A\nZfYuU4DL7OybY4GtxpgNCVGvKIridLZtg3/8A048EXbsgBkz4NlnoXXrpEkIJ+umLfCCHafPACYZ\nY94Rkdki0gYrfLMIuM7efhpwJrAG2A5cGX/ZiqIoLmDGDBgwAH74wTL2990HKeg/CGnojTFLgG5+\nlvcOsL0BBsUuTVEUxaVs3gz//Ce88AIcdhh89BH07JkyOToyVlEUJZ5MngydO8PLL8Mdd0BZWUqN\nPDhk4hFFURTXs2EDDB4Mb7wBRx8N06dbna8OQD16RVGUWDAGnnvO8uKnToXiYliwwDFGHtSjVxRF\niZ5vv7U6W2fOtLJqnnkGDj001aoaoB69oihKpNTUwOjRcMQRMG8ejBkDpaWONPKgHr2iKEpkfPkl\nXH01fPIJnH46PPUU/OEPqVYVFPXoFUVRwqG62sqD79oVVq6El16CadMcb+RBPXpFUZTQLFwI/fvD\n4sVw4YXw+OOw776pVhU26tEriqIEoqoKiorgmGNg0yZ4802YONFVRh7Uo1cURfHPhx9asfjVqy1v\n/uGHITc31aqiQj16RVEUb379FQYNgj//GXbvhvfft9ImXWrkQQ29oijK77z7rpUy+eSTcNNNsHQp\nnHxyqlXFjIZuFEVRfvkFhgyxMmk6d7ZSJ489NtWq4oZ69IqiNF6MgUmT4PDD4bXX4F//gi++SCsj\nD+rRK4rSWFm/HgYOhLfegoICKxb/xz+mWlVCUI9eUZTGhTEwfrwVopkxA0aOtMoYpKmRB/XoFUVp\nTHzzDVxzDcyebWXVPPMMHHxwqlUlHPXoFUVJf2pq4D//gSOPhM8+g7FjLWPfCIw8qEevKEq6s3y5\nNeBpwQI46yzLyLdrl2pVSSWkRy8izUTkUxFZLCLLRWSEvbyjiCwQkdUiMlFEmtjLm9rv19jrOyT2\nEhRFUfywaxf8+9/QrRt8/TW8+iq8/XajM/IQXuhmJ9DbGHMU0BU4XUSOBR4EHjXGHAJsAfrb2/cH\nthhjDgYetbdTFMWhlJSV07N4Nh2LptKzeDYlZeWplhQ7n31mZdLcfTecfz6sWAGXXAIiqVaWEkIa\nemNRab/Nsv8M0Bt43V7+AtDPft3Xfo+9/mSRRtq6iuJwSsrKGfbGUsorqjBAeUUVw95Y6l5jv307\nB44da+XBb94MU6ZYnnybNqlWllLEGBN6I5FMYCFwMDAGGAnMt712RKQ98K4x5ggRWQacboxZZ6/7\nGjjGGPOzzzEHAAMA8vLyuk+YMCGqC6isrCQnJyeqfVOBm/S6SSu4S69TtK76cRu7amobLG+SmUGn\n/VrWvXeK3mDkLlrEoQ8/TPPyctb/5S98fe211DhcM8TWtoWFhQuNMQWhtgurM9YYUwN0FZFc4E3g\ncH+b2f/9ee8N7ibGmHHAOICCggLTq1evcKQ0oLS0lGj3TQVu0usmreAuvU7RemXRVIyfB3sB1hb3\nqnsfi96SsnJGzljF+ooq9s/NZmifTvTrlh+dYH9s3Qq33WbN9HTQQSwaNYquQ4awf/zOkFCS8V2I\nKL3SGFMBlALHArki4rlRtAPW26/XAe0B7PWtgM3xEKsoSnzZPzc7ouWRkvDQ0DvvQJcu8PTTcMst\nsGQJFd26xefYaUQ4WTdtbE8eEckGTgG+BOYA59ubXQ68Zb+eYr/HXj/bhBMfUhQlboTbwTq0Tyey\nszLrLcvOymRon05x0TFyxiqqqmvqLauqrmHkjFWxHfinn+DSS+Hss6F1a2tk68iR0Lx5bMdNU8IJ\n3bQFXrDj9BnAJGPMOyKyApggIvcCZcB4e/vxwEsisgbLk784AboVRQmAx4v2GFiPFw00CJl43icq\ntLK+oiqi5SExBiZMgBtusEI2I0ZYM0A1aRKDyvQnpKE3xiwBGjwLGWO+AXr4Wb4DuCAu6hRFiZhg\nXrQ/A96vW358Y+Ze7J+bTbkfox5VaGjdOrj+eitc06OHVa/miCPioDL90RIIipJmxN2LjoG4hIZq\na2HcOCsWP2sWjBpl1YtXIx82WgJBUdKMuHrRMRJJaMhvdk7LKqsIWWkp9O5tGfyDDkryVbgfNfSK\nkmYM7dOpXowe4tvBGinhhIZ8+xV+3FzJV0Pvpubjl8ls2tTKqunfv9GObI0VNfSKkmYkuoM1EXj3\nK3T66VsefPcxum5YzUeHH8+JMydBvnO1uwE19IqShiSygzURrK+oosnuagbNm8TA+ZPY2iyHwefc\nytTDTmStGvmYUUOvKErSCDRK9pRfv+WW/z1Ep5+/540uhdzT+2q2NG9Ffgr6FdIRNfSKoiQFf/n9\n/57wGUc8MoJxrz7Djzl7c+X5dzPnoD8Bqe1XSDfU0CuKkhR88/uP+24xxdMf54CKH+H661l4yQ18\nNXc94pJ+BTehhl5RlKTgyePfc0clw+Y8yyVL3uOb1vtz0aXFTPzvbfwF+MuJh6VWZJqihl5RlKSw\nf242nT+bw73v/Zd9fqtg7DHn8WjPS9mnTW6qpaU9augVRUk8mzYxcc5jtHtvCl+26cDV597F0raH\naBw+SWgJBEVREocx8PLLcPjhtCudzoqBQ7n2xqdY1vYQWjfPoukeGQyZuCh9pjB0KOrRKykn4RNT\nNHJS1r4//ADXXQfTpllT+40fT+fOnfmQyCpsxpPG+l1TQ6+klFT94BsL4bRv3I1fba0129Ott1qv\nH3sMBg2CzN+Lm0VaYTMeNObvmoZulJSSsIkpFCB0+8Z9BqivvoJevWDgQMuLX7bMqh2fWb+Cpb+i\na5DYCpuN+bumhl5JKU4qqZuOhGrfuBm/3bvhoYfgqKNg6VJ49ll47z3o2LHBpiVl5X4nlobEVths\nzN81NfRKSkn0nKWNnVDtG8jIBfK4/bJ4MRxzjDVB9xlnwIoVcOWVAStNjpyxCn9ziwokJAPHM61i\noPlMG8N3TQ29klISPWdpYydU+wYycgKhwzc7d8Jdd0FBgTX70//+B5MnQ9u2QXcLdHMxxD9W7h2a\n8kdj+a6poVdSSr9u+Txw7pHk52YjQH5uNg+ce2Tad44li1DtO7RPJ79hFAPBwzeffALdusG998Lf\n/mZ58eefH1a9+EA3l0QUMPMXmvI+X2P5roXMuhGR9sCLwH5ALTDOGPOYiAwHrgF+sje93Rgzzd5n\nGNAfqAFuMMbMSIB2JU1wW0ldtxGsfft1y+emiYv8rvPreVdWwh13wOOPQ/v2MH069OkTkZ5kTowS\n6OlBgLlFveN+PqcSTnrlbuBmY8wXItISWCgiM+11jxpjHvbeWEQ6AxcDXYD9gfdF5FBjjP/bqqIo\nKSU/3KkHZ86EAQPg229h8GC4/35o2TLi8yVzYhQnTauYSkIaemPMBmCD/XqbiHwJBPtE+gITjDE7\ngbUisgboAcyLg15FUeJMSA97yxY6Pfig5b136gQffQQnnBDTOZP1FOe0aRVTRUQxehHpAHQDFtiL\nBovIEhF5VkRa28vygR+8dltH8BuDoigpJGgc/803oXNn9nvvPRg2DBYtitnIJxPtA7IQYwIlHfls\nKJIDfADcZ4x5Q0TygJ+x+m3uAdoaY64SkTHAPGPMy/Z+44FpxpjJPscbAAwAyMvL6z5hwoSoLqCy\nspKcnJyo9k0FbtLrJq3gLr1O19pk82YOHj2afT/4gG0HH0zZoEHUdu2aallh4fS29SUWvYWFhQuN\nMQUhNzTGhPwDsoAZwD8DrO8ALLNfDwOGea2bARwX7Pjdu3c30TJnzpyo900FbtLrJq3GuEuvY7XW\n1hrz/PPGtG5tTNOmxtx/vzG7djlXrx/cpNWY2PQCn5swbHjI0I2ICDAe+NIYM8pruXey7F+BZfbr\nKcDFItJURDoChwCfhrzjKIqSWr77zhrwdMUV0LmzFaYZNgyyslKtTImRcLJuegJ/B5aKiCcP63bg\nEhHpihW6+Ra4FsAYs1xEJgErsDJ2BhnNuFEU51JbC//9LxQVWe8ff9yqVZOhw2zShXCybj4Gv2Mq\npgXZ5z7gvhh0KYqSDFatgv79Ye5cKx/+qafggAMiOkRjLf3rJrRMsaI0Rqqr4eGHYcQIaN4cnn8e\nLrssrJGt3qRr6d90u3nps5miNDbKyqBHD7j9djj7bKt8weWXR2zkIT1L/8a9dLMDUI9eURoLO3ZY\nHvzIkdCmjVWA7Nxz/W7awKM9yn83WzqW/k3FpCiJRj16RWkMfPyxVSu+uJh3up7KURf9h55f5fr1\nUv15tOVbqvxum45lptPx5qWGXlHSmW3brLo0J57Ib5VV9L/0PgafMpitzXIChiT8ebS1xvgNx6Rj\nmel0vHlp6EZRfIi1I84xHXkzZlhFyH74AW64gb57ncYaH6e0qrqG4VOW19MXjkfrfY25zbNoukcG\nW6uqw75ex7SRH9KxPo4aekXxItYsEkdkoWzeDEOGwIsvwmGHWWGb44/n66KpfjevqKqmpKy8Tl+o\nio++17hlezXZWZk8elFX97RREJJZXTNZaOhGUbyINYskpVkoxsDrr8Phh8Orr1p148vK4PjjgeCh\nB299/sIxGSJ1Hq2r2yhM+nXLZ25Rb9YWn8Xcot6uNvKghl5R6hFrR1zKOvI2bIDzzoMLLoB27eCz\nz6zZn5o1q9skWOjBW5+/io/5rbPrjJ1r26gRo4ZeUbyItSMu6R15xsBzz1m1ad59Fx58EBYsAD+V\nJvt1y6d1c/91a3z1+Xq0udlZAbcNtTzc7dzc2el01NArihexZpEkNQtl7Vo47TS46io48khYvBhu\nvRX2CNz1dvfZXWLW56o2UgDtjFWUesTaEZeUjryaGhgzxqosmZFhFSS79tqwipDFQ58r2kiphxp6\nRfEh1mnuEjpN3ooVcPXVMG+eVVJ47Fj4wx+Srs/RbaQ0QA290qhwcv52UKqrrfj7PfdAy5Z8fs9j\n3JjZhfX/Xcr+uWvccx1KSlBDryQUJxlWp+dvB2ThQisOv2QJXHQR064Zxs2lG6iq3gG46DqUlKGd\nsUrCcFoVQDfkb9ejqgpuu82qNPnTT1BSAhMmcN9nvyTlOkrKyulZPJuORVNZ9eM2V1dvbOyooVcS\nhtMMa6rztyMynB9+aBUhe+ghy5tfsQL69g2qN57X4XuT3lVT6/pSvY0ZNfRKxHgbrJ7FswP++AMZ\nHn/D65NBKvO37yxZypCJi0Ibzl9/tabx+/OfYfdueP99ePppyM0NqTee1+G0m7QSG2rolYiIJBwT\nyPCIfZxkk6r87ZKycl6Z/z3GZ3kDwzltGnTpYmXSDBkCS5fCySc3ON7QPp3Iyqw/SUhWpsT1OlL9\n9KPEl5CGXkTai8gcEflSRJaLyI328r1EZKaIrLb/t7aXi4iMFpE1IrJERI5O9EUoySMST29on05+\nJxs2kBLP0N/Q/gfOPTLhHZgjZ6xqYOQ9rK+ogp9/hv/7PzjrLNhzT/jkExg1Clq0CHxQnwNW1xg+\n/25z3DTr6NX0Ipysm93AzcaYL0SkJbBQRGYCVwCzjDHFIlIEFAG3AWcAh9h/xwBP2v+VNCAST69f\nt3xumrgoouMkmkTmbwfKMAp4rcbw9x8WQOcrYcsWuPtuaxBU06ZBzzNyxiqqaxveOl6Z/z0FB+wV\nl+tzcqleJ2VyuYWQht4YswHYYL/eJiJfAvlAX6CXvdkLQCmWoe8LvGiMMcB8EckVkbb2cRSXE6qE\nrS/5EW7vVvylbt40cRHDpywnt3kWW7ZX19u+xZbNPP3GU5y6ZgEUFMCsWVYZgzAIdOPwPCn5Gr1o\nDKPv6NUmmRkJf/oJR6drU2RTTEQxehHpAHQDFgB5HuNt/9/X3iwf+MFrt3X2MiUNiDTO3VjqmvgL\naYFV671yx+7fY+rGcNHiGfzttsH0+n4RPPywNco1TCMPwW+SvjeBWFJcvQubddqvZcyGNFgnfrg6\ntZM4OsRyvMPYUCQH+AC4zxjzhohUGGNyvdZvMca0FpGpwAPGmI/t5bOAW40xC32ONwAYAJCXl9d9\nwoQJUV1AZWUlOTk5Ue2bChKlt6Kqmo1bd7CrppYmmRnktWpWr+JgNATSGum5EqEtEr3JYGn51qDr\nMzOE1pt+5IRxT9B+xVJ+PuIIvi4qoio/cuNZUVXND5u3+13XJDODTvu1rHu/6sdt7KqpDbldKGJt\n24qqasq3VFHrZW8yRMhvnU1udlbYOoO185H5reKiNdnEorewsHChMaYg1HZhjYwVkSxgMvCKMeYN\ne/FGT0hGRNoCm+zl64D2Xru3A9b7HtMYMw4YB1BQUGB69eoVjpQGlJaWEu2+qSARekvKyhk2aylV\n1Rl4HtKys2p44NzOMXlh2rbhc0fx7IBpoxm1NVz1+RQGzn/Vqiz51FMsO/hgevXuHfX57ixZ2iCT\nJzsrkwfOPZJeXp/5lUVTMX4e3AVYW9wr7PPF2rY9i2dTXpHZYHl+biZzi3qFrTNQO+fnZvOPv/WK\ni9Zkkwy94WTdCDAe+NIYM8pr1RTgcvv15cBbXssvs7NvjgW2anw+sejjbOrxF6ICOPSnb3nj5aHc\nOWe8lSq5YoU1j2sYlSaDcW+/I3n0oq4hM4ickj0TqhM/XJ2NJRQYb8Lx6HsCfweWiognheJ2oBiY\nJCL9ge+BC+x104AzgTXAduDKuCpWGuD0nGe3Z0mEo9/zfsTby9myvZqsmmoGzvsfg+ZNorJZcz67\n/wn+VDQQxF/CaXTnDSeDyCnZM6E68cPVqSWOoyOcrJuPwW86NECD0Rx2ts2gGHUpERBpJkwycXqW\nRDBjWlJWzvApy6mo+j1jJph+j+EtfXkq7W/5JwdtXMt7R53M7lGjOLP3HyPWFa92a7pHRt1xWjfP\n4u6zuyS97UMZ8kgMuJY4jhytXpkGOMVr80ewsFKqf6zBjCnQoE09BNS/fTv861/0evRRaNsWpkzh\ntLPPrnc+jyEr6lpLRVl5wDaIR7v5Xh/AjuqGHZ7JIBxDrgY8caihTwOc/Djr5LBSqL4Nf0beQwP9\nc+bANdfA119bsz09+CC0alW32tfoemrdgH8PPR7t5rSbrBry1KGGPk1w6o/IyWGlWIxpnf6tW615\nWseNg4MOgtmzobCwwfaRGt14tJuTb7JKctGiZkpCcXKWRLBMj2AGtU7/229D587wzDNwyy3WxCB+\njDxEbnTj0W5OybhRUo8aeiWhpKqQWDA8IzTLK6oaZBl4jGmgdMnWzbN4pHB/+o0cCuecA3vvDfPn\nw8iR0Lx5wHNGanTj0W7JvsmGW75aST4aulEiJtJ0yVjCSvFOzfSNlRuslDKDZUx9j1/v3KcdSr+V\nH8KFl1p140eMgKIiaNIk5Hmj6TCPxwTcDa4hQX03Ts+uauyooVciIpk/6FBZMb4GLDfQgbzwFyv3\nGPm5RfVHqtYztOvWwfXXwzvvwDHHwPjxVu34MIlHkbBoi5Mlw9A6reNXqY8a+iRTUVVNz+LZjsuO\nCZdk/qBHvL3c77lGvL2cHdW1DW4ADxzfMNTiS8QdlLW11gxPQ4eyu7qa/551PY91Pp393t7I0F25\nEV2zt9EtLS2tV6ogFMm4wcby9OSmjl+3D+CLBjX0SaSkrJzyLVV1NT/c+HgbbHrAkiB54ZFSUlbe\noLSvB3/Lq6pr2LjV//behJvNUlJWzmuvzGLIpJEc+8Myvj6yB9eddB2rc6wircn+7BJ9g43mRuJt\nMDNEqPFTINFpHb+NNcSknbFJZOSMVfWq94H7atIE++HGc/LoaNrEX/VDX8LpoHzrs+/4aujdvPDY\nNXTetJZbT7+BU864q87Ie/D32SWqQzKRHnNJWTk3T1ocUb0k37LC/oy8U7KrvGmsdaHU0CcRNz3e\nBiJQNgrE9wcTrE0ClThukhn66xwym2XJEg79ax9unTWeDzsezSn9/8uko07DBKhR460zltrvoUhU\nqqRHsz9DDYE/h0D19zNFHJNd5Y90+A1Gg4Zukoj1o9wWYHlyiTZO6dkm0VMEBgqx5GZnMfycLn4z\nWPJahc5+gQAdlDt3wv33w/3306ZJCwadcxtTDzshZBGyVl43nVjCK6E+j0SVuQhksD0E+m4G+pxr\njWFt8VkxaUokTh7Al0jUo08iQ/t0IsPHcKTi8TYenmdmAAMYrx9MoBDL8HO6BPTKo57MZP58OPpo\n+Pe/4eKL+fs/n2Pq4SeGVWnyt12769otWm8xnM8jUeMRgmkL9t1062AsJw/gSyRq6JNIv2755LfO\nTvngoVjilMEe9eP5gwll2LynuZtb1Du6NvztNxgyBI4/3sqLnzoVXnqJa8/t4dcYNM9q+HOprjF1\n7Rat8Qvn80hUpkggbZkiQb+b/gymYN2knDxYyokD+JKBhm6STG52FnOLetVblux0r1jilMFis9H8\nYIJde0JzwGfNsoqQrV0LAwfCAw/AnnvWnRca5ukPCRGuija8EurziFemiL+2DqQ51Gfp3UaeEcae\nW7/TM1mcWhcqkahHn2IS2YEXiFgeu4PFZn1/PKEyUFJx7VRUwNVXwymnWNP6ffABjBlTZ+Q9eD8x\nDO3TiZEzVhFodmVPu0XrLYb6POKRKRKorYE6zWDdsD3HDvU5eNooPze7Qds0hkwWN6EefYrweFf+\nOoYSPaIwlo69SPLQQ3mhgQzYiLeXJ+YJ5623rNGtmzbBbbfB3XdDdvCbm7+a7t74tls03mKozyPR\nJYs9I4KjfWporJksbkI9+hTg7V0FIpE/kljilIHSK7f8trOe5x6OFxroGrdsr46vl79xI1x0EfTr\nB/vuCwsWQHFxSCMPwbNS4hXfDfV5xKPjM5QxjuWpwa0ds40JNfQpIFRKGyT+RxJtZ2a/bvmc1z2/\nQdXH7dW19QxzoJuYt8HJbR5elkzUYQBj4KWXrFLCJSVw773w2WfQvXvYhwhkIAWi7wT2g+fzePSi\nrgAMmbio7qaZjJLFsXjljTWTxU2ooU8BoX48Tv+RzFn5U8B4NViGOZz0ywBjdPwS8RPO99/DWWfB\nZZdBp06waBHccQdkRZaCmUxvNZw4eqJKFsdynY01k8VNhIzRi8izwF+ATcaYI+xlw4FrgJ/szW43\nxkyz1w0D+gM1wA3GmBkJ0O1qAsW5wX+pXKcRjtGtMYbsrMyg/QBbvSbdDkXYhrW2FsaOtWLwtbXw\n2GMwaBBkhi545o9kzscbKo6eyJLFsV5nY8xkcRPhdMY+DzwBvOiz/FFjzMPeC0SkM3Ax0AXYH3hf\nRA41xgSPUzQyok1pSxSRpncGu1F58Nywgh030HG8U/UgfIOT/cMP0KsXfPQRnHqqNb1fhw4h9wtG\nMmu6J7pTM5gxdvK8w0rshDT0xpgPRaRDmMfrC0wwxuwE1orIGqAHMC9qhWmIk35UwbJjAtV393ej\n8sZjmEN5eYFueOd1z2fOyp/Cb5vdu+GRR/jTXXdBixbw3HNw+eVhjWz1JdBNLxmfTaqH56tXnr6I\nCSNQahv6d3xCN1cAvwKfAzcbY7aIyBPAfGPMy/Z244F3jTGv+znmAGAAQF5eXvcJEyZEdQGVlZXk\n5OREtW8qcJreVT9u81v1sUlmBvk5ElBrRVU1G7fuYFdNLXtkiFXBsNbQJDODvFbNwi5H4H2cSPcF\nyFmzhk4PPUTL1avZcNxxrL35ZnbtvXfY+4PlMW/+rRrjp+chQ4T81tnRl1cIgL/vQUVVNeVbqupV\nOE3U+SPFad/bYLhJK8Smt7CwcKExpiDUdtHm0T8J3IP1hH0P8AhwFTRIxgD899sZY8YB4wAKCgpM\nr169ohJSWlpKtPumAqfpvbJoKsZPn7wAz53ewlFa67FjB9xzDzz4IOyzD7z+Oqv23jtivXeWLOXl\n+d8DgWP4+bmZDUYzx0qg74FTJ8Vw2vc2GG7SCsnRG5WhN8Zs9LwWkaeBd+y364D2Xpu2A9ZHrU5J\nGB6DEmq0pyP55BPo3x9WrrRCNKNGwV57QWlpxId6bcEPIbeJ55gGT7tf3H4bdxTPbmDINXyiJIKo\n0itFpK3X278Cy+zXU4CLRaSpiHQEDgE+jU2i80nUZBOJItSALSemd5aUlXPKiKk8X3AOtSecwPaK\nbTB9Ojz/vGXkoyRQHXZv4nV8BGVjAAAby0lEQVTT8233pJR8UBTCMPQi8hpWZ2onEVknIv2Bh0Rk\nqYgsAQqBIQDGmOXAJGAFMB0YlO4ZNymp1xIjyRjtGU9Kysp5e+TzPD/qKi5b+A4vdjuLHhc/Soc5\nu2O+sQbK9/cQz5teY53dSEk94WTdXOJn8fgg298H3BeLKDeRzMmy40Wo0Z6OYssWsq7pz/iFM/h6\nr3Zc+LdiPm/XpW51OFlCwbjkmPZ2jL4h8R7TEK/0SafG8RXnokXNYsSNBZ1SncYXNm+8AYMG0Wfj\nJsYcewGje17Czj0aziJVVV3DzZMW88hJkWem3NvvSMCK1dcYQ6YIlxzTvm55PIlHuzfWya2V2FBD\nHyPh/Hi9PbCirrVUlJX7/VEG8tTi7cElc7RnVPz4IwweDJMnQ9euXH3BcEqbtwu6S40xlG+poiRA\n2wbj3n5HJsSw+xKPdnfjE6SSerTWTYyEqiHiG8PfVVPrN4YfKNZ/Z8nSuPcBOLY2iTFW52rnzvDO\nO9ZkIJ9+Sr8r/xJwQnJvao1xdLzbu90hunZ34xOkknrUo4+RUKNcw/XAAm3nCSmE2j8a3Sk37N58\n+y1cey289x6ccAI88wx06lSv5HGmCDXGNCiR4I3TDZ6n3UtLS/nH33pFvL9rwm6Ko1BDHweCGc1w\nPbBA2wVK/3O6QfMlYPipttaa4WnYMKtkwRNPWJODZGQ0iEd7CqWd1z3f7w0Q0t/gOT7spjgSDd0k\nmHDLvwabpDmS46aCaKcMfP+ND+Ckk+CGGywvftkyq9JkhvW1DPSUM2flTzxy4VENwjkZImlv8Bwb\ndlMcjXr0CcDbe22VnUVWplBd87v36c8DC1bga/LCcsd6cNFMGbhHzW6unDeJkx54DfbMgRdegL//\nvUERsmBPQ/5CZvmtaxqFwXNc2E1xPGro44yv4auoqiYrQ2jdPIuK7dU0yczw64H5M1yFh7Vhzsqf\n6sWnnVavPpw+CG+D3eXHNTz07mi6bPqGaZ16cuYHkyEvz++xc5tnsWV7w5r1GSJ0LJraoD+kNIoS\nCNGgeeyK21BDH2f8Gb7qWkPzJntQ9q/TrAJGQWqCewyGv/i0AIWHtXGUUQmnD2L/3Gx+/qmCGz95\njQEL3mBz81Zc+9fbWdbjZM4MYORLysqp3LHb7zpPbD4VOeSax664ETX0cSZe6W/DpyxvcMMwwCvz\nv6fggL0cY1TCyQJ5YN+ttB95Ax03lzPxyFO5r3d/qlu24gGvFFRfD3nkjFVU14auQ5PsHPJU5LHr\nE4QSK9oZG2fiMcdoSVk5FQGm2TPgqFzxoOMItm2DwYM5qf+57Nssgxv7P0TRmTfScr82deGrQB21\noWaw8sZzE62oqk54cblk57G7sZaS4jzUo48z8Rr9GAwnpVYGHEfw4xLoey2sWwc33kiLe+/lsZwc\nHvPZP5CH7OmTCIf9c7MpKSunfEsV5RXWTSdRIZVk57HrSFglHqihjzPxmCYwlCGP1qgkKgRQLwvk\nl19gyBB46SU4/HCYOxeOOy7gvsHGD/hOLp6VKWCoF9Lx3EStGu/xH1jmS7Lz2HUkrBIP1NAngFjT\n34JNvp2dlUnhYW3oWTw7IoMdTidigxvBURFUmDYGXn/dqlGzeTPceaf117RpVNcaaHJx8H8THTJx\nUf0pb2zibRA9bTV8yvK68FqzrMRFQHUkrBIP1NA7kECTb+dmZ/GXo9rWy6sPN0QRKgTg70ZQvqUm\nvCJhGzbAwIFQUgLdu1tlDI46KuprDTW5uL9lluHbFmB5/Nm5+/d5drdsrw7rMwi3uJ03ThkJ6+n/\n0A5hd6KdsQ7E3+jH/1zUlUV3n8bUJRuimrwiVAjA340gZJEwY+DZZ60QzfTp8NBDMH9+2EYe4jfS\nc2ifTmT4DLhKlEGMZgKRcIvb+eKEkbC/939oh7BbUY/eofjzZkvKyv0OIILw4vrBQgARx4LXroUB\nA+D9960yBk8/DYceGlRDIOIx0rNft3xKflxBfm5mwr3OaOLmsXSqpnokbLL6P5TEoYbeJZSUlXPz\npMUB14cKUYQKAYQdC66psQqP3X47ZGbCk09aBj8j9Q+HudlZzC3qlfDzRBM3d3On6vqKqqT0fyiJ\nI/W/TiUknsf+YOmGoUIU/brlc173/LoiaZkinNf9d0/RXz58gyJhK1ZYxcduugn+/GdYvhyuuy6u\nRt4NE62HmoPAH/EYX5Eq3KxdsQhncvBnRWSTiCzzWraXiMwUkdX2/9b2chGR0SKyRkSWiMjRiRTf\nWAg2mTdYnmw4WTeTF5bX3SxqjGHywvI6Q+ovFpzfOpt+3fJ569O1PH3qlez841FULFnB5/eOhqlT\nob0fNy8G3DI4KJq4eTQ3B6eQzP4PJTGEE7p5HngCeNFrWREwyxhTLCJF9vvbgDOAQ+y/Y4An7f9K\nDAQbJZqdlcnwc7r4Xeed5ZHhZwBSVXUNw6csD5hbX1paSukr0+j0z8H03bSWKYefxIiTB7B9x948\nsGh9QMMWbb6+mwYHRRo39x1fEai4nRNJZv+HkhhCGnpjzIci0sFncV+gl/36BaAUy9D3BV40xhhg\nvojkikhbY8yGeAlujAQbJRrIWPgriuaPiqrqunzweqmah+3FgWPHkj/pf/zUIperz72L9w+x79lB\njG8sRb/cHMcOB++bQ7Didk4kWf0fSmIQE8Ywc9vQv2OMOcJ+X2GMyfVav8UY01pE3gGKjTEf28tn\nAbcZYz73c8wBwACAvLy87hMmTIjqAiorK8nJyYlq31QQjd6l5VsDrjsyv5Xf5at+3Maumlq/60Jx\nwKoVnPLsGJqXl7Os8FTmXnw5u1o01Ozv3IHO2yQzg077tQx63lj2Bfd8Fyqqqtm9YzsbtlvXlteq\nGbnZWamWFRS3tC24SyvEprewsHChMaYg1HbxzrrxNx2S3zuJMWYcMA6goKDA9OrVK6oTlpaWEu2+\nqSAavXcUzw44ejTQvKNXFk3FRNjXnrNzO0Wlz3HOonfhwANZ9MgjDNrVlfJvwj93oPMKsLbYv1YP\nFT5PA2CFph4498iwvF83fBdKysoZNmspAw+DR5ZaP7/srBoeOLezo0MhbmhbD27SCsnRG226xEYR\naQtg/99kL19H/USsdsD66OUpEN8sj0wRBBp4kIVff8Z74wdyyeIZvHbC+bBkCRVHHx3xuWPJ0HDC\n4KBEE81gK0WJlWgN/RTgcvv15cBbXssvs7NvjgW2anw+duKZ5fHIhUextvgsWjS1vMnW27fy6NsP\n89zrI9jWtDnn/99Iskf/B1q0iOrcsWaX9OuWz9yi3qwtPou5Rb3TyshD+vdDKM4kZOhGRF7D6njd\nR0TWAXcDxcAkEekPfA9cYG8+DTgTWANsB65MgGbXE01WSqxZHp6pCUfOWMWQiYswxnD2lx8y/P2n\naLlzO//peQljjruQ6sws3vQzzWE46Zve8+Q2y8qgYnu1Zmj4oEXKlFQQTtbNJQFWnexnWwMMilVU\nOlNRVc2wWdFPRRfJTSLQ1IR5237m3vee5NQ1C1jU9hBuO+NGVrXpAFgee6T4myc3OyuTRy/qqgbe\nB88IZfh9mkTNSVcSjZZAiCPhGOGNW3dQVV0/YhZurngsqYsjZ6yiatduLl48g9vnPEtWbQ33Fl7F\nswV9qc2wQi1iH7Nn8WyG9ulEbtAj+hzbJfnvqcbTHhtXfYGAPvEoSUENfZwI1whb6YMNu0bCidHG\nYlD3WPsNr05/nOO/X8K8PxxJ0en/4LvW+9etF35Pj/Jof+D4TL/HCle7xp39069bPqVbV4fMQlKU\neKGGPk6Ea4SbZPrv/w4nRhuVQa2pgcceY8Zzt7NLMhnWZzATjjoNI5YOT6jGN25cVV3Dxq3+K2X6\n065xZ0VxLlrULE6Ea4TzWjWLOisl4tTFZcvg+OPh5pupOPZEzrluLK91Pb3OyHvOG0h7uAOu3FzH\nRVEaA2ro40S4Rjg3OyvqXHF/BlWAwsPa1N9w1y4YMQKOPhq++QZee439PniPm67o7fe8gbQHevrw\npTHkvyuKm9HQTZwIZ8q3krJyNv64jeLpi9g/NzvirJR+3fL5/LvNvDL/+7p4ugEmLyyn4IC9rGN9\n+in0729585deCo89BvvsU7e/v/MF0p7XqklE2tSwK4ozUY8+ToTyaj2dtbtqamMqwTtn5U8NakpU\nVdcw+u3FcPPNcNxxsGULvP02vPJKnZGPRrvT668oihIe6tHHkWBebbxSEP3F04/7bgnF00dDxY9w\n7bXw4IPQyn+xs0i0l5aujugYiqI4EzX0SSJeKYjeGS4td/7GsDnPcuniGazba3+YMwcSUBwp2vry\niqI4Aw3dJIl4Tcfm6ZA9ec0CZj5zPRctmcn4486n7O0PEmbk3TDrk6IogVFDnyTilYLYr10TZn42\nlvGT76Eie08GDHyCvcf8h7OPPziecuvQaouK4n40dBNnAoU5Yh76bgy8+irceCPtfv0V/v1vDrvt\nNsY3CT8zJhp01KuiuB819HEkVBmEqIe+//ADXH+9NSH3Mccw65b7+dcaWP+vmXVVKees/CkhMXQd\n9aoo7kdDN3Ek7mGO2loYOxa6dLE6Wh99lJInJjF48a56MfOX53+fsBi6jnpVFPejhj6OxDXMsXo1\n9O5tefI9esDSpXDTTYx8f02Dm4kv8Yyh66hXRXE/GrqJI9GEOXxj+reefBAHvfI0h4wZyc7MLJ44\n7xY6334j/Q5sB4R/04hnDF1HvSqKu1FDH0fCKYPgjW9Mf8+vlnPgY9dyxI9reO+QY7nz1OvZ1HJv\nsl5fAiJ1dWn83Ux80Ri6oigeNHQTR/yFOc7rns/IGavoWDSVnsWzqaj6vfSvJ6bfZHc1//zwJaa8\nMIT9fv2ZgX2LGPDXO9jUcm8AqmsNw6csB/zHzH3RGLqiKN6oRx9nAk3fB1ZHafmWGkrKyunXLZ/1\nFVUcXf4lD747mkN++YHJXQq55+RrqMjes8FxPTeIQHPBRpJ1oyNdFaVxEZOhF5FvgW1ADbDbGFMg\nInsBE4EOwLfAhcaYLbHJdCf+snBqjbHq2xyay4MfPcv5n7zJhpb7cMX5wyk9qCCs48YSM49lOkJF\nUdxJPEI3hcaYrsYYj5UqAmYZYw4BZtnvGyWBOkQ7LpoHRxzBhZ+8wYTuZ9Gn/5iQRr518/hUktSR\nrorS+EhEjL4v8IL9+gWgXwLO4Qp8O0T33FFJ76cf5+WJd0JWFnz4Ic2fHkurvL3rYvr/d+wfyMqU\nevtlZQp3n92l7n1JWTk9i2fXxf0jyZnXka6K0viINUZvgPdExABPGWPGAXnGmA0AxpgNIrJvrCLd\nincWzmlfzeOemU/SZnsFX105iEPHjITsbPrRMGRScMBeAWPosYZedKSrojQ+xBjfaSwi2Flkf2PM\netuYzwT+AUwxxuR6bbPFGNPaz74DgAEAeXl53SdMmBCVhsrKSnJycqLaNxlUrd/EoaNHc9CCufx8\nQEeW3vAP5OhuUR9v1Y/b/M7l2iQzg077tQy5f0VVNeVbqqj1+twzRMhvnd1gohGnt60vbtLrJq3g\nLr1u0gqx6S0sLFzoFTYPSEwevTFmvf1/k4i8CfQANopIW9ubbwtsCrDvOGAcQEFBgekVZYnd0tJS\not03oRgDL70EN90Ev/0G993HPkOHInPnxqT3yqKpGD8RN4Gwa+iEm3Xj2LYNgJv0ukkruEuvm7RC\ncvRGbehFpAWQYYzZZr8+Dfg3MAW4HCi2/78VD6Gu4vvvrZmepk+3pvYbPx4OPzwuh45H6EVHuipK\n4yKWztg84GMRWQx8Ckw1xkzHMvCnishq4FT7feOgthbGjLGKkH30EYwebf2Pk5EHLTKmKErkRO3R\nG2O+AY7ys/wX4ORYRLmSVavg6qvh44/h1FNh3Djo0CHup/E3YEoHPCmKEgwdGRsr1dXwyCMwfDhk\nZ8Nzz8Hll4NIyF2jRUMviqJEghr6WCgrg/79rf/nnmuFbfbbL9WqFEVR6qFFzaJhxw644w74059g\n/Xp4/XWYPFmNvKIojkQ9+kiZO9fy4letskI0o0bBXnulWpWiKEpA1KMPl8pKuOEGOPFEy6OfPh2e\nf16NvKIojkcNfTjMmAFHHAFPPAGDB8OyZdCnT6pVKYqihIUa+mBs3gxXXAGnnw7Nmv2eG++i4dWK\noihq6AMxeTJ07gwvvwy33w6LFkHPnqlWpSiKEjHaGevLhg1WeOaNN6BbNysW37VrqlUpiqJEjXr0\nHoyxOlc7d4apU6G4GBYsUCOvKIrrUY8e4NtvYcAAmDkTTjgBnnkGOmntGEVR0oPG7dHX1MDjj1sZ\nNfPmWSNbP/hAjbyiKGlF4/Xov/zSKkL2ySdWVs3YsXDAAalWpSiKEncan0dfXQ333WfF3leuhBdf\nhGnT1MgripK2NC6P/osv4KqrYPFiuOACK2yTl5dqVYqiKAmlcXj0VVVQVAQ9esDGjVbq5KRJauQV\nRWkUpL9H/9FHViz+q6+sYmQjR0LrBnOVK4qipC3p69H/+isMGgQnnQS7dlmpk888o0ZeUZRGR3oa\n+nfftVImn3wSbrrJKkJ2yimpVqUoipISEmboReR0EVklImtEpChR56nHL7/AZZfBmWdahcfmzoVH\nH4UWLZJyekVRFCeSEEMvIpnAGOAMoDNwiYh0TsS5AKt8waRJcPjh8NprcNdd1vR+xx2XsFMqiqK4\nhUR1xvYA1hhjvgEQkQlAX2BFvE/U5OefrflaS0qge3d4/3344x/jfRpFURTXkihDnw/84PV+HXBM\n3M8ybRo9rrjCKmXw0EMwZAjskf6JRIqiKJEgxpj4H1TkAqCPMeZq+/3fgR7GmH94bTMAGACQl5fX\nfcKECRGfJ7u8nI6jRrF2yBCq2rWLj/gEU1lZSY5LJi5xk1Zwl143aQV36XWTVohNb2Fh4UJjTEHI\nDY0xcf8DjgNmeL0fBgwLtH337t1NtMyZMyfqfVOBm/S6Sasx7tLrJq3GuEuvm7QaE5te4HMThk1O\nVNbNZ8AhItJRRJoAFwNTEnQuRVEUJQgJCWgbY3aLyGBgBpAJPGuMWZ6IcymKoijBSVjPpTFmGjAt\nUcdXFEVRwiM9R8YqiqIodaihVxRFSXPU0CuKoqQ5augVRVHSHDX0iqIoaU5CRsZGLELkJ+C7KHff\nB/g5jnISjZv0ukkruEuvm7SCu/S6SSvEpvcAY0ybUBs5wtDHgoh8bsIZAuwQ3KTXTVrBXXrdpBXc\npddNWiE5ejV0oyiKkuaooVcURUlz0sHQj0u1gAhxk143aQV36XWTVnCXXjdphSTodX2MXlEURQlO\nOnj0iqIoShBcbehTMgF5BIjItyKyVEQWicjn9rK9RGSmiKy2/7dOob5nRWSTiCzzWuZXn1iMttt6\niYgc7QCtw0Wk3G7fRSJypte6YbbWVSLSJ5la7fO3F5E5IvKliCwXkRvt5Y5r3yBaHdm+ItJMRD4V\nkcW23hH28o4issBu24l2iXREpKn9fo29voMDtD4vImu92rarvTwx34NwitY78Q+r/PHXwIFAE2Ax\n0DnVunw0fgvs47PsIaDIfl0EPJhCfScBRwPLQukDzgTeBQQ4FljgAK3DgVv8bNvZ/j40BTra35PM\nJOttCxxtv24JfGXrclz7BtHqyPa12yjHfp0FLLDbbBJwsb18LHC9/XogMNZ+fTEw0QFanwfO97N9\nQr4Hbvbo6yYgN8bsAjwTkDudvsAL9usXgH6pEmKM+RDY7LM4kL6+wIvGYj6QKyJtk6M0oNZA9AUm\nGGN2GmPWAmuwvi9JwxizwRjzhf16G/Al1lzKjmvfIFoDkdL2tduo0n6bZf8ZoDfwur3ct209bf46\ncLKISIq1BiIh3wM3G3p/E5AH+3KmAgO8JyIL7TlyAfKMMRvA+oEB+6ZMnX8C6XNqew+2H3Gf9QqD\nOUqrHSrohuXNObp9fbSCQ9tXRDJFZBGwCZiJ9VRRYYzZ7UdTnV57/VZg71RpNcZ42vY+u20fFZGm\nvlpt4tK2bjb0/u7ITksh6mmMORo4AxgkIielWlAMOLG9nwQOAroCG4BH7OWO0SoiOcBk4CZjzK/B\nNvWzLKma/Wh1bPsaY2qMMV2BdlhPE4cH0ZRSvb5aReQIrHm0DwP+BOwF3GZvnhCtbjb064D2Xu/b\nAetTpMUvxpj19v9NwJtYX8iNnkcx+/+m1Cn0SyB9jmtvY8xG+0dUCzzN7+EDR2gVkSwsw/mKMeYN\ne7Ej29efVqe3L4AxpgIoxYpn54qIZ9Y8b011eu31rQg/DBg3vLSebofLjDFmJ/AcCW5bNxt6R09A\nLiItRKSl5zVwGrAMS+Pl9maXA2+lRmFAAumbAlxmZwUcC2z1hCBShU/s8q9Y7QuW1ovtbIuOwCHA\np0nWJsB44EtjzCivVY5r30Bandq+ItJGRHLt19nAKVj9CnOA8+3NfNvW0+bnA7ON3fOZIq0rvW72\ngtWX4N228f8eJKv3ORF/WD3UX2HF5+5ItR4fbQdiZSYsBpZ79GHFBmcBq+3/e6VQ42tYj+TVWJ5E\n/0D6sB4px9htvRQocIDWl2wtS+wfSFuv7e+wta4CzkhB256A9ci9BFhk/53pxPYNotWR7Qv8ESiz\ndS0D/mUvPxDrhrMG+B/Q1F7ezH6/xl5/oAO0zrbbdhnwMr9n5iTke6AjYxVFUdIcN4duFEVRlDBQ\nQ68oipLmqKFXFEVJc9TQK4qipDlq6BVFUdIcNfRKWiAiNXYVwGUi8j8RaR7DsXqJyDv263MkSGVU\nEckVkYFRnGO4iNwSrUZFiQQ19Eq6UGWM6WqMOQLYBVznvdIegBLx990YM8UYUxxkk1ys6oiK4ljU\n0CvpyEfAwSLSQawa6/8FvgDai8hpIjJPRL6wPf8cqJvbYKWIfAyc6zmQiFwhIk/Yr/NE5E27tvhi\nETkeKAYOsp8mRtrbDRWRz+yCVSO8jnWHWPXb3wc6Ja01lEaPGnolrbBrmZyBNaoQLIP6ojGmG/Ab\ncCdwirGKzX0O/FNEmmHVcjkbOBHYL8DhRwMfGGOOwqqNvxyrpvzX9tPEUBE5DaskQA+sYmDdReQk\nEemOVaajG9aN5E9xvnRFCcgeoTdRFFeQbZeCBcujHw/sD3xnrLreYBW+6gzMtcuRNwHmYVURXGuM\nWQ0gIi8DA2hIb+AysCoSAlul4Qxhp9l/Zfb7HCzD3xJ40xiz3T6HY+oyKemPGnolXagyVinYOmxj\n/pv3Iqx64Jf4bNeV+JWtFeABY8xTPue4KY7nUJSI0NCN0piYD/QUkYMBRKS5iBwKrAQ6ishB9naX\nBNh/FnC9vW+miOwJbMPy1j3MAK7yiv3ni8i+wIfAX0Uk265qenacr01RAqKGXmk0GGN+Aq4AXhOR\nJViG/zBjzA6sUM1UuzP2uwCHuBEoFJGlwEKgizHmF6xQ0DIRGWmMeQ94FZhnb/c60NJYU/VNxKoM\nORkrvKQoSUGrVyqKoqQ56tEriqKkOWroFUVR0hw19IqiKGmOGnpFUZQ0Rw29oihKmqOGXlEUJc1R\nQ68oipLmqKFXFEVJc/4fnmIewMM7eDAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a096f6470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(y_test,y_test_pred)\n",
    "plt.plot([0,350],[0,350],'r')\n",
    "plt.xlabel('Actual')\n",
    "plt.xlabel('Predicted')\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manually computing the coefficients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can manually compute the regression coefficients using the least-squares matrix formula in class.  Of course, using the `sklearn` package is easier.  I illustrate the manual computation only to show what is happening \"under the hood\" of the linear regression routine. \n",
    "\n",
    "To compute the coefficients manually, we first construct the matrix $\\mathbf{A} = [ \\mathbf{1} ~ \\mathbf{X} ]$.  This is performed with the `ones` command and `hstack`.  Note that after we do this, `A` has 11 columns -- one more column that the data matrix `X_tr`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(300, 11)"
      ]
     },
     "execution_count": 150,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ones = np.ones((ns_train,1))\n",
    "A = np.hstack((ones,X_tr))\n",
    "A.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We next use the `lstsq` method to solve $\\mathbf{y} \\approx \\mathbf{A\\beta}$.  This will find the desired least-squares fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "out = np.linalg.lstsq(A,y_tr)\n",
    "beta = out[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 152.34786452,  -16.57607993, -254.66532396,  560.98630022,\n",
       "        278.91811152, -393.41357305,   97.05460405,  -19.0023093 ,\n",
       "        169.46450327,  632.95050374,  114.21638941])"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "beta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that `beta[0]` mathches the intercept in `regr.intercept_` from the regression fit and `beta[1:]` matches the coefficients in `regr.coef_`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ -16.57607993, -254.66532396,  560.98630022,  278.91811152,\n",
       "       -393.41357305,   97.05460405,  -19.0023093 ,  169.46450327,\n",
       "        632.95050374,  114.21638941])"
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "152.34786451820108"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us verify this is the same as direct matrix calculation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [],
   "source": [
    "Am=np.matrix(A)\n",
    "ym=np.matrix(y_tr)\n",
    "ym=np.transpose(ym)\n",
    "Amt=np.transpose(Am)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "beta=np.linalg.inv(Amt*Am)*Amt*ym"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[ 152.34786452],\n",
       "        [ -16.57607993],\n",
       "        [-254.66532396],\n",
       "        [ 560.98630022],\n",
       "        [ 278.91811152],\n",
       "        [-393.41357305],\n",
       "        [  97.05460405],\n",
       "        [ -19.0023093 ],\n",
       "        [ 169.46450327],\n",
       "        [ 632.95050374],\n",
       "        [ 114.21638941]])"
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "beta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}