Crop Science.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<h2 style = 'text-align: center;'>Satellite Data Classification</h2>\n",
    "\n",
    "<p>    For this project, we need several python libraries to process the data, and analyze the data. Supervised Machine Learning algorithms like SVM and knn are applied to the dataset. Several data processing libraries are used. For example:</p>\n",
    "<ul>\n",
    "    <li>dbfread (to access the data in the database)</li>\n",
    "    <li>osgeo (to get the different band data in the satellite and CDL data)</li>\n",
    "    <li>openCV, numpy, matplotlib and pandas(for general image and data analysis</li>\n",
    "\n",
    "</ul>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from dbfread import DBF\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = DBF('mydata.tif.vat.dbf')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cornData = []\n",
    "for rows in data:\n",
    "    cornData.append([rows['CLASS_NAME'], rows['RED'], rows['GREEN'],rows['BLUE'], rows['OPACITY']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import cv2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cornData.insert(0, ['CLASSNAME', 'RED', 'GREEN', 'BLUE', 'ALPHA'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "cornDF = pd.DataFrame(cornData[1:], columns = cornData[0])\n",
    "corn_np_array = np.array(cornData)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "USDA_Map = cv2.imread('CDL_2013_Champaign_north.tif')\n",
    "USDA_data = np.array(USDA_Map)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<p style = 'line-height: 7px;'> Then we have put all the color and class name data in the database to a pandas dataframe </p>\n",
    "<p style = 'line-height: 7px;'> We have also created a numpy array to store the data above </p>\n",
    "<p style = 'line-height: 7px; margin-bottom :30px;'> We have used openCV to read our CDL Image file, we have converted it to a numpy array <p>\n",
    "\n",
    "<p> Then we use gdal to fetch our satellite data, and we can seperate its 5 bands into 5 big matrices. I have plotted the histograms of these 5 matrices. Observe that our every matrix has the same shape as our CDL labelling data. It is very easy for us to apply machine learning algorithms on those data</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from osgeo import gdal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "gtif = gdal.Open('satellite.tif')\n",
    "tif2 = gdal.Open('CDL_2013_Champaign_north.tif')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>gtif is our satellite data, which is our raw input data x. tif2 is our CDL data, which is our labelling data f(x)</p>\n",
    "<p> Observe the our x and f(x) have the same shape and they represent the same region for every pixel. The histogram of each band of the satellite data is plotted below<p>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'AREA_OR_POINT': 'Area'}\n",
      "{'AREA_OR_POINT': 'Area'}\n",
      "5 1\n",
      "the shape of the 1 matrix is (5959, 9425)\n",
      "the shape of the 2 matrix is (5959, 9425)\n",
      "the shape of the 3 matrix is (5959, 9425)\n",
      "the shape of the 4 matrix is (5959, 9425)\n",
      "the shape of the 5 matrix is (5959, 9425)\n",
      "the max of the 1  matrix is 35709\n",
      "the max of the 2  matrix is 35412\n",
      "the max of the 3  matrix is 31680\n",
      "the max of the 4  matrix is 24759\n",
      "the max of the 5  matrix is 24381\n"
     ]
    }
   ],
   "source": [
    "print(gtif.GetMetadata())\n",
    "print(tif2.GetMetadata())\n",
    "srcband = gtif.GetRasterBand(1)\n",
    "count = gtif.RasterCount\n",
    "count2 = tif2.RasterCount\n",
    "print(count,count2)\n",
    "\n",
    "asdfb = srcband.GetDataset()\n",
    "\n",
    "band1 = srcband.ReadAsArray()\n",
    "band2 = (gtif.GetRasterBand(2))\n",
    "band2 = band2.ReadAsArray()\n",
    "band3 = (gtif.GetRasterBand(3))\n",
    "band3 = band3.ReadAsArray()\n",
    "band4 = (gtif.GetRasterBand(4)).ReadAsArray()\n",
    "band5 = (gtif.GetRasterBand(5)).ReadAsArray()\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "count = 0\n",
    "matrix1 = band1.flatten()\n",
    "matrix2 = band2.flatten()\n",
    "matrix3 = band3.flatten()\n",
    "matrix4 = band4.flatten()\n",
    "matrix5 = band5.flatten()\n",
    "for i in enumerate([band1,band2,band3,band4,band5],1):\n",
    "    print('the shape of the',i[0],'matrix is', i[1].shape)\n",
    "    \n",
    "for i in enumerate([band1,band2,band3,band4,band5],1):\n",
    "    print('the max of the' ,i[0],' matrix is', np.max(i[1]))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p></p>\n",
    "<p></p>\n",
    "<p>Since the data size is too large, we clear our memory after loading the raw data </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "band1 = None\n",
    "band2 = None\n",
    "band3 = None\n",
    "band4 = None\n",
    "band5 = None\n",
    "gtif1 = None\n",
    "srcband = None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> We have plotted the histograms of our 5 bands below </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa90cb6fc50>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEICAYAAAB/Dx7IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADztJREFUeJzt3X+MHPV9xvHnqe2YBJxiwom62OWAtokISoxzIWmDaEt/\nYGxa+kcrmf5Q2qK6bZyKRIlaI6oqkYJEUrVBLU0Tp3VMmoSfgSoiSopbiEhEsHuGM7FNDcY4ApfG\nR40LqBItzqd/zPdgfezezZ1ndvfjvF/S6mZnZmefndt9PDs/fI4IAQDy+KFBBwAAzA3FDQDJUNwA\nkAzFDQDJUNwAkAzFDQDJUNxIzfYB27/QZfzP2n56EJmAtlHcAJAMxQ0AyVDcOBG80/Ye28/Z/pzt\nk6bPYDts/3jH/S22P9Zx/3LbE7aP2H7A9tv6FR6YK4obJ4LflHSppHMl/aSkP5vLg21fIGmzpD+Q\n9CZJn5H0FduLG84JNKK14ra92fYh27tqzPvJsrUzYfsx20fayoUT0o0R8VREHJZ0naQr5/j49ZI+\nExHbIuJoRNwk6SVJ7246KNCENre4t0haXWfGiPhgRKyMiJWS/kbSnS3mwonnqY7h70r60Tk+/ixJ\nHyq7SY6UDYcV81gO0BetFXdE3C/pcOc42+fa/rrtHba/afstXR56paSb28qFE9KKjuEfk/QfXeb5\nH0lv6Lj/Ix3DT0m6LiJO7bi9ISJ4H2Io9Xsf9yZJfxwR75D0YUmf6pxo+yxJZ0u6t8+5kNsG28tt\nnybpWkm3dplnQtJv2F5ge7Wkn+mY9llJf2j7Xa6cbHut7SV9yA7MWd+K2/Ypkn5a0u22J1QdAFo2\nbbZ1ku6IiKP9yoUTwpck3SNpv6QnJH2syzxXS/plSUdUHcz8p6kJETEu6fcl3SjpOUn7JP1Oq4mB\n4+A2/5CC7VFJd0fE+bbfKGlvREwv6875H5a0ISIeaC0UACTXty3uiHhe0pO2f12SylfSt09NL/u7\nl0r6dr8yAUBGbZ4OeLOqEn6z7adtX6XqK+pVtndK2i3pio6HrJN0S/C31ABgRq3uKgEANI8rJwEg\nmYVtLPT000+P0dHRNhYNACekHTt2PBsRI3XmbaW4R0dHNT4+3saiAeCEZPu7dedlVwkAJENxA0Ay\nFDcAJENxA0AyFDcAJENxA0AyFDcAJENxA0AyFDcAJNPKlZMZjW786sCe+8D1awf23ADyYYsbAJKh\nuAEgGYobAJKhuAEgGYobAJKhuAEgmVqnA9o+IOkFSUclvRwRY22GAgD0NpfzuH8uIp5tLQkAoBZ2\nlQBAMnWLOyTdY3uH7fXdZrC93va47fHJycnmEgIAjlG3uC+KiFWSLpO0wfbF02eIiE0RMRYRYyMj\ntf5QMQBgHmoVd0QcLD8PSbpL0oVthgIA9DZrcds+2faSqWFJvyRpV9vBAADd1Tmr5AxJd9memv9L\nEfH1VlMBAHqatbgjYr+kt/chCwCgBk4HBIBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4A\nSIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbi\nBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASKZ2cdteYPth23e3GQgA\nMLO5bHFfLenRtoIAAOqpVdy2l0taK+nv240DAJhN3S3uGyT9iaTv95rB9nrb47bHJycnGwkHAHit\nWYvb9uWSDkXEjpnmi4hNETEWEWMjIyONBQQAHKvOFvd7JP2K7QOSbpF0ie0vtJoKANDTrMUdEddE\nxPKIGJW0TtK9EfFbrScDAHTFedwAkMzCucwcEd+Q9I1WkgAAamGLGwCSobgBIBmKGwCSobgBIBmK\nGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCS\nobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgB\nIJlZi9v2Sba3295pe7ftj/YjGACgu4U15nlJ0iUR8aLtRZK+ZftrEfFgy9kAAF3MWtwREZJeLHcX\nlVu0GQoA0Futfdy2F9iekHRI0taI2NZuLABAL7WKOyKORsRKScslXWj7/Onz2F5ve9z2+OTkZNM5\nAQDFnM4qiYgjku6TtLrLtE0RMRYRYyMjI03lAwBMU+eskhHbp5bh10v6RUn/3nYwAEB3dc4qWSbp\nJtsLVBX9bRFxd7uxAAC91Dmr5BFJF/QhCwCgBq6cBIBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbi\nBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBk\nKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkZi1u\n2yts32d7j+3dtq/uRzAAQHcLa8zzsqQPRcRDtpdI2mF7a0TsaTkbAKCLWbe4I+KZiHioDL8g6VFJ\nZ7YdDADQ3Zz2cdselXSBpG1dpq23PW57fHJyspl0AIDXqF3ctk+R9GVJH4iI56dPj4hNETEWEWMj\nIyNNZgQAdKhV3LYXqSrtL0bEne1GAgDMpM5ZJZb0D5IejYi/aj8SAGAmdba43yPptyVdYnui3Na0\nnAsA0MOspwNGxLckuQ9ZAAA1cOUkACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3\nACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRD\ncQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMrMWt+3Ntg/Z3tWPQACAmdXZ\n4t4iaXXLOQAANc1a3BFxv6TDfcgCAKihsX3cttfbHrc9Pjk52dRiAQDTNFbcEbEpIsYiYmxkZKSp\nxQIApuGsEgBIhuIGgGTqnA54s6RvS3qz7adtX9V+LABALwtnmyEiruxHEABAPewqAYBkKG4ASIbi\nBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBk\nKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkKG4ASIbiBoBkFg46\nwHSjG7866AgAMNSGrrh/EA3qH6sD168dyPMCOD7sKgGAZChuAEimVnHbXm17r+19tje2HQoA0Nus\nxW17gaS/lXSZpPMkXWn7vLaDAQC6q3Nw8kJJ+yJivyTZvkXSFZL2tBkM7ftBPIOHA7I4EdQp7jMl\nPdVx/2lJ75o+k+31ktaXuy/a3jvPTKdLenaej+2nLDklsr7CH29sUazT5mXJKbWT9ay6MzZ2OmBE\nbJK06XiXY3s8IsYaiNSqLDklsrYhS04pT9YsOaXBZ61zcPKgpBUd95eXcQCAAahT3P8m6Sdsn237\ndZLWSfpKu7EAAL3MuqskIl62/X5J/yxpgaTNEbG7xUzHvbulT7LklMjahiw5pTxZs+SUBpzVETHI\n5wcAzBFXTgJAMhQ3ACQzNMU9LJfV2z5g+zu2J2yPl3Gn2d5q+/Hyc2kZb9t/XTI/YntVx3LeW+Z/\n3PZ7G8i12fYh27s6xjWWy/Y7yuveVx7rhrN+xPbBsl4nbK/pmHZNed69ti/tGN/1PVEOlG8r428t\nB83nk3OF7fts77G92/bVZfzQrdcZsg7jej3J9nbbO0vWj860fNuLy/19ZfrofF9DQzm32H6yY52u\nLOMH+rk6RkQM/KbqoOcTks6R9DpJOyWdN6AsBySdPm3cJyRtLMMbJX28DK+R9DVJlvRuSdvK+NMk\n7S8/l5bhpceZ62JJqyTtaiOXpO1lXpfHXtZw1o9I+nCXec8rv+/Fks4u74MFM70nJN0maV0Z/rSk\nP5pnzmWSVpXhJZIeK3mGbr3OkHUY16slnVKGF0naVtZB1+VLep+kT5fhdZJune9raCjnFkm/1mX+\ngX6uOm/DssX9ymX1EfG/kqYuqx8WV0i6qQzfJOlXO8Z/PioPSjrV9jJJl0raGhGHI+I5SVslrT6e\nABFxv6TDbeQq094YEQ9G9W77fMeymsrayxWSbomIlyLiSUn7VL0fur4nyhbLJZLu6PK655rzmYh4\nqAy/IOlRVVcKD916nSFrL4NcrxERL5a7i8otZlh+5/q+Q9LPlzxzeg0N5uxloJ+rTsNS3N0uq5/p\nTdmmkHSP7R2uLuOXpDMi4pky/J+SzijDvXL36/U0levMMtx23veXr5ibp3Y/zCPrmyQdiYiXm8xa\nvp5foGqra6jX67Ss0hCuV9sLbE9IOqSqyJ6YYfmvZCrT/7vkaf3zNT1nREyt0+vKOv2k7cXTc9bM\n09rnaliKe5hcFBGrVP1viBtsX9w5sfzLOXTnUA5rrg5/J+lcSSslPSPpLwcb51W2T5H0ZUkfiIjn\nO6cN23rtknUo12tEHI2IlaqutL5Q0lsGHKmr6Tltny/pGlV536lq98efDjBiV8NS3ENzWX1EHCw/\nD0m6S9Wb7nvla4/Kz0Nl9l65+/V6msp1sAy3ljcivlc+JN+X9FlV63U+Wf9L1VfUhdPGz4vtRaqK\n8IsRcWcZPZTrtVvWYV2vUyLiiKT7JP3UDMt/JVOZ/sMlT98+Xx05V5fdUhERL0n6nOa/Ttv7XDWx\no/x4b6qu4Nyv6gDE1MGGtw4gx8mSlnQMP6Bq3/Rf6NiDVZ8ow2t17MGK7fHqwYonVR2oWFqGT2sg\n36iOPeDXWC699iDKmoazLusY/qCqfZeS9FYdewBqv6qDTz3fE5Ju17EHud43z4xWtd/xhmnjh269\nzpB1GNfriKRTy/DrJX1T0uW9li9pg449OHnbfF9DQzmXdazzGyRdP+jf/2uyN7GQRoJUR2wfU7Uv\n7NoBZTinvAl2Sto9lUPV/rZ/lfS4pH/p+KVY1R+ZeELSdySNdSzr91QdTNkn6XcbyHazqq/C/6dq\nX9lVTeaSNCZpV3nMjSpX1TaY9R9LlkdU/V83nYVzbXneveo46t7rPVF+T9vLa7hd0uJ55rxI1W6Q\nRyRNlNuaYVyvM2QdxvX6NkkPl0y7JP35TMuXdFK5v69MP2e+r6GhnPeWdbpL0hf06pknA/1cdd64\n5B0AkhmWfdwAgJoobgBIhuIGgGQobgBIhuIGgGQobgBIhuIGgGT+HxZcSadX2+nkAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa90efdfd30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEICAYAAAB/Dx7IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD2NJREFUeJzt3X+sZHV9xvH34y4/rKCA3CgCcsE0GiQWcEVbjW0wjbgQ\naRubrKkNbUm3rWLBaCrGpNWmJmjTaoxtdbUIqOE3JhSrLa0YNerSXVlwF4osLI0SdJfiFkhbFPz0\njzmLs5f7Y/bunDvz3b5fyc09c+bcM885d+bZM98z526qCklSO54x6QCSpH1jcUtSYyxuSWqMxS1J\njbG4JakxFrckNcbilqTGWNyS1BiLWwekJKsnnUHqi8WtpiQ5PcltSR5Ncm2Sq5P8RZJfSfL9JO9O\n8gPg093y5yTZkmR3km8kednQul6Q5Poku5LsSPLHQ/e9L8k1Sa7oHmtbkjUT2GTpaSxuNSPJwcDn\ngcuAo4ArgV8fWuT53fwTgPVJTgMuBf4AeC7wCeDGJIckeQbwD8DtwLHA64CLkrx+aH1vBK4CjgBu\nBD7W28ZJ+6C34k5yaZKdSbaOsOyHu6OiLUm+m2R3X7nUtFcBq4GPVtVPquoG4Nah+38K/FlVPV5V\n/wOsBz5RVRur6smquhx4vFvPK4CZqvrzqvpxVd0HfBJYN7S+r1fVP1bVk8BngF/ofxOlpfU5DngZ\ngyOUK5ZasKresWc6yduB0/qLpYa9AHig9v7LaN8bmt5VVf87dPsE4LzuObXHwd16ngReMOcgYRXw\ntaHbPxia/m/g0CSrq+qJ/dkIaX/1dsRdVV8FHh6el+RFSb6UZHOSryV5yTw/+mYGb4GluR4Ejk2S\noXnHD03P/VOX3wM+UFVHDH39XFVd2d23Y859h1fV2p63QdpvKz3GvQF4e1W9HHgX8LfDdyY5ATgR\n+PIK51IbvsngSPmCJKuTnAucscjynwT+MMkrM/CsJGcnOZzBEMuj3cnMZyZZleSUJK9Yge2Q9suK\nFXeSw4BfAq5NsoXBiaJj5iy2DriuG1OU9lJVPwZ+Azgf2A28BbiJwbj1fMtvAn6fwZDdj4DtwO90\n9z0JnAOcCuwAHgI+BTynz22QxiF9/kcKSWaBm6rqlCTPBu6uqrllPbz8bcDbquobvYXSASXJRuDj\nVfXpSWeRVsqKHXFX1SPAjiS/CdC9dX3qLH033n0kg7fD0ryS/HKS53dDJecBLwO+NOlc0krq8+OA\nVzIo4Rd3F0acD/wWcH6S24FtwLlDP7IOuKr6fAugA8GLGXz2ejfwTuBNVfXgZCNJK6vXoRJJ0vh5\n5aQkNaaXC3COPvromp2d7WPVknRA2rx580NVNTPKsr0U9+zsLJs2bepj1ZJ0QEryH6Mu61CJJDXG\n4pakxljcktQYi1uSGmNxS1JjLG5JaozFLUmNsbglqTEWtyQ1ps//c7Ipsxd/YWKPff8lZ0/ssSW1\nxyNuSWqMxS1JjbG4JakxFrckNcbilqTGWNyS1BiLW5IaY3FLUmMsbklqjMUtSY2xuCWpMRa3JDXG\n4pakxljcktQYi1uSGmNxS1JjLG5JaozFLUmNGbm4k6xKcluSm/oMJEla3L4ccV8I3NVXEEnSaEYq\n7iTHAWcDn+o3jiRpKaMecX8E+BPgpwstkGR9kk1JNu3atWss4SRJT7dkcSc5B9hZVZsXW66qNlTV\nmqpaMzMzM7aAkqS9jXLE/WrgjUnuB64Czkzy2V5TSZIWtGRxV9V7quq4qpoF1gFfrqq39J5MkjQv\nP8ctSY1ZvS8LV9VXgK/0kkSSNBKPuCWpMRa3JDXG4pakxljcktQYi1uSGmNxS1JjLG5JaozFLUmN\nsbglqTEWtyQ1xuKWpMZY3JLUGItbkhpjcUtSYyxuSWqMxS1JjbG4JakxFrckNcbilqTGWNyS1BiL\nW5IaY3FLUmMsbklqjMUtSY2xuCWpMRa3JDXG4pakxljcktQYi1uSGmNxS1JjLG5JaozFLUmNsbgl\nqTEWtyQ1xuKWpMZY3JLUGItbkhqzZHEnOTTJrUluT7ItyftXIpgkaX6rR1jmceDMqnosyUHA15N8\nsaq+1XM2SdI8lizuqirgse7mQd1X9RlKkrSwkca4k6xKsgXYCdxcVRvnWWZ9kk1JNu3atWvcOSVJ\nnZGKu6qerKpTgeOAM5KcMs8yG6pqTVWtmZmZGXdOSVJnnz5VUlW7gVuAs/qJI0layiifKplJckQ3\n/UzgV4F/7zuYJGl+o3yq5Bjg8iSrGBT9NVV1U7+xJEkLGeVTJXcAp61AFknSCLxyUpIaY3FLUmMs\nbklqjMUtSY2xuCWpMRa3JDXG4pakxljcktQYi1uSGmNxS1JjLG5JaozFLUmNsbglqTEWtyQ1xuKW\npMZY3JLUGItbkhpjcUtSYyxuSWqMxS1JjbG4JakxFrckNcbilqTGWNyS1BiLW5IaY3FLUmMsbklq\njMUtSY2xuCWpMRa3JDXG4pakxljcktQYi1uSGmNxS1JjLG5JaozFLUmNsbglqTFLFneS45PckuTO\nJNuSXLgSwSRJ81s9wjJPAO+sqm8nORzYnOTmqrqz52ySpHksecRdVQ9W1be76UeBu4Bj+w4mSZrf\nPo1xJ5kFTgM2znPf+iSbkmzatWvXeNJJkp5m5OJOchhwPXBRVT0y9/6q2lBVa6pqzczMzDgzSpKG\njFTcSQ5iUNqfq6ob+o0kSVrMKJ8qCfD3wF1V9df9R5IkLWaUI+5XA78NnJlkS/e1tudckqQFLPlx\nwKr6OpAVyCJJGoFXTkpSYyxuSWqMxS1JjbG4JakxFrckNcbilqTGWNyS1BiLW5IaY3FLUmMsbklq\njMUtSY2xuCWpMRa3JDXG4pakxljcktQYi1uSGmNxS1JjLG5JaozFLUmNsbglqTEWtyQ1xuKWpMZY\n3JLUGItbkhpjcUtSYyxuSWqMxS1JjbG4JakxFrckNcbilqTGWNyS1BiLW5IaY3FLUmMsbklqjMUt\nSY2xuCWpMRa3JDVmyeJOcmmSnUm2rkQgSdLiRjnivgw4q+cckqQRLVncVfVV4OEVyCJJGoFj3JLU\nmNXjWlGS9cB6gBe+8IXLXs/sxV8YVyRJOiCN7Yi7qjZU1ZqqWjMzMzOu1UqS5nCoRJIaM8rHAa8E\nvgm8OMn3k5zffyxJ0kKWHOOuqjevRBBJ0mgcKpGkxljcktQYi1uSGmNxS1JjLG5JaozFLUmNsbgl\nqTEWtyQ1xuKWpMZY3JLUGItbkhpjcUtSYyxuSWqMxS1JjbG4JakxFrckNcbilqTGWNyS1BiLW5Ia\nY3FLUmMsbklqjMUtSY2xuCWpMRa3JDXG4pakxljcktQYi1uSGmNxS1JjLG5JaozFLUmNsbglqTEW\ntyQ1xuKWpMZY3JLUGItbkhpjcUtSY1ZPOoBg9uIvTORx77/k7Ik8rqT94xG3JDVmpOJOclaSu5Ns\nT3Jx36EkSQtbsriTrAL+BngDcDLw5iQn9x1MkjS/Uca4zwC2V9V9AEmuAs4F7uwzmPo3qbH1SXJc\nXweCUYr7WOB7Q7e/D7xy7kJJ1gPru5uPJbl7mZmOBh5a5s+uNLP2o7es+eDYV+l+7cf/x6wnjLrg\n2D5VUlUbgA37u54km6pqzRgi9c6s/TBrP8zaj0lkHeXk5APA8UO3j+vmSZImYJTi/jfg55OcmORg\nYB1wY7+xJEkLWXKopKqeSHIB8E/AKuDSqtrWY6b9Hm5ZQWbth1n7YdZ+rHjWVNVKP6YkaT945aQk\nNcbilqTGTE1xT8tl9UnuT/KdJFuSbOrmHZXk5iT3dN+P7OYnyUe7zHckOX1oPed1y9+T5LwxZbs0\nyc4kW4fmjS1bkpd32769+9mMOev7kjzQ7dstSdYO3fee7nHvTvL6ofnzPi+6k+Ubu/lXdyfOl5v1\n+CS3JLkzybYkF3bzp27fLpJ16vZtkkOT3Jrk9i7r+xdbf5JDutvbu/tnl7sNY8x6WZIdQ/v11G7+\nRF9fVNXEvxic9LwXOAk4GLgdOHlCWe4Hjp4z70PAxd30xcAHu+m1wBeBAK8CNnbzjwLu674f2U0f\nOYZsrwVOB7b2kQ24tVs23c++YcxZ3we8a55lT+5+54cAJ3bPhVWLPS+Aa4B13fTHgT/aj6zHAKd3\n04cD3+0yTd2+XSTr1O3bblsP66YPAjZ2+2De9QNvBT7eTa8Drl7uNowx62XAm+ZZfqKvr2k54n7q\nsvqq+jGw57L6aXEucHk3fTnwa0Pzr6iBbwFHJDkGeD1wc1U9XFU/Am4GztrfEFX1VeDhPrJ19z27\nqr5Vg2fZFUPrGlfWhZwLXFVVj1fVDmA7g+fEvM+L7kjlTOC6ebZ7OVkfrKpvd9OPAncxuGJ46vbt\nIlkXMrF92+2fx7qbB3Vftcj6h/f3dcDrujz7tA1jzrqQib6+pqW457usfrEnY58K+OckmzO4jB/g\neVX1YDf9A+B53fRCuVdye8aV7dhueu78cbuge2t56Z6hh2VkfS6wu6qeGHfW7u35aQyOuKZ6387J\nClO4b5OsSrIF2MmgxO5dZP1PZeru/68uz4q8zuZmrao9+/UD3X79cJJD5mYdMdNYnwPTUtzT5DVV\ndTqDv4b4tiSvHb6z+9dyKj9DOc3ZOn8HvAg4FXgQ+KvJxtlbksOA64GLquqR4fumbd/Ok3Uq921V\nPVlVpzK44voM4CUTjrSguVmTnAK8h0HmVzAY/nj3BCM+ZVqKe2ouq6+qB7rvO4HPM3iy/bB7q0P3\nfWe3+EK5V3J7xpXtgW66t8xV9cPuxfFT4JMM9u1ysv4ng7emq+fMX7YkBzEows9V1Q3d7Knct/Nl\nneZ92+XbDdwC/OIi638qU3f/c7o8K/o6G8p6Vjc0VVX1OPBplr9fx/v6Wu7g+Di/GFzBeR+DEw97\nTjK8dAI5ngUcPjT9DQZj03/J3iepPtRNn83eJyhurZ+doNjB4OTEkd30UWPKOMveJ/zGlo2nnzxZ\nO+asxwxNv4PBuCXAS9n75NN9DE48Lfi8AK5l7xNcb92PnGEw5viROfOnbt8uknXq9i0wAxzRTT8T\n+BpwzkLrB97G3icnr1nuNowx6zFD+/0jwCWTfg5U1XQUd7dRaxmcIb8XeO+EMpzU/fJvB7btycFg\nnO1fgXuAfxn6RYTBfzJxL/AdYM3Qun6PwUmU7cDvjinflQzeBv+EwRjZ+ePMBqwBtnY/8zG6K2vH\nmPUzXZY7GPy9m+GyeW/3uHczdLZ9oedF97u6tduGa4FD9iPraxgMg9wBbOm+1k7jvl0k69TtW+Bl\nwG1dpq3Any62fuDQ7vb27v6TlrsNY8z65W6/bgU+y88+eTLR15eXvEtSY6ZljFuSNCKLW5IaY3FL\nUmMsbklqjMUtSY2xuCWpMRa3JDXm/wCxwW9Z4CHwSQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa967390390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEICAYAAAB/Dx7IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADqNJREFUeJzt3X+MHPV9xvHn6dkBBE4w9Qo5mHCAqkQENeBc6Y8g1FK1\nARuV/NPKqImiBslSQyiJGrVGkdpQqRJJ1TZCaZs6qQUkKYQfQY1AJDgNEUQJds5gO7apg8GuwHJz\nRx0K9I+kkE//mO/R9bE/5o6dnftc3y9pdbOz39159mbv8dz88DkiBADI4+faDgAAWBiKGwCSobgB\nIBmKGwCSobgBIBmKGwCSobgBSbZ/3fZzbecA6qC4ASAZihvLku0VbWcAmkJxY9mwfcT2n9reK+m/\nbb/N9r22Z20ftv1HXWNPsX2r7R/bPiDpl9pLDiwMWyVYbq6RtFHScUmPSPqXMm+dpG/aPhgR35D0\n55LOL7dTJT3YTlxg4Rrb4ra9zfaM7X01xv6t7d3l9kPbLzSVC8veLRHxrKQLJXUi4i8i4qcR8Yyk\nz0vaVMb9nqS/jIjjZfwtLeUFFqzJLe5bJX1W0u3DBkbEx+ambV8v6eLmYmGZe7Z8PUfSW+dtBExI\nerRMv7VrrCT9+xiyASPR2BZ3RDyi6tfV19g+3/bXbe+y/ajtd/R46jWS7mgqF5a9uf/u8llJhyPi\n9K7bqojYUB4/Junsrue9bawpgTdg3Acnt0q6PiLeLenjkv6++0Hb50g6V9K3xpwLy89OSS+Vg5Wn\n2J6wfaHtuYOQd0m60fZq2+skXd9eVGBhxlbctk+T9GuS7ra9W9I/Slo7b9gmSfdExKvjyoXlqXyG\nrpJ0kaTDkp6X9AVJbylDblK1e+SwpIckfbGFmMCiuMk/pGB7UtL9EXGh7TdLOhgR88u6e/wTkq6L\niO82FgoAkhvbFndEvCjpsO3flSRX3jX3eNnfvVrS98aVCQAyavJ0wDtUlfDbbT9n+1pJvy/pWtt7\nJO2XdHXXUzZJujP4W2oAMFCju0oAAKPHJe8AkEwjF+CsWbMmJicnm3hpAFiWdu3a9XxEdOqMrVXc\nto9IeknSq5JeiYipQeMnJyc1PT1d56UBAJJs1756dyFb3L8REc8vIg8AYITYxw0AydQt7pD0UPk/\nRjb3GmB7s+1p29Ozs7OjSwgAOEHd4r40ItZLulLSdbYvmz8gIrZGxFRETHU6tfavAwAWoVZxR8TR\n8nVG0n2SLmkyFACgv6HFbftU26vmpiX9tqShfxwBANCMOmeVnCnpPttz4/85Ir7eaCoAQF9Di7v8\nyad3DRsHABgPTgcEgGSW3F95n9zyQCvLPXLzxlaWCwALxRY3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRD\ncQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNA\nMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3ACRDcQNAMhQ3\nACRDcQNAMrWL2/aE7Sds399kIADAYAvZ4r5B0pNNBQEA1FOruG2vk7RR0heajQMAGKbuFvdnJP2J\npJ/1G2B7s+1p29Ozs7MjCQcAeL2hxW37KkkzEbFr0LiI2BoRUxEx1el0RhYQAHCiOlvc75H0O7aP\nSLpT0uW2v9RoKgBAX0OLOyJujIh1ETEpaZOkb0XE+xtPBgDoifO4ASCZFQsZHBHflvTtRpIAAGph\nixsAkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsA\nkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsAkqG4\nASAZihsAkqG4ASAZihsAkqG4ASAZihsAkqG4ASAZihsAkhla3LZPtr3T9h7b+23fNI5gAIDeVtQY\n8xNJl0fEy7ZXSvqO7Qcj4rGGswEAehha3BERkl4ud1eWWzQZCgDQX6193LYnbO+WNCNpe0TsaDYW\nAKCfWsUdEa9GxEWS1km6xPaF88fY3mx72vb07OzsqHMCAIoFnVUSES9IeljSFT0e2xoRUxEx1el0\nRpUPADBPnbNKOrZPL9OnSPotSf/WdDAAQG91zipZK+k22xOqiv6uiLi/2VgAgH7qnFWyV9LFY8gC\nAKiBKycBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCS\nobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgB\nIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSobgBIBmKGwCSGVrcts+2/bDtA7b3275h\nHMEAAL2tqDHmFUl/HBGP214laZft7RFxoOFsAIAehm5xR8SxiHi8TL8k6UlJZzUdDADQ24L2cdue\nlHSxpB09Httse9r29Ozs7GjSAQBep3Zx2z5N0r2SPhoRL85/PCK2RsRUREx1Op1RZgQAdKlV3LZX\nqirtL0fEV5uNBAAYpM5ZJZb0T5KejIi/aT4SAGCQOlvc75H0AUmX295dbhsazgUA6GPo6YAR8R1J\nHkMWAEANXDkJAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ\n3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQ\nDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQDMUNAMlQ3ACQzNDitr3N9oztfeMI\nBAAYrM4W962Srmg4BwCgpqHFHRGPSDo+hiwAgBpGto/b9mbb07anZ2dnR/WyAIB5RlbcEbE1IqYi\nYqrT6YzqZQEA83BWCQAkQ3EDQDJ1Tge8Q9L3JL3d9nO2r20+FgCgnxXDBkTENeMIAgCoh10lAJAM\nxQ0AyVDcAJAMxQ0AyVDcAJAMxQ0AyVDcAJAMxQ0AyVDcAJDM0Csn/7+Y3PJAa8s+cvPG1pYNIB+2\nuAEgGYobAJKhuAEgGYobAJKhuAEgGYobAJKhuAEgGYobAJKhuAEgGYobAJKhuAEgGYobAJKhuAEg\nGYobAJKhuAEgGYobAJKhuAEgGYobAJKhuAEgGYobAJKhuAEgGYobAJKhuAEgmRVtB4A0ueWBVpZ7\n5OaNrSwXwBtTa4vb9hW2D9o+ZHtL06EAAP0NLW7bE5L+TtKVki6QdI3tC5oOBgDorc6ukkskHYqI\nZyTJ9p2SrpZ0oMlgaF5bu2jaxO4hLAd1ivssSc923X9O0i/PH2R7s6TN5e7Ltg8uMtMaSc8v8rlL\nAfnbNTC/PzXGJIuzrL//CbSZ/5y6A0d2cDIitkra+kZfx/Z0REyNIFIryN8u8reL/ONR5+DkUUln\nd91fV+YBAFpQp7i/L+kXbJ9r+02SNkn6WrOxAAD9DN1VEhGv2P6IpG9ImpC0LSL2N5jpDe9uaRn5\n20X+dpF/DBwRbWcAACwAl7wDQDIUNwAks2SKeylfVm/7iO0f2N5te7rMO8P2dttPla+ry3zbvqW8\nj72213e9zgfL+Kdsf7DBvNtsz9je1zVvZHltv7t8Pw6V53oM+T9p+2hZB7ttb+h67MaS5aDt93bN\n7/mZKgfad5T5XykH3UeZ/2zbD9s+YHu/7RvK/BTrYED+FOvA9sm2d9reU/LfNGiZtk8q9w+VxycX\n+77GJiJav6k66Pm0pPMkvUnSHkkXtJ2rK98RSWvmzfu0pC1leoukT5XpDZIelGRJvyJpR5l/hqRn\nytfVZXp1Q3kvk7Re0r4m8kraWca6PPfKMeT/pKSP9xh7Qfm8nCTp3PI5mhj0mZJ0l6RNZfpzkv5w\nxPnXSlpfpldJ+mHJmWIdDMifYh2U78lpZXqlpB3le9VzmZI+LOlzZXqTpK8s9n2N67ZUtrhfu6w+\nIn4qae6y+qXsakm3lenbJL2va/7tUXlM0um210p6r6TtEXE8In4sabukK5oIFhGPSDreRN7y2Jsj\n4rGoPt23d71Wk/n7uVrSnRHxk4g4LOmQqs9Tz89U2TK9XNI95fnd34tR5T8WEY+X6ZckPanqCuQU\n62BA/n6W1Doo38eXy92V5RYDltm9Xu6R9Jsl44Le16jy17FUirvXZfWDPijjFpIesr3L1aX9knRm\nRBwr0/8h6cwy3e+9tP0eR5X3rDI9f/44fKTsStg2t5tBC8//85JeiIhX5s1vRPm1+2JVW33p1sG8\n/FKSdWB7wvZuSTOq/sF7esAyX8tZHv+vknGp/iwvmeJe6i6NiPWq/ofE62xf1v1g2epJc15ltrzF\nP0g6X9JFko5J+ut24wxn+zRJ90r6aES82P1YhnXQI3+adRARr0bERaqu9L5E0jtajjRSS6W4l/Rl\n9RFxtHydkXSfqg/Cj8qvrCpfZ8rwfu+l7fc4qrxHy/T8+Y2KiB+VH8afSfq8qnWgITl7zf9PVbsi\nVsybP1K2V6oqvS9HxFfL7DTroFf+bOugZH5B0sOSfnXAMl/LWR5/S8m4VH+Wl8zByRWqDrycq//b\n2f/OtnOVbKdKWtU1/V1V+6b/SiceaPp0md6oEw807Szzz5B0WNVBptVl+owGc0/qxIN7I8ur1x8Y\n2zCG/Gu7pj+mat+jJL1TJx5AekbVwaO+nylJd+vEg1QfHnF2q9rv/Jl581OsgwH5U6wDSR1Jp5fp\nUyQ9KumqfsuUdJ1OPDh512Lf17huY1tQjW/2BlVHr5+W9Im283TlOq+smD2S9s9lU7UP7F8lPSXp\nm10/UFb1hyeelvQDSVNdr/UhVQc4Dkn6gwYz36HqV9n/UbX/7dpR5pU0JWlfec5nVa7AbTj/F0u+\nvar+r5zuEvlEyXJQXWdX9PtMlXW6s7yvuyWdNOL8l6raDbJX0u5y25BlHQzIn2IdSPpFSU+UnPsk\n/dmgZUo6udw/VB4/b7Hva1w3LnkHgGSWyj5uAEBNFDcAJENxA0AyFDcAJENxA0AyFDcAJENxA0Ay\n/ws2aZxhNctknAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa90ccaa5c0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEICAYAAAB/Dx7IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD71JREFUeJzt3X2MZXV9x/H3x12kFlBAphQBWSRWS20EHB9aCWlpeBBo\nsQ+aNVZppd20QYtNTQs1abWJrZpoLVGrqyJoEaoCKYVY3VbUmgo4qwvusq48bQN0dQdRF22rXfz2\nj3umuU7n4e5y79z5zbxfyc2cOed3z3y/99z97LnnYSZVhSSpHY8bdwGSpH1jcEtSYwxuSWqMwS1J\njTG4JakxBrckNcbg1oqQ5BeSPDCkdf1Wks8PY13SKBjcktQYg1vLTpK1465BWs4Mbi0LSXYm+ZMk\ndwDfS7I2yVOSXJtkOsl9Sf6gb/wTklyR5FtJ7gSeu8j6n5lkU5KHk+xI8tK+ZU9OckOSPUluA06Y\n9dwzu+d8J8m7k3w2ye/0LX9Vku1dLZ9MctzQXhhpDga3lpOXAecChwI/BP4RuB04Gvgl4LVJzurG\n/jm9gD0BOAu4YL6VJjkI2AR8BPgJYD3w7iQndkPeBfw3cBTwqu4x89wjgI8DlwJPBnYAP9+3/Hzg\nT4FfAyaAfwWu3s/+pYGMLLiTXJ5kd5KtA4z96yRbusfXknx7VHVpWbusqu6vqv+itwc9UVV/UVU/\nqKp7gffRC12AlwJvqqqHq+p+4LIF1nsesLOqPlhVe6vqy8C1wEuSrAF+HfizqvpeVW0Frux77jnA\ntqq6rqr2dj/n633Lfw/4q6ra3i3/S+Ak97o1SqM8lngF8E7gQ4sNrKo/nJlO8hrg5NGVpWXs/r7p\n44CnzPpPfA29PVqAp8wa/+8LrPc44Pmz1rUW+DC9veS1C6zrR35OVdWsq1eOA/4mydv65oXep4SF\napL228iCu6o+l2Rd/7wkJ9D7WDoB/Cfwu1X11VlPfRm9j8Faffp/VeX9wH1V9fR5xu4CjgW2dd8/\ndYH13g98tqrOmL2g2+Pe261r5r3Yv65dwDF949P/fbfuN1XVVQv8fGmolvoY90bgNVX1HOB1wLv7\nF3YfL48HPr3EdWn5uQ14pDth+YQka5I8K8nMSciPApcmOSzJMcBrFljXjcBPJXlFkgO6x3OT/HRV\nPQpcB7whyY93x737j5ffBPxskhd3V7tcBPxk3/L3dHX8DECSJyV5yVBeAWkeSxbcSQ6md1LnY0m2\nAO+ldzKo33rg490/Jq1i3XvgPOAk4D7gIeD9wJO6IW+kdyjiPuBT9A57zLeuR4Az6b2//oPeMeq3\nAAd2Q14NHNzNvwL4YN9zHwJeArwV+CZwIjAFfL9bfn23rmuS7AG2Ai96DK1Li8oo/5BCd6jkxqp6\nVpInAjuqanZY94//MnBRVf3byIqSHoMkjwMeAF5eVTePux6tTku2x11Ve4D7Zj5GpufZM8uTPBM4\nDPjCUtUkDSLJWUkOTXIgvUv/Atwy5rK0io3ycsCr6YXwM5I8kORC4OXAhUlup3dS6fy+p6wHrin/\nlpqWn58D7qF3uOaXgRd3lyxKYzHSQyWSpOHzzklJasxIruM+4ogjat26daNYtSStSJs3b36oqiYG\nGTuS4F63bh1TU1OjWLUkrUhJBr7T1kMlktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEDXQ6YZCfw\nCPAosLeqJkdZlCRpfvtyHfcvdr/iUpI0Rh4qkaTGDLrHXcCnkhTw3qraOHtAkg3ABoCnPnWhvyK1\nPK275Kax/eydbz53bD9bUnsG3eM+tapOofeXPS5KctrsAVW1saomq2pyYmKg2+0lSfthoOCuqge7\nr7uB64HnjbIoSdL8Fg3uJAclOWRmmt7f7ts66sIkSXMb5Bj3kcD1SWbGf6Sq/mmkVUmS5rVocFfV\nvcCzFxsnSVoaXg4oSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEG\ntyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BL\nUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaszAwZ1kTZIvJ7lxlAVJ\nkha2L3vcFwPbR1WIJGkwAwV3kmOAc4H3j7YcSdJiBt3jfgfwx8APR1iLJGkAiwZ3kvOA3VW1eZFx\nG5JMJZmanp4eWoGSpB81yB73C4FfSbITuAY4PcnfzR5UVRurarKqJicmJoZcpiRpxqLBXVWXVtUx\nVbUOWA98uqp+c+SVSZLm5HXcktSYtfsyuKo+A3xmJJVIkgbiHrckNcbglqTGGNyS1BiDW5IaY3BL\nUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1\nxuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMM\nbklqjMEtSY0xuCWpMQa3JDVm0eBO8mNJbktye5JtSd64FIVJkua2doAx3wdOr6rvJjkA+HyST1TV\nLSOuTZI0h0WDu6oK+G737QHdo0ZZlCRpfgMd406yJskWYDewqapunWPMhiRTSaamp6eHXackqTNQ\ncFfVo1V1EnAM8Lwkz5pjzMaqmqyqyYmJiWHXKUnq7NNVJVX1beBm4OzRlCNJWswgV5VMJDm0m34C\ncAbw1VEXJkma2yBXlRwFXJlkDb2g/2hV3TjasiRJ8xnkqpI7gJOXoBZJ0gC8c1KSGmNwS1JjDG5J\naozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTG\nGNyS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozB\nLUmNMbglqTEGtyQ1xuCWpMYY3JLUmEWDO8mxSW5OcmeSbUkuXorCJElzWzvAmL3AH1XVl5IcAmxO\nsqmq7hxxbZKkOSy6x11Vu6rqS930I8B24OhRFyZJmts+HeNOsg44Gbh1jmUbkkwlmZqenh5OdZKk\n/2fg4E5yMHAt8Nqq2jN7eVVtrKrJqpqcmJgYZo2SpD4DBXeSA+iF9lVVdd1oS5IkLWSQq0oCfADY\nXlVvH31JkqSFDLLH/ULgFcDpSbZ0j3NGXJckaR6LXg5YVZ8HsgS1SJIG4J2TktQYg1uSGmNwS1Jj\nDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbg\nlqTGGNyS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5J\naozBLUmNMbglqTEGtyQ1xuCWpMYsGtxJLk+yO8nWpShIkrSwQfa4rwDOHnEdkqQBLRrcVfU54OEl\nqEWSNIChHeNOsiHJVJKp6enpYa1WkjTL0IK7qjZW1WRVTU5MTAxrtZKkWbyqRJIaY3BLUmMGuRzw\nauALwDOSPJDkwtGXJUmaz9rFBlTVy5aiEEnSYDxUIkmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpj\ncEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3\nJDXG4JakxhjcktQYg1uSGrN23AUI1l1y01h+7s43nzuWnyvpsXGPW5IaY3BLUmMMbklqjMEtSY1Z\ndicnx3WiTpJa4R63JDXG4JakxhjcktQYg1uSGmNwS1JjBgruJGcn2ZHk7iSXjLooSdL8Fr0cMMka\n4F3AGcADwBeT3FBVd466OI3WOC+99PekSPtvkOu4nwfcXVX3AiS5BjgfMLi13/zFWtL+GyS4jwbu\n7/v+AeD5swcl2QBs6L79bpId+1nTEcBD+/ncVq3GnmEMfectS/nT5uS2Xj32tefjBh04tDsnq2oj\nsPGxrifJVFVNDqGkZqzGnmF19r0ae4bV2fcoex7k5OSDwLF93x/TzZMkjcEgwf1F4OlJjk/yeGA9\ncMNoy5IkzWfRQyVVtTfJq4FPAmuAy6tq2whresyHWxq0GnuG1dn3auwZVmffI+s5VTWqdUuSRsA7\nJyWpMQa3JDVm2QT3SrytPsnOJF9JsiXJVDfv8CSbktzVfT2sm58kl3X935HklL71XNCNvyvJBePq\nZy5JLk+yO8nWvnlD6zHJc7rX8O7uuVnaDuc2T99vSPJgt723JDmnb9mlXQ87kpzVN3/O9313McCt\n3fy/7y4MGKskxya5OcmdSbYlubibv2K39wI9j3dbV9XYH/ROet4DPA14PHA7cOK46xpCXzuBI2bN\neytwSTd9CfCWbvoc4BNAgBcAt3bzDwfu7b4e1k0fNu7e+vo5DTgF2DqKHoHburHpnvuicfe8QN9v\nAF43x9gTu/f0gcDx3Xt9zULve+CjwPpu+j3A7y+Dno8CTummDwG+1vW2Yrf3Aj2PdVsvlz3u/7ut\nvqp+AMzcVr8SnQ9c2U1fCby4b/6HqucW4NAkRwFnAZuq6uGq+hawCTh7qYueT1V9Dnh41uyh9Ngt\ne2JV3VK9d/WH+tY1VvP0PZ/zgWuq6vtVdR9wN733/Jzv+24v83Tg493z+1/DsamqXVX1pW76EWA7\nvTurV+z2XqDn+SzJtl4uwT3XbfULvTitKOBTSTan9ysBAI6sql3d9NeBI7vp+V6DFl+bYfV4dDc9\ne/5y9urusMDlM4cM2Pe+nwx8u6r2zpq/bCRZB5wM3Moq2d6zeoYxbuvlEtwr1alVdQrwIuCiJKf1\nL+z2Klb09Zirocc+fwucAJwE7ALeNt5yRiPJwcC1wGurak//spW6vefoeazberkE94q8rb6qHuy+\n7gaup/dx6RvdR0K6r7u74fO9Bi2+NsPq8cFuevb8ZamqvlFVj1bVD4H30dvesO99f5PeYYW1s+aP\nXZID6AXYVVV1XTd7RW/vuXoe97ZeLsG94m6rT3JQkkNmpoEzga30+po5i34B8A/d9A3AK7sz8S8A\nvtN9/PwkcGaSw7qPY2d285azofTYLduT5AXdscBX9q1r2ZkJr86v0tve0Ot7fZIDkxwPPJ3eSbg5\n3/fdXuvNwG90z+9/Dcem2wYfALZX1dv7Fq3Y7T1fz2Pf1uM8YzvrbOw59M7Y3gO8ftz1DKGfp9E7\nc3w7sG2mJ3rHtP4FuAv4Z+Dwbn7o/cGKe4CvAJN963oVvZMcdwO/Pe7eZvV5Nb2Piv9D7/jchcPs\nEZjs/lHcA7yT7m7fcT/m6fvDXV93dP+Aj+ob//quhx30XSkx3/u+e//c1r0eHwMOXAY9n0rvMMgd\nwJbucc5K3t4L9DzWbe0t75LUmOVyqESSNCCDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXmfwEV\nlLlGz6TU/AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa90cc1e6a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEadJREFUeJzt3X2QXXV9x/H3h/DQKiDBZBQBWVRKB60KZABbtExnqjxY\no1O1YVp5KDZTB3xodaZRZwSdOsVOLRaxIg8RUAdQoG0EFGm1g9UCLjQghEECpk1iMOEZBbHBb/+4\nJ/S67maX5G7ubn7v18ydPfec3z3n973n7mfP/Z1z76aqkCRt/3YYdgckSduGgS9JjTDwJakRBr4k\nNcLAl6RGGPiS1AgDX01K8qEkF0yxbZJ8PsnDSW6e7r5N0IeRJJVkx2FsX9uHeB2+tHlJXgtcChxY\nVT8dUh9GgB8CO1XVxmH0QbOfR/ja7g3gqHg/YNVEYf9s19+9Y/B3T9ucLzoNTZJVST6Q5PYkjya5\nPMmv9S1/Y5LlSR5J8t0kr+xbtiTJvUkeT7IiyVv6lp2U5DtJzkryIHDGONs+I8kXu+lNwyUnJvmf\nJA8k+XC37BTgAuA1SX6S5KNJjkqyJslfJbkf+HySuUmuTrKhG/q5Osk+fdv79yQfT/Id4AngJUme\nl+TCJOuSrE3y10nmdO3nJPm7ri/3AccN9tlXixwP1LC9HTga+BnwHeAk4NwkBwNLgT8ARoE/AZYl\nObCqngLuBV4L3A+8DfhikpdV1bpuvYcDlwEvAHaaYl+OBA4EfgO4OclVVXVhkqeBd1bVkQBJjgJe\nCOxJ7+h/B+A5wOe7euZ0fT8HeHPf+t8BHAPcDQT4MrAeeBnwXOBqYDXwOeDPgDcCBwM/Ba6cYg3S\nhIZ6hJ9kaZL1Se6YQtuzuqO95Ul+kOSRbdFHTbuzq+pHVfUQ8FXg1d38xcDnquqmqnq6qi4GngKO\nAKiqr3SP+0VVXQ7cAxzWt94fVdWnq2pjVT05xb58tKqerKrbgNuAV22m7S+A06vqqe4xD1bVlVX1\nRFU9Dnwc+N0xj7moqu7sxuD3BI4F3ldVP62q9cBZwKKu7duBT1XV6u65+Zsp1iBNaNhH+BfROwq6\nZLKGVfUXm6aTvJvekY9mv/v7pp8AXtRN7wec2O3rTXbetDzJCcBfAiPdsl2BeX1tVw+gL7tupu2G\nqvrZpjtJnkMvsI8G5nazd0syp6qeHqdP+9F757EuyaZ5O/S1edGY9v/9LOqQxjXUI/yqugF4qH9e\nkpcm+XqSW5J8O8lvjvPQ4+ldNaHt12rg41W1R9/tOVV1aZL9gPOB04DnV9UewB30hkk2me7Lz8au\n//30hoMOr6rdgdd18yfq02p671jm9dW3e1W9vFu+Dti3r/2LB9d1tWomnrQ9D3h3VR0KfAD4x/6F\n3S/7/sA3h9A3bTvnA3+e5PDuqpbnJjkuyW70xrsL2ACQ5GTgFUPsK8BuwJPAI0n2BE7fXOPuXMM3\ngE8m2T3JDt3BzqZhoC8D70myT5K5wJLp7LzaMKMCP8muwG8DX0mynN7Jq73GNFsEXNH3Nlnboaoa\npXfi8hzgYWAlvRO6VNUK4JPAfwI/Bn6L3gnfYfoU8OvAA8CNwNen8JgT6A1TraBX4xX8/+v9fOA6\neucSbgWuGnB/1aChf/Cq+0DJ1VX1iiS7A3dX1diQ72//X8CpVfXdbdRFSdouzKgj/Kp6DPhhkrfB\nMx9QeeZKiW48fy69IztJ0rMw7MsyL6UX3gd2H2Q5Bfhj4JQktwF3Agv7HrIIuKyG/bZEkmahoQ/p\nSJK2jRk1pCNJmj5D++DVvHnzamRkZFibl6RZ6ZZbbnmgquZvyWOHFvgjIyOMjo4Oa/OSNCsl2eJP\nXTukI0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjRj2/7SVJjSy5JqhbXvV\nmccNbdvSdPEIX5IaYeBLUiMMfElqhIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJ\naoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjZg08JPsm+RbSVYkuTPJ\ne8dpkyRnJ1mZ5PYkh0xPdyVJW2rHKbTZCLy/qm5NshtwS5Lrq2pFX5tjgAO62+HAZ7ufkqQZYtIj\n/KpaV1W3dtOPA3cBe49pthC4pHpuBPZIstfAeytJ2mLPagw/yQhwMHDTmEV7A6v77q/hV/8okGRx\nktEkoxs2bHh2PZUkbZUpB36SXYErgfdV1WNbsrGqOq+qFlTVgvnz52/JKiRJW2hKgZ9kJ3ph/6Wq\numqcJmuBffvu79PNkyTNEFO5SifAhcBdVfX3EzRbBpzQXa1zBPBoVa0bYD8lSVtpKlfp/A7wDuD7\nSZZ38z4EvBigqs4FrgWOBVYCTwAnD76rkqStMWngV9V/AJmkTQGnDqpTkqTB85O2ktQIA1+SGmHg\nS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWrEVL4tU2rOyJJrhrLd\nVWceN5Ttqg0e4UtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5IaYeBLUiMMfElqhIEv\nSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLU\nCANfkhoxaeAnWZpkfZI7Jlh+VJJHkyzvbh8ZfDclSVtrxym0uQg4B7hkM22+XVVvHEiPJEnTYtIj\n/Kq6AXhoG/RFkjSNBjWG/5oktyX5WpKXT9QoyeIko0lGN2zYMKBNS5KmYhCBfyuwX1W9Cvg08M8T\nNayq86pqQVUtmD9//gA2LUmaqq0O/Kp6rKp+0k1fC+yUZN5W90ySNFBbHfhJXpgk3fRh3Tof3Nr1\nSpIGa9KrdJJcChwFzEuyBjgd2Amgqs4F3gq8K8lG4ElgUVXVtPVYkrRFJg38qjp+kuXn0LtsU5I0\ng/lJW0lqhIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1\nwsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5IaYeBLUiMM\nfElqhIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1YtLA\nT7I0yfokd0ywPEnOTrIyye1JDhl8NyVJW2sqR/gXAUdvZvkxwAHdbTHw2a3vliRp0CYN/Kq6AXho\nM00WApdUz43AHkn2GlQHJUmDMYgx/L2B1X3313TzfkWSxUlGk4xu2LBhAJuWJE3VNj1pW1XnVdWC\nqlowf/78bblpSWreIAJ/LbBv3/19unmSpBlkEIG/DDihu1rnCODRqlo3gPVKkgZox8kaJLkUOAqY\nl2QNcDqwE0BVnQtcCxwLrASeAE6ers5KkrbcpIFfVcdPsryAUwfWI0nStPCTtpLUCANfkhph4EtS\nIwx8SWqEgS9JjZj0Kh1pZMk1w+6CpAHwCF+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY\n+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5IaYeBLUiMMfElqhIEv\nSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEZMKfCT\nHJ3k7iQrkywZZ/lJSTYkWd7d3jn4rkqStsaOkzVIMgf4DPD7wBrge0mWVdWKMU0vr6rTpqGPkqQB\nmMoR/mHAyqq6r6p+DlwGLJzebkmSBm0qgb83sLrv/ppu3lh/mOT2JFck2Xe8FSVZnGQ0yeiGDRu2\noLuSpC01qJO2XwVGquqVwPXAxeM1qqrzqmpBVS2YP3/+gDYtSZqKqQT+WqD/iH2fbt4zqurBqnqq\nu3sBcOhguidJGpSpBP73gAOS7J9kZ2ARsKy/QZK9+u6+CbhrcF2UJA3CpFfpVNXGJKcB1wFzgKVV\ndWeSjwGjVbUMeE+SNwEbgYeAk6axz5KkLTBp4ANU1bXAtWPmfaRv+oPABwfbNUnSIPlJW0lqhIEv\nSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLU\nCANfkhph4EtSI6b0D1AkbRsjS64ZynZXnXncULarbcsjfElqhIEvSY0w8CWpEQa+JDXCwJekRhj4\nktQIL8ucJYZ1uZ6k7cesDPxhhp/XK0uarRzSkaRGGPiS1AgDX5IaYeBLUiMMfElqhIEvSY0w8CWp\nEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjpvTlaUmOBv4BmANcUFVnjlm+C3AJcCjwIPBHVbVqsF2d\nGfzWSkmz1aRH+EnmAJ8BjgEOAo5PctCYZqcAD1fVy4CzgE8MuqOSpK0zlSP8w4CVVXUfQJLLgIXA\nir42C4EzuukrgHOSpKpqgH2VNE38yvE2TCXw9wZW991fAxw+UZuq2pjkUeD5wAP9jZIsBhZ3d3+S\n5O4t6TQwb+y6G9Jq7a3WDdt57Zl4PGC7rnsSm6t9vy1d6Tb9ByhVdR5w3tauJ8loVS0YQJdmnVZr\nb7VuaLf2VuuG6at9KlfprAX27bu/Tzdv3DZJdgSeR+/krSRphphK4H8POCDJ/kl2BhYBy8a0WQac\n2E2/Ffim4/eSNLNMOqTTjcmfBlxH77LMpVV1Z5KPAaNVtQy4EPhCkpXAQ/T+KEynrR4WmsVarb3V\nuqHd2lutG6ap9nggLklt8JO2ktQIA1+SGjHrAj/J0UnuTrIyyZJh92cQkqxK8v0ky5OMdvP2THJ9\nknu6n3O7+Ulydlf/7UkO6VvPiV37e5KcONH2hinJ0iTrk9zRN29gtSY5tHsuV3aPzbatcHwT1H1G\nkrXdfl+e5Ni+ZR/sarg7yRv65o/7+u8uqripm395d4HF0CXZN8m3kqxIcmeS93bzW9jnE9U+vP1e\nVbPmRu+k8b3AS4CdgduAg4bdrwHUtQqYN2be3wJLuuklwCe66WOBrwEBjgBu6ubvCdzX/ZzbTc8d\ndm3j1Po64BDgjumoFbi5a5vusccMu+bN1H0G8IFx2h7UvbZ3AfbvXvNzNvf6B74MLOqmzwXeNeya\nu77sBRzSTe8G/KCrr4V9PlHtQ9vvs+0I/5mveaiqnwObvuZhe7QQuLibvhh4c9/8S6rnRmCPJHsB\nbwCur6qHquph4Hrg6G3d6clU1Q30ruTqN5Bau2W7V9WN1fsNuKRvXUM1Qd0TWQhcVlVPVdUPgZX0\nXvvjvv67I9rfo/e1JvDLz+FQVdW6qrq1m34cuIveJ/Nb2OcT1T6Rad/vsy3wx/uah809gbNFAd9I\nckt6Xz8B8IKqWtdN3w+8oJue6DmYzc/NoGrdu5seO38mO60buli6aViDZ1/384FHqmrjmPkzSpIR\n4GDgJhrb52NqhyHt99kW+NurI6vqEHrfSHpqktf1L+yOXJq4fralWoHPAi8FXg2sAz453O5MnyS7\nAlcC76uqx/qXbe/7fJzah7bfZ1vgT+VrHmadqlrb/VwP/BO9t3A/7t6u0v1c3zWf6DmYzc/NoGpd\n202PnT8jVdWPq+rpqvoFcD69/Q7Pvu4H6Q197Dhm/oyQZCd6gfelqrqqm93EPh+v9mHu99kW+FP5\nmodZJclzk+y2aRp4PXAHv/x1FScC/9JNLwNO6K5mOAJ4tHtrfB3w+iRzu7eIr+/mzQYDqbVb9liS\nI7rxzRP61jXjbAq8zlvo7Xfo1b0oyS5J9gcOoHdictzXf3eE/C16X2sCv/wcDlW3Hy4E7qqqv+9b\ntN3v84lqH+p+H/aZ7Gd7o3cW/wf0zlp/eNj9GUA9L6F31v024M5NNdEbn/s34B7gX4E9u/mh9w9p\n7gW+DyzoW9ef0jvRsxI4edi1TVDvpfTexv4vvTHHUwZZK7Cg+wW6FziH7tPkw75NUPcXurpu737Z\n9+pr/+Guhrvpu+pkotd/9zq6uXs+vgLsMuyau34dSW+45nZgeXc7tpF9PlHtQ9vvfrWCJDVitg3p\nSJK2kIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGvF/4Y3kayID79IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa90cc0a7f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa90cb6fc50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(matrix1)\n",
    "plt.title('blue')\n",
    "plt.figure()\n",
    "plt.hist(matrix2)\n",
    "plt.title('green')\n",
    "plt.figure()\n",
    "plt.hist(matrix3)\n",
    "plt.title('red')\n",
    "plt.figure()\n",
    "plt.hist(matrix4)\n",
    "plt.title('red edge')\n",
    "plt.figure()\n",
    "plt.hist(matrix5)\n",
    "plt.title('near infrared')\n",
    "plt.figure()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> Then we need to obtain our dependent variable (label data) <p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the shape of the CDL data is (5959, 9425)\n"
     ]
    }
   ],
   "source": [
    "labels = tif2.GetRasterBand(1)\n",
    "label_array = labels.ReadAsArray()\n",
    "print('the shape of the CDL data is', label_array.shape)\n",
    "# arr = labels.asarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> We have obtained our labelling data, then we need to see how many different types of crops are in our labelling data. 1 represents the corn, 5 represents the soybeans, and 3 represents other types of crops. We have sorted our CDL data, we can see that corn appears the most in northern illinois and soybeans almost have the same level of production as corns</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(1, 25320078), (5, 21417645), (176, 2522306), (121, 2325752), (122, 1977921), (141, 1287171), (123, 375667), (24, 347272), (111, 154703), (124, 124414), (190, 105600), (36, 80492), (37, 74591), (131, 23415), (26, 11530), (28, 5821), (43, 3318), (61, 2298), (152, 679), (195, 679), (58, 569), (241, 406), (59, 405), (222, 188), (225, 175), (13, 174), (27, 97), (4, 90), (12, 48), (254, 41)]\n"
     ]
    }
   ],
   "source": [
    "label_dict = {}\n",
    "for i in label_array.flatten():\n",
    "    if i not in label_dict:\n",
    "        label_dict[i] = 0\n",
    "    else:\n",
    "        label_dict[i] += 1\n",
    "sorted_labels = sorted([(key,val) for key,val in label_dict.items()], key = lambda x: x[1], reverse = True)\n",
    "print(sorted_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<p> For large classification problems, svm and knn algorithms are good choice. knn has good accuracy when the dimension is low and the number of data points is large. However, knn is more sensitive to outliers and it does not have good performance in high dimensional dataset</p>\n",
    "<p> SVM is a stable algorithm. It is very good for high dimensional data, and it treats outliers pretty well. Even though it has lower accuracy than knn during cross validation, it is more preferred in our task </p>\n",
    "<p> First, we use KNN algorithm to train our model. As a rule of thumb, we should choose the square root of our sample size as our k value. Then, we can use 5-fold or 10-fold cross validation for parameter tuning to find the best k for our knn model</p>\n",
    "<p> The first step is to build our training data, in order to do it, we need to clean and normalizae our data </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "matrix1 = matrix1/np.max(matrix1)\n",
    "matrix2 = matrix2/np.max(matrix2)\n",
    "matrix3 = matrix3/np.max(matrix3)\n",
    "matrix4 = matrix4/np.max(matrix4)\n",
    "matrix5 = matrix5/np.max(matrix5)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#normalize input data\n",
    "input_data_flatten = np.zeros((5929*9425,5))\n",
    "for i in range(5929*9425):\n",
    "    input_data_flatten[i,0] = matrix1[i]\n",
    "    input_data_flatten[i,1] = matrix2[i]\n",
    "    input_data_flatten[i,2] = matrix3[i]\n",
    "    input_data_flatten[i,3] = matrix4[i]\n",
    "    input_data_flatten[i,4] = matrix5[i]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Then we get rid of other crop types other than corns and soybeans <p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "label_flatten = label_array.flatten()\n",
    "#normalize labels\n",
    "#use only 3 labels\n",
    "for i in range(len(label_flatten)):\n",
    "    if label_flatten[i] == 1 or label_flatten[i] == 5:\n",
    "        continue\n",
    "    else:\n",
    "        label_flatten[i] = 3\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> SVM Is good for high dimensional data, but knn is much easier to compute than SVM. Even though when the number of sample increases, the knn algorithm needs more time to find the nearest neighbor, it is still faster than SVM. In our crop data, knn is also highly reliable for some good k values.  </p>\n",
    "\n",
    "<p> We randomly sample our input data without repetition, and take our X_train and Y_train from our samples</p>\n",
    "<p> A sample size of 10000 is actually good for training because a larger sample has the potential to overfit with a little bit increase in accuracy</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#random sampling the training data\n",
    "population = np.arange(5929*9425)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample = np.random.choice(population,10000, False)\n",
    "sample = sorted(sample)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train = np.take(input_data_flatten, sample, 0)\n",
    "Y_train = np.take(label_flatten, sample)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Clear memory for training</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "matrix1 = None\n",
    "matrix2 = None\n",
    "matrix3 = None\n",
    "matrix4 = None\n",
    "matrix5 = None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> First we try our svm model </p>\n",
    "<p>By our assumption, since the every crop has its different colors, and it is rare that if two crop has a very different color and they are of the same type. So, our kernel is not linear, we can not seperate different groups by a high-dimensional line. It is better to use rbf kernel here </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import svm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> Then we use 5-fold cross validation to study the accuracy of our svm model </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.62268866  0.69715142  0.692       0.70685343  0.65832916]\n"
     ]
    }
   ],
   "source": [
    "svm_model = svm.SVC(decision_function_shape = 'ovo', kernel = 'rbf')\n",
    "score = cross_val_score(svm_model, X_train, Y_train, cv=5, scoring='accuracy')\n",
    "print(score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f2a058d5710>"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGkVJREFUeJzt3X+cXXV95/HX2/yAWIIBM2h+mlQDLQoSmAYpC2apmKy1\nhFaLYbUYdoXdtbTVStzE7ipFLd3m0Wqt6cMFpQoVY4qYx4jogAssKws0kxKMmZgwRCAzATKEDD9k\nXJLw2T/Od+jJZSb3zuTO3Jn5vp+Px33knu/53nM+59zc9z33e+6do4jAzMzy8JpGF2BmZiPHoW9m\nlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvmVP0t2SPpLuf1DS7bX0HcJ65kp6QdKEodZqdqQc+mYl\nEfHNiHh3PZYl6VFJ7yot+/GIOCYiDtZj+WZD4dC3YSdpYqNrsOHj53dsceiPE5L+q6QuSc9L2i7p\ntyTNlNQr6fhSv4WSnpY0SdIKSfdK+oKkHkk7Jf1mat8laY+kDx9mncdL+gdJuyXtk7QhtS+W1Jlq\nehL4h9R+maQOSc9IapE0M7Ur1bBH0nOStkh6W5r3Hkntabu6JF3ZTx1HpfrfVmprStt+gqTjJN0q\nqTvVeauk2QNs0wpJPy5Nny/pZ5KelfRlQKV5b5Z0p6S9aZ9+U9K0NO9GYC7wvTSk80lJ8yRFX0im\n56cl7Y8OSZeVln2VpPWSbkjbvlVS82Gei79Nz9lzkjZJOqc0b4KkT0l6JC1rk6Q5ad5bJd2RanhK\n0qdS+9clfa60jMWSOkvTj6bn9yfALyRNlLSqtI52Sb9bUeNlkraV5p8uaaWk71T0+5Kkvx1oW+0I\nRYRvY/wGnATsAmam6XnAm9P9O4HLSn3XAF9J91cAB4BLgQnA54DHgbXAUcC7geeBYwZY7/eBbwPH\nAZOAd6b2xWm5/yMtZwpwHvA0cHpq+zvgntR/CbAJmEYRqr8OzEjzngDOSfePA04foJbrgc+Xpv8Q\n+GG6/3rgfcBrganAPwEbSn3vBj5S2ic/Tvenp+1/f9q+j6ft6uv7FuD8tD1NwD3AF0vLfRR4V2l6\nHhDAxDR9D/D3wNHAaUA3cF6adxXwS+A96bm5Brj/MP8HPpS2cyLwCeBJ4Og0byWwJf0/EfD21Hdq\n2r+fSDVMBc5Mj/k68LnS8hcDnRXbthmYA0xJbb8PzKQ4mPwA8IvS8/j7QBfwG6mGtwBvAmakftNS\nv4nAHuCMRr+uxuut4QX4VocnsXgB7QHeBUyqmPcR4M50XxRvDuem6RXAw6W+p6RQekOpbS9wWj/r\nnAG8DBzXz7zFwEt9oZPavgb8VWn6GGB/CsLzgB3AO4DXVCzrceA/AcdW2QfvAh4pTd8LXDJA39OA\nfaXpu+k/9C8pB23af519fftZ7oXAg6XpRxkg9FNYHgSmluZfA3w93b8K+FFp3slA7yD+T+wD3p7u\nbweW9dPn4nK9FfO+TvXQ/w9Vatjct16gFfiTAfr9gHRgArwXaG/0a2o83zy8Mw5ERAfwMYqg2CNp\nXd/QCfAd4CxJM4BzKYL6/5Qe/lTpfm9aXmXbMf2sdg7wTETsG6Cs7oj4ZWl6JvBYqeYXKN5QZkXE\nncCXKT5h7JF0raRjU9f3URztPibpf0s6a4D13QW8VtKZkuZRBPt3ASS9VtL/lPSYpOcojrCnqfq3\naGZSvEn21RzlaUlvSPu6Ky33Hyk+HdRiJsX+e77U9hgwqzT9ZOn+i8DRGmD8XNKVaejkWUk9wOtK\ntcwBHunnYQO112pXeULSJZI2p6G2HuBtNdQA8A2KTyqkf288gpqsCof+OBERN0XEv6H4yBwUQyuk\nUL6d4uP2vwfWpfA6UruA4/vGsPsrqWJ6d6oNAEm/QjHE0JXq/FJEnEFxRHsixZAEEbExIpYBJwAb\ngPX9rqz4Rsx6iqPXi4FbS4H6CYqhjTMj4liKNz8ojc8P4AmKsOqrWeVp4C/Sdp6SlvuhimUebj/v\npth/U0ttc0n7YzDS+P0ngYsoPnlNA54t1bILeHM/D90F/OoAi/0FxXBYnzf20+eV7ZP0JuA64Arg\n9amGn9ZQAxTP66npnMx7gW8O0M/qwKE/Dkg6SdJ5ko6iGAfupTii73MTxVDF+9P9IxYRT1B8LP/7\ndKJ0kqRzD/OQbwGXSjot1fkXwAMR8aik30hH6JMowuaXwMuSJqv43vzrImI/8FzFdlW6ieLN7YMc\nup1TKfZJj4qT2p+pcTO/D7xV0u+lI+w/5tDwmwq8ADwraRbpjarkKQYI1YjYBfxf4BpJR0s6FfiP\nFJ8WBmsqxbmGbmCipE8Dx5bmfxX4rKQFKpwq6fXArcAMSR9TcTJ8qqQz02M2A+9RcbL+jRSfJA/n\nVyjeBLoBJF1KcaRfruFKSWekGt6S3ihInwhvpnjO/jkiHh/CPrAaOfTHh6OAv6Q4UfokxVHx6tL8\nFmAB8GREPFTH9f4Bxbj8zyjOKQwYDBHxI+C/Uww3PUFx1Lc8zT6W4ihxH8UQx16KE85963g0DZ/8\nZ4pAH2gdD1C8acykeEPq80WKk8lPA/cDP6xl4yLiaYoTkH+ZalpAca6gz59TnJh+luIN4paKRVwD\n/Lc03PGqbx1RfCKZR3HU/13gM2k/DVYrxTbtoNh/v+TQoZe/ofgUdDvFG+fXKE6+Pk9xIvp3KP7f\nPAz82/SYG4GHKMbub6c4YT+giGgH/hq4j+LN7hRK+yoi/gn4PEWwP09xdH98aRHfSI/x0M4wU30+\n6ZuZDZ2kuRQHD2+MiOcaXc945iN9M2soSa8B/pTifJMDf5j5l3Rm1jDphP5TFMNSSxtcThY8vGNm\nlhEP75iZZWTUDe9Mnz495s2b1+gyzMzGlE2bNj0dEU3V+o260J83bx5tbW2NLsPMbEyR9Fj1Xh7e\nMTPLikPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjNYW+pKUqrrvaIWlVP/O/kC6esFnSjnQBBdKf\n0b1PxfU9fyLpA/XeADMzq13V7+mnqwutpfgTrJ3ARkkt6U+pAhARHy/1/yNgYZp8keKSdQ+nKzlt\nktQaET313AgzM6tNLUf6i4COiNgZES8B64Blh+l/McUFM4iIHRHxcLq/m+Jvrlf9xZiZmQ2PWkJ/\nFodekKGTQ6/j+Yp0JZz5wJ39zFsETKaf62RKulxSm6S27u7uWuo2M7MhqPeJ3OXAzel6pa9IF+W+\nEbg0Il51ubuIuDYimiOiuanJHwTMzIZLLaHfxaEXg57NwBdvXk4a2ukj6ViKS8n9WUTcP5Qizcys\nPmoJ/Y3AAknzJU2mCPaWyk6Sfg04juIamX1tkymu/XlDRNxcn5LNzGyoqoZ+RBwArqC4+PI2YH1E\nbJV0taQLSl2XU1zurHxVlouAc4EVpa90nlbH+s3MbBBG3ZWzmpubw39a2cxscCRtiojmav38i1wz\ns4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPf\nzCwjDn0zs4w49M3MMuLQNzPLyMRGF2Bm48+GB7tY07qd3T29zJw2hZVLTuLChbMaXZbh0DezOtvw\nYBerb9lC7/6DAHT19LL6li0ADv5RwMM7ZlZXa1q3vxL4fXr3H2RN6/YGVWRlDn0zq6vdPb2DareR\n5dA3s7qaOW3KoNptZDn0zayuVi45iSmTJhzSNmXSBFYuOalBFVmZT+SaWV31naz1t3dGJ4e+mdXd\nhQtnOeRHKQ/vmJllxKFvZpYRh76ZWUYc+mZmGakp9CUtlbRdUoekVf3M/4Kkzem2Q1JPad6HJT2c\nbh+uZ/FmZjY4Vb+9I2kCsBY4H+gENkpqiYj2vj4R8fFS/z8CFqb7xwOfAZqBADalx+6r61aYmVlN\najnSXwR0RMTOiHgJWAcsO0z/i4FvpftLgDsi4pkU9HcAS4+kYDMzG7paQn8WsKs03ZnaXkXSm4D5\nwJ2DeaykyyW1SWrr7u6upW4zMxuCep/IXQ7cHBEHq/YsiYhrI6I5IpqbmprqXJKZmfWp5Re5XcCc\n0vTs1Naf5cAfVjx2ccVj7669PBsuvsiFWZ5qOdLfCCyQNF/SZIpgb6nsJOnXgOOA+0rNrcC7JR0n\n6Tjg3anNGqjvIhddPb0E/3qRiw0PDvRebmbjRdXQj4gDwBUUYb0NWB8RWyVdLemCUtflwLqIiNJj\nnwE+S/HGsRG4OrVZA/kiF2b5qukPrkXEbcBtFW2frpi+aoDHXg9cP8T6bBj4Ihdm+fIvcjPki1yY\n5cuhnyFf5MIsX/57+hnyRS7M8uXQz5QvcmGWJw/vmJllxEf6ZjXwj9lsvHDom1XR92O2vt829P2Y\nDXDw25jj4R2zKvxjNhtPHPpmVfjHbDaeOPTNqvCP2Ww8ceibVeEfs9l44hO5ZlX4x2w2njj0zWrg\nH7PZeOHhHTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy\n4tA3M8uIQ9/MLCMOfTOzjNQU+pKWStouqUPSqgH6XCSpXdJWSTeV2v8qtW2T9CVJqlfxZmY2OFX/\ntLKkCcBa4HygE9goqSUi2kt9FgCrgbMjYp+kE1L7bwJnA6emrj8G3gncXc+NMDOz2tRypL8I6IiI\nnRHxErAOWFbR5zJgbUTsA4iIPak9gKOBycBRwCTgqXoUbmZmg1dL6M8CdpWmO1Nb2YnAiZLulXS/\npKUAEXEfcBfwRLq1RsS2yhVIulxSm6S27u7uoWyHmZnVoF4ncicCC4DFwMXAdZKmSXoL8OvAbIo3\nivMknVP54Ii4NiKaI6K5qampTiWZmVmlWkK/C5hTmp6d2so6gZaI2B8RPwd2ULwJ/C5wf0S8EBEv\nAD8Azjryss3MbChqCf2NwAJJ8yVNBpYDLRV9NlAc5SNpOsVwz07gceCdkiZKmkRxEvdVwztmZjYy\nqoZ+RBwArgBaKQJ7fURslXS1pAtSt1Zgr6R2ijH8lRGxF7gZeATYAjwEPBQR3xuG7TAzsxooIhpd\nwyGam5ujra2t0WWYmY0pkjZFRHO1fv5FrplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceib\nmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6\nZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZqSn0JS2VtF1Sh6RV\nA/S5SFK7pK2Sbiq1z5V0u6Rtaf68+pRuZmaDNbFaB0kTgLXA+UAnsFFSS0S0l/osAFYDZ0fEPkkn\nlBZxA/D5iLhD0jHAy3XdAjMzq1ktR/qLgI6I2BkRLwHrgGUVfS4D1kbEPoCI2AMg6WRgYkTckdpf\niIgX61a9mZkNStUjfWAWsKs03QmcWdHnRABJ9wITgKsi4oepvUfSLcB84EfAqog4eKSFm5mNBxse\n7GJN63Z29/Qyc9oUVi45iQsXzhq29dUS+rUuZwGwGJgN3CPplNR+DrAQeBz4NrAC+Fr5wZIuBy4H\nmDt3bp1KMjMb3TY82MXqW7bQu784Du7q6WX1LVsAhi34axne6QLmlKZnp7ayTqAlIvZHxM+BHRRv\nAp3A5jQ0dADYAJxeuYKIuDYimiOiuampaSjbYWY25qxp3f5K4Pfp3X+QNa3bh22dtYT+RmCBpPmS\nJgPLgZaKPhsojvKRNJ1iWGdneuw0SX1Jfh7QjpmZsbund1Dt9VA19NMR+hVAK7ANWB8RWyVdLemC\n1K0V2CupHbgLWBkRe9PY/ZXA/5K0BRBw3XBsiJnZWDNz2pRBtdeDImLYFj4Uzc3N0dbW1ugyzMyG\nXeWYPsCUSRO45vdOGfSYvqRNEdFcrV+9TuSamdkg9QX7WPz2jpmZDcGFC2cNa8hX8t/eMTPLiEPf\nzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQ\nNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w4\n9M3MMuLQNzPLSE2hL2mppO2SOiStGqDPRZLaJW2VdFPFvGMldUr6cj2KNjOzoZlYrYOkCcBa4Hyg\nE9goqSUi2kt9FgCrgbMjYp+kEyoW81ngnvqVbWZmQ1HLkf4ioCMidkbES8A6YFlFn8uAtRGxDyAi\n9vTNkHQG8Abg9vqUbGZmQ1VL6M8CdpWmO1Nb2YnAiZLulXS/pKUAkl4D/DVw5eFWIOlySW2S2rq7\nu2uv3szMBqVeJ3InAguAxcDFwHWSpgEfBW6LiM7DPTgiro2I5ohobmpqqlNJZmZWqeqYPtAFzClN\nz05tZZ3AAxGxH/i5pB0UbwJnAedI+ihwDDBZ0gsR0e/JYDMzG161HOlvBBZImi9pMrAcaKnos4Hi\nKB9J0ymGe3ZGxAcjYm5EzKMY4rnBgW9m1jhVQz8iDgBXAK3ANmB9RGyVdLWkC1K3VmCvpHbgLmBl\nROwdrqLNzGxoFBGNruEQzc3N0dbW1ugyzMzGFEmbIqK5Wj//ItfMLCMOfTOzjDj0zcwy4tA3M8uI\nQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy\n4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjExsdAH1suHBLta0bmd3Ty8zp01h\n5ZKTuHDhrEaXZWY2qoyL0N/wYBerb9lC7/6DAHT19LL6li0ADn4zs5JxMbyzpnX7K4Hfp3f/Qda0\nbm9QRWZmo9O4CP3dPb2Dajczy1VNoS9pqaTtkjokrRqgz0WS2iVtlXRTajtN0n2p7SeSPlDP4vvM\nnDZlUO1mZrmqGvqSJgBrgX8HnAxcLOnkij4LgNXA2RHxVuBjadaLwCWpbSnwRUnT6lg/ACuXnMSU\nSRMOaZsyaQIrl5xU71WZmY1ptZzIXQR0RMROAEnrgGVAe6nPZcDaiNgHEBF70r87+jpExG5Je4Am\noKc+5Rf6Ttb62ztmZodXS+jPAnaVpjuBMyv6nAgg6V5gAnBVRPyw3EHSImAy8EjlCiRdDlwOMHfu\n3FprP8SFC2c55M3MqqjXidyJwAJgMXAxcF15GEfSDOBG4NKIeLnywRFxbUQ0R0RzU1NTnUoyM7NK\ntYR+FzCnND07tZV1Ai0RsT8ifg7soHgTQNKxwPeBP4uI+4+8ZDMzG6paQn8jsEDSfEmTgeVAS0Wf\nDRRH+UiaTjHcszP1/y5wQ0TcXLeqzcxsSKqGfkQcAK4AWoFtwPqI2CrpakkXpG6twF5J7cBdwMqI\n2AtcBJwLrJC0Od1OG5YtMTOzqhQRja7hEM3NzdHW1tboMszMxhRJmyKiuVq/cfGLXDMzq41D38ws\nIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dcz\ny4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTN\nzDKiiGh0DYeQ1A08dgSLmA48Xady6sl1DY7rGhzXVbvRWBMceV1vioimap1GXegfKUltEdHc6Doq\nua7BcV2D47pqNxprgpGry8M7ZmYZceibmWVkPIb+tY0uYACua3Bc1+C4rtqNxppghOoad2P6ZmY2\nsPF4pG9mZgNw6JuZZWTMhr6kpZK2S+qQtKqf+UdJ+naa/4CkeaOkrhWSuiVtTrePjEBN10vaI+mn\nA8yXpC+lmn8i6fThrqnGuhZLera0rz49QnXNkXSXpHZJWyX9ST99RnSf1VjTiO8vSUdL+mdJD6W6\n/ryfPiP+WqyxrhF/LZbWPUHSg5Ju7Wfe8O6viBhzN2AC8Ajwq8Bk4CHg5Io+HwW+ku4vB749Supa\nAXx5hPfXucDpwE8HmP8e4AeAgHcAD4ySuhYDtzbg/9cM4PR0fyqwo5/ncUT3WY01jfj+Stt/TLo/\nCXgAeEdFn0a8Fmupa8Rfi6V1/ylwU3/P13Dvr7F6pL8I6IiInRHxErAOWFbRZxnwjXT/ZuC3JGkU\n1DXiIuIe4JnDdFkG3BCF+4FpkmaMgroaIiKeiIh/SfefB7YBsyq6jeg+q7GmEZe2/4U0OSndKr8d\nMuKvxRrraghJs4HfBr46QJdh3V9jNfRnAbtK0528+gXwSp+IOAA8C7x+FNQF8L40JHCzpDnDXFMt\naq27Ec5KH9F/IOmtI73y9NF6IcWRYlnD9tlhaoIG7K80VLEZ2APcERED7qsRfC3WUhc05rX4ReCT\nwMsDzB/W/TVWQ38s+x4wLyJOBe7gX9/R7dX+heLvibwd+Dtgw0iuXNIxwHeAj0XEcyO57oFUqakh\n+ysiDkbEacBsYJGkt43Eequpoa4Rfy1Kei+wJyI2Dfe6BjJWQ78LKL8rz05t/faRNBF4HbC30XVF\nxN6I+H9p8qvAGcNcUy1q2Z8jLiKe6/uIHhG3AZMkTR+JdUuaRBGu34yIW/rpMuL7rFpNjdxfaZ09\nwF3A0opZjXgtVq2rQa/Fs4ELJD1KMfx7nqR/rOgzrPtrrIb+RmCBpPmSJlOc7Gip6NMCfDjdfz9w\nZ6QzI42sq2Lc9wKKsdlGawEuSd9IeQfwbEQ80eiiJL2xbyxT0iKK/6/DHhZpnV8DtkXE3wzQbUT3\nWS01NWJ/SWqSNC3dnwKcD/ysotuIvxZrqasRr8WIWB0RsyNiHkU+3BkRH6roNqz7a2K9FjSSIuKA\npCuAVopvzFwfEVslXQ20RUQLxQvkRkkdFCcLl4+Suv5Y0gXAgVTXiuGuS9K3KL7ZMV1SJ/AZihNb\nRMRXgNsovo3SAbwIXDrcNdVY1/uB/yLpANALLB+BN24ojsb+ANiSxoQBPgXMLdU20vuslpoasb9m\nAN+QNIHiTWZ9RNza6NdijXWN+GtxICO5v/xnGMzMMjJWh3fMzGwIHPpmZhlx6JuZZcShb2aWEYe+\nmVlGHPpmZhlx6JuZZeT/A5u0jkR58krhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2ae35e1588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(np.arange(5),score)\n",
    "plt.title('svm cross validation accuracy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> It looks like our SVM model has a good accuracy of prediction. Since there are a lot of outliers in our raw data like clouds, buildings, neighborhoods, etc... An average of 67.5% accuracy is good</p>\n",
    "\n",
    "<p> Then we want to study the accuracy of our knn model. We use cross validation to obtain the best k for our knn model</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "cv_scores = []\n",
    "neighbors = [i for i in range(50,150,10)]\n",
    "for k in neighbors:\n",
    "    knn = KNeighborsClassifier(n_neighbors=k)\n",
    "    scores = cross_val_score(knn, X_train, Y_train, cv=5, scoring='accuracy')\n",
    "    cv_scores.append(scores.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.77951165660291422, 0.77591195605298902, 0.77351215565303888, 0.77071295495323877, 0.76721475435368858, 0.76331510357877597, 0.76231530337882591, 0.76041620292905066, 0.75821580252895071, 0.75671635230408807]\n"
     ]
    }
   ],
   "source": [
    "print(cv_scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f29cab0b160>"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHw5JREFUeJzt3XuYHGWZ9/HvjwkJISiTkHggiUmQkyhI2BHxBAgiwXVN\nRFYTj6i7vLwrqCi6uOu6LiqK4ht3VzygYDwmCgvZeEBAMSAaJBMJp2AghEAyHByE4RiEJPf7Rz1N\najrdUz2Q6u6Z+X2uq67U4amqu2o6fXc9T9VTigjMzMwGskOrAzAzs/bnZGFmZoWcLMzMrJCThZmZ\nFXKyMDOzQk4WZmZWyMliGJG0TtLrWhzD4ZI2lLj9RyTtMcDylp+Dp0vS8ZKuyk0PeKwDbOcdki7d\nvtHZSOdkYUNKROwSEWsBJC2Q9NlWx1SW/LHWI2m6pJA0KrfeDyPi9eVHaCOJk4UNCfkvw6FgqMU7\nFPictpaTxTAl6UWSbpc0L02vk3SqpOslPSjpx5J2SssOl7RB0kcl/VnS3ZLeO8C23yvpZkkPS1or\n6f8MUPYgSdemsuen/X42t/wfJa2RdL+kJZJ2zy0LSR+QdCtwa27enpJOAN4BfDxV1/w0t9sDC47z\n47njnCPpDZJuSTH8ywDHskDSNyRdlo7nCknTCuLdN5W/X9JqSW/Nld8tHfNDkq4BXli1v5C0Zxof\nK+nLku5Ix3WVpLHAlal4XzoPr6hRnfVKScvTesslvTK3bKmkz0j6XTqmSyVNrHP84yX9TFKvpAfS\n+JTc8gmSviPprrR8cW7ZbEkr07HeJmlWmt+v2lDSpyX9II1XrpreL+lO4PI0/3xJ96TjuVLSi3Pr\n1zxPkn4u6eSq47le0pvr/b2tSkR4GCYDsA54HXAQcCfwxqpl1wC7AxOAm4ET07LDgU3A6cCOwBuA\nx4Dxdfbzt2RfbAIOS2UPym1rQxofDdwBfCht91jgCeCzafkRwH0p3jHAfwNX5vYTwGUp3rG5eXum\n8QWVbQ3yOD+V4vlHoBf4EfAs4MXARmBGneNeADwMHJri/U/gqnrxAuOA9cB7gVHAzHS8+6Xyi4Cf\npHIvAXpqbK9yrGcDS4HJQAfwyhTD9FRuVG694yvbSbE8ALwrxTAvTe+Wli8FbgP2TjEvBb5Q5/h3\nA94C7JzO1/nA4tzynwM/Bsan83tYmn8w8CBwFNkP1MnAvvnPbG4bnwZ+kMYrx/a9dI4qn4H3pf2P\nAb4CrMytX+88vRX4Q67cS4G/AKNb/f92qAwtD8DDdvxjZv/x/gPYABxeY9k7c9NfBL6Rxg8n+5LM\nf+H8GTikwf0uBj6U21YlWRyavgCVK3sVW5PFucAXc8t2AZ4EpqfpAI6o2lcjyaLoODvS9LPS9l6e\nK78CmFPnOBcAi6ri3QxMrRUv8Dbgt1Xb+Cbw7+mL7MnKl2ZadgY1kgXZF+xG4KU1Yqp8odZLFu8C\nrqlaZxlwfBpfCnwyt+yfgF82+Hc/EHggjT8f2EKNHxjpmOcP8JktShZ7DBBDZyqza8F52oksSe6V\nps8CvtbM/59DfXA11PBzIvD7iFhaY9k9ufHHyL7sKv4SEZsGWP4UScdIujpVrfSRXYnUqrrYHeiJ\n9L8zWV+1/I7KREQ8QvZrb3Kd8o0qOs7NaXxj+vfe3PKN1Dnu6nhSvPeTHcc2y4FpwMsl9VUGsqqz\n5wGTyH7p58vfQW0Tyb7sbhsgrnr6nePcfvLneKDz9RRJO0v6ZqrieYisCqxTUgcwFbg/Ih6oserU\npxl7xVPnSFKHpC+kqqyHyJINZOeo7nmKiMfJrnreKWkHsius7z+DmEYcJ4vh50TgBZLml7FxSWOA\n/yH7ZfbciOgEfkFWJVXtbmCypPyyqbnxu8i+UCvbHkdW1dGTKzNQt8it6DL5qfgl7UJWzXNXbnl1\nYrwiIjpzwy4R8X/Jqr820f98vKDOPu8DHqeqTaPG/mrpd45z++mpUbbIR4F9yK7Enk125QjZ3349\nMEFSZ4311lM7doBHyaq1Kp5Xo0z+GN8OzCarbt2V7OqjEsNA5wngu2TJ+kjgsYhYVqec1eBkMfw8\nDMwCDpX0hRK2P5qsDrgX2CTpGKDebZrLyKppTpI0StJssvrrioXAeyUdmJLQGWT1yusajOVeYNDP\nITxDb5D0akmjgc8AV0dEvaufnwF7S3qXpB3T8DJJL0pXNxcCn06/2PcD3lNrIxGxBTgP+H+Sdk+/\nrl+RzlkvWfVPvfPwixTD29Pf4G3Afim2wXoW2ZVXn6QJZNVplRjvBi4GvpYawneUVEkm55L9nY+U\ntIOkyZL2TctWAnNT+S7guAZi+CvZFejOZJ+ZSgwDnSdSctgCfBlfVQyak8UwFBF9ZI2Jx0j6zHbe\n9sPAB8kaZh8g+6W3pE7ZJ8gatd8P9AHvJPuS+mta/ivg38iuVO4m+0U4dxDhnAvsl6p4FheW3j5+\nRPYleT/wN2THVFM6V68nO6a7yKp7ziRLtgAnkVX53EPWHvKdAfZ7KnADsDzt+0xgh4h4DPgc8Lt0\nHg6piuEvwBvJrgr+Anyc7MaH+xo+4q2+QtYIfh9wNfDLquXvImuH+RNZm9eHUwzXkDXyzydr6L6C\nrVc7/0b2d3+ArL3tRwUxfI+sGq0HWJXiyKt5nqrW3x/4QcF+rIr6VyeblUvSH8ganAf6YmxLkhaQ\nNd5/stWx2NMj6d3ACRHx6lbHMtT4ysJKJekwSc9LVSDvAQ5g21+kZqWTtDPZ3V7ntDqWocjJwsq2\nD3AdWTXUR4HjUv22WdNIOpqsfedeiqu6rAZXQ5mZWSFfWZiZWaFh0zHXxIkTY/r06a0Ow8xsSFmx\nYsV9ETGpqNywSRbTp0+nu7u71WGYmQ0pkur1HNCPq6HMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszM\nCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr\n5GRhZmaFnCzMzKyQk4WZmRUqNVlImiVptaQ1kk6rsXy+pJVpuEVSX27ZmZJuTMPbyozTzMwGVto7\nuCV1AGcDRwEbgOWSlkTEqkqZiDglV/5kYGYa/1vgIOBAYAywVNLFEfFQWfGamVl9ZV5ZHAysiYi1\nEfEEsAiYPUD5ecDCNL4fcGVEbIqIR4HrgVklxmpmZgMoM1lMBtbnpjekeduQNA2YAVyeZl0HzJK0\ns6SJwGuBqTXWO0FSt6Tu3t7e7Rq8mZlt1S4N3HOBCyJiM0BEXAr8Avg92dXGMmBz9UoRcU5EdEVE\n16RJk5oZr5nZiFJmsuih/9XAlDSvlrlsrYICICI+FxEHRsRRgIBbSonSzMwKlZkslgN7SZohaTRZ\nQlhSXUjSvsB4squHyrwOSbul8QOAA4BLS4zVzMwGUNrdUBGxSdJJwCVAB3BeRNwk6XSgOyIqiWMu\nsCgiIrf6jsBvJQE8BLwzIjaVFauZmQ1M/b+jh66urq7o7u5udRhmZkOKpBUR0VVUrl0auM3MrI05\nWZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+Rk\nYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOF\nmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVKjVZSJolabWk\nNZJOq7F8vqSVabhFUl9u2Rcl3STpZkn/JUllxmpmZvWNKmvDkjqAs4GjgA3AcklLImJVpUxEnJIr\nfzIwM42/EngVcEBafBVwGLC0rHjNzKy+0pIFcDCwJiLWAkhaBMwGVtUpPw/49zQewE7AaEDAjsC9\nZQS5+NoevnTJau7q28junWP52NH7MGfm5DJ2ZWY2ZJVZDTUZWJ+b3pDmbUPSNGAGcDlARCwDfgPc\nnYZLIuLmGuudIKlbUndvb++gA1x8bQ+fuPAGevo2EkBP30Y+ceENLL62Z9DbMjMbztqlgXsucEFE\nbAaQtCfwImAKWYI5QtJrqleKiHMioisiuiZNmjTonX7pktVsfHJzv3kbn9zMly5Z/TQOwcxs+Coz\nWfQAU3PTU9K8WuYCC3PTbwaujohHIuIR4GLgFds7wLv6Ng5qvpnZSFVmslgO7CVphqTRZAlhSXUh\nSfsC44Fludl3AodJGiVpR7LG7W2qoZ6p3TvHDmq+mdlIVVqyiIhNwEnAJWRf9D+JiJsknS7pTbmi\nc4FFERG5eRcAtwE3ANcB10XET7d3jB87eh/G7tjRb97YHTv42NH7bO9dmZkNaer/HT10dXV1RXd3\n96DX891QZjaSSVoREV1F5cq8dXZImDNzspODmVmBdrkbyszM2piThZmZFXKyMDOzQk4WZmZWyMnC\nzMwKOVmYmVmhEX/rbLvw8x5m1s6cLNpApffbSqeGld5vAScMM2sLroZqA+791szanZNFG3Dvt2bW\n7pws2oB7vzWzdudk0Qbc+62ZtTs3cLeBSiO274Yys3blZNEm3PutmbUzV0OZmVkhJwszMyvkaih7\nip8iN7N6nCwM8FPkZjYwV0MZ4KfIzWxgThYG+ClyMxtYYbKQdLKk8c0IxlrHT5Gb2UAaubJ4LrBc\n0k8kzZKksoOy5vNT5GY2kMJkERGfBPYCzgWOB26VdIakF5YcmzXRnJmT+fyx+zO5cywCJneO5fPH\n7u/GbTMDGrwbKiJC0j3APcAmYDxwgaTLIuLjZQZozeOnyM2snsJkIelDwLuB+4BvAx+LiCcl7QDc\nCjhZmJkNc41cWUwAjo2IO/IzI2KLpDeWE5aZmbWTRhq4Lwbur0xIeraklwNExM1lBWZmZu2jkSuL\nrwMH5aYfqTHPbLtxtyNm7aeRZKGIiMpEqn5yNyFWCnc7YtaeGqmGWivpg5J2TMOHgLVlB2Yjk7sd\nMWtPjSSLE4FXAj3ABuDlwAmNbDw9xLda0hpJp9VYPl/SyjTcIqkvzX9tbv5KSY9LmtP4YdlQ5W5H\nzNpTYXVSRPwZmDvYDUvqAM4GjiJLMsslLYmIVbltn5IrfzIwM83/DXBgmj8BWANcOtgYbOjZvXMs\nPTUSg7sdMWutRvqG2knSByR9TdJ5laGBbR8MrImItRHxBLAImD1A+XnAwhrzjwMujojHGtinDXHu\ndsSsPTVSDfV94HnA0cAVwBTg4QbWmwysz01vSPO2IWkaMAO4vMbiudROIkg6QVK3pO7e3t4GQrJ2\n525HzNpTI3c17RkRfy9pdkR8V9KPgN9u5zjmAhdERL+WTUnPB/YHLqm1UkScA5wD0NXVFbXK2NDj\nbkfM2k8jVxZPpn/7JL0E2BV4TgPr9QBTc9NT0rxa6l09vBW4KCKerLHMzMyapJFkcU56n8UngSXA\nKuDMBtZbDuwlaYak0WQJYUl1IUn7knVMuKzGNuq1Y5iZWRMNWA2VOgt8KCIeAK4E9mh0wxGxSdJJ\nZFVIHcB5EXGTpNOB7oioJI65wKL8g39p39PJrkyuaHSfZmZWDlV9R29bQOqOiK4mxfO0dXV1RXd3\nd6vDMDMbUiStaOQ7vpFqqF9JOlXSVEkTKsN2iNHMzIaIRu6Gelv69wO5ecEgqqTMzGxoa+QJ7hnN\nCMSs3bj3W7OtGnlT3rtrzY+I723/cMzag3u/NeuvkTaLl+WG1wCfBt5UYkxmLefeb836a6Qa6uT8\ntKROsn6ezIYt935r1l8jVxbVHiXrx8ls2KrXy617v7WRqpFeZ38qaUkafgasBi4qPzSz1nHvt2b9\nNXLr7Fm58U3AHRGxoaR4zNpCpRHbd0OZZRpJFncCd0fE4wCSxkqaHhHrSo3MrMXc+63ZVo20WZwP\nbMlNb07zzMxshGgkWYxKb7oDII2PLi8kMzNrN40ki15JTz1XIWk2cF95IZmZWbtppM3iROCHkr6a\npjcANZ/qNjOz4amRh/JuAw6RtEuafqT0qMzMrK008pzFGZI6I+KRiHhE0nhJn21GcGZm1h4aabM4\nJiL6KhPprXlvKC8kMzNrN420WXRIGhMRf4XsOQtgTLlhmVmFu0q3dtBIsvgh8GtJ3wEEHA98t8yg\nzCzjrtKtXRRWQ0XEmcBngRcB+wCXANNKjsvMcFfp1j4a7XX2XrJXqf49cARwc2kRmdlT3FW6tYu6\n1VCS9gbmpeE+4MeAIuK1TYrNbMTbvXMsPTUSQyu6Snfbycg20JXFn8iuIt4YEa+OiP8m6xfKzJqk\nXbpKr7Sd9PRtJNjadrL42p6mxmGtM1CyOBa4G/iNpG9JOpKsgdvMmmTOzMl8/tj9mdw5FgGTO8fy\n+WP3b/overedWN1qqIhYDCyWNA6YDXwYeI6krwMXRcSlTYrRbERrh67S3XZijdwN9WhE/Cgi/g6Y\nAlwL/HPpkZlZ2/BrZm1Q7+COiAci4pyIOLKsgMys/bRL24m1TiMP5ZnZCOfXzJqThZk1pB3aTnz7\nbus4WZjZkOCuT1prUG0WgyVplqTVktZIOq3G8vmSVqbhFkl9uWUvkHSppJslrZI0vcxYzay9+fbd\n1irtykJSB3A2cBTZ2/WWS1oSEasqZSLilFz5k4GZuU18D/hcRFyWXry0paxYzaz9+fbd1irzyuJg\nYE1ErI2IJ4BFZM9r1DMPWAggaT9gVERcBtnb+SLisRJjNbM259t3W6vMZDEZWJ+b3pDmbUPSNGAG\ncHmatTfQJ+lCSddK+lK6Uqle7wRJ3ZK6e3t7t3P4ZtZOfPtua5XaZjEIc4ELIqJSITkKeA1wKvAy\nYA+y92j0k5756IqIrkmTJjUrVjNrgXbp+mSkKvNuqB5gam56SppXy1zgA7npDcDKiFgLIGkxcAhw\nbglxmtkQ0Q63745UZV5ZLAf2kjRD0miyhLCkupCkfYHxwLKqdTslVS4XjgBWVa9rZmbNUdqVRURs\nknQS2Zv1OoDzIuImSacD3RFRSRxzgUUREbl1N0s6lex1rgJWAN8qK1Yzs8EYiQ8HKvcdPaR1dXVF\nd3d3q8Mws2Gu+uFAyBrah2r7iaQVEdFVVK5dGrjNzIaEkfpwoJOFmdkgjNSHA50szMwGYaQ+HOhk\nYWY2CCP14UD3OmtmNgjt9G6PZt6V5WRhZjZI7fBwYLO7bHc1lJnZENTsu7KcLMzMhqBm35XlZGFm\nNgQ1+64sJwszsyGo2XdluYHbzGwIavZdWU4WZmZDVDPvynI1lJmZFXKyMDOzQk4WZmZWyMnCzMwK\nOVmYmVkhJwszMyvkZGFmZoWcLMzMrJCThZmZFXKyMDOzQk4WZmZWyMnCzMwKOVmYmVkhJwszMyvk\nZGFmZoWcLMzMrJCThZmZFXKyMDOzQqUmC0mzJK2WtEbSaTWWz5e0Mg23SOrLLducW7akzDjNzGxg\npb2DW1IHcDZwFLABWC5pSUSsqpSJiFNy5U8GZuY2sTEiDiwrPjMza1yZVxYHA2siYm1EPAEsAmYP\nUH4esLDEeMzM7GkqM1lMBtbnpjekeduQNA2YAVyem72TpG5JV0uaU2e9E1KZ7t7e3u0Vt5mZVWmX\nBu65wAURsTk3b1pEdAFvB74i6YXVK0XEORHRFRFdkyZNalasZmYjTpnJogeYmpuekubVMpeqKqiI\n6En/rgWW0r89w8zMmqjMZLEc2EvSDEmjyRLCNnc1SdoXGA8sy80bL2lMGp8IvApYVb2umZk1R2l3\nQ0XEJkknAZcAHcB5EXGTpNOB7oioJI65wKKIiNzqLwK+KWkLWUL7Qv4uKjMzay71/44eurq6uqK7\nu7vVYZiZDSmSVqT24QG1SwO3mZm1MScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJ\nwszMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScL\nMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhZmaFnCzM\nzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCikiWh3DdiGpF7ij1XFsBxOB+1odRJvwuejP52Mrn4v+\nnsn5mBYRk4oKDZtkMVxI6o6IrlbH0Q58Lvrz+djK56K/ZpwPV0OZmVkhJwszMyvkZNF+zml1AG3E\n56I/n4+tfC76K/18uM3CzMwK+crCzMwKOVmYmVkhJ4sWktQp6QJJf5J0s6RXSJog6TJJt6Z/x7c6\nzmaRdIqkmyTdKGmhpJ0kzZD0B0lrJP1Y0uhWx1kGSedJ+rOkG3Pzan4WlPmvdE6ul3RQ6yIvR53z\n8aX0f+V6SRdJ6swt+0Q6H6slHd2aqMtT63zkln1UUkiamKZL+Xw4WbTWfwK/jIh9gZcCNwOnAb+O\niL2AX6fpYU/SZOCDQFdEvAToAOYCZwLzI2JP4AHg/a2LslQLgFlV8+p9Fo4B9krDCcDXmxRjMy1g\n2/NxGfCSiDgAuAX4BICk/cg+Ky9O63xNUkfzQm2KBWx7PpA0FXg9cGdudimfDyeLFpG0K3AocC5A\nRDwREX3AbOC7qdh3gTmtibAlRgFjJY0CdgbuBo4ALkjLh+35iIgrgfurZtf7LMwGvheZq4FOSc9v\nTqTNUet8RMSlEbEpTV4NTEnjs4FFEfHXiLgdWAMc3LRgm6DO5wNgPvBxIH+nUimfDyeL1pkB9ALf\nkXStpG9LGgc8NyLuTmXuAZ7bsgibKCJ6gLPIfiHdDTwIrAD6cl8QG4DJrYmwJep9FiYD63PlRtp5\nAXgfcHEaH5HnQ9JsoCcirqtaVMr5cLJonVHAQcDXI2Im8ChVVU6R3dc8Iu5tTvXxs8mS6O7AOGpc\ndo9UI+mzUETSvwKbgB+2OpZWkbQz8C/Ap5q1TyeL1tkAbIiIP6TpC8iSx72VS8b0759bFF+zvQ64\nPSJ6I+JJ4ELgVWSX0KNSmSlAT6sCbIF6n4UeYGqu3Ig5L5KOB94IvCO2PiQ2Es/HC8l+WF0naR3Z\nMf9R0vMo6Xw4WbRIRNwDrJe0T5p1JLAKWAK8J817D/C/LQivFe4EDpG0sySx9Xz8BjgulRlJ5wPq\nfxaWAO9Od70cAjyYq64atiTNIquff1NEPJZbtASYK2mMpBlkDbvXtCLGZomIGyLiORExPSKmk/34\nPCh9r5Tz+YgIDy0agAOBbuB6YDEwHtiN7M6XW4FfARNaHWcTz8d/AH8CbgS+D4wB9iD7j78GOB8Y\n0+o4Szr2hWRtNU+m//jvr/dZAAScDdwG3EB2B1nLj6EJ52MNWV38yjR8I1f+X9P5WA0c0+r4m3E+\nqpavAyaW+flwdx9mZlbI1VBmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwsbMSRtFRSqS+3T/v5\nYOpN+Bk/aZy6g9mvoMwCScfVmH+4pJ890xhsZBtVXMTMKiSNiq19VRX5J+B1EbHhme43Iv7hmW7j\n6ZLUERGbW7V/aw++srC2JGl6+lX+rfSOi0sljU3LnroykDQxdXeApOMlLU7vflgn6SRJH0kdNV4t\naUJuF++StDK9O+PgtP649N6Aa9I6s3PbXSLpcrKH5Kpj/Ujazo2SPpzmfYPsgcKLJZ1SVf54SRdK\n+qWyd1V8Mbfs9ZKWSfqjpPMl7VLjmN8v6ZYU57ckfTW3+UMl/V7S2qqrjGdL+nl638M3JO2QtjVP\n0g0p9jNzcTwi6cuSrgNeIekLklal9yOcNZi/pQ0TrX4y0YOHWgMwnayzuAPT9E+Ad6bxpaSnUoGJ\nwLo0fjzZU77PAiaR9Vx7Ylo2H/hwbv1vpfFDgRvT+Bm5fXSSvTNhXNruBmo8TQ/8DdlTsuOAXYCb\ngJlp2TrSU7VV6xwPrAV2BXYC7iDry2cicCUwLpX7Z+BT+WMm62RxHTAB2BH4LfDVVGYB2VPuOwD7\nAWvS/MOBx8mSVwfZeyGOS9u6M52rUcDlwJy0TgBvTeO7kT0ZXXmIt7PVnw8PzR9cDWXt7PaIWJnG\nV5AlkCK/iYiHgYclPQj8NM2/ATggV24hZO8JkPRsZW9dez3wJkmnpjI7AS9I45dFRK33CbwauCgi\nHgWQdCHwGuDagjh/HREPpnVWAdPIEtR+wO+y7rEYDSyrWu9g4IpKLJLOB/bOLV8cEVuAVZLy3dtf\nExFr0zoLU9xPAksjojfN/yFZ8lwMbAb+J637IFmyOTe1fbj9YwRysrB29tfc+GZgbBrfxNYq1J0G\nWGdLbnoL/T/v1f3cBFmfOm+JiNX5BZJeTtaF/PZUfWyj0v4vi4h522m7yo3XOt6BPB6pnSIiNqWq\nuiPJrkhOInsplY0gbrOwoWgdWfUPbO2RdrDeBiDp1WS9cj4IXAKcnHq9RdLMBrbzW2BO6i13HPDm\nNO/puBp4laQ90/7HSdq7qsxy4DBJ45V13f6WBrd9sLL3me9AduxXkXXQeFhq9+kA5gFXVK+Y2k12\njYhfAKeQvQLYRhhfWdhQdBbwE0knAD9/mtt4XNK1ZPX+70vzPgN8Bbg+faneTvbuhLoi4o+SFrC1\nS+xvR0RRFVS9bfUqe1/DQklj0uxPkrWdVMr0SDoj7e9+sl56H2xg88uBrwJ7knX7flFEbJF0WpoW\n8POIqNUF/LOA/5W0Uyr3kadzfDa0uddZsyFG0i4R8Ui6srgIOC8iLmp1XDa8uRrKbOj5tKSVZO/9\nuJ2sQdqsVL6yMDOzQr6yMDOzQk4WZmZWyMnCzMwKOVmYmVkhJwszMyv0/wFi/b+sZWjkCQAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2ae40a9da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(neighbors, cv_scores)\n",
    "plt.xlabel('number of neighbors')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.title('knn algorithm prediction accuracy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>It seems like our knn algorithm has the best accuracy when k = 50, and the overall accuracy is 10% higher than our svm model. However, there is a serious potential that our knn model might overfit and it might also consider a lot of outliers in the sample. When the number of sample increases, the potential of overfitting decreases, so we pick 100 as our k</p>\n",
    "\n",
    "<p> When using 50 as our k, I found that it could only recognize soybeans in our southern illinois map. This is a typical behavior of overfitting </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=100, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn_model = KNeighborsClassifier(n_neighbors=100)\n",
    "knn_model.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> Then we can try larger dataset for knn because it trains really fast </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovo', degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svm_model = svm.SVC(decision_function_shape = 'ovo')\n",
    "svm_model.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sample = np.random.choice(population,1000000, False)\n",
    "sample = sorted(sample)\n",
    "X_train_large = np.take(input_data_flatten, sample, 0)\n",
    "Y_train_large = np.take(label_flatten, sample)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='ball_tree', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=1000, p=2,\n",
       "           weights='distance')"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn_model2 = KNeighborsClassifier(n_neighbors=1000,weights = 'distance', algorithm = 'ball_tree')\n",
    "knn_model2.fit(X_train_large, Y_train_large)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> Up to now we have built our there models. a knn-small-sample model, a knn-large-sample model, and a svm-small-sample model </p>\n",
    "<p> The most serious problem of knn is overfitting. Also, it does not have a mathematical calculation of the boundary of different layers. It only uses Euclidean Distance, which is very inaccurate </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Then we want to use our model to predict the southern part of Illinois. However, the dataset is too large, it may take several years to output the result.</p>\n",
    "<p> Observe that there are a lot of large crop fields in Illinois. We can take the average of the color band values of adjacent areas to compress our Image. </p>\n",
    "<p> However, we don't know how large on average a crop field is. Once we obtain the average size of a corn field, then we can compress our satellite image by that factor </p>\n",
    "<p> We are going to use Depth First Search algorithm to obtain the average size of field of each crop type </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 267,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def check_run(input):\n",
    "    index = 0\n",
    "    dict = {}\n",
    "    cur = input[0]\n",
    "    while index < len(input):\n",
    "        count = 0\n",
    "        cur = input[index]\n",
    "        while index < len(input):\n",
    "            if input[index] == cur:\n",
    "                count +=1\n",
    "                index +=1\n",
    "            else:\n",
    "                break\n",
    "        if cur not in dict:\n",
    "            dict[cur] = count\n",
    "        else:\n",
    "            if count > dict[cur]:\n",
    "                dict[cur] = count\n",
    "                \n",
    "\n",
    "    return dict\n",
    "\n",
    "def get_neighbors(pos, shape):\n",
    "    i,j = pos[0] ,pos[1]\n",
    "    if i == 0 and j == 0:\n",
    "        return [[1,0], [0,1]]\n",
    "    if i == shape[0]-1 and j == shape[1] -1:\n",
    "        return [[i-1,j], [i,j-1]]\n",
    "    \n",
    "    if i == 0 and j == shape[1] -1:\n",
    "        return [[i+1,j], [i, j-1]]\n",
    "    if i == shape[0] -1 and j == 0:\n",
    "        return [[i-1, j], [i, j+1]]\n",
    "    if i == 0:\n",
    "        return [[i+1,j], [i, j+1], [i, j-1]]\n",
    "    if i == shape[0] -1:\n",
    "        return [[i,j], [i, j+1], [i-1, j]]\n",
    "    if j == 0:\n",
    "        return [[i, j+1], [i+1, j], [i-1,j]]\n",
    "    if j == shape[1] -1:\n",
    "        return [[i, j-1], [i-1, j], [i+1, j]]\n",
    "    return [[i+1, j], [i-1, j], [i, j+1], [i, j-1]]\n",
    "\n",
    "\n",
    "\n",
    "def DFS(matrix, pos, visited):\n",
    "    i,j = pos[0],pos[1]\n",
    "    count = 1\n",
    "    stack = []\n",
    "    stack.append((i,j))\n",
    "    visited[i,j] = 1\n",
    "    \n",
    "    while stack != []:\n",
    "        cur_position = stack.pop()\n",
    "        i,j = cur_position[0], cur_position[1]\n",
    "        for k in get_neighbors((i,j), matrix.shape):\n",
    "            if visited[k[0], k[1]] != 1 and matrix[k[0], k[1]] == matrix[i,j]:\n",
    "                stack.append(k)\n",
    "                visited[k[0], k[1]] = 1\n",
    "                count += 1\n",
    "    return count\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 287,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{1: 2451, 131: 99, 4: 1, 5: 1583, 141: 526, 13: 2, 152: 7, 195: 2, 24: 360, 26: 39, 28: 33, 58: 4, 222: 3, 59: 5, 36: 174, 37: 208, 43: 13, 111: 182, 176: 2899, 241: 1, 254: 1, 121: 5300, 122: 2829, 123: 738, 124: 118, 61: 12, 190: 400}\n",
      "{1: 11469596, 131: 9311, 4: 42, 5: 9266367, 141: 678453, 13: 91, 152: 491, 195: 98, 24: 142908, 26: 2649, 28: 2994, 58: 394, 222: 189, 59: 280, 36: 49169, 37: 22128, 43: 862, 111: 67975, 176: 1289984, 241: 49, 254: 28, 121: 967228, 122: 833936, 123: 103647, 124: 25390, 61: 660, 190: 65081}\n"
     ]
    }
   ],
   "source": [
    "visited = np.array([[0]*5000]*5000)\n",
    "dict_count_areas = {}\n",
    "dict_total_areas = {}\n",
    "for i in range(5000):\n",
    "    for j in range(5000):\n",
    "        if visited[i,j] == 0:\n",
    "            count = DFS(label_array[:5000,:5000], [i,j], visited)\n",
    "            if label_array[i,j] not in dict_count_areas:\n",
    "                dict_count_areas[label_array[i,j]] = 1\n",
    "            else:\n",
    "                dict_count_areas[label_array[i,j]] += 1\n",
    "            \n",
    "            if label_array[i,j] not in dict_total_areas:\n",
    "                dict_total_areas[label_array[i,j]] = count\n",
    "            else:\n",
    "                dict_total_areas[label_array[i,j]] += count\n",
    "\n",
    "print(dict_count_areas)\n",
    "print(dict_total_areas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 288,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{1: 4679.557731538148, 131: 94.05050505050505, 36: 282.58045977011494, 5: 5853.674668351232, 124: 215.16949152542372, 141: 1289.8346007604562, 176: 444.9755087961366, 24: 396.96666666666664, 121: 182.49584905660376, 122: 294.78119476846945, 123: 140.4430894308943, 28: 90.72727272727273}\n"
     ]
    }
   ],
   "source": [
    "average_areas = {}\n",
    "for i in dict_average_areas:\n",
    "    average_areas[i] = dict_total_areas[i]/dict_count_areas[i]\n",
    "    \n",
    "print(average_areas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>It seems like corn fields on average occupies 4680 pixels, which is 117000m2 on the map. Soy bean fields occupy 5853 pixels, which is 146325m2 on the map. Then we choose our factor to be 400 pixels, which is about 1/12 of the average size of corn fields to prune our image. Our prediction accuracy will not suffer a lot from this pruning</p>\n",
    "\n",
    "<p> The tricky part is that we can divide the whole satellite image into grids, and pick one pixel for each grid. Since the average size of corn fields and soy bean fields is large. If a cornfield exists, if its size is 12 grids, it must be picked by at least 9 grids. The worst ratio for an average cornfield is 0.75, which means that our prediction will at least covers 75% of the area of a cornfield. \n",
    "This ratio is even better for soybeans. So, there is a possibility that we predict more soybeans than actual. </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#start predicting\n",
    "\n",
    "tif3 = gdal.Open('20130824_RE3_3A_Analytic_Champaign_south.tif')\n",
    "band_1 = (tif3.GetRasterBand(1)).ReadAsArray()\n",
    "band_2 = (tif3.GetRasterBand(2)).ReadAsArray()\n",
    "band_3 = (tif3.GetRasterBand(3)).ReadAsArray()\n",
    "band_4 = (tif3.GetRasterBand(4)).ReadAsArray()\n",
    "band_5 = (tif3.GetRasterBand(5)).ReadAsArray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "band_1 = band_1/np.max(band_1)\n",
    "band_2 = band_2/np.max(band_2)\n",
    "band_3 = band_3/np.max(band_3)\n",
    "band_4 = band_4/np.max(band_4)\n",
    "band_5 = band_5/np.max(band_5)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "297 471\n"
     ]
    }
   ],
   "source": [
    "#constructing grids\n",
    "\n",
    "print(5959//20, 9425//20)\n",
    "compressed_data = np.zeros((5959//20,9425//20,5))\n",
    "for i in range(5959//20):\n",
    "    for j in range(9425//20):\n",
    "        compressed_data[i,j,0] = band_1[20*i, 20*j]\n",
    "        compressed_data[i,j,1] = band_2[20*i, 20*j]\n",
    "        compressed_data[i,j,2] = band_3[20*i, 20*j]\n",
    "        compressed_data[i,j,3] = band_4[20*i, 20*j]\n",
    "        compressed_data[i,j,4] = band_5[20*i, 20*j]\n",
    "            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 338,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "flatten1 = band_1.flatten()\n",
    "flatten2 = band_2.flatten()\n",
    "flatten3 = band_3.flatten()\n",
    "flatten4 = band_4.flatten()\n",
    "flatten5 = band_5.flatten()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#reshape data to fit in our model\n",
    "compressed_data_flatten = compressed_data.reshape(((5959//20)*(9425//20)),5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#first we use knn to predict\n",
    "prediction_south = knn_model.predict(compressed_data_flatten)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 10267.,      0.,      0.,      0.,      0.,  37357.,      0.,\n",
       "             0.,      0.,  92263.]),\n",
       " array([ 1. ,  1.4,  1.8,  2.2,  2.6,  3. ,  3.4,  3.8,  4.2,  4.6,  5. ]),\n",
       " <a list of 10 Patch objects>)"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEI9JREFUeJzt3X+s3XV9x/Hnay0o/gKUhrGWWRKbLZXMiQ3WsBgDGxQw\nlGRoajaphtls4tRtiQP/GJlKAskiyuaPEOgsTgWCOjooMgKYZX9QufyYWJBxgyhtUCrlh84pq773\nx/mgx37u5Z7b9t5zoc9HcnK/38/n8/1+3+fTnr7u+X6/5zRVhSRJw35j3AVIkhYew0GS1DEcJEkd\nw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEmdxeMuYG8dccQRtXz58nGXIUnPG3feeecPq2rJKGOf\nt+GwfPlyJiYmxl2GJD1vJPnuqGM9rSRJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO\n4SBJ6jxvPyEtSeO0/LwbxnLchy86fV6O4zsHSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAkdQwHSVLH\ncJAkdQwHSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAk\ndQwHSVLHcJAkdQwHSVJnpHBI8ldJtiX5VpIvJXlxkmOSbE0ymeTqJAe3sS9q65Otf/nQfs5v7Q8k\nOWWofU1rm0xy3v5+kpKk2ZkxHJIsBd4PrKqqY4FFwDrgYuCSqnoN8ARwTtvkHOCJ1n5JG0eSlW27\n1wJrgE8nWZRkEfAp4FRgJfCONlaSNCajnlZaDBySZDHwEuBR4ETg2ta/CTizLa9t67T+k5KktV9V\nVT+rqu8Ak8Dx7TFZVQ9V1TPAVW2sJGlMZgyHqtoB/APwPQah8BRwJ/BkVe1uw7YDS9vyUuCRtu3u\nNv5Vw+17bDNdeyfJhiQTSSZ27tw5yvOTJO2FUU4rHc7gN/ljgN8CXsrgtNC8q6rLqmpVVa1asmTJ\nOEqQpAPCKKeV/hD4TlXtrKr/A74CnAAc1k4zASwDdrTlHcDRAK3/UODx4fY9tpmuXZI0JqOEw/eA\n1Ule0q4dnATcB9wGnNXGrAeua8ub2zqt/9aqqta+rt3NdAywAvgGcAewot39dDCDi9ab9/2pSZL2\n1uKZBlTV1iTXAncBu4G7gcuAG4CrknystV3RNrkC+HySSWAXg3/sqaptSa5hECy7gXOr6ucASd4H\n3MTgTqiNVbVt/z1FSdJszRgOAFV1AXDBHs0PMbjTaM+xPwXeNs1+LgQunKJ9C7BllFokSXPPT0hL\nkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqG\ngySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySp\nYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjojhUOSw5Jcm+TbSe5P8qYk\nr0xyc5IH28/D29gkuTTJZJJvJjluaD/r2/gHk6wfan9DknvbNpcmyf5/qpKkUY36zuGTwNeq6neB\n1wH3A+cBt1TVCuCWtg5wKrCiPTYAnwFI8krgAuCNwPHABc8GShvznqHt1uzb05Ik7YsZwyHJocCb\ngSsAquqZqnoSWAtsasM2AWe25bXAlTVwO3BYkqOAU4Cbq2pXVT0B3AysaX2vqKrbq6qAK4f2JUka\ng1HeORwD7AT+OcndSS5P8lLgyKp6tI35PnBkW14KPDK0/fbW9lzt26dolySNySjhsBg4DvhMVb0e\n+B9+dQoJgPYbf+3/8n5dkg1JJpJM7Ny5c64PJ0kHrFHCYTuwvaq2tvVrGYTFD9opIdrPx1r/DuDo\noe2Xtbbnal82RXunqi6rqlVVtWrJkiUjlC5J2hszhkNVfR94JMnvtKaTgPuAzcCzdxytB65ry5uB\ns9tdS6uBp9rpp5uAk5Mc3i5Enwzc1PqeTrK63aV09tC+JEljsHjEcX8JfCHJwcBDwLsZBMs1Sc4B\nvgu8vY3dApwGTAI/aWOpql1JPgrc0cZ9pKp2teX3Ap8DDgFubA9J0piMFA5VdQ+waoquk6YYW8C5\n0+xnI7BxivYJ4NhRapEkzT0/IS1J6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ\n6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgO\nkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO\n4SBJ6hgOkqTOyOGQZFGSu5Nc39aPSbI1yWSSq5Mc3Npf1NYnW//yoX2c39ofSHLKUPua1jaZ5Lz9\n9/QkSXtjNu8cPgDcP7R+MXBJVb0GeAI4p7WfAzzR2i9p40iyElgHvBZYA3y6Bc4i4FPAqcBK4B1t\nrCRpTEYKhyTLgNOBy9t6gBOBa9uQTcCZbXltW6f1n9TGrwWuqqqfVdV3gEng+PaYrKqHquoZ4Ko2\nVpI0JqO+c/gE8CHgF239VcCTVbW7rW8HlrblpcAjAK3/qTb+l+17bDNduyRpTGYMhyRvBR6rqjvn\noZ6ZatmQZCLJxM6dO8ddjiS9YI3yzuEE4IwkDzM45XMi8EngsCSL25hlwI62vAM4GqD1Hwo8Pty+\nxzbTtXeq6rKqWlVVq5YsWTJC6ZKkvTFjOFTV+VW1rKqWM7igfGtV/QlwG3BWG7YeuK4tb27rtP5b\nq6pa+7p2N9MxwArgG8AdwIp299PB7Rib98uzkyTtlcUzD5nW3wJXJfkYcDdwRWu/Avh8kklgF4N/\n7KmqbUmuAe4DdgPnVtXPAZK8D7gJWARsrKpt+1CXJGkfzSocqurrwNfb8kMM7jTac8xPgbdNs/2F\nwIVTtG8BtsymFknS3PET0pKkzr6cVpI0heXn3TCW4z580eljOa5emHznIEnqGA6SpI7hIEnqGA6S\npI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7h\nIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnq\nGA6SpI7hIEnqGA6SpI7hIEnqGA6SpM6M4ZDk6CS3JbkvybYkH2jtr0xyc5IH28/DW3uSXJpkMsk3\nkxw3tK/1bfyDSdYPtb8hyb1tm0uTZC6erCRpNKO8c9gN/E1VrQRWA+cmWQmcB9xSVSuAW9o6wKnA\nivbYAHwGBmECXAC8ETgeuODZQGlj3jO03Zp9f2qSpL01YzhU1aNVdVdb/hFwP7AUWAtsasM2AWe2\n5bXAlTVwO3BYkqOAU4Cbq2pXVT0B3AysaX2vqKrbq6qAK4f2JUkag1ldc0iyHHg9sBU4sqoebV3f\nB45sy0uBR4Y2297anqt9+xTtkqQxGTkckrwM+DLwwap6eriv/cZf+7m2qWrYkGQiycTOnTvn+nCS\ndMAaKRySHMQgGL5QVV9pzT9op4RoPx9r7TuAo4c2X9banqt92RTtnaq6rKpWVdWqJUuWjFK6JGkv\njHK3UoArgPur6uNDXZuBZ+84Wg9cN9R+drtraTXwVDv9dBNwcpLD24Xok4GbWt/TSVa3Y509tC9J\n0hgsHmHMCcA7gXuT3NPaPgxcBFyT5Bzgu8DbW98W4DRgEvgJ8G6AqtqV5KPAHW3cR6pqV1t+L/A5\n4BDgxvaQJI3JjOFQVf8JTPe5g5OmGF/AudPsayOwcYr2CeDYmWqRJM0PPyEtSeoYDpKkjuEgSeoY\nDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKk\njuEgSeqM8t+EvuAsP++GsRz34YtOH8txJWm2fOcgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEg\nSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkzoIJhyRr\nkjyQZDLJeeOuR5IOZAsiHJIsAj4FnAqsBN6RZOV4q5KkA9eCCAfgeGCyqh6qqmeAq4C1Y65Jkg5Y\nCyUclgKPDK1vb22SpDFYPO4CZiPJBmBDW/1xkgf2cldHAD/cP1WNLhfPOGQsdY3AumbHv1+zY12z\nkIv3qa5XjzpwoYTDDuDoofVlre3XVNVlwGX7erAkE1W1al/3s79Z1+xY1+xY1+wc6HUtlNNKdwAr\nkhyT5GBgHbB5zDVJ0gFrQbxzqKrdSd4H3AQsAjZW1bYxlyVJB6wFEQ4AVbUF2DJPh9vnU1NzxLpm\nx7pmx7pm54CuK1U1H8eRJD2PLJRrDpKkBeQFGw5JNiZ5LMm3pulPkkvb13V8M8lxC6SutyR5Ksk9\n7fF381TX0UluS3Jfkm1JPjDFmHmfsxHrmvc5S/LiJN9I8l+trr+fYsyLklzd5mtrkuULpK53Jdk5\nNF9/Ntd1DR17UZK7k1w/Rd+8z9eIdY1lvpI8nOTedsyJKfrn9vVYVS/IB/Bm4DjgW9P0nwbcCARY\nDWxdIHW9Bbh+DPN1FHBcW3458N/AynHP2Yh1zfuctTl4WVs+CNgKrN5jzHuBz7bldcDVC6SudwH/\nNN9/x9qx/xr44lR/XuOYrxHrGst8AQ8DRzxH/5y+Hl+w7xyq6j+AXc8xZC1wZQ3cDhyW5KgFUNdY\nVNWjVXVXW/4RcD/9p9Tnfc5GrGvetTn4cVs9qD32vIC3FtjUlq8FTkqSBVDXWCRZBpwOXD7NkHmf\nrxHrWqjm9PX4gg2HESzkr+x4UzstcGOS1873wdvb+dcz+K1z2Fjn7DnqgjHMWTsVcQ/wGHBzVU07\nX1W1G3gKeNUCqAvgj9upiGuTHD1F/1z4BPAh4BfT9I9lvkaoC8YzXwX8e5I7M/h2iD3N6evxQA6H\nheou4NVV9TrgH4F/nc+DJ3kZ8GXgg1X19Hwe+7nMUNdY5qyqfl5Vv8/gE/3HJzl2Po47kxHq+jdg\neVX9HnAzv/ptfc4keSvwWFXdOdfHmo0R65r3+Wr+oKqOY/Bt1ecmefM8HRc4sMNhpK/smG9V9fSz\npwVq8NmPg5IcMR/HTnIQg3+Av1BVX5liyFjmbKa6xjln7ZhPArcBa/bo+uV8JVkMHAo8Pu66qurx\nqvpZW70ceMM8lHMCcEaShxl86/KJSf5ljzHjmK8Z6xrTfFFVO9rPx4CvMvj26mFz+no8kMNhM3B2\nu+K/Gniqqh4dd1FJfvPZ86xJjmfwZzTn/6C0Y14B3F9VH59m2LzP2Sh1jWPOkixJclhbPgT4I+Db\newzbDKxvy2cBt1a7kjjOuvY4L30Gg+s4c6qqzq+qZVW1nMHF5lur6k/3GDbv8zVKXeOYryQvTfLy\nZ5eBk4E973Cc09fjgvmE9P6W5EsM7mI5Isl24AIGF+eoqs8y+DT2acAk8BPg3QukrrOAv0iyG/hf\nYN1cv0CaE4B3Ave289UAHwZ+e6i2cczZKHWNY86OAjZl8B9V/QZwTVVdn+QjwERVbWYQap9PMsng\nJoR1c1zTqHW9P8kZwO5W17vmoa4pLYD5GqWucczXkcBX2+88i4EvVtXXkvw5zM/r0U9IS5I6B/Jp\nJUnSNAwHSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAkdQwHSVLn/wFiGSLkdWt18gAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2ae362d080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(prediction_south)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The histogram shows that We have a lot of soybeans in southern illinois, the number of soybeans is 9 times the number of corns. We have mentioned that knn has several problems before.\n",
    "Also, our cross validation only shows 75% accuracy. This is still reasonable</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "prediction_south2 = svm_model.predict(compressed_data_flatten)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([69957, 12421, 57509])"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unique2,count2 = np.unique(prediction_south2, return_counts = True)\n",
    "count2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 69957.,      0.,      0.,      0.,      0.,  12421.,      0.,\n",
       "             0.,      0.,  57509.]),\n",
       " array([ 1. ,  1.4,  1.8,  2.2,  2.6,  3. ,  3.4,  3.8,  4.2,  4.6,  5. ]),\n",
       " <a list of 10 Patch objects>)"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFUdJREFUeJzt3WGMXfV55/HvLxgSlIQYwqwX2aZGqtWKoA0BizhKFWWD\nYgypMNKmEahbO4iNdxeym2hX6jp9sVZJI5E3TctuSoWCN3aahCBaFjcxcS2gqvaFCUOgECBZphSE\nLcAuJtCUbSLSZ1/cv5sb/2c8d2zP3An+fqSre85z/uec5x57/Jt7zrnXqSokSRr2pnE3IElafAwH\nSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAkdZaMu4FjdfbZZ9eqVavG3YYk/cJ46KGH\n/q6qJkYZ+wsbDqtWrWJycnLcbUjSL4wkz4461tNKkqSO4SBJ6hgOkqSO4SBJ6hgOkqTOrOGQ5FeS\nPDL0eDXJp5OclWRPkqfa85ltfJLcnGQqyaNJLhra1qY2/qkkm4bqFyd5rK1zc5LMz8uVJI1i1nCo\nqh9U1YVVdSFwMfAacBewBbi3qlYD97Z5gMuB1e2xGbgFIMlZwFbgvcAlwNbDgdLGfGJovfUn5NVJ\nko7JXE8rXQr8TVU9C2wAtrf6duCqNr0B2FEDe4GlSc4BLgP2VNWhqnoZ2AOsb8vOqKq9Nfg/S3cM\nbUuSNAZzDYerga+36WVV9XybfgFY1qaXA88NrbOv1Y5W3zdNXZI0JiN/QjrJacCVwGeOXFZVlaRO\nZGMz9LCZwakqzj333GPezqot3zpRLc3JMzd9ZCz7laS5mss7h8uB71bVi23+xXZKiPZ8oNX3AyuH\n1lvRakerr5im3qmqW6tqTVWtmZgY6etBJEnHYC7hcA0/O6UEsBM4fMfRJuDuofrGdtfSWuCVdvpp\nN7AuyZntQvQ6YHdb9mqSte0upY1D25IkjcFIp5WSvBX4MPDvh8o3AXckuQ54FvhYq+8CrgCmGNzZ\ndC1AVR1K8lngwTbuxqo61KavB74MnA7c0x6SpDEZKRyq6h+Adx5Re4nB3UtHji3ghhm2sw3YNk19\nErhglF4kSfPPT0hLkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhI\nkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjojhUOSpUnuTPL9\nJE8meV+Ss5LsSfJUez6zjU2Sm5NMJXk0yUVD29nUxj+VZNNQ/eIkj7V1bk6SE/9SJUmjGvWdwx8C\n366qXwXeDTwJbAHurarVwL1tHuByYHV7bAZuAUhyFrAVeC9wCbD1cKC0MZ8YWm/98b0sSdLxmDUc\nkrwD+ABwG0BV/aSqfghsALa3YduBq9r0BmBHDewFliY5B7gM2FNVh6rqZWAPsL4tO6Oq9lZVATuG\ntiVJGoNR3jmcBxwE/leSh5N8KclbgWVV9Xwb8wKwrE0vB54bWn9fqx2tvm+aeifJ5iSTSSYPHjw4\nQuuSpGMxSjgsAS4Cbqmq9wD/wM9OIQHQfuOvE9/ez6uqW6tqTVWtmZiYmO/dSdJJa5Rw2Afsq6oH\n2vydDMLixXZKiPZ8oC3fD6wcWn9Fqx2tvmKauiRpTGYNh6p6AXguya+00qXAE8BO4PAdR5uAu9v0\nTmBju2tpLfBKO/20G1iX5Mx2IXodsLstezXJ2naX0sahbUmSxmDJiOP+E/DVJKcBTwPXMgiWO5Jc\nBzwLfKyN3QVcAUwBr7WxVNWhJJ8FHmzjbqyqQ236euDLwOnAPe0hSRqTkcKhqh4B1kyz6NJpxhZw\nwwzb2QZsm6Y+CVwwSi+SpPnnJ6QlSR3DQZLUGfWagyRpyKot3xrLfp+56SMLsh/fOUiSOoaDJKlj\nOEiSOoaDJKljOEiSOoaDJKljOEiSOoaDJKljOEiSOoaDJKljOEiSOoaDJKljOEiSOoaDJKljOEiS\nOoaDJKljOEiSOiOFQ5JnkjyW5JEkk612VpI9SZ5qz2e2epLcnGQqyaNJLhrazqY2/qkkm4bqF7ft\nT7V1c6JfqCRpdHN55/Cvq+rCqlrT5rcA91bVauDeNg9wObC6PTYDt8AgTICtwHuBS4CthwOljfnE\n0Hrrj/kVSZKO2/GcVtoAbG/T24Grhuo7amAvsDTJOcBlwJ6qOlRVLwN7gPVt2RlVtbeqCtgxtC1J\n0hiMGg4F/EWSh5JsbrVlVfV8m34BWNamlwPPDa27r9WOVt83TV2SNCZLRhz3a1W1P8m/APYk+f7w\nwqqqJHXi2/t5LZg2A5x77rnzvTtJOmmN9M6hqva35wPAXQyuGbzYTgnRng+04fuBlUOrr2i1o9VX\nTFOfro9bq2pNVa2ZmJgYpXVJ0jGYNRySvDXJ2w9PA+uA7wE7gcN3HG0C7m7TO4GN7a6ltcAr7fTT\nbmBdkjPbheh1wO627NUka9tdShuHtiVJGoNRTistA+5qd5cuAb5WVd9O8iBwR5LrgGeBj7Xxu4Ar\ngCngNeBagKo6lOSzwINt3I1VdahNXw98GTgduKc9JEljMms4VNXTwLunqb8EXDpNvYAbZtjWNmDb\nNPVJ4IIR+pUkLQA/IS1J6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO\n4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6owcDklOSfJw\nkm+2+fOSPJBkKsk3kpzW6m9u81Nt+aqhbXym1X+Q5LKh+vpWm0qy5cS9PEnSsZjLO4dPAU8OzX8e\n+EJV/TLwMnBdq18HvNzqX2jjSHI+cDXwLmA98EctcE4BvghcDpwPXNPGSpLGZKRwSLIC+AjwpTYf\n4EPAnW3IduCqNr2hzdOWX9rGbwBur6ofV9XfAlPAJe0xVVVPV9VPgNvbWEnSmIz6zuEPgN8G/qnN\nvxP4YVW93ub3Acvb9HLgOYC2/JU2/p/rR6wzU72TZHOSySSTBw8eHLF1SdJczRoOSX4dOFBVDy1A\nP0dVVbdW1ZqqWjMxMTHudiTpDWvJCGPeD1yZ5ArgLcAZwB8CS5Msae8OVgD72/j9wEpgX5IlwDuA\nl4bqhw2vM1NdkjQGs75zqKrPVNWKqlrF4ILyfVX1m8D9wEfbsE3A3W16Z5unLb+vqqrVr253M50H\nrAa+AzwIrG53P53W9rHzhLw6SdIxGeWdw0z+G3B7kt8DHgZua/XbgK8kmQIOMfjHnqp6PMkdwBPA\n68ANVfVTgCSfBHYDpwDbqurx4+hLknSc5hQOVfWXwF+26acZ3Gl05Jh/BH5jhvU/B3xumvouYNdc\nepEkzR8/IS1J6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgO\nkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqTOrOGQ5C1JvpPkr5M8nuR3W/28\nJA8kmUryjSSntfqb2/xUW75qaFufafUfJLlsqL6+1aaSbDnxL1OSNBejvHP4MfChqno3cCGwPsla\n4PPAF6rql4GXgeva+OuAl1v9C20cSc4HrgbeBawH/ijJKUlOAb4IXA6cD1zTxkqSxmTWcKiBH7XZ\nU9ujgA8Bd7b6duCqNr2hzdOWX5okrX57Vf24qv4WmAIuaY+pqnq6qn4C3N7GSpLGZKRrDu03/EeA\nA8Ae4G+AH1bV623IPmB5m14OPAfQlr8CvHO4fsQ6M9UlSWMyUjhU1U+r6kJgBYPf9H91XruaQZLN\nSSaTTB48eHAcLUjSSWFOdytV1Q+B+4H3AUuTLGmLVgD72/R+YCVAW/4O4KXh+hHrzFSfbv+3VtWa\nqlozMTExl9YlSXMwyt1KE0mWtunTgQ8DTzIIiY+2YZuAu9v0zjZPW35fVVWrX93uZjoPWA18B3gQ\nWN3ufjqNwUXrnSfixUmSjs2S2YdwDrC93VX0JuCOqvpmkieA25P8HvAwcFsbfxvwlSRTwCEG/9hT\nVY8nuQN4AngduKGqfgqQ5JPAbuAUYFtVPX7CXqEkac5mDYeqehR4zzT1pxlcfziy/o/Ab8ywrc8B\nn5umvgvYNUK/kqQF4CekJUkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS\n1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEmdWcMhycok\n9yd5IsnjST7V6mcl2ZPkqfZ8Zqsnyc1JppI8muSioW1tauOfSrJpqH5xksfaOjcnyXy8WEnSaEZ5\n5/A68F+r6nxgLXBDkvOBLcC9VbUauLfNA1wOrG6PzcAtMAgTYCvwXuASYOvhQGljPjG03vrjf2mS\npGM1azhU1fNV9d02/ffAk8ByYAOwvQ3bDlzVpjcAO2pgL7A0yTnAZcCeqjpUVS8De4D1bdkZVbW3\nqgrYMbQtSdIYzOmaQ5JVwHuAB4BlVfV8W/QCsKxNLweeG1ptX6sdrb5vmrokaUxGDockbwP+FPh0\nVb06vKz9xl8nuLfpeticZDLJ5MGDB+d7d5J00hopHJKcyiAYvlpVf9bKL7ZTQrTnA62+H1g5tPqK\nVjtafcU09U5V3VpVa6pqzcTExCitS5KOwSh3KwW4DXiyqn5/aNFO4PAdR5uAu4fqG9tdS2uBV9rp\np93AuiRntgvR64DdbdmrSda2fW0c2pYkaQyWjDDm/cBvAY8leaTVfge4CbgjyXXAs8DH2rJdwBXA\nFPAacC1AVR1K8lngwTbuxqo61KavB74MnA7c0x6SpDGZNRyq6v8AM33u4NJpxhdwwwzb2gZsm6Y+\nCVwwWy+SpIXhJ6QlSR3DQZLUMRwkSR3DQZLUMRwkSR3DQZLUMRwkSR3DQZLUMRwkSR3DQZLUMRwk\nSR3DQZLUMRwkSR3DQZLUMRwkSR3DQZLUMRwkSZ1R/ptQSXOwasu3xrLfZ276yFj2qzcm3zlIkjqG\ngySpYzhIkjqzhkOSbUkOJPneUO2sJHuSPNWez2z1JLk5yVSSR5NcNLTOpjb+qSSbhuoXJ3msrXNz\nkpzoFylJmptR3jl8GVh/RG0LcG9VrQbubfMAlwOr22MzcAsMwgTYCrwXuATYejhQ2phPDK135L4k\nSQts1nCoqr8CDh1R3gBsb9PbgauG6jtqYC+wNMk5wGXAnqo6VFUvA3uA9W3ZGVW1t6oK2DG0LUnS\nmBzrNYdlVfV8m34BWNamlwPPDY3b12pHq++bpj6tJJuTTCaZPHjw4DG2LkmazXFfkG6/8dcJ6GWU\nfd1aVWuqas3ExMRC7FKSTkrHGg4vtlNCtOcDrb4fWDk0bkWrHa2+Ypq6JGmMjjUcdgKH7zjaBNw9\nVN/Y7lpaC7zSTj/tBtYlObNdiF4H7G7LXk2ytt2ltHFoW5KkMZn16zOSfB34IHB2kn0M7jq6Cbgj\nyXXAs8DH2vBdwBXAFPAacC1AVR1K8lngwTbuxqo6fJH7egZ3RJ0O3NMekqQxmjUcquqaGRZdOs3Y\nAm6YYTvbgG3T1CeBC2brQ5K0cPyEtCSpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqG\ngySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqLJhyS\nrE/ygyRTSbaMux9JOpktinBIcgrwReBy4HzgmiTnj7crSTp5LYpwAC4Bpqrq6ar6CXA7sGHMPUnS\nSWuxhMNy4Lmh+X2tJkkagyXjbmAukmwGNrfZHyX5wTFu6mzg705MV6PL52cdMpa+RmBfc+Pfr7mx\nrznI54+rr18adeBiCYf9wMqh+RWt9nOq6lbg1uPdWZLJqlpzvNs50exrbuxrbuxrbk72vhbLaaUH\ngdVJzktyGnA1sHPMPUnSSWtRvHOoqteTfBLYDZwCbKuqx8fcliSdtBZFOABU1S5g1wLt7rhPTc0T\n+5ob+5ob+5qbk7qvVNVC7EeS9AtksVxzkCQtIm/YcEiyLcmBJN+bYXmS3Ny+ruPRJBctkr4+mOSV\nJI+0x39foL5WJrk/yRNJHk/yqWnGLPgxG7GvBT9mSd6S5DtJ/rr19bvTjHlzkm+04/VAklWLpK+P\nJzk4dLz+3Xz3NbTvU5I8nOSb0yxb8OM1Yl9jOV5JnknyWNvn5DTL5/fnsarekA/gA8BFwPdmWH4F\ncA8QYC3wwCLp64PAN8dwvM4BLmrTbwf+L3D+uI/ZiH0t+DFrx+BtbfpU4AFg7RFjrgf+uE1fDXxj\nkfT1ceB/LvTfsbbv/wJ8bbo/r3EcrxH7GsvxAp4Bzj7K8nn9eXzDvnOoqr8CDh1lyAZgRw3sBZYm\nOWcR9DUWVfV8VX23Tf898CT9p9QX/JiN2NeCa8fgR2321PY48gLeBmB7m74TuDRJFkFfY5FkBfAR\n4EszDFnw4zViX4vVvP48vmHDYQSL+Ss73tdOC9yT5F0LvfP2dv49DH7rHDbWY3aUvmAMx6ydingE\nOADsqaoZj1dVvQ68ArxzEfQF8G/aqYg7k6ycZvl8+APgt4F/mmH5WI7XCH3BeI5XAX+R5KEMvh3i\nSPP683gyh8Ni9V3gl6rq3cD/AP73Qu48yduAPwU+XVWvLuS+j2aWvsZyzKrqp1V1IYNP9F+S5IKF\n2O9sRujrz4FVVfWvgD387Lf1eZPk14EDVfXQfO9rLkbsa8GPV/NrVXURg2+rviHJBxZov8DJHQ4j\nfWXHQquqVw+fFqjBZz9OTXL2Quw7yakM/gH+alX92TRDxnLMZutrnMes7fOHwP3A+iMW/fPxSrIE\neAfw0rj7qqqXqurHbfZLwMUL0M77gSuTPMPgW5c/lORPjhgzjuM1a19jOl5U1f72fAC4i8G3Vw+b\n15/HkzkcdgIb2xX/tcArVfX8uJtK8i8Pn2dNcgmDP6N5/wel7fM24Mmq+v0Zhi34MRulr3EcsyQT\nSZa26dOBDwPfP2LYTmBTm/4ocF+1K4nj7OuI89JXMriOM6+q6jNVtaKqVjG42HxfVf3bI4Yt+PEa\npa9xHK8kb03y9sPTwDrgyDsc5/XncdF8QvpES/J1BnexnJ1kH7CVwcU5quqPGXwa+wpgCngNuHaR\n9PVR4D8meR34f8DV8/0D0rwf+C3gsXa+GuB3gHOHehvHMRulr3Ecs3OA7Rn8R1VvAu6oqm8muRGY\nrKqdDELtK0mmGNyEcPU89zRqX/85yZXA662vjy9AX9NaBMdrlL7GcbyWAXe133mWAF+rqm8n+Q+w\nMD+PfkJaktQ5mU8rSZJmYDhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjr/H10XGldPka0Z\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2ae36a1278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(prediction_south2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>svm model seems to have a good gauge of the different crop types in southern illinois. It agrees with our common sense </p>\n",
    "<p> Our large sample knn fails. It only output one type of crop. The reason might be that it overfits in large sample </p>\n",
    "\n",
    "<p> Then we plot our two predictions in yellow (for corn), green (for soybeans) and black (for others) </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "prediction2 = prediction_south2.reshape((5959//20),(9425//20))\n",
    "prediction = prediction_south.reshape((5959//20),(9425//20))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "dict = {1: [0.0,255.0,255.0],5: [0.0,255.0,0.0],3: [0.0,0.0,0.0]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "img = np.zeros((5929,9425,3))\n",
    "img2 = np.zeros((5929,9425,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for i in range(5959//20):\n",
    "    for j in range(9425//20):\n",
    "        if 20*i+20 < 5928 and 20*j+20 < 9424:\n",
    "            img[20*i:20*i+20, 20*j:20*j+20] = dict[prediction2[i,j]]\n",
    "        else:\n",
    "            img[20*i:,20*j:] = dict[prediction2[i,j]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import cv2\n",
    "cv2.imwrite('prediction_svm_rbf.png', img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(5959//20):\n",
    "    for j in range(9425//20):\n",
    "        if 20*i+20 < 5928 and 20*j+20 < 9424:\n",
    "            img2[20*i:20*i+20, 20*j:20*j+20] = dict[prediction[i,j]]\n",
    "        else:\n",
    "            img2[20*i:,20*j:] = dict[prediction[i,j]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv2.imwrite('prediction_knn_100.png', img2)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}