intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# MSAIL Tutorial Series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "## Intro to ML"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Some Definitions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "__Artificial Intelligence__\n",
    "* _definition_: the theory and development of computer systems able to perform tasks that normally require human intelligence\n",
    "* _examples_:\n",
    "    * __search algorithms__ - pathfinding, graph traversal\n",
    "    * __constraint satisfaction problems__ - map coloring, concert scheduling\n",
    "    * __logic__ - resolution refutation, prolog\n",
    "    * __planning__ - graph planning given some initial state\n",
    "    * __machine learning__ - the interesting stuff..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "__Machine Learning__\n",
    "* _informal definition_: Algorithms that improve their prediction performance at some task with experience (or data).\n",
    "<img src=\"ml.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Categories of ML Algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "__Supervised Learning__\n",
    "* _goal_: given the data X in some feature space and the labels Y, learn to predict Y from X\n",
    "* regression (continuous labels) - stock market prediction, Airbnb\n",
    "* classification (discrete labels) - digit recognition, object recognition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "__Unsupervised Learning__\n",
    "* _goal_: given the data X without any labels, learn the _structures_ of the data\n",
    "* clustering - image compression\n",
    "* component analysis - dimensionality reduction, reconstruction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "__Reinforcement Learning__\n",
    "* _setting_: given a sequence of states X and \"rewards\", agent has to take actions A for each time step\n",
    "* _goal_: how to \"learn to act\" or \"making decisions\" to maximize the sum of future rewards"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Development Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "__IPython Notebooks__\n",
    "* http://continuum.io/downloads\n",
    "* Anaconda is really nice because it comes with a bunch of commonly used Python packages for data science all bundled together"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "__Kaggle__\n",
    "* https://www.kaggle.com/c/titanic\n",
    "* Go ahead and make an account on Kaggle\n",
    "* Download the datasets (train.csv and test.csv) for the Titantic competition\n",
    "* Place the datasets in some directory where you would like to work on MSAIL stuff from"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "__MSAIL Curriculum__\n",
    "* https://github.com/MSAIL/Curriculum\n",
    "* Start by downloading this IPython notebook, start up Anaconda, and opening this intro.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Import some libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "import numpy as np\n",
    "import random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Open the file with the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "with open('train.csv', 'rb') as tf:\n",
    "    csv_file_object = csv.reader(tf, delimiter=',')\n",
    "    header = csv_file_object.next()\n",
    "    data=[]\n",
    "    for row in csv_file_object:\n",
    "        data.append(row)\n",
    "    data = np.array(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Open the file with the training data ~pythonically~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "with open('train.csv', 'rb') as tf:\n",
    "    csv_file_object = csv.reader(tf, delimiter=',')\n",
    "    header = csv_file_object.next()\n",
    "    data = np.array([row for row in csv_file_object])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Explore the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['1' '0' '3' 'Braund, Mr. Owen Harris' 'male' '22' '1' '0' 'A/5 21171'\n",
      " '7.25' '' 'S']\n",
      "['891' '0' '3' 'Dooley, Mr. Patrick' 'male' '32' '0' '0' '370376' '7.75' ''\n",
      " 'Q']\n"
     ]
    }
   ],
   "source": [
    "print data[0]\n",
    "print data[-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's see how many passengers there were, how many survived, and then the ratio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "891 342.0 0.383838383838\n"
     ]
    }
   ],
   "source": [
    "number_passengers = np.size(data[:,1].astype(np.float))\n",
    "number_survived = np.sum(data[:,1].astype(np.float))\n",
    "proportion_survivors = number_survived / number_passengers\n",
    "print number_passengers, number_survived, proportion_survivors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's filter the data by the passenger classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class_data = data[:,2].astype(np.float)\n",
    "class_1_stats = class_data == 1\n",
    "class_2_stats = class_data == 2\n",
    "class_3_stats = class_data == 3\n",
    "\n",
    "class_1_onboard = data[class_1_stats,1].astype(np.float)\n",
    "class_2_onboard = data[class_2_stats,1].astype(np.float)\n",
    "class_3_onboard = data[class_3_stats,1].astype(np.float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "For each class (1-3), let's see what percent of people from those classes survived"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Proportion of class 1 who survived is 0.62962962963\n",
      "Proportion of class 2 who survived is 0.472826086957\n",
      "Proportion of class 3 who survived is 0.242362525458\n"
     ]
    }
   ],
   "source": [
    "proportion_class_1_survived = \\\n",
    "                       np.sum(class_1_onboard) / np.size(class_1_onboard)  \n",
    "proportion_class_2_survived = \\\n",
    "                       np.sum(class_2_onboard) / np.size(class_2_onboard) \n",
    "proportion_class_3_survived = \\\n",
    "                       np.sum(class_3_onboard) / np.size(class_3_onboard) \n",
    "\n",
    "print 'Proportion of class 1 who survived is %s' % proportion_class_1_survived\n",
    "print 'Proportion of class 2 who survived is %s' % proportion_class_2_survived\n",
    "print 'Proportion of class 3 who survived is %s' % proportion_class_3_survived"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Open the file with the test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "with open('test.csv', 'rb') as tf:\n",
    "    csv_file_object = csv.reader(tf, delimiter=',')\n",
    "    header = csv_file_object.next()\n",
    "    data = np.array([row for row in csv_file_object])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's predict the survival of a passenger based solely on which class they were in, write these predictions to a csv file, and submit to Kaggle!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "with open(\"classbasedmodel.csv\", \"wb\") as pf:\n",
    "    predictions = csv.writer(pf, delimiter=',')\n",
    "    predictions.writerow([\"PassengerId\", \"Survived\"])\n",
    "    data[:,-1] = data[:,1].astype(np.float)\n",
    "    for example in data:\n",
    "        if example[-1] == 1:\n",
    "            predictions.writerow([example[0], '1'])\n",
    "        elif example[-1] == 3:\n",
    "            predictions.writerow([example[0], '0'])\n",
    "        else:\n",
    "            prob = random.uniform(0, 1)\n",
    "            if prob < 0.5:\n",
    "                predictions.writerow([example[0], '0'])\n",
    "            else:\n",
    "                predictions.writerow([example[0], '1'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}