DL0101EN-4-1-Convolutional-Neural-Networks-with-Keras-py-v1.0.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://cognitiveclass.ai\"><img src = \"https://ibm.box.com/shared/static/9gegpsmnsoo25ikkbl4qzlvlyjbgxs5x.png\" width = 400> </a>\n",
    "\n",
    "<h1 align=center><font size = 5>Convolutional Neural Networks with Keras</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lab, we will learn how to use the keras library to build convolutional neural networks. We will also use the popular MNIST dataset and we will compare our results to using a conventional neural network.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents\n",
    "\n",
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "\n",
    "<font size = 3>\n",
    "       \n",
    "2. <a href=\"#item2\">Import Keras and Packages</a>   \n",
    "\n",
    "3. <a href=\"#item3\">Convolutional Neural Network with One Convolutional and Pooling Layers</a>  \n",
    "\n",
    "4. <a href=\"#item4\">Convolutional Neural Network with Two Convolutional and Pooling Layers</a>  \n",
    "\n",
    "</font>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Keras and Packages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by importing the keras libraries and the packages that we would need to build a neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import keras\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense\n",
    "from keras.utils import to_categorical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When working with convolutional neural networks in particular, we will need additional packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers.convolutional import Conv2D # to add convolutional layers\n",
    "from keras.layers.convolutional import MaxPooling2D # to add pooling layers\n",
    "from keras.layers import Flatten # to flatten data for fully connected layers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convolutional Layer with two sets of convolutional and pooling layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import data\n",
    "from keras.datasets import mnist\n",
    "\n",
    "# load data\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "\n",
    "# reshape to be [samples][pixels][width][height]\n",
    "X_train = X_train.reshape(X_train.shape[0], 28, 28, 1).astype('float32')\n",
    "X_test = X_test.reshape(X_test.shape[0], 28, 28, 1).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's normalize the pixel values to be between 0 and 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = X_train / 255 # normalize training data\n",
    "X_test = X_test / 255 # normalize test data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's convert the target variable into binary categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = to_categorical(y_train)\n",
    "y_test = to_categorical(y_test)\n",
    "\n",
    "num_classes = y_test.shape[1] # number of categories"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's define a function that creates our model. Let's start with one set of convolutional and pooling layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convolutional_model():\n",
    "    \n",
    "    # create model\n",
    "    model = Sequential()\n",
    "    model.add(Conv2D(16, (5, 5), strides=(1, 1), activation='relu', input_shape=(28, 28, 1)))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n",
    "    \n",
    "    model.add(Flatten())\n",
    "    model.add(Dense(100, activation='relu'))\n",
    "    model.add(Dense(num_classes, activation='softmax'))\n",
    "    \n",
    "    # compile model\n",
    "    model.compile(optimizer='adam', loss='categorical_crossentropy',  metrics=['accuracy'])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's call the function to create the model, and then let's train it and evaluate it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      " - 42s - loss: 0.2870 - acc: 0.9203 - val_loss: 0.1097 - val_acc: 0.9673\n",
      "Epoch 2/10\n",
      " - 43s - loss: 0.0857 - acc: 0.9752 - val_loss: 0.0561 - val_acc: 0.9825\n",
      "Epoch 3/10\n",
      " - 45s - loss: 0.0576 - acc: 0.9829 - val_loss: 0.0556 - val_acc: 0.9826\n",
      "Epoch 4/10\n",
      " - 46s - loss: 0.0425 - acc: 0.9873 - val_loss: 0.0414 - val_acc: 0.9857\n",
      "Epoch 5/10\n",
      " - 44s - loss: 0.0343 - acc: 0.9899 - val_loss: 0.0426 - val_acc: 0.9849\n",
      "Epoch 6/10\n",
      " - 46s - loss: 0.0285 - acc: 0.9918 - val_loss: 0.0384 - val_acc: 0.9878\n",
      "Epoch 7/10\n",
      " - 46s - loss: 0.0241 - acc: 0.9928 - val_loss: 0.0402 - val_acc: 0.9870\n",
      "Epoch 8/10\n",
      " - 45s - loss: 0.0202 - acc: 0.9938 - val_loss: 0.0374 - val_acc: 0.9873\n",
      "Epoch 9/10\n",
      " - 42s - loss: 0.0166 - acc: 0.9950 - val_loss: 0.0411 - val_acc: 0.9874\n",
      "Epoch 10/10\n",
      " - 44s - loss: 0.0138 - acc: 0.9959 - val_loss: 0.0374 - val_acc: 0.9882\n",
      "Accuracy: 0.9882 \n",
      " Error: 1.1800000000000068\n"
     ]
    }
   ],
   "source": [
    "# build the model\n",
    "model = convolutional_model()\n",
    "\n",
    "# fit the model\n",
    "model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200, verbose=2)\n",
    "\n",
    "# evaluate the model\n",
    "scores = model.evaluate(X_test, y_test, verbose=0)\n",
    "print(\"Accuracy: {} \\n Error: {}\".format(scores[1], 100-scores[1]*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convolutional Layer with two sets of convolutional and pooling layers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's redefine our convolutional model so that it has two convolutional and pooling layers instead of just one layer of each."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convolutional_model():\n",
    "    \n",
    "    # create model\n",
    "    model = Sequential()\n",
    "    model.add(Conv2D(16, (5, 5), activation='relu', input_shape=(28, 28, 1)))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n",
    "    \n",
    "    model.add(Conv2D(8, (2, 2), activation='relu'))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n",
    "    \n",
    "    model.add(Flatten())\n",
    "    model.add(Dense(100, activation='relu'))\n",
    "    model.add(Dense(num_classes, activation='softmax'))\n",
    "    \n",
    "    # Compile model\n",
    "    model.compile(optimizer='adam', loss='categorical_crossentropy',  metrics=['accuracy'])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's call the function to create our new convolutional neural network, and then let's train it and evaluate it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      " - 45s - loss: 0.4888 - acc: 0.8611 - val_loss: 0.1632 - val_acc: 0.9527\n",
      "Epoch 2/10\n",
      " - 45s - loss: 0.1349 - acc: 0.9594 - val_loss: 0.0897 - val_acc: 0.9723\n",
      "Epoch 3/10\n",
      " - 44s - loss: 0.0951 - acc: 0.9707 - val_loss: 0.0710 - val_acc: 0.9765\n",
      "Epoch 4/10\n",
      " - 44s - loss: 0.0758 - acc: 0.9760 - val_loss: 0.0647 - val_acc: 0.9789\n",
      "Epoch 5/10\n",
      " - 47s - loss: 0.0647 - acc: 0.9802 - val_loss: 0.0564 - val_acc: 0.9813\n",
      "Epoch 6/10\n",
      " - 47s - loss: 0.0569 - acc: 0.9822 - val_loss: 0.0521 - val_acc: 0.9821\n",
      "Epoch 7/10\n",
      " - 44s - loss: 0.0506 - acc: 0.9841 - val_loss: 0.0461 - val_acc: 0.9840\n",
      "Epoch 8/10\n",
      " - 44s - loss: 0.0458 - acc: 0.9856 - val_loss: 0.0477 - val_acc: 0.9843\n",
      "Epoch 9/10\n",
      " - 45s - loss: 0.0409 - acc: 0.9875 - val_loss: 0.0461 - val_acc: 0.9863\n",
      "Epoch 10/10\n",
      " - 46s - loss: 0.0383 - acc: 0.9881 - val_loss: 0.0391 - val_acc: 0.9876\n",
      "Accuracy: 0.9876 \n",
      " Error: 1.2399999999999949\n"
     ]
    }
   ],
   "source": [
    "# build the model\n",
    "model = convolutional_model()\n",
    "\n",
    "# fit the model\n",
    "model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200, verbose=2)\n",
    "\n",
    "# evaluate the model\n",
    "scores = model.evaluate(X_test, y_test, verbose=0)\n",
    "print(\"Accuracy: {} \\n Error: {}\".format(scores[1], 100-scores[1]*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Thank you for completing this lab!\n",
    "\n",
    "This notebook was created by [Alex Aklson](https://www.linkedin.com/in/aklson/). I hope you found this lab interesting and educational. Feel free to contact me if you have any questions!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is part of a course on **edX** called *Deep Learning Fundamentals with Keras*. If you accessed this notebook outside the course, you can take this course online by clicking [here](http://cocl.us/DL0101EN_edX_Week4_LAB1)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "\n",
    "Copyright &copy; 2018 [IBM Developer Skills Network](https://cognitiveclass.ai/?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}