CNNs_with_Keras.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "CNNs_with_Keras",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/paulgureghian/Google_Colab_Notebooks/blob/master/CNNs_with_Keras.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "Zsa1vXyWlNvD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<a href=\"https://cognitiveclass.ai\"><img src = \"https://ibm.box.com/shared/static/9gegpsmnsoo25ikkbl4qzlvlyjbgxs5x.png\" width = 400> </a>\n",
        "\n",
        "<h1 align=center><font size = 5>Convolutional Neural Networks with Keras</font></h1>"
      ]
    },
    {
      "metadata": {
        "id": "WUFMiyRglNvD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "In this lab, we will learn how to use the keras library to build convolutional neural networks. We will also use the popular MNIST dataset and we will compare our results to using a conventional neural network.\n"
      ]
    },
    {
      "metadata": {
        "id": "iRknh_felNvD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Table of Contents\n",
        "\n",
        "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
        "\n",
        "<font size = 3>\n",
        "      \n",
        "1. <a href=\"#item2\">Import Keras and Packages</a>   \n",
        "2. <a href=\"#item3\">Convolutional Neural Network with One Convolutional and Pooling Layers</a>  \n",
        "3. <a href=\"#item4\">Convolutional Neural Network with Two Convolutional and Pooling Layers</a>  \n",
        "\n",
        "</font>\n",
        "</div>"
      ]
    },
    {
      "metadata": {
        "id": "Lq2rZRLClNvD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Import Keras and Packages"
      ]
    },
    {
      "metadata": {
        "id": "qA95Af-2lNvD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Let's start by importing the keras libraries and the packages that we would need to build a neural network."
      ]
    },
    {
      "metadata": {
        "id": "eGvLOcsPlNvT",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Import Keras \n",
        "import keras\n",
        "from keras.layers import Dense\n",
        "from keras.models import Sequential\n",
        "from keras.utils import to_categorical"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Bupp0MWQlNvT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "When working with convolutional neural networks in particular, we will need additional packages."
      ]
    },
    {
      "metadata": {
        "id": "bFO463j0lNvT",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Import additional Keras packages\n",
        "from keras.layers import Flatten                     ### To flatten data for fully connected layers\n",
        "from keras.layers.convolutional import Conv2D        ### To add convolutional layers\n",
        "from keras.layers.convolutional import MaxPooling2D  ### To add pooling layers"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "QHHW5NnclNvT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Convolutional Network with one set of convolutional and pooling layers"
      ]
    },
    {
      "metadata": {
        "id": "QYbsix7GlNvT",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 12240
        },
        "outputId": "695017b3-2992-4430-dfb0-4443c4763eec"
      },
      "cell_type": "code",
      "source": [
        "### Import data\n",
        "from keras.datasets import mnist\n",
        "\n",
        "### Load data\n",
        "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
        "\n",
        "### Print 'X_train' and 'X_test' \n",
        "print(\"X_train:\\n\",  X_train, '\\n')\n",
        "print(\"X_test:\\n\", X_test, '\\n')\n",
        "\n",
        "### Reshape to be [samples][pixels][width][height]\n",
        "X_train = X_train.reshape(X_train.shape[0], 28, 28, 1).astype('float32')\n",
        "X_test = X_test.reshape(X_test.shape[0], 28, 28, 1).astype('float32')\n",
        "\n",
        "### Print reshaped 'X_train' and 'X_test'\n",
        "print(\"Reshaped X_train:\\n\", X_train, '\\n')\n",
        "print(\"Resahped X_test:\\n\", X_test) "
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "X_train:\n",
            " [[[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " ...\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]] \n",
            "\n",
            "X_test:\n",
            " [[[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " ...\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]\n",
            "\n",
            " [[0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  ...\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]\n",
            "  [0 0 0 ... 0 0 0]]] \n",
            "\n",
            "Reshaped X_train:\n",
            " [[[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " ...\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]] \n",
            "\n",
            "Resahped X_test:\n",
            " [[[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " ...\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "2kwTr_lHPUR5",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        },
        "outputId": "65e8cbcb-ece6-427c-ca32-65bb915d4c31"
      },
      "cell_type": "code",
      "source": [
        "print(\"Y_train:\\n\" , y_train, '\\n')\n",
        "print(\"Y_test:\\n\" , y_test)  "
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Y_train:\n",
            " [5 0 4 ... 5 6 8] \n",
            "\n",
            "Y_test:\n",
            " [7 2 1 ... 4 5 6]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "KZYSzZqylNvi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Let's normalize the pixel values to be between 0 and 1"
      ]
    },
    {
      "metadata": {
        "id": "E671oE3JlNvi",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 10506
        },
        "outputId": "57d2f015-0036-4264-e2e5-e75385e62d4c"
      },
      "cell_type": "code",
      "source": [
        "### Normalize pixel values\n",
        "X_train = X_train / 255 ### Normalize training data\n",
        "X_test = X_test / 255   ### Normalize test data\n",
        "\n",
        "print(\"X_train:\\n\", X_train, '\\n')\n",
        "print(\"X_test:\\n\", X_test) "
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "X_train:\n",
            " [[[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " ...\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]] \n",
            "\n",
            "X_test:\n",
            " [[[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " ...\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]\n",
            "\n",
            "\n",
            " [[[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  ...\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]\n",
            "\n",
            "  [[0.]\n",
            "   [0.]\n",
            "   [0.]\n",
            "   ...\n",
            "   [0.]\n",
            "   [0.]\n",
            "   [0.]]]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "mLPDkR6JlNvi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Next, let's convert the target variable into binary categories"
      ]
    },
    {
      "metadata": {
        "id": "fAH9v19hlNvi",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        },
        "outputId": "81c1bb2d-e46f-484e-e182-f69fbacebceb"
      },
      "cell_type": "code",
      "source": [
        "### Convert target variable to binary\n",
        "y_train = to_categorical(y_train)\n",
        "y_test = to_categorical(y_test)\n",
        "\n",
        "print(\"Y_train:\\n\", y_train, '\\n')\n",
        "print(\"Y_test:\\n\", y_test, '\\n')\n",
        "\n",
        "num_classes = y_test.shape[1] ### Number of categories\n",
        "\n",
        "print(\"Num Classes:\\n\", num_classes) "
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Y_train:\n",
            " [[0. 0. 0. ... 0. 0. 0.]\n",
            " [1. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " ...\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 1. 0.]] \n",
            "\n",
            "Y_test:\n",
            " [[0. 0. 0. ... 1. 0. 0.]\n",
            " [0. 0. 1. ... 0. 0. 0.]\n",
            " [0. 1. 0. ... 0. 0. 0.]\n",
            " ...\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]] \n",
            "\n",
            "Num Classes:\n",
            " 10\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "cepGXuAelNvi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Next, let's define a function that creates our model. Let's start with one set of convolutional and pooling layers."
      ]
    },
    {
      "metadata": {
        "id": "MtbsbmvxlNvy",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Define 'convolutional model()'\n",
        "def convolutional_model():\n",
        "    \n",
        "    ### Create model\n",
        "    model = Sequential()\n",
        "    model.add(Conv2D(16, (5, 5), strides=(1, 1), activation='relu', input_shape=(28, 28, 1)))\n",
        "    model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n",
        "    \n",
        "    model.add(Flatten())\n",
        "    model.add(Dense(100, activation='relu'))\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "    \n",
        "    ### Compile model\n",
        "    model.compile(optimizer='adam', loss='categorical_crossentropy',  metrics=['accuracy'])\n",
        "    \n",
        "    return model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "R9BHCqSTlNvy",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Finally, let's call the function to create the model, and then let's train it and evaluate it."
      ]
    },
    {
      "metadata": {
        "id": "2ZzEDLKglNvy",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 408
        },
        "outputId": "806f2dc4-3bb0-44a1-b787-07acbc903dd3"
      },
      "cell_type": "code",
      "source": [
        "### Instantiate the model\n",
        "model = convolutional_model()\n",
        "\n",
        "### Fit the model\n",
        "model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200, verbose=2)\n",
        "\n",
        "### Evaluate the model\n",
        "scores = model.evaluate(X_test, y_test, verbose=0)\n",
        "print(\"Accuracy: {} \\n Error: {}\".format(scores[1], 100-scores[1]*100))"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 60000 samples, validate on 10000 samples\n",
            "Epoch 1/10\n",
            " - 3s - loss: 0.2926 - acc: 0.9174 - val_loss: 0.1259 - val_acc: 0.9625\n",
            "Epoch 2/10\n",
            " - 2s - loss: 0.0879 - acc: 0.9742 - val_loss: 0.0691 - val_acc: 0.9771\n",
            "Epoch 3/10\n",
            " - 2s - loss: 0.0599 - acc: 0.9825 - val_loss: 0.0519 - val_acc: 0.9827\n",
            "Epoch 4/10\n",
            " - 2s - loss: 0.0462 - acc: 0.9864 - val_loss: 0.0479 - val_acc: 0.9840\n",
            "Epoch 5/10\n",
            " - 2s - loss: 0.0375 - acc: 0.9884 - val_loss: 0.0444 - val_acc: 0.9854\n",
            "Epoch 6/10\n",
            " - 2s - loss: 0.0312 - acc: 0.9902 - val_loss: 0.0400 - val_acc: 0.9859\n",
            "Epoch 7/10\n",
            " - 2s - loss: 0.0263 - acc: 0.9916 - val_loss: 0.0473 - val_acc: 0.9861\n",
            "Epoch 8/10\n",
            " - 2s - loss: 0.0224 - acc: 0.9929 - val_loss: 0.0373 - val_acc: 0.9873\n",
            "Epoch 9/10\n",
            " - 2s - loss: 0.0196 - acc: 0.9940 - val_loss: 0.0425 - val_acc: 0.9856\n",
            "Epoch 10/10\n",
            " - 2s - loss: 0.0156 - acc: 0.9953 - val_loss: 0.0510 - val_acc: 0.9832\n",
            "Accuracy: 0.9832 \n",
            " Error: 1.6800000000000068\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "rVAai4uclNwB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "------------------------------------------"
      ]
    },
    {
      "metadata": {
        "id": "oP7TEkzHlNwE",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Convolutional Network with two sets of convolutional and pooling layers"
      ]
    },
    {
      "metadata": {
        "id": "-69h_SFYlNwG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Let's redefine our convolutional model so that it has two convolutional and pooling layers instead of just one layer of each."
      ]
    },
    {
      "metadata": {
        "id": "HFN5-EdTlNwH",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "### Define 'convolutional model()'\n",
        "def convolutional_model():\n",
        "    \n",
        "    ### Create model\n",
        "    model = Sequential()\n",
        "    model.add(Conv2D(16, (5, 5), activation='relu', input_shape=(28, 28, 1)))\n",
        "    model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n",
        "    \n",
        "    model.add(Conv2D(8, (2, 2), activation='relu'))\n",
        "    model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n",
        "    \n",
        "    model.add(Flatten())\n",
        "    model.add(Dense(100, activation='relu'))\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "    \n",
        "    ### Compile model\n",
        "    model.compile(optimizer='adam', loss='categorical_crossentropy',  metrics=['accuracy'])\n",
        "    \n",
        "    return model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kWPZLhYTlNwK",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now, let's call the function to create our new convolutional neural network, and then let's train it and evaluate it."
      ]
    },
    {
      "metadata": {
        "id": "4BVoKhSTlNwM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 408
        },
        "outputId": "15e7a26f-3021-4553-e1df-0cbcdfd55b6e"
      },
      "cell_type": "code",
      "source": [
        "### Instantiate the model\n",
        "model = convolutional_model()\n",
        "\n",
        "### Fit the model\n",
        "model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200, verbose=2)\n",
        "\n",
        "### Evaluate the model\n",
        "scores = model.evaluate(X_test, y_test, verbose=0)\n",
        "print(\"Accuracy: {} \\n Error: {}\".format(scores[1], 100-scores[1]*100))"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 60000 samples, validate on 10000 samples\n",
            "Epoch 1/10\n",
            " - 3s - loss: 0.4516 - acc: 0.8714 - val_loss: 0.1341 - val_acc: 0.9600\n",
            "Epoch 2/10\n",
            " - 3s - loss: 0.1003 - acc: 0.9695 - val_loss: 0.0718 - val_acc: 0.9760\n",
            "Epoch 3/10\n",
            " - 3s - loss: 0.0715 - acc: 0.9781 - val_loss: 0.0581 - val_acc: 0.9821\n",
            "Epoch 4/10\n",
            " - 3s - loss: 0.0585 - acc: 0.9823 - val_loss: 0.0479 - val_acc: 0.9837\n",
            "Epoch 5/10\n",
            " - 3s - loss: 0.0501 - acc: 0.9849 - val_loss: 0.0478 - val_acc: 0.9831\n",
            "Epoch 6/10\n",
            " - 3s - loss: 0.0435 - acc: 0.9864 - val_loss: 0.0355 - val_acc: 0.9876\n",
            "Epoch 7/10\n",
            " - 3s - loss: 0.0380 - acc: 0.9881 - val_loss: 0.0362 - val_acc: 0.9862\n",
            "Epoch 8/10\n",
            " - 3s - loss: 0.0354 - acc: 0.9891 - val_loss: 0.0368 - val_acc: 0.9880\n",
            "Epoch 9/10\n",
            " - 3s - loss: 0.0323 - acc: 0.9896 - val_loss: 0.0383 - val_acc: 0.9865\n",
            "Epoch 10/10\n",
            " - 3s - loss: 0.0289 - acc: 0.9909 - val_loss: 0.0351 - val_acc: 0.9882\n",
            "Accuracy: 0.9882 \n",
            " Error: 1.1800000000000068\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "QuvO2VxqlNwP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Thank you for completing this lab!\n",
        "\n",
        "This notebook was created by [Alex Aklson](https://www.linkedin.com/in/aklson/). I hope you found this lab interesting and educational. Feel free to contact me if you have any questions!"
      ]
    },
    {
      "metadata": {
        "id": "PEaawM91lNwQ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This notebook is part of a course on **edX** called *Deep Learning Fundamentals with Keras*. If you accessed this notebook outside the course, you can take this course online by clicking [here](http://cocl.us/DL0101EN_edX_Week4_LAB1)."
      ]
    },
    {
      "metadata": {
        "id": "nhizIjiolNwS",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<hr>\n",
        "\n",
        "Copyright &copy; 2018 [IBM Developer Skills Network](https://cognitiveclass.ai/?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/)."
      ]
    }
  ]
}