Lab_8_Building_Least_Square_Binary_Classifier.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Lab 8: Building Least Square Binary Classifier.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyM892Pu2K5u0ZUeC2qwL7rf",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mirsazzathossain/CSE317-Lab/blob/main/Lab_8_Building_Least_Square_Binary_Classifier.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4lcxLlsWZB9u"
      },
      "source": [
        "#### **1. Import necessary packages:**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VLQvwLcFZBqn"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XuidPMi2ZFcr"
      },
      "source": [
        "#### **2. Upload and load dataset:**\n",
        "At first we have to upload the dataset to google colab to start working with it. Please download the **\"Admission_classifier.csv\"** dataset from piazza resourse or [click here](https://piazza.com/class_profile/get_resource/ku1fdd7zhev3r2/kwv4ddrap491yz) to download it. Then click on files form sidebar, drag and drop your file to side bar to upload the dataset.\n",
        "\n",
        "Now, use `data = pd.read_csv(\"Admission_classifier.csv\")` to load the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h4uFtFXFYr6B"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "blie1N5zbScV"
      },
      "source": [
        "#### **3. Preprocess the Data:**\n",
        "* To visualize the loaded data use `print(data.head())`. \n",
        "* Now, after visualizing the data did you observe we have an extra column named `Serial No.`? \n",
        "* This certainly is not a feature, so we will drop this column. Use `data.drop('Serial No.', axis=1, inplace=True)` to drop the column.\n",
        "* See the column `'Label'` is also not a feature rather it is our target. $y_i = 1$ means admitted. $y_i = -1$ means not admitted for $i^{th}$ student.\n",
        "  * We will store it in a seperate variable `Y` using `Y = data['Label']`.\n",
        "  * Convert `Y` to numpy array using `Y = Y.values`\n",
        "  * Dorp the column from `data` using `data.drop('Label', axis=1, inplace=True)`\n",
        "* In `data` we are left with two features gre scores &  toefl scores. Covert it to numpy array and store in a new variable `X` using `X = data.values`. So, `X` is the matrix of feature columns, each column in `X` will be the feature vectors. Assume the admission acceptance is decided based on these two features. So for every $i^{th}$ student has features $x_1$, and $x_2$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cIRwbvSFbTyO"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "My89w3uAdd7X"
      },
      "source": [
        "#### **4. Add a ones column vector to X & partition the data and labels:**\n",
        "Add a new column cosisting ones as $0^{th}$ column to X. Saw the [numpy documentation](https://numpy.org/doc/stable/reference/generated/numpy.c_.html) for more details. Devide data X and Y into X_train, X_test, Y_train and Y_test. Train dataset will contains 300 datapoints and test dataset will contains 100 datapoint."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1ty_IXl8ebt-"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U28akWEDeoQp"
      },
      "source": [
        "#### **5. Solve the system of equation:**\n",
        "This time we design the classifier as follows. The classifier will give the appropriate admission label for an student. If the value $c_0 +c_1 x_1 +c_2x_2$ is positive, sign function outputs $1$. Otherwise it outputs $-1$.\n",
        "$$y_p = np.sign(c_0 +c_1x_1+c_2x_2)$$\n",
        "Now, calculate the coefficients $c = (c_0, c_1, c_2)$ by finding the least square solution for $(Xc = Y)$. Follow the exact procedure followed by earlier lab. So you will find $ĉ$ using the pseudo inverse of X. Use `X_train` and `X_train` as dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7rL-VKk0huQ5"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WlqRl9FPiomB"
      },
      "source": [
        "#### **6. Find predicted labels:**\n",
        "Find the predicted labels `Y_test_pred`, by using the formula `Y_test_pred = np.sign(X_test * c_hat)`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y7Cw0JdejaB1"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DQ4HbwgtkVg8"
      },
      "source": [
        "#### **7. Calculate the accuracy:**\n",
        "Apparently `Y_test` are the ground truths for the test data. Now you have built a classifier that is defined by `c_hat`. Calculate the accuracy of your predicted label: `Y_test_pred` and groundtruth labels: `Y_test`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8eST1-00kwxj"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "904KzezAljx6"
      },
      "source": [
        "#### **8. Seperate the data into positive and negative:**\n",
        "Finally, separate `X_train` into `X_pos`, `X_neg` using the concept of boolean array indexing and `Y_train`. So that all `X_pos` will have labes `1`. And `X_neg` will have `-1` labels."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QGyNtxVImQnD"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hc2jqZi_n_Y2"
      },
      "source": [
        "#### **9. Plot 2D figure with X_pos & X_neg:**\n",
        "Plot 2D figure with `X_pos` and `X_neg` so that for each data point has feature $x_1$ will be on `x-axis` and $x_2$ will be on `y-axis`. Use marker ‘o’ with red color for `X_neg` and marker ‘*’ with blue color for `X_pos`.\n",
        "\n",
        "**Note: Previously we added one vector in $0^{th}$ column of `X`. Hence, feature $x_1=X[1]$ and feature $x_2=X[2]$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XZn-i-mapgoV"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XFSs_f7Yqyuc"
      },
      "source": [
        "#### **10. Plot the discriminating line:**\n",
        "Plot the discriminating line produced by the equation $c_0 + c_1 x_1 + c_2x_2 = 0$ by finding:\n",
        "$$x_2 = -( c_0 + c_1x_1) / c_2$$\n",
        "for $x_1 = 290:350$. Now, plot previous plots along with a new line plot using $x_1, x_2$ to see the boundary"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FL-4ct5Mrn9z"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 10,
      "outputs": []
    }
  ]
}