Lab_6_OLS_Regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Lab 6:  OLS Regression.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyPPOUUwxCRiiW4CSxVuK7Qw",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mirsazzathossain/CSE317-Lab/blob/main/Lab_6_OLS_Regression.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qu8-4GGd0t4n"
      },
      "source": [
        "#### **Problem Statement:**\n",
        "In this experiment we will do Ordinary least squares (multiple) regression for the prediction of Graduate Admissions from an Indian/Bangladeshi perspective. The dataset can be obtained from\n",
        "[Piazza](https://piazza.com/class_profile/get_resource/ku1fdd7zhev3r2/kwaz7m8lx5a52k) and [Kaggle](https://www.kaggle.com/mohansacharya/graduate-admissions). The dataset (Admission_Predict.csv) containss even features arranged into columns in a CSV file. There are 400 sample datapoints. The features are as follows:\n",
        "1. GRE Scores (out of 340)\n",
        "2. TOEFL Scores (out of 120)\n",
        "3. University Rating (out of 5)\n",
        "4. Statement of Purpose and Letter of Recommendation Strength (out of 5)\n",
        "5. Undergraduate GPA (out of 10)\n",
        "6. Research Experience (either 0 or 1)\n",
        "7. Chance of Admit (ranging from 0 to 1)\n",
        "\n",
        "The first column of the dataset contains a serial number, and the final column provide the probability of getting admission, i.e. the target output for each datapoint.We will be using the dataset to create a linear regression model in order to determine the chances of admission of a new sample student, and to assess how well our model works in making a useful forecast."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UTFGVQAzylcJ"
      },
      "source": [
        "#### **1. Import necessary packages:**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7NN7iMIsx7Ds"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PBoPBBely2V1"
      },
      "source": [
        "#### **2. Upload and load dataset:**\n",
        "At first we have to upload the dataset to google colab to start working with it. Please download the **\"Admission_Predict.csv\"** dataset from piazza resourse or [click here](https://piazza.com/class_profile/get_resource/ku1fdd7zhev3r2/kwaz7m8lx5a52k) to download it. Then click on files form sidebar, drag and drop your file to side bar to upload the dataset.\n",
        "\n",
        "Now, use `data = pd.read_csv(\"Admission_Predict.csv\")` to load the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rFjyg4jY16zN"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VCFQyd4_2QWo"
      },
      "source": [
        "#### **3. Preprocess the Data:**\n",
        "* To visualize the loaded data use `print(data.head())`. \n",
        "* Now, after visualizing the data did you observe we have an extra column named `Serial No.`? \n",
        "* This certainly is not a feature, so we will drop this column. Use `data.drop('Serial No.', axis=1, inplace=True)` to drop the column.\n",
        "* See the column `'Chance of Admit'` is also not a feature rather it is our target. \n",
        "  * We will store it in a seperate variable `y` using `y = data['Chance of Admit ']`.\n",
        "  * Convert `y` to numpy array using `y = y.values`\n",
        "  * Dorp the column from `data` using `data.drop('Chance of Admit ', axis=1, inplace=True)`\n",
        "* In `data` we are left with all 7 features. Covert it to numpy array and store in a new variable `X` using `X = data.values`. So, `X` is the matrix of feature columns, each column in `X` will be the feature vectors.\n",
        "\n",
        "☢ Note: Be careful about the space after the column name `'Chance of Admit '`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_ZR3EWYt2Pfu"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fb7WgQrc-bKw"
      },
      "source": [
        "#### **4. Add a ones column vector to X:**\n",
        "Add a new column cosisting ones as $0^{th}$ column to X. Saw the [numpy documentation](https://numpy.org/doc/stable/reference/generated/numpy.c_.html) for more details. Devide data X and y into x_train, x_test, y_train and y_test. Train dataset will contains 300 datapoints and test dataset will contains 100 datapoint."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bF7VdVQV-nhF"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zRuxuheJBZjZ"
      },
      "source": [
        "#### **5. Solve the system of equation:**\n",
        "Solve the system of equations $(Xβ = y)$ to find the values of the $β$ vector $(β_0, β_1, β_2, \\ldots, β_n)$. You can find $β$ using $β = X^† y = (X^T X)^{−1} X^T y = R^{−1} Q^T y$. There is also a numpy function to calculate the psuedo inverse: `np.linalg.pinv()`, saw the [numpy documentation](https://numpy.org/doc/stable/reference/generated/numpy.linalg.pinv.html) for more details. Use `x_train` and `y_train` as dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Cq0xGcwECkJg"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rZ-2uCWUFnZN"
      },
      "source": [
        "#### **6. Find predicted chance of admit:**\n",
        "Find the predicted chance of admit $\\hat y$, by multiplying $X * β$. For prediction use `x_test` as dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_CUbOOGeGX9S"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-XSa1mXOGw9-"
      },
      "source": [
        "#### **7. Find the error vector e:**\n",
        "Find the error vector, $e$, by subtracting $\\hat y$ from `y_test`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PIGN2JfNGwU_"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uiGRMarpHNHn"
      },
      "source": [
        "#### **8. Compute the $r^2$ value:**\n",
        "Recall that, $r^2 = 1 - SSE / SST$, where $SSE$ is the sum of squared errors: $e^Te$ and $SST = \\text{Total sum of squares : } (\\text{y_test} - avg(\\text{y_test}))^T(\\text{y_test} - avg(\\text{y_test}))$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sRduN3A8H3bz"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qao9EbhKIsYS"
      },
      "source": [
        "#### **9. Plot the vectors $y$, $\\hat y$, and $e$:**\n",
        "Plot the vectors $\\text{y_test}$, $\\hat y$, and $e$, and make suitable observations. Use different color for three vectors while ploting."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mH9QI_5gJDUn"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M9VQwKy1UbG9"
      },
      "source": [
        "#### **10. Test with new data:**\n",
        "Introduce a new sample student with your own data, and find where they fall."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "C0bu2bRrYXPP"
      },
      "source": [
        "# Write appropriate code"
      ],
      "execution_count": 10,
      "outputs": []
    }
  ]
}