assignments/assignment_yourname_t81_558_class1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sz3w6YSoKx6i"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/app_deep_learning/blob/main/assignments/assignment_yourname_class1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rMtCUtXxZ3ts"
   },
   "source": [
    "# T81-558: Applications of Deep Neural Networks\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/index.html)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/).\n",
    "\n",
    "**Module 1 Assignment: How to Submit an Assignment**\n",
    "\n",
    "**Student Name: Your Name**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s_t8eej3Z3tu"
   },
   "source": [
    "# Assignment Instructions\n",
    "\n",
    "Assignments are submitted using the **submit** function that is defined earlier in this document.  When you submit an assignment, you are sending both your source code and data.  A computer program will automatically check your data, and it will inform you\n",
    "of how closely your data matches up with my solution file.  You are allowed to submit as many times as you like, so if you see some issues with your first submission, you are allowed to make changes and resubmit.  You may resubmit as many times as you like, I will only count your final submission towards your grade.\n",
    "\n",
    "When you first signed up for the course, I emailed you a student key.  You can see a sample key below.  If you use this key, you will get an error.  It is not the current key.  **Use YOUR student key, that I provided in the email.**\n",
    "\n",
    "You must also provide a filename and assignment number.  The filename is the source file that you wish to submit.  Your data is a Pandas dataframe.\n",
    "\n",
    "**Assignment 1 is  easy!** To complete assignment one, all you have to do is add your student key and make sure that the **file** variable contains the path to your source file.  Your source file will most likely end in **.pynb** if you are using a Jupyter notebook; however, it might also end in **.py** if you are using a Python script.\n",
    "\n",
    "Run the code below, and if you are successful, you should see something similar to:\n",
    "\n",
    "```\n",
    "Success: Submitted assignment 1 for jheaton:\n",
    "You have submitted this assignment 2 times. (this is fine)\n",
    "No warnings on your data. You will probably do well, but no guarantee. :-)\n",
    "```\n",
    "\n",
    "If there is an issue with your data, you will get a warning.\n",
    "\n",
    "\n",
    "**Common Problem #1: Bad student key**\n",
    "\n",
    "If you use an invalid student key, you will see:\n",
    "\n",
    "```\n",
    "Failure: {\"message\":\"Forbidden\"}\n",
    "```\n",
    "\n",
    "You should also make sure that **_class#** appears somewhere in your filename. For example, for assignment 1, you should have **_class1** somewhere in your filename. If not, you will get an error.  This check ensures that you do not submit the wrong assignment, with the incorrect file.  If you do have a mismatch, you will get an error such as:\n",
    "\n",
    "\n",
    "**Common Problem #2: Must have class1 (or another number) as part of the filename**\n",
    "```\n",
    "Exception: _class1 must be part of the filename.\n",
    "```\n",
    "\n",
    "The following video covers assignment submission: [assignment submission video](http://www.yahoo.com).\n",
    "\n",
    "**Common Problem #3: Can't find source file**\n",
    "\n",
    "You might get an error similar to this:\n",
    "\n",
    "```\n",
    "FileNotFoundError: [Errno 2] No such file or directory: '/Users/jeffh/projects/t81_558_deep_learning/t81_558_class1_intro_python.ipynb'\n",
    "```\n",
    "\n",
    "This error means your **file** path is wrong.  Make sure the path matches where your file is stored.  See my hints below in the comments for paths in different environments.\n",
    "\n",
    "**Common Problem #4: Too Much Data**\n",
    "\n",
    "You might get an error similar to this:\n",
    "\n",
    "```\n",
    "Failure: HTTP content length exceeded 10485760 bytes.\n",
    "```\n",
    "\n",
    "I see that error sometimes.  Basically AWS Lambda only allows the HTTP request to go up to a certain size. All of my assignments stay well below the limit you are hitting.  This means that the amount of data you are generating is far enough above what I am experiencing that you get this error.  I would have a look at the number of rows, you are generating.\n",
    "\n",
    "**Common Problem #5: Failure: \"This assignment expects CSV data, you did not send a data frame.\"**\n",
    "\n",
    "If you see the error:\n",
    "\n",
    "```\n",
    "Failure: \"This assignment expects CSV data, you did not send a data frame.\"\n",
    "```\n",
    "\n",
    "This is usally because you do not have the square brackes around the dataframe you are submitting. Rather than:\n",
    "\n",
    "```\n",
    "submit(source_file=file,data=df,key=key,no=1)\n",
    "```\n",
    "\n",
    "Make sure you have:\n",
    "\n",
    "```\n",
    "submit(source_file=file,data=[df],key=key,no=1)\n",
    "```\n",
    "You might also not be submitting a dataframe, try to print the type of the dataframe you are submitting, for example:\n",
    "\n",
    "```\n",
    "print(type(df))\n",
    "submit(source_file=file,data=[df],key=key,no=1)\n",
    "```\n",
    "**Common Problem #6: Google Drive is Disabled**\n",
    "\n",
    "When you attempt to map your Google Drive you are told that it is disabled. This is a WUSTL setting for @wustl.edu email accounts. For some reason you cannot use Google Drive on CoLab in conjunction with WUSTL email accounts. Use a non-WUSTL email acco\n",
    "\n",
    "**Common Problem #7: Secret T81_558_KEY does not exist**\n",
    "\n",
    "You must define your submission key in Google CoLab, under the \"Key Icon\", see Module 1 video.\n",
    "\n",
    "```\n",
    "Mounted at /content/drive\n",
    "Note: using Google CoLab\n",
    "---------------------------------------------------------------------------\n",
    "SecretNotFoundError                       Traceback (most recent call last)\n",
    "<ipython-input-1-1fa45d638be1> in <cell line: 12>()\n",
    "     12 if COLAB:\n",
    "     13   # For Colab, add to your \"Secrets\" (key icon at the left)\n",
    "---> 14   key = userdata.get('T81_558_KEY')\n",
    "     15 else:\n",
    "     16   # If not colab, enter your key here, or use an environment variable.\n",
    "\n",
    "/usr/local/lib/python3.10/dist-packages/google/colab/userdata.py in get(key)\n",
    "     66     raise TimeoutException(key)\n",
    "     67   if not resp.get('exists', False):\n",
    "---> 68     raise SecretNotFoundError(key)\n",
    "     69   if not resp.get('access', False):\n",
    "     70     raise NotebookAccessError(key)\n",
    "\n",
    "SecretNotFoundError: Secret T81_558_KEY does not exist.\n",
    "```\n",
    "\n",
    "**Common Problem #8: ???**\n",
    "\n",
    "If you run into a problem not listed here, just let me know.  If you cannot solve the file not found error, try running the code provided later in this file to list every file in your notebooks folder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vT-h8-JIbVg0"
   },
   "source": [
    "# Google CoLab Instructions\n",
    "\n",
    "If you are using Google CoLab, it will be necessary to mount your GDrive so that you can send your notebook during the submit process.  Running the following code will map your GDrive to ```/content/drive```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VDWQvbmRaLLD",
    "outputId": "b2f9713a-2b1c-4c9e-b484-a439e0bafd04"
   },
   "outputs": [],
   "source": [
    "try:\n",
    "  from google.colab import drive, userdata\n",
    "  drive.mount('/content/drive', force_remount=True)\n",
    "  COLAB = True\n",
    "  print(\"Note: using Google CoLab\")\n",
    "except:\n",
    "  print(\"Note: not using Google CoLab\")\n",
    "  COLAB = False\n",
    "\n",
    "# Assignment Submission Key - Was sent you first week of class.\n",
    "# If you are in both classes, this is the same key.\n",
    "if COLAB:\n",
    "  # For Colab, add to your \"Secrets\" (key icon at the left)\n",
    "  key = userdata.get('T81_558_KEY')\n",
    "else:\n",
    "  # If not colab, enter your key here, or use an environment variable.\n",
    "  # (this is only an example key, use yours)\n",
    "  key = \"Gx5en9cEVvaZnjhdaushddhuhhO4PsI32sgldAXj\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QpThHb0sKx6j"
   },
   "source": [
    "If you are running this notebook with CoLab, the following command will show you your notebooks. You will need to know the location of your notebook when you submit your assignment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fgb13NXeaif2",
    "outputId": "f52ba60f-4588-49a4-c0a5-a863b2bf1345"
   },
   "outputs": [],
   "source": [
    "!ls /content/drive/My\\ Drive/Colab\\ Notebooks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6vA8pJdpZ3tu"
   },
   "source": [
    "# Assignment Submit Function\n",
    "\n",
    "You will submit the ten programming assignments electronically.  The following **submit** function can be used to do this.  My server will perform a basic check of each assignment and let you know if it sees any underlying problems.\n",
    "\n",
    "**It is unlikely that should need to modify this function.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "IhpWWqjYZ3tv"
   },
   "outputs": [],
   "source": [
    "import base64\n",
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import requests\n",
    "import PIL\n",
    "import PIL.Image\n",
    "import io\n",
    "from typing import List, Union\n",
    "\n",
    "# This function submits an assignment.  You can submit an assignment as much as you like, only the final\n",
    "# submission counts.  The paramaters are as follows:\n",
    "# data - List of pandas dataframes or images.\n",
    "# key - Your student key that was emailed to you.\n",
    "# course - The course that you are in, currently t81-558 or t81-559.\n",
    "# no - The assignment class number, should be 1 through 1.\n",
    "# source_file - The full path to your Python or IPYNB file.  This must have \"_class1\" as part of its name.\n",
    "# .             The number must match your assignment number.  For example \"_class2\" for class assignment #2.\n",
    "\n",
    "def submit(\n",
    "    data: List[Union[pd.DataFrame, PIL.Image.Image]],\n",
    "    key: str,\n",
    "    course: str,\n",
    "    no: int,\n",
    "    source_file: str = None\n",
    ") -> None:\n",
    "    if source_file is None and '__file__' not in globals():\n",
    "        raise Exception(\"Must specify a filename when in a Jupyter notebook.\")\n",
    "    if source_file is None:\n",
    "        source_file = __file__\n",
    "\n",
    "    suffix = f'_class{no}'\n",
    "    if suffix not in source_file:\n",
    "        raise Exception(f\"{suffix} must be part of the filename.\")\n",
    "\n",
    "    ext = os.path.splitext(source_file)[-1].lower()\n",
    "    if ext not in ['.ipynb', '.py']:\n",
    "        raise Exception(f\"Source file is {ext}; must be .py or .ipynb\")\n",
    "\n",
    "    with open(source_file, \"rb\") as file:\n",
    "        encoded_python = base64.b64encode(file.read()).decode('ascii')\n",
    "\n",
    "    payload = []\n",
    "    for item in data:\n",
    "        if isinstance(item, PIL.Image.Image):\n",
    "            buffered = io.BytesIO()\n",
    "            item.save(buffered, format=\"PNG\")\n",
    "            payload.append({'PNG': base64.b64encode(buffered.getvalue()).decode('ascii')})\n",
    "        elif isinstance(item, pd.DataFrame):\n",
    "            payload.append({'CSV': base64.b64encode(item.to_csv(index=False).encode('ascii')).decode(\"ascii\")})\n",
    "        else:\n",
    "            raise ValueError(f\"Unsupported data type: {type(item)}\")\n",
    "\n",
    "    response = requests.post(\n",
    "        \"https://api.heatonresearch.com/wu/submit\",\n",
    "        headers={'x-api-key': key},\n",
    "        json={\n",
    "            'payload': payload,\n",
    "            'assignment': no,\n",
    "            'course': course,\n",
    "            'ext': ext,\n",
    "            'py': encoded_python\n",
    "        }\n",
    "    )\n",
    "\n",
    "    if response.status_code == 200:\n",
    "        print(f\"Success: {response.text}\")\n",
    "    else:\n",
    "        print(f\"Failure: {response.text}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "id": "cwHjhzzgZ3ty",
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "source": [
    "# Assignment #1 Sample Code\n",
    "\n",
    "For assignment #1, you only must change two things.  The key must be modified to be your key, and the file path must be modified to be the path to your local file.  Once you have that, just run it, and your assignment is submitted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "k7sKpvkYZ3tz",
    "outputId": "c2915cc8-fa8e-447c-a04f-8b64717f4ce7"
   },
   "outputs": [],
   "source": [
    "# You must identify your source file.  (modify for your local setup)\n",
    "file=\"/content/drive/My Drive/Colab Notebooks/assignment_yourname_t81_558_class1.ipynb\"  # Google CoLab\n",
    "# file='C:\\\\Users\\\\jeffh\\\\projects\\\\t81_558_deep_learning\\\\assignments\\\\assignment_yourname_class1.ipynb'  # Windows\n",
    "# file='/Users/jheaton/projects/t81_558_deep_learning/assignments/assignment_yourname_class1.ipynb'  # Mac/Linux\n",
    "\n",
    "df = pd.DataFrame({'a' : [1, 0, 1, 1], 'b' : [0, 1, 0, 1], 'c' : [0, 1, 1, 0]})\n",
    "\n",
    "submit(source_file=file,data=[df],key=key,course='t81-558',no=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-uIU8SxVZ3t2"
   },
   "source": [
    "# Checking Your Submission\n",
    "\n",
    "You can always double-check to make sure my program correctly received your submission.  The following utility code will help with that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "YtQXDseiZ3t3"
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import pandas as pd\n",
    "import base64\n",
    "import os\n",
    "\n",
    "def list_submits(key, course):\n",
    "  r = requests.post(\"https://api.heatonresearch.com/wu/submit\",\n",
    "    headers={'x-api-key': key},\n",
    "    json={'course':course})\n",
    "  if r.status_code == 200:\n",
    "    print(\"Success: \\n{}\".format(r.text))\n",
    "  else:\n",
    "    print(\"Failure: {}\".format(r.text))\n",
    "\n",
    "def display_submit(key,no,course):\n",
    "  r = requests.post(\"https://api.heatonresearch.com/wu/submit\",\n",
    "    headers={'x-api-key': key},\n",
    "    json={'course':course, 'assignment':no})\n",
    "  if r.status_code == 200:\n",
    "    print(\"Success: \\n{}\".format(r.text))\n",
    "  else:\n",
    "    print(\"Failure: {}\".format(r.text))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YrtN_yZnZ3t5",
    "outputId": "9d80bf31-8108-4b32-cc98-f928bbb87277"
   },
   "outputs": [],
   "source": [
    "# Show a listing of all submitted assignments.\n",
    "\n",
    "list_submits(key, course=\"t81-558\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CPjhGyemZ3t7",
    "outputId": "0b1cd262-3907-436c-8b34-b0f58b308f2d"
   },
   "outputs": [],
   "source": [
    "# Show one assignment, by number.\n",
    "\n",
    "display_submit(key,1, course=\"t81-558\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PMMkC2ChKx6k"
   },
   "source": [
    "# Can't Find Your File in CoLab?\n",
    "\n",
    "The following code will list every file in your Google Colab Notebooks directory along with a true/false indicator of if the file could be opened.  You SHOULD see all true's.  Copy/paste the filename (along with the quotes) and use it as the name of your submitted notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xKSrO3vxKx6k",
    "outputId": "8b76d44d-6fea-486e-cc7c-1fd9d0522292"
   },
   "outputs": [],
   "source": [
    "from os import listdir\n",
    "from os.path import isfile, join\n",
    "import base64\n",
    "\n",
    "if COLAB:\n",
    "  PATH = \"/content/drive/My Drive/Colab Notebooks/\"\n",
    "  for f in listdir(PATH):\n",
    "    full_path = join(PATH, f)\n",
    "    if isfile(full_path):\n",
    "      try:\n",
    "        with open(full_path, \"rb\") as image_file:\n",
    "          encoded_python = base64.b64encode(image_file.read()).decode('ascii')\n",
    "          success = True\n",
    "      except FileNotFoundError:\n",
    "        success = False\n",
    "\n",
    "      report = full_path.replace('\\\\','\\\\\\\\')\n",
    "      report = report.replace('\\\"','\\\\\\\"')\n",
    "      report = report.replace('\\'','\\\\\\'')\n",
    "      print(f'[{success}] - \"{report}\"')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-OvkF33bZ3t9"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.11 (torch)",
   "language": "python",
   "name": "pytorch"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}