03_what_went_wrong.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a0f085d9",
   "metadata": {},
   "source": [
    "<img src='images/Practicum_AI_Logo.white_outline.svg' width=250 alt='Practicum AI logo'> <img src='https://github.com/PracticumAI/practicumai.github.io/blob/main/images/icons/practicumai_python.png?raw=true' align='right' width=50>\n",
    "***\n",
    "\n",
    "# What Went Wrong??\n",
    "\n",
    "Whether it's a typo or trying to solve a complex problem, you will undoubtedly encounter errors when coding! <img src='https://github.com/PracticumAI/python-for-ai/blob/main/images/frustrating_error.png?raw=true' alt='Icon of a person with a hand on head and an exclamation point' width=150 align='right'>\n",
    "\n",
    "Mistakes, errors, problem-solving, and, unfortunately, frustration are sometimes part of coding. Those can also lead to great joy when these problems are solved!\n",
    "\n",
    "But learning to solve problems is also something that can take practice. We will all run into errors, but having resources and experience to solve those errors rather than giving up is critical.\n",
    "\n",
    "This notebook is designed to help lead you through how we solve some common errors. We will look at:\n",
    "\n",
    "* Strategies for interpreting error messages\n",
    "* Online resources for finding help with errors\n",
    "* New possibilities with AI-powered tools to solve errors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52fea84c",
   "metadata": {},
   "source": [
    "## Common typos\n",
    "\n",
    "Let's start trying to solve some of these errors. Each cell below has code that will generate an error. Can you fix them?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0f4d9eb",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "my_value = 5\n",
    "print(f'I am storing the value {myvalue}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21314a7d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def print_message()\n",
    "    print(\"If this prints you fixed the error!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d1ce848d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas\n",
    "\n",
    "url='https://raw.githubusercontent.com/PracticumAI/python-for-ai/main/data/gapminder_all.csv' # Direct link to the dataset on GitHub\n",
    "df = pd.read_csv(url) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57e2c808",
   "metadata": {},
   "source": [
    "Ok, that last one isn't really a typo but a difference in how things are imported. Nothing is wrong with the `import pandas` line, but you will get an error if you try to use the common alias `pd`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8344885d",
   "metadata": {},
   "source": [
    "## The wall of red\n",
    "\n",
    "Sometimes, the error message is very long, and you are faced with a wall of red that can be intimidating. In these cases, the best solution is to start at the bottom of the error and work up a few lines to see if you can determine the source of the error.\n",
    "\n",
    "Have a try at this one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4ddfdac",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "url='https://raw.githubusercontent.com/PracticumAI/python-for-ai/main/data/gapminder_all.csv' # Direct link to the dataset on GitHub\n",
    "df = pd.read_csv(url) # Create the dataframe from the given URL\n",
    "hist = df['gdpPercap_2024'].hist()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1762377",
   "metadata": {},
   "source": [
    "   `KeyError: 'gdpPercap_2024'`\n",
    "\n",
    "It seems like a lot of text to tell you that the column `gdpPercap_2024` doesn't exist. `KeyError`s are pretty common mistakes and mean that you are trying to access a column that doesn't exist. There isn't data for 2024 in this case, but sometimes it's a typo in the name or an extra space somewhere.\n",
    "\n",
    "But, again, scrolling to the bottom gives the best summary of what went wrong.\n",
    "\n",
    "Scrolling back up a ways, you will see the line where the error occurred:\n",
    "\n",
    "   `----> 5 hist = df['gdpPercap_2024'].hist()`\n",
    "   \n",
    "Though that can sometimes be off by a few lines, and the line called out is just where things stopped.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "921f9588",
   "metadata": {},
   "source": [
    "## Not all red is bad!\n",
    "\n",
    "Especially in AI, there are a few common cases where red output text doesn't mean things failed.\n",
    "\n",
    "1. With the rapid development of AI frameworks, it is common that functions are renamed, moved, replaced, or modified in other ways. In most cases, the developers will warn you that this is happening in the future and start issuing what are called **deprecation** warnings--something that is deprecated is being phased out and will, at some point, stop working. Take note of these warnings, especially if you are developing code you hope to use for a while. Changing to the new name/function/way of calling the feature now may save you trouble in the future. But, despite the warning, the function will still work for now.\n",
    "\n",
    "   * In the example below, the deprecated function wasn't even called in our code but by a function we called. In this case, you can try updating the package or ignore it for now.\n",
    "\n",
    "   ![Screenshot of a deprecation warning](images/deprecation_warning.png)\n",
    "  \n",
    "   * The deprecation warning reads, \"The rowNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().rowspan.start instead.\" This means the `rowNum` parameter is being phased out and will not be supported after Matplotlib 3.4 (two minor versions later). A replacement is also suggested.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49293cf9",
   "metadata": {},
   "source": [
    "2. Another common source of red output that can be ignored is when you are using GPUs. (\"GPU\" stands for \"Graphics Processing Unit\". GPUs are specialized computing hardware that can make AI and machine learning work *much* faster. GPUs are available on SCINet's Atlas cluster.) Often, when you first access the GPU, there is some logging and information about the GPU displayed. It would be nice if this were in a different color, but it's shown in red even though nothing is wrong!\n",
    "\n",
    "   This text is common when the GPU is first accessed:\n",
    "   ![Warning seen when a GPU is first accessed](https://github.com/PracticumAI/python-for-ai/blob/main/images/gpu_warning.png?raw=true)\n",
    "   \n",
    "   and this is common when you start a training run.\n",
    "   \n",
    "   ![Warning seen when a GPU is first accessed](https://github.com/PracticumAI/python-for-ai/blob/main/images/gpu_warning2.png?raw=true)\n",
    "   \n",
    "   **Both red blocks are logging messages and can be safely ignored. Despite the color, nothing has gone wrong!**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bf37c16",
   "metadata": {},
   "source": [
    "## CUDA Memory errors\n",
    "\n",
    "\n",
    "Another class of error that you may encounter in working through notebooks, either in this course or in your development, is a CUDA out-of-memory error. CUDA is the programming language that interfaces with the GPU. When things work well, the error is relatively straightforward, e.g.:\n",
    "\n",
    "![CUDA out of memory error](https://github.com/PracticumAI/python-for-ai/blob/main/images/CUDA_OOM.png?raw=true)\n",
    "\n",
    "When things don't work well, the error can be more cryptic, and it can be hard to understand what went wrong. \n",
    "\n",
    "> The Python libraries we use work behind the scenes to communicate with CUDA, so you don't need to worry about it much. But when you see CUDA, know that it means something to do with a GPU.\n",
    "\n",
    "The source of the issue is that because there is time spent in moving data from the CPU memory (the computer's main RAM chips) to the GPU memory (the much more limited RAM built into the GPU itself), Python tries to minimize data movement. While notebooks are pretty good at managing the memory within a single notebook, a notebook will attempt to keep its memory on the GPU and not let other notebooks use the GPU RAM. \n",
    "\n",
    "So, if you have multiple notebooks open in a session, it is common to have odd errors caused by out-of-memory (OOM) situations. In many cases, Python recognizes and reports the error, but sometimes, hard-to-diagnose errors are displayed. In general, especially in our course notebooks, when you get an error, especially one using the GPU, suspect that there may be an out-of-memory error.\n",
    "\n",
    "To solve this, use the \"Running Terminals and Kernels\" panel and stop the kernels for notebooks you are not using. In this screenshot, there are several running notebooks:\n",
    "\n",
    "  ![screenshot of the Running Terminals and Kernels panel showing running notebook kernels](https://github.com/PracticumAI/python-for-ai/blob/main/images/Kernel_control.png?raw=true)\n",
    "  \n",
    "  Either use the \"**Shut Down All**\" button or close individual kernels. Unfortunately, you might need to restart the kernel for the notebook you are using and re-run all the cells from the top."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e9bdf2c",
   "metadata": {},
   "source": [
    "## What do you do when an error doesn't make sense?\n",
    "\n",
    "If you run into an issue where the cause of an error isn't clear, where do you go for help?\n",
    "\n",
    "Google (or another search engine) is usually a good starting point. The top results frequently have good information about the cause of the error. Here are sites with which you will likely become familiar:\n",
    "\n",
    "* [Stack Overflow](https://stackoverflow.com/): This is a great resource, and Google search results frequently feature relevant Stack Overflow posts where a user asks a question that resulted in an error similar to yours, and there are solutions provided. Other users can up-vote solutions to indicate that these are correct and helpful.\n",
    "* [Towards Data Science](https://towardsdatascience.com/) and [Medium](https://medium.com/): Towards Data Science is a blog that has multiple authors covering data science and AI tools. While not always a top hit in finding solutions, this site is frequently a resource when looking for methods for AI applications. Towards Data Science is part of a larger collection of sites under the Medium umbrella, with other sites, some of which also have great AI-focused content. Unfortunately, some of the content on these sites requires registration or subscription. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}