This is Jeopardy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# This is Jeopardy!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This project is slightly different than others you have encountered thus far. Instead of a step-by-step tutorial, this project contains a series of open-ended requirements which describe the project you'll be building. There are many possible ways to correctly fulfill all of these requirements, and you should expect to use the internet, Codecademy, and/or other resources when you encounter a problem that you cannot easily solve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Project Goals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will work to write several functions that investigate a dataset of _Jeopardy!_ questions and answers. Filter the dataset for topics that you're interested in, compute the average difficulty of those questions, and train to become the next Jeopardy champion!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to complete this project, you should have completed the Pandas lessons in the <a href=\"https://www.codecademy.com/learn/paths/analyze-data-with-python\">Analyze Data with Python Skill Path</a>. You can also find those lessons in the <a href=\"https://www.codecademy.com/learn/data-processing-pandas\">Data Analysis with Pandas course</a> or the <a href=\"https://www.codecademy.com/learn/paths/data-science/\">Data Scientist Career Path</a>.\n",
    "\n",
    "Finally, the <a href=\"https://www.codecademy.com/learn/practical-data-cleaning\">Practical Data Cleaning</a> course may also be helpful."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Project Requirements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. We've provided a csv file containing data about the game show _Jeopardy!_ in a file named `jeopardy.csv`. Load the data into a DataFrame and investigate its contents. Try to print out specific columns.\n",
    "\n",
    "   Note that in order to make this project as \"real-world\" as possible, we haven't modified the data at all - we're giving it to you exactly how we found it. As a result, this data isn't as \"clean\" as the datasets you normally find on Codecademy. More specifically, there's something odd about the column names. After you figure out the problem with the column names, you may want to rename them to make your life easier for the rest of the project.\n",
    "   \n",
    "   In order to display the full contents of a column, we've added this line of code for you:\n",
    "   \n",
    "   ```py\n",
    "   pd.set_option('display.max_colwidth', None)\n",
    "   ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 216930 entries, 0 to 216929\n",
      "Data columns (total 7 columns):\n",
      " #   Column       Non-Null Count   Dtype \n",
      "---  ------       --------------   ----- \n",
      " 0   Show Number  216930 non-null  int64 \n",
      " 1   Air Date     216930 non-null  object\n",
      " 2   Round        216930 non-null  object\n",
      " 3   Category     216930 non-null  object\n",
      " 4   Value        216930 non-null  object\n",
      " 5   Question     216930 non-null  object\n",
      " 6   Answer       216928 non-null  object\n",
      "dtypes: int64(1), object(6)\n",
      "memory usage: 11.6+ MB\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import re\n",
    "pd.set_option('display.max_colwidth', None)\n",
    "pd.set_option('display.max_rows', None)\n",
    "\n",
    "filepath = \"jeopardy.csv\"\n",
    "jeopardy_df = pd.read_csv(filepath)\n",
    "jeopardy_df.columns = jeopardy_df.columns.str.strip()\n",
    "jeopardy_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Write a function that filters the dataset for questions that contains all of the words in a list of words. For example, when the list `[\"King\", \"England\"]` was passed to our function, the function returned a DataFrame of 49 rows. Every row had the strings `\"King\"` and `\"England\"` somewhere in its `\" Question\"`.\n",
    "\n",
    "   Test your function by printing out the column containing the question of each row of the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Show Number</th>\n",
       "      <th>Air Date</th>\n",
       "      <th>Round</th>\n",
       "      <th>Category</th>\n",
       "      <th>Value</th>\n",
       "      <th>Question</th>\n",
       "      <th>Answer</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4953</td>\n",
       "      <td>3003</td>\n",
       "      <td>1997-09-24</td>\n",
       "      <td>Double Jeopardy!</td>\n",
       "      <td>\"PH\"UN WORDS</td>\n",
       "      <td>$200</td>\n",
       "      <td>Both England's King George V &amp; FDR put their stamp of approval on this \"King of Hobbies\"</td>\n",
       "      <td>Philately (stamp collecting)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14912</td>\n",
       "      <td>2832</td>\n",
       "      <td>1996-12-17</td>\n",
       "      <td>Jeopardy!</td>\n",
       "      <td>WORLD HISTORY</td>\n",
       "      <td>$100</td>\n",
       "      <td>This country's King Louis IV was nicknamed \"Louis From Overseas\" because he was raised in England</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21511</td>\n",
       "      <td>4650</td>\n",
       "      <td>2004-11-19</td>\n",
       "      <td>Jeopardy!</td>\n",
       "      <td>THE \"O.C.\"</td>\n",
       "      <td>$1000</td>\n",
       "      <td>this man and his son ruled England following the execution of King Charles I</td>\n",
       "      <td>Oliver Cromwell</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>23810</td>\n",
       "      <td>4862</td>\n",
       "      <td>2005-11-01</td>\n",
       "      <td>Jeopardy!</td>\n",
       "      <td>NAME THE YEAR</td>\n",
       "      <td>$400</td>\n",
       "      <td>William the Conqueror was crowned King of England in Westminster Abbey on Christmas Day in this year</td>\n",
       "      <td>1066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>27555</td>\n",
       "      <td>1799</td>\n",
       "      <td>1992-05-28</td>\n",
       "      <td>Double Jeopardy!</td>\n",
       "      <td>HISTORIC IN-LAWS</td>\n",
       "      <td>$600</td>\n",
       "      <td>This member of the Medici family was the mother-in-law of England's King Charles I</td>\n",
       "      <td>Marie de Medici</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index  Show Number   Air Date             Round          Category  Value  \\\n",
       "0   4953         3003 1997-09-24  Double Jeopardy!      \"PH\"UN WORDS   $200   \n",
       "1  14912         2832 1996-12-17         Jeopardy!     WORLD HISTORY   $100   \n",
       "2  21511         4650 2004-11-19         Jeopardy!        THE \"O.C.\"  $1000   \n",
       "3  23810         4862 2005-11-01         Jeopardy!     NAME THE YEAR   $400   \n",
       "4  27555         1799 1992-05-28  Double Jeopardy!  HISTORIC IN-LAWS   $600   \n",
       "\n",
       "                                                                                               Question  \\\n",
       "0              Both England's King George V & FDR put their stamp of approval on this \"King of Hobbies\"   \n",
       "1     This country's King Louis IV was nicknamed \"Louis From Overseas\" because he was raised in England   \n",
       "2                          this man and his son ruled England following the execution of King Charles I   \n",
       "3  William the Conqueror was crowned King of England in Westminster Abbey on Christmas Day in this year   \n",
       "4                    This member of the Medici family was the mother-in-law of England's King Charles I   \n",
       "\n",
       "                         Answer  \n",
       "0  Philately (stamp collecting)  \n",
       "1                        France  \n",
       "2               Oliver Cromwell  \n",
       "3                          1066  \n",
       "4               Marie de Medici  "
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a func that contains all words in a list of words.\n",
    "def filter_words_in_list(word_list=[]):\n",
    "    basic_filter = jeopardy_df[jeopardy_df.Question.str.contains(\"|\".join(word_list),re.I,regex=True)].reset_index()\n",
    "    rows = jeopardy_df[(jeopardy_df.Question.str.contains(word_list[0],re.I,regex=True)==True) & (jeopardy_df.Question.str.contains(word_list[1],re.I,regex=True)==True)]\n",
    "    return rows.reset_index()\n",
    "    \n",
    "list_of_words = ['King','England']\n",
    "\n",
    "# column to be printed\n",
    "cols = filter_words_in_list(list_of_words)\n",
    "cols.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Test your original function with a few different sets of words to try to find some ways your function breaks. Edit your function so it is more robust.\n",
    "\n",
    "   For example, think about capitalization. We probably want to find questions that contain the word `\"King\"` or `\"king\"`.\n",
    "   \n",
    "   You may also want to check to make sure you don't find rows that contain substrings of your given words. For example, our function found a question that didn't contain the word `\"king\"`, however it did contain the word `\"viking\"` &mdash; it found the `\"king\"` inside `\"viking\"`. Note that this also comes with some drawbacks &mdash; you would no longer find questions that contained words like `\"England's\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "The function above seems not to include words that dont start with 'king'or 'King' \n",
    "and includes words like 'England's'.\n",
    "'''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. We may want to eventually compute aggregate statistics, like `.mean()` on the `\" Value\"` column. But right now, the values in that column are strings. Convert the`\" Value\"` column to floats. If you'd like to, you can create a new column with float values.\n",
    "\n",
    "   While most of the values in the `\" Value\"` column represent a dollar amount as a string, note that some do not &mdash; these values will need to be handled differently!\n",
    "\n",
    "   Now that you can filter the dataset of question, use your new column that contains the float values of each question to find the \"difficulty\" of certain topics. For example, what is the average value of questions that contain the word `\"King\"`?\n",
    "   \n",
    "   Make sure to use the dataset that contains the float values as the dataset you use in your filtering function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/hl/zjjjtjfn3_x86lljfycj2bsr0000gn/T/ipykernel_54466/726182867.py:3: FutureWarning: The default value of regex will change from True to False in a future version.\n",
      "  jeopardy_df['float_values'] = jeopardy_df.Value.str.replace(r'\\D+',\"\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The average value is: 771.84\n"
     ]
    }
   ],
   "source": [
    "# Changing type of Value column to float type\n",
    "\n",
    "jeopardy_df['float_values'] = jeopardy_df.Value.str.replace(r'\\D+',\"\")\n",
    "jeopardy_df['float_values'] = pd.to_numeric(jeopardy_df['float_values'])\n",
    "jeopardy_df['float_values'] = jeopardy_df['float_values'].fillna(0)\n",
    "king_values = jeopardy_df[jeopardy_df.Question.str.contains(r\"King|king\")]\n",
    "king_values_mean = king_values['float_values'].mean()\n",
    "\n",
    "print(f\"The average value is: {king_values_mean:0.2f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Write a function that returns the count of unique answers to all of the questions in a dataset. For example, after filtering the entire dataset to only questions containing the word `\"King\"`, we could then find all of the unique answers to those questions. The answer \"Henry VIII\" appeared 55 times and was the most common answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Henry VIII     55\n",
       "Solomon        35\n",
       "Richard III    33\n",
       "Louis XIV      31\n",
       "David          30\n",
       "Name: Answer, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def unique_answers(terms=[]):\n",
    "    filtered_king_values = jeopardy_df[jeopardy_df['Question'].str.contains(\"|\".join(terms),re.I,regex=True)]\n",
    "    unique_counts = filtered_king_values['Answer'].value_counts(dropna=False)\n",
    "    return unique_counts\n",
    "\n",
    "terms = ['King','king']\n",
    "unique_answers(terms).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. Explore from here! This is an incredibly rich dataset, and there are so many interesting things to discover. There are a few columns that we haven't even started looking at yet. Here are some ideas on ways to continue working with this data:\n",
    "\n",
    " * Investigate the ways in which questions change over time by filtering by the date. How many questions from the 90s use the word `\"Computer\"` compared to questions from the 2000s?\n",
    " * Is there a connection between the round and the category? Are you more likely to find certain categories, like `\"Literature\"` in Single Jeopardy or Double Jeopardy?\n",
    " * Build a system to quiz yourself. Grab random questions, and use the <a href=\"https://docs.python.org/3/library/functions.html#input\">input</a> function to get a response from the user. Check to see if that response was right or wrong."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The no. of rows from the 90s is: 104 and from the 2000s is: 268\n"
     ]
    }
   ],
   "source": [
    "# Investigate the ways in which questions change over time by filtering by the date. \n",
    "# How many questions from the 90s use the word \"Computer\" compared to questions from the 2000s?\n",
    "\n",
    "word = 'computer'\n",
    "\n",
    "jeopardy_df['Air Date'] = pd.to_datetime(jeopardy_df['Air Date'],format='%Y-%m-%d')\n",
    "air_dates_after = jeopardy_df[(jeopardy_df['Air Date'].dt.strftime('%Y') >= str(2000)) & (jeopardy_df['Air Date'].dt.strftime('%Y') < str(2010))].sort_values(by='Air Date')\n",
    "filtered_air_dates = jeopardy_df[jeopardy_df['Air Date'].dt.strftime('%Y') < str(2000)].sort_values(by='Air Date')\n",
    "rows_contained = filtered_air_dates[filtered_air_dates['Question'].str.contains(word,case=False)]\n",
    "rows_contained_after = air_dates_after[air_dates_after['Question'].str.contains(word,case=False)]\n",
    "\n",
    "print(\"The no. of rows from the 90s is: {} and from the 2000s is: {}\".format(len(rows_contained),len(rows_contained_after)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "227"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Perhaps another way of doing it...?\n",
    "# This ignores rows with words like supercomputer,picking rows that have computer as an individual word.\n",
    "test = jeopardy_df[(jeopardy_df['Air Date'].dt.strftime('%Y') >= str(2000)) & (jeopardy_df['Air Date'].dt.strftime('%Y') < str(2010)) & (jeopardy_df['Question'].str.contains(r'\\bcomputer\\b',case=False))]\n",
    "len(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              Round  Category\n",
      "0  Double Jeopardy!       381\n",
      "1   Final Jeopardy!        10\n",
      "2         Jeopardy!       105\n"
     ]
    }
   ],
   "source": [
    "# Is there a connection between the round and the category? \n",
    "#Are you more likely to find certain categories, like \"Literature\" in Single Jeopardy or Double Jeopardy?\n",
    "\n",
    "# jeo = jeopardy_df.groupby('Round').get_group('Jeopardy!')\n",
    "# test = jeo[jeo.Category=='LITERATURE']\n",
    "\n",
    "a = jeopardy_df[jeopardy_df['Category'].map(lambda x: x=='LITERATURE')].groupby('Round').count().reset_index()\n",
    "print(a[['Round','Category']])\n",
    "\n",
    "\n",
    "# t = jeopardy_df[jeopardy_df['Category']=='LITERATURE']\n",
    "# p = len(t.groupby('Round').get_group('Double Jeopardy!'))/len(jeopardy_df['Round'])\n",
    "# round(p*100,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7. Compare your program to our <a href=\"https://content.codecademy.com/PRO/independent-practice-projects/jeopardy/jeopardy_solution.zip\">sample solution code</a> - remember, that your program might look different from ours (and probably will) and that's okay!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8. Great work! Visit <a href=\"https://discuss.codecademy.com/t/this-is-jeopardy-challenge-project-python-pandas/462365\">our forums</a> to compare your project to our sample solution code. You can also learn how to host your own solution on GitHub so you can share it with other learners! Your solution might look different from ours, and that's okay! There are multiple ways to solve these projects, and you'll learn more by seeing others' code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}