Assignment2-Ashutosh.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 2\n",
    "___________________________________________________________________________________________________________________________\n",
    "\n",
    " \n",
    " ### _Data analysis with pandas_\n",
    " ===========================================================================================================================\n",
    " \n",
    " *In this task you should use Pandas to answer a few questions about the adult dataset given in the folder.*\n",
    " \n",
    " Unique values of all features:\n",
    "   * `age`: continuous.\n",
    "   \n",
    "   * `workclass`: Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked.\n",
    "   \n",
    "   * `fnlwgt`: continuous.\n",
    "   \n",
    "   * `education`: Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool.\n",
    "   \n",
    "   * `education-num`: continuous.\n",
    "   \n",
    "   * `marital-status`: Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse.\n",
    "   \n",
    "   * `occupation`: Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces.\n",
    "   \n",
    "   * `relationship`: Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried.\n",
    "   \n",
    "   * `race`: White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other, Black.\n",
    "   \n",
    "   * `sex`: Female, Male.\n",
    "   \n",
    "   * `capital-gain`: continuous.\n",
    "   \n",
    "   * `capital-loss`: continuous.\n",
    "   \n",
    "   * `hours-per-week`: continuous.\n",
    "   \n",
    "   * `native-country`: United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands.\n",
    "   \n",
    "   * `salary`: >50K,<=50K"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education</th>\n",
       "      <th>education-num</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>native-country</th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>77516</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>83311</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>215646</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>Private</td>\n",
       "      <td>234721</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>Private</td>\n",
       "      <td>338409</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Wife</td>\n",
       "      <td>Black</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>Cuba</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age         workclass  fnlwgt  education  education-num  \\\n",
       "0   39         State-gov   77516  Bachelors             13   \n",
       "1   50  Self-emp-not-inc   83311  Bachelors             13   \n",
       "2   38           Private  215646    HS-grad              9   \n",
       "3   53           Private  234721       11th              7   \n",
       "4   28           Private  338409  Bachelors             13   \n",
       "\n",
       "       marital-status         occupation   relationship   race     sex  \\\n",
       "0       Never-married       Adm-clerical  Not-in-family  White    Male   \n",
       "1  Married-civ-spouse    Exec-managerial        Husband  White    Male   \n",
       "2            Divorced  Handlers-cleaners  Not-in-family  White    Male   \n",
       "3  Married-civ-spouse  Handlers-cleaners        Husband  Black    Male   \n",
       "4  Married-civ-spouse     Prof-specialty           Wife  Black  Female   \n",
       "\n",
       "   capital-gain  capital-loss  hours-per-week native-country salary  \n",
       "0          2174             0              40  United-States  <=50K  \n",
       "1             0             0              13  United-States  <=50K  \n",
       "2             0             0              40  United-States  <=50K  \n",
       "3             0             0              40  United-States  <=50K  \n",
       "4             0             0              40           Cuba  <=50K  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(r'C:/Users/ashan/Machine-Learning-SMP-2020/Assingments/Week2/adult_data.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. How many men and women are represented in this dataset?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Male :  21790  Female :  10771\n"
     ]
    }
   ],
   "source": [
    "M = df[df.sex == 'Male']\n",
    "m = len(M)\n",
    "F = df[df.sex != 'Male']\n",
    "f = len(F)\n",
    "print(\"Male : \",m,\" Female : \",f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. What is the average age of women?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average age :  36.85823043357163\n"
     ]
    }
   ],
   "source": [
    "total_age_female = df['age'][df.sex == 'Female'].sum()\n",
    "print(\"Average age : \",total_age_female/f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. What is the percentage of german citizens?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "% of Germans :  0.42074874850281013 %\n"
     ]
    }
   ],
   "source": [
    "r = df.shape[0]                          \n",
    "G = df[df['native-country'] == 'Germany']\n",
    "g = len(G)\n",
    "print(\"% of Germans : \", g/r * 100, \"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. What is the mean of age for those who earn more than 50K per year and those who earn less than 50K per year? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean age for >50K :  44.24984058155847\n",
      "Mean age for <= 50K :  36.78373786407767\n"
     ]
    }
   ],
   "source": [
    "mean_age_salary1 = df['age'][df.salary == '>50K']\n",
    "print(\"Mean age for >50K : \",mean_age_salary1.sum()/mean_age_salary1.count())\n",
    "mean_age_salary2 = df['age'][df.salary == '<=50K']\n",
    "print(\"Mean age for <= 50K : \",mean_age_salary2.sum()/mean_age_salary2.count())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. What is the standard deviation of age for those who earn more than 50K per year and those who earn less than 50K per year? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10.51902771985177 14.020088490824813\n"
     ]
    }
   ],
   "source": [
    "salary = df.groupby('salary')\n",
    "rich = salary.get_group('>50K')\n",
    "age = rich['age'].std()\n",
    "poor = salary.get_group('<=50K')\n",
    "age2 = poor['age'].std()\n",
    "print(age, age2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Is it true that people who earn more than 50K have at least high school education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are people who earn more than 50K but still dont have a HS degree.\n"
     ]
    }
   ],
   "source": [
    "to_check = df[df['education-num'] < 9]\n",
    "to_check_final = to_check[df.salary == '>50K']\n",
    "if(len(to_check_final)):\n",
    "    print('There are people who earn more than 50K but still dont have a HS degree.')\n",
    "else:\n",
    "     print('it is true that people who earn more than 50K have at least high school education')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7. Display age statistics for each race and each gender. Use groupby() and describe(). Find the maximum age of men of Amer-Indian-Eskimo race.\n",
    "\n",
    "### 8. Among whom is the proportion of those who earn a lot (>50K) greater: married or single men? Consider as married those who have a marital-status starting with Married (Married-civ-spouse, Married-spouse-absent or Married-AF-spouse), the rest are considered bachelors.\n",
    "\n",
    "### 9. What is the maximum number of hours a person works per week? How many people work such a number of hours, and what is the percentage of those who earn a lot (>50K) among them?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          count       mean        std   min   25%   50%   75%   max\n",
      "sex                                                                \n",
      "Female  10771.0  36.858230  14.013697  17.0  25.0  35.0  46.0  90.0\n",
      "Male    21790.0  39.433547  13.370630  17.0  29.0  38.0  48.0  90.0\n",
      "                      count       mean        std   min   25%   50%   75%  \\\n",
      "race                                                                        \n",
      "Amer-Indian-Eskimo    311.0  37.173633  12.447130  17.0  28.0  35.0  45.5   \n",
      "Asian-Pac-Islander   1039.0  37.746872  12.825133  17.0  28.0  36.0  45.0   \n",
      "Black                3124.0  37.767926  12.759290  17.0  28.0  36.0  46.0   \n",
      "Other                 271.0  33.457565  11.538865  17.0  25.0  31.0  41.0   \n",
      "White               27816.0  38.769881  13.782306  17.0  28.0  37.0  48.0   \n",
      "\n",
      "                     max  \n",
      "race                      \n",
      "Amer-Indian-Eskimo  82.0  \n",
      "Asian-Pac-Islander  90.0  \n",
      "Black               90.0  \n",
      "Other               77.0  \n",
      "White               90.0  \n",
      "82\n"
     ]
    }
   ],
   "source": [
    "#for Q7\n",
    "sex = df.groupby('sex')\n",
    "race = df.groupby('race')\n",
    "Male = sex.get_group('Male')\n",
    "race_men = Male.groupby('race')\n",
    "AIE_men = race_men.get_group('Amer-Indian-Eskimo')\n",
    "print(sex['age'].describe())\n",
    "print(race['age'].describe())\n",
    "print(AIE_men['age'].max())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Married Men\n"
     ]
    }
   ],
   "source": [
    "#for Q8.\n",
    "salary = df.groupby('salary')\n",
    "greater = salary.get_group('>50K')\n",
    "Male = greater[greater.sex == 'Male']\n",
    "no_of_married = Male[Male['marital-status'] == 'Married-civ-spouse']\n",
    "marr = len(no_of_married)\n",
    "no_of_married = Male[Male['marital-status'] == 'Married-spouse-absent']\n",
    "marr += len(no_of_married)\n",
    "no_of_married = Male[Male['marital-status'] == 'Married-AF-spouse']\n",
    "marr += len(no_of_married)\n",
    "single = Male.shape[0] - marr\n",
    "if single > marr:\n",
    "    print('Single Men')\n",
    "elif single == marr:\n",
    "    print('Both are equal')\n",
    "else:\n",
    "    print('Married Men')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Max Hours per week :  99\n",
      "No of people :  85\n",
      "% of rich :  29.411764705882355 %\n"
     ]
    }
   ],
   "source": [
    "#For Q9\n",
    "print('Max Hours per week : ', df['hours-per-week'].max())\n",
    "people = df[df['hours-per-week'] == df['hours-per-week'].max()]\n",
    "no_of_people = len(people)\n",
    "print(\"No of people : \", no_of_people)\n",
    "earn_great = people[people.salary == '>50K']\n",
    "no_of_rich = len(earn_great)\n",
    "print(\"% of rich : \",no_of_rich/no_of_people * 100,\"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 10. Count the average time of work for those who earn a little and a lot for each country. What will these be for Japan?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "native-country  salary\n",
      "?               <=50K     40.164760\n",
      "                >50K      45.547945\n",
      "Cambodia        <=50K     41.416667\n",
      "                >50K      40.000000\n",
      "Canada          <=50K     37.914634\n",
      "                            ...    \n",
      "United-States   >50K      45.505369\n",
      "Vietnam         <=50K     37.193548\n",
      "                >50K      39.200000\n",
      "Yugoslavia      <=50K     41.600000\n",
      "                >50K      49.500000\n",
      "Name: hours-per-week, Length: 82, dtype: float64\n",
      "Average in Japan > 50k:  47.958333333333336 \n",
      " <= 50k :  41.0\n"
     ]
    }
   ],
   "source": [
    "v = df.groupby(['native-country','salary'])\n",
    "print(v['hours-per-week'].mean())\n",
    "v2 = df.groupby('salary')\n",
    "v3 = v2.get_group('>50K')\n",
    "v4 = v2.get_group('<=50K')\n",
    "jap1 = v3['hours-per-week'][v3['native-country'] == 'Japan'].mean()\n",
    "jap2 = v4['hours-per-week'][v4['native-country'] == 'Japan'].mean()\n",
    "print(\"Average in Japan > 50k: \", jap1,\"\\n <= 50k : \", jap2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}