Day_030_HW.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# 作業 : (Kaggle)鐵達尼生存預測\n",
    "https://www.kaggle.com/c/titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [作業目標]\n",
    "- 試著調整特徵篩選的門檻值, 觀察會有什麼影響效果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [作業重點]\n",
    "- 調整相關係數過濾法的篩選門檻, 看看篩選結果的影響 (In[5]~In[8], Out[5]~Out[8])\n",
    "- 調整L1 嵌入法篩選門檻, 看看篩選結果的影響 (In[9]~In[11], Out[9]~Out[11])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass                                               Name  \\\n",
       "0         0       3                            Braund, Mr. Owen Harris   \n",
       "1         1       1  Cumings, Mrs. John Bradley (Florence Briggs Th...   \n",
       "2         1       3                             Heikkinen, Miss. Laina   \n",
       "3         1       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)   \n",
       "4         0       3                           Allen, Mr. William Henry   \n",
       "\n",
       "      Sex   Age  SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0    male  22.0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "1  female  38.0      1      0          PC 17599  71.2833   C85        C  \n",
       "2  female  26.0      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3  female  35.0      1      0            113803  53.1000  C123        S  \n",
       "4    male  35.0      0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 做完特徵工程前的所有準備 (與前範例相同)\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import copy\n",
    "from sklearn.preprocessing import LabelEncoder, MinMaxScaler\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "data_path = 'data/'\n",
    "df = pd.read_csv(data_path + 'titanic_train.csv')\n",
    "\n",
    "train_Y = df['Survived']\n",
    "df = df.drop(['PassengerId'] , axis=1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAdFklEQVR4nO3de7QV5Znn8e8PBBUxOKImXhi8EY0oEkEN2pmgid26zIQk3jUJzqjEWTpxpcfp1raDBJOVNm3SMd5W0+qIdhzxkihts7wExagxXLRBBIPSalq8JMHroHg55zzzR70Htrv2OWdv9q59Cb+Pqxa1q96qemqfcj/7feutdysiMDMzKzWo1QGYmVn7cXIwM7McJwczM8txcjAzsxwnBzMzy3FyMDOznKqSg6SLJK2Q9KSkpZIOrffAkr4k6YJ695P2ta4R+zEzs4wGes5B0iTgx8DkiHhf0g7A0Ih4ecCdS1tERFdjQu33OOsiYnjRxzEz21xUU3PYGVgbEe8DRMTaiHhZ0gspUSBpoqQFaX6GpFmS7gNulLRQ0tjenUlaIGmCpNMlXSlpRNrXoLR+mKQXJQ2RtJekeyQ9LulhSfumMntIekzSYkmXNPYtMTOzLaoocx8wXdIzwC+BORHx0ADbTAD+LCLWS/o2cCJwsaSdgV0i4nFJBwBExFuSlgGfAx4E/itwb0R8KGkWcHZEPJuasq4GjgQuB66JiBslndNXEJKmAdMArv7R9yac+Y1Tqjjd9jDpgKmtDqFmR205qtUh1GRYqNUh1GzPDzsv5nFbvtXqEGp2wPP/Uvcb/eHa56oefmLIDnu23R92wOQQEeskTQA+CxwBzKniXsHciFif5m8F7gcuJksSt1UoPwc4iSw5nAxcLWk4cBhwm7Thfdsy/Xs4cFyavwm4tI/YZwGzoLY/lJnZ5q6amgMR0Q0sABZIWg5MBbrY2Cy1Vdkm75Rs+5Kk1ySNI0sA36xwiLnADyRtT1breADYBngzIsb3FVY1sZuZtURPd6sjqMuA9xwk7SNpTMmi8cDvgBfIPshh47f4vtwC/BUwIiKWl6+MiHXAIrLmorsjojsi3gael3RCikOSDkybPEpWwwA4baBzMDNruu6u6qc2VM0N6eHAbEkrJT0J7AfMAL4LXC7pYWCgFHk72Yf5rf2UmQN8Lf3b6zTgjHRPYgUwJS0/DzhH0mJgRBXnYGbWVBE9VU/taMCurH8qOu2eg29IF883pJtjc70h/cGa5VV/5gzd7YC2+8NWdc/BzMxq1KY1gmo5OZiZFaHDb0g7OZiZFcE1BzMzKxdt2gupWk4OZmZF6OnsmoOH7DYzK0L0VD8NQNLRklZJWl1phApJoyXNTyNnL5C0W73hOzmYmRWhp7v6qR+SBgNXAceQPWd2iqT9yopdBtwYEeOAmcAP6g3fycHMrAiNqzkcAqyOiOci4gOyESemlJXZD5if5h+ssL5mTg5mZkWoYfgMSdMkLSmZppXsaVfgxZLXa9KyUsvYOIzRV4BtJY2sJ3zfkDYzK0INN6RLR5CuoNLT0+VPX58PXCnpdOBXwEtkg6NuMicHM7MCZINZN8QaoHRsmt2Aj/wSZ/plzq8CpJ87OC4i6hq3xM1KZmZFaNw9h8XAmPQLmEPJBjGdW1pA0g69v6YJXAhcX2/4Tg5mZkXo6al+6kdEdAHnAvcCTwO3RsQKSTMlfSkVmwysSr/Y+XHg+/WG72YlM7MiNHD4jIiYB8wrWza9ZP52sp9GaBgnBzOzInR/2OoI6uLkYGZWhA4fPmOzSQ6d9uM5jy2f3eoQavbpsae2OoSaXTh471aHUJPXB7c6gtp9+qUnWh1CzRoyZJ5HZTXrTJ2WGKzDuOZgZmY5Tg5mZlYufEPazMxyfM/BzMxy3KxkZmY5rjmYmVmOaw5mZpbjmoOZmeV0NeRRupZxcjAzK4JrDmZmluN7DmZmluOag5mZ5bjmYGZmOa45mJlZjnsrmZlZTkSrI6iLk4OZWRF8z8HMzHKcHPIkdQPL0/6fBqZGxLt9lJ0BrIuIy4qIxcysJTr8hvSggva7PiLGR8T+wAfA2QUdx8ysPXV3Vz+1oaKSQ6mHgb0BJH1D0pOSlkm6qbygpLMkLU7r75A0LC0/QdJTafmv0rKxkhZJWpr2OaYJ52JmVp2enuqnNlRocpC0BXAMsFzSWOAi4MiIOBA4r8ImP4+Ig9P6p4Ez0vLpwF+k5V9Ky84GLo+I8cBEYE2F40+TtETSkj+++2pDz83MrF9ODhVtLWkpsAT4D+A64Ejg9ohYCxARr1fYbn9JD0taDpwGjE3LHwVukHQWMDgtewz4G0l/DYyOiPXlO4uIWRExMSIm7jjsE408PzOz/kVP9VMbKqq30vr0jX4DSQIG6vh7A/DliFgm6XRgMkBEnC3pUOBYYKmk8RFxs6SFadm9ks6MiAcafB5mZpskejr7OYdm3HPoNR84UdJIAEnbVyizLfCKpCFkNQdS2b0iYmFETAfWAqMk7Qk8FxE/BeYC4wo/AzOzanV4s1LTnnOIiBWSvg88lLq6/htwelmx7wALgd+RdYXdNi3/+3TDWWRJZhlwAfA1SR8CrwIzCz8JM7NqtWkvpGoVkhwiYngfy2cDs8uWzSiZvwa4psJ2X62wux+kycys/bRpjaBafkLazKwITg5mZpbjgffMzCynw2sOzeytZGa2+eiJ6qcBSDpa0ipJqyVd0EeZEyWtlLRC0s31hu+ag5lZERrUW0nSYOAq4CiykSAWS5obEStLyowBLgQOj4g3JO1U73GdHMzMChCNa1Y6BFgdEc8BSLoFmAKsLClzFnBVRLwBEBF/qPegblYyMytCDc1KpePApWlayZ52BV4seb0mLSv1SeCTkh6V9BtJR9cbvmsOZmZFqGHMpIiYBczqY7UqbVL2egtgDNmQQ7sBD0vaPyLerDqIMq45mJkVoXE3pNcAo0pe7wa8XKHMXRHxYUQ8D6wiSxabzMnBzKwIXd3VT/1bDIyRtIekocDJZOPJlboTOAJA0g5kzUzP1RO+m5XMzIrQoKG4I6JL0rnAvWQ/WXB9GqtuJrAkIuamdX8uaSXQDfzviHitnuM6OZiZFaGBQ3ZHxDxgXtmy6SXzAfxlmhrCycHMrAAN7MraEk4OZmZF6PAf+3FyMDMrgpNDZzhqy1EDF2ojnx57aqtDqNm/rah7OJemm3zgma0OoSajB41odQg1+9nIya0OoTX8Yz9mnanTEoN1lk7/DWknBzOzIjg5mJlZjnsrmZlZjmsOZmaW4+RgZmblotvNSmZmVs41BzMzK+eurGZmlufkYGZmOZ19y8HJwcysCNHV2dnBycHMrAidnRucHMzMiuAb0mZmlueag5mZlXPNwczM8lxzMDOzctHV6gjq4+RgZlaA6PCaw6BWB9BL0lckhaR9Wx2LmVndemqY2lDbJAfgFOAR4ORWB2JmVq/oqX5qR22RHCQNBw4HziAlB0mDJF0taYWkuyXNk3R8WjdB0kOSHpd0r6SdWxi+mVmOk0NjfBm4JyKeAV6XdBDwVWB34ADgTGASgKQhwBXA8RExAbge+H6lnUqaJmmJpCVL/9/q4s/CzCyJblU9taN2uSF9CvCTNH9Lej0EuC0ieoBXJT2Y1u8D7A/cLwlgMPBKpZ1GxCxgFsCFu5/a2Z2OzayjtGuNoFotTw6SRgJHAvtLCrIP+wB+0dcmwIqImNSkEM3MahY97VkjqFY7NCsdD9wYEaMjYveIGAU8D6wFjkv3Hj4OTE7lVwE7StrQzCRpbCsCNzPrS6ffc2h5zYGsCenvypbdAXwKWAM8BTwDLATeiogP0o3pn0oaQXYOPwFWNC9kM7P+RXR2zaHlySEiJldY9lPIejFFxLrU9LQIWJ7WLwX+SzPjNDOrRbvWCKrV8uQwgLslbQcMBS6JiFdbHZCZWTV62rQXUrXaOjlUqlWYmXUC35A2M7Oc6FHV00AkHS1plaTVki6osP5sScslLZX0iKT96o3fycHMrAAR1U/9kTQYuAo4BtgPOKXCh//NEXFARIwHfgj8uN7427pZycysUzWwWekQYHVEPAcg6RZgCrByw7Ei3i4pvw3Zs2J1cXIwMytAA7uy7gq8WPJ6DXBoeSFJ5wB/SdaB58h6D+pmJTOzAnR3q+qpdBy4NE0r2VWlLJOrGUTEVRGxF/DXwN/WG79rDmZmBail5lA6DlwFa4BRJa93A17uZ3e3ANdUffA+uOZgZlaABvZWWgyMkbSHpKFkP2swt7SApDElL48Fnq03ftcczMwKMFAvpOr3E12SzgXuJRuY9PqIWCFpJrAkIuYC50r6AvAh8AYwtd7jOjmYmRWgkQ/BRcQ8YF7Zsukl8+c17GCJk4OZWQG6ezq71d7JwcysAI1qVmoVJwczswL0eMhuMzMr599zMDOzHDcrdYhhHZbFLxy8d6tDqNnkA89sdQg1W7Ds2laHUJO3p/63VodQsyeWDG11CC3hZiWzDtVpicE6i3srmZlZToe3Kjk5mJkVwc1KZmaW495KZmaW09PqAOrk5GBmVoCo+DMMncPJwcysAF1uVjIzs3KuOZiZWY7vOZiZWY5rDmZmluOag5mZ5XS75mBmZuUa+CuhLeHkYGZWgB7XHMzMrJwH3jMzsxzfkDYzs5weuVnJzMzKdLc6gDo15aeKJF0kaYWkJyUtlXSopGsl7ZfWr+tju89IWpi2eVrSjGbEa2ZWrx5VP7WjwmsOkiYBXwQOioj3Je0ADI2Ian5weDZwYkQskzQY2KfIWM3MGqXTeys1o+awM7A2It4HiIi1EfGypAWSJvYWkvQjSU9Imi9px7R4J+CVtF13RKxMZWdIuknSA5KelXRWE87DzKxqUcPUjpqRHO4DRkl6RtLVkj5Xocw2wBMRcRDwEHBxWv4PwCpJv5D0TUlblWwzDjgWmARMl7RL+U4lTZO0RNKSJetWN/SkzMz60+nNSoUnh4hYB0wApgF/BOZIOr2sWA8wJ83/M/BnaduZwESyBHMqcE/JNndFxPqIWAs8CBxS4dizImJiREycOHzvxp2UmdkAemqY2lFTeitFRDewAFggaTkwdaBNSrb9d+AaSf8E/FHSyPIyfbw2M2uZ7jatEVSr8JqDpH0kjSlZNB74XYU4jk/zpwKPpG2PlTZ0Fh5D1jvszfR6iqStUrKYDCwuIHwzs03imsPAhgNXSNoO6AJWkzUx3V5S5h1grKTHgbeAk9LyrwP/IOndtO1pEdGd8sUi4F+B/wxcEhEvN+FczMyq0q4f+tUqPDlExOPAYRVWTS4pMzzNfqds25P72fUzETGt7gDNzArQ4T8h7SekzcyK0Ok1h6Y8Id1oETEjIi5rdRxmZn3prmEaiKSjJa2StFrSBRXWbylpTlq/UNLu9cbfkcnBzKzdNeo5hzQ6xFXAMcB+wCm9Qw+VOAN4IyL2Jns+7NJ643dyMDMrQAN7Kx0CrI6I5yLiA+AWYEpZmSlkww1B1tnn8yU9PTeJk4OZWQFqSQ6lozmkqbSzza7AiyWv16RlVCoTEV1kvT5HUgffkDYzK0AtT+VGxCxgVh+rK9UAyndfTZmaODmYmRWggWMmrQFGlbzeDSh/rqu3zBpJWwAjgNfrOaiblczMCtDA3kqLgTGS9pA0FDgZmFtWZi4bhyU6HnggIlxzMDNrNz0NGu4tIroknQvcCwwGro+IFZJmAksiYi5wHXCTpNVkNYb+HiCuipODmVkBGvkQXETMA+aVLZteMv8ecEIDD+nkYGZWhE4fJtrJwcysAJ0+fIaTg5lZAbrU2XUHJwczswJ0dmpwcjAzK4SblTrEnh921uDqrw9udQS1Gz1oRKtDqMnUCf+LK/av6zmhpvvY7P/T6hBq9rFx57c6hJZoVFfWVtlskoNZuU5LDNZZOjs1ODmYmRXCzUpmZpbT3eF1BycHM7MCuOZgZmY54ZqDmZmVc83BzMxy3JXVzMxyOjs1ODmYmRWiq8PTg5ODmVkBfEPazMxyfEPazMxyXHMwM7Mc1xzMzCynO1xzMDOzMn7OwczMcnzPwczMcnzPwczMcjq9WWlQ0QeQ1C1pqaSnJN0maVgD9nm6pCsbEZ+ZWRGihv/aUeHJAVgfEeMjYn/gA+DsajeU1IG/pGxmlvVWqnZqR81IDqUeBvYGkHSnpMclrZA0rbeApHWSZkpaCEySdLCkX0taJmmRpG1T0V0k3SPpWUk/bPJ5mJn1q4eoempHTUsOkrYAjgGWp0X/PSImABOBb0kamZZvAzwVEYcCi4A5wHkRcSDwBWB9KjceOAk4ADhJ0qgKx5wmaYmkJQ+8+2xRp2ZmltNTw9SOmpEctpa0FFgC/AdwXVr+LUnLgN8Ao4AxaXk3cEea3wd4JSIWA0TE2xHRldbNj4i3IuI9YCUwuvzAETErIiZGxMQjh40pX21mVphOv+fQjN5K6yNifOkCSZPJagGTIuJdSQuArdLq9yKiu7cofQ+L/n7JfDfueWVmbaRdm4uq1ex7Dr1GAG+kxLAv8Jk+yv2W7N7CwQCStk3NU2ZmbS0iqp7aUas+aO8Bzpb0JLCKrGkpJyI+kHQScIWkrcnuN3yheWGamW2a7g6vORSeHCJieIVl75PdnB6wfLrfUF6zuCFNvWW+WG+cZmaN5GYlMzPLaVazkqTtJd2fuvXfL+k/VSgzOj06sDQ9PjDg82ZODmZmBWjicw4XkPXeHAPMT6/LvQIcljoHHQpcIGmX/nbq5GBmVoAmdmWdAsxO87OBL+diifggNecDbEkVn/1ODmZmBahl+IzSB3bTNG3gI2zw8Yh4BSD9u1OlQpJGpU5ALwKXRsTL/e3U3ULNzApQS3NRRMwCZvW1XtIvgU9UWHVRDcd4ERiXmpPulHR7RPy+r/JODmZmBWhkb6WI6LMLv6TfS9o5Il6RtDPwhwH29bKkFcBngdv7KudmJTOzAjTxIbi5wNQ0PxW4q7yApN3Ss2Kk3kyHkz1j1icnBzOzAjSxt9LfAUdJehY4Kr1G0kRJ16YynwIWpvHsHgIui4jlFfeWuFnJzKwAzRpQLyJeAz5fYfkS4Mw0fz8wrpb9OjmYmRWgO9p1MO7qODmYmRWgXQfUq5aTg5lZATp9bCUnBzOzArTrj/hUy8nBzKwAPW5WMjOzcq45mJlZjnsrdYhxW77V6hBq8umXnmh1CDX72cjJrQ6hJr98Yhjb93S1OoyafGzc+a0OoWYHPXlZq0NoCTcrmXWoTksM1lncrGRmZjmuOZiZWY5rDmZmltMd3a0OoS5ODmZmBfDwGWZmluPhM8zMLMc1BzMzy3FvJTMzy3FvJTMzy/HwGWZmluN7DmZmluN7DmZmluOag5mZ5fg5BzMzy3HNwczMctxbyczMcnxD2szMctystIkkdQPLSxZ9OSJeaFE4ZmYN5SekN936iBhf60aSBkd0+EDpZvYnr9NrDoNaHUApSbtLeljSE2k6LC2fLOlBSTeTahuSviZpkaSlkv5R0uCWBm9mVqInouqpLUVESyagG1iapl+kZcOArdL8GGBJmp8MvAPskV5/CvgXYEh6fTXwjQrHmAYsSdO0gs6jkP0W/N47ZsfrmD31Oym9wU0naV1EDC9bNgK4EhhPljw+GRHDJE0GLo6II1K5c4G/Af6QNt0a+L8RMaNJ4ZfGvCQiJjb7uPVwzMXrtHjBMdtHtVtvpW8DvwcOJGvyeq9k3Tsl8wJmR8SFTYzNzGyz0Vb3HIARwCsR0QN8HejrPsJ84HhJOwFI2l7S6CbFaGb2J6/dksPVwFRJvwE+yUdrCxtExErgb4H7JD0J3A/s3LQoP2pWi45bD8dcvE6LFxyzlWjZPQczM2tf7VZzMDOzNuDkYGZmOZtdcpB0kaQVkp5MD9Ad2oB9fknSBQ2Kb10NZbvTOTwl6TZJw/opO0PS+Y2IsQiSviIpJO3b6lgqqXTdSLpW0n5pfcW/m6TPSFqYtnla0owmxVv1tVHDPk+XdGUj4qviWL3x9067N+O4tlG7dWUtlKRJwBeBgyLifUk7AEOr3HaLiOiqtC4i5gJzGxdp1TYMQSLpZ8DZwI9bEEcjnAI8ApwMzGhtKB/V13UTEWdWsfls4MSIWJae4t+nyFhLbPK10SZD1Hh4nRbb3GoOOwNrI+J9gIhYGxEvS3oh/Q+PpImSFqT5GZJmSboPuDF9AxzbuzNJCyRN6P1GJWlE2tegtH6YpBclDZG0l6R7JD2ehgjZN5XZQ9JjkhZLuqSOc3sY2Dvt8xvpG+4ySTeVF5R0VjreMkl39H6rlHRC+qa5TNKv0rKxJcOUPClpTB0xViRpOHA4cAZZckDSIElXp2/rd0uaJ+n4tG6CpIfSe3mvpKJ7qvV13SyQtOEBLEk/Ujbsy3xJO6bFOwGvpO26U0+73mvrJkkPSHpW0lkFxl96bdyZ3rcVkqaVxL5O0kxJC4FJkg6W9Ot0LSyStG0quku6jp+V9MMCY86Rh9dprlY/ot3MCRhONlzHM2TdZj+Xlr8A7JDmJwIL0vwM4HFg6/T628B30/zOwDNp/nTgyjR/F3BEmj8JuDbNzwfGpPlDgQfS/FzS0B/AOcC6Gs5nXfp3i3Tc/wGMBVaVnM/2JedyfpofWbKP7wH/M80vB3ZN89ulf68ATkvzQ3vfiwb/Xb4GXJfmfw0cBBwPzCP7AvMJ4I20bEgqs2PJe3x9i66bBcDENB8l79P0kutheor9F8A32Tg8zAxgGdnT/TsALwK7NDDm3LVRdj1sDTzVey2k+E8s+Ts/BxycXn8s7ef0tHwEsBXwO2BUQe954cPreOp/2qyalSJinaQJwGeBI4A5GvhewdyIWJ/mbyV7puJi4ETgtgrl55B9YD1I9i346vTN+DDgNkm95bZM/x4OHJfmbwIureGUtpa0NM0/DFxH9gF0e0SsBYiI1ytst7+k7wHbkX3w3ZuWPwrcIOlW4Odp2WPARZJ2A34eEc/WEF+1TgF+kuZvSa+HALdF9kDkq5IeTOv3AfYH7k/v5WDSN/OiVHnd9JD97QH+mfT+RcTM1Kzz58CpZOc2OZW7K11b69P5HQLc2aCwK10bAN+S9JU0P4rsQ/Y1sg/jO9LyfcgeRl2czuFtgPR+z4+It9LrlcBossTWaJWalYYAV0raMLxOybpFEfF8mv88MAFYnGLemo1D7ViVNqvkAFnVnuwb3wJJy4GpQBcbm9i2KtvknZJtX5L0mqRxZAngmxUOMRf4gaTtyS7QB4BtgDcrXOwbdr2Jp5P7H0jZ/w0D7e8Gst/PWCbpdNKHVUScrewG/bHAUknjI+Lm1NRwLHCvpDMj4oFNjDdH0kjgSLKEFWQf9kH2TbviJsCKiJjUqBiq0cd10+8mJdv+O3CNpH8C/pjO+SNl+nhdj0rXxmTgC8CkiHhXWfNp7/X+Xmxsq+/vGnq/ZL6b5n6GeHidJtqs7jlI2qeszXw8WdX4BbIPctj4Lb4vtwB/BYyIiOXlKyNiHbAIuBy4O7J25reB5yWdkOKQpAPTJo+S2tmB02o/q5z5wIm9H0ApSZXbFnhF0pDSY0raKyIWRsR0YC0wStKewHMR8VOyxDeuATGWOh64MSJGR8TuETEKeD4d/7h07+HjbPy2vQrYUdlNYpTdzxlbaceN0s91U2pQOhfIagiPpG2P1cbq4hiyD9Q30+spkrZKf6vJwOICwi81AngjJYZ9gc/0Ue63ZPcWDgaQtK2kdvgi6eF1mmizSg5kTSizJa1UNuzGfmRtv98FLpf0MNn/vP25nezD/NZ+yswha0efU7LsNOAMScuAFcCUtPw84BxJi8ku/rpExArg+8BD6ViVeqh8B1hI1kT225Llfy9puaSngF+RtYmfBDyVmij2BW6sN8Yyp5CvJdwB7AKsIWsX/8cU71sR8QHZh/Cl6fyWkjXZFamv66bUO8BYSY+T1YRmpuVfB1al9+8msvsSvdfYIuBfgd8Al0TEy8WeBvcAW6RzuCQdNye9xycBV6T3+H7yNepW6MThdTqWh8+wtiVpeGrvH0n2QXp4RLza6rgaQdnzDusi4rJWx2JWSTtUFc36crek7ch6z1zyp5IYzDqBaw5mZpazud1zMDOzKjg5mJlZjpODmZnlODmYmVmOk4OZmeX8f7eubz7aNqmwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 計算df整體相關係數, 並繪製成熱圖\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "corr = df.corr()\n",
    "sns.heatmap(corr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5 Numeric Features : ['Pclass', 'Age', 'SibSp', 'Parch', 'Fare']\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pclass   Age  SibSp  Parch     Fare\n",
       "0       3  22.0      1      0   7.2500\n",
       "1       1  38.0      1      0  71.2833\n",
       "2       3  26.0      0      0   7.9250\n",
       "3       1  35.0      1      0  53.1000\n",
       "4       3  35.0      0      0   8.0500"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 記得刪除 Survived\n",
    "df = df.drop(['Survived'] , axis=1)\n",
    "\n",
    "#只取 int64, float64 兩種數值型欄位, 存於 num_features 中\n",
    "num_features = []\n",
    "for dtype, feature in zip(df.dtypes, df.columns):\n",
    "    if dtype == 'float64' or dtype == 'int64':\n",
    "        num_features.append(feature)\n",
    "print(f'{len(num_features)} Numeric Features : {num_features}\\n')\n",
    "\n",
    "# 削減文字型欄位, 只剩數值型欄位\n",
    "df = df[num_features]\n",
    "df = df.fillna(-1)\n",
    "MMEncoder = MinMaxScaler()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# 作業1\n",
    "* 鐵達尼生存率預測中，試著變更兩種以上的相關係數門檻值，觀察預測能力是否提升?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.7038635542329971"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 原始特徵 + 邏輯斯迴歸\n",
    "train_X = MMEncoder.fit_transform(df)\n",
    "estimator = LogisticRegression()\n",
    "cross_val_score(estimator, train_X, train_Y, cv=5).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "# 篩選相關係數1\n",
    "\n",
    "high_list = list(corr[(corr['Survived']>0.1) | (corr['Survived']<-0.1)].index)\n",
    "\n",
    "print(high_list.remove(\"Survived\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.6791567235397566"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 特徵1 + 邏輯斯迴歸\n",
    "# temp = df[high_list]\n",
    "# print(type(high_list))\n",
    "train_X = MMEncoder.fit_transform(df[high_list])\n",
    "cross_val_score(estimator, train_X, train_Y, cv=5).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n",
      "['Pclass', 'Age', 'Parch', 'Fare']\n"
     ]
    }
   ],
   "source": [
    "# 篩選相關係數2\n",
    "high_list = list(corr[(corr['Survived']>0.04) | (corr['Survived']<-0.04)].index)\n",
    "\n",
    "print(high_list.remove(\"Survived\"))\n",
    "print(high_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.7049872206659582"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 特徵2 + 邏輯斯迴歸\n",
    "train_X = MMEncoder.fit_transform(df[high_list])\n",
    "cross_val_score(estimator, train_X, train_Y, cv=5).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業2\n",
    "* 續上題，使用 L1 Embedding 做特徵選擇(自訂門檻)，觀察預測能力是否提升?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.4116012 , -0.18850803, -0.        ,  0.13666287,  0.01405623])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import Lasso\n",
    "L1_Reg = Lasso(alpha=0.003)\n",
    "train_X = MMEncoder.fit_transform(df)\n",
    "L1_Reg.fit(train_X, train_Y)\n",
    "L1_Reg.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Pclass', 'Age', 'Parch', 'Fare']"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from itertools import compress\n",
    "L1_mask = list((L1_Reg.coef_>0) | (L1_Reg.coef_<0))\n",
    "L1_list = list(compress(list(df), list(L1_mask)))\n",
    "L1_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "c:\\users\\henry\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.7049872206659582"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# L1_Embedding 特徵 + 線性迴歸\n",
    "train_X = MMEncoder.fit_transform(df[L1_list])\n",
    "cross_val_score(estimator, train_X, train_Y, cv=5).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}