Personal.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Detail of other tries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outline\n",
    "***\n",
    "* Preprocessing data\n",
    "* Tokenlization and punctutation cleaning\n",
    "* Hypertuning Word2Vec model\n",
    "* Choose the classifier\n",
    "* Hypertuning classifier\n",
    "* Result\n",
    "* Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Inport the necessary package\n",
    "\n",
    "import pandas as pd\n",
    "import nltk\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 6335 entries, 0 to 6334\n",
      "Data columns (total 5 columns):\n",
      "Unnamed: 0       6335 non-null int64\n",
      "title            6335 non-null object\n",
      "text             6335 non-null object\n",
      "label            6335 non-null object\n",
      "title_vectors    6335 non-null object\n",
      "dtypes: int64(1), object(4)\n",
      "memory usage: 247.5+ KB\n"
     ]
    }
   ],
   "source": [
    "# Check the basic information of the dataset\n",
    "\n",
    "df = pd.read_csv('data/fake_or_real_news.csv', nrows=10000)\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "      <th>title_vectors</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>You Can Smell Hillary’s Fear</td>\n",
       "      <td>Daniel Greenfield, a Shillman Journalism Fello...</td>\n",
       "      <td>0</td>\n",
       "      <td>[ 1.1533764e-02  4.2144405e-03  1.9692603e-02 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Watch The Exact Moment Paul Ryan Committed Pol...</td>\n",
       "      <td>Google Pinterest Digg Linkedin Reddit Stumbleu...</td>\n",
       "      <td>0</td>\n",
       "      <td>[ 0.11267698  0.02518966 -0.00212591  0.021095...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Kerry to go to Paris in gesture of sympathy</td>\n",
       "      <td>U.S. Secretary of State John F. Kerry said Mon...</td>\n",
       "      <td>1</td>\n",
       "      <td>[ 0.04253004  0.04300297  0.01848392  0.048672...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Bernie supporters on Twitter erupt in anger ag...</td>\n",
       "      <td>— Kaydee King (@KaydeeKing) November 9, 2016 T...</td>\n",
       "      <td>0</td>\n",
       "      <td>[ 0.10801624  0.11583211  0.02874823  0.061732...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>The Battle of New York: Why This Primary Matters</td>\n",
       "      <td>It's primary day in New York and front-runners...</td>\n",
       "      <td>1</td>\n",
       "      <td>[ 1.69016439e-02  7.13498285e-03 -7.81233795e-...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title  \\\n",
       "0                       You Can Smell Hillary’s Fear   \n",
       "1  Watch The Exact Moment Paul Ryan Committed Pol...   \n",
       "2        Kerry to go to Paris in gesture of sympathy   \n",
       "3  Bernie supporters on Twitter erupt in anger ag...   \n",
       "4   The Battle of New York: Why This Primary Matters   \n",
       "\n",
       "                                                text  label  \\\n",
       "0  Daniel Greenfield, a Shillman Journalism Fello...      0   \n",
       "1  Google Pinterest Digg Linkedin Reddit Stumbleu...      0   \n",
       "2  U.S. Secretary of State John F. Kerry said Mon...      1   \n",
       "3  — Kaydee King (@KaydeeKing) November 9, 2016 T...      0   \n",
       "4  It's primary day in New York and front-runners...      1   \n",
       "\n",
       "                                       title_vectors  \n",
       "0  [ 1.1533764e-02  4.2144405e-03  1.9692603e-02 ...  \n",
       "1  [ 0.11267698  0.02518966 -0.00212591  0.021095...  \n",
       "2  [ 0.04253004  0.04300297  0.01848392  0.048672...  \n",
       "3  [ 0.10801624  0.11583211  0.02874823  0.061732...  \n",
       "4  [ 1.69016439e-02  7.13498285e-03 -7.81233795e-...  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop('Unnamed: 0', inplace=True, axis=1)\n",
    "label_trans = lambda i: 0 if i == 'FAKE' else 1\n",
    "df.label = df.label.apply(label_trans)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a38ef5470>"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuAAAAEyCAYAAAC7wGWwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFo9JREFUeJzt3WuwZFd5HuD3Q8NNGCMpUhRFUjzCpSJRKjGMJ1gpbAebAEI4DE4IEUWMjLHl2FAxcS4eLhXID6pwbINNYoPFJZYIN3FXLDlYKMRUfnAZ2UQIhKIxCCQh0BhshIFCSHz5cfaItpgz02d0evU5Pc9Ttevsvfbu7u/MqjXzzurVu6u7AwAAjHG/ZRcAAADHEgEcAAAGEsABAGAgARwAAAYSwAEAYCABHAAABhLAAQBgIAEcAAAGEsABAGCgHcsu4L44+eSTe+fOncsuAwCAFXfNNdf8WXefshnPta0D+M6dO7Nv375llwEAwIqrqs9u1nNZggIAAAMJ4AAAMJAADgAAAwngAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAAOAAADCeBHaefeK5ZdAgAA25AADgAAAwngAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAAOAAADCeAAADDQwgJ4VZ1ZVR+oqk9W1Seq6pem9pOq6qqqunH6eeLUXlX1qqraX1XXVtWuRdUGAADLssgZ8LuS/NvuPifJuUmeW1XnJNmb5OruPjvJ1dNxkjwpydnTdlGSVy+wNgAAWIqFBfDuvq27/3ja/2qS65OcnmRPkkumyy5J8tRpf0+SS3vNh5KcUFWnLao+AABYhiFrwKtqZ5JHJflwklO7+7bp1BeSnDrtn57k5pmH3TK1AQDAylh4AK+q70nyziTP7+47Zs91dyfpDT7fRVW1r6r2HThwYBMrBQCAxVtoAK+q+2ctfL+pu981NX/x4NKS6eftU/utSc6cefgZU9tf0d0Xd/fu7t59yimnLK54AABYgEXeBaWSvD7J9d39iplTlye5cNq/MMl7Z9qfNd0N5dwkX5lZqgIAACthxwKf+zFJfirJx6vqY1PbC5O8PMllVfWcJJ9N8vTp3JVJzk+yP8nXkzx7gbUBAMBSLCyAd/f/SVLrnH7cIa7vJM9dVD0AALAV+CZMAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYaGEBvKreUFW3V9V1M20vrapbq+pj03b+zLkXVNX+qrqhqp64qLoAAGCZFjkD/ntJzjtE+yu7+5HTdmWSVNU5SS5I8nenx/xOVR23wNoAAGApFhbAu/uDSb485+V7kry1u7/Z3Z9Jsj/JoxdVGwAALMsy1oA/r6qunZaonDi1nZ7k5plrbpnaAABgpYwO4K9O8v1JHpnktiS/sdEnqKqLqmpfVe07cODAZtcHAAALNTSAd/cXu/vu7v52ktfmO8tMbk1y5sylZ0xth3qOi7t7d3fvPuWUUxZbMAAAbLKhAbyqTps5/MkkB++QcnmSC6rqgVV1VpKzk3xkZG0AADDCjkU9cVW9Jcljk5xcVbckeUmSx1bVI5N0kpuS/HySdPcnquqyJJ9McleS53b33YuqDQAAlmVhAby7n3GI5tcf5vqXJXnZouoBAICtwDdhAgDAQAI4AAAMJIADAMBAAjgAAAwkgAMAwEACOAAADCSAAwDAQAI4AAAMJIADAMBAcwXwqvp7iy4EAACOBfPOgP9OVX2kqn6xqh620IoAAGCFzRXAu/tHkjwzyZlJrqmqN1fV4xdaGQAArKC514B3941JXpzkV5L8oySvqqpPVdU/XVRxAACwauZdA/73q+qVSa5P8uNJ/kl3/51p/5ULrA8AAFbKjjmv+y9JXpfkhd39jYON3f35qnrxQioDAIAVNG8Af3KSb3T33UlSVfdL8qDu/np3v3Fh1QEAwIqZdw34+5M8eOb4+KkNAADYgHkD+IO6+y8PHkz7xy+mJAAAWF3zBvCvVdWugwdV9YNJvnGY6wEAgEOYdw3485O8vao+n6SS/I0k/2JhVQEAwIqaK4B390er6m8necTUdEN3f2txZQEAwGqadwY8Sf5Bkp3TY3ZVVbr70oVUBQAAK2quAF5Vb0zy/Uk+luTuqbmTCOAAALAB886A705yTnf3IosBAIBVN+9dUK7L2gcvAQCA+2DeGfCTk3yyqj6S5JsHG7v7KQupCgAAVtS8AfyliywCAACOFfPehvCPqur7kpzd3e+vquOTHLfY0gAAYPXMtQa8qn4uyTuS/O7UdHqS9yyqKAAAWFXzfgjzuUkek+SOJOnuG5P89UUVBQAAq2reAP7N7r7z4EFV7cjafcCZw869Vyy7BAAAtoh5A/gfVdULkzy4qh6f5O1J/sfiygIAgNU0bwDfm+RAko8n+fkkVyZ58aKKAgCAVTXvXVC+neS10wYAAByluQJ4VX0mh1jz3d0P3/SKAABghc37RTy7Z/YflOSfJzlp88sBAIDVNtca8O7+0sx2a3f/ZpInL7g2AABYOfMuQdk1c3i/rM2Izzt7DgAATOYN0b8xs39XkpuSPH3TqwEAgBU3711QfmzRhQAAwLFg3iUov3y48939is0pBwAAVtu8X8SzO8kvJDl92v5Vkl1JHjptHIGvowcAIJl/DfgZSXZ191eTpKpemuSK7v6XiyoMAABW0bwz4KcmuXPm+M6pDQAA2IB5Z8AvTfKRqnr3dPzUJJcspiQAAFhd894F5WVV9QdJfmRqenZ3/8niygIAgNU07xKUJDk+yR3d/VtJbqmqsxZUEwAArKy5AnhVvSTJryR5wdR0/yT/fVFFAQDAqpp3BvwnkzwlydeSpLs/H7cfBACADZs3gN/Z3Z2kk6SqHnKkB1TVG6rq9qq6bqbtpKq6qqpunH6eOLVXVb2qqvZX1bVVtetofpmtyP2/AQCYNW8Av6yqfjfJCVX1c0nen+S1R3jM7yU5715te5Nc3d1nJ7l6Ok6SJyU5e9ouSvLqOevaUoRtAACOZN67oPx6VT0+yR1JHpHkP3b3VUd4zAeraue9mvckeey0f0mS/521teV7klw6zbJ/qKpOqKrTuvu2OX8PAADYFo44A15Vx1XVB7r7qu7+9939744Uvg/j1JlQ/YV858t8Tk9y88x1t0xt29ahZsPNkAMAcMQA3t13J/l2VT1sM194dk35RlTVRVW1r6r2HThwYDNLGkIIBwA4ts37TZh/meTjVXVVpjuhJEl3/+sNvt4XDy4tqarTktw+td+a5MyZ686Y2r5Ld1+c5OIk2b1794YD/AhCNgAA65k3gL9r2u6ry5NcmOTl08/3zrQ/r6remuSHknzF+m8AAFbRYQN4Vf2t7v5cd1+y0Seuqrdk7QOXJ1fVLUlekrXgfVlVPSfJZ5M8fbr8yiTnJ9mf5OtJnr3R19uKzIQDAHBvR5oBf0+SXUlSVe/s7n827xN39zPWOfW4Q1zbSZ4773Nvdzv3XpGbXv7kZZcBAMASHOlDmDWz//BFFgIAAMeCIwXwXmefWGICAMDGHWkJyg9U1R1Zmwl/8LSf6bi7+3sXWh0AAKyYwwbw7j5uVCEAAHAsOOIX8QAAAJtHAAcAgIEEcAAAGEgABwCAgQRwAAAYSADfQtxXHABg9Qngm2yjIVroBgA4tgjgW4QgDgBwbBDAl+hIoVsoBwBYPQI4AAAMdNivomc5zHwDAKwuM+AAADCQAA4AAAMJ4JvEshEAAOYhgC+JwA4AcGwSwLcJgR0AYDUI4AAAMJAADgAAAwngAAAwkAC+BVjfDQBw7BDA7yPhGQCAjRDAAQBgIAF8E5gFBwBgXgL4FifcAwCsFgF8ixG4AQBWmwAOAAADCeDbiNlxAIDtTwAHAICBBHAAABhIAAcAgIEEcAAAGEgA3wZ8+BIAYHUI4AAAMJAADgAAAwngAAAwkAAOAAADCeDblA9mAgBsTwI4AAAMJIADAMBAAjgAAAwkgAMAwEACOAAADCSAAwDAQAI4AAAMJIADAMBAO5bxolV1U5KvJrk7yV3dvbuqTkrytiQ7k9yU5Ond/efLqA8AABZlmTPgP9bdj+zu3dPx3iRXd/fZSa6ejgEAYKVspSUoe5JcMu1fkuSpS6wFAAAWYlkBvJP8YVVdU1UXTW2ndvdt0/4Xkpy6nNK2l517r1h2CQAAbMCyAvgPd/euJE9K8tyq+tHZk93dWQvp36WqLqqqfVW178CBAwNK3VoEbgCA7W0pAby7b51+3p7k3UkeneSLVXVakkw/b1/nsRd39+7u3n3KKaeMKhkAADbF8ABeVQ+pqoce3E/yhCTXJbk8yYXTZRcmee/o2rYrs+IAANvHMm5DeGqSd1fVwdd/c3f/z6r6aJLLquo5ST6b5OlLqG1bOBi4BW8AgO1neADv7k8n+YFDtH8pyeNG1wMAACNtpdsQAgDAyhPAAQBgIAEcAAAGEsABAGAgARwAAAYSwFfE4W5J6HaFAABbhwAOAAADCeAAADCQAL5CLDUBANj6BPAVJIgDAGxdAjgAAAwkgAMAwEAC+AqzFAUAYOsRwAEAYCABHAAABhLAAQBgIAF8xRxc9239NwDA1iSAAwDAQAL4MWK9GXEz5QAAYwngAAAwkAAOAAADCeDHEMtNAACWTwAHAICBBHAAABhIAAcAgIEEcAAAGEgAP8Yc6oOYPpwJADCOAA4AAAMJ4CvO7DYAwNYigAMAwEAC+DHM7DgAwHgCOAAADCSAAwDAQDuWXQDjWXoCALA8ZsABAGAgARwAAAYSwAEAYCABnEOyThwAYDEEcI6KgA4AcHQEcJIcOlAL2QAAm08A57CEcACAzSWAAwDAQL6Ih3usN9u9c+8VuenlTx5cDQDAajIDDgAAAwngbNjsTPlG14hbUw4AHOsEcOayc+8V3xWeDx7fO5Af6loAANYI4AAAMJAAzoaY2QYAuG8EcDbFektU7ssX/NzXsD/qPwv+UwIAbMSWC+BVdV5V3VBV+6tq77Lr4b471LrwjXyQc9515Ydakz5vfffVPM8hqAMAyRYL4FV1XJLfTvKkJOckeUZVnbPcqliUeUL4PDPohwu26wX/9X5uhhFBeyuG+a1YEwBsRVsqgCd5dJL93f3p7r4zyVuT7FlyTQAAsGm2WgA/PcnNM8e3TG0cAzayjONwS1rWO573mkO9xnrLYI526clmzcDPuyznaGrbbEd6p+Le+/Nef19ed6OP2Y6z/Nux5vviWPt9ge2punvZNdyjqp6W5Lzu/tnp+KeS/FB3P2/mmouSXDQdPiLJDcMLXXNykj9b0mszH320PeinrU8fbQ/6aevTR9vDev30fd19yma8wI7NeJJNdGuSM2eOz5ja7tHdFye5eGRRh1JV+7p797LrYH36aHvQT1ufPtoe9NPWp4+2hxH9tNWWoHw0ydlVdVZVPSDJBUkuX3JNAACwabbUDHh331VVz0vyviTHJXlDd39iyWUBAMCm2VIBPEm6+8okVy67jjksfRkMR6SPtgf9tPXpo+1BP219+mh7WHg/bakPYQIAwKrbamvAAQBgpQngAAAwkAC+QVV1XlXdUFX7q2rvsutZdVV1ZlV9oKo+WVWfqKpfmtpPqqqrqurG6eeJU3tV1aum/rm2qnbNPNeF0/U3VtWFM+0/WFUfnx7zqqqq8b/p9ldVx1XVn1TV70/HZ1XVh6c/17dNdzZKVT1wOt4/nd858xwvmNpvqKonzrQbd5ugqk6oqndU1aeq6vqq+ofG0tZTVf9m+vvuuqp6S1U9yHhavqp6Q1XdXlXXzbQtfPys9xp8t3X66Nemv/Ourap3V9UJM+c2NEaOZhyuq7ttc25ZuzPLnyZ5eJIHJPm/Sc5Zdl2rvCU5Lcmuaf+hSf5fknOS/Ocke6f2vUl+ddo/P8kfJKkk5yb58NR+UpJPTz9PnPZPnM59ZLq2psc+adm/93bckvxykjcn+f3p+LIkF0z7r0nyC9P+LyZ5zbR/QZK3TfvnTGPqgUnOmsbaccbdpvbRJUl+dtp/QJITjKWttWXt258/k+TB0/FlSX7aeFr+luRHk+xKct1M28LHz3qvYZu7j56QZMe0/6szfbThMbLRcXi4zQz4xjw6yf7u/nR335nkrUn2LLmmldbdt3X3H0/7X01yfdb+gdqTtTCR6edTp/09SS7tNR9KckJVnZbkiUmu6u4vd/efJ7kqyXnTue/t7g/12si5dOa5mFNVnZHkyUleNx1Xkh9P8o7pknv30cG+e0eSx03X70ny1u7+Znd/Jsn+rI05424TVNXDsvaP0+uTpLvv7O6/iLG0Fe1I8uCq2pHk+CS3xXhauu7+YJIv36t5xPhZ7zW4l0P1UXf/YXffNR1+KGtf8phscIwc5b9r6xLAN+b0JDfPHN8ytTHA9JbOo5J8OMmp3X3bdOoLSU6d9tfro8O133KIdjbmN5P8hyTfno7/WpK/mPlLb/bP9Z6+mM5/Zbp+o33HxpyV5ECS/1ZrS4VeV1UPibG0pXT3rUl+Pcnnsha8v5LkmhhPW9WI8bPea7BxP5O1dxeSjffR0fy7ti4BnG2hqr4nyTuTPL+775g9N80WuJ/mklTVTyS5vbuvWXYtHNaOrL01++ruflSSr2Xt7ex7GEvLN63v3ZO1/zD9zSQPSXLeUotiLiPGjzF69KrqRUnuSvKmZdeSCOAbdWuSM2eOz5jaWKCqun/WwvebuvtdU/MXp7fsMv28fWpfr48O137GIdqZ32OSPKWqbsraW3U/nuS3svaW68Ev+5r9c72nL6bzD0vypWy879iYW5Lc0t0fno7fkbVAbixtLf84yWe6+0B3fyvJu7I2xoynrWnE+FnvNZhTVf10kp9I8szpPzHJxvvoS9n4OFyXAL4xH01y9vQp2AdkbaH95UuuaaVNa6hen+T67n7FzKnLkxz89PiFSd470/6s6RPo5yb5yvTW3fuSPKGqTpxmmJ6Q5H3TuTuq6tzptZ4181zMobtf0N1ndPfOrI2J/9Xdz0zygSRPmy67dx8d7LunTdf31H7B9Gnys5KcnbUPJRl3m6C7v5Dk5qp6xNT0uCSfjLG01XwuyblVdfz053iwn4ynrWnE+FnvNZhDVZ2XtSWST+nur8+c2tAYmcbVRsfh+ub5VKntr3zC9vys3YnjT5O8aNn1rPqW5Iez9nbbtUk+Nm3nZ21t1dVJbkzy/iQnTddXkt+e+ufjSXbPPNfPZO1DFvuTPHumfXeS66bH/NdM3xBrO6r+emy+cxeUh09/me1P8vYkD5zaHzQd75/OP3zm8S+a+uGGzNxBw7jbtP55ZJJ903h6T9buwmAsbbEtyX9K8qnpz/KNWbtLg/G0/H55S9bW5X8ra+8oPWfE+FnvNWxz99H+rK3PPpghXjNz/YbGyNGMw/U2X0UPAAADWYICAAADCeAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAAOAAAD/X/O7WjZqFexEAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10274cef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Draw a graph of text length verse frequency\n",
    "\n",
    "import matplotlib\n",
    "%matplotlib inline\n",
    "df['text'].str.len().plot(kind = 'hist', bins = 1000, figsize = (12,5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tokenization and punctuation cleaning\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tokenized the word for future use\n",
    "\n",
    "from string import punctuation\n",
    "texts = df.text+df.title\n",
    "mapping_table = {ord(char): u' ' for char in punctuation}\n",
    "tokenized = [nltk.word_tokenize(review.translate(mapping_table)) for review in texts]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Clean the tokenized text which stand for punctuation\n",
    "\n",
    "def clean_text(tokenized_list):\n",
    "    import string\n",
    "    sw = nltk.corpus.stopwords.words('english')\n",
    "    sw.append(\"“\")\n",
    "    sw.append(\"”\")\n",
    "    sw.append(\"’\")\n",
    "    sw.append(\"‘\")\n",
    "    sw.append(\"—\")\n",
    "    new_list = [[token.lower() for token in tlist if token not in string.punctuation \n",
    "                 and token.lower() not in sw] for tlist in tokenized_list]\n",
    "    return new_list\n",
    "cleaned = clean_text(tokenized)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the Word2Vec Model and LogisticRegression for checking the performance of Word2Vec parameters\n",
    "\n",
    "from gensim.models import Word2Vec\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import *\n",
    "target_names = ['FAKE', 'REAL']\n",
    "y = np.array(df['label']) \n",
    "seed = 2\n",
    "test_size = 0.33"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The easy-reading version of report\n",
    "\n",
    "def new_report(y0_test, y0_pred):\n",
    "    print (\"  Accuracy: {:.5f}  Precision: {:.5f}  Recall: {:.5f}  F-1: {:.5f}\"\n",
    "           .format(accuracy_score(y0_test, y0_pred), precision_score(y0_test, y0_pred), \n",
    "                   recall_score(y0_test, y0_pred), f1_score(y0_test, y0_pred)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The building function of word vectors\n",
    "\n",
    "def vectors_build(word_vectors, cleaned, word_model):\n",
    "    for i in range(0, len(df)):\n",
    "        word_vectors[i] = 0\n",
    "        for word in cleaned[i]:\n",
    "            word_vectors[i] += word_model[word]\n",
    "        if len(cleaned[i]) != 0:\n",
    "            word_vectors[i] = word_vectors[i] / len(cleaned[i])\n",
    "    return word_vectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hypertuning Word2Vec model\n",
    "***\n",
    "\n",
    "* We tested all the parameter that a word2vec model has, but it turns out only few of them are changeable.\n",
    "* In order to present clearly, we hide the unchanged parameters.\n",
    "* Changeable parameters including size, window, alpha, iter, batch words and negative.\n",
    "* We did use the randam searching for parameters in the word2vec model, but the result didn't go well.\n",
    "* Because of the number of parameters, grid search is also hard to implement.\n",
    "* Need to mention that although I set the seed value, because of workers = 4, I can not eliminate ordering jitter from OS thread scheduling, which cause some random result appeared.\n",
    "* In conclusion, each parameter can only contribute a very small percentage to the final result."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Original\n",
    "\n",
    "The original model is filled with the default parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  import sys\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec, original:\n",
      "  Accuracy: 0.89479  Precision: 0.90049  Recall: 0.88665  F-1: 0.89351\n"
     ]
    }
   ],
   "source": [
    "# Try the default parameter of Word2Vec Model\n",
    "\n",
    "word_model = Word2Vec(cleaned, seed = seed, min_count = 1, size = 100, \n",
    "                      window = 5, alpha = 0.025, iter = 10, \n",
    "                      batch_words = 10000, negative = 5)\n",
    "word_vectors = np.zeros((len(df), 100))\n",
    "word_vectors = vectors_build(word_vectors, cleaned, word_model)\n",
    "x0_train, x0_test, y0_train, y0_test = train_test_split(word_vectors, y, test_size=test_size, random_state=seed)\n",
    "LR0_model = LogisticRegression().fit(x0_train, y0_train)\n",
    "y0_pred = LR0_model.predict(x0_test)\n",
    "\n",
    "print(\"Word2Vec, original:\")\n",
    "new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing the size\n",
    "\n",
    "We use 50 as the scale, and try to find out the best size within 50 to 500."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  import sys\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec, size = 50 :\n",
      "  Accuracy: 0.88953  Precision: 0.89168  Recall: 0.88569  F-1: 0.88867\n",
      "Word2Vec, size = 100 :\n",
      "  Accuracy: 0.89718  Precision: 0.89865  Recall: 0.89433  F-1: 0.89649\n",
      "Word2Vec, size = 150 :\n",
      "  Accuracy: 0.89622  Precision: 0.89692  Recall: 0.89433  F-1: 0.89562\n",
      "Word2Vec, size = 200 :\n",
      "  Accuracy: 0.90196  Precision: 0.90115  Recall: 0.90202  F-1: 0.90158\n",
      "Word2Vec, size = 250 :\n",
      "  Accuracy: 0.89766  Precision: 0.89569  Recall: 0.89914  F-1: 0.89741\n",
      "Word2Vec, size = 300 :\n",
      "  Accuracy: 0.89861  Precision: 0.90048  Recall: 0.89529  F-1: 0.89788\n",
      "Word2Vec, size = 350 :\n",
      "  Accuracy: 0.90005  Precision: 0.90388  Recall: 0.89433  F-1: 0.89908\n",
      "Word2Vec, size = 400 :\n",
      "  Accuracy: 0.89527  Precision: 0.89672  Recall: 0.89241  F-1: 0.89456\n",
      "Word2Vec, size = 450 :\n",
      "  Accuracy: 0.89622  Precision: 0.89845  Recall: 0.89241  F-1: 0.89542\n",
      "Word2Vec, size = 500 :\n",
      "  Accuracy: 0.89766  Precision: 0.89875  Recall: 0.89529  F-1: 0.89702\n"
     ]
    }
   ],
   "source": [
    "# Testing the influence of size and use 50 as difference\n",
    "\n",
    "for j in range(1,11):\n",
    "    word_model = Word2Vec(cleaned, seed = seed, min_count = 1, size = 50*j, \n",
    "                      window = 5, alpha = 0.025, iter = 10, \n",
    "                      batch_words = 10000, negative = 5, workers = 4)\n",
    "    word_vectors = np.zeros((len(df), 50*j))\n",
    "    word_vectors = vectors_build(word_vectors, cleaned, word_model)\n",
    "    x0_train, x0_test, y0_train, y0_test = train_test_split(word_vectors, y, test_size=test_size, random_state=seed)\n",
    "    LR0_model = LogisticRegression().fit(x0_train, y0_train)\n",
    "    y0_pred = LR0_model.predict(x0_test)\n",
    "    \n",
    "    print(\"Word2Vec, size =\", 50*j, \":\")\n",
    "    new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apparently when the size = 350 we got the best performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing the window\n",
    "Window parameter represent the distance between the current and predicted word, and we change it from 1 to 10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  import sys\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec, window = 1 :\n",
      "  Accuracy: 0.88379  Precision: 0.89272  Recall: 0.87128  F-1: 0.88187\n",
      "Word2Vec, window = 2 :\n",
      "  Accuracy: 0.88714  Precision: 0.89733  Recall: 0.87320  F-1: 0.88510\n",
      "Word2Vec, window = 3 :\n",
      "  Accuracy: 0.89718  Precision: 0.89942  Recall: 0.89337  F-1: 0.89639\n",
      "Word2Vec, window = 4 :\n",
      "  Accuracy: 0.89813  Precision: 0.90351  Recall: 0.89049  F-1: 0.89695\n",
      "Word2Vec, window = 5 :\n",
      "  Accuracy: 0.89957  Precision: 0.90145  Recall: 0.89625  F-1: 0.89884\n",
      "Word2Vec, window = 6 :\n",
      "  Accuracy: 0.90005  Precision: 0.89923  Recall: 0.90010  F-1: 0.89966\n",
      "Word2Vec, window = 7 :\n",
      "  Accuracy: 0.90244  Precision: 0.89971  Recall: 0.90490  F-1: 0.90230\n",
      "Word2Vec, window = 8 :\n",
      "  Accuracy: 0.90292  Precision: 0.90288  Recall: 0.90202  F-1: 0.90245\n",
      "Word2Vec, window = 9 :\n",
      "  Accuracy: 0.90340  Precision: 0.90221  Recall: 0.90394  F-1: 0.90307\n",
      "Word2Vec, window = 10 :\n",
      "  Accuracy: 0.90196  Precision: 0.90192  Recall: 0.90106  F-1: 0.90149\n"
     ]
    }
   ],
   "source": [
    "for j in range(1,11):\n",
    "    word_model = Word2Vec(cleaned, seed = seed, min_count = 1, size = 350, \n",
    "                      window = j, alpha = 0.025, iter = 10, \n",
    "                      batch_words = 10000, negative = 5)\n",
    "    word_vectors = np.zeros((len(df), 350))\n",
    "    word_vectors = vectors_build(word_vectors, cleaned, word_model)\n",
    "    x0_train, x0_test, y0_train, y0_test = train_test_split(word_vectors, y, test_size=test_size, random_state=seed)\n",
    "    LR0_model = LogisticRegression().fit(x0_train, y0_train)\n",
    "    y0_pred = LR0_model.predict(x0_test)\n",
    "    \n",
    "    print(\"Word2Vec, window =\", j,\":\")\n",
    "    new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Easy to see the best window size is 9, which give us the highest result in all the dimension."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing the alpha\n",
    "\n",
    "The range of alpha is from 0.015 to 0.060, and the step is 0.005."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  import sys\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec, alpha = 0.015 :\n",
      "  Accuracy: 0.89909  Precision: 0.90058  Recall: 0.89625  F-1: 0.89841\n",
      "Word2Vec, alpha = 0.02 :\n",
      "  Accuracy: 0.90053  Precision: 0.90010  Recall: 0.90010  F-1: 0.90010\n",
      "Word2Vec, alpha = 0.025 :\n",
      "  Accuracy: 0.90531  Precision: 0.90490  Recall: 0.90490  F-1: 0.90490\n",
      "Word2Vec, alpha = 0.03 :\n",
      "  Accuracy: 0.90674  Precision: 0.90209  Recall: 0.91162  F-1: 0.90683\n",
      "Word2Vec, alpha = 0.035 :\n",
      "  Accuracy: 0.90244  Precision: 0.90202  Recall: 0.90202  F-1: 0.90202\n",
      "Word2Vec, alpha = 0.04 :\n",
      "  Accuracy: 0.90387  Precision: 0.90153  Recall: 0.90586  F-1: 0.90369\n",
      "Word2Vec, alpha = 0.045000000000000005 :\n",
      "  Accuracy: 0.90770  Precision: 0.90848  Recall: 0.90586  F-1: 0.90717\n",
      "Word2Vec, alpha = 0.05 :\n",
      "  Accuracy: 0.90866  Precision: 0.90631  Recall: 0.91066  F-1: 0.90848\n",
      "Word2Vec, alpha = 0.055 :\n",
      "  Accuracy: 0.90961  Precision: 0.90805  Recall: 0.91066  F-1: 0.90935\n"
     ]
    }
   ],
   "source": [
    "for j in range(1,10):\n",
    "    word_model = Word2Vec(cleaned, seed = seed, min_count = 1, size = 350, \n",
    "                      window = 9, alpha = 0.01+ 0.005*j, iter = 10, \n",
    "                      batch_words = 10000, workers = 3, negative = 5)\n",
    "    word_vectors = np.zeros((len(df), 350))\n",
    "    word_vectors = vectors_build(word_vectors, cleaned, word_model)\n",
    "    x0_train, x0_test, y0_train, y0_test = train_test_split(word_vectors, y, test_size=test_size, random_state=seed)\n",
    "    LR0_model = LogisticRegression()\n",
    "    LR0_model = LR0_model.fit(x0_train, y0_train)\n",
    "    y0_pred = LR0_model.predict(x0_test)\n",
    "    \n",
    "    print(\"Word2Vec, alpha =\",(0.01+0.005*j),\":\")\n",
    "    new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing the iters\n",
    "We change the iterier from 6 to 15."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  import sys\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec, alpha = 0.015 :\n",
      "  Accuracy: 0.89431  Precision: 0.89575  Recall: 0.89145  F-1: 0.89360\n",
      "Word2Vec, alpha = 0.02 :\n",
      "  Accuracy: 0.90340  Precision: 0.90375  Recall: 0.90202  F-1: 0.90288\n",
      "Word2Vec, alpha = 0.025 :\n",
      "  Accuracy: 0.89766  Precision: 0.89569  Recall: 0.89914  F-1: 0.89741\n",
      "Word2Vec, alpha = 0.03 :\n",
      "  Accuracy: 0.90100  Precision: 0.89943  Recall: 0.90202  F-1: 0.90072\n",
      "Word2Vec, alpha = 0.035 :\n",
      "  Accuracy: 0.89909  Precision: 0.89449  Recall: 0.90394  F-1: 0.89919\n",
      "Word2Vec, alpha = 0.04 :\n",
      "  Accuracy: 0.90100  Precision: 0.89790  Recall: 0.90394  F-1: 0.90091\n",
      "Word2Vec, alpha = 0.045000000000000005 :\n",
      "  Accuracy: 0.90053  Precision: 0.89856  Recall: 0.90202  F-1: 0.90029\n",
      "Word2Vec, alpha = 0.05 :\n",
      "  Accuracy: 0.90196  Precision: 0.90115  Recall: 0.90202  F-1: 0.90158\n",
      "Word2Vec, alpha = 0.055 :\n",
      "  Accuracy: 0.90483  Precision: 0.89943  Recall: 0.91066  F-1: 0.90501\n",
      "Word2Vec, alpha = 0.060000000000000005 :\n",
      "  Accuracy: 0.90244  Precision: 0.90048  Recall: 0.90394  F-1: 0.90221\n"
     ]
    }
   ],
   "source": [
    "for j in range(1,11):\n",
    "    word_model = Word2Vec(cleaned, size = 300, window = 5, min_count = 1, \n",
    "                          alpha = 0.045, iter= 5 + j, \n",
    "                          batch_words = 10000, workers = 4, negative = 5)\n",
    "    word_vectors = np.zeros((len(df), 300))\n",
    "    word_vectors = vectors_build(word_vectors, cleaned, word_model)\n",
    "    x0_train, x0_test, y0_train, y0_test = train_test_split(word_vectors, y, test_size=test_size, random_state=seed)\n",
    "    LR0_model = LogisticRegression().fit(x0_train, y0_train)\n",
    "    y0_pred = LR0_model.predict(x0_test)\n",
    "    \n",
    "    print(\"Word2Vec, iter =\",(5+j),\":\")\n",
    "    new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the iter did not have a specific change to final result, but we can see that the best iteration here is 14."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing the batch_words\n",
    "Batch words is target size (in words) for batches of examples passed to worker threads. And we try to reduce the parameter here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  import sys\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec, batch_words = 10000 :\n",
      "  Accuracy: 0.90626  Precision: 0.90200  Recall: 0.91066  F-1: 0.90631\n",
      "Word2Vec, batch_words = 9000 :\n",
      "  Accuracy: 0.90579  Precision: 0.90267  Recall: 0.90874  F-1: 0.90570\n",
      "Word2Vec, batch_words = 8000 :\n",
      "  Accuracy: 0.90531  Precision: 0.90181  Recall: 0.90874  F-1: 0.90526\n"
     ]
    }
   ],
   "source": [
    "for j in range(1, 4):\n",
    "    word_model = Word2Vec(cleaned, size = 350, window = 5, min_count = 1, \n",
    "                          alpha = 0.04, iter = 14, \n",
    "                          batch_words = 11000 - 1000*j, workers= 4, negative = 5)\n",
    "    word_vectors = np.zeros((len(df), 350))\n",
    "    word_vectors = vectors_build(word_vectors, cleaned, word_model)\n",
    "    x0_train, x0_test, y0_train, y0_test = train_test_split(word_vectors, y, test_size=test_size, random_state=seed)\n",
    "    LR0_model = LogisticRegression().fit(x0_train, y0_train)\n",
    "    \n",
    "    print(\"Word2Vec, batch_words =\",(11000 - 1000*j),\":\")\n",
    "    y0_pred = LR0_model.predict(x0_test)\n",
    "    new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Surprisingly when the batch_word = 10000 we get the best result."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing the nagetive\n",
    "Last but not least, we attempted to change the nagetive value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  import sys\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec, negative = 5 :\n",
      "  Accuracy: 0.90387  Precision: 0.89924  Recall: 0.90874  F-1: 0.90397\n",
      "Word2Vec, negative = 6 :\n",
      "  Accuracy: 0.90435  Precision: 0.89858  Recall: 0.91066  F-1: 0.90458\n",
      "Word2Vec, negative = 7 :\n",
      "  Accuracy: 0.90626  Precision: 0.90431  Recall: 0.90778  F-1: 0.90604\n",
      "Word2Vec, negative = 8 :\n",
      "  Accuracy: 0.90244  Precision: 0.89668  Recall: 0.90874  F-1: 0.90267\n",
      "Word2Vec, negative = 9 :\n",
      "  Accuracy: 0.90626  Precision: 0.90047  Recall: 0.91258  F-1: 0.90649\n"
     ]
    }
   ],
   "source": [
    "for j in range(1, 6):\n",
    "    word_model = Word2Vec(cleaned, size = 350, window = 5, min_count = 1, \n",
    "                          alpha = 0.04, iter = 14, \n",
    "                          batch_words = 10000, workers= 4, negative = 4+j)\n",
    "    word_vectors = np.zeros((len(df), 350))\n",
    "    word_vectors = vectors_build(word_vectors, cleaned, word_model)\n",
    "    x0_train, x0_test, y0_train, y0_test = train_test_split(word_vectors, y, test_size=test_size, random_state=seed)\n",
    "    LR0_model = LogisticRegression().fit(x0_train, y0_train)\n",
    "    \n",
    "    print(\"Word2Vec, negative =\",(4 + j),\":\")\n",
    "    y0_pred = LR0_model.predict(x0_test)\n",
    "    new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best result we get is when nagetive value = 7, so we choose that as our last parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result\n",
    "\n",
    "* The best combination of parameters:\n",
    "* size = 350, window = 5, alpha = 0.04, iter = 14, batch_words = 10000, negative = 7\n",
    "* The results:\n",
    "* Accuracy: 0.90626  Precision: 0.90431  Recall: 0.90778  F-1: 0.90604"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Choosing the parameter of Classifier\n",
    "***\n",
    "\n",
    "* We use logistic regression, random forest and xgboost to find the best result.\n",
    "* As usual, we hide the unnecessary and unchangeable parameters in the description of model.\n",
    "* We use grid search to help us find the best result.\n",
    "* To be more specific, we only show the result with the best parameter set in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The grid_search function\n",
    "\n",
    "def grid_search(model, X, Y):\n",
    "    grid_search = GridSearchCV(model, param, scoring=\"accuracy\", n_jobs = 1)\n",
    "    grid_result = grid_search.fit(X, Y)\n",
    "    print(\"Best Accuracy:\", grid_result.best_score_)\n",
    "    print(\"Parameter set:\", grid_result.best_params_)\n",
    "    model.set_params(**grid_result.best_params_)\n",
    "    model.fit(X, Y)\n",
    "    y0_pred = model.predict(x0_test)\n",
    "    new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  import sys\n"
     ]
    }
   ],
   "source": [
    "# Use the best result from Word2Vec\n",
    "\n",
    "word_model = Word2Vec(cleaned, size = 350, window = 5, min_count = 1, alpha = 0.04, \n",
    "                      iter = 14, batch_words = 10000, workers= 4, negative = 7)\n",
    "word_vectors = np.zeros((len(df), 350))\n",
    "word_vectors = vectors_build(word_vectors, cleaned, word_model)\n",
    "x0_train, x0_test, y0_train, y0_test = train_test_split(word_vectors, y, test_size=test_size, random_state=seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LogisticRegression\n",
    "**Positive**\n",
    "* The procession of calulation is very fast and do not use lot of storage area.\n",
    "\n",
    "**Nagetive**\n",
    "* Not good when processing a large amount of variables.\n",
    "* Less fitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Accuracy: 0.9147031102733271\n",
      "Parameter set: {'C': 2.0, 'intercept_scaling': 3, 'penalty': 'l2', 'tol': 0.0001}\n",
      "  Accuracy: 0.90292  Precision: 0.89905  Recall: 0.90682  F-1: 0.90292\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "param = {\n",
    "    'C': [1.0, 1.5, 2.0],\n",
    "    'tol':[0.0001, 0.0002, 0.0003],\n",
    "    'penalty': ['l1', 'l2'],\n",
    "    'intercept_scaling': [1,2,3]\n",
    "}\n",
    "\n",
    "grid_search(LogisticRegression(random_state = 2),x0_train,y0_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result\n",
    "\n",
    "* The best combination of parameters:\n",
    "* 'C': 2.0, 'intercept_scaling': 3, 'penalty': 'l2', 'tol': 0.0001\n",
    "* The results:\n",
    "* Accuracy: 0.91470  Precision: 0.90905  Recall: 0.91682  F-1: 0.91292"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Forest\n",
    "** Positive **\n",
    "* Easy to handle high dimension datas.\n",
    "* The result are smoother.\n",
    "\n",
    "** Nagetive **\n",
    "* Only when the number of trees are big enough.\n",
    "* Difficult to handle inbalance dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.7s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.7s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.7s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n",
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.5s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.7s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.7s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n",
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.4s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n",
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.9s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n",
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.9s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.7s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.7s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    1.8s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n",
      "building tree 1 of 10\n",
      "building tree 2 of 10\n",
      "building tree 3 of 10\n",
      "building tree 4 of 10\n",
      "building tree 5 of 10\n",
      "building tree 6 of 10\n",
      "building tree 7 of 10\n",
      "building tree 8 of 10\n",
      "building tree 9 of 10\n",
      "building tree 10 of 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n",
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 25\n",
      "building tree 2 of 25\n",
      "building tree 3 of 25\n",
      "building tree 4 of 25\n",
      "building tree 5 of 25\n",
      "building tree 6 of 25\n",
      "building tree 7 of 25\n",
      "building tree 8 of 25\n",
      "building tree 9 of 25\n",
      "building tree 10 of 25\n",
      "building tree 11 of 25\n",
      "building tree 12 of 25\n",
      "building tree 13 of 25\n",
      "building tree 14 of 25\n",
      "building tree 15 of 25\n",
      "building tree 16 of 25\n",
      "building tree 17 of 25\n",
      "building tree 18 of 25\n",
      "building tree 19 of 25\n",
      "building tree 20 of 25\n",
      "building tree 21 of 25\n",
      "building tree 22 of 25\n",
      "building tree 23 of 25\n",
      "building tree 24 of 25\n",
      "building tree 25 of 25\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.3s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 50\n",
      "building tree 2 of 50\n",
      "building tree 3 of 50\n",
      "building tree 4 of 50\n",
      "building tree 5 of 50\n",
      "building tree 6 of 50\n",
      "building tree 7 of 50\n",
      "building tree 8 of 50\n",
      "building tree 9 of 50\n",
      "building tree 10 of 50\n",
      "building tree 11 of 50\n",
      "building tree 12 of 50\n",
      "building tree 13 of 50\n",
      "building tree 14 of 50\n",
      "building tree 15 of 50\n",
      "building tree 16 of 50\n",
      "building tree 17 of 50\n",
      "building tree 18 of 50\n",
      "building tree 19 of 50\n",
      "building tree 20 of 50\n",
      "building tree 21 of 50\n",
      "building tree 22 of 50\n",
      "building tree 23 of 50\n",
      "building tree 24 of 50\n",
      "building tree 25 of 50\n",
      "building tree 26 of 50\n",
      "building tree 27 of 50\n",
      "building tree 28 of 50\n",
      "building tree 29 of 50\n",
      "building tree 30 of 50\n",
      "building tree 31 of 50\n",
      "building tree 32 of 50\n",
      "building tree 33 of 50\n",
      "building tree 34 of 50\n",
      "building tree 35 of 50\n",
      "building tree 36 of 50\n",
      "building tree 37 of 50\n",
      "building tree 38 of 50\n",
      "building tree 39 of 50\n",
      "building tree 40 of 50\n",
      "building tree 41 of 50\n",
      "building tree 42 of 50\n",
      "building tree 43 of 50\n",
      "building tree 44 of 50\n",
      "building tree 45 of 50\n",
      "building tree 46 of 50\n",
      "building tree 47 of 50\n",
      "building tree 48 of 50\n",
      "building tree 49 of 50\n",
      "building tree 50 of 50\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Accuracy: 0.8897266729500471\n",
      "Parameter set: {'min_samples_split': 3, 'min_weight_fraction_leaf': 0.0, 'n_estimators': 50, 'verbose': 0}\n",
      "  Accuracy: 0.88714  Precision: 0.90130  Recall: 0.86840  F-1: 0.88454\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "param = {\n",
    "    'n_estimators': [10, 25, 50],\n",
    "    'min_samples_split':[2, 3, 4],\n",
    "    'verbose': [0, 3],\n",
    "    'min_weight_fraction_leaf': [0.0, 0.1]\n",
    "}\n",
    "\n",
    "grid_search(RandomForestClassifier(random_state = 2),x0_train,y0_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 100\n",
      "building tree 2 of 100\n",
      "building tree 3 of 100\n",
      "building tree 4 of 100\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.1s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 5 of 100\n",
      "building tree 6 of 100\n",
      "building tree 7 of 100\n",
      "building tree 8 of 100\n",
      "building tree 9 of 100\n",
      "building tree 10 of 100\n",
      "building tree 11 of 100\n",
      "building tree 12 of 100\n",
      "building tree 13 of 100\n",
      "building tree 14 of 100\n",
      "building tree 15 of 100\n",
      "building tree 16 of 100\n",
      "building tree 17 of 100\n",
      "building tree 18 of 100\n",
      "building tree 19 of 100\n",
      "building tree 20 of 100\n",
      "building tree 21 of 100\n",
      "building tree 22 of 100\n",
      "building tree 23 of 100\n",
      "building tree 24 of 100\n",
      "building tree 25 of 100\n",
      "building tree 26 of 100\n",
      "building tree 27 of 100\n",
      "building tree 28 of 100\n",
      "building tree 29 of 100\n",
      "building tree 30 of 100\n",
      "building tree 31 of 100\n",
      "building tree 32 of 100\n",
      "building tree 33 of 100\n",
      "building tree 34 of 100\n",
      "building tree 35 of 100\n",
      "building tree 36 of 100\n",
      "building tree 37 of 100\n",
      "building tree 38 of 100\n",
      "building tree 39 of 100\n",
      "building tree 40 of 100\n",
      "building tree 41 of 100\n",
      "building tree 42 of 100\n",
      "building tree 43 of 100\n",
      "building tree 44 of 100\n",
      "building tree 45 of 100\n",
      "building tree 46 of 100\n",
      "building tree 47 of 100\n",
      "building tree 48 of 100\n",
      "building tree 49 of 100\n",
      "building tree 50 of 100\n",
      "building tree 51 of 100\n",
      "building tree 52 of 100\n",
      "building tree 53 of 100\n",
      "building tree 54 of 100\n",
      "building tree 55 of 100\n",
      "building tree 56 of 100\n",
      "building tree 57 of 100\n",
      "building tree 58 of 100\n",
      "building tree 59 of 100\n",
      "building tree 60 of 100\n",
      "building tree 61 of 100\n",
      "building tree 62 of 100\n",
      "building tree 63 of 100\n",
      "building tree 64 of 100\n",
      "building tree 65 of 100\n",
      "building tree 66 of 100\n",
      "building tree 67 of 100\n",
      "building tree 68 of 100\n",
      "building tree 69 of 100\n",
      "building tree 70 of 100\n",
      "building tree 71 of 100\n",
      "building tree 72 of 100\n",
      "building tree 73 of 100\n",
      "building tree 74 of 100\n",
      "building tree 75 of 100\n",
      "building tree 76 of 100\n",
      "building tree 77 of 100\n",
      "building tree 78 of 100\n",
      "building tree 79 of 100\n",
      "building tree 80 of 100\n",
      "building tree 81 of 100\n",
      "building tree 82 of 100\n",
      "building tree 83 of 100\n",
      "building tree 84 of 100\n",
      "building tree 85 of 100\n",
      "building tree 86 of 100\n",
      "building tree 87 of 100\n",
      "building tree 88 of 100\n",
      "building tree 89 of 100\n",
      "building tree 90 of 100\n",
      "building tree 91 of 100\n",
      "building tree 92 of 100\n",
      "building tree 93 of 100\n",
      "building tree 94 of 100\n",
      "building tree 95 of 100\n",
      "building tree 96 of 100\n",
      "building tree 97 of 100\n",
      "building tree 98 of 100\n",
      "building tree 99 of 100\n",
      "building tree 100 of 100\n",
      "Word2Vec + RF: 0.8923959827833573\n",
      "  Accuracy: 0.89240  Precision: 0.91046  Recall: 0.86936  F-1: 0.88943\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    5.4s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    0.0s finished\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    0.0s finished\n"
     ]
    }
   ],
   "source": [
    "RF0_model = RandomForestClassifier(n_estimators=100, criterion='gini', max_depth=None, min_samples_split=4, \n",
    "                                   min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='auto', \n",
    "                                   max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, \n",
    "                                   bootstrap=True, oob_score=False, n_jobs=1, random_state=None, verbose=3, \n",
    "                                   warm_start=False, class_weight=None)\n",
    "RF0_model = RF0_model.fit(x0_train, y0_train)\n",
    "print(\"Word2Vec + RF:\",RF0_model.score(x0_test, y0_test))\n",
    "y0_pred = RF0_model.predict(x0_test)\n",
    "new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result\n",
    "\n",
    "* The best combination of parameters:\n",
    "* 'min_samples_split': 'min_samples_split': 3, 'min_weight_fraction_leaf': 0.0, 'n_estimators': 100, 'verbose': 0\n",
    "* The results:\n",
    "* Accuracy: 0.89240  Precision: 0.91046  Recall: 0.86936  F-1: 0.88943"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Xgboost\n",
    "\n",
    "** Positive **\n",
    "* Regularization term is added into the algorithem, decrease the complexity and made itself faster.\n",
    "* The variance is also decreased with regularization term added.\n",
    "\n",
    "** Nagetive **\n",
    "* Time\n",
    "\n",
    "** Need to montion **\n",
    "* Because of the time cost, we did not use the grid search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec + XG: 0.8895265423242468\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "       FAKE       0.88      0.90      0.89      1050\n",
      "       REAL       0.90      0.88      0.89      1041\n",
      "\n",
      "avg / total       0.89      0.89      0.89      2091\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n",
      "  if diff:\n",
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n",
      "  if diff:\n"
     ]
    }
   ],
   "source": [
    "from xgboost import XGBClassifier\n",
    "XG0_model = XGBClassifier(max_depth=7, learning_rate=0.2, \n",
    "                        n_estimators=20, silent=True, \n",
    "                        objective='binary:logistic', nthread=-1, \n",
    "                        gamma=0, min_child_weight=1, max_delta_step=0, \n",
    "                        subsample=1, colsample_bytree=1, \n",
    "                        colsample_bylevel=1, reg_alpha=0, \n",
    "                        reg_lambda=1, scale_pos_weight=1, \n",
    "                        base_score=0.5, seed=0, missing=None)\n",
    "XG0_model = XG0_model.fit(x0_train, y0_train)\n",
    "print(\"Word2Vec + XG:\",XG0_model.score(x0_test, y0_test))\n",
    "y0_pred = XG0_model.predict(x0_test)\n",
    "print(classification_report(y0_test, y0_pred, target_names=target_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec + XG: 0.8900047824007652\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "       FAKE       0.88      0.90      0.89      1050\n",
      "       REAL       0.90      0.88      0.89      1041\n",
      "\n",
      "avg / total       0.89      0.89      0.89      2091\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n",
      "  if diff:\n",
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n",
      "  if diff:\n"
     ]
    }
   ],
   "source": [
    "from xgboost import XGBClassifier\n",
    "XG0_model = XGBClassifier(max_depth=6, learning_rate=0.2, \n",
    "                        n_estimators=20, silent=True, \n",
    "                        objective='binary:logistic', nthread=-1, \n",
    "                        gamma=0, min_child_weight=1, max_delta_step=0, \n",
    "                        subsample=1, colsample_bytree=1, \n",
    "                        colsample_bylevel=1, reg_alpha=0, \n",
    "                        reg_lambda=1, scale_pos_weight=1, \n",
    "                        base_score=0.5, seed=0, missing=None)\n",
    "XG0_model = XG0_model.fit(x0_train, y0_train)\n",
    "print(\"Word2Vec + XG:\",XG0_model.score(x0_test, y0_test))\n",
    "y0_pred = XG0_model.predict(x0_test)\n",
    "print(classification_report(y0_test, y0_pred, target_names=target_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec + XG: 0.9096126255380201\n",
      "  Accuracy: 0.90961  Precision: 0.91359  Recall: 0.90394  F-1: 0.90874\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n",
      "  if diff:\n",
      "/Users/lifesaver/miniconda3/lib/python3.6/site-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n",
      "  if diff:\n"
     ]
    }
   ],
   "source": [
    "from xgboost import XGBClassifier\n",
    "XG0_model = XGBClassifier(max_depth=6, learning_rate=0.3, \n",
    "                        n_estimators=200, silent=True, \n",
    "                        objective='binary:logistic', nthread=-1, \n",
    "                        gamma=0, min_child_weight=2, max_delta_step=0, \n",
    "                        subsample=1, colsample_bytree=1, \n",
    "                        colsample_bylevel=1, reg_alpha=0, \n",
    "                        reg_lambda=3, scale_pos_weight=1, \n",
    "                        base_score=0.5, seed=0, missing=None)\n",
    "XG0_model = XG0_model.fit(x0_train, y0_train)\n",
    "print(\"Word2Vec + XG:\",XG0_model.score(x0_test, y0_test))\n",
    "y0_pred = XG0_model.predict(x0_test)\n",
    "new_report(y0_test, y0_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result\n",
    "\n",
    "* The best combination of parameters:\n",
    "* max_depth=6, learning_rate=0.3, n_estimators=250\n",
    "* The results:\n",
    "* Accuracy: 0.90866  Precision: 0.91504  Recall: 0.90010  F-1: 0.90751"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "***\n",
    "\n",
    "* According to result, the best classifier for word2vec is logistic regression with an over 91% accuracy.\n",
    "* Can not assure that the LR classifier is the best because of the comparison to last time result.\n",
    "* LR is not highly stable. XGboost is more stable when we rerun the test for several times.\n",
    "* Grid search is great, but when it comes many parameter, it may not efficient as before.\n",
    "* The score in the grid search is very important."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}