exercise-cross-validation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "04a9d9da",
   "metadata": {
    "papermill": {
     "duration": 0.007464,
     "end_time": "2023-07-02T20:36:56.376017",
     "exception": false,
     "start_time": "2023-07-02T20:36:56.368553",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**This notebook is an exercise in the [Intermediate Machine Learning](https://www.kaggle.com/learn/intermediate-machine-learning) course.  You can reference the tutorial at [this link](https://www.kaggle.com/alexisbcook/cross-validation).**\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcf93c16",
   "metadata": {
    "papermill": {
     "duration": 0.00645,
     "end_time": "2023-07-02T20:36:56.389464",
     "exception": false,
     "start_time": "2023-07-02T20:36:56.383014",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In this exercise, you will leverage what you've learned to tune a machine learning model with **cross-validation**.\n",
    "\n",
    "# Setup\n",
    "\n",
    "The questions below will give you feedback on your work. Run the following cell to set up the feedback system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "63f053a7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:36:56.404792Z",
     "iopub.status.busy": "2023-07-02T20:36:56.404296Z",
     "iopub.status.idle": "2023-07-02T20:36:56.447707Z",
     "shell.execute_reply": "2023-07-02T20:36:56.446577Z"
    },
    "papermill": {
     "duration": 0.054103,
     "end_time": "2023-07-02T20:36:56.450287",
     "exception": false,
     "start_time": "2023-07-02T20:36:56.396184",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Setup Complete\n"
     ]
    }
   ],
   "source": [
    "# Set up code checking\n",
    "import os\n",
    "if not os.path.exists(\"../input/train.csv\"):\n",
    "    os.symlink(\"../input/home-data-for-ml-course/train.csv\", \"../input/train.csv\")  \n",
    "    os.symlink(\"../input/home-data-for-ml-course/test.csv\", \"../input/test.csv\") \n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.ml_intermediate.ex5 import *\n",
    "print(\"Setup Complete\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "823e6d93",
   "metadata": {
    "papermill": {
     "duration": 0.006877,
     "end_time": "2023-07-02T20:36:56.464304",
     "exception": false,
     "start_time": "2023-07-02T20:36:56.457427",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "You will work with the [Housing Prices Competition for Kaggle Learn Users](https://www.kaggle.com/c/home-data-for-ml-course) from the previous exercise. \n",
    "\n",
    "![Ames Housing dataset image](https://storage.googleapis.com/kaggle-media/learn/images/lTJVG4e.png)\n",
    "\n",
    "Run the next code cell without changes to load the training and test data in `X` and `X_test`.  For simplicity, we drop categorical variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6502536e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:36:56.480230Z",
     "iopub.status.busy": "2023-07-02T20:36:56.479810Z",
     "iopub.status.idle": "2023-07-02T20:36:57.801624Z",
     "shell.execute_reply": "2023-07-02T20:36:57.800792Z"
    },
    "papermill": {
     "duration": 1.33282,
     "end_time": "2023-07-02T20:36:57.804147",
     "exception": false,
     "start_time": "2023-07-02T20:36:56.471327",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Read the data\n",
    "train_data = pd.read_csv('../input/train.csv', index_col='Id')\n",
    "test_data = pd.read_csv('../input/test.csv', index_col='Id')\n",
    "\n",
    "# Remove rows with missing target, separate target from predictors\n",
    "train_data.dropna(axis=0, subset=['SalePrice'], inplace=True)\n",
    "y = train_data.SalePrice              \n",
    "train_data.drop(['SalePrice'], axis=1, inplace=True)\n",
    "\n",
    "# Select numeric columns only\n",
    "numeric_cols = [cname for cname in train_data.columns if train_data[cname].dtype in ['int64', 'float64']]\n",
    "X = train_data[numeric_cols].copy()\n",
    "X_test = test_data[numeric_cols].copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93286bd8",
   "metadata": {
    "papermill": {
     "duration": 0.006746,
     "end_time": "2023-07-02T20:36:57.819639",
     "exception": false,
     "start_time": "2023-07-02T20:36:57.812893",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Use the next code cell to print the first several rows of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1acbf288",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:36:57.835781Z",
     "iopub.status.busy": "2023-07-02T20:36:57.834867Z",
     "iopub.status.idle": "2023-07-02T20:36:57.871199Z",
     "shell.execute_reply": "2023-07-02T20:36:57.870109Z"
    },
    "papermill": {
     "duration": 0.047229,
     "end_time": "2023-07-02T20:36:57.873739",
     "exception": false,
     "start_time": "2023-07-02T20:36:57.826510",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>MasVnrArea</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>BsmtFinSF2</th>\n",
       "      <th>...</th>\n",
       "      <th>GarageArea</th>\n",
       "      <th>WoodDeckSF</th>\n",
       "      <th>OpenPorchSF</th>\n",
       "      <th>EnclosedPorch</th>\n",
       "      <th>3SsnPorch</th>\n",
       "      <th>ScreenPorch</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>60</td>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2003</td>\n",
       "      <td>2003</td>\n",
       "      <td>196.0</td>\n",
       "      <td>706</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>548</td>\n",
       "      <td>0</td>\n",
       "      <td>61</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>20</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>1976</td>\n",
       "      <td>1976</td>\n",
       "      <td>0.0</td>\n",
       "      <td>978</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>460</td>\n",
       "      <td>298</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>60</td>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2001</td>\n",
       "      <td>2002</td>\n",
       "      <td>162.0</td>\n",
       "      <td>486</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>608</td>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>70</td>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>1915</td>\n",
       "      <td>1970</td>\n",
       "      <td>0.0</td>\n",
       "      <td>216</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>642</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>272</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>2000</td>\n",
       "      <td>2000</td>\n",
       "      <td>350.0</td>\n",
       "      <td>655</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>836</td>\n",
       "      <td>192</td>\n",
       "      <td>84</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 36 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    MSSubClass  LotFrontage  LotArea  OverallQual  OverallCond  YearBuilt  \\\n",
       "Id                                                                          \n",
       "1           60         65.0     8450            7            5       2003   \n",
       "2           20         80.0     9600            6            8       1976   \n",
       "3           60         68.0    11250            7            5       2001   \n",
       "4           70         60.0     9550            7            5       1915   \n",
       "5           60         84.0    14260            8            5       2000   \n",
       "\n",
       "    YearRemodAdd  MasVnrArea  BsmtFinSF1  BsmtFinSF2  ...  GarageArea  \\\n",
       "Id                                                    ...               \n",
       "1           2003       196.0         706           0  ...         548   \n",
       "2           1976         0.0         978           0  ...         460   \n",
       "3           2002       162.0         486           0  ...         608   \n",
       "4           1970         0.0         216           0  ...         642   \n",
       "5           2000       350.0         655           0  ...         836   \n",
       "\n",
       "    WoodDeckSF  OpenPorchSF  EnclosedPorch  3SsnPorch  ScreenPorch  PoolArea  \\\n",
       "Id                                                                             \n",
       "1            0           61              0          0            0         0   \n",
       "2          298            0              0          0            0         0   \n",
       "3            0           42              0          0            0         0   \n",
       "4            0           35            272          0            0         0   \n",
       "5          192           84              0          0            0         0   \n",
       "\n",
       "    MiscVal  MoSold  YrSold  \n",
       "Id                           \n",
       "1         0       2    2008  \n",
       "2         0       5    2007  \n",
       "3         0       9    2008  \n",
       "4         0       2    2006  \n",
       "5         0      12    2008  \n",
       "\n",
       "[5 rows x 36 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4708ae40",
   "metadata": {
    "papermill": {
     "duration": 0.007472,
     "end_time": "2023-07-02T20:36:57.888656",
     "exception": false,
     "start_time": "2023-07-02T20:36:57.881184",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "So far, you've learned how to build pipelines with scikit-learn.  For instance, the pipeline below will use [`SimpleImputer()`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html) to replace missing values in the data, before using [`RandomForestRegressor()`](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html) to train a random forest model to make predictions.  We set the number of trees in the random forest model with the `n_estimators` parameter, and setting `random_state` ensures reproducibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8086e14c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:36:57.905594Z",
     "iopub.status.busy": "2023-07-02T20:36:57.905210Z",
     "iopub.status.idle": "2023-07-02T20:36:58.281029Z",
     "shell.execute_reply": "2023-07-02T20:36:58.280129Z"
    },
    "papermill": {
     "duration": 0.387341,
     "end_time": "2023-07-02T20:36:58.283608",
     "exception": false,
     "start_time": "2023-07-02T20:36:57.896267",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.impute import SimpleImputer\n",
    "\n",
    "my_pipeline = Pipeline(steps=[\n",
    "    ('preprocessor', SimpleImputer()),\n",
    "    ('model', RandomForestRegressor(n_estimators=50, random_state=0))\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f4e1022",
   "metadata": {
    "papermill": {
     "duration": 0.006917,
     "end_time": "2023-07-02T20:36:58.297952",
     "exception": false,
     "start_time": "2023-07-02T20:36:58.291035",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "You have also learned how to use pipelines in cross-validation.  The code below uses the [`cross_val_score()`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html) function to obtain the mean absolute error (MAE), averaged across five different folds.  Recall we set the number of folds with the `cv` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ff691e99",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:36:58.313422Z",
     "iopub.status.busy": "2023-07-02T20:36:58.313076Z",
     "iopub.status.idle": "2023-07-02T20:37:01.896361Z",
     "shell.execute_reply": "2023-07-02T20:37:01.895200Z"
    },
    "papermill": {
     "duration": 3.593838,
     "end_time": "2023-07-02T20:37:01.898724",
     "exception": false,
     "start_time": "2023-07-02T20:36:58.304886",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average MAE score: 18276.410356164386\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "# Multiply by -1 since sklearn calculates *negative* MAE\n",
    "scores = -1 * cross_val_score(my_pipeline, X, y,\n",
    "                              cv=5,\n",
    "                              scoring='neg_mean_absolute_error')\n",
    "\n",
    "print(\"Average MAE score:\", scores.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b3a864a",
   "metadata": {
    "papermill": {
     "duration": 0.00658,
     "end_time": "2023-07-02T20:37:01.912771",
     "exception": false,
     "start_time": "2023-07-02T20:37:01.906191",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Step 1: Write a useful function\n",
    "\n",
    "In this exercise, you'll use cross-validation to select parameters for a machine learning model.\n",
    "\n",
    "Begin by writing a function `get_score()` that reports the average (over three cross-validation folds) MAE of a machine learning pipeline that uses:\n",
    "- the data in `X` and `y` to create folds,\n",
    "- `SimpleImputer()` (with all parameters left as default) to replace missing values, and\n",
    "- `RandomForestRegressor()` (with `random_state=0`) to fit a random forest model.\n",
    "\n",
    "The `n_estimators` parameter supplied to `get_score()` is used when setting the number of trees in the random forest model.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "aa3a71e3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:37:01.928555Z",
     "iopub.status.busy": "2023-07-02T20:37:01.928177Z",
     "iopub.status.idle": "2023-07-02T20:37:03.067990Z",
     "shell.execute_reply": "2023-07-02T20:37:03.067018Z"
    },
    "papermill": {
     "duration": 1.150427,
     "end_time": "2023-07-02T20:37:03.070208",
     "exception": false,
     "start_time": "2023-07-02T20:37:01.919781",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"1_GetScore\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.impute import SimpleImputer\n",
    "\n",
    "def get_score(n_estimators):\n",
    "    \"\"\"Return the average MAE over 3 CV folds of random forest model.   \n",
    "    Keyword argument:\n",
    "    n_estimators -- the number of trees in the forest\n",
    "    \"\"\"\n",
    "    my_pipeline = Pipeline(steps=[\n",
    "        ('preprocessor', SimpleImputer()),\n",
    "        ('model', RandomForestRegressor(n_estimators, random_state=0))\n",
    "    ])\n",
    "    scores = -1 * cross_val_score(my_pipeline, X, y,\n",
    "                                  cv=3,\n",
    "                                  scoring='neg_mean_absolute_error')\n",
    "\n",
    "    return scores.mean()\n",
    "\n",
    "# Check your answer\n",
    "step_1.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f6020e08",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:37:03.086039Z",
     "iopub.status.busy": "2023-07-02T20:37:03.085733Z",
     "iopub.status.idle": "2023-07-02T20:37:03.094953Z",
     "shell.execute_reply": "2023-07-02T20:37:03.093894Z"
    },
    "papermill": {
     "duration": 0.019259,
     "end_time": "2023-07-02T20:37:03.096882",
     "exception": false,
     "start_time": "2023-07-02T20:37:03.077623",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 3, \"questionType\": 2, \"questionId\": \"1_GetScore\", \"learnToolsVersion\": \"0.3.4\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc99\">Solution:</span> \n",
       "```python\n",
       "def get_score(n_estimators):\n",
       "    my_pipeline = Pipeline(steps=[\n",
       "        ('preprocessor', SimpleImputer()),\n",
       "        ('model', RandomForestRegressor(n_estimators, random_state=0))\n",
       "    ])\n",
       "    scores = -1 * cross_val_score(my_pipeline, X, y,\n",
       "                                  cv=3,\n",
       "                                  scoring='neg_mean_absolute_error')\n",
       "    return scores.mean()\n",
       "\n",
       "```"
      ],
      "text/plain": [
       "Solution: \n",
       "```python\n",
       "def get_score(n_estimators):\n",
       "    my_pipeline = Pipeline(steps=[\n",
       "        ('preprocessor', SimpleImputer()),\n",
       "        ('model', RandomForestRegressor(n_estimators, random_state=0))\n",
       "    ])\n",
       "    scores = -1 * cross_val_score(my_pipeline, X, y,\n",
       "                                  cv=3,\n",
       "                                  scoring='neg_mean_absolute_error')\n",
       "    return scores.mean()\n",
       "\n",
       "```"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#step_1.hint()\n",
    "step_1.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3ac7379",
   "metadata": {
    "papermill": {
     "duration": 0.007233,
     "end_time": "2023-07-02T20:37:03.111882",
     "exception": false,
     "start_time": "2023-07-02T20:37:03.104649",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Step 2: Test different parameter values\n",
    "\n",
    "Now, you will use the function that you defined in Step 1 to evaluate the model performance corresponding to eight different values for the number of trees in the random forest: 50, 100, 150, ..., 300, 350, 400.\n",
    "\n",
    "Store your results in a Python dictionary `results`, where `results[i]` is the average MAE returned by `get_score(i)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "221c4f9f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:37:03.128459Z",
     "iopub.status.busy": "2023-07-02T20:37:03.128133Z",
     "iopub.status.idle": "2023-07-02T20:38:07.169648Z",
     "shell.execute_reply": "2023-07-02T20:38:07.168589Z"
    },
    "papermill": {
     "duration": 64.05943,
     "end_time": "2023-07-02T20:38:07.178966",
     "exception": false,
     "start_time": "2023-07-02T20:37:03.119536",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"2_GetDict\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "results = {} # Your code here\n",
    "for i in range(50,401,50):\n",
    "    a = get_score(i)\n",
    "    results[i] = a\n",
    "\n",
    "# Check your answer\n",
    "step_2.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "464c833c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:38:07.197370Z",
     "iopub.status.busy": "2023-07-02T20:38:07.196941Z",
     "iopub.status.idle": "2023-07-02T20:38:07.201394Z",
     "shell.execute_reply": "2023-07-02T20:38:07.200324Z"
    },
    "papermill": {
     "duration": 0.016151,
     "end_time": "2023-07-02T20:38:07.203514",
     "exception": false,
     "start_time": "2023-07-02T20:38:07.187363",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#step_2.hint()\n",
    "#step_2.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39cad8c0",
   "metadata": {
    "papermill": {
     "duration": 0.008174,
     "end_time": "2023-07-02T20:38:07.219972",
     "exception": false,
     "start_time": "2023-07-02T20:38:07.211798",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Use the next cell to visualize your results from Step 2.  Run the code without changes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "0182f2db",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:38:07.238509Z",
     "iopub.status.busy": "2023-07-02T20:38:07.237588Z",
     "iopub.status.idle": "2023-07-02T20:38:07.524020Z",
     "shell.execute_reply": "2023-07-02T20:38:07.522658Z"
    },
    "papermill": {
     "duration": 0.300182,
     "end_time": "2023-07-02T20:38:07.528394",
     "exception": false,
     "start_time": "2023-07-02T20:38:07.228212",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAGdCAYAAAAbudkLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABYb0lEQVR4nO3de1xU170+/mcGmGHkMoLcRC4iKooEVEwUITFNLahITZOojR5/mrbW1KKJUZtoTU2Tpib1ktR4tMm355hYXyeYxKJEDVErQfEeIngDFUFABC+AMzDAMMys3x/IxBFU0MENM8/79ZrTsveavT97z9R5ztprry0TQggQERER2SC51AUQERERdRYGHSIiIrJZDDpERERksxh0iIiIyGYx6BAREZHNYtAhIiIim8WgQ0RERDaLQYeIiIhslqPUBUjJZDLhypUrcHNzg0wmk7ocIiIiagchBGpqauDv7w+5/N59NnYddK5cuYLAwECpyyAiIqIHUFpaioCAgHu26XDQ2b9/P1auXIns7GyUl5cjNTUVzz77rHl9bW0t3njjDWzbtg2VlZXo27cv5s+fj9/97nettiWEwIQJE5Cent5qO9XV1Zg/fz7S0tIAAD//+c/x0UcfoWfPnuY2JSUl+P3vf499+/ZBpVJh2rRpWLVqFRQKRbuOxc3NDUDziXJ3d+/oqSAiIiIJaLVaBAYGmn/H76XDQUen0yEqKgovvfQSnn/++VbrFyxYgIyMDGzevBl9+/bF7t27MXfuXPj7+2PSpEkWbT/88MO7XjKaNm0aLl++jPT0dADAb3/7W8yYMQNff/01AMBoNCIxMRHe3t7IyspCZWUlZs6cCSEEPvroo3YdS8u+3d3dGXSIiIi6mXYNOxEPAYBITU21WDZkyBDx9ttvWywbPny4WLZsmcWynJwcERAQIMrLy1tt5+zZswKAOHLkiHnZ4cOHBQCRn58vhBBi165dQi6Xi7KyMnObzz//XCiVSqHRaNpVv0ajEQDa3Z6IiIik15Hfb6vfdRUXF4e0tDSUlZVBCIGMjAycP38eCQkJ5jZ1dXV48cUXsW7dOvj5+bXaxuHDh6FWqzFy5EjzslGjRkGtVuPQoUPmNhEREfD39ze3SUhIgF6vR3Z2trUPi4iIiLohqw9GXrt2LWbPno2AgAA4OjpCLpfjn//8J+Li4sxtFixYgNGjR7e6lNWioqICPj4+rZb7+PigoqLC3MbX19divYeHBxQKhbnNnfR6PfR6vflvrVbb4eMjIiKi7qNTgs6RI0eQlpaG4OBg7N+/H3PnzkXv3r0xduxYpKWlYd++fThx4sQ9t9PWdTchhMXy9rS53YoVK/DnP/+5g0dERERE3ZVVL13V19dj6dKlWLNmDZKSkhAZGYnk5GRMnToVq1atAgDs27cPFy9eRM+ePeHo6AhHx+as9fzzz+Ppp58GAPj5+eHq1auttn/9+nVzL46fn1+rnpvq6moYDIZWPT0tlixZAo1GY36VlpZa69CJiIioC7Jq0DEYDDAYDK0m73FwcIDJZAIAvPHGGzh58iRycnLMLwD44IMPsHHjRgBATEwMNBoNjh07Zt7G0aNHodFoMHr0aHOb06dPo7y83Nxm9+7dUCqViI6ObrM+pVJpvsOKd1oRERHZvg5fuqqtrUVBQYH576KiIuTk5MDT0xNBQUEYM2YMFi9eDJVKheDgYGRmZmLTpk1Ys2YNgOaemLYGIAcFBSEkJAQAMHjwYIwbNw6zZ8/Gxx9/DKD59vKJEyciLCwMABAfH4/w8HDMmDEDK1euRFVVFRYtWoTZs2czwBAREVGzjt7SlZGRIQC0es2cOVMIIUR5ebmYNWuW8Pf3F87OziIsLEysXr1amEymu24TbdymXllZKaZPny7c3NyEm5ubmD59uqiurrZoU1xcLBITE4VKpRKenp4iOTlZNDQ0tPtYeHs5ERFR99OR32+ZEEJIF7OkpdVqoVarodFo2AtERETUTXTk95tPLyciIiKbxaBDRERENotBh4iIiGwWgw51ir1nryIt94rUZRARkZ2z+szIRMcvVWH2v76HEEBkHzX6erlIXRIREdkp9uiQVen0TVj4RS5a7uXLKrghbUFERGTXGHTIqt77Jh8lVXXmvw8y6BARkYQYdMhqDly4jn8dKQYAvPazgQCAw4WVMJrsdqomIiKSGIMOWYWm3oA/fHUSADBjVDDmPh0KV6UjbtYZcPaKVuLqiIjIXjHokFW8/fVZlGsaENyrB94YPwiODnKM6tcLAMfpEBGRdBh06KHtPlOBrT9chkwGrJocBRdl8818sf2bg86hiww6REQkDQYdeihVukYsTT0FAPjtk/3weF9P87q4/l4AgGNFVWgwGCWpj4iI7BuDDj0wIQSWbTuFG7WNGODjigW3BiC36O/jCh83JfRNJvxQXC1RlUREZM8YdOiBpeVewa5TFXCQy7BmylA4OzlYrJfJZIi91atzkJeviIhIAgw69ECuahvwp+1nAADJP+mPxwLUbbZrCTpZBZWPrDYiIqIWDDrUYUIIvLH1JDT1BkT0cUfyM/3v2rZlQPKpyzehqTM8qhKJiIgAMOjQA/ji+1JknLsOhYMca6YMhZPD3b9GvdUqhHq7wCSaJw8kIiJ6lBh0qENKq+rw9tdnAQAL4wdioK/bfd/TcvmKt5kTEdGjxqBD7WYyCSz+Khe6RiOigz3wmyf7tet9P47TYdAhIqJHi0GH2u2zw5dwpLAKKicHrJ4cBQe5rF3vG9WvF+QyoPC6DuWa+k6ukoiI6EcMOtQuF6/X4r1v8gEASycMQl8vl3a/V61ywmMBPQEAB3n3FRERPUIMOnRfTUYTFn2ZC32TCXH9vTB9ZHCHtxF36+6rg7x8RUREjxCDDt3XJwcKcaLkJtyUjnj/hUjI23nJ6na3j9MRQli7RCIiojYx6NA95ZVr8cGe8wCAPyWFo09P1QNtZ3iQB5SOclyv0aPgWq01SyQiIrorBh26q8YmE177IhcGo8DYwT54ITrggbfl7OSAJ0KaH/jJu6+IiOhRYdChu/po3wXklWvh0cMJf33uMchkHb9kdTvzc68YdIiI6BFh0KE25ZTexPrvLgIA/vLsY/Bxc37obcaGNgedI4VVaDKaHnp7RERE98OgQ600GIxY+EUOjCaBpCh/JEb2tsp2w/3d0bOHE2r1Tci9rLHKNomIiO6FQYdaWfntOVy8roO3mxJv/3yI1bbrIJdhdChvMyciokeHQYcsHCmsxP8eLAIAvP/8Y/BwUVh1+6NDOU6HiIgeHQYdMqvVN2HxV7kQApg6IhDPDPK1+j7ibg1I/qGkGnWNTVbfPhER0e0YdMjsr7vyUFpVjz49VVg2cXCn7CO4Vw/06amCwShwrKiqU/ZBRETUgkGHAADfnbuG/ztaAgBY+UIk3JydOmU/MpnM3KvDy1dERNTZGHQImjoDXt96EgAwa3RfjL4VRDrLaPNzr/iATyIi6lwMOoS3vj6Dq1o9Qrxc8Pq4QZ2+v5YByWfLtais1Xf6/oiIyH51OOjs378fSUlJ8Pf3h0wmw7Zt2yzW19bWIjk5GQEBAVCpVBg8eDA2bNhg0WbOnDkIDQ2FSqWCt7c3Jk2ahPz8fIs258+fx6RJk+Dl5QV3d3fExsYiIyPDok1JSQmSkpLg4uICLy8vzJ8/H42NjR09JLuWfrocqSfKIJcBqyZHQaVw6PR9erspMcjPDQBw6CJ7dYiIqPN0OOjodDpERUVh3bp1ba5fsGAB0tPTsXnzZuTl5WHBggWYN28etm/fbm4THR2NjRs3Ii8vD99++y2EEIiPj4fRaDS3SUxMRFNTE/bt24fs7GwMHToUEydOREVFBQDAaDQiMTEROp0OWVlZSElJwdatW7Fw4cKOHpLdulGrxx9TTwMA5owJRXSwxyPbd8vjIA5d5DgdIiLqROIhABCpqakWy4YMGSLefvtti2XDhw8Xy5Ytu+t2cnNzBQBRUFAghBDi+vXrAoDYv3+/uY1WqxUAxN69e4UQQuzatUvI5XJRVlZmbvP5558LpVIpNBpNu+rXaDQCQLvb2xKTySR+u+m4CH59h4hfkykaDE2PdP/78q6K4Nd3iLj3//NI90tERN1fR36/rT5GJy4uDmlpaSgrK4MQAhkZGTh//jwSEhLabK/T6bBx40aEhIQgMDAQANCrVy8MHjwYmzZtgk6nQ1NTEz7++GP4+voiOjoaAHD48GFERETA39/fvK2EhATo9XpkZ2e3uS+9Xg+tVmvxslfbcsrw7ZmrcJTLsHpKFJSOnX/J6nZPhHjCUS5DaVU9SirrHum+iYjIflg96Kxduxbh4eEICAiAQqHAuHHjsH79esTFxVm0W79+PVxdXeHq6or09HTs2bMHCkXzLLwymQx79uzBiRMn4ObmBmdnZ3zwwQdIT09Hz549AQAVFRXw9bWc0M7DwwMKhcJ8eetOK1asgFqtNr9agpW9KdfU40/bzwAAXvnpAET0UT/yGlyUjhgW1BMAcJCXr4iIqJN0StA5cuQI0tLSkJ2djdWrV2Pu3LnYu3evRbvp06fjxIkTyMzMxIABAzBlyhQ0NDQAAIQQmDt3Lnx8fHDgwAEcO3YMkyZNwsSJE1FeXm7ehkwma7V/IUSbywFgyZIl0Gg05ldpaakVj7x7EELg9a2nUNPQhKgANX73dKhktbSM08nifDpERNRJHK25sfr6eixduhSpqalITEwEAERGRiInJwerVq3C2LFjzW1belUGDBiAUaNGwcPDA6mpqXjxxRexb98+7NixA9XV1XB3dwfQ3AO0Z88efPbZZ3jjjTfg5+eHo0ePWuy/uroaBoOhVU9PC6VSCaVSac1D7nY+P1aK/eevQ+Eox+opUXB0kG6Ggbj+Xvhw7wUcKrgBk0lALm87oBIRET0oq/7KGQwGGAwGyOWWm3VwcIDJZLrne4UQ0Oub51Spq2ses3HnduRyuXk7MTExOH36tEUPz+7du6FUKs3jeMhSSWUd/rLzLADgDwlh6O/jJmk9UYE94aJwQHWdAXkV9jteioiIOk+He3Rqa2tRUFBg/ruoqAg5OTnw9PREUFAQxowZg8WLF0OlUiE4OBiZmZnYtGkT1qxZAwAoLCzEli1bEB8fD29vb5SVleH999+HSqXChAkTADSHGA8PD8ycORN/+tOfoFKp8P/+3/9DUVGRuacoPj4e4eHhmDFjBlauXImqqiosWrQIs2fPNvcC0Y9MJoFFX+airtGIJ0I88avYEKlLgpODHCP79cK+/Gs4WHADQ/wf/VghIiKybR3u0fn+++8xbNgwDBs2DADw2muvYdiwYfjTn/4EAEhJScHjjz+O6dOnIzw8HO+99x7effddvPzyywAAZ2dnHDhwABMmTED//v0xZcoUuLi44NChQ/Dx8QEAeHl5IT09HbW1tXjmmWcwYsQIZGVlYfv27YiKigLQ3Eu0c+dOODs7IzY2FlOmTMGzzz6LVatWWeXE2Jr/PViEY5eq0EPhgFUvRHWZy0Q/jtPhxIFERGR9MiGEkLoIqWi1WqjVamg0GpvuBSq4VoMJa7PQ2GTCu7+IwPSRwVKXZJZfocW4Dw/A2UmO3OXxj/w2dyIi6n468vvNZ13ZuCajCQu/yEVjkwlPDvDCtCeCpC7JQpivG7xcFWgwmHCi5KbU5RARkY1h0LFxG767iNzLGrg5O+JvL0Te9dZ7qchkMvPlq4O8zZyIiKyMQceGnbmiwd//cwEA8PakIeitVklcUds4nw4REXUWBh0bpW8yYuEXuWgyCSQM8cWzQ/tIXdJdtQSdk5c10DYYJK6GiIhsCYOOjfr73gvIr6iBp4sC7/7isS53yep2fXqqEOLlAqNJ4GhhldTlEBGRDWHQsUE/lFTjH5kXAQB//UUEvFy7/mzQsf17AeA4HSIisi4GHRtT32jEoi9yYRLAL4b1wbiI3lKX1C6xoRyQTERE1segY2PeT89H4Q0dfN2VeCtpiNTltFtMaC/IZMCFa7W4qm2QuhwiIrIRDDo25NDFG/j00CUAwPvPR0Ldw0nagjqgZw8FHuvT/AgI9uoQEZG1MOjYiJoGAxZ/eRIA8OITQXg6zEfiijputPnyFR8HQURE1sGgYyPe3ZmHspv1CPRU4Y+Jg6Uu54HE3TZxoB0/mYSIiKyIQccG7Mu/ipTjpZDJgJUvRMFV2eGH0ncJI/p6QOEoR4W2ARev66Quh4iIbACDTjdXrWvE61tPAQB+FRuCUf16SVzRg3N2csCIYA8AzeONiIiIHhaDTje3PO0Mrtfo0c/bBYsTwqQu56GZHwdxgUGHiIgeHoNON7bzZDnScq/AQS7DmilD4ezkIHVJD61lnM7hwko0GU0SV0NERN0dg043db1Gj2Xbmi9ZzX06FEMDe0pbkJVE9FHD3dkRNQ1NOFWmkbocIiLq5hh0uiEhBJb8+xSq6wwY3Nsd854ZIHVJVuMglyEmtHmc0aGLvM2ciIgeDoNON7T1hzLszbsKJwcZ1kyJgsLRtj7GOI7TISIiK7GtX0g7cOVmPf6cdgYA8OrYgRjc213iiqyvZUBydnE16huNEldDRETdGYNONyKEwB++OokafROGBfXEnKf6SV1SpwjxckFvtTMajSZ8X1wldTlERNSNMeh0I5uPliCr4AacneRYPTkKjg62+fHJZLIfbzPnc6+IiOgh2OYvpQ26dEOHv+7MAwC8Pm4Q+nm7SlxR57r9cRBEREQPikGnGzCaBBZ9mYt6gxGj+nliZkxfqUvqdKNv3Xl15ooW1bpGiashIqLuikGnG/ifrEJ8X1wNV6UjVr4QBblcJnVJnc7H3RkDfV0hRPPkgURERA+CQaeLO3+1Bqu+PQ8AeHPiYAR69pC4okeH43SIiOhhMeh0YQajCa99kYNGowk/CfPGlBGBUpf0SMWGNgedQww6RET0gBh0urD/zijA6TIt1ConvPd8JGQy279kdbuR/TzhIJfhUmUdSqvqpC6HiIi6IQadLurUZQ3W7SsAALw9aQh83Z0lrujRc3N2Mj/D69BF9uoQEVHHMeh0QQ0GIxZ+mYMmk8CEx/zw8yh/qUuSTOytu6+yCjggmYiIOo5Bpwv6YO95nL9aCy9XBf7y7GN2d8nqdi0Dkg8V3IDJJCSuhoiIuhsGnS7m+0tV+GR/IQBgxXOR8HRRSFyRtIYFeUDl5IBKXSPOXa2RuhwiIupmGHS6kLrGJiz8MhdCAM8PD8DPwn2lLklyCkc5RvbzBMBZkomIqOMYdLqQ977JR3FlHXqrnfGnpHCpy+kyWm4zZ9AhIqKOYtDpIrIu3MCmw8UAgL+9EAm1ykniirqOlnE6R4uq0NhkkrgaIiLqThh0ugBtgwF/+CoXADBjVDCeHOAtcUVdyyA/N/RyUaCu0Yic0ptSl0NERN1Ih4PO/v37kZSUBH9/f8hkMmzbts1ifW1tLZKTkxEQEACVSoXBgwdjw4YNFm3mzJmD0NBQqFQqeHt7Y9KkScjPz2+1r507d2LkyJFQqVTw8vLCc889Z7G+pKQESUlJcHFxgZeXF+bPn4/Gxu73AMi3vz6LK5oGBPfqgTfGD5K6nC5HLpch5tZt5rx8RUREHdHhoKPT6RAVFYV169a1uX7BggVIT0/H5s2bkZeXhwULFmDevHnYvn27uU10dDQ2btyIvLw8fPvttxBCID4+Hkaj0dxm69atmDFjBl566SXk5ubi4MGDmDZtmnm90WhEYmIidDodsrKykJKSgq1bt2LhwoUdPSRJ7Tl7FV9lX4ZMBqyaHAUXpaPUJXVJcf05ToeIiB6AeAgARGpqqsWyIUOGiLffftti2fDhw8WyZcvuup3c3FwBQBQUFAghhDAYDKJPnz7in//8513fs2vXLiGXy0VZWZl52eeffy6USqXQaDTtql+j0QgA7W5vbZW1ehH9zh4R/PoO8e7Os5LU0F2UVOpE8Os7ROiSnaKmwSB1OUREJKGO/H5bfYxOXFwc0tLSUFZWBiEEMjIycP78eSQkJLTZXqfTYePGjQgJCUFgYPNDK3/44QeUlZVBLpdj2LBh6N27N8aPH48zZ86Y33f48GFERETA3//HWYMTEhKg1+uRnZ3d5r70ej20Wq3FS0pvbj+NG7V6DPBxxWs/GyhpLV1doGcPBHn2QJNJ4FgRZ0kmIqL2sXrQWbt2LcLDwxEQEACFQoFx48Zh/fr1iIuLs2i3fv16uLq6wtXVFenp6dizZw8UiubJ8QoLmyfMe+utt7Bs2TLs2LEDHh4eGDNmDKqqqgAAFRUV8PW1nGfGw8MDCoUCFRUVbda2YsUKqNVq86slWEnh69wr2HmyHA5yGdZMGQpnJwfJaukuWu6+yrrAoENERO3TKUHnyJEjSEtLQ3Z2NlavXo25c+di7969Fu2mT5+OEydOIDMzEwMGDMCUKVPQ0NAAADCZmm8h/uMf/4jnn3/ePKZHJpPhyy+/NG+jrUcjCCHu+siEJUuWQKPRmF+lpaXWOuwOuaZtwJvbTwMAkn/SH48FqCWpo7vhOB0iIuooq458ra+vx9KlS5GamorExEQAQGRkJHJycrBq1SqMHTvW3LalV2XAgAEYNWoUPDw8kJqaihdffBG9e/cGAISH/zhpnlKpRL9+/VBSUgIA8PPzw9GjRy32X11dDYPB0Kqn5/ZtKJVKax5yhwkh8Ma/T+FmnQFD/N2R/Ex/SevpTlruvDp3tQbXahrg42Z/T3QnIqKOsWqPjsFggMFggFxuuVkHBwdzL83dCCGg1+sBNN+VpVQqce7cOYttX7p0CcHBwQCAmJgYnD59GuXl5eY2u3fvhlKpRHR0tLUOyeq+/P4y9uVfg8JBjjVThsLJgVMZtZeniwJD/N0BAIcv8vIVERHdX4d7dGpra1FQUGD+u6ioCDk5OfD09ERQUBDGjBmDxYsXQ6VSITg4GJmZmdi0aRPWrFkDoHn8zZYtWxAfHw9vb2+UlZXh/fffh0qlwoQJEwAA7u7uePnll7F8+XIEBgYiODgYK1euBABMnjwZABAfH4/w8HDMmDEDK1euRFVVFRYtWoTZs2fD3d39oU9MZ7hcXYe3d5wFACyMH4gwPzeJK+p+4vp74cwVLbIu3MCkoX2kLoeIiLq6jt7SlZGRIQC0es2cOVMIIUR5ebmYNWuW8Pf3F87OziIsLEysXr1amEwmIYQQZWVlYvz48cLHx0c4OTmJgIAAMW3aNJGfn2+xn8bGRrFw4ULh4+Mj3NzcxNixY8Xp06ct2hQXF4vExEShUqmEp6enSE5OFg0NDe0+lkd5e7nRaBIvfnJYBL++Qzy3/qBoMpo6fZ+26Ltz10Tw6ztEzF/3mr9TRERkXzry+y0TQggJc5aktFot1Go1NBpNp/cCfXqwCG99fRYqJwd888qT6Ovl0qn7s1V1jU0Y+uc9aDSakLHoaYTwPBIR2Z2O/H5zgMgjUHi9Fu+lNz/iYsmEQQw5D6GHwhHDg3sCALJ49xUREd0Hg04nM5oEFn6ZiwaDCXH9vfBfI4OlLqnbM99mfoFBh4iI7o1Bp5N9sr8QJ0puwk3piPdfiIRc3vYcP9R+o28FncOFlTCa7PbKKxERtQODTifKr9Digz3nAQB/SgpHn54qiSuyDZF91HBTOkJTb8CZKxqpyyEioi6MQaeTNDaZ8NqWXDQaTRg72AcvRAdIXZLNcHSQY9StyQM5ToeIiO6FQaeTrNt3AWfLtfDo4YS/PvfYXR9LQQ8m9lbQOVTAiQOJiOjuGHQ6QW7pTfz3dxcBAH959jE+qqATxA1oHqdz7FIVGgxGiashIqKuikGnE2ScuwajSSApyh+Jkb2lLscmhXq7wtddicYmE7KLq6Uuh4iIuiirPtSTmr06diAiA9QYFughdSk2SyaTITbUC/8+UYaDBTcQe+tOLCIiotuxR6eTPDPIFx4uCqnLsGkt4eYgByQTEdFdMOhQt9USdE6WaaCpM0hcDRERdUUMOtRt+amdEertAiGAw4Xs1SEiotYYdKhbMz8OgreZExFRGxh0qFvjOB0iIroXBh3q1kb26wW5DCi8ocOVm/VSl0NERF0Mgw51a2qVEyIDegJgrw4REbXGoEPdXhwvXxER0V0w6FC3Zx6nc7ESQgiJqyEioq6EQYe6veHBPeHsJMf1Gj0uXKuVuhwiIupCGHSo21M6OuDxvp4AgKwLvHxFREQ/YtAhm8BxOkRE1BYGHbIJLeN0jhZVwWA0SVwNERF1FQw6ZBPCe7ujZw8n1OqbcPLyTanLISKiLoJBh2yCXC5DbGhzr07WBT4OgoiImjHokM0Y3b8XAODgRY7TISKiZgw6ZDNaBiSfKKmGTt8kcTVERNQVMOiQzQjy7IEADxUMRoFjl6qkLoeIiLoABh2yGTLZj+N0DnI+HSIiAoMO2ZjYAT8+DoKIiIhBh2zK6NDmAcl55VrcqNVLXA0REUmNQYdsiperEoP83AAAh9irQ0Rk9xh0yOa03H11iI+DICKyeww6ZHNaxukcuHADQgiJqyEiIikx6JDNeaKvJ5wcZCi7WY+SqjqpyyEiIgkx6JDNcVE6YligBwDgYAHH6RAR2bMOB539+/cjKSkJ/v7+kMlk2LZtm8X62tpaJCcnIyAgACqVCoMHD8aGDRss2syZMwehoaFQqVTw9vbGpEmTkJ+f3+b+9Ho9hg4dCplMhpycHIt1JSUlSEpKgouLC7y8vDB//nw0NjZ29JDIBrU8zfwgx+kQEdm1DgcdnU6HqKgorFu3rs31CxYsQHp6OjZv3oy8vDwsWLAA8+bNw/bt281toqOjsXHjRuTl5eHbb7+FEALx8fEwGo2ttveHP/wB/v7+rZYbjUYkJiZCp9MhKysLKSkp2Lp1KxYuXNjRQyIbFDeg+TbzQxdvwGTiOB0iIrslHgIAkZqaarFsyJAh4u2337ZYNnz4cLFs2bK7bic3N1cAEAUFBRbLd+3aJQYNGiTOnDkjAIgTJ05YrJPL5aKsrMy87PPPPxdKpVJoNJp21a/RaASAdren7qOxySjC3/xGBL++Q5y6fFPqcoiIyIo68vtt9TE6cXFxSEtLQ1lZGYQQyMjIwPnz55GQkNBme51Oh40bNyIkJASBgYHm5VevXsXs2bPxr3/9Cz169Gj1vsOHDyMiIsKitychIQF6vR7Z2dlt7kuv10Or1Vq8yDY5Ocgxqt+tp5nz8hURkd2yetBZu3YtwsPDERAQAIVCgXHjxmH9+vWIi4uzaLd+/Xq4urrC1dUV6enp2LNnDxQKBQBACIFZs2bh5ZdfxogRI9rcT0VFBXx9fS2WeXh4QKFQoKKios33rFixAmq12vy6PViR7WkZp5PFoENEZLc6JegcOXIEaWlpyM7OxurVqzF37lzs3bvXot306dNx4sQJZGZmYsCAAZgyZQoaGhoAAB999BG0Wi2WLFlyz33JZLJWy4QQbS4HgCVLlkCj0ZhfpaWlD3iU1B20BJ3jl6qgb2o9/ouIiGyfozU3Vl9fj6VLlyI1NRWJiYkAgMjISOTk5GDVqlUYO3asuW1Lr8qAAQMwatQoeHh4IDU1FS+++CL27duHI0eOQKlUWmx/xIgRmD59Oj777DP4+fnh6NGjFuurq6thMBha9fS0UCqVrbZJtmugryu8XJW4UavHD8U3EXPrOVhERGQ/rNqjYzAYYDAYIJdbbtbBwQEmk+me7xVCQK9vfgjj2rVrkZubi5ycHOTk5GDXrl0AgC1btuDdd98FAMTExOD06dMoLy83b2P37t1QKpWIjo625mFRNyWTyRDXn+N0iIjsWYd7dGpra1FQUGD+u6ioCDk5OfD09ERQUBDGjBmDxYsXQ6VSITg4GJmZmdi0aRPWrFkDACgsLMSWLVsQHx8Pb29vlJWV4f3334dKpcKECRMAAEFBQRb7dHV1BQCEhoYiICAAABAfH4/w8HDMmDEDK1euRFVVFRYtWoTZs2fD3d39wc4G2ZzR/b2wLecKsgpuYFFCmNTlEBHRI9bhoPP999/jJz/5ifnv1157DQAwc+ZMfPrpp0hJScGSJUswffp0VFVVITg4GO+++y5efvllAICzszMOHDiADz/8ENXV1fD19cVTTz2FQ4cOwcfHp911ODg4YOfOnZg7dy5iY2OhUqkwbdo0rFq1qqOHRDasZZzOycs3oW0wwN3ZSeKKiIjoUZIJYb9PPdRqtVCr1dBoNOwFsmHPrPoOhTd0+GRGNOKH+EldDhERPaSO/H7zWVdk80ZznA4Rkd1i0CGbF9fy3KuLfMAnEZG9YdAhmxfTzwsyGVBwrRYVmgapyyEiokeIQYdsnrqHEyL7qAHw8hURkb1h0CG7MNp8+YpBh4jInjDokF0wj9MpuAE7vtGQiMjuMOiQXYgO9oDSUY6rWj0uXq+VuhwiInpEGHTILjg7OWBEXw8AQNYFXr4iIrIXDDpkN2J5mzkRkd1h0CG70TJO58jFSjQZ7/2QWSIisg0MOmQ3hvir4e7siBp9E06WaaQuh4iIHgEGHbIbDnIZRoc29+oc4nw6RER2gUGH7ErsgOagk8WgQ0RkFxh0yK7EhjY/4POH4puobzRKXA0REXU2Bh2yKyFeLvBXO6PRaMLxS1VSl0NERJ2MQYfsikwm+/E2c16+IiKyeQw6ZHdagg7H6RAR2T4GHbI7o/s3j9M5W65Fla5R4mqIiKgzMeiQ3fFxc0aYrxuEAA5zlmQiIpvGoEN2iZeviIjsA4MO2aXYW5evDl1k0CEismUMOmSXRvbrBQe5DMWVdSitqpO6HCIi6iQMOmSXXJWOGBbYEwBvMycismUMOmS3RnOcDhGRzWPQIbsVdyvoHL5YCZNJSFwNERF1BgYdsltDA3uih8IBlbpG5FfUSF0OERF1AgYdslsKRzmeCPEEwHE6RES2ikGH7FrL5auDvM2ciMgmMeiQXWuZOPBoYRUam0wSV0NERNbGoEN2LczXDb1cFKg3GHGipFrqcoiIyMoYdMiuyeUy823mB/ncKyIim8OgQ3Yv7tbjIDggmYjI9jDokN0bHdrco5NTehM1DQaJqyEiImti0CG7F+jZA8G9esBoEjhWVCV1OUREZEUdDjr79+9HUlIS/P39IZPJsG3bNov1tbW1SE5ORkBAAFQqFQYPHowNGzZYtJkzZw5CQ0OhUqng7e2NSZMmIT8/37z+0qVL+PWvf42QkBCoVCqEhoZi+fLlaGxstNhOSUkJkpKS4OLiAi8vL8yfP79VG6L2iOXjIIiIbFKHg45Op0NUVBTWrVvX5voFCxYgPT0dmzdvRl5eHhYsWIB58+Zh+/bt5jbR0dHYuHEj8vLy8O2330IIgfj4eBiNRgBAfn4+TCYTPv74Y5w5cwYffPAB/vGPf2Dp0qXmbRiNRiQmJkKn0yErKwspKSnYunUrFi5c2NFDIvpxPh0GHSIimyITQjzwQ35kMhlSU1Px7LPPmpdFRERg6tSpePPNN83LoqOjMWHCBLzzzjttbufkyZOIiopCQUEBQkND22yzcuVKbNiwAYWFhQCAb775BhMnTkRpaSn8/f0BACkpKZg1axauXbsGd3f3+9av1WqhVquh0Wja1Z5sV7WuEcP/sgdCAMeW/hQ+7s5Sl0RERHfRkd9vq4/RiYuLQ1paGsrKyiCEQEZGBs6fP4+EhIQ22+t0OmzcuBEhISEIDAy863Y1Gg08PT3Nfx8+fBgRERHmkAMACQkJ0Ov1yM7Ott4BkV3wcFFgiH/z/1gO8TZzIiKbYfWgs3btWoSHhyMgIAAKhQLjxo3D+vXrERcXZ9Fu/fr1cHV1haurK9LT07Fnzx4oFIo2t3nx4kV89NFHePnll83LKioq4Ovra9HOw8MDCoUCFRUVbW5Hr9dDq9VavIhacJwOEZHt6ZSgc+TIEaSlpSE7OxurV6/G3LlzsXfvXot206dPx4kTJ5CZmYkBAwZgypQpaGhoaLW9K1euYNy4cZg8eTJ+85vfWKyTyWSt2gsh2lwOACtWrIBarTa/7tWDRPYnNvTHcToPcUWXiIi6EKsGnfr6eixduhRr1qxBUlISIiMjkZycjKlTp2LVqlUWbdVqNQYMGICnnnoKX331FfLz85GammrR5sqVK/jJT36CmJgYfPLJJxbr/Pz8WvXcVFdXw2AwtOrpabFkyRJoNBrzq7S01ApHTbbi8b6eUDjIUa5pQNENndTlEBGRFVg16BgMBhgMBsjllpt1cHCAyXTvByYKIaDX681/l5WV4emnn8bw4cOxcePGVtuMiYnB6dOnUV5ebl62e/duKJVKREdHt7kPpVIJd3d3ixdRC5XCAdHBHgB49xURka1w7OgbamtrUVBQYP67qKgIOTk58PT0RFBQEMaMGYPFixdDpVIhODgYmZmZ2LRpE9asWQMAKCwsxJYtWxAfHw9vb2+UlZXh/fffh0qlwoQJEwA09+Q8/fTTCAoKwqpVq3D9+nXz/vz8/AAA8fHxCA8Px4wZM7By5UpUVVVh0aJFmD17NgMMPbDY/r1wuLASWQU3MCOmr9TlEBHRwxIdlJGRIQC0es2cOVMIIUR5ebmYNWuW8Pf3F87OziIsLEysXr1amEwmIYQQZWVlYvz48cLHx0c4OTmJgIAAMW3aNJGfn2/ex8aNG9vcx53lFhcXi8TERKFSqYSnp6dITk4WDQ0N7T4WjUYjAAiNRtPR00A26ofiKhH8+g7x2PJ00WQ0SV0OERG1oSO/3w81j053x3l06E5NRhOGvbMHNQ1N2P77WEQF9pS6JCIiuoOk8+gQdWeODnKM6tf8NHPeZk5E1P0x6BDdoeVxEIcuMugQEXV3DDpEd2iZOPD4pWo0GIwSV0NERA+DQYfoDqHeLvB1V6KxyYTvL1VLXQ4RET0EBh2iO8hkMnOvzkFeviIi6tYYdIja0DJOhxMHEhF1bww6RG1o6dE5VabBzbpGiashIqIHxaBD1AZfd2f093GFEMDhi5VSl0NERA+IQYfoLuI4ToeIqNtj0CG6C/OA5AL26BARdVcMOkR3MbKfJ+QyoOiGDmU366Uuh4iIHgCDDtFduDs7mZ91xbuviIi6JwYdonvgbeZERN0bgw7RPYwO/XGcjhBC4mqIiKijGHSI7mF4cE84O8lxo1aP81drpS6HiIg6iEGH6B6Ujg54IqQXACCLl6+IiLodBh2i+4gNbQ46HKdDRNT9MOgQ3UfLfDpHCythMJokroaIiDqCQYfoPsJ7u8OjhxN0jUbklt6UuhwiIuoABh2i+5DLZea7rzhOh4ioe2HQIWqHWM6nQ0TULTHoELVDy8SBJ0puQqdvkrgaIiJqLwYdonYI6tUDgZ4qNJkEjhVVSV0OERG1E4MOUTvFcpwOEVG3w6BD1E4cp0NE1P0w6BC10+hbEwfmV9Tgeo1e4mqIiKg9GHSI2qmXqxKDe7sDAA5dZK8OEVF3wKBD1AFx/Zt7dQ4VVEpcCRERtQeDDlEHtIzTySq4ASGExNUQEdH9MOgQdcATIZ5wcpCh7GY9iivrpC6HiIjug0GHqAN6KBwxLMgDAHCQ43SIiLo8Bh2iDorjbeZERN0Ggw5RB8W2DEi+WAmTieN0iIi6MgYdog6KDOgJV6UjbtYZcLZcK3U5RER0Dww6RB3k5CDHqH6eAPg4CCKirq7DQWf//v1ISkqCv78/ZDIZtm3bZrG+trYWycnJCAgIgEqlwuDBg7FhwwaLNnPmzEFoaChUKhW8vb0xadIk5OfnW7Sprq7GjBkzoFaroVarMWPGDNy8edOiTUlJCZKSkuDi4gIvLy/Mnz8fjY2NHT0kog4bHcpxOkRE3UGHg45Op0NUVBTWrVvX5voFCxYgPT0dmzdvRl5eHhYsWIB58+Zh+/bt5jbR0dHYuHEj8vLy8O2330IIgfj4eBiNRnObadOmIScnB+np6UhPT0dOTg5mzJhhXm80GpGYmAidToesrCykpKRg69atWLhwYUcPiajD4gY0B51jRVVoMBjv05qIiCQjHgIAkZqaarFsyJAh4u2337ZYNnz4cLFs2bK7bic3N1cAEAUFBUIIIc6ePSsAiCNHjpjbHD58WAAQ+fn5Qgghdu3aJeRyuSgrKzO3+fzzz4VSqRQajaZd9Ws0GgGg3e2JWphMJjHiL3tE8Os7xMGC61KXQ0RkVzry+231MTpxcXFIS0tDWVkZhBDIyMjA+fPnkZCQ0GZ7nU6HjRs3IiQkBIGBgQCAw4cPQ61WY+TIkeZ2o0aNglqtxqFDh8xtIiIi4O/vb26TkJAAvV6P7OzsNvel1+uh1WotXkQPQiaT8TZzIqJuwOpBZ+3atQgPD0dAQAAUCgXGjRuH9evXIy4uzqLd+vXr4erqCldXV6Snp2PPnj1QKBQAgIqKCvj4+LTato+PDyoqKsxtfH19LdZ7eHhAoVCY29xpxYoV5jE/arXaHKyIHkTL08yz+NwrIqIuq1OCzpEjR5CWlobs7GysXr0ac+fOxd69ey3aTZ8+HSdOnEBmZiYGDBiAKVOmoKGhwbxeJpO12rYQwmJ5e9rcbsmSJdBoNOZXaWnpgx4mkfm5V6cu34Sm3iBxNURE1BZHa26svr4eS5cuRWpqKhITEwEAkZGRyMnJwapVqzB27Fhz25ZelQEDBmDUqFHw8PBAamoqXnzxRfj5+eHq1auttn/9+nVzL46fnx+OHj1qsb66uhoGg6FVT08LpVIJpVJprcMlO+ffU4V+3i4ovK7DkcJKJAzxk7okIiK6g1V7dAwGAwwGA+Ryy806ODjAZDLd871CCOj1egBATEwMNBoNjh07Zl5/9OhRaDQajB492tzm9OnTKC8vN7fZvXs3lEoloqOjrXVIRPcUy9vMiYi6tA736NTW1qKgoMD8d1FREXJycuDp6YmgoCCMGTMGixcvhkqlQnBwMDIzM7Fp0yasWbMGAFBYWIgtW7YgPj4e3t7eKCsrw/vvvw+VSoUJEyYAAAYPHoxx48Zh9uzZ+PjjjwEAv/3tbzFx4kSEhYUBAOLj4xEeHo4ZM2Zg5cqVqKqqwqJFizB79my4u7s/9Ikhao/Y/l7415FiBh0ioq6qo7d0ZWRkCACtXjNnzhRCCFFeXi5mzZol/P39hbOzswgLCxOrV68WJpNJCCFEWVmZGD9+vPDx8RFOTk4iICBATJs2zXzbeIvKykoxffp04ebmJtzc3MT06dNFdXW1RZvi4mKRmJgoVCqV8PT0FMnJyaKhoaHdx8Lby+lh3dQ1ipA3dojg13eIKzfrpC6HiMgudOT3WyaEsNunEmq1WqjVamg0GvYC0QObtC4LuZc1WDU5Ci9EB0hdDhGRzevI7zefdUX0kFruvjrEy1dERF0Ogw7RQ2qZODCr4AbsuIOUiKhLYtAhekjDgz2gdJTjWo0eBddqpS6HiIhuw6BD9JCcnRzweF9PAM29OkRE1HUw6BBZQaz5uVd8HAQRUVfCoENkBbH9m597daSwEk3Ge0+OSUREjw6DDpEVDPFXQ61yQq2+CbmXNVKXQ0REtzDoEFmBg1xmfpo5bzMnIuo6GHSIrCT2ttvMiYioa2DQIbKSlqDzQ0k16hqbJK6GiIgABh0iq+nbqwf69FTBYBQ4fqla6nKIiAgMOkRWI5PJzHdf8WnmRERdA4MOkRWZx+lcYNAhIuoKGHSIrGh0aHPQOVuuRZWuUeJqiIiIQYfIirzdlBjk5wYAOHSRvTpERFJj0CGyspZeHY7TISKSHoMOkZXFDWgZkMznXhERSY1Bh8jKngjpBUe5DCVVdSiprJO6HCIiu8agQ2RlrkpHDA3sCQDYl39V2mKIiOwcgw5RJxgX4QcA+GDvBVRoGiSuhojIfjHoEHWCmaP74rE+amjqDVj8VS5MJiF1SUREdolBh6gTODnI8cHUoXB2kuPAhRv415FiqUsiIrJLDDpEnaS/jyuWThgMAPjrrjwUXKuRuCIiIvvDoEPUiWaMCsZTA72hbzLh1S05aGwySV0SEZFdYdAh6kQymQwrX4hEzx5OOF2mxdr/XJC6JCIiu8KgQ9TJfN2d8ddfPAYAWP9dAbKLqySuiIjIfjDoED0CEx7rjeeG94FJAAu25KJW3yR1SUREdoFBh+gReevnQ9CnpwolVXX4y46zUpdDRGQXGHSIHhF3ZyesnhIFmQxIOV6KPWc5azIRUWdj0CF6hEb164XfPtkPAPDG1pO4XqOXuCIiItvGoEP0iL0WPxCD/NxQqWvEG1tPQgjOmkxE1FkYdIgeMaWjAz785VAoHOT4T/41pBwvlbokIiKbxaBDJIFBfu74w7gwAMA7O87i0g2dxBUREdkmBh0iifwqNgQx/XqhrtGIBV/koMnIWZOJiKyNQYdIInK5DKumRMHN2REnSm5i/XcXpS6JiMjmdDjo7N+/H0lJSfD394dMJsO2bdss1tfW1iI5ORkBAQFQqVQYPHgwNmzYYF5fVVWFefPmISwsDD169EBQUBDmz58PjUZjsZ3z589j0qRJ8PLygru7O2JjY5GRkWHRpqSkBElJSXBxcYGXlxfmz5+PxsbGjh4SkWT69FThnUkRAIC//+cCcktvSlsQEZGN6XDQ0el0iIqKwrp169pcv2DBAqSnp2Pz5s3Iy8vDggULMG/ePGzfvh0AcOXKFVy5cgWrVq3CqVOn8OmnnyI9PR2//vWvLbaTmJiIpqYm7Nu3D9nZ2Rg6dCgmTpyIiooKAIDRaERiYiJ0Oh2ysrKQkpKCrVu3YuHChR09JCJJTRrqj4mRvWE0CSz4Igf1jUapSyIish3iIQAQqampFsuGDBki3n77bYtlw4cPF8uWLbvrdr744guhUCiEwWAQQghx/fp1AUDs37/f3Ear1QoAYu/evUIIIXbt2iXkcrkoKyszt/n888+FUqkUGo2mXfVrNBoBoN3tiTpLtU4vRr67VwS/vkO8ue2U1OUQEXVpHfn9tvoYnbi4OKSlpaGsrAxCCGRkZOD8+fNISEi463s0Gg3c3d3h6OgIAOjVqxcGDx6MTZs2QafToampCR9//DF8fX0RHR0NADh8+DAiIiLg7+9v3k5CQgL0ej2ys7Pb3I9er4dWq7V4EXUFPXsosHJyJABg0+FifHfumsQVERHZBqsHnbVr1yI8PBwBAQFQKBQYN24c1q9fj7i4uDbbV1ZW4p133sGcOXPMy2QyGfbs2YMTJ07Azc0Nzs7O+OCDD5Ceno6ePXsCACoqKuDr62uxLQ8PDygUCvPlrTutWLECarXa/AoMDLTOQRNZwZMDvDFrdF8AwOKvTqJax/FmREQPq1OCzpEjR5CWlobs7GysXr0ac+fOxd69e1u11Wq1SExMRHh4OJYvX25eLoTA3Llz4ePjgwMHDuDYsWOYNGkSJk6ciPLycnM7mUzWaptCiDaXA8CSJUug0WjMr9JSTtRGXcsb4wehv48rrtfosTT1FGdNJiJ6SFYNOvX19Vi6dCnWrFmDpKQkREZGIjk5GVOnTsWqVass2tbU1GDcuHFwdXVFamoqnJyczOv27duHHTt2ICUlBbGxsRg+fDjWr18PlUqFzz77DADg5+fXquemuroaBoOhVU9PC6VSCXd3d4sXUVfi7OSAD6cOhaNchm9OV+DfP5RJXRIRUbdm1aBjMBhgMBggl1tu1sHBASbTj5OhabVaxMfHQ6FQIC0tDc7Ozhbt6+rqmou7Yztyudy8nZiYGJw+fdqih2f37t1QKpXmcTxE3VFEHzUW/GwgAGB52hmUVtVJXBERUffV4aBTW1uLnJwc5OTkAACKioqQk5ODkpISuLu7Y8yYMVi8eDG+++47FBUV4dNPP8WmTZvwi1/8AkBzT058fDx0Oh3+53/+B1qtFhUVFaioqIDR2HxbbUxMDDw8PDBz5kzk5ubi/PnzWLx4MYqKipCYmAgAiI+PR3h4OGbMmIETJ07gP//5DxYtWoTZs2ezp4a6vZfHhGJEsAdq9U1Y+EUujCZewiIieiAdvaUrIyNDAGj1mjlzphBCiPLycjFr1izh7+8vnJ2dRVhYmFi9erUwmUz3fD8AUVRUZN7P8ePHRXx8vPD09BRubm5i1KhRYteuXRa1FBcXi8TERKFSqYSnp6dITk4WDQ0N7T4W3l5OXVnxDZ0If/MbEfz6DrHhuwKpyyEi6jI68vstE8J+RztqtVqo1Wrz7e1EXc0Xx0vxh60n4eQgw/bfxyHcn99TIqKO/H7zWVdEXdjkEQGID/eFwSjw6pYTaDBw1mQioo5g0CHqwmQyGVY89xi8XBU4f7UWq749J3VJRETdCoMOURfXy1WJv73QPGvyP7OKcKjghsQVEbVPY5OJA+lJco5SF0BE9/fMIF9MGxmE/ztagoVf5iL91aegVjnd/41EEjCaBD7efxF/33sBADDA1xVhvu4Y5OeGMD83DPJzg7eb8q6TuxJZE4MOUTfxxwmDcajgBi5V1uFP20/j778cJnVJRK2U3azHa1tycLSoyrzsdJkWp8ssny3o0cPpVuhxR9itABTm6wYXJX+WyLp41xXvuqJu5ERJNV74x2EYTQJrXxyGn0f53/9NRI/I17lXsDT1FGoamtBD4YC3fj4ET/T1RH5FDc5V1ODcVS3yK2pw6YYOd7uiFeipatX7E+LlAkcHjrSgH3Xk95tBh0GHupk1e85j7X8uwN3ZEd8ueAq91SqpSyI7V9NgwPLtZ/DvE82PLBka2BMfTh2Kvl4ubbZvMBhRcK32VgDSmoPQtRp9m+0VDnKE+riaw09LAPJzd+blLzvFoNNODDrUHRmMJryw4RByL2sQ278X/vWrkZDL+Y89SSO7uAqvbslBaVU95DIg+Sf9Me+nA+D0AD0w1bpGc/g5d7UG+RU1OF9RA11j29MqqFVOCPO1DD8D/dzg7szxa7aOQaedGHSou7p4vRaJaw+gwWDCnyaG41dxIVKXRHamyWjC2n0FWLfvAkwCCPBQ4cOpQzGir6dV92MyCZTdrG/V+1N4Q3fXO7r69FRZhJ8wPzf083KFwpGXv2wFg047MehQd/avI8V4c9tpKBzl2DkvDgN83aQuiexEcaUOr6TkIKf0JgDguWF98NakIY+0J0XfZMTFazrzuJ9zt17lmoY22zs5yNDPy7VVAOrTU8XLX90Qg047MehQdyaEwEufHsd3564jvLc7tv0+lv8fK3UqIQS+yr6Mt9LOQNdohJuzI979xWNdalC8ps6Ac1cte3/OVdSgRt/UZns3pSMG3h5+fJvvBFP34OWvroxBp50YdKi7u6ZtQMKH+1FdZ8Dcp0Pxh3GDpC6JbNTNukb8MfU0dp4qBwA8EeKJD6YORZ+eXX8wvBACVzQNrcLPxeu1MBjb/gn0c3e26PkJ83NDfx9XKB0dHnH11BYGnXZi0CFbkH66HC9v/gFyGbBlTgwet/IYCaJDF2/gtS25qNA2wFEuw2vxAzHnqVA4dPNB8I1NJhTd0CG/QmsOP/kVNSi7Wd9mewe5DCFeLs0ByNfNPA9QgIeKNwQ8Ygw67cSgQ7Zi0Ze5+Cr7MgI8VPjmlSfhxrtOyAoam0xYvfscPjlQCCGAfl4u+PCXQxEZ0FPq0jpVTYMB52/d9dUSfs5V1EBTb2izvYvCAQN83e64/d0dni6KR1y5/WDQaScGHbIVNQ0GjP/7AVyursfk6ACsnBwldUnUzRVcq8ErKTk4c6V5RuMXnwjCmxMHo4fCPmcuFkLgqlbfqven4FotGo2mNt/j7aZs7u2RySADIJMBMsiAW50/ty+TyW5bf2t583/e/t7Wy9DyXvO6NrZnsT/L7bUMxL61qR/fe3sNt23vnvu7bXu3r4sO9kBiZO8HOe131ZHfb/v8xhLZGDdnJ6yZMhRTPzmML7Mv46eDfTEuwk/qsqgbEkJg89ESvLvzLBoMJnj0cML7z0cifoh9f59kMhn81M7wUzvj6TAf8/ImowmXKnXIK7+t9+eqFqVV9bheo8f1u0yCaE8amoxWDzodwaBDZCOeCPHEnKdC8Y/Mi1iaegrDg3vCx81Z6rKoG7lRq8frX53Ef/KvAQCeHOCF1ZOj4OPO79HdODrI0d/HDf193JB0W0eqTt+E81ebZ3tuvm4iIAQgALRcRxEWy5oXtlp35/pb/0fcZ3sQwnLdbX/fvj2Yl1lur7n1He/vyP5uWzc0sOeDn2ArYNAhsiGv/WwgMs9fR165Fq9/dRL/O+txzhFC7ZJx7hoWf5mLG7WNUDjI8fr4QXhpdF8Osn1ALkpHDAvykLoMAsBJN4hsiMJRjr//cigUjnJknLuO/ztWInVJ1MU1GIxYvv00Xtp4HDdqGxHm64btybH4dVwIQw7ZBAYdIhsz0NcNr9+aT+cvO/JQeL1W4oqoqzp7RYukj7Lw2eFiAMBLsX2xPTkWg3vz5gyyHQw6RDbopdF9Edu/F+oNRiz4IhdNd7krhOyTySTwzwOFePa/D+LCtVp4uynx2a+ewPKkIXB24oR4ZFsYdIhskFwuw6rJUXB3dkRu6U2syyiQuiTqIq5qG/D//e8x/GVnHhqNJowd7Iv0V57EmIHeUpdG1CkYdIhsVG+1Cu88GwEA+GhfgfkBjGS/0k9XIOHD/cgquAFnJzne/UUE/t//F41erkqpSyPqNAw6RDZs0tA++HmUP4wmgQVbclDX2PaDDcm26fRNeGPrSby8ORs36wyI6OOOHfOexPSRwbwrj2wegw6RjXtnUgR6q51RdEOHv+7Kk7ocesRyS28ice0BpBwvhUwG/O7pUPz7d7Ho7+MqdWlEjwSDDpGNU/dwwqpbj4TYfKQEGbcmgyPbZjQJrNt3Ac9vOIRLlXXwVzvj/34zCq+PGwSFI//pJ/vBbzuRHYjt74VfxYYAABZ/dRJVukaJK6LOdLm6Di9+cgSrdp9Hk0kgMbI3vnnlKcSE9pK6NKJHjkGHyE78YVwYBvi44katHkv+fRJ2/Dxfm7Y9pwzjPzyAY5eq4KJwwOrJUVj34jCoe/CJ9mSfGHSI7ISzkwM+mDoUTg4yfHvmKr7Kvix1SWRF2gYDXkk5gVdSclCjb8LwoJ745pWn8Hx0AAcck11j0CGyIxF91Fjws4EAgD9/fRalVXUSV0TWcPxSFcZ/eADbc67AQS7Dq2MH4Is5MQjq1UPq0ogkx6BDZGfmPBWKx/t6oFbfhNe+yIHRxEtY3ZXBaMLq3ecw9ePDKLtZjyDPHvhiTgxeHTsQjg78550IYNAhsjsOchnWTBkKV6Ujjl+qxsf7L0pdEj2Aohs6vPCPw/hoXwFMAnghOgC7XnkS0cF8YjbR7Rh0iOxQoGcPLE8KBwB8sOc8TpdpJK6I2ksIgS3HS5C49gByS2/C3dkR/z1tOFZNjoKr0lHq8oi6HAYdIjv1QnQAEob4wmBsnjW5wWCUuiS6j2pdI363+Qe8vvUU6hqNGNXPE+mvPoXEyN5Sl0bUZTHoENkpmUyGFc9FwstViQvXavG39HNSl0T3kHXhBsb9fT/Sz1TAyUGGJeMH4f9+Mwr+PVVSl0bUpXU46Ozfvx9JSUnw9/eHTCbDtm3bLNbX1tYiOTkZAQEBUKlUGDx4MDZs2GBeX1VVhXnz5iEsLAw9evRAUFAQ5s+fD42mddf5zp07MXLkSKhUKnh5eeG5556zWF9SUoKkpCS4uLjAy8sL8+fPR2MjJ0Ijai9PFwVWvhAJAPjfg0XIunBD4oroTvomI97deRb/9T9HcVWrRz9vF6TOjcWcMaGQy3nbONH9dPiCrk6nQ1RUFF566SU8//zzrdYvWLAAGRkZ2Lx5M/r27Yvdu3dj7ty58Pf3x6RJk3DlyhVcuXIFq1atQnh4OIqLi/Hyyy/jypUr+Oqrr8zb2bp1K2bPno2//vWveOaZZyCEwKlTp8zrjUYjEhMT4e3tjaysLFRWVmLmzJkQQuCjjz56wNNBZH9+MsgH/zUqCJuPlGDRl7n49tWnOLlcF3H+ag1eSclBXrkWAPBfo4LwxwnhUCkcJK6MqPuQiYeYHlUmkyE1NRXPPvuseVlERASmTp2KN99807wsOjoaEyZMwDvvvNPmdr788kv813/9F3Q6HRwdHdHU1IS+ffviz3/+M37961+3+Z5vvvkGEydORGlpKfz9/QEAKSkpmDVrFq5duwZ3d/f71q/VaqFWq6HRaNrVnshW1TU2IXFtFopu6JAU5Y+PXhwmdUl2TQiBTYeL8dddedA3meDposDfno/E2HBfqUsj6hI68vtt9TE6cXFxSEtLQ1lZGYQQyMjIwPnz55GQkHDX97QU6ujY3MH0ww8/oKysDHK5HMOGDUPv3r0xfvx4nDlzxvyew4cPIyIiwhxyACAhIQF6vR7Z2dlt7kev10Or1Vq8iAjooXDEB1OHwkEuw9e5V7A9p0zqkuzW9Ro9fvXpcSxPOwN9kwljBnoj/dUnGXKIHpDVg87atWsRHh6OgIAAKBQKjBs3DuvXr0dcXFyb7SsrK/HOO+9gzpw55mWFhYUAgLfeegvLli3Djh074OHhgTFjxqCqqgoAUFFRAV9fy//he3h4QKFQoKKios19rVixAmq12vwKDAy0xiET2YShgT0x75n+AIBl207jys16iSuyP/vyr2Lch/uRce46FI5yvJUUjk9fehw+bs5Sl0bUbXVK0Dly5AjS0tKQnZ2N1atXY+7cudi7d2+rtlqtFomJiQgPD8fy5cvNy00mEwDgj3/8I55//nlER0dj48aNkMlk+PLLL83t2np+ixDirs91WbJkCTQajflVWlr6sIdLZFOSf9IfUYE9UdPQhEVf5sLEWZMfifpGI97cdhq/+vR7VOoaMcjPDV8nx2FWbAifU0X0kKw6u1R9fT2WLl2K1NRUJCYmAgAiIyORk5ODVatWYezYsea2NTU1GDduHFxdXZGamgonpx8HP/bu3TwnRHh4uHmZUqlEv379UFJSAgDw8/PD0aNHLfZfXV0Ng8HQqqfn9m0olUrrHCyRDXJ0kOPDqUMx4e8HcOhiJf73YBF+82Q/qcuyaWeuaPBKSg4KrtUCAH4dF4LFCWFwduKAYyJrsGqPjsFggMFggFxuuVkHBwdzLw3Q3JMTHx8PhUKBtLQ0ODtbdstGR0dDqVTi3Lkf5/UwGAy4dOkSgoODAQAxMTE4ffo0ysvLzW12794NpVKJ6Ohoax4WkV0J8XLBsomDAQB/+/YczlXUSFyRbTKZBD7ZfxHP/vdBFFyrhY+bEv/69RN4c2I4Qw6RFXW4R6e2thYFBQXmv4uKipCTkwNPT08EBQVhzJgxWLx4MVQqFYKDg5GZmYlNmzZhzZo1AJp7cuLj41FXV4fNmzdbDAr29vaGg4MD3N3d8fLLL2P58uUIDAxEcHAwVq5cCQCYPHkyACA+Ph7h4eGYMWMGVq5ciaqqKixatAizZ8/mHVRED2naE0H4T9417Mu/hle35GDb70dD6cgfX2sp19Rj4Re5OHSxEgCQMMQXK56LhKeLQuLKiGyQ6KCMjAwBoNVr5syZQgghysvLxaxZs4S/v79wdnYWYWFhYvXq1cJkMt3z/QBEUVGReT+NjY1i4cKFwsfHR7i5uYmxY8eK06dPW9RSXFwsEhMThUqlEp6eniI5OVk0NDS0+1g0Go0AIDQaTUdPA5HNu6qtF8Pe3i2CX98hVuzKk7ocm7Hz5BUR+da3Ivj1HWLQsm/E50eLzf8+ElH7dOT3+6Hm0enuOI8O0b19e6YCc/6VDZkMSJk9CiP79ZK6pG6rVt+EP6edwZfZlwEAkQFqfDh1KPp5u0pcGVH3I+k8OkRkOxKG+GHKiAAIAbz2RS5qGgxSl9Qt/VBSjcS1B/Bl9mXIZM13t2393WiGHKJHgEGHiO7pT0lDEOipQtnNeryVdlbqcrqVJqMJf997AZP/cRjFlXXo01OFlNmjsCghDE4O/OeX6FHg/9KI6J5clY74YMpQyGXA1h8u45tT5fd/k53TNxlxtLASv/zkCD7Yex5Gk8DPo/yx65UnefmP6BGz6jw6RGSbRvT1xMtjQrH+u4tYmnoK0cEe8HHnbL0tGgxG/FBSjaOFVThaVIkTJTehb2qeUsNN6Yh3no3As8P6SFwlkX1i0CGidnl17EBknr+OM1e0WPzVSXz60uN2O2uvTt+E7OJqHC2qxLGiKuSWatBoNFm08XJVILa/FxbFhyHQs4dElRIRgw4RtYvCsXnW5IkfZSHz/HVsPlKMGTF9pS7rkahpMOD7S9U4UlSJo4VVOF2mQdMdj8fwdVdiZEgvjOzniZEhvRDq7WK3QZCoK2HQIaJ2G+DrhjfGD8Kfvz6Ld3flYXR/L4Ta4J1DmjoDjl2qwtHCShwtqsKZKxrc+divPj1VGBniaQ42wb16MNgQdUEMOkTUITNj+uI/edeQVXADC7bkYOvvRnf7O4iqdI04VlSJI4VVOFpUhfwKLe6cYSy4Vw+MDPHEEyG9MDLEk5ejiLoJBh0i6hC5XIaVkyOR8MF+nLyswUf7CvDazwZKXVaHXKtpMA8cPlZUhfNXa1u16eftgpEhvTCqnyeeCPFEb7VKgkqJ6GEx6BBRh/VWq/DuLx7DvM9P4L8zCvB0mDeGB3lIXdZdlWvqzcHmaGEVCm/oWrUZ6OtqHmPzRIgnfNx4VxmRLWDQIaIHkhTlj715V7E95wpe25KDnfOfhIuya/yTUlpVh6NFP46xKamqs1gvkwGD/NwxMsQTo/p54vG+nujlqpSoWiLqTF3jXyUi6pbenhSBY0VVuFRZh7/szMOK5x575DUIIVBcWWfurTlaVIWym/UWbeQyIKKPunnwcEgvPN7XE+oeTo+8ViJ69Bh0iOiBqVVOWD05CtP+eRSfHyvB2ME++Olg307dpxACF6/rbgs2lbiq1Vu0cZTL8FiA2nwpakSwB9ycGWyI7BGDDhE9lNH9vfCbuBD8M6sIr289ifRXn4KXFS8DmUwC56/V4FhRlbnH5katZbBROMgRFfhjsBke5NFlLqMRkbT4LwERPbRFCWE4cOEGzl2twZJ/n8InM6IfeE4Zo0kgr1xrHmNz/FIVqussn5qudJRjWFBPi2Dj7ORgjUMhIhvDoENED83ZyQEfTB2KSf+dhT1nr+KL70sx9fGgdr23yWjCmSta86Wo45eqoG1osmijcnLAiL4etybo64XIADWUjgw2RHR/DDpEZBXh/u5YGB+G977Jx5+/PotR/XohuJdLq3YGowknL2vMwSa7uBq1estg46p0vBVsmntsHuuj7vaTEhKRNBh0iMhqZj/ZD/vyr+FYURVe+yIXW347CkYhkFuqMd/qnV1cjXqD0eJ97s6OeOLWHVEj+3kivLc7HBlsiMgKZELcOdG5/dBqtVCr1dBoNHB3d5e6HCKbUFpVh/F/P4BafRMG+rqiuLIO+ibLJ3t7uijwRN/miflG9vPEID93OMj5nCgiap+O/H6zR4eIrCrQswf+/PMhWPhlrvnRCl6uSozs54lRt8bY9Pd2hZzBhogeAQYdIrK654b3gYNchrpGI0b280Q/Lxc+2ZuIJMGgQ0RWJ5PJ8OywPlKXQUQEjvYjIiIim8WgQ0RERDaLQYeIiIhsFoMOERER2SwGHSIiIrJZDDpERERksxh0iIiIyGYx6BAREZHNYtAhIiIim8WgQ0RERDaLQYeIiIhsFoMOERER2SwGHSIiIrJZdv30ciEEAECr1UpcCREREbVXy+92y+/4vdh10KmpqQEABAYGSlwJERERdVRNTQ3UavU928hEe+KQjTKZTLhy5Qrc3Nwgk8msum2tVovAwECUlpbC3d3dqtvuDnj89n38AM+BvR8/wHNg78cPdN45EEKgpqYG/v7+kMvvPQrHrnt05HI5AgICOnUf7u7udvsFB3j89n78AM+BvR8/wHNg78cPdM45uF9PTgsORiYiIiKbxaBDRERENotBp5MolUosX74cSqVS6lIkweO37+MHeA7s/fgBngN7P36ga5wDux6MTERERLaNPTpERERksxh0iIiIyGYx6BAREZHNYtAhIiIim8Wg8xDeeustyGQyi5efn595vRACb731Fvz9/aFSqfD000/jzJkzElb8cPbv34+kpCT4+/tDJpNh27ZtFuvbc7x6vR7z5s2Dl5cXXFxc8POf/xyXL19+hEfxcO53DmbNmtXqOzFq1CiLNt35HKxYsQKPP/443Nzc4OPjg2effRbnzp2zaGPL34P2HL8tfwc2bNiAyMhI8+RvMTEx+Oabb8zrbfmzb3G/c2DLn39bVqxYAZlMhldffdW8rKt9Dxh0HtKQIUNQXl5ufp06dcq87m9/+xvWrFmDdevW4fjx4/Dz88PPfvYz8zO2uhudToeoqCisW7euzfXtOd5XX30VqampSElJQVZWFmprazFx4kQYjcZHdRgP5X7nAADGjRtn8Z3YtWuXxfrufA4yMzPx+9//HkeOHMGePXvQ1NSE+Ph46HQ6cxtb/h605/gB2/0OBAQE4L333sP333+P77//Hs888wwmTZpk/hGz5c++xf3OAWC7n/+djh8/jk8++QSRkZEWy7vc90DQA1u+fLmIiopqc53JZBJ+fn7ivffeMy9raGgQarVa/OMf/3hEFXYeACI1NdX8d3uO9+bNm8LJyUmkpKSY25SVlQm5XC7S09MfWe3Wcuc5EEKImTNnikmTJt31PbZ2Dq5duyYAiMzMTCGE/X0P7jx+IezvO+Dh4SH++c9/2t1nf7uWcyCE/Xz+NTU1YsCAAWLPnj1izJgx4pVXXhFCdM1/A9ij85AuXLgAf39/hISE4Je//CUKCwsBAEVFRaioqEB8fLy5rVKpxJgxY3Do0CGpyu007Tne7OxsGAwGizb+/v6IiIiwqXPy3XffwcfHBwMHDsTs2bNx7do18zpbOwcajQYA4OnpCcD+vgd3Hn8Le/gOGI1GpKSkQKfTISYmxu4+e6D1OWhhD5//73//eyQmJmLs2LEWy7vi98CuH+r5sEaOHIlNmzZh4MCBuHr1Kv7yl79g9OjROHPmDCoqKgAAvr6+Fu/x9fVFcXGxFOV2qvYcb0VFBRQKBTw8PFq1aXl/dzd+/HhMnjwZwcHBKCoqwptvvolnnnkG2dnZUCqVNnUOhBB47bXXEBcXh4iICAD29T1o6/gB2/8OnDp1CjExMWhoaICrqytSU1MRHh5u/oGyh8/+bucAsP3PHwBSUlLwww8/4Pjx463WdcV/Axh0HsL48ePN//2xxx5DTEwMQkND8dlnn5kHn8lkMov3CCFaLbMlD3K8tnROpk6dav7vERERGDFiBIKDg7Fz504899xzd31fdzwHycnJOHnyJLKyslqts4fvwd2O39a/A2FhYcjJycHNmzexdetWzJw5E5mZmeb19vDZ3+0chIeH2/znX1paildeeQW7d++Gs7PzXdt1pe8BL11ZkYuLCx577DFcuHDBfPfVnen02rVrrZKuLWjP8fr5+aGxsRHV1dV3bWNrevfujeDgYFy4cAGA7ZyDefPmIS0tDRkZGQgICDAvt5fvwd2Ovy229h1QKBTo378/RowYgRUrViAqKgp///vf7eazB+5+Dtpia59/dnY2rl27hujoaDg6OsLR0RGZmZlYu3YtHB0dzcfQlb4HDDpWpNfrkZeXh969eyMkJAR+fn7Ys2ePeX1jYyMyMzMxevRoCavsHO053ujoaDg5OVm0KS8vx+nTp23ynABAZWUlSktL0bt3bwDd/xwIIZCcnIx///vf2LdvH0JCQizW2/r34H7H3xZb+w7cSQgBvV5v85/9vbScg7bY2uf/05/+FKdOnUJOTo75NWLECEyfPh05OTno169f1/seWH14sx1ZuHCh+O6770RhYaE4cuSImDhxonBzcxOXLl0SQgjx3nvvCbVaLf7973+LU6dOiRdffFH07t1baLVaiSt/MDU1NeLEiRPixIkTAoBYs2aNOHHihCguLhZCtO94X375ZREQECD27t0rfvjhB/HMM8+IqKgo0dTUJNVhdci9zkFNTY1YuHChOHTokCgqKhIZGRkiJiZG9OnTx2bOwe9+9zuhVqvFd999J8rLy82vuro6cxtb/h7c7/ht/TuwZMkSsX//flFUVCROnjwpli5dKuRyudi9e7cQwrY/+xb3Oge2/vnfze13XQnR9b4HDDoPYerUqaJ3797CyclJ+Pv7i+eee06cOXPGvN5kMonly5cLPz8/oVQqxVNPPSVOnTolYcUPJyMjQwBo9Zo5c6YQon3HW19fL5KTk4Wnp6dQqVRi4sSJoqSkRIKjeTD3Ogd1dXUiPj5eeHt7CycnJxEUFCRmzpzZ6vi68zlo69gBiI0bN5rb2PL34H7Hb+vfgV/96lciODhYKBQK4e3tLX7605+aQ44Qtv3Zt7jXObD1z/9u7gw6Xe17IBNCCOv3ExERERFJj2N0iIiIyGYx6BAREZHNYtAhIiIim8WgQ0RERDaLQYeIiIhsFoMOERER2SwGHSIiIrJZDDpERERksxh0iIiIyGYx6BAREZHNYtAhIiIim8WgQ0RERDbr/wd9FW6Pww4M4wAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "plt.plot(list(results.keys()), list(results.values()))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f785b9f4",
   "metadata": {
    "papermill": {
     "duration": 0.008783,
     "end_time": "2023-07-02T20:38:07.547535",
     "exception": false,
     "start_time": "2023-07-02T20:38:07.538752",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Step 3: Find the best parameter value\n",
    "\n",
    "Given the results, which value for `n_estimators` seems best for the random forest model?  Use your answer to set the value of `n_estimators_best`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b298eb2d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:38:07.567420Z",
     "iopub.status.busy": "2023-07-02T20:38:07.566705Z",
     "iopub.status.idle": "2023-07-02T20:38:07.574882Z",
     "shell.execute_reply": "2023-07-02T20:38:07.573842Z"
    },
    "papermill": {
     "duration": 0.020616,
     "end_time": "2023-07-02T20:38:07.577017",
     "exception": false,
     "start_time": "2023-07-02T20:38:07.556401",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"3_BestEst\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_estimators_best = 200\n",
    "\n",
    "# Check your answer\n",
    "step_3.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "a880013f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-02T20:38:07.596781Z",
     "iopub.status.busy": "2023-07-02T20:38:07.596399Z",
     "iopub.status.idle": "2023-07-02T20:38:07.600534Z",
     "shell.execute_reply": "2023-07-02T20:38:07.599769Z"
    },
    "papermill": {
     "duration": 0.016279,
     "end_time": "2023-07-02T20:38:07.602474",
     "exception": false,
     "start_time": "2023-07-02T20:38:07.586195",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#step_3.hint()\n",
    "#step_3.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f937b157",
   "metadata": {
    "papermill": {
     "duration": 0.008939,
     "end_time": "2023-07-02T20:38:07.620459",
     "exception": false,
     "start_time": "2023-07-02T20:38:07.611520",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In this exercise, you have explored one method for choosing appropriate parameters in a machine learning model.  \n",
    "\n",
    "If you'd like to learn more about [hyperparameter optimization](https://en.wikipedia.org/wiki/Hyperparameter_optimization), you're encouraged to start with **grid search**, which is a straightforward method for determining the best _combination_ of parameters for a machine learning model.  Thankfully, scikit-learn also contains a built-in function [`GridSearchCV()`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html) that can make your grid search code very efficient!\n",
    "\n",
    "# Keep going\n",
    "\n",
    "Continue to learn about **[gradient boosting](https://www.kaggle.com/alexisbcook/xgboost)**, a powerful technique that achieves state-of-the-art results on a variety of datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5935427",
   "metadata": {
    "papermill": {
     "duration": 0.008792,
     "end_time": "2023-07-02T20:38:07.638322",
     "exception": false,
     "start_time": "2023-07-02T20:38:07.629530",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "*Have questions or comments? Visit the [course discussion forum](https://www.kaggle.com/learn/intermediate-machine-learning/discussion) to chat with other learners.*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 83.297082,
   "end_time": "2023-07-02T20:38:08.569758",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2023-07-02T20:36:45.272676",
   "version": "2.4.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}