credit_card_fraud_detection_with_snap_ml.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"gpuType":"T4","authorship_tag":"ABX9TyMePrYxAKg4099Ae9ggdwWj"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","source":["## Credit Card Fraud Detection using Scikit-Learn and Snap ML"],"metadata":{"id":"tFvbam8OzcoT"}},{"cell_type":"markdown","source":["Here we use two popular classification models to recognize fraudulent credit card transactions. These models are: Decision Tree and Support Vector Machine."],"metadata":{"id":"jmBv6vHNzfnf"}},{"cell_type":"markdown","source":["### Installing and Importing necessary packages"],"metadata":{"id":"FGcrgz0VKk3D"}},{"cell_type":"code","execution_count":null,"metadata":{"id":"5GNAw99qDfma"},"outputs":[],"source":["!pip install pandas\n","!pip install matplotlib\n","!pip install numpy\n","!pip install scikit-learn\n","!pip install requests\n","!pip install opendatasets\n","!pip install snapml"]},{"cell_type":"code","source":["import pandas as pd\n","import numpy as np\n","import requests\n","import sklearn.tree as tree\n","import opendatasets as od\n","from __future__ import print_function\n","import matplotlib.pyplot as plt\n","%matplotlib inline\n","from sklearn.model_selection import train_test_split\n","from sklearn.preprocessing import normalize, StandardScaler\n","from sklearn.utils.class_weight import compute_sample_weight\n","from sklearn.metrics import roc_auc_score\n","import time\n","import warnings\n","warnings.filterwarnings('ignore')"],"metadata":{"id":"jHzVxvmXKiq5","executionInfo":{"status":"ok","timestamp":1704530321418,"user_tz":-210,"elapsed":1623,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}}},"execution_count":2,"outputs":[]},{"cell_type":"markdown","source":["### Downloading dataset from IBM Object Storage\n","Beware, It needs Kaggle API Credentials."],"metadata":{"id":"cMZ5KleEK-cD"}},{"cell_type":"markdown","source":["The dataset includes information about transactions made by credit cards in September 2013 by European cardholders. We will use the trained model to assess if a credit card transaction is legitimate or not.\n","\n","The job is to build a model that predicts if a credit card transaction is fraudulent or not. We can model the problem as a binary classification problem. A transaction belongs to the positive class (1) if it is a fraud, otherwise it belongs to the negative class (0).\n","\n","The dataset contains transactions that occured over a certain period of time. The majority of the transactions are normally legitimate and only a small fraction are non-legitimate. Thus, the dataset is highly unbalanced : only 492 transactions out of 284,807 are fraudulent (the positive class - the frauds - accounts for 0.172% of all transactions)."],"metadata":{"id":"03v7I26PLAOP"}},{"cell_type":"code","source":["od.download(\"https://www.kaggle.com/mlg-ulb/creditcardfraud\")"],"metadata":{"id":"84wSS0bN1CZT"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["### Exploring the dataset"],"metadata":{"id":"ETt5Yu8CMrhX"}},{"cell_type":"markdown","source":["Reading the downloaded data set and exploring it."],"metadata":{"id":"dLML6WL02OGM"}},{"cell_type":"code","source":["# read the input data\n","raw_data = pd.read_csv('creditcardfraud/creditcard.csv')\n","print(\"There are \" + str(len(raw_data)) + \" observations in the credit card fraud dataset.\")\n","print(\"There are \" + str(len(raw_data.columns)) + \" variables in the dataset.\")\n","\n","# display the first rows in the dataset\n","raw_data.head()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":290},"id":"PIypqgop2Ed9","executionInfo":{"status":"ok","timestamp":1704530360187,"user_tz":-210,"elapsed":2210,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"3288103c-ffcc-405a-a29e-b2c5398ede0e"},"execution_count":4,"outputs":[{"output_type":"stream","name":"stdout","text":["There are 284807 observations in the credit card fraud dataset.\n","There are 31 variables in the dataset.\n"]},{"output_type":"execute_result","data":{"text/plain":["   Time        V1        V2        V3        V4        V5        V6        V7  \\\n","0   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n","1   0.0  1.191857  0.266151  0.166480  0.448154  0.060018 -0.082361 -0.078803   \n","2   1.0 -1.358354 -1.340163  1.773209  0.379780 -0.503198  1.800499  0.791461   \n","3   1.0 -0.966272 -0.185226  1.792993 -0.863291 -0.010309  1.247203  0.237609   \n","4   2.0 -1.158233  0.877737  1.548718  0.403034 -0.407193  0.095921  0.592941   \n","\n","         V8        V9  ...       V21       V22       V23       V24       V25  \\\n","0  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n","1  0.085102 -0.255425  ... -0.225775 -0.638672  0.101288 -0.339846  0.167170   \n","2  0.247676 -1.514654  ...  0.247998  0.771679  0.909412 -0.689281 -0.327642   \n","3  0.377436 -1.387024  ... -0.108300  0.005274 -0.190321 -1.175575  0.647376   \n","4 -0.270533  0.817739  ... -0.009431  0.798278 -0.137458  0.141267 -0.206010   \n","\n","        V26       V27       V28  Amount  Class  \n","0 -0.189115  0.133558 -0.021053  149.62      0  \n","1  0.125895 -0.008983  0.014724    2.69      0  \n","2 -0.139097 -0.055353 -0.059752  378.66      0  \n","3 -0.221929  0.062723  0.061458  123.50      0  \n","4  0.502292  0.219422  0.215153   69.99      0  \n","\n","[5 rows x 31 columns]"],"text/html":["\n","  <div id=\"df-1cde0b27-2f9e-485a-9a4c-6c69f9b76809\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Time</th>\n","      <th>V1</th>\n","      <th>V2</th>\n","      <th>V3</th>\n","      <th>V4</th>\n","      <th>V5</th>\n","      <th>V6</th>\n","      <th>V7</th>\n","      <th>V8</th>\n","      <th>V9</th>\n","      <th>...</th>\n","      <th>V21</th>\n","      <th>V22</th>\n","      <th>V23</th>\n","      <th>V24</th>\n","      <th>V25</th>\n","      <th>V26</th>\n","      <th>V27</th>\n","      <th>V28</th>\n","      <th>Amount</th>\n","      <th>Class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0.0</td>\n","      <td>-1.359807</td>\n","      <td>-0.072781</td>\n","      <td>2.536347</td>\n","      <td>1.378155</td>\n","      <td>-0.338321</td>\n","      <td>0.462388</td>\n","      <td>0.239599</td>\n","      <td>0.098698</td>\n","      <td>0.363787</td>\n","      <td>...</td>\n","      <td>-0.018307</td>\n","      <td>0.277838</td>\n","      <td>-0.110474</td>\n","      <td>0.066928</td>\n","      <td>0.128539</td>\n","      <td>-0.189115</td>\n","      <td>0.133558</td>\n","      <td>-0.021053</td>\n","      <td>149.62</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>0.0</td>\n","      <td>1.191857</td>\n","      <td>0.266151</td>\n","      <td>0.166480</td>\n","      <td>0.448154</td>\n","      <td>0.060018</td>\n","      <td>-0.082361</td>\n","      <td>-0.078803</td>\n","      <td>0.085102</td>\n","      <td>-0.255425</td>\n","      <td>...</td>\n","      <td>-0.225775</td>\n","      <td>-0.638672</td>\n","      <td>0.101288</td>\n","      <td>-0.339846</td>\n","      <td>0.167170</td>\n","      <td>0.125895</td>\n","      <td>-0.008983</td>\n","      <td>0.014724</td>\n","      <td>2.69</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>1.0</td>\n","      <td>-1.358354</td>\n","      <td>-1.340163</td>\n","      <td>1.773209</td>\n","      <td>0.379780</td>\n","      <td>-0.503198</td>\n","      <td>1.800499</td>\n","      <td>0.791461</td>\n","      <td>0.247676</td>\n","      <td>-1.514654</td>\n","      <td>...</td>\n","      <td>0.247998</td>\n","      <td>0.771679</td>\n","      <td>0.909412</td>\n","      <td>-0.689281</td>\n","      <td>-0.327642</td>\n","      <td>-0.139097</td>\n","      <td>-0.055353</td>\n","      <td>-0.059752</td>\n","      <td>378.66</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>1.0</td>\n","      <td>-0.966272</td>\n","      <td>-0.185226</td>\n","      <td>1.792993</td>\n","      <td>-0.863291</td>\n","      <td>-0.010309</td>\n","      <td>1.247203</td>\n","      <td>0.237609</td>\n","      <td>0.377436</td>\n","      <td>-1.387024</td>\n","      <td>...</td>\n","      <td>-0.108300</td>\n","      <td>0.005274</td>\n","      <td>-0.190321</td>\n","      <td>-1.175575</td>\n","      <td>0.647376</td>\n","      <td>-0.221929</td>\n","      <td>0.062723</td>\n","      <td>0.061458</td>\n","      <td>123.50</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>2.0</td>\n","      <td>-1.158233</td>\n","      <td>0.877737</td>\n","      <td>1.548718</td>\n","      <td>0.403034</td>\n","      <td>-0.407193</td>\n","      <td>0.095921</td>\n","      <td>0.592941</td>\n","      <td>-0.270533</td>\n","      <td>0.817739</td>\n","      <td>...</td>\n","      <td>-0.009431</td>\n","      <td>0.798278</td>\n","      <td>-0.137458</td>\n","      <td>0.141267</td>\n","      <td>-0.206010</td>\n","      <td>0.502292</td>\n","      <td>0.219422</td>\n","      <td>0.215153</td>\n","      <td>69.99</td>\n","      <td>0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>5 rows × 31 columns</p>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1cde0b27-2f9e-485a-9a4c-6c69f9b76809')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-1cde0b27-2f9e-485a-9a4c-6c69f9b76809 button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-1cde0b27-2f9e-485a-9a4c-6c69f9b76809');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-9290c4b7-e75e-40d7-9d97-d978536f8638\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-9290c4b7-e75e-40d7-9d97-d978536f8638')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-9290c4b7-e75e-40d7-9d97-d978536f8638 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","    </div>\n","  </div>\n"]},"metadata":{},"execution_count":4}]},{"cell_type":"markdown","source":["To simulate a real world transaction dataset, we inflate the dataset 10 times."],"metadata":{"id":"9fUrY3JM2Ssz"}},{"cell_type":"code","source":["n_replicas = 10\n","\n","# inflate the original dataset\n","big_raw_data = pd.DataFrame(np.repeat(raw_data.values, n_replicas, axis=0), columns=raw_data.columns)\n","\n","print(\"There are \" + str(len(big_raw_data)) + \" observations in the inflated credit card fraud dataset.\")\n","print(\"There are \" + str(len(big_raw_data.columns)) + \" variables in the dataset.\")\n","\n","# display first rows in the new dataset\n","big_raw_data.head()\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":290},"id":"y80Xr4YbMtMU","executionInfo":{"status":"ok","timestamp":1704530360187,"user_tz":-210,"elapsed":17,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"ccd12328-3c73-4121-f497-cd01bf506a6a"},"execution_count":5,"outputs":[{"output_type":"stream","name":"stdout","text":["There are 2848070 observations in the inflated credit card fraud dataset.\n","There are 31 variables in the dataset.\n"]},{"output_type":"execute_result","data":{"text/plain":["   Time        V1        V2        V3        V4        V5        V6        V7  \\\n","0   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n","1   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n","2   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n","3   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n","4   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n","\n","         V8        V9  ...       V21       V22       V23       V24       V25  \\\n","0  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n","1  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n","2  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n","3  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n","4  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n","\n","        V26       V27       V28  Amount  Class  \n","0 -0.189115  0.133558 -0.021053  149.62    0.0  \n","1 -0.189115  0.133558 -0.021053  149.62    0.0  \n","2 -0.189115  0.133558 -0.021053  149.62    0.0  \n","3 -0.189115  0.133558 -0.021053  149.62    0.0  \n","4 -0.189115  0.133558 -0.021053  149.62    0.0  \n","\n","[5 rows x 31 columns]"],"text/html":["\n","  <div id=\"df-7d7c0074-dce4-432a-9b90-f0b856934353\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Time</th>\n","      <th>V1</th>\n","      <th>V2</th>\n","      <th>V3</th>\n","      <th>V4</th>\n","      <th>V5</th>\n","      <th>V6</th>\n","      <th>V7</th>\n","      <th>V8</th>\n","      <th>V9</th>\n","      <th>...</th>\n","      <th>V21</th>\n","      <th>V22</th>\n","      <th>V23</th>\n","      <th>V24</th>\n","      <th>V25</th>\n","      <th>V26</th>\n","      <th>V27</th>\n","      <th>V28</th>\n","      <th>Amount</th>\n","      <th>Class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0.0</td>\n","      <td>-1.359807</td>\n","      <td>-0.072781</td>\n","      <td>2.536347</td>\n","      <td>1.378155</td>\n","      <td>-0.338321</td>\n","      <td>0.462388</td>\n","      <td>0.239599</td>\n","      <td>0.098698</td>\n","      <td>0.363787</td>\n","      <td>...</td>\n","      <td>-0.018307</td>\n","      <td>0.277838</td>\n","      <td>-0.110474</td>\n","      <td>0.066928</td>\n","      <td>0.128539</td>\n","      <td>-0.189115</td>\n","      <td>0.133558</td>\n","      <td>-0.021053</td>\n","      <td>149.62</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>0.0</td>\n","      <td>-1.359807</td>\n","      <td>-0.072781</td>\n","      <td>2.536347</td>\n","      <td>1.378155</td>\n","      <td>-0.338321</td>\n","      <td>0.462388</td>\n","      <td>0.239599</td>\n","      <td>0.098698</td>\n","      <td>0.363787</td>\n","      <td>...</td>\n","      <td>-0.018307</td>\n","      <td>0.277838</td>\n","      <td>-0.110474</td>\n","      <td>0.066928</td>\n","      <td>0.128539</td>\n","      <td>-0.189115</td>\n","      <td>0.133558</td>\n","      <td>-0.021053</td>\n","      <td>149.62</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>0.0</td>\n","      <td>-1.359807</td>\n","      <td>-0.072781</td>\n","      <td>2.536347</td>\n","      <td>1.378155</td>\n","      <td>-0.338321</td>\n","      <td>0.462388</td>\n","      <td>0.239599</td>\n","      <td>0.098698</td>\n","      <td>0.363787</td>\n","      <td>...</td>\n","      <td>-0.018307</td>\n","      <td>0.277838</td>\n","      <td>-0.110474</td>\n","      <td>0.066928</td>\n","      <td>0.128539</td>\n","      <td>-0.189115</td>\n","      <td>0.133558</td>\n","      <td>-0.021053</td>\n","      <td>149.62</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>0.0</td>\n","      <td>-1.359807</td>\n","      <td>-0.072781</td>\n","      <td>2.536347</td>\n","      <td>1.378155</td>\n","      <td>-0.338321</td>\n","      <td>0.462388</td>\n","      <td>0.239599</td>\n","      <td>0.098698</td>\n","      <td>0.363787</td>\n","      <td>...</td>\n","      <td>-0.018307</td>\n","      <td>0.277838</td>\n","      <td>-0.110474</td>\n","      <td>0.066928</td>\n","      <td>0.128539</td>\n","      <td>-0.189115</td>\n","      <td>0.133558</td>\n","      <td>-0.021053</td>\n","      <td>149.62</td>\n","      <td>0.0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>0.0</td>\n","      <td>-1.359807</td>\n","      <td>-0.072781</td>\n","      <td>2.536347</td>\n","      <td>1.378155</td>\n","      <td>-0.338321</td>\n","      <td>0.462388</td>\n","      <td>0.239599</td>\n","      <td>0.098698</td>\n","      <td>0.363787</td>\n","      <td>...</td>\n","      <td>-0.018307</td>\n","      <td>0.277838</td>\n","      <td>-0.110474</td>\n","      <td>0.066928</td>\n","      <td>0.128539</td>\n","      <td>-0.189115</td>\n","      <td>0.133558</td>\n","      <td>-0.021053</td>\n","      <td>149.62</td>\n","      <td>0.0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>5 rows × 31 columns</p>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7d7c0074-dce4-432a-9b90-f0b856934353')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-7d7c0074-dce4-432a-9b90-f0b856934353 button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-7d7c0074-dce4-432a-9b90-f0b856934353');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-e843a587-0de4-4e9b-9ccd-b522a36f5acc\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e843a587-0de4-4e9b-9ccd-b522a36f5acc')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-e843a587-0de4-4e9b-9ccd-b522a36f5acc button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","    </div>\n","  </div>\n"]},"metadata":{},"execution_count":5}]},{"cell_type":"markdown","source":["Each row in the dataset represents a credit card transaction. As shown above, each row has 31 variables. One variable (the last variable in the table above) is called Class and represents the target variable.\n","\n","Note that for confidentiality reasons, the original names of most features are anonymized V1, V2 .. V28. The values of these features are the result of a PCA transformation and are numerical. The feature 'Class' is the target variable and it takes two values: 1 in case of fraud and 0 otherwise."],"metadata":{"id":"fOqIhm3D2wbq"}},{"cell_type":"markdown","source":["Statistics :"],"metadata":{"id":"ljjTxQuo2_iI"}},{"cell_type":"code","source":["# get the set of distinct classes\n","labels = big_raw_data.Class.unique()\n","\n","# get the count of each class\n","sizes = big_raw_data.Class.value_counts().values\n","\n","# plot the class value counts\n","fig, ax = plt.subplots()\n","ax.pie(sizes, labels=labels, autopct='%1.3f%%')\n","ax.set_title('Target Variable Value Counts')\n","plt.show()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":428},"id":"WMzOSciy3Dhx","executionInfo":{"status":"ok","timestamp":1704530360643,"user_tz":-210,"elapsed":471,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"f5a297a3-0705-416c-9079-873745c4b434"},"execution_count":6,"outputs":[{"output_type":"display_data","data":{"text/plain":["<Figure size 640x480 with 1 Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAYUAAAGbCAYAAAAr/4yjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9JklEQVR4nO3dd3gU5doG8Ht2syW9d0IKBBJCrwpIl4AoKsfQFRAR2xFR8ajHo+JnRwSPSBEFFQmgYkFUWgRRQKQKJBAChEAC6cmmt935/giZQ0gCSdhkdmfv33XlCjs7O/vMbNh75n3fmRFEURRBREQEQCV3AUREZDkYCkREJGEoEBGRhKFAREQShgIREUkYCkREJGEoEBGRhKFAREQShgIREUkYCiS7V199FYIgNOu1ISEhuPPOO284365duyAIAnbt2tWs92kNISEhmD59utxlkI1jKDSRIAiN+rG0L5+9e/fi1VdfRX5+/nXnq6yshJeXFwYOHNjgPKIoIigoCD179jRzldbh/fffhyAI2LFjR4PzrFy5EoIgYNOmTa1Y2c3JyMjAs88+i4iICDg4OMDR0RG9evXC66+/fsO/m9YSGxuLxYsXy12GotnJXYC1WbNmTa3HX3zxBbZv315nemRkZGuWdUN79+7F/PnzMX36dLi5uTU4n0ajQUxMDFasWIGUlBQEBwfXmWf37t1ITU3F3LlzzVLbSy+9hOeff94sy2oNEydOxLx58xAbG4sRI0bUO09sbCw8PT0xevToVq6ueQ4cOIA77rgDRUVFmDp1Knr16gUAOHjwIN5++23s3r0b27Ztk7nK6u164sQJPPXUU3KXolgMhSaaOnVqrcd//vkntm/fXmd6c4iiiLKyMtjb29/0sm7GlClTsHz5cqxbt67eL+vY2FioVCpMnDjxpt6nuLgYjo6OsLOzg52d9fwpBgQEYOjQofj222+xbNky6HS6Ws+npaVh9+7dePjhh6HRaGSqsvHy8/Nx7733Qq1W48iRI4iIiKj1/BtvvIGVK1fKVB21NjYftYDVq1dj2LBh8PHxgU6nQ6dOnbBs2bI689W0h2/duhW9e/eGvb09VqxYAQBISUnB2LFj4ejoCB8fH8ydOxdbt26tt2lq//79GDVqFFxdXeHg4IDBgwdjz5490vOvvvoq5s2bBwAIDQ2VmrjOnz9fb/0DBgxASEgIYmNj6zxXWVmJb775BkOHDkVAQACOHTuG6dOnIywsDHq9Hn5+fnjwwQeRk5NT63U1/QYJCQmYPHky3N3dpSaq+voUGrsNa2zbtg3du3eHXq9Hp06d8O233zY4b1O2XUOmTp0Kg8GAn376qc5z69evh8lkwpQpUwAA7733Hvr37w9PT0/Y29ujV69e+Oabb274Hg31tXz22Wf1fn6//PILbrvtNjg6OsLZ2RljxoxBfHz8Dd9nxYoVSEtLw/vvv18nEADA19cXL730Uq1pS5cuRVRUFHQ6HQICAvD444/XaWJqqI9kyJAhGDJkiPS4pr/nq6++whtvvIE2bdpAr9dj+PDhOHPmTK3X/fTTT0hJSZH+hkNCQqTnP/zwQ0RFRcHBwQHu7u7o3bt3vX/DdH3Ws3tmRZYtW4aoqCiMHTsWdnZ2+PHHH/HYY4/BZDLh8ccfrzVvYmIiJk2ahNmzZ2PWrFno2LEjiouLMWzYMFy+fBlz5syBn58fYmNjsXPnzjrv9euvv2L06NHo1asXXnnlFahUKukL9ffff0ffvn0xbtw4nD59GuvWrcOiRYvg5eUFAPD29q63fkEQMHnyZLz55puIj49HVFSU9NyWLVuQm5srfeFt374d586dw4wZM+Dn54f4+Hh8/PHHiI+Px59//lnnSy0mJgbh4eF48803cb2rtjdlGyYlJWHChAl45JFHMG3aNKxevRoxMTHYsmULbr/99gbfozHbriHjxo3Do48+itjYWIwbN67Wc7GxsQgODsaAAQMAAB988AHGjh2LKVOmoKKiAuvXr0dMTAw2b96MMWPGNPgeTbFmzRpMmzYN0dHReOedd1BSUoJly5Zh4MCBOHLkSK0vz2tt2rQJ9vb2uO+++xr1Xq+++irmz5+PESNG4NFHH0ViYiKWLVuGAwcOYM+ePc0+Onr77behUqnw7LPPwmAw4N1338WUKVOwf/9+AMC///1vGAwGpKamYtGiRQAAJycnANV9OE8++STuu+8+zJkzB2VlZTh27Bj279+PyZMnN6semyXSTXn88cfFazdjSUlJnfmio6PFsLCwWtOCg4NFAOKWLVtqTV+4cKEIQPz++++laaWlpWJERIQIQNy5c6coiqJoMpnE8PBwMTo6WjSZTLXePzQ0VLz99tulaQsWLBABiMnJyY1ar/j4eBGA+MILL9SaPnHiRFGv14sGg6HBdV23bp0IQNy9e7c07ZVXXhEBiJMmTaozf81zV2vqNty4caM0zWAwiP7+/mKPHj2kaTt37mz2tmtITExMrW0hiqJ46tSpOtvt2nWpqKgQO3fuLA4bNqzOukybNk16XN92EUVRXL16da3PsrCwUHRzcxNnzZpVa7709HTR1dW1zvRrubu7i926dbvuPDUyMzNFrVYrjhw5UjQajdL0JUuWiADEVatWNbg+NQYPHiwOHjxYelzz2URGRorl5eXS9A8++EAEIB4/flyaNmbMGDE4OLjOMu+++24xKiqqUetA18fmoxZwdZ+AwWBAdnY2Bg8ejHPnzsFgMNSaNzQ0FNHR0bWmbdmyBYGBgRg7dqw0Ta/XY9asWbXmO3r0KJKSkjB58mTk5OQgOzsb2dnZKC4uxvDhw7F7926YTKZmrUOnTp3Qo0cPrF+/XppWXFyMTZs24c4774SLi0uddS0rK0N2djZuueUWAMDhw4frLPeRRx5p1Ps3ZRsGBATg3nvvlR67uLjggQcewJEjR5Cenl7v8s2x7aZOnYqysrJaTVU1zRU1R1LXrkteXh4MBgNuu+22erdPc2zfvh35+fmYNGmStB7Z2dlQq9Xo169fvUeYVysoKICzs3Oj3mvHjh2oqKjAU089BZXqf18fs2bNgouLS73NaY01Y8YMaLVa6fFtt90GADh37twNX+vm5obU1FQcOHCg2e9P1dh81AL27NmDV155Bfv27UNJSUmt5wwGA1xdXaXHoaGhdV6fkpKCdu3a1Wl6ad++fa3HSUlJAIBp06Y1WIvBYIC7u3uT1wGo/mJ79tlnsXfvXvTv3x/ff/89SkpKan3h5ebmYv78+Vi/fj0yMzPrvPe16lvf+jRlG7Zv377OturQoQMA4Pz58/Dz86uzfHNsu9GjR8PDwwOxsbFS2/m6devQrVu3Wk1umzdvxuuvv46jR4+ivLxcmt7cczOuVbMuw4YNq/f5mgBviIuLCwoLCxv1XikpKQCAjh071pqu1WoRFhYmPd8cbdu2rfW4Ztvn5eXd8LX/+te/sGPHDvTt2xft27fHyJEjMXnyZKkJjxqPoWBmZ8+exfDhwxEREYH3338fQUFB0Gq1+Pnnn7Fo0aI6e583M9KoZlkLFixA9+7d652nps21OSZNmoTnnnsOsbGx6N+/P2JjY+Hu7o477rhDmmf8+PHYu3cv5s2bh+7du8PJyQkmkwmjRo2qd0+7Mevb1G3YHObYdhqNBuPHj8fKlSuRkZGBCxcuICkpCe+++640z++//46xY8di0KBBWLp0Kfz9/aHRaLB69eobdoI2FBpGo7HedVmzZk29AXijkV0RERE4evQoKioqau2p36zr1a9Wq+tMr28agOv2PdWIjIxEYmIiNm/ejC1btmDjxo1YunQpXn75ZcyfP79phds4hoKZ/fjjjygvL8emTZtq7fnc6BD+asHBwUhISIAoirX+Y109EgMA2rVrB6B6T6+h8fI1mrNXWjP08uuvv8Z//vMfbN++HdOnT5e+OPLy8hAXF4f58+fj5Zdfll5Xs+faXE3dhmfOnKmzrU6fPg0ADXawNmXbXU/N8N0NGzYgOTkZgiBg0qRJ0vMbN26EXq/H1q1baw1dXb169Q2XXbOnnJ+fX+vckmv3xmvWxcfHp1nrctddd2Hfvn3YuHFjrdrrU3PeSmJiIsLCwqTpFRUVSE5OrvX+7u7u9Z70lpKSUuu1TXG9v2NHR0dMmDABEyZMQEVFBcaNG4c33ngDL7zwAvR6fbPezxaxT8HMavZ2rt67MRgMjfoSqBEdHY20tLRaZ8OWlZXVGSveq1cvtGvXDu+99x6KiorqLCcrK0v6t6OjIwA0+czUKVOmIDMzE7Nnz0ZlZWWtpqP61hXATZ9x2tRteOnSJXz33XfS44KCAnzxxRfo3r17vXvOQNO23fXUDN/98ssvsWHDBgwePBht2rSptS6CINTauz9//jy+//77Gy675st+9+7d0rTi4mJ8/vnnteaLjo6Gi4sL3nzzTVRWVjZ5XR555BH4+/vjmWeekcL0apmZmXj99dcBACNGjIBWq8V///vfWp/Pp59+CoPBUGs0Vbt27fDnn3+ioqJCmrZ582ZcvHjxuvVcj6OjY73NktcOgdZqtejUqRNEUax3m1DDeKRgZiNHjoRWq8Vdd92F2bNno6ioCCtXroSPjw8uX77cqGXMnj0bS5YswaRJkzBnzhz4+/tj7dq10t5Ozd6SSqXCJ598gtGjRyMqKgozZsxAYGAg0tLSsHPnTri4uODHH38EAOkM1X//+9+YOHEiNBoN7rrrLiksGvKPf/wDjz32GH744QcEBQVh0KBB0nMuLi4YNGgQ3n33XVRWViIwMBDbtm1DcnJyk7fb1Zq6DTt06ICZM2fiwIED8PX1xapVq5CRkXHdIG7Ktrueq4fvAsBrr71W6/kxY8bg/fffx6hRozB58mRkZmbio48+Qvv27XHs2LEbboe2bdti5syZmDdvHtRqNVatWgVvb29cuHBBms/FxQXLli3D/fffj549e2LixInSPD/99BMGDBiAJUuWNPg+7u7u+O6773DHHXege/futc5oPnz4MNatW4dbb70VQPUw5hdeeAHz58/HqFGjMHbsWCQmJmLp0qXo06dPrZM4H3roIXzzzTcYNWoUxo8fj7Nnz+LLL7+Uwq45evXqhQ0bNuDpp59Gnz594OTkhLvuugsjR46En58fBgwYAF9fX5w8eRJLlizBmDFjGt2JTlfIN/BJGeobkrpp0yaxa9euol6vF0NCQsR33nlHXLVqVZ0hocHBweKYMWPqXe65c+fEMWPGiPb29qK3t7f4zDPPiBs3bhQBiH/++WeteY8cOSKOGzdO9PT0FHU6nRgcHCyOHz9ejIuLqzXf//3f/4mBgYGiSqVq0vDUmJgYEYD43HPP1XkuNTVVvPfee0U3NzfR1dVVjImJES9duiQCEF955RVpvprhlVlZWXWWUd/Qy6Zuw61bt4pdu3YVdTqdGBERIX799de1lnftkNSmbrvrqRm+q9PpxLy8vDrPf/rpp2J4eLhU2+rVq+td5/qGcB46dEjs16+fqNVqxbZt24rvv/9+nSGpV69jdHS06OrqKur1erFdu3bi9OnTxYMHDzZqPS5duiTOnTtX7NChg6jX60UHBwexV69e4htvvFFr2K0oVg9BjYiIEDUajejr6ys++uij9a77woULxcDAQFGn04kDBgwQDx482OCQ1Gs/s+TkZBGAuHr1amlaUVGROHnyZNHNzU0EIA1PXbFihTho0CDpc2zXrp04b968OnXTjQmi2IheHLIIixcvxty5c5GamorAwEC5yyEiBWIoWKjS0tI65wD06NEDRqOx3nZfIiJzYJ+ChRo3bhzatm2L7t27w2Aw4Msvv8SpU6ewdu1auUsjIgVjKFio6OhofPLJJ1i7di2MRiM6deqE9evXY8KECXKXRkQKxuYjIiKS8DwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIomd3AUQtYSKKhOyisqRVViOzIIy6d/F5VWoMokwmURUmUQYr/oBALVKgJ1aBY1agJ3qym+1AJ2dGl5OOvi66ODrooePiw5ejjqoVILMa0pkXgwFsiomk4iU3BJcNpQiq/DKl770u0x6bCithCi2bC12KkEKCh8XffVvZ/3/Hjvr4eeqh4ejtmULITIjQRRb+r8OUfNUGU1IyizCiTQD4i8V4ESaAScvF6C4wih3aU3i6ahF50BXdAl0RZc21b8D3OzlLouoXgwFsgjlVUYkphfiRFoBTlwyID7NgFPphSivMsldWovwcqoOiq6BrujSxg1dAl3h56qXuywihgLJIyWnGLuTsnE8NR8n0gqQlFmISqNt/yl6O+uqjyau/PQJ8YCrg0bussjGMBSoVRhNIg6l5CHuVAbiTmbiTGaR3CVZPDuVgN4h7hgR6YuRnfzQ1tNB7pLIBjAUqMUUllXit9NZiDuZiV2JmcgrqZS7JKvWwdcJIyJ9MaKTL3oEuUEQOPKJzI+hQGaVklOMHSczEXcyAwfO59p8k1BL8XbWYXiED0ZE+mJguBf0GrXcJZFCMBToph1KycW2+AzEnWKzkBzsNWoMaO+FkZ18MSzSB15OOrlLIivGUKBmyS4qxzeHUvHVgYs4l10sdzl0hUoABoZ7Y3LfIIyI9IWdmhctoKZhKFCjmUwifjudhfUHLuDXU5lsGrJw3s46xPRqg0l92yLIg53U1DgMBbqhzMIyxO6/gK8OXMQlQ5nc5VATCQIwsL0XJvVti5GdePRA18dQoAb9fTEfq/ck4+fj6agwKvMkMlsT4KrH/beGYHLftjwHgurFUKBaKo0m/Hz8MlbvOY+jF/PlLodaiL1GjXt7BuLBASFo7+MsdzlkQRgKBKD6MhNf/nkBH+8+i4yCcrnLoVZS07T06OB26N/eS+5yyAIwFGyc0SRi4+FUfLAjCWn5pXKXQzIa2N4Lz4+OQOdAV7lLIRkxFGzYlhOX8d620zy3gCSCAIzp4o950R0R7OkodzkkA4aCDdp7NhvvbklknwE1SKMWMLFPWzw5PBzezjwZzpYwFGzI8VQD3t16Cr8nZctdClkJB60aDw0MxcOD28FJx3ty2QKGgg04m1WEhdsS8cuJ9Ba/Gxkpk6ejFo8PbY+ptwRDa8fzHJSMoaBglw2lWLw9Cd8cTpXuQUx0M4I87PH07R1wd7dA3p9aoRgKCmQyifj0j2Qs3J6IskqedEbmF+nvgtfujkKfEA+5SyEzYygozJnMIsz75m8cuZAvdymkcCoBmDEgFPOiO/LS3QrCUFAIo0nEx7vPYfGO04q9rzFZpjAvRyyI6YZewe5yl0JmwFBQgNMZhZj3zTH8zSGmJBOVAMwcGIpnRvKowdoxFKxYldGEFbvP4YO4JFTw6IAsQDtvR7wX0w092vKowVoxFKzUqfQCzPv6GI6nGeQuhagWtUrArNvCMPf2cOjseNRgbRgKVqbKaMLSXWex5NczvJw1WbRwHycsHN8NXdu4yV0KNQFDwYokXCrAvG/+RvylArlLIWoUtUrA7EFheGpEB570ZiUYClYidv8FvLLpBG+BSVYpws8Z/53UAx18ee8GS8dQsHCVRhNe3RSPtfsvyF0K0U1x0tlh8YTuGNHJV+5S6DoYChYsp6gcj649jL+Sc+UuhcgsVALwbHRHPDakvdylUAMYChYq/pIBD39xiDe+IUW6u3sA3vlHV57TYIEYChZo87FLmPf1MZRWGuUuhajFdGvjio8f6A1fF73cpdBVGAoWRBRFvLctER/tPCt3KUStwtdFh4/v741uQW5yl0JXMBQsRFF5FZ5afwQ7TmbKXQpRq9LZqfDOP7rinh6BcpdCYChYhPPZxZj1xUEk8V7JZMMeGdwOz0V35H0aZMZQkNnu01n457ojMJRWyl0KkeyGR/jgg0k9eOtPGTEUZLThwAW8+N0J3hWN6CrhPk5YNb0Pgjwc5C7FJjEUZPLZnmTM35zAeyYT1cPPRY/YWf0Q5u0kdyk2h6Egg2W7zuKdLafkLoPIonk767D2oX68NEYrYyi0sve3n8Z/45LkLoPIKng4arFmZl9EBbjKXYrNYCi0ojd/PomPd5+Tuwwiq+Jqr8GamX15Ce5WwlBoJa9uisdne8/LXQaRVXLW2+GzGX15H+hWwAuct4I3fkpgIBDdhMKyKkxf9RfvQ94KGAotbMHWU1j5e7LcZRBZvcLyKjyw6i8k8CZTLYqh0IIW7zjN6xgRmZGhtBL3f7ofSRmFcpeiWAyFFvLRzjNYvIOjjIjMLae4AlM+2Y/k7GK5S1EkhkIL+HzveSzYmih3GUSKlVlYjikr/8TF3BK5S1EchoKZ/XY6C69tTpC7DCLFu2Qow7RVf6GgjNcNMyeGghmdzSrCE7GHeS0jolZyLrsY/4w9AhP/z5kNQ8FMDCWVeOjzgygsq5K7FCKb8tvpLLzNy8aYDUPBDKqMJjwee5gdX0Qy+Xj3OXx7OFXuMhSBoWAGr21OwB9nsuUug8imvfDtcRzlyW03jaFwk9buT8EX+1LkLoPI5pVXmTB7zUFkFJTJXYpVYyjchH1nc/Dqpni5yyCiKzIKyvHwmkMoqzTKXYrVYig004WcEjy29hAqjRz1QGRJ/r6Yjxe/PS53GVaLodAMReVVmPn5AeSVcHw0kSX69kgaVvzGS8w0B0OhiUwmEU+uO4KkzCK5SyGi63hnyynsTMyUuwyrw1BoogXbEvHrKf6hEVk6kwg8ue4IzmZxB64pGApN8FdyLg9JiaxIYVkVZrPjuUkYCo1UUlGFed/8DZ5NT2RdzmQWYdH203KXYTUYCo309i+nkJLDKzISWaNP/kjGkQt5cpdhFRgKjbD3TDbW/MkT1IisldEk4tmv/2YzUiMwFG6gqLwK8745BpHNRkRW7WxWMZuRGoGhcANv/JSAtPxSucsgIjNgM9KNMRSuY1diJtb9dVHuMojITNiMdGMMhQYYSivx/EaeKk+kNGxGuj6GQgNe+zEB6bzaIpEisRmpYQyFeuxIyMBG3rCDSLHYjNQwhsI18ksq8MJ3bDYiUjo2I9WPoXCN+T8mIKuwXO4yiKgVsBmpLobCVf6+mI/vj6bJXQYRtRKjScTzG4/DxOvXSBgKV3n7l1M8SY3IxiRmFLIP8SoMhSt2JmZi37kcucsgIhks3pGE8ip2OgMMBQDVN85555dTcpdBRDJJyy/Fmn28vhnAUAAAfH80DafSC+Uug4hktHTXWRSW8Ra7Nh8K5VVGLNzGYWlEti63uAIrd5+TuwzZ2XworNmXwgveERGA6iGq2UW2PSTdpkOhoKwSH+08I3cZRGQhSiqM+DAuSe4yZGXTobBs11nklbANkYj+J/avC7iYa7t3WbTZUMgoKMPqPclyl0FEFqbSKGLhtkS5y5CNzYbCou2nUVZpkrsMIrJAP/x9CQmXCuQuQxY2GQpnMgvx9SGewUhE9RNFYMFW2zx3ySZDYdH2JBh5rRMiuo6diVnYb4NXObC5ULiYW4It8elyl0FEVuDDX21vdKLNhcLne8/zKIGIGuWPM9k4nWFbVzuwqVAoLq/ChoMX5S6DiKzI6j3n5S6hVdlUKHx18CIKy6rkLoOIrMj3R9KQX1IhdxmtxmZCwWQS8dne83KXQURWprTSiPUHbKeFwWZCIe5UJlJybPcsRSJqvjX7UmymL9JmQuGLfeflLoGIrFRafim22sioRZsIhQs5JfjjTLbcZRCRFYvdf0HuElqFTYTCugMXeO9lIrope85m44INNEErPhQqjSZ8fZCXtCCimyOKwIaDyj9aUHwobE/IsPmbZhCReXx9MBVVRmVfSFPxoWAr7YBE1PIyC8vx66lMuctoUYoOhQs5Jdhzlh3MRGQ+Sj9nQdGh8POJy+xgJiKz+u10FjILyuQuo8UoOhS2J2TIXQIRKYzRJGLHSeU2ISk2FLKLynHkQp7cZRCRAsWdVO4Op2JDIe5kBmzkrHQiamV7zmajtMIodxktQrGhwKYjImopZZUmxV4lQZGhUFphVOwHRkSWYYdCdzwVGQq7k7JQVqnsE0yISF6/JmZCVODwRkWGApuOiKilZRWW4+jFfLnLMDvFhYLJJCr+jEMisgxxChyaqrhQOHQhD7nFtnPrPCKSzw4FDk1VXCiw6YiIWsup9EKk5inrctoMBSKim6C0UUiKCoUzmYVIzi6WuwwisiFxCuvDVFQoKPl6JERkmfafy0VReZXcZZiNokLhcAqvdUREravCaMJeBZ0sq6hQOJ5mkLsEIrJBSvruUUwoZBWW47JBudc4JyLLxVCwQMfT8uUugYhs1AmGguX5+6JyPhQisi7ZRRW4bCiVuwyzUEwoKOnwjYisz/FUZXwHKSYUjinkAyEi63TiUoHcJZiFIkLhUn4psovK5S6DiGyYUvoVFBEKPEogIrkppQlbEaHAkUdEJLeswnJkFFj/sHhFhAKPFIjIEiihs1kRoaCUwzYism4nLln/d5HVh8KFnBLkl1TKXQYRkSI6m60+FHiUQESWQgnfR1YfChdylXXXIyKyXhkF5cix8uHxVh8KSujtJyLlsPYLczIUiIjMKLPQur+TGApERGaUWcDmI1llWPkHQETKkllo3d9JVh0Koiha/aEaESmLtX8nWXUo5BRXoNIoyl0GEZHE2lsvrDoU0q28l5+IlIfNRzKy9sM0IlKeLCsf/GLVoZBusO5EJiLlySoqhyhab7O2dYeClScyESlPpVFEnhVfj82qQyGToUBEFsiaz5+y6lDgkQIRWSJr7my26lCw9qFfRKRM1tyKYdWhYCipkLsEIqI6eKQgkwqjSe4SiIjqKCqvkruEZmtWKHz00UcICQmBXq9Hv3798Ndff113/q+//hoRERHQ6/Xo0qULfv7552YVe62KKoYCEVkeo8l6h6TaNfUFGzZswNNPP43ly5ejX79+WLx4MaKjo5GYmAgfH5868+/duxeTJk3CW2+9hTvvvBOxsbG45557cPjwYXTu3Pmmiq+SacObykuQ//uXKEnaB1OJAVqfMLiPeBg6/w4AAGNxHvJ2fYay80dgKiuGLigKHiNmQ+MReN3lFhz4AYVHf4axIAsqexc4dBwA98HTINhpAQCGfV+h5PQ+VOamQrDTQhcYCffB06HxbAMAqDJkIG35zHqX7XX383CMGAhjaSFyfnofZReOw849AF53zIHWt500X862ZdC4+cKl7zhzbCoim1RlxZffEcQmnmXRr18/9OnTB0uWLAEAmEwmBAUF4Z///Ceef/75OvNPmDABxcXF2Lx5szTtlltuQffu3bF8+fKbKj783z/Lcu2jrB/eQWVWCjyiH4PayQPF8TtRcOAHBDy0FGonT6R/+SwElR3ch82ESuuAggPfozT5EAJmLoNKq693mcUJu5D98wfwumMOdIGRqMxNQ87Pi+EQcRs8hs8CAGR89TIcIwdB6xcOiEbk//YFKrJTpOWKJiNMJQW1llv49xYU/PUt2jz+BVRae+T++gkq0s/Ac9Q/UXjkZ5SnxsN/2mIAQHnaKeTuWA6/+xdCUKlbdBuSZSo8vBmG/d/CWJwHrU8oPEbMhi6gY73zVmSlwPDHWpSnn4GxIBPuw2bBpc/dteZJXfYgjAWZdV7r1GMMPEc+CgDI2bIEZSlHYSzKhaDRV+/sDJkOjWcQAFjljswDtwbjtbtvbqd39+7dWLBgAQ4dOoTLly/ju+++wz333HPd1+zatQtPP/004uPjERQUhJdeegnTp09v0vs2qfmooqIChw4dwogRI/63AJUKI0aMwL59++p9zb59+2rNDwDR0dENzt8UcgSCqbIcJYl74DZ0BvRBnaFxD4DbwCnQuPuj8MgvqMq7hIpLifAY+Rh0/h2g8WwDj+jHIFZVoPjkbw0utzztJPRtIuHYaQjsXH1hH9oTDpGDUHE5SZrHd/xrcOoyAlrvYGh9wuA5Zi6MBVmoyDgDABBUaqid3Gv9lJzeB4eOA6HS2gMAKnMuwjFyEDQegXDuNgqVORcBAKKxCjnbPoLHyMcZCDaq+ORu5P76CdwGTIL/9A+g9QlF5lcvw1icX+/8YlU57Nz84D54GtSO7vXO4z9tEdo8vkb68ZnwOgDAMWKANI/Wrz0873gKAQ8tg8/41wCIyNjwMkSTEQBg2LcBpopS+E//APq2XZCz5UPpteVpp1BxORHOvWuHkdzM0YpRXFyMbt264aOPPmrU/MnJyRgzZgyGDh2Ko0eP4qmnnsJDDz2ErVu3Nul9mxQK2dnZMBqN8PX1rTXd19cX6enp9b4mPT29SfM3VqVcncwmIyCaIKg1tSYLdjqUp8ZDNFZeeaz933OCCoJag/LUhAYXqwuMRHn6WZRfSgQAVOano/TsQdi3691wKeXFAACV3qne58vTz6Ay8xycuo6Upml9QlGWcgyiyYjS5MPQeIcAAAr2b4Q+qAt0/uHXWXlSsoID38O5WzScut4OrVdbeEQ/DkGjQ9Hx7fXOr/PvAPehD8Kx02Dgmv8PNdQOrrV2UkrP/AU7N3/ogrpI8zh3HwV9UGfYufpC59cebrfdD2NhFqoM1UcY1rgjYzTDDuvo0aPx+uuv4957723U/MuXL0doaCgWLlyIyMhIPPHEE7jvvvuwaNGiJr2v1Y4+kqsjR6VzgC4gAoa961FVmAPRZERR/E6UXzoFY3EeNB5toHbxRv5vn8NYVgTRWAnDn9/AWJgNY1Fug8t17DQEbgOnIH3tv5Cy4G5cWvEQ9G27wPXW8fXOL4om5MWthC6wE7RXvtivVXRsGzSeQdC3iZSmud4SA6jUSFvxEEqS9sFz9BxU5qah6EQcXAdMRM7WJUhbPhNZ378thQ4pn2isREX6GeiDu0vTBEEFfUh3lKedMtt7FCfsglPX2yEIQr3zmCrKUHR8B+xcfWHn4gXAOndk5OjvNFerTJM6mr28vKBWq5GRkVFrekZGBvz8/Op9jZ+fX5Pmb6wG/qZaheedzyDnlw+QtnQaIKig9WsHx8hBKE8/A0FtB+97/42cXz5A6gcTgSv/sfRhvYDr/J2UXTgGw59fwWPko9AFdERV3iXk7liJ/D3r4DZgUp35c7ctQ0VWCvymvFvv8kyV5ShO+A1u/SfUmq7SOcJ77Lxa09LXvQj3oQ+iOH4XqvIzEDBrBXK2fIj8PevgMeyhpm8gKyUIIrSCCDtBhJ3qyr9VIjSCqXqaAGhUJmgEEWpceU4lwg6m6t+CCWoAdqqa+U3V8135t50gQg0T1CoRdhChFkywgwlqoebf1b/VEKGqeQ4iVDXzwAS1YKp+DBNUuDINIlQwQgUTVIIItVg9T52fK9OFmvmlxyZk5hWjm2jCF+G70DfkSPU8ohEv+53FnqQ87Ap6DwJMEEQjhHq6ISPsDHjCdTueCIxvcPtuPJyN6eUFOHh7AgJc36z13IrfL+OlH1JQXGFCBx97fPtkJMK8FwAADPdWYc5X57DvkxhEeejxwbQwaNT/wbhTJ7Hz6S54dfNMxJ3KR88gJ3w0qR1c7Zs8fsbsKhxHAOjWqu/ZUKtMQUEBSktLYW9v36jlNGnrabVa9OrVC3FxcVKHh8lkQlxcHJ544ol6X3PrrbciLi4OTz31lDRt+/btuPXWW5vy1nWoZUwFjbs//Ca/DVNFGUwVJbBz8kDWD+9A41YddDq/9giY8SFM5cUQjVVQO7ji8hdPV3cQNyD/9y/hFDUMzt2iAQBa7xCYKsuRu2UJXPtPgCD876Aud/sylJ49AN/Jb0t7U9cqSdwDsbIcjp2HX3ddio5th0rvCIfwW5D53RtwCL8FgtoODhEDYfj9y6ZuGqsmigLKRQHlAGCUu5rWVVWYA2ALZly6Bzrhf0eWeQWrUFZxAp0vPitNUwtXglEFaIXqYMwwPozlJUOwJW8MNCoRmppQvCpI4/a8jYCO3fGe/ROwq6yeRy1UB56xVzFmdDCgtMiAvXHbEf1ZFp585inYa+2gcjZh6EwRt1+ZPx4mvL1oBe6+LwYvH83F4dxcvPd/s7F0zQ94bIcKj4wfBrVQE5RXB+eVQBWqA1ItmCCI/wvXmoBUXROqgli9HKFmumiEAPHK9OplCFdPE03y7rXepCZH6tNPP41p06ahd+/e6Nu3LxYvXozi4mLMmDEDAPDAAw8gMDAQb731FgBgzpw5GDx4MBYuXIgxY8Zg/fr1OHjwID7++OObKlytkn+jq7R6qLR6GMuKUJp8GO5DZtR+XucIAKjMTUNF+hm43Ta1wWWJleV1/pCkIBBFQKi+/WjejuUoOb0PvpPekkKoPkXHtsGhfV+oHVwbnMdYYkD+3vXwm/JO9QSTCaLpykk3xiqIIs8DsRVqBxdAUNXpVDaW5NfpRDaKKhhFACagpoGxShSQV6mBsaT+vdEqQyYunUmA970v4pes+ndkAAB6QBx1Gy59MBEL4tKr+yuuUXRsO0pEX3zjMgGZcW/APngknknpi5JAAYbfv8Thnrc0Yc1bxgS/ILzTyu/ZUKuMi4tLo48SgGaEwoQJE5CVlYWXX34Z6enp6N69O7Zs2SIdtly4cAEq1f/2avv374/Y2Fi89NJLePHFFxEeHo7vv//+ps9REAQBglD9fdnaSs8dAgDYeQSiKu8y8natgsajDZy6VLfnFZ/6A2oHF6hdfFCZdR65Oz6GQ/gtsA/tKS0je/NCqJ094T54OgDAvn1fFBz4HlqfMGgDOqIq7zLyf/8S9u37Sp1ouduXoTjhN/iMewkqrQOMRXkAAEHnAJVGJy27Mu8Syi/Gwyfm1euuR27cx3Dpcw/snKv/k+raRKI4fifsQ3qi8O8t0AV2Msv2IssnqDXQ+rVHWcrfcOhQfRQviiaUnf8bzr3uvOnlFx3fDrWDK+zb9bnxzGL1T82gjatZy46M1q71u2tvvfXWOicGN6dVplmNb0888USDzUW7du2qMy0mJgYxMTHNeavrUgsCqmRIBVN5CfJ3f46qwmyo9c5w6NgfboMegKCu3pzGolzk/foJjMX5UDu5wylqGFwHTKy1jKqCLOCqJiHX/hMBCMj//UsYi3KgsneFffu+cB90vzRP0ZHqDzxj3Qu1luV5x1NSIAHVe1JqZy/oQ3s0uA6l5w6hKu8yvO58Rprm3PNOVFw+g8trnobOv0O9fRmkXC597kH2T4ug9QuHzr8DCg7+ALGyTPrbunZHRjRWojK7eiQQTFUwFuWgIuMcBK0eGvcAabmiaELR8R1w7Dy8ziihyvx0lJzcDX1oT6gdXFBVkIOC/V9DsNPCPqzuyDtr2ZExRygUFRXhzJkz0uPk5GQcPXoUHh4eaNu2LV544QWkpaXhiy++AAA88sgjWLJkCZ577jk8+OCD+PXXX/HVV1/hp59+atL7NvnkNUvS4aVfeKkLIjMqOPQjCv6qOXktrNbJa+mxz8PO1RdeY+YCaPgMel1QZ/hNflt6XJp8GJlfvYyAWSvqnNVfVZiDnC3/RUX6WZjKiqB2dIMuKApu/SdJZ+pLyzl3CPl/rIXf/e9JTaumyjLk/LQYpcmHoPPvAK+75kHt6GbOTdIsjw5ph3+NiripZezatQtDhw6tM33atGn47LPPMH36dJw/f77WjviuXbswd+5cJCQkoE2bNvjPf/7T5JPXrDoUOr28BSUVNtYjSEQWb87wcMy9vYPcZTSL1Z6nAADuDtobz0RE1Mrk6FMwF+utHICXE0OBiCyPjqEgD29n3Y1nIiJqZS76+i/7YQ0YCkREZubnWv/VkK2BdYeCE0OBiCyPP0NBHl48UiAiC8QjBZnwSIGILI2Tzg7O7FOQB/sUiMjSWPNRAsBQICIyK2vuTwAYCkREZuXnwlCQjYPWDg5ay7oNHxHZNh4pyIxHC0RkSfxcG3/vAktk/aHAEUhEZEF4pCAzHikQkSXh6COZtfV0kLsEIiIJjxRkFhXQ8D2IiYhak71GDTcrv6S/AkLBRe4SiIgAAIHu1t3JDCggFMK8HOHIYalEZAE6K2An1epDQRAERPpb/wdBRNavW5Cb3CXcNKsPBYBNSERkGRgKFiIqkJ3NRCQvjVpQxA6qMkJBAR8EEVm3jn7O0NlZf/+mIkKhg68ztGpFrAoRWalubdzkLsEsFPFNqlGr0MHPSe4yiMiGKaE/AVBIKABAlD/7FYhIPt0ZCpYlKpD9CkQkD0etGu29ldFaoZxQ4OUuiEgmnQNdoVIJcpdhFooJhUh/ZyjkMyEiK6OUpiNAQaHgoLVDhB+bkIio9SmlkxlQUCgAwNAIb7lLICIb1LWNcpqvFRUKwyJ85S6BiGyMt7MObdyVc18XRYVCjyA3eDpa97XMici6DOvoI3cJZqWoUFCpBAzuyCYkImo90Z2V1UKhqFAAgOFsQiKiVuKoVWNAey+5yzArxYXCoA5e0Kg5NpWIWt6Qjj6KuAje1RQXCs56DfqEeMhdBhHZgJFRymuZUFwoAMCwCGV1/BCR5dGoBQxV4HeNIkNhRKTy0puILMstYZ5w0WvkLsPsFBkKIV6OCPNylLsMIlKw6Cg/uUtoEYoMBYBNSETUcgQBGNlJmS0SDAUioibqHuQGHxe93GW0CMWGQp9QDzjr7eQug4gUaGQnZTYdAQoOBY1ahdsVenhHRPKKVuBQ1BqKDQUAmNA7SO4SiEhh2vs4IUwhd1mrj6JDoV+YJ0I5ComIzOiOzsptOgIUHgoAMJ5HC0RkJioBmNC3rdxltCjFh8I/egXCjvfpJCIzGNrRB4Fu9nKX0aIUHwo+znoMUdj1zolIHlNuUfZRAmADoQAAE/uwCYmIbk6gmz2GdFD+DqZNhMLQCOUf8hFRy5rUNwgqG2iKtolQUKsETL0lWO4yiMhKadQCxttIi4NNhAJQ3YSks7OZ1SUiMxrV2R8+zsq8rMW1bOZb0t1Ri7u7B8hdBhFZoYcGhspdQquxmVAAgGn9Q+QugYisTJ8Qd3QLcpO7jFZjU6EQFeCKvrxVJxE1wcyBYXKX0KpsKhQAYMaAELlLICIrEezpoNj7JjTE5kJhVGc/RPq7yF0GEVmBBweE2sQw1KvZXCgIgoB50R3kLoOILJyrvQYxvdvIXUars7lQAIBhEb7oHewudxlEZMEeHdIODlrbu1GXTYYCADw3KkLuEojIQgW62WO6jY5WtNlQ6BvqgSEdveUug4gs0DMjO0CvUctdhixsNhQAYF50Rwi21YdERDfQyd8F9/YIlLsM2dh0KEQFuOKOLv5yl0FEFuTFOyIh2PDeok2HAgA8c3sH3oSHiAAAt4V7YWC4l9xlyMrmQyHM2wn39bK9YWdEVJtKAF4YHSl3GbKz+VAAgDkjwnkFVSIbd2+PNugUwBNb+U0IwN/VHvfzfgtENktnp8KzPKkVAENB8tjQ9nDS2d6JKkQEzBgQCn9X3p0RYChIPBy1eHRIO7nLIKJW5u6gwWND+X+/BkPhKg8PCuPF8ohszBPDwuGi18hdhsVgKFxFo1ZhwX1dOUSVyEaEeTmyP/EaDIVrdA50xSODeShJpHRqlYCF47tBy5GHtXBr1OPJ4eHo4OskdxlE1IJmDwpDj7a8WvK1GAr10NqpsOC+blCzGYlIkSL8nPHUCA5BrQ9DoQHdgtzw0MBQucsgIjPTqlVYNKE7m40awK1yHXNv74Awb0e5yyAiM5ozIpyjDK+DoXAdeo0aC+7rCrYiESlDj7ZuHEhyAwyFG+gV7IFpNnoHJiIlsdeosTCGfYU3wlBohOeiIxDs6SB3GUR0E/41qiPCvDmq8EYYCo1gr1XjnX905V3aiKxU/3aePOJvJIZCI90S5olpt4bIXQYRNZGzzg4LYrrZ9N3UmoKh0AT/HhOJviEecpdBRE3wn7s6IdCNV0BtLIZCE2jUKiyd2hMBrnq5SyGiRhjd2Q/jewfJXYZVYSg0kZeTDh8/0Bt6DTcdkSXr5O+CheO7yV2G1eE3WzN0DnTFO//oKncZRNQAb2cdPp3eGw5a3jirqRgKzXR390DMHhQmdxlEdA2dnQorH+jNO6k1E0PhJvxrVAQGdfCWuwwiusqCmG7oHuQmdxlWi6FwE1QqAR9O6oFQL14ficgSPDk8HGO7BchdhlVjKNwkV3sNPr6/F5x0bLskktOYrv6YOyJc7jKsHkPBDMJ9nbFoQnee8Uwkk65tXLGQJ6iZBUPBTG7v5IunhvOmHUStzc9Fj5UP9IZeo5a7FEVgKJjRk8PbY3RnP7nLILIZ9ho1Vj7QG74uPKHUXBgKZiQIAhZN6I4B7T3lLoVI8QQBeH98N3Rp4yp3KYrCUDAz/ZU9l97BvCE4UUuaF90Ro7v4y12G4jAUWoCD1g6rZvRBl0DuwRC1hCeHtcdjQ9rLXYYiMRRaiItegy8e7IsOvrypB5E5PT60HZ4e2VHuMhSLodCC3B21+PKhfjy5jchMHhncDvOiI+QuQ9EYCi3Mx1mP2FkMBqKb9fCgMDw/moHQ0hgKrcDf1R7rH74FYd4MBqLmmDkwFC/eESl3GTZBEEVRlLsIW5FVWI7JK/9EUmaR3KUQWY3Hh7LJqDUxFFpZTlE5pnyyH6fSC+UuhcjiPTeqI0cZtTKGggzyiisw5ZP9SLhcIHcpRBZJEIBX74rCtP4hcpdicxgKMjGUVuLxtYfxx5lsuUshsihqlYC3x3VBDO+tLAuGgoyqjCa8tjkBX+xLkbsUIougtVNh0fjuGNOVZyrLhaFgAdb8mYL5m+JRZeJHQbbL10WH5VN7oUdbXiJGTgwFC7HnTDYeW3sYhtJKuUshanW9gt2xbGpP+DjzaqdyYyhYkOTsYsz8/ADOZRXLXQpRq5nUNwjzx3aG1o6nTVkChoKFMZRW4onYw/g9iR3QpGwatYBX7orC1FuC5S6FrsJQsEBGk4j/25yAz/ael7sUohbh5aTDsqk90SfEQ+5S6BoMBQu2dn8KXvmBHdCkLN3auGL5/b3g72ovdylUD4aChdt7troDOr+EHdBk/f7Rsw3euLcz76dswRgKVuB8djHmrD+Cv1MNcpdC1Cx2KgEv3hGJBweGyl0K3QBDwUoYTSKW7jyD//6ahEojPzKyHp6OWnw4uQf6t/OSuxRqBIaClUm4VICnvzrKC+qRVbirWwDmj42Ch6NW7lKokRgKVqiiyoQP4k5j+W/nYGQnNFkgH2cdXr+nM0ZG+cldCjURQ8GKHb2Yj2e+OoqzPNmNLEhMrzZ46c5OcLXXyF0KNQNDwcqVVRrx3tZErNqTDB40kJwC3ezx1rguGNTBW+5S6CYwFBTir+RcPPv137iQWyJ3KWRjBAGY2i8Yz4+OgKPOTu5y6CYxFBSkpKIKb/58Emv3XwA/VWoNoV6OeHtcF/QL85S7FDIThoIC/ZGUjVc2nWBfA7UYlQDMHBiKZ0Z25IloCsNQUCijScSGAxexeMdpZBaWy10OKUiEnzPeGteF9z1QKIaCwpVUVOGT35Px8e5zKCqvkrscsmLBng6YO6IDxnYLgEolyF0OtRCGgo3IKSrHf+OSEPvXBZ4RTU3i56LHP4e3x4TeQbBT854HSsdQsDHns4uxYGsifjp+We5SyMJ5OGrx6OB2uP/WYPYb2BCGgo06ejEfb/18EvuTc+UuhSyMs84OM28LxUO3hcGJQ0xtDkPBxsWdzMA7W07hdEaR3KWQzPQaFabdGoJHBreDO69VZLMYCgSjScS3h1Ox8vdzDAcbpFELmNAnCE8OC4ePi17uckhmDAWqZffpLHz6RzJ2J2XxBDiFs9eocU+PQDw6uB3aejrIXQ5ZCIYC1SspoxCr9iTj28NpKK8yyV0OmVGwpwPuvyUYMb2DeNE6qoOhQNeVW1yBdX9dwLq/LiA1r1TucqiZBAEYFO6N6f1DMKSjNwSB5xlQ/RgK1Cgmk4jdSVlYu/8Cfj2Vyfs4WAlvZx3G9QzExD5tEerlKHc5ZAUYCtRk6YYybDhwERsOXMAlQ5nc5dA17FQChnT0wYQ+QRja0ZsnnFGTMBSo2YwmEX8l52JbQjq2J2SweUlm7bwdEdM7CON6BsLHmaOIqHkYCmQ2CZcKsC0hHdviM5BwuUDuchRPoxbQN9QDQzv6YFiED8K8neQuiRSAoUAtIjWvBNsTMrAtPgMHzueiin0QZuHlpMPQjt4YFuGDgeFecNZz9BCZF0OBWlx+SQXiTmZie0IGdidloaTCKHdJVkMQgC6Brhja0QfDI33QJdCVI4eoRTEUqFWVVRqx50w2/jqfi2MXDTiRZkAhL+ldi5PODgPbe2FYpA+GdPRm/wC1KoYCyUoURZzNKsax1HwcSzXg79R8JFwqsJkT5jwctYgKcEEnfxd0CnBBVIALwryceL8Ckg1DgSxOpdGExPRCHEs14FhqPv5ONSApo9Dq+yWCPOzRyd8FUQGu1UEQ4AJ/V3u5yyKqhaFAVqGs0oj4SwaczSpGhqEM6QVlyCgow2VD9e+c4grZr9UkCICbvQbezjp4Oeng72ov7f13CnCBCzuFyQowFEgRKqpMyLgSFOkFZUg3XPm5Mi27qALllUZUGEVUGk2oqDKh0mhq8OhDEACVIEAlAC56DbycdPBy1lb/dtJJX/xeTlrpsaejlieKkdVjKJBNM5lEVJqq+y9UggC1ILA9n2waQ4GIiCQ81iUiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIsn/A2itDEewWoPMAAAAAElFTkSuQmCC\n"},"metadata":{}}]},{"cell_type":"markdown","source":["As shown above, the Class variable has two values: 0 (the credit card transaction is legitimate) and 1 (the credit card transaction is fraudulent). Thus, We need to model a binary classification problem. Moreover, the dataset is highly unbalanced, the target variable classes are not represented equally. This case requires special attention when training or when evaluating the quality of a model. One way of handing this case at train time is to bias the model to pay more attention to the samples in the minority class. The models under the current study will be configured to take into account the class weights of the samples at train/fit time."],"metadata":{"id":"795VV5B23Sd1"}},{"cell_type":"markdown","source":["Histogram:"],"metadata":{"id":"Bx3VSXzK3gK_"}},{"cell_type":"code","source":["plt.hist(big_raw_data.Amount.values, 6, histtype='bar', facecolor='g')\n","plt.show()\n","\n","print(\"Minimum amount value is \", np.min(big_raw_data.Amount.values))\n","print(\"Maximum amount value is \", np.max(big_raw_data.Amount.values))\n","print(\"90% of the transactions have an amount less or equal than \", np.percentile(raw_data.Amount.values, 90))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":497},"id":"4xSSO-9E3hPa","executionInfo":{"status":"ok","timestamp":1704530361079,"user_tz":-210,"elapsed":441,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"be0c2448-ea47-4c11-a414-21f8a837a163"},"execution_count":7,"outputs":[{"output_type":"display_data","data":{"text/plain":["<Figure size 640x480 with 1 Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAiMAAAGsCAYAAAAPJKchAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAiNUlEQVR4nO3de3CU1cHH8d9yyQYKu4AxN1gggIDcAiKEgNzGlEAZSmovlNqCVLDa4EDxVtSSYtuJVbHMWBQdCxlqEaUKTBEpGAQKBC1IhACmBilBzAYUyJKIAbLn/cNh7b4kkA1JTrL5fmZ2xjx7nmfPHtbd7+w+mziMMUYAAACWNLM9AQAA0LQRIwAAwCpiBAAAWEWMAAAAq4gRAABgFTECAACsIkYAAIBVxAgAALCKGAEAAFYRIwAAwKpGFSPbt2/XpEmTFB8fL4fDobVr14Z8DGOMnnnmGfXs2VNOp1MdO3bUH/7wh9qfLAAAqJYWticQirKyMiUmJurnP/+57rjjjhodY86cOdq0aZOeeeYZ9e/fX6dPn9bp06dreaYAAKC6HI31D+U5HA6tWbNGaWlpgW3l5eV67LHH9Oqrr+rs2bPq16+f/vjHP2rMmDGSpMOHD2vAgAHKy8tTr1697EwcAAAEaVQf01zL7NmzlZOTo1WrVmn//v364Q9/qPHjx+vjjz+WJP3jH/9Qt27dtH79eiUkJKhr166aOXMm74wAAGBR2MRIYWGhli9frtWrV2vkyJHq3r27HnzwQd12221avny5JOmTTz7RsWPHtHr1aq1YsUJZWVnau3evfvCDH1iePQAATVejOmfkag4cOKCKigr17NkzaHt5ebluuOEGSZLf71d5eblWrFgRGPeXv/xFgwcPVn5+Ph/dAABgQdjESGlpqZo3b669e/eqefPmQde1adNGkhQXF6cWLVoEBcvNN98s6et3VogRAADqX9jEyKBBg1RRUaGTJ09q5MiRlY4ZMWKELl26pCNHjqh79+6SpP/85z+SpC5dutTbXAEAwDca1bdpSktLVVBQIOnr+Hj22Wc1duxYdejQQZ07d9ZPf/pT7dy5U4sWLdKgQYN06tQpZWdna8CAAZo4caL8fr+GDBmiNm3aaPHixfL7/UpPT5fL5dKmTZss3zsAAJqmRhUjW7du1dixY6/YPn36dGVlZenixYv6/e9/rxUrVujEiROKiorSsGHDtHDhQvXv31+S9Nlnn+n+++/Xpk2b9K1vfUsTJkzQokWL1KFDh/q+OwAAQI0sRgAAQPgJm6/2AgCAxokYAQAAVjWKb9P4/X599tlnatu2rRwOh+3pAACAajDG6Ny5c4qPj1ezZlW//9EoYuSzzz6Tx+OxPQ0AAFADx48fV6dOnaq8vlHESNu2bSV9fWdcLpfl2QAAgOrw+XzyeDyB1/GqNIoYufzRjMvlIkYAAGhkrnWKBSewAgAAq4gRAABgFTECAACsIkYAAIBVxAgAALCKGAEAAFYRIwAAwCpiBAAAWEWMAAAAq4gRAABgFTECAACsIkYAAIBVxAgAALCKGAEAAFa1sD0B2xwLr/5njfENk2FsTwEAEIZ4ZwQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwKqQYyczM1JAhQ9S2bVtFR0crLS1N+fn5V90nKytLDocj6BIZGXldkwYAAOEjpBjZtm2b0tPTtXv3bm3evFkXL17UuHHjVFZWdtX9XC6XioqKApdjx45d16QBAED4aBHK4I0bNwb9nJWVpejoaO3du1ejRo2qcj+Hw6HY2NiazRAAAIS16zpnpKSkRJLUoUOHq44rLS1Vly5d5PF4NHnyZB08ePCq48vLy+Xz+YIuAAAgPNU4Rvx+v+bOnasRI0aoX79+VY7r1auXli1bpnXr1umVV16R3+/X8OHD9emnn1a5T2Zmptxud+Di8XhqOk0AANDAOYwxpiY73nfffXr77be1Y8cOderUqdr7Xbx4UTfffLOmTp2q3/3ud5WOKS8vV3l5eeBnn88nj8ejkpISuVyumky3So6Fjlo9XjgzGTV6qAAAmiifzye3233N1++Qzhm5bPbs2Vq/fr22b98eUohIUsuWLTVo0CAVFBRUOcbpdMrpdNZkagAAoJEJ6WMaY4xmz56tNWvWaMuWLUpISAj5BisqKnTgwAHFxcWFvC8AAAg/Ib0zkp6erpUrV2rdunVq27atvF6vJMntdqtVq1aSpGnTpqljx47KzMyUJD3xxBMaNmyYevToobNnz+rpp5/WsWPHNHPmzFq+KwAAoDEKKUZeeOEFSdKYMWOCti9fvlx33XWXJKmwsFDNmn3zhsuZM2c0a9Yseb1etW/fXoMHD9auXbvUp0+f65s5AAAICzU+gbU+VfcEmJrgBNbq4wRWAEAoqvv6zd+mAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYFVKMZGZmasiQIWrbtq2io6OVlpam/Pz8a+63evVq9e7dW5GRkerfv782bNhQ4wkDAIDwElKMbNu2Tenp6dq9e7c2b96sixcvaty4cSorK6tyn127dmnq1Km6++67tW/fPqWlpSktLU15eXnXPXkAAND4OYwxpqY7nzp1StHR0dq2bZtGjRpV6ZgpU6aorKxM69evD2wbNmyYBg4cqKVLl1brdnw+n9xut0pKSuRyuWo63Uo5Fjpq9XjhzGTU+KECAGiCqvv6fV3njJSUlEiSOnToUOWYnJwcpaSkBG1LTU1VTk5OlfuUl5fL5/MFXQAAQHiqcYz4/X7NnTtXI0aMUL9+/aoc5/V6FRMTE7QtJiZGXq+3yn0yMzPldrsDF4/HU9NpAgCABq7GMZKenq68vDytWrWqNucjSZo/f75KSkoCl+PHj9f6bQAAgIahRU12mj17ttavX6/t27erU6dOVx0bGxur4uLioG3FxcWKjY2tch+n0ymn01mTqQEAgEYmpHdGjDGaPXu21qxZoy1btighIeGa+yQnJys7Ozto2+bNm5WcnBzaTAEAQFgK6Z2R9PR0rVy5UuvWrVPbtm0D53243W61atVKkjRt2jR17NhRmZmZkqQ5c+Zo9OjRWrRokSZOnKhVq1Zpz549eumll2r5rgAAgMYopHdGXnjhBZWUlGjMmDGKi4sLXF577bXAmMLCQhUVFQV+Hj58uFauXKmXXnpJiYmJ+vvf/661a9de9aRXAADQdFzX7xmpL/yekYaB3zMCAAhFvfyeEQAAgOtFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYFXIMbJ9+3ZNmjRJ8fHxcjgcWrt27VXHb926VQ6H44qL1+ut6ZwBAEAYCTlGysrKlJiYqCVLloS0X35+voqKigKX6OjoUG8aAACEoRah7jBhwgRNmDAh5BuKjo5Wu3btQt4PAACEt3o7Z2TgwIGKi4vTt7/9be3cufOqY8vLy+Xz+YIuAAAgPNV5jMTFxWnp0qV644039MYbb8jj8WjMmDH64IMPqtwnMzNTbrc7cPF4PHU9TQAAYInDGGNqvLPDoTVr1igtLS2k/UaPHq3OnTvrr3/9a6XXl5eXq7y8PPCzz+eTx+NRSUmJXC5XTadbKcdCR60eL5yZjBo/VAAATZDP55Pb7b7m63fI54zUhqFDh2rHjh1VXu90OuV0OutxRgAAwBYrv2ckNzdXcXFxNm4aAAA0MCG/M1JaWqqCgoLAz0ePHlVubq46dOigzp07a/78+Tpx4oRWrFghSVq8eLESEhLUt29fffXVV3r55Ze1ZcsWbdq0qfbuBQAAaLRCjpE9e/Zo7NixgZ/nzZsnSZo+fbqysrJUVFSkwsLCwPUXLlzQAw88oBMnTqh169YaMGCA3nnnnaBjAACApuu6TmCtL9U9AaYmOIG1+jiBFQAQiuq+fvO3aQAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVhEjAADAKmIEAABYRYwAAACriBEAAGAVMQIAAKwiRgAAgFXECAAAsIoYAQAAVoUcI9u3b9ekSZMUHx8vh8OhtWvXXnOfrVu36pZbbpHT6VSPHj2UlZVVg6kCAIBwFHKMlJWVKTExUUuWLKnW+KNHj2rixIkaO3ascnNzNXfuXM2cOVP//Oc/Q54sAAAIPy1C3WHChAmaMGFCtccvXbpUCQkJWrRokSTp5ptv1o4dO/SnP/1Jqampod48AAAIM3V+zkhOTo5SUlKCtqWmpionJ6fKfcrLy+Xz+YIuAAAgPNV5jHi9XsXExARti4mJkc/n0/nz5yvdJzMzU263O3DxeDx1PU0AAGBJg/w2zfz581VSUhK4HD9+3PaUAABAHQn5nJFQxcbGqri4OGhbcXGxXC6XWrVqVek+TqdTTqezrqcGAAAagDp/ZyQ5OVnZ2dlB2zZv3qzk5OS6vmkAANAIhBwjpaWlys3NVW5urqSvv7qbm5urwsJCSV9/xDJt2rTA+HvvvVeffPKJHn74YX300Ud6/vnn9frrr+tXv/pV7dwDAADQqIUcI3v27NGgQYM0aNAgSdK8efM0aNAgLViwQJJUVFQUCBNJSkhI0FtvvaXNmzcrMTFRixYt0ssvv8zXegEAgCTJYYwxtidxLT6fT263WyUlJXK5XLV6bMdCR60eL5yZjAb/UAEANCDVff1ukN+mAQAATQcxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArKpRjCxZskRdu3ZVZGSkkpKS9P7771c5NisrSw6HI+gSGRlZ4wkDAIDwEnKMvPbaa5o3b54yMjL0wQcfKDExUampqTp58mSV+7hcLhUVFQUux44du65JAwCA8BFyjDz77LOaNWuWZsyYoT59+mjp0qVq3bq1li1bVuU+DodDsbGxgUtMTMx1TRoAAISPkGLkwoUL2rt3r1JSUr45QLNmSklJUU5OTpX7lZaWqkuXLvJ4PJo8ebIOHjx41dspLy+Xz+cLugAAgPAUUox8/vnnqqiouOKdjZiYGHm93kr36dWrl5YtW6Z169bplVdekd/v1/Dhw/Xpp59WeTuZmZlyu92Bi8fjCWWaAACgEanzb9MkJydr2rRpGjhwoEaPHq0333xTN954o1588cUq95k/f75KSkoCl+PHj9f1NAEAgCUtQhkcFRWl5s2bq7i4OGh7cXGxYmNjq3WMli1batCgQSooKKhyjNPplNPpDGVqAACgkQrpnZGIiAgNHjxY2dnZgW1+v1/Z2dlKTk6u1jEqKip04MABxcXFhTZTAAAQlkJ6Z0SS5s2bp+nTp+vWW2/V0KFDtXjxYpWVlWnGjBmSpGnTpqljx47KzMyUJD3xxBMaNmyYevToobNnz+rpp5/WsWPHNHPmzNq9JwAAoFEKOUamTJmiU6dOacGCBfJ6vRo4cKA2btwYOKm1sLBQzZp984bLmTNnNGvWLHm9XrVv316DBw/Wrl271KdPn9q7FwAAoNFyGGOM7Ulci8/nk9vtVklJiVwuV60e27HQUavHC2cmo8E/VAAADUh1X7/52zQAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKuIEQAAYBUxAgAArCJGAACAVcQIAACwihgBAABWESMAAMAqYgQAAFhFjAAAAKtqFCNLlixR165dFRkZqaSkJL3//vtXHb969Wr17t1bkZGR6t+/vzZs2FCjyQIAgPATcoy89tprmjdvnjIyMvTBBx8oMTFRqampOnnyZKXjd+3apalTp+ruu+/Wvn37lJaWprS0NOXl5V335AEAQOPnMMaYUHZISkrSkCFD9Oc//1mS5Pf75fF4dP/99+vXv/71FeOnTJmisrIyrV+/PrBt2LBhGjhwoJYuXVqt2/T5fHK73SopKZHL5QplutfkWOio1eOFM5MR0kMFANDEVff1u0UoB71w4YL27t2r+fPnB7Y1a9ZMKSkpysnJqXSfnJwczZs3L2hbamqq1q5dW+XtlJeXq7y8PPBzSUmJpK/vVK37qvYPGa7qZP0BAGHr8uvGtd73CClGPv/8c1VUVCgmJiZoe0xMjD766KNK9/F6vZWO93q9Vd5OZmamFi5ceMV2j8cTynRRy9xPum1PAQDQCJ07d05ud9WvISHFSH2ZP39+0Lspfr9fp0+f1g033CCHo/Y+VvH5fPJ4PDp+/Hitf/wD1reusb51i/WtO6xt3WpI62uM0blz5xQfH3/VcSHFSFRUlJo3b67i4uKg7cXFxYqNja10n9jY2JDGS5LT6ZTT6Qza1q5du1CmGhKXy2X9Hyycsb51i/WtW6xv3WFt61ZDWd+rvSNyWUjfpomIiNDgwYOVnZ0d2Ob3+5Wdna3k5ORK90lOTg4aL0mbN2+ucjwAAGhaQv6YZt68eZo+fbpuvfVWDR06VIsXL1ZZWZlmzJghSZo2bZo6duyozMxMSdKcOXM0evRoLVq0SBMnTtSqVau0Z88evfTSS7V7TwAAQKMUcoxMmTJFp06d0oIFC+T1ejVw4EBt3LgxcJJqYWGhmjX75g2X4cOHa+XKlXr88cf16KOP6qabbtLatWvVr1+/2rsXNeR0OpWRkXHFR0KoHaxv3WJ96xbrW3dY27rVGNc35N8zAgAAUJv42zQAAMAqYgQAAFhFjAAAAKuIEQAAYFWTjpElS5aoa9euioyMVFJSkt5//33bU2pwfvvb38rhcARdevfuHbj+q6++Unp6um644Qa1adNG3//+96/4JXeFhYWaOHGiWrdurejoaD300EO6dOlS0JitW7fqlltukdPpVI8ePZSVlVUfd69ebd++XZMmTVJ8fLwcDscVf5/JGKMFCxYoLi5OrVq1UkpKij7++OOgMadPn9add94pl8uldu3a6e6771ZpaWnQmP3792vkyJGKjIyUx+PRU089dcVcVq9erd69eysyMlL9+/fXhg0bav3+1rdrre9dd911xWN5/PjxQWNY38plZmZqyJAhatu2raKjo5WWlqb8/PygMfX5XBBuz93VWd8xY8Zc8fi99957g8Y06vU1TdSqVatMRESEWbZsmTl48KCZNWuWadeunSkuLrY9tQYlIyPD9O3b1xQVFQUup06dClx/7733Go/HY7Kzs82ePXvMsGHDzPDhwwPXX7p0yfTr18+kpKSYffv2mQ0bNpioqCgzf/78wJhPPvnEtG7d2sybN88cOnTIPPfcc6Z58+Zm48aN9Xpf69qGDRvMY489Zt58800jyaxZsybo+ieffNK43W6zdu1a8+GHH5rvfve7JiEhwZw/fz4wZvz48SYxMdHs3r3b/Otf/zI9evQwU6dODVxfUlJiYmJizJ133mny8vLMq6++alq1amVefPHFwJidO3ea5s2bm6eeesocOnTIPP7446Zly5bmwIEDdb4Gdela6zt9+nQzfvz4oMfy6dOng8awvpVLTU01y5cvN3l5eSY3N9d85zvfMZ07dzalpaWBMfX1XBCOz93VWd/Ro0ebWbNmBT1+S0pKAtc39vVtsjEydOhQk56eHvi5oqLCxMfHm8zMTIuzangyMjJMYmJipdedPXvWtGzZ0qxevTqw7fDhw0aSycnJMcZ8/QLRrFkz4/V6A2NeeOEF43K5THl5uTHGmIcfftj07ds36NhTpkwxqamptXxvGo7//2Lp9/tNbGysefrppwPbzp49a5xOp3n11VeNMcYcOnTISDL//ve/A2Pefvtt43A4zIkTJ4wxxjz//POmffv2gbU1xphHHnnE9OrVK/Dzj370IzNx4sSg+SQlJZlf/OIXtXofbaoqRiZPnlzlPqxv9Z08edJIMtu2bTPG1O9zQVN47v7/62vM1zEyZ86cKvdp7OvbJD+muXDhgvbu3auUlJTAtmbNmiklJUU5OTkWZ9Ywffzxx4qPj1e3bt105513qrCwUJK0d+9eXbx4MWgde/furc6dOwfWMScnR/379w/6y82pqany+Xw6ePBgYMz/HuPymKb0b3H06FF5vd6gdXC73UpKSgpay3bt2unWW28NjElJSVGzZs303nvvBcaMGjVKERERgTGpqanKz8/XmTNnAmOa6npv3bpV0dHR6tWrl+677z598cUXgetY3+orKSmRJHXo0EFS/T0XNJXn7v+/vpf97W9/U1RUlPr166f58+fryy+/DFzX2Ne3Qf7V3rr2+eefq6KiIugfTZJiYmL00UcfWZpVw5SUlKSsrCz16tVLRUVFWrhwoUaOHKm8vDx5vV5FRERc8UcMY2Ji5PV6JUler7fSdb583dXG+Hw+nT9/Xq1ataqje9dwXF6Lytbhf9cpOjo66PoWLVqoQ4cOQWMSEhKuOMbl69q3b1/lel8+RrgaP3687rjjDiUkJOjIkSN69NFHNWHCBOXk5Kh58+asbzX5/X7NnTtXI0aMCPwm7fp6Ljhz5kzYP3dXtr6S9JOf/ERdunRRfHy89u/fr0ceeUT5+fl68803JTX+9W2SMYLqmzBhQuC/BwwYoKSkJHXp0kWvv/56k4gEhI8f//jHgf/u37+/BgwYoO7du2vr1q26/fbbLc6scUlPT1deXp527Nhheyphqar1veeeewL/3b9/f8XFxen222/XkSNH1L179/qeZq1rkh/TREVFqXnz5lec6V1cXKzY2FhLs2oc2rVrp549e6qgoECxsbG6cOGCzp49GzTmf9cxNja20nW+fN3VxrhcriYTPJfX4mqPydjYWJ08eTLo+kuXLun06dO1st5N7bHfrVs3RUVFqaCgQBLrWx2zZ8/W+vXr9e6776pTp06B7fX1XBDuz91VrW9lkpKSJCno8duY17dJxkhERIQGDx6s7OzswDa/36/s7GwlJydbnFnDV1paqiNHjiguLk6DBw9Wy5Ytg9YxPz9fhYWFgXVMTk7WgQMHgp7kN2/eLJfLpT59+gTG/O8xLo9pSv8WCQkJio2NDVoHn8+n9957L2gtz549q7179wbGbNmyRX6/P/DElJycrO3bt+vixYuBMZs3b1avXr3Uvn37wJimvt6S9Omnn+qLL75QXFycJNb3aowxmj17ttasWaMtW7Zc8VFVfT0XhOtz97XWtzK5ubmSFPT4bdTrW6enxzZgq1atMk6n02RlZZlDhw6Ze+65x7Rr1y7oTGQY88ADD5itW7eao0ePmp07d5qUlBQTFRVlTp48aYz5+ut8nTt3Nlu2bDF79uwxycnJJjk5ObD/5a+bjRs3zuTm5pqNGzeaG2+8sdKvmz300EPm8OHDZsmSJWH51d5z586Zffv2mX379hlJ5tlnnzX79u0zx44dM8Z8/dXedu3amXXr1pn9+/ebyZMnV/rV3kGDBpn33nvP7Nixw9x0001BXz09e/asiYmJMT/72c9MXl6eWbVqlWnduvUVXz1t0aKFeeaZZ8zhw4dNRkZGo//qqTFXX99z586ZBx980OTk5JijR4+ad955x9xyyy3mpptuMl999VXgGKxv5e677z7jdrvN1q1bg75a+uWXXwbG1NdzQTg+d19rfQsKCswTTzxh9uzZY44ePWrWrVtnunXrZkaNGhU4RmNf3yYbI8YY89xzz5nOnTubiIgIM3ToULN7927bU2pwpkyZYuLi4kxERITp2LGjmTJliikoKAhcf/78efPLX/7StG/f3rRu3dp873vfM0VFRUHH+O9//2smTJhgWrVqZaKioswDDzxgLl68GDTm3XffNQMHDjQRERGmW7duZvny5fVx9+rVu+++ayRdcZk+fbox5uuv9/7mN78xMTExxul0mttvv93k5+cHHeOLL74wU6dONW3atDEul8vMmDHDnDt3LmjMhx9+aG677TbjdDpNx44dzZNPPnnFXF5//XXTs2dPExERYfr27WveeuutOrvf9eVq6/vll1+acePGmRtvvNG0bNnSdOnSxcyaNeuKJ1jWt3KVraukoP9P6/O5INyeu6+1voWFhWbUqFGmQ4cOxul0mh49epiHHnoo6PeMGNO419dhjDF1+94LAABA1ZrkOSMAAKDhIEYAAIBVxAgAALCKGAEAAFYRIwAAwCpiBAAAWEWMAAAAq4gRAABgFTECAACsIkYAAIBVxAgAALCKGAEAAFb9H2nTgNz+xJhcAAAAAElFTkSuQmCC\n"},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Minimum amount value is  0.0\n","Maximum amount value is  25691.16\n","90% of the transactions have an amount less or equal than  203.0\n"]}]},{"cell_type":"markdown","source":["### Preprocessing"],"metadata":{"id":"1f8T7GJXrcvX"}},{"cell_type":"markdown","source":["Scaling and Normalizing"],"metadata":{"id":"UlSWUh7TuPil"}},{"cell_type":"code","source":["# standardize features by removing the mean and scaling to unit variance\n","big_raw_data.iloc[:, 1:30] = StandardScaler().fit_transform(big_raw_data.iloc[:, 1:30])\n","data_matrix = big_raw_data.values\n","\n","# X: feature matrix (for this analysis, we exclude the Time variable from the dataset)\n","X = data_matrix[:, 1:30]\n","\n","# y: labels vector\n","y = data_matrix[:, 30]\n","\n","# data L1-normalization\n","X = normalize(X, norm=\"l1\")\n","\n","# print the shape of the features matrix and the labels vector\n","print('X.shape=', X.shape, 'y.shape=', y.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"HwGxPLVD3xvM","executionInfo":{"status":"ok","timestamp":1704530363868,"user_tz":-210,"elapsed":2792,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"d46be0e8-a497-47e1-e543-6c5cb635c15d"},"execution_count":8,"outputs":[{"output_type":"stream","name":"stdout","text":["X.shape= (2848070, 29) y.shape= (2848070,)\n"]}]},{"cell_type":"markdown","source":["### Dataset Train/Test Split"],"metadata":{"id":"GW28Uj3B4Ei8"}},{"cell_type":"code","source":["X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y)\n","print('X_train.shape=', X_train.shape, 'Y_train.shape=', y_train.shape)\n","print('X_test.shape=', X_test.shape, 'Y_test.shape=', y_test.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"qRfH0tFlsl2I","executionInfo":{"status":"ok","timestamp":1704530366040,"user_tz":-210,"elapsed":2174,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"f8569538-3300-477a-9a2e-62e64913a1c5"},"execution_count":9,"outputs":[{"output_type":"stream","name":"stdout","text":["X_train.shape= (1993649, 29) Y_train.shape= (1993649,)\n","X_test.shape= (854421, 29) Y_test.shape= (854421,)\n"]}]},{"cell_type":"markdown","source":["### Decision Tree Modelling"],"metadata":{"id":"HIlNBShA40q8"}},{"cell_type":"markdown","source":["#### Build a Decision Tree Classifier model with Scikit-Learn"],"metadata":{"id":"DAO4kYwKNeSd"}},{"cell_type":"code","source":["# compute the sample weights to be used as input to the train routine so that\n","# it takes into account the class imbalance present in this dataset\n","w_train = compute_sample_weight('balanced', y_train)\n","\n","# import the Decision Tree Classifier Model from scikit-learn\n","from sklearn.tree import DecisionTreeClassifier\n","\n","# for reproducible output across multiple function calls, set random_state to a given integer value\n","sklearn_dt = DecisionTreeClassifier(max_depth=4, random_state=35)\n","\n","# train a Decision Tree Classifier using scikit-learn\n","t0 = time.time()\n","sklearn_dt.fit(X_train, y_train, sample_weight=w_train)\n","sklearn_time = time.time()-t0\n","print(\"[Scikit-Learn] Training time (s):  {0:.5f}\".format(sklearn_time))"],"metadata":{"id":"v-TnHVXMNhtg","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1704530408356,"user_tz":-210,"elapsed":42319,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"dfe0620a-55d8-4746-a642-661dae228325"},"execution_count":10,"outputs":[{"output_type":"stream","name":"stdout","text":["[Scikit-Learn] Training time (s):  41.52780\n"]}]},{"cell_type":"markdown","source":["#### Or Build a Decision Tree Classifier model with Snap ML"],"metadata":{"id":"sVjNeEkS4hL4"}},{"cell_type":"code","source":["# if not already computed,\n","# compute the sample weights to be used as input to the train routine so that\n","# it takes into account the class imbalance present in this dataset\n","# w_train = compute_sample_weight('balanced', y_train)\n","\n","# import the Decision Tree Classifier Model from Snap ML\n","from snapml import DecisionTreeClassifier\n","\n","# Snap ML offers multi-threaded CPU/GPU training of decision trees, unlike scikit-learn\n","# to use the GPU, set the use_gpu parameter to True\n","# snapml_dt = DecisionTreeClassifier(max_depth=4, random_state=45, use_gpu=True)\n","\n","# to set the number of CPU threads used at training time, set the n_jobs parameter\n","# for reproducible output across multiple function calls, set random_state to a given integer value\n","snapml_dt = DecisionTreeClassifier(max_depth=4, random_state=45, n_jobs=4)\n","\n","# train a Decision Tree Classifier model using Snap ML\n","t0 = time.time()\n","snapml_dt.fit(X_train, y_train, sample_weight=w_train)\n","snapml_time = time.time()-t0\n","print(\"[Snap ML] Training time (s):  {0:.5f}\".format(snapml_time))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"CpK9IVaC4orJ","executionInfo":{"status":"ok","timestamp":1704530417240,"user_tz":-210,"elapsed":8887,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"10804c1c-fd45-4b87-d531-dae63c1a536e"},"execution_count":11,"outputs":[{"output_type":"stream","name":"stdout","text":["[Snap ML] Training time (s):  6.17015\n"]}]},{"cell_type":"markdown","source":["#### Evaluating the Scikit-Learn and Snap ML Decision Tree Classifier Models"],"metadata":{"id":"ZNHHViSQN35J"}},{"cell_type":"code","source":["# Snap ML vs Scikit-Learn training speedup\n","training_speedup = sklearn_time/snapml_time\n","print('[Decision Tree Classifier] Snap ML vs. Scikit-Learn speedup : {0:.2f}x '.format(training_speedup))\n","\n","# run inference and compute the probabilities of the test samples\n","# to belong to the class of fraudulent transactions\n","sklearn_pred = sklearn_dt.predict_proba(X_test)[:,1]\n","\n","# evaluate the Compute Area Under the Receiver Operating Characteristic\n","# Curve (ROC-AUC) score from the predictions\n","sklearn_roc_auc = roc_auc_score(y_test, sklearn_pred)\n","print('[Scikit-Learn] ROC-AUC score : {0:.3f}'.format(sklearn_roc_auc))\n","\n","# run inference and compute the probabilities of the test samples\n","# to belong to the class of fraudulent transactions\n","snapml_pred = snapml_dt.predict_proba(X_test)[:,1]\n","\n","# evaluate the Compute Area Under the Receiver Operating Characteristic\n","# Curve (ROC-AUC) score from the prediction scores\n","snapml_roc_auc = roc_auc_score(y_test, snapml_pred)\n","print('[Snap ML] ROC-AUC score : {0:.3f}'.format(snapml_roc_auc))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"gebahLCZvQYy","executionInfo":{"status":"ok","timestamp":1704530418296,"user_tz":-210,"elapsed":1059,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"c41f3ed9-e506-4ca8-fca8-8a0c534a4817"},"execution_count":12,"outputs":[{"output_type":"stream","name":"stdout","text":["[Decision Tree Classifier] Snap ML vs. Scikit-Learn speedup : 6.73x \n","[Scikit-Learn] ROC-AUC score : 0.966\n","[Snap ML] ROC-AUC score : 0.966\n"]}]},{"cell_type":"markdown","source":["As shown above both decision tree models provide the same score on the test dataset. However Snap ML runs the training routine 12x faster than Scikit-Learn. This is one of the advantages of using Snap ML: acceleration of training of classical machine learning models, such as linear and tree-based models."],"metadata":{"id":"UBF_WHrN5fr6"}},{"cell_type":"markdown","source":["### Building the SVM Model"],"metadata":{"id":"rVWO2ofz57gk"}},{"cell_type":"markdown","source":["#### Build a Support Vector Machine model with Scikit-Learn"],"metadata":{"id":"JCpGgtB95uhB"}},{"cell_type":"code","source":["# import the linear Support Vector Machine (SVM) model from Scikit-Learn\n","from sklearn.svm import LinearSVC\n","\n","# instatiate a scikit-learn SVM model\n","# to indicate the class imbalance at fit time, set class_weight='balanced'\n","# for reproducible output across multiple function calls, set random_state to a given integer value\n","sklearn_svm = LinearSVC(class_weight='balanced', random_state=31, loss=\"hinge\", fit_intercept=False)\n","\n","# train a linear Support Vector Machine model using Scikit-Learn\n","t0 = time.time()\n","sklearn_svm.fit(X_train, y_train)\n","sklearn_time = time.time() - t0\n","print(\"[Scikit-Learn] Training time (s):  {0:.2f}\".format(sklearn_time))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"zCEU-4N-5wjj","executionInfo":{"status":"ok","timestamp":1704530500410,"user_tz":-210,"elapsed":82117,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"5e3be711-1864-4e1e-ae8c-0230683ce136"},"execution_count":13,"outputs":[{"output_type":"stream","name":"stdout","text":["[Scikit-Learn] Training time (s):  82.13\n"]}]},{"cell_type":"markdown","source":["#### Build a Support Vector Machine model with Snap ML"],"metadata":{"id":"GJIxu-ja6d23"}},{"cell_type":"code","source":["# import the Support Vector Machine model (SVM) from Snap ML\n","from snapml import SupportVectorMachine\n","\n","# in contrast to scikit-learn's LinearSVC, Snap ML offers multi-threaded CPU/GPU training of SVMs\n","# to use the GPU, set the use_gpu parameter to True\n","# snapml_svm = SupportVectorMachine(class_weight='balanced', random_state=25, use_gpu=True, fit_intercept=False)\n","\n","# to set the number of threads used at training time, one needs to set the n_jobs parameter\n","snapml_svm = SupportVectorMachine(class_weight='balanced', random_state=25, n_jobs=4, fit_intercept=False, use_gpu=True)\n","# print(snapml_svm.get_params())\n","\n","# train an SVM model using Snap ML\n","t0 = time.time()\n","model = snapml_svm.fit(X_train, y_train)\n","snapml_time = time.time() - t0\n","print(\"[Snap ML] Training time (s):  {0:.2f}\".format(snapml_time))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"dZ6aKLK76e_n","executionInfo":{"status":"ok","timestamp":1704530507138,"user_tz":-210,"elapsed":6731,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"88a43282-e8bb-4201-f308-8e6fcf1717d2"},"execution_count":14,"outputs":[{"output_type":"stream","name":"stdout","text":["[Snap ML] Training time (s):  6.86\n"]}]},{"cell_type":"markdown","source":["#### Evaluate the Scikit-Learn and Snap ML Support Vector Machine Models"],"metadata":{"id":"tTey8uYT6wSs"}},{"cell_type":"code","source":["# compute the Snap ML vs Scikit-Learn training speedup\n","training_speedup = sklearn_time/snapml_time\n","print('[Support Vector Machine] Snap ML vs. Scikit-Learn training speedup : {0:.2f}x '.format(training_speedup))\n","\n","# run inference using the Scikit-Learn model\n","# get the confidence scores for the test samples\n","sklearn_pred = sklearn_svm.decision_function(X_test)\n","\n","# evaluate accuracy on test set\n","acc_sklearn  = roc_auc_score(y_test, sklearn_pred)\n","print(\"[Scikit-Learn] ROC-AUC score:   {0:.3f}\".format(acc_sklearn))\n","\n","# run inference using the Snap ML model\n","# get the confidence scores for the test samples\n","snapml_pred = snapml_svm.decision_function(X_test)\n","\n","# evaluate accuracy on test set\n","acc_snapml  = roc_auc_score(y_test, snapml_pred)\n","print(\"[Snap ML] ROC-AUC score:   {0:.3f}\".format(acc_snapml))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"NlMLPt5c6yCv","executionInfo":{"status":"ok","timestamp":1704530507783,"user_tz":-210,"elapsed":647,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"744d2dc0-1a4f-4508-ead7-b125235d9c96"},"execution_count":15,"outputs":[{"output_type":"stream","name":"stdout","text":["[Support Vector Machine] Snap ML vs. Scikit-Learn training speedup : 11.97x \n","[Scikit-Learn] ROC-AUC score:   0.984\n","[Snap ML] ROC-AUC score:   0.984\n"]}]},{"cell_type":"markdown","source":["As shown above both SVM models provide the same score on the test dataset. However, as in the case of decision trees, Snap ML runs the training routine faster than Scikit-Learn. Moreover, as shown above, not only is Snap ML seemlessly accelerating scikit-learn applications, but the library's Python API is also compatible with scikit-learn metrics and data preprocessors."],"metadata":{"id":"x7AYmiob7Hbe"}},{"cell_type":"markdown","source":["#### Evaluating the quality of the SVM models trained above using the hinge loss metric"],"metadata":{"id":"WCv70kdp8vVN"}},{"cell_type":"code","source":["# get the confidence scores for the test samples\n","sklearn_pred = sklearn_svm.decision_function(X_test)\n","snapml_pred  = snapml_svm.decision_function(X_test)\n","\n","# import the hinge_loss metric from scikit-learn\n","from sklearn.metrics import hinge_loss\n","\n","# evaluate the hinge loss from the predictions\n","loss_snapml = hinge_loss(y_test, snapml_pred)\n","print(\"[Snap ML] Hinge loss:   {0:.3f}\".format(loss_snapml))\n","\n","# evaluate the hinge loss metric from the predictions\n","loss_sklearn = hinge_loss(y_test, sklearn_pred)\n","print(\"[Scikit-Learn] Hinge loss:   {0:.3f}\".format(loss_snapml))\n","\n","# the two models should give the same Hinge loss"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"3Q3201RE8u5D","executionInfo":{"status":"ok","timestamp":1704530781388,"user_tz":-210,"elapsed":911,"user":{"displayName":"Amir Kermanshahani","userId":"15909005352054600711"}},"outputId":"86f41cac-b912-4daf-afe7-ba7a43dd98f1"},"execution_count":16,"outputs":[{"output_type":"stream","name":"stdout","text":["[Snap ML] Hinge loss:   0.233\n","[Scikit-Learn] Hinge loss:   0.233\n"]}]}]}