fastprop_benchmark/air_pollution_prop.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "tags": [
                    "hide_input"
                ]
            },
            "source": [
                "# Propositionalization: Predicting air pollution in Beijing\n",
                "\n",
                "In this notebook we will compare getML to featuretools and tsfresh, both of which open-source libraries for feature engineering. We find that advanced algorithms featured in getML yield significantly better predictions on this dataset. We then discuss why that is.\n",
                "\n",
                "Summary:\n",
                "\n",
                "- Prediction type: __Regression model__\n",
                "- Domain: __Air pollution__\n",
                "- Prediction target: __pm 2.5 concentration__ \n",
                "- Source data: __Multivariate time series__\n",
                "- Population size: __41757__"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "tags": [
                    "remove_cell_on_docs"
                ]
            },
            "source": [
                "<a target=\"_blank\" href=\"https://colab.research.google.com/github/getml/getml-demo/blob/master/fastprop_benchmark/air_pollution_prop.ipynb\">\n",
                "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
                "</a>"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Background\n",
                "\n",
                "A common approach to feature engineering is to generate attribute-value representations from relational data by applying a fixed set of aggregations to columns of interest and perform a feature selection on the (possibly large) set of generated features afterwards. In academia, this approach is called _propositionalization._\n",
                "\n",
                "getML's [FastProp](https://getml.com/latest/user_guide/concepts/feature_engineering/#feature-engineering-algorithms-fastprop) is an implementation of this propositionalization approach that has been optimized for speed and memory efficiency. In this notebook, we want to demonstrate how – well – fast FastProp is. To this end, we will benchmark FastProp against the popular feature engineering libraries [featuretools](https://www.featuretools.com/) and [tsfresh](https://tsfresh.readthedocs.io/en/latest/). Both of these libraries use propositionalization approaches for feature engineering.\n",
                "\n",
                "As our example dataset, we use a publicly available dataset on air pollution in Beijing, China (https://archive.ics.uci.edu/dataset/381/beijing+pm2+5+data). For further details about the data set refer to [the full notebook](https://getml.com/latest/examples/enterprise-notebooks/air_pollution)."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Analysis"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Table of contents\n",
                "\n",
                "1. [Loading data](#1.-Loading-data)\n",
                "2. [Predictive modeling](#2.-Predictive-modeling)\n",
                "3. [Comparison](#3.-Comparison)\n",
                "4. [Conclusion](#4.-Conclusion)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Note: you may need to restart the kernel to use updated packages.\n"
                    ]
                }
            ],
            "source": [
                "%pip install -q \"getml==1.5.0\" \"featuretools==1.31.0\" \"tsfresh==0.20.3\""
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {
                "tags": []
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "getML API version: 1.5.0\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "import os\n",
                "import sys\n",
                "\n",
                "os.environ[\"PYARROW_IGNORE_TIMEZONE\"] = \"1\"\n",
                "from pathlib import Path\n",
                "from urllib import request\n",
                "import getml\n",
                "import pandas as pd\n",
                "\n",
                "print(f\"getML API version: {getml.__version__}\\n\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {},
            "outputs": [],
            "source": [
                "# If we are in Colab, we need to fetch the utils folder from the repository\n",
                "if os.getenv(\"COLAB_RELEASE_TAG\"):\n",
                "    !curl -L https://api.github.com/repos/getml/getml-demo/tarball/master | tar --wildcards --strip-components=1 -xz '*utils*'"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [],
            "source": [
                "parent = Path(os.getcwd()).parent.as_posix()\n",
                "\n",
                "if parent not in sys.path:\n",
                "    sys.path.append(parent)\n",
                "\n",
                "from utils import Benchmark, FTTimeSeriesBuilder, TSFreshBuilder"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 1. Loading data"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 1.1 Download from source\n",
                "\n",
                "We begin by downloading the data from the UCI Machine Learning repository."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {},
            "outputs": [],
            "source": [
                "FEATURETOOLS_FILES = [\"featuretools_training.csv\", \"featuretools_test.csv\"]\n",
                "\n",
                "for fname in FEATURETOOLS_FILES:\n",
                "    if not os.path.exists(fname):\n",
                "        fname, res = request.urlretrieve(\n",
                "            \"https://static.getml.com/datasets/air_pollution/featuretools/\" + fname,\n",
                "            fname,\n",
                "        )"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {},
            "outputs": [],
            "source": [
                "TSFRESH_FILES = [\"tsfresh_training.csv\", \"tsfresh_test.csv\"]\n",
                "\n",
                "for fname in TSFRESH_FILES:\n",
                "    if not os.path.exists(fname):\n",
                "        fname, res = request.urlretrieve(\n",
                "            \"https://static.getml.com/datasets/air_pollution/tsfresh/\" + fname, fname\n",
                "        )"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {},
            "outputs": [],
            "source": [
                "fname = \"PRSA_data_2010.1.1-2014.12.31.csv\"\n",
                "\n",
                "if not os.path.exists(fname):\n",
                "    fname, res = request.urlretrieve(\n",
                "        \"https://archive.ics.uci.edu/ml/machine-learning-databases/00381/\" + fname,\n",
                "        fname,\n",
                "    )"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 1.2 Prepare data for tsfresh and getML\n",
                "\n",
                "Our our goal is to predict the pm2.5 concentration from factors such as weather or time of day. However, there are some **missing entries** for pm2.5, so we get rid of them."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_full_pandas = pd.read_csv(fname)\n",
                "\n",
                "data_full_pandas = data_full_pandas[\n",
                "    data_full_pandas[\"pm2.5\"] == data_full_pandas[\"pm2.5\"]\n",
                "]"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "tsfresh requires a date column, so we build one."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {},
            "outputs": [],
            "source": [
                "def add_leading_zero(val):\n",
                "    if len(str(val)) == 1:\n",
                "        return \"0\" + str(val)\n",
                "    return str(val)\n",
                "\n",
                "\n",
                "data_full_pandas[\"month\"] = [add_leading_zero(val) for val in data_full_pandas[\"month\"]]\n",
                "\n",
                "data_full_pandas[\"day\"] = [add_leading_zero(val) for val in data_full_pandas[\"day\"]]\n",
                "\n",
                "data_full_pandas[\"hour\"] = [add_leading_zero(val) for val in data_full_pandas[\"hour\"]]\n",
                "\n",
                "\n",
                "def make_date(year, month, day, hour):\n",
                "    return year + \"-\" + month + \"-\" + day + \" \" + hour + \":00:00\"\n",
                "\n",
                "\n",
                "data_full_pandas[\"date\"] = [\n",
                "    make_date(str(year), month, day, hour)\n",
                "    for year, month, day, hour in zip(\n",
                "        data_full_pandas[\"year\"],\n",
                "        data_full_pandas[\"month\"],\n",
                "        data_full_pandas[\"day\"],\n",
                "        data_full_pandas[\"hour\"],\n",
                "    )\n",
                "]"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "tsfresh also requires the time series to have ids. Since there is only a single time series, that series has the same id."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_full_pandas[\"id\"] = 1"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The dataset now contains many columns that we do not need or that tsfresh cannot process. For instance, *cbwd* might actually contain useful information, but it is a categorical variable, which is difficult to handle for tsfresh, so we remove it."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "We also want to split our data into a training and testing set."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_train_pandas = data_full_pandas[data_full_pandas[\"year\"] < 2014]\n",
                "data_test_pandas = data_full_pandas[data_full_pandas[\"year\"] == 2014]\n",
                "data_full_pandas = data_full_pandas"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {},
            "outputs": [],
            "source": [
                "def remove_unwanted_columns(df):\n",
                "    del df[\"cbwd\"]\n",
                "    del df[\"year\"]\n",
                "    del df[\"month\"]\n",
                "    del df[\"day\"]\n",
                "    del df[\"hour\"]\n",
                "    del df[\"No\"]\n",
                "    return df\n",
                "\n",
                "\n",
                "data_full_pandas = remove_unwanted_columns(data_full_pandas)\n",
                "data_train_pandas = remove_unwanted_columns(data_train_pandas)\n",
                "data_test_pandas = remove_unwanted_columns(data_test_pandas)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>pm2.5</th>\n",
                            "      <th>DEWP</th>\n",
                            "      <th>TEMP</th>\n",
                            "      <th>PRES</th>\n",
                            "      <th>Iws</th>\n",
                            "      <th>Is</th>\n",
                            "      <th>Ir</th>\n",
                            "      <th>date</th>\n",
                            "      <th>id</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>24</th>\n",
                            "      <td>129.0</td>\n",
                            "      <td>-16</td>\n",
                            "      <td>-4.0</td>\n",
                            "      <td>1020.0</td>\n",
                            "      <td>1.79</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 00:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>25</th>\n",
                            "      <td>148.0</td>\n",
                            "      <td>-15</td>\n",
                            "      <td>-4.0</td>\n",
                            "      <td>1020.0</td>\n",
                            "      <td>2.68</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 01:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>26</th>\n",
                            "      <td>159.0</td>\n",
                            "      <td>-11</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1021.0</td>\n",
                            "      <td>3.57</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 02:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>27</th>\n",
                            "      <td>181.0</td>\n",
                            "      <td>-7</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1022.0</td>\n",
                            "      <td>5.36</td>\n",
                            "      <td>1</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 03:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>28</th>\n",
                            "      <td>138.0</td>\n",
                            "      <td>-7</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1022.0</td>\n",
                            "      <td>6.25</td>\n",
                            "      <td>2</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 04:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>...</th>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>43819</th>\n",
                            "      <td>8.0</td>\n",
                            "      <td>-23</td>\n",
                            "      <td>-2.0</td>\n",
                            "      <td>1034.0</td>\n",
                            "      <td>231.97</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2014-12-31 19:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>43820</th>\n",
                            "      <td>10.0</td>\n",
                            "      <td>-22</td>\n",
                            "      <td>-3.0</td>\n",
                            "      <td>1034.0</td>\n",
                            "      <td>237.78</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2014-12-31 20:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>43821</th>\n",
                            "      <td>10.0</td>\n",
                            "      <td>-22</td>\n",
                            "      <td>-3.0</td>\n",
                            "      <td>1034.0</td>\n",
                            "      <td>242.70</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2014-12-31 21:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>43822</th>\n",
                            "      <td>8.0</td>\n",
                            "      <td>-22</td>\n",
                            "      <td>-4.0</td>\n",
                            "      <td>1034.0</td>\n",
                            "      <td>246.72</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2014-12-31 22:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>43823</th>\n",
                            "      <td>12.0</td>\n",
                            "      <td>-21</td>\n",
                            "      <td>-3.0</td>\n",
                            "      <td>1034.0</td>\n",
                            "      <td>249.85</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2014-12-31 23:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>41757 rows × 9 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "       pm2.5  DEWP  TEMP    PRES     Iws  Is  Ir                 date  id\n",
                            "24     129.0   -16  -4.0  1020.0    1.79   0   0  2010-01-02 00:00:00   1\n",
                            "25     148.0   -15  -4.0  1020.0    2.68   0   0  2010-01-02 01:00:00   1\n",
                            "26     159.0   -11  -5.0  1021.0    3.57   0   0  2010-01-02 02:00:00   1\n",
                            "27     181.0    -7  -5.0  1022.0    5.36   1   0  2010-01-02 03:00:00   1\n",
                            "28     138.0    -7  -5.0  1022.0    6.25   2   0  2010-01-02 04:00:00   1\n",
                            "...      ...   ...   ...     ...     ...  ..  ..                  ...  ..\n",
                            "43819    8.0   -23  -2.0  1034.0  231.97   0   0  2014-12-31 19:00:00   1\n",
                            "43820   10.0   -22  -3.0  1034.0  237.78   0   0  2014-12-31 20:00:00   1\n",
                            "43821   10.0   -22  -3.0  1034.0  242.70   0   0  2014-12-31 21:00:00   1\n",
                            "43822    8.0   -22  -4.0  1034.0  246.72   0   0  2014-12-31 22:00:00   1\n",
                            "43823   12.0   -21  -3.0  1034.0  249.85   0   0  2014-12-31 23:00:00   1\n",
                            "\n",
                            "[41757 rows x 9 columns]"
                        ]
                    },
                    "execution_count": 13,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "data_full_pandas"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "We then **load the data into the getML engine**. We begin by setting a project."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Launching ./getML --allow-push-notifications=true --allow-remote-ips=true --home-directory=/home/user --in-memory=true --install=false --launch-browser=true --log=false --token=token in /home/user/.getML/getml-1.5.0-x64-linux...\n",
                        "Launched the getML Engine. The log output will be stored in /home/user/.getML/logs/20240913121328.log.\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Connected to project <span style=\"color: #008000; text-decoration-color: #008000\">'air_pollution'</span>.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Connected to project \u001b[32m'air_pollution'\u001b[0m.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "getml.engine.launch(allow_remote_ips=True, token=\"token\")\n",
                "getml.engine.set_project(\"air_pollution\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {},
            "outputs": [],
            "source": [
                "df_full = getml.data.DataFrame.from_pandas(data_full_pandas, name=\"full\")\n",
                "df_full[\"date\"] = df_full[\"date\"].as_ts()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "We need to **assign roles** to the columns, such as defining the target column."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  table {\n",
                            "    font-family: Helvetica, sans-serif;\n",
                            "  }\n",
                            "\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right !important;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.sub-header {\n",
                            "    font-weight: normal;\n",
                            "    font-style: italic;\n",
                            "  }\n",
                            "  .join_key,\n",
                            "  .numerical,\n",
                            "  .target,\n",
                            "  .unused_float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "\n",
                            "  .char-align {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  span.left {\n",
                            "    text-align: right;\n",
                            "    width: 3em;\n",
                            "  }\n",
                            "  span.right {\n",
                            "    float: right;\n",
                            "    text-align: left;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th> name</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"time_stamp\">                       date</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"target\"> pm2.5</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"numerical\">     DEWP</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"numerical\">     TEMP</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"numerical\">     PRES</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"numerical\">      Iws</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"numerical\">       Is</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"numerical\">       Ir</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"unused_float\">          id</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header\"> role</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header time_stamp\">                 time_stamp</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header target\">target</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">numerical</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">numerical</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">numerical</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">numerical</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">numerical</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">numerical</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header unused_float\">unused_float</th>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header\"> unit</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header time_stamp\">time stamp, comparison only</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header target\">      </th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">         </th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">         </th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">         </th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">         </th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">         </th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header numerical\">         </th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header unused_float\">            </th>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2010-01-02</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">129</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-16</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-4</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1020</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.79</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2010-01-02 01:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">148</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-15</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-4</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1020</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">2</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.68</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>2</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2010-01-02 02:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">159</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-11</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-5</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1021</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">3</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.57</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>3</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2010-01-02 03:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">181</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-7</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-5</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1022</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">5</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.36</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>4</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2010-01-02 04:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">138</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-7</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-5</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1022</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">6</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.25</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">2</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th></th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">...</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>41752</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2014-12-31 19:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">8</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-23</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-2</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1034</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">231</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.97</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>41753</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2014-12-31 20:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">10</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-22</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-3</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1034</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">237</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.78</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>41754</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2014-12-31 21:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">10</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-22</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-3</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1034</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">242</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.7</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>41755</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2014-12-31 22:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">8</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-22</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-4</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1034</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">246</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.72</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>41756</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2014-12-31 23:00:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">12</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-21</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">-3</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">1034</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "            \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">249</span\n",
                            "              ><span class=\"right\" style=\"width: 2ch\"\n",
                            "                >.85</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align numerical\">\n",
                            "              <span class=\"left\">0</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align unused_float\">\n",
                            "              <span class=\"left\">1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "\n",
                            "  <p>\n",
                            "    41757 rows x 9 columns<br />\n",
                            "    memory usage: 3.01 MB<br />\n",
                            "    name: full<br />\n",
                            "    type: getml.DataFrame<br />\n",
                            "    \n",
                            "  </p>\n"
                        ],
                        "text/plain": [
                            " name                          date    pm2.5        DEWP        TEMP        PRES         Iws          Is          Ir\n",
                            " role                    time_stamp   target   numerical   numerical   numerical   numerical   numerical   numerical\n",
                            " unit   time stamp, comparison only                                                                                 \n",
                            "    0                    2010-01-02      129        -16          -4         1020        1.79           0           0\n",
                            "    1           2010-01-02 01:00:00      148        -15          -4         1020        2.68           0           0\n",
                            "    2           2010-01-02 02:00:00      159        -11          -5         1021        3.57           0           0\n",
                            "    3           2010-01-02 03:00:00      181         -7          -5         1022        5.36           1           0\n",
                            "    4           2010-01-02 04:00:00      138         -7          -5         1022        6.25           2           0\n",
                            "                                ...      ...         ...         ...         ...       ...           ...         ...\n",
                            "41752           2014-12-31 19:00:00        8        -23          -2         1034      231.97           0           0\n",
                            "41753           2014-12-31 20:00:00       10        -22          -3         1034      237.78           0           0\n",
                            "41754           2014-12-31 21:00:00       10        -22          -3         1034      242.7            0           0\n",
                            "41755           2014-12-31 22:00:00        8        -22          -4         1034      246.72           0           0\n",
                            "41756           2014-12-31 23:00:00       12        -21          -3         1034      249.85           0           0\n",
                            "\n",
                            " name             id\n",
                            " role   unused_float\n",
                            " unit               \n",
                            "    0              1\n",
                            "    1              1\n",
                            "    2              1\n",
                            "    3              1\n",
                            "    4              1\n",
                            "                 ...\n",
                            "41752              1\n",
                            "41753              1\n",
                            "41754              1\n",
                            "41755              1\n",
                            "41756              1\n",
                            "\n",
                            "\n",
                            "41757 rows x 9 columns\n",
                            "memory usage: 3.01 MB\n",
                            "type: getml.DataFrame"
                        ]
                    },
                    "execution_count": 16,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "df_full.set_role([\"date\"], getml.data.roles.time_stamp)\n",
                "df_full.set_role([\"pm2.5\"], getml.data.roles.target)\n",
                "df_full.set_role(\n",
                "    [\"DEWP\", \"TEMP\", \"PRES\", \"Iws\", \"Is\", \"Ir\"], getml.data.roles.numerical\n",
                ")\n",
                "df_full"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th.sub-header {\n",
                            "    font-weight: normal;\n",
                            "    font-style: italic;\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right !important;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.numerical {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.numerical {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "\n",
                            "  td.char-align {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  span.left {\n",
                            "    text-align: right;\n",
                            "    width: 3em;\n",
                            "  }\n",
                            "  span.right {\n",
                            "    float: right;\n",
                            "    text-align: left;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th>     </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th>     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>2</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>3</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>4</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th></th>\n",
                            "        \n",
                            "          \n",
                            "            <td>...</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "\n",
                            "  <p>\n",
                            "    41757 rows<br />\n",
                            "    \n",
                            "    type: StringColumnView<br />\n",
                            "    \n",
                            "  </p>\n"
                        ],
                        "text/plain": [
                            "             \n",
                            "    0   train\n",
                            "    1   train\n",
                            "    2   train\n",
                            "    3   train\n",
                            "    4   train\n",
                            "        ...  \n",
                            "\n",
                            "\n",
                            "41757 rows\n",
                            "type: StringColumnView"
                        ]
                    },
                    "execution_count": 17,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "split = getml.data.split.time(\n",
                "    population=df_full, time_stamp=\"date\", test=getml.data.time.datetime(2014, 1, 1)\n",
                ")\n",
                "split"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 2. Predictive modeling"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.1 Propositionalization with getML's FastProp"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<span style='font-size: 1.2rem; font-weight: 500;'>data model</span>\n",
                            "            <div style='margin-top: 15px; margin-bottom: 5px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>diagram</div>\n",
                            "            <div style=\"height:100px;width:660px;position:relative;\"><svg height=\"90\" width=\"650\"><rect y=\"0\" x=\"0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">full</text><rect x=\"51\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"67.0\" y1=\"10\" x2=\"67.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"83.0\" y1=\"10\" x2=\"83.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"26.0\" x2=\"99\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"42.0\" x2=\"99\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><rect y=\"0\" x=\"500\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"575.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"551\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"567.0\" y1=\"10\" x2=\"567.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"583.0\" y1=\"10\" x2=\"583.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"26.0\" x2=\"599\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"42.0\" x2=\"599\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"150\" y1=\"43.0\" x2=\"490\" y2=\"43.0\" style=\"stroke:#808080;;stroke-width:4\" /><polygon points=\"500, 43.0 490, 37.0 490, 49.0 \" style=\"fill:#808080;;stroke-width:0;\" /><rect y=\"10.0\" x=\"249.0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"70\" style=\"fill:#6829c2;stroke-width:0;\" /><text dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\"><tspan y=\"40.0\" x=\"324.0\" font-size=\"7pt\" >date &lt;= date</tspan><tspan y=\"50.0\" x=\"324.0\" font-size=\"7pt\" >Memory: 1.0 days</tspan></text></svg></div>\n",
                            "            </div>\n",
                            "\n",
                            "            <div style='margin-top: 15px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>staging</div>\n",
                            "            <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">data frames</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">staging table              </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_1</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">full</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">FULL__STAGING_TABLE_2</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "            </div>\n",
                            "            \n",
                            "<span style='font-size: 1.2rem; font-weight: 500;'>container</span>\n",
                            "<div style='margin-top: 15px;'>\n",
                            "<div style='float: left; margin-right: 50px;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>population</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">subset</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">rows   </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">full</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">unknown</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">full</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">unknown</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "<div style='float: left;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>peripheral</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">full</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">41757</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">DataFrame</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "data model\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "    joins:\n",
                            "    - right: 'full'\n",
                            "      time_stamps: (population.date, full.date)\n",
                            "      relationship: 'many-to-many'\n",
                            "      memory: 86400.0\n",
                            "      lagged_targets: False\n",
                            "\n",
                            "  full:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "\n",
                            "container\n",
                            "\n",
                            "  population\n",
                            "      subset   name   rows      type\n",
                            "  0   test     full   unknown   View\n",
                            "  1   train    full   unknown   View\n",
                            "\n",
                            "  peripheral\n",
                            "      name    rows   type     \n",
                            "  0   full   41757   DataFrame"
                        ]
                    },
                    "execution_count": 18,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "time_series = getml.data.TimeSeries(\n",
                "    population=df_full,\n",
                "    alias=\"population\",\n",
                "    split=split,\n",
                "    time_stamps=\"date\",\n",
                "    memory=getml.data.time.days(1),\n",
                ")\n",
                "\n",
                "time_series"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['full'],\n",
                            "         predictors=[],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['memory: 1d', 'simple features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['full'],\n",
                            "         predictors=[],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['memory: 1d', 'simple features'])"
                        ]
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "fast_prop = getml.feature_learning.FastProp(\n",
                "    loss_function=getml.feature_learning.loss_functions.SquareLoss,\n",
                "    num_threads=1,\n",
                "    aggregation=getml.feature_learning.FastProp.agg_sets.All,\n",
                ")\n",
                "\n",
                "pipe_fp_fl = getml.pipeline.Pipeline(\n",
                "    tags=[\"memory: 1d\", \"simple features\"],\n",
                "    data_model=time_series.data_model,\n",
                "    feature_learners=[fast_prop],\n",
                ")\n",
                "\n",
                "pipe_fp_fl"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K⠼ Checking... ━━━━━━━━━━━━━━━━━━━━                      50% • 00:01"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:02\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe_fp_fl.check(time_series.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {},
            "outputs": [],
            "source": [
                "benchmark = Benchmark()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Trying 331 features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:08\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:08.079324.\n",
                        "\n",
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:04\n",
                        "\u001b[?25h"
                    ]
                }
            ],
            "source": [
                "with benchmark(\"fastprop\"):\n",
                "    pipe_fp_fl.fit(time_series.train)\n",
                "    fastprop_train = pipe_fp_fl.transform(time_series.train, df_name=\"fastprop_train\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                }
            ],
            "source": [
                "fastprop_test = pipe_fp_fl.transform(time_series.test, df_name=\"fastprop_test\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {},
            "outputs": [],
            "source": [
                "predictor = getml.predictors.XGBoostRegressor()\n",
                "\n",
                "pipe_fp_pr = getml.pipeline.Pipeline(\n",
                "    tags=[\"prediction\", \"fastprop\"], predictors=[predictor]\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 25,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:28\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:28.352024.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['prediction', 'fastprop'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['prediction', 'fastprop'])"
                        ]
                    },
                    "execution_count": 25,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_fp_pr.fit(fastprop_train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 26,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 12:14:20</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">fastprop_train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">38.3028</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">55.2472</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6438</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 12:14:20</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">fastprop_test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">44.2502</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">63.4168</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.5462</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used         target        mae       rmse   rsquared\n",
                            "0   2024-09-13 12:14:20   fastprop_train   pm2.5     38.3028    55.2472     0.6438\n",
                            "1   2024-09-13 12:14:20   fastprop_test    pm2.5     44.2502    63.4168     0.5462"
                        ]
                    },
                    "execution_count": 26,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_fp_pr.score(fastprop_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.2 Using featuretools\n",
                "\n",
                "To make things a bit easier, we have written a high-level wrapper around featuretools which we placed in a separate module (`utils`)."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 27,
            "metadata": {},
            "outputs": [],
            "source": [
                "ft_builder = FTTimeSeriesBuilder(\n",
                "    num_features=200,\n",
                "    horizon=pd.Timedelta(days=0),\n",
                "    memory=pd.Timedelta(days=1),\n",
                "    column_id=\"id\",\n",
                "    time_stamp=\"date\",\n",
                "    target=\"pm2.5\",\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 28,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "featuretools: Trying features...\n"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Selecting the best out of 298 features...\n",
                        "Time taken: 0h:35m:42.348872\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "with benchmark(\"featuretools\"):\n",
                "    featuretools_training = ft_builder.fit(data_train_pandas)\n",
                "\n",
                "featuretools_test = ft_builder.transform(data_test_pandas)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 29,
            "metadata": {},
            "outputs": [],
            "source": [
                "df_featuretools_training = getml.data.DataFrame.from_pandas(\n",
                "    featuretools_training, name=\"featuretools_training\"\n",
                ")\n",
                "df_featuretools_test = getml.data.DataFrame.from_pandas(\n",
                "    featuretools_test, name=\"featuretools_test\"\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 30,
            "metadata": {},
            "outputs": [],
            "source": [
                "def set_roles_featuretools(df):\n",
                "    df[\"date\"] = df[\"date\"].as_ts()\n",
                "    df.set_role([\"pm2.5\"], getml.data.roles.target)\n",
                "    df.set_role([\"date\"], getml.data.roles.time_stamp)\n",
                "    df.set_role(df.roles.unused, getml.data.roles.numerical)\n",
                "    df.set_role([\"id\"], getml.data.roles.unused_float)\n",
                "    return df\n",
                "\n",
                "\n",
                "df_featuretools_training = set_roles_featuretools(df_featuretools_training)\n",
                "df_featuretools_test = set_roles_featuretools(df_featuretools_test)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 31,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['featuretools', 'memory: 1d'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['featuretools', 'memory: 1d'])"
                        ]
                    },
                    "execution_count": 31,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "predictor = getml.predictors.XGBoostRegressor()\n",
                "\n",
                "pipe_ft_pr = getml.pipeline.Pipeline(\n",
                "    tags=[\"featuretools\", \"memory: 1d\"], predictors=[predictor]\n",
                ")\n",
                "\n",
                "pipe_ft_pr"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 32,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K⠦ Checking...                                            0% • 00:00"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe_ft_pr.check(df_featuretools_training)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 33,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:10\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:10.205240.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['featuretools', 'memory: 1d'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['featuretools', 'memory: 1d'])"
                        ]
                    },
                    "execution_count": 33,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_ft_pr.fit(df_featuretools_training)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 34,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used             </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 12:58:20</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">featuretools_training</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">38.277</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">54.8781</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6506</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 12:58:20</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">featuretools_test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">43.9151</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">62.5672</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.5594</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used                target        mae       rmse   rsquared\n",
                            "0   2024-09-13 12:58:20   featuretools_training   pm2.5     38.277     54.8781     0.6506\n",
                            "1   2024-09-13 12:58:20   featuretools_test       pm2.5     43.9151    62.5672     0.5594"
                        ]
                    },
                    "execution_count": 34,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_ft_pr.score(df_featuretools_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.3 Using tsfresh\n",
                "\n",
                "tsfresh is a rather low-level library. To make things a bit easier, we have written a high-level wrapper which we placed in a separate module (`utils`).\n",
                "\n",
                "To limit the memory consumption, we undertake the following steps:\n",
                "\n",
                "- We limit ourselves to a memory of 1 day from any point in time. This is necessary, because tsfresh duplicates records for every time stamp. That means that looking back 7 days instead of one day, the memory consumption would be  seven times as high.\n",
                "- We extract only tsfresh's **MinimalFCParameters** and **IndexBasedFCParameters** (the latter is a superset of **TimeBasedFCParameters**).\n",
                "\n",
                "In order to make sure that tsfresh's features can be compared to getML's features, we also do the following:\n",
                "\n",
                "- We apply tsfresh's built-in feature selection algorithm.\n",
                "- Of the remaining features, we only keep the 40 features most correlated with the target (in terms of the absolute value of the correlation).\n",
                "- We add the original columns as additional features.\n"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 35,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>pm2.5</th>\n",
                            "      <th>DEWP</th>\n",
                            "      <th>TEMP</th>\n",
                            "      <th>PRES</th>\n",
                            "      <th>Iws</th>\n",
                            "      <th>Is</th>\n",
                            "      <th>Ir</th>\n",
                            "      <th>date</th>\n",
                            "      <th>id</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>24</th>\n",
                            "      <td>129.0</td>\n",
                            "      <td>-16</td>\n",
                            "      <td>-4.0</td>\n",
                            "      <td>1020.0</td>\n",
                            "      <td>1.79</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 00:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>25</th>\n",
                            "      <td>148.0</td>\n",
                            "      <td>-15</td>\n",
                            "      <td>-4.0</td>\n",
                            "      <td>1020.0</td>\n",
                            "      <td>2.68</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 01:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>26</th>\n",
                            "      <td>159.0</td>\n",
                            "      <td>-11</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1021.0</td>\n",
                            "      <td>3.57</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 02:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>27</th>\n",
                            "      <td>181.0</td>\n",
                            "      <td>-7</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1022.0</td>\n",
                            "      <td>5.36</td>\n",
                            "      <td>1</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 03:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>28</th>\n",
                            "      <td>138.0</td>\n",
                            "      <td>-7</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1022.0</td>\n",
                            "      <td>6.25</td>\n",
                            "      <td>2</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2010-01-02 04:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>...</th>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>35059</th>\n",
                            "      <td>22.0</td>\n",
                            "      <td>-19</td>\n",
                            "      <td>7.0</td>\n",
                            "      <td>1013.0</td>\n",
                            "      <td>114.87</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2013-12-31 19:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>35060</th>\n",
                            "      <td>18.0</td>\n",
                            "      <td>-21</td>\n",
                            "      <td>7.0</td>\n",
                            "      <td>1014.0</td>\n",
                            "      <td>119.79</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2013-12-31 20:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>35061</th>\n",
                            "      <td>23.0</td>\n",
                            "      <td>-21</td>\n",
                            "      <td>7.0</td>\n",
                            "      <td>1014.0</td>\n",
                            "      <td>125.60</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2013-12-31 21:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>35062</th>\n",
                            "      <td>20.0</td>\n",
                            "      <td>-21</td>\n",
                            "      <td>6.0</td>\n",
                            "      <td>1014.0</td>\n",
                            "      <td>130.52</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2013-12-31 22:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>35063</th>\n",
                            "      <td>23.0</td>\n",
                            "      <td>-20</td>\n",
                            "      <td>7.0</td>\n",
                            "      <td>1014.0</td>\n",
                            "      <td>137.67</td>\n",
                            "      <td>0</td>\n",
                            "      <td>0</td>\n",
                            "      <td>2013-12-31 23:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>33096 rows × 9 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "       pm2.5  DEWP  TEMP    PRES     Iws  Is  Ir                 date  id\n",
                            "24     129.0   -16  -4.0  1020.0    1.79   0   0  2010-01-02 00:00:00   1\n",
                            "25     148.0   -15  -4.0  1020.0    2.68   0   0  2010-01-02 01:00:00   1\n",
                            "26     159.0   -11  -5.0  1021.0    3.57   0   0  2010-01-02 02:00:00   1\n",
                            "27     181.0    -7  -5.0  1022.0    5.36   1   0  2010-01-02 03:00:00   1\n",
                            "28     138.0    -7  -5.0  1022.0    6.25   2   0  2010-01-02 04:00:00   1\n",
                            "...      ...   ...   ...     ...     ...  ..  ..                  ...  ..\n",
                            "35059   22.0   -19   7.0  1013.0  114.87   0   0  2013-12-31 19:00:00   1\n",
                            "35060   18.0   -21   7.0  1014.0  119.79   0   0  2013-12-31 20:00:00   1\n",
                            "35061   23.0   -21   7.0  1014.0  125.60   0   0  2013-12-31 21:00:00   1\n",
                            "35062   20.0   -21   6.0  1014.0  130.52   0   0  2013-12-31 22:00:00   1\n",
                            "35063   23.0   -20   7.0  1014.0  137.67   0   0  2013-12-31 23:00:00   1\n",
                            "\n",
                            "[33096 rows x 9 columns]"
                        ]
                    },
                    "execution_count": 35,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "data_train_pandas"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": [
                        "Rolling: 100%|██████████| 40/40 [00:25<00:00,  1.55it/s]\n",
                        "Feature Extraction: 100%|██████████| 40/40 [00:23<00:00,  1.67it/s]\n",
                        "Feature Extraction: 100%|██████████| 40/40 [00:23<00:00,  1.67it/s]\n"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Selecting the best out of 78 features...\n",
                        "Time taken: 0h:1m:21.479083\n",
                        "\n"
                    ]
                },
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": [
                        "Rolling: 100%|██████████| 40/40 [00:03<00:00, 13.11it/s]\n",
                        "Feature Extraction: 100%|██████████| 40/40 [00:06<00:00,  6.33it/s]\n",
                        "Feature Extraction: 100%|██████████| 40/40 [00:06<00:00,  6.12it/s]\n"
                    ]
                }
            ],
            "source": [
                "tsfresh_builder = TSFreshBuilder(\n",
                "    num_features=200, memory=24, column_id=\"id\", time_stamp=\"date\", target=\"pm2.5\"\n",
                ")\n",
                "\n",
                "with benchmark(\"tsfresh\"):\n",
                "    tsfresh_training = tsfresh_builder.fit(data_train_pandas)\n",
                "\n",
                "tsfresh_test = tsfresh_builder.transform(data_test_pandas)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "tsfresh does not contain built-in machine learning algorithms. In order to ensure a fair comparison, we use the exact same machine learning algorithm we have also used for getML: An XGBoost regressor with all hyperparameters set to their default value.\n",
                "\n",
                "In order to do so, we load the tsfresh features into the getML engine."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 37,
            "metadata": {},
            "outputs": [],
            "source": [
                "df_tsfresh_training = getml.data.DataFrame.from_pandas(\n",
                "    tsfresh_training, name=\"tsfresh_training\"\n",
                ")\n",
                "df_tsfresh_test = getml.data.DataFrame.from_pandas(tsfresh_test, name=\"tsfresh_test\")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "As usual, we need to set roles:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 38,
            "metadata": {},
            "outputs": [],
            "source": [
                "def set_roles_tsfresh(df):\n",
                "    df[\"date\"] = df[\"date\"].as_ts()\n",
                "    df.set_role([\"pm2.5\"], getml.data.roles.target)\n",
                "    df.set_role([\"date\"], getml.data.roles.time_stamp)\n",
                "    df.set_role(df.roles.unused, getml.data.roles.numerical)\n",
                "    df.set_role([\"id\"], getml.data.roles.unused_float)\n",
                "    return df\n",
                "\n",
                "\n",
                "df_tsfresh_training = set_roles_tsfresh(df_tsfresh_training)\n",
                "df_tsfresh_test = set_roles_tsfresh(df_tsfresh_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "In this case, our pipeline is very simple. It only consists of a single XGBoostRegressor."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 39,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['tsfresh', 'memory: 1d'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['tsfresh', 'memory: 1d'])"
                        ]
                    },
                    "execution_count": 39,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "predictor = getml.predictors.XGBoostRegressor()\n",
                "\n",
                "pipe_tsf_pr = getml.pipeline.Pipeline(\n",
                "    tags=[\"tsfresh\", \"memory: 1d\"], predictors=[predictor]\n",
                ")\n",
                "\n",
                "pipe_tsf_pr"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 40,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe_tsf_pr.check(df_tsfresh_training)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 41,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:05\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:05.645952.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['tsfresh', 'memory: 1d'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['tsfresh', 'memory: 1d'])"
                        ]
                    },
                    "execution_count": 41,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_tsf_pr.fit(df_tsfresh_training)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 42,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used        </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 13:00:08</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">tsfresh_training</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">40.8917</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">57.9517</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6099</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 13:00:08</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">tsfresh_test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">47.1106</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">66.6</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.5015</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used           target        mae       rmse   rsquared\n",
                            "0   2024-09-13 13:00:08   tsfresh_training   pm2.5     40.8917    57.9517     0.6099\n",
                            "1   2024-09-13 13:00:08   tsfresh_test       pm2.5     47.1106    66.6        0.5015"
                        ]
                    },
                    "execution_count": 42,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_tsf_pr.score(df_tsfresh_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 3. Comparison"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 43,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>runtime</th>\n",
                            "      <th>num_features</th>\n",
                            "      <th>features_per_second</th>\n",
                            "      <th>normalized_runtime</th>\n",
                            "      <th>normalized_runtime_per_feature</th>\n",
                            "      <th>mae</th>\n",
                            "      <th>rmse</th>\n",
                            "      <th>rsquared</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>getML: FastProp</th>\n",
                            "      <td>0 days 00:00:13.433118</td>\n",
                            "      <td>289</td>\n",
                            "      <td>21.514167</td>\n",
                            "      <td>1.000000</td>\n",
                            "      <td>1.000000</td>\n",
                            "      <td>44.250202</td>\n",
                            "      <td>63.416780</td>\n",
                            "      <td>0.546191</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>featuretools</th>\n",
                            "      <td>0 days 00:35:42.350566</td>\n",
                            "      <td>114</td>\n",
                            "      <td>0.053213</td>\n",
                            "      <td>159.482747</td>\n",
                            "      <td>404.306039</td>\n",
                            "      <td>43.915071</td>\n",
                            "      <td>62.567175</td>\n",
                            "      <td>0.559369</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>tsfresh</th>\n",
                            "      <td>0 days 00:01:21.479423</td>\n",
                            "      <td>72</td>\n",
                            "      <td>0.883658</td>\n",
                            "      <td>6.065563</td>\n",
                            "      <td>24.346701</td>\n",
                            "      <td>47.110594</td>\n",
                            "      <td>66.599982</td>\n",
                            "      <td>0.501524</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "                               runtime  num_features  features_per_second  \\\n",
                            "getML: FastProp 0 days 00:00:13.433118           289            21.514167   \n",
                            "featuretools    0 days 00:35:42.350566           114             0.053213   \n",
                            "tsfresh         0 days 00:01:21.479423            72             0.883658   \n",
                            "\n",
                            "                 normalized_runtime  normalized_runtime_per_feature  \\\n",
                            "getML: FastProp            1.000000                        1.000000   \n",
                            "featuretools             159.482747                      404.306039   \n",
                            "tsfresh                    6.065563                       24.346701   \n",
                            "\n",
                            "                       mae       rmse  rsquared  \n",
                            "getML: FastProp  44.250202  63.416780  0.546191  \n",
                            "featuretools     43.915071  62.567175  0.559369  \n",
                            "tsfresh          47.110594  66.599982  0.501524  "
                        ]
                    },
                    "execution_count": 43,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "num_features = dict(\n",
                "    fastprop=289,\n",
                "    featuretools=114,\n",
                "    tsfresh=72,\n",
                ")\n",
                "\n",
                "runtime_per_feature = [\n",
                "    benchmark.runtimes[\"fastprop\"] / num_features[\"fastprop\"],\n",
                "    benchmark.runtimes[\"featuretools\"] / num_features[\"featuretools\"],\n",
                "    benchmark.runtimes[\"tsfresh\"] / num_features[\"tsfresh\"],\n",
                "]\n",
                "\n",
                "features_per_second = [1.0 / r.total_seconds() for r in runtime_per_feature]\n",
                "\n",
                "normalized_runtime_per_feature = [\n",
                "    r / runtime_per_feature[0] for r in runtime_per_feature\n",
                "]\n",
                "\n",
                "comparison = pd.DataFrame(\n",
                "    dict(\n",
                "        runtime=[\n",
                "            benchmark.runtimes[\"fastprop\"],\n",
                "            benchmark.runtimes[\"featuretools\"],\n",
                "            benchmark.runtimes[\"tsfresh\"],\n",
                "        ],\n",
                "        num_features=num_features.values(),\n",
                "        features_per_second=features_per_second,\n",
                "        normalized_runtime=[\n",
                "            1,\n",
                "            benchmark.runtimes[\"featuretools\"] / benchmark.runtimes[\"fastprop\"],\n",
                "            benchmark.runtimes[\"tsfresh\"] / benchmark.runtimes[\"fastprop\"],\n",
                "        ],\n",
                "        normalized_runtime_per_feature=normalized_runtime_per_feature,\n",
                "        mae=[pipe_fp_pr.mae, pipe_ft_pr.mae, pipe_tsf_pr.mae],\n",
                "        rmse=[pipe_fp_pr.rmse, pipe_ft_pr.rmse, pipe_tsf_pr.rmse],\n",
                "        rsquared=[pipe_fp_pr.rsquared, pipe_ft_pr.rsquared, pipe_tsf_pr.rsquared],\n",
                "    )\n",
                ")\n",
                "\n",
                "comparison.index = [\"getML: FastProp\", \"featuretools\", \"tsfresh\"]\n",
                "\n",
                "comparison"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 44,
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "# export for further use\n",
                "comparison.to_csv(\"comparisons/air_pollution.csv\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "getml.engine.shutdown()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 4. Conclusion\n",
                "\n",
                "We have compared getML's feature learning algorithms to tsfresh's brute-force feature engineering approaches on a data set related to air pollution in China. We found that getML significantly outperforms featuretools and tsfresh. These results are consistent with the view that feature learning can yield significant improvements over simple propositionalization approaches.\n",
                "\n",
                "However, there are other datasets on which simple propositionalization performs well. Our suggestion is therefore to think of algorithms like `FastProp` and `RelMT` as tools in a toolbox. If a simple tool like `FastProp` gets the job done, then use that. But when you need more advanced approaches, like `RelMT`, you should have them at your disposal as well.\n",
                "\n",
                "You are encouraged to reproduce these results."
            ]
        }
    ],
    "metadata": {
        "celltoolbar": "Tags",
        "kernelspec": {
            "display_name": "Python 3 (ipykernel)",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.11.9"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}