DigiWell_UDE.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "efd013e1",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center> <h1>Auto-Completing Models to Uncover Missing Physics:\n",
    "A Deep Dive Tutorial Into Universal Differential Equations\n",
    "</h1> </center>  \n",
    "\n",
    "<center> <h3>Vinicius Viena Santana\n",
    "    </h3>\n",
    "</center>  \n",
    "\n",
    "<center> <h3>Norwegian University of Science and Technology, Department of Chemical Engineering, Idelfonso´s lab </h3></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1141ba67",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Background**\n",
    "- Bachelor and Master's degree in industrial engineering from the Federal University of Bahia 🎓\n",
    "- Ph.D. at the University of Porto, Portugal (Supervised by Idelfonso Nogueira and Chris Rackauckas) 📚"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e30319a5",
   "metadata": {},
   "source": [
    "**About me**\n",
    "\n",
    "Enthusiastic about coding and the Julia programming language ❤️"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e69954b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Research**\n",
    "- Currently a researcher at NTNU, Norway (PSE Group, Idelfonso's lab) 🇳🇴\n",
    "- Research focus: Scientific Machine Learning and AI applied to industrial and chemical engineering 📈"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29c0c2bd",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Modeling in Natural Sciences\n",
    "\n",
    "Building simple and powerful models is time consuming and expensive 💰"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f934325",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Models compete for acceptance. Should we favor rigor or simplicity, exactness or usefulness - the 10 USD or 1000 USD model? "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a43580a2",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Always start by trying the simplest model and then only add complexity to the extent needed. This is the 10 USD approach (Levenspiel, 2002, Modeling in Chemical Engineering)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbdc509c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* How can we do minimal extensions that would allow our **SIMPLE** model to fit the data?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "915ddbe6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Universal Differential Equations [Rackauckas et al. (2020)](https://arxiv.org/abs/2001.04385)\n",
    "\n",
    "~ A small black box into your simple model ~"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09b0c838",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* The universal approximator aim to reduce mechanistic model non-explained residuals through small mechanistic extensions.\n",
    "\n",
    "\n",
    "<center><img src=\"diagram-20231023.svg\" width=\"700\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7298a913",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Case Study: UDE in Separation Engineering -- Fixed Bed Adsorption"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78d2ea3a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* What is Fixed Bed Adsorption?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c877b9f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<center><img src=\"diagram-20231030.svg\" width=\"700\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f017359b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* We need conservation equations to describe the transport of chemicals inside of it. \n",
    "For columns, writing it down results in Partial Differential Equations."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df0d37ce",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The Zero Length Column"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62aa23e7",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* If the column size is sufficiently small, we can use the zero-length column approximation. This simplifies the kinetics estimation, making it easier than with larger column because mass balance is reduced to an ODE instead of a PDE."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b837e5a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$\\textrm{Fluid phase mass balance}$\n",
    "\n",
    "$V_f \\frac{dc}{dt} + M_s \\frac{dq}{dt} = Q(c_{\\text{in}} - c)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8deb8033",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$\\textrm{Solid phase mass balance}$\n",
    "\n",
    "$\\frac{dq}{dt} = f(q, q^*)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ec42ee4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$f(q, q^*) = k_1(q^* - q), k_2(q^* - q)^2, k_3(q^{*2} - q^2)/q,...?$\n",
    "\n",
    "* Why not $f(q, q^*) = \\textrm{ANN}(q^*, q,\\theta)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91852a15",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "q* is typically expressed as a function of c: $q^* = Q\\frac{Kc}{(1 + Kc)}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75e517c7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Hands-on UDE implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1928015",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Loading and Visualizing Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d9f88461",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m\u001b[1m  Activating\u001b[22m\u001b[39m project at `C:\\Users\\Vinic\\.julia\\environments\\v1.9`\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Plots.PlotlyBackend()"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Pkg\n",
    "Pkg.activate() #Activating enviroment\n",
    "using Plots, LaTeXStrings, DelimitedFiles\n",
    "plotly() #Plot backend"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "01757aa6",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "plot_font = \"Computer Modern\"\n",
    "default(fontfamily=plot_font,\n",
    "        linewidth=2, framestyle=:box, label=nothing, grid=false)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "9cb46a89",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.914342738407699,
          "title": "",
          "x": 0.9934383202099737,
          "y": 0.5329861111111112
         },
         "legendgroup": "Exp",
         "marker": {
          "color": "rgba(0, 154, 250, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Exp",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.1149144414942044,
          0.453593327427082,
          0.17166903204340475,
          0.16434298525027757,
          0.43082886105468254,
          0.39036313080466295,
          0.1498263562138032,
          0.6049853300852165,
          0.7288340742910437,
          0.7208278900541426,
          0.9725557011832456,
          1.0645132432308824,
          1.605832870402756,
          1.9623017325357597,
          2.488345947495106,
          2.9372823345460555,
          3.3004647312077497,
          4.44026467007553,
          6.014621388907804,
          7.332105354195911,
          8.600968893100676,
          9.879855941342193,
          11.70106015407636,
          12.820166371191963,
          14.011211986433725,
          15.036464869109007,
          16.06275295941086,
          16.493576791018565,
          16.929248776222778,
          17.310492828215327,
          17.808197157388648,
          18.412709295322795,
          18.90649223578161,
          19.02591785290369,
          19.112996811420775,
          18.883545295888464,
          19.46450909951882,
          19.24859182601517,
          19.527825543706353,
          20.0553701728416,
          19.714848868287742,
          19.63114088016073,
          19.53588580560029,
          19.85638399603698,
          19.803292061991808,
          19.732583577342563,
          19.847534725209762,
          19.71101197633774,
          19.96119154981576,
          20.094516579726115,
          19.74853945973233
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.914342738407699,
          "title": "",
          "x": 0.9934383202099737,
          "y": 0.5329861111111112
         },
         "legendgroup": "Inlet c",
         "line": {
          "color": "rgba(227, 111, 71, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Inlet c",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 400,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1,
         "xanchor": "auto",
         "y": 1,
         "yanchor": "auto"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 600,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.061654272382618835,
          0.9934383202099737
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.07581474190726165,
          0.9901574803149606
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.48447362265275373,
          20.693904643873072
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "5",
          "10",
          "15",
          "20"
         ],
         "tickvals": [
          0,
          5,
          10,
          15,
          20
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"364a91b8-240c-4a6c-8f84-95d492d893e0\" style=\"width:600px;height:400px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('364a91b8-240c-4a6c-8f84-95d492d893e0', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5329861111111112,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.914342738407699,\n",
       "            \"x\": 0.9934383202099737\n",
       "        },\n",
       "        \"name\": \"Exp\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Exp\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.1149144414942044,\n",
       "            0.453593327427082,\n",
       "            0.17166903204340475,\n",
       "            0.16434298525027757,\n",
       "            0.43082886105468254,\n",
       "            0.39036313080466295,\n",
       "            0.1498263562138032,\n",
       "            0.6049853300852165,\n",
       "            0.7288340742910437,\n",
       "            0.7208278900541426,\n",
       "            0.9725557011832456,\n",
       "            1.0645132432308824,\n",
       "            1.605832870402756,\n",
       "            1.9623017325357597,\n",
       "            2.488345947495106,\n",
       "            2.9372823345460555,\n",
       "            3.3004647312077497,\n",
       "            4.44026467007553,\n",
       "            6.014621388907804,\n",
       "            7.332105354195911,\n",
       "            8.600968893100676,\n",
       "            9.879855941342193,\n",
       "            11.70106015407636,\n",
       "            12.820166371191963,\n",
       "            14.011211986433725,\n",
       "            15.036464869109007,\n",
       "            16.06275295941086,\n",
       "            16.493576791018565,\n",
       "            16.929248776222778,\n",
       "            17.310492828215327,\n",
       "            17.808197157388648,\n",
       "            18.412709295322795,\n",
       "            18.90649223578161,\n",
       "            19.02591785290369,\n",
       "            19.112996811420775,\n",
       "            18.883545295888464,\n",
       "            19.46450909951882,\n",
       "            19.24859182601517,\n",
       "            19.527825543706353,\n",
       "            20.0553701728416,\n",
       "            19.714848868287742,\n",
       "            19.63114088016073,\n",
       "            19.53588580560029,\n",
       "            19.85638399603698,\n",
       "            19.803292061991808,\n",
       "            19.732583577342563,\n",
       "            19.847534725209762,\n",
       "            19.71101197633774,\n",
       "            19.96119154981576,\n",
       "            20.094516579726115,\n",
       "            19.74853945973233\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5329861111111112,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.914342738407699,\n",
       "            \"x\": 0.9934383202099737\n",
       "        },\n",
       "        \"name\": \"Inlet c\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Inlet c\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.061654272382618835,\n",
       "            0.9934383202099737\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 400,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            5.0,\n",
       "            10.0,\n",
       "            15.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.48447362265275373,\n",
       "            20.693904643873072\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.07581474190726165,\n",
       "            0.9901574803149606\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"5\",\n",
       "            \"10\",\n",
       "            \"15\",\n",
       "            \"20\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"auto\",\n",
       "        \"xanchor\": \"auto\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 1.0,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.0\n",
       "    },\n",
       "    \"width\": 600\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data = readdlm(\"observations.csv\", ',');\n",
    "scatter(train_data[:, 1], train_data[:, 2], xlabel = \"time (min)\", ylabel = \"Concentration (mg/L)\", label = \"Exp\")\n",
    "plot!(train_data[:, 1], ones(size(train_data, 1))*Cin, xlabel = \"time (min)\",\n",
    "        ylabel = \"Concentration (mg/L)\", label = \"Inlet c\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a36e79a2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Defining parameters and Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "id": "80c8ac3a",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.22"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "begin\n",
    "# Bed parameters\n",
    "D_bed = 5*10^-2 #m\n",
    "L_bed = 5*10^-2 #m\n",
    "V_bed = 4/3*pi*(D_bed/2)^3\n",
    "ϵ = 0.45\n",
    "d_p = 2e-3\n",
    "ρ_p = 1100 #kg/m3\n",
    "V_p = 4/3*pi*(d_p/2)^3 \n",
    "V_solid = V_bed*(1-ϵ)\n",
    "V_gas = V_bed*ϵ\n",
    "M_solid = ρ_p*V_solid #kg\n",
    "\n",
    "# Inlet conditions\n",
    "Q = 5 #L/min\n",
    "Cin = 20 # ppm = mg/L\n",
    "\n",
    "#Isotherm parameters\n",
    "qmax = 55.54 #mg/g\n",
    "k = 1.8 #L/mg\n",
    "kmass = 0.22; #min^-1\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3da882d",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Creating Neural Network using Lux.jl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37606171",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"neural_net.svg\" width=\"600\"/></center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "id": "74d6c89c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#Scaling parameter (maximum concentration at solid phase)\n",
    "qstar_max = qmax*k*Cin/(1 + k*Cin);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "id": "661cde98",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "#Neural Network with 2 inputs and one of this inputs is a function (q*)\n",
    "using Lux\n",
    "ɼ = Lux.Chain(x -> x./qstar_max, Lux.Dense(2, 8, tanh), Lux.Dense(8, 10, tanh),\n",
    "                  Lux.Dense(10, 1)); #Uptake rake"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "id": "80833baf",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "using Random;\n",
    "rng = Random.default_rng(222)\n",
    "pinit, st = Lux.setup(rng, ɼ);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "id": "cf26219f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1-element Vector{Float64}:\n",
       " 0.28480718564744717"
      ]
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ɼ([50; 50], pinit, st)[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca45cf57",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Defining the UDE model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "id": "658d2d49",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "using DifferentialEquations, DiffEqFlux"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "id": "ce888044",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "using ComponentArrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "id": "82fd2ca9",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ude (generic function with 1 method)"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function ude(du, u, p, t)\n",
    "    c = u[1] #Fluid phase concentration\n",
    "    q = u[2] #Solid phase concentration\n",
    "\n",
    "    qstar = qmax*k*c/(1.0 + k*c)\n",
    "\n",
    "    du[1] = Q*(Cin - c)/(V_gas*1e3) - M_solid*1e3*ɼ([qstar q]', p, st)[1][1]/(V_gas*1e3) \n",
    "    du[2] = ɼ([qstar q]', p, st)[1][1]\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8595a959",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Simulating the UDE before training it - randomly initialized parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "id": "931a1901",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "Δt = train_data[2, 1] - train_data[1, 1]; #1 min\n",
    "tinit = train_data[1, 1] #0.0 min\n",
    "tfinal = train_data[end, 1] #50 min\n",
    "u0 = [0.0; 0.0];\n",
    "prob = ODEProblem(ude, u0, (tinit, tfinal), pinit);\n",
    "#Note that the ODE has two dependent variables but only of them are measured"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "id": "a8da4c29",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "sol_ude = solve(prob, AutoVern7(KenCarp5(autodiff = true)), abstol = 1e-7, reltol = 1e-7, saveat = Δt);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "id": "17957b47",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.8629483814523184,
          "title": "",
          "x": 0.9943757030371203,
          "y": 0.5527777777777777
         },
         "legendgroup": "untrained UDE",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "untrained UDE",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0,
          17.314043180381326,
          17.313488977867465,
          17.312985030251856,
          17.312531100244325,
          17.312127091911794,
          17.311772933367894,
          17.311468552218628,
          17.311213535281574,
          17.31100751071993,
          17.31085039507496,
          17.310742104887954,
          17.31068255670018,
          17.31067166705292,
          17.310709352487446,
          17.310795384377677,
          17.310928698141858,
          17.311109019841503,
          17.311336257373007,
          17.311610318632738,
          17.31193111151708,
          17.31229854392242,
          17.31271252374513,
          17.313172958881598,
          17.3136797572282,
          17.31423234673819,
          17.314829023513006,
          17.315469545459816,
          17.316153798547194,
          17.316881668743733,
          17.317653042018016,
          17.31846780433862,
          17.31932584167413,
          17.320227039993124,
          17.321171285264196,
          17.32215846345592,
          17.32318846053688,
          17.32426087320407,
          17.325372995402677,
          17.326524103596256,
          17.327714090926005,
          17.328942850533156,
          17.33021027555892,
          17.33151625914451,
          17.33286069443115,
          17.334243474560054,
          17.335664492672432,
          17.33712364190951,
          17.338620815412497,
          17.340155906322614,
          17.341728807781074
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.8629483814523184,
          "title": "",
          "x": 0.9943757030371203,
          "y": 0.5527777777777777
         },
         "legendgroup": "observation",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "observation",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.1149144414942044,
          0.453593327427082,
          0.17166903204340475,
          0.16434298525027757,
          0.43082886105468254,
          0.39036313080466295,
          0.1498263562138032,
          0.6049853300852165,
          0.7288340742910437,
          0.7208278900541426,
          0.9725557011832456,
          1.0645132432308824,
          1.605832870402756,
          1.9623017325357597,
          2.488345947495106,
          2.9372823345460555,
          3.3004647312077497,
          4.44026467007553,
          6.014621388907804,
          7.332105354195911,
          8.600968893100676,
          9.879855941342193,
          11.70106015407636,
          12.820166371191963,
          14.011211986433725,
          15.036464869109007,
          16.06275295941086,
          16.493576791018565,
          16.929248776222778,
          17.310492828215327,
          17.808197157388648,
          18.412709295322795,
          18.90649223578161,
          19.02591785290369,
          19.112996811420775,
          18.883545295888464,
          19.46450909951882,
          19.24859182601517,
          19.527825543706353,
          20.0553701728416,
          19.714848868287742,
          19.63114088016073,
          19.53588580560029,
          19.85638399603698,
          19.803292061991808,
          19.732583577342563,
          19.847534725209762,
          19.71101197633774,
          19.96119154981576,
          20.094516579726115,
          19.74853945973233
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 250,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1,
         "xanchor": "auto",
         "y": 1,
         "yanchor": "auto"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 700,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.05284651918510186,
          0.9943757030371203
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.12130358705161852,
          0.9842519685039369
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.6028354973917835,
          20.6973520771179
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "5",
          "10",
          "15",
          "20"
         ],
         "tickvals": [
          0,
          5,
          10,
          15,
          20
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"4d76625f-0fe6-41c2-a4f5-9e92ba222e93\" style=\"width:700px;height:250px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('4d76625f-0fe6-41c2-a4f5-9e92ba222e93', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5527777777777777,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.8629483814523184,\n",
       "            \"x\": 0.9943757030371203\n",
       "        },\n",
       "        \"name\": \"untrained UDE\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"untrained UDE\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            17.314043180381326,\n",
       "            17.313488977867465,\n",
       "            17.312985030251856,\n",
       "            17.312531100244325,\n",
       "            17.312127091911794,\n",
       "            17.311772933367894,\n",
       "            17.311468552218628,\n",
       "            17.311213535281574,\n",
       "            17.31100751071993,\n",
       "            17.31085039507496,\n",
       "            17.310742104887954,\n",
       "            17.31068255670018,\n",
       "            17.31067166705292,\n",
       "            17.310709352487446,\n",
       "            17.310795384377677,\n",
       "            17.310928698141858,\n",
       "            17.311109019841503,\n",
       "            17.311336257373007,\n",
       "            17.311610318632738,\n",
       "            17.31193111151708,\n",
       "            17.31229854392242,\n",
       "            17.31271252374513,\n",
       "            17.313172958881598,\n",
       "            17.3136797572282,\n",
       "            17.31423234673819,\n",
       "            17.314829023513006,\n",
       "            17.315469545459816,\n",
       "            17.316153798547194,\n",
       "            17.316881668743733,\n",
       "            17.317653042018016,\n",
       "            17.31846780433862,\n",
       "            17.31932584167413,\n",
       "            17.320227039993124,\n",
       "            17.321171285264196,\n",
       "            17.32215846345592,\n",
       "            17.32318846053688,\n",
       "            17.32426087320407,\n",
       "            17.325372995402677,\n",
       "            17.326524103596256,\n",
       "            17.327714090926005,\n",
       "            17.328942850533156,\n",
       "            17.33021027555892,\n",
       "            17.33151625914451,\n",
       "            17.33286069443115,\n",
       "            17.334243474560054,\n",
       "            17.335664492672432,\n",
       "            17.33712364190951,\n",
       "            17.338620815412497,\n",
       "            17.340155906322614,\n",
       "            17.341728807781074\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5527777777777777,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.8629483814523184,\n",
       "            \"x\": 0.9943757030371203\n",
       "        },\n",
       "        \"name\": \"observation\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"observation\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.1149144414942044,\n",
       "            0.453593327427082,\n",
       "            0.17166903204340475,\n",
       "            0.16434298525027757,\n",
       "            0.43082886105468254,\n",
       "            0.39036313080466295,\n",
       "            0.1498263562138032,\n",
       "            0.6049853300852165,\n",
       "            0.7288340742910437,\n",
       "            0.7208278900541426,\n",
       "            0.9725557011832456,\n",
       "            1.0645132432308824,\n",
       "            1.605832870402756,\n",
       "            1.9623017325357597,\n",
       "            2.488345947495106,\n",
       "            2.9372823345460555,\n",
       "            3.3004647312077497,\n",
       "            4.44026467007553,\n",
       "            6.014621388907804,\n",
       "            7.332105354195911,\n",
       "            8.600968893100676,\n",
       "            9.879855941342193,\n",
       "            11.70106015407636,\n",
       "            12.820166371191963,\n",
       "            14.011211986433725,\n",
       "            15.036464869109007,\n",
       "            16.06275295941086,\n",
       "            16.493576791018565,\n",
       "            16.929248776222778,\n",
       "            17.310492828215327,\n",
       "            17.808197157388648,\n",
       "            18.412709295322795,\n",
       "            18.90649223578161,\n",
       "            19.02591785290369,\n",
       "            19.112996811420775,\n",
       "            18.883545295888464,\n",
       "            19.46450909951882,\n",
       "            19.24859182601517,\n",
       "            19.527825543706353,\n",
       "            20.0553701728416,\n",
       "            19.714848868287742,\n",
       "            19.63114088016073,\n",
       "            19.53588580560029,\n",
       "            19.85638399603698,\n",
       "            19.803292061991808,\n",
       "            19.732583577342563,\n",
       "            19.847534725209762,\n",
       "            19.71101197633774,\n",
       "            19.96119154981576,\n",
       "            20.094516579726115,\n",
       "            19.74853945973233\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.05284651918510186,\n",
       "            0.9943757030371203\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 250,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            5.0,\n",
       "            10.0,\n",
       "            15.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.6028354973917835,\n",
       "            20.6973520771179\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.12130358705161852,\n",
       "            0.9842519685039369\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"5\",\n",
       "            \"10\",\n",
       "            \"15\",\n",
       "            \"20\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"auto\",\n",
       "        \"xanchor\": \"auto\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 1.0,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.0\n",
       "    },\n",
       "    \"width\": 700\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot(sol_ude.t, Array(sol_ude)[1, :], label = \"untrained UDE\", size = (700, 250))\n",
    "scatter!(train_data[:, 1], train_data[:, 2], xlabel = \"time (min)\", ylabel = \"Concentration (mg/L)\", label = \"observation\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c53ed06",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Training using only one of the states as training data - c(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "id": "8ca3d4f6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "using SciMLSensitivity, Optimization, OptimizationOptimisers, OptimizationOptimJL, Zygote"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "id": "ded71d05",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "predict (generic function with 1 method)"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function predict(p, states)\n",
    "    new_prob = remake(prob; p = p)\n",
    "    sensealg = InterpolatingAdjoint(autojacvec = ReverseDiffVJP(true)) #that is a very important choice to make\n",
    "    out = Array(solve(new_prob, AutoVern7(KenCarp4(autodiff = false)), abstol = 1e-6, reltol = 1e-6,\n",
    "    sensealg = sensealg, saveat = Δt))[states, :]; #g = [0, 1] * u\n",
    "    out\n",
    "end\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "id": "efb77ce3",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loss (generic function with 1 method)"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function loss(p)\n",
    "    y = predict(p, 1) #Call predict function\n",
    "    loss = sum(abs2, (y .- train_data[:, 2])/Cin) #squared error\n",
    "    loss\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "599bb905",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Setting up optimization problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "id": "a121cac8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12.241571739579275"
      ]
     },
     "execution_count": 170,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss(pinit) #High initial loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "id": "d539de79",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "adtype = Optimization.AutoZygote()\n",
    "optf = Optimization.OptimizationFunction((x,p) -> loss(x), adtype);\n",
    "optprob = Optimization.OptimizationProblem(optf, ComponentArray(pinit));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "id": "97515274",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "#47 (generic function with 1 method)"
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "losses = Float64[]\n",
    "callback = function (p, l)\n",
    "    push!(losses, l)\n",
    "    if length(losses) % 10 == 0\n",
    "        println(\"Current loss after $(length(losses)) iterations: $(losses[end])\")\n",
    "    end\n",
    "    return false\n",
    "end\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "id": "7e905a15",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current loss after 10 iterations: 0.9912751016369831\n",
      "Current loss after 20 iterations: 0.05233927397387695\n",
      "Current loss after 30 iterations: 0.18011964069645936\n",
      "Current loss after 40 iterations: 0.08956623192614221\n",
      "Current loss after 50 iterations: 0.06762473870592468\n",
      "Current loss after 60 iterations: 0.02934085739176759\n",
      "Current loss after 70 iterations: 0.022607886716461845\n",
      "Current loss after 80 iterations: 0.022410285687602102\n",
      "Current loss after 90 iterations: 0.018872805287614713\n",
      "Current loss after 100 iterations: 0.01704543106013229\n",
      "Current loss after 110 iterations: 0.015542540768194571\n",
      "Current loss after 120 iterations: 0.01423458567782907\n",
      "Current loss after 130 iterations: 0.013064919704580066\n",
      "Current loss after 140 iterations: 0.011966802434716766\n",
      "Current loss after 150 iterations: 0.010987784025897857\n",
      "Current loss after 160 iterations: 0.010131311805941621\n",
      "Current loss after 170 iterations: 0.009390651707241132\n",
      "Current loss after 180 iterations: 0.008742915632061329\n",
      "Current loss after 190 iterations: 0.008174523315993765\n",
      "Current loss after 200 iterations: 0.007676540409905058\n",
      "Current loss after 210 iterations: 0.007242898215593904\n",
      "Current loss after 220 iterations: 0.006867787536750321\n",
      "Current loss after 230 iterations: 0.006539729535884639\n",
      "Current loss after 240 iterations: 0.006251219598803126\n",
      "Current loss after 250 iterations: 0.005998180310079479\n",
      "Current loss after 260 iterations: 0.005778377923188015\n",
      "Current loss after 270 iterations: 0.005585555215862424\n",
      "161.054523 seconds (281.48 M allocations: 501.918 GiB, 29.87% gc time, 4.18% compilation time: 2% of which was recompilation)\n"
     ]
    }
   ],
   "source": [
    "@time res1 = Optimization.solve(optprob, ADAM(0.075), callback = callback, maxiters = 270);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "id": "f5c95fe0",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data = readdlm(\"test_data.csv\", ',');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "id": "35d3b4e3",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Trained UDE",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Trained UDE",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0,
          0.0482181609352962,
          0.08825970743944449,
          0.1340026767178737,
          0.18668515960388074,
          0.24797480025058502,
          0.32009395341784425,
          0.4059241843150957,
          0.5095466831494949,
          0.636540847109897,
          0.7952533314593695,
          0.9970393689457062,
          1.259618646587108,
          1.6075573877296543,
          2.0764349796672215,
          2.7112137784393235,
          3.5578434495654556,
          4.640726862817994,
          5.935208178533228,
          7.373022285956178,
          8.856993129420967,
          10.308425675559352,
          11.661026852754796,
          12.885320688350228,
          13.96793943379242,
          14.907678336486317,
          15.715868020666473,
          16.40376966783547,
          16.986551680564595,
          17.478047007711243,
          17.891185876414056,
          18.237901807109772,
          18.528246272079155,
          18.77127972466813,
          18.97435845222581,
          19.14414036216897,
          19.28584695926092,
          19.40421939374176,
          19.503000664453314,
          19.585392918212648,
          19.654214165998756,
          19.71156460049967,
          19.759437257516854,
          19.79939491327772,
          19.832666473775664,
          19.860449620268668,
          19.883644141969953,
          19.902946986142553,
          19.919047986580296,
          19.932494434063106,
          19.943700212093113
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Train data",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Train data",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.1149144414942044,
          0.453593327427082,
          0.17166903204340475,
          0.16434298525027757,
          0.43082886105468254,
          0.39036313080466295,
          0.1498263562138032,
          0.6049853300852165,
          0.7288340742910437,
          0.7208278900541426,
          0.9725557011832456,
          1.0645132432308824,
          1.605832870402756,
          1.9623017325357597,
          2.488345947495106,
          2.9372823345460555,
          3.3004647312077497,
          4.44026467007553,
          6.014621388907804,
          7.332105354195911,
          8.600968893100676,
          9.879855941342193,
          11.70106015407636,
          12.820166371191963,
          14.011211986433725,
          15.036464869109007,
          16.06275295941086,
          16.493576791018565,
          16.929248776222778,
          17.310492828215327,
          17.808197157388648,
          18.412709295322795,
          18.90649223578161,
          19.02591785290369,
          19.112996811420775,
          18.883545295888464,
          19.46450909951882,
          19.24859182601517,
          19.527825543706353,
          20.0553701728416,
          19.714848868287742,
          19.63114088016073,
          19.53588580560029,
          19.85638399603698,
          19.803292061991808,
          19.732583577342563,
          19.847534725209762,
          19.71101197633774,
          19.96119154981576,
          20.094516579726115,
          19.74853945973233
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 350,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1.05,
         "xanchor": "lower",
         "y": 0.07,
         "yanchor": "right"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 800,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.046240704286964134,
          0.9950787401574803
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.08664541932258472,
          0.9887514060742407
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.6028354973917835,
          20.6973520771179
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "5",
          "10",
          "15",
          "20"
         ],
         "tickvals": [
          0,
          5,
          10,
          15,
          20
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"d4f5ec5d-7aa5-4816-bc58-803e4778a9e2\" style=\"width:800px;height:350px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('d4f5ec5d-7aa5-4816-bc58-803e4778a9e2', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Trained UDE\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Trained UDE\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            0.0482181609352962,\n",
       "            0.08825970743944449,\n",
       "            0.1340026767178737,\n",
       "            0.18668515960388074,\n",
       "            0.24797480025058502,\n",
       "            0.32009395341784425,\n",
       "            0.4059241843150957,\n",
       "            0.5095466831494949,\n",
       "            0.636540847109897,\n",
       "            0.7952533314593695,\n",
       "            0.9970393689457062,\n",
       "            1.259618646587108,\n",
       "            1.6075573877296543,\n",
       "            2.0764349796672215,\n",
       "            2.7112137784393235,\n",
       "            3.5578434495654556,\n",
       "            4.640726862817994,\n",
       "            5.935208178533228,\n",
       "            7.373022285956178,\n",
       "            8.856993129420967,\n",
       "            10.308425675559352,\n",
       "            11.661026852754796,\n",
       "            12.885320688350228,\n",
       "            13.96793943379242,\n",
       "            14.907678336486317,\n",
       "            15.715868020666473,\n",
       "            16.40376966783547,\n",
       "            16.986551680564595,\n",
       "            17.478047007711243,\n",
       "            17.891185876414056,\n",
       "            18.237901807109772,\n",
       "            18.528246272079155,\n",
       "            18.77127972466813,\n",
       "            18.97435845222581,\n",
       "            19.14414036216897,\n",
       "            19.28584695926092,\n",
       "            19.40421939374176,\n",
       "            19.503000664453314,\n",
       "            19.585392918212648,\n",
       "            19.654214165998756,\n",
       "            19.71156460049967,\n",
       "            19.759437257516854,\n",
       "            19.79939491327772,\n",
       "            19.832666473775664,\n",
       "            19.860449620268668,\n",
       "            19.883644141969953,\n",
       "            19.902946986142553,\n",
       "            19.919047986580296,\n",
       "            19.932494434063106,\n",
       "            19.943700212093113\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Train data\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Train data\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.1149144414942044,\n",
       "            0.453593327427082,\n",
       "            0.17166903204340475,\n",
       "            0.16434298525027757,\n",
       "            0.43082886105468254,\n",
       "            0.39036313080466295,\n",
       "            0.1498263562138032,\n",
       "            0.6049853300852165,\n",
       "            0.7288340742910437,\n",
       "            0.7208278900541426,\n",
       "            0.9725557011832456,\n",
       "            1.0645132432308824,\n",
       "            1.605832870402756,\n",
       "            1.9623017325357597,\n",
       "            2.488345947495106,\n",
       "            2.9372823345460555,\n",
       "            3.3004647312077497,\n",
       "            4.44026467007553,\n",
       "            6.014621388907804,\n",
       "            7.332105354195911,\n",
       "            8.600968893100676,\n",
       "            9.879855941342193,\n",
       "            11.70106015407636,\n",
       "            12.820166371191963,\n",
       "            14.011211986433725,\n",
       "            15.036464869109007,\n",
       "            16.06275295941086,\n",
       "            16.493576791018565,\n",
       "            16.929248776222778,\n",
       "            17.310492828215327,\n",
       "            17.808197157388648,\n",
       "            18.412709295322795,\n",
       "            18.90649223578161,\n",
       "            19.02591785290369,\n",
       "            19.112996811420775,\n",
       "            18.883545295888464,\n",
       "            19.46450909951882,\n",
       "            19.24859182601517,\n",
       "            19.527825543706353,\n",
       "            20.0553701728416,\n",
       "            19.714848868287742,\n",
       "            19.63114088016073,\n",
       "            19.53588580560029,\n",
       "            19.85638399603698,\n",
       "            19.803292061991808,\n",
       "            19.732583577342563,\n",
       "            19.847534725209762,\n",
       "            19.71101197633774,\n",
       "            19.96119154981576,\n",
       "            20.094516579726115,\n",
       "            19.74853945973233\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.046240704286964134,\n",
       "            0.9950787401574803\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 350,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            5.0,\n",
       "            10.0,\n",
       "            15.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.6028354973917835,\n",
       "            20.6973520771179\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.08664541932258472,\n",
       "            0.9887514060742407\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"5\",\n",
       "            \"10\",\n",
       "            \"15\",\n",
       "            \"20\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"right\",\n",
       "        \"xanchor\": \"lower\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.05\n",
       "    },\n",
       "    \"width\": 800\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 186,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Cin = 20\n",
    "plot(sol_ude.t, predict(res1.u, 1), label = \"Trained UDE\", size = (800, 350))\n",
    "scatter!(train_data[:, 1], train_data[:, 2], xlabel = \"time (min)\", \n",
    "    ylabel = \"Concentration (mg/L)\", label = \"Train data\",\n",
    "legend=:outerbottomright)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "id": "8501e41d",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Test set UDE",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Test set UDE",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99,
          100
         ],
         "xaxis": "x",
         "y": [
          0,
          -0.052646471993779045,
          -0.039523730332327124,
          -0.025700734638002227,
          -0.011113899274619032,
          0.004290218401462251,
          0.02059367952673621,
          0.03786862639247079,
          0.056194254664573916,
          0.07569192185382216,
          0.09645461506223704,
          0.11858899407964521,
          0.14227209012443842,
          0.16766976551128873,
          0.19488902039510916,
          0.22418179454662537,
          0.25583074915157183,
          0.2900761455761923,
          0.3271549818945908,
          0.36750964274231346,
          0.4115167659418983,
          0.4596765003444433,
          0.5125291870684767,
          0.5708594378697114,
          0.635259833408547,
          0.7067774841405069,
          0.7865674917421221,
          0.8757644105406498,
          0.9759681146480044,
          1.0889997365418889,
          1.2169685173177573,
          1.3624302818533436,
          1.5280003560584898,
          1.717010934469918,
          1.9325329136228329,
          2.1785390637698314,
          2.4570105114411573,
          2.771056775523566,
          3.120481867932941,
          3.5035161311320544,
          3.9166656298544247,
          4.352263543590262,
          4.8022465371565515,
          5.255236164540081,
          5.702618012895213,
          6.135029191932576,
          6.546114819084976,
          6.929847288327538,
          7.283842867183321,
          7.606817027139107,
          7.898581715824753,
          8.160154952565637,
          8.392806849883417,
          8.599235394948124,
          8.781042373250438,
          8.940954010801518,
          9.080904344815579,
          9.20324783961261,
          9.309933008718483,
          9.402729944103035,
          9.48343109338991,
          9.553432519323533,
          9.614132892719603,
          9.66673156205382,
          9.712231106552533,
          9.751616651776963,
          9.785659620748628,
          9.815053286925743,
          9.840467965607822,
          9.86241368562716,
          9.881336256268618,
          9.897677183684731,
          9.911791166542615,
          9.923951699318458,
          9.934429178131536,
          9.943481739483135,
          9.951292006981907,
          9.958014778072599,
          9.963804853124714,
          9.968808886505247,
          9.973126358013145,
          9.97684042766141,
          9.980034270046668,
          9.982790742395741,
          9.985175112869635,
          9.987229399539814,
          9.988994449767095,
          9.990511110912294,
          9.9918201883204,
          9.9929542194805,
          9.99393216106365,
          9.994772390720101,
          9.99549328610011,
          9.996113224853932,
          9.99665019376576,
          9.997115502310987,
          9.997516485276822,
          9.99786066615587,
          9.99815556844073,
          9.998408715624004,
          9.998627631198293
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Test data",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Test data",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99,
          100
         ],
         "xaxis": "x",
         "y": [
          0.4116816151357062,
          0.20991569680016633,
          0.34274799493835084,
          0.2077971111405163,
          0.3454918924040904,
          0.36372387207451196,
          0.04246359638375827,
          0.10755411672364348,
          0.23032216181593876,
          0.32030939121781066,
          0.3149902084588786,
          0.3403263382394859,
          0.06179159214774943,
          0.1989538597349111,
          0.3917351134973381,
          0.6122288233236826,
          0.48079384682181436,
          0.7539686757638644,
          0.5988240201294828,
          0.5784879535011717,
          0.8624445479228227,
          0.5997542323701074,
          0.5302786639171492,
          1.097078150109682,
          0.7294070605067997,
          0.8237227594423556,
          0.7557120006056826,
          0.9076335151817955,
          1.3677931639356355,
          1.4236099841013925,
          1.9478574758392795,
          1.865490581707562,
          1.7572465435626854,
          1.9349872784435318,
          1.974440410476193,
          2.7702857130629006,
          2.985709176031708,
          2.6720411332723386,
          3.3004051957797325,
          3.7760950435917495,
          3.933150064878424,
          4.317028550262893,
          4.776392130525876,
          5.3464742770512474,
          5.438113913182881,
          5.922517309812299,
          6.325061528350176,
          6.308504744509279,
          6.747898759762233,
          7.50558221919882,
          7.4213451193020115,
          8.181139416360317,
          8.091521973725762,
          8.33492637101121,
          8.7688791920082,
          8.610681256363089,
          8.417851194079272,
          9.142725531764805,
          9.331969062864399,
          9.301195243368989,
          9.890484939055444,
          9.229282531013475,
          9.726499740494209,
          9.915636244635145,
          9.435441981130777,
          9.343521365138896,
          9.69634377620142,
          9.574712833220847,
          9.651109928460812,
          9.792802051855237,
          9.81427401349818,
          9.981882804893436,
          9.855662341268406,
          9.829763063435925,
          9.415613586052926,
          9.87834537957247,
          9.689179083467199,
          10.097819429988325,
          10.082602006989859,
          9.897746006816357,
          10.507142751663256,
          10.000031962146524,
          10.000330555430585,
          10.245318275277679,
          9.720045384693796,
          9.97241755004881,
          9.94204499574352,
          9.958768585592319,
          9.730197795652414,
          10.064810454921503,
          10.018731681658988,
          10.101159031528036,
          10.116155770522326,
          9.977403541367945,
          9.901577494477456,
          9.667068930167236,
          10.031587789791216,
          10.203233982096604,
          9.940260412820981,
          9.95856469844549,
          10.087155098125692
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 350,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1.05,
         "xanchor": "lower",
         "y": 0.07,
         "yanchor": "right"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 800,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.06846292650918635,
          0.9950787401574803
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -3,
          103
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "25",
          "50",
          "75",
          "100"
         ],
         "tickvals": [
          0,
          25,
          50,
          75,
          100
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.08664541932258472,
          0.9887514060742407
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.3694401487034895,
          10.823936428372967
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0.0",
          "2.5",
          "5.0",
          "7.5",
          "10.0"
         ],
         "tickvals": [
          0,
          2.5,
          5,
          7.5,
          10
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"1e33a014-b796-47af-aa22-e3243fa6dea4\" style=\"width:800px;height:350px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('1e33a014-b796-47af-aa22-e3243fa6dea4', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Test set UDE\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Test set UDE\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            -0.052646471993779045,\n",
       "            -0.039523730332327124,\n",
       "            -0.025700734638002227,\n",
       "            -0.011113899274619032,\n",
       "            0.004290218401462251,\n",
       "            0.02059367952673621,\n",
       "            0.03786862639247079,\n",
       "            0.056194254664573916,\n",
       "            0.07569192185382216,\n",
       "            0.09645461506223704,\n",
       "            0.11858899407964521,\n",
       "            0.14227209012443842,\n",
       "            0.16766976551128873,\n",
       "            0.19488902039510916,\n",
       "            0.22418179454662537,\n",
       "            0.25583074915157183,\n",
       "            0.2900761455761923,\n",
       "            0.3271549818945908,\n",
       "            0.36750964274231346,\n",
       "            0.4115167659418983,\n",
       "            0.4596765003444433,\n",
       "            0.5125291870684767,\n",
       "            0.5708594378697114,\n",
       "            0.635259833408547,\n",
       "            0.7067774841405069,\n",
       "            0.7865674917421221,\n",
       "            0.8757644105406498,\n",
       "            0.9759681146480044,\n",
       "            1.0889997365418889,\n",
       "            1.2169685173177573,\n",
       "            1.3624302818533436,\n",
       "            1.5280003560584898,\n",
       "            1.717010934469918,\n",
       "            1.9325329136228329,\n",
       "            2.1785390637698314,\n",
       "            2.4570105114411573,\n",
       "            2.771056775523566,\n",
       "            3.120481867932941,\n",
       "            3.5035161311320544,\n",
       "            3.9166656298544247,\n",
       "            4.352263543590262,\n",
       "            4.8022465371565515,\n",
       "            5.255236164540081,\n",
       "            5.702618012895213,\n",
       "            6.135029191932576,\n",
       "            6.546114819084976,\n",
       "            6.929847288327538,\n",
       "            7.283842867183321,\n",
       "            7.606817027139107,\n",
       "            7.898581715824753,\n",
       "            8.160154952565637,\n",
       "            8.392806849883417,\n",
       "            8.599235394948124,\n",
       "            8.781042373250438,\n",
       "            8.940954010801518,\n",
       "            9.080904344815579,\n",
       "            9.20324783961261,\n",
       "            9.309933008718483,\n",
       "            9.402729944103035,\n",
       "            9.48343109338991,\n",
       "            9.553432519323533,\n",
       "            9.614132892719603,\n",
       "            9.66673156205382,\n",
       "            9.712231106552533,\n",
       "            9.751616651776963,\n",
       "            9.785659620748628,\n",
       "            9.815053286925743,\n",
       "            9.840467965607822,\n",
       "            9.86241368562716,\n",
       "            9.881336256268618,\n",
       "            9.897677183684731,\n",
       "            9.911791166542615,\n",
       "            9.923951699318458,\n",
       "            9.934429178131536,\n",
       "            9.943481739483135,\n",
       "            9.951292006981907,\n",
       "            9.958014778072599,\n",
       "            9.963804853124714,\n",
       "            9.968808886505247,\n",
       "            9.973126358013145,\n",
       "            9.97684042766141,\n",
       "            9.980034270046668,\n",
       "            9.982790742395741,\n",
       "            9.985175112869635,\n",
       "            9.987229399539814,\n",
       "            9.988994449767095,\n",
       "            9.990511110912294,\n",
       "            9.9918201883204,\n",
       "            9.9929542194805,\n",
       "            9.99393216106365,\n",
       "            9.994772390720101,\n",
       "            9.99549328610011,\n",
       "            9.996113224853932,\n",
       "            9.99665019376576,\n",
       "            9.997115502310987,\n",
       "            9.997516485276822,\n",
       "            9.99786066615587,\n",
       "            9.99815556844073,\n",
       "            9.998408715624004,\n",
       "            9.998627631198293\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0,\n",
       "            51.0,\n",
       "            52.0,\n",
       "            53.0,\n",
       "            54.0,\n",
       "            55.0,\n",
       "            56.0,\n",
       "            57.0,\n",
       "            58.0,\n",
       "            59.0,\n",
       "            60.0,\n",
       "            61.0,\n",
       "            62.0,\n",
       "            63.0,\n",
       "            64.0,\n",
       "            65.0,\n",
       "            66.0,\n",
       "            67.0,\n",
       "            68.0,\n",
       "            69.0,\n",
       "            70.0,\n",
       "            71.0,\n",
       "            72.0,\n",
       "            73.0,\n",
       "            74.0,\n",
       "            75.0,\n",
       "            76.0,\n",
       "            77.0,\n",
       "            78.0,\n",
       "            79.0,\n",
       "            80.0,\n",
       "            81.0,\n",
       "            82.0,\n",
       "            83.0,\n",
       "            84.0,\n",
       "            85.0,\n",
       "            86.0,\n",
       "            87.0,\n",
       "            88.0,\n",
       "            89.0,\n",
       "            90.0,\n",
       "            91.0,\n",
       "            92.0,\n",
       "            93.0,\n",
       "            94.0,\n",
       "            95.0,\n",
       "            96.0,\n",
       "            97.0,\n",
       "            98.0,\n",
       "            99.0,\n",
       "            100.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Test data\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Test data\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.4116816151357062,\n",
       "            0.20991569680016633,\n",
       "            0.34274799493835084,\n",
       "            0.2077971111405163,\n",
       "            0.3454918924040904,\n",
       "            0.36372387207451196,\n",
       "            0.04246359638375827,\n",
       "            0.10755411672364348,\n",
       "            0.23032216181593876,\n",
       "            0.32030939121781066,\n",
       "            0.3149902084588786,\n",
       "            0.3403263382394859,\n",
       "            0.06179159214774943,\n",
       "            0.1989538597349111,\n",
       "            0.3917351134973381,\n",
       "            0.6122288233236826,\n",
       "            0.48079384682181436,\n",
       "            0.7539686757638644,\n",
       "            0.5988240201294828,\n",
       "            0.5784879535011717,\n",
       "            0.8624445479228227,\n",
       "            0.5997542323701074,\n",
       "            0.5302786639171492,\n",
       "            1.097078150109682,\n",
       "            0.7294070605067997,\n",
       "            0.8237227594423556,\n",
       "            0.7557120006056826,\n",
       "            0.9076335151817955,\n",
       "            1.3677931639356355,\n",
       "            1.4236099841013925,\n",
       "            1.9478574758392795,\n",
       "            1.865490581707562,\n",
       "            1.7572465435626854,\n",
       "            1.9349872784435318,\n",
       "            1.974440410476193,\n",
       "            2.7702857130629006,\n",
       "            2.985709176031708,\n",
       "            2.6720411332723386,\n",
       "            3.3004051957797325,\n",
       "            3.7760950435917495,\n",
       "            3.933150064878424,\n",
       "            4.317028550262893,\n",
       "            4.776392130525876,\n",
       "            5.3464742770512474,\n",
       "            5.438113913182881,\n",
       "            5.922517309812299,\n",
       "            6.325061528350176,\n",
       "            6.308504744509279,\n",
       "            6.747898759762233,\n",
       "            7.50558221919882,\n",
       "            7.4213451193020115,\n",
       "            8.181139416360317,\n",
       "            8.091521973725762,\n",
       "            8.33492637101121,\n",
       "            8.7688791920082,\n",
       "            8.610681256363089,\n",
       "            8.417851194079272,\n",
       "            9.142725531764805,\n",
       "            9.331969062864399,\n",
       "            9.301195243368989,\n",
       "            9.890484939055444,\n",
       "            9.229282531013475,\n",
       "            9.726499740494209,\n",
       "            9.915636244635145,\n",
       "            9.435441981130777,\n",
       "            9.343521365138896,\n",
       "            9.69634377620142,\n",
       "            9.574712833220847,\n",
       "            9.651109928460812,\n",
       "            9.792802051855237,\n",
       "            9.81427401349818,\n",
       "            9.981882804893436,\n",
       "            9.855662341268406,\n",
       "            9.829763063435925,\n",
       "            9.415613586052926,\n",
       "            9.87834537957247,\n",
       "            9.689179083467199,\n",
       "            10.097819429988325,\n",
       "            10.082602006989859,\n",
       "            9.897746006816357,\n",
       "            10.507142751663256,\n",
       "            10.000031962146524,\n",
       "            10.000330555430585,\n",
       "            10.245318275277679,\n",
       "            9.720045384693796,\n",
       "            9.97241755004881,\n",
       "            9.94204499574352,\n",
       "            9.958768585592319,\n",
       "            9.730197795652414,\n",
       "            10.064810454921503,\n",
       "            10.018731681658988,\n",
       "            10.101159031528036,\n",
       "            10.116155770522326,\n",
       "            9.977403541367945,\n",
       "            9.901577494477456,\n",
       "            9.667068930167236,\n",
       "            10.031587789791216,\n",
       "            10.203233982096604,\n",
       "            9.940260412820981,\n",
       "            9.95856469844549,\n",
       "            10.087155098125692\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0,\n",
       "            51.0,\n",
       "            52.0,\n",
       "            53.0,\n",
       "            54.0,\n",
       "            55.0,\n",
       "            56.0,\n",
       "            57.0,\n",
       "            58.0,\n",
       "            59.0,\n",
       "            60.0,\n",
       "            61.0,\n",
       "            62.0,\n",
       "            63.0,\n",
       "            64.0,\n",
       "            65.0,\n",
       "            66.0,\n",
       "            67.0,\n",
       "            68.0,\n",
       "            69.0,\n",
       "            70.0,\n",
       "            71.0,\n",
       "            72.0,\n",
       "            73.0,\n",
       "            74.0,\n",
       "            75.0,\n",
       "            76.0,\n",
       "            77.0,\n",
       "            78.0,\n",
       "            79.0,\n",
       "            80.0,\n",
       "            81.0,\n",
       "            82.0,\n",
       "            83.0,\n",
       "            84.0,\n",
       "            85.0,\n",
       "            86.0,\n",
       "            87.0,\n",
       "            88.0,\n",
       "            89.0,\n",
       "            90.0,\n",
       "            91.0,\n",
       "            92.0,\n",
       "            93.0,\n",
       "            94.0,\n",
       "            95.0,\n",
       "            96.0,\n",
       "            97.0,\n",
       "            98.0,\n",
       "            99.0,\n",
       "            100.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            25.0,\n",
       "            50.0,\n",
       "            75.0,\n",
       "            100.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -3.0,\n",
       "            103.0\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.06846292650918635,\n",
       "            0.9950787401574803\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"25\",\n",
       "            \"50\",\n",
       "            \"75\",\n",
       "            \"100\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 350,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            2.5,\n",
       "            5.0,\n",
       "            7.5,\n",
       "            10.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.3694401487034895,\n",
       "            10.823936428372967\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.08664541932258472,\n",
       "            0.9887514060742407\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0.0\",\n",
       "            \"2.5\",\n",
       "            \"5.0\",\n",
       "            \"7.5\",\n",
       "            \"10.0\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"right\",\n",
       "        \"xanchor\": \"lower\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.05\n",
       "    },\n",
       "    \"width\": 800\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Cin = 10.0\n",
    "test_prob = remake(prob; p = res1.u, tspan = (0.0, 100.), u0 = [0.0, 0.0])\n",
    "sol_testprob = Array(solve(test_prob, AutoVern7(KenCarp4(autodiff = true)), abstol = 1e-6, reltol = 1e-6,\n",
    "        saveat = Δt))[1, :];\n",
    "plot(0.0:1.0:100.0, sol_testprob, label = \"Test set UDE\", size = (800, 350))\n",
    "scatter!(test_data[:, 1], test_data[:, 2], xlabel = \"time (min)\", \n",
    "    ylabel = \"Concentration (mg/L)\", label = \"Test data\",\n",
    "legend=:outerbottomright)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6fda16b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Smoothed Collocation for Fast Two-Stage Training"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9d94c31",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Training ANNs in ODEs can be slow. Can we speed it up with a warm start?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ffd29ff",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$ANN(q^*, q) = \\frac{dq}{dt}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1533f6f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "How do we estimate dq/dt? First we have to estimate q(t) from c(t)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52e59ded",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$V_f \\int_{0}^{c}{dc} + M_s \\int_{0}^{q}{dq} = Q(c_{\\text{in}}t - \\int_{0}^{c}{c}dt)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0e63c44",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$q(t) = Q/M_s(c_{\\text{in}}t - \\int_{0}^{c}{c}dt) - V_f/M_s c(t) $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "id": "b5766214",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "using DataInterpolations, Integrals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "id": "f6c9e601",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "t = train_data[:, 1]\n",
    "c_exp = train_data[:, 2]\n",
    "c_itp = AkimaInterpolation(c_exp, t); #Interpolate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "id": "db64dd5b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "q (generic function with 1 method)"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "integrand(t) = Q/(M_solid*1e3)*c_itp(t)\n",
    "int_integrand(t) = solve(IntegralProblem((t, p) -> integrand(t), 0.0, t), QuadGKJL()).u\n",
    "q(t) = Q/(M_solid*1e3)*Cin*t - int_integrand(t) - V_gas/M_solid*c_itp(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "id": "46dccc55",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "q_train = q.(t);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34e87faa",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can use Kernel smoothing to estimate dq(t)/dt from q(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "id": "64239d01",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([2.535207827004392 2.532016773743905 … 0.010450459894075337 0.008616109173400943], [0.09093161908405922 2.576075130934778 … 53.90842337568725 53.93068327921643])"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dq_dt_est, q_smooth = collocate_data(q_train[:, :]', train_data[:, 1], EpanechnikovKernel()); #From DiffEqFlux"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "id": "ec867084",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.885790317876932,
          "title": "",
          "x": 0.9943757030371203,
          "y": 0.5439814814814815
         },
         "legendgroup": "q_exp (estimated)",
         "marker": {
          "color": "rgba(0, 154, 250, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "q_exp (estimated)",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          -8.547355152461484e-05,
          2.4814107896581175,
          4.967579586702007,
          7.47404948542323,
          9.961362883953916,
          12.432971958293201,
          14.92577487064249,
          17.40484795364266,
          19.84464461273799,
          22.27918475649282,
          24.698399663466496,
          27.096279788111115,
          29.453840504843374,
          31.754458202536718,
          33.99849169798882,
          36.180685952564865,
          38.31255019624902,
          40.35868007842807,
          42.221570358298756,
          43.90260295011116,
          45.42113425176046,
          46.77899152199593,
          47.93964494404699,
          48.91582511692857,
          49.74477607417078,
          50.43547530325556,
          50.99557633668802,
          51.460101718337484,
          51.875014512230635,
          52.2382735670106,
          52.5474769292266,
          52.785912669371875,
          52.95084471542189,
          53.08021979888443,
          53.19737254856705,
          53.323254511002794,
          53.42801781718497,
          53.510220146406915,
          53.58945124963068,
          53.61254771337104,
          53.62533132070481,
          53.66667420895877,
          53.71939142023197,
          53.75799493356266,
          53.779237457908316,
          53.80865926592188,
          53.83605070540048,
          53.86501410870763,
          53.886265633225506,
          53.8809801337087,
          53.884569423712065
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.885790317876932,
          "title": "",
          "x": 0.9943757030371203,
          "y": 0.5439814814814815
         },
         "legendgroup": "Smoothed q",
         "line": {
          "color": "rgba(227, 111, 71, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Smoothed q",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.09093161908405922,
          2.576075130934778,
          5.050372070372813,
          7.5107808529693685,
          9.9544626311752,
          12.377298682642465,
          14.773740878793777,
          17.137340534246366,
          19.46114554555089,
          21.737971760152398,
          23.96095288235943,
          26.12409446079012,
          28.222816925681293,
          30.254847072370247,
          32.22176634577417,
          34.129047737731284,
          35.950017489851035,
          37.67991619960537,
          39.31455783764501,
          40.850674627849486,
          42.28584783724031,
          43.61868395821497,
          44.848855914263666,
          45.976858787556,
          47.003982656778334,
          47.9322006707627,
          48.764373770075956,
          49.50445779002067,
          50.15726498508232,
          50.728170252790214,
          51.223059048962696,
          51.648434044451086,
          52.01139160598544,
          52.31933767465385,
          52.57926152931159,
          52.79755682966158,
          52.98028264960206,
          53.138531626350094,
          53.27411008049722,
          53.38931252681684,
          53.48676897160287,
          53.56898863371275,
          53.63821049909828,
          53.69653798625737,
          53.74601613991268,
          53.788431129490476,
          53.82508744541845,
          53.856781384762144,
          53.88422932261284,
          53.90842337568725,
          53.93068327921643
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 300,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1.05,
         "xanchor": "lower",
         "y": 0.07,
         "yanchor": "right"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 700,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.05284651918510186,
          0.9943757030371203
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.10108632254301551,
          0.9868766404199475
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.6180085361345604,
          55.54860634179947
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"6408a1a5-3ea2-4c4c-a49b-25b433fefe76\" style=\"width:700px;height:300px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('6408a1a5-3ea2-4c4c-a49b-25b433fefe76', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5439814814814815,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.885790317876932,\n",
       "            \"x\": 0.9943757030371203\n",
       "        },\n",
       "        \"name\": \"q_exp (estimated)\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"q_exp (estimated)\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            -8.547355152461484e-5,\n",
       "            2.4814107896581175,\n",
       "            4.967579586702007,\n",
       "            7.47404948542323,\n",
       "            9.961362883953916,\n",
       "            12.432971958293201,\n",
       "            14.92577487064249,\n",
       "            17.40484795364266,\n",
       "            19.84464461273799,\n",
       "            22.27918475649282,\n",
       "            24.698399663466496,\n",
       "            27.096279788111115,\n",
       "            29.453840504843374,\n",
       "            31.754458202536718,\n",
       "            33.99849169798882,\n",
       "            36.180685952564865,\n",
       "            38.31255019624902,\n",
       "            40.35868007842807,\n",
       "            42.221570358298756,\n",
       "            43.90260295011116,\n",
       "            45.42113425176046,\n",
       "            46.77899152199593,\n",
       "            47.93964494404699,\n",
       "            48.91582511692857,\n",
       "            49.74477607417078,\n",
       "            50.43547530325556,\n",
       "            50.99557633668802,\n",
       "            51.460101718337484,\n",
       "            51.875014512230635,\n",
       "            52.2382735670106,\n",
       "            52.5474769292266,\n",
       "            52.785912669371875,\n",
       "            52.95084471542189,\n",
       "            53.08021979888443,\n",
       "            53.19737254856705,\n",
       "            53.323254511002794,\n",
       "            53.42801781718497,\n",
       "            53.510220146406915,\n",
       "            53.58945124963068,\n",
       "            53.61254771337104,\n",
       "            53.62533132070481,\n",
       "            53.66667420895877,\n",
       "            53.71939142023197,\n",
       "            53.75799493356266,\n",
       "            53.779237457908316,\n",
       "            53.80865926592188,\n",
       "            53.83605070540048,\n",
       "            53.86501410870763,\n",
       "            53.886265633225506,\n",
       "            53.8809801337087,\n",
       "            53.884569423712065\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5439814814814815,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.885790317876932,\n",
       "            \"x\": 0.9943757030371203\n",
       "        },\n",
       "        \"name\": \"Smoothed q\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Smoothed q\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.09093161908405922,\n",
       "            2.576075130934778,\n",
       "            5.050372070372813,\n",
       "            7.5107808529693685,\n",
       "            9.9544626311752,\n",
       "            12.377298682642465,\n",
       "            14.773740878793777,\n",
       "            17.137340534246366,\n",
       "            19.46114554555089,\n",
       "            21.737971760152398,\n",
       "            23.96095288235943,\n",
       "            26.12409446079012,\n",
       "            28.222816925681293,\n",
       "            30.254847072370247,\n",
       "            32.22176634577417,\n",
       "            34.129047737731284,\n",
       "            35.950017489851035,\n",
       "            37.67991619960537,\n",
       "            39.31455783764501,\n",
       "            40.850674627849486,\n",
       "            42.28584783724031,\n",
       "            43.61868395821497,\n",
       "            44.848855914263666,\n",
       "            45.976858787556,\n",
       "            47.003982656778334,\n",
       "            47.9322006707627,\n",
       "            48.764373770075956,\n",
       "            49.50445779002067,\n",
       "            50.15726498508232,\n",
       "            50.728170252790214,\n",
       "            51.223059048962696,\n",
       "            51.648434044451086,\n",
       "            52.01139160598544,\n",
       "            52.31933767465385,\n",
       "            52.57926152931159,\n",
       "            52.79755682966158,\n",
       "            52.98028264960206,\n",
       "            53.138531626350094,\n",
       "            53.27411008049722,\n",
       "            53.38931252681684,\n",
       "            53.48676897160287,\n",
       "            53.56898863371275,\n",
       "            53.63821049909828,\n",
       "            53.69653798625737,\n",
       "            53.74601613991268,\n",
       "            53.788431129490476,\n",
       "            53.82508744541845,\n",
       "            53.856781384762144,\n",
       "            53.88422932261284,\n",
       "            53.90842337568725,\n",
       "            53.93068327921643\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.05284651918510186,\n",
       "            0.9943757030371203\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 300,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.6180085361345604,\n",
       "            55.54860634179947\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.10108632254301551,\n",
       "            0.9868766404199475\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"right\",\n",
       "        \"xanchor\": \"lower\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.05\n",
       "    },\n",
       "    \"width\": 700\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 203,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scatter(t, q_train, xlabel = \"time (min)\", ylabel = \"q (mg/L)\", label = \"q_exp (estimated)\")\n",
    "plot!(t, q_smooth[:], xlabel = \"time (min)\",\n",
    "ylabel = \"Concentration (mg/L)\", legend=:outerbottomright, label = \"Smoothed q\", size = (700, 300))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ce5d434",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Setting up learning problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "id": "cf185b19",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×51 Matrix{Float64}:\n",
       "  0.176155   0.461967  0.242618  0.234628  …  0.999947  1.00013   0.999656\n",
       " -1.5817e-6  0.045919  0.091926  0.138309     0.997175  0.997077  0.997144"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y = dq_dt_est[:]; #target\n",
    "X = [qmax*k*c_exp./(1.0 .+ k*c_exp) q_train]'/qstar_max; #Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "id": "26f9321f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "ɼ_fast = Lux.Chain(Lux.Dense(2, 8, tanh), Lux.Dense(8, 10, tanh),\n",
    "                  Lux.Dense(10, 1)); #Uptake rake"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "id": "92e77af4",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "using Random;\n",
    "rng = Random.default_rng(222)\n",
    "pinit_fast, st_fast = Lux.setup(rng, ɼ_fast);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "id": "102c9a81",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loss_fast (generic function with 1 method)"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predicting and loss function\n",
    "function predict_fast(p)\n",
    "        Y_hat = ɼ_fast(X, p, st_fast)[1][:]\n",
    "        Y_hat\n",
    "end\n",
    "\n",
    "function loss_fast(p)\n",
    "         Y_hat = predict_fast(p)\n",
    "         loss = sum(abs2, (Y_hat .- Y))\n",
    "         return loss\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "id": "55e0b278",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "132.19156537433574"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss_fast(pinit_fast) #testing initial loss"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbec8aad",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Continuing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "id": "2197f119",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "adtype = Optimization.AutoZygote();\n",
    "optf_fast = Optimization.OptimizationFunction((x,p) -> loss_fast(x), adtype);\n",
    "optprob_fast = Optimization.OptimizationProblem(optf_fast, ComponentArray(pinit_fast));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "b451514f",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "#55 (generic function with 1 method)"
      ]
     },
     "execution_count": 198,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "losses_fast = []\n",
    "callback_fast = function (p, l)\n",
    "    push!(losses_fast, l)\n",
    "    if length(losses_fast) % 100 == 0\n",
    "        println(\"Current loss after $(length(losses_fast)) iterations: $(losses_fast[end])\")\n",
    "    end\n",
    "    return false\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "id": "096ba1db",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current loss after 100 iterations: 4.378522721813006\n",
      "Current loss after 200 iterations: 1.6674762088658721\n",
      "Current loss after 300 iterations: 0.5836468296748255\n",
      "Current loss after 400 iterations: 0.33954833406485174\n",
      "Current loss after 500 iterations: 0.23219823413579016\n",
      " 14.679808 seconds (32.51 M allocations: 1.927 GiB, 5.03% gc time, 98.57% compilation time)\n"
     ]
    }
   ],
   "source": [
    "@time res_fast = Optimization.solve(optprob_fast, ADAM(1e-2), callback = callback_fast, maxiters = 500);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "id": "54d167ae",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0mComponentVector{Float32,SubArray...}(weight = Float32[-0.3489829 -0.11738045 … -0.38437322 0.68859684], bias = Float32[0.009310756;;])"
      ]
     },
     "execution_count": 200,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p_modified = copy(ComponentArray(pinit))\n",
    "p_modified.layer_2 = res_fast.u.layer_1\n",
    "p_modified.layer_3 = res_fast.u.layer_2\n",
    "p_modified.layer_4 = res_fast.u.layer_3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "id": "34b261dc",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.9238602119179546,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.529320987654321
         },
         "legendgroup": "First Stage UDE",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "First Stage UDE",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0,
          0.0828229726718144,
          0.14726531748445526,
          0.22253409219055278,
          0.31085970746540165,
          0.415000068269376,
          0.5383791917322911,
          0.6850241465506676,
          0.8600580498587933,
          1.0698898752018828,
          1.3216139132017508,
          1.6252532250609353,
          1.9905740129175362,
          2.4313337617513255,
          2.9592819456959525,
          3.58966631878375,
          4.331227045260835,
          5.193908045785073,
          6.172267241362913,
          7.257797708300421,
          8.425987227964244,
          9.637333255268079,
          10.855233437092366,
          12.038472811672392,
          13.146988337542657,
          14.162912023337233,
          15.06589917780104,
          15.858157253785143,
          16.538805200931833,
          17.11992105604304,
          17.609501280070038,
          18.02078819319933,
          18.36359292620602,
          18.648982452717156,
          18.885470669161702,
          19.081366823873662,
          19.243128351655926,
          19.376789988449804,
          19.486938205514225,
          19.57775055526326,
          19.652609056406515,
          19.714150697726993,
          19.764889082605563,
          19.806645159207264,
          19.84093831987095,
          19.86921608957312,
          19.89247926597457,
          19.91155725352757,
          19.927265670629197,
          19.940200807009585,
          19.950826027983656
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.9238602119179546,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.529320987654321
         },
         "legendgroup": "observation",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "observation",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.1149144414942044,
          0.453593327427082,
          0.17166903204340475,
          0.16434298525027757,
          0.43082886105468254,
          0.39036313080466295,
          0.1498263562138032,
          0.6049853300852165,
          0.7288340742910437,
          0.7208278900541426,
          0.9725557011832456,
          1.0645132432308824,
          1.605832870402756,
          1.9623017325357597,
          2.488345947495106,
          2.9372823345460555,
          3.3004647312077497,
          4.44026467007553,
          6.014621388907804,
          7.332105354195911,
          8.600968893100676,
          9.879855941342193,
          11.70106015407636,
          12.820166371191963,
          14.011211986433725,
          15.036464869109007,
          16.06275295941086,
          16.493576791018565,
          16.929248776222778,
          17.310492828215327,
          17.808197157388648,
          18.412709295322795,
          18.90649223578161,
          19.02591785290369,
          19.112996811420775,
          18.883545295888464,
          19.46450909951882,
          19.24859182601517,
          19.527825543706353,
          20.0553701728416,
          19.714848868287742,
          19.63114088016073,
          19.53588580560029,
          19.85638399603698,
          19.803292061991808,
          19.732583577342563,
          19.847534725209762,
          19.71101197633774,
          19.96119154981576,
          20.094516579726115,
          19.74853945973233
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 450,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1.05,
         "xanchor": "lower",
         "y": 0.07,
         "yanchor": "right"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 800,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.046240704286964134,
          0.9950787401574803
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.06739088169534369,
          0.9912510936132983
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.6028354973917835,
          20.6973520771179
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "5",
          "10",
          "15",
          "20"
         ],
         "tickvals": [
          0,
          5,
          10,
          15,
          20
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"ea67e4c1-91d4-4f16-b3ca-5467d9ebaeaf\" style=\"width:800px;height:450px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('ea67e4c1-91d4-4f16-b3ca-5467d9ebaeaf', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.529320987654321,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.9238602119179546,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"First Stage UDE\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"First Stage UDE\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            0.0828229726718144,\n",
       "            0.14726531748445526,\n",
       "            0.22253409219055278,\n",
       "            0.31085970746540165,\n",
       "            0.415000068269376,\n",
       "            0.5383791917322911,\n",
       "            0.6850241465506676,\n",
       "            0.8600580498587933,\n",
       "            1.0698898752018828,\n",
       "            1.3216139132017508,\n",
       "            1.6252532250609353,\n",
       "            1.9905740129175362,\n",
       "            2.4313337617513255,\n",
       "            2.9592819456959525,\n",
       "            3.58966631878375,\n",
       "            4.331227045260835,\n",
       "            5.193908045785073,\n",
       "            6.172267241362913,\n",
       "            7.257797708300421,\n",
       "            8.425987227964244,\n",
       "            9.637333255268079,\n",
       "            10.855233437092366,\n",
       "            12.038472811672392,\n",
       "            13.146988337542657,\n",
       "            14.162912023337233,\n",
       "            15.06589917780104,\n",
       "            15.858157253785143,\n",
       "            16.538805200931833,\n",
       "            17.11992105604304,\n",
       "            17.609501280070038,\n",
       "            18.02078819319933,\n",
       "            18.36359292620602,\n",
       "            18.648982452717156,\n",
       "            18.885470669161702,\n",
       "            19.081366823873662,\n",
       "            19.243128351655926,\n",
       "            19.376789988449804,\n",
       "            19.486938205514225,\n",
       "            19.57775055526326,\n",
       "            19.652609056406515,\n",
       "            19.714150697726993,\n",
       "            19.764889082605563,\n",
       "            19.806645159207264,\n",
       "            19.84093831987095,\n",
       "            19.86921608957312,\n",
       "            19.89247926597457,\n",
       "            19.91155725352757,\n",
       "            19.927265670629197,\n",
       "            19.940200807009585,\n",
       "            19.950826027983656\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.529320987654321,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.9238602119179546,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"observation\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"observation\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.1149144414942044,\n",
       "            0.453593327427082,\n",
       "            0.17166903204340475,\n",
       "            0.16434298525027757,\n",
       "            0.43082886105468254,\n",
       "            0.39036313080466295,\n",
       "            0.1498263562138032,\n",
       "            0.6049853300852165,\n",
       "            0.7288340742910437,\n",
       "            0.7208278900541426,\n",
       "            0.9725557011832456,\n",
       "            1.0645132432308824,\n",
       "            1.605832870402756,\n",
       "            1.9623017325357597,\n",
       "            2.488345947495106,\n",
       "            2.9372823345460555,\n",
       "            3.3004647312077497,\n",
       "            4.44026467007553,\n",
       "            6.014621388907804,\n",
       "            7.332105354195911,\n",
       "            8.600968893100676,\n",
       "            9.879855941342193,\n",
       "            11.70106015407636,\n",
       "            12.820166371191963,\n",
       "            14.011211986433725,\n",
       "            15.036464869109007,\n",
       "            16.06275295941086,\n",
       "            16.493576791018565,\n",
       "            16.929248776222778,\n",
       "            17.310492828215327,\n",
       "            17.808197157388648,\n",
       "            18.412709295322795,\n",
       "            18.90649223578161,\n",
       "            19.02591785290369,\n",
       "            19.112996811420775,\n",
       "            18.883545295888464,\n",
       "            19.46450909951882,\n",
       "            19.24859182601517,\n",
       "            19.527825543706353,\n",
       "            20.0553701728416,\n",
       "            19.714848868287742,\n",
       "            19.63114088016073,\n",
       "            19.53588580560029,\n",
       "            19.85638399603698,\n",
       "            19.803292061991808,\n",
       "            19.732583577342563,\n",
       "            19.847534725209762,\n",
       "            19.71101197633774,\n",
       "            19.96119154981576,\n",
       "            20.094516579726115,\n",
       "            19.74853945973233\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.046240704286964134,\n",
       "            0.9950787401574803\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 450,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            5.0,\n",
       "            10.0,\n",
       "            15.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.6028354973917835,\n",
       "            20.6973520771179\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.06739088169534369,\n",
       "            0.9912510936132983\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"5\",\n",
       "            \"10\",\n",
       "            \"15\",\n",
       "            \"20\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"right\",\n",
       "        \"xanchor\": \"lower\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.05\n",
       "    },\n",
       "    \"width\": 800\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 202,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot(t, predict(p_modified, 1), label = \"First Stage UDE\", size = (800, 450))\n",
    "scatter!(train_data[:, 1], train_data[:, 2], xlabel = \"time (min)\",  ylabel = \"Concentration (mg/L)\", label = \"observation\", legend=:outerbottomright)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ae98d68",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Conclusions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03104e21",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "* The Universal Differential Equation (UDE) framework serves as a powerful tool for enhancing the capabilities of basic mathematical models, allowing them to better fit data.\n",
    "* We explored a straightforward implementation of UDE for estimating uptake kinetics in adsorption processes.\n",
    "* Additionally, we delved into the application of a fast, two-stage training methodology that employs Kernel smoothing techniques to warm up our ANN parameters.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d48ddab",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Thank you!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a448e715",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Feel free to contact me if you want to know more about it:\n",
    "    \n",
    "Email: vinicius.santana@ntnu.no\n",
    "\n",
    "Code and files are available at:"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Julia 1.9.0",
   "language": "julia",
   "name": "julia-1.9"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}