NorwAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "efd013e1",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center> <h1>Scientific Machine Learning: Bridging Domain Knowledge and Artificial Intelligence for Sustainable Industrial Transition - Part 2: Hands on\n",
    "</h1> </center>  \n",
    "\n",
    "<center> <h3>Vinicius Viena Santana\n",
    "    </h3>\n",
    "</center>  \n",
    "\n",
    "<center> <h3>Norwegian University of Science and Technology, Department of Chemical Engineering, PSE Group, Idelfonso´s lab </h3></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1141ba67",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Background**\n",
    "- Bachelor and Master's degree in industrial engineering from the Federal University of Bahia, Brazil (Supervised by Idelfonso Nogueira). 🎓\n",
    "- Ph.D. at the University of Porto, Portugal (Supervised by Idelfonso Nogueira, Chris Rackauckas and Ana Ribeiro) 📚"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e30319a5",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**About me**\n",
    "\n",
    "Enthusiastic about coding and the Julia programming language ❤️"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e69954b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Research**\n",
    "- Currently a Postdoctoral researcher at NTNU, Norway (PSE Group, Idelfonso's lab)\n",
    "- Research focus: Scientific Machine Learning and AI applied to industrial and chemical engineering 📈"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc92e79a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Modeling in Natural Sciences\n",
    "\n",
    "Building simple and powerful models is time consuming and expensive 💰. Yet, researchers/engineers dedicate a great fraction of their time to it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c056d21",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Models compete for acceptance. Should we favor rigor or simplicity, exactness or usefulness? "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d325103",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# An engineering problem: Two approaches"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "935d6e44",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "A waste treatment plant operates a process where contaminated water flows through a filtration system to remove these impurities. Developing a mathematical model this process is important.\n",
    "\n",
    "\n",
    "With it, we can answer questions such as:\n",
    "\n",
    "* **How much time it takes for the outlet concentration to reach a certain value?**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b12016e5",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The contaminant binds to the particles and are removed from the water stream.\n",
    "\n",
    "<center><img src=\"contaminant.svg\" width=\"450\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4843bb83",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<center><img src=\"models.png\" width=\"1000\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb755e6f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "How can one build this model? \n",
    "\n",
    "* First we need data!\n",
    "* We propose a model with unknown parameters.\n",
    "* We fit the data and check the fitting quality.\n",
    "* Finally, we use our model to control/design our unit. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ee9dd7b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Trying to solve the problem in a complete ''Data Driven\" way"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78e66681",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The data was collected where flow rate (Q) is 5.0 L/min and inlet concentration (Cin) is 20 mg/L:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 408,
   "id": "c461e681",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.885790317876932,
          "title": "",
          "x": 0.9921259842519685,
          "y": 0.5439814814814815
         },
         "legendgroup": "Outlet c",
         "marker": {
          "color": "rgba(0, 154, 250, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Outlet c",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.1149144414942044,
          0.453593327427082,
          0.17166903204340475,
          0.16434298525027757,
          0.43082886105468254,
          0.39036313080466295,
          0.1498263562138032,
          0.6049853300852165,
          0.7288340742910437,
          0.7208278900541426,
          0.9725557011832456,
          1.0645132432308824,
          1.605832870402756,
          1.9623017325357597,
          2.488345947495106,
          2.9372823345460555,
          3.3004647312077497,
          4.44026467007553,
          6.014621388907804,
          7.332105354195911,
          8.600968893100676,
          9.879855941342193,
          11.70106015407636,
          12.820166371191963,
          14.011211986433725,
          15.036464869109007,
          16.06275295941086,
          16.493576791018565,
          16.929248776222778,
          17.310492828215327,
          17.808197157388648,
          18.412709295322795,
          18.90649223578161,
          19.02591785290369,
          19.112996811420775,
          18.883545295888464,
          19.46450909951882,
          19.24859182601517,
          19.527825543706353,
          20.0553701728416,
          19.714848868287742,
          19.63114088016073,
          19.53588580560029,
          19.85638399603698,
          19.803292061991808,
          19.732583577342563,
          19.847534725209762,
          19.71101197633774,
          19.96119154981576,
          20.094516579726115,
          19.74853945973233
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.885790317876932,
          "title": "",
          "x": 0.9921259842519685,
          "y": 0.5439814814814815
         },
         "legendgroup": "Inlet c",
         "line": {
          "color": "rgba(227, 111, 71, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Inlet c",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20,
          20
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 300,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1,
         "xanchor": "right",
         "y": 0.07,
         "yanchor": "bottom"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 500,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.0739851268591426,
          0.9921259842519685
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.10108632254301551,
          0.9868766404199475
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.48447362265275373,
          20.693904643873072
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "5",
          "10",
          "15",
          "20"
         ],
         "tickvals": [
          0,
          5,
          10,
          15,
          20
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"81119101-01b4-4719-ad46-e8c3c8eb1691\" style=\"width:500px;height:300px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('81119101-01b4-4719-ad46-e8c3c8eb1691', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5439814814814815,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.885790317876932,\n",
       "            \"x\": 0.9921259842519685\n",
       "        },\n",
       "        \"name\": \"Outlet c\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Outlet c\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.1149144414942044,\n",
       "            0.453593327427082,\n",
       "            0.17166903204340475,\n",
       "            0.16434298525027757,\n",
       "            0.43082886105468254,\n",
       "            0.39036313080466295,\n",
       "            0.1498263562138032,\n",
       "            0.6049853300852165,\n",
       "            0.7288340742910437,\n",
       "            0.7208278900541426,\n",
       "            0.9725557011832456,\n",
       "            1.0645132432308824,\n",
       "            1.605832870402756,\n",
       "            1.9623017325357597,\n",
       "            2.488345947495106,\n",
       "            2.9372823345460555,\n",
       "            3.3004647312077497,\n",
       "            4.44026467007553,\n",
       "            6.014621388907804,\n",
       "            7.332105354195911,\n",
       "            8.600968893100676,\n",
       "            9.879855941342193,\n",
       "            11.70106015407636,\n",
       "            12.820166371191963,\n",
       "            14.011211986433725,\n",
       "            15.036464869109007,\n",
       "            16.06275295941086,\n",
       "            16.493576791018565,\n",
       "            16.929248776222778,\n",
       "            17.310492828215327,\n",
       "            17.808197157388648,\n",
       "            18.412709295322795,\n",
       "            18.90649223578161,\n",
       "            19.02591785290369,\n",
       "            19.112996811420775,\n",
       "            18.883545295888464,\n",
       "            19.46450909951882,\n",
       "            19.24859182601517,\n",
       "            19.527825543706353,\n",
       "            20.0553701728416,\n",
       "            19.714848868287742,\n",
       "            19.63114088016073,\n",
       "            19.53588580560029,\n",
       "            19.85638399603698,\n",
       "            19.803292061991808,\n",
       "            19.732583577342563,\n",
       "            19.847534725209762,\n",
       "            19.71101197633774,\n",
       "            19.96119154981576,\n",
       "            20.094516579726115,\n",
       "            19.74853945973233\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5439814814814815,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.885790317876932,\n",
       "            \"x\": 0.9921259842519685\n",
       "        },\n",
       "        \"name\": \"Inlet c\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Inlet c\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.0739851268591426,\n",
       "            0.9921259842519685\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 300,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            5.0,\n",
       "            10.0,\n",
       "            15.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.48447362265275373,\n",
       "            20.693904643873072\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.10108632254301551,\n",
       "            0.9868766404199475\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"5\",\n",
       "            \"10\",\n",
       "            \"15\",\n",
       "            \"20\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"bottom\",\n",
       "        \"xanchor\": \"right\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.0\n",
       "    },\n",
       "    \"width\": 500\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 408,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data = readdlm(\"observations.csv\", ',')\n",
    "Cin = 20\n",
    "Q = 5\n",
    "scatter(train_data[:, 1], train_data[:, 2], xlabel = \"time (min)\", ylabel = \"Concentration (mg/L)\", label = \"Outlet c\")\n",
    "\n",
    "plot!(train_data[:, 1], ones(size(train_data, 1))*Cin, xlabel = \"time (min)\",\n",
    "        ylabel = \"Concentration (mg/L)\", label = \"Inlet c\", size = (500, 300), legend = :bottomright)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbc85cff",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "💡 Ok! I can fit a model with this data. \n",
    "\n",
    "\n",
    "Whenever someone needs to estimate the time it takes for the contaminat concentration to reach a certain value, I can do it.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e475589",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "I will use a neural networks because it can approximate any function. Inputs can be $t$, $Cin$ and $Q$ and my outputs will be the concentration $C$. \n",
    "\n",
    "That means I want to find a function $f$ that establish this relationship:\n",
    "\n",
    "$f: (Cin, Q, t)$ $\\rightarrow$ $C$ or better $f$: t $\\rightarrow$ $C$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e3dcbc1",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Hands on coding in Julia"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 445,
   "id": "b78d75b1",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Chain(\n",
       "    layer_1 = Dense(1 => 50, sigmoid_fast),  \u001b[90m# 100 parameters\u001b[39m\n",
       "    layer_2 = Dense(50 => 50, sigmoid_fast),  \u001b[90m# 2_550 parameters\u001b[39m\n",
       "    layer_3 = Dense(50 => 1),           \u001b[90m# 51 parameters\u001b[39m\n",
       ") \u001b[90m        # Total: \u001b[39m2_701 parameters,\n",
       "\u001b[90m          #        plus \u001b[39m0 states."
      ]
     },
     "execution_count": 445,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Lux, Statistics, Optimisers, Zygote, ADTypes, Printf\n",
    "\n",
    "C = Chain(Dense(1, 50, sigmoid), Dense(50, 50, sigmoid), Dense(50, 1)) #Initializing NN structure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 446,
   "id": "ccac22d1",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "using Random;\n",
    "rng = Random.default_rng(222)\n",
    "pinit, st = Lux.setup(rng, C);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 456,
   "id": "493ec2d5",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "scaled_data = (train_data .- minimum(train_data, dims = 1))./(maximum(train_data, dims = 1) .- minimum(train_data, dims = 1))\n",
    "    \n",
    "train_index = Int(round(.7*size(scaled_data, 1)))\n",
    "data = scaled_data[1:train_index, :];\n",
    "test_data =  scaled_data[train_index:end, :];"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 457,
   "id": "02522017",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loss (generic function with 2 methods)"
      ]
     },
     "execution_count": 457,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function loss(model, ps, st, data)\n",
    "    y_pred, st = Lux.apply(model, data[1:1, :], ps, st)\n",
    "    mse_loss = mean(abs2, y_pred .- data[2:2, :])\n",
    "    return mse_loss, st, ()\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 458,
   "id": "aed85505",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.13481271325277333, (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple()), ())"
      ]
     },
     "execution_count": 458,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss(C, pinit, st, data')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 459,
   "id": "d6ead833",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AutoZygote()"
      ]
     },
     "execution_count": 459,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "opt = Adam(2e-3)\n",
    "tstate = Lux.Experimental.TrainState(rng, C, opt)\n",
    "vjp_rule = AutoZygote()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 460,
   "id": "63f3cb4c",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "function train(tstate::Lux.Experimental.TrainState, vjp, data, epochs)\n",
    "    for epoch in 1:epochs\n",
    "        grads, loss_val, stats, tstate = Lux.Experimental.compute_gradients(\n",
    "            vjp, loss, data, tstate)\n",
    "        if epoch % 500 == 1 || epoch == epochs\n",
    "            @printf \"Epoch: %3d \\t Loss: %.5g\\n\" epoch loss_val\n",
    "        end\n",
    "        tstate = Lux.Experimental.apply_gradients(tstate, grads)\n",
    "    end\n",
    "    return tstate\n",
    "end\n",
    "\n",
    "tstate = Lux.Experimental.TrainState(rng, C, opt);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 461,
   "id": "88e03817",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch:   1 \t Loss: 0.75181\n",
      "Epoch: 501 \t Loss: 0.010638\n",
      "Epoch: 1001 \t Loss: 0.0097962\n",
      "Epoch: 1501 \t Loss: 0.005072\n",
      "Epoch: 2001 \t Loss: 0.0003627\n",
      "Epoch: 2501 \t Loss: 0.00016123\n",
      "Epoch: 3001 \t Loss: 0.00012634\n",
      "Epoch: 3501 \t Loss: 0.00010523\n",
      "Epoch: 4001 \t Loss: 9.4122e-05\n",
      "Epoch: 4501 \t Loss: 8.8831e-05\n",
      "Epoch: 5000 \t Loss: 8.6489e-05\n"
     ]
    }
   ],
   "source": [
    "tstate = train(tstate, vjp_rule, data', 5000);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 462,
   "id": "2df7b3e8",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "ŷ = C(data[:, 1]', tstate.parameters, tstate.states)[1]\n",
    "ŷ_test = C(test_data[:, 1]', tstate.parameters, tstate.states)[1];"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 463,
   "id": "907787f8",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.936716243802858,
          "title": "",
          "x": 0.9921259842519685,
          "y": 0.5185185185185185
         },
         "legendgroup": "train prediction",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "train prediction",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          0.02,
          0.04,
          0.06,
          0.08,
          0.1,
          0.12,
          0.14,
          0.16,
          0.18,
          0.2,
          0.22,
          0.24,
          0.26,
          0.28,
          0.3,
          0.32,
          0.34,
          0.36,
          0.38,
          0.4,
          0.42,
          0.44,
          0.46,
          0.48,
          0.5,
          0.52,
          0.54,
          0.56,
          0.58,
          0.6,
          0.62,
          0.64,
          0.66,
          0.68,
          0.7
         ],
         "xaxis": "x",
         "y": [
          -0.0006954666418632419,
          0.0014842592294414111,
          0.003869933070097039,
          0.006511723880687551,
          0.009478650093085269,
          0.012865962923143712,
          0.016805239381413123,
          0.021477999873386895,
          0.027133730816800265,
          0.0341130545890173,
          0.04287619001429518,
          0.0540353330941486,
          0.06838645370135972,
          0.08693036358247791,
          0.11086418092141846,
          0.14151359773722144,
          0.18016910529108432,
          0.2277982889889979,
          0.28464963121443004,
          0.34984855413563504,
          0.4211792675735139,
          0.4952564406878215,
          0.568138840122672,
          0.6361790307703588,
          0.6967439684366407,
          0.7485308612229243,
          0.7914545490836697,
          0.8262810090704813,
          0.8542162773270711,
          0.8765819022193783,
          0.8946147730238204,
          0.9093728760508746,
          0.9217109239731555,
          0.9322936052805718,
          0.9416245275280042,
          0.9500783920893868
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.936716243802858,
          "title": "",
          "x": 0.9921259842519685,
          "y": 0.5185185185185185
         },
         "legendgroup": "train data",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "train data",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          0.02,
          0.04,
          0.06,
          0.08,
          0.1,
          0.12,
          0.14,
          0.16,
          0.18,
          0.2,
          0.22,
          0.24,
          0.26,
          0.28,
          0.3,
          0.32,
          0.34,
          0.36,
          0.38,
          0.4,
          0.42,
          0.44,
          0.46,
          0.48,
          0.5,
          0.52,
          0.54,
          0.56,
          0.58,
          0.6,
          0.62,
          0.64,
          0.66,
          0.68,
          0.7
         ],
         "xaxis": "x",
         "y": [
          0,
          0.016951232741757135,
          0.002840626662960308,
          0.002473950352669402,
          0.015811847371873387,
          0.013786495216707767,
          0.0017473778745970717,
          0.024528560939321126,
          0.030727320221360925,
          0.03032660232009797,
          0.04292584275479161,
          0.04752841398776284,
          0.07462202793596219,
          0.09246366760759928,
          0.11879273118553389,
          0.1412624672666087,
          0.1594401263685749,
          0.2164883063564396,
          0.2952865080393284,
          0.3612279595343529,
          0.42473590779708303,
          0.4887455431938914,
          0.5798987203259428,
          0.6359111578796487,
          0.695524237609732,
          0.7468392175368554,
          0.7982060106892578,
          0.8197691944116855,
          0.8415750332962613,
          0.8606566971529716,
          0.8855673197834862,
          0.9158237850399883,
          0.9405381380607593,
          0.9465155152024969,
          0.9508739082232696,
          0.9393896197001641
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.936716243802858,
          "title": "",
          "x": 0.9921259842519685,
          "y": 0.5185185185185185
         },
         "legendgroup": "test prediction",
         "line": {
          "color": "rgba(62, 164, 78, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "test prediction",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0.7,
          0.72,
          0.74,
          0.76,
          0.78,
          0.8,
          0.82,
          0.84,
          0.86,
          0.88,
          0.9,
          0.92,
          0.94,
          0.96,
          0.98,
          1
         ],
         "xaxis": "x",
         "y": [
          0.9500783920893868,
          0.957930439898591,
          0.9653809940479058,
          0.9725748496752237,
          0.9796160913073976,
          0.98657917337045,
          0.9935170875571163,
          1.0004673247139138,
          1.007456198430937,
          1.014501966941122,
          1.0216170810501866,
          1.0288098000831154,
          1.0360853525843559,
          1.043446769907569,
          1.050895485135238,
          1.0584317637970422
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.936716243802858,
          "title": "",
          "x": 0.9921259842519685,
          "y": 0.5185185185185185
         },
         "legendgroup": "test data",
         "marker": {
          "color": "rgba(195, 113, 210, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "test data",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0.7,
          0.72,
          0.74,
          0.76,
          0.78,
          0.8,
          0.82,
          0.84,
          0.86,
          0.88,
          0.9,
          0.92,
          0.94,
          0.96,
          0.98,
          1
         ],
         "xaxis": "x",
         "y": [
          0.9393896197001641,
          0.9684674661763287,
          0.9576605806332735,
          0.9716365204822887,
          0.9980406813602355,
          0.9809972336380082,
          0.9768075612137093,
          0.9720399450268903,
          0.9880812149290271,
          0.985423908057857,
          0.9818848743894166,
          0.9876382996614461,
          0.9808051931797721,
          0.9933269426994629,
          1,
          0.9826834830043117
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 300,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1,
         "xanchor": "right",
         "y": 0.07,
         "yanchor": "bottom"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 500,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.07898512685914261,
          0.9921259842519685
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.030000000000000027,
          1.03
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0.00",
          "0.25",
          "0.50",
          "0.75",
          "1.00"
         ],
         "tickvals": [
          0,
          0.25,
          0.5,
          0.75,
          1
         ],
         "title": "",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.050160396617089535,
          0.9868766404199475
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.032469283555030426,
          1.0902055807102093
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0.00",
          "0.25",
          "0.50",
          "0.75",
          "1.00"
         ],
         "tickvals": [
          0,
          0.25,
          0.5,
          0.75,
          1
         ],
         "title": "",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"99b9a8b2-85d6-411a-9a91-1f958a16734a\" style=\"width:500px;height:300px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('99b9a8b2-85d6-411a-9a91-1f958a16734a', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5185185185185185,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.936716243802858,\n",
       "            \"x\": 0.9921259842519685\n",
       "        },\n",
       "        \"name\": \"train prediction\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"train prediction\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            -0.0006954666418632419,\n",
       "            0.0014842592294414111,\n",
       "            0.003869933070097039,\n",
       "            0.006511723880687551,\n",
       "            0.009478650093085269,\n",
       "            0.012865962923143712,\n",
       "            0.016805239381413123,\n",
       "            0.021477999873386895,\n",
       "            0.027133730816800265,\n",
       "            0.0341130545890173,\n",
       "            0.04287619001429518,\n",
       "            0.0540353330941486,\n",
       "            0.06838645370135972,\n",
       "            0.08693036358247791,\n",
       "            0.11086418092141846,\n",
       "            0.14151359773722144,\n",
       "            0.18016910529108432,\n",
       "            0.2277982889889979,\n",
       "            0.28464963121443004,\n",
       "            0.34984855413563504,\n",
       "            0.4211792675735139,\n",
       "            0.4952564406878215,\n",
       "            0.568138840122672,\n",
       "            0.6361790307703588,\n",
       "            0.6967439684366407,\n",
       "            0.7485308612229243,\n",
       "            0.7914545490836697,\n",
       "            0.8262810090704813,\n",
       "            0.8542162773270711,\n",
       "            0.8765819022193783,\n",
       "            0.8946147730238204,\n",
       "            0.9093728760508746,\n",
       "            0.9217109239731555,\n",
       "            0.9322936052805718,\n",
       "            0.9416245275280042,\n",
       "            0.9500783920893868\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            0.02,\n",
       "            0.04,\n",
       "            0.06,\n",
       "            0.08,\n",
       "            0.1,\n",
       "            0.12,\n",
       "            0.14,\n",
       "            0.16,\n",
       "            0.18,\n",
       "            0.2,\n",
       "            0.22,\n",
       "            0.24,\n",
       "            0.26,\n",
       "            0.28,\n",
       "            0.3,\n",
       "            0.32,\n",
       "            0.34,\n",
       "            0.36,\n",
       "            0.38,\n",
       "            0.4,\n",
       "            0.42,\n",
       "            0.44,\n",
       "            0.46,\n",
       "            0.48,\n",
       "            0.5,\n",
       "            0.52,\n",
       "            0.54,\n",
       "            0.56,\n",
       "            0.58,\n",
       "            0.6,\n",
       "            0.62,\n",
       "            0.64,\n",
       "            0.66,\n",
       "            0.68,\n",
       "            0.7\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5185185185185185,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.936716243802858,\n",
       "            \"x\": 0.9921259842519685\n",
       "        },\n",
       "        \"name\": \"train data\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"train data\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            0.016951232741757135,\n",
       "            0.002840626662960308,\n",
       "            0.002473950352669402,\n",
       "            0.015811847371873387,\n",
       "            0.013786495216707767,\n",
       "            0.0017473778745970717,\n",
       "            0.024528560939321126,\n",
       "            0.030727320221360925,\n",
       "            0.03032660232009797,\n",
       "            0.04292584275479161,\n",
       "            0.04752841398776284,\n",
       "            0.07462202793596219,\n",
       "            0.09246366760759928,\n",
       "            0.11879273118553389,\n",
       "            0.1412624672666087,\n",
       "            0.1594401263685749,\n",
       "            0.2164883063564396,\n",
       "            0.2952865080393284,\n",
       "            0.3612279595343529,\n",
       "            0.42473590779708303,\n",
       "            0.4887455431938914,\n",
       "            0.5798987203259428,\n",
       "            0.6359111578796487,\n",
       "            0.695524237609732,\n",
       "            0.7468392175368554,\n",
       "            0.7982060106892578,\n",
       "            0.8197691944116855,\n",
       "            0.8415750332962613,\n",
       "            0.8606566971529716,\n",
       "            0.8855673197834862,\n",
       "            0.9158237850399883,\n",
       "            0.9405381380607593,\n",
       "            0.9465155152024969,\n",
       "            0.9508739082232696,\n",
       "            0.9393896197001641\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            0.02,\n",
       "            0.04,\n",
       "            0.06,\n",
       "            0.08,\n",
       "            0.1,\n",
       "            0.12,\n",
       "            0.14,\n",
       "            0.16,\n",
       "            0.18,\n",
       "            0.2,\n",
       "            0.22,\n",
       "            0.24,\n",
       "            0.26,\n",
       "            0.28,\n",
       "            0.3,\n",
       "            0.32,\n",
       "            0.34,\n",
       "            0.36,\n",
       "            0.38,\n",
       "            0.4,\n",
       "            0.42,\n",
       "            0.44,\n",
       "            0.46,\n",
       "            0.48,\n",
       "            0.5,\n",
       "            0.52,\n",
       "            0.54,\n",
       "            0.56,\n",
       "            0.58,\n",
       "            0.6,\n",
       "            0.62,\n",
       "            0.64,\n",
       "            0.66,\n",
       "            0.68,\n",
       "            0.7\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5185185185185185,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.936716243802858,\n",
       "            \"x\": 0.9921259842519685\n",
       "        },\n",
       "        \"name\": \"test prediction\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"test prediction\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(62, 164, 78, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.9500783920893868,\n",
       "            0.957930439898591,\n",
       "            0.9653809940479058,\n",
       "            0.9725748496752237,\n",
       "            0.9796160913073976,\n",
       "            0.98657917337045,\n",
       "            0.9935170875571163,\n",
       "            1.0004673247139138,\n",
       "            1.007456198430937,\n",
       "            1.014501966941122,\n",
       "            1.0216170810501866,\n",
       "            1.0288098000831154,\n",
       "            1.0360853525843559,\n",
       "            1.043446769907569,\n",
       "            1.050895485135238,\n",
       "            1.0584317637970422\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.7,\n",
       "            0.72,\n",
       "            0.74,\n",
       "            0.76,\n",
       "            0.78,\n",
       "            0.8,\n",
       "            0.82,\n",
       "            0.84,\n",
       "            0.86,\n",
       "            0.88,\n",
       "            0.9,\n",
       "            0.92,\n",
       "            0.94,\n",
       "            0.96,\n",
       "            0.98,\n",
       "            1.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5185185185185185,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.936716243802858,\n",
       "            \"x\": 0.9921259842519685\n",
       "        },\n",
       "        \"name\": \"test data\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"test data\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(195, 113, 210, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.9393896197001641,\n",
       "            0.9684674661763287,\n",
       "            0.9576605806332735,\n",
       "            0.9716365204822887,\n",
       "            0.9980406813602355,\n",
       "            0.9809972336380082,\n",
       "            0.9768075612137093,\n",
       "            0.9720399450268903,\n",
       "            0.9880812149290271,\n",
       "            0.985423908057857,\n",
       "            0.9818848743894166,\n",
       "            0.9876382996614461,\n",
       "            0.9808051931797721,\n",
       "            0.9933269426994629,\n",
       "            1.0,\n",
       "            0.9826834830043117\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.7,\n",
       "            0.72,\n",
       "            0.74,\n",
       "            0.76,\n",
       "            0.78,\n",
       "            0.8,\n",
       "            0.82,\n",
       "            0.84,\n",
       "            0.86,\n",
       "            0.88,\n",
       "            0.9,\n",
       "            0.92,\n",
       "            0.94,\n",
       "            0.96,\n",
       "            0.98,\n",
       "            1.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            0.25,\n",
       "            0.5,\n",
       "            0.75,\n",
       "            1.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.030000000000000027,\n",
       "            1.03\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.07898512685914261,\n",
       "            0.9921259842519685\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0.00\",\n",
       "            \"0.25\",\n",
       "            \"0.50\",\n",
       "            \"0.75\",\n",
       "            \"1.00\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 300,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            0.25,\n",
       "            0.5,\n",
       "            0.75,\n",
       "            1.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.032469283555030426,\n",
       "            1.0902055807102093\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.050160396617089535,\n",
       "            0.9868766404199475\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0.00\",\n",
       "            \"0.25\",\n",
       "            \"0.50\",\n",
       "            \"0.75\",\n",
       "            \"1.00\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"bottom\",\n",
       "        \"xanchor\": \"right\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.0\n",
       "    },\n",
       "    \"width\": 500\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 463,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot(data[:, 1], ŷ', label = \"train prediction\")\n",
    "scatter!(data[:, 1], data[:, 2], label = \"train data\", legend = :bottomright)\n",
    "plot!(test_data[:, 1], ŷ_test', label = \"test prediction\")\n",
    "scatter!(test_data[:, 1], test_data[:, 2], label = \"test data\", size = (500, 300))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bceb5bb3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "What if we want to have a model to predict what happens when the inlet is less contaminated, i.e., $Cin < 20.0$?\n",
    "\n",
    "* One may collect more data at several values of $Cin$, and train a bigger model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e12f09d6",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Can we use the same data and physics knowledge to obtain a model that can extrapolate for other contaminant concentration or flow rate?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "915ddbe6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Solving the problem with domain knowledge - Universal Differential Equations [Rackauckas et al. (2020)](https://arxiv.org/abs/2001.04385)\n",
    "\n",
    "~ A small black box into your simple domain specific model ~"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09b0c838",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* The universal approximator aim to reduce mechanistic model non-explained residuals through small mechanistic extensions.\n",
    "\n",
    "\n",
    "<center><img src=\"diagram-20231023.svg\" width=\"700\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b837e5a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$\\textrm{Fluid phase mass balance}$\n",
    "\n",
    "$V_f \\frac{dc}{dt} + M_s \\frac{dq}{dt} = Q(c_{\\text{in}} - c)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8deb8033",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$\\textrm{Solid phase mass balance}$\n",
    "\n",
    "$\\frac{dq}{dt} = f(q, q^*)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ec42ee4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$f(q, q^*) = k_1(q^* - q), k_2(q^* - q)^2, k_3(q^{*2} - q^2)/q,...?$ $\\rightarrow$ mechanistic approach\n",
    "\n",
    "* Why not $f(q, q^*) = \\textrm{ANN}(q^*, q,\\theta)$ ?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91852a15",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "q* is typically expressed as a function of c: $q^* = Q\\frac{Kc}{(1 + Kc)}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75e517c7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Hands-on UDE implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a36e79a2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Defining parameters and Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 559,
   "id": "80c8ac3a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.22"
      ]
     },
     "execution_count": 559,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "begin\n",
    "# Bed parameters\n",
    "D_bed = 5*10^-2 #m\n",
    "L_bed = 5*10^-2 #m\n",
    "V_bed = 4/3*pi*(D_bed/2)^3\n",
    "ϵ = 0.45 + randn()*(0.05*0.45)\n",
    "#ϵ = 0.48\n",
    "d_p = 2e-3\n",
    "ρ_p = 1100 #kg/m3\n",
    "V_p = 4/3*pi*(d_p/2)^3 \n",
    "V_solid = V_bed*(1-ϵ)\n",
    "V_gas = V_bed*ϵ\n",
    "M_solid = ρ_p*V_solid #kg\n",
    "\n",
    "# Inlet conditions\n",
    "Q = 5 + randn()*(0.05*5)\n",
    "Cin = 20 # ppm = mg/L\n",
    "\n",
    "#Isotherm parameters\n",
    "qmax = 55.54 #mg/g\n",
    "k = 1.8 #L/mg\n",
    "kmass = 0.22 #min^-1\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c153dac",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* None of those would help if one go for pure data driven approach"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3da882d",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Creating Neural Network using Lux.jl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37606171",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"neural_net.svg\" width=\"600\"/></center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 560,
   "id": "74d6c89c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#Scaling parameter (maximum concentration at solid phase)\n",
    "qstar_max = qmax*k*Cin/(1 + k*Cin);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 561,
   "id": "661cde98",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "#Neural Network with 2 inputs and one of this inputs is a function (q*)\n",
    "using Lux\n",
    "∂q∂t = Lux.Chain(x -> x./qstar_max, Lux.Dense(2, 8, tanh), Lux.Dense(8, 10, tanh),\n",
    "                  Lux.Dense(10, 1)); #Uptake rake"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 562,
   "id": "80833baf",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "using Random;\n",
    "rng = Random.default_rng(222)\n",
    "pinit, st = Lux.setup(rng, ∂q∂t);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 563,
   "id": "cf26219f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1-element Vector{Float64}:\n",
       " -0.15931046575460905"
      ]
     },
     "execution_count": 563,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "∂q∂t([50; 50], pinit, st)[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca45cf57",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Defining the UDE model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 564,
   "id": "658d2d49",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "using DifferentialEquations, DiffEqFlux"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 565,
   "id": "ce888044",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "using ComponentArrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 566,
   "id": "82fd2ca9",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ude (generic function with 1 method)"
      ]
     },
     "execution_count": 566,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function ude(du, u, p, t)\n",
    "    c = u[1] #Fluid phase concentration\n",
    "    q = u[2] #Solid phase concentration\n",
    "\n",
    "    qstar = qmax*k*c/(1.0 + k*c)\n",
    "\n",
    "    du[1] = Q*(Cin - c)/(V_gas*1e3) - M_solid*1e3*∂q∂t([qstar q]', p, st)[1][1]/(V_gas*1e3) \n",
    "    du[2] = ∂q∂t([qstar q]', p, st)[1][1]\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8595a959",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Simulating the UDE before training it - randomly initialized parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 567,
   "id": "931a1901",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "Δt = train_data[2, 1] - train_data[1, 1]; #1 min\n",
    "t₀ = train_data[1, 1] #0.0 min\n",
    "tf = train_data[train_index, 1] #50 min\n",
    "u0 = [0.0; 0.0];\n",
    "prob = ODEProblem(ude, u0, (t₀, tf), pinit);\n",
    "#Note that the ODE has two dependent variables but only of them are measured"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 568,
   "id": "a8da4c29",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "sol_ude = solve(prob, AutoVern7(KenCarp5(autodiff = true)), abstol = 1e-7, reltol = 1e-7, saveat = Δt);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 569,
   "id": "17957b47",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.8629483814523184,
          "title": "",
          "x": 0.9943757030371203,
          "y": 0.5527777777777777
         },
         "legendgroup": "untrained UDE",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "untrained UDE",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35
         ],
         "xaxis": "x",
         "y": [
          0,
          24.72747221494482,
          24.751926936723592,
          24.776179886636978,
          24.800225424399862,
          24.82405362397324,
          24.847657444230556,
          24.871033542506204,
          24.894170052592816,
          24.917057514877367,
          24.939693127302547,
          24.96207399683499,
          24.984185723468453,
          25.006017570267083,
          25.027566939931383,
          25.048831235161842,
          25.069807858658965,
          25.09048236457724,
          25.110840519634735,
          25.130880171125526,
          25.150599166380506,
          25.16999535273058,
          25.189066577506647,
          25.20780432607752,
          25.226188460974786,
          25.244216297069848,
          25.261886299966626,
          25.279196935269017,
          25.29614666858093,
          25.312733965506276,
          25.328957220965172,
          25.344800262638277,
          25.360252718087047,
          25.375316231270638,
          25.38999244614819,
          25.404283006678867
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.8629483814523184,
          "title": "",
          "x": 0.9943757030371203,
          "y": 0.5527777777777777
         },
         "legendgroup": "observation",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "observation",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35
         ],
         "xaxis": "x",
         "y": [
          0.1149144414942044,
          0.453593327427082,
          0.17166903204340475,
          0.16434298525027757,
          0.43082886105468254,
          0.39036313080466295,
          0.1498263562138032,
          0.6049853300852165,
          0.7288340742910437,
          0.7208278900541426,
          0.9725557011832456,
          1.0645132432308824,
          1.605832870402756,
          1.9623017325357597,
          2.488345947495106,
          2.9372823345460555,
          3.3004647312077497,
          4.44026467007553,
          6.014621388907804,
          7.332105354195911,
          8.600968893100676,
          9.879855941342193,
          11.70106015407636,
          12.820166371191963,
          14.011211986433725,
          15.036464869109007,
          16.06275295941086,
          16.493576791018565,
          16.929248776222778,
          17.310492828215327,
          17.808197157388648,
          18.412709295322795,
          18.90649223578161,
          19.02591785290369,
          19.112996811420775,
          18.883545295888464
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 250,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1,
         "xanchor": "auto",
         "y": 1,
         "yanchor": "auto"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 700,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.05284651918510186,
          0.9943757030371203
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.0500000000000007,
          36.05
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30"
         ],
         "tickvals": [
          0,
          10,
          20,
          30
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.12130358705161852,
          0.9842519685039369
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.7621284902003662,
          26.166411496879235
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "5",
          "10",
          "15",
          "20",
          "25"
         ],
         "tickvals": [
          0,
          5,
          10,
          15,
          20,
          25
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"2d241c0a-bc68-4f5d-8732-1de4c2db0995\" style=\"width:700px;height:250px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('2d241c0a-bc68-4f5d-8732-1de4c2db0995', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5527777777777777,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.8629483814523184,\n",
       "            \"x\": 0.9943757030371203\n",
       "        },\n",
       "        \"name\": \"untrained UDE\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"untrained UDE\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            24.72747221494482,\n",
       "            24.751926936723592,\n",
       "            24.776179886636978,\n",
       "            24.800225424399862,\n",
       "            24.82405362397324,\n",
       "            24.847657444230556,\n",
       "            24.871033542506204,\n",
       "            24.894170052592816,\n",
       "            24.917057514877367,\n",
       "            24.939693127302547,\n",
       "            24.96207399683499,\n",
       "            24.984185723468453,\n",
       "            25.006017570267083,\n",
       "            25.027566939931383,\n",
       "            25.048831235161842,\n",
       "            25.069807858658965,\n",
       "            25.09048236457724,\n",
       "            25.110840519634735,\n",
       "            25.130880171125526,\n",
       "            25.150599166380506,\n",
       "            25.16999535273058,\n",
       "            25.189066577506647,\n",
       "            25.20780432607752,\n",
       "            25.226188460974786,\n",
       "            25.244216297069848,\n",
       "            25.261886299966626,\n",
       "            25.279196935269017,\n",
       "            25.29614666858093,\n",
       "            25.312733965506276,\n",
       "            25.328957220965172,\n",
       "            25.344800262638277,\n",
       "            25.360252718087047,\n",
       "            25.375316231270638,\n",
       "            25.38999244614819,\n",
       "            25.404283006678867\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5527777777777777,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.8629483814523184,\n",
       "            \"x\": 0.9943757030371203\n",
       "        },\n",
       "        \"name\": \"observation\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"observation\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.1149144414942044,\n",
       "            0.453593327427082,\n",
       "            0.17166903204340475,\n",
       "            0.16434298525027757,\n",
       "            0.43082886105468254,\n",
       "            0.39036313080466295,\n",
       "            0.1498263562138032,\n",
       "            0.6049853300852165,\n",
       "            0.7288340742910437,\n",
       "            0.7208278900541426,\n",
       "            0.9725557011832456,\n",
       "            1.0645132432308824,\n",
       "            1.605832870402756,\n",
       "            1.9623017325357597,\n",
       "            2.488345947495106,\n",
       "            2.9372823345460555,\n",
       "            3.3004647312077497,\n",
       "            4.44026467007553,\n",
       "            6.014621388907804,\n",
       "            7.332105354195911,\n",
       "            8.600968893100676,\n",
       "            9.879855941342193,\n",
       "            11.70106015407636,\n",
       "            12.820166371191963,\n",
       "            14.011211986433725,\n",
       "            15.036464869109007,\n",
       "            16.06275295941086,\n",
       "            16.493576791018565,\n",
       "            16.929248776222778,\n",
       "            17.310492828215327,\n",
       "            17.808197157388648,\n",
       "            18.412709295322795,\n",
       "            18.90649223578161,\n",
       "            19.02591785290369,\n",
       "            19.112996811420775,\n",
       "            18.883545295888464\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.0500000000000007,\n",
       "            36.05\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.05284651918510186,\n",
       "            0.9943757030371203\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 250,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            5.0,\n",
       "            10.0,\n",
       "            15.0,\n",
       "            20.0,\n",
       "            25.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.7621284902003662,\n",
       "            26.166411496879235\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.12130358705161852,\n",
       "            0.9842519685039369\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"5\",\n",
       "            \"10\",\n",
       "            \"15\",\n",
       "            \"20\",\n",
       "            \"25\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"auto\",\n",
       "        \"xanchor\": \"auto\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 1.0,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.0\n",
       "    },\n",
       "    \"width\": 700\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 569,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot(sol_ude.t, Array(sol_ude)[1, :], label = \"untrained UDE\", size = (700, 250))\n",
    "scatter!(train_data[1:train_index, 1], train_data[1:train_index, 2], xlabel = \"time (min)\", ylabel = \"Concentration (mg/L)\", label = \"observation\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c53ed06",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Training using only one of the states as training data - c(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 570,
   "id": "8ca3d4f6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "using SciMLSensitivity, Optimization, OptimizationOptimisers, OptimizationOptimJL, Zygote"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 571,
   "id": "ded71d05",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "predict (generic function with 1 method)"
      ]
     },
     "execution_count": 571,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function predict(p, states)\n",
    "    new_prob = remake(prob; p = p)\n",
    "    sensealg = InterpolatingAdjoint(autojacvec = ReverseDiffVJP(true)) #that is a very important choice to make\n",
    "    out = Array(solve(new_prob, AutoVern7(KenCarp4(autodiff = false)), abstol = 1e-6, reltol = 1e-6,\n",
    "    sensealg = sensealg, saveat = Δt))[states, :]; #g = [0, 1] * u\n",
    "    out\n",
    "end\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 572,
   "id": "efb77ce3",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loss (generic function with 2 methods)"
      ]
     },
     "execution_count": 572,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function loss(p)\n",
    "    y = predict(p, 1) #Call predict function\n",
    "    ∑e² = sum(abs2, (y .- train_data[1:train_index, 2])/Cin) #squared error\n",
    "    ∑e²\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "599bb905",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Setting up optimization problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 573,
   "id": "a121cac8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29.493708874501223"
      ]
     },
     "execution_count": 573,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss(pinit) #High initial loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 574,
   "id": "d539de79",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "adtype = Optimization.AutoZygote()\n",
    "optf = Optimization.OptimizationFunction((x,p) -> loss(x), adtype);\n",
    "optprob = Optimization.OptimizationProblem(optf, ComponentArray(pinit));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 575,
   "id": "97515274",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "#25 (generic function with 1 method)"
      ]
     },
     "execution_count": 575,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "losses = Float64[]\n",
    "callback = function (p, l)\n",
    "    push!(losses, l)\n",
    "    if length(losses) % 20 == 0\n",
    "        println(\"Current loss after $(length(losses)) iterations: $(losses[end])\")\n",
    "    end\n",
    "    return false\n",
    "end\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 576,
   "id": "7e905a15",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current loss after 20 iterations: 0.11002935773969906\n",
      "Current loss after 40 iterations: 0.07633692037702387\n",
      "Current loss after 60 iterations: 0.018820959090987955\n",
      "Current loss after 80 iterations: 0.010282066208861962\n",
      "Current loss after 100 iterations: 0.00845530771777609\n",
      "Current loss after 120 iterations: 0.007276689738899456\n",
      "Current loss after 140 iterations: 0.006508718369714101\n",
      "Current loss after 160 iterations: 0.005840410724441356\n",
      "Current loss after 180 iterations: 0.005299524168406174\n",
      "Current loss after 200 iterations: 0.004875550501966513\n",
      "Current loss after 220 iterations: 0.004530111161347893\n",
      "Current loss after 240 iterations: 0.0042513514198873935\n",
      "136.559751 seconds (337.27 M allocations: 400.877 GiB, 18.98% gc time, 1.40% compilation time: 3% of which was recompilation)\n"
     ]
    }
   ],
   "source": [
    "@time res1 = Optimization.solve(optprob, ADAM(0.075), callback = callback, maxiters = 250);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 581,
   "id": "f5c95fe0",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "test_data = readdlm(\"test_data.csv\", ',');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 582,
   "id": "745d5472",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "predict_full (generic function with 1 method)"
      ]
     },
     "execution_count": 582,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function predict_full(p, states)\n",
    "    new_prob = remake(prob; p = p, tspan = (train_data[1, 1], train_data[end, 1]))\n",
    "    sensealg = InterpolatingAdjoint(autojacvec = ReverseDiffVJP(true)) #that is a very important choice to make\n",
    "    out = Array(solve(new_prob, AutoVern7(KenCarp4(autodiff = false)), abstol = 1e-6, reltol = 1e-6,\n",
    "    sensealg = sensealg, saveat = Δt))[states, :]; #g = [0, 1] * u\n",
    "    out\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 583,
   "id": "35d3b4e3",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Trained UDE",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Trained UDE",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35
         ],
         "xaxis": "x",
         "y": [
          0,
          0.02073331737219587,
          0.06213452270574857,
          0.10961069410147348,
          0.16448425572506806,
          0.22845079327154158,
          0.30393744866018907,
          0.39388795680294336,
          0.5026431688739282,
          0.6361289832406704,
          0.8029844853776736,
          1.015201244866789,
          1.2906282760855052,
          1.654272844292825,
          2.140072816563985,
          2.7886308483609175,
          3.6376122048188972,
          4.7026718682515725,
          5.9585749517971385,
          7.348367173414643,
          8.791951785843553,
          10.217939264643324,
          11.569731409782666,
          12.80644863879982,
          13.913002525602474,
          14.881193287376997,
          15.716472104652716,
          16.429605896944185,
          17.03217881412114,
          17.5390569331494,
          17.962584772586037,
          18.315723646709746,
          18.60897353391634,
          18.852208185998382,
          19.053464102739817,
          19.219894169736584
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Train data",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Train data",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.1149144414942044,
          0.453593327427082,
          0.17166903204340475,
          0.16434298525027757,
          0.43082886105468254,
          0.39036313080466295,
          0.1498263562138032,
          0.6049853300852165,
          0.7288340742910437,
          0.7208278900541426,
          0.9725557011832456,
          1.0645132432308824,
          1.605832870402756,
          1.9623017325357597,
          2.488345947495106,
          2.9372823345460555,
          3.3004647312077497,
          4.44026467007553,
          6.014621388907804,
          7.332105354195911,
          8.600968893100676,
          9.879855941342193,
          11.70106015407636,
          12.820166371191963,
          14.011211986433725,
          15.036464869109007,
          16.06275295941086,
          16.493576791018565,
          16.929248776222778,
          17.310492828215327,
          17.808197157388648,
          18.412709295322795,
          18.90649223578161,
          19.02591785290369,
          19.112996811420775,
          18.883545295888464,
          19.46450909951882,
          19.24859182601517,
          19.527825543706353,
          20.0553701728416,
          19.714848868287742,
          19.63114088016073,
          19.53588580560029,
          19.85638399603698,
          19.803292061991808,
          19.732583577342563,
          19.847534725209762,
          19.71101197633774,
          19.96119154981576,
          20.094516579726115,
          19.74853945973233
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Time extrapolation",
         "line": {
          "color": "rgba(62, 164, 78, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Time extrapolation",
         "showlegend": true,
         "type": "scatter",
         "x": [
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          19.219894169736584,
          19.357243169778336,
          19.47067961865076,
          19.56412028327142,
          19.64119631051121,
          19.704685622476322,
          19.75694238084654,
          19.800036377065936,
          19.83545554106332,
          19.864609275071572,
          19.888647467697098,
          19.908386347172353,
          19.92461980690975,
          19.938011662802563,
          19.949009698808204,
          19.958044603568826
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 350,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1.05,
         "xanchor": "lower",
         "y": 0.07,
         "yanchor": "right"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 800,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.046240704286964134,
          0.9950787401574803
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.08664541932258472,
          0.9887514060742407
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.6028354973917835,
          20.6973520771179
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "5",
          "10",
          "15",
          "20"
         ],
         "tickvals": [
          0,
          5,
          10,
          15,
          20
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"d25cc65b-8122-4182-804f-c5d9a0a95180\" style=\"width:800px;height:350px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('d25cc65b-8122-4182-804f-c5d9a0a95180', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Trained UDE\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Trained UDE\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            0.02073331737219587,\n",
       "            0.06213452270574857,\n",
       "            0.10961069410147348,\n",
       "            0.16448425572506806,\n",
       "            0.22845079327154158,\n",
       "            0.30393744866018907,\n",
       "            0.39388795680294336,\n",
       "            0.5026431688739282,\n",
       "            0.6361289832406704,\n",
       "            0.8029844853776736,\n",
       "            1.015201244866789,\n",
       "            1.2906282760855052,\n",
       "            1.654272844292825,\n",
       "            2.140072816563985,\n",
       "            2.7886308483609175,\n",
       "            3.6376122048188972,\n",
       "            4.7026718682515725,\n",
       "            5.9585749517971385,\n",
       "            7.348367173414643,\n",
       "            8.791951785843553,\n",
       "            10.217939264643324,\n",
       "            11.569731409782666,\n",
       "            12.80644863879982,\n",
       "            13.913002525602474,\n",
       "            14.881193287376997,\n",
       "            15.716472104652716,\n",
       "            16.429605896944185,\n",
       "            17.03217881412114,\n",
       "            17.5390569331494,\n",
       "            17.962584772586037,\n",
       "            18.315723646709746,\n",
       "            18.60897353391634,\n",
       "            18.852208185998382,\n",
       "            19.053464102739817,\n",
       "            19.219894169736584\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Train data\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Train data\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.1149144414942044,\n",
       "            0.453593327427082,\n",
       "            0.17166903204340475,\n",
       "            0.16434298525027757,\n",
       "            0.43082886105468254,\n",
       "            0.39036313080466295,\n",
       "            0.1498263562138032,\n",
       "            0.6049853300852165,\n",
       "            0.7288340742910437,\n",
       "            0.7208278900541426,\n",
       "            0.9725557011832456,\n",
       "            1.0645132432308824,\n",
       "            1.605832870402756,\n",
       "            1.9623017325357597,\n",
       "            2.488345947495106,\n",
       "            2.9372823345460555,\n",
       "            3.3004647312077497,\n",
       "            4.44026467007553,\n",
       "            6.014621388907804,\n",
       "            7.332105354195911,\n",
       "            8.600968893100676,\n",
       "            9.879855941342193,\n",
       "            11.70106015407636,\n",
       "            12.820166371191963,\n",
       "            14.011211986433725,\n",
       "            15.036464869109007,\n",
       "            16.06275295941086,\n",
       "            16.493576791018565,\n",
       "            16.929248776222778,\n",
       "            17.310492828215327,\n",
       "            17.808197157388648,\n",
       "            18.412709295322795,\n",
       "            18.90649223578161,\n",
       "            19.02591785290369,\n",
       "            19.112996811420775,\n",
       "            18.883545295888464,\n",
       "            19.46450909951882,\n",
       "            19.24859182601517,\n",
       "            19.527825543706353,\n",
       "            20.0553701728416,\n",
       "            19.714848868287742,\n",
       "            19.63114088016073,\n",
       "            19.53588580560029,\n",
       "            19.85638399603698,\n",
       "            19.803292061991808,\n",
       "            19.732583577342563,\n",
       "            19.847534725209762,\n",
       "            19.71101197633774,\n",
       "            19.96119154981576,\n",
       "            20.094516579726115,\n",
       "            19.74853945973233\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Time extrapolation\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Time extrapolation\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(62, 164, 78, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            19.219894169736584,\n",
       "            19.357243169778336,\n",
       "            19.47067961865076,\n",
       "            19.56412028327142,\n",
       "            19.64119631051121,\n",
       "            19.704685622476322,\n",
       "            19.75694238084654,\n",
       "            19.800036377065936,\n",
       "            19.83545554106332,\n",
       "            19.864609275071572,\n",
       "            19.888647467697098,\n",
       "            19.908386347172353,\n",
       "            19.92461980690975,\n",
       "            19.938011662802563,\n",
       "            19.949009698808204,\n",
       "            19.958044603568826\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.046240704286964134,\n",
       "            0.9950787401574803\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 350,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            5.0,\n",
       "            10.0,\n",
       "            15.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.6028354973917835,\n",
       "            20.6973520771179\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.08664541932258472,\n",
       "            0.9887514060742407\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"5\",\n",
       "            \"10\",\n",
       "            \"15\",\n",
       "            \"20\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"right\",\n",
       "        \"xanchor\": \"lower\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.05\n",
       "    },\n",
       "    \"width\": 800\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 583,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Cin = 20\n",
    "plot(sol_ude.t[1:train_index], predict_full(res1.u, 1)[1:train_index], label = \"Trained UDE\", size = (800, 350))\n",
    "scatter!(train_data[:, 1], train_data[:, 2], xlabel = \"time (min)\", \n",
    "    ylabel = \"Concentration (mg/L)\", label = \"Train data\",\n",
    "legend=:outerbottomright)\n",
    "plot!(train_data[train_index:end, 1], predict_full(res1.u, 1)[train_index:end], label = \"Time extrapolation\", size = (800, 350))    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc21ff39",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## What if I have a less contaminated stream?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 584,
   "id": "8501e41d",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Test set UDE",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Test set UDE",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99,
          100
         ],
         "xaxis": "x",
         "y": [
          0,
          -0.05902289402747805,
          -0.04456019167260559,
          -0.02933115509908147,
          -0.013278604902032318,
          0.0036583365089452725,
          0.021555466680400877,
          0.04048256496687251,
          0.06053914476712138,
          0.08181256650085521,
          0.1044195146546994,
          0.12849379397521388,
          0.15415233782339918,
          0.1815929128410971,
          0.21096415945546698,
          0.24248119902448914,
          0.27642355057284723,
          0.31298852079340483,
          0.35254597083057315,
          0.3954811587232388,
          0.4421107880017676,
          0.4930356249321961,
          0.548790004347282,
          0.6099942262577663,
          0.6774552396949596,
          0.7520542752146451,
          0.8349347557556295,
          0.9272512130462771,
          1.030543993404661,
          1.1465307203544672,
          1.27718193211485,
          1.4248760881077789,
          1.5920352641723443,
          1.7818275036374815,
          1.9968918791917083,
          2.2406694575709007,
          2.515337664188176,
          2.8233392609893206,
          3.1646523712656527,
          3.5386149785786087,
          3.9409962643952245,
          4.366538905855957,
          4.806837722776487,
          5.253202351931156,
          5.695941750395703,
          6.126976618381884,
          6.538178070952461,
          6.9247496349496345,
          7.282863097953255,
          7.610271604848184,
          7.906833533099712,
          8.17288997993937,
          8.409780140276062,
          8.619526294934726,
          8.8041098127126,
          8.965908870280103,
          9.10722481520745,
          9.230213715043577,
          9.337019874287863,
          9.429540870335439,
          9.509544166345677,
          9.578627455346318,
          9.63818110741823,
          9.689482986929574,
          9.733615787130471,
          9.771566273436807,
          9.804163999565176,
          9.832160315314006,
          9.85618162572593,
          9.876794392513345,
          9.894463717841786,
          9.90961825835204,
          9.922600602609227,
          9.93372849700648,
          9.943260976401103,
          9.951422933295605,
          9.958418095890254,
          9.964404173469068,
          9.96953126181894,
          9.973922725893656,
          9.977678498637365,
          9.980895952058974,
          9.983651234248365,
          9.98600670197745,
          9.988023961017383,
          9.989752455707539,
          9.991229810968237,
          9.992493192279332,
          9.993576753987123,
          9.994503885348374,
          9.995295834340435,
          9.995973809674988,
          9.996555501162872,
          9.99705287420059,
          9.9974774147006,
          9.997840608575352,
          9.998152619701763,
          9.998419736857866,
          9.998647681262426,
          9.998842174301396,
          9.999008897420008
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.902105986751656,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.5376984126984127
         },
         "legendgroup": "Test data",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Test data",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99,
          100
         ],
         "xaxis": "x",
         "y": [
          0.4116816151357062,
          0.20991569680016633,
          0.34274799493835084,
          0.2077971111405163,
          0.3454918924040904,
          0.36372387207451196,
          0.04246359638375827,
          0.10755411672364348,
          0.23032216181593876,
          0.32030939121781066,
          0.3149902084588786,
          0.3403263382394859,
          0.06179159214774943,
          0.1989538597349111,
          0.3917351134973381,
          0.6122288233236826,
          0.48079384682181436,
          0.7539686757638644,
          0.5988240201294828,
          0.5784879535011717,
          0.8624445479228227,
          0.5997542323701074,
          0.5302786639171492,
          1.097078150109682,
          0.7294070605067997,
          0.8237227594423556,
          0.7557120006056826,
          0.9076335151817955,
          1.3677931639356355,
          1.4236099841013925,
          1.9478574758392795,
          1.865490581707562,
          1.7572465435626854,
          1.9349872784435318,
          1.974440410476193,
          2.7702857130629006,
          2.985709176031708,
          2.6720411332723386,
          3.3004051957797325,
          3.7760950435917495,
          3.933150064878424,
          4.317028550262893,
          4.776392130525876,
          5.3464742770512474,
          5.438113913182881,
          5.922517309812299,
          6.325061528350176,
          6.308504744509279,
          6.747898759762233,
          7.50558221919882,
          7.4213451193020115,
          8.181139416360317,
          8.091521973725762,
          8.33492637101121,
          8.7688791920082,
          8.610681256363089,
          8.417851194079272,
          9.142725531764805,
          9.331969062864399,
          9.301195243368989,
          9.890484939055444,
          9.229282531013475,
          9.726499740494209,
          9.915636244635145,
          9.435441981130777,
          9.343521365138896,
          9.69634377620142,
          9.574712833220847,
          9.651109928460812,
          9.792802051855237,
          9.81427401349818,
          9.981882804893436,
          9.855662341268406,
          9.829763063435925,
          9.415613586052926,
          9.87834537957247,
          9.689179083467199,
          10.097819429988325,
          10.082602006989859,
          9.897746006816357,
          10.507142751663256,
          10.000031962146524,
          10.000330555430585,
          10.245318275277679,
          9.720045384693796,
          9.97241755004881,
          9.94204499574352,
          9.958768585592319,
          9.730197795652414,
          10.064810454921503,
          10.018731681658988,
          10.101159031528036,
          10.116155770522326,
          9.977403541367945,
          9.901577494477456,
          9.667068930167236,
          10.031587789791216,
          10.203233982096604,
          9.940260412820981,
          9.95856469844549,
          10.087155098125692
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 350,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1.05,
         "xanchor": "lower",
         "y": 0.07,
         "yanchor": "right"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 800,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.06846292650918635,
          0.9950787401574803
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -3,
          103
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "25",
          "50",
          "75",
          "100"
         ],
         "tickvals": [
          0,
          25,
          50,
          75,
          100
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.08664541932258472,
          0.9887514060742407
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.37600786339820047,
          10.824127721033978
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0.0",
          "2.5",
          "5.0",
          "7.5",
          "10.0"
         ],
         "tickvals": [
          0,
          2.5,
          5,
          7.5,
          10
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"b8886344-6d1f-43b6-9e7a-b14969bb4ced\" style=\"width:800px;height:350px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('b8886344-6d1f-43b6-9e7a-b14969bb4ced', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Test set UDE\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Test set UDE\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            -0.05902289402747805,\n",
       "            -0.04456019167260559,\n",
       "            -0.02933115509908147,\n",
       "            -0.013278604902032318,\n",
       "            0.0036583365089452725,\n",
       "            0.021555466680400877,\n",
       "            0.04048256496687251,\n",
       "            0.06053914476712138,\n",
       "            0.08181256650085521,\n",
       "            0.1044195146546994,\n",
       "            0.12849379397521388,\n",
       "            0.15415233782339918,\n",
       "            0.1815929128410971,\n",
       "            0.21096415945546698,\n",
       "            0.24248119902448914,\n",
       "            0.27642355057284723,\n",
       "            0.31298852079340483,\n",
       "            0.35254597083057315,\n",
       "            0.3954811587232388,\n",
       "            0.4421107880017676,\n",
       "            0.4930356249321961,\n",
       "            0.548790004347282,\n",
       "            0.6099942262577663,\n",
       "            0.6774552396949596,\n",
       "            0.7520542752146451,\n",
       "            0.8349347557556295,\n",
       "            0.9272512130462771,\n",
       "            1.030543993404661,\n",
       "            1.1465307203544672,\n",
       "            1.27718193211485,\n",
       "            1.4248760881077789,\n",
       "            1.5920352641723443,\n",
       "            1.7818275036374815,\n",
       "            1.9968918791917083,\n",
       "            2.2406694575709007,\n",
       "            2.515337664188176,\n",
       "            2.8233392609893206,\n",
       "            3.1646523712656527,\n",
       "            3.5386149785786087,\n",
       "            3.9409962643952245,\n",
       "            4.366538905855957,\n",
       "            4.806837722776487,\n",
       "            5.253202351931156,\n",
       "            5.695941750395703,\n",
       "            6.126976618381884,\n",
       "            6.538178070952461,\n",
       "            6.9247496349496345,\n",
       "            7.282863097953255,\n",
       "            7.610271604848184,\n",
       "            7.906833533099712,\n",
       "            8.17288997993937,\n",
       "            8.409780140276062,\n",
       "            8.619526294934726,\n",
       "            8.8041098127126,\n",
       "            8.965908870280103,\n",
       "            9.10722481520745,\n",
       "            9.230213715043577,\n",
       "            9.337019874287863,\n",
       "            9.429540870335439,\n",
       "            9.509544166345677,\n",
       "            9.578627455346318,\n",
       "            9.63818110741823,\n",
       "            9.689482986929574,\n",
       "            9.733615787130471,\n",
       "            9.771566273436807,\n",
       "            9.804163999565176,\n",
       "            9.832160315314006,\n",
       "            9.85618162572593,\n",
       "            9.876794392513345,\n",
       "            9.894463717841786,\n",
       "            9.90961825835204,\n",
       "            9.922600602609227,\n",
       "            9.93372849700648,\n",
       "            9.943260976401103,\n",
       "            9.951422933295605,\n",
       "            9.958418095890254,\n",
       "            9.964404173469068,\n",
       "            9.96953126181894,\n",
       "            9.973922725893656,\n",
       "            9.977678498637365,\n",
       "            9.980895952058974,\n",
       "            9.983651234248365,\n",
       "            9.98600670197745,\n",
       "            9.988023961017383,\n",
       "            9.989752455707539,\n",
       "            9.991229810968237,\n",
       "            9.992493192279332,\n",
       "            9.993576753987123,\n",
       "            9.994503885348374,\n",
       "            9.995295834340435,\n",
       "            9.995973809674988,\n",
       "            9.996555501162872,\n",
       "            9.99705287420059,\n",
       "            9.9974774147006,\n",
       "            9.997840608575352,\n",
       "            9.998152619701763,\n",
       "            9.998419736857866,\n",
       "            9.998647681262426,\n",
       "            9.998842174301396,\n",
       "            9.999008897420008\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0,\n",
       "            51.0,\n",
       "            52.0,\n",
       "            53.0,\n",
       "            54.0,\n",
       "            55.0,\n",
       "            56.0,\n",
       "            57.0,\n",
       "            58.0,\n",
       "            59.0,\n",
       "            60.0,\n",
       "            61.0,\n",
       "            62.0,\n",
       "            63.0,\n",
       "            64.0,\n",
       "            65.0,\n",
       "            66.0,\n",
       "            67.0,\n",
       "            68.0,\n",
       "            69.0,\n",
       "            70.0,\n",
       "            71.0,\n",
       "            72.0,\n",
       "            73.0,\n",
       "            74.0,\n",
       "            75.0,\n",
       "            76.0,\n",
       "            77.0,\n",
       "            78.0,\n",
       "            79.0,\n",
       "            80.0,\n",
       "            81.0,\n",
       "            82.0,\n",
       "            83.0,\n",
       "            84.0,\n",
       "            85.0,\n",
       "            86.0,\n",
       "            87.0,\n",
       "            88.0,\n",
       "            89.0,\n",
       "            90.0,\n",
       "            91.0,\n",
       "            92.0,\n",
       "            93.0,\n",
       "            94.0,\n",
       "            95.0,\n",
       "            96.0,\n",
       "            97.0,\n",
       "            98.0,\n",
       "            99.0,\n",
       "            100.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5376984126984127,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.902105986751656,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"Test data\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Test data\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.4116816151357062,\n",
       "            0.20991569680016633,\n",
       "            0.34274799493835084,\n",
       "            0.2077971111405163,\n",
       "            0.3454918924040904,\n",
       "            0.36372387207451196,\n",
       "            0.04246359638375827,\n",
       "            0.10755411672364348,\n",
       "            0.23032216181593876,\n",
       "            0.32030939121781066,\n",
       "            0.3149902084588786,\n",
       "            0.3403263382394859,\n",
       "            0.06179159214774943,\n",
       "            0.1989538597349111,\n",
       "            0.3917351134973381,\n",
       "            0.6122288233236826,\n",
       "            0.48079384682181436,\n",
       "            0.7539686757638644,\n",
       "            0.5988240201294828,\n",
       "            0.5784879535011717,\n",
       "            0.8624445479228227,\n",
       "            0.5997542323701074,\n",
       "            0.5302786639171492,\n",
       "            1.097078150109682,\n",
       "            0.7294070605067997,\n",
       "            0.8237227594423556,\n",
       "            0.7557120006056826,\n",
       "            0.9076335151817955,\n",
       "            1.3677931639356355,\n",
       "            1.4236099841013925,\n",
       "            1.9478574758392795,\n",
       "            1.865490581707562,\n",
       "            1.7572465435626854,\n",
       "            1.9349872784435318,\n",
       "            1.974440410476193,\n",
       "            2.7702857130629006,\n",
       "            2.985709176031708,\n",
       "            2.6720411332723386,\n",
       "            3.3004051957797325,\n",
       "            3.7760950435917495,\n",
       "            3.933150064878424,\n",
       "            4.317028550262893,\n",
       "            4.776392130525876,\n",
       "            5.3464742770512474,\n",
       "            5.438113913182881,\n",
       "            5.922517309812299,\n",
       "            6.325061528350176,\n",
       "            6.308504744509279,\n",
       "            6.747898759762233,\n",
       "            7.50558221919882,\n",
       "            7.4213451193020115,\n",
       "            8.181139416360317,\n",
       "            8.091521973725762,\n",
       "            8.33492637101121,\n",
       "            8.7688791920082,\n",
       "            8.610681256363089,\n",
       "            8.417851194079272,\n",
       "            9.142725531764805,\n",
       "            9.331969062864399,\n",
       "            9.301195243368989,\n",
       "            9.890484939055444,\n",
       "            9.229282531013475,\n",
       "            9.726499740494209,\n",
       "            9.915636244635145,\n",
       "            9.435441981130777,\n",
       "            9.343521365138896,\n",
       "            9.69634377620142,\n",
       "            9.574712833220847,\n",
       "            9.651109928460812,\n",
       "            9.792802051855237,\n",
       "            9.81427401349818,\n",
       "            9.981882804893436,\n",
       "            9.855662341268406,\n",
       "            9.829763063435925,\n",
       "            9.415613586052926,\n",
       "            9.87834537957247,\n",
       "            9.689179083467199,\n",
       "            10.097819429988325,\n",
       "            10.082602006989859,\n",
       "            9.897746006816357,\n",
       "            10.507142751663256,\n",
       "            10.000031962146524,\n",
       "            10.000330555430585,\n",
       "            10.245318275277679,\n",
       "            9.720045384693796,\n",
       "            9.97241755004881,\n",
       "            9.94204499574352,\n",
       "            9.958768585592319,\n",
       "            9.730197795652414,\n",
       "            10.064810454921503,\n",
       "            10.018731681658988,\n",
       "            10.101159031528036,\n",
       "            10.116155770522326,\n",
       "            9.977403541367945,\n",
       "            9.901577494477456,\n",
       "            9.667068930167236,\n",
       "            10.031587789791216,\n",
       "            10.203233982096604,\n",
       "            9.940260412820981,\n",
       "            9.95856469844549,\n",
       "            10.087155098125692\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0,\n",
       "            51.0,\n",
       "            52.0,\n",
       "            53.0,\n",
       "            54.0,\n",
       "            55.0,\n",
       "            56.0,\n",
       "            57.0,\n",
       "            58.0,\n",
       "            59.0,\n",
       "            60.0,\n",
       "            61.0,\n",
       "            62.0,\n",
       "            63.0,\n",
       "            64.0,\n",
       "            65.0,\n",
       "            66.0,\n",
       "            67.0,\n",
       "            68.0,\n",
       "            69.0,\n",
       "            70.0,\n",
       "            71.0,\n",
       "            72.0,\n",
       "            73.0,\n",
       "            74.0,\n",
       "            75.0,\n",
       "            76.0,\n",
       "            77.0,\n",
       "            78.0,\n",
       "            79.0,\n",
       "            80.0,\n",
       "            81.0,\n",
       "            82.0,\n",
       "            83.0,\n",
       "            84.0,\n",
       "            85.0,\n",
       "            86.0,\n",
       "            87.0,\n",
       "            88.0,\n",
       "            89.0,\n",
       "            90.0,\n",
       "            91.0,\n",
       "            92.0,\n",
       "            93.0,\n",
       "            94.0,\n",
       "            95.0,\n",
       "            96.0,\n",
       "            97.0,\n",
       "            98.0,\n",
       "            99.0,\n",
       "            100.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            25.0,\n",
       "            50.0,\n",
       "            75.0,\n",
       "            100.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -3.0,\n",
       "            103.0\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.06846292650918635,\n",
       "            0.9950787401574803\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"25\",\n",
       "            \"50\",\n",
       "            \"75\",\n",
       "            \"100\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 350,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            2.5,\n",
       "            5.0,\n",
       "            7.5,\n",
       "            10.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.37600786339820047,\n",
       "            10.824127721033978\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.08664541932258472,\n",
       "            0.9887514060742407\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0.0\",\n",
       "            \"2.5\",\n",
       "            \"5.0\",\n",
       "            \"7.5\",\n",
       "            \"10.0\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"right\",\n",
       "        \"xanchor\": \"lower\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.05\n",
       "    },\n",
       "    \"width\": 800\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 584,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Cin = 10.0\n",
    "test_prob = remake(prob; p = res1.u, tspan = (0.0, 100.), u0 = [0.0, 0.0])\n",
    "sol_testprob = Array(solve(test_prob, AutoVern7(KenCarp4(autodiff = true)), abstol = 1e-7, reltol = 1e-7,\n",
    "        saveat = Δt))[1, :];\n",
    "plot(0.0:1.0:100.0, sol_testprob, label = \"Test set UDE\", size = (800, 350))\n",
    "scatter!(test_data[:, 1], test_data[:, 2], xlabel = \"time (min)\", \n",
    "    ylabel = \"Concentration (mg/L)\", label = \"Test data\",\n",
    "legend=:outerbottomright)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de5635d5",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<center><img src=\"sparsereg.PNG\" width=\"1000\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6836e38",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<center><img src=\"sparsereg2.PNG\" width=\"1000\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6fda16b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Smoothed Collocation for Fast Two-Stage Training"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9d94c31",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Training ANNs in ODEs can be slow. Can we speed it up with a warm start?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ffd29ff",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$ANN(q^*, q) = \\frac{dq}{dt}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1533f6f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "How do we estimate dq/dt? First we have to estimate q(t) from c(t)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52e59ded",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$V_f \\int_{0}^{c}{dc} + M_s \\int_{0}^{q}{dq} = Q(c_{\\text{in}}t - \\int_{0}^{c}{c}dt)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0e63c44",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$q(t) = Q/M_s(c_{\\text{in}}t - \\int_{0}^{c}{c}dt) - V_f/M_s c(t) $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "id": "b5766214",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "using DataInterpolations, Integrals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "id": "f6c9e601",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "t = train_data[:, 1]\n",
    "c_exp = train_data[:, 2]\n",
    "c_itp = AkimaInterpolation(c_exp, t); #Interpolate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "id": "db64dd5b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "q (generic function with 1 method)"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "integrand(t) = Q/(M_solid*1e3)*c_itp(t)\n",
    "int_integrand(t) = solve(IntegralProblem((t, p) -> integrand(t), 0.0, t), QuadGKJL()).u\n",
    "q(t) = Q/(M_solid*1e3)*Cin*t - int_integrand(t) - V_gas/M_solid*c_itp(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "id": "46dccc55",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "q_train = q.(t);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34e87faa",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can use Kernel smoothing to estimate dq(t)/dt from q(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "id": "64239d01",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([2.535207827004392 2.532016773743905 … 0.010450459894075337 0.008616109173400943], [0.09093161908405922 2.576075130934778 … 53.90842337568725 53.93068327921643])"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dq_dt_est, q_smooth = collocate_data(q_train[:, :]', train_data[:, 1], EpanechnikovKernel()); #From DiffEqFlux"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "id": "ec867084",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.885790317876932,
          "title": "",
          "x": 0.9943757030371203,
          "y": 0.5439814814814815
         },
         "legendgroup": "q_exp (estimated)",
         "marker": {
          "color": "rgba(0, 154, 250, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "q_exp (estimated)",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          -8.547355152461484e-05,
          2.4814107896581175,
          4.967579586702007,
          7.47404948542323,
          9.961362883953916,
          12.432971958293201,
          14.92577487064249,
          17.40484795364266,
          19.84464461273799,
          22.27918475649282,
          24.698399663466496,
          27.096279788111115,
          29.453840504843374,
          31.754458202536718,
          33.99849169798882,
          36.180685952564865,
          38.31255019624902,
          40.35868007842807,
          42.221570358298756,
          43.90260295011116,
          45.42113425176046,
          46.77899152199593,
          47.93964494404699,
          48.91582511692857,
          49.74477607417078,
          50.43547530325556,
          50.99557633668802,
          51.460101718337484,
          51.875014512230635,
          52.2382735670106,
          52.5474769292266,
          52.785912669371875,
          52.95084471542189,
          53.08021979888443,
          53.19737254856705,
          53.323254511002794,
          53.42801781718497,
          53.510220146406915,
          53.58945124963068,
          53.61254771337104,
          53.62533132070481,
          53.66667420895877,
          53.71939142023197,
          53.75799493356266,
          53.779237457908316,
          53.80865926592188,
          53.83605070540048,
          53.86501410870763,
          53.886265633225506,
          53.8809801337087,
          53.884569423712065
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.885790317876932,
          "title": "",
          "x": 0.9943757030371203,
          "y": 0.5439814814814815
         },
         "legendgroup": "Smoothed q",
         "line": {
          "color": "rgba(227, 111, 71, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "Smoothed q",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.09093161908405922,
          2.576075130934778,
          5.050372070372813,
          7.5107808529693685,
          9.9544626311752,
          12.377298682642465,
          14.773740878793777,
          17.137340534246366,
          19.46114554555089,
          21.737971760152398,
          23.96095288235943,
          26.12409446079012,
          28.222816925681293,
          30.254847072370247,
          32.22176634577417,
          34.129047737731284,
          35.950017489851035,
          37.67991619960537,
          39.31455783764501,
          40.850674627849486,
          42.28584783724031,
          43.61868395821497,
          44.848855914263666,
          45.976858787556,
          47.003982656778334,
          47.9322006707627,
          48.764373770075956,
          49.50445779002067,
          50.15726498508232,
          50.728170252790214,
          51.223059048962696,
          51.648434044451086,
          52.01139160598544,
          52.31933767465385,
          52.57926152931159,
          52.79755682966158,
          52.98028264960206,
          53.138531626350094,
          53.27411008049722,
          53.38931252681684,
          53.48676897160287,
          53.56898863371275,
          53.63821049909828,
          53.69653798625737,
          53.74601613991268,
          53.788431129490476,
          53.82508744541845,
          53.856781384762144,
          53.88422932261284,
          53.90842337568725,
          53.93068327921643
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 300,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1.05,
         "xanchor": "lower",
         "y": 0.07,
         "yanchor": "right"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 700,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.05284651918510186,
          0.9943757030371203
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.10108632254301551,
          0.9868766404199475
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.6180085361345604,
          55.54860634179947
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"6408a1a5-3ea2-4c4c-a49b-25b433fefe76\" style=\"width:700px;height:300px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('6408a1a5-3ea2-4c4c-a49b-25b433fefe76', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5439814814814815,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.885790317876932,\n",
       "            \"x\": 0.9943757030371203\n",
       "        },\n",
       "        \"name\": \"q_exp (estimated)\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"q_exp (estimated)\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            -8.547355152461484e-5,\n",
       "            2.4814107896581175,\n",
       "            4.967579586702007,\n",
       "            7.47404948542323,\n",
       "            9.961362883953916,\n",
       "            12.432971958293201,\n",
       "            14.92577487064249,\n",
       "            17.40484795364266,\n",
       "            19.84464461273799,\n",
       "            22.27918475649282,\n",
       "            24.698399663466496,\n",
       "            27.096279788111115,\n",
       "            29.453840504843374,\n",
       "            31.754458202536718,\n",
       "            33.99849169798882,\n",
       "            36.180685952564865,\n",
       "            38.31255019624902,\n",
       "            40.35868007842807,\n",
       "            42.221570358298756,\n",
       "            43.90260295011116,\n",
       "            45.42113425176046,\n",
       "            46.77899152199593,\n",
       "            47.93964494404699,\n",
       "            48.91582511692857,\n",
       "            49.74477607417078,\n",
       "            50.43547530325556,\n",
       "            50.99557633668802,\n",
       "            51.460101718337484,\n",
       "            51.875014512230635,\n",
       "            52.2382735670106,\n",
       "            52.5474769292266,\n",
       "            52.785912669371875,\n",
       "            52.95084471542189,\n",
       "            53.08021979888443,\n",
       "            53.19737254856705,\n",
       "            53.323254511002794,\n",
       "            53.42801781718497,\n",
       "            53.510220146406915,\n",
       "            53.58945124963068,\n",
       "            53.61254771337104,\n",
       "            53.62533132070481,\n",
       "            53.66667420895877,\n",
       "            53.71939142023197,\n",
       "            53.75799493356266,\n",
       "            53.779237457908316,\n",
       "            53.80865926592188,\n",
       "            53.83605070540048,\n",
       "            53.86501410870763,\n",
       "            53.886265633225506,\n",
       "            53.8809801337087,\n",
       "            53.884569423712065\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.5439814814814815,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.885790317876932,\n",
       "            \"x\": 0.9943757030371203\n",
       "        },\n",
       "        \"name\": \"Smoothed q\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"Smoothed q\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.09093161908405922,\n",
       "            2.576075130934778,\n",
       "            5.050372070372813,\n",
       "            7.5107808529693685,\n",
       "            9.9544626311752,\n",
       "            12.377298682642465,\n",
       "            14.773740878793777,\n",
       "            17.137340534246366,\n",
       "            19.46114554555089,\n",
       "            21.737971760152398,\n",
       "            23.96095288235943,\n",
       "            26.12409446079012,\n",
       "            28.222816925681293,\n",
       "            30.254847072370247,\n",
       "            32.22176634577417,\n",
       "            34.129047737731284,\n",
       "            35.950017489851035,\n",
       "            37.67991619960537,\n",
       "            39.31455783764501,\n",
       "            40.850674627849486,\n",
       "            42.28584783724031,\n",
       "            43.61868395821497,\n",
       "            44.848855914263666,\n",
       "            45.976858787556,\n",
       "            47.003982656778334,\n",
       "            47.9322006707627,\n",
       "            48.764373770075956,\n",
       "            49.50445779002067,\n",
       "            50.15726498508232,\n",
       "            50.728170252790214,\n",
       "            51.223059048962696,\n",
       "            51.648434044451086,\n",
       "            52.01139160598544,\n",
       "            52.31933767465385,\n",
       "            52.57926152931159,\n",
       "            52.79755682966158,\n",
       "            52.98028264960206,\n",
       "            53.138531626350094,\n",
       "            53.27411008049722,\n",
       "            53.38931252681684,\n",
       "            53.48676897160287,\n",
       "            53.56898863371275,\n",
       "            53.63821049909828,\n",
       "            53.69653798625737,\n",
       "            53.74601613991268,\n",
       "            53.788431129490476,\n",
       "            53.82508744541845,\n",
       "            53.856781384762144,\n",
       "            53.88422932261284,\n",
       "            53.90842337568725,\n",
       "            53.93068327921643\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.05284651918510186,\n",
       "            0.9943757030371203\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 300,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.6180085361345604,\n",
       "            55.54860634179947\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.10108632254301551,\n",
       "            0.9868766404199475\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"right\",\n",
       "        \"xanchor\": \"lower\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.05\n",
       "    },\n",
       "    \"width\": 700\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 203,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scatter(t, q_train, xlabel = \"time (min)\", ylabel = \"q (mg/L)\", label = \"q_exp (estimated)\")\n",
    "plot!(t, q_smooth[:], xlabel = \"time (min)\",\n",
    "ylabel = \"Concentration (mg/L)\", legend=:outerbottomright, label = \"Smoothed q\", size = (700, 300))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ce5d434",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Setting up learning problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "id": "cf185b19",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×51 Matrix{Float64}:\n",
       "  0.176155   0.461967  0.242618  0.234628  …  0.999947  1.00013   0.999656\n",
       " -1.5817e-6  0.045919  0.091926  0.138309     0.997175  0.997077  0.997144"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y = dq_dt_est[:]; #target\n",
    "X = [qmax*k*c_exp./(1.0 .+ k*c_exp) q_train]'/qstar_max; #Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "id": "26f9321f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "ɼ_fast = Lux.Chain(Lux.Dense(2, 8, tanh), Lux.Dense(8, 10, tanh),\n",
    "                  Lux.Dense(10, 1)); #Uptake rake"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "id": "92e77af4",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "using Random;\n",
    "rng = Random.default_rng(222)\n",
    "pinit_fast, st_fast = Lux.setup(rng, ɼ_fast);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "id": "102c9a81",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loss_fast (generic function with 1 method)"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predicting and loss function\n",
    "function predict_fast(p)\n",
    "        Y_hat = ɼ_fast(X, p, st_fast)[1][:]\n",
    "        Y_hat\n",
    "end\n",
    "\n",
    "function loss_fast(p)\n",
    "         Y_hat = predict_fast(p)\n",
    "         loss = sum(abs2, (Y_hat .- Y))\n",
    "         return loss\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "id": "55e0b278",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "132.19156537433574"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss_fast(pinit_fast) #testing initial loss"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbec8aad",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Continuing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "id": "2197f119",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "adtype = Optimization.AutoZygote();\n",
    "optf_fast = Optimization.OptimizationFunction((x,p) -> loss_fast(x), adtype);\n",
    "optprob_fast = Optimization.OptimizationProblem(optf_fast, ComponentArray(pinit_fast));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "b451514f",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "#55 (generic function with 1 method)"
      ]
     },
     "execution_count": 198,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "losses_fast = []\n",
    "callback_fast = function (p, l)\n",
    "    push!(losses_fast, l)\n",
    "    if length(losses_fast) % 100 == 0\n",
    "        println(\"Current loss after $(length(losses_fast)) iterations: $(losses_fast[end])\")\n",
    "    end\n",
    "    return false\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "id": "096ba1db",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current loss after 100 iterations: 4.378522721813006\n",
      "Current loss after 200 iterations: 1.6674762088658721\n",
      "Current loss after 300 iterations: 0.5836468296748255\n",
      "Current loss after 400 iterations: 0.33954833406485174\n",
      "Current loss after 500 iterations: 0.23219823413579016\n",
      " 14.679808 seconds (32.51 M allocations: 1.927 GiB, 5.03% gc time, 98.57% compilation time)\n"
     ]
    }
   ],
   "source": [
    "@time res_fast = Optimization.solve(optprob_fast, ADAM(1e-2), callback = callback_fast, maxiters = 500);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "id": "54d167ae",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0mComponentVector{Float32,SubArray...}(weight = Float32[-0.3489829 -0.11738045 … -0.38437322 0.68859684], bias = Float32[0.009310756;;])"
      ]
     },
     "execution_count": 200,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p_modified = copy(ComponentArray(pinit))\n",
    "p_modified.layer_2 = res_fast.u.layer_1\n",
    "p_modified.layer_3 = res_fast.u.layer_2\n",
    "p_modified.layer_4 = res_fast.u.layer_3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "id": "34b261dc",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "data": [
        {
         "colorbar": {
          "len": 0.9238602119179546,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.529320987654321
         },
         "legendgroup": "First Stage UDE",
         "line": {
          "color": "rgba(0, 154, 250, 1.000)",
          "dash": "solid",
          "shape": "linear",
          "width": 2
         },
         "mode": "lines",
         "name": "First Stage UDE",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0,
          0.0828229726718144,
          0.14726531748445526,
          0.22253409219055278,
          0.31085970746540165,
          0.415000068269376,
          0.5383791917322911,
          0.6850241465506676,
          0.8600580498587933,
          1.0698898752018828,
          1.3216139132017508,
          1.6252532250609353,
          1.9905740129175362,
          2.4313337617513255,
          2.9592819456959525,
          3.58966631878375,
          4.331227045260835,
          5.193908045785073,
          6.172267241362913,
          7.257797708300421,
          8.425987227964244,
          9.637333255268079,
          10.855233437092366,
          12.038472811672392,
          13.146988337542657,
          14.162912023337233,
          15.06589917780104,
          15.858157253785143,
          16.538805200931833,
          17.11992105604304,
          17.609501280070038,
          18.02078819319933,
          18.36359292620602,
          18.648982452717156,
          18.885470669161702,
          19.081366823873662,
          19.243128351655926,
          19.376789988449804,
          19.486938205514225,
          19.57775055526326,
          19.652609056406515,
          19.714150697726993,
          19.764889082605563,
          19.806645159207264,
          19.84093831987095,
          19.86921608957312,
          19.89247926597457,
          19.91155725352757,
          19.927265670629197,
          19.940200807009585,
          19.950826027983656
         ],
         "yaxis": "y"
        },
        {
         "colorbar": {
          "len": 0.9238602119179546,
          "title": "",
          "x": 0.9950787401574803,
          "y": 0.529320987654321
         },
         "legendgroup": "observation",
         "marker": {
          "color": "rgba(227, 111, 71, 1.000)",
          "line": {
           "color": "rgba(0, 0, 0, 1.000)",
           "width": 1
          },
          "size": 8,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "observation",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          0.1149144414942044,
          0.453593327427082,
          0.17166903204340475,
          0.16434298525027757,
          0.43082886105468254,
          0.39036313080466295,
          0.1498263562138032,
          0.6049853300852165,
          0.7288340742910437,
          0.7208278900541426,
          0.9725557011832456,
          1.0645132432308824,
          1.605832870402756,
          1.9623017325357597,
          2.488345947495106,
          2.9372823345460555,
          3.3004647312077497,
          4.44026467007553,
          6.014621388907804,
          7.332105354195911,
          8.600968893100676,
          9.879855941342193,
          11.70106015407636,
          12.820166371191963,
          14.011211986433725,
          15.036464869109007,
          16.06275295941086,
          16.493576791018565,
          16.929248776222778,
          17.310492828215327,
          17.808197157388648,
          18.412709295322795,
          18.90649223578161,
          19.02591785290369,
          19.112996811420775,
          18.883545295888464,
          19.46450909951882,
          19.24859182601517,
          19.527825543706353,
          20.0553701728416,
          19.714848868287742,
          19.63114088016073,
          19.53588580560029,
          19.85638399603698,
          19.803292061991808,
          19.732583577342563,
          19.847534725209762,
          19.71101197633774,
          19.96119154981576,
          20.094516579726115,
          19.74853945973233
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "annotations": [],
        "height": 450,
        "legend": {
         "bgcolor": "rgba(255, 255, 255, 1.000)",
         "bordercolor": "rgba(0, 0, 0, 1.000)",
         "borderwidth": 1,
         "font": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "title": {
          "font": {
           "color": "rgba(0, 0, 0, 1.000)",
           "family": "Computer Modern",
           "size": 15
          },
          "text": ""
         },
         "tracegroupgap": 0,
         "traceorder": "normal",
         "x": 1.05,
         "xanchor": "lower",
         "y": 0.07,
         "yanchor": "right"
        },
        "margin": {
         "b": 20,
         "l": 0,
         "r": 0,
         "t": 20
        },
        "paper_bgcolor": "rgba(255, 255, 255, 1.000)",
        "plot_bgcolor": "rgba(255, 255, 255, 1.000)",
        "showlegend": true,
        "width": 800,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0.046240704286964134,
          0.9950787401574803
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -1.5,
          51.5
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "10",
          "20",
          "30",
          "40",
          "50"
         ],
         "tickvals": [
          0,
          10,
          20,
          30,
          40,
          50
         ],
         "title": "time (min)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.06739088169534369,
          0.9912510936132983
         ],
         "gridcolor": "rgba(0, 0, 0, 0.100)",
         "gridwidth": 0.5,
         "linecolor": "rgba(0, 0, 0, 1.000)",
         "mirror": true,
         "range": [
          -0.6028354973917835,
          20.6973520771179
         ],
         "showgrid": false,
         "showline": true,
         "showticklabels": true,
         "tickangle": 0,
         "tickcolor": "rgb(0, 0, 0)",
         "tickfont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 11
         },
         "tickmode": "array",
         "ticks": "inside",
         "ticktext": [
          "0",
          "5",
          "10",
          "15",
          "20"
         ],
         "tickvals": [
          0,
          5,
          10,
          15,
          20
         ],
         "title": "Concentration (mg/L)",
         "titlefont": {
          "color": "rgba(0, 0, 0, 1.000)",
          "family": "Computer Modern",
          "size": 15
         },
         "type": "-",
         "visible": true,
         "zeroline": false,
         "zerolinecolor": "rgba(0, 0, 0, 1.000)"
        }
       }
      },
      "text/html": [
       "    <div id=\"ea67e4c1-91d4-4f16-b3ca-5467d9ebaeaf\" style=\"width:800px;height:450px;\"></div>\n",
       "    <script>\n",
       "        requirejs.config({\n",
       "        paths: {\n",
       "            plotly: 'https://cdn.plot.ly/plotly-2.6.3.min'\n",
       "        }\n",
       "    });\n",
       "    require(['plotly'], function (Plotly) {\n",
       "\n",
       "    Plotly.newPlot('ea67e4c1-91d4-4f16-b3ca-5467d9ebaeaf', [\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"lines\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.529320987654321,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.9238602119179546,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"First Stage UDE\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"First Stage UDE\",\n",
       "        \"line\": {\n",
       "            \"color\": \"rgba(0, 154, 250, 1.000)\",\n",
       "            \"shape\": \"linear\",\n",
       "            \"dash\": \"solid\",\n",
       "            \"width\": 2\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.0,\n",
       "            0.0828229726718144,\n",
       "            0.14726531748445526,\n",
       "            0.22253409219055278,\n",
       "            0.31085970746540165,\n",
       "            0.415000068269376,\n",
       "            0.5383791917322911,\n",
       "            0.6850241465506676,\n",
       "            0.8600580498587933,\n",
       "            1.0698898752018828,\n",
       "            1.3216139132017508,\n",
       "            1.6252532250609353,\n",
       "            1.9905740129175362,\n",
       "            2.4313337617513255,\n",
       "            2.9592819456959525,\n",
       "            3.58966631878375,\n",
       "            4.331227045260835,\n",
       "            5.193908045785073,\n",
       "            6.172267241362913,\n",
       "            7.257797708300421,\n",
       "            8.425987227964244,\n",
       "            9.637333255268079,\n",
       "            10.855233437092366,\n",
       "            12.038472811672392,\n",
       "            13.146988337542657,\n",
       "            14.162912023337233,\n",
       "            15.06589917780104,\n",
       "            15.858157253785143,\n",
       "            16.538805200931833,\n",
       "            17.11992105604304,\n",
       "            17.609501280070038,\n",
       "            18.02078819319933,\n",
       "            18.36359292620602,\n",
       "            18.648982452717156,\n",
       "            18.885470669161702,\n",
       "            19.081366823873662,\n",
       "            19.243128351655926,\n",
       "            19.376789988449804,\n",
       "            19.486938205514225,\n",
       "            19.57775055526326,\n",
       "            19.652609056406515,\n",
       "            19.714150697726993,\n",
       "            19.764889082605563,\n",
       "            19.806645159207264,\n",
       "            19.84093831987095,\n",
       "            19.86921608957312,\n",
       "            19.89247926597457,\n",
       "            19.91155725352757,\n",
       "            19.927265670629197,\n",
       "            19.940200807009585,\n",
       "            19.950826027983656\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    },\n",
       "    {\n",
       "        \"showlegend\": true,\n",
       "        \"mode\": \"markers\",\n",
       "        \"xaxis\": \"x\",\n",
       "        \"colorbar\": {\n",
       "            \"y\": 0.529320987654321,\n",
       "            \"title\": \"\",\n",
       "            \"len\": 0.9238602119179546,\n",
       "            \"x\": 0.9950787401574803\n",
       "        },\n",
       "        \"name\": \"observation\",\n",
       "        \"yaxis\": \"y\",\n",
       "        \"legendgroup\": \"observation\",\n",
       "        \"marker\": {\n",
       "            \"symbol\": \"circle\",\n",
       "            \"color\": \"rgba(227, 111, 71, 1.000)\",\n",
       "            \"line\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"width\": 1\n",
       "            },\n",
       "            \"size\": 8\n",
       "        },\n",
       "        \"y\": [\n",
       "            0.1149144414942044,\n",
       "            0.453593327427082,\n",
       "            0.17166903204340475,\n",
       "            0.16434298525027757,\n",
       "            0.43082886105468254,\n",
       "            0.39036313080466295,\n",
       "            0.1498263562138032,\n",
       "            0.6049853300852165,\n",
       "            0.7288340742910437,\n",
       "            0.7208278900541426,\n",
       "            0.9725557011832456,\n",
       "            1.0645132432308824,\n",
       "            1.605832870402756,\n",
       "            1.9623017325357597,\n",
       "            2.488345947495106,\n",
       "            2.9372823345460555,\n",
       "            3.3004647312077497,\n",
       "            4.44026467007553,\n",
       "            6.014621388907804,\n",
       "            7.332105354195911,\n",
       "            8.600968893100676,\n",
       "            9.879855941342193,\n",
       "            11.70106015407636,\n",
       "            12.820166371191963,\n",
       "            14.011211986433725,\n",
       "            15.036464869109007,\n",
       "            16.06275295941086,\n",
       "            16.493576791018565,\n",
       "            16.929248776222778,\n",
       "            17.310492828215327,\n",
       "            17.808197157388648,\n",
       "            18.412709295322795,\n",
       "            18.90649223578161,\n",
       "            19.02591785290369,\n",
       "            19.112996811420775,\n",
       "            18.883545295888464,\n",
       "            19.46450909951882,\n",
       "            19.24859182601517,\n",
       "            19.527825543706353,\n",
       "            20.0553701728416,\n",
       "            19.714848868287742,\n",
       "            19.63114088016073,\n",
       "            19.53588580560029,\n",
       "            19.85638399603698,\n",
       "            19.803292061991808,\n",
       "            19.732583577342563,\n",
       "            19.847534725209762,\n",
       "            19.71101197633774,\n",
       "            19.96119154981576,\n",
       "            20.094516579726115,\n",
       "            19.74853945973233\n",
       "        ],\n",
       "        \"type\": \"scatter\",\n",
       "        \"x\": [\n",
       "            0.0,\n",
       "            1.0,\n",
       "            2.0,\n",
       "            3.0,\n",
       "            4.0,\n",
       "            5.0,\n",
       "            6.0,\n",
       "            7.0,\n",
       "            8.0,\n",
       "            9.0,\n",
       "            10.0,\n",
       "            11.0,\n",
       "            12.0,\n",
       "            13.0,\n",
       "            14.0,\n",
       "            15.0,\n",
       "            16.0,\n",
       "            17.0,\n",
       "            18.0,\n",
       "            19.0,\n",
       "            20.0,\n",
       "            21.0,\n",
       "            22.0,\n",
       "            23.0,\n",
       "            24.0,\n",
       "            25.0,\n",
       "            26.0,\n",
       "            27.0,\n",
       "            28.0,\n",
       "            29.0,\n",
       "            30.0,\n",
       "            31.0,\n",
       "            32.0,\n",
       "            33.0,\n",
       "            34.0,\n",
       "            35.0,\n",
       "            36.0,\n",
       "            37.0,\n",
       "            38.0,\n",
       "            39.0,\n",
       "            40.0,\n",
       "            41.0,\n",
       "            42.0,\n",
       "            43.0,\n",
       "            44.0,\n",
       "            45.0,\n",
       "            46.0,\n",
       "            47.0,\n",
       "            48.0,\n",
       "            49.0,\n",
       "            50.0\n",
       "        ]\n",
       "    }\n",
       "]\n",
       ", {\n",
       "    \"showlegend\": true,\n",
       "    \"xaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            10.0,\n",
       "            20.0,\n",
       "            30.0,\n",
       "            40.0,\n",
       "            50.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -1.5,\n",
       "            51.5\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.046240704286964134,\n",
       "            0.9950787401574803\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"10\",\n",
       "            \"20\",\n",
       "            \"30\",\n",
       "            \"40\",\n",
       "            \"50\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"y\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"time (min)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"paper_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"annotations\": [],\n",
       "    \"height\": 450,\n",
       "    \"margin\": {\n",
       "        \"l\": 0,\n",
       "        \"b\": 20,\n",
       "        \"r\": 0,\n",
       "        \"t\": 20\n",
       "    },\n",
       "    \"plot_bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "    \"yaxis\": {\n",
       "        \"showticklabels\": true,\n",
       "        \"gridwidth\": 0.5,\n",
       "        \"tickvals\": [\n",
       "            0.0,\n",
       "            5.0,\n",
       "            10.0,\n",
       "            15.0,\n",
       "            20.0\n",
       "        ],\n",
       "        \"range\": [\n",
       "            -0.6028354973917835,\n",
       "            20.6973520771179\n",
       "        ],\n",
       "        \"domain\": [\n",
       "            0.06739088169534369,\n",
       "            0.9912510936132983\n",
       "        ],\n",
       "        \"mirror\": true,\n",
       "        \"tickangle\": 0,\n",
       "        \"showline\": true,\n",
       "        \"ticktext\": [\n",
       "            \"0\",\n",
       "            \"5\",\n",
       "            \"10\",\n",
       "            \"15\",\n",
       "            \"20\"\n",
       "        ],\n",
       "        \"zeroline\": false,\n",
       "        \"tickfont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"zerolinecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"anchor\": \"x\",\n",
       "        \"visible\": true,\n",
       "        \"ticks\": \"inside\",\n",
       "        \"tickmode\": \"array\",\n",
       "        \"linecolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"showgrid\": false,\n",
       "        \"title\": \"Concentration (mg/L)\",\n",
       "        \"gridcolor\": \"rgba(0, 0, 0, 0.100)\",\n",
       "        \"titlefont\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 15\n",
       "        },\n",
       "        \"tickcolor\": \"rgb(0, 0, 0)\",\n",
       "        \"type\": \"-\"\n",
       "    },\n",
       "    \"legend\": {\n",
       "        \"yanchor\": \"right\",\n",
       "        \"xanchor\": \"lower\",\n",
       "        \"bordercolor\": \"rgba(0, 0, 0, 1.000)\",\n",
       "        \"bgcolor\": \"rgba(255, 255, 255, 1.000)\",\n",
       "        \"borderwidth\": 1,\n",
       "        \"tracegroupgap\": 0,\n",
       "        \"y\": 0.07,\n",
       "        \"font\": {\n",
       "            \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "            \"family\": \"Computer Modern\",\n",
       "            \"size\": 11\n",
       "        },\n",
       "        \"title\": {\n",
       "            \"font\": {\n",
       "                \"color\": \"rgba(0, 0, 0, 1.000)\",\n",
       "                \"family\": \"Computer Modern\",\n",
       "                \"size\": 15\n",
       "            },\n",
       "            \"text\": \"\"\n",
       "        },\n",
       "        \"traceorder\": \"normal\",\n",
       "        \"x\": 1.05\n",
       "    },\n",
       "    \"width\": 800\n",
       "}\n",
       ");\n",
       "    });\n",
       "    </script>\n"
      ]
     },
     "execution_count": 202,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot(t, predict(p_modified, 1), label = \"First Stage UDE\", size = (800, 450))\n",
    "scatter!(train_data[:, 1], train_data[:, 2], xlabel = \"time (min)\",  ylabel = \"Concentration (mg/L)\", label = \"observation\", legend=:outerbottomright)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ae98d68",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Conclusions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03104e21",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "* The Universal Differential Equation (UDE) framework serves as a powerful tool for enhancing the capabilities of basic mathematical models, allowing them to better fit data.\n",
    "* We explored a straightforward implementation of UDE for estimating uptake kinetics in adsorption processes.\n",
    "* Additionally, we delved into the application of a fast, two-stage training methodology that employs Kernel smoothing techniques to warm up our ANN parameters.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d48ddab",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Thank you!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a448e715",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Feel free to contact me if you want to know more about it:\n",
    "    \n",
    "Email: vinicius.santana@ntnu.no\n",
    "\n",
    "Code and files are available at:"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Julia 1.10.0",
   "language": "julia",
   "name": "julia-1.10"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}