python/lecture3-conformance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Applied Process Mining Module\r\n",
    "\r\n",
    "This notebook is part of an Applied Process Mining module. The collection of notebooks is a *living document* and subject to change. \r\n",
    "\r\n",
    "# Lecture 3 - 'Conformance Checking' (Python / PM4Py)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Setup\n",
    "\n",
    "<img src=\"https://pm4py.fit.fraunhofer.de/static/assets/images/pm4py-site-logo-padded.png\" alt=\"PM4Py\" style=\"width: 200px;\"/>\n",
    "\n",
    "In this notebook, we are using the [PM4Py library](https://pm4py.fit.fraunhofer.de/) in combination with several standard Python data science libraries:\n",
    "\n",
    "* [pandas](https://pandas.pydata.org/)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## Perform the commented out commands to install the dependencies\r\n",
    "# %pip install pandas\r\n",
    "# %pip install matplotlib\r\n",
    "# %pip install pm4py"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "import pandas as pd\r\n",
    "import pm4py\r\n",
    "import os"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Conformance Checking"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Process Models\n",
    "\n",
    "A common industry standard for Process Model is [BPMN](https://www.bpmn.org/), which can be created by web-based tools like [BPMN.io](https://bpmn.io/) and loaded and used by PM4Py. We load a manually drawn BPMN model that described the expected process behaviour of the `Sepsis Process`. This is a process that describes several logistical and diagnostic activities performed for patients in a hospital that are suspected to have a life-threatening sepsis condition. An event log that was extracted from the information system of a hospital is publicly available and serves as ouor running example."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "sepsis_bpmn = pm4py.read_bpmn(os.path.join(\"..\", \"data\", \"sepsis.bpmn\"))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "pm4py.vis.view_bpmn(sepsis_bpmn)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Alignments\n",
    "\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "sepsis = pd.read_csv(\"../data/sepsis.csv\", sep=';')\n",
    "num_rows = len(sepsis)\n",
    "print(\"Number of rows: {}\".format(num_rows))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "sepsis.head()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "#sepsis['timestamp'] = pd.to_datetime(sepsis['timestamp'])\n",
    "sepsis_log = pm4py.format_dataframe(sepsis, case_id='case_id', activity_key='activity', timestamp_key='timestamp')\n",
    "sepsis_log = pm4py.convert_to_event_log(sepsis_log)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now let us discover a process map as we have seen in Lecture 1:"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from pm4py.algo.discovery.dfg import algorithm as dfg_discovery\n",
    "from pm4py.algo.discovery.dfg import algorithm as dfg_discovery\n",
    "from pm4py.visualization.dfg import visualizer as dfg_visualization\n",
    "\n",
    "dfg = dfg_discovery.apply(sepsis_log)\n",
    "\n",
    "dfg = dfg_discovery.apply(sepsis_log, variant=dfg_discovery.Variants.PERFORMANCE)\n",
    "gviz = dfg_visualization.apply(dfg, log=sepsis_log, variant=dfg_visualization.Variants.PERFORMANCE)\n",
    "dfg_visualization.view(gviz)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "This is clearly not usable at all. In Lecture 2, we have seen how to use Process Discovery methods to overcome this problem to a certain degree.\n",
    "However, when having a normative process model such as the handmade BPMN model of the Sepsis Process there is another way to get value from the event log: by means of conformance checking with alignments.\n",
    "\n",
    "In PM4Py, we can convert the BPMN model to a Petri net and then compute an optimal alignment between the traces contained in the event log and our model. This will give us, for each trace, a mapping from the events to the path through the model (or run) that is most similar to what has been recorded in the event log. (**Warning**: This is computationally expensive and may take a while)\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from pm4py.objects.conversion.bpmn import converter as bpmn_converter\n",
    "\n",
    "sepsis_net, sepsis_im, sepsis_fm = bpmn_converter.apply(sepsis_bpmn)\n",
    "\n",
    "sepsis_alignment = pm4py.conformance_diagnostics_alignments(sepsis_log, sepsis_net, sepsis_im, sepsis_fm) "
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "This is how an *alignment* of an individual trace looks like:"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "sepsis_alignment[0]"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "This can be used to project statistics on top of the Petri net representation:"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from pm4py.visualization.petri_net import visualizer as pn_vis\n",
    "\n",
    "pn_vis.apply(sepsis_net, sepsis_im, sepsis_fm, sepsis_log, variant = pn_vis.Variants.ALIGNMENTS)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "The statistics can be read as (number of skips or move on model, number of correct or synchronous events)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [],
   "outputs": [],
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}