python/lecture2-discovery.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Applied Process Mining Module\r\n",
    "\r\n",
    "This notebook is part of an Applied Process Mining module. The collection of notebooks is a *living document* and subject to change. \r\n",
    "\r\n",
    "# Lecture 2 - 'Process Discovery with the Heuristics Miner' (Python / PM4Py)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Setup\n",
    "\n",
    "<img src=\"https://pm4py.fit.fraunhofer.de/static/assets/images/pm4py-site-logo-padded.png\" alt=\"PM4Py\" style=\"width: 200px;\"/>\n",
    "\n",
    "In this notebook, we are using the [PM4Py library](https://pm4py.fit.fraunhofer.de/) in combination with several standard Python data science libraries:\n",
    "\n",
    "* [pandas](https://pandas.pydata.org/)\n",
    "* [plotnine](https://plotnine.readthedocs.io/en/stable/)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## Perform the commented out commands to install the dependencies\r\n",
    "# %pip install pandas\r\n",
    "# %pip install matplotlib\r\n",
    "# %pip install pm4py"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "import pandas as pd\r\n",
    "import pm4py\r\n",
    "import plotnine\r\n",
    "from plotnine import ggplot, geom_point, aes, theme_bw, coord_flip, scale_y_discrete, theme, element_text, geom_bin2d"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Process Discovery"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Process Models\n",
    "\n",
    "A common industry standard for Process Model is [BPMN](https://www.bpmn.org/), which can be created by web-based tools like [BPMN.io](https://bpmn.io/) and loaded and used by PM4Py."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "import pm4py\r\n",
    "import os\r\n",
    "\r\n",
    "bpmn_graph = pm4py.read_bpmn(os.path.join(\"..\", \"data\", \"sepsis.bpmn\"))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "pm4py.vis.view_bpmn(bpmn_graph)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Quality Dimensions\n",
    "\n",
    "The quality dimensions `fitness`, `precision`, `simplicity`, and `generalisation` are best illustrated by using a small example event log.\n",
    "We are using an example event log in XES format that is used in the book `Process Mining - Data Science in Action` by Wil van der Aalst, which is downloaded and stored in the `../data` directory with the code below:"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Just to be able to download and extract the file\n",
    "def load_file_from_zip(url, file, target):\n",
    "    from io import BytesIO\n",
    "    from urllib.request import urlopen\n",
    "    from zipfile import ZipFile        \n",
    "    with urlopen(url) as zipresp:\n",
    "        with ZipFile(BytesIO(zipresp.read())) as zfile:\n",
    "            zipInfo = zfile.getinfo(file)\n",
    "            zipInfo.filename = os.path.basename(target)\n",
    "            zfile.extract(zipInfo, os.path.dirname(os.path.realpath(target)))\n",
    "\n",
    "load_file_from_zip('http://www.processmining.org/_media/processminingbook/chapter_7.zip', 'Chapter_7/Lfull.xes', '../data/Lfull.xes')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from pm4py.objects.log.importer.xes import importer as xes_importer\n",
    "example_log = xes_importer.apply('../data/Lfull.xes')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Let us have a look at the event log in tabular form. The mapping of the activity labels to actual activities is:\n",
    "\n",
    "* a = register request, \n",
    "* b = examine thoroughly, \n",
    "* c = examine casually, \n",
    "* d = check ticket, \n",
    "* e = decide, \n",
    "* f = reinitiate request, \n",
    "* g = pay compensation, and \n",
    "* h = reject request."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "pm4py.convert_to_dataframe(example_log)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now let us discover a process map as we have seen in Lecture 1:"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "pm4py.view_dfg(*pm4py.discover_directly_follows_graph(example_log))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Not really very insightful the directly-follows based process map visualization."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Four Process Models - One Event Log 🚧"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "This part is under construction."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Heuristics Miner 🚧"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "L_heur_1 = pd.read_csv(\"../data/L_heur_1.csv\", sep=';')\n",
    "num_rows = len(L_heur_1)\n",
    "print(\"Number of rows: {}\".format(num_rows))\n",
    "L_heur_1_log = pm4py.format_dataframe(L_heur_1, case_id='CASE_concept_name', activity_key='activity_id', timestamp_key='timestamp')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "dfg, sa, ea = pm4py.discover_directly_follows_graph(L_heur_1_log)\n",
    "pm4py.view_dfg(dfg, sa, ea)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Dependency Graph 🚧"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "In PM4Py, there is no option to review the intermediate data structures used for the Heuristics Miner such as the dependency graph.\n",
    "\n",
    "🚧"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Causal net"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "The Causal net formalism is called `Heuristics Net` in PM4Py along with the definitions used in the original paper that proposed the Heuristics Miner."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from pm4py.algo.discovery.heuristics import algorithm as heuristics_miner\n",
    "\n",
    "# due to bug here: https://github.com/pm4py/pm4py-core/issues/220\n",
    "# we need to convert this to a event log object\n",
    "\n",
    "L_heur_1_log = pm4py.convert_to_event_log(L_heur_1_log)\n",
    "L_heur_1_map = pm4py.discover_heuristics_net(L_heur_1_log, dependency_threshold = 0.8, and_threshold = 0.65, loop_two_threshold = 0.5)\n",
    "pm4py.view_heuristics_net(L_heur_1_map)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Convert to BPMN"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from pm4py.objects.conversion.wf_net.variants import to_bpmn\n",
    "\n",
    "# we need to have an intermediate step to get a Petri net\n",
    "L_heur_1_petrinet, im, fm = pm4py.convert_to_petri_net(L_heur_1_map)\n",
    "L_heur_1_bpmn = to_bpmn.apply(L_heur_1_petrinet, im, fm)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "pm4py.view_bpmn(L_heur_1_bpmn)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from pm4py.objects.petri_net.exporter import exporter as pnml_exporter\n",
    "pnml_exporter.apply(L_heur_1_petrinet, im, \"petri_final.pnml\", final_marking=fm)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [],
   "outputs": [],
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}