analyze_data_results.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"toc\"></a>\n",
    "# ANALYZE DATA & RESULTS\n",
    "\n",
    "This notebook provides functions and scripts for running basic analysis on simulation data and parsed results.\n",
    "\n",
    "---\n",
    "- [WORKSPACE VARIABLES](#workspace-variables)\n",
    "- **[SITE ARCHITECTURE](#site-architecture)**\n",
    "- **[ESTIMATED HEMODYNAMICS](#estimated-hemodynamics)**\n",
    "- **[EXACT HEMODYNAMICS](#exact-hemodynamics)**\n",
    "- **[VASCULAR DAMAGE](#vascular-damage)**\n",
    "- **[VASCULAR FUNCTION](#vascular-function)**\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following basis analyses are performed for every simulation set:\n",
    "\n",
    "- `analyze_metrics` extracts number of cells (`COUNTS`), total volume (`VOLUMES`), average cell cycle length (`CYCLES`), colony diameter (`DIAMETERS`), number of cell of each type (`TYPES`), number of cells in each populations (`POPS`), growth rate (`GROWTH`), symmetry (`SYMMETRY`), and activity (`ACTIVITY`) across time\n",
    "- `analyze_seeds` extracts the above metrics per seed at selected timepoints\n",
    "- `analyze_locations` extracts cell counts, volumes, types, and populations per location at selected timepoints\n",
    "- `analyze_distribution` extracts distribution of cell types and populations as a function of radius from the center of the simulation\n",
    "- `analyze_outlines` extracts colony outline at selected timepoints\n",
    "- `analyze_concentrations` extracts concentration profiles as a function of radius from the center of the simulation over time\n",
    "- `analyze_centers` extracts concentration of glucose, oxygen, and TGF$\\alpha$ at the center of the simulations\n",
    "\n",
    "The following basic analyses are performed only for simulation sets using graph sites to represent vasculature:\n",
    "\n",
    "- `analyze_graphs` extracts vascular graph edge and node information (coordinates of edge, pressure and oxygen partial pressure at each node, radius, length, wall thickness, shear stress, circumferential stress, and flow rate)\n",
    "- `analyze_measures` extracts graph measures (number of edges, number of nodes, graph radius and diameter, degree, eccentricity, shortest path, clustering, closeness centrality, betweenness centrality, and number of components)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each of the simulation sets has a corresponding script of the same name, which contains a class (of the same name) with relevant condition variables and methods to iterate through these condition.\n",
    "\n",
    "The `analyze.py` file contains general analysis functions to perform each of the above analyses as well as specific methods to calculate the above listed metrics and measures.\n",
    "\n",
    "Analysis can take some time, so all resulting `.json` and `.csv` files are provided in the `analysis` directory.\n",
    "Note that not all analyses need to be run at the same time.\n",
    "\n",
    "These files are compressed using the provided `compress.sh` script, which takes the simulation set name as an argument:\n",
    "\n",
    "```bash\n",
    "$ ./compress.sh SITE_ARCHITECTURE\n",
    "$ ./compress.sh ESTIMATED HEMODYNAMICS\n",
    "$ ./compress.sh EXACT_HEMODYNAMICS\n",
    "$ ./compress.sh VASCULAR_DAMAGE\n",
    "$ ./compress.sh VASCULAR_FUNCTION\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from scripts.analyze import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"workspace-variables\"></a>\n",
    "\n",
    "### WORKSPACE VARIABLES\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Set up workspace variables for analyzing simulations.\n",
    "\n",
    "- **`DATA_PATH`** is the path to data files (`.tar.xz` files of compressed simulation outputs)\n",
    "- **`RESULTS_PATH`** is the path to result files (`.pkl` files generated by parsing)\n",
    "- **`ANALYSIS_PATH`** is the path for analysis files (`.json` and `.csv` files, `.tar.xz` compressed archives)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_PATH = \"/path/to/data/files/\"\n",
    "RESULTS_PATH = \"/path/to/result/files/\"\n",
    "ANALYSIS_PATH = \"/path/to/analysis/files/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **`TIMEPOINTS`** is a list of select timepoint indices\n",
    "- **`ALL_TIMEPOINTS`** is a list of all timepoint indices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "TIMEPOINTS = [2, 16, 30]\n",
    "ALL_TIMEPOINTS = [2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"site-architecture\"></a>\n",
    "\n",
    "### SITE ARCHITECTURE\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Analyze `SITE_ARCHITECTURE` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.SITE_ARCHITECTURE import SITE_ARCHITECTURE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SITE_ARCHITECTURE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_metrics, timepoints=TIMEPOINTS)\n",
    "SITE_ARCHITECTURE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_seeds, timepoints=TIMEPOINTS)\n",
    "SITE_ARCHITECTURE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_locations, timepoints=TIMEPOINTS)\n",
    "SITE_ARCHITECTURE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_distribution, timepoints=TIMEPOINTS)\n",
    "SITE_ARCHITECTURE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_outlines, timepoints=TIMEPOINTS)\n",
    "SITE_ARCHITECTURE.load(ANALYSIS_PATH, DATA_PATH, analyze_concentrations, timepoints=ALL_TIMEPOINTS)\n",
    "SITE_ARCHITECTURE.load(ANALYSIS_PATH, DATA_PATH, analyze_centers, timepoints=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"estimated-hemodynamics\"></a>\n",
    "\n",
    "### ESTIMATED HEMODYNAMICS\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Analyze `ESTIMATED_HEMODYNAMICS` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.ESTIMATED_HEMODYNAMICS import ESTIMATED_HEMODYNAMICS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ESTIMATED_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_metrics, timepoints=TIMEPOINTS)\n",
    "ESTIMATED_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_seeds, timepoints=TIMEPOINTS)\n",
    "ESTIMATED_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_locations, timepoints=TIMEPOINTS)\n",
    "ESTIMATED_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_distribution, timepoints=TIMEPOINTS)\n",
    "ESTIMATED_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_outlines, timepoints=TIMEPOINTS)\n",
    "ESTIMATED_HEMODYNAMICS.load(ANALYSIS_PATH, DATA_PATH, analyze_concentrations, timepoints=ALL_TIMEPOINTS)\n",
    "ESTIMATED_HEMODYNAMICS.load(ANALYSIS_PATH, DATA_PATH, analyze_centers, timepoints=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"exact-hemodynamics\"></a>\n",
    "\n",
    "### EXACT HEMODYNAMICS\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Analyze `EXACT_HEMODYNAMICS` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.EXACT_HEMODYNAMICS import EXACT_HEMODYNAMICS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "EXACT_HEMODYNAMICS_ROOT_GRAPHS = EXACT_HEMODYNAMICS.GRAPHS[1:] # ignore PATTERN layout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "EXACT_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_metrics, timepoints=TIMEPOINTS)\n",
    "EXACT_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_seeds, timepoints=TIMEPOINTS)\n",
    "EXACT_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_locations, timepoints=TIMEPOINTS)\n",
    "EXACT_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_distribution, timepoints=TIMEPOINTS)\n",
    "EXACT_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, analyze_outlines, timepoints=TIMEPOINTS)\n",
    "EXACT_HEMODYNAMICS.load(ANALYSIS_PATH, DATA_PATH, analyze_concentrations, timepoints=ALL_TIMEPOINTS)\n",
    "EXACT_HEMODYNAMICS.load(ANALYSIS_PATH, DATA_PATH, analyze_centers, timepoints=-1)\n",
    "EXACT_HEMODYNAMICS.load(ANALYSIS_PATH, DATA_PATH, analyze_graphs, timepoints=ALL_TIMEPOINTS, extension=\".GRAPH\")\n",
    "EXACT_HEMODYNAMICS.load(ANALYSIS_PATH, DATA_PATH, analyze_measures, graphs=EXACT_HEMODYNAMICS_ROOT_GRAPHS, timepoints=ALL_TIMEPOINTS, extension=\".GRAPH\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-damage\"></a>\n",
    "\n",
    "### VASCULAR DAMAGE\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Analyze `VASCULAR_DAMAGE` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.VASCULAR_DAMAGE import VASCULAR_DAMAGE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "VASCULAR_DAMAGE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_metrics, timepoints=TIMEPOINTS)\n",
    "VASCULAR_DAMAGE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_seeds, timepoints=TIMEPOINTS)\n",
    "VASCULAR_DAMAGE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_locations, timepoints=TIMEPOINTS)\n",
    "VASCULAR_DAMAGE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_distribution, timepoints=TIMEPOINTS)\n",
    "VASCULAR_DAMAGE.run(ANALYSIS_PATH, RESULTS_PATH, analyze_outlines, timepoints=TIMEPOINTS)\n",
    "VASCULAR_DAMAGE.load(ANALYSIS_PATH, DATA_PATH, analyze_concentrations, timepoints=ALL_TIMEPOINTS)\n",
    "VASCULAR_DAMAGE.load(ANALYSIS_PATH, DATA_PATH, analyze_centers, timepoints=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-function\"></a>\n",
    "\n",
    "### VASCULAR FUNCTION\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Analyze `VASCULAR_FUNCTION` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.VASCULAR_FUNCTION import VASCULAR_FUNCTION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "VASCULAR_FUNCTION_ROOT_GRAPHS = VASCULAR_FUNCTION.GRAPHS[1:] # ignore PATTERN layout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "VASCULAR_FUNCTION.run(ANALYSIS_PATH, RESULTS_PATH, analyze_metrics, timepoints=TIMEPOINTS)\n",
    "VASCULAR_FUNCTION.run(ANALYSIS_PATH, RESULTS_PATH, analyze_seeds, timepoints=TIMEPOINTS)\n",
    "VASCULAR_FUNCTION.run(ANALYSIS_PATH, RESULTS_PATH, analyze_locations, timepoints=TIMEPOINTS)\n",
    "VASCULAR_FUNCTION.run(ANALYSIS_PATH, RESULTS_PATH, analyze_distribution, timepoints=TIMEPOINTS)\n",
    "VASCULAR_FUNCTION.run(ANALYSIS_PATH, RESULTS_PATH, analyze_outlines, timepoints=TIMEPOINTS)\n",
    "VASCULAR_FUNCTION.load(ANALYSIS_PATH, DATA_PATH, analyze_concentrations, timepoints=ALL_TIMEPOINTS)\n",
    "VASCULAR_FUNCTION.load(ANALYSIS_PATH, DATA_PATH, analyze_centers, timepoints=-1)\n",
    "VASCULAR_FUNCTION.load(ANALYSIS_PATH, DATA_PATH, analyze_graphs, timepoints=ALL_TIMEPOINTS, extension=\".GRAPH\")\n",
    "VASCULAR_FUNCTION.load(ANALYSIS_PATH, DATA_PATH, analyze_measures, graphs=VASCULAR_FUNCTION_ROOT_GRAPHS, timepoints=ALL_TIMEPOINTS, extension=\".GRAPH\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}