generate_figure_inputs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"toc\"></a>\n",
    "# GENERATE FIGURE INPUTS\n",
    "\n",
    "This notebook provides functions and scripts for generating input files for visualization with D3.\n",
    "\n",
    "---\n",
    "- [WORKSPACE VARIABLES](#workspace-variables)\n",
    "- **[SITE ARCHITECTURE](#site-architecture)**\n",
    "    - [point distances](#site-architecture-point-distances)\n",
    "    - [type grid](#site-architecture-type-grid)\n",
    "    - [site scatter](#site-architecture-site-scatter)\n",
    "    - [outline graph](#site-architecture-outline-graph)\n",
    "    - [site ladder](#site-architecture-site-ladder)\n",
    "- **[ESTIMATED HEMODYNAMICS](#estimated-hemodynamics)**\n",
    "    - [value sensitivity](#estimated-hemodynamics-value-sensitivity)\n",
    "- **[EXACT HEMODYNAMICS](#exact-hemodynamics)**\n",
    "    - [capillary density](#exact-hemodynamics-capillary-density)\n",
    "    - [value sensitivity](#exact-hemodynamics-value-sensitivity)\n",
    "    - [layout merged](#exact-hemodynamics-layout-merged)\n",
    "- **[VASCULAR DAMAGE](#vascular-damage)**\n",
    "    - [value sensitivity](#vascular-damage-value-sensitivity)\n",
    "- **[VASCULAR FUNCTION](#vascular-function)**\n",
    "    - [value sensitivity](#vascular-function-value-sensitivity)\n",
    "    - [layout merged](#vascular-function-layout-merged)\n",
    "    - [edge counts](#vascular-function-edge-counts)\n",
    "- ***[MULTIPLE SIMULATION SETS](#multiple-simulation-sets)***\n",
    "    - [pattern compare](#multiple-simulation-sets-pattern-compare)\n",
    "    - [pattern types](#multiple-simulation-sets-pattern-types)\n",
    "    - [layout merged](#multiple-simulation-sets-layout-merged)\n",
    "    - [layout scatter](#multiple-simulation-sets-layout-scatter)\n",
    "    - [property distribution](#multiple-simulation-sets-property-distribution)\n",
    "    - [measure ladder](#multiple-simulation-sets-measure-ladder)\n",
    "    - [delta ladder](#multiple-simulation-sets-delta-ladder)\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each of the simulation sets has a corresponding script of the same name, which contains a class (of the same name) with relevant condition variables and methods to iterate through these condition.\n",
    "The `loop` method will be used the most often to iterate through all conditions, extract relevant information, and then compile the information into a single output file.\n",
    "The `loop` method works with both individual files for each condition (`.csv` or `.json`) or a `.tar.xz` compressed archive of the files for each condition.\n",
    "For a given figure, not all conditions may be used.\n",
    "\n",
    "All generated figure inputs are provided in the `analysis` directory.\n",
    "As much as possible, files are provided in compressed archive form, using the provided `compress.sh` script, taking three arguments: `[NAME OF SIMULATION SET] [EXTENSION NAME] [json OR csv]`.\n",
    "The corresponding arguments for each of the following figures is provided in the description.\n",
    "\n",
    "The `generate.py` file contains functions to generate figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.generate import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"workspace-variables\"></a>\n",
    "\n",
    "### WORKSPACE VARIABLES\n",
    "\n",
    "Set up workspace variables for analyzing simulations.\n",
    "\n",
    "- **`DATA_PATH`** is the path to data files (`.tar.xz` files of compressed simulation outputs)\n",
    "- **`RESULTS_PATH`** is the path to result files (`.pkl` files generated by parsing)\n",
    "- **`ANALYSIS_PATH`** is the path for analysis files (`.json` and `.csv` files, `.tar.xz` compressed archives)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_PATH = \"/path/to/data/files/\"\n",
    "RESULTS_PATH = \"/path/to/result/files/\"\n",
    "ANALYSIS_PATH = \"/path/to/analysis/files/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"site-architecture\"></a>\n",
    "\n",
    "### SITE ARCHITECTURE\n",
    "\n",
    "Generate `SITE_ARCHITECTURE` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.SITE_ARCHITECTURE import SITE_ARCHITECTURE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"site-architecture-point-distances\"></a>\n",
    "\n",
    "**point distances**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/point_distances.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The point distances figure shows the maximum distance of cells from the center of the simulation as a function of point source distance from the center.\n",
    "\n",
    "The function `make_point_distances` calculates this distance for each of the point sources.\n",
    "These files are then merged into a single file `SITE_ARCHITECTURE.DISTANCES.csv` that is used as inputs for D3.\n",
    "Files created by the function `make_point_distances` can be compressed with `compress.sh` using `SITE_ARCHITECTURE .DISTANCES csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sites = [\"SOURCE\"]\n",
    "layouts = {\n",
    "    \"SOURCE\": [\n",
    "        \"point0\",\"point2\",\"point4\",\"point6\",\"point8\",\"point10\",\n",
    "        \"point12\",\"point14\",\"point16\",\"point18\",\"point20\",\n",
    "        \"point22\",\"point24\",\"point26\",\"point28\",\"point30\",\n",
    "        \"point32\",\"point34\",\"point36\",\"point38\",\n",
    "    ]\n",
    "}\n",
    "\n",
    "SITE_ARCHITECTURE.run(ANALYSIS_PATH, RESULTS_PATH, make_point_distances, timepoints=[None], \\\n",
    "                      sites=sites, layouts=layouts)\n",
    "SITE_ARCHITECTURE.loop(ANALYSIS_PATH, merge_point_distances, save_point_distances, \".DISTANCES\", \\\n",
    "                       timepoints=[None], sites=sites, layouts=layouts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"site-architecture-type-grid\"></a>\n",
    "\n",
    "**type grid**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/type_grid.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The type grid figure shows cell type distribution across the colony for different combinations of source grid spacing in the horizontal and vertical directions.\n",
    "\n",
    "The function `make_type_grid_borders` extracts the farthest border for each of the grid sources across seeds.\n",
    "These files are then merged into a single file `SITE_ARCHITECTURE.BORDERS.csv` that is used as inputs for D3.\n",
    "Files created by the function `make_type_grid_borders` can be compressed with `compress.sh` using `SITE_ARCHITECTURE.BORDERS csv`.\n",
    "\n",
    "The `.LOCATIONS` files produced in the basic analysis step are also used as input.\n",
    "Make sure to EXTRACT THE ARCHIVE FILES from `SITE_ARCHITECTURE.LOCATIONS.tar.xz`.\n",
    "Only the `SOURCE_constant`, `SOURCE_x*y*`, and `SOURCE_grid#` (# = 2, 3, 4, 5) conditions with extension `.LOCATIONS.150.csv` are needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sites = [\"SOURCE\"]\n",
    "layouts = {\n",
    "    \"SOURCE\": [\n",
    "        \"constant\",\"grid2\",\"grid3\",\"grid4\",\"grid5\",\n",
    "        \"x1y2\",\"x1y3\",\"x1y4\",\"x1y5\",\n",
    "        \"x2y1\",\"x2y3\",\"x2y4\",\"x2y5\",\n",
    "        \"x3y1\",\"x3y2\",\"x3y4\",\"x3y5\",\n",
    "        \"x4y1\",\"x4y2\",\"x4y3\",\"x4y5\",\n",
    "        \"x5y1\",\"x5y2\",\"x5y3\",\"x5y4\",\n",
    "    ]\n",
    "}\n",
    "\n",
    "SITE_ARCHITECTURE.run(ANALYSIS_PATH, RESULTS_PATH, make_type_grid_borders, timepoints=[30], \\\n",
    "                      sites=sites, layouts=layouts)\n",
    "SITE_ARCHITECTURE.loop(ANALYSIS_PATH, merge_type_grid_borders, save_type_grid_borders, \".BORDERS\", \\\n",
    "                       timepoints=[None], sites=sites, layouts=layouts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"site-architecture-site-scatter\"></a>\n",
    "\n",
    "**site scatter**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/site_scatter.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The site scatter figures shows scatter plots of emergent metrics, center concentrations, or cell type distribution as a function of sites (either point sources or grid sources).\n",
    "\n",
    "The `.SEEDS` and `.CENTERS` files produced in the basic analysis step are merged into single files per metric (`SITE_ARCHITECTURE.SEEDS.*.json` and `SITE_ARCHITECTURE.CENTERS.json`) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\", \"ACTIVITY\", \"TYPES\"]\n",
    "for metric in METRICS:\n",
    "    SITE_ARCHITECTURE.loop(ANALYSIS_PATH, merge_seeds, save_seeds, f\".SEEDS.{metric}\", timepoints=[None])\n",
    "SITE_ARCHITECTURE.loop(ANALYSIS_PATH, merge_centers, save_centers, \".CENTERS\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"site-architecture-outline-graph\"></a>\n",
    "\n",
    "**outline graph**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/outline_graph.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The outline graph figure shows colony outlines of each seed for different graph sources.\n",
    "\n",
    "The `.OUTLINES` files produced in the basic analysis step are used as input.\n",
    "Make sure to EXTRACT THE ARCHIVE FILES from `SITE_ARCHITECTURE.OUTLINES.tar.xz`.\n",
    "Only the `GRAPH`, `PATTERN`, and `SOURCE_constant` conditions with extension `.OUTLINES.150.csv` are needed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"site-architecture-site-ladder\"></a>\n",
    "\n",
    "**site ladder**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/site_ladder.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The site ladder figures show ladder plots of emergent metrics and center concentrations for different graph source layouts.\n",
    "\n",
    "These figures use the same merged files (`SITE_ARCHITECTURE.SEEDS.*.json` and`SITE_ARCHITECTURE.CENTERS.json`) as the **site scatter** figures (see above)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"estimated-hemodynamics\"></a>\n",
    "\n",
    "### ESTIMATED HEMODYNAMICS\n",
    "\n",
    "Generate `ESTIMATED_HEMODYNAMICS` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.ESTIMATED_HEMODYNAMICS import ESTIMATED_HEMODYNAMICS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"estimated-hemodynamics-value-sensitivity\"></a>\n",
    "\n",
    "**value sensitivity**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/value_sensitivity.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The value sensitivity figures shows sensitivity of emergent metrics or center concentrations to changes in estimated hemodynamic factor weights along with value from exact hemodynamics simulations.\n",
    "\n",
    "The `.SEEDS` and `.CENTERS` files produced in the basic analysis step are merged into single files per metric (`ESTIMATED_HEMODYNAMICS.SEEDS.*.json` and `ESTIMATED_HEMODYNAMICS.CENTERS.json`) which are then used as inputs for D3.\n",
    "\n",
    "These figures include files from the `EXACT_HEMODYNAMICS` set; see corresponding [value sensitivity](#exact-hemodynamics-value-sensitivity) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\", \"ACTIVITY\"]\n",
    "for metric in METRICS:\n",
    "    ESTIMATED_HEMODYNAMICS.loop(ANALYSIS_PATH, merge_seeds, save_seeds, f\".SEEDS.{metric}\", timepoints=[None])\n",
    "ESTIMATED_HEMODYNAMICS.loop(ANALYSIS_PATH, merge_centers, save_centers, \".CENTERS\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"exact-hemodynamics\"></a>\n",
    "\n",
    "### EXACT HEMODYNAMICS\n",
    "\n",
    "Generate `EXACT_HEMODYNAMICS` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.EXACT_HEMODYNAMICS import EXACT_HEMODYNAMICS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"exact-hemodynamics-capillary-density\"></a>\n",
    "\n",
    "**capillary density**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/capillary_density.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The capillary density figure shows capillaries per area for different graph source layouts as a ladder plot.\n",
    "\n",
    "The function `merge_graph` extracts individual graph metrics from the `.GRAPH` files produced in the basic analysis step.\n",
    "Capillary density is approximated as the number of capillary edges that cross vertical sections equally spaces across the environment.\n",
    "Values across conditions and times are merged into a single file `EXACT_HEMODYNAMICS.GRAPH.DENSITY.json` that is used as input for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ALL_TIMES = [\"010\", \"020\", \"030\", \"040\", \"050\", \"060\", \"070\", \"080\", \"090\", \"100\", \"110\", \"120\", \"130\", \"140\", \"150\"]\n",
    "EXACT_HEMODYNAMICS.loop(ANALYSIS_PATH, merge_graph, save_graph, \".GRAPH.DENSITY\", timepoints=ALL_TIMES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"exact-hemodynamics-value-sensitivity\"></a>\n",
    "\n",
    "**value sensitivity**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/value_sensitivity.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The value sensitivity figures shows sensitivity of emergent metrics or center concentrations to changes in estimated hemodynamic factor weights along with value from exact hemodynamics simulations.\n",
    "\n",
    "The `.SEEDS` and `.CENTERS` files produced in the basic analysis step are merged into single files per metric (`EXACT_HEMODYNAMICS.SEEDS.*.json` and `EXACT_HEMODYNAMICS.CENTERS.json`) which are then used as inputs for D3.\n",
    "\n",
    "These figures include files from the `ESTIMATED_HEMODYNAMICS` set; see corresponding [value sensitivity](#estimated-hemodynamics-value-sensitivity) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\", \"ACTIVITY\"]\n",
    "for metric in METRICS:\n",
    "    EXACT_HEMODYNAMICS.loop(ANALYSIS_PATH, merge_seeds, save_seeds, f\".SEEDS.{metric}\", timepoints=[None])\n",
    "EXACT_HEMODYNAMICS.loop(ANALYSIS_PATH, merge_centers, save_centers, \".CENTERS\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"exact-hemodynamics-layout-merged\"></a>\n",
    "\n",
    "**layout merged**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/layout_merged.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The layout merged figures compares emergent metrics or center concentrations for different graph layouts.\n",
    "\n",
    "The functions `make_layout_merged_metrics` and `make_layout_merged_concentrations` extracts timecourse of metrics or concentrations, respectively, for each seed.\n",
    "Files created by the functions can be compressed with `compress.sh` using `EXACT_HEMODYNAMICS .MERGED json`.\n",
    "\n",
    "Files for these figures are combined with files from the `VASCULAR_FUNCTION` set; see corresponding [layout merged](#vascular-function-layout-merged) section and [combined layout merged](#multiple-simulation-sets-layout-merged) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "EXACT_HEMODYNAMICS.run(ANALYSIS_PATH, RESULTS_PATH, make_layout_merged_metrics)\n",
    "EXACT_HEMODYNAMICS.load(ANALYSIS_PATH, DATA_PATH, make_layout_merged_concentrations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-damage\"></a>\n",
    "\n",
    "### VASCULAR DAMAGE\n",
    "\n",
    "Generate `VASCULAR_DAMAGE` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.VASCULAR_DAMAGE import VASCULAR_DAMAGE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-damage-value-sensitivity\"></a>\n",
    "\n",
    "**value sensitivity**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/value_sensitivity.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The value sensitivity figures shows sensitivity of emergent metrics or center concentrations to changes in vascular damage fraction along with value from simulations with degradation and remodeling.\n",
    "\n",
    "The `.SEEDS` and `.CENTERS` files produced in the basic analysis step are merged into single files per metric (`VASCULAR_DAMAGE.SEEDS.*.json` and `VASCULAR_DAMAGE.CENTERS.json`) which are then used as inputs for D3.\n",
    "\n",
    "These figures include files from the `VASCULAR_FUNCTION` set; see corresponding [value sensitivity](#vascular-function-value-sensitivity) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\", \"ACTIVITY\"]\n",
    "for metric in METRICS:\n",
    "    VASCULAR_DAMAGE.loop(ANALYSIS_PATH, merge_seeds, save_seeds, f\".SEEDS.{metric}\", timepoints=[None])\n",
    "VASCULAR_DAMAGE.loop(ANALYSIS_PATH, merge_centers, save_centers, \".CENTERS\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-function\"></a>\n",
    "\n",
    "### VASCULAR FUNCTION\n",
    "\n",
    "Generate `VASCULAR_FUNCTION` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.VASCULAR_FUNCTION import VASCULAR_FUNCTION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-function-value-sensitivity\"></a>\n",
    "\n",
    "**value sensitivity**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/value_sensitivity.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The value sensitivity figures shows sensitivity of emergent metrics or center concentrations to changes in vascular damage fraction along with value from simulations with degradation and remodeling.\n",
    "\n",
    "The `.SEEDS` and `.CENTERS` files produced in the basic analysis step are merged into single files per metric (`VASCULAR_FUNCTION.SEEDS.*.json` and `VASCULAR_FUNCTION.CENTERS.json`) which are then used as inputs for D3.\n",
    "\n",
    "These figures include files from the `VASCULAR_DAMAGE` set; see corresponding [value sensitivity](#vascular-damage-value-sensitivity) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\", \"ACTIVITY\"]\n",
    "for metric in METRICS:\n",
    "    VASCULAR_FUNCTION.loop(ANALYSIS_PATH, merge_seeds, save_seeds, f\".SEEDS.{metric}\", timepoints=[None])\n",
    "VASCULAR_FUNCTION.loop(ANALYSIS_PATH, merge_centers, save_centers, \".CENTERS\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-function-layout-merged\"></a>\n",
    "\n",
    "**layout merged**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/layout_merged.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The layout merged figures compares emergent metrics or center concentrations for different graph layouts.\n",
    "\n",
    "The functions `make_layout_merged_metrics` and `make_layout_merged_concentrations` extracts timecourse of metrics or concentrations, respectively, for each seed.\n",
    "Files created by the functions can be compressed with `compress.sh` using `VASCULAR_FUNCTION .MERGED json`.\n",
    "\n",
    "Files for these figures are combined with files from the `EXACT_HEMODYNAMICS` set; see corresponding [layout merged](#exact-hemodynamics-layout-merged) section and [combined layout merged](#multiple-simulation-sets-layout-merged) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "VASCULAR_FUNCTION.run(ANALYSIS_PATH, RESULTS_PATH, make_layout_merged_metrics)\n",
    "VASCULAR_FUNCTION.load(ANALYSIS_PATH, DATA_PATH, make_layout_merged_concentrations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-function-edge-counts\"></a>\n",
    "\n",
    "**edge counts**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/edge_counts.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The edge counts figure shows the fraction of edges remaining in the simulation over time for different root layouts.\n",
    "\n",
    "The function `make_edge_counts` calculates the fraction of edges for each root layout.\n",
    "These files are then merged into a single file `VASCULAR_FUNCTION.EDGES.json` that is used as inputs for D3.\n",
    "Files created by the function `make_edge_counts` can be compressed with `compress.sh` using `VASCULAR_FUNCTION .EDGES json`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "VASCULAR_FUNCTION.load(ANALYSIS_PATH, DATA_PATH, make_edge_counts, extension=\".GRAPH\")\n",
    "VASCULAR_FUNCTION.loop(ANALYSIS_PATH, merge_edge_counts, save_edge_counts, \".EDGES\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"multiple-simulation-sets\"></a>\n",
    "\n",
    "### MULTIPLE SIMULATION SETS\n",
    "\n",
    "Generate figure inputs that draw from files from more than one simulation set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"multiple-simulation-sets-pattern-compare\"></a>\n",
    "\n",
    "**pattern compare**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/pattern_compare.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The pattern compare figures shows emergent metrics or center concentrations over time for the pattern layout with varying hemodynamics and damage (using the `EXACT_HEMODYNAMICS`, `VASCULAR_DAMAGE`, and `VASCULAR_FUNCTION` simulation sets).\n",
    "\n",
    "For emergent metrics, the `.METRICS` files produced in the basic analysis step are merged into single files per metric (`PATTERN_COMPARE.*.json`) which are then used as inputs for D3.\n",
    "For concentrations, we loop through select data files `.tar.xz` to extract glucose and oxygen concentration at the center of the simulation for all timepoints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NAMES = [\"EXACT_HEMODYNAMICS\", \"VASCULAR_DAMAGE\", \"VASCULAR_FUNCTION\"]\n",
    "CONTEXTS = [\"C\", \"CHX\"]\n",
    "CASES = {\n",
    "    \"EXACT_HEMODYNAMICS\": [\n",
    "        [(\"graphs\", \"PATTERN\")],\n",
    "    ],\n",
    "    \"VASCULAR_DAMAGE\": [\n",
    "        [(\"value\", \"000\"), (\"frac\", \"000\")],\n",
    "        [(\"value\", \"100\"), (\"frac\", \"000\")],\n",
    "        [(\"value\", \"000\"), (\"frac\", \"100\")],\n",
    "        [(\"value\", \"100\"), (\"frac\", \"100\")],\n",
    "    ],\n",
    "    \"VASCULAR_FUNCTION\": [\n",
    "        [(\"graphs\", \"PATTERN\")],\n",
    "    ],\n",
    "}\n",
    "\n",
    "for metric in [\"GROWTH\", \"SYMMETRY\", \"CYCLES\", \"ACTIVITY\"]:\n",
    "    make_pattern_compare_metrics(ANALYSIS_PATH, metric, NAMES, CONTEXTS, CASES)\n",
    "make_pattern_compare_concentrations(DATA_PATH, ANALYSIS_PATH, NAMES, CONTEXTS, CASES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"multiple-simulation-sets-pattern-types\"></a>\n",
    "\n",
    "**pattern types**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/pattern_types.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The pattern types figures shows cell type distribution across the colony for varying hemodynamics and damage (using the `EXACT_HEMODYNAMICS`, `VASCULAR_DAMAGE`, and `VASCULAR_FUNCTION` simulation sets).\n",
    "\n",
    "The function `make_pattern_types_locations` merges selected `.LOCATIONS` files produced in the basic analysis step into a single file (`PATTERN_TYPES.LOCATIONS.csv`).\n",
    "The function `make_pattern_types_borders` extracts the farthest border for each of the combinations across seeds, merged into a single file `PATTERN_TYPES.BORDERS.csv`.\n",
    "Both files are used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NAMES = [\"VASCULAR_DAMAGE\",\"EXACT_HEMODYNAMICS\",  \"VASCULAR_FUNCTION\"]\n",
    "CONTEXTS = [\"C\", \"CHX\"]\n",
    "CASES = {\n",
    "    \"EXACT_HEMODYNAMICS\": [\n",
    "        (\"PATTERN\", \"exact\", \"no\"),\n",
    "    ],\n",
    "    \"VASCULAR_DAMAGE\": [\n",
    "        (\"000_000\", \"static\", \"no\"),\n",
    "        (\"000_100\", \"simple\", \"no\"),\n",
    "        (\"100_000\", \"static\", \"yes\"),\n",
    "        (\"100_100\",  \"simple\", \"yes\"),\n",
    "    ],\n",
    "    \"VASCULAR_FUNCTION\": [\n",
    "        (\"PATTERN\", \"exact\", \"yes\"),\n",
    "    ],\n",
    "}\n",
    "\n",
    "make_pattern_types_locations(ANALYSIS_PATH, NAMES, CONTEXTS, CASES)\n",
    "make_pattern_types_borders(RESULTS_PATH, ANALYSIS_PATH, NAMES, CONTEXTS, CASES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"multiple-simulation-sets-layout-merged\"></a>\n",
    "\n",
    "**layout merged**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/layout_merged.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The layout merged figures compares emergent metrics or center concentrations for different graph layouts.\n",
    "\n",
    "The `make_layout_merged` function loops through the simulation sets and contexts to calculated mean, standard deviation, and confidence interval for the PATTERN layout and all five GRAPH layouts.\n",
    "\n",
    "Files for these figures use outputs from the `EXACT_HEMODYNAMICS` and `VASCULAR_FUNCTION` sets: see corresponding `EXACT_HEMODYNAMICS`[layout merged](#exact-hemodynamics-layout-merged) section and `VASCULAR_FUNCTION` [layout merged](#vascular-function-layout-merged) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\", \"ACTIVITY\", \"GLUCOSE\", \"OXYGEN\"]\n",
    "for metric in METRICS:\n",
    "    make_layout_merged(ANALYSIS_PATH, metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"multiple-simulation-sets-layout-scatter\"></a>\n",
    "\n",
    "**layout scatter**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/layout_scatter.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The layout scatter figures show different hemodynamic properties for different layouts and coupling, with overlaid experimental data.\n",
    "\n",
    "The `make_layout_scatter` function loops through selected `.GRAPH` files produced in the basic analysis step from the `EXACT_HEMODYNAMICS` (uncoupled) and `VASCULAR_FUNCTION` (coupled) sets.\n",
    "Entries are separated into pattern layout (`PATTERN`) or the root layout (`Lav`, `Lava`, `Lvav`, `Sav`, and `Savav`) outputs.\n",
    "\n",
    "Note that the figure only shows a randomly selected subset of edges for clarity, but kernel smoothed lines are calculated across all edges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_layout_scatter(ANALYSIS_PATH)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"multiple-simulation-sets-property-distribution\"></a>\n",
    "\n",
    "**property distribution**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/property_distribution.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The property distribution figures show distribution of different hemodynamic properties for different layouts and coupling as violins.\n",
    "\n",
    "The `make_property_distribution` function loops through selected `.GRAPH` files produced in the basic analysis step from the `EXACT_HEMODYNAMICS` and `VASCULAR_FUNCTION` sets to extract relevent properties (`RADIUS`, `WALL`, `SHEAR`, `CIRCUM`, `FLOW`, and `PRESSURE`.\n",
    "Merged files (`PROPERTY_DISTRIBUTION.*.json`) contain dictionaries of binned data keyed on the context and layout type.\n",
    "\n",
    "The `VASCULAR_FUNCTION` set uses the `C` (colony) and `CH` (tissue) contexts.\n",
    "The `EXACT_HEMODYNAMICS` set (which does not alter the vascular structure) uses the `C/CH` context, indicating that properties do not differ between contexts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_property_distribution(ANALYSIS_PATH)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"multiple-simulation-sets-measure-ladder\"></a>\n",
    "\n",
    "**measure ladder** \n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/measure_ladder.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The measure ladder figures show ladder plots of graph measures for different root layouts.\n",
    "\n",
    "The `make_graph_measures` function loops through selected `.MEASURES` files produced in the basic analysis step from the `EXACT_HEMODYNAMICS` and `VASCULAR_FUNCTION` sets to extract graph measures at day 15 into a single file (`GRAPH_MEASURES.csv`)\n",
    "The `VASCULAR_FUNCTION` set uses the `C` (colony) and `CH` (tissue) contexts.\n",
    "The `EXACT_HEMODYNAMICS` set (which does not alter the vascular structure) uses the `C/CH` context, indicating that measures do not differ between contexts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_graph_measures(ANALYSIS_PATH)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"multiple-simulation-sets-delta-ladder\"></a>\n",
    "\n",
    "**delta ladder**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/delta_ladder.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The delta ladder figures show ladder plots of change in emergent metrics and center concentrations for different graphs with and without coupling.\n",
    "\n",
    "These figures use the same merged files (`*.SEEDS.*.json`, `*.CENTERS.json`) from the `EXACT_HEMODYNAMICS` and `VASCULAR_FUNCTION` sets as the **value sensitivity** figures (see above)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}