LHS_PRCC.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "LHS-PRCC.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/scmassey/model-sensitivity-analysis/blob/master/LHS_PRCC.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LynXJ5FPHndO",
        "colab_type": "text"
      },
      "source": [
        "# Latin Hypercube Sampling & Partial Rank Correlation Coefficients  <br/> *~ a method for analyzing model sensitivity to parameters ~*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PHmmvxwMHndP",
        "colab_type": "text"
      },
      "source": [
        "#### Importing packages that will be used."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZC6x-a7WHndQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "from scipy import special\n",
        "\n",
        "import random\n",
        "\n",
        "from ipywidgets import interact, interactive, fixed, interact_manual\n",
        "import ipywidgets as widgets\n",
        "from IPython.display import display\n",
        "\n",
        "import pandas as pd\n",
        "\n",
        "import matplotlib.pyplot as plt\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U0_okb6XHndS",
        "colab_type": "text"
      },
      "source": [
        "### Specify the number of parameters to sample and the number of samples to draw from each parameter distribution. \n",
        "#### *Do not include any parameters that should be left fixed in parameterCount - those will be specified later. When choosing number of samples to draw, note that more samples (~ 1000) yields better results while fewer (~50) is faster for testing, since it requires fewer model solves.* "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EZy2ndbCHndT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Number of parameters to sample \n",
        "parameterCount = 2;\n",
        "\n",
        "# Number of samples to draw for each parameter\n",
        "sampleCount = 100; "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i2vYkow7HndV",
        "colab_type": "text"
      },
      "source": [
        "#### This defines functions for specifying sampled parameters' names and distributions  as well as drawing samples from a user-specified parameter distribution. Does not need any user edits."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O0kGl8MKHndV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def parNameDist(Name,Distribution):\n",
        "    paramTemp = {}\n",
        "    paramTemp['Name']=Name\n",
        "    paramTemp['Dist']=Distribution\n",
        "    \n",
        "    return paramTemp\n",
        "  \n",
        "def sampleDistrib(modelParamName,distrib,distribSpecs): \n",
        "    \n",
        "    if distrib == 'uniform':\n",
        "        \n",
        "        mmin = distribSpecs[0].value\n",
        "        mmax = distribSpecs[1].value\n",
        "        \n",
        "        intervalwidth = (mmax - mmin) / sampleCount      # width of each \n",
        "                                                         # sampling interval\n",
        "        samples = []\n",
        "        \n",
        "        for sample in range(sampleCount):\n",
        "            \n",
        "            lower = mmin + intervalwidth * (sample-1)    # lb of interval\n",
        "            upper = mmin + intervalwidth * (sample)      # ub of interval\n",
        "            \n",
        "            sampleVal = np.random.uniform(lower, upper)  # draw a random sample \n",
        "                                                         # within the interval\n",
        "            samples.append(sampleVal)\n",
        "\n",
        "    \n",
        "    \n",
        "    elif distrib == 'normal':\n",
        "        \n",
        "        mmean= distribSpecs[0].value\n",
        "        mvar = distribSpecs[1].value\n",
        "        \n",
        "        lower = mvar*np.sqrt(2)*special.erfinv(-0.9999)+mmean # set lb of 1st\n",
        "                                                              # sample interval\n",
        "        samples = []\n",
        "        \n",
        "        for sample in range(sampleCount):\n",
        "          \n",
        "            n = sample + 1\n",
        "            \n",
        "            if n != sampleCount:\n",
        "                upper = (np.sqrt(2*mvar)*special.erfinv(2*n/sampleCount-1)\n",
        "                         + mmean)                        # ub of sample interval\n",
        "            else:\n",
        "                upper = np.sqrt(2*mvar)*special.erfinv(0.9999) + mmean\n",
        " \n",
        "            sampleVal = np.random.uniform(lower, upper)  # draw a random sample \n",
        "                                                         # within the interval\n",
        "    \n",
        "            samples.append(sampleVal)\n",
        "\n",
        "            lower = upper           # set current ub as the lb for next interval\n",
        "            \n",
        "\n",
        "    \n",
        "    elif distrib == 'triangle':\n",
        "        \n",
        "        mmin = distribSpecs[0].value\n",
        "        mmax = distribSpecs[1].value\n",
        "        mmode= distribSpecs[2].value\n",
        "    \n",
        "        samples = []\n",
        "        \n",
        "        for sample in range(sampleCount):\n",
        "          \n",
        "            n = sample + 1\n",
        "            \n",
        "            intervalarea = 1/sampleCount \n",
        "            \n",
        "            ylower = intervalarea*(n-1) # use cdf to read off area as y's &\n",
        "            yupper = intervalarea*(n)   # get corresponding x's for the pdf\n",
        "        \n",
        "        \n",
        "            # Check to see if y values = cdf(x <= mmode) \n",
        "            # for calculating correxponding x values:\n",
        "            \n",
        "            if ylower <= ((mmode - mmin)/(mmax - mmin)):     \n",
        "                lower = np.sqrt(ylower*(mmax - mmin)*(mmode - mmin)) + mmin \n",
        "\n",
        "            else:\n",
        "                lower = mmax-np.sqrt((1 - ylower)*(mmax - mmin)*(mmax - mmode))\n",
        "\n",
        "            \n",
        "            if yupper <= ((mmode - mmin)/(mmax - mmin)):    \n",
        "                upper = np.sqrt(yupper*(mmax - mmin)*(mmode - mmin)) + mmin; \n",
        "\n",
        "            else:\n",
        "                upper = mmax-np.sqrt((1 - yupper)*(mmax - mmin)*(mmax - mmode))\n",
        "\n",
        "                \n",
        "            sampleVal = np.random.uniform(lower, upper)  \n",
        "            \n",
        "            samples.append(sampleVal)\n",
        "            \n",
        "    \n",
        "    b = int(np.ceil(sampleCount/10))\n",
        "    plt.hist(samples, density = 1, bins = b) \n",
        "    \n",
        "    B=str(b)\n",
        "    \n",
        "    plt.title('Histogram of ' + modelParamName \n",
        "              + ' parameter samples for ' + B + ' bins')\n",
        "    \n",
        "    plt.ylabel('proportion of samples');\n",
        "    plt.xlabel(modelParamName + ' value')\n",
        "    \n",
        "    plt.show()\n",
        "    \n",
        "    return samples"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lB1cTlDffxwk",
        "colab_type": "text"
      },
      "source": [
        "#### Calls the function to ask for user input to name parameters and specify distributions. \n",
        "### Type these in text input boxes and dropdowns that will appear below after running the cell."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CdbdLiSadt7V",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "params = {}\n",
        "\n",
        "for i in range(parameterCount):\n",
        "  \n",
        "    s=str(i)\n",
        "    \n",
        "    params[i] = interactive(parNameDist,\n",
        "                            Name='Type parameter ' + s + ' name', \n",
        "                            Distribution=['uniform','normal','triangle'])\n",
        "    \n",
        "    display(params[i])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9K7C0uUHHndX",
        "colab_type": "text"
      },
      "source": [
        "### Input parameter distribution specifics in the interactive boxes that appear below after running this cell."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "id": "mDjrr9jbHndY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "distribSpecs={}\n",
        "\n",
        "for i in range(parameterCount):\n",
        "  \n",
        "    parName = params[i].result['Name']\n",
        "    \n",
        "    print('Enter distribution specifics for parameter ' + parName + ':')\n",
        "    \n",
        "    if params[i].result['Dist'] == 'normal':\n",
        "\n",
        "        distribSpecs[parName] = {}\n",
        "        \n",
        "        distribSpecs[parName][0] = widgets.FloatText(\n",
        "                value=2,\n",
        "                description='Mean:'\n",
        "              )\n",
        "        distribSpecs[parName][1] = widgets.FloatText(\n",
        "                value=1,\n",
        "                description='Variance:'\n",
        "              )\n",
        "\n",
        "        display(distribSpecs[parName][0], distribSpecs[parName][1])\n",
        "\n",
        "    elif params[i].result['Dist'] == 'uniform':\n",
        "\n",
        "        distribSpecs[parName] = {}\n",
        "\n",
        "        distribSpecs[parName][0] = widgets.FloatText(\n",
        "                value=0,\n",
        "                description='Minimum:'\n",
        "              )\n",
        "        distribSpecs[parName][1] = widgets.FloatText(\n",
        "                value=2,\n",
        "                description='Maximum:'\n",
        "              )\n",
        "\n",
        "        display(distribSpecs[parName][0], distribSpecs[parName][1])\n",
        "\n",
        "\n",
        "    elif params[i].result['Dist'] == 'triangle':\n",
        "      \n",
        "        distribSpecs[parName] = {}\n",
        "\n",
        "        distribSpecs[parName][0] = widgets.FloatText(\n",
        "                value=0,\n",
        "                description='Minimum:'\n",
        "              )\n",
        "        distribSpecs[parName][1] = widgets.FloatText(\n",
        "                value=2,\n",
        "                description='Maximum:'\n",
        "              )\n",
        "        distribSpecs[parName][2] = widgets.FloatText(\n",
        "                value=1,\n",
        "                description='Mode:'\n",
        "              )\n",
        "\n",
        "        display(distribSpecs[parName][0], distribSpecs[parName][1], distribSpecs[parName][2])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6_FBOVRXgizZ",
        "colab_type": "text"
      },
      "source": [
        "#### This passes the distributions to the code for generating parameter samples, and histogram plots of samples for each parameter will appear below. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X1Clr2PYd6Mg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "parameters = {}\n",
        "for j in range(parameterCount):\n",
        "  \n",
        "    parameters[params[j].result['Name']] = sampleDistrib(params[j].result['Name'],\n",
        "                                                         params[j].result['Dist'],\n",
        "                                                         distribSpecs[params[j].result['Name']])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PdQAUyAB_QYY",
        "colab_type": "text"
      },
      "source": [
        "#### Randomly permute each set of parameter samples in order to randomly pair the samples to more fully sample the parameter space for the Monte Carlo simulations."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HFdaJckHHnde",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "LHSparams=[]\n",
        "\n",
        "for p in parameters:\n",
        "    \n",
        "    temp = parameters[p]\n",
        "    random.shuffle(temp)\n",
        "    \n",
        "    LHSparams.append(temp)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ad1XNtD_lvN",
        "colab_type": "text"
      },
      "source": [
        "### Define your model function.\n",
        "#### Two examples have been provided below: (1) a linear function with two sampled parameters: slope and intercept, and (2) a Lotka-Volterra predator - prey model.\n",
        "\n",
        "Note that the order and number of the parameters needs to match the order  and number of parameters speficied above to ensure accuracy when the model is solved below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T67BWVNT_p7F",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def testlinear(x,sampledParams,unsampledParams):\n",
        "\n",
        "    m = sampledParams[0]\n",
        "    b = sampledParams[1]\n",
        "    \n",
        "    a = unsampledParams\n",
        "\n",
        "    y = m * x + b + a;\n",
        "\n",
        "    return y    "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XIcGIOHUTN43",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def myodes(y, t, sampledParams, unsampledParams):\n",
        "\n",
        "    q, r = y                           # unpack current values of y\n",
        "\n",
        "    alpha, beta = sampledParams # unpack sampled parameters\n",
        "\n",
        "    delta, lambdaa, gamma = unsampledParams   # unpack unsampled parameters\n",
        "\n",
        "    derivs = [alpha*q*r - lambdaa*q,   # list of dy/dt=f functions\n",
        "              beta*r - gamma*q*r - delta*r]\n",
        "\n",
        "    return derivs\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gFMg4Lzl_z3T",
        "colab_type": "text"
      },
      "source": [
        "#### Run Monte Carlo simulations for each parameter sample set.\n",
        "### *Be sure to specify a call to your model function and any necessary arguments below.* \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aAtqOL4P_2GP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# EDIT THE FOLLOWING VARIABLES, UNSAMPLED PARAMETERS, & ANY OTHER ARGS HERE,\n",
        "# AS WELL AS THE CALL TO YOUR OWN MODEL FUNCTION INSIDE THE FOR LOOP BELOW\n",
        "\n",
        "x = np.linspace(0, 10, num=101) \n",
        "\n",
        "unsampledParams = 2; \n",
        "\n",
        "Output = [] \n",
        "\n",
        "for j in range(sampleCount):   \n",
        "  \n",
        "    sampledParams=[i[j] for i in LHSparams] \n",
        "    \n",
        "    sol = testlinear(x,sampledParams,unsampledParams)\n",
        "    Output.append(sol)\n",
        "    \n",
        "# EDIT THE STRING TO NAME YOUR SIM OUTPUT (for fig labels, filenames):\n",
        "labelstring = 'y'\n",
        "    \n",
        "    \n",
        "    \n",
        "# # EXAMPLE CODE FOR A COUPLED ODE MODEL:\n",
        "\n",
        "# import scipy.integrate as spi\n",
        "\n",
        "# t = np.linspace(0,17,num=171) # time domain for myodes\n",
        "\n",
        "# # odesic = [q0, r0]\n",
        "# odesic = [500,1000]\n",
        "\n",
        "# lambdaa = np.log(2)/7\n",
        "# delta = 0.5\n",
        "# gamma = 1\n",
        "\n",
        "# unsampledParams = [lambdaa, delta, gamma]\n",
        "\n",
        "# Simdata={}\n",
        "\n",
        "# Output = [] \n",
        "\n",
        "# for i in range(sampleCount):\n",
        "  \n",
        "#     Simdata[i]={}\n",
        "    \n",
        "#     Simdata[i]['q']=[]\n",
        "#     Simdata[i]['r']=[]\n",
        "    \n",
        "# for j in range(sampleCount):\n",
        "\n",
        "#     sampledParams=[i[j] for i in LHSparams] \n",
        "  \n",
        "#     sol=spi.odeint(myodes, odesic, t, args=(sampledParams,unsampledParams)) \n",
        "\n",
        "#     Simdata[j]['q'] = sol[:,0] # solution to the equation for variable r\n",
        "#     Simdata[j]['r'] = sol[:,1] # solution to the equation for variable s\n",
        "    \n",
        "#     Ratio = np.divide(sol[:,0],sol[:,1]) # compute ratio to compare w/ param samples\n",
        "    \n",
        "#     Output.append(Ratio)  \n",
        "                                                             \n",
        "\n",
        "# labelstring = 'predator to prey ratio (q/r)'; # id for fig labels, filenames\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9HH8nfMzc1qi",
        "colab_type": "text"
      },
      "source": [
        "### Plot the range of simulation output generated by the all of the Monte Carlo simulations using errorbars."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UcWtaGt-QqBY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "yavg = np.mean(Output, axis=0)\n",
        "yerr = np.std(Output, axis=0)\n",
        "\n",
        "plt.errorbar(t,yavg,yerr)\n",
        "plt.xlabel('x')\n",
        "# plt.xlabel('time (days)')   # for myodes\n",
        "plt.ylabel(labelstring)\n",
        "plt.title('Error bar plot of ' + labelstring + ' from LHS simulations')\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3XF6evQB_6ZN",
        "colab_type": "text"
      },
      "source": [
        "### Compute partial rank correlation coefficients to compare simulation outputs with parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O5Tbw-Id6ghf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "SampleResult=[]\n",
        "\n",
        "x_idx = 11          # time or location index of sim results \n",
        "x_idx2= x_idx+1     #   to compare w/ param sample vals\n",
        "\n",
        "LHS=[*zip(*LHSparams)]\n",
        "LHSarray=np.array(LHS)\n",
        "Outputarray=np.array(Output)\n",
        "subOut=Outputarray[0:,x_idx:x_idx2]\n",
        "\n",
        "LHSout = np.hstack((LHSarray,subOut))\n",
        "SampleResult = LHSout.tolist()\n",
        "\n",
        "\n",
        "Ranks=[]\n",
        "              \n",
        "for s in range(sampleCount):\n",
        "\n",
        "    indices = list(range(len(SampleResult[s])))\n",
        "    indices.sort(key=lambda k: SampleResult[s][k])\n",
        "    r = [0] * len(indices)\n",
        "    for i, k in enumerate(indices):\n",
        "        r[k] = i\n",
        "\n",
        "    Ranks.append(r)\n",
        "\n",
        "  \n",
        "C=np.corrcoef(Ranks);\n",
        "\n",
        "if np.linalg.det(C) < 1e-16: # determine if singular\n",
        "    Cinv = np.linalg.pinv(C) # may need to use pseudo inverse\n",
        "else:\n",
        "    Cinv = np.linalg.inv(C) \n",
        "\n",
        "resultIdx = parameterCount+1\n",
        "prcc=np.zeros(resultIdx)\n",
        "\n",
        "for w in range(parameterCount): # compute PRCC btwn each param & sim result\n",
        "    prcc[w]=-Cinv[w,resultIdx]/np.sqrt(Cinv[w,w]*Cinv[resultIdx,resultIdx])\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "86N98EtCOflg",
        "colab_type": "text"
      },
      "source": [
        "### Plot the PRCCs for each parameter"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eSa2rx_jOepg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "xp=[i for i in range(parameterCount)]\n",
        "\n",
        "plt.bar(xp,prcc[0:parameterCount], align='center')\n",
        "\n",
        "bLabels=list(parameters.keys())\n",
        "plt.xticks(xp, bLabels)\n",
        "\n",
        "plt.ylabel('PRCC value');\n",
        "\n",
        "N=str(sampleCount)\n",
        "loc=str(x_idx)\n",
        "plt.title('Partial rank correlation of params with ' + labelstring \n",
        "          + ' results \\n from ' + N + ' LHS sims, at x = ' +loc);\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uZG94ERF6it_",
        "colab_type": "text"
      },
      "source": [
        "### Can also do PRCCs over time... "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bja6BJWBYhou",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "SampleResult=[]\n",
        "\n",
        "resultIdx = parameterCount+1\n",
        "prcc=np.zeros((resultIdx,len(x)))\n",
        "\n",
        "LHS=[*zip(*LHSparams)]\n",
        "LHSarray=np.array(LHS)\n",
        "Outputarray=np.array(Output)\n",
        "\n",
        "for xi in range(len(x)):  # loop through time or location of sim results \n",
        "    xi2  = xi+1           # to compare w/ parameter sample vals\n",
        "\n",
        "\n",
        "    subOut = Outputarray[0:,xi:xi2]\n",
        "\n",
        "    LHSout = np.hstack((LHSarray,subOut))\n",
        "    SampleResult = LHSout.tolist()\n",
        "\n",
        "\n",
        "    Ranks=[]\n",
        "              \n",
        "    for s in range(sampleCount):\n",
        "              \n",
        "      indices = list(range(len(SampleResult[s])))\n",
        "      indices.sort(key=lambda k: SampleResult[s][k])\n",
        "      r = [0] * len(indices)\n",
        "      for i, k in enumerate(indices):\n",
        "          r[k] = i\n",
        "      \n",
        "      Ranks.append(r)\n",
        "\n",
        "  \n",
        "    C=np.corrcoef(Ranks);\n",
        "\n",
        "    if np.linalg.det(C) < 1e-16: # determine if singular\n",
        "        Cinv = np.linalg.pinv(C)   # may need to use pseudo inverse\n",
        "    else:\n",
        "        Cinv = np.linalg.inv(C) \n",
        "\n",
        "    for w in range(parameterCount):  # compute PRCC btwn each param & sim result\n",
        "        prcc[w,xi]=-Cinv[w,resultIdx]/np.sqrt(Cinv[w,w]*Cinv[resultIdx,resultIdx]) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ay8DGC1Veyrn",
        "colab_type": "text"
      },
      "source": [
        "### Plot PRCC values as they vary over time or space.\n",
        "#### *Notice PRCC can change with respect to the independent variable (x-axis). This may be helpful for certain applications, as opposed to only looking at a \"snapshot.\"*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "01aiOrI7b2VN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for p in range(parameterCount):\n",
        "    plt.plot(x,prcc[p,])\n",
        "\n",
        "\n",
        "labels=list(parameters.keys())\n",
        "plt.legend(labels)\n",
        "\n",
        "plt.ylabel('PRCC value');\n",
        "plt.xlabel('x')\n",
        "\n",
        "N=str(sampleCount)\n",
        "plt.title('Partial rank correlation of params with ' + labelstring \n",
        "          + ' results \\n from ' + N + ' LHS sims');\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2z8fXWcYxalX",
        "colab_type": "text"
      },
      "source": [
        "##That's it, you're done! Happy modeling."
      ]
    }
  ]
}