The-Grammar-of-Diffusion-Simulations.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import Base: reset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Distributions, LightGraphs, ProgressMeter, RCall, DataFrames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MersenneTwister(UInt32[0x01332bfa], Base.dSFMT.DSFMT_state(Int32[-1772545288, 1073534108, 1077066014, 1072915095, -2146195133, 1072843413, 301764553, 1073404181, 750472136, 1073628106  …  -1491411563, 1073194977, 716119449, 1072893711, 1632331784, 758890923, 1433693833, -13012230, 382, 0]), [1.85292, 1.34669, 1.51821, 1.75166, 1.45124, 1.02233, 1.94478, 1.93202, 1.84752, 1.83554  …  1.3986, 1.70216, 1.46308, 1.27622, 1.57881, 1.51321, 1.79293, 1.26551, 1.60442, 1.31848], 382)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "srand(20130810)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The grammar is a set of simple building blocks that generalize any diffusion simulation. \n",
    "\n",
    " > `reset(N) -> seed(node_status, N, seed_fn, seed_size) -> \n",
    "            evolve(node_status, N, evolve_fn, ...) -> \n",
    "            sum(node_status)`  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data structure on which diffusion simulations are executed is a combination of the network connections between the nodes and their *immutable* properties. All of them are initialized together at the beginning of each realization and is not tampered with in the course of the simulation. This data structure might hold different parameters for different kinds of models, for e.g., it might hold thresholds or influence of neighbors or influence of advertising. We illustrate our design using a simple simulation involving thresholds.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "struct Network\n",
    "    G::LightGraphs.SimpleGraphs.SimpleGraph{Int}\n",
    "    threshold::Vector{Float64}\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Initialization\n",
    "\n",
    "The starting point of any diffusion simulation is the `reset` function that takes a network object and sets the status of all the nodes in the network to `false`. The output is a BitVector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "reset (generic function with 4 methods)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function reset(N::Network)\n",
    "    return falses(nv(N.G))\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Seeding\n",
    "\n",
    "The next step is to seed a subset of the network using a seeding function that dictates which specific nodes should be targeted. This mutates the state of the `node_status` vector in place. A purer version of this function would initialize a new vector that is the copy of `node_status`, update this vector and return a copy of the new vector. Let us define these two versions and check if there are any speed differences between the versions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "seed! (generic function with 1 method)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function seed!(node_status::BitVector, \n",
    "               N::Network,\n",
    "               seed_fn::Function,\n",
    "               seed_size::Int)\n",
    "    \n",
    "    seed = seed_fn(N, seed_size) # should return a vector of indices\n",
    "    node_status[seed] = true\n",
    "    \n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "seed (generic function with 1 method)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function seed(node_status::BitVector,\n",
    "              N::Network,\n",
    "              seed_fn::Function,\n",
    "              seed_size::Int)\n",
    "    \n",
    "    new_status = copy(node_status)\n",
    "    seed = seed_fn(N, seed_size) # should return a vector of indices\n",
    "    new_status[seed] = true\n",
    "    \n",
    "    return new_status\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We illustrate the working of these functions and measure the timing using two examples. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1 Random seeding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "seed_random (generic function with 1 method)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function seed_random(N::Network, seed_size::Int)\n",
    "    return sample(vertices(N.G), seed_size, replace = false)\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Network({10000, 10000} undirected simple Int64 graph, [0.084108, 0.0325621, 0.0378909, 0.0202239, 0.122579, 0.0759791, 0.132814, 0.107071, 0.00187806, 0.00291333  …  0.0861788, 0.0601747, 0.0314736, 0.0978747, 0.0534731, 0.0600227, 0.0795918, 0.117065, 0.0288046, 0.0631423])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "N = Network(watts_strogatz(10^4, 3, 0.5), rand(TruncatedNormal(0.05, 0.05, 0, Inf), 10^4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "node_status = reset(N);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.304894 seconds (113.94 k allocations: 6.082 MiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "true"
      ],
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@time seed!(node_status, N, seed_random, 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "1000"
      ],
      "text/plain": [
       "1000"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(node_status)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "node_status = reset(N);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.033944 seconds (5.62 k allocations: 387.856 KiB)\n"
     ]
    }
   ],
   "source": [
    "@time node_status = seed(node_status, N, seed_random, 1000);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "1000"
      ],
      "text/plain": [
       "1000"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(node_status)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 Seeding the nodes with highest pagerank centrality "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "seed_pagerank (generic function with 1 method)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function seed_pagerank(N::Network, seed_size::Int)\n",
    "    return sortperm(pagerank(N.G))[1:seed_size]\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.277631 seconds (111.82 k allocations: 6.251 MiB, 3.03% gc time)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "1000"
      ],
      "text/plain": [
       "1000"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_status = reset(N)\n",
    "@time seed!(node_status, N, seed_pagerank, 1000)\n",
    "sum(node_status)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.027593 seconds (1.26 k allocations: 379.803 KiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "1000"
      ],
      "text/plain": [
       "1000"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_status = reset(N)\n",
    "@time node_status = seed(node_status, N, seed_pagerank, 1000)\n",
    "sum(node_status)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows an interesting pattern when the bottleneck is shifted to the seeder algorithm. The more complex the seeding algorithm, lesser is the difference in speed between mutating and non-mutating updates."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By keeping the `seeder` function is independent of the `seed` function, we can quickly compare the relative benefits of the diffusion process across seeding methods."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Evolution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different simulation design will employ different kinds of rules for node behavior. These rules are encoded into the `evolve_fn`. The `evolve_fn` function might be passed additional variables from its calller to support the evolution logic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "evolve! (generic function with 1 method)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function evolve!(node_status::BitVector, \n",
    "                 N::Network,\n",
    "                 evolve_fn::Function)\n",
    "    \n",
    "   for node in shuffle(vertices(N.G)) # by default nodes are updated in random order\n",
    "        if node_status[node] == false\n",
    "            evolve_fn(node_status, N, node)\n",
    "        end\n",
    "    end\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This higher level abstraction allows us to specify the evolution separate from the mechanics of evolving the diffusion process. The type and number of inputs to the evolver function can change depending on the complexity of the evolution rule."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We illustrate how the evolution works using a few example functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "evolve_threshold! (generic function with 1 method)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function evolve_threshold!(node_status::BitVector, N::Network, node::Int)\n",
    "    \n",
    "    n_engaged_nbrs = sum([node_status[nbr] for nbr in neighbors(N.G, node)])\n",
    "    \n",
    "    if n_engaged_nbrs/nv(N.G) > N.threshold[node] \n",
    "        node_status[node] = true\n",
    "    end\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.003651 seconds (27.00 k allocations: 1.312 MiB)\n"
     ]
    }
   ],
   "source": [
    "@time evolve!(node_status, N, evolve_threshold!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "1002"
      ],
      "text/plain": [
       "1002"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(node_status)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `evolve` function is typically called multiple times in a typical simulation design."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Executing the simulation\n",
    "\n",
    "The vocabulary of functions defined above is called several times from the function `simulate` that is incharge of putting together the parameter space required to execute the simulations. The function should collect the results into a relational database.\n",
    "\n",
    "As an example illustrating the entire process, we illustrate the simulation of the following problem:\n",
    "\n",
    "*Given a network, seeding individuals with high pagerank provides more bang for the buck compared to random seeding*\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us consider Watts-Strogatz networks with $10,000$ nodes and with each individual having an average degree of $1 - 10$. The threshold for each node can vary from $0.05 - 0.3$ (but we assume that everyone has the same threshold). We consider ten values of threshold in between these values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "simulate (generic function with 1 method)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function simulate(n::Int, p::Float64; T = 30, n_realizations = 20)\n",
    "    parameter_space = [(k, phi) for k in 1:10, \n",
    "                                    phi in linspace(0.05, 0.3, 10)]\n",
    "    \n",
    "    output = DataFrame(r = Int[], k = Int[], phi = Float64[], engaged_random = Int[], engaged_pagerank = Int[])\n",
    "    \n",
    "    @showprogress 1 \"Simulating...\" for (k, phi) in parameter_space\n",
    "          for r in 1:n_realizations\n",
    "            N = Network(watts_strogatz(n, k, p), fill(phi, n))\n",
    "            \n",
    "            # Begin the diffusion process starting with a random seed\n",
    "            \n",
    "            node_status = reset(N)\n",
    "            \n",
    "            seed!(node_status, N, seed_random, 1) # start with a single random seed\n",
    "            \n",
    "            for t = 1:T\n",
    "                evolve!(node_status, N, evolve_threshold!)\n",
    "            end\n",
    "            \n",
    "            n_engaged_rand = sum(node_status)\n",
    "            \n",
    "            # Begin the diffusion process starting with purposive seeding\n",
    "            \n",
    "            node_status = reset(N)\n",
    "            \n",
    "            seed!(node_status, N, seed_pagerank, 1)\n",
    "            \n",
    "            for t = 1:T\n",
    "                evolve!(node_status, N, evolve_threshold!)\n",
    "            end\n",
    "            \n",
    "            n_engaged_pagerank = sum(node_status)\n",
    "            \n",
    "            push!(output, [r, k, phi, n_engaged_rand, n_engaged_pagerank])\n",
    "            \n",
    "        end\n",
    "    end\n",
    "    \n",
    "    return output\n",
    "    \n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32mSimulating...100%|██████████████████████████████████████| Time: 0:04:45\u001b[39m\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>r</th><th>k</th><th>phi</th><th>engaged_random</th><th>engaged_pagerank</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>2</th><td>2</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>3</th><td>3</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>4</th><td>4</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>5</th><td>5</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>6</th><td>6</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>7</th><td>7</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>8</th><td>8</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>9</th><td>9</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>10</th><td>10</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>11</th><td>11</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>12</th><td>12</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>13</th><td>13</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>14</th><td>14</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>15</th><td>15</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>16</th><td>16</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>17</th><td>17</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>18</th><td>18</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>19</th><td>19</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>20</th><td>20</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>21</th><td>1</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>22</th><td>2</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>23</th><td>3</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>24</th><td>4</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>25</th><td>5</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>26</th><td>6</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>27</th><td>7</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>28</th><td>8</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>29</th><td>9</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>30</th><td>10</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>&vellip;</th><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td></tr></tbody></table>"
      ],
      "text/plain": [
       "2000×5 DataFrames.DataFrame\n",
       "│ Row  │ r  │ k  │ phi  │ engaged_random │ engaged_pagerank │\n",
       "├──────┼────┼────┼──────┼────────────────┼──────────────────┤\n",
       "│ 1    │ 1  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 2    │ 2  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 3    │ 3  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 4    │ 4  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 5    │ 5  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 6    │ 6  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 7    │ 7  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 8    │ 8  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 9    │ 9  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 10   │ 10 │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 11   │ 11 │ 1  │ 0.05 │ 1              │ 1                │\n",
       "⋮\n",
       "│ 1989 │ 9  │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1990 │ 10 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1991 │ 11 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1992 │ 12 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1993 │ 13 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1994 │ 14 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1995 │ 15 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1996 │ 16 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1997 │ 17 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1998 │ 18 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1999 │ 19 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 2000 │ 20 │ 10 │ 0.3  │ 1              │ 1                │"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = simulate(10^4, 0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>r</th><th>k</th><th>phi</th><th>engaged_random</th><th>engaged_pagerank</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>2</th><td>2</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>3</th><td>3</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>4</th><td>4</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>5</th><td>5</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>6</th><td>6</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>7</th><td>7</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>8</th><td>8</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>9</th><td>9</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>10</th><td>10</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>11</th><td>11</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>12</th><td>12</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>13</th><td>13</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>14</th><td>14</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>15</th><td>15</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>16</th><td>16</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>17</th><td>17</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>18</th><td>18</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>19</th><td>19</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>20</th><td>20</td><td>1</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>21</th><td>1</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>22</th><td>2</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>23</th><td>3</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>24</th><td>4</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>25</th><td>5</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>26</th><td>6</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>27</th><td>7</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>28</th><td>8</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>29</th><td>9</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>30</th><td>10</td><td>2</td><td>0.05</td><td>1</td><td>1</td></tr><tr><th>&vellip;</th><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td></tr></tbody></table>"
      ],
      "text/plain": [
       "2000×5 DataFrames.DataFrame\n",
       "│ Row  │ r  │ k  │ phi  │ engaged_random │ engaged_pagerank │\n",
       "├──────┼────┼────┼──────┼────────────────┼──────────────────┤\n",
       "│ 1    │ 1  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 2    │ 2  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 3    │ 3  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 4    │ 4  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 5    │ 5  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 6    │ 6  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 7    │ 7  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 8    │ 8  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 9    │ 9  │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 10   │ 10 │ 1  │ 0.05 │ 1              │ 1                │\n",
       "│ 11   │ 11 │ 1  │ 0.05 │ 1              │ 1                │\n",
       "⋮\n",
       "│ 1989 │ 9  │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1990 │ 10 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1991 │ 11 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1992 │ 12 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1993 │ 13 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1994 │ 14 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1995 │ 15 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1996 │ 16 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1997 │ 17 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1998 │ 18 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 1999 │ 19 │ 10 │ 0.3  │ 1              │ 1                │\n",
       "│ 2000 │ 20 │ 10 │ 0.3  │ 1              │ 1                │"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@rput results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1m\u001b[33mWARNING: \u001b[39m\u001b[22m\u001b[33mRCall.jl: -- Attaching packages --------------------------------------- tidyverse 1.2.1 --\n",
      "v ggplot2 2.2.1     v purrr   0.2.4\n",
      "v tibble  1.4.2     v dplyr   0.7.4\n",
      "v tidyr   0.8.0     v stringr 1.2.0\n",
      "v readr   1.1.1     v forcats 0.2.0\n",
      "-- Conflicts ------------------------------------------ tidyverse_conflicts() --\n",
      "x dplyr::filter() masks stats::filter()\n",
      "x dplyr::lag()    masks stats::lag()\u001b[39m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RCall.RObject{RCall.StrSxp}\n",
       " [1] \"forcats\"   \"stringr\"   \"dplyr\"     \"purrr\"     \"readr\"     \"tidyr\"    \n",
       " [7] \"tibble\"    \"ggplot2\"   \"tidyverse\" \"stats\"     \"graphics\"  \"grDevices\"\n",
       "[13] \"utils\"     \"datasets\"  \"methods\"   \"base\"     \n"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R\"\"\"\n",
    "library(tidyverse)\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RCall.RObject{RCall.VecSxp}\n",
       "# A tibble: 100 x 4\n",
       "# Groups:   k [10]\n",
       "       k    phi engaged_random engaged_pagerank\n",
       "   <int>  <dbl>          <dbl>            <dbl>\n",
       " 1     1 0.0500           1.00             1.00\n",
       " 2     1 0.0778           1.00             1.00\n",
       " 3     1 0.106            1.00             1.00\n",
       " 4     1 0.133            1.00             1.00\n",
       " 5     1 0.161            1.00             1.00\n",
       " 6     1 0.189            1.00             1.00\n",
       " 7     1 0.217            1.00             1.00\n",
       " 8     1 0.244            1.00             1.00\n",
       " 9     1 0.272            1.00             1.00\n",
       "10     1 0.300            1.00             1.00\n",
       "# ... with 90 more rows\n"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R\"\"\"\n",
    "summary_results <- results %>% group_by(k, phi) %>% \n",
    "                              summarize_all(funs(mean)) %>% \n",
    "                              select(-r)\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RCall.RObject{RCall.StrSxp}\n",
       "       k             phi         engaged_random engaged_pagerank\n",
       " Min.   : 1.0   Min.   :0.0500   Min.   :1      Min.   :1       \n",
       " 1st Qu.: 3.0   1st Qu.:0.1056   1st Qu.:1      1st Qu.:1       \n",
       " Median : 5.5   Median :0.1750   Median :1      Median :1       \n",
       " Mean   : 5.5   Mean   :0.1750   Mean   :1      Mean   :1       \n",
       " 3rd Qu.: 8.0   3rd Qu.:0.2444   3rd Qu.:1      3rd Qu.:1       \n",
       " Max.   :10.0   Max.   :0.3000   Max.   :1      Max.   :1       \n"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R\"\"\"\n",
    "summary(summary_results)\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 0.6.2",
   "language": "julia",
   "name": "julia-0.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}