adaptive_distance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adaptive distance\n",
    "\n",
    "[ABC](https://elfi.readthedocs.io/en/latest/usage/tutorial.html#approximate-bayesian-computation) provides means to sample an approximate posterior distribution over unknown parameters based on comparison between observed and simulated data. \n",
    "This comparison is often based on distance between features that summarise the data and are informative about the parameter values.\n",
    "\n",
    "Here we assume that the summaries calculated based on observed and simulated data are compared based on weighted distance with weight $w_i=1/\\sigma_i$ calculated based on their standard deviation $\\sigma_i$.\n",
    "This ensures that the selected summaries to have an equal contribution in the distance between observed and simulated data.\n",
    "\n",
    "This notebook studies [adaptive distance](https://projecteuclid.org/euclid.ba/1460641065) [SMC-ABC](https://elfi.readthedocs.io/en/latest/usage/tutorial.html#sequential-monte-carlo-abc) where $\\sigma_i$ and $w_i$ are recalculated between SMC iterations as proposed in [[1](#Reference)]. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import scipy.stats as ss\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import elfi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1:\n",
    "\n",
    "Assume we have an unknown parameter with prior distribution $\\theta\\sim U(0,50)$ and two simulator outputs $S_1\\sim N(\\theta, 1)$ and $S_2\\sim N(\\theta, 100)$ whose observed values are 20."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def simulator(mu, batch_size=1, random_state=None):\n",
    "    \n",
    "    batches_mu = np.asarray(mu).reshape((-1,1))\n",
    "    \n",
    "    obs_1 = ss.norm.rvs(loc=batches_mu, scale=1, random_state=random_state).reshape((-1,1))\n",
    "    obs_2 = ss.norm.rvs(loc=batches_mu, scale=100, random_state=random_state).reshape((-1,1))\n",
    "    \n",
    "    return np.hstack((obs_1, obs_2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "observed_data = np.array([20,20])[None,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here the simulator outputs are both informative about the unknown model parameter, but $S_2$ has more observation noise than $S_1$. We do not calculate separate summaries in this example, but compare observed and simulated data based on these two variables.\n",
    "\n",
    "Euclidean distance between observed and simulated outputs or summaries can be used to find parameter values that could produce the observed data. Here we describe dependencies between the unknown parameter value and observed distances as an ELFI model `m` and sample the approximate posterior distribution with the [rejection sampler](https://elfi.readthedocs.io/en/latest/usage/tutorial.html#inference-with-rejection-sampling)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = elfi.new_model()\n",
    "theta = elfi.Prior(ss.uniform, 0, 50, model=m)\n",
    "sim = elfi.Simulator(simulator, theta, observed=observed_data)\n",
    "d = elfi.Distance('euclidean', sim)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "rej = elfi.Rejection(d, batch_size=10000, seed=123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us sample 100 parameters with `quantile=0.01`. This means that we sample 10000 candidate parameters from the prior distribution and take the 100 parameters that produce simulated data closest to the observed data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Progress [==================================================] 100.0% Complete\n"
     ]
    }
   ],
   "source": [
    "sample = rej.sample(100, quantile=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Method: Rejection\n",
       "Number of samples: 100\n",
       "Number of simulations: 10000\n",
       "Threshold: 6.66\n",
       "Sample means: theta: 19.6"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEJCAYAAACNNHw2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAN0UlEQVR4nO3dcYykdX3H8fenQIMVUkDWyxW5rm1JGzT1SDdEg3+AVkvFFkyMLWktRpMjURJMbM3Vf7RtTDCpaJu0NmchkEatpEqhQlovlJZqG+yd0nKIBGvOyPXgjqoBG6MFv/1jnrXbY+9mb2fm9r6z71eymXmeeZ6d3y/Mvu/h2XlmU1VIkvr5kY0egCRpfQy4JDVlwCWpKQMuSU0ZcElqyoBLUlNjA57k/CT3JvlykoeSXD+sf1+SA0keGL5eN/vhSpKWZdz7wJNsBbZW1ReTnAnsBa4C3gR8p6r+cPbDlCQd6dRxG1TVQeDgcP/pJA8D563nyc4999xaXFxcz66StGnt3bv3yapaOHL92ICvlGQRuAi4H7gEuC7JbwF7gHdV1beOtf/i4iJ79uw5nqeUpE0vyddXW7/mX2ImOQP4FPDOqnoK+Ajw08B2RkfoHzzKfjuS7Emy5/Dhw8c9cEnS6tYU8CSnMYr3x6rq0wBV9URVPVtVPwA+Cly82r5VtauqlqpqaWHhOf8HIElap7W8CyXATcDDVXXjivVbV2z2BmDf9IcnSTqatZwDvwR4M/BgkgeGde8Brk6yHShgP3DtTEYoSVrVWt6F8jkgqzx09/SHI0laK6/ElKSmDLgkNWXAJakpAy5JTR3XlZjSWizuvGvd++6/4YopjkSabx6BS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDU1NuBJzk9yb5IvJ3koyfXD+nOS7E7y6HB79uyHK0latpYj8GeAd1XVhcDLgXckuRDYCdxTVRcA9wzLkqQTZGzAq+pgVX1xuP808DBwHnAlcOuw2a3AVbMapCTpuY7rHHiSReAi4H5gS1UdHB56HNgy1ZFJko7p1LVumOQM4FPAO6vqqSQ/fKyqKkkdZb8dwA6Abdu2TTZazb3FnXete9/9N1wxxZFIJ781HYEnOY1RvD9WVZ8eVj+RZOvw+Fbg0Gr7VtWuqlqqqqWFhYVpjFmSxNrehRLgJuDhqrpxxUN3AtcM968B7pj+8CRJR7OWUyiXAG8GHkzywLDuPcANwG1J3gZ8HXjTbIYoSVrN2IBX1eeAHOXhV093OJKktfJKTElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNXXqRg9AJ5/FnXdt9BAkrYFH4JLUlAGXpKYMuCQ1NTbgSW5OcijJvhXr3pfkQJIHhq/XzXaYkqQjreUI/Bbg8lXWf6iqtg9fd093WJKkccYGvKruA755AsYiSToOk5wDvy7Jvw+nWM6e2ogkSWuy3veBfwT4A6CG2w8Cb11twyQ7gB0A27ZtW+fTSeNN8v71/TdcMcWRSCfGuo7Aq+qJqnq2qn4AfBS4+Bjb7qqqpapaWlhYWO84JUlHWFfAk2xdsfgGYN/RtpUkzcbYUyhJPgFcCpyb5DHgvcClSbYzOoWyH7h2hmOUJK1ibMCr6upVVt80g7FIko6DV2JKUlMGXJKaMuCS1JQBl6Sm/IMOc8o/yiDNP4/AJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLU1NiAJ7k5yaEk+1asOyfJ7iSPDrdnz3aYkqQjreUI/Bbg8iPW7QTuqaoLgHuGZUnSCTQ24FV1H/DNI1ZfCdw63L8VuGrK45IkjbHec+BbqurgcP9xYMuUxiNJWqOJf4lZVQXU0R5PsiPJniR7Dh8+POnTSZIG6w34E0m2Agy3h462YVXtqqqlqlpaWFhY59NJko603oDfCVwz3L8GuGM6w5EkrdVa3kb4CeBfgJ9N8liStwE3AK9J8ijwi8OyJOkEOnXcBlV19VEeevWUxyJJOg5eiSlJTRlwSWrKgEtSU2PPgUubweLOuybaf/8NV0xpJNLaeQQuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmTp1k5yT7gaeBZ4FnqmppGoOSJI03UcAHl1XVk1P4PpKk4+ApFElqatKAF/DZJHuT7JjGgCRJazPpKZRXVtWBJC8Edif5SlXdt3KDIew7ALZt2zbh00mSlk10BF5VB4bbQ8DtwMWrbLOrqpaqamlhYWGSp5MkrbDugCd5fpIzl+8DrwX2TWtgkqRjm+QUyhbg9iTL3+fjVfW3UxmVJGmsdQe8qr4GvGyKY5EkHQffRihJTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1NS6/yq9Zm9x510bPQRJJzGPwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUxMFPMnlSR5J8tUkO6c1KEnSeOsOeJJTgD8Bfhm4ELg6yYXTGpgk6dgmOQK/GPhqVX2tqr4P/CVw5XSGJUkaZ5KAnwd8Y8XyY8M6SdIJMPM/6JBkB7BjWPxOkkfW+a3OBZ6czqjacM5N5AMT7d5yzhNyzsfnJ1dbOUnADwDnr1h+0bDu/6mqXcCuCZ4HgCR7qmpp0u/TiXPeHJzz5jCLOU9yCuVfgQuSvDjJjwK/Dtw5nWFJksZZ9xF4VT2T5Drg74BTgJur6qGpjUySdEwTnQOvqruBu6c0lnEmPg3TkHPeHJzz5jD1Oaeqpv09JUkngJfSS1JTLQK+GS7ZT3JzkkNJ9q1Yd06S3UkeHW7P3sgxTlOS85Pcm+TLSR5Kcv2wfp7nfHqSLyT5t2HOvzesf3GS+4fX9yeHNwXMlSSnJPlSks8My3M95yT7kzyY5IEke4Z1U39tn/QB30SX7N8CXH7Eup3APVV1AXDPsDwvngHeVVUXAi8H3jH8d53nOX8PeFVVvQzYDlye5OXAB4APVdXPAN8C3raBY5yV64GHVyxvhjlfVlXbV7x1cOqv7ZM+4GySS/ar6j7gm0esvhK4dbh/K3DVCR3UDFXVwar64nD/aUY/3Ocx33OuqvrOsHja8FXAq4C/GtbP1ZwBkrwIuAL482E5zPmcj2Lqr+0OAd/Ml+xvqaqDw/3HgS0bOZhZSbIIXATcz5zPeTiV8ABwCNgN/Afw7ap6ZthkHl/fHwbeDfxgWH4B8z/nAj6bZO9wNTrM4LU980vpNR1VVUnm7i1DSc4APgW8s6qeGh2cjczjnKvqWWB7krOA24Gf2+AhzVSS1wOHqmpvkks3ejwn0Cur6kCSFwK7k3xl5YPTem13OAJf0yX7c+qJJFsBhttDGzyeqUpyGqN4f6yqPj2snus5L6uqbwP3Aq8AzkqyfDA1b6/vS4BfTbKf0enPVwF/xHzPmao6MNweYvQP9cXM4LXdIeCb+ZL9O4FrhvvXAHds4FimajgPehPwcFXduOKheZ7zwnDkTZLnAa9hdO7/XuCNw2ZzNeeq+t2qelFVLTL62f37qvoN5njOSZ6f5Mzl+8BrgX3M4LXd4kKeJK9jdB5t+ZL992/wkKYuySeASxl9YtkTwHuBvwZuA7YBXwfeVFVH/qKzpSSvBP4JeJD/Ozf6Hkbnwed1zj/P6JdXpzA6eLqtqn4/yU8xOjo9B/gS8JtV9b2NG+lsDKdQfruqXj/Pcx7mdvuweCrw8ap6f5IXMOXXdouAS5Keq8MpFEnSKgy4JDVlwCWpKQMuSU0ZcElqyoBr7iQ5K8nbh/uXLn8C3nHs/5YkPzGb0UnTY8A1j84C3j7B/m8BDLhOer4PXHMnyfInVj4C/A/w38CTwEuBvYwuGqkkvwDcCJwxPP4WRpd+38Lo0u7vMrrU/XeAXwGeB/wzcG35g6OTgAHX3Bk+3fAzVfXS4eq/O4CXAP8JfJ5RkO8H/hG4sqoOJ/k14Jeq6q1J/oHRFYM//CD+5SvmkvwFoyso/+bEzkp6Lj+NUJvBF6rqMYDho1wXgW8zOiLfPXwC4inAwaPsf1mSdwM/xujS74cAA64NZ8C1Gaz8jI1nGb3uAzxUVa841o5JTgf+FFiqqm8keR9w+qwGKh0Pf4mpefQ0cOaYbR4BFpK8AkYfbZvkJavsvxzrJ4fPLn8j0knCI3DNnar6rySfH/5A9HcZfbrjkdt8P8kbgT9O8uOMfhY+zOj0yC3AnyVZ/iXmRxl9HOjjjD7eWDop+EtMSWrKUyiS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpr6X49PfnqAUOsfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(sample.samples_array,range=(0,50),bins=20)\n",
    "plt.xlabel('theta');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The approximate posterior sample is concentrated around $\\theta=20$ as expected in this example. However the sample distribution is much wider than we would observe in case the sample was selected based on $S_1$ alone.\n",
    "\n",
    "Now let us test adaptive distance in the same example. \n",
    "\n",
    "First we switch the distance node `d` to an adaptive distance node and initialise adaptive distance SMC-ABC. Initialisation is identical to the rejection sampler, and here we use the same batch size and seed as earlier, so that the methods are presented with the exact same candidate parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "d.become(elfi.AdaptiveDistance(sim))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "ada_smc = elfi.AdaptiveDistanceSMC(d, batch_size=10000, seed=123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since this is an iterative method, we must decide both sample size (`n_samples`) and how many populations are sampled (`rounds`). In addition we can decide the $\\alpha$ quantile (`quantile`) used in estimation. \n",
    "\n",
    "Each population with `n_samples` parameter values is sampled as follows: 1. `n_samples/quantile` parameters are sampled from the current proposal distribution with acceptance threshold determined based on the previous population and 2. the distance measure is updated based on the observed sample and `n_samples` with the smallest distance are selected as the new population. The first population is sampled from the prior distribution and all samples are accepted in step 1.\n",
    "\n",
    "Here we sample one population with `quantile=0.01`. This means that the total simulation count will be the same as with the rejection sampler, but now the distance function is updated based on the 10000 simulated observations, and the 100 parameters included in the posterior sample are selected based on the new distance measure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ABC-SMC Round 1 / 1\n",
      "Progress [==================================================] 100.0% Complete\n"
     ]
    }
   ],
   "source": [
    "sample_ada = ada_smc.sample(100, 1, quantile=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Method: AdaptiveDistanceSMC\n",
       "Number of samples: 100\n",
       "Number of simulations: 10000\n",
       "Threshold: 0.462\n",
       "Sample means: theta: 19.8"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_ada"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEGCAYAAAB8Ys7jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAOJElEQVR4nO3df6ydBX3H8fdnLQanbhW5NozKLotkBs3E7IZB9A/F6Zg44Q/CNG6pGUmX6BLM3Fz1n81lJvCPP5Zs2ToxNIs/ICorw2SzqTg3t6Ct4ASRgKREukKLQsTF6Irf/XGejrvbW+5p7zn39nv6fiXNOc9zzun5PuHcN0+ee56nqSokSf38zHoPIEk6OQZckpoy4JLUlAGXpKYMuCQ1tXEt3+zss8+u+fn5tXxLSWpv3759j1fV3NL1axrw+fl59u7du5ZvKUntJXl4ufUeQpGkpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlNjfY0wyX7gKeBp4EhVLSQ5C7gZmAf2A9dU1RPTGVOStNSJ7IG/rqouqqqFYXk7sKeqLgD2DMuSpDWymkMoVwI7h/s7gatWP44kaVzjnolZwBeSFPC3VbUD2FxVB4fHHwU2L/fCJNuAbQDnnXfeKsdVB/PbP3/Sr91//RUTnESabeMG/DVVdSDJi4HdSb69+MGqqiHuxxhivwNgYWHBf/5HkiZkrEMoVXVguD0E3ApcDDyW5ByA4fbQtIaUJB1rxYAneV6SFxy9D7wRuAe4Ddg6PG0rsGtaQ0qSjjXOIZTNwK1Jjj7/k1X1T0m+BtyS5FrgYeCa6Y0pSVpqxYBX1UPAK5dZ/z3g9dMYSpK0Ms/ElKSmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJampjes9gLTY/PbPn/Rr919/xQQnkU597oFLUlMGXJKaMuCS1JQBl6Smxg54kg1J7kpy+7B8fpI7kzyY5OYkz5nemJKkpU5kD/w64L5FyzcAH66qlwJPANdOcjBJ0rMbK+BJtgBXAB8blgNcBnxmeMpO4KppDChJWt64e+AfAd4L/HRYfhHwZFUdGZYfAc5d7oVJtiXZm2Tv4cOHVzWsJOkZKwY8yZuBQ1W172TeoKp2VNVCVS3Mzc2dzF8hSVrGOGdivhp4S5I3AWcCPwd8FNiUZOOwF74FODC9MSVJS624B15V76uqLVU1D7wV+GJVvR24A7h6eNpWYNfUppQkHWM13wP/E+APkzzI6Jj4jZMZSZI0jhO6mFVVfQn40nD/IeDiyY8kSRqHZ2JKUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWpqxYAnOTPJV5N8I8m9ST4wrD8/yZ1JHkxyc5LnTH9cSdJR4+yB/xi4rKpeCVwEXJ7kEuAG4MNV9VLgCeDa6Y0pSVpqxYDXyA+HxTOGPwVcBnxmWL8TuGoqE0qSljXWMfAkG5LcDRwCdgPfAZ6sqiPDUx4Bzp3OiJKk5YwV8Kp6uqouArYAFwMvG/cNkmxLsjfJ3sOHD5/kmJKkpU7oWyhV9SRwB3ApsCnJxuGhLcCB47xmR1UtVNXC3NzcqoaVJD1jnG+hzCXZNNx/LvAG4D5GIb96eNpWYNe0hpQkHWvjyk/hHGBnkg2Mgn9LVd2e5FvAp5P8BXAXcOMU55QkLbFiwKvqP4FXLbP+IUbHwyVJ68AzMSWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJampFQOe5CVJ7kjyrST3JrluWH9Wkt1JHhhuXzj9cSVJR42zB34EeE9VXQhcArwryYXAdmBPVV0A7BmWJUlrZMWAV9XBqvr6cP8p4D7gXOBKYOfwtJ3AVdMaUpJ0rBM6Bp5kHngVcCewuaoODg89Cmw+zmu2JdmbZO/hw4dXMaokabGxA57k+cBngXdX1Q8WP1ZVBdRyr6uqHVW1UFULc3NzqxpWkvSMsQKe5AxG8f5EVX1uWP1YknOGx88BDk1nREnScsb5FkqAG4H7qupDix66Ddg63N8K7Jr8eJKk49k4xnNeDfwu8M0kdw/r3g9cD9yS5FrgYeCa6YwoSVrOigGvqn8DcpyHXz/ZcSRJ4/JMTElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTa0Y8CQfT3IoyT2L1p2VZHeSB4bbF053TEnSUuPsgd8EXL5k3XZgT1VdAOwZliVJa2jFgFfVl4HvL1l9JbBzuL8TuGrCc0mSVnCyx8A3V9XB4f6jwObjPTHJtiR7k+w9fPjwSb6dJGmpVf8Ss6oKqGd5fEdVLVTVwtzc3GrfTpI0ONmAP5bkHIDh9tDkRpIkjeNkA34bsHW4vxXYNZlxJEnjGudrhJ8C/gP45SSPJLkWuB54Q5IHgF8fliVJa2jjSk+oqrcd56HXT3gWSdIJ8ExMSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpqRWvB67Tz/z2z6/3CJLG4B64JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JTfA9fMWM331/dff8UEJ5HWhnvgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1NSqAp7k8iT3J3kwyfZJDSVJWtlJBzzJBuCvgN8ELgTeluTCSQ0mSXp2q9kDvxh4sKoeqqqfAJ8GrpzMWJKklazmYlbnAt9dtPwI8GtLn5RkG7BtWPxhkvtP8v3OBh4/ydd25Tavkdyw1u/4//jfefatdnt/cbmVU78aYVXtAHas9u9JsreqFiYwUhtu8+nBbZ5909re1RxCOQC8ZNHylmGdJGkNrCbgXwMuSHJ+kucAbwVum8xYkqSVnPQhlKo6kuQPgH8GNgAfr6p7JzbZsVZ9GKYht/n04DbPvqlsb6pqGn+vJGnKPBNTkpoy4JLUVIuAnw6n7Cf5eJJDSe5ZtO6sJLuTPDDcvnA9Z5ykJC9JckeSbyW5N8l1w/pZ3uYzk3w1yTeGbf7AsP78JHcOn++bhy8FzJQkG5LcleT2YXmmtznJ/iTfTHJ3kr3Duol/tk/5gJ9Gp+zfBFy+ZN12YE9VXQDsGZZnxRHgPVV1IXAJ8K7hv+ssb/OPgcuq6pXARcDlSS4BbgA+XFUvBZ4Arl3HGaflOuC+Rcunwza/rqouWvT974l/tk/5gHOanLJfVV8Gvr9k9ZXAzuH+TuCqNR1qiqrqYFV9fbj/FKMf7nOZ7W2uqvrhsHjG8KeAy4DPDOtnapsBkmwBrgA+NiyHGd/m45j4Z7tDwJc7Zf/cdZplrW2uqoPD/UeBzes5zLQkmQdeBdzJjG/zcCjhbuAQsBv4DvBkVR0ZnjKLn++PAO8Ffjosv4jZ3+YCvpBk33A5EZjCZ3vqp9JrMqqqkszcdz6TPB/4LPDuqvrBaOdsZBa3uaqeBi5Ksgm4FXjZOo80VUneDByqqn1JXrve86yh11TVgSQvBnYn+fbiByf12e6wB346n7L/WJJzAIbbQ+s8z0QlOYNRvD9RVZ8bVs/0Nh9VVU8CdwCXApuSHN2ZmrXP96uBtyTZz+jw52XAR5ntbaaqDgy3hxj9j/pipvDZ7hDw0/mU/duArcP9rcCudZxloobjoDcC91XVhxY9NMvbPDfseZPkucAbGB37vwO4enjaTG1zVb2vqrZU1Tyjn90vVtXbmeFtTvK8JC84eh94I3APU/hstzgTM8mbGB1HO3rK/gfXeaSJS/Ip4LWMLjv5GPCnwD8AtwDnAQ8D11TV0l90tpTkNcC/At/kmWOj72d0HHxWt/lXGP3yagOjnadbqurPk/wSo73Ts4C7gN+pqh+v36TTMRxC+aOqevMsb/OwbbcOixuBT1bVB5O8iAl/tlsEXJJ0rA6HUCRJyzDgktSUAZekpgy4JDVlwCWpKQOumZNkU5J3Dvdfe/QKeCfw+nck+YXpTCdNjgHXLNoEvHMVr38HYMB1yvN74Jo5SY5esfJ+4H+A/wYeB14B7GN00kgl+VXgQ8Dzh8ffwejU75sYndr9I0anuv8x8FvAc4F/B36//MHRKcCAa+YMVze8vapeMZz9twt4OfBfwFcYBflO4F+AK6vqcJLfBn6jqn4vyZcYnTH4fxfiP3rGXJK/Z3QG5T+u7VZJx/JqhDodfLWqHgEYLuU6DzzJaI9893AFxA3AweO8/nVJ3gv8LKNTv+8FDLjWnQHX6WDxNTaeZvS5D3BvVV36bC9Mcibw18BCVX03yZ8BZ05rUOlE+EtMzaKngBes8Jz7gbkkl8Lo0rZJXr7M64/G+vHh2uVXI50i3APXzKmq7yX5yvAPRP+I0dUdlz7nJ0muBv4yyc8z+ln4CKPDIzcBf5Pk6C8x/47R5UAfZXR5Y+mU4C8xJakpD6FIUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTf0vgX2fL8DG7bAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(sample_ada.samples_array,range=(0,50),bins=20)\n",
    "plt.xlabel('theta');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the posterior distribution over unknown parameter values is narrower than in the previous example. This is because the simulator outputs are now normalised based on their estimated standard deviation. \n",
    "\n",
    "We can see $w_1$ and $w_2$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0.06940134, 0.0097677 ])]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_ada.adaptive_distance_w"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2:\n",
    "\n",
    "This is the normal distribution example presented in [[1](#Reference)]. \n",
    "\n",
    "Here we have an unknown parameter with prior distribution $\\theta\\sim N(0,100)$ and two simulator outputs $S_1\\sim N(\\theta, 0.1)$ and $S_2\\sim N(1, 1)$ whose observed values are 0. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def simulator(mu, batch_size=1, random_state=None):\n",
    "    \n",
    "    batches_mu = np.asarray(mu).reshape((-1,1))\n",
    "    \n",
    "    obs_1 = ss.norm.rvs(loc=batches_mu, scale=0.1, random_state=random_state).reshape((-1,1))\n",
    "    obs_2 = ss.norm.rvs(loc=1, scale=1, size=batch_size, random_state=random_state).reshape((-1,1))\n",
    "    \n",
    "    return np.hstack((obs_1, obs_2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "observed_data = np.array([0,0])[None,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$S_1$ is now informative and $S_2$ uninformative about the unknown parameter value, and we note that between the two output variables, $S_1$ has larger variance under the prior predictive distribution. This means that normalisation estimated based on output data observed in the initial round or based on a separate sample would not work well in this example.\n",
    "\n",
    "Let us define a new model and initialise adaptive distance SMC-ABC."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = elfi.new_model()\n",
    "theta = elfi.Prior(ss.norm, 0, 100, model=m)\n",
    "sim = elfi.Simulator(simulator, theta, observed=observed_data)\n",
    "d = elfi.AdaptiveDistance(sim)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "ada_smc = elfi.AdaptiveDistanceSMC(d, batch_size=2000, seed=123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we sample 1000 parameter values in 5 rounds with the default `quantile=0.5` which is recommended in sequential estimation [[1](#Reference)]:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ABC-SMC Round 1 / 5\n",
      "Progress [==================================================] 100.0% Complete\n",
      "ABC-SMC Round 2 / 5\n",
      "Progress [==================================================] 100.0% Complete\n",
      "ABC-SMC Round 3 / 5\n",
      "Progress [==================================================] 100.0% Complete\n",
      "ABC-SMC Round 4 / 5\n",
      "Progress [==================================================] 100.0% Complete\n",
      "ABC-SMC Round 5 / 5\n",
      "Progress [==================================================] 100.0% Complete\n"
     ]
    }
   ],
   "source": [
    "sample_ada = ada_smc.sample(1000, 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Method: AdaptiveDistanceSMC\n",
       "Number of samples: 1000\n",
       "Number of simulations: 32000\n",
       "Threshold: 0.925\n",
       "Sample means: theta: -0.195"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_ada"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEGCAYAAACevtWaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAPcklEQVR4nO3df6xfdX3H8edrVMAfmwW56bClu01stgBx0d0wCMvCxIQqzLKFsTq3FSXpjGzT6YZF/sB/TDAu/lqmSydIXQhKUEenc7PrdGxu1BUlClS0QZB2hV5UplOjVt/743vQby633N7v+d572899PhJyz/l8zvl+35/c3hfnfu45n2+qCklSW35mqQuQJI2f4S5JDTLcJalBhrskNchwl6QGrVjqAgBOO+20mpycXOoyJOm4ctdddz1WVROz9R0T4T45OcmePXuWugxJOq4keehIfU7LSFKDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSg46JJ1SlY9nk1o+PfO6D1188xkqko+eVuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSg+YM9yQ3JjmU5J6htrcl+VKSLyT5aJKVQ33XJNmX5P4kFy1U4ZKkIzuaK/ebgA0z2nYCZ1fV84EvA9cAJDkT2ASc1Z3zniQnjK1aSdJRmfM+96q6I8nkjLZPDu3eCVzWbW8EPlhV3we+mmQfcA7wX2OpVsuW95pL8zOOOfdXAZ/otlcDDw/17e/aJEmLqFe4J7kWOAzcPMK5W5LsSbJnenq6TxmSpBlGDvckVwCXAK+oquqaDwBnDB22pmt7kqraVlVTVTU1MTHrh3dLkkY0Urgn2QBcDbysqr471LUD2JTkpCTrgPXAZ/uXKUmajzn/oJrkFuAC4LQk+4HrGNwdcxKwMwnAnVX16qq6N8mtwH0MpmuuqqofLVTxkqTZHc3dMi+fpfmGpzj+LcBb+hQlSerHJ1QlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDVozo/Zk7Q0Jrd+fORzH7z+4jFWouOR4a7m9QlJ6XjltIwkNchwl6QGGe6S1CDDXZIaNGe4J7kxyaEk9wy1nZpkZ5KvdF9P6dqT5N1J9iX5QpIXLmTxkqTZHc2V+03AhhltW4FdVbUe2NXtA7wEWN/9twV473jKlCTNx5zhXlV3AN+Y0bwR2N5tbwcuHWr/QA3cCaxMcvq4ipUkHZ1R73NfVVUHu+1HgFXd9mrg4aHj9ndtB5khyRYGV/esXbt2xDJ0PPF+c2nx9P6DalUVUCOct62qpqpqamJiom8ZkqQho4b7o09Mt3RfD3XtB4Azho5b07VJkhbRqOG+A9jcbW8Gbh9q/8Purplzgf8dmr6RJC2SOefck9wCXACclmQ/cB1wPXBrkiuBh4DLu8P/EXgpsA/4LvDKBahZkjSHOcO9ql5+hK4LZzm2gKv6FiVJ6scnVCWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGjLvkr6Si4zLGWilfuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBvcI9yZ8luTfJPUluSXJyknVJdifZl+RDSU4cV7GSpKMzcrgnWQ38KTBVVWcDJwCbgLcC76iq5wHfBK4cR6GSpKPX98M6VgBPT/JD4BnAQeBFwO91/duBNwPv7fk+Okb44RPS8WHkcK+qA0n+Evga8D3gk8BdwONVdbg7bD+werbzk2wBtgCsXbt21DI0AgNaal+faZlTgI3AOuC5wDOBDUd7flVtq6qpqpqamJgYtQxJ0iz6TMu8GPhqVU0DJPkIcD6wMsmK7up9DXCgf5mS5qPPb2cPXn/xGCvRUulzt8zXgHOTPCNJgAuB+4BPAZd1x2wGbu9XoiRpvkYO96raDdwGfA74Yvda24A3Aq9Psg94DnDDGOqUJM1Dr7tlquo64LoZzQ8A5/R5XUlSPz6hKkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqUK8P65DUnj6fvwp+Buuxwit3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIa1Cvck6xMcluSLyXZm+S8JKcm2ZnkK93XU8ZVrCTp6PS9cn8X8E9V9UvALwN7ga3ArqpaD+zq9iVJi2jkcE/ybODXgRsAquoHVfU4sBHY3h22Hbi0b5GSpPnpc+W+DpgG3p/k80nel+SZwKqqOtgd8wiwaraTk2xJsifJnunp6R5lSJJm6hPuK4AXAu+tqhcA32HGFExVFVCznVxV26pqqqqmJiYmepQhSZqpT7jvB/ZX1e5u/zYGYf9oktMBuq+H+pUoSZqvkcO9qh4BHk7yi13ThcB9wA5gc9e2Gbi9V4WSpHnru+TvnwA3JzkReAB4JYP/Ydya5ErgIeDynu8hSZqnXuFeVXcDU7N0XdjndSVJ/fiEqiQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUG9wz3JCUk+n+Rj3f66JLuT7EvyoSQn9i9TkjQf47hyfy2wd2j/rcA7qup5wDeBK8fwHpKkeegV7knWABcD7+v2A7wIuK07ZDtwaZ/3kCTNX98r93cCVwM/7vafAzxeVYe7/f3A6tlOTLIlyZ4ke6anp3uWIUkaNnK4J7kEOFRVd41yflVtq6qpqpqamJgYtQxJ0ixW9Dj3fOBlSV4KnAz8HPAuYGWSFd3V+xrgQP8yJUnzMfKVe1VdU1VrqmoS2AT8a1W9AvgUcFl32Gbg9t5VSpLmZSHuc38j8Pok+xjMwd+wAO8hSXoKfaZlfqKqPg18utt+ADhnHK8rSRqNT6hKUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIatGLUE5OcAXwAWAUUsK2q3pXkVOBDwCTwIHB5VX2zf6l6wuTWjy91CdIR9fn3+eD1F4+xkuWtz5X7YeANVXUmcC5wVZIzga3ArqpaD+zq9iVJi2jkcK+qg1X1uW7728BeYDWwEdjeHbYduLRvkZKk+RnLnHuSSeAFwG5gVVUd7LoeYTBtM9s5W5LsSbJnenp6HGVIkjq9wz3Js4APA6+rqm8N91VVMZiPf5Kq2lZVU1U1NTEx0bcMSdKQXuGe5GkMgv3mqvpI1/xoktO7/tOBQ/1KlCTN18jhniTADcDeqnr7UNcOYHO3vRm4ffTyJEmjGPlWSOB84A+ALya5u2t7E3A9cGuSK4GHgMv7lShJmq+Rw72q/gPIEbovHPV1JUn9+YSqJDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQQsW7kk2JLk/yb4kWxfqfSRJT7Yg4Z7kBOCvgZcAZwIvT3LmQryXJOnJFurK/RxgX1U9UFU/AD4IbFyg95IkzbBigV53NfDw0P5+4FeHD0iyBdjS7f5fkvsXqJaFdBrw2FIXscgcc/uWbLx561K8K3D8fo9/4UgdCxXuc6qqbcC2pXr/cUiyp6qmlrqOxeSY27fcxgttjnmhpmUOAGcM7a/p2iRJi2Chwv2/gfVJ1iU5EdgE7Fig95IkzbAg0zJVdTjJHwP/DJwA3FhV9y7Eey2x43paaUSOuX3LbbzQ4JhTVUtdgyRpzHxCVZIaZLhLUoMM93lK8rYkX0ryhSQfTbJyqO+abrmF+5NctJR1jlOS30lyb5IfJ5ma0dfkmGF5LKGR5MYkh5LcM9R2apKdSb7SfT1lKWsctyRnJPlUkvu6f9ev7dqbGrfhPn87gbOr6vnAl4FrALrlFTYBZwEbgPd0yzC04B7gt4E7hhtbHvMyWkLjJgbfu2FbgV1VtR7Y1e235DDwhqo6EzgXuKr73jY1bsN9nqrqk1V1uNu9k8E9/DBYXuGDVfX9qvoqsI/BMgzHvaraW1WzPUHc7JhZJktoVNUdwDdmNG8Etnfb24FLF7WoBVZVB6vqc932t4G9DJ6qb2rchns/rwI+0W3PtuTC6kWvaHG1POaWxzaXVVV1sNt+BFi1lMUspCSTwAuA3TQ27iVbfuBYluRfgJ+fpevaqrq9O+ZaBr/e3byYtS2Uoxmzlp+qqiRN3i+d5FnAh4HXVdW3kvykr4VxG+6zqKoXP1V/kiuAS4AL66cPChzXSy7MNeYjOK7HPIeWxzaXR5OcXlUHk5wOHFrqgsYtydMYBPvNVfWRrrmpcTstM09JNgBXAy+rqu8Ode0ANiU5Kck6YD3w2aWocRG1POblvITGDmBzt70ZaOo3twwu0W8A9lbV24e6mhq3T6jOU5J9wEnA17umO6vq1V3ftQzm4Q8z+FXvE7O/yvElyW8BfwVMAI8Dd1fVRV1fk2MGSPJS4J38dAmNtyxxSWOX5BbgAgZL3j4KXAf8PXArsBZ4CLi8qmb+0fW4leTXgH8Hvgj8uGt+E4N592bGbbhLUoOclpGkBhnuktQgw12SGmS4S1KDDHdJapDhrmUlycokr+m2L0jysXmef0WS5y5MddL4GO5ablYCr+lx/hWA4a5jnve5a1lJ8sTqjvcDPwS+AzwGnA3cBfx+t67IrwBvB57V9V8BnM9gidwDwPeA84C/AH4TeDrwn8AflT9UOgYY7lpWulUAP1ZVZye5gMEj5mcB/wN8hkFY7wb+DdhYVdNJfhe4qKpeleTTwJ9X1Z7u9U594inGJH8H3FpV/7C4o5KezIXDtNx9tqr2AyS5G5hksMTC2cDObqXAE4CDRzj/N5JcDTwDOBW4FzDcteQMdy133x/a/hGDn4kA91bVeU91YpKTgfcAU1X1cJI3AycvVKHSfPgHVS033wZ+do5j7gcmkpwHg+Vhk5w1y/lPBPlj3drgl427WGlUXrlrWamqryf5TPeB0N9jsBLizGN+kOQy4N1Jns3g5+SdDKZcbgL+JskTf1D9WwafMfsIg2WCpWOCf1CVpAY5LSNJDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoP+Hw5D6Q8VrSPIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(sample_ada.samples_array, range=(-25,25), bins=20)\n",
    "plt.xlabel(theta);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sample distribution is concentrated around $\\theta=0$ but wider than could be expected. However we can continue the iterative estimation process. Here we sample two more populations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ABC-SMC Round 6 / 7\n",
      "Progress [==================================================] 100.0% Complete\n",
      "ABC-SMC Round 7 / 7\n",
      "Progress [==================================================] 100.0% Complete\n"
     ]
    }
   ],
   "source": [
    "sample_ada = ada_smc.sample(1000, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Method: AdaptiveDistanceSMC\n",
       "Number of samples: 1000\n",
       "Number of simulations: 48000\n",
       "Threshold: 0.868\n",
       "Sample means: theta: 0.0183"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_ada"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEGCAYAAACevtWaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAQDklEQVR4nO3df6zddX3H8edrFNENIzDuulq6lbguC5BZ3A2D6B8o20DcVlyUlWRalKQuYqKJmyn6hy4ZCWYTjMtkqYFQDYrN1NEpbtaOjekCeGEMKZV5pxDalfYC/sDpcC3v/XG+lWO57bn3nnvupZ/7fCQ35/v9fD7f831/cttXT7/3+/3cVBWSpLb8zGIXIEmaf4a7JDXIcJekBhnuktQgw12SGrRssQsAOPXUU2v16tWLXYYkHVPuueeex6tqbLq+50W4r169momJicUuQ5KOKUkeOVKfl2UkqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBz4snVKVRWr3pC4t27oeved2inVtLm5/cJalBhrskNchwl6QGGe6S1CDDXZIaNDDck7wwyd1J/iPJziR/1rWfnuSuJJNJPp3kBV37Cd3+ZNe/erRTkCQdbiaf3J8GXlNVLwfWAhclORf4IHBdVf0K8B3gim78FcB3uvbrunGSpAU0MNyr5wfd7vHdVwGvAf62a98CXNJtr+v26fovSJJ5q1iSNNCMrrknOS7JfcB+YDvwX8B3q+pAN2Q3sLLbXgk8CtD1fw/4+Wnec2OSiSQTU1NTw81CkvRTZhTuVXWwqtYCpwHnAL827ImranNVjVfV+NjYtL/fVZI0R7O6W6aqvgvcDpwHnJTk0PIFpwF7uu09wCqArv8lwBPzUq0kaUZmcrfMWJKTuu0XAb8N7KIX8m/ohm0Abu22t3X7dP3/VFU1n0VLko5uJguHrQC2JDmO3j8GW6vq80keBG5J8ufAvwM3dONvAD6RZBJ4Elg/grolSUcxMNyr6n7g7Gnav0Xv+vvh7f8LvHFeqpMkzYlPqEpSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkho0MNyTrEpye5IHk+xM8s6u/QNJ9iS5r/u6uO+Yq5JMJnkoyYWjnIAk6bmWzWDMAeDdVXVvkhcD9yTZ3vVdV1V/2T84yRnAeuBM4KXAl5P8alUdnM/CJUlHNvCTe1Xtrap7u+2ngF3AyqMcsg64paqerqpvA5PAOfNRrCRpZmZ1zT3JauBs4K6u6R1J7k9yY5KTu7aVwKN9h+1mmn8MkmxMMpFkYmpqataFS5KObMbhnuRE4DPAu6rq+8D1wMuAtcBe4EOzOXFVba6q8aoaHxsbm82hkqQBZhTuSY6nF+w3V9VnAapqX1UdrKpngI/x7KWXPcCqvsNP69okSQtkJnfLBLgB2FVV1/a1r+gb9nrggW57G7A+yQlJTgfWAHfPX8mSpEFmcrfMK4E3AV9Pcl/X9l7gsiRrgQIeBt4GUFU7k2wFHqR3p82V3ikjSQtrYLhX1VeATNN121GOuRq4eoi6JElD8AlVSWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwaGe5JVSW5P8mCSnUne2bWfkmR7km92ryd37UnykSSTSe5P8opRT0KS9NNm8sn9APDuqjoDOBe4MskZwCZgR1WtAXZ0+wCvBdZ0XxuB6+e9aknSUQ0M96raW1X3dttPAbuAlcA6YEs3bAtwSbe9Dvh49dwJnJRkxbxXLkk6olldc0+yGjgbuAtYXlV7u67HgOXd9krg0b7Ddndth7/XxiQTSSampqZmWbYk6WhmHO5JTgQ+A7yrqr7f31dVBdRsTlxVm6tqvKrGx8bGZnOoJGmAGYV7kuPpBfvNVfXZrnnfocst3ev+rn0PsKrv8NO6NknSApnJ3TIBbgB2VdW1fV3bgA3d9gbg1r72N3d3zZwLfK/v8o0kaQEsm8GYVwJvAr6e5L6u7b3ANcDWJFcAjwCXdn23ARcDk8APgbfMa8WSpIEGhntVfQXIEbovmGZ8AVcOWZfUhNWbvjDnYx++5nXzWImWGp9QlaQGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ2ayX3u0qIb5pZCaSnyk7skNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1aGC4J7kxyf4kD/S1fSDJniT3dV8X9/VdlWQyyUNJLhxV4ZKkI5vJJ/ebgIumab+uqtZ2X7cBJDkDWA+c2R3z0STHzVexkqSZGRjuVXUH8OQM328dcEtVPV1V3wYmgXOGqE+SNAfDXHN/R5L7u8s2J3dtK4FH+8bs7tqeI8nGJBNJJqampoYoQ5J0uLmG+/XAy4C1wF7gQ7N9g6raXFXjVTU+NjY2xzIkSdOZU7hX1b6qOlhVzwAf49lLL3uAVX1DT+vaJEkLaE7hnmRF3+7rgUN30mwD1ic5IcnpwBrg7uFKlCTN1rJBA5J8CjgfODXJbuD9wPlJ1gIFPAy8DaCqdibZCjwIHACurKqDoyldknQkA8O9qi6bpvmGo4y/Grh6mKIkScPxCVVJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDBoZ7khuT7E/yQF/bKUm2J/lm93py154kH0kymeT+JK8YZfGSpOnN5JP7TcBFh7VtAnZU1RpgR7cP8FpgTfe1Ebh+fsqUJM3GwHCvqjuAJw9rXgds6ba3AJf0tX+8eu4ETkqyYr6KlSTNzFyvuS+vqr3d9mPA8m57JfBo37jdXdtzJNmYZCLJxNTU1BzLkCRNZ+gfqFZVATWH4zZX1XhVjY+NjQ1bhiSpz1zDfd+hyy3d6/6ufQ+wqm/caV2bJGkBzTXctwEbuu0NwK197W/u7po5F/he3+UbSdICWTZoQJJPAecDpybZDbwfuAbYmuQK4BHg0m74bcDFwCTwQ+AtI6hZkjTAwHCvqsuO0HXBNGMLuHLYoiRJw/EJVUlqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoOWDXNwkoeBp4CDwIGqGk9yCvBpYDXwMHBpVX1nuDIlSbMxH5/cX11Va6tqvNvfBOyoqjXAjm5fkrSARnFZZh2wpdveAlwygnNIko5i2HAv4EtJ7kmysWtbXlV7u+3HgOXTHZhkY5KJJBNTU1NDliFJ6jfUNXfgVVW1J8kvANuTfKO/s6oqSU13YFVtBjYDjI+PTztGkjQ3Q31yr6o93et+4HPAOcC+JCsAutf9wxYpSZqdOYd7kp9L8uJD28DvAA8A24AN3bANwK3DFilJmp1hLsssBz6X5ND7fLKq/iHJ14CtSa4AHgEuHb5MSdJszDncq+pbwMunaX8CuGCYoiRJw/EJVUlqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoNGFu5JLkryUJLJJJtGdR5J0nONJNyTHAf8NfBa4AzgsiRnjOJckqTnGtUn93OAyar6VlX9GLgFWDeic0mSDrNsRO+7Eni0b3838Jv9A5JsBDZ2uz9I8tCIahmlU4HHF7uIBeacF0g+uNBn/Am/x8eOXz5Sx6jCfaCq2gxsXqzzz4ckE1U1vth1LCTn3L6lNl9oc86juiyzB1jVt39a1yZJWgCjCvevAWuSnJ7kBcB6YNuIziVJOsxILstU1YEk7wD+ETgOuLGqdo7iXIvsmL6sNEfOuX1Lbb7Q4JxTVYtdgyRpnvmEqiQ1yHCXpAYZ7rOU5C+SfCPJ/Uk+l+Skvr6ruuUWHkpy4WLWOZ+SvDHJziTPJBk/rK/JOcPSWEIjyY1J9id5oK/tlCTbk3yzez15MWucb0lWJbk9yYPdn+t3du1Nzdtwn73twFlV9evAfwJXAXTLK6wHzgQuAj7aLcPQggeAPwDu6G9sec5LaAmNm+h97/ptAnZU1RpgR7ffkgPAu6vqDOBc4Mrue9vUvA33WaqqL1XVgW73Tnr38ENveYVbqurpqvo2MElvGYZjXlXtqqrpniBuds4skSU0quoO4MnDmtcBW7rtLcAlC1rUiFXV3qq6t9t+CthF76n6puZtuA/nrcAXu+3pllxYueAVLayW59zy3AZZXlV7u+3HgOWLWcwoJVkNnA3cRWPzXrTlB57PknwZ+MVput5XVbd2Y95H7793Ny9kbaMykzlr6amqStLk/dJJTgQ+A7yrqr6f5Cd9LczbcJ9GVf3W0fqTXA78LnBBPfugwDG95MKgOR/BMT3nAVqe2yD7kqyoqr1JVgD7F7ug+ZbkeHrBfnNVfbZrbmreXpaZpSQXAe8Bfr+qftjXtQ1Yn+SEJKcDa4C7F6PGBdTynJfyEhrbgA3d9gagqf+5pfcR/QZgV1Vd29fV1Lx9QnWWkkwCJwBPdE13VtUfd33vo3cd/gC9/+p9cfp3ObYkeT3wV8AY8F3gvqq6sOtrcs4ASS4GPsyzS2hcvcglzbsknwLOp7fk7T7g/cDfAVuBXwIeAS6tqsN/6HrMSvIq4F+BrwPPdM3vpXfdvZl5G+6S1CAvy0hSgwx3SWqQ4S5JDTLcJalBhrskNchw15KS5KQkb++2z0/y+Vkef3mSl46mOmn+GO5aak4C3j7E8ZcDhrue97zPXUtKkkOrOz4E/B/wP8DjwFnAPcAfdeuK/AZwLXBi13858Ep6S+TuAX4EnAf8KfB7wIuAfwPeVv6l0vOA4a4lpVsF8PNVdVaS8+k9Yn4m8N/AV+mF9V3AvwDrqmoqyR8CF1bVW5P8M/AnVTXRvd8ph55iTPIJYGtV/f3Czkp6LhcO01J3d1XtBkhyH7Ca3hILZwHbu5UCjwP2HuH4Vyd5D/CzwCnATsBw16Iz3LXUPd23fZDe34kAO6vqvKMdmOSFwEeB8ap6NMkHgBeOqlBpNvyBqpaap4AXDxjzEDCW5DzoLQ+b5Mxpjj8U5I93a4O/Yb6LlebKT+5aUqrqiSRf7X4h9I/orYR4+JgfJ3kD8JEkL6H39+TD9C653AT8TZJDP1D9GL3fMfsYvWWCpecFf6AqSQ3ysowkNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ36fxbRVsDARoGBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(sample_ada.samples_array, range=(-25,25), bins=20)\n",
    "plt.xlabel('theta');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We observe that the sample mean is now closer to zero and the sample distribution is narrower.\n",
    "\n",
    "Let us examine $w_1$ and $w_2$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0.01023228, 1.00584519]),\n",
       " array([0.00921258, 0.99287166]),\n",
       " array([0.01201937, 0.99365522]),\n",
       " array([0.02217631, 0.98925365]),\n",
       " array([0.04355987, 1.00076738]),\n",
       " array([0.07863284, 0.9971017 ]),\n",
       " array([0.13892778, 1.00929049])]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_ada.adaptive_distance_w"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that $w_2$ (second column) is constant across iterations whereas $w_1$ increases as the method learns more about possible parameter values and the proposal distribution becomes more concentrated around $\\theta=0$.\n",
    "\n",
    "## Notes\n",
    "\n",
    "The adaptive distance SMC-ABC method demonstrated in this notebook normalises simulator outputs or summaries calculated based on simulator output based on their estimated standard deviation under the proposal distribution in each iteration. This ensures that all outputs or summaries have an equal contribution to the distance between simulated and observed data in all iterations. \n",
    "\n",
    "It is important to note that the method does not evaluate whether outputs or summaries are needed or informative. In both examples studied in this notebook, results would improve if inference was carried out based on $S_1$ alone. Hence one should choose the summaries used in adaptive distance SMC-ABC with the usual care. ELFI tools that aid in the selection process are discussed in the diagnostics notebook available [here](https://github.com/elfi-dev/notebooks/tree/master)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[1] Prangle D (2017). Adapting the ABC Distance Function. Bayesian Analysis 12(1): 289-309, 2017. https://projecteuclid.org/euclid.ba/1460641065"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6.9 64-bit ('elfi36': conda)",
   "language": "python",
   "name": "python36964bitelfi36conda8bec451552304612bd355a97e3742bbb"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}