variational-autoencoders.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Variational Autoencoder: Theory and Code.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "jnLK0BmafFCY",
        "oCjTpfo1XmG2"
      ],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jarrydmartinx/deep-rl/blob/master/variational-autoencoders.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JdumEh0rBT5s",
        "colab_type": "text"
      },
      "source": [
        "# Theory: Generative Models and Variational Autoencoders"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "riEyUKUcBd58",
        "colab_type": "text"
      },
      "source": [
        "## What are generative models?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rQcpyHa1Jtsz",
        "colab_type": "text"
      },
      "source": [
        "The term is used in many ways, here it refers to any model that takes a training set $\\mathbf{X} = \\{\\mathbf{x}^{(i)}\\}^N_{i=1}$, consisting of samples $\\mathbf{x}^{(i)}$ drawn from a 'true' distribution $p_{\\text{data}}(\\mathbf{x})$, and learns to represent an estimate of that distribution somehow. The result is a probability distribution $p_{\\text{model}}(\\mathbf{x})$. \n",
        "\n",
        "Different models do one or more of the following:\n",
        "* Represent this distribution $p_{\\text{model}}(\\mathbf{x})$ explicitly.\n",
        "* Allow the distribution $p_{\\text{model}}(\\mathbf{x})$, or some approximation to it, to be computed numerically, for any $\\mathbf{x}$\n",
        "* Allow generation of samples from the model distribution $p_{\\text{model}}(\\mathbf{x})$.\n",
        "\n",
        "Spoiler: Variational Autoencoders do all three things.\n",
        "\n",
        "Here lets restrict ourselves, following Goodfellow (2016), to generative models that work using the principle of maximum likelihood. Note that some of the models mentioned are often used with principles other than maximum likelihood.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gu7xcBOR-XFx",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "### Maximum Likelihood and Generative Models\n",
        "* The basic idea of maximum likelihood is to define a model that outputs an estimate of a probability distribution over the input space.\n",
        "* We then refer to the **likelihood** as the probability that model assigns to the training data. For i.i.d data:\n",
        "$$ \\prod_{i}p_\\text{model}(\\mathbf{x}^{(i)};\\mathbf{\\theta})$$\n",
        "* The **maximum likelihood principle** dictates that we choose the parameters $\\theta$ that maximise this likelihood. This is easiest to do in log space; it simplifies the expressions and is more numerical stable. Hence:\n",
        "$$ \n",
        "\\mathbf{\\theta}^* = \\arg\\max_\\mathbf{\\theta} \\sum_i \\log p_\\text{model}(\\mathbf{x}^{(i)};\\mathbf{\\theta})\n",
        "$$\n",
        "* We can **think of** maximum likelihood as minimising the **KL divergence** between the data generating distribution and the model:\n",
        "$$\n",
        "\\mathbf{\\theta}^* = \\arg\\min_\\mathbf{\\theta}D_{KL}\\big( p_\\text{data}(\\mathbf{x}) \\Vert p_\\text{model}(\\mathbf{x};\\mathbf{\\theta}) \\big)\n",
        "$$\n",
        "  * If we had access to  $p_\\text{data}(\\mathbf{x})$, and it were a member of the family of distributions $ p_\\text{model}(\\mathbf{x};\\mathbf{\\theta}) $, then we would recover $p_\\text{data}(\\mathbf{x})$  exactly.\n",
        "  * However, we don't have access to $p_\\text{data}(\\mathbf{x})$; instead, we only have $N$ samples from $p_\\text{data}(\\mathbf{x})$. \n",
        "* We define an **empirical distribution** $\\hat{p}_\\text{data}(\\mathbf{x})$ that places mass only at these N points, approximating.\n",
        "  * Minimising the KL divergence between the empirical distribution $\\hat{p}_\\text{data}(\\mathbf{x})$ and  $p_\\text{model}(\\mathbf{x};\\mathbf{\\theta}) \\big) $ is **exactly equivalent** to maximising the log likelihood of the training set.\n",
        "  * The justifications for use of the KL (as the only divergence that is a reasonable measure of relative information) therefore provide some justification for the maximum likelihood principle.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "86geP0csR3K3",
        "colab_type": "text"
      },
      "source": [
        "### Taxonomy of Generative Models using Maximum Likelihood Principle\n",
        "We can distinguish the way different generative models work by contrasting the ways they compute the likelihood and its gradients, or approximations to these quantities.\n",
        "\n",
        "\n",
        "![texto alternativo](https://cdn-images-1.medium.com/max/800/1*otb47mG5KQWNh91prmyxzg.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o1MVpm0uBjsx",
        "colab_type": "text"
      },
      "source": [
        "## The VAE Generative Model $p_\\text{model}(\\mathbf{x};\\theta)$\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "fbOIEqF7T-FY"
      },
      "source": [
        "In the taxonomy above, VAEs are generative models that provide an explicit, but **intractable** density function.\n",
        "* This avoids the disadvantages of designing models with tractable density functions (e.g. FVBNs above).\n",
        "* However, we must resort to **approximation** to maximise the likelihood"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oyo_6V-8DRGr",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "### VAE Modelling Assumptions\n",
        "Recall that we have a training set $\\mathbf{X} = \\{\\mathbf{x}^{(i)}\\}^N_{i=1}$, consisting of $N$ independent samples $\\mathbf{x}^{(i)}$ of a random variable $x$ with density $p_\\text{data}(\\mathbf{x})$. The VAE model makes the following 3 assumptions about $p_\\text{data}(\\mathbf{x})$\n",
        "1. Assume the data are generated by a random process involving an unobserved continuous random variable $\\mathbf{z}$. Assume the process has two steps:\n",
        "  * a value $\\mathbf{z}^{(i)}$ is generated from some prior distribution $p_{\\theta^*}(\\mathbf{z})$, then\n",
        "  * a value $\\mathbf{x}^{(i)}$ is generated from some conditional distribution $p_{\\theta^*}(\\mathbf{x} \\mid \\mathbf{z})$\n",
        "  \n",
        "This amounts to assuming that, for a single sample $\\mathbf{x}^{(i)}$:\n",
        "  $$ p_\\text{data}(\\mathbf{x}^{(i)}) = \\int p_{\\theta^*}(\\mathbf{x}^{(i)},\\mathbf{z})dz = \\int p_{\\theta^*}(\\mathbf{z})p_{\\theta^*}(\\mathbf{x}^{(i)}|\\mathbf{z}) dz\n",
        "  $$\n",
        "\n",
        "\n",
        "\n",
        "3. Assume their pdfs are differentiable in $\\theta$ and $z$ almost everywhere\n",
        "\n",
        "### The VAE Model of the Marginal Likelihood $p_\\text{model}(\\mathbf{x}^{(i)};\\theta)$\n",
        "The VAE model introduces latent variables and expresses the marginal likelihood of observed variables as an expectation over the latent variables.\n",
        "$$\n",
        "\\boxed{p_\\text{model}(\\mathbf{x}^{(i)};\\theta) = \\int p_\\theta(\\mathbf{x}^{(i)},\\mathbf{z})dz = \\int p_\\theta(\\mathbf{z})p_\\theta(\\mathbf{x}|\\mathbf{z}) dz}\n",
        "$$\n",
        "The true parameters $\\theta^*$ and the values of the latent variables $z$ are unknown to us\n",
        "\n",
        "### What assumptions did we NOT make?\n",
        "Note that we do **not** make the common assumptions necessary to make the following tractable:\n",
        "  * The model marginal likelihood $p_\\text{model}(\\mathbf{x};\\theta)$\n",
        "  * The model posterior $p_\\theta(\\mathbf{z} \\mid \\mathbf{x})$. Note that intractibility precludes the use of the EM algorithm.\n",
        " \n",
        "In general, these are not tractable. Note that these intractabilities are quite common, and appear in the case of even moderately complicated likelihood functions $p_{\\theta}(\\mathbf{x}|\\mathbf{z})$ , e.g. a neural network with one nonlinear hidden layer. \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "umrw5TuRDUkz",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "### Intractability of the Model Marginal Likelihood $p_\\text{model}(\\mathbf{x};\\theta)$\n",
        "* To find the optimal parameters $\\theta^*$ by maximum likelihood, we must compute an estimate of the marginal likelihood for each training sample $\\mathbf{x}^{(i)}$.\n",
        "* However, **we cannot in general evaluate the marginal likelihood $p_\\text{model}(\\mathbf{x};\\theta)$ exactly**. We can of course simulate the data generating process:\n",
        "  1. Sample a value from the prior, to get $z^{(i)} \\sim p_\\theta(z)$\n",
        "  2. Compute the likelihood $p_\\theta(x|z^{(i)})$ using the decoder.\n",
        "  3. As the number of samples $\\to\\infty$, the sample mean of the likelihood approaches $p_\\text{model}(\\mathbf{x};\\theta)$\n",
        "  \n",
        "  As we mentioned, this is not tractable in general. If p(x|z) is small for most $z$ drawn from the prior, it will take a large number of samples to get an accurate estimate (i.e. to include enough samples where p(x|z) is large).\n",
        "* Instead of computing this directly, the VAE approach is to compute a lower bound on $\\log p_\\text{model}(\\mathbf{x};\\theta)$.\n",
        "* Note also that sampling-based solutions like Monte-Carlo EM would be too slow for large datasets where we want to make parameter updates using small minibatches or even single datapoints. This is because it typically involves an expensive sampling loop per datapoint."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Api5FE_4Mlww",
        "colab_type": "text"
      },
      "source": [
        "### The Variational Lower Bound  $\\mathcal{L}(\\theta, \\phi; \\mathbf{x}^{(i)})$ on the Marginal Log-Likelihood $\\log p_\\text{model}(\\mathbf{\\mathbf{x}^{(i)}};\\theta)$\n",
        "\n",
        "\n",
        "The marginal log-likelihood under our model for a single training sample $\\mathbf{x}^{(i)}$ can be rewritten as:\n",
        "\\begin{align}\n",
        "\\log p(\\mathbf{x}^{(i)}; \\mathbf{\\theta}) \\quad\n",
        "&= \\quad \\mathbb{E}_{q(z|x)}\\left[\\log p_\\theta(\\mathbf{x}^{(i)})\\right] \\\\\n",
        "&= \\quad \\mathbb{E}_{q(z|x)}\\left[\\log \\frac{p_\\theta(\\mathbf{x}^{(i)},\\mathbf{z})}{p_\\theta(\\mathbf{z}\\mid \\mathbf{x}^{(i)})}\\right] \\\\\n",
        "&= \\quad \\mathbb{E}_{q(z|\\mathbf{x})}\\left[\\log \\frac{p_\\theta(\\mathbf{x}^{(i)}, \\mathbf{z})\\ q_\\phi(\\mathbf{z} \\mid \\mathbf{x}^{(i)})}{q_\\phi(\\mathbf{z} \\mid \\mathbf{x}^{(i)})\\ p_\\theta(\\mathbf{z}\\mid \\mathbf{x}^{(i)})} \\right]\\\\\n",
        "&= \\quad \\mathbb{E}_{q(z|\\mathbf{x})}\\left[\\log \\frac{p_\\theta(\\mathbf{x}^{(i)},\\mathbf{z})}{q_\\phi(\\mathbf{z} \\mid \\mathbf{x}^{(i)})} + \\log \\frac{q_\\phi(\\mathbf{z} \\mid \\mathbf{x}^{(i)})}{p_\\theta(\\mathbf{z}\\mid \\mathbf{x}^{(i)})}\\right]\\\\\n",
        "&= \\quad \\mathbb{E}_{q(\\mathbf{z}|\\mathbf{x})}\\left[\\log \\frac{p_\\theta(\\mathbf{x}^{(i)}, \\mathbf{z})}{q_\\phi(\\mathbf{z} \\mid \\mathbf{x}^{(i)})}\\right] +  \\mathbb{E}_{q(z|\\mathbf{x})}\\left[\\log \\frac{q_\\phi(\\mathbf{z} \\mid \\mathbf{x}^{(i)})}{ p_\\theta(\\mathbf{z}\\mid \\mathbf{x}^{(i)})}\\right] \\\\\n",
        "&= \\underbrace{\\mathcal{L}(\\theta, \\phi ; \\mathbf{x}^{(i)})}_\\text{Var. lower bound on marg. log-likelihood of $\\mathbf{x}^{(i)}$} + \\qquad   \\underbrace{D_{KL} (q_\\phi(z \\mid \\mathbf{x}^{(i)}) \\Vert p_\\theta(\\mathbf{z}\\mid \\mathbf{x}^{(i)}))}_\\text{KL bw approximate and 'true' model posterior ($\\theta$-dependent)} \n",
        "\\end{align}\n",
        "\n",
        "We can write the evidence lower bound (ELBO) as:\n",
        "\\begin{align}\n",
        "\\log p_\\theta(\\mathbf{x}^{(i)}) \n",
        "&\\geq \\quad \\mathcal{L}(\\theta, \\phi; \\mathbf{x}^{(i)}) \\\\\n",
        "&= \\quad \\mathbb{E}_{q(z\\mid\\mathbf{x}^{(i)})}\\left[\\log \\frac{p(\\mathbf{x}^{(i)}, z)}{q(z|\\mathbf{x}^{(i)})}\\right] \\\\\n",
        "&= \\quad \\mathbb{E}_{q_\\phi (z\\mid \\mathbf{x}^{(i)})}\\big\\{\\log p_\\theta (\\mathbf{x}^{(i)}, z) -\\log q_\\phi (z \\mid \\mathbf{x}^{(i)}) \\big\\} \\\\\n",
        "&= \\quad \\underbrace{\\mathbb{E}_{q_\\phi (z\\mid \\mathbf{x}^{(i)})}\\big\\{\\log p_\\theta (\\mathbf{x}^{(i)}, z) \\big\\}}_\\text{Exp. complete data log-likelihood} \\qquad +  \\underbrace{H(q_\\phi (z\\mid \\mathbf{x}^{(i)}))}_\\text{Entropy of approx. posterior $q$}\\\\\n",
        "\\end{align}\n",
        "\n",
        "\n",
        "For our purposes, we will rewrite this equation in a form that best reflects the way VAEs compute an estimate of the (negative) ELBO:\n",
        "\\begin{align}\n",
        "\\boxed{- \\mathcal{L}(\\theta, \\phi; x^{(i)}) = \\qquad \\underbrace{- \\mathbb{E}_{q_\\phi(z \\mid x^{(i)})} \\big\\{\\log p_\\theta (x^{(i)}\\mid z) \\big\\}}_\\text{exp'd recontruction error}  \\qquad + \\underbrace{D_{KL}(q_\\phi (z\\mid x^{(i)}) \\Vert p_\\theta(z))}_{\\text{regularizes $\\phi$ so approx. posterior $q_\\phi$ is close to prior $p_\\theta$}}}\n",
        "\\end{align}\n",
        "\n",
        "(it's nice because it's equivalent to maximising the expectation pictured above, but with respect to q(z|x) instead of p(z), with a KL correction term which penalises how far q(z|x) is from p(z))\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QtOjQqzZr1d0",
        "colab_type": "text"
      },
      "source": [
        "![texto alternativo](https://pbs.twimg.com/media/C31GCqdWEAA_xVm.jpg =300x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5wn4UDiziTR1",
        "colab_type": "text"
      },
      "source": [
        "### Estimating the ELBO $\\mathcal{L}(\\theta, \\phi; x^{(i)})$ and its gradients\n",
        "* VAEs train by maximizing the evidence lower bound (ELBO) on the marginal log-likelihood, which is an expectation taken against $q_\\phi (z'\\mid x)$. In practice we often only compute the single sample Monte Carlo estimate of this expectation:\n",
        "\\begin{align}\n",
        "&\\mathbb{E}_{q_\\phi (\\mathbf{z}^{(i)}\\mid \\mathbf{x}^{(i)})}\\big\\{\\log p_\\theta (\\mathbf{x}^{(i)}, \\mathbf{z}^{(i)}) -\\log q_\\phi (\\mathbf{z}^{(i)} \\mid \\mathbf{x}^{(i)}) \\big\\} \\\\\n",
        "\\approx \\quad &\\frac{1}{L}\\sum_{l = 1}^{L} \\log p_\\theta (\\mathbf{z}^{(i,l)}) + \\log p_\\theta (\\mathbf{x}^{(i)} \\mid \\mathbf{z}^{(i,l)}) - \\log q_\\phi (\\mathbf{z}^{(i,l)} \\mid \\mathbf{x}^{(i)}) \\qquad \\mathbf{z}^{(i,l)}  \\sim q_\\phi (\\mathbf{z}\\mid \\mathbf{x}^{(i)}) \\\\\n",
        "\\approx \\quad &\\log p_\\theta (\\mathbf{z}^{(i,l)}) + \\log p_\\theta (\\mathbf{x}^{(i)} \\mid \\mathbf{\\mathbf{z}}^{(i,l)}) - \\log q_\\phi (\\mathbf{z}^{(i,l)} \\mid \\mathbf{x}^{(i)}) \\qquad \\mathbf{z}^{(i,l)}  \\sim q_\\phi (\\mathbf{z} \\mid \\mathbf{x}^{(i)})\n",
        "\\end{align}\n",
        "\n",
        "If the KL term can be integrated analytically, we need only compute an estimate of the expected reconstruction error.\n",
        "  * The single sample Monte Carlo estimate of the expected reconstruction error is:\n",
        "$$\\log p(\\mathbf{x}^{(i)}| z') \\qquad z'\\sim q_{\\phi}(z|\\mathbf{x}^{(i)})$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uyGPtV21MzI6",
        "colab_type": "text"
      },
      "source": [
        "### How do we sample from $q_{\\phi}(z|\\mathbf{x}^{(i)})$\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gZiyZzLtZ1zP",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "![texto alternativo](https://jaan.io/images/encoder-decoder.png =350x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E9zDll8oR24q",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "## Using a VAE as a Generative Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J9k47CJ7MLMQ",
        "colab_type": "text"
      },
      "source": [
        "**THE NOISE VARAIBLE HAS NOTHING TO DO WITH THE PRIOR OVER THE LATENT VARIABLES**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "96WRz9c61shl",
        "colab_type": "text"
      },
      "source": [
        "We'll discuss two ways in which we can use a trained VAE as a generative model: computing estimates of likelihoods, and sampling.\n",
        "###  Compute an estimate of the Marginal Likelihood $p_\\text{model}(\\mathbf{x})$ \n",
        "\n",
        "* As mentioned, we can't compute our model likelihood exactly, as $\n",
        "p_\\text{model}(\\mathbf{x}^{(i)};\\theta) = \\int p_\\theta(\\mathbf{x}^{(i)},\\mathbf{z})dz = \\int p_\\theta(\\mathbf{z})p_\\theta(\\mathbf{x}|\\mathbf{z}) dz $ is intractable in general.\n",
        "* However, we can compute the **ELBO** as an **approximation to $p_\\text{model}(\\mathbf{x})$**, for any $x$, by the following procedure:\n",
        "  * Pass x into the encoder to get the parameters of the distribution q(z\\mid x).\n",
        "  * \n",
        "\n",
        "  1. Pass our \n",
        "\n",
        "The quality of this approximation is only limited by the quality of our approximation to the posterior\n",
        "### Sample from $p_\\text{model}(\\mathbf{x}^{(i)};\\theta) = p_\\theta(z) p_\\theta(x\\mid z)$\n",
        "  1. Sample a value from the prior, to get $z^{(i)} \\sim p_\\theta(z)$\n",
        "  2. Sample a value from the conditional distribution $p_\\theta(x|z^{(i)})$ output by the decoder\n",
        "  3. As the number of samples $\\to\\infty$, the sample mean approaches $p_\\text{model}(\\mathbf{x};\\theta)$\n",
        "\n",
        "\n",
        "TO SUMMARIZE:\n",
        "\n",
        "Two ways to use VAE as an approximate generative model of p(x):\n",
        "1. evaluate the expectation below by sampling from the prior and likelihood\n",
        "*expectation above\n",
        "2. evaluate the ELBO to lower bound p(x), by sampling from the variational approximation q(z|x) (encoder)\n",
        "NOW PART II\n",
        "Sampling: \n",
        "is just what you said.\n",
        "you just \n",
        "1.sample values in the latent space, according to the prior,\n",
        "2. pipe that value through the decoder to get parameters for your p(x|z) decoder distribution\n",
        "3. sample from that p(x|z) and that does give you a sample distributed according to p(x)\n",
        "DONE\n",
        "\n",
        "And my confusion before, regarding the fact that this shouldn't work since we're sampling from p(z), and the decoder is trained to decode values from q(z|x), was just a confusion about the fact that our decoder was trained using a variational approximation. sampling from the prior would be perfect if our approximation to the posterior p(z|x) was perfect."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1ghc_fAFFHVm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "! pip install -q dm-sonnet\n",
        "! pip install -q plotnine"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AheyE1e4CUb3",
        "colab_type": "code",
        "outputId": "942c0f00-bfd2-4ff4-d78a-2e07c67eedd7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 165
        }
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "# import plotnine as gg\n",
        "import sonnet as snt\n",
        "import tensorflow as tf\n",
        "\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
            "For more information, please see:\n",
            "  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
            "  * https://github.com/tensorflow/addons\n",
            "If you depend on functionality not listed there, please file an issue.\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hkQWD4QqW53L",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from __future__ import absolute_import, division, print_function\n",
        "\n",
        "# to generate gifs\n",
        "# !pip install tf-nightly-2.0-preview\n",
        "import tensorflow as tf\n",
        "!pip install imageio\n",
        "\n",
        "import os\n",
        "import time\n",
        "import numpy as np\n",
        "import glob\n",
        "import matplotlib.pyplot as plt\n",
        "import PIL\n",
        "import imageio\n",
        "\n",
        "from IPython import display"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_zz3hNm_W4Lo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "cellView": "form",
        "id": "ZiF3aRIje_0R",
        "colab": {}
      },
      "source": [
        "#@title TODO\n",
        "class Encoder(snt.AbstractModule):\n",
        "  \n",
        "  def __init__(self, latent_size: int):\n",
        "    super(self).__init__(name='encoder')\n",
        "    pass\n",
        "  \n",
        "  def _build(self, inputs: tf.Tensor):\n",
        "    pass"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jnLK0BmafFCY",
        "colab_type": "text"
      },
      "source": [
        "## Examples"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k3qGF_JgLNPu",
        "colab_type": "code",
        "outputId": "e1c25bfa-826d-423f-a880-f6294bcb3158",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        }
      },
      "source": [
        "initial_state = tf.get_variable(name='weight', shape=(16,), dtype=tf.float32, initializer=tf.initializers.truncated_normal(), trainable=False)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RAyhZ8DiHxdB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# with tf.variable_scope('sickone'):\n",
        "#   y = tf.random_normal(shape=(16,), name='latent')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pXWStVpHKQlY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# with tf.Session() as sess:\n",
        "#   sess.run(tf.global_variables_initializer())\n",
        "#   result = sess.run(x)\n",
        "#   state_result = sess.run(initial_state)\n",
        "#   print()\n",
        "#   print('result: {}'.format(result))\n",
        "#   print('state: {}'.format(state_result))\n",
        "#   print('state: {}'.format(state_result))\n",
        "#   print('y sample: {}'.format(sess.run(y)))\n",
        "#   print('y sample: {}'.format(sess.run(y)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oCjTpfo1XmG2",
        "colab_type": "text"
      },
      "source": [
        "## Make a simple conv"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6gvgXTZTQcjL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Reset the motherfucking graph.\n",
        "tf.reset_default_graph()\n",
        "\n",
        "# Make a convolutional layer consisting of 32 independent filters.\n",
        "conv_module = snt.Conv2D(output_channels=32, kernel_shape=(5, 5), stride=(2, 2))\n",
        "\n",
        "# Make some dummy (noise) inputs.\n",
        "inputs = tf.random_normal(shape=(1, 84, 84, 3))  # [N, H, W, C]\n",
        "\n",
        "# Pass the noise through our conv filters.\n",
        "filtered_image = conv_module(inputs)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xr_-qoJrUQL0",
        "colab_type": "code",
        "outputId": "e29c0e68-f489-40db-f6b5-c47636ec9735",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 74
        }
      },
      "source": [
        "tf.global_variables()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'conv_2d/w:0' shape=(5, 5, 3, 32) dtype=float32_ref>,\n",
              " <tf.Variable 'conv_2d/b:0' shape=(32,) dtype=float32_ref>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7Y89Rl8WUGdb",
        "colab_type": "code",
        "outputId": "2d3c3ed6-e795-412c-c273-85aabb464e4c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "filtered_image"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor 'conv_2d/BiasAdd:0' shape=(1, 42, 42, 32) dtype=float32>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ezwcSJBFViGl",
        "colab_type": "code",
        "outputId": "ba6c5074-728e-4cb8-bfdb-35eb0e1d56da",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 92
        }
      },
      "source": [
        "with tf.train.MonitoredSession() as sess:\n",
        "  input_v = sess.run(inputs)  # Get some noise inputs out of TF\n",
        "  output_v = sess.run(filtered_image)  # Get our filtered image out (NOTE: not the *same* image as above)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Graph was finalized.\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "abux9WapWBKh",
        "colab_type": "code",
        "outputId": "7d8b3678-b7a6-45d9-a50d-45d8387890fa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "input_v.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1, 84, 84, 3)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7dIVQEm8V6-t",
        "colab_type": "code",
        "outputId": "70b7c9fa-c067-42fd-87a2-954aca5ed7b1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 405
        }
      },
      "source": [
        "plt.imshow(input_v.squeeze())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7fb4763d8630>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAFMCAYAAABCsp4mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsfXmgjVX3/2fLlDFkJqVkJkLmFJkV\nMl6UzIlI5hCSlDFDFKHRkJApRElkyJBMkXnIWDJf4/r9cc591nC5zlvvV/X+9uefu/Zd+9l7P8PZ\n51nrrPVZjogIHh4eHh5xIt7fvQAPDw+PfwP8Zunh4eERAfxm6eHh4REB/Gbp4eHhEQH8Zunh4eER\nAfxm6eHh4REB4v/ZAwcOHIhNmzbBOYeePXuiQIEC/811eXh4ePyj8Kc2y7Vr12L//v2YNm0adu/e\njZ49e2LatGn/7bV5eHh4/GPwpzbLVatWoUKFCgCA+++/H6dPn8a5c+eQLFmyG/Z37XYDADb3zIr8\nAw9iwZijga4mSqu+l+OYtyg4fv4HqyTH8rv3KVWXNntVe3D47+bNm5E/f349DK7wutFC6TbVnxzI\nBfcO08etfTmQ/zBLS3XP5/ofd9Xm437qGV5MByD/23AYyP16OX3cACEnraN152bgpsgQpde68FAg\nu0JJlG4BugMASm8ughT59f1MPIjl17vrKfaY1IYxbmUgH0EppctYkzu3SqXP8dgk1n3xwuFA3twz\nHRpmzhS0H0Middx83KnXg2e4QYuVLpVbEcin9LLxhMjR+MroyIXXunkzkD8/vp3LfctVN/cqs5B/\nLW5GklbYPUozA18Ech37lGc3w+ypEl7OO8ifv5nWlbvE8rLvlaqU+HiMOaxUeOgNcSPvMOfU0cx/\nl5DDF3IzNiM/8mMyPghUTXc/ow6bMZ/HrdNhvxn0U91M9jbL544qVTa5NCqrl+ouhhc0Ba7CBj3m\nsbmBOAkfKVXTOHJ0/pTP8uTJk0iVKlXQTp06NU6cOHHL4/JlSvhnpvs/Q758+f7uJTDypf+7V6CQ\nPF/Sv3sJCvkyJfi7l8D4Jz03APLlu/fvXkKAfPhnXRvku/+/NpT7M+mOvXv3xqOPPhq8XTZs2BAD\nBw7Efffdd8P+W369/I/bKD08PDz+E/wpMzxdunQ4efJk0D5+/DjSpk170/75Bx4EANDo++Ha7f7H\nmOFEBOe0qfG3meE0EHA9/zFmeBV6NNa1+TvNcBqdGfkdP2N/qxlOBDj3jzHDiRbAuce17m8ywwkE\nB/fPMcNpLVyGz/SYf9IM/1Nvlhs2bMCoUaMwadIkbN26FQMGDMCUKVNuPsmnoYtDUQT3qQM1EjrY\n6cUNSn1Nq36XH+xLSlVOyMvsAswUacNTHCcCqpgHomL7QCxTepRSrRBNytNF6bKvGBzIO+frIZOb\n5+GHbE8Hcn76PLzE0EO2SSynQByXxsG4PUh/WR0RfTPaC5ClYSAmin5TqS6dDD2CRIRFTn8hVMa7\nPB12mKWZ6ygfqyRGJ/YK6qNVOauxvJNeEfMNQCexkQxBTXXcHZvaqzYSjmQ5j/1C7B/Iv5ov67I0\nLpAvuelKd3B7eP5cTwI/z8HE3E8Fuub2Y+TkhrhZzz/o/UBePqi5nn+IGEJ/VwOoplqE8INGhDH3\n63Ns15s36JF7Vytd+wm8VveruTcrhFxmp1JtowdVO49ZTXgpcA54WTwPQ3foa0M554rWk3r+Hrr5\n+lZ+BnvNeUkr7TMn56gZ/jKdFY3GTm/WH9OHPMImff+p4ArcDH/qzbJw4cLImzcvGjRoAOccXn31\n1T8zjIeHh8e/Bn86zrJz587/zXV4eHh4/KPxp8zw/xSNwq/Ln4DQCA5VljwS6Bo/8Yrpza/lLxjN\nO0JOpVrA72jLjZVllM6V+k61Jx8MnfKzWYAxWbQ/sflh9rUlpuF6HCdN36xmdRcDaTHlVZqG2Kba\nJ6uze6H0/C8BACuoGkq7+VjZoTp3rNtSHUelx/NarPlCc1TzOeHDmoSlZq03N1/gQo4puj4cJ43P\n8gdhM1fd0V/pCuYy/t2q5QP5/vkNlG6VkNO5ONYiQERwP4l/1DfHpT2gmum/Y1/gsQpmMPGoYILR\nbV/IcpXKSrVoTOi5qQhgMYCKYNdTR2i/8DOuSSAXNn4xJz5x55129SRNJNwJl44r3SCkU+3u4XEI\nhAKFtijd5h8XBDJ91U3PP+Jhbsxfp3Qdic3eAzleVrqZyVUT9Aeb98POh57NTscmYFj6Fqh5jF0N\n2Z3eYhz4N4SHK2szeP1C+9vHpkBKYzQjxa1qpG8VnqUcAIDJ2InPUFDp6g0WD1JX4yKIYzf06Y4e\nHh4eEcBvlh4eHh4RwG+WHh4eHhHgtvgsY+L1grhG6SaJMrECOxeIxodKNYM4HKOdcVkdzcZRcTv3\nV1S6HLA+kxTh9ZyBqzxTD7SQYyDR/DGloveXBfL3rfRhpZ48wv1q1DLz5dKdMZn7InA8hV2JHMB4\ndupT8iCkmFOSG5/oc1rudCxjLnBsnfUCOREuZW+/yxwOAZmAWHGWoN4sT9A+yxQtdd8zMrJqsFKB\nionQlTVmDtksNZqPWfEC5HII+ga4IuNVW7riqiXWUyC6EssTFmldixpijtl6jgIhHW2aD1ewGmg6\nP6tjc+nr2PYNcY176GfM4flAzoBjSqcjCfWY75u437so5BesDQJcdaVz/Ypw49XflK4K8XVd8Kx+\nxpcc+CaQn1imVHjWPCsfqOdDhw7RCLGWjtYv/b6QdejUXNRW7RrZGgfyA/u1bpe8r23NNjYgvOfc\nSZjsuipVUxE69n17HZ9bcuR/Od3Rw8PD4/83+M3Sw8PDIwLcHjM8bFvFZKnIIIe8Lp7pez2QyZgh\ny8EhD2mwXulqiK49pur5oxs2VO23EOpwkAgmcgMvHWaTIYVObkFftR5tWqhMJBPGA2hzOoEI6/l4\nRSisp14pYPpKYFdpHveV1SasQWXN2fk1EQcl4+v6drMjStdRmhrvGROpaPj4h2KngqIQi9k2atVJ\n3cS0NTxHtUd05oUDZ4ZQ5W+0buEFbojILeoISDvc6WgcULRuv9YyYyD3gT7/DkJ+2zz942qwq+fI\n3J+Url+nsKk5DHCdAAzn9ZQy46x0nFL6CbIonUhgi5W/lkLIZ05rnUupey/MHaJFrLStfqwMqp3Y\nE8jtftF5kose5LxVl0KHFZ07w3IyM6Z2igHlwGFOFaqFuB92zBuCnNU74+11QwPdZO1pwDScD+SZ\n2ZcrXe3dVVSbdvI5T8yp1yMN+F1mbQ+UDN+rlUCJ+MWUbtVqztJyRR9Wuri2Q/9m6eHh4REB/Gbp\n4eHhEQH8Zunh4eERAf50bvh/hBhXQzg8Jl9VwR6U/ILue5bFBs21jyL5Gl7u+C3aabXXcWpWV5qn\ndE831E7MQ8JTlKW9nmO4ampf0x/gNMa3ofGboIizaVmWWcfh60CufyZ0XD0Q6p9xmPmlSNVMc1Yd\nlwfMCbp9spmjaV3VbnCO+bSmnXhAd+7BoTNDy2lGnJfvYeZ46795QfgM36mhVEgxR9OJPe0mBnL0\nxBFKV0OQep833G4krqNymXbU/bo3Vk241ppZRnrYnrhLa0YIDr0EpTV71MaqnBr7rnHZ1ot5kIcR\n6g13mC5islLDghmSGlEqpSH3O6/bHNVEEMPNT3nVaHXvyuEsWkJ9oJq+PhUWcN9D0DhF7Kf86IVB\nSvfZO5zTmAkaz0DT+YEEs1PZ7YG480xzfLmemYWmZtbsRdPu4XHW7l6rdIVsauTHz4r59PR3iWi5\ncSvNc/RLTBrz01hdoYTSTZB+yv/gFxv/Zunh4eERAfxm6eHh4REBblPoUIjVmzADDnVA7nOh09kN\nr9MbgfzKSv2K/qgIq/l2oTYtG1bm4IFPdVA+yvfS7QQ9Q1TRC7EHexueUbrsU1Nyg3TmDbmfA3lU\ndc2k8+JcNvWf1NEwmGNIrG/Ei3uBCEmcw0UZgWRsNJmjYfiFgfd0c2srHihvLGOPhGRCkMqGQy6+\nBSpE6UykJVM4o0VnmgDnGuk0mRxbmYWp8Y+dlK4ymNmm8SRjzz8nXShinQTcJ+zyvSIJBwDcJs2w\nTUcFC5AhynZ9OIMG23NrJV684fzhI8NrCYVVDWrKmknGDh8xjNnZq6TXGWUyaae5YBQHgGZgs7MU\nflW6wpil2huyhoLw6MA7sRn/iYmCXYJNSndtGhvmzR7TNWo+SL07kK+b849X3DxHmlM4PG845Oys\nCAlKrlnAvgAzRD2k+M6BbIaNvd5wXsP0t8380rofnEOvI+kvMQuCc6b+wo+CZb+g9tFQLO58hn+z\n9PDw8IgAfrP08PDwiAB+s/Tw8PCIALfFZ2ljhxII10NKM/1vTvqQfoYGV16sZFIKF7mnRcuc0ufG\n1/G0YEgxTC5wgjq7qGZYpx9EBbmvdahM2aRFA/m7R6Dxq1lPRhmxdS284lAqqDrlI7pAVIPHuUjY\nrpVjlW57qbaqLetbbomj8FmLhlo5YYrwyzWBxsdxsJrXNe3PeNz1S/VxhcsL1iE7TlXhw1rAIS5E\nbYEPRFiRIYsa+d4k1W6fvmkgZ4LOhXzUsVN7alOlwtnJ7PssCp0m++PkkK8t0bP34NIHB3CqKa+1\ntAll2tWN44qq5J+odAshmY0sI5H07xpq8nmm8FngliR8BO2zO7qQffhdq2jGf4DTT+k5rXGT5POg\n706bkrrvj99z39WkU5rh7uY5TDKsy8WhPCcK6s942mkZVJuIi+Q5pys4yhC95Mv02tZ+FUr/TTvg\nqC7CBsAdFr83NNBhhZ4p3cPDw+Mvwm+WHh4eHhHgtpjhOcJhDb8QIYdz2AVhllCzmxwFkInmb/IZ\nk3hOXjpG6eKP45fyTnRQ6YZZs9yFCnkTZYtVMGtcQu674rLJ7oGod9xgpNKNn8qZFy3MfNqYBnYO\nZfog9/Lq8FpCGSutRN7Eu4Ytx+EwNyiz0o2Zpud4oT4TnF59ooDSxb9T1NyeY65N+BLTWzcg/xXn\nVayw1t1TRd+P5a9zcNNxG+hE3JdcO6VSowr3BWUELuRjeyrpVh2O0gA6BKyjMGGLm1OcK86rOtro\n+b9nRhoYszNYS0x4TCox8Cl9PV6ow/JoE7p02nHoyl2WLsmJylvNdOZL7jWa96dV+Bp0JIJ7/Xel\ne7MXfx66mfOg77nY37Umzytd/FWCtHiXrk3/qqGR7t9NhGetCLkvaAXBlXZYLUzfT8znYQg4XCeR\n09RKsTxGs8T+8LV2Z6ga4/nMgT+GfGGUdbVJBQPKikm+y6EJhTnzJzb8m6WHh4dHBPCbpYeHh0cE\n8Julh4eHRwS4TaFDMYzTBQD8hK6i6PkF4xZbLhi2Nxe7uc8M35XTqpzLWE5vfBRmlD3hr4iD1wj3\nGW6VvQ99G8hukw7H6CymHxLLn8cpVX0N71DfIZo9CLJ2fZZwGNMhwGUB7hFM7SNN6EjNYsIvV9+k\nt72smlgJDmWqBR3mdAJviJZOhVs/Zy8AoHCN1cCTJhVyzr2isQ+Ropomw8a8sjJ0KI5wJAEigivF\nrE+7v9+m9PfP1emvVINDZ5zxhNUQLOJDodP9cggCfqdJtIFwVBE1JLgpDkjCvtdENXXoVrQ4r6bz\ndAzW5IHMmLXze722nLJR2dxjk2PqNobScQnb4Qx7FonoGFPLDkgr5jypHxxaz+zn7uH7oJVmq+jA\nc658IaQr+SDw/U6gVM57+bCtZv6n9vMcPxpdMtMW4VsEndLqIJnHLOsUhwdan6V+HnQxM6K37AIC\n+DdLDw8PjwgQ0Wa5c+dOVKhQAR9//DEA4MiRI2jSpAmioqLQoUMHXL58+RYjeHh4ePy7ccvN8sKF\nC3jttddQogRH3Y8cORJRUVH49NNPkS1bNsyYMeP/dJEeHh4efzdu6bO8evUqrl69ivHjxyNVqlRo\n3LgxHn/8cSxcuBAJEybExo0bMXHiRIwaNermgximdNfg3kD12tT9qutVsZq1TnObXe3NwWuLG2of\n0T15OHbsID4xC1ipWsEURHCJyuuul3uKjhXMcezfulx3t9IlOs4+QiQxvp4yOiYSiTn97Imwy2gx\nESo6J3iybxBz1lLoxmtttOFzk4Rp1mdnfTgS6trc/YpWFhEVJM8fVqoHVqRU7V0Qx8aRbknG2Xqn\n46qA0SLzlbaZapO68CLKFNB+we8KsT/rtQ26b295+i9O0MqRJv31BgjiLAUvHqGl6uOmch4hlaqp\ndcW4TTqUFg7TRcvkkFrOPndfeD17Yvt+XRGW/1indUzijr2n9c25T7jbtzRSqhtQ/Ynp+oaX+CrB\n9XNAX+Ff361973R/HOPYfyQR13X0eD2OoOkbbz5ircaGfZZtbhQvLP2UuoRrXLthxD/wjBo1Ktgs\nS5QogVWrVgEADhw4gK5du2Lq1Kk3P3gLYgeNenh4ePyL8Jdr8ES018aUdfFvliH4N0tW+TdLrfNv\nlhr/oDfLP7VZJkmSBNHR0UicODGOHTuGdOnSxdm/RfgSTAChBRxensIfkKUP6wrs8+pwCEiybn31\nQGM4Ncy9psswFRMs0sMa6hS6ulP0ZulyhY4lAA0Hfq10n76s2xocxjD5s/ZK88wE3kg+zKsZ3vGi\nCevYwrL8iGUDQBv4bj26eog67nhbLq7lRtoPx813pBZ/6L4Tbk4sg0TXQ9W8LgHA3IFaWUIU0Dqo\n6eB33fOEaqspYj2B9QJpIkYrTXvRdfC9l8xxwnopoFMo3/xMh4CUrMObZe/1SqVIaH43m+OT4C+y\nxtAFwz5+cA03HqwD7OS7t9BeyAYyPMrs1pc5FdUl1NXs2o3kazP6RWgM1s3fRAE9u4+iCoeLucyG\nPeg8r61KHBtgc8Pwv6bgL/oftTi0jmLCwV4FqC/h6zk8bnkdnYUM4kvgmGDNBwC0yqqaGz7hDbLw\nNp0a7ZJz+qN9wlq1Eb4/ss+m+EH69zjCEw3+VOhQyZIlsWjRIgDA4sWLUaZMmVsc4eHh4fHvxi3f\nLLds2YI333wThw8fRvz48bFo0SIMGTIE3bt3x7Rp05ApUybUrFnzVsN4eHh4/Ktxy80yX758+Oij\nj2L9f9KkSTfo7eHh4fG/idtT3TFNmEX5N4JL40C/s89ivCm91yqloExqaeiwBydlecB5pfqxJZ/G\nQxlu5YdgNvACxmfznGh2MtxqtEP4obLqUKlRh9jB1M7MZ53vX/Zg70eTN0LjnKC2SOvewcmCL3DH\nembd15ax3OcxpbJpAZKsfR5NV7qsjv1iVUhXXpzvQl+MRMdjOcY3ifMqiIR6Qrqim/k+DmS31dCI\nC3xgrtVr4lrtEv8nIkwYxbqW1p8Xx69I75uUxmYoJXpp2jOIy09jVPIhHO0IzxTDBi76XoTu+0IC\nbryv7470L8b6YWaVkAebc5p5Y/9i6AenYvp/EH7zilrnloj012t6rC6O3592G5/tHpRT7S1g5vb7\nC4fY0H9eD+R6GNixXlC9Of3DTEJR0XHGszoZ+cQHj6r2NRQK5NYYp3SUS+wH2/V+IHqhurls8+U1\nz2J6H/wv+yw9PDw8/n+D3yw9PDw8IsBtMcMfCZtza4jwiHNYA2ZrcaTjJZ8Tpp/1itYRS50RR6xg\nbOi4x0QXQqwz0XcCR5y2nzKCi1l1MybSW5BF0p686WwlftXtVQeMa7i4NGEvAJCxe4zZ5s7UXCps\nhgqaDR6k4z7JcQjWQ2d0OMamrRwDebW4Zgq/I8aaSUqxwoqcsB/JuCicjnICnuLFv4akStU7fM4A\ngIzmJI+IglWFmWaH1gPLxfVpbszuXYnMOJcWsVy3itbNE/LFjVr3xEOiYS7AV6LQnQPOiNtR/QfD\nHC9qhH2jQ/lwWJrvsNguZL3uLKTjXquGA8/eAwHugNI5Yeoahwkui1lrmnOc/bZg3XlRh2PFdzp0\n6Oo0wcoVdhmFQ6kBx/eYkMSs7YNALoGmSrcqhXYZlT/N8UtLRRE0AMBJjnPF3brQGd4NfT6pFcGJ\nAnUAMBu/BXJNaNojopvQ48O/WXp4eHhEBL9Zenh4eEQAv1l6eHh4RIDbEzr0XjhUpxXBvedArVi3\n2um80RKiSh8Zf4pwg+BQq6eVbkBVrso2LsrGlehKjFFJQqf8yXnEypOW7sZSPXRcwfE32E84tq0+\nrohwsDY3zOCri2pfU/ZCrwfyno1h3wp9AeeeAhH7RWPndA8IxD7opVT9ZumuWMLhGm5MK6OU4xqG\n6XAMDOEzZDXX5pA4Lta9ifUUiXEn6PCc+CLDsJx1C/ZmeddrLD8Qy6crs+iBo/fqlLYM8fYFcu49\ni5Vuu3wAret7g/B1Fq6kVEPDvtZOdCeGuYt4OQX7wu46rQf6UDSfNPcxq5hU16QEvhBdH7VrK6eb\nblnoL1Fs/zITMgAuzRStmsI66MuGq+V5AfG/nqt0ZatpP/3y+TKx/fHwWrbBuTx4af9LgWbY8l3q\nuA1N2In7MKrqBdCXqrlWuEkXpNJd+25mn3bZx/UPBctdqDoBXS97g9xwyb5urk0c26F/s/Tw8PCI\nAH6z9PDw8IgAf5miLSK0Dv9tFZIHpWY6tRTIo/tS6UB899XntCpJxUB21FDparluoqOJ1bisWYjO\nJmJGEgcdcgOZ7fGGZjYiFOD5z+kMnroXuVj86iLaRH6DXlftHsIq+EKEI32BOZCBK7FNK7ZR+0Oz\n7vSfUEi1l2QeKxeuQMIseUkzrWG8IGs5ZEySg+K4WC4CncCD3xJy38pLta5Qfz72PXOS7wvT+5KY\n4wEAX4pElCprtf24d5+eA8ShM9ug3RDO1QrkaaSvY3vHpvdxM+TSDaGMnk44gKUbcoJmivAtG2Y1\nnM3LN1/Sym4dO3Jj+Aile92xrfkoTulBv9GfFXLVxfSG2QqioNt88xl75PdAPOe0qR1fjmNIp5b3\nhAFz6NHcP4S8HVHZuLjYBPRQR7VIyFe2ZqG8Sjc7vjbDixXhZ2D0GnORiYkIl6Ow0bHQ2DDUPVCE\nPyvOmOFx+ST9m6WHh4dHBPCbpYeHh0cE8Julh4eHRwS4PT7LHlruUfcEt+/XTNlw7ENqY/wwrYlL\nF7RzJ5TuPKUVY0xTupImN+8OCDZ00kzZWepwOMih4zoV8sPvmGUFpTVT+nQRHfNoae356LHS+pO4\nswuH2BCApwCUjCOLkzT/uNK5+ZrWWhbEoHf1ODLMxzJspxf1GjI6HVY0W/j+FuvIHTzxhW5nEGs9\ntqmf0qXrwylm8br9oHSfV2H2mC8FGXtbAFUEUTk5TRvuHu+s2nVbC//qe2OhwSme9V19pdkA9kUX\nvpZe6RbUDvsoZwML+h2E+0KGJOnnOMHvHFbT1TjCumFrIMcKnRJ+StdXH9jYhMB8TCG/JOHN2JxL\nJPyU1tX3eWrRMAf2fzgQm/TUzr6PsFf3vZadp7gjlNJIBLgaSfAELQt05Zvpkh9ukgjJMpVLEj85\nR7UvCsIm94thIXN3iYY+kRKT+aQ/NmTseFKkP88pgEjh3yw9PDw8IoDfLD08PDwigN8sPTw8PCLA\n7Ul3TBCKO6QrA0LyVU7bG1pCx1l1WiXiw0xpnz2zGwTyKxsnKt2U2sy75fb2VzrQZtVsjFDc5UcY\nDafDJdHiAPs6ug7Rl+ZB5fyxl222kGtplQ4XBU3i+DwXpmqOof1STkTj+4TwfebXGmw2bQL7W4e+\nkVjpXu7B/qwCtF3pNj8fPn4sIY3xkf0u5MbQ6aaF8LlqL2YWLNRKo/2pbWACL9XC3Y1EgAh7xXqy\nQyM2T7qMOzU0bHQyEKs4vZY7/+B43c9T7tNjupilhFIvz4kQzaQma8/9xH5Z2qiiZ3G6Bq82ZRp9\nHR3xdcziKirdQWjG8f50LwCgDwjO6dRMkKho2k8ztXfty/Tob5L+kCV17Be8YLaG08b5mQJcRti5\n0OePaBacqwUSnwcbk5tHVJvcZuJ8J5ob2Uw42MlQ/eUm9m//bJ7VmDOMRwT3k/nNQPjXi/RWKqz7\nb1d39PDw8Pj/DX6z9PDw8IgAt8UMF/YL4BwcGgUq2rpNd83LJhMlXaJ156O5QbqY1F0N2Jw/NU2b\nHS+X08sZtixmOQGvM4MjJ+A+0peGGotwlDT6MHzFIQjfJvxJqcpe0YlzM8FhTgmpLwCgOvpiHvqi\nhusrJqytjmvvmFqo/Fq9tjXF9HkMFKE97gnNlA6RYnqIFinNILcBADCKCqFPLR3G0X82s864N4xp\n08MUaWsmwrfeb6D77mR5qr6NSETMvl07GVPO0DmCE5mpyR84po4729cwZfdlsZQxrQYJ+76008+K\nW8iFxiZVSqZ0ndw5AMDvBKR2wClBGbQ4xSXVt2IjTiNs+olmxJkMZsWaZZ6/mmKpGaGv27a1uphX\nqj1hn0kDwgjoC/nS0YzcqNNN6bKseD+QD83T7hPUWC0ajyBuiLWXC/2hbwjuMQdwTTxQU3OUfKx0\n5BZS1depyq9e47V3/NwUviPp+9AhgDEUZTdmZOJnzC34QGmoimdK9/Dw8PhL8Julh4eHRwTwm6WH\nh4dHBLgtPsvs94WcBnv2ErLf57BHZE0tdppWvGJlLrruTupUOOpZhHW1b54XuMrEHJWAZQoP+TqI\nCM7QUa9ZzrlR6amT0i0QXa/U1petw0wR8iKqxwHA+rbawfn1O3xsTDBEWwLecUBb6dBzO9VxTgYM\nZb1X6QYczKjavYpzuh2tgoYIx3A239GF4pyIJgJlTUrld3EwpXfRjPfl0p0N5GVdOiodgWnJLMP6\nWjFssRTimNOaKb27PgwLTN7cT4U4PA279VqLnuHQmWdwh9LJJFbC93qSVmF/1rsEtHZowBFImGZj\nuUSG5+NG9bVsxPr0iX+4lErTf6DmVe/TI+YIgnM6Pq0mJgdyXzNDZiHf/YZRiuoArpqmQcRe7TOl\n/mW479MpeS1wWC0ueXGcVMc1Aldp/MRMv9pkNBbPxuFCtR9+TOlm9hFlOvuZC9k8XJ3hfQoqNQTr\nXsu/bziRegoANOHm22FEueFvvfUW1q9fj6tXr6J169bInz8/unbtimvXriFt2rQYPHgwEia0BTc9\nPDw8/ndwy81y9erV+OWXXzBt2jScOnUKtWrVQokSJRAVFYUqVapg2LBhmDFjBqKiom7Hej08PDz+\nFtzSDL927RouXbqEJEmS4NrbAtKwAAAgAElEQVS1ayhZsiSSJk2KhQsXImHChNi4cSMmTpyIUaNG\nxTWMQl/H2RX9bHbFEfHKnNGE7sSRQeMEW4yL1gXLSNk9AFVdGxaK2nplWC/mePPdd5TuSKu2gbz6\nLaXClQNiPaNnKN3xfXqt6e4TWRsrQ2YglSS47x2o1M1N5N+/YzbqVGXvUjqH6qoNEgzYzlBey5AU\n+kNpjtUKxU6lm0X4cJ+e/9l72XrI2keH3HR7TZ9jO2LKcwqHIwXLIZHtdESpMCMTy3eL/z9KBAdB\nQ3S3vjf0mw7zccIMpaEPa9389YH8SgYdVvX69+yGeWSfNh/XUIitKAg4e4Kvz+86AgWpZSTTGMN4\n3p5Zj8gwZCWuytcxeoFxdcCAYv4QHnc65KaQ+FwN0/z7UE6MNNrUx2+yYKB2Jzlod9Ie8DW/D+fC\nB4XCA+8Uj0N0HExaKKKfm47rdOfh4nO+2VyB/OLQYcafEzjQCDBk8MCTbIZTA11MD1Mexs1wyx94\n7rjjDiRJEqJfmjFjBsqWLYuLFy8GZneaNGlw4sSJuIbw8PDw+Ncj4h94lixZgnfffRcTJ05ExYoV\nsWpV6FeD/fv3o1u3bpg6der/6UI9PDw8/k5E9APPd999h3HjxmHChAlInjw5kiRJgujoaCROnBjH\njh1DunTp/qNJvRnuzfAA3gxXKm+G/3PN8Fu+WZ49exZRUVGYPHky0qQJXazevXujSJEieOqppzBg\nwADkzJkTdevWvekYZcIn+R0IZeCwwnEaFWG16mur1EkUFfIPNuZCRCDRm+Yhm2tYiOL1CfUjAjQZ\nOpCVj31Sf1Yw9wynXu1JOUjp7hMb22xzCjWhWbxLv8ShTSuHlwqvJSYr9ObnP17ux+53pduE1Lrz\nfhZzdtDVJnfMHoBbgYiAM3eq/7mU8mLp+03Ypdcazaw7reKZe1Wfz7HoTK37oZ8IwZJM4QSkFdfm\n7tqaZ+nnSrraI1of5UNtdNSTzAo1PUFBpavXum8grztQTekKrwzRDLmJBGrmEG8S60a/Yr4sXheT\n6n0cMftKCJp9/BRWBHKqOpphnPR3MPBd+G9pgoMO64H7VhyodUVFuN5aM6T8WrPVLYtDr6cwPgrk\nveEoI/qU4KIcRnFxR2Qsqw5Dne/iqMRqQGKfH2jCsxJ8zHLXWEd+Fjqe6gBOB5pVrcxzNlioj3om\nju3wlm+WCxYswKlTp9BRlO8cNGgQevXqhWnTpiFTpkyoWbNmHCN4eHh4/Ptxy82yfv36qF+/fqz/\nT5o06Qa9PTw8PP43cVsyeH4NOwYzEeFX55BZWlfDr+gFdUogWrrSEImMGutrlGcRy5DV/Kq4a2bI\nnDpFVeFO6CwdpONi7d8Zs6OMJDaxkf4bm7Nc6H2tM8S4+UqxP6/WytYAgP70PPq4sXgNzwe62tD+\nxM+xRa5G6UwCB6aI7zKTh4GcwoWxI9bVCsXLEn2iMmaAuAumWWhtUqNleyofmirNlkrsz2smCJHe\nJ0INVzpoz6MVUDigZ3wwG8s6DwpQ/m7SWWKDH2VmpXXLjyrdw+G6d112EAbndOgqBjY14XBtOmeX\nPF+3qdI5x/6dE+bj9wU4XrmFIpQG8L3xGQ0MHUvzEOsDMUCc4xhzN46AHcOtmmlGpPcEp7ZJrgMZ\nczqFWPo97n4AwBbahXzuAcxezm6ZHGvNsyIsbzqhi9kBr6qW5Gam8vpaxaf3Avmaa610WzeF/uYp\nQFiWRc9f9hCPE8+wToFunlzjc8M9PDw8IoDfLD08PDwigN8sPTw8PCJARHGWfxWZXw/9pbB8XrgQ\nMj/1oOr7jfC1lFM+OsC1zMWNwj8rXXcx5hcp9I9PT51pqtp/BD4cAtIaqu497Kf81lbFmsD+jd3I\nrVT3r5Lr0cXUYjmFt6wJRJcz5BfrD+C1nJmBHXwiS42v0wnnawvSxeHpq0267yTpp9FhTu/EY115\n406kszExH5YPxoR17dRxnUsf3GR6c6DXQmgqmWnivmaB9jW91oV9lrVycBGu9wHME3F/Re7Qfqh1\n11QTqWTD+AWXPMcnXeHFC0pX81sOkOxiXG1Jd3KFsp93zgfAoUWt7V2ux89RW9yndUM54y2t8TXm\naSYa1vVdRzcvjufn2FXQvj4skSxYGk7GyxlKpPe2Cfamsq9rpYlOu9/x83gmPW8lF9MXwQNlb+7T\nViFJTTNr5V0mJrg8R9rYsMKfRTOXuTblvg+d9fECwGOmKJoeJpFSUexPawD/Zunh4eERAfxm6eHh\n4REBbk/dcBdKPyQaGZY5NdHOrq0SrbxTHHfReBAug1MRTVA+njRt+iZcR7vcRbj22kTKPnpfIO/J\nZw7cIjJaSJtvcJw2RcZEdq6QapOIloqJhqHVBFfcQSU0uSrquIPHhwdy1q4HlQ6TdQaLvHL1TAjW\ndIgQLNI1zimmKBoRvr9Lmz2DTnO621TSqXBJ2qom3FiWvyRdwK1yPb4+bro+TlqFX4uMDdqsyX9f\nMs/NMKeLqz0v7vqzuu4dipcXpLU2PkZk2zQ+p8//I84vBJw2ZxOVaa/6XloubGhn45RFimNGo5Kp\neUViG9AaoQeJKD5qGNU84d54kbSpe8WxO2vsFr22EeJR7agfW5iaaOgsMgUXvRDKyvlpzCAUeKE7\nNr8jXD+D9bPyZRdOZK1i/QCr0+r2o8IVF6WJejFJ2tfGnJ8Zeh6oVg24IebiDBHF5frqVG1qo8P1\nJPybpYeHh0cE8Julh4eHRwTwm6WHh4dHBLhNPstw8SCiWCl09U0q3F46H8gTjatBuhBjeXOIQx5e\ndiuVboik9gLg3g6P8WLscATJUONIc7J84nYHcqNVJomwhJBN3Xoy9d+/ES7Ex8Npije6NhYy5OI9\n6DS1VyiTajvH4VPdoHMhnxGLzQtdzaxb+MoOIuB+aD/oHvdVIJMJuQDptLGZIvKowroeSpd6BIeH\nXEuoz3nqMpYbiHr3RKZEWu6Rev7POujl3MX8qm5EA913iDiv7l9pXRmR7lZNBSDhj+qhMK+U84DT\n1YFZ83lFz31vnshf2L+MZ01KrUBtNFLtmRfEhcuqx8zXT4ekbRkdSrGl7Td4dn7j+/NMmhpK9aGg\nL3rZzD/0TZ6/W3c9/5uvmjn69WQ5TLtH2AqHvMCn21jXSPsFk4sn+axhaITxk9YB3+cZVXUoIRZw\nuqPdEDqFlzqUCAnMsq+Q8FkH1E03HEbBv1l6eHh4RAC/WXp4eHhEAL9Zenh4eESA2+SzDDntiCaE\nZY5B2/CR7nu2CfsFZ6WdoHQjjguOKKdL7zbFgUCejOR6UCqqmznDtFw7CM5NUbrm4Pb7H+rYveVN\n2gVyWTdazyFdiBktNffN4+W+DHtJKhOw0AFVwLGcP5s0wVzguEsy9N/znL6QNcA+u/az5ivdyFp9\nxEqGKF3C8L25RA2Qz5Sj6F6VOcmazDd1FMw5no3HpRSSXzd5ckNE387an0ZgarOcwp+2gz5WDPS1\nSptrvMIy5wu9oRaTnilKmUdp3OntgXzSfDSmhu/bCyCMgUM7oTvmdCmTWu8wk/3K59soHQTV3Ptm\n2c0d+/McBivdSNNuH1OCZAVwwbhwP1rE5z9mpZ5k82nRWPC90lFVZjF3b2p/3oBKOjW110PsM0xy\nPjTH+SRA0gvAyiQ8fyFLp8gMhUjXW39WT0DnGBOYcX8M6d83KoBjnXOZOU6E79XddB3OLdPK1OVY\n/t38ZhHHdujfLD08PDwigN8sPTw8PCLA3xM6dJcIAfjD5hRy1cIXsE9pxgjdc+X0UeWXcY2gxqtH\nKN0QTSqOzlfCDNMUO3RIhuekczZRks3yPkZzRph2dY2JXMrZokyyvJK+Nh3F7Rhh1o0VYtxYd+2q\nao2+l9NB2+3XPQl9efYP+mrlswjWU62wPo8FG8V9K2zu23qbtyrkRXqcnZWGBnJ+vKx0yYT7IDFx\nyM8hrqkYa/jQP0qpZu43mEn96R6avmcAOJarD2n2pP6rRL5hCTNLOJSGBoVqYBV4m/X3ROuu6UX2\n3YhzhZUueb6nuLFij9JNcOzeOLlFV8zrnnmRaiNViCGKaB6ApUq1tmeFQN5gUirbvKgLwUnICKQa\n8/T5zzYFRONlYjeZyxSqEkk/1IErOgPfruMifTOa6vMYObkyH4dm0NBF4jqJ9RUxi42KK9Sud+gP\n9Sc4aEqqYo4/G2vN+XvWIQ8PD4+/CL9Zenh4eEQAv1l6eHh4RIDb4rO8fjDka4mXtRKuH1yE+CnY\nZ0Ep7YpuPg4JNrWiX2vdOkNGrdDTOG0GhkIQiH7BceP3SNWOQ5emj75f6RqT4B0TVRgBAI5TyFKb\ncpJtDOX28CXf8ro3hsKa8tCd2OYuIm93Dh2aP7KzOq5atPC9pNd+WRy5UzW7HGX6sNQZdNceMswp\n0xqtxCMAbpx+qWo72uiss1lU2/1yiBs5zBQy+/IrHXJzzyH2Guc4zf7LJSkIGR374Y6SPl9IXyOA\nmlV47bNP664Un8NRcPUhvW55kpoMH5gfuseU/Wm4PZ8ja/XEgergNu3Q+01cuje6vaR0Q+aKVMjd\n5uN3iQ+07OvjdJQPXDgdlAixqjv2WcBy6qqaxX4omCIvr/nALUQvHv/XAUrXw6TxDmvPuYrRw8Lh\nUC+dA4Yng8soaBjTvaDX/Tg/m26yPkd62Fz0saICwTta1VjIH8faN/h3CThdKcB9IdIvn2qudD50\nyMPDw+Mvwm+WHh4eHhHgbwkdaiNemSt+/bbqW/sxwSJNmj0IEIXIyhkz+ItWfFjnUUp153jdNdG1\n0Cn/cQduwPRzB4/zYhOtajc5EJ3JCpl+lIvF10NVrcxuYoD2sBmSMGE2AMClS/uRKFE2XCvBmUhX\nl+lb4yDsaWeY2hNolwHuYVOTdmuVg2RTKqZ09yUJzbnnfOxr86swCzNZ2mzsVC31VDmtc+AidTZU\n46l7ec45+/j6E30IOKZjb4d+6rgx0Mw2uF+Me0Sfx7U/mLk9yR+a1f5SOrVwPWbF8HO86DpcpXig\nRdy3iyZdwhBBHpRjqtb9spjllubTN74aswXNmX9J6Wq4X1TbIfTZIRoV+zlewANvrap1ecuxHG+Z\nPuy6kOdX1ydVbq7OIErqmOmLYtjoyxOw1CFPBXbLbFMGM9D1Xc4oG9h6s9INh2acf6gLf5YrPqDX\nKqOMSBNEIU+X0Odo25iscL3NtXmNs7QsVT+RDQpk+DdLDw8Pjwhwy1K4Fy9eRPfu3fHbb7/h0qVL\naNu2LXLlyoWuXbvi2rVrSJs2LQYPHoyECRPeaigPDw+Pfy1uuVl+8803yJcvH1q2bInDhw+jWbNm\nKFy4MKKiolClShUMGzYMM2bMQFRU1K2G8vDw8PjX4j/yWa5btw4jR47EoUOHsHDhQiRMmBAbN27E\nxIkTMWrUqJsfGONPIQKcg+R7tslO0rtw6DGte1SEC23Or3VnRSZeepvel836/tiHakMudHFJE9Yg\n+3U2YTVDWRuLfV0vB06kcRLu5bmcg0M9Mb32pzgROVTSfM0lkWUJASzBrEDeZBjPC4oFPZhXry7n\ntlBY1Rw6H8sP9u17POmj+XV6pWKKBzAPZwP5LdJxRstFaNUMzFC6aDFnYxEdRR0ILrkIZTnXWx1H\nR01VvkeYAd7tN6z2xH2vHdKqIsK/vbH1da3MHPJa3ZDVfqhu9nyZqXWKkl5rLXnoPH39l1djn9lB\npxl4Mi3WjPdvVwlNOutqJ6C+vseYziUtXW2dmkpfiQ/LWWidWM41nUGJBBXSq/bHRY4FcqMYN/wW\nAvI5OEVqfvOcSiIdD+Re12VCqRfT5afJqWOnft+BmyKYkQjOhvmB001fNRVU+8axHUa8WTZo0ABH\njx7FuHHj8Nxzz2HVqtDDeODAAXTt2hVTp069+cFbtgD5bA64h4eHx78HtzTDYzB16lRs374dXbp0\nUYGbEe21+fPHdPZvlvBvlv7NUjT8m6VW/YPfLG+5WW7ZsgVp0qRBxowZkTt3bly7dg1JkyZFdHQ0\nEidOjGPHjiFdunRxjhFzaQLemF28oJfOm4euIIvlv8mrVLsdF1lPslifVF2ZbJCtux5zsJ4jXZfY\na7shKlRSTbdExHyYD8coER7xmRnGzlG9472BvC+c3HEvgH37AcrGoVOPOZ36kvSz1wP50hO7lC5t\nDRM6dJFnLdCtqVkQb8I7zeqiWrIca5NvxRskldc6m0BVvbDYIC35Kx0J5I5OZ4m8LR/JjqIoWgfo\nDfItzSSDDHeophOr74YKSvemuENHsurFHZIhSP11sMizRv5AFKLrr8mT0Adyg/xQK8EEu6hyVGnK\nQjDjal5kmO88tE4UM2knuOk6S6bsId4s6aIu9JX8ESZjPtvtd6UrK1KYvrM37tox1WwsLnljChdP\nA+C2NEdyx4xEZx/6Vh0nty4XX2+OhjwLrhd/sXbcoUMJR8j1mU3OLXXBen5KoD8rBa7ytTkcZ4ky\njVuGDq1btw4TJ4beBU+ePIkLFy6gZMmSWLQolMK4ePFilCljucQ8PDw8/rdwyzfLBg0a4JVXXkFU\nVBSio6PRp08f5MuXD926dcO0adOQKVMm1KxZ83as1cPDw+Nvwy03y8SJE2Po0KGx/j9p0qQb9Pbw\n8PD438TtSXfMHvL90Z5FcNkrYdoe9lO87l5TfX+i9YEcy2fmpIP5GG6KK6adwLRzhcffTkjrtCfi\n5N1i1hPa+YteTOXSa652mg/4if2pDxuvRPLv9JlUEkzq34T/LgKhEhwaizTK15brcKxfLosSWYm0\nP0knwgHDOBMNSw9o3U7pQruJ0/aGP2LIR6WmZvmhLzSr/HBBBv6SJt3BBJFt1tK4WlFRzHGP+CFg\nP+nf4nQtN7xeW6dfZk3FKZZpT/VUug3EhdheOalT+HA3O7QLmSJs80uGFpBxJeFIKYeM3xcPdPZH\nhH1lmA3/p2G6YNmTRThV05Dqi1oAwGn6UivNjxGSWWe4GWeQuLH2k+LEu09F7ZbHYvlYk/anWhqm\nlo6L0o0Xa3EOyNCU5z86WY/SWbiF3z5WSOmuYKPu7Dg19dNe+lfdr3/gOSZM1ydCKcOs8kSxHvFU\n4B/4TjmdJ+lZhzw8PDz+Ivxm6eHh4REBbosZPjH8HtyMQnJRvBroCqC/6rtVyHmf14WOMOruQPwq\nu172iur8st3X1El2mvsUskiYZaTZIPo+bK0eMH2Po6xKc/08F8hCMhMgagyBl4TJNgJLwmvZCece\nhDaoTRWsVswklKqc/p5LF6UDBndUrB3Icxfp69FwLK/nbFsbZ3o9vJ7Y8aKSGXcy6RiPps76OsT6\nem5QmqEDmXDXRNzcFESk1/OGeWyTmWpaL3L98WeMOfkhBBm007XpcbeIBzp5UakoIBwOB8E5Lqjm\nhv6k+qL9Nj5u811K5Y6y+QrLvdyf3RkEXbfePa3vxythMt4BRIAbo/uKqub2A/4Zl2NH4U+09ozw\nC9hy62etPSsi+1w4jIg2EVxBhzy1WLctif6sUHZmJHqu7iqlSwxNxjuOOCSqfF1d+G2pCNElw1YE\nhF1YN4illhdkm1Hl8Wa4h4eHx1+D3yw9PDw8IoDfLD08PDwiwG3xWeJC+Gf9JItCctIhvADocIB4\n3dn31LzUcqX7pAYnsl6A/cmf8ySdW6Z1o/Vy3AuhNDpCNJzTCbAkU+PG60pfbt15bpzUSbUDRDGn\nXsZLdK/xmZyuwulupxYMDK8/Hpy7Dvn9Vd/4DKeKcU1UCz5padIWhSvO5V6sO2+ryGMan00nhK7H\nYXocroZWLprHoVuVSpqAlO+ND/kOPrb0tbpKR2M53dA9b06kiJhzneQg0D7UbVP1cXkamHx80XSn\n1ysdUpxhuZZm8e4/+3Ag94FmCq8ZjkGbRaH8bslr8KT17wq3NX2jVaqnKSaX9wj76cfHu1vpSl7v\nYQZ6IzT+DcK8+mVmeYzO9sTxD3BT5BDP2C/mnL6ppa95/lmsvzucwkk0CM51B+FNXib05xjE3tA7\nzGXLAs2HsF9wJViQJJl4ReseOBJKcdxF92PlW42ULnG3twL54RKZlQ6mKJyEf7P08PDwiAB+s/Tw\n8PCIAH6z9PDw8IgAEfNZ/hW4pCFOOqKQXFgUebfUSjPAld+amgKCF2TW2LhTSrdC+ikHGD9YO+0Y\nGdkuXDWPgHue1qTFbgYfu8xp6jkSPHgXxugKji0+F2U1Cuv59q03lRhdEjHmD2FpKQhPAM8waacz\nMXhpfhZ5g0OGK936Cbr0XXGSbRNLKJYXZXgorz8bw71GoHnvaiW1ZtmEklYz/q1jd4l7sDyXnl5w\naKYVvJsAMH4l3+Saj8sxCe9V4OPyLIGG8X1dlmGg8bsqXYZFpQL5gPBRAkACwXTWYZauK5XiV17L\nLDgUJZFXa53ISVn8wPB5kjvHhx19Vum2JGFKMldUDxnjo2TwPR5rNM+L0yJdQBXug5tXsJR+ytLm\nlMoZv+jQquK5XhDzTA8C8CacZEV7QEdsphLDaII4YJ7xUdbYwItYWsgGejIxoGu2V2kqEefRlur2\nqTmO227V40pDMCSeAv7N0sPDwyMC+M3Sw8PDIwLcHtYhx+mFzjlIoyQ5aRPpV/dxILeCZgp/F8y4\n/JA1EYT8tn1bJ00nl2dKUwDA1oaxM6EkG3Whr3Sa2kacDuQvzGFPifVYJhmHS3o5YAbwnmFLYuAe\nQs/sDm/svbmJJJG6hmafvnPuo6p9qIMYJTqFnn8cp5g6dFA6mHtlFn5TWNXz4tBx7xllfNG7mZlD\nWmzL2Q9BVEyvh0yMx8KSul1YzPGOnuMuJv3BXvP4pxdz6LsGJEUo5Oo8LUJSVwkXwCFZ7Uzf0eqK\naIpDEmzsDZ55Ruk++JDpxxPBsMGbqxxz74jexuz0+hwPisiu7105pbtHfK5+XTde6d7vNI7nr6/Z\nkqDJ2HWZlW5NQv8b9CFc92eAxR8FupUbNa1+qWPsQ3nIrPtHaKwT8iyjaw0uwZHVpE3HeEUIhApO\nlxVZCna9xXqkfbqjh4eHx1+D3yw9PDw8IoDfLD08PDwiwG3xWeY+HfJLbE9JyH3aYbtwBTrjNaBs\nzJTcY99TSjfIMVM5VphauDOE7+O8ZjGPP0SPc7V+iIeNvpyD3MZltl3ENTgT11BQ9N3kNGs06i9i\neYqmpLKO0U/FOUclDvsILxLcnU6xshGmqeO+QP1ArolaSod7tUen9T6W34VO98KqTwKx4bO6FO2n\nu+qEhGsEN1iz2Ffrwj6i+bdwZ0q1TVibRuyXq+60Xy5xDz7nGW+Ia0rvG59lGnXc7DO/qXbNlD+L\nlg5dQhxuYaWqaM5qsfbnPp6JVV//alI6RRnjek5XKf2sKtO3xWL1/4pT8YZA+/M7288KxVQxHQQ8\noodJu5ap5k6SDo9DbZG2O8ucY2++IP307cds6HPc8KAIrds5JmZRoedduJSdcSdnBIfOHYEJq4Mu\nt4tSwvmqizsipZA1sRswTDC3Txk/W+mi3hZ0gls1tZ9nSvfw8PD4i/CbpYeHh0cEuD2sQ/eGX+33\nUUjmekFwhU3fb9i8Ogr9ipxBMJm8TZqupdRwPo0i5bRtVbKwrpi1EuHsFyK4jvq1/5c/jgdyDsMi\nXaISj7tqnlKhueMMngnQGQOmXhdGyIJhTcJLQYgJnByf40B0U8e9MkWsp6E+x2mL3lHtwZV6B/I6\naMZ5eVZJoXEBIaZuojmxQocoTadALnJymNKtS2Ieo0IcHvL5908oVR3IzKDWSicfR+daiv+Ph8sh\n1iMzRIBYfgDFOhRfZ760u8rsPZvNMN8KFqrupENeBoXZ+Wk34O4HsEdnGEmU6cG6Gibx5nUh/4GH\n9RwiWGbNaX39Z6VMqdqudYg+iMY9BZdgnNJ1Lshs/LV66iJg0U9zkTB9hgZP6xQy6q5t/QVFmZ28\na5iZfAsR8jmHrWBXw3nSbpC2VbjYXZkLR5SuxUOGWWkSF4WjpqWVyq0UVEOGWArLw/eqLPC2eY47\nCrn6NqXC3NzeDPfw8PD4S/CbpYeHh0cE8Julh4eHRwS4PemOz4ZDLj4guGcdSPjsWuvCf3hP1ly3\n2X73sphon1ZdUumGNfT80CllnSnETj4YG2KnO4JTAwlnlGaimKO5OVB6V+e20H5QN8GWuWfsLBAa\nNMcm4JeCwKu7sge6KT20P+fz55mG5uk03+mBLOvNGrGiR+po3aznArGKqWn/ZcGwr6c2Yvksr4op\n4lc2J7JQ09HXdYMDufZz+5Wuocg+JexRugcy88CTD+0M5NIguCc4HOjlJTqFsLBhVopCXOAT6f25\n1gyos090u0/pYiKFDhMhs3M4LF41TAQUGotLN2zuKKXrUuNEILeArpJZBiNF65zS9TF04P3PhnzR\nlGwcXB7N6k7bBWXX+9mUzu3jVNlHtmp/9i81mZ39t2d0VUyXR/9OQFU57MwNDrElheteIp54dlaJ\nNGUAKCY/2J9pRqL+dSardh/Xi+dAMqXDOyI8rC0MOHTIbiTSF56IdHxUtA1dEvBvlh4eHh4RIKLN\nMjo6GhUqVMDMmTNx5MgRNGnSBFFRUejQoQMuX7586wE8PDw8/uWIyAwfPnw4VqxYgUaNGuGHH35A\n2bJlUaVKFQwbNgwZMmRAVFTcRk+jTaHX4E8KEhptcnjwIdZ9i69V32o0JZBfLq8ZUdqIrt3Mqu+T\noRv3a9Pmk/MJVJsGpQYANG76G5yzwSN3spheE+rSMZHdY+ZPDTZ7fjOpPwfxsmrfIzIYQGHmGFyH\nQ7zYA98M5raRMZl1UIvuK4NMkmzvo3SHcofMkp5EsQqW7ZjH3Do5MUjpqmXUQUjzVohrtUuHNaEi\n20yWoSm3WOt2qSCgalpunkqnD1xlQkAcBONtRc26hMXshiDSB1Zx7L5YuHi3WbfO4KHnOTyq+1hd\nFWwQ5eG1xNehU7jGa1sEXc2sFtiFdOFDQ+n7jKm8hzW8nh56nL6DmNS2r838wc2fYxL/sB4qZ/4j\nD50QbjQH4X04xB/AulAoph0AACAASURBVF8r6Oe/ZwkuWEjWRDZz3iGShN42yT7tlJfCfFZjYsuI\n8CrKKF1/t4IbDfSYNOUvhA7t3r0bu3btQrly5QAAa9asQfnyoeisxx57DKtWrbrVEB4eHh7/etzy\nzbJVq1bo3bs3Zs+ejcyZM2Pw4MHBBnngwAF07doVU6dOjWsIHLy4BVnvzBdnHw8PD49/MuKswTN7\n9mw89NBDyJo16w31kf6Q3n1niPTCm+EheDPcm+Ex8Gb4v8cMj3OzXLZsGQ4ePIhly5bh6NGjSJgw\nIZIkSYLo6GgkTpwYx44dQ7p06eIaAgDw6UMHAACfUEhuPOOeQPfN+7ry1df7+CY/bWoHzdzNRcI2\nSgYiAGvEZc6BX5XOZsbd3TS0mTUGQDRG6bI6TlU8tPms0rl04kJW1Lf193O8QTv8rHSFmqRVbRy7\njhuDEE985t6ao9nAB81h+pap9iEzm+eLYHZo53To1O7LPE72ETp0ImbUngAwT/PBt2j7JK90TC+l\nc26nal8WO3KqCc8r3Y+nOKXVta+vdN+NZPZx55iBh7AEX57kxFE68JE6zk0yFczaSqp4rXrQFQvk\nZE6HHDUXoUwLc9qP7odKdmP5iyav6TlIsOkMvqorhi0Ww3bCHUp3YfsP3Mitma0OmTmy7OBNKP+g\nTkr3Kjjspy9phir9vaqZreSGSFihdZnNAkTUT/OYwygk3wfO8dzXW4cHYUtGlh/S4Xm4or/Yrom1\nWjZ67OTibp/P0exZrvNPMctB/3qaRT1fUt5jtpxIbEe9KeLcLEeMGBHIo0aNQubMmbFx40YsWrQI\nTz31FBYvXowyZcrEMYKHh4fH/wb+4zjL9u3bY/bs2YiKisIff/yBmjVr/l+sy8PDw+MfhduTwZMr\n9P5OPx+Gy5UZzXakCnQ7B1dRfVd0Ef6Mpbpus7siSkhVNqbtnSIT4aJZwJkHVZNS7AgLgHPGRIDM\njNGkwYtEmbJK5ihIa7Z3Ra1LuFh3FaGpMdxANywQpl22QAshG2dTM5OlNDEbm6lL9+lhFNPMWu3r\nwiOPBevpYuzXIappSksd0oXPKIvM2hipdKvBbDXFzRzyq/eLTWK8AoSDou89awV1FQAU66vbYKto\nu9NujwmDeZyh/YwfTvjBHBpr3a/hYnoZCTjigIyyr4GwplFMq14FM+lk6LZa6RoKV/BdItMkBF35\nLQmFKKvO46MbZKLxP6LoE6X51DEZdLvr+jkaHY+PM3zCWI3UZoa7RStUt5voMpxLiGKipnpbszZZ\nPiwRdDGxqVGasauB8FkUPqkdjHcSu2VyuoFK91D4s9r+Rp8rbAmkfNA/PG/25L8eHh4efw1+s/Tw\n8PCIAH6z9PDw8IgAt8dn6XR8GkrzlCd0dAKkJ5J6mPAYQdy8Q4dnIqdiHdIsJ66/9qel6hPyt/1O\nBPeYnqO0CIFsvkCHMVwH+5deaZlH6caNZ4bzHkt0oankn+g50grWnXkUU73tFIBUKJaCfXFrz+iw\nijHCiXncxG72O2cCG/aIUKaC2teTVzCwbx1XSOmmtQn9rUcUK64OTwt5pmb9WfuYjrvMEp+ZdR7/\nSg/z8wPiZuUyPsPdrPt5O+tymfWkc4nUccdxSbVlbKlbYq6/CM86Dg3lCnaHtZIyhccOewRVATVN\nh57LdQ/kn+uY6yiiXChptFK58yKUhTTvwgNO+/BjQuKICMX044CF4vakiX5W6T4cwiFQz/Tuog/c\nwmxRT5fTLO6VT+ZU7ZYk/IvhRD4qQXCrHCCLlMUVy/ndi1o5S7OhYxiHlsUuiteZdQV1nO+3d/QF\nADy6PnZ8qKTHp/yRb3/+zdLDw8MjAvjN0sPDwyMC+M3Sw8PDIwLcJp9lyKFCNBTOvYwUmB/ozpBJ\nN3IiuM6WHjxfguW1k7TuQVFBTodu4vOi+hS/GRnyYYwiApz2w3wDTttb/ZZSoYdwRRoVupFMTTRV\n5WPFwEkwo3PIBRZn5wAdjAfn7Vh5u5yamBCacf1KTuGL26H9cpXD43zJC4oIticRp7Q6d0DpKoKZ\nw7cka690P4s4xxTiHIlMKp55NpzOqASG4uaQj/xHpkqmcNPVz6vz5hFOhST6CM41AcDXrtwD2om+\nbJesUmgWQyKWt2xxrRKhna61vsctoDEB/FtAOnMHTojltNCE62jMjwbK6QKqqLyX78eX27RfOPa7\nFfvpPwrP34QIHzmHJmJ+F0+fR6dyLA+zT05zM8NelrtG6fTPt1rybwrdSFeijAERoYi7U/3vGbCf\nuAMeMv1N6QAB/2bp4eHhEQH8Zunh4eERAeIk0viv4cRQJedIe1fQXA9t6jQQ2V9Tq2gTpWoKJhoe\nX0wXbs8MNvUsJdkkw088umVIPwqAg2bLAZjZKMFJzWzUU4w7w3KLCdO7oLHCN5me5YS8zOhoAqcp\nXtjbROm6CdorzWMDlDDLcTQnkIsb3eodKj5G6RbeVAM8KzwmH5TTRiGV0rmZymSersfpWY/JVxaf\nza90KdxPgVzAEM2t2MzrzijDmACbi4n3BJlRK5NuKN0L9lmRnHEviBArADgJyZC1EzWxNmh9WlOz\nBzUZwimWn1fTrDeFHT/Xn03PqHSunmDMMs9YjayazWq8oCG6x9yss21YnvBYU6VL31bcu/13K93C\nbfy5cvbatNGTzBnLZngToWoCwD0uQ8kMQ5dsWC+gG6aa3cTns9nX72pddU7cpXkp9TA/nA7k9aR9\nDevPCvqkFIa+Lw74N0sPDw+PCOA3Sw8PD48I4DdLDw8Pjwjwt6Q7thK692LnMDGMThJXr8EHSrcB\nrwfy46T9MHCacTzGT0IA0kL7LE46Zke3NSs/lSmVv2iG99UPcumKEtCUbL/cr1PK/thdLpCLolp4\nSTeikrLgOZPM1qEqF2rGCt4Rkqa9csQUXTDlGGIov0PrsfPzujFPp5TOqr5WtWuScBQ67WF1IvAq\nHXTo0vHcIuRmG6cQEuz10Q/H0TJ6sZ2Xs/wxDJy8Ng2NiutJbTbnn5+OhY9JB4fjqHQqfaBbZPzU\nUXfwOV/dqmPZphPTAlZ1Oqyrp/gAlH5Ls7h/0kVTrTXKHaLXo+2E9cYvWEQ2VX0OAE8+Gog0ZplS\nOVEBYEFbpcKQg7r9tWSMyxD6PBKegcOHQG+mDKQBOhXUhXj4AQCfGdq3ui/pkix0TqRGjtfneFiU\ncvkMO5SuY8zxRGi3Sh83hrMkcdJsDWk8RZuHh4fHX4PfLD08PDwiwG0xw8uF34KXUUge2b5poCsw\nSpvTKlyhoLGD7n4hED9dqguNRbFFBNJRHHC6fhn3o4A/RnTOxvJX2bSuAtt2zhVVqi0ijCRvfcOW\nNM3wqidZxHI4rogeILhdDu1FdMgok12hlpqohFJluaTjo2Rxp+7QWExcCa6ie9xo2WVy7+/6PPZ3\nZDcAXdcM6/U+1Y/RQpHBc6aZzuBxIisD35jMo31sormy7Fqh/UCv57jvAJPO4kqZ8iZuNstJP9S6\n8xyS9bO5/znPcpiPS65dBISCYeF5wI0FwHaqs/dqnnB1wFZl5EoBaYxGfhp6Gd1G0q4OtyUU5kb5\nTmBBan0e1U7JlmbRHxSfr0f3q3YFYtY3X1KaNV31PRbWLE6FXSSbiZDfOWxJxrrK5iP2mSDTStZX\nj2kIz4FXhrPcd6tSJRGsTBecuceZQmxadPgGGTliPaa4I6bGsRv6N0sPDw+PCOA3Sw8PD48I4DdL\nDw8Pjwhwe0KHYvxgCLNdSzLku0zn+Zyn9e1PmpGo7B2TecwrurrfxYeyBvKdpGMcqP0i1XajQz5E\nIsI6pwvQF+3Gvi56U1d3dMTVHRO4X5TuMjGziXNZ9NpMmMudkKEUPYO1OOd0REwBaGyWsUtGN107\nhu6tzx32Gr/cT+JgQ9DEgTwElDSxQx+J4x54ZoM+8IPCZqSbh0FdEHISN03pclPHQN65g8NqruYk\nJBQ1NS+nyK5n66jDxWg854Y665gaIajKk+qYn8nnMwVy+xFKhTMB4X7Y1y1O0cX6FPE/+gkGKAB4\ntdLcQB6nH020cZJlX3Nbpc2iJzlxiP3Lx2rp651hNscLkUm4deC0wanQvmd5qbpP1vMNajpLLxa1\neY47wn2vAogPuGvMWPSoKVP6LXG4FqmnAXDZ9IZAB3gNL0H7vpcQh+ht7qxZ5Y8MDf2+kZEIrvR9\nSnd4BZd/zeyO6fl86JCHh4fHX4PfLD08PDwiwO1hHaoZNplnh+WVIqyi63zVdf18jnl4ePlUpcPP\nHEZAo0x4zpfVREMXaBo4SYfZSBO26Js6gwNF2Qx3E77QOjHl5ekjjY4JbdGtr1KlNGEdEFFPtYVl\nURtADtFt0GZdFK3HUh5nkKmlhZm68Ng+IVeDxnzw9Th872qt/JUvzvfLtSpN1tcCOdP015UuO3Th\nrRXyIpvTT4JSgdwb9ZXu5ZWPBHLK0sKdQsCVu9lm/ebs++q4ha+ZWCIxfS/D1Dzg7jqBvOGkZhYq\nJA5s2lGp0KpD6ETeA9AKUMblZ8YvUrcdn8erY3TIj0Qb94j+x2ARLtVFL3zGoXJ6DmEypp+dXI9D\nTJTrogwz8iNseh/tePMUuh+WaNLiyjPzqvb7tflpdVfDxMjIHpJFCN7BYfrziBncLj3azK8tbc1e\nZa5xUrTkxlDNXvV5eNx2AGilfsacexN/Bv7N0sPDwyMC3PLNcs2aNejQoQNy5Ah9izz44INo0aIF\nunbtimvXriFt2rQYPHgwEiZMeIuRPDw8PP69iMgML1asGEaOZLOzR48eiIqKQpUqVTBs2DDMmDED\nUVGWdsLDw8Pjfwd/yme5Zs0a9OvXDwDw2GOPYeLEiXFvll/EsB/3C8nx5gWqVsZ94ATl84B2Oqyk\nF14U/cwcXXYHIkH7KN9pavq6GGodAN3qaV0t4cP7TadigpgRxbkftE5SKXV/VakuxzMpZcc4GXHm\nMD7fmYDy9cwgzday280M5E6k1/bWtL6qnRtfBfKvg3TIh+vHbORIqH2Way+FfD1F4eDKQoOY1X5R\nV62qZMKMSBWC+lHPTxwTdfzMYKW7KwUzNB0gfjyzgrD1N76ueaHZxyeapWIT+6l+7GaplTh0pjDZ\n4nLrWCb98RiPEMP5e2GZkuwPdHPM+T8o5ISaWAhbxLMye66+/mO78DPvphl/3oM2NTfcJoLrfNbo\nhJy1lFKNnsLet3rGofw52J+79GPtpaOaJ8wUzDyVoXeY8f01IEPv7DgiGM7dTPNsFOHr777VYXbf\nG79kS8EW72KlJndgkTSzVSUXOud2L6yEQzqlGytO6/nriBgRbZa7du1CmzZtcPr0abRr1w4XL14M\nzO40adLgxIkTtxjBw8PD49+NWwalHzt2DOvXr0eVKlVw8OBBPPPMM7hw4QLWrg39wrd//35069YN\nU6dOvekYW7YcQ7586W+q9/Dw8Pin45ZvlunTp0fVqiF2k3vuuQd33303Nm/ejOjoaCROnBjHjh1D\nunTp4hwjf/53AABE/eDcq0C8foGu1TXdt80YQT7aTuukGY63TOhOlycCkdxXSvWO+Tp4IWyGE22J\nTXBbqxPLs25uhsPpYmpoJcypN82E8QzrTD9xYsM0MTLAZvj9pM8xTjPc6TCn3OCMoqSDzivdj9IM\nz6pdBmt3hOySonCxyYjFaZnEE1SK1TUyM7z1GV2U7V1phov/ZwVhm4vcDH/uRzbDazykzfB5wgxH\nLDO8vNDZj0eoCBZhFRxKgJIKM/yCtrUl3XPcZrh+VsaCzfBF03TISywzPBwtRkRwXYxuiJCzavrj\n0QeZoaoeNHvX08IM/w762tAMnSXl6lQI5Ay9Qmb4kdeAjL2BI6K4nisdhxk+Qt/H73FItaUZrjmH\noMxwxDLDQ2tfSCvhTBG0sfFeDmRrhsf17njLN8s5c+bgxIkTaN68OU6cOIF69eqhePHiKF68OJ56\n6ikMGDAAOXPmRN26dW8+SeIQJRVFnwrJKzlVsU553bfu6VcC+WHzIXugGPs6aZCJsxRMTk3nKhUm\nXzULih9mSidgjdPOt+LFxQa1+pI+bg7P//CTOnZtPSQV1BmlA+nUPFm0z60P/68wwW1woJOsa1Xp\nHXXceEEJRpt0pT9XUFe7pMzith5+WfcFx8/RMX0dm2QKHffRVWDWtDpKV7vB54Fct4Z+bPYm1+Nk\nFOTs5nbgkjg0ketvtKLa5yOC0Xw10CjFiqD9yZky8qDYX3pUg8WEegUFe7Pc+Egypes69lwg5zds\n/D9RTHxgKN0xD54LdNudTo0tKngC1+Yyay3P8aND3tHxgZ0FGz61GKB0boL2PSJeyFqja0dR4E6t\n2txZXBDt+gd+Ec/uUfO5/VZQ9tU3W4Np5r3Kc5z6KXROhwtnQeYNh/BrYfZFvlhF35zSooRovVsl\nWzuRjllBVydoLqj+3t8NDVoeXnKZWFR3r8dRkCCu3fCWb5aPP/44OnfujKVLl+LKlSvo27cvcufO\njW7dumHatGnIlCkTataseathPDw8PP7VuOVmmSxZMowbNy7W/ydNmvR/siAPDw+PfyL+loJl6n0+\nVo17GSqgMVBk/32srR5Z2+oG0KZGjo+nAwB2NgISX9GzRAl/0iTtTgN+F4utrENekJzNeYL+MctR\nTtUe25TzCCt+8DQAIDvNwB5XB9mJTd285gJsJT5p5+ZpJUw1qUScNrr/kk5T+3UGn0eJOmaSsMuU\n9hGQwOiusOhGf6l1wx5WTdrLfuzT8fT1v+v6Z4GsPU3AUOHBOJBcFhYD6ognwnjzUM8s9U2wX6Y/\ndN7iOZG45oZrfxb2s4si1whd6Wtw/1ABseq9O2LeayNQow8zib9knuPhgobo7c+1rsNwkcL3rJ7D\ntRBpm24l4kLME9iZKHZYzXLhlnlUu2wapuMU34LHdDG7bmIc547qMTtl0O1h/ED0plDBvv4g9IHD\na2I5doMpT80D+R3otNWz5jRkPYKxZpznkZkbGXX+7xdHQrM+SYjlo8ksPD2HrRfIsw55eHh4/DX4\nzdLDw8MjAvjN0sPDwyMC3CafZegvUUh+7y32IbQyaXNyNWuN/6JPAi4Ov+jKt1rJBfNQ85RWzX5U\n+wzx7Y7wXIQ0Zo6RxL63Rk6zgY9cyosr/PgrSlc8M5elS/yrvqTXvtOTFCnDjtG7Wodizr4aBzzR\nBljyrvBnmTvztig22fFerZxiLsdMEToyfehJpbuakOPlEmiCaVBMfN7LN/CDTRRzNtM+2zJjuqj2\niXZ87M/QNGRUdU0g32/i3OaLYLrLB8sFcgH6Rq/Haf9ZA+hgxqkilOnHs5opuyAeCOSPHtAxuc+k\nFvT01EbpsDYUuhXD+C9DwJIVvKi6nu/EXtUBC2co3StTmHQmdhRLaiH/rjSWVXwMkgAAXiCCu2xG\nSsQhcC+ceVGpxqS4N5C/VmR+gKz1qWs7AsM7mX8MRSwE10aUSczaUK+t/etPB3K3ntqhm9p4OHsI\nf2NF46kuuJNjotLk0Mf1dKFY7k70Khwq60W6jHwcJivVSe+z9PDw8Phr8Julh4eHRwS4LWb4HeE3\n6WsUkq+LAk5k8jucIAz6+TOlQq59olE6Vnn0m85PS3Q7Q4VyAICj9A2ambCCieJyODdb6aZV5rXW\nX9gZGhzXVMNo5tbQlcdqzeV0wxg+oJiwqrhuhlxprH6vdVDNn3tx1tBvTutKy2JaWfT5v3qQgvFt\nuiPhftHQRe2dK63a0WKOxFgIDTaLUkGb01t7sDmdcaGYfwNpkzWhKZh2xYQuybUNN36Zl4TPpq2+\nkt+9w7OUgWFfrxEKHaI5BPekAy7z87FxkU7MKCRueZKv9TBbBQnVfcZ6Tv//2vvO+KqK7e1nbkIo\nAQRCQHqVolQB6U0QRFABL0UMvQhI7yVAAOlFEJBeFKUjGCnCBQkoBJBOkK7SCS10Ql3vh32yZ62J\nCef/4k30/ub5kpmzZu89u5zJXs9Z61mTNb1zuYssJ6fKy2vceYcTWvQ5lYt1r35m7QqfQYLFOX32\ng/z+daujv5sq2AjsCpHPEUcMmzaGCH2VEqXWyNDqV69d0bYIGVamLhu8HE8HbSfPsSf7rk5QUtnr\n9B/XAQC5cwBoKjPqQr5+yW2nNr5IPeL5Bto3SwsLCwsvYBdLCwsLCy9gF0sLCwsLL5AgnKUZO6RI\ny3dtUFJZqKZIhTSUbPrp9qVYBdrYdo2yCEuXpf1E/3OP1BsRAXnkMbIy9ZILSqYQPkA2t53cFCmj\nmm7TYF0w1igueZepufm703aUbErV0bZpsvAlZkKfdL5psiphvzNyLLiYzmBDB2+cVsRB79bCRJjj\nTsfkwXgAkJmI50uSw02itEbYYyNyC8dZ+4Bh42RbJ5buaM7nX4w/BVCgq5SdOcp5uhIyBKdoDx2e\nc6hJZWE7UUvHYOUzMjoJMaFE0wF0gFI60GavUd2yBOlcWVLNhC2C/N12ISV5wV6v67CiTUYqbpnD\ncj4LyeEQ72ESUhj3qhbp0K7IpTKsa3tjJkM0p4WwRbap4rbr4KCw/YKLok9MdUl5RPJiwgM5w77s\n0U9iu4ZJGPeqdghbiYZSFq7VUh0vN1LJ0o8X2PeD3okQtiyqkDOGCHgm1YrUXB26tL2t1IE0BPsE\n7JulhYWFhRewi6WFhYWFF/j/Klj2f0VMQAx52hVZuEj21DIqn9iv/GqnlAS5mG8w4sJw5rJnlyI7\naI7Oon+EFz77TQqqRjGllz7/kgrTyQ8whZhWBYWNy+S+lg0Shhsq5GZbOG4YLQBUi8lIue0N1/QG\nSf+9tdKu9+VP5D5LyS5+CYiHXdm7jHWMa0pxdrCzmr7Ggfsh8dJ40WUCRaDjMuRFcV87ibGfTvp5\nqLqGq2ZnFVK8zU+/BY4CuaQbrtBfd/Ych2HUaCJTn/L9oBXnSb0nbBs9G9ag6dioZgDQdMbTM7IO\nFXXTrnesLB3+wQTJQ/QL1/fjyxXy+pc19vQgxoWnSYgkWfgrNVMMoq1yP3+QDgHKyQrkAYB6k1UA\nKCLd7m6TpLIPRrL25Jh0np5AxgnwidSmikmk+LEQv64ur9vepfIc287Sc79gnH+Gd9h+PG73n0H9\nK0x+MEr3yz+SawN9YkOHLCwsLF4IdrG0sLCw8AJ2sbSwsLDwAgkSOuTrSYV7Qj/DV1VAGcZZ+VYp\nJsaGhWkOZ7HBUTQJ10xYsbKS7JLMT0M5gfKSe8FOJ5SBngBqoFEXcCRTfaF60sZCRbDJCBCqzpRN\nCshCXzgmVWfo3aJuu0ioE55xCIQiUOjOlF1aNZUK13eP13DbKT+UYRQwKkF+rrQ6eBeYleQZ2bNb\nptThF4fro453YilMK8aM3qNfhO3RshGin2ayVlZSOwLlMRjBauxG3PGirH2ACKiqrRvDmoOj5nlZ\nXIyysnTMUQPkMfqzQnRlrwgbF7k/t1qefwwV/WeK/4vMZ5W1M9FCYTvqo8OK0jyT93gnac462ih8\nV+WkrK7Yw8dRk5qQm2JX4mRIjjDRfyDST40SAyP1fvrPkqYVw2Wfmuqwn5OVQpzG1g1A5ZpQ2za6\ntt3GfN7UjzHubpDfo/pqrOivZPS2+tk4x+sstOyOcZAZnvRnagQomYCsoKsMNIYsj7OYWiAu2DdL\nCwsLCy9gF0sLCwsLL5Ao4r/c2XoDH4qxu7HYbdNUuZ8HLNg+BWToDkXosB414hu54V3ZxZcfOduk\nJagCxqv9L+xyDJKZDzkm6fCYM6ooJHS2w9yKYcKSbpsc+dtH2mXtucgRn3VdO6ZWc0h6XSi8O1R3\n7soMImVcLGIhISq54bI/+Mhtps8ts52unbrv2X4KjjWX16YgK+Fc1qiPFi7Lb+OzUvo6Dv5c7udO\nIS3+i8VSGBgsEejeQL2PFB0BlVlnaVAlWW9bzfhI9ClNJd2J/FiOfXkB67UQtsPsWeltnNPZ4c6z\neSS4MV77dAmODNI+oqozTQ4W9eTkV2wptAxWc8gQKCkhLFFnv9zPWk9kFdUBVAt5jVsxVmLeIbnd\n3SJ6bEpDmBnfsntTX5r2IpPol2AZPUk9h4gGkAxASUYLbCcjFSqfDiWbfDKVMHU1dHqb/qAftE2Q\nMXmX2DOWnWSW0Jk9U5wGLQM8QsAx8EGI234WqzS6DR2ysLCweCHYxdLCwsLCC9jF0sLCwsILJAhn\n+bWHvwgiwtdKoSmz0Y18YuzL6U647RCDP2jPC8DHOgrLKWxrpLf5FJD9UQ6/SWkAJbMtQcv0nsuT\n5MVeHqRlX7791LxscYdumOCCSX3h8HBEr0GpI0AwS9v61NiQtFoK1BxhGgGpHjQQuhDU0mRyN42i\ni+tdihJVgKIfPZ/vjx2OwlLYeIiNA6mso1hy4rMiRgjSoUlue9nObsLWsIxWb2r5iS40Nm/qdBHJ\nZOimx7r6XVhK6+eQYVX8iS/st0fYDj/WoVSPIMOKkqKWZ/v1UKoWwLi4wd/nFWOHPmLpl/+W+bcK\nuiobQabpFSXNyx5SksGkxlK5Xw1x+H4qQLHDvPhJmvdxO7OFShPGiIsssUXyq2itq91RhdlO46tX\ngGYnkX6h/l4PNXbUqRE7xlJZ+A5kxCupk27zW0he/hjpnN8BG+TzjwJ7nd3lOAClZNo0ag5xm/kM\nEf/jlrO0sLCweDF4JaQRGhqKOXPmwNfXF126dEH+/PnRp08fPH36FIGBgRg3bhz8/PyevyMLCwuL\nfyie+2YZFRWFadOmYdGiRZgxYwY2b96Mzz//HE2aNMGiRYuQI0cOrFix4nm7sbCwsPhH47mc5bp1\n67B7926EhIS4n7355pv44Ycf4Ofnh/3792PevHmYMmVKnPsY5+cErPV+dBfj/FKiz+PXmVXyMN2Y\n1NSkDpKzmTNdt9vERxGaZ1RMqor7HhwNAHhMQBGzgiHLzIvoY+yIh12+bUzgBybbXF8WjqdvZUql\nYvsdOtbZz2AiDFMKQ0RZQsQNY2pVDHP6RzptbrmfTCl9Q2leaN13ckeB7zvnQVQbUDJNsNW7muCd\n973kmhSVFP107s3i+QAAIABJREFU6prbvgGZCvkf1jZYMEiqjXFilATN2QX5soWx4WZ5Huo8k+zL\nLG1nL2rbJePxzxTOLno5yXUXJIfr/hVOLc+jbD4nlJQxz0eMizQ5w8ss7jCjkW7JK3Hmvyltx//8\nqxpLRR5GdcvMcvx/wjRnfOAjyRlPZumn54zDzeog++10RiOUR/ydDhFUEQUcTsMmI8/jkNIcbhHq\nL2yk5Nib0LxkD+MLMS//K26747GTwjZ9p0cWsgygolvIYyTXfLpZTzW+1fC5i+WsWbPw22+/4ebN\nm7h9+zY6d+6MHj16IDw8HABw9uxZ9OnTB0uWxF2K9uqRXxH42qtx2i0sLCz+7vCKs7x58yamTp2K\nixcvolmzZuDrqzc/pi8o7rzl2DdLB/bNUsO+Wdo3SzHvv/Gb5XMXy4CAABQvXhy+vr7Inj07/P39\n4ePjg+joaCRLlgyRkZHIkCFDvPvo89gJB+ntaRP7aiv8KsZOUnEnfA2dodskhbEhNjOeTYKsbqY6\nxOzoJg6FyRQuVZkpFDWXc0sFfZPvrDfUerqyAlqfG0pG5sqWfJDbHJLXsQ32tIWKtLGX6tApdZuU\nXJA/+nyL6B/sorVeVCzdFz2fwPfi+pIRykF6A+m/1yme6nuZUkqQfRXOblB/SY033MLla9IIm4L+\nshD4j4aExYt07+uH8po+O2/QQAH6HwRfHAGIpXtlLLUenWNKJBWBVD3PQrZqMo7W6yr+zxefIMPM\noPj8jPt/ij1XLxvPEccBWWgLyaXS0jXEKCsRWhjiQev7s2MOlOf41iStwL9H1ktD73JaIWmwCPID\nhleX59FuONsvF5Y6DJQ7p+/jDkOMvzALjyIkF7ZjkFX6qh3TOc7BRgQgjusFcrZhorKeCgi0HUgu\nSxUMYmma+WJ9y14gdKhChQrYuXMnnj17hqioKNy/fx/lypXDhg1ObvLGjRtRsWLF5+zFwsLC4p+N\n575ZZsyYETVr1kTDho5GZHBwMAoXLoy+ffti6dKlyJw5M+rWrftfn6iFhYVFYsIrzrJx48Zo3Lix\n+Gz+/PlxjLawsLD430OCVHcErRRtlb2B7qbtIscyOamnkDyU7y967Nk2xiEkZSdgkrhzvmC2ypfk\nfkZrDqPTNqnGPq0224//DGHDfU2odDRoj1hpg0IxivF3pwDVno01DpE1nh9/2raVcnIoqRWny8gi\nkQjfozeuZcjHqfS6EueOpU+ljQnQtzKOr4xzprKMwzWE45ew+3q6tHwEg3e1d9v5ST8nxwG8HqXH\n7Vpick3GSS5j2vnVSgjTAGhOWWUwpPau6B+moAyF92YspTF1XmCmPul77aTUYPGeen6nphcRttvl\nD+lDIB0krutm8rbStEAyc2EtHM7yAwAL+kfKofGR3ywVtOQXpk3ztC/nf05K7wluL87axdCuvr7+\nOwpJNXS+F8ofLWwFG8v0WxQY7TY7Qqb4ZqS5bvuxCjeO4fzg6MxQ/og0fJCOCz+TOxjewqY7WlhY\nWHgBu1haWFhYeIEEUR1a5HnvbkJOOym0q/Nv03/OyHy9K3WMPTFXiwzOtAuLs5piVMHKuEj28zhx\nZrSdYnkWirkvZNjGsSiH8kasQvkgHrshX/vvyTry8A88rzvXnbhSSkdQNxQoAHFCTiePNB41Yqm4\n5/O9uScd5jIZMj6wa0OPG74UGG3QB/3S60dFXYsVnyXwEjPfrCPTYZev0epJDbFZ2PZ8X91tl6yj\ni44RmkLxK1DKCKvp/6PoTk+qx3b4XCpbYYNWtjLvcTZ2IsvoD2ErixgVJALkbABl0AAs7CYdZIDi\njYlsyx7m149RTyQpKlINRH9utuUAgNZnyZwNZjLVqU+VlDwPZmNTQ1IUH0KrMJnKTm8bl7E9e67r\neSiSGMX/amzcNkiKqBT5uO0dSsY8xVKT4mr1ZsUDLbiO2lIoHWtidkOEk8ZznI809VFG3RC28HiW\nQ/tmaWFhYeEF7GJpYWFh4QXsYmlhYWHhBRKEsyzsKUR4uKbTjkjGOITK5oxYVT7MNIzrWLunNFUv\no9sHZRgBrpgpfTH5t5NiR1XwDzoaVhZmYV60UpxqXWbkIv9H7udMdZ1Um1152jQOUL2hMB5xgycj\nGom6JOOMFIvzISw19qOrHaqJsioiulf2bEOxQ56gw6PmkSxc31JJ6XT1Jbs/XeX9qH2znNtea1zJ\nXL/qbLCsr2oechslEbzcR0bC89cshQ0w+N2cBqH1B8s4M25kSp8wt333WRVpXOpUIqSGt6GWpcaT\nRndck89oOVT1Yxxy11vSOIkp1RvX+KWzekK3ZTFDoIPxHLtaCX9yr1hE2tPga8IUpnRKZzVMlNs1\nZWrkC1vJ40GG2dyB1h9IFZNS6ynh2pIrwCeNENvNf6jbnYzrPy2F/KDWA31e63vvFbZM4zTfegnl\nhI2ww52P+k0eA3niFl2Ibzm0b5YWFhYWXsAulhYWFhZeIEHc8BilLfLztJNqpZn0l6qIsUUyfee2\nN6OTsCnSYRVmyMfWctoNr7xjp9yuoRwcssz5O4Rihw5t7t7DbVff30Iaw3QmRlbI7I5ztMptp1RS\ngeYehog+immJrOH7HV8qGB3wKaYjI/T5t1OymtIp0u57XlUTEmdE7yPkcNvfIK2wDWisU2FGLE5t\n7KeX5++gWNJir+TXEjWnjhuZV1MMFzFSS/I8HC7Dk5LN1qpLY9rJeJC+mdrpTg/tr1GvFfiNzSeP\nUczN/5Wkon+vkY45+azXdWHr3oOpZJ0yJNIq1XCbwZDKTp++63HDQwnqPYXzofoaZFWyKBrAU8xe\nEZa6dM5tr1ayCFdO9nX8wLj+E4wjdPUMnfQnbngg+1aXNJ7x9dwtzV1cGi/u0+3MsgjYDJ+5ov9x\n6q1ue3yUQ9/0ImC8Am6xL5YUCAQykg4Pq/jSJmm8LbuLRy52211+lVlSV79GnIj2nH9SENRy4wL0\n1OtP43Pvy+PRgjj3ad8sLSwsLLyAXSwtLCwsvIBdLC0sLCy8QMJwlh7KwBNVgKaMcPzKoBNOrdAq\nNHlXPJH7WaK5HozPKmz+v+kdXZ8mTEiWrLr84GFyz3xCY3E9H0DzIiuxWNhEyQMp8IzujCadaJA0\nimTIQzeWYjbJk3pIOASFIgAGsg2lLB5CdfNxPikbnWSoISP9uuZQjWwzYKjmKd8ef0eYfvB0iQhr\njWszg4UnrQlsL2xZZHQKfvpS87bpm0vOMjWbEEEq8qhnWtX9wyOamFxUKLW4V7LgBZCcaoh+sGI1\nD4wnfB47rdYYKWx0Su/ZyC5Ex24OTzyt2Q/45Ku38UVzxinTWDlY6VImhcPkBA5X4ddVKgttgS75\n8cwg5t8xvisPSStEmemO7CnCp4Yaulqhecqsb+4XtrzDdDsM0hbeWj7YQdX0MU97eFAaSFAjFHiU\nkclZLmJfpFf9ZKUCPL4suifYzctnhC754w+3ncFQg08V4RCaB1cThhnXZrDSz+5KI+TugxdRSrew\nsLCwsIulhYWFhVdIGDe8i/MaTJ+T097IDnnZCDm5xQU+pZLLZFZAq5uoEQgQc8PmGe5Ka/PV2lPC\nkXAV7xtjQxe00B2/L6WRV9cbavpErJ1UHs8/y0uif++CzuhI43ERooiQVilEMfGcHoYO6sTKbL99\n5fGrycgh/NhBjz1luCFMwhal5WZoX87ZrsV2PEeRyXxsDshubV2ZUa0xjsK3nSYPcpUpJ6fPxbJL\nfu8BtZMdv6ws9KVoveh/xHZrJNcgG+nsp4FNJZ0zYjjLvHnPKDyXyQkzoQ2roWrWBfLoMC8yRHRT\ns+PfMZKkyrPabttl5BDQ6QPWkWpNeeVIVy8q2Z9Ud0QSdo0fSRMPCaNwWaUzqqymSNJJISfAeB75\n16q4J+RnX2rC67cV9qfWIUEdVTWx2Rekw3W+LSDPqv5xmVF0qKmm5d5dKGk5/sjnPyEz2o7l8ygL\n0adotu+ssPUrofuvUgV4C/tmaWFhYeEF7GJpYWFh4QXsYmlhYWHhBRKEs4yd76hDLsqRTKnyV1p+\n+T+majJT/N47TKqhl7hdym3TcXn45ZIyQcPNnpALAtTVQtKYgSkbzWoiTFl7bXfb52/Jy2aqx3D8\navRfbaX/R6l5VT1z2QSlqgNMOZzqy2OolczWa6uwnZ0wTPRzQKcbvk3JhO2HoowLPmSkO37hpJhS\nh2nwN86p+mu6/V1OuZl6Xfb/M1zzsm8NNo4xLO5rxaubUUqmbHSnGdQctp1RyysWki/T+ynXUJha\nbtbXtQikGn93rHHbg4zQneFMfRtKQbGU17IHpBx++FkWEvOeLLy3Npc+x9q/HxW21/+lebl9z74S\nNpDkF2O4XyJAlTeu6fZZbFw7YbpQ2t9tn9sl02YzsCp5uZFB2AoZqvbkq79YM544c6lAwM8KqMgL\ntvnJonB5R7E8xfwfCBuOnZB9fsqy7pn4JcJ8oqadaQQA6Jh9iZm1+yej2T6t6pCFhYXFi8EulhYW\nFhZewC6WFhYWFl4gQTjLXB568fdSTvuPUvEVgNdVC1vilLDMf5cN/n47JMq7LTLiKtNilejvh1Pt\nLicRfI0JbON7VBvkIQIYvzPGOHwbzn1GID7wSx4THxdTFQ+kg9lI7RDbKdqlO5tlmiCqx8cDSvDM\nxAaTpG3LNKdKJp3IC4XOwjY0RJNGQ0I+EbaiSpaQ5BGKbUvL+/GAxUv+avBQauw7rLPWbRIBfkF6\nu8ffwMBj0Xs9iqWKzpXxivt43qARAkos+zM5pAzdhwMcGbZ5IwitBirMH8mvj+QlAXavvjMO8h67\ndoZEWxRedttnEClsRaUqINRVR/qN6AT2GZzlHPboTIcBn8/cZvKn3YTpAVi1w6XphA2NZCVWGqKf\n1eFDSwIABtEvGK5KoRvpFNfyvzwU2x3WPy+ghpLk82jyF/3XlZ6rIYYOBGoulAYdEiY1w8PnzgYU\nT30FQNDf4+6oKGyf0TbEBd84LR4sX74coaE6KTkiIgKLFy9GSEgIACB//vwYOnTo83ZjYWFh8Y/G\ncxfLBg0aoEEDR1Fg9+7dWL9+PUaMGIEBAwagSJEi6NmzJ7Zu3YrKlc1iOhYWFhb/O3juYskxbdo0\njBo1CkFBQShSxHEDq1ativDw8HgXy9/f0LJDv7+hwIXEP4NMKey+oLnbnrdLSvvM/4LL/hipaAyf\n1pfqzze/lal4OVni2NPXhAnZjvCeDDl5XEkX5Zp9UrpIHZnbsaSdDEdp/GSt6MtQhqKiXaqILiYG\nkvlmcy9q17t1F+l2SS10IIp7ySekWkv6UkxmvKCx4dgYVW8CGS5i51m6H21slswgc0rw6e0yKALG\nWKgjB4VpX1adbldSUCSEx8t1SmPOOknEdtdfk7JH+6ozlfO9/xY2Xusu+W0jPEtMVc470ybP2BGe\ndpJ7ru38SamUnuWZdi9VLkkD9GLhUYdJSmSljY9NuSJpqaNK38cSX0mXmVRdtz09j7z/rZ7mdtvz\nSIbO7azFlL0atRS2YSNkvqPKpq/dOvR32yVRHUez7nH7h8fKa8yZn0pGeNYx4/yJ9OipSp7jnQhN\nGQQWOi9sWeAJSZq9ElXfuydsarZu71hoyIfFA69/4Dl06BAyZcoEHx8fpE6t4+YCAgJw9erVeLa0\nsLCw+OfD6x94Bg8ejNq1ayNnzpz4+OOPsXq1E/S8Y8cOrFy5EhMmmBVCGCIigEKF4rZbWFhY/M3h\ntRu+a9cuBAcHQymFmzdvup9HRkYiQ4YM8WwJoLDH747JfOBu+OG43XDaJTNP1BcPWMd0w3Um0PD6\nUm50kOGGU4wbTiehXpPv/WeZG55dSbWgx/X0fGbnM9zw0fp/zvPccMzn/5+cuRIdgFLFUKqwFp/d\nfaiR2GzeRb1d67cMN9xIE4rPDQd3w0325L7zh+pSrIJlnVlSyHgjg0beKcSXJAFw6sN0wyczN7yr\n/hXzKRFUUp35lbOG6YZLPuXOJpZBs9fIEiHtoic3IicexOOGD3jDGTtiFzCwNDByP3fD5a+40g1/\nW9iEGy5nhg3xXTf6cze8AM2HOh0ghzI3XJluOLgbLtW7hBv+g+mGy8mt4254M8cNr0WjsF71R0AW\nrfVU2nTDWf22SqXid8M/ZJvG54ZPNNzwpJ5IhvO0Em++L6Nhtsyu77Z3LJSi0WV7GhEwDF4tlpGR\nkfD394efnx8AIHfu3NizZw9KliyJjRs3omnTpvFuX8nDS26LaR/Sb6HdlHyQurXQvFx5MioYcmkp\ng2pQtfTiOefbbw2jXCxP/OE8dPkAfGYsMtlVSd3J2UPYktRg6Y8ygwvEQln2zFojbbGkzvq57apK\nLxZVcRCba+kFcnUsqTn9QYowaesqla1Qm1FhpQ4Ol/NZppXbVU0pbYY6noelLqDQSpgqt9PK8Unb\nGbpjmCN6A9j61LG8HJm1O+t8JEOgindJ4bafdX1Z2PBIy7L9kc/gIc9LXnDUXi0D1p+kDUu11N99\nGTkilsda9VMJ20gPETsChJHvKxEdlPV3uR8Eaomy0uWlUv/4yVqNP8lyqcYvYTh9xj+vggNjRs0H\n8soKlgu5vGH7n4RtXtQx3WnZXNjKrGcnpaKEjQaaFSw59E2the5IcYEJ412T8w5gX6MwYy/pjhgf\nQC9sOyG/199wZ5WMKp1Kj90SWk/aQvUbwhbIxbFsT8QJrxbLq1evIl06vaoPGDAAgwcPxrNnz1C0\naFGUK2cGQFlYWFj8b8GrxbJQoUKYM0e/OeTNmxeLFi36r03KwsLC4u+GBMngWeBRmWkxmLBgmMKg\nIdp2vncWOThfJ7e5v21/YSo+QStTX88jeYiAugt0x/xxPtA4xbqerJlVsYvTE5O49rslfe3HzJ0n\nSH9BzdKv+q9VkMopF17dKfpRRcro7Q7yolMQfuD7xrS/Y0ajPhOS15TKPsE/aA5vgpL/2Hoy14ZI\nFqXSxeVic5a+bD5PTaX0LcaE6rP2EamGneaudhF/XrhQ2AoxxmAP5XPbJXAcgeqS289FstDVL7Gk\nZbK7rX8brtZy6OJuZqEv/m3obhS++6yTJxOHdgKqDJqzYJK3ES7GNoEOMwvKJfnthfySpzGuI1Pr\nuTJemlIb55jslIfCyrMO2CFtT9ZqvtF35FNhU8RiZ5TkobbV1Hx/pYGSIkNFg8/jc78lM9F4qb0/\nSNI5VdnvDaMgXWQysu3UA61qHpE8m7DNRIjbnrLLSIwpzRSZjOtWld3kLbOM+9/Oqg5ZWFhYvBDs\nYmlhYWHhBexiaWFhYeEF/k/pjv+/aDnY+dvC0z7OxNF/Oifjo9qU6eW2Z74t6/LVzKbDQdJ9Kyu9\n4ckC3faRYSV3jByyVFpEHLWN8Ax1m42tZ1QsVJoHVHRa2n7XPOWRdlJKJ3NFqTozcZsmbQ+08XC0\nc6biQJtOKPa7JsrUfqk6BJbFOWKQNOGmjLPb2kVzaJVhlBBsqkkzpWSa4CKlOaPfjWvzNB41eFSV\n4SnUiu03kxG6obRStiLJWVIjHXallI5XJQKuXdbq61eRWWxnMN+4wDhEZYydzGK5/NX7wqbUd6wn\nz/8rj4r4dQABaIEbaO/avjS4z8pFdPxepcHyHFUafoR3he06y+2Y+pswYV5VOZ/zP3pU/fMAqnwZ\nYXsIHUqk6DthQ0E91/tGWFcnRksmkz8ZINq4jeDhUoZA0ZK2PDX5VWETDP4FyT2rzFIN/helvyvv\nYb6w+fCOoWyFrjHKX4XQUskKjiNVUrcdZU5cisoL2DdLCwsLCy9gF0sLCwsLL5AgoUOmwO0xlrX1\nVt72Yuw50lKl64xcsHdYsgeRdC0zB+iQo0vXzZQZo5oW7XP+gDDBGDosn/5g08lmwlaKdGqmGXLC\nndB9cpdYavS7svZActLiTmIWXkE7lD2u3bKFJ2XGwvw6Ok2npZI6Q4YzgRs8bchUckELt33r+gJh\nSxPgVHsjyofpxoYd4/HCzeTTlghy22H4WtiqkvZDqddNYVNCYkBnhRBNhFJHWV+6dimNud0Twr2T\npZE98bGyq3inZ11pzO24s9SRoL5QANMpRi45lOvzjISMASrN3lHqQmaJrWaTU8bk8hnTOfGdc7GI\nekAtM0JguGCWed/Yw6KuGzYWEjbR2LD70u5ybEMtzKt48TQFXGTbZiaZCcVvwCvF7grLSZn9KtSs\n6hvhaiwxE9uMVSyGeYsiYOlFeR6NTc6Gz8wWLLOwsLB4MdjF0sLCwsIL2MXSwsLCwgskEGfp6IUR\nTfO0v2BWWcxoLWN7av8si5JVqaCLmYVBhu4MYO2RrYxTWmyQNvedikmE3UhbX3JGN7ewAkmSToMg\nuyoY++R1jn6WqYf+c2+L/t0FmntTylGUJgqCUl+jJbSC06uGinnvloz76y1Vf+4aYT0pxayfCVs7\n9j9y9plbwoZOjgwahTbE3K1yn22q8H22EDYFmVKXlylnt8cSYQvBMD1veXT8i90OPmuaQOARILRd\nzq0Hqoj+KN8wt53UTx5D3dPP1RrI2Jk6jBf7iiQT3AyOCg+BoKBAisuypZfHwBnEBcG335K8uPpE\nPzs/fiOf4zfjTM0kqE3SVvwt3d4f6yvOxqYybHcq6XYbGQ72gxSWQk0frZiU45kjn3iGWiKHmo+z\nTLGKSHKdqoT+jqXeK0zwbRYm+jeyr9OdEWPl4NqsndSQ2mvm3Jtk799Dnm53hG3HZC1R97KRGx3f\namjfLC0sLCy8gF0sLSwsLLyAXSwtLCwsvEDCcJZrPXGWtQlqrcLY2prfCt4ikpbw6Hvd/uUzYUIp\nVNUdkgXfZ9TWp9F+nZSdoh1V5HzKOTxFTNwnB09wlMUpADBlqyttuwhTBmgV6TzUQthOqwVyPiKW\nLuYzh0m6BK34PEBlFNvNe8S284uv/gBkLGGnjtKWhWmPDZD7Uek98XJXgeTGtXmAQD2uneR6ks+S\nj1G6gjpNjY7K8hgXt7MYSSN0D7v//LyICIpVAvV7XcrePdo3XW4g5n5O2qBTKpMYhR0e99V8+gGZ\nbYsmntM4spTwWiOFK1pBDv5/yLF/fK2vx1mDa8zOSjmoLlJFHZ+zPNaIT4VpfmE515Y5nfos9Hvs\n53ggHrntCVVlCY6XmJpZZMW4A01HSzoV/b7MLvqLldbh+9BThdHlcxlPLQXbnGfLtb0py1rgzbdk\nn1HB712Qz1jozCqst1Vu536vCEqZ8uuMKadQY7MRiAv2zdLCwsLCC9jF0sLCwsILJIgbjjd2O393\nvwG8sRvHb2sVngLGT/6I0u4DUiSVtuO8I5VcckOXHvwNMuQBaWRKJW46LhsR8Olj6YYM8tMqNO9B\nqrWEcjGhsXGHY6wn6fb16iAVnnNP19sGerabC0JrKGxgqj/nDRVpke1VXrrWP+/4QvQrBOhrcOi6\nrMpV5Kx2p5HNkJX31HKirQTMNlz0dlXcdhOj1FSsIiOkpZ2okczTG7hMF6IbMUSqbytWSwsz2e5e\nIijuohsqM70NV1dq11CcPQWpenMSl9221HcHYkrZ7SFCSaVQaramgmYseiTGhmzRqkMh8ZS6/Any\nvr0HfV+jzMGdJJ2BqQ5lQ/QjlJIppSBWRFAZ34fW7Hm4ISs4YpVOXE1bT/JgUatkmB1Xo4cnpZEQ\nAYVCAHd9SYaVQWmqaaZx/dsZy5FgF1LIpNpk93V4UnQW4xpf8ByaCKq0EXLFsi/VZrmZTXe0sLCw\neEHYxdLCwsLCC9jF0sLCwsILJAxnOdnDWXVdDUyuC9WNc4HJxNBRFO22Iw1JpkmiFzcPRCTVp5W6\nJwf0cNKfaMJuKFPbLIrtd6Vh+0A312G5ML3jUdF2YBAhskgkPn5bh1KsG+OkjJ0lILsCzjFZrvtS\n/BzJmVK6MgrtwVfmZhK0DNokFBa27jxcxrj7X3g4yw5bCRWNa/wzux+0xuAz6yBu+El+td4ezdOt\nk4UP8fCtKm47KYW57WgY1SZHy4kv7yu03bBNacX9qUJLDQBpPvGYIaiXVGmeOFdGQxPtsicEzhMe\nA3zlmm6kkHE2m+7r+TUwFebHa1uqy9J0Z6Qeq5JUlcbSMlyuzW4npXI2ZYd6LIfCT3PRgfUzCFNa\npvxnJmW2Z8vBpFflvN84WkL0fwHLVezzMwCAxpSH6rsdGFuTjTR40WFM1b2mTNulN4xr9b7+Xh39\n4KgwvRrApPekCTn2OOfxxxLEWio82dfO8aaZ+oWWs7SwsLB4IdjF0sLCwsILJIwbbh6UvRd/arwF\nB7N21zdl6NDkuTl1Z6SII8Ko2fo0Hhrv3UPMcITvPe7Uu4So09uELW1erbpiqqFznzWZYYsGV+6W\nGURH6ZDo/6q0W/wBHP+VaA2UqgPCWn38S3JuyMRDgIxwCLPwmrDnlmPbs0pYM4QJDeH4/svoGrDW\nOH/maiuz1N0TIzyH0QlTW0q5mp8W6LCvpUbmhWIiM6il1YkIg6CYIhPlMaiV0zLQRylWFQ/SnW1/\n5pLbniGTUkBD2HkMNagG5YTAEP0LSj2DKJl1AhKv8A0zGcaL7ICGid23g4HSWOSaMR8P1UIUBaWq\nCVtlRgVtjYeyiuV2csoghbQ1uS+HLmqu6RWCJ4NqAQEtFPAlyzbrKCmrnV9oGXdZZg3Ax8b0WO3B\n6LfkfE4V13M9ZZQnqDfGk4nW1wg/AkDrWdpYrUmG0brhFhYWFi+E55bCvXfvHvr27Ytbt27h8ePH\n+OSTTxAYGIiQkBAAQP78+TF06ND4d2JhYWHxD8dzF8tVq1YhV65c6NmzJyIjI9G8eXMEBgZiwIAB\nKFKkCHr27ImtW7eicuXKCTFfCwsLi0TBcznLtWvXYufOnRg+fDhOnjyJQYMG4cqVK/jxxx8BAGvW\nrEFERAT69esX90EaeDjC5QTVQKFZqD7kl49CjNH6LTVndjm1M2kZMXHQCM8By0V8Waqo02UZOqPg\nkC9/pjqEiazdQ5bsOxeiq8pPlIdAi9VaGbzo7VHSGIsy0ucVcNuJnbmWKiPS34lEmdQvu7Y1segk\nFktUNYWy7fUPAAAMIElEQVQ0Xj0rjxCR021v+VAGiJRprtM/U7xtkJYx2xPFIns4K3jaINuKG6FE\n+5kKFF43+NX98922MsJK9rD93mYXrqpxr0Ya833fCI/6jYVH1cEqYatNOjwoTMnnlqClhh48lBUk\n87R3uOhT8wh5WylUna/nOgeLxVi0b+I2yxmiNzt+ysp6hiIU0/wnpjYPAKqjoStfdoozLohw3lAx\nr9FW54qGYr2wvcLTeJN9L2wtozUxPd8IqzpQT1ZJLfaOvh/UVspncc58m/GsVGzNwqPmymtMMBWC\n9Nj3Dct3kTrsSA36Ru4nJvuZCErNNY7R5k/2HjM87uXQqx94WrdujbNnz+L27duYPn06hg0bhtWr\nndzf8PBwrFixAhMmTIhz+4izESiUvdDzDmNhYWHxt8Vz3fDvvvsOmTNnxty5c3Hs2DF88sknSJVK\nZ6J782N64Z4e3T37ZhkzI7dl3yztm6WEfbOU+Pu8WT53sdy3bx8qVHAqRRUoUAAPHz7EkydPXHtk\nZCQyZMgQ1+YOlsv2V0xF96v8IXLsCb1Yqn7yVK4yLdTAj9cIG83UJ1nRECtSZMQ8rGcX5I5xcVLp\nY7YSZdCA9CH6In9mXObPwBdI+VDBeOjujNGZB6lSdXXb11NlxFpdrw1KCtIAuK6bG9tJUxIzrISd\nl1mwbbFeICM2yfMvVI09yObhO7NQmYvStL+2FHHGz+yLtNoIa+rBbCvMg7Djaz1lEIBuU7Qwa/9O\nA43t5D1+TUxeqjcJ+ab5EcJEj/V9VEmlys1pnHca84DT84fiNAuPgpFthtl6sdwhI7cAOq/b4+Uz\nht68IzN26J5M91FNPSFpQUC2KJnulY9uuO1V5no8Xf+nPxudR5jyc9FoJYV5562ST0Tjb1l4UBvn\nnw5hNBTJf0CVjH+6NJcdZJVcyFTdGpAfSI0oYcqoF0iSAmFQnv8Vztathe0YC6UrzQokPg/PDR3K\nkSMHDh48CAC4cOEC/P39kSdPHuzZ41Ql3LhxIypWrBjfLiwsLCz+8Xjum2WjRo0wYMAABAUF4cmT\nJwgJCUFgYCAGDx6MZ8+eoWjRoihXrlxCzNXCwsIi0fDcxdLf3x+TJ0+O9fmiRbHkXi0sLCz+Z5Ew\n6Y4xnIXnRwP1ilZKP3xylxgqaHqjQBFYetPTSGnzEbyIEfNJRjEjDw9CBKiskgd5/UKY2x6DHcL2\nFJpueBuSUS9dQ/+gsHODUVSepekBwO8FNeEXciwzAGABEVoohez0s2urqV4S2y19qq/OFCNSi8YZ\nvOBHHbRtqxzsfz6n2zbYXL0NEdQkeW063Nbt6UOM8aONvMF+WpEn25kqwlQph57rN1y5BoDaWNJt\nN6mh5/0NjcJb6jW339dIKd1gPMab2fOw/xdjrqXY8RA3Ymnh8yJYUKjO+LRN2GWMZol8QWOEJfXX\nfd32bWorbKtW6zTOehPlC0nvYrIo279LOj/UvdEMQLIp8vAPdUE9ldpIRU3BeGFD9UjKyJs//smK\nA1VYOyyF88MM3XsVyv9X0H1d+k/lkspCWLZMt0s1FKZYq1E8tLAQmS+xRNomO8petIigYkuLIS5Y\npXQLCwuLF4RdLC0sLCy8QIK44eNedt6le18mjHtZ4TETfB1o1I1uwaJsqhnec1OmUFMXlYRtNXtf\n/5WkEmjBklNFv94Tp272qgOx4yxfZm3TQ+ERae/Gchd0CBBBcrymq5eymA6zuXPAU8zJjWvUISeK\nyort2qrObjuLsc8QkkH/AUrTApvpobDNUlpweRokSnr8sD0EqD+Mmedlrt79J8JEKeSNfJPVqNoi\nLz/QibVfNUKXftXH5EE9Tt1wDhmuNljJOuJR0CExpSHxIfM1/yXDDKGas6Ncn28YW+q5KAVWGhvU\nuIAcKtRojev4M2vvqylMezprVeeSaqawjUMt0e/tKRjmxMR2EDYF7Yavh4xlrMWnJsW7UOKCbu/9\nRNpOGCpE+e6z8/JEA1IbgpqjcHmzHpt5iTz/Zzy072E8fjcA3NOKUWlTyFCqEmzoJlombFAO9UG0\nB+qM3GeZ83oBCq+QUW5n3XALCwuLF4NdLC0sLCy8gF0sLSwsLLxAoiilW1hYWPzTYN8sLSwsLLyA\nXSwtLCwsvIBdLC0sLCy8gF0sLSwsLLyAXSwtLCwsvIBdLC0sLCy8wHMl2v4qjBw5EgcPHoRSyq0M\nmdA4ceIEOnbsiBYtWiAoKAiXLl1Cnz598PTpUwQGBmLcuHHw8/NLkLmMHTsWe/fuxZMnT/Dxxx+j\ncOHCiTaXBw8eoF+/frh+/ToePnyIjh07okCBAok2HwCIjo5GnTp10LFjR5QtWzbR5rJr1y507doV\nr7zyCgAgX758aNOmTaJem9DQUMyZMwe+vr7o0qUL8ufPnyjzWb58OUJDQ91+REQEFi9enGhlsv/r\nZbspAbBr1y5q164dERGdOnWKGjZsmBCHFbh37x4FBQVRcHAwLVy4kIiI+vXrR+vWrSMiogkTJtA3\n33yTIHMJDw+nNm3aEBHRjRs3qHLlyok2FyKitWvX0qxZs4iI6Pz581SjRo1EnQ8R0cSJE6l+/fq0\ncuXKRJ3Lzp07qXPnzuKzxJzPjRs3qEaNGnTnzh2KjIyk4ODgRL9XRM53PCQkhIKCgujgwYNERNSj\nRw8KCwtLsDksXLiQxo8fT0REly9fppo1a/6l80kQNzw8PBzVq1cHAOTJkwe3bt3C3bt3n7PVXws/\nPz/Mnj1b1AvatWsXqlWrBgCoWrUqwsPDE2QupUqVcgWVU6dOjQcPHiTaXADgnXfeQdu2jq7ipUuX\nkDFjxkSdz+nTp3Hq1ClUqVIFQOLdp7iQmPMJDw9H2bJlkTJlSmTIkAHDhw//W1yfadOmoW3btrhw\n4YLrNSb0XNKmTYubN28CAG7fvo00adL8pfNJkMXy2rVrSJs2rdtPly4drl69mhCHduHr64tkyZKJ\nzx48eOC6KwEBAQk2Jx8fH6RI4VRnXLFiBSpVqpRoc+Fo3LgxevXqhQEDBiTqfMaMGSPq0Cf2tTl1\n6hTat2+PDz/8ENu3b0/U+Zw/fx7R0dFo3749mjRpgvDw8ES/PocOHUKmTJng4+OD1KlTu58n9Fxq\n166Nixcv4q233kJQUBD69Onzl84nwThLDvobZlgmxpw2bdqEFStWYN68eahRQ1e1S6zrs2TJEhw9\nehS9e/cWc0jI+axevRrFihVDtmzZ/tSe0NcmZ86c6NSpE2rVqoVz586hWbNmePpU688lxr26efMm\npk6diosXL6JZs2aJdq9isGLFCtSrVy/W5wk9l7+ibHd8SJDFMkOGDLh27Zrbv3LlCgIDAxPi0PEi\nRYoUiI6ORrJkybwr6fsX4qeffsKMGTMwZ84cpEqVKlHnEhERgYCAAGTKlAkFCxbE06dP4e/vnyjz\nCQsLw7lz5xAWFobLly/Dz88vUa9NxowZ8c477wAAsmfPjvTp0+Pw4cOJNp+AgAAUL14cvr6+yJ49\nO/z9/eHj45No8wEcWiI4OBhKKdcNBrwsk/0X4i8p2x0PEsQNL1++PDZscERNjxw5ggwZMiBlypQJ\nceh4Ua5cOXdeCVnS986dOxg7dixmzpyJNGnSJOpcAGDPnj2YN8+pkX3t2jXcv38/0eYzadIkrFy5\nEsuWLUODBg3QsWPHRL02oaGhmDvXqW199epVXL9+HfXr10+0+VSoUAE7d+7Es2fPEBUVlaj3CnAW\nIH9/f/j5+SFJkiTInTt3opXJ/m+X7U4w1aHx48djz549UEphyJAhKFCgwPM3+gsRERGBMWPG4MKF\nC/D19UXGjBkxfvx49OvXDw8fPkTmzJkxatQoJEmS5L8+l6VLl2LKlCnIlSuX+9no0aMRHByc4HMB\nnDCdgQMH4tKlS4iOjkanTp1QqFAh9O3bN1HmE4MpU6YgS5YsqFChQqLN5e7du+jVqxdu376Nx48f\no1OnTihYsGCiXpslS5ZgxYoVAIAOHTqgcOHCiTafiIgITJo0CXPmOAX8Tp06Jcpk9+/fP0HmATih\nQwMGDMD169fx5MkTdO3aNVbZ7heZj5Vos7CwsPACNoPHwsLCwgvYxdLCwsLCC9jF0sLCwsIL2MXS\nwsLCwgvYxdLCwsLCC9jF0sLCwsIL2MXSwsLCwgvYxdLCwsLCC/w/WJ3wIWkODOgAAAAASUVORK5C\nYII=\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BP2Zb-8IXQp4",
        "colab_type": "code",
        "outputId": "08c4a27b-9f8f-43e2-b41d-30a1e443568b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "output_v.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1, 42, 42, 32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QCu6jYNiXJoU",
        "colab_type": "code",
        "outputId": "963a088b-1117-4c4f-b2f6-77258b98322a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 404
        }
      },
      "source": [
        "plt.imshow(output_v.squeeze()[:, :, :3])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7fb474373710>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAFKCAYAAACU6307AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztnXtgjuX/x99XWMyhMaaISGQZhVQj\nOZNDRKJmOeQQy/nLLMwhFTOnHDKTUw6ZKClqc+zICh1MSlQszRwT2aR1/f54brP73j7Xcydt8nu/\n/rHrej+f6/4813M/H/d9PZ/7cymttQYhhBAjN+S1A4QQ8l+AwZIQQlzAYEkIIS5gsCSEEBcwWBJC\niAsYLAkhxAX5r9TwpZdewldffQWlFEaOHIkaNWqIr01Wyta+ec8eHK1eHQBQfu4p0e6xfiVEbQ1e\nlZ0L7iVrALDd3tyzZw+qW/4kR3cQzUYPryVqSzBaPp4xOSva7gu6ozoWexoXx4hWaQXOmwZFQ3VO\n1BL1INlw00K7Pw/sQfUdnrnx6STPzfiH1onac2//KWrPQonaHL3U7gtaozrWe3wJfV20A4DQ5e+K\nWodb5GO2SakgD7rrJ7s/gXtQfZ9nbtpu/ks0OxYuX5PsmCqfHD8ObST7AqBic/t3Z0/sGlTv85in\n4d9dtKuxcqiofa0TRa2Dul/UlsL+3Si4Jw7p1TsDAApjt2h3I/qJ2gXMFTUAePuuW0RtxDcptva6\nPXvQ1vqOfxs5W7TTzz8ralcULD/77DMcOnQIcXFxOHjwIEaOHIm4uDjX9gWCgq7ksP8aQdeQP0Eo\nmdcu2Agqci3NjV9eu2AjqNC1MzcAEFThjrx2IZMbgq4dXwCg8lX4jl/Rbfj27dvRtGlTAEClSpVw\n5swZnDsnX8kQQsh/HXUlT/BERkaiQYMGmQEzJCQEL774IipWrJjj6y8mJV1zV5OEEPJ3uOI1y6x4\ni7eX1icvUU7rzHXMa2HNUmsNZfmT12uWGsOgMMXTuAbWLHVTDbXJMzd5vWap0QUKyz2+XANrlrqW\nhtrtGetaWLPUCV9BNb/b08jjNUtfvQvnVW0A18aa5T6tEWh9x690zfKKbsMDAgJw4sSJzPaxY8dQ\nqlSpKxmKEEL+E1xRsKxXrx7i4+MBAHv37kVAQACKFClyVR0jhJBriStaswSAKVOmYOfOnVBKYezY\nsahatarhKI7bHq0z+4pipGh2Tr8kj3m7LA28ob8sAhhx0H4ZXkZr/GL5U7ar4Vf9ep1kbZd8a4dY\ngzNj19qaelw7qHFvAwBGjqsrmr2kzFfyzSNkfxKqixL0b46OvhqI8Yy1vd8K0S4YT4pa5bN3itqB\n7+X3UafWJ7b2Z1rjPutz+lxeEfGw+0VRahY9StTODn9M1HZgja2ddfkGCw1fo/byZ6HbymbqI1nz\nMN7hzxgo9bynEbdLNussL5l8bjiavGAGBDvaRbXGWWtuihnsUO9bWUs2xBQA+Q/7iNovC+1LGKV6\nvI/jix4GAAQ9HS/apRrC4RWvWQ4bNuxKTQkh5D8Hn+AhhBAXMFgSQogLGCwJIcQFDJaEEOICBktC\nCHHBFacO/a2DbLSnTuhmOrOvvCED5nBNOR1FazmNRSk5sx8A1uujtnYraGywniRprWrKx8QXovYz\nbhW1W5f8LGqqW1f7MfQSKNXNo71SW7RreZfhKRwAGxoaZZGXHafDQAAzrb8XPCCnwKTJD37ge9MB\nCxm0NHsza6rO4adMgwLll5p1EdPXwZECZ0sdQs6P+nroISqDb5Kf0pp+xjAkgAzY087y6XbIUJ60\nsw5jQ0S7dW0MT3/VaWo4YmtZKjjE1tRpGqqQNTdhhiGnyZJeZ6438UJbObd7Fezf46/1btRQnnyz\nPT3k77E2pIDxypIQQlzAYEkIIS5gsCSEEBcwWBJCiAsYLAkhxAUMloQQ4oKrUvzXK80dbX2577Be\n43x1Jq16hYvaabXVcMCjBg14zZEB00pf7gtbKacV4AlZKveMnB6k58rpCMcRk0Ofp4ZLqTC5MOoP\ncqaS55gGbQ++ErXq6j7HQJ9hoNVXzjCmXBYY+NWgDUyTtWdQOlvfJ1Zf+aWphlEB9DCUVnppj6yp\ncbK2a63c96l8HmNApCjNmCjX+Xk97B15TABH8Uu2vnxWX6fxm0W7dTUfNoza0aD1laX0HCp9WX0v\nTpOL7cr1nwC03WJScUwfEbU9qmz2Piv1b8wiw6ALZYlXloQQ4gIGS0IIcQGDJSGEuIDBkhBCXMBg\nSQghLmCwJIQQF+RK1SG0ceTqvKsz+1Qv+fD60c9ETc2U9zDGoIJmf3S6vQkNZVUdSjQk3dz3iVx1\nRz1omMbyBl8OGyrZzDDYOTcWc6DlYjaIMmxLFQF7CojWO6DUA1ZLLi1UA2+IWkk8LmrG5JCV79t9\n6dwCKs6z2dSAJ94zWWKWMdFJrliFTfIe1+ht/4z1D4CyNs7b+KN8bjQzeGL68qk6yw0q8Pvn9spC\nvho4b7nxoWE/9oRadURt+m75OwfDmM4cxItao4B1Hv9peJfNTX4aPAGADyCnQD2UbD93cKsGfvYc\n66Dh1KhkCIe8siSEEBcwWBJCiAsYLAkhxAUMloQQ4gIGS0IIcQGDJSGEuIDBkhBCXHBFeZaJiYkY\nNGgQKleuDACoUqUKIiPlMlTKsCte1buPi3bf1pogO7HopCgVnLRMtgOQHmHIbbzXMB2fG8rCfdJI\n1rrKUr4f7L78qTXyW75kDMnJwuLwlwYReNbnbVGb7SMXTVNLptvaWefGVDDsEbmaHPRcWZOz7ABf\nR/t3rVHY8uX3XQZDAKq2/DkWMRzVtJ+gdua2FtXAWWusYs/JvmCiYVQD1bx8NfdOsjW1joBSnj4N\nkz/d5TF19rJvmTxiyHx0nBxZc5dxSDZDkadlVz431EsDENtS1pxSOa2RbJ075Q0V6vR78pxfcT3L\n++67DzNnzvT+QkIIuQ7gbTghhLjgim/Dx48fj/Lly+PMmTPo378/6tWrJ74+KSkJQUFB/8hRQgjJ\nS64oWKampmLXrl1o2bIlkpOT0bVrVyQkJMDHxyfng3DNUoRrljJcs+SapYncXrO8otvw0qVLo1Wr\nVlBKoXz58ihZsiRSU73siUIIIf9hrihYrlu3DgsWLAAAHD9+HCdPnkTp0tk3lyKEkOuFK/o1vHHj\nxhg2bBg2b96MixcvYty4ceItOADg4G1i37eVSolmvovlWkrnfzosaukRsisAgPc/lPueN9wYrjOM\ned5wyzRHHjMjh1uCjEtDmW5fCj1gEIE5pdNlrV+0bNha7jtj2IkRxwza8r2idLJLNVHzz6HvvPVv\nr2aGHRoB4OnvROmA4e7O9F/+8hdX2dpdJl3uC4W8m6KJQZ8WErXDOx812r410LnUEgFYt9+m5Y2+\nWCxqMYPuFLUb35HvHisrx8xpIMhyIqmx7MvMLYGipiaJEgBgzwn5Pry8/wanOyhvrThqZZodmSsK\nlkWKFEFMTPYtXAkh5HqFqUOEEOICBktCCHEBgyUhhLiAwZIQQlzAYEkIIS7Ild0dazt+qt+ldWaf\n6UEMVfgFWfzY8PN/zVFGf5wJEAFa45jljzFbdGcnWbt3lShVNqRqjHU8TdFFayy3fAkNf1Y+XtQ3\nsgbgZSU/bdQB8taP5R5/2dbWq36F6uTn+fuN/aKdWmOYudd3iNIfd+8UtU6R9p0m39Ia7a25WSsf\nzYMeLEpRkNOVHg3uLWp3bnccIstTKkGQP6skNU7UEC6nziFKlgBg/uEKtnav8j/i1cMVAQC9b/tR\ntLvJkFh0pqnhgBvHilJbZU8PfFv3QDu1yPO3Y8fQrCj8LmoNKppTfD6Q3yLecuyo+ughjbW3ecZr\nb9gZVZ/m7o6EEPKPYLAkhBAXMFgSQogLGCwJIcQFDJaEEOICBktCCHFBrqQO3afetbU/020y++LO\nPiLa3V5Udk3jRlErnK1srJ3zGx+zj9X0VahNvQAAFZstEO1aGcacc0DOuXj9jk2ittKeqYO1AzUe\nnelJcdg2SD5eGloYvAH+wHxR06qSqCldwv5afRRK3exp3LFIPuABeXZ0DdlMff2uqI3UbWztF6Ex\nykp7eclL4ZjPDVod3COL8w1Flc/Ym/p/Gmqqx5HOXeTUoR63zBG1FqgraurHT2RfAGhnao3WgJVa\npSAX1QbktLKh+eX0uFV/yiO+DHuVqw76RrypLnj+/qygaKfuM5xT+j5ZA/Bjl3tFreKKfPah9Fko\nVdTTaNdOPuRauXA4rywJIcQFDJaEEOICBktCCHEBgyUhhLiAwZIQQlzAYEkIIS7IldSh4cq+sVC0\nfi+zbyveE+12GqqjGDNHVpj9KRFib5/UGv5WysUOrBbt6qOJqKWiuHzA8xtkrZA95ca23zJiRTNd\nqY88JoBxP+SwKZvFeDwkG2pnM6s/s0SzkhggaifUUPl4DaaK0l8f2D9lpXXmZlOfySMCAEzbuZkK\n68hJXgDgqMikB0KpmQCAcnMGilbJYYYhTZtnxb5q9ObJPj1t7RUaCLGGe/28oSqVryHtTP8sSg+q\n6qL2m2M7r6/0btytagEAMnrcLdrtXbRYdqWooeoYAHX2V1l8Y4p9rI4aarU1OYa8Mh3FqkOEEPKP\nYLAkhBAXMFgSQogLGCwJIcQFDJaEEOICBktCCHFBfu8vAfbv34+wsDB0794doaGhSElJQXh4ODIy\nMlCqVClER0fDx8dHtI+u9b7Yp3Yb0oNuEyUUOyRrv/UdKYsAXu/5Ug59nn+rLJA35QI6ikqHC7LV\nmzea6hU50LicF6Xl9CB1sLNxmL5vyulBeq6cHqEed3webwB43Pq70jrR7oRhc62uzxwQtdfmyXbO\n/8l11r7OXjLe4uTzqqzJzjCsvzqRvQ+evuRnDSlAcoEsHDS4Ete3l0EFlu9ylEHCUHy9a5rnz1Fy\nutYUyOlBOw1v4/V+e0RNvZI9de5r/RMAQKsvRLto+XAoeNb8PW6n5Wu9UOf70MAb1nkcCbnSmWmT\nOK9XlufPn8eECRMQHByc2Tdz5kyEhIRgxYoVuO2227B6tZybSAgh1wNeg6WPjw/mz5+PgICAzL7E\nxEQ0aeJJ0G7UqBG2b98umRNCyHWB19vw/PnzI39++8vS0tIyb7v9/f1x/Lip0CghhPz3cf2446xZ\ns1C8eHGEhoYiODg482ry0KFDGDFiBFauXCkbH0gC7gi6Kg4TQkhe4OoHHie+vr5IT09HwYIFkZqa\nartFz5HOjmdKd2mgtlX+frfB7kp/4ClmXhiOf9z+A0/zVzUSenn8abEg+48/l5HHNf/AY3THhtYa\n6tLzwsb/xrz9wBMnanNNP/D42VfG9Rv68o8+lZrJB4zaKEpdn2kraq/Ne1se01EBwDY3/+AHnm4G\nsyXGH3gibe0T+nmUVGMAACcxQTY0POJ90PAbTpyXRbLln9ufq0+qNRRBuz0/8OxdZviBZ7o8NzsN\nx3u9n6w5f+DROAUFzxYlWp0W7Uw/8ETiL4MKPPw3fuDpqDVWW+eO6QeefVr+EfOKUofq1q2L+Ph4\nAEBCQgLq169/JcMQQsh/Bq+34UlJSYiKisKRI0eQP39+lC5dGlOmTEFERAQuXLiAMmXKYOLEiShQ\noIDhKH72tv41s6/5A870h8ts3GFwTWdP47h8vFKyBgCOQjd6qIaaZv1X1FCu1hNcW07HOWY43E+G\nKc5Q8tVTFz1GtFu+IHs6lm2c0JtE7ceCf4han1s/sLU3Jms0K+fxJ71Xek4mHjvIm1J1HSfnB4X6\nTBS1F/74yda+TWscsuamZw6vz8rmhXLFmjueHi1qcpITgNhvbU3d+06o+d8BAD6dWlU0q/udaVDD\n5npfnTcZAt0cm8t9cRKqpr+n8eUp0exRw9dqrTL8/qC9fK+yvtRWrSpQfN3OCftE7d4xS7wcxXCP\n4PjOZc3IQ2v5ylqvlyfH6214UFAQli5dmq1/0SLDrmyEEHKdwSd4CCHEBQyWhBDiAgZLQghxAYMl\nIYS4gMGSEEJckCsblmXblEnrzD7Vw2BXz+BaLznlAC/fa/ZnoD0lw5bmYNoJravsj88S2fCPZob3\nscmQeG38ZIaYRGg1XdRegzw/XVHeMdAaQD0GAFB403DAt2Tt50dFaWq5dqL2AewJwm9rjXbW3KyT\nM4487jy3S9TUk91lwxXJouSvxtnaJ/QglFSeTcxOYLBoV0Y+GpImyR9yiQjjtnxQhx3pMeUAZbnf\npLxsu7mY4RrpjJw69FdQiKjdkH+U3Zcv60Pd85Gn8ZVhg7yTsoQ98vEAAA297EyY1Z+s3yvDuaMj\nuGEZIYT8IxgsCSHEBQyWhBDiAgZLQghxAYMlIYS4gMGSEEJcwGBJCCEuuKLiv3+XBx3tj7P07XaK\nWejcUy7S+TqSRO3e/L8b/cko4ihxeg7IKPI5ACAf7hPtRpSWc9eiTClxchU6oEMOfU2sf+vLOV9d\nj7xoGBR4UU6zxKghconX6bAX6h0CYDru8TTap8mDqkRZe0rOs/yfriJqOoc5vVQq2Oc5+XAAgKG1\nZW2awe53OQnvxMxBct/AN0S7jvhE1Ey5lOYsSwAvOF4xT2f2yUXogEK/yUWc34W/7E+SfD5qZy41\nNLSVX6k6bJbt3pereKsuT4vapWPIGGbvZYNZhCzxypIQQlzAYEkIIS5gsCSEEBcwWBJCiAsYLAkh\nxAUMloQQ4oJcKdGmDDsYmo4u73wMVFLxohaDGKM/SbCXE7P5Y7ALNWhjDIbNlCw6qmzZdqHrMkdO\nf1jmZaM9ZdhWvI7BbrujnU/rzB0o878mv4/4p54RtRY3V5QPmGrI1XDs/a07aahVl/YNl80A4B7D\nxn9fL5a1DPWOqJXWbWztVAClrb+P5ZQCZjH+zd2iNlbVkg2zpeM4iKpra+rhn0BF1/M0hss5efc4\nSs1l5UvTbpPtDSf5W4ZSg+VkMyQb3uObXkKTYc7jHO1OWmOV5Y8cOYAFhoDEK0tCCHEBgyUhhLiA\nwZIQQlzAYEkIIS5gsCSEEBcwWBJCiAtcpQ7t378fYWFh6N69O0JDQxEREYG9e/fCz88PANCzZ080\nbNhQPoijAohtN8WRhgMbdmGb3TlS1PrfMcEwKOD/gr19AholLX+aKLnq0Nd9nYk1lykcI1dBujH6\nsKh9ut9e5UfH/gXVx/o/LP4v0Q6HzGkl2pCudIehIstBR86FLV2nk7yDIQy7SWrD8ZShBJDGXkfH\nq4DqZdktkH0BgOH/E6Uno6eIWqMbZF/79D5tdyfGD6rvrwCA5jH1RbsEtUfUyudUWsnicBNR8jDU\n4U9rP6j1Hn/QzE80y3+XfMw/TRujOvNxstLe3tRvaqgO1nHeOiqaaX2zqCnlZZfW9XL1LP2W4/yf\nD6C39ferhu+OIRx6LdF2/vx5TJgwAcHBwbb+oUOHolGjRt7MCSHkusDrbbiPjw/mz5+PgICA3PCH\nEEKuSVw/wTNr1iwUL1488zb8+PHjuHjxIvz9/REZGYkSJUqItklIQhCCrprThBCS21xRpfR27drB\nz88PgYGBiI2NxezZszFmzBjx9dVR3dbmmuVluGbJNUsJrlleW2uWV/RreHBwMAIDAwEAjRs3xv79\n+69kGEII+c9wRcFywIABSE5OBgAkJiaicuXKV9UpQgi51vB6G56UlISoqCgcOXIE+fPnR3x8PEJD\nQzF48GAUKlQIvr6+mDjRcL8MYG4LR0f85b7lCfJl70c3dBG1O+LqihpuM7oDn7sW2TtCAJ8Vnr6S\nz/YQ7b6dk0/ULhqmoMFwgzNP5NB31pqTQ+dEszpeCtIoXdWgnhaVwqq4vaMTUNiq8HPv/TVFu22G\nW+1ShqSJ6HzyLRo2BefQ6enT3m7DB0wVpW+iZW2GYeVjxDzH3MTozL6bJsnncQKyb3R2icOyK8gX\naP6QM9o4ljD080Bm3/Oi3Z/NDVWg4uRNAlPqLRG1w2/9mq0v0boVbikfzVhYSb8n32YDQCnDwCsc\n52PIfI0V1u33g1/IduUNx/MaLIOCgrB06dJs/S1aOCMgIYRcv/AJHkIIcQGDJSGEuIDBkhBCXMBg\nSQghLmCwJIQQF+T5hmXvGuxMGwvNQhlR+x29jP4ULmB/Skf/0RrKZ72n4WPYQOt3+Skd405npgyQ\nlY5hOmuoSxt1de4rmrXAh4ZBgXh8LYtfyilQ39S0OxuoNfZZn1Wg4Xh9qx8TtXf27BO1X9BA1PRM\nx6QOADDL8+fvA81pNYUNH4jasVbUXnmgvaiNbG5vn47XKN7C48fpBNkXdVdO+WEW36yUNW84NlDT\neAcKntSfrkr+Zr0Gebe71YZdwDqaNgIMcHzHUzVUaU/fNvnUQIOcssMsSn4qawBwIv4jUVMP25+o\nyhpzhhjOjWmG7zGvLAkhxAUMloQQ4gIGS0IIcQGDJSGEuIDBkhBCXMBgSQghLsil1CF7gU+tj2b2\njb85VbS7sciDovbdgc9FbRHSzQ71cKQ5LNRQT3v6nn1Vno7ZhsKwSo2Tj1fPoH1ib2ZNcUAfw0cT\na06dmfWpbDugrmyrnWkVGpmpTwpyEWNjqadfDUlHMXJa0ZAYuy/TfgSGWgVzevxUPQeLy2yDnOY1\nEB+I2hxDKtOzk36ztfWIolBRZwEAe0YUE+22qyGidhPkosl+mCxqANAC4XZ/sp47urRo9zU6i1qN\nYjPlA/5icMZRyEqX01DJli+GUj4acqpec5iL/87ScgHwqsPt55WO1lDDLX+myHFFazkdiVeWhBDi\nAgZLQghxAYMlIYS4gMGSEEJcwGBJCCEuYLAkhBAX5Erq0P3Kvqd4on4+s2+/oQLKr5D3W+6k5e2D\nVqm5Rn+yvWGtM3dOqmyoSFJWzvJAzBA5HScIz4rac4i1tSfoPxCpfAAA897bm5MJAOB4yyqyM4Bx\n/+ObDbtEHXWksmg9GErN8DReklNgfjbs/x4uS9hTQa66s2eSvbKU7lwfKs5K7XjiIcOowI8GrYJB\nU1gli8U72f05paFKWHN5t2w2dNs9ojYt6ktR+ytWlAAAxQ7a22e1RlHrsz1rsFPD5XNjyGT53Jg+\n/JQ8aJR9Mzd9A6Cszd8O5ZPHNG0QprxEpnBD9txkx/c4y1ccx5fLhiVDrvK+4YQQ8v8NBktCCHEB\ngyUhhLiAwZIQQlzAYEkIIS5gsCSEEBfkSurQp45UlbpaZ/aNN9i18w8QtWfX3Sdq+HyT0Z/1HexV\niVqV09hgVUhpbcplwJ2iovGdQZO5IVuRHw1llfnR37wh2qnOnxlGBQJqRYvasSUGj4Y7KjLZqrW8\nKtvd21OUduysKWoPQE6dme1oP6s15ljnzbeHmsq+AJj9uHwOdPpMrhC0Cr+J2uj99vaEyhqR33v8\nmdD8YA4WHrb+VEnUJuBP2W6xvLEcAGCl47N6T0O19PRV6NVMNPup40Z5zGqGb+TWsaJUIcB+Tv2o\ngYqWe1N7lBDtmiw6LWpt2ogSAOBj026H6GZrab0YSnW3WktEK1M4lJMVszB58mTs2rULf/75J555\n5hlUr14d4eHhyMjIQKlSpRAdHQ0fHx83QxFCyH8Sr8Fyx44d+P777xEXF4fTp0+jffv2CA4ORkhI\nCFq2bIlp06Zh9erVCAkJyQ1/CSEkT/C6ZlmnTh28/PLLAIBixYohLS0NiYmJaNKkCQCgUaNG2L59\n+7/rJSGE5DF/a80yLi4OO3fuxMcff5wZIA8fPozw8HCsXCk/tnY+KQm+QUH/3FtCCMkjXK1ZAsCm\nTZuwevVqLFy4EM2bN8/sdxNrv6xu3wKAP/Bchj/w8Ace0Y4/8BjJ7R94XKUOffTRR4iJicH8+fNR\ntGhR+Pr6Ij3dE3BSU1MRECAHNUIIuR7wemV59uxZTJ48GYsXL4afnx8AoG7duoiPj0e7du2QkJCA\n+vXrG8eohwK2ts7SpyFvHqRObjUMavhvpcWNRn8GH/re1m41DRg83eprX1k2TJU33lLH5M2lxhyQ\nr56gHP9bZ9kgbIQukO3lmSTJV44AELxH3njrbRjKtTxcVezT0W+LZkr1ErUHdAdRK/6dPDdVqy7K\n3gdP3/Ly8hUJALQ7LF89xSXfLmqryslXlhOWOjqev9ynai2WnTn5gygVOWv4Cr4lSwCA93O49LL6\nfjpo2FzOUJSrWr9BorY3QL6y/AmPOnrWZvalJ8nz/RB2idrX7xQUNQDIr+JErVzjdo6exajY2HNF\nWXSLcVj5eN5esGHDBpw+fRqDBw/O7Js0aRJGjx6NuLg4lClTBo8+6pwoQgi5vvAaLDt37ozOnbNv\nnbloUfb/9Qkh5HqFjzsSQogLGCwJIcQFDJaEEOICBktCCHEBgyUhhLggV0q03e4o0faD1pl9P94o\nP/mB9G6iNEGNE7VIeEmk0in2Jm6GwlFPQ91iMJSfRJgyQn4+ftgJ+ckfOB6KsT3BM1A2U3fJGgC8\ncLP8sY5ub8iz1I85/FkNhY5Wa7VoNud5ecwwOT0Pb94YJWodxo2wd0RoYJJ1nOfkMQFAafmJGj39\nBdkuXM7yeNfxsE1rrbHeOo9bm3wxaPfPeFPUEgedN1gCQBdbK0uKLnCT4ahyKqnxjdy5XtYeQjVb\nO1YnoY/yPOIcCzl3+Q2sFbVOGCUfEEARvChqzkzrE1qjpPVZnZRPDejbubsjIYT8IxgsCSHEBQyW\nhBDiAgZLQghxAYMlIYS4gMGSEEJc4Lr47z8hp63MMvsuHJIN1UOiFGlKeDLlagDAb470oGL6cp9x\n3Bai9HiUnB407IsahjE/tLc1APWrR/omSbbbIpe2A4Dv3zGkXTSW90sKVWey+ROqPLVL/SCXaAvb\nYXDGkDrU90K4qK2pHWlrLwfQpbbnzJmpzSXqoOSCuwry+9eoIGpVUM/Wbg1giJW+0xrLzf4I7Bgs\nl69Tg0XJoo69qasAyqpQbDiP10XItd9Co14Rte+C5ALX3yZlTwGMxaW+naLdhu4PixoWy74AwLuY\nLmoNO2UvX3eyk1WGsZKhDKNh3nhlSQghLmCwJIQQFzBYEkKICxgsCSHEBQyWhBDiAgZLQghxQa5U\nHVKOqkNa68y+wK/kw++725ADFP6yKI0uYSjXA+CFrvad7/Qt5aFSPH268G2indpsmKoOsq+FIFdP\nSkNxuy96OpQa4mkkzZCPN1Mztz89AAAeDUlEQVSWAKDbPFlbYij0NC3Z3h6SoDG9uee9DTFsNx0K\neW/wbrqfqDVXfeRB+9qbeq6G6mfN81zzafv0TPnzODcoWNRGQK4eVduRV6I1cOnU1oPl46kZjUWt\nCeQSOMchp84BQD7Yt5vcrTVqWQ59gSminX5vmKiplvLxtCGvxvnus85N63vkuVn/ZS3DAT+QNQBQ\nRQ22Ze1N/AyFWwEAtdQR0WzXP903nBBC/r/DYEkIIS5gsCSEEBcwWBJCiAsYLAkhxAUMloQQ4gJX\nqUOTJ0/Grl278Oeff+KZZ57Bli1bsHfvXvj5+QEAevbsiYYNG4r25RypQ8laZ/b9bEhHWGgoH/S0\nwevmKqc6R5fxxx+29gqtEWL586TBn7aG6jnNxsu+bjRUsgF+srWyplXl08dFq1mqlGFMIMzwPh7R\nsq/r1AR7hx4NKM8GX48tGi3avdndkOZ1SyFR0ikVRe1t9Y2t3U5rvG3NzXkUkI8HIMTxGWelSyfZ\nrlGc/D56OaSsnxX6GU7IVwJkrY38GT+2vpFsB6A4ttra87VGb8ufV7d2lw0/fVyUaoxqK2pf409R\n2+j4rjbVGpssX5rhXtFu2BG5ItGmsubQ9OV2+bPSwY6KRbofoOYCABSeku10EVHzWqJtx44d+P77\n7xEXF4fTp0+jffv2eOCBBzB06FA0amT+MAkh5HrBa7CsU6cOatTw1GMsVqwY0tLSkJGR8a87Rggh\n1xJ/6wmeuLg47Ny5E/ny5cPx48dx8eJF+Pv7IzIyEiVKyHtqf5eUhDuDgq6Kw4QQkhe4DpabNm3C\nvHnzsHDhQiQlJcHPzw+BgYGIjY3F0aNHMWbMGNGWa5YVDN78ZGtxzfIyXLPkmqWJ3F6zdPVr+Ecf\nfYSYmBjMnz8fRYsWRXBwMAIDAwEAjRs3xv79+90MQwgh/1m8BsuzZ89i8uTJmDdvXuav3wMGDEBy\nsqfiQmJiIipXNuxpQQgh1wFef+DZsGEDTp8+jcGDL++e1KFDBwwePBiFChWCr68vJk6caBzj5xyK\nnFzqaztFvpSWb1CAWwx3fRW1oXQKgHnKvvHWCgCvW3+/brhFhSonShv1Kdmujrye+/a4m7P3vevp\na1ehpGg3UU+TjwcAKk2U1hneYrVk++3t3ix939wq36KZ6sN0SpGrvMT6yRWi+uhvsvW1s+7Mbv1c\nnhsAwHvym1xuWE55ZFV3UWujF+XQ53HoXWWY1FeaiFLku4+K2nrVSh4TwBrYN2WbD+BV6+9PGy0W\n7eqirKh99Z3hx9s75d8dzmR+gy5Tx+rTI58W7VTZWFHTv46QfQGg/ORbeLXjNftYANSObz1/9+ll\nGHWlqHgNlp07d0bnzp2z9bdv396bKSGEXDfwCR5CCHEBgyUhhLiAwZIQQlzAYEkIIS5gsCSEEBfk\nyoZl3dQFW3uJvjGz7yw+E+3exP2ipiCn4yBYzsIHANyUamvq9zRUSyv1433ZrJshAWDJC3VErWW1\nz0VtO5bY2qd1VxRXnrSHtEf3inYX1k6WnQFw8/woUSvQS34S5/Acx5MvzwKY4/nzzjvkJ0r2t3hE\n1I6roaJW0rCZlfrS3s76xMxDerVoBwAFVEdR21QxUtS2/jhB1BrDnsqj9Xoo1drTyL9BtKstP/iC\nNfqsqFX4zbAhF4DKN9nb+7VGFWt+vtdy6lj1pvITVfM2y8eTt0ADmj5rDyP9ZgNz+1t/z5G/q1/i\ntKhdNBwPANYatJcc7aznzgqD3ZPcsIwQQv4ZDJaEEOICBktCCHEBgyUhhLiAwZIQQlzAYEkIIS7I\nldShYo7CoL9BZ/adrS7btQyUK+u894acjvKuF3/6vG5/y0eeAMpaxUYGLZWr4IzYsFjUAvC1qKUa\niv8qdLe1tV4Mpaw+vVG0K6R+ETUAkOvcAO9gvaiNRV9be7w+jLGqPAAgHcmi3WQcNByxkqjcaihS\n/LOjApSGhrp0LhmK/HhefKes3VpVlL6t+raoVd3kOEQWf+S6OkAyzovay8pX1O4yjOk5vrNDA1Z6\njCm1LvhVuULWnRXlz3HRO/Ln2H2GfXIW6yborjx5SPsMyTqJ+FLU1OLdogYAurusKdhTy7R+DEqt\nAQDMafqVaBe28XlR45UlIYS4gMGSEEJcwGBJCCEuYLAkhBAXMFgSQogLGCwJIcQFuZI65Mzy0LY+\nOQfkJzwoahXu/1jUVk01+/N4fadDWVIuXqol2ukm2TcXu4QKXiofMMNf1kx7UTf4TjQbsW2XPCaA\nKISI2v7WL4palQ32NBetX4BSVpWiv3aIdrqHXK5G/WE4xV4fL2tJ1ezHqNYRaq+VEpIq730NAGhs\n0H41aIaPEQOcqUx/QV263lgnpwCh7TlZG2TY+3qmwRcAlWHfsOt7XRuVlee8ODRA3qv74nbD57FT\n9seUyvSA7mprL8AS9EQ3AMBmJZ83RfGRqP2CYoYjAqe0nOa3Sc23tZtojc3W96qpYUxTOOSVJSGE\nuIDBkhBCXMBgSQghLmCwJIQQFzBYEkKICxgsCSHEBQyWhBDigvzeXpCWloaIiAicPHkSFy5cQFhY\nGKpWrYrw8HBkZGSgVKlSiI6Oho+PjzyIKdFSdxbN5ik5lm+RN+FD4/r3ySKAqdq+++NQANO0lZin\ntoh270AubYXjckksuSAY8O1NNbN3Wn1dPpBz0I6pnoZRgWLpcp5l7Q3/E7VmKOjoeQHN4MnL3HiD\nvKOi0oZtAQ0FzBJeXyRqbYNS7B0aKBjkyb39AfLOlwBwk7+cFVj4hJxL2HKAPGbBuY4cvH5Ae6vv\nrZJyLmVU2RhRu9dUac9LBvQB5Sy1VxsHLpXfm2UwjPlQ1j6vJ0pNHbmLdl8c862X4Ki1S+kthtzF\nO5X8WdyOo6IGAG8reWfYbtH2YzYB8JrV9/4pb/X9csZrsNy6dSuCgoLQu3dvHDlyBE8//TRq1aqF\nkJAQtGzZEtOmTcPq1asREiJ/OQkh5L+O19vwVq1aoXfv3gCAlJQUlC5dGomJiWjSxFNetlGjRti+\nffu/6yUhhOQxrh93fOKJJ3D06FHExMSgR48emQHy8OHDCA8Px8qVK0XbpCQgKOjqOEwIIXmB19vw\nS6xcuRL79u3D8OHDbc9Puom11R1bR2R5FBvQT4h2zxnWLJtNeF3UGkf+3TXLzZhmbcTwP8Oa5TrD\nmmVb05plSdmXb/3sa5b6191Qfp7n07uceVa084F5zXJNuuGZ44LpovaAY80yQWs0tz6sjZDXLKE7\nGrzpISoJyrBmCfuaZZq+GYWUZx3rB9P6Mf7BmqVhzIJz7e03+2l0mOsZ662S8vcgarBpzbKfqDX2\n8tVSyv5cvdZjoNSlbRHGyoYxH8jaMyNFaaBpzdLx5Ph6rdHaOm9OXeGa5Wlva5ZoIWrdou3bVSwZ\nBnSb4vk7xLBm2eKlf/BseFJSElJSPCdtYGAgMjIyULhwYaSne75wqampCAgI8DYMIYT8p/EaLHfu\n3ImFCxcCAE6cOIHz58+jbt26iI+PBwAkJCSgfn1nGR9CCLm+8LpmmZ6ejlGjRiElJQXp6eno378/\ngoKCMGLECFy4cAFlypTBxIkTUaBAAfkgjkttWxmyJvfIB98s53FsNdyGNhqXQzpOFgr98IWtfX6J\nhm83jz9pS+TpeEzdKmprcEQ+oHGGdbZW5mwpQ/5HsFyeCgBuM/zmlhEk75o54qR918z+v2jMLuPx\naEBKThYuMLz/8C2xoja5SR/7MFnOm9aolpNJJuufl1OL+v8iOzR7nSGt5D17U9fQUF9br7/7KdGs\nvqHum5wcBmBuc5MK3XeSo6cmAOvcflIuYac2HpAHrSBLy3fJc/O5oz1dawyxPqsZ8pDQ+owsqpsM\nlkDsJvlzvL+p3de7tcZXlj/3jJE/Dz0+VNS8rlkWLFgQU6dmLxC5aJG81kQIIdcbfIKHEEJcwGBJ\nCCEuYLAkhBAXMFgSQogLGCwJIcQFubK744OO1KGPtc7sS5nyhmj3Q0M5/aGKvHkd9t8sp/h4Dlra\n1tTYCQXPgHVuCRTNPj+6TB7TVHTnkEF7+na7L/oglKrkafT8QTRrWNYwJoBq8kMzmFNxpyzq2vYm\nshSIyndCNJvkW0rUnjsnn2JvyQ+3YPLKk7b2p1v9UbeRp+/CNsNjUQAa3ihr09LXiNpqJZez6viw\nPeVMvweoS4/8TDSkHN2zUdamNxOl5vJmogCAhBBH2pntyTjD9pZqq6xFGQ5Y1KCFyb6U0fLc/KLk\nB1o+TDpmOCDwUJB8Xp13HLMQNNKsM/kjw0fVnLs7EkLIP4PBkhBCXMBgSQghLmCwJIQQFzBYEkKI\nCxgsCSHEBa6L//4TXlid/ef4S32bOsq/4/c2jFnBVMrmaBWzQ2qXva0v950xbKAGufYpdBN5Uy5V\nb4Fsl203N0DDkzJUQTbDtrPmjK8NReV5PV5CzrualW1zOY1UKwdEaTkfqaQhHWPB+jhRm9BaLv6c\nveaORgMrZehTvC8fEMD/LjwsatNUB1HrqB+TB81WqFYD73v67s6QzYr47pZ9eVu2u9/0BbAOL/Vp\nQ3rQO7XaiVrbso/KxwuVqxWtQldHz2uZfZ2UfK7eZzhPH/K2u8JCWVo/zN7uOOVy3wxD3WhTnSde\nWRJCiAsYLAkhxAUMloQQ4gIGS0IIcQGDJSGEuIDBkhBCXJArVYfUHMeGZc/qy327cjC49LpFhUWt\nEM6JWlAO6ThZcdbcsW2g9r3B0FA9qI1/W1F7t+Y62TDOPgG6Uy2oVZ5Ukyc6187JAgBQxrwLGqYa\n5kDhK1HTv79i7/CNAc739dgVdqaHZKFuPVmrbPB1sZzGgqb2Uk560zmopp4938M2/y7bAdhj0PoY\ntKfwmaiN1vb96CdAI9Ka5/vUnaJdAr4VtdkGX15Ga4MKDFq+3tbWIYBa4fn7aBf587/ZuDv6WlE5\nA7mU003OjfeyVkAyvQ+9QZQM27sDAEq+NlTU9DLHpnzxGmhhDZgg740O/aIo8cqSEEJcwGBJCCEu\nYLAkhBAXMFgSQogLGCwJIcQFDJaEEOICr6lDaWlpiIiIwMmTJ3HhwgWEhYUhPj4ee/fuhZ+fHwCg\nZ8+eaNiwoXwQtdjW1rp7lr4tol0TLBW1zemyz9ULjpJFAN1hTw8YqjWmWXkOwyFvoNSjjLyB0g2/\nyMebv26vLD5S0dbUKASFNKvlK5pplSCPCaCYoX7KbwY7Few4zqcaqq5nbqLf6STadS15VtSevPie\nqG0pYHBmoCMd5WVADbIa+bzklVw0aE0NWrsastagjt2fD16FatDL8/dT8gZq6uFJ8pjlqspate9k\nDcAER5rb6AsaL9zomZfRpXMwuETyLaKkYDiRjSl5fraW1qehVHGr9atsJu9JCB3jJavRX/ZnseO8\n6v6HxmIfz+vrXFwp2lUzVB3zWqJt69atCAoKQu/evXHkyBE8/fTTqFmzJoYOHYpGjRp5MyeEkOsC\nr8GyVatWmX+npKSgdGnTf1mEEHJ94nrN8oknnsCwYcMwcqQn+33ZsmXo2rUrhgwZglOnDNU0CSHk\nOuBvPe64b98+hIeHY+TIkfDz80NgYCBiY2Nx9OhRjBkzRrRLSjqNoKDiok4IIdc6Xm/Dk5KS4O/v\nj1tuuQWBgYHIyMhAlSpV4O/vDwBo3Lgxxo0bZxyjenV77Xz+wJMF/sAjwx94ZA38gSe3f+Dxehu+\nc+dOLFzo2ezixIkTOH/+PMaMGYPk5GQAQGJiIipXruxtGEII+U/j9TY8PT0do0aNQkpKCtLT09G/\nf3/4+voiOjoahQoVgq+vLyZOnJh5pZkTI5R9w6YoXSuzb9EJubLOcfk/a0C/Ikq3tf/UYAgcWrvM\nPlTWqkPTcjC4hFzkBI3lCytsKfqDYdDbZV/GGcxMGoBC98taWvDtoqbv+dDe0a0ssOSI5+/uIYYj\nthcVNd9wZdH7JsOYM+2+6R+h1KUr8Z8MdgD6ylLPGLnSzbM4L2r3o6Ot/YfW8LE+K9OF7JVy0ou+\n5HN7e8i9GtN3evwZUicHA4tS+mtRO5EkX1nfXF0e0zndY7XGeGtuxsF0hSxXa/LGd4bdxe5MCLS1\ntf4GSt0FAOih5bvZhbhZ1LzehhcsWBBTp07N1r9mzRpvpoQQct3AJ3gIIcQFDJaEEOICBktCCHEB\ngyUhhLiAwZIQQlyQOxuWKXvyqC09RncQ7eLUm6Imp44CD0Wa/QmdYG/31hrzLX/k9GngrQvyxlO4\n0ZBcbJrhI8PtLy07GepIOABgju/mnCwAAItLpIkaAJw27K7W0/B/5HOOjeBsn5WJew1aK1ma/byc\nINMf9nS0rL7M7NzG6M6qOHmTrLdXyCeIf8gXolYL5WztXVqjtuXP7ldlXzb0lB8gGKiWiNr3OCIP\nCqA7ttnai7VGd8ufJYYHGtBWTo+CoTaOrvauqKme9s9DH9ZQ5a3z5rD8BXjptHxujWxt+E4BwPai\nojTzLnte1YC9GrOqeY418BvnloWX0VpOZeSVJSGEuIDBkhBCXMBgSQghLmCwJIQQFzBYEkKICxgs\nCSHEBQyWhBDiglzJsyzpyNM7oXVm39nlPqLd4i5yXlcIDovabdpUURYoq7ra2p9ooJ7l4ieQS4Yp\nFJIHffCoKBX+WM4l+xlrbW0/3Q6/Kk+x5OIwFTE2FBQGsKN2oKi13vWNqA11FHgdqTVesj6rUd99\nmJMJAEDf+ZColS26X9RCf1skalHKUYZP/wooT5HZY6aCsgB6FZfnfG9N2a7LFvnr8Lyz+K3WgDU3\nypRMu2+XrLU1JKjurytrACLn20sRPt9bY8x8jz8H+8h29Q1j9hsta5MGyu8xpJR9bspBI9mar/Jy\nDW/o7+Ux1fOG5FUAz5XvJWoTk+XcbjmzG1hjCIe8siSEEBcwWBJCiAsYLAkhxAUMloQQ4gIGS0II\ncQGDJSGEuCB3SrQNd/yMH60v90Vf4aDVDXW/7ulhNNXLOjo6kLkl8j7D3sh3GcasCTk95AvIZZ8A\n+x7GWneGUnEAgFN4QrQqYRjRw/OiMrS5nDo0ba7Dn9s11A/WnIyS/Rm7Uv48TqOrqD1qqAnWeK59\nFz7dF1AxVqOvl7JxBtl0wquGc0UtYFs/WztVa5S20lHkHeWBEh/I2ik54wqY4uWrOWywrakxAwqe\nvleKzxDNwn4tJ2o/42dRu9Uwc2My7BM+/gaNsX95+p7PJ5p50q8klLedH+WUtOyHyVJq8HfD63yZ\nOkQIIf8IBktCCHEBgyUhhLiAwZIQQlzAYEkIIS5gsCSEEBfkd/Oi9PR0tGnTBmFhYQgODkZ4eDgy\nMjJQqlQpREdHw8dHrhwEACgo9+m7RopmC/e9KGo9TxuSZ4pvMLqjouzpARqAirL+HiHblUBTUfsC\ntWRDLZe56YbseRXdtKevhCk7xlvC1+4xojTNkMn0QaUnsx3nUl8DJIl24+uuFDV8KkuvV9sqi/0c\nE9BXX+7rl/3lNuRNGqFKGexuLSxKy3L4GC/1VZM3acTAGrLWDj1F7TNt3t1xdo5fX09ftdPfi3Zh\nSk4PKms4sbQhH6tgPrvdeA1EWX0tDBWg4pWcVvauNqcGteknp7LdFJP9fLxUT+xMxFh50Jmy5OrK\ncu7cubjpJs+hZs6ciZCQEKxYsQK33XYbVq9e7WYIQgj5T+M1WB48eBAHDhxAw4YNAQCJiYlo0qQJ\nAKBRo0bYvn37v+ogIYRcC3h9gqdPnz6IjIzE2rVrUbZsWURHR2cGyMOHDyM8PBwrVxpuwQAkpSYh\nqHTQ1fOaEEJyGeOa5dq1a3HPPfegXLmcH49y+6Rk9dnV7XYTNFSkZx1Dr/k31ixPmx1yrlmGA2qy\n9fcIeX3F37BmeQob5eNpeT2zG+zvfzE6ojs8SxtL1OOGMWUJALDboJnWLGFfs3xIr8CHKgSAec0S\ndffImmHNslQ1WTvuKAZve2TNG4Y1SxjXLF8TpYSa9vW1Zrs1Ntby+PPvrFmOlw0BzMZ0W1tjChSG\nAQA+wDOiXZyqImpzzA+DikpBh126BgpaL29oWrM8/ZSovasNJdbx99Ysf9Uafta5c2aAvGapZ44T\nNWOw3LZtG5KTk7Ft2zYcPXoUPj4+8PX1RXp6OgoWLIjU1FQEBASYhiCEkOsCY7CcMePyw/izZs1C\n2bJl8cUXXyA+Ph7t2rVDQkIC6tc37ehBCCHXB66rDl0Klg8++CBGjBiBCxcuoEyZMpg4cSIKFDBv\nEKZUc1tb64TMPm24fd1suH1p+rWs6Xbrjf40Ome/9N+66SQaNfUHADzV6ZRo1/Ok7JDvSNmh8yZn\ndA5pTNbf8XPl25dpYaZBgXhtuGWGvH7cvqd9yeXNBYfRoWd5AMBbCweIdgEIFzVTRZ6yiBe1cmhh\na2/XGsHWrdQOr+sQ8twd1vJGcOUny0s/j4y439Zep3egrXoAAPAO5Ao5n0yWvx/1wheIGtDFoAFA\nGVtL68lQyvM51KoyWbTa/bthKcOQrbRBLmSEVvYCSLYlkzcNvyN3wF+i9mSUfDwAeP1+Wfurof3c\nUfp9aPUwACAat4p24VreJM1VniUADBhw+YuyaJG8Ix8hhFyP8AkeQghxAYMlIYS4gMGSEEJcwGBJ\nCCEuYLAkhBAX5MqGZYQQ8l+HV5aEEOICBktCCHEBgyUhhLiAwZIQQlzAYEkIIS5gsCSEEBcwWBJC\niAtcVx26Wrz00kv46quvoJTCyJEjUaOGoQ7bv0hiYiIGDRqEypUrAwCqVKmCyMjIXPdj//79CAsL\nQ/fu3REaGoqUlJS/v3vmv+hPREQE9u7dCz8/PwBAz549M/dj+reZPHkydu3ahT///BPPPPMMqlev\nnqdz4/Rny5YteTI3aWlpiIiIwMmTJ3HhwgWEhYWhatWqeTY3OfkTHx+fZ+cNcBV2pM0JnYskJibq\nPn36aK21PnDggO7UqVNuHt7Gjh079IABA/Ls+Fpr/fvvv+vQ0FA9evRovXTpUq211hEREXrDhg1a\na62nTp2qly9fnqf+jBgxQm/ZsiXXfLjE9u3bda9evbTWWp86dUo3aNAgT+cmJ3/yam7Wr1+vY2Nj\ntdZa//zzz7p58+Z5Ojc5+ZNXc3OJadOm6Q4dOug1a9ZctbnJ1dvw7du3o2lTzz42lSpVwpkzZ3Du\n3LncdOGawsfHB/Pnz7dtzZGXu2fm5E9eUadOHbz88ssAgGLFiiEtLS1P5yYnfzIyMnLt+Flp1aoV\nevfuDQBISUlB6dKl83RucvInL/m3dqTN1WB54sQJFC9ePLNdokQJHD9+PDddsHHgwAH07dsXTz75\nJD755JNcP37+/PlRsGBBW19aWlrmLYK/v3+uzk9O/gDAsmXL0LVrVwwZMgSnTsmV5K8m+fLlg6+v\nLwBg9erVeOihh/J0bnLyJ1++fHkyN5d44oknMGzYMIwcOTJP5yYnf4C8OW8AICoqChEREZntqzU3\nub5mmRWdh4+lV6hQAf3790fLli2RnJyMrl27IiEhIVfXwLyRl/NziXbt2sHPzw+BgYGIjY3F7Nmz\nMWbMmFw7/qZNm7B69WosXLgQzZtf3p4kr+Ymqz9JSUl5OjcrV67Evn37MHz4cNt85NXcZPVn5MiR\neTI3V2tH2pzI1SvLgIAAnDhxIrN97NgxlCpl2pf036N06dJo1aoVlFIoX748SpYsidTU1DzxJSuX\nds8EcE3snhkcHIzAwEAAQOPGjbF///5cO/ZHH32EmJgYzJ8/H0WLFs3zuXH6k1dzk5SUhJSUFABA\nYGAgMjIyULhw4Tybm5z8qVKlSp7MzbZt27B582Z06tQJb7zxBl555ZWrdt7karCsV68e4uM9G1Tt\n3bsXAQEBKFKkSG66kMm6deuwYIFns6jjx4/j5MmTeb7WAgB169bNnKNrYffMAQMGIDk5GYBn7edS\n9sC/zdmzZzF58mTMmzcv8xfVvJybnPzJq7nZuXMnFi5cCMCztHX+/Pk8nZuc/BkzZkyezM2MGTOw\nZs0arFq1Co8//jjCwsKu2tzkeom2KVOmYOfOnVBKYezYsahatWpuHj6Tc+fOYdiwYfjtt99w8eJF\n9O/fHw0aNMhVH5KSkhAVFYUjR44gf/78KF26NKZMmYKIiIi/tXvmv+lPaGgoYmNjUahQIfj6+mLi\nxInw9/f/132Ji4vDrFmzULFixcy+SZMmYfTo0XkyNzn506FDByxbtizX5yY9PR2jRo1CSkoK0tPT\n0b9/fwQFBf3tXVf/TX98fX0RHR2d63OTlX+yI21OsJ4lIYS4gE/wEEKICxgsCSHEBQyWhBDiAgZL\nQghxAYMlIYS4gMGSEEJcwGBJCCEu+D89aQM7mY3KnAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bd-Hu3Hr_wW9",
        "colab_type": "text"
      },
      "source": [
        "## Make an MLP Encoder/Decoder\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wAWX_eKccAyc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "(train_images, _), (test_images, _) = tf.keras.datasets.fashion_mnist.load_data()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-gXEQUI5A6Vi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_images = train_images.reshape(train_images.shape[0], 28, 28, 1).astype('float32')\n",
        "test_images = test_images.reshape(test_images.shape[0], 28, 28, 1).astype('float32')\n",
        "\n",
        "# Normalizing the images to the range of [0., 1.]\n",
        "train_images /= 255.\n",
        "test_images /= 255.\n",
        "\n",
        "# Binarization\n",
        "train_images[train_images >= .5] = 1.\n",
        "train_images[train_images < .5] = 0.\n",
        "test_images[test_images >= .5] = 1.\n",
        "test_images[test_images < .5] = 0."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rsCm0NWOAAv9",
        "colab_type": "code",
        "cellView": "code",
        "colab": {}
      },
      "source": [
        "#@title EncoderMLP\n",
        "\n",
        "class EncoderGaussMLP(snt.AbstractModule):\n",
        "  \n",
        "  def __init__(self):\n",
        "    super().__init__(name='encoder')\n",
        "  \n",
        "    self._mlp = snt.nets.MLP(output_sizes=[500, 100], \n",
        "                             activation=tf.nn.tanh,\n",
        "                             activate_final=True)\n",
        "\n",
        "    self._mu_layer = snt.Linear(output_size=N_LATENT)\n",
        "\n",
        "    self._log_var_layer = snt.Linear(output_size=N_LATENT)\n",
        "  \n",
        "  def _build(self, input_images: tf.Tensor):\n",
        "    flat_image = snt.BatchFlatten()(input_images)\n",
        "    \n",
        "    encoding = self._mlp(flat_image)\n",
        "    \n",
        "    mu = self._mu_layer(encoding)\n",
        "    \n",
        "    log_variance = self._log_var_layer(encoding)\n",
        "    \n",
        "    return latent_mu, latent_log_var\n",
        "  \n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "etXyUezxAgiP",
        "colab_type": "code",
        "cellView": "code",
        "colab": {}
      },
      "source": [
        "#@title DecoderBernMLP\n",
        "\n",
        "class DecoderBernMLP(snt.AbstractModule):\n",
        "  \n",
        "  def __init__(self):\n",
        "    super().__init__(name='decoder')\n",
        "\n",
        "    self._mlp = snt.nets.MLP(output_sizes=[100, 500], \n",
        "                             activation=tf.nn.tanh,\n",
        "                             activate_final=True)\n",
        "    \n",
        "    self._theta_layer = snt.Linear(output_size=N_OBSERVED)\n",
        "        \n",
        "  def _build(self, latent: tf.Tensor, apply_sigmoid=False):\n",
        "    \n",
        "    decoding = self._mlp(latent)\n",
        "        \n",
        "    logits = self._theta_layer(decoding)\n",
        "    \n",
        "    if apply_sigmoid == True:\n",
        "      return tf.sigmoid(logits)\n",
        "    \n",
        "    return logits"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zm_AevofBr8Z",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class VAE(snt.AbstractModule):\n",
        "  def __init__(self, \n",
        "               encoder: EncoderMLP, \n",
        "               decoder: DecoderMLP):\n",
        "    super().__init__(name='vae')\n",
        "    \n",
        "    self._encoder = encoder\n",
        "    self._decoder = decoder\n",
        "  \n",
        "  def _build(self, input_images:tf.Tensor):  \n",
        "    \n",
        "    latent_mu, latent_log_var = self._encoder(input_images)\n",
        "    latent_sample = self._sample_latent_z(latent_mu, latent_log_var)\n",
        "    logits = self._decoder(latent_sample)\n",
        "    \n",
        "    return logits, latent_mu, latent_log_var\n",
        "  \n",
        "  def _sample_latent_z(self, mean, logvar, noise=None): # Produce a sample in the latent space\n",
        "    if noise == None:\n",
        "      noise = tf.random.normal(shape=mean.shape)\n",
        "    return noise * tf.exp(logvar * .5) + mean\n",
        "    \n",
        "  def generate_sample_x(self, noise = None): # Produce a sample in the input space\n",
        "    self._sample_latent_z(\n",
        "    return self._decoder.() \n",
        "  \n",
        "  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1AD2dPlRounh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "N_TRAINING_STEPS = 6000\n",
        "N_LATENT = 2\n",
        "N_OBSERVED = 784\n",
        "BATCH_SIZE = 128\n",
        "N_TRAINING_SAMPLES = train_images.shape[0]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EKElexg1gS5C",
        "colab_type": "code",
        "cellView": "code",
        "colab": {}
      },
      "source": [
        "#@title Build the Graph\n",
        "\n",
        "encoder = EncoderMLP()\n",
        "decoder = DecoderBernMLP()\n",
        "\n",
        "vae = VAE(encoder, decoder)\n",
        "\n",
        "input_batch = tf.placeholder(tf.float32, shape=(BATCH_SIZE, 28, 28))\n",
        "\n",
        "theta, latent_mu, latent_log_var = vae(input_batch)\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Iw54FryvjjA0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K7I9HZzWNops",
        "colab_type": "code",
        "cellView": "code",
        "colab": {}
      },
      "source": [
        "#@title Build the Graph\n",
        "\n",
        "tf.reset_default_graph()\n",
        "\n",
        "input_batch = tf.placeholder(tf.float32, shape=(BATCH_SIZE, 28, 28))\n",
        "    \n",
        "encoder = EncoderMLP()\n",
        "decoder = DecoderMLP()\n",
        "\n",
        "\n",
        "# Encode the inputs into the latent space\n",
        "latent_mu, latent_log_var, flat_image, encoding = encoder(input_batch)\n",
        "\n",
        "# Sample from the posterior over the latents\n",
        "noise = tf.random_normal(shape=[N_LATENT])\n",
        "\n",
        "latent = latent_mu + noise * latent_log_var\n",
        "\n",
        "test_mu = decoder(latent_mu)\n",
        "\n",
        "# Decode the sample latents into the input space\n",
        "dec_mu = decoder(latent)\n",
        "\n",
        "# Compute the Reconstruction Loss term\n",
        "\n",
        "flat_input = snt.BatchFlatten()(input_batch)  # [B, HW]\n",
        "\n",
        "# recon loss for binarized data is just binary cross-entropy\n",
        "recon_loss = - (flat_input * tf.log(dec_mu + 1e-15) \n",
        "                + (1 - flat_input) * tf.log(1-dec_mu + 1e-15))  # [B, HW]\n",
        "\n",
        "recon_loss = tf.reduce_sum(recon_loss, axis=-1)  # [B]  (sum -log likelihoods)\n",
        "recon_loss = tf.reduce_mean(recon_loss)  # []  (reduce mean over batch)\n",
        "\n",
        "# Compute the KL Loss term\n",
        "latent_variance = tf.exp(latent_log_var)  # [B, L]\n",
        "\n",
        "kl_loss = 0.5 * (latent_variance + latent_mu**2 - latent_log_var - 1)  # [B, L]\n",
        "kl_loss = tf.reduce_sum(kl_loss, axis=-1)  # [B]\n",
        "kl_loss = tf.reduce_mean(kl_loss)  # []\n",
        "\n",
        "# Combine the loss terms\n",
        "loss = recon_loss + kl_loss  # []\n",
        "\n",
        "# Pick an optimizer and add the minimizer op to the graph\n",
        "optimizer = tf.train.AdamOptimizer(learning_rate=1e-4)\n",
        "sgd_op = optimizer.minimize(loss)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ULbF0i0HYvw5",
        "colab_type": "code",
        "outputId": "b1d70e1e-f630-4659-881e-76cbd7da2bcc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 926
        }
      },
      "source": [
        "# Run the Graph\n",
        "\n",
        "fetches = {\n",
        "           'sgd_op': sgd_op,\n",
        "           'loss': loss,\n",
        "            'dec_mu': tf.reduce_sum(dec_mu),\n",
        "           #'latent': latent,\n",
        "#            'dec_mu': tf.reduce_mean(dec_mu),\n",
        "#            'test_mu': test_mu,\n",
        "#            'recon_a': tf.reduce_sum(tf.log(dec_var)),\n",
        "#            'recon_b': tf.reduce_sum(sq_error / dec_var),\n",
        "           'recon_loss': recon_loss,\n",
        "           'kl_loss': kl_loss,\n",
        "    'test_mu': test_mu,\n",
        "    'input_batch': input_batch,\n",
        "          }\n",
        "sess = tf.Session()\n",
        "sess.run(tf.global_variables_initializer())\n",
        "for step in range(N_TRAINING_STEPS):\n",
        "\n",
        "  batch_indexes = np.random.choice(N_TRAINING_SAMPLES, BATCH_SIZE)\n",
        "  mini_batch = train_images[batch_indexes,:,:].astype(np.float32).squeeze(axis=-1)\n",
        "  np_outputs = sess.run(fetches, feed_dict={input_batch: mini_batch})\n",
        "\n",
        "  if step % (N_TRAINING_STEPS // 50) == 0:\n",
        "    truth = np_outputs.pop('input_batch')\n",
        "    reconstructed = np_outputs.pop('test_mu')\n",
        "    print(np_outputs)\n",
        "#       print('Step: {}, Loss: {}, Recon. Loss: {}, kl_loss: {}'.format(step,\n",
        "#           np_outputs['loss'], np_outputs['recon_loss'], np_outputs['kl_loss']))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'sgd_op': None, 'loss': 549.42267, 'dec_mu': 50126.703, 'recon_loss': 549.25494, 'kl_loss': 0.16769901}\n",
            "{'sgd_op': None, 'loss': 333.0927, 'dec_mu': 33150.875, 'recon_loss': 329.27692, 'kl_loss': 3.815787}\n",
            "{'sgd_op': None, 'loss': 289.15002, 'dec_mu': 33925.945, 'recon_loss': 286.43164, 'kl_loss': 2.7183986}\n",
            "{'sgd_op': None, 'loss': 284.25897, 'dec_mu': 33138.355, 'recon_loss': 282.51556, 'kl_loss': 1.7434018}\n",
            "{'sgd_op': None, 'loss': 264.55307, 'dec_mu': 30248.844, 'recon_loss': 262.60864, 'kl_loss': 1.9444281}\n",
            "{'sgd_op': None, 'loss': 265.0794, 'dec_mu': 31521.232, 'recon_loss': 263.21985, 'kl_loss': 1.8595575}\n",
            "{'sgd_op': None, 'loss': 240.96706, 'dec_mu': 29871.453, 'recon_loss': 238.70335, 'kl_loss': 2.2636983}\n",
            "{'sgd_op': None, 'loss': 235.00769, 'dec_mu': 32276.723, 'recon_loss': 232.49045, 'kl_loss': 2.5172422}\n",
            "{'sgd_op': None, 'loss': 234.04305, 'dec_mu': 32867.707, 'recon_loss': 231.61987, 'kl_loss': 2.4231744}\n",
            "{'sgd_op': None, 'loss': 242.2762, 'dec_mu': 32558.883, 'recon_loss': 240.40015, 'kl_loss': 1.8760478}\n",
            "{'sgd_op': None, 'loss': 222.79446, 'dec_mu': 28816.28, 'recon_loss': 220.7079, 'kl_loss': 2.086567}\n",
            "{'sgd_op': None, 'loss': 219.81287, 'dec_mu': 32902.78, 'recon_loss': 217.65662, 'kl_loss': 2.1562428}\n",
            "{'sgd_op': None, 'loss': 227.17154, 'dec_mu': 28339.984, 'recon_loss': 225.18005, 'kl_loss': 1.9914844}\n",
            "{'sgd_op': None, 'loss': 227.86751, 'dec_mu': 31580.078, 'recon_loss': 226.01685, 'kl_loss': 1.8506553}\n",
            "{'sgd_op': None, 'loss': 230.2058, 'dec_mu': 30690.0, 'recon_loss': 228.17793, 'kl_loss': 2.027856}\n",
            "{'sgd_op': None, 'loss': 230.94473, 'dec_mu': 29945.156, 'recon_loss': 229.1618, 'kl_loss': 1.7829319}\n",
            "{'sgd_op': None, 'loss': 223.70378, 'dec_mu': 32700.918, 'recon_loss': 221.81146, 'kl_loss': 1.8923175}\n",
            "{'sgd_op': None, 'loss': 211.13516, 'dec_mu': 32806.14, 'recon_loss': 208.89474, 'kl_loss': 2.240419}\n",
            "{'sgd_op': None, 'loss': 229.03938, 'dec_mu': 31335.477, 'recon_loss': 227.13791, 'kl_loss': 1.9014809}\n",
            "{'sgd_op': None, 'loss': 222.9147, 'dec_mu': 31953.322, 'recon_loss': 221.23845, 'kl_loss': 1.6762613}\n",
            "{'sgd_op': None, 'loss': 218.95581, 'dec_mu': 30629.291, 'recon_loss': 217.18343, 'kl_loss': 1.772378}\n",
            "{'sgd_op': None, 'loss': 219.27379, 'dec_mu': 34067.496, 'recon_loss': 217.57861, 'kl_loss': 1.6951692}\n",
            "{'sgd_op': None, 'loss': 208.4253, 'dec_mu': 30412.121, 'recon_loss': 206.54489, 'kl_loss': 1.8804083}\n",
            "{'sgd_op': None, 'loss': 225.26321, 'dec_mu': 32595.768, 'recon_loss': 223.13705, 'kl_loss': 2.1261537}\n",
            "{'sgd_op': None, 'loss': 215.80708, 'dec_mu': 30217.27, 'recon_loss': 213.67236, 'kl_loss': 2.134713}\n",
            "{'sgd_op': None, 'loss': 211.49223, 'dec_mu': 29598.658, 'recon_loss': 209.54492, 'kl_loss': 1.9473169}\n",
            "{'sgd_op': None, 'loss': 215.8091, 'dec_mu': 37178.883, 'recon_loss': 213.90436, 'kl_loss': 1.9047405}\n",
            "{'sgd_op': None, 'loss': 222.65442, 'dec_mu': 35385.805, 'recon_loss': 221.06572, 'kl_loss': 1.5886921}\n",
            "{'sgd_op': None, 'loss': 197.66559, 'dec_mu': 33478.56, 'recon_loss': 195.75134, 'kl_loss': 1.9142451}\n",
            "{'sgd_op': None, 'loss': 213.3292, 'dec_mu': 32644.613, 'recon_loss': 211.48236, 'kl_loss': 1.8468248}\n",
            "{'sgd_op': None, 'loss': 213.58461, 'dec_mu': 33373.133, 'recon_loss': 211.47134, 'kl_loss': 2.1132648}\n",
            "{'sgd_op': None, 'loss': 210.60124, 'dec_mu': 30992.484, 'recon_loss': 208.49377, 'kl_loss': 2.1074705}\n",
            "{'sgd_op': None, 'loss': 211.98373, 'dec_mu': 33433.734, 'recon_loss': 210.25818, 'kl_loss': 1.7255592}\n",
            "{'sgd_op': None, 'loss': 192.5676, 'dec_mu': 31625.492, 'recon_loss': 190.74054, 'kl_loss': 1.8270595}\n",
            "{'sgd_op': None, 'loss': 202.49765, 'dec_mu': 31374.826, 'recon_loss': 200.35825, 'kl_loss': 2.1394038}\n",
            "{'sgd_op': None, 'loss': 210.69203, 'dec_mu': 32662.719, 'recon_loss': 208.60739, 'kl_loss': 2.0846438}\n",
            "{'sgd_op': None, 'loss': 218.01297, 'dec_mu': 33303.883, 'recon_loss': 216.29623, 'kl_loss': 1.7167431}\n",
            "{'sgd_op': None, 'loss': 203.24138, 'dec_mu': 35215.22, 'recon_loss': 201.2674, 'kl_loss': 1.9739863}\n",
            "{'sgd_op': None, 'loss': 197.09256, 'dec_mu': 32148.723, 'recon_loss': 194.87698, 'kl_loss': 2.2155735}\n",
            "{'sgd_op': None, 'loss': 216.86229, 'dec_mu': 31608.023, 'recon_loss': 215.17874, 'kl_loss': 1.6835525}\n",
            "{'sgd_op': None, 'loss': 203.83833, 'dec_mu': 30733.074, 'recon_loss': 201.71304, 'kl_loss': 2.1252851}\n",
            "{'sgd_op': None, 'loss': 194.14667, 'dec_mu': 30451.273, 'recon_loss': 192.20317, 'kl_loss': 1.9434932}\n",
            "{'sgd_op': None, 'loss': 209.07292, 'dec_mu': 31390.52, 'recon_loss': 206.97803, 'kl_loss': 2.0948977}\n",
            "{'sgd_op': None, 'loss': 214.05865, 'dec_mu': 30183.816, 'recon_loss': 212.49359, 'kl_loss': 1.5650635}\n",
            "{'sgd_op': None, 'loss': 198.7952, 'dec_mu': 32008.646, 'recon_loss': 196.93306, 'kl_loss': 1.862142}\n",
            "{'sgd_op': None, 'loss': 197.34239, 'dec_mu': 31366.484, 'recon_loss': 195.52551, 'kl_loss': 1.8168778}\n",
            "{'sgd_op': None, 'loss': 192.89459, 'dec_mu': 29936.227, 'recon_loss': 190.95142, 'kl_loss': 1.9431707}\n",
            "{'sgd_op': None, 'loss': 186.05423, 'dec_mu': 32265.684, 'recon_loss': 184.08258, 'kl_loss': 1.9716443}\n",
            "{'sgd_op': None, 'loss': 190.8939, 'dec_mu': 30730.676, 'recon_loss': 189.07895, 'kl_loss': 1.8149524}\n",
            "{'sgd_op': None, 'loss': 188.80919, 'dec_mu': 32386.422, 'recon_loss': 186.61432, 'kl_loss': 2.1948705}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6hU-NtlEuwDS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = np.linspace(-3, 3, num=5)\n",
        "y = np.linspace(-3, 3, num=5)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ugN6c0T9wuvd",
        "colab_type": "code",
        "outputId": "d5fbf5ce-0a5a-4a9d-e3e3-bcc16fd176a3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 472
        }
      },
      "source": [
        " np.sort(np.random.randn(25, 2), axis=0).astype(np.float32)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-1.6110584 , -2.3263366 ],\n",
              "       [-1.4989392 , -1.9535222 ],\n",
              "       [-1.416607  , -1.9180276 ],\n",
              "       [-1.2709504 , -1.4216558 ],\n",
              "       [-1.1695166 , -1.3741621 ],\n",
              "       [-0.94293964, -1.2163731 ],\n",
              "       [-0.79104453, -1.1193346 ],\n",
              "       [-0.68776935, -0.9141928 ],\n",
              "       [-0.4738726 ,  0.06631131],\n",
              "       [-0.45178416,  0.11228297],\n",
              "       [-0.4474064 ,  0.11814801],\n",
              "       [-0.4464775 ,  0.15939535],\n",
              "       [-0.37798703,  0.27463874],\n",
              "       [-0.18666294,  0.34359834],\n",
              "       [ 0.1594285 ,  0.5146453 ],\n",
              "       [ 0.22617787,  0.52469444],\n",
              "       [ 0.22908174,  0.617307  ],\n",
              "       [ 0.23722091,  0.65636265],\n",
              "       [ 0.3232248 ,  0.8523616 ],\n",
              "       [ 0.33665165,  0.9091204 ],\n",
              "       [ 0.44892156,  0.91362834],\n",
              "       [ 1.3943167 ,  0.96468204],\n",
              "       [ 1.5853802 ,  1.435113  ],\n",
              "       [ 1.712998  ,  1.5947464 ],\n",
              "       [ 1.9626383 ,  1.7867622 ]], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 185
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jBGh2xCevWeR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "budge_latent_traversal = 3 * np.sort(np.random.randn(25, 2), axis=0).astype(np.float32)\n",
        "budge_latent_traversal[:,1] = np.zeros(25)\n",
        "budge_latent_traversal = tf.convert_to_tensor(budge_latent_traversal)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GmPiN84Muk-8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "traversals = decoder(budge_latent_traversal)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ff3vRwH_wLVb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sik1 = sess.run(traversals)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y3P5U-cDwM-Q",
        "colab_type": "code",
        "outputId": "ffcb55da-9277-4402-83e4-21120c4f37ff",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 365
        }
      },
      "source": [
        "plt.imshow(np.reshape(sik1[13], (28, 28)))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f1baebb7198>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 195
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAFKCAYAAACU6307AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG/RJREFUeJzt3V9sVGX+x/HPdKZDO6WlUNqy9R8b\nrJGouLrBWAxogXWDyUYx2bA2QHbDBbqRgIQYQgRNSEQqMRFNlj+CF5JNmvTKCzdl0WyWGChZsusG\nbgpslrCIpYXaP9v/bH8XxqadmTP9PtNz5kz5vV9XznMez3nOOdMPM+fM9zyRsbGxMQEAMioIewAA\nMBMQlgBgQFgCgAFhCQAGhCUAGBCWAGAQy8VGwv51UiQSMfe1jtVlnXfu3PF9+6Ojo+Z19vf3m/sO\nDg6mba+srFRHR8f469u3b5vX2dPTY+47e/ZsU79EImFe57fffpu2/ec//7nOnz8/qW1oaMi0zljM\n/qfz4IMPmvuWl5eb+86aNSulLRaLpbw3CgqC+Uxk/Rtw+VvJtI7kv41cbl8K+ZOlXzuB4BUWFoY9\nBN+VlJSEPQTfBRWMYcuHrMj6k+W7776rb775RpFIRLt27dKSJUv8HBcA5JWswvLcuXO6evWqmpqa\ndOXKFe3atUtNTU1+jw0A8kZWn9nPnDmj1atXS5IWLVqk7u5u9fX1+TowAMgnWX2y7Ozs1COPPDL+\net68eero6Mh4cd7rmkM+XIuYKIjxRKNR39fpcoOhqKjIl23W1NSk/e98l+kGy4oVK3I4ktyIx+Nh\nDyEQYV+P9eVuuOUObro+6e5wBYG74dO/G15TUzPprvLdcDd8xYoV+utf/zqpbabfDY/H4xoeHp7U\ndjfcDS8oKND//ve/0LYvZfk1vKqqSp2dneOvb968qcrKSl8GBAD5KKuwfOaZZ9TS0iJJunjxoqqq\nqsyfCABgJsrqa/iTTz6pRx55RL/5zW8UiUT09ttv+z0uAMgrWV+z3LFjh5/jAIC8FsnFk9K9NjGd\nEiYXyReGM7Fu32uf0l2Itt40kKSRkRFTP5efanV1dZn7nj17Nm37pk2bdOzYsfHXf/7zn83rPHXq\nlLlvb2+vqd/8+fPN6/S6wTM2NpZyvhcuXGha5+9+9zvz9l1u8CxatMjct7q6OqVt4cKF+ve//z2p\nbbo3jbxYb3K5/E173YxK93c13XU6r8eXtQDAXY6wBAADwhIADAhLADAgLAHAgLAEAAPCEgAMCEsA\nMCAsAcAgJxOWZfoFf7YVOy6FRy7bsFYJeFXaFBUVpTwiq7u72/ftuzz2zaWC48knnzQtKysrM6/T\n5SErX375pamfS1VUbW2tedmvf/1r0zozHSeX7SdzefSbV7VNcrtLBZfLuSotLTX1c9knF9YM8Csr\n+GQJAAaEJQAYEJYAYEBYAoABYQkABoQlABgQlgBgQFgCgAFhCQAGhCUAGOSk3DFsLuVOfpQ7Ji8r\nLCw0b99qdHTU3NdlwqZME4FNXOZSwuYy0VRNTY2p3/fff29e5z333OO5bNOmTZNeP/zww6Z1PvDA\nA+btuxz/wcFBc9/i4uK07cnvd69+6QRRGuxy/l2OVRCTG2bCJ0sAMCAsAcCAsAQAA8ISAAwISwAw\nICwBwICwBAADwhIADAhLADAgLAHAIO/KHV1KE4NgLc3KNM7kZS4zMVpL0xKJhHmdLtvPtN558+aN\n/3dJSYl5nS597733XlM/l3LPTLMQ/uIXv5j02jq7YUVFhXn7LuV+RUVF015v8vEeGBgwr9OljNX6\nt+qy/y5/V9bSSGZ3BIAcIiwBwICwBAADwhIADAhLADAgLAHAgLAEAAPCEgAMCEsAMMi7Ch6rICYh\nc+k7PDxsXuZXBUG2/JowbWJ1h8vEUi7bt1Ym9ff3m9eZqdpm0aJFk15bz7/LPgV1/uPxuKndZfte\nE/GlYz0GfkyCFo1GU5YxYRkA5KGsPlm2trZq69atqq2tlSQ99NBD2r17t68DA4B8kvXX8KeeekoH\nDx70cywAkLf4Gg4ABlmH5eXLl/Xqq6/qlVde0ddff+3nmAAg70TGsniAZHt7u86fP681a9bo2rVr\n2rhxo06ePOl5dw4AZrqsrllWV1frhRdekCTdf//9mj9/vtrb23XfffdNe0DW7A7qp0PWn07897//\nTds+f/58dXZ2Tmpzefiu9UG50WjUvE6Xn/l4mTVrloaGhsZfuxxTlwf1eh3XZH78dGjOnDnq7u6e\n1DaTfjqU7j1QXFyc8rDfiedtKi4/HbL+zMvlgcJe79XCwsKUsbn8DUx3+1KWX8M///xzHTt2TJLU\n0dGhW7duqbq6OrvRAcAMkNUny5UrV2rHjh368ssvNTIyonfeeYev4ADualmF5ezZs3Xo0CG/xwIA\neSsn5Y5e12wikUjKsiCuWfpRbpUs47WNpGWZSiOD2H4yl/23HleXdbpcs7JO2ObyTSbT9pOXWffL\nr0m4psPrWnhyu8vxd7kWaz1WYU9C6FdZJL+zBAADwhIADAhLADAgLAHAgLAEAAPCEgAMCEsAMCAs\nAcCAsAQAA8ISAAzybnbHIEqjXB4RZi2Ncimhsz52TJL6+vpM/Vye8uTHI7KSl7mU+7mwPnbLr3LP\n5GXW8+9SQucyVj9mYkxud3n/FxUVmftaj4HLo9Qy7X/yMut7cLqPvfsRnywBwICwBAADwhIADAhL\nADAgLAHAgLAEAAPCEgAMCEsAMCAsAcDg/0UFTxATdrlwmQTq+vXrpn4uVRH33HOPuW+map+Jx9Gl\nKsWl2sc6EZlLVYrL5HJ+rDOZy/tvaGjI3PfWrVspbXPnzk1pnzVrlnmd1gnjJHtlmF8ThoWNT5YA\nYEBYAoABYQkABoQlABgQlgBgQFgCgAFhCQAGhCUAGBCWAGBAWAKAQd6VO1pLo1zKEnNdQpm8zGX7\n7e3tpn4u5XYLFiww98203olli0FN2GUtjcy2THEqQZTmuZR7Wiesk6SrV6+mtD344IMp7TU1NeZ1\nVlRUmPtaubz/XY7/nTt3TP1cJuzLhE+WAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoABYQkABoQlABgQ\nlgBgQFgCgEFOyh29yp0ikUjKMpfSMCuX0jhr30yzCybvg0u5lXX/XWY3tJaFSVJRUZHnMpcZJSdy\nKXcLqozRSy72yaXvyMiIuW9bW1tK26pVq1La77//fvM6gygNdjmnQZRG+rVPpr1oa2vT6tWrdeLE\nCUnSjRs3tGHDBjU0NGjr1q0aHh72ZTAAkK+mDMv+/n7t3btXdXV1420HDx5UQ0OD/vjHP+qBBx5Q\nc3NzoIMEgLBNGZbxeFxHjx5VVVXVeFtra6tWrVolSaqvr9eZM2eCGyEA5IEpL6bFYrGUa24DAwOK\nx+OSfnikU0dHRzCjA4A8Me0bPJaLp5FIxPNibPLF31xf4E9WWFho6ldcXOy5bM6cORlfZ3LfffeZ\n++ZatjdD8pn1fOfKwoULzX1fe+01p/aZ7scPaGHJKiwTiYQGBwdVVFSk9vb2SV/R0xkbG0sbqgUF\nBSl3f13u3AbBepfZ66bWnDlz1N3dPamtp6fHvP3W1lZTv+rqavM6H3/8cXPfkpKStO3RaDTrcxP2\ng5q9FBYWOt19nshlnC7HzfrwZ0n605/+lNL22muv6Q9/+MOkttWrV5vX6fKgYOs/ni7/yHod13g8\nnvWNZJcPYJl+uZLVx7hly5appaVFknTy5EktX748m9UAwIwx5SfLCxcuaP/+/bp+/bpisZhaWlp0\n4MAB7dy5U01NTaqpqdFLL72Ui7ECQGimDMtHH31Un332WUr7p59+GsiAACAf/b+YsMyF9fpKpmsb\nyctcLkzPmjXL1M+lgieIm2YuE0u59LWeV7+qgpLPdy6vmabjUu01f/58U7vLNdMg3lcu5z9TBVvY\n54bacAAwICwBwICwBAADwhIADAhLADAgLAHAgLAEAAPCEgAMCEsAMCAsAcAg1AnL0i2zljS5lFC5\nsE4Ylmn7yctcHlHlVcKWzOUZmS4ylbBNXBZ26VlQ5z+I9bqs06U01as0NrndpYTR5b0axLGyvv8k\n+3759RxWPlkCgAFhCQAGhCUAGBCWAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoABYQkABnk3u6O13Mul\n3M6lLMtaGuUyY561hNKFy4yRLjMGeh3XSCQyaVnY5aYu59+l3C3scluX82qd3bGkpGRaY/ISxOyO\nLuuxnle//v74ZAkABoQlABgQlgBgQFgCgAFhCQAGhCUAGBCWAGBAWAKAAWEJAAZ5V8ETxERYLhUE\n1l/7u0ys5LL94uJiUz+vyarScTmm1snlgjimLusNYp+C4rINl2qrkZERU/vAwIB5nS7nNdeTu2W7\nPSYsA4AcIiwBwICwBAADwhIADAhLADAgLAHAgLAEAAPCEgAMCEsAMCAsAcAgJ+WOLuVmQUxY5sJa\nUuVS7uhSblVUVORrP8m/ErZsy81c/r8gJgwLYp+CmjDPpW9ZWZmpvaOjw7zOmpoac9/CwkJzX6sg\nSlNdym0z/a3yyRIADExh2dbWptWrV+vEiROSpJ07d+pXv/qVNmzYoA0bNugvf/lLkGMEgNBN+TW8\nv79fe/fuVV1d3aT27du3q76+PrCBAUA+mfKTZTwe19GjR1VVVZWL8QBAXoqMGa+afvTRR5o7d67W\nr1+vnTt3qqOjQyMjI6qoqNDu3bs1b948z/93bGwskGffAUCuZHU3/MUXX1R5ebkWL16sI0eO6OOP\nP9aePXs8+9+5cyf9xmMxjY6OTmoL4m54EA+q9eoXj8c1PDw8qa2vr8+8/Zs3b5r6VVRUmNdZWlpq\n7huPx9O2FxQUTNrnIO5wu/R1WafXeyp5n1y4bN9lG/39/ea+//rXv1LannjiCf3973+f1NbT02Ne\n5+OPP27um0gkTP1cfg3idVzTZYX1uLq8VzPd4c/qbnhdXZ0WL14sSVq5cqXa2tqyWQ0AzBhZheWW\nLVt07do1SVJra6tqa2t9HRQA5Jspv4ZfuHBB+/fv1/Xr1xWLxdTS0qL169dr27ZtKi4uViKR0L59\n+3IxVgAIzZRh+eijj+qzzz5Laf/lL38ZyIAAIB/lpNzRpdwsF7PtZRJEuaPXDa50rCVkLqVmfs2u\nOHFZEDM25oMg3n9Bze7odTMuud2v2Q2TWW/G+nX+k9djXa9f26fcEQAMCEsAMCAsAcCAsAQAA8IS\nAAwISwAwICwBwICwBAADwhIADAhLADDISbljEIIqobOuN1MJY3IpoMtYvUrYkrmUxVnL0lwEsU4p\n9+WGuSivDeq96vWc0uT2wcFB8zqDmIkyqGefUu4IAHmIsAQAA8ISAAwISwAwICwBwICwBAADwhIA\nDAhLADAgLAHAIO8mLAubdSIul6qQ/v5+8/atFQwuE4a5VEV49Y1EIllXu/ix/emsM4j3mMv2Xc6V\nS2VWcXGxqX14eNi8TpfJ9azHNexJCP3CJ0sAMCAsAcCAsAQAA8ISAAwISwAwICwBwICwBAADwhIA\nDAhLADAgLAHAICflji6lgflW/ujFZZ8GBgbM6y0pKZn29pP5VRo5cVnYE8a5CHvCMpfJ3VzGU1hY\naGp3Of8upbllZWWmfkFNbpdrd8deAEDACEsAMCAsAcCAsAQAA8ISAAwISwAwICwBwICwBAADwhIA\nDAhLADDISbmjC2u5l0tZXBCzC2YqIUteNjQ0ZN6+14x9yVzK0kpLS819gygNDKI002Wd0WjU3Ncq\niPeU5Faa6LVfye2Dg4Pmdbq8V60zQc6k0thMTGHZ2Nio8+fPa3R0VJs3b9Zjjz2mN998U3fu3FFl\nZaXef/99xePxoMcKAKGZMizPnj2rS5cuqampSV1dXVq7dq3q6urU0NCgNWvW6IMPPlBzc7MaGhpy\nMV4ACMWU1yyXLl2qDz/8UNIPTxkZGBhQa2urVq1aJUmqr6/XmTNngh0lAIRsyrCMRqNKJBKSpObm\nZq1YsUIDAwPjX7srKirU0dER7CgBIGTmGzynTp1Sc3Ozjh8/rueff3683XLxuqCgwPNibBAX36fD\nOh6vZwlKGv/H5UdLliyZ1pjyRSyWd/cDp20m71NRUVHa9uQbej9+C5zpws4K0zvl9OnTOnTokD75\n5BOVlpYqkUhocHBQRUVFam9vV1VVVcb/3+sOXzQaNd9RSxbU3XDr3ciRkZG07YlEIuVO9eXLl83b\nLy8vN/XLFNbJXO6Ge/0BxmIxjY6Ojr8O+/j7cTc8eZ9c5MPd8HTvwdLSUvX29k5qO3funHmdCxcu\nNPe99957Tf38+AdpOlnhuh0vU34N7+3tVWNjow4fPjz+h7xs2TK1tLRIkk6ePKnly5f7NFQAyE9T\nRv4XX3yhrq4ubdu2bbztvffe01tvvaWmpibV1NTopZdeCnSQABC2KcNy3bp1WrduXUr7p59+GsiA\nACAf5d2EZbnefjLrdZFM17qSl92+fdu8feuP+62VPunGk4n1XLlcP3K5vmm9ZueyTq+xxmKxlGVB\nbD+o67vDw8Omdpfte12LT8ePajeraDSash7rfvlV6UNtOAAYEJYAYEBYAoABYQkABoQlABgQlgBg\nQFgCgAFhCQAGhCUAGBCWAGCQk3LHTOVGycuCeAxTEI/I6uvrS9teVlaWsqy9vd28/QULFpj6uUxY\nVlJSYu7rVe5WWFg4aVlBgf3fWZdzGkS5YaaxZltC57L/033smhevsSa3ez12Lx2XB3lbH9HmItPx\nTz6O1udb+lVSzSdLADAgLAHAgLAEAAPCEgAMCEsAMCAsAcCAsAQAA8ISAAwISwAwICwBwCAn5Y4u\nrOVmLiV0LuVOQcw26VLuZt1+ECWckn3WSmupmeRWwmc9r7HY9N+6xcXFKTMhWmfXDKKEU3Ir4/Qa\na3J7aWmpeZ2dnZ3mvn7M2pjMZSZY6zlwea9mwidLADAgLAHAgLAEAAPCEgAMCEsAMCAsAcCAsAQA\nA8ISAAwISwAwyLsKHmtVQBBVMZI0NDRk6pdc+ZFpmUtVhrVvT0+PeZ0u1S5lZWWe7S6TpE0UxLly\n2adMfZMrlqwTkblMWOZSweTC61glb8/6npYyv6+TZar2ylamc5Xt5HJU8ABADhGWAGBAWAKAAWEJ\nAAaEJQAYEJYAYEBYAoABYQkABoQlABgQlgBgkJNyR5dJiKzlbkGUEEr2Eq7vv//evMylhMxaGucy\nYZZLmaLXsVqwYIF6e3vHX8+aNcu8TpdyU+t+uawz01iTS+Gs23cp9RsYGDD3dSmN9JqwbHBwcNJr\nl9LQvr4+c19rGavL31+mMlKXEtNst5+J6Sg2Njbq/PnzGh0d1ebNm/XVV1/p4sWLKi8vlyRt2rRJ\nzz33nC8DAoB8NGVYnj17VpcuXVJTU5O6urq0du1aPf3009q+fbvq6+tzMUYACN2UYbl06VItWbJE\n0g9PnhkYGHD6CggAd4MpLwJEo1ElEglJUnNzs1asWKFoNKoTJ05o48aNeuONN3T79u3ABwoAYYqM\nGa+Unzp1SocPH9bx48d14cIFlZeXa/HixTpy5Ii+++477dmzx/P/HRsb8+0iKwCEwXSD5/Tp0zp0\n6JA++eQTlZaWqq6ubnzZypUr9c4772T8/72+tsdisZS7ita7jC53Q11MvOObybfffpu2/Wc/+5n+\n8Y9/TGr75z//ad7+E088YepnHaek8W8GFiUlJWnba2trdenSpfHXYd8N97oTnM7s2bPTtpeXl2f8\nVUMmLvuUy7vhP/nJT3Tjxo1Jbe3t7eZ1Xrlyxdz32WefNfVzea8UFhambS8qKkq5yz/ddaaT6UHB\nU34N7+3tVWNjow4fPjx+93vLli26du2aJKm1tVW1tbXmwQDATDTlJ8svvvhCXV1d2rZt23jbyy+/\nrG3btqm4uFiJREL79u0LdJAAELYpw3LdunVat25dSvvatWsDGRAA5CPKHQHAICfljpkummY785rL\njIEuF82t5V6Z+iUvcykhs16Mdvl1wX/+8x9zX6+bIbW1tbp+/XpW2/daZzrWmyFz5841r9OrTK68\nvDylFNRaGupS7ujS1+XGldeNk+n88uS7774z9+3q6jL1q6ysNK8zU2lm8k0163769UscPlkCgAFh\nCQAGhCUAGBCWAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoCB+XmW0+G1iUgkEtij1iZyqaCwTi42NDSU\ntn3evHkpD0N2mTDMWpni8rR6lyopr/WWlZWpp6dn/LVLVYTLObYef5dJuLz6zp49O6W6ynpcXY6p\ny0Rb0z2uiUQi5f3m8miz7u5uc19rZZZLBZdXBVu6xzlaj5Vfx59PlgBgQFgCgAFhCQAGhCUAGBCW\nAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoBBTsodAWCm45MlABgQlgBgQFgCgAFhCQAGhCUAGBCWAGBg\nf9y0j95991198803ikQi2rVrl5YsWRLGMHzV2tqqrVu3qra2VpL00EMPaffu3SGPKnttbW36/e9/\nr9/+9rdav369bty4oTfffFN37txRZWWl3n//fcXj8bCH6SR5n3bu3KmLFy+qvLxckrRp0yY999xz\n4Q7SUWNjo86fP6/R0VFt3rxZjz322Iw/T1Lqfn311Vehn6uch+W5c+d09epVNTU16cqVK9q1a5ea\nmppyPYxAPPXUUzp48GDYw5i2/v5+7d27V3V1deNtBw8eVENDg9asWaMPPvhAzc3NamhoCHGUbtLt\nkyRt375d9fX1IY1qes6ePatLly6pqalJXV1dWrt2rerq6mb0eZLS79fTTz8d+rnK+dfwM2fOaPXq\n1ZKkRYsWqbu7O2UeFIQrHo/r6NGjqqqqGm9rbW3VqlWrJEn19fU6c+ZMWMPLSrp9mumWLl2qDz/8\nUNIPcyQNDAzM+PMkpd8vlzmngpLzsOzs7Jw0Kde8efPU0dGR62EE4vLly3r11Vf1yiuv6Ouvvw57\nOFmLxWIqKiqa1DYwMDD+da6iomLGnbN0+yRJJ06c0MaNG/XGG2+kTDSX76LRqBKJhCSpublZK1as\nmPHnSUq/X9FoNPRzFco1y4nulmrLhQsX6vXXX9eaNWt07do1bdy4USdPnpyR14umcrecsxdffFHl\n5eVavHixjhw5oo8//lh79uwJe1jOTp06pebmZh0/flzPP//8ePtMP08T9+vChQuhn6ucf7KsqqpS\nZ2fn+OubN2+qsrIy18PwXXV1tV544QVFIhHdf//9mj9/vtrb28Melm8SicT4lKrt7e13xdfZuro6\nLV68WJK0cuVKtbW1hTwid6dPn9ahQ4d09OhRlZaW3jXnKXm/8uFc5Twsn3nmGbW0tEiSLl68qKqq\nKqd5hfPV559/rmPHjkmSOjo6dOvWLVVXV4c8Kv8sW7Zs/LydPHlSy5cvD3lE07dlyxZdu3ZN0g/X\nZH/8JcNM0dvbq8bGRh0+fHj8LvHdcJ7S7Vc+nKtQnjp04MAB/e1vf1MkEtHbb7+thx9+ONdD8F1f\nX5927Nihnp4ejYyM6PXXX9ezzz4b9rCycuHCBe3fv1/Xr19XLBZTdXW1Dhw4oJ07d2poaEg1NTXa\nt2+fCgsLwx6qWbp9Wr9+vY4cOaLi4mIlEgnt27dPFRUVYQ/VrKmpSR999JF++tOfjre99957euut\nt2bseZLS79fLL7+sEydOhHqueEQbABhQwQMABoQlABgQlgBgQFgCgAFhCQAGhCUAGBCWAGBAWAKA\nwf8BecOWlgqHlGMAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ErrG-AjMvEle",
        "colab_type": "code",
        "outputId": "e1785a32-fc76-4f39-86c0-36782d8d7b6b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "yy.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(50, 50)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 150
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yrniya0krc1v",
        "colab_type": "code",
        "outputId": "e9b1e5a4-310b-4109-d103-4cb4cb2cc540",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 365
        }
      },
      "source": [
        "plt.imshow(truth[6])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f1bc3c464a8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 142
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAFKCAYAAACU6307AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAD2NJREFUeJzt3V9IVPn/x/HX+ToNOlSYkzNLF+1G\nGCtLXiwYjdEfTRYMlqirTSwWuigWpVYiRNIuhCyLIOvCP1t7IwsDXnWxoEQsROhEXgR6Y3UREq2N\nrVSitirzvfjxk2/f1e+8HWfmzOjzAV042PI+nbPPPjPHT8eJxWIxAQD+p3+5PQAAZANiCQAGxBIA\nDIglABgQSwAwIJYAYEAsAcCAWAKAgSfR33jlyhU9e/ZMjuOosbFRJSUlyZwLADJKQrF88uSJXr16\npXA4rJcvX6qxsVHhcDjZswFAxkjobfjAwIAqKyslSTt37tT79+81NTWV1MEAIJMkFMuJiQlt2bJl\n8euCggJFo9GkDQUAmSYpN3j4tzgArHUJxTIQCGhiYmLx67dv36qwsDBpQwFApkkolvv27VNfX58k\naWRkRIFAQBs3bkzqYACQSRK6G/7tt9/qm2++0Q8//CDHcXT58uVkzwUAGcXhH/8FgPgS/qF0rG+O\n47g9wqrFYrGEj4M1xvrDdkcAMCCWAGBALAHAgFgCgAGxBAADYgkABsQSAAyIJQAYEEsAMGAHDxKS\nqh0sbu8MYmcOlsPKEgAMiCUAGBBLADAglgBgQCwBwIBYAoABsQQAA2IJAAbEEgAMiCUAGPB0RwAw\nYGUJAAbEEgAMiCUAGBBLADAglgBgQCwBwIBYAoABsQQAA2IJAAbEEgAMiCUAGBBLADAglgBgQCwB\nwIBYAoABsQQAA2IJAAbEEgAMiCUAGBBLADAglgBgQCwBwMCTyG+KRCI6d+6cioqKJEm7du1SU1NT\nUgcDgEySUCwlac+ePWpvb0/mLACQsXgbDgAGCcfyxYsXOnv2rE6cOKHHjx8ncyYAyDhOLBaLrfQ3\njY+Pa2hoSFVVVRobG9OpU6fU398vr9ebihkBwHUJrSyDwaCOHDkix3G0fft2bd26VePj48meDQAy\nRkKxvH//vu7evStJikajevfunYLBYFIHA4BMktDb8KmpKV24cEEfPnzQ3NycamtrdfDgwVTMBwAZ\nIaFYAsB6w48OAYABsQQAA2IJAAbEEgAMiCUAGBBLADAglgBgQCwBwIBYAoABsQQAA2IJAAYJP1YC\nqeE4jun72NIPpBcrSwAwIJYAYEAsAcCAWAKAAbEEAANiCQAGxBIADIglABgQSwAwYAdPlrLu9JHY\n7bNWLXUNxGKxf7zO+U8OVpYAYEAsAcCAWAKAAbEEAANiCQAGxBIADIglABgQSwAwIJYAYEAsAcCA\n7Y4Zxro1bSXbHQGsHitLADAglgBgQCwBwIBYAoABsQQAA2IJAAbEEgAMiCUAGBBLADAglgBgwHZH\nYI3jSaDJYVpZjo6OqrKyUj09PZKkN2/e6OTJk6qurta5c+f0999/p3RIAHBb3FhOT0+rpaVFoVBo\n8bX29nZVV1frt99+05dffqne3t6UDgkAbosbS6/Xq+7ubgUCgcXXIpGIDh8+LEkqLy/XwMBA6iYE\ngAwQ9zNLj8cjj+fzb5uZmZHX65Uk+f1+RaPR1EwHABli1Td4+EDYHfy5Y7lrgGsjNRKKpc/n0+zs\nrHJzczU+Pv7ZW3SkB3c4sdQ1EIvFVvUPQ3OtLC+hn7MsKytTX1+fJKm/v1/79+9P6lAAkGmcWJy/\nSoaHh3Xt2jW9fv1aHo9HwWBQN27cUENDgz59+qRt27aptbVVGzZsSNfMECtLsLJMt7ixRGYiliCW\n6cV2RwAwIJYAYEAsAcCAWAKAAbEEAANiCQAGxBIADIglABgQSwAwIJYAYEAsAcCAB5alwWr26gLI\nDKwsAcCAWAKAAbEEAANiCQAGxBIADIglABgQSwAwIJYAYEAsAcCAWAKAAdsdsxSPLAXSi5UlABgQ\nSwAwIJYAYEAsAcCAWAKAAbEEAANiCQAGxBIADIglABgQSwAwIJYAYEAsAcCAWAKAAbEEAANiCQAG\nxBIADIglABgQSwAwIJYAYEAsAcCAWAKAAbEEAANTLEdHR1VZWamenh5JUkNDg77//nudPHlSJ0+e\n1B9//JHKGQHAdXGfGz49Pa2WlhaFQqHPXq+vr1d5eXnKBgOATBJ3Zen1etXd3a1AIJCOeQAgI8Vd\nWXo8Hnk8//y2np4e/frrr/L7/WpqalJBQUFKBlwLYrGY2yNgDVruuuJ6S424sVzK0aNHlZ+fr+Li\nYnV1denOnTtqbm5O9mxrhuM4Sf9v8j8ElrquYrHYqq43rqvlJXQ3PBQKqbi4WJJUUVGh0dHRpA4F\nAJkmoVjW1dVpbGxMkhSJRFRUVJTUoQAg0zixOOvu4eFhXbt2Ta9fv5bH41EwGFRNTY26urqUl5cn\nn8+n1tZW+f3+dM2cdXgbjlTgbXh6xY0lVo9YIhWIZXoldIMHgPusd8NT8Zf1esR2RwAwIJYAYEAs\nAcCAWAKAAbEEAANiCQAGxBIADIglABgQSwAwIJYAYEAsAcCAWAKAAbEEAANiCQAGxBIADIglABgQ\nSwAwIJYAYEAsAcCAWAKAAQ8sA7BoJQ83W29PgmRlCQAGxBIADIglABgQSwAwIJYAYEAsAcCAWAKA\nAbEEAANiCQAGxBIADIglABgQSwAwIJYAYEAsAcCAWAKAAbEEAANiCQAGxBIADIglABgQSwAwIJYA\nYMDTHTPMentiHpAtTLFsa2vT0NCQ5ufndebMGe3evVsXL17UwsKCCgsLdf36dXm93lTPCgCucWJx\nljKDg4O6e/euuru7NTk5qWPHjikUCunAgQOqqqrSzZs39cUXX6i6ujpdM2cdnsUMN63k+luJ9Xat\nxv3MsrS0VLdu3ZIkbd68WTMzM4pEIjp8+LAkqby8XAMDA6mdEgBcFjeWOTk58vl8kqTe3l4dOHBA\nMzMzi2+7/X6/otFoaqcEAJeZ74Y/ePBAvb29am5u/uz19bYUT0QsFjP/ApJtJdcf1+ryTDd4Hj16\npI6ODv3yyy/atGmTfD6fZmdnlZubq/HxcQUCgVTPmdX4zBJu4jPL5Ii7svz48aPa2trU2dmp/Px8\nSVJZWZn6+vokSf39/dq/f39qpwQAl8W9Gx4Oh3X79m3t2LFj8bWrV6/q0qVL+vTpk7Zt26bW1lZt\n2LAh5cNmK1aWcBMry+SIG0usHrGEm4hlcrDdEQAMiCUAGBBLADAglgBgQCwBwIBYAoABsQQAA2IJ\nAAbEEgAMiCUAGBBLADDggWXAGreSPdyp2ke+FrCyBAADYgkABsQSAAyIJQAYEEsAMCCWAGBALAHA\ngFgCgAGxBAADYgkABsQSAAyIJQAYEEsAMCCWAGBALAHAgFgCgAGxBAADYgkABsQSAAyIJQAYEEsA\nMCCWAGBALAHAgFgCgAGxBAADYgkABsQSAAyIJQAYEEsAMCCWAGBALAHAgFgCgIHH8k1tbW0aGhrS\n/Py8zpw5o4cPH2pkZET5+fmSpNOnT+vQoUOpnBMAXBU3loODg3r+/LnC4bAmJyd17Ngx7d27V/X1\n9SovL0/HjADgurixLC0tVUlJiSRp8+bNmpmZ0cLCQsoHA4BM4sRisZj1m8PhsJ4+faqcnBxFo1HN\nzc3J7/erqalJBQUFqZwzqzmOY/7eFZwOIOm4VpdnjuWDBw/U2dmpe/fuaXh4WPn5+SouLlZXV5f+\n/PNPNTc3p3pWAHCN6W74o0eP1NHRoe7ubm3atEmhUEjFxcWSpIqKCo2OjqZ0yGznOI75F+AmrtXl\nxY3lx48f1dbWps7OzsW733V1dRobG5MkRSIRFRUVpXZKAHBZ3Bs8v//+uyYnJ3X+/PnF144fP67z\n588rLy9PPp9Pra2tKR0SANy2ohs8SAwfmiNbcK0ujx08AGBALAHAgFgCgAGxBAADYgkABsQSAAyI\nJQAYEEsAMCCWAGBALAHAgFgCgIHpGTwA1of1tt97JVhZAoABsQQAA2IJAAbEEgAMiCUAGBBLADAg\nlgBgQCwBwIBYAoABsQQAA7Y7pgFbyIDsx8oSAAyIJQAYEEsAMCCWAGBALAHAgFgCgIErPzp05coV\nPXv2TI7jqLGxUSUlJW6MkVSRSETnzp1TUVGRJGnXrl1qampyearEjY6O6qefftKPP/6ompoavXnz\nRhcvXtTCwoIKCwt1/fp1eb1et8dckf8+poaGBo2MjCg/P1+SdPr0aR06dMjdIVeora1NQ0NDmp+f\n15kzZ7R79+6sP0/SP4/r4cOHrp+rtMfyyZMnevXqlcLhsF6+fKnGxkaFw+F0j5ESe/bsUXt7u9tj\nrNr09LRaWloUCoUWX2tvb1d1dbWqqqp08+ZN9fb2qrq62sUpV2apY5Kk+vp6lZeXuzTV6gwODur5\n8+cKh8OanJzUsWPHFAqFsvo8SUsf1969e10/V2l/Gz4wMKDKykpJ0s6dO/X+/XtNTU2lewz8D16v\nV93d3QoEAouvRSIRHT58WJJUXl6ugYEBt8ZLyFLHlO1KS0t169YtSdLmzZs1MzOT9edJWvq4FhYW\nXJ7KhVhOTExoy5Yti18XFBQoGo2me4yUePHihc6ePasTJ07o8ePHbo+TMI/Ho9zc3M9em5mZWXw7\n5/f7s+6cLXVMktTT06NTp07p559/1l9//eXCZInLycmRz+eTJPX29urAgQNZf56kpY8rJyfH9XPl\n+nbHtbIV8KuvvlJtba2qqqo0NjamU6dOqb+/Pys/L4pnrZyzo0ePKj8/X8XFxerq6tKdO3fU3Nzs\n9lgr9uDBA/X29urevXv67rvvFl/P9vP0n8c1PDzs+rlK+8oyEAhoYmJi8eu3b9+qsLAw3WMkXTAY\n1JEjR+Q4jrZv366tW7dqfHzc7bGSxufzaXZ2VpI0Pj6+Jt7OhkIhFRcXS5IqKio0Ojrq8kQr9+jR\nI3V0dKi7u1ubNm1aM+fpv48rE85V2mO5b98+9fX1SZJGRkYUCAS0cePGdI+RdPfv39fdu3clSdFo\nVO/evVMwGHR5quQpKytbPG/9/f3av3+/yxOtXl1dncbGxiT932ey//+TDNni48ePamtrU2dn5+Jd\n4rVwnpY6rkw4V07MhbX6jRs39PTpUzmOo8uXL+vrr79O9whJNzU1pQsXLujDhw+am5tTbW2tDh48\n6PZYCRkeHta1a9f0+vVreTweBYNB3bhxQw0NDfr06ZO2bdum1tZWbdiwwe1RzZY6ppqaGnV1dSkv\nL08+n0+tra3y+/1uj2oWDod1+/Zt7dixY/G1q1ev6tKlS1l7nqSlj+v48ePq6elx9Vy5EksAyDbs\n4AEAA2IJAAbEEgAMiCUAGBBLADAglgBgQCwBwIBYAoDBvwEtkabTIHyGYQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qUR03xUVreqL",
        "colab_type": "code",
        "outputId": "89f10f5a-a252-47a9-fa68-5b75188e5d22",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 365
        }
      },
      "source": [
        "plt.imshow(np.reshape(reconstructed[6], (28, 28)))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f1bc3edb978>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 143
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAFKCAYAAACU6307AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHGlJREFUeJzt3W9sVFX+x/FPaSn9Oy30n9YF3WVL\nJPyJWRfWYkCKrLuYbACfoATIJiaL2ZWAhCAhgiYkIkhMRE2gKGYj2WyTPvKBWQhrNKyButtEsxA3\nRTYhFaEU6EKhBQrL78Hm17TTmennDDOdKb5fj5hzT889d+6dL3fume85OXfu3LkjAEBCYzLdAQAY\nDQiWAGAgWAKAgWAJAAaCJQAYCJYAYMjLdAeQnJBffN26dcuu29vbG7O8uLhY165d63999epVu82Q\nurm5uVa9/Px8u814x/Tggw/q9OnTg8r6+vqsNisrK+39RyIRu+7YsWPtumPGDL3XycnJGXJt5OTk\n2G2G1P2h4c4SFjeIjSbjxo3LdBdSjmCXPknfWb7++uv6+uuvlZOTo82bN2vmzJmp7BcAZJWkguWX\nX36p06dPq6mpSadOndLmzZvV1NSU6r4BQNZI6mv40aNHtXDhQknS5MmTdfny5aDnUgAw2iR1Z3nh\nwgVNmzat//WECRPU2dmpkpKSlHUMiYU8mwoZNEhUd+BARcigRTabMmVKpruQcrEGfnD3UjIazlwc\nI2+kR8MjkYiuXLnS//peGA2fMmWK2traBpWN9tHwMWPG6L///e+gMkbDUyOp/4Kqq6t14cKF/tfn\nz59XVVVVyjoFANkmqWD5+OOP6+DBg5KkEydOqLq6mq/gAO5pSX0N/9nPfqZp06bp2WefVU5Ojl59\n9dVU9wsAskrSzyw3bNiQyn4AQFbLYab09It+4J7I9evXrXoDUw+H09PTY9c9f/58zPJZs2bp73//\ne//rv/3tb3ab69evt+u6j3NCsm8uX74cs7yvr2/IgMqcOXOsNl9++WV7/yEj7iEDV4WFhUPKqqqq\n1NnZOWy9eELeV3fUPRUDTHebxpkK/MYAAAwESwAwECwBwECwBAADwRIADARLADAQLAHAQLAEAAPB\nEgAMLFiWpHjTnuXl5Q3ZNnBqs2TbjZaOTIvhVFdX9//7F7/4hf13jY2Ndt0jR45Y9c6dO2e3OWvW\nrLjbNm7cOOj13LlzrTYnT55s7z9kkplUZHtFl4dcfyF9HT9+vFUv5PqLVzdWBo8rVZk+3FkCgIFg\nCQAGgiUAGAiWAGAgWAKAgWAJAAaCJQAYCJYAYCBYAoCBYAkABtIdo7jpZjdu3IhZnpeXF3ebw03N\nunnzpt1mXp5/miORiLWtqKjIbjMkhW7ixIlWPXdhN0mqra2Nu23x4sWDXldVVVltFhcX2/sPSWHs\n6+uz6/b29sYsj17MLiQ1NuR9da/zTKTmpkP29gwAsgjBEgAMBEsAMBAsAcBAsAQAA8ESAAwESwAw\nECwBwECwBAADwRIADD+IdMeQVeHc1LTbt2/b27q6uuz9uyvm5efn222GSJRuNjBtLSQtLVG6YTQ3\nNdJdBVOSSktL426LTq90jytkxcB0pfvFS42MTm/t6Oiw20yU7hrNPa6xY8fabSb6rLK6IwCMAgRL\nADAQLAHAQLAEAAPBEgAMBEsAMBAsAcBAsAQAA8ESAAyjNoMn2V/zD8fN4Onp6YlZHolEhmxLlO2T\nbRJlOwzcFpJBFLJgmitkwbaCggJ7W25urtVmyDkNyTYKqRsvgya6PCSDJWTBslRku0WL9/7n5uYO\n2Z97rlKFO0sAMCT1X35LS4vWrl2ruro6SdKUKVO0ZcuWlHYMALJJ0t+PZs+erd27d6eyLwCQtfga\nDgCGpIPlt99+qxdeeEHPPfecvvjii1T2CQCyTs6dJIaVOzo61NraqkWLFqm9vV2rVq3SoUOH0jbH\nIgBkWlLPLGtqavT0009LkiZNmqTKykp1dHQMmUg1ndL106F4E6pGu3TpUszy++67T+fOnRtUduXK\nFXv/lZWVVr10/ccU7+cYhYWF6u3tTapN9ycmkux9hPx0qLCwMGb5+PHjh0zMnOmfDrnXX7w+TJw4\nUe3t7YPKvv/+e7vNkIl677vvPqteWVmZ3Wa863rs2LFD3hv3XIVMqJywnWT+6OOPP9YHH3wgSers\n7NTFixdVU1OTkg4BQDZK6s5ywYIF2rBhg/7617+qr69Pr732Gl/BAdzTkgqWJSUl2rNnT6r7AgBZ\na9SmO6ZLOhYsi5caGcu1a9eseiEpbCHPbBLVHficOF37d59DhTxbS5RuGb0t5JmhK13PLOOlJkZf\nbyH7j/d8N5arV69a9YqLi+02E52r6M+mew1m9JklAPzQECwBwECwBAADwRIADARLADAQLAHAQLAE\nAAPBEgAMBEsAMBAsAcCQdemO6Zh6LaTNkOnEXJ2dnXZdd4qySZMm2W2GpAa6dUPSHUO4+w85p+6K\nlZI/9V1IWmJIul3I1G/x0g2jy//973/bbT788MN23ZKSEqteyPEnOq/R29I1TWM83FkCgIFgCQAG\ngiUAGAiWAGAgWAKAgWAJAAaCJQAYCJYAYCBYAoAh6zJ4Ms3N4AnJCom3sFQs//znP616IQtL1dbW\n2nUTLW41cFvI0schWVHpyAxKlEESvc3ta6qyUqKFZAbFysx59NFHgzJ2oo0bNy7ldUPOf6K6yS5Y\nlqpsL+4sAcBAsAQAA8ESAAwESwAwECwBwECwBAADwRIADARLADAQLAHAQLAEAMMPIt0xHYuQJUpL\ni9528+ZNu92LFy9a9b777ju7zerqaruum8YZkpYYkhronquQcxqyCJbb13QtlpUo3TTaN998Y5WP\nHz/ebjPkfb1x44ZVLxKJ2G2GcD9XIam5iXBnCQAGgiUAGAiWAGAgWAKAgWAJAAaCJQAYCJYAYCBY\nAoCBYAkABoIlABhGbbpjqlZsi3b79m2rXqJUq+htISv29fT0WPU6OjrsNt20NEkqLS216qUr3c9N\nt8vLS8+l614rIddUSN1r167ZdeNdA9HlISs2hlwrbhphqj5/0dvcVVOLi4vt/d/16o5tbW1auHCh\nDhw4IEk6e/asVq5cqeXLl2vt2rVBuc8AMBoNGyx7enq0bds21dfX95ft3r1by5cv15/+9Cc9+OCD\nam5uTmsnASDThg2W+fn52rdv36CZa1paWvTkk09KkhoaGnT06NH09RAAssCwD37y8vKGPB/q7e3t\nf15RUVGhzs7O9PQOALLEXT8lT/WD/nQ8YA9RVlZ21/Xq6uoSvk7k2WefteuONHfwZzQJefifrJKS\nErtuTU2NXXf27Nkxy9977z27jdEk+vob6esxqWBZVFSk69evq6CgQB0dHUGTyw7HDb4hQTqk7tWr\nV61658+fj1leV1enkydPDiprbW2199/S0mLVmz59ut3mggUL7LqVlZUxy0tLS9Xd3d3/euzYsXab\nIdzJb0MmyY3X1+Li4iGjz+7kv+6vJqSwEe7Tp0/bdf/4xz8OKXvvvff0hz/8YVDZpEmT7Dbnz59v\n133ggQeseuXl5Xab8T6r0def5I+GV1RU2PtPdP6T+p3lnDlzdPDgQUnSoUOHNHfu3GSaAYBRY9g7\ny+PHj2vHjh06c+aM8vLydPDgQe3atUubNm1SU1OTamtrtWTJkpHoKwBkzLDBcvr06froo4+GlH/4\n4Ydp6RAAZKOsy+DJ9DNL95lVouyF6G1FRUX2/t0f+Ic8MwxZMCzhM5uAdtItNzfXrusuwjZc3YFC\nMogKCgrsuiEJHoWFhVZ5yCJkIdeV+16FfP5CMnjc40rVIHT2XP0AkMUIlgBgIFgCgIFgCQAGgiUA\nGAiWAGAgWAKAgWAJAAaCJQAYCJYAYBiRdMd46UY5OTlpW/hqoJDptNwUqkQLO0VvC5mi6ic/+YlV\nL2SOxJAUtkTnY+C2dC0Yl44p0kKkI4Uv5PhDFheLd16jyyORiN1mSEqr+1kJmU4vUbpn9OfKXQgw\nVXPfcmcJAAaCJQAYCJYAYCBYAoCBYAkABoIlABgIlgBgIFgCgIFgCQAGgiUAGEYk3TFkdT03jS1V\nK8ZF6+3tteolSvWK3haSbuWmJoakpYWsGOimO4YIWV0wpK4rZMXAkFUjXSHnP9GqodHGjx9vlXd3\nd9ttXr58+a73Hy3kWo23YuVw2xJhdUcAGEEESwAwECwBwECwBAADwRIADARLADAQLAHAQLAEAAPB\nEgAMo3bBspCsgJAMCrduokWYord9//339v7d47p+/brdZkgGUzoyeNLx/ruLVUmJs3KijykdC+iF\nHH9env+RjNfX6PKQ/YcsLuZegyHvaaLrP/o4QmJAKnBnCQAGgiUAGAiWAGAgWAKAgWAJAAaCJQAY\nCJYAYCBYAoCBYAkABoIlABhGJN0xhJualY60tJB2QxZhu3r1qr3/0tJSq15RUZHdZsgiYOlIdwz5\nO7ev7sJuIW2G1A1JIQw5/pB0R3fBshMnTthtugv2SVJxcbFVLyQtMVEaa/S2dCwulwh3lgBgsIJl\nW1ubFi5cqAMHDkiSNm3apN/85jdauXKlVq5cqc8++yydfQSAjBv2nr+np0fbtm1TfX39oPL169er\noaEhbR0DgGwy7J1lfn6+9u3bp+rq6pHoDwBkpZw75tPnd955R+PHj9eKFSu0adMmdXZ2qq+vTxUV\nFdqyZYsmTJgQ92/v3LkT9EAcALJNUqPhixcvVnl5uaZOnarGxka9++672rp1a8K/iRWTY03+645G\nhowwhkwUe/nyZavexYsXY5ZPmzZtyOjjsWPH7P27o6HxRkJjmTp1ql033jeIsrKyQe9Nfn6+3WbI\nuXInKg75zzfeNRWJRHTlypVBZe77n64JdUMmiv7888+HlP3ud79TY2PjoLKQ0fBf/vKXdt1HHnnE\nqheJROw2471XEyZM0KVLl+x2BiorK7PrJhphT2o0vL6+vv8DuGDBArW1tSXTDACMGkkFyzVr1qi9\nvV2S1NLSorq6upR2CgCyzbDfOY4fP64dO3bozJkzysvL08GDB7VixQqtW7dOhYWFKioq0vbt20ei\nrwCQMcMGy+nTp+ujjz4aUv6rX/0qLR0CgGyUdemOrpBBg5CH8e5g0I0bN+xtBQUF9v7dAaaJEyfa\nbYakmyV6wJ1selk6fgmRqtUdo7nXVboGGENSMysrK63ykAGWRL9qiRbSV1eiazV6m3tdper6I90R\nAAwESwAwECwBwECwBAADwRIADARLADAQLAHAQLAEAAPBEgAMBEsAMGRdumM6Vm1050iU/BSuRKvg\nRW+LN/dlLG5qZEiqWXd3t103XgqdNPh9TNfqhiGpgaloM9n9hbz/IfsIWd3RTXcMmXu0p6fHruuu\nsBly/kPSHUcad5YAYCBYAoCBYAkABoIlABgIlgBgIFgCgIFgCQAGgiUAGAiWAGDIugyeW7duWfVC\nMkjSsWBWYWGhvS1Rtk+y0pFpIfnZLiHZFCHZLm62R0ibiRaXiz437v5DFkELea9CFrcbN26cVV5e\nXm63efPmTbtuovd1oJBjSvRZjT7n7vuaqqxA7iwBwECwBAADwRIADARLADAQLAHAQLAEAAPBEgAM\nBEsAMBAsAcBAsAQAw4ikO8ZLN8rJyRmyzU1NDEl3S0e6XaK0zOhtIeluRUVFVr2QFMZUvVcDt6Xr\n/XcX90rVOU32+gsRkhoZclzx+hpdHrJgXkVFhV3X7WvIgm2JPivRnyt3IbZUnVPuLAHAQLAEAAPB\nEgAMBEsAMBAsAcBAsAQAA8ESAAwESwAwECwBwECwBABD1q3u6KYbhqQwhaxY59ZNtGJj9LaQlRgn\nT55s1Qs5fnfFTCnx8Q/clq7VHd3UuJBjSlT3+vXrg167KXQhKxaGpPuFiNfX6PKSkpK07N89B6lK\nd4xuJ+QcpIIVLHfu3KnW1lbdunVLq1ev1owZM7Rx40bdvn1bVVVVevPNN+2LDABGo2GD5bFjx3Ty\n5Ek1NTWpq6tLS5cuVX19vZYvX65FixbprbfeUnNzs5YvXz4S/QWAjBj2u9SsWbP09ttvS5IikYh6\ne3vV0tKiJ598UpLU0NCgo0ePpreXAJBhwwbL3Nzc/mnDmpubNW/ePPX29vZ/7a6oqFBnZ2d6ewkA\nGZZzxxxROXz4sPbu3av9+/frqaee6r+bPH36tF5++WX9+c9/jvu3d+7cScs8gQAwUqwBniNHjmjP\nnj16//33VVpaqqKiIl2/fl0FBQXq6OhQdXV1wr+/c+dOzFHuMWPGDBkpDRm5doWMRl+4cMGqd/bs\n2ZjlTzzxhD7//PNBZZ999pm9/xkzZlj13EmCJammpsauW1VVFbP8Rz/6kb777rv+1yEjkSGj4Tdu\n3LDqpWI0vK6uTidPnhxUFolErDbTNRp+7do1u25XV9eQskceeURfffXVoLLDhw/bbdbW1tp1f/7z\nn1v1iouL7TbjjYbff//9Qz5zpaWlVpshn5VEo/HDfg3v7u7Wzp07tXfvXpWXl0uS5syZo4MHD0qS\nDh06pLlz59qdAYDRaNg7y08++URdXV1at25df9kbb7yhV155RU1NTaqtrdWSJUvS2kkAyLRhg+Wy\nZcu0bNmyIeUffvhhWjoEANko6zJ4Ms19ZhaSwdPe3m7v/6GHHrLquZlOoXVv375tbQtp031PpbDn\ny65EfY0+XveZecgiZKnKNooW71lodPm4cePsNt1n9lLiz8BAIYO7iZ5vRp/HkCyyVCA3HAAMBEsA\nMBAsAcBAsAQAA8ESAAwESwAwECwBwECwBAADwRIADARLADCM2nTHkHS7EInS/QZKNO1Y9LaQKcrc\ntYxiTc8VzwMPPGDXTZSaOHBbXp5/6YSkO/7nP/+x67oSTeUVfW7cdMOQ6y8kNTLkWiksLLTKQ95/\nd4o6yf+sjB071m4zUd3obe77mqq5dLmzBAADwRIADARLADAQLAHAQLAEAAPBEgAMBEsAMBAsAcBA\nsAQAA8ESAAxZl+7opiaFpIWFrAJXUFBg1YuXahZrW8j+E7U70LVr1+w23VX4JOnq1avWtpB0x5Bz\n5Z7/kBTCRG1Gb3Pf/5AUupDUyJBrJV5qbHT5T3/6U7vNc+fO2XXdc5CudMd0pTzHw50lABgIlgBg\nIFgCgIFgCQAGgiUAGAiWAGAgWAKAgWAJAAaCJQAYRiSDJySDws1gSEdWiORnG4RkGhQVFdn7dxcs\nC1mEyl1YSkqcFTFwW0hWUEi2z82bN616ZWVldpsh1597XYVkj/T19dl1Q96reHXHjRs36PX9999v\nt/nNN9/YdaP3E09IVlLIuUrVQmQu7iwBwECwBAADwRIADARLADAQLAHAQLAEAAPBEgAMBEsAMBAs\nAcBAsAQAQ9alO6aizbup66a7uQt7SVJtba29//Lycqueu7CWFJZu53LTMiXp1q1bdt2Q1FBXSGqq\nm5oXsmCam8Iphb1X8c5B9DGEXP8h59VNDQ5J4Uz0/oekTaaDdRQ7d+5Ua2urbt26pdWrV+vTTz/V\niRMn+j/Yzz//vObPn5/OfgJARg0bLI8dO6aTJ0+qqalJXV1dWrp0qR577DGtX79eDQ0NI9FHAMi4\nYYPlrFmzNHPmTElSJBJRb29v0Cw2AHAvGPYhQG5ubv9zpObmZs2bN0+5ubk6cOCAVq1apZdeekmX\nLl1Ke0cBIJNy7pgT8x0+fFh79+7V/v37dfz4cZWXl2vq1KlqbGzUuXPntHXr1nT3FQAyxhrgOXLk\niPbs2aP3339fpaWlqq+v79+2YMECvfbaaynrkDtyG/IoIGQ0sqOjw6p36tSpmOW//vWv9Ze//GVQ\n2VdffWXvf968eVa9f/3rX3abIZO/VldXxyx/9NFH1dra2v86EonYbYaM8LqTCrsjsVL8EfbJkycP\nOY/FxcVWmyGjxiETJV+/ft2uG6sPEydOVHt7+6Cys2fP2m0eOXLErrtkyRKrXsi1UlBQELO8tLRU\n3d3dg8rccxByrhL9cmDYr+Hd3d3auXOn9u7d2z/6vWbNmv4T0tLSorq6OrszADAaDXtn+cknn6ir\nq0vr1q3rL3vmmWe0bt06FRYWqqioSNu3b09rJwEg04YNlsuWLdOyZcuGlC9dujQtHQKAbES6IwAY\nRiTdMYSbRhaSwhUywOBK9HOp6G0hg1HuqoWVlZV2myErQV65csXaFjJoFpIaePHiRauemxYqJV6J\n8dq1a4Neu9dKyDGFXH8h56q0tDRmefQgaToGOKXE18pAIQM8iVIjo7e56Y+pWgWSO0sAMBAsAcBA\nsAQAA8ESAAwESwAwECwBwECwBAADwRIADARLADDY81mOZiGH6GZbxMu0KCkpGbJgWUgGh7sQmbuw\nWmjdeNlGkUhkUMZGyCJoIQtNudkWIW3GO/9lZWW6fPmy3c5AIRk8IQt23a2CgoIh07yFZJCFTCfn\nXqvjxo2z24x3XseMGTPkOnavFTJ4AGAEESwBwECwBAADwRIADARLADAQLAHAQLAEAAPBEgAMBEsA\nMBAsAcDwg0h3BIC7xZ0lABgIlgBgIFgCgIFgCQAGgiUAGAiWAGAYuSmcB3j99df19ddfKycnR5s3\nb9bMmTMz0Y2Uamlp0dq1a1VXVydJmjJlirZs2ZLhXiWvra1Nv//97/Xb3/5WK1as0NmzZ7Vx40bd\nvn1bVVVVevPNN5Wfn5/pbgaJPqZNmzbpxIkTKi8vlyQ9//zzmj9/fmY7GWjnzp1qbW3VrVu3tHr1\nas2YMWPUnydp6HF9+umnGT9XIx4sv/zyS50+fVpNTU06deqUNm/erKamppHuRlrMnj1bu3fvznQ3\n7lpPT4+2bdum+vr6/rLdu3dr+fLlWrRokd566y01Nzdr+fLlGexlmFjHJEnr169XQ0NDhnp1d44d\nO6aTJ0+qqalJXV1dWrp0qerr60f1eZJiH9djjz2W8XM14l/Djx49qoULF0qSJk+erMuXLw9ZswaZ\nlZ+fr3379qm6urq/rKWlRU8++aQkqaGhQUePHs1U95IS65hGu1mzZuntt9+W9L81knp7e0f9eZJi\nH1fIOkLpMuLB8sKFCxo/fnz/6wkTJqizs3Oku5EW3377rV544QU999xz+uKLLzLdnaTl5eWpoKBg\nUFlvb2//17mKiopRd85iHZMkHThwQKtWrdJLL72kS5cuZaBnycvNzVVRUZEkqbm5WfPmzRv150mK\nfVy5ubkZP1cZeWY50L2SbfnQQw/pxRdf1KJFi9Te3q5Vq1bp0KFDo/J50XDulXO2ePFilZeXa+rU\nqWpsbNS7776rrVu3ZrpbwQ4fPqzm5mbt379fTz31VH/5aD9PA4/r+PHjGT9XI35nWV1drQsXLvS/\nPn/+vKqqqka6GylXU1Ojp59+Wjk5OZo0aZIqKyvV0dGR6W6lTFFRUf8Sqx0dHffE19n6+npNnTpV\nkrRgwQK1tbVluEfhjhw5oj179mjfvn0qLS29Z85T9HFlw7ka8WD5+OOP6+DBg5KkEydOqLq6WiUl\nJSPdjZT7+OOP9cEHH0iSOjs7dfHiRdXU1GS4V6kzZ86c/vN26NAhzZ07N8M9untr1qxRe3u7pP89\nk/3/XzKMFt3d3dq5c6f27t3bP0p8L5ynWMeVDecqI7MO7dq1S//4xz+Uk5OjV199VQ8//PBIdyHl\nrl69qg0bNujKlSvq6+vTiy++qCeeeCLT3UrK8ePHtWPHDp05c0Z5eXmqqanRrl27tGnTJt24cUO1\ntbXavn27xo4dm+mu2mId04oVK9TY2KjCwkIVFRVp+/btqqioyHRXbU1NTXrnnXf04x//uL/sjTfe\n0CuvvDJqz5MU+7ieeeYZHThwIKPniinaAMBABg8AGAiWAGAgWAKAgWAJAAaCJQAYCJYAYCBYAoCB\nYAkAhv8DwQehnczkGjoAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RJ4kOSKXOU1d",
        "colab_type": "code",
        "cellView": "code",
        "colab": {}
      },
      "source": [
        "#@title Test the reconstructions\n",
        "  \n",
        "batch_indices = np.random.choice(N_TRAINING_SAMPLES, BATCH_SIZE)\n",
        "training_samples = train_images[batch_indices,:,:]\n",
        "plt.imshow(training_samples[1,:,:])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "feE4ejQzayYp",
        "colab_type": "code",
        "outputId": "0b2cc5a3-e9f7-41c4-fc14-0832a1a2730b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 402
        }
      },
      "source": [
        "with tf.train.MonitoredSession() as sess:\n",
        "\n",
        "  reconst_vectors = sess.run(test_mu, feed_dict={input_batch: training_samples})\n",
        "\n",
        "  reconst_image = np.reshape(reconst_vectors[1,:], (28,28))\n",
        "  plt.imshow(reconst_image)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Graph was finalized.\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAFKCAYAAACU6307AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8zVf+x/FPGk2JfYu9eKgUtfaB\nihBiX2vtAxmig1arNQRtUUuHGUvogk5LLG1Rlam2dNGRqqVqiWJoLTOJLZYSW+pHRS3N748+mod7\nc+/N+5uSiHk9/3LP+fic782Nj++9555z/NLS0tIMAODTfTl9AQCQG1AsAUBAsQQAAcUSAAQUSwAQ\nUCwBQJAnOwaJjIz02D5lyhQbO3asS9ujjz4q5dy9e7c8vvsYvrz++utS3IMPPuixfdCgQbZgwQJ5\nPHdBQUFS3PXr1+WcBw8elGODg4M9tvfs2dNWrFiR/rhYsWJyznPnzsmx+fLlk+KWL18u5wwNDfXY\n/vTTT1tMTIxL22OPPSblTEhIkMevX7++HNuxY0c5duDAgRnaoqKiMvwOnz9/Xs75888/y7HVqlWT\n4goUKCDn9Pb8K1eubEeOHHFpe/nll6WctWvXlsefOHGi174cvbMsX758Tg5/R5QsWTKnL+GOcFIc\ncwv1P6bcpHTp0jl9CXfEAw88kNOXkPU7yylTptiePXvMz8/Pxo4d66h6A0Buk6ViuX37dktKSrLY\n2Fg7dOiQjR071mJjY2/3tQHAXSNLb8O3bt1qrVq1MjOzKlWq2MWLF+3y5cu39cIA4G7il5W14ePH\nj7dmzZqlF8yIiAj7+9//bpUrV/YYf+LEiXvy80kA/ztuy2x4ZvXW22z04sWLM8yU5/bZ8DFjxtjU\nqVPl8dzdrbPh7jPH98Js+Lhx4+xvf/ubS1tunw2fPn26vfTSSy5t98JseLVq1ew///mPS1uumA0P\nCgpy+Qdw5syZe3YWGADMslgsQ0NDbc2aNWZmtm/fPgsKCnL0vwcA5DZZehv+6KOP2iOPPGK9e/c2\nPz8/n7euAHAvyPJnlqNGjbqd1wEAd7VsWe5Yt25dua9KlSpSTm8TLJ4sWbJEju3Tp48Ut2nTJq99\nv/76q8vjcuXKyeMfP35cinvooYfknE5Wqvz0009SX9WqVeWccXFxcuyJEyekuGvXrsk5161b57F9\n3LhxGfo6deok5XSypHXPnj1yrPr7Z+b934B7+zPPPCPndDJxliePVj7KlCkj55w7d67H9jfeeCND\n37Fjx6ScNWrUkMf3hY00AEBAsQQAAcUSAAQUSwAQUCwBQECxBAABxRIABBRLABBQLAFAkC0reNxX\ntPjqW7VqlZQzf/788vgrV66UY0+fPi3F5c2bV87RqFEjefw///nPUpyT5aZJSUlybLt27bz2lShR\nIv3PP/zwg5yzVKlScqy6MufSpUtyTl8rTdx3y1K3k2vWrJk8fmJiohzr5Hl5W8Xl3h4fHy/nVLeo\nM/ttE3CFk/EvXLjgtc99q7mnn35aynm7NibnzhIABBRLABBQLAFAQLEEAAHFEgAEFEsAEFAsAUBA\nsQQAAcUSAAQUSwAQZMtyx+DgYLmvfv36Uk4nB5bdd5/+f0JgYKAU52u5o/sBYb6We7obPny4FPf7\nue0KJ4dgHT58WOpzstw0MjJSjlWfl69lce7atm3rtW/EiBEujxctWiTl7Ny5szz+jRs35FhfP393\nnpbxtm3bNkN7vXr15JwHDhyQYwsUKCDFFStWTM4ZFhbmta9r164uj7/++mspp5MDA33hzhIABBRL\nABBQLAFAQLEEAAHFEgAEFEsAEFAsAUBAsQQAAcUSAAQUSwAQZMtyx1dffdVj++OPP56hb/bs2VLO\nt99+Wx6/dOnScuyJEyekOG9LrcaPH29ffPGFS5uT5W7333+/FDd//nw55/r16+VYX8tIb+27evWq\nnPO5556TY9PS0qS4kSNHyjlnzZrlsX3ZsmUZ+p566ikpZ3R0tDz+Aw88IMeGhobKsVu2bPHY7n66\naNGiReWcTmLV38GKFSvKOX/88UeP7T169LC4uDiXttatW0s5P/zwQ3l8X7izBAABxRIABBRLABBQ\nLAFAQLEEAAHFEgAEFEsAEFAsAUBAsQQAQbas4Bk4cKDcd+bMGSnn0aNH5fG7d+8ux968eVOKe/bZ\nZ732uR965WQFgXoIlPsqIV9q164tx27fvt1r38GDB9P//Pjjj8s5L126JMeqh1s5ObCsTZs2cp+6\nMmzq1Kny+J999pkc6+RwLW+He7m3f/PNN3LOHj16yLGNGjWS4ooUKSLn/Pzzz732JSYmujzet2+f\nlHPUqFHy+L5wZwkAgizdWcbHx9uwYcOsatWqZvbbcbbjx4+/rRcGAHeTLL8Nb9iwobzpBQDkdrwN\nBwBBlovlwYMH7ZlnnrE+ffrY5s2bb+c1AcBdxy9N3UDwFsnJybZz505r3769HT9+3CIjIy0uLs4C\nAgI8xqekpDjaJw8A7jZZ+syyVKlS1qFDBzP7bUPYEiVKWHJyslWoUMFjvLevTkRGRtrixYtd2sqU\nKSNdQ0xMjHy9I0aMkGPVrw55+4pLjRo1bP/+/S5tTr465OfnJ8U52VD4dnx1KDo62l588cX0x06+\nOrRx40Y5Vv3qUOHCheWc165d89j+5JNP2rvvvuvStnr1ainn2LFj5fGdfHWoSpUqcmxycnKGtqio\nKHv99ddd2g4fPizndPLVoXXr1klxt+OrQ+vWrbMWLVq4tHl7Xd05+epQ165dvfZl6W34p59+agsX\nLjQzs7Nnz9r58+etVKlSWUkFALlClu4sW7RoYaNGjbKvv/7arl+/bq+88orXt+AAcC/IUrEsUKCA\nzZ0793ZfCwDctbJluWOlSpXkvuvXr//hnO5CQkLkWHVmPyUlRe5LSEiQx2/cuLEUV758eTnnqVOn\n5NgDBw5IfW3btpVzhoeHy7GrVq2S4py8prt37/bal9XPlydNmiSPf+TIETm2X79+cuywYcMytEVF\nRdmGDRtc2pwcmFa2bFk5Vn0N1CXMZmbVqlWT+1q2bCnldHK4ni98zxIABBRLABBQLAFAQLEEAAHF\nEgAEFEsAEFAsAUBAsQQAAcUSAAQUSwAQZMtyx61bt3psDwsLy9Dnvg2TN7eeNJiZ06dPy7EfffSR\nFNe8eXOvfefPn3d5rG67ZmZWp04dKU7dSs7M7IMPPpBjBw8eLPVNmDBBzum+DZ8v6tI8J8s9N23a\nJPdFRUVJOZ1se9awYUM5dteuXXJscHCw1O5kCaOTpYnu29t5U7lyZTnn5cuX5b5p06ZJOT/++GN5\nfF+4swQAAcUSAAQUSwAQUCwBQECxBAABxRIABBRLABBQLAFAQLEEAEG2rODxtYLFvS8uLk7KqR7s\nZebskPVmzZpJceXKlZP7EhMT5fHVFSxdunSRc1atWlWOLVq0qNTXp08fOefbb78tx/paQXSrtWvX\nyjknTpwo96mrjQoUKCCP/9prr8mx6goiM++ruNzbd+zYIeesWLGiHKuuTKtVq5ac09fP1X1lUteu\nXaWcW7Zskcfv1auX1z7uLAFAQLEEAAHFEgAEFEsAEFAsAUBAsQQAAcUSAAQUSwAQUCwBQECxBABB\ntix39HW4lnvfX//6VynnypUr5fF79Oghx44ZM0aK83VgWaFChVweO1lueOzYMSluyZIlcs4pU6bI\nsd6e/zfffOPSV7duXTmnk6Wpvg6sutXFixflnHv37vXY3q5duwx9ERERUs5BgwbJ49euXVuOLVy4\nsBzboUMHqT05OVnOmZSUJMe+8MILUtzChQvlnL7+reTPn9/l8b59+6ScTpZw+sKdJQAIKJYAIKBY\nAoCAYgkAAoolAAgolgAgoFgCgIBiCQACiiUACCiWACDIluWO3pbmjRkzJkOfuoRp27Zt8vg7d+6U\nY9WT+M6dO+e178qVKy6Pn376aXn8I0eOSHFr1qyRczo5XdBXbOXKldP/3Lp1aznnG2+8IceqS9Ou\nXbsm56xZs6bXvhs3brg8Llu2rJQzJCREHv/ChQty7OnTp+XYdevWZWhr1aqVLV++3KWte/fucs4v\nv/xSjv3++++luIYNG8o5//Wvf3lsj4qKsg0bNmQp762/t3+EdGeZkJBgrVq1sqVLl5qZ2alTp6xf\nv34WERFhw4YNc/SLCwC5UabF8sqVKzZ58mSX/0lnz55tERERtmzZMqtYsaKtWLHijl4kAOS0TItl\nQECAzZ8/34KCgtLb4uPjrWXLlmZmFh4eblu3br1zVwgAd4FMP7PMkyeP5cnjGpaammoBAQFmZla8\neHE7e/bsnbk6ALhL+KWlpaUpgXPmzLGiRYta3759LSQkJP1uMikpyV566aUMHyrfKjEx0dGejgBw\nt8nSbHhgYKBdvXrV8ubNa8nJyS5v0T3p0qWLx/b9+/dbjRo1XNpWrVolXYOT2fC8efPKsepGtd5m\nw+vUqWN79uxxaXOyUa06G75582Y553PPPSfHepsNf++996x///7pj3v27CnnvFtnw0ePHm3Tpk1z\naWvTpo2Uc9asWfL4TmbDw8PD5dj9+/dnaFuwYEGGjYmdzIYvW7ZMjr3//vuluNsxG75q1aoMdUTN\nGxYWJo/ftGlTr31Z+p5l48aN07+6EhcX53MAALgXZHpnuXfvXps+fbqdPHnS8uTJY2vWrLGZM2fa\n6NGjLTY21sqWLWtdu3bNjmsFgByTabGsWbOmxy+Vv/POO3fkggDgbpQtK3j69esn96mfL544cUIe\nf+PGjXJsQkKCFOdtXqxOnTr22WefubQ5mdxSP4s8ePCgnHPLli1ybLt27aS+HTt2yDknTpwox6am\npkpxu3fvlnP6+lkdOnTI5bF6raGhofL4Tg7MGjFihBzbsWNHKa5YsWJyTvXAPjPPK4g88fTZqje+\nPgd178tsruR3BQoUkMf3hbXhACCgWAKAgGIJAAKKJQAIKJYAIKBYAoCAYgkAAoolAAgolgAgoFgC\ngCBbljs+8MADct+MGTOknE62XVIPQTMzq1evnhR39OhRr33uy8t87fXprnjx4lJc8+bN5Zzu2+D5\n4u3n36dPH3v//ffTH0+ZMkXOuXLlSjlW3XXfySFUAwYMkPvUJXxODhZzskXdnDlz5Nj8+fN7bHff\nki0pKUnO6YT6b8XJ5uDDhw/32jd9+nSXx0888YSU8+GHH5bH/+CDD7z2cWcJAAKKJQAIKJYAIKBY\nAoCAYgkAAoolAAgolgAgoFgCgIBiCQACiiUACLJlueOlS5fkvl69ekk5P/nkE3n8qKgoOXbx4sVS\nnK8TG3/99VeXx/fdp/+fdPXqVSlOPYXSzKxp06ZybIUKFaS+Dz/8UM5ZqFAhOfbw4cNSXJUqVeSc\nmzZt8tgeEhKSoU89NbR8+fLy+OPGjZNj/f395dibN29K7U5eqxdeeEGO9bXk91ZOTmL9+eef5b6X\nX35Zyvn555/L4/vCnSUACCiWACCgWAKAgGIJAAKKJQAIKJYAIKBYAoCAYgkAAoolAAiyZQXP/v37\n5b7evXtLOZ2soDh16pQc279/fylu1qxZXvu+++47l8dFixaVx69fv74UFxAQIOc8fvy4HOtrBdWt\nfQcOHJBzqquizMxCQ0OluCtXrsg5U1JS5D5fK0huVaRIEXn82bNny7HBwcFy7JNPPumx/dq1ay6P\n27RpI+d0stqlZs2aUlzXrl3lnMeOHfPYXrt27Qx9hw4dknI6OdzPF+4sAUBAsQQAAcUSAAQUSwAQ\nUCwBQECxBAABxRIABBRLABBQLAFAQLEEAEG2LHc8c+aM3DdkyBApZ7ly5eTxnSwhW716tRT37bff\nyn1ODoFq2LChFPfee+/JOdu2bSvHrl+/3mN78+bNXfoefPBBOeeYMWPk2JiYGCmuY8eOcs78+fN7\n7atevbrLY/VwOScHxrVo0UKO7dChgxzrvqzWzKxHjx62Z88el7aNGzfKORcsWCDH+jq071YfffSR\nnHPLli0e2zt16pThd1M9CM/J0mhfuLMEAIFULBMSEqxVq1a2dOlSMzMbPXq0de7c2fr162f9+vWz\nDRs23MlrBIAcl+nb8CtXrtjkyZMtJCTEpX3EiBEWHh5+xy4MAO4mmd5ZBgQE2Pz58y0oKCg7rgcA\n7kp+aWlpaUrgnDlzrGjRota3b18bPXq0nT171q5fv27Fixe38ePHW7Fixbz+3SNHjljlypVv20UD\nQHbL0mx4ly5drEiRIla9enWLiYmxN9980yZMmOA1fsCAAR7b169fn+GtvDob6WQ2XN1Q2Mxs165d\nUtw777zjsf3QoUNWpUoVl7bcNBseHx/vsf2vf/2rTZw4Mf2xk9nwkiVLyrHZORseGRmZYWPi//73\nv1LOy5cvy+O7b8bryx+dDZ80aVKGf4v3wmz4q6++aiNHjnRpU2fDmzRpIo/fsmVLr31Zmg0PCQlJ\n/8pFixYtHH2NAgByoywVy6FDh6YfVRAfHy//DwMAuVWmb8P37t1r06dPt5MnT1qePHlszZo11rdv\nXxs+fLjly5fPAgMDberUqdlxrQCQYzItljVr1rQlS5ZkaHfyORgA5HbZstyxXr16cl/Pnj2lnFev\nXpXH3717txzr7QNmd74+NHfv8zYZ5Ik6cdOsWTM559atW+XY1q1be+279cNv9RRMM7OIiAg5Vl0a\nqJ4saOZ70tB9Mu7o0aNSzk6dOsnjz58/X45966235NjOnTt7bC9durTL48jISDln37595diKFStK\nceqJpWZmdevWlftSU1OlnE7+/d/2CR4A+F9DsQQAAcUSAAQUSwAQUCwBQECxBAABxRIABBRLABBQ\nLAFAQLEEAEG2LHesXbu23BcXFyflTEpKksf3tLbdm23btklxkydP9tg+b968DH116tSRxy9YsKAU\n9+WXX8o5ixYtKsdu3rzZY3tYWJhLn5OcycnJcmyZMmWkOE97OXpz8eJFj+2hoaEZft9q1aol5VR/\nT8z052T2289ZNW7cuAxtQ4YMsSlTpri0OdnH4YknnpBj33//fSnO399fzum+VPNW7vuUL1y4UMrp\nZO9TX7izBAABxRIABBRLABBQLAFAQLEEAAHFEgAEFEsAEFAsAUBAsQQAQbas4PG1gsO97/Lly1JO\nJ4cg7dmzR45VDyxr2LCh3Ld+/Xp5/F9++UWKO3jwoJzT1yFk7goXLiz1NW/eXM7pZAVJTEyMFOfk\nwLZy5crJfWXLlpVyBgQEyONv375djn344Yfl2DfeeENqd7Iq6Ntvv5Vje/XqJcXdvHlTzvnDDz/I\nfcuWLZNyOllB5gt3lgAgoFgCgIBiCQACiiUACCiWACCgWAKAgGIJAAKKJQAIKJYAIKBYAoAgW5Y7\nXrhwQe47c+aMlPOpp56Sx09JSZFjY2NjpbgmTZp47UtISHB53KhRI3l8dWmWkyVcJ06ckGObNm3q\nta9ChQrpf65Zs6ac86effpJjS5QoIcU5WW7o6/m79+XLl0/K6etgLXcnT56UY30d7ufu66+/9tju\nfphfoUKF5Jx58ugloXr16lLc999/L+ds2bKl3Ld//34pZ3x8vDx+48aNvfZxZwkAAoolAAgolgAg\noFgCgIBiCQACiiUACCiWACCgWAKAgGIJAAKKJQAIsmW5440bN+S+hx56SMr5448/yuN/9tlncuyw\nYcOkOF+nSw4cONDl8TvvvCOPX7x4cSlu7969cs727dvLsb6WJt7a991338k51SWEZmadO3eW4tas\nWSPn9Pb6T5o0yRYvXuzSpi5NVU8BdRr773//W45dtWpVhraRI0dmaHdyuuaBAwfk2MDAQCnu008/\nlXMuWbLEY3tCQoL95S9/cWmbOXOmlLNy5cry+L5IxTI6Otp27txpN27csMGDB1utWrXsxRdftJs3\nb1rJkiVtxowZjtbqAkBuk2mx3LZtmyUmJlpsbKylpKRYt27dLCQkxCIiIqx9+/b22muv2YoVKywi\nIiI7rhcAckSmn1k2aNDAZs2aZWa/7V6Smppq8fHx6TuAhIeH29atW+/sVQJADsu0WPr7+6d/NrFi\nxQoLCwuz1NTU9LfdxYsXt7Nnz97ZqwSAHOaXlpaWpgSuXbvW5s2bZ4sWLbI2bdqk300mJSXZSy+9\nZMuXL/f6d0+dOmVlypS5PVcMADlAmuDZtGmTzZ071xYsWGAFCxa0wMBAu3r1quXNm9eSk5MtKCjI\n59/3Nmv16quv2siRI13aChcuLF24r01q3Y0ePVqO/aOz4cHBwRk2/70Ts+FTpkyRc77wwgtybPny\n5T229+vXz2WmMjExUc7pZDa8Vq1aUtztmA0/evSoVapUyaUtN82Gz5gxI0PbN998Y2FhYS5tb731\nlpzziy++kGPV2XD3bxz4cvHiRY/tCQkJFhwc7NKmzoY7eefr/k2WW2X6NvzSpUsWHR1t8+bNsyJF\nipjZb7sJ//7LGhcX56hwAUBulOmd5erVqy0lJcWGDx+e3jZt2jQbN26cxcbGWtmyZa1r16539CIB\nIKdlWix79eplvXr1ytDu5K0lAOR22bKCp127dnLfvn37pJy7du2Sx/f2OZwn169fl+K8fbYUHByc\noU/NaWY2aNAgKe7o0aNyzmbNmsmxcXFxXvsOHTqU/udff/1VznnlyhU59tixY1Jc0aJF5Zz9+/eX\n+3r37i3lVA+2MzPbvHnzHYn1tjLHvd39M3Rf1APjzPRrdXK4XcmSJb32ub+D/eqrr6Sc3j4H9eQP\nfWYJAKBYAoCEYgkAAoolAAgolgAgoFgCgIBiCQACiiUACCiWACCgWAKAIFuWO3o7hKh169YZ+ho0\naCDl/Pjjj+Xx//nPf8qx7777rhRXpUoVr3333ef6f1CXLl3k8aOioqS4ggULyjk3bdokx6oHlrlv\nA+bLJ598IseGhoZKcdWqVZNz7t+/32uf+/I6X0tzbzVt2jR5fCdLGN23IfPF2xZp7nvHduzYUc65\ndOlSOTZ//vxSXMOGDeWcP/zwgxyrLrl1P+gsq7izBAABxRIABBRLABBQLAFAQLEEAAHFEgAEFEsA\nEFAsAUBAsQQAAcUSAATZstzR14lx7n2+Tne71SOPPCKP72S54a3no/vi68RC977Dhw/L49erV0+K\nU0+2MzM7d+6cHJsvXz6vfZcuXUr/80cffSTnHDVqlBy7ceNGKe6LL76Qc0ZHR3vtc19euXr1aimn\nk9NF/f395djChQvLsStXrszQNmDAgAw/m1tP5cxMqVKl5NhatWpJcU6W2+bNm9dr39WrV10e//LL\nL1JO9XfKzKx+/fpe+7izBAABxRIABBRLABBQLAFAQLEEAAHFEgAEFEsAEFAsAUBAsQQAQbas4Ond\nu7fct3DhQinnpEmT5PGdHML05ptvSnE1a9b02N6/f3/bsmWLS5t6CJuZ2YEDB6S4Fi1ayDlv3rwp\nxx47dsxr360HVDk5WMv95+GLemDVoEGD5JyLFy/22F6vXr0MfZ06dZJy1q1bVx7/ww8/lGMTExPl\n2D/96U9S+4YNG+ScKSkpcuyaNWukuOXLl8s5ff1b9fPzc3lcvnx5KaeTA9N84c4SAAQUSwAQUCwB\nQECxBAABxRIABBRLABBQLAFAQLEEAAHFEgAEFEsAEGTLcsdXXnnFY/vq1asz9DVu3FjK+e2338rj\nP/bYY3LshQsXpLjk5GSvfe7LC50s9xs8eLAUFxERIefs1q2bHPvjjz9KfRUqVJBzpqamyrHbt2+X\n4py8pj169JD71J/rxx9/LI/funVrOXb+/Ply7NGjRzO0de/ePcNBZidOnJBzjhs3To7t2bOnFHfq\n1Ck5p5PDDevUqSPldHIInC9SsYyOjradO3fajRs3bPDgwbZu3Trbt2+fFSlSxMzMBg4caM2bN78t\nFwQAd6NMi+W2bdssMTHRYmNjLSUlxbp162aNGjWyESNGWHh4eHZcIwDkuEyLZYMGDax27dpmZlao\nUCFLTU11tIsNANwLMp3g8ff3t8DAQDMzW7FihYWFhZm/v78tXbrUIiMjLSoqSv6cDwByK7+0tLQ0\nJXDt2rU2b948W7Roke3du9eKFCli1atXt5iYGDt9+rRNmDDB6989evSoVapU6XZdMwBkO6lYbtq0\nyWbNmmULFixIn9T53cGDB+2VV17xuWlnhw4dPLavXr06Q586G169enUpzsysVKlScuyXX34pxXmb\nDV+wYEGGjWl//fVXefycng33NMNq9tu7iltnP53MRrtv2urLe++9J8UNHTpUzlmjRg2P7U2aNMnw\nrYo7MRt+5swZOdbJbHjBggUztC1evNgiIyNd2u7UbLjq8uXLcuyePXs8to8fP94mT57s0qbOhju5\nUfv9I0dPMn0bfunSJYuOjrZ58+alF8qhQ4fa8ePHzcwsPj7eqlatKl8MAORGmU7wrF692lJSUmz4\n8OHpbd27d7fhw4dbvnz5LDAw0KZOnXpHLxIAclqmxbJXr17Wq1evDO1O3toBQG7HckcAEGTLckdf\nn2m696mny9WvX18ef8GCBXKs+mG0r9MF3fu++uorefy1a9dKcc8++6yc08nSxDJlynjtCwkJSf/z\nkCFD5JzuEw6+PP7441KcOhFkZvbEE094bG/SpInt2LHDpU1daLFr1y55/GvXrsmxXbt2lWN/nzdw\n537yZsWKFeWc7j8PX8Qv0tg//vEPOae3pdFmZuXKlXN5rE6y1apVSx7/D03wAAAolgAgoVgCgIBi\nCQACiiUACCiWACCgWAKAgGIJAAKKJQAIsmUFT6NGjeS+2NhYKeeKFSvk8Z2sYx8wYIAU5+1gpUaN\nGmVYsdOvXz95/BkzZkhxTradq1KlihzrbYs2M7MbN26k/3n37t1yTveVF76oK6icbN6SL18+r32h\noaEuj923IPRmzZo18vj+/v5ybOnSpeVYb1vPlS1b1uWx+pzMzNatWyfH/vTTT1LcsmXL5JwbNmzw\n2ud+mJ669WH+/Pnl8X3hzhIABBRLABBQLAFAQLEEAAHFEgAEFEsAEFAsAUBAsQQAAcUSAAQUSwAQ\n+KWppw4BwP8w7iwBQECxBAABxRIABBRLABBQLAFAQLEEAEG27JTubsqUKbZnzx7z8/OzsWPHWu3a\ntXPiMm6r+Ph4GzZsmFWtWtUPdsyhAAAD1klEQVTMzIKDg238+PE5fFVZl5CQYEOGDLEnn3zS+vbt\na6dOnbIXX3zRbt68aSVLlrQZM2ZYQEBATl+mI+7PafTo0bZv3770ncQHDhxozZs3z9mLdCg6Otp2\n7txpN27csMGDB1utWrVy/etklvF5rVu3Lsdfq2wvltu3b7ekpCSLjY21Q4cO2dixY+WjJO52DRs2\ntNmzZ+f0ZfxhV65cscmTJ1tISEh62+zZsy0iIsLat29vr732mq1YscIiIiJy8Cqd8fSczMxGjBhh\n4eHhOXRVf8y2bdssMTHRYmNjLSUlxbp162YhISG5+nUy8/y8GjVqlOOvVba/Dd+6dau1atXKzH47\nG+bixYvyuSvIHgEBATZ//nwLCgpKb4uPj7eWLVuamVl4eLht3bo1py4vSzw9p9yuQYMGNmvWLDMz\nK1SokKWmpub618nM8/O6efNmDl9VDhTLc+fOWdGiRdMfFytWzM6ePZvdl3FHHDx40J555hnr06eP\nbd68OacvJ8vy5MljefPmdWlLTU1NfztXvHjxXPeaeXpOZmZLly61yMhIi4qKsgsXLuTAlWWdv7+/\nBQYGmtlvB/iFhYXl+tfJzPPz8vf3z/HXKkc+s7zVvbLaslKlSvb8889b+/bt7fjx4xYZGWlxcXG5\n8vOizNwrr1mXLl2sSJEiVr16dYuJibE333zTJkyYkNOX5djatWttxYoVtmjRImvTpk16e25/nW59\nXnv37s3x1yrb7yyDgoLs3Llz6Y/PnDljJUuWzO7LuO1KlSplHTp0MD8/P3vwwQetRIkSlpycnNOX\nddsEBgba1atXzcwsOTn5nng7GxISYtWrVzczsxYtWlhCQkIOX5FzmzZtsrlz59r8+fOtYMGC98zr\n5P687obXKtuLZWhoaPqZy/v27bOgoCArUKBAdl/Gbffpp5/awoULzczs7Nmzdv78eUdne9/tGjdu\nnP66xcXFWdOmTXP4iv64oUOH2vHjx83st89kf/8mQ25x6dIli46Otnnz5qXPEt8Lr5On53U3vFY5\nsuvQzJkzbceOHebn52cTJ060atWqZfcl3HaXL1+2UaNG2f/93//Z9evX7fnnn7dmzZrl9GVlyd69\ne2369Ol28uRJy5Mnj5UqVcpmzpxpo0ePtl9++cXKli1rU6dOtfvvvz+nL1Xm6Tn17dvXYmJiLF++\nfBYYGGhTp0614sWL5/SlymJjY23OnDlWuXLl9LZp06bZuHHjcu3rZOb5eXXv3t2WLl2ao68VW7QB\ngIAVPAAgoFgCgIBiCQACiiUACCiWACCgWAKAgGIJAAKKJQAI/h/FRvGOdAxFQQAAAABJRU5ErkJg\ngg==\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rsZpwWxeynEl",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Ignore this\n",
        "\n",
        "def log_normal_pdf(sample, mean, logvar, raxis=1):\n",
        "  log_2pi = N_OBSERVED * tf.log(2. * np.pi)\n",
        "  return tf.reduce_sum(\n",
        "      -.5 * ((sample - mean)**2. * tf.exp(-logvar) + logvar + log2pi),\n",
        "      axis=raxis)\n",
        "\n",
        "def compute_loss(sample, mean, logvar\n",
        "  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-uDdtoqWa4kd",
        "colab_type": "text"
      },
      "source": [
        "## Making a convnet encoder"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y5qMUqGPkhnc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Encoder(snt.AbstractModule):\n",
        "  \n",
        "  def __init__(self):\n",
        "    # TODO(jaslanides) lol this is bullshit ay\n",
        "    super().__init__(name='encoder')\n",
        "    \n",
        "    self._conv_net = snt.nets.ConvNet2D(output_channels=[32, 16, 8], \n",
        "                                        kernel_shapes=[5, 3, 3],\n",
        "                                        strides=[2, 2, 1], \n",
        "                                        paddings=['SAME'])\n",
        "    \n",
        "    self._mlp = snt.nets.MLP(output_sizes=[50, 50])\n",
        "    \n",
        "    self._mu_layer = snt.Linear(output_size=N_LATENT)\n",
        "    \n",
        "    self._sigma_layer = snt.Linear(output_size=N_LATENT)\n",
        "    \n",
        "  def _build(self, input_image: tf.Tensor):  # Think of this as __call__ :)\n",
        "    filtered_image = self._conv_net(input_image)\n",
        "    flat_image = snt.BatchFlatten()(filtered_image)\n",
        "    \n",
        "    embedding = self._mlp(flat_image)\n",
        "    \n",
        "    mu = self._mu_layer(embedding)  # [N_LATENT]\n",
        "    sigma = self._sigma_layer(embedding)\n",
        "    \n",
        "    return mu, sigma"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JTkQQJtml2Qr",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Decoder(snt.AbstractModule):\n",
        "  \n",
        "  def __init__(self):\n",
        "    super().__init__(name='decoder')\n",
        "\n",
        "    self._deconv_net = snt.nets.ConvNet2D(output_channels=[32, 16, 8], \n",
        "                                        kernel_shapes=[5, 3, 3],\n",
        "                                        strides=[2, 2, 1], \n",
        "                                        paddings=['SAME']).transpose()    \n",
        "   \n",
        "    self._mlp = snt.nets.MLP(output_sizes=[50, 3528])\n",
        "    \n",
        "    \n",
        "  def _build(self, latent: tf.Tensor):\n",
        "    embedding = self._mlp(latent)\n",
        "    \n",
        "    filtered_image = tf.reshape(embedding, [1,21,21,8])\n",
        "    \n",
        "    output = self._deconv_net(filtered_image)\n",
        "    \n",
        "    return outputlatent_variance"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SC7BV12XeUM-",
        "colab_type": "code",
        "outputId": "eddffee5-6362-4463-b438-058d518ef382",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 708
        }
      },
      "source": [
        "tf.global_variables()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'mlp/linear_0/w:0' shape=(784, 50) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp/linear_0/b:0' shape=(50,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear/w:0' shape=(50, 16) dtype=float32_ref>,\n",
              " <tf.Variable 'linear/b:0' shape=(16,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_1/w:0' shape=(50, 16) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_1/b:0' shape=(16,) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp_1/linear_0/w:0' shape=(16, 50) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp_1/linear_0/b:0' shape=(50,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_2/w:0' shape=(50, 784) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_2/b:0' shape=(784,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_3/w:0' shape=(50, 784) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_3/b:0' shape=(784,) dtype=float32_ref>,\n",
              " <tf.Variable 'beta1_power:0' shape=() dtype=float32_ref>,\n",
              " <tf.Variable 'beta2_power:0' shape=() dtype=float32_ref>,\n",
              " <tf.Variable 'mlp/linear_0/w/Adam:0' shape=(784, 50) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp/linear_0/w/Adam_1:0' shape=(784, 50) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp/linear_0/b/Adam:0' shape=(50,) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp/linear_0/b/Adam_1:0' shape=(50,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear/w/Adam:0' shape=(50, 16) dtype=float32_ref>,\n",
              " <tf.Variable 'linear/w/Adam_1:0' shape=(50, 16) dtype=float32_ref>,\n",
              " <tf.Variable 'linear/b/Adam:0' shape=(16,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear/b/Adam_1:0' shape=(16,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_1/w/Adam:0' shape=(50, 16) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_1/w/Adam_1:0' shape=(50, 16) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_1/b/Adam:0' shape=(16,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_1/b/Adam_1:0' shape=(16,) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp_1/linear_0/w/Adam:0' shape=(16, 50) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp_1/linear_0/w/Adam_1:0' shape=(16, 50) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp_1/linear_0/b/Adam:0' shape=(50,) dtype=float32_ref>,\n",
              " <tf.Variable 'mlp_1/linear_0/b/Adam_1:0' shape=(50,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_2/w/Adam:0' shape=(50, 784) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_2/w/Adam_1:0' shape=(50, 784) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_2/b/Adam:0' shape=(784,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_2/b/Adam_1:0' shape=(784,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_3/w/Adam:0' shape=(50, 784) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_3/w/Adam_1:0' shape=(50, 784) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_3/b/Adam:0' shape=(784,) dtype=float32_ref>,\n",
              " <tf.Variable 'linear_3/b/Adam_1:0' shape=(784,) dtype=float32_ref>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dz6grYm1fYWI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.reset_default_graph()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6ynfhMhhizIR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.reset_default_graph()\n",
        "\n",
        "inputs = tf.random_normal(shape=(1, 84, 84, 3))  # [N, H, W, C]\n",
        "\n",
        "encoder = Encoder()\n",
        "decoder = Decoder()\n",
        "\n",
        "latent_mu, latent_sigma = encoder(inputs)\n",
        "\n",
        "noise = tf.random_normal(shape=[N_LATENT])\n",
        "\n",
        "latent = latent_mu + noise * latent_sigma  # sik1\n",
        "\n",
        "output = decoder(latent)\n",
        "\n",
        "reconstruction_loss = tf.square(inputs - ae_output)\n",
        "\n",
        "# TODO(jarryd): Figrue out the closed form\n",
        "\n",
        "kl_loss = 0.5 * (latent_sigma^2 + latent_mu^2 - tf.log(latent_sigma^2) - 1)\n",
        "\n",
        "loss = reconstruction_loss + kl_loss  # + for sure\n",
        "\n",
        "\n",
        "optimizer = tf.train.AdamOptimizer(learning_rate=1e-4)\n",
        "\n",
        "sgd_op = optimizer.minimize(loss)\n",
        "\n",
        "\n",
        "with tf.train.MonitoredSession() as sess:\n",
        "  for step in range(N_TRAINING_STEPS):\n",
        "    sess.run(sgd_op)\n",
        "    \n",
        "  # Profit here."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "cellView": "form",
        "id": "PeDoKg6tiCnp",
        "colab": {}
      },
      "source": [
        "# @title examples\n",
        "\n",
        "# tf.reset_default_graph()\n",
        "\n",
        "# conv_net = snt.nets.ConvNet2D(output_channels=[32, 16, 8], kernel_shapes=[5, 3, 3], strides=[2, 2, 1], paddings=['SAME'])\n",
        "\n",
        "# mlp = snt.nets.MLP(output_sizes=[50, 50, N_LATENT])\n",
        "\n",
        "# inputs = tf.random_normal(shape=(1, 84, 84, 3))  # [N, H, W, C]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s2aWFmi0hbWP",
        "colab_type": "text"
      },
      "source": [
        "## The Classical Variational Inference Recipe\n",
        "\n",
        "1. Pick a model $p(x,z)$\n",
        "2. Pick a variational approximation $q(z;\\phi)$\n",
        "3. Write down the objective (the ELBO):\n",
        "$$ \n",
        "\\mathcal{L}(\\phi) = \\mathbb{E}_{q(z;\\phi)} \\big[ \\log p(x,z) - \\log q(z;\\phi) \\big]\n",
        "$$\n",
        "4. Take the expectation\n",
        "  * We need to take the expectation in order to make the dependence on the variational parameters $\\phi$ explicit, in order to take gradients in $\\phi$ to optimise\n",
        "5. Compute the gradient $\\nabla_\\phi \\mathcal{L}$\n",
        "6. Optimise: take a gradient step $\\phi^{\\tau + 1} = \\phi^{\\tau} + \\alpha\\nabla_\\phi\\mathcal{L}$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fVLWW2OxlwFs",
        "colab_type": "text"
      },
      "source": [
        "### The problem with the recipe and the solution\n",
        "* Problem: Step 4. **We often can't take this expectation analytically**.\n",
        "  * Therefore we can't make the dependence on the variational parameters explicit.\n",
        "  * Therefore we can't take the gradient. We could evaluate it by sampling and use non-gradient methods, but very inefficient.\n",
        "  \n",
        "### The Solution: push the gradient inside the expectation\n",
        "* Let's try to switch the order of steps 4 and 5.\n",
        "* We want to apply the gradient operator $\\nabla_\\phi\\mathcal{L}$ before we take the expectation in $\\mathcal{L}(\\phi) = \\mathbb{E}_{q(z;\\phi)} \\big[ \\log p(x,z) - \\log q(z;\\phi) \\big]$\n",
        "* We need a general way to take gradients of expectations\n",
        "* Then we use stochastic optimisation to take the expectation and profit."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aocJig3_jM9A",
        "colab_type": "text"
      },
      "source": [
        "### Computing Gradients of Expectations\n",
        "Let $g(z,\\phi) =  \\log p(x,z) - \\log q(z;\\phi)$ so that we can rewrite $\\mathcal{L}(\\phi) $ more simply\n",
        "\\begin{align}\n",
        "\\mathcal{L}(\\phi) \n",
        "&= \\mathbb{E}_{q(z;\\phi)} \\big[ \\log p(x,z) - \\log q(z;\\phi) \\big] = \\mathbb{E}_{q(z;\\phi)} \\big[g(z,\\phi)\\big] \n",
        "\\end{align}\n",
        "Now we will derive a new expression for the gradient of the ELBO by pushing the gradient operator inside the expectation:\n",
        "\\begin{align}\n",
        "\\nabla_\\phi\\mathcal{L} \n",
        "&=\\nabla_\\phi  \\int q(z;\\phi) g(z, \\phi)dz \\\\\n",
        "&=\\int \\big(\\nabla_\\phi q(z;\\phi)\\big)g(z, \\phi) + q(z;\\phi)\\big(\\nabla_\\phi g(z, \\phi)\\big) dz \\\\\n",
        "&=\\int \\big(q(z;\\phi)\\nabla_\\phi \\log q(z;\\phi)\\big)g(z, \\phi) + q(z;\\phi)\\big(\\nabla_\\phi g(z, \\phi)\\big) dz\\\\\n",
        "&=\\int q(z;\\phi)  \\big(\\nabla_\\phi \\log\\{q(z;\\phi)\\}g(z, \\phi) +\\nabla_\\phi g(z, \\phi)\\big) dz \\\\\n",
        "&=\\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\log\\{q(z;\\phi)\\}g(z, \\phi) +\\nabla_\\phi g(z, \\phi)\\big]\n",
        "\\end{align}\n",
        "Things to note:\n",
        "*  we have used the log-derivative trick: $\\nabla_\\phi \\log q = \\frac{\\nabla_\\phi q}{q}$\n",
        "* we have been able to introduce the density $q$ into both terms.\n",
        "  * q is a distribution we know and can sample from\n",
        "  * Once an expectation is in terms of a distribution i can sample from, i can use Monte Carlo to construct noisy unbiased gradients\n",
        "  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g6fA5kou_GXM",
        "colab_type": "text"
      },
      "source": [
        "## The score function estimator"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NRViItRkx4Ed",
        "colab_type": "text"
      },
      "source": [
        "### The score function\n",
        "The score function $\\nabla_\\phi  \\log q(z;\\phi) $ is a statistical measure of how sensitive a likelihood function is to its parameters. It's expectation $\\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\log q(z;\\phi)\\big] = 0$.\n",
        "\n",
        "### Rewriting the ELBO with the score function\n",
        "Now recall the expression for the gradient of the ELBO derived above:\n",
        "\\begin{align}\n",
        "\\nabla_\\phi\\mathcal{L}\n",
        "&= \\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\log\\{q(z;\\phi)\\}g(z, \\phi) + \\nabla_\\phi g(z, \\phi)\\big]\\\\\n",
        "&=  \\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\log\\{q(z;\\phi)\\}g(z, \\phi)\\big] +  \\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi g(z, \\phi)\\big]\n",
        "\\end{align}\n",
        "Expanding the right hand term:\n",
        "\\begin{align}\n",
        "\\nabla_\\phi g(z, \\phi)\n",
        "&= \\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\big(  \\log p(x,z) - \\log q(z;\\phi) \\big)\\big] \\\\\n",
        "&= \\mathbb{E}_{q(z;\\phi)}\\big[ \\nabla_\\phi \\big(- \\log q(z;\\phi) \\big) \\big]\\\\\n",
        "&=  - \\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\log q(z;\\phi)\\big] = 0\n",
        "\\end{align}\n",
        "\n",
        "Hence we can rewrite the gradient of the ELBO as:\n",
        "\\begin{align}\n",
        "\\nabla_\\phi\\mathcal{L}= \\quad &\\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\log\\{q(z;\\phi)\\}g(z, \\phi)\\big]\\\\\n",
        "= \\quad &\\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\log\\{q(z;\\phi)\\} \\big(\\log p(x,z) - \\log q(z;\\phi)\\big) \\big] \\\\\n",
        "\\approx \\quad &\\frac{1}{L}\\sum_{l=1}^L \\nabla_\\phi \\log\\{q(z^{(l)};\\phi)\\} \\big(\\log p(x,z^{(l)}) - \\log q(z^{(l)};\\phi) \\big) \\qquad z^{(i)} \\sim q(z;\\phi)\n",
        "\\end{align}\n",
        "\n",
        "This is the score function estimator. It is also called the likelihood ratio (from Monte Carlo literature), or REINFORCE gradient.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X6Ie6sNx6IXn",
        "colab_type": "text"
      },
      "source": [
        "### The Problem with the Score Function Estimator: High Variance\n",
        "The variance of the noisy unbiased gradients produced by the score function estimator can be a problem:\n",
        "\\begin{align}\n",
        "\\mathbb{V}_{q(z;\\phi)} = \\mathbb{E}_{q(z;\\phi)} \\bigg[\\big(\\nabla_\\phi \\log\\{q(z;\\phi)\\} (\\log p(x,z) - \\log q(z;\\phi) ) - \\nabla_\\phi\\mathcal{L}\\big)^2 \\bigg]\n",
        "\\end{align}\n",
        "\n",
        "Sampling values that are rare (wrt q) can lead to large scores and thus high variance in the gradient estimates."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d_bLFuDa9I9x",
        "colab_type": "text"
      },
      "source": [
        "### Solution to the Problem with the Score Function Estimator: Control Variates\n",
        "* Replace the function we're trying to compute the expectation of, with another function having the same expectation, but that leads to lower variance in the estimates.\n",
        "  * Replace $f$ with $\\hat{f}$ where $\\mathbb{E}[\\hat{f(z)}] = \\mathbb{E}[f(z)]$\n",
        "  * A general class is $\\hat{f(z)} = f(z) - a(h(z) - \\mathbb{E}[h(z)])$\n",
        "    * a is chosen to minimise the variance\n",
        "    * h is a function of our choice: good h have high correlation with the original function\n",
        "    \n",
        "  * To do black box variational inference, we want an $h$ that has known expectation for a large number of $q$\n",
        "  * A great $h$ is therefore the score function itself, with known expectation  $\\mathbb{E}_{q(z;\\phi)}\\big[\\nabla_\\phi \\log q(z;\\phi)\\big] = 0$."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iv1HT0vT_kRV",
        "colab_type": "text"
      },
      "source": [
        "### Summary of Requirements to do Black Box VI with the score function estimator\n",
        "* Sampling from variational approximation $q(z;\\phi)$\n",
        "* Evaluating $\\nabla_\\phi\\log q(z; \\phi)$\n",
        "* Evaluating $\\log p(x, z)$ and $\\log q(z)$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VSYmobF7AI1i",
        "colab_type": "text"
      },
      "source": [
        "## Pathwise Estimator\n",
        "Two additional assumptions:\n",
        "1. We can write our variational approximation in terms of a noise source that's parameter free. In other words: \n",
        "  * given some **parameter free source of noise** $\\epsilon\\sim s(\\epsilon)$, that is $\\phi$-independent, \n",
        "  * we can transform that noise variable with a deterministic parameterised function $z = t(\\epsilon, \\phi)$ that is dependent on the variational parameters, to get $z \\sim q(z; \\phi)$\n",
        "2. The model $p(x, z)$and the variational approximation $q(z; \\phi)$ are differentiable with respect to the variational parameters (smoothness in $\\phi$)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ldg8TqijI1Mg",
        "colab_type": "text"
      },
      "source": [
        "### The reparametrization trick\n",
        "Let's dwell a little on the trick used above, whereby we rewrite an expectation w.r.t.  $q(z;\\phi)$ such that the Monte Carlo estimate of the expectation is explicitly differentiable w.r.t. $\\phi$. Here we prove that this rewriting is valid:\n",
        "* Given the deterministic mapping $\\mathbf{z} = t(\\epsilon,\\phi)$, we know that\n",
        "\\begin{align} \n",
        "q_\\phi(\\mathbf{z})\\prod_i dz_i &= p(\\epsilon) \\prod_i d\\epsilon_i \\\\\n",
        "\\implies \\int q(z;\\phi)f(z)dz &= \\int p(\\epsilon)f(z)d\\epsilon \\\\\n",
        "&= \\int p(\\epsilon) f(t(\\epsilon;\\phi))d\\epsilon\n",
        "\\end{align}\n",
        "Honestly I don't know why the implication holds, or even really what the statement about infinitesimals means, let alone why it is true. It seems to say that since $t$ is a deterministic map, there must be some kind of preservation of measure in the infinitesimal cube. \n",
        "\n",
        "* It follows that a differentiable estimator can be constructed\n",
        "$$ \\int q(z \\mid x;\\phi)f(z)dz \\quad \\simeq \\quad \\frac{1}{L} \\sum_{l=1}^{L} f(t(x, \\epsilon,\\phi)) \\qquad \\epsilon\\sim p(\\epsilon)$$\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-w1cX8vSIy3o",
        "colab_type": "text"
      },
      "source": [
        "### Computing the Gradient of the ELBO in terms of our transformation and noise source\n",
        "We wil rewrite the expression for the gradient of the ELBO derived above in terms of $z = t(\\epsilon, \\phi)$\n",
        "\\begin{align}\n",
        "\\nabla_\\phi\\mathcal{L}\n",
        "&= \\mathbb{E}_{q(z;\\phi)}\\big[(\\nabla_\\phi \\log\\{q(z;\\phi)\\})g(z, \\phi) + \\nabla_\\phi g(z, \\phi)\\big]\\\\\n",
        "&=\\mathbb{E}_{s(\\epsilon)}\\big[\\nabla_\\phi (\\log\\{t(\\epsilon,\\phi)\\})g(t(\\epsilon, \\phi), \\phi) + \\nabla_\\phi g(t(\\epsilon, \\phi), \\phi)\\big]\\\\\n",
        "\\end{align}\n",
        "\n",
        "Note that the first term in the expectation \n",
        "$\\mathbb{E}_{s(\\epsilon)}\\big[(\\nabla_\\phi\\log s(\\epsilon))g(t(\\epsilon,\\phi),\\phi)\\big]       $ vanishes because $\\nabla_\\phi\\log s(\\epsilon)= 0$, since it is independent of $\\phi$. Hence we are left with the right hand term (which interestingly was the one the vanished when we derived the score function estimator!):\n",
        "\n",
        "\\begin{align}\n",
        "\\nabla_\\phi\\mathcal{L}\n",
        "&=\\mathbb{E}_{s(\\epsilon)}\\big[\\nabla_\\phi g(t(\\epsilon, \\phi),\\phi)\\big] \\\\\n",
        "&=\\mathbb{E}_{s(\\epsilon)}\\big[\\nabla_z g(z,\\phi)\\nabla_\\phi t(\\epsilon,\\phi)   + \n",
        "\\nabla_\\phi g(z,\\phi)\\big] \\\\\n",
        "&=\\mathbb{E}_{s(\\epsilon)}\\big[\\nabla_z (\\log p(x,z) - \\log q(z;\\phi))\\nabla_\\phi t(\\epsilon,\\phi)   + \n",
        "\\nabla_\\phi \\log p(x,z) - \\log q(z;\\phi)\\big] \\\\\n",
        "&=\\mathbb{E}_{s(\\epsilon)}\\big[\\nabla_z (\\log p(x,z) - \\log q(z;\\phi))\\nabla_\\phi t(\\epsilon,\\phi)   -\n",
        "\\nabla_\\phi \\log q(z;\\phi)\\big] \\\\\n",
        "&=\\mathbb{E}_{s(\\epsilon)}\\big[\\nabla_z (\\log p(x,z) - \\log q(z;\\phi))\\nabla_\\phi t(\\epsilon,\\phi)\\big] \\\\\n",
        "\\end{align}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HEM05rQUGiS9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}