diff --git a/tensorflow_probability/examples/jupyter_notebooks/Variational_Gaussian_Process_Multiclass_Classification.ipynb b/tensorflow_probability/examples/jupyter_notebooks/Variational_Gaussian_Process_Multiclass_Classification.ipynb new file mode 100644 index 0000000000..fa7c81e649 --- /dev/null +++ b/tensorflow_probability/examples/jupyter_notebooks/Variational_Gaussian_Process_Multiclass_Classification.ipynb @@ -0,0 +1,504 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Sparse Variational Gaussian Process in Multiclass Classification\n", + "\n", + "In this notebook, sparse variational gaussian process model (VGP) is applied to a multiclass classification problem. VGP is easily scalable to large scale dataset." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Background\n", + "\n", + "$\\qquad$ Consider making inference about a stochastic function $f$ given a likelihood $p(y|f)$ and $N$ observations $y=\\{y_1, y_2, \\dots, y_N\\}^T$ at observation index points $X=\\{x_1, x_2, \\dots, x_N\\}^T$. Place a GP prior on $f$: $p(f) \\sim N(f|m(X), K(X, X))$. The joint distribution of data and latent stochastic function is \n", + "\n", + "$$p(y, f) = \\prod_{i=1}^{N}p(y_i|f_i)N(f|m(X), K(X, X)) \\tag{1}$$ \n", + "\n", + "$\\qquad$ The main interest is the posterior over the function values given the observations $p(f|y)$. The posterior is intractable when the likelihood $p(y|f)$ is non-Gaussian, which is often the case in classification problems; and the computational complexity is $O(N^3)$ due to the inversion of $K_{X, X}$, which is also intractable for large dataset.\n", + "\n", + "$\\qquad$ To reduce the computational complexity, $M << N$ inducing index points $Z=\\{z_1, z_2, \\dots, z_M\\}^T$ and inducing variables $u=f(Z)$ are introduced. Assuming a GP prior on the joint density $p(f, u)$, $$p(f, u) = N\\begin{pmatrix} \\begin{bmatrix} f \\\\ u\\end{bmatrix}| \\begin{bmatrix} m(X) \\\\ m(Z)\\end{bmatrix}, \\begin{bmatrix} K(X, X) & K(X, Z) \\\\ K(Z, X) & K(Z, Z)\\end{bmatrix}\\end{pmatrix},$$ and a GP prior on $u$ $$p(u) = N(u|m(Z), K(Z, Z)),$$ the conditional of $f$ is $p(f|u) = N(f|\\mu, \\Sigma)$, where for $i, j = 1, \\dots, N$\n", + "\n", + "$$[\\mu]_i = m(x_i) + \\alpha(x_i)^T(u-m(Z)), \\tag{2}$$\n", + "\n", + "$$[\\Sigma]_{ij} = K(x_i, x_j) - \\alpha(x_i)^T K(Z, Z)\\alpha(x_j), \\tag{3}$$\n", + " where $\\alpha(x_i) = K(Z, Z)^{-1}K(Z, x_i)$, and the joint density of $y, f, u$ becomes\n", + "\n", + "$$p(y, f, u) = p(f|u; X, Z)p(u; Z)\\prod_{i=1}^{N}p(y_i|f_i)$$\n", + "\n", + "$\\qquad$ The goal is still finding the posterior of the function values $f$, however, the likelihood $p(y_i|f_i)$ is not Gaussian, so no closed-form solution for the posterior of $f$. Therefore, a variational posterior is used to solve the difficulty. \n", + "\n", + "$\\qquad$ Replacing the posterior $p(u|y)$ by an arbitrary full-rank Gaussian distribution $q(u)$ [Hensman et al. (2013)], then the variational posterior for $y$ and $u$ jointly becomes \n", + "\n", + "$$q(f, u; X, Z) = p(f|u; X, Z)q(u), \\tag{4}$$\n", + "\n", + "$$\\mbox{where } q(u) \\sim N(\\mathbf{m}, \\mathbf{S})$$\n", + "\n", + "$\\mathbf{m}, \\mathbf{S}$ are parameters to be chosen by optimizing an evidence lower bound (ELBO). \n", + "\n", + "$\\qquad$ Since both $p(f|u; X, Z)q(u)$ are Gaussian, the marginal variational posterior of $f$ can be computed analytically\n", + "\n", + "$$q(f|\\mathbf{m}, \\mathbf{S}; X, Z) = \\int p(f|u; X, Z)q(u) du \\sim N(\\tilde{\\mu}, \\tilde{\\Sigma}) \\tag{5}$$\n", + "\n", + "with $[\\tilde{\\mu}]_i = \\mu_{\\mathbf{m}, Z}(x_i), [\\tilde{\\Sigma}]_{ij} = \\Sigma_{\\mathbf{S}, Z}(x_i, x_j)$, and\n", + "\n", + "$$\\mu_{m, Z}(x_i) = m(x_i)+\\alpha(x_i)^T(\\mathbf{m}-m(Z)) \\tag{6}$$\n", + "$$\\Sigma_{S, Z}(x_i, x_j) = K(x_i, x_j) - \\alpha(x_i)^T[K(Z, Z) - \\mathbf{S}]\\alpha(x_j) \\tag{7}$$\n", + "\n", + "$\\qquad$ The variation parameters $Z, \\mathbf{m}, \\mathbf{S}$ in $q(f|\\mathbf{m}, \\mathbf{S}; X, Z)$ are determined by maximizing the lower bound \n", + "\n", + "$$L = \\sum_{i=1}^N \\mathbb{E}_{q(f_i|\\mathbf{m}, \\mathbf{S}; X, Z)}[logp(y_i|f_i)] - KL[q(u)|| p(u)], \\tag{8}$$\n", + "\n", + "where the expected log-likelihood can be computed with Gauss–Hermite quadrature.\n", + "\n", + "$\\qquad$ The variational posterior is given as $q(f)$ in (5). To make predictions for a set of test index points $X^*$, the new latent function values $f^*$ is approximated by\n", + "\n", + "\\begin{equation}\\begin{array}{rcl}p(f^*|y) &=& \\int p(f^*|f, u)p(f, u|y) df du\\\\ &\\approx& \\int p(f^*|f, u)p(f|u)q(u)df du \\\\ &=& \\int p(f^*|u)q(u) du \\\\ &=& q(f^*)\\end{array}\\end{equation}\n", + "\n", + "where the last line is following (5), (6) and (7) by replacing $x_i$ by $x_i^*$.\n", + "\n", + "$\\qquad$ With the variational posterior in (5), the predictive mean and variance of $y^*$ are computed as \n", + "\n", + "$$\\hat{y}^* = \\mathbb{E}(y^*) = \\int\\int y^* p(y^*|f^*)q(f^*) df^* dy^* \\tag{9}$$\n", + "$$\\hat{\\mathbb{V}}(y^*) = \\int\\int y^{*2} p(y^*|f^*)q(f^*) df^* dy^* \\tag{10}$$\n", + "\n", + "## References\n", + "\n", + "[1]: Titsias, M. \"Variational Model Selection for Sparse Gaussian Process Regression\", 2009. http://proceedings.mlr.press/v5/titsias09a/titsias09a.pdf \n", + "\n", + "[2]: Hensman, J., Lawrence, N. \"Gaussian Processes for Big Data\", 2013. https://arxiv.org/abs/1309.6835\n", + "\n", + "[3]: Salimbeni, H. and Deisenroth, M. \"Doubly stochastic variational inference for deep Gaussian processes.\" Advances in Neural Information Processing Systems. 2017. https://arxiv.org/pdf/1705.08933.pdf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Imports" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import tensorflow.compat.v2 as tf\n", + "import tensorflow_probability as tfp\n", + "import pandas as pd\n", + "from sklearn.preprocessing import LabelEncoder\n", + "from sklearn.model_selection import train_test_split\n", + "from scipy.cluster.vq import kmeans2\n", + "\n", + "tf.enable_v2_behavior()\n", + "\n", + "tfb = tfp.bijectors\n", + "tfd = tfp.distributions\n", + "tfk = tfp.math.psd_kernels\n", + "\n", + "dtype = np.float64" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load Glass Data\n", + "\n", + "A standard imbalanced machine learning dataset referred to as the “Glass Identification” dataset, or simply “glass”.\n", + "\n", + "The dataset describes the chemical properties of glass and involves classifying samples of glass using their chemical properties as one of six classes. The dataset was credited to Vina Spiehler in 1987." + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "--2020-11-08 09:08:57-- https://raw.githubusercontent.com/jbrownlee/Datasets/master/glass.csv\n", + "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.192.133, ...\n", + "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", + "HTTP request sent, awaiting response... 200 OK\n", + "Length: 11154 (11K) [text/plain]\n", + "Saving to: 'glass.csv.1'\n", + "\n", + "100%[======================================>] 11,154 --.-K/s in 0.001s \n", + "\n", + "2020-11-08 09:08:57 (11.2 MB/s) - 'glass.csv.1' saved [11154/11154]\n", + "\n" + ] + } + ], + "source": [ + "!wget https://raw.githubusercontent.com/jbrownlee/Datasets/master/glass.csv" + ] + }, + { + "cell_type": "code", + "execution_count": 102, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(171, 9) (43, 9) (214,)\n" + ] + } + ], + "source": [ + "data = pd.read_csv('glass.csv', header=None)\n", + "data = data.values\n", + "X = data[:,0:9].astype(dtype)\n", + "Y = data[:,9]\n", + "\n", + "encoder = LabelEncoder()\n", + "encoder.fit(Y)\n", + "encoded_Y = encoder.transform(Y)\n", + "encoded_Y = encoded_Y.astype(dtype)\n", + "num_outputs = 6\n", + "\n", + "X_train, X_test, y_train, y_test = train_test_split(X, encoded_Y, test_size=0.2, random_state=42)\n", + "print(X_train.shape, X_test.shape, encoded_Y.shape)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Defining Trainable Variables in VGP\n", + "\n", + "* Using kmeans to initialize 30 representative `inducing_index_points` $Z$ and make them learnable variable" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "metadata": {}, + "outputs": [], + "source": [ + "num_inducing_points_ = 30\n", + "inducing_index_points_init = kmeans2(X_train, num_inducing_points_, minit=\"points\")[0] #50, 60\n", + "inducing_index_points = tf.Variable(inducing_index_points_init, dtype=dtype, name='inducing_index_points')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* Initializing RBF kernel and kernel parameters, which are `amplitude` and `length_scale` (the same length scale is used for all $X$ columns)\n", + "* Initializing the variational mean and covariance $\\mathbf{m}, \\mathbf{S}$ in $q(u)$" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": {}, + "outputs": [], + "source": [ + "amplitude = tfp.util.TransformedVariable(\n", + " 1., tfb.Softplus(), dtype=dtype, name='amplitude')\n", + "length_scale = tfp.util.TransformedVariable(\n", + " 1., tfb.Softplus(), dtype=dtype, name='length_scale')\n", + "kernel = tfk.ExponentiatedQuadratic(amplitude=amplitude, length_scale=length_scale)\n", + "\n", + "observation_noise_variance = tfp.util.TransformedVariable(1., tfb.Softplus(), dtype=dtype, name='observation_noise_variance')\n", + "\n", + "variational_inducing_observations_loc = tf.Variable(np.zeros([num_outputs, num_inducing_points_], dtype=dtype), name='variational_inducing_observations_loc')\n", + "\n", + "Ku = kernel.matrix(inducing_index_points, inducing_index_points)\n", + "variational_inducing_observations_scale_init = np.linalg.cholesky(Ku + np.eye(num_inducing_points_)*1e-6)\n", + "variational_inducing_observations_scale = tf.Variable(np.tile(variational_inducing_observations_scale_init[None, :, :], [num_outputs, 1, 1]), \n", + " name='variational_inducing_observations_scale')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* Defining log probability. For multiclass classification, Categorical distribution is used. The `observations` is a flat array of batch size; since the expected log likelihood in VGP is approximated by Gauss–Hermite quadrature, the input logits is reshaped to (`quadrature_size, batch_size, num_outputs`) to adapt to the `sparse_softmax_cross_entropy_with_logits` in `log_prob`\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "metadata": {}, + "outputs": [], + "source": [ + "def log_prob(observations, f):\n", + " #f is (6, 20, 64)\n", + " berns = tfd.Independent(tfd.Categorical(logits=tf.transpose(f, perm=[1,2,0])), 1) #(20, 64, 6), n_quadrature, bs, n_outputs\n", + " return berns.log_prob(observations) #sparse_softmax_cross_entropy_with_logits: have logits of shape [batch_size, num_classes] and have labels of shape [batch_size]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Constructing Model and Training" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "metadata": {}, + "outputs": [], + "source": [ + "vgp = tfd.VariationalGaussianProcess(\n", + " kernel,\n", + " index_points=X_test,\n", + " inducing_index_points=inducing_index_points,\n", + " variational_inducing_observations_loc=variational_inducing_observations_loc, #TensorShape([6, 30])\n", + " variational_inducing_observations_scale=variational_inducing_observations_scale, #TensorShape([6, 30, 30])\n", + " observation_noise_variance=observation_noise_variance)" + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "metadata": {}, + "outputs": [], + "source": [ + "batch_size = 64\n", + "\n", + "optimizer = tf.optimizers.Adam(learning_rate=.01)\n", + "\n", + "@tf.function\n", + "def optimize(x_train_batch, y_train_batch):\n", + " with tf.GradientTape() as tape:\n", + " # Create the loss function we want to optimize.\n", + " recon = vgp.surrogate_posterior_expected_log_likelihood(\n", + " observations=y_train_batch,\n", + " observation_index_points=x_train_batch,\n", + " log_likelihood_fn=log_prob,\n", + " quadrature_size=20)\n", + "\n", + " elbo = -tf.reduce_sum(recon) + tf.reduce_sum(vgp.surrogate_posterior_kl_divergence_prior())\n", + "\n", + " grads = tape.gradient(elbo, vgp.trainable_variables)\n", + " optimizer.apply_gradients(zip(grads, vgp.trainable_variables))\n", + " return elbo" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Training by Batch" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0 114.67260603059552\n", + "120 67.871934322895\n", + "240 60.0149377742081\n", + "360 56.29566984959375\n", + "480 53.7197290981309\n", + "600 51.48258717207506\n", + "720 52.6730473693974\n", + "840 44.445047997931475\n", + "960 50.50109738207267\n", + "1080 44.75255389124902\n", + "1199 51.55517736426542\n" + ] + } + ], + "source": [ + "num_iters = 1200\n", + "num_logs = 10\n", + "num_training_points_ = X_train.shape[0]\n", + "\n", + "for i in range(num_iters):\n", + " batch_idxs = np.random.randint(num_training_points_, size=[batch_size])\n", + " x_train_batch = X_train[batch_idxs, ...]\n", + " y_train_batch = y_train[batch_idxs]\n", + " loss = optimize(x_train_batch, y_train_batch)\n", + "\n", + " if i % (num_iters / num_logs) == 0 or i + 1 == num_iters:\n", + " print(i, loss.numpy())\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Computing the Predictive Mean and Variance\n", + "\n", + "To compute the predictive mean and variance for a set of new $X^*$, the `predict_mean_and_var` from gpflow is used to compute (9) and (10). " + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "metadata": {}, + "outputs": [], + "source": [ + "from gpflow.likelihoods.multiclass import Softmax\n", + "\n", + "Fmu = tf.cast(tf.transpose(vgp.mean()), tf.float32) #TensorShape([6, 43])\n", + "Fvar = tf.cast(tf.transpose(vgp.variance()), tf.float32)##TensorShape([6, 43])\n", + "\n", + "S = Softmax(num_outputs)\n", + "m, v = S.predict_mean_and_var(Fmu, Fvar) #shape=(43, 6)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Results and Plots" + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Multiclass classification accuracy for 6 is 0.7209302325581395\n" + ] + } + ], + "source": [ + "acc =np.mean(np.argmax(m, 1).astype(int) == y_test.astype(int))\n", + "print(\"Multiclass classification accuracy for {} is {}\".format(num_outputs, acc))" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Multiclass classification variances for 6 is [0.23027779 0.21305028 0.23469782 0.2065194 0.23566929 0.18067268\n", + " 0.21430728 0.2347648 0.2097274 0.21258086 0.2179921 0.21743922\n", + " 0.2189273 0.22974259 0.22108854 0.20057674 0.1710678 0.21648297\n", + " 0.19312762 0.18159598 0.18870537 0.22279613 0.2087079 0.2288218\n", + " 0.19952135 0.21122998 0.21671966 0.22578193 0.2281411 0.24098359\n", + " 0.17000449 0.2195439 0.20093569 0.20745346 0.19372132 0.24052121\n", + " 0.15541928 0.22103915 0.19316027 0.20382655 0.16869901 0.22654828\n", + " 0.19985217]\n" + ] + } + ], + "source": [ + "indices = np.argmax(m, 1)\n", + "v_ = tf.reduce_sum(tf.one_hot(indices, 6)*v, 1)\n", + "print(\"Multiclass classification variances for {} is {}\".format(num_outputs, v_))" + ] + }, + { + "cell_type": "code", + "execution_count": 98, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.decomposition import PCA\n", + "\n", + "pca = PCA(n_components=1)\n", + "X_test_pca = pca.fit_transform(X_test)\n", + "y_preds = np.argmax(m, 1).astype(int)\n", + "y_sd = (v_)**0.5" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Each dot is the predicted class for each $x_i^*$, and the error bar is one sd of $y_i^*$. If the a dot and a cross overlap, this is a correct prediction." + ] + }, + { + "cell_type": "code", + "execution_count": 99, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Text(0.5, 0, 'X_test_pca')" + ] + }, + "execution_count": 99, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA24AAAE+CAYAAAD1QEO5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU1f3G8c9JMkkmOyEhhH1RBCQIGEVccKlWXKrV/nCrtW5FbW1rXWtrK7V73apdrLhWrWhxFxHRKiKImKAsAWRfQjayJySTZJbz+yOLhIRkgJnMBJ63r7zMPXPnnO/cZCbzcO85Y6y1iIiIiIiISPiKCHUBIiIiIiIi0jUFNxERERERkTCn4CYiIiIiIhLmFNxERERERETCnIKbiIiIiIhImFNwExERERERCXNRoS5gT2lpaXbYsGGhLkNERERERCQkli9fXmatTd+7PayC27Bhw8jNzQ11GSIiIiIiIiFhjNneWbsulRQREREREQlzCm4iIiIiIiJhTsFNREREREQkzIXVHDcREREREQkvbrebnTt30tDQEOpSDimxsbEMGjQIh8Ph1/4KbiIiIiIisk87d+4kMTGRYcOGYYwJdTmHBGst5eXl7Ny5k+HDh/t1H10qKSIiIiIi+9TQ0EDfvn0V2gLIGEPfvn336yymgpuIiIiIiHRJoS3w9veYKriJiIiIiEhAzcnNZ05ufsD6i4yMZMKECYwbN47p06dTX19/wH1dffXVvPLKKwBcf/31rF27tsM+zz77LDfffDMAb7zxRrt9fv3rX/PBBx8c8PgHSsFNRERERETCmtPpZMWKFeTl5REdHc2//vWvdrd7vd4D6vfJJ59k7NixXe6zd3C77777OPPMMw9ovIOh4CYiIiIiIgGzrqiaBWuKeTknn4ff38C6ouqA9n/KKaewadMmFi5cyOmnn84VV1xBVlYWXq+XO+64g+OOO47x48fz+OOPA80Lgdx8882MHTuW8847j127drX1ddppp5GbmwvAM888w6hRozj11FNZsmQJAJ9++ilvvfUWd9xxBxMmTGDz5s3tztj973//Y+LEiWRlZXHttdfS2NgIwLBhw7j33nuZNGkSWVlZfPXVVwf9uBXcREREJKgCfcmUiISvdUXVzFq0FVeTlxRnFNUuN7MWbQ1YePN4PLz77rtkZWUB8Pnnn/P73/+etWvX8tRTT5GcnExOTg45OTk88cQTbN26lddff53169ezevVqnnjiCT799NMO/RYVFXHvvfeyZMkS3n///bYzbCeeeCIXXHAB999/PytWrGDkyJFt92loaODqq6/m5ZdfZvXq1Xg8Hh577LG229PS0vjiiy+46aabeOCBBw76sSu4iYiISI8JVYjralwFS5HAmZ9XQrLTgTM6EmMMyU4HyU4H8/NKDqpfl8vFhAkTyM7OZsiQIVx33XUAHH/88W3L6S9YsIDnnnuOCRMmMHnyZMrLy9m4cSOLFi3i8ssvJzIykgEDBnDGGWd06H/ZsmWcdtpppKenEx0dzaWXXtptTevXr2f48OGMGjUKgO9///ssWrSo7faLL74YgGOPPZZt27Yd1OMHfY6biIiI9LCcbRUATM8efFD9tIatrvpRIBPpWQVVLjKTY9u1JcZGUVDlOqh+W+e47S0+Pr7te2stf/vb3zj77LPb7TNv3jy/VnDc31UerbVd3h4TEwM0L6zi8Xj2q+/O6IybiIiIiIgExMAUJ7UN7UNKbYOHgSnOoI999tln89hjj+F2uwHYsGEDdXV1TJ06lZdeegmv10tRUREfffRRh/tOnjyZhQsXUl5ejtvtZs6cOW23JSYmUltb2+E+o0ePZtu2bWzatAmA559/nlNPPTVIj07BTUREREREAmTauAyqXW5cTV6stVS73FS73EwblxH0sa+//nrGjh3LpEmTGDduHDfccAMej4eLLrqII488kqysLG666aZOw1VmZiYzZ85kypQpnHnmmUyaNKnttssuu4z777+fiRMnsnnz5rb22NhYnnnmGaZPn05WVhYRERHceOONQXt8prtTfD0pOzvbtq7qIiIiIoeGPS9pnJObT862Co4blhqSSyU729effkQOZ+vWrWPMmDH+719UzUMLNlBZ7+akI9KYNi6DMZnJQayw9+rs2Bpjlltrs/feV3PcREREREQkYMZkJvPNo/sD+geRQFJwExERERGRgFJgCzzNcRMREREREQlzCm4iIiIiItKlcFoX41Cxv8dUwU1ERERERPYpNjaW8vJyhbcAstZSXl5ObGxs9zu30Bw3ERERERHZp0GDBrFz505KS0tDXcohJTY2lkGDBvm9v4KbiIiIiIjsk8PhYPjw4aEu47CnSyVFRERERETCnIKbiIiIiIhImFNwExERERERCXMKbiIiQWK9pVjr6qS9DOurD0FFIj3HevLbvo+OqCbK1GE9O4iLLPp6H18V1leL9ew44L47a7e+Cqxvd8v216vgRUdUt7X73J2Pub+1iIj0lKAGN2PMNmPMamPMCmNMbjDHEhEJJ9Y2YSu/j628oV14s95SbMWV2OpbQlidSHDZpi+xZdOwdf/GWi8npd3NNzJuwJadzdn9v8+E9E9wmBpsxdXYsguxZedg3Wv867thfnPfDe+1b69/EVt2Nr7Gz7AV12Ervoctv5Bjkv8BWKIjqjkl7XZsxVXY8gug/JvYumf26vuj5lpcbwTqUIiIBExPnHE73Vo7wVqb3QNjiYiEBWOiMfE3QNPnbeGtObR9D3zFmPgfhLpEkeBxjIOY07G1v6e4+AleXnMOzogSrPUCXvIKKjgq9id43V+BbyfEToOo0f71HX0yOLLwVd5CadlbvJyTzwfL/4qtmQkxUzHRkzAJN4JnHW5fLEckvk58w28Y6/wxcRHbaWrcRJMvCaJPwdb+keKix1iwppiN+W/jqfgRLnsExJwezKMjInJATDA/SM8Ysw3IttaW+bN/dna2zc3ViTkROXRY15vY6rsgagT46sFWYfo8gYk+LtSliQSVtW5qim8m0XzEmtKxjO77FRHGh9sbiSPSi8UQYSyfFZ5In8yHGJOZ6nffXxUV4qi5gcGJm9hQMYaxaXmsKTsWR99HGZ2ZDkB+0ev05+fUNzlJjKnHZw0NHgcuTzKPrZzJ9MmTGMA9JJr/saZ0LEf1XU9J/XAeW/kLvndiFmMyk4N1aEREumSMWd7ZSa9gn3GzwAJjzHJjzIwgjyUiEnaM80JM4h3g2QS+QkzKowptclgwxsGzeT/F44vi6PS1REb4eOiz6URHeTEGIoyluiGR+z+9lvl5fv37bpt383bz8sZ7iYrwMjYtD4A3tt7Fu3mVbfu8knc07267gcSY5vmkEcYS52hi5id34YsYwPy8cp7N+zFuXzRHp68lKsLL29t/R7SjD/PzSgJ3IEREAiTYwe0ka+0k4BzgR8aYqXvvYIyZYYzJNcbk6tPYReRQY72l2Po5X2/XPdnpgiUih6KEiKVERXjatq/Mer/d7fHRdYxJW0NB1f49JwqqXBzX/5N2bePTv2jXT+XuEib3n9/hvmcNm0dibCQFVS4SIj4jyrjbbhvb5wMSY6P2ux4RkZ4Q1OBmrS1s+f8u4HXg+E72mWWtzbbWZqenpwezHBGRHtVuTlvqfzDJ97eb8yZyKLMNH3LVmAcp2H0E9yx6gNomJxkJVQDkFo7A44vA44vi16c8ypRB6/ar72kjPuIbgx5j5a5J3PrhvyisG8O5Q+7n1CFfNI/tq+BHE35Ln5id1LtjqHQl4PUZSuuTuWDUh5zY73FOHryKq8Y8QGHdSG778DGWFx/P1AFPMjb5NQamOAN+PEREDlbQgpsxJt4Yk9j6PfBNIC9Y44mIhJPmVSWvbg5tLXPajPNCTPKfm8NblVaVlEOXbVqBrfoxTWYUj624m2uy/okzqrHt9lfXnsi9C39AVIQHn43iwmG/xbrX+td3w3ucMfCfrCk7lkdyfsTupgSeW/crdtQewblD/tK2qmTf2EK8vihqmxK57q27+PVH19MntoYKV18m93+Hbw39PU3mSB5b+QtK6+J5cuUPWVsxhbOHPsMl45YH69CIiBywqCD2nQG8boxpHedFa23HaxZERA5BxkRD/AyIHNBuTptxXghEQES/0BUnEmyOsRB/HXHx1/O9kyzvr/wWrqYmJmYWU7a7kSX5WXhJ4uUNt3HaUckMTCiEqFH+9R19EsTfgMNehSNqG5X1bpwxabiTHsckzMZET4KEGUSQQE1lLn/NPYomm8BnRf34y9IoxgzI4ORhxaTH1RCX+DO+dyI8tGAD5fU+/ldwFxl93mJAxrnBPT4iIgcgqKtK7i+tKikiInLomZPb/MHY07MHMyc3n5xtFRw3LJXp2YMD1m93+7TqbF9/+hER6SmhWlVSREREREREDpKCm4iIiIiISJhTcBMREREREQlzCm4iIiIiIiJhTsFNREREREQkzCm4iYiIiIiIhDkFNxERERERkTCn4CYiIiIiIhLmFNxERERERETCnIKbiIiIiIhImFNwExERERERCXMKbiIiIiIiImFOwU1ERERERCTMRYW6ABEREZEDMT17cKhLEBHpMQpuIiIi0mOmZw/u0cDlz1gKgCLSGyi4iYiISFApGImIHDzNcRMREREREQlzCm4iInuYk5vPnNz8UJchIiIi0o6Cm4iIiIiISJhTcBMREREREQlzCm4iIiIiIiJhTsFNREREREQkzCm4iYiIiIiIhDkFNxERERERkTCn4CYiIiIiIhLmFNxERERERETCnIKbiIiIiIhImFNwExERERERCXMKbiIiIiIiImFOwU1ERERERCTMKbiJiIiIiIiEOQU3ETnsWWv3q10Ob9baTn839tUucqjRa6ZIaAQ9uBljIo0xXxpj5gZ7LBGR/VXnqWHW5l+wZXdeu/Z6Ty2zNv+CzbtXh6gyCUfWWhYU/4d3Cp9u9ybVWssHJbN5u/BJvXmVQ5rXenl5x0MsLXunQ/t/8//KktK3Q1SZyKEvqgfG+CmwDkjqgbFERPaLxUeDt57ntv6OU5N+xoI1EVQ31bDavIZ1lOKznlCXKGHGa90sLW9+03regGsB+KBkNgt3vcJxqWdhsRhMKEsUCSKL13qYW/gUAFPSzsNrvbyS/yirqj4hM3ZoiOsTOXQFNbgZYwYB5wG/B24N5lgiIgciISqF60b8hn9uuIf3Kx/ExJ/HsKOW4o2qoGHHJXjSR0BiqKuUcGGM4ZzMqwFYUtZ8ZiE6MpaPd73KcalnccHAG4gwmoUgh65IE8WlQ27l5R0PMbfwKXzWx07XJlZVfcLZ/a9kar+LQ12iyCEr2Gfc/grcid72iEgYS3CkEFl8DSb1EdJHvQlAUuW1OHzDmZ9XwpjM5BBXKOGks/CWrdAmh5GoCAeXDrmV2dvvZ17RMwAKbSI9IGh/YYwx5wO7rLXLu9lvhjEm1xiTW1paGqxyRES6VFTVhNnzJdFGkhgbRUGVK3RFSViLiohu+95hHLo8Ug4rxkQQFeFo23ZExISwGpHDQzD/afAk4AJjzDbgJeAMY8wLe+9krZ1lrc221manp6cHsRwRkc7Ve2qJG/YCNqKemm3n43GlUdPnWarZyMAUZ6jLkzDTuhDJx7teJTv1LE5MO5+l5fM6LFgicqhqndOWV72UMzMuZ2zSZOYWPtVhwRIRCaygXSpprb0buBvAGHMacLu19spgjSciciDqPbU8vWUm1lFKw45LqCwZRGPNcNJHv4Sn33NM6HNbqEuUMNIa2loXIrlg4A2Ylv9aL5s8b8C1GKOzb3Jo2nMhktbLIz0+d9ucN2hesEREAk8X44vIYc3bsmrkVcPv5vpjz8IZHUlFbQyp1TPoGz2QAX0c3fQgh5smn6vdQiStc95OSvsWTb4GLDrrJocua324fY3t5rS1znkbmzSZJl9DiCsUOXSZcLqsIzs72+bm5oa6DBE5zPisr21RiTm5+QBMzx7crl2klbUWi+3wu7GvdpFDzb5eG/WaKRIYxpjl1trsvdt74nPcRETC2r7eaOgNiHTGGNPpQiT7ahc51Og1UyQ09AwTEREREREJcwpuIiIiIiIiYU7BTUREREREJMwpuImIiIiIiIQ5BTcREREREZEwp+AmIiIiIiIS5vYruBlj+hhjxgerGBEREREREemo2+BmjFlojEkyxqQCK4FnjDEPBb80ERERERERAf/OuCVba2uAi4FnrLXHAmcGtywRERERERFp5U9wizLGZAKXAHODXI+IiIiIiIjsxZ/gdh/wHrDZWptjjBkBbAxuWSIiIiIiItIqqrsdrLVzgDl7bG8BvhPMokRERERERORr/ixOMsoY8z9jTF7L9nhjzD3BL01ERERERETAv0slnwDuBtwA1tpVwGXBLEpEJFSmZw9mevbgUJchIiIi0o4/wS3OWvv5Xm2eYBQjIiIiIiIiHfkT3MqMMSMBC2CM+T+gKKhViYiIiIiISJtuFycBfgTMAkYbYwqArcCVQa1KRERERPZpTm4+gC7tDjAdVwln/qwquQU40xgTD0RYa2uDX5aIiIiIHGoUjEQOnD+rSv7UGJME1AMPG2O+MMZ8M/iliQg0/5Fr/UMnIiIiIocnf+a4XWutrQG+CfQDrgH+FNSqREREREREpI0/wc20/P9c4Blr7co92kRERERERCTI/Aluy40xC2gObu8ZYxIBX3DLEhERERERkVb+rCp5HTAB2GKtrTfGpNJ8uaSIiIiIiIj0AH/OuE0B1ltrq4wxVwL3ANXBLUtERERERERa+RPcHgPqjTHHAHcC24HnglqViIiIiIiItPEnuHmstRa4EHjEWvsIkBjcskRERERERKSVP3Pcao0xdwNXAlONMZGAI7hliYiIiIiISCt/zrhdCjQC11lri4GBwP1BrUpERERERETadHvGrSWsPbTH9g40x01ERERERKTHdHvGzRhzgjEmxxiz2xjTZIzxGmO0qqSIiIiIiEgP8edSyb8DlwMbASdwPfCPYBYlIiIiIiIiX/MnuGGt3QREWmu91tpngNO6u48xJtYY87kxZqUxZo0x5jcHWauIiF/e2r6G1RVFHdrn7ljLqvLCEFQkInJwrLU8tX4ZJa7aDu1Pr19GcX1NiCrrnsvj5m9rPqHJ623X3rCP9p72SfEWFhVt7tC+pGQrC4s2haAikc75s6pkvTEmGlhhjPkLUATE+3G/RuAMa+1uY4wDWGyMedda+9lB1CtyWFlXVM2CNcVU1rvZWeli2rgMxmQmh7qssNbo9fBw3sdUNbp47rQryErNBJrD3G3L3uSMAUfy+MnTQ1yliMj+Kaiv5pG8Rcze/AUzx17AgjXFVNQ3Mafgc5Y3fEWNu4Fbxp0a6jI7tah4M3/NW8RnRTuJLRhBdb2XbRW7+czmsLIqn4l9B3Jy/xEhqc1ayyN5i1hTWcw9Y89j4ZoGKuvdLN21jXfrljA6pR9T+48kwpiQ1CeyJ3/OuH0PiARuBuqAwcB3uruTbba7ZdPR8mUPsE6Rw866ompmLdqKq8lLijOKapebWYu2sq5IU0y7EhMZxQunfZek6BiuWvgiqyuK2kLbcWmDefiEC0NdoojIfhsUn8LTUy+juL6WHy6bQ7W7nsrE7Sxv+IqhDOas1GNCXeI+nT1oNDeMnMpn5VtYGbmSBKfljepPWFGVz09HfSNkoQ3AGMMTp1zCIGcfZubNpci7C198NXNrFxPji+OuUecqtEnY6Da4WWu3W2td1toaa+1vrLW3tlw62S1jTKQxZgWwC3jfWrvsYAsWOVzMzysh2enAGR2JMYZkp4Nkp4P5eSWhLi3sDYxP5sXTv0dSdAzffv9pfvbZGxyXNpgnp15KXFR0qMsTETkg2emDOTfuZBppZHniYrY7tjI2ZjinxE5iwZpdoS6vS47KDI5zjKfSUconzoUU+0o50TERT1lqqEujT0wcpztOJNkksi5+BStjviAlMpFpsSezdEP4XoIqh599BjdjzGpjzKp9ffnTecucuAnAIOB4Y8y4TsaZYYzJNcbklpaWHvgjETnEFFS5SIxtfzVzYmwUBVWuEFXUuwyMT+bykZPatm8Zd6pCm4j0et7dcfSLSmnbPt55NElOR9j/bSiocjExfmS7tqz44WFTd0WNj+PiRrdtT4o9inRnfNjUJwJdn3E7H/hWF19+s9ZWAQuBaZ3cNstam22tzU5PT9+fbkUOaQNTnNQ2eNq11TZ4GJjiDFFFvctb29fw4OqFDI5PITUmjhsWz+l0wRIRkd7CWkt+zGYKPWXE++KJtJG8UbOQEldt2P9tyEiO5q2axWAh0ZcEwLyapfRPjglxZc188TX8ry6HaBtDjI3hw7pc1tcVhv1xlcPLPoNbyyWS+/zqrmNjTLoxJqXleydwJvBV4EoXObRNG5dBtcuNq8mLtZZql5tql5tp4zJCXVrY23NO27xpP+CNs65tN+dNRKS3sdby4OqFbXPasnZPZnzTJHb7GpjfsJjjjvRn3bjQaPC4+czmUOwrZWTDGLIbJ3OcYzz5viK+jPgy5KtKLinZyrt1S4gjnvG7j+e4hhNIMol81PQZ/QY0hrQ2kT11dankdcaYO/bYLjDG1Bhjao0xN/nRdybwUctllTk0z3Gbe/AlixwexmQmM2PqcJzRkVS5PCQ7HcyYOlyrSnaj0evh4dUL281p23PO29/XfBLqEkVE9ltBfTX/3pjDpSMm8LdTLiAuOgrqErgw8RR8kU3k7g7fZes/Lt7Mquqd3DLqGww1g6lyeZiceBQ3jJzKsvItfF7a7fmAoLHW8vDqjxme2Jd/nXAJSQ4nda4ILk4+lWHxqcwpyMFntbaehAdj9/HLaIzJAaZZa8tbtr+01k40xsQCC6y1UwNdTHZ2ts3NzQ10tyK92pzcfACmZw8OcSW9R2FdNSkxzg5z2grra0h2xBLv0Fw3Eel9ttSUMywxlQhj2v1t2LM9XG2qKeOIpLQOf9M215QxMiktlKVR0ViPtZa+sfHt6qtsrMdrLWmx4Xs2Uw5Nxpjl1trsvdu7+hy3iNbQ1mIOgLW2oeXSRxGRsDQgvvOzkgPiknq4EhGRwBmR1He/2sPJEfsIZ6EObQCpMXGdtvfZR7tIqHS1OEm7dz7W2j8AGGMigPB/hRARERERETlEdBXcFhhjftdJ+33AgiDVIyIiIiIiInvp6lLJO4AnjTGbgJUtbccAucD1wS5MREREREREmu0zuFlr64DLjTEjgKNbmtdaazf3SGUiIiIiIiICdH3GDQBr7RZgSw/UIiIiIiIiIp3oao6biIiIiIiIhIGuPoB7eE8WIiIiIiIiIp3r6ozbKwDGmP/1UC0iIiIiIiLSiS4/gNsYcy8wyhhz6943WmsfCl5ZIiIiIiIi0qqrM26XAQ00h7vETr5ERERERESkB3T1cQDrgT8bY1ZZa9/twZpERERERERkD91+HADwqTHmIWBqy/bHwH3W2urglSUiraZnDw51CSIiIiISYv58HMDTQC1wSctXDfBMMIsSERERERGRr/lzxm2ktfY7e2z/xhizIlgFiYiIiMihSVeRiBw4f4KbyxhzsrV2MYAx5iTAFdyyRERERGRfFICCQ8dVwpk/we1G4DljTHLLdiXw/eCVJCIiIiIiInvqNrhZa1cCxxhjklq2a4JelYiIiIiIiLTx54wboMAm3ZuTmw/oMgMRERERkUDzZ1VJERER6cSc3Py2f7QSEREJJgU3ERERERGRMNdtcDPGTDfGJLZ8f48x5jVjzKTglyYiIiIiIiLg3xm3X1lra40xJwNnA/8GHgtuWSIiIiIiItLKn+Dmbfn/ecBj1to3gejglSQiIiIiIiJ78ie4FRhjHgcuAeYZY2L8vJ+IiIiIiIgEgD8B7BLgPWCatbYKSAXuCGpVIiIiIiIi0qbb4GatrQd2ASe3NHmAjcEsSkRERERERL7mz6qS9wJ3AXe3NDmAF4JZlIiIiIiIiHzNn0slLwIuAOoArLWFQGIwixIREREREZGv+RPcmqy1FrAAxpj44JYkIiIiIiIie/InuP23ZVXJFGPMD4APgCeCW5aIiIiIiIi0iupuB2vtA8aYs4Aa4Cjg19ba94NemYiIiIiIiAD+LU4SD3xorb2D5jNtTmOMI+iVSQflu+vZVbO7Q3tlnYuS6o7tB+urotJu2+sam9hRXtVhn7rGJraXdWwXEREREZH958+lkouAGGPMQJovk7wGeLa7OxljBhtjPjLGrDPGrDHG/PTgSj28WWv5yfNvcfWsOe3CW2Wdi2uffIUbn30dr88XsPHmr1rPdx59gTmfr27X/tTHOXzn0Rf4dON2AH4x5z2uevy/bCutbNunrrGJm559g2ufnEOD2xOwmkREREREDlf+BDfT8lluFwN/s9ZeBIz1434e4DZr7RjgBOBHxhh/7iedMMZw+7lTKa2tawtvraFte1kld543lcgIf36c/jl9zEimHjWcma9/0Bbenvo4h4fmL+ac8Udx/IjBAPz4rCl4fT6ufPxl3vxyK7OXbeOCh2fz5fZCbj93KrGObq/GFRHpldYVVbNgTTEv5+Tz8PsbWFdUHeqSRETkEGaaF4zsYgdjvgR+CDwMXGetXWOMWW2tzdqvgYx5E/h7V/PjsrOzbW5u7v50e9j5cnshM55+jajICCIjIqhvbOIf37+QKUcMDfhYjW4Pt/xnLovWb2Vo3xS2l1dxzvij+NMl04iK/DokLsjbxp0vz8Xr9REZEYHH6yFr6Bh+feEJjMlMDnhdIiKhtq6omlmLtlJa00CsI4JBqfFUu9zMmDpcr3siInJQjDHLrbXZe7f7c4rmFpo/fPv1ltA2AvhoPwcfBkwElu3P/aSjiUMH8OdLz6HG1UhlnYt7LzozKKENIMYRxV+/ez4A21vmse0d2gDWFDVxyuiJ+KwXt9fNpOFHctSAQczPKwlKXSIioTY/r4RkpwNndCTGGJKdDpKdDr3uiYhI0HQb3Ky1H1trL7DW/tkYEwGUWWt/4u8AxpgE4FXgFmttTSe3zzDG5BpjcktLO18MQ75WWefib+9/2rb9+IfLOl2wJFBe+PTLdtuvL1/TYZ/t5bWsyd/Ytr2uYDs+XwMFVa6g1SUiEkoFVdI5xpsAACAASURBVC4SY9tfCp4YG6XXPRERCRp/VpV80RiT1LK65FpgvTHmDn86b1l98lXgP9ba1zrbx1o7y1qbba3NTk9P35/aDzt7zml78rqLeeHGS9vNeQu0Pee05fzm5g5z3qB5IZLV2/LYVV3FqAFHMWH4RHzW8s4Xy0iI9ga8JhGRcDAwxUltQ/vFl2obPAxMcYaoIhEROdT5c6nk2JYzZd8G5gFDgO91dydjjAGeAtZZax86qCoFay03P/8m28sq2+a0TRw6gFnXXkxpbR03PBPYVSXfW72hLbT96ZJpxEU7+Ot3z28Lb3uuKrmrppKsoWOIj03FGR3H1LHZeK2Pj/JytaqkiBySpo3LoNrlxtXkxVpLtctNtcvNtHEZoS5NREQOUf4sTrIGmAC8SPPiIh8bY1Zaa4/p5n4nA58Aq4HWRPELa+28fd1Hi5N0bfm2Apo8ng5z2r7cXkh9o5uTRgVurluj28OLS1fwvZMmtZvTtnf7huIytpVVMrhvPx5asIHKejcnHZHG0ZnRWNvE2VmjAlaTiEg4WVdU3e51b9q4DC1MIiIiB21fi5P4E9x+AtwFrATOo/mM2wvW2lMCXaSCW+82JzcfgOnZg0NciYhIz9DrnoiIBNq+glu3H7JlrX0UeHSPpu3GmNMDWZyIiIiIiIjsm1+fjmyMOQ84Gojdo/m+oFQkIiIiIiIi7fizquS/gEuBHwMGmA4E54PDREREREREpAN/VpU80Vp7FVBprf0NMAXQxfwiIiIiIiI9xJ/g1vppovXGmAGAGxgevJJERERERERkT/7McZtrjEkB7ge+ACzwZFCrEhERERERkTb+rCr525ZvXzXGzAVirbXVwS1LREREREREWu0zuBljLu7iNqy1rwWnJBEREREREdlTV2fcvtXFbRZQcBMREREREekB+wxu1tprerIQERERERER6dw+V5U0xtxqjLmuk/YfG2NuCW5ZIiIiIiIi0qqrjwO4Fni+k/ZZLbeJiIiIiIhID+hqjpu11jZ10thojDFBrElERKRXmJ49ONQliIjIYaLLjwMwxmRYa0v2bgtuSdJb6Q2MiIiIiEhwdHWp5P3AO8aYU40xiS1fpwFvAw/0SHUiIiIiIiLS5aqSzxljSoH7gHE0fwTAGuBea+27PVSfiIiIiIjIYa/LSyVbAtphHdLm5OYD/l0GuD/7BnpsERERERE5dHV1qaTsp5xtFeRsq/Br3zm5+W3B7EAc7P1FDkd63oiIBJZeV0V6joKbhERPvtDrj4qIiIiI7Kk3vj9UcBMJY73xRUU6p5+liIiIHIx9BjdjzGRjTFLL905jzG+MMW8bY/5sjEnuuRJFRA6cApOIiIgcCro64/Y0UN/y/SNAMvDnlrZnglyXiIiIiIiItOhqVckIa62n5ftsa+2klu8XG2NWBLkuERERERERadHVGbc8Y8w1Ld+vNMZkAxhjRgHuoFcmIiIiIiIiQNfB7XrgVGPMZmAssNQYswV4ouU2ERERERER6QH7vFTSWlsNXG2MSQRGtOy701pb0lPFiYiIiIiISNdz3ACw1tYCK3ugFhEREREREemEPsdNREREREQkzCm4iYiIiIiIhDkFNxERERERkTCn4CYiIiIiIhLmFNw64fX6/Gr3en1tX/6076vvvfdt/d7XSb+B4u9jDKT9GTMQ9YXiMQaKtbZX1x9u9n4utdKxFBEROXz09vdWQQtuxpinjTG7jDF5wRojGIqLq7juB0+Rk7u1y/a33v6Sm3/6PN/93mNcdc0s6uoaAShdu4Obf/Ic373qX1x1zeNt7QDPPb+YX/7qFZqaPG1tucu38n+X/I3b7phNU5OHxUs2MOPGZ/ho4TpefuQdaip2A7Dk0w384ManKSurPejH+OWK7Vz7gycpKKxs175iZeftgbB7dwM/vuV53pm3skP7T255gbnzVrS1bd1WytXXPsG6dYXt9m1tX7u2oNvxPl26kR/c+DSlex2vpZ9t4vobOraHm38/v5h7fv0qXo+3Q/vev0PStV07y3np4bls3rKrXfv69UVcfe0THdpFRETk0LNl6y6uvvYJvlpf1K5969ZSrrnuCdZ9VbiPe4aPbj8O4CA8C/wdeC6IYwSc0xlNlCOCX937KjN+Oo0FRS7KSmt59rM8cHtISowFoE+fODZuLMbhiKSx0cOV1z5BcUoKzi07IcKAzzJqVH8cjsi2vvv2TeTznC3cevd/cR09gupGH4lb8qmucbFhYzFlZTUkJzvZWVDBb3//Jo6kON5ZW8KGTSUsf+tzRh2ZQazTwbqiahasKaay3s3OShfTxmUwJjPZ78eYlBhLdVU9t90+mwcfuJyBA/qwYuV27v7lHPr3TyHOGR3w4xodHUVSopMHH36Xoup6ljVFUV7lYvaj86jdVc0Vl5/Qtm98XAw+a7nz5y/zlz9dypgxA9i6rZTb75iNiTAktPwMunyMSU527arh5lteIO6kcdQSyfKcLSx/cxlHjOxHbIwj4I8xkNL6JvLc50tYvbOKmOPGsLPShXfjDua9nsvZ38wiMlIny/2xrqiaz/Orqalr5KafvkDSyeMgKYG8tYXkvrqE5CQn8XGB/33vrI6Dec6KiIQjvbZJbxIfH4PP5+OOu17iR3ecz4JNVZQWVfHU0jycMVEkJHT//jLUgvbuz1q7CKgIVv/Bkpwcx4N/uZz0jGT+8dA8ytblU7d4Fe4GN4POOhZfUhwAp5x8FPf+6tu43V6iHJFUV+zGuWUnPtMc2nyJcZxz3RlER3+djc879xiuuPZU1q7aQeFHq/Ct3872FVtJHJyOiTDcdudL7NhRgc9aoPm0bdSuSnLfXEZsaiLX33IO+TWNzFq0FVeTlxRnFNUuN7MWbWVdUbXfj3HkyAwevP9yGhqauO322cx7d2VbaHvo/svp0yc+sAeV5uB238yLGTt+MC8+/TGlq7fhWppHdUkVg884htThGW379uuXxMMPXEFSspM7f/4y785f1RbaHnrgCoYM7tvteOOOHsRNt51HecVudi74goidJeS88RnRfRK4/mfnkuhH+AulkZOGM+jko6kvLKchZx0rPlzFvNdzOeGUo7j91nMU3PywrqiaWYu24o2NIWXqMXgslH+8Cvf2Ipb+dzHeiEh+eMf59O+f0iN1HMxzVkQk3Oi1TXqbjH7JPPTgFcTGRfPAH96kfP1Odi9eBcaQeeYkdkdGdt9JiOndXyeSk+M4+sITiIiMoHr5Bjx1DUy55GQyh6QxP6+kbb/W8OZxf305W4S1RCbFEXPSeP6T0/GSvtq0vhx5xngaiyuoWrOdAaMHMeGCyWT/30mUlNTw4MPvYiIiOPr84/DVNVCWs56Y+FgmXDSFj7dUMT+vhGSnA2d0JMYYkp0Okp2OdnX5ozW87Sqt4YGH3sVaghbaWkVHRzHszEkk9e9DzcrNNJbXkH3hZIaMHtSh/tbwVlffyP0PzqOyqt7v0NZqQ6Mh68IT8O52Ufb5eqKdMUy4aAqLtlYF+qEF3Py8EkZMHEH65NG4CsvJz91E+qiBpE0eq9Dmpz2fK5UmgviTszBuD7XLN+JpdDPhOyeSU+Lq0ToO5jkrIhJO9NomvVFGv2QmfOckfE0eqnLW42t0c9Llp9Cvf0qv+N0N+TtAY8wMY0yuMSa3tLQ01OW0KSjfDfbr7aaGJhJjoyioav9Gr6qqvsN9rcdLgsNQUtPQsd8qF6bR3bbtbmgiPjqCorLdbW0+n223j/X5cEYaCqpcFFS5SIxtf4VrZ3X5o3b31/VZa6l3Ne13H/trZ3kd1vt10G2sb9xn/XX1je23dzd22KcrBVUuHL6vx7I+H3EtxzHctf6cPa49HrPbw87KutAV1cvs+VxxuX1E2fYTj2OwPfK7EMjnrIhIuNBrm/RWhXu85wbwNLp7ze9uyIObtXaWtTbbWpudnp4e6nKA5oVItszLwUQYMs+YQHRKAjmvL2X7hkIGpjjb9nt77pc8/Mh7zXPaaM55PmPw1TdS+cEXpDs7TiGsW7uNDUvWkTCiP2nHH0Xptl0s/s8iNr+Xy4jh6fz5D5dgDOS9v4LolAT6n3YMniYPn85eRN8oGJjipLah/cIUtQ2ednX5o3VO29ChafzlT5fijHVw2+2zg7IwSavduxvYsWA5u8tryThlHM4BfVn13pdsWL65Q/2tc9pSU+O5/8+XkZmZwp0/f7nDgiVdiSqrYvkby4hJTaT/6cfg8/pYMvsTUsP/TDgDU5zkfbyGylVbSRyRSdZZEyjdWkLhRyu1MImf9nyuOOvrqf5kFTY2Gps9mtiEWD57eTGJTcH/x4pAPWdFRMKJXtukN9q6tZTN83KIjo8l8xsTiUqIZel/l1C4vaxX/O6GPLiFm+KSam69Yza4PQyflo1JSyHzGxNwJiewZm4OQ2l+ozf3nRU8/Mh7zXPYWua01Y4ahrEWIiLA1Yj3k5XtVpV8/j9LWL94HX2OHEDipKNIOnIgAyeMYPeuKqIdUfzx99Npcnualy43Bk+jG0dyHOMumEzj7gZWvvYpUwYlUO1y42ryYq2l2uWm2uVm2riMfT2kDvZciOSh+y8n+9jh7ea8FRYF/lLC3XUN3Hn3y9TuqmboNyYQmZlGxtRx9BmSzsYPV5FYVt6277btZe3mtB07aVi7OW/+rPqz9LNNLG+ZG9hn6njiBvTl6Asm01TffBzDfVVJ36Z8tn++Aeew/qRNHk2f0YMZdPLR7NpSwr2/eV3hzQ/TxmU0Pz+KqzDL10NUFPXHjKLf0HSyLpoCEYblry5hy9bgrirZWsfBPGdFRMKNXtukt9m6rZTb7pyNMyaKEeceh+mbzIAzJxEV42DlG0sZE2dCXWK3gvlxALOBpcBRxpidxpjrgjVWIMXGOOiXnsRDf7mcW74zAWd0JLU2gpMuP4UjjuzP2CHNc6ySkmKZcsJIhg7py5jRmfzsnm9DZl+qjxqGo18KGQNTyUxPbLeqZEpyHGefNY77fvkt4mKjqHJ5GJE1lPSMZE44fiSpqQkkJMQyYfwQbrn7W0QnOql1W4aM7M9P7/oWA/unMG5wKjOmDscZHUmVy0Oy08GMqcP3axWn+PgYRo3q325OW+uct4z+ScTGBn7FxWhHFH1S4pn5629zx/dOwBkdSU2TZcr/nUjWxGGMHvL13DWn08HAgant5rS1znkbPjyd+LiYbsdLTIglK2sQ9/32/4hPiKXK5WHIiAx+9vMLGJiZHParSh4xqA8nnDKazBPHUt3oJdnp4O4ZU7n1lmmkJDs1z80PYzKTm58rcdFEJsUx8eIpDB+Uitca+mf24RczL2bYkLSgrKLaaR0H8ZwVEQk3em2T3sYZ62DQwFQefei7/Phb43BGR7I7IoqTrpjKsKFpjB2SGuoSu2Wstd3v1UOys7Ntbm5uqMvAWosxzal7Tm4+ANOzB7drb92vlTGGO19p/oyyP39nfLv2zvreu989923d5785OzDGMD17cJd1Hexj9Kc9EPw9roGqr7Njvb99hJK1lleW7wTa/5x7S/3hYk5uPtZaLjluSEh/Fw72OSsiEo702ia9yf68Fw0lY8xya2323u3B/By3XmtfP7i92/3dr7vb9tWvv+MdiAOpvSfHDER9oXiMgdTb6w8nOpYiIiLS298P6HorERERERGRMKfgJiIiIiIiEuYU3ERERERERMKcgpuIiIiIiEiYU3ATEREREREJcwpuIiIiIiIiYU7BTUREREREJMwpuImIiIiIiIQ5BTcREREREZEwp+AmIiIiIiIS5qJCXYCISDBNzx4c6hJEREREDpqCm0gYU+g4dOhnKSIiIgdDwa2X6u1vAnuy/t5+rEREREQksHrj+0MFtwA6bliq3/v2xl8Wkd5OzzsRkcDS66pIzzHW2lDX0CY7O9vm5uaGugwREREREZGQMMYst9Zm792uVSXD2JzcfObk5nf4XprpmIiIiIjI4ULBLQACFSC66idnWwU52yr8vq+/Nc3JzefOV1Z2uW+oA1KoxxcRERERCTUFN2nTEwG0tzuUH5uIiIiIhC8FNxERERERkTCn4CYiIiIiIhLmFNxERERERETCnIKbiIiIiIhImFNwExERERERCXMKbiIiIiIiImFOwU1ERERERCTMKbiJiIiIiIiEOQU3ERERERGRMKfgJiIiIiIiEuYU3ERERERERMKcgpuIiIiIiEiYU3ATEREREREJc0ENbsaYacaY9caYTcaYnwdzrENF4fJNvP2vBQBsW5TH3B/+E5/Px+612yldsByAZfO+4PYzZvLczP+Sv76g7b5FX2zmrcfe44XfvkLl1pK29rzF63jrn++1G2f7up08f98cyjcXUfzKJ1hrAajavovnZv6X7WvzeeG3r7S1d8Vay39+9yrb1uR3aH/xD6+xNW/HgR2MTnz1+UZefXhuh/b1uZuZ8+DbARtnT68+PJevPt/Yof21R95h7WcbAjqW1+PlibteoKywon27t6W9oDyg44mIiIhI7xAVrI6NMZHAP4CzgJ1AjjHmLWvt2mCNGQrriqpZsKaYyno3OytdTBuXwZjM5APup+Cp/+HOXc/qHRV8+upnNG4s4PxBN9JUVoMnJZELI6Kof2IuBtiyajvvzHqfm16+nQVbatj55Ad4ctcDkHzBFDj7BFZ9so4vfvUcGUPSOPua04hxxgDw0ezF/Od3rxI5rD/ebcU8WlAJ2WNwPv02Diyv/2M+jqgIjrzgeBasKeny8VWV1vDWY+/xxt/f5aaXbmXB9joq6pr434Nvsvnlj6mvqef6P10ZkONb/fwClrzwMQVltawedQSV9W7ylm3ky18+S3LfRM79wTeIT4o7oLE6U1/r4s1/zOf5++Zw439uYUGph8p6N4tmLeCrJ+dz7vXfYOwJowI23s4Nhbz92Ht8+ubn3P/hTNIGpOL1evnL9//Ohy8upv/wfnzrxm8GbDwRERER6R2CecbteGCTtXaLtbYJeAm4MIjj9bh1RdXMWrQVV5OXFGcU1S43sxZtZV1R9QH3kznjPBImHclHf3oVb9YI6JuEu7gSay31J2dR/8TbWCznP3wdDy+6D7fXx58v/CO1O3bh7JPQ1me9w0FsfjGf//JZTEoCM178WVtoAzj+xmkkfGMS3m3FNPXrg3PJKmL++So+jwefhQaPj2/964e89FVFt4+vT79kHvxoJj4Df/n2X6jdVgLvLmXzyx+TeuYkpvz4vIAd36ppJ3LUOcfy9h9fo+ytT3EWl7HsrqfxxMZw40u3BTS0AcQlOnngo5nEpsTz8CUPUrN+J2bhF3z15HySp4zlrF9dEtDxho4dzB/n30N5YSV3nDGTXfllbaHtuj9codAmIiIicpgKZnAbCOx57dzOlrZDxvy8EpKdDpzRkRhjSHY6SHY6mJ9X0v2d99FPhCMKxzXnwtHD8bz2CZTXYCMjiPD6SH7jEwAirjmPxTFJDB07mAm/vwZjofzeZ6l9fzkxpx4DRw/H/eoiSv40G2d6CuN/dzVLSxvbjfneml14LzgZz5QsondVNvfr9hDhasIXYcj6/dW8Xdzo9+MbNGoAE/5wLRGRhvLf/Juad5Yx5LzjGf3D81mwtjRgxzclIYaS804hbWoWu19fTPFvnyc6KY7xv7uaz6s8BzROd/oNTmP8764mOjGOij+9SNWcj8k8bTxH3/4d3v+qLODjHX3iUfxx/j2U5pfz3aE3tYW2y35+UcDHEhEREZHeIZjBzXTS1mHClDFmhjEm1xiTW1p6YG/wQ6WgykVibPurTRNjoyioch1UP7UeH/FnTGzb9owe1va9iYslaeIRlNQ0AFCdlEDKqK/zsGfKOOK/Maltu//JR9N3QJ8ONRVUuWjy+uDEcR0LGpxBv5H9Kalp2K/HVxUfR+rRQ9u2h15wAklx0ft9PPassbPxd9U1ccRFJ7a1pWcfSfqQtAMexx8V0TFknDC6bXvYhVNIio8J2pijJx9Bn/4pbdtnXnVqUMYRERERkd4hmMFtJzB4j+1BQOHeO1lrZ1lrs6212enp6UEsJ/AGpjipbWh/lqe2wcPAFOdB9ePML6Hu8bexqUm4U5NwrNmCBbyx0di6Bqr//CL9EmKw1lI752NKczYQO3YoEUlx+P72KnWPvYlNTSJm5AA2z17Ihtc+7VDTwBQnjvIaIme9idcZjY1s/lXwJsVj12wl7/F36ZcY4/fjs9ZS9/piShavIXbMECKT41l211OUbCjc7+Oxr+PSOn7fyipyfvkskamJxBw5iB1zP2fdfz464HH80fh+LtvfXErMkQOJ7JtEzj3/pnDVtqCM2TqnrXjrLqZckI0zIZY7zpjZYcESERERETl8BDO45QBHGmOGG2OigcuAt4I4Xo+bNi6DapcbV5MXay3VLjfVLjfTxmUccD+ur3aw+x+v40uOJ+a8yTgqa7A0n76sP3UiZPaFnaVE/30Oj9/+HFtfXUzq2dkk/eQ79LnkNGxdA9brJfbKs8i46zJSJ49my6x5xOWuaTfmxFhL099fxXq8GAy+2Bjqjh5BZE0dMUP6UfDGp4xctpKq+qZuH5+1lqd/ObttTlvST/6PjLsuwxrDqnv+zTGOA7uEsbPjW5y3nca/v0ZEgpM+t19Gxp2XkjY1i+3P/4/IhcsPaJzu/Pf+N9vmtCX/7BL63305UQlO8mY+zxh3XUDH2nMhkuv+cAX3vXFXuzlvCm8iIiIih6egBTdrrQe4GXgPWAf811q7put79S5jMpOZMXU4zuhIqlwekp0OZkwdvt+rSu7ZT8UHXxLXL4UbZt+KWdp8uDJv/jaNRw3BmfsVSXdeRtqRAyj9qoD3nv2Ib998Dvc+dQNxMVHUrNiCs38f4pLjid1aSLXbcvyvLmf8tIkse24hja6v57ltW/Al8Y4InJNGYSMiqL3mfMzlZxFxyjH4qnZz/PQT2TJ/OZeP6dvt46sqrWHBsx9x/g1n8etnbiQuNoq65CSmPPAD4p1RbH73i4Ad34x1m0lNT+KXb99NQmYfqpt8ZP98OsdeNJnPX/yEupr6AxprX+prXbz9rwWcdtlJzHzpp8Q5Hex2xjHlgevpk57E2teXBXS8nesL+fTNnHZz2lrnvJUXVrL0zZyAjiciIiIivYPx53O6ekp2drbNzc0NdRn7bU5u8xos07MHd7Nn9/14mzxMOzKVlPRkXvpsG5Wbirjpyinc/tIXeOsaePi6E/F4PGxdtYM+Gcn0HZCKMab5vm4PZ49Mxev28GFhHcYYpmcPxt3kZndVPX36fR24rLWUF1bwYcFulq7I58RJQ9vazxgQT98BqZQXVpA2sK9fj6+ssILU/ilERES023/P9oM5Lq39eT1eqstqSO3fZ5/tgVZRXElyWhKRUZHtxqwsqSKpbyKRUZEBHa+soJy0gX39bhcRERGRQ4cxZrm1Nnvv9qB9jpscmMjoKFLSmwNWZFQkaaMHARARFUlEcjwAUVFRHDlpRMf7OqLawpkp+vrMkyPa0S60ARhjSBvYF1NUjyM1sUM7sF8hIW1A6n61H6jIqMhOw9m+2gNhX/32yUjptP1g7eu4K7SJiIiIHL6COcdNREREREREAkDBTUREREREJMwpuImIiIiIiIS5/2/v3mPlKOswjn+ftGC5FBXFiLRS/wCVVEBz0CKgBBCLNBjRBjESEo0a4wWIRkX+MtGkSb2LoggNCgTiDTUYrVUxSBVpgdIUWwyRQBsb6yUGkABCf/6xA6nNKZy2ZzrT2e8nOcmZ3dmd5+ybnrPPzjtvLW6SJEmS1HMWN0mSJEnqOYubJEmSJPWcxU2SJEmSes7iJkmSJEk9Z3GTJEmSpJ6zuEmSJElSz1ncJEmSJKnnLG6SJEmS1HMzuw6g4Vk8MbfrCK0Z8s8mSZKk/rK4TYM98Wb+uHkH79T+U820eGJu78tI3/NJkiRJbUtVdZ3haRMTE7V69equY/TS91dvBCwxkiRJ0pAlub2qJra/3TNuewkLmyRJkjS+XJxEkiRJknrO4iZJkiRJPWdxkyRJkqSes7hJkiRJUs9Z3CRJkiSp5yxukiRJktRzFjdJkiRJ6jmLmyRJkiT1nMVNkiRJknrO4iZJkiRJPWdxkyRJkqSeS1V1neFpSf4O3L+TD3sh8I8W4qh/HOvx4ViPF8d7fDjW48OxHh+O9fQ7vKoO2f7GXhW3XZFkdVVNdJ1D7XOsx4djPV4c7/HhWI8Px3p8ONZ7jlMlJUmSJKnnLG6SJEmS1HNDKG6Xdx1Ae4xjPT4c6/HieI8Px3p8ONbjw7HeQ/b6a9wkSZIkaeiGcMZNkiRJkgZtEMUtydIkG5KsTXJDkud1nUntSLI4yd1JtiZxBaMBSrIwyT1J7k3yqa7zqD1JliXZkmRd11nUniRzk9yUZH3z+/uCrjOpPUlmJbktyV3NeH+m60xqV5IZSe5McmPXWYZuEMUNWAHMr6qjgT8DF3ecR+1ZB5wN3Nx1EE2/JDOArwNnAEcB5yY5qttUatFVwMKuQ6h1TwAfq6pXAguAD/nvetAeA06pqmOAY4GFSRZ0nEntugBY33WIcTCI4lZVv6yqJ5rNW4E5XeZRe6pqfVXd03UOtea1wL1V9Zeqehy4Hnhrx5nUkqq6GfhX1znUrqraXFV3NN8/xOgN3mHdplJbauThZnOf5ssFFQYqyRzgTOCKrrOMg0EUt+28B/h51yEk7ZLDgI3bbG/CN3jSYCSZB7wa+GO3SdSmZurcGmALsKKqHO/h+jLwCWBr10HGwcyuA0xVkl8BL57krkuq6ifNPpcwmpJx7Z7Mpuk1lbHWYGWS2/ykVhqAJAcCPwQurKoHu86j9lTVk8CxzZoDNySZX1VeyzowSRYBW6rq9iQnd51nHOw1xa2qTnum+5OcDywCTi3/j4O92rONtQZtEzB3m+05wF87yiJpmiTZh1Fpu7aqftR1Hu0ZVfXvJL9ldC2rxW14TgDOSvIWYBZwUJJrqurd2iBazwAAA89JREFUHecarEFMlUyyEPgkcFZVPdJ1Hkm7bBVwRJKXJdkXeCfw044zSdoNSQJcCayvqi92nUftSnLIU6t7J9kPOA3Y0G0qtaGqLq6qOVU1j9Hf699Y2to1iOIGXArMBlYkWZPkm10HUjuSvC3JJuB44GdJlnedSdOnWWTow8ByRgsYfK+q7u42ldqS5DrgD8DLk2xK8t6uM6kVJwDnAac0f6PXNJ/Qa5gOBW5KspbRh3Erqspl4qVpEGcVSpIkSVK/DeWMmyRJkiQNlsVNkiRJknrO4iZJkiRJPWdxkyRJkqSes7hJkiRJUs9Z3CRJkiSp5yxukqReSzI3yX1JDm62n99sHz7JvvOSvGs3jvXp3ckqSVJbLG6SpF6rqo3AZcCS5qYlwOVVdf8ku88Ddrm4ARY3SVIvWdwkSXuDLwELklwInAh8YQf7LQFOSrImyUVJZiRZmmRVkrVJPgCQ5NAkNzf7rUtyUpIlwH7NbddO9uTNGb0NSb7TPN8Pkuzf3Hdckt8nuSvJbUlmN/v/Lskdzdfrp/+lkSSNg1RV1xkkSXpWSd4M/AI4vapW7GCfk4GPV9WiZvv9wIuq6rNJngOsBBYDZwOzqupzSWYA+1fVQ0kerqoDnyHDPOA+4MSqWplkGfAn4KvABuCcqlqV5CDgEWBfYGtVPZrkCOC6qprY/VdDkjRuZnYdQJKkKToD2AzMByYtbpM4HTg6yTua7ecCRwCrgGVJ9gF+XFVrdiLHxqpa2Xx/DfBRYDmwuapWAVTVgwBJDgAuTXIs8CRw5E4cR5Kkp1ncJEm91xSfNwELgFuSXF9Vm6fyUOAjVbV8kud8A3AmcHWSpVX13SnG2X6qSjXHmWwKy0XA34BjGF2e8OgUjyFJ0v/xGjdJUq8lCaPFSS6sqgeApcDnd7D7Q8DsbbaXAx9szqyR5MgkBzQrUm6pqm8DVwKvafb/71P7PoOXJjm++f5c4BZG0yRfkuS45jizk8xkdIZvc1VtBc4DZkz5B5ckaRsWN0lS370PeGCb69q+AbwiyRsn2Xct8ESzQMhFwBWMrkG7I8k64FuMZpucDKxJcifwduArzeMvB9buaHGSxnrg/CRrgYOBy6rqceAc4GtJ7mI0lXNWk/X8JLcymib5n116BSRJY8/FSSRJmqJmcZIbq2p+x1EkSWPGM26SJEmS1HOecZMk7XWSvAq4erubH6uq103T878A+PUkd51aVf+cjmNIkrQzLG6SJEmS1HNOlZQkSZKknrO4SZIkSVLPWdwkSZIkqecsbpIkSZLUcxY3SZIkSeq5/wE9ZhMOvIGaJQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.figure(figsize=(15, 5))\n", + "plt.scatter(X_test_pca, y_test,\n", + " marker='x', s=50, c=y_test, zorder=10)\n", + "\n", + "plt.errorbar(X_test_pca, y_preds, yerr=y_sd, fmt='o', capthick=1, label='Predidtion', alpha=0.5)\n", + "plt.legend(loc='upper right')\n", + "plt.ylabel('6 Classes of Glasses')\n", + "plt.xlabel('X_test_pca')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "JupyterPy2", + "language": "python", + "name": "ipykernel_py2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}