From e7f8faa0716f7f75d4d98a589fda686579161344 Mon Sep 17 00:00:00 2001 From: Luyao Peng Date: Sun, 8 Nov 2020 10:17:01 -0700 Subject: [PATCH] Adding an example notebook of variational GP Demonstrating how variational GP is used in multiclass-classification --- ...an_Process_Multiclass_Classification.ipynb | 504 ++++++++++++++++++ 1 file changed, 504 insertions(+) create mode 100644 tensorflow_probability/examples/jupyter_notebooks/Variational_Gaussian_Process_Multiclass_Classification.ipynb diff --git a/tensorflow_probability/examples/jupyter_notebooks/Variational_Gaussian_Process_Multiclass_Classification.ipynb b/tensorflow_probability/examples/jupyter_notebooks/Variational_Gaussian_Process_Multiclass_Classification.ipynb new file mode 100644 index 0000000000..fa7c81e649 --- /dev/null +++ b/tensorflow_probability/examples/jupyter_notebooks/Variational_Gaussian_Process_Multiclass_Classification.ipynb @@ -0,0 +1,504 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Sparse Variational Gaussian Process in Multiclass Classification\n", + "\n", + "In this notebook, sparse variational gaussian process model (VGP) is applied to a multiclass classification problem. VGP is easily scalable to large scale dataset." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Background\n", + "\n", + "$\\qquad$ Consider making inference about a stochastic function $f$ given a likelihood $p(y|f)$ and $N$ observations $y=\\{y_1, y_2, \\dots, y_N\\}^T$ at observation index points $X=\\{x_1, x_2, \\dots, x_N\\}^T$. Place a GP prior on $f$: $p(f) \\sim N(f|m(X), K(X, X))$. The joint distribution of data and latent stochastic function is \n", + "\n", + "$$p(y, f) = \\prod_{i=1}^{N}p(y_i|f_i)N(f|m(X), K(X, X)) \\tag{1}$$ \n", + "\n", + "$\\qquad$ The main interest is the posterior over the function values given the observations $p(f|y)$. The posterior is intractable when the likelihood $p(y|f)$ is non-Gaussian, which is often the case in classification problems; and the computational complexity is $O(N^3)$ due to the inversion of $K_{X, X}$, which is also intractable for large dataset.\n", + "\n", + "$\\qquad$ To reduce the computational complexity, $M << N$ inducing index points $Z=\\{z_1, z_2, \\dots, z_M\\}^T$ and inducing variables $u=f(Z)$ are introduced. Assuming a GP prior on the joint density $p(f, u)$, $$p(f, u) = N\\begin{pmatrix} \\begin{bmatrix} f \\\\ u\\end{bmatrix}| \\begin{bmatrix} m(X) \\\\ m(Z)\\end{bmatrix}, \\begin{bmatrix} K(X, X) & K(X, Z) \\\\ K(Z, X) & K(Z, Z)\\end{bmatrix}\\end{pmatrix},$$ and a GP prior on $u$ $$p(u) = N(u|m(Z), K(Z, Z)),$$ the conditional of $f$ is $p(f|u) = N(f|\\mu, \\Sigma)$, where for $i, j = 1, \\dots, N$\n", + "\n", + "$$[\\mu]_i = m(x_i) + \\alpha(x_i)^T(u-m(Z)), \\tag{2}$$\n", + "\n", + "$$[\\Sigma]_{ij} = K(x_i, x_j) - \\alpha(x_i)^T K(Z, Z)\\alpha(x_j), \\tag{3}$$\n", + " where $\\alpha(x_i) = K(Z, Z)^{-1}K(Z, x_i)$, and the joint density of $y, f, u$ becomes\n", + "\n", + "$$p(y, f, u) = p(f|u; X, Z)p(u; Z)\\prod_{i=1}^{N}p(y_i|f_i)$$\n", + "\n", + "$\\qquad$ The goal is still finding the posterior of the function values $f$, however, the likelihood $p(y_i|f_i)$ is not Gaussian, so no closed-form solution for the posterior of $f$. Therefore, a variational posterior is used to solve the difficulty. \n", + "\n", + "$\\qquad$ Replacing the posterior $p(u|y)$ by an arbitrary full-rank Gaussian distribution $q(u)$ [Hensman et al. (2013)], then the variational posterior for $y$ and $u$ jointly becomes \n", + "\n", + "$$q(f, u; X, Z) = p(f|u; X, Z)q(u), \\tag{4}$$\n", + "\n", + "$$\\mbox{where } q(u) \\sim N(\\mathbf{m}, \\mathbf{S})$$\n", + "\n", + "$\\mathbf{m}, \\mathbf{S}$ are parameters to be chosen by optimizing an evidence lower bound (ELBO). \n", + "\n", + "$\\qquad$ Since both $p(f|u; X, Z)q(u)$ are Gaussian, the marginal variational posterior of $f$ can be computed analytically\n", + "\n", + "$$q(f|\\mathbf{m}, \\mathbf{S}; X, Z) = \\int p(f|u; X, Z)q(u) du \\sim N(\\tilde{\\mu}, \\tilde{\\Sigma}) \\tag{5}$$\n", + "\n", + "with $[\\tilde{\\mu}]_i = \\mu_{\\mathbf{m}, Z}(x_i), [\\tilde{\\Sigma}]_{ij} = \\Sigma_{\\mathbf{S}, Z}(x_i, x_j)$, and\n", + "\n", + "$$\\mu_{m, Z}(x_i) = m(x_i)+\\alpha(x_i)^T(\\mathbf{m}-m(Z)) \\tag{6}$$\n", + "$$\\Sigma_{S, Z}(x_i, x_j) = K(x_i, x_j) - \\alpha(x_i)^T[K(Z, Z) - \\mathbf{S}]\\alpha(x_j) \\tag{7}$$\n", + "\n", + "$\\qquad$ The variation parameters $Z, \\mathbf{m}, \\mathbf{S}$ in $q(f|\\mathbf{m}, \\mathbf{S}; X, Z)$ are determined by maximizing the lower bound \n", + "\n", + "$$L = \\sum_{i=1}^N \\mathbb{E}_{q(f_i|\\mathbf{m}, \\mathbf{S}; X, Z)}[logp(y_i|f_i)] - KL[q(u)|| p(u)], \\tag{8}$$\n", + "\n", + "where the expected log-likelihood can be computed with Gauss–Hermite quadrature.\n", + "\n", + "$\\qquad$ The variational posterior is given as $q(f)$ in (5). To make predictions for a set of test index points $X^*$, the new latent function values $f^*$ is approximated by\n", + "\n", + "\\begin{equation}\\begin{array}{rcl}p(f^*|y) &=& \\int p(f^*|f, u)p(f, u|y) df du\\\\ &\\approx& \\int p(f^*|f, u)p(f|u)q(u)df du \\\\ &=& \\int p(f^*|u)q(u) du \\\\ &=& q(f^*)\\end{array}\\end{equation}\n", + "\n", + "where the last line is following (5), (6) and (7) by replacing $x_i$ by $x_i^*$.\n", + "\n", + "$\\qquad$ With the variational posterior in (5), the predictive mean and variance of $y^*$ are computed as \n", + "\n", + "$$\\hat{y}^* = \\mathbb{E}(y^*) = \\int\\int y^* p(y^*|f^*)q(f^*) df^* dy^* \\tag{9}$$\n", + "$$\\hat{\\mathbb{V}}(y^*) = \\int\\int y^{*2} p(y^*|f^*)q(f^*) df^* dy^* \\tag{10}$$\n", + "\n", + "## References\n", + "\n", + "[1]: Titsias, M. \"Variational Model Selection for Sparse Gaussian Process Regression\", 2009. http://proceedings.mlr.press/v5/titsias09a/titsias09a.pdf \n", + "\n", + "[2]: Hensman, J., Lawrence, N. \"Gaussian Processes for Big Data\", 2013. https://arxiv.org/abs/1309.6835\n", + "\n", + "[3]: Salimbeni, H. and Deisenroth, M. \"Doubly stochastic variational inference for deep Gaussian processes.\" Advances in Neural Information Processing Systems. 2017. https://arxiv.org/pdf/1705.08933.pdf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Imports" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import tensorflow.compat.v2 as tf\n", + "import tensorflow_probability as tfp\n", + "import pandas as pd\n", + "from sklearn.preprocessing import LabelEncoder\n", + "from sklearn.model_selection import train_test_split\n", + "from scipy.cluster.vq import kmeans2\n", + "\n", + "tf.enable_v2_behavior()\n", + "\n", + "tfb = tfp.bijectors\n", + "tfd = tfp.distributions\n", + "tfk = tfp.math.psd_kernels\n", + "\n", + "dtype = np.float64" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load Glass Data\n", + "\n", + "A standard imbalanced machine learning dataset referred to as the “Glass Identification” dataset, or simply “glass”.\n", + "\n", + "The dataset describes the chemical properties of glass and involves classifying samples of glass using their chemical properties as one of six classes. The dataset was credited to Vina Spiehler in 1987." + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "--2020-11-08 09:08:57-- https://raw.githubusercontent.com/jbrownlee/Datasets/master/glass.csv\n", + "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.192.133, ...\n", + "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", + "HTTP request sent, awaiting response... 200 OK\n", + "Length: 11154 (11K) [text/plain]\n", + "Saving to: 'glass.csv.1'\n", + "\n", + "100%[======================================>] 11,154 --.-K/s in 0.001s \n", + "\n", + "2020-11-08 09:08:57 (11.2 MB/s) - 'glass.csv.1' saved [11154/11154]\n", + "\n" + ] + } + ], + "source": [ + "!wget https://raw.githubusercontent.com/jbrownlee/Datasets/master/glass.csv" + ] + }, + { + "cell_type": "code", + "execution_count": 102, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(171, 9) (43, 9) (214,)\n" + ] + } + ], + "source": [ + "data = pd.read_csv('glass.csv', header=None)\n", + "data = data.values\n", + "X = data[:,0:9].astype(dtype)\n", + "Y = data[:,9]\n", + "\n", + "encoder = LabelEncoder()\n", + "encoder.fit(Y)\n", + "encoded_Y = encoder.transform(Y)\n", + "encoded_Y = encoded_Y.astype(dtype)\n", + "num_outputs = 6\n", + "\n", + "X_train, X_test, y_train, y_test = train_test_split(X, encoded_Y, test_size=0.2, random_state=42)\n", + "print(X_train.shape, X_test.shape, encoded_Y.shape)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Defining Trainable Variables in VGP\n", + "\n", + "* Using kmeans to initialize 30 representative `inducing_index_points` $Z$ and make them learnable variable" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "metadata": {}, + "outputs": [], + "source": [ + "num_inducing_points_ = 30\n", + "inducing_index_points_init = kmeans2(X_train, num_inducing_points_, minit=\"points\")[0] #50, 60\n", + "inducing_index_points = tf.Variable(inducing_index_points_init, dtype=dtype, name='inducing_index_points')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* Initializing RBF kernel and kernel parameters, which are `amplitude` and `length_scale` (the same length scale is used for all $X$ columns)\n", + "* Initializing the variational mean and covariance $\\mathbf{m}, \\mathbf{S}$ in $q(u)$" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": {}, + "outputs": [], + "source": [ + "amplitude = tfp.util.TransformedVariable(\n", + " 1., tfb.Softplus(), dtype=dtype, name='amplitude')\n", + "length_scale = tfp.util.TransformedVariable(\n", + " 1., tfb.Softplus(), dtype=dtype, name='length_scale')\n", + "kernel = tfk.ExponentiatedQuadratic(amplitude=amplitude, length_scale=length_scale)\n", + "\n", + "observation_noise_variance = tfp.util.TransformedVariable(1., tfb.Softplus(), dtype=dtype, name='observation_noise_variance')\n", + "\n", + "variational_inducing_observations_loc = tf.Variable(np.zeros([num_outputs, num_inducing_points_], dtype=dtype), name='variational_inducing_observations_loc')\n", + "\n", + "Ku = kernel.matrix(inducing_index_points, inducing_index_points)\n", + "variational_inducing_observations_scale_init = np.linalg.cholesky(Ku + np.eye(num_inducing_points_)*1e-6)\n", + "variational_inducing_observations_scale = tf.Variable(np.tile(variational_inducing_observations_scale_init[None, :, :], [num_outputs, 1, 1]), \n", + " name='variational_inducing_observations_scale')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* Defining log probability. For multiclass classification, Categorical distribution is used. The `observations` is a flat array of batch size; since the expected log likelihood in VGP is approximated by Gauss–Hermite quadrature, the input logits is reshaped to (`quadrature_size, batch_size, num_outputs`) to adapt to the `sparse_softmax_cross_entropy_with_logits` in `log_prob`\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "metadata": {}, + "outputs": [], + "source": [ + "def log_prob(observations, f):\n", + " #f is (6, 20, 64)\n", + " berns = tfd.Independent(tfd.Categorical(logits=tf.transpose(f, perm=[1,2,0])), 1) #(20, 64, 6), n_quadrature, bs, n_outputs\n", + " return berns.log_prob(observations) #sparse_softmax_cross_entropy_with_logits: have logits of shape [batch_size, num_classes] and have labels of shape [batch_size]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Constructing Model and Training" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "metadata": {}, + "outputs": [], + "source": [ + "vgp = tfd.VariationalGaussianProcess(\n", + " kernel,\n", + " index_points=X_test,\n", + " inducing_index_points=inducing_index_points,\n", + " variational_inducing_observations_loc=variational_inducing_observations_loc, #TensorShape([6, 30])\n", + " variational_inducing_observations_scale=variational_inducing_observations_scale, #TensorShape([6, 30, 30])\n", + " observation_noise_variance=observation_noise_variance)" + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "metadata": {}, + "outputs": [], + "source": [ + "batch_size = 64\n", + "\n", + "optimizer = tf.optimizers.Adam(learning_rate=.01)\n", + "\n", + "@tf.function\n", + "def optimize(x_train_batch, y_train_batch):\n", + " with tf.GradientTape() as tape:\n", + " # Create the loss function we want to optimize.\n", + " recon = vgp.surrogate_posterior_expected_log_likelihood(\n", + " observations=y_train_batch,\n", + " observation_index_points=x_train_batch,\n", + " log_likelihood_fn=log_prob,\n", + " quadrature_size=20)\n", + "\n", + " elbo = -tf.reduce_sum(recon) + tf.reduce_sum(vgp.surrogate_posterior_kl_divergence_prior())\n", + "\n", + " grads = tape.gradient(elbo, vgp.trainable_variables)\n", + " optimizer.apply_gradients(zip(grads, vgp.trainable_variables))\n", + " return elbo" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Training by Batch" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0 114.67260603059552\n", + "120 67.871934322895\n", + "240 60.0149377742081\n", + "360 56.29566984959375\n", + "480 53.7197290981309\n", + "600 51.48258717207506\n", + "720 52.6730473693974\n", + "840 44.445047997931475\n", + "960 50.50109738207267\n", + "1080 44.75255389124902\n", + "1199 51.55517736426542\n" + ] + } + ], + "source": [ + "num_iters = 1200\n", + "num_logs = 10\n", + "num_training_points_ = X_train.shape[0]\n", + "\n", + "for i in range(num_iters):\n", + " batch_idxs = np.random.randint(num_training_points_, size=[batch_size])\n", + " x_train_batch = X_train[batch_idxs, ...]\n", + " y_train_batch = y_train[batch_idxs]\n", + " loss = optimize(x_train_batch, y_train_batch)\n", + "\n", + " if i % (num_iters / num_logs) == 0 or i + 1 == num_iters:\n", + " print(i, loss.numpy())\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Computing the Predictive Mean and Variance\n", + "\n", + "To compute the predictive mean and variance for a set of new $X^*$, the `predict_mean_and_var` from gpflow is used to compute (9) and (10). " + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "metadata": {}, + "outputs": [], + "source": [ + "from gpflow.likelihoods.multiclass import Softmax\n", + "\n", + "Fmu = tf.cast(tf.transpose(vgp.mean()), tf.float32) #TensorShape([6, 43])\n", + "Fvar = tf.cast(tf.transpose(vgp.variance()), tf.float32)##TensorShape([6, 43])\n", + "\n", + "S = Softmax(num_outputs)\n", + "m, v = S.predict_mean_and_var(Fmu, Fvar) #shape=(43, 6)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Results and Plots" + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Multiclass classification accuracy for 6 is 0.7209302325581395\n" + ] + } + ], + "source": [ + "acc =np.mean(np.argmax(m, 1).astype(int) == y_test.astype(int))\n", + "print(\"Multiclass classification accuracy for {} is {}\".format(num_outputs, acc))" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Multiclass classification variances for 6 is [0.23027779 0.21305028 0.23469782 0.2065194 0.23566929 0.18067268\n", + " 0.21430728 0.2347648 0.2097274 0.21258086 0.2179921 0.21743922\n", + " 0.2189273 0.22974259 0.22108854 0.20057674 0.1710678 0.21648297\n", + " 0.19312762 0.18159598 0.18870537 0.22279613 0.2087079 0.2288218\n", + " 0.19952135 0.21122998 0.21671966 0.22578193 0.2281411 0.24098359\n", + " 0.17000449 0.2195439 0.20093569 0.20745346 0.19372132 0.24052121\n", + " 0.15541928 0.22103915 0.19316027 0.20382655 0.16869901 0.22654828\n", + " 0.19985217]\n" + ] + } + ], + "source": [ + "indices = np.argmax(m, 1)\n", + "v_ = tf.reduce_sum(tf.one_hot(indices, 6)*v, 1)\n", + "print(\"Multiclass classification variances for {} is {}\".format(num_outputs, v_))" + ] + }, + { + "cell_type": "code", + "execution_count": 98, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.decomposition import PCA\n", + "\n", + "pca = PCA(n_components=1)\n", + "X_test_pca = pca.fit_transform(X_test)\n", + "y_preds = np.argmax(m, 1).astype(int)\n", + "y_sd = (v_)**0.5" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Each dot is the predicted class for each $x_i^*$, and the error bar is one sd of $y_i^*$. If the a dot and a cross overlap, this is a correct prediction." + ] + }, + { + "cell_type": "code", + "execution_count": 99, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Text(0.5, 0, 'X_test_pca')" + ] + }, + "execution_count": 99, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA24AAAE+CAYAAAD1QEO5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU1f3G8c9JMkkmOyEhhH1RBCQIGEVccKlWXKrV/nCrtW5FbW1rXWtrK7V73apdrLhWrWhxFxHRKiKImKAsAWRfQjayJySTZJbz+yOLhIRkgJnMBJ63r7zMPXPnnO/cZCbzcO85Y6y1iIiIiIiISPiKCHUBIiIiIiIi0jUFNxERERERkTCn4CYiIiIiIhLmFNxERERERETCnIKbiIiIiIhImFNwExERERERCXNRoS5gT2lpaXbYsGGhLkNERERERCQkli9fXmatTd+7PayC27Bhw8jNzQ11GSIiIiIiIiFhjNneWbsulRQREREREQlzCm4iIiIiIiJhTsFNREREREQkzIXVHDcREREREQkvbrebnTt30tDQEOpSDimxsbEMGjQIh8Ph1/4KbiIiIiIisk87d+4kMTGRYcOGYYwJdTmHBGst5eXl7Ny5k+HDh/t1H10qKSIiIiIi+9TQ0EDfvn0V2gLIGEPfvn336yymgpuIiIiIiHRJoS3w9veYKriJiIiIiEhAzcnNZ05ufsD6i4yMZMKECYwbN47p06dTX19/wH1dffXVvPLKKwBcf/31rF27tsM+zz77LDfffDMAb7zxRrt9fv3rX/PBBx8c8PgHSsFNRERERETCmtPpZMWKFeTl5REdHc2//vWvdrd7vd4D6vfJJ59k7NixXe6zd3C77777OPPMMw9ovIOh4CYiIiIiIgGzrqiaBWuKeTknn4ff38C6ouqA9n/KKaewadMmFi5cyOmnn84VV1xBVlYWXq+XO+64g+OOO47x48fz+OOPA80Lgdx8882MHTuW8847j127drX1ddppp5GbmwvAM888w6hRozj11FNZsmQJAJ9++ilvvfUWd9xxBxMmTGDz5s3tztj973//Y+LEiWRlZXHttdfS2NgIwLBhw7j33nuZNGkSWVlZfPXVVwf9uBXcREREJKgCfcmUiISvdUXVzFq0FVeTlxRnFNUuN7MWbQ1YePN4PLz77rtkZWUB8Pnnn/P73/+etWvX8tRTT5GcnExOTg45OTk88cQTbN26lddff53169ezevVqnnjiCT799NMO/RYVFXHvvfeyZMkS3n///bYzbCeeeCIXXHAB999/PytWrGDkyJFt92loaODqq6/m5ZdfZvXq1Xg8Hh577LG229PS0vjiiy+46aabeOCBBw76sSu4iYiISI8JVYjralwFS5HAmZ9XQrLTgTM6EmMMyU4HyU4H8/NKDqpfl8vFhAkTyM7OZsiQIVx33XUAHH/88W3L6S9YsIDnnnuOCRMmMHnyZMrLy9m4cSOLFi3i8ssvJzIykgEDBnDGGWd06H/ZsmWcdtpppKenEx0dzaWXXtptTevXr2f48OGMGjUKgO9///ssWrSo7faLL74YgGOPPZZt27Yd1OMHfY6biIiI9LCcbRUATM8efFD9tIatrvpRIBPpWQVVLjKTY9u1JcZGUVDlOqh+W+e47S0+Pr7te2stf/vb3zj77LPb7TNv3jy/VnDc31UerbVd3h4TEwM0L6zi8Xj2q+/O6IybiIiIiIgExMAUJ7UN7UNKbYOHgSnOoI999tln89hjj+F2uwHYsGEDdXV1TJ06lZdeegmv10tRUREfffRRh/tOnjyZhQsXUl5ejtvtZs6cOW23JSYmUltb2+E+o0ePZtu2bWzatAmA559/nlNPPTVIj07BTUREREREAmTauAyqXW5cTV6stVS73FS73EwblxH0sa+//nrGjh3LpEmTGDduHDfccAMej4eLLrqII488kqysLG666aZOw1VmZiYzZ85kypQpnHnmmUyaNKnttssuu4z777+fiRMnsnnz5rb22NhYnnnmGaZPn05WVhYRERHceOONQXt8prtTfD0pOzvbtq7qIiIiIoeGPS9pnJObT862Co4blhqSSyU729effkQOZ+vWrWPMmDH+719UzUMLNlBZ7+akI9KYNi6DMZnJQayw9+rs2Bpjlltrs/feV3PcREREREQkYMZkJvPNo/sD+geRQFJwExERERGRgFJgCzzNcRMREREREQlzCm4iIiIiItKlcFoX41Cxv8dUwU1ERERERPYpNjaW8vJyhbcAstZSXl5ObGxs9zu30Bw3ERERERHZp0GDBrFz505KS0tDXcohJTY2lkGDBvm9v4KbiIiIiIjsk8PhYPjw4aEu47CnSyVFRERERETCnIKbiIiIiIhImFNwExERERERCXMKbiIiQWK9pVjr6qS9DOurD0FFIj3HevLbvo+OqCbK1GE9O4iLLPp6H18V1leL9ew44L47a7e+Cqxvd8v216vgRUdUt7X73J2Pub+1iIj0lKAGN2PMNmPMamPMCmNMbjDHEhEJJ9Y2YSu/j628oV14s95SbMWV2OpbQlidSHDZpi+xZdOwdf/GWi8npd3NNzJuwJadzdn9v8+E9E9wmBpsxdXYsguxZedg3Wv867thfnPfDe+1b69/EVt2Nr7Gz7AV12Ervoctv5Bjkv8BWKIjqjkl7XZsxVXY8gug/JvYumf26vuj5lpcbwTqUIiIBExPnHE73Vo7wVqb3QNjiYiEBWOiMfE3QNPnbeGtObR9D3zFmPgfhLpEkeBxjIOY07G1v6e4+AleXnMOzogSrPUCXvIKKjgq9id43V+BbyfEToOo0f71HX0yOLLwVd5CadlbvJyTzwfL/4qtmQkxUzHRkzAJN4JnHW5fLEckvk58w28Y6/wxcRHbaWrcRJMvCaJPwdb+keKix1iwppiN+W/jqfgRLnsExJwezKMjInJATDA/SM8Ysw3IttaW+bN/dna2zc3ViTkROXRY15vY6rsgagT46sFWYfo8gYk+LtSliQSVtW5qim8m0XzEmtKxjO77FRHGh9sbiSPSi8UQYSyfFZ5In8yHGJOZ6nffXxUV4qi5gcGJm9hQMYaxaXmsKTsWR99HGZ2ZDkB+0ev05+fUNzlJjKnHZw0NHgcuTzKPrZzJ9MmTGMA9JJr/saZ0LEf1XU9J/XAeW/kLvndiFmMyk4N1aEREumSMWd7ZSa9gn3GzwAJjzHJjzIwgjyUiEnaM80JM4h3g2QS+QkzKowptclgwxsGzeT/F44vi6PS1REb4eOiz6URHeTEGIoyluiGR+z+9lvl5fv37bpt383bz8sZ7iYrwMjYtD4A3tt7Fu3mVbfu8knc07267gcSY5vmkEcYS52hi5id34YsYwPy8cp7N+zFuXzRHp68lKsLL29t/R7SjD/PzSgJ3IEREAiTYwe0ka+0k4BzgR8aYqXvvYIyZYYzJNcbk6tPYReRQY72l2Po5X2/XPdnpgiUih6KEiKVERXjatq/Mer/d7fHRdYxJW0NB1f49JwqqXBzX/5N2bePTv2jXT+XuEib3n9/hvmcNm0dibCQFVS4SIj4jyrjbbhvb5wMSY6P2ux4RkZ4Q1OBmrS1s+f8u4HXg+E72mWWtzbbWZqenpwezHBGRHtVuTlvqfzDJ97eb8yZyKLMNH3LVmAcp2H0E9yx6gNomJxkJVQDkFo7A44vA44vi16c8ypRB6/ar72kjPuIbgx5j5a5J3PrhvyisG8O5Q+7n1CFfNI/tq+BHE35Ln5id1LtjqHQl4PUZSuuTuWDUh5zY73FOHryKq8Y8QGHdSG778DGWFx/P1AFPMjb5NQamOAN+PEREDlbQgpsxJt4Yk9j6PfBNIC9Y44mIhJPmVSWvbg5tLXPajPNCTPKfm8NblVaVlEOXbVqBrfoxTWYUj624m2uy/okzqrHt9lfXnsi9C39AVIQHn43iwmG/xbrX+td3w3ucMfCfrCk7lkdyfsTupgSeW/crdtQewblD/tK2qmTf2EK8vihqmxK57q27+PVH19MntoYKV18m93+Hbw39PU3mSB5b+QtK6+J5cuUPWVsxhbOHPsMl45YH69CIiBywqCD2nQG8boxpHedFa23HaxZERA5BxkRD/AyIHNBuTptxXghEQES/0BUnEmyOsRB/HXHx1/O9kyzvr/wWrqYmJmYWU7a7kSX5WXhJ4uUNt3HaUckMTCiEqFH+9R19EsTfgMNehSNqG5X1bpwxabiTHsckzMZET4KEGUSQQE1lLn/NPYomm8BnRf34y9IoxgzI4ORhxaTH1RCX+DO+dyI8tGAD5fU+/ldwFxl93mJAxrnBPT4iIgcgqKtK7i+tKikiInLomZPb/MHY07MHMyc3n5xtFRw3LJXp2YMD1m93+7TqbF9/+hER6SmhWlVSREREREREDpKCm4iIiIiISJhTcBMREREREQlzCm4iIiIiIiJhTsFNREREREQkzCm4iYiIiIiIhDkFNxERERERkTCn4CYiIiIiIhLmFNxERERERETCnIKbiIiIiIhImFNwExERERERCXMKbiIiIiIiImFOwU1ERERERCTMRYW6ABEREZEDMT17cKhLEBHpMQpuIiIi0mOmZw/u0cDlz1gKgCLSGyi4iYiISFApGImIHDzNcRMREREREQlzCm4iInuYk5vPnNz8UJchIiIi0o6Cm4iIiIiISJhTcBMREREREQlzCm4iIiIiIiJhTsFNREREREQkzCm4iYiIiIiIhDkFNxERERERkTCn4CYiIiIiIhLmFNxERERERETCnIKbiIiIiIhImFNwExERERERCXMKbiIiIiIiImFOwU1ERERERCTMKbiJiIiIiIiEOQU3ETnsWWv3q10Ob9baTn839tUucqjRa6ZIaAQ9uBljIo0xXxpj5gZ7LBGR/VXnqWHW5l+wZXdeu/Z6Ty2zNv+CzbtXh6gyCUfWWhYU/4d3Cp9u9ybVWssHJbN5u/BJvXmVQ5rXenl5x0MsLXunQ/t/8//KktK3Q1SZyKEvqgfG+CmwDkjqgbFERPaLxUeDt57ntv6OU5N+xoI1EVQ31bDavIZ1lOKznlCXKGHGa90sLW9+03regGsB+KBkNgt3vcJxqWdhsRhMKEsUCSKL13qYW/gUAFPSzsNrvbyS/yirqj4hM3ZoiOsTOXQFNbgZYwYB5wG/B24N5lgiIgciISqF60b8hn9uuIf3Kx/ExJ/HsKOW4o2qoGHHJXjSR0BiqKuUcGGM4ZzMqwFYUtZ8ZiE6MpaPd73KcalnccHAG4gwmoUgh65IE8WlQ27l5R0PMbfwKXzWx07XJlZVfcLZ/a9kar+LQ12iyCEr2Gfc/grcid72iEgYS3CkEFl8DSb1EdJHvQlAUuW1OHzDmZ9XwpjM5BBXKOGks/CWrdAmh5GoCAeXDrmV2dvvZ17RMwAKbSI9IGh/YYwx5wO7rLXLu9lvhjEm1xiTW1paGqxyRES6VFTVhNnzJdFGkhgbRUGVK3RFSViLiohu+95hHLo8Ug4rxkQQFeFo23ZExISwGpHDQzD/afAk4AJjzDbgJeAMY8wLe+9krZ1lrc221manp6cHsRwRkc7Ve2qJG/YCNqKemm3n43GlUdPnWarZyMAUZ6jLkzDTuhDJx7teJTv1LE5MO5+l5fM6LFgicqhqndOWV72UMzMuZ2zSZOYWPtVhwRIRCaygXSpprb0buBvAGHMacLu19spgjSciciDqPbU8vWUm1lFKw45LqCwZRGPNcNJHv4Sn33NM6HNbqEuUMNIa2loXIrlg4A2Ylv9aL5s8b8C1GKOzb3Jo2nMhktbLIz0+d9ucN2hesEREAk8X44vIYc3bsmrkVcPv5vpjz8IZHUlFbQyp1TPoGz2QAX0c3fQgh5smn6vdQiStc95OSvsWTb4GLDrrJocua324fY3t5rS1znkbmzSZJl9DiCsUOXSZcLqsIzs72+bm5oa6DBE5zPisr21RiTm5+QBMzx7crl2klbUWi+3wu7GvdpFDzb5eG/WaKRIYxpjl1trsvdt74nPcRETC2r7eaOgNiHTGGNPpQiT7ahc51Og1UyQ09AwTEREREREJcwpuIiIiIiIiYU7BTUREREREJMwpuImIiIiIiIQ5BTcREREREZEwp+AmIiIiIiIS5vYruBlj+hhjxgerGBEREREREemo2+BmjFlojEkyxqQCK4FnjDEPBb80ERERERERAf/OuCVba2uAi4FnrLXHAmcGtywRERERERFp5U9wizLGZAKXAHODXI+IiIiIiIjsxZ/gdh/wHrDZWptjjBkBbAxuWSIiIiIiItIqqrsdrLVzgDl7bG8BvhPMokRERERERORr/ixOMsoY8z9jTF7L9nhjzD3BL01ERERERETAv0slnwDuBtwA1tpVwGXBLEpEJFSmZw9mevbgUJchIiIi0o4/wS3OWvv5Xm2eYBQjIiIiIiIiHfkT3MqMMSMBC2CM+T+gKKhViYiIiIiISJtuFycBfgTMAkYbYwqArcCVQa1KRERERPZpTm4+gC7tDjAdVwln/qwquQU40xgTD0RYa2uDX5aIiIiIHGoUjEQOnD+rSv7UGJME1AMPG2O+MMZ8M/iliQg0/5Fr/UMnIiIiIocnf+a4XWutrQG+CfQDrgH+FNSqREREREREpI0/wc20/P9c4Blr7co92kRERERERCTI/Aluy40xC2gObu8ZYxIBX3DLEhERERERkVb+rCp5HTAB2GKtrTfGpNJ8uaSIiIiIiIj0AH/OuE0B1ltrq4wxVwL3ANXBLUtERERERERa+RPcHgPqjTHHAHcC24HnglqViIiIiIiItPEnuHmstRa4EHjEWvsIkBjcskRERERERKSVP3Pcao0xdwNXAlONMZGAI7hliYiIiIiISCt/zrhdCjQC11lri4GBwP1BrUpERERERETadHvGrSWsPbTH9g40x01ERERERKTHdHvGzRhzgjEmxxiz2xjTZIzxGmO0qqSIiIiIiEgP8edSyb8DlwMbASdwPfCPYBYlIiIiIiIiX/MnuGGt3QREWmu91tpngNO6u48xJtYY87kxZqUxZo0x5jcHWauIiF/e2r6G1RVFHdrn7ljLqvLCEFQkInJwrLU8tX4ZJa7aDu1Pr19GcX1NiCrrnsvj5m9rPqHJ623X3rCP9p72SfEWFhVt7tC+pGQrC4s2haAikc75s6pkvTEmGlhhjPkLUATE+3G/RuAMa+1uY4wDWGyMedda+9lB1CtyWFlXVM2CNcVU1rvZWeli2rgMxmQmh7qssNbo9fBw3sdUNbp47rQryErNBJrD3G3L3uSMAUfy+MnTQ1yliMj+Kaiv5pG8Rcze/AUzx17AgjXFVNQ3Mafgc5Y3fEWNu4Fbxp0a6jI7tah4M3/NW8RnRTuJLRhBdb2XbRW7+czmsLIqn4l9B3Jy/xEhqc1ayyN5i1hTWcw9Y89j4ZoGKuvdLN21jXfrljA6pR9T+48kwpiQ1CeyJ3/OuH0PiARuBuqAwcB3uruTbba7ZdPR8mUPsE6Rw866ompmLdqKq8lLijOKapebWYu2sq5IU0y7EhMZxQunfZek6BiuWvgiqyuK2kLbcWmDefiEC0NdoojIfhsUn8LTUy+juL6WHy6bQ7W7nsrE7Sxv+IqhDOas1GNCXeI+nT1oNDeMnMpn5VtYGbmSBKfljepPWFGVz09HfSNkoQ3AGMMTp1zCIGcfZubNpci7C198NXNrFxPji+OuUecqtEnY6Da4WWu3W2td1toaa+1vrLW3tlw62S1jTKQxZgWwC3jfWrvsYAsWOVzMzysh2enAGR2JMYZkp4Nkp4P5eSWhLi3sDYxP5sXTv0dSdAzffv9pfvbZGxyXNpgnp15KXFR0qMsTETkg2emDOTfuZBppZHniYrY7tjI2ZjinxE5iwZpdoS6vS47KDI5zjKfSUconzoUU+0o50TERT1lqqEujT0wcpztOJNkksi5+BStjviAlMpFpsSezdEP4XoIqh599BjdjzGpjzKp9ffnTecucuAnAIOB4Y8y4TsaZYYzJNcbklpaWHvgjETnEFFS5SIxtfzVzYmwUBVWuEFXUuwyMT+bykZPatm8Zd6pCm4j0et7dcfSLSmnbPt55NElOR9j/bSiocjExfmS7tqz44WFTd0WNj+PiRrdtT4o9inRnfNjUJwJdn3E7H/hWF19+s9ZWAQuBaZ3cNstam22tzU5PT9+fbkUOaQNTnNQ2eNq11TZ4GJjiDFFFvctb29fw4OqFDI5PITUmjhsWz+l0wRIRkd7CWkt+zGYKPWXE++KJtJG8UbOQEldt2P9tyEiO5q2axWAh0ZcEwLyapfRPjglxZc188TX8ry6HaBtDjI3hw7pc1tcVhv1xlcPLPoNbyyWS+/zqrmNjTLoxJqXleydwJvBV4EoXObRNG5dBtcuNq8mLtZZql5tql5tp4zJCXVrY23NO27xpP+CNs65tN+dNRKS3sdby4OqFbXPasnZPZnzTJHb7GpjfsJjjjvRn3bjQaPC4+czmUOwrZWTDGLIbJ3OcYzz5viK+jPgy5KtKLinZyrt1S4gjnvG7j+e4hhNIMol81PQZ/QY0hrQ2kT11dankdcaYO/bYLjDG1Bhjao0xN/nRdybwUctllTk0z3Gbe/AlixwexmQmM2PqcJzRkVS5PCQ7HcyYOlyrSnaj0evh4dUL281p23PO29/XfBLqEkVE9ltBfTX/3pjDpSMm8LdTLiAuOgrqErgw8RR8kU3k7g7fZes/Lt7Mquqd3DLqGww1g6lyeZiceBQ3jJzKsvItfF7a7fmAoLHW8vDqjxme2Jd/nXAJSQ4nda4ILk4+lWHxqcwpyMFntbaehAdj9/HLaIzJAaZZa8tbtr+01k40xsQCC6y1UwNdTHZ2ts3NzQ10tyK92pzcfACmZw8OcSW9R2FdNSkxzg5z2grra0h2xBLv0Fw3Eel9ttSUMywxlQhj2v1t2LM9XG2qKeOIpLQOf9M215QxMiktlKVR0ViPtZa+sfHt6qtsrMdrLWmx4Xs2Uw5Nxpjl1trsvdu7+hy3iNbQ1mIOgLW2oeXSRxGRsDQgvvOzkgPiknq4EhGRwBmR1He/2sPJEfsIZ6EObQCpMXGdtvfZR7tIqHS1OEm7dz7W2j8AGGMigPB/hRARERERETlEdBXcFhhjftdJ+33AgiDVIyIiIiIiInvp6lLJO4AnjTGbgJUtbccAucD1wS5MREREREREmu0zuFlr64DLjTEjgKNbmtdaazf3SGUiIiIiIiICdH3GDQBr7RZgSw/UIiIiIiIiIp3oao6biIiIiIiIhIGuPoB7eE8WIiIiIiIiIp3r6ozbKwDGmP/1UC0iIiIiIiLSiS4/gNsYcy8wyhhz6943WmsfCl5ZIiIiIiIi0qqrM26XAQ00h7vETr5ERERERESkB3T1cQDrgT8bY1ZZa9/twZpERERERERkD91+HADwqTHmIWBqy/bHwH3W2urglSUiraZnDw51CSIiIiISYv58HMDTQC1wSctXDfBMMIsSERERERGRr/lzxm2ktfY7e2z/xhizIlgFiYiIiMihSVeRiBw4f4KbyxhzsrV2MYAx5iTAFdyyRERERGRfFICCQ8dVwpk/we1G4DljTHLLdiXw/eCVJCIiIiIiInvqNrhZa1cCxxhjklq2a4JelYiIiIiIiLTx54wboMAm3ZuTmw/oMgMRERERkUDzZ1VJERER6cSc3Py2f7QSEREJJgU3ERERERGRMNdtcDPGTDfGJLZ8f48x5jVjzKTglyYiIiIiIiLg3xm3X1lra40xJwNnA/8GHgtuWSIiIiIiItLKn+Dmbfn/ecBj1to3gejglSQiIiIiIiJ78ie4FRhjHgcuAeYZY2L8vJ+IiIiIiIgEgD8B7BLgPWCatbYKSAXuCGpVIiIiIiIi0qbb4GatrQd2ASe3NHmAjcEsSkRERERERL7mz6qS9wJ3AXe3NDmAF4JZlIiIiIiIiHzNn0slLwIuAOoArLWFQGIwixIREREREZGv+RPcmqy1FrAAxpj44JYkIiIiIiIie/InuP23ZVXJFGPMD4APgCeCW5aIiIiIiIi0iupuB2vtA8aYs4Aa4Cjg19ba94NemYiIiIiIiAD+LU4SD3xorb2D5jNtTmOMI+iVSQflu+vZVbO7Q3tlnYuS6o7tB+urotJu2+sam9hRXtVhn7rGJraXdWwXEREREZH958+lkouAGGPMQJovk7wGeLa7OxljBhtjPjLGrDPGrDHG/PTgSj28WWv5yfNvcfWsOe3CW2Wdi2uffIUbn30dr88XsPHmr1rPdx59gTmfr27X/tTHOXzn0Rf4dON2AH4x5z2uevy/bCutbNunrrGJm559g2ufnEOD2xOwmkREREREDlf+BDfT8lluFwN/s9ZeBIz1434e4DZr7RjgBOBHxhh/7iedMMZw+7lTKa2tawtvraFte1kld543lcgIf36c/jl9zEimHjWcma9/0Bbenvo4h4fmL+ac8Udx/IjBAPz4rCl4fT6ufPxl3vxyK7OXbeOCh2fz5fZCbj93KrGObq/GFRHpldYVVbNgTTEv5+Tz8PsbWFdUHeqSRETkEGaaF4zsYgdjvgR+CDwMXGetXWOMWW2tzdqvgYx5E/h7V/PjsrOzbW5u7v50e9j5cnshM55+jajICCIjIqhvbOIf37+QKUcMDfhYjW4Pt/xnLovWb2Vo3xS2l1dxzvij+NMl04iK/DokLsjbxp0vz8Xr9REZEYHH6yFr6Bh+feEJjMlMDnhdIiKhtq6omlmLtlJa00CsI4JBqfFUu9zMmDpcr3siInJQjDHLrbXZe7f7c4rmFpo/fPv1ltA2AvhoPwcfBkwElu3P/aSjiUMH8OdLz6HG1UhlnYt7LzozKKENIMYRxV+/ez4A21vmse0d2gDWFDVxyuiJ+KwXt9fNpOFHctSAQczPKwlKXSIioTY/r4RkpwNndCTGGJKdDpKdDr3uiYhI0HQb3Ky1H1trL7DW/tkYEwGUWWt/4u8AxpgE4FXgFmttTSe3zzDG5BpjcktLO18MQ75WWefib+9/2rb9+IfLOl2wJFBe+PTLdtuvL1/TYZ/t5bWsyd/Ytr2uYDs+XwMFVa6g1SUiEkoFVdI5xpsAACAASURBVC4SY9tfCp4YG6XXPRERCRp/VpV80RiT1LK65FpgvTHmDn86b1l98lXgP9ba1zrbx1o7y1qbba3NTk9P35/aDzt7zml78rqLeeHGS9vNeQu0Pee05fzm5g5z3qB5IZLV2/LYVV3FqAFHMWH4RHzW8s4Xy0iI9ga8JhGRcDAwxUltQ/vFl2obPAxMcYaoIhEROdT5c6nk2JYzZd8G5gFDgO91dydjjAGeAtZZax86qCoFay03P/8m28sq2+a0TRw6gFnXXkxpbR03PBPYVSXfW72hLbT96ZJpxEU7+Ot3z28Lb3uuKrmrppKsoWOIj03FGR3H1LHZeK2Pj/JytaqkiBySpo3LoNrlxtXkxVpLtctNtcvNtHEZoS5NREQOUf4sTrIGmAC8SPPiIh8bY1Zaa4/p5n4nA58Aq4HWRPELa+28fd1Hi5N0bfm2Apo8ng5z2r7cXkh9o5uTRgVurluj28OLS1fwvZMmtZvTtnf7huIytpVVMrhvPx5asIHKejcnHZHG0ZnRWNvE2VmjAlaTiEg4WVdU3e51b9q4DC1MIiIiB21fi5P4E9x+AtwFrATOo/mM2wvW2lMCXaSCW+82JzcfgOnZg0NciYhIz9DrnoiIBNq+glu3H7JlrX0UeHSPpu3GmNMDWZyIiIiIiIjsm1+fjmyMOQ84Gojdo/m+oFQkIiIiIiIi7fizquS/gEuBHwMGmA4E54PDREREREREpAN/VpU80Vp7FVBprf0NMAXQxfwiIiIiIiI9xJ/g1vppovXGmAGAGxgevJJERERERERkT/7McZtrjEkB7ge+ACzwZFCrEhERERERkTb+rCr525ZvXzXGzAVirbXVwS1LREREREREWu0zuBljLu7iNqy1rwWnJBEREREREdlTV2fcvtXFbRZQcBMREREREekB+wxu1tprerIQERERERER6dw+V5U0xtxqjLmuk/YfG2NuCW5ZIiIiIiIi0qqrjwO4Fni+k/ZZLbeJiIiIiIhID+hqjpu11jZ10thojDFBrElERKRXmJ49ONQliIjIYaLLjwMwxmRYa0v2bgtuSdJb6Q2MiIiIiEhwdHWp5P3AO8aYU40xiS1fpwFvAw/0SHUiIiIiIiLS5aqSzxljSoH7gHE0fwTAGuBea+27PVSfiIiIiIjIYa/LSyVbAtphHdLm5OYD/l0GuD/7BnpsERERERE5dHV1qaTsp5xtFeRsq/Br3zm5+W3B7EAc7P1FDkd63oiIBJZeV0V6joKbhERPvtDrj4qIiIiI7Kk3vj9UcBMJY73xRUU6p5+liIiIHIx9BjdjzGRjTFLL905jzG+MMW8bY/5sjEnuuRJFRA6cApOIiIgcCro64/Y0UN/y/SNAMvDnlrZnglyXiIiIiIiItOhqVckIa62n5ftsa+2klu8XG2NWBLkuERERERERadHVGbc8Y8w1Ld+vNMZkAxhjRgHuoFcmIiIiIiIiQNfB7XrgVGPMZmAssNQYswV4ouU2ERERERER6QH7vFTSWlsNXG2MSQRGtOy701pb0lPFiYiIiIiISNdz3ACw1tYCK3ugFhEREREREemEPsdNREREREQkzCm4iYiIiIiIhDkFNxERERERkTCn4CYiIiIiIhLmFNw64fX6/Gr3en1tX/6076vvvfdt/d7XSb+B4u9jDKT9GTMQ9YXiMQaKtbZX1x9u9n4utdKxFBEROXz09vdWQQtuxpinjTG7jDF5wRojGIqLq7juB0+Rk7u1y/a33v6Sm3/6PN/93mNcdc0s6uoaAShdu4Obf/Ic373qX1x1zeNt7QDPPb+YX/7qFZqaPG1tucu38n+X/I3b7phNU5OHxUs2MOPGZ/ho4TpefuQdaip2A7Dk0w384ManKSurPejH+OWK7Vz7gycpKKxs175iZeftgbB7dwM/vuV53pm3skP7T255gbnzVrS1bd1WytXXPsG6dYXt9m1tX7u2oNvxPl26kR/c+DSlex2vpZ9t4vobOraHm38/v5h7fv0qXo+3Q/vev0PStV07y3np4bls3rKrXfv69UVcfe0THdpFRETk0LNl6y6uvvYJvlpf1K5969ZSrrnuCdZ9VbiPe4aPbj8O4CA8C/wdeC6IYwSc0xlNlCOCX937KjN+Oo0FRS7KSmt59rM8cHtISowFoE+fODZuLMbhiKSx0cOV1z5BcUoKzi07IcKAzzJqVH8cjsi2vvv2TeTznC3cevd/cR09gupGH4lb8qmucbFhYzFlZTUkJzvZWVDBb3//Jo6kON5ZW8KGTSUsf+tzRh2ZQazTwbqiahasKaay3s3OShfTxmUwJjPZ78eYlBhLdVU9t90+mwcfuJyBA/qwYuV27v7lHPr3TyHOGR3w4xodHUVSopMHH36Xoup6ljVFUV7lYvaj86jdVc0Vl5/Qtm98XAw+a7nz5y/zlz9dypgxA9i6rZTb75iNiTAktPwMunyMSU527arh5lteIO6kcdQSyfKcLSx/cxlHjOxHbIwj4I8xkNL6JvLc50tYvbOKmOPGsLPShXfjDua9nsvZ38wiMlIny/2xrqiaz/Orqalr5KafvkDSyeMgKYG8tYXkvrqE5CQn8XGB/33vrI6Dec6KiIQjvbZJbxIfH4PP5+OOu17iR3ecz4JNVZQWVfHU0jycMVEkJHT//jLUgvbuz1q7CKgIVv/Bkpwcx4N/uZz0jGT+8dA8ytblU7d4Fe4GN4POOhZfUhwAp5x8FPf+6tu43V6iHJFUV+zGuWUnPtMc2nyJcZxz3RlER3+djc879xiuuPZU1q7aQeFHq/Ct3872FVtJHJyOiTDcdudL7NhRgc9aoPm0bdSuSnLfXEZsaiLX33IO+TWNzFq0FVeTlxRnFNUuN7MWbWVdUbXfj3HkyAwevP9yGhqauO322cx7d2VbaHvo/svp0yc+sAeV5uB238yLGTt+MC8+/TGlq7fhWppHdUkVg884htThGW379uuXxMMPXEFSspM7f/4y785f1RbaHnrgCoYM7tvteOOOHsRNt51HecVudi74goidJeS88RnRfRK4/mfnkuhH+AulkZOGM+jko6kvLKchZx0rPlzFvNdzOeGUo7j91nMU3PywrqiaWYu24o2NIWXqMXgslH+8Cvf2Ipb+dzHeiEh+eMf59O+f0iN1HMxzVkQk3Oi1TXqbjH7JPPTgFcTGRfPAH96kfP1Odi9eBcaQeeYkdkdGdt9JiOndXyeSk+M4+sITiIiMoHr5Bjx1DUy55GQyh6QxP6+kbb/W8OZxf305W4S1RCbFEXPSeP6T0/GSvtq0vhx5xngaiyuoWrOdAaMHMeGCyWT/30mUlNTw4MPvYiIiOPr84/DVNVCWs56Y+FgmXDSFj7dUMT+vhGSnA2d0JMYYkp0Okp2OdnX5ozW87Sqt4YGH3sVaghbaWkVHRzHszEkk9e9DzcrNNJbXkH3hZIaMHtSh/tbwVlffyP0PzqOyqt7v0NZqQ6Mh68IT8O52Ufb5eqKdMUy4aAqLtlYF+qEF3Py8EkZMHEH65NG4CsvJz91E+qiBpE0eq9Dmpz2fK5UmgviTszBuD7XLN+JpdDPhOyeSU+Lq0ToO5jkrIhJO9NomvVFGv2QmfOckfE0eqnLW42t0c9Llp9Cvf0qv+N0N+TtAY8wMY0yuMSa3tLQ01OW0KSjfDfbr7aaGJhJjoyioav9Gr6qqvsN9rcdLgsNQUtPQsd8qF6bR3bbtbmgiPjqCorLdbW0+n223j/X5cEYaCqpcFFS5SIxtf4VrZ3X5o3b31/VZa6l3Ne13H/trZ3kd1vt10G2sb9xn/XX1je23dzd22KcrBVUuHL6vx7I+H3EtxzHctf6cPa49HrPbw87KutAV1cvs+VxxuX1E2fYTj2OwPfK7EMjnrIhIuNBrm/RWhXu85wbwNLp7ze9uyIObtXaWtTbbWpudnp4e6nKA5oVItszLwUQYMs+YQHRKAjmvL2X7hkIGpjjb9nt77pc8/Mh7zXPaaM55PmPw1TdS+cEXpDs7TiGsW7uNDUvWkTCiP2nHH0Xptl0s/s8iNr+Xy4jh6fz5D5dgDOS9v4LolAT6n3YMniYPn85eRN8oGJjipLah/cIUtQ2ednX5o3VO29ChafzlT5fijHVw2+2zg7IwSavduxvYsWA5u8tryThlHM4BfVn13pdsWL65Q/2tc9pSU+O5/8+XkZmZwp0/f7nDgiVdiSqrYvkby4hJTaT/6cfg8/pYMvsTUsP/TDgDU5zkfbyGylVbSRyRSdZZEyjdWkLhRyu1MImf9nyuOOvrqf5kFTY2Gps9mtiEWD57eTGJTcH/x4pAPWdFRMKJXtukN9q6tZTN83KIjo8l8xsTiUqIZel/l1C4vaxX/O6GPLiFm+KSam69Yza4PQyflo1JSyHzGxNwJiewZm4OQ2l+ozf3nRU8/Mh7zXPYWua01Y4ahrEWIiLA1Yj3k5XtVpV8/j9LWL94HX2OHEDipKNIOnIgAyeMYPeuKqIdUfzx99Npcnualy43Bk+jG0dyHOMumEzj7gZWvvYpUwYlUO1y42ryYq2l2uWm2uVm2riMfT2kDvZciOSh+y8n+9jh7ea8FRYF/lLC3XUN3Hn3y9TuqmboNyYQmZlGxtRx9BmSzsYPV5FYVt6277btZe3mtB07aVi7OW/+rPqz9LNNLG+ZG9hn6njiBvTl6Asm01TffBzDfVVJ36Z8tn++Aeew/qRNHk2f0YMZdPLR7NpSwr2/eV3hzQ/TxmU0Pz+KqzDL10NUFPXHjKLf0HSyLpoCEYblry5hy9bgrirZWsfBPGdFRMKNXtukt9m6rZTb7pyNMyaKEeceh+mbzIAzJxEV42DlG0sZE2dCXWK3gvlxALOBpcBRxpidxpjrgjVWIMXGOOiXnsRDf7mcW74zAWd0JLU2gpMuP4UjjuzP2CHNc6ySkmKZcsJIhg7py5jRmfzsnm9DZl+qjxqGo18KGQNTyUxPbLeqZEpyHGefNY77fvkt4mKjqHJ5GJE1lPSMZE44fiSpqQkkJMQyYfwQbrn7W0QnOql1W4aM7M9P7/oWA/unMG5wKjOmDscZHUmVy0Oy08GMqcP3axWn+PgYRo3q325OW+uct4z+ScTGBn7FxWhHFH1S4pn5629zx/dOwBkdSU2TZcr/nUjWxGGMHvL13DWn08HAgant5rS1znkbPjyd+LiYbsdLTIglK2sQ9/32/4hPiKXK5WHIiAx+9vMLGJiZHParSh4xqA8nnDKazBPHUt3oJdnp4O4ZU7n1lmmkJDs1z80PYzKTm58rcdFEJsUx8eIpDB+Uitca+mf24RczL2bYkLSgrKLaaR0H8ZwVEQk3em2T3sYZ62DQwFQefei7/Phb43BGR7I7IoqTrpjKsKFpjB2SGuoSu2Wstd3v1UOys7Ntbm5uqMvAWosxzal7Tm4+ANOzB7drb92vlTGGO19p/oyyP39nfLv2zvreu989923d5785OzDGMD17cJd1Hexj9Kc9EPw9roGqr7Njvb99hJK1lleW7wTa/5x7S/3hYk5uPtZaLjluSEh/Fw72OSsiEo702ia9yf68Fw0lY8xya2323u3B/By3XmtfP7i92/3dr7vb9tWvv+MdiAOpvSfHDER9oXiMgdTb6w8nOpYiIiLS298P6HorERERERGRMKfgJiIiIiIiEuYU3ERERERERMKcgpuIiIiIiEiYU3ATEREREREJcwpuIiIiIiIiYU7BTUREREREJMwpuImIiIiIiIQ5BTcREREREZEwp+AmIiIiIiIS5qJCXYCISDBNzx4c6hJEREREDpqCm0gYU+g4dOhnKSIiIgdDwa2X6u1vAnuy/t5+rEREREQksHrj+0MFtwA6bliq3/v2xl8Wkd5OzzsRkcDS66pIzzHW2lDX0CY7O9vm5uaGugwREREREZGQMMYst9Zm792uVSXD2JzcfObk5nf4XprpmIiIiIjI4ULBLQACFSC66idnWwU52yr8vq+/Nc3JzefOV1Z2uW+oA1KoxxcRERERCTUFN2nTEwG0tzuUH5uIiIiIhC8FNxERERERkTCn4CYiIiIiIhLmFNxERERERETCnIKbiIiIiIhImFNwExERERERCXMKbiIiIiIiImFOwU1ERERERCTMKbiJiIiIiIiEOQU3ERERERGRMKfgJiIiIiIiEuYU3ERERERERMKcgpuIiIiIiEiYU3ATEREREREJc0ENbsaYacaY9caYTcaYnwdzrENF4fJNvP2vBQBsW5TH3B/+E5/Px+612yldsByAZfO+4PYzZvLczP+Sv76g7b5FX2zmrcfe44XfvkLl1pK29rzF63jrn++1G2f7up08f98cyjcXUfzKJ1hrAajavovnZv6X7WvzeeG3r7S1d8Vay39+9yrb1uR3aH/xD6+xNW/HgR2MTnz1+UZefXhuh/b1uZuZ8+DbARtnT68+PJevPt/Yof21R95h7WcbAjqW1+PlibteoKywon27t6W9oDyg44mIiIhI7xAVrI6NMZHAP4CzgJ1AjjHmLWvt2mCNGQrriqpZsKaYyno3OytdTBuXwZjM5APup+Cp/+HOXc/qHRV8+upnNG4s4PxBN9JUVoMnJZELI6Kof2IuBtiyajvvzHqfm16+nQVbatj55Ad4ctcDkHzBFDj7BFZ9so4vfvUcGUPSOPua04hxxgDw0ezF/Od3rxI5rD/ebcU8WlAJ2WNwPv02Diyv/2M+jqgIjrzgeBasKeny8VWV1vDWY+/xxt/f5aaXbmXB9joq6pr434Nvsvnlj6mvqef6P10ZkONb/fwClrzwMQVltawedQSV9W7ylm3ky18+S3LfRM79wTeIT4o7oLE6U1/r4s1/zOf5++Zw439uYUGph8p6N4tmLeCrJ+dz7vXfYOwJowI23s4Nhbz92Ht8+ubn3P/hTNIGpOL1evnL9//Ohy8upv/wfnzrxm8GbDwRERER6R2CecbteGCTtXaLtbYJeAm4MIjj9bh1RdXMWrQVV5OXFGcU1S43sxZtZV1R9QH3kznjPBImHclHf3oVb9YI6JuEu7gSay31J2dR/8TbWCznP3wdDy+6D7fXx58v/CO1O3bh7JPQ1me9w0FsfjGf//JZTEoCM178WVtoAzj+xmkkfGMS3m3FNPXrg3PJKmL++So+jwefhQaPj2/964e89FVFt4+vT79kHvxoJj4Df/n2X6jdVgLvLmXzyx+TeuYkpvz4vIAd36ppJ3LUOcfy9h9fo+ytT3EWl7HsrqfxxMZw40u3BTS0AcQlOnngo5nEpsTz8CUPUrN+J2bhF3z15HySp4zlrF9dEtDxho4dzB/n30N5YSV3nDGTXfllbaHtuj9codAmIiIicpgKZnAbCOx57dzOlrZDxvy8EpKdDpzRkRhjSHY6SHY6mJ9X0v2d99FPhCMKxzXnwtHD8bz2CZTXYCMjiPD6SH7jEwAirjmPxTFJDB07mAm/vwZjofzeZ6l9fzkxpx4DRw/H/eoiSv40G2d6CuN/dzVLSxvbjfneml14LzgZz5QsondVNvfr9hDhasIXYcj6/dW8Xdzo9+MbNGoAE/5wLRGRhvLf/Juad5Yx5LzjGf3D81mwtjRgxzclIYaS804hbWoWu19fTPFvnyc6KY7xv7uaz6s8BzROd/oNTmP8764mOjGOij+9SNWcj8k8bTxH3/4d3v+qLODjHX3iUfxx/j2U5pfz3aE3tYW2y35+UcDHEhEREZHeIZjBzXTS1mHClDFmhjEm1xiTW1p6YG/wQ6WgykVibPurTRNjoyioch1UP7UeH/FnTGzb9owe1va9iYslaeIRlNQ0AFCdlEDKqK/zsGfKOOK/Maltu//JR9N3QJ8ONRVUuWjy+uDEcR0LGpxBv5H9Kalp2K/HVxUfR+rRQ9u2h15wAklx0ft9PPassbPxd9U1ccRFJ7a1pWcfSfqQtAMexx8V0TFknDC6bXvYhVNIio8J2pijJx9Bn/4pbdtnXnVqUMYRERERkd4hmMFtJzB4j+1BQOHeO1lrZ1lrs6212enp6UEsJ/AGpjipbWh/lqe2wcPAFOdB9ePML6Hu8bexqUm4U5NwrNmCBbyx0di6Bqr//CL9EmKw1lI752NKczYQO3YoEUlx+P72KnWPvYlNTSJm5AA2z17Ihtc+7VDTwBQnjvIaIme9idcZjY1s/lXwJsVj12wl7/F36ZcY4/fjs9ZS9/piShavIXbMECKT41l211OUbCjc7+Oxr+PSOn7fyipyfvkskamJxBw5iB1zP2fdfz464HH80fh+LtvfXErMkQOJ7JtEzj3/pnDVtqCM2TqnrXjrLqZckI0zIZY7zpjZYcESERERETl8BDO45QBHGmOGG2OigcuAt4I4Xo+bNi6DapcbV5MXay3VLjfVLjfTxmUccD+ur3aw+x+v40uOJ+a8yTgqa7A0n76sP3UiZPaFnaVE/30Oj9/+HFtfXUzq2dkk/eQ79LnkNGxdA9brJfbKs8i46zJSJ49my6x5xOWuaTfmxFhL099fxXq8GAy+2Bjqjh5BZE0dMUP6UfDGp4xctpKq+qZuH5+1lqd/ObttTlvST/6PjLsuwxrDqnv+zTGOA7uEsbPjW5y3nca/v0ZEgpM+t19Gxp2XkjY1i+3P/4/IhcsPaJzu/Pf+N9vmtCX/7BL63305UQlO8mY+zxh3XUDH2nMhkuv+cAX3vXFXuzlvCm8iIiIih6egBTdrrQe4GXgPWAf811q7put79S5jMpOZMXU4zuhIqlwekp0OZkwdvt+rSu7ZT8UHXxLXL4UbZt+KWdp8uDJv/jaNRw3BmfsVSXdeRtqRAyj9qoD3nv2Ib998Dvc+dQNxMVHUrNiCs38f4pLjid1aSLXbcvyvLmf8tIkse24hja6v57ltW/Al8Y4InJNGYSMiqL3mfMzlZxFxyjH4qnZz/PQT2TJ/OZeP6dvt46sqrWHBsx9x/g1n8etnbiQuNoq65CSmPPAD4p1RbH73i4Ad34x1m0lNT+KXb99NQmYfqpt8ZP98OsdeNJnPX/yEupr6AxprX+prXbz9rwWcdtlJzHzpp8Q5Hex2xjHlgevpk57E2teXBXS8nesL+fTNnHZz2lrnvJUXVrL0zZyAjiciIiIivYPx53O6ekp2drbNzc0NdRn7bU5u8xos07MHd7Nn9/14mzxMOzKVlPRkXvpsG5Wbirjpyinc/tIXeOsaePi6E/F4PGxdtYM+Gcn0HZCKMab5vm4PZ49Mxev28GFhHcYYpmcPxt3kZndVPX36fR24rLWUF1bwYcFulq7I58RJQ9vazxgQT98BqZQXVpA2sK9fj6+ssILU/ilERES023/P9oM5Lq39eT1eqstqSO3fZ5/tgVZRXElyWhKRUZHtxqwsqSKpbyKRUZEBHa+soJy0gX39bhcRERGRQ4cxZrm1Nnvv9qB9jpscmMjoKFLSmwNWZFQkaaMHARARFUlEcjwAUVFRHDlpRMf7OqLawpkp+vrMkyPa0S60ARhjSBvYF1NUjyM1sUM7sF8hIW1A6n61H6jIqMhOw9m+2gNhX/32yUjptP1g7eu4K7SJiIiIHL6COcdNREREREREAkDBTUREREREJMwpuImIiIiIiIS5/2/v3mPlKOswjn+ftGC5FBXFiLRS/wCVVEBz0CKgBBCLNBjRBjESEo0a4wWIRkX+MtGkSb2LoggNCgTiDTUYrVUxSBVpgdIUWwyRQBsb6yUGkABCf/6xA6nNKZy2ZzrT2e8nOcmZ3dmd5+ybnrPPzjtvLW6SJEmS1HMWN0mSJEnqOYubJEmSJPWcxU2SJEmSes7iJkmSJEk9Z3GTJEmSpJ6zuEmSJElSz1ncJEmSJKnnLG6SJEmS1HMzuw6g4Vk8MbfrCK0Z8s8mSZKk/rK4TYM98Wb+uHkH79T+U820eGJu78tI3/NJkiRJbUtVdZ3haRMTE7V69equY/TS91dvBCwxkiRJ0pAlub2qJra/3TNuewkLmyRJkjS+XJxEkiRJknrO4iZJkiRJPWdxkyRJkqSes7hJkiRJUs9Z3CRJkiSp5yxukiRJktRzFjdJkiRJ6jmLmyRJkiT1nMVNkiRJknrO4iZJkiRJPWdxkyRJkqSeS1V1neFpSf4O3L+TD3sh8I8W4qh/HOvx4ViPF8d7fDjW48OxHh+O9fQ7vKoO2f7GXhW3XZFkdVVNdJ1D7XOsx4djPV4c7/HhWI8Px3p8ONZ7jlMlJUmSJKnnLG6SJEmS1HNDKG6Xdx1Ae4xjPT4c6/HieI8Px3p8ONbjw7HeQ/b6a9wkSZIkaeiGcMZNkiRJkgZtEMUtydIkG5KsTXJDkud1nUntSLI4yd1JtiZxBaMBSrIwyT1J7k3yqa7zqD1JliXZkmRd11nUniRzk9yUZH3z+/uCrjOpPUlmJbktyV3NeH+m60xqV5IZSe5McmPXWYZuEMUNWAHMr6qjgT8DF3ecR+1ZB5wN3Nx1EE2/JDOArwNnAEcB5yY5qttUatFVwMKuQ6h1TwAfq6pXAguAD/nvetAeA06pqmOAY4GFSRZ0nEntugBY33WIcTCI4lZVv6yqJ5rNW4E5XeZRe6pqfVXd03UOtea1wL1V9Zeqehy4Hnhrx5nUkqq6GfhX1znUrqraXFV3NN8/xOgN3mHdplJbauThZnOf5ssFFQYqyRzgTOCKrrOMg0EUt+28B/h51yEk7ZLDgI3bbG/CN3jSYCSZB7wa+GO3SdSmZurcGmALsKKqHO/h+jLwCWBr10HGwcyuA0xVkl8BL57krkuq6ifNPpcwmpJx7Z7Mpuk1lbHWYGWS2/ykVhqAJAcCPwQurKoHu86j9lTVk8CxzZoDNySZX1VeyzowSRYBW6rq9iQnd51nHOw1xa2qTnum+5OcDywCTi3/j4O92rONtQZtEzB3m+05wF87yiJpmiTZh1Fpu7aqftR1Hu0ZVfXvJL9ldC2rxW14TgDOSvIWYBZwUJJrqurd2iBazwAAA89JREFUHecarEFMlUyyEPgkcFZVPdJ1Hkm7bBVwRJKXJdkXeCfw044zSdoNSQJcCayvqi92nUftSnLIU6t7J9kPOA3Y0G0qtaGqLq6qOVU1j9Hf699Y2to1iOIGXArMBlYkWZPkm10HUjuSvC3JJuB44GdJlnedSdOnWWTow8ByRgsYfK+q7u42ldqS5DrgD8DLk2xK8t6uM6kVJwDnAac0f6PXNJ/Qa5gOBW5KspbRh3Erqspl4qVpEGcVSpIkSVK/DeWMmyRJkiQNlsVNkiRJknrO4iZJkiRJPWdxkyRJkqSes7hJkiRJUs9Z3CRJkiSp5yxukqReSzI3yX1JDm62n99sHz7JvvOSvGs3jvXp3ckqSVJbLG6SpF6rqo3AZcCS5qYlwOVVdf8ku88Ddrm4ARY3SVIvWdwkSXuDLwELklwInAh8YQf7LQFOSrImyUVJZiRZmmRVkrVJPgCQ5NAkNzf7rUtyUpIlwH7NbddO9uTNGb0NSb7TPN8Pkuzf3Hdckt8nuSvJbUlmN/v/Lskdzdfrp/+lkSSNg1RV1xkkSXpWSd4M/AI4vapW7GCfk4GPV9WiZvv9wIuq6rNJngOsBBYDZwOzqupzSWYA+1fVQ0kerqoDnyHDPOA+4MSqWplkGfAn4KvABuCcqlqV5CDgEWBfYGtVPZrkCOC6qprY/VdDkjRuZnYdQJKkKToD2AzMByYtbpM4HTg6yTua7ecCRwCrgGVJ9gF+XFVrdiLHxqpa2Xx/DfBRYDmwuapWAVTVgwBJDgAuTXIs8CRw5E4cR5Kkp1ncJEm91xSfNwELgFuSXF9Vm6fyUOAjVbV8kud8A3AmcHWSpVX13SnG2X6qSjXHmWwKy0XA34BjGF2e8OgUjyFJ0v/xGjdJUq8lCaPFSS6sqgeApcDnd7D7Q8DsbbaXAx9szqyR5MgkBzQrUm6pqm8DVwKvafb/71P7PoOXJjm++f5c4BZG0yRfkuS45jizk8xkdIZvc1VtBc4DZkz5B5ckaRsWN0lS370PeGCb69q+AbwiyRsn2Xct8ESzQMhFwBWMrkG7I8k64FuMZpucDKxJcifwduArzeMvB9buaHGSxnrg/CRrgYOBy6rqceAc4GtJ7mI0lXNWk/X8JLcymib5n116BSRJY8/FSSRJmqJmcZIbq2p+x1EkSWPGM26SJEmS1HOecZMk7XWSvAq4erubH6uq103T878A+PUkd51aVf+cjmNIkrQzLG6SJEmS1HNOlZQkSZKknrO4SZIkSVLPWdwkSZIkqecsbpIkSZLUcxY3SZIkSeq5/wE9ZhMOvIGaJQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.figure(figsize=(15, 5))\n", + "plt.scatter(X_test_pca, y_test,\n", + " marker='x', s=50, c=y_test, zorder=10)\n", + "\n", + "plt.errorbar(X_test_pca, y_preds, yerr=y_sd, fmt='o', capthick=1, label='Predidtion', alpha=0.5)\n", + "plt.legend(loc='upper right')\n", + "plt.ylabel('6 Classes of Glasses')\n", + "plt.xlabel('X_test_pca')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "JupyterPy2", + "language": "python", + "name": "ipykernel_py2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}