hw4_v6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Distributed Deep Q-Learning \n",
    "\n",
    "The goal of this assignment is to implement and experiment with both single-core and distributed versions of the deep reinforcement learning algorithm Deep Q Networks (DQN). In particular, DQN will be run in the classic RL benchmark Cart-Pole and abblation experiments will be run to observe the impact of the different DQN components. \n",
    "\n",
    "The relevant content about DQN can be found Q-Learning and SARSA are in the following course notes from CS533.\n",
    "\n",
    "https://oregonstate.instructure.com/courses/1719746/files/75047394/download?wrap=1\n",
    "\n",
    "The full pseudo-code for DQN is on slide 45 with prior slides introducing the individual components. \n",
    "\n",
    "\n",
    "## Recap of DQN \n",
    "\n",
    "From the course slides it can be seen that DQN is simply the standard table-based Q-learning algorithm but with three extensions:\n",
    "\n",
    "1) Use of function approximation via a neural network instead of a Q-table. \n",
    "2) Use of experience replay. \n",
    "3) Use of a target network. \n",
    "\n",
    "Extension (1) allows for scaling to problems with enormous state spaces, such as when the states correspond to images or sequences of images. Extensions (2) and (3) are claimed to improve the robustness and effectiveness of DQN compared. \n",
    "\n",
    "(2) adjusts Q-learning so that updates are not just performed on individual experiences as they arrive. But rather, experiences are stored in a memory buffer and updates are performed by sampling random mini-batches of experience tuples from the memory buffer and updating the network based on the mini-batch. This allows for reuse of experience as well as helping to reduce correlation between successive updates, which is claimed to be beneficial. \n",
    "\n",
    "(3) adjusts the way that target values are computed for the Q-learning updates. Let $Q_{\\theta}(s,a)$ be the function approximation network with parameters $\\theta$ for representing the Q-function. Given an experience tuple $(s, a, r, s')$ the origional Q-learning algorithm updates the parameters so that $Q_{\\theta}(s,a)$ moves closer to the target value: \n",
    "\\begin{equation}\n",
    "r + \\beta \\max_a' Q_{\\theta}(s',a') \n",
    "\\end{equation}\n",
    "Rather, DQN stores two function approximation networks. The first is the update network with parameters $\\theta$, which is the network that is continually updated during learning. The second is a target network with parameters $\\theta'$. Given the same experience tuple, DQN will update the parameters $\\theta$ so that $Q_{\\theta}(s,a)$ moves toward a target value based on the target network:\n",
    "\\begin{equation}\n",
    "r + \\beta \\max_a' Q_{\\theta'}(s',a') \n",
    "\\end{equation}\n",
    "Periodically the target network is updated with the most recent parameters $\\theta' \\leftarrow \\theta$. This use of a target network is claimed to stabilize learning.\n",
    "\n",
    "In the assignment you will get to see an example of the impact of both the target network and experience replay.\n",
    "\n",
    "Further reading about DQN and its application to learning to play Atari games can be found in the following paper. \n",
    "\n",
    "Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G. and Petersen, S., 2015. Human-level control through deep reinforcement learning. Nature, 518(7540), p.529.\n",
    "https://oregonstate.instructure.com/courses/1719746/files/75234294/download?wrap=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip3 install --user gym[Box2D]\n",
    "!pip3 install --user torch\n",
    "!pip3 install --user JSAnimation\n",
    "!pip3 install --user matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install the packages for enviroment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gym\n",
    "import torch\n",
    "import time\n",
    "import os\n",
    "import ray\n",
    "import numpy as np\n",
    "\n",
    "from tqdm import tqdm\n",
    "from random import uniform, randint\n",
    "\n",
    "import io\n",
    "import base64\n",
    "from IPython.display import HTML\n",
    "\n",
    "from dqn_model import DQNModel\n",
    "from dqn_model import _DQNModel\n",
    "from memory import ReplayBuffer\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "FloatTensor = torch.FloatTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Useful PyTorch functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensors\n",
    "\n",
    "This assignment will use the PyTorch library for the required neural network functionality. You do not need to be familiar with the details of PyTorch or neural network training. However, the assignment will require dealing with data in the form of tensors.  \n",
    "\n",
    "The mini-batches used to train the PyTorch neural network is expected to be represented as a tensor matrix. The function `FloatTensor` can convert a list or NumPy matrix into a tensor matrix if needed. \n",
    "\n",
    "You can find more infomation here: https://pytorch.org/docs/stable/tensors.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[3, 2, 1], [6, 4, 5], [7, 8, 9]]\n",
      "<class 'torch.Tensor'>\n",
      "tensor([[3., 2., 1.],\n",
      "        [6., 4., 5.],\n",
      "        [7., 8., 9.]])\n"
     ]
    }
   ],
   "source": [
    "# list\n",
    "m = [[3,2,1],[6,4,5],[7,8,9]]\n",
    "print(m)\n",
    "\n",
    "# tensor matrix\n",
    "m_tensor = FloatTensor(m)\n",
    "print(type(m_tensor))\n",
    "print(m_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensor.max()\n",
    "Once you have a tenosr maxtrix, you can use torch.max(m_tensor, dim) to get the max value and max index corresponding to the dimension you choose.\n",
    "```\n",
    ">>> a = torch.randn(4, 4)\n",
    ">>> a\n",
    "tensor([[-1.2360, -0.2942, -0.1222,  0.8475],\n",
    "        [ 1.1949, -1.1127, -2.2379, -0.6702],\n",
    "        [ 1.5717, -0.9207,  0.1297, -1.8768],\n",
    "        [-0.6172,  1.0036, -0.6060, -0.2432]])\n",
    ">>> torch.max(a, 1)\n",
    "torch.return_types.max(values=tensor([0.8475, 1.1949, 1.5717, 1.0036]), indices=tensor([3, 0, 0, 1]))\n",
    "```\n",
    "You can find more infomation here: https://pytorch.org/docs/stable/torch.html#torch.max"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([3., 6., 9.]) tensor([0, 0, 2])\n"
     ]
    }
   ],
   "source": [
    "max_value, index = torch.max(m_tensor, dim = 1)\n",
    "print(max_value, index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize Environment\n",
    "### CartPole-v0:  \n",
    "CartPole is a classic control task that is often used as an introductory reinforcement learning benchmark. The environment involves controlling a 2d cart that can move in either the left or right direction on a frictionless track. A pole is attached to the cart via an unactuated joint. The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. A reward of +1 is provided for every timestep that the pole remains upright. The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center.  \n",
    "(You can find more infomation by this Link: https://gym.openai.com/envs/CartPole-v0/)  \n",
    "  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the Env name and action space for CartPole\n",
    "ENV_NAME = 'CartPole-v0'\n",
    "# Move left, Move right\n",
    "ACTION_DICT = {\n",
    "    \"LEFT\": 0,\n",
    "    \"RIGHT\":1\n",
    "}\n",
    "# Register the environment\n",
    "env_CartPole = gym.make(ENV_NAME)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set result saveing floder\n",
    "result_floder = ENV_NAME\n",
    "result_file = ENV_NAME + \"/results.txt\"\n",
    "if not os.path.isdir(result_floder):\n",
    "    os.mkdir(result_floder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper Function\n",
    "Plot results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_result(total_rewards ,learning_num, legend):\n",
    "    print(\"\\nLearning Performance:\\n\")\n",
    "    episodes = []\n",
    "    for i in range(len(total_rewards)):\n",
    "        episodes.append(i * learning_num + 1)\n",
    "        \n",
    "    plt.figure(num = 1)\n",
    "    fig, ax = plt.subplots()\n",
    "    plt.plot(episodes, total_rewards)\n",
    "    plt.title('performance')\n",
    "    plt.legend(legend)\n",
    "    plt.xlabel(\"Episodes\")\n",
    "    plt.ylabel(\"total rewards\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparams\n",
    "When function approximation is involves, especially neural networks, additional hyper parameters are inroduced and setting the parameters can require experience. Below is a list of the hyperparameters used in this assignment and values for the parameters that have worked well for a basic DQN implementation. You will adjust these values for particular parts of the assignment. For example, experiments that do not use the target network will set 'use_target_model' to False. \n",
    "\n",
    "You can find the more infomation about these hyperparameters in the notation of DQN_agent.init() function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "hyperparams_CartPole = {\n",
    "    'epsilon_decay_steps' : 100000, \n",
    "    'final_epsilon' : 0.1,\n",
    "    'batch_size' : 32, \n",
    "    'update_steps' : 10, \n",
    "    'memory_size' : 2000, \n",
    "    'beta' : 0.99, \n",
    "    'model_replace_freq' : 2000,\n",
    "    'learning_rate' : 0.0003,\n",
    "    'use_target_model': True\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "# Part 1: Non-distributed DQN\n",
    "\n",
    "In this part, you will complete an implementation of DQN and run experiments on the CartPole environment from OpenAI Gym.  \n",
    "Note that OpenAI Gym has many other environments that use the same interface---so this experience will allow the curious student to easily explore these algorithms more widely. \n",
    "\n",
    "Below you need to fill in the missing code for the DQN implementation. \n",
    "\n",
    "The Run function below can then be used to generate learning curves. \n",
    "\n",
    "You should conduct the following experiments involving different features of DQN. \n",
    "\n",
    "1. DQN without a replay buffer and without a target network. This is just standard Q-learning with a function approximator.\n",
    "    The corresponding parameters are: memory_size = 1, update_steps = 1, batch_size = 1, use_target_model = False  \n",
    "    \n",
    "2. DQN without a replay buffer (but including the target network).   \n",
    "    The corresponding parameters are: memory_size = 1, update_steps = 1, batch_size = 1, use_target_model = True  \n",
    "\n",
    "3. DQN with a replay buffer, but without a target network.   \n",
    "    Here you set use_target_model = False and otherwise set the replay memory parameters to the above suggested values \n",
    "   \n",
    "4. Full DQN\n",
    "\n",
    "For each experiment, record the parameters that you used, plot the resulting learning curves, and give a summary of your observations regarding the differences you observed. \n",
    "***\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## DQN Agent\n",
    "\n",
    "The full DQN agent involves a number of functions, the neural network, and the replay memory. Interfaces to a neural network model and memory are provided. \n",
    "\n",
    "Some useful information is below:   \n",
    "- Neural Network Model: The network is used to represent the Q-function $Q(s,a)$. It takes a state $s$ as input and returns a vector of Q-values, one value for each action. The following interface functions are used for predicting Q-values, actions, and updating the neural network model parameters. \n",
    "    1. Model.predict(state) --- Returns the action that has the best Q-value in 'state'.\n",
    "    2. Model.predict_batch(states) --- This is used to predict both the Q-values and best actions for a batch of states. Given a batch of states, the function returns: 1) 'best_actions' a vector containing the best action for each input state, and 2) 'q_values' a matrix where each row gives the Q-value for all actions of each state (one row per state).   \n",
    "    3. Model.fit(q_values, q_target) --- It is used to update the neural network (via back-propagation). 'q_values' is a vector containing the Q-value predictions for a list of state-action pairs (e.g. from a batch of experience tuples). 'q_target' is a vector containing target values that we would like the correspoinding predictions to get closer to. This function updates the network in a way that the network predictions will ideally be closer to the targets. There is no return value.  \n",
    "    4. Model.replace(another_model) --- It takes another model as input, and replace the weight of itself by the input model.\n",
    "- Memory: This is the buffer used to store experience tuples for experience replay.\n",
    "    1. Memory.add(state, action, reward, state', is_terminal) --- It takes one example as input, and store it into its storage.  \n",
    "    2. Memory.sample(batch_size) --- It takes a batch_size int number as input. Return 'batch_size' number of randomly selected examples from the current memory buffer. The batch takes the form (states, actions, rewards, states', is_terminals) with each component being a vector/list of size equal to batch_size. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DQN_agent(object):\n",
    "    def __init__(self, env, hyper_params, action_space = len(ACTION_DICT)):\n",
    "        \n",
    "        self.env = env\n",
    "        self.max_episode_steps = env._max_episode_steps\n",
    "        \n",
    "        \"\"\"\n",
    "            beta: The discounted factor of Q-value function\n",
    "            (epsilon): The explore or exploit policy epsilon. \n",
    "            initial_epsilon: When the 'steps' is 0, the epsilon is initial_epsilon, 1\n",
    "            final_epsilon: After the number of 'steps' reach 'epsilon_decay_steps', \n",
    "                The epsilon set to the 'final_epsilon' determinately.\n",
    "            epsilon_decay_steps: The epsilon will decrease linearly along with the steps from 0 to 'epsilon_decay_steps'.\n",
    "        \"\"\"\n",
    "        self.beta = hyper_params['beta']\n",
    "        self.initial_epsilon = 1\n",
    "        self.final_epsilon = hyper_params['final_epsilon']\n",
    "        self.epsilon_decay_steps = hyper_params['epsilon_decay_steps']\n",
    "\n",
    "        \"\"\"\n",
    "            episode: Record training episode\n",
    "            steps: Add 1 when predicting an action\n",
    "            learning: The trigger of agent learning. It is on while training agent. It is off while testing agent.\n",
    "            action_space: The action space of the current environment, e.g 2.\n",
    "        \"\"\"\n",
    "        self.episode = 0\n",
    "        self.steps = 0\n",
    "        self.best_reward = 0\n",
    "        self.learning = True\n",
    "        self.action_space = action_space\n",
    "\n",
    "        \"\"\"\n",
    "            input_len The input length of the neural network. It equals to the length of the state vector.\n",
    "            output_len: The output length of the neural network. It is equal to the action space.\n",
    "            eval_model: The model for predicting action for the agent.\n",
    "            target_model: The model for calculating Q-value of next_state to update 'eval_model'.\n",
    "            use_target_model: Trigger for turn 'target_model' on/off\n",
    "        \"\"\"\n",
    "        state = env.reset()\n",
    "        input_len = len(state)\n",
    "        output_len = action_space\n",
    "        self.eval_model = DQNModel(input_len, output_len, learning_rate = hyper_params['learning_rate'])\n",
    "        self.use_target_model = hyper_params['use_target_model']\n",
    "        if self.use_target_model:\n",
    "            self.target_model = DQNModel(input_len, output_len)\n",
    "#         memory: Store and sample experience replay.\n",
    "        self.memory = ReplayBuffer(hyper_params['memory_size'])\n",
    "        \n",
    "        \"\"\"\n",
    "            batch_size: Mini batch size for training model.\n",
    "            update_steps: The frequence of traning model\n",
    "            model_replace_freq: The frequence of replacing 'target_model' by 'eval_model'\n",
    "        \"\"\"\n",
    "        self.batch_size = hyper_params['batch_size']\n",
    "        self.update_steps = hyper_params['update_steps']\n",
    "        self.model_replace_freq = hyper_params['model_replace_freq']\n",
    "        \n",
    "    # Linear decrease function for epsilon\n",
    "    def linear_decrease(self, initial_value, final_value, curr_steps, final_decay_steps):\n",
    "        decay_rate = curr_steps / final_decay_steps\n",
    "        if decay_rate > 1:\n",
    "            decay_rate = 1\n",
    "        return initial_value - (initial_value - final_value) * decay_rate\n",
    "    \n",
    "    def explore_or_exploit_policy(self, state):\n",
    "        p = uniform(0, 1)\n",
    "        # Get decreased epsilon\n",
    "        epsilon = self.linear_decrease(self.initial_epsilon, \n",
    "                               self.final_epsilon,\n",
    "                               self.steps,\n",
    "                               self.epsilon_decay_steps)\n",
    "        \n",
    "        if p < epsilon:\n",
    "            #return action\n",
    "            return randint(0, self.action_space - 1)\n",
    "        else:\n",
    "            #return action\n",
    "            return self.greedy_policy(state)\n",
    "        \n",
    "    def greedy_policy(self, state):\n",
    "        return self.eval_model.predict(state)\n",
    "    \n",
    "    # This next function will be called in the main RL loop to update the neural network model given a batch of experience\n",
    "    # 1) Sample a 'batch_size' batch of experiences from the memory.\n",
    "    # 2) Predict the Q-value from the 'eval_model' based on (states, actions)\n",
    "    # 3) Predict the Q-value from the 'target_model' base on (next_states), and take the max of each Q-value vector, Q_max\n",
    "    # 4) If is_terminal == 1, q_target = reward, otherwise q_target = reward + discounted factor * Q_max\n",
    "    # 5) Call fit() to do the back-propagation for 'eval_model'.\n",
    "    def update_batch(self):\n",
    "        if len(self.memory) < self.batch_size or self.steps % self.update_steps != 0:\n",
    "            return\n",
    "\n",
    "        batch = self.memory.sample(self.batch_size)\n",
    "\n",
    "        (states, actions, reward, next_states,\n",
    "         is_terminal) = batch\n",
    "        \n",
    "        states = states\n",
    "        next_states = next_states\n",
    "        terminal = FloatTensor([1 if t else 0 for t in is_terminal])\n",
    "        reward = FloatTensor(reward)\n",
    "        batch_index = torch.arange(self.batch_size,\n",
    "                                   dtype=torch.long)\n",
    "        \n",
    "        # Current Q Values\n",
    "        _, q_values = self.eval_model.predict_batch(states)\n",
    "        q_values = q_values[batch_index, actions]\n",
    "        \n",
    "        # Calculate target\n",
    "        if self.use_target_model:\n",
    "            actions, q_next = self.target_model.predict_batch(next_states)\n",
    "        else:\n",
    "            actions, q_next = self.eval_model.predict_batch(next_states)\n",
    "            \n",
    "        #INSERT YOUR CODE HERE --- neet to compute 'q_targets' used below\n",
    "        max_value,index= torch.max(q_next,dim=1)\n",
    "        \n",
    "        q_target = []\n",
    "        for i in range(len(terminal)):\n",
    "            if terminal[i]==1:\n",
    "                q_target.append(reward[i])\n",
    "            else:\n",
    "                q_target.append(reward[i]+ self.beta*max_value[i])\n",
    "        # update model\n",
    "        q_target = FloatTensor(q_target)\n",
    "        self.eval_model.fit(q_values, q_target)\n",
    "    \n",
    "    def learn_and_evaluate(self, training_episodes, test_interval):\n",
    "        test_number = training_episodes // test_interval\n",
    "        all_results = []\n",
    "        \n",
    "        for i in range(test_number):\n",
    "            # learn\n",
    "            self.learn(test_interval)\n",
    "            \n",
    "            # evaluate\n",
    "            avg_reward = self.evaluate()\n",
    "            all_results.append(avg_reward)\n",
    "            \n",
    "        return all_results\n",
    "    \n",
    "    def learn(self, test_interval):\n",
    "        for episode in tqdm(range(test_interval), desc=\"Training\"):\n",
    "            state = self.env.reset()\n",
    "            done = False\n",
    "            steps = 0\n",
    "            self.episode =  self.episode + 1\n",
    "            while steps < self.max_episode_steps and not done:\n",
    "                #INSERT YOUR CODE HERE\n",
    "                # add experience from explore-exploit policy to memory\n",
    "                # update the model every 'update_steps' of experience\n",
    "                # update the target network (if the target network is being used) every 'model_replace_freq' of experiences\n",
    "                \n",
    "                \n",
    "                action = self.explore_or_exploit_policy(state)\n",
    "                observation,reward, done, info = self.env.step(action)\n",
    "                self.memory.add(state,action,reward,observation,done)\n",
    "                \n",
    "                steps = steps + 1\n",
    "                self.steps = self.steps + 1\n",
    "                if (self.steps%self.update_steps)==0:\n",
    "                    self.update_batch()\n",
    "                    \n",
    "                    \n",
    "                if (self.steps%self.model_replace_freq)==0:\n",
    "                    self.target_model.replace(self.eval_model)\n",
    "                    \n",
    "                \n",
    "                \n",
    "                state = observation\n",
    "                \n",
    "                \n",
    "    def evaluate(self, trials = 30):\n",
    "        total_reward = 0\n",
    "        for _ in tqdm(range(trials), desc=\"Evaluating\"):\n",
    "            state = self.env.reset()\n",
    "            done = False\n",
    "            steps = 0\n",
    "\n",
    "            while steps < self.max_episode_steps and not done:\n",
    "                steps += 1\n",
    "                action = self.greedy_policy(state)\n",
    "                state, reward, done, _ = self.env.step(action)\n",
    "                total_reward += reward\n",
    "\n",
    "        avg_reward = total_reward / trials\n",
    "        print(avg_reward)\n",
    "        f = open(result_file, \"a+\")\n",
    "        f.write(str(avg_reward) + \"\\n\")\n",
    "        f.close()\n",
    "        if avg_reward >= self.best_reward:\n",
    "            self.best_reward = avg_reward\n",
    "            self.save_model()\n",
    "        return avg_reward\n",
    "\n",
    "    # save model\n",
    "    def save_model(self):\n",
    "        self.eval_model.save(result_floder + '/best_model.pt')\n",
    "        \n",
    "    # load model\n",
    "    def load_model(self):\n",
    "        self.eval_model.load(result_floder + '/best_model.pt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 121.11it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 577.18it/s]\n",
      "Training:  24%|██▍       | 12/50 [00:00<00:00, 119.25it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.2\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 147.70it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 539.01it/s]\n",
      "Training:  30%|███       | 15/50 [00:00<00:00, 146.77it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.366666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 130.99it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 544.06it/s]\n",
      "Training:  26%|██▌       | 13/50 [00:00<00:00, 125.07it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.533333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 138.18it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 547.90it/s]\n",
      "Training:  26%|██▌       | 13/50 [00:00<00:00, 127.10it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.633333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 138.38it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 356.69it/s]\n",
      "Training:  24%|██▍       | 12/50 [00:00<00:00, 118.53it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14.7\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 133.34it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 271.79it/s]\n",
      "Training:  22%|██▏       | 11/50 [00:00<00:00, 94.73it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19.133333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 93.37it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 352.53it/s]\n",
      "Training:  14%|█▍        | 7/50 [00:00<00:00, 65.56it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11.6\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 103.83it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 407.80it/s]\n",
      "Training:  22%|██▏       | 11/50 [00:00<00:00, 101.71it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.766666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 92.65it/s] \n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 190.95it/s]\n",
      "Training:  16%|█▌        | 8/50 [00:00<00:00, 71.36it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21.2\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 74.53it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 26.02it/s]\n",
      "Training:   8%|▊         | 4/50 [00:00<00:01, 39.99it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "187.6\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:01<00:00, 47.15it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 63.16it/s]\n",
      "Training:  12%|█▏        | 6/50 [00:00<00:00, 57.56it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "98.9\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 80.95it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 104.81it/s]\n",
      "Training:  14%|█▍        | 7/50 [00:00<00:00, 66.34it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "55.766666666666666\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 86.77it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 53.29it/s]\n",
      "Training:   8%|▊         | 4/50 [00:00<00:01, 36.62it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "110.56666666666666\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 85.75it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 48.16it/s]\n",
      "Training:  18%|█▊        | 9/50 [00:00<00:00, 89.15it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "127.33333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 80.02it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.93it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:01, 27.57it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 51.61it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.33it/s]\n",
      "Training:  10%|█         | 5/50 [00:00<00:00, 45.69it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "199.53333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 73.12it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.53it/s]\n",
      "Training:  16%|█▌        | 8/50 [00:00<00:00, 75.42it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "196.53333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 72.78it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.94it/s]\n",
      "Training:  14%|█▍        | 7/50 [00:00<00:00, 64.40it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 67.16it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.54it/s]\n",
      "Training:   8%|▊         | 4/50 [00:00<00:01, 29.51it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:01<00:00, 44.81it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.02it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:01, 27.30it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:01<00:00, 37.33it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.29it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:01, 24.95it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:01<00:00, 34.13it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.31it/s]\n",
      "Training:   8%|▊         | 4/50 [00:00<00:01, 36.76it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:01<00:00, 37.41it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 32.42it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:02, 21.99it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:01<00:00, 23.04it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.95it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 14.67it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:01<00:00, 27.65it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.72it/s]\n",
      "Training:   8%|▊         | 4/50 [00:00<00:01, 23.62it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 24.35it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.04it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.68it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 24.09it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.79it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:02, 18.88it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 14.81it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.54it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.51it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 12.24it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.41it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  6.07it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.19it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.87it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.50it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.09it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 38.83it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "154.23333333333332\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 14.63it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 51.59it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:02, 18.35it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "114.23333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 15.34it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 244.56it/s]\n",
      "Training:  20%|██        | 10/50 [00:00<00:00, 98.99it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21.366666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:00<00:00, 104.83it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 235.45it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:01, 27.43it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22.233333333333334\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 16.71it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 45.34it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.75it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "134.1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 19.77it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 369.96it/s]\n",
      "Training:  18%|█▊        | 9/50 [00:00<00:00, 87.35it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13.5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 12.75it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 58.22it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:02, 16.69it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "107.53333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 15.56it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.79it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.61it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "199.86666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 17.34it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 64.38it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 10.39it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "97.26666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.94it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 41.30it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 15.33it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "150.23333333333332\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 16.76it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 38.54it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.41it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "170.53333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 12.09it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 34.20it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  5.48it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "172.1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 19.83it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 50.05it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.11it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "129.93333333333334\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 15.92it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.14it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.28it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "197.56666666666666\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 16.14it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 41.95it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 11.40it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "121.16666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.29it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 42.72it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 12.37it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "140.93333333333334\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 12.42it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 50.93it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 11.37it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "115.9\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 11.29it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 38.14it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 10.93it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "157.83333333333334\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.66it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 42.68it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 13.43it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "146.63333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 13.42it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 42.90it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 12.81it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "140.23333333333332\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 16.38it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 42.65it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 14.98it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "147.46666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.64it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 49.40it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 10.75it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "119.33333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 12.43it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.12it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.29it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.52it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.92it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 12.48it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 14.16it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 53.31it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:01, 27.16it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "111.56666666666666\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.19it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 43.70it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 11.76it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "131.8\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.06it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 46.74it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:02, 16.26it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "127.1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.20it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.81it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 12.96it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "199.03333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 17.85it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 285.63it/s]\n",
      "Training:  20%|██        | 10/50 [00:00<00:00, 92.09it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17.833333333333332\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 12.57it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.76it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  5.50it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.43it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.46it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.64it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.80it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.19it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.93it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.40it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.14it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "198.43333333333334\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.60it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.84it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00,  8.78it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 45.91it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "112.76666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00,  9.65it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 42.96it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  5.68it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "116.16666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.06it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 39.85it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:02, 16.36it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "151.4\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 18.80it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 62.67it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 11.88it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "99.43333333333334\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 16.76it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 74.74it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 14.11it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "83.6\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:02<00:00, 17.20it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 53.62it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 14.40it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "118.5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 15.19it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 35.70it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.84it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "157.46666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.24it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.99it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:03, 13.94it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "149.36666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 12.85it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 36.65it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 11.90it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "163.93333333333334\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.51it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.91it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "198.76666666666668\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 14.75it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 134.92it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:01, 26.07it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "39.766666666666666\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 12.31it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 33.75it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.48it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "187.76666666666668\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.46it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.37it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.96it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.60it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.11it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:04,  9.99it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.53it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.27it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.75it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.47it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.01it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.90it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.04it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.86it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.34it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 32.07it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.95it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.15it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.64it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.36it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00, 10.16it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.03it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 11.85it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.24it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.69it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.73it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.09it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.87it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.80it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.95it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.99it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.84it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.64it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.75it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.07it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.28it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 26.91it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.75it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00,  8.54it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.71it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.53it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.76it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.49it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.29it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.03it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.87it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00,  8.88it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.99it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.38it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.09it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.23it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.03it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.07it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.37it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.15it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.87it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.63it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.43it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.93it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.76it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.11it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.76it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.73it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 13.09it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.82it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.51it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.26it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.97it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.68it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.31it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.92it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:04,  9.85it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 12.01it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.09it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.50it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.31it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.36it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.25it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.38it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.65it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  6.01it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.16it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.51it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:09,  5.29it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.03it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.94it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.33it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.86it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 32.29it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:04,  9.99it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.41it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.08it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:02, 17.51it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.37it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.91it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 12.18it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.02it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.01it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.87it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.08it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.47it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.69it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.85it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.42it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00,  9.45it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.59it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.33it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.71it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.13it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  5.78it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.67it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.72it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.25it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.98it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  8.12it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.18it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.97it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.58it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  8.42it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 27.95it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.46it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.63it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.48it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.70it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.31it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.07it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 10.54it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.26it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 33.09it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.16it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.37it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.98it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.57it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  9.33it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 27.73it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  5.46it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  9.10it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.02it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.03it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 12.63it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.70it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.34it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.25it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.70it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.17it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.62it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.83it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.20it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.67it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.76it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.65it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.60it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.99it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.16it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.83it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.78it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.37it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00,  9.06it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 25.01it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:04, 10.00it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.65it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.92it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.47it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.70it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 27.74it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.58it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00,  8.26it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 27.75it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.08it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00,  7.61it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 25.94it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.72it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.39it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.70it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.05it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.68it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.57it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.23it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.92it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.16it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.01it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.28it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.30it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.20it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 30.57it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  8.09it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.70it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.81it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.19it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.61it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.64it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.49it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.73it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.77it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.39it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.85it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.77it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.36it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.24it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.73it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.14it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.92it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.58it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.59it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.04it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.86it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.53it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.60it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.33it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.68it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:02, 18.53it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.09it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.08it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.74it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.72it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 28.70it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:04,  9.85it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.19it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.63it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.50it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.19it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.72it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 10.62it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.25it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.83it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  8.05it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00,  9.99it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.07it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.76it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.80it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.97it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.30it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.28it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 27.75it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.62it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.92it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.80it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.24it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.27it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.92it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00,  9.95it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 25.32it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.30it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.88it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.78it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.20it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.79it/s]\n",
      "Training:   4%|▍         | 2/50 [00:00<00:04, 10.45it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.86it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.98it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.25it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.52it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.06it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 12.88it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.92it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.87it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.67it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.05it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.56it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.29it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.04it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.30it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 30.70it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  6.06it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.03it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.70it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.23it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.29it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.01it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.77it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.56it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.09it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.56it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.74it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.60it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.59it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.45it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.23it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.38it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.55it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 47.22it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "124.96666666666667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.24it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.88it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.05it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.59it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.98it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.21it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 11.45it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.85it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.57it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.29it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.99it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.46it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.13it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.75it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.10it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.25it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.87it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.40it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.43it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.83it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.69it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.31it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.10it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.88it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 12.39it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 47.61it/s]\n",
      "Training:   6%|▌         | 3/50 [00:00<00:01, 25.05it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "121.2\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:03<00:00, 10.61it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.24it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:08,  5.47it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.31it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.99it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.43it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.26it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.94it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.42it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.33it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 29.03it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.02it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.70it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.83it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.39it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.12it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.87it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.39it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 31.12it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.92it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  9.21it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 23.73it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.53it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00,  9.04it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.99it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.62it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.68it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:00<00:00, 30.97it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  8.90it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.21it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 25.20it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00,  9.99it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 25.02it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:07,  6.66it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:04<00:00, 10.04it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 25.26it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.48it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:06<00:00,  7.63it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 24.96it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:06,  7.88it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:05<00:00, 10.26it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:01<00:00, 25.03it/s]\n",
      "Training:   2%|▏         | 1/50 [00:00<00:05,  9.74it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:11<00:00,  1.71it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:02<00:00, 10.71it/s]\n",
      "Training:   0%|          | 0/50 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training: 100%|██████████| 50/50 [00:29<00:00,  1.65it/s]\n",
      "Evaluating: 100%|██████████| 30/30 [00:02<00:00, 12.24it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200.0\n",
      "\n",
      "Learning Performance:\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsvXm8XHV5+P9+Zrn3Zl9IgEBCAjRB\nyEKAyCarFhHEUlQUKhqoFkHRWlsV6VdBa3/FXbFWxFbQqoCAVEpRQcUiVIWggGGJbCGEhCQkZONu\nM3Oe3x9nmTNzz8w9c++cmXPnPu/Xa1537mfO8jmzfJ7z7KKqGIZhGEY1mXZPwDAMw0gnJiAMwzCM\nSExAGIZhGJGYgDAMwzAiMQFhGIZhRGICwjAMw4jEBIRhhBCRg0TkDyKyS0Q+2O75GEY7ybV7AoaR\nMj4K/EpVD2v3RAyj3ZgGYRiAiPg3S/OBR0d5DMPoCExAGGMeEVkrIh8XkcdE5GURuVZEerzXzhCR\nh0Rku4j8n4gsq9rvYyLyCPCKiPwSOBn4VxHZLSKLRGSaiHxXRLaIyHMi8v9EJOPtf76I3CciXxaR\nbcAVVWPbReQZETnWG39eRDaLyMrQHN7ombR2eq9fEXptgYioiKwUkXUi8pKI/GPo9ayIXCYiT3sm\nsQdFZJ732qtE5C4R2SYia0TkbQl/DEYnoqr2sMeYfgBrgdXAPGAmcB/wGeBwYDNwFJAFVnrbdof2\ne8jbb4I39ivgPaFjfxf4MTAFWAD8CXi399r5QBH4AK65dkJo7ALvnJ8B1gFfB7qB1wO7gMneMU4C\nluLerC0DNgF/6b22AFDgW96xDwUGgIO91z8C/BE4CBDv9T2AScDz3hxy3vvwErC43Z+VPcbWo+0T\nsIc9RvvwFvqLQv+fDjwNfAP4p6pt1wAnhvb766rXAwHhLfADwCGh19+L66PwBcS6qv3PB54M/b/U\nW+T3Co1tBZbXuJavAF/2nvsCYm7o9fuBc0LXcmbEMd4O/Lpq7JvA5e3+rOwxth5mMzU6hedDz58D\n9sH1J6wUkQ+EXuvyXovar5pZ3vbPVR1732H23xR63gegqtVjkwFE5CjgSmCJd65u4Kaq470Yet7r\n74ur+Twdcf75wFEisj00lgP+M2Jbw6iJ+SCMTmFe6Pl+wAbcxfufVXV66DFRVa8PbVuvnPFLQAF3\nwQ0f+4WY+8fhB8BtwDxVnQZcjWsuisPzwIE1xv+36ronq+rFo5yrMc4wAWF0Cu8XkbkiMhO4DLgR\n13Z/kYgcJS6TPKfwlDgHVNUS8EPgn0VkiojMBz4MfK+J854CbFPVfhE5EvirBvb9d+CfRGShd33L\nRGQP4HZgkYi8U0Ty3uPVInJwE+dtjANMQBidwg+AO4FnvMdnVHUV8DfAvwIvA0/h+gga4QPAK94x\n7/XO8+3mTBmA9wGfFpFdwCdxBVJcvuRtfyewE/gPXGf7Llxn+Dm4mtSLwGdxzVeGERtRtYZBxthG\nRNbiOpZ/3u65GEYnYRqEYRiGEYkJCMMwDCMSMzEZhmEYkZgGYRiGYUQyphPlZs2apQsWLGj3NAzD\nMMYUDz744EuqOnu47ca0gFiwYAGrVq1q9zQMwzDGFCLy3PBbmYnJMAzDqIEJCMMwDCMSExCGYRhG\nJCYgDMMwjEhMQBiGYRiRJCYgRGSeiNwtIo+LyKMi8rfe+EyvFeKT3t8Z3riIyFUi8pSIPCIihyc1\nN8MwDGN4ktQgisDfq+rBwNG45ZgPAS4FfqGqC4FfeP8DnAYs9B4X4nYDMwzDMNpEYnkQqroR2Og9\n3yUij+N24joTtw8vwHdwWzx+zBv/rrq1P34rItNFZI53nDHBA2u3sW5rL6991Z7MmNTFYNHh2vue\n5ZWBYlOOf9rSORw8Z2qsbV/aPcD23kH+bM9YrQ8aYvULO7jz0Rfpzmc57+j5TJuQr7ntfz+8gSc3\n7XL/EeGNS+dw0N5T+MXjm9hn+oTY1zMctz28gaf88xjGKOjKZXjnMQuYNiHPD363jhd39MXab4/J\n3bzrmPm8MljiO/+3loFCKdF5Ltp7Cmcs22f4DUdBSxLlRGQBcBjwO9zevL7g2Cgie3qb7Utl+8b1\n3liFgBCRC3E1DPbbb79E590on7n9MR5ev4OefIabLzqWxzbs5F9+8gQAErdHWA1UYe3WXq4697BY\n23/1509y31Mv8ct/OGl0J47g63c/xU9Wu10w586YwJnL9614/aerX+SI+TOYPaWbj9z8MP0FBxH3\nGq6771lWHruAr/3yKc5cvg9fPSfe9YTpL5R4ZaDIHpPd9gaFksPf3fgQJUdH/T4b4xu/NN3cGRM5\ncdFsLrv1j8Dwv19/v9cdvCePbtjJ53+2JtZ+o+GMZfuMfQEhIpOBW4APqepOqf2ORb0wpJKgql4D\nXAOwYsWKVFUa7C84LJ83nbVbX+FLd/2JF3f0c9BeU/jph46nznXH4rSv/pq+Bu5Idg8U2d0kzaWa\ngaLD9Il5tvcWGCw6Fa/1F0pc/P0H+dgbXsVFJx5IsaS8/+QD+cipr+L5bb287Zu/4Wu/fApwF/aR\ncM09z/DDVc9z78deC8CLO/opOcpn37KUt786XTcNxtjihe19vObKXzJYdBjwvtv/8ualnHtk/e/V\njx96gb+94aGK/X7+4RMS0eBbSaJRTCKSxxUO31fVH3nDm0Rkjvf6HGCzN76eyr7Cc3G7YY0ZCo7D\n3BkTeM9x+/PLJzbz2MadrDx2waiFA0B3LhN88eJQdJSSU19+3vOnLbzxql83vFAXHaU75351qs8x\nWHJQhaJ3TEeVjHf982ZO5Ad/czR/+7qF7D9rEoXSyOT7xh19bHtlMPh//cuuCWDf6RNHdDzD8Mln\n3O9qwXGC30UuM/zvN+ttU3Q0+O5nM2M/SDTJKCbBbYH4uKp+KfTSbcBK7/lK4Meh8Xd50UxHAzvG\nkv8BoFhS8tkM7zp2AVN7ckztyfGXhzVHBezOZehvQIMoOQ6lYUq5P7ZxJ49u2Mnu/sY0jWLJoSef\nBaBQJSCK3qLvDztKhYDcf9Yk/u6URUzqzg4rwGrRO1jCCV3bhu2egJgxYUTHMwyfXNZdEoslpeh9\nP/PZ4ZfJnCcMCiUn+A3EESxpJ0kT02uAdwJ/FJGHvLHLgCuBH4rIu4F1wNnea3cAp+P2De4FLkhw\nbolQLDnkMsLUnjxfPecwHFUmdjXnLe7OZ9nRV2hgLkppmDt0/06n4DSoQZSUnpwrIEpV2kdYc/B7\njUT9TnKZTPADbJTewRLhKb/gCYg503pGdDzD8MllPQ2iFNIgssMv9Hlvm5LTmGBJO0lGMd1LtF8B\n4HUR2yvw/qTm0woKjgZ3ICe/as9htm6MnlyGzQ1pEDrsAjzoCZBig6aeguPQk/futKrO4WsUjpa1\niEyEiS2XkUCYNEp/oVKDeOHlPmZN7g60GsMYKflAE9CQiSmGBpEt71d04guWtDP2RVyKKJSc4E6i\n2XTns0McwvUoOjqsicn/ATQqIEqO0u0txtVmooI3RyfkA4nSILIZGZUGEb62F7b3mXnJaAr+ol4M\nmYri/KZ930Wx5AS+tbz5IIwwvg8iCRr3QQzvpB6pialQ0uBuvXqR9++eHNXgLj/KSZ/PZkblg1Al\nMGG9sL2PudNNQBijJxc4qcOawPC/6UgntWkQRphCyUlMrWw8ismh5JT9AFH4dzqNLtTFkkNPruzM\nizqmo+XY8CgTU3YUJqa+QdeprupqKqZBGM1CRALzZ1kTGP43XTYxOcFNkzmpjQqKjiamVvbks40J\niNBCXUtm+SamRsNcS47Slcsg4kZLRZ1XQxpEtJN6dCYmgJIqL/cOMlh02Nc0CKNJ5LLiaQLeQh9D\ng6hwUgemqbF//z32ryAl+Db3ZDWI+CYmf/Et1jEfjdQHUXAc8tkMuYwMCXMdLA01MUU6qbMyYhNT\nnycgHFVeCHIgTEAYzSGfybhRTA04m3OZSie1SNnsNJYxAdEk/C9Tcj6ILIXS8H4FH3+7etv7KnQ9\nIRJFsaRkM0I2M3SRL4e5lqOYovIEc96PcCT4GeWq5RBXMzEZzSKXFTcPogFnc+DcdlzTVCeYl8AE\nREM8v62Xr9/9VKRdP+nkmG4vrDSuFlGMJSB8E1ODPghHyWfFzWWo2tc/b2UeRLQPYiQaxGDRqbg2\nS5Izmk0um6HoOMHNTjwNwo9iUi8fqjOW1s64ihbxs0df5PM/W8POiMzjpO2OvlN4oBDvrtv3DcQR\nEI2amPwfgGsmqpyPf0ytyIMYegzfztsovnkJXCG0e8D9f2pP7YqyhtEI+Yy4eRBBwlucRLlKJ3Un\n5ECACYiG8BfbqOibsokpuTwIILajuuyDqL0I+4JhJJnU2YxE+iDKzvGQDyJCQriRIo0LiN5CWTg7\nXhRTh2jzRkrIZTNeHkQjiXLhTGqnIxzUYAKiIfy1MOquvJGIh5HgF8eLmwvhz9GpIyAGR6pBeCam\nbEaGlPMoxHRSZ0dYaqM3rEE47jk6wRlopIdc1r3xKf+mG3BSe/uZD2Ic4i94UQtbI5UfR0J3rkEN\nojS8BlE2MTVazdUhl81E1lMKl9qolweRjzBPxaHaxFRSbUq1XMPwyWdcDaKRwJNcVSa1CYhxiBNy\njlbjL7ZJZlJDfCd1nCimsokp/p28qgY/gCgfRDHwQdTPg8iO1MQUEhAlVRxHyZqAMJpIdRRTnMW+\nXKJDKXk3UJ1AZ1xFi/DX0ajwzKQrOPaM0AcRz0kd/07eP1wukyFbxwdRcnT4Yn0jcVKHTGy+I7xD\nbtaMlJDLZig4oWJ9sRLlfBOT4xXt7IwvpQmIBvDviOtpEIklyuUb9UF4i3+cPIgG7uTD15mL8EEM\nhvMgHL8W09Dj5EZYi8kvs+Gew80LiXKCG8ZIyXulNso3fY2HuXZCoT4wAdEQWscH0Ujlx5HQ3WCY\na0N5EA34AsI/migfRDGmk9qNgGrcB1FhYvJqTUUd3zBGStnEFD+KqbJYn2kQ45JSHA0ioTuHRp3U\nQUhunUU4CIVtQIPwNYaslwdRfXz/mBV5EBFvSTYjQbG9RggLCFX3M7EoJqOZ5LMZr+Vo/Ju+iiJ/\njjmpxyX1fBCFBkLiRkLPCDOp692k+/0lGil5Ec73iMqGLkTlQdQo9x2eZ1yqo5hcH0Rn/BiNdODn\n6BQdh2xGYkfJ+cmf5qSOgYh8W0Q2i8jq0NiNIvKQ91jrtyIVkQUi0hd67eqk5jUa6vkg/DvprgRr\nMQH0x86kbqBYXwOLdDmyIxOZ7FYI+SB8k1zUD6yskjdmZqo2MVminNFsctlM0Fu6EU0gKPLXQWGu\nSZb7vg74V+C7/oCqvt1/LiJfBHaEtn9aVZcnOJ9R4wSLbvsS5eJoEKoaL8y1TmZ47X3K+R65zFBH\nc6UPwh2rVe47PIe4hKOY3IKAZmIymkve0wQKDTYAC/sumtWLvt0kpkGo6j3AtqjXxL2lfBtwfVLn\nT4J6mdSDSSfKBSam4Rfz8PzqOqkDE9MINIiseBmnVbWYAh/E8E5qYEgU1HAMjWIyE5PRXPJZXxNo\nrIWwW+RPrRZTEzge2KSqT4bG9heRP4jI/4rI8bV2FJELRWSViKzasmVL8jMN4S94kXkQCRfrC5zU\nMUxMxbgCIgiFHYEGkc3UL/ftlP0fkYlyobjxRugd4oPQSCe4YYwUv0pxsUFfQrgTXaeYmNr10zqX\nSu1hI7Cfqh4GfBj4gYhMjdpRVa9R1RWqumL27NktmGr43O7fej6IpO4cshkhnxX6Y5iYwvNrdh5E\nuJ1iVLnvKCd1lA8i0CAajWIKm5gc9zymQRjNJJ+VwJcQp92oT4WTukPuWlp+FSKSA94M3OiPqeqA\nqm71nj8IPA0savXchqNUxwfhL4xJOanB1SJiaRChRdsPzfVZ/cIOvn3vs0EHPBihicmr5jo0iins\npHbHokphhBOLGmFILSYrtWE0mXLL0cY0CN9J3Ul5EO3wpPw58ISqrvcHRGQ2sE1VSyJyALAQeKYN\nc6tLUKwvYlFrpLnISInbdjRsMgrb+B/bsJO/+tZv2dlf5Jwj55W3acDME645lY3wQUT2pI74jZU7\ncDWaB1Hpg1CNLiduGCMlF7QcbWyh953UfkveTiDJMNfrgd8AB4nIehF5t/fSOQx1Tp8APCIiDwM3\nAxepaqSDu534a1mUzT7pRDlw6zE16qQORyr99XUPBM2Odg+UF9pGivX5x8vW0iD8RkXDmpgy3lwb\n80H0hTQoR91rNflgNJN8KBqpkZIZfmWBTir3nZgGoarn1hg/P2LsFuCWpObSLLROHkQjWZcjpTuX\niVWLKXxX7i/S2/sKvLiznwNnT+LpLa+wK9QVr6Ew13AUU10fRLijXB0TU8OJckUmdmXpHSx5BQHN\nB2E0l3LL0RFoEI51lBu31PNBhKN7kqIrlxmxBrG9dxCAfaa7vZt3VwiIRjSIsonJrchau9x3uSf1\n0ONkR+iD6B0sMak7F5zDBITRbMItRxuPYrKe1OOWwMQUsai1RIOIaWKqDHN1t3+5twDA3lN7ACo0\niIZMTKWyiSmbrV9qw3+t2aU2pngCwi8pbolyRjPxhcJAodRgFFPnOalNQDRA2cRUJw8iSR9ELsNA\nDBNTeH6+9Wi7LyCm+QKiEGzTWCZ1+Tqjejr4GoUbguqORd3gZ4Mw18bzIHwNwnwQRhL4i3t/odTQ\nQh9kYJuTenwyXMvRjCQbUdOdz9I/Yg3CNTEFAiLspG4ozDXcDyJTtye1xsikbuTcjqP0FUpM6naT\nBgMTk0kIo4n4N3l9hVJjpTY60EltAqIB/LUs0kndggqO3TE1iLAJzBcWO+qYmBrJpPbNUX7L0SGl\nNoIw12Gc1Fk/iim+gPDNa5N9E5P5IIwE8LWGvkKpoYXez6QuWrnv8Um51EZ0sb5G7JUjoTuXCUp0\n1yOqFtPLvYPkMsLMSV3AyJ3UpbilNmL0pIbGfBB+aO7kKhOTJcoZzcS/eekbbLDURlaCmxgr9z0O\nqe+DcMjnkn074+ZBRNVierm3wPSJeSZ0ueaZsA+ioX4QVZnUQ30QjZXaaMT/8dLuAQD29LSgoB+E\nfYuNJuLf6PUXSg0X6/Mz/c1JPQ6pW2rD0cRD2+LmQURpEDv6Bpk+sYuenC8g3LvxjIysH4Qb5poZ\n0hXO13DCpTYiy32PIJN6085+AOZ4fhQn6AfRGT9GIx34d/+9g8WGftP5jAS/T+tJPQ6pV+67UGys\nNPBI6M7F1SCc0HNPg3ilwPQJeXryroDwzTUT8tmG7uJ97Snr+SCgsiJrueXocOW+G/dBbN7pahCB\ngFDrB2E0H/937GhjmkAuW76BMw1iHKL1fBAtyJ7szserxVTLBzF9YlfQunSnZ2Ka0JVtKJIonO9R\nDlUNOcVL4VIb7ljUAp4NopjiCydfg9jLMzGVHKWk0SYswxgpYa2hsVIbEjS0Mif1OKSsQUTXYkpa\nrezJuYv5cHfdUT6IHX2uD2KIBtGVHXE/iKhyGUGinEPIBzH0OP5dWiMaxKZd/cwI+VH8bO0OuVkz\nUkL4Rq/RUhv+19mc1OOQuqU2GmxPOBLKXeXqaxGliDDXl3sHmTExH7Qu9X0QE/O5IVFM61/u5T3f\nWVXhyK4+nu+khsooqLh5EI1EMf3s0RcZKJbYtHOAvab2BMcrJ8qZhDCaR9hU3GgeRPl5Z3wnTUA0\ngBNEMUXXYkrcxOT3pR6mJ0RFsT5H6S+U6C84TJ/YhYjQncsEYa49XdkhuQy/X7ednz++iT+u30E1\n4X4Q2aBcRpQPYrhifZmK49Vi3dZe3vufD/Kj37/A5p397Dm1J3B6l7xyHpYoZzSTkS70IxUsaaYz\nrqJF+FE5UTb7wVJjhb1GQtB2dBhHdXWxPr/MxvSJecANl/W1g4n57JBF2ne0rdvWO+TYvo8hG9Ig\nSk60BlEvDyIXmJjqX4vvK3lsw05Xg5jSHQgc9ftBmHwwmkiliamRPIiQYOkQu6cJiAZwhsuDSHil\n6olpYqpoGOQ4QZmNGRO7guO84sVrRzmpB+oJCC9LVCTaxFQMFesr12IaeblvX1g9umEHW3ZXm5iU\nkkUxGU0mfPffUB5E6HtoJqZxyHA+iORNTK4G0T+ciSncctQpF+qbPqGsQfhEOan94z//ct/QY4ei\ntaJyGQpBue9yfsRoyn37USEPr99ByVH2mlrWIErWk9pIgMqFfqQ+iM5YWjvjKlqEb2KqVYspabuj\nr0EMlyxXGebqBL0gpvsaRK4sIBo1MYWjtbJVXeFUNTKTejTlvv3MVP+a9pzaE2ROO2qJckbzyY/Q\nVDTS6Kc0k2TL0W+LyGYRWR0au0JEXhCRh7zH6aHXPi4iT4nIGhE5Nal5jYZ6PakLpVYICF+DGM7E\nVOmD8HtBzJjkaxDlebompioNwjNhPR8hIEqOkvU1iCozUWUnu/pO6qhy36+EKsyW51I5tz1DPgjH\n+kEYCZCrcDabkzoprgPeEDH+ZVVd7j3uABCRQ3B7VS/29vk3EclG7NtWSkG57+h+EEnbHX0B0Tes\nBuHOLyOuUNve52kQE1wNonuIialag3D33/bKYEXvanAd9L76XO2DCAvOylpMQ+dYXe77iRd3suxT\nd/L0lt2VcxmsvNa9pvYEAsEPczUFwmgmIzUVWZhrA6jqPcC2mJufCdygqgOq+izwFHBkUnMbKXVL\nbbREg/BNTPHCXHs889GO3gJduUyQYBb2QfTkspSccs6Ce/zyolytRRRL5ZIi1T6IwZAmolrOPI8K\nQxWRimqwz2/ro+QoG7f3V2znC8Npnv9k9pTuQCA4QaJcZ/wYjXTQFXZSN1CA00xMzeESEXnEM0HN\n8Mb2BZ4PbbPeGxuCiFwoIqtEZNWWLVuSnmsFqkNNKT6tKLUxIaaJyV90u3MZSo66Xdi6wkLB/di7\nsplgsQ9fU1gAVfshik45aqjaBxGu6RSOYqp1M5UNVYPtHXQ1lcFS5bX513rE/BnsOaWbfDYzJIrJ\nfBBGM6kwMTXUD8Kc1KPlG8CBwHJgI/BFbzzqU4j0XqrqNaq6QlVXzJ49O5lZ1qCeD6JYSr6aa6M+\niK6c2+FqsOjQFboT8o+Ty0oQux2+pv5iiVmTXXOUr0GEhaOvKeWrTUxVJT7qOamh3GAlfE3V/S58\nDeLyNx3CN847AiDQGBxHKTnJdvEzxh8jz4PoPA0i18qTqeom/7mIfAu43ft3PTAvtOlcYEMLpxYL\n/wY5ygcxWHLoyrVGgxjeB+FrEFlKqt7cwgLCW+BD9ZQKjsMEvES8Qom9pvYwUHQCAfHhHz5MyVGK\nJSfYp7pchu/szns1aer1pAYq+kn0er6G6iTAvkKJrlyG+XtMYv4ekwAqSm2oWk9qo7mEa6qZk7qF\niMic0L9nAX6E023AOSLSLSL7AwuB+1s5tziUGwZFaRBOCzWIeHkQ3Tm3Z/Rg0amwq/rHyWejk936\nCw49+Szz95jIMy+9gqryqzWbeWrzbgqlsomp2gdRKJUFk19ID6jpI8hlM8F7WUtADBScwCTmI6Ew\nV0uUM5pNhSYwzp3UiWkQInI9cBIwS0TWA5cDJ4nIclzz0VrgvQCq+qiI/BB4DCgC71fV4etatxin\nng+iJYly5Wbq9Sg5DiLuAlx0lKLj0BXKfSgLiEzIxFRemPsKJXryGQ7aewr//dAG1m3r5eXeAlN6\n8pRC+R61fBDduUyQowC1TUyuD8Ldx893GGJiGiwFzvVgv5APwvIgjGYz4jyIESbYpZnEBISqnhsx\n/B91tv9n4J+Tmk8z8OVCZB5ECxLlMhm30N5ADB+EX23VUaVQbWLynueyEqjFhQondYnpE/KsmD+D\nH/xuHbc8uB5w8xQqMqmrtA9fg+jKZegdLNXNgwDXh+Hv21fHBzEhXykgwiYmR2sf3zBGQnihb7Tl\naPl5Z3wnO0PMtQj/jrimBtECtXJCVzaWDyKbkSBKaLDoBNoHlPMg/LahUKlB9BdK9OSzrJg/E4Dv\n/24dALsGihWZ1NUmJl8b6PI1CD8Posa3LJuVISamweqkPW8uYfybM98R3iHmXiMlZEeoCXSik9p+\nWg1Qq1ifX2KiFY6pnlw2VhRTLpPx8gwcBktVAiIU5hq0Da3yQXTnM8ybOYE9p3Sz9RU30W6w6NBX\ncMo+iBpO6q5sxsuDcI9XO4opE2gufV6Ya3Up874oARGq5mr9IIxmIyJDcn3ikB9hJ7o00xlX0SIC\nE1OVBhFuw5k0PfnMsE7qsAZRcmo7qV0TU7lC7OMbdwbPe/JZRIQVC9xUFV8Y7OwrBD+aah9E4KTO\nZwIHMtTOg8h5AgxCJqaIPIhaJqagWF+HOASN9OBrDo3c9IU1j6xpEOOPWnkQ4TacSdOTH97EVHSc\nwAcRCIiIPIhwmOuPH9rAG6/6NZt39btRTJ5T+wjPzLR83nQAtvcOBj+aWqU2urKVJqa6TupSlYmp\nWG1icoY4qf3foZ+MZ/LBaDbVfrY4VIS5mgYx/vB9ENVhrv6dcyt8ED354U1MQ3wQtfIgMplgsV+3\ntRdH4aVdg57d3x0/5eC9WLLvVN6wZG/A7W1dM8y1wgdRNjHVzIPIlvMg+urkQYSLC7rHE0TKJiYr\ntWE0myAZ1BoGGXEpm5gqF7FikCCW/Ns5IYaA8B3m2YzgRJmYPO0gn5Pgi7xl9wDgaghFRwMtY789\nJnL7B45n0V5TAPc9yAcmpioBUfQFhJsHMVyYay6TGZIoFxXmWu2D8I/p7xvVkMgwRkMu07gPwhoG\njXNq9aQu+yBaYWKK6YPwkuDqldoIRzFt3uUWyfMFRfVd+6TuckR0uZqr54PwBKS/YHd7GsRwYa5h\nH0StUhsDxaE+CHBzIfzzWaKc0WzKZtT4v+mw6bVTblpMQDSALyCqW3T60TutUCvjhLn6UUwZcX0Q\nA0MERPnL72sDW3YNVPytvmufXCEgamVSl01MlbWYoueZzUjwXgaZ1KWIRLkIASFS9nmYgDCajf/d\nbiTwpFzEsnO+jyYgGqBWuW9/gWxJFFOMMFffB5HLlp3U3RGZ1F25crE+XysJNIhclYDoCQmIWg2D\nQiU+oCxQa91N5StKbQwNc1XVyDBXqMzC7pCbNSNFlG+CGtEgfKHSOctq51xJCyiX2oj2QbQivb6n\nK1pAbNhe7h9d8ArqZT0bfy0zn6wMAAAgAElEQVQndS6TGWIr3bLTFRDdVSamyV1hAeGX2vDDTSs1\nCF9AFJ36hfTC5b7LYa7l93aw5OAoQ6KYwDVb+dqHOamNZlNdsTgO/u+iUxzUEENAiMhrRGSS9/w8\nEfmSiMxPfmrpY7goptZpEJUCau1Lr3Dslb/kwedeDuaX83wQvjCpTJQr+yCq73bKPojKRXlSd/l/\nX6jkgyxsP4qpXKzPn0c9ddsv910oOcF7OFgsCz//OqM0CNfE5HfO65wfpJEOAi25EQ0iSCDtnPvu\nOFfyDaBXRA4FPgo8B3w30VmlFF8uFEqVHdgKrYxi6soM8UFs63Uznbd6i7vb1Mf1Qfi2/ahEua5Q\nFJNPLR9ELpup0DzArQ0lUtaoiiEfhPu/1nXW+Saw8PWEndS+cKt2mIPvv3CCeRhGM8llGtcGckFo\nbOd8H+OsaEV1V8Mzga+q6leBKclOK504IaEQViJamijntQgthEwxvkYTjrLyE+X8/IKaTuqqu51A\nQES0Wpzc7bb9DP8A/HNccO39rFrrajC+MCo5Tl0Tkx/m2hfqOx3Og/DHo5zU4TBXkw9Gswn8CQ1o\nA53opI5TzXWXiHwcOA84QUSyQD7ZaaWTsIAoOg7ZjLtwBSamFhXrA9dmnw8WYl8whOcmZLMS2PRr\nhrlW3e342kiUWWdyd5aXdleVFMgIz7/cy91ryu1fK30Qtd+TrGdiCguICg2iWE9AWBSTkRwj0SDG\nq5P67cAA8G5VfRG3V/TnE51VSnF0qGMWygtVKzSI7qBpUHlRDXwjVRpE2Hlb0Yjd60XdlRvqpPZl\nYKSA8CKZwteZz2TYunuwYjtfGA1XSM/PpPbNYD35TIWT2hccPTWc1OUoJhMQRnMZSamNcn5Q53wf\nh9UgPKHwpdD/6xivPghHyXt283AuRCvzIHzTTzgctBhoEOWEtZ68VNxZd1WZjD731mUsnzejYrHf\na2o3m3ZGJ8oBTPIimcKaUjYrvOT5PsC9sw9nWNdbu/1aUX0FN8R1+oSuCg3C901Uh9y657EoJiM5\n/Dpljdx85Efg2E47NQWEiOzC7fwWiapOTWRGKcZRpSvrRhGFNYhwmeukCZuYfMqag/d/yAfh010l\nIM46bC4AO/sLwdj8mZNCAiLKxOR+XbJVPoiXQhpELpsJtIZiyRnGxJShUCprENMn5tm8qyxsfCEY\nFeYazoPooN+jkRJymaEBHMMh4t6UjQsntapO8YTAV4BLcU1Lc4GPAZ8Z7sAi8m0R2Swiq0NjnxeR\nJ0TkERG5VUSme+MLRKRPRB7yHleP9sKSwFGC1p3hXIjBFkYx+XfTUSamoKFRyY1iqqdB+ISdcPNm\nThxynjC+iSm8TzYj7B5wNYAjF8ykO5cJnMbD5UG42ljZBzFtQj5Sg6iVSe1rEGZiMppNPjs0gCMO\nfpHMTiHOO3Cqqv6bqu5S1Z2q+g3gLTH2uw54Q9XYXcASVV0G/An4eOi1p1V1ufe4KM7kW4m/+Hb5\n5SVCJqZtXkOdGZOS990HGkTIsVus5YOIISD8u6SMwL7Te4Lx6kQ5KNdjqtXU/V/espQvnn1oEHY6\nnA/CT5TzBcH0iVUCYrgoJk8wm4nJaDa5bOMaBLjm104p9Q3xBERJRN4hIlkRyYjIO4D6tR4AVb0H\n2FY1dqeqFr1/f4urkYwJ/AimfMgB6/PS7kFEYObErsTn4fsG+kMLaXUCX9FxgmJ9PtUmJh9/m2kT\n8kzz5i8Svf2U7qFOav9HNLUnx4GzJ/P6xXsHd/SF4fIgvH4QgYlpQpebPe1dhx/FVCsPIghz7Zzf\no5ES3Ai/xr9YuYjIwLFMnHfgr4C3AZu8x9ne2Gj5a+Anof/3F5E/iMj/isjxtXYSkQtFZJWIrNqy\nZUutzZqOLw98P0OxQkAMMGNiV8saBkG0BlGdBxFOIOvKDr0LB9c8k8sIMyZ1McUzIXXnMpELe6BB\nVIW5AsyYVBaO/h39sHkQXi2mvpAPAsomu3pRTGETk2VSG83mzOX78N4TDmh4v3xWxoeTGsDLeThL\nVc9s5klF5B+BIvB9b2gjsJ+qbhWRI4D/EpHFqrqzel9VvQa4BmDFihU1nejNxl98yyGc5Tv4rbsH\nmDU5ee0BygJiIFSSoroMedEv1hfDxASuFjBjYhdTPQER5aCGspM6fFxfnZ4+oWxeq/RB1NcgCo4T\nmJimhQREuDHScCYmExBGszl+4WyOXzi74f3c5NPO+T7WFXWqWsLNoG4aIrISOAN4h5ehjaoOqOpW\n7/mDwNPAomaed7QEJibv7iAc5vrS7kFmTe5uyTwmRGgQpSoTU9kHUf546wmIfCbDjIl5pvS4C3SU\ngxrKAiLsjPc1iOkh81q5Z/TwxfpUYfdAkWxGguP7foj+guNFhUSYmKwfhJFCOs1JHSeT+j4R+Vfg\nRuAVf1BVf9/oyUTkDbhRUCeqam9ofDawTVVLInIAsBB4ptHjJ0lgYorwQWzdPcDSudNbMo+eiES5\n4hAfhB/FVN6vroDIZZg+sWxiirL5QzmKKfwD8O2tMyaWNQgJaRD1fBD+wr+rv8DEfDbwe/jlNvoK\n0b0g/HP4kWQd9Hs0xjhduUzgp+wE4giIY72/nw6NKfDaejuJyPXAScAsEVkPXI4btdQN3OUtHL/1\nIpZOAD4tIkVcB/hFqrot8sBtIjAxRfogBltmYgo0iEKEk3pIFFO4gmvtL+3fv34RB8+ZWtYgaizK\nkwINYqgPopYGUe9uyn9tV3+Rnq5sIMQGQwKi1lyynoM7fD7DaDd///pF7DW1Z/gNxwhxMqlPHsmB\nVfXciOH/qLHtLcAtIzlPq1BvPa72QfQXSuweKLbMxOQv9FEaRDkPwmnIB/GOo9zq7X5GdHeNRdnX\nMPJVpTag7GCGclTRcHkQuZCAmNiVDRzpgYlpsMSEruh5V2RSmwphpIQzlu3T7ik0lTgaBCLyRmAx\nEIhGVf107T06j1LggyiHcEJ5UW2VBpHJCN25TBACCmEntTfXqDyIGJEVgYmphjA5dO50/t8bD+Y1\nfzYrGAuimCI1iPqZ1GUBUWBCyMQUCIhiqaY/JFNhYjIBYRhJMKyA8LKaJwInA/8OvBW4P+F5pY5y\nFFO5GQ4QFKrbY1JrNAhwTUD9UU5qDfkgso0LiO6ca+apZ9Z5z/GVoX++D2J6hQ+inExYb+3OBj4I\nT4MIfBDutfUOlpgYEeIKrqAMTEymQRhGIsTxphyrqu8CXlbVTwHHAPOSnVb6qOWDCDSIKa0UEJmK\nrnKlahOTo+RDpTbyWYm9iE7tydV0UkcR7YMozyuOBrGzr8CECB9E72BtH4RrYjIntWEkSZyVwG92\n3Csi+wAFYP/kppRONIhiKptPIKxBtMbEBK6juqJYX0iDUNXAOewvwI0UETxw9mTm7zEp9vZ+qY1w\nFFNQrC+mgNiwo58DZk0uaxChRLmaGoSUhbSV2jCMZIjjg7jdK6r3eeD3uBFM30p0VinEX4S7qvIg\n/B7Os1uqQWRrhrn68wz7IOo5qKv5/nuOasim7y/y0ycM1SAKJadu9FS4JME7j5kfaARlDaLIxK6J\nkftmRIJrtWJ9hpEMcaKY/sl7eouI3A70qOqOZKeVPqoT5cI+iMnduZqmkCTozmUqWnOGazEFd9XZ\nkQmIRssE+KW/p08aqkEMX6zPPddxfzaLRXtN4anNu4BQHsRgKbLUd/gc7nFMQBhGEsRxUv8auAf4\nNXDfeBQOEDYxDfVB7NGiCCafbEYq2p+G8x/CGoR/d99dIxKoGfiail/ID6pMTHXkzeRud14XvGYB\nwJAw175CbRNTWCiYfDCMZIhzu7gSWINb4vv/vEJ5X052WunDX3jLGoTng3hloGU5ED65TKai3Hjg\npNaQBpEpN+5pRINolHw2w/QJ+Qozjy8UhtMgjl84mxsuPJrXvmpPoFxiPOykrqVBhA9rUUyGkQxx\nTEzPiEgfMOg9TgYOTnpiaaO6WJ/vg9i6e5D9ZkbbyZMim5GKUh+lGj4I38afZKe7dx0znxMWVRY1\nK4e5OsOW2jj6gD2C//15DhZLlBxloOjULLVRYWIyH4RhJEIcE9PTwEvAD3AzoT+gqk79vTqP6nLf\n/kK8e6AY1ChqFbmsVCTK+SYmV4Pw23CWS20kqUEsmzudZVV1qOIW66umK1SLyY/SimdiMgFhGEkQ\nZ+W4ClgHnAt8EFgpIgcmOqsUolUahG/KGSzWj9RJgiEaRKlGFFMLTExRxC33XU04D6J30O0rNaEr\nWvhmKkxMI5unYRj1GfanpapfVdWzgT8HHgSuwG0XOq4oVfeD8EMyS06iJpwo/E5s1XMrOeVWqOGy\nw60WYBXF+hoMmc2I+5765cwn1qzmahqEYSRNHBPTF4HjgMnAb4BP4kY0jSv8/kD57FANotV36LV9\nEE5Zg8i2xgcRRWW570b2E7q8EN5hTUwW5moYiRPHeP5b4HOquinpyaSZaid1WEBENbRJklw2E/ga\nIJxJTcujmKLIjOLuviub8UxMtduNQqVZyeSDYSRDnJXjFuAUEfkEgIjsJyJHJjut9BHkQWTL5hPH\nS0xr9QKcq9IgfOHl1MyDaKOAaPDU3fmsq0EMY2IajRAyDCMecX6+X8ct0PdX3v+7vLFxRakqk7pY\nUgY9P0Q7TEzhhkXFkJO6MoqpvU5q9/noNIiJNZ3UZmIyjKSJY2I6SlUPF5E/AKjqyyLS2tThFODf\npWc9R2rRcYKSEO1wUlf4IHwntdbIg2i1gAgt2I3WSXLLiJRCUUy1i/WVn5uAMIwkiLNyFEQki1uk\nz+8fHSsPQkS+LSKbRWR1aGymiNwlIk96f2d44yIiV4nIUyLyiIgcPoLrSQwNCYhcJkPR0SDjt/Vh\nrpkKDSJc7rvsgwiFuWZbVycKqs0/je3blXM1iMDEVKcfRNRzwzCaR9w8iFuBPUXkn4F7gf8v5vGv\nA95QNXYp8AtVXQj8wvsf4DRgofe4EPhGzHO0BH89zoh7Z15y2mdiGqJBOFEaRGZMmpi6cxk3zNWL\nYoqTSW3ywTCSIU6pje+LyIPA6wAB/lJVH49zcFW9R0QWVA2fCZzkPf8O8CvgY974d9W9Vf+tiEwX\nkTmqujHOuZKmXFra8wGUlEKxjT6IUqiaa6hYXzgPIteCTOoomqFB+D6IOCYmK7VhGMlQV0CISAZ4\nRFWXAE806Zx7+Yu+qm4UkT298X2B50PbrffGKgSEiFyIq2Gw3377NWlKw+Mvwhlxo4OKjlPWIFps\nwqnWIIoRmdRuFzn39VabwMLrdaM+iJ58ll39RfoGS2Sk9tyzo/BzGIYRj7orh1dz6WERacVKHPUr\n1yEDqteo6gpVXTF79uyIXZLBD3PNZoS8F2kz2C4NIlsZxRTWIAqhKCZfg2hrmGuDa/esyd1sfWXA\n60edq7n4i0UxGUbixIlimgM8KiL3A6/4g6r6FyM85ybfdCQic4DN3vh6KntdzwU2jPAcTce/M8+I\ne5c7UCxHMeWzrV2gavkgHNWgLlMukwl6S0/qbm0xwdHkKMye0s2WXQP0FYo1zUtQaVYy+WAYyRBn\n5fhUk895G26PiSu9vz8OjV8iIjcARwE70uJ/gPJduojQk8/QXyi1T4PwophUFRGpaDkajmKaPrGL\n7737KA6fP73e4ZrOaJzUsyd3019w2LxzoGYE05BzmIQwjESI46T+35EeXESux3VIzxKR9cDluILh\nhyLybtwqsWd7m98BnA48BfQCF4z0vEkQmJhEgp7Qvg+i1SYcP0O65Ci5bLm7XEmpqMUEcNzCWS2d\nG1QV0mtw8Z41xU2xWbett2YEU/U5zEltGMmQqO1BVc+t8dLrIrZV4P1Jzmc0hJ3U3bkM/YWQD6LF\nTmrf5l50lFy27KR2qjKp20WlBtHYvrMn9wCugDhkn6k1t7N+EIaRPFZJPybhMFfXB9E+E1NYg4BK\nJ3U4k7pdjNYHAW7ToPgmpsbmZxhGPOynFZNwolx3LutqECU3Vr8deRBAhe/B/xv2QbSLyhDUxvb1\nBQTAhHxtBdeK9RlG8tT8BYrIH4kIM8UNR1VVXZbYrFJIuNRGTz5Df4o0iFqZ1O1CRuGknj4h7+WZ\naH0NImM+CMNImno+iDNaNosxQFmDgO5cloEKH0Tr+0EAgb+hpGEfRPs1iNHkQWQywqzJ3by4s7+u\nkzp8XJMPhpEMNQWEqj7XyomknVJUmGupsolQq6jWIIJMatWgFepY9UGAa2Z6cWd/3TwI/7gZsUxq\nw0iKYVc2ETlaRB4Qkd0iMigiJRHZ2YrJpQkNopjKiXLt0iACH0RpqJM60CBanLwXpvLuvvF5zJrs\nhrrWd1JLxV/DMJpPnJXtX4FzgSeBCcB7gK8lOak04lT7INqYKJfLRvsgqjvKtQsZZZaz76iOJSAs\nSc4wEiNWHoSqPiUiWVUtAdeKyP8lPK/U4RdP9aOYio7SWyhWdG5rFdlMZV/ssJM6HT6I8PORmZgA\nJtToJgfgK23moDaM5IgjIHq9DnIPicjncKurTkp2WumjXGqDoMbRrv5iy81LEBHFFJiYSEUU02h7\nNcye7AmIGJnUpkAYRnLEWUXe6W13CW6xvnnAm5OcVBqpDHN1F66dfYWWm5f8OUA5isn7Q8lxAg2i\nnQtnWECMxAcxe4qbTW0mJsNoL3FWt79U1X5V3amqn1LVDzMOQ2DDiXI9OXfh2tVfbIuAGBLF5Ie7\nOkrJcchlpK2RPRJ6S0ZjYqonIHzFzZzUhpEccVa3lRFj5zd5HqknXGqj2zMx7ewvtMXENDST2h13\n1K/P1N5Fc7QmpsP3m86lp72K4xfW7vfhn8N6QRhGctTLpD4X+CtgfxG5LfTSVGBr0hNLG1pRrK9s\nYmp1JVco+xfKUUwhDaKkbfU/QKXjeCQLeC6b4aITD6y7jfkgDCN56jmp/w/XIT0L+GJofBfwSJKT\nSiO+iSnrJcoB7OwvssekrpbPpToPojqKqd131aNpORoXX0kyE5NhJEfNW01VfU5Vf6Wqx+D2o57i\nPdararFVE0wL4XLfYSd1vh1RTNkqJ7UnvPxy3+3MgYDRm5hinSNjiXKGkTRxMqnPBu7HbezzNuB3\nIvLWpCeWNgIfRKbcIGig6LQ5iqnKSe0V62u3BjHaPIh45zAfhGEkTZw8iP8HvFpVNwOIyGzg58DN\nSU4sbWg4iikUn9/WPIigUZA7rgqDRR0fGkQQ5prM8Q3DiCcgMr5w8NjKKPpIiMhBwI2hoQOATwLT\ngb8Btnjjl6nqHSM9T7MJSm1UC4gUaRAAgyWnrXWYoDU+iIz5IAwjceIIiJ+KyM+A673/3w78ZKQn\nVNU1wHIAEckCLwC34vag/rKqfmGkx04S384fzqSG9ggI3+9RchRVxVHIZ4VCSRksltoexSQiiLga\nTWImJk9CWKkNw0iOYQWEqn5ERN4MHIfbLOgaVb21Sed/HfC0qj6X9pLNTkSYK7Rbg3ACwdWVzVAo\nuQUE02CXz4hQUk3cxJTyr41hjGniOKk/q6o/UtUPq+rfqeqtIvLZJp3/HMqaCcAlIvKIiHxbRGbU\nmM+FIrJKRFZt2bIlapNEcEIlLMIaRHebazH55qW8J6gGS+2PYoKQCSihufiHTYMwNIxOJc7qdkrE\n2GmjPbFXAPAvgJu8oW8AB+KanzZSmXsRoKrXqOoKVV0xe3btTNtmE+RBZMqlNqDdGoQGDmrfWZ4W\nDUISvsPPWpirYSROvUzqi4H3AQeISDgxbgpwXxPOfRrwe1XdBOD/9c79LeD2JpyjaYQ7yom4C/Jg\nqT1hruFMan9e/jwGiynTIBJawMuZ1O2/VsPoVOr5IH6A64z+F+DS0PguVd3WhHOfS8i8JCJzVHWj\n9+9ZwOomnKNpaJU9vTvnCoh2JMqFNQg/1NXXIAaKTt0id60iGyzgCR/fwlwNIzHq9aTeAezAXcib\niohMxDVdvTc0/DkRWQ4osLbqtbbjaGUCWnc+y66BNldzLTmBBpEPmZim9uRbPqdqkm4JGvggTIMw\njMSI1VGu2ahqL7BH1dg72zGXuDhaGdPvO6rbUs01W9YgfCd1Vyi7Ox0+CPdv4iamFFyrYXQqpqDH\nxHEqTUx+sly7+0H4Tuq8JzQGS07by31DuFZSMsc3J7VhJI8JiJg4qhWLkV+PqR3lvit8EBFO6jRo\nEEl3fDMTk2EkjwmImDhauRi1V4MIRTGVhvog0hTFlFypDUuUM4ykMQERE0e1YjFqpw/CX3yLISd1\nd0WiXPs/1qQb+gSlNlIgDA2jU2n/SjJGcBytMJf4yXLt0CBEhFxGXBNTlZO65Gjbi/VB8nkQVqzP\nMJLHBERMnKrCc35f6nYICHDvnEuOBv2ow/kY6TAxtSoPov3XahidigmImFQ7qQMNog0mJiCkQVT6\nICAdZpeyjyDpTOpEDm8YBiYgYuNUZ1J7Tup82zWIyigmSIcGkXQehEUxGUbymICIieNULna+k7od\n1VzB1RiKTtlJ3VWhQbT/Y03cxJQxE5NhJE37V5IxwpBSG210UkNYg6h0UkM6NIikE9nMxGQYyWMC\nIiZuqY3y/z1tdlLnMkKxVHZSd6XMB+G/V0lZgKwfhGEkjwmImAxxUrcxUQ7cekylGk7qNGgQSRfr\n8wVD2jsRGsZYxgRETKqd1D259iXKgZtNXazhpB4feRDWk9owksYEREwcrXSITvFKak/ubktB3LIP\nIij3XZ5bmjSIpORn0k5wwzDaVO57LFJtYnrjsjnMntLNnlN72jIfNw/CCXplV2gQKYhikoTzIPxL\ntCgmw0iO9q8kY4Soct8nLGpdT+xqfA2i6AwNc02HBuH/TdbEZKU2DCM52qZBiMhaYBdQAoqqukJE\nZgI3Agtwu8q9TVVfbtccw1RrEO2mOpO6UoNo/zyTNgGZD8IwkqfdGsTJqrpcVVd4/18K/EJVFwK/\noLIXdluprsXUbtKeSd2yYn0puFbD6FTaLSCqORP4jvf8O8BftnEuFagqKTDtB+QyGQoRPakhHRpE\n2QeRzPHNSW0YydPOJU+BO0XkQRG50BvbS1U3Anh/92zb7KooOekyMfkahDNOfRC+EEyDMDSMTqWd\nUUyvUdUNIrIncJeIPBFnJ0+YXAiw3377JTm/CtJmYsplhf5iyEldkQfRflUn+VIbJHp8wzDaqEGo\n6gbv72bgVuBIYJOIzAHw/m6O2O8aVV2hqitmz25dFFF1oly7qdYgwiamfAommnStpKQFkGEYbRIQ\nIjJJRKb4z4HXA6uB24CV3mYrgR+3Y35RpDKKqVTWIMKJcmkwu7SqJ3UKLtUwOpZ2mZj2Am71Fo8c\n8ANV/amIPAD8UETeDawDzm7T/IZQXe673eQymYpM6oooplSU2kh2Afc/ijQIQ8PoVNoiIFT1GeDQ\niPGtwOtaP6PhcVIWxZTNVmZSd6cskzoQEAkt4NmEM7UNw0hfmGtq0bQ5qYdkUmcrXms3ZSdyMsdP\nutaTYRgmIGJTSpkPIutlUgdO6lzafBDJOpEtk9owkscEREwc1cSSvkaCr0FEJcqlQYNIPJPau1wz\nMRlGcpiAiImj6bgz98nW6weRgnm2TINIwbUaRqdiAiImmjITU6BBRGZSt/9jTbrURjkPIpnjG4Zh\nAiI2JSd9iXLFkhPKg0ibBuH/TWYu+WyGZXOncdDeUxM5vmEY1jAoNqkrtRHKpM5I5Z10GvIggjv8\nhG5BshnhtkuOS+bghmEApkHEJm0mJjcPwnVSZzOCiKSqgJ019DGMsY8JiJikLVEu3DAoEAzeYpyG\nKKak8yAMw0ieFC156abkaKpCKrNeqY1iSQPB4AuwNGkQaXrPDMNoDBMQMVFNV1KWryUMlkoRGkT7\nP9akndSGYSRP+1eSMUIay30DDBadkEM4jT6INk/EMIwRYwIiJmkrtZGLEBD+33T4IMxJbRhjHRMQ\nMXGcdNnTfWEwEBIQvmBIQ5hruR9Ee+dhGMbIMQERE1VNVeVQPzFusOiUndSp8kGkx9xlGMbIaP9K\nMkZIW6Jc4IMoOUN8D2lYlH0Zlab3zDCMxjABEZOSpivM1TcnDRSc4HkmVXkQydZiMgwjeVouIERk\nnojcLSKPi8ijIvK33vgVIvKCiDzkPU5v9dzqoSmNYhqI0iBS4IOoNnsZhjH2aEctpiLw96r6exGZ\nAjwoInd5r31ZVb/QhjkNS9rKffuO6IFCKdAY0hTFZHkQhjH2abmAUNWNwEbv+S4ReRzYt9XzaBQn\nZWGuft/pwZITlPr2F+U0CDKxPAjDGPO01QchIguAw4DfeUOXiMgjIvJtEZnRtolF4JbaaPcsykTl\nQfjRS2mKYkqT38YwjMZo20oiIpOBW4APqepO4BvAgcByXA3jizX2u1BEVonIqi1btrRsvmkrtRHO\ngwic1ClqolM2MbV3HoZhjJy2CAgRyeMKh++r6o8AVHWTqpZU1QG+BRwZta+qXqOqK1R1xezZs1s2\nZ7eaa3pWu7AGUXZSu+NpuGvPZMxJbRhjnZb7IMRdvf4DeFxVvxQan+P5JwDOAla3em71SJuJydcg\ndvQV2HtqjzsW6gnRbqQNTupCocD69evp7+9v2TkNI8309PQwd+5c8vn8iPZvRxTTa4B3An8UkYe8\nscuAc0VkOaDAWuC9bZhbJI6jFB0lnwLbvs9+MycyuTvHXx62D//w+oMA9649DRFMEPZBtO6c69ev\nZ8qUKSxYsCAVWpRhtBNVZevWraxfv579999/RMdoRxTTvUDUr/eOVs+lFiVHKZQcevJZADbt6qfk\nKHtP62nzzMocMHsyqz91asVYmjSIdoS59vf3m3AwDA8RYY899mA0vtr03BKniC/f9SdO+fL/oqoA\nPL+tD3Dv2tNMJiPkUlIwql3lvk04GEaZ0f4e0rGapAjHUW5+cD3Pb+tj/cuuYHh+Wy8A81IuINKk\nQVi5b8MY+7TDB5Fq/vD8y7y403VyPrphB/NmTmTdtl5EYJ/p6TExRZHLpscH8fpD9gJIVeSXYRiN\nYRpEFf/zyIt0ZTNkMxW03aUAABLhSURBVMLqF3YC8PzLvew9tYfuXLbNs6tPJkUaxJJ9p/HhUxa1\nexotZe3atSxZsqShfa677jo2bNgw7DaXXHLJaKbWECeddBKrVq2qu81XvvIVent7m3req6++mu9+\n97vA0PdlwYIFvPTSS3X3f+ihh7jjjta4MuN8bs3k/PPP5+abbx71No1iGkQIx1Hu+ONGTlg0m/Uv\n9/LHF3YAsH5bH/NmpNu8BG7oa1o0iHbzqf9+lMc27GzqMQ/ZZyqXv2lxU4953XXXsWTJEvbZZ5+m\nHjdpvvKVr3DeeecxcWLzfhcXXXRR8Hwk78tDDz3EqlWrOP30+HU+i8UiuVzjy+BY/dwaxTSIEKs3\n7ODFnf2ctmRvluw7jdUv7EBVef7lXubOnNDu6Q1LRtLjpB6vFItFVq5cybJly3jrW98a3GV/+tOf\n5tWvfjVLlizhwgsvRFW5+eabWbVqFe94xztYvnw5fX19PPDAAxx77LEceuihHHnkkezatQuADRs2\n8IY3vIGFCxfy0Y9+tO4cJk+eHDy/+eabOf/88wH3DvOiiy7i+OOPZ9GiRdx+++0A9PX1cc4557Bs\n2TLe/va309fXF+x/8cUXs2LFChYvXszll18OwFVXXcWGDRs4+eSTOfnkkwG48847OeaYYzj88MM5\n++yz2b17d8WcNm/ezBFHHAHAww8/jIiwbt06AA488EB6e3u54oor+MIXvhD5vgB87Wtf4/DDD2fp\n0qU88cQTFccfHBzkk5/8JDfeeCPLly/nxhtv5P777+fYY4/lsMMO49hjj2XNmjWAu7ifffbZvOlN\nb+L1r389juPwvve9j8WLF3PGGWdw+umnB3fiDz74ICeeeCJHHHEEp556Khs3bqw5v2oWLFjAZZdd\nxjHHHMOKFSv4/e9/z6mnnsqBBx7I1VdfDbihqB/5yEdYsmQJS5cu5cYbbwzGL7nkEg455BDe+MY3\nsnnz5uC4UXNKDFUds48jjjhCm8mX71qjCy69XV/a1a/X3vuMzv/Y7frcS6/ogktv1y/duaap50qC\nS295RN/+zf9r9zTaxmOPPdbW8z/77LMK6L333quqqhdccIF+/vOfV1XVrVu3Btudd955etttt6mq\n6oknnqgPPPCAqqoODAzo/vvvr/fff7+qqu7YsUMLhYJee+21uv/+++v27du1r69P99tvP123bl3N\neUyaNCl4ftNNN+nKlStVVXXlypV66qmnaqlU0j/96U+67777al9fn37xi1/UCy64QFVVH374Yc1m\ns8Gc/HkXi0U98cQT9eGHH1ZV1fnz5+uWLVtUVXXLli16/PHH6+7du1VV9corr9RPfepTQ+Z1yCGH\n6I4dO/RrX/uarlixQr/3ve/p2rVr9eijj1ZV1csvvzx4v8Lvi3++q666SlVVv/71r+u73/3uIce/\n9tpr9f3vf3/wv//+qaredddd+uY3vznYbt999w2u7aabbtLTTjtNS6WSbty4UadPn6433XSTDg4O\n6jHHHKObN29WVdUbbrgheJ+q5xfF/Pnz9d/+7d9UVfVDH/qQLl26VHfu3KmbN2/W2bNnq6rqzTff\nrH/+53+uxWJRX3zxRZ03b55u2LBBb7nllmD8hRde0GnTpg07p5UrV+pNN900ZB5RvwtglcZYY83E\nFOLuJzazfN509pjczdK50wC487EXUU1/BBPAJ884hKLjtHsa45p58+bxmte8BoDzzjuPq666in/4\nh3/g7rvv5nOf+xy9vb1s27aNxYsX86Y3vali3zVr1jBnzhxe/epXAzB16tTgtde97nVMm+Z+Jw85\n5BCee+455s2b1/D83va2t5HJZFi4cCEHHHAATzzxBPfccw8f/OAHAVi2bBnLli0Ltv/hD3/INddc\nQ7FYZOPGjTz22GMVrwP89re/5bHHHguue3BwkGOOOWbIuY899ljuu+8+7rnnHi677DJ++tOfoqoc\nf/zxseb+5je/GYAjjjiCH/3oR8Nuv2PHDlauXMmTTz6JiFAoFILXTjnlFGbOnAnAvffey9lnn00m\nk2HvvfcOtKI1a9awevVqTjnlFABKpRJz5syJNVefv/iLvwBg6dKl7N69mylTpjBlyhR6enrYvn07\n9957L+eeey7ZbJa99tqLE088kQceeIB77rknGN9nn3147Wtf27Q5NYIJCI8tuwZ4eP0O/t5zrB48\nZyrduQzf+vUzAMybkX4T04SuLJBuR3qnUx13LiL09/fzvve9j1WrVjFv3jyuuOKKyHIgWqdrYXd3\nd/A8m81SLBZjzaH6PFHzixoHePbZZ/nCF77AAw88wIwZMzj//PNrzvuUU07h+uuvrzkngOOPP55f\n//rXPPfcc5x55pl89rOfRUQ444wz6u7n478Hw12/zyc+8QlOPvlkbr31VtauXctJJ50UvDZp0qSK\n+UehqixevJjf/OY3seZXb86ZTKbiM8xkMhSLxZrnhujPpBlzaoRxb7DuL5RYt7WXH656HoCTX7Un\nABO7cvzTmUvYtHMAGBsahNF+1q1bF/x4r7/+eo477rhgUZ01axa7d++uiDSZMmVK4Gd41atexYYN\nG3jggQcA2LVrV6yFsJq99tqLxx9/HMdxuPXWWyteu+mmm3Ach6effppnnnmGgw46iBNOOIHvf//7\nAKxevZpHHnkEgJ07dzJp0iSmTZvGpk2b+MlPfhI576OPPpr77ruPp556CoDe3l7+9Kc/DZnXCSec\nwPe+9z0WLlxIJpNh5syZ3HHHHYHmESZ8/LhU77Njxw723ddtNXPdddfV3O+4447jlltuwXEcNm3a\nxK9+9SsADjroILZs2RJ8noVCgUcffXTE84vihBNO4MYbb6RUKrFlyxbuuecejjzySE444QRuuOEG\nSqUSGzdu5O677x52TkkwLjWIRzfs4G9veIhNO/vZ1V/+Ae47fQKL9ymr9W979TyeeHEX//PHDew1\nNd05EEY6OPjgg/nOd77De9/7XhYuXMjFF1/MxIkT+Zu/+RuWLl3KggULAhMSlB3HEyZM4De/+Q03\n3ngjH/jAB+jr62PChAn8/Oc/b3gOV155JWeccQbz5s1jyZIlFQ7jgw46iBNPPJFNmzZx9dVX09PT\nw8UXX8wFF1zAsmXLWL58OUce6RZSPvTQQznssMNYvHgxBxxwQMVCfuGFF3LaaacxZ84c7r77bq67\n7jrOPfdcBgbcG6rPfOYzLFpUGea8YMECwF0UwV2Y169fz4wZQ1u/VL8vcTj55JO58sorWb58OR//\n+Mf56Ec/ysqVK/nSl74UmGiieMtb3sIvfvELlixZwqJFizjqqKOYNm0aXV1d3HzzzXzwgx9kx44d\nFItFPvShD7F48eIh85swYWQWhrPOOovf/OY3HHrooYgIn/vc59h7770566yz+OUvf8nSpUtZtGgR\nJ554IkDdOSWB1FNx0s6KFSt0uHjtKNa/3Ms//8/j7Dmlmz2n9jB7Sjezp3RzyJypkYKgWHIsOmgM\n8Pjjj3PwwQe3exqp5fzzz+eMM87grW99a7unkjp2797N5MmT2bp1K0ceeST33Xcfe++9d7un1RSi\nfhci8qCqrhhu33GpQcydMZFvnHdE7O1NOBhGZ3PGGWewfft2BgcH+cQnPtExwmG0jEsBYRidwFFH\nHRWYdHz+8z//k6VLl0ZuX88OP97x/Q4j4ayzzuLZZ5+tGPvsZz/LqaeeWmOPsYMJCKOjqBcJ1Gn8\n7ne/G34jI3GqAwHSxGhdCGY7MTqGnp4etm7dOuofhWF0Auo1DOrpGXmAjWkQRscwd+5c1q9fP6oG\nKYbRSfgtR0dK6gSEiLwB+Cpuxte/q+qVbZ6SMUbI5/Mjbq1oGMZQUmViEpEs8HXgNOAQ3D7Vh7R3\nVoZhGOOTVAkI4EjgKVV9RlUHgRuAM9s8J8MwjHFJ2gTEvsDzof/Xe2MBInKhiKwSkVVmazYMw0iO\ntPkgouITK0JSVPUa4BoAEdkiIs+N4nyzgPptqjoPu+bxgV3z+GCk1zw/zkZpExDrgXAN47lAzb5+\nqjp7NCcTkVVx0s07Cbvm8YFd8/gg6WtOm4npAWChiOwvIl3AOcBtbZ6TYRjGuCRVGoSqFkXkEuBn\nuGGu31bV5GrZGoZhGDVJlYAAUNU7gDtadLprWnSeNGHXPD6wax4fJHrNY7rct2EYhpEcafNBGIZh\nGCnBBIRhGIYRybgUECLyBhFZIyJPicil7Z7PaBCReSJyt4g8LiKPisjfeuMzReQuEXnS+zvDGxcR\nucq79kdE5PDQsVZ62z8pIivbdU1xEZGsiPxBRG73/t9fRH7nzf9GLxIOEen2/n/Ke31B6Bgf98bX\niEiqC/iLyHQRuVlEnvA+72M6/XMWkb/zvterReR6EenpxM9ZRL4tIptFZHVorGmfrYgcISJ/9Pa5\nSiRmTXxVHVcP3Oiop4EDgC7gYeCQds9rFNczBzjcez4F+BNuHavPAZd645cCn/Wenw78BDcp8Wjg\nd974TOAZ7+8M7/mMdl/fMNf+YeAHwO3e/z8EzvGeXw1c7D1/H3C19/wc4Ebv+SHe598N7O99L7Lt\nvq461/sd4D3e8y5geid/zrhVFJ4FJoQ+3/M78XMGTgAOB1aHxpr22QL3A8d4+/wEOC3WvNr9xrTh\ngzgG+Fno/48DH2/3vJp4fT8GTgHWAHO8sTnAGu/5N4FzQ9uv8V4/F/hmaLxiu7Q9cJMofwG8Frjd\n++K/BOSqP2fcsOljvOc5bzup/uzD26XtAUz1FkupGu/Yz5ly6Z2Z3ud2O3Bqp37OwIIqAdGUz9Z7\n7YnQeMV29R7j0cQ0bL2nsYqnUh8G/A7YS1U3Anh/9/Q2q3X9Y+19+QrwUcDx/t8D2K6qRe//8PyD\na/Ne3+FtP5au+QBgC3CtZ1b7dxGZRAd/zqr6AvAFYB2wEfdze5DO/pzDNOuz3dd7Xj0+LONRQAxb\n72ksIiKTgVuAD6nqznqbRoxpnfHUISJnAJtV9cHwcMSmOsxrY+aace+IDwe+oaqHAa/gmh1qMeav\n2bO5n4lrFtoHmITbCqCaTvqc49DodY74+sejgGio3tNYQETyuMLh+6r6I294k4jM8V6fA2z2xmtd\n/1h6X14D/IWIrMUtCf9aXI1iuoj4yZ/h+QfX5r0+DdjG2Lrm9cB6VfUbUd+MKzA6+XP+c+BZVd2i\nqgXgR8CxdPbnHKZZn+1673n1+LCMRwHRUfWevGiE/wAeV9UvhV66DfCjGFbi+ib88Xd5kRBHAzs8\n9fVnwOtFZIZ35/Z6byx1qOrHVXWuqi7A/fx+qarvAO4G3uptVn3N/nvxVm979cbP8aJf9gcW4jrz\nUoeqvgg8LyIHeUOvAx6jgz9nXNPS0SIy0fue+9fcsZ9zFU35bL3XdonI0d77+K7QserTbsdMm5xB\np+NG+zwN/GO75zPKazkOV118BHjIe5yOa3v9BfCk93emt73gdu17GvgjsCJ0rL8GnvIeF7T72mJe\n/0mUo5gOwP3hPwXcBHR74z3e/095rx8Q2v8fvfdiDTEjO9p4rcuBVd5n/V+4kSod/TkDnwKeAFYD\n/4kbidRxnzNwPa6fpYB7x//uZn62wArvPXwa+Feqgh1qPazUhmEYhhHJeDQxGYZhGDEwAWEYhmFE\nYgLCMAzDiMQEhGEYhhGJCQjDMAwjEhMQhgGISElEHgo96lb5FZGLRORdTTjvWhGZNdrjGEYSWJir\nYQAisltVJ7fhvGtx49hfavW5DWM4TIMwjDp4d/ifFZH7vcefeeNXiMg/eM8/KCKPebX5b/DGZorI\nf3ljvxWRZd74HiJyp1dw75uE6uSIyHneOR4SkW+K2+8iKyLXidsP4Y8i8ndteBuMcYoJCMNwmVBl\nYnp76LWdqnokbgbqVyL2vRQ4TFWXARd5Y58C/uCNXQZ81xu/HLhX3YJ7twH7AYjIwcDbgdeo6nKg\nBLwDN3t6X1VdoqpLgWubeM2GUZfc8JsYxrigz1uYo7g+9PfLEa8/AnxfRP4LtwQGuCVQ3gKgqr/0\nNIdpuI1h3uyN/4+IvOxt/zrgCOABr9nXBNzibP8NHCAiXwP+B7hz5JdoGI1hGoRhDI/WeO7zRtza\nOEcAD3qVROuVWI46hgDfUdXl3uMgVb1CVV8GDgV+Bbwf+PcRXoNhNIwJCMMYnv+/vTtUqSAK4jD+\n/a9BBIMYbOKrWO2C+CiCgj6BLyGY7GIRLReLiMGnMJg0GMZwzsVFDjeJxe9Xlt3DLmyaM8Mw52By\nnU8XksyA7aq6pR1gtAGsA/e0EhFJdoHXaud0TJ/v0QbuQRvGtp9kq69tJtnpHU6zqroCTmgjvqU/\nYYlJataSPE3ur6tq0eq6muSBtqE6/PHeCnDRy0cBzqvqLckp7fS3Z+Cd77HNZ8BlkkfgjjbSmqp6\nSXIM3PSg80nLGD76dxabuaPf+2VpOdtcpSVsQ9V/ZolJkjRkBiFJGjKDkCQNGSAkSUMGCEnSkAFC\nkjRkgJAkDX0BjsVoBxuBP1gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "training_episodes, test_interval = 10000, 50\n",
    "agent = DQN_agent(env_CartPole, hyperparams_CartPole)\n",
    "result = agent.learn_and_evaluate(training_episodes, test_interval)\n",
    "plot_result(result, test_interval, [\"batch_update with target_model\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "# Part 2: Distributed DQN\n",
    "***\n",
    "\n",
    "Here you will implement a distributed version of the above DQN approach. The distribution approach can be the same as that used for the table-based distribution Q-learning algorithm from homework 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## init Ray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ray.shutdown()\n",
    "ray.init(include_webui=False, ignore_reinit_error=True, redis_max_memory=500000000, object_store_memory=5000000000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Distributed DQN agent\n",
    "The idea is to speedup learning by creating actors to collect data and a model_server to update the neural network model.\n",
    "- Collector: There is a simulator inside each collector. Their job is to collect exprience from the simulator, and send them to the memory server. They follow the explore_or_exploit policy, getting greedy action from model server. Also, call update function of model server to update the model.  \n",
    "- Evaluator: There is a simulator inside the evaluator. It is called by the the Model Server, taking eval_model from it, and test its performance.\n",
    "- Model Server: Stores the evalation and target networks. It Takes experiences from Memory Server and updates the Q-network, also replacing target Q-network periodically. It also interfaces to the evaluator periodically. \n",
    "- Memory Server: It is used to store/sample experience relays.\n",
    "\n",
    "An image of this architecture is below. \n",
    "\n",
    "For this part, you should use our custom_cartpole as your enviroment. This version of cartpole is slower, which allows for the benefits of distributed experience collection to be observed. In particular, the time to generate an experience tuple needs to be non-trivial compared to the time needed to do a neural network model update. \n",
    "\n",
    "<span style=\"color:green\">It is better to run the distributed DQN agent in exclusive node, not in Jupyter notebook</span>\n",
    "```\n",
    "Store all of your distrited DQN code into a python file.\n",
    "ssh colfax (get access to the Devcloud on terminal)\n",
    "qsub -I -lselect=1\n",
    "python3 distributed_dqn.py\n",
    "```\n",
    "\n",
    "<img src=\"distributed DQN.png\">\n",
    "\n",
    "For this part of the homework you need to submit your code for distributed DQN and run experiments that vary the number of workers involved. Produce some learning curves and timing results and discuss your observations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from memory_remote import ReplayBuffer_remote\n",
    "from dqn_model import _DQNModel\n",
    "import torch\n",
    "from custom_cartpole import CartPoleEnv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the Env name and action space for CartPole\n",
    "ENV_NAME = 'CartPole_distributed'\n",
    "\n",
    "# Set result saveing floder\n",
    "result_floder = ENV_NAME + \"_distributed\"\n",
    "result_file = ENV_NAME + \"/results.txt\"\n",
    "if not os.path.isdir(result_floder):\n",
    "    os.mkdir(result_floder)\n",
    "torch.set_num_threads(12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0, dtype=torch.uint8)\n"
     ]
    }
   ],
   "source": [
    "print(1.0 == FloatTensor(1)[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "a =[True,False]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "if 1==a[0]:\n",
    "    print(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (Intel, 2019 update 2)",
   "language": "python",
   "name": "c009-intel_distribution_of_python_3_2019u2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}