批量归一化和残差网络.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"9B8F749E4C954AD8AFE6B96DD1F8B830","mdEditEnable":false},"source":"# 批量归一化（BatchNormalization）\n#### 对输入的标准化（浅层模型）\n处理后的任意一个特征在数据集中所有样本上的均值为0、标准差为1。  \n标准化处理输入数据使各个特征的分布相近\n#### 批量归一化（深度模型）\n利用小批量上的均值和标准差，不断调整神经网络中间输出，从而使整个神经网络在各层的中间输出的数值更稳定。"},{"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"3E15ABE688814DCB811DF2D4D3A0BBDB","mdEditEnable":false},"source":"### 1.对全连接层做批量归一化\n位置：全连接层中的仿射变换和激活函数之间。  \n**全连接：**  \n$$\n\\boldsymbol{x} = \\boldsymbol{W\\boldsymbol{u} + \\boldsymbol{b}} \\\\\n output =\\phi(\\boldsymbol{x})\n $$   \n\n\n**批量归一化：**\n$$ \noutput=\\phi(\\text{BN}(\\boldsymbol{x}))$$\n\n\n$$\n\\boldsymbol{y}^{(i)} = \\text{BN}(\\boldsymbol{x}^{(i)})\n$$\n\n\n$$\n\\boldsymbol{\\mu}_\\mathcal{B} \\leftarrow \\frac{1}{m}\\sum_{i = 1}^{m} \\boldsymbol{x}^{(i)},\n$$ \n$$\n\\boldsymbol{\\sigma}_\\mathcal{B}^2 \\leftarrow \\frac{1}{m} \\sum_{i=1}^{m}(\\boldsymbol{x}^{(i)} - \\boldsymbol{\\mu}_\\mathcal{B})^2,\n$$\n\n\n$$\n\\hat{\\boldsymbol{x}}^{(i)} \\leftarrow \\frac{\\boldsymbol{x}^{(i)} - \\boldsymbol{\\mu}_\\mathcal{B}}{\\sqrt{\\boldsymbol{\\sigma}_\\mathcal{B}^2 + \\epsilon}},\n$$\n\n这⾥ϵ > 0是个很小的常数，保证分母大于0\n\n\n$$\n{\\boldsymbol{y}}^{(i)} \\leftarrow \\boldsymbol{\\gamma} \\odot\n\\hat{\\boldsymbol{x}}^{(i)} + \\boldsymbol{\\beta}.\n$$\n\n\n引入可学习参数：拉伸参数γ和偏移参数β。若$\\boldsymbol{\\gamma} = \\sqrt{\\boldsymbol{\\sigma}_\\mathcal{B}^2 + \\epsilon}$和$\\boldsymbol{\\beta} = \\boldsymbol{\\mu}_\\mathcal{B}$，批量归一化无效。\n\n### 2.对卷积层做批量归⼀化\n位置：卷积计算之后、应⽤激活函数之前。  \n如果卷积计算输出多个通道，我们需要对这些通道的输出分别做批量归一化，且每个通道都拥有独立的拉伸和偏移参数。\n计算：对单通道，batchsize=m,卷积计算输出=pxq\n对该通道中m×p×q个元素同时做批量归一化,使用相同的均值和方差。\n\n### 3.预测时的批量归⼀化\n训练：以batch为单位,对每个batch计算均值和方差。  \n预测：用移动平均估算整个训练数据集的样本均值和方差。\n### 从零实现"},{"metadata":{"id":"39F35F2C0FE148B498B49A9E7D5A96C1","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"collapsed":false,"scrolled":false},"cell_type":"code","outputs":[],"source":"#目前GPU算力资源预计17日上线，在此之前本代码只能使用CPU运行。\n#考虑到本代码中的模型过大，CPU训练较慢，\n#我们还将代码上传了一份到 https://www.kaggle.com/boyuai/boyu-d2l-deepcnn\n#如希望提前使用gpu运行请至kaggle。","execution_count":2},{"cell_type":"code","execution_count":1,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"96E72C7A9429485180F9B63F7FB07CB9","scrolled":false},"outputs":[],"source":"import time\nimport torch\nfrom torch import nn, optim\nimport torch.nn.functional as F\nimport torchvision\nimport sys\nsys.path.append(\"/home/kesci/input/\") \nimport d2lzh1981 as d2l\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\ndef batch_norm(is_training, X, gamma, beta, moving_mean, moving_var, eps, momentum):\n    # 判断当前模式是训练模式还是预测模式\n    if not is_training:\n        # 如果是在预测模式下，直接使用传入的移动平均所得的均值和方差\n        X_hat = (X - moving_mean) / torch.sqrt(moving_var + eps)\n    else:\n        assert len(X.shape) in (2, 4)\n        if len(X.shape) == 2:\n            # 使用全连接层的情况，计算特征维上的均值和方差\n            mean = X.mean(dim=0)\n            var = ((X - mean) ** 2).mean(dim=0)\n        else:\n            # 使用二维卷积层的情况，计算通道维上（axis=1）的均值和方差。这里我们需要保持\n            # X的形状以便后面可以做广播运算\n            mean = X.mean(dim=0, keepdim=True).mean(dim=2, keepdim=True).mean(dim=3, keepdim=True)\n            var = ((X - mean) ** 2).mean(dim=0, keepdim=True).mean(dim=2, keepdim=True).mean(dim=3, keepdim=True)\n        # 训练模式下用当前的均值和方差做标准化\n        X_hat = (X - mean) / torch.sqrt(var + eps)\n        # 更新移动平均的均值和方差\n        moving_mean = momentum * moving_mean + (1.0 - momentum) * mean\n        moving_var = momentum * moving_var + (1.0 - momentum) * var\n    Y = gamma * X_hat + beta  # 拉伸和偏移\n    return Y, moving_mean, moving_var"},{"cell_type":"code","execution_count":2,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"04A65ABC9F20404E83F166F43226873D","scrolled":false},"outputs":[],"source":"class BatchNorm(nn.Module):\n    def __init__(self, num_features, num_dims):\n        super(BatchNorm, self).__init__()\n        if num_dims == 2:\n            shape = (1, num_features) #全连接层输出神经元\n        else:\n            shape = (1, num_features, 1, 1)  #通道数\n        # 参与求梯度和迭代的拉伸和偏移参数，分别初始化成0和1\n        self.gamma = nn.Parameter(torch.ones(shape))\n        self.beta = nn.Parameter(torch.zeros(shape))\n        # 不参与求梯度和迭代的变量，全在内存上初始化成0\n        self.moving_mean = torch.zeros(shape)\n        self.moving_var = torch.zeros(shape)\n\n    def forward(self, X):\n        # 如果X不在内存上，将moving_mean和moving_var复制到X所在显存上\n        if self.moving_mean.device != X.device:\n            self.moving_mean = self.moving_mean.to(X.device)\n            self.moving_var = self.moving_var.to(X.device)\n        # 保存更新过的moving_mean和moving_var, Module实例的traning属性默认为true, 调用.eval()后设成false\n        Y, self.moving_mean, self.moving_var = batch_norm(self.training, \n            X, self.gamma, self.beta, self.moving_mean,\n            self.moving_var, eps=1e-5, momentum=0.9)\n        return Y"},{"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"4FE611706B7F4C1B8B57182BC1CE450B","mdEditEnable":false},"source":"### 基于LeNet的应用"},{"cell_type":"code","execution_count":3,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"1B18E8F1B4C843E6852A0B50CFFD9C53","scrolled":false},"outputs":[{"output_type":"stream","text":"Sequential(\n  (0): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))\n  (1): BatchNorm()\n  (2): Sigmoid()\n  (3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n  (4): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))\n  (5): BatchNorm()\n  (6): Sigmoid()\n  (7): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n  (8): FlattenLayer()\n  (9): Linear(in_features=256, out_features=120, bias=True)\n  (10): BatchNorm()\n  (11): Sigmoid()\n  (12): Linear(in_features=120, out_features=84, bias=True)\n  (13): BatchNorm()\n  (14): Sigmoid()\n  (15): Linear(in_features=84, out_features=10, bias=True)\n)\n","name":"stdout"}],"source":"net = nn.Sequential(\n            nn.Conv2d(1, 6, 5), # in_channels, out_channels, kernel_size\n            BatchNorm(6, num_dims=4),\n            nn.Sigmoid(),\n            nn.MaxPool2d(2, 2), # kernel_size, stride\n            nn.Conv2d(6, 16, 5),\n            BatchNorm(16, num_dims=4),\n            nn.Sigmoid(),\n            nn.MaxPool2d(2, 2),\n            d2l.FlattenLayer(),\n            nn.Linear(16*4*4, 120),\n            BatchNorm(120, num_dims=2),\n            nn.Sigmoid(),\n            nn.Linear(120, 84),\n            BatchNorm(84, num_dims=2),\n            nn.Sigmoid(),\n            nn.Linear(84, 10)\n        )\nprint(net)"},{"metadata":{"id":"F11883676ED64A2DB05BC3480AF28BFF","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"collapsed":false,"scrolled":false},"cell_type":"code","outputs":[],"source":"#batch_size = 256  \n##cpu要调小batchsize\nbatch_size=16\n\ndef load_data_fashion_mnist(batch_size, resize=None, root='/home/kesci/input/FashionMNIST2065'):\n    \"\"\"Download the fashion mnist dataset and then load into memory.\"\"\"\n    trans = []\n    if resize:\n        trans.append(torchvision.transforms.Resize(size=resize))\n    trans.append(torchvision.transforms.ToTensor())\n    \n    transform = torchvision.transforms.Compose(trans)\n    mnist_train = torchvision.datasets.FashionMNIST(root=root, train=True, download=True, transform=transform)\n    mnist_test = torchvision.datasets.FashionMNIST(root=root, train=False, download=True, transform=transform)\n\n    train_iter = torch.utils.data.DataLoader(mnist_train, batch_size=batch_size, shuffle=True, num_workers=2)\n    test_iter = torch.utils.data.DataLoader(mnist_test, batch_size=batch_size, shuffle=False, num_workers=2)\n\n    return train_iter, test_iter\ntrain_iter, test_iter = load_data_fashion_mnist(batch_size)","execution_count":4},{"cell_type":"code","execution_count":5,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"9F9DC5C22F5942A48A8A4B656B33D543","scrolled":false},"outputs":[{"output_type":"stream","text":"training on  cpu\nepoch 1, loss 0.5476, train acc 0.824, test acc 0.829, time 86.9 sec\n","name":"stdout"},{"output_type":"error","ename":"RuntimeError","evalue":"DataLoader worker (pid 76) is killed by signal: Killed. ","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mConnectionResetError\u001b[0m                      Traceback (most recent call last)","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py\u001b[0m in \u001b[0;36m_try_get_data\u001b[0;34m(self, timeout)\u001b[0m\n\u001b[1;32m    723\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 724\u001b[0;31m             \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_data_queue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    725\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/multiprocessing/queues.py\u001b[0m in \u001b[0;36mget\u001b[0;34m(self, block, timeout)\u001b[0m\n\u001b[1;32m    112\u001b[0m         \u001b[0;31m# unserialize the data after having released the lock\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 113\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0m_ForkingPickler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloads\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mres\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    114\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/reductions.py\u001b[0m in \u001b[0;36mrebuild_storage_fd\u001b[0;34m(cls, df, size)\u001b[0m\n\u001b[1;32m    293\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 294\u001b[0;31m         \u001b[0mfd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdetach\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    295\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/multiprocessing/resource_sharer.py\u001b[0m in \u001b[0;36mdetach\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     56\u001b[0m             \u001b[0;34m'''Get the fd.  This should only be called once.'''\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 57\u001b[0;31m             \u001b[0;32mwith\u001b[0m \u001b[0m_resource_sharer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_connection\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_id\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mconn\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     58\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mreduction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrecv_handle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/multiprocessing/resource_sharer.py\u001b[0m in \u001b[0;36mget_connection\u001b[0;34m(ident)\u001b[0m\n\u001b[1;32m     86\u001b[0m         \u001b[0maddress\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mident\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 87\u001b[0;31m         \u001b[0mc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mClient\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maddress\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mauthkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mprocess\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_process\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mauthkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     88\u001b[0m         \u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mos\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetpid\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/multiprocessing/connection.py\u001b[0m in \u001b[0;36mClient\u001b[0;34m(address, family, authkey)\u001b[0m\n\u001b[1;32m    497\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mauthkey\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 498\u001b[0;31m         \u001b[0manswer_challenge\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mauthkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    499\u001b[0m         \u001b[0mdeliver_challenge\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mauthkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/multiprocessing/connection.py\u001b[0m in \u001b[0;36manswer_challenge\u001b[0;34m(connection, authkey)\u001b[0m\n\u001b[1;32m    740\u001b[0m             \"Authkey must be bytes, not {0!s}\".format(type(authkey)))\n\u001b[0;32m--> 741\u001b[0;31m     \u001b[0mmessage\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconnection\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrecv_bytes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m256\u001b[0m\u001b[0;34m)\u001b[0m         \u001b[0;31m# reject large message\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    742\u001b[0m     \u001b[0;32massert\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mCHALLENGE\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mCHALLENGE\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'message = %r'\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/multiprocessing/connection.py\u001b[0m in \u001b[0;36mrecv_bytes\u001b[0;34m(self, maxlength)\u001b[0m\n\u001b[1;32m    215\u001b[0m             \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"negative maxlength\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 216\u001b[0;31m         \u001b[0mbuf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_recv_bytes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmaxlength\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    217\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mbuf\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/multiprocessing/connection.py\u001b[0m in \u001b[0;36m_recv_bytes\u001b[0;34m(self, maxsize)\u001b[0m\n\u001b[1;32m    406\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_recv_bytes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmaxsize\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 407\u001b[0;31m         \u001b[0mbuf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_recv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    408\u001b[0m         \u001b[0msize\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstruct\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munpack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"!i\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbuf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetvalue\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/multiprocessing/connection.py\u001b[0m in \u001b[0;36m_recv\u001b[0;34m(self, size, read)\u001b[0m\n\u001b[1;32m    378\u001b[0m         \u001b[0;32mwhile\u001b[0m \u001b[0mremaining\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 379\u001b[0;31m             \u001b[0mchunk\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremaining\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    380\u001b[0m             \u001b[0mn\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchunk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mConnectionResetError\u001b[0m: [Errno 104] Connection reset by peer","\nDuring handling of the above exception, another exception occurred:\n","\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)","\u001b[0;32m<ipython-input-5-4ecbbea10726>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mlr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_epochs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0.001\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m5\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0moptimizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptim\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mAdam\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnet\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mparameters\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlr\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mlr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0md2l\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain_ch5\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnet\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrain_iter\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtest_iter\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbatch_size\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moptimizer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdevice\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_epochs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m","\u001b[0;32m~/input/d2lzh1981/utils.py\u001b[0m in \u001b[0;36mtrain_ch5\u001b[0;34m(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)\u001b[0m\n\u001b[1;32m    233\u001b[0m     \u001b[0;32mfor\u001b[0m \u001b[0mepoch\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_epochs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    234\u001b[0m         \u001b[0mtrain_l_sum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrain_acc_sum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbatch_count\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstart\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0.0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0.0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 235\u001b[0;31m         \u001b[0;32mfor\u001b[0m \u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtrain_iter\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    236\u001b[0m             \u001b[0mX\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mX\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdevice\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    237\u001b[0m             \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdevice\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py\u001b[0m in \u001b[0;36m__next__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    802\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    803\u001b[0m             \u001b[0;32massert\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_shutdown\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tasks_outstanding\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 804\u001b[0;31m             \u001b[0midx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    805\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tasks_outstanding\u001b[0m \u001b[0;34m-=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    806\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py\u001b[0m in \u001b[0;36m_get_data\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    769\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    770\u001b[0m             \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 771\u001b[0;31m                 \u001b[0msuccess\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_try_get_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    772\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0msuccess\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    773\u001b[0m                     \u001b[0;32mreturn\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py\u001b[0m in \u001b[0;36m_try_get_data\u001b[0;34m(self, timeout)\u001b[0m\n\u001b[1;32m    722\u001b[0m         \u001b[0;31m#   (bool: whether successfully get data, any: data if successful else None)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    723\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 724\u001b[0;31m             \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_data_queue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    725\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    726\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/signal_handling.py\u001b[0m in \u001b[0;36mhandler\u001b[0;34m(signum, frame)\u001b[0m\n\u001b[1;32m     64\u001b[0m         \u001b[0;31m# This following call uses `waitid` with WNOHANG from C side. Therefore,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     65\u001b[0m         \u001b[0;31m# Python can still get and update the process status successfully.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 66\u001b[0;31m         \u001b[0m_error_if_any_worker_fails\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     67\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mprevious_handler\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     68\u001b[0m             \u001b[0mprevious_handler\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msignum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mframe\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mRuntimeError\u001b[0m: DataLoader worker (pid 76) is killed by signal: Killed. "]}],"source":"lr, num_epochs = 0.001, 5\noptimizer = torch.optim.Adam(net.parameters(), lr=lr)\nd2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)"},{"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"AC692570D35A492BB82A50E9BBBBB598","mdEditEnable":false},"source":"### 简洁实现"},{"cell_type":"code","metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"EAC5ACDF548B4831992A653DF4FD4348","scrolled":false},"outputs":[],"source":"net = nn.Sequential(\n            nn.Conv2d(1, 6, 5), # in_channels, out_channels, kernel_size\n            nn.BatchNorm2d(6),\n            nn.Sigmoid(),\n            nn.MaxPool2d(2, 2), # kernel_size, stride\n            nn.Conv2d(6, 16, 5),\n            nn.BatchNorm2d(16),\n            nn.Sigmoid(),\n            nn.MaxPool2d(2, 2),\n            d2l.FlattenLayer(),\n            nn.Linear(16*4*4, 120),\n            nn.BatchNorm1d(120),\n            nn.Sigmoid(),\n            nn.Linear(120, 84),\n            nn.BatchNorm1d(84),\n            nn.Sigmoid(),\n            nn.Linear(84, 10)\n        )\n\noptimizer = torch.optim.Adam(net.parameters(), lr=lr)\nd2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)","execution_count":null},{"attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAnsAAAHaCAYAAAB4nI9BAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAGUtSURBVHhe7d0NuFVlnffxW7MipcQiw8TEwsTCRMXExBENE0dUfMSCUUN8SR01NfURUx5w0NRRU1NHSUlIDEgcKTQxUChxwMTEEZMSkxKTMZooaSIH89nf+6z7sNjufd6PnL3O93Nd69p7r7POPvuQ59f/fl1bvFUSJEmSVEhbZo+SJEkqIIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPbWJDRs2hLVr12avJKmY1qxZkz2TaofFntrEggULwtSpU7NXklQ869atC+eff372SqodFntqE1OmTAkTJ07MXklS8UyfPj0eq1evzs5ItcFiT622fv36MGvWrLBs2bKwZMmS7KwkFcuMGTPilBVHMVRrLPbUajNnzozDG5g0aVJ8lKQiWbVqVZg3b158bs6p1ljsqdUYwk0Y4qCnT5KKZPLkydmzEJYvXx4WL16cvZI6Pos9tUq+tQtW5FLwSVKR5Bu1cI6yaonFnlol39pNykNRkmrZwoULw4oVK7JXdfLTV6SOzmJPrXLPPfdkzzZiG5byYJSkWlUp5yj0HMVQrbDYU4sxZ4W5K126dAndu3eP5/r27RsfncAsqQiYg5yKuj59+sRHc061xmJPLZaGa4cPHx622Wab+PyUU06Jj2xNwBYFklTL2FaKuci9e/cOAwYMiOdGjRoVttpqq/oGr9TRWeypxbp27Rp79I4//vjw17/+NWy55ZbhoIMOCr169Yqh6O3TJNU6Gq0UehR4L7/8cjz3+uuvh2HDhoWBAwc6b081YYu3SrLnUrOl3rtdd901rFy5MsyfPz8WegztSlJRkHWnnXZaXJQ2bty4MGbMGHNONcOePbUKQxkceQagpKIx51TLLPYkSZIKzGJPkiSpwCz2JEmSCsxiT5IkqcAs9tQmyicvS1LRmHOqVRZ7ahM9e/bMnklSMe24447ZM6m2WOxJkiQVmMWeJElSgVnsSZIkFZjFniRJUoFZ7EmSJBXYFm+VZM/1Dtliiy2yZ6oV/plIzWPO1R5zrrjs2ZMkSSowe/Y2g9Ti/cp8/+k7um8fXPe/lX8mUvOYc7XDnCs+e/YkSZIKzGJPkiSpwCz2JEmSCsxiT5IkqcAs9iRJkgrMYk+SJKnALPYkSZIKzGJPkiSpwCz2CujPr6wIKxfOyl5t9LulC8L//Pfq7NXbvfnG+uxZ072+emWLvq+5/vev68Kv5kyOj+XW/nZ5/N0kdR5kwYuPTg9vrFubnanzhxVLYyZU05K8Wv+nNfF4J/z6JzNjhpfj9yTX34m8VfF4B43NoK12lv/ZHZfEAPjcV28OW75rq+xs3XmKn2G3LsrO1Jl85HZh0MV3hV4Dh2VnNlr6vavDi/NnhC9MuD+8v0ev7Gydv/x+Vfj7mxvqA48wfe35J8Jrv1gci8f9z7oh7DH8vOzqjR44/+DsWeM+edio8MkhJ2Wv6hB4P7r48HDcd56NP2fayF3CyGkvhXe/r2u477S9wjG3Pxm2/mCPsPj2i8JvFz0Qvjjl+ew72447y0st01Y5x9/2f37/+nDI2Gnx7z2h8Jk3/rgwavYfYyYkPx57TMwwcqnc6mcXxu85+Ot3hx33GZydrUO2UUBy8PyPLy2LOfdfpZwjiz5+0PAwePy92dUbLbr1/JiJTfGh3v3e9rnI1n8v5Rmfia+Tc/uMGhfzkAzte+y5MbP5d5jz9SPDl2f9PnTZtnv23W3DnCs+e/Zq2C7/cGwMvEcnjIytPXq+CAce161eGZ+ngoswojD88G794+tynzrqjBiQFFG8JwH03WEfjiFwzxd3igF0/xn7hodKxdfyH00K7+naLfQ/eUIssCoVethhz4MaPegZrNYr94Ede4euH+4ZfvHD27MzdShmd/rskPrgpxXPtZKKp3sps8gjsoyGJ1nGc4qsLUqN3IdLBRCvU8FFnnT/5D7xebntPzUg7H7k6bFoemry+Pi+FIfkHHlHzs08eY/4fk9NuTwWfp8edlZsOFNsVvKhT+xZMdvyB/hcfyvrhQQNdbKcn5f3wtypsZH7sf2HxtfkHEVeWxd66hzs2dsM2vKekbQ4CS5C7P0f2Tm2RPcceXH82l9LQfFIqRDk51AwLfnO2NgqbAgB+PzsibGIoxBL4URhRZjyfvSulff+Ndfvf7kkPP6tc8KGUpj+w4V3xM9fCS1xfscd+g2q79l7btatoV/pd0yhx3mKz14HHB1fJ7SGaSm3hi1eqWXaMucouuZ/48Tw3y8tCweef1t8ni++aPCmnr/vj9o9DP/Os+GDu/TNvvp29JI9WnqPId+YHbYpNSjJOry3lCNkCZnCKEj5aENzUaw9cftF4eWfzYk9erseekL2lU3REP/51CvDgDOujT+bnj10LeXsR0vZhwXXjA6vPDUv9PnHU+LrhIyrNFrTHOZc8VnsbQZtfYPwNA/v+VJB98rPH6kPA4YiGOLk5zB0wVyQPAo2iqdyBA+BV46WKS3exoo9fs62O/auWGgR1hSdhNZeJ1waPvOlC2PLljB/qfR9uxw0vH5IhsKSoWLQAqdVTzCnIWuGN/occWq46x/fH38WQQ0+J8UjgUkPYGsYglLLtHXOkQH0bpFr5BmFUULOMcTK0CuNyHKVPkO1nAN/940VezREya1KGcNnfLpUvC1/8M5Y4H32tKvqfxb52L2UV2k04tmZN4Zl990Un4Oc49qUg2TZ50uFLCMrvG/K3jWlxjfFLVNg+v3TmHiupcy54nMYt4YRNgQgf/BpSJPC73fP/CQe9PKB8GBolpbl0BvmxwCjOCJAEoZ+U+u2WgA2FS1Zfh49chR2fMaEuTf8HFreBNSKuVNjUBNitFx5TOi9O6gUuJ84ZER2JoTdSp+dcxy0ZtPQzRHXzY2/G3MOwe/W2kJP0uZHgUeu0chLvXVMW0k5x8Hrt0o582ypaCLfyALyju/heUIukZtobc4xneWZadfEgo+pJanRDXoOKerIpYHn3xbWlTKYApV8Ix/TZwA5Rp4xLYaRlK1KRR4FXso5cpDfj3+H/UoFLr8PB1Nc+FprCz11DhZ7NYqAoeeL4Yt8MdVjj4GxRcpBMIBhTzAvjyEBhj+7faxP/dApQUJw/eCs/et70hjOpbWXP+jVA8MM5V/j+nKEFoFX/hkpNGmd0gP30+tOi3NeKuE6ilhaxwd89eZ4jrCmFUzQ8R58blBYIhWLDfU8SqodFFJkT76YIsNSznHw+reLH4w5sOvg42POMbrASEAaBsWrpcKQ92I+HMig8izjAI3P8vMpA/Pe9Z4u8eeWf0Z+PhnLOeY68xnT1JM8soq51OTavqWCjxEK5lwzJ5EClhxk1TGZn3IOFJAM80pNYbFXo+jip9VICzFfTNFDl4KJogxpgQOrykBBRAgmhBXvRaH44IWHxgIttY7zR1pFRq9Z+dcqDXdQqPE15ueVF3y0UpkYzVButXksFHazSgUow7EEHui5Y/iD8/RYEoJ8/lVPzYtfJwBTL6ek2scqVYqkfDHF6EC+COM1OUdPXuo1I+fKG31kGHOEf1Iq5Bh1YDi1PMs4QI9Z+fmUgXn8TOYLkkvlBR8FGgtIyC92TaiE0Qka2nzWTx9zdjz3mS9eEA8ykgb4L2ZPjDnHNB3wvgxDm3VqKou9GkbvHAFEqPGHv89J4+PcFFqTe594WXw++kevxy0Ddj/qjPC7LChoHbKYI48gIbCYA0cRRfDQIs4fqeDi/cu/Vq0njTA6/JqHwkf3/nxYUyr6Vv1sTiz0CKs9jj03fuZqWJ1Gjx4BSKDyM/iczNVhGJhikFYwxWD+d3ufASgVBg1bMoS/f3q8yBuyjWKMBirPORi5oBhLBdGff/dixSygYUrW/amUFYw+lGcZB7rttNvbzqcMLEc+0Qhm/hyLzijOKEA5mE7C17imEkZeWICWFpzQEOdz0Qj+0tQX4nMKPhamUMgyEpN6+Cz21FQWezUuFXz0mtGrx0EAMEzBcxY9sHXAxwYcEYsjrqMlWWmrEsKIsKy2lUpL8RlpmbNnHlslUPTRss7PxauE7Q5Sr9/SadfEz5Z6JPkdCV5av1xDCFKkrnry4arby0iqTango6GZcg783dOjx+ue+x4WCzlGEuhd47FaI5QGcEMFWEvwGZl/x+gIOyHwGejpY+uVtNiiEhq0HHwWpqzsWGoY8/lAzpGdFLL8buQfOc7qXjK8tfMO1Xm4GnczaOtVaqCAY44HAcPB8ERCa5NVrt8d9uHYOmS1Gluw0EtWjtWyv54/I+w9alwsypgXmNCi5L0JnHxIMjcwhRMYPuZn0mtHcfnzKZfHQCZcX5h3T5yHwns8d/8tcYiGgKQFzPflV/qmhRtgIQet+PwcFXr40hwYQpWvs7KNod7yDVNbiiEi+GciNU975Bz4W99QyiJ6u1IvHGj80SD84VcPrN+3jq1VyIVy5BJ5yQIHsihNeUnII7IlX6SRofkVwMzpo6eRxjaYUvLYDWeGvU+4NH7vrx6eEnOQvKVRmkYx+Flp02RQ4K1e9nh8zi4F/Mw0nxrsrpB+B+ZGM++Q7KZRy6KOtmDOFZ89ewVA+BFOhM4nDv5S+EgpKHieDkKFAKEnjQKKQqhSoYdfzLo1thwp5ghU3pdg4mD7ACZDM4k4nePrDFtUQhizQSmtUO52kQowevd4L1qzrFBjAjYFYDUEIK3YVOgxZE1LPj8H8FPDzgrPzLgutnTbqtCT1HGQXcyzI9PIIDIsn3NpPh3FEduekGH5oimPEQ+2qiKLQI6RjynXyDkKu/San5UanuX4XhZgUDweOv7euJ1UQvby2di7lGvIrmrYNobpLfnPTAHJiEXCdBx6M1lYQiEpNZXFXo1jqII/fIZK8ceVz8UwoPXIQSs3IQQJLIZ0KyGIflkqothhPo/AqnZU8+b//i0OZxBcR33rsfreunxhyPczT4UevrTAIo+wpZXM56JXMK28o6CjNzA/X4XXBLLbrUjFw6gCDbytu+8YX5N7NBBTznGkooiijevJgmrDtGzRUp4haRi40pG/rlzaUiXfoKVoS+iBo+BjrjLTUcoxT5pFI3wPRWzKuY/ueVCck8znTPgcPUu/F72RlabiSNVY7NW4X/5oUiyoKPYIOHrRaImyPxOhl28VMkSLNGRQjkIP1VbHNse73v3ecNg3ZsfQogij944h4d8semCTrVYY/iUkqxVp7CpPryStZ3ryGKal5zE/nAL+j4CikJYw10oqDv7mKYZ6Z/N8mZtLvpFzjBKkfAALL8hCCkIeyzHlhVGH3cruRNFS9CjSoKUAI4cY3qVnkdcJ+Xz0rYvisHElTFOhGAQ5RoOd4V9W8OZHYchwRjpYoMG/idRUFns1jCBbUSpu0h0zmPtB4NHVz1wWVpMlzLWj4GJlK8FCUViOVWEEaFtN+s3Pp9mu16fj52HolpZsXuqVrIQg504ZDJEw55BH3iPfqiUk03la1rS080O8kmob899oGJJNFGsUQwxjkjH5YU/+7hfecGa8nyz5WH6/WZBzjDS01SgA8+lSQUYusdMBc/TSPqcJvXLVspXcpWHMnOa7j90hjopwLj8XmmKX28TtXPrdKDDZo7Ta0LJUzgUam0FbTVxmxS1/8Cy24I+e+XH06qU5IxR3tG5ZBMG8PoKR3dyXfu/q2PKk5y0VZOz4zj12WQGbwpPhYb6PYYxqaMkSUvlrGFIh1KptU1AJw7vs71d+KzY+P72X/K5so0AvH3Nbeh96QhziZcd59rHiM7Mwg3+H+07bK/5shn7zE6xbwonLUsu0Vc4xOnHPF3eKe4GyRQm3DaPXjMUXoOHKfnQnzf5jLIbo5WPYlKKQ/KLBmDKRfOC9yMn8nSf4O6f4q7ZtE1tGMXxK1iX04KXzzcF8QRqm+czkd6SxzibxFHUM/dKDRz6zWI4GMY1Yrjv2jqdjvjIHkN81bUvTGuZc8dmzV8Mogmj9UdAwcXm7XfqGvsPPi4URmw5T1PXoe0Bs6RKOaVNPgo9gy290zL5Q3Ioo30renBjKINQ5KDoJxy9OeT4GHffAJMgZ0iDM+cwUtGAVHOGXtneRVNsogihm6LWncUmBw64C6e+fQo+vseKfHj8asWQiPW70rrGwI01h4fuZ69dQA/adRtFIA5keR0Zp/qnU4KWwTUXdq6X8S/MCOZ96B2nMknfMy640XC3l2bO3GbRVi5eCh+EDWn355wxrMneN52wXQOHD8/wQAkUe51NxR5DQasz3xqUJxfnWbDmCKr81AAhXisz8xOLG8PP5vvx2KoQ3W8g0VIDSG8jPKZ+IzWfn92ntylxbvFLLtFXOMQWFv2d6uej1ovhJw7Q0aOmNY54xrykEy/fZpABM+cQ1jBbk8wrkGPPpyMlK+DnkUn4KCkUoudXce9OSc70OOLr+MzCqQh7zO1VbUEIvH9mcsjHh34OewtYWr+Zc8VnsbQbttf+U2p4hKLWMOVc7zLnicxhXkiSpwCz2JEmSCsxiT5IkqcAs9iRJkgrMYk+SJKnALPYkSZIKzGJPkiSpwNxnbzNI+0+pdvhnIjWPOVd7zLnismdPkiSpwOzZU5t54IEHwtChQ7NXklQ85pxqkcWe2sTKlSvDgQceGF566aWw1VaV7+8oSbVul112CQ899FDo06fyfXSljshhXLWJyZMnh1WrVoU5c+ZkZySpWBYsWBAbtpMmTcrOSLXBYk9tYsqUKZs8SlLRpHybOnVq2LBhQ3wu1QKLPbVaau2C+Sxr1qyJzyWpKNatWxdmzpwZn69evdpRDNUUiz21Wr43b/369fWBKElFQa5R8CX33HNP9kzq+FygoVYh/HbYYYdNQnDAgAFh0aJF2StJqn0HH3xwHMVIunTpEl599dXQrVu37IzUcdmzp1Ypb+1i8eLFYcWKFdkrSaptTFPJF3pwFEO1xGJPrZKGcNN2K+nRhRqSioLdBmDOqVZZ7KnFUmuX4Bs8eHA8lzYbTeEoSbUuFXVDhgypfyT3Fi5c6CiGaoLFnlosFXQE33vf+974/KCDDgrdu3ePe+6VD3tIUq1Juw2Qa4ccckh2dmPDlm1YpI7OBRpqMbZYmT59eujbt2+49dZb4/yVG2+8MWy77baha9euMQyZxCxJtYq5eWwptXbt2vC3v/0tnH322eGYY44JX/va18LSpUvDiBEjYiEodWQWe2oTo0ePjj19d911VzjppJOys5JUHGQcWUfGkXVSrXAYV5IkqcAs9iRJkgrMYk+SJKnALPYkSZIKzGJPklQTuFsPq/5ZKJHu3LNhw4Zm7XW3bNmy7JnUeVjsSZI6vNWrV8ctT/r37x8mTZoUbrnllnju/PPPj1s9NRUbIbvpuzobiz1JUod35plnhqOPPjr06tUrjBs3LvzjP/5jOPLII8Oll14aevTokV3VuDPOOCMWiXfeeWd2Rio+iz1JUofG0OusWbPC8OHD42tuz0iRN2rUqGYVesmFF14YN4JfsmRJdkYqNos9SVKHNmPGjNCnT5/6wm7evHnxNmb00uXRY5eGabnzBZjbxy3NuIVjwn1tzz333HDRRRdlZ6Ris9iTJHVI3I6RO1Zcd9118TXPGX6dOHFi7N2jaEu4nRm9dccdd1z9ddzScd999w0nnnhiHAbOGzBgQCwYly9fnp2RistiT5LUIXHf2WuvvTb20l1wwQXxFmWnnnpq7Nnbc889s6vqdOvWLUyYMCGcddZZ8fUjjzwSCzyGehm2TecTegrB8LBUdBZ7kqQOi2FZDBw4MD7Sg8dRzaBBg+LjAw88EA466KAwZsyYWDAOGTIkns/r3r17ePHFF7NXUnFZ7EmSOqynnnoq9tqlnjiGZhvSt2/f+Mj+e5UKvDy2bFm5cmX2Siouiz1JUoe1dOnS+t46NLb6lgIu7bvH9zaE4eGePXtmr6TistiTJHVYDOPut99+2au6Yo7h17/97W/ZmU3deOONsScQTzzxRHxMK3PLsXp35513zl5JxWWxJ0nqkLgNGvPz8j17YL+9fK/dnDlzwsiRI8PVV18dCzz24AOrbRcvXhyuvPLK+DovrcJtbKhXKgKLPUlSh0RBR08et0jLO+WUU2IRx7w83HPPPXGblptuuinccMMNsRjk+9g0mb30UvGXRyHI/D62YJGKzmJPktQh/eQnP4m9evn99EDxN2zYsPp73FLgzZ49O7zwwgtxDh7DvE8++WSYP39+eOyxx0KXLl3idQlFInvy8X1SZ2CxJ0nqkNgDj168Sm6++ebYo8e8O4q7oUOH1i/MAKt3y4d/E4Z72XePjZmlzsBiT5LUYey///71t0Pr3bt37MGrhMJu2rRpcT4eBV9TcWcNev/Kb7UmFZnFniSpw2CIlh67KVOmxKHZhrANC0OxDW2yXI7NmU866aTsldQ5WOxJkjoMhme5LRpHfli2GubzpQ2Xm6I510pFYbEnSZJUYBZ7kiRJBWaxJ0mSVGAWe5IkSQVmsSdJklRgFnuSJFWxatWqcMstt4Tzzz8/fPvb347n2APwzDPPjJsz5+/RK3VUFnuSJOVwOzVuxXbggQeGnXbaKZxzzjnhxhtvDIsWLYpfX7lyZbj99tvDJZdcEvbaa6+w6667hiuuuKJZ+/11FGxIvXDhwrBu3brsTIj3FK7F30XVWexJkpSZPn162H333cPo0aNjEbTlu7YKH+03KOx94mXhgK/eHPY9+Yow8Lx/C/uMGhc+ftDw8O73dQ0rVqwIY8eODbvssksYP358LBZrAUXsHnvsEYtailk+98iRI8O+++4bDxXHFm+VZM+lFiMYaQmzEaq700uqNRQ6FD/02OH9PXqFvU64NHzikBGxoKvm729uCK88NS88PfXKsPrZhfHcgAEDwv333x/v8NHRcfu40047Ld655IgjjgjPPfdc6NKlS/jEJz4RC1cVg8We2oTFnqRaxRDmMcccE+/JS0/e3qPGhT2/dGF413u6ZFc0za9/MjMsvOHMsP5Pa+L9d+fOndvh79hBryTD0BR43IeY+w2reBzGlSR1aiy2oNCjB+8LE+6PQ7bNLfTAsO5R33osdPtYn7iw4/DDD+/wc9969+4db0u3fv36cPrpp2dnVTQWe5KkTouFFVOnTo3F3eHXPBQ+tv/Q7CstQ6E39Ib54QM79o4LOZgD19Hn8PXq1Ss+urK4uCz2JEmdEkOYl19+eXw+6OK7Qo89BsbnrbX1B3uEId+YHd7TtVuYM2dOLCY7KraRoRcSTz31VHxU8VjsSZI6pYsuuij2un1yyElxIUZboofvc2fdEJ+zRQvDpB0FRS6/+8yZM2Oxe/PNN8fzFH7MX2RlsYrFYk+S1OkwZDlr1qy6BRknXJqdbVu9Dz0hfKh3v7iXXVrl2xGwvcx1110Xh5jHjRsXhg8fHrp16xZ7+Fisccopp2RXqigs9iRJnc59990XH3c5aHicX9dUD5x/cPjDiqbNbaOQ/MwXL4jPZ8yYER87grPPPjvcdttt4emnnw6DBg2KK3Hnz58fzz377LP1c/hUHBZ7kqROh7l02OUfjo2PTfW7pQvC39Y1fYXtzvsPjUXf4sWL6+fGbW704p1xxhmhb9++2ZkQ+vXrF8917949O6MisdiTJHUqa9asibcEYwXuTp8dkp1tHyzS2P5TA+Jz5sRJm4PFniSpU0k9bKyabejuGG0lDROzFUtjli9fHhd0sCWM1FYs9iRJnUoq9rgl2jvh/R/ZOT7+4Q9/iI/l6Gm85ZZb4v1ouS/v1VdfHW666aaauceuOj6LPUlSp9LUu1pMG7lL+PbBW2xygEUalc43Jv9z2YqF1cDcpm2HHXaI9+VlaJnFEqyO5daTUlvx3rhqE94bV1KtYO7cwQcfHLdFOfaOp7Ozb/faLxaHDW9suj8ehd7+Z90Qvzfvo/0GZc/ebtGt54dnZ95Yv83JxIkT4/Yn9OiVY4HENttsE/7617+GrbfeOjur9rDVVlvFhSkHHHBAGDJkSIe/j3FrWOypTVjsSaoVzJ3bZZdd4py9E+57NTvbNPTicTu0hoq7cvPGHxd+/ZOZ4corrwyvv/56vKNGYytzt9xyy/D3v/89e6X2RuE3ZsyYcOmll8be1aKx2FObsNiTVCuYC8fQKT1r9OyV99I1pCXF3neHfTis/9Oa8Nhjj4WBA+tuyTZv3rwwZcqUeBeL/N016GliU+NDDjnEnr12xrA6Q+ePPPJI7GlF//79w6JFi2LxVyQWe2oTFnuSasmJJ54Ye9j2PvGy0P/kCdnZxjW32GMoeNZZ+8fh2VdfffVtRQS3J6PQoPBbuHBhdjaEnj17hhdeeKGQvUwdEfsu8v9j3O2E4fbx48dnXykGF2hIkjqdww47LD6ueHR6+Pub7bfq9RezJ8bHYcOGVewt6tq1azj11FNjrx/F3WWXXRYLPXr4LPTeOczZu+OOO+Jzhtubsk1OLbHYkyR1OiyUoKj68ysrwoq5U7OzjRs57aXwkWyT5Mak96bIO+uss7Kz1fXu3TtMmDAhvPzyy/WFh945Q4cOjQfD/Ple1iKw2JMkdTr0mlFY4akpl4f/+e/V8Xlj2JuPO280ht7Cx791Tnw84YQTYk9dc/To0SN7pncSK3Px1FNPxceisNiTJHVKFGFMyH999crw6ISRbTqcu+Q7Y8PLP5sT70PLHDDVhlRkL1u2LD4WhcWeJKlTYnh19uzZ8f/gf7d0QSz43izbV68l/nPGdWHp966Oz6dNmxZ69Xpn7tShtlO0u5dY7EmSOi0Kvfvvvz8O67IX3uzzD27ykG45egZ/et1pYfHtF8XX9Ogx8V/a3Cz2JEmd2oABA8L8+fNj4cdWKd8ftXv4+d1XNKuX78VHp8fvW/7gnbHH8Oabby7c9h2qXRZ7kqROj4LvySefjHP43li3Ns6549649NTR41fe28c1q59dGK+befIe4ZEJI+PqWwrGhx56KJx99tnZldLmZ7EnSVIJW7FQ8DHPjm1QKPDoqeN2Z1OP3SFuqEwBOOmw94XJR24XfvjVA2MP4H+/tCwuxLjqqqvCSy+9FAYPHpy9o9QxWOxJkpQzYsSI8Pzzz4e5c+fGHjoKv4SVu2l4l7tisKL37rvvjnvjcW9VN0JWR2SxJ0lSGebd0UPH3DvubMGdRSnouC0kvXe8/v3vfx8LPQo+7oQhdVQWe5IkNQHDvFdccUVYv77127NI7ySLPUmSmoBbaK1YsSJMmTIlOyPVBos9SZKa4J577omPU6dOLdymuyo2iz1JkhrB0O306dPj81WrVoUFCxbE51ItsNiTJKkRM2fODGvXrs1eBYdyVVMs9iRJakR5cUfxt27duuyV1LFZ7EmS1ACGbefNm5e9qpMf1pU6Oos9SZIawN56YO+9/KNDuaoVFnuSJDVg0qRJ8XHIkCH1jxR8bMWycuXKeE7qyCz2JEmqglW3FHTcGu2QQw7JzoYwdOjQ+Jh6/aSOzGJPkqQqBgwYEO69995w1VVXhS222CI7G8IFF1wQb6XGvXOljs5irwCWLVsWbrnllk1amGvWrAmrV6/OXjWMycdNvVZS50ZW3HnnnXE1asKqVHKkKZpzbUfQpUuXMHz48HDqqaeGbt26xXP08g0cODAWejyXOjqLvRpH4F5zzTUxiEaPHh2HHJhHcuWVVzY5hHr06BHfY8mSJdkZSXq75cuXh3POOSeMGDEinHbaaWHOnDmxsXnRRRfVF0KN6dq1a7wDhZsSS+8ci70axgafBO64ceNi6/P+++8PW265ZSz0rr322voVY43hOq6//vrrY3BLUiUjR44M5557bizYbrvttrDtttuGM888M+YH55pqzJgx8dZjNEwltT+LvRp2++23h549e4bevXvH1wwrjBo1KoZwUwu9hOuZk0LvoBuFSirHKALTQ8gZsEDh5JNPjrnRnEIv4ftorDqFRGp/Fns1jD2eBg0alL0KsWeO17169crO1M2PWbp0aQzq/Jy+NM8vX9jxfX379o1FpCTl0ROXCj2QEwzd5s8hbUCc33CYIpGh2/ztxphmMnjw4DgS0d7yOZgfveBz2ruozsBirwaNHTs2HHPMMXH+DMFFb9yjjz4aJ00fdNBB2VV1aDXfd9994bjjjovXcf0DDzwQ9tprrzj3hu/J22+//cLEiROzV5I6uxtvvDEO35IbbEFCjjBXj73nDjvssOyqOhRUNDqPPPLI+D0UUuTUHnvsEU488cRw+eWXZ1fWOeCAA2IRyN0o2hNFHvMKUw6C14ceemg4+OCDYzEqFZnFXg2aMGFC+NKXvhSf33HHHeGuu+4KH/jAB2Jg5Xv1wBAv16fWNwFN4PJ9F154YVxlltenT5+wYsUK5+5Jis4777xw/PHHhw0bNsQpIuQNOUERV543/fr1CzfccEMYNmxYfE1Dkzl9bFPCylXeJ4/r6e0rvxVZWzvppJPi5wYL0eiV5PMzlMz8QVfUqugs9mrUE088EVfRpvl6aRd35vBVwl5RoCeP0CP8mFRdfn0KPQo+SQI9dszLY5oHGssbRgjAVJFLL700NizZk65///7xfMLCMrwTd6HIz2+eMWNGXNBGoUdjWCo6i70atXjx4voCDmnuXbWFGSl807y8atJE6/zcGkmd21NPPRXzJuVL2ievWt7QYweKOeblNaahYVQ2Mm7q0ZiUfUcccUSzF7FJtcxirwYxv4WWdn5+XuqRq1ak0QsIhmcbWm2b5s40dc8sScXH3Dvm1yVp+LZalqS84esMlzamobx56623mnw0Jn2uxx9/PD5KnYXFXg2iYKMoy/fsMYcG1Yo9Jk2nQE3z8SoFddoGoXwujqTOiV48et7yeZPyoVqPHJu05xuYqLQIg3mAeCfyhqHitJFz2kCen58+g1RkFns1iFY2w635+S/MRaHgo8cvGT9+fFwBx6o4Vs2lbVpYpME2LJUmRdMKZ25LQ0O9kjoPCiOGPPNbrJAR5M8zzzyTnambD0zesFsA+ZQWf7FlC6t3WXVbjkKQod7y7VvaCsUon+mKK66Iq3C5xy2/CwUshR8rcp2yos7AYq8GMX+GeTDlc05Y6fbwww9nr+r24UsBe8YZZ4RTTjklPqfQI+DSirm8Rx55JJx11lnOZ5EUMeRJYVe+cTJ5km8wprxhixYWf6UdA2bNmhVX5XJLx3K8N7dea6/VsDR++UwUoOQjjdhUhLJ9Fb+DK3HVGVjs1SB69phgXI4tElhFm3r3nnzyyfDss8+GadOmxdfseM+5l19+OV5bju+lZ48tEiQJ9OyV76cHijd65VLBR8489NBDYdGiRfW9dfPnzw9PP/103OqpHA1OCkNW67YXRjPmzp0bXnjhhfpik8/CSlxy0BEMdRYWezWGQo65LyeccEJ2ZiNa3gxTsI8e81BosZaHGS30NJcmj+sZ0uD7y1vwkjonhkFZ+V8pb+j9T3nD/F+GdocMGVK/nQoottLK3HKXXHJJ3LcvbYfSHviMjILkfwb5xqiGOafOxGKvBjCxeIcddogtYfaHYiPQfKDmEazjxo2LQdrUicdcR2Cz31Ra6CGpcyJnuOMFucOdJ9iTs1pBxsIKCjbuxtPQKv9yV199dRzmzd/uUVL7sdirAWkyNBuU7rzzzjF8G0LBx9BIUyce03q/+OKLHdKQFHu86P2/6aabwnPPPRc3Q24IucE11VbmlqMoJMMs9KR3jsVeDWAoYvbs2eGyyy6LCy2agm1WmjrxmGB3SEMSyBvmuXHbM4q4pizWIj+aun1KKiYlvXMs9iRJkgrMYk+SJKnALPYkSZIKzGJPkiSpwCz2JEmSCsxiT5IkqcAs9qrYYostGj0aU+l7yo/GVPqe/NGYSt9TfjSm0vfkj8ZU+p7yozGVvid/NKbS95Qfjan0PfmjMS35Hqm9VfrvMn80ptL3lB+NqfQ9+aMxlb6n/GhMpe/JH42p9D3lR2MqfU/+aEyl7yk/GlPpe8qPxjT3erU/i71WaOg/4rb4D7wp71GUz9Da96jVz9DUzya1h/b677pca9/Dz1CnqT+nIW3xezSk0nu0xfuqdSz2qnjrrbeadFRT6dpKR0MqXV/pqKbStZWOhlS6vtJRTaVrqx3VVLq20lFNpWurHdVUurbSUU1zrpXeKZX+u6x0VFPp2mpHNZWurXRUU+naakc1la6tdFRT6dpqRzWVrq10NKTS9ZWOaipdW+loSHOu1TvHYk+SJKnALPYkSZIKzGJPkiSpwCz2pM3IOS2Sisy5ex2DxV4Vrh5SezMAtbmRc2adVHwWew0wBCVJUq2z2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkzYTFwBJKjpXfHcMFnvSZmQISpLa2xZvudlXRen/hNvjn8f/g6897fnfgX+C2lzMOeWZcyFMnjw5jB49OgwaNCjMnz8/O1v77NlrgP8nLEmSap09e5tBaul8Zb7/9B3dtw9u/54P/wRVROZc7TDnNrJnT5IkSTXHYk+SJKnALPYkSZIKzGJPkiSpwCz2pM3IxRmSpPZmsVdFWkEktRcLPUlFR86ZdZufxV4DLPgkFRkZZ85JxWexJ0mSVGAWewWx+PaLwsqFs7JXLbf2t8vDgmtGh/V/WpOdab7XfrE4/P3NDdmr1nvu/lvCU5PHZ6829ca6tfHz/vdLy7Izkopq+YN3hqXfuzp71TrkxupnF2avmu/Pr6wI//Pfq7NXrcdneWTCyKrZ2VYZr87JO2hU0Z67fqf3bsud5aeN3CV88rBRYZ+TKhdFTfW7pQvCA+cfHEZOeym8v0eveK5aoZX0GjgsfKh3v/j8zTfWh+kn7Bo+8qkB4ZCx08KW79oqngfh2pBeBxwd3yuPYu57pd/tw7v1D7sOPj47W+eTQ06KAfifM64LnzrqjPC+7T6SfaVO/nO1VHvuLC9tbrWWc2TIutUrw9AbWn9nA/62B118V8wR/GrO5PB66b2rIUvy+TRv/HGxkcln2fqDPbKzIRaja1/+Zfbq7brttFvo909jslcbkbv8/H1GjcvO1On52SFhzS+XhDlfPzLsVHq+/e77ZV+pU/65WsKc28g7aKjTWvn4D8JTUy4PLz/5cHj1mZ9sciyddk34w4ql2ZUhvOs9XcIR182NIUh4/e9f12VfaRl+LgXjW6XW7s/uuCQWd796eEo86IVcNvPGsH2psOTnPD31yviZXv+v38SjtT9bUufxhxefiXnz4vwZb8u5Z++7KeZg3kGlQvEDO/YO/37aXjGLWuPXP5kZe/ZoYJOpP73utPqco7j9j1vPj197X6mofH72xPgZU861ZhRGnYc9e1UUqWePr1Vy1LceC9t8uGf2qk6lnr2//H5VuOeLO4Vjbn8y9rAl9LpNPnK7cOwdT7+tB43hDd6Hluj+Z92QnW2eFx+dHoc1KB533GdwHB5+8MJD488jZGlZ/77U4h3+nWfDu9/XNRaCXMPv1Vba+56R/vlpcypSzx5Zwd9/uf3OuDZ8/KDh2auNynv2wKjEp448PXzmSxdmZ+qQoX2PPTfsMfy87EwdhlwfLf3cP72yIuZSS9Aw/sFZ+8f37n/yhJir9568R/jsaVeFXQ89IU5jWVQq9shfcva3ix4Ij37jxDBi6guhy7bds3dpHXv2Nipqz57FXhW1EII//OqBsRADj1uVCp73du0WX+PzY6fFXi/+kBk2YPgADDEw1JAKOoYOqhWEDClQQH532Idj+PQ54tTsK3WtUVqgJ83+Y3ZmUxR8FGEcLUGo0WIm0NLwCi3Z939k5/j8jb/8Kf6M9Hu9+b9/i63c3oOPDx/tNyiea632Lvbgn6A2l1rIORpxL5WyBvx9U2Dlh01TEUbj8j2l/GM6SEJxmC/o0t9zOfKCApIGJKMTB3/97uwrpczJ8rFSoxZ8HvI3NY6bi5GR5T+aFD93mkP4lzWvxN9xiy3fFd76+5thTemaD39yn/g1/Pl3L4Yd9/78JoVqa5hzG1nsdTK1EILPzrwxtgLj8/tuCt1LQbTDngfF1wxHEF6EGH/I6TnKe++YZ/dfWYv4t4sfjHPgKBQZMuDrHMwX2XbH3pv00i284cywrhRyQ74xOztTGcOpKawbQ8Fa3grns/6t9HtWCtpyq342J+xeapm3du5iYgiqyGoh51iUkKaKMJRK5jGKAYY504gGOUH+5f/2+fvNF3tkHyjO5n/jxNgI7rnvYbGRTL6QfS/Mu2eTXjpGGGjUjv7R69mZ6mgAb2ji9JFdSjmXbwgzN5ohWuboNYZ/Dz5zpR7OljDnNnLOnjocWrMEGwd/+Cno8mHXFLRkKQQ5mAgMWsi8Tq1VCsn83DwQnD36HpC9qkMrmPPpoOeNcKb4TAdz72hx58+lY1mpaK2Ez/HR0u/X2MHnllQcLD5IuUZBRhak1ymfmirlHCtpE16nhiSPf3xp2SYrYpmz12OPgdmrOmRaPufSnD3yK59n5Bx5lz+XjtRQz4u5WyHXyo/m/t6SxV4DaqnHhaKqtfM3CKw0jMD8l/y2BB8pFXW8TsOpzI3heubS5bGijRZ2OuhpY14gvYjpYNjlg7v03eRcOqrNt4tFZCl0GzsqBaik6mop5/7aBjlHIUePIOjJY4QkYU7yFu/aKvbmgVGPFaXnDJnmMayazzkWVYD8SlnG6AiLy/7PHU9vknHpKJ8vjVhEVsi18iPlsNRUFntV1FIAglBKQwI8b4nnZt0aeh96QnzOXL2HLj68vsXKQovtSgVa6nn7z+9fH1vE+QUboLXNsA1HpdYnQcvP+dSws+Jrwo15MhSrDbFnT2p7tZhzTPVINrQg6xgW3qGUXdjrhEtjL9sLc6fG1+QH2ziRb2BfP+TnKoPsSzlXbd4cU2sYqk3zC5l7+HKp8dsQe/bUXiz2CiBtKJxaio0VTpXQU/fKU/Pq97Jj6ISAYxVYssex54Zfzpkch3MJTHromotFF8zh2y0LSMKNSdcpXKuxZ08SWdc11yNGT19zkA9PlIquvUtFHiiamNP3+LfOqW8kf7rUEGUol+FZCjayqrmNSDKYjCQzk3d32SYs+c7Y7FVl9uypvVjsFQDFF8MFrLxtCUKOuSX9Rl4chzAStizIL8ig12+r93QJPx57TOzla+5GnvTqMX+FAGSeYLL3qHGxBd1QocZcGiZiN3awqERS8bCogoYhU0paiv3q6NVj66aEHGP4NWUSX+PcT0qZyM/cs5SLzUVRx8by+UxmjjVbtFBEVkN+Vcq18iPNMZSaymKvANb86qlYfLV0ixN66wi6NISbUEB2+1if7FXda3r7aFXS+m0uNkCmVy+/ezwtYFavUUTyOSohAF8tBSTDvcyPIYTLD86z3xUFo0O5UvGwpyYoolqCKSnMxdv35AnZmY2YQ5zHin5yjp0B8tu8NAWjJAwL5xvK5BK9kix0q7YIjdziOlYJpzwrzzmm1tDYJg9t2Ko5LPYKgHkg5QslmoP5J8zRo5hrCGH5ix/eHq9jQnJzdm6nNftkqbXL9i0Mmcw6a/+4d9/UY3cIc0tFHEH3TOk986vgEiY6M6F5wBnXxtflE505wO3ZeE4LWlKxrHry4ThHuKWNOXrsKPQqLYzIo0FKVpFzbKVSaaPmaugJZMNjCkT2zqM4Y0N6Np9nX1Qat+R1mnqTR26RX2mRWsqz/EGhy8gIz8lFqaks9mocwUERRtd+ufwGyw2h966xIVlCir32uPaLU56PQ7/0tFUqzsB5rkm3LGNuDfNj6EEkRNlAdOD5t8W7X5zy8F/jI0PIK7KJ0mAj0/zBBOdK5zlASzh/Lr/KrqOqtQny0uZAjxc9/5VyrqnFH7lTfmeMcuQWPWvcpYNVtDSE6UmjiKuGkYm0tx7fz60dyTqyj/vYshk9d78g58hOsjY/R5mdD/K5RVGI8jzjYD9U5hHmz/H9HZ05t/m5qXIVbATZXv80bbXZKGg5suHwsFsXZWdC3CKFwDj++y/HViwbZjZ0B408euAYQsh/jT2pKPTAJp60WpknyM9g/gstzPwQMj1+fC5arwQsX//Y/kOzr1bHezI0kYZN8nNbCFBCjfepFPh8ZoZN8nNZ+Pzlv19ztedmo9LmVgubKoO8YtXsife9Wl/c0ZhkdIBGI7cVIwP4WkN30Mgr33CZ4oxCj4Vqh1/zUNxbj3P3n7Fv/PoXJty/ybQW8P6pgcrKXnYjaAzDw3z29F7kHhme/Lz0e/K60m0mWTCX5i8naUPo1jDnNvIOGp1MLYRguncswcTWKKAwY7d3wiLtAs8fMgVUmoBM0NBT15RiL92Hkef8nPz8Fa59OOvtO+wbs+u/RqHHROQjS4Uhn5EhkU8cMiLe7oe5hY1hqCK/gIO5Ogz9ptZ2/jMk/I75u4S0FUNQRVYLOUem3XfaXuHTx5wde8lAg/LpqVfGaSVkFZlAbrH/XX5Eg8KqKcUeoyNpCyiKuvwmyvTqcU9uehcHj7+3/mupACUXU2OUomunfQ+LOZfPsErIzXyW8XPISopH3rPS1Bx+x/K7hLQFc24j76ChDoXhAoY1CUAKPZb58wfLjbwJrhSKSX7+B8+bgqLwsRvODDvvPzTOIykvsiisOE9Ipf2oKA5pGR9aCkV6FRk2OeK6uTEomcOSFllUOygU82jJcpNwtluoVuhJKi4KIObb9T95QswRco4ePTKH3rR8JqT5bOloKlbPMo2E4dbyu2WQYzQkeaTABMUmhR7ziMlBCjO+l0bxr396X2xwV8q3/EFOJzSKmdvHli9H37qoVXOwpUrs2auiFlq89HixiowWJIUZ8zkqdenTuiUQy3v2CC+GWfMIU1rHqXeN6xorsHj//HsRYuXDHS2V5srw/g3hzh3cU7Kti0FbvCqyWsg5Mog5cWm7lLRggpzL955xHav683P4yCbuuFFppwJGJlLvWppb3NCOBlzDkTKmLXOOxjvDueWb1JdjjjY/v7XDtuXMuY0cxu1kaiEE1f4MQRWZOSeYcxs5jCtJkqSaY7EnbSap50OSioqcM+s2P4s9aTMyBCVJ7c1iT5IkqcAs9hrgZFVJklTrLPaqsNCTVHTknFknFZ/FniRJUoG5z95m4KT82tMefybpvwP/BFVE5lztMefcZ0+SJEk1yGJvM6CFU8RjwoQJFc8X4ZDUPJX+jopwmHOqRRZ7ahOLFy8O11xzTVi3ru4ek2oaA1aqHevXrw/XX399WLBgQXZGqg0We1U436R5pkyZEgu96dOnZ2fUGAs9qbbMmjUrrF27NkyaNCk7o8bYa9gxWOw1wIKvaWjtpiKPok9SbSDjzLmmS/k2c+ZMRzFUUyz21GqptYuFCxeGFStWxOeSVBSrVq0K8+bNi8/zDVypFljsqdXKe/Ps3ZNUNFOnTg0bNmzIXplzqi0We2qVfGs3YZ8iSSqS8uLOUQzVEos9tQqFXb61CwpAV6tJKgoKu+XLl2evNqK3T6oFFntqldTa7dq1a3zs1q1bfHSIo3FOjJdqgzmnWmexpxZLwxgE4NChQ+O5ESNGxEdXqzWNBZ/UseUXY6R8GzJkSCz4Vq5c6ShGI1zx3TFY7KnFUsgRgK+99lp83rNnz9CnT584tLtkyZJ4TpJqFY1a8qx3795h1113jedeeeWV+sKPDeWljs5iTy122WWXheeffz5ceuml4WMf+1g8t+OOO4a77747vPzyy/FG0pJUywYPHhzzbNq0aWH77beP5z7xiU+Eiy++OObfmDFj4jmpI7PYa4C7fjeOXrxevXplr+r0798/dO/ePXslqSMz5xpHnpFreeQe+SfVAou9KgxASUVnzkmdg8WeJElSgVnsSZIkFZjFniRJUoFZ7EmSJBWYxZ60GTlBXlLRmXObn8VeFe74rfZmAGpz8+4Gam/mXMewRel/CP+XqCAFYHv88xiutcc/ExWROac8cy6EyZMnh9GjR8ebAsyfPz87W/vs2ZMkSSowe/aqeCdavD/8pf/0Hd1Ru7XffwfS5mbOCebcRvbsSZIkqeZY7EmSJBWYxZ60mTiBXVLRkXNm3eZnsSdtRoagJKm9WexJkiQVmMWeJElSgVnsVcESdJehSyoyc07qHCz2JEmSCsxir4at/5918Sj37M8WhBn/dkX2aqOHpt0eXv3NiuzVpt58c0P445rV2aum4frF82aFP/9xTXam7a1ZvSo+8jNuumR0/c/Kf9Zn/mNe/N0kFVOlbOLcXf960du+9vPH5sRMqCZlSlP95c9r43uu+vXy7Ezby38mfqdfPLUwPs//blxzz01jK2a+1BiLvRo2ccI5Yfyph7+t2Hpi3g/Csid/kr2q88bf1odJV50fXntlZXZmU/dPui6cM3SPTUKSaykcOR57cHp45N8nhx9OuTFc+7WR4dRDdgmjDtghfPOiE2PBVwnXNPXgfcv95oVl4Ywv7BoDj4Dj5/NI6J5+aN15PLngwfDwjInxuaRi4e/+vKP3Cr9+fml2ps6yJxbERt427++Wnalz37eveVv+Jb9cujicVsob8i4hRzhPzpFl/Dze95bLTouZOHLf7cKE048Ms797U/Ydm0p52JSDa8vR0L7g2H3rc/TxOTNjo5zMHvNPB9afX7FsScVGvNQU3i5tM2ir2wgRUoQQBdD4Ox+KITXtlsuzr27EzyHMLvrS/mHak38M23xg03BMCEBajseednH44j9fFguqSsXh3gcOKR2HhU/tMzDssnu/8K53bZV9ZVN8nob85fW1MZjfeGN9uOib0+L7lvvGWceE7XfsFY4adW4MyzsffSncXArhT+19QBh5zvh4DQXv+7buGi7+1r3xdVtqz9sIpf8O/BNUEbVVzlEMUXhRBF166/1hy622CpeeeHD21Y2uvHt+zKSR/bcLX7v27jBg8LDsK5uil+6mMaPDbv0GhHOvuivcWWoEV8qqj5eybb/PHx3fk2u7lDKmEt7vj79/e89jwuefM31ieKlUrI44e1z4Uilby9HYffT+KeHGWU/HnBtZuo6evP9c9Ej8vTDz21fH9yED25o5t1FRb5dmsbcZtFUIgkLv6q8eFz7Ss1fo9qGPhNd+95sYYKBQIzj4ORRy991xzSYF0Xve0yWGWN6T8x+IBSPFIyGF7br3iI+0fAlZwoYCrKV4X8L1u9dfEvb83OBw6tdvqP8Z5ejFo+eye4+e8XeZ+OMXwi+fWRw+d9jw8J73donX0MOInh/vEx+TY79yccUCsjnaOwT981NRtWXOgYbo4rmzYl7Q4MsXPWTDuVffFRt9Xzt231jsfaiUGckenx2UPatDrjAqcfrYm2Nu0PDMZxp/9+To5//PSdmZliEzv/0v54Qu23SNP6t33/7ZVzZFL95/PDwzDDrqhPpib/tSppOLKdfoFaSw/HiffvF1slep4T38K2OyVy1jsbeRxV4n057/R9zWIZiKsu//2xXhiUd+EFuj+Mvrf4otRn4ORRrBk0e4NaeV2NRij59Jj195wIJikpY0CL9UjDEvhu87atR59T2PlT5zHoHI9QyzHD7yjFjsgmKV84ccMyq2zlujPUNQKrK2zjlQFNHYo8f/qC+fm50t/Yzv3hS+fuv9sSes0lBncz9DU4o9hlfJrUrXUEzeNu7M8OvlS8OXv3ZVzKeEnOtbysaUTdNuHl9xRCYhR+ndO2/YXqFbqfjbbc/94vlH7p8S34Ocq9aL2VTm3EZFLfacs9eAFFYdFd36DOUyjFptKBXMfaNouvp7j8XQo0giQFKhR7HI4od8YZWfr5cOhiGQ5rfkj/xw7w+n3BTn0zDHZuzoQ+vn1uE/fnxf6P6RnuHm2c/GQo/ePVqstKwJPB6TC2+YFj8jrdbUA3nECWfH3j3OU9Dxs0GgMqw7JAtVevVaW+hJRUfGdfSce/zhmfULFlJvfiX/WyoEf/z9O8Px506IOUeBxNBrvtCj0PrxvXdmr+oameVZljLllZd++bbz+XmDNKwZemV0hZxLnxFkJLlHVlHo0chl/l9do/am+iwFOUaeMeqyw869Y28e+cyQLufJQUY3fvOrZWHIiNNjznG8a6utwoH/+KVWF3rqHCz2ahQt3CULHoy9X/liaoeP9Y4tTY7PHXZsPDe7FC5Ixc+f1/5hkyEOUCz+v5MODQt+ODW+ZpiV984fd36jrkeO4qz8a5XmvOz8yb7xfS8/9fAYiAnDE4Q2oUv4HnH8WdlXNkXo0ULmutH/99p4jtfjTzs8hi49gHNnTornmbyMtFil2rCwpNryq2eeiPNy88UUw7Up5zh4TU8XWVifc6Us2O7Dm+YAxR89bqx4pZFLz1t5lnGAxnT5+UlZBubxnn33PSh+RvIpIYM+sF33mFVk5qHHnVJxvnQ6x3ufeskN4d1ZQfv/SgUk8xRZgEK+8nlffO6p+DX8YfWq0K3s95OqsdirURRLzKsjTAghQo7QeKFU9LCwgnkrzEkhcCjg+NqzT9S1WAlB5vglFGRnX3FHnDzM9zAMklqb+YNFFKCHsPxrXF+O9x1Taq0ShgQhYfXmhrohZ1rJhC5DJUyALse1DG8QklzDnD2cU/qcLCC558axccXaz386J/5uzyx6JH6d36018wkldSw09L5w3Kn1BR9zjbcks4buEY+vl/KP1/9BYVTKgrQS98//vSaOIuTxPuTXgh9MjUPBzH8rzzIO8HPLz9PLVgmLLnhvsphRDvIL5BGNXb5WKSNBz+W5w/YKhw4/Jex78NB4jqHZsbfPDnNmTAyv/nZFbNTyuy3Ndkugh5AG/0fMOjWRxV4Ni0VUqeBjOJSWbSrQWPTARF+ef+enL4dTSq3FYaMvqC+ICKMPbr9jfJ5HYFFY9fxEnxgsFE35I7UiKbzKv8b1laTP+OULrooByxYuBDZFKnNsDjxiRHblpigUX/3ti2H8HQ/FAKS1y9AGjwTnjT94Ovy09F70EjKEm0JwxbNL3taal1TbWJRxzMkXxNxgSgfZ9sUzLw0f3bl3fSFGg5UGado+6ve/+03Mh3J8P0O8vT+9T8Wc4wA/q/x8QyMGfEYWiVDg0Rhl+gw9fZwffXHdyEQlq15cHgvLtLsAn48M4/HfHno+FnY0bPndyDde0yMJ5vBJTWGxV+MopggTlukz7y5tPExvHs/ZsoDg+IdSUcVqL1qDzBdhiLUShkQOOGx49qpt8BmfXvhwXJjBz13/l3Vx/yjmnDSEFXVprh6/x0FHHV8fthSDBx99QgzJA4YMj78T4frz0s/pVyp2JRULxRArU1POPf/zx+MQKaMRvGZ0g4Yv+UdxxLy5Hjt9IvvuTfE+qbhqS+Qr8/eYewd6I8lm8qqa1CsIRlVouKeFa3wfGUjGc478JNuZE82ISENzGKU8i70CYD4cf/Tbf3Tn2Ooj7JgHx2sOWq+EDy3Ve2+/MhZfqYgqR6/bhDOOjKHFUCurtNLBEAXYGiB/vtJ8vYSg4/2YoPzNmU/GVbp7/8OQuBKXYRSGaqvJzw3k+5l7mF5z0MvH5GR+L3r/Zn3n+vB86fPv/4W6uYodXUefGC91JGneL5lGhpEJLDbb4WOfiOdSQ5BtmRj+ZNVuGhYtx3uRITQQkc8zDlBA5s/xuhryknmATE2hB44h2Z137Ru+cff8uBnzNV89bpN5y3n5uYGz7ro+bq+SXnPQU8nCNDC8y157NGrTnGypKSz2CuCfD989zsdLLV8mKPM8HWkLFIog9tsjAKu1NCmo6HnLtxgZ8qh2NIRd6M85co84wZh5Mqm1C4aZ/+WuuXHLBFq/60pFajnCjHBb81+r4mcedvIF8fWn+g+MQc7cnYSvMUzMwpNaWoVrwSc1zb+VCikKLjJtz/0/Hxux+ZxjVALkxoNTb4lDnNWygAYq85vzc97oPauUcRwNZQrFGRlMUXbt9xfV98qBffWuv+/JOJzLvLxUXObRQ0eusRcfDVdWE/OakQxyLo8eQPKQ4VyK2lpQCyu+OwOLvRqXdm9PLVjChP2fUmuUXriEa2iB7j3wsOzMpuiF43tZ3p9HsVjtqOZ/3/hb+O43L4ktb4ZjaYmDXseEkLt2xqI4BPtKbhVbwnAyPYGsOjtz/G0xzAcdfUL4xZKFpc94xibzBOmppKdvtz0r91hKql2MWDANJe0hyqpUzuV73tL2T+QKWbDPP1TfUP3Be26N00BSLiHtDVrpKL8lW973b7sy7LHfoNigTfP98jlHjyNFII/0Rpbj8zJ3mQKOuc1phTFzDmm854vHOCpTyjgatWnRmtQUFns1ju0GKOJoERJ+tC4ZRmDCctquJLm3FEpgGKCSH5UCkAJqvzbYt+nd73lv3EsvrUCj9447XRB2u+99QDwHwoxJyNWGlbnrBwE399474zyc684fGefmMDk7jwnRzAFMC0AkFQeLsciB1KglR9KCNIqsPHr1yEK2LckXXQkNZHraGGptCxeW8paFbXw+pqXQwGZIl+Ix4WssVDvmlAuzM5uip5HPyqbQzDVkS6ofTL4xNnLz+Owc7Eeav7+v1BiLvQZ09N3ECYcn5s2KXf74/u1Xxhbo54YMjy1MCsC0CIKJv/TasWKNlVwEYh49fuxnR4uyrSb95luebHJMq5WevDQZOcn30JWjV5CDDUxP/8KucZsCfl9+t4S5hQQfRe5RJ50XJ2zn9x6UVNvYeoSpH0w/odBjYQb71pFzKWfYfoUsYD9QFj3wNebQlaNXj+HVarcua658ztFDx2gGhV35ClwKvmrTZ9h4meFetovhHub8Diw0yxeMZBoL1dhk/mv/ene8fVylYWGpEm+Xthmk+QutvY0QBRt3nZjy+KtxCIAVruwDxfAn+zwRDLRwGZadcPqR4Stjb46hQsFED9mZl99WX3ixwosiiV42ettAcDJBOA1NVMLQSflthdLQcXP2gHrjjfWxRUtLPf08ClB66uJk69LXWIxBYcjcPFal8fswxMvnpsVMMcnvfcGx+8bvZU5ga4c6GB5Ce/yZpP8O/BNUEbVVzlHQcPcJ7ihBpjAPmNEHFnmBRio9aYwk8Lffd79BseFHNrEPHzsRkHUUWpxjH9KUhQl/5wyzMvxbydpSoUUhR9YlzB+kAV1+r9rGMA8vTUtJ6K2be++k+H5kG9tn8XsxL/n48ybEkQ82vWfeM3MI+V0mTjgn5jb78TEU3Brm3EbeG1dtpq1CkBYg8zeYWEzBw+2CuOUOIcFtyOj5o1fssR/NiEVPfkiAYQJu2zNx7gsxOFgtxvUESZKKvXzAlSPwKhV7TGhO82uagp9NYKdijyKVnjxWsBHsbKScJkkznMvwLj18fL7hp4+Jk5oTWsBsZMoEbVrYrWEISi3TVjlHllEMUeyRbcyRu+WBZ8N/lQo3/s6Za0yjdcdddotZxwpYetGQdhdgRIFGLA3kKaX3+26pgZyuAX/n9JjxHpWQNxRc5cUejVCyqTnIOabYpMxkqxam3zA38MhR59bf/owMJKf53Px88ixtUg8atjeNGR2eXPBAmLLw1VaNyJhzG1nsqc20VQhSCNGDxZAmBQ5FG88pnGgN01IlKAhD9torH0LguvxwKO+Rti8ArWDmkrDSrRq+Tqsyv9KWgEqTnZsqBRvzcNKwLkHN9gUNDfPye3JNOf5t+L1TL2VLGYJSy7RVzlHUsAiNBivPaezxd81zMoJ96cgb7s7DXLZ8hqGxnANz7Si+qo1iUGxSZOV70OiFI7fyDd2mIOf6lrIxNV7JMH6HhkZQmGNIxpZnOLhVZGuHpM25jSz21GbaKgTV/gxBqWXMudphzm1U1GLPBRqSJEkFZrEnbUb26kkqOnNu87PYqyJ1PUvtxQDU5kbOmXVqT+Zcx2Cx1wBDUJIk1TqLPUmSpAKz2JMkSSowiz1JkqQCc5+9KtpzbyDnAtYe/0xUROac8sw599mT1Mb8P0NJRUfOmXWbn8XeZkDrqYjHzTffXPF8EY72YgiqqCr9HRXhMOdUiyz21CaWLl0aLrnkkrB+/frsjCQVy4YNG8Lll18eFi5cmJ2RaoPFntrElClTwrp168L06dOzM5JULA888EBYs2ZNmDRpUnZGqg0We1XYrd10tHanTp0an99zzz3xUVLHZ841D41azJw501EM1RSLPbVaau1i3rx5YdWqVfG5JBUFGUfWgVEMCj6pVljsqdVSazdh6bokFQmjF4xiJOW5J3VkFntqlXxrN3EoV1LRlBd3CxYscBRDNcNiT61S3trF8uXLw+LFi7NXklTb2G2AIy8/V1nq6Cz21CqptdulS5f42LVr1/ho756kojDnVOss9tRiqbVL8A0dOjSeGzFiRHxkCxZXqzXMlZBSx5fvwUv5Rt6Re8uWLQtLliyJ51SZOdcxWOxV4S1eGjdjxoz4OHz48PDnP/85Pt91111Dr1694lw+Nx6VOjZzrnHsMECekWuf+cxn4rnXXnst5h5+8IMfxEepI7PYU4tdddVV4bHHHgsXX3xx+OAHPxjPbbvttuGuu+4KL7zwQhg8eHA8J0m1asiQITHPyLWtt946ntt+++1j7pF/EyZMiOekjsxiT60ycODA0KdPn/q5LO9973vDoEGDQu/eveNrSap15Bm5Rr6BvCP3yD+pFljsSZIkFZjFniRJUoFZ7EmSJBWYxZ60mbgSUlLRmXMdg8WeJElSgVnsSZIkFZjFXgPc9VuSJNW6LUoFjRXNO8z5C7WnPf5M0n8H/gmqiMy52mPOhTB58uQwevTouK/i/Pnzs7O1z549SZKkArNnbzNILZ0f/tJ/+o7uqN3ar1Vqz56KzJyrHebcRvbsSZIkqeZY7EmSJBWYxZ60GTmEK0lqbxZ7VaR5BlJ7sdDT5kbOmXVqT+ScWbf5Wew1wBCUJEm1zmJPkiSpwCz2JEmSCsxiT5IkqcAs9grih1NuDM/+bEH2quVee2VlmHbz+PCXP6/NzjQf79GWHn94Znho2u3Zq0298bf18fO++psV2RlJRbV43qzwyL9Pzl61Drnx6+eXZq+a789/XBPW/8+67FXr/eaFZeG7118S3nxzQ3ZmUw9OvaVNMl6dk3fQqKI9d/1O792WO8ufesgu4fPHjAojzxmfnWkZwuTSEw8Odz76Uth+x17xXGPh+ql9BoYddu4dn1Mknvp5PstJ4dSv3xDPJYRrQ/ruNyjs8dlB2as6FHOnlX633nv0D8NOviA7W4drZ/zbFeGem8aGkWePq/+8Sf5ztVR77yzvn582p1rLuZsuGR1eW7UyXHl36+9swN/2uVfdFT7/f06Kr3/+2Jzwx9+vjs8rIUvIlIScpDAbf+dDocvWXbOzdXnZUIOXnEo/M+8bZx0Ti89zr74rO1Pn4336xUJwzD8dGA48YkTYe+Bh2VfqlH+ulvAOGhsV9Q4aFntV1FoItmexN+qAHcIf16wO3Xv0DFu+a6t4LvnD6lXh7Cvu2CS8fvHUwhhce35ucPjatXeHd2Xfw/s25JDS5y8PQYq5e2+/MnTr3iO2pPGB7brHx6u/91j458N3D9t9uEfYrvR1fu7Ou/at//qxX7k47H3gkPi8pQxBFVmt5Vx7FnvXfm1keOzB6TE/8sUb1pbyj0KL6xMy8fJTDw9vbtgQ/mXy3JhBuPMb54eXGugx3GX3fm9rCKfcJXPpLaTR/KFS3oIM5T0pIMm3F5YtCV0/0C3s8LG6huxeBx4Whn9lTHzeUubcRhZ7nUythWBDxV61IuvCG6bVB1RSqdhjiPT0L+wabpz1dPh4KagSQmlk/+1i0bVbvwHZ2Tqrfr08vs+go08Io//vtdnZ5nnmP+aF8aUwJewI2ifnPxC++X9PDDfPfjYWnt+86MTYGr/5gWfj78H/EVAQjr19dvYOrWcIqshqLecaKvaqFVnVGn3lxR5OPminOIJw1KjzsjN1aFTSGC0vqshAMooRiG/e92R2tnnWlBrMFxy7bxhw6LBw5vjb4utzjtwjfO1f7w77Hjw0/PjeO8Mtl50WJtw1Nzag6TlkNIMc3KZU9LUFc24ji71OphZCkJbo2mzY4ZfPLI69Xx/JDWWeUmo9Upzxh0ygbf/RneP51373mxgYqaAjXBgiwP+WQivfi3fUqHNj8I3cd7tw+tibw6CjTojXgULr6q8eF6Yt+WN9710eBR9hVF5QNhWfccVzT4V9Bx1RP7yy7MmfhE9+Zr/wnvd2CatffjGe333vA+LX/vL62vDKS78Mh33p9E2K0tYwBFVktZBzM799dXj6sYfjczLljTfWx6HNJI0IpEZt388eFB8x7ZbLNynoaBQn9JSlXjwaqxd9c1ockeAcoxUJecjoxrUzFr2tUQsKPj5X7779szPNQwN71neuD8eednH93GNyj7zms5Bry372k7Df54+OX8Mzix4JB/7jl2Ix2BbMuY0s9tRm2ioE7/rXi+qHNh+fMzN8dOfecYgAFEq0fpnXxh9yeo7y3jvCiu/Hr/7zibgYgt44gob3o3AaO/rQ+JjvpaMlzVwSWpwNodVLMdoU73lPl7cFKp/1v0rBnC9kq/n18qXhqC+f2+rh7MQQlFqmrXKOnq3nf/54fM5UjfV/WRf2/oe6nrqf/3ROGDLi9Pj3Tk5Q6OX/9vn7zRd75CIooMivLxx3amwsMhWE3j+mjTzxyA826aVjUch1pYb1jKdfr9iozfvl0sWxGG2K3fYcEButCXOaf/jdmzYpZKtJedgWw9kw5zay2FObaasQzCsfxs0XeI0Ve3nXfPW4uPq1vBVLYfmbXy2Lk5GT84btFVub+XBlrglBmlAwUpDmW9QN4fPwufL4rNv37BVb8I2ZVArw8s/UGoag1DLtkXPlw7j5Aq8pxV7CytaJE84JX77gqk2GZpkqcs25x4WZ//nX7MzG4eF8YUUDlnl8Cb2DZB0519TdCMrzl2KPQpMRmcY8ev+UNpu7CHNuo6IWe269UhAUVK2dv8FwBa1YMJzB0ERC63dZqUjkGtCjx8oxVtDmsQUMgZcOegwJNAI/HQwHl59LR3mhlzz7xIIw/ebLGz1o8UoqprbIOcyZPjE+3nPj2Ni4TWjgsuAi5SCrbVm0kR8aBiMV+ZyjMQzyK2UZc4nLz+WP8oY2yK9KuVZ+kIdSc1jsFQRDsdu8vy4Eq+3T1Jgff//O+jkgaQ5MKu44z+qwOdl+d7On3BSHdVNvYcL8PsKNg3l/lTx4z63hiOPPis/pCaRF29i+fnuUisoR54xr9GjKUK+k2sQQbr7Ya0nWMRSctmQ64oSzw01jRsfFYKB3jkVl5Bso9BipGDLyjPg6YQg25dwBhw3Pzm7qR6WcIzdTUcf0GIZ5G0J+Vcq18oM8lJrDYq8A0qReggpsh9JcDD08OmtKnP8CHum1IwjBXBUKNFrEXLvgh1Pftu8dCGLCjaN8mxYQqnw/c2XA9QxdMC+nIfbsSXr1tyvqcw4sLmsOisNvTzgnjDh7XHy9S5894zzkq889Lg7N4sgvnxunujBywcIJFqWVLzJjrl3KuS7bbLpNC2i8zv/B1PpGLVg89v3br8xeVWbPntqLxV4BsCgBLd1YkwBkZS/B9O7chOFzrrgjblacDBlxRpx8zBAvvXxsh9JcU28aG98n3zoneAnVFLaVDBg8LG422tjR8+N9su+QVCQM4VLcpdX3LcEdKijQ8qv1Dx95Rhhz0731iyX4Gos1vnXJ6DhPmVWyzXXfHdfEBXP5LV/YAobGbkN37SC/KuVa+UEeSs1hsVcAv3rmibjZZkvnsiwotUAJUoqwvLQlQUIYHl66hrAaNvqCRlemlWMlHPMA8wUk+OxgyKSa8rmA1Y7Ghkk6EiYsuzhDapr0t93SRi0jICzMYFFGOfavy2PLKXLuc0OGN/suPGQceVW+0IK87PvZQWH2d+uGiCvhd6yUa+UH718rzLmOwWKvClYQpVVEHR1d+p/q3/Lb5ez8yb7hi2dcusk2AJUwf++nWUE2d+akOE+wqQjOO686Pwb1Q9NvD7eNPzPOCYz7931h1/DX0nvRGq6EFWdMaE6715dPdOZAuq6tVuJKRVdLOccem/R85Ydxm4PRCOboNdb7z0gH84rBXqL5hWqNodHMZu/8LFbwspJ3whlHxs2aTxjw4bjIjUZtpeFncov8Yh4gUp7lD+ZI01jmeVutxFXnYLFX42gJrli2JBwy7O3bkryv7JY/1bAZKJOSG0JhN+H0I2NBSMgwd+bmy07LvlrZ30uhmSZQ8zmZXM2wCBuk/rX0nAKVAm7ij18IUxf/PgZlWgUHisH8kYrB8vMcYNuV/Lm0p5ak2sYUD+b1Vtp+qfzWZtWQXZV69coxp48GNNtMcUsypq00tICM1bscIMPIYzanf+xHM+L8ad6Dn8sdiNirj15Epq0kFIT53GJKDcrzjIMpO4/cP2WTc3y/1Bj32auiPfcGSu9N66y1CIbflgqotMwfhM3Xjt03thCZn8IeStyLMW24nFqc6et5lfbgI8AIPIZBKPRoGadbmX3hi6eGr4y9eZMhXYKZ2/uwiIOJzWMnzm7S7vK0omkRp2Hd8mKN7Q0YVv7cF47NzmzE/lvsl7XjLrtlZ+ruQZmfm9MS7bn/lLS51UrOsZKVIuw7P325frEEDUl6y0695Ib63QPK9+PkXKV99lC+Bx/vRy6y4wC3kmSVLaMZ5x29VxzKvfhb975toQbDqXddU7ftSvk9wqthlOP1UqamoWNyL90hCIya8HMZbSlHg5f8ZS/RJG0I3Rrm3EZuqtzJ1EIIUnBxZwvuHZtuY0YPHCtoaQHSY4b0h1yuKcUehSO3RKOlStjl58swHMGQBbvZc6uh1MJmiHbx3FmxyKMVStF2zMkXxIKPIePGUPDli0da1f9Wek82PL2p1DquNIeG35FCtHwrmNYyBFVktZBzFD7nDN0jLkpItzGjMPvh5BvjgouJc1+IWUVukV/lmlLs8TPIsmVPLIiNVxZtJOwp+v9OOjTOif76rffXDwOn/GPBBD175B45vNeBh8UGa2PTYhiOzvdK8jvdP+m6uPdfuh94OX7H8o2j24I5t5GbKqtDIRgYRqX1ScDQOmTiLi1dhkLLF0Hk5380da4HPXQM3dJyvP6+J982MZowoqBjyIOQAgUorXDCiuKODZRZ1cv2KvRClk80Lj9OP3TXWFgm7ME3auAOdUXn9x5r9mRpSbWNnjOKIkYnaPiRE+QcPf2MLOQbrGk+Wzqa6rZxZ8YRErIxX+iBwo3z5CHXgc9B/g4/fUzMXwpG8omtU7jmnw/f/W3ZVn6w319CZnP/XXruKB5bstOB1BB79qqohRYvPV0UYLQ4CR8Cg+f9Pjd4kxYjrV3ut5hW63ItPX/l92YELVzuN3nAkOHxPWjV0pJtaOUthRjXcD1FKK3j8tVtCe+fL+bKsTdffjNmfj7DyI312FEUErjlPZWtZYtXRVYLOccCCUYs0lSQtGp/91L25bOC4dGupYzLZwDZRzaVD7+CEQfykwYkuYRK1yVcw7zj1OCkYctepJWykYzN3zaykvwIBr8feb7f4GEN9gjymRmqdgSj/TiM28nUQgiq/bVnCLbnf2NSU5hzgjm3kcO4kiRJqjkWe5IkSQVmsdcAh9ckSVKts9irwkJPUtGRc2adVHwWe5IkSQVmsSdJklRgFnuSJEkF5j57m0Had0i1oz3+TNxnT0VmztUec8599iS1Aws9SVJ7s9iroj1bpfwffBGPu+++u+L5Ihztob3eV2oqcq69sq78b6gox7Rp0yqeL8LRHtrzvdV0FnsNcBii6RYuXBjGjh2bvZKk4lm/fn047bTTwqpVq7IzUm2w2FObmDFjRli5cmWYN29edkaSiuWBBx4I69atC9OnT8/OSLXBYk+tRms3hd+UKVPioyQVTco3c061xmJPrTZr1qywZs2a+HzmzJmx5StJRcLQ7Zw5c+LzZcuWhSVLlsTnUi2w2FOr5Vu59PJR8ElSkUydOjVs2LAhexXCpEmTsmdSx2exp1ahtVs+T88hjqZxAZBUO8pzjakrNG7VsPZc8a2ms9hTq7ABZb61iwULFsTFGmqcISh1fOw2sHz58uxVnbVr18YpLFItsNhTq6ShjC5dusTHrbbaKj5SBEpSEVTLOUcxVCss9tRiqQevR48eoX///vHckCFD4uM999wTHyWplrHgLM1DHjZsWHzkVloUfCzYcM891QKLvSrc9btx7K2Hk046KQ5pYPDgwaF79+5hxYoVYfHixfGcpI7JnGtc2ltvwIAB4UMf+lA8t/XWW8esAws3pI7OYk8tdvPNN4d77703nH766fXDGzvttFO44447wtNPPx3DUZJq2YgRI8KiRYvCVVddVT98S86NGzcuzJ07N1x44YXxnNSRWeypxQi+4cOHh169eoWuXbvGcx/84AfjUEe/fv3ia0mqdTRcGbrt1q1bfM3oBefo3UsFoNSRWexJkiQVmMWeJElSgVnsSZIkFZjFnrSZuBJSUtGZcx2DxV4V3uJFUtGZc1LnYLEnSZJUYBZ7kiRJBWaxJ0mSVGAWe5IkSQVmsdeINIG5oUnM+WsaOhpS6fpKRzWVrq10NKTS9ZWOhhx88MEVvyd/NKTS9ZWOaipdW+2optK1lY5qKl1b7ZA6gqb8N5m/prGjmkrXVjqqqXRttaOaStdWOhpy+eWXV/ye8qOaStdWOhpS6fpKRzWVrq10NKTS9ZUObX4We03U0NLxd2pZeUf/DO+Utvh3aM/foyN8Bqkl/NtqO23xb9laHf0z6J1jsVcF/4Hmj8aUX1/paEil68uPxlT6nvKjIZWuLz8aM3/+/Irflz8aUun68qMxlb6n/GhIpesrHQ2pdH35IW1uzf1vsvz6SkdDKl1f6WhIpevLj4ZUur7S0ZBx48ZV/J780ZhK31N+NKTS9eVHYyp9T/nRkErXlx/qGCz2JEmSCsxiT5IkqcAs9iRJkgrMYk+SJKnALPYkSZIKzGJPkiSpwCz2JEmSCsxiT5IkqWTDhg3xsVu3bvGxKCz21CZWrVqVPZOkYnrllVeyZyqq5557Lj7us88+8bEoLPbUJlJrSJKKypwrtnXr1oVZs2bF5/369YuPRWGxJ0mSOr3zzz8/rFy5MvTp0ycMHjw4O1sMFnuSJKnTYhrSkUceGe68886w1VZbhbvuuit06dIl+2oxbPGWdypWG9hll11ii2j+/Plh0KBB2VlJKo5jjz02/Pu//3vo1atXPFT7KPRWrFgRn3ft2jXccMMN4dRTT42vi8RiT23CYk9S0Q0YMCA88cQT2SsVBb14/P/WbbfdVtgi3mJPbcJiT1LRffnLXw533313OProo8OwYcOys6pVDNn27ds3HjwvMos9tQmLPUlFN3r06DB58uQwbty4MH78+Oys1PG5QEOSJKnALPYkSZIKzGJPkiSpwCz2JEmSCsxiT5IkqcAs9iRJkgrMYk+SJKnALPYkSZIKzGJPkiSpwCz2JEmSCsxiT5IkqcAs9iRJkgrMYk+SJKnALPYkSZIKzGJPbWLw4MGhZ8+eYcCAAdkZSSqWgw46KHTr1i0MHTo0OyPVhi3eKsmeS5IkqWDs2ZMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQC2+Ktkuy51CzLly8Py5Yti8egQYPisW7dunDLLbeEnj17hhNOOCG7UpJq0+rVq8OSJUti3nXr1i2ceuqp8fzUqVPDypUrw4UXXhi6dOkSz0kdlcWeWmzhwoXhmmuuCQ888EAYMGBAeOyxx8IxxxwTX2+11Vbh9ddfNwQl1TSKvIkTJ4Ybb7wx5tmrr74abr/99nDJJZfEr8+dOzcMHjw4Ppc6Kos9tcqqVavCTjvtFJ+fd955Ydtttw0DBw4M69evD0OHDo3nJanW7brrrmHFihXh7LPPjr19Z511Vnx90kknxcat1JFZ7KnVdthhhxh+DG/ccccd2VlJKo6RI0eG6dOnh759+4Ynn3zSUQvVFBdoqNX69+8fHz/96U/HR0kqmn322Sc+9u7d20JPNcdiT23m8ccfz55JUjExV1mqNRZ7apV58+aFNWvWxOdLly6Nj7xmzp4kFQGZ9uCDD9Y/Z67yhg0b4vQVqRY4Z0/Nxuo0Vt0yd4XQmz17dpy3R/jdfffd4bnnngsTJkxw0rKkmkWeHXzwwXHIdu3ateGuu+6KuceijDFjxsR8O/300+M2U1JHZ8+emo29pSj4FixYEK699trQvXv3+r2n2KLg4osvttCTVNPYM5T99Ri2Pf7442Pj9tJLL41fY6HG0UcfbaGnmmHPnlqE4QuKvFTUMWxL67dHjx7xtSTVOjKNjOvatWt2pq6x26tXr+yVVBss9iRJkgrMYVxJkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKjCLPUmSpAKz2JMkSSowiz1JkqQCs9iTJEkqMIs9SZKkArPYkyRJKqwQ/j9GoR3k1ZMM5wAAAABJRU5ErkJggg=="}},"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"50F442253C51495181611BB088576C75","mdEditEnable":false},"source":"# 残差网络（ResNet）\n深度学习的问题：深度CNN网络达到一定深度后再一味地增加层数并不能带来进一步地分类性能提高，反而会招致网络收敛变得更慢，准确率也变得更差。\n### 残差块（Residual Block）\n恒等映射：  \n左边：f(x)=x                                                  \n右边：f(x)-x=0 （易于捕捉恒等映射的细微波动）\n\n![Image Name](https://cdn.kesci.com/upload/image/q5l8lhnot4.png?imageView2/0/w/600/h/600)\n\n在残差块中，输⼊可通过跨层的数据线路更快 地向前传播。"},{"cell_type":"code","execution_count":6,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"3F859601FD264D1A8D5E5E51B9D83D0D","scrolled":false},"outputs":[],"source":"class Residual(nn.Module):  # 本类已保存在d2lzh_pytorch包中方便以后使用\n    #可以设定输出通道数、是否使用额外的1x1卷积层来修改通道数以及卷积层的步幅。\n    def __init__(self, in_channels, out_channels, use_1x1conv=False, stride=1):\n        super(Residual, self).__init__()\n        self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1, stride=stride)\n        self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1)\n        if use_1x1conv:\n            self.conv3 = nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=stride)\n        else:\n            self.conv3 = None\n        self.bn1 = nn.BatchNorm2d(out_channels)\n        self.bn2 = nn.BatchNorm2d(out_channels)\n\n    def forward(self, X):\n        Y = F.relu(self.bn1(self.conv1(X)))\n        Y = self.bn2(self.conv2(Y))\n        if self.conv3:\n            X = self.conv3(X)\n        return F.relu(Y + X)"},{"cell_type":"code","execution_count":7,"metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"2E2CF8846F5C43618B6DB004FA95BC0E","collapsed":false,"scrolled":false},"outputs":[{"output_type":"execute_result","metadata":{},"data":{"text/plain":"torch.Size([4, 3, 6, 6])"},"transient":{},"execution_count":7}],"source":"blk = Residual(3, 3)\nX = torch.rand((4, 3, 6, 6))\nblk(X).shape # torch.Size([4, 3, 6, 6])"},{"cell_type":"code","execution_count":8,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"C8F6DFFB58344B669C24A18721B8A66F","scrolled":false},"outputs":[{"output_type":"execute_result","metadata":{},"data":{"text/plain":"torch.Size([4, 6, 3, 3])"},"transient":{},"execution_count":8}],"source":"blk = Residual(3, 6, use_1x1conv=True, stride=2)\nblk(X).shape # torch.Size([4, 6, 3, 3])"},{"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"CEFB948C07AE428D8F5F756CDE556CF9","mdEditEnable":false},"source":"### ResNet模型\n卷积(64,7x7,3)  \n批量一体化  \n最大池化(3x3,2)  \n\n残差块x4 (通过步幅为2的残差块在每个模块之间减小高和宽)\n\n全局平均池化\n\n全连接"},{"cell_type":"code","execution_count":9,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"6117AE144A6B4EF58B382D31F1C2F760","scrolled":false},"outputs":[],"source":"net = nn.Sequential(\n        nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3),\n        nn.BatchNorm2d(64), \n        nn.ReLU(),\n        nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"},{"cell_type":"code","execution_count":9,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"D1E8052B1A8242F285BDD3E54418419B","scrolled":false},"outputs":[],"source":"def resnet_block(in_channels, out_channels, num_residuals, first_block=False):\n    if first_block:\n        assert in_channels == out_channels # 第一个模块的通道数同输入通道数一致\n    blk = []\n    for i in range(num_residuals):\n        if i == 0 and not first_block:\n            blk.append(Residual(in_channels, out_channels, use_1x1conv=True, stride=2))\n        else:\n            blk.append(Residual(out_channels, out_channels))\n    return nn.Sequential(*blk)\n\nnet.add_module(\"resnet_block1\", resnet_block(64, 64, 2, first_block=True))\nnet.add_module(\"resnet_block2\", resnet_block(64, 128, 2))\nnet.add_module(\"resnet_block3\", resnet_block(128, 256, 2))\nnet.add_module(\"resnet_block4\", resnet_block(256, 512, 2))"},{"cell_type":"code","execution_count":10,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"93D7FFC7581F438A82E92D85E3F6D24E","scrolled":false},"outputs":[],"source":"net.add_module(\"global_avg_pool\", d2l.GlobalAvgPool2d()) # GlobalAvgPool2d的输出: (Batch, 512, 1, 1)\nnet.add_module(\"fc\", nn.Sequential(d2l.FlattenLayer(), nn.Linear(44944, 10))) "},{"cell_type":"code","execution_count":11,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"EE1FD4E1510941AB98B3C8840DA47E44","scrolled":false},"outputs":[],"source":"X = torch.rand((1, 1, 224, 224))\nfor name, layer in net.named_children():\n    X = layer(X)\n    print(name, ' output shape:\\t', X.shape)"},{"cell_type":"code","execution_count":13,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"7A68C8B9480C4DB8935436EC136068F1","scrolled":false},"outputs":[],"source":"lr, num_epochs = 0.001, 5\noptimizer = torch.optim.Adam(net.parameters(), lr=lr)\nd2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)"},{"attachments":{"1576638871%281%29.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAA5AAAAGBCAYAAAAQWV4zAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAHYcAAB2HAY/l8WUAAFFRSURBVHhe7d37111Fnedx+k/oH+e3Zi37B9MzSXqMM0ZsAoYWh/QEoQcvYYLNrWEQBUSwyRiFGJUlSLiMRAd9smA0XAZID9dkCE+aSwCbxCEIRDq0SLCDPcGB9EyH5fnBml1779qnqnbtS+2zT50653m/1vou5XlOzjnPvlTVp/btGAEAAAAAQAsESAAAAABAKwRIAAAAAEArBEgAAAAAQCsESAAAAABAKwRIAAAAAEArBEgAAAAAQCsESAAAAABAKwRIAAAAAEArBEgAAAAAQCsESAAAAABAKwRIAAAAAEArBEgAAAAAQCsESAAAAABAKwRIAAAAAEArBEigo127dokTTjihqGOOOaZU6ndXX311WgCAjN6GutrPhVL0EQCmDQES8KAGPK5BQNtioABgIZPtn6ttpLKifwAQOwIk0MK4BjwMFAAsJCeeeKKzLaTMom8AEDMCJFCjTXD8F0tPSOsDn726VOp3rn+nF4MFALPOFR7r2s+FVPZykUW/ACBWBEjAoe5UVTXg+bNNu8Rf7hKtq26gIEt+3u9+97v8GwDA7LDDY5c2dNbL1T8QIgHEiAAJWGR4/L3f+71SR97ngKcuSM7Pz+ffBACmn30mh2xL/3L+d862caEXIRLANCBAAhp7oKPqz26Yd3b2o1ZVkGTAAGAWEB79S05UHmNNYjKxCCAmBEgg5wqPoQY7/+KPy9cGESIBTDv7bI5xTcbNWtmTi1ziACAmBEgg4QqPsgN3dezjKk5dAjBL7HY1dJs67WVPLHIUEkAsCJBY8GIIj6pcIZJBA4BpZB995NRVv7JPZeUoJIBYECCxoLlumDPpWXJXiGTQAGCacPSxn+IoJIAYESCxoNm3lo9lkGMPGph5BjBNOPrYT3EUEkCMCJBYsGKfIWfmGcA0ss/s6LttlaGqqlyvH1e5Pl+W67WjlN0XECABTBoBEgtW7DPkciDCzDOAaWNPzvXdtrpO85cV+hEh9qM2VPV9p1m7L2AyEcCkESCxIE3L9TkchQQwbfTJuXGEuoUeIJlMBDBpBEgsSNNyfQ4DBwDTZpynr8rSA2Qsz5Uc93fSJxPpBwBMGgESC8603R3Qnm3nKCSAWNnt6zjC1EIPkLIIkAAmiQCJBWec1+fIQYQq1++7lH0UUn5/AIjRONtXVQsxQHIdJICYECCx4Izz9Cq9g+9z4DSu05fk3RJlAUAfCJDTFSDpAwB0QYDEgjLu28uHCJCyRhk8yGUgQ6h6L66nAdCXcV//KGshBkhZfZ2JQh8AYFQESCwo454dH1eAHHX2WQ4Y5N+uDxr0YvAAoA8EyPF9p1HORJF9gB0c9aIPAOCDAIkFZVoDpKwus891Awa9uJ4GQB8IkHEFSPoAAONAgMSCcuKJww54HM8MiyFAth0wyNfI1wJAXwiQkw+Q9AEAxo0AiQVlmgNk3eBBDgLqTlHVS75Glny9rM9+9rPF/6f6LwZoWCjktq4HyHEFKQKkuw+Q1aYPkEUfUC7aaqA9AiQWlHHPjocOkD4DBmqypQZswKwiQIb7u2XRB4ynVKAEUI0AiQVllgKk7ODUf1PTU3LdMdONWUSADPd3y6IPGG8RIoFqBEgsKKOcwirvhNpUeoCUgwjXa/RyfU5VVZ2+pGZL1e+o6SgGJ5g1BMhwf7cs+oDxFxN+gBsBEgtK1wApw54eDvsqnwCrf35V+JA/b3M6k3wdnWIYcjlXDe70iQBg2hEgx/ud6q6BVGRbU9Xe6KVeh4xqp6uWHXepBUwESCwosx4gdVUdoV5yEMIgIhzXOmFgglky7ssEZBEg200+tekDZNEHmFzLjck+wESAxIJidww+AVIOEJpKf2/X7+1yfVZV+QZIRb7W1SHqRccYjmtdsPwxKwiQ8QRIpU0fIH8Pk73MWEbAEAESC8q4T7HSQ57P0cWm0gcnsrp2ZPLfyYGH/l7MrIZnD0xYB5gVevs6jkclyVqoAbLrJKJO/ju7/ZFF++Omn7UkizNGgAwBEgvOOAc4oQJkH529GkR0HYhgNAxMMItGuVFZ2yJA9tNu0wc0syedmewDMgRILDjjHOCMK0D2PXDA5NkDE9YrZoE9MRJzgLSvbR/lu447QNrflQmncOxtmgAJECCxAI3zNNZxBEj76CNBY3boAxNmtjELxn2ZgKyFGCDtfoC2Ihx7mya8AwRILEB2Z9DnUcgQAZKBw+wgQGLWECAJkLNI36aZxAUIkFig7FNS+urw+w6Q9qCBjmu2yPWpr18GhZgF45qgU9VnWJMhUn5H+V4xB0i9b6EfCI8ACZgIkFiQxnUUsu8Aqb+fLALGbCFAYhaN+zrIvsOabGfle7p+17bGGSDtI6UEmPAIkICJAIkFaxxHIfsMkPqARBad1uwhQGIWjfs01j7DmnqvmAOk3RfQToRHgARMBEgsWPYgR9aoHb980LM8mjnqEU17wCCLQcPsIUBiFo3zOnNZfYU1dWSvj+83zgCpT0xyrfRkECABEwESC5o9gO97oNO19AGDLDqs2USAxKwa13XmsvSwpibsZMmfu15fVzJEun7epvQJQ1nj/ltl0RdMBgESMBEgseDZAx05EJhkiJSDEv370FnNLgIkZtU4j0LaoWocn9Gm7Ik+VX0GSPszaCMmgwAJmAiQQCKGEKnfDVAVpyvNNgIkZtm4jkLKtlKGSLtGOZrYpVzfQZbrtV1Kvpe+/Aguk0OABEwESCDhuh5SVt/XslSVuhbH/nwCxWwjQGKWjftayFkvjj7GgwAJmAiQQK4qRPY5o+wqe5ZZljzyKL8PZhsBErPOPgo57vZ0Voqjj3EhQAImAiRgsQc8qvoe+LhOWZXFaasLBwESs841MUeIrC/CY3wIkICJAAk42AN7vUYZ/MjQWBUcZdExLSwESCwErvaUEOkuOzzKol2YPAIkYCJAAhXqQqQsGQJlZ1934wYVGOXrqkKjqvn5+fyTsVAQILFQVLWnBMnhTYFcy4d+IQ4ESMBEgAQaNAVJu5qCol10RgsXARILSVNbKtvOhVau5aCK8BgPAiRgIkACLfkGyaaiEwIBEgtN3+3oLBY3UYsPARIwESABT7Lz6DIIkoMCBgbQESCxUBEk3cVRxzgRIAETARIYkQyEKlTaJX+nCrDJbUQfPBIgsdDo7aeaZAtZS5cudf48ZNFHxI8ACZgIkEDP5GNACAJogwAJTBbtNdogQAImAiTQIxUIGJCgDQIkMDnyqJ8MBpw2iiYESMBEgAR6pDoZBiRogwAJTI7a/+RppOx7qEOABEwESKAnehhgQII2CJDA5OihgH0PdQiQgIkACfSEMABfbDPAZNj7HmeNoA4BEjARIIGe6B0MAxK0QYAEJkPePEff9zhrBHUIkICJAAn0wA4CshiQoAkBEgjP1V6z/6EOARIwESCBHthHH1VxFBJ1CJBAeFUBkvYaVQiQgIkACYyoajAiiwEJ6hAggfCqJvw4awRVCJCAiQAJjKguQDIgQR0CJBBWXXsti30QLgRIwESABEZUNZutiqOQqEKABMKivUYXBEjARIAERtA0my2Lo5CoQoAEwqG9RlcESMBEgARGYN8KvqoYkMCFAAmE0yZAymI/hI0ACZgIkMAImk6HUsVpUXAhQALh0F6jKwIkYCJAAh3VzWbL06Ds/yYcwEaABMKoa6/tor2GjQAJmAiQQEd6hyIHHHpotP9bFgMS2AiQQBj20Ud736O9Rh0CJGAiQAId2IMP+d92gNy1a5fR6XBaFGwESGD8XO21/TPZPtNeowoBEjARIIEOXAP/Y489tvjvpUuXpj/TOx0ZKgkI0Lm2IwD9or3GqAiQgIkACXQkOxE5yFCdiX5HVjUgkT+Tr1FHJAEdARIIQ+5rqqSq9lr+N+01bARIwESABHqiD0iYvUYbBEhgMmiv4YMACZgIkEBPGJDAFwESmAzaa/ggQAImAiTQEwYk8EWABCaD9ho+CJCAiQAJ9IQBCXwRIIHJoL2GDwIkYCJAAj1hQAJfBEhgMmiv4YMACZgIkEBPGJDAFwESmAzaa/ggQAImAiTQEwYk8EWABCaDQAAfbC+AiQAJ9IQACV8ESGAyCATwwfYCmAiQQE8IkPBFgAQmg0AAH2wvgIkACfSEAAlfBEhgMggE8MH2ApgIkEBPCJDwRYAEJoNAAB9sL4CJAAn0hAAJXwRIYDIIBPDB9gKYCJBATwiQ8EWABCaDQAAfbC+AiQAJ9IQACV8ESGAyCATwwfYCmAiQQE8IkPBFgAQmg0AAH2wvgIkACfSEAAlfBEggvF27dhmBYH5+Pv8N4EaABEwESKAnBEj4IkAC4REg4YsACZgIkEBPCJDwRYAEwiNAwhcBEjARIIGeECDhiwAJhEeAhC8CJGAiQAI9oYOBLwIkEB4BEr7o3wETARLoCR0MfBEggfAIkPBF/w6YCJBAT+hg4IsACYRHgIQv+nfARIAEekIHA18ESCA8AiR80b8DJgIk0BM6GPgiQALhsd/BF/07YCJAAj2hg4EvBrJAeOx38EX/DpgIkEBP6GDgi4EsEB77HXzRvwMmAiTQEzoY+GIgC4THfgdf9O+AiQAJ9IQOBr4YyALhsd/BF/07YCJAAj2hg4EvBrJAeOx38EX/DpgIkEBP6GDgi4EsEB77HXzRvwMmAiTQEzoY+GIgC4THfgdf9O+AiQAJ9IQOBr4YyALhsd/BF/07YCJAAj2hg4EvBrJAeCeeeGKxz51wwgnsd2hE/w6YCJBAT+hg4IsACYRHgIQv+nfARIAEekIHA18ESCA8AiR80b8DJgIk0BM6GPgiQALhESDhi/4dMBEggZ7QwcAXARIIjwAJX/TvgIkACfSEDga+CJBAeARI+KJ/B0wESKAndDDwRYAEwiNAwhf9O2AiQAI9oYOBLwIkEB4BEr7o3wETATJCsnOTnRo1XaU6F1l/8Ad/4HwNFW9NYlBAgJxeu3btcm5HVPyl73O01dNVss2U+15oBEjARICMjD2gpCgqXM3Pz+d7YhgEyOmlDygpigpXMkiGbisJkICJABkZ/dQaiqLCVuiBAQFyOskjIARIippcESCBySJARsYOkPbpG1TcJU+HWrp0qfN3VJyl728ESLRhB0jXdkXFXbTV01V6OymLAAlMFgEyMlzcD4Q1yYEBAXI62QEy9KnPwEIz6baSAAmYCJCRIUACYREg4YsACYRFgATiQoCMDAESCIsACV8ESCAsAiQQFwJkZAiQQFgESPgiQAJhESCBuBAgI0OABMIiQMIXARIIiwAJxIUAGRkCJBAWARK+CJBAWARIIC4EyMgQIIGwCJDwRYAEwiJAAnEhQEaGAAmERYCELwIkEBYBEogLATIyBEggLAIkfBEggbAIkEBcCJCRIUACYREg4YsACYRFgATiQoCMDAESCIsACV8ESCAsAiQQFwJkZAiQQFgESPgiQAJhESCBuBAgI0OABMIiQMIXARIIiwAJxIUAGRkCJBAWARK+CJBAWARIIC4EyMgQIIGwCJDwRYAEwiJAAnEhQEaGAAmERYCELwIkEBYBEogLATIyBMjpIgeSdYX4ESDhS+7bBMjpYbfLrkLcCJBAXAiQkSFATg856NA7lbqS65JOJ04ESPiy930CZNzs/ayu5Gtpq+Mz6baSAAmYCJCRIUBOD59BiSrWaXwIkPBFgJwuer/atggJcZl0W0mABEwEyMgQIKeH3qGpI4xVpV7Heo0PARK+CJDTxd7Hq0q2zep1rNe4yPWjrxsCJDBZBMjIECCnh76umgYaDDjjRYCEL/bn6aGvqzZ9qr5P0gfHgwAJxIUAGRkC5PTQO5Q268kncCIcAiR8ESCnh76Ptd2/fQInwph0W0mABEwEyMgQIKeD76y21GUgg/EjQMIXAXJ6dGl39X6YfTIOk24rCZCAiQAZGQLkdOgyKOnybzB+BEj4IkBOjy7rSf837JNxmHRbSYAETATIyBAgp4PembXtTBiUxGmSA4NJD4rQDQFyevi2u13adozfpNtKAiRgIkBGhgA5HXwGj3KwKdelej2dT1wIkPBFgJwO+v7Vpj/1fT3CmXRbSYAETATIyBAgp4Pdmdgl150q9Tr1WsSFAAlfBMjpoO9fsi2W/22Xq52W/82+GBe5rvR1RIAEJosAGRkCZPzswWObYl3GiwAJXwTI6aD3p22LdRknAiQQFwJkZAiQ8bM7Mp9icBIfAiR8ESCng76OfIqAEJ9Jt5UESMBEgIwMATJ+ekdW15HIQab8vd3xMdiMCwESvgiQ00FfR3X7lmqr1WtlhW4LUG/SbSUBEjARICNDgIxf20GJTu/8WK9xIUDCFwEyfvq+1Xa/Zr3Ga9JtJQESMBEgI0OAjF+XAGkPTFiv8ZjkwGDSgyJ0Q9CIn75v+ezXeh9MUIjHpNtKAiRgIkBGhgAZN70T810/BMg4ESDhiwAZP70v9Vk/+j5JUIjHpNtKAiRgIkBGhgAZt66DC/3fsV7jQoCELwJk/PT147NfcQQyTpNuKwmQgIkAGRkCZNy6zmozKIkXARK+CJBx09ePTz9qr1f2x3gQIIG4ECAjQ4CMW5fBBSEhbpMcGLBtTCcCZNz0/artPm2v09BtAepNuq1k2wBMBMjIECDjZg8a5aCjquT6U69VRccTHwIkfMn9mwAZL32/ku2w3TbbZbfV9L3xmXRbSYAETATIyBAg42V3YL5FpxMnAiR8ydBBgIyX3o/6lgqciMuk20oCJGAiQEaGABkvuwNrWwxI4kaAhC8CZNz0ddO2ZDtNMIjXpNtKAiRgIkBGhgAZLzlo9C3EjwAJX3LfJkDGy26H2xTiRoAE4kKAjAwBEgiLAAlfMnAQIIFwCJBAXAiQkSFAAmERIOGLAAmERYAE4kKAjAwBEgiLAAlfBEggLAIkEBcCZGQIkEBYBEj4IkACYREggbgQICNDgATCIkDCFwESCIsACcSFABkZAiQQFgESvgiQQFgESCAuBMjIECCBsAiQ8EWABMIiQAJxIUBGhgAJhEWAhC8CJBAWARKICwEyMgRIICwCJHwRIIGwCJBAXAiQkSFAAmERIOGLAAmERYAE4kKAjAwBEgiLAAlfBEggLAIkEBcCZGQIkEBYBEj4IkACYREggbgQICNDgATCIkDCFwESCIsACcSFABkZAiQQFgESvgiQQFgESCAuBMjIECCBsAiQ8EWABMIiQAJxIUBGhgAJhEWAhC8CJBAWARKICwEyMgRIICwCJHwRIIGwCJBAXAiQkSFAAmERIOGLAAmERYAE4kKAjAwBEgiLAAlfBEggLAIkEBcCZGT0AClLhkiKosZX+v5GgEQbdoB0bVcURfVbk2wrCZCAiQAZGTtAUhQVrgiQaMMOkBRFhS0CJDBZBMjIMDChqMkVARJtMdlHUZOpSQQ4AiRgIkBGSDZOrtM3KIoaX8nJm9AIkNNLbi+01RQVvibRVhMgARMBEuiRPCpBCEBbBEggPLnfTSKEYHoRIAETARLoiQoD3JERbREggfDkRJ88ksX+hrYIkICJAAn0RIUBOhe0RYAEwtLvM8D+hrYIkICJAAn0RHUwzGyjLQIkEJa+z3G2CNoiQAImAiTQAzsIMDBBGwRIICw9CDDZh7YIkICJAAn0QO9cGJigLQIkEI69v8lisg9tECABEwESGJF+TY1ehAE0IUAC4bgCJJN9aIMACZgIkMCIXIMSWcxsowkBEgjHNdFHgEQbBEjARIAERuQalMhiYIImBEggjKqJPllM9qEJARIwESCBEdQNSmQxMEEdAiQQhnz2o76v6cVkH5oQIAETARIYQVOApKNBHQIkMH5V16nrxb6HOgRIwESABEbQNChhZht1CJDA+DVN9MnibBHUIUACJgIk0FGbQYksBiaoQoAExq9pok8Wk32oQ4AETARIoKM2gxJZDExQhQAJjFfbiT5ZTPahCgESMBEggQ7sa2pkSFT/31UEA7gQIIHxsvexuraayT5UIUACJgIk0IE9KJEz13YHY/8esBEggfGy22U9QMr/b/83+yBcCJCAiQAJdGAffbSPSNqBkoEJXAiQwPjY+5crQLrabsBGgARMBEjAk2tQ4hqE6AMV9TNAZ29LBEigP/azH+X+deyxxxb/vXTp0vRnTPahCQESMBEggQ5kB6IG/3KwsWnTplKAVKFSzXIDNgIkMF5qH1ODfleAlEGTdhp1CJCAiQAJ9IDToNAFARIISz8qydFGtEWABEwESKAHBEh0QYAEwiJAogsCJGAiQAI9IECiCwIkEBYBEl0QIAETARLoAQESXRAggbAIkOiCAAmYCJBADwiQ6IIACYRFgEQXBEjARIAEekCARBcESCAsAiS6IEACJgIk0AMCJLogQAJhESDRBQESMBEggR4QINEFARIIiwCJLgiQgIkACfSAAIkuCJBAWARIdEGABEwESKAHBEh0QYAEwiJAogsCJGAiQAI9IECiCwIkEBYBEl0QIAETARLoAQESXRAggbAIkOiCAAmYCJBADwiQ6IIACYRFgEQXBEjARIAEekCARBcESCAsAiS6IEACJgIk0AMCJLogQAJhESDRBQESMBEggR4QINEFARIIiwCJLgiQgIkACfSAAIkuCJBAWARIdEGABEwESKAHBEh0QYAEwiJAogsCJGAiQAI9IECiCwIkEBYBEl0QIAETARLoAQESXRAggbAIkOiCAAmYCJBADwiQ6IIACYRFgIQvu52mfwcIkEAvCJDogsEsEBb7HHwx0QeUESCBHhAg0QWnRQFhESDhww6PtNNAhgAJ9IAACV+cFgWER4CED71fl8X2AmQIkEAPCJDwYYdHBrJAGARItCH7dLl96O00Rx+BIQIk0AMCJNqytxW2FyAcAiSa2BN8sgiPgIkACfSAAIk2GJgAk6UHyKVLl6ZttWy/KUpOKNhHHWUx0QCUESCBHsjOhwCJKmpwog9KZDEwAcI69thjS/shRVUVE3yAGwES6IEdIOXMtprNpBZu6QMRu+TvCY9AWL//+7/v3B8pyi4mgoFqBEigB5s2bSpd10ZRVcWsNjAZf/zHf+zcJylKlpzYkxPCAOoRIIEeECCpNsXgBJgsroGkXAXADwES6IHsgDiFlXKVPNrIAAWIA3dhBYDRESCBHtgBkmsnACA+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESCBHhAgASB+BEgAGB0BEugBARIA4keABIDRESAjc/XVV6edmurgKIoKUwwm4UO21a7tiKKo8Zbc9wBMFgEyMgxKKGpyxZFjtKWfcUBRVNhisg+YLAJkZPTTayiKClvMbKMN+5R1iqLCFgESmCwCZGTsAPmZz19NTVGtPP0vnD+n4i19fyNAog07QP7Lf3OCc9ui4izZTtNWT1fJfUxvqwmQwGQRICOjB0jZYP6Pn/9OPPCqoChqTKUHAQIk2rAD5MY75p3bFkVR/ZQMkWp/k0WABCaLABkZAiRFhS0CJHwRICkqbBEggbgQICNDgKSosEWAhC8CJEWFLQIkEBcCZGQIkBQVtgiQ8EWApKiwRYAE4kKAjAwBkqLCFgESvgiQFBW2CJBAXAiQkSFAUlTYIkDCFwGSosIWARKICwEyMgRIigpbBEj4IkBSVNgiQAJxIUBGhgBJUWGLAAlfBEiKClsESCAuBMjIECApKmwRIOGLAElRYYsACcSFABkZAiRFhS0CJHwRICkqbBEggbgQICNDgKSosEWAhC8CJEWFLQIkEBcCZGQIkBQVtgiQ8EWApKiwRYAE4kKAjAwBkqLCFgESvgiQFBW2CJBAXAiQkSFAUlTYIkDCFwGSosIWARKICwEyMgRIigpbBEj4IkBSVNgiQAJxIUBGhgBJUWGLAAlfBEiKClsESCAuBMjIECBno2Rnp+ob/22X8zVUHEWAhC8C5GyUbJtpq6ej5PpR+5ssAiQwWQTIyBAgp7/sjo71GHcRIOGLADkbpa9D1mPcRYAE4kKAjAwBcvrLHpSwHuMuAiR8ESCnv+xAIkv+zPVaavJFgATiQoCMDAFyuss1KJHFeoy3CJDwRYCc/lLrT2+z6XPjLQIkEBcCZGQIkNNb8voZfVCid3gMMOMtAiR8ESCnu/S2Wf5/2dfK/0+fG28RIIG4ECAjQ4Cc3rIHJXqgZIAZbxEg4YsAOb1lt9PyZ//q3w77XfrcOIsACcSFABkZAuR0lmtQYh+RtP8NFUcRIOGLADm95WqrCZDxl77eZBEggckiQEaGADmdpQ8m9bCofs66jLcIkPBFgJzOcoVH++esyziLAAnEhQAZGQLk9FXVoEQWATL+IkDCFwFyOktfZ3pbzeUG8RcBEogLATIyBMjpK31QYq8vTo2KvwiQ8EWAnL6qm+jTAyT9bpxFgATiQoCMDAFyuqpuUCJLD5AMMuMsAiR8ESCnr/T1ZferBMj4iwAJxIUAGRkC5PSUPuiQ5VpXeqfHIDPOIkDCFwFyuqppok9WU1tOTbYIkEBcCJCRIUBOT7UZlHAn1viLAAlfBMjpKTt4VPWpXG4QdxEggbgQICNDgJyOajso4dSo+IsACV8EyOkpva2um8TjcoO4iwAJxIUAGRkC5HSUPnhU66qqWJ9xFwESvgiQ01F26LDbZrtYn/EWARKICwEyMgTI+MvuyHyK9RlfESDhiwA5HWVP9LUt+t74igAJxIUAGRkCZPylD0rkOpIdW10xsx13ESDhiwAZf8m2V62fNu20LP319L1xlb5+ZBEggckiQEaGABl36Z2Y/P+u19il/xsGmvEVARK+CJDxl75+2rbV+r+h742rCJBAXAiQkSFAxl1dBhjciTXu0tcpARJtECDjLj1s+LS53Ik13iJAAnEhQEaGABlvdR2UcCfWuIsACV8EyHirzfN5q4o7scZbBEggLgTIyBAg4yy78/JdLwTIeIsACV8EyHir60SfLP3fsk7jKgIkEBcCZGQIkHGWPlj0HZTI4tSoeIsACV8EyDjLDhm+bS1ni8RbBEggLgTIyBAg46tRByWyODUq3iJAwhcBMs7S10mXiT4CZLxFgATiQoCMDAGSosIWARK+CJAUFbYIkEBcCJCRIUBSVNgiQMIXAZKiwhYBEogLATIyBEiKClsESPgiQFJU2CJAAnEhQEaGAElRYYsACV8ESIoKWwRIIC4EyMgQICkqbBEg4YsASVFhiwAJxIUAGRkCJEWFLQIkfBEgKSpsESCBuBAgI0OApKiwRYCELwIkRYUtAiQQFwJkZAiQFBW2CJDwRYCkqLBFgATiQoCMDAGSosIWARK+CJAUFbYIkEBcCJCRIUBSVNgiQMIXAZKiwhYBEogLATIyBEiKClsESPgiQFJU2CJAAnEhQEaGAElRYYsACV8ESIoKWwRIIC4EyMgQICkqbBEg4YsASVFhiwAJxIUAGRkCJEWFLQIkfBEgKSpsESCBuBAgI0OApKiwRYCELwIkRYUtAiQQFwJkZAiQFBW2CJDwRYCkqLBFgATiQoCMDAGSosIWARK+CJAUFbYIkEBcCJCRIUBSVNgiQMIXAZKiwhYBEogLATIyBEiKClsESPgiQFJU2CJAAnEhQEaGAElRYYsACV8ESIoKWwRIIC4EyMjoAZKiqLBFgEQbdoCkKCpsESCBySJARoYASVGTKwIk2iBAUtRkiwAJTBYBMjIMTChqcsWgBG0x2UdRkykm+oDJI0BGSjaQFEWFKTlxA/hybUsURY2vaKuBOBAgAQAAAACtECABAAAAAK0QIAEAAAAArRAgAQAAAACtECABAAAAAK0QIAEAAAAArRAgAQAAAACtECABAAAAAK0QIAEAAAAArRAgAQAAAACtECABAAAAAK0QIAEAAAAArRAgAQAAAACtECABAAAAAK0QIAEAAAAArRAgAQAAAACtECCnzXv7xNz5fyX++te/y38AoBX2HQAAgJERIKfJ4efFTWd+XFz24C/Eb/Mf1XtN3PX5T4pzr7pWbPtZ/qMp8eR3VomVHztFXPcUg330wHvfGTp431fFuTfcKXYfOJr/ZJrsEz/84uXiurkdYv+7+Y+CeUbceOoZ4tyvbhMvRbQbHz1yRAzy/19psFNsOOkUsebSH4on/jH/mZcXxL1f3yiuufeF/L9rvHdAPD63XlzyX3/qvW26vSP2P7ldbN++XTyx/538ZyN4d7u48dLN4s7dB0TdHvDu9pvFJd/ZIrbteSP/yeiOHnhEXH/+enHfQfqBfhwSu76fbJdf/77YdSj/0SQc2i42bdwiHv3Z4fwHLpF81yr77k2+20Zx7778v8dg370R//0+pnp9D8SRIy36/lfvEOecNGJ/d/Ae8bnPrBdzf/N6cx/11A3ipGSMvPI7T4pJtY4EyGkxeFnMnblcfGruJY9Bxry4eukS8b73rxW3H5imDvg1cfuZ8nsfL67dzcABI+q07yj7xK1/lm2LX3ns/0ysoe7sxe+Jjy9aLN63/FviqRHSycE9WSBxVVVIGTz1LfHB5LMXXf6QeKftgssHZb3V9+eTYYnyjvjpfz1HLHn/ymRd/u/adTnYebX4I7nc/mKreKPTSt8p1i9Ktpv1j4nfNf37wTPi2hOS1y65QNzV7cMsPxdzn5Tb7GJx+tz+kbfZQ3dfKP4wea/6ZTEQO7/6wfR1fXxm5rDYsW5F+p6LLnlA/G/PN/37uy/JBli91xfEXUH70+GEQHM92TBRpLaNz4i5n4f8G0zvPvhF8YfJ/nXy5hdqtpVu3/Xd/U86lkv3qpyESdoI+Td85bHfmX/D4Ij49a9+JX7lU79xB5SdX12SbP/jX1eDI78Wv/rZbrH9zs3imkvPFiv/5EPi4gf76++md32/IR6+6uPij5asFXMv13egr86tSduqZd94suNE4EA8/Y2svatfTrl8+2vVx4wJAXIqHE4akpPER9bv9OtEX7tDfFpuYMuvEyPnsF9uE1/690nnecEdomE/6sHL4rbTJ9/JYRZ03Hdy7yaNtAxB71v1PbGv66YYdN8xvbj5EyN2aplsIJMsB0e5A8M7yb+RneHxYuOTHp+sOkXH53SqM7YIvQk5nLz/R+T714a1d8TDX8rC0Nqtv+w4iPIIkInDyQBLBtbjNj4h/m/+s+76DJBviK1ny3W/TFz2UM2A8t2HxRfTycpzxI97CcG5g/eI85bKdXm8uHJ7fei31QbIlceLRXL7+MBHrd99VCyWP19yvFhh/FwvM0CqgWNp2xuhzPU2XJ/NpfeZA3HkLRlQfi2OFIczYgiQ74gHL5Xb1Kni1tpGtdt37Xt9VO5DVQHy1S3iz33bsIp2YhwB8tBPHxPb5m4Q13z9crEm2Z6Xf8DxfeR+8Z0nvPa3atO8vgfilbmzxB/J361IxtGVXdlwotmrv9OpNnT518RjbWZcpy1AVg4iliSN7dr1Yu7x+lNcxkLbWVvNUqrXWwOLmKWDi9qN122w4z9nG1iyXFofAag03JGOSzbYLoPx9vLB1/vXie2/7fOD1Pvm261essE87UJx5a3bxN6D03iqYhfa8lh1i9jbuH2p1ycN3KRaLE9d953McPB88QN+g1fTOPedJ8V1J7sGubJOFsuWZNv34j9x/b65Lrr7QPp3Z22/vd6z7UF2uOqU82HlA/H3f0gsN35u1cV3i9eMt8w6xdGDT76tltr5d8Sua05K+7FFa7Zkgf61u8XnjGWoltsysWyl/vOqch2Ryj/f6Ny1/c27fPa5HgOk6i8b2od3H74yO2J7/j3iH0b6QNtA/PTmbBJk8fn/Xfyir/dWf1dp8FW13VQrBrClMNq91H6XUevzPLHpUfdRE1mbkrbKHHy7BuTdBum9UgPlVd8Tz7xSd/Tox+LKj8lt/xRx5Y9cv1f1vDiYv7Wk1sfKq37seG1W2bJaLM668VHn72X96KpT0td0DpCrLiqfEWHXFWvFMrntVISAtgEyO9W1vjbPZ+diFGdXyM+V7bOaTDn7ZjG/97XS0dBy215VFUfmp359/1zcfuYH098V/bc6fbSolv1dUTeIJ6xFpSZ7W7fZMQRIn89VAdIYjKiNL61lYsU1yQLOXx+EFiBbzVL2HSDf3S92zK0XZ5x3x3gC6btJh/bh6tPn+p59Maq0jN4Qd/+lnJn3nw32cugecV6xTkevYQOvBnDWTm5sw7KS7fjirWLfzOdIc0B7+m1Np3iq1/cUICe871ROiPVRwfYdcx32Xaoz8wuQqjMdBrAVy5el71cKssEDZEKdMir/PrnNG31Il3IN8PLPNzr3urDfVOUBR7VRA6T2Pf/kQ9nf6AhHw5CjJlqSUL785NLr9Bpe0/6G2OMYxDnrvk3iKxvuEH/t+p1dT+4XrS71HUeA7DyQOyqOvpf/Xye1Puvb3XLYiDNAqskGuW3+fO4z2bIbqczlotZH3bav2v5S+NM0vk9TgGyzPdivTfrEJ7TtOQs+VYFqGKTa9GXF3/HuIfH6r34lfq0OSzd83/b9pHubmon1bZ8JsfNrWXgrvpNvWfvy4QfF52XILk121/QZNW2zKnMiqn/HPOAxHV65EgaHxHNzF4uPpAum4VSXvqmNf/nx2UzOig1ivu6+Aer1fQXIvt/Pkm7UNafPVZ+m4zuL7ih7cJcY7L0lu6aq81GdFtQyTben0ascIF0dsTzd5yXx+OZLxIr8qM2i1deJp4LOhoSWL48PJvuO/JuXrBW3127EdcuvgwnvO/Uzq+UQ5FWT2ncK6pSaftrjrO2313u2PdgdrjpNdNH59wyvmcvX9bKrttcfgQ0RIBPvzm8QH1n2SXH9/D8ke75OLbem060cGtot/W86/PwW8blPbxA7q9qXwzvE+v9wvrj+kS5n9YwaINV+7v47VBXvra6zdbzGrnJb7H5d57LXd9U1terIT+ko0UXiZPnz5WvFl4yf52VcU5tRA8/hAHwgXt+xRdz3Yrs1l50lcbGYe/aQtS0qsxQg1WRDdqpzeu2dfT1gUT8RN6eXspwhbn72TcfvVb1t7CNjCxT2mQr5AN6cFLtBPPHzLeJ0uT90CZA+Yx9tW8/+norltO3L9W2q/R0szcuqbpua4vVtefX2s8SSlZeIrfv+Of9JTh1hXXKFeKjTqX7qLAv9TKc3xNOPPC3efC/yAFm30G31K2F4Aej7kkFCr2ce1lEb/xk3ipsuyWa5j/v27uojKcXrexq09v1+hmww0+k6HHUUb0nS6fS6MtT1QWOcKEgGd+kpFmvuKA3CR9MuAA3e2iG+kh+hOO6ax5O/eFap5bFO3HTzqenfawz6S9otv9Zi3XcSw2sf25za21aAfSenrqmrvHbz4PPZLHbLO2ZmbX9zgBy8skV8Ws7Ulq4xHIhXbpMd9LL6mxnlAXLZmr8qD969Kg8CldvWQBx9rzxcd974Rw0cm+52Zwww8wkIrXPXO/MiZKvTaA3Dm8d0O+W5rwDp3s/NgdY7ybaRfdfqzxp+n+HYIb9LbWPl6zEJ9Oc5f2+VHfDy7Ul+di/l2J7U8lAD8H96PG87lpwmri1NUJiK/aV2v1DLr77dzfbRyANkPtnQ7sZa3b7r2AKFFSDdZ1WMGCAPzYvN2vZ83iq5bRwvPnWFto072rbyutc0TcrZ38HSvKxq1tM0r+8S95kCB7eem/4748Y3+SmurQKcYxxUjD9OuVW8UPUGqm1rs52NyTE+g/TGlaD+oLEMCCvoC/+Ne8S5aWN8qrhpT8UQpe9Ba9/vp0vf+wxx28v+b6zuetXP9Y8mNcjqfofCesW1m73vGO0D0LBj73AkYmpoy+O36rS+uuv9pihAjrDviMFPxU35BfF933l13PtOqjhFsyaoenY+WdtvX4N1szgreQ/V4Q4nXirucpreDVdeS/IhcebcC8Kax82o75X0M72U17alZsutGyGo7dSrPcr3lcp/M7wW87gbfmIsq8NJ+5eGWGe4bEMNxOoHVdU8AqRaNrU3fhh+n7oBnNuIbU6+PZWWQ+U6zT+vtN3kf4Nje1LLY/heR8WLWz+fn5FVs60fTj5LTVQm/7Z6okAtv/oQnYUNffDtGpC7fmY5/JS4/qxTxOrLtojn65664G14kMFYH++97TjKJKvFESnH3UtDBYpnvp39LaX38GkvGl6bfU97XZW3UffrcsY+4Lij74/WiZXJ3yGvgXxU/3l+R9+d65uWVdU2NVvr22nwlNj4YfmdrZuHVbU7JerSlpXiWnVakjb+qL3cxbMPH4djfBrmppVQG1rkaa5b14s1xfVmy8Sy1RdWn6JTer2c6TlDnLvxIfFq/pKU2gHznUkeZk7vmHTGD9ydr/X6sqPiwCObxLmnqVPY5OeeLa7a+px4S59KrB3o9DPATpfn0g1i3vut1F2vFotz7/qH9jtDW4OkAZOH7N9/gbir3zsmpNSOvPLmvT1/d5/BSEXjV2i5naTMz82ebXba8BRjuR/sqHruT/lz5HVGay69QzzjODQ6eOs5sXXd2cXsqLzB1arzN4lHnM8wNL9Xelqf3KaXrxPbnaOZpuU3C/uOOlKWLOfao7EdjXnf0b9/dRuX8Ox8VNvvWmdy//jtK/eIC0/I/vvffeeJ9Bob5wDg5b8WX1KD5ou2iKftG1bl32v1Lc+JN13/vnVtE5dbg6wmleHeZ0BYyPeVun/z7ry4cd0W8Zy+cxTXwiT7gf+hx5wa0A3XUVOZbVz9fj4caP2tePASOfhpGiip73OKuGmP79/U1OY0yLen0verXKf555W2m/xvcGxPanmY7zUQb+3YIE7Ijy6u+vYucUhvA4vJlDYTBT7rUx/Euwb2rp+ZVDCS7/ep23s8Ba64jsxaH7X9QUM59q9iffRU7m37GXFtGhxqAmSbmyqp8a2znVDjOHtdlbfR9gHSf1vqHCBnan27qaOPpbucV7U7lr//8TlZXjntm+JeGdrlY1QuPi3dJhpvCqqWo3PbCSMJkNUNia0+QKok7bj1edJY3v7Z5enKkYPf1edfngwyh3cILM2+6Y1r8Xp1u2GrI7EDYfJvb0s35mVJ4+dYebUB8rDYec3HsxUqd/61l4svrB3e5tho6NVdmFQDsES/7bfPDQ+q/fTmVeIPnd+zwbvJICTtdMc1SB0G1Isf6P9UPNWBjTqILC4SL/gNRoq7lZUmRDy2k9Twcx/YvUmsShrVdDLk0gvFanUe+/s/JC57yJ5tyh5BIZeF/jlZOCzvt4NX7hD/cZl8L3ndXv7+xQSM66iQvTy0U+ecp+7WLb8Z2XdevSM/BbPpetCuxrvvDLQj55VnYUienU/W9ufrPW9Ds37gqDhw/zpxQtqWrxQXfHudWC3XaU2dvul+cetZartfKTY8qT24omXH609tu9p3Mf52NQhyHHWuDBt18s8r/o3j832r9ef7DBKzMpd33X6eLA4VIDfvFA9vPFN8cGV2Te9g4J4CaxNaqtV/l0ZV21PlOs0/r9R25H+Do01xB8jM4edvyfdHOWFyT94Gau16q4kCtfy+LO5/093Xydp2hdxH9WXsWu7N6+Lg3Rdm7bh+VGRkg6Rfz//mpIz1ka+j8l00a+7KqY6aOZa5Wh91N3RS/VLd3anVJKyrLSomm5Lfjy9AVq2r8jbaPkCqR7uo+rn40fnyM5LvcMX91ngre/xLtwA5A+tbrUf5uXkZf0d6k76KfallP1YcdNM+Iy39EhDrxkpF3XhetnxLR47zanl5yiiSANn+NK+qADk4sl88cs3paaNTHjyrxjIJdDc/a8zCyaMlN6VB0ey01QZx3Prt1qzdEbH/kZ1iX/6fKbWStZ1p8MoPsp+5nvXleH1Gu9X+5feLv9Mnxt87IO7/UrYzlDaIyvcbXbq8O5yCqmZF+r+l+tArt30y/Yxl395du4N0obaz0k7lWeWd13Mw4ly3XbYT9bkrxXErVorLHvyFNlt1VLx4W37U3Hpepwqwi9b8wLoj7FHx5tM7xDP6xT7qdKgk+Nz0rH7dzUC89Vw+iCmdZuZYHod3iC+njaJr8FC1/GZk39EmrprvSNvd2PYd7fvXXgcu5R2ce+DSIF9vWT+QrPvbzxFLPpDfkKbFQ7SziZ3D4vm5S8Qn1u8wJw/tjrfydKcW9dYRbT/Qrrm7ILve1/jb1XXXrtMEK2+4oupes09K5ftK8Rltr/mrqXtfSN+5mRrQudrANqr280wRIPP3luv8xa2XihNWfk3sdIah5tBSrf67NMq3p9LgUg3cS4P8/OyJJfZzIPMJb0dbVRcgpeHEnmzLbxH33bw2a+9bPJw8owb98uYh1cvy6G/ka5qe+dhmXeSfV/Fw+05UW5/f7NDYLisH2zXfVb1fTaCo2/arxrK66vcZPidWrtMr79kurl93j3hFrcqa71ZS+9qKv1/d30LbFrO/p2KdVi7fRHGwofr7Ni8rx/echfWtXY/6pTXHp8tI/4y/v/2s9PXOG27lfUzltfzqWm0ZDuf3iNfSPuslcdfFH07e0zr4pf52+Vk+1Wb7G9ExTQtVp1aCkeK1Iyh/epnj0QfqItqKw7GD528UJ8mFow322qzsglq41iBfPTuq9LnO1yeKZ9VU3DTj8Dbxn+SO9rHk923erwfpcvDeCNRdBB2dZuuqeJ6PprhOcQw3TGrzTKO6yq4FcZ1+4zkYca3bTtuJ+tzF4uSb95YH9sV59OYpXm0aRiV7hlDV9YsD8fymVel7mUe93MtjeAMW+2+sWH4zse8MQ3DWnnWrNutqPPvOsM3TO+CDexwzk7KaZi/z618y+XUzakYzX2/D9vmoOFI62t+RPbjI/9u1rBurartS76lvH6N8jrM9yfeVK24XT9+5SZy7+ttil+u7yOsuP7danHvDg9ryHpUaiLXbHsvq28lyu6T1t87TMWsGho3qv0ujfL26+sIVK336x+4BUhq8/qD4Yn7qdlpLThZfc10n3Egty8vEA42zY67lPsq66EqdnXaquOmBH6Q3mDG2S3ufL6jv6jiTSvUdjmU+tkChHNyaXvu9LAlH6T62Zm06GbH4rDuybb/mu5XUvlb9/dbzsB39Zvb3VKzTyuWrxg3Je8lav1384kD5UprmZWVvUzO2vhOu36mfFfu0TznaEXUJ0aI11mPNqo5ANlWII5By0Pm1x1sEtYRaCeUF8gnxtf/5i9KGJ6WnkiWf8ZXHflvxGXkHoR19UacwLrvqYXHINSDVOXamVHEjCetC1IrXq0PJ1RthxcZd9fk9SJe351GU4oHOpXXkUy06FzXgGsPfParqxsJzMOJYt922E/W5VdcADcSOdeXvfOjuC7P97YzvNzyT8qfippOT96+7426+vsyjXlXLY3g6unkkzv36Wdh3Bs9cl13/mW7/HR7dkU+kVS8DzRj2HXVXT3v/rW6zm0pvA/L1pzrvfL3p22rXSR/1cOuCPbioegxDm3I8ciGllr8+GKk70vncfxGnyteXTvFSpd9W/qh4+7U94tG5L+V3D1XlGBQl1N32SoOGkaj9reX2WKL2c/37l8t4b/3od7Jczcli9X3sfqXd57QvR7tedTfEdHywXPzp+T8UP9HOoK72mrjr86ucj+cpBpLOEKAMhtcmy1pxsbjL6+ijKnWjkerTWd8u7hTpWu5V62KM8gnS9LKIvO0wtp18fyxfslLzt6p9MnigUPdGOFXcuvlr6bqUl4bsXJ+dZZOepvxK3q/Vbg851Qc6X6uus7S2a0e/mf09FevUblMVNXG9ZFl2NH7FSnGcPPp1898az3JvXlbWNjVT6zvj+l3tI0nyR6dUXoZlnB2T0M4gG8+lM+Mx2jWQ6SmlG8Sfpef4r0x3JPOdDiUD0ezfmA29q7Sd5OB94kJ1ysfK88X19+8Rb1Y9aNexMynFTUH0Z0NWvF6/cLy+rOVV8/mj8r+OazhwqHx8gRo8jfp9+3qfMahuLKoCU4VXfpA1WtqdirttJ+pzrZlEjfM7J4Oc60/KBmXv+8AnxVVzj4v9riM96pQW4/MrymiAa5aH81pA9+unft/R7obo/J5tVHXSLj3vO8PrHpOBQHot4vD7Z6e1OSrv4KpDkeNUOLXt5Ovt5AuGQe3u/DoZc303l728micjeqCWf5sBnqS204bX/9NjG8WH836rqBVniw3JfvvSbw44Hpx/l/hKerbEh8V5tz5k/a5cT+xv+0ChYT/QbTmq/fxDYrk9UZJU5bVhxU0zlolz79L7n6rQoj5HW14jVct2PRm6qSOm3R6TYlKDy+rtIz/NO/2OK8WKleoGZ2vFbfYz5UqG67JtDfsQ13KvWhfj9cz3N4ld8gh7vi8Z247aHx1/S2M5lvk4A0VxeZS80Vb+MPn0PQaviwflpRqyv3xshMd46AY7xJfl/nHaD4RxlZmj38z+nop1WtE3qb9v7Te+JU6Xy/Kib4lrV8ttc5lYcc3w8rHmZVXepmZlfSttPsNQsczdtMv8XPdtiVgSIN0zoy5VK6GY/S5dY6U2rDYz+ubNM+T1kXOX/fvibo7ymoQzNj4i/s4Oko6daUi7KYi6Jqji9epvq7vINivr9M7azx9NegRqSfs7SRanHtbdUl3tvKN+377ep3evidvPlNuc62hfTWByUEcA9aN23baT5s+tbODeOyAekTeqyG84lZ0qvkU8q9+5UW2DyT5iXrfjKONZdnXfazhjPrwbqfv1U73vGEdOtogbzpDLuMPgyqfD6HHfGWh3P5WD4fvTINfi+6vv0GaQo9px9dp8vRXrWz8q88x16RG10l3pWlId9ZXbq85Y6YHX355Q22nT6/P3XXzKhUlovFaclfwdw38zelhqPXgp+l2ff6Orb6/qBlNFH2Rc36eOpLQ57dLW3HY6qRt1OUu/S7Tr921qOF5Ry8O9fRwWz9+sTo1fKS574Bfit4efz+/9IJfTaeLaJ/XjPbZDYtf3taPq6hmAjuuu1HVawz7EFRZdPwso35eMbUc9k7ZU2WOC3vexdeK/2b9rcVOVumfJqstcikkwR5Wve/t5MraQ6y1/zI8eIOWvkxC5X97tXLUXlZcHaFXzdxQT2Pa9LNT7a5dAZH1wxTp19U3qbs/LvyWeelkLvIefEteeqvrDLER2f4xHYqrX91Bdm+fkWuZO2qUzn/y+2HNgj9i1fZuY+87G7IaJF2xN+la7DWhT3xe7nKff9OsY+8YddSoHua6glqobzLdz9OAecf8NF4o/rbrDpdqZqgZjakdRdyWseP2ua7JT9rzvjNj0+aPYe4s4qe2z7LRnv5mzvxY1eBr1+6r36eFGPaNe86jq3vROFnUdpM9gxP1MuG7bSfPnVu9bGXmjqsfn1osz8tl/4+59r90hPi3XhX2NYaOG71U8j0hdW+l+/dTuO0l4vPtz2alH2fWbIwyuWncYiR73HfXYItUuZp19i++vvoNz0GvLl4t6bb7eytuqdnpXx2enZvvB8eLae+8WFzkH7l3KurOv9be/u/9Jc7Bilxq81A0I5fUm7yYDyOKxJPm+Uizfo+Jt7QjvL5OAc4r8DqtuFE+XTkXcIzZ/Sm6Hp4obnho+D618V+kqajv2GPAY6tuF+sHUYfHgpbI9+JD4xA//V/775vavWsd/WxUgi7tSu4+uDqspZLYIkOqolPydvOZxu3ZzMyNEelwPqY5KOdpM9T2G+6WrPRuhjetD3naUth352La5TeLuV7RtXPVrrnZSXRfmuM5L9aXpsu2h1Hf9p+3r0smRYkI1b0dK7aDq1xzvVVmOdlidjVG62Zp6f+3f+AXI4TNo0zPV7PfLz8hRN+/rPUBKU7C+dXabV3lvAVX5PQbKd5odVnpGyT/eJy6Uf7P2+Ual62TYnrevMPv3MYuSL9j2Rg61g9zi9JXPJIO28oB75OcRJg3uDflCNAaqauOvGYQWt6WWz4Z82f16dUcl7zsjtvj87rJr2ypPRy0MT8mpvJGJogZPI35ftbwWXfP4aOs10VcDkG2X+fWA718rbi/dCKj9YEQ9zLuf7aT5c2v3LV0yKLn/0myyZviczHlxdTpJ0v5sgkzz9yquDUyfDblTfMXx+uncd/Rb6avrkUYYXJU66Wp97jtyouO+G+4Qqh+uHUjorBBVL18u6rX5eitvq2r5NQ3OkzKOhCvq3yfb8e58Vlyun5HL2r6tv70IAKNUaTnm+5Zr+RZHvd2PWhnsvSW78ZwaqHobDjjabI9l9e1CfYBMHHxKPGI8gb59u1s2yr+1aaeuGhPdNu2oQFN/mnAFyMFbf1Mcyak8yvjePnGbCpHyEiA9YFZJJ8Zc21t+2n7yXsP90tWeuX4WUEWgUEeu9QME6i7kfv2KekzS8eLq7W9qkzJmZY88WSwuv7/6NT+5JbtbdvFd05vFaePbvB0ptYOqX6u8PECrmmv71KUhpYlZxzZQ2+5bfVNxxuCK7PE7xffVt9/XXxA/y3fh5rFJzTY1zetbY7d5fYxZs/d6Wdx2+gfTMzRXn79OXHPrnUm43C1eSr5P/bXMVcLu38foR1ea1G9Iw8ZZ7/iKm7q0aIibODsutfHXDkLVqQfLxKc23yIulivQfn1+t9j3LblMPOBzUYT6fPs89Z6kd8la9T1RN6FfnJfvfPSCJW9MRh20q4mBPh403Dib07L2HJTvVjfgaDMY0R4AbZyGleu0nTR/bnMjrVHrsGjwh7cVd97ltVKb5TGcrTzumu+Jb6TPabJeP437zuCQ2CGfW2ms4xEaX6uTrtPnvmOrHUjoSttQnXy5qNfm6628reY3GnEFRlX5zYaGkx8adUv5yiPp2o1IHM/PO/zWoeYBuGT97bU3Q5DVeBOdpEqPPMj3rdLy1Sb71t4qfvKG9e+ScJk9x7jhOZ611Hbcbnssq28XGgNkSZt2psoo/9ZUDJxP/KaYf7t6Syle52r/HcwAmfQfuzaJ09S1sCsuFj+uu85RPxKpTnHNf+Wi7prpmpDP9n39poiu9izsALOkIlAkCyL5/nk/c/kD4he/VX1a3Q0YXeomkIfa9Lfl7Txpf158edjG5O1I6T1Uv9amba18rTrt2zEp7Gi7a9v9/PXZ37FH3HiSfN+VyXaSB9OG79u8rGq2qale30P27yrvLaCq6SY6SfmfUdJmnw27fx/j8xmNK6F4sKZ259PiNDi5oVjPiEukz5DcuF78+FX1jofEMw86bhZSzNqapxQWG39DIBo+GzK/41Tp9erUGzkrcot41ngAZeK9g2L33CXiRvuiKnVKyfvPET/uNl1cL531cjzcWikGHE2zqjnV+IwUINVO2/3U5LEpnmvkGnDUDEYGR8Trex8Vt158crZ9VJ5W1GU7aR4EufatfTu3ib3FKXHKsOHVj66poxbpIOS+vxPmkGUgjux/RGxYt1UUu1mq5eCseDakukmL/fop3XeS733ggH6kZITG1+ik64x336kdSOgcg5Bq+XJRr83b3LrOuFLNciqee1pxZHY4c55sf1Z4PPjwfxYnLC3fQdDJ629PNAyw3PJ9y/Fvjh58Wmxdd2b2WIhkn1r26fVi7vH94sh76pTq5O+YG+U5pGo7brM9utS3C94BUrXJHtfzD7Vsoxr9o3jiunPEh/NLYdLlvlpeq/qo2Pu6uiPiUfHi1s+Lj6S/X5ls3+1OKzUC5N6bxQlyW0n+e9Hq68RTjRtjogiRDdtvcZmKu70s7/v543eMR/K0aOMOPyWuP+sUsfqyLcI4kNyHykCRKMZ4i8XiP/9kfq3nt8RTPjuCuqFcw7Y2aqBI5e1I6T182ouK1w6e+pbzDKiUo/2qbffz12d/RzaB9ZH1O4c3j2r4vs3LqmabmpH17d3mGcu8u6O/kTez8xmX+Lx2dMfk/9tKm5VQnC6qDo8n9Afppjf6OO1CceWlZ2vXI+izB2oBZDfekYd15cWky/OGv3TXNLXxNwYibfZaluv1h3eKq9VdL9PTsM4WX7jqQrG6uB7C9cgT7aGyH/ho8n0vFGtWVj3zq5t0mWrLc0g/Dc/1ewfV+IwSINURJ4/rZ4NRF547/z41GDFPsVN3FVS1aOUlYmvtrLHvdtI8CHLtW+pn6TU4ay8XV55/RvFdy89bS7bv4k5/ye+Xn1zsO8Xfp91NNtN+cFac0pu+v+P1U7fvuIzQ+LbtMMa872TbTIvv7xiEVMuXi3pt3uZefv/PxO5tW8R1l14t7note2UTdWOq8vWy9TPQh5+/pbgrsOuo0OCtHeJr6fZn3kGwZHBI7PpGfgSw7XM4GwZYbvm+Vfdv3juQXtf8iSLUZLX0iodHvDOo2o67DmDq2wXvwVTrPtqlfRvVzlHx5t5HxdzG4X0VZC1afppYfUr+TGt5yul8i9NJc0aATL7i4Z0bxF9cs0P8ou0bSEn7OTf3fNKjVyuud3Y+U1vd8b5p329u4/S7avd+lkRdoJCMI7KLxb/+6k7xzx5fQF03qD9X3GXUQJHK29DSe/i0F87XDidkXZdgqOcI66d6Zn/P8eJTVzjuD5E/1L74Ow6+LF7Rnznb8H3Vsqq+Ac1fiU8tl+vLsU3Nwvp+b5+4/S9XpN9v9W0vVf57Q6cAmT0Calfap56dj9tku+czLhlhDNNB7wFSP+JoPEPu8M/Sm+Gszk9hkiUHxmvWbRGPyztXFd4R+x/cnAx88wf3ytcuSUJnMoD+7iMHkkVsURt/m85J+26Vrx+8Lv4m6dTXFOFWDcY3ifv3VZwilWxgW7942nBGefWt4ie9rrssKBqzRtL+H4rT5N/e8lSbVA8BUp1Gc/LmFzx2jhCGkwTua3vVYCRbr8PKJivWXLpZ3L/3YHkbc/HaTpoHQa596+DTW8SGJDSqyZM0mJ12odiw9Tmh34RVd/hn28T19r9Jwpx8BMiB0qNwfAZnw06t8vXTtO84qcY32x6MUy+bquVzIMe972Tbkew8ful4bIRW+UX+TXcKzB4dkS+X5LVbb90ovvDp4fpVVdsnaMrXaeXUNfSlZ5kOxFvPqfDYcMdK/Q6C6WlR+c+VwfCmSYuWyM5ZHvF5tjpsKj4DwkK+bzX8m/Ro5JdPs5Znsv19er3Y+rSjLap7XmVR6nlq9adQuSq77qaqnTSr7eCouIylNIHVhk8b5ett8dMfXSpOsP+2ZLyx6vxvirlH94jXW5xmZgfIcRg+qqfi1Gb1TL/GO902DzCLgwD6aY59qQ0UR8WB4rFww5J3Na59lFtB3fiu+Z4bIwUKpSlAdrwGsrjvwJIrxEOOdanaUP17qb9HX252Vf4dDe1bm/fOyrFNTfv61vqUrM9wneHl0DJAHtxzp/ju1y8Xa7SbdqnKxk4/Fi/MSoDEBKWH+5dbpzYlgWnuHHHm3MvZz1RDYG2IncsVMtPTApMNdMkF4q5xnHZYR92JK6ndP7MaYvnw7s0Xi+PS775MXPZQzx0fppdz33FRja+1H3hUbYcRYN8ZBkh1wyP392xbp3/vdrHOOkKWBeyzxRe+s0Vs2/2S41qO4R1Hh79LfrbnDnHeh+W/t6/rGYhnvp1fD6Ofhn9wp7j+nPyUclkf+Kg7vOulT1CedcfwKL0+EJB3F3x3ePrUouVniqvmtlltytvD8KbfAbAywGmvTwyev7HiZifZLPOjc98U567WJlpWXiLmnj0o3ty3TWzQAnr63bY+PRxMqQnA/Pd9VzbAUqHNPFNDlfs5kG9kExbJ9qAvl9ee3iIuyB8zMzxiMvp+Vl9VYVNuly/lR83PFsNJNjlovVzcum2bmFtn/jzd1kunupqKANlnaX3v0Re3FGdwpfvHa8+IB40+8CXxyDWn5zcK1Pvs4X5YrI/Hv5Vf6lA3wJSXPOxxXD7RA2egSILE41vEVX+u9t0PiTM3/604eOARseE/DPfndHtce7m4Tu2r1nXHTcFLZweKwcBas4ND4v58wrTyKGxTgCy+d4tS7YR2kCObZDwkdt0tb6yST+rduUGsSidbjxfXaqewZH/PGeLmZx03icmvx6vsm1oGyOob0KgJq7YBcjrWt/mIrO3i0FtJn5o/LzoNuXfOi72vacvhLa19yB9VIidgK6+xT14vJ2vkZ6bv+SdniHOv2izunN8jXjP+1i7tJQEStvRw/8fFZQ9qF9onO0Kx0b52t/jcyeUOv3Ppz3dLDW8A4Xezlp4U1ze6dphhLT5ra+ejq5hRrn2nZITZu8YZxzD7zjBA/nNp8Nilfn1EHnleIpbIzm1jVWC0qRtAuPfP4655XBiPxP/lVvEf04GAFaxV4Ha8RxZiHW1WXirkFCFSXZ+vXz85eF3s2Pip4rTvUWrR5Q+Jd9T1ONrPs0HgP4k9d35ZrPmYdlZNWtmRxrnH7TNrsmuWjfC85KPiuqeT7137fMPR67qn5LKpP+rnPjIj7yZYvc7N61Zb3GxppMoesfH/XrxPXPf1jemp//pjOYr6wEezM6D228EwP821FCZloP+C+O+vm8vk7+++pP91kve9h+a/WRyhKbbl4tpx87vJ6/Y3PP5/si+VqtkPG27MNzZ6oEgG2XfecKFYle+rshb/+TfFw8aNMgbirX3yrBrtmeBWLT7tv4jn/u9wkN3mXhB2oBDzG7Kj5Nr7ZlXzSKKmALnqIsfpnlZdsVYsk5+ThrfhDeuGlySpu4yq75PVojV3GGOcYbvv+K5NfVPLAFn6Ows1/eY0r+/8uxuXCyXjiLmLP26dMdKt0r4hWSaPNvap6m+t7/OyUn0MARIu7+0Tc+f/lfjrX49/47AVNyJqfU1Z394Qj303a3izQYG248jrar++Wdy523GaMyA17jvjC5Ch9p3agURHpdnaRu6AIK/J/e79LzhOv87C9ccdwfrgHjXLax7ha5Q/y2zxWVvSZ5lJr96+Qczpzx3LDY68LvbOZw9vLg3w0tOLht+//Pus7tor31cFr2Q9LzlerLpsi/jb/KK24iZXyc/TyzHu3C1eKt25tezogSxILu78SI8unhTXpROR1vMzcyowXXS3fmQmWYe3/yexOl9WwzpbfOHWbWJf1Tn342RPQMgj2MURjZbbU3pztW3iu5dmNz0qTX6MW74dH3fRPdo178N+sKhb7xS7jUuBpHfE9qtXmhMk6TL4ptjhdYFmj/RAUdxEZZlYcU7NZQ7Ke2+Ll3bL/VQ/5U+dZivPxjpL/FHLO4E/+Z2sfcomTBLqGYTaspLXxV6y9YXq0xWbAmRFIDNYr82u9Tbvpn/wsVuNdX2d43KU7FnaFQ+P33dv+u82z1c8Wb7h+6rndGfP2XZRD7p3fP5Ur+93xM4brhM7HRcmH/3NS2L3dnn66XC9FJXe+yFr/9Zc6vh9WreKx9KnBrThMy4ZYQzTAQES7ag7wPlcbwlMlYE48pYMK/LOZ/mP2spPbXTOJLLvNBscFUd7HtMOXj8gXu/5PbuTpxN6huDCQBx9L5o/ZKqoCYjmo+YtJPv48Nls4QzeOiT6vhnqxOSng2fXVicOH7RO1/ORtNe/0Y8e23fVHrO8zS9tE4Mj4tfyDI42f5fjta0fR9SXmof0j2yW1vfE+IxLRhjDdECARCvFedydd35gYWLfAQAAs4QACQAAAABohQAJAAAAAGiFAAkAAAAAaIUACQAAAABohQAJAAAAAGiFAAkAAAAAaIUACQAAAABohQAJAAAAAGhBiP8Px3YliW1ggbAAAAAASUVORK5CYII="}},"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"610FDA7246E64E6AAA8669EEDEA33BB1","mdEditEnable":false},"source":"# 稠密连接网络（DenseNet）\n\n![Image Name](https://cdn.kesci.com/upload/image/q5l8mi78yz.png?imageView2/0/w/600/h/600)\n\n###主要构建模块：  \n稠密块（dense block）： 定义了输入和输出是如何连结的。  \n过渡层（transition layer）：用来控制通道数，使之不过大。\n### 稠密块"},{"cell_type":"code","execution_count":13,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"644303248ACD46C9886553FB41750A4F","scrolled":false},"outputs":[],"source":"def conv_block(in_channels, out_channels):\n    blk = nn.Sequential(nn.BatchNorm2d(in_channels), \n                        nn.ReLU(),\n                        nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))\n    return blk\n\nclass DenseBlock(nn.Module):\n    def __init__(self, num_convs, in_channels, out_channels):\n        super(DenseBlock, self).__init__()\n        net = []\n        for i in range(num_convs):\n            in_c = in_channels + i * out_channels\n            net.append(conv_block(in_c, out_channels))\n        self.net = nn.ModuleList(net)\n        self.out_channels = in_channels + num_convs * out_channels # 计算输出通道数\n\n    def forward(self, X):\n        for blk in self.net:\n            Y = blk(X)\n            X = torch.cat((X, Y), dim=1)  # 在通道维上将输入和输出连结\n        return X"},{"cell_type":"code","execution_count":14,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"72F1294D9DDD454D9831BCA94764B9DE","scrolled":false},"outputs":[{"output_type":"execute_result","metadata":{},"data":{"text/plain":"torch.Size([4, 23, 8, 8])"},"transient":{},"execution_count":14}],"source":"blk = DenseBlock(2, 3, 10)\nX = torch.rand(4, 3, 8, 8)\nY = blk(X)\nY.shape # torch.Size([4, 23, 8, 8])"},{"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"23FDE43573EA40389189D2D4659A59D9","mdEditEnable":false},"source":"### 过渡层\n$1\\times1$卷积层：来减小通道数  \n步幅为2的平均池化层：减半高和宽"},{"cell_type":"code","execution_count":15,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"ED891AD5805040F6A93AD06B1E7C0FA8","scrolled":false},"outputs":[{"output_type":"execute_result","metadata":{},"data":{"text/plain":"torch.Size([4, 10, 4, 4])"},"transient":{},"execution_count":15}],"source":"def transition_block(in_channels, out_channels):\n    blk = nn.Sequential(\n            nn.BatchNorm2d(in_channels), \n            nn.ReLU(),\n            nn.Conv2d(in_channels, out_channels, kernel_size=1),\n            nn.AvgPool2d(kernel_size=2, stride=2))\n    return blk\n\nblk = transition_block(23, 10)\nblk(Y).shape # torch.Size([4, 10, 4, 4])"},{"cell_type":"markdown","metadata":{"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"AE6DADC26CE04D2C86F49867321D491F","mdEditEnable":false},"source":"### DenseNet模型"},{"cell_type":"code","execution_count":16,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"E32E30B6657E4CB5B3027B663927AA83","scrolled":false},"outputs":[],"source":"net = nn.Sequential(\n        nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3),\n        nn.BatchNorm2d(64), \n        nn.ReLU(),\n        nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"},{"cell_type":"code","execution_count":17,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"8464B9866CBF4EB280381A63A9F4557E","scrolled":false},"outputs":[],"source":"num_channels, growth_rate = 64, 32  # num_channels为当前的通道数\nnum_convs_in_dense_blocks = [4, 4, 4, 4]\n\nfor i, num_convs in enumerate(num_convs_in_dense_blocks):\n    DB = DenseBlock(num_convs, num_channels, growth_rate)\n    net.add_module(\"DenseBlosk_%d\" % i, DB)\n    # 上一个稠密块的输出通道数\n    num_channels = DB.out_channels\n    # 在稠密块之间加入通道数减半的过渡层\n    if i != len(num_convs_in_dense_blocks) - 1:\n        net.add_module(\"transition_block_%d\" % i, transition_block(num_channels, num_channels // 2))\n        num_channels = num_channels // 2"},{"cell_type":"code","execution_count":18,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"76C89C4331674B8C8F531F4157CACAB1","scrolled":false},"outputs":[{"output_type":"stream","text":"0  output shape:\t torch.Size([1, 64, 48, 48])\n1  output shape:\t torch.Size([1, 64, 48, 48])\n2  output shape:\t torch.Size([1, 64, 48, 48])\n3  output shape:\t torch.Size([1, 64, 24, 24])\nDenseBlosk_0  output shape:\t torch.Size([1, 192, 24, 24])\ntransition_block_0  output shape:\t torch.Size([1, 96, 12, 12])\nDenseBlosk_1  output shape:\t torch.Size([1, 224, 12, 12])\ntransition_block_1  output shape:\t torch.Size([1, 112, 6, 6])\nDenseBlosk_2  output shape:\t torch.Size([1, 240, 6, 6])\ntransition_block_2  output shape:\t torch.Size([1, 120, 3, 3])\nDenseBlosk_3  output shape:\t torch.Size([1, 248, 3, 3])\nBN  output shape:\t torch.Size([1, 248, 3, 3])\nrelu  output shape:\t torch.Size([1, 248, 3, 3])\nglobal_avg_pool  output shape:\t torch.Size([1, 248, 1, 1])\nfc  output shape:\t torch.Size([1, 10])\n","name":"stdout"}],"source":"net.add_module(\"BN\", nn.BatchNorm2d(num_channels))\nnet.add_module(\"relu\", nn.ReLU())\nnet.add_module(\"global_avg_pool\", d2l.GlobalAvgPool2d()) # GlobalAvgPool2d的输出: (Batch, num_channels, 1, 1)\nnet.add_module(\"fc\", nn.Sequential(d2l.FlattenLayer(), nn.Linear(num_channels, 10))) \n\nX = torch.rand((1, 1, 96, 96))\nfor name, layer in net.named_children():\n    X = layer(X)\n    print(name, ' output shape:\\t', X.shape)"},{"cell_type":"code","execution_count":20,"metadata":{"collapsed":false,"jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"F70449A99F984956B8BEC295A53E0A99","scrolled":false},"outputs":[],"source":"#batch_size = 256\nbatch_size=16\n# 如出现“out of memory”的报错信息，可减小batch_size或resize\ntrain_iter, test_iter =load_data_fashion_mnist(batch_size, resize=96)\nlr, num_epochs = 0.001, 5\noptimizer = torch.optim.Adam(net.parameters(), lr=lr)\nd2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)"}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.7.3","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}