{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Neural networks and gradient calculations with Pytorch"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Introduction\n",
    "\n",
    "The latest Reinforcement Learning assignment involves some basic neural networks and gradient computations. Pytorch, an open-source machine learning framework, offers a bunch of features to construct, train and deploy neural networks as well as calculate derivatives, etc. This tutorial aims to offer you some basic background of neural networks and gradient calculations to help to start your assignment. This tutorial includes two parts. In the first part, we cover the basic linear regression model and neural network and you can skip this part if you have prior knowledge. In the second part, we talk about how to stop gradient. This operator is used when implementing the DQN algorithm. So please make sure you understand this operator before Exercise 4.\n",
    "\n",
    "If you are interested in diving into the Pytorch and deep learning, please check these excellent materials. \n",
    "\n",
    "1. Pytorch Tutorial: https://pytorch.org/tutorials/\n",
    "\n",
    "2. Dive into Deep Learing: https://d2l.ai/\n",
    "\n",
    "3. Deep Learning course: CS-E4890, Aalto; CS231N, Stanford http://cs231n.stanford.edu/"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "import random\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Part I\n",
    "## 1. Let's start with linear regression\n",
    "Suppose we have a bunch of data points generated from a linear model $y_i = wx_i + b$ with additive noise. Our task is to decide the linear model's weight $w$ and bias $b$ using these data on hand."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "# generate synthetic data\n",
    "def synthetic_data(w, b, num_examples):  #@save\n",
    "    \"\"\"Generate y = Xw + b + noise.\"\"\"\n",
    "    X = torch.normal(0, 1, (num_examples, len(w)))\n",
    "    y = torch.matmul(X, w) + b\n",
    "    y += torch.normal(0, 0.5, y.shape)  # additive noise\n",
    "    return X, y.reshape((-1, 1))\n",
    "\n",
    "true_w = torch.tensor([-3.4])\n",
    "true_b = torch.tensor([4.2])\n",
    "\n",
    "# generate data\n",
    "features, labels = synthetic_data(true_w, true_b, 1000) # generate 1000 data points"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "# plot data\n",
    "plt.scatter(features, labels)"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7fb57f245d00>"
      ]
     },
     "metadata": {},
     "execution_count": 3
    },
    {
     "output_type": "display_data",
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAaiElEQVR4nO3df4zc9X3n8dd710MZk7ZrhFvhCcYWipyLz2VXXRFXPp2Cm2BSCkyoqIvINdLdhfaP9GqOW3VJfLFJXLF3vhyWTlVVqqLmFB81CWQPzmlNWnPK1TrTrON1jAu+kgQM6yi4Zw892KGMd9/3x853PTs7353f8/0xz4e0wjP79cxnZPPej9/f9+f9NncXACD5BqJeAACgMwjoAJASBHQASAkCOgCkBAEdAFJiVRRvet111/mGDRuieGsASKwTJ078vbuvDft+JAF9w4YNmpqaiuKtASCxzOz1lb5PygUAUoKADgApQUAHgJQgoANAShDQASAlIqlyacXkyRntP3JW5wtFrRvKamzHJuVHclEvCwBiIxEBffLkjB5+5rSKpTlJ0kyhqIefOS1JBHUAKEtEymX/kbOLwTxQLM1p/5GzEa0IAOInEQH9fKHY1PMA0I8SEdDXDWWbeh4A+lEiAvrYjk3KZgaXPJfNDGpsx6aIVgQA8ZOIm6LBjU+qXAAgXCICurQQ1AngABAuESkXAEB9BHQASAkCOgCkBAEdAFKCgA4AKUFAB4CUIKADQEo0HNDN7Akze8vMXqp4bq+ZzZjZdPnrV7qzTABAPc3s0P9U0u01nn/M3YfLX9/qzLIAAM1qOKC7+3ckXeziWgAAbejE0f/PmdlvSpqS9JC7X6p1kZk9IOkBSVq/fn0H3rZ7mI4EIInavSn6h5JukjQs6ceSvhJ2obs/7u6j7j66du3aNt+2e4LpSDOFolxXpiNNnpyJemkAsKK2Arq7/8Td59x9XtIfS7qlM8uKDtORACRVWwHdzK6vePgpSS+FXZsUTEcCkFQN59DN7ElJH5N0nZm9KWmPpI+Z2bAkl/SapN/q/BJ7a91QVjM1gncwHYn8OoC4Mnfv+ZuOjo761NRUz9+3EUEOvTLtks0M6tF7tkhS6PcI6gC6zcxOuPto2Pc5KVolP5LTo/dsUW4oK5OUG8ouBmzy6wDiLDETi3opbDoS+XUAccYOvQlBHr3R5wGglwjoTRjbsUnZzOCS57KZQY3t2BTRigDgClIuTQjSMFS5AIgjAnqTwvLrABA1Ui4AkBIEdABICVIuEeC0KYBuIKD3yO7J03ryxTc0V3UyN+jmKImgDqAtpFx6YPfkaX3t+LllwTzAaVMAnUBA74EnX3yj7jUzhaK2TRyl7zqAlhHQeyBsZ16NYRoA2kFA74FBs4avJf0CoFUE9B6476M3NHU9zb4AtIIqlzY0Wn64L7/QS/1rx8819Lo0+wLQCnboLWp2mPS+/BatWZ2p+7o0+wLQKnboLVpp2EWtXfrkyRm9897lmq9lWpjhlxvK6tYPr9X+I2f14KFp/Ww2IzOpMFviABKAugjoLWp22MX+I2dVmq9d7eJa2Jnf+uG1evrEzOIPikKxtHjNTKGosa+f0iPPnSHAA6iJlEuLmh12Ue9GZ7E0pydffGPZrr9Sad51abbUUIoHQP8hoLeo2WEXjdzobLRePUCJI4BKBPQWrTRMupZaPwA6gRJHAAFy6G1oZthF5bSjmUJx8UZou4YaqJwB0B/YofdQfiSnsR2blBvKytXcCdIwTWZpAKQYO/QeCmrXgxufc+7KZgZ1dWZAl2ZLy67PlfPuMyukVd4uLv99APoTO/QeCqtdd1foDdZ6uXdOlQIIENB7KOwG5tvF0rIbrL/2i7nFA0ZXZwaUzSz/ozLRdhfAFaRcemjdULZm+mTdUHbJDdbq1Myl2ZKymUF9eut6vfDKhcXXCNLnM4Wixr5xStLKU48YfQekGzv0Hmq0dj0sNfPCKxd0bHx7zZ4wpTnXrkPTGvnS8zV367snT+vBQ9MN954BkDwE9B5qtHa9XluBWjdQA5dmSxr7xqklgXry5IwOHj+3rEyyWJrT3mfPtPRZAMQPKZcea6R2faXUTCNKc66Hnjq12ODrH94rhda8F4olTZ6cIfUCpAA79Biql5oZytY/TDTnLtdCwA7pCbaI9gFAOrBDj6HKU6W1bmD+6s3XNzwsoxFBKoebpkCymUdw1HB0dNSnpqZ6/r5psW3i6IqHjZqVKwfvysoaaWmfdoI7ED0zO+Huo6HfJ6Anz8bxwx3pAyNJA7YQtOv9NcgMmD5w9Sp6sQMRqhfQyaEnULunQ4eyGZmk1ZkBzXtj/WDoxQ7EX8MB3cyeMLO3zOyliueuNbNvm9nflf+7pjvLRKV2WvEOZTOa3nObfjRxh/7xcuv7/GJpTg89dUobxw9zUhWIiWZ26H8q6faq58Yl/ZW7f0jSX5Ufo8sq69lbEQThZgdqVAsqadixA/HQcEB39+9Iulj19N2Svlr+9Vcl5TuzLNSTH8np2Ph2Hdg5rGaa8BaKV9ImncT0JCB67ebQf97dfyxJ5f/+XPtLQjPyI7mO3SBtF9OTgGj17KaomT1gZlNmNnXhwoVevW1faDX10mlMTwKi1W5A/4mZXS9J5f++FXahuz/u7qPuPrp27do23xaVujWvtFlvz5bIowMRajegPyvpM+Vff0bSf2/z9dCCdm+Sdsq8pIef+X6kawD6WTNli09K+t+SNpnZm2b2ryRNSPqEmf2dpE+UHyMCwU3S9qeUtqdYmmeXDkSk4V4u7n5fyLd+uUNrQQeEdWrspV2HprX/yFlOkwI9xknRlIlLPr1WbfrkyRltmzjKYSSgSwjoKROXfLq0tDY9GKvHxCSgewjoKVR56Kh6t26Stt10bc928UH6J2ysHoeRgM4hoKdYrZF3j+0c1sHP/tKS563Ld1J3T54OPXQ0UyiySwc6hPa50OTJGY19/ZRK9UYbtWF1ZkCzpfma3zNJ929dr335LV17fyANaJ+LuvIjOe2/9+au5t3Dgrm00I/94PFz7NSBNhHQIelK3v21iTt0YOewVmeW/9XoZt7dxWxToF2kXBCq1ozRL3zztN59f67+b26RaaGW/tYPr9ULr1xgvilQoV7KhSHRCJUfyS0JorsnuxvMJS2WNFYOwQ5KHIM1AaiNgI6GPfniG5G9d7E0p73Pnln2LwYCPHAFAR0Na3fCUbsKxZIKxZIkdu1ALdwURcMGu12w3iQOJgFLEdDRsPs+ekPUS1iGKUnAFQR0NGxffos+vXV9rHbqLtHoCyijbBEt2zZxNPJWvYHMoOmaq1bp7WKJG6ZILU6Komvi0qpXkkpzrkKxRCdH9DUCOlpW3ao3TqkYbpiiH1G2iLZUHz7aOH5Y0RY3XhGXdBDQKwR0dFQcRuAFTFpMu3AgCf2AgI6OGtuxSQ8/c3rJMAuTItm1u6TPP/N9FUvzi+/PgSSkGTl0dFStoRr3b12vzEA0+fXZimAeIL+OtGKHjo6rzqtL0uiN1+rBQ9Oxya9zIAlpREBHTwQBvjodE5Wh1RlJCzn2R547o0uzCz1ihrIZ7b1rM+kYJBIBHT0TBMngBuXQ6sxiIO21d967rN2Tp3Xou2+oNHfl3w2FYkljXz+1ZL1AUnBSFJGK8rTpoFloB8ncUFbHxrf3eEXAyjgpilgb27EpshumK7UDJseOJCLlgkgFaY3PP/P9FQdJ95pLuunhb2nOXTlq15EQ7NARufxITn/75U/q01vXR72UJYIdPL1hkBQEdMTGvvwWHdg5vKSG/cDOYW276dqol6ZiaU4PPXWKoI5YI+WCWKlVwy5Jx35wMYLVLDXnzilTxBo7dMRenE51csoUcUZAR+zFreJkplDUyJeeJ/2C2CHlgtiLUwfHwKXZknYdmtauQ9OSpAGT5l1UxCBS7NARe3GajBRmvlzSPlMo6sFD09o9eTraBaEvEdARe9WTkeLOJR08fo6UDHqOgI5EyI/kdGx8u+Iz5G5lrnjdzEV/IKAjUdaF7NLjNM80ELebuUi/jgR0M3vNzE6b2bSZ0XULXVMrn57NDK7YlyUyJtIu6KlO7tBvdffhlTqBAe2qNREprvl1d2nXoWlt/uJfENjRE5QtInHCTpPGZXhGtXffn9OuQ9Oaev2i9uW3SFrYuTO4Gp3WqYDukp43M5f0R+7+ePUFZvaApAckaf36eDVhQvJVDs+YKRQX68Lj5ODxcxq9caEvTeUPHwZXo1M6MuDCzNa5+3kz+zlJ35b0O+7+nbDrGXCBXqjcBQ+sMMyil4LUUK2DUgzVQD31Blx0ZIfu7ufL/33LzL4p6RZJoQEd6IXK1MzkyZlYpGRWOvFKVQza1fZNUTO7xsx+Ovi1pNskvdTu6wKdlITDSWElmUCjOlHl8vOS/trMTkn6G0mH3f0vOvC6QEcFh5MO7ByOXSuBzIBpbMemqJeBhGs75eLuP5R0cwfWAvREkIZ55LkzujRbing1C+IzfA9JxklR9KX8SE577tysiOZTLzM373rkuTNRLwMJRx06+tb+I2djVdp4abbEYGq0hR06+tZKVSWZiLbulYOpdx2aZpAGmkJAR98KqyrJDWW1/96bNZTN9HhFy12aLenhZ04T1NGQjhwsahYHixAHtWrTs5lBPXrPlpqpjt2Tp/W14+d6ucRFAyb9zNUZvV0s0Sqgj/XkYBGQRJXtAur1VJk8OaOnT0S3S553qVBcqMihVQDCsEMHGrBt4mjs5ppKzDDtN/V26OTQgQbE9Vh+cPN0w/hhbZs4Sq69zxHQgQYk4Vh+kIohqPcvAjrQgFqTkjKDMTmVVKFYmtNDT50iqPcpcuhAg6qHUrz7j5cXb1TGkWlhUAF59vSgygXokOpJSRvHD0e4mvqCrRpVMf2DlAvQoiTk1QPF0pz2PkuvmLQjoAMtqpVXj7NCsURuPeUI6ECLkjA0oxodHdONHDrQhuoxd3HqsV7LpdmFXTq59HQioAMdEscZprXsP3JW+ZHcsqodKmGSj4AOdEEQGB966tRiS9y4OF8oLvuBQyVMOpBDB7okP5LTfMyCuSTJpH/71PSyfz0US3Paf+RsRItCJxDQgS4KK2208teg9f60qbtCJzXFtWcNGkNAB7qoVmljNjOox3YO60cTd+grvx6v+epJqq3HcgR0oIsqSxtNC8fwKwdo5EdyWrM6+slI0sIPmrEdm6JeBtrATVGgy6pbBlTbc+fmWFTEVObQuTGaTDTnAmIgKCGMyxCNNaszuuMXrtcLr1ygrDFG6jXnIqADMRN2QCnonhilNasz2nPnZgJ7RJhYBCRMfiSnk1+8TQd2Di/Jvd+/dX3kvWMuzZYYohFj5NCBmKqVex+98VrtffZMpH3Yg86N7NLjhx06kCD5kZym99wW9TJUKJa0YfywPvLv/5zdeowQ0AG0bLY0r12HprV78nTUS4EI6EAiDWXjUbseOHj8HDv1GCCgAwm0967NygzEZ0i1S/SBiQECOpBA+ZGc9t9782IVzJrVmcgD/EyhqI3jh7Vt4ii79YhQhw6kRGV/86HVGRVmS5HVrZuk+7eu1778lohWkE716tApWwRSorrMMcohG66FvProjddS3thD7NCBFKvctQ+YRTZsI0frgI5ghw70scpd+8bxw5GtY6ZQ1Ng3Ti2uCd3BTVGgT0Td67w053rkuTM1vzd5ckbbJo5yU7VNHQnoZna7mZ01s1fNbLwTrwmgs8Z2bFLUhY7VDcekK7n+mUJRrivzTQnqzWs7oJvZoKQ/kPRJSR+RdJ+ZfaTd1wXQWfmRnO7fuj7qZSyz/8hZ5pt2SCd26LdIetXdf+ju70v6M0l3d+B1AXTYvvwWHdg5rGwmmmxrrROuYXNMmW/avE78qeYkvVHx+M3yc0uY2QNmNmVmUxcuXOjA2wJoRX4kp5e//Mll7XkP7BzWgZ3DXR2JVyiWluXIw3L7Uef8k6jtskUzu1fSDnf/1+XH/0LSLe7+O2G/h7JFIN52T57WwePnunYwKTNgumrVgN59v3aNfDYzuGT2Khb0YsDFm5JuqHj8QUnnO/C6ACKyL79Fj5V38N1QmvfQYL5mdYZg3qJOBPTvSvqQmW00s6sk/YakZzvwugAilB/J6dj49q4F9TCXZkvaf+QsVS4taPtgkbtfNrPPSToiaVDSE+5eu9gUQOKM7dikBw9N97QvTFC6OPX6RQZVN4Gj/wDq2hDRKdPqwdj9nltnSDSAtvU67RKo3m5Sn74yAjqAusZ2bFI2Mxj1MiRRn74SAjqAuvIjOT16z5bIduqVBszo+RKCbosAGlLduTGq4RlBC+DgxmmwNrBDB9CCuJziJKe+FAEdQNPilFOfKRRJv5QR0AE0rTKnHvSCiRItdxeQQwfQkuoZptsmjmqmRgXKmtUZrb5qlc6X+513S5B+yY/kloze66cDSezQAXRErTRMNjOoPXdu1rHx7frRxB0atO6O2DhfKPb1wAwCOoCOqJWGqT7Ved9Hbwh/gQ5YN5Tt64EZpFwAdEx1GqbavvwW/ejCOzr2g4tdef+g70wt/XAgiR06gJ46+NlfWhyuUc+nt65vqpomP5Lr64EZBHQAPRe05n1t4o7QwJ4bympffosevWdLU7n3sFz+2I5Nba05CQjoACJVLwDnR3Kab7Ar7LaJo5JUN5efVuTQAUQqCLS1ygyD8sNGyx2DipZH79miY+Pbm1pHGkod6YcOIJaC8sPqipVG5IaydQN6ZQD/2WxG775/WaW5K/Ewjr3X6YcOIJFqlR82ql5FS3WteqFYWhLMpWSWOhLQAcTSSkG5XuVLvYqWRn9YJK3UkRw6gFhaN5St2UogV85vP/LcGV2aLS37fnVFS63ceKOBOmmljuzQAcRSveqX90rzy37PmtWZJXnvWm0AHjw0rWymfuhLYqkjO3QAsbRS9cu2iaM1Uyarr1q15CbmI8+dWXadS5otzSszaEvy5pkB0weuXqXCbCmxVS4EdACxFdZKICxlEjw/eXImNCUTuOaqVbrmp1YlukyxGikXAImz0vH+IM2yUjCXpLeLJR0b367Hdg5Lkh48NL1kUMbkyRltmziaqPml7NABJM7Yjk3LatSDnHejFSyVwT+4Psixf33qnL537u0lzydhfik7dACJs1Kr3kYqWFYK/i7p2A8uJrIFLzt0AIkUll8PK3cMDGUz2nvXZuVHcqGtdsOs9LpxwA4dQKqEDbAeymZ0YOewpvfctviDoNk6825PXGoXO3QAqbJSuWO1Wz+8Vl87fq7h155z1+TJmWWvFdbYq9cNvwjoAFKn3uSkwAuvXGj6tStvjtYqjwxuoE69flFPn5jp6Y1Vui0C6Fsbxw833Jq30lA2IzOtWBo5aKa5GvG1kU6QYei2CAAhWu3VUiiW6ta51wrmUncbfhHQAfStsBuonTAQcv+0mw2/COgA+lZQz96N6pX5Ghv0bjf8IqAD6Gv5kZy+8us3d22nHjBJv/aLjd2sbRUBHUDfq3XydCibqXntUDbTUvB3tVZV0wyqXACghlozTYM5o9KVOveh1Rm9895llWrlWGowqeWa9HpVLm3VoZvZXkmflRT82Pm8u3+rndcEgDiod0CpMhgHB4gaaQ0QDNroRk16Jw4WPebu/6kDrwMAsbLSAaVap0AlLdvVhwmafXUyoJNDB4Am1RptF+y4g1y8VL/3S6dr0juxQ/+cmf2mpClJD7n7pVoXmdkDkh6QpPXr13fgbQEgGrXa7gY77mPj25fsurdNHA1NxXS6Jr3uDt3M/tLMXqrxdbekP5R0k6RhST+W9JWw13H3x9191N1H165d26n1A0DP1RuB18i1kjpek153h+7uH2/khczsjyX9j7ZXBAAxF9ZzvdaOO+zaNaszHa9JbyuHbmbXVzz8lKSX2lsOAMRfrZYBYadAw67dc+fmjq+r3Rz6fzSzYS1U4rwm6bfaXRAAxF0zPdebubZdHCwCgISgfS4A9AkCOgCkBAEdAFKCgA4AKUFAB4CUiKTKxcwuSHq9529c23WS/j7qRXRAGj5HGj6DxOeImzR8juAz3OjuoUftIwnocWJmUyuVASVFGj5HGj6DxOeImzR8jkY/AykXAEgJAjoApAQBXXo86gV0SBo+Rxo+g8TniJs0fI6GPkPf59ABIC3YoQNAShDQASAlCOiSzOzLZvZ9M5s2s+fNbF3Ua2qWme03s1fKn+ObZjYU9ZpaYWb3mtkZM5s3s8SVmpnZ7WZ21sxeNbPxqNfTCjN7wszeMrPEzjcwsxvM7AUze7n89+l3o15TK8zsajP7GzM7Vf4cj6x4PTl0ycx+xt3/ofzrfyPpI+7+2xEvqylmdpuko+5+2cz+gyS5++9FvKymmdk/kTQv6Y8k/Tt3T0yfZTMblPR/JH1C0puSvivpPnf/20gX1iQz++eS3pH0X939n0a9nlaUh+9c7+7fM7OflnRCUj6BfxYm6Rp3f8fMMpL+WtLvuvvxWtezQ5cUBPOya7QwsCNR3P15d79cfnhc0gejXE+r3P1ldz8b9TpadIukV939h+7+vqQ/k3R3xGtqmrt/R9LFqNfRDnf/sbt/r/zr/yfpZUmdnyjRZb7gnfLDTPkrND4R0MvM7PfN7A1J90v6YtTradO/lPTnUS+iD+UkvVHx+E0lMIikjZltkDQi6cWIl9ISMxs0s2lJb0n6truHfo6+Cehm9pdm9lKNr7slyd2/4O43SDoo6XPRrra2ep+hfM0XJF3WwueIpUY+R0JZjecS96+9NDGzD0h6WtKuqn+JJ4a7z7n7sBb+1X2LmYWmwdqdKZoY7v7xBi/9b5IOS9rTxeW0pN5nMLPPSPpVSb/sMb450sSfRdK8KemGiscflHQ+orX0vXLO+WlJB939majX0y53L5jZ/5R0u6SaN6z7Zoe+EjP7UMXDuyS9EtVaWmVmt0v6PUl3ufts1OvpU9+V9CEz22hmV0n6DUnPRrymvlS+mfgnkl529/8c9XpaZWZrg4o1M8tK+rhWiE9UuUgys6clbdJCdcXrkn7b3WeiXVVzzOxVST8l6f+WnzqetEodSTKzT0n6L5LWSipImnb3HZEuqglm9iuSDkgalPSEu/9+tCtqnpk9KeljWmjZ+hNJe9z9TyJdVJPM7J9J+l+STmvh/2tJ+ry7fyu6VTXPzH5B0le18PdpQNJT7v6l0OsJ6ACQDqRcACAlCOgAkBIEdABICQI6AKQEAR0AUoKADgApQUAHgJT4/whekE1DgF3ZAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "# process the training data\n",
    "def data_iter(batch_size, features, labels):\n",
    "    num_examples = len(features)\n",
    "    indices = list(range(num_examples))\n",
    "    # The examples are read at random, in no particular order\n",
    "    random.shuffle(indices)\n",
    "    for i in range(0, num_examples, batch_size):\n",
    "        batch_indices = torch.tensor(indices[i:min(i +\n",
    "                                                   batch_size, num_examples)])\n",
    "        yield features[batch_indices], labels[batch_indices]"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Initialize model parameters and define the model\n",
    "To learn a model from data points, let's first define the model form as $y = wx + b$ and initialize our model."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "w = torch.normal(mean=torch.tensor([0.0]), std=torch.tensor([0.1]))\n",
    "b = torch.zeros(1)\n",
    "print(f'The initial weight is: {w}, the intial bias is {b}')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "The initial weight is: tensor([-0.0432]), the intial bias is tensor([0.])\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "However, this is not enough! If we want to calculate the gradient w.r.t $w$ and $b$, we have to define it explicitly by setting `requires_grad=True`. This is important, otherwise, we can't optimize these parameters by the optimizer. In practice, you should always take care of which parameter should be set `requires_grad` as `True` and when to set them. Also, in some cases, we need to set `requires_grad` as `False`, and we will discuss it in Part 2."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "w.requires_grad = True\n",
    "b.requires_grad = True\n",
    "print(f'After set requires_grad=Ture, the w is: {w}, and b is {b}')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "After set requires_grad=Ture, the w is: tensor([-0.0432], requires_grad=True), and b is tensor([0.], requires_grad=True)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "def model(x, w, b):\n",
    "    \"\"\" Define the linear regression model.\"\"\"\n",
    "    return w * x + b"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Define the loss function and define the optimization algorithm\n",
    "The optimization objective we used here is the **mean square error**: $L = \\frac{1}{N} \\sum_{i=1}^N \\frac{1}{2}(y_i - \\hat{y_i})^2$"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "source": [
    "def loss_fn(y_hat, y):\n",
    "    return (y - y_hat)**2 * 0.5"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Once we define the model and the loss function, there are many methods to define the parameters $w$ and $b$ to minimize the loss function. For example, we can calculate the optimal parameters analytically, or we perform the optimization with gradient descent. This tutorial will use the second one.\n",
    "\n",
    "In order to use the gradient descent method, we need to first calculate the gradient as:\n",
    "\n",
    "$$\\frac{dL}{dw} = \\frac{1}{N}\\sum_{i=1}^N -x_i(y_i - w x_i - b) = \\frac{1}{N}\\sum_{i=1}^N(w x_i^2 + b x_i - x_i y_i)$$\n",
    "\n",
    "$$\\frac{dL}{db} = \\frac{1}{N} \\sum_{i=1}^N -(y_i - w x_i - b) = \\frac{1}{N} \\sum_{i=1}^N (w x_i + b - y_i)$$\n",
    "\n",
    "With the gradient w.r.t $w$ and $b$, we finally perform the gradient descent step:\n",
    "\n",
    "$$ w \\leftarrow w - \\alpha \\frac{dL}{dw}$$\n",
    "\n",
    "$$ b \\leftarrow b - \\alpha \\frac{dL}{db}$$\n",
    "\n",
    "Notice that the gradient is estimated using the whole training set. This is fine when the training set is small, but when the training set is large, e.g., ImageNet, calculating over the whole training set to perform one gradient step is expensive. Thus, the simple yet efficient solution is estimating the gradient with a mini batch of training data, we call this method as stochastic gradient descent (SGD)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "def sgd(params, lr, batch_size):\n",
    "    with torch.no_grad(): # freeze params for saving resources\n",
    "        for param in params:\n",
    "            param -= lr * param.grad / batch_size\n",
    "            param.grad.zero_()  # reset the gradient as 0 !!"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Training\n",
    "Now, we can finally perform the training process. But there is still something we should take care of, that is the hyperparameters. Specifically, in our toy linear regression example, we should define the learning rate $\\alpha$ and *batch size*. Hyperparameters are important for machine learning since they hugely influence the training results. So you should always tune them carefully."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "source": [
    "# hyperparameters\n",
    "learning_rate = 0.01\n",
    "batch_size = 10\n",
    "num_epoch = 10"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "source": [
    "# training\n",
    "for epoch in range(num_epoch):\n",
    "    for x, y in data_iter(batch_size, features, labels):\n",
    "        y_hat = model(x, w, b)\n",
    "        loss = loss_fn(y_hat, y).sum()\n",
    "        # compute gradient on loss w.r.t w and b\n",
    "        loss.backward()\n",
    "        sgd([w, b], learning_rate, batch_size)\n",
    "    \n",
    "    # print\n",
    "    with torch.no_grad():\n",
    "        train_loss = loss_fn(model(features, w, b), labels)\n",
    "        print(f'epoch {epoch+1}, loss {float(train_loss.mean())}')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "epoch 1, loss 2.0713047981262207\n",
      "epoch 2, loss 0.3864479064941406\n",
      "epoch 3, loss 0.15654605627059937\n",
      "epoch 4, loss 0.12499469518661499\n",
      "epoch 5, loss 0.12078550457954407\n",
      "epoch 6, loss 0.12004215270280838\n",
      "epoch 7, loss 0.11994075030088425\n",
      "epoch 8, loss 0.11994903534650803\n",
      "epoch 9, loss 0.11994881927967072\n",
      "epoch 10, loss 0.11993353068828583\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Notice that instead of calcuating the gradient w.r.t $w$ and $b$, we directly call the function `.backward()` on loss. The function is to calculate the gradient w.r.t parameters with `requires_grad=True`. The resulting gradient for each parameter is stored in `param.grad`, see our `sgd()` function. For more detail, please check the document https://pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "source": [
    "# compare w and b with the ground truth\n",
    "print(f'The estimated w is {w.item()}, the true w is {true_w.item()};') \n",
    "print(f'The estimated b is {b.item()}, the true b is {true_b.item()}')\n",
    "\n",
    "# plot it\n",
    "plt.scatter(features, labels)\n",
    "x = range(-4, 5)\n",
    "y = [model(i,w,b).detach().numpy() for i in x]\n",
    "plt.plot(x, y, 'r')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "The estimated w is -3.4095938205718994, the true w is -3.4000000953674316;\n",
      "The estimated b is 4.212475299835205, the true b is 4.199999809265137\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fb57f0bf730>]"
      ]
     },
     "metadata": {},
     "execution_count": 12
    },
    {
     "output_type": "display_data",
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAqAUlEQVR4nO3de3zO9f/H8cd7M2yUkalMUopSskVRviXSkGgVSaSSDg6Jcgo59KUUosOQnCqHSBGRc/JLlDmfUqRkKasssWGH9++Pa/Pd5ppd17Zr17Vrz/vttlu7rutzePl+57W39+f1fr2NtRYREfFPAd4OQEREPEdJXkTEjynJi4j4MSV5ERE/piQvIuLHSng7gMwqVqxoq1Wr5u0wRESKlM2bN/9prQ1z9plPJflq1aoRGxvr7TBERIoUY8wvOX2m6RoRET+mJC8i4seU5EVE/JiSvIiIH1OSFxHxY0ryIiJ+TEleRMSP+UeSP3UKnnsOjhzxdiQiIj7FP5L8d9/Bu+9CrVowfTqoR76ICOAvSf7222HHDqhdGzp3hqgoOHjQ21GJiHidfyR5gBo1YO1amDABNm6E66+HN9+E1FRvRyYi4jX+k+QBAgKga1fYvRsaNYJeveC222DPHm9HJiLiFf6V5DNUrQpLlsCHH8K+fRAZCSNGQHKytyMTESlU/pnkAYyBjh1h716IjoaXXoJ69WDzZm9HJiJSaPw3yWeoVAnmzoUFCyA+Hm6+Gfr3h6SkXE9duDWOhqPWcMWAJTQctYaFW+MKIWARkYLj/0k+Q3S0Y26+c2d4/XWoUwfWrcvx8IVb43jx053EJSRhgbiEJF78dKcSvYgUKcUnyQOEhsJ778GqVZCS4ng4260bHD9+zqGjl+8jKTlrZU5Sciqjl+8rpGBFRPKveCX5DHfeCTt3Qu/eMGkSXHcdLF2a5ZDfEpxP5+T0voiILyqeSR6gTBl44w345hu48EJo2dLxoPbPPwGoHBrs9LSc3hcR8UXFN8lnaNAAtmyBIUMcD2hr1YK5c+kbVYPgoMAshwYHBdK3WU0vBSoi4j4leYBSpWD4cEeyv/xyeOghood3Z9xtYYSHBmOA8NBgXr2/NtGR4d6OVkTEZcb6UDOvevXq2djYWO8GkZLiaIcweLAj+Y8ZA0884ai7FxHxQcaYzdbaes4+00g+uxIl4IUXHA9mIyLgySehaVP46SdvRyYi4jYl+ZxcdRWsWeNoYbxpk6Ph2bhxangmIkWKkvz5BATAU085FlHdeSc8/zw0bAi7dnk7MhERlyjJu6JKFVi0CGbPhgMH4MYbHQ9qz5zxdmQiIudVIEneGDPNGHPUGLMr03vDjDFxxpht6V93F8S9vMYYaN/eMapv2xaGDYO6dR1TOSIiPqqgRvIzgOZO3h9nrY1I/1rq5POiJywMZs1yjOyPHXPU2ffpA4mJ3o5MROQcBZLkrbXrgL8L4lpFRqtWjs1JnnwSxo6FG25w7EyVC3W2FJHC5Ok5+R7GmB3p0znlnR1gjHnKGBNrjImNj4/3cDgFrFw5R++bNWscrxs3hqefhn/+cXq4OluKSGHzZJKfCFQHIoAjwFhnB1lrJ1tr61lr64WFhXkwHA9q3NixkXifPjBliqPh2eLF5xymzpYiUtg8luSttX9Ya1OttWnAe8DNnrqXTwgJgdGjHZuIV6gArVvDww87NipJp86WIlLYPJbkjTGXZnp5H1A8istvugliYx0llvPnOxqezZ4N1ubYwbJccJDm6UXEIwqqhHIOsAGoaYw5bIx5AnjdGLPTGLMDaAz0Loh7FQklSzq6Wm7dCtWrQ4cO0Lo1L0VccE5ny6AAw8kzKZqnFxGPUIMyT0tNhbfegkGDoEQJtj07iB5l6hJ3/DSVQ4NJPJPCscTkc04LDw1m/YAmXghYRIoaNSjzpsBAxw5Uu3bBTTcR8coAvl45koNdarJ+QBMSnCR40Dy9iBQMJfnCcuWVjr1lp0xxTOPUrg1jxnDZBSWdHq4dqESkICjJFyZjHL3p9+yBZs2gb18+m92XOn8fynKYdqASkYKiJO8NlSvDggUwdy7l439jwfSeDNk0l1IpydqBSkQKVAlvB1BsGQMPPgh33klA7950/vBDOv++GaZOBRcS/MKtcYxevo/fEpKoHBpM32Y19YtBRM6hkby3XXQRfPABLF0K//4Lt97q6Ft/8qTTwwcv3MkVLy6h19xtKrsUkVwpyfuKFi0cFThduzp2oKpdG1avznLI4IU7mbnxEM6qXtUeQUScUZL3JRdeCDEx8NVXjr1mmzaFLl0gIQGAOd/+et7T4xKStGJWRLJQkvdFt98O27dD//4wY4ajNcJnn5HqwsI1Td2ISGZK8r4qOBhGjYJvv4VKlSA6mpjPXqPiyWO5nqqpGxHJoCTvY87ZVCTgEscWgyNH0mz/RlZO6cZ9u9bgdGI+E62YFRFQkvcpOW4qsusoDBxIiR3bia98OeOWvMH0+cOofPxojtfSilkRATUo8ykNR60hzskIPHOzstteWUmTLz+h31fvY41hVKPHmBXZAmv+9/vaABYINIZUawlXHb2IX1ODsiLClU1FDh8/w/t1W9HsiRi2VL6GESsn8tHsF7ni7/89aM34tZ3xoDYuIYnec7dRTf3qRYodJXkfktMUS+b3M74/XO5iOj34Mn3u7sU18T+zbFoPntk4n8C0VKfXyEj8qr4RKV6U5H1I32Y1z9lUJHuzsizHGMP82k1p2mUSa6rfxICvZrDwg+ep9cdP572Pqm9Eig8leR8SHRnOq/fXJjw0GANOm5VlHBMaHHT2vfiy5el630CeiX6RS078xaL3e9Fn3QeUSjmT472czf2LiP9RgzIfEx0Z7tID0jKlSpCQlHz24SrAspoN2VD1Bl5aM4UeG+bRfN839GvxHFuqXHvO+YHGFHjsIuJ7NJIvYjKXWYLj4WrmdP1P8AX0admbTm2HUzrlDPNn9WPoqncJOZN15O7K6lkRKfqU5IuY0cv3kZSc9eGqBbKPy9ddWZeoJ2J4v+49PLr5c1ZM7c5tB7dkOWbwwp1O73HOgiw9pBUpspTki5icyiwtjjn8zBJLBjO86dO07fAap0uU5MN5Qxi9ZDzlkv4FYObGQ1wxYEmWZJ/jgiwlepEiSUm+iMmpzDJjwZSzmfbNVWpx9+Nv8c4tD3Lf7jWsmtqV5vvWA45fDjM3Hjqb6Ict2n3OvxSSklMZtmh3Qf4xRKSQKMkXMbmVWeb0S+B0iZKMub0TrR8dzx9lL2LSwleZuOAVwk44Gp7N3HiIa1/6goSkZKfnJyQlazQvUgQpyRcxuZVZ5vRLIGOEv+fiK7m30xu81uhRmhzYxMqpXWmzcxVYS1Jy2nnvrdp6kaJHvWv8kLP9Xz+OPcT6A39nOe7Kvw4zatlb3Hx4D+uqRTKweQ8Ol7s4x+saYFy7CEYv30dcQpJ644j4iPP1rlGSLyZyan5mbBodtn7BgK9mYKzl9UaP8sGNLbM0PHNV+ZAghra6TslepJCpQZnkXJVjAph5Y0uadY5hU5XrGL7qXT6e1Z/qf55/q0FnjiUmqxJHxMcUSJI3xkwzxhw1xuzK9F4FY8xKY8yP6f8tXxD3krzJrb98XLlKPNZ2GL1bPk/1vw6zdMazdNswjxKpKW7dJyk5lRfmbVeNvYiPKKiR/Aygebb3BgCrrbVXA6vTX4uXOHsgew5jWHB9E+7qMoGVVzWg37oP+OyD57nu9/1u3SvVWtXYi/iIAkny1tp1wN/Z3r4XeD/9+/eB6IK4l+RNRlVO+ZCgXI/9s0x5ekQP4On7BhJ28hifffA8/b6aQank027fVx0vRbzLk3PyF1trjwCk/7eSs4OMMU8ZY2KNMbHx8fEeDEeiI8PZOiTK5eOX17iVpl0mMr92U7ptnM/SGT256ddduZ+YjTpeiniP1x+8WmsnW2vrWWvrhYWFeTucYiF7+4PzOV66LANa9KRDuxGUTE3h49kDeHnFRMqcTnT5Gmp4KeI9nkzyfxhjLgVI/2/Ou05LoXJpfj6b9dUiiOocw9R699Jx61JWTO3OHQdcK3f1oSpdkWLHk0l+EfBo+vePAp958F7ihsyrZt2RVLI0/73zSdp0fJ3EkqWZMX8YYz8fS2jS8VzP1f6yIt5RIIuhjDFzgDuAisAfwFBgITAPqAocAtpaa7M/nM1Ci6EKX06LpHJTMiWZ7hvm0m3jxySUvoChdz3D0poNc52bCQ4K5NX7awOcsypXi6hE8kYrXiVHGa2Fs3eedNW1R3/itS/e4obf97P86gYMjupGfNkK5z0nNDiI0ylpWe6ZkfyV6EXcpxWvkqOMqZu8bge4t9KV3PfIWF6543EaHdzC6ildabtjxXkn4hOSkp22Mx6+WO2MRQqakrwQHRnO2AfrOH0YGxIUgOH8szCpAYFMrv8ALR5/m72VrmD0F28xc+5gLkv43a04jiUm57hblYjkjaZr5Cxn3Sszpk8Wbo2j78fbSU47/8+LsWk8vG0ZA9ZOJ9CmMea2Tsyoew9pAa5X84xvF6FpGxE3aE5eCsTCrXEMW7Q7x41FMrv0eDwjl8fQ5KdYtlSuSb8Wz7G/YlWX7hMaHMSw1tfpwayIi5TkpUAt3BrH8MW7OZaYS7K3lnv3rGXo6vcocyaRd25px6QGbUgOzL21QnZ6MCuSMyV58aiI4SvOO7q/6GQCQ1dPpvXedewNq0a/Fs+x89Kr3b5PoDGkWauRvUg2qq4Rj8pt+uavMqH0bN2PLve/RPmk4yz88AUGfDnN7YZn6m4p4j4leSk0q66uT9QTE5h7w108892nLJveg/qH8lZNo+6WIq5Rkpd8c6V9cYbjpcsysPmztH9oJAHWMnfOi4xYHkNZNxqeZYhLSNJoXiQXmpOXfFu4NY6+87eTnOrez1LwmVM8//VMOscu4o+yFRjUrDtfVr8pTzFkVORonl6KI83Ji0dFR4Yzuk2dsw3PXF07m1SyNCObdOGBjqM5UTKE6fOHM27xGMon/uN2DAlJyfT9eLtG9iLZaCQvBc6devoMJVOS6bZxHt02fMy/pUIY1vRpFl97u9vN6MNDg1k/oIm7IYsUaSqhFK9YuDWOXnO3uXVOzfifee2LN4k48iMrr6rP4Kiu/HFBRbeuMb5dhBZSSbGiJC9ek5dWxgFpqXSO/YwX/m8WyQGBvNK4Mx/VaZbnLaa0kEr8nebkxWvysgtVWkAgU26+n+ad32b3JdUZtfwdZn80iKrHjuQpBpVbSnGmJC8elXkXKoN75Za/lK/Mww+N5MVmPbj+9/0sn9aDJ75bQECa+73vM8otG45awxXapUqKEU3XSKGLfHlF7n1vsrnk+J+MWBFD0wOb2HZpDfq16MkPYdXyFYemccRfaLpGfMrQVte5PYXz+4UV6fLAEHq26stlCb/z+YxePPf1bIJS3ftlkZmmcaQ40EhevCJz7/rQkCC3RvYVEv9hyOrJRO/5iu8rXk7/Fj3ZXrlmnmMxoCocKdJUXSM+b/DCnczceMitc5rs/46Ry2OodPIY0+q1ZuxtHTkVVDrPMWj6RooqTdeIzxsRXZuODVzbVCTDmqtuJqrLBD6qE8WTmxayfFoPbvllR55j0D6z4o+U5MVnjIiuzc+jWtKwegWXz/m3VBkGNevBQ+1fwWKY89FAXln2NhecPpmnGI4lJqvqRvyKpmvEp7mzmKp08il6fz2bLpsWEl8mlEHNurP6qvpu37N8SBBbh0S5fZ6It2hOXoqsKwYswd2f0BuO/MBrX7zFtfE/s+ja2xnW9Gn+Dinn1jUaVq/A7t/+Pdt/p3xIEENbqcul+CbNyUuRVTm9s6U7dlxag9aPjmPsfzrQfN83rJrSlXt3fwluDGjWH/g7S4O1Y4nJ9Jq7jcEL87bJiYi3KMmLT8tLWwSA5MAg3m7YnpaPvckvoZfy5udjmfrJy1x6PD5f8czaeEhz9lKkKMmLT8toi5BXP4ZdzgMdX+flJk9yy6EdrJjajQ5bl2JsWp6uZ0ELqKRI8XiSN8b8bIzZaYzZZozRhLu4LToy/OyGJHmRFhDItJvupVnnGLZfWoORKyYwZ85Aqv2dtxG5th2UoqSwRvKNrbUROT0YEMlNXqdtMvs19BI6thtBv+Y9qXX0IMumP8tT335CYB4anml+XoqKEt4OQMQVGVUtmTcDSTyT4najM4xhXp0o1l5ZlxErJzJw7XTu+f7/6N+iJ3srXenWpWZuPES9yyucE5faI4gv8XgJpTHmIHAMx3Tmu9baydk+fwp4CqBq1ap1f/nlF4/GI/5j4dY4Xvx0J0nJ7o/EAbCWlt9/zfBVkyh36gQT67fhnVsf4kwJ19shlyoRQIAxWWJQewQpbF6tkzfGVLbW/maMqQSsBJ611q5zdqzq5MVdGY3O3N19KrPQpOO8tPo9Htj9JT9edBn9W/RkS/i1+YpLe81KYfKZxVDGmGHACWvtGGefK8lLfmTubFkuOIjjp5JJc+PH+44DsYxcHsOl//7JjLqtGH17J5JK5q3hmQEOjmqZp3NF3OW1JG+MKQMEWGv/Tf9+JfCytXaZs+OV5KUgLdwaxwvztpPqxs942dOJ9PvqfTptXcKv5S5mQPNnWV8twu17ayQvhcmbK14vBr42xmwHvgOW5JTgRQpadGQ4Yx+s49Y5J0qFMCSqKw8+PIrkgEBmzR3Ma0vf5MJTJ9y6TuNrwtw6XsRTPJrkrbU/WWvrpH9dZ60d6cn7iWQXHRnuVlfLDN9ddj0tHn+bifXb8MCu1ayc2o2oHza4fP7sb7UyVnyDGpSJ33Onk6Uz1/++n9e/eJNaRw/yec3/MOyup/mzTHm3rqEGZ+JJalAmxdpv+UjwALsuuYrWncYx+rZHuGv/RlZN6cr9u1a71fDsWGIyfedv1+heCp2SvPi9nDpZGjeukRJYgphb23H3Y29zoEIV3lgyjhkfD6Py8aMuXyM51dJr7jYajlqjZC+FRtM14vecLZrKvmDJnT1mA9JSeWTrUvp99T7WGF5r9CgzI+/GGvfHTOFaISsFQNM1UqxldLIMDw3G4Eis2Vekfvm96y2I0wICeb9uK5o9EcOWytfw35WTmDt7AFf+ddjt2OISkug1dxsRw1dodC8eoZG8CHnbgQoAa2mzazUvrX6P0ilnGP+fh5l88/2kBrjfTE3tECSvfGbFa26U5MVb8luBE3biGC+vnEiLH75h58XV6d/iOfZc7F7DM4DgoAAqlCmlZmfiFk3XiOQiv62M48uWp+t9A3km+kUuOfEXi97vRZ91H1Aq5Yxb10lKTiMuIQmLYyrnxU93ahpH8kVJXoSs8/b5saxmQ5o+MZGF1zWmx4Z5LJ3ek7qH9+T5eknJqdqJSvJFSV4kXXRkOOsHNGF8uwiCAt0psMzqn+AL6NOyN53aDqdUymk+ntWfoaveJeRM3qaD8jONJKIkL5JNdGQ4o9vUISQof3891l1Zl2adY3i/7j08uvlzVkztzm0Ht7h9HZP33zciSvIizkRHhrPnvy0Y3y6CMiXzPld/slQIw5s+TdsOr3G6REk+nDeE0UvGUy7pX5evYS2al5c8U3WNiIvyW4FTKuUMz37zEc9snM+xkAt56a6uLKvZ0O3rqA+OZKfqGpEC0LdZTbdaIWR3ukRJxtzeidaPjuePshcxaeGrTFjwCmEnjrl1HfXBEXcoyYu4KDoyPG8LprLZc/GVRD8yllGNHuPOA5tYObUrbXaucqvhWXKqVdWNuERJXsQN+S2xzJASWIJJDdrQ4vG3+aFiVcYsHc8H84ZQ5Z8/XL5GfrtrSvGgJC/iBmeLpoKDAhnfLoLyIUFuX++ni6rQ7uFRDL6rKzf+9j3Lp3bn0c2LMTYt13MDjGHh1jgWbo2j4ag1XDFgiTpcyjn04FXETZk3DM/cesBZt0t3hP9zlJHLY7jj4GZiw6+lf/OeHKh4mdvXMUCHBlUZEV07T3FI0aPeNSKFJOMXQJ6rcKzlvt1fMmT1e4QkJ/FmQ0fDs5TAEm5dxgDj2kWoAqeYUJIX8ZK8Jv2KJ48xbNVk7vn+/9hd6Ur6tejJ7kuucusaocFBbBsa5dY5UjQpyYt4WV6ncqJ+2MCIFROokPgPk+vfz5u3tud0UCmXzw8KgNFtNaL3d6qTF/GyvDZAW1HjFpp2mcj82k3ptnE+S2f0pN7h3S6fn5wGveZuY/DCne6GLH5CSV6kkGQ0QAt0sxnN8dJlGdCiJx3ajaBkagrzZ/Vn+MqJlDmd6PI1Zm08pKqbYkpJXqSQta/vfsUMwPpqEUR1jmFqvXt5ZMtSVkztzh0HXJvetKBNxIspJXmRQjYiujYdG1R1e0QPkFSyNP+980ke6DiakyWDmTF/GGM/H0to0nGXzo9LSFJLhGJGD15FvCyvFTglU5LpvmEu3TZ+TELpCxh61zMsrdnQ5d7EanTmP1RdI1JE5KUK55qjB3n9ize54ff9LL+6AYOjuhFftoJL5wYGGMa2raNEX8R5tbrGGNPcGLPPGLPfGDPA0/cTKcryUoXzfaUruO+Rsbxyx+M0OriF1VO60nbHCpcanqWmWYYvzlqtozYJ/sWjSd4YEwjEAC2AWkB7Y0wtT95TpKjLqMJxJ9GnBgQyuf4DNH/8bfZWuoLRX7zFh3NfokrC77meeywx+ez3Gf+S0Gbi/sPTI/mbgf3W2p+stWeAj4B7PXxPEb/Qt1lNt/ea/blCOA+1f4VBUd2IOLKPFdO683jsZwSkuTb9M3r5vnOmirSZeNHm6SQfDvya6fXh9PdEJBcZe826y5oAZkXeTdQTE9h4WW2Grn6P+bP6cdWfh3I8J/LlFSzcGpdj+2K1NS66PJ3knQ1DskwUGmOeMsbEGmNi4+PjPRyOSNESHRnO+HYRBORhS6ojF4bRuc1QerbqQ7VjR1gyoyfPrp9DUGryOcceS0ym19xtOW6KUrmA+uhL4fN0kj8MZF75UQX4LfMB1trJ1tp61tp6YWFhHg5HpOiJjgznjQcjCA8NxuDYuKRjg6qubTBuDItq3UHTLhNZXuNWXvh6Fove703tIz+6fP8A45g6kqLJoyWUxpgSwA/AnUAcsAl42FrrtPmGSihF3ONujX3TH79lxIoYwk4m8N5N0Yz7TweXGp6FBgfxT1Jylv754ju8VkJprU0BegDLgb3AvJwSvIi4L6MSx9VdqVZdXZ+7ukxk7g138cx3n7Jseg/qH8q9eVlCUrKqbYooj9fJW2uXWmtrWGurW2tHevp+IsVRQuK58+w5+bdUGQY2f5b2D40kwFrmznmREctjKOtiwzNV2xQt6l0j4gfy8mB0w+V1aNb5Hd67KZr225ezYmo3Gh/Y5NK5qrYpOpTkRfyAsw3GXXEqqDQjm3ThgY6j+bdUCNPnD2fc4jGUT/znvOdZ0GrYIkJJXsQP5HVTkgzbKtfknsfeZHzD9rT8/mtWTelKqz1fnbc1QlxCEn0/VkdLX6cGZSJ+puGoNXnfSByoGf8zr33xJhFHfmTlVfUZHNWVPy6oeN5zwlV141XqQilSjOR1P9nMAtJSeTx2EX3+bybJAYG80rgzH9Vpdt42xsFBgTxQN5wvv4/nt4QklVsWIiV5kWImo37+t4QkygUHkZDkevVNZlWPHWHUsre59dAOvql6AwOaP8uh8pfmeLwh65L24KBAXr2/thK9hynJixRzgxfuZNbGQ1kScPaEnCNreWj7cgZ+OY2gtFTG3NaR6fVakxbg2oPe8NBg1g9okoeoxVVe7ScvIt43Iro249plbY0wrl0EHRtUzf1kY/goojl3dZnA19Xq8NKXU/l0Zl9qxP/s0r1VbuldGsmLFHMd3tvA+gN/u3awtbTau45hq97lgtOJxNzyIBNuaUtyYM4rbjWS9zyN5EUkR7OevIXx7SIIDXahNYIxLK7ViKZdJrL0mob0Xj+bxTN6Uec3xwrY7P3vg4MC1dzMy5TkRYToyHC2DY3Ksc4+OChrqjgWUo5erfrS+YEhlDt1gk9n9mXgmqmUPnOK8iFBZ6eE9NDV+zRdIyJnOSu/DAowYCA51XmuuOD0SQasnU6Hbcv4JfQSxrTpw9vvveD02pkrfoxx9NxRqWX+abpGRFySeeVsxmi8bOkSOSZ4cDQ8G9SsBw+1f4U0Y3h7Sh94+mn453+tEbLvHZuQlMyxRHW2LAwayYvIeV0xYIlrpZZA6eRTvBQ7jw5fz4dLLoFJk6BVK5dW4eoBbd5pJC8ieeZOh8vU0sGUGf8Gaz9YzP60UtC6NYtrNSLpt99zPVellp6hJC8i5+VOh8syJUsA8OQeaNHxDcb+pwPN9n3Dqildab1n7XkbnmkfWc/QdI2I5CrzQ9PKocE5Tr0YIKRkICfP/O/B7dXxv/D6F28ReWQfq6vfxOCobhy5MOyc8yxqdJZXamsgIgUqpzn20Bz65ASkpfLY5s/p838fkGoCeLVxZ+bUaYY1504mqN+N+zQnLyIFytkUTnBQYI5NKtMCApl207006xzD9ktr8MryGObMGUi1v8+tqElKTmX4Ym0FXVCU5EXEbc5KLV+9v3aue80eDr2Eju1G0LdFT2odPciy6c/y1LefEJiWtS3yscRklVQWEE3XiEiBOV+pZPapnEr//sWIlROJ+nEj2y+5mv4tevJ9pSuyHL9taJTTa2V/RlDc5/E1Jy8ihcLZilkDdGhQlRHRtYkYviLrnL213L1vPcNXTiL01L9MaNCWmFvacaaEo49OxwZV+fL7eOISkgg0hlRrKR8SxIlTKSSn/S93Ffd5fM3Ji0ihcDaNM65dBCOiawNONpYyhqXX/Ie7ukxg0bW389w3H7FkRk9ujNsLwMyNh87+yyA1fUB6LDE5S4IHxzz+6OX7PPpnK6o0kheRQpPb6tk7DsQycnkMl/77JzPqtmL07Z1IKlnapWsb4OColgUSZ1GjkbyI+ITcFjytrV6PqCdi+PDGu+m8eRErpnWn4c/bCuTaxZWSvIgUmr7NapLzVuAOJ0uFMPSurrR9eBTJAYHMmjuY15a+yYWnTuR4TlCgUd/6HCjJi0ihiY4Mp0ODqrkmeoBNl11Pi8ffZkKDNjywazUrp3Yj6ocNzg/2nVlnn6MkLyKFKvt+s4E5rKAywOmgUrze6DHu7fQGf4aEMnnBSN5ZOIqKJ49lOTY5zfLCvO2qrXfCYw9ejTHDgCeB+PS3Blprl57vHD14FSl+nJVdBgcFcmPVcmz86djZqpoSqSk89d2nPLd+NklBpXn5zif59LomWUp2ggIMo9vWKXallN588DrOWhuR/nXeBC8ixZOzsssH6oaz5dA/ZxM8QEpgCSbc8iB3P/42+y+6jDeWjGPGx8OofPzo2WOS0yzDFqklQmaeHsmfsNaOcfUcjeRFBHJeOZvRrdLYNDptWUK/r97HGsNrjR5lZuTdZxue/VzMSim9OZLvYYzZYYyZZowp7+wAY8xTxphYY0xsfHy8s0NEpJjJaQORjHbE1gTwft1WNHsihi2Vr+G/Kycxd/YArvzrcOEGWgTkK8kbY1YZY3Y5+boXmAhUByKAI8BYZ9ew1k621taz1tYLCwtzdoiIFDM51bxnbBH486iWlA8J4nC5i+n04Mu8cHdvavx5iC+mP8vzmz+F5PM3SitO8pXkrbVNrbXXO/n6zFr7h7U21VqbBrwH3FwwIYuIv8uplXHmWvihra4jKNCAMXxS+07uemIiX151Mz1XTYP69WHr1sIO2yd5bLrGGHNpppf3Abs8dS8R8S85tTLOXDUTHRnO6DZ1zh5TskplTs2ZC/Pnw2+/wU03waBBcOqU1/4cvsCTD14/xDFVY4GfgaettUfOd44evIpIgfj7b+jTB6ZPh5o1YepUaNjQ21F5jFcevFprH7HW1rbW3mCtbZ1bghcRKTAVKsC0abB8uWMkf9tt0LMnnMi5NYK/0opXEfFfUVGwaxc8+yy88w5cd50j8RcjSvIi4t/KloU334Svv4aQEGjeHB57zDGlUwwoyYtI8XDrrY6Km0GDYOZMqFULPvnE21F5nJK8iBQfpUvDiBEQGwvh4dCmDTzwABzx30eGSvIiUvxERMC338KoUbBkiWNUP2MG+NBOeQVFSV5EiqcSJaB/f9ixA2rXhscfh2bN4OefvR1ZgVKSF5HirUYNWLsWYmJgwwa4/np46y1ITc311KJASV5EJCAAunWD3bsdNfXPPQe33w5793o7snxTkhcRyVC1KixdCh9+CN9/75i7HzmySDc8U5IXEcnMGOjY0TGKj46GwYMdfXA2b/Z2ZHmiJC8i4kylSjB3LixYAEePOjpbDhgASc573fsqJXkRkfOJjoY9exyrZF97DerUgXXrvB2Vy5TkRURyExoKU6bAqlWQkgKNGkH37nD8uLcjy5WSvIiIq+68E3buhF69YOJER7nlF194O6rzUpIXEXFHmTIwbhx88w1ccAHcfTd06gR//eXtyJxSkhcRyYsGDWDLFhgyBObMgWuvhXnzfK41gpK8iEhelSoFw4c7yisvvxzatYP77nNsP+gjlORFRPLrhhscLRFGj3ZsSlKrlmPLQR8Y1SvJi4gUhBIlHPvK7tzpWCnbpQs0bQo//eTVsJTkRUQK0lVXwZo1MGkSbNrk6HA5frzXGp4pyYuIFLSAAHj6acciqsaNoXdvaNjQ0QCtsEMp9DuKiBQXVarA4sUwezYcOACRkfDyy3DmTKGFoCQvIuJJxkD79o5RfZs2MHQo1KvnmMopBEryIiKFISzMMaJftAj+/ttRZ9+3LyQmevS2SvIiIoWpVSvH3PyTT8KYMY6GZ2vXeux2SvIiIoWtXDlH9c2aNY5a+saN4YUXPHIrJXkREW9p3NixkXifPlC9ukduka8kb4xpa4zZbYxJM8bUy/bZi8aY/caYfcaYZvkLU0TET4WEOFbKduvmkcuXyOf5u4D7gXczv2mMqQU8BFwHVAZWGWNqWGv9Y/tzEZEiIl8jeWvtXmvtPicf3Qt8ZK09ba09COwHbs7PvURExH2empMPB37N9Ppw+nvnMMY8ZYyJNcbExsfHeygcEZHiKdfpGmPMKuASJx8NstZ+ltNpTt5z2o7NWjsZmAxQr14977dsExHxI7kmeWtt0zxc9zBwWabXVQDfabAsIlJMeGq6ZhHwkDGmlDHmCuBq4DsP3UtERHKQ3xLK+4wxh4FbgCXGmOUA1trdwDxgD7AM6K7KGhGRwpevEkpr7QJgQQ6fjQRG5uf6IiKSP8b6wPZUGYwx8cAv+bhEReDPAgqnICku9ygu9ygu9/hjXJdba8OcfeBTST6/jDGx1tp6uR9ZuBSXexSXexSXe4pbXOpdIyLix5TkRUT8mL8l+cneDiAHiss9iss9iss9xSouv5qTFxGRrPxtJC8iIpkoyYuI+DG/TPLGmD7GGGuMqejtWACMMf81xuwwxmwzxqwwxlT2dkwAxpjRxpjv02NbYIwJ9XZMcP7NaLwUT/P0zW/2G2MGeDueDMaYacaYo8aYXd6OJYMx5jJjzJfGmL3p/x8+5+2YAIwxpY0x3xljtqfHNdzbMWVmjAk0xmw1xnxe0Nf2uyRvjLkMuAs45O1YMhltrb3BWhsBfA4M8XI8GVYC11trbwB+AF70cjwZMjajWeftQIwxgUAM0AKoBbRP3xTHF8wAmns7iGxSgBestdcCDYDuPvK/12mgibW2DhABNDfGNPBuSFk8B+z1xIX9LskD44B+5NDa2BustcczvSyDj8RmrV1hrU1Jf7kRR7dQrzvPZjTecDOw31r7k7X2DPARjk1xvM5auw7429txZGatPWKt3ZL+/b84EpfTvSQKk3U4kf4yKP3LJ/4eGmOqAC2BKZ64vl8leWNMayDOWrvd27FkZ4wZaYz5FeiA74zkM+sMfOHtIHyQyxvgSFbGmGpAJPCtl0MBzk6JbAOOAiuttT4RFzAex8A0zRMXz+8er4XufJuYAAOBqMKNyCG3zVWstYOAQcaYF4EewFBfiCv9mEE4/pk9qzBicjUuH+HyBjjyP8aYssAnQK9s/5L1mvROuBHpz54WGGOut9Z69XmGMeYe4Ki1drMx5g5P3KPIJfmcNjExxtQGrgC2G2PAMfWwxRhzs7X2d2/F5cRsYAmFlORzi8sY8yhwD3CnLcRFE3ncjMYbtAGOm4wxQTgS/Cxr7afejic7a22CMWYtjucZ3n5o3RBobYy5GygNXGiMmWmt7VhQN/Cb6Rpr7U5rbSVrbTVrbTUcfzlvLIwEnxtjzNWZXrYGvvdWLJkZY5oD/YHW1tpEb8fjozYBVxtjrjDGlAQewrEpjjhhHCOsqcBea+0b3o4ngzEmLKN6zBgTDDTFB/4eWmtftNZWSc9ZDwFrCjLBgx8leR83yhizyxizA8d0kk+UlQHvABcAK9PLOyd5OyDIeTMab0h/MN0DWI7jIeK89E1xvM4YMwfYANQ0xhw2xjzh7ZhwjEwfAZqk/0xtSx+letulwJfpfwc34ZiTL/ByRV+ktgYiIn5MI3kRET+mJC8i4seU5EVE/JiSvIiIH1OSFxHxY0ryIiJ+TEleRMSP/T/uTQBKslycnAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Change different optimizers\n",
    "In the above example, we define the `sgd()` optimizer by ourselves. However, Pytorch already offers a lot of different optimizers, see https://pytorch.org/docs/stable/optim.html#how-to-use-an-optimizer. Here, we will show how to use the `Adam` optimizer offered by Pytorch. We will show it by keeping the model the same, but change the training loop."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "source": [
    "# let's first define the adam optimizer\n",
    "optimizer = torch.optim.Adam([w, b], lr=learning_rate)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Again, let's first initialize the parameters. Since we already define the $w$ and $b$, we can easily modify their data by changing `w.data` and `b.data`. Notice that "
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "source": [
    "# Because we want to show \n",
    "w.data = torch.normal(mean=torch.tensor([0.]), std=torch.tensor([0.1]))\n",
    "b.data = torch.zeros(1)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "source": [
    "# training loop\n",
    "for epoch in range(num_epoch):\n",
    "    for x, y in data_iter(batch_size, features, labels):\n",
    "        optimizer.zero_grad() # ! Remember to call zero_grad()\n",
    "        y_hat = model(x, w, b)\n",
    "        loss = loss_fn(y_hat, y).sum()\n",
    "        \n",
    "        # calculat gradient on loss w.r.t w and b\n",
    "        loss.backward()\n",
    "        # preform the gradient descent step to update parameters\n",
    "        optimizer.step()\n",
    "    \n",
    "    # print\n",
    "    with torch.no_grad():\n",
    "        train_loss = loss_fn(model(features, w, b), labels)\n",
    "        print(f'epoch {epoch+1}, loss {float(train_loss.mean())}')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "epoch 1, loss 9.069252967834473\n",
      "epoch 2, loss 5.062711715698242\n",
      "epoch 3, loss 2.670809507369995\n",
      "epoch 4, loss 1.3396475315093994\n",
      "epoch 5, loss 0.6380116939544678\n",
      "epoch 6, loss 0.32141920924186707\n",
      "epoch 7, loss 0.1893029361963272\n",
      "epoch 8, loss 0.14160916209220886\n",
      "epoch 9, loss 0.12600655853748322\n",
      "epoch 10, loss 0.12143859267234802\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Here, we want to stress the `optimizer.zero_grad()` function. The `backward()` function, by default, accumulates gradients in the `.grad` buffer. This means that every call to `backward` **adds the gradient** to what is currently stored in the buffer, instead of overwriting it. This is useful when dealing with large models and dataset/batch sizes, where the whole data doesn't fit into memory and the gradients have to be calculated for a larger number of samples and averaged. In our case, we want to calcualte the current gradient value at each iteration, so we have to manually zero out the gradients by calling `optimizer.zero_grad()`.\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "source": [
    "# compare w and b with the ground truth\n",
    "print(f'The estimated w is {w.item()}, the true w is {true_w.item()};') \n",
    "print(f'The esimated b is {b.item()}, the true b is {true_b.item()}')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "The estimated w is -3.384695053100586, the true w is -3.4000000953674316;\n",
      "The esimated b is 4.161173343658447, the true b is 4.199999809265137\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 2. Basic Neural Network\n",
    "In our assignments, we will use very basic neural networks, especially multilayer perceptrons (MLPs). \"For certain choices of the activation function, it is widely known that MLPs are **universal approximators**. Even with a single-hidden-layer network, given enough nodes (possibly absurdly many), and the right set of weights, **we can model any function**, though actually learning that function is the hard part.\" MLPs are vastly adopted in reinforcement learning to represent the policy, the value function or the dynamic model. In this tutorial, we will quickly go through how to define MLPs and how to train it. For more detail, please check http://www.d2l.ai/chapter_multilayer-perceptrons/mlp.html"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Define the network"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "source": [
    "class Net(nn.Module):\n",
    "    def __init__(self, input_dim, output_dim, hidden_dim):\n",
    "        super(Net, self).__init__()\n",
    "        self.layer1 = nn.Linear(input_dim, hidden_dim)\n",
    "        self.layer2 = nn.Linear(hidden_dim, hidden_dim)\n",
    "        self.output_layer = nn.Linear(hidden_dim, output_dim)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.layer1(x)\n",
    "        x = F.relu(x)\n",
    "        x = self.layer2(x)\n",
    "        x = F.relu(x)\n",
    "        x = self.output_layer(x)\n",
    "        return x\n",
    "\n",
    "net = Net(1, 1, 10)\n",
    "print(net)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Net(\n",
      "  (layer1): Linear(in_features=1, out_features=10, bias=True)\n",
      "  (layer2): Linear(in_features=10, out_features=10, bias=True)\n",
      "  (output_layer): Linear(in_features=10, out_features=1, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "The above `Net` class defines a simple two-layer neural network. In `__init__` function, you can initialize the network, and the operations on inputs are implemented in the `forward` function. As a subclass of `nn.Module`, the backpropagation is automatically built after implementing the `forward` function. \n",
    "\n",
    "`nn.Linear()` is used to define the linear layer $y = w x + b$, and `F.relu()` is the activition function, defined as $relu(x) = max(x, 0)$."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Train the network\n",
    "To be concise, we will use the defined MLPs to approximate a linear model similar to our linear regression example. Please remember that the MLPs is a powerful function approximator, so they can represent much more complex models. We will use the neural network to solve a more complex task later.\n",
    "\n",
    "In this toy example, we will use the same loss function, the same dataset as above, the only difference is instead of using the linear model, we use two-layer MLPs here."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "The parameters of the MLPs are automatically initialized when defining them. The default initialized values depend on the type of layers. Specifically, for linear layers, the initialized value is given by $U(-\\sqrt{k}, \\sqrt{k})$, where $k = \\frac{1}{in\\_features}$ (see https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/linear.py#L435-L68). \n",
    "\n",
    "The initialized values are:"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "source": [
    "for w in net.parameters():\n",
    "    print(w)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Parameter containing:\n",
      "tensor([[-0.1063],\n",
      "        [-0.4625],\n",
      "        [-0.5468],\n",
      "        [ 0.1303],\n",
      "        [ 0.8797],\n",
      "        [-0.5968],\n",
      "        [ 0.1255],\n",
      "        [-0.7445],\n",
      "        [-0.9135],\n",
      "        [-0.0865]], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([ 0.6870,  0.6928, -0.6203, -0.5075,  0.0336,  0.0019,  0.5913, -0.6671,\n",
      "         0.1970,  0.1624], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([[-0.0963,  0.3123,  0.1501, -0.1534,  0.3066, -0.2851,  0.1462,  0.1528,\n",
      "         -0.0935,  0.1652],\n",
      "        [-0.3105, -0.2966,  0.1985,  0.0468, -0.1024, -0.1075, -0.1202,  0.2795,\n",
      "          0.2169, -0.2171],\n",
      "        [-0.0121, -0.1958, -0.1180,  0.2977, -0.0958,  0.3156, -0.0798,  0.0919,\n",
      "         -0.0384, -0.2942],\n",
      "        [ 0.0118, -0.0936, -0.0148, -0.1755,  0.1892,  0.2182, -0.2960,  0.2528,\n",
      "          0.1387, -0.0884],\n",
      "        [-0.2091,  0.1648,  0.2598, -0.0867,  0.0978,  0.0121,  0.0657, -0.0814,\n",
      "         -0.2462,  0.0156],\n",
      "        [ 0.1214, -0.1721,  0.2962, -0.2099, -0.0256, -0.0120, -0.0604, -0.1480,\n",
      "          0.0247,  0.0110],\n",
      "        [ 0.0870,  0.1939, -0.0793, -0.0017, -0.2207, -0.0785, -0.0534,  0.1873,\n",
      "          0.1641, -0.1795],\n",
      "        [ 0.1016, -0.2960, -0.1523,  0.1536,  0.2137,  0.2172, -0.2464,  0.1437,\n",
      "         -0.3109,  0.2700],\n",
      "        [ 0.2133, -0.2963,  0.1176, -0.2578,  0.2897,  0.0235, -0.3033,  0.1579,\n",
      "         -0.1550, -0.1988],\n",
      "        [ 0.2454, -0.0261, -0.0632,  0.1918, -0.0111,  0.0924, -0.2880, -0.0513,\n",
      "         -0.0835,  0.2051]], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([-0.0711,  0.1636, -0.0502, -0.0231,  0.1933,  0.1284,  0.0158, -0.2339,\n",
      "         0.2303, -0.1696], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([[-0.0017, -0.1119,  0.1393,  0.0677, -0.2677,  0.1061, -0.0124,  0.0365,\n",
      "          0.2570,  0.1801]], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([-0.1774], requires_grad=True)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now, let's train our network similar to the above example."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "source": [
    "# Define the optimizer, \n",
    "num_epoch=15\n",
    "optimizer = torch.optim.Adam(net.parameters(), lr=0.01)\n",
    "# training loop\n",
    "for epoch in range(num_epoch):\n",
    "    for x, y in data_iter(batch_size, features, labels):\n",
    "        optimizer.zero_grad() # ! Remember to call zero_grad()\n",
    "        y_hat = net(x)\n",
    "        loss = loss_fn(y_hat, y).sum()\n",
    "        \n",
    "        # calculat gradient on loss w.r.t w and b\n",
    "        loss.backward()\n",
    "        # preform the gradient descent step to update parameters\n",
    "        optimizer.step()\n",
    "    \n",
    "    # print\n",
    "    with torch.no_grad():\n",
    "        net.eval()\n",
    "        train_loss = loss_fn(net(features), labels)\n",
    "        print(f'epoch {epoch+1}, loss {float(train_loss.mean())}')\n",
    "        net.train()"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "epoch 1, loss 0.21708008646965027\n",
      "epoch 2, loss 0.1381392478942871\n",
      "epoch 3, loss 0.12913581728935242\n",
      "epoch 4, loss 0.127198725938797\n",
      "epoch 5, loss 0.1336274892091751\n",
      "epoch 6, loss 0.12410368770360947\n",
      "epoch 7, loss 0.13476374745368958\n",
      "epoch 8, loss 0.12787781655788422\n",
      "epoch 9, loss 0.1228647381067276\n",
      "epoch 10, loss 0.13053394854068756\n",
      "epoch 11, loss 0.14114776253700256\n",
      "epoch 12, loss 0.2078314572572708\n",
      "epoch 13, loss 0.15255621075630188\n",
      "epoch 14, loss 0.124310702085495\n",
      "epoch 15, loss 0.12995916604995728\n"
     ]
    }
   ],
   "metadata": {
    "tags": []
   }
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "source": [
    "# plot it\n",
    "plt.scatter(features, labels)\n",
    "x = torch.tensor(range(-4, 5), dtype=torch.float32).unsqueeze(-1)\n",
    "y = [net(i).detach().numpy() for i in x]\n",
    "plt.plot(x, y, 'r')"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fb570135940>]"
      ]
     },
     "metadata": {},
     "execution_count": 20
    },
    {
     "output_type": "display_data",
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAqOElEQVR4nO3de3zPdf/H8cd788UQIyST6MSlZCuVcJUk58NSksOlpNNVkkPLlK4o4TKn0lVXXeUQipKW86GLIqHIUL8oRTIVlVVs+G57//74bq7Z8Xvavt/v9rzfbrtt3+8+h5ey197en9f79TbWWkREJHSFBToAERHxjRK5iEiIUyIXEQlxSuQiIiFOiVxEJMSVC8RNa9asaRs0aBCIW4uIhKzt27f/Yq2tlfv9gCTyBg0asG3btkDcWkQkZBljvs/vfU2tiIiEOCVyEZEQp0QuIhLilMhFREKcErmISIhTIhcRCXFK5CIiIS60EvnHH8PUqaDWuyIiZ4RWIl+wAEaMgG7d4JdfAh2NiEhQCK1EPmMGvPgirF0L0dGwcWOgIxIRCbjQSuTGwMMPw5YtEBEBbdrAuHGQkRHoyEREAia0Enm2mBj4/HPo0weeegrat4cffwx0VCIiARGaiRzgnHNg7lyYORM2b3ZNtaxdG+ioRERKXOgmcnBNtQwcCNu2Qe3a0KEDPPEEpKd7dJnEHcm0mriOhvHLaTVxHYk7kospYBER/wvtRJ6tSRPYuhXuvRcmTIAbb4SDB906NXFHMqMW7yY5JQ0LJKekMWrxbiVzEQkZpSORA1SqBK++Cm+9Bbt3u6Zaliwp8rSE1XtJc579sDTNmUHC6r3FFKiIiH+VnkSe7c47XQ9CGzaEHj1g6FA4darAww+npHn0vohIsCl9iRzgkkvgk0/g0Ufh+eehZUvYty/fQ+tGRnj0vohIsCmdiRygQgWYPh0SE2H/frjqKtfK0FziOjQiwhF+1nsRjnDiOjQqmThFRHxUehN5th49ICkJmjZ11Z3fdx+kpp75dmxMFBN6NiUqMgIDREVGMKFnU2JjogIWsoiIJ4wNQAOq5s2b2xLffNnphKefhokTXVUub7/t+iwiEiKMMduttc1zv1/6R+TZHA4YPx5WrYKjR6F5c9diInVSFJEQV3YSebb27WHnTrj+ehg0CPr3hz//DHRUIiJeK3uJHKBOHVizBp591vUA9KqrXCWLIiIhqGwmcoDwcBg9Gtavh7Q01wh9xgxNtYhIyCm7iTzbDTe4qlpuuQWGDIGePeHYsUBHJSLiNiVygJo1YelSmDIFli93Le/fvDnQUYmIuMXtRG6MmWmMOWKM+SLHe2OMMcnGmKSsj87FE2YJMAaGD3ftCxoeDn/9K0yaBJmZgY5MRKRQnozIZwMd83l/mrU2OutjhX/CCqBrr4UdO+DWW2HkSOjcGY4cCXRUIiIFcjuRW2s3AL8VYyzBo1o114Khl1+GDz90TbWsX+/RJdTjXERKij/myAcbY3ZlTb1UL+ggY8z9xphtxphtR48e9cNti5kx8OCDrj7nVavCzTe7Voa6sT+oepyLSEnyNZG/DFwMRAM/AlMKOtBa+6q1trm1tnmtWrV8vG0JatbMtQPRgAHwzDOuhJ5ceEJWj3MRKUk+JXJr7c/W2gxrbSbwH+Ba/4QVZKpUgdmzYc4cV1KPjoYVBT8OUI9zESlJPiVyY8z5OV7eCnxR0LGlwoABrkRety506QJxcXD6dJ7DCuplXi3CoXlzEfE7T8oP3wI2A42MMYeMMYOAScaY3caYXcBNwLBiijN4NG4MW7bA3/8Okye7yhT37z/rkPx6nDvCDCdOp2veXET8ruy0sS0O77zj2vDZGHj9dbjttjPfStyRTMLqvRxOSaNuZASpp9M5lurMc4moyAg2xbctyahFJEQV1Ma2XCCCKTV69YKrr3btE3r77a5R+tSpULEisTFRZ21O0TB+eb6X0Ly5iPhKS/R9ddFFrtWgI0a46s5btIC9eatTtDeoiBQXJXJ/KF/eNV++bBkcOuQapc+de9Yh2htURIqLErk/deni6qR41VWuCpeBA+HECUB7g4pI8dHDzuKQnu5aPDRuHDRqBAsXwpVXBjoqEQlx2rOzJJUr50rka9dCSgpcdx288oo2rRCRYqFEXpxuvtk11XLDDa6+LXfeCb//7tEl1HxLRIqi8sPidt55sHKlq7f56NGulaELFsA11xR62ujE3czfevCsQXz2IiJAc+sicoZG5CUhLAzi42HDBtf8eatWMG1agVMtoxN3M2/LwXy/reZbIpKbEnlJatnStWlF586u3Yi6d4dff81z2Ftbfyj0MskpaZpmEZEzlMhLWo0a8N578PzzsGaNq5Pixo1nHZLhxkNR9WoRkWxK5IFgDAwZAp98AhUqQJs2rlLFrE0rwo1x6zKaZhERUCIPiDOVKO/8xC39p3Holm7w1FPQoQP89BN9rrvA7WupV4uIKJGXsNzbwH1zMoxbmj/AjqcmuUbozZoxrvJPbl9PvVpERIm8hOW7DVx6JoMrXQ2ffgo1a0KHDjz1yVzCM4veHzT1dLrmyUXKOCXyElboNnBXXAGffQb33MOgjQtZ+GY8df84Uuj1jqU6GbowiZhn1iihi5RRSuQlrMh2tpUqwWuv8Wi3x2h89AArZg2h3Tdbi7zusVQnwxYm0UArQEXKHCXyEuZuO9ttLTvR9a7pHKp2Hq8tfpZ/fPAq5dPz7jCUU3bRokoTRcoWJfIS5m4727gOjfj5vPr07D+ZWVd3457tS1g0P476x3506z4qTRQpO9TGNogl7khmzJIvSUlz0v7rzSSsmE6YzeSJDoNZ2uRGt65xYGKXYo5SREqK2tiGqMoVXH3N/tuoJZ0HzuDrmhcyY2kC41fNoKLzZKHnuruwSERCmxJ5kMpZbw6uZfuHq9Wmd9+JvNTidvruXM37bwznkl8OFngNd5b6i0joUyIPUvnVm1sgI7wck268mwG9xnJu6u8snTOMXrvWFNhJURUsIqWfEnmQKqje3OJ6QLrhoqvpPHAGO+o2ImHlC0xbNoXKp1LzHJ+ckkbcop35JnNtWiFSOiiRB6mC6s2jIiPYFN+WqMgIjlSpQf/ezzK1dT+6f7WBpXOGcvnP3+Y5x5lhGbYw6axEnbtVgEoWRUKXEnmQKqrePHvEnhkWzgut+tD3zueo5DzJ4rkjGLB9aZ6pFgtnjczHLPkyb6sAlSyKhCQl8iBVVL157hH71vpN6TxwBpsujOaZD17h34njqXry+FnHODMsY5d+Scwza0hJy39xkbopioQe1ZGHqOypkZyj6ghHOMZm0PeTxYz8aA4/VzmXId3j+DzqL25fNzLCQdLT7YsjZBHxkerIS5mCRuw9m9fntWt70qvfJKwxvD1/JA9sXYSxmW5d1xg9BBUJNRqRlzKtJq47U3te9eRxJqyaQZe9m/io4VUM7zKcXytHFnkNw//6tuR8HRUZQVyHRnnaCYhIydCIvIzIOcf9R8UqPNwjntHtH6LFwd2smD2EFgd3FXmN3L/aczbjUstckeCjRF7K5ClbNIZ5MZ2JHTCF4+UjmL9gNEM/nk+YG5tWFORYqlOliiJBxO1EboyZaYw5Yoz5Isd7NYwxa40x32R9rl48YYq78itbBPiq9kV0u2s6iZe3Yeimt3hzwZOc9+cvXt8nzZnBiLd3ah5dJAh4MiKfDXTM9V488F9r7aXAf7NeSwBlPwTNr2FWavkIRnQZzojOw7jyp29YMWsIbb71/llFhrVaTCQSBNxO5NbaDcBvud7uAczJ+noOEOufsMQXsTFRTLmjGQX1Pny36c10u2s6R6rUYPaiMcSvn0m5jHSf7qnFRCKB4+sc+XnW2h8Bsj7XLuhAY8z9xphtxphtR48e9fG2UpTYmKg8Dy1z+vbcC4j92xTmRXfiwU8X8878kdT7/Wef7qnFRCKBUWIPO621r1prm1trm9eqVaukblumRRXQryXbKUcFRnd4mId6xHPxrz+wYtYQOu7d5PX9Iis5vD5XRLznayL/2RhzPkDW58K3fJcSVdCDz9xWNG5Nl4Ev8F2Nuvw7cQLPrHmZCumnPb7f8ZOF7ykqIsXD10S+BLgr6+u7gPd9vJ74Uc7Vn0X5IbIOvfpN4tVrbmXAjuW8N3cEDX/z7OGlMxM98BQJAE/KD98CNgONjDGHjDGDgInALcaYb4Bbsl5LEImNiTrT9rYoznAH49sOYuDtT1Pnz19ZNvtRbv1inUf3G/52kpK5SAnTEv0yIr8mW4Wp88cvPL80gesOfck7V7TjH7c8SFr5im6d6wg3JNzeDHDtdHQ4JY26Wt4v4rOClugrkZchiTuSGbowye3jwzMzGLLpLR75ZCHfnluPwT1GsrdWA7fOreQIw2LydGfM2YpXRDyjXitCbEwU/VvUd/v4jLBwpv21P/17P0u1k8d5/43h9ElaVeD+oDmlOjO1cYVICVEiL2PGxTalf4v6Z1Z+hhtDq4trFDqH/kmDaDoPfIFP613OhNUv8uKSSZxz6oRX909OSSN6rJpuifiTplbkLKMTdzN/y8F8FxMZm8mDW99lxIa5JFerzeDuI9l9/qVe3ccRZkjo1UzTLCIe0NSKuGVcbFOm9Y4mMiLv4h5rwni5RS96951IuYwM3p0Xx6DPEt2aasnNmWk1zSLiJxqRS4ESdyTzxOJdpDrz7i5ULe1PJq+Yzi37trL2kmuJ6zyUlIiqHt8j3BgyrNWmFSJuUNWK+CzmmTUcS82xetNa7t6+lFEfzuTXSpEM6R7HtnqXe319VbWIFE5TK+Kzk7lr0I1hdvPu3NZ/MqfDHSx4cxQPf7LQ600r1ONcxDtK5OK2tHymWAC+qHMJXe9+nhWNWxO3cS5z3n6aWsePeXUP9TgX8ZwSufjF8QqVGNItjpEdH6F58lesmPUIrQ4k+XRN1Z2LuEeJXNxWvag2tcawsFkHegyYwrGIqsxd+BQjNswl3If9QZNT0jTNIlIEPewUtyXuSCZu0U6cGUX/nanoPMmYD17lzl1r+LReEx7tFsePVb3vQ28AC6pukTJNDzvFZ7ExUSTc3uzMKtCCtpIDOOmoSHynIQzp9hhNjuxnxawh3Lxvq9f3zv7Voblzkbw0IhevJe5IZsySL0lJK3xDiQa/JfPikklc8fO3vN68BxPb3I0z3LfdhKIiI9gU39ana4iEGo3Ixe9iY6JIero903tHn+ndkp8DNaLo2X8ys67uxqBt77No3uPUP/ajT/fW/qAi/6MRufhFw/jlhW72nK3915tJWDGdMJvJ6PYP8f7lN3l1v0qOMKpXrqBe51KmaEQuxaquGzsQAay57Ho6D5zBnloNeX7ZFKYsm0LlU6ke3y/VmUlyStqZmvO4RTs1by5llhK5+IW7Gz0DJFerzZ19JzCtVV9i/+8jls9+lCt//Nqn+zszLGOXfunTNURClRK5+EXujZ4Lq2gB16YVz7fuS+++EyiXmc678+J4cMsijM1/9ag7jqU6SdyRTKuJ67TMX8oUzZFLsUjckcyIt3eS4cbfr6onjzN+1Yt03fsxmy68kmFdRnDknHO9um92vXk2NeKS0kRz5FKiYmOimHJHM7emW/6oWIXBPUbyeMchxBzey6pZj9DuG+9qznP/2tAyfykLlMil2OScbilqqgVjeLtZe7re9TyHq9bitcXPMnbty1RwnvI5DpUqSmmnqRUpMaMTdzNvy8Eijyuf7uSxDW9w/2fvsafmhQzpHsfXtRp4fd8w49rESGWKEuo0tSIBl73xc1FOl3Mwvu0gBvQay7lpv7PkjeH0/3y5V1vKAWRa1BpXSjWNyCUg+v1nM5u+/a3I42qeOEbCiunc9N121l5yHY93GsKxStV8une4MUy5Qxs/S+jRVm8S1BrELy/we8ZmMnDbUkZ+NItjEVUZ1nUEmy9s5tP9VM0ioUhTKxLUCuvVYk0YM6/pwa1/m8rx8pWYv2A0j380m3IZ6V7fT9UsUpookUtQcKfe/P/Ou4hud01nQbP2PLRlEYvmx/nUfCs5JY2YZ9Zo8ZCEPE2tSFBoNXEdyR6UCXba8zETV80g3Gby1C1/570r/NfSNjLCwZjul2vaRYKOplYkqHnSqwVgZePWdLznRb4872KmLZ/KtKWTqeJF8638pKQ5iXtHTbgkdCiRS1DIXjxU2Fx5bj9WrUWfO59jSut+dPtqAytmPUJM8h6/xOPMtJpDl5ChRC5BIzYmikwPp/oyw8KZ0aoPd/T9J2HW8s78x3lo89uE+bDhczatCJVQ4ZdEbow5YIzZbYxJMsZo8lu85m5f89w+r/cXOg98gRWNW/P4hjeYv3A0df74xadYqkX4th2dSEnx54j8JmttdH4T8SLu8nSuPKc/KlZhSLc4Hus8lCt//IaVsx6h/debvY4lJc1JzDNrNFcuQa9coAMQySm7UiRh9d4z27ilnk7nWGrhGzyfYQyLmrZjW9RfeGFpAq++9xzzojsxru0gTjoqehzPsVQncYt2nnmdMy71bZFg4ZfyQ2PMfuAYrpYWr1hrX83nmPuB+wHq169/9ffff+/zfaVsSNyRzKjFu0lzejbv7chwMmLDXB78dDFfn1ufId3j2FO7oVcxVC4fTqblrBi0OlRKWrEu0TfG1LXWHjbG1AbWAo9YazcUdLzqyMVTiTuSSVi9l+SUtDybRxSl9f4dTF0+lWonjzP+pnuYc1VX8KA6pjBRkRFsivdfDbtIYUqs14oxZgxw3Fo7uaBjlMjFF94k9Rqpv5OwYjo3f/sZ/734GuI6D+U3H5tvgWtHov0Tu/h8HRF3FNuCIGNMZWPMOdlfA+2BL3y9rkhBYmOi2BTflgMTuzCtd7Rbtee/VarGoNv+wdPtHqD1gSRWzRxM6/07fI7F2yobEX/yR9XKecDHxpidwKfAcmvtKj9cV6RI2VvKuTVRYgxzru5GjwFT+b1iFea9/RTx62fiyHDzQWoujjBDXIdGXp0r4k8+J3Jr7XfW2mZZH5dba5/zR2Ai7oqNiaKfGxtWZNtTuyHd7prGvOhOPPjpYt6dF0eD37woMfTPNLuIz7SyU0qFcbFNaXVxDbePP+moyOgOD/PArU9wQcrPLJ/9KLfv/sCjXYicGVrGL8FBiVxKjQO/er6kfvVlLek0cAa7zr+UySum88LSBKqePO72+ckpaTSIX66FQxJQSuRSanjbG+WnqjXp13sck24YQKe9m1g1czAtDyR5dI1jqU6GLkxidOJur2IQ8YUSuZQavlSQZIaF89L1d9Cz/2TSHBV5c+Fo/vHBq1RwnvLoOvO2HCR6rEbnUrKUyKXUyK9PS4QjnOm9ozkwsQvTe0cTHlb4E8rd519Kl7unM+vqbtyzfQnL5gzl8p/2eRRHSpprdK6ELiVFiVxKjeye5lGRERhcqy5zLqFPWL2XjMyiH2aedFRkbLsH+Nsdz3DOqRMkzh3BQ5vfJtzD1rjZCV3z51LctNWblBkN45d7tLQfoFran4xb8xLd9mxke93GDOs6goPVz/c6hig12xIfaKs3KfO8mUP/PeIcHukxkiHd4rjk1x9YOesR7kxa5VGZYk7JKWmMWrxbI3TxKyVyKTN86XW+pMmNdLznRXbUbcTE1S/y2rvPUPPEMa+ulebMYMySL706VyQ/mlqRMiW74dbhlDSPp1kAjM3k7u1LGfnRHE44KjKq4yOsuex6r+OJjHAwpvvlmmoRt5RY90N3KJFLMGg1cR3JXtaeX/LLQaYvm8IVP3/Lwqa38OzN93G8QiWvruUIMyT0aqZkLkXSHLlILnEdGnndLmVfzfrc+rfJzLi+N7d/8V9WznqEa37wrumnM9NqqkV8okQuZZanzbZyc4Y7mHLD3+jV959kmDAWvjmKkR/Opny6590UU9KcegAqXlMilzJtXGxTpveOpnolh9fX+LzeX+g88AUWNOvA37cuInHucC47esDj64xdqlG5eEdz5CI5+DJvDtB236f8c+ULVD11nIQbBvD6NbFY4/54aXrvaG3wLAXSHLmIG3wpUQRYd8m1dBj0Lz68qDmj18/kzQVPEvX7EbfPH7owieSsihrVnIu7lMhFcshe5u+L3ypV44FbnySu06Nc8dM+Vs4cTM8v/uvVIqI0Z4Z6nkuRlMhFcomNiSLK1704jeGdK2+h08AZfFW7IVOXT+OlxAlUT/3d40t5255Xyg4lcpF85DfF4k2p4qHIOvTpM54Jbe6m3b5PWT1zMG2+9ez5kDZ4lqIokYvkI79OitOy2uF6WuGSGRbOK9fdTo+7pvJbRFVmLxrDs2teIuL0SbfOv6lxLS/+BFKWqGpFxEOJO5KJe2cnTjda4uZWIf00wzfO475P3+NA9fMZ3nUESXUbFXleJUcYFRzhpKQ6Vc1ShqlqRcRPYmOiSOjVjMgIz2vPT5Urz4Sb7qFvn+con+Fk0bw4hm2cT7mM9ELPS3VmcizVeaaaZZi2lZMcNCIX8VHijmRGLd5NmtOzjSfOOXWCMR+8wm1frGNnnUsZ1nUE351bz+3zDTCtd7RG5mWIRuQixSTnfDq4/1D0zwqVGdFlOH/vEU/9lJ9YPvtRBmxf6naZogX1aBFAI3IRv8tulevJCtFax39j0srnuem77WxoEMNjnYdy5Jxz3T6/1cU1mH+f9+10JTSoja1ICWsQv9yzE6ylX9JKnlz/OqfDHTzZ/mGW/+Wvbp9+ae3KrB3exrN7SkjR1IpICavk8PDHyxjmx3Smy90vcKB6Xf615J9MWzqZqiePu3X6N0dOaDl/GaVELlJMxve8kjAvVhHtrxHFbf0TmNK6H92+2sCqmYNpeSDJrXM1Z142KZGLFJPYmCim3uFdi9yMsHBmtOpDz79NJs1RkTcXjiZ+/UwcGYX3Ok9Jc9IgfjmtJq7T6LwM0Ry5SAnIuVdoZCUHKVk14e6o6DzJ6HWv0z9pJTvrXMqQ7nF8X71ukec5wg0Jt2sLudJEDztFgog3tecd9n7CP1e9QLnMDEa3f4jEy29y67zqlRw83U0bPJcGetgpEkS8aZe7ulFLOg2cwZe1L2L6silMWTaFyqdSizzvWKqTEe/s1FRLKaZELhIg3rTL/bFqLfr0Gc/U1v2I/b+PWD77Ua788esiz8vItNpKrhTzSyI3xnQ0xuw1xuwzxsT745oiZUFch0Yet8fNDAvnhVZ96N13Ao6MdN6dF8f9W9/F2MxCzzuWevaD0sQdybSauI6Gejga8nxO5MaYcOBfQCegCdDHGNPE1+uKlAWxMVH0a1Hfq3O31bucTvfMYO0l1/HEh7OY8/bT1Dr+m1vnZs/Ra1u50sEfI/JrgX3W2u+staeBBUAPP1xXpEwYF9uU/l4m8z8qVuGh2FHEdxjMNYf+j5WzHqHNt58VeHx2ok5YvTfPg1ZtKxe6/JHIo4Afcrw+lPXeWYwx9xtjthljth09etQPtxUpPbKTuTe7EGEMC6I70u2uaRytXJ3Zi8by1H//Q/n0vDXncYtcDz0L2j5O28qFJn8k8vz+7uWpabTWvmqtbW6tbV6rlnY8EcltXGxTpvWOPmtXov4t6lO5fHiR5wLsq1mf2AFTmX1VVwZte5/35o7gol8PnXWMM8MybGFSgTXs2lYuNPlcR26MuR4YY63tkPV6FIC1dkJB56iOXMQziTuSGbowye3j232zlUkrn6di+imebvcg7zRtB6bw8b4jzJDQSwuIglmxLQgyxpQDvgZuBpKBz4C+1toCa52UyEU812riOo9a45735y9MWzaVlgd3seQvN/Bkh4f5s0LlAo8PDzOcU6Ecv6dpO7lgVWwLgqy16cBgYDXwFfB2YUlcRLzjaaniz+fUpH/vZ5l0wwA67/mYFbOGcFXyVwUen5FpSUlzqoolBPmljtxau8Jae5m19mJr7XP+uKaInC02Jsrt/izZMsPCeen6O+jVbxIAb88fycOfLCQss+jWAKpiCR1a2SkSQjxdCZptR1RjOg98gRWNWxO3cS7zF47mvD9/KfI8VbGEBiVykRAS16ERDm+anOPaI3RItzge6zyUZj9+zaqZj9Dum62FnmNBqz5DgBK5SAiJjYkioVczIiM873EOgDEsatqOrnc9T3K12ry2+FnGrP03FdJPF3hKckoacWq6FdSUyEVCTGxMFElPt/d6mgXgu3Pr0bP/ZF5r3oO7P19G4hvDueSXgwUe78y0jFq8y+v7SfFSIhcJUXEdGhHhcG+xUH5Ol3Mw7ub7uPv2MdQ6cYylc4bRJ2kVFFCSnObMJHrsGo3Mg5A2lhAJYTl3HvLlJ7nW8WNMWT6VGw7sYOVlLYnv+Ai/R5yT77ERjnBuuzqK9XuOcjglTTXnJUg7BImUcgUtGAo3hgw3fs6NzeS+T98jbsMbHKlcg6HdRvDZBVfkfyxn9+GIcIQzoWdTJfNiph2CREq5/KZaIhzhTLmjGeFFLM8HsCaMV6+7jdv6J+AML8eCt55g6MfzCc+n5jz3rwXVnAeWErlIKZG9fVzOplvZo+Q+113g9nV2nX8ZXe5+nsQmNzJ001u89dYo6v5xpMjzVHMeOJpaESkj+v1nM5u+dW/jiWyxX65n3JqXyDBhjOw0hFWNWhV4bFRkBJvi2/oaphRCUysiZdz8+65neu/oImvQc07DJF5+E13ufp79Nery78QJjF/1IhWdJ/Odwonr0KhY4paiKZGLlCFF1aBHRUaQmetf6d9Xr0uvfpP493W30XfnKpbMGc5D557IdwpHAkNTKyJlUPaenTm3e3OEGapULJdnk+ac/rr/c6Yun0rVkyfY89jTNJvwRKF9znOWR6pM0XcqPxSRs+RMstUiHJw4nY4zo+h8cO6JFCavmMZN322Hbt1g+nS46KI8101OSVOZop8pkYtIgTzdtMLYTAZuW8o/Nr0BGRkwcCCMHk3ir+F5Rvq56aGo9/SwU0QK5GnpoDVhrL7lTvj2W7j/fpg9Gy69lPSHB1PlWOGbq6tM0f+UyEWkwE2XKznC8p0Cz65SSTwCrS64jdaDXmFJs3bEblnCxlfu5cl1r3HuiRSP7iXeUyIXkXxXhTrCDc4Mm6eHVvVKDib0bMq2739j2MIkklPSOFStNkPaPkTb+15heePW3LNtCRtfGcTID2cTmfbHmXNVplg8NEcuIkDeCpMTp9JJSctbwRIVGcFNjWsxb0vBbW8v/vUHHt30Fl2/2siJ8hWZ2TyWFe3u5O+xV+tBpw/0sFNEPNIwfnmBHRVzV6PkFm4MmdbS6uRPTNr9LnXXreT0OdWYdf1tzGjSkWrnnatSRC/oYaeIeKSguexwY4psmZtpLf1a1GdzxPm0vOZhutz9PBvPa8QDa2ay4ZV76bJ6HmMXfqbe5n6iEbmI5Cu/RUMRjvBCSwv/d1wYac7MPO9HH97L8I3zuOHADo5WjmT+TX0Z+s4UqFjRr7GXVhqRi4hHCuqmWNQWcxGOcE6l503iAEl1GzGg97Pc3u+f7Dv3AoYuewkuuQRefhlOF7xvqBROI3IR8Uh+I/VsUVnL8IcuTHLrWq2/38W8g8vgk0/gwgvhqadgwABweLm5dClX0Ii8XCCCEZHQlf2AsqAeKp7Me3984ZXwZjysXu1K4vfey6HH/8HU6+/ks5YdubFJHW0p5waNyEXErzxZ7m+Aab2jARi75Auu3rWJ4R/Po8mR/eyrUY/prfuyvHFrrHHNApf1Xi0qPxSRElFY2WJ+jOGsRUfGZtJx7ycM+/hNLvv1IF/VasD01n1Zfen1YEyZ7tWih50iUiI8XYKfeyxpTRgrG7em4z0zGNLtMSqkn+aV98azdM5Qbvr2Mw4fS/VjtKWDErmI+FV+y/29kRkWzpImbbjl3pd5rPNQqp08zqxFY1n61uOwdm3e3wBlmBK5iPhVdtliUVvKuSsjLJxFTdvR9r5XGN1pMBc5f4f27eHGG+Gjj/xyj1CnRC4ifpe9pdz03tFn6tB9lR5ejoXRnfggcSPMmAH79kGbNtCuHWze7Ic7hC4lchEpNrExUWyKb8v+iV2KXEjkDmemZVjiHhJb3erqhT5lCuzaBS1bQufOsH27H6IOPT4lcmPMGGNMsjEmKeujs78CE5HSJd9WuWGG6pUcZ1aO9m9Rv8jrZFjLqMW7SdzzGwwfDt99BxMmwNat0Lw5xMbCzp3F84cIUv4YkU+z1kZnfazww/VEpBTKb8l/Qq9mPN3tcupGRnA4JY31e45SvVLRc+tpzgwSVu91vahSBeLjYf9+GDsW1q+H6Gi4+moYPx727CnWP1cw8KmO3BgzBjhurZ3syXmqIxcRyH+5vyPMgMGtjaAPTOyS981jx+D11+Hdd2HLFtd7TZpAz55w223QrBn5bnsUAoqzjnywMWaXMWamMaZ6IQHcb4zZZozZdvRo4Xv6iUjZkLB6b56eLc5MS+Xy5YqcUw8vKBlXrw6PPeZ6APrDD64Ho7Vru0bnMTFw8cX/+35m/s29Qk2RI3JjzAdAnXy+9SSwBfgFV4/5Z4HzrbX3FHVTjchFBApeBWqA/Vmj7Qbxyws8P98ReUGOHoX334fFi+GDD8DphLp14dZbXaP1G26AcsHdfsrrEbm1tp219op8Pt631v5src2w1mYC/wGuLY7gRaR0KmgVaM73CxqZe1wFU6sW3HsvrFjhSurz5kGLFjBzJtx8M9SpA4MGub5/6pRn1w4wX6tWzs/x8lbgC9/CEZGyJL9KltwbNLtzjMeqVYN+/Vzz6EePuj536ACLFkGXLq6k37ev6/0TJ7y/Twnx9WHnXCAa19TKAeABa+2PRZ2nqRURyZZ70+f8WtW6c4xfnDoF69a5EnhiIvz6K0REQMeOrumXrl0hMtL/93WTuh+KiHgiPR02bnTNqS9eDIcPuza8uPlmV/VLjx6ukXsJUiIXEfFWZiZ8+qlrpL54sWsRUliY6wFpz56uB6b16hV7GErkIiL+YK2rLUB2Uv/yS9f71133v1r1iy8ullsrkYuIFIe9e10J/d13/9fr5corXQm9Z0+4/HK/LUBSIhcRKW7ffw/vvedK6ps2uUbvl13mSug9e7p6wfiQ1LVDkIhIcbvwQhg61PWQ9PBhePll13sJCXDtta6v163z+22VyEVEikOdOvDgg7BmDRw5ArNnu1oENGjg91sF93pUEZHSoEYNuOsu10cx0IhcRCTEKZGLiIQ4JXIRkRCnRC4iEuKUyEVEQpwSuYhIiFMiFxEJcUrkIiIhLiC9VowxR4HvvTy9Jq59QoON4vKM4vKM4vJMsMYFvsV2obU2TxP0gCRyXxhjtuXXNCbQFJdnFJdnFJdngjUuKJ7YNLUiIhLilMhFREJcKCbyVwMdQAEUl2cUl2cUl2eCNS4ohthCbo5cRETOFoojchERyUGJXEQkxIV0IjfGPGaMscaYmoGOBcAY86wxZpcxJskYs8YYUzfQMQEYYxKMMXuyYnvPGBMZ6JgAjDG9jDFfGmMyjTEBLxUzxnQ0xuw1xuwzxsQHOh4AY8xMY8wRY8wXgY4lJ2PMBcaY9caYr7L+Hz4a6JgAjDEVjTGfGmN2ZsU1NtAx5WSMCTfG7DDGLPPndUM2kRtjLgBuAQ4GOpYcEqy1V1pro4FlwD8CHE+2tcAV1torga+BUQGOJ9sXQE9gQ6ADMcaEA/8COgFNgD7GmCaBjQqA2UDHQAeRj3RghLX2L0AL4OEg+e91CmhrrW0GRAMdjTEtAhvSWR4FvvL3RUM2kQPTgMeBoHlaa639I8fLygRJbNbaNdba9KyXW4B6gYwnm7X2K2vt3kDHkeVaYJ+19jtr7WlgAdAjwDFhrd0A/BboOHKz1v5orf086+s/cSWnqMBGBdbleNZLR9ZHUPwcGmPqAV2A1/x97ZBM5MaY7kCytXZnoGPJzRjznDHmB6AfwTMiz+keYGWggwhCUcAPOV4fIggSUygwxjQAYoCtAQ4FODN9kQQcAdZaa4MiLmA6rsFnpr8vHLSbLxtjPgDq5POtJ4EngPYlG5FLYXFZa9+31j4JPGmMGQUMBp4OhriyjnkS1z+J55dETO7GFSRMPu8FxUgumBljqgDvAkNz/Ys0YKy1GUB01rOg94wxV1hrA/qMwRjTFThird1ujGnj7+sHbSK31rbL731jTFOgIbDTGAOuaYLPjTHXWmt/ClRc+XgTWE4JJfKi4jLG3AV0BW62Jbh4wIP/XoF2CLggx+t6wOEAxRISjDEOXEl8vrV2caDjyc1am2KM+RDXM4ZAPyxuBXQ3xnQGKgJVjTHzrLX9/XHxkJtasdbuttbWttY2sNY2wPUDeFVJJPGiGGMuzfGyO7AnULHkZIzpCIwEultrUwMdT5D6DLjUGNPQGFMeuBNYEuCYgpZxjaJeB76y1k4NdDzZjDG1squyjDERQDuC4OfQWjvKWlsvK2fdCazzVxKHEEzkQW6iMeYLY8wuXFM/QVGSBbwInAOszSqN/HegAwIwxtxqjDkEXA8sN8asDlQsWQ+DBwOrcT24e9ta+2Wg4slmjHkL2Aw0MsYcMsYMCnRMWVoBfwPaZv2dSsoabQba+cD6rJ/Bz3DNkfu11C8YaYm+iEiI04hcRCTEKZGLiIQ4JXIRkRCnRC4iEuKUyEVEQpwSuYhIiFMiFxEJcf8PaXvwo0bkpvcAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Part II\n",
    "In this part, we will introduce an operator that you will see in your assignment -- the `detach()` operator. Please pay more attention to this operator since many silence bugs are caused by it. **The `detach()` operator will return a new Tensor, detached from the current computational graph, and the returned Tensor will never require gradient.** \n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3. The `detach()` operator\n",
    "It might be more straightforward to understand the `detach()` operator by showing how it influences the `w` parameter."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "source": [
    "print(f'Before calling detach(), w is {w}')\n",
    "print(f'After calling detach(), w is {w.detach()}')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Before calling detach(), w is Parameter containing:\n",
      "tensor([0.1407], requires_grad=True)\n",
      "After calling detach(), w is tensor([0.1407])\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Obviously, by calling the `detach()` on the parameter `w`, the returned new Tensor doesn't require grad anymore, such its value won't be updated by the optimizer.\n",
    "\n",
    "Notice that the `w.detach()` returns a new Tensor, but the original `w` is not modified, which means the `w` still requires grad. "
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "source": [
    "w"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([0.1407], requires_grad=True)"
      ]
     },
     "metadata": {},
     "execution_count": 22
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Looks simple right? Now we will show how this simple operator is used in practice by two concrete examples."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4. Example 1: Scaling gradients\n",
    "Imagine a scenario where we have a parametrized function (for example, a neural network) $f_{\\theta_1}(x)$ and we are interested in **scaling** the calculated gradients by the values of some other function $g_{\\theta_2}(x)$, which **shares the first layer with network $\\theta_1$**.\n",
    "\n",
    "Without using the `detach()` operator, the parameter will be updated as: $$ \\theta_1 \\leftarrow \\theta_1 + \\alpha \\nabla_{\\theta} \\sum_i^N g_{\\theta_2}(x_i) f_{\\theta_1}(x_i) $$\n",
    "\n",
    "However, notice that this gradient is actually not what we want. We only what to **scale** the gradient $\\nabla_{\\theta} f_{\\theta_1}(x_i) $ by the function $g_{\\theta_2}(x_i)$, but here, the gradient flow will pass through the $g_{\\theta_2}(x_i)$ since it shares the first layer with $f_{\\theta_1}$. \n",
    "\n",
    "This problem can be easily fixed by using the `detach()` operator on $g_{\\theta_2}$. After calling `detach()`, the $g_{\\theta_2}$ doesn't require grad anymore and gradient flow won't pass through it. So we can safely wright the above formula as: \n",
    "$$ \\theta_1 \\leftarrow \\theta_1 + \\alpha  \\sum_i^N g_{\\theta_2}(x_i) \\nabla_{\\theta} f_{\\theta_1}(x_i) $$"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Pseudocode\n",
    "f_values = f(x) # Parametrized by theta_1\n",
    "g_values = g(x) # Parametrized by theta_2, but shares the first layer with theta_1\n",
    "g_stopgrad = g_values.detach() # Same as g_values, but not parametrized by theta anymore\n",
    "\n",
    "objective = torch.sum(g_stopgrad*f_values)\n",
    "objective.backward()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 5. Example 2: SimSiam Network\n",
    "To better understand this `detach()` operator, let's take SimSiam (https://arxiv.org/pdf/2011.10566.pdf) architecture as an example.\n",
    "\n",
    "SimSiam architecture is used to learn image representation via unsupervised learning such that the similar samples stay close, while dissimilar ones are far apart (contrastive learning). SimSiam, as shown in the following figure, achieves this by learning an encoder `f` such that for an image `x` with different augmentation `aug1(x)` and `aug2(x)`, we can encode them into **similar** latent representations `z1` and `z2`. \n",
    "                                                     \n",
    "                                                       <---X--- stop-grad\n",
    "                         ---> aug1(x)---> encoder (f) ----------------------\n",
    "                        |                                                   |\n",
    "            image x --- |                                               Similarity\n",
    "                        |                                                   |\n",
    "                         ---> aug2(x)---> encoder (f) ---> predictor (h) ---\n",
    "\n",
    "Notice that both encoders `f` share the same parameters, if we directly maximize the similarity, the training will collapse, aka, for all images, the encoder f with always output the same z. Why? If for any input `x`, the encoder returns the same vector, say $[0, 0, 0]^T$, the similarity is always maximized, thus the optimizer will lead to this solution.\n",
    "\n",
    "In order to get the non-collapsing solutions, we need to **change the computational graph**. How? SimSiam proposes to use the **stop-grad** operator on one encoder and only update another encoder. To achieve this, in Pytorch, we can easily call `detach()` on the encoder to remove (see the `D(p, z)` function in the pseudocode). \n",
    "\n",
    "The Pytorch-like pseudocode of SimSiam is:"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Pseudocode\n",
    "# f: encoder network, with parameters \\theta\n",
    "# h: prediction mlp\n",
    "\n",
    "for x in loader: # load a minibatch x with n samples\n",
    "    x1, x2 = aug_1(x), aug_2(x)  # random image augmentation\n",
    "    z1, z2 = f(x1), f(x2)\n",
    "    p1, p2 = h(z1), h(z2)\n",
    "    \n",
    "    loss = D(p1, z2)/2 + D(p2, z1)/2  # loss is the similarity between p1 and p2\n",
    "    \n",
    "    loss.backward()\n",
    "    update(f, h)\n",
    "    \n",
    "def D(p, z):  # negative cosine similarity\n",
    "    \n",
    "    z = z.detach() # Notice here, stop grad, the key operator\n",
    "    \n",
    "    p = normalize(p, dim=-1) # l2 normalize\n",
    "    z = normalize(x, dim=-1) # l2 normalize\n",
    "    \n",
    "    return - (p * z).sum(dim=-1).mean()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Optional: Image Classification\n",
    "We have shown how to use an MLP to fit a simple model on a toy example, now let's use neural networks to solve a real task -- training an image classifier. The task is relatively straightforward: we have a bunch of images, we want to distinguish which classes they belong to. This example serves as a full example to help you understand how to prepare the data, how to build the model and how to train it. \n",
    "\n",
    "References: 1. http://www.d2l.ai/chapter_convolutional-neural-networks/index.html  \n",
    "            2. https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Prepare Data\n",
    "Like the above examples, the first thing is to prepare the dataset. Instead of generating a synthetic dataset, this time we use the CIFAR10 dataset, which is easy to download and process with the help of PyTorch. The image pre-processing includes two steps: change them to `Tensor` format and then normalize them. After loading and pre-processing the `trainset` and `testset`, in order to train the model, we typically want to pass samples in `minibatches`, reshuffle the data at every epoch to reduce model overfitting, and use Python’s multiprocessing to speed up data retrieval. This can be done easily using the `DataLoader` API, which wraps an iterable around the `Dataset` to enable easy access to the samples.\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "batch_size = 4 # training minibatch size\n",
    "\n",
    "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n",
    "                                        download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,\n",
    "                                          shuffle=True, num_workers=2)\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
    "                                       download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,\n",
    "                                         shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "source": [
    "# let's plot some samples\n",
    "figure = plt.figure(figsize=(8, 8))\n",
    "cols, rows = 3, 3\n",
    "for i in range(1, cols * rows + 1):\n",
    "    sample_idx = torch.randint(len(trainset), size=(1,)).item()\n",
    "    img, label = trainset[sample_idx]\n",
    "    # process data inorder to show them\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    npimg = np.transpose(npimg, (1, 2, 0)) # notice the default image is 3*32*32, we need to change them to 32*32*3\n",
    "\n",
    "    figure.add_subplot(rows, cols, i)\n",
    "    plt.title(classes[label])\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(npimg)\n",
    "plt.show()\n"
   ],
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAHRCAYAAAABukKHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACbF0lEQVR4nO39d7Qd2XneCb+7wsnn3HPPzREZaAANNDqy2STFJsUgJokiRUm0LMsKXmNbHo2W57O0rE+2NR7bGvsb+7M1GqcZy0GjSIoji5QVqGZWR3YG0Mi4F7g5nZzqVJg/AHnwPJd10G2KOM3u97cWV/NFnVO1q2rv2vfsp573NVEUiaIoiqIou7EG3QBFURRFeb2ik6SiKIqixKCTpKIoiqLEoJOkoiiKosSgk6SiKIqixKCTpKIoiqLE8KabJI0xR4wxzxtj6saYnxp0e5Q3B9rvlNcLxpj/YIz5B4Nux7cLzqAbMAB+RkS+FEXRvYNuiPKmQvudonwb8qb7JSkie0TkzDfaYIyx73BblDcP2u8U5duQN9UkaYz5goi8S0R+2RjTMMb8ujHmXxlj/osxpiki7zLGHDXGfMkYUzHGnDHGfPct3x8xxnzWGFMzxjxjjPkHxpivDeyElG8LtN8pg8QYc68x5rmbS/2/JSKpW7b9FWPMJWPMjjHm94wx07dse58x5rwxpmqM+ZfGmC8bY35iICcxQN5Uk2QURe8Wka+KyN+IoignIp6I/AUR+YcikheRp0TksyLyxyIyLiL/vYj8mjHmyM1d/O8i0hSRSRH5kZv/U5S+aL9TBoUxJiEivysivyoiJRH5lIh8/Oa2d4vIL4rI94vIlIgsishv3tw2KiKfFpG/LSIjInJeRB65s61/ffCmmiRj+M9RFP1pFEWhiJwSkZyI/C9RFHlRFH1BRD4nIp+8uST2cRH5e1EUtaIoOisi/3FgrVa+3dF+p9wJHhYRV0T+eRRFvSiKPi0iz9zc9kMi8itRFD0XRVFXbkyIbzXG7BWRD4rImSiKPhNFkS8ivyQia3e++YNHJ0mR67f8/2kRuX7zwfVnLIrIjIiMyY0Xna7HfFdRXgva75Q7wbSILEdYyWLxlm1/9v8liqKGiGzLjX43Lbf0s5vfX/qWt/Z1iE6SIrd2nhURmTPG3Hpd5kVkWUQ2RcQXkdlbts1965unvEHRfqfcCVZFZMYYY275t/mb/12RGy+UiYiIMSYrN5ZWl29+b/aWbUawD75p0EkSeUpuaD8/Y4xxjTGPishHROQ3oygKROQzIvILxpiMMeYuEflLA2up8kZC+53yreIJufFH1k8ZYxxjzMdE5KGb235dRH7UGHPKGJMUkX8kIk9FUbQgIr8vIieMMR81xjgi8pNyQxN/06GT5C1EUeSJyHeLyAdEZEtE/qWI/KUois7d/MjfEJEhubE2/6si8hsi0h1AU5U3ENrvlG8VN/vWx0TkL4tIWUR+QG780SVRFD0mIn9HRH5HbvxyPCAiP3hz25aIfEJE/oncWII9JiJflzdhvzNadPm/HWPMPxaRySiK9G1D5Y6h/U6509yUApZE5IeiKPrioNtzJ9Ffkq8BY8xdxpiT5gYPiciPi8j/Peh2KW9stN8pg8AY835jTPHmUuzPiYgRkScH3Kw7zpsxLd03Q15uLHVNi8iGiPxTEfnPA22R8mZA+50yCN4qN3TLhIicFZGPRlHUHmyT7jy63KooiqIoMehyq6IoiqLEoJOkoiiKosTQV5P8wPd+FNZiTRjgl1MZiEulcYinpicgTrgpiGu1JsS9ngdxFPUgNrQ0PJzPQux5uFy+2cDv94IQ4kwKiy+ksgWI3QQez4p8iFtN3J8EeLxUOgexEfx8r4vnPzYxD3HXp7etPfx8q4nn22mUId5cXYd4bt8+iH0rCbEjeL7/6z/+J0YGQBB0oSEh3XfLYLc19LdeEGE/xUQ2Ipah+0ahF9D3fbzOXboPO2t4nf2tSxAvXngO958ahvjlcxcgvvziUxB/4C13Q3z94jLE29YOxPe8/aMQX1vDbGKH5rFf7h0dgfilV7D9lRqEUu5gt3j0vR+GeHICnwNf/fxnIP6V//R7EH/k4x+D+Gf/zi/d8X734e98Lz7rTP/fD5Zl9Y1ZxAr5X6hPG/4Cxfau/UcUUx8PcbsfUie38BLvvuD4Lz6NKTH0jQhjHrO7DmD1vx48MSUc+hfaXy/AZ3NIHwgCiumC//Hn/yS2z+kvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJmnROnZEmlshR5pgF9eF19dRC8nnUQtJJFDT9ELUCF0nAXHXx3Xkahfb12qghtdqdyA2Nu6/U69D7Pfw/EjCFIdWrX0fzzeby0PcbDVwe9KFOJXC6xeGeHyLdIAgxOOl82mIx4fp+tI6fauDmm+mhFpU1MHrNShsG7VSO+Zz/y+kWdI3QtIYW3TfdzZWIK5WVnHvHdQg69UtjNdRI8xG2xAHAd73pZVNiB9//Ax+vo737YtPnIM4UyxBfNdD74c4tLDj5kwL4lIStfd2DUXH8ZEhiLe3cRxPTY5CbLqkyQbTEE8ePAnxyvanIF5cRE11EFikeYUB9imbNTzSLG9UNLs17i8yBtQnQ1bEKA5ZOOf28OFIw/Rv93PoNhKiWHh92DpoUXsc4euFu2PN0pBG6fKYpvcIIgdPyOYD4KNUfNJoIxoj/dBfkoqiKIoSg06SiqIoihKDTpKKoiiKEkNfTTJBItzQMGpYHvkas0XU5Gp11ORGR9E/tbqKWlA6iXP2aB7jC0uoBW1uVSBuRejDTJHPMWvhQrVraOGa1t1HcqiZrlVQm0rmURtKZVDLkRCvj5BXx3JQq+rW8fwcB8/HMvh546KmefTwXogbG1jAvtbF77td1KrMLnfXYFi6fBr/wcbrViyNQZzJFSFuNVDjWlp8BeLzZzB+4evPQpxNoxY+lsf7GgV43Q7OokZnZ6Ygvn4VNcpKuQLx3BRpw9SNCknUuyYPHYE4TRpjuI2a6wxp5fkMnl+FPp9MU3si1FDztL+F8y9D7FuobR88hD7Pj333OyHOJmkcDgBDY98ijSyk9wPYtxhE/Pn+GqLY/X+fGNLYQvKoM3bEKiId/za+T/ZdsojI7dlle6T3AAy1J+LrSefDmqOx2NvcX8N0SBP26H547PG3cQz0Q39JKoqiKEoMOkkqiqIoSgw6SSqKoihKDH01yUIBfXepFPrXoi5qCSFpcN0O+hZfOYN+sFYXP3/qAGodOa8C8Z4kakHpJG5/8hJqKz36G+DoOGpyZMuU3FAR4lQH/W+JHu6vU0U/WzWJvsVkBjVNm3yQfhP9aVsb1yDOFDD37czBYxC7lBt2dQ01zWvXFiCuJ9G/VhpCTZM10kHxpc/9NsR7jqIGd+zkWyHO5VCj9H28zgtXr0L8+c9jYfXqNua8zZEGuEB+2OECXreNFdTWH37vIxB3fdQk17eqEF9dRB/i+AiOg6npQxCHbdRXDo9jP5u8axbiS6+8CLF4OC5TGRzXTdKXHMq53Oqg3nPhPPpKhw7guDZp1O6PHsD2+W28PoOAfX4R5y4lz3jImqGwJsneXVLxbqPxsQZokWYqpJHOz2A+4Cw+iuT8FdSVe4KaHEmsu3yN5ja+R9Yg2fjJm21634U9+RJgvmQ3QXm2k9jnWXNtRNgHPZJ0g/DVv3+hvyQVRVEUJQadJBVFURQlBp0kFUVRFCWG22iSaNjaIX+XY+PXW03UOnqU63V9GX17oUGtI5rFdec6rRsnyH9VSKAGOVfEz281cF366B70ac7tRa2n3sT9byyhFjUzjAvblQpqSU6b1rkzRyFc28AclzZ9P1/EdfbGNubsvNBGLesY5ey8Tvtv0v0Ievj9KERtyHFfH5rkvuMPQ3zobsz9WSC/LesLhSJuP0oa5gvPY/3GbhN9k9tbqLFVyI87OoSa4dgYapTrW+gPznBtvzreh72TqC0/9OA9EBdH0Ye5fQG1fYfqkra6qDe1QozNBmmwEzMQd2vY78fGixBf28Z3A3bqeH7lNmrtK12sp5nNo9aezONzYBDs0iRJU0zQs04sfFaxT9Le9fuDfH67fIj0abY50j+kKNfs4f0HIS4N4fFbVewj63TPAmov54LdnVuWNUhuMPkcLRIF6f0Vm7zHWfp+wcX2JRPYRw3p5mEPv9/2qFbxa/CE6y9JRVEURYlBJ0lFURRFiUEnSUVRFEWJoa8m2aP6ihbVP0xmUUupbaAXh5d9O7QsPZTCdeKDc5jzMkHp9bbXUDPLFiiXbFSBeIYa0Oui92ZzHf1ZjQrGE/OohUmEfrjpEmp+i2t4/NoW+h5DMuuwN4m9PFEDtbGs4Dr8tctXID60D7W4TBY1Tjx7kYB8m2Fw+8qNd4IH3vEoxCyH2JyHksKADLDjk3sg/v4f+hGIn3n8KxA/9YXfh7hIuU6HqS7qsXtQex5K4XVf3XwJ4gePoK9zfgL1lJkRjG2X6rTuQU/c9iaOux2qs1qvot7jJvGKNpsYN6qozXepsOrpc+g7DUnLr2xQ3dnuAsQHjr0N91+pyKAxNBh31VMkdtUn5PqIu2yRt9njLl8kbnYo12uRvLoR1b/02ngPxovYhxvtCsTbDdw/17q1DT6cDD2LEjZpsg4+GyMq8Nhp4PskXYoDGnNex6EYx3hmCL24ZP2VFqXRrvVefb5g/SWpKIqiKDHoJKkoiqIoMegkqSiKoigx9NUk19bIxzeOmqEf4Lp0MoVaSodyt1rkNRqjunnZJGqOr5w7C7GhdfFsBuvoTZVwh+tV1BgdG9exq1WsOzhSRK2p3kJNsJTDde/lOmqUNtWTDCuYSzVhob9uZhavZyqDOoCfx3X5s2fRb9bqYR3EyTH0AzYoN2xuGBM6BgEu1KcT2L6BQfpLp4KaW0T+21QetWnLwevmUk7a+X0HIC6VUOM7OI+aYdTGcdDcwPs6TOOiWUcf4p5R7FeTI9i+jIuaXi5C9bhDml/KxXHU6+G42NzCfu/6eJ/bDdQorzcWIc5SXsyAHhObO3h+Y2kUgGwP450qfr5E6nhqiB4EAyCkZ0NEOiwriuyzs0hEtO3++r4hI2JI9R9tH/eXpfczxofwH4I2eqTLNbzHTg99kvkIn32+YB9K2ujNzeXwWXtgL+r8xQL2mdNnMF9wrYbHzxVwrmjaOJf0yNfYJJ3dSWAf67XIm5wpQpyg/VtsRO2D/pJUFEVRlBh0klQURVGUGHSSVBRFUZQY+mqSQ6QVpJP48QppL8ZQXboWrosnqYbYxAiuY3/966ixzc7PQ7yzifUdm01ch85R6tF8DrWgTA7bNzxchNhK4Dp5eZ20qBZqfJkcaohODzXYroVxpoS5Ui+exvO959T9EHvkYxzJo86xsog5SF+m+pXXrqNmej9pZ7trxpFRc0DUrz4D8Stf+TLEuRLVTywWcQcW+WkPPABxcgzvQ5r0lmMnMHdqYxUv1Ab5be0A9ZadCuYoHkrgdc05/T12nMu07ZHh2EdPmUueNGminoOK5u7ah0k6nwLV02y3Uc+ZyKM2PzuO46CYRm276eNzQrKkT7m4v0HAPsZoVzJVDFlzvJ0Gyff4dk5M18E+M0IPt3yCPM4e3sOAYvY5TpCGOFpAjS6Tx3s4u+c4xMXSJDbYptyopPmFAWmeVJs4R557K4fXh32iXA6y0aFavVRTNmCNmb3WfdBfkoqiKIoSg06SiqIoihKDTpKKoiiKEkNfTXJkDOu+rW6iRud1UavweB2Yaowdnkc/20gG16m367j/tVXMTZrNonbEOkGtjdrMxPgcxHYStZ56B9ftXfINJg3qDJZLXpzxfRC//OJpiKtlqhu4HzXeSgm1mKUV9APmE9ieoTxqP/OUa7VXw/Z98F2Ye3alRj5T0mwdm8xYA+L3/vU/hjgVYDd95CMfhjhavwxx2EQP2MoSarejx9FPOrTnCMTba7i/Ky/gfU1HeF+CHvomWxX010ZU33HUoF816aI+49ioP9mkbUcOjrPFTdT+0xnsF4a07RTVGpwj/+xyBb/vkZ7zgZPTECep33RsHGcO5bI1SdQkmzXSLAdASHK8S7o211fscd5jetaxRsmapGWwT1O5REka9sZSe3v47OJcr66NJ2QL6fR58nSTTzNXQp9kOode4laXfZ7YR50k+Ti5PmWEmmOSns0u1etM0v5YV48c7FNeiHNTSLFlv/oapvpLUlEURVFi0ElSURRFUWLQSVJRFEVRYuirSdqkjbDXxCEtpdpAH2Gnjevq1W3U6JYrqKXMzWE9xFoZ8xGGSdJuaCE+mcN19GodNTpTR3/Z3H7M4VknLcmmmmbsrbl2DXNeZujzB44eg/jChfMQHydf5PoOrqs3NrE9bgI1zJkZ1CjbIWo/Wzuoza1XsH0zhylHJ+kwg+LyFfR3fu+P/iTEo/vxurZC0kPSpIFRDuHgGuaV3Fp5BRuQxus0PoHafGcD64R6EfYrsak2oIP9tkuFQ7PkSWv28Pv1bRxHfhu326RvDWVRb/EN1fKrY3yV9heQBy9P18Proiabp9qG2f2ocdrjeP1bHezXde/V1/b7VuG6eI42i3zskySRLQq5Vix9n2yRNmmUCfZRhniNu3SNDGl4mQT2gZA825yr1BjOXYrHb1U3cH8BPhsSWcxvnBnC9iRd/LxFxsYEPWsSlNc7keAYx1BIz+Iibe/S+zLdbgW//xp+H+ovSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJulRTa8s5Sq1LVwHvrKAOSu3t9E/lvZxYT5fwHXqbA41yQN70YdY3UEf4dbmEsRjU+h3q1Nu2dYWaoiXzlHu1wxejqlZrJnWbOI6dy6N1yc3ilrMRhXPr1zBOn8XLl+F+NQDD0L8uQvoE33w1F3YnjJqjovLqJWtrKJWtvcQ5iRNpdArVW2jZjwofviv/jTE++95COLIw/NKDaFWaztU2y+J+oxP+lG7hvpLUMb7bHKYpzKMSOPzUIu3QhwXk8Oo1xRS2E+aXWxPPUCtPuGixuqtYz9KUN3RRJb2T/3Esih3K2npR+/CnMlOHvWnly5jP+t5OG5qHmqOfpeubxc159kk5tYdBA7VHGXfo6HcoQ49+zjVaxiwLxK32zbdA4N9biiNHuasi5+fnJqBOOjgmAgM+wTJN2nh+QWUW7VHmqZXx3toke+ybbCPB10cIwkH+4hFF2x3PU7WJHEM2JRnu0eaq2XhGMr2SOMNtZ6koiiKonzT6CSpKIqiKDHoJKkoiqIoMfTVJCOuO0c+xVoN8wcmEqhtpFK4bjw9i36zE/vQ1zg5fRgbEKJG1vVQkzy4H+sCLq5ehHh2L/rpqjZ5h5LoT6tu4/7XN1BT5WXsBPlEd6roc1xaQp/nyDDmI1y9jprqyAjmtp2Y2wvxH3zpOYiHEqg73H33QWxvhNdvbv8hiIMAv+/3Bu9XExHZd8+9EIc26Q2kn6QdvI9BEs+j20GfoU2+RSuJ2qyhvJuZImpy6ysLEOcE9zdaQj2JPXYRjTqX/qHZQD3Fp/Nt+hnajv2uR3VHh/Okt7Vo3ArqNedexHGQmsb93TeLeTzPX0QNcmsLr2d7CK9PMnmOPv9eGTSsgYn09z1GEcX0bUO6r0XeWUP5f4v0vkdpFPv0tasLEB+i46eoHmOUxWdxu419yo76j33OxWqRL9FQn5EI+0ghj3m2G+RZt1y83g7FfL0dB8/HTZIXOsL7xWPY7eL5GAvHTD/0l6SiKIqixKCTpKIoiqLEoJOkoiiKosTQV5NstXAdu0v1FtlbMz2FfrJ6Hf1jO2X0LY695TjE5y6jf+roUfRrnXwQfZTpEH2ZYQ/rBuZpnX5zE0/XpoSKafKD9droPcoP4Tq376PW4+PlknvvPgVxbQd9mstL6JM8++zzEBf2o69x7hDWh8wlcF09n8LzPXYENd/SFGrCIWmSvdbrwycpAeUmpX4WUV7IcAj1nIg8Y8khvA5CnqoCaekB6UcXr2A/q2+hZleYQu19qV7E/Xl4vJE03rch8nlyLcM6aZbPV8k3uYra93tRzpLhAravRrls/Q7e95dfxHqa3aex3y/MYT8r7cNxuv/owxBfamE/L46iz3OndUleb7DvkVXH3b5D/L3BuUd7XdLkQtTJjUHdtlKuQNwpo0/R72CfDl30xkYO3vP8MGmWXeyD3S7q1CHVc+R6mwF5hb029vEueew59+qu3LiEQxqx41CNVCrwaVv4ea+HDW41ae7CR19f9JekoiiKosSgk6SiKIqixKCTpKIoiqLE0FeT7PRw4XZ4GDXBkLw+w0PojQl9XOduN1GjvHBxAeI9ezD3ao+0nK9++U8hPnYYtRyf1plD0pbyOVwXL1ewPY02rnOnqbxiJKhdra8t4PEMakUbG3j+TZSCZHIv5l/s1FHUdCm/49w+8pF2sP3nrqKWdNeJUxCnC+jDbFcwp2fOYbfXYHCSeOEj0idMSPUNKUdvmMTtnEeTNc4WaZQrlzA36eYlrANaGkf954uPo170Co2bR05izt2xBOpRIdUypObL6g6279Im6ltr59FvW3SnIH7gVBHiRIo8ZKzNJ7H9f/T8AsTWKo67v/X+j0A8M78X4va1t0FcouOvvw40yYh9drfRJC0yTR8+iHmm90zjNVp89ou4uw5qgjN7cGx3u9hHDpewD/U8fFZcW8AxUOliJzqyF9uzf08RYq4N7Cbw2d7uUC7YCC9Ql2qMcm7WBPVxoXqWtt0/d6vj4P5CwfZw+1pNGmMhft9x8D2GfugvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJumSf2xyCjXJ1ZU1iA8ewlyptoPazZe+iOvyG2X0s917D9VjXEFtKEUa6Moyamq9Hs751evLEPttynFJ+QwzLq5T97roP1u8juv2I0X0KhlDWg/F65u4v0MHMJfq2TOY0zKbwvNJu7iuPjSMmubMPOoiLtVgq1fQD9epYM7NoI7+tUERUPE9n/y5IWuQHdR+Lcq72WxjP4laqFcsUp3QJx4/DfG9o7i/MMR+srKKGmGTavktX0eNde8h/L5fx35pk96yvYTtz1K/3FPCYbxTRQ/bWh2vz4F57FftdfJNpvD6b6eLEH/v934M4kOnvgM/v4XjarSI/byxgX7hrIO5Xl8PGNJpuSCkRb68+Wl8NhaT2GcXySNt2hWI15bwfY4dD58dJRvv6Tw9i0t5fD/BTZDOTDVWuZZsnrzEPa6/6GEfaTWxz4YNHAM+6fyBj3EYUs1W0iSFfKaRzSIxxoHPY5TrS+L+g9fw81B/SSqKoihKDDpJKoqiKEoMOkkqiqIoSgx9NclKpQLxU0+hT/Gekw9CPDI6DfHqGmopYyPo36o2SfNbwhyZ1WX0/d19eC/E5y5jTsiJefQS1Sn3qghqdKURTHLZbuA6+/g41s1braK2ZCzUKH3Kh1iuYI7P8SJ6lapb6G8rDWP7mnXUEJ0WrvuPlHB/Hcqv2CFfakS+SifA67OyjBruoIgC1D/CBra7W8M6n34HNcqIcg5XSXv26G/D0Mb7Wu6i/vPl0wsQP3wSP3/fAdSHjlPtu7Fp1E8aVdR7bKr32CGPV8Zg/F0HUbuvrpCeRfUoU/kixGYE83i2y6hFmxy2/3u/590Qf//Hvgfil587C/HGKvbrE+/A67NWxRzFOx0yJA+ARAJznQYe+QLJJ0nlCqVbq0Ds5/Ce7z3xVohJIpSvLmCffuopvKafePcDEBdGMA/zcBE90Kkk9hEnhTG/L2KRL9ElzbVH52+o/mQijftLkuU60eHcqZSLlTVHCkPSFD0qfcv1Pbn2cSqN97NjXv3vQ/0lqSiKoigx6CSpKIqiKDHoJKkoiqIoMfT3STqoFbiUf69K6/CXr16EuNVFbeTIXQch/vrTqCXVOrSunEcvz9U11E6SGdQUJybnIG5Qe+b2oma5uoGaZpdyzdoWaj3TY+hvW1hCzXEog9cnT7luK2XUGFl7Gy3h57eXUYvK0bp7N0AvkEPr+o1mBdtjoS6wtIb1Ox8nbWlQcApZwwIQ6SdRhF+oUm7Wa6sViJsWXve5SfSbJui+feaLT0Bcb+P+Hyat/P5R1JZHplADvFojz5iFepLr4bjIlrAfDFHtvjPks4xIL9o7ie8CJKk239UejvMKaboPvX0vxOsL6Od97LOfhvgY1YFN26idp8gHWr6IdWAHQTqB1yQyLPBXIJws4T0NmuTFLWBt3T0PvgfioVG8J3J5AcJqFcdyzcP4ygq+T5ClhL8zM/g+hW+jZuiSL9Gi+owhjSmf7llkkw+RPNzGJQ2TNF9DXmKh4xuDYzzwcbtHmrHt0OcpN6xDnn+39+rzVOsvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJpki70vQw3XezQ3M/emmyDdIdfzGiqjx7T98FOJXFtAX+dZ7MRdp0EJN0g1QQ/ziF/4YYovWwatl1FoeefejEF87j1pLJKgFJelqzUyj9nLplecgtlN4/LFp1EzPv4ya4Nr2AsRtF3UNJ40abI90AzdEHaWQxr+BdhauQLy0gLlxr62hZjooWGMMKA+k7eN9cUPUGLepcOcf/dHX8fPTqOFNjD8K8d13Y67Rrzw5C/FLW6g/pYbQF5jooJ81FaHmOTvGeTKxX3YCbF+3jPs7e+4qxXh8bwT3n/BRc5z0sCNvuaiJPlHFcXb2Faz3ePgw3o/778FaiHMH0MPnrWK/evnXMI/pWAfPfxAMZ/CaeJvYRqu6AHHKLkJc62CfmNuDz4a6h2Px4rNYo3R2Gp+NRw6hLn72DOq2G2XsI/MTlDt2DEJJp6hGK2mAPBFEwhofaraBkIZLcciSI/ku+YAhvUfgh3i9/A7q7D3S3R0Hn7WdHj4DOj5+PqT29kN/SSqKoihKDDpJKoqiKEoMOkkqiqIoSgx9NclGA704vS5qQeOkrYTkPUlRvsBQ0KsyMYnr6PV6BeLVMi5sT7g4p7eoLuDJEycgfvpFrAt4AKUhSRlcpz94ELWVhQXUDSpN1GrqNdRE73vwEYjPnXsG4i4lHEzmSHOkdf/8EGo7QnUVJcL9ZShf4eYi1vsMdjYg3t5AHSXskZAwIKI25W4lv2fYRY1uaQN9hZ//yhmIr53G+3h3Aftdx8V+sO/AfogfPH4AYjOK2nDNYHt+7Uuo/R65gPdhfgZzHM8PFyGuNyoQX7iM2v/6Mp5vCeUYGfNR47v6BGrt+Xvw+HtLePz3v+MkxJdbeH1aFexH7ziGnjy3gP348S/g/bj8OPqX/8KPYl7TQZAz2OdqOwsQh1X0RC9v4j0YOYhjb3sV3zfICj4Lox7qtNeefxbimQy9z/HI/Xi8YfRZFul9D4bKYUoQksZHmmBAOn+Xnv1dqjfpdfDzEb2PYkgDNKSJskYaUD3Irkc1ZUPcH/smex5+P2Gjd9mkNXeroiiKonzT6CSpKIqiKDHoJKkoiqIoMfTVJHkduNFErWOoRP41F3fnUD69Xo81NQyzlIt1c7MC8fs+hNrF9SuotUxN7oH4/WO4bt+oojbUqKPG2Alx4f7uk/fh8c59CeJ2DX2G25Uixju4Tl+IcF3/oQcfgviVS+gTdVJ4/arbWG9TMnh+lTren24DNcca3b/z1/B6cP7EgUH+V86bWd7Gdv/7Tz0G8VXS7N5/EjXFU+/+EMTOEPogwxrdp4/i9zccPP7iy3ifvkZ5KV98Ce+bcwb1rVwG9apiBjXAYhL/lg2oll6VxtVWhHrXKnny7GIR4kNvvQfik/txXI+uYj1Pr4v9Ouvj9sXzWJf0sS9g/chHvvttED/w/nfIoGmTzmp5eE52GvM2j9x9N8T5EdS5G1TzNPLxmiUpF2q3hsfPTeL7Ch16H8Oh3LJhiH3AMvgsYx8if97zureJWRPEPtIhzdInzXNXfUhKjWsL9vl0hnyPXXw/xotQk3Q5NyzZIBOU/9nwXNQH/SWpKIqiKDHoJKkoiqIoMegkqSiKoigx9NUks1lchze00OtTja+1tXWIJyZwXV0oJ2eafJQp8vn1SAfYqpJPcxr9bDs7qMWMjBYhbtWxfdkUrqtvrKBWtBRiewsFrIuXy6L2sriCseNiPcxKGTXCixcwH2Oliuvu6+uv4PFKqEG65M9LCOUnpHyFz55BzbNFtkjrdfInU7uCHjO7hxpls4PX6ewreF7TU5gjt3TvAxBPPfhe/P55zPnrkn504Ph3QHzl9EvY4AxqlCf2oiF3eQvHzQtLFYgrVTy/YgL73eFp9CEeugv7wcVLeL3KWzhOpkroZ753L2qQhyhHc3cFfYx50qe6ddR8V2ooOL1wsQLxA+97N8Tf/X0Yu030gQ6CHmlunNnTuHiNEvT+xE4F9X5DPsOEjfe0OIr3JJ0iEzdpcobel+h6eI8TeDixaDDzs8AP8Puex9vx2Rj0UETk3Knsa/Tp2dnzabuP7bPJq+wbnAvsBJ6/TXOPz+eLhxObRFHLRs9+P14nj0VFURRFef2hk6SiKIqixKCTpKIoiqLE0FeTDMhbkxvC/IBuAtd1Aw8/79I6cy6DGiPnAywUcDt7bc5dwjp6xsOaatkcrmOHpKE2dtC7lJxDjdFqoVfppUX8/NWLmAv21ImDECds1ESn9qO/bruMfrnKFmo7wyX0Wj1/BjVJ2UTvVlIwHhvG8zn7CmpLl1dRE024eL3a0esjd2vSQ80xIn2k5WD9w+94B/rshnvYL8bux7yXTfLENTeegLhsUU7cpeMQnyr8FYj/8NlPQzyZxfvynh/E42/8x9+H2KqgJnmkhMNyfBzPN1FEvapLAtoD92E9zB//wbfj/mdx3GUMab5N7MeJAupvqTzqaY0m9puj7zgG8fFHfxDiHOlt5Reonw+AWh2vwQzVbg2S+GxaodqstSq+H5CmJ2t2H74/4UyirpwhjbNNzz6P8hnX2+Tz80kzdej3D41tP8D9+aR5koQogY/PdtYwI6r/GAQYez75FG3sA70Ij99ukQffxvNzyGTvU81ZEWxvFHIuWZyb+qG/JBVFURQlBp0kFUVRFCUGnSQVRVEUJYa+miRriraF68rlbdTUJqlOXkjr4I6L37dMDmJDNd28Dq5T79DxxrO43QnRq5QbwvqSjU38m2Dh4iWIpYe6QtJBHeLIMdSmigXUaK8toV/OJLC9XC+Tfae5FLbv/pOoeTbqlG+xhdrXuS3U4l4+T/4zg7fbIt1ibqh/Tbo7xWoNNTFLsJ+cP4fX+eA9WP9w/yGsC+rVMcfuF7/4KxAvVXH75F709y5fwdyjhjxcOR/9sAdPYB3Qk6Oo3f+1v4g+zesV8tAJ6mOXFtEH2W7guPrQ93wXxN/1CLb/UJb0nSZq09HwXohzE6hthyGeb2YO9bXty0sQJ8Yxr2kii+0J66iVN8voX8aj3xn+4AnURd+yDzWw4ydQ5x3LFyG26P2Czhq+P9HaxnPsVDFfcJDHsWdobK6tood7bR3H/okj+yBOJ1CTS9iULJU0PNYUQ8qNGtB7AWyqDtgXSfuz6P2HKMD9R/R7zbbZ90j1Jslzb5n+++Ncs5xLth/6S1JRFEVRYtBJUlEURVFi0ElSURRFUWLoq0mmKadjm7w5XFfOGPbK4LrxSAG1pTBEraZH9RYnSCP7oyu4zn9iP2ofKxefgfiCwTp6iQjPZ3kBta35WVRDbEFt7O7j90Lst1A72rPvCMR10lSNhbrA7DS259JlzAlareHtGR1Hf9r6OrbvEvkgjYOa8uFp1M6GhjIQz08OQg3azWNfOwvxxB7UZo+8FX1/B49hLlKvi9r0madehPjyEvbLShXPe3wfatG5AvbT1WXs92Rhk+Hpo/h59qhlsJ+fe/Y5iK+QP3b/CLbnwEnUxpM0rsbmsR+GDeznz51HDfHKeazL+p770cOXzWE/fOECavef+fdYz3M080WIZ/8etn9sgvQja/B1TNcbqFn9wbN4jXYMXpP7H8Q+GRSxD/V6OPZWywsQFzZQY5wu4tj0SJP8yhMvQJyzcew+dJI0SYeuacA+SXwWBSHekx5pgB2yUAf0rBeDY8SQDXGXjZHebzER769vuAvuQVaS3qfhL7DG2gf9JakoiqIoMegkqSiKoigx6CSpKIqiKDH01STbbdTc2h3UJLlmmU0rvxZ5bZwA8/1Nz1JdvEXUAYbH0W+WpnVmcdB70/Rx+/o1zK/4jrdjHbtTDz0K8YtPoZZiKH/i1774ZYgtOv7d9z8I8bVnUGM8vB/9ZRkfNcUFB9fJdxq40l7vol+u2kRtq0O5cN/24F0Qv/NB1HA3VtAn2nsNNda+ldxzP/pbnTTeV1cwt+vqha9D3CLf3ZVXFiA+th81zCP75iFeDLDfFCewnxaL2N4tqvt5+vGv4P42KhBfXkBtvSeo2U3uQZ/ngaOoN73lbQ9DvLCMuWjPYVlTGc/hOHpiETXfx7/4JLbvLF6PzCh+/6WnUMM8NoL99KP30bjdwHcF/Dyen5NCTXUQvPs9+GzoUO3XQ0ewzT0L+6Tl4KO0m8c+86dPou785GXso0ev4E3rkEZXr6MH+rs+8haIsxnyPfbwWcCe9dBQfUryNbZ6uD0iDZTzYjMW6cwW+zJJkxS2cdLnLRs/b1P7Oc+4od9/EfkqzWsonqu/JBVFURQlBp0kFUVRFCUGnSQVRVEUJYa+mmQyiZpbrY7+s12aJK1DpxKUv4/2X8riv7yV8iM+/+IL+P0e5iJt1IYhLo6g1yjEZXx5+iXUjvaN4Tp7eQ33v8M5JUew3mOZatBdI021OIzr6rOTeyB+8jHUQOtNXGefmMc6gl9/GnUNn8xBxw/OQTw3jd6tsSLqKPk0ak/XtjAf5KCotVA/MQF205KD+sMrL2A9yFQStdW7KJfr2gpqePXKNsSjY3ifxKZhwn5Xuq9DxTGIx9v4+T1HMdes46JfNpdDf/D0zAwe3uC7Aakt1GifehJzzc7PFiE2KazlZzvY7z7/BGqOW80zEO8t4rjZ9wD6Ngvkq9xer0Ac+hcgLo7iuB0EJ47hs8fmpxX7Bkn/t8ip1+nh2F8so7e0VcM+eG0Tn60nDqMO/ZH3Yk3S6RJpfF3sA9RcCelZzT7Ohoean2ewT3I9R0OaYK9H9SXpehjSFKOA5w7aThpiEFJ9SDo/y+Jcs/h538dnCvf5fugvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJulTvj9eh7ZpnTmXRS0om8Ht+SHUHl55+VmIew1ct3/5/GmIZ4ZJI6Uabs89h/6v/CTWbPPTeD7l2hbEx6ZR0zMJ1IaaHdSuhHKj+pQPcGkRNc25UdQIPQfP99oKruvXfKy71+nhQvzYMPrL5kdRNxjK4f3wfEwyauVJs0yixjsodioViJd28Lre+5a9+HnSMOeGKAfvEGqvjevYb9ZJzxmuYD9PWqjBcSZJP4v9LDWKOXaPDeF9OtBCsfwrX/tTiNeW0L9aoLqpQYj+2tUlyk17CfvdpfPYT3M5bI9Pft9KB9sXkh7U8lATffx51PJdwbqyzauYOzYR4rh+94fRoziQDMKkcXnk6eZnXzKDY81l35/h3KB4gFwmD/Hdd6EG+b63oo4+msP9GbpHQrq2F+E97wnG9R7ec0miV9e1cGpgjZNzpYbSv54jK4Ah7YE1TKF6krzdEvJRkubKqWUj+nxkXn2+YP0lqSiKoigx6CSpKIqiKDHoJKkoiqIoMfTVJD0PvTu2y/n6cKU54ZKfrYjaRy9Ab9HXX0L/ldfE40V0/PIGajH1RgXiY3dhHb2mje3rUA7LpIPnsxWituVmUaNL5XAdf2MRc6laVfQ+2TbqAGdJm8lNYm7VMfr+wrPodytkURs7dRi1nxL5RPdN4fVPZVG3WK2hrmHbpFMMiDZpr8vXFyF2UqifVLcqEEdN1I7PX0WNbv063rfeBB6vkMF+nLXxOg6V8D6k06jZraxizt6JHubxZP/x2Ahq3/vnSSsewc9bWdSrJsaxfSNjqGl+4Yvor52cQ1/nsVPYL66uoH933wRrsshqA8//334Bc9OeOjoJ8azg/bhwEf3F+9/X93DfEmwXx6ohTSzwKTeoxblBUfMqldAre+L4fRAfGMPvP3wUvbBZgx5s1thaVP8xorzVbcE+0/BJs3PxnhrSMIVqAYe7CkIiNmuY/HESJdl6HND+/ZCvN37eon9gTZJ9lQ496/1dqmo8+ktSURRFUWLQSVJRFEVRYtBJUlEURVFi6KtJVsivFpHXZozqPSaTqI3s7KDmZSzMGcl15LotXIc/cgQ1u1SCvErkU7x4HTW9RAZznw6V0L8WkfcpTfn8sgG2v1lFTbVYwHX/0hBqS6P7UWeYmcd6jstLCxA/8yLWMRTy9jx0L9ajHM6ghjo6jufboNyyHnmZel3Kj9i/RNwdI0sFG0ujqEF2aqhp9SpY+295E32PL19GP+72Omrb+05iv0geR62XPWLbTdSWx0dQ4wsi7OcNytt5hWoHTkyjZjc9hv3o4uXzEC9uok/x6EHU4h9+BGsNOpyrVXDcZcjPnEqhXjU3jJ7BJ5/H3KuTeeznFxeuQezSuwrTB05B3IqomOAAsDgXKP1+sB3SwHYZ8UjXxksuH/6OYxAP25gn2RV8/4J9h77BMdDwUUeWEH2XPmmEtoue6ShiHyPVj8S9S0T1KPn0+R/Yl3hbW+JtcrFGIfsisb3sY3Vc9rBj+y1O/toH/SWpKIqiKDHoJKkoiqIoMegkqSiKoigx9NUkA/LKJJP48dFh9BGmM6il+B3UxCoNzB2azOLn82lcN/dpnfvaFmpJWxWMi/NHaX/oGxwr4P5yCdRCNjdR01xfQ+2oWd2hGDXL+SnMETq/F+sGprO4zv7KK89AfPEi+tuOHkJ/3Z4ZPJ9sAtfpW5TPMaCcnCHVuEskcLubIJ1jQDgkF9x1CLXp+b0HIW5XUAPbXkOf3vA0apjbqxWIMxOGYtR38i5e96pPuVQD7NelEfTItXuo8W2Usb3D5NO8uIj9sEvHSwhqnAtXUMvuBdhv3v8+1Ci3l/B6vHQecxg7edRIjzyAmuu1Jo7T9A7qafvG8HotruP+x6g+5tv34P0dCEH/XKAMa2ycq9SifMDFBF4jQ7lhJYF9xDMUk47s2yh69lizI00zYTi3LIbsaySJb5fGZ1He7p7Peb77a36sMe7KC865X23WUDn3K8K+S+HcrruMnPHoL0lFURRFiUEnSUVRFEWJQSdJRVEURYmhryZpW7gOPZRDr06xgOvkvFDP68JJ8iHaVMOs63HdPNxfM4G+zOwcaoC5UdRSSjatkzdRC2qsojbDOUC7Xfx+Ywe3v3IBc4AGFvoUP/DRvwTx//9//bsQX7yEWtJQAa/HqZPoiywOo4ZrBeiX29nchHh2bi/EvYh9oXj7t3dQcx0U1dXrEO87iv5Si0TLyYPH8fOHsC7o3XWsH+mRfzQiKfZs7TLEwyNYL3LzlXMQp4ZxB9Vt7BceHk7e8Qjm8Ww1UK+qR5yLlbT7KdSeE+0itrdIdURbqJlOTqPGePEy9ptqGfvBtavYnvseQI1z5QXMwdxu4wmPkmmwRPU2R0o4bgeBxZoXaWQR+er8gHOLokZnGbwHThKfpZGNfaYT4ufbXfI1Ul7lHr1fEAi2J7FLM8RnhU3ny17gkDS7nocfMPS+iEvHY98p52JlDdElz3voU/3HqP/cYjv9Td6305j7fve/+ZuKoiiK8gZHJ0lFURRFiUEnSUVRFEWJoa8mWRwqQjw5MQ7xaAl9kgEtRHtUPzLH3p0M+tGcHGovw3nUQnKkWfZaqDUVLNRCeluobfWaqL1s7qDfrBvg3wztOnqdalXc/9Ia+jQPn8J19a3KAsTJBJ7f5jauq3/iE1hIb+84Xi83hd/fuI4aay6H/jSPvViGas6RjzWVRB1kUIyMoc/vrqP7IG5TXsqV66sQZ6lfTUygD8+dpjyPKbwPZ1/G/XkJvG4ra9iPpqbRF9mqY7+4eBb9rxNzByDOJfB8pqbwfC0L83zaFo673DBqfKdfwFyvY9PoK33gLacgvv8hPP+7TqJGurmMvk3XxXGy/7sehdjroAbaaKBP8vIZ1HRfePYJiO9+21+UO41hzZB8k6zRca7SXZqmjWOxZljTw7HZIw1OSINkTc4iDS7ySKPskqZnc71Hfv+DNUDOjYrn5/D+WDOk62fo89x+rs8pFtWX7NGzTNirzPeLdke5XrnWbz/0l6SiKIqixKCTpKIoiqLEoJOkoiiKosRgeK1bURRFUZQb6C9JRVEURYlBJ0lFURRFiUEnSUVRFEWJQSdJRVEURYlBJ0lFURRFiUEnSUVRFEWJQSdJRVEURYlBJ0lFURRFiUEnSUVRFEWJQSfJbxJjTGSMOXj7TypvZowxC8aY93yDf3+HMeb8N/rOa92XosRhjPkPxph/MOh2fDvyppgk9aGivF6JouirURQdGXQ7FEX5xrwpJsl+GGP61tRUlEGhfVN5PfNm6Z9v+EnSGPOrIjIvIp81xjSMMT9zc4n0x40x10TkC8aYR40xS/S9//rr0xhjG2N+zhhz2RhTN8Y8a4yZ+wbHersx5rox5l135OSUbzceNMacNcaUjTH/3hiT4r53s9/9rDHmJRFpGmMcY8wPG2MWjTHbxpj/7wDbr3ybYIy51xjz3M3n1W+JSOqWbR82xrxgjKkYYx43xpy8Zdu0MeZ3jDGbxpirxpifumXbLxhjPm2M+b+MMTUR+ct39KQGxBt+koyi6IdF5JqIfCSKopyI/PbNTe8UkaMi8v5XsZu/KSKfFJEPikhBRH5MRKB8uzHm/SLyGyLy8SiKvvjn03rlDcYPyY3+dkBEDovIz8d87pMi8iERKd783L8SkR8WkWkRGRGR2W91Q5VvX4wxCRH5XRH5VREpicinROTjN7fdJyK/IiL/ndzoS/9GRH7PGJM0xlgi8lkReVFEZkTkO0Xkp28+2/6M7xGRT8uNvvlrd+B0Bs4bfpLswy9EUdSMoqj9Kj77EyLy81EUnY9u8GIURdu3bP+EiPxbEflgFEVPf0taq7wR+OUoiq5HUbQjIv9QbkyG34hfuvm5toh8n4h8Loqir0RR1BWRvyMi4R1qr/LtycMi4orIP4+iqBdF0adF5Jmb2/6KiPybKIqeiqIoiKLoP4pI9+Z3HhSRsSiK/n4URV4URVdE5P8QkR+8Zd9PRFH0u1EUha/y2fltz5tiTTmG66/hs3MicrnP9p8Wkf8URdHL31SLlDc6t/a5Rbnxy/B2n5u+NY6iqGmM2d79FUX5r0yLyHKExYIXb/53j4j8iDHmv79lW+LmdwIRmTbGVG7ZZovIV2+JX8tz8w3Bm+WX5DeqLH3rvzVFJPNngTHGFpGxW7ZflxtLZHF8QkQ+aoz56W+ijcobn1t17HkRWYn53K19c/XW7xljMnJjmUxR4lgVkRljjLnl3+Zv/ve6iPzDKIqKt/wvE0XRb9zcdpW25aMo+uAt+/lGz9I3NG+WSXJdRPb32X5BRFLGmA8ZY1y5oRUlb9n+f4rI/2yMOWRucNIYc+uDakVurN//lDHmr/95N155w/CTxphZY0xJRH5ORH7rVXzn0yLy4ZsvhSVE5O/Lm2fcKv9tPCEivtx4HjnGmI+JyEM3t/0fIvJXjTFvufksy9587uVF5GkRqd18cSx984XFu40xDw7oPF4XvFkG2y+KyM/fXEb4Pt4YRVFVRP663JgMl+XGL8tb33b9Z3LjhZ8/FpGaiPw7EUnTPq7JjYnyZ40xP/HnfwrKG4Bflxt96MrN/93W3B1F0RkR+cmb310VkbJg31QUIIoiT0Q+JjfePi2LyA+IyGdubvu63NAlf/nmtks3PydRFAUi8hEROSUiV0VkS248E4fuYPNfdxhctlYURVEU5c94s/ySVBRFUZTXjE6SiqIoihKDTpKKoiiKEoNOkoqiKIoSQ99kAv+/f/I/wls99eYWbLfdDsQ72w2Ix8cn8PN2/zm5MJSn/bsQDxcxXerIMFoX7z52P36+hHayYNcRKXGJodDCf/Dp836Ee+SXoCyD37eNjTF9PvB93H8PY7YohSaBcUQnIPj9aq0M8XNffwZiv4Of/+T3/yDv8I7wj//R34UT9bwebA9CvA9t2p5Ow4vHYug+dDrYb1Mp/Hyvh/tLJpMQ1+t1iKMI22NZ2M8NdSy+qJ7nQdxsNrF9GWwf79/3e7Qdh3XCxX5iO7id+20yieMuDLGfV6t4/kL9LqDPdzqQwVHyeRznId3PX/oX/+qO97tH3nMULoJF12hsbAzi6VnMDPjwWx+GOFXIQewbPMdUKgMxvz/Z7eA9dZwExdi+TrsLMd+Dbge3uwm8x9wH19fWMV5dhrhSrUDMfbZJYyRo4ZibGZvE7TTmLl68CPH1a/hCNz165Z4T90Ds0/lcvLQIcSJTgPjC2YuxfU5/SSqKoihKDDpJKoqiKEoMOkkqiqIoSgx9NcmINLhOF5O+Wz5qDaWRIsSui8u8jUaDtuM6e+Dj8ZLpFMSpFK7zu7RO77q0Tt/DdfCehQv/qQTu3yJtxaa/IQwJB7bB7SwJRqQhsnbFkqhloWaZSOH52A5uT4aoK0R0/XoRrvPXSfNsN/F+eC3ULQYFa1btNt7HOrU7CEjL9VGPSKXwPrNG6Xl43t0ufp81O/q6ZLM52o4fCAO6L6SXMCMjqKVbpOU3WzjuEgnUTLPZLMR8Pqw5sh7G1481w2QSxx3vnzXSToeeGzZfz8EXNRkeRc3RSeA5zuzZA/GhI4ch9ulZ0GzjOUcGz7lRQw3PoWehRe8vtHeNzf5JYLjP8z3nPpqiezozMwVxPou6+Po6apbLK6hZtuo4Rtv0HkCDtu9sbkLsezimD+zHrKL1Cn6/VqlB/Oh3vBPiQwePQXzuMmqU/dBfkoqiKIoSg06SiqIoihKDTpKKoiiKEkNfTTKXQy+P56P2Uq6i724kjdrI0FAR4mYT1+kLedRehgoY5wr4/ckJ9EnmM+jDNKQLdJro1anWKhDPzcxD7FqkUbKjjWyLhv/GIM1QKAzJ3+bZuP8e+8t6qPU0SItrkfep1sDzLdd2IH7lwisQX1lEL1IU4gkOqpTJhUuXIG63UYPrkbZqSMtlPSaXw35bq5HPj32MhrV0/Dx71Fiz47hLegz7JjMZHGfsOePzTaX7nx+ffxiyZouaIetbrLkmk7i/dJr/tsbrY1M/Tybw/rTaVYjbXby/g+ADH/koxNl8rm+cG0KfXZs0yFq9ArHjch8jz7SNfcqxUUO0LI5xf5GwZxv7oEPHc5z+GiX32c0uxiukQS4vYywBtidFXuNKBeeOvXv2Qnz44CGIN9Y3IH726WdpO2qaG6trED/y9ndAvLr96uuW6y9JRVEURYlBJ0lFURRFiUEnSUVRFEWJoa8m6bi4js0aZcfDdWb2W3ld/IdMahjioQLm7xspoTen20PNLZfFAtl75vf1PX61iuvUm5R/cHQEvVFhFs+nTfkEa6QVdUjbCUhjbAWoKVZpXX+D1v3r5CertPF42w3UctbJp1puo2bZa2G8ubYKsQnweKWgv3/vTnHl6lWIHYf8qqSZsceM/bwRJXpkX6Hv43bOY5lIsB8X9RwmID3Go37UbuJ9Yw2Sc7MWitjv2QfJPk7OPct6125fJGqWXeqnYYjt5Vy3bPhNp/D6BiGeTyKJDSiV0IM4CGbn8f0EIc2v7XFuVLrmdA17lE94pITPmnQG7yH3mcBnjzW2x6H3Hwx5wJeXMNcp69SJXX0Yv99qYZ8cHx+H2CfdnfMpe+RtLmZQ071++QrE586dh3iDcsfu34c+yfvuvw/3t3gdj1dAzXhsFN938f1Xr4PrL0lFURRFiUEnSUVRFEWJQSdJRVEURYmhrybpebhu61EuVLKnSaWCmtnONmo7e+fR+5JN4zqxCVHz5PSEYYDr8i5pSx7lkGxSvEM5Kc9vo/emtYO+wpUK+r+utjA/4HZAOoVDOUHJu9QgDbJNKSvbpHE2A4y7FPuUz7G+hN6giHLlelW8P2MZvH5henfFzUFg2aS32DEfvAnntGW9xiO/aTaHuWEdB29EvV7rGycS7EOk+pbkmWtRrlWHBg5rijadf5f0MNYceaBwHdcHH3xL3/1fuYL6EOflbJM2zhqkTZqxQ7lmUzZqmKxRch7OQRAEeI4RmZw5v6zh/LakM3frGFeS/ftQj+4x57lOZ0hTTKLGR11QEuRtDSh/8Mp1fD+Dn53cJ4uT2Kc6pOO3GtTnq9j+ynoF4s0tvB6NCl6vjQ38/Crldt23Fz3zbgrvV4ZyzWbJi8xjpB/6S1JRFEVRYtBJUlEURVFi0ElSURRFUWLoq0m2mqhNNJvsl0KvzXYN15ldyg84VCxCPDKC68J7549AvLK2AvHWxhbEY+OoKZZrqNWcXURN8fw6rqN3ly9DHCSwvT75RMtpXNffFqrXSH43v0b5ATu4faOH6/idMsaG6iROki5x8RJqSdEO6hod0pKySfyb6MFp9Kk6O3i/BwV7ztiDxT5J9v11u3gddudixevCeSW5jmrPZ99k//qUSdofNVds0iS5vezLZH1oh7RzPv4jj7wd4vvuexD3Rx6/u44ch/j6day199zzz0BcLuM4ZF9lmzx2Q+RZcxOoD6WS6BkcBIuLeM7jU+jZZt9ikWrb2iTnt2qoydkpvKeOg/eQ6z2yR523+z72mSrlI7bp/Yighw1cXrkG8dI1jNlb23oOv7+zWYF4OI1e3ve8/X0QZ1L4HsDlK3i9F6/is2zl+gLGm+iDLD//MsSlPPaxxg5ej1IJfZ5FyhveD/0lqSiKoigx6CSpKIqiKDHoJKkoiqIoMfTVJLsd1Ny8Lmo1Hco1yjk2yaoj2zuYT7Dn4f6GhzG3q6GF/rNnXoA4tLB9Zy5jjbIvPo71E1dpndpOo04wf3AvxOMzqNkl6XxHSCtyyEfpNCsQu0k83+fXsZ5jsoe3Y5z8fKem0Ru0msLrGRbxgtczqGNkk6hTzKdQGyrt9O0Od4zRkVGIuf4g+xI57yXnqeRcq6xh2lQf0oo4t2gJYs4hzL5F1iRZ8+xSHdBKpULtw+OnM+j54vby+RZIA1xewpy9nPuVVdOZ2VmIM5TT+A//8Pch5ly5Rco1y3VX2d+864YMgMVFHEu5PD6LbKpVu76M+r1L3k+/i32C+2yC3n/ge8i+SK5pur2DunCTdWCqd5mm/U9N4RiLQtTdL57HZ1NtB99HefThd0E8OoS5abMuHv+hh1Anf893YHtqdXx293p4Plukgz/++Fch/v3f/R2I19bwfZXrpLl60at/1ukvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaHvwqzr4jqzMaTp2eTnMuiXqtZwndnz0P9WyaDWVKlhfr6QvEKrq+iVWV7DdeZLq6hNVcsYHx9HbWlhGXUIH6UbKYygdtMqo/eJc3Jmc6i15HKUP3AENcCcj16poIHXx5AWlS9Rrluq29duU47PBOWfpLudo5p244kZeT3Q9VCP6FJeSYt8hrZD1z1fhJg1wiT5HIvk321TLbytLapLuol6FGtwzRaOk1WqY9rr4X32qPbgUAH355EHjutN7tvL+hJur9VwnLGG67rYMXwf9bLRUfQzv/e9H4D4scc+D3GDct1yrtdyHdvTauL1HgQN8oDvbOPYbNdwu+Whjjo/Pg1xhvR+zr/L9RMLQ+i7PHAA6yc2m5TLNY37TyQxtkhDbTZIBy+T5ke+x/IO3sMPvP27IH77fY9AfP4Mnk+3ifd46coliMenUfceyuH5p3PY5w4cOgbxXYfR25uiuejJL/8xbk+R77P+6vuc/pJUFEVRlBh0klQURVGUGHSSVBRFUZQY+mqSllWE2Fi4Dl+usX8N/VL1KmqStoPaiuOg1mPZVyFuUO7SkHyKtS2Mq4L5FvftOwHxX/vhH4D4Dz6PWsp6Bdfhk1SHsBlhe+oBrmtnKX9gy9D1IL+fofyIPYvqElpcnxLjDl2flkW5aSlpqE91B9m/Vxoj/9qAqFIdT/aQOS7qD4Y0ymYTr3u5jNc9m0X9gz/vU+5Yzo3qUy099mRxLlbW/DgvJvuDC6RJssfOdfD8R0cxL6VPeUY3NlBDnZjA7bt9k7i9VsV4pIQaKOeKfeyxP4H42nXMy8nn43uD90lepZqam2voyws9vOfjeXy/we1hH6n52KdW25jHeX0da7/u2YMeaIe8sFwjletF8vsInofPrmHS3cdIZz77MnrKx4fRI/6Bd6EmWVtHzTai9wZcGpNLi5gnO5XBPmy7+OzuUZ+zSFONIrze3/99n4Q4EWF7Llw4DfFmuSKvFv0lqSiKoigx6CSpKIqiKDHoJKkoiqIoMfTVJJttXIffIC/NVhm1jjR5UfwuendavK4sVH8xRO0oohyPRdJiRuh49TquQ29soPHxT77yJYg3q6hBblXRixQkKhAPZ1ArGqGr5wbYPumRF4d8j/uPHoV4p0L+wApdD9Ip8pPoY7UDXKffqqBu4LAfj65voYka8qAYHUE9hAtIJpKJfpt3+Sq9CM+7Ucf7XK+xVk7XOY85dIeH0a9q26i/sH+2R9p2u43bWy3OfUp6F+We9SgvaIb6VRiipsp5MS0b9Z5OBzXaZIpyxVp4Pdod7DfjE6hvvec974H4yae+BvGFC5QXlPrpIMjQWKhXsU175vZBPDmKfbTaxGeJRbV2kxbnUcbjeZT/lnPFFos41h2X609iH26QT9Hzsc9vbaEmmiGd/IPv/SDE0+N4j5ubeH3CkPJ8ky80V0Qdu9vFuaXVomeljX04RfU72zTGXMrjfe8990L8wnNPQ7y2huffD/0lqSiKoigx6CSpKIqiKDHoJKkoiqIoMfTVJNsd1FK65L3hGmYJF7Wa+VnMr9eg+orZHK6rX11AraJaw3Xte9/6AMQHpjHX6Ke+9HWIl1rY3i89/QzEvQi1pK0arovvc1CLShfRj9ajOniNNh4vQ1pViWrU7Z3BfI+JFGpH6+TNshNcNxD3F5K/LyKN0fJRvMsXcJ3fClDXGBQTE3hfObdpLo++vgrpR1yekDVFzn2ao7yRrEl2Otgvki7eBz5eNoN+2Wazv+bJmt8M9esKadOUsldmZ/Hz1Sp+vtFAvaxL+le9jhpkKoXX13Up922SNFJ6LmSzeL1nZ+ch3t6uQOzYVHh2ABjBseO63Eewjfks1Xskb+zyCuaZLnvYB4IQP1+v4jVcX8O80hOk+7L3ttlA7yn3qQ3yZXK9yKkR3P/xI5grtdum2sE2PkvGRllHx+tjJfD9FLLiSpc88ELtF8pH3Gri3NNrVyC+dhU990nyVieS7A2OR39JKoqiKEoMOkkqiqIoSgw6SSqKoihKDP3rSTqoPZRGUANrtFB7yRdwXXpuP9ZEiyj36bG7D0H827/9mxC3PKzDNzFehHh2CuOhDJ7Ojofr9JR6VhKU63Q0hxpi1EZNb3UbdYaWxf49XOd2yAvVbeLfJENXyOvTw/ZPulTfsYr347CH17ueoHqSEeoaU9Te3BDqBH4d9z8oEqS9ptN4XTNZqrNZ4Fys6AkTynnbaHBdVLzvmQxeF6/LPke8zqxxcm5SztW6Z88eiO89dT/E01Rrb30d/chcH5LrZXZI3+n1UNPl71er2D6+/i7pOZxLd2cbx8n8PGqQyQTeL0N/m/P9HQQ7W6Rrk/l2+RqOpXSEY2WIzqFerUCcGcZr0GAvbRfj9bUViFcpP2+BnrU90u3TVM8yS8+moRx6vg8fOILtpe93qL1CfdyhZ8sO1VxN5XDuyBm8fkGEz+Ik1cvsUZ/h67WzifmTry1iLt75ecyN27Fe/bNOf0kqiqIoSgw6SSqKoihKDDpJKoqiKEoMfTXJq5evQewmcR3a69A6sYfazvom1mSb34NaSxjh4e+9/2GIJyfR2+MmcZ26XKtAvLKKGuZqBduXcFC7CR1cB2+0cF0/XMd17+wW14MkjY/8YawLbNDx02nU0oT8c4a8UMtp1H7mhvB49RpqcaMF1DRnknh/unX0z3kN1vIGw9LSIsTsK+TcrTbV2ttd/xG1cNbstrexn7KvkjW5hEMeMNIk2de5vYP7f/Ah1CAnJjAP6PIy6lGTk7h9agpr712/juN0fQPHzRadX7RLM8X2FwoYsw/0+jXU5oeGyK8b4hcKQ9jPuf5lFFLy3QFQymNu0X0HDkK8fw++X7F/BnXlVhl17tBQvlsLdewXTz8PcT6H73eUCug5b1TwWVSroHe3SPUic1Qj1LVRg7Po91GKfI1hD/tIq47n12mjT1EoX3BItW4tNkaSBulRPuMm+SCzpGlaNMa9Ll6PhIPnNz6FuW/LPo7RfugvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJhmRZvjK2fMQ5yj3ajKJ69Bf+TLW8EpnXoB4YhJ1AF5XT7qoZfghakVt8n/1SNuYmMB1/THKTxg4+PkXLqK2s7GJOkLexzjo4Lp5JOgNchP4NwhrZ7ZLdfsSeL3n5lHD3bMPdRCfjl+9hFrRyYP4/Zki5zxF7csnH+ugaHVQK2VRLOGhfmJI/0mn8boODaEnbHi4SLtH/YXrN7LPMUk+wgRpllw/cor0kLlZ9GydP38B4uvX0ZPXblPuWNJkl5ex37IPdKiA58/1Lnd2yrQd+3maPGvbO+gpXF9HzbNG2vhwCfU228JxEODlHwhdqgW7skZ5jy3UeddXtyH2qR7iUBGfXRNUT/GufehL3NrB/a0t4TVtDKGGlyHd3KcxUqlUIE6RL7CQxHuSpj7dofy+zTrur7aN16NBmqXv4bNkcxPPr0QaqUs1USv0vkmnjc8EQ5rm9ia+x8DPgDHS/fcIfr8f+ktSURRFUWLQSVJRFEVRYtBJUlEURVFi6KtJViq4DjxKmt48aWZXruK68MICamTdDmoh5QquuycpJ+TYKO5/48FT+HkbtZPtGq6j75skr1AXj5dycN16egjXyetl1FaSZOfyhDRG8uZk0pQLlrxCTboefgfFmZRQvkbydW6UMV/hVhm1ovIa+jInhjF2i+g9cgXzQQ6KTgf1DfabjpSKENt0HyPWMBOoN2VI/+DafA3yi25tbUIcBngfDOWG5X48Oop61LXrOC6eeQbrnG5v431cW0Otu1BAPSqRwH7WIc+YQzmYOTcra66pJG73yVOWoPqSy0sLELOGGglqsKUSvisgVNd1EHQsPKdGGT3RG5SLNeXiPQ/pfYUE6dyjBdQAxyfQ6zo/gdfIC7AP93qkOe5ge165gO+LZMj3ODmEunhE+Yhb1OebTeyDtR3UydeW8HidLmqQXY/ef2jj9XKHihDPTuKzziff5fXLZ7E92zgmeqSBJpL4LNupYh9uUn3MfugvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJlkjb8z8HGqEKyurEG9soHYzO4O5QxNpPNzzz6KPslFHHWCJ1s0/S+viLkkZGdJMsxuoQY6mcV0/izKEJA3qCjmb6wbiOrZxcB08Iu+OHaEu4JD/LlVEbSyVxL9ZNlcWIJ4gzXSYNNszTdTyFqhO4PEi1YxjH6dPF2RADJfwPlrUTYdL6Hky5LtjnyNrdEJasu+HFLO+gv2gQr5CrkfJ9STZh9looNZ/7fpViI3B+xJskp7SxHHS83GcsAZZKqEmagx+nvWkQgE13BHymY6QdG2RB6/dQX0rncZxsL1NGi/leh0EnOvTIY2yQDVMk0nsk2GA9yggjbJLnuzVGvahTAf76Ngw3rOMRbVyqf5lukm1cyktdJM80GTtlcXFixD3tvHZbjz0ZG+RJhiFOAYSCbp+GXz/IUU+ybBDY66J129nG59tjRrlaiUNlve/Q+/X1Fs4BvqhvyQVRVEUJQadJBVFURQlBp0kFUVRFCWGvprkW996H8RjY6gVdUkz9MhPdXVxAeL5IfQCZbK4jtwLUPNLpnCdu1rFdXyX/FoWeWWWNvDzVTpeuosa4tY2fj4kXcIx2L4wwHXynkf1NtvY/my+CPHoFHqDOqQ1lTdQg82cPIwxSTldyh+51cL2Vau4Lv/wsaP4/TZrd4Ph3lMPQOx55EskzS6i2nKtFuon7At0yBdYreB998kHyblGOccw6y/su+xRjmH2cd5zz0mIeVyxxsn1KwNq7672UMy+UYvqSXa6qAeZGuXNFK7fiefj0ssC7DOtc55Pf/A5gzuk57OXtkdtbHs4VtIpelYk8ftOmr2qeA/S5K21XbwnLXo25S3s8/kJ9EF6CTxel34OWfTs8EgT3SEfYdhFT/f2FranSDVF81n0NgvV0pUe6tY763j9L1/CfMbNNh6vG2J7my3SHDv4+WQHdXWb8jf3Q39JKoqiKEoMOkkqiqIoSgw6SSqKoihKDH01yY9+70cgZq2E7F9SpRpgqxtYc6xex3XogPITupSD01A9S8em7aRN+eTPatUoJ2cT18WjDTwBkrYk8FHb6jZQM+xQ3T5jUIvhuoOWjdpRLn8A4yTlxLyK/rkrC3g9x8bw80nSUIMWnt+zT74E8d17DkJcp5ypg4I1xSTpO50OamZJqnfImhxrgt027580yxD7za79dXF/SbpvrPFduID6ytws+luzGczrGUXYjzc30ZPGmmSK9Kz1dewnbEN0qK4pa4RBgPrb2CjqXYU8trfdRr2q66GeNT6O389k8H5x/ctBYFF9wS55a5tcO5bGekjPIqvLOnER4grlSjXUZ9Lk9Z2cxvdB6luYt7nl4zW0fdI8PdyfR75EquAqVg5zrw6l0Bw7PoZjJJvG43XofYhWtwJxFS+ntKgP7ZTx+zky55bJa3z6DOZ2NVSk9Pipe3B/Q6SZ9kF/SSqKoihKDDpJKoqiKEoMOkkqiqIoSgx9Ncku+aXa5J1ZWVmBOKTcpXv27IV4a2sbYvY5ptOYcNCxcXsmTf4tC4/XIq+M10INMbA4NymKkFw3L/Bx/z1aN7fI7+YmyB/XxYX3MuVDtELUeE/cfRzi61Sfc6dJfsEC6gpODq9fx8PrceXqAsSXFq5AvFnHfJCoSN85Tr+M2unU1DTEjoP6R4I1StIQA9IkO9SP83msz8haeY98mhnqp+zz41yuRlhDZI0V+0Gjjv0sTZor+zDZJ1kkD1gQUj+m6xHWUL9pkmewNIwes+0d9D222qivDd/Gg9Zs4rjg9gyCHTon7mM5yl+bIR046uE1TqVQAxylPlYYIc94BveXS+LxNxaxBuml6wsQC9UUNRH2uVIONb2Ui31wh2uYbqAOPjOCuWTnJ7EmaId06EYdnz090gjbDdR8q3X8/v33vQ3i0Smsv7nyR38I8bWr2N4k5S8+eAyf9SNJ1Fz7ob8kFUVRFCUGnSQVRVEUJQadJBVFURQlhr6aJPuxHAc/nk7junZEOSb9HmpFafLSjI7iOnm1SnXy6Ps+1WzLki6QyWAuVIeKpnVIY2ywV4mMn0nSGHPJIsQp8sexZsuabreD2y+ffRbi+RHMfzg8gsdbreD1kSHUduwc3o/uJuosVojnmy9g+4cn0Lc5KNgnefHieYgPHDgEcYJ8jmGIGuHKCubAzWVJ+3aLEA8Noa+vVkONzqP7yqyuok/Rp37Xphy5rGmyZmc7/bVzzgXreVQrj/4UZs2U6z1ybcQU5SX1fTxeGGK/5HHFPkxuL+emHQT8bEmkKL+tYJtzrEmSF9Ul32NImpxF6Wpr2xWIF7axFi49iiUzjjVVQ/KUN1v4rNn08Pwmyes6u28PxFfP45h75Qp6fTe2KBdqgGMiQfmR2/TegEOaYCKN8dPPPwfxo8OPQvzhD31QEBwjTz71JMQWtSdov/oapvpLUlEURVFi0ElSURRFUWLQSVJRFEVRYuirSXKORdYy9u7dC3G9gVrK9RX0BZYr6MVhvxfnwNyd65XyDdZwnZ1rwOWohlkxj1pU6KN20yRdwnXwb4iJUcyfWKM6e+xXM7ROniQv08q1SxA//SRpqILt32mizrArvyLVy/Q81IIO7UEdI4rwfCfGUIsbFCHd5wTVxrMM6jvDRfSgcT894KLe0vW4XiNeB69HdTUNa2jYPr7v3E+5X9ZqqOGxZscaYY/9uj1sP/s8LTp/9knaFo47j/QqztHcpH7Xo7qtEmE/Hx5GbZ01UtYs+d2HQTBKYzudwmdfJkNjjXyTDuVajWjs17t4zeqr5DVt8TXB9mVprKfp/YNUAts7jK97yOYm6uRp0gCPHroL4pWrqOPvbF+DuFTE9z8uXMY809sbeH4heY/vPoG5VPdPoG/0hZdehNh+HDXGI8eOQHz/w/dDXO/is7kwRGOEExr3YfC9U1EURVFep+gkqSiKoigx6CSpKIqiKDH01SRZO2B/U5LyCx49iuvanO9wbQ3XxVlLYW1maWkZG2vj8V588WWIz5x+BeLNTfQaOU7/dWiLfJKB19/nyDknR0YwnyFrYztlzF3bIi3sypXL2CDKbWsX8HpmBbWzIQuPxz7LyQnMf9ho4fk1Xwd1/UR2a3pd0nM2NlDrbpE/9ba5VEnwYY2S+5lLflsqO7orDignL90WEdJULRv7ZZvqXRoqdJpM9ffncv1LzpFsk2fMJe1+ehrrXY6UUGNst/D8ul0c1w7lzWTNkd914HcRBsHB/fjs4jaxl9Nx8aZ26VnB72dYpFnyNSgUUURkTZJr3dr0PgeVsxTHwi+kyPc5TPl4q7UyxNcWMa9zRLrz9Nw8xC3K3bpexv2F1OfTWfRp5ofw/OvkRV5axzGfH8M+ubaBecQvX8Nn6aETqGGWxvH4/dBfkoqiKIoSg06SiqIoihKDTpKKoiiKEkNfTZK1HfaD8XbWesbHsAYZaxu7tAnSWti/1Wz092utrmxAvLGONcbalBOUtSTOv8haUESapUs6w8kTJyDeoXX57TL6RHPDeH3alJvVtfB4h/fPQHzP3ZhrNWlQJ8lm0Us1P4daE/vx2pQzdFAUCqhV93zSTluondqUUzjcdZ9Qj4koDyff916PNUKs75jhnMW0A9s2FON9SSax3xSLWC8zm0UPG4+DXXoZ1+rroFZt2Xi8RAKvR7uNGmOStifIg2cZ0reKOK5Zv2Ptnp8bLtVuHASOzXmg8R6wp7tLGpzncU1T/HyC7lkuk6TteI9syv3a6WAfdBzWHPHz3Mf30bNjrISaZJnyDYeC58O6upPE8zt+7934gSS278IZzAXbJS9yjd6HqLVwe4p091QO+2RnGdtbof2NTmMfndmL598P/SWpKIqiKDHoJKkoiqIoMegkqSiKoigx9NUkWWthjY79T7zdIy2CRcCIckoGpL1YdPyEi8d7x9sfhrhUxHXmr/3pn0J87jz6KNmX2SOtxKLzscl7lKDcrs0G5gus7qBPM6Kcm3v3Y77CLTo/j7SlQ/OoXdmUe/biJfQGPfwwXp/REvo4Oadml67/oCgOo4eJteBKtQIx50YdHsbzDClPI/t/E+T3DUjj2517FLVji7TgkRHMazk5gfetQ/eVNTtjWMPEuFFv0nY8/0waxwHrYRZppqxR8rsC2Qzej24br1/WxetTHEU/rli4f49yw3J9zUHAtWebpIkNFTHvc4r0/E4X78n0NGqA+TxqnD7dc9bRKxV8nyFDGmCX+pBH+YTTlMf5wD70Nfao3mSTxkSLts/OHYQ4P4Z9bHQeNb9Tj2Ju1s/95mchfvKPMRfr+YuYG7YT9L8+pRJez2ye3sfIocY7NV2EOJPH7f3QX5KKoiiKEoNOkoqiKIoSg06SiqIoihJDX02S/U7sb2LNcldduF2aIq7725RwMCQfJu8vTdpKZhprwI2PYf6/o8cOQfz157BG2UsvvQRxrYYa5eLCAsQe5RMcG0Wf47PPPA0x+0rZlzhCfsCgi16oy5exJtv6KuYvTMyg75GsUbv9fKQBc37H0Lz6dfpvJdeuX4eYPWoO+SJ90hA7HdYQUS/a2kL/rLGon5JGxxqmT1pwFHFyVmxfLov6TaGAGh/vf2cb/bRej+tfUh5R0o+iCMdNgvJ2Fous+eK4TqVxnHEu3Yhy07o21UqsVyC2HNSPeqR98/0ZBPkijk32QQYRjs0c5Z1O51AHz1O9x4D6jJvCscc6dD7C7/P7E9k868Z4jQ35KDe20QeZIONjgjTMiVnUVPcexvcnciP4rBiewePnCtiHjpxETfO5rz4P8eYaarD5UbyeGfLQs05/9Cjuv97EMcSecfZO90N/SSqKoihKDDpJKoqiKEoMOkkqiqIoSgyvySfJsOa22+/VP4cl7/92vszbtYf3Pzk5CfGj70At5sTxYxC3KHfp6dNnIH7mmWcgZs10fBz9cawjDJEW5FDu19HRMYiLlBNzbh69TkM51EUO7N2H7ZlAzZbvz646i1y0bkBkM6jhsdTNBD72i3IZfYxhgNs5566bwPNuNlF/CgLu16THFEk/yeB9qVbRP8v94na5TtmDx/fJUJ5Mv4fjMpXGYc71OFfXsG5rrYp6jk25X8Me5bJ1UKuvdVFf6gX8rgLXx0S9aBBMTKEPslLFc+r56EtsdfB9gST5GCtNvMetNu4vn8fjsZc3pLzNmRJeo3wWvz9ULELskmbXJm9rirzBW5fwfDwHz/e+t2M9xqFx0plt1HBbTdRgPeoz6Ty9n4KldqVLuWrF4PXpkNeWaxcfO4m5ZCPSYFudV+/N1V+SiqIoihKDTpKKoiiKEoNOkoqiKIoSQ19NkrmdRsg+ylQKvTevVZPk/d0uV+wuTY3y/YmgVsO5XkdHqObYDOag3Ld3L8SPPfYFiPn88qQZ+qRtcd3ETKa/hlgqoQ90zwx6l0rD2H6GNeTbabyDo//fbqwpZrN4HRMJ8rx18DoOl4oQFwqor1QqFYg512q3S/srol9WBPuB7+Pnt8kHyZok92PWw1wX98+1Bn0f73OrjfpYinyTlqE6sQHnlqX6lAbHpZDfNkOeu1od9TiuO+s4g/9bfZg0v0YbzzGRony69GxzKe9ymzTI0TEc29ynyjuo4w7Rs4mfpW4Cj+emSIfO4Hauv2iRBtrsYJ88Tr7G/Agev9JEEbHTo3qaLl7PBNW7PHz3XohrNdTdNzdw/5Uq7m9zB32fs8Po6yyO47Nyu8o6uWqSiqIoivJNo5OkoiiKosSgk6SiKIqixPCaNEnWThjWUjjH5u0+z5rj7XLDvlafpetwrlg8n5Bym1qUDPXo0bsgHhlBLeq5556D+MKFCxCzBnnq1CmIOYdnmnKvTk2hRpolf5l/m3qQfD1Yo7zd/b1TzM3tgbjVwuvS6aAni+s9ug7meRwiD1WXcvA26vj9hIsetFwW9Y0e5VI1hvUoag/licxmUQMdGUGf5a5cruX+njzPw/MhS9iu3KxdD+97IsG5cSmvZwKvp0d5TbuUl9RO4vUokoePNdN8Hu/PIBgbx3vQ8chXmKJ8vJQ7lXVhrj1bIN18awtrzSYMXxPsg1wzNUnPhnQONcMmeWvLZTyeTa8jTIzi8eancQx26Z4ZG4+XS+D16LFuX8J7/ODbTkHMtW5fOYPPzj0H8f2LAmmkgUGNMZHF+1VuoObaaZMPsw/6S1JRFEVRYtBJUlEURVFi0ElSURRFUWLoKxqyZsXctp4kwb5H5na5Q2+Xe3SX34w0RYvayxqkTYcP6PNcV2+E6km+853vhPjkyZN4fNIppqYx1yt7oRjW3kKPrmfY3/d4O83Rvo2GfKeYpVp2jQbmPuV+4JEWy/2SfXi9Hp5nz8PPD9N9ZY2OfYfc720b9x9FrH3jfY44byfdp3Sa9B7ypHl0Pbhfp5OkZ1HezmoF9auEi5ppknynPA4sIZ9kCr9foNy2bM/t9fo/Z+4E+RxqcjPT0xAnEujNTSYxZl240UafX62LurIfoeaZzeA1HKb6li7dM0N92qM+E5FXt0f5jbuU+3RqHsdckjTQDunsDvWJVJrfj8A+FdnYR9MFPJ/DDxyGeP8DByDOU/1Ol+4He6e5T/l0fTo+3q9+6C9JRVEURYlBJ0lFURRFiUEnSUVRFEWJoa8IxRri7XyIyST6qW6nUTK38z3eLsflLt+l4VyxfETORUtbKWZtKQhpnZ7yN87Moa5hODct+e160j/HpU1/0+RonT4iXaXVQi8Q54sMWXOV14dP8sknn4DY0I1IpZIUkz6SQg0vTX7S0gh6tgIfz3ttbQVivo58/HodNdNaDetZsi8yRzl9NzdRr2JYD0uRRsnnz/UvM6QX2Tbl/STN1TLkm3Rx/wUXz9/vYL/NUD1QrkfJMT02BkLDw3tokddzl++Q6iVa9OzZrpAvr4N9QtirauE14VymtkseaAc/z49yfv9hOIe6cJjlPM4Yd8hL7PBcYPD821V6X4LzZlO+YdbtjYVjPJPG80snsA+bsL8OHtKY7nr07Otge/uhvyQVRVEUJQadJBVFURQlBp0kFUVRFCWGb8onyd6U15pb9XbczjfJminnig1JY2PtiZvDmueu+otkpGSfJrc2oJyW3L4U5fTk43Eu1t3bOUco11nEdfo0aVdd8tc1upiTc1C0Ka9iEGI7fR9FrEYT9SRDHrHtbcxbyffhdto25xp1fNw+Moq5XVmbZv2F+81eqlPKx++Q5hcJj0vW6sknSsfn69ts4n1PJ1HbzlItQoc0UitNmnEW9aMu9dMO6UF8PQbBdm0DYs5X22riNYv4fQbyutoOjrVSkd4f6OKzK5/mfMH4+RZphAH1yaTFvkgcM9znLeqjOzWst+gH9H4HPfsM6disa7N319Cz1Ungs09IYzRdbJ/Fujadb7OO96fn8fs0pOn2+ue5hmO/6k8qiqIoypsMnSQVRVEUJQadJBVFURQlhr6a5O18iqyB8ed3aXq30Sh3aXy30Yp2aXQU26QbcPs5B+juHJzks7T6+yp3XS/SiiKuX7lLxUS4vRyz75H9fLUaanUJ0gE432I68TowrIlIwuV6iahh7c6dyj4/9pDhfZ0YRx8f+3u53qJH+oZLeSunprDW3dzcXmwv3Rf2CXKtQO6H7Dlr0/5YM2WtnnO95qneJtff5By+No3zboeuh4P92OvhdkOPmWSi/zgbBCF5S7sdvOapJN7zfL4Ise/z+w90T8gTnUzjNRnO4f64zw4NT0DM+YR7DewTnLeaPdC1Bvo2k4LPhnwWNdIoYp0e95/LUU1Q7NLiU59IkIe918H9uQ7q4LkMvl/h03sKtS6ej+OSTk46eibBz4h49JekoiiKosSgk6SiKIqixKCTpKIoiqLEYF6rd1FRFEVR3izoL0lFURRFiUEnSUVRFEWJQSdJRVEURYlBJ0lFURRFiUEnSUVRFEWJQSdJRVEURYlBJ0lFURRFiUEnSUVRFEWJQSdJRVEURYlBJ8lvEmNMZIw5OOh2KIqivBqMMf/BGPMPBt2ObxfeFJOkMWbBGPOeQbdDURRF+fbiTTFJ9sMY07empqK83tA+qyh3jjf8JGmM+VURmReRzxpjGsaYn7m5RPrjxphrIvIFY8yjxpgl+t5//fVpjLGNMT9njLlsjKkbY541xsx9g2O93Rhz3Rjzrjtycsq3JcaYOWPMZ4wxm8aYbWPMLxtjDhhjvnAz3jLG/JoxpnjLdxaMMT9rjHlJRJo6USqvFmPMvcaY524+u35LRFK3bPsrxphLxpgdY8zvGWOmb9n2PmPMeWNM1RjzL40xXzbG/MRATmKAvOEnySiKflhEronIR6IoyonIb9/c9E4ROSoi738Vu/mbIvJJEfmgiBRE5MdEBEqBG2PeLyK/ISIfj6Loi38+rVfeaBhjbBH5nIgsisheEZkRkd8UESMivygi03KjX86JyC/Q1z8pIh8SkWIURb4oym0wxiRE5HdF5FdFpCQinxKRj9/c9m650ee+X0Sm5Eaf/M2b20ZF5NMi8rdFZEREzovII3e29a8P3sx/jf5CFEVNERFjzO0++xMi8jNRFJ2/Gb9I2z8hIn9VRD4YRdHLf66tVN5oPCQ3JsK/dctE97Wb/71087+bxph/JiJ/j777S1EUXb8DbVTeODwsIq6I/PPoRl3ETxtj/ubNbT8kIr8SRdFzIiLGmL8tImVjzF4R+Q4RORNF0WdubvslEfn/3OnGvx54M0+Sr+VhMycil/ts/2kR+U86QSqvgjkRWeRfgsaYcRH5JRF5h4jk5cYqT5m+qxOk8lqZFpHlCAsHL96y7bk/+8coihrGmG25sboxLbf0tyiKIpak3iy84Zdbb/KNKkvf+m9NEcn8WXBzSWzslu3XReRAn/1/QkQ+aoz56W+ijcqbg+siMv8NNMVflBt98mQURQUR+YtyYwn2VrRCuvJaWRWRGYPLZfM3/7siInv+7B+NMVm5sbS6fPN7s7dsM7fGbybeLJPkuojs77P9goikjDEfMsa4IvLzIpK8Zfv/KSL/szHmkLnBSWPMyC3bV0TkO0Xkp4wxf/3Pu/HKG4qn5cYD6H8xxmSNMSljzNvkxq/HhohUjDEzIvK3BtlI5Q3DEyLiy41nk2OM+ZjcWPIXEfl1EflRY8wpY0xSRP6RiDwVRdGCiPy+iJwwxnz05h90Pykik3e++YPnzTJJ/qKI/LwxpiIi38cboyiqishflxuT4bLc+GV569LCP5MbL/z8sYjUROTfiUia9nFNbkyUP/tmfANMeXVEURSIyEdE5KDceKFsSUR+QET+JxG5T0SqcuMB9ZlBtVF54xBFkSciHxORvyw3lu9/QG72rSiKHhORvyMivyM3/nA7ICI/eHPbltxYIfsnIrItIsdE5Osi0r2jJ/A6wOBStaIoiqIgxhhLbvxB90Nvtrf33yy/JBVFUZTXgDHm/caY4s2l2J+TGxr5kwNu1h1HJ0lFURTlG/FWufFW/5bckAg+GkVRe7BNuvPocquiKIqixKC/JBVFURQlBp0kFUVRFCWGvhl3/rcvPw1rselUANutXhNiO8Td9bIJiBsmCXE6xO2+70Ec2OiltkJsX8LgUrFL6Sxt+hOgHWH7Qgs/ENLKM/8FEUbYAIv84L6P36jZ+H07wuuXoQMkHdxf1MPzCclLHpDV3Da4f0Ne9IDOz6cT5vR8/8ND9982X9+3gp/48R+Bhv30T/0kbC8VixBvbq1C/L//638J8drOeYhDynp6+MAJiF868xzEXgfS9Eoum4XYUD8SC+9jMoXbberX2Txud6h9Po2jqOtCHPawo62tbuL+HDye18G3+L0Wbp+en4bYrzQgftvxByEeve8IxOlEBuL1a9cgNiTxJOl6/sRf+1t3vN+99aFT0Khu2IPtuRG8p60y3qTsEN4jz8Nnme/h99MuPgvzQ9RHsuAwE4mwPWmD2x0fx34vQc9OM4T7C8O+cTpTwP0LtjeRxv07bgrigPbnU+zQmEm59CxP4v59H+NOF6/Hjfwvt+zfxvZ2vBrEtqC0+r/98m/E9jn9JakoiqIoMegkqSiKoigx6CSpKIqiKDH01SRdGzeXBNfZ2ysvQNyqoUaZP/4wxIkcHS7AdfSOi3O2RevWxsPPpyxcRs45uC4eBqgbVCLcboUdiF3BdW7fQp2hG+G6N656yy4RNLRI8xP8fmjw8xHpFEIaa0SaZESaLJ8Pa7IkqYrN+bMHokDupkdarE96S62OGlmO9JP3vecDEH/qd3cgrnuo2S1ew+IGbAXLF3D/nTZeZxbL3QT2804XL2zSRX2ogc2Tuw4eh3gkOwzxM2ceh9gPcnj8CPvtzk4V4lKhhN938Xzr9To2iPrdy6sXIU4+i5rwdz7wboiP7huH2HGxfQtLGzJoej1634LGXnkLn21eD++xlaNnF73/0CPdeGQMNcVEEq+x16T2JFGHjix8ljkh9rGEQZ03kR3BOEW6cBLvSejj8QIS8tMZPD8T4fl7XdTx8wncvyHdvtnBPhrQmHHoYWbo2Rn2+P0SfJZHpGma8NUXwNJfkoqiKIoSg06SiqIoihKDTpKKoiiKEkPfhVknJA2wiYXSl575fYi75XWI3QC1jeQclnR00rhOnhsZg9ikUGvxDfkULdTwQtIMO7wOTevYeYe8OgHuv0man09eHIt8jx55q3zyHUakoYYhary9NsZJOt6umrv0Jw5rkBYd35AouVuCfH2IkqxJsj91VzNJ37jv3vshXl5Zgfgzn/1ViOvVZdw9aXphgBrk7PRBiMtV7Peej3pMo4ka6mgJ9SInQR60HB5/afs6xJaN358qTUBc2I+a5isXX4K420WfZKGA46zRxfZOT89DfO9Db4G42cBxns+PQvzb//kPIX7f+1GznNxzSAZNOo3XYH4EvZ7ZYdQAO/R+xMQQbs+SBvfiAurgXgL7dKGYhzikZ1evSzp3E7+fSmN7PdLc0jZqjOkUbrdderZSn+z5uD0kj3UvwD4fCH4er4aIuNgHDT07kxY++9IOari5FLavhjZI8ejZmiHfqcvvFfRBf0kqiqIoSgw6SSqKoihKDDpJKoqiKEoM/X2SAa4bX3r2q/jlncsQD7fR6yJn/wDC5dMkJqVRSxk/cR/EyWHUNqZmULuI8phjshzgOnUkqBOMRrhOvW8I/W8JH1fOlxu4Tp4g36hNuVZr5G2qdHHdm32SuQT+jZIhb4+zyxcJofismZJkybqBxbbIXblrXx9l0zi3J+eVNHRd2h30sL340ssQLy9fhbjXxX7gONhvPA/vW+ThfRsdw37ZbKMes7NTwe9Te7co16zn4X1wDepLcwdRy+9soK+wOIza/vTcHMQLy5cgDklLt22bYhwH+/cegPi+kw9AfPpF1DyvXMXr/dTTL0B84tjdELsZ1t7vPOOkA5+YRY3y5KEZiMub+H7GUA4H1+b6FsTuPHpTXyxTPuA8PkvcfBH3dxmfxUI6eYp8jlES+5Dj4v67HfTGBi189ticKDuFOrVLuWPTLsaRw+9j0JhycAy6u3ylOKbp9sihg9gnn30Gx4RNia1LQt5fH+9fP/SXpKIoiqLEoJOkoiiKosSgk6SiKIqixNBXk8waXAe/uoY5G9NbuO6e6eI6d7J3FuLxPPq7Vihn5vlrqGVYWdQqUg+/DeLc3nshzk+iPyybRc0z55M/jNbl0wlsXzfB2hiuoxvSckwS/+aocsFHChOkQQ5T3UCHZAEhDZHSFYpHPtF6i3QA0qKCiHPLvj40ycDH62KRZyqinLxeG/WLjU30Pe6UMTnqxCj2i2YL9aEuaesWZek9fwH7dZOvM4nDQ6UixIGH5/fQ/divP/ah74P49LlzEC9dR19mlsYVlTIUn7T2dAr1qpaH4+KuI1gf8twFPH65XKH94/dPHj8K8d/9+b8BcaWC17tDvthB8NDsLMTJBGpwjXV8VjWWMM7sx/y0V9exD03PoQf8fXvxeJe28f2ORhmfTQ7lnc4O4+D36X2JBPkiI3r4dHvYB71wG/fXQ908Qc+e4Qx6Z12qJ7m5jd5kzqWa6uCYcrC5kkwXIQ7I97mxgX1umDThZIfeO+Bn9+weebXoL0lFURRFiUEnSUVRFEWJQSdJRVEURYmhryY5QfUf93zoHRB/+d98AeI05eMbLlKNMaqzd3UDtaRrZVw3LhZRqzATuHDdqqIuYK+egTg7ievOmSH0OjlZjK0Uajtp8uakSfsKqKZcnnJqFpO4PeAUpJSvkHUCm/x7Fmug5JMMqCZexP7CXblcyUdpvT7+ZoqoHVw3k6XaXKEI8bvf9X6IS8OTEJ85jf2k3kA9ZmMVPVdLK6gX7Z87hsfPY97Nr3zlMYibTWx/ljpWu436l+2QD5Rq840UpyA2EY6L02fw/HrUrwzVTXXI78uaa4c04pPHT0HsUj9tNjGRZlAiT57gc2LP3r0yaIZL6ItMoO1PmhV81mzvYB/ZewS9qfsOoa7bId366Dj6Jg9NUc1Nqv94/RpqhGukC1c7+Gxot/GaRyHek0ySRhHVsk1QzU+XfJClPNZEzVKebZs0x6ZHNVep/maDhHTj47M/T/tv7OCzOB3g+Y0XSXd30GO/TN7mfrw+noqKoiiK8jpEJ0lFURRFiUEnSUVRFEWJoX/u1hDX0Rst1G5KU7hu3l7Gz+dS6J2pNnHdvFZDjSydRk1vyEJ/m9lcgDjVQ10grKAWY23jOv/68D0Q3/PoD0PcdikXag/PxxH2LuG6txXhOnrGpvqRFFsBrvs3e7hu79G6vKF8ipy7NQxYe0K4viTXt7ScwefQFBHp0XlzfckU1boLQjwPhzxj995HOYGpn/k+armf/vSnIXZTRYinZzBncDKBOYDvOYX1FkfGhyG+egV9h4ZytW5uo//YJr3IovqZFvWLRhPHjeejXvPIW9CXefostme0iO1N5/H8Rkvo+WMfqmVje373c/8JYjeBGmqugOP0Qx/6mNxpTBrH+ijlw92p4j0J6Jq7Lmpm9+zD9yG2a6i5RT7ew7l9hyGePYa6t/MlrN07ksBnUS/E3zudLj4b6nXUvbNZev8ijTo39zlDvktL8HzS5Cm/q4TnE5IHu2Ph9xc3sI9ub2P9zYkMjpF6Dc8/X8D7VZzC9002NnH/nTYVoOyD/pJUFEVRlBh0klQURVGUGHSSVBRFUZQY+mqSdoCbo1YF4wbmkOxsUU7NLOUapXx6uTwnH0VdoES5TFt13H8YUO7VFHpnek3UUIM65lNc/wKuS+f3nYQ4M4I1yyIL8xW2DfrjLNIcc+x3I6NkSLlWqWyheFRAcog+70ZYI80PUVvrUv3KkBIkWlRvMhmxA3EwJKidrK0G1G6X/LdhgNvZBzg9jXrFzg72k5kp1BwnxlDvCOg+pLLkIRvB7xeHcbtFtfjGRlEP2n/gIMSLq6j5tWgcjU+gD3RiDcfl6uoCxF4PPWIpenfgAx/4MMRLy+jRKw6hR+7Z55+A2DI4TssV9EVmsqidt1uYs3kQRC72sp0G5TlOYw3R8Xn03bWTuH2SPr9/DnXXZoO8o6QpblzHa17dwfqHIXuJKf+uzfUmLRwTNo11mzzTEen0Ab3vkHSpFi15srP07C8OFSFud/DzBfKYbxoewzjmhsmn2Uugjr5KnnahZ3HCQg25H/pLUlEURVFi0ElSURRFUWLQSVJRFEVRYuirSXKy0V4VtQ6XanRNzKJ/6gppKZFDGtsIHj7t4brxCNUIc8lrE1JOyqqH6+huAtuf8q9DXN/AuoPbV/4Y4vYsen2y049CnJl4AOJ0FtfFXRvjmsF18CDCv1FSNuViJe2rFKJ2lLBQh2jR3ewY1PYC8kVGIWpHDl6+gfFDf+ETEDebVKeUEmv6XdQfuL6hCfHzNuk5jQpq1fMzmFfz/Dmso2pIf3n4LeiLfPrJZyDe3kY/79QUatssurLP06L7wn/ZZrNFiEtDOA6HCqhZZjL4+TDAcZ0gDZdzAg8V8frMzaGGurGO42ykhHlNWbvP51DbHwS9CDWxhQV8NpTIKzozfhfEL17CPjo2g2N1KIsaZa2Fvssu5TbduoB9rtfGPp0u4P6aLdQsvQ5plBa+32Hb5IOkXK0R6f6c59km3yR9XBqUq9Z2sD0u6eClCdTxLZp7KmX0TWZHcEyvtvG9gmQR54bJPOXpnkNNsx/6S1JRFEVRYtBJUlEURVFi0ElSURRFUWLon7vVx3Xz+hrW1cvl9tPeUGsIaJ3cBLg9SRqjS/kAfZQ0xS3gOrhDPsvkMK6zd6leY9TAdfKOj8dv0Lp/t3EBP38JdQpv8g/x+JNHIbYm74e4MIT5GKMUakdN8jVGFuYrdEhj9A3qCFxfsmCo/iT5KH3Swjzr9eGTXF7EflYYQp/iMOXVNJTLtNOp4A4NnnfCRT2kUsW8lo0GarXPv/AyxHN79kGcTOL+Hn4Y77vXxX5Vq6N+9fu/91mIV5dQ03vLg6cgfvABzEFcqeJAKQyhxndgH/p9j911HOKjR+6GOMG5VSlv5hce+zLEHdLTvvM7UVMOQuyHF87juLq+uCiDJiKf3fwseknDDo6tzDB+/vgoXvMi5et1qFZtu4s6dSfAPlInT7dFmqJLurxNuVwj0gxTaWyfS+3hPsw1SCkUSq8sXOXVpy/UBcfYEHmhHaqVWxpFX2nXwu3NFD7LJ/K4P6+L57tnTxHiItUW7of+klQURVGUGHSSVBRFUZQYdJJUFEVRlBj6+ySpPmO0iRplcfghiJ946SsQ17ep5hglJx2mdegRsq4UhnHd3JBkVtnEdfjOBmpPuRLuP4vL7tIlUc4jP51p4d8Q2RBzpdpUY665fAXiVgH9cr0CroOn9qG/Lrkf6x4mMqhZRuSz7NrY3gR5mdI9zs+It3ubNMoym50GxLETpyAOKBercJ3OXX/r4XkaqqOZIY2tR3kpaw28z5ksfj5foLyRpL+QjVKyWdRHej7pN0J+1Qzu3yGPnUXa9HAKfYunhlAzvesk6mvpNGr3mSz2qyjC8zmQxX5YaeP2RAr3t1NHz9pv/jbW53zlxdMQe5R39F//m38hd5rrOxWID+ZwbImPzxq/S9foLvRR2qQBbu2gbhxaeM8t6oMJyo0a0LMpIHNtRM9SQ15gQ7lPLdIEmUQCNdCQng0h+Rhtam+Xro/j4rOGa+XWacylDO5v7gB61iukce6sX4N4pIQ6vMU1aj18v6Qf+ktSURRFUWLQSVJRFEVRYtBJUlEURVFi6KtJrl94DOJkDdeZL6+jBnf3W78L4kvnVyC+/ixqEc0m+iibVC8yO4xeoQRpikWKeyiFSJtPb5TyDdr4N0LZw88P0Tp60MX2mQppO+QzTAiue3caGLe28PrVzz0P8fRe9K8Vp1BrulzD9n79KvoLk5Tz8/iJhyH2Z3DdvrzL+zQYjIsaYCKJekrPIw2QctL2fNJ3ElSok/B7qDcND6MmODGOmuDoCMYhabttypu5s4P9uNHBXLP3PPQeiNeq2E/PkY/Q66FHzOtiv3Pperk0DFzSjzZ2sK5qEGBHKGTw3YDZvejTzGZQv/r85z8H8bmzr0DcaeL5J9Ok/w2ANsneq3W8RktL2KcOpbDNox18FjTX8J532qgLu/hxseiap9KoE3fIU25IU7RdqhVLmp5NPkOLnn2cP5jzG7vUiWzSQP2Iav3SCyQRjVGuCdvz8P0VriGbp/cIbPLcpylfcSaD7fV8yu/s9ddkb0V/SSqKoihKDDpJKoqiKEoMOkkqiqIoSgx9NcnqBmpcVh21l8wwrisP7UOt4l3v+zGIrz+LvsGXvvT7EDcqqGHmp4oQd6qY73CrjHXwyP4l9QpqJWfWsf0dg+vg1hCue0+P4g7bpAukLNQlUk3UHVIdXKfPJSi3qoXeqfLqCxB7K6hFXaP8in9yGnWPr11B3+ZQEb1af/AHqDF/54/+dxCPHkPf66BIkF7S7qIGF5Ee4VLOYPbdcW5Xj/SPDF3XKnnmfuxHfxzi8UmsfddqoN4Rkj5j6HxSWcxLGdXwvgUhaoR3n0S9ZWMbPWKf/4PHIX7wQdSyx8bRR1ksYeydRm28VkPP2pkz+C5BbR7fJXj4QcxVu7OJ2z/4HtRcWQ0ydn/N+E7ghziWNrr47Di9tApxj+o5TlfIu9s8D7FL+YNHx/D77NUdKuJ2J0KNzyEfZreFz57bWeDZO8wqoB9gey32XZLm6XVxTEXk2WZNkrez7zRF3ts2PVubZg3iBOX19iNsf0i5ZS0Xfa390F+SiqIoihKDTpKKoiiKEoNOkoqiKIoSQ9+Fa99B7WJkCtfpnzqLGqJFNdeCLGp2e74DNcu5ExMQd5q4rl3Cw0nrOtb1u/wk5opdJl9mO0R/V8ug9rO+g+vgV69TAUuDl2dmCNe152bx49NFXBffM4R+O65J1+3g8S4vobY1NY/HM8O4/yvL+PmhNGprOQePN4ZlGKV3/SWI5w+hb/L1QjaFHcHvoebotbHf9Kh+o+NQndE2+iKF9Iss5e1MkCeu1yPNkb7Pw8pKof6xTL7HmTnUKMensN9w/ctch/JkdvB8qlXqR5Okaa6joTiTpLqklJvWIo9dl3K3utTv5vfOQ/zKS/guQookyDa1fxBwvtpeiD7FkRGsD1mr4DVerlQgnnKxDwYB9tFaBce+QxphoYSD1XbxHtkJ7NMW6e5iY7y7Uiz9PiLNMyJNko2LXohjoNvFe2ioNq6QhhlF9B5BG3V4J+K83xhvbqHnPEzg/kpZzJPdpVrBdoC6fj/0l6SiKIqixKCTpKIoiqLEoJOkoiiKosTQV5MMcqghvnjpSxCHeVw3HxmbhLgY4jqw1bwK8fb1S7j/F9D3uL6Mn99axPqW+yZRc3xuEf1dXdIZ9s3i6d5/F7a3/BSuU7cTuK794hJqSQvkjxvD5shbThQh3juKWk+tQf6/BOoCO3WM6+T/W62zFkb5HSPUBcYbpINcegritVYFd/fQd8sgaJPnyyK9JqQ8lzZ5sCTg2nGol9g26jupPN4XsnCJ41Keyh7qIxX2STrot/3U7/xbiMensaO8eAbv8/692O8KedTDlpcqEF++cBbilVXspxcvonh+8sghiF96CX2QV5fQjxyFeP7XF9BXubGNHsKZSWz/9ia21/Tw/kZm8HVMs9SGbQ81snwR72khhzryCnm8k3kcq0X6PVJr4dgf3pUPmHKhUv1GlzRIzs0q5GsMKL9ws0bPAvq+Q7lauaZrYFAT7bTx+wkH2xtQzVVJ43aPNE2HvMY2eY35/JfXcC7Jz7P3FnX51Q2cW9BZjOgvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJnn47R+EeH4vrpuXe6hJpnO4Tl0iDewrn8NcrZ/6DMbPv4zr3iFJTSQLyMQHUKvZaOI69+nrFYhfuYjf/+53oSa5tYM6wfhx1FZsqoPXoLhew+9bC6hLrFcw32ByB9fhky1cRzdUB7FJ6/ghHk5c8hO2y/j5DW8T4qMpXPdvVqkg54DwKFercJ5Jrn1Hekutink4Uz72KzeBHjib9J/qFmpy9TL6ayt030cmD0P8xBNfh/gZ8vOevAc/v7SMWvzlM9i+Rh37ydIS3SfSDDlP5emXsD2famG/rFdwfy06v30Hj0K8dxY1zmeWcGBdoXcT0ln0USZIEw581tbvPL7Ba5JJ4sMmcvGa9my8Zp0yaphlF3Or5rKUsZZ0bcfBRzHnNjVh/5g/b1O9yQ55Uf0mjhH+vEP5jE0SPfMNym3bbOD1ybn47M/kcEz7AX6/28Y+l3bwPYF2gO0PQzze5BDmU66V8f4sbmAf7Qm+v/JeiUd/SSqKoihKDDpJKoqiKEoMOkkqiqIoSgx9NcleoQjxnzy7BPFLr3wV4v/h7x6DeHMbtZ0XT6OX6J6P/CDEk2/D4//Rf/kSxI1yBWLbRb8VWZvEtnFdveWgDrBWQy2k3kRt6sWvfBFilsYi9iXS3xzNDra3kcPtR6le5Q7lFN1cRW9RlzTJ/QdxXX8kjedzzwzVnCOtaYpywabJmzQoul2ujYf0aHu9jFprlzXJNPoSG+uY9/HKuQsQVzy8zpc/h3U4v+cHPglxNl+EeN8+zF06ObcH4meewRzEloMaZLmK97FLuWmtgGvzUa3CUXx3YHwMtfvRcdQMZ6bnIJ6dRi1+dBQ9ZkPDmIs2TVp4fgi3FwqoZ/3tn/0fIX7xKfTrDoKdHml65OtrkHc3FPRUF9L4KG22cCz5wzgWUzZqoJQ6VXoeft8lzbJH7bNdHMs2fb5NuWXDHmqEtqHcrYJjxjfY/jLloa7R+yDb9P5DfgifdU4Dn4WdDj6bogL2oS1q/wsX0as7M4I6eaONmuRGGTXSYFe+5Xj0l6SiKIqixKCTpKIoiqLEoJOkoiiKosTQV5OsNXBd+POfxxyPPaqJ9uRjT0J87CT6qw6+7f0Qn/rI90DcrOM6Pdm55NrzL+I/uLiOXeviurhvKJ9gRPn/bKobmMTL4XTIl0jr9pxykmu2BSRi2llc129b6NW5654ixDNbuD2ZQt3hLe/Edf7rV/CC3TOLOgpnM2w10f9nBYPPoSkisnHpPMScK7VBmmOX6kdyrTuLfHj5EfT33vPwoxA7edTQIhomq6uot/zTf/ovIJ6eRs1v7+xeiI8dQp/k2CjmAZ2ZQU1wanIK4mwWPXyJBGqC8/N4vHweNU/W1oXGieNgvL2N+o6bwON7dH8iuv71bXxOTI2hxjkyhuc3CPIpfJas71Du0S6eU8bBexyQxtf1UDNsk4aXohqcnS5+PtlCzTM5jDqzRblM2238fJM86j16OtmUd7se4D2v1/FZ09rEZ0U+i/cwylH+4wr2yUaDzof6kM3P4jT22YVNHHMXl69DvLOD96/n4f3oWfis5GdhP/SXpKIoiqLEoJOkoiiKosSgk6SiKIqixNC/nmQSNbTMMK5DJ33Uhi49/wTEI3vRL5aZQK3FT+K6s0f59tIuah3FHIopXCOtS94iQ76/jI3ft+j7huoSWiQ6Jin9Iklhu+ritUirWSKvT6eB2x+87wjER/bgOn4mwnX5KRt9le1h1Dl80iE65P9rR3i+iV2q6mC4eu4cxDbVc+z6qN8YQ7XzKI/l1J4DED/0rg9DPD63D+JUgvQL8pydPYs+x/e/930Qh5Rc9hMf/xjE2Rx60BwXjyd0X3p0vvU6GoIdG9vX8bBfV1aw33Q6rJ/RuwBN7HcbGzgu8znUk8Ym8LkQUr9ffg5zx+6jWoV/0sHnyCBIk0g1M44aoEca41CyCLFPtWuFfJCFHOrcGbrlrS3M62wL7i+dwWcl+yqvr6KHvUZ9oBtSsVvSobmG6uV19Lg7ghrjniPoic+Qjr1xFk8wiNjbi3E6Q9eHvMfNLewj7TI+jANqX32HfK1JvB7pNN2APugvSUVRFEWJQSdJRVEURYlBJ0lFURRFiaGvJlmxcV234uA6u9NAbWS0gV6apXMvQJwlb1BuHNfZezuYj88qYw2w8UwF4skcrts/fBeuMy/XcKF9vIDr2MN58iZh88S2cZ3dZ81TCFrY7wa4Dp4iTdfQ5f+t/4I5RB/Yj9fn8CwecXkb29fqohaUxhJrkk6jlsQ6StC/O9wxPNJLQiosatkkDnMOXQu/P3fqnRAXx1Erz2dRr+n18D4vLWGuV9dFT9fd9z4I8cLiNYhfPncZ4hTlOg198uRR3OQ6pg30z3ItQYb9vfz5MMDzDUgD3bMHNdtjR++CuDiMHjmbjJjDQ3h9/+gLj0P8137kh79Rs+8o9TK+f3FkbgLiDuVK7fl4zXqkiTUpD3ODdGCfXmiweniP/QCv2c46ao4B9dGVLdKVKZdrq4M6cz6Dz4JsDts/VKI+SD7M9W18Vg0HeH1cer8hmcX8wR3SeO0U/l7z+H2TLm4vZanWLz0zhnIYBwm8Htauh3c8+ktSURRFUWLQSVJRFEVRYtBJUlEURVFi6F9P0sF13OlDmIv1wtkzEG9voZ9qYhFzrW6uL0C8cu5LEO+bwHXswwWsPzlzALUgp46a6ftP4elECVzXz+VxHf7cImqotsF18oRLNdWkv3+N17lZKbq2it6jRITnk/TxG+vPrUO80EAvUaWCGqRPusnhcVqnH8Hjjc5hHcFC+vWRuzWkWm8k/4hN98Gn3KyTU5iXMryCGph77DjGLuoxFvV71nKf+fpzEG9s4X21LPzbs1YtQ8y5UQ2Z3gLSCLkjpdN4H+ukUSZu4/O0yFfKSYjTaR43qNetrOD5rlKcTLG+VYT46Pd9HOL1Mn5/EMyM7Yc4mcB71PZQY1wlH6Ef4DW1kxi/soi6tCWoIe7J4j3KJPEa+m181l1bxNylC1Xssz613yVNzu9in7m2ifHULPb50STu79rCJYg7KexzcwXsM46D59MljdP0MF5bwTHdqOOzbZTqc0YhbvcNvmASkdc6mXz1nnD9JakoiqIoMegkqSiKoigx6CSpKIqiKDH01SRNB9eFH7r/EYhP/94fQHz+Kq4rP3gE1/Hv3YvaRyOqQDyDtkAZwpJtknFxfy2q8RbgsrcYwXx/jsF1/Tr5JMdHcB27nkCv1M4WapjSxvNxSNuJLKpHGZD3KEJvVODi3yxUzlKeuk51FEM8XuDjBbi4g5plMof385FxzGlaMLh9UDjk67NJ4+uSSNkN8fPDE7MQF/ajBrm4gnkyn/r6MxBzXsyhIfR4OZR7tNVEv3DCIX2KNM7Qx+2sERr629UisZtzq/JfulzPMeiR5kmfN+RrDJKoRy0sLEC8voq+0cWFqxBPTuHAvfcU+kgLc+hxOziP9TQHwd55bFN5C/PdBgH5Dkn/5/qIFuUKNQY1v24N79pmDzXKEdJ1i6SLp7OoS/fIw03lKSVAm6Z4HXwW+oLPisoqfj5HyWYt8oW2Avx+16cary08XkSDrEG6vd/EZ323TZomvb/hkgZrXOzTmRTltY5e/bNOf0kqiqIoSgw6SSqKoihKDDpJKoqiKEoMfTXJx//Lv4PYauLCdmEEvSrXFnHd/fx5zDf44L2kNVHduXYNt48Xad2ZtJaUj+vy3Qrl56P8iEEFF+oPzaEfbHIY17XP7eC6uEe6QUSaoJCmZyjnKPvVQgvPxyMtziFtrNemdXS6exbVz0zQn0CdFl7vxx77MsSl/KuvsfathOsjGsrV6gr2w/FJTFIbFTBH7tUy3veXH/ttiJ9+4qu4f6rv+NDDb4V43/6DEKdJ7wjJPyvUT0Ku20n+WsPadsQfwO0+5QjenVS4vycsIk3SJ99llbT4//yZ34J4eQnHOfsqv+8TqDe95ZG3Qcx61aOYaveOsHAFddUeve+wWcY2tqleo3HwnlhUDzKbpPzChsYa3cMuaYZBEsdEdqgIcVRGDTVJuzdkvbVIN/f4fHzagYV9PJHB8+mQCFquoGc+59IYIa+wQ2bgDj1rd1oY+3T9epTrNZtFDZOfve02XZA+6C9JRVEURYlBJ0lFURRFiUEnSUVRFEWJoa8muXX19yDO2riuPI9l5mRhjbQS8pN1Kb9eknJQJocpX6GgHywMSDvp4v5Xr6GGWHJRi/HIT5cdwVyoa8voQ1xZRV2Avr7rLwySisSif8jT+W618PwC9v6QN4vX7Q1ptB4u80uH2uNZrGXhCflV9PsNioDqQXJ9QkN1Pken0e9pOXidX3jxaYg/+zu/CfHO9g7EPukbW5Sb9eOf+CTE2Qwer9HBfmiZV58n8hvB9R85RzDn7LXo+vnccYkU9UuH9LEvf/FPIGYNMkO1CTtUS/FPPv95iCcnpyCObtO+O4FFz5YU5fps1lFjq7Zw7FHJT8k28Bo2k1QTld7HsEhzbJEG1zIViP0Emspt0p0TCWxf5GB7QnqWZC189qYoj3O+hO2rNbDPVTbwWZakZ9PkGN7zNuVbzlGfDkmHT+bw/ZEh0jgNe4k9era18AaFt6nBeiv6S1JRFEVRYtBJUlEURVFi0ElSURRFUWLoq0mmI9IEPdTsCgVcxz5xmOs54jrx56/h/ik9oUxHqANMUf3HEnljMg6u6w8fx3XmXI60mQb+TbBKOTOvbKH/rkd/Q+RLGA+TGWl9lXSBFK7bj42if6y8gMfLJvDzGfJW7VRxnT9HXqdsDtfpK1RvM0014WzKd9jrvXrv0LeSXosSTSbxvErj6Iu8cA0TTba6ixBfunQR4jblPj1wEDXNJNXGu0IeulfOvAzxW97yFto/6h9el/QnykVrLM7V2j92SXMkyVYs8pkmE5zUGL/Avs5N0mCvLVyB+IEHHoL40OFDEJ89cxbi69ex9uEK5X4dHx2VQTM0irVVL17Cc7DS+H5AzlDu0hbVM+yQZzvEe5JOYeyFnJ+Y8j6ThtcjE3UqhzqwUG7SwFB7AtT4RoaGIHbT+P1amfJgW6hjT0/is7rQoTzU9P5DuYNj3CaPdyaN1zdD7xl0PXw2ZhIs1OP7FTbVFk4JJQrvg/6SVBRFUZQYdJJUFEVRlBh0klQURVGUGPpqkmEX14WDENeRfR/j6VH0HdbX8fvL26j1LC1gvsGJEVxX/+RH7oX4yjp6lcZyuE6+Z74AccdGncDN47r42UXUirapgONIAdfx778PtZO1JTz/tVX0NtnktbrnHqxreH3laxBPTOC6fuhj+3aauG4/MY0aJ5VZlBqm4JRiAdfh223UzlIsEg+I4VnUCLMlvO6VJuXRXLsMsUsesI21Fdz/KOZ2ve9B1BQd8lwZ0q4vXzgH8fGjRyEukr7TJU2S6zfu0ihZZGSoFp9PNz4gn2NEmqNHvkrPw3iNNEP2UR4/cQLi6Sn0wLFPMyR/9DbVahwp4vUaBDse9qn1OurWrovXIApw7GRTuN118B46FPtU47Pn4bPES2CfEYNjvUWaZY9q80YBvR/SQ13fpVymPKaCNh4/lcLjN+l9hyy1d3zqCMQ79PmtMj0rbTz/RIJys9IYakf4bPYi3N5s0PsYOfy8cXB7P/SXpKIoiqLEoJOkoiiKosSgk6SiKIqixNBXk2zWyU8V4ZzKKRd3NnGdt7aFWkTaQc2tODIMcaON69JBBrWj9QhFtj/5cgViI6iFUJU9ccibtFxDDbTTQ01uD9XLTHC+wjJ6cSyDxyepSS5fWYA4srCFKfKz1cl7xUrV5BRez4XFCh6fcrk6Lu6hW8X9J6gu4qCoRnifnnr8cYgvnXkJ4lHSGGenJiH2qK7o+ARqaPkCamIJ0mvGxiYgPr3+IsTLK5jLtNPB68p5IndrkqyBCm3HjmRT7lr2WboO5fylmL/foetTLrMfugjxcBHHLZ/P+Pg4xJx71lBcKuG7DINgdQm9tKwhBj6OzSii2rKUp5lLzXoe6cY9HPv5DN6T4gheQ4e8uxblF241sM8mbHz/wETk66T8uobzOnfJqztMOjc9jcZK9B5BAcfgSn0BYiePz5pmFTXeNh2/Q5NNq4tzjU9P+3wOx6z0cH/t1+AJ11+SiqIoihKDTpKKoiiKEoNOkoqiKIoSQ19NstfGdWBetw4CWscmP1Zhguo5Uo204WHU/KrbuK785EuvQNxp45x+ZgOP1w3RKxOQX26UfJzbLVxXz+ZxHd/r4rp9q47HEzp/Y+E6d8/D/Z+7gDkwKZ2hdLvkQ+3h/pMJ9l7hunwkeP6JBN5ei+6f45COErKKOxh+79O/DnGK6pLOTo5AnM6jP3arTnoLaXypJOozLmt89IXSCPo0WW9KpVEv+uj3fBy/P4rtzVL9RUq1usvnGFJH4VJ4IeXp5O276k9SHtDTp09DfOalFyBO0/VySLNNkUdwbW0d4kwWx9VdR49h++j6D4IW1X80pIu7VBPUSWEu0IA8zaHgsyKge+TQKe+fRZ18bBL7HKWJlkS5jNtNEdtLeZpDyovt07M9oD7m0vsjnQaOqUIW95e2UGOsNvFZ36H6kNkift+h9lkpPL508AIYg9c7k8b954awz4ZdyoVLuWf7ob8kFUVRFCUGnSQVRVEUJQadJBVFURQlhr6aZII0NkM1zBxaJx8dxX8IE7gubZHPstdB3+PoOK4j92qY43FpBdf5I5dyjdL+xSffZr0CcUDiUruHXp0G+RjrAXun8HqwtBKRtuV1SdPFj0sY0PWW/v4yrlMo9PkkaUmsOe6uW3ibnKF3iPtPYn3CXA71H59q761vbEG8tLgG8Xa5AnGS6lPWarg9TRrbxjpqbOy7fPH5F3B/ZUqay/UbSXPk3KvdDuk5pFV3qBZfq4X9tkMeOI+/zx45G69ns4XjZnQM9bHNTbwejQaeb71eoxi3r61h/c+IctEOAs8jzYq2JykPdD6D3tNiFvvM6jZe850avS9g4/dHhtEXOUI1U3eq+Cw8fRlrpjohPQvp9YkeaY6W4JhyXX7/gepddvHhViaP+IUe5vvN5PA9gR6/r+LiGOAxnsmiJumQjp5I0Zgxm7SdNEfSaFsePzvj0V+SiqIoihKDTpKKoiiKEoNOkoqiKIoSQ39N0qA2wuUGOTdoFKDWYVHNtGQC5+RcFrWhRoDr1tOztI5OktnCOrbP9/F0spSzUrq4jm2x5kd1AyPyQnVIC7NNBeICeXPEwc/Xq3h9ZqfQt2m6qN3YFp4/+x67HfY2cU5OqsFGOUUDyofYIG/ToJiemYV4ab0C8fYO6g+VMmpy5SrmHi3vYB1SQ3kn9+zZAzFrtWtUj7JSwf0/9dSTfeM/f16rdsx/C/f/vkUaaqeD4+bqVfT7jo1h7tzTL7+M+yOxvtkkDZU01kEQRniOIyUc+7023vNOh95fII2PH62pJD6Lsjmq7UrPluUN3P/1VezDHQ81uyPzeyFOJHCsL2/jsyXwUbOzk3gPkuQ7DLuU19mah9DnfMSk6lohXr92BZ9NYyW8HokMHq+1hRps3cf3EELKnbtDNUsNPTt7wasfQ/pLUlEURVFi0ElSURRFUWLQSVJRFEVRYuirSQ4luS4drtsnUzjH+h7GqQSuK4fkDPTJm1R08fNBvQGxHVBNNVpWtg22b36a1rnJPFQYxdyxG1VaRycvk22hfyxLOTv3H98P8fOnr+L3SaMU0kBt8qtFEa7bdyjfYq1M+RdJ1+B8jay1TZEmevzu4/J6YGl1B+IWaVZGuD4inlcmjf3IH8J6kdVKBeJnnn4a4hR9f2N9A+LhoSLEaeoHxma943YaIPUDi/sBJWOl/fF9jUhrt2zO+YvXj7/fIw9Zedf1Qs01R/paghKN3nPPvRAP0fVLJQdfx9R1sc0e5V12XTxHofqOXZ/6YIFift+Bat8urmAf20jis49sgjI7uxfiEtXm7bTRxzgyQmMi4NyuuJ17XEQ+w935drFPZTKoebY432+A7z8Usqjp1ikX7g55b0M6fiqF579Deb17FuW6zb36Pqe/JBVFURQlBp0kFUVRFCUGnSQVRVEUJYa+muT0KNY46/RwXbfbxXXzFO0u0eMaa6gZJkgLSVINsT13nYB4rY66wFfOoDbSoxyYU2O4zj03jRrkoeMHIL5wcQniawuoE6yt4rp4q4m6xenTFyDe2cF1dy7XeG0Rc2BScyVBNs/7Tt0FcUQ5NgMX2/MjP/FjEDu0js9a0uQU3u9B0fNREXEt7FcmxVpuEeLhYdQn2PfHml63R/5RynN58jjWPxweRi2Xr6tzm5zCIWuGJK47FmvLrBD1ry9JpfvEkIeMNU/Wxvlo/C/sr3UdPN9cDscZa6AOdexUgpJAD4CI6hM2W+hpnhjF3KrJPOrcNnmUIxrstsFzzGbwGiUp12jgc21a1OUT1MU6lBu11aMxQ33UovdDXJvacxud2FB9TYncb/zBm3hUG7fdppqpgs9Kk8E4l6FcsJR6dTiP+YU9en9ms4Jxp4FxP/SXpKIoiqLEoJOkoiiKosSgk6SiKIqixGB2e7AURVEURRHRX5KKoiiKEotOkoqiKIoSg06SiqIoihKDTpKKoiiKEoNOkoqiKIoSg06SiqIoihLD/wMZoCd8i/JaYgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 576x576 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Define the Convolutional Neural Network\n",
    "The convolutional neural network (CNN) is designed for processing images. CNN-based architectures are now ubiquitous in the field of computer vision, and become the default choice when inputs are pixels. Here, we only introduce the simplest CNN model, for modern CNN architectures that are widely used in DL, please check http://www.d2l.ai/chapter_convolutional-modern/index.html. \n",
    "\n",
    "In this example, we use two convolutional layers following three fully connected layers as our model. The output is a vector with size 10 corresponding to the probability of each class. To make predictions, we choose the label with the maximal probability."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = torch.flatten(x, 1) # flatten all dimensions except batch\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "net = Net()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Train the Model\n",
    "The model training procedure is followed the previous examples, thus we won't discuss more. The full example can be found in PyTorch's tutorial."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)\n",
    "\n",
    "for epoch in range(2):  # loop over the dataset multiple times\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # get the inputs; data is a list of [inputs, labels]\n",
    "        inputs, labels = data\n",
    "\n",
    "        # zero the parameter gradients\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # print statistics\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:    # print every 2000 mini-batches\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[1,  2000] loss: 2.169\n",
      "[1,  4000] loss: 1.843\n",
      "[1,  6000] loss: 1.670\n",
      "[1,  8000] loss: 1.586\n",
      "[1, 10000] loss: 1.509\n",
      "[1, 12000] loss: 1.460\n",
      "[2,  2000] loss: 1.412\n",
      "[2,  4000] loss: 1.379\n",
      "[2,  6000] loss: 1.367\n",
      "[2,  8000] loss: 1.351\n",
      "[2, 10000] loss: 1.322\n",
      "[2, 12000] loss: 1.288\n",
      "Finished Training\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "source": [
    "# save the model\n",
    "PATH = './cifar_net.pth'\n",
    "torch.save(net.state_dict(), PATH)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Visualize the Results"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "source": [
    "# let's plot some samples\n",
    "figure = plt.figure(figsize=(8, 8))\n",
    "cols, rows = 3, 3\n",
    "for i in range(1, cols * rows + 1):\n",
    "    sample_idx = torch.randint(len(testset), size=(1,)).item()\n",
    "    img, label = testset[sample_idx]\n",
    "\n",
    "    # calculate outputs by running images through the network\n",
    "    outputs = net(img.unsqueeze(dim=0))\n",
    "    # the class with the highest energy is what we choose as prediction\n",
    "    _, predicted = torch.max(outputs.data, 1)\n",
    "\n",
    "    # process data inorder to show them\n",
    "    npimg = (img/2+0.5).numpy() # unnormalize and convert to numpy\n",
    "    npimg = np.transpose(npimg, (1, 2, 0)) # notice the default image is 3*32*32, we need to change them to 32*32*3\n",
    "    \n",
    "    figure.add_subplot(rows, cols, i)\n",
    "    plt.title(f'GT: {classes[label]}, Pred: {classes[predicted]}')\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(npimg)\n",
    "plt.show()"
   ],
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAHRCAYAAAABukKHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACf30lEQVR4nO39eZwkaXXfC58TEbln7V1dvW/Ts/XsMwwwDMtI7AKsxdoFFlpsazPWtWwjy7KEJSHLsq5kvfdeX3hlvcISRhJaQQgESDDAMAzDMgvMPt3Te1dXdVVlZeWeEfG8f2Q2yt8vJzKrppesGc738+nPzKmIjHgi4oknMp9f/M5R55wYhmEYhtGPN+oGGIZhGMZmxR6ShmEYhpGAPSQNwzAMIwF7SBqGYRhGAvaQNAzDMIwE7CFpGIZhGAls6oekqn6nqp5Q1Yqq3jLq9lwsVPXtqnrPBtZ/QZ4H4/Kiqu9S1fePuh0Goqr7VNWpapCw/BdU9X9erO1dTlT1qKq+Zp3rvk9Vf23A8oqqHtjAvgdub71clIekqn6/qn5RVauqutD9/59S1b3dAzv/z3XXOR+/Ysimf0tEfsY5V3TOPXAx2rpeuie41W3nsqp+UlWvuZxt6GFk5+H5xiXsi8+lLRserFT1blVtdNt0TlX/UlW3X+y2XSpU9S5VPTnqdlwqRtG/nHO/7pz78Yt5HOfp6aPn23lUVX/+UuzrQumOf0cu934v+CGpqj8nIr8rIv9NRLaJyJyI/ISI3Cki890DKzrnit2P3NTzt88N2fxeEXkkYb+X41vSb3bbvUtEFkTkfc/SDlXVS/2LfNTn4XnBJe6Ll5Of6bbxKhGZFJHf4RXsul9+NmP/uoj9YLLb7h8QkV9S1Tdcwn1ddFTVv1TbvqDBXVUnRORXROSnnHN/7pxbcx0ecM79kHOu+Ry3m1HVioj4IvKQqh7u/v2oqr5TVR8WkaqqBqr6T1T1EVUtdb+FX9uznVtV9QFVXVPVP1PVP30uP7+dczUR+YCIXN/d7t2q+m5V/byI1ETkgKpe0/21uayqT6jq9/a0Y0ZVP6yqZVW9X0SueD6eh83MpeqLPdt/uare2z2/J1T17d2/v6l7bsvdv7+r52Of7f631P2WfsdG9umcWxaRv5B/7HfPdt1f2tOuh1T1rp4271fVz3Sv+ydFZMsGj/nbVfXB7rEdPj9wquqPqOpj3e0eUdV/2f17QUQ+JiI7en6Z7NjIPjcrl7p/dflRVT2tqme6D+Tz+/7GNHnPL78fU9XjIvIpVfVV9be0M/NwRETe9Fwb4Jz7gnS+kF+v3VmBbp+bF5E/UFVPVX++2x+WVPWDqjrd09a3qeqx7rL/+ByasKU7jq51++7enm07VT3Y/f/3qer/q6ofVdWqiHyLqt6iql/tfvZPRST7XM8D4Jx7zv9E5A0iEopIsM71nYgc7Ik/IiI/v4H1j4rIgyKyW0Ry0vmmXRWR14pISkT+vYg8LSLp7r9jIvKvu8u+S0RaIvJr62zr+86vKyJF6TwkP9eN7xaR4yJynYgEIjIhIidE5Ee68a0ick5Eruuu/yci8kERKUhnwDslIvc8H87D8+XfpeyLIrJHRNak8y07JSIzInJzd9ldInKDdL5w3igiZ0XkO7rL9nX3s6429fStH+/+/xYR+ZSI/FHCdd8pIksi8m3d/b+2G8921/+CiPy2iGRE5JXdY3h/z74eFpEfTGjHi0VktbtNr7uva7rL3iSdL3oqIq+SzhfFW3vOx8lR94fnWf8630/+uDtG3CAiiyLymu7yd52/bj3r/mF33Zx0fs0+3u0X0yLy6fX2u94+2r2ed3av56u71zIUkf/a7UM5EflZEblPOrNrGRF5r4j8cXdbh0Sk0u1rmW7fC3uO4+UiUhrQlvd1++j5z/+u4Dj5jXPaXXe1215PRMalM879H9K5R79bRNpyEca5C+04b5XONEPv3+4VkZKI1EXklYM6znPoaEdF5Ed74v8kIh/siT3pPIDu6p7oUyKiPcvvWe9J616ERvdY5kXkwyJyRXfZ3SLyKz3rfp90H6A9f3uviPyydH4FtqU7wHSX/Xrvxd/M5+H58u9S9kUR+Q8i8lfrXPe/i8jvdP9/nzy3h2St2+5TIvK/5R8fenzd3yndB2jP3z4uIj8snQd7KCKFnmUfkJ6H5JB2vPf8caxj3b8WkX/d/f+75IX5kLyU/et8P+kdI35TRH6/+//vkv6H5IGedT8lIj/RE79uvf2uZ3slEVkRkcdE5B0917IlItme9R8TkVf3xNulM74FIvJLIvInPcsK3c+/Zp3n4X30+aKIRCKym89pd90/7Fn3lSJyWnCcu1cuwjh3oXPMS9L5eRw450IREefcy0REtCPeXwqt7kTP/++QzrcH6e47VtUT0vnWG4nIKdc9W8/y2fXwW865X1xHO/aKyEtUtdTzt0BE/khEZrv/37v+MblwLud5eD5wKfvibhE5/GwLVPUlIvIb0pkhSEvnG/CfXcC+RDqDVNLbjNzvvkdV39Lzt5R0fknsEJEV51y1Z9kx6RzLetgtIh99tgWq+kbpfAG8SjrnNS8iX1vndp+vXI6xjseIG9a57o5n+exG2XL+uIhF51yjJ94rIn+lqnHP3yLp6LPQDudcVVWXNtiO3s9XVHWZt/ts63bX4XHuYoyzF3xhvyAiTRH59ovQlvXSexJOS+eiiUjnJRrp3NynROSMiOzs/u086x0gNtqOEyLyGefcZM+/onPuJ6UzbRLSvvdc5P2P8jxsFi5lXzwhyTryB6Qzy7DbOTchIu+RzrSVCF6jiwX3uz+ifldwzv2GdK77VFcnPM9G+t2zHrOqZqSjk/6WiMw55yal8zC9lMe8GbgcYx2PEacHrNt7ns88y2cvFnw9T4jIG6nPZZ1z58eab7RDVfPSkSY2Qu/ni9KZPk46D3wOeJy7KOfhgh6SzrmSiPxnEfkfqvrdqlrsCrs3S+en9qXmgyLyJlV9taqmROTnpNOR75VOp45E5Ge6Lzh8u3R0lm/QFYLvugjt+IiIXNUVrVPdf7er6rXOuUhE/lJE3qWqeVU9JJ3psIvJBZ2HFwKXuC/+bxF5jap+b/ccznS3KyIyJiLLzrmGqr5YRH6w53OLIhKLyDe8XT0vXuy7wDaJiLxfRN6iqq/vvryR7b5sscs5d0xEviwi/1lV06r6chF5y+DNAb8vIj/S7VOequ7UjgXq/K/lRREJu78qX9fzubMiMtN90eUFw2Ua6/5Td4y4TjrvN/zpOj/3QRF5h6ruUtUpEQELh3Ze/Ln7IrXxPSLy7vMv1KjqbHdMERH5cxF5s3ZecktL50WnjT5jvq3n878qIl90zq1n5usL0vkx8o7uPfpdcpHGuQueInDO/aaI/BvpvCyyIJ2b5L3S0UvuHfRZVf2Yqv7CBez7CeloBf+XdF6UeYuIvMU513LOtaTzksqPSWe+/a3SeZg1u/veJR2R+YKniZxza9IZKL5fOt965uUfxW4RkZ+Rzvz6vHTm0v+g9/OjPA8vJC5VX3TOHZfOyzE/JyLL0nl55qbu4p8SkV9R1TXpaDIf7PlcTUTeLSKf187bpy+VzjflY9L5lX9BdAePbxeRX5DOQ+uEiPw7+cf7+gdF5CXdNv+ydF72+AbaeRv6hxK2fb90Burfkc4LEp8Rkb3dvv6O7nGudPfx4Z7PPS6dF1COdI/5BfF2q8hlGes+I50X7v5BOlLPJ9bZtN+Tjhb9kIh8VTpfynvZLSKfX+e2hvG70rnen+j2+fuk08fEOfeIiPy0dGZXzkinf3zDM6uqr9DO2/qD+IB0+uqyiNwmIs/aP5mece7t3f1+n/Sfh+eE4hTuCxtV/aKIvMc59weq+lbpvH36H0bdrstN73kYdVu+2VDVX5SOzvPeUbfF+OZAVR+Uzss2G9UHDXmBPyRV9VUi8oR0fl39kHSmCg44586MtGGXGTsPhmEYz41Nm0HhInG1dKaFitJ5O/G7v0kfDHYeDMMwngMv6F+ShmEYhnEhbOoqIIZhGIYxSuwhaRiGYRgJDNQk9+w9CHOxQYCrc+x5mIjd9z1aPjjm7aVSKYrTEGezGYgL+SLExeIYxbi8UEB7Uy6L+XAzGdx+Oo37b7VaEDeb6Kqo1+sQ12p1Wl6j7eHn223cfhRFEPNUeceS2bN+jAk0+PNhiMvjNm7v7z/9cZXRMEQDiCkevHoseNxtaUMcUtyg81in8/LUYXRv/K//+QGIP/mRf4B4aWUF4ma7AXE6wON523fcBfG33IoV2kLqd8MEE9/H+7KvX2ewX8fUT9bW1iBGv/azjANUFCeiyxWGuH0eJ97607982fvd3KFpOI3VRhWWBzEexMwYji1X7cT8HF6I62fSeM7HC/j5ybFxiKtr6JSgzUmcx+2drWIfe/rEcYhLq6u0Adygi/GURzH9furrZHjP5HPYhyTCD0QtXD9FfUYzONb7ORyLWzSWRm0au/h4aGzkPu8JHu+Jx1YS+5z9kjQMwzCMBOwhaRiGYRgJ2EPSMAzDMBIYqEmylsFaBON5uJw/37/+YI1y2P6YmOahWXNjDZE1T4/2x/vneW+OWZNst3EePooGz6Pz4fL54Hn2YfPwLCT0a5gUy+YgjtsDlzul60wtj/pi0mJJq22FpEnGuP4DX8cCIP/lV34T4vs/ex82MKQzSRc2n8N+d+O1+yDevXUS4rVyGeJ6DbVs3xt8n7XarAfh+rkC6j9xiu7LCOOUR/qOR/2SzrdHw4ySvrZWQc1zFHh9he1ZV6V7lzQ3Vf69gQc5N7cN4iy9XxHQ5/neHh+bhLhBI/fyqRJub8j7Iw16H4JvOZXBOnWQxvY2WSdv4/osTOdzOVxMGmGthpqw33d+B8MaJI/1bWrvIOyXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcBATZI1sWEM0xSHaZD9MeoEnj9EsySNbZgvkDVD1iRZs+N5fdYNWPPs8yEO0RCHxUPp03AHa7q8eRdvDlVSdWNaqaPDZE0yZK06Zm0XNzA/jxrg//e96IO8/9OoQSpp745EN9bqd2zbDvH2rVhN6sTpZYiXVsinSNtL030S0PH13cfUD6sn8XhX6qitb5tCzdInzbY4hsvHiug/Zr2L/cN8X42CVgvvVR57+jRK0th4rJEhnmZ+X6O0jD5H1tD63qfI4zmuVVDDa5OGOka+znaTvbZ4PGkPrwlLjK0GVdrzyfvKPkWfNNEGeoUj6tOhj3GLvMU+XQ9+Ngzz4Ds27w7AfkkahmEYRgL2kDQMwzCMBOwhaRiGYRgJDBQDhs2jD9MShvogh2ierDn6Q7a3UQkvohySbUXxhI+fdYeNap68fKMM03hJKloHF6iBXiL6tFL2PZIGGQv79JAG5cCtVlETy+XyEH/9sSMQP/ww+iTT06ghBpzTl657mnICLwnm6fzE189BHEWo9wTs3yWtOcu5WKkfF+k2bdQwj+fKGp6PGw8egDiulmh/uP/xSdRY2+QXbtSxY7KncHJySkZNvY6aV5AhDcsNG2vwnEzRMbFvj8eClRXUoYukOTbbmMt1z86t2F4aiyPSldlDzn2ySPl7Dx04BPHx45gL9sipZyD2af++j/vLZnB/DdI0Q3oPQXzUZId5yh35VtmzzhovjymDsF+ShmEYhpGAPSQNwzAMIwF7SBqGYRhGAoN9kkM0QdZG+lx5NA/Oy4M+jZHmtXm5ckwanc+5V3F/MddbdKgL+GS4CykHZ0g5PnmenHUGjofVg7xQtO87z5CYdBY+H6OC9YI+/+gQTTJy5GGj2nh/8Rcfg/jvP/FZiBdWShCXzs1DnE1h3kmhvJ8xaZzpyRmIUynUp7SOHrdGhepPNjHPZj6P+681cPm5FdQ4HXnMJKJcrlls7xPPoCZbOo399vWvvhPi/fsPQtyooMZZXkWfJ2u2zSauPwpCqk8YpAfXss3nUDMco9q1nJuUa3I2Kf8ua4p9tWmbg99v4M9z/Uh+nyFL7a+sYR88/OTTEE9PTUOcy+Lx1R22N8e+REFSKX704BotGivZt+riwT5Vfl+jXMbzn96AN9d+SRqGYRhGAvaQNAzDMIwE7CFpGIZhGAkMnJhlzXGYJsh17YbFvL3A5/VxOdcUG6aJOmFfI2uGdHz0+f7cteybHDwvPkyDHJbLlXWE4bldaX0yDLqYz9Dg7W9WXF+OWjqvDn2R6uN1r1MezM9/4QGIPboujeoSxFEL9SEV1K/8PHrk2L5aV/xLc4W2X8FcqrFi+6ukl8WklWtfPU66b9PoWXNkPF08i5rojkO7ID5wAOMd27dAzHk9a9MTELNHjj2KoyBNPr4saW4B30w0FjVIQ8xQLlHOFRqRpjgxjucoJG+vI92bc5/yOeVcrzyWcKrZlMM+VVoq4QptbD9rlItrqINnyHcZUa7YNPkWQ9LJWcTkW559mXy87OnvG3v5eg7AfkkahmEYRgL2kDQMwzCMBOwhaRiGYRgJDNQkeV6X57WH5RLty73KmiPVGAsC0iB5/1RzrC/36+Dykn1eHZ6nDtlrQzrCMP8ea5AXWj9y2PneaP3JPimvf41hK4yGIXUyIzfYI/XYY8cg/vy9D0OczhQhzuc5byfqHbUW5tns03uqqOlVKPepBKjHsG+xz9eo3M9Iv+nzC9Ntrbg/z0O9KGxRbUDS206dweN95sRZiG+8Fn2Sk2O4f/Z1cj9tNlhDvfywBsn3bo1qYE5RblXO/9umc8rvU2Qpd2ouj5+fGNuG2yPPdnoGNcw6aZR9tXb73j/AOCCPejpF9RxJh5/dhfmL19ro+4xivGd4/2HISj0uj3nspaGN60fy8ib5Slkj3chIZ78kDcMwDCMBe0gahmEYRgL2kDQMwzCMBDakSQ6rJzlsOeff6683OWw5+yQ3CHlj+mqUsaTXV69yY5ri0OZscHsb3T6fL8732L+9zeqT5PNAi6kjcK7QP//ARyH+6pefgDhIoT60RrlPWzXKLRpTe9iXGFMOXPZktTmvJXu2SJumHLvKeg0df8R+WMqTmc2jniUe5u1sh6gnnSvh8j/8q7shPnISz9e33oG1CG+6di/EaarVKB5ptCOgVsU+o6TJOdLk+BiUOmWNcqFOTOA55/cxFpcWIc7nUSd3Aeb7rZHm1qY+oaxLExqyj5Nf4MDtcXszlO+33qA81qSrZ30+X7i+R77JfA5rrrbJZxlS/t+YvL5cv7LdorEvMp+kYRiGYVww9pA0DMMwjATsIWkYhmEYCQyuJ8m5U4dolMN8kP3bG+yr3HDMXpu+XKnsQxzsg/Ro/f5crkifhjhw7eG+x2H0e6EGt2fY8s2Su1WHaJC8vFnFXKd/9yHUIP/uIx+HOIqwn6ay6KFqtXn/qJeo4vpOuA4n5yimepPk6+wXWWkxlwHlP5CmGpCelS5MYpzH2ofsWJMa6nMS4xoLi7j8Lz/6BYi//HWsRfh9b3kFxK972Y0Qp9Kj73ch5XXmWyvdN1KSBllDDZLz16ZmBuuu6TRqjk3K3Ro2ScML0NfJtXh5e3w8nLuVNVVxqPl51Oez1OfS1J5WG89HSBqgz77JIZphit5vyWbxfLZJo+T3WzjPtuvzaSZjvyQNwzAMIwF7SBqGYRhGAvaQNAzDMIwENuSTHOpbHKKR8eJ+jZG253Hu2CEaJqkrhSzOy3O+RK4vyb5J3+N8g7h+u81+OAz7jZysQfJy2twF+yZZI92YRjkq+hxMPp631SWsv/jwV1ATu/fuz0K8vLIKsaax3mMqg3rK3JXXQVyror5SK+P+60sL2N4KanZxyLUI+UYQok+EJfC+yE1gPcfidvQl+pQrtrKKuWUdae9+GvWnOKRcspxDmXLfnizj9v7gr++FOEP39Wtfged7FHAt25C8rl5qcB7rNvkom+Trq1Cf4LFobg5ztfL7E5VV7MMeaYYu5uKxg/Nc+wG9n0G6M2uUAV3z2UmsJzlRRJ273sR8vzFpgOkU1ZuksbhJ57NeRa9yJj3Yo8/Xh8d2zrM9CPslaRiGYRgJ2EPSMAzDMBKwh6RhGIZhJHBBuVv7NEpcLJ5PWkfA8+TsW6R5Zfab0bx4kML9z4yjtjQxhnEqzfUpOYcm+SJ9nDdvt3Beu1pBrcaR9hSRbtCkeflWC9vfjjemCfZrlFTP0nGuVor7/H2bA85levr0SYjv+8ynIV5eOA3xyROoETbIY5ZK4XmqUz3D8SzfFtRvyN971523QpyjuqjHj6Nv8MnHj0LM/YrZtXsO4jtfejPEsztRg/z4/YchXiljP41C1HdSrA+xvMV5PendgCCDtRXzU1shLpdRA/3kvQ9BfNO1u2TU5HI4VoR0b6RJF2420QfZ5vyztP6w2rJV0r1Zx47JR1hbRc2zSfUuI34fga4Z+0I9bi/FPg3uM5Ok6/e9PzLYw95qYftbpIny/tMZPL8xaYrD3tdgz7wzTdIwDMMwLhx7SBqGYRhGAvaQNAzDMIwEhuRuHTLPTDHPa3NuVtYUvb7tDc6tyhroxHiRYtQV2DfJOT/TVGdPSUPt81EG2J6JCfRhcj7ASh21IE3h9nMF/Py5c6hL9H+H2ZjPkXWQfq/QYE1zVJw+fgTij37oLyGulrB+YZNEtKeOooYZ0nGn6TwpaaBho4LLI9RPJEQNc3YOPWNvfPWdEH/xXvRxHn0GNdR2s4bbp/tg217UJK97CWqgx0/i5+uk90RUH1LoOjfq+PmoL68l3djUDTlHc0Aap1ItxNOLJYiXS5QrdgT0+Qjp/QiPDprfz5icwnqRxa2zELMGxzpw33I6p3nKVVqnPlMgTZU1uwZ5uiPB7d1+J/bZs0+hjt6i+pjVNbxH6HWT/uPhPhQNHpuEdP2YNEv2haaoHiWPhT5d33hIHu5e7JekYRiGYSRgD0nDMAzDSMAekoZhGIaRwBCf5OBcqaxZej4vH5JrlX2P9MjOpHleGrUgzilZq+E89VgxDzHnS2RfZ0TeKLIWSRxTjTfK5co10gLy4/Hx12ukRQ2tQDmMjX5+c+RqZT7wh/8L4vLyIsQzk3hd5+cxT2SNavmx/7ZN5722ihpnhfSgdA7zUmYK4xD/w/1HIT6Mm5Ms1cKTFG5PFfuVR+ufXcN++9efP4XLjx+DeOXMcYhbVdT8QqpVqKSPeRRzkuG4jeeX9aGI71Py5zYanOcUPX6jgN+PcBSzTstjWzqDuqtHvz84V2ihgN7SPt9gE8e2nEf5dKl+IudCdeSF5fy941t2Q/ymf/o2iL/22U9BfO+n/h7iJbonfRq8h9XW5Vy5rIKzzzNmDZPiLHmb+X2SiDXPwWnGAfslaRiGYRgJ2EPSMAzDMBKwh6RhGIZhJDDEJzn4GcoaG3tRhvkqmTzNK+/fvwfipWXMycnz9nGE89xcHpFzs9brqK3UGlQ3j5qbonl3Pt4UabjZPOoOrP04Rz7KVcoFy+0fVl9SXhicIZ9kgWrHnVtGPeaZk2chrrYpBy95yAJHuUqrZYjDBmqc7RWsH1klfSfI4vaeJp/l7K59EOdnMVdpqoB5MNuUF7RcwlqCS/fdA3FzFY/fRfj5mGoL+qSxBln0G7frrJUPhvtdizx87TbGfo58q5mBw9BloUUan0/vQ/RZRR0ew+ICanST1OfyedTRp6fRW7uwgGNbNoM6dG0Z89/GdNZZM63VUJP00tgel0Jf52NH5iF+9PAJiFuUTPXEGfT6Soby+ZKuHvD7GvzCB2uUtDii+pIMP6uaTXq/hJ49aaqZOnDb617TMAzDML7JsIekYRiGYSRgD0nDMAzDSGBDYkCfL5KTtW7AeyLSP09cLKKGlyEtanICtZSVFdReWB1hCZTn7UPKZ9ioUb1HytE5VsR5/Qr5z7ZQ/kb+fK2E+Q/zhRmIM2nUOWp19E4N1STjwblb+z9P3iMWQUfEeAHPs5AfdbmMGtfpJTyvkVDuUA/1oD5PWRb7Xb9/Fa+j0nWNKqj/VKmfpcin6ZNfN0oN/q7aWEbNsUWaoSNPWF8iTc6BnMLjzeaxH0YNPJ6Q302g2onsAfTovhKqhbh7H+Y13b1jm4yaFunAAedupWNs0zGmx/ic4HL2VJ84cYKWY5+amUCdOgyoPVRvss93SD7CNl2D7Xuvhrjp8J5bWME+0CZdO6Y+Mj4xCXGO+ri08fyl2TtM5ytFY3+d60ey75HyL3NuXa6fyZruIOyXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcAFGpQG15ljhmlorRrqAiHl4JwlzU8inBdv1KgBNC8f0bx41NKByxtV3H/pHPrfTp9Gr1CQwv3tPYA+z0oFtbNt2zDf49YtqA0dP4Z1EeM+3yflm5TBDNM0+4yZI4LzaKZIq/bIMlUV1FO8NOoPHtXNbMaU2zWL10FJL3HkoWMtl5P8Rg3UqpePPkar0/aVM1di+x31S75OSu8KuJg9aOxRw8XtGvpCQ8q9qpSXlDsinw/ODTtOWvurbjkI8ewU5sIdBXv24r26vIIJeKsrJYg1xHMQpKi+5Bh6T9dK6MXNZbDPZnPYx8dmcKyr19Er61XwnLk26tZjBdRIm24nxNNbXgbxkyfwmlUj8tKmUNf3YvLCVrDPzE1vh3h+/gzELdLtHWmKroWe8YB09VwRNU+fdH1NU31JGjOieP21c+2XpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcBATVJ1cC7WfhVssFGS12afJefTq66hBpjNcK5UnHeuU73HmLSodkx+uyWc56+R94c1wLBFNeGyqAtoirxTNI+fzSjFqEsc2Ic5PcXh/k6cxvyKYXOwVtannT1PKBZQb5icRa221qZ+GaHeEXjkQyT9oa82IPtJlS48p/FMo/6kEWnDpIW7iPyugnE/tH/OgUz3GTeXPWOOahGy3zlqoUbr+9gv0znsxw3yrKVIj1O6D2++Cvv1HbdcQ/snX+UIaFI9xy3T2Ofaa/g+AY901QqOVQXqI/xrhL2tV1xzLcT1Fmp+qSJqlFmHmiFrcukAr3k6s5fai/dISPdIfgJzy65VMZ9yo0beZKoZWphATZZzucaku0fkE3Ux515lHZxq9VKfTqXw2aAZXB631z822i9JwzAMw0jAHpKGYRiGkYA9JA3DMAwjgSGa5LBkrBvL3cqLefsxaR3lMvrNJiZxHj4V5CgmrYq+AzTqqDnWqujFiViKonyJ2RzGBfLvVZuoSzz51NMQLy+jHy0iLYy9VukM6gp+QDXkhuR25fSGfb7ITcqefeixyuTxOi+voB4ykUX94Ywjv62wPxdPTIpzuY6j/hOusjjN+gl5vi74PA/xH1MuVcfvDpBWnxnHPKC5MYwrVKc1m8XzrT5vn/SfNPoopwu4/hu/5TaIrzyIGmWrNUyjvfQ0KXcrHbKMjaPGlqF7s025Wbk+ZZrOGevMpRWqF0ne1lwOrxmPVW1XpRg/PzGO+XF96mPVNRybFuePQ9xYw3qZLsT9rdF7ApOku8fkBeZ8v2nyKnuK55drB7OGGrAHnLYf0/sZ/Tp+MvZL0jAMwzASsIekYRiGYSRgD0nDMAzDSOCCcrdyjk2WMFlz5LhN8/ZcUy0OcV670aC4jjpCtUrz4DHl5OzzfeLhT01OQsz1Ih15gebnT0F8jOKIjJbFIuoap+ZRC+L8hnv2UO7XKmqqfDyxY78Za5RD6lFuEs1y3z70dLF+0d6P/Wj3Tsyh+/hx9L+yXtEMyQ/r4/JUFrXvmPpZXGdNkjXEIX5i3aB/te+rLN1X5BGTDHrgcmOoRznyJbbJ8xbzjUwet1QWfZOawuU7ZrGf79+BepqPcpOk+Q8jgL2zEWmMrNNmKJ/tBOUS5fqO7Mvz/cFDr6eoK6/S+wdU0lOcUj5ibxLisSm8p/ryATdxrGs1UaNMUYlXIe+t481FqCHG5N0NMuSbHFbblkRir8+iP9hL3KZnh7r1/z60X5KGYRiGkYA9JA3DMAwjAXtIGoZhGEYCF6RJ6gYfsTyNHLZx3n+G6ilmKf/el7/0ZYhXy+hLvPaa6yDO51E3CALc3rY59OO1KOdmOkXeKfr8EtWcq1Qx32K+gNrWltk5iIvks4zIq1VZY50BdYCYc4QO8UXG5C2KOWfpJtEkuV9wTt9dO/A83n7DFRA/9RRel2PnUG8hS5e0WqhRSojXkX2BXhavW1zH9fsuxNBKn0SfoXjw1vreDSCNNCaPWaNMfl2q/yiUV9Sn+zDDPsoI78MD29EHOT01CbEjfSlDtf9GAb8fsdJEnXZmjPI00znnPhqQZ7tFeaGrVTxnExGOfSSRyhrljk1l+Jphr8gVt0JcnMB6kkIaadgoQex72GfSWVy/TWOF5wKK+X0H3L1HuWbrpPNnyDeZ5vzAdBewTzUk721MJzTwTJM0DMMwjAvGHpKGYRiGkYA9JA3DMAwjgQvSJIcmax3iRfHIK9SgeeW57ejvUtIuDh8/CnFEfrGDV6JWNTmJOTmnpjBeXcL8iZksm5FwnrtMNeSc4v6DNHqn0hnUstTDefcUe4eGaIjDNMWNaoxDU/VeJtZWyxBns3geW6ShHZhDvejNd+J1//iXnoH4kadRH2qStktyicRkylK6Tl6LPHGcNLcvx3FfdcEhMft7qX2kgQbsR27i+WxRjmGhvJps40yxJ66B+tGhPbMQv+GVt0K8beskbjAk3yrVsxwFXHOULwHXi6xTvt82+STzadRtPdLACnkcC3h/IXnEA7ooOcorXUihF3Vy6iDESjVWSWaWJmmSGdIg/Qx5a8lrm86gd7ZdRQ01W8Sxrk55s6e2oibLPkefkq321UCl9xiaDdy+kkfcZ2/xAOyXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcCGNEn2Brm4L4EeRDHNI3OuUfYtPvUMakenKJdpkMd575j8VkdOHoP4bGkJ4okJ1CB98r858hrddv1VEB86dCXEc4+jX+/0WfTnpXyeV4ewr6ZZzMKEx+ebtS6uY3hhvkeuizgqKmvoawzJw9Ygz1lEGtc1B9H/Okba87bpJyH+6iOY+7XUQH0jinB/YRtjTwbnYlWPNcUheSo5N+tQsZj0FvKUtUnDVfI5evR5pVytacHzf81ePJ/f86YXQbx3F+pjjQbqUwzfh5sBri/J16hZwPy0XPs2R9cgoPy2fcdMl7hFmlsckXe3jWPn3lkcm2QS44pH739QrtbQYZ8OSIP0AvLOko7faqEG6KfIR5rH9s7uoHzCEY11dL6rZcrHTO+npEkjZh9kjby8rKkOwn5JGoZhGEYC9pA0DMMwjATsIWkYhmEYCQzRJDeqjdCnaX3PG1zzS4Xz+eG8fKGA3qLZWcxPWCdvTLWG8/jlMvrFeN47TQa5bVtwf1deux/iN73l9RCfPHoC4rOnz0KsSvkFmxgrzatzWcL+XKyDNchh9SP7Pr/RHKOXCI+Pm7TSFJ2nSgX1Cp+K7V2xG/tJLoc72LsbNbS//czXID51Fvfv0fZjjzTJIV89HYvTMelTMWqAnPjS0Q6CDGqMfoCetEYZ/b9K/uRiHvW1nbPoO73xyh0Qf+sdqNXffCPeF6xxNmt4X0Yx50i+QLv2RYBzt/qkabH3lL27+TzVm6Rcrq6JfYi9oe3lEi6PsD1ZOkWVEvb5meLVEC+HOHbx+yNhTLq+YHsCKvGZzeIfOPVpo0nni94XyebxfPV5fTEUjzTUfAHfR2nVqSYs1cfkep79tY0td6thGIZhXDD2kDQMwzCMBOwhaRiGYRgJDBQDLnUuT/YepdM8b40NqNfR77VnN2ohZdKmzi6gJsjepzDEef8oxOWPPnUY4kaMGuIdL0V/2G0334jbP4TrH376KMRcD3OB4miIpjiMYRokwxrxqCgto3YcUk5fnzxSQYpjPI5aDf2yaQ+v+8tuRz1nZQX3/xd/ez/EcQbrhCoJRo5q8XmUGzWdobyfLWxvFFGtQPYb+/h5pTyhrQr6Eq/Zg56066/aDfHeHZg385qDWHtwz07MzTpWRA0zpuvTapPeRp60Br070Aw2w3d1ukbUpoBunYC8pDnShVOUl3m1hmNTmjS6Gp2zp48dh3isjX1g6zReo+IOvEbtHHq4wybeI+3SAsRBG8eekMYOpXtsnDTYQPAax47rXeL5rfV5oXH1mDzrqYA1RVzfS+P1CNn7S5qqONrhADZD7zQMwzCMTYk9JA3DMAwjAXtIGoZhGEYCgw1KQyQwZUMbMczHxzXWOHUozzsHPmlPlPt1Kk25WOkAVlbQL9YijbLdxM+v1bFBjzyJOgHXLDu9/yTE+/cegLg4MYnrn0WtLKR5+2Ga5HDfJMb9Plf2320Ojh09BXFEvsFJysU6QXVCoxb2qzT1izHSU9KkR81N4PIc1Y9cc1w3lGrp0XW76Tr0Gc5NoS/zni9hLtlKc3A9ymxhC+6vjvrOwd3okfs3P/4miK8nzZG7RTafoeV4vE3y97LU3VcLkGofpslDGHABzxHAbXY0FgXkjQ1JI2y18SQ0SYeNaahdPIfX7ESpBDHngd6bxT6zdQZ18SdPosaY2YNjm4vxHOcVPeTLzUVcXqAaqXS8ns81WGksUjzeVp28spSrtd2KaTnlhiVNMqA+xRpxXylj0iz9IbWOexl97zQMwzCMTYo9JA3DMAwjAXtIGoZhGEYCAzXJmDQtGaKRsebl+4OfwX0aG9db3KBvjzXLyUmcx2cNs0p+Mo5jqivYpLp8R45gHcL50ziv/+UHHoX46quuxe3T6W0PEWW5niRriLy9flsl12zjeLCP8nLRJtNUnjxl42OYWzSXGeyvZY2NT0vUwus6PoEa49gU6j/lZbxOPvkWowA/v3Mr9sNdW6Yh/iKlbnUx59EkjTSL289E6MF79ctvhvi269FPrJRrltJeSkgetcCnM0aCHd/lKbrPOM725ZqVkeMifp8Bj8ope3Epl2iE658+W4J4bRU1wLNnlyFebWAfzNA5mp5BH2S9ib7G0+cwnhlD3TkfoE6dzWCfaYbYnlwa76FGjT3cqDHGpNN7lD/Yo4vM70+wxsj1NpW8wh4L6TTY+R5+Pkv5nkMu5jsA+yVpGIZhGAnYQ9IwDMMwErCHpGEYhmEkMFANcJyPj/1Q5HP0BOfth9GvSZJGxpLkMImSpplZoxwrop8uncJ5f/aDiaJGmaF8jhHpGCFptpUanr+nn0Gf5fbt6J9jrw8fD2uMrCF6gu2J6AMRaX0uxhPKuXRHRT5LGly6SGtQu6ljZqjWHx9ViuoX5rKo10yMo94ScG5RR56vNupNjmoBrq7h8knSWCNaX0KqjZdDfafVwu3t3Y7n5+arMTdru4ntdexPjtlvS+8asJ+2z8+LcZY04j4PG1+QTaCF+5IZuLzZwkbPn0XP9QKdowbVO2yu0TXAoUECDzWzyTzq2I58mqcX5yFuh3jOqytHIJ6bw/y9jTrmtRbFPthqUk1Qrn1LFSDzBRxbY/Ih8ljcbuH54Hy/UUhjW9/7LoN9kkLPLl5/2PsysO91r2kYhmEY32TYQ9IwDMMwErCHpGEYhmEkMFiTJK2ANSuPa3aRN4XzBfIjmXO39u//wuonsl+O95fNog4xPo7z6ry9OukMrfZgr06/bxRPN+cn7JtWH8qQXKx9uV5l4PLNwkQBfYnpDOlFdNgkrUqjhfpGkTxtaapH2cDLKvd/+SmIz54pQeyRZhc2MA+nC3F5eXUS4tUi1X8MqfYe3ZaslcctbPCt114D8ewkapRLyyWI83k8H9wP+HwHffcpacBUazCT4fbjBeqrD5oavVGSZWGyiopPnW6lhL5B9pQ7ureVfIQZyl+bouVbpjA/b4k83KVqCdtXQO9tWME80rO7sU+coj7r+3jNfXq/YaKIfabRwBMWeNinGyHq5hH1GdYIeWx2lBu2TT5SIe9t1Mb2cB9utbHP8fsvg7BfkoZhGIaRgD0kDcMwDCMBe0gahmEYRgJDNMmN1S/k/HoRLff78iFemCbGWkd/vcSNkSZtZXwcc4TyPHqD6kny+eH1uY5eX/7BIfDx8fnnepQx+ddYY+7TJDeJROmU9APqN9kM6iueN1jf4By4EZ0n/q54ah5r+dXXUH9Sn7V3ErTovG6ZQI21Qp6zeot9kejbDCNc/4od2C+/5aWHIB4jzXOYFt0mPYdxlFczRXZi1tr770OuBYj3GfutR0GKco3ySYvZG9p3UjnxNHmoqWYpnUKZKOI15/0trWKuVfHpfQcPr2GzjD7KLQHWtj3VRI2z0UAdP5fF9zOy5CVOUyfgsSYVkKYY4vaH5flWpT5BL2zwo4Pv6RblP85kcOxthTwGJDP63mkYhmEYmxR7SBqGYRhGAvaQNAzDMIwELkiT7NMe+jQvyhHJ8/jkJfI2kE/v2dioJsmaHu8/kx6cz5E1Rz4/nE8wlWIt5sI0VGa4Rntx93fJoH7jKZ7HdhP1Fz+F+kIhj/qJozyQIXm2xscw7+UbX/0iiGPKA3lqHvWhVArP61VX7YL46oMHIf7rf7hfBsE5eH3SZ246hPUh9+9CTx1f5Tb5NqtV1KNYw80X8PzxbemRINRsov7TarJnDTVS1r5DNimOgL78tBSzDzIkDY4kQQk8yldLJ5E1Sk3j8rPnMLdqpYG+w7FxvMrNJi7PRKjBbS9iH3+whX04dvj5ag3b02pyDVUcGyNKRpvKs2bK9TrJI57C/dUp7zVrkDFpitzn2TfZIM1yIzq4/ZI0DMMwjATsIWkYhmEYCdhD0jAMwzAS2JAm2b/CxnbGGqDQvLAO0UCZYblZLxTWDFMpnOdnL09ffUeah0+RRtnfXj7ejWmIw3OxDl6+WXK58nnh+o9pyrnLehEf5rA8kS3KhXr9VVh776q9b4Z4eZlytdJl5PZ+6j7MBXv6+Bn8QJs8ZRFqhjHpVSsrqxTj+hNjqAHyfcf6EGvxLfbMUT1Lj32TXAeWk+nS9Wi1qHZhOHpNMqJ6hjE1OqT3J1izZM1MyTeZp9ynrMM2IuyDDdJ5ha5ZTJpem/IVjweUG7aNuU8Dxf2lsvj5VAo1zGajjPtvkA+U7sFUhMefpny+vjd4rPHIB8qe+oDOB9c0dZRfuVZHT3s2u/7ax/ZL0jAMwzASsIekYRiGYSRgD0nDMAzDSGCgJhnTvLrXl5+Q542F4o1pYDyPzJrgheZq5f31aYLcHvq8R5piwJpkX8rQjWqmF9fHuNH6mqyjjIoCebqyWdRXOM+j76O+wBolH1W/Jon6kJKGlyMNbm4Gc7GWy6gJriyh52xlYRnb1yIPWL+zEaKwge155ImnIT5z9jqIsxnUVGPWaOm+4n7M7wbEVP+xTf2E5CBR8nm2SQ9rNIbobyMgIt8da5Ic9g09fO8H5D3No05cXsNzslbDPsR4nDBXSVPz2MdI9z736TTmP26toE4e+HhNHPWZJvdh8jI3a7i/qEX5l3N0T9OTKO3h8bXo+DgvOGvG/DDq65NNWn8A9kvSMAzDMBKwh6RhGIZhJGAPScMwDMNIQDeLN84wDMMwNhv2S9IwDMMwErCHpGEYhmEkYA9JwzAMw0jAHpKGYRiGkYA9JA3DMAwjAXtIGoZhGEYC9pA0DMMwjATsIWkYhmEYCdhD0jAMwzASeEE/JFX1LlU9Oep2MKr6dlW9Z9TtENm85+j5ymY9n9bnNh+qelRVXzPqdlxsLuZxqapT1YMXY1vPlXU/JFX1+1X1i6paVdWF7v//lKruVdVKzz/XXed8/IoB29zXXX9gya5RoKrvU9VW9xiWVfWTqnrNCNqxac/Rpcb6nPW5UXAp+t1mouf6nm/3UVX9+VG3a7Oyroekqv6ciPyuiPw3EdkmInMi8hMicqeIzDvniuf/dT9yU8/fPnchDRzxjfqb3WPaJSILIvI+XkE7jPQX+QtxMLM+Z31uFIyy310MNtg3JrvH8QMi8kuq+oZn2d4L8jpvhKEnU1UnRORXROSnnHN/7pxbcx0ecM79kHPuQiqmfrb731L3G80d3Wmhz6vq76jqsoi8S1Xfparv72kTfNNV1WlV/QNVPa2qK6r61wnH8g5VfVRVd22kkc65moh8QESu727nblV9t6p+XkRqInJAVa/pfvNfVtUnVPV7e/Y7o6ofVtWyqt4vIldsYPfPi3N0MbE+Z31uFFzifieq+jZVPaaqS6r6H2mZp6o/r6qHu8s/qKrTPctfqqr3qmpJVR9S1bt6lvX1jY20yzn3BRF5RESu1+5UuKq+U1XnReQP1tG2xONaxzl5n6q+p9uP11T1M6q6N2HdN6nqA90+fUJV39Wz7Hzf+2FVPa6q53rbMuwYBrGebxx3iEhGRD60ng0yqvoRTf4p/8rufye738S+0I1fIiJHRGSriLx7Hbv5IxHJi8h13c/8zrO04z+JyNtF5FXOuQ3pIapaFJEfEpEHev78NhH5FyIyJiKLIvJJ6QxqW6Xzzex/qOr5kvH/j4g0RGS7iPxo91/v9p/35+giY33O+twouGT9TlUPicj/K51ruENEZqQzW3Ced4jId4jIq7rLV6RzDUVVd4rI34rIr4nItIj8WxH5C1Wd7fl8b984toE2q6reKZ1rdL6vbevuZ293m4PaNvC4VPXlqloa0owfEpFfFZEtIvKgiPzvhPWqIvLPRGRSRN4kIj+pqt9B67xcRK4WkVdL59fxtd2/Jx7DUJxzA/+JyFulM83Q+7d7RaQkInUReSUtcyJycNh2u+vu664f9Pzt7SJynNZ7l4i8/9k+J51BIBaRqWfZ/l0ickpEfltE7hGRifW0q/vZ90lnkCmJyLyIfFhEruguu1tEfqVn3e8Tkc/R598rIr8sIr6ItEXkmp5lvy4i9zzfz9Gl+md9zvrcC7Df/ZKI/ElPXBCRloi8phs/JiKv7lm+vXsNAxF5p4j8EW3v4yLyw8/WNzZwfUvSeVg8JiLv6Lk2LRHJ9qw/qG0Dj2udfb7380URiURk97BzLCL/XUR+h45pV8/y+0Xk+4cdw7A2rme+eUlEtqhq4JwLRUSccy8TEdHOG2qXQhs5sYF1d4vIsnNuJWH5pHS+DX2fc251g+34LefcLyYs623jXhF5CX1jCqTzTXq2+/+966/7m94ANss5uhRYn3t2rM9dWi5lv9shPefPOVdV1aWe5XtF5K9UNe75WyQdTXSviHyPqr6lZ1lKRD7dE2/k2pxny/njJBadc411tm3Yca2H3s9XutP5sF0REVV9iYj8hnQkiLR0fvX/GW1rvuf/a9J56A47hlODGreei/4FEWmKyLevY92NklTxmf9elc60zXm29fz/CRGZVtXJhG2tiMibpTO3fudzaWQCvW08ISKfcc5N9vwrOud+UjrTYqF0Bo3z7HmO+xn09814jp4r1ueeHetzl5ZL2e/OSM/1UNW8dKYmz3NCRN5I1zPrnDvVXfZHtKzgnPuNns8nXbPnAm9rUNuGHdd66P18UTpTvaefZb0PSGd2ZbdzbkJE3iMius59DDqGgQx9SDrnSiLyn6Wjd3y3qha7IujN0vlpfSEsSmdKZpjQ/KCIvFJV93TF9f/Q074zIvKxbvumVDWlqq/s/bBz7m7pzHv/VffbiIh8w4Nz1wUeg4jIR0Tkqq6Aner+u11Vr3XORSLyl9J50SHfncP/4Q1se6TnaBRYn1sX1ucuMpe43/25iLy5q9GlpfOCUO/4+x4Rebd2X1pR1VlVPf+wfr+IvEVVX6+qvqpmtfOCTeKLTtp5qeruC2zzeto27LjWw7f1fP5XReSLzrln+2U8Jp3ZiYaqvlhEfvAiHcNA1nUwzrnfFJF/IyL/XjqvpZ+Vjv7xTunM2Seiqh9T1V9I2G5NOi8AfF47b229NGG9T4rIn4rIwyLyFekMEL28TTrzy4932/ezCdv4ERH5sKre1u1gFRH52qD2rwfn3JqIvE5Evl8634DmReS/Smc6QETkZ6Tzs39eOnPwf9D7+c16jgYd86XG+txgrM9dGi5hv3tERH5aOr+Gzkjnl3Tvi0q/K51fSZ9Q1TURuU86L0pJ94Hx7SLyC9L5AnNCRP6dDB6/d4vI5we1dwMMatvA41LVV6hqZcj2PyAdLX1ZRG6TzhenZ+OnRORXum34JRH54MU4hmFoV8T8pkNV3yoi1znn/sPQlQ3jImB9zrhcqOqD0nlRZaP64GVFVd8nIicH6PAj55vWKOqce//wtQzj4mF9zrhcOOduHnUbXii8oHO3GoZhGMaF8E073WoYhmEYw7BfkoZhGIaRwEBN8g8eqMHPzDh6Nt9pMk5i+gNbWryBy/t/5Ya0fPD24xiXh337wzDt1iBuBi1cPc5AnG6lIPZ4/x7uINII26PUfj4csgCFnvIKuP+I9icRxbgDPr2e70P8zrv2rteDdFHZtvdOaNnBW18Py6cm0Ya1dwt2YydtiFUHH4aLcf04xPXrjQbGdfTHFwpTEFcb2I/yuSLEgZ+HOI5wfxH185iuWxThdW23sf0u5vsI42YTU5DW63h85WoV4laI+/eon7abZYxD/LzzcxBnsngf5Qp4H33xo++97P3uilf8IPQ5L419ygswzqTpGHJjEGfTGBdy2EcKeXSUdFw7/0gY8ViHN2scY9ym9as1vAaVagk/TzkEHOVEj2lsjeMWLcf2Cg4dwoNZ39gY4v6VxmruADpk7Itjag/dI47jNh7PE/f8WWKfs1+ShmEYhpGAPSQNwzAMIwF7SBqGYRhGAgM1yVwGtQLWTvpre+K8chCj9qE0r86aW+xoezHGXpiGmDW+tofaTDhsnpy0mMk11JI0wHnzlk/z5j5NjAe4w5jitnKMp98PaV6ddQePdAMSApSn5el4Y9IoWZP0g81hm202MN/1yuICxON51PjikDRI6gcypAYta+exYj9jvSc3hppinvSlIIv7U+GYmsf9iPSbtI/tYYk1pPvCV7xvef+xQz2mVkdNMbuMyyfykxB7QRY/36pj3MQEK+2I9TQ83+1wY+86XAomJ7ZDnC2gjqo+nkM/wHOepjjue78A772Y3kdgTdDR2Mevc0SkwcV0jYMUaqaO+nQY4jXx6NZn3Zs1Ut/nZwGNfRGP7bh938P2xTHp8HR8Pm3fo7EwioZopiRi0uUZiP2SNAzDMIwE7CFpGIZhGAnYQ9IwDMMwEhgoQmXoERr1JefBPyjFXp8/bUh2H1qdvTqO5sED8sNJiFpIXENtq11Ff1vjNFZjiY5hPD6O2pO/YzfELZr3jwKap0+jduPIW+UFqB0FdDkC0jl4Hr8t+HnheXthDZhQ/Iu/Sb4y1asnIV5efhJiF5+DOFrF8zQ2jh61Wq0GcUB+UCEPlp8ahzgMWf9BD9qSYA7psSJe99jxecbrzNo+a/UeXRjWY/p8oB6eD8+jfkEaaMpHPSifx/ZlSYNttEmfS6FGXChQv3V0n1JHjPoHlstOPo/XXKmPpNN0r9K9HUXkyXb4PoZE+P5DVGevK3mch2VC63sdgoZy6tOsUUYsEtJYyn0sUO6zNLaQaKrc5/o0ycHLY2WvL67A7wG0Wti+1RLq5Or4fYz1q5KbZFg0DMMwjM2HPSQNwzAMIwF7SBqGYRhGAgM1ySxpIVFf7lWaN6ZnrvNw3p6NeZyvj+1icYDzyI2I/Fdl1IKaC6chrpw+BvHqOdS63PIpiNNnFiHOXnUlxNuuIU2SvDypNmlXTdR6whZqYzFpiFWap2+Po06SzuDyBvkkeXv9qgbP8+PSzaJJ5lKoaeVRTpGlc4fxD2hvlXQaPW7nzqGGyRpeGKEes3PnQYhnpqchfvzJByF2EbZ3//492N5F7FdeQJoee/B8POBUmvzBdN8Ui6gZ5grYb/i+TNH5ZU9co4F6Wpn8w+yTTJGfWgPyI9PxBeT79II+Q/Nlh3129Tqeg4A1yQx1StbQyInXbJMXlyRLzo/LGhz32RR5mtMO26OkIfZdI9IcnWtQTGMLxe02HkCuiBphvY6aYNSi/MKcN5re5/Bi9vbi+ZmaxD5ermAfrdSwveyzlHj9fW6TDIuGYRiGsfmwh6RhGIZhJGAPScMwDMNIYEM+ybDvkcqqF2k97PeiieiAvENxFeeVq2XUcqorRyFeO30cP392HmJ/tQTxZBO3n4tQI9Q2Lp+aRG1r264dEAdFrBE3kUOtJkihDsB+u1oF/XbnSPuJizjvvkY+yTOrqHm2OWFkX5bQwd4r9lWOinYL9YLK8jLEHmnVjvyiNaqdV6tgHJInrR2ivjI9gdc1nsB+UC6hFp5JT0C8RLlmT516BrdHeTNTpMe0SCNkH2WKNL65WeyX6QnsNz7nIeWcvzQMtKqo/VdX8HjTGdzexCT2yyzaJkWzqOnWG2zAHr0mWa2VIG5F7MEmb2ob28z3OntTI9IchXRp9dnriiF7bR1pamETNVX2WHuUu5Vzx/YVUxzy/kk2i2PdlplJiOcX8J7yOLcsba9vpKLTxaV3c3k83+Uq3jOsIadYg43W//vQfkkahmEYRgL2kDQMwzCMBOwhaRiGYRgJDNQkfcrPx7k+tW8mmbw+TdT8GiX0q5XmUVOsk4+xvYS+x1TpDMQF0jDTLdQ4M22cp1ea6A5o3v0c+R5X6PhSIdX5oxyW7RzqBOOU3zEUbN8ytTdI4fbzuYBiFHvqIXqRVtY4x+jg3Lp8+TaLT7LFeRaXMafuzNatEOcm9kEc1lETzK2h9tvguqKkB7Wq2G+jBl4nrtfoU55PL4WaXUyeuuL0AYjzlOt0/sxR/Hw4OA9lGGE/oXKRIh7VdeV+TTmFxWfPH9enxPZm0iEtx3cJ8gH2u8UVbE+jhedrFFSa2EccFWdNUW5UzqMc0r3lk+7rKJcq5z7lXKeMc6xTU71Grk9JPkqPxr44YM867Y/2TzK5ZFPkw6Sxy/fIA09e2FSfD5Ny2TrsU6kUnU+up0ljPcn8fbl4o756k8lskmHRMAzDMDYf9pA0DMMwjATsIWkYhmEYCQzUJD3yZ8WK88Qx+frW5lFDLC8egbh1FjVI7xxqjF69DPFYiPPMBdINvMFWHhGaF6dpawm4HiN5mWrkp1tp4AbSDv1kuTTXoENNsrqGWtfRE5g7tryEWtrWHdvx86QFpcZRm0vR8TSErx/pBlRTzuvzRo0I8ngJ9TsuP5jPoi8w5nqLuyYhTlOu07kxvA2u3Iq+w5nZLRBfsW8fxLU6XtexGdSOZ0/vhfhEGbdXJzF4OrUL4p2zWB9zdvskxKwpthqUI5g8f2sV8kFS3styHe+zsuD2c3n0hfp51He8GDXc6TTuvzWN1/dsna73CGBvappM4n6KanqSbhuS5sdjJyem7nMkx4N/ryjdm1wTNaKCjJHgPRM4uidIU/Ri1lBp7OjTVPF46/R+CGuGnIs1atF7B3T4jv6QonqYzSr2qbCOcZF8ki3K3doK1z/W2S9JwzAMw0jAHpKGYRiGkYA9JA3DMAwjgYGapJIGuHjkIYirxzEnZTiPGmOqjrlUJxqrEOcapDGyf420qZUa+sVC1p6yOG+9VkE/Vq2O8/RT5O+K2RtF3p4c19dsoLazsoDHs0C5ZWuUE/Ppp5+GuLyCOUrXqK7fU8fQR5qm3LF7D14D8cROrGsoHuWX5Jpug7vDZWNqdg7i1QXsV1zbzlOqhUd6Um7yEMTF8RmId2/H7b3+rhdBPEG16yrUD5sh7l/Ig1apogb6wb/5KsRPnUUt2k2gprkaoY/wxbd9F8R+Cmv5eVSsME2XNSStv1HHflmpYj8+cwrfNTh1GN81yE+Ttt+m80XGzVwe9SN/FX2VoyBFJykIWCRDDatNtWNZJ2dNjn2NXG8yYp8j57km3yMv5xcyAvJ15kjDDCMa+0iTDcgL3ApxfTZVNxqkO1Mt3YA0z6ki6tqtNuWhrvH+yMtcxz4VNrH9W2ZwbFyk90GE01wPwH5JGoZhGEYC9pA0DMMwjATsIWkYhmEYCQwUoThnZvnLn4U4VUYtZYLy4WXJmJileoc8LVylfIZVmgdfJi2q1sYtLFGdvybNk3Pu2XqA2s042bWm86gFpbkmWgZPXxTj/gr0+S3TOE8+N4fam0fnr1hArem22/F8lOvY/iCL2lduDOf9y1W8Hms1jLmG26i49UW3Qnz3xz4CcVrZs4XnoRpTLbki+hSPL6CmWFlCbV0j1IZzdB0yVDe0QBqiKmrjuye3QXz7FdiPv/6FL0Bc86i+Yw615a88jP7amel92J4xyrWaIs+aYr9MpVBDzG3B9l0xsRviZ57G+35pDe+rWNFnungOz/eMj+f35mtullFDXUoiFhmJmAT9mPV9Ws6+yT5NcYhtL+axkzTEiO7dqXG897eksQ+HJbwGbWpAhjzeddLhWbMM23gPtlqDB5M5yr989uxZiJXPP8U10hh5/UyA92DUQh+nJ+uvYWq/JA3DMAwjAXtIGoZhGEYC9pA0DMMwjAQGapLZEP1S0+Rnmggx12qtjfPW52o4L51J4Tx2tY7rn6V571qM8+RjeZxXr5OvsRyjZueoBlncoLp8tH2fc2CGuP0GfX5mchJij/ILMqxy5HKkDXFRN/JaZSh/ZIG0t+ktqHkWc3i+ToWoQ0TshXKbwye5soyal4Sod3gUNyn359iuGyBuF3dC7FafgDjl4/Y+8bcfhXi1jP2ca9Ptu+oqbK+HGuUUad9veOPLIK5TXdGVNezHuTnU+J565GFcfwY1yt17UAP1KI8le/aU9LLAG5wnNDc2CfHJs+jfzY1jv0tlqa5qA/vhHj5/IyAi3yDDNTT594XHuU25niT5LPt9jgj7IlmTzJNO3qL2l0voSZ/bgmNNhsaagPJORzSWeuzDpONrhnz8VG+SjicM8f2Neg2fNWk6nzFpnlELP5+lZ0uL8mw3a1RTNWWapGEYhmFcMPaQNAzDMIwE7CFpGIZhGAkMrifp4bxtkbSMAmlmVfL5Pd1EDa9dJ28RaZAe+cvy5NWR+gqGlRLEk6TxMSmS/K7wSSshWWIsh9pSdgzjSgvbv7aA7VlcRA23ThrsuXPnIP7aA5jTs9VErSqbQx/kxDTmIM2RZjtRJA2X5vkLO9D/dtVB1PJEtssoWD6NuULbLew3nDdT03geprffDPHJ5RIuz+J1mSbN8GvzmHO41Ub9Iya95mmq39ikOqQ7tmI9yFT+WyDOjGG/jcnD5rF2Tt9tz1Gd1pWloxCzPiR0n2qfY5lMe6Rdc+3Aegs120kfNdR9e1BzzESo7/3vj/wdxL/8zh+Vy40TPCdxX71B8pr2+ey43iTF7JOMB+dqZXza3swUvn9wjjTIchk940tnMf9xi2u0BniNYzJNBx7nS8Y+xX3M93EsT2fwmrPOzzVPPYftazZwLJyZmMT2kW6+VsZ8xDHVCmZdfhD2S9IwDMMwErCHpGEYhmEkYA9JwzAMw0hgoCYZco00qrEWtnBeN6R56gZpHSFpIezVKdI08RbKfzgZ4ApbU6jRtWle+tw51Hb2z6DGtpt8gSdoXl/J2xRR7td2C7WsL3/5ywNjj+pfZjI4b//4U4chDkkkjUm30L7vOJy/EM9HagLPV4tqtP3sP/vntL3bZRSkPTyvs3OoOdbbqE+0Y9QEvYA0twb6CPcWqf4kearadJ1zRdQMK2slirGfCelP7RbqMT71q3yAHrFGA/NYZqhe5fTkLMSFMdSWWVuPY9y/Un1HoZzDzuM8nKjvnDt9DNcXbF86h7lgT6zg+dy/6zqIU+OoqY4C1tyU8jK7mL2l+HmuD6l0r/Zpjn2+Seoz4eDcp8srqAs78nT7pJlWqniPpDLk44x4bMF4nK5pIYt9Ko6xT6ng+x4eaY7lZXwfoz/3K+WVpofRKtU8LVItYU/x+Py+3LmybuyXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcBATTKIaZ5Y0OcnpJX4NG9/JWmEYYzrj4/j7rM0rz+VwonjNOUqreI0vDRJKJiamsb2kfemFOHxNOls5Gl7GZ+8QJSPsFoh7YZ8kAXKt8gaZWEcvU9+hub1sXniWOOl/IU7x/H4IyHtKcL27lrD9o6Kma143FdcdRDiuz/zOYiPPIn1GOcOXg2xK6OGNrYLr9vjR9EX2aQ8mGGd65KSVo1LJaAcvk3Srl2I/W4iR1eWciBHLdSTatUSbr+N11189PO2SW/yBPWeYp5qAwq2v0J1S1VQo/XJU+eR5FlX/MPZRex3V++9RUZNX25Sujf7bj6PfI6kebH3lOtTxo59eqSBkmbGuVuXlvH9iXHKj5vN4Fjj+1TbV7CPp0gDjULUPLMZfJ+Bx9IMPysoH3FlFXX7FOn+Pvk0s2P4HgJZo6Vaxz7UWMU4X0Rv8gTVfG2S5jkI+yVpGIZhGAnYQ9IwDMMwErCHpGEYhmEkMFCTdBHOMy9VMB+gq+E88OwYamoHfZx3bpK3ZYxykbZI49Q6ajGNFs6b10nrickX2FfzjDTRFvnp6jQvPk6+zIB0BPbeBJRfMWoPrpnmk7YVku4QRxin6PhUB9eso3KbfZql1HCFs59HbU/+lYyEpQXUW2oV1MgKedRaq1XM6Tt/5D6I/TbqJ3PTmKP204uoxTquZRfheQrouu3dhT7F6W3ox90yhu2fIN/lJMVCOYHbtRLEYRvvu4VFvE8mZrE9zifNUPC+CbLUj2Lsx3FMnj/F9mQyOC6ITkKYDvD42thcCevrz6N5qeDcquw55vcH+oyS1Gd4uUfvL7AHm3+ucD3JkPqgkKZXbaPGlvbwXk/R8RWyeE2apHOfIi9s9Swuz9BYn6LBpkH1Hvl9lWwBNdSZ2a0QSwY1xDML2MdYw2Svb0Q6eTFLnvq2aZKGYRiGccHYQ9IwDMMwErCHpGEYhmEkMFCTbJHm9QzVJFsqo9ZzLdWDvMonXyTtzQspn18FxYoopJptbB6iaX2f6lv6lBs2S5phmnSGKiW9jBy2T2meW/q0KlxcKZdwfynMdzi3Bb1AqxXULNmB50gbikk3CSle43qepNHmKcdnkXJ8joprr34RxE8fOQExd4OA9KBnvv4gxG96zV0QF0gPcdTPxwqoh9RLJYhfdcedEP/zH3s7xLlp/HyO+n0mg/0qRXkwp6YmIG6Tz1AVP7//4BUQz84dwPU9bEDgYb9IBdQRFNuza3ILxJUJrAVYyON9vljCcWBp5STEZdJYD5e4NuMPy+WG6wuyLzHFej7nbmWfJHnEPdbQyOMck+8yII+48lCIYZ9uHJLm1g4xLlG+4sYy1nD12+iTzI9hH2il8B6KKPeqlx74aJG2j+djoYQ6u0c1XlkC9un8BuQTZY14jTzs/Rp0MvZL0jAMwzASsIekYRiGYSRgD0nDMAzDSGDgxHGjRnX7GhgfoZyXecrfd2A3aivsNXKUP4/rH/JEfL81ifL/CddEw3l/j+btsxEePucvFNI42bsUkNdochxrrnFNuWYDtaC1VfQD8jeWIEWXx3H+R/oAHX+TvEMN/jxpqueqOG8/Kt75zndC/F9+8zcgPnHmSYiLRcxTubKM2vaOWfRVsl6RyuLnXQvzTO7Ygh6uf/LaN0IctKlfUV5M7tch9Sv1UQMcH8O8k3VaXq2UIJ4kX2SxsACxl0IfaRijpqhK9TXJT5wu4H26bQ8ubzRI4y1hnG7j/sqVpyBeKpNeNwK4VivfG6xZ8mDE1R/Z+Tm0fCG97xDFnBEYYU82v48R1vAeaJJHW3wcGwoB7n9iEn2FzQm8R87Q+yJxSKMX174lDbHlSLONOf8v9jmfNNB0enCN1nZf7V3yvVo9ScMwDMO4cOwhaRiGYRgJ2EPSMAzDMBIYqEmePfYMxFOkkWmdcpFGqMnxtHpMmp/GOO+cY0mSvUQ0j+y4jh0pAx593nPYfo80T49ypToqYtYiX6dPwsPV11wD8Wtf9zqIH/n61yEeJw1zKofz/lXKf7i2hpphq0F1CumEezk8YQ3Kn9ikmmwLBdQhRsXCuUWIX37nyyGurKHG9eDDj0E8VkCNa2KSOhaFedIk85Tn8c1veDXEBR/Xf/IrpyDefT1q8Vt2oXbNOYZdjHGjSfUs6caoUU7jZ55+FOKtuRLEs3vRZ3q0ippu5FArD+hGS5Gek6PcrKUy1u+sVK/D9YuzELsKar6ZADXOUcAaJGtqIXmolXRtpVqzUZ+mSWMZ15sUft8A10+nsU8HdC+HdM+4VewjrSaN1UXKMx2wT5PGQvJxxjGejzQ9G1hj9NPonQ1JQ4zIx8kaYoq2HzdxbOT3RdoRtw81TOUClQOwX5KGYRiGkYA9JA3DMAwjAXtIGoZhGEYCAzXJpdPHIS6SllIgr0oU4jxvaQ3nwfM0D51iLwvNI3s07x479uLQvDnN4zua93fkbYpoubLmSbrC008/jZ+nepoe6RT79+2DOCCv0P79+yGe2bYD4hNnMFfuh//mIxBXq6jNscaZyuDlrVH9zRL5IjNXY3tGxT2fuxfi8XH0Dd5w/c0QN6p4XBNj5PmaQg1RSb9482tfDHEU3QbxVbt2Qlybp7qkTfKAUd3TWh01v0oL+/nC2bPYPvJZCtVZnd2C11kEj2/rGHbkXWN4nRtrqLnGKYyjiGoN1skT18TlURk/H1KezHB8EuJUFn2r+3ajXjUK2AcZ0NjG+YL7a73i2MS5QftzrdIfPH/wchqqQ7qXq6cx9+pUgOc0lSINkvrYGtX6jRWXBxkca/JZzE/M9SqFxnJNcf5gPh78uC+Dfaoe+URZ421SLV+PnKvsIx2E/ZI0DMMwjATsIWkYhmEYCdhD0jAMwzASGKhJRk2ct/VJA8xRjsnTJZzXPjiJ89aOcp22qQZZpDxvTvn4PM5lShok+TB9LsJG2/do3rtFGmSKdIcW5ROsrOI8fZbqAnJ+wYMHD0J84ADW/ZuaRT/Z+NQkxJ/6+09CvEbz8lOk3RUKeL4FS8RJi7xX3hb0940K1nbL5TItx+O+8+WoKe7eg+exHWI/LmSxH73oJtQc//pvPgbxkUcfgvjN3/qdELeo33mkBUckSLVa2M/qpFlmU9hvUylcf3aMPXuoWZZL5KOkOKhvx+XzmIN5gd4lyBZ2QRzV8Hir5aMQ15v4LsO+G7EW4a6tqEnmvdHnDFYyPSvVjwzofYi+PNSkKSrHPmucNHb5rFqS5kb790lzzE7ivbt8ht4noT6ZJR3bVzzeZfJkj+dw8BjPYp+NFbdfjVjTRN06oPqanHc7CvHZUKP3P9iXGdH7Lg0aq2OqrylNrt2bjP2SNAzDMIwE7CFpGIZhGAnYQ9IwDMMwEhioSZbLSxBnajiv2/BwXvzYWcwfeMMO1CKaeXwmt0jzi32e98d56xY3V8lMRJ9X0lDZehQIemlatL0czZPv2bsH4gXK3Xr8xFGIb7rpZmwPeX04btM8eo40zdtuxu3Nn0UtKZPB66F0xNNF1C12bMc6iemZzaFJskbH+g+ftzW8jBIpabERanJxjNtbq+J5Okl1UlVIn9mK+89P4v5WKqgRluZR/1hewTqiLcrRWyqhBsu1/8oV9FW2IvIl0vFyndSdk6jZrvJ9rVi7r5E+B3F9tQSx55E+Ra8CtEizHCPf5K3XY67XUeCTJukoN6kq3osp0tRCGgsiyn/LNkKfPNPOJ8+2ci5Y/HxM71sUd6Bu3KyVIC6dQ8/1OHvMaexsh1Srl2rTTvrkac/h2JPhXKl5fF+i0SbfI93jIeW6bbbwnqrXqRYxjX0+ba9FeapTdDyDsF+ShmEYhpGAPSQNwzAMIwF7SBqGYRhGAgM1yUaDfJLkWyyT3yukmmE07SzLlB+wWcd5Zl7fRThv3OT6j9Rej5KvBrS9mOa5swGu3yZv09mFBYjDE1g38MwJ9CJ9/WtYL/KGG26EuFhE3+iJE1jn79wy1tnryx/J+QkbqAXlsnj+Czn0JqWyJBYVcPtV8q2OCq7Fx3GjQZof9UPvq/dDfONW7CmFDOo3QRa18+/6zjdBPD6B5zU/gfrKhz70dxB/4f6vQbxaWoG4Qj7E0grGbfJ0xY76Bek9xQm8zltJc5ycwHjPPvTn5tL4+a98+csQpwI8v7e+4kUQf/qzn4Z4pYLtry3ifVHw9kL88ENk4JV/IZcbj2yKSu9LiMNjouy6EpG3l3177JtkVyStzqlc+/NKky5fD1BHntiF53itTWNvDc95m3yYU9THs3SC0jHeg8UU1cadxHvKFeYgXlzFsatWRZ2e60c26fy2yXzs09hYIF9oo4Wfz5BvdRD2S9IwDMMwErCHpGEYhmEkYA9JwzAMw0hgoCZ5iDS1h+9BbUWiEoR7p2cgHitMQhySN8cjjTHL+ftIF2hTkbWYfZAUN0nL8sgfl/JxHn+lgUrDZ//8QxCXP/55iEOuV0m5YJ955ijEN998E8ScC7YZ4v45d+vc3DaIM1ls//GjxyBeOI3eKMc6SQ4v/zX7ULsaFbkc+g5DOi8PPPAgxI7qIU5PUU5eh9r69Cz202Ie9Ynpa1GzzOVRb/nox+6B+Pd+709xf5wnkkiTdh+Tp258BvWguTnMtTqex+u0ZQu2N0161loLj3+eNNHyAvab0jJq8d/6ypshvvUW1DS/9siDEO/ei+299bZbIG618D585LHHZdR4dA1SJAKqwz4Ykve0QRqZ+pQXmUTGkMY6F+G96Wfo/Qs3WNNsCNWjTOE9lJvCPh+Rpri6hBrl5Di+P5HL0fsRlFuV83oH7GWuYx8Maeyr17BPqkPNkmsHj9H7HU16fyag3LhFel+jXbV6koZhGIZxwdhD0jAMwzASsIekYRiGYSQwUJPcthNzlT45txvi5mPPQLx4Bv1cH62R94U0PA1x3rhA3hie105TvsQ01UMMfPy8R/kRA9IdCmR2Or6G8+RHFzFnZaOGvs0GaWVZEoM+/GHUNJeWcHsrK6jxsia5ay96nZpN1AFi0lynpqlOXwrn4ettbH89Ql3i6aePymaAc7MGAV7XTAb1kTPzeF693ddAvJTF46wt43VOPYH+1/EpytOZxn79sY9iXc8dO1Ar3rZ1CuLDh09C3GiS1k5+4Mlp9Jht2bYD4sWzeF+tHMX7MCK/a8x6mqN+XKfaeuSLDLJ4n51exJzO4+N4vLe/+A6I/+n3Yv3NUgnbf/vtN8ioURobfMrjzLlcHWuAXFOUPOUtyj0qNDbF9L5A5FGyVoftiykZbKy8HK9ZKosantDYkE6hBri2hhplZRx1eb+A13ytgX0qlcI+uEZjVbWG5yMmjdSRrp/y+X0S8uyT5ikNzH+cz7Cmyk7XZOyXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcBATXKGtJDrbrsd4gcefxLiFZo3f6yK88rs0/NjqsuXohpg5J3xySeZpRpi01M4T57J4Ly8o3nxVgm1lZUGzqNHYziPXyftKCaNtd3G+LHHHoP4a1/DnJ5cQy2g48nk0f+3dw9qlNdddwjbQ+cnTdvj7Y+l0I936ghqe6OCzwvH11yDmuMTh/8W4g99AjXDm6++AuKbbsd+sncM/abTM6wpPgVxaRU1yh/7sR+C+NqrDkL8x3/81xB/9OOfgpjzdFYpx/GxY5gjOJMuQDw2hnpRrYJ6y9oa1tKrUd5O1iRTeNtIhfzD8+dwe0o+1Ykp1MZzlEO4Tdp4Oj1wGLoscO7WiPIDK+V59gO6d0kzC1KkOdL+wpj/QjVTaSl7aWOqh0iWc3E0tgYp7DMR5XpNke7P9TCX1lDzi4rkQ6Sx26d6j6Um9hnW4V2I6/OvN0ee+SjEz3vki0xR4m6flnt9TtNk7JekYRiGYSRgD0nDMAzDSMAekoZhGIaRwEAxwFH+wUobtYkoh/PaM7swh2R4BuehUx56VTgf4gx5cWpNqrPnUz1I3v8s5rTkHKAe+e9qpHmWDj8NcUDakCNfY7GA8/xcv5G1pQxpglnKvbpn/36IZ7agX65Bmuki1bssrZQgjkj78chnOjE9AbF6o9eGRETOnMGcs+zZKpfRA1Vr4HVZK6F+8pWHD0N8w0tuhnj3XtTeY/JoPU7a+/g4armFIvbrWh3bd+VVqIn6n0RNskEeuqCB2/PI85an+o/nFk5DXFrG81EnzTEi/UdIrwlJT3v0MTp/N9wM8Svveg3EO3fvhPjUKfKh0vm76SbMaTwK1HEeZlYF8fdESLla2ZPN9SN9DQYuj6leok8ap8eqJnldPcob7UW8PxxrmkLva9C9z3mw27R8jeozSptz3ZKGSaJpO8KxPyBROEX7q3OfpdyxXHtYQ7ynCnkcez1/cH5lWHfdaxqGYRjGNxn2kDQMwzCMBOwhaRiGYRgJDBahaJ6c43HSENdqOA89MYHzzGkftZaTJ49C7ATnketNzCcYelRTzOH+C03UCEPSPIMA56VDmrenEmfSWMP9Z8nnOUd+sO27Mbct+yRZk+QcpZUyam9jRdRuYtKEGzVsX5oMbgHVUKvVcR7/8UefgDiLl2dkfOhDmPO2XsfjZJ9dqORpI72lQf7Yh7+G16XVxu1zbtFPfeqzEG+bQ+09m8PzvLxcgrhKtet8yjHsUz+sUb3HtRXsF2dPom+yH/zuOzGB/XTrVuynu3ejJnvV1VdBfO2h6yG++aZbId5PWnqO9J+A9DpHfupabf21/S4VrQbdSxnU8NqkuUWUK9VjX+QQG16W3l8IY+yjPg+95PvjXKYx+wQpt2s6xHsipD4Rk8YakQbqF3H94gzWDNUABw8eiziP9GoNvcauheef8w07GvtblHs1Q2NdRGNjhcYQn42xA7BfkoZhGIaRgD0kDcMwDCMBe0gahmEYRgIDNck2zRPv3Ynz0N/9ljdDfPTkCYg//rl7IV6toG+yHqBGVpxEjfGKbaj9COWQbLOISN4bjfA7gHMYtxucsJG9PriY8xs2Kb9jSPPkKZqXXyBfI2uSq6vor6vQ+Yopf+Ey+TanpzHn6G7yq5VJ61qYx/aMT6CvdFSUSiWIlS1rTEj6jFDu0soixPfdcx/ED3zpAVy/hteh1WK9CHP6zp85C/GBvZi79atf+TjE1SrlCHaccxf1ndlteF337EFN8cAVqAkePHA1xLfcdDPE+6/AHMBbZ9GPWxxDvSwgf3Od6qqW10oQn1pCX2RpFftdjfQirou6c8cb5XLDOmlfTLlWHY0tnAqUfZdRTGMV5V71uT5kODi/bRxyHmzURAtp+jzlnp3OYa3g5fwMxHWqn+nncWzOTs5BPDWDuvbMDGqY5WW8R5afRi90m3K7+vReAfsqIxprI/KhRmlsLz/LONfrIOyXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcBgnyTNo+eohtr4FpzHnqX6iydP4jz0pz/3OYijGvkiyZ82vv8ANpaMfCuUo/L0acxhGZCGWCxg+zJZ9FX6VBMtXUSvVKWG8+ZCOgHnGGU/3/Q0ztNznUT204Ut/HyziVpWndZfIt2k1lcnkGvI4bx/7IaJf5cHstX11clkvai/1bh+SHkfV1bxvGbSqMU68qA5uk6nzqKW+1cf+nuID+5/BuLVCl6nV7zyFRBffyP6EG+8/jqIrzyIGidrkjN0HwbkR2418PjLlRLEi2fnIX7iSbwP10hDjUM8vwF5+AoF7FcF8lNvmcP25ug+HAlDhG9+38AjzzX/2vDI6Bizhka5R326ZkI6bUBDtdJNwvUm1aEGp3SXZLOY5zrr4djI9STz46iLp0ijLM6iJpnO0/aW8L0ApXuabjlpNfEe5bE04jziAd3zVNs3RRqraZKGYRiGcRGwh6RhGIZhJGAPScMwDMNIYIgmSdoDGQd9yq83lkEt4jUvfTHEM1T/sVJFP1rEXhhqznIJNcF2FTXAqIEaXKWOmmeL6kfO7UKv0OxW1AxzNWzvymHUalgHqFH9yApplAdJW+Kcl+02HnGN8g0uL2O+w0qF/WeUA5N0i527UePlaXnHxtAR4ahf9alF1E5lDZP8rlwbMKae1aZ461bUV8bGMIcu5+C9hnKdvvg2zG36kz99A8RXXonrT5Bmp6SpVskvu0xa/DPPoAbK2nhI+g1/NQ5IEMoUUU+a3bUN4pnJSYjH8qjppgLK1Up5MtukF9VI8xwF3MfYw8y6eMD1IinmtNecK9SRD9Ijn59P965HvkufWtyi5fUG9ul0jjRCumn4/ZJ6CfvQzBb0yM+RZ75FvtHSyhLEZfKAt6l9fl8tW+yzcYtiuh5CuV3rZRwrW20cK/N0Dw/CfkkahmEYRgL2kDQMwzCMBOwhaRiGYRgJDNYkad6bfX3sR+N4bitqLd9610sgblHu1XYb43qDfIF1XL62hlrNahk1Q84BWqL1q6TVcK5UL0JNczKLp6tRxu1XSQsLm+QDreA8f7NK+QpT6JXySdjIZrhmG7YnlUK/2ZVXXwPx3DbUEaIQdQHHCShHxLA8mgz3y0wWteStW7dCzH7VSYpf+7rXQ/yyO18GcUD1IEPS0tOkJ3E9zEcffRRi1pK5jipb+PJ5zK06PjEB8XaqD5kjzXCsiPcl+xQj0neE8n6mPXz3wKOcyCHpR2ureHy1KmvtqLEe3Ie5ZS8HHvsOI+5zVD+Scq068jVyrlHx6X0O0iAzpONyPcmQ+kRMy3l/+TRqjOrjNS5T/czxCewT4+Qpn9uC+X0ba3jNTpzCvN1hk3R18sA3a+QDTfN7BuRZT/E9h59v0PG36PrlfNIgfezDg7BfkoZhGIaRgD0kDcMwDCMBe0gahmEYRgIDNUmWgrjunUeamVLMT+As5SfMUi5W1ijHyT/mkZcmovqKrGm2SXNsR7i82mafIcbs7Zk/ey3Ep06dg/jMaYyXyNfYopppTz32MMRCOgfnT+RctPt2oE4wPUO5dEmDzOZRl+D6mEFq/fP0l5LZWcwrmcuhpjY1hXkk5+awth1/fscO1OjGyffo0Xm44cYbIX7VnXcMbO/dn8GcxJ+7F+uosq+S28/tu2Lblbj+DK7PmmQ6jfdRQPeho/ukRZphSNq0ROQRpLqr1RZq7Y0qxvOUQ/nUKfRxPv30UxCfPo31J1/1ysHn+1LQP9Zh7GfwnLM5l3Ovtim/bZbeNygUUUdOpVEDzGVQV3d0zttt1BQzpKnObdkHcY081e0K5h+ulHDsmpjAsaW0cBziZ04chbhJuWnHC9hnMynyZU7iewDq4dhcqaKG6YS8t3TBIhrbfTp/vqAvNeRksQOwX5KGYRiGkYA9JA3DMAwjAXtIGoZhGEYCAzXJMvv4aHma6kuyt8dj7w/FbMtjv5vn8Tw0rs+5XpkM+QqzpGkWqH5iaivOo7MXqlrDXKvVCvo4q1Wc914poZdocRHn/at0fjkdodL+s+T/Y22KNcvcGOoe+SJqcaxlcTwqfuAHfgBi1iQLBdRWud199SbJaDjMh/nUk49DvHUL6iesIe7di/UdCwW8LlNT+HnW2tPUT5Xuq5hulJBq63Fu1iblxWyR37gdor5VJs/b0iLm3VxZwOUL86g5lpZQ31olfcvF6JMsFvF6XncF3nejwEujbpwtoL6fpntNScYt0r0V0VCXJr2fNTa/MAnxzDTq6kGM13zxzEn8fJ509iweT5NymTapxmqGjmfh5JO0Pq6QKmL7xgoYZ7J4jSMf+0C5gn3KUa3cIMCxrhXSsyAkTVaQ2OH2xlI4uDaUPf7J2C9JwzAMw0jAHpKGYRiGkYA9JA3DMAwjAR2WF9MwDMMwvlmxX5KGYRiGkYA9JA3DMAwjAXtIGoZhGEYC9pA0DMMwjATsIWkYhmEYCdhD0jAMwzASsIekYRiGYSRgD0nDMAzDSMAekoZhGIaRwPPiIamqR1X1NaNux8VmI8elqjlV/RtVXVXVP7vUbTOenRdCX1TVu1X1x0fdDqMfVd2nqk5Vn7VCk6r+gqr+z4u1vcvJ8/XeuaCHpKp+v6p+UVWrqrrQ/f+fUtW9qlrp+ee665yPX3GxDuBS0tPBzrf7qKr+/Iia890iMiciM8657xlRGzYtL7S+SO08paq/rapcrW7ToqrvUtX3j7odl5JR9Dnn3K875y7JF5xNNt49J1T1far6axdzm8/5IamqPycivysi/01EtklnAP8JEblTROadc8Xz/7ofuannb5+70IZfKNphvcc/2T2OHxCRX1LVNzzL9i71N7W9IvKkc+5Zi2huhm+Ko+L53hcHcFO3za8WkR8UkX/OK3wzX/dRshn73EXsC5thvNs8OOc2/E9EJkSkKiL/dJ3rOxE5uIHtv01EjonIkoj8RxE5KiKv6S7zROTnReRwd/kHRWS657MvFZF7RaQkIg+JyF09y+4WkXeLyOdFpD6sTSKyr9v2oOdvXxKRfysid4nISRF5p4jMi8gfraNticc1pB3/WURaItIWkYqI/JiIvL17HL8jIssi8mvd6/KHIrLY3c8viojX3YYvIv+niJwTkWdE5Gf42J6P/0bcFzMi8t9F5HT3338XkUzPZ/+9iJzpLvvxjeyb1xWRPxOR/7unT/6YiBwXkc92l/+oiDwmIisi8nER2dvz2deKyOMistrdxmdE5MfX2Q5fRH6h26fXROQrIrK7u+x3ReSEiJS7f39F9+9voP760Kj7yfOlz/Vc33/R7TdnROTnepa/S0TeT+t+oy90r9dvde/zIyLy0+u9z2WTjHfdz+akM14d6/bbe0Qk13MvzHf//lkRua7793/R7XOtbr/7m4tyvZ9jJ3mDiITrOfHP1klE5CMi8vMJ6x7qHuArpTMI/XZ3X+cHpp8VkftEZFd3+XtF5I+7y3Z2L8i3dS/ga7vxbHf53d3OdJ2IBCKSWm+nERGVzrfEmnS+2d/Vbdd/7bYjN6Rtw47r5SJSGtCWd0n35ujGb+9+/l9125eTzgPyQyIy1m37kyLyY931f0JEHu22bUpE/l5eGA/JUfbFX+le760iMiudL2e/2tOu+W5fy0tnUHlOD8luO+alMxie75N/KCKF7nX/DhF5WkSu7faFXxSRe7uf3SKdh9h3i0hKRP6P7jH8eHf5Hul8odyT0I5/JyJfE5Gru/fATdKZ8hcReauIzHT3+XPdNmafrb++kP5d4j53/vr+cff63iCdL73n+9w3zmtCX/gJ6Xwh2i0i0yLy6fXe57K5xrv/Rzrj9U7pPPhfJt0voNL5Qjgm//gl9cGez71PRH7tol7v59hJ3iqdKYXev53/9VYXkVcO6iRDtv1LIvInPXFBOt8Mzp/cx0Tk1T3Lt0vn20MgnW85f0Tb+7iI/HD3/+8WkV/ZwHGe7zQl6XxDf0xE3tFddle3Xdme9Qe1beBxraMt75L+h+TxntgXkaaIHOr5278Ukbu7//8pEfmXPcteIy+Mh+Qo++JhEfm2nuWvF5Gj3f///4nIf+lZdnCD+3bSebitdPfza9L54ne+Tx7oWfdj0v0y1I096Qxue0Xkn4nIfT3LVDq/CNb7S/IJEfn2da67Ip1pxb7++kL6d4n73Pnre03P335TRH6fz2tCX/iUiPxET/y69d7nsknGu27/rZ/vS0PWney2eaIbv08u8kPyuc4rL4nIFlUNXFcjc869TEREVU/Khb0QtEM6UzjS3W5VVZd6lu8Vkb9S1bjnb5F0NIG9IvI9qvqWnmUp6XybOs8J2Thb3LNrgYvOucY62zbsuJ4LvceyRUTS0pmeOM8x6XwTE96/PLfzsBkZZV/cIf3ne0fPsi/3LHsu5/tW59zTvX9Q1Wfb3l4R+V1V/T97V5XOtedjcKq6kbbsls5Duo+uLvfj3X04ERmXTj98oXMp+9x5eq/RMen8olzPunyfH5ONM+rxbouIZOVZ+l335bV3i8j3SGf2Ju75zOo6t78hnuvF/IJ0frV8+0Vsy3nOSOfGFBERVc1LZ0rnPCdE5I3Oucmef1nn3Knusj+iZQXn3G/0fN5dxLbytga1bdhxXej+z0nnW9zenr/tEZFT3f8/I51pkfPslhcGo+yLp6X/fJ/u+eylPN+91/6EdGYJevtdzjl3r/Qfg26wLSdE5Ar+Y/cNzXeKyPeKyJRzblI6g9T5p/jFvM82G5eyz52n9xr19qtno/dcw/XufvZicbnGu3Mi0pBn6XfSeYHt26UzEzYhnV+/Ipew3z2nh6RzriSdl0n+h6p+t6oWVdVT1Zul87P6QvhzEXmzqr5cVdPS0X162/keEXm3qu4VEVHVWVU931nfLyJvUdXXq6qvqllVvUtVd0kC3VfV777ANq+nbcOO64JwzkXSEc7frapj3Tb8G+mcE+ku+9equlNVJ6UzwD3vGXFf/GMR+cXudd4inSmm3vP9I6p6bXeA+KXeDavq21X16AW27zzvEZH/oKrXdbc9oarnbUJ/KyLXqep3dd9IfId03sZcL/9TRH5VVa/svhF+o6rOSEcTCqWjlwWq+kvS+SV5nrMism8Db5A/b7jEfe48/0lV891r+iMi8qfr/NwHReQdqrpLVaek82LNN3g+jHfOuVg6csVvq+qO7lh+h6pmpNPvmtL5NZ8XkV+nj58VkQMXemC9POcO7Jz7TekMwv9eRBak07j3SmfwvXfQZ1X1Y6r6CwnbfUQ6b2R9QDrfRlako6Gc53dF5MMi8glVXZOOcPyS7mdPSOdbxi9I5+Y9IZ0XDwYd527pvCV6MRjUtoHHpaqvUNXKBe7/X0nnrbsj0nkb7APS6WwiIr8nIp8QkYdF5AER+ah0BrnoAvc5ckbYF39NOlOqD0vn5Zavdv8mzrmPicj/RzpT/U9L59eHSOcGF7mI/c4591fSeaHiT1S1LCJfF5E3dpedk87U1G9IZ2C5sne/qrqn64lL+sXx29IZeD8hHY3096Xz0sbHpaOFPimdKb2G4DTf+YQXS6r61YtwmJuKS9XneviMdPrNP4jIbznnPrHOpv2edK7NQ9Lpj39Jy58v492/lc499SXpvL3/X6Uzjv+hdPrbKem8iHgffe73ReSQqpZU9a8v8Pg6be2Knd+0qOqD0hGfL1QffF6hqm8Ukfc45/YOXdm4YFT1Wuk8vDLOuVBVPyEi/9o599iIm2Z8E/HNOt5dCN/0D8lvFlQ1JyLfIp1fBHMi8hfSeevxZ0fZrhcyqvqd0pnuLIjI/xKR2Dn3HSNtlGEYG+IFpxcYiah0dJQV6Uy3PiakkxkXnX8pnWn/w9KZ1v7J0TbHMIyNYr8kDcMwDCMB+yVpGIZhGAnYQ9IwDMMwEhiYccf3czAX27Gv9Mb0Ac1g7FoQZot52vsYhM3SGdxcvghxdgqTeWzffS3Ere3oVV0++hDE+eI07q+B7RsLSxD7qUWIazIJ8XizAfH+fZik4tEHmxAvnlWInY/H70Jsj4vwfKufgjgO2xB79J1Hc7i+q9Mb12RhcxFuL44XscGXibs//nfQs7IBdtNMNg2xn8UKUuphXCvXIB4bH4fYUzzMKEJXzNQ09psa9Zsz8/O4/TG8ruk0tjdDMV5lkZj80GtraxA3Gtjvzp07B3Grhe3bNoe2yHQG9z8zg/cNt5+JYmxxTDGfv2w2O3B7tUYd4gP7rrjs/e7t3/kqOOmnz+I5veoQ+trHJ3Fs+tqTX4f4yFkcy1IBXXMaS7kKWtqjscLhvUxdXNI09OZyeM/kaewdn0U7Z7GQg9j3MPZi7FNBHZPbVOrYRxeXsA+0WniN62vYvvIijkXf973fC3E2j+fjs3d/BOIf/6n/CPGhG7EaWb2J+48iHKuvPXhlYp+zX5KGYRiGkYA9JA3DMAwjAXtIGoZhGEYCG6oC0m8W4b/wMxc3H4c4D5yieXPPw9iR9pOmeWVN4/5i0nqm4mX8POXXTfk4T59aOgXxnquxvWtt1PRKxyGUY4dpHj4inWEMdYBmGTVLTKYvIj4JD440R0Wdwi/g/tPjqAXVY9QcpY3n1xMSNkZElgSWkDS4bCZFMbWb9B0hKXysiHoSa3ztNp6ngK5DSJqfT9puIYt6TpDC9ippoOkU9tuTp1HjfPLJJyF++mkoDCJHjhyBmG/Lffv3QZzP4wk5sB9TXV519dUQT0xMQFwoYr/yU3h+fDpfnkf3KWua4bMVnLi8PHb0GYhV8BhSdM1iGotcCk+6R2MTj4zK7w/Q9v2Al+PnHZ1TD7uYCA8dNLbE9EJJTNkpfcVrkuZ7sor7DxT7VCGL229V+QBo7MVbRu6/D7PNzUxgnwvbeIBf+sInIb7y2hdBnMvjDsIN9Dn7JWkYhmEYCdhD0jAMwzASsIekYRiGYSQwUJN0jie2SexQmocnLcjRMzh0qPWk06Rd0Lx61MJ58iLN8xdI03SF7RA3aP3KCia+b9RQC9qWJa0rj/PwcYjz2plZPH4/R/P4BWzfU0/g8QQpjNsRa4yoBcVNPH+OtJ3UVtQ8WZNsNLC9WqoKwo690RCSJujCiGJsZ7OB2m6GNEHW4FgT49SMrA+1WqxR4nUdy+N5V5bqY/xDSP7WpWX0nH3hC/dDfM8990B84jiK4eyLHB+fxO0v4fZ5/Ue2PQHxtYePQnzlwSshvuXW6yHO5QZrrv3Hj/dJHI2+WttqrQxxIcBrujx/FuKZHejZbtK9GQR4DtiEF9DY6HgNOofsi+yzqMvg/fEnIuqDdMtJoPS+CAn7jTLuodEkH2YBvcXLEZ6/u177LRAvnDkBcaWEvsuXfuurIV6tlCD+6r0fhfipR78M8XW3vxJifrYMwn5JGoZhGEYC9pA0DMMwjATsIWkYhmEYCQzRJHlmm3IwOsyJ6YR8dwFqeukM5yOkfIbk+5M0zRsHpCVRPsFUFjW8Ugt1BXGkYZJ/zQtIW2ni8U5PYnsXV9ErVVtDv938KVy/Xcbj6dtfBtsbsSbMGm4Gz0dI2lsUktbGx5/FHJ1xizXK0TA5jteRBZM0+SS5F6fpusbUjViTzJDPMgj4tsDzmKNcpAEJRvx5j3yDjx0+DPFHP44erycOo2fv3CLmET11Cv28hSL5b5uo0Z5FOUiaTbxvVldRswyoXy4sYA7j8TE8/muvQc0ySFGu3QydDxbUyN88CnzSAGcm8Jym6JgqNbz3G+Q5TtM5pC4nSmOrT7la+3RdCj36g8er888fimPSiWO6SULSicM2Hu/E2A6Ia6TpXncD+hRvejGOTd/6ra+H+JlnHoX4+AnMffsd//RtEK+u4T1x+ImvQPzAl1DHv+aWOyDO5NbvCbdfkoZhGIaRgD0kDcMwDCMBe0gahmEYRgIDNUllwxdrWpzrk3OLChsfWeMkzYy8SSxdUFlAya7RH5Tq2OUpx6TD2GtgDs+ZPGpfe+bQb1cmH+FKDXWIpUU83kaMmp+XpxpyIWpVMeV6zef5/ODyepvENqo/Gdcpv2QbtSrHwgXVkBsVrLewhsb9knOfpsjHuFbHfsKaJNd75Fyj3BE516tPmiR71lZLJYg//rG/g/hvPvI3EM9sw/qPUYy+wkoVt+cHXH8Scwyz39mRHhVRHdE9e/dA3KA8pefOoUaZTl8HMfup+XxwLly3sRTSl4RsCu/VbVu3QpwqYB+pxKjjtqk+IdcoZV3asUbJt2KfhkjL/cG5Xn2fPNwe1/zEax7FHsW4vEpj3bV78ZrfcNtOiK+4Cmv9Xn3oNojzWRzrt26dw+0fQp9kLo1jd2YGPfF3vuoNEP/DR/4S4jPH8T2AfVdSnx2A/ZI0DMMwjATsIWkYhmEYCdhD0jAMwzASGCwGsPmGvDRcc0153p0m1kNKzkpWIvFzqIlRqlfxSTtK07z7agu1k/0Hx7F9lBv16w/i55uTuP9VsinWa6j1hDRvP7ENt1eh+pIea7ZUhy+m8zc1i5pvROezXicvFs3bqyMtL0X5Ij3cvhej1jYqyqsliAOWxmPSZquoOXqkUXJ9SM5dmiJfJecWZY2yQVqvR4JSTL7Lxx5BD9jDDz0Eca2O/tSJCK/D0vIKxO0Ql0cxae2kP+XyqL17lJczCLgeJITSbOL5jchDx77RhcUFiBsNPF+Tk1O4g/5CtZedbJru/TXU71seamQVh75A51F+YRobSUIUn3K7subYd0poLHakebZJt87S+xdBSJ5xuqnYS+zTWJGmsWJiDMfWu177JohT5JGfHMNcruzTDCgPdyGH2+f6lx49ul780rsg/uw/fAziL33uUxDv2HUFxEJdEvdlGIZhGMazYg9JwzAMw0jAHpKGYRiGkcBATdJLk5+qTTPlNI+tWcodSqv7NO8bzKAfLM6g/ypeRl2AfZohaUGrVDcvvxW9T9UVyrVawc8/E+D+dA2PZ6aJ2pHXxrhRwfVbZTp/NA9fmEGdoOBTblpSJmo11jjJt0q+SymQLpFB3aK1SpolrT8qqqRhpUl/aTRQG46pNh77JL0ANcrVFdT4OM8m+yajNp43Lhi5VkF9yvcw9+zxEychPru8DHFMmudZypXKuVU5r2aK2lscw/0HpA9t34Z5N4tF1CyVBLRzZzFP5koZ9TlHPtGQNN+QrmexgPdJm/KejoLZKez7YYxjQZl02ZUW9gmlWrqeT/o/iX4x5W5VOoeO+lhfjVLSQMfzWN/S1XCwXavg8Y1PUL7hAtbaDcnTfuiKmyC+4faX4/bGZyDO5XDs5dq3/blpaezqW598nHTP79yO3t4XvfQVEN/z0b+F+CUvx9yxe3bskiTsl6RhGIZhJGAPScMwDMNIwB6ShmEYhpHA4HqSKc4HiPPaVM5R1Md5aM4p6XuoJaWo3qHkSdtoUi5UkiiXmpSr1EOt46kF8q8tojYS1XFeu57D7S2TZrfaRB2iTn6z0jJ6rWKf6kVOYfuapOEKaUdVkgibMbZflih3bYQnKBKq15nG6yNawuWks4yKiSnU1Opl1OSqa6iJ+aT1Nit4XvJb8DxnM6Q/cb1K8k2yfOIo8Wa9gfubGCM9hrTlBuX5ZD9xqYTHy7liubhgm3L4FgrY/okJPJ+ZLPYzJQ8ea6B18oUunEP9qkY3ZjZDuXDp5YUwxPuK9apRsI3yNJ9dQd14sc33FmmKffUZWVNjTzl5pkl0ZIs6d8KxHF7TV774n0D89OOoIy+Xcew9/sxxiOcc9okdO+n3U4jHc+3VWC8yX0Bdu++a0vGEfbljcbnPyWv7CmQSlHv2RS95JcSf/uhfQ/zV+9A3+eKXvCxx0/ZL0jAMwzASsIekYRiGYSRgD0nDMAzDSGCwT9LneWbUGkTJZ+eRxuiRjy/Gef8wRp9hJocT1ynS8FRx3jw1jvP8tXHM93f2OGoh6RWcl9cxnBd3W2chJglT4kXSUtbwO0aTNFlvjH2PuL+4hn69aAp9o35M5480Wxexdkb1IEnbkjKeb0c5QrU1em1IRGScrmORNLSArpNPeR1j8pxxzt8tW9BTxrlb2cPlB3gdsnSaiwX0u/L29uzZC/HMDHrK1ignMO/f0fEF1J5KBTXadBrP17ZtWOuvSrluWfPMUQ5lzl372OOPQXzmzBmIt07g+Qhb5GOl8yPe6Ptd5RyegzMtjGPyjHMu1oBExJA0SvZJtkmEYw2OuqykKHdqzsN6il+672sQP/zQExDX6Bq3uBZtCn2Ge7ejlzYO8Vnw6NcxH3FhHH2Z45N4D8+QLp6ie5qszX19njVeXl6p4tg2s2U3xFfddCvEX3/wPlkv9kvSMAzDMBKwh6RhGIZhJGAPScMwDMNIYKAmqeTzizl3q0caXRtzWGoK56Ujn/xTOZyXbjRp3j7E/ftzOO/dTg0uwtZqkdGQasT5E6Q9jePpaCzT8S2hSOnWyCiaodyv9B1EyRsUk2br5ThnKBXUpHyNHtU9VKopF1Edwoj8g0Iar6Q3x3emVhPPa4tzuQbYzmwKz+MaaXR+SOeNPVgE6x9p8v1lSDDKUP1IrreYTmM/zlL9Rc5LOQxuX4s0P66fOT2NtfzY91ipoJ7D9TQZ9nEuUy7aAzvncH9V9gDi8abSg8vaXg488s7mhepL1lDT86jNrWGXkFOV0mDlkabpCfkshbyywVkIsxnymJNnurSCY3M2Tx5ter+hvEoa5Cpe47Orn4H41jtfCvFkDn2atSWsMTo9g+8F7D+4H+KpGRzrU3TPe6Tbp6iWbiaD1++uV2O9y997+NdkvWyOUdEwDMMwNiH2kDQMwzCMBOwhaRiGYRgJDBQD/HHUXoI8znO7Oj5j2y30e/VpXGnUKNtV0hRDMgfRvH+TNEDXQO2nqadoe+RVSlG+xRRqL2EDtRnJY3tlDn2QqugPc1EJ4rhM/jvWIQKc94+apJmSX81RnT7l3LWCmqP6qDt4KTxfmqIabv7o6/qJ9PsEI8qVGlEtvpj0nJj0nxxpiL5Px000+TrQ+r7PeTkp92kZ9Z8nnngS4loNrxNrpKw5Mrw+e8a4/ceOHaX1B25eVihvKWuUE5OocTZI42SNMpfB+7hOGjNb9rYObt4lYW7rJMTnTuM1rAjey57DaxCyj49fkOB7n3K1cv3IvtSnZMxshdRHU3jOJ6g+5loVderZOdSNZ6ZxbGOdeu9VV0N80x2Y65Q1xZNPPgLxJ/72IxAvLOF7A7fcdjvEP/i278X970ffZkidJugzWlKu2WuwHuaBq6+T9WK/JA3DMAwjAXtIGoZhGEYC9pA0DMMwjAQGapIpTLcnUYO0ogrOC7uYcqGuoU9PGri7MIeaX+Dj9mJBjcyjmmd+iN4h//gX8fOk3UR51BUcaZC6hjqEl0UtqpXDeX4vR7lV2+QvK+JyjWj9Gmm8NfQWRSn0EsUr6E/zyVepVI9SaPtSwPZ5Y6gN+eHoc2iKDM+dyl/tWJP0yTc5LDcr5y7lmNfn2n5N0orLq9iPxsdR7ylQrlfWIFlz7Ns/wcfH2zt1CrX6YnGMYtTGl5awXiRrnjt2YV5MziVbrVIuWof3TTui+prkYx0FxSzeS47qEwpdkxTVh+Rcrm1Hnug+TdKnmGr3UpwOshRjH1LSTDM0NuToGt/+EqwHKU3s8+lx1CxfetdrIJ4co/qbJ09C3G7hNU2RD7W8egLi++67H+Ktc5if+TWvuwvibduxfUGKfaJ4D0wUUUe/9aW4vUHYL0nDMAzDSMAekoZhGIaRgD0kDcMwDCOBgZpkQPPm9RzOg0ua6xGSBhfQRL1DjSym2JGfyvdJUyui5qYp0iiXH8LtC/kcCziP7dZKGNM8trRp/y3S9Cj/4WyetJ1lPD9t8oHGY6gjpKbofGVQ19AM7Z+0rcgjL9c2bI9XI02T6mnOTNH1HRGcl5Fzn0aco5Y8UZyXkjU11hzZt7i4uIjtIZ9kcQw1vTZpkqwRTk5hHkqm1cJ+xr5E1iQ51ysfX7+GSe8SUG5Zbi9vL6L7gnPVpsiHmkqRvseWQWpen+Y8AiLKGz1Gx3SujjprRLV1+/LRelwzk64hLWddN03nMJ2hmqV0DbiGaBzj+wa7dmO9yFtuvhnixdOoKdZCbO9DX7gH4uoqvm9SyKNGuXUO3a7p4iTEKcrbXaugjv/Ag1+H2Evh+d2zG32TW7fi/rbO4fscU5O4/2sO3SjrxX5JGoZhGEYC9pA0DMMwjATsIWkYhmEYCQwUAxoV1FriAmpcSlqGKPkOd1Bu0gZ5hWo4z9/OkYbG6fgKqI3UyHuTdqhF+eRNCgXnvWPBeXt15H0qoLcmVcDtZ0kHuO6KnRDf/3Wc549onp8ziIYBaoS+h9pZkCXNUfH6RKRFBWlqbw012jzpIrvoeo2KlVXM65gJyK9KGl6d4lwe+0WO6kGGTTzPtQr2wyXKXXp8HnP0sia4czvqIwcPHIT4ga+hVn702DMQs6bZJr+tkkcvm8XrmuFaiKTJhiHXJcUwdOzTJA2W7stx0t7TpN/lx8hgzdsnjTLtj/67ehRjo2ZIY1usU/7ZFq6fSvFQiscUkK4dBFQrlzTIQo6uMeVxDuieSFNe6kIe4wMHroB4chLHgloZdfbqwjzEp84cg5jzULtxvOblEnq+V5bRe3vLLbdCvG/vXoinZ1Fj3Ll7G8QB1ZcsV3HMWHoc9z8+hpru9l14zw5i9L3TMAzDMDYp9pA0DMMwjATsIWkYhmEYCQw2KJGXJUXz6uEkahOuifPAYZp8h5Rv0LVRi5mcwnn3q2dRE9wygZ9/mspXnvJRm9EazlNXyLe4Z8c+iLOkQ+g4bm+R6leOp3H59hk8H1ny7rTJdzkX4Lz+ah11hNoqrh9lqB5kljRIH7WsjOD6ecHrsX0M/XtTm6SeZLWOvsUm5/qsoZZcI01yLYPf/caLeJ0aFdz+04dRI/zSI+jReuyZIxB79N1y2xzqJePko/zi/ZiXcmFxAeJhuVsd1c9Mp/B4Jidm6PO4fqmBnraAzg977sYnUF96+YteDPH+Kw5AzB60yRn0qDnS+/psk97g3LSXgyxpVjvGyPPcxmt05CyfU645yl5SvGZZ8v5maHkmT7V8KTlsQGbTdhPv3ZmteA337cFrIjGuz17VmRnsU2PUJ9jHGdA1XCmVIL7qKtRE3/TmN0A8PY1jEXt9czw203sBdappWq3h+zHVCj4LGg3ywA/AfkkahmEYRgL2kDQMwzCMBOwhaRiGYRgJDNQkvRnUVuI18hVSfUXZhRpcn/hA8+r5nbj7W7btgvjlu7ZDfIh8fPc8gRroF2s4794cQ6+RV8TtXXHFLdg88tM1m49DnCuj3yxNNefWaLkveH62ZdCbtG8Mz8cxh5ppuIreolobNcY0+QFjak9OcXsTdHxpD3WWyRz5XkeF4nEuk8cqptymbfLh1ZrkM3SoN50+hb7Hz3we81Le9+CDEJfqqG+kfTyPx46ih6xFvkfOFRvHnGtVKMY/9K+Py3fswH4dU3nGiQnU9utt6mdUz3FmBvWnW265HuJp0hyH5Z716V0Gn+tlcjHGEZDPYh8JSHMbz5cgTmXxGufo8ynKK835bDPkeWafpfrYpyN6v6BN3tdKC+N8AceGuTmqYdqmWrJUyzdL9TXTOjivc6uF29uxE32Ib/y210O8i8b2iHR57jOs27epz/H5m92C9SinSTdvtfAeGIT9kjQMwzCMBOwhaRiGYRgJ2EPSMAzDMBIYqEm2m5jr1LXI9xihd8VNo/bhUx26MEZtZ3wavTgzOdQ6ji/ivP/8qeMQL5EGuNPH3KntLOV2Hcd56hTNsyvlVg2a+PkZ8mFWfNbCIJQJH493/xjqFFnSYs4oajnZLOUs9fDzQtpOmhqQ8bkuIc37k84hbnPUk8xQXVHPp/qGpHlxfcI4wuVrlJv12MkTED/8KPoiOQ9kRAUR2zH2u5ByrXI9RtZTOGaNMQioViF50LZuxbqoN910E8TlVTzeySm8r5565mmInzn2FMRpyt1aoTqk1xy6BuJCEbX/dkg5hWNsP9evFDd6TdIjDZG9sI7S3wq9bxAo542WgXGKdOaYPNSZNnlLHd6rq03Ke01jc4P65Ne/8lWIt+3AXKlC17zZxO0FAXnQlV84Qe546asg3rePxmZqH9dsZW+wT/l9vezgmqpt8uBz/uBUsP73L+yXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcBgn2SdvCQh1akr4Dx+kMZ5dQ1w/XHKERl46HsMazzPjfPu51ZK2L7cJMQp9h75GIekBQVN3F6QptyyHmqsaT0FcYE0ydUmtn8L1XQLUqjtzFdw/TLpFK0C5coN8fymI9Q8MyG2J09+wz1bUVfYOoO+zVJl/fkMLyUZ6ieT5HGqVzH3Kvskoxg/X2/g+vPnFiFeXsPrEkaUI1fxOkS0fBjs+WJYnynQfZXPY7/k88Pt2bdvH8QzM6hhPnkYNchqBfvRzrk9EPP551qExSK9m+DweOr1Fi3Hfuj5g1NIXw7Yq8ojUSFNGlgDj2l6G9VibeK9VKugjzDMkM8yh30kRZpkrYHnNEf5e7Ok+7Yor/XReXyfo1hEL+ytL3kpxPMnMb/w8ROYv7hcxnvqdW94C8Q33oDeWhXOP4y6O8vSrDH25/8d7B3me441zphrrA7AfkkahmEYRgL2kDQMwzCMBOwhaRiGYRgJDBQD4hppCTyNO4eao5Jmlili7tcbd6CW0VxBLeip42ch3kG+woksah9C2/epDmFI3qOYfIdNqhFXaeHnW2THixT3X6Sabwtl0jyjEsQpxfXL5Fusxahb1Fs47z7m4eXK0OdTVL9zbjue7z3T2P5TFTzAU5Sbd1SsrKA/NyQ9p0YaWkQ+wjb1w2PHsF7k1x97FOJyA7fvSN/xyBNGlrU+PYTrQXIcUJ5Jzp3KuV+npsh/HGA/rpJGe+gQ+oHn5+chPnr08MD9zVBu1oMHr4Y4l8P7nvNmttttinH7aaqlGLvBnrvLQYP6gE9D41gB793JPLWZ+oQf4/phC/u0pAMK8ZyUG3QOI+xDhQzq1JzblXXqiOovNslXefsdd0K8eBrzEX/tPfdCnM1hDdWrr70W4kwGjycizzyrvhHdA0L39DDvMcP3JGvOouv/fWi/JA3DMAwjAXtIGoZhGEYC9pA0DMMwjAQGapJhE5+hPuUC9VdJAyyjOaeYQ3/WGM1TK+XkPHLuNMRnTqCmdv1uyqWawXn7agm1qrqPGlwUoFaVT6G2c6aMdQuF5tFnx1ADXaB8jQVFTRVbI7JA2s8qxT5pNznFyzOTx/0XA5zHzyjqKrNohZI6aazP0PmqNdbvHbqUVEmDdNSuKEKBoU7LwwjP49ISXtf5BbxOrG4o6SER+U9jillz7K8HyXkw8brmqC5ri2oDVqt4nW65Beuu1ut4vj73uc9CvFJahnitgu8CjI2jx2///oMQ79l9AOJCEdvr033E54NztabTqNe1WY8aAaUl8srS+wqaphqeEzg2nVjAPjY5gXmpfcpv65MvkizWUmnQ/lnzpBqlpRXMN9ym3z+tFm6gtIZjdaWyAvEX7vs0Lq9in3zt618N8d49uyEelp+YNdOQdOyAdGvWJJl+DZJFyOeue9svScMwDMNIwB6ShmEYhpGAPSQNwzAMI4GBmmSG/E9CuVmjGmod0sR54EKMIlh5FbWiVkQ1vUKcV186cQ7iZzzUYppZ1AXCCs5zl2Oct8+kUNtZJd1hcaEE8f59UxAXyFuzGpHXx0Ndo0qaY4PyD9YoN26OvFMFn/16eHzFDLZntoDaUpPm/Z8ibWqhThok6RyjYoJ8gRHlyQzpvHo11Mar1A84T2Q2TbUDSb5gOaNFuWFdn0WLPFy0AtcmVNbsyPfI9Rcj0nfyBewnjz5Kvs8SapTZHB5/RPU2s2nUGMfG8L6K6YTU6fwHNIwElDczRRok593k8zUKzi3h2FMlnXdsAu+tFOUijVr0PkQT34fwM7h+o45jKZ8zj3yRTerjjSznhcbP1ys09mXwGt9++20Qr549CvFXvvAViGdm9kN8w02YmzVP+YbZ+8u1ic8cewLi5WXUSA8cugXiAr0P0q8wcv1NXKoyWCMdhP2SNAzDMIwE7CFpGIZhGAnYQ9IwDMMwEhioSU6P48RutohaxeEjqMEFhb0Qxw61iNOnsY7f+Fac509PsdcFdYLjZ89AvBhg/sLAoxpiNI9fzKBXqVVBbaV8GGuu7ZolbaWIWlk7opprDWwv58TMyuDcsU3SUAuUj3F6CnUOzgdZp7p8j5fx+i0soReqWUbdxWtuDp9kgeoVxlRv0ZF/tdDE81yr0HVu4XEe3I++v/kF7Jdtdk5yOUjyrMV9Gpsbspy2T/UqOTfr3Db0G2/fiR68x57A81OhXK6sCeay5O+lepVF8vTxKNEgT1uO+h1JrtKi9ZXOh78J6kkK1cDsPwbSxUnXdQ6XnzuHY1W+QHmn6Rx4NHYFVP/Qhbi/NOUTzmXxGleq2OcPXY/5d7fm8X2QL33skxCvnuMaoDh2HD+O+X/37N1B6+PnTxy5D+LHHrwH4qPHsVbvuXMnIL760Ishnp7dDnGmQLWASTPWvnuSTOOSlSTsl6RhGIZhJGAPScMwDMNIwB6ShmEYhpHAQDEgm6eaamOohcwdwHx9+QKKNbkCahG7Z7DO3bYZ1J7O+qiFfP5xzLGZc5gfsXQK55krLZxnp2l+aQeoBbW5XmYL58XbFfTmyNatuLyM2o/L4PaKedQh4hDbm0nhvHlLyIcakh+NdJBmiNfnCcp5erpKdf4qNE/P0/LpomwKOA8jaXQ+XViP1ncO9Ymt27D23eQkasv9u+dcrOSTZIvVEJvfsDySUYz3Dee9vP76GyC++mqs3ffF+74K8ews9ru9+zjXK3r6DhzYh9u/BvUrn3LNpslnmiEfKt936uHxcn1JzmU7Cgo5PKY0+QpXyTe5tIq+PvaONsknGFKtVtat+Rz7lLd5bhITMd9x9T6IK2V8P2SiiMczN4lj2SNfeQDipUXU5ScncH1PcCz/+498BOJaqQTxtdfj+ymHH78f4lPzWON1tYIe7oe/hNs/+hj28auvvx3ifVffBPHcbsw/7FMfXVlBD35ulsb6HuyXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcBAMeCOG1GDbMeo5Wz3cd59C/kqxzzU8HZtxVyoXhbnucdx2lhObMM/7NyC8+wlqvFWKaFGWFnD5SdPoeYYU67YfB41wXQKtycRxu0WfseoUy5WL0WaJ+kOmQxqZ3nSoiZyqLG2SRcoU13DZWy+xKQFBUXKJ0n7S/t4fkdFQPUHY8pBS4fVlwtUArwubdJ/FpdR/2iRfuRxzmLWHDlx5JA0kMNq3TnyTXKtvSz1kzhC0S+fm4T4pW94GcRTM/j5+fmTEL/61Vgb8LpDhyAW8uT53uBcrTHlbHZkLOV6k3y8oyAiITVN9QxbZfQJrtapli5tj3VszpdLaZwljimvNXXyfXP7IC56dI6py+6cwbF3eRE91UrtmdoyCfE0XfPHj8xD/NAJ9JRXl+l9kUXM9Ro59I1WSOP1ArwnSks4VjereD6zk3jA5VV8f+VF9H7Fzn3ojT56FHPHbpvdI0nYL0nDMAzDSMAekoZhGIaRgD0kDcMwDCOBgZrklftRg/R89JIcPofz8lPkPytQzsgwwnn3mSLOe+8k+9rO7eijvPo69FHmd6Nmun0Gn/mPPlKC+P2/fxTia25CDW4txO3Nz6P3aG4GjzdNPlK/irpGNo1xGKGmGIWkYXLdQg91kTNcU65GvsqARF3K7eo59H7tmCGNcpNokinKQ8m5WoVizoXa5jqazxyF+Mgx1FNiTqVKsYsH+xzZBsk+R45V8bqSpCf5HF6X6SnUl1pN3H+hgB66G2+8EeJF0nfuuAM1y5e8BPNiBin2iVKtPk5tS3peSH5g1ozTNC6026zoXX7UY58inoMMvU8QUSchGbavJqjnccyaJZ6jK7divt7JAMeOagbHggNXo+ZW3LoT4oXTqKk+9cQxiFlmbzV4rML2Lqzg+xwH9uA916ychrjSRh+meuQNbpcgblTwhBZ34dhUbePY1pjHsfArn/4wxKk3fBfETzz5CMQvvf21koT9kjQMwzCMBOwhaRiGYRgJ2EPSMAzDMBIYqEmmfXyGzlB+u3ID55XTJK6s1nEe3FFdOS+Fn9+zDbWa7Tfh/nLjqFG+8lbKBbsTNcu1VdzeFQdRm/nut2GOyqdP4Oc/8idfg7jVRB1g5xYIpVHA9m4hidCLMGfmEZxWlxrVRVyLsf2NJolXDpdHVA+yPY/epBzVXNu+E71M51pDDH+XCfZJOp+0WvY1Uq29p44cgfhvPvZRiJcpz2Qmi1ptyL7Mvlyy7PNjI+VgX6Tvcy5avA0nJ7E2364dV0BcXsWOE1B7FhdRD9q7D3Ms3/nyOyBmT2CfKMu1D306HtIk221cn2svcu5WzmM6CpptvOYt0klrZGystnB9zvfrSMfleoYeJQC+Zi/m171l2yR+PoP3dmEr1afM4f5Xy+gbzOZxf4cOYT1GP8Cxb/4MeomXSIO8ah/mZl0t49h28hTGk1tRM9QI38dYI03RV9TZUzk8/rCF26/VSHOl2rlCebIf/grmkpUfkkTsl6RhGIZhJGAPScMwDMNIwB6ShmEYhpHAQE1y/hzOQy+UKP9eC+eJJ8dwXjlFBQsjRe2iUsFndHo7inwvexGKekXyj83OoHbyzBGcR3/qGWzvwevRiJkZm4S4rKiVjO1GDXTxDOYn3EHfMTITOO+dpvYWs3i6dyjqGnVafrJJl6dNWhFpWVvHsD2hotdqR4HOJ9W7PEb5KUeFR/UjXUi5TkmaTZGmViE/6fwi1o4jS1pf7cAs6UuNBuonLdKOHSeTJRzpURxHER5fPov9tEKescVzmEdzmWrjPXPsSYhf/dpX4fbzqD+FId7HrDE6Tl5Lh8uaK3sCHfkm+XxuAklSmpSLtdLE3KKn1lAHjujnhT+kD1CXlQkaG158Dfoaxwt4jpaWyDeYIo8z+S5PnsQ+kk1hH5+l2r7ZIt5D124lzdLDPvLgV45CvLiE52/pHJ6P2Z3Y/laVdOkWnqFMnsbWNG6vWcPrUy/h9vN59FU+9tXPQlw6jprtIOyXpGEYhmEkYA9JwzAMw0jAHpKGYRiGkcBgTbKEGpwnOA8cZijZqlYgLGao3mKD6imOo3ZRx2lviT2cVz5D9SLDp3Ee+9hZ1A1K5H0qTKMmd2oJ5+2PLuC8/NxezOUaPoHaz/GjqNnWD2GOzbNlnCcPUOKUOw7gH87RvHrlDOoSizWcx08JaksHtmAcFdFvt5X9h5SqNUN2uVGRSlHOX6qjyS6+Uhmvw+Ejz0BcJ/0iX8BaczffdBPE2TSemNOn0XdYWi1BfO4c5qVcI/3KOeyHEfk6PUH/6jaqHbhK+zt2HI/v3OICxMfzuL0nnnwc4tlZ7Nd50qa5PqbHf+Dl3mB/rcd5UOn8Rpw8dwREgvdajeodNkij9EhI9dkbS/E0nePbDqAGqT6Onc0A+/Tsdlx/egKvYbmCY3W9iX0ip6gxci5Wj3LTNpr4fsnUFF6zKw7g+yN79+PyVoxja9jAeyifw/bObEXNtFFHn2RlGc9PlTVivHySovzD2QyOjbum6Nk1APslaRiGYRgJ2EPSMAzDMBKwh6RhGIZhJDBQk8z6OO+eS+E89lJIdeFaOG8fKc5rr1ZxXjkMcZ54ZQvWbyw3cPulEmpTKdIR1lo4jz27BZfni6gLpFKoTW2bpNyyVP/Sz+LnH7kffZkHt+PxT6Up96w/CfHOMdSmxlJ4vo9WMI7rnBMUNVltk++UpJ5MAf/QIM32wBRej1HBmpVQnssmCRCnz6LnaWERNcJdOzEv5swM6ilv/cG3DmzPyZMnIV4mjfDwEfQlPvTgAxBXSqSfUO3BiQK2b8d2yqm7RLX5qtjvquQLPXbsBMQf/OCfQBzH2M9f85rXQJwi/y07AFmC9IZ81WbJMZXC65vigpwjIMjgUeazKNBnGrg8GNZkOqY9kzjWzBZw++0m3ovZcXy/Ycf+fRBPTmAfPj2PfSRu4klP53CsiGIam5dRIyxT7tOwgevvPDAFcZDFa7q0RGMzeXnTPvbBqa14Pkpk2V6Zxz6fzaHmmiri/tsRjtWujdej1iQRcwD2S9IwDMMwErCHpGEYhmEkYA9JwzAMw0hgoCYZkGaXIU3v0adwnnh7Hud5p9O4+TNr6P2Zonx8x8+SZkjSVIX8bk/Ok/8ti/6wq/bi/k8soBi0jNPkQqlNZYV8izPTqBNc9RLSNDM4b/+aF6O29NRR1I6ckD9NUJfYVSQvU4w6wEoJa6p5LTzeseIkxAsl8n45bI+EmyCJpoj4Ph5Hmq4L5wqdmJyE+Lbbb4f4lttug7hAuUsP3XA9xDHVDtxzAK9jrYbn/eZbboD4hutxe0eePIbxU6hx7tqOn1eq57i0hJrrK1/5coiPHsPtryyhts/1MJdJf3Lu4voU+fpEVJ8zpjjgepYjoNoin6KjPMgkrAY0dE7mcLCaGkff3xYaXMqreO/laCzwYno/gBIWNyjfbqWM7VtbxPYs0vEVSJNsh3i8zQaO1QHlio0rqLOHa3g8KfJdpjLbIG61sX1ZMmk7h++DRE3s09N7sX1rVTy/fkDvX1Tw86uUO3YQ9kvSMAzDMBKwh6RhGIZhJGAPScMwDMNIYKAm6dG07VoFNcdSmbwrDufhl5fIFxnjvDpZfeQw1a/M0PYa5CWqko/y4HbUQrYV0TepgvPmpXIJ4rUqanZjWapxliKNs4nz8rfM7sP1M6hpzm3B/Z9dxHnzUgX3x1qN0nea7aSt5ci8xdraKtWnrDZRw12aR61qVAR0nn0fz0s2i/3i0KFDEO/chb7DmJx+AeWwLRbwPEQx9rNJh3keHRkd94V7Ib7mKtQkF06VIH7koSMQnz6F99Fn770b4mYLfZ//5Du+FeLv+M7vhLi8ivcd57Hcth3rjOZyrI0Phn2SfH08Mk5yzPU0NwPlOo49K1Vsc4U8yvu34vsBt12NuVG3TuO9efwI+vrEx+2PFbEPLi/iWDSzBa/pCnl1Vxaw/eUSji3sKU9lcXBvBXhNPLzFJJXD9rYpt2tAY08Q4Pq5cTwf6Yhqlsb4+bCJRslUhsYE0kgbVbynwzaOtbUKLq+trl+Ht1+ShmEYhpGAPSQNwzAMIwF7SBqGYRhGAgM1yTPkt1qooTdnew59glEN55UXF3EevDBJXpuY5v3X0H9Wr+A8/ERhEve/BeMTZ9D4eMUOnAefpfqV86QJeqRdtRqoTZ2kmm1V0kjbbTzeBuVPDAJcv0U13aoNnKdfa1H9SPLPbZtCzbXZQI30aTKC1iifYYG0vXJr9HX9REQcaanCGhb58FhTy1LtPpI7+jQx1+eBYy2YIJ8fJzct5lCvynoovtdW8bw/ffjjEDdbqMds2zEL8dQUbv/A/gPUQGyx52ED1WPtemPXfdj5a5JeVafajH2aJK4+ErhNrLvum5qE+OAMji2e4tiVSuHyQh7HykDw3q6Rb6+Mp0xW6f2Pc/T+RrWCGly7hQewVsE+26jhWOUwzbRs2YV9LOKbgHyIEpAmKKxhoo4f0VinARUTFlzfz7JHHO/x8hLVJsZHh1TofRPOnzwI+yVpGIZhGAnYQ9IwDMMwErCHpGEYhmEkoJvRs2QYhmEYmwH7JWkYhmEYCdhD0jAMwzASsIekYRiGYSRgD0nDMAzDSMAekoZhGIaRgD0kDcMwDCOB/z9SUuMwa/yZigAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 576x576 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "source": [
    "# let's calculate the overall test accuracy\n",
    "correct = 0\n",
    "total = 0\n",
    "# since we're not training, we don't need to calculate the gradients for our outputs\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        # calculate outputs by running images through the network\n",
    "        outputs = net(images)\n",
    "        # the class with the highest energy is what we choose as prediction\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Accuracy of the network on the 10000 test images: 50 %\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [],
   "outputs": [],
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.8.8 64-bit ('rl_pytorch': conda)"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "interpreter": {
   "hash": "79cde69fcce212f2dcfc86fc7d14ff7fa4294891a60d8a7090286894fe9f5e75"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}