Basics_of_NN_exercise_solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST Code Along with ANN\n",
    "Before we start working with Convolutional Neural Networks (CNN), let's model the <a href='https://en.wikipedia.org/wiki/MNIST_database'>MNIST dataset</a> using only linear layers.<br>\n",
    "In this exercise we'll use the same logic laid out in the ANN notebook. We'll reshape the MNIST data from a 28x28 image to a flattened 1x784 vector to mimic a single row of 784 features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Perform standard imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F          # adds some efficiency\n",
    "from torch.utils.data import DataLoader  # lets us load data in batches\n",
    "from torchvision import datasets, transforms\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.metrics import confusion_matrix  # for evaluating results\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the MNIST dataset\n",
    "PyTorch makes the MNIST dataset available through <a href='https://pytorch.org/docs/stable/torchvision/index.html'><tt><strong>torchvision</strong></tt></a>. The first time it's called, the dataset will be downloaded onto your computer to the path specified. From that point, torchvision will always look for a local copy before attempting another download.\n",
    "### Define transform\n",
    "As part of the loading process, we can apply multiple transformations (reshape, convert to tensor, normalize, etc.) to the incoming data.<br>For this exercise we only need to convert images to tensors. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "transform = transforms.ToTensor()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the training set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset MNIST\n",
       "    Number of datapoints: 60000\n",
       "    Root location: ./dataset\n",
       "    Split: Train\n",
       "    StandardTransform\n",
       "Transform: ToTensor()"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data = datasets.MNIST(root='./dataset', train=True, download=True, transform=transform)\n",
    "train_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the test set\n",
    "There's a companion set of MNIST data containing 10,000 records accessible by setting train=False. As before, torchvision will only download this once, and in the future will look for the local copy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset MNIST\n",
       "    Number of datapoints: 10000\n",
       "    Root location: ./dataset\n",
       "    Split: Test\n",
       "    StandardTransform\n",
       "Transform: ToTensor()"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data = datasets.MNIST(root='./dataset', train=False, download=True, transform=transform)\n",
    "test_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examine a training record"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0118, 0.0706, 0.0706, 0.0706,\n",
       "           0.4941, 0.5333, 0.6863, 0.1020, 0.6510, 1.0000, 0.9686, 0.4980,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.1176, 0.1412, 0.3686, 0.6039, 0.6667, 0.9922, 0.9922, 0.9922,\n",
       "           0.9922, 0.9922, 0.8824, 0.6745, 0.9922, 0.9490, 0.7647, 0.2510,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1922,\n",
       "           0.9333, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922,\n",
       "           0.9922, 0.9843, 0.3647, 0.3216, 0.3216, 0.2196, 0.1529, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0706,\n",
       "           0.8588, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.7765, 0.7137,\n",
       "           0.9686, 0.9451, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.3137, 0.6118, 0.4196, 0.9922, 0.9922, 0.8039, 0.0431, 0.0000,\n",
       "           0.1686, 0.6039, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0549, 0.0039, 0.6039, 0.9922, 0.3529, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.5451, 0.9922, 0.7451, 0.0078, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0431, 0.7451, 0.9922, 0.2745, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.1373, 0.9451, 0.8824, 0.6275,\n",
       "           0.4235, 0.0039, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3176, 0.9412, 0.9922,\n",
       "           0.9922, 0.4667, 0.0980, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1765, 0.7294,\n",
       "           0.9922, 0.9922, 0.5882, 0.1059, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0627,\n",
       "           0.3647, 0.9882, 0.9922, 0.7333, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.9765, 0.9922, 0.9765, 0.2510, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1804, 0.5098,\n",
       "           0.7176, 0.9922, 0.9922, 0.8118, 0.0078, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.1529, 0.5804, 0.8980, 0.9922,\n",
       "           0.9922, 0.9922, 0.9804, 0.7137, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0941, 0.4471, 0.8667, 0.9922, 0.9922, 0.9922,\n",
       "           0.9922, 0.7882, 0.3059, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0902, 0.2588, 0.8353, 0.9922, 0.9922, 0.9922, 0.9922, 0.7765,\n",
       "           0.3176, 0.0078, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0706, 0.6706,\n",
       "           0.8588, 0.9922, 0.9922, 0.9922, 0.9922, 0.7647, 0.3137, 0.0353,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.2157, 0.6745, 0.8863, 0.9922,\n",
       "           0.9922, 0.9922, 0.9922, 0.9569, 0.5216, 0.0431, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.5333, 0.9922, 0.9922, 0.9922,\n",
       "           0.8314, 0.5294, 0.5176, 0.0627, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000]]]), 5)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling the first record from <tt>train_data</tt> returns a two-item tuple. The first item is our 28x28 tensor representing the image. The second is a label, in this case the number \"5\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape: torch.Size([1, 28, 28]) \n",
      "Label: 5\n"
     ]
    }
   ],
   "source": [
    "image, label = train_data[0]\n",
    "print('Shape:', image.shape, '\\nLabel:', label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the image\n",
    "Matplotlib can interpret pixel values through a variety of <a href='https://matplotlib.org/tutorials/colors/colormaps.html'>colormaps</a>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADgpJREFUeJzt3X+MVfWZx/HPs1j+kKI4aQRCYSnEYJW4082IjSWrxkzVDQZHrekkJjQapn8wiU02ZA3/VNNgyCrslmiamaZYSFpKE3VB0iw0otLGZuKIWC0srTFsO3IDNTjywx9kmGf/mEMzxbnfe+fec++5zPN+JeT+eM6558kNnznn3O+592vuLgDx/EPRDQAoBuEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxDUZc3cmJlxOSHQYO5u1SxX157fzO40syNm9q6ZPVrPawFoLqv12n4zmybpj5I6JQ1Jel1St7sfSqzDnh9osGbs+ZdJetfd33P3c5J+IWllHa8HoInqCf88SX8Z93goe+7vmFmPmQ2a2WAd2wKQs3o+8Jvo0OJzh/Xu3i+pX+KwH2gl9ez5hyTNH/f4y5KO1dcOgGapJ/yvS7rGzL5iZtMlfVvSrnzaAtBoNR/2u/uImfVK2iNpmqQt7v6H3DoD0FA1D/XVtDHO+YGGa8pFPgAuXYQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EVfMU3ZJkZkclnZZ0XtKIu3fk0RTyM23atGT9yiuvbOj2e3t7y9Yuv/zy5LpLlixJ1tesWZOsP/XUU2Vr3d3dyXU//fTTZH3Dhg3J+uOPP56st4K6wp+5zd0/yOF1ADQRh/1AUPWG3yXtNbM3zKwnj4YANEe9h/3fcPdjZna1pF+b2f+6+/7xC2R/FPjDALSYuvb87n4suz0h6QVJyyZYpt/dO/gwEGgtNYffzGaY2cwL9yV9U9I7eTUGoLHqOeyfLekFM7vwOj939//JpSsADVdz+N39PUn/lGMvU9aCBQuS9enTpyfrN998c7K+fPnysrVZs2Yl173vvvuS9SINDQ0l65s3b07Wu7q6ytZOnz6dXPett95K1l999dVk/VLAUB8QFOEHgiL8QFCEHwiK8ANBEX4gKHP35m3MrHkba6L29vZkfd++fcl6o79W26pGR0eT9YceeihZP3PmTM3bLpVKyfqHH36YrB85cqTmbTeau1s1y7HnB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgGOfPQVtbW7I+MDCQrC9atCjPdnJVqffh4eFk/bbbbitbO3fuXHLdqNc/1ItxfgBJhB8IivADQRF+ICjCDwRF+IGgCD8QVB6z9IZ38uTJZH3t2rXJ+ooVK5L1N998M1mv9BPWKQcPHkzWOzs7k/WzZ88m69dff33Z2iOPPJJcF43Fnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqr4fX4z2yJphaQT7r40e65N0g5JCyUdlfSAu6d/6FxT9/v89briiiuS9UrTSff19ZWtPfzww8l1H3zwwWR9+/btyTpaT57f5/+ppDsveu5RSS+5+zWSXsoeA7iEVAy/u++XdPElbCslbc3ub5V0T859AWiwWs/5Z7t7SZKy26vzawlAMzT82n4z65HU0+jtAJicWvf8x81sriRltyfKLeju/e7e4e4dNW4LQAPUGv5dklZl91dJ2plPOwCapWL4zWy7pN9JWmJmQ2b2sKQNkjrN7E+SOrPHAC4hFc/53b27TOn2nHsJ69SpU3Wt/9FHH9W87urVq5P1HTt2JOujo6M1bxvF4go/ICjCDwRF+IGgCD8QFOEHgiL8QFBM0T0FzJgxo2ztxRdfTK57yy23JOt33XVXsr53795kHc3HFN0Akgg/EBThB4Ii/EBQhB8IivADQRF+ICjG+ae4xYsXJ+sHDhxI1oeHh5P1l19+OVkfHBwsW3vmmWeS6zbz/+ZUwjg/gCTCDwRF+IGgCD8QFOEHgiL8QFCEHwiKcf7gurq6kvVnn302WZ85c2bN2163bl2yvm3btmS9VCrVvO2pjHF+AEmEHwiK8ANBEX4gKMIPBEX4gaAIPxBUxXF+M9siaYWkE+6+NHvuMUmrJf01W2ydu/+q4sYY57/kLF26NFnftGlTsn777bXP5N7X15esr1+/Pll///33a972pSzPcf6fSrpzguf/093bs38Vgw+gtVQMv7vvl3SyCb0AaKJ6zvl7zez3ZrbFzK7KrSMATVFr+H8kabGkdkklSRvLLWhmPWY2aGblf8wNQNPVFH53P+7u5919VNKPJS1LLNvv7h3u3lFrkwDyV1P4zWzuuIddkt7Jpx0AzXJZpQXMbLukWyV9ycyGJH1f0q1m1i7JJR2V9N0G9gigAfg+P+oya9asZP3uu+8uW6v0WwFm6eHqffv2JeudnZ3J+lTF9/kBJBF+ICjCDwRF+IGgCD8QFOEHgmKoD4X57LPPkvXLLktfhjIyMpKs33HHHWVrr7zySnLdSxlDfQCSCD8QFOEHgiL8QFCEHwiK8ANBEX4gqIrf50dsN9xwQ7J+//33J+s33nhj2VqlcfxKDh06lKzv37+/rtef6tjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQjPNPcUuWLEnWe3t7k/V77703WZ8zZ86ke6rW+fPnk/VSqZSsj46O5tnOlMOeHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCqjjOb2bzJW2TNEfSqKR+d/+hmbVJ2iFpoaSjkh5w9w8b12pclcbSu7u7y9YqjeMvXLiwlpZyMTg4mKyvX78+Wd+1a1ee7YRTzZ5/RNK/uftXJX1d0hozu07So5JecvdrJL2UPQZwiagYfncvufuB7P5pSYclzZO0UtLWbLGtku5pVJMA8jepc34zWyjpa5IGJM1295I09gdC0tV5Nwegcaq+tt/MvijpOUnfc/dTZlVNByYz65HUU1t7ABqlqj2/mX1BY8H/mbs/nz193MzmZvW5kk5MtK6797t7h7t35NEwgHxUDL+N7eJ/Iumwu28aV9olaVV2f5Wknfm3B6BRKk7RbWbLJf1G0tsaG+qTpHUaO+//paQFkv4s6VvufrLCa4Wconv27NnJ+nXXXZesP/3008n6tddeO+me8jIwMJCsP/nkk2VrO3em9xd8Jbc21U7RXfGc391/K6nci90+maYAtA6u8AOCIvxAUIQfCIrwA0ERfiAowg8ExU93V6mtra1sra+vL7lue3t7sr5o0aKaesrDa6+9lqxv3LgxWd+zZ0+y/sknn0y6JzQHe34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCCrMOP9NN92UrK9duzZZX7ZsWdnavHnzauopLx9//HHZ2ubNm5PrPvHEE8n62bNna+oJrY89PxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EFWacv6urq656PQ4dOpSs7969O1kfGRlJ1lPfuR8eHk6ui7jY8wNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUObu6QXM5kvaJmmOpFFJ/e7+QzN7TNJqSX/NFl3n7r+q8FrpjQGom7tbNctVE/65kua6+wEzmynpDUn3SHpA0hl3f6rapgg/0HjVhr/iFX7uXpJUyu6fNrPDkor96RoAdZvUOb+ZLZT0NUkD2VO9ZvZ7M9tiZleVWafHzAbNbLCuTgHkquJh/98WNPuipFclrXf3581stqQPJLmkH2js1OChCq/BYT/QYLmd80uSmX1B0m5Je9x90wT1hZJ2u/vSCq9D+IEGqzb8FQ/7zcwk/UTS4fHBzz4IvKBL0juTbRJAcar5tH+5pN9IeltjQ32StE5St6R2jR32H5X03ezDwdRrsecHGizXw/68EH6g8XI77AcwNRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCavYU3R9I+r9xj7+UPdeKWrW3Vu1Lorda5dnbP1a7YFO/z/+5jZsNuntHYQ0ktGpvrdqXRG+1Kqo3DvuBoAg/EFTR4e8vePsprdpbq/Yl0VutCumt0HN+AMUpes8PoCCFhN/M7jSzI2b2rpk9WkQP5ZjZUTN728wOFj3FWDYN2gkze2fcc21m9msz+1N2O+E0aQX19piZvZ+9dwfN7F8L6m2+mb1sZofN7A9m9kj2fKHvXaKvQt63ph/2m9k0SX+U1ClpSNLrkrrd/VBTGynDzI5K6nD3wseEzexfJJ2RtO3CbEhm9h+STrr7huwP51Xu/u8t0ttjmuTMzQ3qrdzM0t9Rge9dnjNe56GIPf8ySe+6+3vufk7SLyStLKCPlufu+yWdvOjplZK2Zve3auw/T9OV6a0luHvJ3Q9k909LujCzdKHvXaKvQhQR/nmS/jLu8ZBaa8pvl7TXzN4ws56im5nA7AszI2W3Vxfcz8UqztzcTBfNLN0y710tM17nrYjwTzSbSCsNOXzD3f9Z0l2S1mSHt6jOjyQt1tg0biVJG4tsJptZ+jlJ33P3U0X2Mt4EfRXyvhUR/iFJ88c9/rKkYwX0MSF3P5bdnpD0gsZOU1rJ8QuTpGa3Jwru52/c/bi7n3f3UUk/VoHvXTaz9HOSfubuz2dPF/7eTdRXUe9bEeF/XdI1ZvYVM5su6duSdhXQx+eY2YzsgxiZ2QxJ31TrzT68S9Kq7P4qSTsL7OXvtMrMzeVmllbB712rzXhdyEU+2VDGf0maJmmLu69vehMTMLNFGtvbS2PfePx5kb2Z2XZJt2rsW1/HJX1f0n9L+qWkBZL+LOlb7t70D97K9HarJjlzc4N6Kzez9IAKfO/ynPE6l364wg+IiSv8gKAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8E9f/Ex0YKZYOZcwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(train_data[0][0].reshape((28,28)), cmap=\"gray\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADmtJREFUeJzt3W+sVPWdx/HPFwT/UFQIV3ulKF00ZgmJYEbYhI2iRLSbKvCgBmIQTQM+ANkmEBfhATxwE6PbdlVMk4slQFJpGyorJGYtGo1L3BgGJQiLbNVc6V0QLqFYqw9Q+O6De2hu8c5vhpkzc+byfb8ScmfO9/zmfDPczz0z85uZn7m7AMQzpOgGABSD8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCOqSVh5szJgxPn78+FYeEgilu7tbJ06csFr2bSj8ZnavpGclDZX0ors/ldp//PjxKpfLjRwSQEKpVKp537of9pvZUEkvSPqBpImS5pvZxHpvD0BrNfKcf6qkj9z9E3c/LenXkmbn0xaAZmsk/GMl/bHf9Z5s298ws8VmVjazcm9vbwOHA5CnRsI/0IsK3/p8sLt3uXvJ3UsdHR0NHA5AnhoJf4+kcf2uf0/SkcbaAdAqjYR/t6SbzOz7ZjZc0jxJ2/NpC0Cz1T3V5+7fmNlSSa+pb6pvg7sfyK0zAE3V0Dy/u78q6dWcegHQQry9FwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAaWqXXzLolfSHpjKRv3L2UR1PIz5kzZ5L1zz//vKnHX7duXcXaV199lRx76NChZP2FF15I1lesWFGxtmXLluTYyy67LFlfuXJlsr5mzZpkvR00FP7Mne5+IofbAdBCPOwHgmo0/C7p92a2x8wW59EQgNZo9GH/dHc/YmbXSNppZh+6+9v9d8j+KCyWpOuvv77BwwHIS0Nnfnc/kv08LmmbpKkD7NPl7iV3L3V0dDRyOAA5qjv8ZjbCzEaeuyxplqT9eTUGoLkaedh/raRtZnbudl5y9//MpSsATVd3+N39E0m35NjLRevw4cPJ+unTp5P1d955J1nftWtXxdqpU6eSY7du3ZqsF2ncuHHJ+mOPPZasb9u2rWJt5MiRybG33JL+1b7jjjuS9cGAqT4gKMIPBEX4gaAIPxAU4QeCIvxAUHl8qi+8999/P1m/6667kvVmf6y2XQ0dOjRZf/LJJ5P1ESNGJOsPPvhgxdp1112XHDtq1Khk/eabb07WBwPO/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFPP8ObjhhhuS9TFjxiTr7TzPP23atGS92nz4m2++WbE2fPjw5NgFCxYk62gMZ34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIp5/hyMHj06WX/mmWeS9R07diTrU6ZMSdaXLVuWrKdMnjw5WX/99deT9Wqfqd+/v/I6Ls8991xyLJqLMz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBFV1nt/MNkj6oaTj7j4p2zZa0m8kjZfULekBd/9T89oc3ObMmZOsV/te/2rLSe/bt69i7cUXX0yOXbFiRbJebR6/mkmTJlWsdXV1NXTbaEwtZ/6Nku49b9tKSW+4+02S3siuAxhEqobf3d+WdPK8zbMlbcoub5KUPrUBaDv1Pue/1t2PSlL285r8WgLQCk1/wc/MFptZ2czKvb29zT4cgBrVG/5jZtYpSdnP45V2dPcudy+5e6mjo6POwwHIW73h3y5pYXZ5oaRX8mkHQKtUDb+ZbZH035JuNrMeM/uxpKck3W1mf5B0d3YdwCBSdZ7f3edXKM3MuZewrrzyyobGX3XVVXWPrfY+gHnz5iXrQ4bwPrHBiv85ICjCDwRF+IGgCD8QFOEHgiL8QFB8dfdFYO3atRVre/bsSY596623kvVqX909a9asZB3tizM/EBThB4Ii/EBQhB8IivADQRF+ICjCDwTFPP9FIPX12uvXr0+OvfXWW5P1RYsWJet33nlnsl4qlSrWlixZkhxrZsk6GsOZHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYp7/IjdhwoRkfePGjcn6I488kqxv3ry57vqXX36ZHPvQQw8l652dnck60jjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQVef5zWyDpB9KOu7uk7JtayUtktSb7bbK3V9tVpNonrlz5ybrN954Y7K+fPnyZD31vf9PPPFEcuynn36arK9evTpZHzt2bLIeXS1n/o2S7h1g+8/dfXL2j+ADg0zV8Lv725JOtqAXAC3UyHP+pWa2z8w2mNmo3DoC0BL1hv8XkiZImizpqKSfVtrRzBabWdnMyr29vZV2A9BidYXf3Y+5+xl3PytpvaSpiX273L3k7qWOjo56+wSQs7rCb2b9P041V9L+fNoB0Cq1TPVtkTRD0hgz65G0RtIMM5ssySV1S3q0iT0CaAJz95YdrFQqeblcbtnx0HynTp1K1nfs2FGx9vDDDyfHVvvdnDlzZrK+c+fOZP1iVCqVVC6Xa1rwgHf4AUERfiAowg8ERfiBoAg/EBThB4Jiqg+FufTSS5P1r7/+OlkfNmxYsv7aa69VrM2YMSM5drBiqg9AVYQfCIrwA0ERfiAowg8ERfiBoAg/EBRLdCNp3759yfrWrVuT9d27d1esVZvHr2bixInJ+u23397Q7V/sOPMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFDM81/kDh06lKw///zzyfrLL7+crH/22WcX3FOtLrkk/evZ2dmZrA8ZwrkthXsHCIrwA0ERfiAowg8ERfiBoAg/EBThB4KqOs9vZuMkbZb0XUlnJXW5+7NmNlrSbySNl9Qt6QF3/1PzWo2r2lz6Sy+9VLG2bt265Nju7u56WsrFbbfdlqyvXr06Wb///vvzbCecWs7830ha7u5/L+kfJC0xs4mSVkp6w91vkvRGdh3AIFE1/O5+1N3fyy5/IemgpLGSZkvalO22SdKcZjUJIH8X9JzfzMZLmiLpXUnXuvtRqe8PhKRr8m4OQPPUHH4z+46k30n6ibv/+QLGLTazspmVe3t76+kRQBPUFH4zG6a+4P/K3c990uOYmXVm9U5Jxwca6+5d7l5y91JHR0cePQPIQdXwm5lJ+qWkg+7+s36l7ZIWZpcXSnol//YANEstH+mdLmmBpA/MbG+2bZWkpyT91sx+LOmwpB81p8XB79ixY8n6gQMHkvWlS5cm6x9++OEF95SXadOmJeuPP/54xdrs2bOTY/lIbnNVDb+775JUab3vmfm2A6BV+NMKBEX4gaAIPxAU4QeCIvxAUIQfCIqv7q7RyZMnK9YeffTR5Ni9e/cm6x9//HFdPeVh+vTpyfry5cuT9XvuuSdZv/zyyy+4J7QGZ34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCCrMPP+7776brD/99NPJ+u7duyvWenp66uopL1dccUXF2rJly5Jjq3099ogRI+rqCe2PMz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBBVmnn/btm0N1RsxceLEZP2+++5L1ocOHZqsr1ixomLt6quvTo5FXJz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoc/f0DmbjJG2W9F1JZyV1ufuzZrZW0iJJvdmuq9z91dRtlUolL5fLDTcNYGClUknlctlq2beWN/l8I2m5u79nZiMl7TGznVnt5+7+b/U2CqA4VcPv7kclHc0uf2FmByWNbXZjAJrrgp7zm9l4SVMknftOrKVmts/MNpjZqApjFptZ2czKvb29A+0CoAA1h9/MviPpd5J+4u5/lvQLSRMkTVbfI4OfDjTO3bvcveTupY6OjhxaBpCHmsJvZsPUF/xfufvLkuTux9z9jLuflbRe0tTmtQkgb1XDb2Ym6ZeSDrr7z/pt7+y321xJ+/NvD0Cz1PJq/3RJCyR9YGbn1ppeJWm+mU2W5JK6JaXXqQbQVmp5tX+XpIHmDZNz+gDaG+/wA4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBFX1q7tzPZhZr6RP+20aI+lEyxq4MO3aW7v2JdFbvfLs7QZ3r+n78loa/m8d3Kzs7qXCGkho197atS+J3upVVG887AeCIvxAUEWHv6vg46e0a2/t2pdEb/UqpLdCn/MDKE7RZ34ABSkk/GZ2r5kdMrOPzGxlET1UYmbdZvaBme01s0KXFM6WQTtuZvv7bRttZjvN7A/ZzwGXSSuot7Vm9n/ZfbfXzP6poN7GmdmbZnbQzA6Y2T9n2wu97xJ9FXK/tfxhv5kNlfS/ku6W1CNpt6T57v4/LW2kAjPrllRy98LnhM3sdkl/kbTZ3Sdl256WdNLdn8r+cI5y939pk97WSvpL0Ss3ZwvKdPZfWVrSHEkPq8D7LtHXAyrgfivizD9V0kfu/om7n5b0a0mzC+ij7bn725JOnrd5tqRN2eVN6vvlabkKvbUFdz/q7u9ll7+QdG5l6ULvu0RfhSgi/GMl/bHf9R6115LfLun3ZrbHzBYX3cwArs2WTT+3fPo1BfdzvqorN7fSeStLt819V8+K13krIvwDrf7TTlMO0939Vkk/kLQke3iL2tS0cnOrDLCydFuod8XrvBUR/h5J4/pd/56kIwX0MSB3P5L9PC5pm9pv9eFj5xZJzX4eL7ifv2qnlZsHWllabXDftdOK10WEf7ekm8zs+2Y2XNI8SdsL6ONbzGxE9kKMzGyEpFlqv9WHt0tamF1eKOmVAnv5G+2ycnOllaVV8H3XbiteF/Imn2wq498lDZW0wd3/teVNDMDM/k59Z3upbxHTl4rszcy2SJqhvk99HZO0RtJ/SPqtpOslHZb0I3dv+QtvFXqbob6Hrn9dufncc+wW9/aPkv5L0geSzmabV6nv+XVh912ir/kq4H7jHX5AULzDDwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUP8Pt/ALPExulGgAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(train_data[0][0].reshape((28,28)), cmap=\"gist_yarg\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch loading with DataLoader\n",
    "Our training set contains 60,000 records. If we look ahead to our model we have 784 incoming features, hidden layers of 120 and 84 neurons, and 10 output features. Including the bias terms for each layer, the total number of parameters being trained is:<br>\n",
    "\n",
    "$\\quad(784\\times120)+120+(120\\times84)+84+(84\\times10)+10 =\\\\ 94080+120+10080+84+840+10 = 105,214$\n",
    "\n",
    "For this reason it makes sense to load training data in batches using <a href='https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader'><tt><strong>DataLoader</strong></tt></a>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(101)  # for consistent results\n",
    "\n",
    "train_loader = DataLoader(train_data, batch_size=100, shuffle=True)\n",
    "\n",
    "test_loader = DataLoader(test_data, batch_size=500, shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the cell above, <tt><strong>train_data</strong></tt> is a PyTorch <a href='https://pytorch.org/docs/stable/data.html#torch.utils.data.Dataset'><em>Dataset</em></a> object (an object that supports data loading and sampling).<br>\n",
    "The <tt><strong>batch_size</strong></tt> is the number of records to be processed at a time. If it's not evenly divisible into the dataset, then the final batch contains the remainder.<br>\n",
    "Setting <tt><strong>shuffle</strong></tt> to True means that the dataset will be shuffled after each epoch.\n",
    "\n",
    "<div class=\"alert alert-info\"><strong>NOTE:</strong> DataLoader takes an optional <tt><strong>num_workers</strong></tt> parameter that sets up how many subprocesses to use for data loading. This behaves differently with different operating systems so we've omitted it here. See <a href='https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader'>the docs</a> for more information.</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View a batch of images\n",
    "Once we've defined a DataLoader, we can create a grid of images using <a href='https://pytorch.org/docs/stable/torchvision/utils.html#torchvision.utils.make_grid'><tt><strong>torchvision.utils.make_grid</strong></tt></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Labels:  [   0    5    7    8    6    7    9    7    1    3    8    4]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAABUCAYAAACm27J5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAH21JREFUeJztnXucTeX6wL8vY8ZUkzCGwVQUSZ1TKYpKhZyoUGY0fSp9jk5hdDGchNM5Ssqlg5KKklSnXxqhSUlJRBe3Dg5yQgiHGpPp4pKaPe/vj73fd/ae2Xtue/Zae/N8P5/12XuvtWb2M8+8613Pem6v0lojCIIgCIIgVI0abgsgCIIgCIIQy4gxJQiCIAiCEAZiTAmCIAiCIISBGFOCIAiCIAhhIMaUIAiCIAhCGIgxJQiCIAiCEAZhGVNKqeuUUl8rpbYrpYZXl1CCIAiCIAixgqpqnymlVE1gK3AtsBdYA9yqtf6q+sQTBEEQBEGIbsLxTLUDtmutd2itfwNmAz2rRyxBEARBEITYIC6Mn20C7PH7vBe4tKwfUEpJu3VBEARBEGKFfK11g/JOCseYUkH2lTKWlFL3APeE8T2CIAiCIAhu8G1FTgrHmNoLpPl9bgrsK3mS1voF4AUQz5QgCIIgCMcf4eRMrQFaKKWaKaXigUzgneoRSxAEQRAEITaosmdKa12olLoX+ACoCczUWm+uNskEQRAEQRBigCq3RqjSl0mYTxAEQRCE2OFLrfUl5Z0kHdAFQRAEQRDCQIwpQRAEQRCEMBBjShBimP79++PxePB4PKSnp7stjiCckCQmJpKYmMhrr73GU089xVNPPcU555xDo0aNaNSokdviCQ4QTmsEQRBcokuXLgA88cQTOJn3KAhCaU4//XQAbrjhBurUqQPA/fffz0cffQRAt27dAPB4PO4IKEQc8UwJgiAIgiCEwXFXzVe7dm2effZZADIzMxk6dCgAY8aMoWXLlgAcPHgw0mJUmI4dOwLeJxqlvE3lzVNMXl4ef/rTnwD4/fff3RHwOOe8887j7LPPBqBHjx4A/PnPf7bHV6xYwbBhwwBYtWqV8wIGITExkWXLlgFw8cUXM3/+fABuv/12jh075qJkghBZcnJyAsLZb775JgC33nqrWyIFMHjwYFq0aAHA3XffTVycN/gzY8YMAO65J3YWA+nSpQs333wzAOnp6WzcuBGAhIQE+vXrB8DWrVsdl2vIkCGAV9fGIxhhKlTNh9basQ3vcjMR3Xr06KELCwuDbllZWTorKyviMlR069Spk87Pz9f5+fm6sLBQezwe7fF4AmTu3Lmz7ty5swb0BRdcYLfmzZvr5s2bu/43xOKWkpKi586dq+fOnauLioqs3oNtRUVFevbs2Xr27Nmuy222t99+28r36aef6oSEBJ2QkOC6XMfjNmTIEH3gwAF94MABXVRUpEeOHKlHjhzpulwn0paamqqHDRumhw0bptevXx9wfebl5em8vDzXZQwl97p16/S6dev0oUOH9KFDh+xcHq1bcnKynjNnjp4zZ07APanknLhz5069c+dO3apVK92qVStHZczJydE5OTlaa63bt2+v27dvH+nvXFsR+0bCfIIgCIIgCGFw3CWg/+1vfwv4/PTTTwPFobNoICkpCYDx48dz2mmnlXnurFmzANi8eTPXXnstAFprG6q89NJL2blzZ6VlOO2006wcR44c4Ycffqj074g1TJh35cqVNkkUsKGxJUuWALB27VqysrIAaNCgAb/++qvDkganSZMmAFxySbHHedGiRa6F9tq0acOHH34IwOzZs8nMzASgXr16IZPiTSjbHF+zZg1r1661v6OgoACAn3/+mT179kRU/mBcc801AIwYMQKAzp0722Naax588EEAJkyYQGFhYcTkuPLKKwFYunQpNWqE98yrlOLAgQMAPPbYYwA888wz4QnoAO3btwdg8eLFJCYmljq+f/9+zjnnHKfFqjD79+9n6tSpALz44osAjBw50s4zbmHCkJMmTQK889/XX38NeFMcTPVhXl4eK1asAGDTpk2cccYZAPTu3duG12655RYAHn30UUdkT0tLIyMjo0LnDhkyhC+++ALAvkYS8UwJgiAIgiCEwXHjmZo4cSLgfWo3T72///4777zjXXu5a9eurslWklGjRgFw0UUXsWPHDgA+//xz7rjjjlLnpqamBrwa6tWrB8App5xSJRlWrlxpE6/37t1rk6uVUgFehX//+98AHD58GIAvv/wy4Pc4YfFXB+3bt+fjjz8GID4+3u4fPXo0ixYtAor/1oyMDBISEgCv127y5MkOS1uauLg4Bg8eDHjHwvTp0wFvYYVbJCQk2CfXAQMG2P1+OZIcO3bMekWaNm0acA54r1fjaRswYID1XOXl5dmxZZJgI01KSoqdL0466aSg55x66qkAZGVlMWXKlIjJsnTpUqD09VgVtNbUr18fgKeeegqAJ5980h5fuHAh48aNA2D16tVhfVd1ctlllwEE9UoBjB071s5L0UjNmjUDvMgADRs2tP8Lt6IB5nrq3r17qWPHjh2zXtlZs2bx/ffflzqnqKgooEjHScwcCDBnzpyg9x/j0Zw4cSJz5swBnLlPHRfGVL9+/Rg4cGCp/T/++COffPKJCxKFpnv37vbG8+233/KPf/wD8IY4ghlTofj2228ByM/Pr5IcLVq0sJN006ZN7Y2u5OTdu3dvux8oNbGbybfkfhNe1VpbI8aNycMYnZMnT7ZG1L59+2xI6rPPPrPndurUCYBXX33V7hs1ahQbNmxwStyQtG3bluzsbMCr09zcXJcl8k5QZnyYG0RJioqKbCWqMVD9qV27tp2Ys7KyaNCgAeANr1544YWRELsU5qby0ksvWSPqp59+AmDZsmXW0IiLi7PVY926dYuoMdWrVy8Ahg0bxuWXX17mueaBbPLkyXbs9uvXjzPPPNOeU7duXQD69u0LQK1ateyxnj172uvX/D/dpnHjxtx9991Bjx06dAggKq7LsujRowf9+/cP2Pfdd99Ru3ZtlyTyMnbsWKD0nA1eI3v8+PFl/vzcuXMdN6aMgWQq+QDeeuutoOempaXZ906mCkiYTxAEQRAEIQxi2jOVkpICFCf3AUyfPt0m1vl7q5RS1rviNImJidaK7t69O0VFRYA3rLF48WJ73sKFCwFv7yDDV199BWBDUQbz+/bv318lmbKysmw/rnC49NJLgdJPOW+88Ybdb7w/N9xwA7/88kvY31kZTIJku3bt7BPtNddcw/bt2+05bdq0AeDdd98FvGPFJEJHgwcIAgsrBg0axAcffOCiNMWYsWxCeVXBeHjuvPNO65lauXKlIz15LrnkEpsikJKSwpYtWwDo06cPUHz9GTZt2gR4Q/TJyclA1b3DZWHG4pIlS6y3bPjw4UBgCHLz5s289tprAAHXVkmvWc2aNQH461//CsBNN93EAw88AEDr1q2tJ2zo0KFWH24yffr0oMnlO3bssEUCe/fudVqsCmE8T6ZYwZ8DBw7YucUtgt0HzXxiUlDKokOHDo7fS41nCrChu5ycnHJ/7n//+1/EZCpJTBtTxuXnfyPPzc21N0rTsNOc07x5c0flM9Vyr7zyim2+WVRUZOVt1qyZzQc4evQoN954o2OyTZ8+nQULFgDFBgdAdnZ2gD5NrpbJTahVq1ZADoOpNDI31WD7r7jiCgDq1KnjuDF13nnnAd7/v2k6529I1alTh9tuuw0oDkPl5eXZBp7mZ9zCTCKdOnXixx9/BEob1rFMvXr1WLNmDQBnnHGGvdEMGTKklCETCbKzs22V56pVq7j++usBgt7wateubfMMGzRowHPPPQcUG16R4OjRoxw9ehQIfnOuKGYZExNqnzFjhq0q878eOnTo4KoxZZoYm5C7wYRds7Ozo86IqlGjhq0O7tOnj03d8M9z7dmzJwAffvihq411MzIy7Pxu5uLbbrutUnPKxRdfbH9HJB4kSpKWlmZzpfbs2RNwXw+GecAHZw1uCfMJgiAIgiCEQUx6poy3wfQCArjqqquAwITikjhd0WeSnI2XoyTPPfecrfa47777HH9i2bdvH0BAtVrJyjXzpL5t2zbA60nwr8ryT4o2pKam2uTXaFqE1/REguLE9Pnz51vP2ZEjRwDv31yyatEtjK7j4+NtONsUH8QyZqHm3NxcGxYpKCjgvvvuA5yrKktMTLT/9wEDBgT1SBlv1K233hrg3Tae5Hbt2kVVFVxFCeYlNuE+tzApDiWTtB966CGgOPwZDZhqw+zs7JC9j/7+978D8P777wNEtDdZRfAvFNm8eTNQMZ3WrFmTu+66C4DrrrvOFjq88MILEZAykIyMDJtUPmnSpHKTyv1Dgk5Wm8ekMWWaV5rcga+++qpC4RhjPDjFueeeW+45Zo2juLg46143gzwaeO+998o8HqyiIjs7m3/+85/2s6nmy8vLq17hKoB/TpkxpqdMmWIbRTZr1syGKE0jw2gxpKD4gSE/P9/muJTEVD1NmzaN2bNnA9jQZbRh8tOM0Z6QkGAN7szMTD766CPHZTIhsJSUFJ544gmguNrtiiuu4I9//CNQ+gZvDK9169Y5JWq10bZt24Ayc4PJCXODq666itGjR5fan5uba/NkognzwBjKkNqwYYNtReG2EWXSLirbZsQYMS+++KK9727dutWGMiO5ZqxJ45k4caI1oIw+g2GMKH9jSqr5BEEQBEEQYoSY80zVq1fP9mkyy3wMHjyYn3/+Oej5ZgmOunXr8vzzzzsjpI9p06YBgdb78uXLbcJtRkaGdQP37dvXLnljlrBwIgHXCczSN7/99pvj3z1z5kzAG8Jp27Yt4PU8maURtNa2ksU8bUUD5qnXeEVC6e7qq6+2lVtaa9d72JRFYmKivQaN11ZrbRtILlu2zHGZtm3bZpODy6uQ/Oyzz6x+L774YuvRjOTTeXVz8sknA/D4448HLJVj/hY3msCaquxHHnkkoFLRJJ2PHTvWFl9EEyYasmDBAhvyq1u3LnFx3tvqoUOHoqapqPFI+f/PTbVk48aNA6I25p6Znp5ux8NJJ51kG8n27dvXkSiPv+fUVKGW5Wny7y8FzjeUFs+UIAiCIAhCGMScZ2r06NF2oUazOKPJyQmGOTc1NdXx3hhbt24FipMnS/Lss8/ap/Hc3FzOOussoNgid6LPTiRws6dXSUzO1DfffGOfuPw7Q48dO7bU4tjRwNVXXw1gS65NDzKDKQyYMGFCQDfrb775xhkBq8Abb7xhCy5MntTrr7/OyJEjXZPp3Xff5fbbbwegUaNGti+NWYD84MGDNsl23Lhxtlt6bm5umcUu0YYZ86Zv2vnnnx9w3OQLupGbZBZzN20RDGbMG09+tGFyW41nE7y5RSZR+9RTT7XLfZked27RuHFjIHBuNl3xV69ezb333gt4O+Cb60FrbVcQGTdunKO97fr06WM9TV988UXInlImPyotLY309PSAY02bNrU/99Zbb1WoL1U4lGtMKaXSgFeBRkAR8ILW+mmlVD3gTeBMYBfQR2sd8W5kZ511lp2IH3/88XLPNwPdf72waMJckGeffbZNhDUu2SlTpriaEFpV/HXtvy5YQkKC4xWLJuE5OTk54P//9ttvA0SlIQWUWtPLrBkHXje8SeAu2TvN3JiiiXbt2gGB1bSmsjKc3knVwYoVK6zh2qpVK7tGpTGmCgoKAhqS+j8YzZ8/3zlBw6Br1652CRF/I8qEJ5csWeJKbykTcrryyitLHVu4cCH333+/0yKFjQnLA/zhD3+wlaDr1693SySAgPm45L7U1FTmzp1r98+bNw8INEDMvckp/A0jf6MoVLJ/MNLS0gJCf5E2pioS5isEhmqtzwUuAwYppVoDw4ElWusWwBLfZ0EQBEEQhBML40Wo6AbkAtcCXwOpvn2pwNcV+Fld1a1jx466Y8eO2uPx6Hnz5ul58+bpGjVq6Bo1aoT8meTkZH3kyBF95MgRXVhYqBMTE3ViYmKVZYj0Nn78eD1+/HhdWFioCwsL9bRp01yXqSpbdna29ng82uPx6MLCQr1r1y69a9cuXb9+fUfluOqqq/Tq1av16tWrrTxmmzFjhp4xY0bUjof8/Hydn59vx0KDBg3sseXLlwfo99ChQ/rQoUP6xhtv1AkJCTohIcF1+c12zjnnBOh906ZNetOmTbpJkya6SZMmrstXma1ly5a6oKBAFxQUaI/Ho08//XR9+umnuy5XqC0pKUknJSUFjBezHT16VGdnZ+vs7GxXZLv22mv12rVr9dq1a0vJ5vF4dIcOHVzXX2W2+Ph4HR8fr9etW6eLiop0UVGR3rFjh27YsKFu2LCh6/Klpqbq1NRUvWfPHitfML2vW7fOdVkBPWTIEF0eu3fv1jk5OTonJ0dPnDhR7969W+/evdse//zzz3WfPn10nz59dFpaWjjyrK2IbVSpnCml1JnARcAqoKHWej+A1nq/UiqlMr+rsvivI7dz506g9BImJWnatCnx8fEAvPzyy3ZZhmjl4YcfBrzhBsBWnMU6K1euBIqXsog0Jhfg5ZdftrkieXl5vPnmm4B3+Ryz6vl3331n9R5NmKaixhX/0UcfBYRoTN+YL7/80q5BGU25Jca9Pnr06IDQgllWyck1s6qLlJQUTj31VMDbODVUBXE00LVrV7vOmqk0A2yY/eGHHy7VoNdJ2rRpw0UXXVRqvwmHmXzYaCI9Pd2GrIcNG2b316pVi7/85S8AXHDBBXb/qlWr+P77750VMgg1a9Zk0KBBQHHuVCj8+wO6yaRJkwKWgjFVfGVV6Jk5x7w6kSflT4WNKaXUKcBcYLDW+ueKJhgrpe4BYjOTWhAEQRAEoRwqZEwppWrhNaRe11rP8+3+XimV6vNKpQJB21trrV8AXvD9Hh3snMrin4xbFv5VFk4/haWkpNieKbt27arQz5iEUNNTJSMjg9atWwOx1XNq8uTJAQmtRg9xcXER7wScmJhon0b8F87t0aOHXfLjk08+sVVLvXr1ikrPlPEqGE/OZZddFuDhMeN5xowZUTk2rrnmGsBbHWQYNGhQTHqkgvHDDz/YPnfRhEny/9e//hWwdIi/Rwq8T/5uYJYRMv31/Dly5IiVyykvdkUwXuIxY8bYZbX8mTJlCv3797efly9fDmD7IbpNp06dGDFihP1s5j6z0P3UqVNJSkoCokvv4XqV/D2yTlCRaj4FvARs0Vr7X4HvAHcC43yvuRGRsFgOoDi8URamnDwrK8tW3zh9w3n00Udta4Phw4fbBp7B1sMqiflbo6W9QFXwrx4x5eSNGjWK+CreSUlJAY3pTCNO/7XTCgoKrG5r165tGxlGS4M9gMceewzwtj4AGDFiRIDRZ6oQozF0ffLJJwc03Fu8eDHgbY0Qi5jq2m7dulmj5L333iMhIQEgaoyqU045xVbtlTSk3DaiDGbtvcTExFLH5s2bx+uvv+60SOViQmMtW7akZcuWgHetPfOgW7JZ5COPPAIUNx11m8svv9zOd4MGDSrVvPrss8+OqobFsUpFPFOXA3cAG5VSpr5zJF4jKkcpdRewG6h4zaIgCIIgCMJxQrnGlNb6UyCUi6RziP3VjvF0FBUV2eaWpqGYP3Xq1LEu5G3btrnSPwW8S8mYBWjHjh1L3759Afj000+tlwqKF0tNSkqyya2mEdmyZcuiMoRTVTIzMx1JcCyvn5h/H6zk5GSb6B+NujaekOeee856F/bt2xeVHinDQw89ZPvtFBQUcMsttwBEdcJ2WZilfPr168d//vMfoNj7EA0Yb8+ECRO48MILSx1/4oknXPdIGcyyIMGIluTnkpiGmwUFBba4xYTfAXbs2MH27dsBbwgtGpu5mvmua9eupTxTc+bMsffM9PR0Fi1a5Lh81YEpdKpML6rqJGY6oJsB8MwzzzB16lSg2L36/PPPk5mZCXhzTUxehsnbcIMdO3bYpn433XSTXYusdevW1siC4g7d9evXt2ED/wqu44nu3btHfMIsKiqyBkhCQoJd7/Cnn35ixYoVAHYtOPAa3NFoRJXk6NGjtru5E+tiVYVevXoB3nwYU2k7derUmDWiDGaeUUrZkGW00LlzZ5vKYJqPgtcANAbUq6++6oZoxw0m5/WCCy7ggQceAGDo0KG20eWsWbN477333BKvXMwDAHhzR01jX/N3GQMRcLypcnUS6RSS8pC1+QRBEARBEMKhsk07w9mohmZe77//vm1kGGz773//q2+++WZ98803u954zDTNe/LJJ21jRdNoMdhmGqcdOHBAHzhwQJ9//vmu/w1V3UI1hnPiu3v37q179+6tf/nllzIb1Hk8Ht2rVy/XdXW8bBs3btQbN24MGOO33Xab63KFuy1YsEAvWLBAezwePWHCBD1hwgTXZTLbli1bgo7rRYsWuS5bya1Dhw4B86DZTPNO/8a0slXfVrt27YAGqcHmxMOHD+vDhw/rc88913V5q7qZBp3+TT2r6XdXqGmneKYEQRAEQRDCIGZypgz9+vWzi8Ca8vDGjRvb1dBHjRrFwYMHXZPPH9MG4cEHH+Tjjz8G4Prrrw/Zf8Qk1Jsy1Vhc5Nhg8noaNWpk95n8sEhjchnWr19v89bOO++8ANnMItlmwWMhPLp06WLLxqG4R0ystkPwJzU11b43PeDcxsyBKSmBC0+YOcN05I4mPv/8c1s4kZiYaHO6xo0bB0RXj6PjiV9//dX2exs4cCBNmjQBivtnNW7c2LbU2LJliztCVgOmS7rBJKQ7hSqv8qlav6yamnYK0Y8xGE2xAHh7nEyfPt0tkYQIYPoFrVixwlaSHT16lCuuuAKADRs2uCZbdWEaqPbv3582bdoA3mWI3MT0OFq6dCnJycmAtxr1uuuuA2JzuR5BiFK+1FpfUt5JEuYTBEEQBEEIA/FMCREhLs4bQS4Z5vN4PG6JJESQu+66y4Zrnn76acaMGeOyRCcGAwcOtOGamTNnOhZKF4QTiAp5psSYEgRBEARBCI6E+QRBEARBECKNGFOCIAiCIAhhIMaUIAiCIAhCGIgxJQiCIAiCEAZON+3MBw77XoWKk4zorLKIziqP6KzyiM4qj+is8ojOKk916eyMipzkaDUfgFJqbUUy44ViRGeVR3RWeURnlUd0VnlEZ5VHdFZ5nNaZhPkEQRAEQRDCQIwpQRAEQRCEMHDDmHrBhe+MdURnlUd0VnlEZ5VHdFZ5RGeVR3RWeRzVmeM5U4IgCIIgCMcTEuYTBEEQBEEIA8eMKaXUdUqpr5VS25VSw5363lhDKbVLKbVRKbVeKbXWt6+eUmqxUmqb77Wu23K6iVJqplIqTym1yW9fUB0pL1N84+4/Sqk27knuHiF09ohS6n++sbZeKdXd79gIn86+Vkr9yR2p3UUplaaUWqqU2qKU2qyUesC3X8ZaCMrQmYy1ECilaiulViulNvh09qhvfzOl1CrfOHtTKRXv25/g+7zdd/xMN+V3gzJ0NksptdNvnF3o2x/5a1NrHfENqAl8AzQH4oENQGsnvjvWNmAXkFxi3wRguO/9cGC823K6rKOOQBtgU3k6AroD7wMKuAxY5bb8UaSzR4C/Bjm3te8aTQCa+a7dmm7/DS7oLBVo43ufBGz16UbGWuV1JmMttM4UcIrvfS1glW/85ACZvv3TgIG+91nANN/7TOBNt/+GKNLZLCA9yPkRvzad8ky1A7ZrrXdorX8DZgM9Hfru44GewCu+968AvVyUxXW01suBgyV2h9JRT+BV7WUlcJpSKtUZSaOHEDoLRU9gttb6mNZ6J7Ad7zV8QqG13q+1/rfv/S/AFqAJMtZCUobOQnHCjzXfeDnk+1jLt2mgE/CWb3/JcWbG31tAZ6WUckjcqKAMnYUi4temU8ZUE2CP3+e9lH2Bncho4EOl1JdKqXt8+xpqrfeDd7ICUlyTLnoJpSMZe2Vzr8/tPdMvfCw6K4EvlHIR3idgGWsVoITOQMZaSJRSNZVS64E8YDFeD92PWutC3yn+erE68x3/CajvrMTuU1JnWmszzh73jbPJSqkE376IjzOnjKlgVrOUEQbncq11G6AbMEgp1dFtgWIcGXuheR44C7gQ2A9M9O0XnfmhlDoFmAsM1lr/XNapQfadkHoLojMZa2WgtfZorS8EmuL1zJ0b7DTfq+iM0jpTSp0PjABaAW2BesBDvtMjrjOnjKm9QJrf56bAPoe+O6bQWu/zveYB8/FeWN8bl6TvNc89CaOWUDqSsRcCrfX3vgmpCHiR4vCK6MyHUqoWXqPgda31PN9uGWtlEExnMtYqhtb6R2AZ3rye05RSZv1cf71YnfmO16HiIfzjDj+dXecLM2ut9THgZRwcZ04ZU2uAFr7qhHi8SXPvOPTdMYNS6mSlVJJ5D3QFNuHV1Z2+0+4Ect2RMKoJpaN3gL6+ao7LgJ9MiOZEp0TOwE14xxp4dZbpqxpqBrQAVjstn9v48lBeArZorSf5HZKxFoJQOpOxFhqlVAOl1Gm+94lAF7y5ZkuBdN9pJceZGX/pwMfal2V9ohBCZ//1e8hReHPM/MdZRK/NuPJPCR+tdaFS6l7gA7yVfTO11pud+O4YoyEw35dLGAf8n9Z6kVJqDZCjlLoL2A1kuCij6yil3gCuBpKVUnuBUcA4gutoId5Kju3AEeDPjgscBYTQ2dW+0mGNt4q0P4DWerNSKgf4CigEBmmtPW7I7TKXA3cAG325GQAjkbFWFqF0dquMtZCkAq8opWridXDkaK3fVUp9BcxWSo0B1uE1UvG9vqaU2o7XI5XphtAuE0pnHyulGuAN660HBvjOj/i1KR3QBUEQBEEQwkA6oAuCIAiCIISBGFOCIAiCIAhhIMaUIAiCIAhCGIgxJQiCIAiCEAZiTAmCIAiCIISBGFOCIAiCIAhhIMaUIAiCIAhCGIgxJQiCIAiCEAb/D16wOwYpHD5nAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 720x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from torchvision.utils import make_grid\n",
    "np.set_printoptions(formatter=dict(int=lambda x: f'{x:4}')) # to widen the printed array\n",
    "\n",
    "# Grab the first batch of images\n",
    "for images,labels in train_loader: \n",
    "    break\n",
    "\n",
    "# Print the first 12 labels\n",
    "print('Labels: ', labels[:12].numpy())\n",
    "\n",
    "# Print the first 12 images\n",
    "im = make_grid(images[:12], nrow=12)  # the default nrow is 8\n",
    "plt.figure(figsize=(10,4))\n",
    "# We need to transpose the images from CWH to WHC\n",
    "plt.imshow(np.transpose(im.numpy(), (1, 2, 0)));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the model\n",
    "For this exercise we'll use fully connected layers to develop a <a href='https://en.wikipedia.org/wiki/Multilayer_perceptron'>multilayer perceptron</a>.<br>\n",
    "Our input size is 784 once we flatten the incoming 28x28 tensors.<br>\n",
    "Our output size represents the 10 possible digits.<br>\n",
    "We'll set our hidden layers to [120, 84] for now. Once you've completed the exercise feel free to come back and try different values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MultilayerPerceptron(nn.Module):\n",
    "    def __init__(self, in_sz=784, out_sz=10, layers=[120,84]):\n",
    "        super().__init__()\n",
    "        self.fc1 = nn.Linear(in_sz,layers[0])\n",
    "        self.fc2 = nn.Linear(layers[0],layers[1])\n",
    "        self.fc3 = nn.Linear(layers[1],out_sz)\n",
    "    \n",
    "    def forward(self,X):\n",
    "        X = F.relu(self.fc1(X))\n",
    "        X = F.relu(self.fc2(X))\n",
    "        X = self.fc3(X)\n",
    "        return F.log_softmax(X, dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultilayerPerceptron(\n",
       "  (fc1): Linear(in_features=784, out_features=120, bias=True)\n",
       "  (fc2): Linear(in_features=120, out_features=84, bias=True)\n",
       "  (fc3): Linear(in_features=84, out_features=10, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.manual_seed(101)\n",
    "model = MultilayerPerceptron()\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"><strong>NOTE:</strong> You may have noticed our shortcut for adding ReLU to the linear layer. In the last section this was done under the <tt>__init__</tt> section as\n",
    "<pre style='background-color:rgb(217,237,247)'>\n",
    "layerlist = []\n",
    "for i in layers:\n",
    "    layerlist.append(nn.Linear(n_in,i)) \n",
    "    layerlist.append(nn.ReLU(inplace=True))\n",
    "self.layers = nn.Sequential(*layerlist)</pre>\n",
    "    \n",
    "Here we're calling <tt>F.relu()</tt> as a functional wrapper on the linear layer directly:\n",
    "<pre style='background-color:rgb(217,237,247)'>\n",
    "def forward(self,X):\n",
    "    X = F.relu(self.fc1(X))</pre></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Count the model parameters\n",
    "This optional step shows that the number of trainable parameters in our model matches the equation above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_parameters(model):\n",
    "    params = [p.numel() for p in model.parameters() if p.requires_grad]\n",
    "    for item in params:\n",
    "        print(f'{item:>6}')\n",
    "    print(f'______\\n{sum(params):>6}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 94080\n",
      "   120\n",
      " 10080\n",
      "    84\n",
      "   840\n",
      "    10\n",
      "______\n",
      "105214\n"
     ]
    }
   ],
   "source": [
    "count_parameters(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define loss function & optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Flatten the training data\n",
    "The batch tensors fed in by DataLoader have a shape of [100, 1, 28, 28]:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Batch shape: torch.Size([100, 1, 28, 28])\n"
     ]
    }
   ],
   "source": [
    "# Load the first batch, print its shape\n",
    "for images, labels in train_loader:\n",
    "    print('Batch shape:', images.size())\n",
    "    break\n",
    "    \n",
    "# EQUIVALENT TO:\n",
    "# dataiter = iter(train_loader)\n",
    "# images, labels = dataiter.next()\n",
    "# print('Batch shape:', images.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can flatten them using <tt>.view()</tt>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([100, 784])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "images.view(100,-1).size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll do this just before applying the model to our data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the model\n",
    "This time we'll run the test data through the model during each epoch, so that we can compare loss & accuracy on the same plot.\n",
    "\n",
    "<div class=\"alert alert-info\"><strong>A QUICK NOTE: </strong>In the section below marked &nbsp;<tt><em>#Tally the number of correct predictions</em></tt> &nbsp;we include the code\n",
    "<pre style='background-color:rgb(217,237,247)'>\n",
    "predicted = torch.max(y_pred.data, 1)[1]</pre>\n",
    "This uses the <a href='https://pytorch.org/docs/stable/torch.html#torch.max'><tt><strong>torch.max()</strong></tt></a> function. <tt>torch.max()</tt> returns a tensor of maximum values, and a tensor of the indices where the max values were found. In our code we're asking for the index positions of the maximum values along dimension 1. In this way we can match predictions up to image labels.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch:  0  batch:  200 [ 20000/60000]  loss: 0.35221729  accuracy:  82.695%\n",
      "epoch:  0  batch:  400 [ 40000/60000]  loss: 0.32761699  accuracy:  87.340%\n",
      "epoch:  0  batch:  600 [ 60000/60000]  loss: 0.31156573  accuracy:  89.490%\n",
      "epoch:  1  batch:  200 [ 20000/60000]  loss: 0.20120722  accuracy:  94.800%\n",
      "epoch:  1  batch:  400 [ 40000/60000]  loss: 0.14656080  accuracy:  95.185%\n",
      "epoch:  1  batch:  600 [ 60000/60000]  loss: 0.12691295  accuracy:  95.478%\n",
      "epoch:  2  batch:  200 [ 20000/60000]  loss: 0.13621402  accuracy:  96.815%\n",
      "epoch:  2  batch:  400 [ 40000/60000]  loss: 0.07235763  accuracy:  96.790%\n",
      "epoch:  2  batch:  600 [ 60000/60000]  loss: 0.04241359  accuracy:  96.878%\n",
      "epoch:  3  batch:  200 [ 20000/60000]  loss: 0.09474990  accuracy:  97.635%\n",
      "epoch:  3  batch:  400 [ 40000/60000]  loss: 0.06394162  accuracy:  97.600%\n",
      "epoch:  3  batch:  600 [ 60000/60000]  loss: 0.07836709  accuracy:  97.562%\n",
      "epoch:  4  batch:  200 [ 20000/60000]  loss: 0.05509195  accuracy:  98.135%\n",
      "epoch:  4  batch:  400 [ 40000/60000]  loss: 0.06395346  accuracy:  98.125%\n",
      "epoch:  4  batch:  600 [ 60000/60000]  loss: 0.05392118  accuracy:  98.105%\n",
      "epoch:  5  batch:  200 [ 20000/60000]  loss: 0.03487724  accuracy:  98.515%\n",
      "epoch:  5  batch:  400 [ 40000/60000]  loss: 0.03120600  accuracy:  98.433%\n",
      "epoch:  5  batch:  600 [ 60000/60000]  loss: 0.03449132  accuracy:  98.402%\n",
      "epoch:  6  batch:  200 [ 20000/60000]  loss: 0.04473587  accuracy:  98.770%\n",
      "epoch:  6  batch:  400 [ 40000/60000]  loss: 0.05389304  accuracy:  98.770%\n",
      "epoch:  6  batch:  600 [ 60000/60000]  loss: 0.04762774  accuracy:  98.685%\n",
      "epoch:  7  batch:  200 [ 20000/60000]  loss: 0.01370908  accuracy:  98.885%\n",
      "epoch:  7  batch:  400 [ 40000/60000]  loss: 0.01426961  accuracy:  98.945%\n",
      "epoch:  7  batch:  600 [ 60000/60000]  loss: 0.04490321  accuracy:  98.902%\n",
      "epoch:  8  batch:  200 [ 20000/60000]  loss: 0.02279496  accuracy:  99.150%\n",
      "epoch:  8  batch:  400 [ 40000/60000]  loss: 0.03816750  accuracy:  99.060%\n",
      "epoch:  8  batch:  600 [ 60000/60000]  loss: 0.02311455  accuracy:  99.055%\n",
      "epoch:  9  batch:  200 [ 20000/60000]  loss: 0.01244260  accuracy:  99.330%\n",
      "epoch:  9  batch:  400 [ 40000/60000]  loss: 0.00740430  accuracy:  99.340%\n",
      "epoch:  9  batch:  600 [ 60000/60000]  loss: 0.01638621  accuracy:  99.280%\n",
      "\n",
      "Duration: 275 seconds\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "start_time = time.time()\n",
    "\n",
    "epochs = 10\n",
    "train_losses = []\n",
    "test_losses = []\n",
    "train_correct = []\n",
    "test_correct = []\n",
    "\n",
    "for i in range(epochs):\n",
    "    trn_corr = 0\n",
    "    tst_corr = 0\n",
    "    \n",
    "    # Run the training batches\n",
    "    for b, (X_train, y_train) in enumerate(train_loader):\n",
    "        b+=1\n",
    "        \n",
    "        # Apply the model\n",
    "        y_pred = model(X_train.view(100, -1))  # Here we flatten X_train\n",
    "        loss = criterion(y_pred, y_train)\n",
    " \n",
    "        # Tally the number of correct predictions\n",
    "        predicted = torch.max(y_pred.data, 1)[1]\n",
    "        batch_corr = (predicted == y_train).sum()\n",
    "        trn_corr += batch_corr\n",
    "        \n",
    "        # Update parameters\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        \n",
    "        # Print interim results\n",
    "        if b%200 == 0:\n",
    "            print(f'epoch: {i:2}  batch: {b:4} [{100*b:6}/60000]  loss: {loss.item():10.8f}  \\\n",
    "accuracy: {trn_corr.item()*100/(100*b):7.3f}%')\n",
    "    \n",
    "    # Update train loss & accuracy for the epoch\n",
    "    train_losses.append(loss)\n",
    "    train_correct.append(trn_corr)\n",
    "        \n",
    "    # Run the testing batches\n",
    "    with torch.no_grad():\n",
    "        for b, (X_test, y_test) in enumerate(test_loader):\n",
    "\n",
    "            # Apply the model\n",
    "            y_val = model(X_test.view(500, -1))  # Here we flatten X_test\n",
    "\n",
    "            # Tally the number of correct predictions\n",
    "            predicted = torch.max(y_val.data, 1)[1] \n",
    "            tst_corr += (predicted == y_test).sum()\n",
    "    \n",
    "    # Update test loss & accuracy for the epoch\n",
    "    loss = criterion(y_val, y_test)\n",
    "    test_losses.append(loss)\n",
    "    test_correct.append(tst_corr)\n",
    "        \n",
    "print(f'\\nDuration: {time.time() - start_time:.0f} seconds') # print the time elapsed            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the loss and accuracy comparisons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd8VGXWwPHfSSeFloQaIAFCJ7RID01UiqIiIqwNXyu2de1lbezrq+uqy2Lv7toQEV1UsCAgvYQi0pPQEmoSSCCEhJTn/eMOcQgpA0xyJ5nz/Xzmk8zcduYmc+4zz33uuWKMQSmllHfwsTsApZRS1UeTvlJKeRFN+kop5UU06SullBfRpK+UUl5Ek75SSnkRTfrK44jIMyLyid1xlCYik0RkyTku215E1onIMRG5192xlbPNc463OohItIgYEfGzOxZvokm/lhCRXSIy3O44SqssLhEZIiJp1RmTTR4GFhpjwowx0+wORnkvTfpKVY9WwCa7g1BKk74XEJFbRSRZRA6LyGwRaeZ4XUTknyJySESyRWSDiHRxTBslIpsd3RF7ReTBctbdRkTmi0imiGSIyKciUt8x7WOgJfCtiOSIyMOllg0B5gLNHNNzTsUGBIjIfxzb3yQi8U7LNRORr0QkXUR2VtRdIiKBIvKSiOwRkYMi8paI1HFMGyIiaSLygGMf7BeRm5yWDXfsr6MisgpoU8l+HuOINUtEFopIR8fr84GhwGuO99iujGXricj7jhj2isj/iohvZfvYMb2FiMxy7I9MEXmt1LpfEpEjjn01soL4y92vji63mSLyheNvslZEujlN7+h4z1mOfTDGaVodEXlZRHY7/s+WnPobOFzr+PtkiMgTFe1j5QbGGH3UggewCxhexuvDgAygJxAIvAoscky7BFgD1AcE6Ag0dUzbDyQ4fm8A9Cxnu22BixzrjgQWAVMri8tp+hAgrdRrzwB5wCjAF3geWOGY5uOI+SkgAGgN7AAuKWf9U4HZQEMgDPgWeN5p24XAFMDfsb1coIFj+nRgBhACdAH2AkvK2U474LhjX/hjdeckAwGO6QuBWyrYD98Abzu21QhYBdxe2T527J/fgH86lg0CBjqmTQIKgFsd800G9gFSxvYr3K+Ov0kBMM7x/h4Edjp+93e818cdyw4DjgHtHcu+7nj/zR1x9He8l2jAAO8CdYBuQD7Q0e7PU21+2B6APtz0hyw/6b8PvOj0PNTx4Y12fDi3A30Bn1LL7QFuB+qeZRxXAOsqi8tp+hDKTvrznJ53Ak44fu8D7Ck1/2PAh2WsWxyJuI3Ta/2AnU7bPgH4OU0/5Ngfvo791MFp2v9RftJ/Epjh9NwH6yAxxPF8IeUkfaCxI9nVcXptIrCgsn3seD/pzu/Bab5JQLLT82BHkm1SxrwV7lfH32RFqfe3H0hwPA44/w8BnzuW8XHs425lbDPaEU+U02urgAl2fpZq+0PPmtd+zYC1p54YY3JEJBNoboyZ7+gKeB1oKSJfAw8aY44CVwF/BV4QkQ3Ao8aY5aVXLiKNgGlYH/wwrA/5ETfEfcDp91wgyDHKoxVWd1CW03RfYHEZ64jESnRrRKQkZMf8p2QaYwpLbSvUsawfkOo0bXcF8TZznm6MKRaRVKzWbWVaYbWW9zvF6XNq25Xs4xbA7lLvwVnJfjTG5DrWH1pODJXt15J94Xh/aVjvGyDVGFPsNO9urPcegfXtI6Wc+E6LkT/2v6oi2qdf++3D+kADJf3o4VitUIwx04wxvYDOWF0UDzleX22MuRyrq+EbrG6OsjyP1VqLM8bUBa7DSqynVFbG9WzLvKZitdTrOz3CjDGjypg3A6uV2dlp3nrGGFeSSjpW108Lp9daVjB/6f0sjmX3urCtVKyWfoRTnHWNMZ0d0yvax6lYB+zzbcC5sl9L9oWI+ABRWO97H9DC8dopLbHeewZWV12F50NU9dGkX7v4i0iQ08MP+Ay4SUS6i0ggVhfFSmPMLhG5QET6iIg/VjdIHlAkIgEicq2I1DPGFABHgaJythkG5ABZItIcx0HDyUGs/uHyHATCRaSei+9xFXBURB5xnCD0FZEuInJB6RkdLc93gX86WsuISHMRuaSyjRhjioBZwDMiEiwinYAbK1hkBjBaRC507M8HsBL5Mhe2tR/4CXhZROqKiI/j5O1gxywV7eNVWN0sL4hIiOPvPqCybZbBlf3aS0TGOv6v7nO8vxXASqz/n4dFxF9EhgCXAdMdf4MPgFccJ4p9RaSf439R2UCTfu0yB6tle+rxjDHmF6z+5q+wkkMbYIJj/rpYSfEI1tfxTOAlx7TrgV0ichS4A6t1WZZnsU4SZwPfYyVKZ88Df3WM6jhjBJAxZitW/+8OxzzNSs9Tav4irITSHetEYgbwHlDeQeMRrJOMKxzvZR7QvqJtOLkbq6vhAPAR8GEFcW3D2kevOmK6DLjMGHPSxW3dgHUSdDPW32Mm0NQxrdx97LQ/2mKdh0kDrnFxm87xu7Jf/+tY9xGs/4+xxpgCx3scA4x0LPcGcIPjbwvWSd/fgdXAYeDvaO6xjRijN1FRSlVMRJ4B2hpjyjv4qxpCj7ZKKeVFNOkrpZQX0e4dpZTyItrSV0opL+JxF2dFRESY6Ohou8NQSqkaZc2aNRnGmMjK5vO4pB8dHU1iYqLdYSilVI0iIhVdMV5Cu3eUUsqLaNJXSikvoklfKaW8iMf16Sulql9BQQFpaWnk5eXZHYqqRFBQEFFRUfj7+5/T8pr0lVKkpaURFhZGdHQ0TuWdlYcxxpCZmUlaWhoxMTHntA7t3lFKkZeXR3h4uCZ8DycihIeHn9c3Mk36SikATfg1xPn+nWpN0s/KPcm0X5LYuDfb7lCUUspj1ZqkLyL865ckfth4oPKZlVIeJSsrizfeeOOclh01ahRZWVkVzvPUU08xb968c1p/adHR0WRkZLhlXXaoNUm/Xh1/ureoz+KkdLtDUUqdpYqSflFReTdts8yZM4f69etXOM+UKVMYPnz4OcdXm9SapA+QEBvBhr3ZHDnu6s2KlFKe4NFHHyUlJYXu3bvz0EMPsXDhQoYOHcqf/vQnunbtCsAVV1xBr1696Ny5M++8807Jsqda3rt27aJjx47ceuutdO7cmYsvvpgTJ04AMGnSJGbOnFky/9NPP03Pnj3p2rUrW7daN/hKT0/noosuomfPntx+++20atWq0hb9K6+8QpcuXejSpQtTp04F4Pjx44wePZpu3brRpUsXvvjii5L32KlTJ+Li4njwwTNuIldtatWQzYTYSKbOS2JpSgaXxlV41z2lVDme/XYTm/cddes6OzWry9OXdS53+gsvvMDGjRtZv349AAsXLmTVqlVs3LixZGjiBx98QMOGDTlx4gQXXHABV111FeHh4aetJykpic8//5x3332X8ePH89VXX3HddWfe7CsiIoK1a9fyxhtv8NJLL/Hee+/x7LPPMmzYMB577DF++OGH0w4sZVmzZg0ffvghK1euxBhDnz59GDx4MDt27KBZs2Z8//33AGRnZ3P48GG+/vprtm7diohU2h1VlWpPS98Yuh/6muZBeSzeXnP725RSlt69e582Fn3atGl069aNvn37kpqaSlJS0hnLxMTE0L17dwB69erFrl27ylz32LFjz5hnyZIlTJhg3T56xIgRNGjQoML4lixZwpVXXklISAihoaGMHTuWxYsX07VrV+bNm8cjjzzC4sWLqVevHnXr1iUoKIhbbrmFWbNmERwcfLa7w21caumLyAjgX4Av8J4x5oVS0+8A7gKKgBzgNmPMZse0x4CbHdPuNcb86L7wnWQm4zv3Yd4M7sXk7Q9ijNEhaEqdg4pa5NUpJCSk5PeFCxcyb948li9fTnBwMEOGDClzrHpgYGDJ776+viXdO+XN5+vrS2FhIWBd+HQ2ypu/Xbt2rFmzhjlz5vDYY49x8cUX89RTT7Fq1Sp++eUXpk+fzmuvvcb8+fPPanvuUmlLX0R8gdex7nTfCZgoIp1KzfaZMaarMaY78CLwimPZTsAEoDMwAnjDsT73i4iFEc8Tl7uCy4/PICX9eJVsRinlfmFhYRw7dqzc6dnZ2TRo0IDg4GC2bt3KihUr3B7DwIEDmTFjBgA//fQTR44cqXD+QYMG8c0335Cbm8vx48f5+uuvSUhIYN++fQQHB3Pdddfx4IMPsnbtWnJycsjOzmbUqFFMnTq1pBvLDq609HsDycaYHQAiMh24HNh8agZjjHMHYAhw6hB4OTDdGJMP7BSRZMf6lrsh9jNdcAvHU5bywNYv+XnFMNqOmVglm1FKuVd4eDgDBgygS5cujBw5ktGjR582fcSIEbz11lvExcXRvn17+vbt6/YYnn76aSZOnMgXX3zB4MGDadq0KWFhYeXO37NnTyZNmkTv3r0BuOWWW+jRowc//vgjDz30ED4+Pvj7+/Pmm29y7NgxLr/8cvLy8jDG8M9//tPt8buq0nvkisg4YIQx5hbH8+uBPsaYu0vNdxdwPxAADDPGJInIa8AKY8wnjnneB+YaY2aWWvY24DaAli1b9tq926V7AZTt5HF2Pt+XCMkm7M/LoV7zc1+XUl5iy5YtdOzY0e4wbJWfn4+vry9+fn4sX76cyZMn29oir0hZfy8RWWOMia9sWVdO5JbVMX7GkcIY87oxpg3wCPDXs1z2HWNMvDEmPjKy0rt9VSwghNntX8CnKJ/iL2+EQh2+qZSq3J49e7jgggvo1q0b9957L++++67dIVUJV7p30oAWTs+jgH0VzD8dePMcl3WLjl178dCG23gjbRr8/CSM/HtVb1IpVcPFxsaybt06u8Oocq609FcDsSISIyIBWCdmZzvPICKxTk9HA6fGUs0GJohIoIjEALHAqvMPu2L92oTzI/1Y3WQirHwLfp9Z+UJKKeUFKm3pG2MKReRu4EesIZsfGGM2icgUINEYMxu4W0SGAwXAEeBGx7KbRGQG1knfQuAuY0zF11S7QViQPz1b1ue5kxP4psVOmH0vNO4CjTpU9aaVUsqjuXRxljFmjjGmnTGmjTHmOcdrTzkSPsaYPxtjOhtjuhtjhhpjNjkt+5xjufbGmLlV8zbOlBAbyW/7j3N49DsQEAwzrof88oeEKaWUN6g9V+SWkhAbgTGw5KA/jPsAMpOtFv9ZXoChlFK1Sa1N+nFR9akb5Mfi7ekQMwgufAo2zYKVb9sdmlLKDUJDQwHYt28f48aNK3OeIUOGkJiYWOF6pk6dSm5ubslzV0o1u+KZZ57hpZdeOu/1uFutTfq+PsLA2AgWJ2VYl0sPuA/aj4afnoA9K+0OTynlJs2aNSupoHkuSid9V0o112S1NumD1a9/4GgeyYdyQASueAPqtYAvJ0GO1t1XylM88sgjp9XTf+aZZ3j55ZfJycnhwgsvLCmD/N///veMZXft2kWXLl0AOHHiBBMmTCAuLo5rrrnmtNo7kydPJj4+ns6dO/P0008DVhG3ffv2MXToUIYOHQqcfpOUskonV1TCuTzr16+nb9++xMXFceWVV5aUeJg2bVpJueVTxd5+/fVXunfvTvfu3enRo0eF5SnORa0qrVzawLYRACxKyiC2cRjUqQ/j/wPvXwRf/Q9c/w34VE0pIKVqrLmPwoHf3bvOJl1h5AvlTp4wYQL33Xcfd955JwAzZszghx9+ICgoiK+//pq6deuSkZFB3759GTNmTLnFFN98802Cg4PZsGEDGzZsoGfPniXTnnvuORo2bEhRUREXXnghGzZs4N577+WVV15hwYIFREREnLau8konN2jQwOUSzqfccMMNvPrqqwwePJinnnqKZ599lqlTp/LCCy+wc+dOAgMDS7qUXnrpJV5//XUGDBhATk4OQUFBLu9mV9Tqln6LhsG0jgg5/W5aTeNg9CuwcxEseM6+4JRSJXr06MGhQ4fYt28fv/32Gw0aNKBly5YYY3j88ceJi4tj+PDh7N27l4MHD5a7nkWLFpUk37i4OOLi4kqmzZgxg549e9KjRw82bdrE5s2by1sNUH7pZHC9hDNYxeKysrIYPHgwADfeeCOLFi0qifHaa6/lk08+wc/PaoMPGDCA+++/n2nTppGVlVXyurvU6pY+WKN4vkhMJb+wiEA/R6u+x7WQuhIWvwxRvaH9CHuDVMqTVNAir0rjxo1j5syZHDhwoKSr49NPPyU9PZ01a9bg7+9PdHR0mSWVnZX1LWDnzp289NJLrF69mgYNGjBp0qRK11NRXTJXSzhX5vvvv2fRokXMnj2bv/3tb2zatIlHH32U0aNHM2fOHPr27cu8efPo0MF91xjV6pY+WP36eQXFrNlVqkzqyBehaTf4+jY4vNOe4JRSJSZMmMD06dOZOXNmyWic7OxsGjVqhL+/PwsWLKCyYoyDBg3i008/BWDjxo1s2LABgKNHjxISEkK9evU4ePAgc+f+cclQeWWdyyudfLbq1atHgwYNSr4lfPzxxwwePJji4mJSU1MZOnQoL774IllZWeTk5JCSkkLXrl155JFHiI+PL7mdo7vU+pZ+3zbh+PkIi5Iy6N/Wqc/OP8jq3397EMy4AW7+Cfzr2BeoUl6uc+fOHDt2jObNm9O0aVMArr32Wi677DLi4+Pp3r17pS3eyZMnc9NNNxEXF0f37t1Lyh5369aNHj160LlzZ1q3bs2AAQNKlrntttsYOXIkTZs2ZcGCBSWvl1c6uaKunPL8+9//5o477iA3N5fWrVvz4YcfUlRUxHXXXUd2djbGGP7yl79Qv359nnzySRYsWICvry+dOnVi5MiRZ729ilRaWrm6xcfHm8rG1Z6t8W8v53h+Id/fW8ZRevuP8Nl46HE9XP6aW7erVE2hpZVrlqourVzjDYqNYNO+o2Tk5J85sd0lkPAgrPsY1n5c/cEppVQ18oqknxBr1ehfmlzODdOHPg4xg2HOg7D/t2qMTCmlqpdXJP0uzetRP9ifRdvLSfo+vlZ9nuBwq3//RMX3xlSqNvK0rl5VtvP9O3lF0vf1EQa2jWBxUnr5OywkAq7+N2Tvha8nQ3Fx9QaplI2CgoLIzMzUxO/hjDFkZmae1wVbtX70zimDYiP5bsN+th/MoX2Tcm523OICuOQ5mPswLJ0KCfdXb5BK2SQqKoq0tDTS07U8iacLCgoiKirqnJf3mqQ/MNYarrk4Kb38pA/Q+zbrwq35f4PmvaD14GqKUCn7+Pv7ExMTY3cYqhp4RfcOQLP6dWjbKJRFSeX0658iApdNg/BYmPk/cLTKb+mrlFLVxmuSPlglGVbuyCSvoJI7NgaGwjWfQGGeVZGzqKBa4lNKqarmVUl/UGwk+YXFJJYuyVCWyHYw5lWrq+fnp6o+OKWUqgZelfT7tG6Iv6+cXnWzIl3GQp/JsOIN2DiraoNTSqlq4FVJPzjAj/hWDSvv13d20RRo0Qdm3wPp26suOKWUqgZelfQBEtpFsGX/UQ4dq7isagm/ALj6I/ALghnXQ35OlcanlFJVyeuS/qDKSjKUpW4zGPc+ZGyHb/8MegGLUqqG8rqk36lpXRqGBLC4vJIM5Wk9BIb9FTbOhFXvVkVoSilV5VxK+iIyQkS2iUiyiDxaxvT7RWSziGwQkV9EpJXTtCIRWe94zHZn8OfCx1GSYVFSxtlfcj7gL9BuJPz4OKSurpoAlVKqClWa9EXEF3gdGAl0AiaKSKdSs60D4o0xccBM4EWnaSeMMd0djzFuivu8JMRGkJGTz9YDZ3mXeR8fuPJNq7vnyxvh+Fl+W1BKKZu50tLvDSQbY3YYY04C04HLnWcwxiwwxuQ6nq4Azr0wRDU4VWrZ5aGbzuo0gGs+thL+VzdDcSUXeimllAdxJek3B1Kdnqc5XivPzcBcp+dBIpIoIitE5IqyFhCR2xzzJFZHwacm9YJo1ziUxWczdNNZ024w+mXYsRAWPu/W2JRSqiq5kvTPvLU8lNkZLiLXAfHAP5xebum4hdefgKki0uaMlRnzjjEm3hgTHxkZ6UJI5y8hNpKVOw9XXpKhPD2vt26xuOgf1i0XlVKqBnAl6acBLZyeRwFnVCETkeHAE8AYY0zJfQmNMfscP3cAC4Ee5xGv2yTERnCysJhVOw+f+0pG/QOadIVZt8GRXW6LTSmlqoorSX81ECsiMSISAEwAThuFIyI9gLexEv4hp9cbiEig4/cIYACw2V3Bn48+MeEE+PqcW7/+Kf51YPzHgLHuuFXg4gVfSillk0qTvjGmELgb+BHYAswwxmwSkSkicmo0zj+AUODLUkMzOwKJIvIbsAB4wRjjEUm/ToAvF8Q0OPd+/VMaxsCVb1v31p37sHuCU0qpKuLSTVSMMXOAOaVee8rp9+HlLLcM6Ho+AValhNhIXpi7lUNH82hU99xvP0b7kTDwfljyilWnp8e17gtSKaXcyOuuyHWWUHI3LTeMtx/6BMQMgu/vhwO/n//6lFKqCnh10u/YpC4RoQHn169/iq8fXPWBNY7/i+shM+X816mUUm7m1Un/VEmGJckZFBe7oYhaaCSM/w8cT4fX+8CPT8CJrPNfr1JKuYlXJ32w+vUzck6y5cBR96ywRW+4Zy10mwDLX4dXe8Lq96Co0D3rV0qp86BJ3539+qeENYbLX4Pbf4XIjvD9A/DWQEiZ775tKKXUOfD6pN+obhAdmoS5p1+/tKbdYNJ31lj+glz4+Er47BrISHL/tpRSygVen/TBau2v3nmE3JNV0AUjAp3GwN2rYfizsGspvNEXfngMTrhwg3allHIjTfpY/foni4pZeT4lGSrjFwgD74N710KP62DlWzCtB6x8B4oKqm67SinlRJM+0DumIQF+Pmd/N61zEdoILvsX3L4IGneBuQ/BmwMgaV7Vb1sp5fU06QNB/r70iWlYNf365WnSFW78FiZ8BkUn4dOr4JNxkL6t+mJQSnkdTfoOCbERJB3KYX/2ierbqAh0GA13rYSL/xdSV8Ib/WDOw5BbhV1NSimvpUnf4Y+7adlwC0S/QOh/D9y7DnpNgtXvWv39K97S/n6llFtp0nfo0CSMiNBAe5L+KSERcOkrcMdSaNYdfnjEavlv/xHO9ibuSilVBk36DiLCoNgIliSlu6ckw/lo3Amu/wYmfgEY+Gw8fDIWDm2xNy6lVI2nSd9JQrsIjuQWsGmfm0oynA8RaD8CJi+HS56HvWusUT7fPwDHM+2OTilVQ2nSdzKgrVWSYVF1juKpjF8A9LsT7l0PF9wMiR9a/f3LX4fCk3ZHp5SqYTTpO2kUFkTHpnWrd+imq4IbWvfknbwMouLhx8etK3u3zdX+fqWUyzTplzIoNoI1u49wPN9Dq2I26gDXz4JrZ4KPL3w+Af5zORzcZHdkSqkaQJN+KQmxkRQUGVbu9PB+89iLrFb/yBet+/O+NRC+vQ9yPPBbilLKY2jSLyU+ugGBfj4sqo6SDOfL1x/63G6N7+99O6z72Krfv3QaFObbHZ1SygNp0i8lyN+XPq3DPbNfvzzBDWHkC9ZIn5b94OcnrTt3bZtrd2RKKQ+jSb8Mg2IjSEk/zt6saizJ4A6R7eDaGXDdV9ZVvp9PgHWf2B2VUsqDaNIvw6mSDEtqUmvfWdvhcPtiaD0UZt+rLX6lVAlN+mVo1ziURmGBLLKzJMP58guAaz6GpnHw5STYs9LuiJRSHkCTfhlEhITYSJYmZ1Bkd0mG8xEYBn/6Euo2t0o5aBkHpbyeS0lfREaIyDYRSRaRR8uYfr+IbBaRDSLyi4i0cpp2o4gkOR43ujP4qjSoXQRZuQVs3JttdyjnJzTSGtfvFwgfj4WsVLsjUkqVJScd9m+o8s1UmvRFxBd4HRgJdAImikinUrOtA+KNMXHATOBFx7INgaeBPkBv4GkRaeC+8KvOqZIMNWoUT3kaRMN1s+BkjlW4TWv1K+U5iosh8QN4rRfMutV6XoVcaen3BpKNMTuMMSeB6cDlzjMYYxYYY3IdT1cAUY7fLwF+NsYcNsYcAX4GRrgn9KoVERpI52Z1a3a/vrMmXWDi53BkN3x6NZw8bndESqn9G+D9i+C7v0CTOBj/MfhUba+7K2tvDjj3CaQ5XivPzcCp4SIuLSsit4lIoogkpqd7Tss6ITaStbuPkOOpJRnOVvRAGPc+7FsLM27UG7QoZZf8Y/DDY/DOYDiyC658x7p9amS7Kt+0K0lfynitzLObInIdEA/842yWNca8Y4yJN8bER0ZGuhBS9RgUG0FhsWFFioeXZDgbHS+D0a9A8s/w37ur/KukUsqJMbDpG3itN6x407pT3j2J0O0aq5x6NfBzYZ40oIXT8yhgX+mZRGQ48AQw2BiT77TskFLLLjyXQO3QK7oBQf4+LE5KZ3inxnaH4z7xN8HxdFjwHIQ2gov/ZndEStV+h3fCnIesBleTrtaQ6qj4ag/DlaS/GogVkRhgLzAB+JPzDCLSA3gbGGGMOeQ06Ufg/5xO3l4MPHbeUVeTQD9f+rYOt/cWilVl0EOQcwiWTbMSf/977I5IqdqpMN+qh7X4JfDxgxEvwAW3gq8r6df9Kt2qMaZQRO7GSuC+wAfGmE0iMgVINMbMxurOCQW+FOsryh5jzBhjzGER+RvWgQNgijGmRg0dSYiN5G/bNpN6OJcWDYPtDsd9RGDk360W/09/hZBI6DbB7qiUql12LoLv7ofMJOh0BYx4Huo2szUklw41xpg5wJxSrz3l9PvwCpb9APjgXAO026BYa+jmkuQMJvZuaXM0bubjC2PfgROH4b93QXC4VbJZKXV+cg5ZjakNX1hDpq/9CmLLTZPVSq/IrUTbRqE0qRtUO8brl8UvEK75FBp1ghk3QFqi3REpVXMVF8Pq9+G1eNg4y+pGvXOFxyR80KRfKaskQwRLkmp4SYaKBNW1KnOGNoZPx0H6NrsjUqrm2f8bvD8cvr/fGnM/eRkM+yv417E7stNo0ndBQrtIjuYVsiEty+5Qqk5oI6tcg4+/Va4he6/dESlVM5SMuR8CWXtg7LvVNub+XGjSd8HAthGIUDtH8Thr2Bqumwl52fDJVVquQamKGAObvobXLnCMub8J7l4NceOrbcz9udCk74KGIQF0aVav9vbrO2vaDSZ8CodT4POJcDK38mWU8jaHd1hdoV9Oska+3fILXPpM6LyfAAAgAElEQVQK1PH80mKa9F2UEBvB2j1ZHMvzgtIFrQdbX1FTV8LM/4GiWlKGQqnzVZgPv/4D3uhn3aNixAtw6wKI6mV3ZC7TpO+ihNhIiooNy2tTSYaKdL4CRr8E2+fCd3+2vsoq5c12/ApvDoAF/wvtRsDdq6DvZNsusjpXNStaG/VsVZ/gAF8WJ2VwcecmdodTPS64xRpv/OvfIaQRDH/a7oiUqn45h+DHJ+D3GdAgxqPG3J8LTfou+qMkgxf06zsb8hjkHIQlr1gjfPpOtjsipapHcRGs+RDmTYHCEzDoYUi43+OGYJ4tTfpnISE2gvlbD7EnM5eW4bWoJENFRKyqnLmZ8MOj1kmrruPsjkqpqrVvvVXjft9aiBlkfQYiYu2Oyi20T/8sJMRaZZ8XJ3tZa9/HF8a+B60GwNd3QPIvdkekVNXIOwpzH4F3h0J2qvV/f8PsWpPwQZP+WWkTGUKzekEs3l7Lx+uXxT8IJnwGke3hi+th7xq7I1LKfYyxyia8dgGsfBvi/wfuToS4qz16zP250KR/FqySDJEsTcmgsMgLbz5Sp75VriEk3LrlYkay3RG5JjvN+iBv+VZHIanTFebDzsXWxYgzb7LOW93yC4x+2fp/r4W0T/8sJbSL4IvEVH5Ly6ZXK8+/EMPtwprA9d/A+xfDx1fCzT9B3aZ2R3WmYwesOxRtmmVdb3BKq4FWSekmXeyLTdmnuBgObYYdC2DHQti9DApyISAMRvzdGrFWw4Zgnq3a/e6qwIA2p0oypHtn0gcIbwPXfgn/vsy6KnHS957RKjqeAZv/a10av2sJYKBRZ6voVacrrNd+mQJvJ0D8zTD0cQhuaHfUqqplp1kJPmUB7PzVuocEQEQ76HEdtB4C0QlW4UEvIMbDvu7Gx8ebxETPLu97+WtL8PP14avJ/e0OxV4p8+HT8dCit9XtY8dQttzDsPU7qz925yIwRdaHufNY6DLWOgfh7MQRWPB/sPo9CKoPFz4JPW+0Tlar2uFElnWA37HQatFnOrohQxpZCf7Uo15zmwKsGiKyxhhT6f0XNemfg5d+3Mabv6aw9smLqFfH3+5w7PX7TPjqZuhwKYz/T/Ukz7xs2DrH6rpJmQ/FhdZFM13GWsm+cefKT74d2GiN0ti9xCqDO+of0LJv1ceu3K/wJKStciT5hdYgA1MM/iEQPeCPJN+oU607KevM1aSv3TvnICE2gtcWJLM8JZMRXbzk6tzydB1ndav88IhVR/zSqVXzwcrPge0/WC365HlQlA/1WkDfO61k37T72W23SReY9J3VFfTTX+GDS6DreLhoimeeo1B/MAYObvojye9eavXLiw807wUJD1pJPuoC8AuwN1YPpEn/HPRo2YCQAF8WJ6Vr0gfoewccPwSLX7a+Qg97wj3rLTgB23+0WvTbf7Kuigxrag2n63IVRMWf3wFGxDpgtLsElvzTunn11u9h8EPWwcQv0D3vQ52/7L1/nHzd8av1/wYQHgvdr4U2QyF6IATVszXMmkCT/jkI8POhX5vw2l9f/2wMe9KqUbLoRWvYW+9bz209hfnWxV+bZsG2uXAyx7oKuMe1VtdNy37g4+aRxgEh1sne7tdaNVbmPQNrP7YqKLa72L3bUq7Jy7b65VMciT4zyXo9JNLRXTPUqgZbL8rGIGsmTfrnKCE2knlbDrE78zitwkPsDsd+IlbXTm4mzHnIusl6l7GuLVtUYH2wN86yWtr52VZd8i5XWetoNbB6htE1jIGJn0HSPKvkxGdXW9UUL/k/a8SSqjqFJyFt9R8nX0v65YOtK8F7TbJa87W8X746aNI/RwmxEQAsSsrgek36Fl8/GPeBNX5/1m3WcMjWQ8qet6gQdi22WvRbvrVG1QTWg46XWi361oPB16aT5LHDIWYZrHobFv4d3ugL/e6GhAcgMNSemGqjo/utv/+OhbBrKRQcd+qXf8DRL99b++XdTEfvnCNjDAP/voDOzeryzg2VnjD3LieOwIejICvVOlnarLv1enEx7Flmtei3zLbGSweEQvtRVou+zTDP60c/dgDmPQu/fQZhzeDiv1nfQLS1eW6MsYbWrn7P+lZniqx++dZDHOPlB3rGNR81kA7ZrAaPzdrAd7/tZ+1TF+HvqxUtTnN0n3XVbmEeXPYv61L3zd/Asf3gV8c6edplLMReXDNK1aausrqt9q+3ziuMfBGaxtkdVc1x4gis/xwSP7D65+s0sM6hxP+Pdp25iVuTvoiMAP4F+ALvGWNeKDV9EDAViAMmGGNmOk0rAn53PN1jjBlT0bZqUtKf8/t+7vx0LTPv6Ed8tF7ZeYb07dZQyBOHwTcQYi+Czlda/eQ1sZukuBjWfQy/PGslsV43WSeA9are8u1bZ7Xqf//KGn0VdYF1NXTnK2rGwb4Gcds4fRHxBV4HLgLSgNUiMtsYs9lptj3AJODBMlZxwhjT3aWoa5j+bcLxEatfX5N+GSLbwU1z4eBGq0Vf0y9z9/GBXjdCpzGw8AVY9a7VJz3sr9YBQK/qtZzMtfbL6vetevT+wRA3Hi64GZp2szs6r+fKidzeQLIxZgeAiEwHLgdKkr4xZpdjmleVnqwfHEBcVH0WJ6Vz/0Xt7A7HMzXqYD1qkzoNrKJtPW+EuQ/D9w9A4kcw6kVo5cWlOTKSre6b9Z9CXhZEtLe6wbpN0PHzHsSVjujmQKrT8zTHa64KEpFEEVkhIleUNYOI3OaYJzE9vWbdoGRQbAS/pWaRnVtgdyiqujXuBDd+C1f/20pyH46EmTdb5zO8RVEhbJ4N/7kcXutljXhqMxRu/A7uWgl9bteE72FcSfplDVM4m7O/LR39TH8CporIGWdtjDHvGGPijTHxkZGRZ7Fq+yW0i6TYwLIUvVDLK4lY/dN3rYLBj1jDT1+Nt65OLsy3O7qqc3S/1cU1tSvMuN5q5Q/9K/xlM1z9EcQk6AgnD+VK904a0MLpeRTgclPGGLPP8XOHiCwEegApZxGjR+veoj6hgX4sSspgZFet2eK1AoKtUs3d/2Rd1fvLlD+u6m0/wu7o3MMYqzTx6vf/GG7Z5kLrhiOxF9f6OvS1hSt/pdVArIjEAHuBCVit9kqJSAMg1xiTLyIRwADgxXMN1hP5+/rQv004i7anY4xBtHXj3RpEw4RPreqfcx+Fz6+xEuIlz0NEW7ujOzdnDLdsCP3ugviboGFru6NTZ6nSpG+MKRSRu4EfsYZsfmCM2SQiU4BEY8xsEbkA+BpoAFwmIs8aYzoDHYG3HSd4fYAXSo36qRUS2kXy0+aD7MrMJSZCr85VWBeaTV4Kq96xukHe6Av97oRBD0FgmN3RuWbvWkh832m4ZW+48m3rhjT+QXZHp86RXpzlBrszjzP4HwuZcnlnbugXbXc4ytMcO2h196z/BEKbWEXcQhs7Ho2s10IbWc8Dgu2N9YzhliHWzcHjb9aL0Tyc1tOvRq3CQ2jZMJhF2zM06aszhTWGK163ukN+mWKViT5+yCooVlpAmDV/yQGh8emPU9OCw917XUDJcMtPrAqXkR1g5D+g2zU6+qaW0aTvJgmxEXyzbi8FRcVakkGVLSoebpxt/V5cZFUkzTlofRPIOfU4BDkHrJ8HfoecXyD/6JnrEh+rzPBpB4VG1o3rSx8syrv6uagQts2xrpjd+Sv4+EPHy6ybg7fqr6NvailN+m6SEBvJpyv3sG5PFr1j9OpcVQkfX0dybgRNulY878njjoOB0wHh1EHi1AHj4Cbr20Nx4ZnL+4f8cSA49U3Bx9/qxjm237oD2bAnoecN1nyqVtOk7yb92oTj6yMsTkrXpK/cKyDEqvXfMKbi+YqLrTpHpQ8IzgeJQ1usUsZ5R6HthXDpP63RRVpCwmto0neTenX86d6iPouSMnjg4vZ2h6O8kY8PhERYj8adK563uEgTvZfSzmc3SoiNYENaFlm5J+0ORamKacL3Wpr03SghNhJjYGlypt2hKKVUmTTpu1G3qHqEBfmxOKlmFY1TSnkPTfpu5Ofrw4A2ESxOysDTLnpTSinQpO92Ce0i2Jt1gi37j9kdilJKnUGTvptd0rkJYYF+vPDDVm3tK6U8jiZ9N4sIDeQvF7Vj0fZ0ftx00O5wlFLqNJr0q8AN/VrRoUkYf/tuMydOFtkdjlJKldCkXwX8fH14dkxn9mad4I2FyXaHo5RSJTTpV5E+rcO5onsz3v51B7syjtsdjlJKAZr0q9TjozoS4OfDs99u0pO6SimPoEm/CjWqG8R9w2NZsC2deVsO2R2OUkpp0q9qN/aPpl3jUJ79dhN5BXpSVyllL036Vczf14dnx3Qh7cgJ3lyYYnc4Sikvp0m/GvRrE85l3Zrx5q8p7MnMtTscpZQX06RfTZ4Y1RF/H2HKd5vsDkUp5cU06VeTJvWCuPfCWOZtOcT8rXqlrlLKHpr0q9FNA2JoExnCM7M360ldpZQtNOlXowA/H6Zc3oU9h3N5Z9EOu8NRSnkhTfrVbEDbCEZ3bcrrC5JJPawndZVS1culpC8iI0Rkm4gki8ijZUwfJCJrRaRQRMaVmnajiCQ5Hje6K/Ca7InRHfER4W/fbbY7FKWUl6k06YuIL/A6MBLoBEwUkU6lZtsDTAI+K7VsQ+BpoA/QG3haRBqcf9g1W7P6dbjnwrb8tPkgC7fplbpKqerjSku/N5BsjNlhjDkJTAcud57BGLPLGLMBKC617CXAz8aYw8aYI8DPwAg3xF3j3TKwNa0jQnhm9ibyC/WkrlKqeriS9JsDqU7P0xyvucKlZUXkNhFJFJHE9HTvuKl4gJ8Pz4zpzK7MXN5bvNPucJRSXsKVpC9lvOZqyUiXljXGvGOMiTfGxEdGRrq46ppvULtIRnRuwqvzk9ibdcLucJRSXsCVpJ8GtHB6HgXsc3H957OsV3jyMuv0yP/qSV2lVDVwJemvBmJFJEZEAoAJwGwX1/8jcLGINHCcwL3Y8ZpyaF6/DncPbcvcjQdYnOQdXVtKKftUmvSNMYXA3VjJegswwxizSUSmiMgYABG5QETSgKuBt0Vkk2PZw8DfsA4cq4EpjteUk1sHtSY6PJinZ2/iZGHpc+FKKeU+4ml3dIqPjzeJiYl2h1HtFmw7xE0fruaRER2YPKSN3eEopWoYEVljjImvbD69ItdDDG3fiIs6NebV+Unsz9aTukqpqqFJ34M8dWkniooN//v9FrtDUUrVUpr0PUiLhsHcOaQt32/Yz9LkDLvDUUrVQpr0Pcztg1vTsqGe1FVKVQ1N+h4myN+Xpy/rRPKhHD5aplfqKqXcS5O+B7qwY2Mu7NCIf81L4uDRPLvDUUrVIpr0PdTTl3WmoNjwnJ7UVUq5kSZ9D9UyPJg7Brdh9m/7WJ6SaXc45+zQ0Tw27cu2OwyllIMmfQ9255A2RDWow9OzN1JQVLNO6hpjmLU2jWEv/8roaUsY/9ZyFm1Px9MuBlTK22jS92BB/r48eWknth/M4d/LdtkdjsuyTxTw5+nruX/Gb3RsGsYTozqSeiSXGz5YxRWvL+XnzQcpLtbkr5Qd/OwOQFXs4k6NGdwukqnzkhjTrRmN6gbZHVKFVu08zF++WM+Bo3k8eHE7Jg9pi6+PcGP/aGatTeONhSnc+p9EOjQJ466hbRnVtSm+PmVV4FZKVQVt6Xs4EeGZMZ05WVjM83O32h1OuQqKinn5p21MeGc5fr7CzDv6cfew2JKEHuDnw4TeLZn/wGD+eU03CoqKuefzdVz0yq/MXJNW47qvlKqpNOnXADERIdw6KIav1+1l1U7PK1K6K+M4495azqvzk7mqZxTf35tAj5Zl3wrZz9eHK3tE8dNfBvPGtT0J9PflwS9/Y+hLC/l05W69daRSVUyrbNYQuScLGf7yr9St48939wzEz9f+47Uxhi/XpPHM7E34+QjPj41jdFzTs17H/K2HeHV+MutTs2hSN4jbBrVmYu+W1AnwraLIlap9tMpmLRMc4MeTl3Zi64FjfLxit93hkJ1bwN2frePhmRuIi6rHD/cNOuuED1b31YUdG/P1nf355OY+tAoPZsp3mxn49/m8uTCFY3kFVRC9Ut5LW/o1iDGGGz5Yxfo9Wcx/cAiRYYG2xLE8JZP7Z6wn/Vg+D1zcntsGtXbrydhVOw/z2oJkFm1Pp14df24aEM2k/tHUDw5w2zaUqm1cbelr0q9hUtJzGDF1EWO6Nefl8d2qddsnC4t55eftvL0ohZjwEP41oQddo+pV2fZ+S83itQXJ/Lz5IKGBflzXtxW3JMQQEWrPwU4pT6ZJvxZ7Ye5W3vo1ha8m96NXq4bVss2U9Bzum76e3/dmM7F3C568tBPBAdUz4nfrgaO8viCF7zbsI9DPh4m9W3L7oDY0qefZw1eVqk6a9Gux4/mFDH/lVxoEB/DtPQOrdJy7MYbpq1OZ8u1mAv19eGFsHCO6NKmy7VUkJT2HNxem8PW6vfiKMC4+ismD29CiYbAt8SjlSTTp13LfbdjH3Z+tY8rlnbmhX3SVbOPI8ZM8OmsDP246yIC24bx8dXePaF2nHs7lrV9T+DIxjSJjuKJ7c+4c2oY2kaF2h6aUbTTp13LGGK59byUb92az4MEhhLu5n3tJUgYPfLmew8dP8vAlHbh5YAw+Hnbl7IHsPN5ZtIPPVu0mv7CY0V2bctfQtnRsWtfu0JSqdpr0vUDyoWOMmLqYsT2b8+I495zUzS8s4uWftvPOoh20iQxh2sQedG5WdSdr3SEjJ5/3l+zkP8t2cfxkERd1aszdQ9vSrUV9u0NTqtpo0vcS/zdnC+8s2sGsO/vTs5yrYF2VfOgY936+ns37j3Jd35Y8MapTjbpAKiv3JB8t28WHS3eRfaKAQe0iuXtoW3rHVM/JbqXspEnfS+TkF3LhywtpFBbEN3cNOKeTusYYPl25h//9fjPBAX68eFUcwzs1roJoq8exvAI+WbGH95fsICPnJL1jGnLPsLYMbBuBiGd1USnlLm69IldERojINhFJFpFHy5geKCJfOKavFJFox+vRInJCRNY7Hm+d7RtRFQsN9OPxUR35fW8201fvOevlM3PyufU/ifz1m430jgnnh/sSanTCBwgL8mfykDYsfngYT1/WiT2ZuVz//ipGTVvCR0t3cuT4SbtDVMo2lbb0RcQX2A5cBKQBq4GJxpjNTvPcCcQZY+4QkQnAlcaYaxzJ/ztjTBdXA9KW/tkzxjDhnRVsO3iMBQ8MoUGIa1eu/ro9nQe//I3s3AIeHdmBSf2jPe5krTvkFxYxa+1ePl25m417jxLg68PFnRszPr4FA9pGaGlnVSu4rXtHRPoBzxhjLnE8fwzAGPO80zw/OuZZLiJ+wAEgEmiFJv1qse3AMUZNW8z4+BY8P7ZrhfPmFRTx4g/b+GDpTto1DuVfE3p4zYiXTfuy+TIxja/X7SX7RAHN69fhql5RXN0rSsf7qxrN1aTvyiWVzYFUp+dpQJ/y5jHGFIpINhDumBYjIuuAo8BfjTGLywj2NuA2gJYtW7oQkiqtfZMwJvWP5oOlO5nYuwVxUWWPXNl24Bh/nr6OrQeOMal/NI+O7ECQf805WXu+OjerR+cx9Xh0ZAd+3nyQGYmpvDo/iWm/JDGgbTjj41twSecmXrVPlHdxpaV/NXCJMeYWx/Prgd7GmHuc5tnkmCfN8TwF6A3kAKHGmEwR6QV8A3Q2xhwtb3va0j93x/IKGPbyrzSrX4evJ/c/ravGGMN/lu/muTlbqBvkxz+u7sbQ9o1sjNZz7M06wczENL5ck0rakRPUDfLjih7NGR/fgi7NPXu46tnILyxi/Z4slqZksm7PEfx8hJBAP8KC/AgN9CM00J/QID9CA32dfremhwSemsdPu8M8lDtb+mlAC6fnUcC+cuZJc3Tv1AMOG+uIkg9gjFnjOBi0AzSrV4GwIH8eH9WBv3zxGzMSU5nQ2/rWlH4sn4dn/saCbekM69CIF8fFadEyJ83r1+HPw2O5Z1hblu/I5IvVqUxfncp/lu+mU9O6jI+P4ooezWtclc+iYsOmfdksTc5kWUoGq3cdJq+gGB+BDk3q4ucr7M7M5Vh+ITl5hZwocO0GNsEBvtYBIOiPA8Gp52GBjgOE43drHn9CAn0JK3UgCfTz0dFUNnClpe+HdSL3QmAv1oncPxljNjnNcxfQ1elE7lhjzHgRicRK/kUi0hpY7Jiv3Ns/aUv//BhjuObtFSQdOsaCB4ewds8RHvpyAzn5hTwxuiPX922lHzQXZOcW8N/f9jIjMfWMk78D20Z45AlvYwzJh3JYlpLJ0uQMVuzI5GheIQDtGofSv00E/duE06d1OPXq+J+xfGFRMcdPFpHjOAjk5BdwLK+QnPxCjucXlvyec+pn/unPjzm9XuTCje99fYTI0EDG9mzO9f1a0bReHbfvE2/i1nH6IjIKmAr4Ah8YY54TkSlAojFmtogEAR8DPYDDwARjzA4RuQqYAhQCRcDTxphvK9qWJv3zt2X/US59dQnR4cGkpB+nQ5Mwpk3sQbvGYXaHViN58snftCO5LHO05JelZHLoWD4AUQ3qMKBNBP3bhtOvTTiNwqqvZpIxhvzC4tMOEsfyCzieX0ROfoHjufX69oM5zN96EBFhRJcm3NQ/ml6tGmjD5BzoxVlebsq3m/lg6U5uHhjDwyPaE+inJybPV15BUcnJ3yXJGQAMaBPB1fFR1XbyNyMnn2UpmSxPyWBpciZ7DucCEBEaQL82EQxoE86AthG2H4zORurhXD5esZvpq/ZwNK+Qrs3rMal/NJd2a6r/t2dBk76XKyo27Ms6UaM+/DVJ2pFcZq5J48vENPZmVd3J36N5BazacZilKRksT8lk64FjAIQF+tGndTgD2obTv00E7RqH1vjWce7JQmat3ctHy3aRfCiHiNAA/tSnFdf1aUmjuvZXd/V0mvSVqgbFxYZlKZnMSEzlh00HOFlYfF4nf/MKiliz+wjLHC353/dmU1RsCPTz4YLohvRztOS7NKuLn2/tvMW1MYYlyRl8tHQX87cdws9HGN21KZMGxNBdi+iVS5O+UtUsK/cks3/bxxerU9m074+Tv9dc0IIBbco++VtYVMyGvdksS7aS/Jo9RzhZWIyvj9Atqh4D2kbQr004PVs28MprB3ZlHOffy3fxZWIaOfmF9GhZn0n9oxnVtSn+tfSgd6406Stlo417s/kyMZVv1u8rOfk7rlcU43pFkZNfyNJkq7tm5c7D5ORbI2w6Nq1L/zZWl03vmHBCA6vndpQ1wbG8Ar5ak8ZHy3axKzOXxnUDua5PK/7Up6Xb7yVRU2nSV8oD5BUU8dPmg3zpOPnr/HGLDg+mf1trGGW/1uGavFxQXGz4dXs6HyzdyeKkDAL8fBjTrRk3DYj2+Ps+VDVN+kp5mLQjucz5fT8NggPo3zaC5vV1XPr5SD50jI+W7eKrNXs5UVBE7+iG3DQgmos6Na615zsqoklfKeUVsk8U8GViKh8t20XakRM0qxfE9f2imdi7RY27ivp8aNJXSnmVomLDL1sO8uHSXSzfkUmQvw9X9mjOpP4xtG9S+y9M1KSvlPJaWw8c5d/LdjFr7V7yC4vp3yacmwbEMKxDo1pbME6TvlLK6x05fpLpq1P5ePku9mXn0bJhMDf0a8XV8S3KrD9Uk2nSV0oph8KiYn7afJCPlu5i1a7DBAf4Mq5XFDf2j6ZNZKjd4bmFJn2llCrDxr3ZfLRsF7PX7+NkUTGD20VydXwUbRuF0qJBMCE19PoITfpKKVWBjJx8Pl+5h49X7C6pTgpW8boWDYNp6Xic+r1VeDCNw4I8sqw2aNJXSimXnCwsZuuBo+w5nMuew7mkOn7uOZzLvqy80+4NEODrQ1TDOiUHBOeDQsuG9n5LcOeds5RSqtYK8PMhLqp+mfeVLigqZl/WiZKDgPNBYc3uIxxz3KTmlPK+JbRsGEyTup7xLUGTvlJKlcPf14dW4SG0Cg8pc3p2bgG7Dx8/44Cwds8Rvtuw3yO/JWjSV0qpc1Qv2J+44PK/JezPymPP4dySA0Nl3xL6tYng1Yk9qjRmTfpKKVUF/H19aBkeTMvwYAYSccb07NyC07qN9hzOpWFI1V87oElfKaVsUC/Yn67B9egaVb3VQb2vFJ1SSnkxTfpKKeVFNOkrpZQX0aSvlFJeRJO+Ukp5EU36SinlRTTpK6WUF9Gkr5RSXsTjqmyKSDqw+zxWEQFkuCmcmk73xel0f5xO98cfasO+aGWMiaxsJo9L+udLRBJdKS/qDXRfnE73x+l0f/zBm/aFdu8opZQX0aSvlFJepDYm/XfsDsCD6L44ne6P0+n++IPX7Ita16evlFKqfLWxpa+UUqocmvSVUsqL1JqkLyIjRGSbiCSLyKN2x2MnEWkhIgtEZIuIbBKRP9sdk91ExFdE1onId3bHYjcRqS8iM0Vkq+N/pJ/dMdlJRP7i+JxsFJHPRSTI7piqUq1I+iLiC7wOjAQ6ARNFpJO9UdmqEHjAGNMR6Avc5eX7A+DPwBa7g/AQ/wJ+MMZ0ALrhxftFRJoD9wLxxpgugC8wwd6oqlatSPpAbyDZGLPDGHMSmA5cbnNMtjHG7DfGrHX8fgzrQ93c3qjsIyJRwGjgPbtjsZuI1AUGAe8DGGNOGmOy7I3Kdn5AHRHxA4KBfTbHU6VqS9JvDqQ6PU/Di5OcMxGJBnoAK+2NxFZTgYeBYrsD8QCtgXTgQ0d313siEmJ3UHYxxuwFXgL2APuBbGPMT/ZGVbVqS9KXMl7z+rGoIhIKfAXcZ4w5anc8dhCRS4FDxpg1dsfiIfyAnsCbxpgewHHAa8+BiUgDrF6BGKAZECIi19kbVdWqLUk/DWjh9DyKWv4VrTIi4o+V8D81xsyyOx4bDQDGiMgurG6/YSLyib0h2SoNSDPGnPrmNxPrIOCthgM7jTHpxpgCYBbQ3+aYqlRtSfqrgVgRiRGRAKwTMbNtjsk2IiJYfbZbjDGv2B2PnUW/VJUAAAC0SURBVIwxjxljoowx0Vj/F/ONMbW6JVcRY8wBIFVE2jteuhDYbGNIdtsD9BWRYMfn5kJq+YltP7sDcAdjTKGI3A38iHX2/QNjzCabw7LTAOB64HcRWe947XFjzBwbY1Ke4x7gU0cDaQdwk83x2MYYs1JEZgJrsUa9raOWl2TQMgxKKeVFakv3jlJKKRdo0ldKKS+iSV8ppbyIJn2llPIimvSVUsqLaNJXSikvoklfKaW8yP8D9RANmyLkI1YAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(train_losses, label='training loss')\n",
    "plt.plot(test_losses, label='validation loss')\n",
    "plt.title('Loss at the end of each epoch')\n",
    "plt.legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows some evidence of overfitting the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XlcVPX+x/HXB1ARVwQ1N8Tcl0QRt1wzNTOvlrm1at30tpnee+22/OqqmdU1s73utdI2K82ytJtWlmiWpuCWG2CJiqiMiggiyPL9/TEjFw1khBkOM/N5Ph4+ZObMnPOZM8ObM9/zme+IMQallFKez8/qApRSSrmGBrpSSnkJDXSllPISGuhKKeUlNNCVUspLaKArpZSX0EBXFZ6IhIuIEZEAq2u5mKOuFqW4n4jIQhFJFZFN7qitmO2Wqt7yIiLRInKP1XV4Kg30cuZ4waaKSBWra7GCiMwQkQ9LuE2iiAwsr5os0hsYBDQ2xnSzuhjlHTTQy5GIhAN9AAMML+dtV7ijWx/XFEg0xpyxuhDlPTTQy9edwEbgXWB84QUiUlVEXhCRAyKSJiLrRaSqY1lvEflZRE6JyCERmeC4/oK3pyIyQUTWF7psROQBEUkAEhzXvexYx2kRiRWRPoVu7y8ij4vIbyKS7ljeREReF5EXLqp3hYhMLepBFrcNERkCPA6MFZEMEdlexH0/AMKAFY7b/KPQ4ttE5KCIHBeR/yt0Hz8RedRR9wkRWSIidYp7EkRkmIhsc+zPn0WkY6FliSIyTUR2OJ6HxSISWGj5wyJyRESSReTu4rbhuG1DEVkuIidFZJ+ITHRc/2fgbaCn4zHOLOb+d4vIHsc7um9EpGlJ+9ixrMjnsdCqB4pIgmO9r4uIFLP9Yver/G8YbJJjXxwRkb8Xum8VEXnJsSzZ8XOVQstHOJ6D0471Dym06aYi8pOj9m9FJPRS+1kVYozRf+X0D9gH3A90AXKA+oWWvQ5EA40Af+BqoAr2cEsHbgEqASFAJ8d9ooF7Cq1jArC+0GUDfAfUAao6rrvdsY4A4O/AUSDQsexh4FegNSBAhOO23YBkwM9xu1Ags3D9Fz3OS21jBvBhCfspERhY6HK447G8BVR11JUNtHUsn4r9D2Vjxz77D/BxMeuOBFKA7o79PN6xvSqFtr0JaOjYb3uAex3LhgDHgA5ANeAjR10titnWWuANIBDoBNiAa4t6roq4742O10tbx358AvjZyX1c5PNY6DXxFVAb+2vLBgwppoZi92uh5+Rjx764yrGugY7lTznuWw+oC/wMzHIs6wakYR9y8sP+mm9T6DX9G9DK8VxHA89Z/bvrKf8sL8BX/mEfM80BQh2X9wJ/dfzsB5wFIoq432PAsmLWGU3JgT6ghLpSz28XiANGFHO7PcAgx88PAl9fxmMvvI0ZlD7QGxe6bhMwrlBt1xZa1sCxrwOKWPeb54Ol0HVxQL9C27690LI5wL8dPy8oHC6O0Cky0IEmQB5Qo9B1zwLvFvVcFXH/lcCfC132w/5HtGkZn0cD9C50eQnw6CWe8yL3a6HnpM1F++odx8+/AUMLLbsO+xAT2P8wvHiJ1/QThS7fD6y6nN81X/6nQy7lZzzwrTHmuOPyR/xv2CUU+1Hcb0Xcr0kx1zvrUOELIvJ3x9v4NBE5BdRybL+kbb2H/agQx/8fFLfBErZRFkcL/ZwJVHf83BRY5hhCOYU9iPKA+kWsoynw9/O3ddy+CfYj8pK205AL9+eBS9TaEDhpjEm/6PaNLnGfi+t8uVCNJ7EfbTeCMj2PUPzjK6qGkvbrxfvj/H5syIX7p/AyV9WnLqInysqB2MfCxwD+InL+xVoFqC0iEdjfHmcBzYGLx5UPYX+LWpQzQFChy1cUcZuC6TQd46yPANcCu4wx+SKSij0ozm+rObCziPV8COx01NsW+KKogpzYhjPTe17uFKCHgLuNMT85edvZxpjZl7kNgCPYw+i8sEvcNhmoIyI1CoV6GHDYyW2dr3PRxQvK+DxejmL3q9hP8IN9f+x1/ByG/XHj+L8psKuIZefrUy6mR+jl40bsRzbtsI+ldsIeij8Cdxpj8rG/nZ/nOJHmLyI9HSeRFmE/iTVGRAJEJEREOjnWuw0YKSJBYu8t/nMJddQAcrGPdQaIyD+BmoWWvw3MEpGWYtdRREIAjDFJwGbsR+afGWPOlnIbx4BwEbnUa+8YcGUJj6WwfwOzz580FJG6IjKimNu+BdwrIt0dj7GaiNwgIjWc2M4SYIKItBORIGB6cTc0xhzCPm78rIgEiv3E65+xP5/OPqbHRKS94zHVEpHRjmWlfh4vkzP79UnH6689cBew2HH9x8ATjvuEAv/EflAA8A5wl4hc6zjx2khE2pSiPnURDfTyMR5YaIw5aIw5ev4f8Br2zo0AYBr2I/XN2N9e/wv7SciDwFDsJ75OYg/xCMd6XwTOYQ/A9yg5LL7BPjYbj/0tcBYXvmWehz20vgVOY//Fq1po+XvYT34VO9zixDY+dfx/QkS2FLOOZ7GHwSkRmVbCYwJ4GVgOfCsi6dhPxnUv6obGmBhgIvZ9n4r9xOMEJ7aBMWYl8BLwg+N+P5Rwl1uwjzUnA8uA6caY75zc1jLsr4FPROQ09qPt6x2Ly/o8OsuZ/boW+774HphrjPnWcf3TQAywA/vreovjOowxm7CH/4vYT46uxX40r8pIHCcelCqRiPTFfpQV7nhXoXyUY8hlP1DJGJNrbTXqPD1CV04RkUrAFOBtDXOlKiYNdFUiEWkLnMLetvaSxeUopYqhQy5KKeUl9AhdKaW8RLn2oYeGhprw8PDy3KRSSnm82NjY48aYuiXdrlwDPTw8nJiYmPLcpFJKeTwRudSnkgvokItSSnkJDXSllPISGuhKKeUlNNCVUspLaKArpZSX0EBXSikvoYGulFJeQr/gQinllc6ey2NJzCFOZGRbXQoAN0U2plloNbduQwNdKeVVjDF8/etRZv93N8lpWYiUfJ/yENk0WANdKaWctefIaWYs38Uv+0/StkFNXhrXmW7N6lhdVrnRQFdKebzUM+eY9108i345QK2qlZh9UwfGdQ3D36+CHJ6XEw10pZTHyss3fLTpIC98G0d6Vi539gxn6sCW1A6qbHVpltBAV0p5pI2/n2DG8l3sPZpOzytDmD68HW2uqFnyHb2YBrpSyqMknzrLM1/v4asdR2hUuypv3hbJkA5XIBXl7KeFNNCVUh4hKyeP+et+543ofRgDUwe25C99m1O1sr/VpVUYGuhKqQrNGMM3u47y9H/3kJR6lhuuasBjQ9vQODjI6tIqHA10pVSFFX8snZkrdvHTvhO0rl+DjyZ25+rmoVaXVWFpoCulKpy0zBxeXB3PBxsPUL1KAE+NaM+t3cII8NfZSi5FA10pVWHk5RsWbz7E3G/jOJV5jlu7h/G3Qa2pU8032xAvlwa6UqpCiEk8yfTlu9iVfJpu4XWYPrwd7RvWsrosj6KBrpSy1NG0LJ5duYcvtyXToFYgr97SmWEdG2gbYilooCulLJGVk8c76/fz+pp95OYbJg9owX39mxNUWWOptHTPKaXKlTGG1XtSmPXVbg6ezOS69vV54oZ2NKmjbYhlpYGulCo3+1LSmbliNz8mHKdlvep8+Ofu9G6pbYiuooGulHK701k5vLw6gfd+TqRqZX/+Oawdd/RsSiVtQ3QppwJdRKYAEwEB3jLGvCQinYB/A4FALnC/MWaT2ypVSnmc/HzD0tgk5nyzlxNnzjGuaxOmDW5NSPUqVpfmlUoMdBHpgD3MuwHngFUi8l9gDjDTGLNSRIY6Lvd3Y61KKQ8SeyCVmSt2sSMpjciw2iyc0I2rGmsbojs5c4TeFthojMkEEJG1wE2AAc7PVVkLSHZLhUoVwxjDt7uP8dEvB8nJy7e6HFVIdm4+sQdSqVejCi+N7cSITg21DbEcOBPoO4HZIhICnAWGAjHAVOAbEZkL+AFXF3VnEZkETAIICwtzRc1KXXByrUmdqlxRM9DqklQhfgIPXtOCe/s3p3oVPVVXXsQYU/KNRP4MPABkALuxB7s/sNYY85mIjAEmGWMGXmo9UVFRJiYmpuxVK5+VdtZ+cu39DYkEVfbnr4NacXsPPbmmvJuIxBpjokq8nTOBftGKnwGSgGeB2sYYI/b3UmnGmEt+XYgGuiqt/HzDp7GHmLMqjpOZ5xjXNYxpg1vpyTXlE5wNdGe7XOoZY1JEJAwYCfQEJgP9gGhgAJBQ+nKVKl7sgZPMWL6bXw+nEdU0mPeGd6NDIz25ptTFnB3c+swxhp4DPGCMSRWRicDLIhIAZOEYJ1fKVY6dzuK5lXtZtvUwV9QM5OVxnRgeoSfXlCqOU4FujOlTxHXrgS4ur0j5vOzcPBasT+TVHxLIzTM8cE1z7u/fgmp6ck2pS9LfEFVhGGP4Ya99jo/EE5kMalefJ25oS9OQalaXppRH0EBXFcJvtgxmfbWb6DgbzetW4/27u9G3VV2ry1LKo2igK0ulZ+Xw6g/7WLB+P1Ur+fPEDW0Zf3W4tiEqVQoa6MoS+fmGz7Yk8a9VcZw4k82YLk2Ydl1r6tbQNkSlSksDXZW7bYdOMX35LrYfOkXnsNq8Mz6KiCa1rS5LKY+nga7KTUp6FnNWxbE0Nom6Naowb0wEN3ZqhJ+ftiEq5Qoa6MrtzuXm8+7P+3nl+31k5+Zxb7/mPDighc7xoZSL6W+Ucqs1cSnMWrGb34+fYUCbejw5rB3NQrUNUSl30EBXbpF4/AyzvtrN93tTaBZajYUTunJNm3pWl6WUV9NAVy6VkZ3La442xEr+wmPXt+GuXs2oHKBtiEq5mwa6cgljDF9sO8yzX+8lJT2bmyMb88iQ1tTTecqVKjca6KrMdiSdYsbyXWw5eIqIxrX4zx1d6BwWbHVZSvkcDXRVasczsnl+VRxLYg8RUq0yc0Z1ZFRkY21DVMoiGujqsuWcyyZ26Rxaxs9nlslkdqDgbwRZCay0ujqlKqhbPoYWl/xStzLTQFeX5dd1y6gZ/SQ98g/xa2AkjdteTXC1SlaXpVTFVzvc7ZvQQFdOOfz7HlKW/p3OmT+RJFewrfd/iBgwBvHT7hWlKgoNdHVJmRlpbP94OpFJHxKMHxuufJDIsf9H48Agq0tTSl1EA10VyeTnE/v124TFPEtPThJTaxBhY5+nZ6NmVpemlCqGBrr6g33bfyLnq4eJytnFPv/mnBz8H6K6D7a6LKVUCTTQVYFU2xHiP3mErseXkyY12HTVDLqMmIx/gL5MlPIE+puqyM05R+xnL9B276t0MWfZVH8MbW95hm7BoVaXppS6DBroPm7n+uVU++EJuucf4Ncqnal54wv0aNvF6rKUUqWgge6jjhyI48in04jMWEey1GPr1a/TaeCt2oaolAfTQPcxZ8+ks+2TmXQ++C618GND+L10HvskDYOqW12aUqqMNNB9hMnPZ8uq92i0aTY9sRFbcwCNxjxPzyYtrC5NKeUiGug+YP+uX8j8chpdzu3gN/9m7B74Kl16Xm91WUopF9NA92JpJ46x9+NHibIt47RU55d2TxA18q/ahqiUl9LfbC+Ul5tLzOfzaL37ZaLMGWLqjqTNLc/SPaS+1aUppdxIA93L7N6wkiqrH6N73n52VY4gaMTzdG/f3eqylFLlQAPdSxw9tI/DS6bRJX0NR6nLlu4v0fm68dqGqJQPceq3XUSmiMhOEdklIlMLXT9ZROIc189xX5mqOFmZGWxY+Ai13u5J+9Pr2RA2iVoPbyXy+rs0zJXyMSUeoYtIB2Ai0A04B6wSkf8CjYERQEdjTLaI1HNrpeoCJj+frd8t4oqNT9HTpLClRj8ajH6enk1bW12aUsoizgy5tAU2GmMyAURkLXATEAU8Z4zJBjDGpLityooiOx2OJ1hdBYeO2Tj1zbNEZm9lv19Tdg78kMhef7K6LKWUxZwJ9J3AbBEJAc4CQ4EYoBXQR0RmA1nANGPMZrdVarWMFHj7Wjh10OpKaALUpBob2zxG1M1/I6BSZatLUkpVACUGujFmj4j8C/gOyAC2A7mO+wYDPYCuwBIRudIYYwrfX0QmAZMAwsLCXFt9eTmXCR+Pgwwb3PQfCKxdrpvPM4b1Ccf5fGsSZ7Jz6duqHn+6YQQ9Qq8o1zqUUhWbU10uxph3gHcAROQZIAn7UMznjgDfJCL5QChgu+i+84H5AFFRUReEvUfIz4dlf4HDW2Dsh9B2WLluPibxJNOX72JXcj7dwlswfXg72jesVa41KKU8g1OBLiL1jDEpIhIGjAR6AvnAACBaRFoBlYHjbqvUKt/PhD3LYfDT5RrmR9OyeG7lHr7YlkyDWoG8ektnhnVsgIiUWw1KKc/ibB/6Z44x9BzgAWNMqogsABaIyE7s3S/jLx5u8Xix78FPL0GXu6Dng+WyyaycPN5Zv5/X1+wjN98weUAL7uvfnKDK+pEBpdSlOTvk0qeI684Bt7u8oori92j479+g+QAY+jy4+cjYGMPqPSk8/d/dHDiRyXXt6/PEDe1oUifIrdtVSnkPPewrSspeWHwnhLSE0e+CfyW3bm5fSgZPfbWbdfE2WtSrzod/7k7vlvr1b0qpy6OBfrEMG3w0GgKqwG1LINB9JyBPZ+XwyuoE3v05kaqV/fnnsHbc0bMplfz1E55KqcungV5Yzln45BZ7qE/4L9R2T5tlfr5haWwSc77Zy4kz5xgb1YRp17UmtHoVt2xPKeUbNNDPy8+HL+6DpBgY8x40ds8XJW85mMrM5bvYnpRGZFhtFk7oxlWNtQ1RKVV2GujnrXkadi2DQU9BuxEuX33K6SyeW7WXz7ccpl6NKrw0thMjOjXUNkSllMtooANs/RB+fAEix8PVD7l01dm5eSz8KZFXv08gJ89wX//mPHBNC6pX0V2vlHItTZX962DFFLjyGrjhBZe2J67Zm8JTX+1m//EzDGxbjyduaEd4aDWXrV8ppQrz7UC3xcPi2yGkhX3c3EXtib/bMpj11W7WxNm4sm413r2rK/1b6+zCSin38t1AP3Pc3p7oXxludU17YkZ2Lq/+kMCC9fupEuDP/w1ty/irw6kcoG2ISin3881Az8mCT26F9KP29sTgpmVaXX6+YdnWwzy3ai+29GxGd2nMw0NaU69GoIsKVkqpkvleoBsDXz4Ah36B0e9B46gyrW77oVNMX76LbYdO0alJbd66M4pOTcp3el2llAJfDPQ1z8DOpXDtdGh/Y6lXY0vP5vlv9rIkJonQ6lWYOzqCkZ0b4eenbYhKKWv4VqBv+xjWzYHOd0Dvv5ZqFedy83l/QyIvr04gKzePv/S9kgcHtKBGoHvne1FKqZL4TqAnroflk6FZXxj2YqnaE2MPpPKPpdv5zXaGfq3q8s8/taN53epuKFYppS6fbwT68X3wyW1QpxmMeb/U7YkPf7rdPl/5+CgGtKmnn/JUSlUo3t9Pd+aEvT3RL8Denlg1uFSrOXDiDL8fP8PEvldybdv6GuZKqQrHu4/Qc7Nh8W2QdhgmfGU/Qi+l6Dj7V6XqB4SUUhWV9wa6MfYx84MbYNQCaNKtTKuLjkuhaUgQzfSj+0qpCsp7h1zW/gt2LIYBT0CHm8u0qqycPDb8foL+req6qDillHI97wz0HUsg+lmIuBX6TCvz6n7Zf5KsnHwdblFKVWjeF+gHfrZ/EjS8D/zpZZfMnhgdl0LlAD96XBniggKVUso9vCvQT/xmb0+sHWZvTwyo7JLVro2z0ePKEKpW9nfJ+pRSyh28J9AzT8JHY+w/3/YpBNVxyWoPnsjk9+NndPxcKVXheUeXS+45WHwHnDoIdy6HOle6bNXR8SkA9G+tga6Uqtg8P9CNgRUPwYH1MPJtaNrTpauPjrMRVkfbFZVSFZ/nD7msmwvbP4b+j0PH0S5ddVZOHj//dpz+revqJ0OVUhWeZwf6r0thzdPQcRz0+4fLV7+poF1Rh1uUUhWf5wb6wV/gi/uhaS8Y/opLv9z5vOg4G5UD/Oh5ZajL162UUq7mmYF+8nf45Bao1RjGfggBVdyymej4FLo3q6Ptikopj+B5gX42FRaNAZPv0vbEix06mcnvtjP66VCllMfwrC6X8+2JqYlw55cQ0txtm4qO03ZFpZRnceoIXUSmiMhOEdklIlMvWjZNRIyIuHeg2Rj46q+Q+COMeB3Ce7l1c9FxNprUqcqV2q6olPIQJQa6iHQAJgLdgAhgmIi0dCxrAgwCDrqzSADWz4NtH0K/RyBirFs3ZW9XPEH/VvqtREopz+HMEXpbYKMxJtMYkwusBW5yLHsR+Adg3FSf3c7P4fun4KrR0P8xt24KYHPiSc7m5Olwi1LKozgT6DuBviISIiJBwFCgiYgMBw4bY7Zf6s4iMklEYkQkxmazla7Kk79DWE8Y/ppb2hMvFh1no7K/Hz2b6+yKSinPUeJJUWPMHhH5F/AdkAFsB3KB/wMGO3H/+cB8gKioqNIdyfedBr2mlPrLnS9XdFwK3a+sQ1BlzzpnrJTybU6dFDXGvGOMiTTG9AVOAolAM2C7iCQCjYEtInKFuwotrzA/dDKT32xn6KezKyqlPIyzXS71HP+HASOB940x9Ywx4caYcCAJiDTGHHVbpeUkOl6/DFop5ZmcHVP4TERCgBzgAWNMqhtrstTauBQaB1eleV1tV1RKeRanAt0Y06eE5eEuqcZi2bn2dsWRkY20XVEp5XE876P/brR5fyqZ5/Lo30qHW5RSnkcDvZDouBQq+/txdQttV1RKeR4N9EKi4210a6btikopz6SB7pCUmsm+lAz9dKhSymNpoDtEx51vV9RAV0p5Jg10h+g4G41qV6V53epWl6KUUqWigc75dkX9MmillGfTQAdiEh3tivrpUKWUB9NAp1C7os6uqJTyYBro2MfPuzYLploVbVdUSnkunw/0w6fOkpCSoZ8OVUp5PJ8P9LXarqiU8hI+H+jRcSk0ql2VFvW0XVEp5dl8OtDP5ebz077j9NN2RaWUF/DpQI85cJIz5/Lor99OpJTyAj4d6GvjbFTyF65uEWp1KUopVWY+HejRcTa6htehurYrKqW8gM8GevKps8QdS9fuFqWU1/DZQF+rXwatlPIyPhvo0XEpNKwVSEttV1RKeQmfDHR7u+IJ+rWup+2KSimv4ZOBHnsglYzsXB0/V0p5FZ8M9Oj4FCr5C720XVEp5UV8MtDXxtmIaqrtikop7+JzgX4k7Sx7j2q7olLK+/hcoP9vdkVtV1RKeRefC/ToOBsNagXSqr62KyqlvItPBXpOnn12Rf0yaKWUN/KpQI89kEp6di799NuJlFJeyKcCPTrORoCf0KuFfhm0Usr7OBXoIjJFRHaKyC4Rmeq47nkR2SsiO0RkmYjUdm+pZRcdl0JUeDA1AitZXYpSSrlciYEuIh2AiUA3IAIYJiItge+ADsaYjkA88Jg7Cy2ro2lZjnZFHW5RSnknZ47Q2wIbjTGZxphcYC1wkzHmW8dlgI1AY3cV6Qpr41MA/TJopZT3cibQdwJ9RSRERIKAoUCTi25zN7DS1cW5UnScjStqBtK6fg2rS1FKKbco8bPvxpg9IvIv7EMsGcB24PyROSLyf47Li4q6v4hMAiYBhIWFuaDky5eTl8/6hOPc0LGBtisqpbyWUydFjTHvGGMijTF9gZNAAoCIjAeGAbcZY0wx951vjIkyxkTVrWvNcMcWR7uiDrcopbyZU7NTiUg9Y0yKiIQBI4GeIjIEeAToZ4zJdGeRZRUdf75dUWdXVEp5L2enG/xMREKAHOABY0yqiLwGVAG+cwxjbDTG3OumOsskOs5Gl6barqiU8m5OBboxpk8R17VwfTmud+x0FnuOnOaRIW2sLkUppdzK6z8p+r/ZFXX8XCnl3bw+0KPjU7iiZiBtrtB2RaWUd/PqQM/Ny+fHhOP0a6WzKyqlvJ9XB/qWg6dIz9J2RaWUb/DqQI+OS7G3K7bUdkWllPfz8kC3Edk0mJrarqiU8gFeG+gpp7PYfeS0DrcopXyG1wZ6dLyjXVG/nUgp5SO8NtDXxtmoX7MKbRtou6JSyjd4ZaDb2xVt2q6olPIpXhnoWw+d4nRWrn47kVLKp3hloEfHpeCvsysqpXyMlwa6jS5hwdSqqu2KSinf4XWBnpKexa7k0/TTdkWllI/xukDX2RWVUr7K6wI9Ot5GvRpVaNegptWlKKVUufKqQM/Ny+fHeG1XVEr5Jq8K9G3arqiU8mFeFejRcTb8/YTeOruiUsoHeVegx6cQGVZb2xWVUj7JawI9JT2LnYdP63CLUspneU2gr4s/DkC/VtquqJTyTV4T6NFxKdStUYX2DbVdUSnlm7wi0PXLoJVSyksCfXvSKdLO5uinQ5VSPs0rAj06zoafQJ8WGuhKKd8VYHUBrhAdZyMyLJhaQdquqDxHTk4OSUlJZGVlWV2KqiACAwNp3LgxlSqVLss8PtBt6dn8ejiNaYNbWV2KUpclKSmJGjVqEB4erud+FMYYTpw4QVJSEs2aNSvVOjx+yGXd+S+D1v5z5WGysrIICQnRMFcAiAghISFlesfm8YEeHW8jtLrOrqg8k4a5KqysrwePDvS8fFPwZdB+fvqLoZTybU4FuohMEZGdIrJLRKY6rqsjIt+JSILj/2D3lvpH2w6d4lSmtisqVRqnTp3ijTfeKNV9hw4dyqlTpy55m3/+85+sXr26VOtXpVNioItIB2Ai0A2IAIaJSEvgUeB7Y0xL4HvH5XK1Ni7F3q6osysqddkuFeh5eXmXvO/XX39N7dq1L3mbp556ioEDB5a6Pivk5uZaXUKZONPl0hbYaIzJBBCRtcBNwAigv+M27wHRwCOuL7F40fE2OocFUzuocnluVimXm7liF7uTT7t0ne0a1mT6n9oXu/zRRx/lt99+o1OnTgwaNIgbbriBmTNn0qBBA7Zt28bu3bu58cYbOXToEFlZWUyZMoVJkyYBEB4eTkxMDBkZGVx//fX07t2bn3/+mUaNGvHll19StWpVJkyYwLBhwxg1ahTh4eGMHz+eFStWkJOTw6effkqbNm2w2WzceuutnDhxgq5du7Jq1SpiY2MJDb3wIO2+++5j8+bNnD17llGjRjFz5kwANm/ezJQpUzjJLfQjAAAOXUlEQVRz5gxVqlTh+++/JygoiEceeYRvvvkGEWHixIlMnjy5oObQ0FBiYmKYNm0a0dHRzJgxg+TkZBITEwkNDeWZZ57hjjvu4MyZMwC89tprXH311QDMmTOHDz74AD8/P66//nomTpzI6NGj2bJlCwAJCQmMGzeO2NhYlz6XznIm0HcCs0UkBDgLDAVigPrGmCMAxpgjIlJkm4mITAImAYSFhbmkaIDjGdnsSErj74O0XVGp0njuuefYuXMn27ZtAyA6OppNmzaxc+fOgra5BQsWUKdOHc6ePUvXrl25+eabCQkJuWA9CQkJfPzxx7z11luMGTOGzz77jNtvv/0P2wsNDWXLli288cYbzJ07l7fffpuZM2cyYMAAHnvsMVatWsX8+fOLrHX27NnUqVOHvLw8rr32Wnbs2EGbNm0YO3YsixcvpmvXrpw+fZqqVasyf/589u/fz9atWwkICODkyZMl7ovY2FjWr19P1apVyczM5LvvviMwMJCEhARuueUWYmJiWLlyJV988QW//PILQUFBnDx5kjp16lCrVi22bdtGp06dWLhwIRMmTLjMZ8J1Sgx0Y8weEfkX8B2QAWwHnH5fYoyZD8wHiIqKMqWs8w+0XVF5k0sdSZenbt26XdAD/corr7Bs2TIADh06REJCwh8CvVmzZnTq1AmALl26kJiYWOS6R44cWXCbzz//HID169cXrH/IkCEEBxd9Km7JkiXMnz+f3Nxcjhw5wu7duxERGjRoQNeuXQGoWdPe6bZ69WruvfdeAgLs8VanTp0SH/fw4cOpWrUqYP/A14MPPsi2bdvw9/cnPj6+YL133XUXQUFBF6z3nnvuYeHChcybN4/FixezadOmErfnLk59sMgY8w7wDoCIPAMkAcdEpIHj6LwBkOK+Mv8oOs5GaPXKOruiUi5UrVq1gp+jo6NZvXo1GzZsICgoiP79+xfZI12lSpWCn/39/Tl79myR6z5/O39//4KxamNKPsbbv38/c+fOZfPmzQQHBzNhwgSysrIwxhTZ5lfc9QEBAeTn5wP84XEUftwvvvgi9evXZ/v27eTn5xMYGHjJ9d58880F7zS6dOnyhz945cnZLpd6jv/DgJHAx8ByYLzjJuOBL91RYFHy8g3rEmz01XZFpUqtRo0apKenF7s8LS2N4OBggoKC2Lt3Lxs3bnR5Db1792bJkiUAfPvtt6Smpv7hNqdPn6ZatWrUqlWLY8eOsXLlSgDatGlDcnIymzdvBiA9PZ3c3FwGDx7Mv//974I/GueHXMLDwwvGtj/77LNia0pLS6NBgwb4+fnxwQcfFJwgHjx4MAsWLCAzM/OC9QYGBnLddddx3333cdddd5V5n5SFs33on4nIbmAF8IAxJhV4DhgkIgnAIMflcrE96Xy7og63KFVaISEh9OrViw4dOvDwww//YfmQIUPIzc2lY8eOPPnkk/To0cPlNUyfPp1vv/2WyMhIVq5cSYMGDahRo8YFt4mIiKBz5860b9+eu+++m169egFQuXJlFi9ezOTJk4mIiGDQoEFkZWVxzz33EBYWRseOHYmIiOCjjz4q2NaUKVPo06cP/v7+xdZ0//33895779GjRw/i4+MLjt6HDBnC8OHDiYqKolOnTsydO7fgPrfddhsiwuDBg129iy6LOPOWx1WioqJMTExMmdcz77t4XvshgS1PDtIOF+Wx9uzZQ9u2ba0uw1LZ2dn4+/sTEBDAhg0buO+++wpO0nqSuXPnkpaWxqxZs8q8rqJeFyISa4yJKum+Hjk519q4FDo1qa1hrpSHO3jwIGPGjCE/P5/KlSvz1ltvWV3SZbvpppv47bff+OGHH6wuxfMC/URGNjsOp/HXgdquqJSna9myJVu3brW6jDI536VTEXjcXC7rEmwYg37cXymlLuJxgX6+XbFDw1pWl6KUUhWKRwV6Xr5hXbyNvi21XVEppS7mUYG+I+kUqZk59NPhFqWU+gOPCvTzXwbdt6UGulJWqF69OgDJycmMGjWqyNv079+fktqTX3rppYIP6IBz0/GqknlWoMfbiGhSm+Bq2q6olJUaNmzI0qVLS33/iwPdmel4KxJjTME0AhWJx7QtnsjIZkfSKaZeq+2KygutfBSO/uradV5xFVxf/Ae4H3nkEZo2bcr9998PwIwZM6hRowZ/+ctfGDFiBKmpqeTk5PD0008zYsSIC+6bmJjIsGHD2LlzJ2fPnuWuu+5i9+7dtG3b9oK5XIqa9vaVV14hOTmZa665htDQUNasWXPB1Lbz5s1jwYIFgH3iq6lTp5KYmFjsNL2FrVixgqeffppz584REhLCokWLqF+/PhkZGUyePJmYmBhEhOnTp3PzzTezatUqHn/8cfLy8ggNDeX7779nxowZVK9enWnTpgHQoUMHvvrqKwCuv/56rrnmGjZs2MAXX3zBc8895/S0vkOHDuXVV18tmMisV69evPnmm3Ts2LEsz/IFPCbQf0w4ru2KSrnQuHHjmDp1akGgL1myhFWrVhEYGMiyZcuoWbMmx48fp0ePHgwfPrzY77t88803CQoKYseOHezYsYPIyMiCZUVNe/vQQw8xb9481qxZ84d5z2NjY1m4cCG//PILxhi6d+9Ov379CA4Odmqa3t69e7Nx40ZEhLfffps5c+bwwgsvMGvWLGrVqsWvv9r/aKampmKz2Zg4cSLr1q2jWbNmTk2zGxcXx8KFCwu+GORypvW95557ePfdd3nppZeIj48nOzvbpWEOHhTo0XEphFSrzFWNtF1ReaFLHEm7S+fOnUlJSSE5ORmbzUZwcDBhYWHk5OTw+OOPs27dOvz8/Dh8+DDHjh3jiiuuKHI969at46GHHgKgY8eOF4RUUdPeXirE1q9fz0033VQwf8rIkSP58ccfGT58uFPT9CYlJTF27FiOHDnCuXPnCqYCXr16NZ988knB7YKDg1mxYgV9+/YtuI0z0+w2bdr0gjltLmda39GjRzNr1iyef/55FixY4JZ50z0i0PPzDesSjuuXQSvlYqNGjWLp0qUcPXqUcePGAbBo0SJsNhuxsbFUqlSJ8PDwIqfNLayoo/fipr29lEvNLeXMNL2TJ0/mb3/7G8OHDy/4NqLz6724Rmem2YULp9otPM3u5U7rGxQUxKBBg/jyyy9ZsmRJiSeOS8MjToruOJzGyTPndLhFKRcbN24cn3zyCUuXLi3oWklLS6NevXpUqlSJNWvWcODAgUuuo2/fvixatAiAnTt3smPHDqD4aW+h+Kl7+/btyxdffEFmZiZnzpxh2bJl9OnTx+nHk5aWRqNGjQB47733Cq4fPHgwr732WsHl1NRUevbsydq1a9m/fz9w4TS7579SbsuWLQXLL3a50/qC/ZzAQw89RNeuXZ16R3C5PCLQo+NSEIE+2q6olEu1b9+e9PR0GjVqRIMGDQD7VLAxMTFERUWxaNEi2rRpc8l13HfffWRkZNCxY0fmzJlDt27dgOKnvQWYNGlSwQnGwiIjI5kwYQLdunWje/fu3HPPPXTu3NnpxzNjxgxGjx5Nnz59Lhiff+KJJ0hNTaVDhw5ERESwZs0a6taty/z58xk5ciQRERGMHTsWsH9hxcmTJ+nUqRNvvvkmrVoV3YhxudP6gn2oqGbNmm6bN90jps9dvPkgsQdSmTMqwg1VKWUNnT7X9yQnJ9O/f3/27t2Ln1/Rx9NlmT7XI47Qx3YN0zBXSnm0999/n+7duzN79uxiw7ysPOKkqFJKebo777yTO++8063b8IgjdKW8VXkOeaqKr6yvBw10pSwSGBjIiRMnNNQVYA/zEydOEBgYWOp16JCLUhZp3LgxSUlJ2Gw2q0tRFURgYCCNGzcu9f010JWySKVKlQo+paiUK+iQi1JKeQkNdKWU8hIa6Eop5SXK9ZOiImIDLj0xRPFCgeMuLMfT6f74H90XF9L9cSFv2B9NjTElzn1SroFeFiIS48xHX32F7o//0X1xId0fF/Kl/aFDLkop5SU00JVSykt4UqDPt7qACkb3x//ovriQ7o8L+cz+8JgxdKWUUpfmSUfoSimlLkEDXSmlvIRHBLqIDBGROBHZJyKPWl2PVUSkiYisEZE9IrJLRKZYXVNFICL+IrJVRL6yuhariUhtEVkqInsdr5OeVtdkFRH5q+P3ZKeIfCwipZ/G0ENU+EAXEX/gdeB6oB1wi4i0s7Yqy+QCfzfGtAV6AA/48L4obAqwx+oiKoiXgVXGmDZABD66X0SkEfAQEGWM6QD4A+Osrcr9KnygA92AfcaY340x54BPgBEW12QJY8wRY8wWx8/p2H9ZG1lblbVEpDFwA/C21bVYTURqAn2BdwCMMeeMMaesrcpSAUBVEQkAgoBki+txO08I9EbAoUKXk/DxEAMQkXCgM/CLtZVY7iXgH0C+1YVUAFcCNmChYwjqbRGpZnVRVjDGHAbmAgeBI0CaMeZba6tyP08IdCniOp/utRSR6sBnwFRjzGmr67GKiAwDUowxsVbXUkEEAJHAm8aYzsAZwCfPOYlIMPZ38s2AhkA1Ebnd2qrczxMCPQloUuhyY3zgrVNxRKQS9jBfZIz53Op6LNYLGC4iidiH4gaIyIfWlmSpJCDJGHP+XdtS7AHviwYC+40xNmNMDvA5cLXFNbmdJwT6ZqCliDQTkcrYT2wst7gmS4iIYB8f3WOMmWd1PVYzxjxmjGlsjAnH/rr4wRjj9UdhxTHGHAUOiUhrx1XXArstLMlKB4EeIhLk+L25Fh84QVzhv4LOGJMrIg8C32A/U73AGLPL4rKs0gu4A/hVRLY5rnvcGPO1hTWpimUysMhx8PM7cJfF9VjCGPOLiCwFtmDvDtuKD0wBoB/9V0opL+EJQy5KKaWcoIGulFJeQgNdKaW8hAa6Ukp5CQ10pZTyEhroSinlJTTQlVLKS/w/dEt9wmxnzUgAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot([t/600 for t in train_correct], label='training accuracy')\n",
    "plt.plot([t/100 for t in test_correct], label='validation accuracy')\n",
    "plt.title('Accuracy at the end of each epoch')\n",
    "plt.legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate Test Data\n",
    "We retained the test scores during our training session:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[tensor(9439), tensor(9635), tensor(9666), tensor(9726), tensor(9746), tensor(9758), tensor(9737), tensor(9749), tensor(9746), tensor(9725)]\n",
      "\n",
      "Test accuracy: 97.250%\n"
     ]
    }
   ],
   "source": [
    "print(test_correct) # contains the results of all 10 epochs\n",
    "print()\n",
    "print(f'Test accuracy: {test_correct[-1].item()*100/10000:.3f}%') # print the most recent result as a percent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, we'd like to compare the predicted values to the ground truth (the y_test labels), so we'll run the test set through the trained model all at once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract the data all at once, not in batches\n",
    "test_load_all = DataLoader(test_data, batch_size=10000, shuffle=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test accuracy: 9725/10000 =  97.250%\n"
     ]
    }
   ],
   "source": [
    "with torch.no_grad():\n",
    "    correct = 0\n",
    "    for X_test, y_test in test_load_all:\n",
    "        y_val = model(X_test.view(len(X_test), -1))  # pass in a flattened view of X_test\n",
    "        predicted = torch.max(y_val,1)[1]\n",
    "        correct += (predicted == y_test).sum()\n",
    "print(f'Test accuracy: {correct.item()}/{len(test_data)} = {correct.item()*100/(len(test_data)):7.3f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not bad considering that a random guess gives only 10% accuracy!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Display the confusion matrix\n",
    "This uses scikit-learn, and the predicted values obtained above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[   0    1    2    3    4    5    6    7    8    9]]\n",
      "\n",
      "[[ 968    0    1    0    1    2    5    0    4    1]\n",
      " [   0 1126    4    0    0    0    2    5    0    4]\n",
      " [   2    1 1007    4    3    0    0   10    5    0]\n",
      " [   1    0    5  985    0    2    1    3    4    7]\n",
      " [   1    0    1    0  962    1    3    1    3   12]\n",
      " [   2    1    1   13    0  882   33    1   23   16]\n",
      " [   1    2    2    0    5    1  913    0    1    0]\n",
      " [   1    1    5    5    3    2    0 1003    7    7]\n",
      " [   2    4    6    2    1    2    1    1  925    8]\n",
      " [   2    0    0    1    7    0    0    4    2  954]]\n"
     ]
    }
   ],
   "source": [
    "# print a row of values for reference\n",
    "np.set_printoptions(formatter=dict(int=lambda x: f'{x:4}'))\n",
    "print(np.arange(10).reshape(1,10))\n",
    "print()\n",
    "\n",
    "# print the confusion matrix\n",
    "print(confusion_matrix(predicted.view(-1), y_test.view(-1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows that the model had the greatest success with ones, twos and sevens, and the lowest with fives, sixes and eights."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Examine the misses\n",
    "We can track the index positions of \"missed\" predictions, and extract the corresponding image and label. We'll do this in batches to save screen space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "275"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "misses = np.array([])\n",
    "for i in range(len(predicted.view(-1))):\n",
    "    if predicted[i] != y_test[i]:\n",
    "        misses = np.append(misses,i).astype('int64')\n",
    "        \n",
    "# Display the number of misses\n",
    "len(misses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  61,   62,   81,  104,  115,  151,  193,  217,  247,  259],\n",
       "      dtype=int64)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Display the first 10 index positions\n",
    "misses[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set up an iterator to feed batched rows\n",
    "r = 12   # row size\n",
    "row = iter(np.array_split(misses,len(misses)//r+1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that everything is set up, run and re-run the cell below to view all of the missed predictions.<br>\n",
    "Use <kbd>Ctrl+Enter</kbd> to remain on the cell between runs. You'll see a <tt>StopIteration</tt> once all the misses have been seen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index: [  61   62   81  104  115  151  193  217  247  259  264  320]\n",
      "Label: [   8    9    6    9    4    9    9    6    4    6    9    9]\n",
      "Guess: [   2    5    5    5    9    8    8    5    6    0    4    8]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAABUCAYAAACm27J5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXucjXX+wN/fZoZmjbSuy0yFZVCsyxJllTQs05aaIrrwSrJIpaWSlGjdJvyKkFGUjYQxq7S5LFkUEioxhGRcJpdW5dLSzHx/fzzn+50zY87MGWfOec7web9ez+uc85xn5nzO53yf5/l+P1eltUYQBEEQBEG4MC5zWwBBEARBEITSjEymBEEQBEEQAkAmU4IgCIIgCAEgkylBEARBEIQAkMmUIAiCIAhCAMhkShAEQRAEIQACmkwppToqpXYppfYopYaUlFCCIAiCIAilBXWhdaaUUhHAN0B74CCwCeiutd5RcuIJgiAIgiCEN4FYpq4H9mitv9VanwPmAZ1LRixBEARBEITSQWQAfxsLHPB6fRBoWdgfKKWk3LogCIIgCKWF41rrKkUdFMhkShWw77zJklKqD9AngM8RBEEQBEFwg/3+HBTIZOogcJXX6zjgcP6DtNYpQAqIZUoQBEEQhIuPQGKmNgF1lVK1lFJlgG7A+yUjliAIgiAIQunggi1TWusspdQAYBkQAczUWm8vMckEQRAEQRBKARdcGuGCPkzcfIIgCIIglB42a62bF3VQIDFTgiAIghAUfvOb3wAwb948vv32WwAGDhzopkiC4BNpJyMIgiAIghAA4uYTSoTOnTvTvXt3AA4ePMgzzzwDQHZ2tptiCYJQSomPjwdg586d/PLLLwDExcVx4sQJN8WyNGrUCIBVq1ZRuXJlAFq0aMHnn3/uplhCySNuPiG4JCUlMXjwYMC5sJQrV86+99JLLwHw008/uSLbxU7fvn0BmDp1Kvfccw8AixYtclMkoZQRGelc/ps3b07Tpk0BaNasGfXr1wecSczUqVMB2Lp1qztCejh69CgA586dc1UOwxtvvEGPHj0AiIiI4JtvvgHg+++/d1MswUXEzScIgiAIghAAF41l6rrrrgNyV1sAX375pVviXLRERETQpUsXAObMmYNSBRXCF4JJ//79mTRpEgBaa06ePOmyRHm57777+OMf/wicHzB82WXO+u3TTz8FYMmSJaSkpADwww8/hFDKS5OoqCjAcUcNGjQIgLvuuqvAY1u3bk2zZs0A7O/pFh999BEAp0+fdlUOQ8eOHYmIiABg9+7ddOzYEXBCHMKdv//97wA88sgjAJw8eZJu3boBiIsyAEr1ZCo6OhqAhx9+mAkTJgB5J1Pbtm2joJiwTz/9lIULFwK5gydUN6SJEyfaG8yWLVv47rvvAIiNjeWTTz4Bck3qq1evJjMzE4CcnJyQyFcUjRs3Zu7cuQW+ZyavH330UVhc9K644gpGjhwJwOOPP24nft5jYsmSJTz22GMA7N/vV9cA17jxxhsBeOWVVzh79iwAPXr0YMWKFW6KBTgX6AEDBgDOeWluNPnPPzOOW7ZsaR8bNmwIwP333x8qcfntb38LOBO/IUOGAM45mJ9//vOfzJ492z4vrdSrVw9wxg7An//85wKPO378ONu2bbOvzbnhBv369QMc156R222M27NatWrWtdepUyd7HQ93WrdubeNZzcKmcuXKjBkzBoD27du7Jps/tGjRgj59nO508fHx7NmzB3BCHD777DMAjh075ops4uYTBEEQBEEIgFKZzWcsUmlpaQB06NChQAuUUqrI/Tt37gQgISHBWoGCQUJCAgApKSnWvH727Fk6d+4MQM2aNWnePG/CQPny5dmwYQPgrNoPHDgQNPmK4vLLLwecmi933HGH3f/jjz8C8Nxzz/HWW28B2Mwbt2jdujXg6NoE03qzfv16u1KvWLGidS/Vrl0bgFOnToVIUv+59tprraujevXqdgxNnjzZTbEYNWoUAIMGDcpjFTaJB4sXL2bJkiWAM97ff9/pOOV9Xm7evBmA2267jePHjwdd5ujoaP71r38BcNNNN9n9q1ev5quvvgKwVoc777zTWgQffPDBUmWdMr/HyJEjefTRRwHnmmIwv9G0adOYN28e4AR6h0MQ9dVXX80XX3wBOGOlUqVKLkvkYKweFStWpEmTJgB5LHnhikkO2rRpEzVr1gRg7NixAHTt2pUGDRoAjqU4HF191atXB2DDhg1cdZXTEjg7O9tawCH3OvK3v/0NgLVr15bUx/uVzSeWKUEQBEEQhAAodTFTLVu25LXXXgPyBkUaf+nSpUvtvpUrV1KrVi3AsZYYK8rdd99tYwbMjHzs2LH07NkzaHL36tULgMOHD1uLGmBXyAUxaNAgW/n3v//9b9Bk84dx48YBTj0pY1XIzs62q7OMjAzXZPOmTZs21hISExPDkSNHAGe1snfvXsCJSTM1YkaOHEliYiIAf/nLXwDsKj0cMKvIpUuX2tXZoEGDXLdIgWPJM/ELR48etbF0s2bNsins+/bts8cbKxZgx/WJEyfseVyrVq2QWKb69OljLVL79u3j448/BpzA/l9//TXPsdOnT2fOnDmAMy5MLTXvczhcGT16NIAtX+LNsmXLeOqppwD4+uuvQyqXPyQkJHDllVcC8Oyzz7osjUOvXr2sTPPnzyc9Pf28Y2rUqEG7du3s61WrVgHOdd9NjNegXr16/OMf/wCw8aT/+9//bMzU8OHDmTJlCpD3Xuo2JtYyJibGWlS7d+9uf48xY8bY64jxnJSgZco/tNYh2wAd6JacnKyzs7PzbOvXr9eVKlXSlSpV8vv/zJgxQ8+YMcP+j23btgUsW2Hb9u3b9fbt2/WsWbOC+jklvRm97tixQ+/YsUPn5OTo/fv36/379+vGjRu7Lp/ZYmJidExMjP7yyy/tb7phwwZdp04dXadOHZ9/FxcXp48cOaKPHDmiT58+rU+fPq1btGjh+veJjIzUkZGROjU1VaempuqsrCz98ssv65dffvm8YyMiInRERIT2uNFDtn399dc6KytLZ2Vl6YULFxZ5fFxcnP1tzPeqUaOG3rt3r967d6+eOXNmSOTevXu3lSM+Pr7I46Ojo3V0dLReuHCh3rVrl961a5euWrWq62OksHEzduxY+x1zcnL02bNn9dmzZ/WECRP0hAkTdHR0tOuyFrRVrVpVV61aVe/cuVNnZmbqzMxMXa1aNdflAvTAgQOtTgcOHGj3d+rUSa9evVqvXr1aZ2Rk5Lk3ZWRk6IyMDJ2enq7T09P12rVrdZcuXXSXLl10rVq1Qib77t277bhv3ry5bt68uX0vOjo6z1g5efKkPnnypK5QoYLrOs+/LV68WOfk5OicnBydkJBg98fFxelRo0bpUaNG6RMnTugTJ07ou+66q6Q+93N/5jfi5hMEQRAEQQiAUufmK4jExMRitRhITEyka9euQZQoL2XLlqVs2bJAeJrUC8O4zEwgd3Z2tnUfhFMdL5Pe3rBhQ2sSHj16tE2d9cXBgwdt2woT1B0TExNESf3DlM8wNYDmzZtn3TLeXHbZZbz77rsALF++nDfeeCPostWoUQNwgoSLw8GDB22NG/N7vfnmm/b9gkoTBJuWLVvaYHNfmISKF154wZahSEtLs4kO4YTR79NPP2337d+/37p0Zs2a5Ypc/tKpUyfASXs35WuOHDlik44iIyNdq6tmgvgB3nvvPRsWsGDBAsqUKVPg3+Qf0/Hx8TahIT093f6PcCqtYBpM16xZM6yu8eDUNrz99tsB59rx4osvAo5rz9REM8H25nuEilI3mfKOwTB06dLFFv4rDJOtNWPGjPNumCYTIBjExsbaG1CFChWC9jnBoE6dOgC2RtMbb7zB9OnT3RSpQLwLD27atAnAZo4VhfHBv/DCCyUv2AVwzTXX2Po+JlPI3AzzExcXZ9vJNGjQgHfeeQdw4iCChTl3TJ0awMYVFcXw4cMBbHaW929U1MS3pHjyySdtXNzYsWOtjk32mC927Nhhb6jTpk2zN3i3s1dN1t7o0aPtggByW690797dZgWHM+XKlePBBx+0r5OTkwHn+5nfq1q1ajbGMVRxpCaW1sQvgrPwMtecMmXK2BqB48eP59ChQz7/17333st9990HOOerqc1WUFxbqMjKyrLxpOZ6D+ETB+vNmTNnbMzuVVddlWcxZmrv9e7dG/D/mlRSiJtPEARBEAQhEEpbADqgp0yZoqdMmWID0Q4dOqTr16+v69evf96x8fHxOj4+Xk+fPt0e770tWbJEL1myRFevXj2ogXPz5s3T8+bN02fOnNFVqlTRVapU8XlscnKyTk5O1u3bt3c94O/YsWP62LFjVl+jR492XaaCtlOnTulTp07p7OxsPWzYMD1s2DDXZbrQberUqVbfTzzxhH7iiSfOOyYqKkpHRUXp2bNn22NXrFgRUjkzMjJsAHq9evWK9bf16tXT9erV08ePH7f/Y9myZSGT/amnntJPPfWUPn36tD58+LA+fPiwvuOOOwo8tmHDhrphw4b6gw8+0E2aNNFNmjTR33//vR4wYIAeMGCA6+OlV69eulevXudd29q0aaPbtGnjunz+bsOGDbOyr1y50u7v0KFDnu/VuHHjkCa/PP300/rpp58+L/HJbJmZmbp27dq6du3afv2/m2++Wd988806OzvbJgZ06dIlqN9h8ODBevDgwTo7O1v37NlT9+zZM8/7aWlpOi0tTefk5Oj169fr9evX68jISNfHREFbUlKSTkpK0iNHjswTOL9mzRq9Zs2aYHymXwHopc7NB7nuGNPpvFWrVjZu5MYbb7Qp5PHx8TYNtGLFitY8eODAARYsWADkuk+C7YffvXs34BS/NKmb3iZKb0yKp1LK1VYht956q009NT2nTGuNgjAtOu6++24eeughwEkJNmbyM2fOBFPcUo8xsT/00EN8+OGHALYHX35M4boHHnjA7vPXrVlSjBs3jldffRVwYhZefvllv//WuOu93ZGRkZHWZZWVlVWCkp6PkTU9Pd1eO1JTU607ybgjv/32W3sOJCYm2iK/KSkptoXV3LlzXStd0rp1aysHYEs79O/fn3Xr1rkiU3Ex7YRMmQ2AmTNnUrlyZSBvYdrMzMywKCoKuQU8u3XrZkt9+IO5Fxw9epSqVasCTuyeuScFA1Pwd8yYMUybNg2ApKQkwHFvt2rVyh5rihgH+xy8UBYtWmQfTexlZGRkofemUCBuPkEQBEEQhAAolZYpM3O+7bbbAFizZg1/+MMfAKd5sLFMXXHFFdYa9cMPP9gmla+++mqxsv9KAtO8GHJbs+SnS5cuQG7mXCgyswrDu1S/CdI1K/P83HbbbXbFYDrTG/79738DBDVw3VgXevXqZQuk/vzzz2zZsgWAdevW2WyPP/3pT7adTNu2bQv8f8aiuWvXrpAVaDRZKmXLlrUB/74wY8WbYK5sC2LDhg38/PPPgGNNM9YZXxZXyNW3abb6u9/9zr53880322KapthhsFmyZIldlT///PM2ONi0edq6dWue4n/GijJ06FCee+45IG8gfqgw42PAgAF5klpMK6SyZcvabKZffvklbBqlG6KioujYsSOAtZSYJB1wsiVNYeW6deva/dnZ2dZ6WbZsWRt0HGqOHz/O66+/DsB//vOfYv2tKeDZrVs3O8579+4d1CD07du3A45FtW/fvkBukWLzaAgXy19xyV9wN9SIZUoQBEEQBCEASmWjY4PxsQ8fPjzPCteQmZlJv379AKeJqVv1SSB39bpmzRqbYluvXj1Onz5tjzHlHUxqZ+vWrVm/fn1oBc2HiQsw7XqMNdBgUtw/++wz27onP6ZFiIkPCAZmdf7pp5/maW5s0sPnzJljYwSKU57i8OHDNobt6NGjJSVugVx77bWAE8NgVt/GKjZmzBjbgLRu3bq2lEdMTIy1YPbt2zfkFghTisG0WQEnvs6s2vNjapQZOWfPnm3j6xISEli2bBngxN2FuqmwUsq2lzLtN6pWrWrj0wBr6U5LS7PjKSkpicWLF4dUVlNLx59r2uTJk21T22A2c/cHc+6lpaX5tAr7y4EDB+y1MpixpaZml2m5As5YNW2ILpTY2FhbfuDkyZM2Ni/YdOjQAXDitACuvPJKW07m3Xfftdb8G264wV4/w5FKlSpZK1pERISNRTa1EUsQvxodF+nmU0pdBcwGfgfkACla61eVUhWB94CawHdAV6110HxnZgD07t3b1tUpismTJ/PBBx8ES6RiYW4e8+fP55VXXgGck9QUHbv88stp0aIFQJEunnDC3ES9J1Jnzpyxem/atCkVK1YMuhymVtSgQYOs+yW/SzI1NfW8v0tJSSlwAvLwww8D8Ne//pUrrrgCCP5kaseOHQCMGDHCJkaYWja33HKLndDeeOONttZTdna2Tchww5VjJkKTJ0+2E7/Y2FheeumlAo83F21zDhjXMDgLHnPTb9myZcgnU1pr+xtcf/31AFSpUoW4uDjAKYhpXFPeY6tBgwYhn0yNGDHC72Mfe+wxbr31VgD7aHpWhpIKFSowfvx4IK973Swox48fb93G3bt3p3nzwu9fv/76q3XdhzpRx1e4Q3EwAfahZvny5XkevZk7d67tt1quXLmwnkzdfvvtNhTl3LlzwZhEFQt/3HxZwCCtdQOgFfCoUupaYAiwUmtdF1jpeS0IgiAIgnBJUaRlSmudCWR6np9USqUDsUBnoK3nsLeB1cAzJSVYjRo1bKBc7969rRvPq2aVDWxevny5dXtMnjzZuktGjBjBzJkzgVx3ldtMmjSJu+++G3ACXs0KOCoqyrYUaNy4sVvi+aRatWqAU6LflDi47LLLrGkVcoNf7733XhuUuXbt2pBYpgxLly61K65ALDXeVpFHHnkEyA2aDjajRo2yFYnHjRsHOKUQjHXWm7Vr17oaMGqCPjdu3GjHbd++ffNYKk2Q8EsvvWTHiLFAeJOVlWUtLgsWLLCtUYYNGxa8L1AEx44ds9eOVatWWcvUO++8YwN327RpYy1qocJ7LBhXn3F5GW655RbA+T3MNbFHjx4AxSpjESgmIWX8+PHW4uuNKUMxceJEm5zj/Ztrrfnqq68AWLlypbVAbNmypcBxFO4YfQwdOtTuM61zwomkpKRCk0ncxju0JxzkLFY2n1KqJtAU2AhU80y00FpnKqVKJCBm7ty5ALRr144qVarY/eaCvGDBAmsqNrVqzp07Z11/pmUMOGX+f//73wPhM5kCuPPOOwHHlXTNNdcAzo3GzZuGL0yrDBM3NGzYMOuCysnJISEhwR47atQowKlpYuofNW3aNOTumpJwd3m7Wt3I1jL1jowbLCIiguuuuw5w4tOMa8S42cIBc475cvH5g3ErZGVl2dYi4XhenDlzxtZeM5mubmHiu/Jnc5raR2ZRCnlbooQKk42XfyJl4u1MrTJwFmKQty7g0qVLz4vVDCWmRtOLL75oe6zef//99gZe3MxwM7G955577OJi4sSJJSVuQGzdutW6+Vq0aBEWk5T8mDhd77pk4TAZ9XsypZSKAVKBgVrrn/2N61FK9QH6FHmgIAiCIAhCKcSvyZRSKgpnIjVHa20iRo8opap7rFLVgQKjc7XWKUCK5/8Umc1nOribFYrBVI1dsWKFDco1q6xGjRrZaujeHD58mPT09KI+MuSYlYyxsBnCrUM35FqbTC2YIUOG2Iyn/HWjTKBuv3798qw2TTPP0kJiYmIeE7IJ/HYD7wBQ7+bcJitr//79IZcpFOzdu5dOnToBzgo+HFaevnDb1eSdEezNs88+G2JJCsZkw3mzb98+nn/+ecBJojAYb4TW2tZ6c9v6aiyPixYtsgk348aNs27VyZMn22uhr6rhJju3Tp06vPfee3a/sSaaxAe32bVrl7VMmVp84YbJCve2spoEJDfxJ5tPAW8C6Vprb1vk+0BPYKznsUTSWUx2W5kyZWyH9vLly9OoUSMA3n777SL/hymKdtNNN4WFkkszxsxrYqM6duxoCxqaR4P5vbxZtmxZ0LPgSpq6devmcfN5F1x1E++ifqaEwMXKuHHjbMzPAw88EHaTqWrVqtGuXTsAPvnkE5elycXE44wdO9YuOiH3mpicnBxSeSpVqmR/R8hdHNx3330FLgRiY2MBJ6zDTDrCpeDohAkTrOsxKSnJui8nTZpkC7+aMjAGU5Sza9eugPO9TXHbJ598MuDyCiXN0qVLrSEjOjo6ZO2dLgSlFKEs7VQU/limWgMPAtuUUl949g3FmUTNV0o9DGQA55dkFgRBEARBuMjxJ5tvHeArQOrWkhUnl6FDhzJlyhTAKbtvWsSY1aA33jPU1NRUXnvtNSA8TH/FwTuYO1wwKxITADp69GjbXNIXx48ft4XfevfuHZarmsIwbSzAyZYzWZZu0qRJE9q3b++2GMUmKirKNnCGXLexOV8XLlxoWwF5M3DgQOu6nzVrVggkLR61a9e2mWduWAnNZzZs2NC2wGnVqpWVybtxLWBrr4XaLRwVFZWnfZa5jmzcuLHA443lbPbs2WFjETZs3brVJkUkJydb92XXrl3tb5Cfxx9/PM/r77//3masmiLN4cQHH3xgk71atGhhE4lMkkM4EU5WKQjz3nyHDh0CyNMV/WLG3LSNWyzcLibglHNYs2YN4GRemtT41NRUGws2ffp0W9m3tHPu3LmwmAyWK1fOunBOnTpls17DnUcffTRPbKBxn5oL4ahRo6xrB3Izvh544AEbS+MrJshNvOORDhw4EPLPN2n1bdu2tZm2V199dZ5jTMmMoUOHMnv27NAK6CWDKaviDyYW0O1K7UWxbds2O7EaM2aMHQ/5J1XmHmZiqhYtWhSWcbyGEydO2LjlRo0accMNNwBOr1Lj8nOrH2J+ws3NJ735BEEQBEEQAiCsLVOXGnv27AFy62g0bNjQ9mMLF7Kzs1m6dClASItxukVERIStM+VmIOyxY8dssdTNmze73rPRXzIyMmxRyfLly5/3vq9MyV9++YWpU6cC/iWdhJrGjRtbi5QbK3VjEU5JSbFFZZs3b26LF2/dutUWPb1YrMThyo4dO6yVyjyWZkzbrUaNGtmMxXPnztn2N959ON0knKxSIJYpQRAEQRCEgBDLVBhiVp3hGCtyKeDdnqVt27a2qr6xHLrBN998k6fOVGlh0aJFfPjhh4ATVG5ipkyNIVNR2mCqinfo0MHGboQjP/30k20abCxvbjBjxgxmzJjh2ucLFx8mvqtr1662Wnv//v1tmQe3MU26N2/ebOMFP/nkE9atWwfgWqKOTKbCCBOIawK5wzlQ8WLmySeftAXh9u7d60qA8cWEcYOZPoNAyHvZlQTbtm2zLayWL1/u6uRaEIKFSYAytR3DDVOnKzEx0dZPK1OmjM3idwtx8wmCIAiCIASACmUQlz/tZARBEARBEMKEzVrr5kUdJJYpQRAEQRCEAJDJlCAIgiAIQgDIZEoQBEEQBCEAZDIlCIIgCIIQADKZEgRBEARBCIBQ15k6Dpz2PAr+UxnRWXERnRUf0VnxEZ0VH9FZ8RGdFZ+S0tk1/hwU0tIIAEqpz/1JMxRyEZ0VH9FZ8RGdFR/RWfERnRUf0VnxCbXOxM0nCIIgCIIQADKZEgRBEARBCAA3JlMpLnxmaUd0VnxEZ8VHdFZ8RGfFR3RWfERnxSekOgt5zJQgCIIgCMLFhLj5BEEQBEEQAiBkkymlVEel1C6l1B6l1JBQfW5pQyn1nVJqm1LqC6XU5559FZVSK5RSuz2Pv3VbTjdRSs1USh1VSn3tta9AHSmHSZ5x95VSqpl7kruHD529qJQ65BlrXyilEr3ee9ajs11KqT+7I7W7KKWuUkp9rJRKV0ptV0o94dkvY80HhehMxpoPlFKXK6U+U0p96dHZCM/+WkqpjZ5x9p5Sqoxnf1nP6z2e92u6Kb8bFKKzt5RS+7zGWRPP/uCfm1rroG9ABLAXqA2UAb4Erg3FZ5e2DfgOqJxvXzIwxPN8CDDObTld1tFNQDPg66J0BCQCHwEKaAVsdFv+MNLZi8DgAo691nOOlgVqec7dCLe/gws6qw408zwvD3zj0Y2MteLrTMaab50pIMbzPArY6Bk/84Funv2vA/08z/sDr3uedwPec/s7hJHO3gLuKeD4oJ+bobJMXQ/s0Vp/q7U+B8wDOofosy8GOgNve56/Ddzpoiyuo7VeA/w3325fOuoMzNYOG4ArlVLVQyNp+OBDZ77oDMzTWp/VWu8D9uCcw5cUWutMrfUWz/OTQDoQi4w1nxSiM19c8mPNM15OeV5GeTYNtAMWevbnH2dm/C0EblVKqRCJGxYUojNfBP3cDNVkKhY44PX6IIWfYJcyGliulNqslOrj2VdNa50JzsUKqOqadOGLLx3J2CucAR6z90wv97HoLB8eV0pTnBWwjDU/yKczkLHmE6VUhFLqC+AosALHQvej1jrLc4i3XqzOPO//BFQKrcTuk19nWmszzkZ5xtn/KaXKevYFfZyFajJV0KxZ0ggLprXWuhnQCXhUKXWT2wKVcmTs+WYa8HugCZAJTPDsF515oZSKAVKBgVrrnws7tIB9l6TeCtCZjLVC0Fpna62bAHE4lrkGBR3meRSdcb7OlFINgWeB+kALoCLwjOfwoOssVJOpg8BVXq/jgMMh+uxShdb6sOfxKJCGc2IdMSZJz+NR9yQMW3zpSMaeD7TWRzwXpBxgBrnuFdGZB6VUFM6kYI7WepFnt4y1QihIZzLW/ENr/SOwGieu50qllOmf660XqzPP+xXw34V/0eGls44eN7PWWp8FZhHCcRaqydQmoK4nO6EMTtDc+yH67FKDUqqcUqq8eQ50AL7G0VVPz2E9gcXuSBjW+NLR+0APTzZHK+An46K51MkXM3AXzlgDR2fdPFlDtYC6wGehls9tPHEobwLpWuuJXm/JWPOBL53JWPONUqqKUupKz/NoIAEn1uxj4B7PYfnHmRl/9wCrtCfK+lLBh852ei1yFE6Mmfc4C+q5GVn0IYGjtc5SSg0AluFk9s3UWm8PxWeXMqoBaZ5YwkhgrtZ6qVJqEzBfKfUwkAF0cVFG11FKvQu0BSorpQ4Cw4GxFKyjf+FkcuwBzgAPhVzeL05yAAAA0ElEQVTgMMCHztp6Uoc1ThbpXwG01tuVUvOBHUAW8KjWOtsNuV2mNfAgsM0TmwEwFBlrheFLZ91lrPmkOvC2UioCx8AxX2u9RCm1A5inlPo7sBVnkorn8R9KqT04FqlubgjtMr50tkopVQXHrfcF0NdzfNDPTamALgiCIAiCEABSAV0QBEEQBCEAZDIlCIIgCIIQADKZEgRBEARBCACZTAmCIAiCIASATKYEQRAEQRACQCZTgiAIgiAIASCTKUEQBEEQhACQyZQgCIIgCEIA/D+zacwAovxk2QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 720x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "nextrow = next(row)\n",
    "print(\"Index:\", nextrow)\n",
    "print(\"Label:\", y_test.index_select(0,torch.tensor(nextrow)).numpy())\n",
    "print(\"Guess:\", predicted.index_select(0,torch.tensor(nextrow)).numpy())\n",
    "\n",
    "images = X_test.index_select(0,torch.tensor(nextrow))\n",
    "im = make_grid(images, nrow=r)\n",
    "plt.figure(figsize=(10,4))\n",
    "plt.imshow(np.transpose(im.numpy(), (1, 2, 0)));"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}