4.Pytorch Image Classification using NN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "### Image classification using Pytorch\r\n",
    "- Dataset: MNIST, Fashion MNIST\r\n",
    "- Architecture : Neural Network, Convolution Neural Network\r\n",
    "- Framework : Pytorch"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Digit MNIST\r\n",
    "### 1. Load Libraries"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "source": [
    "# torch loading...!!!\r\n",
    "import torch\r\n",
    "import torch.nn as nn\r\n",
    "import torch.nn.functional as F\r\n",
    "import torch.optim as optim\r\n",
    "\r\n",
    "# torchvision loading...!!!\r\n",
    "import torchvision\r\n",
    "import torchvision.transforms as transforms\r\n",
    "from torchvision import datasets\r\n",
    "from torchsummary import summary\r\n",
    "\r\n",
    "# other libraries\r\n",
    "import time\r\n",
    "import numpy as np\r\n",
    "import matplotlib.pyplot as plt"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "* `torchvision` is one of the most important modules of pytorch. you can download many dataset using `torchvision`.\r\n",
    "* `torchvision.transform` will help us in the transformation of the pixel value of the images. Mainly `Normalization` and `Standarization`.\r\n",
    "\r\n",
    "\r\n",
    "### 2. Define the transforms\r\n",
    "Steps :\r\n",
    "1. Convert dataset to tensor.\r\n",
    "2. Normalize the dataset. Normalize indicates the value 0.5 means and standard deviation for each channels for grayscale MNIST dataset."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "# tranforms used for the training, validation, and testing sets\r\n",
    "transform = transforms.Compose([transforms.ToTensor(),transforms.Normalize((0.5,), (0.5,))])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.Download Dataset\r\n",
    "* you can download the dataset with `torchvisions.datasets`\r\n",
    "* MNIST dataset has one argument transform arguments `transform=transform`, which is directly transform data into tensor and normalize.\r\n",
    "* Dataset Contains `60000 Training samples` and `15000 batches` dataloader.\r\n",
    "* `torch.utils.data.DataLoader` Data Loader to convert data into batches."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "# get the data\r\n",
    "trainset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)\r\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)\r\n",
    "testset = datasets.MNIST(root='./data', train=False, download=True, transform=transform)\r\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=64, shuffle=False)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ./data\\MNIST\\raw\\train-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "9913344it [00:00, 12039338.86it/s]                             \n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Extracting ./data\\MNIST\\raw\\train-images-idx3-ubyte.gz to ./data\\MNIST\\raw\n",
      "\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ./data\\MNIST\\raw\\train-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "29696it [00:00, 30283017.65it/s]         "
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Extracting ./data\\MNIST\\raw\\train-labels-idx1-ubyte.gz to ./data\\MNIST\\raw\n",
      "\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ./data\\MNIST\\raw\\t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "1649664it [00:00, 13109794.68it/s]                           "
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Extracting ./data\\MNIST\\raw\\t10k-images-idx3-ubyte.gz to ./data\\MNIST\\raw\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ./data\\MNIST\\raw\\t10k-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "5120it [00:00, ?it/s]                   "
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Extracting ./data\\MNIST\\raw\\t10k-labels-idx1-ubyte.gz to ./data\\MNIST\\raw\n",
      "\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "\n",
      "C:\\Users\\SHIVA\\miniconda3\\envs\\pytorch19\\lib\\site-packages\\torchvision\\datasets\\mnist.py:498: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ..\\torch\\csrc\\utils\\tensor_numpy.cpp:180.)\n",
      "  return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 4.Visualize the data"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "source": [
    "for batch_1 in trainloader:\r\n",
    "    batch = batch_1\r\n",
    "    break\r\n",
    "print(\"Image Shape:\", batch[0].shape) # as batch[0] contains the image pixels -> tensors\r\n",
    "print(\"Target Shape:\",batch[1].shape) # batch[1] contains the labels -> tensors\r\n",
    "plt.figure(figsize=(12, 8))\r\n",
    "for i in range (batch[0].shape[0]):\r\n",
    "    # index to print first 4 images per batch\r\n",
    "    if i == 4:\r\n",
    "        break\r\n",
    "    plt.subplot(1, 4, i+1)\r\n",
    "    plt.axis('off')\r\n",
    "    plt.imshow(batch[0][i].reshape(28, 28), cmap='gray')\r\n",
    "    plt.title(f\"class: {int(batch[1][i])}\")\r\n",
    "    # plt.savefig('digit_mnist.png')\r\n",
    "plt.show()"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Image Shape: torch.Size([64, 1, 28, 28])\n",
      "Target Shape: torch.Size([64])\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqwAAACvCAYAAAA4yYy3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUQklEQVR4nO3de7DN9b/H8febXRxFJXQokeugc9yaTFJyCUVKhU4X0lEkqj2TaqTr2UbJCKFmHDVkurj8dhciRqNz2m6/yBj8VFSUk0um3d4oZe/P+WNvv5+f92dt322tvddnrfV8zKzJvNb38ln67LXf62u9vx91zgkAAAAQqirJHgAAAABQFgpWAAAABI2CFQAAAEGjYAUAAEDQKFgBAAAQNApWAAAABC3jC1ZVvVdVP0/2OIDyYu4iVTF3kYqYt8mV8QVrCFT1DlX9m6oeUdVdqnpNsscEnI6qrlbV31X1cOnjq2SPCYjipDl74lGkqq8me1zA6WRyvZCV7AFkOlW9XkReEpHBIrJBROond0RAuYx2zv13sgcBlIdz7twTf1bVc0Vkn4gsTN6IgNPL9HohY66wqmpDVf2Lqh5U1UOqOiPGdtNU9QdVLVDVjSd/elHVK1X1i9Ln9qvqlNK8uqrOLz1uvqr+VVUviji050XkBefcOudcsXNur3Nub/yvGOki4LkLlClF5u5tInJARP73jF4k0k7A8zaj64WMKFhVtaqILBGR3SLSWEQuFpF3Y2z+VxFpJyK1ReRtEVmoqtVLn5smItOcc7VEpKmILCjNh4rIeSLSUEQuFJGRIvJb6bmfVNUlZYzrChGpq6o7VfVHVZ2hqv9y5q8W6STUuXuSiar6s6rmqep15Xx5SGMpMHdPGCoi8xzrlEPCnbfUCyLinEv7h4hcJSIHRSTL89y9IvJ5Gfv+IiJtS//8P1LyCafOKdvcJyJrROTfyzmuBiLiROQLKbm0X0dE8kRkQrL/zniE8Qh17pbu20lEaopINSl5Ey4UkabJ/jvjEcYj5Ll70jEaiUiRiFyW7L8vHmE8Qp231AsuM66wSsknmd3OueOn21BVHyv9QvOvqpovJZ+E6pQ+/Z8i0kJEdpRexu9Xmr8lIp+IyLuq+n+qOklVz4owrt9K//uqc+4n59zPIjJFRG6M/tKQ5kKdu+KcW++cK3TOHXPOzZWSN0/mLk4Idu6e5B4pKUC+K+d+SF+hztuMrxcypWD9QUQuVdUym8xKv3/yuIgMEpELnHPni8ivIqIiIs65b5xz/yEi9aTki8+LVPUc59yfzrnnnXOtRaSziPQTkSGnG5Rz7hcR+VFKPjX9PS7vi0NaC3LuxuBOnA+Q1Ji7Q0Rkbjn3QXoLct5SL2ROwbpBRH4SkRdV9ZzSLz1f7dmupogcl9J/DlDVZ0Sk1oknVfVuVa3rnCsWkfzSuFhVu6nqv5V+x6RARP4UkeKIY3tTRMaoaj1VvUBEsqXk+zOASKBzV1XPV9XepePJUtW7RORaEVkex2tFegly7p503M5S8v1E7g6Ak4U8bzO6XsiIgtU5VyQiN4lIMxHZIyWfUgZ7Nv1ESn7hfi0lX7j+XUo+bZ3QR0S2qephKflC9R3Oud9E5F9FZJGUTL6/ichnUnLZX1R1nKouK2N4/yUlX9z+unTfL0Vkwhm9UKSdgOfuWSKSIyVv1j+LyBgRucU59/UZv1iklYDn7glDReQvzrnCM3qBSEuBz9uMrhe09Mu8AAAAQJAy4gorAAAAUhcFKwAAAIJGwQoAAICgUbACAAAgaKe7zxgdWYibc67S783J3EUiMHeRqip77jJvkQhlzVuusAIAACBoFKwAAAAIGgUrAAAAgkbBCgAAgKBRsAIAACBoFKwAAAAIGgUrAAAAgkbBCgAAgKBRsAIAACBoFKwAAAAIGgUrAAAAgkbBCgAAgKBRsAIAACBoFKwAAAAIGgUrAAAAgpaV7AEAAACkk6pVq5rsoYceMtmUKVNMVq1aNe8xi4qK4h9YCuMKKwAAAIJGwQoAAICgUbACAAAgaBSsAAAACBpNV+XQq1cvk/Xr189kLVu2jHzMt99+22Rz584t38AAAEBSZGXZUmrJkiUm89UQ+fn5JmvXrp33PBs3biz32NIJV1gBAAAQNApWAAAABI2CFQAAAEGjYAUAAEDQMqrpSlVN1qBBA++2Tz/9tMmGDx9uMuecyd59912TFRYWes/Tvn17k9F0BQBAapg5c6bJunbtarL+/fub7OOPPzZZcXFxYgaWZrjCCgAAgKBRsAIAACBoFKwAAAAIGgUrAAAAgqa+pqG/P6ka+8nA1apVy2Q5OTkmGz16dORjbtmyxWS33367yXbu3Bn5mJnAOWe73SpYKs9dn5EjR5qse/fu3m07dOhgsiZNmpgsLy/PZN9++63Jli5d6j3P9u3bTbZ161bvtqmKuYtUVdlzNxPmbawVqNauXWuyYcOGmczXkI1/Vta85QorAAAAgkbBCgAAgKBRsAIAACBoFKwAAAAIWlo0XfkarBYsWGCyXr16mezXX3/1HnPUqFEmW7x4scn++OOPKEPMaDSulM8DDzxgstdff91kZf3sRuFb+a08x/z9999NVlBQENcxfXzj/OKLL0w2efJkk23atMl7zFgrz52KuVs+1atXN9msWbNMNnTo0ISf2zdP9uzZ4902Nzc30jE/+OADk61evbpc40oWmq4Sb82aNd78kksuMVmbNm1MFvV9J5PRdAUAAICURcEKAACAoFGwAgAAIGgUrAAAAAgaBSsAAACCFuxdArKyskzWuXNn77YLFy40We3atU02fPhwky1btsx7zAMHDpxuiIiITuvYfN2lK1euNFnLli1NFutn99ChQyY7fvx4pPHUrVvXZFWqRP9cG++dB+I5ZnFxscl69uzpPeZnn30W6dzM3fIZOHCgyd55553I+x85csRk69evj7Rvjx49TBbv3MvPzzfZ4MGDTbZq1aq4zlMRuEtA4m3bts2bz54922RTp06t4NGkJ+4SAAAAgJRFwQoAAICgUbACAAAgaBSsAAAACJrtbArEPffcY7I5c+Z4t/3hhx9Mdt9995nsrbfein9gwBlo0qSJN/ctHVqzZs1Ix/Q1G4qIjB071mS+nxGfrl27mmzAgAHebTt06GCyLl26RDpPvDZu3GiyCRMmmCxqcxUSo0WLFpG28y2tKyJy9913m2znzp2Rjnn55ZdH2i4W3++Xjh07muzee+81WYhNV6g877//frKHkBG4wgoAAICgUbACAAAgaBSsAAAACBoFKwAAAIIWRNOVr8nk0UcfNVms1Xo6depksn379sU9LiBRYjVS1apVK9L+vtWeHnnkEe+2+/fvjz6wU/ialGhcQlS+VdZ8fKu5iURvsPLZunXrGe8rIpKXl2cyX9MVMoOvgbB58+ZJGAlO4AorAAAAgkbBCgAAgKBRsAIAACBoFKwAAAAIWhBNV08//bTJWrVqZbLhw4d796fB6h+qVavmzWvUqBFp/4KCApMVFRXFNSaI3Hbbbd7cORdp/5kzZ5rs8OHDcY0JSLRFixaZ7PrrrzfZjz/+WBnDialevXomGzFiRKR9V6xYkejhIEAvvPCCybKygiiZMhZXWAEAABA0ClYAAAAEjYIVAAAAQaNgBQAAQNAoWAEAABC0IFrebr75ZpMtX77cZPPmzauM4QSpYcOGJvPdNaF3797e/a+88spI58nJyTHZM888E2lfVJwxY8YkewjAaf38888mi3WHjGSaP3++yXx3WNm2bZvJcnNzK2RMCItvGdb8/HzvtkePHq3g0cTWtGlTkw0cONBku3btMtnChQsrZEwVhSusAAAACBoFKwAAAIJGwQoAAICgUbACAAAgaFrW0pCqGm3dyDj5xjB58mSTjR07tjKGk3TNmjUz2apVq0zma8T65ptvvMdcv369ybp162ayiy++2GRVqsT3ucY5p3Ed4AxU1tyNqm3btt5806ZNkfavWrVqIoeDiJi7qS3Wz83mzZtN1qhRI5P5lpX1vZeGqLLnbrrNW997c6x6qWPHjgk999lnn+3NZ8yYYbL7778/0jF9y663adPGu20yl04ua95yhRUAAABBo2AFAABA0ChYAQAAEDQKVgAAAAQtiJWufFauXJnsIVSKwYMHm2zSpEkm+/LLL002YMAAk+3YscN7Ht9KHB999JHJfE1XiF9hYaE3P3z4sMlq1qxpspEjR5rs22+/9R5z7dq1kc8PpLNYq2y1atXKZO+8847JUqXBConne2+uUaNGpZz7vffe8+Z9+vQx2bhx40zma76ePn26yWrXru09TzKbrsrCFVYAAAAEjYIVAAAAQaNgBQAAQNAoWAEAABC0YJuu0s15553nzX0NVrt27TLZ7bffbrLjx4/HP7BT/PTTTwk/JmI3SM2ePdtk2dnZJps1a5bJYq26cujQIZN98sknJvOt5DJ//nyTHTx40HseICT169c32auvvhp5/yVLliRyOEhxvt/NU6dO9W57zjnnmOzIkSORztO/f3+Tde7c2but7/fAxIkTI52ne/fuJrvpppu8227ZsiXSMSsbV1gBAAAQNApWAAAABI2CFQAAAEGjYAUAAEDQgm26uuOOO0y2YsWKJIwkMapVq+bNGzZsaLLhw4ebLN4Gq4cffthk11xzjcmefPLJuM6D8nn++edNVlRUZDLfnDj//PO9x7zwwgtNduedd5rsrrvuMtl1111nsjFjxnjPs2fPHm8OJINvNTjfz4KIyLZt20yWm5ub8DEhdbVr185kF110kXfbWrVqmczXdNW7d2+TLVq0yGS+hlgRkZycHG8ehW+VrqiNYaHgCisAAACCRsEKAACAoFGwAgAAIGgUrAAAAAgaBSsAAACCprGWdxQRUdXYTyZQYWGhydatW2eyG264wbt/RSxRmmiq6s1vueUWk61Zs8Zk+/fvN1nbtm1NFquLsE+fPibzLQnnW8KzuLjYe8yonHP+F1+BKmvuVhZft3OHDh0i7+9blm/gwIEmu+yyy0w2efJk7zGfeOKJyOdPVczdMHXt2tVkq1evNtnRo0e9+7dv395kX3/9ddzjCkllz910m7cdO3Y02eeff+7ddvTo0SabM2eOyd577z2T+e4c0LhxY+958vPzvfmpfHeQ8d15oGfPnt79Yy0lXhnKmrdcYQUAAEDQKFgBAAAQNApWAAAABI2CFQAAAEELounK17wxceJEk7344ove/ceNG5fwMSXTFVdcYTJf09SDDz5osvr163uP2bdvX5MtW7bsDEZXfjSupA7fMoHdunXzbtuiRQuTHTp0KOFjSibmbphWrlxpMl9jYXZ2tnf/6dOnJ3xMoaHpKvF8S/qKiCxevNhkM2bMMNmuXbtMNmXKFJM9++yzZzC6fxgxYoTJhgwZYrKrr746rvNUBJquAAAAkLIoWAEAABA0ClYAAAAEjYIVAAAAQctK9gBERKZOnWoy34o7Y8eO9e6/Z88ek3333Xcm863iFK8ePXqYrHnz5ibr1KmTd39fg1XTpk1NVlRUZLKXXnrJZLm5ud7zbN++3ZsDJ/OtcDJgwADvtr65v2DBgoSPCZlt9uzZJuvSpYvJ9u7da7I333yzQsaEzLR8+XJvPmzYMJO98cYbJtu3b5/JduzYEdeYfCtyPfTQQybzNV2lGq6wAgAAIGgUrAAAAAgaBSsAAACCRsEKAACAoAXRdHXs2DGTjRo1ymS+LzGLiIwfP95krVu3NlnNmjVNdsEFF0QZooiI5Ofnm6ygoMBkVapE/xzg+xJ3r169THbw4EGTHT9+PPJ5kFxPPfWUN580aZLJ/vzzz4oeTkLceOONJqPpCvFo3LixyXwrWJ111lkmGzRokMkKCwsTMi5AJPb7m6+h6eWXXzZZXl6eyerVqxf5/L5572tK9K0eunnz5sjnCRVXWAEAABA0ClYAAAAEjYIVAAAAQaNgBQAAQNCCaLryKS4uNtmGDRu82/bv3z/SMZs1a2ayli1bRh7Tzp07TfbVV19F3h+ZoUGDBiZ77LHHvNsePXrUZK+88krCx+TjWy2oX79+kfdv0qRJIoeDDFK9enVvvmLFCpM1atTIZMuWLTPZunXr4h8YUIb169d78ylTpphswoQJJvv+++9Ndu2115qsTp063vOMGDHCZL5VrVauXOndP9VxhRUAAABBo2AFAABA0ChYAQAAEDQKVgAAAASNghUAAABBU+dc7CdVYz8JROSc08o+Z2hzNzs725vn5OSYzLek33PPPWcy31LDIiJXXXWVyXzLWz7++OPe/aPau3evyRo2bBjXMUPD3K0Y5557rjf3LX/t47vDBXcJ+GeVPXczYd7G4luOvW/fviabNm2ayQ4cOGCy3bt3e8/je8+OtW2qKmvecoUVAAAAQaNgBQAAQNAoWAEAABA0ClYAAAAEjaYrVDgaV0Tq1q3rzT/99FOTtW7d2mS+L+ZnZflXVq5du3akMana/y1lvR+cavr06SaL1VyWqpi78fPN03nz5nm3HTRokMmWLl1qsltvvdVkRUVFZzC69EXTFVIRTVcAAABIWRSsAAAACBoFKwAAAIJGwQoAAICg0XSFCkfjSmyXXnqpyXJzc03Wvn17k5WnQcrH13RVUFBgsrlz53r3Hz9+vMkKCwvjGlNomLvxGz16tMmmTp3q3fbYsWMm69atm8k2bNgQ97jSHU1XSEU0XQEAACBlUbACAAAgaBSsAAAACBoFKwAAAIJG0xUqHI0r8RsyZIjJunfv7t22X79+Jvvll19MtnjxYpO99tprJtu9e3eUIaYl5m751K9f32R5eXkm8zUbioh8+OGHJvOtaoXTo+kKqYimKwAAAKQsClYAAAAEjYIVAAAAQaNgBQAAQNAoWAEAABA07hKACkenNVIVczd+s2bNMplvqWERkezsbJOtW7cu4WPKBNwlAKmIuwQAAAAgZVGwAgAAIGgUrAAAAAgaBSsAAACCRtMVKhyNK0hVzF2kKpqukIpougIAAEDKomAFAABA0ChYAQAAEDQKVgAAAAStzKYrAAAAINm4wgoAAICgUbACAAAgaBSsAAAACBoFKwAAAIJGwQoAAICgUbACAAAgaP8PHF3aZmWSD64AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 864x576 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5. Define the Neural Network"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "source": [
    "class Net(nn.Module):\r\n",
    "    def __init__(self):\r\n",
    "        super(Net, self).__init__()\r\n",
    "        self.conv1 = nn.Conv2d(in_channels=1, out_channels=20, kernel_size=5, stride=1)\r\n",
    "        self.conv2 = nn.Conv2d(in_channels=20, out_channels=50, kernel_size=5, stride=1)\r\n",
    "        self.fc1 = nn.Linear(in_features=800, out_features=500)\r\n",
    "        self.fc2 = nn.Linear(in_features=500, out_features=10)\r\n",
    "        \r\n",
    "    def forward(self, x):\r\n",
    "        x = F.relu(self.conv1(x))\r\n",
    "        x = F.max_pool2d(x, kernel_size=2, stride=2)\r\n",
    "        x = F.relu(self.conv2(x))\r\n",
    "        x = F.max_pool2d(x, kernel_size=2, stride=2)\r\n",
    "        x = x.view(x.size(0), -1)\r\n",
    "        x = F.relu(self.fc1(x))\r\n",
    "        x = self.fc2(x)\r\n",
    "        return x"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 6.Check the GPU Availablity."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\r\n",
    "model = Net().to(device)\r\n",
    "print(model) \r\n",
    "print(str(summary(model, (1, 28, 28))))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Net(\n",
      "  (conv1): Conv2d(1, 20, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (conv2): Conv2d(20, 50, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (fc1): Linear(in_features=800, out_features=500, bias=True)\n",
      "  (fc2): Linear(in_features=500, out_features=10, bias=True)\n",
      ")\n",
      "==========================================================================================\n",
      "Layer (type:depth-idx)                   Output Shape              Param #\n",
      "==========================================================================================\n",
      "├─Conv2d: 1-1                            [-1, 20, 24, 24]          520\n",
      "├─Conv2d: 1-2                            [-1, 50, 8, 8]            25,050\n",
      "├─Linear: 1-3                            [-1, 500]                 400,500\n",
      "├─Linear: 1-4                            [-1, 10]                  5,010\n",
      "==========================================================================================\n",
      "Total params: 431,080\n",
      "Trainable params: 431,080\n",
      "Non-trainable params: 0\n",
      "Total mult-adds (M): 2.29\n",
      "==========================================================================================\n",
      "Input size (MB): 0.00\n",
      "Forward/backward pass size (MB): 0.12\n",
      "Params size (MB): 1.64\n",
      "Estimated Total Size (MB): 1.76\n",
      "==========================================================================================\n",
      "==========================================================================================\n",
      "Layer (type:depth-idx)                   Output Shape              Param #\n",
      "==========================================================================================\n",
      "├─Conv2d: 1-1                            [-1, 20, 24, 24]          520\n",
      "├─Conv2d: 1-2                            [-1, 50, 8, 8]            25,050\n",
      "├─Linear: 1-3                            [-1, 500]                 400,500\n",
      "├─Linear: 1-4                            [-1, 10]                  5,010\n",
      "==========================================================================================\n",
      "Total params: 431,080\n",
      "Trainable params: 431,080\n",
      "Non-trainable params: 0\n",
      "Total mult-adds (M): 2.29\n",
      "==========================================================================================\n",
      "Input size (MB): 0.00\n",
      "Forward/backward pass size (MB): 0.12\n",
      "Params size (MB): 1.64\n",
      "Estimated Total Size (MB): 1.76\n",
      "==========================================================================================\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 7.Optimizer and Loss Function"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "source": [
    "# loss function for the network to optimize\r\n",
    "criterion = nn.CrossEntropyLoss()\r\n",
    "\r\n",
    "# optimizer for the network\r\n",
    "optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 8.Training the Neural Network on Data"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "source": [
    "def train(net):\r\n",
    "    start = time.time()\r\n",
    "    epoch_list = []\r\n",
    "    loss_list = []\r\n",
    "    # 1. Define the training loop\r\n",
    "    for epoch in range(10): \r\n",
    "        running_loss = 0.0\r\n",
    "        # 2. Load the training data\r\n",
    "        for i, data in enumerate(trainloader, 0):\r\n",
    "            # 3. Split the data into inputs and labels\r\n",
    "            inputs, labels = data[0].to(device, non_blocking=True), data[1].to(device, non_blocking=True)\r\n",
    "            \r\n",
    "            # 4. set the parameters gradients to zero\r\n",
    "            optimizer.zero_grad()\r\n",
    "            \r\n",
    "            # 5. predict the output\r\n",
    "            outputs = net(inputs)\r\n",
    "            \r\n",
    "            # 6. Calculate the loss\r\n",
    "            loss = criterion(outputs, labels)\r\n",
    "            \r\n",
    "            # 7. backpropagate the loss\r\n",
    "            loss.backward()\r\n",
    "            \r\n",
    "            # 8. updatet the weigths\r\n",
    "            optimizer.step()\r\n",
    "            \r\n",
    "            # 9. print the mini-batch loss\r\n",
    "            running_loss += loss.item()\r\n",
    "        epoch_list.append(epoch)\r\n",
    "        loss_list.append(running_loss)\r\n",
    "        print(f\"[Epoch: {epoch+1}, Step: {i+1}] Loss: {running_loss:.3f}\")\r\n",
    "                \r\n",
    "    end = time.time()\r\n",
    "    print(\"Done Training...!!!\")\r\n",
    "    print(f\"Training time: {(end-start)/60:.2f} Minutes\")\r\n",
    "    return epoch_list, loss_list      "
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "source": [
    "epochs, losses = train(model)      "
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch: 1, Step: 938] Loss: 0.038\n",
      "[Epoch: 2, Step: 938] Loss: 0.037\n",
      "[Epoch: 3, Step: 938] Loss: 0.036\n",
      "[Epoch: 4, Step: 938] Loss: 0.034\n",
      "[Epoch: 5, Step: 938] Loss: 0.034\n",
      "[Epoch: 6, Step: 938] Loss: 0.032\n",
      "[Epoch: 7, Step: 938] Loss: 0.032\n",
      "[Epoch: 8, Step: 938] Loss: 0.031\n",
      "[Epoch: 9, Step: 938] Loss: 0.030\n",
      "[Epoch: 10, Step: 938] Loss: 0.030\n",
      "Done Training...!!!\n",
      "Training time: 1.99 Minutes\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 9.Testing Network on the Test Set"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "source": [
    "correct = 0\r\n",
    "total = 0\r\n",
    "with torch.no_grad():\r\n",
    "    for data in testloader:\r\n",
    "        inputs, labels = data[0].to(device, non_blocking=True), data[1].to(device, non_blocking=True)\r\n",
    "        outputs = model(inputs)\r\n",
    "        _, predicted = torch.max(outputs.data, 1)\r\n",
    "        total += labels.size(0)\r\n",
    "        correct += (predicted == labels).sum().item()\r\n",
    "        \r\n",
    "print(f\"Accuracy of the network on the {total} test images: {100 * correct / total:.2f}%\")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Accuracy of the network on the 10000 test images: 99.26%\n"
     ]
    }
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "orig_nbformat": 4,
  "language_info": {
   "name": "python",
   "version": "3.8.0",
   "mimetype": "text/x-python",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "file_extension": ".py"
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.8.0 64-bit ('pytorch19': conda)"
  },
  "interpreter": {
   "hash": "884a8f81666a19c0851426c83cd6eaa7b212468ad852fb3caa21591c98d4369f"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}