BE_cGAN_Enonce.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "BE cGAN - Enonce.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Neyri/Projet-cGAN/blob/master/BE_cGAN_Enonce.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "UGwKsKS4GMTN"
      },
      "cell_type": "markdown",
      "source": [
        "<h1 ><big><center>Deep Learning 2019/2020</center></big></h1>\n",
        "\n",
        "<h3><big><center>Liming Chen</center></big></h3>\n",
        "\n",
        "\n",
        "<h2><big><center> Assignment (3?): GANs </center></big></h2>\n",
        "\n",
        "<h5><big><center>Adapted from <i>Projet d'Option</i> of : Mhamed Jabri, Martin Chauvin, Ahmed Sahraoui, Zakariae Moustaïne and Taoufik Bouchikhi\n",
        "\n",
        "\n",
        "<p align=\"center\">\n",
        "<img height=300px src=\"https://cdn-images-1.medium.com/max/1080/0*tJRy5Chmk4XymxwN.png\"/></p>\n",
        "<p align=\"center\"></p>"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "16aVF81lJuiP"
      },
      "cell_type": "markdown",
      "source": [
        "The aim of this assignment is to discover GANs, understand how they are implemented and then explore one specific architecture of GANs that allows us to perform image to image translation (which corresponds to the picture that you can see above this text ! )\n",
        "\n",
        "Before starting the exploration of the world of GANs, here's  what students should do and send back for this assignement : \n",
        "* In the \"tutorial\" parts of this assignement that focus on explaining new concepts, you'll find <font color='red'>**questions**</font> (there are 6) that aim to test your understanding of those concepts. \n",
        "* In some of the code cells, you'll have to complete the code and you'll find a \"TO DO\" explaining what you should implement."
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "rtRoiHZqKI0p",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Importing all the libraries needed\n",
        "import matplotlib.pyplot as plt\n",
        "import imageio\n",
        "import glob\n",
        "import random\n",
        "import os\n",
        "import numpy as np\n",
        "import math\n",
        "import itertools\n",
        "import time\n",
        "import datetime\n",
        "import cv2\n",
        "from pathlib import Path\n",
        "from PIL import Image\n",
        "\n",
        "\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "import torchvision.transforms as transforms\n",
        "from torchvision.utils import save_image, make_grid\n",
        "from torchvision import datasets\n",
        "from torch.autograd import Variable\n",
        "\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "wPOeHZlaIHQa"
      },
      "cell_type": "markdown",
      "source": [
        "# Part 1 : Introduction to GANs.\n",
        "*** \n",
        "\n",
        "**Generative Adverserial Networks** belong to the set of algorithms named **generative models**. These algorithms belong to the field of unsupervised learning, meaning that they try to learn the **underlying structure** of the given data, without specifying a target value.               \n",
        "Generative models learn the intrinsic distribution function of the input data p(x) (or p(x,y) if there are multiple targets/classes in the dataset), allowing them to generate both synthetic inputs x’ and outputs/targets y’, typically given some hidden parameters.\n",
        "\n",
        "Generative Adversarial Networks are composed of two models:\n",
        "\n",
        "* The first model is called a **Generator** and it aims to generate new data similar to the expected one. You can think of the Generator as a manufacturer that creates fake clothes of luxuary brands.\n",
        "* The second model is named the **Discriminator**. This model’s goal is to recognize if an input data is ‘real’ — belongs to the original dataset — or if it is ‘fake’ . In this scenario, a Discriminator is analogous to the police (or a fashion expert maybe), which tries to detect clothes as truthful or fraud.\n",
        "\n",
        "How do these two components interact? **Through competition** ! Well, as we often say, adversity and competition makes better and that's what happens here : the Generator needs to learn how to create data in such a way that the Discriminator isn’t able to distinguish it as fake anymore. The Discriminator needs to learn how to detect fake data that comes from the generator. The competition between these two is what improves their knowledge, until the Generator succeeds in creating realistic data that fools the Discriminator.\n",
        "\n",
        "Here's a plot showing all of this ! \n",
        "\n",
        "![GANs](https://cdn-images-1.medium.com/max/1200/1*5rMmuXmAquGTT-odw-bOpw.jpeg)\n"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "gLoFU9UnDK8q"
      },
      "cell_type": "markdown",
      "source": [
        "## Mathematically, how do they work ?"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "rr-IlmeaFBNn"
      },
      "cell_type": "markdown",
      "source": [
        "A neural network G(z, θ₁) is used to model the Generator, Its role is mapping input noise variables *z* to the desired data space *x*. Conversely, a second neural network D(x, θ₂) models the discriminator and outputs the probability that the data came from the real dataset. In both cases, θᵢ represents parameters defining the network.\n",
        "\n",
        "As a result, the Discriminator is trained to correctly classify the input data as either real or fake. This means **its weights are updated as to maximize the probability that any real data input x is classified as belonging to the real dataset, while minimizing the probability that any fake image is classified as belonging to the real dataset**. In more technical terms, **the loss function used maximizes the function D(x), and it also minimizes D(G(z)).**\n",
        "\n",
        "Furthermore, the Generator is trained to fool the Discriminator by generating data as realistic as possible, which means that the Generator’s weights are optimized to **maximize the probability that any fake image is classified as belonging to the real dataset**. Formally this means that the **loss function used for this network maximizes D(G(z)).**\n",
        "\n",
        "Here's the resulting optimization problem : \n",
        "\n",
        "$$ \\underset{G}{\\min} \\underset{D}{\\max} L(D,G) = \\mathbb{E}_{x\\sim p_{data}(x)}[\\log D(x)] + \\mathbb{E}_{z\\sim p_z(z)}[\\log (1-D(G(z)))] $$\n",
        "\n",
        "After several steps of training, the Generator and Discriminator reach a point at which both cannot improve anymore : The generator generates realistic synthetic data and the discriminator is unable to differentiate between the two types of input."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "8sfBAbN-F9a8"
      },
      "cell_type": "markdown",
      "source": [
        " <font color='red'>**Question 1**</font>                                                                                     \n",
        "Explain in your own words why/how an optimization problem of the form $\\underset{G}{min}\\ \\underset{D}{max}\\ \\mathcal{L}(D,G)$ is adequate for the problem we're trying to solve."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "H_wBONLINZQL"
      },
      "cell_type": "markdown",
      "source": [
        "## Let's implement a basic GAN then ! "
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "-Jsy4IoNg06E"
      },
      "cell_type": "markdown",
      "source": [
        "### Discriminator\n",
        "\n",
        "The discriminator network is going to be a pretty typical linear classifier. It will be made of 3 hidden layers, will be activated with a leaky ReLu and use dropout layers."
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "DteCngtHE5IW",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Discriminator(nn.Module):\n",
        "\n",
        "    def __init__(self, input_size, hidden_dim, output_size):\n",
        "        super(Discriminator, self).__init__()\n",
        "        \n",
        "        # define hidden linear layers\n",
        "        self.fc1 = nn.Sequential(\n",
        "            nn.Linear(input_size, hidden_dim*4),\n",
        "            nn.LeakyReLU(negative_slope=0.2, inplace=True),\n",
        "            nn.Dropout(0.3, inplace=True)\n",
        "        )\n",
        "        self.fc2 = nn.Sequential(\n",
        "            nn.Linear(hidden_dim*4, hidden_dim*2),\n",
        "            nn.LeakyReLU(negative_slope=0.2, inplace=True),\n",
        "            nn.Dropout(0.3, inplace=True)\n",
        "        )\n",
        "        self.fc3 = nn.Sequential(\n",
        "            nn.Linear(hidden_dim*2, hidden_dim),\n",
        "            nn.LeakyReLU(negative_slope=0.2, inplace=True),\n",
        "            nn.Dropout(0.3, inplace=True)\n",
        "        )\n",
        "        \n",
        "        # final fully-connected layer\n",
        "        self.fc4 = nn.Linear(hidden_dim, output_size)\n",
        "        \n",
        "        \n",
        "        \n",
        "    def forward(self, x):\n",
        "        # flatten image\n",
        "        x = x.view(-1, 28*28)\n",
        "        # all hidden layers\n",
        "        x = self.fc1(x)\n",
        "        x = self.fc2(x)\n",
        "        x = self.fc3(x)\n",
        "        # final layer\n",
        "        out = self.fc4(x)\n",
        "\n",
        "        return out"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "Vg5jasGzhneR"
      },
      "cell_type": "markdown",
      "source": [
        "### Generator\n",
        "\n",
        "The generator network will be almost exactly the same as the discriminator network, except that hidden layers' dimension become higher as we get closer to the output layer and we're applying a tanh activation function to our output layer."
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "Qd6gFzBTDNNG",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Generator(nn.Module):\n",
        "\n",
        "    def __init__(self, input_size, hidden_dim, output_size):\n",
        "        super(Generator, self).__init__()\n",
        "        \n",
        "        # define hidden linear layers\n",
        "        self.fc1 = nn.Sequential(\n",
        "            nn.Linear(input_size, hidden_dim),\n",
        "            nn.LeakyReLU(negative_slope=0.2, inplace=True),\n",
        "            nn.Dropout(0.3, inplace=True)\n",
        "        )\n",
        "        self.fc2 = nn.Sequential(\n",
        "            nn.Linear(hidden_dim, hidden_dim*2),\n",
        "            nn.LeakyReLU(negative_slope=0.2, inplace=True),\n",
        "            nn.Dropout(0.3, inplace=True)\n",
        "        )\n",
        "        self.fc3 = nn.Sequential(\n",
        "            nn.Linear(hidden_dim*2, hidden_dim*4),\n",
        "            nn.LeakyReLU(negative_slope=0.2, inplace=True),\n",
        "            nn.Dropout(0.3, inplace=True)\n",
        "        )\n",
        "        \n",
        "        # final fully-connected layer\n",
        "        self.fc4 = nn.Sequential(\n",
        "            nn.Linear(hidden_dim*4, output_size),\n",
        "            nn.Tanh()\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        # all hidden layers\n",
        "        x = self.fc1(x)\n",
        "        x = self.fc2(x)\n",
        "        x = self.fc3(x)\n",
        "        # final layer\n",
        "        out = self.fc4(x)\n",
        "\n",
        "        return out"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "Z1fNAEiEiCg7"
      },
      "cell_type": "markdown",
      "source": [
        "### Build the network"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "z3cv4klXiBss",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Discriminator hyperparams\n",
        "\n",
        "# Size of input image to discriminator (28*28)\n",
        "input_size = 784\n",
        "# Size of discriminator output (real or fake)\n",
        "d_output_size = 1\n",
        "# Size of last hidden layer in the discriminator\n",
        "d_hidden_size = 32\n",
        "\n",
        "# Generator hyperparams\n",
        "\n",
        "# Size of latent vector to give to generator\n",
        "z_size = 100\n",
        "# Size of discriminator output (generated image)\n",
        "g_output_size = 784\n",
        "# Size of first hidden layer in the generator\n",
        "g_hidden_size = 32"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "MM8QUM5ZiKsE",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# instantiate discriminator and generator\n",
        "D = Discriminator(input_size, d_hidden_size, d_output_size)\n",
        "G = Generator(z_size, g_hidden_size, g_output_size)\n",
        "\n",
        "# check that they are as you expect\n",
        "print(D)\n",
        "print()\n",
        "print(G)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "roFIhYV1BUev"
      },
      "cell_type": "markdown",
      "source": [
        "<font color='red'>**Question 2**</font>                                                                  \n",
        "\n",
        "How many parameters are there to learn for each network ? (the Generator and Discriminator)"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "5fbSgsrE1GqC"
      },
      "cell_type": "markdown",
      "source": [
        "# Part2: cGAN"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "7SjXNoT7BUey"
      },
      "cell_type": "markdown",
      "source": [
        "Let's take the example of the set described in the next picture.\n",
        "![Map to satellite picture](https://raw.githubusercontent.com/Neyri/Projet-cGAN/master/BE/img/map_streetview.png)\n",
        "We have a picture of a map (from Google Maps) and we want to create an image of what the satellite view may look like.\n",
        "\n",
        "As we are not only trying to generate a random picture but a mapping between a picture to another one, we can't use the standard GAN architecture. We will then use a cGAN.\n",
        "\n",
        "A cGAN is a supervided GAN aiming at mapping a label picture to a real one or a real picture to a label one. As you can see in the diagram below, the discriminator will take as input a pair of images and try to predict if the pair was generated or not. The generator will not only generate an image from noise but will also use an image (label or real) to generate another one (real or label).\n",
        "![Diagram of how a cGan works](https://raw.githubusercontent.com/Neyri/Projet-cGAN/master/BE/img/cgan_map.png)\n"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "0JRaeHfzl6cO"
      },
      "cell_type": "markdown",
      "source": [
        "### Generator\n",
        "\n",
        "In the cGAN architecture, the generator chosen is a U-Net\n",
        "![U-Net](https://raw.githubusercontent.com/Neyri/Projet-cGAN/master/BE/img/unet.png)\n",
        "\n",
        "A U-Net takes as input an image, and outputs another image. \n",
        "\n",
        "It can be divided into 2 subparts : an encoder and a decoder. \n",
        "* The encoder takes the input image and reduces its dimension to encode the main features into a vector. \n",
        "* The decoder takes this vector and map the features stored into an image.\n",
        "\n",
        "A U-Net architecture is different from a classic encoder-decoder in that every layer of the decoder takes as input the previous decoded output as well as the output vector from the encoder layers of the same level. It allows the decoder to map low frequencies information encoded during the descent as well as high frequencies from the original picture. \n",
        "\n",
        "![U-Net](https://www.researchgate.net/profile/Baris_Turkbey/publication/315514772/figure/fig2/AS:485824962797569@1492841105670/U-net-architecture-Each-box-corresponds-to-a-multi-channel-features-maps-The-number-of.png)"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "xFqMOsoYwzFe"
      },
      "cell_type": "markdown",
      "source": [
        "The architecture we will implement is the following (the number in the square is the number of filter used).\n",
        "![UNet Architecture](https://raw.githubusercontent.com/Neyri/Projet-cGAN/master/BE/img/unet_architecture.png)\n",
        "The encoder will take as input a colored picture (3 channels: RGB), it will pass through a series of convolution layers to encode the features of the picture. It will then be decoded by the decoder using transposed convolutional layers. These layers will take as input the previous decoded vector AND the encoded features of the same level. "
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "Q_jf9H_NDESm"
      },
      "cell_type": "markdown",
      "source": [
        "Let's first create a few classes describing the layers we will use in the U-Net."
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "Zk5a6B5hILN2",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# code adapted from https://github.com/milesial/Pytorch-UNet/blob/master/unet/unet_parts.py\n",
        "\n",
        "# Input layer\n",
        "class inconv(nn.Module):\n",
        "    def __init__(self, in_ch, out_ch):\n",
        "        super(inconv, self).__init__()\n",
        "        self.conv = nn.Sequential(\n",
        "            nn.Conv2d(in_ch, out_ch, kernel_size=4, padding=1, stride=2),\n",
        "            nn.LeakyReLU(negative_slope=0.2, inplace=True)\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.conv(x)\n",
        "        return x\n",
        "\n",
        "# Encoder layer\n",
        "class down(nn.Module):\n",
        "    def __init__(self, in_ch, out_ch):\n",
        "        super(down, self).__init__()\n",
        "        self.conv = nn.Sequential(\n",
        "            nn.Conv2d(in_ch, out_ch, kernel_size=4, padding=1, stride=2),\n",
        "            nn.BatchNorm2d(out_ch),\n",
        "            nn.LeakyReLU(negative_slope=0.2, inplace=True)\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.conv(x)\n",
        "        return x\n",
        "\n",
        "# Decoder layer\n",
        "class up(nn.Module):\n",
        "    def __init__(self, in_ch, out_ch, dropout=False):\n",
        "        super(up, self).__init__()\n",
        "        if dropout :\n",
        "            self.conv = nn.Sequential(\n",
        "                nn.ConvTranspose2d(in_ch, out_ch, kernel_size=4, padding=1, stride=2),\n",
        "                nn.BatchNorm2d(out_ch),\n",
        "                nn.Dropout(0.5, inplace=True),\n",
        "                nn.ReLU(inplace=True)\n",
        "            )\n",
        "        else:\n",
        "            self.conv = nn.Sequential(\n",
        "                nn.ConvTranspose2d(in_ch, out_ch, kernel_size=4, padding=1, stride=2),\n",
        "                nn.BatchNorm2d(out_ch),\n",
        "                nn.ReLU(inplace=True)\n",
        "            )\n",
        "\n",
        "    def forward(self, x1, x2):\n",
        "        x1 = self.conv(x1)\n",
        "        x = torch.cat([x1, x2], dim=1)\n",
        "        return x\n",
        "\n",
        "# Output layer\n",
        "class outconv(nn.Module):\n",
        "    def __init__(self, in_ch, out_ch):\n",
        "        super(outconv, self).__init__()\n",
        "        self.conv = nn.Sequential(\n",
        "              nn.ConvTranspose2d(in_ch, out_ch, kernel_size=4, padding=1, stride=2),\n",
        "              nn.Tanh()\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.conv(x)\n",
        "        return x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "1rZ5Qz1mBUe8"
      },
      "cell_type": "markdown",
      "source": [
        "Now let's create the U-Net using the helper classes defined previously."
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "4Tbp_535EVPW",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class U_Net(nn.Module):\n",
        "    ''' \n",
        "    Ck denotes a Convolution-BatchNorm-ReLU layer with k filters.\n",
        "    CDk denotes a Convolution-BatchNorm-Dropout-ReLU layer with a dropout rate of 50%\n",
        "    Encoder:\n",
        "      C64   - C128   - C256   - C512  - C512  - C512 - C512 - C512\n",
        "    Decoder:\n",
        "      CD512 - CD1024 - CD1024 - C1024 - C1024 - C512 - C256 - C128\n",
        "    '''\n",
        "    def __init__(self, n_channels, n_classes):\n",
        "        super(U_Net, self).__init__()\n",
        "        # Encoder\n",
        "        self.inc = inconv(n_channels, 64) # 64 filters\n",
        "        # To do :\n",
        "        # Create the 7 encoder layers called \"down1\" to \"down7\" following this sequence\n",
        "        # C64   - C128   - C256   - C512  - C512  - C512 - C512 - C512\n",
        "        # The first one has already been implemented\n",
        "        \n",
        "        \n",
        "        # Decoder\n",
        "        # To do :\n",
        "        # Create the 7 decoder layers called up1 to up7 following this sequence :\n",
        "        # CD512 - CD1024 - CD1024 - C1024 - C1024 - C512 - C256 - C128\n",
        "        # The last layer has already been defined\n",
        "        \n",
        "        \n",
        "        self.outc = outconv(128, n_classes) # 128 filters\n",
        "\n",
        "    def forward(self, x):\n",
        "        x1 = self.inc(x)\n",
        "        x2 = self.down1(x1)\n",
        "        x3 = self.down2(x2)\n",
        "        x4 = self.down3(x3)\n",
        "        x5 = self.down4(x4)\n",
        "        x6 = self.down5(x5)\n",
        "        x7 = self.down6(x6)\n",
        "        x8 = self.down7(x7)\n",
        "        # At this stage x8 is our encoded vector, we will now decode it\n",
        "        x = self.up7(x8, x7)\n",
        "        x = self.up6(x, x6)\n",
        "        x = self.up5(x, x5)\n",
        "        x = self.up4(x, x4)\n",
        "        x = self.up3(x, x3)\n",
        "        x = self.up2(x, x2)\n",
        "        x = self.up1(x, x1)\n",
        "        x = self.outc(x)\n",
        "        return x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "1hmcejTWJSYY",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# We take images that have 3 channels (RGB) as input and output an image that also have 3 channels (RGB)\n",
        "generator=U_Net(3,3)\n",
        "# Check that the architecture is as expected\n",
        "generator"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "xIXFtHzcBUfO"
      },
      "cell_type": "markdown",
      "source": [
        "You should now have a working U-Net."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "RqD1katYBUfP"
      },
      "cell_type": "markdown",
      "source": [
        "<font color='red'>**Question 3**</font>                                                                  \n",
        "Knowing the input and output images will be 256x256, what will be the dimension of the encoded vector x8  ?\n",
        "\n",
        "<font color='red'>**Question 4**</font>                                                                  \n",
        "As you can see, U-net has an encoder-decoder architecture. Explain why it works better than a traditional encoder-decoder."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "cchTp3thBUfR"
      },
      "cell_type": "markdown",
      "source": [
        "### Discriminator\n",
        "\n",
        "In the cGAN architecture, the chosen discriminator is a Patch GAN. It is a convolutional discriminator which enables to produce a map of the input pictures where each pixel represents a patch of size NxN of the input.\n",
        "![patch GAN](https://raw.githubusercontent.com/Neyri/Projet-cGAN/master/BE/img/patchGAN.png)\n",
        "The size N is given by the depth of the net. According to this table :\n",
        "\n",
        "| Number of layers | N |\n",
        "| ---- | ---- |\n",
        "| 1 | 16 |\n",
        "| 2 | 34 |\n",
        "| 3 | 70 |\n",
        "| 4 | 142 |\n",
        "| 5 | 286 |\n",
        "| 6 | 574 |\n",
        "\n",
        "The number of layers actually means the number of layers with `kernel=(4,4)`, `padding=(1,1)` and `stride=(2,2)`. These layers are followed by 2 layers with `kernel=(4,4)`, `padding=(1,1)` and `stride=(1,1)`.\n",
        "In our case we are going to create a 70x70 PatchGAN."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "ge6I7M0aBUfT"
      },
      "cell_type": "markdown",
      "source": [
        "Let's first create a few helping classes."
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "RYqomFO8BUfV",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class conv_block(nn.Module):\n",
        "    def __init__(self, in_ch, out_ch, use_batchnorm=True, stride=2):\n",
        "        super(conv_block, self).__init__()\n",
        "        if use_batchnorm:\n",
        "            self.conv = nn.Sequential(\n",
        "                nn.Conv2d(in_ch, out_ch, kernel_size=4, padding=1, stride=stride),\n",
        "                nn.BatchNorm2d(out_ch),\n",
        "                nn.LeakyReLU(negative_slope=0.2, inplace=True)\n",
        "            )\n",
        "        else:\n",
        "            self.conv = nn.Sequential(\n",
        "                nn.Conv2d(in_ch, out_ch, kernel_size=4, padding=1, stride=stride),\n",
        "                nn.LeakyReLU(negative_slope=0.2, inplace=True)\n",
        "            )\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.conv(x)\n",
        "        return x\n",
        "    \n",
        "\n",
        "class out_block(nn.Module):\n",
        "    def __init__(self, in_ch, out_ch):\n",
        "        super(out_block, self).__init__()\n",
        "        self.conv = nn.Sequential(\n",
        "            nn.Conv2d(in_ch, 1, kernel_size=4, padding=1, stride=1),\n",
        "            nn.Sigmoid()\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.conv(x)\n",
        "        return x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "5m4Dnup4BUfc"
      },
      "cell_type": "markdown",
      "source": [
        "Now let's create the Patch GAN discriminator.\n",
        "As we want a 70x70 Patch GAN, the architecture will be as follows :\n",
        "```\n",
        "1. C64  - K4, P1, S2\n",
        "2. C128 - K4, P1, S2\n",
        "3. C256 - K4, P1, S2\n",
        "4. C512 - K4, P1, S1\n",
        "5. C1   - K4, P1, S1 (output)\n",
        "```\n",
        "Where Ck denotes a convolution block with k filters, Kk a kernel of size k, Pk is the padding size and Sk the stride applied.\n",
        "*Note :* For the first layer, we do not use batchnorm."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "AH6u5a-PBUfg"
      },
      "cell_type": "markdown",
      "source": [
        "<font color='red'>**Question 5**</font>                                                                  \n",
        "Knowing the input and output images will be 256x256, what will be the dimension of the encoded vector x8  ?Knowing input images will be 256x256 with 3 channels each, how many parameters are there to learn ?"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "g_9LxNhGBUfi",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class PatchGAN(nn.Module):\n",
        "    def __init__(self, n_channels, n_classes):\n",
        "        super(PatchGAN, self).__init__()\n",
        "        ## TODO :\n",
        "        # create the 4 first layers named conv1 to conv4\n",
        "        self.conv1 =\n",
        "        self.conv2 =\n",
        "        self.conv3 =\n",
        "        self.conv4 =\n",
        "        # output layer\n",
        "        self.out = out_block(512, n_classes)\n",
        "        \n",
        "    def forward(self, x1, x2):\n",
        "        x = torch.cat([x2, x1], dim=1)\n",
        "        x = self.conv1(x)\n",
        "        x = self.conv2(x)\n",
        "        x = self.conv3(x)\n",
        "        x = self.conv4(x)\n",
        "        x = self.out(x)\n",
        "        return x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "W_sevZRnBUfn",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# We have 6 input channels as we concatenate 2 images (with 3 channels each)\n",
        "discriminator = PatchGAN(6,1)\n",
        "discriminator"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "v_QubOycBUfv"
      },
      "cell_type": "markdown",
      "source": [
        "You should now have a working discriminator."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "DiI2CByRBUfz"
      },
      "cell_type": "markdown",
      "source": [
        "### Loss functions\n",
        "\n",
        "As we have seen in the choice of the various architectures for this GAN, the issue is to map both low and high frequencies.\n",
        "To tackle this problem, this GAN rely on the architecture to map the high frequencies (U-Net + PatchGAN) and the loss function to learn low frequencies features. The global loss function will indeed be made of 2 parts :\n",
        "* the first part to map hight frequencies, will try to optimize the mean squared error of the GAN.\n",
        "* the second part to map low frequencies, will minimize the $\\mathcal{L}_1$ norm of the generated picture.\n",
        "\n",
        "So the loss can be defined as $$ G^* = arg\\ \\underset{G}{min}\\ \\underset{D}{max}\\ \\mathcal{L}_{cGAN}(G,D) + \\lambda \\mathcal{L}_1(G)$$"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "k4G_xewPBUf4",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Loss functions\n",
        "criterion_GAN = torch.nn.MSELoss()\n",
        "criterion_pixelwise = torch.nn.L1Loss()\n",
        "\n",
        "# Loss weight of L1 pixel-wise loss between translated image and real image\n",
        "lambda_pixel = 100"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "c12q2NwkBUf7"
      },
      "cell_type": "markdown",
      "source": [
        "### Weights loading and training\n",
        "\n",
        "We will now load some pre-trained weights and finish the training of the cGAN. For this purpose we made 2 scripts that do it for you "
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "vGKjO0UMBUf9",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# parameters\n",
        "epoch = 0 #  epoch to start training from\n",
        "n_epoch = 200  #  number of epochs of training\n",
        "batch_size =10  #  size of the batches\n",
        "lr = 0.0002 #  adam: learning rate\n",
        "b1 =0.5  #  adam: decay of first order momentum of gradient\n",
        "b2 = 0.999  # adam: decay of first order momentum of gradient\n",
        "decay_epoch = 100  # epoch from which to start lr decay\n",
        "img_height = 256  # size of image height\n",
        "img_width = 256  # size of image width\n",
        "channels = 3  # number of image channels\n",
        "sample_interval = 500 # interval between sampling of images from generators\n",
        "checkpoint_interval = -1 # interval between model checkpoints\n",
        "cuda = True if torch.cuda.is_available() else False # do you have cuda ?"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "6DHT9c0_BUgA"
      },
      "cell_type": "markdown",
      "source": [
        "Configure the dataloader"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "rxi_QIpgBUgB",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class ImageDataset(Dataset):\n",
        "    def __init__(self, root, transforms_=None, mode='train'):\n",
        "        self.transform = transforms.Compose(transforms_)\n",
        "\n",
        "        self.files = sorted(glob.glob(os.path.join(root, mode) + '/*.jpg'))\n",
        "        if mode == 'val':\n",
        "            self.files.extend(\n",
        "                sorted(glob.glob(os.path.join(root, 'val') + '/*.jpg')))\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "\n",
        "        img = Image.open(self.files[index % len(self.files)])\n",
        "        w, h = img.size\n",
        "        img_A = img.crop((0, 0, w / 2, h))\n",
        "        img_B = img.crop((w / 2, 0, w, h))\n",
        "\n",
        "        if np.random.random() < 0.5:\n",
        "            img_A = Image.fromarray(np.array(img_A)[:, ::-1, :], 'RGB')\n",
        "            img_B = Image.fromarray(np.array(img_B)[:, ::-1, :], 'RGB')\n",
        "\n",
        "        img_A = self.transform(img_A)\n",
        "        img_B = self.transform(img_B)\n",
        "\n",
        "        return img_A, img_B\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.files)\n",
        "    \n",
        "# Configure dataloaders\n",
        "transforms_ = [transforms.Resize((img_height, img_width), Image.BICUBIC),\n",
        "               transforms.ToTensor()]  # transforms.Normalize((0.5,0.5,0.5), (0.5,0.5,0.5))\n",
        "\n",
        "dataloader = DataLoader(ImageDataset(\"facades\", transforms_=transforms_),\n",
        "                        batch_size=16, shuffle=True)\n",
        "\n",
        "val_dataloader = DataLoader(ImageDataset(\"facades\", transforms_=transforms_, mode='val'),\n",
        "                            batch_size=8, shuffle=False)\n",
        "\n",
        "# Tensor type\n",
        "Tensor = torch.cuda.FloatTensor if cuda else torch.FloatTensor"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "Okb3LU76BUgG"
      },
      "cell_type": "markdown",
      "source": [
        "Check the loading works and a few helper functions"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "xuxq4TZRBUgJ",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def plot2x2Array(image, mask):\n",
        "    f, axarr = plt.subplots(1, 2)\n",
        "    axarr[0].imshow(image)\n",
        "    axarr[1].imshow(mask)\n",
        "\n",
        "    axarr[0].set_title('Image')\n",
        "    axarr[1].set_title('Mask')\n",
        "\n",
        "\n",
        "def reverse_transform(image):\n",
        "    image = image.numpy().transpose((1, 2, 0))\n",
        "    image = np.clip(image, 0, 1)\n",
        "    image = (image * 255).astype(np.uint8)\n",
        "\n",
        "    return image\n",
        "\n",
        "def plot2x3Array(image, mask,predict):\n",
        "    f, axarr = plt.subplots(1,3,figsize=(15,15))\n",
        "    axarr[0].imshow(image)\n",
        "    axarr[1].imshow(mask)\n",
        "    axarr[2].imshow(predict)\n",
        "    axarr[0].set_title('input')\n",
        "    axarr[1].set_title('real')\n",
        "    axarr[2].set_title('fake')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "m2NxLrQEBUgM",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "image, mask = next(iter(dataloader))\n",
        "image = reverse_transform(image[0])\n",
        "mask = reverse_transform(mask[0])\n",
        "plot2x2Array(image, mask)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "zAvaxAbxBUgQ"
      },
      "cell_type": "markdown",
      "source": [
        "Initialize our GAN"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "dVgF3qfDBUgR",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Calculate output of image discriminator (PatchGAN)\n",
        "patch = (1, img_height//2**3-2, img_width//2**3-2)\n",
        "\n",
        "if cuda:\n",
        "    generator = generator.cuda()\n",
        "    discriminator = discriminator.cuda()\n",
        "    criterion_GAN.cuda()\n",
        "    criterion_pixelwise.cuda()\n",
        "    \n",
        "# Optimizers\n",
        "optimizer_G = torch.optim.Adam(generator.parameters(), lr=lr, betas=(b1, b2))\n",
        "optimizer_D = torch.optim.Adam(discriminator.parameters(), lr=lr, betas=(b1, b2))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "S58kJj9HBUgV"
      },
      "cell_type": "markdown",
      "source": [
        "Loading the pretrained weights"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "Ld26GCo-BUgX",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "load_weights = True\n",
        "\n",
        "def weights_init_normal(m):\n",
        "    classname = m.__class__.__name__\n",
        "    if classname.find('Conv') != -1:\n",
        "        torch.nn.init.normal_(m.weight.data, 0.0, 0.02)\n",
        "    elif classname.find('BatchNorm2d') != -1:\n",
        "        torch.nn.init.normal_(m.weight.data, 1.0, 0.02)\n",
        "        torch.nn.init.constant_(m.bias.data, 0.0)\n",
        "\n",
        "if load_weights and 'generator.pth' in os.listdir() and 'discriminator.pth' in os.listdir():\n",
        "    # load weights \n",
        "    checkpoint_generator = torch.load('generator.pth')\n",
        "    generator.load_state_dict(checkpoint_generator['model_state_dict'])\n",
        "    optimizer_G.load_state_dict(checkpoint_generator['optimizer_state_dict'])\n",
        "    epoch_G = checkpoint_generator['epoch']\n",
        "    loss_G = checkpoint_generator['loss']\n",
        "    \n",
        "    checkpoint_discriminator = torch.load('discriminator.pth')\n",
        "    discriminator.load_state_dict(checkpoint_discriminator['model_state_dict'])\n",
        "    optimizer_D.load_state_dict(checkpoint_discriminator['optimizer_state_dict'])\n",
        "    epoch_D = checkpoint_discriminator['epoch']\n",
        "    loss_D = checkpoint_discriminator['loss']\n",
        "else :\n",
        "    # Initialize weights\n",
        "    generator.apply(weights_init_normal)\n",
        "    discriminator.apply(weights_init_normal)\n",
        "    epoch_D = 0\n",
        "    epoch_G = 0\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "rN3cbiWaBUgf"
      },
      "cell_type": "markdown",
      "source": [
        "Start training"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "msmQQUX-BUgh",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def save_model(epoch):\n",
        "    # save your work\n",
        "    torch.save({\n",
        "                'epoch': epoch,\n",
        "                'model_state_dict': generator.state_dict(),\n",
        "                'optimizer_state_dict': optimizer_G.state_dict(),\n",
        "                'loss': loss_G,\n",
        "                }, 'generator_'+str(epoch)+'.pth')\n",
        "    torch.save({\n",
        "                'epoch': epoch,\n",
        "                'model_state_dict': discriminator.state_dict(),\n",
        "                'optimizer_state_dict': optimizer_D.state_dict(),\n",
        "                'loss': loss_D,\n",
        "                }, 'discriminator_'+str(epoch)+'.pth')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "6UXrZLLNBUgq"
      },
      "cell_type": "markdown",
      "source": [
        "**You don't need to train the network, we already did it for you** but if you want to have a look, here is the code :"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "4D0EAaJDBUgs"
      },
      "cell_type": "markdown",
      "source": [
        "```python\n",
        "# ----------\n",
        "#  Training\n",
        "# ----------\n",
        "losses = []\n",
        "num_epochs = 200\n",
        "\n",
        "# train the network\n",
        "discriminator.train()\n",
        "generator.train()\n",
        "print_every = 400\n",
        "\n",
        "for epoch in range(epoch_G, num_epochs):\n",
        "    for i, batch in enumerate(dataloader):\n",
        "\n",
        "        # Model inputs\n",
        "        real_A = Variable(batch[0].type(Tensor))\n",
        "        real_B = Variable(batch[1].type(Tensor))\n",
        "\n",
        "        # Adversarial ground truths\n",
        "        valid = Variable(Tensor(np.ones((real_B.size(0), *patch))), requires_grad=False)\n",
        "        fake = Variable(Tensor(np.zeros((real_B.size(0), *patch))), requires_grad=False)\n",
        "\n",
        "        # ------------------\n",
        "        #  Train Generators\n",
        "        # ------------------\n",
        "\n",
        "        optimizer_G.zero_grad()\n",
        "\n",
        "        # GAN loss\n",
        "        fake_A = generator(real_B)\n",
        "        pred_fake = discriminator(fake_A, real_B)\n",
        "        loss_GAN = criterion_GAN(pred_fake, valid)\n",
        "        # Pixel-wise loss\n",
        "        loss_pixel = criterion_pixelwise(fake_A, real_A)\n",
        "\n",
        "        # Total loss\n",
        "        loss_G = loss_GAN + lambda_pixel * loss_pixel\n",
        "\n",
        "        loss_G.backward()\n",
        "\n",
        "        optimizer_G.step()\n",
        "\n",
        "        # ---------------------\n",
        "        #  Train Discriminator\n",
        "        # ---------------------\n",
        "\n",
        "        optimizer_D.zero_grad()\n",
        "\n",
        "        # Real loss\n",
        "        pred_real = discriminator(real_A, real_B)\n",
        "        loss_real = criterion_GAN(pred_real, valid)\n",
        "\n",
        "        # Fake loss\n",
        "        pred_fake = discriminator(fake_A.detach(), real_B)\n",
        "        loss_fake = criterion_GAN(pred_fake, fake)\n",
        "\n",
        "        # Total loss\n",
        "        loss_D = 0.5 * (loss_real + loss_fake)\n",
        "\n",
        "        loss_D.backward()\n",
        "        optimizer_D.step()\n",
        "        \n",
        "        # Print some loss stats\n",
        "        if i % print_every == 0:\n",
        "            # print discriminator and generator loss\n",
        "            print('Epoch [{:5d}/{:5d}] | d_loss: {:6.4f} | g_loss: {:6.4f}'.format(\n",
        "                    epoch+1, num_epochs, loss_D.item(), loss_G.item()))\n",
        "    ## AFTER EACH EPOCH##\n",
        "    # append discriminator loss and generator loss\n",
        "    losses.append((loss_D.item(), loss_G.item()))\n",
        "    if epoch % 100 == 0:\n",
        "        print('Saving model...')\n",
        "        save_model(epoch)\n",
        "```"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "Ed-ZbuVWBUgu"
      },
      "cell_type": "markdown",
      "source": [
        "Observation de l'évolution de l'entrainement"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "ZImdTy5iFrDH",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "```python\n",
        "fig, ax = plt.subplots()\n",
        "losses = np.array(losses)\n",
        "plt.plot(losses.T[0], label='Discriminator')\n",
        "plt.plot(losses.T[1], label='Generator')\n",
        "plt.title(\"Training Losses\")\n",
        "plt.legend()\n",
        "```"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "i0TC5qK3BUg4"
      },
      "cell_type": "markdown",
      "source": [
        "### Evaluate your cGAN"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "fYBRR6NYBUg6",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def load_model(epoch=200):\n",
        "    if 'generator_'+str(epoch)+'.pth' in os.listdir() and 'discriminator_'+str(epoch)+'.pth' in os.listdir():\n",
        "        if cuda:\n",
        "            checkpoint_generator = torch.load('generator_'+str(epoch)+'.pth')\n",
        "        else:\n",
        "            checkpoint_generator = torch.load('generator_'+str(epoch)+'.pth', map_location='cpu')\n",
        "        generator.load_state_dict(checkpoint_generator['model_state_dict'])\n",
        "        optimizer_G.load_state_dict(checkpoint_generator['optimizer_state_dict'])\n",
        "        epoch_G = checkpoint_generator['epoch']\n",
        "        loss_G = checkpoint_generator['loss']\n",
        "\n",
        "        if cuda:\n",
        "            checkpoint_discriminator = torch.load('discriminator_'+str(epoch)+'.pth')\n",
        "        else:\n",
        "            checkpoint_discriminator = torch.load('discriminator_'+str(epoch)+'.pth', map_location='cpu')\n",
        "        discriminator.load_state_dict(checkpoint_discriminator['model_state_dict'])\n",
        "        optimizer_D.load_state_dict(checkpoint_discriminator['optimizer_state_dict'])\n",
        "        epoch_D = checkpoint_discriminator['epoch']\n",
        "        loss_D = checkpoint_discriminator['loss']\n",
        "    else:\n",
        "        print('There isn\\' a training available with this number of epochs')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "4V0DwQomBUg9",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "load_model(epoch=200)\n",
        "\n",
        "# switching mode\n",
        "generator.eval()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "gyvmvkIvBUhB",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# show a sample evaluation image on the training base\n",
        "image, mask = next(iter(dataloader))\n",
        "output = generator(mask.type(Tensor))\n",
        "output = output.view(16, 3, 256, 256)\n",
        "output = output.cpu().detach()\n",
        "for i in range(8):\n",
        "    image_plot = reverse_transform(image[i])\n",
        "    output_plot = reverse_transform(output[i])\n",
        "    mask_plot = reverse_transform(mask[i])\n",
        "    plot2x3Array(mask_plot,image_plot,output_plot)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "nqvrxBoGBUhD",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# show a sample evaluation image on the validation dataset\n",
        "image, mask = next(iter(val_dataloader))\n",
        "output = generator(mask.type(Tensor))\n",
        "output = output.view(8, 3, 256, 256)\n",
        "output = output.cpu().detach()\n",
        "for i in range(8):\n",
        "    image_plot = reverse_transform(image[i])\n",
        "    output_plot = reverse_transform(output[i])\n",
        "    mask_plot = reverse_transform(mask[i])\n",
        "    plot2x3Array(mask_plot,image_plot,output_plot)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "qkFVjRsOBUhG"
      },
      "cell_type": "markdown",
      "source": [
        "<font color='red'>**Question 6**</font>                                                                  \n",
        "Knowing the input and output images will be 256x256, what will be the dimension of the encoded vector x8  ?Compare results for 100 and 200 epochs"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "_GbMIfRXBUhH",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "if cuda:\n",
        "    torch.cuda.empty_cache()"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}