vision-transformer-classification-generation.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "94e07331-51ae-4601-8c3e-ce549cef2b64",
      "metadata": {
        "id": "94e07331-51ae-4601-8c3e-ce549cef2b64"
      },
      "source": [
        "# CS-503 Graded Notebook on Transformers\n",
        "\n",
        "#### Goals:\n",
        "\n",
        "The goal of this notebook is to familiarize yourself with the following topics:\n",
        "- Self-attention\n",
        "- Basic tokenization\n",
        "- Basic positional encodings\n",
        "- Transformer encoder-only (e.g. ViT) and decoder-only (e.g. GPT) models\n",
        "- Vision Transformer (ViT)\n",
        "- Supervised training\n",
        "- Autoregressive modelling\n",
        "\n",
        "This notebook will not cover:\n",
        "- Cross-attention\n",
        "- Full transformer encoder+decoder\n",
        "- Weight initialization techniques\n",
        "- Stochastic depth, dropout, etc\n",
        "- Masked or contrastive pre-training\n",
        "- More advanced tokenization techniques\n",
        "- Distributed training\n",
        "- NLP\n",
        "- Multi-modality\n",
        "- etc...\n",
        "\n",
        "This notebook should give you a solid foundation of working with Transformer models and get you \"thinking with tokens\".\n",
        "\n",
        "If you want to know more about these topics, please see some of the reading material in the lectures and at the bottom of this notebook, and feel free to ask the TAs.\n",
        "\n",
        "\n",
        "#### Instructions:\n",
        "\n",
        "- Your task is to fill in the missing code (denoted by ???), run the training loops and get some results.\n",
        "- We also ask several open questions. Please limit your answers to 2-3 sentences.\n",
        "- Submit the notebook with all cells executed.\n",
        "- We should be able to \"Restart Kernel and Run All Cells\" and get similar outputs to your solution.\n",
        "- The notebooks are individual homework.\n",
        "\n",
        "#### **Submission and due date**: → see Moodle"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ed23d351-746b-4365-af0d-f38b2c54959b",
      "metadata": {
        "id": "ed23d351-746b-4365-af0d-f38b2c54959b"
      },
      "source": [
        "---\n",
        "\n",
        "## 1 Setup\n",
        "\n",
        "We highly recommend running the notebook on a GPU, e.g. by opening it on [Colab](https://colab.research.google.com/).\n",
        "That said, if you have the patience, you can also run it on regular CPUs or on Apple Silicon (neither is recommended)."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0905ddcd-4fb6-455d-99e2-b2561e6ed9d2",
      "metadata": {
        "id": "0905ddcd-4fb6-455d-99e2-b2561e6ed9d2"
      },
      "source": [
        "### 1.1 Dependencies\n",
        "\n",
        "We assume you know how to set up Python virtual environments, install packages and use Colab. If not, please come to the exercise session for troubleshooting.\n",
        "\n",
        "This notebook requires:\n",
        "- PyTorch & torchvision\n",
        "- einops\n",
        "- matplotlib\n",
        "- tqdm"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c7ab39ab-bddc-401a-a877-948429479af5",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c7ab39ab-bddc-401a-a877-948429479af5",
        "outputId": "4173f8f7-ccc5-40fa-afb3-08f28f6b7caa"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: torch in /usr/local/lib/python3.9/dist-packages (1.13.1+cu116)\n",
            "Requirement already satisfied: torchvision in /usr/local/lib/python3.9/dist-packages (0.14.1+cu116)\n",
            "Collecting einops\n",
            "  Downloading einops-0.6.0-py3-none-any.whl (41 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.6/41.6 KB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.9/dist-packages (3.7.1)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (4.65.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from torch) (4.5.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from torchvision) (2.27.1)\n",
            "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.9/dist-packages (from torchvision) (8.4.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from torchvision) (1.22.4)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/dist-packages (from matplotlib) (0.11.0)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib) (23.0)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib) (3.0.9)\n",
            "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.9/dist-packages (from matplotlib) (2.8.2)\n",
            "Requirement already satisfied: importlib-resources>=3.2.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib) (5.12.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib) (1.4.4)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib) (1.0.7)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib) (4.39.2)\n",
            "Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.9/dist-packages (from importlib-resources>=3.2.0->matplotlib) (3.15.0)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->torchvision) (2022.12.7)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->torchvision) (2.0.12)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->torchvision) (1.26.15)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->torchvision) (3.4)\n",
            "Installing collected packages: einops\n",
            "Successfully installed einops-0.6.0\n"
          ]
        }
      ],
      "source": [
        "# Dependencies. If you already have them installed, no need to run this cell.\n",
        "!pip install torch torchvision einops matplotlib tqdm"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3515ed22-b852-42fa-a85b-456edc5a1c92",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3515ed22-b852-42fa-a85b-456edc5a1c92",
        "outputId": "58cd7681-657a-4070-8752-5620bdc552d9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Wed Mar 22 21:44:27 2023       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 525.85.12    Driver Version: 525.85.12    CUDA Version: 12.0     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   51C    P0    26W /  70W |      3MiB / 15360MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n",
            "Selected device: cuda\n"
          ]
        }
      ],
      "source": [
        "from tqdm.auto import tqdm\n",
        "import math\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "from torch.utils.data import DataLoader\n",
        "import torchvision\n",
        "from torchvision.datasets import MNIST\n",
        "from einops import rearrange, repeat\n",
        "import matplotlib.pyplot as plt\n",
        "from mpl_toolkits.axes_grid1 import ImageGrid\n",
        "\n",
        "if torch.cuda.is_available():\n",
        "    device = torch.device('cuda')\n",
        "    !nvidia-smi\n",
        "elif torch.backends.mps.is_available() and torch.backends.mps.is_built():\n",
        "    device = torch.device('mps')\n",
        "else:\n",
        "    device = torch.device('cpu')\n",
        "    \n",
        "print('Selected device:', device)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fd6fb739-cd0b-4f4a-bacf-4f2ce3c100cd",
      "metadata": {
        "id": "fd6fb739-cd0b-4f4a-bacf-4f2ce3c100cd"
      },
      "source": [
        "### 1.2 Data\n",
        "\n",
        "To reduce the computational cost of running this notebook, we'll operate on downsized 14x14 MNIST images. \n",
        "Even though we will only work on toy problems, the principles shown in this notebook are very similar to, if not the same in real-world models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f2e64236-6187-4dfd-abaf-f95c795dcdb2",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 477,
          "referenced_widgets": [
            "ac119fa734f546498441be6d7f2b8be9",
            "b1849889c6684962b870d02520aaad73",
            "f8177954e77e4faf971a763c78133030",
            "c0b3e3709e5840f88b1c907a33417f5f",
            "5fcda9418c204f788fd2ed1839a7d208",
            "6a33a35f1447480495c1c4447094fed8",
            "29802a016636479a86807967a9df5d94",
            "54ee3648fec94055b57fde13bd0aded5",
            "75388f4a628240cba9fa225745edd318",
            "f81e089cc735476cb5a22cc2bb26197a",
            "d4cf7a541ba941dc82411e9a7a0946f8",
            "d63f450d96e94fb1b1c856a28e9f2703",
            "7251e216f9144b8a86a63dd72c3c3ef0",
            "4d58ff4816934cf8bc3b8a48ce5a9dcf",
            "4b73705c99c1403db27318149ee39ce8",
            "be185ac21d4046b28c635b6d5a988f9e",
            "f17fcedc467f4957abcdac2b9ce4c164",
            "bbacdb46c586476abbd8e38c68904d48",
            "870922ce6c294ed0a67b7a3a90159bdf",
            "2795973dc1b14bc4b0fedcd6182cb44f",
            "01247f2a46524ce7a1fbb70fd24241cd",
            "a41eb04cdc644fa188e16c54a59f7470",
            "1e85194f74e6443b9c68ee066589ff39",
            "2ef5bc7213d14e27bf6452ecd6a4b3cf",
            "91f6d977c5134906a56ebaec6abfad0c",
            "6e2d164bec60427ea0fcf5475a0cb763",
            "945d5e1d58a2489c8437e9b5f317c57a",
            "6a0d23126dc8481f8a5fe0f3a67d2319",
            "f3053da2315c4988b316a7967e7d2a2c",
            "e33334969c534e0f9250bbb00a8c9967",
            "b43cd727390241e388bd7756617c4480",
            "633a8c7973e04fbfb24fea49f96abc05",
            "74bffc782dd247db9a4adbdaec1db9d0",
            "3043d4b1c3df4166931ba7589383a5c9",
            "6aa5a826d1264afe8e949737b0dc0838",
            "953ce56b49b747809dabe0dce1172c20",
            "340771315b63457cb745f3d07abe694f",
            "9162e2df1d5e417c9d382f619fbcc0dd",
            "fd73fc9d8ae54649ba27690670784b85",
            "912172a35481491fae2b3a1174e92e90",
            "4a05adbaba514ef398918e0ce4a2f33d",
            "82bcfff5dfbe44248dc2529c0c5f7528",
            "558940b629c648c3b7bd4453c4948f2e",
            "e9abe64c78794d9fb469f277293ee5db"
          ]
        },
        "id": "f2e64236-6187-4dfd-abaf-f95c795dcdb2",
        "outputId": "564bf0c4-ce55-476b-b894-a3da168ea307"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ./data/MNIST/raw/train-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ac119fa734f546498441be6d7f2b8be9",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/9912422 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Extracting ./data/MNIST/raw/train-images-idx3-ubyte.gz to ./data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ./data/MNIST/raw/train-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d63f450d96e94fb1b1c856a28e9f2703",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/28881 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Extracting ./data/MNIST/raw/train-labels-idx1-ubyte.gz to ./data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "1e85194f74e6443b9c68ee066589ff39",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/1648877 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Extracting ./data/MNIST/raw/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "3043d4b1c3df4166931ba7589383a5c9",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/4542 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Extracting ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw\n",
            "\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.9/dist-packages/torch/utils/data/dataloader.py:554: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.\n",
            "  warnings.warn(_create_warning_msg(\n"
          ]
        }
      ],
      "source": [
        "image_size = 14\n",
        "batch_size = 64\n",
        "num_workers = 10\n",
        "\n",
        "transform = torchvision.transforms.Compose([\n",
        "    torchvision.transforms.Resize((image_size, image_size)),\n",
        "    torchvision.transforms.ToTensor()\n",
        "])\n",
        "\n",
        "dataset_train_val = MNIST(root='./data', train=True, download=True, transform=transform)\n",
        "dataset_train, dataset_val = torch.utils.data.random_split(dataset_train_val, [50_000, 10_000])\n",
        "dataset_test = MNIST(root='./data', train=False, download=True, transform=transform)\n",
        "\n",
        "loader_train = DataLoader(dataset_train, batch_size=batch_size, num_workers=num_workers, shuffle=True, drop_last=True)\n",
        "loader_val = DataLoader(dataset_val, batch_size=batch_size, num_workers=num_workers, drop_last=False)\n",
        "loader_test = DataLoader(dataset_test, batch_size=batch_size, num_workers=num_workers, drop_last=False)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8zwAL-hJR_J4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8zwAL-hJR_J4",
        "outputId": "47679c82-1e8b-4c70-f215-7a927c9430e2"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "torch.Size([1, 14, 14])"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_train[0][0].shape"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3745db34-c2b2-4147-840c-3384c3f08022",
      "metadata": {
        "id": "3745db34-c2b2-4147-840c-3384c3f08022"
      },
      "source": [
        "Let's look at some images from the training set (it's just good old MNIST):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "07fae5c8-4ec7-4f13-a734-569115b71fc5",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 593
        },
        "id": "07fae5c8-4ec7-4f13-a734-569115b71fc5",
        "outputId": "399904e7-4e3f-4ffe-91e0-1192c99e9d17"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkEAAAJACAYAAACKWLIIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAB7WklEQVR4nO3debxX8/Y/8NdbmtM8aD7lRpImR0JlzEy4LmXKmEuZp1zzWBJuEW6RMpVrKFxE5ErGTpdUSGlQaTrSQGny/v3h+P5aa70/5xznM+1P+/V8PDyydu999vussz/77D57fdbbee9BREREFDc7ZXsCRERERNnAmyAiIiKKJd4EERERUSzxJoiIiIhiiTdBREREFEu8CSIiIqJYSuomyDl3lHNujnNunnNuQKomRURERJRurqx9gpxz5QB8C6AHgCUApgHo7b3/KtE+devW9Xl5eWU6Xq5auHAhCgsLXaK/Z04s5kSKYz4AYPr06YXe+3qhv2NOrEzm5LfffhOxc67YOJ2ikpMoYU6k4q6vOyfxdTsDmOe9nw8AzrlxAHoCSHgTlJeXh4KCgiQOmXvy8/OL/XvmxGJOpDjmAwCcc4sS/R1zYqUrJ/qGBwA2bNgg4goVKhQbpxPPE4s5kYq7vibzOKwxgMXbxUuKtgnOub7OuQLnXMGqVauSONyOgzmxmBOJ+bCYE4s5sZgTizlJLO2F0d77Ed77fO99fr16wXfnYoc5sZgTifmwmBOLObGYE4s5SSyZx2FLATTdLm5StI0okkJv62/cuFHEP//8s4iXLFli9pk1a1axXwMAzjzzTBFXq1at1PPMddu2bTPbdI3ITjvxg6m5Yu3atWbbddddJ+Jzzz1XxAcccEBa50SUKslciaYBaOWca+GcqwCgF4BXUzMtIiIiovQq8ztB3vutzrn+AN4CUA7AKO/97JTNjIiIiCiNknkcBu/9GwDeSNFciIiIiDKGD+aJiIgolpJ6JyiX6KaQa9asMWN074Ty5cubMfvtt5+IK1eunPzk0mDLli1m2wcffCDir76SLZ3at29v9unYsaOIc7nAN1TgeeWVV4q4sLBQxNWrVzf77LrrriJu27atGZPJZnHppF83v/zyixnz7rvvinjatGlmzBFHHCHiAw880IwpV65cWaaYdrqgXp9H06dPN/vUr19fxKFzJFeKwz/++GOz7Z133hFxv379MjWdrNCvg9WrV5sxy5YtE7H+AMX69evNPvvuu6+IW7dubcZUqlSp1POMGp03/Tso9Em1PffcM61z0nLjVUhERESUYrwJIiIioljiTRARERHF0g5RExRaBFY3vXv77bdFPGzYMLPPlClTRBx6Zj969GgRn3766WZMNmobdIM6/ewVAE477TQR6/V/Qs+eX31Vtn4K1XJElf7+3nzzTTNGN3X729/+JuKKFSuafXSt2M4725dRrtYE6fNo7ty5Ir799tvNPu+//76IQ00p33rrLRG/9957ZkwU6s1CjR51jdMdd9wh4tBrrXPnziIeNWqUGdOsWTMRR/WcWbTILkNVs2ZNEes6uR2NrgG6+OKLzZipU6cW+zVC55bO4zPPPGPG6HWvonqehGzevFnEl112mYhDTTUfeeSRtM5J4ztBREREFEu8CSIiIqJY4k0QERERxRJvgoiIiCiWdojC6E2bNpltuoDztddeE3GPHj3MPrVq1RLxwoULzZguXbqIOCpFaroI+KqrrjJjqlatKuIbb7xRxDfffLPZZ/HixSmYXfqFinFfeuklEYeKGSdOnChiXagYlZ9vOvz6669m2xtvyFVwdDPJUCGjLnIONdcbOHCgiENFotmgz5tQ40O9Qvo333xT4tfVOdAfugCAM844Q8RROde2bt0qYt0EELDngb525rLQB20+++wzEf/nP/8xY3RD0Ntuu03EoQa2V1xxhYhHjhxpxuy9994izuXmiRs3bhRx6Hd3pvGdICIiIool3gQRERFRLCX1OMw5txDAegDbAGz13ucXvwcRERFRNKSiJugQ731hycPS58svvzTbdAOzJ598UsQrV640++iGWKG6oUaNGok4Kosg6hqLVatWmTH6mbVu6qjrioBoPLMtjdBz/AULFog4tPinrsPQ9RChRXRzlf75Dh061Ix54IEHRHz55ZcXGwO2yaFeOBKw51pU6l/0deCGG24wY0qqAQp9L4cddpiIjzrqKDMmKtcOTdeuhGojW7ZsKWL989VN8gBbD1KhQgUzRte7ZOM8CV3znn/++RL3u+iii0SsF6TW3z9gFxANLfIcurblCj13vSC1rnfKhmi+ComIiIjSLNmbIA/gbefcdOdc39AA51xf51yBc64g9O5EHDEnFnMiMR8Wc2IxJxZzYjEniSV7E9TVe98JwNEA+jnnuusB3vsR3vt8732+fusvrpgTizmRmA+LObGYE4s5sZiTxJK6CfLeLy36cyWA8QA6F78HERERUTSUuTDaOVcVwE7e+/VF/38EgDtK2C0tQoXRe+21l4h1QZZukgcAl1xyiYhzaWVkvdr5oYceasaMHz9exOPGjRNxqOGgLoCMqlCRqf55hgqjr7vuOhEffPDBIu7Xr5/Zp2HDhiKOSpHv9kJF7oMGDRKxbmAI2GLp8847T8ShRm36vJk/f74Zo4tEQ0Wx6aaL3gHghRdeEHFJK4ED9uetvzfA5rZu3bqlmWIk6A9ZrFu3zozRP/OffvpJxI8++qjZR+dav9YAe47qBq+ZELoOrlmzRsS77LKLGaOvlfo80TkCgEWLFom4d+/eZkw2Xiupoq+5usj+wAMPzOBswpL5dFgDAOOLftA7A3jOe2/vLIiIiIgiqMw3Qd77+QDsP4GIiIiIcgA/Ik9ERESxlJMLqOoGTPXr1zdjZs6cKeLvvvtOxLrWAcit5/aartW46667zBj9bPnzzz8XcaiWQzfCi6pQXY7+ed55551mjG7sN2TIEBFfffXVZp9HHnlExFFYPFI/ew/V+9xzzz0ivv76680YvVioPq9Cjdt0zYheUBUATj75ZBHrGrZMCC3aqq8LoUUuNf2a0AtlAkDr1q1FHMW6sUR0o0NdtwIAO+8sf3UMHjxYxA8//LDZRzch7NSpkxmTS3kqic6jbtgLAD/88IOIDznkEDNGN6KMCv16Cl0bPv30UxEXFsq+ylGoOeU7QURERBRLvAkiIiKiWOJNEBEREcVS5GuCQs/xP/74YxH/97//NWN0a/ApU6aIOLSgoX7OnUv0s/RmzZqZMcOHDxfx4sWLRfzXv/419ROLkFC/jY4dO4p4xIgRIg7VVultd999txkT6qeTKqHXxNNPP13inAYMGCDim2++2YypXLmyiPVz/lCvkzvukO3Bli1bZsboxXuzUfsR+vnrPlAzZswwYz777LNiv87cuXPNPqFeM7lCn7t77rmnGfPmm2+KWPfN+fXXX80+ukbv7LPPNmP0+ZfLdK+uyZMnmzF6Qe7QdTuqdP1c6JzXdad/+9vfRKz792UD3wkiIiKiWOJNEBEREcUSb4KIiIgolngTRERERLEU+UpgXbwLALfffruIL7jgAjOmc2e5oL0uSgs1dsplujFXaLFIXXSom/xlo4FdWenCS12AB9iiQ73wKWAbkVWpUkXEF110kdlHN/7TDRcBID8/32xLlY0bN5ptuug91HRNLygbeg3opov6AwahhpPvvvuuiJ977jkzpkmTJmZbpoWKsfVroEaNGmaMbpipizmfffZZs89JJ50k4t12263U88w2XfjdoUMHM0Yvhrp27VoRhxaf/sc//iHi0OKZUWiWGJqDvjbq6y1gFwedNGmSiEPXqFNPPVXEofMvqnQBfeh3zrx580S8zz77iLh8+fKpn9ifxHeCiIiIKJZ4E0RERESxVOJNkHNulHNupXNu1nbbajvnJjnn5hb9mf3Fk4iIiIj+hNLUBI0G8DCAp7bbNgDAu977Qc65AUWxXY2xDHTDpQkTJpgxBx10kIhPOOEEM0Y/j9UL1YUWSkxng7tUCjWleuutt0Q8duxYM0bXhOgFDUNN7nbaKZpvFq5fv17EI0eONGP04pjt2rUzY3SDTH0O6PMGsI3hMr2AaqhmQdfcTJw40Yw59thjRdyiRQszRudDNwLcfffdzT66PiS0MGYUzqNQk0nd9E8vBgvYmsPly5eLWDfYBIDvv/9exLlUE6TPgbPOOsuM0QtU69oOvRAvAOy7774ijur1NtRUs3v37iIeP368GaOvr6tXrxZx7dq1S9wnl+oyNf37BLC/h3XT1Cgo8crkvZ8CYLXa3BPAmKL/HwPgxNROi4iIiCi9yvrPswbe+z/eNlgOoEGigc65vs65Audcgf6kSVwxJxZzIjEfFnNiMScWc2IxJ4kl/R61//1ztgk/b+69H+G9z/fe59erVy/Zw+0QmBOLOZGYD4s5sZgTizmxmJPEynoTtMI51xAAiv5cmbopEREREaVfWZslvgqgD4BBRX++krIZKbp5HQB8+eWXIn7sscfMGL2q9plnniniXF6tOFQYq1dDf+mll8wY3cBNFzPqBpNANJrchegVqR9++GEzZunSpSKeNm2aGbNo0SIR6+LZrl27mn0GDhwo4lBjuHQKnbtPPvmkiL/44gszpjQN7XTxpm4MqPMO2NdoFBrehYQ+DPHpp5+KWJ8zAHDvvfeKWJ8zunkmEC4Oz1Wh8+Rf//qXiHXhe6jAN6rnhaYbqAJAr169RPzZZ5+ZMU899ZSI9de54YYbzD5t2rQRca7kKCSUty5duohYN7CNgtJ8RH4sgI8B7OGcW+KcOx+/3/z0cM7NBXB4UUxERESUM0p8J8h73zvBXx2W4rkQERERZUz2m3cQERERZUHkFlDVz5ZPO+00M0bXKYQ+8jd48GARH3DAASLWDcFySei5cePGjUX80EMPmTG6qaRuOHjkkUeafUINvqJA5yBUO9aqVati41wVajyo6zaOOuqoTE0nZ4Sa4F122WUibt++fYlfp0+fPiJu27atGZPLNYda6HoTer3tyOrUqSNiXRcIAN26dRNxtWrVRNyjRw+zTxQWEE2VUB3YXXfdJeIovi74ThARERHFEm+CiIiIKJZ4E0RERESxxJsgIiIiiqXIVwfXqFHDbNONq8gWL+qVzoFwUzeiuAgVlO++++7FxkQhoaZ/F1xwQRZmEh2hAvrWrVtnYSZ/Dt8JIiIioljiTRARERHFEm+CiIiIKJac9z5zB3NuFYBFAOoCKMzYgZOXzHybe+/rJfpL5sRiTqTt8pHsMbIh3TmJUz4A5iSEObGYEylxPjJ5E/R/B3WuwHufn/EDl1Em5sucZOcYqcScWOmeL/ORnWOkEnNiMSdWuubLx2FEREQUS7wJIiIioljK1k3QiCwdt6wyMV/mJDvHSCXmxEr3fJmP7BwjlZgTizmx0jLfrNQEEREREWVbUu8EOeeOcs7Ncc7Nc84NSNWkiIiIiNKtzO8EOefKAfgWQA8ASwBMA9Dbe/9Von3q1q3r8/LyynS8XLVw4UIUFhbafuJFmBMrkzn57bffiv370FIL6VBcTuJ4jgDA9OnTCxN9rDWbOdHXzNA1NF3nTVRzkk3MicWcSMVdX5NZO6wzgHne+/kA4JwbB6AngIQ3QXl5eSgoKEjikLknP7/4T/QxJ1a6chL6ZfXrr7+KeNu2bSKuWrWq2Se0Rk6yistJHM8RAHDOLUr0d9nMyaZNm0S8ZcsWM0afN6k6Z6Kak2xiTizmRCru+prMP1caA1i8XbykaJvgnOvrnCtwzhWsWrUqicPtOJgTizmRmA+LObGYE4s5sZiTxNL+Xr/3foT3Pt97n1+vXsImwbHCnFjMicR8WMyJxZxYzInFnCSWzOOwpQCabhc3KdpGFEn6MQYA3HvvvSJetEi+i/zggw+afWrWrJnSeVG06Uek48aNK3Gfs846S8TpeISaKVu3bi1xm85R6BFhxYoVRVypUiUzJlfyFHq0vnnzZhHrR+2hOrEqVaqIuFy5cimYXXToHPzvf/8T8eTJk80+l1xyiYhr166d+oltJ5l3gqYBaOWca+GcqwCgF4BXUzMtIiIiovQq8ztB3vutzrn+AN4CUA7AKO/97JTNjIiIiCiNknkcBu/9GwDeSNFciIiIiDKGa4cRERFRLCX1TlAu0YV869atM2N0cV8uV9GHCvd0YbCO9fdfWrrgURf7AdkpeNTfz9SpU82YJ598UsRdu3YV8c47x+YlAsA2j1y2bJmIv//+e7PPwoULRXzooYeaMfXr1xdxVAtgQ6+b+fPni/jll18W8bBhw8w+mWqymQ4//vijiB977DEzZs6cOSJev359sX8PAB07dhTxnXfeaca0aNFCxFE9T3SOAPshizfekA9JQh+ouP/++0XcuXNnMyZXziVdBA3YIucvvvhCxIMHDzb7hH5/pFNuZJeIiIgoxXgTRERERLHEmyAiIiKKpR224EE/2//kk09EPHToULPPNddcI+JcrgkK1W7ceuutItaNqkJN0XRNTOi59h133CHiE044wYzJxrP9JUuWiFh//4D9ni+//HIRh9YOy1W6mVthYaEZM2PGDBHfd999Iv7yyy/NPrq+7oILLjBjHnjgARGHGuVFwYYNG8y22267TcTdunUTcePGZrWgnKbPk9BroH379iLec889Rbx27Vqzz/XXXy9iXY8H2NdoVGrydPPHp556yox56aWXRNy0aVMRT5s2zeyjXxejR482YzJdI1NWEydONNv0z/iHH34QccOGDdM6p9LgO0FEREQUS7wJIiIioljiTRARERHFEm+CiIiIKJaiUXWWpFCDs19++UXEDz/8sIh1cy/AFvflso8++shs003edCH4XnvtZfapXr26iGvVqmXG6P2y0dwrVNT9wgsviFg36gKAG264QcQdOnQo8Vj6fItiQ7c1a9aYbfo1ECruXL58uYh1UWzv3r3NPt99952IQ43ydGFpVAqj9c9y+vTpZszs2XJJRN3gLirFu6my6667ilh/WACwefv5559F/Pjjj5t99OvkoIMOMmOi2hhQzz3U1HDkyJEirly5sohPPvlks48+d6J4LUlEX3P1hygAW+iuz60oiOYZR0RERJRmvAkiIiKiWErqfVzn3EIA6wFsA7DVe5+fikkRERERpVsqHmYf4r23XdfSSC/UNmvWLDPm6aefFvHzzz8v4quvvtrskytNqUL0wpd60UfANu+64oorRKzrf3KJrmUBgBdffFHEe+yxhxnTp08fEetaso8//tjss2DBAhHn59t7f91MLt01MLrm5rnnnjNjdGO2Nm3amDE9evQQ8dFHHy3iAw44wOzz2muvifjTTz81Y8qXL2+2RYG+lgwfPtyMOfLII0Vcu3bttM4pk/R1AwBWrlwp4s8++8yM0Y0233vvPRF/+OGHZp9LL71UxKFzKao1Qbp2Ry+0DNgamTFjxog49L3pxqJ6Meoo0w0xQ3Wo+vdwFGueonnGEREREaVZsjdBHsDbzrnpzrm+oQHOub7OuQLnXMGqVauSPNyOgTmxmBOJ+bCYE4s5sZgTizlJLNmboK7e+04AjgbQzznXXQ/w3o/w3ud77/NzeS2uVGJOLOZEYj4s5sRiTizmxGJOEkvqJsh7v7Toz5UAxgOwzROIiIiIIqjMhdHOuaoAdvLery/6/yMA3FHCbn+aLlwE7ArwDz30kBnTvbt8U0oXeOriVsCunpxLTdB0gWNohfAKFSoUu08u0UWIugAPsE37QkWvuqC4X79+In7llVfMPhs3bhRx6F9WgwcPFvEZZ5xhxpT1/Nq2bZvZpgu4dRE0YJu13X333WaMLszUb50/+uijZh/ddLFTp05mTKihaRQsWbJExKGCXr2KvP65rV692uzzzTffiFh/KAEAmjRpIuJsFI3q8x+wP+NQ40N9XdY5uPDCC80+F110kYhz+YMoIfpc0r+XQs1499lnHxFHtTA8RDfI7NixoxmjV4nX580777xj9qlZs6aI99tvPzMmlR+0SOa3fAMA44teuDsDeM57PzElsyIiIiJKszLfBHnv5wNoX+JAIiIiogjKnffeiIiIiFIo8kUv+jkrYGs7zj33XDNGb9N1ROPHjzf76EUnc+mZtX6WHKpB0AuI6sXtbrrpJrNP3bp1RRyVZlebNm0S8SeffGLG6GfJ3bp1M2N07Y5uqhlqXqYXFdXN5QCgoKBAxL169TJjyloTFKqT09/HsmXLzBhdE6abHALA119/LWK9CG3o6+r6LF3rAoTrmKLgq6++EvEuu+xixujmiFOnThXx7bffbvbRNTKhBovjxo0TcTY+taPPCQC48sorRXz22WebMfpn/p///EfEw4YNM/vo18AhhxxixkTl+lKS0ALFTz75pIj1deGee+4x+1SrVi2l88qm+vXrm226FlBfX++8806zz7p160Q8Y8YMM6ZRo0ZlmWIQ3wkiIiKiWOJNEBEREcUSb4KIiIgoliJfExR6Rqz7CLz55ptmzMyZM0X85ZdfijjUH0M/i0zlc8d00zVBf/vb38yYH3/8UcQTJkwQsV48FAAefvhhEad7IdDSqly5sogPO+wwM2by5Mki1t8vYGvDdO+kUC2LrhMK9bE44ogjRJzKnlOh/k66RmHDhg1mjO7/EupJovOq5x06th4TqvWI6sKQ+pxv27atGaMXEL355ptFfNlll5l9atSoIeLrrrvOjNE/s2zUBJXm+qrjEF07qBcPBWzdUKhGL6oL7eoaxGeffdaMefDBB0V8/PHHi1j3rgOAcuXKpWB20RDqsaWvr/q6NGnSJLOPrudN9znBd4KIiIgolngTRERERLHEmyAiIiKKJd4EERERUSxFvjA61PRv4kS5RFloAUPdhGratGkiDjXXa968eVmmGEmNGzc22/RCkLoQUxdBA3ax0KgURuui3n333deMGTRokIhDBfS6QF43jws1MzvqqKNEHFqIVDcMTGUBZKjB3bHHHiviUDGhLvIONQM99NBDRdysWTMRX3rppWYfXQQcal4a1cWI27RpI+IBAwaYMS+++KKIdTFxqPHq8uXLRXz99debMem+3uiGhoBd2DV0DuhzN3S+6SZ4esHmFStWmH309Saqi4WGPgzx3nvvifi+++4zY9avXy9i/Xtp9OjRZh+dg1DDwWOOOUbEUWniqwvmdRPZ0LYTTjhBxHrxZcAuuqqvL6kWzbOQiIiIKM14E0RERESxVOJNkHNulHNupXNu1nbbajvnJjnn5hb9WSu90yQiIiJKrdI8qB8N4GEA2z+8GwDgXe/9IOfcgKLYPvROgdDzaN3EsDRNDXVty1tvvWXGbN68WcS6cVxU6OfxgG36FlpkUz9//u6770TcunVrs08o/1HUvn17s+3GG28Usa4lA+zzf73IYygneluobiid9Q6hxoNXXHGFiC+++OISv06oTkfXuzzxxBMiDtWZnHTSSSKuU6dOiV83Kvbee28RhxaV1Y3/dNPNPffc0+xz//33i7hFixZmTLqbwOkGf4BdaHfKlClmzBlnnCHi0AKquv5l4MCBIm7ZsqXZ57TTThNxVBsFrl271mzTi59+//33Zoz+fnRu9cK7gH0tn3POOWaMrgmKiurVq4tYL2wO2AaS77zzjohff/11s4/OW7p/B5V4pfbeTwGgK497AvjjyjAGwImpnRYRERFRepX1n6sNvPfLiv5/OYAGiQY65/o65wqccwWrVq0q4+F2LMyJxZxIzIfFnFjMicWcWMxJYkm/Z+9/fzZjn8/8/78f4b3P997nZ2NtnChiTizmRGI+LObEYk4s5sRiThIr603QCudcQwAo+nNl6qZERERElH5l7WD2KoA+AAYV/flKymaUJrp4ev78+WaMXtH25JNPNmOi0OAr1MxLF6XNnj3bjNHFjJ9//rmIx40bZ/aJanG4Fmog1rdvXxGHig41/f1G4eddGnrepfm5hQrsFyxYIGLdQFM3FwSAK6+8UsRRXTE+RBezhgrs7733XhHffvvtIg41EI3Cauihc0AXbL/99ttmjC5M/cc//mHG/PDDDyLeZ599RHznnXeaffLy8hLONUpCP7sOHTqIONQY8IADDhCxfq2ECnz1uzJt27Y1Y6J6DdYfdjjuuOPMGN0w8qeffhLx5MmTzT66WWe6leYj8mMBfAxgD+fcEufc+fj95qeHc24ugMOLYiIiIqKcUeI7Qd773gn+6rAUz4WIiIgoY3LjvX4iIiKiFIvmqoZpoBvaDRkyxIzR9R+//fZbiWOyITSHLl26iDjU9E0/k3/22WdFnJ+fX6pj5Qr9bD8KdRpREqoJ0vU8N910k4hbtWpl9tGLPka1MWJZ5ep5FHrt6hqU008/3Yw55ZRTRKybyAL22qjrokL1L7lyXoSan+omk6G6TP3a0fkvzfefKzkKCdUCXnjhhVmYyZ+Tu7/hiIiIiJLAmyAiIiKKJd4EERERUSzxJoiIiIhiKTaF0bopWqdOnbI0k+SFCh4PPPBAEb/xxhtmjC7o1MWMuVyUR39e6Dxq3LixiEMriNOOI/Sa1wWuudT8MhVCOQk1xKQdA98JIiIioljiTRARERHFEm+CiIiIKJZcqGFa2g7m3CoAiwDUBVCYsQMnL5n5Nvfe10v0l8yJxZxI2+Uj2WNkQ7pzEqd8AMxJCHNiMSdS4nxk8ibo/w7qXIH33rYnjqhMzJc5yc4xUok5sdI9X+YjO8dIJebEYk6sdM2Xj8OIiIgolngTRERERLGUrZugEVk6blllYr7MSXaOkUrMiZXu+TIf2TlGKjEnFnNipWW+WakJIiIiIsq2pN4Jcs4d5Zyb45yb55wbkKpJEREREaVbmd8Jcs6VA/AtgB4AlgCYBqC39/6rRPvUrVvX5+Xllel4uWrhwoUoLCxMuB4Fc2IxJ1Ic8wEA06dPL0z0sVbmxGJOLObEimNOiru+JrN2WGcA87z38wHAOTcOQE8ACW+C8vLyUFBQkMQhc09+fvGf6GNOLOZEimM+AMA5tyjR3zEnFnNiMSdWHHNS3PU1mcdhjQEs3i5eUrRNcM71dc4VOOcKVq1alcThdhzMicWcSMyHxZxYzInFnFjMSWJp/3SY936E9z7fe59fr17CJsGxwpxYzInEfFjMicWcWMyJxZwklszjsKUAmm4XNynaFgm//fabiDdu3CjiLVu2mH2qVq0q4vLly6d+YhH2yy+/mG2vvvqqiHfbbTczpnPnzmmbU6bpGrkNGzaYMeXKlRNxpUqV0jonIiJKj2TeCZoGoJVzroVzrgKAXgBeLWEfIiIiokgo8ztB3vutzrn+AN4CUA7AKO/97JTNjIiIiCiNknkcBu/9GwDeSNFciIiIiDKGa4cRERFRLCX1TlBUhBo+zp4tn8xddtllIv7222/NPpdccomIL7/8cjOmWrVqZZliJOg8ff/99yI+99xzzT6FhYUiHjlyZOonFiFr164VsT4nAOC0004T8QknnGDGOJewF2RarF69WsT//e9/zZgePXqIeJdddvnTx9m2bZvZpj9yW7duXTNm5513iEsNAPs6Wr58uRkzffp0Ebdt29aMadq0qYh1wX0u2bRpk4i//PJLM0Z/fx06dDBjdtppx/l3uf5wTuhDFkuXys8SVahQwYypWLGiiGvXrl3imGz49ddfzbZ58+aJWL8ufvzxR7NPs2bNRBzq8aPHJHPe7DhnHBEREdGfwJsgIiIiiiXeBBEREVEs7RAP6kNN/gYOHChiXcvTv39/s8+9994r4tDz2SuuuELEudRQceXKlSI+/vjji40B4KabbhJx5cqVUz+xLNL1HTNmzBDxSy+9ZPbp3bt3WudUFt98842Ir776ajPm9ddfF3GbNm3+9HEWL15stt16660iHjp0qBlTs2bNP32sbAjVF+rryyeffCLiRx991OzzyiuviLh79+5mzJgxY0Ssa4SiTNeGTZgwQcShWrrBgweLOFQTlMt0A95p06aJOPS6+OCDD0QcqiXUzVhDtZvXXHNNqeeZKps3bxbxP//5TzNGn+MdO3YUcZMmTcw++ho8bNgwM+app54ScTILwvKdICIiIool3gQRERFRLPEmiIiIiGKJN0FEREQUSzlZGK2LF3VxGQC8//77ItaroYeal+nmgQ8++KAZc+aZZ4q4YcOGxU82SzZu3Gi23X777SLWBXahYvFcKvwuC13cpwuhQ4Wy9evXF3GmGyOG6MZs+vsCgJ9//lnEoe+tpO/l448/NtumTp0q4q1btxb7NaJEv06++OILM2bUqFEi1kXAIfoDBFOmTDFj/vOf/4i4b9++ZkwUGiiGGmTq82DIkCEl7lOrVi0R53JjxFBjQP3z1B9OCL0uTj/9dBHrHAHAkiVLRNy+fXszJhvnib5WhJqv9unTR8S6AXHoHPj0009FfN9995kxoWtXWeXuWUhERESUBN4EERERUSwl9TjMObcQwHoA2wBs9d7bRT6IiIiIIigVNUGHeO8LSx6WOrp52RNPPGHGnHjiiSLWNUChRoh6gcnQYqGLFi0ScVRqgvQzUt2oCwD23ntvEZ9//vki3tHrf0J++OEHET/99NMi1s/sAaBdu3ZpnVNZNGrUSMShBUtfeOEFEYea1enXha5j+Prrr80+ujYgKrUeuk4q1Ojx4YcfFvHo0aPNmBo1aoi4X79+Ij7vvPPMPrquQddGALZO6JxzzjFjstGcVNeTvfvuu2aMbnyom8iG6qZ0LV2I/pmFatTSXYMXqjfRCyuHGvgNHz5cxPr71Q18Afs7J/S61a/B0HU6G685PY999tnHjHn55ZdFrBfa1bW6APDmm2+KWC9+DqS2sWg0rlZEREREGZbsTZAH8LZzbrpzzn60AYBzrq9zrsA5V7Bq1aokD7djYE4s5kRiPizmxGJOLObEYk4SS/YmqKv3vhOAowH0c86ZBXK89yO89/ne+/x69eolebgdA3NiMScS82ExJxZzYjEnFnOSWFI3Qd77pUV/rgQwHkDnVEyKiIiIKN3KXBjtnKsKYCfv/fqi/z8CwB0pm1mRUJGabtSlV8cGgLFjx4o4VAit6QZfoWPrwr2o0E3fQgWed999t4g3bNgg4scff9zss3r1ahFfdNFFZkyu/Msi1MBNNzhbs2aNiHv16mX20as6R4Fejfmvf/2rGaMbQerCeADYY489RKzz8dprr5l9dIF1VPJTWCg/r6Gb1wG2MDO0Qvd1110nYl2UGSpUrVKlioj32msvM2bu3LkizkaTyVBTVf1Bk9Dq4LqxY4sWLUSsVwIHbK6XLl1qxrRs2VLEoeL9dH+AI9RodNCgQSIONdLVr50nn3xSxKHvpTRNDqPQMLOsdIG8bkisXycAcMcd8jZCnxNAanOSzKfDGgAYX1SpvzOA57z3E1MyKyIiIqI0K/NNkPd+PgDbv5uIiIgoB/Aj8kRERBRLkV9AVTdXAoDnn39exIcddpgZo5tQ6QZboQXw3n77bRE3btzYjMnLy0s412wKPV/X9LPuM844o8Sv8e2334q4evXqZkyomVUUheofdJ1CmzZtRNy5s631j8KCqZqukwgtxjlp0iQRX3XVVWbMvffeK2JdR7VgwQKzT8+ePUVcsWLF4iebBlu2bDHbnnnmGRGHGrOdeuqpIr7rrrvMmLp164q4ND9/3eQwVDf31VdfiThUs5Zq+nr66KOPmjG6dqpVq1ZmzKxZs4r9OqFria4VCzWa3X333UUcah6YbqGfw8KFC0UcqhvS9S66LvUvf/mL2Uc34swluobts88+M2N03Vv37vID5HrhXcDmJN3XW74TRERERLHEmyAiIiKKJd4EERERUSxFviZo/vz5Ztu4ceNEfOmll5ox+vmz7u/zzjvvmH10T4OHHnrIjGnQoEHCuWaT7t+iF0sFbK2G7gtUp04ds49+ZqvrP3KJrm8CgOnTp4v4tttuE3GoBiqK9HPzUB2Hrtvo37+/GaN/vrr3iV68GLA1FKE6vlA/kFTSPa8AuwByrVq1zJjLL79cxDVr1vzTxw7199ELzc6ePduM6dSpk4gzUUul5zpz5kwzplu3biJu395+CHjfffcVsb7e6LpNwNYaRbUeJtTn6sorrxRxs2bNzJiJE2WHGN2rrXfv3maf0KKjURSqk3rvvfdEPH78eDNmzz33FPHxxx8v4tA5kOmaS74TRERERLHEmyAiIiKKJd4EERERUSzxJoiIiIhiKfKF0T/++KPZposzQ0XAugBw8uTJIg4tptinTx8Rh4qAo7qY3bp160R83HHHmTG77rqriHXDyGHDhpl9ateuLeJQA8mo0sV8oWZ569evF3GXLl1EHNWfd0lC895///1FHCpk1IsR6zGhImDdGC/USC7dhdE77WT/PVe/fn0Rf/PNN2aMLng96KCDzBhdZK6PpZvkAbZRnn4dAcA111wj4kwURuufw/Dhw80YvXB0aPFp3cRw6tSpItYLEwPRbDIK2O83VNivF8DVCxYDwIoVK0S8ZMkSEetrdC5ZtWqV2fbAAw+I+P777zdj3nzzTRHrhq1HH3202ac0i52nEt8JIiIioljiTRARERHFUok3Qc65Uc65lc65Wdttq+2cm+Scm1v0p23AQURERBRhpakJGg3gYQBPbbdtAIB3vfeDnHMDiuLrUz+9cA1KtWrVRKyfvwPAjBkzRKwbxZ133nlmn1tuuUXEehHEKNN1UnfeeacZc+2114r4ueeeE7FeKBIABgwYIOJsLGhYVrpB5k8//VTiGB3vSHSdUIsWLcyYSy65RMRdu3YVcajW7KSTThJxNprgVa1a1Wz717/+JeJQDZReCHTOnDlmjK5r0OdI8+bNzT6nnHKKiHW9IWAb7oXqmlJN1+Wkq1ardevWZlumaz1KS9cA3XTTTWbMd999J+JQHZhuxqoXqO7YsWNZp5h1oQVx9eKooXNJNx/98MMPRRxa+DhyNUHe+ykAVqvNPQGMKfr/MQBOTO20iIiIiNKrrP/0aOC9X1b0/8sBJFxLwjnX1zlX4JwrCFWYxxFzYjEnEvNhMScWc2IxJxZzkljS77/63z9f6Iv5+xHe+3zvfX69evWSPdwOgTmxmBOJ+bCYE4s5sZgTizlJrKw3QSuccw0BoOjPlambEhEREVH6lbXK9VUAfQAMKvrzlZTNSGnUqJHZdtppp4n4xRdfNGOWLVsm4nvuuUfEoULFXXbZpSxTjISLL75YxHo1dMAWePbt21fEbdq0MftEtZixNHQRqG4WCQCnn366iHXjzag2eEsXXZyri6cPPPBAs8+ECRNEfOihh5oxocLlVAoVFevi3Ouuu86M0YWZocJ4vUK9bq4X+gCFbnxYvnx5M2ZHoovuQ80hM1H4XRZ67qFCd/3hmzp16pgxl19+uYj79esn4mx8YCBVdBE/YFeI19dSwL42dMPI0Or0mVaaj8iPBfAxgD2cc0ucc+fj95ufHs65uQAOL4qJiIiIckaJ7wR573sn+KvDUjwXIiIiooyJ5vuTRERERGkW+c53oUUF9UKfgwcPNmN0LYt+NpmrC2Mm0rJlSxE/9dRTCUbGh27sqBfLBOwz6XQv9JlrqlevLuJBg+yT79mzZ4s4qq+t0LxKM1eeEyXT9SENGtiuKVGti9LzuvDCC80YXecWqnnSzWb1183l+sJQI91Ro0aJ+KOPPjJjdJPJfffdV8RReG3xnSAiIiKKJd4EERERUSzxJoiIiIhiiTdBREREFEuRL4wOFZPpYqooFFdR9FWqVCnbU8g5+vUXWnk+tI3ipVatWsXGuSTU/LJt27ZZmEl0hH4P6+U3evbsmanppBTfCSIiIqJY4k0QERERxRJvgoiIiCiWnF4MMK0Hc24VgEUA6gIozNiBk5fMfJt77+sl+kvmxGJOpO3ykewxsiHdOYlTPgDmJIQ5sZgTKXE+MnkT9H8Hda7Ae5+f8QOXUSbmy5xk5xipxJxY6Z4v85GdY6QSc2IxJ1a65svHYURERBRLvAkiIiKiWMrWTdCILB23rDIxX+YkO8dIJebESvd8mY/sHCOVmBOLObHSMt+s1AQRERERZVtS7wQ5545yzs1xzs1zzg1I1aSIiIiI0q3M7wQ558oB+BZADwBLAEwD0Nt7/1WiferWrevz8vLKdLxctXDhQhQWFtqe40WYE4s5keKYDwCYPn16YaKPtTInFnNiMSdWHHNS3PU1mbXDOgOY572fDwDOuXEAegJIeBOUl5eHgoKCJA6Ze/Lzi/9EH3NiMSdSHPMBAM65RYn+jjmxmBOLObHimJPirq/JPA5rDGDxdvGSom2Cc66vc67AOVewatWqJA6342BOLOZEYj4s5sRiTizmxGJOEkv7KvLe+xEoqurOz89nFTYyk5MtW7aUuE2vlhxaKThTeJ5IyeZj06ZNZtuUKVNEvOeee5oxjRvLf8dk85zQeI5YzImViZz89ttvZlvomru9ChUqmG2Zen3xPEksmXeClgJoul3cpGgbERERUeQlcxM0DUAr51wL51wFAL0AvJqaaRERERGlV5kfh3nvtzrn+gN4C0A5AKO897NTNjMiIiKiNEqqJsh7/waAN1I0FyIiIqKMSXthdDps27ZNxKtXrzZjvvzySxHrivgWLVqYfRo0aCDipk2bmjHlypUr9TyzafZs+6bco48+KuKhQ4eKuFKlSmmdE2VOqP/X448/LuJdd93VjBkyZIiIy5cvn9qJxdTPP/9stv3vf/8TcZcuXcyYUDFtFP3yyy8iHjt2rBmzzz77iLhDhw5mTDYK8XWR84YNG8yY6dOnFxsDwLx580SsX4O33Xab2Uf/zqHM4wKqREREFEu8CSIiIqJY4k0QERERxVLka4L0s2YA+Ne//iXiUP3L7rvvLuJmzZqJ+KOPPjL7jBw5UsQXXnihGXP++eeLuHr16mZMFNSoUcNs++yzz0S8Zs0aEYdqRHYkoToZvW3z5s0iXrZsmdnn/fffF7E+twCge/fuIt5558y+1EK1JG3bthXxuHHjzBhdDxE6j+JGnyOlqVvR+7z55ptmzMyZM0V8wAEHlGF20bBu3ToRX3HFFWbMlVdeKeL27dubMdmoCdK/P66//nozZtKkSSIOzf2aa64R8Ysvvijip59+2uyjcxLVmtNQI8jvvvtOxB9++KEZ06hRIxHrureaNWuafTJ9DvCdICIiIool3gQRERFRLPEmiIiIiGKJN0FEREQUS5EvjNZNDgHgmWeeEfFNN91kxnTq1EnEunnXQw89ZPbRq2q3atXKjAkV10ZR3bp1zbYff/xRxK+//rqIddF3lOmfw/r1680YXfCoV1EHgF122UXEX3zxRbFfA7BF9ccee6wZo4tcM10YvXXrVrNt+fLlIg4VdOvmiLox6Q8//GD2WbFihYibNGlixtSqVUvE2VxRuzihAlB93ujrhC7+BICffvpJxKEi9Pvuu0/EmT5HUunXX38VsT5vokyfm6Gi50suuUTE++23nxlTp04dEevmiZ988onZR59vUS2MDl0H9QeHVq5cacbo7++oo44S8c0332z2qVy5sohDDSVTea3gO0FEREQUS7wJIiIioljiTRARERHFUlIPoZ1zCwGsB7ANwFbvfX4qJrW9UH2BXuTxrrvuMmPee+89EevFQf/xj3+Yff7+97+LOFRXE4W6hdKoWLGi2dauXTsRT5gwQcRnnnlmqb5OFOgmmhdccIEZ88Ybb4g41Azy+OOPF7F+Zq1rAQDg8ssvF3Godizb9R2hJo86H1dddVWJX2fMmDEivv/++82Yxo0bizhU7/OXv/xFxAMGDDBjsrGYpK4tKygoMGOGDRsm4tGjR4tYL8AJ2BrE5s2bmzGhBZpzha71eO6550Ssa4QAoE2bNiKOyrVUn7933nmnGaNrdUozdz2mfv36JX7dqNKNdgF7DdbXFwCYNWuWiG+44QYR6xpMADjiiCNEfMcdd5gxqVxYOBVX6kO894Up+DpEREREGcPHYURERBRLyd4EeQBvO+emO+f6hgY45/o65wqccwWhj7vHEXNiMScS82ExJxZzYjEnFnOSWLI3QV29950AHA2gn3Ouux7gvR/hvc/33ufXq1cvycPtGJgTizmRmA+LObGYE4s5sZiTxJKqCfLeLy36c6VzbjyAzgBsR7okhApMdSO6vfbay4zRhdC6IVaVKlXMPtWqVRNxVAr3ykI3vQOAgw8+WMT33HOPiDdt2mT2iWphtLb33nubbRdddFGJY/QqxjvtJP9d8Pjjj5t9pk2bJuJLL73UjNFfJ910c8SRI0eaMboQt1evXmaMbgx44403ivjWW281++ivU1hoSwR1I85QkzRdgJwJS5YsEfHFF19sxjz66KMirlGjhogXLlxo9nnkkUdE/Nprr5kxoddortANMvX5dsghh5h9jjnmGBFH5fqq55GuDzVEtUFoaXTu3Nlsa9iwoYhDzSD1NVc3Fg2tPH/WWWeJON2vkzJfqZ1zVZ1zu/zx/wCOADCr+L2IiIiIoiGZW94GAMYX3cnuDOA57/3ElMyKiIiIKM3KfBPkvZ8PwC6yQkRERJQDcnLFPr0w3YIFC8wY3dCsatWqIn7ggQfMPnpB0ZNPPtmMyXStR1mFnjW3bNlSxD///LOIc2nRQ/3z1E24gNI1ONPN8vS5NHToULNP//79RRxaQDXT54luXBZasFM3KKxdu7YZM3fuXBHrc+b00083+1SvXl3EelFawDZAC9Vabdy40WxLhm5i+P3335sx+++/v4j1IrMA8Oabb4r43//+t4hDizHn58u+saFGebkitBjvk08+KeKlS5eKWNdEAeHzIk5CjRFzpSYoVE957bXXijj0O1W/NvRC1/o6ng258RudiIiIKMV4E0RERESxxJsgIiIiiqXI1wSFFifUPTfatm1rxujFT3WNhu5ZAdjn3HpxTSB3+uaE6F5JLVq0EHG2F/38M1LV22PdunUi1gvr6t5KgO1xU7ly5TIdO5V0bdPmzZvNGN2/J1SDs/vuu4t4/vz5In7nnXfMPh06dBBxaCFFvcBmaGFa3acrWXoRz1APoLy8PBGHFojV9TwTJ8oPwYaaz73wwgsiDvUlyxWhDsN6Yd3u3WWf3G7dupl9cqWesqz0a1D/7gq93nRdZihHP/74o4hDdTTpboAYqmfq0aOHiLt06WLGLF68WMT6WhnqO/bVV1+JONS/TvcBTMaOfVYSERERJcCbICIiIool3gQRERFRLPEmiIiIiGIp8pWwocJo3dCtcePGZkxJRXiladKkC91ynS4eXrZsmYhzqVliWWzYsMFsGzJkiIh1oeI///lPs0+qC3hTQRcchpoa3nfffSKeNGmSGaM/UKCbB/71r381++imi3vuuacZc9ttt4n4uOOOM2NCC0wmQ3+IQX/wAbAL6IYKLnUjwGuuuUbEY8eONfs0adJExLnSFA8AtmzZIuJnn33WjNFN7wYPHiziKL5GSit0nXj77bdF/PXXX5sx+vqpG5bqgl8AmDp1qoh141HAFiXrD28AwJFHHmm2pZuel34thbbpHLVp08bsoz9Yoc9HgIXRREREREnjTRARERHFEm+CiIiIKJZKrAlyzo0CcByAld77tkXbagN4HkAegIUATvXe/5SOCYaaNHXu3FnEeoFDwC4oqesN9DNewNZD5FLzwLLQz/V1c7lcp5tsPfHEE2bMokWLRKwX9mzQoIHZJ4r1Hfr81g0dAbuIaahGQdfBde3atcRj16hRQ8RHH320GVOnTh0RZ6Jxnr527LrrriXuE2oyeccdd4j40EMPFfGBBx5o9oniOVJas2fPFrGumwOACy+8UMS6YW0uf/+rV68224YPHy7iOXPmmDGtW7cW8Zo1a0RcmoaDOgZsbtPdGDGd9HkRqoHSNXihn0cqF+MtzZVoNICj1LYBAN713rcC8G5RTERERJQzSrwJ8t5PAaBvxXoC+KNv+hgAJ6Z2WkRERETpVdb3pBt47//4fPVyAPaZQRHnXF/nXIFzriC0Bk0cMScWcyIxHxZzYjEnFnNiMSeJJf1g3v9eRJCwoY73foT3Pt97n5/LzzJTiTmxmBOJ+bCYE4s5sZgTizlJrKyVvyuccw2998uccw0BrEzlpLYXKrDThZe6URcAXHrppSLWDd1Cd8MDBw4U8Y5WGK2LU/XKz6lsQJVpoUaPL7/8sohDBfRDhw4VsS6EzpUCTz3PUDNQXdCrY7JFwQCwYMECEesV4suXL5/WOaVTqBnt5MmTRdyxY0cz5vLLLxexbkyZyxo1amS2vfTSSyIO5U3nQK+grhsjAkDv3r1FrJue7mj0dSpUCK6buv73v/81Y/r06ZOyOZX1naBXAfwxiz4AXknNdIiIiIgyo8SbIOfcWAAfA9jDObfEOXc+gEEAejjn5gI4vCgmIiIiyhklPu/x3vdO8FeHpXguRERERBmTk0UvenHCUK3HBx98UOw+++23n9kntADcjmT33XcX8euvvy7iKlWqZHI6KaUX1QWACRMmiDhUO7bbbruJOFdqgCg9dO0gYM8b3Rwyl4XO92OOOUbEocU5dWPZHUmokWeoqV9JWrVqJeIWLVqYMTta3WlJ9PnWsGFDM0YvvqsbH6cal80gIiKiWOJNEBEREcUSb4KIiIgolngTRERERLGUk1VZejXePfbYw4wJbYs7XYSnC9ByiV7tPFTgeeutt4pYr/IMZGY1c8odzZs3L9W2HUXodRN6ndCfp3Oby0010yXUZPOGG24Qcbt27dI6B/4GICIioljiTRARERHFEm+CiIiIKJacrq1I68GcWwVgEYC6AAozduDkJTPf5t77hMv2MicWcyJtl49kj5EN6c5JnPIBMCchzInFnEiJ85HJm6D/O6hzBd77/IwfuIwyMV/mJDvHSCXmxEr3fJmP7BwjlZgTizmx0jVfPg4jIiKiWOJNEBEREcVStm6CRmTpuGWVifkyJ9k5RioxJ1a658t8ZOcYqcScWMyJlZb5ZqUmiIiIiCjb+DiMiIiIYimpmyDn3FHOuTnOuXnOuQGpmhQRERFRupX5cZhzrhyAbwH0ALAEwDQAvb33XyXap27duj4vL69Mx8tVCxcuRGFhoV2gpwhzYjEnUhzzAQDTp08vTNTbgzmxmBOLObHimJPirq/JLKDaGcA87/18AHDOjQPQE0DCm6C8vDwUFBQkccjck59ffFsD5sRiTqQ45gMAnHOLEv0dc2IxJxZzYsUxJ8VdX5N5HNYYwOLt4iVF2wTnXF/nXIFzrmDVqlVJHG7HwZxYzInEfFjMicWcWMyJxZwklsw7QaXivR+Boo+25efnp+WjaKFHer/88ouIN27cKOJKlSqZfapVqyZi5xI+sUlKJnKSa5gTifmwopqTLVu2mG2bNm0ScZUqVcyYnXZK/nMp2crJb7/9JuLNmzeLeOed7a+W0LZ0iOp5kk1RyYk+b+bMmWPGLFmyRMTdu3c3YypWrJiyOSXzKlwKoOl2cZOibURERESRl8xN0DQArZxzLZxzFQD0AvBqaqZFRERElF5lfn/Se7/VOdcfwFsAygEY5b2fnbKZEREREaVRUg9pvfdvAHgjRXMhIiIiypjMVKqlmC46nDBhghlz2223iVhXxDdt2hTa4MGDRXzooYeaMeXKlSvlLNPn119/Ndt0wVnlypXNmJIKvUMF5tu2bRNxqJgzFQWe6aC/H10cDwDLli0T8VdfyQ4Ps2bNMvs0b95cxD179jRjqlatWup5Ron+eevXWqgIuHz58iIuy7mXS3QR8LBhw8wYfR4NHz7cjAnlKYr09wsA7733noifeeYZER944IFmn7PPPlvEoWLxXKFfJ4D9MM7WrVtFHPowTmnOAX0di+r1NkTP/bvvvhPxueeea/bR15x3333XjIlKYTQRERFRzuJNEBEREcUSb4KIiIgolnKyJuinn34S8b///W8z5qKLLhLxKaecIuIHH3zQ7NO7d28RL1iwwIzZZZddSj3PVNHPVV944QUz5o03ZH36E088YcaU9Aw+9Ox/zJgxIu7cubMZ0759exFHpf5DP6Pv16+fGTN27FgR165dW8Sh2rEZM2aIWNdHAOGaiGzTP9/58+ebMa+//rqIp02bJuLQa0KvQ3TXXXeZMbvttpuIc6muQdN5u/76682Yxx57TMSprGFIN3290ecAYOt79Lk1ceJEs0+NGjVEfOqpp5oxUai5DNH1Pa++arvB6GuJrpUL1VzqOtQGDRqYMfo6VqtWreInmyWh72/9+vUivv3220U8c+ZMs88///lPEaf7d27uXomIiIiIksCbICIiIool3gQRERFRLPEmiIiIiGIpJwuj69WrJ2LdqAuwjal0YVuoCFgXRkelmFE35vr888/NmK+//rrYfUpDF7EBwC233CLiq666yozRhdFRsXr1ahGvWLHCjNHfny7WnDdvntnnkksuEXEUGiPqosTCwkIz5uGHHxbxW2+9ZcYcfPDBIj7nnHNErFd4BmzBeajgWhfYt2nTxoyJKp3bTz/9VMShDwIcccQRIs6lQnD9/X700UdmzMqVK0WsrwuLFi0y+zz55JMiPu6448yYbHzwRAsV+H788cci1oXvgP2wjf5Qxc0332z20QXWoeaB+ndXVIWa+D7wwAMinjx5soh1U2MAOP3000WsC8xTLXdemUREREQpxJsgIiIiiiXeBBEREVEsJVUT5JxbCGA9gG0Atnrv81MxqZLohlqhRej0c933339fxFOnTjX76OezFSpUKOsUU0rXHIQaijVq1EjEO+/853+0ejFRwNbRbNiwwYzRuY5Ks8SGDRuKeMiQIWbM9OnTRXz//feLeNKkSWafK664QsRRqG/RDUSvvvpqM0bXyY0ePdqMadmypYh1bUdp9ikoKDBj9ALGuUTXD7788ssi3nfffc0+9evXT+ucMilU89aiRQsR9+3bV8S66SYAPP/88yKOaq1L6Bp35513irh///5mjL4O6Ovgnnvuafa58sorRdy2bVszRtcW6ZrYbNHX/VDjw0cffVTEuu42lMdMLyycisLoQ7z3tgqTiIiIKML4OIyIiIhiKdmbIA/gbefcdOdc39AA51xf51yBc64gl98STyXmxGJOJObDYk4s5sRiTizmJLFkb4K6eu87ATgaQD/nXHc9wHs/wnuf773Pj8qzzGxjTizmRGI+LObEYk4s5sRiThJLqibIe7+06M+VzrnxADoDmJKKif3JeZhtuqBz4MCBIh42bJjZp3HjxqmdWIroRmu77767GfPvf/9bxKG7/WbNmhV7nFAjPE0XwQLRKYTWfvvtNxGHGpw98sgjItbnUseOHc0+egXtTBfQh853XdQ/d+5cM+aVV14RcWmKd/UFM9Tc7I477hDxpk2bzJgoFI+XlS461w3f7r33XrNPVBqtloV+3SxevNiM0YXR+oMZoaaH+voT1RyFzl/dNDV0XdDXQb36+/Lly80+NWrUEHH16tXNmFmzZolYF0oD6W8oGLJu3ToRh36n1q1bV8R///vfRRz6MM7ChQtF3KlTJzOmZs2apZxlycr8TpBzrqpzbpc//h/AEQBmFb8XERERUTQk805QAwDji+5+dwbwnPd+YkpmRURERJRmZb4J8t7PBxDNRaOIiIiISpCTC6jqmojvv//ejNFNmPr06SPiAw44wOwT1UUO9bPmE0880Yx57rnnRKybewHA5ZdfLuI1a9aI+PHHHzf76Gf/Rx55ZInziwr9nHzAgAFmzEUXXSRiXacQarA4e/ZsEXft2tWMSWdOQjVBP//8s4hDi8Xq2g5djwDY+iY9Ri+cCQAffvihiK+55hozplatWmZbFIVyq2uAdJO/Y4891uwT1WtJaeiFh/XPFwD2339/EevmrKGatKg2VdVCtUr6OqhrMAF7bXzqqadEfN9995l9Bg0aJOI99tjDjNE1WtkQamw5duxYEX/wwQdmjF409u233xbx0KFDzT76GqMXXwaAk046ScTJnEu5+0olIiIiSgJvgoiIiCiWeBNEREREsZSTNUG6lqVfv35mzG677SbiU089VcShRUhzRZ06dcy2J554QsT/+te/zBhdE6TrP0ILX7Zr107Eof4fUaXrMpo0aWLG6G3btm0TcaixmF6M98ADDzRj0lnvEKo3Oe2000Qc6vmkn8+H6g90rZzukaPrAACgc+fOIj7rrLPMmLIs6JsNoR4x+rV0zDHHiHjXXXdN65wyTdfzfPfdd2aM7g2lXzfffvut2Uf3GIvqOVGlShWzTfeCeuCBB8yYGTNmiFhfJ0K9bc4880wRhxYP1X1yMlFLpeu3QjVew4cPF3GoVlXXKuo+Y2vXrjX7HHzwwSLeZ599zJhU5oDvBBEREVEs8SaIiIiIYok3QURERBRLvAkiIiKiWIpmZdp2QoWK1157rYhDRVJ33XWXiCtVqpTaiWVRqDBWN/MKNUv89ddfRazz9uijj5p9FixYIOKoFDPqgju9WCFgi5pDedMFgLpQL9Qc8IQTThBxFJq+6e819PPXi+qGild1AaRuVPa///3P7PPee++JONSEMar0z/+bb74xY6ZMkWtC64ZvmV5AN910c75QA0l9XdYfqpg2bZrZRzcmjcq1RAu9nnVx8qhRo8yY9evXi/iMM84Q8eGHH2720UX1UbiWAPYc0IsvA3bx09DrQC9crhdd1QuqAsAtt9wi4gYNGhQ/2STxnSAiIiKKJd4EERERUSzxJoiIiIhiqcSHss65UQCOA7DSe9+2aFttAM8DyAOwEMCp3vufEn2NZIRqgiZMmCDipk2bmjEvv/yyiHWTP/38FgD23ntvEXfs2NGMicozW03PK/R8Vm/Tz31DC/Xpr5uN7z9Uk6DrUHQjQMAuYKibtQHA0qVLRawXyzzuuOPMPnqhxCieE3rxWABo1KhRsTEAtG7dWsSPPPKIiM8++2yzT4cOHUQcxXwkomvJQjUKuhmmbiiZS99vaegmmvrnCwA33nijiHXz2fbt25t9unTpIuJcypuuJwwtsvrRRx+JWDcsDTURjWrTXv2zqV69uhmjXzuhBr26+ePFF18sYt10EwBq165d7FxSrTTvBI0GcJTaNgDAu977VgDeLYqJiIiIckaJN0He+ykAVqvNPQH88bGRMQBOTO20iIiIiNKrrDVBDbz3f3w+bjmAhJ9hc871dc4VOOcK9Ed044o5sZgTifmwmBOLObGYE4s5SSzpwmj/e8GGLdr4/38/wnuf773PDy1GGUfMicWcSMyHxZxYzInFnFjMSWJl7Va1wjnX0Hu/zDnXEIDtKJciVatWNdt00bNuyAQAAwbIMqXly5eL+NBDDzX7hFYGjpNQMXUUihdDc9BFvYsXLzZjdAFzt27dzBhdrKnHtG3b1uyzozTHC33oQK8MvWHDBhFfddVVZp9QkWiu0M3bfvrJfr5j5MiRIg6tMr4jqVu3roj1tRSwDWt1g8x77rnH7BNaRT1X6cazADBkyBAR9+zZU8T169dP65xSSReCh4q6mzVrJuLQ9UR/0ELvU61aNbNPpn/nlPWdoFcB9Cn6/z4AbDtJIiIioggr8SbIOTcWwMcA9nDOLXHOnQ9gEIAezrm5AA4viomIiIhyRomPw7z3vRP81WEpngsRERFRxkRzBbvthJpJ6bqN/fbbz4zZvHmziLds2SJi3cQJsIusRqEeJp30c98999zTjNFNJkOLkGaDbuCmF3AEgM8++0zEoYLA/fffX8RxOgdCi87OmzdPxLrO4S9/+YvZJ5dzpBdnDC38qWuAcvn7LQ39GtfNIgHgnXfeEbHOSahuakfKW6j+RW87/fTTRRzVxoiloX8PAOFGsrkoGr/RiIiIiDKMN0FEREQUS7wJIiIioljiTRARERHFUuQLo0N0gV2oWVsuN3DLlkMOOaTEMVEp7tPzaNGihRkT2kb/X6h53YgRI0SsP0AQlZ9/qujvJ7RadtyFfuahQtk4CX3/L730kohDjQApevhOEBEREcUSb4KIiIgolngTRERERLHkfl8EPkMHc24VgEUA6gIozNiBk5fMfJt77xMu28ucWMyJtF0+kj1GNqQ7J3HKB8CchDAnFnMiJc5HJm+C/u+gzhV47/MzfuAyysR8mZPsHCOVmBMr3fNlPrJzjFRiTizmxErXfPk4jIiIiGKJN0FEREQUS9m6CRpR8pBIycR8mZPsHCOVmBMr3fNlPrJzjFRiTizmxErLfLNSE0RERESUbUm9E+ScO8o5N8c5N885NyBVkyIiIiJKtzK/E+ScKwfgWwA9ACwBMA1Ab+/9V4n2qVu3rs/LyyvT8XLVwoULUVhY6BL9PXNiMSdSHPMBANOnTy9M9LFW5sRiTizmxIpjToq7viazdlhnAPO89/MBwDk3DkBPAAlvgvLy8lBQUJDEIXNPfn7xn+hjTizmRIpjPgDAObco0d8xJxZzYjEnVhxzUtz1NZnHYY0BLN4uXlK0jYiIiCjy0v7pMOdcX+dcgXOuYNWqVek+XE5gTizmRGI+LObEYk4s5sRiThJL5nHYUgBNt4ubFG0TvPcjUPTRtvz8fH4UDdnLia7/2rhxo4h32sneE1eqVCmtc/pDJnJSmvo3PWbTpk0ljqlQoYIZs/POyby0+LoJYU6sqORky5YtIt68eXOJ+5QrV85sq1ixooidS1g6mFBUrq+//vqriEPXV/39pktUzpMoSuadoGkAWjnnWjjnKgDoBeDV1EyLiIiIKL3K/M9V7/1W51x/AG8BKAdglPd+dspmRkRERJRGSb1n771/A8AbKZoLERERUcZw7TAiIiKKpeSqNyNs69atItaFeqGC31Dh2o5k3bp1Ir700ktFfPXVV5t92rdvn9Y5ZdKKFSvMtk8++UTE33zzjYhnzpxp9mnbtq2ITz31VDOmRYsWIt7Rz61c8NNPP5lta9asEXGoiVxJxbmhgvuyFPTmkvXr14t45MiRIn777bdL/Bpdu3Y126699loRZ6pweHuhn+eGDRtEPHu2rfz4/vvvRfz666+LeP/99zf7nH766SKuWrWqGZPucyn0/eoPhJQvX96MCRW2/9ljhT54sm3bNhFXqVLFjEllTnhlJiIioljiTRARERHFEm+CiIiIKJZ2iJqg3377zWx75513RPzee++J+Pbbbzf76Dqh0PNK3QSvLM9FMyGUk/Hjx4tY16nsscceJX6d0LPYKNQ/hL5fXQMyb948M+aRRx4R8V577SXiXr16mX06dOgg4gYNGpgx6cxJ6Bm+bnyp6xMAYNasWSL+7rvvzBhdS6fP9zZt2ph9unTpIuI6deqYMVGoiQrVBD3//PMiDtXF6WaYugne1KlTzT4HHHCAiEN1DVGlz6/CwkIzZuDAgSLWeTzooIPMPu+++66IQw1F9fmXjZqg0LXkhRdeEHHoPPn5559FrOtQX3nlFbOP/j11/fXXmzHt2rUTcag+Jxm63gkAHnjgARGHzt/69ev/6WPp184HH3xgxujzYvjw4WZM5cqV//SxE8n+lYmIiIgoC3gTRERERLHEmyAiIiKKJd4EERERUSzlZGG0Lp4LFVfdd999Ih4wYICIQyt///LLLyJ++umnzRjd3Kp69erFTzZLVq5cabbp72fQoEElfp1nn31WxMcdd5wZU6tWrT85u9TTRYkA8NJLL4n4pJNOMmN0TnbZZRcRR7GgVa/YDQBDhgwR8TPPPGPGLFy4UMShAku9Tb8m9GsPAJo3by5iXSQLAJ07dxZxNorpa9asabbNmTNHxKHVz/W1Qp9rN910k9ln3LhxIg41YYwq/YGQe+65x4yZMGGCiP/xj3+IeLfddjP7fPjhhyI+4YQTzJhsFEJroSL+atWqiTh0vdFN/vr06SNi/ToBgLFjx4r4lFNOMWP0NVh/EAFI7oMHoSLj888/X8ShD1HMnz9fxKHrkn6d77rrriIO/f6sV6+eiEO/q1OJ7wQRERFRLPEmiIiIiGIpqcdhzrmFANYD2AZgq/c+PxWTIiIiIkq3VNQEHeK9t920UkQ/ZwWASZMmifjEE080Y/r27Sti3XAq9OxfP8d//PHHzRhdExQVuuGVrhEBbAM3vTiqrhkBgFtuuUXERx11VBlnmF6rVq0y2/QChq1btzZjdOPD0MK6URNq0KnrK0L1FlqoyaOuD1i7dq2Ip02bZvbRC8g++uijZozOczZqP0J1SLqBYqhRnqbrL0JftzRfJ6r097PvvvuaMfr8atasmYivueYas4+u9QhdS0INFDMt1CRXNxoN1dPpBpG6DjVUJ9WjRw8R699bAHDvvfeKeMyYMWZMqN6ttEL1RI0aNSo2BoBu3br96WPp37v6dzlgG7KmuyExH4cRERFRLCV7E+QBvO2cm+6cs7ewRERERBGV7E1QV+99JwBHA+jnnOuuBzjn+jrnCpxzBaFHFnHEnFjMicR8WMyJxZxYzInFnCSW1E2Q935p0Z8rAYwH0DkwZoT3Pt97n6+fCccVc2IxJxLzYTEnFnNiMScWc5JYmavQnHNVAezkvV9f9P9HALgjZTMrsnz5crNNF/02bdrUjHn55ZdFrFcwDhWpTZ48WcSh1bBDzeIyLVR0+eKLL4r4888/N2P0Ssi6uE+vaAwAnTp1ErFuJhgVoeK5mTNnijjULLFly5YiPvnkk0UcKjhs0aKFiHXBOWDzlMrmgKHvVRcep4r+/kMrTuvvLVRwHYWC1xBdBBt6belV1fVK2KHGebpxXOi6oX+O2WggGaKb0+nCd8A2Y73hhhtE/Mknn5h9/vWvf4m4cePGZZ1iWukGmoCd+/HHH2/GPPjggyLWvz9CxdS66Pzggw82Y8aPHy/i0O/EZAqjM0lfPxYsWGDGXHHFFRmaze+SuTI1ADC+6IW7M4DnvPcTUzIrIiIiojQr802Q934+gPYlDiQiIiKKIH5EnoiIiGIpmg/qtxN61qkXlAstcrlu3ToR33nnnSIeMWKE2efyyy8Xcej5bNWqVRNNNWPWr19vtj3wwAMiHjhwoBmjFzrVORo1apTZRy+MGGqspesoklnMr6xCzbxee+01Ef/nP/8xY0pqkKkXCQRs7cYll1xixgwePFjEUVyItTR0zcxjjz1mxuift66rCo3JhtAcdLPE4cOHmzE1atQQ8RdffCFi3UgPAG677TYRh5qsHn300SLORt1UqFZJX190bR0ADBs2TMSfffaZiO+//36zz5FHHini0M9D119lok5Kn+OvvPKKGaOb/B1zzDFmjC44TlWTP52TXKbrrULXRb3Iarpl/8pERERElAW8CSIiIqJY4k0QERERxVLka4JCNTilqcvRz/qnT58u4lBNUJ8+fUSs+2VExZo1a8w2XbsSepaua4B0X6CvvvrK7KOfl3/44YdmzP777y/ibCxCGvpZ7bXXXiLefffdzRhdz6N7wPzvf/8z++h+UlOmTDFjdK5zpSZIL1isv9dHHnnE7NO/f38Rh3oWRaEHTui6ofvbTJgwwYzRPX/0z7ZatWpmn0MOOUTE+jUCpH9hyFAtib52PPPMM2aM7icWqgnSX0f3ygotrqn723z77bdmTNeuXUWsF/RNBz2vp556yozRC52GFu0u6ecZqr/65ptvRPzxxx+bMT179hRxVPsrhehzUH9/+rUEZP5awXeCiIiIKJZ4E0RERESxxJsgIiIiiiXeBBEREVEsRb4wujQ2btxotulmXnoh0DPOOMPsE9VCaC20QKVuznbPPfeYMbowdOJEudRbw4YNzT5r164V8eGHH27GVKxYMeFc00UX3C1btsyM0QsW1q1b14zRC53q+IgjjjD76KaToULwXFnQUNNFohdeeKGI//KXv5h9rrnmGhFn43wojVAzwhNOOEHEuoEhYM81vXjoX//6V7PPcccdJ+LQuZdq3ntRxD1t2jQzRi/y+eqrrwa/zvYOPPBAM6ZVq1Yi1o1IzzzzTLOPLoINNaPVhdGZ8Msvv4hYX/MA29hRf2AEsAW9+vt98803zT5Dhw4VcfPmzc0Y3bA2VIgfVfpDBXphb92EMhv4ThARERHFEm+CiIiIKJZKvAlyzo1yzq10zs3abltt59wk59zcoj9rFfc1iIiIiKKmNDVBowE8DGD7DlIDALzrvR/knBtQFF+f+umF6eeM//znP82YFStWiFjXCGWiCVe6hGpQ9OKvffv2NWNWrVol4qOOOkrEoUVXdVOwKCyECdi6hVD9g14c9frr7SnaunVrEevGcKEGaHoRwPPPP9+MyYX6Mv06AoDnn39exPqcCTXXC9WS5Qp9PpemnklfO+rUqWPGZKNhqK4JCjXx1PVs55xzjhnTrl07EYdqnnTtnK65DDUZ1WN69eplxmQjb7rG8qCDDjJjdM3T999/b8bo80LX14UWvu7SpYuI7777bjOmWbNmIo5C49HSWr16tYhffvllEZ966qlmn3Q3EdVK/I3mvZ8CYLXa3BPAmKL/HwPgxNROi4iIiCi9yvrP+gbe+z8+jrMcgP24EhEREVGEJf1sw//+XMIuUlPEOdfXOVfgnCvQb63HFXNiMScS82ExJ9b2OSksLMz2dCKB54nFnCRW1pugFc65hgBQ9OfKRAO99yO89/ne+/x69eqV8XA7FubEYk4k5sNiTqztc5KJXkS5gOeJxZwkVtZmia8C6ANgUNGfr6RsRkpoJeSFCxeKWDecAoCRI0eKWBcE5lJxWWnoYrLSNNTSDc90jqJMF7TqVawB4N///reI3377bTNG/+J47733RBxqFKe3NWnSpMT5RZEuWgSAIUOGiPhvf/ubiPfbbz+zTy58r+kUet3oRp2ZsNNOO4ni3CuvvNKM0dfT0HVQN5UsTaHqBRdcIOLQiun664aaV2ZD7dq1RXz//febMXfddZeI33//fTNGNxI9+eSTRXzSSSeVuI9uaAvk9u8qXRz+888/i7hnz55mn0y/dkrzEfmxAD4GsIdzbolz7nz8fvPTwzk3F8DhRTERERFRzijxVtx73zvBXx2W4rkQERERZUy838cmIiKi2IrGQ9k/afr06SLed999zZj9999fxLn8XDVV9PPmsWPHijj0LDZX8hYqCtV1YaWpf9BNJ0vT9DBXcqTppo+AXYhWLwSajWZ2UaPr7fr372/GZKtZ5vbnYiYXso1qvU9p6Nevbk4IAA899JCIQwuo6u9ZnwOhnOTqtaO0dENW3RxSN+YEMp8TvhNEREREscSbICIiIool3gQRERFRLPEmiIiIiGIp8tVroSIp3WDpmGOOMWNK0ywwbnQu9UrQuSx0nlSpUiULM8kdzZs3N9t0s8Tu3buLeEcv5CwNvVp4qKEm5a7QOa4/EMAPCJRO+/btRTxx4kQRR+H3NN8JIiIioljiTRARERHFEm+CiIiIKJZcaIHStB3MuVUAFgGoC6AwYwdOXjLzbe69T7hsL3NiMSfSdvlI9hjZkO6cxCkfAHMSwpxYzImUOB+ZvAn6v4M6V+C9z8/4gcsoE/NlTrJzjFRiTqx0z5f5yM4xUok5sZgTK13z5eMwIiIiiiXeBBEREVEsZesmaESWjltWmZgvc5KdY6QSc2Kle77MR3aOkUrMicWcWGmZb1ZqgoiIiIiyLal3gpxzRznn5jjn5jnnBqRqUkRERETpVuZ3gpxz5QB8C6AHgCUApgHo7b3/KtE+devW9Xl5eWU6Xq5auHAhCgsLE641wJxYzIkUx3wAwPTp0wsTfayVObGYE4s5sTKZk23btom4XLlyGTmuVtz1NZm1wzoDmOe9nw8AzrlxAHoCSHgTlJeXh4KCgiQOmXvy84v/RB9zYjEnUhzzAQDOuUWJ/o45sZgTizmx0pUTfcMDAL/88ouIq1atKuJM3RQVd31N5nFYYwCLt4uXFG0jIiIiiry0fzrMOdfXOVfgnCtYtWpVug+XE5gTizmRmA+LObGYE4s5sZiTxJJ5HLYUQNPt4iZF2wTv/QgUfbQtPz+fH0UDcxKSiZxs2bLFbNu4caOIK1SoIOKKFSuafZxLWM6UMjxHLObEikpOdG3p5s2bzRj9+qtSpYoZs9NOyf+7PB050Y96Fi2yT5v0mJYtW4o4W/UwQGbOk7lz55ptF198sYifffZZETdq1CgdU/lTkjnjpgFo5Zxr4ZyrAKAXgFdTMy0iIiKi9CrzO0He+63Ouf4A3gJQDsAo7/3slM2MiIiIKI2SeRwG7/0bAN5I0VyIiIiIMoZrhxEREVEsJfVOUDr8+uuvIg5Vsuuiu1Chau3atUVcvXp1EaeiAC8O9M/j888/N2M6deok4lAxcbqFmn4uW7ZMxAMHDjRjZs2aJeK6deuKuH///mafrl27ijibBY9/0EWZn376qRkzefLkEr9OtWrVRLzXXnuJWP+sAaBmzZoijkI+KLP0dWLw4MFmzJdffini4cOHmzG77rpraieWIvp3zpAhQ8wY/TvlvPPOE/Hee+9t9ilfvnyJx9bXtkx8MKMsQh88mTp1qojnzZsn4oYNG5p9Mv398U6AiIiIYok3QURERBRLvAkiIiKiWIpcTdALL7wg4nPPPdeM0c9IQzUoum5DN2mqVy+4tlysbd261Wx74w354b/HH3/cjHn++edFnI2aoE2bNplt9913n4jr1KljxowaNUrEs2fLLg9XX3212Wfo0KEiPuCAA8yYTD/X/u2330T87bffmjFPPvmkiEM/p/3331/E+ucfeu5/zjnniLhXr15mTDbOiVQpaZHp0N/r8zF0fuo6mlq1apkxUc2b/p5nzpwpYv0aAey1XNeSRZl+Pe+8s/3V+eKLL4r43XffFbG+TgJAu3btSjx2aRYhjUKdkK7DBezvlGeeeUbEXbp0MfvohrXpxneCiIiIKJZ4E0RERESxxJsgIiIiiiXeBBEREVEsRa4wukWLFiIOFabqBm66wRsAvPbaayJ+5ZVXRKwbWQHxa6Coi9Zef/11M6Zv374iPuuss8yYTBeyhYSKU3XB8rHHHmvG6JWs8/LyRLxu3Tqzj27yts8++5gxlSpVSjjXdNBN10I/pxNOOEHEofNd50MX9E6bNs3so1+jOocA0L17dxFHoZAzJFT4/f7774t4wYIFIl6yZInZRzd5Xbp0qRmzceNGEYc+dNCsWbPEk80ifV6MGTOmxH369Okj4qgWfYfoa9wNN9xgxtSoUUPEjz76qIgXLlxo9mnTpo2IQwXXUW0+qq+506dPN2N22WUXEevXiv5wAMDCaCIiIqKM4E0QERERxVJSj8OccwsBrAewDcBW731+KiZFRERElG6pqAk6xHtfmIKvA8DWcejmbYCtZQjVgyxfvrzYMVGtSUgnnYMZM2aIOFQn1bhxYxFfe+21ZkwUnu2HanBOOukkEZfm2bo+Lw499FAzRtcDrFmzxozJ9kKQesFHAFi/fr2IdYNFwNayzJ8/X8RfffWV2Wf16tUiDtU+6JqgqNCviTlz5pgxDzzwgIj19xJq+KZrPXRtRIhe5DnK9M/4pZdeEvF+++1n9mnZsqWIc+karF8rofrJRx55RMT6uvDPf/7T7PPDDz+IOFTLV7Vq1VLOMrP03O+8804z5uabbxaxbqqpF7kGMv864OMwIiIiiqVkb4I8gLedc9Odc31LHE1EREQUEcneBHX13ncCcDSAfs458563c66vc67AOVeg32qPK+bEYk4k5sNiTizmxGJOLOYksaRugrz3S4v+XAlgPIDOgTEjvPf53vt8Llr6O+bEYk4k5sNiTizmxGJOLOYksTIXRjvnqgLYyXu/vuj/jwBwR7ITKkvDwlBDuw8//FDE1113nYhzqSgvVXQhmy5yDq3q/NRTT4m4QYMGKZ9XKoR+nqHGY39WqJg6F5pqTp061Ww77rjjRBwqntYFvU2bNhVx6HsfMGCAiE8++WQzJqqvN10sfvfdd5sxRx99tIh1A9EofDAgnUINJPVq4CtWrBDx6aefbvbRjThziT5/N2zYYMbo86BJkyYi1h9EAYDZs2eLeLfddjNjDj/88GLnkgmha8WwYcNErJsYA8AFF1wg4kGDBok4VBi9xx57lGWKZZbMb4kGAMYX/UB2BvCc935iSmZFRERElGZlvgny3s8H0D6FcyEiIiLKmOi/r09ERESUBpFbQLU0tm3bJmL9fDo0Rtc2hJ7p6oZYoWfh+tloqLFT5cqVzbZMCy1Md99994n4vffeE/G4cePMPu3atRNxLtTDpJJuBAgAhxxyiIhDtVTZ1qFDB7PtsssuE/GkSZPMGF2TcMwxx4j4iCOOMPs0atRIxLmy4CMA/Pe//xVx6DWgGwP++OOPIv773/9u9sl2s8xk6DzNmzfPjHniiSdErBcRPuyww8w+uXzt0Oe0XgwWsAs0698DixcvNvvoRpyffPKJGaOvN6modfyzQj+7c889V8Shc143CdWLna9duzYFs0tO7p6VREREREngTRARERHFEm+CiIiIKJZysiZIP5MfOHCgGaOfNerFNEM1M7pnyHfffWfGlC9fXsSjR482Y0455RSzLd309/Pwww+bMbp2StcInXDCCWafXHmOH6r32Lhxo4hD/aT0Ioe6vidUI6LrZKLYJybUEO2ee+4RsV4IFgCmTZsm4pdfflnEn332mdlHn++hxVKjmCMAOPDAA0X89ddfmzG6ruGWW24Rsa6PAWxuc+V1BNhayDFjxpgxK1euFLFeKLN+/fqpn1iGhBYW1jWkuuYUsDUxuo5o7ty5Zp8vvvhCxHvvvXdpp5lRoTqk1q1bl7ifrqHVveo6duxY4tcI1ebq3Cbz+sqdVyYRERFRCvEmiIiIiGKJN0FEREQUS7wJIiIioljKycJoXcC8adMmM+bnn38W8YIFC0QcauhWoUIFEZ955plmzL777ivirl27Fj/ZDJk5c6aIdfEmAFx00UUi7t+/v4grVaqU+olliP75AsCFF14oYt2oC7ANzd555x0R6yJ8AHj77bdFHCqSzEZDs5Loov7atWubMboZ4kEHHSTib775xuxz5ZVXiji0KKJ+LWWiUHjRokUi1j9bAOjWrZuIQ81Pv//+exHr5oknnnii2SeqC8aWhr6efvTRR2aMLgI+/vjjRRzF87+09AcqAHuOFxQUmDG6qFl/GOC1114z++jrj16sF4hG89FQsbgujte/lwF7vXj99ddFXLVq1RKPrT+sAQBXXHGFiGvVqlXi10mE7wQRERFRLPEmiIiIiGKpxJsg59wo59xK59ys7bbVds5Ncs7NLfqz7O9FEREREWVBaR7cjgbwMICntts2AMC73vtBzrkBRfH1qZ9eWLNmzUT8v//9z4zRzzB1E7xQTYJ+jq9rhABbV5GNZ/+hRo+PPfaYiHUTOAC49tprRZzLNUBaqN5D16Y8/fTTZszkyZNFrHMbWjD0yy+/FLGuEwOyv6hqqK5BNy4LNTDcunWriHV9iP4agH0tTZ8+3Yzp3bt3sfukQ5UqVUoc8/jjj4tY1/sAQN26dUWs6xF69Ohh9snlmqA5c+aI+PPPPzdjdI1XqDlnripNDc7s2bPNNt34UP/+yM/PN/vcdtttIg41S4zCuRS6nui6U10rCdjrYIsWLUQ8a9YsaLpBa6juNpXNV0u8EnnvpwDQS2n3BPBHG9ExAE5M2YyIiIiIMqCs/xxr4L3/45/ZywE0SNF8iIiIiDIi6fek/e+LNtmFm4o45/o65wqccwWrVq1K9nA7BObEYk4k5sNiTizmxGJOLOYksbLeBK1wzjUEgKI/VyYa6L0f4b3P997n70jPjZPBnFjMicR8WMyJxZxYzInFnCRW1o5WrwLoA2BQ0Z+vpGxGpaCLk5s2bZrJw2fdvHnzzLb58+eLeMiQIWZMw4YN0zanbNMruwO2QDeUk06dOol45MiRItZFsYAtHo7iCuEzZsww23QhY6jB2JIlS0S8Zs0aEa9du9bsc/DBB4v44osvNmOy0TxPX+zPO+88M0avUB0q/NZz10WZUShcTSXd/HOvvfYyY84++2wRp7JQNdtC38udd94p4iOPPNKM0c0DGzSQVSKhAl99jkbxWgLYpo4A8MADD4i4NK8d/bs7RH+d0AeUQvMpq9J8RH4sgI8B7OGcW+KcOx+/3/z0cM7NBXB4UUxERESUM0r855n3vneCvzosxXMhIiIiyphovvdGRERElGa5u8pdjOlnzQAwatQoEYfqpHa02oXtNW7c2GwbNmyYiEML7eqGeqWpXSnNc+1sa9eundl28803i1g3fQSAOnXqiLhRo0Yi1o1KAbuYZqimIgrnXmgOut4gVH8QN/vss4+IJ02aZMbomowo/HxTJfS96HP8lFNOydR0IiFUqxRakDoVUlnvUxp8J4iIiIhiiTdBREREFEu8CSIiIqJY4k0QERERxRILo3NQqONn3LuAhooZdYHujtTQrSShFdS7detWbEwE2A8H7LLLLlmaCVH68Z0gIiIiiiXeBBEREVEs8SaIiIiIYsl57zN3MOdWAVgEoC6AwowdOHnJzLe59z5hwQ5zYjEn0nb5SPYY2ZDunMQpHwBzEsKcWMyJlDgfmbwJ+r+DOlfgvc/P+IHLKBPzZU6yc4xUYk6sdM+X+cjOMVKJObGYEytd8+XjMCIiIool3gQRERFRLGXrJmhElo5bVpmYL3OSnWOkEnNipXu+zEd2jpFKzInFnFhpmW9WaoKIiIiIsi2pd4Kcc0c55+Y45+Y55wakalJERERE6Vbmd4Kcc+UAfAugB4AlAKYB6O29/yrRPnXr1vV5eXllOl6uWrhwIQoLC+2aDkWYE4s5keKYDwCYPn16YaKPtTInVpRy8ttvv5ltO+2UnuqLXMlJJjEnUnHX12TWDusMYJ73fj4AOOfGAegJIOFNUF5eHgoKCpI4ZO7Jzy/+E33MicWcSHHMBwA45xYl+jvmxMpmTvRNz8aNG82YypUrizhVN0VRzUk2MSdScdfXZM7CxgAWbxcvKdomOOf6OucKnHMFq1atSuJwOw7mxGJOJObDYk4s5sRiTizmJLG0fzrMez/Ce5/vvc+P+0rnf2BOLOZEYj4s5sRiTizmxGJOEkvmcdhSAE23i5sUbSMiio1Q/Yt+HFSpUiUzply5cmmbU7rpWtKFCxeK+I033jD7XHDBBSIO5SSXbd26VcRbtmwRsX4cSNGQzDtB0wC0cs61cM5VANALwKupmRYRERFRepX5nSDv/VbnXH8AbwEoB2CU9352ymZGRERElEbJPA6D9/4NAPZ9TyIiIqKI49phREREFEtJvROULdu2bRPxr7/+asboojQtVJRXoUIFEaeruVc6rF+/XsQzZ840Y9asWSPiH374QcQtWrQw++y1114irlOnjhlTvnz50k4zqzZt2mS26Tzp/hkdO3Y0++ieE7lc4Ep/3ubNm0X8wQcfmDGDBg0S8eDBg82YDh06iNi5hP1DI0e/lp555hkRH3nkkWafihUrpnVO2TZ7tqwGGTlypIgffPBBs0+uXDt3ZLnzW56IiIgohXgTRERERLHEmyAiIiKKpcjVBOl6n6VLbf/Fl19+WcQTJ040Y77++msR77yz/Fa7dOli9jnppJNEfOyxx5oxUWh4tW7dOrNt6NChIv7www/NmAMPPFDETZs2FfEnn3xi9hk3bpyIQzUy5513noij0gRN12488cQTZsznn38u4mXLlolYn0eAzUGma4JCix6H6uI0XfOWqnnrZoGh2pZcqnfR9Hmkaz1uuOEGs49+rel9AGDgwIEirlGjRlmnmFah823GjBki1rUtut4JyO1zoDSmTJki4v/85z8iHjJkiNknqjVB+vfw1KlTzZhFi+TyZO3btzdjqlWrJmJdh6prWQFg9913F3HLli3NmFTW6/KdICIiIool3gQRERFRLPEmiIiIiGKJN0FEREQUS1ktjNbFVwDwzjvviPiKK64o8et069bNbDvllFNErFd1fv/9980+/fr1E/E333xjxuiVkHfdddcS55dqS5YsMdvWrl0r4lGjRpkx9erVE7EuFg81mNQFcRdffLEZ0717dxG3bdvWjMkGXbg3efJkM0Y3MHvuuedEPGfOHLNPqFA0k37++Wez7ZprrhFxqHi+U6dOIj744IPNGF0srwsQFyxYYPaZNm2aiDt37mzG6AaTUW1EGvrZfvvttyK+//77RXzyySebfa677joR33PPPWaMfh1HtTBaN1kFbNHvueeeK+IdvTFiaegPK2T7uvFnbN26VcT6AzIA8Nhjj4m4Zs2aZswuu+wi4kaNGolYfxAFAPbcc08R6w9CAUCVKlXMtrKK5pWIiIiIKM14E0RERESxlNTjMOfcQgDrAWwDsNV7n1/8HkRERETRkIqaoEO894Vl2fH7778322688UYR77333maMXpxQP2cE7DNp/Tz2jDPOMPvo59q33HKLGdOmTRsRh+oB0i0vL89s+9vf/ibizz77zIxp3LixiHfbbTcRz5071+xz1113ibhdu3ZmTIMGDRLONZv0Yq+hJo4TJkwQ8fz580UcOv+yvWDqL7/8YrbpRogHHXSQGfPxxx+L+JVXXjFjdJ2erg0oLLQv9YULF4r4r3/9qxnz9NNPizgKTUdD9PcLAGPHjhXxqaeeKuLLLrvM7DNixAgR6xwB4etWFOhz4K233jJj9tlnHxHra1Ko/kXXHIaaJ0a1eWBp6GvjihUrRKybbgLRfR3o64mugwOA2267TcShn7n+OvrnO3z4cLOPrjdLNz4OIyIiolhK9ibIA3jbOTfdOdc3NMA519c5V+CcK1i1alWSh9sxMCcWcyIxHxZzYjEnFnNiMSeJJXsT1NV73wnA0QD6Oee66wHe+xHe+3zvfb7+iHZcMScWcyIxHxZzYjEnFnNiMSeJJXUT5L1fWvTnSgDjAdgGIUREREQRVObCaOdcVQA7ee/XF/3/EQDu+DNfI1RwpxsUPvTQQ2ZMixYt9Fz+zGET7qObPYWal+lGTtkQKkCbOXOmiEOF0a+//rqI69evL2JdyAcAhxxyiIjvu+8+M6Zu3bqJJ5tFtWrVEvHtt99uxpx++uki1nkLNQnTq6Znmi74BmwBu25SBgDnnHOOiEMNFfUqz7rhW+h71w1EW7dubcbkSsFrqHhVF8sfcMABIv7HP/5h9tFF588++6wZU7169bJMMe02bNggYr06OmC/502bNon4o48+MvvoAvM99tjDjOnfv7+Io1o4HDJp0iQR698nudRAUv9+DDUnLEvDQv3Bg1DjX93UVRdXp1oynw5rAGB8UbJ2BvCc935iSmZFRERElGZlvgny3s8H0D6FcyEiIiLKGH5EnoiIiGIpqwuo6noDAKhataqIQ7UNWqhOQTf8+vHHH0V86623mn2eeuopEeuaEiAatQ06RwBw1llniTj0vP21114T8VdffSXiUD1Ew4YNRVy7dm0zpiw1WZmg5xVq6qibvHXt2lXEuukdYGugDj300BKPnUqhc7A0i2/qZpGh5pG6TkzXn4Xq+PSCrscff7wZoxfrzYZQLZ0+5/VrAgDefPNNEes6sd13393s88QTT4i4R48eZky2m24mopvYhhpk6gWpdWPZUA2UXiwztBhzLpsxY4aIdTPMKLwGsm3lypUinjjRVtA888wzIk7364TvBBEREVEs8SaIiIiIYok3QURERBRLWX1I2b59yR8u69evn9l2/fXXizi0oKReLPLDDz8UcaiuomPHjiIO9ScI1cRkWqi2QS9+etVVV5kxemFLXUekax8A+3w2tDhmly5dRBzVGqFFixaZbboGRvcS0ucNADz55JMi1t8/EK7bykW6bu/55583Y4444ggRhxbZzQZdK6h7kAHA0KFDRfzCCy+U+HWGDRsm4tNOO83so+vGdtopd/69qc/d5cuXmzFnn322iHWfslA/tZtvvlnEZ555phkTqlOLIr0YLAAsXrxYxHoR4zjWBOnXzquvviriUI+5vfbaS8Tp/n2SO69MIiIiohTiTRARERHFEm+CiIiIKJZ4E0RERESxlNVKLb0QIQDcfffdIr7//vvNmL///e8iDjVTatWqlYh1A7c+ffqYffTioLq5FxCNBf1CTSYHDhwoYr3ILADcc889ItaNKEMFrWvXrhVxaMG7qNIF5O+//74Zoxf71EWh++23n9nnscceE7HOUejr5Aqds3nz5ol4+vTpZp9Ro0aJOCrFrV9//bWIe/bsacbsuuuuIr7wwgvNGP396OtPFBqoplK9evVE3Lx5czNGnyc33nijiEPXdt3QM6rNIktDN+MFgNWrV4u4ZcuWmZpOZOmc6OazuskmAFSrVi2tc9L4ThARERHFEm+CiIiIKJZKvAlyzo1yzq10zs3abltt59wk59zcoj/tIltEREREEVaamqDRAB4GsP3qogMAvOu9H+ScG1AUXx/Yt1gVK1Y023StzjHHHGPG6OaIoWfLuglTaWp5dDMrXQ8B2MUis1EjFGoepRs7hhZD1XTDL93sK2Tvvfcu1XyiSNd/AHZxSF3rEKoL043volAnlir6nNCLCh9++OFmH31OROV80Iuhdu7c2YzRtXRTp041Y9atWyfiHb3pnX5NhGoQdYPIww47TMQ70msiJFQTtGrVKhHXrFkzQ7OJLv160o03u3XrZvbJ9PWjxHeCvPdTAKxWm3sCGFP0/2MAnJjaaRERERGlV1lrghp47//4J/JyAA0SDXTO9XXOFTjnCvSdclwxJxZzIjEfFnNiMScWc2IxJ4klXRjtf39+YBez+v9/P8J7n++9z9cfvYwr5sRiTiTmw2JOLObEYk4s5iSxst4ErXDONQSAoj9Xpm5KREREROlX1gq/VwH0ATCo6M9XUjUh3XisUaNGqfrSQqiwrUOHDiLWq4UDwOzZs0V88MEHp3JapRJa3b5Xr14iPvfcc82Yq6++WsSNGzcWcagRnv46f/nLX0o9z2zTBXadOnUyY+644w4R66L7L774wuyjc129evUyzjB65s6dK2Jd2KhXUAfC52MUtG/fXsSvvGIvU7p5m359A8ANN9wg4qgUfqeLvi7oZpiAbSC5ozWMLIleHR2wDWq7dOmSqelE1tKlS4v9+yicN6X5iPxYAB8D2MM5t8Q5dz5+v/np4ZybC+DwopiIiIgoZ5T4TpD3vneCvzoswXYiIiKiyGPHaCIiIoqlHbvrVzFCDRZPOOEEEb/00ktmzA8//JC2OZWWbtYH2Nqk559/3oz58ssvRbzbbruJOLSorK4PyOVGcU2bNjXbrrjiChFPmzZNxNdcc43ZZ4899hBxri4EGWqoOWbMGBHruoZcapapz+/hw4ebMbo2MPSzzPSCjtmmX+N6oWUKL5D8ySefiLhKlSqZmk5k6XPnp59+EnFoQe42bdqkdU4a3wkiIiKiWOJNEBEREcUSb4KIiIgolngTRERERLGUu1WuaaCb3ukVtIHoNoarWLGiiEOr84a2xUmo6LVdu3bFxjuyjRs3mm0zZ84U8Y033ihifZ5Fmf5516hRI0szoR1N6MMpO1LT1FQ59thjRXzrrbeKeOvWrZmcThDfCSIiIqJY4k0QERERxRJvgoiIiCiWnPc+cwdzbhWARQDqAijM2IGTl8x8m3vv6yX6S+bEYk6k7fKR7DGyId05iVM+AOYkhDmxmBMpcT4yeRP0fwd1rsB7n5/xA5dRJubLnGTnGKnEnFjpni/zkZ1jpBJzYjEnVrrmy8dhREREFEu8CSIiIqJYytZN0IgsHbesMjFf5iQ7x0gl5sRK93yZj+wcI5WYE4s5sdIy36zUBBERERFlGx+HERERUSzxJoiIiIhiKeM3Qc65o5xzc5xz85xzAzJ9/JI450Y551Y652Ztt622c26Sc25u0Z+1Uni8SOcDYE5CmBMp0/ko+vrMiT0mc2KPyZzI40U6H0Bmc5LRmyDnXDkAwwEcDaANgN7OuTaZnEMpjAZwlNo2AMC73vtWAN4tipOWI/kAmJOQ0WBOtjcaGcoHwJyEMCcWcyLlSD6ADOYk0+8EdQYwz3s/33u/GcA4AD0zPIdiee+nAFitNvcEMKbo/8cAODFFh4t8PgDmJIQ5kTKcD4A5CWFOLOZEinw+gMzmJNM3QY0BLN4uXlK0LeoaeO+XFf3/cgANUvR1czUfAHMSwpxI6coHwJyEMCcWcyLlaj6ANOWEhdF/kv+9pwD7CmyHObGYE4n5sJgTizmxmBMrlTnJ9E3QUgBNt4ubFG2LuhXOuYYAUPTnyhR93VzNB8CchDAnUrryATAnIcyJxZxIuZoPIE05yfRN0DQArZxzLZxzFQD0AvBqhudQFq8C6FP0/30AvJKir5ur+QCYkxDmREpXPgDmJIQ5sZgTKVfzAaQrJ977jP4H4BgA3wL4DsCNmT5+KeY3FsAyAFvw+/PS8wHUwe/V6HMBvAOgdlzywZwwJ1HMB3PCnDAnO2Y+Mp0TLptBREREscTCaCIiIool3gQRERFRLPEmiIiIiGKJN0FEREQUS7wJIiIioljiTRARERHFEm+CiIiIKJb+H/6TqYPn3/J7AAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 720x720 with 128 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "imgs, cls_idxs = next(iter(loader_train))\n",
        "\n",
        "fig = plt.figure(figsize=(10., 10.))\n",
        "grid = ImageGrid(fig, 111, nrows_ncols=(8, 8), axes_pad=0.1)\n",
        "for i, img in enumerate(imgs):\n",
        "    grid[i].imshow(img[0], cmap='Greys')\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "586d679e-8069-460a-8ed3-152d8f53480b",
      "metadata": {
        "id": "586d679e-8069-460a-8ed3-152d8f53480b"
      },
      "source": [
        "---\n",
        "\n",
        "## 2 Image classification with a Vision Transformer (ViT)\n",
        "\n",
        "We will now implement a simple [ViT](https://arxiv.org/abs/2010.11929) and perform digit classification on MNIST.\n",
        "\n",
        "![Vision Transformer](https://production-media.paperswithcode.com/methods/Screen_Shot_2021-01-26_at_9.43.31_PM_uI4jjMq.png)\n",
        "\n",
        "Remember that a ViT is a Transformer encoder, which consists of several stacked blocks of self-attention and MLPs. \n",
        "We will first implement the MLP and self-attention, which can then be assembled into a Transformer encoder block.\n",
        "Afterwards, we will look at positional embeddings and the patching operation to turn images into sequences of tokens.\n",
        "\n",
        "![Transformer encoder](https://www.researchgate.net/publication/334288604/figure/fig1/AS:778232232148992@1562556431066/The-Transformer-encoder-structure.ppm)\n",
        "\n",
        "Please consult the [Attention Is All You Need](https://arxiv.org/abs/1706.03762) and [ViT](https://arxiv.org/abs/2010.11929) papers, as well as the reading material at the bottom of this notebook and the course slides, should you require further details."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9e887046-fe97-4bce-bc88-a6c91596702d",
      "metadata": {
        "id": "9e887046-fe97-4bce-bc88-a6c91596702d"
      },
      "source": [
        "### 2.1 MLP layer (10 pts)\n",
        "\n",
        "In the following cell, implement the following two-layer Perceptron:\n",
        "\n",
        "$$ \\text{MLP}(X) = \\text{GeLU}(X W_1^T + b_1) W_2^T + b_2 $$\n",
        "\n",
        "Here, $\\text{GeLU}$ denotes the [GeLU activation function](https://pytorch.org/docs/stable/generated/torch.nn.GELU.html). \n",
        "The first linear layer projects x from dimension `dim` to `int(mlp_ratio * dim)`, while the second projects it back to `dim`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "503072a7-3b35-4bc7-bdab-25a9d60d4459",
      "metadata": {
        "id": "503072a7-3b35-4bc7-bdab-25a9d60d4459"
      },
      "outputs": [],
      "source": [
        "class Mlp(nn.Module):\n",
        "    def __init__(self, dim, mlp_ratio=4.):\n",
        "        super().__init__()\n",
        "        \n",
        "        # TODO\n",
        "        self.gelu_ = nn.GELU()    \n",
        "        self.linear_1 = nn.Linear(dim, int(mlp_ratio * dim))\n",
        "        self.linear_2 = nn.Linear(int(mlp_ratio * dim), dim)    \n",
        "\n",
        "    def forward(self, x):\n",
        "        \n",
        "        # TODO\n",
        "        return self.linear_2(self.gelu_(self.linear_1(x)))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "28f8904a-09de-4792-b17d-b2faaf71f071",
      "metadata": {
        "id": "28f8904a-09de-4792-b17d-b2faaf71f071"
      },
      "source": [
        "### 2.2 (Masked) self-attention layer (20 pts)\n",
        "\n",
        "Next, we ask you to implement a layer that performs (optionally masked) multi-headed self-attention.\n",
        "\n",
        "Remember the scaled dot-product attention formula seen in the course:\n",
        "$$ \\text{Attention}(Q, K, V) = \\text{softmax}_\\text{row} \\left( \\frac{Q K^T}{\\sqrt{d_k}} \\right) V $$\n",
        "\n",
        "The queries $Q$, the keys $K$, and the values $V$ are all linear projections of $X$:\n",
        "\n",
        "$$ Q(X) = X W_Q^T $$\n",
        "$$ K(X) = X W_K^T $$\n",
        "$$ V(X) = X W_V^T $$\n",
        "\n",
        "The scaling factor $d_k$ is the dimensionality of the keys $K$, i.e. `dim // num_heads`.\n",
        "\n",
        "The attention is performed on `num_heads` heads in parallel (don't use a for loop) in `dim // num_heads`-dimensional subspaces and the results are concatenated along the feature dimension.\n",
        "\n",
        "In addition, we want to enable masking of the attention matrix, e.g. for implementing a Transformer decoder.\n",
        "For this, the forward function takes an additional `mask` argument, specifying where to zero-out the attention matrix.\n",
        "In practice, this is implemented by replacing the values of the attention matrix (pre softmax) to minus infinity wherever we don't want any attention flow."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6f35b40e-b6e8-4875-a078-a090e6793536",
      "metadata": {
        "id": "6f35b40e-b6e8-4875-a078-a090e6793536"
      },
      "outputs": [],
      "source": [
        "class Attention(nn.Module):\n",
        "    def __init__(self, dim, num_heads=8):\n",
        "        '''\n",
        "        Self-attention layer.\n",
        "        \n",
        "        params:\n",
        "            :dim: Dimensionality of each token\n",
        "            :num_heads: Number of attention heads\n",
        "        '''\n",
        "        super().__init__()\n",
        "        self.num_heads = num_heads\n",
        "        head_dim = dim // num_heads\n",
        "        self.scale = head_dim ** -0.5\n",
        "\n",
        "        # TODO: Define here the linear layers producing K, Q, V from the input x\n",
        "        self.query = nn.Linear(dim, num_heads * head_dim)\n",
        "        self.key = nn.Linear(dim, num_heads * head_dim)\n",
        "        self.value = nn.Linear(dim, num_heads * head_dim)\n",
        "\n",
        "        # Projection\n",
        "        self.proj = nn.Linear(num_heads * head_dim, dim)\n",
        "\n",
        "    def forward(self, x, mask=None):\n",
        "        '''\n",
        "        Performs a forward pass through the multi-headed self-attention layer.\n",
        "        \n",
        "        params:\n",
        "            :x: Input of shape [B N C]. B = batch size, N = sequence length, C = token dimensionality\n",
        "            :mask: Optional attention mask of shape [B N N]. Wherever it is True, the attention matrix will\n",
        "            be zero.\n",
        "            \n",
        "        returns:\n",
        "            Output of shape [B N C].\n",
        "        '''\n",
        "        B, N, C = x.shape\n",
        "        \n",
        "        # TODO: Compute the keys K, queries Q, and values V from x. Each should be of shape [B num_heads N head_dim].\n",
        "        # q, k, v = self.query(x), self.key(x), self.value(x)\n",
        "        q = rearrange(self.query(x), \"B N (num_heads head_dim) -> B num_heads N head_dim\", num_heads=self.num_heads)\n",
        "        k = rearrange(self.key(x), \"B N (num_heads head_dim) -> B num_heads N head_dim\", num_heads=self.num_heads)\n",
        "        v = rearrange(self.value(x), \"B N (num_heads head_dim) -> B num_heads N head_dim\", num_heads=self.num_heads)\n",
        "\n",
        "        # print(\"q.shape:\", q.shape)\n",
        "\n",
        "        # breakpoint()\n",
        "        # TODO: Compute the attention matrix (pre softmax) and scale it by 1/sqrt(d_k). It should be of shape [B num_heads N N].\n",
        "        # Take the dot product between \"query\" and \"key\" to get the raw attention scores.\n",
        "        attn = torch.matmul(q, k.transpose(-1, -2)) * self.scale\n",
        "\n",
        "        if mask is not None:\n",
        "            # TODO: Apply the optional attention mask. Wherever the mask is True, replace the attention matrix value by negative infinity → zero attention weight after softmax.\n",
        "            mask = rearrange(mask, \"b n1 n2 -> b 1 n1 n2\")\n",
        "            attn = attn.masked_fill(mask, float('-inf')) # Replace masked values with negative infinity\n",
        "\n",
        "        # TODO: Compute the softmax over the last dimension\n",
        "        attn = nn.functional.softmax(attn, dim=-1)\n",
        "\n",
        "        # TODO: Weight the values V by the attention matrix and concatenate the different attention heads\n",
        "        x = torch.matmul(attn, v)\n",
        "        x = rearrange(x, \"B num_heads N head_dim -> B N (num_heads head_dim)\") # concatenate different attention heads\n",
        "\n",
        "        # One final projection\n",
        "        x = self.proj(x)\n",
        "        return x"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b7e812c7-9512-45aa-8e6f-f44d273b3169",
      "metadata": {
        "id": "b7e812c7-9512-45aa-8e6f-f44d273b3169"
      },
      "source": [
        "### 2.3 Open ended question on attention (10 pts)\n",
        "\n",
        "#### Question:\n",
        "What are the main differences between self-attention and convolutions?\n",
        "\n",
        "#### Answer:\n",
        "\n",
        "1- In convolutions with a $ K\\times K $ kernel size, the receptive field is limited only to a $ K \\times K$ neighborhood grid. In contrast, in self attension, the receptive field is the full image, since the values of every other pixel in the image, has a contribution to the new value of a pixel.\n",
        "[reference: https://doi.org/10.48550/arXiv.1911.03584]\n",
        "\n",
        "2- The self attention, inherently does not care about the ordering of the sequence and will always produce the same output, regardless of the order of the elements in the input sequence. This is why we add positional embeddings to the input sequence which are learned throughout training. [reference: https://doi.org/10.48550/arXiv.1911.03584]\n",
        "\n",
        "3- CNNs are typically used for inputs that have an inherent spatial relationship in them, such as images. However, self-attentions can be used for other types of inputs that do not have this property, such as texts.\n",
        "\n",
        "4- It is easier to interpret the weights of CNN as they are designed to learn the local (kernel neighborhood) relationships in the images. "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "018d9e95-8d26-4225-9a81-ee6fec7d694e",
      "metadata": {
        "id": "018d9e95-8d26-4225-9a81-ee6fec7d694e"
      },
      "source": [
        "### 2.4 Transformer encoder block (10 pts)\n",
        "\n",
        "In the next cell, implement a Transformer block. It is defined as:\n",
        "\n",
        "$$ X_a = X + \\text{Attention}(\\text{LN}(X)) $$\n",
        "$$ X_b = X_a + \\text{MLP}(\\text{LN}(X_a)) $$\n",
        "\n",
        "Here, $\\text{LN}$ denotes two LayerNorm layers.\n",
        "\n",
        "Don't forget to pass the optional mask to the self-attention layer!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1ce5f348-c986-49ca-9bd0-980f61afdbd9",
      "metadata": {
        "id": "1ce5f348-c986-49ca-9bd0-980f61afdbd9"
      },
      "outputs": [],
      "source": [
        "class Block(nn.Module):\n",
        "    def __init__(self, dim, num_heads, mlp_ratio=4.):\n",
        "        '''\n",
        "        Transformer encoder block.\n",
        "        \n",
        "        params:\n",
        "            :dim: Dimensionality of each token\n",
        "            :num_heads: Number of attention heads\n",
        "            :mlp_ratio: MLP hidden dimensionality multiplier\n",
        "        '''\n",
        "        super().__init__()\n",
        "\n",
        "        # TODO\n",
        "        self.attention = Attention(dim, num_heads)\n",
        "        self.mlp = Mlp(dim, mlp_ratio)\n",
        "        self.layernorm_1 = nn.LayerNorm(dim)\n",
        "        self.layernorm_2 = nn.LayerNorm(dim)\n",
        "\n",
        "\n",
        "    def forward(self, x, mask=None):\n",
        "        '''\n",
        "        Performs a forward pass through the multi-headed self-attention layer.\n",
        "        \n",
        "        params:\n",
        "            :x: Input of shape [B N C]. B = batch size, N = sequence length, C = token dimensionality\n",
        "            :mask: Optional attention mask of shape [B N N]. Wherever it is True, the attention matrix will\n",
        "            be zero.\n",
        "            \n",
        "        returns:\n",
        "            Output of shape [B N C].\n",
        "        '''\n",
        "        # TODO\n",
        "        # Multi-Headed Self-Attention\n",
        "        x = self.attention(self.layernorm_1(x), mask) + x\n",
        "        \n",
        "        # Feedforward MLP\n",
        "        x = self.mlp(self.layernorm_2(x)) + x\n",
        "\n",
        "        return x\n",
        "        "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c7cc9e29-9fa1-4e93-90cf-317a9fb84779",
      "metadata": {
        "id": "c7cc9e29-9fa1-4e93-90cf-317a9fb84779"
      },
      "source": [
        "### 2.5 Patch embedding layer (10 pts)\n",
        "\n",
        "So far, we implemented everything needed to build a vanilla Transformer encoder and we could more or less directly use it to train e.g. a [BERT](https://arxiv.org/abs/1810.04805) model.\n",
        "If we want to use it for images, we first have to turn these images into sequences of tokens (since that's what Transformers operate on).\n",
        "If we were to do this naively and simply flatten an HxWxC image into (H * W) C-dimensional tokens, we quickly run into difficulties...\n",
        "Even moderately-sized images would result in massive sequence lengths. Let's say our images have size 224x224. Then, our sequence length would be 50'176. Given how the cost of self-attention scales quadratically with the sequence length, we need to find another solution.\n",
        "\n",
        "We can address this problem by spending a little bit of compute and using the implicit bias of image-like data that neighboring pixels are often highly correlated. \n",
        "[Vision Transformers](https://arxiv.org/abs/2010.11929) split images into patches (e.g. of size 16x16) and then linearly project each of them to the desired token dimension.\n",
        "The same 224x224 image from before would be turned into 14x14 patches, each of size 16x16x3, flattened to 768-dimensional vectors.\n",
        "Each of these vectors is then linearly projected to the token dimension of the Transformer.\n",
        "\n",
        "In the next cell, your task is to implement the patching + patch-wise linear projection operation.\n",
        "\n",
        "Hint: Can you perform this operation with a single Conv2d layer? Using einops can yield an alternative and elegant solution as well."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ded45059-e83c-49da-b2b1-0e902088b9b0",
      "metadata": {
        "id": "ded45059-e83c-49da-b2b1-0e902088b9b0"
      },
      "source": [
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "43db2db3-ff29-4e31-be8f-b5100695d659",
      "metadata": {
        "id": "43db2db3-ff29-4e31-be8f-b5100695d659"
      },
      "outputs": [],
      "source": [
        "class PatchEmbed(nn.Module):\n",
        "    def __init__(self, img_size=14, patch_size=2, in_channels=1, embed_dim=192):\n",
        "        '''\n",
        "        Image to Patch Embedding.\n",
        "        \n",
        "        params:\n",
        "            :img_size: Image height and width in pixels\n",
        "            :patch_size: Patch size height and width in pixels\n",
        "            :in_channels: Number of input channels\n",
        "            :embed_dim: Token dimension\n",
        "        '''\n",
        "        super().__init__()\n",
        "        \n",
        "        # TODO\n",
        "        self.projection = nn.Conv2d(in_channels=in_channels, out_channels=embed_dim, kernel_size=patch_size, stride=patch_size)\n",
        "\n",
        "    def forward(self, x):\n",
        "        '''\n",
        "        Performs a forward pass through the patch embedding.\n",
        "        \n",
        "        params:\n",
        "            :x: Input of shape [B C H W]. B = batch size, C = number of channels, H = image height, W = image width\n",
        "            \n",
        "        returns:\n",
        "            Output of shape [B N C].\n",
        "        '''\n",
        "        \n",
        "        # TODO\n",
        "        embeddings = self.projection(x).flatten(2).transpose(1, 2)\n",
        "        return embeddings\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7d54bd0e-9777-49b6-8c00-c5aabb66b2d2",
      "metadata": {
        "id": "7d54bd0e-9777-49b6-8c00-c5aabb66b2d2"
      },
      "source": [
        "### 2.6 Positional embeddings\n",
        "\n",
        "Attention is a permuation invariant operation for keys and values, and equivariant for queries, meaning that if we don't \"tag\" each token with some positional information, the Transformer would have no way of keeping track of where some piece of information comes from and goes to.\n",
        "We can add this positional information by summing to each token a unique positional embedding.\n",
        "Should this information be useful for the task, then the Transformer can \"choose\" to propagate it throughout the layers and keep track of it.\n",
        "\n",
        "Just like the tokenization, positional embeddings depend on the type of data too. \n",
        "If we deal with sequences like text, 1-dimensional embeddings can be either hardcoded, or learned end-to-end.\n",
        "\n",
        "Here are examples of common sine-cosine positional embeddings in both 1D and 2D:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4615bcaf-65ba-4004-be38-99e86fa0b3f9",
      "metadata": {
        "id": "4615bcaf-65ba-4004-be38-99e86fa0b3f9"
      },
      "outputs": [],
      "source": [
        "def build_1d_sincos_posemb(max_len, embed_dim=1024, temperature=10000.):\n",
        "    \"\"\"Sine-cosine positional embeddings from MoCo-v3, adapted back to 1d\n",
        "    Returns positional embedding of shape [1, N, D]\n",
        "    \"\"\"\n",
        "    arange = torch.arange(max_len, dtype=torch.float32)\n",
        "    assert embed_dim % 2 == 0, 'Embed dimension must be divisible by 2 for 1D sin-cos position embedding'\n",
        "    pos_dim = embed_dim // 2\n",
        "    omega = torch.arange(pos_dim, dtype=torch.float32) / pos_dim\n",
        "    omega = 1. / (temperature ** omega)\n",
        "    out = torch.einsum('m,d->md', [arange, omega])\n",
        "    pos_emb = torch.cat([torch.sin(out), torch.cos(out)], dim=1)[None, :, :]\n",
        "    return pos_emb"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0295372b-089b-4208-9bdf-7ebbf6bd7d4b",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        },
        "id": "0295372b-089b-4208-9bdf-7ebbf6bd7d4b",
        "outputId": "5df77021-b582-4542-c1fb-7525f7950e6c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7fa3e413f640>"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB3CAYAAAAaaxszAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAB9XklEQVR4nOz9d5CkWZbdB/7e+6Rr99CREZG6MqsyS1d1dbWY1oORmBmMAGYGIAYgSCyxxC5opO2CXLNdGkAsuaDRFgsYSZBNcGkAuMAAg9F6pqdlTYuqLl2VWkdGhg4P1598b/94n3t4REZkZlVlZVf1xDVzcxmenhGfXz9+7jnnCq01+7Vf+7Vf+/XhK/m9fgH7tV/7tV/79e5qv4Hv137t1359SGu/ge/Xfu3Xfn1Ia7+B79d+7dd+fUhrv4Hv137t1359SGu/ge/Xfu3Xfn1I6z01cCHEDwshzgshLgkh/vP79aL2a7/2a7/26+4l3q0OXAhhAReAHwRuAi8Bv6C1PnP/Xt5+7dd+7dd+7VX2e/jZ54BLWusrAEKIXwZ+EtizgVccT096hffwT+7Xfr0/VaptIiYPf69fxn7t16718stX17TW4ztvfy8NfAaYH7p+E/jozgcJIf4m8DcBZvIuf/LjR0lTC62FOSmxdVlL1LbrAqW33w8MHgMMzvuPA9Bs3dYvPXR5r9t33jdcOx93r/ftVop39vj9ev/rEz/yZeL/6PNg+SBtEDZIB2H55jIgpI0Q5tS/DgyuA8ihy+ZL6u41/Lj92q+7lS3/yvVdb3+//2Gt9ReBLwI8/VhNj/yDKaxuAxEFiKCL7LYhSaDXhSiGIIFYoyPQoYVOLFTooNOh80SiEhudWKSJhVbSnGcfDCqVKC1QQ9f7HwBp2v8QkNs+HMxrlbd9OKgdHxR3+nDY2fz3+nCQd3jMnX7+Xmr/w+Hd1eaVGcZ/819AwQbfhUIR7XrofBHt+mjbQflFtOWh3Dxa2mjb22r4Vg6yBo+wB80e2LXhp4S3NfG9Gv5+s9+vveq9HBkLwNzQ9dnstj1LWw5peY60PAc6QagEkYSQhMi4i0giZNAxzT0KkUEXkgSr14WoB1ELQo1OQIcSnVjo0EGnkjR00Yk0DT62TROPbbQWpLGNTqVp0FkzT1MLpSRaCXP+DtH/nZB//3bY3oTFbshf6G2PHy4h9Dtu8JK9Zxr7zX3vunjpGNevHaJcaON5IYVKCzcX4lZWsPIBViHALqXgCSjmwPPR+YJp8q6H9gto20V5JbS00G4BZXsGvdsZirdzAAjpb2vmeyH6fuNOdZrdvo/o92t7vZe/+kvAQ0KII5jG/fPAL97xJ4SE3DiOP4UlfSzpDe5KVUiqApI0QCVtdNxCBGuIJET26oioh+w2EVGICHrIbgfSBLoBRCl22IZIoGNpkHpioWPboPDYRsUWKmvuWgnSxDT5YdSe3obYTaMHsiZ/701diqzBiq3GLoab+o7GbWXXb0Ptu/wa+48R4vZmfaeGv1dz32/s8OrKFGcaLlM5RcVJmc51KboRo4U2pXyHfK43aOreSAMrF2CV68i8hpwNxTw4LrpYRts22i+gXB/t5lBOHqSN7iN3y0dbPkjnjqhd76RodLInalc62Xb7fkP/s1Hv+q+stU6EEH8b+EPAAv6/Wuu37/hDt+rY/+3/hHOkAWNV0rEJkrFZVK6GKh9GWD6WU8X2xsAbQ+fn0DohTQO0CiBuQdxGRh3T1JMoa+oBIuiZUxwhuz1IIuipLbQe2Vkjt9GJREUuagix95G6aeoSpSzT6JVEK3lHtJ4qmf1OtnP5w7ft1dQtdqD1XRr7+9nUYffG/metqZ9puPy7zguUe1MUVJEJXaNsl5jwx6m5mqqbMuWHlJyIsUKbohdQLrXJFzq4+QC/2sLyQ5yR6wgvRZYEVt4bIHVcD5Uvom0X7fooN4e2PLRbAGmZpt5H6v3GLn1gO/eu2IHUs8a9k3vf2dD7td/Yv7/qPf01tda/B/zevT6+181x6cXHmV5aoDC1jjNzAffgBqo6QjQdov0yaekI0i5i2UUs20cIC61TlE5QboBSBqGrXB3SAJnbQMZdZNBGBB1DveTaEEXgdxFRgggTdBRBFCNCG1KJChPDr/ebeioRVgoZn65SC5VK5A6ELpRGK4WUAqUM+hGp3kG7aIQwDTBNLYTQ29C30gLBVmOWWa/sN2atBXKokYsd6FzuQrsMN+6dDftO9+1Vw039z0Izbyeaeu88TXkDKTyW3GnyusZ4e4pRUWTEsVnKFag4OaaDHCUnZrzToZLrUsh1KbfzuH5IvpPD8iOcahuZDxH5AFHqgetgBd2MbskjXd8g9LibNfI8Og0yZG4bxG4l5rK20UMIXaeJad7ZMEUIe9CwpbDRd6Bc1C4Nf78+vPWudeDvpgpyTH8q/5d4uGxzuBBxsNjixNQtKtUGow9fw651EEdrqLFJkpFpkuph8GrY/hTSup1y0TohTdrmwI/qiHBzT8pFBD2IowHlokM9oFx0bKMi21AvOqNgUnkb5aKVMMPTO1Au/UHpbjw6bB+U7hx+7jUovVcFzc777/S4e7lvr/p+bOh/dHOa36svs6DP00s2iJINAAQWtlXCs0tU7TmKusKsHqds20zlBCOuouYlTPk9im7ERLFJzg8oldrkih3cQg+v2kJ6MXa1i/A1lBzI57ZRLipfRtsOuo/MbW9AudzroPRuqhhz3/6g9MNYtvwrL2utn73t9gf5IiLR46y4THfzEKuBz63eCK3YZWKtw0OBR2mkQbl9E3vqLO7EMtaBOkl5jKTaMo3cHUNaPlLY5oATNtigpU8qbLSVQ8cttLQRTgekhYgCpO2AbSOiEKSEJEF0A3AVBCnYGmkphNQGiQuFTi1EbCOERqVbFEn/XAhrMGRMxXAjz9Bx1qTFjkYOCikloAeP6atSBtSINk18gLT7N98FmYNB5zub+G6Pu5f79iqJ/r5r4gfzEZ9OpnirWWXV3eCm9RZR2iFOW6SqRy8OUVrRsYqkIqEcV+ilBZqezWbsEqSSUugTpRbFXkQQepR6PrlOj0LkYPsRXmph5UJkFCKCBHwbkSQI1wWlwHZQfmxUL2mIUgnC9tB2YpC5SsDy0Zm8UafJAJkDWwfKHsgc2BOd7yPzD2c9UAQuhKWF8NA6xXMmKTqTPKyfZNLxeWpEM50LeHh0lamxVWoHVigeX0COaNSxY6jyCPH4cbQ/hvTHsZ0qlvQGB5zSyQCVJ9Gm4cx7q4g0QHbryLBlmnm3ad407WYmX+wZ+WKs0F2MWiUwlIoK3S3OPLZ358t1hsp34cu1NjTL3dQtO/ly8/8Zuu0+69rv1qz/LOraJ6sbjI5t8OKZ01xtlXhhxWYx7nJevEYnXiVK1tA6BkCKPLZVouhOUpZTjKSjzFhlSo5kJg9lRzHlh9S8gLIXMpqh8kqtgZML8Mod7GIPKx8gKxHCE1DOEHm+gPZzeypb9C7KlmFEDvembOnXPiL/cNQHAoEXxAiHc5/jWvoaqQ5px8tcdN9mMR0lXDvElF+gHrocblaY3RhlbrNMYXSTQusS9qiDnNsgHZkiLU0SlQ8j7CKON4YUNkJYWNJDaxvcKioNUMJGxy3STAEgoi7adpBRgJDS0Cu2DWFg+HERIVKFFjE6FQhLITLtuZAarRKEVKiM11YZzQKAEghloVI9eMzWYHG7SQklQWhSJREiHbp/63e1DZVnn7EDrnwH+h5G3e+EL9+rUb9TVN7nyz/MjXxq7ha1z83zqUqLRxcmGfVOcqOTZ3L949zULRbcq9Tj6yRph1T1SFSHVnSL2O7SterE6hDFIE+YFqi4Fr0kRzN2qEQeQWJTdEPixCbnBxR6Pn63g50PcOMWwoux4i54ZggvotA08MSgcVRqhp8qNdSK7aF1Yhq4NigcOwe6b0ACrTOefAcqH1zdR+TfF/VA/zoVR/ADxSnCzsMspxdoh9dZjVdYFTabuZOM9GZZCeaY79Y41CnyWJBjfKXBXOTgj27iBZeRk5vIsTqxSlG5ERJpI6WP41QGtIoQFlgFEstH2UWUU0I5dXDbaGmh/BDLdgb0irBtcCLTIZMUIWNEotFWDFJDKhFSoRPTyGVqoWILYjujXkzDJtUIYSG1QAk9kCAaNG4Gm2lqIaVCa4El1VBjVVgW2xA5mPfdMLUCtzfyndQK3K4hf6fUym7Pcbf6MDfy8ol5op/+ryk89C8pX32L4u82uTU/Q9U5zNlmhYut05x1HRrJAu1wHqW6KNUiSRt0xQqB28C3KvSSY5TjPJ3YYzN2qLkWYWpRdHyi1KboBVQjl0Lo4uV76FRi+RGkbUQuRqbG0CZ8D6FStBuCSsF2UX6KTkN0kvHjtmd+4xnK1pl6ZbiRQyZJVAlC2oPGvhu1sj/0/PDVA/2rTFQb/F8++SJPnXmUVzcO83qrzXnxGu1omUZwjbZcZtNbYr55ggOtMsvBFFMbozzRKjFabnBg5SaFA2s402/jH15FVUeJpjdI/DJp8QiWXcS2i6aJChvbKhh+3DIn7QYoy4c0QFseMu6iXX9LveK6Rr1idyFKEE6KdGOIBFjKNPJMYy4sG2EpdCpB2lgqzdQrCpVK8xosRZI5Rfu0ihB6oF5JU7krR25ZDJo/sCtHDvdHvQLvD0duXteHp5EnmwWY/yPCg8/C3JOMjn6FsStfYeabD3H18hHeXp3kxbXTLPQe5g37Aq10hVY4j9IhSnVoR7foijUit0te1mjGs4zGRaq2Qyv2KTkundih5MSMBT7VbsGoVwLPqFcCFysf4vQ6yHyEyMemkbsOVobIRRSg/IJxheoEEdlbjVxlipWMI99Xr/zZqAf615AVmPvsK3yy5+PJoyhdpNk5zqIL9d55VNqj3uuReCFtcQi3eZBm7JK3RmhHHo6doFKLYiLxvFVkGGD7BdJizTRp32S92HYRskGnAizpgw1K2KReAkkPrRJUhlwkGC2uUmbwqRTICIgQOgGpETqFRCGU0Wr3m6qWmZEiNci7XwoQSmBJge43QWEGnVKm2SB0d6SttUDK/jCU7Y8ZQt0CPRh2Dt8OvGNapf+4+4XGzev98Aw7w80S5Uuv0316DqvyCL3HfZyJNxnlW9hujGvHdJLDlByXzfohlmWO0G4SJQ1SHaNVF02PTiyJrR7YkOop4riME3gEqcSVDpGSg79HqiSWVCSRg2WnONmHtq0EUkdIGUKSgmWbYxJzDGjXN8er7Rowkf0fND4QDP5PWoIAtDI/2OfBd6JxMA36brTKzsft1/e+HugQ8+ln5vRX//Cn8C99A/Xlqyy8dJrffftxzjY8vtS5xpq6TiO4AEiEcBjJnWCEAzxmzXEgLzhd6XG43GC6ts7MkXlDqzzchFqVZPYoSWUKVTqALMxtp1UwB57WKUnSNvLDeBPilpEeBk2jJe82jTmo3TCKlSAw/HiSQi+GBFRPQiKM1DDO8lhCB7QkDZ2Bu7PPjw9f32nhH3Z73quFH+4to+W9DDl3e9y93rdXfdAbeaokWsOnfvTLuM9Iup/+G0bpEbfwz/8x8vw5Vr90ksVb03z52lFudB1eaXS5ac2zFl+lFy2hdWg+UoWDJQvknTEK1hjT6hBlcsz4PhUXJvyUcS+m7MRMFtrk3ZCRcgM/F5Avt/GKXexiD6fSRvgJsqzBt430MJdHZ6YgQ6tkg04nD31+fCA73NvtuR/I9eGqD8QQk7iD7K0THnqK3OPLzKizPLM2Tt6aYKE7x3VR5IKzSZy0ULpHO15G2Qk34xppp0DJ9rGlQSKFfJdSbGNXO1jxBpZvciYSQFk+2q0hLR8t0oFaRYG5TSeo1Ec7Bo0ABo2rFBlJtB8bJN5/3TIyTRyFcBQaiVDp4L8llTCDTssgqpQt2kMrAVZq0HMqsSzzc300DiClvh1pD64ptJaDx+xE2Hsh4zuh8btx47s97l7v+7DW9VaZq+0ccy8+xsHuRdyDXyMeP4YceZRo+iSOlNQWLmP7EY80avhWhc0oj+gdRDmKVR0SJy1S1QadoFRAmLYA2LCLJKpKPrJJtYUrLSxhPjxztkeqBL4TobXAtpMBQpdOgkwjhBMgdGwksEIiAGE7oJQ5B4S00GSIW2bDS5lsHUhW37VpkDfszo3vrL248f36YNSDbeBrLfwv/RrBz/99up+eg+frfHTqi5x+Y4bCVz7F240R/nDtM1xzzrLRu0AQLRJGS7zltVnQczTWj7McFDjY9eklDhPrTY4kFoWxTXLBAnargRzfJE5DVGGcuJQg7SIMSQ6lZXg/4dmoNCCVPtqqo53AfC11umhpGaWKbSOkhUhc8/qT1ChVXIW2FFqac4RGp+adolMLhEYlenDgayVJAUuoAf+9Eyn3lShpmn3F3qFUMbTK7UqVAdd9n5Uq/cf9WVGqfGs1zy+3fpOFF3+SZy6d5JfCP6T45Hfp/tQocuIZxOwPkVZ/heqN83yy8C1OXJ2levkEb24WGWue5C0nR92+RT24hFY9lO4QxSFx0iDRIU2rQsxRqmGZIM3TSSyariTWgpKdkGpJITBKlWLo4gceKpU4+RCUROZCZJJRKnGEVArteoZSsR2jVHFTtDJgQWdNGcs3ShUwqDwNtg04gX2lyoe4HuhvvtUocelXH+P4+D8iPnqKeOajqEcfoVi6wnPzF6jdnGMzmqLYfJKzOYf14CKp6hAmG9SB63YOuztHqh1KTolebFPwe4yELtJJ8NQ6MrmJYzukUY/Y8km9mhnWWMlWExc2WqRm4KMTlFNCSweVBuYrpU5AWkiVGhI6kogkATvJricIpc1lqZFaoBMNSqCkRmKDTtFZA9w63zLtCCXQWYbKltXdDDCV2mq+A75baPomoD4SFxkXfydefLjerVLlfqLtDyovLoVpRK8Ha7SWRzj5rY/w8OI4s7V/QXzyScKDn0aPHEdbHoUn/oDpfMhjvRxCzOLIIsHGLEuUidw2YdIiTtYA0DokSloonbDpFEllgpVYyNAj1RJf2sRK4Fu5AScOJjxN2kn2zS7FUgJkD0l2DMrsmJQSXA8pLdN47eyboe0ZQEIA+EAPlI22c7fx4nuh7518952UKvtN/HtTD/S3vhrY/Ju3HuNv/84SlR98hXjuE4SP/UX08UUONb5I9ZV1mtHzeLJCsnmKnlOnEyUk6Sap6rGoY5Sr6HYPkLfztGKLojtOEHpYdkoZcNM60ltERCHKKxmUbflAdcu9CVjSQ2hr8BtQwka7MVq0zYBTZJQKmDeJUoYXd7M3DRECDZE2DjmhEcoMH7P3BirRSEeYZESdooXcug8Ly0q3IXFDqZgm3Z9NSKG3DTeFeOdN/G4qk/fSxL9fhpuWAEfmORt9mYsyR/Xqn+f5VoWf+9I65eglktpB7JGnEJVHiD4R4o+/xtH4HFIqPDlDK66R71bZVEfZFAs0VQetemgS0rSBUgENYRPbxoNAPEqsXFxpESiBJz3i4QFnamHZKSqxkU5iqDgAegilEDIAXxkJ7PCAU6WobLCOtECm2Xcfv39Q7jrc1DrZhsR3G27CfhP/oNWDReC6xW+sNnD/6Av8xPWznFj+h0Q/9COkY6eIPvM5KpPf5bPpi4y8+Sg1bxJ7+ZNcy9/kZu9FlIoI4hVWhKRnN3Cbj7IRuniyRj00qW1TsU2l61MQtxCjbRzbJi2PkgBJfhrlVnHdsYHMUPa/StpZyL5ODBJna7iDtBC2i1TK2PHBZKoAECGkBjI7PqDjLfu7FJkFv49806H7smcQ2RtTpUZ22G/ifSQupRpIDfUQUt+tifefey8k/l414/eLToEPXhP/xHibg4VP8a9Wl7mRvMGXwrdZmD+B/ZXP8JlbZzi4+q+I/9wS8eRTiAMfI/DK5Pg6x/w3KZ1rk+hHuN4uEK8eYZFxrvgW7WiZON0wH946IIzrZphuK1KZEKQ1rF6JTiKxhEM3NR/wqZbEiY1lpUShi5QpTjYoRwtkEmLJ2NApfSTeL9sxtJ+MMkRu8oM0gOqjlduReL8GSpUhnny/iX9w64H+tpUOuaJe4SvLn+ZA7gijL64y8vhF0soc6cwPEEqL6sIfc3yzTCv0uNwaJenOsGJXzMGvOoRxHa0VS840OhzhVs9HCJ9au4xfD7HtFG+0gU0HWdsAaaFydaMDt3xSFRhZ4ZC1uE+nKGVu12kPrZMsIS4EQLueebNEoUE8rsoGmykkGoEyX1+1MEhcCfNBkJoGL/QWfdLXewstQYLUetCcxY4mnabbHZRbnPntTVzvaMh3auJ3qz9rdMqR6gYPjaa8uHaQdXuUpd6bpH7MwdUnmcgdpvx6nerD51G5Gvb0Z4hHITrRxK1/lbF4nsMrEwBcaVdQvSKrjBPbXVLVI9VtQKF0jziBnrBx7TxSShqJj8ahFEskFnnLwZMKiSafAZMwMOeWFyMc821Peon5BhiY41NEdvbNEEQSGy9PduwiLVAWOkPguyHxPmDpI/EBgMlqv4l/MOuB/qaLYpQ5+ShfCX+F7tWf4lzzs/yfkm8w/skv0vtL/2fSyacJfnKE2cqvUv3OKurrn+DtzQrhyg+wkLvGUvcVlO4RRB1uCIdNewqreYp65JLqGmFi0wlyCKEpjNXJ24vIVgtHpYg0JE0Do1LJBpt9JD6gUxxjwU91grZyhkYBRNTNFCoBSimEHSKCrB1HMUJFYGdIXOoB7dHXcuvEICfTpG0gyaz26QCBA7vSKX0kvtOCn/3Ee0biew024c8WEn/iuVfI/6TH3/2/C/5k/mn+8WrIZjzPv1N1Ni99hqutT/OX0hcYefZX6f1MAVGYg0M/Qmh5eNNvcDp9jcmrM0TqFFfbeeT6ca7ICkv+FRrhNVLVQesIpRVBrNBaEdpthJS0kzJWr0g3kSTaIVaCIItrKEYuQmhykUnK1ImFnXcQUiPCGIsMiStltOJJMtCKb+PEVWpkhrCFxMndRqcMao9grP0m/sGqB/pbLtmCx9xJzoeSBesWr9UPc/7yUdx8j8KnXiEePQqV46hDhyk0LnHs7ApRanGwXiaN59hwrhIlDZTuEqVN2kKyLg/ihWXWAouqk8OzE8aaJYSV4m3Wse0AWdnE8gtoyyP1qihME5dwGxLX0jaLbAFt++g0GGRRKMCKAoPA7RgSG2wFdtaAbG2Gm3aKSPtoSII0uSrmnzNoXGiFtAQKidB60Hx3IvFhmf7wQHNbo90hMbzt8XcZbO6lBd/r/u9HJO6Uu4THPsGJh1+lF7v8zsrj3HAush5c5CqbjGzWeP7KIZxcQOG514imQ+TkJ4kqc8SzXfy5rzMSW8zenCVKLeb9PK1OlY41QddaR2tlDD9o0CGJ6hGmDl2nhZSSdpLDEoJOIsjbEiex6SUOUmgKoYcQGseNsdwYYaeo0DHHSxQjZHY82pHh15MEZIxI4uwXbKGlhVBDSBxAmft3Q+J71X4T/2DVA/0NT41s8p+cvMKZ736Bm+ocfxR8k9Hzf5WPb4zxN47+a/JPX0J9/r8keOwnkTPneXjzS4y/fYzV8DnGGzXanadYlOdoh9dJkjrttMU136MtDiFacyh8eqmFbyeM93K4uRC/s4lnr2ApBUmEtj1ULiCxfKT0EY41ONCEsLCtAmCGmgrDHRpdbWKQt0oRtoOQctuAkyRFZG9QgTKPk9og7yRrxnIrG6W/Qg22OPHhJEKDuEHKrbCr/kLmnTpx8zL2lhi+U9fmvTTn77fBpnQSkuIM5b91hc+8/Tv89X/wV3lh9Wl+ufc2Z9Kvc70zTvXSx3iuPsqPz/wmucdv0vvsw1jFI1B6CPXxOoXpC5zaPE91foZIHcKRPk77EIkb0pALtMMQrWO0jkjSBkpFNIRNaLVxcImiIuABFrESONInSC1sqUiUtTW7SCXSSrHiyACCNEbKrFkrZZq4Sk1gm0q3jq++d8EGnQSDDBW43bU5qKyp7+XavFPtN/H3vx7ob1eU4dRzr/NT136KL68+w3fkLc4HDeRahc+98BQHe2dQj3wJcqOokZPYJ7/MqLrKqcvH0HqMG50DRHZElHaIkjXQCb1kg03XYyUdoRKUcKTNZLeAEJrqehUhtDH7eJtYto0q1gBQXg3cKqky0/n+UFNlB6oQBolryxiEtBtCBNp2MwNFbPIpwOzmJAIrQbgarTQiNdP/PvImtczPWQKplOHHhXkj9s0+ZF+bJaCEHnRqvYPmgNut93A7Er8but43+2zV5qVZxn79n5A8/SzpI4/y2YfO4VoP8WL0KdbUdVrRImcaCkGVZ155hKnkAtbxr5JMnMYpPUQ8fhRUSuXoNVRic7hRpRHbBKnDam8abAiSTdK0i9Jd0ApFSKQ6ALStJra28BObfGwjhaRkm7dnKXawpMK1EhzHJGK6gRlOWqFB6dqJEFamkHKigdlHk5l8bDcz9qSIJLwns887sd7vm32+N/VgPx7LM7g/V+Pnz7xJ57XHeWNjhDPqBRbCGT528WE+mdg88ok/pXf60+RHnyd48tO4lTc4de4Stky51pmjuzlL162zlrYMlZKs0QIWvRK54BjgMeblSLVgZMM0a6fYw/dWEXYdq7wOgMqNGCpF+tvkhQNlSlbaLhoU7mTIPA230EyGaAYqgCQ1GnFXD6Rd9FGzlQ7eJ0JlA02lMjSVqU+EQsgMeVupCcHKKJbd+G6GJIJ9xyZs56L7fPy9DDXfTRO/n/W9ROHXrh5mfXmc44+3iB75Cxz/mb9P7g+6fG3543wjStjsvc139WtsbJzkIxdP8GRicfixl9FOHlF+BCaeISxMkD99lRF5jSP1Cu3YI1IlVoIyKGjYCwSASgI0KejUuI5lQttaBwmecvDiPCAp2UbbXYhMs7Zlip1x2q4bmW9yvlFECTtF2CkQgZu5M93MvSktRGJuQ9ho2zNN3GZoqGm+aQ67M4E9NeI7a59K+d7Ug/3NdlcRUZ6H/v3X+YX/IUfrrS/wG92XqMfX+d1bj9FKTnLg125Saf02nc+OwshDhG6e8ed+DcuNWWqXUbqCajzG616bTrRIkjaJkwYb8hq+XYBgilrHJ1KS0UYNpSSeHyCcBI86srAMSYzySqQ6yTTi5jfRX9l2m9EHjDIF0EloGrFKjdEHzIYfMBpxc4PhwlG7aMRtpM4QeKZM0dm5uW1LmaIyCeGwvPA2jfhQU91SptzehN+NMuXPkkb8mwtznGk4/Hf/4rcp/ug/JfrEZ5mRX+GvXj9M7spxfkPXaSVLnJVt/mTp03SShxn50jLl5CsEbh7GnsQqHiE++QSOfJOZ+hWi2AEOsB5W8bdpxIOBRlzpHlopOsk62OBIFyexUdohZ1sk2iJvGSewLRVOFsXguBFaSywvNimXlgI7MBpxN8y+JQ5pxKV1u0Y86Rt4700jfjcqZb+JP/iSd3/IfaxeF9ncpPvJn+X442d5fqxDlUlSFfIWl3i9bnP1/DHSiyn22jmEW0OPPIo4MU7l6AJHamvM5WOmXY+SNYFjlTDIMyRKGjRZZV1sUo809VCyHuTY7BXotIpEjQJpMwetFrLdRIYtZNBExy2UCkhVMLAGAwOFymDXoJUD28/CgjyTBGe7xsZs20MnK7P1YT4epQapjWpAKoRQGdI2g0ukQojsPqkRIrsujJpl2JFpztW22/qP2UavyB3Xhy4P12637/XYO5W4w8/c6b4PUl1pO3wpOM/lt06Svlknmf4IyRPP8PjpM5yuhhxUJ0jSgFY4z5lWwJlGgaWrc6RXFM7qFbRKsOwi8cQjpDOH8GdXGRtfZ7rUYMJPGfUkVTVCXtawZWHwjc9oxEPitEOQNuiKFl1Cuiqll0AnEXRTSScbavZilzByiCOXKHRNgFrkmHC1wDbRx1Fivg0mCSKJzTfEJDLNPFNkoVKEShAqMSBDJWaoqbbeA1olAz14/7p5zVu3Db9n9uvB1wP9WFxfGuXS/3icA/+ySv7HJb8w9qtc/uJf4UtrP8LLvd/l6yRMnD/NL2jBE9bXUH+xDBPP0H3qh8n5X+NU43VSLbHkDPVbJ7jiFLiaNFA6JEmbNKKbJE5INSjTTXwqbp4otfDsGGmZPZelwg1ktIqdLyKSiNj2UDL7ypnRJwM+HJCWb5CHSky+hDI2exMXa4wUOsnCr5QC0f9MjIw6IFEZuhGQDT61ltmGny0qZZAqqATobIu9AiFVhs4lEkU/avZu4Vfm/3M7lWJu2Eov7Nf7SaW8m8UQDxqFN2NY6L3K//jmz/PplWl+/mO/SjJxkMrfivjZ8DtIPsp/f+ujXIte4lX9dbobH2X8/CP8iBYc5yWsfJlo5nmc2mOEQC7oMRZdRNoJ690iJbvEelglF7uEbpt2LAdr2rRWJGmLAJDCwbIcEjWCH1WItSRvW6TaDL4tYY4Bx07IpRLHjc3vVhigACDdKJO2mrkL0jKDTcywW9suQnbQlmcGmX0ePDvXMjOs7WLqebfhV/so/P2pB4rA24nkrflDeFe/gc4XsJ/O82i1xSP5ElI4tNM1LjQTLq+P07wwh7t4HtVbRBTmSCbm8OdWmRhZZy7fYdJ3GFdjOHYJKTxAkaoeQdpgUzRpJDGNSNKMHZpBjk4nT9jOk7Zy6JZGdNvIbgsRdSBumc32WeTs4JeTNfI+CjcN3jYDIGkbBC4ts01cyoxvHDpZwrg8LTVA4QiNtNJtqBy51X77yNygcj1AxDuRbF9jvlsZKuXOyFewRcNs/dy9oeXdUPr9Rtr98KsHVQUbRvxjXO2EvLxeQrxxCWfpGuHck9QeusHj48sc0pOU3Gl68QbL1i0utx3m18ZpXz2As3QN0bxmjpHcOMnYAayJmOJEnfFCizE/oOZKKtKnKEexZQ4pcvTfgpoYpUNi1SPSXXqiS6BSuommmwiC1GjDg9QmSGzixCaKHeLIIYkcVGKbfa1xhsJjbXa9pn0UnskK+yh8cMpQuM5QeHautbmshxH5Hmh7Jwoffg/d6XH79d7rwSLwZJ0vXiryiX/UY/ynbxH/+H/Fj/zxP+TAi0/z1TeeYCO5xlfi36e28OcR3/kof+HEv8NPYtTzf4do7hPIT3aYbZ5BCM2NzpO4ssjN4ASbcp5W0CRNWwQqYjl/jVhEjHTmSLRDwS7h2ebg8Sst/NjGrW0glcL2CyTSoO3U8jNb/XZpoQSk9NGZ3R4yxKsy00SGwHUSG+Si1IB7FK5JghNuio7MsIk+9231kfhWRop5bhNg1FelWFZKmu3l7EsL77YQAtgdqb+D4Ks/S9LCZ0c7jHvP8L/Vz/JGs8FP/csf55mPv4T7Dz+C+JGXeb76B/zY6i+RW3qO39YXWA7O8GVdoLw4h/XqE3zqyO/jJy8QjT6J7Y6RHPw0sl0nZ5/lyOINPDtmoZvDsxwazRlSJ2GDlF4UorWRF5rgNklb2GhLUdcF0iRHIfIBiSMtHOmitSDnxKRa4jrZomVLGWAACCdB6ATpZA1TygECF3bPSAv7ktb+QggbtMgeL27Xh7+ThRB3qn0kfn/rgSJwKWw2dY8vv/Y0wXc84sZZCo8tcer0WT5iHWPGOkWUNLge9Hi9XmbjraPIM5eIOtcAiCZPYB8KGDt0i0PFFgcLKVPpDCVrCinyICRah/SSTVpssJnE1CPYiBw2ezmanSJBo0TcyqMbCXTayG4TGTYRcRudBqg0QOt0W1SmWY/mD+SFWDnz9XLAhzumgdvODk7cAiEQGQoXdmrQePZ1V9gZH26l25H3MN+dIWkx4Mpv57oH+dFDaF2Ioc697W9wF2S+x/23fQO4z1z597pOjy/zUw9d5Gh6DIBvLE5z4ZVHUVd+He3mEE/M8dTEEk+NaMreISzpsqwuc7UNZ+sjNC4chAsLpI2zpPEmtjdGMnIAPT1NcXKd0dENpnIBY55izMpRUaPkrBqWzCGEg+G1FEpHxKpHqNt0ZIcOEb1U0Umgm0i6iUWQWvRihzB2iBObOHJII4c0dLcWjUSWQeEZF94/GRSeDk5i6PI2FL5LmW+oGQ++Cx++/bG7o/D9ur/1QD8KbeHRlm3+2eURTn7nSR679m3i5z9Jdeocn/laiFqa5qxqck6+DutP8IPnT+B4EbnPnyEdeQh74nnSk29RCN/m+JklYmUxVy8RJ7Os25eJkg00EWGyQRNYtWZxwiqjgU3VyeFaKeM7XZqlIZemP/bOXZqAjrsZhRghlEK7KoufvUeXptQDmeG+S3PHv/GAUPiJU+cp/7l1Hnv7JBfbeX59tQEc5clv/TPiZ54n/Oh/wGNP/0+kSvJrq6e5Yb1No3eOt+RJ/PUDPH/5MLYfkv/kOaIpG7dwjM7oI0RA7tgfM6IEcwszRKnFYi9Po21cmh1rNdsWFRlpoeoRpy5CyIFLs5W5NNv7Ls392lF3ReBCiDkhxFeEEGeEEG8LIf5OdvuIEOKPhRAXs/Pa3Z5rynH5ofIBXtEv8EfXjtD4p2aNWXjyI/zwo2/wwwd6jBWeJlEhZ8TLfHN5ijfeeBTvu1/Gnf82lvSJpx9GnTrB7NEbPDy5yImy4KhdpeoexLGqgEWqeoTJJmviFmuqw0qgWQpcVnp5NhoVWvUq0XqFdMODeh3Z3MDqrCOCNXTcIk3aKJ3cpkqxZObetHywcminaFQpTt4YJWzXhF7ZNrguOK7R5NoGhWNr08j7aNzO0LeVGqlhX31iqaFzNbg+jML7KFvuQO7DqpRhFL4XUofb+fC9FCz3A0F/UFF4bmqd3nM/xs8cuckvlJ9gnnP86ari7L/+BM63v4VOA4qfWOOpj77Mp8tjnOYjICTL+gqvB2u8ujTDpbdPYr/+Xbwb3yVKGgi3Rjr6COLwGPljS8xOLHOo3GA2n3LA85hMJ8nbo7h2BWEkS2hSlAqI0w5dVactGjR1QCtJacXQiiWtxKIVO3Ril27kEYYeUeiRhC5x1ycNPIPEQwsdYVQpUWROSYyIQkQSGQ48iYwqJdlSpmxx4VkT3wN13w2F36n2+fD7U/dCoSTAf6a1PgU8D/zHQohTwH8O/InW+iHgT7Lrd6xyrsfj1Q6t8Cpvblq8/tZpZLtBOvIQB546x6MTSxzQxwHYDK5woWlxrj5KehXspRumqRZniKZPUJhdYXxilelcxIQvqTCOYxXMJ75OSJV5AzRlg80opRmbgWYryNHt5Yg7OdKOD70Y0e1km+m7ELe3fVXcWYOdgtneQS3tbJDpDigUQ6n0h5kikxaS8Yg7ZIWDy1sDzX6zNn+hPoWSNWe5synfeWC58/4H0UA/jANN6UeIkUd55OQFPjreoJusc0Us8J3rRwkujaKjOulDpyg8ucwj5YA5L4cQHt1knQUucKXtc6M+SjLvYi3dJIk2kZaPnZ8jnTiAmPQp1RqMFFqMeTFVV1AWPr4o48jc4NseaDPQVBGx6hLqNj0REqiEINXZMFMQpoZKCROHKLWJ44xKSSxUbA0GmjqWW1RKup1C6V8eHmgCW7LC/mUY0Cpa37lp7zfmB1t3/R6jtV4EFrPLLSHEWWAG+EngM9nD/jnwVeDv3um5nAOCv/D5L/NPrv8M18MO/91bYzz3GxE5+/dQP/EoT/ICf+nGL/Jby0/zor7Ed8IbdJZm+fTXnuVI5y3Sx74FxTnE9Cewn/42Y94Fnrx2CMkk890ZIidiUfWI4hW0junEy6y7DnmKlLo1JDZjfhGlBZU184XBKgS72+zZXVYohD3Yq7nNZp/YKNesutrVZu+kCJQhLHQ20OwHXCkxSCW8F5u9lmrbTk2t75/N/p1mpXy/2OzX3niI8S/+l4z9kMVPPnKNj/y9X2SZNb54PWHuyz/A50/+U4LP/STxp2b4kd95jfyZR3kx/iTLyQXqvQu8pJ8hSGs8+eKjzMTnsE58g2TyKXKlkwQzj+LYLrWH3wTgWLtMOxklVg7rnTmkbQ3Z7DtmBkOPMG2hUTTtGlJJ3NjCtxxAUrBtlBYU7BgpNLZMB/tWHS9CK4n0Yiyh0VZkYmj3sNkzyOjx9m32H7J6R0NMIcRh4CngO8Bk1twBloDJPX7mbwohviuE+O5aU5A/ssjT+REsLM7LKyxeOoQ8d4l49Aj2kYhTtQazThHbKrHJMvNxm/m1cZrzkzjLFyCsY9lF0rFJrKmU0XKDiVyPMdeiqkbwrNIAzSgVEak2XdGmnSpaiaATO3Rjh6DnE3V80p4HvcSYjIIOIu5A2rujrND8v4yU0OwXtAanLQRuuEWENJSKlGagmaHwgXxQ6NtQ+ODy0NByr6Hk8H3vBvnupFHeyc/0690MNN9pvd8ofHlpglsvPA6FAtbpIqdKPuNqlKvyDBc3a3Ren0KGTVTpAGPH5jlaW+egOkDOrqJ1yIqos9jTrGyM0lkYx165AcGqMer4Y6SlcayRkFytSS3fpurGVBxNkRx5XTKRxsLGMM4KrVNSFZKokEh3CUVApFOCVBMpiJQgUpIotYhSi0RZpKlFmlikiY1KJTqRBoWnEtItFE6SGMemUtuWcxvknW6XFWa1kzZ5J7UvK3z/6p4buBCiCPwq8J9orZvD92nj7d71Haa1/qLW+lmt9bNjNRt5rMz/7ZMv8Wgpx432V/m1tx/lrX/zGbRXIXrmE3z+C1/h81Mhx71P0ohuckZ9ky/dmuS1t09hv/hdnJUzSGETHv4oyWNPM3fqEo9M3+SRSsJhq8qEdRzbKiGERara9OIN1vVNltM2S72UxZ7LUrfARrNCo14lWKuQrruw0UI2N7Cba4hw03xlvhMXbvlmYbI9xIUPOzRdD1zPcOFWpkixjS4cWyMcNeDCkXobFy6zy/0mLjMd+d5cuB7QJA+CC78f9W4+bN7PJv7HV4/xX3zpo7CySe+pH+T/8Pg5fmjSZa3zGn+0aPPbX/oc7tsvAuD+eI0nP/4Sn590OKZOI4THtfQ1Xk6u8p3FGS6cO4F47RLewptE0VpGozyOPnGI/ENLzMwscqTc4FAxYtr1mFITFOxxHLuEoI9Wd+HC6WVcuM64cEkrcWhFHp3Qpxf4hKFH3POIe0NceGCjAyDMuPA4gig0y0mSyPDgfS58uInfgQt/J4qUO9V+E39vdU8NXBid068C/z+t9a9lNy8LIaaz+6eBlbs+z0YDdbHF3Kde5bFqQDl3ggtNmxdvHsS/9A1k1CP3VINHRld53J7Gs0rEaYerbc2lRo3u+Umcm5cIezfBq5HUZvEPrDE6tsF0vseELxlVo7hWGZGZe5SOCNImLdmilca0EkkztmkFPt1ejqiTI23n0B29xYUHTUiCoQN1Owo3/+e+wSfjwq3MZt/nwqVE95F3X1KYyQoNHz7EhWfNetjsY/6NLc570GSHuPDhJvh+ceEfJHPP+1lLgc3r8SLNV+fwz3+TY0+e4dmxVfLeHBtpyIvrJeI3wLv2beLph/CPrvF4rc4Rr0jRmyXVIZv6FtfaLjcbNYLrE1i3rqNaVwGwnCrpyCRi1KMwukkt32bUC6m4gorlkJMVXFnIjtthc09Eqg0K78neDi48Q+ADJG4MPkliD3jwbVx4wtDyh76MUO3Nhe8w9wxuG6p9Lvx7W/eiQhHA/wqc1Vr/v4fu+i3gl7LLvwT85t2eq1WvsPbqSYK/9rf5+MFrnOIjvBKs8HsLPuK7Z5HNDXqf+TlOPXKej09EFK0JUtXjjfQGb2x6zJ85jr60hFh/G8sbQ4+eRj5UonpkgcOVOnOFlGm7QMEezXJSQGeKlCZrbIg29RDWI4t6kKPZy9NtFYlbedKmD502otsxOSlhA520UWlw28F4LzkpDA01t+nCB4qUzNRjZZSJnRrdd9bMDepW2xDzcE7K4LXsyD3ZS5Gy/fW/95yU7zdFykJXc7bzW7zx6uOk31zH+5lRnnv2VY5ZH2FZrvD7rWvMv3wa67U30Ac+jnr6NE+fPMdj1ZRp62GStEczvMHZZszZRonliwfR1zZwVs6idYLjVInHj6Nm5sjNrDI2ts50ocWkrxj1rIEuXEp/wBn3c1KitEOo2nRFiw4RnTTNdOGCTmLRS+yBLjyMXOLIIYrcTBvumCYeZYqUJDUOzSjKtOHxtoGmadbpdhTeL7V3875bTsq+Lvz9qXtB4J8A/j3gc0KI17LTjwL/L+AHhRAXgS9k1+9Yq4HNV15/EuI2hx+5yF87JFEoXtdXuPT7H0P/6Q1E9RFqT13iM0cucVo/QsU/zHJ6gfPtgNduHqL5xmH8898mDdeQdpHk4HHcIw3mphc5XOhysCAZ13MUnPHB19FUdeikazTkOhtRykYoqEce9V6eVqtIb7NE0ixAK0C0GljdJiJsQtxC7dLE++aeYXt9P6aTIRSO7UK/iVtDTdzKuHBLb8WA3okLzy4jt+iQbRSJ3E6V7NYYB7TJHij8/eLCPyyKlIMFOJ3/Kb5+6wDf+O0fJBmZpvzUdf7ixCgH1BQ349f500snWPjjx1Gtq6TVSSY/9jbPjK/xrDtL3hlDa8VVscCFpuTSwizt87M4l98m7c6TJG0oHSGZOIh1UFOZWmOytsFULmLC14zqKhXGce1SZrG32OLCe0SqTaDbdEWXro7pJdo08VTSTS26qWniQewSRS5JZrFP+0FXsZUpUtQ2Rcrg1EffSQR9e32/1A4aZYfF/l5rnwu//3XXBq61fkFrLbTWj2utn8xOv6e1Xtdaf15r/ZDW+gta6427PVdTt3h1o4TsrJA/ssgnj1zCwmY1vsSbNw/RvDSL7VQRR8c49MglDuYdquIA3WiFm/IWl1oFNhbH4eYSxC2k5ROPH4OpMSrjG4znO4z7KVVdJi9r2ddR0DomUh06uk4rjWnHmlZs04ldeqFPHHgkXR/dBboBIuggo55xZ+otGuV2JD5MozimiQ+aeT8nxb49IyWTFBoahUHz7tMqCLVNF36bQ3Mw4Lx9sLkbF771et+7O/ODwIW/HzXhJzzhTXK2Ifna4qQJejo8xccnlxhzXMJoibPNPBevH8ZqLaJyNXh0loPjyxwtKnKWUS6tqqvcCkJutMs0lsbQi01kZxmlAmxvjLQ0BaOjeCNNiuUWNTek6qaULIeiKuHKojmmBuYlhdIJqYqIdY9QBITEhGpomDmgUmyS1CJJLeLY2ZITphY6MacBjTLkzkSlWVphZsXfy52ZNfHh2k1y+2648P16d/VArfSalAtNRfcfnAMJJ/72BT5ZGmPEOcS/vVbhKy8+h77wb0gnDpD/CZvPTzX4hHcUgLX0Ct9cVbx67Rj1bx01XGT7KmLkUaKjpymdvs7R6QUeLjc55OeYSQ/iOjWEzAEmOL8Tr7IqNlmJIlYCi5XAZ71TpNko092okGwU0I0E2dxEtuuGCw/rqKRNqoJdFSn9tEJh+Vs0yjAXPkyjWENcuLMj6Mo2g0tpZQafYV34MCK3tgaYco8Gv5ML340rvxNK3y3o6l7rw4rCf/j4Bf7rH/wWG1HCr20s4fzGH0G9zg/81d/mC1MRldzDfHOjx29cO4D3J7+Pu3CG3lN/gcMffYMvzN3kUf0YFf8w7XCBS9ZlXl7Pc/HGIVqvH8K9/jp68yyW9KF0hOjgSdzDLUYOLjFb2WQu32PKt5iQJUrWBJ5dzY5bAI3WoTGnpW269Oc5Ca1Y0YoFrcTEzXYyc08QeoSRSxj4xKFLErikgYcOHXQgIMwUKXE0iJvdQuNmmCmS8B0HXe2GyveDrt7feqAN3BHGEvzW24+QNnKEJz/CkULMbDrH1XiTy60izsU3AIgOPsyh2gaHCwrbKpDokEXqLHTz1JfHsdaWEN1lLLuIKowhxj2K1SYjuS41V1ARPq4sIIUJw9cZiumJLgEx3dTkLPdihyD0TLZy4KF7FkQRIgqRcRfSwEgKVbIrF27OMzmh2KJS9JC0ECmz60PmHmloFKTJORmg8P5zZ8PMvTJSth6nBo/fjUbZeb799e/eyHer99Od+UGo8YlVpj/+FjnLosEaG28eQy3F6MePc6S8yZx4hDW5wdV2Qu/SGHJ5ATs/h3Ogw8zEMpOeQ1lOoXRIV9VZDhTLnSLNtRpyYw3ZWQVAWD6qMAbVAk6lTSnXo+yGlBzIW5K8LmFLLztuJSAG3/5SHZIQEhMRa0WsNbGCWAliJUiUzE5GUqiUHELfEp1KtBYmnCobZpp9rvFgsLldVri1dWq/Ppj1QBv4lOvyeA3+3hujLLzyCMx8ih97+Aw/Pe1xTn2Tryxrln75INbCNazDP8GjH3mNHzw4z7T3GLbwuJh8i5fXXV66eoz0zTre1ZcRwkZVj5OcfIzaQ/McnlngaDHmUMGhYs+Qc0YAKwvN36Shl1iVa6wGmtXAYjXIsdEp0WyWiOolks0iNJoGhXebyCDjwpXRhe9s4n0ULi0f7O3DTOXmMkmhbySFtm3s9X0Ubmd/AVtvoXDboHCEQtqZ0mQHCmdYPrhzqLlLE99VFriDC7+bpPC91p0a/QdBUjjy+GWiv/n3eHpEM6qn+Fcv/ABX/vg51PN/h4888xo/Oz5Kiw2+o1/kpa9/jN4L5m+ePn6ag59/mY+ORTwpjiKwaEfLvKmu8cZmkXPXDpOe7eJdeZM43jSpliOPks4cwj4SMTG5wkx1g+lczFROMqZGKFkTuHYJI/4ytiytQ5Js6UOHOm16dNKYdqJpJ4JO2kfhLr3YJYhdwsAAkzh0UVnYlQ5sdCghyoaZAxQeZ5LCeJukcK+lD8Mo/H4tfdhH4e+8HmgDL/s9Hq9tclGeY355Cnnpt5g8fJPnphZxZJ5N3ePty8dRV9ok8Sa5QyscnFngSDpH2Z4mTlqshDHX23m6NyaRC/Mk8SZC2CS1WeyJLuXROmN+wKinqapRPKs8QMrohFh1CURnMMnvJNYWCu95pIEHoYIwGDL2ZJLC7ODcmVQIW3z4AIkPTD4yM/XIHRb7XYw9wCCpcDiFUOpdEPXdkwp3q5268PtR3wtjz/2utJ2DlZc5VWnykfwEr9Udzt2aI2pdJD+9xscmV6gwRqy6vLUxxvLFg0Sbb6LyZcTRMY6UmhwuSkPbCcmmvsVKT7DQKREsjsLyCmlvEaUCY0Qrj0Gtil9pUSp2qLoxZUdTEA6+LuDIPEI4GReus6TChESHJDokFCEhCWEKYQpRKgmVOUWJRZQYPrxvrU/7csJUQiKMsUfpLRSeGXuGUfcAjd8jCr9XLnxfkXL/6oE2cHcaPvrIGRY73+bV1UmsP/4O5Wdv8vSnv03ZmmLVWuH35qeov3nM6GefPML082/xWMVjJj1IqhpcFQu8uWmxdGWO+LyN6i0a2mL0NBycoHh4kdlik5lcxJiuUGYim+obDj5MW3RVnYYK2Iw0m7HFZuTRDPIEnTxRK49qSWh3zdKHoI2MOkZSuIMHB7bLCYVtULjcolEGipThxm3bbFu7loVcDSSFUt02zNy2ei1r8nB789xOteyN1Icfv/V/eX+Hmfcbhd/PClZq+K98iWdPnOcn59b4SvQ2X1saxVl4Ced4wPOf/lPm9DipTnhhxeX1a0fxrvwpKlel98TneWTuOo9WO1ScWYSQNIJrXO0FXGjmWZ+fJr6aQzZvoJI2njtKUjtEOjVHbnKDytgGE7keo27CiGtTVTXysmZolL4lnTTjwk1GSk906YmIQCmCFLqZLjxMLcLUJk5t4tghjh2S2Mk04VamCbeNnDtKt8fMDuvBk2ggKbydBx9avbZLNsq7Uajs17urB9rAlZuneuQWJwo/ylJg8+1/98NQ9PCfV3zCfpjpdIpvNhqcP/8Q+Rd/g2RkEnl6go+M9rf2FGiyyqWwyZXlaeoXDuHcehXdmTcT/okZ7LmIqZF1ZgptJj2XUTVm0t6EB+jB1p6maNNKEjYjyWbk0go9et3c1taejjKa8KBjJIWZvX7nMBO2N3GTF+5vlxQOWexvkxRmxp7b7PVCbRtm7rTXD0sId9N73wvKvhNKh/s/zLzfdT9plLffPMVr/+SjjB2b51Of/BZKp9zoKNJ/dR6ilNwXbJ4esTlqPc1bySKvblQR3ziHvX4Dqg8zeeIaj00sclSdpOLMolSHJbnC1TYsLE/SvHYAZ/kionnN/IP+OHFtBmeqQ36szmi+zbgfMuIJKiJHgSqOVci2TWWKFK1IVba1R/QGW3uMsQeCVJqtPYnZ2hMljtnaE2eSwthGRbZRo0QZCo+S7fb6HVt73qm9fn9rz4OtB9rAhXRwxzd5zJqjEQm+sjADtk104glOV03Q/QX9Epfro6RnW6hclejwKY6PrjKXV1hWnl5a55a8zs12idWVMey1W8jeBq5dIalMweQElVqD0XyHEU9QFTk8q4yUZpipVESSBnRkm46O6STQSyzDHQY+YeBlw0wJYYCIAmTUM87Mex1mDkkKdw4zt5+EMfbsdGZuo1D0Nknh9n93i+veaa8HtqlNbkflu9vr7/o3/D4dZp5fH+e3Lh/Dna5T/IF1LOGwFkdcefk0OtR0n/hznCh1mRMj3FLnudQUNM4dwtpcx/On8I+uMTOzyAGnQIlRNAmbLLMYBSy3yjTXq1hry1gdk/8t7CKqdABqPu5Ii3K+Q9kNKTuavLTI66IZZmbHLfRRuBnG9/NRQp0SKTWQFIZKEGsjK4xTy7gyUzPAHEgKU4lOhQHPQxSKoU+Ghpk7h5qDF7K7pHC/Hnw92K30i6u0zh/k//H8Gaqu5n9bv0T0mkQGHX729Jt8ZlLRCq/y7bUCb/7ep7DrCyQjxzn17Ot8fHKFQ95zSOGwGp7jtXqON2/NwVvXceffIlUhae0I0eFTVA8vMDu1xKFCzEzOoWrN4FglBLYZBqkOLbXChtykHinWI4uNyKXZy9NuF7IN9nlo9swG+24TGTbuaZgpsn2ZAxrltg32OySFUmY0ijH2MMgK3xpm3iYpHNKN79U4dzPp3Asqfr+HmXeq7+Uw86X1PP/zxsukm3nSIyf4icIjTLse//Prp9l8+Siy/BCfOnmGH59JCJJNzsbrfP2lZ0lfqRN0r6EfPc7ER8/wkdGUR8QcQvi0kxWuyIucaZS5uDCLuriBc/MiUbRuBt+FOdT0LNZMzNj4OpOlJlN+zLhvMapL5K1+LER/mKlROkTpkFC1CXWbLgHdod2ZvUTSS2yC1DZRs4lxZfbNPWmYDTOj/u5MNdiduUWlRFt2+z7qHnZmDkXN7sxH6df+MPPB1ANt4EEnR31pjKPPvE3VTVmJLtC+OYG9cpPp49c5VGwhcFgLFRfXJrA2TLxK7tAK07UNptQktvBI0y6rgWap5xMvV5Aba6QqACtHWhjHGWmRL5nEt5IDJbWVuazRaB2brGUCemlKkAp6iUUQO4ShN1hPZRbDRtlBHd42zBz8EncbZsqtYeaWhDAbZu6UFIotFL7TnLMTjW/jwYceI3egcPNaHmw+yr38/AcVqfdSaEQ3CdcryG6b05WAiZzgjWaPzZVRVG+RkdllTtbWsaRLWzS51CzTuzWGaF4jrY4jZnLM5HuM+xaWLKB0QjtdYT20WAvyJGslRKNOGm8CGAlssQrlIl6hS8HvUbATCjbkpIUr8tjSZKPsNPYoHRMTEIuYWCsipUk0xFoQ9+WEqSRVkiS1UAPkbSSFaGEifRVDQ8w+AlfbEPggH2UYjQ832X0jz/esHmgDn2+7fPn8KdR/+rM8PrJBonp857UnaP66R/6nc3z8ideZzD/JjbjFr8+XSb67ibP0JuqTT3L0yTN8bMRnQhxG6YALySqv1R0W3zpOerZL3LqIsHzEyKPIozkqRxc4VGpyMB8zRY2yPY3dz0fRMUHSoM06G7rLRggbw/kozaJB4Q0HOh1Eu2nyUaIOWgUoFZCqcG9nZraxZ2c+is6s9VsBV30UznZ3prU9qdDszRySFA43613yUcxr2SktvHM+ynC9E7R9p8fudd8HcZhZdWHae4xzr5+m95sd/uJnv8rnptf5VvirvHjlIfJf+xeUPr7Mc5/5Jofsp4hFyB8sJZx//RS5l/+QZPQI0WPP88yhKzxRCxn1HwKgHd3iUivhbKPIxqU50kshcu1tVNLGtgrE48dIp+cozKwyNr7OdKHNmJcy6tlU1Sh5afZmDoaZWT5KrHrEqkdHtOkS0k0Vndjko3RTSTdbu9bLrPVxhsLTyEH17fXR8MaeePvOzGxjz3A+ym08+PAwM6u7ofDtj91Xo7zXeqANPBQBiz1jbx8rtDjufZIrzQqXzj5EMjJJeXaZR/VpLCSXo02al2dxrp0nrs3gz65zrBgwqUaQMs+mXGcljFnbGCFYGsGqX0WnRqKlKzXskQ61fJuaF1F1bPK6ZDb2ZEhG6ZBIdemJHr1U0UuFyZNInMy9ZpxrBJmpJ+qZjT1Jb9cpe9+ZOdgbuC0rfAuF6wx5D7LCB8YeBqfbhpk7DD7b+PFh6eBthp97kwzezZk5/Pzfr3W8FPKjheNcq49y4ZVHKZ+6ztHxJYSQLHRz9L6Zh0IB/8Q6x8UkVTXKDXmL+c0R0rNdRBKSlqYYO7DMXKHNpD6EKwtoHbKueqwGgnq9SrRawW4soeMWANqroIo1rGoXr9Cl5AWUnJSSA7728UQRKbyMRslKK5RKSFRILEKiDIXH2tjrEyUyJG4NUHiaSlRqobJznVjQlxT2UXiS7JAUDvPh5pjftrVnuO4xH2WfHrm/9UAbeKzaXGtbyNYtpqaW+ZHyNK/VXX736lGS2kG8Rzt8ZkLg4XAm/ToXLxwneS1Ajz+BeHiSZ6Zvcjjv49k1NpN5rooFLq5PsH7jAM7iRQjrxrU5dQgxV2Z8bJ3pfJtxX1BTNZNXkSEZY01u0pYNWklKM4J27NAMPRMz2/VJsphZel1EFCD6zky1tb1+uISwEMLOEuXMMFP3ufBMjcKussKhgKshHfigWQ+HWu06lNyuFd9Zd2rQOxv8ro/7M7Ds4fkD8/wfn3qTVzfK/PKF4ySfe54jT5zFs0e41HL4zjeeR5crxM99jOfGYEaMcCP4Nm/UKyy98jCyV4fSEaqPXeX45CLH7RFyVg2tY25ZN7nRjc1ikoVJ7KUbiO6y2VaTmyapHUROWOTG69RKTUa9kJqrKAufkipjW34WY5wteyBF6XAQMxuILoFOBssejBplK2I2TMz2+iS2jRKlf4psdCq2lj3orGkPEgqj7W7M/vk9UCb3SqPsq1HeWz3QBl4Qo0zlNPwv38RxEn7h5CU2I8XLGxr/4rfBtvjCoWvM+T5x0uKt1UkWXn7ELFcYm+bIycs8VEqYdk6TqJANfZPL7TwLy5PIq1ewmvNonZJUZlDjU1TGN5goNZnwU0ZljqIcxZI5g8J1QqJ69HSTtg5pJYpWYtFJHLpBjqDnm4CrnpUFXHUHAVckvcEgc88DbefGnqGAK9O0syZu2VsovK8JzwKuyPTgQqiBJhz6qpOhprwNod++7GFw37vQhN8L3/39EHB15KErnPz5b7IZCV6pR8huE3e0yRfcP0eUwh/MTyMWb6H8Ih+fXOJU2SzPvtKWvHX5OO61M+hgFXF8lOmDC5woCyb1IaTI00pXWKLOfLvM6so43FzBrt8wzkzLR/tjUKvhjLQoldrUvICqk1KxbYo6jyuL2XG7tZrMBFz1TT29QcBVkGqCbFtPmOWE98OtksRGJaaJGzVK32YPJHoo3GpLlSKGTgzz4HcJuBpc3l/28L7Wg23gtsWom/D2nz6DsFJOfeolumnKWX0dcf0GACceP8tUTqN0j4stn6uLM4hwk7RYo/TQPLOFHpNqAqUTs1C2a7HcKpPesrGbZpip85MkI9P4Y5uUS21qbkLJtiiqitHVCjvb/h0RqjYdERgOMQsF6g8zk8BFhQ6EymzyjgJEf5i5x+LjPo0yHDW7XU64Nczcno1inJmwE4UPZZ0MqVHMbUNuzB1OzPtdD1KN8m7qvaLw0pFbdH/sP6SXwiXriolSKEQ8N2qoiW/Xe+jVBG15PHT4GkeKEVrHLIURZ+s1WF5BhA3igw9RmF1hLh8yootI6ROkm6yLJZYDl41WiXTNw2quk2YNXLg1VHUUWUvJl9sZjZJQsCV54eLKPJZ0DeUGkA3idebMjLWhUSKlBtkoUSqMlFBJYmUNhplpktEoiVm5ZmgUMmem2kLhQzTKgAeHLWv9cKnbt/PcqfYb8/2rB9rAD1Q3+czBa/w3rx6hsV5D/3sf5UDOIRYh1379GaK3XAo/6/HcWIPx/JO8uZnw1VtT+G9/DRm04WPHeOLAPM9WinhWiSTtcL4dcKZRpn72MNb1iyah0K2RjBzDPdRgZGqFg8U203nBhK7h2xUzFMokWWHapCk3aaiAZizYjC2aQY52N0/YyZM0C6iWRLRbyG4bEXVNzGxfkbIjobBPoww04ZafOTSHh5kGfevhgWbGg4v+yrWMA0dohK0Y3tazM+BqmAq5mxplN3nh+61G+TBs61Ghi9WY52BBcSg9TP1f5+henuKvf/IFZvLwSvonLH/rNP5rX2X6C2/y8blrVHOPsMImL6xadF6exD/7AvGBp3BOa56ZWuBE0afqHyZJe2wm81xqWVxq1GhenkEu3EBuXgLAdqvEY7MwMUJ+ciPb89ql5kLVdsiLGr5VQYohFK6Tweb6CEOj9HRCN1V0EwiUIFAykxNmNEpm6knjLCe8T6PEMnNm7uHKHEbhcHvMbL/2oFbu1tT3h5nvvh5oA5ellLGxdV7XVwhCn3T8cUY9TVFXuLo4Q3dxlPDQ40wVmxzQx9jQXW50JGLhFrLbIpo+wchInelcgm9VAFgXm6yHFq16BeptRHfZDBK9GlR8vEqHshtQshV5y8aR+czdBmCcbQkhITFBlikRpDZRapuEwtAxmtk4giTOQn7SwdLj3dDENns9DFA4sLuxB7YPM2GbsQeGTDv9RrtLFvjdaJFh3vxuFMmd6m7beu5HPegGH9VLeBdfZtxLOOj7XLt8mM7qCKNPX6DqpERJg5WVcdSNLhycYmJilTF5iEhEzCcNmktjiOVlhFtD10YZG9lgzNOUmQAgSQM2o5R66NLZLEMjQHbWzFZ3YaNyI+h8AbvYJecH5J2YvK3J2wJP+9giW3os+uvWdMaFm2MwEhExCbFSpNqg8HQwzDSDTKVkNsSU0E8qTC1Q2+WEu5l7tm3r2a3uAXnvywvvfz1YI091hrFj81zvfI1b9RqyeY2Pjdf5dOEAv3J1mjfeeBR39oc4efIiPzxSY9Va4sXwJutfO468eB5r9vOMP3qZj00tMquO41glbuqzXGolXFiYI7gwirvwFgCWN0Z66Bj+3CrTo2vM5EMO5CwqYgrfrgCWsSanXbqqzqassxkpNmPBZuTSDHL0ujnibGcm7Z5B4UHbJBSmvWyQub2JD3Zm9ikUO2dQuLSMvX54c/02a32/oWcofEiNspUFvrXoeLdtPbDVnN+rGuVOCYX3Ut8LTfh7oVFefekpvvUPnuH5A/P8BycX+OfnjvPimdMEf/mv8dhIHc+p8cL8YS794fP0Tn+aiWfP8tn8YVztcjZ9gTcuPUTr21MgbOLZhzjw1DkerTZ5mINY0iNJW1xRq1xp28wvThNereLcujLQhOvyYZKJWezpkPLoJuOlBuNeYlC4qpAXNWwrh8ChP8xEJyRpz3gadJueCAl0Si/VdBOzrSdIrYGpZxiFJ30Evm1nph4sexgkFO7Cg++aUMj2bT13a9b7pp77Uw92oUPUJO7kGM8/Ti9x8L/5O8xVN3i8GnCtE3O1WSXs3SQ/2uDRagtHezRZY+nmNMm8ixA29kSXA2NrjIkieXuUMG1SVwGLnSK99QrW2jIqaRtUU6whaoJSuUXZiSjZmoIumqS3TE6oMaaeSPQIUpVlSpg8iSg2UZxp6BhTTxwZHjwNIQn2lE7dZuoBg8ClZXJR+ggctqtRMlOP+dkhBC62ywWHzT7bG/UuqPwOyx3udNv9rg8qd96vK40aX5qfY2xkg4ePX+JSK+Zmuwi5cSaKTY5Yz7DQdbmwNAOAHFM8Uomo6BJx2uJmu8TGzUl0uIpy8jizHSYLbSZ8G9cqgJA0ZZ16pFnvFgk3Koj6Ojqqo9IAIX2UX4Sih1fqkPMDCnZC0dbkcPC0j9VH4WyhcFCkOiYlJiXJUHgmLMkywlOdne9A4QaJC7QSRk6ot6tRgK2EQth+fp9zwvdplHdXDzYLpdGguTLCZ+1naccu879yiqMnLvODj7zF6+I1Xq8XsG6+gD+3yqdOv0leF2kkC3z35iHqZw+bJzk8xYFHL3K06DAmDhLFdW5ZC1xs5Vi7NYm+tgY9E54fjx6FyQlKk+vZmymlpssU5Shb69ZSYtWjpxq0VUIr1rRji3bs0gl8wgyF667INOEBIuoh+ose7rRubTihsJ+NYnlbjXyvdWs27Fy3tltC4W30yLbAq9vlgbutW9ut+T/IhMJ7fe73u76zVuC/X/smEw9dZ/xnb/G6eIO3G+YYOTC9xGdKU5xvar68OGq+gU1P8qm5a8y4Pkna5M3NHGduHMZeOwe2R/roaQ5OLXKinFC0JpDCZjOZZzEKuNEusrkyil7oIVu3UCrA8cZIS9Po8UnckSblapOaF1JxE0qWQ0mV8KwiUroZQDDNW6nIZKOo7mDdWqRMuFWYZaMEqUWs+lvrt9Qo2xD4YN2a3lKi9NetZZLC2xY+vIN1a+9FjbJfe9cDbeCt9Srnrh7lb5xcwhKa33j9KfKTG8w+9xYCyWqgcb70NbA1kx99m0PSaLff2ixw48Yc0carqHIN/2SdY8WYQ3oSgI6uc7MLSxujhDdHsJrzpEkb8tOkIxN4E3WqhTZjfkjNdikrs/3bcBUpSWqWxnYJ6SQpncQyTrbYJQo9ksBD9RzoJYigh4wCY+pR8a5b6/umHhiiUuTtpp5tXLjI9OB9FD6kCR+oTga5KDuyUeTuzVfs0ujfbb1f2SgfFBrF8MZdhK3QhRIjTJNqyH/5n+KXOvzo7DLtNOFcM8Y7+x2IIo4/fpYjJXDtcW52FWc3q7hX3sJqLRNPP8zIzDLHSm3G9SyuXSFINlmXdRZ7LhsbNcKlGnbjFipYNceLW0WVR7CrXXLlNhUvoOIklBxJHg9PFLGkN0SjmICrVBlNeCh6JidcGWt9pARRhsKNnFCSJGZbT5pYpInJB9dJZrNPMCh82MSTpRQOKmvad1SjvMulx3vVPo2ydz3QBr4ROlzcrPH8p76FEJo/WpTYIy3sJ4s4wqcZp2y8YizI4pnDTOUscrLC5VbK9c1R7I3LqHwJdewYM/ku075xp4Vpk8UgYrlTors8gt1cQydtLKdKWqxhjQSUSy2qbkjRluR1zrwRMneb1sbV1hM9eiqlmwp6qVn0EEYOSehm1mMQUZhtLjFyQoaWHu+sPn0CbA0ys6atB7koOxF4hsKHslGG1SfI2xvUTl14Pxtlt3on2Sj3uz7oNApk30RsmxFVRWkIvmPj5EKeeuQsgU64LObRV+oQBhSeWGImF5FzRriVtLnccuDmCrJdR9ceJj+9zkx5k1FdwrcqpGmXBmusBIL1Tolwo4xsbyKCNaNeckqkxRqiBE6pQ9ELKDoxOUuQlzauyGEJb5ucMIsUNDSKjklEYlyZanjdmrxdTjjkylSpBSpzZSYMttb3aZThQeYdUfge9V4jZvdr73qgDbyuGlxue1j/4aMUnJhvJF9CdXzSsUk+Zp2iYFv8zrefJ16s0Hv0h3l2tMvj+iRvcpVXNyq43/waIgoIHvo4p6YXeLQa4zmjJCrkqjXPlVaBWzdmsW5eRW6cN5tPKjOI2RGqE+tMlxuM+zBm5fHtqlkyi7HVx2nHuDIJaMXQiC3akUu3l6PXyZF2fVTHgV7XyAnj7XLC3SJmpdixpcfyBzSKtl3TzPuDzD32ZbKjeQ+s9Tub9hAVstXId5p6huJnbxty3s6V3w0d77qqbRfVy4ehPj7e4T+b/DyrFw7T+hXFz83YPFyO+Y3f+hHinsf4L6wxbvt0qHP1S8+RnIXwB77AExNLPMHHWLEWebvdZfPFI1iXL+DlZrGPxRw9epWjBY9JcRRNTDtd4Xo3Yr5VZvXmFHLhBnb9BlqnSLtIUpmF8RrueINqscWo32XE01QyH4NnFbPjNtuXid6iUTDbpgKdDBY9BNmihyi1CBOjrjJLHjJTT+ig+1RKP2K2vy8zSW7fl/keePD9RQ/3vx7sQgdtBizJ9EdwZEqYbJC0c8igx5EilBzBmc084WYJKz/HTKHNdM6mqZZYCQTxfBERdBGFOaq1TaZyQbYyTZp42Mhio1WCegOrt4kUNtqroosl3JJJeyvZiqIlcaRB4dkrQ+tkyBBhspX7csI4NjGcOnSGNpj0bcbxHXm9ARcusyyLwfCyj8SHV6ztgsJhG41innTIOj9s5nkXWu53EjE7eO73YSv8XvWgPgQOlRv8wNQSGxs1bl46zOnRNaZyAS9vlEgCl/DEU1RdiYXDjZVJgpUa6fjjjFU2mfU9It1jTW4Ybns1MTx1pUJxos6Ip6mqMgCx6tHQPTYjh3anAK0A2a6bjPCMRtH5ArIQ43khvhOTszS+JfC0hy085GDVGmxZ62PjziQhISXVmlRDqoWhh7Qg0Rn6Vma5sRosORaZlLAvJ9yiUbavWsuOwaFAq200yo5wq70a9n7E7P2rB9rA86LC4UKMvfgSrpUynXuKpXNHUN9c4KePXeHxasSvtS5z69Ih0nCNJx66wGcmW7SjZS50elz85lPIhXkcf4qJ05d57MA8M+IkvlWhEV7jWhvO10eJrxVwbl4ybwq3RnzgGP6BNcbG15nKRYz5gjITeFaZvpxQ6R5dVacpN2nGimYsaMYOnSwbJe7kSLo+tMOtTT1RB+K2kRLuwoULYW3nwPsIfLBuzdlq5ANTj7Vl6pEgrOGNPMMc+ND14aGl3ELawwh5Jwq/U+1l6nk3zfTDIid89rmX+dTff4nv3DzEF18/zQ/81B/y7OHL/O/Nbxv7++zneLIWcVKd5PdvTHPpzYfxi8eZefgKn53sIZAspud47eoxGm8dJorWSQ4cJvfkBo+Uuxz1i0iRI1E9FuRVbnQdrtdHieeLWEsLxOEaAFZumnRkEjGZpzTSYKTUZMRNqLhQJkdOlDNVS1+NorebenSXngjpZdko/U09UWoRKWn8DX0UnmzPRhnszBzKRhmg7l1MPYOI2eGMcLbkhIPL+/W+1QNt4GXb5kiphfPlLwPwnHyY5fUx1t86xsHDN5gttFlPrrDaqGIvfpfS5DpHauvYlk+XgPn1cfRyhEoD7IkOo6Mbxl0pyqSqx2YSsxo6hOsVWF83g0xho3I1ZCnFL3UoOjElpy8nzLG18FiR6szUo5RZFqsEQZoFAWURnMQa0kwnm4Z7DmzkEP89LCdk53kfhcN2U48Q2/864nY54c7amRFuzm9v2Dtdmdsa/btceHw/Ef+7fe73Uk6pQ3T4FEuBy7VOinUEKrVN2vEyjU6RtHmR2UKbowWP6x241agRRWu4tRbHqhvkKZPqhBsdozBJu/No14fxMcZyXcZ9sK0CAssAhQjqoU+0WYRGE50YICAtH+UXoFDELfTI+QE5OyFva3xp4+gMgW9D4aCHePDUYPCMA88ywgeGHmHSCTM5oU7759aWnPCOKPzd8dT7C4/fn3qgDXyi3OTp4xf51r/8UZQW/JVjG7y1OsnXXn+C8T93mdOzN2gFl7hUH8X9ztfJP7zII4+eJW+P0pQNvrMyRvfqJEl3HnF0itGHr3Gk4DGmD6B1wBIbXO9Y1G9NkN6ENEM1qnocJsvkJjeYyHUY9xIqukROVLbkhAzLCWM6iZETdmKHbuQRByYbRQcCgnAQbjWcEb6XnNBcyGiUnQuPB+FW9tD5cLysZvu6tXe/8Hj769rJnd8b330v9WEYVu4su9ohPfAxrrclb3CB6InnKc2tEMZrLLfK5M59lVMz83xivMPL6jJv1Guw8jL2TI/HHnubcTWGJuXNTYerizM4t15FeSXCY09xaGyFI8WQgjOOEJJOtMxiEDPf9WmvjJIuWYju4iAjPC1NoaqjeNUWhVKHETek7KSUbLNqzZNFAxCGXZkZhWLCrfpywmzVWuYwNijcJlEWaWqbU2KCrUw2ynC41dbCY4aa9x0zwvtywnvICN+v+1P33MCFEJYQ4lUhxO9k148IIb4jhLgkhPg3Qgj3bs9h1xKq06v828sHAfj0J77FQtfj22sV1KHDjE6v4NqTrAQu9ReOIEZ9ig/Pc1CdxNM+ZxqCjflpnMVXSEcmcA52OFJMmaICCJqyzlIvYX19hHBpBNG+OVCjmIzwNtV8l4obUZYORV0xA6HsjWAWxnbpEhg3WyroJTa92CHKMsJVz4EgMeFWSWQ29WQZ4XvJCfvhVsh7lBMOm3q2ceHbM8J35oRv/3ttoeuduSn3Ug86I/x7PfDcPHuY3C//P3EtOJIewb1xnrTnciT/WVqxS/h7DcYOLPPM3DXaeo35joX/ypfAtag8fpWDboGiPcH1bsjVZgXn8tuIuEtaO8L4xCoHi22q4gCuVSBVHeq6x1ooadQrRKsVrNYSOqpnc5sKaXkUq9w13xrdkJKdUrAFheGMcPoZ4QqjCd/KCDeberRRo/QRuBZGSpjJCZPEGkLi1mBfJoos3EoPxctuzwg3/+xdmvE+v/2+1ztB4H8HODt0/R8C/0hrfRyoA3/jbk+gy9PkJjf4nc55AEp/PmEpkLzRCIinjlCYWqfkTrMeWly+cBxKZeRDI8zZFRxcLid1VtdGsW9dJS2PwcwkM7mAMc9GYNHTDVZUh7V2ie56BauzCmnPrK4qjyBHoJTvUHVDSrZFQRWwZY4+l6hUlCGYcLBqLUgtwtQ2OwVDFx3b21at9WmULc5vbznhQJECu8oJ9U53ZvYX2s2YM3juOyQS7pVO+G5pkm3P8R4GmR/EjPDlhSkW/vhxchYcyvlwc4mk5/OYPEwnsbn8ymmKsyscPHmFMG2yEiiSt5WZWZyaYzovKDPOglxmoeujr7eQUQ8rP0dxaoOpUoMRNYori2gd0pRN1gPNZrtEUC9jNdcgbpsX45RIc1VkKcYp9si7UUajgI9jaBQ5RMtB9u0vRmUUSiSiTE7IQE6YKEmit+SEpnlbg0UPOpFbcsJ+NooecmT2a0ew1b1urL/X2s8Iv/e6pwYuhJgFfgz4Z9l1AXwO+HfZQ/458FN3faJwk8aVGW4Fr+NaCd1nf5oohVtyBffWZWQu5Ie9j9NNBb995QiEAfHBh/jomOKQVeWy+i7n1iZJX1xHOXmigyc5Nb7EsaLGsceI0jYL8irXWxWWb03h3LqMbN3CsQqDjfXVkU3GC21GfUFN5PGscpZOKIwefMfG+s6OjfVJ10d1pNlYH3S2bazfa5Bpzrc21uv+MHO3YKuBO/MuG+uH0wl3cWXub6x/Z/UnVx7iv/ryx/nIaIu/fuImZ//tJ7l24Sj/0Umzl/VfnXkEe7RN8RNrjNiH6KQpb3/5efR6SO/0Z3m82uG0PMhSeoFLLcH6Kw9hL93Asov4x1aYm13gkFNmVMwCmk2WWYgCFpsV6svjWCu3sBs3t22sF6M+7qiRE454AWVHU7RsitrseDUb64fUKPruG+sjtbXoYSAnjK3dN9YPywlVasLc3sPG+v2B5v2ve0Xg/x/g/4r5XAYYBTb11l/kJjCz2w8KIf6mEOK7Qojvri+3aayNYEkP107AKWFlr0As3wIlebiSojRcbAHdHsovcrDQpeZKevEaa4FPd2EcoVJUYYzRWp1RL8GzS6Q6oavqbEQOjU4R0agjwyYAyi+jimXcfI+CF1CwNb6UuDI/+DVoDIKJdJeIhEjpzI5s5FfJNttx5lJLojsetMCWHnzbjdmSh51NHLZolP5fqH/K+O5tSx6Gf9d32MqzV72Tx++VYPhh5Lx31kLX4dvdJWbLm5w4epWzi7Ostso8cvQKUmjebiiwQU9OMZqahMHL6xOojoMoHmEy32bCl4Rpi40opb46Ci1z7ImaR2F0kxFPUFQlQBCqNk3RYTPyaHfy0DJBaVqnhm5zSpDLI3IxvheRs2N8S+FbEk+7WJmccPgtrPuSQjPCJM3khP1slFRnA0wtB1JCrSVKWeZyf9XaHnJC4PYh5vDG+nus/SUP96/u2sCFED8OrGitX343/4DW+ota62e11s/anRp/dO4UP134CQp+QP5r/wuzecWj1iyrv3OIzs0Jfv6JVyk5mq8l3yW+6GJ1G3zsxDkeq8Yk6Sbnmg7n3jyF1VpFFSaZOnWFh2sbTFsPI4WkEy1yrW1zdXMEfb2FvXIDpRN0bpR44hCFqXVGRupMeAmjnkWR0WxXZiYnVAGBbtKWbVqxop1I2rFDJ/TpBT5x10N1PehGRpMeBcioA2l/kLl7xOxgyYPlm2XHQwhcD/Tg1nYEPqQH37nkYdum+t2WPAylFt7+N739tnelDvmA68HfCY2y2INzwZd44pnXmfyFRf73KyVeWRvjwH/cpujEfCX+A1TDJR2Z5BOVGiOOze/crNK+PINlFzl15CrPjXZJVciC3uC1G4dJL6ZEvZuoQ0cpPLzAiVLMjFVGCI8wbbIsrnOz67PYqKFuxVgrC2ZLj/SxvDHUyBhyTFIst6jmu2YxiSMo4uHJIvZARWUoQK3jTE7YI9EhPRHRUylRqgnSbMlDhsDj1Dan2DZceGyjkqF9mcneq9YGPLhOtuSEsF1OuGPJw/1KJ9yv7XUvCPwTwE8IIa4Bv4yhTv4xUBVbsHIWWLjbE7Vim4Wey2cne1hWQu/rkik/4kRZcOPaQRqrI0ydukLJUbTiJXrLI1grC9QOrDCd6yKEQzPS3GzUsDZXQSW4k5uMFlqMqhEcmUepiEak2YhcknoB2dwkSTtg5VC5EaxiD7/QpWCnFGzI6Ty27G/pMRnLqY6JiTI5YfbVs2/oiRxUlPHgw1u81e6cX3/JA+yw1g+29ezY0gNsc2UKMXBO3xZQtXOQuZd9fgeNYh57Z9v8/V7y8H7/7HutqiuYzD2ON9JE5wsspm1aiUV46HF8KyVOO8T1EtbmGsdLITVPcL0X0FqvknTnKY/VmS02sa0cPdFloZsnXK0i2vOkxSpi1GPUi6i6FpbMoVEEqkkzkmyGPmkzj+i0SOPNgZxQ+wXI+zi5AN+NyFspvgU5aeEIH2somdCUQqMyHrwvJ9QkeguBx1k64XY5YZ86ydC3Fmi9PSO8T5cIpXanUPaSF96jfHC/3l3dtYFrrf8LrfWs1vow8PPAl7XWfxn4CvCz2cN+CfjNuz3XWrLBhabmF3/2N7GtlN/5/R/kkdFVfmjuJn9w/TBXFmZxfnqWKT8iiJdYuHwIztyi9NwtTk4s4VgjLEYhL69XEAsLiLCBfGiEAzOLHPEL+KKM0gGLUcjNjkP75gQsr5NEmwjLRxdnsSZi8iMNRv2Aqquo6jKuLG4tedAJUdomEB26Ohnw4L3EIYhdksAlDTwjJwwDo0aJO4i4sy2dcGdt29IzHG61jUaRty88HpYTwnY9+NBtQg7JCXfhsu9Uu+nB73fdD9rl/Wzwj5RDfqb0OM5kExGFXJNnacYCd/aHKLsBAPVrBxCXr/LJuWscK8a8qr/O/KJRRRUfnufhw9fI26MEosMbmw4bN6ZxF8+SVg6gDh3lSHWDubzGs0dQKqEbr7IcCJZ7OXpLo7DeHCRpWtInKY+hKzX8aptioU3ZjSjamoJt4euCiUUe1oNrhe5nouhwsOQhGvgaGOjBY2URp5ZJJkwlaXauBhRhxoMrvXs64fCC453Lju+hSd+NRtnXg99bvRcd+N8F/lMhxCUMJ/6/3u0HhLCYykmcHxwljDx+/1aF2YllTj1ynjfrkvlmhfDgs0zmu/jOFDfWJmieOYQ+dJip6WUm/IcJibnahuS6jbN6mXRqjtL0GgcLUGEMgaAumqyGguZajXTNRvcWzX/WrUK1gFtrUckWxxaEY2RZ0qggNXowCAqIBnkSvdgmjB2jRglcVGhDFEMmJxSqH2p152XHA1v9jl2Zt2WEg0Hg/bptmJk151020u+VFz58287LHxQ9+PcKhT89dYt//9FzhDdH6Pyh5Ig6xZiniK/+Gq6VcsL9FLcWp2i9MsPhE1c4UmrRi9dYaFZx3nwJOSYZPXKTSXEUG4/rvYCV1TGs6xfRtkdSnWCytsFkLqJsTyGlTaoC6lHCemjTqZdJ1x1kZyXLs7fQnpnb2MUufi6g6IQUbEXeEuR0HlfkMz14ljnfj5jVyeBbZExKnNnqTbCVaeCJkiTKQilp9OBqy8yjU4lWmZwwSbfkhMPbeYZzwmE7jQJ31YPv1/2pd9TAtdZf1Vr/eHb5itb6Oa31ca31z2mtw7v9vBAWE35K/NG/RTf0+FL4GiPTq9Q+cokz6SLzXR9/9COM5DpUnTmut8rcujZLNHOS0oEVZtKDhCLmatSktziGtbFMPH4Ub2qd6VxEWZVASDblOuthwmajQrxeRnaMksCyi+hSBavSo5LrUnISiraFT3EoFwUzyFRdQhFmdmRBpMySh/4mEx06EMXb9eBqb1dmXw9ubtiZUpih79tWrPW5cLa48G3NWbGXbHBncuH92NJzx8d9yAeZR49e5dG/9FU6S6Ncev0Uh90So16Md+YlPCfmUWeSm40aty4dovTEdeaqGyTpJou9HPE5B8bHyB1fY0qN4mqXm/IWy60Ker6DljZpZYba+DrjXkCZMaP3Vj0aKmIjErRbRZLNAlZnHZ0GSGGjvAoqX8Yq9nDzATknJm8pPItBLooQ9tC2epXNcRJDo4iYSMQDOWGizZKHWG/RJ6mSpINUwn4Tl2bZsZGXbx9iDtXOZMLbmni/7jFedp8Hf+f1QJ2YB5w8P370Mlz/fW41qqyHl8lNrcPxGRqssRZaBPVXGa/W+bTzBOebPt+5fhSVr+EfXOOjlRKedrgq3mbx8kE4N48qH0YedDg1ss6MU8CxRmilK9zSTRY2R2gtjuGsXkFHdSzpo0YmkBMW1VKTUS+g5krKupqFYpk3gtJmENQRbXppmtEoNr3EbKuPex5pz82+k0bIKEDGXUh7d4yXhe1USn/F2jYE3s9FGQ62ynjwbfngw3z2zu312xD23Zv0u+G378eatQ+SHrxydIHej/0i33z1Sf7F2RP84tENnppY4tIXH0YIzV8+tsb5RpmvXD6BfvwEc4fmce1JFroOl771JNrPo08c4rGKx4QaZyk5y5VmhcZbh5HdDbQ/RuXoAodqGxzUk/hWBY1mVa6x1FOsbtboroxgr92C0IRbkRsnLYwiRxR+tUUl36HsxlRcTR6PvC5lg0wbBvngGqVjY+jRIXGmB4+UJkyzUCslCZW1DYUPomX7uShZRrjJB9cmXjbLBh/mwbeyUHbQKFm9F9S9rwe/ez3QBl5yQw7MLeCee4VG7CKERDgJ2s8hhUWswFk5g58LOFbS1CO40TGRr6IiOVSIcbDpJKusNyvEy2WEXYRKlbFSk7IjcawCserRFk02I49eu4BsNyAJEMIizZehUMDPBeSdiLwNnnawhTf4dejMEBGLkFhrIgVhf3qfbNmPdQREEQySCbPmvceatW2ZKFk64YBGgdsHmbCFwIefa3h4eYdB43A++G45Keaf2cvsszdC/7Cj7d1KejG6OMfFZpmzzZhTszcYq2zyxvxhpNA8Mned9dDiatslGZkmX2uSd8ZoxYIba0ZWmFbHmcollKVDGNdZD22a6zXz4S4d7JEOlUKbqmPjiSIAPdGhk6a0Qp+om0O0W2aeolOE9NF+GXwbyw/x3AjfSnGlxhMWrnazXJRhiaqBzRo1NMhUW+mESpBo0DtQuFLS0CfapBLqgZSQ7eh7uHFntaseHPb14A+gHmgD92cV4z9wnm/9tx9jsefzBe8nSJoF5IVLnFInKNgafuU1/EKXv3D8EstBzEsbCnfxLBTzfGruGpOOTxivcHZ9nOW3j4FOSMemOXj8KkeKKWPOERLVY0Pf5Go7z9LqOGJ+Hqu1iNapyZkYm6A8Vmes2GLUS6nJHEU5mtmThVkWq3r0dIO2Dukkyhh6Eocg9Ah6PknXR/esbM1aiIx6iLhtbPV34cEHQ0wxNMjclouSDTKtYVMPDK9Zu00PvkvyILC9wfezUvqPu838s33N2raXfI+37bz9w7JmbfP8IfL/+h/yworknLzE0b/8GgdOXOV/uOijtODwL51jORB8a7ODtl386XWel5+gnWi+dGsCsblBUpvloxPLHCvZpKrJlbbk7PXDOEvXIO0hTkwxMbvIkSKM6CmjqkqXWKLOQqfI+toIrGxgN4yc0LKLaH8MRkdxRlpGTuiGZnZjWeR1bhCLvEWjaMOBqzBzFfcIiQmVHvgaIiUHgCTJBpn9NWvpwNAznIuid+Si7LLcYRiF32HBw265KPsN/d3XA23g2vHREbxwawal4fkx6K1X6J0b43jJoeoqlt86DsDs4XlSNOuqi3XrOiQJU1PLVF0BSFYCj42NmlkK6/r4E3UTu6lG0VoRqy6bkUWzl0M3FDJsGT24U0DlyzjFHr4XUrAUvpR4OvsqKmQWDmSGQTEpsdJDGtrsYB9M6/W2Dd59CmW32paLAgMufLuF/s6DzN2GlduSB4c58nsItbofdb/XrN3vuhcaZenmNPN/9BSx1oynU+jpGaStuGJdRmtBcuwUGk1bdLE3biHslIdK5u93o6Og3kAkIZO1Dca8FJC0E8VSt4CoG5t8Wh3Fr7YZ9RLyOo8UHokK6Yg2zcihG+RQLYkMOtmiY7NPVft5RC7FdSNyjjH0eFLg4WAJByGyDJ1+aYXWJp1QoUhElg8+MPQYSWGijalHDS073jUfvL9mra8H38mHDx3vuxl63outfq/ap1FMPdgGLm3CG6P845XXcaTmr3/8m9y6Pssb33qWL0yvc6TY4ffeeBKdWoz+2AISwYK8RvJGF9ohkx97m7lCiiVzXG5ZnFudwqpfRbs53Id7HCy2mJM1hJAEySaLXc1Cp0R4awRrc8XEy+bGSSpTuKMNyuUWNS+i4kiKqoRt5bKAIIXSEWHapid6dLM1a93EIohdotghDl1U4KJDbQaZUWDWrGWDzJ08+GBH5vD5YM3a8Mb6PdasDYKthmz0cNu5ee5dlhoPrVm7jYt+H9asvZd42e+FEuWrV4/z33z9ozhC8Hy5Rjx1GK0EN9vfAECd/DksYQLTrPNn0Uryuel1LCF4WV0kvp7H6qwzd+oSJ8ptpPBZSjqcbeTR803sxk3iyYfwZtc4XOwwYRVw7BJR2qTJCkuBzUqrTLRcRW6uocNVhLDNWsDyCKJi45U7FLyAkhNTcMyaNU+YWOStSbdxFGudmHhkHRo1ilYZHSiI0r4KRRKnhgcfjphVsT3Qhg/WrG1ToqSDYCsG+vAhQ8+2dMLdpYX3umZtv+5cD7SBy7UVzr/4BK1kiQO5gMkvnOHq2iQvLh3g8cNXmCk1+PaayRyJTz5B2baxhEP97GHSFQdxYpzpXEjRnWE9TLnRyeOsXkNEPfT0ASZKTQ7kLGyZQ2vFWhyzHroEaxXk5gZpuDbgFWU5JFfqUHEiCg7k8bN8cBsTkq+yRbEmUyJIBYGSBIkJtuobenQoMx483r4ncw8e3JwPhVqJIRQ+rAcfttYPLXiADGn39eDAtlyUnY18Jy++E7XvsY3+XuiRd1Mf1FyUGx2HF7q3eGpE8KOzK7ivf5P6zUny3hxCaJLNN6m5MJ5O0XtjjHi9zKMHr1J2YCO5TufmBPata+QOrTBT3sR3x+mJkJsdCG+NGEdwYQI5JpkqNhlxLfK2+bYYpk02I8Fm5BHVS4hWAxEaQ48QNipfNoaeQo+8F1KwY3IW+FLiiK188EFptZUPTjwYZPb3ZCaagaGn37yVsrKmndnq+/GyWpg8q2FL/Q4K5Z4MPe+y9vXgd64H2sB7jRLfunmIIF7nQLFF74d/gUuNMi+uOcw8c5apkXW+E94woVGHP0fVFVg4zM/PEq7UCB96hul8hxE5x1ra42bXRi4vIJKY+MAxRioNJnyFLT20TqjrDvXIpl2vQKMBYd043LyqQTTFLiUvpGhr8sLGymRZkOWiqIRQ9AiIslxlMRQEZKMiFx1ZgwUPxlq8RaHcCU2IAQe+xYMDW00cdqDwjEqRw8+xXSo4qCGuGzK54V2Q7jvhuN9pfRh48IUunOv+Lk+N1nnuqdeI3nJZXRln3DmOFBpn9SIjXsqkVWTx/BGCjTIzT52j5Gg60RL15TFYXEUcrTE5skHFniEQXW5GPbrLI8j1FURuGmpVxiqb1FzIyxpaJ0Rph81Isxm59JpFaLaR3TqAWfDglaFQxC728L0go1H6a9Zy2Zo1e2Do6eeD92mUZMcgM1F9CkVs5aOkfSWKadxoCXp4kKl3ZKJs14FvM/S8g9rnwd9bPdAGvtjx+JMlh58o/BWOz/3/2zvXELuuKo7/1rmveT8yk2amM0kmE5JoULChhCpt/GDRJtimVZCI0IqCCAoWEYkEpF+r6AdBLIrFKrUtUosBEapSFIS0NjFpUpOYSZPGJDOZaSeTZOa+71l+2Ptmzty5d5LU5Dxg/+Byzqx7Zs5/1j533XP23muvc2iuj9PzGU6UL5Pe3sPA8DRnSge4cq0bgI3dVTb663jj4lpm3hnFH9rO+sFpPuqNMevNMXGtQu14ifT0eaqrP8LA3ZfY1J2nJzWE52WZSU0xVRCmZwbxp2uk584CINl+GFhFbuAK/R3z9GVr9GfStHu9pFNmZUJTZq1kS1QVyFex64NnKJazlOsFHkoZU+ChPpBZXx+8RRD3GmeiBBJ6NNCNsrgmyvJ1UYJZmGZg0m+a0GPeXxqkWw1SNh7T6r3F/6P5MXHtB78Rwx3C5o6d7Pj4AfofnuW1Pz7IwclRHu/fhCc+/gvH2dIzzycGlX9MbGby3AjpTw4w0l5G8Dh9cYSFI0NUxreyev0FtvgfAuDd1Bmmzg/jTyzgpUwB77tGJxnpqHBX7W5EPGp+gZlKielihrnZPvz3IH1lCr9mMkD99lX4Pf2kegt0di3QnSvSnfHpTAtt1xN6gtXqAXxqdnnkipTMQKb6lO1UQrM8RIqKbwcya6nrCT1+zVtS4KExoWdJP3iThB6pr4niLy/w4NZFub2EGsAXKHKhOs9Yl0dn7zVItXO1osx5s1SH1pFpL1GpzlKumQDXm6nSlUozXcywkO8g1zZER3uBvoxHSYpc0xKVuS4oFvHaVpPpLNCTLZkBSVKUdJ5CVcmXc2ghg5QLACatvq0dL1chly2T85S0J6Qks6QUmikSW6EqVaq+UvHtHYt61AJZa41pxsD1rMxWLJn6FSx0vOzAQBN5jdvWc7yXZGvegFutk9n0b9zmha3C7mrpSClr/EG6xi9SGf8wE1f6mCll2didB+DquSF6skXWtJeYLORYKHRQHt1MZ9q08VypnfzlHmqdg2S68/SmTWZv3r9MvthObd50z2lbJ9nuPF3pGh2Y6vKqPgXKppuulENLGaRcXLx+0m1oOoPkfFLpGtl0lYwoGQ/SmlocyLSV6rGLhio1VGtmfRR8fNXrtRpUMXmbKvgIiuD7gtpVCmkczIRl66Ism0p4mwo83Cwu0IOohvdBEZEZYAF4L7ST/n8MkhytkCy9SdIKTu+dJElaIRq961V1daMx1AAOICJvquq9oZ70A5IkrZAsvUnSCk7vnSRJWiFeekPtQnE4HA7H7cMFcIfD4UgoUQTwn0dwzg9KkrRCsvQmSSs4vXeSJGmFGOkNvQ/c4XA4HLcH14XicDgcCcUFcIfD4UgooQVwEXlIRE6KyISI7A3rvDeLiKwVkddE5N8i8raIfMvanxKRCyJy2L52Ra0VQETOishRq+lNa1slIn8WkVN22x+1TgAR2RLw32ERuSoiT8bJtyLyrIhMi8ixgK2pP8XwE3stvyUi22Kg9YcicsLqeUVE+qx9TEQKAR8/E6bWFfS2bHsR+Z717UkR+UwMtL4U0HlWRA5be+S+RVXv+AtIAaeBcSALHAG2hnHuW9A4DGyz+93Af4CtwFPAd6LW10TvWWCwwfYDYK/d3ws8HbXOFtfCFLA+Tr4FdgDbgGM38iewC/gTJu3xPuD1GGj9NJC2+08HtI4Fj4uRb5u2vf3MHQFywAYbN1JRam14/0fA9+Pi27DuwLcDE2rqaJaBF4HdIZ37plDVSVU9ZPevAceBkWhV3TK7gefs/nPAo9FJacmngNOq+m7UQoKo6t+B2QZzK3/uBn6thgNAn4gMhyKU5lpV9VVdXLvhADAalp4b0cK3rdgNvKiqJVU9A0xg4kcorKRVRAT4AvBCWHpuRFgBfAT4b+Dn88Q4OIrIGHAP8Lo1fdM+mj4bl24JzFIWr4rIQRH5mrWtUdVJuz8FrIlG2orsYekHII6+rdPKn3G/nr+CeUKos0FE/iUifxORB6IS1YRmbR9n3z4AXFLVUwFbpL51g5gNiEgX8DLwpKpeBX4GbAQ+BkxiHqHiwP2qug3YCXxDRHYE31TzjBerOaIikgUeAX5nTXH17TLi6M9miMg+TAmG561pElinqvcA3wZ+KyI9UekLkJi2D/BFlt58RO7bsAL4BWBt4OdRa4sVYlbFfxl4XlV/D6Cql1S1pqo+8AtCfJxbCVW9YLfTwCsYXZfqj/J2Ox2dwqbsBA6p6iWIr28DtPJnLK9nEfky8FngS/YLB9sV8b7dP4jpU94cmUjLCm0fV9+mgc8BL9VtcfBtWAH8n8AmEdlg78L2APtDOvdNYfu3fgkcV9UfB+zBvs3HgGONvxs2ItIpIt31fcwA1jGMT5+whz0B/CEahS1ZcgcTR9820Mqf+4HH7WyU+4Arga6WSBCRh4DvAo+oaj5gXy22FJSIjAObgHeiUbnICm2/H9gjIjkR2YDR+0bY+prwIHBCVc/XDbHwbYiju7swMztOA/uiHLltoe9+zCPyW8Bh+9oF/AY4au37geEYaB3HjNQfAd6u+xMYAP4KnAL+AqyKWmtAcyfwPtAbsMXGt5gvlkmggul3/Worf2Jmn/zUXstHgXtjoHUC03dcv3afscd+3l4jh4FDwMMx8W3Ltgf2Wd+eBHZGrdXafwV8veHYyH3rUukdDocjobhBTIfD4UgoLoA7HA5HQnEB3OFwOBKKC+AOh8ORUFwAdzgcjoTiArjD4XAkFBfAHQ6HI6H8D6AqXsgZlUuwAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Sequence length 49, with token dimensionality 192\n",
        "plt.imshow(build_1d_sincos_posemb(49,192)[0], cmap='inferno')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "000a8f0a-7101-4bed-86ef-52c124e9a653",
      "metadata": {
        "id": "000a8f0a-7101-4bed-86ef-52c124e9a653"
      },
      "outputs": [],
      "source": [
        "def build_2d_sincos_posemb(h, w, embed_dim=1024, temperature=10000.):\n",
        "    \"\"\"Sine-cosine positional embeddings from MoCo-v3\n",
        "    Source: https://github.com/facebookresearch/moco-v3/blob/main/vits.py\n",
        "    Returns positional embedding of shape [B, N, D]\n",
        "    \"\"\"\n",
        "    grid_w = torch.arange(w, dtype=torch.float32)\n",
        "    grid_h = torch.arange(h, dtype=torch.float32)\n",
        "    grid_w, grid_h = torch.meshgrid(grid_w, grid_h)\n",
        "    assert embed_dim % 4 == 0, 'Embed dimension must be divisible by 4 for 2D sin-cos position embedding'\n",
        "    pos_dim = embed_dim // 4\n",
        "    omega = torch.arange(pos_dim, dtype=torch.float32) / pos_dim\n",
        "    omega = 1. / (temperature ** omega)\n",
        "    out_w = torch.einsum('m,d->md', [grid_w.flatten(), omega])\n",
        "    out_h = torch.einsum('m,d->md', [grid_h.flatten(), omega])\n",
        "    pos_emb = torch.cat([torch.sin(out_w), torch.cos(out_w), torch.sin(out_h), torch.cos(out_h)], dim=1)[None, :, :]\n",
        "    return pos_emb"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3fa585cc-4acd-484c-a409-61d52fe6b606",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 208
        },
        "id": "3fa585cc-4acd-484c-a409-61d52fe6b606",
        "outputId": "85638c8f-b63c-4fba-b8ad-366d85b3fa2c"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.9/dist-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3190.)\n",
            "  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7fa3e442a130>"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB3CAYAAAAaaxszAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAABB80lEQVR4nO29eZAk133f+XnvZWZl3dXd1df0XD0HDgIDECAEkCJ4ibpA06Jt2bJEH/JaDtpeO2yt12tLtnZXjpAjLJ/rXXstc1eMlby2KMuUTMohLUVRpEiKF25ggMEAgxnMPdNXdVXXmZnvvf0js6prerrnAGZ6uqfzE9HRVdldVe+9zvz1L3/v9/v+hLWWlJSUlJTth7zTA0hJSUlJeXukBjwlJSVlm5Ia8JSUlJRtSmrAU1JSUrYpqQFPSUlJ2aakBjwlJSVlm/KODLgQ4oeFEMeFECeEED9zqwaVkpKSknJ9xNvNAxdCKOB14AeAc8DTwE9Ya1+9dcNLSUlJSdkI5x289nHghLX2JIAQ4rPAJ4ANDXjZzdhJL/8OPjKlTzbbwdk/hUDc6aHcFdhLp1mple/0MO4KhLAUdncRhek7PZS7hmefPbVgrR1fe/ydGPAZ4OzQ83PAE2t/SQjxKeBTABNejn/74Pe/g49M6XPk/mOMfebvIsU7+ROm9LH/+a9z8bcOEEUORku0djBGYIxEGxl/1wprBdYKjBUYKzE2/gfaP2aT5/3j/Z8Nfx/GXOcf8Hqv2eoIYfnw//T7tJ/8KHgVcLII6SOVjxAOQjoI4QzO3fhmPiY9n9fHkX/+9LrHb/cHW2s/DXwa4J78aFq3n7IlaX/spxl5/BXcxTPIdh0xfwnaXWzdYFoepusRNXPonkvYyaBDBx26RJFCRw5R6GKMQGuVGHw1+AfQN+yWVSNv1hj1tc8Hj8XqJbOdjH3nOzlytV+FXVVsLo8enUTnyphMCZMpg1tAuEUQTmrY3wHvZFXOA3uGnu9OjqWkbDuE9DG5UXq5UUTUQ03NI7tN1PICqtlAtZq49TlsW6AbWUzPJWr7RF0PHbhEXQ9jFEHPG3jwWseee6QV1kq0lld46sMGXazx5NdirUBypf+z1qALceXP75RBt1bwwje/h8x3Q/bsOUd2ZIXsnufJjIUwUsCOTWByBaLKJNbNo7Oj4PjgFhHSR0gHmXzvG3RtdWrQ1+GdrMLTwGEhxCyx4f5x4JO3ZFQpKZuMVD7aq+AWZnFUHms1UdQkaJ9FNs8jWws4C+eR7SbO/AJ02jj1JqaTQXc8olYWHTo4HR8TKqLQJYocdKRQkYO1gkirq0Ix2kisiA143+MefryRMQcGBn0jz3zYoG+mMTdW8MUze1gOJI8vTDKdbzJ75jyViUVyk0t4e06iii5yuo7JFdClKiY7gvVy2EwFq7JYpzAw5LZvyAfzUhgbAakhf9uzt9ZGQoi/CXwRUMBnrLWv3LKRpaRsIpmv/itaX5S4TzbQe2bpHfoQIjOOW5jFZKfRI02ikbPI9hLu6Blku4msLSAbTVSrjVNvY3ou7koeEyrCtk8UuJjIIQwdjFZEoROHWJIwi7UCmRjz/nPLqjE3VqCEva4xH/bMr2fMN8OQWwt/tBDwqnyZlxtHmPameGhunNlCi93lZfbOnKcw0qAwewFZDlDTJWx5BFMox8Y8U8JkR7GOj0m8cql8zDphFjP0uTvRmL+jGVtrfwf4nVs0lpSUO4Z9fZ6jz/wQD2eexm+8ilOZJKp0EZkqrlsBt0JPOBh/nBBQuWWUlEjlILwWkh6iF4EV6J4HgFSGKIhNjJEmDpUkhlQIizaxT2lt/3tsyKWIDVPf4zTJz2RixEXfqCfvM2yUJfaasfLNMuTzYpmF9iu85Le5FM0SLe1hJSqxErlIYRlt5pFuRKa5gifriCBARhFIhYgCkAqru1jAusl6KH+wMIbYYNuh0Iqx0Y4z4jtrtikpG3Dii+/lf3h2lB+78Cc4MrbI+9/4GrkDf0Dw5IcIqvfilu8n408BEHgVwmAZXXwLVbmAai4j85cRnTYi00R1JHoli+5mUB0P1fHRoYNUJgmpaHSkMEbFm6BGIoTEWonQ9qosFwkDD3zYqANX/FO40dAK3F5DLgTc50zSy32Yk+2v0hCnOeeOcbT2MLO1KU439zGT6/Fwo8xoqc7kwgX8iRru+HnUdA1VLCGrLYyfR+fbGL+EdQtotxh7304BIZxBaIXEcO/E0MrOmGVKynVYaeU5KY7yfO19dM04+166h8mVcxQnXkREAaHyoTCLkj7KKQBg8pNgNACi20ZIiegGCGmwUTB4b6sVQliMlghhrkg3lHI4+8QgRN8Ig7V24I0PPHBunTe+3mtuFUdGDLmV/SyYWTrREr2wxrx6CwSMtmbomQyjmRGCyMHPduOxSIPrN8EYpJ8Do7FOfDdjAISDle4glCLxQRIbcdiR3vjdP8OUlBug0c2x1HmF/6prfD08QPelh3j32X18JPwOpXd/C/XuZbqHPoApzOJnJjEqT+QUCN1iEq/1UO0GSkpoNVFOG7GikV4UG+0gNkQ6ig2MloYo6hueVSMuhB2kIEI/vVBe4YHfrBGH63vjt9KIS2H5c+9+Pt7E/eYHebk3z9Hwt6h1jrHMCbrifezqzqBtmZlWjkA77GrmqTazlKOLOJUmkguofAGMRuQ6iKiHyUVYtxDPSPkYQNg4DbF/R7LTNjrv3pmlpNwEu6tzPO7/OMd5jqXwNE8vHqITVZh+5T7u0ZIReQzPzxNONgmqfuyJSx/jVtCAydfj+G17BSklRBppehhCZOAgpMWECqSDNasSRNYKjJbIJEYeG2SDlDL2wIk9cTtkgG/WE78RbqURF8Iy88hrOCMt/viZWaYWxlkQT1ALT9ML51iKThOqHhPNBwmNS8krIZOQjpvt4YcOmfwiIoiQXmbwvtbJxF65dGIjLpw4Hq67g1zyncbmzljcudSmlJRrMTl7jqcmjnBmoUy9e4Lv+t+kvvwo0xdmUNJwrx9QHH8NgLA4i3UrZLwxTBJO0dkxrHSQhQZIiQyCOBRAiA0CrLTIXgjCYpPME2CwkamMRmuFksmmp2EQThHCguWKcEr/+FYMp0gszvvLdB76Mb7/6FcYe/Ehjh59gJdduBTO0QouEagWp9xdRK0RKp6PKyxKGPL5NtYKnFILZbqIXD3+hygV1mtgTA7rxnIcVjqQhFGsia65uQl3pye+qTMqji/z4b/yBcyKxHQ8TM9Dt/1BdZvRiqjnDarbdKQGaVfGiityaIfTrm66um2I653U671mK9BcLlH9vZ9DlyqYQgWdH8N6OUx2DJSPcIuIxCuRygdIq9yugT9e495Sk59WD9CKHsJXhqKjmc6t0Ak8Tr16mNLZFTJ+D9f/DYS0tJTGJuehCZNinUihdQ4dzQ7O1+Fy/LUFPSbJQOkb8vXK8bfl+etlcEeO4P6tY3z40jN86JWL6HqWaGUMhEE4IW7pt5G5ADEioFTA5ouYyhjGnyTKv4vQy2Mz5bjAR/mD89lNz+cBmzpTW6wQfPAJnKWLOMuL0GpCLa5ui5Zz2NAhbGbRgTuobtORQxi6g+q2KFKDoojhi0EbObjN7F8MN1IQcTM5tP3XbAWCwMOenEOO1JAjI4jRFiZXim8xvRzWRliVRSgfa6OBIR/2UGA1VriTTvr1cEZbzI4ssH9kfU2TZjdLo5Mb/P62NKqbSdDDmgh53yfR94H8iIMCnGsY2iTan3IT7OyrNiUlofujn+SRR59BLi1Arwu1dlw238zE1ZbdDGE7gwmd+G7RKMLATdIBZSyCNfC01TXL5tcKYPUf99nuolcGQfj1FbzFn8NOTsVl82uqLVFZxFC1ZaqD8vZIVyYlBaA4S2+fRk4sIYMOaukSotvBqdeg0cB2LF4jE2ugJKJWUde7oWrLa4X9hL067Lc2Fn0jd4pb6S7RWsHlVw5SvLhMYf9FVOk0ajoHxRKmNEJUmcC6OXRhHKuSasskRDJszCE23mbN+6cGfZV0JVJSAOkU0IXdyIn34qg82vTQURPdPotcuYBqLeAsXEA1G7iLC9Ax6LqL6XjoboaomcWEDkEng9WSIIjDf0aveudR35Cv1UK5SYMu12h6Xk/Uavj9NgNjBb/6zGM0AsljYw0m801md5+jPL5Eduos3swxKPvYyV2rWiiZEsYvYbwKSPea3vlOL58fZmfPPiUlQb32OTLPPIN+6AF647PY8YdjjzA7jVFZTH4C42VR7TrKzyLaLVR2GdlqoVpdVCZA9zykG2FCB9nT8V5OpJChiVMFE0MupRlscPaN83qiVtul8nI9XlmWnAoaLIdlpvwSD7YK7L60zOT5JapzF/BHG3j7TqKKGeR4DVMaweRK6Hw1FbW6CXbmrFNS1uA88wzH/suT3LP4Xfz7ztJ5vIQp7CGbPwiZSazV9LLT6KCGyr+Bai2iCpcR9Rqi2UbUmqieig14z0X2pWYjB9mLDbjoWYyRiMjBCIuUZtA4AhJvG4EVYluW0PexFl6MzvJG53c5JY4w1T7A2fYUh1YKHFge5f5mnkq5zkT3LG65idM9h6rUkeURxGiA8Qtoo7FOBuuFWCeLldEg1zs15KvsrNmmpGzA5a88wD97+kH+5OVp7pm6wOG3PovcX6D76A9gSvtxc7txM1WsVyGULrpYw/ELqMICsllH+POIdhfXXcF0XKQboTIZdM9FKo0OnYGAlQxWPXCZHFvbAGKtIV9baXgrDPntMuJCwCF2Uc8/xmL3DVpqnlpvltOdezixUmS+l2F6ocO72jnKlTqj9Ut4EzVUdQWn3cLm8ohqB+tl0fke1s1j3QJmTQOI1JCnBjwlBYBLFyf5cu8lxi4/xFLPp/riHOXWWbzJVwkA7VXwvCpCZrD+FNopENkIKxWOVKigB46DCBtIGUDfOIrE+04MpoiSSszYzqwxogowYGRsoIeKdyTrx8OvxY0oE94uIz6Tc6g372eR4wRhjTndQvshnc5h8o0SrUhR8qoEgYfnBRQAzyyjvGVEFKH8HMYPsSoTFy8BVrpxJatMMlVwsDba0cqEd/8MU1JugEYnx+XOd/lP9NhVP0g9+CBHXlngicVvk3voN4kePk133/fi5PbgumUcp0CYNIEw+QUcz0e2Gyh1EdFpo7wuIhMiO15SRh+X0+tQI6UeNHzoa5+ISKGFRYjVGgZtJAILyKvkZW/EE4c7Iy8rheWvHTlON/T4J8//WY7b87zR/iLznVeoybdott7HrmaVTjTBzEqZd/V8pmoVRicXKPYu4lTqSB0h8wVEFGByK+hsJfau+xooIjbeUvqDCkySddkJFZh97t6ZpaTcBPlMl5HsITq6xln5Gi8vfy9CjLH/xF5mcl28yutE5Ski6eIk5fPKKWC8kdgLzFYAkLlGbC67ASIySBMiM2H8Mzc2KEZLpDE4REOl7xKV6KEopQfhlTicsrEqYZ+tUkbff6+9B99CZUIeO7MXtbSbs+4UoV4h1MvMcwYEXOjsQgiPaquI74a4bkim1ATALbfisfu5uIze8SDIYDwg6mCdLJgIK6LBf7T1VAnvdlIDnpICHHrgOH/15FP830vPM9d+gS9geKP9KHCE99fL3LvyCnnxDaL9c/QOFnC8Cp5TRvoOkVMgAoy/iDAa2VyOPUFnBeGFKNNBuhHWCIyT3P4n4lZxWGQ1vdBhtZIYVj3xjbRQBp74FtNCqfzpDp0HPsSnVv6Q544+QP3VH+QV9yXmWs+y1H6NhjqPYz/AXHcUa8u0Qpd2L4OQlmJzmaJzAdVqxtWZUTiQ7SXqYRIN8H6jh2Fp2VQL5TZihUIXpzDZEUS1kTSNnUe2m7i1GrTbOI0VbMdFd1Zza/tdwGOdlCSn9hpdwKUwb6tg4nr5tbB1Cibmlkfofu4jjFUXKVZr5PecwCn1cGdKsabE6AQ6V8JkipjcKCgfnRm5QhB/uGDCiqu9lrvxhN8Ir9xib77LH+u8h0X5CCVXMp6xjHotuqHH/Fsz2N+/iFc+Tqb6DMI1WAeEATcCp6ewoROftz0X3c2ge0V0pAYNj8PAjTcvI3VF5aa1gijpm3mjlZv95+s9hjtfuWm9DFTuo/rUf+UjsxeYGX2U+cYMC+17cITBUYaJQoNCdp7q+CL5ag1vtIG7qwXFDHZ8ApsrEFYmMJki1i9ds4ITGDR1GOZuP4c3d3bSgdJhsrl9GBvFqVmdc9juPM7SiTiGuHARuVJHNZu4tRVMz8FdzmNCh6iTpGYlxlwbGVfCrVPKPLyjv94FsTaGOMza3fz4965tzIdftxmca1T4+ok9HChEHCyucM/UBcqVOmMHz+GW51G7L6PKJUxlFF0Zx/gFTL6J7RvyIXEgIRxQ/qCzSTy3nbWj743VmS3FX2vpRS6nLs5w6uLMHRjZ9sQ6LrncPvixf4cHPHiTr+9fSTsjEPL2ufuvzJSUG8B89BE+Uvw8ZiX2pKOV7MBp0D0XHcVOg9FyIK52pdOwqjy4ntMAd5HS4HUwCHj6JCb6R+jRyTivOz8ey8C6BcSadMBUB+Xtk65OSgoQHPwo0dg+nNpZZHuFzMJl6DShvoBpSUzLI2oOKWYmoREdOUSRQxTGIZAwdAeVlWvvBPvSscN3gjcT0oPrl81vFYPePDFDPrqAs/sC5POY6kQse5wrxx2M3Dw6MwJyNaSXdp2/edIVSUkBlFtBF2aIqu+OjUnUxIYryOZ51MplZLOGnxj17OIctivQjSy6r4XS9q8w6jcigfx29MC3ug4KxHP5T1/6frpa8u7xearFBtN7zpOrniQzWUPuklAsYqqTmFwRnR9LGhfnYU14b1jL3opVCeQ+O92o7+zZp6QkmNrL+Kefp7fvEUxhD45XxTpx9V+YKSPzS1gvi2w3kM4lRLuD8trIVg/VCZBulFRdmrgDfc8km5UmyVmWg0pMIRIdFBunlVgrUNJs6/L5tRxrZLjUMXTNJNONMvd1fSYWF6nUShRal3EqNWQYxLneo90rcr2t8rHJhnua631tduasU1LWkP3/fo0Tn3uCAx/9LPJwhc57fhibaKHY7G6MjQjHLmGDGmriDVSzhlq6jFypIxtNVK2D6TnoRiI1m8TOo8AdbLir4Q13peNjG2y4i6Ewi0pCK+9kwx02L4PKWvhy8xyvt7/IH+l3M21neXR+PwcLuzlQbHHfWxcpl+tUD5zFG62hZuZQ5RJOZRRdqV4RM7duAbsmZr7TN9yH2VmzTUnZgNZLk/y3Yw/wQ5HDzOtnKen/hp7eQ+fwR5H+OI5bQbkVjPLRpe6gL6NyXITrIVhC9UKgjUiqL6XSSBWbXaWTEIlOutInSoQRQ70v18n5Hi6hXy/n++1UX8LtLaMHGDdjXMjsoRFeIHJ6OK0HaIUlVqIS1sLkShEhLIXlOoXoMqq6jOh1UUYjcnERj3FbcSd6G4HKDnK+Sb7v5BL6Pjtnpikp1+DNl+7n38ydZDF4gIcv7OGHnK+TPxR3og8m78NUjuC6ZaQYoyd9dG6Z0MthcpdQjQWUlNBpo2giMxHC0QilkW4ct9VRYriT8nkZxUqEsGp8bRIPX833liAs9OPj3Hgn+jtpxIWAvb5PrfcQr/Z+m25wgbp6iwvyQc4tHqIdlZlp5zAIqssNpiOHfHMJr1NDmkuofPzPUfp5IsD6EdaN4nuMfim9XW0RGM+3/9k7yxu/7gyFEHuAXwUmiTVlPm2t/ddCiFHg14H9wFvAj1lra7dvqCkpt49Wz+dS7zl+Z2mM1+qjOOL93PP6Be7tfofcAyfo3XeR7tTjOJkqTlJIEgGR8jFu3CtTtuPKQZFpo2QXoQymGxsTFcYGXIQGIS1CGkRkEcKg+165SeLZRmwYH98OqoRSWH58doEf7Bb4t6f+LOfkSS61n2UxeJO2U0PUH+dix0cyznSrgNaSaitLqZUlF11GlZdQgC0UAdC6h830MCbCOj54I9hB+MTZ0d74jcwuAv5Ha+1zQogi8KwQ4kvAXwK+bK39J0KInwF+Bvj7t2+oKSm3D20k2nQ4ob/DvJjh4MLDtLXD+KtzjHlv4BVep1OcRisfJ5PHUXnIxBeHAXTYxjoeTrcNgIhaSJNooAQOCAeVSMpaK1BGgBOX18PVqoTGMNA8WS+kspZrGeLNViUUwvLQoTcA+KP570U2D3KZFwiiJcKozlv5Kt1gNxOtMqEVVDIVlDIoaXCLbTBNVHElLqP3MgBoQDiZWJVQdQfNugEwIJQzMNg7yYhfd2bW2ovAxeTxihDiGDADfAL4cPJrvwJ8ldSAp2xT7r/vdf7Km5/kN1tPM9c5yudEgdcas2jzOI/PVdm/8Aq54PcI9p6mt+/7UU4BzykjhBNLywIysxRrofh1hFxAOE2kF+AApueClZgwNjpxGCXJdR6Ki/e5blw8+b1bFRe/1UZ85k8dRx9+F3+38zzfPnmIf3PyT/GmeIml9lEudV5kyXkL1fgglzoFtB2j0cuwt+tjrSA/0iAvL6DKS0hjEFGEiMI4nOLm41CS42PdIkL6cShFd3ekJ35TsxJC7AceAb4DTCbGHeAScYhlvdd8CvgUwJ4pF3X2q7Rm3ot0CnF6kHDALWKyIwDIQguRCNeIsImUIaoXIKTFaoU1yYmqJcKowcmv1No9eoW1dvVkv8YFsNajWe9kvpGY4mbS1YrLXUNGObiyQLE2Rhi6+PkOfscj688hTSyqhJSIKAThYL0cRjpYE8YpWomyXt+b2WkXQJ/sSIODhZDDKw+QyRWo6nF8V9LRiuVmkca5ScaOvYFXq2GW57CeT+DnEVGAE4V4jSVE0INaDboBdsViWh6mGxcAmVARtn105KADd90myMaKQUXnRk2Q1+aGb8WKTonFjk/Qm/0Aex77ZZSKuNh+hDebT3KaJ3ClJCMFB4uCiUzEPeU6u8o1xsaWKE3P41WaqGoEpRy2VMYUyuhcCZspYby4mhMnOzDeO7maU1i7TrnXer8oRAH4Q+AfW2t/UwixbK2tDP28Zq0dudZ7PFgp2j/40b2MfuwCevZeevvfh1s8jKPyWKuJoiZR+yyyeR7ZWsBZOI9sN2F+AToRpq4wnQy64xG1sujQIez4mFAN9JX7QkH9wonhC2JtE1lY3TDayhfEenzr8gQ/f+YzZL1djLkHeI+4l5mc5PHqCpO5Fgd3xU1k81OLuDN1RNnFTu/C+jmiGxQI2kkXQ/TP/ypf+c9/DG3jdD4pLAKLkhYpLEoYhLCx57xOsQxc6Qhc6/yBrXEO3S6UMDz5z4+h3v8PCHULo7voqBkrCNr+HUgcVgJW0wPhmuce3L3n3/Vw5J9/1lr72FXHb+TFQggX+BzwH621v5kcviyEmLbWXhRCTANz13sfa0W8Gx/pVXnIwWeoOEVIunHHDalASqxUCCliV3m4DG34Iho6vrYSTQiLlBath7STsfAOvOmt4I1bwNoQbQIC2ybEEhoIjSTq5xrr5A7FAMaCie87xJq1vxnuVm9cPVDm8cefRQdu7BknuidR0lleawet5VB3+SsrKm9EQXDYICuuvsvbLmXy18MgkKdO0p36NUx+HKv81QrLIa+5X2W5kxyFW82NZKEI4JeBY9bafzn0oy8APwn8k+T752/LCFNSNoH2B34K74HjuIunEe0V5PxpaHex9QjTcjHtzFWhkFjeWMVhkUhdEQrpNyu+mVCI5Dp3ftex31vJwHdeqpLtfAl2jWILxYGolcmOoDNlcHyEN7KhqJVe05QhNejrcyOr8n7gLwAvCyFeSI79A2LD/Z+FED8FnAZ+7LaMMCVlE5BOAZOfoFfcFT9vXUYEbZz6JWSzjmos49aXoBOhay625xI1s7FSYTdD2PPWDeWtp/l9o6G8mxW32go6KP3P/OoffAD1FcOhXRcoluuU9r6OW20gq8B4FVMooUenMF429tLdAih/oFSokr2Zvpee6qCsz41koXyDjeMNH721w0lJuTNYE4Huogr3x8ajdD9Gd+lVLyGa51CteZyFc4h2C1VcgHYHudzFdDxMJ4NqZjGRQ9j2cSKVbFS6V4Rf+gbZGIXQdmDIATRytWR+nfL5a2WXwPXL5zfTkBsr+OZclaWe4HtaRabzTQ7WKlSqSxSmFsnsnkOWFxDdDjaXR1c6ceMRv4TNxlWXmqurLiHVQVnLzp15SsoQ2d/7Fyx8fhfVD/0H7MFZOu/6foQ/jpffj83txugu3epZRGcRd/EUsr2CXJpHrqxAq45Ta2J6Lm4jaT7S9okCFx04RKGLTjJOjFFx2EVd6ZWLfhGPlVd0j+provQFrtbydpqP3PZuPBa+vLjCK+brfKv1PcyIUY5cGudgocu+Yp2Du85TqDSoHDiPqlzE3XUGyhVMaYSoMoF1c+hCHDs3bnGgTiilj12nA0/fkPeP7SR21mxTUjZAHw957ugDPOEHFJffIlN4mnBsH3qygFQ+rlvBZiOMWyQyEdKv4QBSSnCaSNNGtA0Yiem5cYGONAiZbBz3c7R1bEiFthgjsUpfUUI/3HNe2ysbGN9ICf2NcLt1UABCERFEdU6Ll2g4e1D1Q/R0lq5WOMpQbedxvBC/1cBXC4hoERlFKCmxfh4rFdbLx5v1yXsOl9D3nw+X0MPdu8m+ETtnpikp1+D4Vx7n77wW8hMLP8BD31nm+858ndzh54k+eJlw4n4YOUImMwmZSQK3QhQuo/NvoSqJFkr2MqLTxvHb2K5AZsKBVrhyNDpSqIFWuEFGKg6lDIpvVjNbpE287UGVjtxWErNCwL3uCB3xQV5vf5GV3ikuqqMc5UFm6/s4255mJlfl3c0C1fIyU7UL5CaWcCeWUK0m5AuIiS7Gz6PzY3G6q1u4QmK2n8WSaqGkpKTQ7ma5ZN7g2cVpGsEoM88/yJ6lMuPlp1EHl+nORphyHB/vb7Dp3CTGJHnNQQ/pOBCECCdE6u7gva2WIJ1B0dnAe9Z2kKkiraBfk6G1TDJSEm89Mbb9ArSbEbSCzddCAXjfeMju9gy/Lt5LLTxLJ5yjFp0FByorh+loj4o7Qi9yyHgh1khyVpJxlxBBhPSzcfGZVPSTXq10sCob54/LZN9CMgiprGUneON39+xSUm6QZs+n0T3B77tNnm3sw3/zCA/XqnxfJqBcP4YPdA/5UJjFz0yiZIZI+YROFpMpA6DaDZQx4LeRNBGqg3Bi8yMDD4xEKwUyLgiKxGpWSv+rjzHD6YV9Q3zzRhw2v4xeCsvHH3wRYySXvv4eXjaTvNz7HCu9UzR75zB5w8XGbhxRZLHnAbCr51HteZSFwe2sIJ05RLGTLIbGGB3PW3VvWJUQ7n4jfvfOLCXlJji4+yxPHf3v+Lb9NkvhKb6yeID5XoXS8w9zpJVjvBcb8WB6jt7U+5HKjyuIvSpaOOiwhXVziKCLcFyEMQjRQcoeygiEYzCRRESrl5yQSaaJljiORms5NKJVKQgwicHeWApirRGHO6eFIoRlz5MvIHdJfvLSLr4zN0Fz/ge4HL1Ou3eaheAEHbdOufEeWpGLp0YITZx2qbyQbDdDNnMZ0Q3iPQajEUZjpQJPx554IhRGooNyRVx8B0lBbOqsrBVEPQ/aASLoQtjE6C5W+qu/lFRhWqkG1ZhIGQfWJMnGUHJrmXgywKDMGbiq8nKYjU7Utcc3Y6PnnSAAQXJLjiE0hsgqIiMJTdIdXUtsJCESEGlEFGL7OjNGx49NBCK86v2tjQYXw05gZNccT1QFryyO09DneNM9hmo+yKu1MSrn2mQrDUq7TuIB3fwEZEZwvCo6asYx2eR9rJSI/jkriW/x+/Kxyfk6fK4KYePzWV+ZKSJEbKqFsHCd83A9T/xOIrGIR3bRfugp3vON3yXzwhFeWJrlKB5vmQ5CSIyNaNuIZujQDBWt0KMTZAjaPm4mwHRclBdCECCCHiIKEboHkYN1I5BJWb6JEMrBmmiQdriT2NQZn2tl+IU/eD9/8s2DPHjoBBOP/7/wvnsJpu9FTD6Bcgr4uf2EbgVTbNLLjiI7S7i5AqLZQNYWkStdbLuLzPYwPQ/pRJjIIep6ccuqyCHoeSgtUYPyZ4OUcZfwfvmzsPHJbsVQ+yq4Ihd3rbob3PhG0e3Ovy27msP5p7ikX2e+/RJ/5C2S74xx4dwjzGTHeHezwOxcnZkLS0wvnCc70iC7cglZvIycXMaUKphChag4Ee/2627ci7CftpVsEun+htFd3r7Kn1rkyMgyP5fdQ2D24UiDryIqXot2L8PrR+8n+8YsjtK47ouIRBsFSMIfMmmNVsbaCtrsv6IrPXBF8c5wwc7N6PDA9XVUtoIRx3HxKkcQP9fksfp5Pnvm89BpQ7uT/HObh/wlrJfBFkoYPx9/ZQ9ivRy9vkbPOg2OnR2m03MtNnXGXbq82Qo40yyxa2mEynwFf3kRWakNjIKSGUx/kyhTxpoI4+eRUYjI+BBEsbykMknHkwisQCqDVAar41ZWAMaYgSwnkFx0yfNt5tWsxZWGsqkwJ1ys7dIJ59C2x4K6B79XZDFwqHSzFNs5RlpZHC8k0/JQXpB6NevgTHW5Z+rCatn7wCCv9qrs9Px3ZHzvZgGrYQwC0Vgm6JxDVu7HjhwhmP34hkJVAnB3oPG9FaSrlpICtP/Ef8+B976EWroYK2AuLkInxNQlpuXHCphtP66w7Pix0FXPG4hdhaE7UMDsV1v27/YG1ZZrWqata/z7/sV29bwTut+E3OI/helJTK6AroxjspW42jJTHpTNr/Wu+wY+VSG8MdJVSUkBRGacaGQvOl9FRD3UxDyy3UAuLyFbTZxmHbfRwHYdokYurrZcp/N8NNx5PtFB6YdS1JrO8/3Hw5WXfWM+fOf3dqoth1+32VgrOPnsAxSOtxjbewGvcgl35gSM5LDlEczoRJLjXY316TMVdBIukU4Bs46+97Daf2rMV0lXIiUFUE6BKDeNO74fV+XRpreuPr1qNlCLSwN9et3y0d0Mup25pj79qjHfWJ/e2nhbeuCdbxDGs0OphXD9fZj+azYLYwX/5dj9LIeSx4+vMFVYYXb6POXxJXLTi3gz51HlDM7kVOKdT2IyJayXQ9+APn1qzFfZ2bNPSUlwnv73yK+dw3msQrD7EOHMEwingJPbg/ZGiAo1THYE2WvglM4hmw1kfRm50sJpttGNLKbnoprhFaJW/U31YVGrSJr1Ra0EWIY21e21Ra36z2/GmG+WIX+uZjlu3+Jsay9T2SJHaqPsKzaYGVliauYC2ZEGudkzqDKoyfnBprrOj8VeeXbsqk11uybMEs93lZ1ozHfejFNS1kEcPcFrX/kw90ZP497/DDpfxZT24vhTOE4B/Cl63ghRUMOqDLK0jONdQGTqiGwL5bSRnbhIx/RchLDowCUKYhMTx8ktUsapgcZItLCAc0XRztrWf3D1Yxiq5lyn+vJOldD3sRZO2Iu82f4SS9n7mWwdoBFOc7lb5UAnRxA6jNTqTADeyAqunkN22ohuB4zB+HkATF8LxUYIuWq8kTsv33sjdtZsU1I24OyXHuVnv3WIHz23l3eNX+bRH/w9nNmA3nt/AD0yi1s8jJupQqZK6BbRQQ2dH0v0wpeRuTlEu4ObbWA7CtXMojsZnK43aAAhewajJTI06EghtRpkO61tACGIU1wH+ifbTAvlPrmLdv5JLrSfpSHOcdnuZn/3AfYvlznbnmU62+VIvcxoZZnqwkX8iRrOxFmcyTo2X0RXW3FaYb4VG3K3gB5qYiyEM1Am3Kk6KJAa8JQUAJaWRnieF9m7+B7aehd7jh5kpH2ezPRRAhsReSO4/hRKZrCZKpFwMPku2ui4aXTQQygHEdRAamQUYJPsE6MVQlqMlomRuTKeHYdMNEIIdN+owxXe+EYe9/W0UK6XuXK70mRnC5Jo5R4WnBOEeoV69y3OZB2i6BCVZpWuzjLmjxAZhZ/txqqN0uBkVhDGxFooRmMdL642TXpoWifWRAEGtyY72Ru/+2eYknIDNDo55tpf49dNg/H2LI3wvTz0eo0PrHyb4sNfQj90gd6hD6Fye/AzkzgqT+gUiDIj6MI8ruMh2w2UlIhOG+W0EX6EaXlXhlUihRAeWhoi6SThlCtFruJK4tV+m3E5p72i4GzgiW9BQSspLJ965EXCyOEXvvNxXg0v82rnt1lqv0ZNnKAtnmSqNUlgRplZKdIOMuyqlxiplSj3LuJUmsjoHBRLEAWYQgcRtNGFcdDdWANFOHHWCv7AiDO0LjvFG797Z5aSchNUy8sczH2UOX2CeXOKl2r70HaEfccPsN+JyGeP45SqRNWI0CkghYPjFDBeBQPo7BIAstCMTWU3QJoITIAKkwYEyXfjxhrgyupBlaaUcdGQlAZjQKkrVQntkAHe6hrhQlhm7juJU+jwwVOHKNamOG/voR0uEOolFsxpjDScbc1irMtIpogjNY6jyRTa+EbgFevxXL1MLKkBWC+HdaJ4c1O6sUcuooEnvpEq4d3MzptxSso67LnnFJ987WP8+8VF5trP8SUkZ+YeJa8O82TP5z79CoXC88hui9Afx8lU8Zwynlclkj5RuYP1S4goQDoO0hiQK0gZgukglAEj0D13yNCaxPjGaYVSWkiayxjDoGo41kPZOJxyo9KymxVOkViyTzl0HvrT/NkLX+Xwiw9x9pXv5aXMS1xuzVPvnqClLpO1OZbDKq4sEBpJZBSOG1LoZihle6igi3BqsVedhFOMm4uzUZy4O48hbrk27ImvF06Bu9MT39QZRbbHq+JVvnDuCLXevTyxUuI94dNk7/1DAiAY2YcdfQQhHFy3gslOxx1Qoh7Ki3eh4w4oLaTpIXpxGb12Vrue9DdnTJKqNSwa1K+IG+6Aoo1cd9f/Vl0Yt2uzaE9hhb95wOcrlz7EV9QUl3qvEkYrnFAv0DCzyNpu2lGBVhTfpo+tFBBK4zcbeCwjel1U0IulOnOleIxeLt71d5MYolPYMXHGzGiDg8UWf1E/Sj33HgqOpewZZnJNuqHLhRP7GKkvksnN45V/CaM0PcdgI4m1EhE6mEjS7mbiKs2giomm0JGKqzSNuKJa05j4dWurNTfqXL9eqf56z7dMtaaXQZUOU/mJL/HRJ/4r7/nuLM1amVbz/QhhcRxNofQ6mcILZMdrOKMryLKBaiXpYv8goZ/HZEcwfgncArhFRNLweLiLPXBVef7dcl5ej01WIwyZ6xzl694CzH8EwzRTxw4wHUly+08gjEaXDuO4FZTM4GaqaOWjiyvgZBBREMt0SokIQoQ02G4AwsaqewnxZpFEGTnwblbHIACzbb2bPhOFBo89eJTw6ce5dOYgC/IUQThHvXuCntsgY31UcxzwKXsVIq0ollawWqHyXZTsgmwg/Vy8Zl42Xh/lJ5tFWYzu3pB3czdcLE6lyZ5inT3JrfuV2tySS0tjXFiqXvd9tlI5+53G88bgw/8z9sMw8lMOIzf4uiTqn3IDbP8rLyXlFhD+yA/xgQNfg0YTegZdc7GhImrm4tZogUvUzcSVlT0PrRU6UqvFOUMaKGtFsK6ngdL/2fD3YbaMV32DGAT6Gxdwlv8eZmIa4+cJiuOxDoqbh8wISAeZeNJrdVDixztbZfBGSVcmJQUwk4/S9XKolcvIbgu5NIdot3Dqy9iWwbYUuplF9zyiZhYTOoS92LD3wyRGr9VAuTpMcjvK5reKBsowy6/vpdBcILP3GE7RQU1OYgsl9KDaMo/JjoFwVjvPp9WWN026GikpgHQK6MIMcuK9WOUT6i46XMa2zqKa86iVeZyFi7jtFixewPYspu6h24lSYcfHhA5BJ4PVijB00ENaKMNKhTqJga8nbLU27r12D0VxdShurVFfa9CHX78ZaCP5la99kJVI8li1xnh+hX27z1EYr+FPncKf7kEljxmfwuaKRKUqNlNCe3m0VwEnO/DOh2PdUjjoofAdpAZ9Z88+JaXP+a+Re+VbBPc9SlDZjyzOIlTcA1O7BUxuBOv5ca63dynO9faayGaIanvIlQgTeAilMaGD6HnoIM71jlMDV0vo+7nfw6GW7ZbrfT1eazic74R09ShTfol6J8uuhRrV+REqjYu4lSaq/RYUC4hqK9ZByZUx2R7WzWMy0WqYxTpp5eUG7MxZp6SsIfOHv8vx33g/h77v63j3Pkfnez6GLewhmz8I2Rms1fRKl7C9eVTlDVS7jlq6jKjXUM02sraM7SnUcgETOjhtn6jroSOHKBGzCnrekNSsuaKE3pgkhMKaLlFDglb9MMtabrZL1PBrbgfWwje7Z3ij/bs8xxF2cYh3L09yaKHKwUu7uO/yBCMjy4wvnsMdaeDMvIEcKaNKZfToJMYvoPOt2JC7HXCyaQn9BtzwbEV83/IMcN5a+3EhxCzwWWAMeBb4C9ba4PYMMyXl9lL71gF+5eUH+XijzOyes0zXfg27ZzedBz6CyO/B9adwvApG+WjAZBexjofyc8hsDeHUEO0AR65gOnH1pVAaGaxWYAJorQYeeL+JsZSCKHKuMNoCgUZeoUZ4M1lRcOc8ciFgj51iIfcAzfAyp1WboHcfS8E0c90Snchhsj7CvaFLsVyn3LmIO15HjDVxgh6mEOfTm0wxvvNxC1i3iLHRhtWXOyHnez1uZpZ/GzgGlJLnvwj8K2vtZ4UQvwT8FPDvbvH4UlI2hbMn9/HZ+mtw+j4eblR4aqRBYeEcmcrLBNMRxq3gumVQeULhxBKzUsVG3PPj9tJeE6mbCLcLgFQa2Y3NjAwdMHHrPwAdWYTot28TWKUHG579rvOrhni1E/3NVF9uZif6tezyPRa77+Kl6AsE0QJtOceK/yBLjQNIUWC65+E7EWOtHEIZCqGDFy4j5SIyCLCOi4iCuLm50XF9QvLeBhDWGXjkxkZXeONw96S3Xo8bmqEQYjfwx4B/DPwdESdRfx/wyeRXfgX4eVIDnrJNaXRyXOz8Eb+By3eWdtOJPsT9R+d4ZPlbZB86RvjAGTozT+D4U7huGal8IuUTZSqY7GWsVHEHH3kJ0e6gnC6yFSJaEQiDScrodVJYFUoHo1e9cgBjDULEDR+EjQsTYoN9pRGH9eVl1ysauxOFZlJY/sp9Z2j2MvzLY5/khDzFmdY3mOu+yrJzll79e5lulgjNBDMrJbqhx2S9SGV5kXzvEqqyhNIRNl9ERCE6V8JEPUw2wjqJRviQvKxIKjLv1hqFa3Gjs/vfgL8HFJPnY8Cytcn9CpwDZtZ7oRDiU8Cn3sEYU1JuO67SeE6R+fAEbafGC7XvwTDB3jf2MpF/Hbd8gqg8g1Y+rltGyQy4FULdxZgIXWjEqoTtFgIQQQSRRg5poahMCIA1AscKIkDq1RxxjMQKi5QWY1Y1T9ZqhK+Xbngtb3qzC82EsBw+cBKk5eELH4b6LOfl8xgb0AkucTF3msjsZqZdBXzGmiU8FeF4IV6xDYBTbsfr6GeRyZ2OcDKxF66yWLqpKiE3YMCFEB8H5qy1zwohPnyzH2Ct/TTw6fi95NX5TSkpW4B77z/OXz/5Mf5D/RkW2q/wX5C82niQyLyH985XObz4CtngS4T7z9Cd/UEcr4LnlMGfQjuFJC5eh0QKVQqJcJooLwTayJ4LxGEVIS1CWmQi0jTsiQth0SIOrwCIfpn9BkZ8vUyVt+OJw63VQhn/M5fovet7+OneN3nm+H30jn+C1+SLLLRe4HLnJWrOaZzmB7ncLaPtCI3AY2/PxxpJoZ6nKC8gK404vh2FiChAG43J9OJ/ao6f6KDEm5s71YjfyKzeD/yIEOJjgE8cA//XQEUI4SRe+G7g/PXeyJNFRrL3Uw/PcVk3eXOlzOm5KVw3ZN+ZZ3GAsHqKML8H6cd5n1b5aLeI0V2MX0D4LWQUInwPTA/hR0grUJkQqxXC0UhlMICSBpsovFkrsXZVK2U9LwdWL4rb4eXcSnLZDmMPneDI6T2cah7g9NJDnAHawXki02FRXGIhKFPsZVjo+rhSs9IoIoQls1xAZEKk20HkmwgpkUEnviC83Kpn4zr0TxFrIoRyBhfD3XaBZCor7M8HvG/lPVzw72WMHLvyDo5s0+r5LJ+dRD19Gvfkt3FnXgTXw3oeThThGAOtFgQRdsVie4qw5aO7JUzPjbvZR4qo62GMIkyKf9ZrfjwsI3u9HPH+4z5bqYrTlsroyUeZevIXebLYwtjHONN6lIudJ1ACPGnZm+8wmqmxd2SR0dEapWqN/O45VKWNmClAfgQ9OoHJldDZCiY3CsqHzMiqnOxQz0y4WhNl+NjdyHVnZq39WeBnARIP/O9aa/+cEOI3gD9NnInyk8Dnr/deY06GD7mP8uutZ3jZ/QPe6kyTO/teHlop8cdHGlTuOUOm9B2CmRbhmI/njcVeTg4ip0AEWKlQXhYnCsFrI3UDq3qDzxBKY43AhHHxxLDT3/dq+hKexLVwYGRs8DHxBcPVXs7a9K07vWlUHV8g/Bu/wKPVf8D4by/S+uYTfGu5yNN8jiiqcUk/R5TrUW8dwpMjLAUOWSdkopNFOREFrfCiGspZQgYBOC7Cz8eVcb7GuhFWulgbgRPHGvsRs7vRy/HG69w7ssThSv/vE9/KS2Hphh6vnToApw4Ad75QZjtgHZd84V74G59hFPiRQbT1aq513mzfM2pzeCfr8/eBzwohfgF4Hvjl671g+BS3NukVeDNBlaE/tJVq9f12svKNXL1llhv889hwjY0A0qgWgHxiD0+2v4zuZDChE3vLSbm8iRyiSKEjJ+ltuaoq2K+s7GeTrC2XB656DNf2nG/kTm4r/8MwCMTRE0S5/4OoPBU3Kc6UY0XBNTndazvO7xTP+VZxUytkrf0q8NXk8Ung8Vs/pJSUzafzyJ9B7juOUzsX97hcnIN2B1sLsR2FbvmrGiitbFwmn4RCosghCp1Y4ErHLdOMkQPDro28ZjgErjTsivUzQ4YN+3p3Aeu95k7RPj5FzjxPZqaEzRcxoxNxNkkiD6v78rCJMTfryMOmGijXJ12VlBRiLRRT3EUwem8cLgpqiN4yauUSqrGA06zj1hah3cXWIkzHjcWtOh4mcAeNi6OeN/DWY7XCVaO+WnV5fR0UuNpb3w46KP3P+/yXPor+ouSBiYuMlOtU917EH3+dTLWFmC5AvoCuxkqFOl9Fr9H8lms0vzU9pHBSD30N6QqkpACmO49aegM9/hAyU0Fmqphclyg3jS5eRLZrOLkzyHYT4c+j2l1kbQXV9tCdDNLV6J6LVBoVOujQJQxclNFEoYsxcQFOf7Ny2JBfVT6fjOlWdqDfTCNurODFpQKLPcFKtIepWpV72nmqCwsUJxfJduaQ5SYqipC5PGI0QIctbKaEyXSxbiGes3CuCLUM68Ps5PL5YXbuzFNShsj9zv/J2d86wu4P/wvEPVO0H/5BRH4PfuEQJr8fayN61XPQq+EsvYlq1lBLc6iVBqq5glluYrsObiOHCTzCdgYvcIkCNw61GEkUuOtmnlzPK19ryIdZa8jj37u25OztNubWwu/XF3m19/t8NXiE3XoPD1++j4OFkNlig0OTF6lU6owePItTuYCz6zzOSAVTqqAr4xi/gMmPxxlomZFYarZfddnXDx/aNN/JoZadNduUlA0IXsvz9dfexUeciNGLZ8l6f0g4tZ/e7idRbgWpfFSmGmuh6C7WjTsZScdFOHWkWcG6AY4V6E7S4k8Opa1qhTUSkewoR9HqpSeESHS/Wb9wJ/m9jbKh4OarL2+3Ec9YDyUzLISnCJ0ebuMeetqnq8soYZno5HDckOzKCjl5CRksIsNYSknmukTCiTc/IdZC6bf4g7s6G+pm2TkzTUm5Bq9/+xH+19OXOV7/Xh4+VucHGt+gcPhryA93CCbvw6kcwfPGkJlJem4FXVwmyI2gSnGMXGWGJGbbATITIFs+JvCQTYOOFEIYdOQQChcpDVo7iGi15d9w84dhiVmStNe1fVvj17y9Nn+3q1dr/N5w2C/S7r2PVzu/Tbt3mjn1KifUEfa1ZpnvVZmul3m05zNeWma6lSU3uYQ3vozs9qBYhCiIe7Xme1g/CauYWMzK2mjQyWdtuGmnhVbu/hmmpNwAra7PfPga31jczaVuhdGnH2X23CR7ve+SvecivcMtuqMP4maqSQuwClFhD5okpTUKEV4GEUUIt4eih5AW3TFYIxCBizUSkrqEaMi4aC2RVmCtXX0OwJWNudfzxocN8VaJj0th+aFdTR7ujPGZy08xb99iuXOCpfAtQqdLvv4Qy4FLwRllpZdBSkM1cMj3XHw5j2wvohwHEcT1HcbouDYBsCaWI0D5GN0dtGPre+PD7ARv/O6eXUrKDdILXdrBeZ52v8SJzj4mzh/h4Xae0rM1KtFJPD9LN1NGKx8/M4mRGYR0CAGtfEQUxKqEQQ+cNkI3kcQhARtJhLQDQz5MbHjNII+8jzFiUGB2lSph30BvoA++ljsRUnny3mMYLTm6/Bgvd3PUOEYnuEA3mOdYPketPUPJLdGMFFknTOYvUX4PJ2ihcrVBgVkfIxXWxAVmwMCIx1WZO1OV8O6dWUrKTXDvwZP8xOt/mS/2vsV87zV+e2mCUyuTSPE471kYYWb5GNngywS7z9Ld/X0op4Cj8uBPEUmfyEaYzBIYjWzWYyEm1UI6PRzA9CJsIicrRBwb71cJ93tnGjOc5530zRR201QJb6WY1d6nnoO9U/yteplvn9/Lp89/gjP2FRqd17ncfZWGe4ls/X0s9nxgjFbo0enG3nRupU7OuYgsL8Xzi0JElAiBeTmMdK7wxOO57sy4+E6uYUxJGZAbrXNPyeCIDJGuc84e462gyZlmnvmlMbpzI0n3nRomasabasJBSR/lFMAtYP0S1vOxXgZcDzwH4YFwI4QySDdCXCFmpRGyb9ANMtn07BvgjYp1bhe36vMkFrt/hva7P8a+Q6c4XKkxxQi+LGGxhHqZdjjPolhmMYioBQ7LQYaVXpZuxydoZTEdD9sFwgAR9BBBB6F7EPXARGAjrI1ijzz5fi3MNUr5tzOiH3fblA8TYh5oAQub9qHvjCrbZ6ywvca7ncYK6XhvJ9tprHBnxrvPWju+9uCmGnAAIcQz1trHNvVD3ybbaaywvca7ncYK6XhvJ9tprLC1xpuGUFJSUlK2KakBT0lJSdmm3AkD/uk78Jlvl+00Vthe491OY4V0vLeT7TRW2ELj3fQYeEpKSkrKrSENoaSkpKRsU1IDnpKSkrJN2TQDLoT4YSHEcSHECSHEz2zW594oQog9QoivCCFeFUK8IoT428nxnxdCnBdCvJB8fexOjxVACPGWEOLlZEzPJMdGhRBfEkK8kXwfudPjBBBC3Du0fi8IIRpCiJ/eSmsrhPiMEGJOCHF06Ni66yli/vfkXH5JCPHoFhjrPxNCvJaM57eEEJXk+H4hRGdojX9pM8d6jfFu+LcXQvxssrbHhRA/tAXG+utD43xLCPFCcvyOry3W2tv+Rdw9+E3gAOABLwLv2ozPvokxTgOPJo+LwOvAu4CfJ27kfMfHuGa8bwHVNcf+KfAzyeOfAX7xTo9zg3PhErBvK60t8EHgUeDo9dYT+Bjwu8RtXt8LfGcLjPUHASd5/ItDY90//HtbaG3X/dsn19yLQAaYTeyGupNjXfPzfwH8L1tlbTfLA38cOGGtPWmtDYg72X9ikz77hrDWXrTWPpc8XgGOATN3dlQ3zSeAX0ke/wrwJ+7cUDbko8Cb1trTd3ogw1hrvwYsrTm80Xp+AvhVG/NtoCKEmN6UgbL+WK21v2ftoF7828DuzRrP9dhgbTfiE8BnrbU9a+0p4ASb2Hv3WmMVQgjgx4Bf26zxXI/NMuAzwNmh5+fYwsZRCLEfeAT4TnLobya3pp/ZKmEJ4nbyvyeEeFYI8ank2KS19mLy+BIweWeGdk1+nCsvgK24tn02Ws+tfj7/ZeI7hD6zQojnhRB/KIT4wJ0a1Dqs97ffymv7AeCytfaNoWN3dG3TTcw1CCEKwOeAn7bWNoB/BxwE3g1cJL6F2go8aa19FHgK+BtCiA8O/9DG93hbKkdUCOEBPwL8RnJoq67tVWzF9VwPIcQ/BCLgPyaHLgJ7rbWPAH8H+E9CiNKdGt8Q2+ZvP8RPcKXzccfXdrMM+Hlgz9Dz3cmxLYUQwiU23v/RWvubANbay9Zaba01wP/FJt7OXQtr7fnk+xzwW8Tjuty/lU++z925Ea7LU8Bz1trLsHXXdoiN1nNLns9CiL8EfBz4c8k/HJJQxGLy+FnimPI9d2yQCdf422/VtXWAPwX8ev/YVljbzTLgTwOHhRCziRf248AXNumzb4gkvvXLwDFr7b8cOj4c2/yTwNG1r91shBB5IUSx/5h4A+so8Zr+ZPJrPwl8/s6McEOu8GC24tquYaP1/ALwF5NslPcC9aFQyx1BCPHDwN8DfsRa2x46Pi4SYWwhxAHgMHDyzoxylWv87b8A/LgQIiOEmCUe73c3e3zr8P3Aa9bac/0DW2JtN3F392PEmR1vAv/wTu7cbjC+J4lvkV8CXki+Pgb8B+Dl5PgXgOktMNYDxDv1LwKv9NcTGAO+DLwB/D4weqfHOjTmPLAIlIeObZm1Jf7HchEIieOuP7XRehJnn/zb5Fx+GXhsC4z1BHHsuH/u/lLyuz+anCMvAM8Bf3yLrO2Gf3vgHyZrexx46k6PNTn+/wB/bc3v3vG1TUvpU1JSUrYp6SZmSkpKyjYlNeApKSkp25TUgKekpKRsU1IDnpKSkrJNSQ14SkpKyjYlNeApKSkp25TUgKekpKRsU/5/HCok5brFASsAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Sequence length 49 (= 7x7 patches), with token dimensionality 192\n",
        "plt.imshow(build_2d_sincos_posemb(7,7,192)[0], cmap='inferno')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "00283495-430f-43b8-adfb-4af7cc48b000",
      "metadata": {
        "id": "00283495-430f-43b8-adfb-4af7cc48b000"
      },
      "source": [
        "### 2.7 Assembling the blocks into a ViT (10 pts)\n",
        "\n",
        "Now we have all the building blocks to create a ViT! \n",
        "\n",
        "Implement the following operations:\n",
        "- Turning images into patches and projecting them to the token dimension\n",
        "- Adding positional embeddings to the tokens\n",
        "- Running the tokens through all Transformer blocks\n",
        "- Taking the mean over all tokens of each batch and projecting to logits"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "02f3fb60-74c3-4ecf-8b9b-baf6530b37a9",
      "metadata": {
        "id": "02f3fb60-74c3-4ecf-8b9b-baf6530b37a9"
      },
      "outputs": [],
      "source": [
        "class ViT(nn.Module):\n",
        "    \n",
        "    def __init__(self, \n",
        "                 img_size=14, \n",
        "                 patch_size=2, \n",
        "                 in_channels=1, \n",
        "                 embed_dim=192, \n",
        "                 num_classes=10, \n",
        "                 depth=4, \n",
        "                 num_heads=4, \n",
        "                 mlp_ratio=4., \n",
        "                 **kwargs):\n",
        "        '''\n",
        "        A Vision Transformer for classification.\n",
        "        \n",
        "        params:\n",
        "            :img_size: Image height and width in pixels\n",
        "            :patch_size: Patch size height and width in pixels\n",
        "            :in_channels: Number of input channels\n",
        "            :embed_dim: Token dimension\n",
        "            :num_classes: Number of classes\n",
        "            :depth: Transformer depth\n",
        "            :num_heads: Number of attention heads\n",
        "            :mlp_ratio: MLP hidden dimensionality multiplier\n",
        "        '''\n",
        "        super().__init__()\n",
        "        \n",
        "        self.patch_embed = PatchEmbed(img_size, patch_size, in_channels, embed_dim)\n",
        "        self.pos_embed = nn.Parameter(\n",
        "            build_2d_sincos_posemb(img_size//patch_size, img_size//patch_size, embed_dim=embed_dim), \n",
        "            requires_grad=False\n",
        "        )\n",
        "        \n",
        "        self.blocks = nn.ModuleList([\n",
        "            Block(\n",
        "                dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio)\n",
        "            for i in range(depth)\n",
        "        ])\n",
        "        self.head = nn.Linear(embed_dim, num_classes, bias=False)\n",
        "            \n",
        "    def forward(self, x):\n",
        "        '''\n",
        "        Forward pass through the ViT.\n",
        "        \n",
        "        params:\n",
        "            :x: Input of shape [B C H W]. B = batch size, C = number of channels, H = image height, W = image width\n",
        "            \n",
        "        returns:\n",
        "            Output of shape [B num_classes]\n",
        "        '''        \n",
        "        # TODO: Project images to patches\n",
        "        x = self.patch_embed(x) # x.shape: [64, 49, 192]\n",
        "        \n",
        "        # TODO: Add the positional embeddings to the tokens\n",
        "        x = x + self.pos_embed # self.pos_embed.shape: [1, 49, 192]\n",
        "\n",
        "        # TODO: Forward pass through Transformer blocks\n",
        "        for block in self.blocks:\n",
        "          x = block(x)\n",
        "            \n",
        "        # TODO: Perform average pooling (compute the mean over the sequences)\n",
        "        x = x.mean(dim=1)\n",
        "        \n",
        "        # TODO: Compute the logits\n",
        "        logits = self.head(x)\n",
        "        \n",
        "        return logits"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1c1e31c1-f1cc-4f71-81c7-0d049a60f6f2",
      "metadata": {
        "id": "1c1e31c1-f1cc-4f71-81c7-0d049a60f6f2"
      },
      "source": [
        "### 2.8 Training the ViT for digit classification\n",
        "\n",
        "Finally, let's train a very small ViT on MNIST. Here's a simple training loop that should only take 5-10 minutes on a V100 or Tesla T4 GPU:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a982ad3d-1300-4de5-b714-76e137ec1afa",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a982ad3d-1300-4de5-b714-76e137ec1afa",
        "outputId": "872dc050-172c-4192-bff8-8fda628c10cb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Number of parameters: 1,791,744\n"
          ]
        }
      ],
      "source": [
        "vit = ViT(\n",
        "    img_size=14, patch_size=2, in_channels=1, \n",
        "    embed_dim=192, num_classes=10, depth=4, \n",
        "    num_heads=4, mlp_ratio=4., \n",
        ").to(device)\n",
        "optimizer = torch.optim.AdamW(vit.parameters())\n",
        "num_parameters = sum([p.numel() for p in vit.parameters()])\n",
        "print(f'Number of parameters: {num_parameters:,}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1eb5df51-b12e-4d3d-bf2b-ef1b06a498e6",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000,
          "referenced_widgets": [
            "8f810e3829684e159fc1086055cfb2d8",
            "c9d6341ccd244039a2f7462d8aa9141c",
            "df24f33c7a62458d8f913c3df7b37788",
            "167a1ec127bc47cc8814d06907542979",
            "9579c7d841ef4bac997b804cf2a43d6e",
            "5c54a4c5d84d47f38fd060cc8e588b7f",
            "f5dce73bce444df18b3ed1761b68a378",
            "99fccfad239f489999c9678f00886707",
            "0da23eeab8d4481993525980dc1aeea9",
            "bfd6f20c551e4850b78464b000a79651",
            "815c8cd821d34c649cf6cfd32cf88e15",
            "85d6dbc5ee754df2bcaeddf02e6789fd",
            "adef5d80e5dd4b2d9326e0bcbedbdb16",
            "82020938a5a34a28bd3334979f0ac843",
            "74d448d9a3634f2ca987b2125d22d457",
            "223bfcde129642afa38da0d98fc9b9ce",
            "c2ec115f26c14506b12054b08562e0be",
            "c1ad6fb0575545d4812bda1591809721",
            "e2d9da8b00a442bfb0c1f304a0400fcf",
            "44763f7208804cf099a90833d548d72b",
            "eb78842cbd2a41b79f08810246e4adb2",
            "409997d07e434d0c89a9b344b7d065ee",
            "3045f82c018d4ced87bb00138eb56302",
            "43bf6b5771d14d66a68921c961885637",
            "6012215f59c34da593a64d97b099db44",
            "57e99e364bdf4588ab5bb832deb73cf1",
            "6654c11fac444393bbb8840ddd9976b1",
            "a6cd6cfde5274f3cb478ab99110ad7bd",
            "8488709617e14b5a8e084e6287764c17",
            "3a45bac4e8084d42884da7a1110f0b19",
            "26c1a007b7734943acc816ee2103cf2e",
            "8890eff393b142a3a0633f8f43e7f143",
            "4e9d6998e8c0417ab8d49f6e28910e7a",
            "99bc0ee17d1e4e6882c2fc6fe7ebd84b",
            "305c1bb60a884d4fa718abba2c43ca0e",
            "865fd577e9524a78b74b8fc2faf44dbd",
            "bb4cbbe7d4ba4ce0800ee5f27f0407c1",
            "8fcf71d367d5416998c3ade31ac3ac9c",
            "effd6edac09e4d92bd7419a42813e688",
            "7e815ba7d37c47d5a3535782bea9fbc8",
            "2d03da80f5f94c70b3bf26a8701d2c76",
            "08327570b1814a158889e8bfc4f831fc",
            "6229c61810774c1c93413503be2283c4",
            "5126718b1dc345c290071b41a8e2cc7b",
            "44ea713552704e75b9d7c866799dea71",
            "6fac5e472fd543878fd1903ad6c9148e",
            "d557cb1336fd4076b035216c10c874ce",
            "602f1c6221714d70a00b65636734375f",
            "1becf7be262e4ad78a1c0b9c07ed9fcb",
            "71044263d0064fb880c872bfd2b1c4cf",
            "9b8ac3ca4a6f4b75bfc083f27f455e97",
            "6a78a38b280740ea9344c08be3f87485",
            "3e36fd8723714c1aabbb9d73e58cf75a",
            "a103eb5850e04df781913a635d409914",
            "efd8220fec5d410799e70ba4593d1975",
            "5f2ebdf404df438d81bdbdd3ecc63a2b",
            "1c8056b0b5c544f0a0636b29dcd28e8a",
            "a481397f471345bfbd922299ef872da5",
            "e25ee5853ae8423a9cf17f7d410302c2",
            "05857940f3aa4501a8729b6eb44b679c",
            "eb5d07765390477d9503441820c92c57",
            "c5975d294acf4e5abb6d14cfec4709e2",
            "505c3ec78ba84c3493508754df165607",
            "d2a6821f07624643bda3c06ef06bf393",
            "6cf34c949d3c4b26aff36588648b0068",
            "e8a6bc5e9dee4376a629a4d966c89ab5",
            "0a686322ff3b4aa3a6e98e477b417553",
            "151cbfa15151457fa306df4b319a5007",
            "f4764482aaa94b06a8683fbd22e14829",
            "4a26185b05bb4183b8362eb3c2996913",
            "907e0819834444aab4d94f600576b873",
            "b32f6272195747ce83c49adf7867df4e",
            "a7908a2231144a1494f39a8df5b581e5",
            "441dce0b30b14c3fb3408874f53f66d2",
            "1c6be90a12744a53a9ea107b285e04ba",
            "a91e4e9e5dbe4b5cb8a0e2d9cf2da5b4",
            "4fbd96c5aa8945a5ab3949f70295cbdf",
            "f803f85c510f4cbba4014f4aa8be7d7f",
            "fcf942446de947bf985f4b5dde41dbb4",
            "f28c019f68b548b58d4a083df59baeed",
            "0ff80f19f250454eb2359ab9d6f7df3d",
            "8702e5ad0948471dad7060f1c36ec6bb",
            "75495f03a2ba487c833f43728338e211",
            "021b281f677a4f16b808f55f33154971",
            "64c7ecae57af44edad6ffe9c8b60169b",
            "5a3c09ae450946978ef5e34eb6cc77f3",
            "4b997ef33b7e4e67811fc69de88ee1d6",
            "bbdd0ba0a5a941718b29f8b633524f1a",
            "63b20e94e1a64996a7cf1d526d622068",
            "78e31d8b3a7b49dca42cf4f12faceed0",
            "e7261ab88f6f4d0ba467b93e857f3fa0",
            "33896793d8c54fc6a08fd2a253e5f707",
            "d915cc0f06974f07bed91c6d58589d6f",
            "4828b3e350bd4605b3af7648d451bc2c",
            "9ee9a714253548659da00556bb0d0d6a",
            "08f1bb9f95184ff795b7d1d5edce9472",
            "3e5268ab9d64413a95bf9999a5376bf9",
            "b786e576fcbd4bb68bc14ccccf88f977",
            "f97e214116a443bab73bbf6ff941c252",
            "14e9e0ea5a7e4556b4035e070701f95f",
            "e3154cb5142b4cad821330904f847dea",
            "84a79ecba8b047a1932d1ba7fdec2e97",
            "5cc45c37464b4e46b726069fafc22d20",
            "e5c4b5dbeaf9496fa6d51b3377b3b944",
            "0a1700ab652c4e14b1d10404b3963235",
            "f0da0adc084e4ec3a118fa8ccdbcd325",
            "7f1e9186142f4c7f9fffe394f31af487",
            "333cd168c68c4362b64a95463cd896be",
            "b2fe8edcbcd64b15b9e8aabed1a4ed6a",
            "943af8e34b3147d8a33b1ca09cb18e8c",
            "6446fe6a2c134793bcef51a456b48c48",
            "cb1242f93e714c3b968a86fa479ba212",
            "e354dfba004445f5b506fca946773f4d",
            "7ba80ffdabb04d24989bf0bba9319b70",
            "318fc9b6a8334c3aaa49f14bf74ac7ca",
            "db02cd0a19d24bd08763724a7522c61f",
            "7d1021bfe3064cb0b9a0b2b15d4cb4c9",
            "60883aeb21bb4247ad62e5d828b7497a",
            "d5581485f10b48faaea9eaf72ba9afae",
            "a3348693e09f420caa595cfd2a37082a",
            "c0d8d1e307034d74b1410fef51f624f8",
            "658fdf59fab649b99ac3cc76e0ef045a",
            "26e2a9f1657844e89a9cf06ba0b8e3ec",
            "b057f955348d4576bfe7c34dbc9c53dc",
            "f58b4867d34f497abdb22861e6a67a7f",
            "8a24d21743d34b988ac456a52aeced80",
            "4ee102b6a6114c08a75d5d55b5854395",
            "28e76d9ec4d64639a441760f9700684c",
            "0fe9602e32ec45e293b50ab3fc982cda",
            "b1c91a48f00445a18b895025dd22f729",
            "7a7fd17e44764bf3bb1dc2d5f5e8081a",
            "fa64d5f0f74e44ffa436c26830c80594",
            "937408e39c86440cb6aa5ca2814c5c0f",
            "881cdd41388b49c3a0024cc23fc0f23a",
            "d4dbb9427d0e4cb6933e6560dd030bde",
            "9e9df9c1866c477697203d288da69e8e",
            "b59ef284e5dc494180e4d093b1ad7e2b",
            "1632d25d9f70459f8896f58fd282c494",
            "9679e09815ee4a8684c52c63cf269006",
            "a322bd3bc97b4b9989da72ec0ccb0ea0",
            "3b52f063eb9b4dcdbd518cb8c59bb692",
            "58ecd9217c4f4041a21e5631a0d7541b",
            "f6de9b206baf4fd280a8ef0f48c5bce5",
            "9671b868aa414ca2b21046170ef7955f",
            "50add4dbfcb949f9a88cb387e260160d",
            "8191e371f2394e69b2415f27bf818234",
            "110c483f388344f7a45ebe1a89539d59",
            "a5948c21d65a45c69704fa3a08751836",
            "51a8927325a243948371089e9623deeb",
            "dd5318f1059848c583f38d3420b1d17a",
            "c7b77df568094dab89d2a95538f9a597",
            "ce2b1bc8776b45d9b5c6d5fedb08ff0a",
            "073e630263e54fd3be751588ed56326c",
            "450560edee374196b933593f0bc61def",
            "387771a90e714c01bceb76a3a9da4303",
            "dd1c876d96e84b578b2fc4b5ff1e6280",
            "96cbf761e13b41358ef20eda1c2cf6a7",
            "86895473d2a949388d87abc5ccc4e410",
            "df64bc3a3c934e0fb5bc840b87acbdba",
            "df9e136b398847bf8eb1a54c7f9dc2a2",
            "32013f4694af4b83bb6fcee3036ae746",
            "a6a5dd909ba14bd7ab47c294af9077af",
            "a38562dd5aea49639812e9668455c645",
            "0980528b56264f19a809a99b0f2e71a6",
            "019aea4bf4934391ba265db774734b7a",
            "8dd04c97854047e587fd54b3d4dfa041",
            "04f3be74a824493ab1cf34a36775156a",
            "4a27ab657ef549c189f78ef100333f31",
            "aa326dd4cfeb437caa4edadccc94bd5b",
            "8b600c06a3594e90ad9cb3bc73c7118e",
            "060945a179274ac3a5e19e36724ab2e6",
            "e317352c96524cfb9f587ad1ca17bb91",
            "2085c9bcffde4e4ab2a75d91d5f5b5ff",
            "d7450585236c4e7abf94348f71ef693e",
            "12e00bbdd71a47bd96e9ec9528a752c8",
            "e8b279b6d4ab46658ec7ab0dec786c23",
            "d43a049331404cc9b191b5ecaa2b7752",
            "e5be6f76bf244baa8c9fba0b13ef00fb",
            "9272fe045e8a4b2e83434e83e61e9128",
            "f660234977c6456983ee16aedbaba5bf",
            "368b7eb9c13e4a8f86c576bbe6028845",
            "a0d9789ed5aa4134b331e23e6d4355b9",
            "01d2ff0e4b9a4687b9869fe5fac16089",
            "f243d5c87dfb47a18d52264cf442c159",
            "17aa582e221744209c5a9aaf6bf27c3a",
            "178c940ec6d5415594bd139d6dd70673",
            "5de361c077604266a257e593df56dd6f",
            "40b560b94bc4428eac05f03efae7569d",
            "2ddf5de2b14c46318989e59f8e23ba03",
            "bee7740b8113472885599de0bf4300f8",
            "6f7f2ff1a3d84d9ab141f3cb7e24149d",
            "558897da5b95456c9ca500d9ffff7b32",
            "e725176a05704b87a0147ac9d9fbcc69",
            "5205553804e4457e98e702c61b019a8f",
            "036a414b407a405f8f1967bf269d8e2a",
            "eaaa5544fb6b45968e0873026af49704",
            "694da459b1704fd3a42c108ca51c2353",
            "a4f964aa65c54128a346f976e5ac3b03",
            "eb1ee4da7b93487ca44f4a47ef925622",
            "4dc11e04cc564e0ebaff0e5bc091d22a",
            "7a7a32ef7e3947fc8081fdbf31269c6d",
            "524134a8fa5e467298954ad22beb92ab",
            "156c5fda89fc43c7941cd8bcceb23a6d",
            "5b8781a399c54223921a6b41d2dade9d",
            "202a7306f806476b8513383606fa187b",
            "a5eef95245694ff19cf8bc9b10a4ac3d",
            "3c5d22b32b1649f883c1b20673415182",
            "e6cc7003780f4f2085eb888ea3a69d45",
            "4ff0c5ed0eab463a81aaa30d9c635fb3",
            "27bded8d14ac423b99aed42725d8c73c",
            "5b170f0d31d34f73b4aa59893e37ab96",
            "9ac3ed08af36466a82ae29fd8bacee2c",
            "0b5d93d232ff4d16b7d3711214901014",
            "c8a3ff14902d4fe9b9d1e676a77359db",
            "b4924076c123415eaf5be8dd463c727c",
            "3b5a5332167b4311b58fe41de7724ee5",
            "d313b4fef20c401a9c694092d000eb7b",
            "07b167d4959e473cb80f85b9478574d3",
            "9934ee1fdd5b449ca9e60be26da212fd",
            "6b11a10db0de404c96b240fc42cf55f4"
          ]
        },
        "id": "1eb5df51-b12e-4d3d-bf2b-ef1b06a498e6",
        "outputId": "c2da4d95-e7be-48e9-eff2-fd8507e156d9"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "8f810e3829684e159fc1086055cfb2d8",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.639, val loss: 0.242\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "85d6dbc5ee754df2bcaeddf02e6789fd",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.199, val loss: 0.157\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "3045f82c018d4ced87bb00138eb56302",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.171, val loss: 0.149\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "99bc0ee17d1e4e6882c2fc6fe7ebd84b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.146, val loss: 0.128\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "44ea713552704e75b9d7c866799dea71",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.124, val loss: 0.119\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "5f2ebdf404df438d81bdbdd3ecc63a2b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.117, val loss: 0.121\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0a686322ff3b4aa3a6e98e477b417553",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.118, val loss: 0.110\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "f803f85c510f4cbba4014f4aa8be7d7f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.110, val loss: 0.102\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "63b20e94e1a64996a7cf1d526d622068",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.089, val loss: 0.114\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "14e9e0ea5a7e4556b4035e070701f95f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.086, val loss: 0.107\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6446fe6a2c134793bcef51a456b48c48",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.094, val loss: 0.114\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "658fdf59fab649b99ac3cc76e0ef045a",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.082, val loss: 0.085\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "937408e39c86440cb6aa5ca2814c5c0f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.081, val loss: 0.060\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9671b868aa414ca2b21046170ef7955f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.076, val loss: 0.085\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "387771a90e714c01bceb76a3a9da4303",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.077, val loss: 0.081\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "8dd04c97854047e587fd54b3d4dfa041",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.064, val loss: 0.080\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d43a049331404cc9b191b5ecaa2b7752",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.069, val loss: 0.086\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "40b560b94bc4428eac05f03efae7569d",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.073, val loss: 0.081\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "eb1ee4da7b93487ca44f4a47ef925622",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.057, val loss: 0.081\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "27bded8d14ac423b99aed42725d8c73c",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.055, val loss: 0.072\n"
          ]
        }
      ],
      "source": [
        "num_epochs = 20\n",
        "\n",
        "train_losses = []\n",
        "val_losses = []\n",
        "\n",
        "for _ in range(num_epochs):\n",
        "    \n",
        "    # Train loop\n",
        "    vit.train()\n",
        "    epoch_loss_train = 0\n",
        "    pbar = tqdm(total=len(loader_train))\n",
        "    for imgs, cls_idxs in loader_train:\n",
        "        inputs, targets = imgs.to(device), cls_idxs.to(device)\n",
        "\n",
        "        logits = vit(inputs)\n",
        "        loss = F.cross_entropy(logits, targets)\n",
        "\n",
        "        vit.zero_grad(set_to_none=True)\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "        \n",
        "        epoch_loss_train += loss.item()\n",
        "        \n",
        "        pbar.update(1)\n",
        "        pbar.set_description(f'Train loss: {loss.item():.3f}')\n",
        "    pbar.close()\n",
        "    \n",
        "    epoch_loss_train /= len(loader_train)\n",
        "    train_losses.append(epoch_loss_train)\n",
        "    \n",
        "    \n",
        "    # Validation loop\n",
        "    vit.eval()\n",
        "    epoch_loss_val = 0\n",
        "    for imgs, cls_idxs in loader_val:\n",
        "        inputs, targets = imgs.to(device), cls_idxs.to(device)\n",
        "\n",
        "        with torch.no_grad():\n",
        "            logits = vit(inputs)\n",
        "        loss = F.cross_entropy(logits, targets)\n",
        "        \n",
        "        epoch_loss_val += loss.item()\n",
        "    \n",
        "    epoch_loss_val /= len(loader_val)\n",
        "    val_losses.append(epoch_loss_val)\n",
        "    \n",
        "    print(f'Train loss: {epoch_loss_train:.3f}, val loss: {epoch_loss_val:.3f}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d2f2321f-a8ae-4995-882a-997d47976989",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "id": "d2f2321f-a8ae-4995-882a-997d47976989",
        "outputId": "9f5a8a29-e14a-4f9a-c9b3-85034d7ab0cd"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAAvNElEQVR4nO3deXxU9b3/8ddnlsxkmyEkIQmbAQWUHU1RVAS07lbaWq1cvZVq609vrba9Wm1rba+1j6q16rW1i23VLlfRWrVYUdyFVpFN9kUh7EvIAlnJMjPf3x/nJBnCJCSQkwk5n+fjMY85c+bMzCcnk/PO95zz/R4xxqCUUsq9PMkuQCmlVHJpECillMtpECillMtpECillMtpECillMv5kl1AV+Xk5JjCwsJkl6GUUseVZcuWlRljchM9d9wFQWFhIUuXLk12GUopdVwRkW3tPae7hpRSyuU0CJRSyuU0CJRSyuWOu2MESqm+qampiZ07d1JfX5/sUo5rwWCQwYMH4/f7O/0aDQKlVK+wc+dOMjMzKSwsRESSXc5xyRhDeXk5O3fuZNiwYZ1+ne4aUkr1CvX19WRnZ2sIHAMRITs7u8utKg0CpVSvoSFw7I5mHbomCJZsreCB1zegw24rpdShXBMEq3dW8pv3NnOgrinZpSileqHy8nImTpzIxIkTyc/PZ9CgQS2PGxsbO3zt0qVLufXWW7v0eYWFhZSVlR1Lyd3GNQeL88NBAPZW1ZOVnpLkapRSvU12djYrVqwA4Mc//jEZGRncfvvtLc9HIhF8vsSbzKKiIoqKinqiTEe4pkXQEgSVemqaUqpzZs+ezU033cTpp5/Od7/7XRYvXsyUKVOYNGkSZ555Jhs3bgTgvffe47LLLgOsELn++uuZPn06w4cP57HHHjvi5zz88MOMHTuWsWPH8uijjwJQW1vLpZdeyoQJExg7dizPPfccAHfddRejR49m/PjxhwTVsXBPiyDU2iJQSvVu//PKWtbtrurW9xw9MMSPPjemy6/buXMnH3zwAV6vl6qqKhYuXIjP5+Ott97i+9//Pn//+98Pe82GDRt49913qa6uZtSoUdx8883tnte/bNkynnrqKT766COMMZx++ulMmzaN4uJiBg4cyKuvvgpAZWUl5eXlvPTSS2zYsAER4cCBA13+eRJxTYsgNzOAR2CPtgiUUl1w5ZVX4vV6AWtjfOWVVzJ27Fi+/e1vs3bt2oSvufTSSwkEAuTk5DBgwABKSkraff9//etffOELXyA9PZ2MjAy++MUvsnDhQsaNG8ebb77JnXfeycKFCwmHw4TDYYLBIDfccAMvvvgiaWlp3fIzuqZF4Pd6yMkIUKJBoFSvdzT/uTslPT29ZfqHP/whM2bM4KWXXmLr1q1Mnz494WsCgUDLtNfrJRKJdPlzR44cyfLly5k3bx5333035513Hvfccw+LFy/m7bff5oUXXuBXv/oV77zzTpffuy3XtAgACsJB9uiuIaXUUaqsrGTQoEEAPP30093ynlOnTuXll1+mrq6O2tpaXnrpJaZOncru3btJS0vj2muv5Y477mD58uXU1NRQWVnJJZdcwiOPPMLKlSu7pQbXtAgA8kJBtpXXJbsMpdRx6rvf/S7XXXcd9913H5deemm3vOepp57K7NmzmTx5MgBf+9rXmDRpEvPnz+eOO+7A4/Hg9/v5zW9+Q3V1NTNnzqS+vh5jDA8//HC31CDHWweroqIic7QXpvnRP9bw0se7WPXjC7u5KqXUsVq/fj2nnHJKssvoExKtSxFZZoxJeI6rq3YN5YWDVNVHqGvs+v46pZTqqxwNAhG5SEQ2isgmEbmrnWWuEpF1IrJWRJ5xsp4C7UuglFKHcewYgYh4gceB84GdwBIRmWuMWRe3zAjge8BZxpj9IjLAqXrAOkYAVl+C4bkZTn6UUkodN5xsEUwGNhljio0xjcAcYGabZb4OPG6M2Q9gjNnnYD0UhFMBbREopVQ8J4NgELAj7vFOe168kcBIEfm3iCwSkYsSvZGI3CgiS0VkaWlp6VEXpL2LlVLqcMk+WOwDRgDTgVnA70WkX9uFjDFPGGOKjDFFubm5R/1hqSlewql+bREopVQcJ4NgFzAk7vFge168ncBcY0yTMWYL8AlWMDgmPxTUIFBKHWbGjBnMnz//kHmPPvooN998c7uvmT59OolOZ29vfm/lZBAsAUaIyDARSQGuBua2WeZlrNYAIpKDtauo2MGayA8HddeQUuows2bNYs6cOYfMmzNnDrNmzUpSRT3HsSAwxkSAW4D5wHrgeWPMWhG5V0QutxebD5SLyDrgXeAOY0y5UzWBtgiUUol96Utf4tVXX225CM3WrVvZvXs3U6dO5eabb6aoqIgxY8bwox/9qEvv++yzzzJu3DjGjh3LnXfeCUA0GmX27NmMHTuWcePG8cgjjwDw2GOPtQwxffXVV3fvD9gBR4eYMMbMA+a1mXdP3LQBvmPfekR+OEhpTQNN0Rh+b7IPkSilEnrtLti7unvfM38cXHx/u0/379+fyZMn89prrzFz5kzmzJnDVVddhYjw05/+lP79+xONRjnvvPNYtWoV48ePP+JH7t69mzvvvJNly5aRlZXFBRdcwMsvv8yQIUPYtWsXa9asAWgZTvr+++9ny5YtBAKBbhtiujNctyXMDwcxBkqrG5JdilKql4nfPRS/W+j555/n1FNPZdKkSaxdu5Z169Z19DYtlixZwvTp08nNzcXn83HNNdewYMEChg8fTnFxMd/85jd5/fXXCYVCAIwfP55rrrmGv/71r+1eDc0Jrhp0DlqvVLansp6B/VKTXI1SKqEO/nN30syZM/n2t7/N8uXLqaur47TTTmPLli089NBDLFmyhKysLGbPnk19/bHtXs7KymLlypXMnz+f3/72tzz//PM8+eSTvPrqqyxYsIBXXnmFn/70p6xevbpHAsF9LQK7L0GJHjBWSrWRkZHBjBkzuP7661taA1VVVaSnpxMOhykpKeG1117r9PtNnjyZ999/n7KyMqLRKM8++yzTpk2jrKyMWCzGFVdcwX333cfy5cuJxWLs2LGDGTNm8MADD1BZWUlNTY1TP+ohXNciKIhrESilVFuzZs3iC1/4QssuogkTJjBp0iROPvlkhgwZwllnndXp9yooKOD+++9nxowZGGO49NJLmTlzJitXruSrX/0qsVgMgJ/97GdEo1GuvfZaKisrMcZw66230q9fPyd+xMO4ahhqAGMMJ//wda47s5DvX6JD3irVW+gw1N1Hh6E+AhGxrlSmLQKllAJcGARgjUK6t/JgsstQSqlewZVBUKC9i5XqlY63XdW90dGsQ1cGQV44SEllg37plOpFgsEg5eXl+nd5DIwxlJeXEwwGu/Q61501BFAQCtIYjVFR20h2RiDZ5SilgMGDB7Nz506OZah5ZQXq4MGDu/QaVwZBfKcyDQKlege/38+wYcOSXYYruXLXUL59pTLtVKaUUm4NgpB2KlNKqWauDILczABej2iLQCmlcGkQeD1CbkZAWwRKKYVLgwCsA8baIlBKKTcHQUiHmVBKKXBzEISDlGgQKKWUe4OgIBykuiFCTUMk2aUopVRSuTYImjuV6YXslVJu594gCGkQKKUUuDkImlsEeuaQUsrlXBsEeS0tAr0ugVLK3VwbBEG/l6w0v7YIlFKu59ogAGvwOT1GoJRyO0eDQEQuEpGNIrJJRO5K8PxsESkVkRX27WtO1tNWfiigLQKllOs5dj0CEfECjwPnAzuBJSIy1xizrs2izxljbnGqjo7kh1NZvasyGR+tlFK9hpMtgsnAJmNMsTGmEZgDzHTw87osPxSkrKaRxkgs2aUopVTSOBkEg4AdcY932vPaukJEVonICyIyJNEbiciNIrJURJZ252XsCuxTSHXwOaWUmyX7YPErQKExZjzwJvCnRAsZY54wxhQZY4pyc3O77cPzNAiUUsrRINgFxP+HP9ie18IYU26MabAf/gE4zcF6DlMQ1iuVKaWUk0GwBBghIsNEJAW4Gpgbv4CIFMQ9vBxY72A9h2nuVKYtAqWUmzl21pAxJiIitwDzAS/wpDFmrYjcCyw1xswFbhWRy4EIUAHMdqqeREJBH2kpXm0RKKVczbEgADDGzAPmtZl3T9z094DvOVlDR0SE/FBQO5UppVwt2QeLky4/HNROZUopV9Mg0BaBUsrlNAjsi9jHYibZpSilVFJoEISDRGKGstqGIy+slFJ9kAZB8ymklRoESil30iBo6VSmF6hRSrmTBoEOM6GUcjnXB0FOegCfR7RTmVLKtVwfBB6PkBfSvgRKKfdyfRAA5IUC2pdAKeVaGgRAQThVWwRKKdfSIMAahXRvZT3GaKcypZT7aBBgXZegrjFKdUMk2aUopVSP0yCg9UplepxAKeVGGgS0XqlMg0Ap5UYaBLQOM6FBoJRyIw0CYEAoAKBnDimlXEmDAAj4vGSnp2jvYqWUK2kQ2JqvS6CUUm6jQWDLDwW1RaCUciUNApu2CJRSbqVBYMsPBamobaS+KZrsUpRSqkdpENiar0uwr0qvVKaUchcNApteqUwp5VYaBLaW3sV6nEAp5TKOBoGIXCQiG0Vkk4jc1cFyV4iIEZEiJ+vpSJ72LlZKuZRjQSAiXuBx4GJgNDBLREYnWC4TuA34yKlaOiMz6Ccj4NNTSJVSruNki2AysMkYU2yMaQTmADMTLPcT4AEg6VvgvFBATyFVSrmOk0EwCNgR93inPa+FiJwKDDHGvNrRG4nIjSKyVESWlpaWdn+ltoJwqrYIlFKuk7SDxSLiAR4G/vtIyxpjnjDGFBljinJzcx2rKS+kncqUUu7jZBDsAobEPR5sz2uWCYwF3hORrcAZwNxkHjAuCAfZV91ANKaXrFRKuYeTQbAEGCEiw0QkBbgamNv8pDGm0hiTY4wpNMYUAouAy40xSx2sqUP54SDRmKGsRjuVKaXcw7EgMMZEgFuA+cB64HljzFoRuVdELnfqc49F8wVq9DiBUspNfE6+uTFmHjCvzbx72ll2upO1dEZ+/CUrhxxhYaWU6iO0Z3Gc1iDQYSaUUu6hQRCnf1oKKV4Pe3XgOaWUi2gQxPF4hAGhgLYIlFKuokHQRkE4qAPPKaVcpVNBICLpdgcwRGSkiFwuIn5nS0uOvFBQB55TSrlKZ1sEC4CgiAwC3gD+E3jaqaKSqblFYIx2KlNKuUNng0CMMXXAF4FfG2OuBMY4V1by5IWC1DfFqDzYlOxSlFKqR3Q6CERkCnAN0DxAnNeZkpKrIJwK6AVqlFLu0dkg+BbwPeAlu3fwcOBdx6pKovxwANDexUop9+hUz2JjzPvA+9AyamiZMeZWJwtLlny7RVCiQaCUconOnjX0jIiERCQdWAOsE5E7nC0tOQZkBhDRFoFSyj06u2totDGmCvg88BowDOvMoT7H7/WQk6FXKlNKuUdng8Bv9xv4PDDXGNME9NnzK/NDQW0RKKVco7NB8DtgK5AOLBCRE4Aqp4pKtvywXqlMKeUenQoCY8xjxphBxphLjGUbMMPh2pJGWwRKKTfp7MHisIg83HwBeRH5BVbroE/KDwepPNjEwcZosktRSinHdXbX0JNANXCVfasCnnKqqGRrvlKZdipTSrlBZ69QdqIx5oq4x/8jIiscqKdXKAg3X7LyIMNy+mzDRymlgM63CA6KyNnND0TkLKDPDtqfZweBHjBWSrlBZ1sENwF/FpGw/Xg/cJ0zJSWfXsReKeUmnR1iYiUwQURC9uMqEfkWsMrB2pImPeAjM+jTYSaUUq7QpSuUGWOq7B7GAN9xoJ5eoyCsp5AqpdzhWC5VKd1WRS+UF9JOZUopdziWIOizQ0yAtgiUUu7R4TECEakm8QZfgFRHKuol8kNBSmsaaIrG8HuPJS+VUqp363ALZ4zJNMaEEtwyjTFHPNAsIheJyEYR2SQidyV4/iYRWS0iK0TkXyIy+lh+mO6UH07FGCitbkh2KUop5SjH/tUVES/wOHAxMBqYlWBD/4wxZpwxZiLwIPCwU/V0VfOVyrR3sVKqr3Nyn8dkYJMxptgY0wjMAWbGLxB3BhJYYxf1muMO+SH72sV6nEAp1cd1tkPZ0RgE7Ih7vBM4ve1CIvINrFNRU4BzE72RiNwI3AgwdOjQbi80kXy7d7EGgVKqr0v6UVBjzOPGmBOBO4G721nmCWNMkTGmKDc3t0fqykrzk+Lz6K4hpVSf52QQ7AKGxD0ebM9rzxysK6D1CiJCfiioLQKlVJ/nZBAsAUaIyDARSQGuBubGLyAiI+IeXgp86mA9XZYf1iBQSvV9jh0jMMZEROQWYD7gBZ40xqwVkXuBpcaYucAtIvJZoIleOJBdfijIih0Hkl2GUko5ysmDxRhj5gHz2sy7J276Nic//1gVhIO8vrYeYwwifXpEDaWUiyX9YHFvlhcK0hiJsb+uKdmlKKWUYzQIOhB/pTKllOqrNAg6oFcqU0q5gQZBB1pbBBoESqm+S4OgA7kZATyCXqlMKdWnaRB0wOf1kJsZ0BaBUqpP0yA4gvxQUIeZUEr1aRoER6C9i5VSfZ0GwRHoeENKqb5Og+AI8sOpVDdEqGmIJLsUpZRyhAbBERTodQmUUn2cBsER5IW0U5lSqm9zVxBU7e7yS7RTmVKqr3NPECx4CH41GWpKu/SyfB1mQinVx7knCE65HJrq4P37u/SyoN9LvzS/DjynlOqz3BMEuSOh6Kuw9Cko/aRLL7VOIW1wqDCllEou9wQBwLS7wJ8Gb/2oSy/LDwfZW6UtAqVU3+SuIMjIhanfho3zYMvCTr9MWwRKqb7MXUEAcMZ/QWgQvHE3xGKdekl+OEhZTQONkc4tr5RSxxP3BYE/Fc67B/asgDUvdOol+XZfgn3VeuaQUqrvcV8QAIy7CvLHw9v3QtORN+752rtYKdWHuTMIPB644D6o3AEf/faIi7cEgfYlUEr1Qe4MAoDh02DkRbDwF1Bb3uGiBaFUQFsESqm+yb1BAHD+vdBYC+8/0OFioVQfQb9Hg0Ap1Se5OwhyR8Fp18HSP0LZpnYXExEKwqns0V1DSqk+yNEgEJGLRGSjiGwSkbsSPP8dEVknIqtE5G0ROcHJehKa/j3wBY/YySwvFNCL2Cul+iTHgkBEvMDjwMXAaGCWiIxus9jHQJExZjzwAvCgU/W0K2MAnP0t2PBP2PZBu4sVhFN1BFKlVJ/kZItgMrDJGFNsjGkE5gAz4xcwxrxrjKmzHy4CBjtYT/vO+AZkDoT5P2i3k1leKMi+6noaItEeLk4ppZzlZBAMAnbEPd5pz2vPDcBriZ4QkRtFZKmILC0t7dow0p2Skgbn/RB2L4e1LyZcZOKQfjRFDRc+soB3NpR0fw1KKZUkveJgsYhcCxQBP0/0vDHmCWNMkTGmKDc315kixn8Z8sfBW/+TsJPZRWPz+dP1k/F4hOufXsr1Ty9hS1mtM7UopVQPcjIIdgFD4h4PtucdQkQ+C/wAuNwYk7yR3Txeu5PZdlj8RMJFpo3M5fXbzuEHl5zC4i0VXPjIAh54fQO1emF7pdRxzMkgWAKMEJFhIpICXA3MjV9ARCYBv8MKgX0O1tI5w6fDiAusq5nVVSRcJMXn4evnDOed/57G5yYM5DfvbebcX7zHP1bswhjTs/UqpVQ3cCwIjDER4BZgPrAeeN4Ys1ZE7hWRy+3Ffg5kAH8TkRUiMredt+s5598LjdXwfscnMA0IBfnFVRP4+81nMiAzyG1zVnDlbz9kza7KHipUKaW6hxxv/8UWFRWZpUuXOvshr9wGH/8VvrEYsk884uKxmOFvy3bw4Osb2V/XyKzJQ7n9glFkpac4W6dSSnWSiCwzxhQleq5XHCzudaZ/H7wBeOvHnVrc4xG+/JmhvHP7dK47s5A5S3Yw/aH3+MuHW4lE9RoGSqneTYMgkcw8q5PZ+rmwfVGnXxZO9fOjz41h3q1TGV0Q4of/WMvnfvVvPirueFA7pZRKJg2C9kz5BmQWWJ3Murj7bFR+Js98/XR+fc2pVB1s4stPLOKbz37Mnkq97rFSqvfRIGhPSjqcezfsWgprX+ryy0WES8YV8NZ3pnHreSOYv3Yv5z70Pg/N30hVfZMDBSul1NHRg8UdiUXhd+dAQzXcsgR8gaN+qx0VdTw4fyOvrNxNVpqfb547gmvOGErA5+3GgpVSKjE9WHy0PF644CdwYBss/v0xvdWQ/mn8ctYkXrnlbEYPDHHvP9dx3i/e5+WPdxGLHV9hrJTqWzQIjuTEc+Gkz8KCB9vtZNYV4waH+b+vncFfbphMONXPt55bwWW//BcLPinVDmlKqaTQIOiM839i7R567/52RyftqqkjcnnllrP536snUlXfxFeeXMy1f/yIVTsPdMv7K6VUZ+kxgs6a+01Y/mfwp0PuSMg9BQac3HofHgIiR/XWDZEoz3y0nV++s4mK2kYuG1/A7ReMojAnvZt/CKWUW3V0jECDoLOaDsKq52HfOti3Hko3QE3ccNQpGdalL+MDIncUhAd3OiCq65t4YkExf1i4haZojP84fSi3njeCnIyjP0itlFKgQeCcugorEJqDoXQD7NsAtXHj56VkWoEw4GQomAjjvgSpWR2+7b6qev737U+Zs2QHQXuQu69NHU5GwOfsz6OU6rM0CHpaXYUdDuutYGgOidpS8KfBhFlw+k3WLqYObC6t4RdvbGTe6r3kZKTwzXNH8OXPDCHo11NOlVJdo0HQW+xdDYt+C6ufh2gjnHQ+nHGzdWZSB7uPPt6+n5+9toHFWyron57CtWecwFemnKC7jJRSnaZB0NvU7IOlT8GSP1i7kXJPtloI479sXTYzAWMMH22p4A8Li3lr/T5SfB6+OGkQX5s6jJMGZPbwD6CUOt5oEPRWkQZY8yIs+jXsXWUdOzhtNnzm6xBu//LOm0tr+OO/tvD3ZTtpiMSYMSqXr08dzpQTs5GjPHNJKdW3aRD0dsbA9g+tQNjwKiAw5vNwxn/B4IS/NwDKaxr4y6Jt/OXDbZTXNjJmYIivTx3OpeML8Hu1i4hSqpUGwfFk/zbrmsnL/wwNVTCoyDqOMHomeP0JX1LfFOWlj3fxh4XFbC6tpSAcZPaZhVw9eSjh1MSvUUq5iwbB8aihGlY8Cx/9BiqKIXOgtdvohDOhYAIEQ4e9JBYzvPfJPn6/YAsfFpeTnuLly58ZylfPKmRI/8THHpRS7qBBcDyLxeDTN6xAKH6vdX72CBg4CQZOtO7zx0Mgo+XpNbsq+eO/tvDKyt3EjOHicQVcN6WQsYNCpKVofwSl3EaDoK+oLYPdK2D3x7DHvq/aZT8pVse1gZOsW8FEyB/HnoPC0x9s5ZmPtlNdHwEgPxRkWE46w3PT4+4zGJyV2v6xhVgMYhHw6XWYlToeaRD0ZdUlraGwewXsXt469IV4rKEuBk6iYcA4VhzMo2R/Jfv3V1BddYC66kp8TTWky0EyqCdD6slNaaS/v5GQp4EMOUgwVocvUounqc56v7yx1u6poVOsW2beIeXEYoamWIymqCESte6bojEiUYMIDOyXitejZzYp1dM0CNymao8dDHG3urJ2F495gzR60zkoqdQQ5EAkQHlTCtUmQI1JpZYgjd40wgEPo6MbOCW6kSCNAGwnn2XmZBbHTmZRZCRbTB7Q/oY+4PNw0oAMRuVlMiIvk1H5GYwYkMmgfql4NCCUcowGgdsZY+1CKt9sDXERyLAGyQtkWvfew48ZxGKGPVX1bCmtZUtZDcVltew5UI/XI6R4ogxr3MRJ9asYVreKE2pWkRatAqDWn82e8ERKsk6lNOs0arJG4fP68Hs9NEVjbC6t4ZOSGj4pqWZPZX3L56WneDkpL5NReRmMzMtkZF4mo/IzGZAZOLa+EZEG2LEYtrwPe9dY/TOyT7KOseScZI0a63FoyI6GGti/xVrvFZvh4H447auQfaIzn6dUBzQIlLNiMSj7BLZ/ANs+hO2LoHK79VxKJgyZDCdMgSFnQN4YSOsPQOXBJj4tqW4JhuZbWU1jy1uHgj5G5VuthzEDQ1w2fmDHp8TGotausuL3rY3/9kUQqbd2a2WPgOq90FDZurw3AP2HW6GQPcIKiRz73q6zQ4211lld5Zut+4rNUGFv/Gv2Hrqsx2fVcda3YOp3wJ/aufWrVDfQIFA9r3KnHQofWBvjfetan8vIs4bVGHDKofep/QCro1xzOGwsqebTkmo27q2mqj5CRsDHrMlDuP7sYRSEU63WTtknrRv+rQuh3t7Q554Cw6fBsGlQeBYEw9bytaVQ9imUb4LyT6FskzW9f4t1QLxZWvahrYfQIOvnqihu3fi33dinD7D+4+9/IvQfZk8Pt26NtfDG3bD6b9BvKFz8IIy62Nnfg1K2pAWBiFwE/C/gBf5gjLm/zfPnAI8C44GrjTEvHOk9NQiOU3UVsGu5FQgtQ3dvhKba1mUyBx56sZ/mazoEQxhjWLOrit8vLGb56jWc6VnDVf03MyGyCn+dfXC831Broz9sGgw757AD2UcUbbI69LUERHNYbDr02hMtG/vhrbfmx4FOjPu0ZQG8ejuUbYSRF8PF90NWYddqVaqLkhIEIuIFPgHOB3YCS4BZxph1ccsUAiHgdmCuBoHLxGJQuaM1GJqH7i79BCIHW5cLDbaCISPPal1UbAag3IT4IDaa0twpTDjnck6dMMm5sZbqK62D8KGBCTvzdVmk0eob8t4DYKIw9XY461bw6YiyyhkdBYGTPYsmA5uMMcV2EXOAmUBLEBhjttrPdc+FgNXxxeOBrBOs28gLW+fHonBgm30th/Wt97tXwKDT4DM3wLBpeDNPYttHO3j6g62UzdnDuIV13HjOcC4em4+vu8daCoatW3fxpcBZt8HYK2D+9+Hd+2Dls3DJz+Gk87rvc5TqBCdbBF8CLjLGfM1+/J/A6caYWxIs+zTwz/ZaBCJyI3AjwNChQ0/btm2bIzWr41N9U5QXl1tjLRWX1TKkfyo3nDWMqz4z5PjpRb3pLZj3Xau1M3omXPizDkegVaqrkrVrqNuCIJ7uGlLticUMb64v4Xfvb2b59gP0S/PzlTNO4CtnFvb4RXyq65v4pKSadXuq2bCnig17q9m5v47RBSHOGJ7N6cOzGTswdGjLJdIA/34MFj4E4oXpd1oj0LYz2KBSXZGsXUO7gCFxjwfb85RyhMcjXDgmnwvH5LN0awW/W1DMY+9s4ncLirnitMFcf1YhJ2Snd+sQ3bGYYVtFHRv2VLF+bzXr91SxYW8VOypaj3GEgj5OLggxZXg2q3dV8u7GUgAyAj6KCrM4Y3g2ZzQHw7Q7YPxV8Ppd8OY9sOIZuOQhGDa122pWqi0nWwQ+rIPF52EFwBLgP4wxaxMs+zTaIlAO2LSvhj8sLObF5btojFqHotJSvIRT/YRT/YSCfkLN06m+Q+aHU/2E01qn/V5h074aNuytZsPeKtbvsU5rPdgUBcAjMCwnnVMKQpxSEOLk/ExOLggxMBw85CD2vup6Fm+pYFFxOYuKK9i0rwawOtUVFfa3g6E/42o/xDf/TjiwHcZdBRf8BDLze34lHs9iUavvyIFtULXb6hsSHgrhweAPJru6HpXM00cvwTo91As8aYz5qYjcCyw1xswVkc8ALwFZQD2w1xgzpqP31CBQR2NfdT3z15awv7aRyoNNVB5sosq+rzzYRHV9hMqDTdQ0RI78ZkC/ND+n5Ic4uSDT2vDnhxiRl0HQ3/VeyqXVDXHBUM6ndjCkpXiZMjSdm7wvc9qOPyO+AHL2bXD6zYeMNOs4Y+CT12HZn6zdVGnZ1i09p3U6fl5PdpQzxhqM8cA22L/VCs0D26zTgA9sgwM7INaU+LXpA6DfEKt3eb+h1i08pHVed5wd1otohzKlOikSjbWEQuXBJqrqW8OivinG8Nx0TskPkRc6xqEvOlBWc2gwfFJSQ6Hs4W7/M3zWs4wKwvxf4CreSrsEX0qQoN9Dqt9LwO8l6PMS9HsI+r2k+lunA34vaX4v2RkpDMgMMiAUoH9aSsfjOxljDX3+zn2wa6l1Gm8gw9rwHqwA087Jfv60w8MhLRtS+1vDmYjX6mHtse/jb4fMa54Wa76JWafwHrKh3w5NdYd+flqOtVHPOgH6ndA6nTnQqvvADuu05QPb7fsdVkfBaMOh7xMMW62H5mAIDwJvirVeMHH3sQTzmu+x15Oxfpb+w63RgbOGWWfN9SANAqWOY+V2MKzYcYCsihV8dvdvOaluBWXePF4I/Sdv+6dTF4GDTVEammLUN0Wpb4pysClKrIM/b59HyMkIMCAUYEBmgNzMIAMyrccj69dwyvrHyNi7CBMejEy7Eyb8R+u4VLEY1B+AunLrVlvWOh03z8Q9lsaa7lkhgTBkDbU38vbpx/2Gtm70j6a1FItZPc4PbLeGR2kJi7j7xuruqb/5ZygY33o9kYKJVkg4eM1xDQKl+hJjYPM78Pa91rhKuSfDuXfDyZcdsiExxtAUNdRHrGCoa4hSVtPAvuoG9lXVW/fVrY9LqxsYWLeB//b9jenelewz/fhVZCZzoueSkZ5OTkYKghCJxYgZiMRiRKOGSMwQM9Z9NGqINk/bt2Z+IozOS+X0YVlMKczi1BPChANeaz++iVkd60zMurXMM63zwTpGkprVwyscq46GamsIEhFArHvxtE4nvPe0Tscirf1hmoeOL1kLUXtsrUAYBk6wQqE5ILKGdVs4aBAo1RcZA+v+Ye26Kf/U6mx33j0wfHrX36tkHbz7U9jwT2LBLPaOu4n1Q65m70FhX5UVFuU1DYiAz+PB4xF8HsHrEbwieL3tPBbB6/Hg8woNkRgfb9/Pkq0V1DfF8AiMGxRmyok5nHliNkWFWUnp99EUjVFR20hZTQPlNY2U1zZQVt1IWa39uKaBcvvY0mlDs7h0fAFnj8gh4OuGUWsjjXY4fNwaEPHhEAxbl6ZtbjWccOZRnzCgQaBUXxaNWL2S37sfqnZaYy199kdWMBxJ+WZ472ew+gVrnKQpt8AZNzt6oLQhEmXF9gN8sLmcDzeX8/GO/TRFDX6vMGlIFlNOzObME7OZOLTfUW9sozFDeU0DJVUN7Kuub7lv2dDXtG74Kw8mPpic4vWQnZFCdkYKORkBgj4vHxaXU3mwicygj/NH53HZ+ALOPimXFF837u+PNFpjcsVfcKpkrXXQ+5KHYPLXj+ptNQiUcoOmelj6pNUhra7c2lV07g+tcZraOrAd3n/Q6qfgC8Dp/w/OvLVzQ293s7rGCEu37reDoYzVuyqJGQj6PXymsL8dDDmMHWiFU1lNIyUtu7asjXxp3Ma+pMpqvSQ6PtIvzU92egrZGQFy7A18dnrA3tg3z7ceZwZ8h50Q0BiJ8e/NZby6ag/z1+6luj5CZtDHBaPzuWx8AWedlNO9odAs0mCFQ+bArg+maNMgUMpNGqrhw1/DB7+0RnedMAum3WkdVK3eCwt/AUufsvY9F91gXRshY0Cyq25RebCJxVsq+GBzGR9uLmfDXusgbcDnoTEao+0mSwSy060D3nmhQMtZUQNC9sHvzAB5oSA5GYFu3Ug3RmL8e1MZ/1y1hzfWWaEQCvq4YEw+l44v4KwTHQqFo6RBoJQb1ZbDvx6Gxb+3DraOvBA2vW3tYph0LZxzh9Wxqpcrq2lgUXE5K7YfID3gs89yCrZs9HMyUrp/kMEuaohEW0LhzbUlVDdECKf6uXBMHpeMs1oK3dmj/WhoECjlZpW74P0HYM3frd1F0++0TlVUjmiIRFn4SRnzVu/hjXUl1DRE6Jfm5/xT8hjSP420FC+pKV7r3u8jrXk6xUtaiq912u/t1oDTIFBKqSSob4qy8NMyXl21m7c37KO6vnM915uleD2toZHi5VufHcnlEwYeVS3JGnROKaVcLej3cv7oPM4fbR3gbYrGqGuMcrAxSl1jxJpuitrzrMetz1vPtcxvipKV5sxItBoESinVQ/xeD+FUD+HU3jW0eO85pK2UUiopNAiUUsrlNAiUUsrlNAiUUsrlNAiUUsrlNAiUUsrlNAiUUsrlNAiUUsrljrshJkSkFNh2lC/PAcq6sZzupvUdG63v2PX2GrW+o3eCMSY30RPHXRAcCxFZ2t5YG72B1ndstL5j19tr1PqcobuGlFLK5TQIlFLK5dwWBE8ku4Aj0PqOjdZ37Hp7jVqfA1x1jEAppdTh3NYiUEop1YYGgVJKuVyfDAIRuUhENorIJhG5K8HzARF5zn7+IxEp7MHahojIuyKyTkTWishtCZaZLiKVIrLCvt3TU/XZn79VRFbbn33YdUHF8pi9/laJyKk9WNuouPWyQkSqRORbbZbp8fUnIk+KyD4RWRM3r7+IvCkin9r3We289jp7mU9F5Loequ3nIrLB/v29JCL92nlth98Fh2v8sYjsivs9XtLOazv8e3ewvufiatsqIivaeW2PrMNjYozpUzfAC2wGhgMpwEpgdJtl/gv4rT19NfBcD9ZXAJxqT2cCnySobzrwzySuw61ATgfPXwK8BghwBvBREn/Xe7E6yiR1/QHnAKcCa+LmPQjcZU/fBTyQ4HX9gWL7PsuezuqB2i4AfPb0A4lq68x3weEafwzc3onvQId/707V1+b5XwD3JHMdHsutL7YIJgObjDHFxphGYA4ws80yM4E/2dMvAOeJiPREccaYPcaY5fZ0NbAeGNQTn92NZgJ/NpZFQD8RKUhCHecBm40xR9vTvNsYYxYAFW1mx3/P/gR8PsFLLwTeNMZUGGP2A28CFzldmzHmDWNM85XUFwGDu/Mzu6qd9dcZnfl7P2Yd1WdvO64Cnu3uz+0pfTEIBgE74h7v5PANbcsy9h9DJZDdI9XFsXdJTQI+SvD0FBFZKSKviciYnq0MA7whIstE5MYEz3dmHfeEq2n/jy+Z669ZnjFmjz29F8hLsExvWJfXY7XwEjnSd8Fpt9i7r55sZ9dab1h/U4ESY8yn7Tyf7HV4RH0xCI4LIpIB/B34ljGmqs3Ty7F2d0wAfgm83MPlnW2MORW4GPiGiJzTw59/RCKSAlwO/C3B08lef4cx1j6CXneutoj8AIgA/9fOIsn8LvwGOBGYCOzB2v3SG82i49ZAr/976otBsAsYEvd4sD0v4TIi4gPCQHmPVGd9ph8rBP7PGPNi2+eNMVXGmBp7eh7gF5GcnqrPGLPLvt8HvITV/I7XmXXstIuB5caYkrZPJHv9xSlp3mVm3+9LsEzS1qWIzAYuA66xg+ownfguOMYYU2KMiRpjYsDv2/nspH4X7e3HF4Hn2lsmmeuws/piECwBRojIMPu/xquBuW2WmQs0n53xJeCd9v4Qupu9P/GPwHpjzMPtLJPffMxCRCZj/Z56JKhEJF1EMpunsQ4qrmmz2FzgK/bZQ2cAlXG7QHpKu/+FJXP9tRH/PbsO+EeCZeYDF4hIlr3r4wJ7nqNE5CLgu8Dlxpi6dpbpzHfByRrjjzt9oZ3P7szfu5M+C2wwxuxM9GSy12GnJftotRM3rLNaPsE6m+AH9rx7sb70AEGsXQqbgMXA8B6s7WysXQSrgBX27RLgJuAme5lbgLVYZ0AsAs7swfqG25+70q6hef3F1yfA4/b6XQ0U9fDvNx1rwx6Om5fU9YcVSnuAJqz91DdgHXd6G/gUeAvoby9bBPwh7rXX29/FTcBXe6i2TVj71pu/g81n0Q0E5nX0XejB9fcX+/u1CmvjXtC2RvvxYX/vPVGfPf/p5u9d3LJJWYfHctMhJpRSyuX64q4hpZRSXaBBoJRSLqdBoJRSLqdBoJRSLqdBoJRSLqdBoFQbIhKVQ0c47bYRLUWkMH4ES6V6A1+yC1CqFzpojJmY7CKU6inaIlCqk+xx5R+0x5ZfLCIn2fMLReQde3C0t0VkqD0/zx7rf6V9O9N+K6+I/F6s61G8ISKpSfuhlEKDQKlEUtvsGvpy3HOVxphxwK+AR+15vwT+ZIwZjzV422P2/MeA9401+N2pWD1LAUYAjxtjxgAHgCsc/WmUOgLtWaxUGyJSY4zJSDB/K3CuMabYHjhwrzEmW0TKsIY/aLLn7zHG5IhIKTDYGNMQ9x6FWNcfGGE/vhPwG2Pu64EfTamEtEWgVNeYdqa7oiFuOooeq1NJpkGgVNd8Oe7+Q3v6A6xRLwGuARba028DNwOIiFdEwj1VpFJdof+JKHW41DYXIn/dGNN8CmmWiKzC+q9+lj3vm8BTInIHUAp81Z5/G/CEiNyA9Z//zVgjWCrVq+gxAqU6yT5GUGSMKUt2LUp1J901pJRSLqctAqWUcjltESillMtpECillMtpECillMtpECillMtpECillMv9f6R6kKM74N5nAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "plt.figure()\n",
        "plt.plot(train_losses, label='Train loss')\n",
        "plt.plot(val_losses, label='Val loss')\n",
        "plt.xlabel('Epoch')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "50faa034-c780-40ab-8721-c11aaeb1be0f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "50faa034-c780-40ab-8721-c11aaeb1be0f",
        "outputId": "90f34fc0-2070-48f2-95a6-f233477229de"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Test loss: 0.064\n",
            "Test top-1 accuracy: 98.07000000000001%\n"
          ]
        }
      ],
      "source": [
        "test_loss = 0\n",
        "correct = 0\n",
        "\n",
        "vit.eval()\n",
        "for imgs, cls_idxs in loader_test:\n",
        "    inputs, targets = imgs.to(device), cls_idxs.to(device)\n",
        "\n",
        "    with torch.no_grad():\n",
        "        logits = vit(inputs)\n",
        "    loss = F.cross_entropy(logits, targets)\n",
        "    test_loss += loss.item()\n",
        "    \n",
        "    pred = logits.argmax(dim=1, keepdim=True)\n",
        "    correct += pred.eq(targets.view_as(pred)).sum().item()\n",
        "\n",
        "test_loss /= len(loader_test)\n",
        "accuracy = correct / len(loader_test.dataset)\n",
        "\n",
        "print(f'Test loss: {test_loss:.3f}')\n",
        "print(f'Test top-1 accuracy: {accuracy*100}%')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "92d59c82-cb03-489c-8f4b-c33460fbaa1d",
      "metadata": {
        "id": "92d59c82-cb03-489c-8f4b-c33460fbaa1d"
      },
      "source": [
        "For reference, we get over 97% top-1 test set accuracy with this simple training loop over 20 epochs. Some stochasticity around that value is expected, but if you're far below, check your implementation again."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "889ab07c-3ef0-4586-b7a2-72ee757c75d4",
      "metadata": {
        "id": "889ab07c-3ef0-4586-b7a2-72ee757c75d4"
      },
      "source": [
        "### 2.9 Open ended question on Vision Transformers (10 pts)¶\n",
        "\n",
        "#### Question:\n",
        "This ViT was designed for image classificaton. How could we modify it for solving dense prediction tasks, such as pixel-wise depth estimation or semantic segmentation?\n",
        "\n",
        "#### Answer:\n",
        "In ViT, in order to do classification, we use a feed-forward classification head which maps the hidden state to class logits. For dense prediction tasks we have to modify this head such that it maps the hidden state to an appropriate output. For instance, for pixel-wise depth estimation, the head should map the hidden state to pixel_wise depths. In semantic segmenention, the head should map the hidden state to image segments in a suitable way.\n",
        "\n",
        "Moreover, in ViT because we are doing classification, we either use the \\<cls\\> token encoding as the input of the classification head, or if we are not using the \\<csl\\> token, we do some kind of pooling such as averge pooling across all the encodings as the input for classification head. However, for dense prediction tasks, we might have to use all the encodings as the input of the head prediction layer. For example for the pixel-wise depth estimation task, we probably should use the encodings of all the pixel tokens as the input to the prediction head.\n",
        "\n",
        "Also, in pixel-wise depth estimatin task, as we want to predict the depth for all the pixels, we cannot perform batch down-sampling on the inputs, as we would lose pixel-wise information."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ac016e02-5231-4716-b31b-0d99f35b25d7",
      "metadata": {
        "id": "ac016e02-5231-4716-b31b-0d99f35b25d7"
      },
      "source": [
        "---\n",
        "\n",
        "## 3 Image generation with GPT\n",
        "\n",
        "In this part, we will implement a GPT model and train it on images, similar in spirit to [iGPT](https://openai.com/research/image-gpt).\n",
        "What this means is that we turn images into a sequence of discrete tokens and train a decoder-only Transformer to perform next-token-prediction.\n",
        "At inference time, we can then autoregressively generate new images from the training distribution!"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "251b72c7-5151-448e-a580-ec695fa92924",
      "metadata": {
        "id": "251b72c7-5151-448e-a580-ec695fa92924"
      },
      "source": [
        "### 3.1 Tokenization\n",
        "\n",
        "First, we need to turn images into sequences of discrete tokens.\n",
        "\n",
        "A common way of performing tokenization of images is to train a discrete VAE, such as (VQ-GAN)[https://arxiv.org/abs/2012.09841] to autoencode images through a discrete bottleneck.\n",
        "Since we don't want to open that can of worms in a homework about Transformers, we can instead suggest you to check this out in your own time.\n",
        "\n",
        "Here, we will tokenize the images in a much simpler way:\n",
        "- First, we turn the grayscale images into black and white → each image is now a sequence of 14x14 zeros and ones. In this manner, we would already have a sequence of discrete tokens we could model, but the sequence length of 14x14=192 could be shorter.\n",
        "- To reduce the sequence length further, we divide the image into 2x2 patches and turn each of the 2x2 patterns into a unique index. If we flatten each 2x2 patch into a sequence of 4 zeros and ones, we can directly interpret them as integers between 0 and 15. This reduces the sequence length by a factor of four. Much more manageable in our toy setting!\n",
        "\n",
        "While this way of tokenizing is really quite \"toy\", it is indicative of a common problem when dealing with images and transformers: naively turning images to tokens can result in very large sequence lengths, which do not go well with the quadratic complexity of self-attention."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3e4688cb-d86a-44b4-aaef-d2f3bd681cf4",
      "metadata": {
        "id": "3e4688cb-d86a-44b4-aaef-d2f3bd681cf4"
      },
      "outputs": [],
      "source": [
        "def dec2bin(x, bits):\n",
        "    mask = 2 ** torch.arange(bits - 1, -1, -1).to(x.device, x.dtype)\n",
        "    return x.unsqueeze(-1).bitwise_and(mask).ne(0).float()\n",
        "\n",
        "def bin2dec(b, bits):\n",
        "    mask = 2 ** torch.arange(bits - 1, -1, -1).to(b.device, b.dtype)\n",
        "    return torch.sum(mask * b, -1).long()\n",
        "\n",
        "def tokenize_MNIST(imgs, patch_size=2):\n",
        "    imgs = (imgs[:,0] > 0.5).int()\n",
        "    bits = rearrange(\n",
        "        imgs, \n",
        "        'b (nh ph) (nw pw) -> b (nh nw) (ph pw)', \n",
        "        ph=patch_size, pw=patch_size\n",
        "    )\n",
        "    return bin2dec(bits, patch_size**2)\n",
        "\n",
        "def detokenize_MNIST(imgs_tokenized, patch_size=2):\n",
        "    bits = dec2bin(imgs_tokenized, patch_size**2)\n",
        "    N = int(math.sqrt(imgs_tokenized.shape[-1]))\n",
        "    return rearrange(\n",
        "        bits, \n",
        "        'b (nh nw) (ph pw) -> b (nh ph) (nw pw)', \n",
        "        ph=patch_size, pw=patch_size, nh=N, nw=N\n",
        "    )"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "85f65314-fdd2-419f-b006-eb05d6884e98",
      "metadata": {
        "id": "85f65314-fdd2-419f-b006-eb05d6884e98"
      },
      "source": [
        "Here are some examples of tokenized images:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9bf85f2c-a8b3-4810-9f70-cacd3c3a6361",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "9bf85f2c-a8b3-4810-9f70-cacd3c3a6361",
        "outputId": "c1b7200d-b42b-437a-f43c-6386feece0ea"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAATEAAAEvCAYAAAAtufaDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAANW0lEQVR4nO3db6jlBZ3H8fdnZ5LSJA0vUjOy4wNxEWnXuIQltKAGU4n2YB8oGboJ82Qri0CUHvRsEYooKIpBTSFRFjOS6I+DFbJg0h0VU8dSrHRMmxOyFfXAhr774B7tzp07zplzfnPP+er7BTL3/Dze83Gu8+Z3z/zmZ6oKSerqn+Y9QJJmYcQktWbEJLVmxCS1ZsQktWbEJLW2dTNf7LTTTqsdO3Zs5ktKep3Yu3fvH6pqaf3xTY3Yjh07WFlZ2cyXlPQ6keS3Gx3320lJrRkxSa0ZMUmtGTFJrc0UsSQ7k/wyydNJrh9qlCRNauqIJdkCfA34IHAOcEWSc4YaJkmTmOVM7D3A01X1TFW9DNwJXDbMLEmazCwR2wY8t+bx/vExSdo0x/2N/SS7kqwkWRmNRsf75SS9wcwSseeBM9Y83j4+doiq2l1Vy1W1vLR02J8YkKSZzBKxnwNnJTkzyQnA5cA9w8ySpMlM/Wcnq+pgkk8APwK2ALdU1eODLZOkCcz0B8Cr6vvA9wfaIknHzCv2JbVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktTbT/zxXGlKSeU84oqqa9wQdgWdiklozYpJaM2KSWjNiklozYpJamzpiSc5I8pMkTyR5PMm1Qw6TpEnMconFQeCzVfVQkpOBvUn2VNUTA22TpKOa+kysql6oqofGH/8Z2AdsG2qYJE1ikPfEkuwAzgMeHOLzSdKkZo5YkrcC3wY+XVV/2uDv70qykmRlNBrN+nKSdIiZIpbkTawG7Paqunuj51TV7qparqrlpaWlWV5Okg4zy+9OBrgZ2FdVXxpukiRNbpYzsQuAjwEXJnlk/NeHBtolSROZ+hKLqvpfYHFvOyDpDcEr9iW1ZsQktWbEJLVmxCS15u2pNZNFvqX0kIb89/RW18PyTExSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa95jXzMZ8n7xb5T79WtYnolJas2ISWrNiElqzYhJas2ISWpt5ogl2ZLk4STfG2KQJB2LIc7ErgX2DfB5JOmYzRSxJNuBDwM3DTNHko7NrGdiXwauA/5+pCck2ZVkJcnKaDSa8eUk6VBTRyzJJcCBqtr7Ws+rqt1VtVxVy0tLS9O+nCRtaJYzsQuAS5P8BrgTuDDJtwZZJUkTmjpiVXVDVW2vqh3A5cCPq+rKwZZJ0gS8TkxSa4PcxaKqfgr8dIjPJUnHwjMxSa0ZMUmtGTFJrRkxSa15e2otDG91rWl4JiapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqm1mSKW5JQkdyV5Msm+JO8dapgkTWLW/wP4V4AfVtV/JDkBOHGATZI0sakjluRtwPuBqwGq6mXg5WFmSdJkZvl28kxgBHwzycNJbkpy0kC7JGkis0RsK/Bu4OtVdR7wF+D69U9KsivJSpKV0Wg0w8tJ0uFmidh+YH9VPTh+fBerUTtEVe2uquWqWl5aWprh5STpcFNHrKpeBJ5Lcvb40EXAE4OskqQJzfq7k58Ebh//zuQzwH/OPkmSJjdTxKrqEWB5mCmSdOy8Yl9Sa0ZMUmtGTFJrRkxSa0ZMUmuzXmKhTZBk3hOkheWZmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNa8PbU0gaqa9wQdgWdiklozYpJaM2KSWjNiklozYpJamyliST6T5PEkjyW5I8mbhxomSZOYOmJJtgGfApar6lxgC3D5UMMkaRKzfju5FXhLkq3AicDvZp8kSZObOmJV9TzwReBZ4AXgj1V171DDJGkSs3w7eSpwGXAm8E7gpCRXbvC8XUlWkqyMRqPpl0rSBmb5dvJi4NdVNaqqvwF3A+9b/6Sq2l1Vy1W1vLS0NMPLSdLhZonYs8D5SU5MEuAiYN8wsyRpMrO8J/YgcBfwEPCL8efaPdAuSZrITHexqKrPA58faIskHTOv2JfUmhGT1JoRk9SaEZPUmhGT1Jr32D8OVi+b0+vJkF9T79c/LM/EJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktebtqY+DoW8/vMi3u17UWy0v8s/Z0NsW9WuwWTwTk9SaEZPUmhGT1JoRk9SaEZPU2lEjluSWJAeSPLbm2NuT7Eny1PjHU4/vTEna2CRnYrcCO9cdux64r6rOAu4bP5akTXfUiFXV/cBL6w5fBtw2/vg24CPDzpKkyUz7ntjpVfXC+OMXgdMH2iNJx2TmN/Zr9XLhI14ynGRXkpUkK6PRaNaXk6RDTBux3yd5B8D4xwNHemJV7a6q5apaXlpamvLlJGlj00bsHuCq8cdXAd8dZo4kHZtJLrG4A3gAODvJ/iTXADcCH0jyFHDx+LEkbbqj3sWiqq44wt+6aOAtknTMvGJfUmtGTFJrRkxSa0ZMUmtGTFJr3mO/gTf6PdSn4c/ZG4dnYpJaM2KSWjNiklozYpJaM2KSWjNiklozYpJaM2KSWjNiklozYpJaM2KSWjNiklozYpJaM2KSWjNiklozYpJaM2KSWjNiklozYpJaM2KSWjNiklozYpJaM2KSWjNiklo7asSS3JLkQJLH1hz7QpInkzya5DtJTjmuKyXpCCY5E7sV2Lnu2B7g3Kp6F/Ar4IaBd0nSRI4asaq6H3hp3bF7q+rg+OHPgO3HYZskHdUQ74l9HPjBAJ9Hko7ZTBFL8jngIHD7azxnV5KVJCuj0WiWl5Okw0wdsSRXA5cAH62qOtLzqmp3VS1X1fLS0tK0LydJG9o6zT+UZCdwHfDvVfXXYSdJ0uQmucTiDuAB4Owk+5NcA3wVOBnYk+SRJN84zjslaUNHPROrqis2OHzzcdgiScfMK/YltWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1lqravBdLRsBvJ3jqacAfjvOcaSzqLnDbtNw2nXls++eqWlp/cFMjNqkkK1W1PO8d6y3qLnDbtNw2nUXa5reTklozYpJaW9SI7Z73gCNY1F3gtmm5bToLs20h3xOTpEkt6pmYJE1koSKWZGeSXyZ5Osn1897ziiRnJPlJkieSPJ7k2nlvWi/JliQPJ/nevLesleSUJHcleTLJviTvnfcmgCSfGX8tH0tyR5I3z3nPLUkOJHlszbG3J9mT5Knxj6cuyK4vjL+ejyb5TpJTNnvXWgsTsSRbgK8BHwTOAa5Ics58V73qIPDZqjoHOB/4rwXa9oprgX3zHrGBrwA/rKp/Af6VBdiYZBvwKWC5qs4FtgCXz3cVtwI71x27Hrivqs4C7hs/3my3cviuPcC5VfUu4FfADZs9aq2FiRjwHuDpqnqmql4G7gQum/MmAKrqhap6aPzxn1n9hbhtvqv+Icl24MPATfPeslaStwHvB24GqKqXq+r/5jrqH7YCb0myFTgR+N08x1TV/cBL6w5fBtw2/vg24CObuQk23lVV91bVwfHDnwHbN3vXWosUsW3Ac2se72eBQvGKJDuA84AH5zxlrS8D1wF/n/OO9c4ERsA3x9/q3pTkpHmPqqrngS8CzwIvAH+sqnvnu2pDp1fVC+OPXwROn+eYI/g48IN5DlikiC28JG8Fvg18uqr+NO89AEkuAQ5U1d55b9nAVuDdwNer6jzgL8znW6JDjN9buozVyL4TOCnJlfNd9dpq9TKChbqUIMnnWH2r5fZ57likiD0PnLHm8fbxsYWQ5E2sBuz2qrp73nvWuAC4NMlvWP0W/MIk35rvpFftB/ZX1StnrXexGrV5uxj4dVWNqupvwN3A++a8aSO/T/IOgPGPB+a851VJrgYuAT5ac75Oa5Ei9nPgrCRnJjmB1Tda75nzJgCShNX3dfZV1ZfmvWetqrqhqrZX1Q5Wf85+XFULcVZRVS8CzyU5e3zoIuCJOU56xbPA+UlOHH9tL2IBfsNhA/cAV40/vgr47hy3vCrJTlbfvri0qv467z0LE7HxG4WfAH7E6n9Q/1NVj8931asuAD7G6lnOI+O/PjTvUU18Erg9yaPAvwH/Pd85MD4zvAt4CPgFq78O5noFepI7gAeAs5PsT3INcCPwgSRPsXr2eOOC7PoqcDKwZ/xr4RubveuQjV6xL6mzhTkTk6RpGDFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrf0/ywX2efaA5DcAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 360x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAAEvCAYAAAAdNeeiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAAQ/0lEQVR4nO3dMW9TTZvG8evavOIT+KkAYVZCSCk3EeVbh21oSY20FR8AaT8HDUVEBzVFJFoaCuxiJShYZZERQSuBxQfgRbq3eGPJZu3jScbjM2P+P+kpbDJ3rsfHvnXG52TGESEAqN2/9B0AAFLQrAA0gWYFoAk0KwBNoFkBaALNCkAT/lai6GAwiOFwWKJ0kslkoul06mX/1nc2SRqPx9OI+GvZv5FvPfLlqTlf12e3SLMaDocajUYlSic5PDxc+W99Z5Mk259X/Rv51iNfnprzdX12mQYCaALNCkATkpqV7SPbH22f2X5SOhQA/G5ts7K9J+mppPuS9iUd294vHQwA5qWcWd2TdBYRnyLip6SXkh6UjQUAi1Ka1XVJX+Yen188BwBbs7Ev2G3/h+2R7dH37983VXYjas4mkS8X+fLUnm8mpVl9lXRz7vGNi+cWRMSziDiMiMO//lp6v1lvas4mkS8X+fLUnm8mpVm9k3TH9m3b1yQ9lPSqbCwAWLT2DvaI+GX7saTXkvYknUTEh+LJAGBO0p/bRMSppNPCWQBgJe5gB9AEmhWAJtCsADSBZgWgCTQrAE2gWQFoAs0KQBNoVgCakLKe1Yntb7bfbyMQACyTcmb1XNJR4RwA0Glts4qIN5J+bCELAKzEelYVIF8e8uWpPd/MxppVzWvi1JxNIl8u8uWpPd8MVwMBNIFmBaAJKbcuvJD0VtJd2+e2H5WPBQCLUlYKPd5GEADowjQQQBNoVgCaQLMC0ARHxOaL2t8lfe74kYGkacavWDf+VkQsvWEkIVtK/XVK5svNllKDfHnjyXf18auzlWhW69geRcRhX+NL1y+ZbxO1yVfnsd1E/V3OxzQQQBNoVgCa0Fezetbz+NL1S+bbRG3y9Te+dP2dzdfLd1YAcFlMAwE0Ye2f21zFYDCI4XBYonSSyWSi6XTqZf/WdzZJGo/H01WXZ3c933g8vvLYOSvz2c6aKhwcHOQMl/RnH99cXZ/dIs1qOBxqNBqVKJ3k8HD1ldG+s0mS7ZX3sex6Pnvp+/Cy1t0nd2WbeO3/5OObq+uzyzQQQBNoVgCakNSsbB/Z/mj7zPaT0qEA4Hcpi+/tSXoq6b6kfUnHtvdLBwOAeSlnVvcknUXEp4j4KemlpAdlYwHAopRmdV3Sl7nH5xfPAcDWsG9gBciXZz5f31mWaen1qzHfTEqz+irp5tzjGxfPLah577Gas0nkyzWfr+8sy7T0+tWYbyalWb2TdMf2bdvXJD2U9KpsLABYlLK7zS/bjyW9lrQn6SQiPhRPBgBzkv7cJiJOJZ0WzgIAK3EHO4Am0KwANIFmBaAJRZaIwe4aj8ebWualSrv8/9Y6zqwANIFmBaAJNCsATaBZAWhCynpWJ7a/2X6/jUAAsEzKmdVzSUeFcwBAp7XNKiLeSPqxhSwAsBLrWVWgpXx9Z1mmpXy1H98a881srFnVvCZOzdmktvL1nWWZlvLVfnxrzDfD1UAATaBZAWhCyq0LLyS9lXTX9rntR+VjAcCilJVCj7cRBAC6MA0E0ASaFYAm0KwANMERsfmi9ndJnzt+ZCBpmvEr1o2/FRFLbxhJyJZSf52S+XKzpdQgX9548l19/OpsJZrVOrZHOTfw5Y4vXb9kvk3UJl+dx3YT9Xc5H9NAAE2gWQFoQl/N6lnP40vXL5lvE7XJ19/40vV3Nl8v31kBwGUxDQTQhCL7Bg4GgxgOhyVKJ5lMJppOp0s3gOs7mySNx+Ppqsuzu55vPB5feeyclfls1zBV+GOPb66uz26RZjUcDjUa9bcO2uHh6iujfWeTJNsr72PZ9Xwb2kR03X1yfftjj2+urs8u00AATaBZAWhCUrOyfWT7o+0z209KhwKA36Usvrcn6amk+5L2JR3b3i8dDADmpZxZ3ZN0FhGfIuKnpJeSHpSNBQCLUprVdUlf5h6fXzwHAFvDvoEVIF8e9g3MU3u+mZRm9VXSzbnHNy6eW1Dz3mM1Z5PIl4t9A/PUnm8mpVm9k3TH9m3b1yQ9lPSqbCwAWJSyu80v248lvZa0J+kkIj4UTwYAc5L+3CYiTiWdFs4CACtxBzuAJtCsADShyKoLwCqbWOxxQys3oDGcWQFoAs0KQBNoVgCaQLMC0ISUJWJObH+z/X4bgQBgmZQzq+eSjgrnAIBOa5tVRLyR9GMLWQBgJb6zAtAE1rOqAPnysJ5VntrzzWysWdW8Jk7N2STy5WI9qzy155thGgigCSm3LryQ9FbSXdvnth+VjwUAi1IW3zveRhAA6MI0EEATaFYAmkCzAtAEb2IxtP9X1P4u6XPHjwwkTTN+xbrxtyJi6TXYhGwp9dcpmS83W0oN8uWNJ9/Vx6/OVqJZrWN7lHNPTO740vVL5ttEbfLVeWw3UX+X8zENBNAEmhWAJvTVrJ71PL50/ZL5NlGbfP2NL11/Z/P18p0VAFwW00AATaBZAWhCkU1OB4NBDIfDEqWTTCYTTafTpTth9p1Nksbj8XTVvSTkW68rn+0avtdo9vXrO1/XZ7dIsxoOhxqN+lsH7fBw9W0cfWeTJNsrb7oj33pd+SrR7OvXd76uzy7TQABNoFkBaEJSs7J9ZPuj7TPbT0qHAoDfpawUuifpqaT7kvYlHdveLx0MAOalnFndk3QWEZ8i4qekl5IelI0FAItSmtV1SV/mHp9fPAcAW8O+gRUgXx72DcxTe76ZlGb1VdLNucc3Lp5bUPPeYzVnk8iXi30D89SebyalWb2TdMf2bdvXJD2U9KpsLABYlLIV1y/bjyW9lrQn6SQiPhRPBgBzkv7cJiJOJZ0WzgIAK3EHO4Am0KwANKHIqgu4uvF4LHvpChlVqD0fdhdnVgCaQLMC0ASaFYAm0KwANCFliZgT299sv99GIABYJuXM6rmko8I5AKDT2mYVEW8k/dhCFgBYie+sADSB9awq0NJ6TH1nWaalfLW//2rMN7OxZlXzmjg1Z5PaWo+p7yzLtJSv9vdfjflmmAYCaELKrQsvJL2VdNf2ue1H5WMBwKKUxfeOtxEEALowDQTQBJoVgCbQrAA0wRGx+aL2d0mfO35kIGma8SvWjb8VEUuvwSZkS6m/Tsl8udlSapAvbzz5rj5+dbYSzWod26Oce2Jyx5euXzLfJmqTr85ju4n6u5yPaSCAJtCsADShr2b1rOfxpeuXzLeJ2uTrb3zp+jubr5fvrADgspgGAmhCkX0DB4NBDIfDEqWTTCYTTafTpZvb9Z1Nksbj8bTj0nENp7p/bL6Dg4Oc4ZK6j2/t77++83V9dos0q+FwqNGov6WFDg9XXxntO5sk2V53n1ff/th8m3hvdB3f2t9/fefr+uwyDQTQBJoVgCbQrAA0IalZ2T6y/dH2me0npUMBwO9SVgrdk/RU0n1J+5KObe+XDgYA81LOrO5JOouITxHxU9JLSQ/KxgKARSnN6rqkL3OPzy+eA4CtYd/ACrS0713fWZZpKV/t778a882kNKuvkm7OPb5x8dyCmvceqzmb1Na+d31nWaalfLW//2rMN5PSrN5JumP7tu1rkh5KelU2FgAsStmK65ftx5JeS9qTdBIRH4onA4A5SX8bGBGnkk4LZwGAlbiDHUATaFYAmlBkiZhdZi9dagc7guNbL86sADSBZgWgCTQrAE2gWQFoQsoSMSe2v9l+v41AALBMypnVc0lHhXMAQKe1zSoi3kj6sYUsALAS31kBaALrWVWgpfWY+s6yTEv5an//1ZhvZmPNquY1cWrOJrW1HlPfWZZpKV/t778a880wDQTQhJRbF15Ieivpru1z24/KxwKARSmL7x1vIwgAdGEaCKAJNCsATaBZAWiCI2LzRe3vkj53/MhA0jTjV6wbfysill6DTciWUn+dkvlys6XUIF/eePJdffzqbCWa1Tq2Rzn3xOSOL12/ZL5N1CZfncd2E/V3OR/TQABNoFkBaEJfzepZz+NL1y+ZbxO1ydff+NL1dzZfL99ZAcBlMQ0E0IQi+wYOBoMYDoclSieZTCaaTqdLN4DrO5skjcfjacel4xpOdVfm4/VLUizfwcFBznBJ3a9f38e367NbpFkNh0ONRv0tLXR4uPrKaN/ZJMn2uvu8+rYyH69fkmL5NvHad71+fR/frs8u00AATaBZAWgCzQpAE5Kale0j2x9tn9l+UjoUAPwuZaXQPUlPJd2XtC/p2PZ+6WAAMC/lzOqepLOI+BQRPyW9lPSgbCwAWJTSrK5L+jL3+PziuQU1b+dTczaJraRytfT69Z1lmdqP7wxbcVWAraTytPT69Z1lmdqP70xKs/oq6ebc4xsXzwHA1qQ0q3eS7ti+bfuapIeSXpWNBQCLUrbi+mX7saTXkvYknUTEh+LJAGBO0t8GRsSppNPCWQBgJe5gB9AEmhWAJtCsKnNwcKCI6PW/ltX++uXm+5PRrAA0gWYFoAk0KwBNoFkBaELKEjEntr/Zfr+NQACwTMqZ1XNJR4VzAECntc0qIt5I+rGFLACwEt9ZAWjCxppVzQt41ZxNIl8u8uWpPd8Mi+9VgHx5yJen9nwzTAMBNCHl1oUXkt5Kumv73Paj8rEAYFHK4nvH2wgCAF2YBgJoAs0KQBNoVgCa4BILetn+Lulzx48MJE0zfsW68bciYuk12IRsKfXXKZkvN1tKDfLljSff1cevztbH6oO2RzkbPuaOL12/ZL5N1CZfncd2E/V3OR/TQABNoFkBaEJfzepZz+NL1y+ZbxO1ydff+NL1dzZfL99ZAcBlMQ0E0ISk7eMvazAYxHA4LFE6yWQy0XQ69bJ/6zubJI3H4+mqy7PkW498eWrO1/XZLdKshsOhRqNRidJJDg9XXxntO5sk2V55Hwv51iNfnprzdX12mQYCaALNCkATaFYAmpDUrGwf2f5o+8z2k9KhAOB3KSuF7kl6Kum+pH1Jx7b3SwcDgHkpZ1b3JJ1FxKeI+CnppaQHZWMBwKKUZnVd0pe5x+cXzy2oeTufmrNJ5MtFvjy155thK64KkC8P+fLUnm8mpVl9lXRz7vGNi+cAYGtSmtU7SXds37Z9TdJDSa/KxgKARSlbcf2y/VjSa0l7kk4i4kPxZAAwJ+lvAyPiVNJp4SwAsBJ3sANoAs0KQBNoVgCaQLMC0ASaFYAm0KwANIFmBaAJKUvEnNj+Zvv9NgIBwDIpZ1bPJR0VzgEAndY2q4h4I+nHFrIAwEp8ZwWgCRtrVjUv4FVzNol8uciXp/Z8Myy+VwHy5SFfntrzzTANBNCElFsXXkh6K+mu7XPbj8rHAoBFKYvvHW8jCAB0YRoIoAk0KwBNoFkBaIIjYvNF7e+SPnf8yEDSNONXrBt/KyKWXoNNyJZSf52S+XKzpdQgX9548l19/OpsJZrVOrZHEXHY1/jS9Uvm20Rt8tV5bDdRf5fzMQ0E0ASaFYAm9NWsnvU8vnT9kvk2UZt8/Y0vXX9n8/XynRUAXBbTQABN2Gqzsn1k+6PtM9tPrjC+6BLL5CNfX9lqz1fFsY2IrfwnaU/S/0j6V0nXJP2XpP1L1vi7pH+T9J585NtmvpLZas9Xy7Hd5pnVPUlnEfEpIn5KeinpwWUKRNkllsmXZ6fzFc4m1Z2vimO7zWZ1XdKXucfnF8/Vgnx5yJen5nxVZOMLdgBN2Gaz+irp5tzjGxfP1YJ8eciXp+Z8VWTbZrN6J+mO7du2r0l6KOnVFn//OuTLQ748NeerI1uJKxsdVwT+XdJ/659XFv7zCuNfSPpfSf/QP+fNj8hHvm3kK52t9nw1HFvuYAfQBL5gB9AEmhWAJtCsADSBZgWgCTQrAE2gWQFoAs0KQBNoVgCa8H9bELKAbEWJEQAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 360x360 with 98 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Tokens: tensor([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  7, 11,  0,  0,  0,  0,  0,  4,\n",
            "        15,  0,  0,  0,  0,  3,  3, 15,  0,  0,  0,  1, 15, 15, 11,  0,  0,  0,\n",
            "         0, 12,  0, 12,  8,  0,  0,  0,  0,  0,  0,  0,  0]) \n",
            "\n",
            "\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAATEAAAEvCAYAAAAtufaDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAANBklEQVR4nO3db4jlBb3H8ffn7iSlSRoOUrty1wfiRaR7jSEsoQtqsJW4PbgPlAy9CfvkVhaBKD3oWQhFFBTFoqbQolw2I4n+uFghF0yaVTF1LcVK19b2hPSHemBL3/tgjjY7O+ueOefsnPO19wuWOee3Z+f3wXXf/M6Zs7OpKiSpq3+Z9QBJmoQRk9SaEZPUmhGT1JoRk9SaEZPU2sJmnuyss86q7du3b+YpJb1O7N+///dVtbj2+KZGbPv27SwvL2/mKSW9TiT5zXrHfTopqTUjJqk1IyapNSMmqbWJIpZkR5JfJHkmyU3TGiVJoxo7Ykm2AF8F3g9cAFyd5IJpDZOkUUxyJfYu4JmqeraqXgbuBnZOZ5YkjWaSiG0Fnl91/+DwmCRtmpP+wn6SXUmWkywPBoOTfTpJ/2QmidgLwDmr7m8bHjtKVe2uqqWqWlpcPOZvDEjSRCaJ2M+A85Kcm+QU4Crg3unMkqTRjP13J6vqSJKPAT8EtgC3V9UTU1smSSOY6C+AV9X3gO9NaYskbZjv2JfUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9Ta2BFLck6SHyd5MskTSW6Y5jBJGsXCBL/2CPDpqno4yenA/iT7qurJKW2TpBMa+0qsqg5V1cPD238GDgBbpzVMkkYxldfEkmwHLgIemsbnk6RRTRyxJG8GvgV8sqr+tM7P70qynGR5MBhMejpJOspEEUvyBlYCtqeq7lnvMVW1u6qWqmppcXFxktNJ0jEm+epkgNuAA1X1xelNkqTRTXIldgnwEeDSJI8Of3xgSrskaSRjv8Wiqv4PyBS3SNKG+Y59Sa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrY39j+eqp8R/73gcVTXrCToOr8QktWbEJLVmxCS1ZsQktWbEJLU2ccSSbEnySJLvTmOQJG3ENK7EbgAOTOHzSNKGTRSxJNuADwK3TmeOJG3MpFdiXwJuBP5+vAck2ZVkOcnyYDCY8HSSdLSxI5bkCuBwVe1/rcdV1e6qWqqqpcXFxXFPJ0nrmuRK7BLgyiS/Bu4GLk3yzamskqQRjR2xqrq5qrZV1XbgKuBHVXXN1JZJ0gh8n5ik1qbyXSyq6ifAT6bxuSRpI7wSk9SaEZPUmhGT1JoRk9Sa355aE/HbNmvWvBKT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JrfY/+fjN8TX683XolJas2ISWrNiElqzYhJam2iiCU5I8neJE8lOZDk3dMaJkmjmPSrk18GflBV/5XkFODUKWySpJGNHbEkbwHeC1wHUFUvAy9PZ5YkjWaSp5PnAgPgG0keSXJrktOmtEuSRjJJxBaAdwJfq6qLgL8AN619UJJdSZaTLA8GgwlOJ0nHmiRiB4GDVfXQ8P5eVqJ2lKraXVVLVbW0uLg4wekk6VhjR6yqXgSeT3L+8NBlwJNTWSVJI5r0q5MfB/YMvzL5LPDfk0+SpNFNFLGqehRYms4USdo437EvqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyaptYkiluRTSZ5I8niSu5K8cVrDJGkUY0csyVbgE8BSVV0IbAGumtYwSRrFpE8nF4A3JVkATgV+O/kkSRrd2BGrqheALwDPAYeAP1bVfdMaJkmjmOTp5JnATuBc4O3AaUmuWedxu5IsJ1keDAbjL5WkdUzydPJy4FdVNaiqvwH3AO9Z+6Cq2l1VS1W1tLi4OMHpJOlYk0TsOeDiJKcmCXAZcGA6syRpNJO8JvYQsBd4GPj58HPtntIuSRrJwiS/uKo+C3x2SlskacN8x76k1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWjJik1oyYpNaMmKTWThixJLcnOZzk8VXH3ppkX5Knhx/PPLkzJWl9o1yJ3QHsWHPsJuD+qjoPuH94X5I23QkjVlUPAC+tObwTuHN4+07gQ9OdJUmjGfc1sbOr6tDw9ovA2VPaI0kbMvEL+1VVQB3v55PsSrKcZHkwGEx6Okk6yrgR+12StwEMPx4+3gOrandVLVXV0uLi4pink6T1jRuxe4Frh7evBb4znTmStDGjvMXiLuBB4PwkB5NcD9wCvC/J08Dlw/uStOkWTvSAqrr6OD912ZS3SNKG+Y59Sa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrZ0wYkluT3I4yeOrjn0+yVNJHkvy7SRnnNSVknQco1yJ3QHsWHNsH3BhVb0D+CVw85R3SdJIThixqnoAeGnNsfuq6sjw7k+BbSdhmySd0DReE/so8P0pfB5J2rCJIpbkM8ARYM9rPGZXkuUky4PBYJLTSdIxxo5YkuuAK4APV1Ud73FVtbuqlqpqaXFxcdzTSdK6Fsb5RUl2ADcC/1lVf53uJEka3ShvsbgLeBA4P8nBJNcDXwFOB/YleTTJ10/yTkla1wmvxKrq6nUO33YStkjShvmOfUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa2lqjbvZMkA+M0IDz0L+P1JnjOOed0FbhuX28Yzi23/WlWLaw9uasRGlWS5qpZmvWOted0FbhuX28YzT9t8OimpNSMmqbV5jdjuWQ84jnndBW4bl9vGMzfb5vI1MUka1bxeiUnSSOYqYkl2JPlFkmeS3DTrPa9Ick6SHyd5MskTSW6Y9aa1kmxJ8kiS7856y2pJzkiyN8lTSQ4kefesNwEk+dTw9/LxJHcleeOM99ye5HCSx1cde2uSfUmeHn48c052fX74+/lYkm8nOWOzd602NxFLsgX4KvB+4ALg6iQXzHbVq44An66qC4CLgf+Zo22vuAE4MOsR6/gy8IOq+jfg35mDjUm2Ap8AlqrqQmALcNVsV3EHsGPNsZuA+6vqPOD+4f3NdgfH7toHXFhV7wB+Cdy82aNWm5uIAe8CnqmqZ6vqZeBuYOeMNwFQVYeq6uHh7T+z8gdx62xX/UOSbcAHgVtnvWW1JG8B3gvcBlBVL1fVH2Y66h8WgDclWQBOBX47yzFV9QDw0prDO4E7h7fvBD60mZtg/V1VdV9VHRne/SmwbbN3rTZPEdsKPL/q/kHmKBSvSLIduAh4aMZTVvsScCPw9xnvWOtcYAB8Y/hU99Ykp816VFW9AHwBeA44BPyxqu6b7ap1nV1Vh4a3XwTOnuWY4/go8P1ZDpiniM29JG8GvgV8sqr+NOs9AEmuAA5X1f5Zb1nHAvBO4GtVdRHwF2bzlOgow9eWdrIS2bcDpyW5ZrarXlutvI1grt5KkOQzrLzUsmeWO+YpYi8A56y6v214bC4keQMrAdtTVffMes8qlwBXJvk1K0/BL03yzdlOetVB4GBVvXLVupeVqM3a5cCvqmpQVX8D7gHeM+NN6/ldkrcBDD8envGeVyW5DrgC+HDN+H1a8xSxnwHnJTk3ySmsvNB674w3AZAkrLyuc6CqvjjrPatV1c1Vta2qtrPy3+xHVTUXVxVV9SLwfJLzh4cuA56c4aRXPAdcnOTU4e/tZczBFxzWcS9w7fD2tcB3ZrjlVUl2sPLyxZVV9ddZ75mbiA1fKPwY8ENW/of636p6YrarXnUJ8BFWrnIeHf74wKxHNfFxYE+Sx4D/AD432zkwvDLcCzwM/JyVPwczfQd6kruAB4HzkxxMcj1wC/C+JE+zcvV4y5zs+gpwOrBv+Gfh65u966iNvmNfUmdzcyUmSeMwYpJaM2KSWjNiklozYpJaM2KSWjNiklozYpJa+387e95nMAjlzAAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 360x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAAEvCAYAAAAdNeeiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAAQKklEQVR4nO3dMYtT3b7H8d/vzsVXkKdScXtBhCmvwfLW421snVq4lS9AOK/DxmKw09piwNbGwqQ4oIWHuTLiyAUNvgCP8L/FmUAiSfaaWVnZaw3fDzxFMq41X8iTP9nJnh1HhACgdv82dAAApGBYAWgCwwpAExhWAJrAsALQBIYVgCb8e4lNR6NRdF1XYuskp6enms1mXvWzodskaTqdziLir1U/o68ffXlq7tv03C0yrLqu02QyKbF1kvF4vPZnQ7dJku0v635GXz/68tTct+m5y2EggCYwrAA0IWlY2T6w/cn2ie2npaMA4E+9w8r2nqRnkh5I2pd0aHu/dBgALEp5ZXVf0klEfI6IX5JeSXpYNgsAlqUMq+uSvi7cPju/DwB2ZmtvsNv+H9sT25MfP35sa9utqLlNoi8XfXlq75tLGVbfJN1cuH3j/L4lEfE8IsYRMf7rr5Xnmw2m5jaJvlz05am9by5lWL2XdMf2bdvXJD2S9LpsFgAs6z2DPSJ+234i6Y2kPUlHEfGxeBkALEj6c5uIOJZ0XLgFANbiDHYATWBYAWgCwwpAExhWAJrAsALQBIYVgCYwrAA0gWEFoAkp17M6sv3d9oddBAHAKimvrF5IOijcAQAb9Q6riHgr6ecOWgBgLa5nVQH68tCXp/a+ua0Nq5qviVNzm0RfLvry1N43x6eBAJrAsALQhJRTF15Keifpru0z24/LZwHAspQrhR7uIgQANuEwEEATGFYAmsCwAtAER8T2N7V/SPqy4Z+MJM0yfkXf+lsRsfKEkYS2lP37lOzLbUvZg7689fRdfv36thLDqo/tSUSMh1pfev+SfdvYm746H9tt7H+V+zgMBNAEhhWAJgw1rJ4PvL70/iX7trE3fcOtL73/le0b5D0rALgoDgMBNKH3z20uYzQaRdd1JbZOcnp6qtls5lU/G7pNkqbT6Wzdx7P09aMvT819m567RYZV13WaTCYltk4yHq//ZHToNkmyvfY8Fvr60Zen5r5Nz10OAwE0gWEFoAlJw8r2ge1Ptk9sPy0dBQB/Srn43p6kZ5IeSNqXdGh7v3QYACxKeWV1X9JJRHyOiF+SXkl6WDYLAJalDKvrkr4u3D47vw8AdobvDawAfXnoy1N731zKsPom6ebC7Rvn9y2p+bvHam6T6MtFX57a++ZShtV7SXds37Z9TdIjSa/LZgHAspRvt/lt+4mkN5L2JB1FxMfiZQCwIOnPbSLiWNJx4RYAWIsz2AE0gWEFoAkMKwBNYFgBaALDCkATGFYAmsCwAtAEhhWAJqRcz+rI9nfbH3YRBACrpLyyeiHpoHAHAGzUO6wi4q2knztoAYC1uJ5VBejLQ1+e2vvmtjasar4mTs1tEn256MtTe98cnwYCaALDCkATUk5deCnpnaS7ts9sPy6fBQDLUq4UeriLEADYhMNAAE1gWAFoAsMKQBMcEdvf1P4h6cuGfzKSNMv4FX3rb0XEyhNGEtpS9u9Tsi+3LWUP+vLW03f59evbSgyrPrYnETEean3p/Uv2bWNv+up8bLex/1Xu4zAQQBMYVgCaMNSwej7w+tL7l+zbxt70Dbe+9P5Xtm+Q96wA4KI4DATQhN4/t7mM0WgUXdeV2DrJ6empZrOZV/1s6DZJmk6ns3Ufz+b2TafTS69dsLbPdg0vxYv13bt3L2e5pLKP7zbU3LfpuVtkWHVdp8lkUmLrJOPx+k9Gh26TJNtrz2PJ7bNXPs4X1Xce2tCK9W3j/42Sj+821Ny36bnLYSCAJjCsADQhaVjZPrD9yfaJ7aelowDgTykX39uT9EzSA0n7kg5t75cOA4BFKa+s7ks6iYjPEfFL0itJD8tmAcCylGF1XdLXhdtn5/cBwM7wvYEVaKlv6JZVWuqr/fGtsW8uZVh9k3Rz4faN8/uW1PzdYzW3SW31Dd2ySkt9tT++NfbNpQyr95Lu2L5t+5qkR5Jel80CgGUp327z2/YTSW8k7Uk6ioiPxcsAYEHSn9tExLGk48ItALAWZ7ADaALDCkATGFa4kHv37ikiBv2vZB/qxbAC0ASGFYAmMKwANIFhBaAJKZeIObL93faHXQQBwCopr6xeSDoo3AEAG/UOq4h4K+nnDloAYC3eswLQBK5nVQH68tCXp/a+ua0Nq5qviVNzm0RfLvry1N43x2EggCaknLrwUtI7SXdtn9l+XD4LAJalXHzvcBchALAJh4EAmsCwAtAEhhWAJrjEBcds/5D0ZcM/GUmaZfyKvvW3ImLlZ7AJbSn79ynZl9uWsgd9eevpu/z69W1DXB3R9iTnO95y15fev2TfNvamr87Hdhv7X+U+DgMBNIFhBaAJQw2r5wOvL71/yb5t7E3fcOtL739l+wZ5zwoALorDQABNYFgBaELv3wZexmg0iq7rSmyd5PT0VLPZzKt+NnSbJE2n09m6c0no60dfnpr7Nj13iwyrrus0mUxKbJ1kPF5/GsfQbZJke+1Jd/T1oy9PzX2bnrscBgJoAsMKQBOShpXtA9ufbJ/Yflo6CgD+lHKl0D1JzyQ9kLQv6dD2fukwAFiU8srqvqSTiPgcEb8kvZL0sGwWACxLGVbXJX1duH12fh8A7AzfG1gB+vLQl6f2vrmUYfVN0s2F2zfO71tS83eP1dwm0ZeLvjy1982lDKv3ku7Yvm37mqRHkl6XzQKAZSlfxfXb9hNJbyTtSTqKiI/FywBgQdKf20TEsaTjwi0AsBZnsANoAsMKQBMYVgCawLAC0ASGFYAmMKwANIFhBaAJKZeIObL93faHXQQBwCopr6xeSDoo3AEAG/UOq4h4K+nnDloAYC3eswLQBK5nVQH68tCXp/a+ua0Nq5qviVNzm0RfLvry1N43x2EggCaknLrwUtI7SXdtn9l+XD4LAJalXHzvcBchALAJh4EAmsCwAtAEhhWAJjgitr+p/UPSlw3/ZCRplvEr+tbfioiVn8EmtKXs36dkX25byh705a2n7/Lr17eVGFZ9bE8iYjzU+tL7l+zbxt701fnYbmP/q9zHYSCAJjCsADRhqGH1fOD1pfcv2beNvekbbn3p/a9s3yDvWQHARXEYCKAJSV8ff1Gj0Si6riuxdZLT01PNZjOv+tnQbZI0nU5n6z6epa8ffXlq7tv03C0yrLqu02QyKbF1kvF4/SejQ7dJku2157HQ14++PDX3bXruchgIoAkMKwBNYFgBaELSsLJ9YPuT7RPbT0tHAcCfUq4UuifpmaQHkvYlHdreLx0GAItSXlndl3QSEZ8j4pekV5Iels0CgGUpw+q6pK8Lt8/O7wOAneF7AytAXx768tTeN5cyrL5Jurlw+8b5fUtq/u6xmtsk+nLRl6f2vrmUYfVe0h3bt21fk/RI0uuyWQCwLOWruH7bfiLpjaQ9SUcR8bF4GQAsSPrbwIg4lnRcuAUA1uIMdgBNYFgBaALDCkATGFYAmsCwAtAEhhWAJjCsADQh5RIxR7a/2/6wiyAAWCXlldULSQeFOwBgo95hFRFvJf3cQQsArMV7VgCawPWsKkBfHvry1N43t7VhVfM1cWpuk+jLRV+e2vvmOAwE0ISUUxdeSnon6a7tM9uPy2cBwLKUi+8d7iIEADbhMBBAExhWAJrAsALQBEfE9je1f0j6suGfjCTNMn5F3/pbEbHyM9iEtpT9+5Tsy21L2YO+vPX0XX79+rYSw6qP7UlEjIdaX3r/kn3b2Ju+Oh/bbex/lfs4DATQBIYVgCYMNayeD7y+9P4l+7axN33DrS+9/5XtG+Q9KwC4KA4DATQh6evjL2o0GkXXdSW2TnJ6eqrZbOZVPxu6TZKm0+ls3cez9PWjL0/NfZueu0WGVdd1mkwmJbZOMh6v/2R06DZJsr32PBb6+tGXp+a+Tc9dDgMBNIFhBaAJDCsATUgaVrYPbH+yfWL7aekoAPhTypVC9yQ9k/RA0r6kQ9v7pcMAYFHKK6v7kk4i4nNE/JL0StLDslkAsCxlWF2X9HXh9tn5fUtq/jqfmtsk+nLRl6f2vjm+iqsC9OWhL0/tfXMpw+qbpJsLt2+c3wcAO5MyrN5LumP7tu1rkh5Jel02CwCWpXwV12/bTyS9kbQn6SgiPhYvA4AFSX8bGBHHko4LtwDAWpzBDqAJDCsATWBYAWgCwwpAExhWAJrAsALQBIYVgCakXCLmyPZ32x92EQQAq6S8snoh6aBwBwBs1DusIuKtpJ87aAGAtXjPCkATtjasar6AV81tEn256MtTe98cF9+rAH156MtTe98ch4EAmpBy6sJLSe8k3bV9Zvtx+SwAWJZy8b3DXYQAwCYcBgJoAsMKQBMYVgCa4IjY/qb2D0lfNvyTkaRZxq/oW38rIlZ+BpvQlrJ/n5J9uW0pe9CXt56+y69f31ZiWPWxPYmI8VDrS+9fsm8be9NX52O7jf2vch+HgQCawLAC0IShhtXzgdeX3r9k3zb2pm+49aX3v7J9g7xnBQAXxWEggCYkfX38RY1Go+i6rsTWSU5PTzWbzbzqZ0O3SdJ0Op2t+3iWvn705am5b9Nzt8iw6rpOk8mkxNZJxuP1n4wO3SZJtteex0JfP/ry1Ny36bnLYSCAJjCsADSBYQWgCUnDyvaB7U+2T2w/LR0FAH9KuVLonqRnkh5I2pd0aHu/dBgALEp5ZXVf0klEfI6IX5JeSXpYNgsAlqUMq+uSvi7cPju/b0nNX+dTc5tEXy768tTeN8dXcVWAvjz05am9by5lWH2TdHPh9o3z+wBgZ1KG1XtJd2zftn1N0iNJr8tmAcCylK/i+m37iaQ3kvYkHUXEx+JlALAg6W8DI+JY0nHhFgBYizPYATSBYQWgCQwrAE1gWAFoAsMKQBMYVgCawLAC0ISUS8Qc2f5u+8MuggBglZRXVi8kHRTuAICNeodVRLyV9HMHLQCwFu9ZAWjC1oZVzRfwqrlNoi8XfXlq75vj4nsVoC8PfXlq75vjMBBAE1JOXXgp6Z2ku7bPbD8unwUAy1Iuvne4ixAA2ITDQABNYFgBaALDCkATHBHb39T+IenLhn8ykjTL+BV9629FxMrPYBPaUvbvU7Ivty1lD/ry1tN3+fXr20oMqz62JxExHmp96f1L9m1jb/rqfGy3sf9V7uMwEEATGFYAmjDUsHo+8PrS+5fs28be9A23vvT+V7ZvkPesAOCiOAwE0ISdDivbB7Y/2T6x/fQS64teYpk++oZqq72visc2Inbyn6Q9Sf8r6T8kXZP0d0n7F9zjvyT9p6QP9NG3y76SbbX31fLY7vKV1X1JJxHxOSJ+SXol6eFFNoiyl1imL8+V7ivcJtXdV8Vju8thdV3S14XbZ+f31YK+PPTlqbmvijbeYAfQhF0Oq2+Sbi7cvnF+Xy3oy0Nfnpr7qmjb5bB6L+mO7du2r0l6JOn1Dn9/H/ry0Jen5r462kp8srHhE4H/lvQP/euThb9dYv1LSf8n6Z/613HzY/ro20Vf6bba+2p4bDmDHUATeIMdQBMYVgCawLAC0ASGFYAmMKwANIFhBaAJDCsATWBYAWjC/wMkEJWb/YFrVwAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 360x360 with 98 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Tokens: tensor([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  5, 12,\n",
            "         8,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,\n",
            "         0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0]) \n",
            "\n",
            "\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAATEAAAEvCAYAAAAtufaDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAANY0lEQVR4nO3cf4jldb3H8efr7halSRoOUq7c9Q/xItK9xhCW0AU12Eq0P+4fSobehP3nVhaBKP3RfxehiIKiWNQUEuViRhL9cLFCLpg0q2LqWoqVrml7Qm5F/WHS+/4xR5udnXXPnPPdOefdPh+wzDnfPc73zY7z5HO+851PqgpJ6uqf5j2AJM3CiElqzYhJas2ISWrNiElqzYhJam37Vp7s1FNPrZ07d27lKSX9g9i3b9/vq2pp/fEtjdjOnTtZWVnZylNK+geR5DcbHfftpKTWjJik1oyYpNaMmKTWZopYkl1JfpHk6STXDzWUJE1q6ogl2QZ8FfgAcA5wRZJzhhpMkiYxy0rs3cDTVfVMVb0M3AlcNsxYkjSZWSJ2OvDcmucHxsckacsc8wv7SXYnWUmyMhqNjvXpJB1nZonY88AZa57vGB87RFXtqarlqlpeWjrsNwYkaSazROxnwFlJzkzyRuBy4J5hxpKkyUz9u5NV9UqSjwM/BLYBt1TV44NNJkkTmOkXwKvqe8D3BppFkjbNO/YltWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1NnXEkpyR5MdJnkjyeJJrhxxMkiaxfYb/9hXgM1X1UJKTgH1J9lbVEwPNJklHNfVKrKpeqKqHxo//BOwHTh9qMEmaxCDXxJLsBM4DHhzi80nSpGaOWJK3AN8CPlVVf9zg73cnWUmyMhqNZj2dJB1ipogleQOrAbu9qu7e6DVVtaeqlqtqeWlpaZbTSdJhZvnpZICbgf1V9cXhRpKkyc2yErsA+ChwYZJHxn8+ONBckjSRqW+xqKr/BTLgLJK0ad6xL6k1IyapNSMmqTUjJqm1WX53UlpYq3cAabOqat4jbJorMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtuce+FsaQ++Iv8l7x7v8/LFdiklozYpJaM2KSWjNiklozYpJamzliSbYleTjJd4cYSJI2Y4iV2LXA/gE+jyRt2kwRS7ID+BBw0zDjSNLmzLoS+xJwHfC3I70gye4kK0lWRqPRjKeTpENNHbEklwAHq2rf672uqvZU1XJVLS8tLU17Okna0CwrsQuAS5P8GrgTuDDJNweZSpImNHXEquqGqtpRVTuBy4EfVdWVg00mSRPwPjFJrQ2yi0VV/QT4yRCfS5I2w5WYpNaMmKTWjJik1oyYpNbcnlozOV62lF5kx/u/mysxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa25x/5xZsg98cH93TV/rsQktWbEJLVmxCS1ZsQktTZTxJKcnOSuJE8m2Z/kPUMNJkmTmPWnk18GflBV/5HkjcAJA8wkSRObOmJJ3gq8D7gaoKpeBl4eZixJmswsbyfPBEbAN5I8nOSmJCcONJckTWSWiG0H3gV8rarOA/4MXL/+RUl2J1lJsjIajWY4nSQdbpaIHQAOVNWD4+d3sRq1Q1TVnqparqrlpaWlGU4nSYebOmJV9SLwXJKzx4cuAp4YZCpJmtCsP538BHD7+CeTzwD/OftIkjS5mSJWVY8Ay8OMIkmb5x37klozYpJaM2KSWjNiklozYpJac3vqBobcUvp42U566G24h3S8fA22iisxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrbk9dQPHy3bGbimtabgSk9SaEZPUmhGT1JoRk9SaEZPU2kwRS/LpJI8neSzJHUneNNRgkjSJqSOW5HTgk8ByVZ0LbAMuH2owSZrErG8ntwNvTrIdOAH47ewjSdLkpo5YVT0PfAF4FngB+ENV3TvUYJI0iVneTp4CXAacCbwDODHJlRu8bneSlSQro9Fo+kklaQOzvJ28GPhVVY2q6q/A3cB717+oqvZU1XJVLS8tLc1wOkk63CwRexY4P8kJWf2lt4uA/cOMJUmTmeWa2IPAXcBDwM/Hn2vPQHNJ0kRm2sWiqj4HfG6gWSRp07xjX1JrRkxSa0ZMUmtGTFJrRkxSa+6xr5ks6r747ol//HAlJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTUjJqk1IyapNSMmqTW3pz7OLOp20uCW0pqOKzFJrRkxSa0ZMUmtGTFJrRkxSa0dNWJJbklyMMlja469LcneJE+NP55ybMeUpI1NshK7Fdi17tj1wH1VdRZw3/i5JG25o0asqu4HXlp3+DLgtvHj24APDzuWJE1m2mtip1XVC+PHLwKnDTSPJG3KzBf2a/U26yPeap1kd5KVJCuj0WjW00nSIaaN2O+SvB1g/PHgkV5YVXuqarmqlpeWlqY8nSRtbNqI3QNcNX58FfCdYcaRpM2Z5BaLO4AHgLOTHEhyDXAj8P4kTwEXj59L0pY76i4WVXXFEf7qooFnkaRN8459Sa0ZMUmtGTFJrRkxSa0ZMUmtucd+A+6LLx2ZKzFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtGTFJrRkxSa0ZMUmtuT31MTD0dtJuAS0dmSsxSa0ZMUmtGTFJrRkxSa0ZMUmtHTViSW5JcjDJY2uOfT7Jk0keTfLtJCcf0ykl6QgmWYndCuxad2wvcG5VvRP4JXDDwHNJ0kSOGrGquh94ad2xe6vqlfHTnwI7jsFsknRUQ1wT+xjw/QE+jyRt2kwRS/JZ4BXg9td5ze4kK0lWRqPRLKeTpMNMHbEkVwOXAB+p1/m9mKraU1XLVbW8tLQ07ekkaUNT/e5kkl3AdcC/V9Vfhh1JkiY3yS0WdwAPAGcnOZDkGuArwEnA3iSPJPn6MZ5TkjZ01JVYVV2xweGbj8EskrRp3rEvqTUjJqk1IyapNSMmqTUjJqk199g/BtwTX9o6rsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1ZsQktWbEJLVmxCS1lq3cSjnJCPjNBC89Ffj9MR5nGos6FzjbtJxtOvOY7Z+ramn9wS2N2KSSrFTV8rznWG9R5wJnm5azTWeRZvPtpKTWjJik1hY1YnvmPcARLOpc4GzTcrbpLMxsC3lNTJImtagrMUmayEJFLMmuJL9I8nSS6+c9z6uSnJHkx0meSPJ4kmvnPdN6SbYleTjJd+c9y1pJTk5yV5Ink+xP8p55zwSQ5NPjr+VjSe5I8qY5z3NLkoNJHltz7G1J9iZ5avzxlAWZ6/Pjr+ejSb6d5OStnmuthYlYkm3AV4EPAOcAVyQ5Z75TveYV4DNVdQ5wPvBfCzTbq64F9s97iA18GfhBVf0L8K8swIxJTgc+CSxX1bnANuDy+U7FrcCudceuB+6rqrOA+8bPt9qtHD7XXuDcqnon8Evghq0eaq2FiRjwbuDpqnqmql4G7gQum/NMAFTVC1X10Pjxn1j9Rjx9vlP9XZIdwIeAm+Y9y1pJ3gq8D7gZoKperqr/m+tQf7cdeHOS7cAJwG/nOUxV3Q+8tO7wZcBt48e3AR/eyplg47mq6t6qemX89KfAjq2ea61FitjpwHNrnh9ggULxqiQ7gfOAB+c8ylpfAq4D/jbnOdY7ExgB3xi/1b0pyYnzHqqqnge+ADwLvAD8oarune9UGzqtql4YP34ROG2ewxzBx4Dvz3OARYrYwkvyFuBbwKeq6o/zngcgySXAwaraN+9ZNrAdeBfwtao6D/gz83lLdIjxtaXLWI3sO4ATk1w536leX63eRrBQtxIk+Syrl1pun+ccixSx54Ez1jzfMT62EJK8gdWA3V5Vd897njUuAC5N8mtW34JfmOSb8x3pNQeAA1X16qr1LlajNm8XA7+qqlFV/RW4G3jvnGfayO+SvB1g/PHgnOd5TZKrgUuAj9Sc79NapIj9DDgryZlJ3sjqhdZ75jwTAEnC6nWd/VX1xXnPs1ZV3VBVO6pqJ6v/Zj+qqoVYVVTVi8BzSc4eH7oIeGKOI73qWeD8JCeMv7YXsQA/cNjAPcBV48dXAd+Z4yyvSbKL1csXl1bVX+Y9z8JEbHyh8OPAD1n9H+p/qurx+U71mguAj7K6ynlk/OeD8x6qiU8Atyd5FPg34L/nOw6MV4Z3AQ8BP2f1+2Cud6AnuQN4ADg7yYEk1wA3Au9P8hSrq8cbF2SurwAnAXvH3wtf3+q5DpnRO/YldbYwKzFJmoYRk9SaEZPUmhGT1JoRk9SaEZPUmhGT1JoRk9Ta/wPB1v93KDK87AAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 360x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAAEvCAYAAAAdNeeiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAARHUlEQVR4nO3dv24T35/G8efZrLgC/ypAmJUQUspNRLl12IaW1EhbcQFIex00FBEd1BSRaGko8BQrQcEqi4IIWgksLoBF+mzxiyWbn/8cfOZ4zsn3/ZIo7OQcP4zlRzOeyRlHhACgdv80dAAASEFZAWgCZQWgCZQVgCZQVgCaQFkBaMI/l5h0NBrFeDwuMXWS8/NzTadTL/vZ0Nkkqeu6aUT8bdnPyLcZ+fLUnG/dZ7dIWY3HY00mkxJTJzk8PFz5s6GzSZLtz6t+Rr7NyJen5nzrPrscBgJoAmUFoAlJZWX7yPZH22e2n5QOBQC/21hWtvckPZV0X9K+pGPb+6WDAcC8lD2re5LOIuJTRPyU9FLSg7KxAGBRSlldl/Rl7vHF5XMAsDO9fcFu+z9sT2xPvn//3te0vag5m0S+XOTLU3u+mZSy+irp5tzjG5fPLYiIZxFxGBGHf/vb0uvNBlNzNol8uciXp/Z8Myll9U7SHdu3bV+T9FDSq7KxAGDRxivYI+KX7ceSXkvak3QSER+KJwOAOUl/bhMRp5JOC2cBgJW4gh1AEygrAE2grAA0gbIC0ATKCkATKCsATaCsADSBsgLQhJT1rE5sf7P9fheBAGCZlD2r55KOCucAgLU2llVEvJH0YwdZAGAl1rOqAPnykC9P7flmeiurmtfEqTmbRL5c5MtTe74ZzgYCaAJlBaAJKZcuvJD0VtJd2xe2H5WPBQCLUlYKPd5FEABYh8NAAE2grAA0gbIC0ARHRP+T2t8lfV7zKyNJ04yX2DT+VkQsvWAkIVvK/JuUzJebLWUO8uWNJ9/241dnK1FWm9ieRMThUONLz18yXx9zk6/O97aP+a9yPg4DATSBsgLQhKHK6tnA40vPXzJfH3OTb7jxpee/svkG+c4KAP4Uh4EAmrDxz222MRqNYjwel5g6yfn5uabTqZf9bOhsktR13XTV6VnybUa+PDXnW/fZLVJW4/FYk8mkxNRJDg9XnxkdOpsk2V55HQv5NiNfnprzrfvschgIoAmUFYAmJJWV7SPbH22f2X5SOhQA/C5l8b09SU8l3Ze0L+nY9n7pYAAwL2XP6p6ks4j4FBE/Jb2U9KBsLABYlFJW1yV9mXt8cfkcAOwM9w2sAPnykC9P7flmUsrqq6Sbc49vXD63oOZ7j9WcTSJfLvLlqT3fTEpZvZN0x/Zt29ckPZT0qmwsAFiUcnebX7YfS3otaU/SSUR8KJ4MAOYk/blNRJxKOi2cBQBW4gp2AE2grAA0gbIC0IQiS8QAQ+m6TvbS5ZCqUHu+mrFnBaAJlBWAJlBWAJpAWQFoQsp6Vie2v9l+v4tAALBMyp7Vc0lHhXMAwFobyyoi3kj6sYMsALAS61lVgHx55vMNnWWZlvLV+P7O9FZWNa+JU3M2iXy55vMNnWWZlvLV+P7OcDYQQBMoKwBNSLl04YWkt5Lu2r6w/ah8LABYlLJS6PEuggDAOhwGAmgCZQWgCZQVgCY4Ivqf1P4u6fOaXxlJmma8xKbxtyJi6QUjCdlS5t+kZL7cbClzkC9vPPm2H786W4my2sT2JOcCudzxpecvma+PuclX53vbx/xXOR+HgQCaQFkBaMJQZfVs4PGl5y+Zr4+5yTfc+NLzX9l8g3xnBQB/isNAAE0oct/A0WgU4/G4xNRJzs/PNZ1Ol96cbehsktR13XTV6dmrnq/ruq3Hzml2+9mu4VCm2u237rNbpKzG47Emk+HWGTs8XH1mdOhskmR75XUsVz1fTzf4bHb7VaLa7bfus8thIIAmUFYAmpBUVraPbH+0fWb7SelQAPC7lMX39iQ9lXRf0r6kY9v7pYMBwLyUPat7ks4i4lNE/JT0UtKDsrEAYFFKWV2X9GXu8cXlcwCwM9w3sALky9NSvqGzLFP79ptJKauvkm7OPb5x+dyCmu89VnM2iXy5Wso3dJZlat9+Myll9U7SHdu3bV+T9FDSq7KxAGBRyt1tftl+LOm1pD1JJxHxoXgyAJiT9Oc2EXEq6bRwFgBYiSvYATSBsgLQhCKrLuDq6roua+WEPhZ77Gnlhib9lbcfe1YAmkBZAWgCZQWgCZQVgCakLBFzYvub7fe7CAQAy6TsWT2XdFQ4BwCstbGsIuKNpB87yAIAK/GdFYAmsJ5VBVrKN3SWZdh+eWrffjO9lVXNa+LUnE1qK9/QWZZh++WpffvNcBgIoAkply68kPRW0l3bF7YflY8FAItSFt873kUQAFiHw0AATaCsADSBsgLQBPexmNc/TGp/l/R5za+MJE0zXmLT+FsRsfQcbEK2lPk3KZkvN1vKHOTLG0++7cevzlairDaxPcm55iR3fOn5S+brY27y1fne9jH/Vc7HYSCAJlBWAJowVFk9G3h86flL5utjbvINN770/Fc23yDfWQHAn+IwEEATKCsATShyk9PRaBTj8bjE1EnOz881nU6X3slx6GyS1HXddNW1JLn5uq7beuycYvn6UHL79WFdPttZ37scHBzkDJdU9/Zb99ktUlbj8ViTyXDrjB0err6MY+hskmR75UV3ufl6uttusXx9KLn9+rAuX64+/m81b791n10OAwE0gbIC0ISksrJ9ZPuj7TPbT0qHAoDfpawUuifpqaT7kvYlHdveLx0MAOal7Fndk3QWEZ8i4qekl5IelI0FAItSyuq6pC9zjy8unwOAneG+gRUgX56W8g2dZZnat99MSll9lXRz7vGNy+cW1HzvsZqzSeTL1VK+obMsU/v2m0kpq3eS7ti+bfuapIeSXpWNBQCLUm7F9cv2Y0mvJe1JOomID8WTAcCcpD+3iYhTSaeFswDASlzBDqAJlBWAJhRZdQHb67oua+WEPlZ+7WnlhkHkbr/SDg4OBl8VolXsWQFoAmUFoAmUFYAmUFYAmpCyRMyJ7W+23+8iEAAsk7Jn9VzSUeEcALDWxrKKiDeSfuwgCwCsxHdWAJrAelYVYL2jPGy/PLXnm+mtrGpeE6fmbBLrHeVi++WpPd8Mh4EAmpBy6cILSW8l3bV9YftR+VgAsChl8b3jXQQBgHU4DATQBMoKQBMoKwBNcB+Ltf3DpPZ3SZ/X/MpI0jTjJTaNvxURS8/BJmRLmX+Tkvlys6XMQb688eTbfvzqbCXKahPbk5xrYnLHl56/ZL4+5iZfne9tH/Nf5XwcBgJoAmUFoAlDldWzgceXnr9kvj7mJt9w40vPf2XzDfKdFQD8KQ4DATShyH0DR6NRjMfjElMnOT8/13Q6XXrzuKGzSVLXddNVp2ever6u67YeO2dlPttZhwoHBwc5wyX9td/fXOs+u0XKajweD3ojx8PD1WdGh84mSbZXXsdy1fP1dAPSTdfJba2Pbf9Xfn9zrfvschgIoAmUFYAmUFYAmpBUVraPbH+0fWb7SelQAPC7lJVC9yQ9lXRf0r6kY9v7pYMBwLyUPat7ks4i4lNE/JT0UtKDsrEAYFFKWV2X9GXu8cXlcwCwM9w3sALky8N9A/PUnm8mpay+Sro59/jG5XMLar73WM3ZJPLl4r6BeWrPN5NSVu8k3bF92/Y1SQ8lvSobCwAWpdyK65ftx5JeS9qTdBIRH4onA4A5SX8bGBGnkk4LZwGAlbiCHUATKCsATaCs8Ee6rpPtrf+VdnBwoIjY+h/qRVkBaAJlBaAJlBWAJlBWAJqQskTMie1vtt/vIhAALJOyZ/Vc0lHhHACw1sayiog3kn7sIAsArMR3VgCawHpWFWgp39BZlmlp+5Fve72VVc1r4tScTWor39BZlmlp+5FvexwGAmhCyqULLyS9lXTX9oXtR+VjAcCilMX3jncRBADW4TAQQBMoKwBNoKwANMElFhyz/V3S5zW/MpI0zXiJTeNvRcTSc7AJ2VLm36RkvtxsKXOQL288+bYfvzrbEKsj2p7kXLOTO770/CXz9TE3+ep8b/uY/yrn4zAQQBMoKwBNGKqsng08vvT8JfP1MTf5hhtfev4rm2+Q76wA4E9xGAigCUm3j/9To9EoxuNxiamTnJ+fazqdLr1J3dDZJKnruumq07O5+bqu23rsnJX5bGftih8cHOQMl1R2+/WBfNtb99ktUlbj8ViTyXBLHx0erj4zOnQ2SbK98jqW3Hw93Uh003VoW+tj25fcfn0g3/bWfXY5DATQBMoKQBMoKwBNSCor20e2P9o+s/2kdCgA+F3KSqF7kp5Kui9pX9Kx7f3SwQBgXsqe1T1JZxHxKSJ+Snop6UHZWACwKKWsrkv6Mvf44vK5BTXfzqfmbFJb+YbOskxL24982+NWXBVoKd/QWZZpafuRb3spZfVV0s25xzcunwOAnUkpq3eS7ti+bfuapIeSXpWNBQCLUm7F9cv2Y0mvJe1JOomID8WTAcCcpL8NjIhTSaeFswDASlzBDqAJlBWAJlBWlem6Tra3/lfawcGBImLrf8C2KCsATaCsADSBsgLQBMoKQBNSlog5sf3N9vtdBAKAZVL2rJ5LOiqcAwDW2lhWEfFG0o8dZAGAlfjOCkATeiurmhfwqjmbxOJ2uciXp/Z8Myy+VwEWt8tDvjy155vhMBBAE1IuXXgh6a2ku7YvbD8qHwsAFqUsvne8iyAAsA6HgQCaQFkBaAJlBaAJLrEgmu3vkj6v+ZWRpGnGS2wafysilp6DTciWMv8mJfPlZkuZg3x548m3/fjV2YZYvdH2JOeaotzxpecvma+PuclX53vbx/xXOR+HgQCaQFkBaMJQZfVs4PGl5y+Zr4+5yTfc+NLzX9l8g3xnBQB/isNAAE1Iun38nxqNRjEej0tMneT8/FzT6XTpTfRys3Vdt/XYOdNVp2eH3naS1HUd+TKQb3vrPrtFymo8HmsyGW5ppsPD1WdGc7P1dCPRldexDL3tJMk2+TKQb3vrPrscBgJoAmUFoAmUFYAmJJWV7SPbH22f2X5SOhQA/C5lpdA9SU8l3Ze0L+nY9n7pYAAwL2XP6p6ks4j4FBE/Jb2U9KBsLABYlFJW1yV9mXt8cfncgppv51NzNol8uciXp/Z8M9yKqwLky0O+PLXnm0kpq6+Sbs49vnH5HADsTEpZvZN0x/Zt29ckPZT0qmwsAFiUciuuX7YfS3otaU/SSUR8KJ4MAOYk/W1gRJxKOi2cBQBW4gp2AE2grAA0ocgSMVdZHyur9rTMDPCXwp4VgCZQVgCaQFkBaAJlBaAJKUvEnNj+Zvv9LgIBwDIpe1bPJR0VzgEAa20sq4h4I+nHDrIAwEp8ZwWgCb2VVc0LeNWcTSJfLvLlqT3fDIvvVYB8eciXp/Z8MxwGAmhCyqULLyS9lXTX9oXtR+VjAcCilMX3jncRBADW4TAQQBMoKwBNoKwANMF9LCb3D5Pa3yV9XvMrI0nTjJfYNP5WRCw9B5uQLWX+TUrmy82WMgf58saTb/vxq7OVKKtNbE8i4nCo8aXnL5mvj7nJV+d728f8Vzkfh4EAmkBZAWjCUGX1bODxpecvma+Puck33PjS81/ZfIN8ZwUAf4rDQABN2GlZ2T6y/dH2me0nW4wvusQy+cg3VLba81Xx3kbETv5J2pP0P5L+RdI1Sf8laf8P5/g3Sf8q6T35yLfLfCWz1Z6vlvd2l3tW9ySdRcSniPgp6aWkB38yQZRdYpl8ea50vsLZpLrzVfHe7rKsrkv6Mvf44vK5WpAvD/ny1Jyvimx8wQ6gCbssq6+Sbs49vnH5XC3Il4d8eWrOV0W2XZbVO0l3bN+2fU3SQ0mvdvj6m5AvD/ny1JyvjmwlzmysOSPw75L+W38/s/CfW4x/Iel/Jf2f/n7c/Ih85NtFvtLZas9Xw3vLFewAmsAX7ACaQFkBaAJlBaAJlBWAJlBWAJpAWQFoAmUFoAmUFYAm/D9kumRNHLK2IgAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 360x360 with 98 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Tokens: tensor([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  3,  3,  0,  0,  0,  0,  6,\n",
            "         0, 14,  0,  0,  0,  6,  0,  3,  8,  0,  0,  0,  0,  7,  8,  0,  0,  0,\n",
            "         0,  7,  8,  0,  0,  0,  0,  4,  0,  0,  0,  0,  0]) \n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "imgs, _ = next(iter(loader_train))\n",
        "\n",
        "tokens = tokenize_MNIST(imgs, patch_size=2)\n",
        "reconst = detokenize_MNIST(tokens, patch_size=2)\n",
        "bits = rearrange(reconst, 'b (nh ph) (nw pw) -> b (nh nw) ph pw', ph=2, pw=2)\n",
        "\n",
        "for i in range(3):\n",
        "    plt.figure(figsize=(5., 5.)); plt.imshow(reconst[i], cmap='Greys'); plt.show()\n",
        "    \n",
        "    fig = plt.figure(figsize=(5., 5.))\n",
        "    grid = ImageGrid(fig, 111, nrows_ncols=(7, 7), axes_pad=0.1)\n",
        "    for j, img in enumerate(bits[i]):\n",
        "        grid[j].imshow(img, cmap='Greys', vmin=0, vmax=1)\n",
        "    plt.show()\n",
        "    \n",
        "    print('Tokens:', tokens[i], '\\n\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "12cf128e-357c-4a80-a2d7-0be4c54b09e2",
      "metadata": {
        "id": "12cf128e-357c-4a80-a2d7-0be4c54b09e2"
      },
      "source": [
        "### 3.4 GPT implementation (20 pts)\n",
        "\n",
        "In the next cell, implement a GPT model. Specifically, implement the following:\n",
        "- Embed the token indices into the desired Transformer token dimension\n",
        "- Add to each token a unique positional embedding\n",
        "- Create a causal mask that restricts the self-attention to the same and previous tokens\n",
        "- Run the tokens through the Transformer blocks\n",
        "- Compute per-token logits"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "146e2633-b6c2-4e74-897a-69a12cb2bd96",
      "metadata": {
        "id": "146e2633-b6c2-4e74-897a-69a12cb2bd96"
      },
      "outputs": [],
      "source": [
        "class GPT(nn.Module):\n",
        "    def __init__(self, max_tokens=49+1, vocab_size=16+2, embed_dim=192, depth=4, num_heads=4, mlp_ratio=4., **kwargs):\n",
        "        '''\n",
        "        GPT to autoregressively model sequences.\n",
        "        \n",
        "        params:\n",
        "            :max_tokens: Maximum sequence length (should include start-of-sequence token)\n",
        "            :vocab_size: Vocabulary size (should include start-of-sequence and end-of-sequence token)\n",
        "            :embed_dim: Token dimension\n",
        "            :depth: Transformer depth\n",
        "            :num_heads: Number of attention heads\n",
        "            :mlp_ratio: MLP hidden dimensionality multiplier\n",
        "        '''\n",
        "        super().__init__()\n",
        "        \n",
        "        self.embed = nn.Embedding(num_embeddings=vocab_size, embedding_dim=embed_dim)\n",
        "        # For simplicity, we will use a 1D positional embedding here, even though the data is 2D.\n",
        "        self.pos_embed = nn.Parameter(build_1d_sincos_posemb(max_tokens, embed_dim=embed_dim), requires_grad=False)\n",
        "        \n",
        "        self.blocks = nn.ModuleList([\n",
        "            Block(\n",
        "                dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio)\n",
        "            for i in range(depth)\n",
        "        ])\n",
        "        self.head = nn.Linear(embed_dim, vocab_size, bias=False)\n",
        "            \n",
        "    def forward(self, x):\n",
        "        '''\n",
        "        Forward pass through the GPT.\n",
        "        \n",
        "        params:\n",
        "            :x: Input of shape [B N]. B = batch size, N = sequence length\n",
        "            \n",
        "        returns:\n",
        "            Output of shape [B N vocab_size]\n",
        "        '''\n",
        "        B, N = x.shape\n",
        "        \n",
        "        # TODO: Embed the input x\n",
        "        # print(B, N)\n",
        "        x = self.embed(x)\n",
        "        \n",
        "        # TODO: Add to x the positional embeddings. Make sure to deal with the case, where N is smaller than the maximum sequence length.\n",
        "        x = x + self.pos_embed[:, :N, :]\n",
        "        # print(self.pos_embed.shape)\n",
        "        \n",
        "        # TODO: Create a causal mask of shape [B N N]. The mask should set values in the attention matrix \n",
        "        # to zero, such that tokens can only attend to previous tokens or themselves, but not at \"future\" tokens.\n",
        "        mask = ~torch.tril(torch.ones(B, N, N)).bool().to(device) # ??? device necessary?\n",
        "\n",
        "        # TODO: Forward pass through Transformer blocks. Make sure to pass the causal mask.\n",
        "        for block in self.blocks:\n",
        "          x = block(x, mask=mask)\n",
        "            \n",
        "        # TODO: Compute the logits for each token\n",
        "        logits = self.head(x)\n",
        "        \n",
        "        return logits\n",
        "    \n",
        "    @torch.no_grad()\n",
        "    def generate(self, seq, max_new_tokens, temperature=1.0, do_sample=True, top_k=None):\n",
        "        '''\n",
        "        Given a start sequence, generate the remaining tokens.\n",
        "        \n",
        "        params:\n",
        "            :seq: Input sequence of shape [B n]. To generate samples from scratch, pass a [B 1] tensor full of the \"start sequence\" token.\n",
        "            :max_new_tokens: Number of tokens to generate\n",
        "            :temperature: Sample tokens with the desired temperature\n",
        "            :do_sample: Set to True to sample and False for simply picking the most likely token\n",
        "            :top_k: Optionally restrict the logits to top k options\n",
        "        '''\n",
        "        for _ in range(max_new_tokens):\n",
        "            logits = self(seq)\n",
        "             # pluck the logits at the final step and scale by desired temperature\n",
        "            logits = logits[:, -1, :] / temperature\n",
        "            # optionally crop the logits to only the top k options\n",
        "            if top_k is not None:\n",
        "                v, _ = torch.topk(logits, top_k)\n",
        "                logits[logits < v[:, [-1]]] = -float('Inf')\n",
        "            # apply softmax to convert logits to (normalized) probabilities\n",
        "            probs = F.softmax(logits, dim=-1)\n",
        "            # either sample from the distribution or take the most likely element\n",
        "            if do_sample:\n",
        "                idx_next = torch.multinomial(probs, num_samples=1)\n",
        "            else:\n",
        "                _, idx_next = torch.topk(probs, k=1, dim=-1)\n",
        "            # append sampled index to the running sequence and continue\n",
        "            seq = torch.cat((seq, idx_next), dim=1)\n",
        "            \n",
        "        return seq"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a5819551-f63e-4014-bbac-479a2b491d17",
      "metadata": {
        "id": "a5819551-f63e-4014-bbac-479a2b491d17"
      },
      "source": [
        "### 3.5 Train iGPT\n",
        "\n",
        "Let's train the GPT on MNIST and generate some samples!\n",
        "We prepend every sequence with a start-of-sequence token and append an end-of-sequence token.\n",
        "We train the Decoder-only transformer to predict a shifted version of the input and thanks to the causal mask, tokens can only attend to previous tokens and themselves.\n",
        "\n",
        "Don't worry if not all of the generated images look like MNIST digits, though most of them should look somewhat plausible."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "af31a235-1f40-4cc2-b3ea-12da5072e188",
      "metadata": {
        "id": "af31a235-1f40-4cc2-b3ea-12da5072e188"
      },
      "outputs": [],
      "source": [
        "patch_size = 2\n",
        "vocab_size = 2**(patch_size*patch_size)\n",
        "max_tokens = (image_size//patch_size)**2\n",
        "\n",
        "gpt = GPT(\n",
        "    max_tokens=max_tokens+1, # number of tokens for one image + one for start-of-sequence token\n",
        "    vocab_size=vocab_size+2, # two additional vocabulary items for start-of-sequence and end-of-sequence token\n",
        "    embed_dim=192, \n",
        "    depth=4, \n",
        "    num_heads=4, \n",
        "    mlp_ratio=4.\n",
        ").to(device)\n",
        "optimizer = torch.optim.AdamW(gpt.parameters())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5a91c11d-2c7b-420c-b820-a2b056fcedd1",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 510,
          "referenced_widgets": [
            "c53979b4234f4ebfb280ff2f7eddf293",
            "12f4bc5979f04627af30adebfdf1b221",
            "dfa2923ad7d34c8a9d37212c91da8fc4",
            "8cab9058b9aa4b4792149b9ab4ddc64e",
            "32903d82d192473e8b37796617467a97",
            "945248631b81441abca170f766754603",
            "98104971d2774746b914e801f3d527b6",
            "57307bac1b314c7bb1541cc6c4bf77c5",
            "893c262eb59e48fa95dfe316177e82f3",
            "d2e2f6883ee847e985c4f9199e3079f3",
            "ea098e2276654d62a3f4326b309602a6",
            "0cf4a1e0413b4092a842b6383d76a91c",
            "0fd7ddd5f1d64151ba8c87a41219c98f",
            "372fb52fe7b148edad97ba7290817610",
            "14aae9b86e7449aba3558cccb16d8abd",
            "6c0a4272ad2e4ec485fd0589e9bfce88",
            "559523f1ba444ec2a17af9d00e56e0f5",
            "d34673fcd913450a8ac5869b59b57275",
            "f56b98e2535f48969fa64bcf4c32ed6f",
            "c8d6ff0e5df04fc19e1f5ecc0c3dcc3a",
            "303b1a517fc843da9cd8f349f7b2c425",
            "ef2b0b2022ed48a7b9d67acdb958b6a2",
            "51db5d4ebae046d999acb1b378054e35",
            "15575d779c504287803d20a563848058",
            "f008f48ed3244f969592774a627e66ec",
            "29e4df8f59774788b8865bf9743ecabc",
            "96ed74eff0ca4be583223afab0a145e7",
            "98575417888545f49819162112a2e56e",
            "5f755c5160004595aa9afc51a457add4",
            "dd37bf2d158047a3ba6b6b5701dd3761",
            "38045f1d73c44939a7b35e32ba7e5ecb",
            "d99936049aad4b55867639bf2df2ae32",
            "cf6678ff63034625b5050f2653931cfc",
            "ab23f1dbc29c4a7da080fd65d7823753",
            "e04e364f2eb242999324cbe1df91628c",
            "0067fc4fc58244639c6a83fdab0b7e74",
            "6728963c26d346d79fb9a26f4d112d8e",
            "5bfb15a87f17454b893293fb8a7dc110",
            "3cc4e194924e4ea18fbdfcdfe5d6305e",
            "bccaad81be33461286ef5bc347c17c26",
            "c9219051a617407ab1db49590bc0b7d9",
            "af915aa8f67d45e6ad5cd311846e5ec8",
            "6da293f3e86044bc9e22f27968f1733c",
            "40b0ac09afe54e98a7c839815b504bcd",
            "5d61b5810b124ca8b778c40dfead927c",
            "cfcfb371ca8949769ff6f79bc367c298",
            "7d678bf7171e4b659a967a6ca127805a",
            "df47640718b4433597228fb8f9dfec0e",
            "46ac8c9ef4e241d2865c4cfc10895b62",
            "d6f1815e37df4b5e91cafb0c096b7935",
            "f6635ac72ee24817903921c52d72031e",
            "e0eed1f3b47c475c99298e0b5d58bea4",
            "aa4925cf508046cf86bcc2b07fc5e988",
            "9c12f610fd334ed8ae618959153e4acb",
            "8c4299113ba642498129668cbd066d6e",
            "4f45a8840b27419a8cc5ab4cfecd2410",
            "09ed4e0249d347a6a76030d208a4ac09",
            "f76e2720d2df499ca254afc5a3217ee0",
            "00abf37d10bb430fa4bc7e49705aa725",
            "9b11ae43dbd74a99913c2bdd3756e8b8",
            "63373abdf7d84dd38c1531604b02c36e",
            "f25db568553a4a3aa78f63e5795d4c2a",
            "ec704019e7b64d51b10f12d0d06b29be",
            "1ec9a65c205e445e88a9e1770a2a070d",
            "43282135126a40949fcaf86c302fab84",
            "1412119ab3564f6aaf53d9d8bd694150",
            "ec4a11ff4a34403db7f53590eefa05f4",
            "4f35d75810f142aeade2f3b06e5cb699",
            "305d94d8ab2b44f8b81d4edd9fe079fa",
            "126edea5f38c45799959d5c0282ce914",
            "fe6d2eddf40e493e861ec0cb4d2168d2",
            "41a2b31c5adc48a49c5e7a81a4f7e80a",
            "88120b2053854c1a8f7e472bfafb388a",
            "19a38e9cb8b442f88363e3cc2be1c92b",
            "7f5d4bab37e7453daae7d4785d8047e8",
            "3cd9332d489a466398f5c4e30dcc3846",
            "7c4bebfc5f30459aaad9f93812583fe1",
            "4b7ef8d3b7284cf4addea2ea63d642e4",
            "1a99277fe0ff4629a9d34753e5f0a6c3",
            "8cae1aa38f8e44faa35337c99982fc7f",
            "76fd901f7f6b4c09bb4ce8fca806c4d8",
            "9fb8fdb1f4034480841c1fa364e10edc",
            "ee3dce92cc3e4f4b892a91b2bccf1ef2",
            "1530d6fe1d354981bebf1e6fe6ebc34c",
            "adc2ad5f22b845329301c33b9842c9fc",
            "beb5b3e87563463a920f68f065b911e7",
            "9649d26dfe40423ebea71132d24b7848",
            "227fbdfa52884cd79383e414726a05d6",
            "5cee094c55004d058098a05d02c8b5cb",
            "9fb50fc960484d2187594c3671da6b7b",
            "077a521bfa64484d9057e5efe49f8078",
            "e0ba5051ac0a4d9ea6f80b2c0fe50c7c",
            "0e4983f18aaa4059a0ba4492c2e97ec7",
            "3233c47e64374330bf6323559883f5c0",
            "22900dc085094bb0a3c03b7f06708529",
            "1d3e3c3e3e5548f5a7564df29e3a7917",
            "79c39a1ff5354255b62f1fe9723cbb3b",
            "cac074db984d48ceba46f0eb83e8b776",
            "c8b506a740ab455c8ef0b6b1a7363752",
            "cbcbc3daea274f67b60783c7b1dd008b",
            "5bffe63966ad4232864fa9b3e60b0f1c",
            "ad7d0ee981914dceadb4cb8610512207",
            "ebdd5f1af9684cc4ae4966c4a312cad7",
            "af98ab56df7a4d5aa7b9448e0c277aad",
            "3c724e55d52d4d17b784f6a5b8f80a73",
            "bc90ab55ca2d4889ab12113123901283",
            "3905c161624544cea4381919e11885e5",
            "e7fac7d5a34346089efe5135aedb5822",
            "274f524ff843460b87fdef21efeabf32",
            "dfe02df2573941c18cd17a6c6d3c3360"
          ]
        },
        "id": "5a91c11d-2c7b-420c-b820-a2b056fcedd1",
        "outputId": "6a2f820b-e3e9-4b6b-e255-6aac17f47ba0"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "c53979b4234f4ebfb280ff2f7eddf293",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.621, val loss: 0.544\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0cf4a1e0413b4092a842b6383d76a91c",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.527, val loss: 0.516\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "51db5d4ebae046d999acb1b378054e35",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.513, val loss: 0.510\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ab23f1dbc29c4a7da080fd65d7823753",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.504, val loss: 0.505\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "5d61b5810b124ca8b778c40dfead927c",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.498, val loss: 0.503\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4f45a8840b27419a8cc5ab4cfecd2410",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.494, val loss: 0.500\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ec4a11ff4a34403db7f53590eefa05f4",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.490, val loss: 0.500\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4b7ef8d3b7284cf4addea2ea63d642e4",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.486, val loss: 0.497\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "5cee094c55004d058098a05d02c8b5cb",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.490, val loss: 0.496\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "cbcbc3daea274f67b60783c7b1dd008b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/781 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train loss: 0.477, val loss: 0.494\n"
          ]
        }
      ],
      "source": [
        "num_epochs = 10\n",
        "\n",
        "train_losses = []\n",
        "val_losses = []\n",
        "\n",
        "start_of_seq_token = vocab_size\n",
        "end_of_seq_token = vocab_size+1\n",
        "\n",
        "for _ in range(num_epochs):\n",
        "    \n",
        "    # Train loop\n",
        "    gpt.train()\n",
        "    epoch_loss_train = 0\n",
        "    pbar = tqdm(total=len(loader_train))\n",
        "    for imgs, _ in loader_train:\n",
        "        x = tokenize_MNIST(imgs, patch_size=patch_size).to(device)\n",
        "        inputs = torch.nn.functional.pad(x.clone(), (1,0,0,0), value=start_of_seq_token)\n",
        "        targets = torch.nn.functional.pad(x.clone(), (0,1,0,0), value=end_of_seq_token)\n",
        "\n",
        "        logits = gpt(inputs)\n",
        "        loss = F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1), ignore_index=-1)\n",
        "\n",
        "        gpt.zero_grad(set_to_none=True)\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "        \n",
        "        epoch_loss_train += loss.item()\n",
        "        \n",
        "        pbar.update(1)\n",
        "        pbar.set_description(f'Train loss: {loss.item():.3f}')\n",
        "    pbar.close()\n",
        "    \n",
        "    epoch_loss_train /= len(loader_train)\n",
        "    train_losses.append(epoch_loss_train)\n",
        "    \n",
        "    \n",
        "    # Validation loop\n",
        "    gpt.eval()\n",
        "    epoch_loss_val = 0\n",
        "    for imgs, _ in loader_val:\n",
        "        x = tokenize_MNIST(imgs, patch_size=patch_size).to(device)\n",
        "        inputs = torch.nn.functional.pad(x.clone(), (1,0,0,0), value=start_of_seq_token)\n",
        "        targets = torch.nn.functional.pad(x.clone(), (0,1,0,0), value=end_of_seq_token)\n",
        "\n",
        "        with torch.no_grad():\n",
        "            logits = gpt(inputs)\n",
        "        loss = F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1), ignore_index=-1)\n",
        "        \n",
        "        epoch_loss_val += loss.item()\n",
        "    \n",
        "    epoch_loss_val /= len(loader_val)\n",
        "    val_losses.append(epoch_loss_val)\n",
        "    \n",
        "    print(f'Train loss: {epoch_loss_train:.3f}, val loss: {epoch_loss_val:.3f}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a2f1f200-d220-41db-87ff-5af03fa78d5c",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a2f1f200-d220-41db-87ff-5af03fa78d5c",
        "outputId": "32953eeb-151a-46c3-9864-b506394d5b94"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1EAAANQCAYAAADJ2eUPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAABFe0lEQVR4nO3dv49kZ7kv+ufBFgnaAdxpjRAgmsDa0tGRkFDLJ0FoI7aRuTqSiRCOHCBNAn+ApRtsaUf8AzeZwLIT4OzEwoHFj+OEZAduMkACfK2xGAuYHnECZ/caPTeY4jAeurr7rfWsqlVVn4806q7qnn7f9e13rZpHq95nsqoCAACAm/nYricAAACwTxRRAAAAAxRRAAAAAxRRAAAAAxRRAAAAAxRRAAAAAyYVUZn5fGb+NjPfycyXuyYFAACwVLnp/xOVmU9FxO8i4rmIuB8Rb0fEi1X1m3V/59atW3V6errReMfm3r178fDhw7zsa3K8OTn2uCrHCFmOsCZ7yLGHHHu4RvaxJnvIscdVOT494ec+GxHvVNW7ERGZ+aOIeCEi1hZRp6encX5+PmHI43F2drb2a3K8OTn2uCrHCFmOsCZ7yLGHHHu4RvaxJnvIscdVOU55O99nIuIPjz2+v3oOAADgYM3eWCIz72TmeWaeX1xczD3cwZJjDzn2kWUPOfaQYw859pFlDzn2kGO/KUXU+xHxuccef3b13EdU1d2qOquqs5OTkwnDHTc59pBjH1n2kGMPOfaQYx9Z9pBjDzn2m1JEvR0Rz2TmFzLz4xHx7Yh4o2daAAAAy7RxY4mq+jAzvxcRP42IpyLilar6ddvMAAAAFmhKd76oqjcj4s2muQAAACze7I0lAAAADokiCgAAYIAiCgAAYIAiCgAAYIAiCgAAYIAiCgAAYIAiCgAAYMCk/ycKHpeZs/zcqprl58KxmOvcvCnn8COX/R5kczWZsTRTr6fW75gl/9vSnSgAAIABiigAAIABiigAAIABiigAAIABiigAAIABB9udb5vdqI6x04qOST3mWKd+D5s55DU99Th23d0PmMeSO58t1U2PzXXzaofw7x93ogAAAAYoogAAAAYoogAAAAYoogAAAAZMaiyRmfci4oOI+GtEfFhVZx2T2mAek/7+ZRvRbAh8ZF0Oh7xpdA4j68l63A1ruoccH3HOzueQm8DMSW7zcb6vN8e/0ZeiozvfV6vqYcPPAQAAWDxv5wMAABgwtYiqiPhZZv4yM+90TAgAAGDJphZRX66qL0XENyLiu5n5lSe/ITPvZOZ5Zp5fXFxMHO54ybGHHPvIsocce8ixhxz7yLKHHHvIsd+kIqqq3l99fBARr0fEs5d8z92qOquqs5OTkynDHTU59pBjH1n2kGMPOfaQYx9Z9pBjDzn227ixRGZ+IiI+VlUfrD7/ekT8e9vMZrLkLh+7pnPP9unEx9JYfwDTHeO/nw65E99lpnTnux0Rr68CezoiflBVP2mZFQAAwEJtXERV1bsR8cXGuQAAACyeFucAAAADFFEAAAADpuyJWoxtbkTbt01vU41sEhzJ5tg2H66b77HlwPLddE2tW7sa1NBJ8x2W5qbrb933uR4+cgg5uBMFAAAwQBEFAAAwQBEFAAAwQBEFAAAw4CAaS0xlk+oju97kt+vxd+Gmm6aPMRuWbaRZyjFusNYQYbuOcY2xG1Ob7/DIIfxbx50oAACAAYooAACAAYooAACAAYooAACAAYooAACAAbrzrbFvHUKWSsck9sUhdAoCGOG6R6epXUn3bT26EwUAADBAEQUAADBAEQUAADBAEQUAADDg2sYSmflKRPz3iHhQVf919dynIuJ/RMRpRNyLiG9V1f+ab5p9Rja4MWbfNgQugfW4HNbqmJG1K1s6Td28fqych/Px75/5LLlB2U3uRL0aEc8/8dzLEfFWVT0TEW+tHgMAABy8a4uoqvpFRPzliadfiIjXVp+/FhHf7J0WAADAMm26J+p2Vf1x9fmfIuJ203wAAAAWbXJjiXr0psS1b0zMzDuZeZ6Z5xcXF1OHO1py7CHHPrLsIccecuwhxz6y7CHHHnLst2kR9efM/HRExOrjg3XfWFV3q+qsqs5OTk42HA459pBjH1n2kGMPOfaQYx9Z9pBjDzn2u7Y73xpvRMRLEfH91ccft81oB5bQ4WPf6EQzHzlyE0vtRmb9AodAB9Ie67JZ6mvYiGvvRGXmDyPiPyPinzPzfmZ+Jx4VT89l5u8j4l9XjwEAAA7etXeiqurFNV/6WvNcAAAAFm9yYwkAAIBjoogCAAAYsGljCY6cTZSwW87B/eN3Nh/Z0s2amtch5OtOFAAAwABFFAAAwABFFAAAwABFFAAAwABFFAAAwICj6853CN1AOBzWIwDA/nEnCgAAYIAiCgAAYIAiCgAAYIAiCgAAYEBuc2N7Zl5ExHurh7ci4uHWBt+OzmP6fFWdXPYFOQ455hwj+o5rbY4RH8lSjtc75jUpxx5y7OMa2cOa7CHHHtvJcVfdwTLzvKrOdjL4THZxTHLc3zG3YdvHJcf9HnNucuwhxz6ukT2syR5y7LGtY/J2PgAAgAGKKAAAgAG7LKLu7nDsuezimOS4v2Nuw7aPS477Pebc5NhDjn1cI3tYkz3k2GMrx7SzPVEAAAD7aNKdqMx8PjN/m5nvZObLXZMCAABYqo3vRGXmUxHxu4h4LiLuR8TbEfFiVf2mb3oAAADL8vSEv/tsRLxTVe9GRGTmjyLihYhYW0TdunWrTk9PJwx5PO7duxcPHz7My74mx5uTY4+rcoyQ5Qhrsocce8ixh2tkH2uyhxx7XJXjlCLqMxHxh8ce34+I/3bVXzg9PY3z8/MJQx6Ps7P17e3leHNy7HFVjhGyHGFN9pBjDzn2cI3sY032kGOPq3KcvTtfZt7JzPPMPL+4uJh7uIMlxx5y7CPLHnLsIccecuwjyx5y7CHHflOKqPcj4nOPPf7s6rmPqKq7VXVWVWcnJycThjtucuwhxz6y7CHHHnLsIcc+suwhxx5y7DeliHo7Ip7JzC9k5scj4tsR8UbPtAAAAJZp4z1RVfVhZn4vIn4aEU9FxCtV9eu2mQEAACzQlMYSUVVvRsSbTXMBAABYvNkbSwAAABwSRRQAAMAARRQAAMCASXui9lHm2v9Q/EaqqmkmAADAPnInCgAAYIAiCgAAYIAiCgAAYIAiCgAAYMDRNZZgGS5r8KFpB93maiQz8nOta66ybi1ZN49oBgWH6RDObXeiAAAABiiiAAAABiiiAAAABiiiAAAABiiiAAAABujOx6ymdl+Bm5pjrR1jF75tnrOHktlNuR6uN1c2OsGyDYfQaW7bph7zEs5td6IAAAAGKKIAAAAGKKIAAAAGTNoTlZn3IuKDiPhrRHxYVWcdkwIAAFiqjsYSX62qhw0/p5UNvMt2jJso56DxwbiRHA7lOrLUdbKEjcFs367Pq3XjH+Pa00hm3K7XL8vh7XwAAAADphZRFRE/y8xfZuadjgkBAAAs2dQi6stV9aWI+EZEfDczv/LkN2Tmncw8z8zzi4uLicMdLzn2kGMfWfaQYw859pBjH1n2kGMPOfabVERV1furjw8i4vWIePaS77lbVWdVdXZycjJluKMmxx5y7CPLHnLsIccecuwjyx5y7CHHfhsXUZn5icz8p799HhFfj4hfdU0MAABgiaZ057sdEa+vupQ8HRE/qKqftMyqwbouMFO7qugmtZ5s1purG9Sh57vNTnyHYKld+FjP72HcMXbYHOE60GOutSPzw7FxEVVV70bEFxvnAgAAsHhanAMAAAxQRAEAAAxQRAEAAAyY0lhiL03d0HeMm1QvI4cemnH0uWluHWt3Cb8jm8eXzTXycM61JXMdmI8GJsswV2OuDu5EAQAADFBEAQAADFBEAQAADFBEAQAADDi6xhLMZwmb/JZKNvObY2Pvvv3eljrfJW8M3qZjO17mN7X5gTU5H9mO27cGHe5EAQAADFBEAQAADFBEAQAADFBEAQAADFBEAQAADDiI7nxzdJzZtw4hc9FVi23Y5vl2KGt3qcehAxhz8/rMLrnGbdeSs3UnCgAAYIAiCgAAYIAiCgAAYMC1RVRmvpKZDzLzV48996nM/Hlm/n718ZPzThMAAGAZbnIn6tWIeP6J516OiLeq6pmIeGv1eFEyc9KfdarqH/4cipEcGDO6zuhxyOfrUh1b5uvO7WPL4TKXZTCawxzXzY55LZk12cPr9rwOIdtri6iq+kVE/OWJp1+IiNdWn78WEd/snRYAAMAybbon6nZV/XH1+Z8i4nbTfAAAABZtcmOJenQ/eO094cy8k5nnmXl+cXExdbijJccecuwjyx5y7CHHHnLsI8secuwhx36bFlF/zsxPR0SsPj5Y941Vdbeqzqrq7OTkZMPhkGMPOfaRZQ859pBjDzn2kWUPOfaQY79Ni6g3IuKl1ecvRcSPe6YDAACwbE9f9w2Z+cOI+JeIuJWZ9yPi3yLi+xHxH5n5nYh4LyK+Neckr3NZh5mpXT50rXlEDj3kOD8Zw36Y4zV7ZKxDd4zHvE3ync++ZXttEVVVL6750tea5wIAALB4kxtLAAAAHBNFFAAAwABFFAAAwIBr90Ttq33bnLYEMmNXrD32lbXbQ47s0mWNTazJeR1Cvu5EAQAADFBEAQAADFBEAQAADFBEAQAADFBEAQAADDjY7nwAAHCdQ+gUx/a5EwUAADBAEQUAADBAEQUAADBAEQUAADAgt7mZLjMvIuK91cNbEfFwa4NvR+cxfb6qTi77ghyHHHOOEX3HtTbHiI9kKcfrHfOalGMPOfZxjexhTfaQY4/t5LirjiSZeV5VZzsZfCa7OCY57u+Y27Dt45Ljfo85Nzn2kGMf18ge1mQPOfbY1jF5Ox8AAMAARRQAAMCAXRZRd3c49lx2cUxy3N8xt2HbxyXH/R5zbnLsIcc+rpE9rMkecuyxlWPa2Z4oAACAfTTpTlRmPp+Zv83MdzLz5a5JAQAALNXGd6Iy86mI+F1EPBcR9yPi7Yh4sap+s+7v3Lp1q05PTzca79jcu3cvHj58mJd9TY43J8ceV+UYIcsR1mQPOfaQYw/XyD7WZA859rgqx6cn/NxnI+Kdqno3IiIzfxQRL0TE2iLq9PQ0zs/PJwx5PM7O1ndmlOPNybHHVTlGyHKENdlDjj3k2MM1so812UOOPa7Kccrb+T4TEX947PH91XMAAAAHa/bufJl5JzPPM/P84uJi7uEOlhx7yLGPLHvIsYcce8ixjyx7yLGHHPtNKaLej4jPPfb4s6vnPqKq7lbVWVWdnZycTBjuuMmxhxz7yLKHHHvIsYcc+8iyhxx7yLHflCLq7Yh4JjO/kJkfj4hvR8QbPdMCAABYpo0bS1TVh5n5vYj4aUQ8FRGvVNWv22YGAACwQFO680VVvRkRbzbNBQAAYPFmbywBAABwSBRRAAAAAya9ne+QZa79j8f/QVXNOBNgG5zzAMBNuRMFAAAwQBEFAAAwQBEFAAAwQBEFAAAwQGOJsKGc/XXZ2j3WNTpyHgP7a65z/VivnYyZuv6ss8PhThQAAMAARRQAAMAARRQAAMAARRQAAMAARRQAAMCAo+vOd9OuKrqnsCQ6z11vjnP2UHLf9XG4nnKdubrk7nrtd9m34zi2c/7YjpdH3IkCAAAYoIgCAAAYoIgCAAAYoIgCAAAYMKmxRGbei4gPIuKvEfFhVZ11TAqO2VwbrLnavm3cvsy6Y9B0Y/t2nc+hXBvmOI5DyabD1Cx2vc7ndMjHRo+O7nxfraqHDT8HAABg8bydDwAAYMDUIqoi4meZ+cvMvHPZN2Tmncw8z8zzi4uLicMdLzn2kGMfWfaQYw859pBjH1n2kGMPOfabWkR9uaq+FBHfiIjvZuZXnvyGqrpbVWdVdXZycjJxuOMlxx5y7CPLHnLsIccecuwjyx5y7CHHfpOKqKp6f/XxQUS8HhHPdkwKAABgqTZuLJGZn4iIj1XVB6vPvx4R/942s4mmdlXp6MqiAxBdrKXNTD2P9y33Jcx3CXPYJt00l2GbnSl3Yd1xWH/L4PdwnKZ057sdEa+vFs7TEfGDqvpJy6wAAAAWauMiqqrejYgvNs4FAABg8bQ4BwAAGKCIAgAAGDBlT9RidDSBgF2wdvscWxOJbbJO1xtZN5flaN1d7aZr71hz3OZxH9t1YGq2x5bXMXInCgAAYIAiCgAAYIAiCgAAYIAiCgAAYMBBNJbYtWPd0Mo8rKfNXJabjb3zsU7ptG49OYeXzXXgEev0OLkTBQAAMEARBQAAMEARBQAAMEARBQAAMEARBQAAMEB3vkE60bCJdZ17rCeW5rK1emzrtON8leMY3c2Ww++ih/P98LkTBQAAMEARBQAAMEARBQAAMODaIiozX8nMB5n5q8ee+1Rm/jwzf7/6+Ml5pwkAALAMN7kT9WpEPP/Ecy9HxFtV9UxEvLV6vLeq6sZ/4DqZ+Q9/+LvL8pnrz2Wc21ynY41YYz1umuPINYDNWNOPWGf8zbVFVFX9IiL+8sTTL0TEa6vPX4uIb/ZOCwAAYJk23RN1u6r+uPr8TxFxu2k+AAAAiza5sUQ9up+79p5uZt7JzPPMPL+4uJg63NGSYw859pFlDzn2kGMPOfaRZQ859pBjv02LqD9n5qcjIlYfH6z7xqq6W1VnVXV2cnKy4XDIsYcc+8iyhxx7yLGHHPvIsocce8ix36ZF1BsR8dLq85ci4sc90wEAAFi2p6/7hsz8YUT8S0Tcysz7EfFvEfH9iPiPzPxORLwXEd+ac5LXOdYOMewPa3QzcoPjdtPOZ64Vm9FZDjZ3bRFVVS+u+dLXmucCAACweJMbSwAAABwTRRQAAMAARRQAAMCAa/dEAWNscL6afIAnuS70mdoswu/iavLhb9yJAgAAGKCIAgAAGKCIAgAAGKCIAgAAGKCIAgAAGKA7HwD/m85TsN+cw7Ad7kQBAAAMUEQBAAAMUEQBAAAMUEQBAAAMyG1uQMzMi4h4b/XwVkQ83Nrg29F5TJ+vqpPLviDHIcecY0Tfca3NMeIjWcrxese8JuXYQ459XCN7WJM95NhjOznuqotLZp5X1dlOBp/JLo5Jjvs75jZs+7jkuN9jzk2OPeTYxzWyhzXZQ449tnVM3s4HAAAwQBEFAAAwYJdF1N0djj2XXRyTHPd3zG3Y9nHJcb/HnJsce8ixj2tkD2uyhxx7bOWYdrYnCgAAYB9NuhOVmc9n5m8z853MfLlrUgAAAEu18Z2ozHwqIn4XEc9FxP2IeDsiXqyq36z7O7du3arT09ONxjs29+7di4cPH+ZlX5Pjzcmxx1U5RshyhDXZQ4495NjDNbKPNdlDjj2uyvHpCT/32Yh4p6rejYjIzB9FxAsRsbaIOj09jfPz8wlDHo+zs/WdGeV4c3LscVWOEbIcYU32kGMPOfZwjexjTfaQY4+rcpzydr7PRMQfHnt8f/UcAADAwZq9O19m3snM88w8v7i4mHu4gyXHHnLsI8secuwhxx5y7CPLHnLsIcd+U4qo9yPic489/uzquY+oqrtVdVZVZycnJxOGO25y7CHHPrLsIccecuwhxz6y7CHHHnLsN6WIejsinsnML2TmxyPi2xHxRs+0AAAAlmnjxhJV9WFmfi8ifhoRT0XEK1X167aZAQAALNCU7nxRVW9GxJtNcwEAAFi82RtLAAAAHBJFFAAAwABFFAAAwABFFAAAwABFFAAAwABFFAAAwABFFAAAwIBJ/08UsB2ZeePvraoZZwIAXGfkdXudQ349n5rPErJxJwoAAGCAIgoAAGCAIgoAAGCAIgoAAGCAIgoAAGDAQXTn6+iAMsUSOoSss+ts1llyZhw2HZPgeFx2vjt//25qPuuup8eW8SF0mpvL1DWy1H/HRrgTBQAAMEQRBQAAMEARBQAAMEARBQAAMGBSY4nMvBcRH0TEXyPiw6o665jUqDk25B3KZsldz3ddjjb79pDZ1awz2A9L3jx+KFwPl+HYMp96vEvOq6M731er6mHDzwEAAFg8b+cDAAAYMLWIqoj4WWb+MjPvdEwIAABgyaYWUV+uqi9FxDci4ruZ+ZUnvyEz72TmeWaeX1xcTBzueMmxhxz7yLKHHHvIsYcc+8iyhxx7yLHfpCKqqt5ffXwQEa9HxLOXfM/dqjqrqrOTk5Mpwx01OfaQYx9Z9pBjDzn2kGMfWfaQYw859tu4sURmfiIiPlZVH6w+/3pE/HvbzOBI6VK1GZ2nYD+MXONueg4f63VzjixHxjrGa+yxrrVdWXK37Cnd+W5HxOurg3s6In5QVT9pmRUAAMBCbVxEVdW7EfHFxrkAAAAsnhbnAAAAAxRRAAAAA6bsiQImuukG1SVsoFw6GbF0NuY/ss1jlu+4JW/k3yfyOnzuRAEAAAxQRAEAAAxQRAEAAAxQRAEAAAzQWCJs9oVD5dyGw3PThjzHYI4mEq6R7Mq+rUd3ogAAAAYoogAAAAYoogAAAAYoogAAAAYoogAAAAbozsdOLLnbyhxGukkdWzZd9q2rD8fHepyPbK/nGjlGF0iu404UAADAAEUUAADAAEUUAADAAEUUAADAgGsbS2TmKxHx3yPiQVX919Vzn4qI/xERpxFxLyK+VVX/a75psg9swmSXbJDuMfU89nugk9eVcesyc27OR7bjDuHcvsmdqFcj4vknnns5It6qqmci4q3VYwAAgIN3bRFVVb+IiL888fQLEfHa6vPXIuKbvdMCAABYpk33RN2uqj+uPv9TRNxe942ZeSczzzPz/OLiYsPhkGMPOfaRZQ859pBjDzn2kWUPOfaQY7/JjSXq0RtB174ZtKruVtVZVZ2dnJxMHe5oybGHHPvIsocce8ixhxz7yLKHHHvIsd+mRdSfM/PTERGrjw/6pgQAALBc13bnW+ONiHgpIr6/+vjjthnN7LJuILqq0Gmk44y1x9JYkyydNUq3Q+gUdwj27dy+9k5UZv4wIv4zIv45M+9n5nfiUfH0XGb+PiL+dfUYAADg4F17J6qqXlzzpa81zwUAAGDxJjeWAAAAOCaKKAAAgAGbNpbgyN10E+a+bRLscIzHDDAH19NxMhsns+07hMzdiQIAABigiAIAABigiAIAABigiAIAABigiAIAABhwdN35DqEbyBLIEQCAY+VOFAAAwABFFAAAwABFFAAAwABFFAAAwIDcZoOAzLyIiPdWD29FxMOtDb4dncf0+ao6uewLchxyzDlG9B3X2hwjPpKlHK93zGtSjj3k2Mc1soc12UOOPbaT4666rGXmeVWd7WTwmezimOS4v2Nuw7aPS477Pebc5NhDjn1cI3tYkz3k2GNbx+TtfAAAAAMUUQAAAAN2WUTd3eHYc9nFMclxf8fchm0flxz3e8y5ybGHHPu4RvawJnvIscdWjmlne6IAAAD20aQ7UZn5fGb+NjPfycyXuyYFAACwVBvficrMpyLidxHxXETcj4i3I+LFqvrNur9z69atOj093Wi8Y3Pv3r14+PBhXvY1Od6cHHtclWOELEdYkz3k2EOOPVwj+1iTPeTY46ocn57wc5+NiHeq6t2IiMz8UUS8EBFri6jT09M4Pz+fMOTxODtb35lRjjcnxx5X5RghyxHWZA859pBjD9fIPtZkDzn2uCrHKW/n+0xE/OGxx/dXzwEAABys2bvzZeadzDzPzPOLi4u5hztYcuwhxz6y7CHHHnLsIcc+suwhxx5y7DeliHo/Ij732OPPrp77iKq6W1VnVXV2cnIyYbjjJscecuwjyx5y7CHHHnLsI8secuwhx35Tiqi3I+KZzPxCZn48Ir4dEW/0TAsAAGCZNm4sUVUfZub3IuKnEfFURLxSVb9umxkAAMACTenOF1X1ZkS82TQXAACAxZu9sQQAAMAhUUQBAAAMmPR2PgCOU+al/4H7papqxpkAwPa5EwUAADBAEQUAADBAEQUAADBAEQUAADBAYwnajGw0v4zN5+Muy/xYc5y6/kYcW8bWGRwmr9vrbfM1ZcQhZ75v3IkCAAAYoIgCAAAYoIgCAAAYoIgCAAAYoIgCAAAYcBDd+XbdQeWQO6XsOlse8Xvoc8jn61xkxpLMdT089HV+09zW5eB1aD6HvvYOlTtRAAAAAxRRAAAAAxRRAAAAAxRRAAAAAyY1lsjMexHxQUT8NSI+rKqzjkl1mGuT3iFvrLzs2EZyHMnGJsoex5ijdQb7berr6NTz+pBfxyPWH5/r4Xxke5w6uvN9taoeNvwcAACAxfN2PgAAgAFTi6iKiJ9l5i8z807HhAAAAJZsahH15ar6UkR8IyK+m5lfefIbMvNOZp5n5vnFxcXE4Y6XHHvIsY8se8ixhxx7yLGPLHvIsYcc+00qoqrq/dXHBxHxekQ8e8n33K2qs6o6Ozk5mTLcUZNjDzn2kWUPOfaQYw859pFlDzn2kGO/jRtLZOYnIuJjVfXB6vOvR8S/t81swBxdUQ69e89N7bqL0jGa2iWRv5Ml7M4SOmm6Bow7xn//HOMxM92U7ny3I+L11cJ7OiJ+UFU/aZkVAADAQm1cRFXVuxHxxca5AAAALJ4W5wAAAAMUUQAAAAOm7Ik6Soe8IfWyY7PZcl42PY+bms+6NS33R6xJ5qYZ1G5oFDWfJTRRYfvciQIAABigiAIAABigiAIAABigiAIAABigsUTYkHqVdRsgb5qZTfyPWGPLMbKmD3mdOjeZ2xLW0hLmsG0jx+y1afuO7bXmkLkTBQAAMEARBQAAMEARBQAAMEARBQAAMEARBQAAMEB3vjV0Shl3WWY6/1zNOgMYo7vZZm76enyMWc5xzP79c/jciQIAABigiAIAABigiAIAABigiAIAABhwbWOJzHwlIv57RDyoqv+6eu5TEfE/IuI0Iu5FxLeq6n/NN80+NqRu17psj+33cMjHtm9s9oX9cWyvFdt2bFmuu/5PzcHrynG6yZ2oVyPi+Seeezki3qqqZyLirdVjAACAg3dtEVVVv4iIvzzx9AsR8drq89ci4pu90wIAAFimTfdE3a6qP64+/1NE3G6aDwAAwKJNbixRj95IuvbNpJl5JzPPM/P84uJi6nBHS4495NhHlj3k2EOOPeTYR5Y95NhDjv02LaL+nJmfjohYfXyw7hur6m5VnVXV2cnJyYbDIccecuwjyx5y7CHHHnLsI8secuwhx37Xdudb442IeCkivr/6+OO2GTXRKWUZ5uqEs2S6SfXZ5nl8bL+jYztels1rdh9ZjvNawyauvROVmT+MiP+MiH/OzPuZ+Z14VDw9l5m/j4h/XT0GAAA4eNfeiaqqF9d86WvNcwEAAFi8yY0lAAAAjokiCgAAYMCmjSX2lg1989FQ4ZFjPOa5yBKOm2tAH1nKgF7uRAEAAAxQRAEAAAxQRAEAAAxQRAEAAAxQRAEAAAw42O58OrDMS74AdPGa0keWsB3uRAEAAAxQRAEAAAxQRAEAAAxQRAEAAAzIbW5AzMyLiHhv9fBWRDzc2uDb0XlMn6+qk8u+IMchx5xjRN9xrc0x4iNZyvF6x7wm5dhDjn1cI3tYkz3k2GM7Oe6qi0tmnlfV2U4Gn8kujkmO+zvmNmz7uOS432POTY495NjHNbKHNdlDjj22dUzezgcAADBAEQUAADBgl0XU3R2OPZddHJMc93fMbdj2cclxv8ecmxx7yLGPa2QPa7KHHHts5Zh2ticKAABgH026E5WZz2fmbzPzncx8uWtSAAAAS7XxnajMfCoifhcRz0XE/Yh4OyJerKrf9E0PAABgWZ6e8HefjYh3qurdiIjM/FFEvBARa4uoW7du1enp6YQhj8e9e/fi4cOHednX5HhzcuxxVY4RshxhTfaQYw859nCN7GNN9pBjj6tynFJEfSYi/vDY4/sR8d+u+gunp6dxfn4+YcjjcXa2vr29HG9Ojj2uyjFCliOsyR5y7CHHHq6RfazJHnLscVWOs3fny8w7mXmemecXFxdzD3ew5NhDjn1k2UOOPeTYQ459ZNlDjj3k2G9KEfV+RHzuscefXT33EVV1t6rOqurs5ORkwnDHTY495NhHlj3k2EOOPeTYR5Y95NhDjv2mFFFvR8QzmfmFzPx4RHw7It7omRYAAMAybbwnqqo+zMzvRcRPI+KpiHilqn7dNjMAAIAFmtJYIqrqzYh4s2kuAAAAizd7YwkAAIBDoogCAAAYoIgCAAAYMGlPFABwM5n/+J/eV9UOZgLAVO5EAQAADFBEAQAADFBEAQAADFBEAQAADDi6xhKXbeydysbg9dblLbP1pq5R2f7dSJZyW0+O4zSRAFhvjn+PrzPXtdedKAAAgAGKKAAAgAGKKAAAgAGKKAAAgAGKKAAAgAEH251vrm5S2+wmwuGb2sHLeqSbrnJjdCBdtkNfz7vunHko61/n5u2bms8Szm13ogAAAAYoogAAAAYoogAAAAZM2hOVmfci4oOI+GtEfFhVZx2TAgAAWKqOxhJfraqHDT+n1Rwb1o7RoWwa3TU5ciyWsNl3Lod8bLs29RrpNfvvrMlxc/yb0et+jyXn6O18AAAAA6YWURURP8vMX2bmnY4JAQAALNnUIurLVfWliPhGRHw3M7/y5Ddk5p3MPM/M84uLi4nDHS859pBjH1n2kGMPOfaQYx9Z9pBjDzn2m1REVdX7q48PIuL1iHj2ku+5W1VnVXV2cnIyZbijJscecuwjyx5y7CHHHnLsI8secuwhx34bF1GZ+YnM/Ke/fR4RX4+IX3VNDAAAYImmdOe7HRGvr7pmPB0RP6iqn7TMastGuvosoRvIUo3kqMvVGJ2n+lhnV7ssH9fI7Tvka+Q2r2eHntk2j++Q1+RUU6+bPLJva2zjIqqq3o2ILzbOBQAAYPG0OAcAABigiAIAABigiAIAABgwpbEER2zqhsklbxTcJjn2sYl3u6y9HlMb8kQc9u/CeX21fduIf6j8HnqMNOhYQr7uRAEAAAxQRAEAAAxQRAEAAAxQRAEAAAzQWCL8T9NXGdm4Z2PleutyuOk6kyPb4jxeb2oOU7P1unS8nIPbN/V8W3JDBHq4EwUAADBAEQUAADBAEQUAADBAEQUAADBAEQUAADBAdz5YGJ172Aado+Z1085eIx3ADuV3s+suh1zNteFqcuBv3IkCAAAYoIgCAAAYoIgCAAAYcG0RlZmvZOaDzPzVY899KjN/npm/X3385LzTBAAAWIab3Il6NSKef+K5lyPirap6JiLeWj3emcxs/8PVLsusqv7hD49YZ+yStbd/XE9hty47B52H87rpvy2X8nu4toiqql9ExF+eePqFiHht9flrEfHN3mkBAAAs06Z7om5X1R9Xn/8pIm43zQcAAGDRJjeWqEf31NbeV8vMO5l5npnnFxcXU4c7WnLsIcc+suwhxx5y7CHHPrLsIccecuy3aRH158z8dETE6uODdd9YVXer6qyqzk5OTjYcDjn2kGMfWfaQYw859pBjH1n2kGMPOfbbtIh6IyJeWn3+UkT8uGc6AAAAy/b0dd+QmT+MiH+JiFuZeT8i/i0ivh8R/5GZ34mI9yLiW3NO8jqXdemY2n1qKZ0/OGzW2bzky65Ye/ORLWzPNru57tu5fW0RVVUvrvnS15rnAgAAsHiTG0sAAAAcE0UUAADAAEUUAADAgGv3RO2rfductm/ku942N2Hyd9bkerIBLnPZ65XrBY+zHtZzJwoAAGCAIgoAAGCAIgoAAGCAIgoAAGCAIgoAAGDAwXbng13RyQaAfeD1CjbnThQAAMAARRQAAMAARRQAAMAARRQAAMCA3Oamwsy8iIj3Vg9vRcTDrQ2+HZ3H9PmqOrnsC3Iccsw5RvQd19ocIz6SpRyvd8xrUo495NjHNbKHNdlDjj22k+OuOrNk5nlVne1k8Jns4pjkuL9jbsO2j0uO+z3m3OTYQ459XCN7WJM95NhjW8fk7XwAAAADFFEAAAADdllE3d3h2HPZxTHJcX/H3IZtH5cc93vMucmxhxz7uEb2sCZ7yLHHVo5pZ3uiAAAA9tGkO1GZ+Xxm/jYz38nMl7smBQAAsFQb34nKzKci4ncR8VxE3I+ItyPixar6zbq/c+vWrTo9Pd1ovGNz7969ePjwYV72NTnenBx7XJVjhCxHWJM95NhDjj1cI/tYkz3k2OOqHJ+e8HOfjYh3qurdiIjM/FFEvBARa4uo09PTOD8/nzDk8Tg7W9+ZUY43J8ceV+UYIcsR1mQPOfaQYw/XyD7WZA859rgqxylv5/tMRPzhscf3V88BAAAcrNm782Xmncw8z8zzi4uLuYc7WHLsIcc+suwhxx5y7CHHPrLsIccecuw3pYh6PyI+99jjz66e+4iqultVZ1V1dnJyMmG44ybHHnLsI8secuwhxx5y7CPLHnLsIcd+U4qotyPimcz8QmZ+PCK+HRFv9EwLAABgmTZuLFFVH2bm9yLipxHxVES8UlW/bpsZAADAAk3pzhdV9WZEvNk0FwAAgMWbvbEEAADAIVFEAQAADJj0dr5Dlrn2Px6/sapqmMn+mJrZseUFHJfLrpGuewD7yZ0oAACAAYooAACAAYooAACAAYooAACAARpLRE8TiWNz08zWbZqW+SNz5GCjOuyeJhLzmev1w++Hx1lnXMedKAAAgAGKKAAAgAGKKAAAgAGKKAAAgAGKKAAAgAFH151PV7geusssdy2tm5ffGTcxtfPmsVnqdeCQjaw9v59xx9iVbuoxL/nY5rDU82rbvwd3ogAAAAYoogAAAAYoogAAAAYoogAAAAZMaiyRmfci4oOI+GtEfFhVZx2TWoJ1m9OWupmO5dj1BlNr9O8uy2LXv5+lk08POW7XyHXvWH83msbQZZtrZMn/punozvfVqnrY8HMAAAAWz9v5AAAABkwtoioifpaZv8zMO5d9Q2beyczzzDy/uLiYONzxkmMPOfaRZQ859pBjDzn2kWUPOfaQY7+pRdSXq+pLEfGNiPhuZn7lyW+oqrtVdVZVZycnJxOHO15y7CHHPrLsIccecuwhxz6y7CHHHnLsN6mIqqr3Vx8fRMTrEfFsx6QAAACWauPGEpn5iYj4WFV9sPr86xHx720zm0innmVY93uQOdtgndFp112ivK49Ioered0dd1k2I+tMJ9jtW0K+U7rz3Y6I11cL5+mI+EFV/aRlVgAAAAu1cRFVVe9GxBcb5wIAALB4WpwDAAAMUEQBAAAMmLInajHm2GRq4+q4XW+63rZ1v/djywEed2wbrKcem+vF1W6azyGvMXZj6mu8Bh+Hz50oAACAAYooAACAAYooAACAAYooAACAAQfRWOIyNu7Ny2bf5ZI5u3Rs629qY4iRvI6xCcVl+RxjDiNGGiIc2/k6YmqDsXV/3+9hvX3Lxp0oAACAAYooAACAAYooAACAAYooAACAAYooAACAAXvVnW+ujjxTO7AcCh2Pemxzjfidwf6YqxPfIb8uwb4a6ZLIfnInCgAAYIAiCgAAYIAiCgAAYMC1RVRmvpKZDzLzV48996nM/Hlm/n718ZPzThMAAGAZbnIn6tWIeP6J516OiLeq6pmIeGv1eFEy88Z/LlNVl/45BCM57NNYAHOa+pow8rq07jXoUF+X1vH6wa5s83w7xn8rHcLxXltEVdUvIuIvTzz9QkS8tvr8tYj4Zu+0AAAAlmnTPVG3q+qPq8//FBG3m+YDAACwaJMbS9Sje5tr729m5p3MPM/M84uLi6nDHS059pBjH1n2kGMPOfaQYx9Z9pBjDzn227SI+nNmfjoiYvXxwbpvrKq7VXVWVWcnJycbDocce8ixjyx7yLGHHHvIsY8se8ixhxz7bVpEvRERL60+fykiftwzHQAAgGV7+rpvyMwfRsS/RMStzLwfEf8WEd+PiP/IzO9ExHsR8a05J/k367qiTO3ocejdjW5KDsu1j11r4Bi4bo7zmj2vq7oOMw9ruse+5XBtEVVVL6750tea5wIAALB4kxtLAAAAHBNFFAAAwABFFAAAwIBr90Ttg33biLZr8loOm1GBY+O61eey1xD5zku+/I07UQAAAAMUUQAAAAMUUQAAAAMUUQAAAAMUUQAAAAMOojsf7CtdfgDYlNcQ9tUhrF13ogAAAAYoogAAAAYoogAAAAYoogAAAAbkNjd2ZeZFRLy3engrIh5ubfDt6Dymz1fVyWVfkOOQY84xou+41uYY8ZEs5Xi9Y16Tcuwhxz6ukT2syR5y7LGdHHfVHSMzz6vqbCeDz2QXxyTH/R1zG7Z9XHLc7zHnJscecuzjGtnDmuwhxx7bOiZv5wMAABigiAIAABiwyyLq7g7HnssujkmO+zvmNmz7uOS432POTY495NjHNbKHNdlDjj22ckw72xMFAACwjybdicrM5zPzt5n5Tma+3DUpAACApdr4TlRmPhURv4uI5yLifkS8HREvVtVv1v2dW7du1enp6UbjHZt79+7Fw4cP87KvyfHm5NjjqhwjZDnCmuwhxx5y7OEa2cea7CHHHlfl+PSEn/tsRLxTVe9GRGTmjyLihYhYW0Sdnp7G+fn5hCGPx9nZ+s6Mcrw5Ofa4KscIWY6wJnvIsYcce7hG9rEme8ixx1U5Tnk732ci4g+PPb6/eg4AAOBgzd6dLzPvZOZ5Zp5fXFzMPdzBkmMPOfaRZQ859pBjDzn2kWUPOfaQY78pRdT7EfG5xx5/dvXcR1TV3ao6q6qzk5OTCcMdNzn2kGMfWfaQYw859pBjH1n2kGMPOfabUkS9HRHPZOYXMvPjEfHtiHijZ1oAAADLtHFjiar6MDO/FxE/jYinIuKVqvp128wAAAAWaEp3vqiqNyPizaa5AAAALN7sjSUAAAAOiSIKAABgwKS38+2jzH/8T4eragczAQAA9pE7UQAAAAMUUQAAAAMUUQAAAAMUUQAAAAOOrrHEZTSbYEmsx81cltsIGQPA7oy8ji/hNdudKAAAgAGKKAAAgAGKKAAAgAGKKAAAgAGKKAAAgAG68zGrqR3TIpbRgYXDYT09su7clE+Pjmvfk/xueNIc62ydY1t/su0xtePeNn8Po9yJAgAAGKCIAgAAGKCIAgAAGKCIAgAAGDCpsURm3ouIDyLirxHxYVWddUyKPkvekAed9m1D6q5N3cisMcUj1lgPm/j7zHVuH7KpxzyS+WVjHcr19LLj2LdjGNHRne+rVfWw4ecAAAAsnrfzAQAADJhaRFVE/Cwzf5mZdzomBAAAsGRTi6gvV9WXIuIbEfHdzPzKk9+QmXcy8zwzzy8uLiYOd7zk2EOOfWTZQ4495NhDjn1k2UOOPeTYb1IRVVXvrz4+iIjXI+LZS77nblWdVdXZycnJlOGOmhx7yLGPLHvIsYcce8ixjyx7yLGHHPtt3FgiMz8RER+rqg9Wn389Iv69bWYsmu4/wLEZuW7pFrne1BznGmsfHXLns23adY67Hr/LoRzHTU3pznc7Il5fXaCejogfVNVPWmYFAACwUBsXUVX1bkR8sXEuAAAAi6fFOQAAwABFFAAAwIApe6LYA7ve5Hfom3rh0Fx2zl52Hdn1tWXppl77DjnfOY5tXd6HnCPL5t8/4276+rMU7kQBAAAMUEQBAAAMUEQBAAAMUEQBAAAM0FiCxVjy5kGAqdcoG80B/9Z55KbXwyU3jXEnCgAAYIAiCgAAYIAiCgAAYIAiCgAAYIAiCgAAYIDufAD8b0voeHRM5D3usm5dcmSXdN4cd9NzdsnZuhMFAAAwQBEFAAAwQBEFAAAwQBEFAAAw4NoiKjNfycwHmfmrx577VGb+PDN/v/r4yXmnyT7IzH/4s05V/cMfYHsuO1+XvIF338iWfXDZOr3s9dlr9Dg5Hr6b3Il6NSKef+K5lyPirap6JiLeWj0GAAA4eNcWUVX1i4j4yxNPvxARr60+fy0ivtk7LQAAgGXadE/U7ar64+rzP0XE7ab5AAAALNrkxhL16A2ea9/kmZl3MvM8M88vLi6mDne05NhDjn1k2UOOPeTYQ459ZNlDjj3k2G/TIurPmfnpiIjVxwfrvrGq7lbVWVWdnZycbDgccuwhxz6y7CHHHnLsIcc+suwhxx5y7LdpEfVGRLy0+vyliPhxz3Q4NLrTADziWjjupt3jYFt03pzPvnWLvEmL8x9GxH9GxD9n5v3M/E5EfD8insvM30fEv64eAwAAHLynr/uGqnpxzZe+1jwXAACAxZvcWAIAAOCYKKIAAAAGXPt2vkOzlM1osI412keW68mmx7pN5fLtIUf2gXV6nNyJAgAAGKCIAgAAGKCIAgAAGKCIAgAAGKCIAgAAGHB03fkAoIuuXOwz63eczOazb9m6EwUAADBAEQUAADBAEQUAADBAEQUAADAgt7mJKzMvIuK91cNbEfFwa4NvR+cxfb6qTi77ghyHHHOOEX3HtTbHiI9kKcfrHfOalGMPOfZxjexhTfaQY4/t5LirThiZeV5VZzsZfCa7OCY57u+Y27Dt45Ljfo85Nzn2kGMf18ge1mQPOfbY1jF5Ox8AAMAARRQAAMCAXRZRd3c49lx2cUxy3N8xt2HbxyXH/R5zbnLsIcc+rpE9rMkecuyxlWPa2Z4oAACAfTTpTlRmPp+Zv83MdzLz5a5JAQAALNXGd6Iy86mI+F1EPBcR9yPi7Yh4sap+s+7v3Lp1q05PTzca79jcu3cvHj58mJd9TY43J8ceV+UYIcsR1mQPOfaQYw/XyD7WZA859rgqx6cn/NxnI+Kdqno3IiIzfxQRL0TE2iLq9PQ0zs/PJwx5PM7O1ndmlOPNybHHVTlGyHKENdlDjj3k2MM1so812UOOPa7Kccrb+T4TEX947PH91XMAAAAHa/bufJl5JzPPM/P84uJi7uEOlhx7yLGPLHvIsYcce8ixjyx7yLGHHPtNKaLej4jPPfb4s6vnPqKq7lbVWVWdnZycTBjuuMmxhxz7yLKHHHvIsYcc+8iyhxx7yLHflCLq7Yh4JjO/kJkfj4hvR8QbPdMCAABYpo0bS1TVh5n5vYj4aUQ8FRGvVNWv22YGAACwQFO680VVvRkRbzbNBQAAYPFmbywBAABwSBRRAAAAAxRRAAAAAxRRAAAAAxRRAAAAAxRRAAAAAxRRAAAAAxRRAAAAAyb9Z7uHIjP/4bmq2sFMODaXrb2prF12aeqatn5he5yv47xu9ziEf3u7EwUAADBAEQUAADBAEQUAADBAEQUAADBAEQUAADBAdz6uNNKFZq6uKofQwWVdjlOPY44uQfCkJVwH9o0OXvM5hNeEXbjpmpTlI163uY47UQAAAAMUUQAAAAMUUQAAAAMm7YnKzHsR8UFE/DUiPqyqs45JAQAALFVHY4mvVtXDhp/Dju168/hcmzi3bdebng8lx1G7zv1QyHHM1PPNJnO67fq1fB/Ncd1zbo/bt9cfb+cDAAAYMLWIqoj4WWb+MjPvdEwIAABgyaYWUV+uqi9FxDci4ruZ+ZUnvyEz72TmeWaeX1xcTBzueMmxhxz7yLKHHHvIsYcc+8iyhxx7yLHfpCKqqt5ffXwQEa9HxLOXfM/dqjqrqrOTk5Mpwx01OfaQYx9Z9pBjDzn2kGMfWfaQYw859tu4iMrMT2TmP/3t84j4ekT8qmtiAAAASzSlO9/tiHh91Unj6Yj4QVX9pGVWLN7UDiqH0LVmCV3wLhvrELLdxJI7+OyTm64peV9t6nkoX7ZBJz925RD+/bJxEVVV70bEFxvnAgAAsHhanAMAAAxQRAEAAAxQRAEAAAyY0liCAzO1MYSN1MCxmXrd2reN1CzLHI0h1v3MQ24ws80mB4eS2Rz2LRt3ogAAAAYoogAAAAYoogAAAAYoogAAAAZoLMFGpm7C3LfNg7AP1p2DU5vG0OOQN+azG9tcP9bqGHmN27drpDtRAAAAAxRRAAAAAxRRAAAAAxRRAAAAAxRRAAAAA46uO5/OUz1umuOSu6ocgn3rZMO85vrdW1PAIZvaXdi/LefT0XV2Lu5EAQAADFBEAQAADFBEAQAADLi2iMrMVzLzQWb+6rHnPpWZP8/M368+fnLeaQIAACzDTe5EvRoRzz/x3MsR8VZVPRMRb60e762q+oc/jJPjfDLz0j8y53Hr1snIHwAu/zeN11ged20RVVW/iIi/PPH0CxHx2urz1yLim73TAgAAWKZN90Tdrqo/rj7/U0TcbpoPAADAok1uLFGP7m2uvb+ZmXcy8zwzzy8uLqYOd7Tk2EOOfWTZQ4495NhDjn1k2UOOPeTYb9Mi6s+Z+emIiNXHB+u+saruVtVZVZ2dnJxsOBxy7CHHPrLsIccecuwhxz6y7CHHHnLst2kR9UZEvLT6/KWI+HHPdAAAAJbt6eu+ITN/GBH/EhG3MvN+RPxbRHw/Iv4jM78TEe9FxLfmnCS7s65blw416zOYo8OZvLkJ62Q5dDqcj3Xexzpl6ZZ8vl9bRFXVi2u+9LXmuQAAACze5MYSAAAAx0QRBQAAMEARBQAAMODaPVGHZskb1Hbtsg2m8honM+Ayrg3s0k2bSBzjOp16zMeY2Rz2LUd3ogAAAAYoogAAAAYoogAAAAYoogAAAAYoogAAAAYcXXc+1tu3rigAS+V6ytJYk9DLnSgAAIABiigAAIABiigAAIABiigAAIABuc2Nhpl5ERHvrR7eioiHWxt8OzqP6fNVdXLZF+Q45JhzjOg7rrU5RnwkSzle75jXpBx7yLGPa2QPa7KHHHtsJ8dddWvJzPOqOtvJ4DPZxTHJcX/H3IZtH5cc93vMucmxhxz7uEb2sCZ7yLHHto7J2/kAAAAGKKIAAAAG7LKIurvDseeyi2OS4/6OuQ3bPi457veYc5NjDzn2cY3sYU32kGOPrRzTzvZEAQAA7KNJd6Iy8/nM/G1mvpOZL3dNCgAAYKk2vhOVmU9FxO8i4rmIuB8Rb0fEi1X1m3V/59atW3V6errReMfm3r178fDhw7zsa3K8OTn2uCrHCFmOsCZ7yLGHHHu4RvaxJnvIscdVOT494ec+GxHvVNW7ERGZ+aOIeCEi1hZRp6encX5+PmHI43F2tr4zoxxvTo49rsoxQpYjrMkecuwhxx6ukX2syR5y7HFVjlPezveZiPjDY4/vr577iMy8k5nnmXl+cXExYbjjJscecuwjyx5y7CHHHnLsI8secuwhx36zd+erqrtVdVZVZycna/8zb64hxx5y7CPLHnLsIccecuwjyx5y7CHHflOKqPcj4nOPPf7s6jkAAICDNaWIejsinsnML2TmxyPi2xHxRs+0AAAAlmnjxhJV9WFmfi8ifhoRT0XEK1X167aZAQAALNCU7nxRVW9GxJtNcwEAAFi82RtLAAAAHBJFFAAAwIBJb+cDpslc+x/c30hVNc3kMI3kK0sA2J3LXrOX/NrsThQAAMAARRQAAMAARRQAAMAARRQAAMAAjSWAgzC1ScchN6HYt826MOqQz18Oi+vxI1Nfs5fAnSgAAIABiigAAIABiigAAIABiigAAIABiigAAIABR9edb45uIIfSVWXXnVIOJccRNz3mXf9ulmSbXbjWjaW70nqyecRrTQ/Xvv3kOmDtHgN3ogAAAAYoogAAAAYoogAAAAZM2hOVmfci4oOI+GtEfFhVZx2TAgAAWKqOxhJfraqHDT9nY1M3741sdrxsrHXj79smyjnma2PlOJn93TabSIz8zKX+jrY5r6VmsG1Tr/9yHDf1NftYbbPZyb79+2eqXb9WsRvezgcAADBgahFVEfGzzPxlZt7pmBAAAMCSTS2ivlxVX4qIb0TEdzPzK09+Q2beyczzzDy/uLiYONzxkmMPOfaRZQ859pBjDzn2kWUPOfaQY79JRVRVvb/6+CAiXo+IZy/5nrtVdVZVZycnJ1OGO2py7CHHPrLsIccecuwhxz6y7CHHHnLst3ERlZmfyMx/+tvnEfH1iPhV18QAAACWaEp3vtsR8fqqI8nTEfGDqvpJy6zWWEL3k8t+7rp5Xfa8riyPHFsO2+wguY+WcG4fG13Llu3Y1rn1uBzHtvbmIsfDt3ERVVXvRsQXG+cCAACweFqcAwAADFBEAQAADFBEAQAADJjSWGJW+7TRfN34x7ZR9tiOdy67Xs9LIgs4bq4BfW6a5brXcr+LR/xbh79xJwoAAGCAIgoAAGCAIgoAAGCAIgoAAGDAYhtLXGapmxptMlxvqb8zuIl9O7fnOt/2LYdtGsn8shyP8Ro5x3qyRtmlYzyPcScKAABgiCIKAABggCIKAABggCIKAABggCIKAABgwF5159s3h9ytRScktmGObmYda/eQz+3LXHa8rgF0mtrlsOPnLtk2z7dDyWwq1ziu404UAADAAEUUAADAAEUUAADAgGuLqMx8JTMfZOavHnvuU5n588z8/erjJ+edJgAAwDLc5E7UqxHx/BPPvRwRb1XVMxHx1upxq6r6hz+ZeekfluGy3xmPXJbNunzWrfOb/jl028xn5PcGjzvGc3MON83x0M/Vdcc3xx+uJjP+5toiqqp+ERF/eeLpFyLitdXnr0XEN3unBQAAsEyb7om6XVV/XH3+p4i43TQfAACARZvcWKIe3cdcey8zM+9k5nlmnl9cXEwd7mjJsYcc+8iyhxx7yLGHHPvIsocce8ix36ZF1J8z89MREauPD9Z9Y1Xdraqzqjo7OTnZcDjk2EOOfWTZQ4495NhDjn1k2UOOPeTYb9Mi6o2IeGn1+UsR8eOe6QAAACzb09d9Q2b+MCL+JSJuZeb9iPi3iPh+RPxHZn4nIt6LiG/NOcm/WWoHlKXOi+W7bO0cYxevkXNoJB/nJktiPa53Vdc9mNtl68/a2759y/zaIqqqXlzzpa81zwUAAGDxJjeWAAAAOCaKKAAAgAGKKAAAgAHX7okCtmvfNlZum3yWwe+BTVk7LI01uX2HkLk7UQAAAAMUUQAAAAMUUQAAAAMUUQAAAAMUUQAAAAN052Mjh9BVBWAurpEAh82dKAAAgAGKKAAAgAGKKAAAgAGKKAAAgAG5zc2vmXkREe+tHt6KiIdbG3w7Oo/p81V1ctkX5DjkmHOM6DuutTlGfCRLOV7vmNekHHvIsY9rZA9rsocce2wnx111EMrM86o628ngM9nFMclxf8fchm0flxz3e8y5ybGHHPu4RvawJnvIsce2jsnb+QAAAAYoogAAAAbssoi6u8Ox57KLY5Lj/o65Dds+Ljnu95hzk2MPOfZxjexhTfaQY4+tHNPO9kQBAADsI2/nAwAAGLD1Iiozn8/M32bmO5n58rbH75KZr2Tmg8z81WPPfSozf56Zv199/OSM48uxbw57n6Uc++w6Szm2jS/HvjnsfZZy7LPrLOXYNr4cJ9pqEZWZT0XE/x0R34iI/xIRL2bmf9nmHBq9GhHPP/HcyxHxVlU9ExFvrR63k2OfA8ry1ZBjl1fDud3h1ZBjh1fDud3h1ZBjl1fDud3h1ZBjh1djRzlu+07UsxHxTlW9W1X/b0T8KCJe2PIcWlTVLyLiL088/UJEvLb6/LWI+OZMw8uxz0FkKcc+zu0ecuzh3O4hxz7O7R5y7LHLHLddRH0mIv7w2OP7q+cOxe2q+uPq8z9FxO2ZxpFjn0POUo59nNs95NjDud1Djn2c2z3k2GMrOWosMZN61PZQ68OJ5NhDjn1k2UOOPeTYQ459ZNlDjj3mzHHbRdT7EfG5xx5/dvXcofhzZn46ImL18cFM48ixzyFnKcc+zu0ecuzh3O4hxz7O7R5y7LGVHLddRL0dEc9k5hcy8+MR8e2IeGPLc5jTGxHx0urzlyLixzONI8c+h5ylHPs4t3vIsYdzu4cc+zi3e8ixx3ZyrKqt/omI/zMifhcR/09E/F/bHr/xOH4YEX+MiP8vHr2X9DsR8X/Eoy4gv4+I/xkRn5LjsnM8lCzleDhZylGOS8rxULKU4+FkKUc5LiXHXE0AAACAG9BYAgAAYIAiCgAAYIAiCgAAYIAiCgAAYIAiCgAAYIAiCgAAYIAiCgAAYIAiCgAAYMD/D4pywOK7cOXDAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 1080x1080 with 200 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "seq = repeat(torch.LongTensor([[start_of_seq_token]]).to(device), '1 d -> n d', n=100)\n",
        "gen_seq = gpt.generate(seq, 49, temperature=0.7, do_sample=True).detach().cpu()\n",
        "reconst = detokenize_MNIST(gen_seq[:,1:], patch_size=patch_size)\n",
        "\n",
        "fig = plt.figure(figsize=(15., 15.))\n",
        "grid = ImageGrid(fig, 111, nrows_ncols=(10, 10), axes_pad=0.1)\n",
        "for i in range(100):\n",
        "    grid[i].imshow(reconst[i], cmap='Greys')\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "de0adbae-2808-4667-8206-268e16748753",
      "metadata": {
        "id": "de0adbae-2808-4667-8206-268e16748753"
      },
      "source": [
        "## 5 Further reading\n",
        "\n",
        "Having implemented self-attention, as well as encoder-only and decoder-only Transformers, you should not have a hard time implementing cross-attention and a full encoder-decoder Transformer to train on arbitrary sequence-to-sequence tasks.\n",
        "That said, here is some further reading material should you want to dive deeper.\n",
        "\n",
        "### Papers & Blogs\n",
        "\n",
        "- [Attention Is All You Need, Vaswani et al. 2017](https://arxiv.org/abs/1706.03762)\n",
        "- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, Dosovitskiy et al. 2020](https://arxiv.org/abs/2010.11929)\n",
        "- [DeiT III: Revenge of the ViT](https://arxiv.org/abs/2204.07118) → a more modern and simplified training recipe for supervised ViTs\n",
        "- [Transformer Circuits Thread](https://transformer-circuits.pub/)\n",
        "- [The Transformer Family Version 2.0, Lilian Weng 2023](https://lilianweng.github.io/posts/2023-01-27-the-transformer-family-v2/)\n",
        "- [The Illustrated Transformer, Jay Alammar 2018](https://jalammar.github.io/illustrated-transformer/)\n",
        "\n",
        "### Code bases\n",
        "\n",
        "If you want to use transformers in your project, we recommend taking a look at the following codebases:\n",
        "- [Self-Supervised Vision Transformers with DINO](https://github.com/facebookresearch/dino)\n",
        "- [Masked Autoencoders Are Scalable Vision Learners (MAE)](https://github.com/facebookresearch/mae)\n",
        "\n",
        "You might find that, in many ways, these repos are not that much different from what we implemented here!"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.13"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "0067fc4fc58244639c6a83fdab0b7e74": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c9219051a617407ab1db49590bc0b7d9",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_af915aa8f67d45e6ad5cd311846e5ec8",
            "value": 781
          }
        },
        "00abf37d10bb430fa4bc7e49705aa725": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_43282135126a40949fcaf86c302fab84",
            "placeholder": "​",
            "style": "IPY_MODEL_1412119ab3564f6aaf53d9d8bd694150",
            "value": " 781/781 [00:31&lt;00:00, 33.60it/s]"
          }
        },
        "01247f2a46524ce7a1fbb70fd24241cd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "019aea4bf4934391ba265db774734b7a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "01d2ff0e4b9a4687b9869fe5fac16089": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "021b281f677a4f16b808f55f33154971": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "036a414b407a405f8f1967bf269d8e2a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "04f3be74a824493ab1cf34a36775156a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_060945a179274ac3a5e19e36724ab2e6",
            "placeholder": "​",
            "style": "IPY_MODEL_e317352c96524cfb9f587ad1ca17bb91",
            "value": "Train loss: 0.053: 100%"
          }
        },
        "05857940f3aa4501a8729b6eb44b679c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "060945a179274ac3a5e19e36724ab2e6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "073e630263e54fd3be751588ed56326c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "077a521bfa64484d9057e5efe49f8078": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1d3e3c3e3e5548f5a7564df29e3a7917",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_79c39a1ff5354255b62f1fe9723cbb3b",
            "value": 781
          }
        },
        "07b167d4959e473cb80f85b9478574d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "08327570b1814a158889e8bfc4f831fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "08f1bb9f95184ff795b7d1d5edce9472": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0980528b56264f19a809a99b0f2e71a6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "09ed4e0249d347a6a76030d208a4ac09": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_63373abdf7d84dd38c1531604b02c36e",
            "placeholder": "​",
            "style": "IPY_MODEL_f25db568553a4a3aa78f63e5795d4c2a",
            "value": "Train loss: 0.511: 100%"
          }
        },
        "0a1700ab652c4e14b1d10404b3963235": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0a686322ff3b4aa3a6e98e477b417553": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_151cbfa15151457fa306df4b319a5007",
              "IPY_MODEL_f4764482aaa94b06a8683fbd22e14829",
              "IPY_MODEL_4a26185b05bb4183b8362eb3c2996913"
            ],
            "layout": "IPY_MODEL_907e0819834444aab4d94f600576b873"
          }
        },
        "0b5d93d232ff4d16b7d3711214901014": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9934ee1fdd5b449ca9e60be26da212fd",
            "placeholder": "​",
            "style": "IPY_MODEL_6b11a10db0de404c96b240fc42cf55f4",
            "value": " 781/781 [00:30&lt;00:00, 37.56it/s]"
          }
        },
        "0cf4a1e0413b4092a842b6383d76a91c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0fd7ddd5f1d64151ba8c87a41219c98f",
              "IPY_MODEL_372fb52fe7b148edad97ba7290817610",
              "IPY_MODEL_14aae9b86e7449aba3558cccb16d8abd"
            ],
            "layout": "IPY_MODEL_6c0a4272ad2e4ec485fd0589e9bfce88"
          }
        },
        "0da23eeab8d4481993525980dc1aeea9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0e4983f18aaa4059a0ba4492c2e97ec7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0fd7ddd5f1d64151ba8c87a41219c98f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_559523f1ba444ec2a17af9d00e56e0f5",
            "placeholder": "​",
            "style": "IPY_MODEL_d34673fcd913450a8ac5869b59b57275",
            "value": "Train loss: 0.521: 100%"
          }
        },
        "0fe9602e32ec45e293b50ab3fc982cda": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0ff80f19f250454eb2359ab9d6f7df3d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4b997ef33b7e4e67811fc69de88ee1d6",
            "placeholder": "​",
            "style": "IPY_MODEL_bbdd0ba0a5a941718b29f8b633524f1a",
            "value": " 781/781 [00:30&lt;00:00, 38.71it/s]"
          }
        },
        "110c483f388344f7a45ebe1a89539d59": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_073e630263e54fd3be751588ed56326c",
            "placeholder": "​",
            "style": "IPY_MODEL_450560edee374196b933593f0bc61def",
            "value": " 781/781 [00:30&lt;00:00, 37.49it/s]"
          }
        },
        "126edea5f38c45799959d5c0282ce914": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3cd9332d489a466398f5c4e30dcc3846",
            "placeholder": "​",
            "style": "IPY_MODEL_7c4bebfc5f30459aaad9f93812583fe1",
            "value": " 781/781 [00:32&lt;00:00, 26.47it/s]"
          }
        },
        "12e00bbdd71a47bd96e9ec9528a752c8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "12f4bc5979f04627af30adebfdf1b221": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_945248631b81441abca170f766754603",
            "placeholder": "​",
            "style": "IPY_MODEL_98104971d2774746b914e801f3d527b6",
            "value": "Train loss: 0.554: 100%"
          }
        },
        "1412119ab3564f6aaf53d9d8bd694150": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "14aae9b86e7449aba3558cccb16d8abd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_303b1a517fc843da9cd8f349f7b2c425",
            "placeholder": "​",
            "style": "IPY_MODEL_ef2b0b2022ed48a7b9d67acdb958b6a2",
            "value": " 781/781 [00:31&lt;00:00, 33.83it/s]"
          }
        },
        "14e9e0ea5a7e4556b4035e070701f95f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e3154cb5142b4cad821330904f847dea",
              "IPY_MODEL_84a79ecba8b047a1932d1ba7fdec2e97",
              "IPY_MODEL_5cc45c37464b4e46b726069fafc22d20"
            ],
            "layout": "IPY_MODEL_e5c4b5dbeaf9496fa6d51b3377b3b944"
          }
        },
        "151cbfa15151457fa306df4b319a5007": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b32f6272195747ce83c49adf7867df4e",
            "placeholder": "​",
            "style": "IPY_MODEL_a7908a2231144a1494f39a8df5b581e5",
            "value": "Train loss: 0.382: 100%"
          }
        },
        "1530d6fe1d354981bebf1e6fe6ebc34c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "15575d779c504287803d20a563848058": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_98575417888545f49819162112a2e56e",
            "placeholder": "​",
            "style": "IPY_MODEL_5f755c5160004595aa9afc51a457add4",
            "value": "Train loss: 0.508: 100%"
          }
        },
        "156c5fda89fc43c7941cd8bcceb23a6d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1632d25d9f70459f8896f58fd282c494": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "167a1ec127bc47cc8814d06907542979": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bfd6f20c551e4850b78464b000a79651",
            "placeholder": "​",
            "style": "IPY_MODEL_815c8cd821d34c649cf6cfd32cf88e15",
            "value": " 781/781 [00:38&lt;00:00, 29.74it/s]"
          }
        },
        "178c940ec6d5415594bd139d6dd70673": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "17aa582e221744209c5a9aaf6bf27c3a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "19a38e9cb8b442f88363e3cc2be1c92b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1a99277fe0ff4629a9d34753e5f0a6c3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ee3dce92cc3e4f4b892a91b2bccf1ef2",
            "placeholder": "​",
            "style": "IPY_MODEL_1530d6fe1d354981bebf1e6fe6ebc34c",
            "value": "Train loss: 0.486: 100%"
          }
        },
        "1becf7be262e4ad78a1c0b9c07ed9fcb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1c6be90a12744a53a9ea107b285e04ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1c8056b0b5c544f0a0636b29dcd28e8a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eb5d07765390477d9503441820c92c57",
            "placeholder": "​",
            "style": "IPY_MODEL_c5975d294acf4e5abb6d14cfec4709e2",
            "value": "Train loss: 0.047: 100%"
          }
        },
        "1d3e3c3e3e5548f5a7564df29e3a7917": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1e85194f74e6443b9c68ee066589ff39": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2ef5bc7213d14e27bf6452ecd6a4b3cf",
              "IPY_MODEL_91f6d977c5134906a56ebaec6abfad0c",
              "IPY_MODEL_6e2d164bec60427ea0fcf5475a0cb763"
            ],
            "layout": "IPY_MODEL_945d5e1d58a2489c8437e9b5f317c57a"
          }
        },
        "1ec9a65c205e445e88a9e1770a2a070d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "202a7306f806476b8513383606fa187b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2085c9bcffde4e4ab2a75d91d5f5b5ff": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "223bfcde129642afa38da0d98fc9b9ce": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "227fbdfa52884cd79383e414726a05d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "22900dc085094bb0a3c03b7f06708529": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "26c1a007b7734943acc816ee2103cf2e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "26e2a9f1657844e89a9cf06ba0b8e3ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4ee102b6a6114c08a75d5d55b5854395",
            "placeholder": "​",
            "style": "IPY_MODEL_28e76d9ec4d64639a441760f9700684c",
            "value": "Train loss: 0.101: 100%"
          }
        },
        "274f524ff843460b87fdef21efeabf32": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2795973dc1b14bc4b0fedcd6182cb44f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "27bded8d14ac423b99aed42725d8c73c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5b170f0d31d34f73b4aa59893e37ab96",
              "IPY_MODEL_9ac3ed08af36466a82ae29fd8bacee2c",
              "IPY_MODEL_0b5d93d232ff4d16b7d3711214901014"
            ],
            "layout": "IPY_MODEL_c8a3ff14902d4fe9b9d1e676a77359db"
          }
        },
        "28e76d9ec4d64639a441760f9700684c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "29802a016636479a86807967a9df5d94": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "29e4df8f59774788b8865bf9743ecabc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d99936049aad4b55867639bf2df2ae32",
            "placeholder": "​",
            "style": "IPY_MODEL_cf6678ff63034625b5050f2653931cfc",
            "value": " 781/781 [00:31&lt;00:00, 34.59it/s]"
          }
        },
        "2d03da80f5f94c70b3bf26a8701d2c76": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2ddf5de2b14c46318989e59f8e23ba03": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e725176a05704b87a0147ac9d9fbcc69",
            "placeholder": "​",
            "style": "IPY_MODEL_5205553804e4457e98e702c61b019a8f",
            "value": "Train loss: 0.122: 100%"
          }
        },
        "2ef5bc7213d14e27bf6452ecd6a4b3cf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6a0d23126dc8481f8a5fe0f3a67d2319",
            "placeholder": "​",
            "style": "IPY_MODEL_f3053da2315c4988b316a7967e7d2a2c",
            "value": "100%"
          }
        },
        "303b1a517fc843da9cd8f349f7b2c425": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3043d4b1c3df4166931ba7589383a5c9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6aa5a826d1264afe8e949737b0dc0838",
              "IPY_MODEL_953ce56b49b747809dabe0dce1172c20",
              "IPY_MODEL_340771315b63457cb745f3d07abe694f"
            ],
            "layout": "IPY_MODEL_9162e2df1d5e417c9d382f619fbcc0dd"
          }
        },
        "3045f82c018d4ced87bb00138eb56302": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_43bf6b5771d14d66a68921c961885637",
              "IPY_MODEL_6012215f59c34da593a64d97b099db44",
              "IPY_MODEL_57e99e364bdf4588ab5bb832deb73cf1"
            ],
            "layout": "IPY_MODEL_6654c11fac444393bbb8840ddd9976b1"
          }
        },
        "305c1bb60a884d4fa718abba2c43ca0e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_effd6edac09e4d92bd7419a42813e688",
            "placeholder": "​",
            "style": "IPY_MODEL_7e815ba7d37c47d5a3535782bea9fbc8",
            "value": "Train loss: 0.032: 100%"
          }
        },
        "305d94d8ab2b44f8b81d4edd9fe079fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_19a38e9cb8b442f88363e3cc2be1c92b",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7f5d4bab37e7453daae7d4785d8047e8",
            "value": 781
          }
        },
        "318fc9b6a8334c3aaa49f14bf74ac7ca": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "32013f4694af4b83bb6fcee3036ae746": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3233c47e64374330bf6323559883f5c0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "32903d82d192473e8b37796617467a97": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "333cd168c68c4362b64a95463cd896be": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "33896793d8c54fc6a08fd2a253e5f707": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b786e576fcbd4bb68bc14ccccf88f977",
            "placeholder": "​",
            "style": "IPY_MODEL_f97e214116a443bab73bbf6ff941c252",
            "value": " 781/781 [00:30&lt;00:00, 38.11it/s]"
          }
        },
        "340771315b63457cb745f3d07abe694f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_558940b629c648c3b7bd4453c4948f2e",
            "placeholder": "​",
            "style": "IPY_MODEL_e9abe64c78794d9fb469f277293ee5db",
            "value": " 4542/4542 [00:00&lt;00:00, 309241.75it/s]"
          }
        },
        "368b7eb9c13e4a8f86c576bbe6028845": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "372fb52fe7b148edad97ba7290817610": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f56b98e2535f48969fa64bcf4c32ed6f",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c8d6ff0e5df04fc19e1f5ecc0c3dcc3a",
            "value": 781
          }
        },
        "38045f1d73c44939a7b35e32ba7e5ecb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "387771a90e714c01bceb76a3a9da4303": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_dd1c876d96e84b578b2fc4b5ff1e6280",
              "IPY_MODEL_96cbf761e13b41358ef20eda1c2cf6a7",
              "IPY_MODEL_86895473d2a949388d87abc5ccc4e410"
            ],
            "layout": "IPY_MODEL_df64bc3a3c934e0fb5bc840b87acbdba"
          }
        },
        "3905c161624544cea4381919e11885e5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3a45bac4e8084d42884da7a1110f0b19": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3b52f063eb9b4dcdbd518cb8c59bb692": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3b5a5332167b4311b58fe41de7724ee5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3c5d22b32b1649f883c1b20673415182": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3c724e55d52d4d17b784f6a5b8f80a73": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3cc4e194924e4ea18fbdfcdfe5d6305e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3cd9332d489a466398f5c4e30dcc3846": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3e36fd8723714c1aabbb9d73e58cf75a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3e5268ab9d64413a95bf9999a5376bf9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "409997d07e434d0c89a9b344b7d065ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "40b0ac09afe54e98a7c839815b504bcd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "40b560b94bc4428eac05f03efae7569d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2ddf5de2b14c46318989e59f8e23ba03",
              "IPY_MODEL_bee7740b8113472885599de0bf4300f8",
              "IPY_MODEL_6f7f2ff1a3d84d9ab141f3cb7e24149d"
            ],
            "layout": "IPY_MODEL_558897da5b95456c9ca500d9ffff7b32"
          }
        },
        "41a2b31c5adc48a49c5e7a81a4f7e80a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "43282135126a40949fcaf86c302fab84": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "43bf6b5771d14d66a68921c961885637": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a6cd6cfde5274f3cb478ab99110ad7bd",
            "placeholder": "​",
            "style": "IPY_MODEL_8488709617e14b5a8e084e6287764c17",
            "value": "Train loss: 0.094: 100%"
          }
        },
        "441dce0b30b14c3fb3408874f53f66d2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "44763f7208804cf099a90833d548d72b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "44ea713552704e75b9d7c866799dea71": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6fac5e472fd543878fd1903ad6c9148e",
              "IPY_MODEL_d557cb1336fd4076b035216c10c874ce",
              "IPY_MODEL_602f1c6221714d70a00b65636734375f"
            ],
            "layout": "IPY_MODEL_1becf7be262e4ad78a1c0b9c07ed9fcb"
          }
        },
        "450560edee374196b933593f0bc61def": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "46ac8c9ef4e241d2865c4cfc10895b62": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4828b3e350bd4605b3af7648d451bc2c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4a05adbaba514ef398918e0ce4a2f33d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4a26185b05bb4183b8362eb3c2996913": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a91e4e9e5dbe4b5cb8a0e2d9cf2da5b4",
            "placeholder": "​",
            "style": "IPY_MODEL_4fbd96c5aa8945a5ab3949f70295cbdf",
            "value": " 781/781 [00:29&lt;00:00, 37.06it/s]"
          }
        },
        "4a27ab657ef549c189f78ef100333f31": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2085c9bcffde4e4ab2a75d91d5f5b5ff",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d7450585236c4e7abf94348f71ef693e",
            "value": 781
          }
        },
        "4b73705c99c1403db27318149ee39ce8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_01247f2a46524ce7a1fbb70fd24241cd",
            "placeholder": "​",
            "style": "IPY_MODEL_a41eb04cdc644fa188e16c54a59f7470",
            "value": " 28881/28881 [00:00&lt;00:00, 954816.77it/s]"
          }
        },
        "4b7ef8d3b7284cf4addea2ea63d642e4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1a99277fe0ff4629a9d34753e5f0a6c3",
              "IPY_MODEL_8cae1aa38f8e44faa35337c99982fc7f",
              "IPY_MODEL_76fd901f7f6b4c09bb4ce8fca806c4d8"
            ],
            "layout": "IPY_MODEL_9fb8fdb1f4034480841c1fa364e10edc"
          }
        },
        "4b997ef33b7e4e67811fc69de88ee1d6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4d58ff4816934cf8bc3b8a48ce5a9dcf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_870922ce6c294ed0a67b7a3a90159bdf",
            "max": 28881,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2795973dc1b14bc4b0fedcd6182cb44f",
            "value": 28881
          }
        },
        "4dc11e04cc564e0ebaff0e5bc091d22a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5b8781a399c54223921a6b41d2dade9d",
            "placeholder": "​",
            "style": "IPY_MODEL_202a7306f806476b8513383606fa187b",
            "value": "Train loss: 0.049: 100%"
          }
        },
        "4e9d6998e8c0417ab8d49f6e28910e7a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4ee102b6a6114c08a75d5d55b5854395": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4f35d75810f142aeade2f3b06e5cb699": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_41a2b31c5adc48a49c5e7a81a4f7e80a",
            "placeholder": "​",
            "style": "IPY_MODEL_88120b2053854c1a8f7e472bfafb388a",
            "value": "Train loss: 0.471: 100%"
          }
        },
        "4f45a8840b27419a8cc5ab4cfecd2410": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_09ed4e0249d347a6a76030d208a4ac09",
              "IPY_MODEL_f76e2720d2df499ca254afc5a3217ee0",
              "IPY_MODEL_00abf37d10bb430fa4bc7e49705aa725"
            ],
            "layout": "IPY_MODEL_9b11ae43dbd74a99913c2bdd3756e8b8"
          }
        },
        "4fbd96c5aa8945a5ab3949f70295cbdf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4ff0c5ed0eab463a81aaa30d9c635fb3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "505c3ec78ba84c3493508754df165607": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "50add4dbfcb949f9a88cb387e260160d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_51a8927325a243948371089e9623deeb",
            "placeholder": "​",
            "style": "IPY_MODEL_dd5318f1059848c583f38d3420b1d17a",
            "value": "Train loss: 0.045: 100%"
          }
        },
        "5126718b1dc345c290071b41a8e2cc7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "51a8927325a243948371089e9623deeb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "51db5d4ebae046d999acb1b378054e35": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_15575d779c504287803d20a563848058",
              "IPY_MODEL_f008f48ed3244f969592774a627e66ec",
              "IPY_MODEL_29e4df8f59774788b8865bf9743ecabc"
            ],
            "layout": "IPY_MODEL_96ed74eff0ca4be583223afab0a145e7"
          }
        },
        "5205553804e4457e98e702c61b019a8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "524134a8fa5e467298954ad22beb92ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e6cc7003780f4f2085eb888ea3a69d45",
            "placeholder": "​",
            "style": "IPY_MODEL_4ff0c5ed0eab463a81aaa30d9c635fb3",
            "value": " 781/781 [00:30&lt;00:00, 36.83it/s]"
          }
        },
        "54ee3648fec94055b57fde13bd0aded5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "558897da5b95456c9ca500d9ffff7b32": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "558940b629c648c3b7bd4453c4948f2e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "559523f1ba444ec2a17af9d00e56e0f5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "57307bac1b314c7bb1541cc6c4bf77c5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "57e99e364bdf4588ab5bb832deb73cf1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8890eff393b142a3a0633f8f43e7f143",
            "placeholder": "​",
            "style": "IPY_MODEL_4e9d6998e8c0417ab8d49f6e28910e7a",
            "value": " 781/781 [00:29&lt;00:00, 39.22it/s]"
          }
        },
        "58ecd9217c4f4041a21e5631a0d7541b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5a3c09ae450946978ef5e34eb6cc77f3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5b170f0d31d34f73b4aa59893e37ab96": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b4924076c123415eaf5be8dd463c727c",
            "placeholder": "​",
            "style": "IPY_MODEL_3b5a5332167b4311b58fe41de7724ee5",
            "value": "Train loss: 0.058: 100%"
          }
        },
        "5b8781a399c54223921a6b41d2dade9d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5bfb15a87f17454b893293fb8a7dc110": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5bffe63966ad4232864fa9b3e60b0f1c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3c724e55d52d4d17b784f6a5b8f80a73",
            "placeholder": "​",
            "style": "IPY_MODEL_bc90ab55ca2d4889ab12113123901283",
            "value": "Train loss: 0.497: 100%"
          }
        },
        "5c54a4c5d84d47f38fd060cc8e588b7f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5cc45c37464b4e46b726069fafc22d20": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b2fe8edcbcd64b15b9e8aabed1a4ed6a",
            "placeholder": "​",
            "style": "IPY_MODEL_943af8e34b3147d8a33b1ca09cb18e8c",
            "value": " 781/781 [00:30&lt;00:00, 36.26it/s]"
          }
        },
        "5cee094c55004d058098a05d02c8b5cb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9fb50fc960484d2187594c3671da6b7b",
              "IPY_MODEL_077a521bfa64484d9057e5efe49f8078",
              "IPY_MODEL_e0ba5051ac0a4d9ea6f80b2c0fe50c7c"
            ],
            "layout": "IPY_MODEL_0e4983f18aaa4059a0ba4492c2e97ec7"
          }
        },
        "5d61b5810b124ca8b778c40dfead927c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_cfcfb371ca8949769ff6f79bc367c298",
              "IPY_MODEL_7d678bf7171e4b659a967a6ca127805a",
              "IPY_MODEL_df47640718b4433597228fb8f9dfec0e"
            ],
            "layout": "IPY_MODEL_46ac8c9ef4e241d2865c4cfc10895b62"
          }
        },
        "5de361c077604266a257e593df56dd6f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5f2ebdf404df438d81bdbdd3ecc63a2b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1c8056b0b5c544f0a0636b29dcd28e8a",
              "IPY_MODEL_a481397f471345bfbd922299ef872da5",
              "IPY_MODEL_e25ee5853ae8423a9cf17f7d410302c2"
            ],
            "layout": "IPY_MODEL_05857940f3aa4501a8729b6eb44b679c"
          }
        },
        "5f755c5160004595aa9afc51a457add4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5fcda9418c204f788fd2ed1839a7d208": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6012215f59c34da593a64d97b099db44": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3a45bac4e8084d42884da7a1110f0b19",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_26c1a007b7734943acc816ee2103cf2e",
            "value": 781
          }
        },
        "602f1c6221714d70a00b65636734375f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a103eb5850e04df781913a635d409914",
            "placeholder": "​",
            "style": "IPY_MODEL_efd8220fec5d410799e70ba4593d1975",
            "value": " 781/781 [00:30&lt;00:00, 36.79it/s]"
          }
        },
        "60883aeb21bb4247ad62e5d828b7497a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6229c61810774c1c93413503be2283c4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "63373abdf7d84dd38c1531604b02c36e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "633a8c7973e04fbfb24fea49f96abc05": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "63b20e94e1a64996a7cf1d526d622068": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_78e31d8b3a7b49dca42cf4f12faceed0",
              "IPY_MODEL_e7261ab88f6f4d0ba467b93e857f3fa0",
              "IPY_MODEL_33896793d8c54fc6a08fd2a253e5f707"
            ],
            "layout": "IPY_MODEL_d915cc0f06974f07bed91c6d58589d6f"
          }
        },
        "6446fe6a2c134793bcef51a456b48c48": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_cb1242f93e714c3b968a86fa479ba212",
              "IPY_MODEL_e354dfba004445f5b506fca946773f4d",
              "IPY_MODEL_7ba80ffdabb04d24989bf0bba9319b70"
            ],
            "layout": "IPY_MODEL_318fc9b6a8334c3aaa49f14bf74ac7ca"
          }
        },
        "64c7ecae57af44edad6ffe9c8b60169b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "658fdf59fab649b99ac3cc76e0ef045a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_26e2a9f1657844e89a9cf06ba0b8e3ec",
              "IPY_MODEL_b057f955348d4576bfe7c34dbc9c53dc",
              "IPY_MODEL_f58b4867d34f497abdb22861e6a67a7f"
            ],
            "layout": "IPY_MODEL_8a24d21743d34b988ac456a52aeced80"
          }
        },
        "6654c11fac444393bbb8840ddd9976b1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6728963c26d346d79fb9a26f4d112d8e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6da293f3e86044bc9e22f27968f1733c",
            "placeholder": "​",
            "style": "IPY_MODEL_40b0ac09afe54e98a7c839815b504bcd",
            "value": " 781/781 [00:31&lt;00:00, 33.24it/s]"
          }
        },
        "694da459b1704fd3a42c108ca51c2353": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6a0d23126dc8481f8a5fe0f3a67d2319": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6a33a35f1447480495c1c4447094fed8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6a78a38b280740ea9344c08be3f87485": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6aa5a826d1264afe8e949737b0dc0838": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fd73fc9d8ae54649ba27690670784b85",
            "placeholder": "​",
            "style": "IPY_MODEL_912172a35481491fae2b3a1174e92e90",
            "value": "100%"
          }
        },
        "6b11a10db0de404c96b240fc42cf55f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6c0a4272ad2e4ec485fd0589e9bfce88": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6cf34c949d3c4b26aff36588648b0068": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6da293f3e86044bc9e22f27968f1733c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6e2d164bec60427ea0fcf5475a0cb763": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_633a8c7973e04fbfb24fea49f96abc05",
            "placeholder": "​",
            "style": "IPY_MODEL_74bffc782dd247db9a4adbdaec1db9d0",
            "value": " 1648877/1648877 [00:00&lt;00:00, 25712883.40it/s]"
          }
        },
        "6f7f2ff1a3d84d9ab141f3cb7e24149d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_694da459b1704fd3a42c108ca51c2353",
            "placeholder": "​",
            "style": "IPY_MODEL_a4f964aa65c54128a346f976e5ac3b03",
            "value": " 781/781 [00:30&lt;00:00, 37.88it/s]"
          }
        },
        "6fac5e472fd543878fd1903ad6c9148e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_71044263d0064fb880c872bfd2b1c4cf",
            "placeholder": "​",
            "style": "IPY_MODEL_9b8ac3ca4a6f4b75bfc083f27f455e97",
            "value": "Train loss: 0.111: 100%"
          }
        },
        "71044263d0064fb880c872bfd2b1c4cf": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7251e216f9144b8a86a63dd72c3c3ef0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f17fcedc467f4957abcdac2b9ce4c164",
            "placeholder": "​",
            "style": "IPY_MODEL_bbacdb46c586476abbd8e38c68904d48",
            "value": "100%"
          }
        },
        "74bffc782dd247db9a4adbdaec1db9d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "74d448d9a3634f2ca987b2125d22d457": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eb78842cbd2a41b79f08810246e4adb2",
            "placeholder": "​",
            "style": "IPY_MODEL_409997d07e434d0c89a9b344b7d065ee",
            "value": " 781/781 [00:30&lt;00:00, 38.75it/s]"
          }
        },
        "75388f4a628240cba9fa225745edd318": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "75495f03a2ba487c833f43728338e211": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "76fd901f7f6b4c09bb4ce8fca806c4d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9649d26dfe40423ebea71132d24b7848",
            "placeholder": "​",
            "style": "IPY_MODEL_227fbdfa52884cd79383e414726a05d6",
            "value": " 781/781 [00:31&lt;00:00, 33.55it/s]"
          }
        },
        "78e31d8b3a7b49dca42cf4f12faceed0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4828b3e350bd4605b3af7648d451bc2c",
            "placeholder": "​",
            "style": "IPY_MODEL_9ee9a714253548659da00556bb0d0d6a",
            "value": "Train loss: 0.043: 100%"
          }
        },
        "79c39a1ff5354255b62f1fe9723cbb3b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "7a7a32ef7e3947fc8081fdbf31269c6d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a5eef95245694ff19cf8bc9b10a4ac3d",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3c5d22b32b1649f883c1b20673415182",
            "value": 781
          }
        },
        "7a7fd17e44764bf3bb1dc2d5f5e8081a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7ba80ffdabb04d24989bf0bba9319b70": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a3348693e09f420caa595cfd2a37082a",
            "placeholder": "​",
            "style": "IPY_MODEL_c0d8d1e307034d74b1410fef51f624f8",
            "value": " 781/781 [00:30&lt;00:00, 28.91it/s]"
          }
        },
        "7c4bebfc5f30459aaad9f93812583fe1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7d1021bfe3064cb0b9a0b2b15d4cb4c9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7d678bf7171e4b659a967a6ca127805a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e0eed1f3b47c475c99298e0b5d58bea4",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_aa4925cf508046cf86bcc2b07fc5e988",
            "value": 781
          }
        },
        "7e815ba7d37c47d5a3535782bea9fbc8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7f1e9186142f4c7f9fffe394f31af487": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7f5d4bab37e7453daae7d4785d8047e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "815c8cd821d34c649cf6cfd32cf88e15": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8191e371f2394e69b2415f27bf818234": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c7b77df568094dab89d2a95538f9a597",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ce2b1bc8776b45d9b5c6d5fedb08ff0a",
            "value": 781
          }
        },
        "82020938a5a34a28bd3334979f0ac843": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e2d9da8b00a442bfb0c1f304a0400fcf",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_44763f7208804cf099a90833d548d72b",
            "value": 781
          }
        },
        "82bcfff5dfbe44248dc2529c0c5f7528": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8488709617e14b5a8e084e6287764c17": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "84a79ecba8b047a1932d1ba7fdec2e97": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7f1e9186142f4c7f9fffe394f31af487",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_333cd168c68c4362b64a95463cd896be",
            "value": 781
          }
        },
        "85d6dbc5ee754df2bcaeddf02e6789fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_adef5d80e5dd4b2d9326e0bcbedbdb16",
              "IPY_MODEL_82020938a5a34a28bd3334979f0ac843",
              "IPY_MODEL_74d448d9a3634f2ca987b2125d22d457"
            ],
            "layout": "IPY_MODEL_223bfcde129642afa38da0d98fc9b9ce"
          }
        },
        "865fd577e9524a78b74b8fc2faf44dbd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2d03da80f5f94c70b3bf26a8701d2c76",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_08327570b1814a158889e8bfc4f831fc",
            "value": 781
          }
        },
        "86895473d2a949388d87abc5ccc4e410": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0980528b56264f19a809a99b0f2e71a6",
            "placeholder": "​",
            "style": "IPY_MODEL_019aea4bf4934391ba265db774734b7a",
            "value": " 781/781 [00:30&lt;00:00, 38.16it/s]"
          }
        },
        "8702e5ad0948471dad7060f1c36ec6bb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "870922ce6c294ed0a67b7a3a90159bdf": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "88120b2053854c1a8f7e472bfafb388a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "881cdd41388b49c3a0024cc23fc0f23a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1632d25d9f70459f8896f58fd282c494",
            "placeholder": "​",
            "style": "IPY_MODEL_9679e09815ee4a8684c52c63cf269006",
            "value": "Train loss: 0.015: 100%"
          }
        },
        "8890eff393b142a3a0633f8f43e7f143": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "893c262eb59e48fa95dfe316177e82f3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8a24d21743d34b988ac456a52aeced80": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8b600c06a3594e90ad9cb3bc73c7118e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8c4299113ba642498129668cbd066d6e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8cab9058b9aa4b4792149b9ab4ddc64e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d2e2f6883ee847e985c4f9199e3079f3",
            "placeholder": "​",
            "style": "IPY_MODEL_ea098e2276654d62a3f4326b309602a6",
            "value": " 781/781 [00:31&lt;00:00, 33.45it/s]"
          }
        },
        "8cae1aa38f8e44faa35337c99982fc7f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_adc2ad5f22b845329301c33b9842c9fc",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_beb5b3e87563463a920f68f065b911e7",
            "value": 781
          }
        },
        "8dd04c97854047e587fd54b3d4dfa041": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_04f3be74a824493ab1cf34a36775156a",
              "IPY_MODEL_4a27ab657ef549c189f78ef100333f31",
              "IPY_MODEL_aa326dd4cfeb437caa4edadccc94bd5b"
            ],
            "layout": "IPY_MODEL_8b600c06a3594e90ad9cb3bc73c7118e"
          }
        },
        "8f810e3829684e159fc1086055cfb2d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c9d6341ccd244039a2f7462d8aa9141c",
              "IPY_MODEL_df24f33c7a62458d8f913c3df7b37788",
              "IPY_MODEL_167a1ec127bc47cc8814d06907542979"
            ],
            "layout": "IPY_MODEL_9579c7d841ef4bac997b804cf2a43d6e"
          }
        },
        "8fcf71d367d5416998c3ade31ac3ac9c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "907e0819834444aab4d94f600576b873": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "912172a35481491fae2b3a1174e92e90": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9162e2df1d5e417c9d382f619fbcc0dd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "91f6d977c5134906a56ebaec6abfad0c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e33334969c534e0f9250bbb00a8c9967",
            "max": 1648877,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b43cd727390241e388bd7756617c4480",
            "value": 1648877
          }
        },
        "9272fe045e8a4b2e83434e83e61e9128": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f243d5c87dfb47a18d52264cf442c159",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_17aa582e221744209c5a9aaf6bf27c3a",
            "value": 781
          }
        },
        "937408e39c86440cb6aa5ca2814c5c0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_881cdd41388b49c3a0024cc23fc0f23a",
              "IPY_MODEL_d4dbb9427d0e4cb6933e6560dd030bde",
              "IPY_MODEL_9e9df9c1866c477697203d288da69e8e"
            ],
            "layout": "IPY_MODEL_b59ef284e5dc494180e4d093b1ad7e2b"
          }
        },
        "943af8e34b3147d8a33b1ca09cb18e8c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "945248631b81441abca170f766754603": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "945d5e1d58a2489c8437e9b5f317c57a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "953ce56b49b747809dabe0dce1172c20": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4a05adbaba514ef398918e0ce4a2f33d",
            "max": 4542,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_82bcfff5dfbe44248dc2529c0c5f7528",
            "value": 4542
          }
        },
        "9579c7d841ef4bac997b804cf2a43d6e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9649d26dfe40423ebea71132d24b7848": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9671b868aa414ca2b21046170ef7955f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_50add4dbfcb949f9a88cb387e260160d",
              "IPY_MODEL_8191e371f2394e69b2415f27bf818234",
              "IPY_MODEL_110c483f388344f7a45ebe1a89539d59"
            ],
            "layout": "IPY_MODEL_a5948c21d65a45c69704fa3a08751836"
          }
        },
        "9679e09815ee4a8684c52c63cf269006": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "96cbf761e13b41358ef20eda1c2cf6a7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a6a5dd909ba14bd7ab47c294af9077af",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a38562dd5aea49639812e9668455c645",
            "value": 781
          }
        },
        "96ed74eff0ca4be583223afab0a145e7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "98104971d2774746b914e801f3d527b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "98575417888545f49819162112a2e56e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9934ee1fdd5b449ca9e60be26da212fd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "99bc0ee17d1e4e6882c2fc6fe7ebd84b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_305c1bb60a884d4fa718abba2c43ca0e",
              "IPY_MODEL_865fd577e9524a78b74b8fc2faf44dbd",
              "IPY_MODEL_bb4cbbe7d4ba4ce0800ee5f27f0407c1"
            ],
            "layout": "IPY_MODEL_8fcf71d367d5416998c3ade31ac3ac9c"
          }
        },
        "99fccfad239f489999c9678f00886707": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9ac3ed08af36466a82ae29fd8bacee2c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d313b4fef20c401a9c694092d000eb7b",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_07b167d4959e473cb80f85b9478574d3",
            "value": 781
          }
        },
        "9b11ae43dbd74a99913c2bdd3756e8b8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9b8ac3ca4a6f4b75bfc083f27f455e97": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9c12f610fd334ed8ae618959153e4acb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9e9df9c1866c477697203d288da69e8e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_58ecd9217c4f4041a21e5631a0d7541b",
            "placeholder": "​",
            "style": "IPY_MODEL_f6de9b206baf4fd280a8ef0f48c5bce5",
            "value": " 781/781 [00:30&lt;00:00, 37.07it/s]"
          }
        },
        "9ee9a714253548659da00556bb0d0d6a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9fb50fc960484d2187594c3671da6b7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3233c47e64374330bf6323559883f5c0",
            "placeholder": "​",
            "style": "IPY_MODEL_22900dc085094bb0a3c03b7f06708529",
            "value": "Train loss: 0.497: 100%"
          }
        },
        "9fb8fdb1f4034480841c1fa364e10edc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a0d9789ed5aa4134b331e23e6d4355b9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a103eb5850e04df781913a635d409914": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a322bd3bc97b4b9989da72ec0ccb0ea0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a3348693e09f420caa595cfd2a37082a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a38562dd5aea49639812e9668455c645": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a41eb04cdc644fa188e16c54a59f7470": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a481397f471345bfbd922299ef872da5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_505c3ec78ba84c3493508754df165607",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d2a6821f07624643bda3c06ef06bf393",
            "value": 781
          }
        },
        "a4f964aa65c54128a346f976e5ac3b03": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a5948c21d65a45c69704fa3a08751836": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a5eef95245694ff19cf8bc9b10a4ac3d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a6a5dd909ba14bd7ab47c294af9077af": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a6cd6cfde5274f3cb478ab99110ad7bd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a7908a2231144a1494f39a8df5b581e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a91e4e9e5dbe4b5cb8a0e2d9cf2da5b4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa326dd4cfeb437caa4edadccc94bd5b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_12e00bbdd71a47bd96e9ec9528a752c8",
            "placeholder": "​",
            "style": "IPY_MODEL_e8b279b6d4ab46658ec7ab0dec786c23",
            "value": " 781/781 [00:31&lt;00:00, 28.43it/s]"
          }
        },
        "aa4925cf508046cf86bcc2b07fc5e988": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ab23f1dbc29c4a7da080fd65d7823753": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e04e364f2eb242999324cbe1df91628c",
              "IPY_MODEL_0067fc4fc58244639c6a83fdab0b7e74",
              "IPY_MODEL_6728963c26d346d79fb9a26f4d112d8e"
            ],
            "layout": "IPY_MODEL_5bfb15a87f17454b893293fb8a7dc110"
          }
        },
        "ac119fa734f546498441be6d7f2b8be9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_b1849889c6684962b870d02520aaad73",
              "IPY_MODEL_f8177954e77e4faf971a763c78133030",
              "IPY_MODEL_c0b3e3709e5840f88b1c907a33417f5f"
            ],
            "layout": "IPY_MODEL_5fcda9418c204f788fd2ed1839a7d208"
          }
        },
        "ad7d0ee981914dceadb4cb8610512207": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3905c161624544cea4381919e11885e5",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e7fac7d5a34346089efe5135aedb5822",
            "value": 781
          }
        },
        "adc2ad5f22b845329301c33b9842c9fc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "adef5d80e5dd4b2d9326e0bcbedbdb16": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c2ec115f26c14506b12054b08562e0be",
            "placeholder": "​",
            "style": "IPY_MODEL_c1ad6fb0575545d4812bda1591809721",
            "value": "Train loss: 0.380: 100%"
          }
        },
        "af915aa8f67d45e6ad5cd311846e5ec8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "af98ab56df7a4d5aa7b9448e0c277aad": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b057f955348d4576bfe7c34dbc9c53dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0fe9602e32ec45e293b50ab3fc982cda",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b1c91a48f00445a18b895025dd22f729",
            "value": 781
          }
        },
        "b1849889c6684962b870d02520aaad73": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6a33a35f1447480495c1c4447094fed8",
            "placeholder": "​",
            "style": "IPY_MODEL_29802a016636479a86807967a9df5d94",
            "value": "100%"
          }
        },
        "b1c91a48f00445a18b895025dd22f729": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b2fe8edcbcd64b15b9e8aabed1a4ed6a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b32f6272195747ce83c49adf7867df4e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b43cd727390241e388bd7756617c4480": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b4924076c123415eaf5be8dd463c727c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b59ef284e5dc494180e4d093b1ad7e2b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b786e576fcbd4bb68bc14ccccf88f977": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bb4cbbe7d4ba4ce0800ee5f27f0407c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6229c61810774c1c93413503be2283c4",
            "placeholder": "​",
            "style": "IPY_MODEL_5126718b1dc345c290071b41a8e2cc7b",
            "value": " 781/781 [00:29&lt;00:00, 37.85it/s]"
          }
        },
        "bbacdb46c586476abbd8e38c68904d48": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bbdd0ba0a5a941718b29f8b633524f1a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bc90ab55ca2d4889ab12113123901283": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bccaad81be33461286ef5bc347c17c26": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "be185ac21d4046b28c635b6d5a988f9e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "beb5b3e87563463a920f68f065b911e7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "bee7740b8113472885599de0bf4300f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_036a414b407a405f8f1967bf269d8e2a",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_eaaa5544fb6b45968e0873026af49704",
            "value": 781
          }
        },
        "bfd6f20c551e4850b78464b000a79651": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c0b3e3709e5840f88b1c907a33417f5f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f81e089cc735476cb5a22cc2bb26197a",
            "placeholder": "​",
            "style": "IPY_MODEL_d4cf7a541ba941dc82411e9a7a0946f8",
            "value": " 9912422/9912422 [00:00&lt;00:00, 206152032.95it/s]"
          }
        },
        "c0d8d1e307034d74b1410fef51f624f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c1ad6fb0575545d4812bda1591809721": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c2ec115f26c14506b12054b08562e0be": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c53979b4234f4ebfb280ff2f7eddf293": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_12f4bc5979f04627af30adebfdf1b221",
              "IPY_MODEL_dfa2923ad7d34c8a9d37212c91da8fc4",
              "IPY_MODEL_8cab9058b9aa4b4792149b9ab4ddc64e"
            ],
            "layout": "IPY_MODEL_32903d82d192473e8b37796617467a97"
          }
        },
        "c5975d294acf4e5abb6d14cfec4709e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c7b77df568094dab89d2a95538f9a597": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c8a3ff14902d4fe9b9d1e676a77359db": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c8b506a740ab455c8ef0b6b1a7363752": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c8d6ff0e5df04fc19e1f5ecc0c3dcc3a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c9219051a617407ab1db49590bc0b7d9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c9d6341ccd244039a2f7462d8aa9141c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5c54a4c5d84d47f38fd060cc8e588b7f",
            "placeholder": "​",
            "style": "IPY_MODEL_f5dce73bce444df18b3ed1761b68a378",
            "value": "Train loss: 0.539: 100%"
          }
        },
        "cac074db984d48ceba46f0eb83e8b776": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cb1242f93e714c3b968a86fa479ba212": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_db02cd0a19d24bd08763724a7522c61f",
            "placeholder": "​",
            "style": "IPY_MODEL_7d1021bfe3064cb0b9a0b2b15d4cb4c9",
            "value": "Train loss: 0.029: 100%"
          }
        },
        "cbcbc3daea274f67b60783c7b1dd008b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5bffe63966ad4232864fa9b3e60b0f1c",
              "IPY_MODEL_ad7d0ee981914dceadb4cb8610512207",
              "IPY_MODEL_ebdd5f1af9684cc4ae4966c4a312cad7"
            ],
            "layout": "IPY_MODEL_af98ab56df7a4d5aa7b9448e0c277aad"
          }
        },
        "ce2b1bc8776b45d9b5c6d5fedb08ff0a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "cf6678ff63034625b5050f2653931cfc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cfcfb371ca8949769ff6f79bc367c298": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d6f1815e37df4b5e91cafb0c096b7935",
            "placeholder": "​",
            "style": "IPY_MODEL_f6635ac72ee24817903921c52d72031e",
            "value": "Train loss: 0.525: 100%"
          }
        },
        "d2a6821f07624643bda3c06ef06bf393": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d2e2f6883ee847e985c4f9199e3079f3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d313b4fef20c401a9c694092d000eb7b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d34673fcd913450a8ac5869b59b57275": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d43a049331404cc9b191b5ecaa2b7752": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e5be6f76bf244baa8c9fba0b13ef00fb",
              "IPY_MODEL_9272fe045e8a4b2e83434e83e61e9128",
              "IPY_MODEL_f660234977c6456983ee16aedbaba5bf"
            ],
            "layout": "IPY_MODEL_368b7eb9c13e4a8f86c576bbe6028845"
          }
        },
        "d4cf7a541ba941dc82411e9a7a0946f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d4dbb9427d0e4cb6933e6560dd030bde": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a322bd3bc97b4b9989da72ec0ccb0ea0",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3b52f063eb9b4dcdbd518cb8c59bb692",
            "value": 781
          }
        },
        "d557cb1336fd4076b035216c10c874ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6a78a38b280740ea9344c08be3f87485",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3e36fd8723714c1aabbb9d73e58cf75a",
            "value": 781
          }
        },
        "d5581485f10b48faaea9eaf72ba9afae": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d63f450d96e94fb1b1c856a28e9f2703": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7251e216f9144b8a86a63dd72c3c3ef0",
              "IPY_MODEL_4d58ff4816934cf8bc3b8a48ce5a9dcf",
              "IPY_MODEL_4b73705c99c1403db27318149ee39ce8"
            ],
            "layout": "IPY_MODEL_be185ac21d4046b28c635b6d5a988f9e"
          }
        },
        "d6f1815e37df4b5e91cafb0c096b7935": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d7450585236c4e7abf94348f71ef693e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d915cc0f06974f07bed91c6d58589d6f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d99936049aad4b55867639bf2df2ae32": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "db02cd0a19d24bd08763724a7522c61f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dd1c876d96e84b578b2fc4b5ff1e6280": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df9e136b398847bf8eb1a54c7f9dc2a2",
            "placeholder": "​",
            "style": "IPY_MODEL_32013f4694af4b83bb6fcee3036ae746",
            "value": "Train loss: 0.030: 100%"
          }
        },
        "dd37bf2d158047a3ba6b6b5701dd3761": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dd5318f1059848c583f38d3420b1d17a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "df24f33c7a62458d8f913c3df7b37788": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_99fccfad239f489999c9678f00886707",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0da23eeab8d4481993525980dc1aeea9",
            "value": 781
          }
        },
        "df47640718b4433597228fb8f9dfec0e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9c12f610fd334ed8ae618959153e4acb",
            "placeholder": "​",
            "style": "IPY_MODEL_8c4299113ba642498129668cbd066d6e",
            "value": " 781/781 [00:31&lt;00:00, 32.40it/s]"
          }
        },
        "df64bc3a3c934e0fb5bc840b87acbdba": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df9e136b398847bf8eb1a54c7f9dc2a2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dfa2923ad7d34c8a9d37212c91da8fc4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_57307bac1b314c7bb1541cc6c4bf77c5",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_893c262eb59e48fa95dfe316177e82f3",
            "value": 781
          }
        },
        "dfe02df2573941c18cd17a6c6d3c3360": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e04e364f2eb242999324cbe1df91628c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3cc4e194924e4ea18fbdfcdfe5d6305e",
            "placeholder": "​",
            "style": "IPY_MODEL_bccaad81be33461286ef5bc347c17c26",
            "value": "Train loss: 0.503: 100%"
          }
        },
        "e0ba5051ac0a4d9ea6f80b2c0fe50c7c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cac074db984d48ceba46f0eb83e8b776",
            "placeholder": "​",
            "style": "IPY_MODEL_c8b506a740ab455c8ef0b6b1a7363752",
            "value": " 781/781 [00:32&lt;00:00, 28.48it/s]"
          }
        },
        "e0eed1f3b47c475c99298e0b5d58bea4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e25ee5853ae8423a9cf17f7d410302c2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6cf34c949d3c4b26aff36588648b0068",
            "placeholder": "​",
            "style": "IPY_MODEL_e8a6bc5e9dee4376a629a4d966c89ab5",
            "value": " 781/781 [00:30&lt;00:00, 27.57it/s]"
          }
        },
        "e2d9da8b00a442bfb0c1f304a0400fcf": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e3154cb5142b4cad821330904f847dea": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0a1700ab652c4e14b1d10404b3963235",
            "placeholder": "​",
            "style": "IPY_MODEL_f0da0adc084e4ec3a118fa8ccdbcd325",
            "value": "Train loss: 0.086: 100%"
          }
        },
        "e317352c96524cfb9f587ad1ca17bb91": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e33334969c534e0f9250bbb00a8c9967": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e354dfba004445f5b506fca946773f4d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_60883aeb21bb4247ad62e5d828b7497a",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d5581485f10b48faaea9eaf72ba9afae",
            "value": 781
          }
        },
        "e5be6f76bf244baa8c9fba0b13ef00fb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a0d9789ed5aa4134b331e23e6d4355b9",
            "placeholder": "​",
            "style": "IPY_MODEL_01d2ff0e4b9a4687b9869fe5fac16089",
            "value": "Train loss: 0.101: 100%"
          }
        },
        "e5c4b5dbeaf9496fa6d51b3377b3b944": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e6cc7003780f4f2085eb888ea3a69d45": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e725176a05704b87a0147ac9d9fbcc69": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e7261ab88f6f4d0ba467b93e857f3fa0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_08f1bb9f95184ff795b7d1d5edce9472",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3e5268ab9d64413a95bf9999a5376bf9",
            "value": 781
          }
        },
        "e7fac7d5a34346089efe5135aedb5822": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e8a6bc5e9dee4376a629a4d966c89ab5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e8b279b6d4ab46658ec7ab0dec786c23": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e9abe64c78794d9fb469f277293ee5db": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ea098e2276654d62a3f4326b309602a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "eaaa5544fb6b45968e0873026af49704": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "eb1ee4da7b93487ca44f4a47ef925622": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4dc11e04cc564e0ebaff0e5bc091d22a",
              "IPY_MODEL_7a7a32ef7e3947fc8081fdbf31269c6d",
              "IPY_MODEL_524134a8fa5e467298954ad22beb92ab"
            ],
            "layout": "IPY_MODEL_156c5fda89fc43c7941cd8bcceb23a6d"
          }
        },
        "eb5d07765390477d9503441820c92c57": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eb78842cbd2a41b79f08810246e4adb2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ebdd5f1af9684cc4ae4966c4a312cad7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_274f524ff843460b87fdef21efeabf32",
            "placeholder": "​",
            "style": "IPY_MODEL_dfe02df2573941c18cd17a6c6d3c3360",
            "value": " 781/781 [00:31&lt;00:00, 32.69it/s]"
          }
        },
        "ec4a11ff4a34403db7f53590eefa05f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4f35d75810f142aeade2f3b06e5cb699",
              "IPY_MODEL_305d94d8ab2b44f8b81d4edd9fe079fa",
              "IPY_MODEL_126edea5f38c45799959d5c0282ce914"
            ],
            "layout": "IPY_MODEL_fe6d2eddf40e493e861ec0cb4d2168d2"
          }
        },
        "ec704019e7b64d51b10f12d0d06b29be": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ee3dce92cc3e4f4b892a91b2bccf1ef2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ef2b0b2022ed48a7b9d67acdb958b6a2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "efd8220fec5d410799e70ba4593d1975": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "effd6edac09e4d92bd7419a42813e688": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f008f48ed3244f969592774a627e66ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dd37bf2d158047a3ba6b6b5701dd3761",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_38045f1d73c44939a7b35e32ba7e5ecb",
            "value": 781
          }
        },
        "f0da0adc084e4ec3a118fa8ccdbcd325": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f17fcedc467f4957abcdac2b9ce4c164": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f243d5c87dfb47a18d52264cf442c159": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f25db568553a4a3aa78f63e5795d4c2a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f28c019f68b548b58d4a083df59baeed": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_64c7ecae57af44edad6ffe9c8b60169b",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_5a3c09ae450946978ef5e34eb6cc77f3",
            "value": 781
          }
        },
        "f3053da2315c4988b316a7967e7d2a2c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f4764482aaa94b06a8683fbd22e14829": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_441dce0b30b14c3fb3408874f53f66d2",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1c6be90a12744a53a9ea107b285e04ba",
            "value": 781
          }
        },
        "f56b98e2535f48969fa64bcf4c32ed6f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f58b4867d34f497abdb22861e6a67a7f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7a7fd17e44764bf3bb1dc2d5f5e8081a",
            "placeholder": "​",
            "style": "IPY_MODEL_fa64d5f0f74e44ffa436c26830c80594",
            "value": " 781/781 [00:29&lt;00:00, 38.43it/s]"
          }
        },
        "f5dce73bce444df18b3ed1761b68a378": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f660234977c6456983ee16aedbaba5bf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_178c940ec6d5415594bd139d6dd70673",
            "placeholder": "​",
            "style": "IPY_MODEL_5de361c077604266a257e593df56dd6f",
            "value": " 781/781 [00:30&lt;00:00, 37.14it/s]"
          }
        },
        "f6635ac72ee24817903921c52d72031e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f6de9b206baf4fd280a8ef0f48c5bce5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f76e2720d2df499ca254afc5a3217ee0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ec704019e7b64d51b10f12d0d06b29be",
            "max": 781,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1ec9a65c205e445e88a9e1770a2a070d",
            "value": 781
          }
        },
        "f803f85c510f4cbba4014f4aa8be7d7f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_fcf942446de947bf985f4b5dde41dbb4",
              "IPY_MODEL_f28c019f68b548b58d4a083df59baeed",
              "IPY_MODEL_0ff80f19f250454eb2359ab9d6f7df3d"
            ],
            "layout": "IPY_MODEL_8702e5ad0948471dad7060f1c36ec6bb"
          }
        },
        "f8177954e77e4faf971a763c78133030": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_54ee3648fec94055b57fde13bd0aded5",
            "max": 9912422,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_75388f4a628240cba9fa225745edd318",
            "value": 9912422
          }
        },
        "f81e089cc735476cb5a22cc2bb26197a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f97e214116a443bab73bbf6ff941c252": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fa64d5f0f74e44ffa436c26830c80594": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fcf942446de947bf985f4b5dde41dbb4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_75495f03a2ba487c833f43728338e211",
            "placeholder": "​",
            "style": "IPY_MODEL_021b281f677a4f16b808f55f33154971",
            "value": "Train loss: 0.221: 100%"
          }
        },
        "fd73fc9d8ae54649ba27690670784b85": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fe6d2eddf40e493e861ec0cb4d2168d2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}