Creating_Poetry_With_Bidirectional_LSTMs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bFWbEb6uGbN-"
   },
   "source": [
    "# Week 4: Predicting the next word\n",
    "\n",
    "Welcome to this assignment! During this week you saw how to create a model that will predict the next word in a text sequence, now you will implement such model and train it using a corpus of Shakespeare's sonnets, while also creating some helper functions to pre-process the data.\n",
    "\n",
    "\n",
    "Let's get started!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Cp4A-ZBwSN11"
   },
   "source": [
    "_**NOTE:** To prevent errors from the autograder, please avoid editing or deleting non-graded cells in this notebook . Please only put your solutions in between the `### START CODE HERE` and `### END CODE HERE` code comments, and also refrain from adding any new cells._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "D2LlcbnTmUxi",
    "outputId": "37001250-4141-4d9f-fd86-930732edd0de"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m475.2/475.2 MB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m56.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m36.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.5/5.5 MB\u001b[0m \u001b[31m69.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m442.0/442.0 kB\u001b[0m \u001b[31m24.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "tensorstore 0.1.64 requires ml-dtypes>=0.3.1, but you have ml-dtypes 0.2.0 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "# Install packages for compatibility\n",
    "\n",
    "# NOTE: You can safely ignore errors about version incompatibility of\n",
    "# Colab-bundled packages (e.g. xarray, pydantic, etc.)\n",
    "\n",
    "!pip install tf-keras==2.15 --quiet\n",
    "!pip install tensorflow==2.15 --quiet\n",
    "!pip install keras==2.15 --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "BOwsuGQQY9OL",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.utils import to_categorical\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BTxqlHqKHzhr"
   },
   "source": [
    "For this assignment you will be using the [Shakespeare Sonnets Dataset](https://www.opensourceshakespeare.org/views/sonnets/sonnet_view.php?range=viewrange&sonnetrange1=1&sonnetrange2=154), which contains more than 2000 lines of text extracted from Shakespeare's sonnets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HgTglNRhmUxj"
   },
   "outputs": [],
   "source": [
    "!pip install gdown==5.1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "WZ4qOUzujMP6",
    "outputId": "1edba406-68a0-40e3-db7a-2d8776328ffe",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.10/dist-packages/gdown/__main__.py:132: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.\n",
      "  warnings.warn(\n",
      "Downloading...\n",
      "From: https://drive.google.com/uc?id=108jAePKK4R3BVYBbYJZ32JWUwxeMg20K\n",
      "To: /content/sonnets.txt\n",
      "100% 93.6k/93.6k [00:00<00:00, 57.7MB/s]\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# sonnets.txt\n",
    "!gdown --id 108jAePKK4R3BVYBbYJZ32JWUwxeMg20K"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "Pfd-nYKij5yY",
    "outputId": "db58e9d0-543f-4fc0-d6ff-4f2534b9d1bc",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 2159 lines of sonnets\n",
      "\n",
      "The first 5 lines look like this:\n",
      "\n",
      "from fairest creatures we desire increase,\n",
      "that thereby beauty's rose might never die,\n",
      "but as the riper should by time decease,\n",
      "his tender heir might bear his memory:\n",
      "but thou, contracted to thine own bright eyes,\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Define path for file with sonnets\n",
    "SONNETS_FILE = './sonnets.txt'\n",
    "\n",
    "# Read the data\n",
    "with open('./sonnets.txt') as f:\n",
    "    data = f.read()\n",
    "\n",
    "# Convert to lower case and save as a list\n",
    "corpus = data.lower().split(\"\\n\")\n",
    "\n",
    "print(f\"There are {len(corpus)} lines of sonnets\\n\")\n",
    "print(f\"The first 5 lines look like this:\\n\")\n",
    "for i in range(5):\n",
    "  print(corpus[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "imB15zrSNhA1"
   },
   "source": [
    "## Tokenizing the text\n",
    "\n",
    "Now fit the Tokenizer to the corpus and save the total number of words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "AAhM_qAZk0o5",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "tokenizer = Tokenizer()\n",
    "tokenizer.fit_on_texts(corpus)\n",
    "total_words = len(tokenizer.word_index) + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "77-0sA46OETa"
   },
   "source": [
    "When converting the text into sequences you can use the `texts_to_sequences` method as you have done throughout this course.\n",
    "\n",
    "In the next graded function you will need to process this corpus one line at a time. Given this, it is important to keep in mind that the way you are feeding the data unto this method affects the result. Check the following example to make this clearer.\n",
    "\n",
    "The first example of the corpus is a string and looks like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "tqhPxdeXlfjh",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "corpus[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XFMP4z11O3os"
   },
   "source": [
    "If you pass this text directly into the `texts_to_sequences` method you will get an unexpected result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "EMSEhmbzNZCE",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "tokenizer.texts_to_sequences(corpus[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pPZmZtpEPEeI"
   },
   "source": [
    "This happened because `texts_to_sequences` expects a list and you are providing a string. However a string is still and `iterable` in Python so you will get the word index of every character in the string.\n",
    "\n",
    "Instead you need to place the example whithin a list before passing it to the method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "Qmgo-vXhk4nd",
    "outputId": "9c0ee71b-580d-4f0c-dfe8-566b0dc21f06",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[34, 417, 877, 166, 213, 517]]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "tokenizer.texts_to_sequences([corpus[0]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0DU7wK-eQ5dc"
   },
   "source": [
    "Notice that you received the sequence wrapped inside a list so in order to get only the desired sequence you need to explicitly get the first item in the list like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "kpTy8WmIQ57P",
    "outputId": "fdde5e4b-cbb5-4177-a53b-4cb9d5bd9ac5",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[34, 417, 877, 166, 213, 517]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "tokenizer.texts_to_sequences([corpus[0]])[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-oqy9KjXRJ9A"
   },
   "source": [
    "## Generating n_grams\n",
    "\n",
    "Now complete the `n_gram_seqs` function below. This function receives the fitted tokenizer and the corpus (which is a list of strings) and should return a list containing the `n_gram` sequences for each line in the corpus:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "deletable": false,
    "id": "iy4baJMDl6kj",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTION: n_gram_seqs\n",
    "def n_gram_seqs(corpus, tokenizer):\n",
    "    \"\"\"\n",
    "    Generates a list of n-gram sequences\n",
    "\n",
    "    Args:\n",
    "        corpus (list of string): lines of texts to generate n-grams for\n",
    "        tokenizer (object): an instance of the Tokenizer class containing the word-index dictionary\n",
    "\n",
    "    Returns:\n",
    "        input_sequences (list of int): the n-gram sequences for each line in the corpus\n",
    "    \"\"\"\n",
    "    input_sequences = []\n",
    "\n",
    "    ### START CODE HERE\n",
    "    for line in corpus:\n",
    "        # Tokenize the line\n",
    "        token_list = tokenizer.texts_to_sequences([line])[0]\n",
    "\n",
    "        # Generate n-grams\n",
    "        for i in range(1, len(token_list)):\n",
    "            n_gram_sequence = token_list[:i+1]\n",
    "            input_sequences.append(n_gram_sequence)\n",
    "    ### END CODE HERE\n",
    "\n",
    "    return input_sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "DlKqW2pfM7G3",
    "outputId": "465a22b2-d8db-44e2-d7f7-698c4613dcfa",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n_gram sequences for first example look like this:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[[34, 417],\n",
       " [34, 417, 877],\n",
       " [34, 417, 877, 166],\n",
       " [34, 417, 877, 166, 213],\n",
       " [34, 417, 877, 166, 213, 517]]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function with one example\n",
    "first_example_sequence = n_gram_seqs([corpus[0]], tokenizer)\n",
    "\n",
    "print(\"n_gram sequences for first example look like this:\\n\")\n",
    "first_example_sequence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0HL8Ug6UU0Jt"
   },
   "source": [
    "**Expected Output:**\n",
    "\n",
    "```\n",
    "n_gram sequences for first example look like this:\n",
    "\n",
    "[[34, 417],\n",
    " [34, 417, 877],\n",
    " [34, 417, 877, 166],\n",
    " [34, 417, 877, 166, 213],\n",
    " [34, 417, 877, 166, 213, 517]]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "wtPpCcBjNc4c",
    "outputId": "e77bd581-9b97-4a1b-fefd-2cf307de4d2a",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n_gram sequences for next 3 examples look like this:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[[8, 878],\n",
       " [8, 878, 134],\n",
       " [8, 878, 134, 351],\n",
       " [8, 878, 134, 351, 102],\n",
       " [8, 878, 134, 351, 102, 156],\n",
       " [8, 878, 134, 351, 102, 156, 199],\n",
       " [16, 22],\n",
       " [16, 22, 2],\n",
       " [16, 22, 2, 879],\n",
       " [16, 22, 2, 879, 61],\n",
       " [16, 22, 2, 879, 61, 30],\n",
       " [16, 22, 2, 879, 61, 30, 48],\n",
       " [16, 22, 2, 879, 61, 30, 48, 634],\n",
       " [25, 311],\n",
       " [25, 311, 635],\n",
       " [25, 311, 635, 102],\n",
       " [25, 311, 635, 102, 200],\n",
       " [25, 311, 635, 102, 200, 25],\n",
       " [25, 311, 635, 102, 200, 25, 278]]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function with a bigger corpus\n",
    "next_3_examples_sequence = n_gram_seqs(corpus[1:4], tokenizer)\n",
    "\n",
    "print(\"n_gram sequences for next 3 examples look like this:\\n\")\n",
    "next_3_examples_sequence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EIzecMczU9UB"
   },
   "source": [
    "**Expected Output:**\n",
    "\n",
    "```\n",
    "n_gram sequences for next 3 examples look like this:\n",
    "\n",
    "[[8, 878],\n",
    " [8, 878, 134],\n",
    " [8, 878, 134, 351],\n",
    " [8, 878, 134, 351, 102],\n",
    " [8, 878, 134, 351, 102, 156],\n",
    " [8, 878, 134, 351, 102, 156, 199],\n",
    " [16, 22],\n",
    " [16, 22, 2],\n",
    " [16, 22, 2, 879],\n",
    " [16, 22, 2, 879, 61],\n",
    " [16, 22, 2, 879, 61, 30],\n",
    " [16, 22, 2, 879, 61, 30, 48],\n",
    " [16, 22, 2, 879, 61, 30, 48, 634],\n",
    " [25, 311],\n",
    " [25, 311, 635],\n",
    " [25, 311, 635, 102],\n",
    " [25, 311, 635, 102, 200],\n",
    " [25, 311, 635, 102, 200, 25],\n",
    " [25, 311, 635, 102, 200, 25, 278]]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dx3V_RjFWQSu"
   },
   "source": [
    "Apply the `n_gram_seqs` transformation to the whole corpus and save the maximum sequence length to use it later:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "laMwiRUpmuSd",
    "outputId": "4e1c9fd8-90ab-4d61-d36a-f06d55549b2f",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n_grams of input_sequences have length: 15462\n",
      "maximum length of sequences is: 11\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Apply the n_gram_seqs transformation to the whole corpus\n",
    "input_sequences = n_gram_seqs(corpus, tokenizer)\n",
    "\n",
    "# Save max length\n",
    "max_sequence_len = max([len(x) for x in input_sequences])\n",
    "\n",
    "print(f\"n_grams of input_sequences have length: {len(input_sequences)}\")\n",
    "print(f\"maximum length of sequences is: {max_sequence_len}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2OciMdmEdE9L"
   },
   "source": [
    "**Expected Output:**\n",
    "\n",
    "```\n",
    "n_grams of input_sequences have length: 15462\n",
    "maximum length of sequences is: 11\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zHY7HroqWq12"
   },
   "source": [
    "## Add padding to the sequences\n",
    "\n",
    "Now code the `pad_seqs` function which will pad any given sequences to the desired maximum length. Notice that this function receives a list of sequences and should return a numpy array with the padded sequences:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "cellView": "code",
    "deletable": false,
    "id": "WW1-qAZaWOhC",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTION: pad_seqs\n",
    "def pad_seqs(input_sequences, maxlen):\n",
    "    \"\"\"\n",
    "    Pads tokenized sequences to the same length\n",
    "\n",
    "    Args:\n",
    "        input_sequences (list of int): tokenized sequences to pad\n",
    "        maxlen (int): maximum length of the token sequences\n",
    "\n",
    "    Returns:\n",
    "        padded_sequences (array of int): tokenized sequences padded to the same length\n",
    "    \"\"\"\n",
    "    ### START CODE HERE\n",
    "    padded_sequences = pad_sequences(input_sequences, maxlen=maxlen, padding='pre')\n",
    "\n",
    "    return padded_sequences\n",
    "    ### END CODE HERE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "IqVQ0pb3YHLr",
    "outputId": "8e932c11-6f0e-4eee-ca4b-52164c11b51c",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,   0,   0,   0,  34, 417],\n",
       "       [  0,   0,   0,  34, 417, 877],\n",
       "       [  0,   0,  34, 417, 877, 166],\n",
       "       [  0,  34, 417, 877, 166, 213],\n",
       "       [ 34, 417, 877, 166, 213, 517]], dtype=int32)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function with the n_grams_seq of the first example\n",
    "first_padded_seq = pad_seqs(first_example_sequence, max([len(x) for x in first_example_sequence]))\n",
    "first_padded_seq"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Re_avDznXRnU"
   },
   "source": [
    "**Expected Output:**\n",
    "\n",
    "```\n",
    "array([[  0,   0,   0,   0,  34, 417],\n",
    "       [  0,   0,   0,  34, 417, 877],\n",
    "       [  0,   0,  34, 417, 877, 166],\n",
    "       [  0,  34, 417, 877, 166, 213],\n",
    "       [ 34, 417, 877, 166, 213, 517]], dtype=int32)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "j56_UCOBYzZt",
    "outputId": "a2055463-f6cc-42b3-b557-d2407fb1df4b",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,   0,   0,   0,   0,   0,   8, 878],\n",
       "       [  0,   0,   0,   0,   0,   8, 878, 134],\n",
       "       [  0,   0,   0,   0,   8, 878, 134, 351],\n",
       "       [  0,   0,   0,   8, 878, 134, 351, 102],\n",
       "       [  0,   0,   8, 878, 134, 351, 102, 156],\n",
       "       [  0,   8, 878, 134, 351, 102, 156, 199],\n",
       "       [  0,   0,   0,   0,   0,   0,  16,  22],\n",
       "       [  0,   0,   0,   0,   0,  16,  22,   2],\n",
       "       [  0,   0,   0,   0,  16,  22,   2, 879],\n",
       "       [  0,   0,   0,  16,  22,   2, 879,  61],\n",
       "       [  0,   0,  16,  22,   2, 879,  61,  30],\n",
       "       [  0,  16,  22,   2, 879,  61,  30,  48],\n",
       "       [ 16,  22,   2, 879,  61,  30,  48, 634],\n",
       "       [  0,   0,   0,   0,   0,   0,  25, 311],\n",
       "       [  0,   0,   0,   0,   0,  25, 311, 635],\n",
       "       [  0,   0,   0,   0,  25, 311, 635, 102],\n",
       "       [  0,   0,   0,  25, 311, 635, 102, 200],\n",
       "       [  0,   0,  25, 311, 635, 102, 200,  25],\n",
       "       [  0,  25, 311, 635, 102, 200,  25, 278]], dtype=int32)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function with the n_grams_seq of the next 3 examples\n",
    "next_3_padded_seq = pad_seqs(next_3_examples_sequence, max([len(s) for s in next_3_examples_sequence]))\n",
    "next_3_padded_seq"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3rmcDluOXcIU"
   },
   "source": [
    "**Expected Output:**\n",
    "\n",
    "```\n",
    "array([[  0,   0,   0,   0,   0,   0,   8, 878],\n",
    "       [  0,   0,   0,   0,   0,   8, 878, 134],\n",
    "       [  0,   0,   0,   0,   8, 878, 134, 351],\n",
    "       [  0,   0,   0,   8, 878, 134, 351, 102],\n",
    "       [  0,   0,   8, 878, 134, 351, 102, 156],\n",
    "       [  0,   8, 878, 134, 351, 102, 156, 199],\n",
    "       [  0,   0,   0,   0,   0,   0,  16,  22],\n",
    "       [  0,   0,   0,   0,   0,  16,  22,   2],\n",
    "       [  0,   0,   0,   0,  16,  22,   2, 879],\n",
    "       [  0,   0,   0,  16,  22,   2, 879,  61],\n",
    "       [  0,   0,  16,  22,   2, 879,  61,  30],\n",
    "       [  0,  16,  22,   2, 879,  61,  30,  48],\n",
    "       [ 16,  22,   2, 879,  61,  30,  48, 634],\n",
    "       [  0,   0,   0,   0,   0,   0,  25, 311],\n",
    "       [  0,   0,   0,   0,   0,  25, 311, 635],\n",
    "       [  0,   0,   0,   0,  25, 311, 635, 102],\n",
    "       [  0,   0,   0,  25, 311, 635, 102, 200],\n",
    "       [  0,   0,  25, 311, 635, 102, 200,  25],\n",
    "       [  0,  25, 311, 635, 102, 200,  25, 278]], dtype=int32)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "rgK-Q_micEYA",
    "outputId": "aad0e826-b561-4bb3-9edb-88f71ba880ad",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "padded corpus has shape: (15462, 11)\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Pad the whole corpus\n",
    "input_sequences = pad_seqs(input_sequences, max_sequence_len)\n",
    "\n",
    "print(f\"padded corpus has shape: {input_sequences.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "59RD1YYNc7CW"
   },
   "source": [
    "**Expected Output:**\n",
    "\n",
    "```\n",
    "padded corpus has shape: (15462, 11)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZbOidyPrXxf7"
   },
   "source": [
    "## Split the data into features and labels\n",
    "\n",
    "Before feeding the data into the neural network you should split it into features and labels. In this case the features will be the padded n_gram sequences with the last word removed from them and the labels will be the removed word.\n",
    "\n",
    "Complete the `features_and_labels` function below. This function expects the padded n_gram sequences as input and should return a tuple containing the features and the one hot encoded labels.\n",
    "\n",
    "Notice that the function also receives the total of words in the corpus, this parameter will be very important when one hot enconding the labels since every word in the corpus will be a label at least once. If you need a refresh of how the `to_categorical` function works take a look at the [docs](https://www.tensorflow.org/api_docs/python/tf/keras/utils/to_categorical)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "cellView": "code",
    "deletable": false,
    "id": "9WGGbYdnZdmJ",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTION: features_and_labels\n",
    "def features_and_labels(input_sequences, total_words):\n",
    "    \"\"\"\n",
    "    Generates features and labels from n-grams\n",
    "\n",
    "    Args:\n",
    "        input_sequences (list of int): sequences to split features and labels from\n",
    "        total_words (int): vocabulary size\n",
    "\n",
    "    Returns:\n",
    "        features, one_hot_labels (array of int, array of int): arrays of features and one-hot encoded labels\n",
    "    \"\"\"\n",
    "    ### START CODE HERE\n",
    "    # Separate features and labels\n",
    "    features = np.array(input_sequences)[:, :-1]  # All elements except the last\n",
    "    labels = np.array(input_sequences)[:, -1]     # The last element\n",
    "    # One-hot encode the labels\n",
    "    one_hot_labels = to_categorical(labels, num_classes=total_words)\n",
    "    ### END CODE HERE\n",
    "\n",
    "    return features, one_hot_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "23DolaBRaIAZ",
    "outputId": "54fcaa53-3b81-4789-de63-c394853fec4a",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "labels have shape: (5, 3211)\n",
      "\n",
      "features look like this:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[  0,   0,   0,   0,  34],\n",
       "       [  0,   0,   0,  34, 417],\n",
       "       [  0,   0,  34, 417, 877],\n",
       "       [  0,  34, 417, 877, 166],\n",
       "       [ 34, 417, 877, 166, 213]], dtype=int32)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function with the padded n_grams_seq of the first example\n",
    "first_features, first_labels = features_and_labels(first_padded_seq, total_words)\n",
    "\n",
    "print(f\"labels have shape: {first_labels.shape}\")\n",
    "print(\"\\nfeatures look like this:\\n\")\n",
    "first_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7t4yAx2UaQ43"
   },
   "source": [
    "**Expected Output:**\n",
    "\n",
    "```\n",
    "labels have shape: (5, 3211)\n",
    "\n",
    "features look like this:\n",
    "\n",
    "array([[  0,   0,   0,   0,  34],\n",
    "       [  0,   0,   0,  34, 417],\n",
    "       [  0,   0,  34, 417, 877],\n",
    "       [  0,  34, 417, 877, 166],\n",
    "       [ 34, 417, 877, 166, 213]], dtype=int32)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "GRTuLEt3bRKa",
    "outputId": "a15be75b-7bbe-432c-9d98-1f3e01bbd9e3",
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "features have shape: (15462, 10)\n",
      "labels have shape: (15462, 3211)\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Split the whole corpus\n",
    "features, labels = features_and_labels(input_sequences, total_words)\n",
    "\n",
    "print(f\"features have shape: {features.shape}\")\n",
    "print(f\"labels have shape: {labels.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xXSMK_HpdLns"
   },
   "source": [
    "**Expected Output:**\n",
    "\n",
    "```\n",
    "features have shape: (15462, 10)\n",
    "labels have shape: (15462, 3211)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ltxaOCE_aU6J"
   },
   "source": [
    "## Create the model\n",
    "\n",
    "Now you should define a model architecture capable of achieving an accuracy of at least 80%.\n",
    "\n",
    "Some hints to help you in this task:\n",
    "\n",
    "- An appropriate `output_dim` for the first layer (Embedding) is 100, this is already provided for you.\n",
    "- A Bidirectional LSTM is helpful for this particular problem.\n",
    "- The last layer should have the same number of units as the total number of words in the corpus and a softmax activation function.\n",
    "- This problem can be solved with only two layers (excluding the Embedding) so try out small architectures first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "cellView": "code",
    "deletable": false,
    "id": "XrE6kpJFfvRY",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTION: create_model\n",
    "def create_model(total_words, max_sequence_len):\n",
    "    \"\"\"\n",
    "    Creates a text generator model\n",
    "\n",
    "    Args:\n",
    "        total_words (int): size of the vocabulary for the Embedding layer input\n",
    "        max_sequence_len (int): length of the input sequences\n",
    "\n",
    "    Returns:\n",
    "        model (tf.keras Model): the text generator model\n",
    "    \"\"\"\n",
    "    model = Sequential()\n",
    "    ### START CODE HERE\n",
    "    # Embedding layer\n",
    "    model.add(Embedding(input_dim=total_words, output_dim=100, input_length=max_sequence_len-1))\n",
    "\n",
    "    # Bidirectional LSTM layer\n",
    "    model.add(Bidirectional(LSTM(150)))\n",
    "\n",
    "    # Dense output layer with softmax activation\n",
    "    model.add(Dense(total_words, activation='softmax'))\n",
    "\n",
    "    # Compile the model\n",
    "    model.compile(loss='categorical_crossentropy',\n",
    "                  optimizer='adam',\n",
    "                  metrics=['accuracy'])\n",
    "\n",
    "    ### END CODE HERE\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0IpX_Gu_gISk",
    "outputId": "be469de4-981a-4c26-fd79-b030049d157c",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "484/484 [==============================] - 37s 66ms/step - loss: 6.8929 - accuracy: 0.0220\n",
      "Epoch 2/50\n",
      "484/484 [==============================] - 33s 69ms/step - loss: 6.4393 - accuracy: 0.0321\n",
      "Epoch 3/50\n",
      "484/484 [==============================] - 31s 63ms/step - loss: 6.2041 - accuracy: 0.0408\n",
      "Epoch 4/50\n",
      "484/484 [==============================] - 30s 63ms/step - loss: 5.9427 - accuracy: 0.0533\n",
      "Epoch 5/50\n",
      "484/484 [==============================] - 30s 62ms/step - loss: 5.6411 - accuracy: 0.0638\n",
      "Epoch 6/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 5.2960 - accuracy: 0.0770\n",
      "Epoch 7/50\n",
      "484/484 [==============================] - 34s 70ms/step - loss: 4.9141 - accuracy: 0.0933\n",
      "Epoch 8/50\n",
      "484/484 [==============================] - 30s 62ms/step - loss: 4.5087 - accuracy: 0.1229\n",
      "Epoch 9/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 4.0931 - accuracy: 0.1692\n",
      "Epoch 10/50\n",
      "484/484 [==============================] - 31s 65ms/step - loss: 3.6855 - accuracy: 0.2403\n",
      "Epoch 11/50\n",
      "484/484 [==============================] - 30s 63ms/step - loss: 3.3080 - accuracy: 0.3074\n",
      "Epoch 12/50\n",
      "484/484 [==============================] - 30s 62ms/step - loss: 2.9615 - accuracy: 0.3719\n",
      "Epoch 13/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 2.6594 - accuracy: 0.4354\n",
      "Epoch 14/50\n",
      "484/484 [==============================] - 31s 65ms/step - loss: 2.3950 - accuracy: 0.4884\n",
      "Epoch 15/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 2.1613 - accuracy: 0.5372\n",
      "Epoch 16/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 1.9554 - accuracy: 0.5860\n",
      "Epoch 17/50\n",
      "484/484 [==============================] - 32s 67ms/step - loss: 1.7754 - accuracy: 0.6224\n",
      "Epoch 18/50\n",
      "484/484 [==============================] - 32s 65ms/step - loss: 1.6162 - accuracy: 0.6599\n",
      "Epoch 19/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 1.4725 - accuracy: 0.6903\n",
      "Epoch 20/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 1.3535 - accuracy: 0.7194\n",
      "Epoch 21/50\n",
      "484/484 [==============================] - 32s 67ms/step - loss: 1.2332 - accuracy: 0.7467\n",
      "Epoch 22/50\n",
      "484/484 [==============================] - 31s 65ms/step - loss: 1.1402 - accuracy: 0.7658\n",
      "Epoch 23/50\n",
      "484/484 [==============================] - 30s 63ms/step - loss: 1.0566 - accuracy: 0.7820\n",
      "Epoch 24/50\n",
      "484/484 [==============================] - 31s 65ms/step - loss: 0.9869 - accuracy: 0.7966\n",
      "Epoch 25/50\n",
      "484/484 [==============================] - 30s 63ms/step - loss: 0.9211 - accuracy: 0.8072\n",
      "Epoch 26/50\n",
      "484/484 [==============================] - 34s 70ms/step - loss: 0.8756 - accuracy: 0.8144\n",
      "Epoch 27/50\n",
      "484/484 [==============================] - 31s 63ms/step - loss: 0.8279 - accuracy: 0.8221\n",
      "Epoch 28/50\n",
      "484/484 [==============================] - 32s 66ms/step - loss: 0.7862 - accuracy: 0.8271\n",
      "Epoch 29/50\n",
      "484/484 [==============================] - 31s 63ms/step - loss: 0.7571 - accuracy: 0.8317\n",
      "Epoch 30/50\n",
      "484/484 [==============================] - 31s 63ms/step - loss: 0.7267 - accuracy: 0.8358\n",
      "Epoch 31/50\n",
      "484/484 [==============================] - 30s 62ms/step - loss: 0.7069 - accuracy: 0.8393\n",
      "Epoch 32/50\n",
      "484/484 [==============================] - 32s 66ms/step - loss: 0.6825 - accuracy: 0.8417\n",
      "Epoch 33/50\n",
      "484/484 [==============================] - 30s 61ms/step - loss: 0.6755 - accuracy: 0.8414\n",
      "Epoch 34/50\n",
      "484/484 [==============================] - 30s 63ms/step - loss: 0.6609 - accuracy: 0.8437\n",
      "Epoch 35/50\n",
      "484/484 [==============================] - 30s 62ms/step - loss: 0.6403 - accuracy: 0.8454\n",
      "Epoch 36/50\n",
      "484/484 [==============================] - 33s 69ms/step - loss: 0.6294 - accuracy: 0.8459\n",
      "Epoch 37/50\n",
      "484/484 [==============================] - 30s 63ms/step - loss: 0.6231 - accuracy: 0.8475\n",
      "Epoch 38/50\n",
      "484/484 [==============================] - 31s 63ms/step - loss: 0.6227 - accuracy: 0.8461\n",
      "Epoch 39/50\n",
      "484/484 [==============================] - 30s 63ms/step - loss: 0.6077 - accuracy: 0.8497\n",
      "Epoch 40/50\n",
      "484/484 [==============================] - 32s 66ms/step - loss: 0.6038 - accuracy: 0.8484\n",
      "Epoch 41/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 0.5929 - accuracy: 0.8489\n",
      "Epoch 42/50\n",
      "484/484 [==============================] - 30s 63ms/step - loss: 0.5926 - accuracy: 0.8481\n",
      "Epoch 43/50\n",
      "484/484 [==============================] - 32s 65ms/step - loss: 0.5890 - accuracy: 0.8489\n",
      "Epoch 44/50\n",
      "484/484 [==============================] - 32s 65ms/step - loss: 0.5906 - accuracy: 0.8470\n",
      "Epoch 45/50\n",
      "484/484 [==============================] - 30s 62ms/step - loss: 0.5811 - accuracy: 0.8487\n",
      "Epoch 46/50\n",
      "484/484 [==============================] - 34s 70ms/step - loss: 0.5761 - accuracy: 0.8483\n",
      "Epoch 47/50\n",
      "484/484 [==============================] - 31s 64ms/step - loss: 0.5735 - accuracy: 0.8488\n",
      "Epoch 48/50\n",
      "484/484 [==============================] - 30s 62ms/step - loss: 0.5709 - accuracy: 0.8489\n",
      "Epoch 49/50\n",
      "484/484 [==============================] - 31s 63ms/step - loss: 0.5725 - accuracy: 0.8498\n",
      "Epoch 50/50\n",
      "484/484 [==============================] - 30s 62ms/step - loss: 0.5794 - accuracy: 0.8465\n"
     ]
    }
   ],
   "source": [
    "# Get the untrained model\n",
    "model = create_model(total_words, max_sequence_len)\n",
    "\n",
    "# Train the model\n",
    "history = model.fit(features, labels, epochs=50, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gy72RPgly55q"
   },
   "source": [
    "**To pass this assignment, your model should achieve a training accuracy of at least 80%**. If your model didn't achieve this threshold, try training again with a different model architecture, consider increasing the number of unit in your `LSTM` layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 887
    },
    "id": "1fXTEO3GJ282",
    "outputId": "ddb0564d-2336-4e87-a5ab-fde1a9c73966",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGzCAYAAAD9pBdvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA850lEQVR4nO3deVxV1d7H8e8B4aApOCFOGGallanlwENldq+kTQ4NNypLs7pmaVpeS63UhpuYTWaalrfJsjRNLctMo7SnssyptK5Tak4BWgkoCsrZzx/rOQdRHEBgneHzfr32ay82e5/zOzv1fFt77bVdjuM4AgAAsCTMdgEAACC0EUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGgCBx++23KyEhoVTHPvbYY3K5XGVbEACcJMIIUM5cLtdJLYsWLbJdKgBY4eLZNED5euedd4r8PGXKFC1cuFBvv/12ke2XX3654uLiSv0+Bw8elMfjkdvtLvGxhw4d0qFDhxQVFVXq9weA0iKMABWsf//+mjBhgk70Vy83N1dVqlSpoKpwMhzH0YEDB1S5cmXbpQBBhcs0gB+47LLL1Lx5cy1fvlyXXnqpqlSpoocffliS9OGHH+rqq69W/fr15Xa71aRJEz355JMqKCgo8hpHjhnZsmWLXC6Xnn32Wb366qtq0qSJ3G632rZtqx9++KHIscWNGXG5XOrfv7/mzJmj5s2by+1267zzztP8+fOPqn/RokVq06aNoqKi1KRJE73yyisnPQ7lf//3f/WPf/xDjRo1ktvtVnx8vB544AHt37//qH3Xrl2rG2+8UbGxsapcubKaNm2qRx55pMg+O3bs0J133uk7X40bN9Y999yj/Pz8Y35WSXrzzTflcrm0ZcsW37aEhARdc801+uyzz9SmTRtVrlxZr7zyiiTpjTfe0N///nfVqVNHbrdb5557riZOnFjsZ/z000/VoUMHVatWTdHR0Wrbtq3effddSdLIkSMVERGhXbt2HXVcnz59VL16dR04cOCE5xEIZJVsFwDA+OOPP3TllVfqpptu0q233uq7ZPPmm2+qatWqGjRokKpWraovvvhCI0aMUHZ2tp555pkTvu67776rnJwc3X333XK5XBozZoyuu+46bdq0SREREcc99uuvv9asWbN07733qlq1aho3bpyuv/56bd26VbVq1ZIkrVy5UldccYXq1aunxx9/XAUFBXriiScUGxt7Up97xowZys3N1T333KNatWpp6dKleumll7R9+3bNmDHDt99PP/2k9u3bKyIiQn369FFCQoJ+/fVXzZ07V0899ZQkaefOnWrXrp327NmjPn36qFmzZtqxY4dmzpyp3NxcRUZGnlRNh1u3bp1uvvlm3X333frnP/+ppk2bSpImTpyo8847T127dlWlSpU0d+5c3XvvvfJ4POrXr5/v+DfffFN33HGHzjvvPA0bNkzVq1fXypUrNX/+fN1yyy267bbb9MQTT2j69Onq37+/77j8/HzNnDlT119/PZfPEPwcABWqX79+zpF/9Tp06OBIciZNmnTU/rm5uUdtu/vuu50qVao4Bw4c8G3r1auXc/rpp/t+3rx5syPJqVWrlvPnn3/6tn/44YeOJGfu3Lm+bSNHjjyqJklOZGSks3HjRt+2H3/80ZHkvPTSS75tXbp0capUqeLs2LHDt23Dhg1OpUqVjnrN4hT3+VJTUx2Xy+X89ttvvm2XXnqpU61atSLbHMdxPB6Pr92zZ08nLCzM+eGHH456Te9+xX1Wx3GcN954w5HkbN682bft9NNPdyQ58+fPP6m6O3fu7Jxxxhm+n/fs2eNUq1bNSUxMdPbv33/MupOSkpzExMQiv581a5Yjyfnyyy+Peh8g2HCZBvATbrdbvXv3Pmr74eMTcnJytHv3brVv3165ublau3btCV83JSVFNWrU8P3cvn17SdKmTZtOeGxycrKaNGni+7lFixaKjo72HVtQUKDPP/9c3bt3V/369X37nXnmmbryyitP+PpS0c+3b98+7d69WxdddJEcx9HKlSslSbt27dJXX32lO+64Q40aNSpyvPeSi8fj0Zw5c9SlSxe1adPmqPcp7a3LjRs3VufOnY9bd1ZWlnbv3q0OHTpo06ZNysrKkiQtXLhQOTk5Gjp06FG9G4fX07NnT33//ff69ddffdumTp2q+Ph4dejQoVR1A4GEMAL4iQYNGhR7GeHnn3/Wtddeq5iYGEVHRys2Nla33nqrJPm+9I7nyC9vbzD566+/Snys93jvsZmZmdq/f7/OPPPMo/Yrbltxtm7dqttvv101a9ZU1apVFRsb6/sC9n4+b/hp3rz5MV9n165dys7OPu4+pdG4ceNit3/zzTdKTk7WaaedpurVqys2NtY3zsdbtzdcnKimlJQUud1uTZ061Xf8xx9/rB49ejD/C0ICY0YAP1HcHRp79uxRhw4dFB0drSeeeEJNmjRRVFSUVqxYoSFDhsjj8ZzwdcPDw4vd7pzEjXSncuzJKCgo0OWXX64///xTQ4YMUbNmzXTaaadpx44duv3220/q85XUsb7cjxwQ7FXcf5dff/1VHTt2VLNmzfT8888rPj5ekZGRmjdvnl544YUS112jRg1dc801mjp1qkaMGKGZM2cqLy/PFzqBYEcYAfzYokWL9Mcff2jWrFm69NJLfds3b95ssapCderUUVRUlDZu3HjU74rbdqTVq1dr/fr1euutt9SzZ0/f9oULFxbZ74wzzpAkrVmz5pivFRsbq+jo6OPuIxX2DO3Zs0fVq1f3bf/tt99OWK/X3LlzlZeXp48++qhI79GXX35ZZD/vJa41a9acsKeoZ8+e6tatm3744QdNnTpVF1xwgc4777yTrgkIZFymAfyYt2fi8J6I/Px8vfzyy7ZKKiI8PFzJycmaM2eOdu7c6du+ceNGffrppyd1vFT08zmOoxdffLHIfrGxsbr00kv1+uuva+vWrUV+5z02LCxM3bt319y5c7Vs2bKj3su7nzcgfPXVV77f7du3T2+99dYJ6z1e3VlZWXrjjTeK7NepUydVq1ZNqampR92ee2Tv0pVXXqnatWvr6aef1uLFi+kVQUihZwTwYxdddJFq1KihXr16acCAAXK5XHr77bfL7DJJWXjssce0YMECXXzxxbrnnntUUFCg8ePHq3nz5lq1atVxj23WrJmaNGmiwYMHa8eOHYqOjtYHH3xQ7HiWcePG6ZJLLtGFF16oPn36qHHjxtqyZYs++eQT3/uMGjVKCxYsUIcOHdSnTx+dc845+v333zVjxgx9/fXXql69ujp16qRGjRrpzjvv1IMPPqjw8HC9/vrrio2NPSroHEunTp0UGRmpLl266O6779bevXs1efJk1alTR7///rtvv+joaL3wwgu666671LZtW91yyy2qUaOGfvzxR+Xm5hYJQBEREbrppps0fvx4hYeH6+abbz6pWoBgQM8I4Mdq1aqljz/+WPXq1dOjjz6qZ599VpdffrnGjBljuzSf1q1b69NPP1WNGjU0fPhwvfbaa3riiSfUsWPHE86PERERoblz56pVq1ZKTU3V448/rrPOOktTpkw5at+WLVvqu+++06WXXqqJEydqwIAB+uCDD9S1a1ffPg0aNND333+vG264QVOnTtWAAQM0ZcoUXXbZZb7ZbCMiIjR79mw1adJEw4cP17hx43TXXXcVmePjRJo2baqZM2fK5XJp8ODBmjRpkvr06aOBAwcete+dd96pjz76SNHR0XryySc1ZMgQrVixoti7jbyXqjp27Kh69eqddD1AoGM6eADlonv37vr555+1YcMG26UEjB9//FGtWrXSlClTdNttt9kuB6gw9IwAOGVHTt2+YcMGzZs3T5dddpmdggLU5MmTVbVqVV133XW2SwEqFGNGAJyyM844Q7fffrvOOOMM/fbbb5o4caIiIyP10EMP2S4tIMydO1e//PKLXn31VfXv31+nnXaa7ZKACsVlGgCnrHfv3vryyy+Vnp4ut9utpKQkjRo1ShdeeKHt0gJCQkKCMjIy1LlzZ7399tuqVq2a7ZKACkUYAQAAVjFmBAAAWEUYAQAAVgXEAFaPx6OdO3eqWrVqPDQKAIAA4TiOcnJyVL9+fYWFHbv/IyDCyM6dOxUfH2+7DAAAUArbtm1Tw4YNj/n7gAgj3pHl27ZtU3R0tOVqAADAycjOzlZ8fPwJ7xALiDDivTQTHR1NGAEAIMCcaIgFA1gBAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWBcSD8gAAgSsvT9q4UVq71iwHDkgxMWaJji669rajoqSDB4+95OdLHo/Z78jF7ZbCivlf7YICaf9+sxw4ULRdrZpUp45Us2bxx55IQYG0Z4+UmyuFh0uVKh299ra9+x/vsx06ZD6fxyM5jlm87cO3eT9Hbu6x11WrSmefLZ11lllq1Dil/5zlgjACACHM+4X2119m+fPPwvXh7ZwcqUoVExaqVy+69rarVZN27CgMHd5l0ybz5VuRIiNNMImMNF/u+/ebL/oTCQuTYmPNUqdO0UWS/vij+OWvv8y5DAS1axcGk7POKgwqzZpJlSvbqYkwAgABxuMxX4CZmYWh4c8/zbYj29nZ5ss4L69wfXg7P79iao6ONl92zZqZ0JKVZWorbl1caAgPlyIiTLiIiDBLWJj5DN7eAY+ncP/8/ON/tshI88Xr7U3JyTHny+ORMjLMUhqRkeY1Dh0q+bHezxUZWdiL4nKZz1nc2uUytVeubILi4evD23/+KW3YYJbff5d27zbLkiVF33/+fKlz59J97lNFGAGAcubxmOCwfbuUnm62RUSYLxzv+vB2eLj5stixo/hl586T+7/8kggLM933NWua9eHtmjVNmMjNNZcisrLMcmQ7O1uKiysMHeecU9iuW9d8eZ6It6cmL6/wyzki4uSOPXTIHHv4kpdXGDy8i9tdeLnkcAcPmvOemVl02bXLhBOXS6pV69hLzZqmVu/n8IaSggKz9rYLCop+toiIwuBR3vbuNZfMNmyQ1q8vDCkbNpjeEVtcjuP/HUvZ2dmKiYlRVlaWoqOjbZcDAEXs3Wv+Yd+40QSOHTvM2rvs3Fm6/1M+HpfLBAXvl6B3ffhSq5YJEW63+UJ2u4tvR0WZ3orSjJUAjudkv7/pGQGAk+DxSNu2SevWmWXt2sL1jh0nPj4szPQO1KtngoT3/5QPHiy69i41a0oNGhx7qVev8P/CgUBHGAGAI/z5p/TTT9KPP5rlp5+kX34x4xKOJTbWdHM3aiQ1bGgCQ8OGhUvduuYSDICj8VcDQMjyeMy18lWrioaP7duL3z8iQjrzTKlpU7M0a1bYrlmzQksHggphBEBI8AaP5cvNsmyZtHKluYuiOAkJUsuWhUvz5tIZZ9C7AZQH/loBCDqOI23ZIn33nQkdxwseUVFSixZFg8f555u5MwBUDMIIgICXm2sCx5IlZvnuu+LniahcWWrVSmrdunA55xx6OwDb+CsIIOD89ZeZoOnbb034+PHHo2+djYiQLrxQateuMHg0a0bwAPwRfy0BBITsbOnDD6Xp06UFC46e9Kt+fSkpqXC58EJzCQaA/yOMAPBbe/dKH39sAsinn5rZNL2aN5c6diwMH/HxFTODJYCyRxgB4Fdyc03wmD7dBJHD5/Zo1kxKSZFuvFE691x7NQIoW4QRANb99ZcJHrNmSZ99VjSANGliAkhKirnLhd4PIPgQRgBYsWOHGQMye7a0aFHRAainn256P1JSzNgPAggQ3AgjACrMb79J06aZAPL990V/17y5dO21ZmnVigAChJJSPaNxwoQJSkhIUFRUlBITE7V06dLj7j927Fg1bdpUlStXVnx8vB544AEdOHCgVAUDCDw//ST16GEuuQwdWhhEkpKkMWPMzKirV0tPPCFdcAFBBAg1Je4ZmT59ugYNGqRJkyYpMTFRY8eOVefOnbVu3TrVqVPnqP3fffddDR06VK+//rouuugirV+/XrfffrtcLpeef/75MvkQAPyP40hffy2NHi3Nm1e4/W9/M5dgunUzT54FAJfjOE5JDkhMTFTbtm01fvx4SZLH41F8fLzuu+8+DR069Kj9+/fvr//+979KS0vzbfvXv/6l77//Xl9//fVJvWd2drZiYmKUlZWl6OjokpQLoIJ5PGYw6ujRZkIySQoLk/7xD2nIENPzASA0nOz3d4ku0+Tn52v58uVKTk4ufIGwMCUnJ2uJ91+dI1x00UVavny571LOpk2bNG/ePF111VXHfJ+8vDxlZ2cXWQD4t4MHpSlTzB0v3bqZIOJ2S3ffLa1bZ8aKEEQAFKdEl2l2796tgoICxcXFFdkeFxentWvXFnvMLbfcot27d+uSSy6R4zg6dOiQ+vbtq4cffviY75OamqrHH3+8JKUBsGjBAqlvX2nzZvNzdLR0773SwIFS3bp2awPg/0o1gLUkFi1apFGjRunll1/WihUrNGvWLH3yySd68sknj3nMsGHDlJWV5Vu2bdtW3mUCKIU//pB69ZI6dzZBJC7OXJ7ZulVKTSWIADg5JeoZqV27tsLDw5VxxOMwMzIyVPcY/+oMHz5ct912m+666y5J0vnnn699+/apT58+euSRRxQWdnQecrvdcrvdJSkNQAVyHOn996UBA6TMTHP3y4AB0r//LVWtars6AIGmRD0jkZGRat26dZHBqB6PR2lpaUpKSir2mNzc3KMCR3h4uCSphGNnAfiB7dvNmJCbbjJB5NxzpW++kcaOJYgAKJ0S39o7aNAg9erVS23atFG7du00duxY7du3T71795Yk9ezZUw0aNFBqaqokqUuXLnr++ed1wQUXKDExURs3btTw4cPVpUsXXygB4P88HmnyZOmhh8wTdCMipIcfloYNMwNVAaC0ShxGUlJStGvXLo0YMULp6elq1aqV5s+f7xvUunXr1iI9IY8++qhcLpceffRR7dixQ7GxserSpYueeuqpsvsUAMrV+vVSnz7S4sXm58RE6bXXpPPOs1sXgOBQ4nlGbGCeEcAOx5EmTJAefFA6cECqUkUaNUrq31+iYxPAiZzs9zfPpgFQrIwM6Y47CmdP7dRJeuUVKSHBalkAglC539oLIPDMmye1aGHWUVHS+PHS/PkEEQDlg54RAD7795sp2196yfx8/vnSe+8xNgRA+aJnBIAk89Tcdu0Kg8j990tLlxJEAJQ/wggQ4hxHevFFqW1bac0aM4vqp59KL7xgLtEAQHnjMg0QwjIzzXTu8+ebn6+5Rnr9dSk21m5dAEILYQQIUWvWSF26SFu2mB6Q5583D7tzuWxXBiDUEEaAEDR/vnTjjVJOjnTmmdKcOYwNAWAPY0aAEPPSS9LVV5sg0qGD9N13BBEAdhFGgBBx6JCZOXXAAPOcmTvukBYskGrVsl0ZgFDHZRogBGRlmcsyCxaYMSFPPy0NHsz4EAD+gTACBLlNm8xA1V9+Mc+WmTpV6t7ddlUAUIgwAgSxr7+Wrr1W2r1batBAmjtXuuAC21UBQFGMGQGC1MyZUseOJoi0bm1mUyWIAPBHhBEgCH37rdSjh5SfL113nfTVV1L9+rarAoDicZkGCDK//WYuzeTnm/WMGVIY/9sBwI/xTxQQRPbulbp2NdO8t2wpTZlCEAHg//hnCggSHo90223STz+Zh9199JFUtartqgDgxAgjQJB49FEzrbvbLc2eLTVqZLsiADg5hBEgCLz9tpSaatqvvSYlJdmtBwBKgjACBLglS6S77jLtYcPMXTQAEEgII0AA27rVzKaan2/W//637YoAoOQII0CAOvLOmbff5s4ZAIGJf7qAAOS9c+bHH6U6dbhzBkBgI4wAAWj4cHPnTGSkWXPnDIBARhgBAszs2dKoUabNnTMAggFhBAgg69dLvXqZ9gMPSLfearceACgLhBEgQOzbJ11/vZSTI11yifT007YrAoCyQRgBAoDjSHffLa1ZI9WtK73/vhQRYbsqACgbhBEgALz8sjR1qhQeboJIvXq2KwKAskMYAfzcd9+Z8SGSNGaM1L693XoAoKwRRgA/lpkp3XCDdPCgWXtDCQAEE8II4KcOHZJuvlnasUNq2lR6/XXJ5bJdFQCUPcII4KeGD5e++EI67TRp1iypWjXbFQFA+SCMAH7oww+l0aNN+7XXpHPPtVsPAJQnwgjgZzZskHr2NO2BA6WUFLv1AEB5I4wAfiQ310xslp0tXXyx9MwztisCgPJHGAH8yIAB0urV5km8TGwGIFQQRgA/MXWqGR/icknvvivVr2+7IgCoGIQRwA+sXy/17Wvaw4dLHTvarQcAKhJhBLDswAHpxhulvXulyy6TRoywXREAVCzCCGDZAw9IP/4oxcYWPn8GAEIJYQSw6P33pUmTzDiRd95hnAiA0EQYASzZuFG66y7THjZM6tTJbj0AYAthBLAgL89MZpaTI11yifT447YrAgB7CCOABYMHSytWSLVqSe+9J1WqZLsiALCHMAJUsA8+kMaPN+0pU6SGDe3WAwC2EUaACrRpk3Tnnab94IPSVVfZrQcA/AFhBKgg+fnSTTdJWVlSUpL01FO2KwIA/0AYASrImDHSDz9INWpI06bx3BkA8CKMABVg8+bCnpDx46VGjezWAwD+hDACVICBA82073/7m3TzzbarAQD/QhgBytncuWapVMn0irhctisCAP9CGAHKUW6uNGCAaQ8aJJ17rt16AMAfEUaAcpSaKm3ZYuYSGT7cdjUA4J8II0A52bDB3EEjSWPHSlWrWi0HAPwWYQQoB44j9e9v5hbp3Fm67jrbFQGA/yKMAOXggw+kBQukyEjppZcYtAoAx0MYAcrY3r3S/feb9pAh0llnWS0HAPweYQQoY088Ie3YITVuLA0bZrsaAPB/hBGgDP38s/TCC6Y9bpxUubLdegAgEBBGgDLiOFK/ftKhQ1LXrtI119iuCAACA2EEKCPvvSctXmx6Q1580XY1ABA4CCNAGcjKkv71L9N+5BEpIcFqOQAQUAgjQBkYPlxKT5fOPlsaPNh2NQAQWAgjwClatsw8AE+SJkyQ3G679QBAoCGMAKfg0CHp7rvN4NUePaTkZNsVAUDgIYwAp2D8eGnFCql6dem552xXAwCBiTAClNL27YVP4h0zRoqLs1sPAAQqwghQSgMGmKnfL75YuvNO29UAQOAqVRiZMGGCEhISFBUVpcTERC1duvS4++/Zs0f9+vVTvXr15Ha7dfbZZ2vevHmlKhjwBx99JM2eLVWqJE2aJIUR6wGg1CqV9IDp06dr0KBBmjRpkhITEzV27Fh17txZ69atU506dY7aPz8/X5dffrnq1KmjmTNnqkGDBvrtt99UvXr1sqgfqHB790r9+5v24MFS8+Z26wGAQOdyHMcpyQGJiYlq27atxv//vYwej0fx8fG67777NHTo0KP2nzRpkp555hmtXbtWERERpSoyOztbMTExysrKUnR0dKleAygrgwebwaqNG0tr1khVqtiuCAD808l+f5eoczk/P1/Lly9X8mH3L4aFhSk5OVlLliwp9piPPvpISUlJ6tevn+Li4tS8eXONGjVKBQUFx3yfvLw8ZWdnF1kAf7BqlTR2rGlPmEAQAYCyUKIwsnv3bhUUFCjuiNsG4uLilJ6eXuwxmzZt0syZM1VQUKB58+Zp+PDheu655/Tvf//7mO+TmpqqmJgY3xIfH1+SMoFyUVBg5hQpKJBuvFG68krbFQFAcCj3YXcej0d16tTRq6++qtatWyslJUWPPPKIJk2adMxjhg0bpqysLN+ybdu28i4TOKFJk6SlS6Xo6MLeEQDAqSvRANbatWsrPDxcGRkZRbZnZGSobt26xR5Tr149RUREKDw83LftnHPOUXp6uvLz8xUZGXnUMW63W27m1IYf2blTevhh0x41SqpXz249ABBMStQzEhkZqdatWystLc23zePxKC0tTUlJScUec/HFF2vjxo3yeDy+bevXr1e9evWKDSKAP3rgASk7W2rXTurb13Y1ABBcSnyZZtCgQZo8ebLeeust/fe//9U999yjffv2qXfv3pKknj17atiwYb7977nnHv35558aOHCg1q9fr08++USjRo1Sv379yu5TAOXo00+l99+XwsOlV14xawBA2SnxPCMpKSnatWuXRowYofT0dLVq1Urz58/3DWrdunWrwg6bASo+Pl6fffaZHnjgAbVo0UINGjTQwIEDNWTIkLL7FEA58XjMrbySdP/9UqtWNqsBgOBU4nlGbGCeEdgyd67UtasZtLp1qxQTY7siAAgc5TLPCBBqnn7arPv2JYgAQHkhjADH8M03ZomMlAYOtF0NAAQvwghwDN5ekdtuk+rXt1sLAAQzwghQjJ9/NuNFXC7pwQdtVwMAwY0wAhTj2WfNunt3qWlTq6UAQNAjjABH2L5dmjrVtLkDHQDKH2EEOMILL0gHD0odOkiJibarAYDgRxgBDvPXX9Krr5o2vSIAUDEII8BhXn5Z2rtXOv986YorbFcDAKGBMAL8v/37pRdfNO2HHjJ30gAAyh9hBPh/b74p7dolNWokpaTYrgYAQgdhBJB06FDh7bz/+pcUEWG3HgAIJYQRQNKsWdKmTVKtWtKdd9quBgBCC2EEIc9xCqd+799fOu00u/UAQKghjCDkpaVJK1ZIlSubMAIAqFiEEYQ8b6/IXXdJtWvbrQUAQhFhBCFt+XLp88+l8HBp0CDb1QBAaCKMIKSNGWPWKSlSQoLVUgAgZBFGELJ+/VWaOdO0H3rIbi0AEMoIIwhZzz0neTxm2veWLW1XAwChizCCkJSZKb3xhmnzQDwAsIswgpA0bpx04IDUrp3UoYPtagAgtBFGEHJycqQJE0x7yBAeiAcAthFGEHImT5b27JHOPlvq1s12NQAAwghCSn6+9MILpj14sJlfBABgF2EEIeW996Tt26W6daXbbrNdDQBAIowghHg8hZOc3X+/FBVltRwAwP8jjCBkfPKJ9MsvUnS01Lev7WoAAF6EEYQM7wPx+vaVYmLs1gIAKEQYQUj45huzREaaSzQAAP9BGEFI8I4V6dlTqlfPbi0AgKIIIwh6v/wiffSRmdxs8GDb1QAAjkQYQdB75hmz7t5datrUaikAgGIQRhDUtm+Xpk41bR6IBwD+iTCCoPbCC9LBg+ZheImJtqsBABSHMIKg9ddf0quvmja9IgDgvwgjCFoTJ0p790otWkhXXGG7GgDAsRBGEJQOHJBefNG0H3rI3EkDAPBPhBEEpffekzIzpUaNpBtvtF0NAOB4CCMIOo4jvfSSaffrJ0VE2K0HAHB8hBEEnW++kVaulCpXlu66y3Y1AIATIYwg6IwbZ9Y9ekg1a9qtBQBwYoQRBJXt26VZs0z7vvvs1gIAODmEEQSViROlggIzyVmLFrarAQCcDMIIgsaBA4WTnA0YYLcWAMDJI4wgaEybJu3ebW7n7drVdjUAgJNFGEFQcJzCgav33itVqmS3HgDAySOMICh8+625nTcqitt5ASDQEEYQFA6/nbdWLbu1AABKhjCCgLd9u/TBB6bN7bwAEHgIIwh4kyaZ23kvvVRq2dJ2NQCAkiKMIKAdOCC98oppczsvAAQmwggC2vTp5nbe+HipWzfb1QAASoMwgoDF7bwAEBwIIwhYS5ZIK1ZwOy8ABDrCCAKWt1fklluk2rXt1gIAKD3CCALSjh3SzJmmze28ABDYCCMISN7bedu3l1q1sl0NAOBUEEYQcLidFwCCC2EEAWfWLGnXLqlhQ6l7d9vVAABOFWEEAeedd8y6d29u5wWAYEAYQUDJzJQWLDDtW2+1WwsAoGwQRhBQpk83A1fbtpXOPtt2NQCAskAYQUCZOtWse/SwWwcAoOwQRhAwNm6Uvv9eCguTUlJsVwMAKCuEEQQMb6/I5ZdLdevarQUAUHYIIwgIjsMlGgAIVoQRBIQffpA2bJCqVJGuvdZ2NQCAskQYQUDw9op06yZVrWq3FgBA2SKMwO8dOiRNm2baXKIBgOBTqjAyYcIEJSQkKCoqSomJiVq6dOlJHTdt2jS5XC51Zw5vlMDnn5vJzmrXljp1sl0NAKCslTiMTJ8+XYMGDdLIkSO1YsUKtWzZUp07d1ZmZuZxj9uyZYsGDx6s9u3bl7pYhCbvJZqUFCkiwm4tAICyV+Iw8vzzz+uf//ynevfurXPPPVeTJk1SlSpV9Prrrx/zmIKCAvXo0UOPP/64zjjjjFMqGKFl3z5p9mzTZvp3AAhOJQoj+fn5Wr58uZKTkwtfICxMycnJWrJkyTGPe+KJJ1SnTh3deeedJ/U+eXl5ys7OLrIgNH34oQkkTZpIiYm2qwEAlIcShZHdu3eroKBAcXFxRbbHxcUpPT292GO+/vprvfbaa5o8efJJv09qaqpiYmJ8S3x8fEnKRBDxXqK55RbJ5bJbCwCgfJTr3TQ5OTm67bbbNHnyZNWuXfukjxs2bJiysrJ8y7Zt28qxSvirXbukzz4zbe6iAYDgVakkO9euXVvh4eHKyMgosj0jI0N1i5mf+9dff9WWLVvUpUsX3zaPx2PeuFIlrVu3Tk2aNDnqOLfbLbfbXZLSEIS8T+ht00Zq2tR2NQCA8lKinpHIyEi1bt1aaWlpvm0ej0dpaWlKSko6av9mzZpp9erVWrVqlW/p2rWr/va3v2nVqlVcfsFxeS/RMHAVAIJbiXpGJGnQoEHq1auX2rRpo3bt2mns2LHat2+fevfuLUnq2bOnGjRooNTUVEVFRal58+ZFjq9evbokHbUdONyvv0rffccTegEgFJQ4jKSkpGjXrl0aMWKE0tPT1apVK82fP983qHXr1q0KC2NiV5wab69IcjJP6AWAYOdyHMexXcSJZGdnKyYmRllZWYqOjrZdDsqZ40jNmknr10tTpki33Wa7IgBAaZzs9zddGPA7y5ebIFK5ssSTAwAg+BFG4Hfeecesu3eXqlWzWgoAoAIQRuBXeEIvAIQewgj8SlqalJHBE3oBIJQQRuBX3n3XrG+8kSf0AkCoIIzAb+zfL82aZdpcogGA0EEYgd/4+GNp714pIUEqZkJfAECQIozAb3gv0dx8M0/oBYBQQhiBX/jrL2nePNO+5Ra7tQAAKhZhBH5h1iwpP186/3yJxxYBQGghjMAveC/R0CsCAKGHMALrdu6UvvzStG+6yW4tAICKRxiBddOnm4fjXXyxuZMGABBaCCOwjks0ABDaCCOwav16adkyKTxc+sc/bFcDALCBMAKr3nvPrDt1kmJj7dYCALCDMAJrHIdLNAAAwggsWrHCXKapXFnq1s12NQAAWwgjsMbbK9K1q1Stmt1aAAD2EEZgRUGBNG2aaXOJBgBCG2EEVnz1lZnsrEYN6YorbFcDALCJMAIrvJdobrhBioy0WwsAwC7CCCpcXp40c6Zpc4kGAEAYQYWbP1/as0dq0EBq3952NQAA2wgjqHDeSzQ33WRmXgUAhDbCCCpUTo700UemzSUaAIBEGEEFmzNHOnBAatpUuuAC29UAAPwBYQQV6vDp310uu7UAAPwDYQQVJjNTWrjQtG++2W4tAAD/QRhBhZkxw8y82ratdNZZtqsBAPgLwggqjPcSDb0iAIDDEUZQIbZtk7791owTSUmxXQ0AwJ8QRlAhZsww6/btpfr17dYCAPAvhBFUiOnTzfrGG+3WAQDwP4QRlLstW6SlS6WwMOn6621XAwDwN4QRlLv33zfrDh2kunXt1gIA8D+EEZQ7bxhh4CoAoDiEEZSrjRul5cvNA/Guu852NQAAf0QYQbny3kXz979LsbF2awEA+CfCCMoVd9EAAE6EMIJys26d9OOPUqVK0rXX2q4GAOCvCCMoN96Bq8nJUq1admsBAPgvwgjKDXfRAABOBmEE5eKXX6Q1a6SICKlbN9vVAAD8GWEE5cLbK9K5s1Sjht1aAAD+jTCCMuc4hXfRcIkGAHAihBGUudWrpbVrJbdb6trVdjUAAH9HGEGZ816iufJKKTrabi0AAP9HGEGZOvwSDROdAQBOBmEEZWrVKvM8mqgoqUsX29UAAAIBYQRlytsrcvXVUtWqdmsBAAQGwgjKjOMw0RkAoOQIIygzy5ZJmzdLVapIV11luxoAQKAgjKDMeHtFunSRTjvNbi0AgMBBGEGZOPwSDXfRAABKgjCCMvH999LWrWbQ6pVX2q4GABBICCMoE967aLp2lSpXtlsLACCwEEZwyjweacYM0+YuGgBASRFGcMp++EHascNcounUyXY1AIBAQxjBKZs1y6yvvtrMvAoAQEkQRnBKHKcwjFx3nd1aAACBiTCCU/Lzz+ZZNG43d9EAAEqHMIJTMnu2WV9+uVStmt1aAACBiTCCU8IlGgDAqSKMoNQ2b5ZWrZLCwswU8AAAlAZhBKXmvUTToYNUu7bdWgAAgYswglLzXqK59lq7dQAAAhthBKWSni59+61pd+9utRQAQIAjjKBUPvzQzDHSrp0UH2+7GgBAICOMoFS840W4RAMAOFWlCiMTJkxQQkKCoqKilJiYqKVLlx5z38mTJ6t9+/aqUaOGatSooeTk5OPuD/+3Z4+Ulmba3NILADhVJQ4j06dP16BBgzRy5EitWLFCLVu2VOfOnZWZmVns/osWLdLNN9+sL7/8UkuWLFF8fLw6deqkHTt2nHLxsOOTT6RDh6Rzz5XOPtt2NQCAQOdyHMcpyQGJiYlq27atxo8fL0nyeDyKj4/Xfffdp6FDh57w+IKCAtWoUUPjx49Xz549T+o9s7OzFRMTo6ysLEVHR5ekXJSD6683d9I8+qj05JO2qwEA+KuT/f4uUc9Ifn6+li9fruTk5MIXCAtTcnKylixZclKvkZubq4MHD6pmzZrH3CcvL0/Z2dlFFviH3Fxp/nzTZrwIAKAslCiM7N69WwUFBYqLiyuyPS4uTunp6Sf1GkOGDFH9+vWLBJojpaamKiYmxrfEc7uG31iwwASS00+XLrjAdjUAgGBQoXfTjB49WtOmTdPs2bMVFRV1zP2GDRumrKws37Jt27YKrBLHc/izaFwuu7UAAIJDpZLsXLt2bYWHhysjI6PI9oyMDNWtW/e4xz777LMaPXq0Pv/8c7Vo0eK4+7rdbrnd7pKUhgpw8KA0d65pc4kGAFBWStQzEhkZqdatWyvNe1+nzADWtLQ0JSUlHfO4MWPG6Mknn9T8+fPVpk2b0lcLqxYtMrf11qkjXXSR7WoAAMGiRD0jkjRo0CD16tVLbdq0Ubt27TR27Fjt27dPvXv3liT17NlTDRo0UGpqqiTp6aef1ogRI/Tuu+8qISHBN7akatWqqlq1ahl+FJQ370Rn3bpJ4eF2awEABI8Sh5GUlBTt2rVLI0aMUHp6ulq1aqX58+f7BrVu3bpVYWGFHS4TJ05Ufn6+brjhhiKvM3LkSD322GOnVj0qjMcjzZlj2kx0BgAoSyWeZ8QG5hmxb8kSc2kmOlratUuKjLRdEQDA35XLPCMIXd67aK65hiACAChbhBGckOMUvaUXAICyRBjBCa1eLW3aJEVFSVdcYbsaAECwIYzghLy9Ip07S6edZrcWAEDwIYzghLy39DLRGQCgPBBGcFyrV0s//WTmFenSxXY1AIBgRBjBcY0aZdbXXisd50HLAACUGmEEx7R2rTR9umk/+qjdWgAAwYswgmMaNcrc1tu9u9Sype1qAADBijCCYm3cKE2datr0igAAyhNhBMVKTTXPo7n6aql1a9vVAACCGWEER9m8WZoyxbSHD7dbCwAg+BFGcJTRo6VDh6ROnaTERNvVAACCHWEERWzbJr3xhmnTKwIAqAiEERTx9NPSwYPS3/4mXXKJ7WoAAKGAMAKfnTul//zHtOkVAQBUFMIIfJ55RsrLMz0il11muxoAQKggjECSlJEhvfKKaQ8fLrlcdusBAIQOwggkSc89J+3fb+6eufxy29UAAEIJYQTavVt6+WXTplcEAFDRCCPQCy9I+/ZJF14oXXWV7WoAAKGGMBLi/vpLeukl06ZXBABgA2EkxL34opSTI7VoIXXtarsaAEAoIoyEsKwsE0Yk82TeMP40AAAs4OsnRDmOdN990p490jnnSNdfb7siAECoIoyEqP/8R3r7bSk8XJo0iV4RAIA9fAWFoBUrTK+IJD31lHTppXbrAQCENsJIiNmzR/rHP8y07126SA8+aLsiAECoI4yEEMeReveWNm2SEhKkt97i8gwAwD6+ikLI889Lc+ZIkZHSjBlSjRq2KwIAgDASMr7+WhoyxLTHjpXatLFaDgAAPoSREJCZKaWkSAUF0i23SH372q4IAIBChJEg5w0gO3ea+UReeYUp3wEA/oUwEuSeeEJKS5OqVJFmzpSqVrVdEQAARRFGgthnn0lPPmnar74qnXuu3XoAACgOYSRIbd4s9ehhbuft29e0AQDwR4SRIDR7trlb5o8/pAsvlF54wXZFAAAcG2EkiOTmml6Q666T/vxTat3aBJOoKNuVAQBwbISRILFqlQkfr7xifn7oIenbb6VGjayWBQDACVWyXQBOjccjjRtnJjTLz5fq1ZOmTJGSk21XBgDAySGMBLCMDOn226X5883PXbtKr70m1a5ttSwAAEqEyzQBat48qUULE0SioqSXXzbPnSGIAAACDT0jAWb3bumRR8y8IZJ0/vnSe+9J551nty4AAEqLnpEAcfCgGRty1lmFQWTAAGnpUoIIACCw0TMSAD7/XBo4UPrlF/Nzy5YmmFx6qd26AAAoC/SM+LFNm6Rrr5Uuv9wEkVq1pEmTpOXLCSIAgOBBz4gf2rtXSk2VnntOysuTwsOlfv2kxx6TatSwXR0AAGWLMOJH8vOld981A1R37jTbkpOlsWMZFwIACF6EET+QkyP95z/S889L27ebbY0bm5+7dZNcLrv1AQBQnggjFmVkmIGoL78s7dljtsXFSYMGmTtleKYMACAUEEYs2LDBjAd5800zJkSSzj5bevBB6dZbCSEAgNBCGKlAy5ZJo0dLs2ZJjmO2JSaa58p07WoGqgIAEGoIIxVg40Zp2DBp5szCbVdfbZ6s2749Y0IAAKGNMFKOdu+WnnxSmjjRzKDqckk9epgQcv75tqsDAMA/EEbKwf79ZmBqaqqUlWW2XXGFNGYMIQQAgCMRRsqQxyNNnWrmCdm2zWxr1Up65hkzXwgAADgaYaQMOI6UlmYuv6xcabY1bCg99ZS5OyaMSfcBADgmwsgpyMiQ3nlHev31wofYRUebwaoDB0qVK9utDwCAQEAYKaFDh6RPPzUB5OOPzc+SmRvkn/+Uhg+XYmPt1ggAQCAhjJyktWtNAHn7bSk9vXB7YqJ0xx1SSooUE2OvPgAAAhVh5DgOHpRmzJDGj5eWLCncHhsr9ewp9e7NA+wAADhVhJFi7NkjTZ5sbs/1PrguPFy66irTC3L11VJEhNUSAQAIGoSRw2zeLL34ovTaa9LevWZbXJzUv790111S3bp26wMAIBgRRiR99515cN2sWWauEElq3tw8PfeWWyS32259AAAEs5ANIx6PNHu2CSGHjwfp1MmEkE6deGYMAAAVIWTDiOOYp+X++qsUGWmeGfPAA0zXDgBARQvZMBIebuYE2bDBjAlhPAgAAHaEbBiRpF69bFcAAAB4agoAALCKMAIAAKwijAAAAKtKFUYmTJighIQERUVFKTExUUuXLj3u/jNmzFCzZs0UFRWl888/X/PmzStVsQAAIPiUOIxMnz5dgwYN0siRI7VixQq1bNlSnTt3VmZmZrH7f/vtt7r55pt15513auXKlerevbu6d++uNWvWnHLxAAAg8Lkcx3FKckBiYqLatm2r8ePHS5I8Ho/i4+N13333aejQoUftn5KSon379unjjz/2bfuf//kftWrVSpMmTSr2PfLy8pSXl+f7OTs7W/Hx8crKylJ0dHRJygUAAJZkZ2crJibmhN/fJeoZyc/P1/Lly5WcnFz4AmFhSk5O1pLDpzE9zJIlS4rsL0mdO3c+5v6SlJqaqpiYGN8SHx9fkjIBAEAAKVEY2b17twoKChQXF1dke1xcnNLT04s9Jj09vUT7S9KwYcOUlZXlW7Zt21aSMgEAQADxy0nP3G633DydDgCAkFCinpHatWsrPDxcGRkZRbZnZGSo7jHmU69bt26J9gcAAKGlRGEkMjJSrVu3Vlpamm+bx+NRWlqakpKSij0mKSmpyP6StHDhwmPuDwAAQkuJL9MMGjRIvXr1Ups2bdSuXTuNHTtW+/btU+/evSVJPXv2VIMGDZSamipJGjhwoDp06KDnnntOV199taZNm6Zly5bp1VdfLdtPAgAAAlKJw0hKSop27dqlESNGKD09Xa1atdL8+fN9g1S3bt2qsLDCDpeLLrpI7777rh599FE9/PDDOuusszRnzhw1b9687D4FAAAIWCWeZ8SGrKwsVa9eXdu2bWOeEQAAAoR3nrA9e/YoJibmmPv55d00R8rJyZEk5hsBACAA5eTkHDeMBETPiMfj0c6dO1WtWjW5XK4ye11vYqPHpWJwvisW57ticb4rFue7YpX2fDuOo5ycHNWvX7/IEI4jBUTPSFhYmBo2bFhurx8dHc0f5grE+a5YnO+KxfmuWJzvilWa8328HhGvUj21FwAAoKwQRgAAgFUhHUbcbrdGjhzJ1PMVhPNdsTjfFYvzXbE43xWrvM93QAxgBQAAwSuke0YAAIB9hBEAAGAVYQQAAFhFGAEAAFYRRgAAgFUhHUYmTJighIQERUVFKTExUUuXLrVdUlD46quv1KVLF9WvX18ul0tz5swp8nvHcTRixAjVq1dPlStXVnJysjZs2GCn2CCQmpqqtm3bqlq1aqpTp466d++udevWFdnnwIED6tevn2rVqqWqVavq+uuvV0ZGhqWKA9vEiRPVokUL30yUSUlJ+vTTT32/51yXn9GjR8vlcun+++/3beN8l63HHntMLperyNKsWTPf78vrfIdsGJk+fboGDRqkkSNHasWKFWrZsqU6d+6szMxM26UFvH379qlly5aaMGFCsb8fM2aMxo0bp0mTJun777/Xaaedps6dO+vAgQMVXGlwWLx4sfr166fvvvtOCxcu1MGDB9WpUyft27fPt88DDzyguXPnasaMGVq8eLF27typ6667zmLVgathw4YaPXq0li9frmXLlunvf/+7unXrpp9//lkS57q8/PDDD3rllVfUokWLIts532XvvPPO0++//+5bvv76a9/vyu18OyGqXbt2Tr9+/Xw/FxQUOPXr13dSU1MtVhV8JDmzZ8/2/ezxeJy6des6zzzzjG/bnj17HLfb7bz33nsWKgw+mZmZjiRn8eLFjuOY8xsREeHMmDHDt89///tfR5KzZMkSW2UGlRo1ajj/+c9/ONflJCcnxznrrLOchQsXOh06dHAGDhzoOA5/tsvDyJEjnZYtWxb7u/I83yHZM5Kfn6/ly5crOTnZty0sLEzJyclasmSJxcqC3+bNm5Wenl7k3MfExCgxMZFzX0aysrIkSTVr1pQkLV++XAcPHixyzps1a6ZGjRpxzk9RQUGBpk2bpn379ikpKYlzXU769eunq6++ush5lfizXV42bNig+vXr64wzzlCPHj20detWSeV7vgPiqb1lbffu3SooKFBcXFyR7XFxcVq7dq2lqkJDenq6JBV77r2/Q+l5PB7df//9uvjii9W8eXNJ5pxHRkaqevXqRfblnJfe6tWrlZSUpAMHDqhq1aqaPXu2zj33XK1atYpzXcamTZumFStW6Icffjjqd/zZLnuJiYl688031bRpU/3+++96/PHH1b59e61Zs6Zcz3dIhhEgWPXr109r1qwpco0XZa9p06ZatWqVsrKyNHPmTPXq1UuLFy+2XVbQ2bZtmwYOHKiFCxcqKirKdjkh4corr/S1W7RoocTERJ1++ul6//33Vbly5XJ735C8TFO7dm2Fh4cfNQI4IyNDdevWtVRVaPCeX8592evfv78+/vhjffnll2rYsKFve926dZWfn689e/YU2Z9zXnqRkZE688wz1bp1a6Wmpqply5Z68cUXOddlbPny5crMzNSFF16oSpUqqVKlSlq8eLHGjRunSpUqKS4ujvNdzqpXr66zzz5bGzduLNc/3yEZRiIjI9W6dWulpaX5tnk8HqWlpSkpKcliZcGvcePGqlu3bpFzn52dre+//55zX0qO46h///6aPXu2vvjiCzVu3LjI71u3bq2IiIgi53zdunXaunUr57yMeDwe5eXlca7LWMeOHbV69WqtWrXKt7Rp00Y9evTwtTnf5Wvv3r369ddfVa9evfL9831Kw18D2LRp0xy32+28+eabzi+//OL06dPHqV69upOenm67tICXk5PjrFy50lm5cqUjyXn++eedlStXOr/99pvjOI4zevRop3r16s6HH37o/PTTT063bt2cxo0bO/v377dceWC65557nJiYGGfRokXO77//7ltyc3N9+/Tt29dp1KiR88UXXzjLli1zkpKSnKSkJItVB66hQ4c6ixcvdjZv3uz89NNPztChQx2Xy+UsWLDAcRzOdXk7/G4ax+F8l7V//etfzqJFi5zNmzc733zzjZOcnOzUrl3byczMdByn/M53yIYRx3Gcl156yWnUqJETGRnptGvXzvnuu+9slxQUvvzyS0fSUUuvXr0cxzG39w4fPtyJi4tz3G6307FjR2fdunV2iw5gxZ1rSc4bb7zh22f//v3Ovffe69SoUcOpUqWKc+211zq///67vaID2B133OGcfvrpTmRkpBMbG+t07NjRF0Qch3Nd3o4MI5zvspWSkuLUq1fPiYyMdBo0aOCkpKQ4Gzdu9P2+vM63y3Ec59T6VgAAAEovJMeMAAAA/0EYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFX/B9Y7NftN/vJkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGzCAYAAABzfl4TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9lklEQVR4nO3deXgUVb7G8beTkD3psAYCYREQEIRx2EQEZgRRRAR0FBUUUEeBKCDoIOMC4hJcBlRERNQwioqigsuIArIpLoMIAwIiKITdoEJCCCQhqfvHuWkIa9Lp7url+3meenKqurr7R93cyeupU+c4LMuyBAAA4AFhdhcAAACCB8ECAAB4DMECAAB4DMECAAB4DMECAAB4DMECAAB4DMECAAB4DMECAAB4DMECAAB4DMECCBGDBg1S/fr13Xrv+PHj5XA4PFtQGVWkbgC+R7AAbOZwOMq0LV261O5SAeCsHKwVAthr1qxZpfZfe+01LVy4UK+//nqp45deeqmSk5Pd/p7CwkIVFxcrKiqq3O89evSojh49qujoaLe/312DBg3S0qVLtW3bNp9/N4Dyi7C7ACDUDRgwoNT+N998o4ULF550/ER5eXmKjY0t8/dUqlTJrfokKSIiQhER/M8FgLPjVggQAP7yl7+oRYsWWrVqlTp37qzY2Fj985//lCR98MEH6tmzp1JSUhQVFaWGDRvqkUceUVFRUanPOHGswrZt2+RwOPT000/rpZdeUsOGDRUVFaW2bdtq5cqVpd57qjEWDodDd955p+bNm6cWLVooKipKzZs316effnpS/UuXLlWbNm0UHR2thg0bavr06RUat3Ho0CGNHj1aqampioqKUpMmTfT000/rxA7YhQsX6uKLL1ZSUpLi4+PVpEkT13UrMWXKFDVv3lyxsbGqXLmy2rRpozfffNOtugDQYwEEjN9//109evTQ9ddfrwEDBrhui8ycOVPx8fEaNWqU4uPjtXjxYj300EPKycnRU089ddbPffPNN3Xw4EHdcccdcjgcevLJJ3X11Vfrl19+OWsvx5dffqn3339fw4YNU0JCgp577jldc8012r59u6pWrSpJWr16tS6//HLVqlVLDz/8sIqKijRhwgRVr17dretgWZauuuoqLVmyRLfeeqv+9Kc/6bPPPtO9996rXbt2afLkyZKk9evX68orr1TLli01YcIERUVFacuWLVqxYoXrs2bMmKHhw4frb3/7m0aMGKEjR45o7dq1+vbbb3XjjTe6VR8Q8iwAfiUtLc068f81u3TpYkmyXnzxxZPOz8vLO+nYHXfcYcXGxlpHjhxxHRs4cKBVr1491/7WrVstSVbVqlWtP/74w3X8gw8+sCRZH330kevYuHHjTqpJkhUZGWlt2bLFdex///ufJcmaMmWK61ivXr2s2NhYa9euXa5jmzdvtiIiIk76zFM5se558+ZZkqxHH3201Hl/+9vfLIfD4apn8uTJliRr3759p/3s3r17W82bNz9rDQDKjlshQICIiorS4MGDTzoeExPjah88eFC//fabOnXqpLy8PP34449n/dx+/fqpcuXKrv1OnTpJkn755Zezvrdbt25q2LCha79ly5ZKTEx0vbeoqEiLFi1Snz59lJKS4jqvUaNG6tGjx1k//1Q++eQThYeHa/jw4aWOjx49WpZlaf78+ZKkpKQkSeZWUXFx8Sk/KykpSTt37jzp1g8A9xEsgABRu3ZtRUZGnnR8/fr16tu3r5xOpxITE1W9enXXwM/s7Oyzfm7dunVL7ZeEjP3795f7vSXvL3lvVlaWDh8+rEaNGp103qmOlUVmZqZSUlKUkJBQ6nizZs1cr0smMHXs2FG33XabkpOTdf311+udd94pFTLGjBmj+Ph4tWvXTo0bN1ZaWlqpWyUAyo9gAQSI43smShw4cEBdunTR//73P02YMEEfffSRFi5cqCeeeEKSTvtf6scLDw8/5XGrDE+iV+S93hYTE6Ply5dr0aJFuummm7R27Vr169dPl156qWtga7NmzbRp0ybNnj1bF198sd577z1dfPHFGjdunM3VA4GLYAEEsKVLl+r333/XzJkzNWLECF155ZXq1q1bqVsbdqpRo4aio6O1ZcuWk1471bGyqFevnnbv3q2DBw+WOl5y26devXquY2FhYeratasmTZqkDRs26LHHHtPixYu1ZMkS1zlxcXHq16+fMjIytH37dvXs2VOPPfaYjhw54lZ9QKgjWAABrKTH4PgegoKCAr3wwgt2lVRKeHi4unXrpnnz5mn37t2u41u2bHGNhSivK664QkVFRXr++edLHZ88ebIcDodr7MYff/xx0nv/9Kc/SZLy8/MlmSdtjhcZGanzzjtPlmWpsLDQrfqAUMfjpkAAu+iii1S5cmUNHDhQw4cPl8Ph0Ouvv+4XtyJKjB8/XgsWLFDHjh01dOhQVyho0aKF1qxZU+7P69Wrl/7617/q/vvv17Zt29SqVSstWLBAH3zwgUaOHOkaTDphwgQtX75cPXv2VL169ZSVlaUXXnhBderU0cUXXyxJ6t69u2rWrKmOHTsqOTlZGzdu1PPPP6+ePXueNIYDQNkQLIAAVrVqVX388ccaPXq0HnjgAVWuXFkDBgxQ165dddlll9ldniSpdevWmj9/vu655x49+OCDSk1N1YQJE7Rx48YyPbVyorCwMH344Yd66KGH9PbbbysjI0P169fXU089pdGjR7vOu+qqq7Rt2za9+uqr+u2331StWjV16dJFDz/8sJxOpyTpjjvu0BtvvKFJkyYpNzdXderU0fDhw/XAAw947N8PhBrWCgFgiz59+mj9+vXavHmz3aUA8CDGWADwusOHD5fa37x5sz755BP95S9/sacgAF5DjwUAr6tVq5YGDRqkc845R5mZmZo2bZry8/O1evVqNW7c2O7yAHgQYywAeN3ll1+ut956S3v37lVUVJQ6dOigxx9/nFABBCF6LAAAgMcwxgIAAHgMwQIAAHiMz8dYFBcXa/fu3UpISJDD4fD11wMAADdYlqWDBw8qJSVFYWGn75fwebDYvXu3UlNTff21AADAA3bs2KE6deqc9nWfB4uSaXJ37NihxMREX389AABwQ05OjlJTU8863b3Pg0XJ7Y/ExESCBQAAAeZswxgYvAkAADyGYAEAADyGYAEAADyGKb0BIARYlqWjR4+qqKjI7lLgp8LDwxUREVHhqSDKFSzq16+vzMzMk44PGzZMU6dOrVAhAADvKCgo0J49e5SXl2d3KfBzsbGxqlWrliIjI93+jHIFi5UrV5ZKuz/88IMuvfRSXXvttW4XAADwnuLiYm3dulXh4eFKSUlRZGQkkxPiJJZlqaCgQPv27dPWrVvVuHHjM06CdSblChbVq1cvtT9x4kQ1bNhQXbp0cevLAQDeVVBQoOLiYqWmpio2NtbucuDHYmJiVKlSJWVmZqqgoEDR0dFufY7bYywKCgo0a9YsjRo16ozpNz8/X/n5+a79nJwcd78SAOAmd//rE6HFE78nbn/CvHnzdODAAQ0aNOiM56Wnp8vpdLo2pvMGACB4uR0sXnnlFfXo0UMpKSlnPG/s2LHKzs52bTt27HD3KwEAgJ9zK1hkZmZq0aJFuu222856blRUlGv6bqbxBgDYpX79+nrmmWfKfP7SpUvlcDh04MABr9UUjNwKFhkZGapRo4Z69uzp6XoAACHO4XCccRs/frxbn7ty5UrdfvvtZT7/oosu0p49e+R0Ot36vrIKtgBT7sGbxcXFysjI0MCBAxUR4R/zax05Ir3+ujR/vvTuuxJjlAAgcO3Zs8fVfvvtt/XQQw9p06ZNrmPx8fGutmVZKioqKtPfoxOfbDybyMhI1axZs1zvgRs9FosWLdL27dt1yy23eKMet1iWdO+90ty50uLFdlcDAP7NsqRDh3y/WVbZ6qtZs6Zrczqdcjgcrv0ff/xRCQkJmj9/vlq3bq2oqCh9+eWX+vnnn9W7d28lJycrPj5ebdu21aJFi0p97om3QhwOh15++WX17dtXsbGxaty4sT788EPX6yf2JMycOVNJSUn67LPP1KxZM8XHx+vyyy8vFYSOHj2q4cOHKykpSVWrVtWYMWM0cOBA9enTx93/c2n//v26+eabVblyZcXGxqpHjx7avHmz6/XMzEz16tVLlStXVlxcnJo3b65PPvnE9d7+/furevXqiomJUePGjZWRkeF2LWVR7mDRvXt3WZalc8891xv1uCUmRrrxRtN+9VV7awEAf5eXJ8XH+37z5MSf9913nyZOnKiNGzeqZcuWys3N1RVXXKHPP/9cq1ev1uWXX65evXpp+/btZ/ychx9+WNddd53Wrl2rK664Qv3799cff/xxhmuXp6efflqvv/66li9fru3bt+uee+5xvf7EE0/ojTfeUEZGhlasWKGcnBzNmzevQv/WQYMG6bvvvtOHH36or7/+WpZl6YorrlBhYaEkKS0tTfn5+Vq+fLnWrVunJ554wtWr8+CDD2rDhg2aP3++Nm7cqGnTpqlatWoVquesLB/Lzs62JFnZ2dke/dyVKy1LsqyoKMv64w+PfjQABKzDhw9bGzZssA4fPuw6lptr/vfS11tubvnrz8jIsJxOp2t/yZIlliRr3rx5Z31v8+bNrSlTprj269WrZ02ePNm1L8l64IEHjrsuuZYka/78+aW+a//+/a5aJFlbtmxxvWfq1KlWcnKyaz85Odl66qmnXPtHjx616tata/Xu3fu0dZ74Pcf76aefLEnWihUrXMd+++03KyYmxnrnnXcsy7Ks888/3xo/fvwpP7tXr17W4MGDT/vdJzrV70uJsv799o9BEh7QurV0/vnSunXS7NnS0KF2VwQA/ik2VsrNted7PaVNmzal9nNzczV+/Hj95z//0Z49e3T06FEdPnz4rD0WLVu2dLXj4uKUmJiorKys054fGxurhg0buvZr1arlOj87O1u//vqr2rVr53o9PDxcrVu3VnFxcbn+fSU2btyoiIgItW/f3nWsatWqatKkiTZu3ChJGj58uIYOHaoFCxaoW7duuuaaa1z/rqFDh+qaa67R999/r+7du6tPnz666KKL3KqlrIJmmKPDIQ0ebNrcDgGA03M4pLg432+eXKIkLi6u1P4999yjuXPn6vHHH9cXX3yhNWvW6Pzzz1dBQcEZP6dSpUonXBvHGUPAqc63yjp4xEtuu+02/fLLL7rpppu0bt06tWnTRlOmTJEk9ejRQ5mZmbr77ru1e/dude3atdStG28ImmAhSQMGSBER0nffSWvX2l0NAMBXVqxYoUGDBqlv3746//zzVbNmTW3bts2nNTidTiUnJ2vlypWuY0VFRfr+++/d/sxmzZrp6NGj+vbbb13Hfv/9d23atEnnnXee61hqaqqGDBmi999/X6NHj9aMGTNcr1WvXl0DBw7UrFmz9Mwzz+ill15yu56yCJpbIZJUvbp01VXS++9LGRnS5Ml2VwQA8IXGjRvr/fffV69eveRwOPTggw+6ffuhIu666y6lp6erUaNGatq0qaZMmaL9+/eXaUXZdevWKSEhwbXvcDjUqlUr9e7dW3//+981ffp0JSQk6L777lPt2rXVu3dvSdLIkSPVo0cPnXvuudq/f7+WLFmiZs2aSZIeeughtW7dWs2bN1d+fr4+/vhj12veElTBQjK3Q95/X5o1S3riCakCS8oDAALEpEmTdMstt+iiiy5StWrVNGbMGFsWvRwzZoz27t2rm2++WeHh4br99tt12WWXKTw8/Kzv7dy5c6n98PBwHT16VBkZGRoxYoSuvPJKFRQUqHPnzvrkk09ct2WKioqUlpamnTt3KjExUZdffrkm//9/WUdGRmrs2LHatm2bYmJi1KlTJ82ePdvz//DjOCwf3xzKycmR0+lUdna2V6b3PnpUSk2V9u6V3ntPuvpqj38FAASMI0eOaOvWrWrQoIHby2DDfcXFxWrWrJmuu+46PfLII3aXc1Zn+n0p69/voBpjIZkxFgMHmjaDOAEAvpSZmakZM2bop59+0rp16zR06FBt3bpVN5ZMthQCgi5YSMeeDpk/X9q9295aAAChIywsTDNnzlTbtm3VsWNHrVu3TosWLfL6uAZ/EnRjLCSpSRPpooukr74ya4iMGWN3RQCAUJCamqoVK1bYXYatgrLHQpJKljJ59dWyz08PAAAqJmiDxXXXmVnefvrJ9FwAQCizexInBAZP/J4EbbBISDDhQjJzWgBAKCp5JDHPkyuAIWiV/J6cOMNoeQTlGIsSgwdLM2dKb78tPfOMWV0PAEJJeHi4kpKSXOtZxMbGlmmyJoQWy7KUl5enrKwsJSUllWnejdMJ6mDRqZPUqJG0ZYv07rvSoEF2VwQAvlezZk1JOuPiWoAkJSUluX5f3BXUwaJkYbL77ze3QwgWAEKRw+FQrVq1VKNGDRUWFtpdDvxUpUqVKtRTUSLoZt480c6dUr16UnGxGcjZuLHXvxIAgKATsjNvnqhOHal7d9OeOdPWUgAACHpBHyykY3Na/PvfUlGRvbUAABDMQiJYXHWVVKWKtGuXtHCh3dUAABC8QiJYREVJ/fubNguTAQDgPSERLKRjt0M++ED6/Xd7awEAIFiFTLD405+kCy6QCgqkcePsrgYAgOAUMsFCktLTzc+pU6W5c+2tBQCAYBRSweKyy6R77zXtW26RMjPtrQcAgGATUsFCkh59VGrXTjpwQLrxRolJ6AAA8JyQCxaRkdJbb0mJiWY59fHj7a4IAIDgEXLBQpLOOUd6+WXTTk+XFi2ytx4AAIJFSAYLSbr2Wun22yXLkgYMkH791e6KAAAIfCEbLCRp8mSpeXMTKm6+2SxUBgAA3BfSwSI2Vnr7bSkmRlqwQHrqKbsrAgAgsIV0sJBMj8Vzz5n2/fdLX39tbz0AAASykA8WknTrrVK/fmbl0xtuMI+iAgCA8iNYSHI4pOnTzdMimZnSbbeZQZ0AAKB8CBb/z+mUZs+WIiKk996TZsywuyIAAAIPweI4bdseW09k9Gim/AYAoLwIFicYNUrq2FHKzeWWCAAA5UWwOEFYmPTqq1J0tJmRs2SGTgAAcHYEi1M491yzWJlkbols325vPQAABAqCxWmMHCldeKF08OCxqb8BAMCZESxOIzxcysiQoqKkzz6TZs60uyIAAPwfweIMmjaVJkww7bvvlnbutLceAAD8XbmDxa5duzRgwABVrVpVMTExOv/88/Xdd995oza/MGqU1K6dlJ0t3XEHt0QAADiTcgWL/fv3q2PHjqpUqZLmz5+vDRs26F//+pcqV67srfpsFxFhbolERkqffCK9/rrdFQEA4L8cllX2/wa/7777tGLFCn3xxRduf2FOTo6cTqeys7OVmJjo9uf4Wnq69M9/SklJ0vr1UkqK3RUBAOA7Zf37Xa4eiw8//FBt2rTRtddeqxo1auiCCy7QjLPMfZ2fn6+cnJxSWyC6916pdWuzQNmQIdwSAQDgVMoVLH755RdNmzZNjRs31meffaahQ4dq+PDh+ve//33a96Snp8vpdLq21NTUChdth4gI82RIpUrSRx9Jb75pd0UAAPifct0KiYyMVJs2bfTVV1+5jg0fPlwrV67U119/fcr35OfnKz8/37Wfk5Oj1NTUgLsVUuLRR6UHH5SqVDG3RGrWtLsiAAC8zyu3QmrVqqXzzjuv1LFmzZpp+xmmpoyKilJiYmKpLZCNGSNdcIH0xx/SXXfZXQ0AAP6lXMGiY8eO2rRpU6ljP/30k+rVq+fRovxZpUrmKZGwMOndd6WlS+2uCAAA/1GuYHH33Xfrm2++0eOPP64tW7bozTff1EsvvaS0tDRv1eeXWrUyAzglM/V3UZGt5QAA4DfKNcZCkj7++GONHTtWmzdvVoMGDTRq1Cj9/e9/L/P7A/Vx0xP99pvUuLF5SmT6dLOeCAAAwaqsf7/LHSwqKliChSQ9+6zpsaheXdq8WXI67a4IAADv8MrgTZQ2bJhZT2TfPumRR+yuBgAA+xEsKqBSJWnSJNN+7jnTawEAQCgjWFRQjx5mKyyURo+2uxoAAOxFsPCASZPMzJwffSQtXGh3NQAA2Idg4QFNm0olT9zefbd09Ki99QAAYBeChYeMGydVrWqm+Z4+3e5qAACwB8HCQypXliZMMO2HHjJTfgMAEGoIFh50++1SixYmVDz8sN3VAADgewQLD4qIkCZPNu2pU6WNG+2tBwAAXyNYeFi3btJVV5n1Q0aNsrsaAAB8i2DhBU8/bSbP+vRT6ZNP7K4GAADfIVh4QePG0ogRpj1qFI+fAgBCB8HCSx54wDx+ummT9NZbdlcDAIBvECy8xOmU7r3XtCdMoNcCABAaCBZelJYmVasmbdkivfmm3dUAAOB9BAsvio8/1mvxyCP0WgAAgh/BwsuGDTvWa/HGG3ZXAwCAdxEsvIxeCwBAKCFY+EDJWIuff5ZmzbK7GgAAvIdg4QNxcdI//mHajz5KrwUAIHgRLHxk2DCpenXTa/H663ZXAwCAdxAsfOTEXovCQnvrAQDAGwgWPjR0qFSjhvTLL4y1AAAEJ4KFDx3fa/HII/RaAACCD8HCx0p6LbZuZawFACD4ECx8LDZWGjPGtBlrAQAINgQLGwwZIiUnm16L116zuxoAADyHYGGDE3stCgrsrQcAAE8hWNjkjjtMr8W2bfRaAACCB8HCJrGx0n33mTa9FgCAYEGwsNEdd0g1a0qZmdLbb9tdDQAAFUewsFFMjDRihGk//bRkWfbWAwBARREsbHbHHWbirLVrpUWL7K4GAICKIVjYrHJl6bbbTPvpp+2tBQCAiiJY+IGRI6WwMGnBAul//7O7GgAA3Eew8AP160vXXmva//qXraUAAFAhBAs/cc895udbb0k7d9pbCwAA7iJY+Ik2baQuXaSjR6XnnrO7GgAA3EOw8CMlvRbTp0s5OfbWAgCAOwgWfuSKK6SmTU2oePllu6sBAKD8CBZ+JCxMGj3atJ95hiXVAQCBh2DhZwYMkGrUkHbskObMsbsaAADKp1zBYvz48XI4HKW2pk2bequ2kBQdLd11l2kzzTcAINCUu8eiefPm2rNnj2v78ssvvVFXSBs61Kx+unq1tGSJ3dUAAFB25Q4WERERqlmzpmurVq2aN+oKaVWrSrfcYtpM8w0ACCTlDhabN29WSkqKzjnnHPXv31/bt28/4/n5+fnKyckpteHsSqb5nj9f+uEHu6sBAKBsyhUs2rdvr5kzZ+rTTz/VtGnTtHXrVnXq1EkHDx487XvS09PldDpdW2pqaoWLDgUNG0pXX23akybZWwsAAGXlsCz3hwceOHBA9erV06RJk3Trrbee8pz8/Hzl5+e79nNycpSamqrs7GwlJia6+9Uh4dtvpQsvlCpVkjIzpVq17K4IABCqcnJy5HQ6z/r3u0KPmyYlJencc8/Vli1bTntOVFSUEhMTS20om/btpYsvNvNZTJlidzUAAJxdhYJFbm6ufv75Z9XiP6W9pmSa72nTpNxce2sBAOBsyhUs7rnnHi1btkzbtm3TV199pb59+yo8PFw33HCDt+oLeb16SY0bSwcOSBkZdlcDAMCZlStY7Ny5UzfccIOaNGmi6667TlWrVtU333yj6tWre6u+kBcWZp4QkaTnn5eKi20tBwCAM6rQ4E13lHXwB47JzZVq1zaLk336qXTZZXZXBAAINT4ZvAnfiI+XBg827eees7cWAADOhGARINLSzM9PPpE2b7a3FgAATodgESAaN5auuMK0p061txYAAE6HYBFASlY9zcjg0VMAgH8iWASQ7t1Nz0VOjvTaa3ZXAwDAyQgWASQs7FivxZQpkm+f5wEA4OwIFgFm4EDzlMiPP0qLFtldDQAApREsAkxiojRokGmzfggAwN8QLALQnXeanx9/LP3yi721AABwPIJFAGrSxMy+aVk8egoA8C8EiwBVMojzlVd49BQA4D8IFgGqRw+pYUMpO1uaNcvuagAAMAgWASos7NhYi+ef59FTAIB/IFgEsMGDpbg4af16ackSu6sBAIBgEdCcTjOvhcSqpwAA/0CwCHAlt0M++kjats3WUgAAIFgEumbNpG7dpOJi6YUX7K4GABDqCBZBoOTR05dflvLy7K0FABDaCBZBoGdPqUEDaf9+6Y037K4GABDKCBZBIDxcSkszbR49BQDYiWARJAYPlqKjpbVrpa++srsaAECoIlgEiSpVpBtvNG3WDwEA2IVgEUSGDTM/331X+vVXe2sBAIQmgkUQad1aat9eKiw0T4gAAOBrBIsgUzKI88UXpaNH7a0FABB6CBZB5tprpWrVpJ07pY8/trsaAECoIVgEmeho6dZbTZtBnAAAXyNYBKEhQySHQ1q0SNq0ye5qAAChhGARhOrXl6680rSnTbO1FABAiCFYBKmSR09nzpQOHbK1FABACCFYBKnu3aVGjaTsbNYPAQD4DsEiSIWFSUOHmvbUqawfAgDwDYJFEBs0iPVDAAC+RbAIYsevH/LCC/bWAgAIDQSLIFcyE+ecOawfAgDwPoJFkPvzn1k/BADgOwSLEFDSazF9OuuHAAC8i2ARAkrWD9mxg/VDAADeRbAIAawfAgDwFYJFiGD9EACALxAsQgTrhwAAfIFgEUJKBnFmZEi5ufbWAgAIThUKFhMnTpTD4dDIkSM9VA686dJLpcaNpZwc6fXX7a4GABCM3A4WK1eu1PTp09WyZUtP1gMvCguT7rzTtJ9/nvVDAACe51awyM3NVf/+/TVjxgxVrlzZ0zXBiwYOlOLipA0bpCVL7K4GABBs3AoWaWlp6tmzp7p163bWc/Pz85WTk1Nqg32cThMuJGnKFHtrAQAEn3IHi9mzZ+v7779Xenp6mc5PT0+X0+l0bampqeUuEp5Vcjvkww+lzEx7awEABJdyBYsdO3ZoxIgReuONNxQdHV2m94wdO1bZ2dmubceOHW4VCs9p1kzq2lUqLubRUwCAZzksq+xD+ObNm6e+ffsqPDzcdayoqEgOh0NhYWHKz88v9dqp5OTkyOl0Kjs7W4mJie5Xjgr54AOpTx+ztPrOnVJMjN0VAQD8WVn/fkeU50O7du2qdevWlTo2ePBgNW3aVGPGjDlrqID/uPJKM2nWtm3SW29Jt9xid0UAgGBQrlshCQkJatGiRaktLi5OVatWVYsWLbxVI7wgPFwaNsy0p0zh0VMAgGcw82YIu+UWs0DZmjXSV1/ZXQ0AIBiU61bIqSxdutQDZcAOVatK/ftLr7xiei06drS7IgBAoKPHIsTddZf5+d570u7d9tYCAAh8BIsQ16qV1KmTdPSoNH263dUAAAIdwQKuCbOmT5cKCuytBQAQ2AgWUN++Uu3a0q+/SnPm2F0NACCQESygSpWkIUNMm/VDAAAVQbCAJOnvf5ciI6Vvv5VWrrS7GgBAoCJYQJKUnCxdd51pP/+8vbUAAAIXwQIuJY+ezp4tZWXZWwsAIDARLODSrp3ZCgqkGTPsrgYAEIgIFiil5NHTadOkwkJ7awEABB6CBUq57joz3mLXLuntt+2uBgAQaAgWKCUqShoxwrSffJJVTwEA5UOwwEmGDJHi4qR166QFC+yuBgAQSAgWOEnlytLtt5v2k0/aWwsAILAQLHBKI0dK4eHS4sXSqlV2VwMACBQEC5xS3brS9deb9lNP2VsLACBwECxwWvfea37OmSNt3WpvLQCAwECwwGm1aiV17y4VF0uTJtldDQAgEBAscEb/+If5+eqr0u+/21sLAMD/ESxwRpdcIl1wgZSXJ73wgt3VAAD8HcECZ+RwHOu1mDJFOnzY3noAAP6NYIGz+tvfpPr1pX37pH//2+5qAAD+jGCBs4qIkO6+27SffloqKrK3HgCA/yJYoExuvVWqUkX6+Wdp3jy7qwEA+CuCBcokLk4aNsy0WZwMAHA6BAuU2V13mdVP//tf6Ysv7K4GAOCPCBYosxo1pEGDTJvFyQAAp0KwQLmMHm0eQf3Pf6QNG+yuBgDgbwgWKJfGjaW+fU376aftrQUA4H8IFii3ksXJZs2Sdu60txYAgH8hWKDcLrxQ6txZKiyUnnjC7moAAP6EYAG3jBtnfr70krRrl721AAD8B8ECbvnrX6VOnaSCAmniRLurAQD4C4IF3OJwSOPHmza9FgCAEgQLuI1eCwDAiQgWcNvxvRYzZtBrAQAgWKCCSnot8vN5QgQAQLBABTHWAgBwPIIFKoxeCwBACYIFKoxeCwBACYIFPIJeCwCARLCAh9BrAQCQCBbwoL/+Vbr4YnotACCUlStYTJs2TS1btlRiYqISExPVoUMHzZ8/31u1IcCc2Guxe7et5QAAbFCuYFGnTh1NnDhRq1at0nfffadLLrlEvXv31vr1671VHwLMJZfQawEAocxhWZZVkQ+oUqWKnnrqKd16662nfD0/P1/5+fmu/ZycHKWmpio7O1uJiYkV+Wr4qc8/l7p1k6KipF9+kVJS7K4IAFBROTk5cjqdZ/377fYYi6KiIs2ePVuHDh1Shw4dTnteenq6nE6na0tNTXX3KxEg6LUAgNBV7h6LdevWqUOHDjpy5Iji4+P15ptv6oorrjjt+fRYhCZ6LQAguHitx6JJkyZas2aNvv32Ww0dOlQDBw7Uhg0bTnt+VFSUa7BnyYbgd3yvxUMP2V0NAMBXKjzGolu3bmrYsKGmT59epvPLmngQ+L7+WrroIvO0yKpV0gUX2F0RAMBdXh9jUaK4uLjUrQ6gRIcO0g03SJYljRxpfgIAglu5gsXYsWO1fPlybdu2TevWrdPYsWO1dOlS9e/f31v1IcA98YQUEyMtXy69957d1QAAvK1cwSIrK0s333yzmjRpoq5du2rlypX67LPPdOmll3qrPgS41FTpH/8w7XvvlY4csbceAIB3VXiMRXkxxiL0HDokNWli1g957DHpn/+0uyIAQHn5bIwFcDZxccfms3j8cWnPHnvrAQB4D8ECPnHjjdKFF5reC3osACB4ESzgEw6H9Mwzpj1zpvTdd3ZWAwDwFoIFfKZ9e2nAANPm8VMACE4EC/jUxIlSbKy0YoX0zjt2VwMA8DSCBXyqdm3pvvtM+x//kA4ftrceAIBnESzgc6NHm/kttm+Xnn7a7moAAJ5EsIDPxcZKTz5p2hMnmvktAADBgWABW/TrJ3XsKOXlSWPH2l0NAMBTCBawxfGPn77+uvTtt7aWAwDwEIIFbNOmjTRwoGkPGSIVFtpbDwCg4ggWsNWTT0pVqkhr1hwbdwEACFwEC9iqRg3puedMe8IEacMGe+sBAFQMwQK2u/FG6corpYIC6ZZbpKIiuysCALiLYAHbORzSiy9KiYlmEOezz9pdEQDAXQQL+IXataV//cu0H3hA2rLF3noAAO4hWMBv3Hqr1LWrmeb7ttuk4mK7KwIAlBfBAn7D4ZBmzDAzcy5bJr30kt0VAQDKi2ABv9KggZnmW5LuvdesJwIACBwEC/idtDQz3XdurnT77ZJl2V0RAKCsCBbwO2Fh0iuvSFFR0mefSa+9ZndFAICyIljALzVpYibMkqSRI6U9e2wtBwBQRgQL+K1Ro8x6IgcOSMOGcUsEAAIBwQJ+KyJCevVVqVIlad486Z137K4IAHA2BAv4tfPPl+6/37SHDuUpEQDwdwQL+L2xY6W2baX9+6UbbmB5dQDwZwQL+L3ISGn2bLOWyFdfSePG2V0RAOB0CBYICOecI738smmnp0sLFthbDwDg1AgWCBjXXmvGWUjSTTfxCCoA+COCBQLKpElSy5ZSVpY0YIBUVGR3RQCA4xEsEFCio6W335bi4qTFi6XHH7e7IgDA8QgWCDhNm0rTppn2+PFmJVQAgH8gWCAg3XSTNHCgVFws3XijtG+f3RUBACSCBQLY88+b3ovdu4+FDACAvQgWCFjx8Wa8RXS0NH++GdgJALAXwQIBrWVL6ZlnTHvsWOmbb2wtBwBCHsECAe/226XrrpOOHpX69TOPogIA7EGwQMBzOKSXXpIaNTKLlPXtKx05YndVABCaCBYICk6n9NFH5udXX0l//7tkWXZXBQChh2CBoNG0qfTuu1J4uDRrFpNnAYAdCBYIKt26mcdQJemBB0zQAAD4DsECQWfIEGn4cNO++Wbpu+/srQcAQkm5gkV6erratm2rhIQE1ahRQ3369NGmTZu8VRvgtn/9S+rRQzp8WOrdW9q1y+6KACA0lCtYLFu2TGlpafrmm2+0cOFCFRYWqnv37jp06JC36gPcEhEhzZ4tNW9uZubs1Uvi1xQAvM9hWe6Pnd+3b59q1KihZcuWqXPnzmV6T05OjpxOp7Kzs5WYmOjuVwNlsnWr1K6d9Ntv0tVXS3PmSGHcAASAcivr3+8K/U9sdna2JKlKlSqnPSc/P185OTmlNsBXGjSQ5s2TIiOl9983AzoBAN7jdrAoLi7WyJEj1bFjR7Vo0eK056Wnp8vpdLq21NRUd78ScEvHjtIrr5h2err02mv21gMAwcztWyFDhw7V/Pnz9eWXX6pOnTqnPS8/P1/5+fmu/ZycHKWmpnIrBD53//1mbovISDOZVvfudlcEAIHDq7dC7rzzTn388cdasmTJGUOFJEVFRSkxMbHUBtjhkUfMmiIFBVKfPtKyZXZXBADBp1zBwrIs3XnnnZo7d64WL16sBg0aeKsuwOPCwqTXX5d69jSPoV55JauhAoCnlStYpKWladasWXrzzTeVkJCgvXv3au/evTp8+LC36gM8KjLSzMbZrZuUmytdfrn0/fd2VwUAwaNcYywcDscpj2dkZGjQoEFl+gweN4U/OHTIhIovv5SqVpWWLpXOMAYZAEJeWf9+R5TnQysw5QXgV+LipP/8x/RcrFxpfi5fLp17rt2VAUBgY6oghKzEROnTT6VWraRff5W6djUTagEA3EewQEirUkVasEBq1kzaudOEi5077a4KAAIXwQIhr0YNadEiqWFD02PRtavpwQAAlB/BApCUkiJ9/rlUt670009mzMW+fXZXBQCBh2AB/L969Uy4qFVL+uEHqVMnaft2u6sCgMBCsACO06iRtGSJlJoqbdpk1hnZuNHuqgAgcBAsgBM0aSKtWHFsQGenTtJ//2t3VQAQGAgWwCmkpkpffCG1ayf9/rt0ySXSwoV2VwUA/o9gAZxG1apmzMWll5qZOnv2lObMsbsqAPBvBAvgDOLjzRLr114rFRZK/fpJL75od1UA4L8IFsBZREVJb70lDRkiWZY0dKj06KOmDQAojWABlEF4uPTCC9KDD5r9Bx+U7r5bKi62ty4A8DcEC6CMHA5pwgTp2WfN/rPPStdfL+Xl2VsXAPgTggVQTsOHS7NmSZUqmcGcnTqxvggAlCBYAG7o39+sL1KtmvT99+axVOa6AACCBeC2zp1NmGjRQtqzx+y/+abdVQGAvQgWQAU0aCB99ZXUq5eUn296Mh54gEGdAEIXwQKooIQEae5cacwYs//YY9I110i5ufbWBQB2IFgAHhAeLk2cKP3731JkpDRvnlnALDPT7soAwLcIFoAH3XyztHSpVKOGtHatGdT5xRd2VwUAvkOwADysQwdp5UqpVSspK0v6y1+kf/7TjMEAgGBHsAC8oG5d6csvTQ9GcbGUnm56L9autbsyAPAuggXgJfHxZszFe++Z+S7WrpXatpWeeEIqKrK7OgDwDoIF4GVXXy398IN5JLWgQLrvPqlLF+nnn+2uDAA8j2AB+EBysvTBB9Krr5rHU1esMGMwpk9nlVQAwYVgAfiIwyENHmxuiXTpIh06ZJZiv+IKafduu6sDAM8gWAA+Vr++tHixNGmSFBUlffqpmRZ8zhy7KwOAiiNYADYIC5PuvtssYPbnP0v790vXXSfddJN04IDd1QGA+wgWgI3OO0/6+muzvkhYmFmOvWVL06MBAIGIYAHYLDJSeuQRM+9Fw4bSjh1S167SqFHSkSN2VwcA5UOwAPxEhw7SmjXS7beb/cmTpTZtzDEACBQEC8CPxMebR1A/+sg8orp+vZmxc+JEJtUCEBgIFoAfuvJKad06qU8fqbBQGjtW6tzZrEECAP6MYAH4qerVpffflzIyzKRaX31lei+uu07avNnu6gDg1AgWgB9zOKRBg8yU4DffbPbnzJGaNZOGDpX27LG7QgAojWABBIC6dc2CZmvWmJk6i4qkF1+UGjUyj6pmZ9tdIQAYBAsggLRsKf3nP9KyZdKFF0p5edJjj5nHVCdPlvLz7a4QQKgjWAABqHNnM+bi/felJk2k3383816ce655qoSAAcAuBAsgQDkcUt++ZvzFjBlSSoq0fbtZ2KxhQ+nZZ02PBgD4EsECCHAREdJtt5knRZ55RqpdW9q1Sxo50ix4NnGilJNjc5EAQgbBAggSsbHSiBHSzz+b2yENGkj79pk5MOrVk8aPl/74w+4qAQQ7ggUQZKKizLTgP/0kvfaa1LSpWTH14YdNwBgzxvRoAIA3lDtYLF++XL169VJKSoocDofmzZvnhbIAVFREhFmG/YcfpHfekVq1knJzpSefNI+v9u4tffIJU4UD8KxyB4tDhw6pVatWmjp1qjfqAeBh4eHStddKq1ebNUg6d5aKi6UPP5R69jQDPR99lMm2AHiGw7Isy+03OxyaO3eu+vTpU+b35OTkyOl0Kjs7W4mJie5+NYAK2LhReuklM+nW/v3mWHi4dNVV0h13SJdeKoVxoxTAccr699vr/9ORn5+vnJycUhsAezVrZibU2rXLjMPo2NHcEpk7V7r8cjOj52OPSTt32l0pgEDj9WCRnp4up9Pp2lJTU739lQDKKCbGjMP48kuzmupdd0lOp7R1q5kqvF49EzTeeUc6csTuagEEAq/fCsnPz1f+cdMA5uTkKDU1lVshgJ/KyzMLnb36qrR8+bHjlStLN94oDR4s/fnPZoIuAKHDb26FREVFKTExsdQGwH/FxkoDB5r1SLZsMT0XdeqYsRhTp0pt2kh/+pOZjOvXX+2uFoC/YXgWgNNq2FB65BFp2zbps8+k668382SsXSvdfbdUs6Z5jHXUKLM42sGDdlcMwG4R5X1Dbm6utmzZ4trfunWr1qxZoypVqqhu3boeLQ6AfwgPl7p3N9v+/dLs2VJGhrRypQkZa9eawaAREVL79lLXrma78EIpMtLu6gH4UrnHWCxdulR//etfTzo+cOBAzZw586zv53FTIHhkZUlLlkiff262X34p/XpsrJk3o3t36bLLzNMojM0AAlNZ/35XaPCmOwgWQPDauvVYyFi82ASP49WpcyxkdO0qVa1qT50Ayo9gAcBWlmUeYV20yIzPWL689COrDofUtu2xWyzt2pnxGwD8E8ECgF85fFj64gsTMhYsMGuYHC862ozJ6NTJ3D7p0EGKi7OnVgAnI1gA8Gu7dkkLF5qg8fnnZon340VEmPkyOnc2W8eOUpUq9tQKgGABIIBYlrRpk+nRWL7cbNu3n3xe/fpS8+alt2bNzCBRAN5FsAAQ0DIzSweNTZtOfZ7DITVocCxotGghnX++1LQpj7oCnkSwABBUfv/djMtYv7709ttvpz4/IkJq0sSEjOO3evV45BVwB8ECQEjYt+9YyPjhB7OtWydlZ5/6/IQE6dxzpbp1pdTUY1vJfq1aZkIwAKURLACELMsyS76vW1d627hRKiw883vDw6XatU3QaNTo5M3p9M2/AfA3BAsAOEFhofTTT9LPP0s7dpht+/Zj7V27pKNHz/wZ1aodCxkNGkiJiWbwaGyseTz2xHZcnJkILCGBWzAIbAQLACinoiJp714TMrZuNQFky5ZjW0VWc42KkqpXl2rUMNuJ7ZJekrp1ecoF/olgAQAedvBg6bCxfbt06JDZ8vLMdmI7N9dMDlYe1aqZQaZ165b+WbOmVLmylJRktuhoekHgOwQLAPATeXlmkOm+fWb9lKys0u2sLDMmJDOzfEvPR0YeCxklW8mtmZiYY9uJ+9HRUliY+YySYOJwnNyuUsX0qCQnm0BT8h6EprL+/S73sukAgPKJjTU9DvXqnf3cAwdMT0hm5sk/s7LM6wcOSMXFUkHBsWDibeHh5pZNcvKxWzjJyVJ8vFSpkgk5kZGnbsfGlu5pobcluBEsAMCPlPzhbdny9OdYlrnFUhIy9u8/1s7JMbde8vLMzxPbJVvJ55T8PL4tmfEmf/xhQsv+/cfGn+zd65l/Z2Rk6bARH186aJyq7XCYsFKylYSXE9uWZYJXyVZUdHI7Otr07pxpi4sz50VFmS0QHkMuLjaL/cXE2BfcCBYAEGAcDvOUSUKCmXvD2woKTr51k5VlBrPm5ZnXCwvNz5KtZD8/35xTEnyO72359deKDYj1tYiIY0EjOtpskZHmeESECR6nap/p2PGvlQSiUwWhkp9Hjhwbw3PimJ6STTITytm1tg7BAgBwRpGR5qmV2rUr/lmWZcaRHB80DhwwPTAlrx//8/h2cbEJLCVbSYA5se1wmD/UYWHHthP3jxwxvTun27KzzR/p4+s4etTUWVKrP8vLI1gAAEKAw3HsVkPdunZXc2aWZcLEkSOm5yU//1j7+GNFRea847fjjxUWmv2znVeWQBQdfeZ5U0racXH2XTeCBQAAp3D8mI6EBLurCRw8PAQAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADyGYAEAADzG56ubWv+/uH1OTo6vvxoAALip5O92yd/x0/F5sDh48KAkKTU11ddfDQAAKujgwYNyOp2nfd1hnS16eFhxcbF2796thIQEORwOj31uTk6OUlNTtWPHDiUmJnrsc3FqXG/f4nr7Ftfbt7jevuXu9bYsSwcPHlRKSorCwk4/ksLnPRZhYWGqU6eO1z4/MTGRX0wf4nr7Ftfbt7jevsX19i13rveZeipKMHgTAAB4DMECAAB4TNAEi6ioKI0bN05RUVF2lxISuN6+xfX2La63b3G9fcvb19vngzcBAEDwCpoeCwAAYD+CBQAA8BiCBQAA8BiCBQAA8BiCBQAA8JigCRZTp05V/fr1FR0drfbt2+u///2v3SUFheXLl6tXr15KSUmRw+HQvHnzSr1uWZYeeugh1apVSzExMerWrZs2b95sT7EBLj09XW3btlVCQoJq1KihPn36aNOmTaXOOXLkiNLS0lS1alXFx8frmmuu0a+//mpTxYFv2rRpatmypWsGwg4dOmj+/Pmu17ne3jNx4kQ5HA6NHDnSdYzr7Vnjx4+Xw+EotTVt2tT1ureud1AEi7ffflujRo3SuHHj9P3336tVq1a67LLLlJWVZXdpAe/QoUNq1aqVpk6desrXn3zyST333HN68cUX9e233youLk6XXXaZjhw54uNKA9+yZcuUlpamb775RgsXLlRhYaG6d++uQ4cOuc65++679dFHH2nOnDlatmyZdu/erauvvtrGqgNbnTp1NHHiRK1atUrfffedLrnkEvXu3Vvr16+XxPX2lpUrV2r69Olq2bJlqeNcb89r3ry59uzZ49q+/PJL12teu95WEGjXrp2Vlpbm2i8qKrJSUlKs9PR0G6sKPpKsuXPnuvaLi4utmjVrWk899ZTr2IEDB6yoqCjrrbfesqHC4JKVlWVJspYtW2ZZlrm2lSpVsubMmeM6Z+PGjZYk6+uvv7arzKBTuXJl6+WXX+Z6e8nBgwetxo0bWwsXLrS6dOlijRgxwrIsfr+9Ydy4cVarVq1O+Zo3r3fA91gUFBRo1apV6tatm+tYWFiYunXrpq+//trGyoLf1q1btXfv3lLX3ul0qn379lx7D8jOzpYkValSRZK0atUqFRYWlrreTZs2Vd26dbneHlBUVKTZs2fr0KFD6tChA9fbS9LS0tSzZ89S11Xi99tbNm/erJSUFJ1zzjnq37+/tm/fLsm719vnq5t62m+//aaioiIlJyeXOp6cnKwff/zRpqpCw969eyXplNe+5DW4p7i4WCNHjlTHjh3VokULSeZ6R0ZGKikpqdS5XO+KWbdunTp06KAjR44oPj5ec+fO1Xnnnac1a9ZwvT1s9uzZ+v7777Vy5cqTXuP32/Pat2+vmTNnqkmTJtqzZ48efvhhderUST/88INXr3fABwsgGKWlpemHH34odT8U3tGkSROtWbNG2dnZevfddzVw4EAtW7bM7rKCzo4dOzRixAgtXLhQ0dHRdpcTEnr06OFqt2zZUu3bt1e9evX0zjvvKCYmxmvfG/C3QqpVq6bw8PCTRrL++uuvqlmzpk1VhYaS68u196w777xTH3/8sZYsWaI6deq4jtesWVMFBQU6cOBAqfO53hUTGRmpRo0aqXXr1kpPT1erVq307LPPcr09bNWqVcrKytKf//xnRUREKCIiQsuWLdNzzz2niIgIJScnc729LCkpSeeee662bNni1d/vgA8WkZGRat26tT7//HPXseLiYn3++efq0KGDjZUFvwYNGqhmzZqlrn1OTo6+/fZbrr0bLMvSnXfeqblz52rx4sVq0KBBqddbt26tSpUqlbremzZt0vbt27neHlRcXKz8/Hyut4d17dpV69at05o1a1xbmzZt1L9/f1eb6+1dubm5+vnnn1WrVi3v/n5XaOinn5g9e7YVFRVlzZw509qwYYN1++23W0lJSdbevXvtLi3gHTx40Fq9erW1evVqS5I1adIka/Xq1VZmZqZlWZY1ceJEKykpyfrggw+stWvXWr1797YaNGhgHT582ObKA8/QoUMtp9NpLV261NqzZ49ry8vLc50zZMgQq27dutbixYut7777zurQoYPVoUMHG6sObPfdd5+1bNkya+vWrdbatWut++67z3I4HNaCBQssy+J6e9vxT4VYFtfb00aPHm0tXbrU2rp1q7VixQqrW7duVrVq1aysrCzLsrx3vYMiWFiWZU2ZMsWqW7euFRkZabVr18765ptv7C4pKCxZssSSdNI2cOBAy7LMI6cPPviglZycbEVFRVldu3a1Nm3aZG/RAepU11mSlZGR4Trn8OHD1rBhw6zKlStbsbGxVt++fa09e/bYV3SAu+WWW6x69epZkZGRVvXq1a2uXbu6QoVlcb297cRgwfX2rH79+lm1atWyIiMjrdq1a1v9+vWztmzZ4nrdW9fbYVmWVbE+DwAAACPgx1gAAAD/QbAAAAAeQ7AAAAAeQ7AAAAAeQ7AAAAAeQ7AAAAAeQ7AAAAAeQ7AAAAAeQ7AAAAAeQ7AAAAAeQ7AAAAAe83+qYPYb8mTFggAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Take a look at the training curves of your model\n",
    "\n",
    "acc = history.history['accuracy']\n",
    "loss = history.history['loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'b', label='Training accuracy')\n",
    "plt.title('Training accuracy')\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'b', label='Training Loss')\n",
    "plt.title('Training loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OjvED5A3qrn2"
   },
   "source": [
    "Before closing the assignment, be sure to also download the `history.pkl` file which contains the information of the training history of your model and will be used to compute your grade. You can download this file by running the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 17
    },
    "id": "9QRG73l6qE-c",
    "outputId": "9ede449d-e1f2-4bba-fb3c-97c347cc2894",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "    async function download(id, filename, size) {\n",
       "      if (!google.colab.kernel.accessAllowed) {\n",
       "        return;\n",
       "      }\n",
       "      const div = document.createElement('div');\n",
       "      const label = document.createElement('label');\n",
       "      label.textContent = `Downloading \"${filename}\": `;\n",
       "      div.appendChild(label);\n",
       "      const progress = document.createElement('progress');\n",
       "      progress.max = size;\n",
       "      div.appendChild(progress);\n",
       "      document.body.appendChild(div);\n",
       "\n",
       "      const buffers = [];\n",
       "      let downloaded = 0;\n",
       "\n",
       "      const channel = await google.colab.kernel.comms.open(id);\n",
       "      // Send a message to notify the kernel that we're ready.\n",
       "      channel.send({})\n",
       "\n",
       "      for await (const message of channel.messages) {\n",
       "        // Send a message to notify the kernel that we're ready.\n",
       "        channel.send({})\n",
       "        if (message.buffers) {\n",
       "          for (const buffer of message.buffers) {\n",
       "            buffers.push(buffer);\n",
       "            downloaded += buffer.byteLength;\n",
       "            progress.value = downloaded;\n",
       "          }\n",
       "        }\n",
       "      }\n",
       "      const blob = new Blob(buffers, {type: 'application/binary'});\n",
       "      const a = document.createElement('a');\n",
       "      a.href = window.URL.createObjectURL(blob);\n",
       "      a.download = filename;\n",
       "      div.appendChild(a);\n",
       "      a.click();\n",
       "      div.remove();\n",
       "    }\n",
       "  "
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "download(\"download_46a0f284-7708-4693-a7cc-33883d9d9a7d\", \"history.pkl\", 942)"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def download_history():\n",
    "  import pickle\n",
    "  from google.colab import files\n",
    "\n",
    "  with open('history.pkl', 'wb') as f:\n",
    "    pickle.dump(history.history, f)\n",
    "\n",
    "  files.download('history.pkl')\n",
    "\n",
    "download_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wdsMszk9zBs_"
   },
   "source": [
    "## See your model in action\n",
    "\n",
    "After all your work it is finally time to see your model generating text.\n",
    "\n",
    "Run the cell below to generate the next 100 words of a seed text.\n",
    "\n",
    "After submitting your assignment you are encouraged to try out training for different amounts of epochs and seeing how this affects the coherency of the generated text. Also try changing the seed text to see what you get!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6Vc6PHgxa6Hm",
    "outputId": "5a4cac9d-629e-4782-b98e-0f0877a3fce5",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help me Obi Wan Kenobi, you're my only hope of single life mistaking skill green it loveliness green the very face less of your part sweet policy days forgetful deeds going confounding great rage knows be green me young with thine eyes to write words do go new pen again again again again again again dost odour accusing his rents slanderers doth stand new again again again again breast doth see not forbidden part she still new ill grave ' knows her strong treasure more is not well made treasure more are of men's eyes shall cover new hate gone ' quite penury must be thought ' her own\n"
     ]
    }
   ],
   "source": [
    "seed_text = \"Help me Obi Wan Kenobi, you're my only hope\"\n",
    "next_words = 100\n",
    "\n",
    "for _ in range(next_words):\n",
    "    # Convert the text into sequences\n",
    "    token_list = tokenizer.texts_to_sequences([seed_text])[0]\n",
    "    # Pad the sequences\n",
    "    token_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n",
    "    # Get the probabilities of predicting a word\n",
    "    predicted = model.predict(token_list, verbose=0)\n",
    "    # Choose the next word based on the maximum probability\n",
    "    predicted = np.argmax(predicted, axis=-1).item()\n",
    "    # Get the actual word from the word index\n",
    "    output_word = tokenizer.index_word[predicted]\n",
    "    # Append to the current text\n",
    "    seed_text += \" \" + output_word\n",
    "\n",
    "print(seed_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oQVDbdcYqSux"
   },
   "source": [
    "## Download your notebook for grading\n",
    "\n",
    "Along with the `history.pkl` file earlier, you will also need to submit your solution notebook for grading. The following code cells will check if this notebook's grader metadata (i.e. hidden data in the notebook needed for grading) is not modified by your workspace. This will ensure that the autograder can evaluate your code properly. Depending on its output, you will either:\n",
    "\n",
    "* *if the metadata is intact*: Download the current notebook. Click on the File tab on the upper left corner of the screen then click on `Download -> Download .ipynb.` You can name it anything you want as long as it is a valid `.ipynb` (jupyter notebook) file.\n",
    "<br>\n",
    "\n",
    "* *if the metadata is missing*: A new notebook with your solutions will be created on this Colab workspace. It should be downloaded automatically and you can submit that to the grader."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZUEiIXZEShHw",
    "outputId": "6f19d5d9-0915-446c-a95d-2007c81fff72"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2024-08-13 08:00:21--  https://storage.googleapis.com/tensorflow-1-public/colab_metadata_checker.py\n",
      "Resolving storage.googleapis.com (storage.googleapis.com)... 209.85.145.207, 142.250.125.207, 173.194.206.207, ...\n",
      "Connecting to storage.googleapis.com (storage.googleapis.com)|209.85.145.207|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1997 (2.0K) [text/x-python-script]\n",
      "Saving to: ‘colab_metadata_checker.py’\n",
      "\n",
      "\r",
      "          colab_met   0%[                    ]       0  --.-KB/s               \r",
      "colab_metadata_chec 100%[===================>]   1.95K  --.-KB/s    in 0s      \n",
      "\n",
      "2024-08-13 08:00:21 (9.29 MB/s) - ‘colab_metadata_checker.py’ saved [1997/1997]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Download metadata checker\n",
    "!wget -nc https://storage.googleapis.com/tensorflow-1-public/colab_metadata_checker.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ek87UXX7Sj6b",
    "outputId": "6c224feb-5037-4bf3-9f66-2146f00900ef"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Grader metadata detected! You can download this notebook by clicking `File > Download > Download as .ipynb` and submit it to the grader!\n"
     ]
    }
   ],
   "source": [
    "import colab_metadata_checker\n",
    "\n",
    "# Please see the output of this cell to see which file you need to submit to the grader\n",
    "colab_metadata_checker.run('C3W4_Assignment_fixed.ipynb')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FDi8qq3XmUxq"
   },
   "source": [
    "**Please disregard the following note if the notebook metadata is detected**\n",
    "\n",
    "_Note: Just in case the automatic download fails when the metadata is missing, you can also do these steps:_\n",
    "* _Click the Folder icon on the left side of this screen to open the File Manager._\n",
    "* _Click the Folder Refresh icon in the File Manager to see the latest files in the workspace. You should see a file ending with a `_fixed.ipynb`._\n",
    "* _Right-click on that file to save locally and submit it to the grader._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6r-X-HXtSc8N"
   },
   "source": [
    "**Congratulations on finishing this week's assignment!**\n",
    "\n",
    "You have successfully implemented a neural network capable of predicting the next word in a sequence of text!\n",
    "\n",
    "**We hope to see you in the next course of the specialization! Keep it up!**"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "dlai_version": "1.2.0",
  "jupytext": {
   "main_language": "python"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}