Trabalho_Final_Deep_Learning_Análise_de_Sentimentos.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.7"
    },
    "colab": {
      "name": "Trabalho Final - Deep Learning - Análise de Sentimentos.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "omtMl3UrQoJ-"
      },
      "source": [
        "# Trabalho Final - Deep Learning - Análise de Sentimentos\n",
        "\n",
        "**FURB - Data Science - Turma 02**<br>\n",
        "**Aluno:** Felipe Eduardo Gomes <br>\n",
        "**Baseado no projeto:** https://www.kaggle.com/arunmohan003/sentiment-analysis-using-lstm-pytorch\n",
        "\n",
        "---\n",
        "\n"
      ],
      "id": "omtMl3UrQoJ-"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "94471057"
      },
      "source": [
        "### Classificação de textos para análise de sentimentos\n",
        "\n",
        "Base de dados \n",
        "\n",
        "Instruções:\n",
        "- O objetivo deste trabalho é criar um modelo binário de aprendizado de máquina para classificação de textos. \n",
        "Para isso, será utilizado a base de dados [IMDb](http://ai.stanford.edu/~amaas/data/sentiment/), que consiste de dados textuais de críticas positivas e negativas de filmes\n",
        "- Uma vez treinado, o modelo deve ter uma função `predict` que recebe uma string como parâmetro e retorna o valor 1 ou 0, aonde 1 significa uma crítica positiva e 0 uma crítica negativa\n",
        "- O pré-processamento pode ser desenvolvidado conforme desejar (ex.: remoção de stopwords, word embedding, one-hot encoding, char encoding)\n",
        "- É preferível que seja empregado um modelo de recorrência (ex.: rnn, lstm, gru) para a etapa de classificação\n",
        "- Documente o código (explique sucintamente o que cada função faz, insira comentários em trechos de código relevantes)\n",
        "- **Atenção**: Uma vez treinado o modelo final, salve-o no diretório do seu projeto e crie uma célula ao final do notebook contendo uma função de leitura deste arquivo, juntamente com a execução da função `predict`\n",
        "\n",
        "Sugestões:\n",
        "- Explorar a base de dados nas células iniciais do notebook para ter um melhor entendimento do problema, distribuição dos dados, etc\n",
        "- Após desenvolver a estrutura de classificação, é indicado fazer uma busca de hiperparâmetros e comparar os resultados obtidos em diferentes situações\n",
        "\n",
        "Prazo de entrega:\n",
        "- 01-08-2021 às 23:59hs GMT-3\n",
        "\n",
        "Formato preferível de entrega:\n",
        "- Postar no portal Ava da disciplina o link do projeto no github (ou anexar o projeto diretamente no portal Ava)\n",
        "\n",
        "luann.porfirio@gmail.com"
      ],
      "id": "94471057"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qN1v_AvCShrn"
      },
      "source": [
        "###Imports de biblioteca e verificação de dispositivo de hardware"
      ],
      "id": "qN1v_AvCShrn"
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2021-06-10T00:36:57.559764Z",
          "start_time": "2021-06-10T00:36:52.638020Z"
        },
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2577de36",
        "outputId": "2a7df02b-a1fc-46f7-e7f8-09d91b58c44b"
      },
      "source": [
        "!pip install torchtext"
      ],
      "id": "2577de36",
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: torchtext in /usr/local/lib/python3.7/dist-packages (0.10.0)\n",
            "Requirement already satisfied: torch==1.9.0 in /usr/local/lib/python3.7/dist-packages (from torchtext) (1.9.0+cu102)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from torchtext) (4.41.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torchtext) (1.19.5)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from torchtext) (2.23.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch==1.9.0->torchtext) (3.7.4.3)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->torchtext) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->torchtext) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->torchtext) (2021.5.30)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->torchtext) (3.0.4)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2021-06-10T00:44:12.514627Z",
          "start_time": "2021-06-10T00:44:12.509125Z"
        },
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a80224ad",
        "outputId": "b7c145c5-302f-4e8f-c520-06f2d7d796b1"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "from torchtext import datasets\n",
        "from nltk.corpus import stopwords \n",
        "from collections import Counter\n",
        "from sklearn.model_selection import train_test_split\n",
        "import string\n",
        "import re\n",
        "import seaborn as sns\n",
        "from tqdm import tqdm\n",
        "import matplotlib.pyplot as plt\n",
        "from torch.utils.data import TensorDataset, DataLoader\n",
        "import nltk\n",
        "\n",
        "nltk.download('stopwords')"
      ],
      "id": "a80224ad",
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3arh3ubbZU7t",
        "outputId": "2820a5bc-50f2-432f-c732-9d0e6dfcadfa"
      },
      "source": [
        "if torch.cuda.is_available():\n",
        "  device = torch.device('cuda')\n",
        "else:\n",
        "  device = torch.device('cpu')\n",
        "\n",
        "print('Using {} device'.format(device))"
      ],
      "id": "3arh3ubbZU7t",
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using cuda device\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cy2IbR0DlKOv"
      },
      "source": [
        "### Ativar o Google Drive\n"
      ],
      "id": "cy2IbR0DlKOv"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OWm1Do2_lKjL",
        "outputId": "b189b874-4461-42b7-f2ea-342b9786e1b7"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "id": "OWm1Do2_lKjL",
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wI8xTawnSUuB"
      },
      "source": [
        "### Carregar banco de dados e dividir o dataset em treino e teste."
      ],
      "id": "wI8xTawnSUuB"
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2021-06-10T00:50:27.424355Z",
          "start_time": "2021-06-10T00:49:16.448387Z"
        },
        "id": "907e3626",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d6a971b7-2f4f-4e9e-9d52-0dcfcef1d435"
      },
      "source": [
        "train_iter, test_iter = datasets.IMDB()\n",
        "ds_imdb = list(train_iter + test_iter)"
      ],
      "id": "907e3626",
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "aclImdb_v1.tar.gz: 100%|██████████| 84.1M/84.1M [00:00<00:00, 94.7MB/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-v3DZhBIW09z"
      },
      "source": [
        "###Carregar o dataset para um Dataframe (Pandas)"
      ],
      "id": "-v3DZhBIW09z"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wHn9NDk4Xc9Z"
      },
      "source": [
        "df = pd.DataFrame(data=ds_imdb, columns=['sentiment', 'review'])"
      ],
      "id": "wHn9NDk4Xc9Z",
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RIXO_rQqW6pT"
      },
      "source": [
        "###Exibe a estrutura do dataframe"
      ],
      "id": "RIXO_rQqW6pT"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "DgAxYXV1W7CT",
        "outputId": "2702f0e2-d414-498e-f607-10c958c96619"
      },
      "source": [
        "df.head()"
      ],
      "id": "DgAxYXV1W7CT",
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>review</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>neg</td>\n",
              "      <td>I rented I AM CURIOUS-YELLOW from my video sto...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>neg</td>\n",
              "      <td>\"I Am Curious: Yellow\" is a risible and preten...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>neg</td>\n",
              "      <td>If only to avoid making this type of film in t...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>neg</td>\n",
              "      <td>This film was probably inspired by Godard's Ma...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>neg</td>\n",
              "      <td>Oh, brother...after hearing about this ridicul...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  sentiment                                             review\n",
              "0       neg  I rented I AM CURIOUS-YELLOW from my video sto...\n",
              "1       neg  \"I Am Curious: Yellow\" is a risible and preten...\n",
              "2       neg  If only to avoid making this type of film in t...\n",
              "3       neg  This film was probably inspired by Godard's Ma...\n",
              "4       neg  Oh, brother...after hearing about this ridicul..."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ksP2vPlDYCE8"
      },
      "source": [
        "###Informações do Dataset"
      ],
      "id": "ksP2vPlDYCE8"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PME__XZzYCZT",
        "outputId": "05abce47-3f0e-4a30-8363-c1666e0224eb"
      },
      "source": [
        "df.info()"
      ],
      "id": "PME__XZzYCZT",
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 50000 entries, 0 to 49999\n",
            "Data columns (total 2 columns):\n",
            " #   Column     Non-Null Count  Dtype \n",
            "---  ------     --------------  ----- \n",
            " 0   sentiment  50000 non-null  object\n",
            " 1   review     50000 non-null  object\n",
            "dtypes: object(2)\n",
            "memory usage: 781.4+ KB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rHU9x8-GYbbI"
      },
      "source": [
        "###Exibe o resumo do Dataset"
      ],
      "id": "rHU9x8-GYbbI"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 173
        },
        "id": "88m3WfJ9YbuG",
        "outputId": "dd96b219-55af-4b86-9ac8-98b7d9cf72bd"
      },
      "source": [
        "df.describe()"
      ],
      "id": "88m3WfJ9YbuG",
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>review</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>50000</td>\n",
              "      <td>50000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>unique</th>\n",
              "      <td>2</td>\n",
              "      <td>49582</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>top</th>\n",
              "      <td>neg</td>\n",
              "      <td>Loved today's show!!! It was a variety and not...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>freq</th>\n",
              "      <td>25000</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       sentiment                                             review\n",
              "count      50000                                              50000\n",
              "unique         2                                              49582\n",
              "top          neg  Loved today's show!!! It was a variety and not...\n",
              "freq       25000                                                  5"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DcQpSeK_YkTG"
      },
      "source": [
        "###Separar os dados em treino e teste"
      ],
      "id": "DcQpSeK_YkTG"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8LE8HdyXYkqC",
        "outputId": "14fe4668-a4fa-44e5-d354-2a609632b5ff"
      },
      "source": [
        "X,y = df['review'].values,df['sentiment'].values\n",
        "x_train,x_test,y_train,y_test = train_test_split(X,y,stratify=y)\n",
        "print(f'shape of train data is {x_train.shape}')\n",
        "print(f'shape of test data is {x_test.shape}')"
      ],
      "id": "8LE8HdyXYkqC",
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "shape of train data is (37500,)\n",
            "shape of test data is (12500,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-b0PtskUY7gh"
      },
      "source": [
        "###Plotar análise de sentimentos"
      ],
      "id": "-b0PtskUY7gh"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "p4qaDBhHY8FZ",
        "outputId": "72a0fbe7-b83d-4d32-bc6b-89fe55745247"
      },
      "source": [
        "dd = pd.Series(y_train).value_counts()\n",
        "sns.barplot(x=np.array(['negative','positive']),y=dd.values)\n",
        "plt.show()"
      ],
      "id": "p4qaDBhHY8FZ",
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD4CAYAAAAO9oqkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAATdUlEQVR4nO3df5Bd5X3f8fenkiGObYIwGw2WUCRjkRRoLIcdTJrY40Q1CKYT4YQSqbElO4xlxtCp66apaDuF2iFDarueYeLgyLEGMcHIGEJRGTlYVoPdeKqgla3qByCz/CrSyEgB28S1SyLn2z/us/Wx2JVWe1e7Ar1fM2f2Od/znHOey1ztZ895zr2kqpAkndz+wXQPQJI0/QwDSZJhIEkyDCRJGAaSJGDmdA9gos4888yaP3/+dA9Dkl5Wtm3b9tdVNXB4/WUbBvPnz2doaGi6hyFJLytJnh6t7m0iSZJhIEkyDCRJGAaSJAwDSRKGgSQJw0CShGEgScIwkCTxMv4E8mS48N/cPt1D0Alm28dWTPcQAPjfH/lH0z0EnYDm/cedx+3YXhlIkgwDSZJhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJIlxhEGStUkOJNnVqX0+yfa2PJVke6vPT/KDzrZPd/a5MMnOJMNJbkmSVj8jyaYkj7Wfs47HC5UkjW08Vwa3AUu6har6zapaVFWLgHuAP+tsfnxkW1Vd06nfCrwfWNiWkWOuBjZX1UJgc1uXJE2ho4ZBVX0VeH60be2v+6uAO490jCRnAadV1ZaqKuB24Iq2eSmwrrXXdeqSpCnS75zB24Bnq+qxTm1Bkm8k+UqSt7XaHGBvp8/eVgOYXVX7W/tbwOw+xyRJOkb9fmvpcn78qmA/MK+qnktyIfBfk5w/3oNVVSWpsbYnWQWsApg3b94EhyxJOtyErwySzAR+Hfj8SK2qXqyq51p7G/A4cC6wD5jb2X1uqwE8224jjdxOOjDWOatqTVUNVtXgwMDARIcuSTpMP7eJ/gnwaFX9/9s/SQaSzGjtN9KbKH6i3QZ6IcnFbZ5hBXBf220DsLK1V3bqkqQpMp5HS+8E/ifws0n2Jrm6bVrGSyeO3w7saI+a3g1cU1Ujk88fBP4EGKZ3xfDFVr8ZeGeSx+gFzM19vB5J0gQcdc6gqpaPUX/vKLV76D1qOlr/IeCCUerPAYuPNg5J0vHjJ5AlSYaBJMkwkCRhGEiSMAwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJDGOMEiyNsmBJLs6tRuT7EuyvS2Xd7Zdn2Q4yZ4kl3bqS1ptOMnqTn1Bkr9q9c8nOWUyX6Ak6ejGc2VwG7BklPonq2pRWzYCJDkPWAac3/b5oyQzkswAPgVcBpwHLG99Af6gHetNwLeBq/t5QZKkY3fUMKiqrwLPj/N4S4H1VfViVT0JDAMXtWW4qp6oqr8F1gNLkwT4VeDutv864IpjfA2SpD71M2dwXZId7TbSrFabAzzT6bO31caqvx74TlUdOqw+qiSrkgwlGTp48GAfQ5ckdU00DG4FzgEWAfuBT0zaiI6gqtZU1WBVDQ4MDEzFKSXppDBzIjtV1bMj7SSfAe5vq/uAsztd57YaY9SfA05PMrNdHXT7S5KmyISuDJKc1Vl9FzDypNEGYFmSU5MsABYCDwFbgYXtyaFT6E0yb6iqAv4CuLLtvxK4byJjkiRN3FGvDJLcCbwDODPJXuAG4B1JFgEFPAV8AKCqdie5C3gYOARcW1U/bMe5DngAmAGsrard7RT/Flif5PeAbwCfnbRXJ0kal6OGQVUtH6U85i/sqroJuGmU+kZg4yj1J+g9bSRJmiZ+AlmSZBhIkgwDSRKGgSQJw0CShGEgScIwkCRhGEiSMAwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEuMIgyRrkxxIsqtT+1iSR5PsSHJvktNbfX6SHyTZ3pZPd/a5MMnOJMNJbkmSVj8jyaYkj7Wfs47HC5UkjW08Vwa3AUsOq20CLqiqnwe+CVzf2fZ4VS1qyzWd+q3A+4GFbRk55mpgc1UtBDa3dUnSFDpqGFTVV4HnD6t9qaoOtdUtwNwjHSPJWcBpVbWlqgq4HbiibV4KrGvtdZ26JGmKTMacwW8DX+ysL0jyjSRfSfK2VpsD7O302dtqALOran9rfwuYPdaJkqxKMpRk6ODBg5MwdEkS9BkGSf49cAi4o5X2A/Oq6i3Ah4HPJTltvMdrVw11hO1rqmqwqgYHBgb6GLkkqWvmRHdM8l7gnwKL2y9xqupF4MXW3pbkceBcYB8/fitpbqsBPJvkrKra324nHZjomCRJEzOhK4MkS4DfBX6tqr7fqQ8kmdHab6Q3UfxEuw30QpKL21NEK4D72m4bgJWtvbJTlyRNkaNeGSS5E3gHcGaSvcAN9J4eOhXY1J4Q3dKeHHo78JEkfwf8PXBNVY1MPn+Q3pNJr6Y3xzAyz3AzcFeSq4Gngasm5ZVJksbtqGFQVctHKX92jL73APeMsW0IuGCU+nPA4qONQ5J0/PgJZEmSYSBJMgwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIYZxgkWZvkQJJdndoZSTYleaz9nNXqSXJLkuEkO5L8Qmefla3/Y0lWduoXJtnZ9rkl7X+sLEmaGuO9MrgNWHJYbTWwuaoWApvbOsBlwMK2rAJuhV54ADcAbwUuAm4YCZDW5/2d/Q4/lyTpOBpXGFTVV4HnDysvBda19jrgik799urZApye5CzgUmBTVT1fVd8GNgFL2rbTqmpLVRVwe+dYkqQp0M+cweyq2t/a3wJmt/Yc4JlOv72tdqT63lHqL5FkVZKhJEMHDx7sY+iSpK5JmUBuf9HXZBzrKOdZU1WDVTU4MDBwvE8nSSeNfsLg2XaLh/bzQKvvA87u9Jvbakeqzx2lLkmaIv2EwQZg5ImglcB9nfqK9lTRxcB32+2kB4BLksxqE8eXAA+0bS8kubg9RbSicyxJ0hSYOZ5OSe4E3gGcmWQvvaeCbgbuSnI18DRwVeu+EbgcGAa+D7wPoKqeT/JRYGvr95GqGpmU/iC9J5ZeDXyxLZKkKTKuMKiq5WNsWjxK3wKuHeM4a4G1o9SHgAvGMxZJ0uTzE8iSJMNAkmQYSJIwDCRJGAaSJAwDSRKGgSQJw0CShGEgScIwkCRhGEiSMAwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkugjDJL8bJLtneWFJB9KcmOSfZ365Z19rk8ynGRPkks79SWtNpxkdb8vSpJ0bGZOdMeq2gMsAkgyA9gH3Au8D/hkVX282z/JecAy4HzgDcCXk5zbNn8KeCewF9iaZENVPTzRsUmSjs2Ew+Awi4HHq+rpJGP1WQqsr6oXgSeTDAMXtW3DVfUEQJL1ra9hIElTZLLmDJYBd3bWr0uyI8naJLNabQ7wTKfP3lYbq/4SSVYlGUoydPDgwUkauiSp7zBIcgrwa8AXWulW4Bx6t5D2A5/o9xwjqmpNVQ1W1eDAwMBkHVaSTnqTcZvoMuDrVfUswMhPgCSfAe5vq/uAszv7zW01jlCXJE2BybhNtJzOLaIkZ3W2vQvY1dobgGVJTk2yAFgIPARsBRYmWdCuMpa1vpKkKdLXlUGS19B7CugDnfJ/TrIIKOCpkW1VtTvJXfQmhg8B11bVD9txrgMeAGYAa6tqdz/jkiQdm77CoKr+D/D6w2rvOUL/m4CbRqlvBDb2MxZJ0sT5CWRJkmEgSTIMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIwDCRJGAaSJAwDSRKGgSQJw0CShGEgScIwkCRhGEiSmIQwSPJUkp1JticZarUzkmxK8lj7OavVk+SWJMNJdiT5hc5xVrb+jyVZ2e+4JEnjN1lXBr9SVYuqarCtrwY2V9VCYHNbB7gMWNiWVcCt0AsP4AbgrcBFwA0jASJJOv6O122ipcC61l4HXNGp3149W4DTk5wFXApsqqrnq+rbwCZgyXEamyTpMJMRBgV8Kcm2JKtabXZV7W/tbwGzW3sO8Exn372tNlZdkjQFZk7CMX65qvYl+WlgU5JHuxurqpLUJJyHFjarAObNmzcZh5QkMQlXBlW1r/08ANxL757/s+32D+3ngdZ9H3B2Z/e5rTZW/fBzramqwaoaHBgY6HfokqSmrzBI8pokrxtpA5cAu4ANwMgTQSuB+1p7A7CiPVV0MfDddjvpAeCSJLPaxPElrSZJmgL93iaaDdybZORYn6uqP0+yFbgrydXA08BVrf9G4HJgGPg+8D6Aqno+yUeBra3fR6rq+T7HJkkap77CoKqeAN48Sv05YPEo9QKuHeNYa4G1/YxHkjQxfgJZkmQYSJIMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIwDCRJGAaSJAwDSRJ9hEGSs5P8RZKHk+xO8i9b/cYk+5Jsb8vlnX2uTzKcZE+SSzv1Ja02nGR1fy9JknSsZvax7yHgX1fV15O8DtiWZFPb9smq+ni3c5LzgGXA+cAbgC8nObdt/hTwTmAvsDXJhqp6uI+xSZKOwYTDoKr2A/tb+2+SPALMOcIuS4H1VfUi8GSSYeCitm24qp4ASLK+9TUMJGmKTMqcQZL5wFuAv2ql65LsSLI2yaxWmwM809ltb6uNVR/tPKuSDCUZOnjw4GQMXZLEJIRBktcC9wAfqqoXgFuBc4BF9K4cPtHvOUZU1ZqqGqyqwYGBgck6rCSd9PqZMyDJq+gFwR1V9WcAVfVsZ/tngPvb6j7g7M7uc1uNI9QlSVOgn6eJAnwWeKSq/kunflan27uAXa29AViW5NQkC4CFwEPAVmBhkgVJTqE3ybxhouOSJB27fq4Mfgl4D7AzyfZW+3fA8iSLgAKeAj4AUFW7k9xFb2L4EHBtVf0QIMl1wAPADGBtVe3uY1ySpGPUz9NEfwlklE0bj7DPTcBNo9Q3Hmk/SdLx5SeQJUmGgSTJMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIwDCRJGAaSJAwDSRKGgSQJw0CShGEgScIwkCRxAoVBkiVJ9iQZTrJ6uscjSSeTEyIMkswAPgVcBpwHLE9y3vSOSpJOHidEGAAXAcNV9URV/S2wHlg6zWOSpJPGzOkeQDMHeKazvhd46+GdkqwCVrXV7yXZMwVjO1mcCfz1dA9iuuXjK6d7CHop35sjbshkHOVnRiueKGEwLlW1Blgz3eN4JUoyVFWD0z0O6XC+N6fGiXKbaB9wdmd9bqtJkqbAiRIGW4GFSRYkOQVYBmyY5jFJ0knjhLhNVFWHklwHPADMANZW1e5pHtbJxttvOlH53pwCqarpHoMkaZqdKLeJJEnTyDCQJBkGeqkkpyf5YGf9DUnuns4x6eST5JokK1r7vUne0Nn2J35LweRyzkAvkWQ+cH9VXTDNQ5EASPIg8DtVNTTdY3ml8srgZSjJ/CSPJPlMkt1JvpTk1UnOSfLnSbYl+R9Jfq71PyfJliQ7k/xeku+1+muTbE7y9bZt5CtAbgbOSbI9ycfa+Xa1fbYkOb8zlgeTDCZ5TZK1SR5K8o3OsXQSau+ZR5Pc0d6rdyf5ySSL2/tjZ3u/nNr635zk4SQ7kny81W5M8jtJrgQGgTvae/LVnffdNUk+1jnve5P8YWu/u70ftyf54/YdaBpLVbm8zBZgPnAIWNTW7wLeDWwGFrbaW4H/3tr3A8tb+xrge609Ezittc8EhoG04+867Hy7WvtfAf+ptc8C9rT27wPvbu3TgW8Cr5nu/1Yu0/oeLeCX2vpa4D/Q+9qZc1vtduBDwOuBPfzoTsXp7eeN9K4GAB4EBjvHf5BeQAzQ+16zkfoXgV8G/iHw34BXtfofASum+7/Libx4ZfDy9WRVbW/tbfT+8f1j4AtJtgN/TO+XNcAvAl9o7c91jhHg95PsAL5M7zuiZh/lvHcBV7b2VcDIXMIlwOp27geBnwDmHfOr0ivJM1X1tdb+U2AxvfftN1ttHfB24LvA/wU+m+TXge+P9wRVdRB4IsnFSV4P/BzwtXauC4Gt7T25GHjjJLymV6wT4kNnmpAXO+0f0vsl/p2qWnQMx/gten9ZXVhVf5fkKXq/xMdUVfuSPJfk54HfpHelAb1g+Y2q8ssDNeLwCcnv0LsK+PFOvQ+dXkTvF/aVwHXArx7DedbT+8PkUeDeqqokAdZV1fUTGvlJyCuDV44XgCeT/DOA9Ly5bdsC/EZrL+vs81PAgRYEv8KPvs3wb4DXHeFcnwd+F/ipqtrRag8A/6L9IyTJW/p9QXrZm5fkF1v7nwNDwPwkb2q19wBfSfJaeu+ljfRuQ775pYc64nvyXnpfeb+cXjBA75bplUl+GiDJGUlG/bZO9RgGryy/BVyd5H8Bu/nR/xPiQ8CH2+2gN9G7LAe4AxhMshNYQe8vK6rqOeBrSXZ1J+c67qYXKnd1ah8FXgXsSLK7revktge4NskjwCzgk8D76N3K3An8PfBper/k72/vz78EPjzKsW4DPj0ygdzdUFXfBh4BfqaqHmq1h+nNUXypHXcTP7ptqlH4aOlJIMlPAj9ol8/L6E0m+7SPjhsfT375cc7g5HAh8IftFs53gN+e5vFIOsF4ZSBJcs5AkmQYSJIwDCRJGAaSJAwDSRLw/wA0FUMjVzrRwgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FYwQdpLda_kk"
      },
      "source": [
        "###Tratamento das strings removendo caracteres (com exceção de letras e números) e espaços em branco"
      ],
      "id": "FYwQdpLda_kk"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x8WTIiMba_71"
      },
      "source": [
        "def preprocess_string(s):\n",
        "    # Remove all non-word characters (everything except numbers and letters)\n",
        "    s = re.sub(r\"[^\\w\\s]\", '', s)\n",
        "    # Replace all runs of whitespaces with no space\n",
        "    s = re.sub(r\"\\s+\", '', s)\n",
        "    # replace digits with no space\n",
        "    s = re.sub(r\"\\d\", '', s)\n",
        "\n",
        "    return s"
      ],
      "id": "x8WTIiMba_71",
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ujXGQUYjba3g"
      },
      "source": [
        "###Tokenização de palavras removendo stopwords"
      ],
      "id": "ujXGQUYjba3g"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ET6VGYwAbeqb"
      },
      "source": [
        "def tockenize(x_train,y_train,x_val,y_val):\n",
        "    word_list = []\n",
        "\n",
        "    stop_words = set(stopwords.words('english')) \n",
        "    for sent in x_train:\n",
        "        for word in sent.lower().split():\n",
        "            word = preprocess_string(word)\n",
        "            if word not in stop_words and word != '':\n",
        "                word_list.append(word)\n",
        "  \n",
        "    corpus = Counter(word_list)\n",
        "    # sorting on the basis of most common words\n",
        "    corpus_ = sorted(corpus,key=corpus.get,reverse=True)[:1000]\n",
        "    # creating a dict\n",
        "    onehot_dict = {w:i+1 for i,w in enumerate(corpus_)}\n",
        "    \n",
        "    # tockenize\n",
        "    final_list_train,final_list_test = [],[]\n",
        "    for sent in x_train:\n",
        "            final_list_train.append([onehot_dict[preprocess_string(word)] for word in sent.lower().split() \n",
        "                                     if preprocess_string(word) in onehot_dict.keys()])\n",
        "    for sent in x_val:\n",
        "            final_list_test.append([onehot_dict[preprocess_string(word)] for word in sent.lower().split() \n",
        "                                    if preprocess_string(word) in onehot_dict.keys()])\n",
        "            \n",
        "    encoded_train = [1 if label =='positive' else 0 for label in y_train]  \n",
        "    encoded_test = [1 if label =='positive' else 0 for label in y_val] \n",
        "    return np.array(final_list_train), np.array(encoded_train),np.array(final_list_test), np.array(encoded_test),onehot_dict"
      ],
      "id": "ET6VGYwAbeqb",
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-3ihNZ6Kb5FK",
        "outputId": "7ae2068f-846b-45c8-ebd2-0a9f247b661e"
      },
      "source": [
        "x_train,y_train,x_test,y_test,vocab = tockenize(x_train,y_train,x_test,y_test)"
      ],
      "id": "-3ihNZ6Kb5FK",
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:28: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SPWnZwVpcFsd"
      },
      "source": [
        "###Imprimir tamanho do vocabulário"
      ],
      "id": "SPWnZwVpcFsd"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kcp1Am8UcGMb",
        "outputId": "f2838644-fa64-457a-dd57-c717f3bf9f1b"
      },
      "source": [
        "print(f'Length of vocabulary is {len(vocab)}')"
      ],
      "id": "kcp1Am8UcGMb",
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Length of vocabulary is 1000\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MNZzuPXQcPbF"
      },
      "source": [
        "###Analisando tamanho do review"
      ],
      "id": "MNZzuPXQcPbF"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 418
        },
        "id": "DVJdWi9EcQb5",
        "outputId": "c6a3d1fe-de82-4ee1-eba3-791b47d1db01"
      },
      "source": [
        "rev_len = [len(i) for i in x_train]\n",
        "pd.Series(rev_len).hist()\n",
        "plt.show()\n",
        "pd.Series(rev_len).describe()"
      ],
      "id": "DVJdWi9EcQb5",
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD4CAYAAAAO9oqkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAQuklEQVR4nO3df6zddX3H8edr1B8Mf7SIu2koWTE2GiYT8QZqNMsVslJgGSwhRkKkOmaXWBJNmsyyJWOTmeAf6AZzZN3sKAkTmT/WBtHaVW7M/gApipQfsl6xhDZAp0VYMdHVvffH+Vx2Vm97b8+9ved+7fORnJzv932+53te53Lo657v/d5zU1VIkk5svzbsAJKk4bMMJEmWgSTJMpAkYRlIkoBFww4wqNNOO62WL18+0H1feuklTjnllLkNNE+6mr2ruaG72buaG8x+PD344IM/qqo3Hj7vbBksX76cnTt3DnTf8fFxxsbG5jbQPOlq9q7mhu5m72puMPvxlOSpqeYeJpIkWQaSJMtAkoRlIEnCMpAkYRlIkrAMJElYBpIkLANJEh3+DeTZ2LXvBT644avz/rh7brx03h9TkmbCdwaSJMtAkmQZSJKwDCRJWAaSJCwDSRKWgSQJy0CShGUgScIykCRhGUiSsAwkSVgGkiQsA0kSloEkCctAkoRlIEnCMpAkYRlIkrAMJElYBpIkLANJEpaBJAnLQJKEZSBJwjKQJDGDMkhyRpJ7kzyW5NEkH23zU5NsT7K7XS9p8yS5OclEkoeTnNu3rzVt+91J1vTN35lkV7vPzUlyPJ6sJGlqM3lncAhYX1VnASuBdUnOAjYAO6pqBbCjrQNcDKxol7XArdArD+B64HzgPOD6yQJp23y4736rZ//UJEkzNW0ZVNUzVfWdtvxfwOPA6cBlwOa22Wbg8rZ8GXB79dwHLE6yFLgI2F5VB6rqeWA7sLrd9rqquq+qCri9b1+SpHmw6Fg2TrIceAdwPzBSVc+0m54FRtry6cDTfXfb22ZHm++dYj7V46+l926DkZERxsfHjyX+y0ZOhvVnHxrovrMxaN5+Bw8enJP9zLeu5obuZu9qbjD7MMy4DJK8BvgS8LGqerH/sH5VVZI6Dvn+n6raCGwEGB0drbGxsYH2c8sdW7hp1zH14JzYc9XYrPcxPj7OoM97mLqaG7qbvau5wezDMKOziZK8gl4R3FFVX27j59ohHtr1/jbfB5zRd/dlbXa0+bIp5pKkeTKTs4kCfA54vKo+3XfTVmDyjKA1wJa++dXtrKKVwAvtcNI2YFWSJe0Hx6uAbe22F5OsbI91dd++JEnzYCbHSt4NfADYleShNvtT4EbgriTXAE8B72u33QNcAkwAPwU+BFBVB5LcADzQtvtEVR1oyx8BbgNOBr7WLpKkeTJtGVTVvwNHOu//wim2L2DdEfa1Cdg0xXwn8LbpskiSjg9/A1mSZBlIkiwDSRKWgSQJy0CShGUgScIykCRhGUiSsAwkSVgGkiQsA0kSloEkCctAkoRlIEnCMpAkYRlIkrAMJElYBpIkLANJEpaBJAnLQJKEZSBJwjKQJGEZSJKwDCRJWAaSJCwDSRKWgSQJy0CShGUgScIykCRhGUiSsAwkSVgGkiQsA0kSMyiDJJuS7E/ySN/sL5LsS/JQu1zSd9t1SSaSPJHkor756jabSLKhb35mkvvb/AtJXjmXT1CSNL2ZvDO4DVg9xfwzVXVOu9wDkOQs4P3Ab7X7/F2Sk5KcBHwWuBg4C7iybQvwqbavNwPPA9fM5glJko7dtGVQVd8CDsxwf5cBd1bVz6rqh8AEcF67TFTVk1X1c+BO4LIkAS4Avtjuvxm4/BifgyRplhbN4r7XJrka2Amsr6rngdOB+/q22dtmAE8fNj8feAPwk6o6NMX2vyTJWmAtwMjICOPj4wMFHzkZ1p99aPoN59igefsdPHhwTvYz37qaG7qbvau5wezDMGgZ3ArcAFS7vgn4w7kKdSRVtRHYCDA6OlpjY2MD7eeWO7Zw067Z9OBg9lw1Nut9jI+PM+jzHqau5obuZu9qbjD7MAz0L2JVPTe5nOQfgLvb6j7gjL5Nl7UZR5j/GFicZFF7d9C/vSRpngx0ammSpX2rfwBMnmm0FXh/klclORNYAXwbeABY0c4ceiW9HzJvraoC7gWuaPdfA2wZJJMkaXDTvjNI8nlgDDgtyV7gemAsyTn0DhPtAf4YoKoeTXIX8BhwCFhXVb9o+7kW2AacBGyqqkfbQ3wcuDPJXwHfBT43Z89OkjQj05ZBVV05xfiI/2BX1SeBT04xvwe4Z4r5k/TONpIkDYm/gSxJsgwkSZaBJAnLQJKEZSBJwjKQJGEZSJKwDCRJWAaSJCwDSRKWgSQJy0CShGUgScIykCRhGUiSsAwkSVgGkiQsA0kSloEkCctAkoRlIEnCMpAkYRlIkrAMJElYBpIkLANJEpaBJAnLQJKEZSBJwjKQJGEZSJKwDCRJWAaSJCwDSRKWgSSJGZRBkk1J9id5pG92apLtSXa36yVtniQ3J5lI8nCSc/vus6ZtvzvJmr75O5Psave5OUnm+klKko5u0Qy2uQ34W+D2vtkGYEdV3ZhkQ1v/OHAxsKJdzgduBc5PcipwPTAKFPBgkq1V9Xzb5sPA/cA9wGrga7N/agvP8g1fnfU+1p99iA8OsJ89N14668eW9Ktr2ncGVfUt4MBh48uAzW15M3B53/z26rkPWJxkKXARsL2qDrQC2A6sbre9rqruq6qiVziXI0maVzN5ZzCVkap6pi0/C4y05dOBp/u229tmR5vvnWI+pSRrgbUAIyMjjI+PDxb+5N532F00aPZBv1Zz5eDBg0PPMKiuZu9qbjD7MAxaBi+rqkpScxFmBo+1EdgIMDo6WmNjYwPt55Y7tnDTrlk/9aFYf/ahgbLvuWps7sMcg/HxcQb97zVsXc3e1dxg9mEY9Gyi59ohHtr1/jbfB5zRt92yNjvafNkUc0nSPBq0DLYCk2cErQG29M2vbmcVrQReaIeTtgGrkixpZx6tAra1215MsrKdRXR1374kSfNk2uMNST4PjAGnJdlL76ygG4G7klwDPAW8r21+D3AJMAH8FPgQQFUdSHID8EDb7hNVNflD6Y/QO2PpZHpnEf1KnkkkSQvZtGVQVVce4aYLp9i2gHVH2M8mYNMU853A26bLIUk6fvwNZEmSZSBJsgwkSVgGkiQsA0kSloEkCctAkoRlIEnCMpAkYRlIkrAMJElYBpIkLANJEpaBJAnLQJKEZSBJwjKQJGEZSJKwDCRJWAaSJCwDSRKWgSQJy0CShGUgScIykCRhGUiSsAwkSVgGkiQsA0kSloEkCctAkoRlIEnCMpAkYRlIkrAMJEnMsgyS7EmyK8lDSXa22alJtifZ3a6XtHmS3JxkIsnDSc7t28+atv3uJGtm95QkScdqLt4ZvLeqzqmq0ba+AdhRVSuAHW0d4GJgRbusBW6FXnkA1wPnA+cB108WiCRpfhyPw0SXAZvb8mbg8r757dVzH7A4yVLgImB7VR2oqueB7cDq45BLknQEsy2DAr6R5MEka9tspKqeacvPAiNt+XTg6b777m2zI80lSfNk0Szv/56q2pfkN4DtSb7ff2NVVZKa5WO8rBXOWoCRkRHGx8cH2s/IybD+7ENzFWteDZp90K/VXDl48ODQMwyqq9m7mhvMPgyzKoOq2teu9yf5Cr1j/s8lWVpVz7TDQPvb5vuAM/ruvqzN9gFjh83Hj/B4G4GNAKOjozU2NjbVZtO65Y4t3LRrtj04HOvPPjRQ9j1Xjc19mGMwPj7OoP+9hq2r2buaG8w+DAMfJkpySpLXTi4Dq4BHgK3A5BlBa4AtbXkrcHU7q2gl8EI7nLQNWJVkSfvB8ao2kyTNk9l8ezwCfCXJ5H7+uaq+nuQB4K4k1wBPAe9r298DXAJMAD8FPgRQVQeS3AA80Lb7RFUdmEUuSdIxGrgMqupJ4O1TzH8MXDjFvIB1R9jXJmDToFkkSbPjbyBLkiwDSZJlIEnCMpAkYRlIkrAMJElYBpIkLANJEpaBJAnLQJKEZSBJwjKQJGEZSJKwDCRJWAaSJGb/N5DVEcs3fHUoj7vnxkuH8riSjo3vDCRJloEkyTKQJGEZSJKwDCRJWAaSJCwDSRKWgSQJy0CShGUgScIykCRhGUiSsAwkSVgGkiQsA0kS/j0DHWeTf0dh/dmH+OA8/00F/5aCNHO+M5AkWQaSJMtAkoRlIEliAZVBktVJnkgykWTDsPNI0olkQZxNlOQk4LPA7wJ7gQeSbK2qx4abTF22fI7OXjrWM6E8i0ldtFDeGZwHTFTVk1X1c+BO4LIhZ5KkE0aqatgZSHIFsLqq/qitfwA4v6quPWy7tcDatvoW4IkBH/I04EcD3nfYupq9q7mhu9m7mhvMfjz9ZlW98fDhgjhMNFNVtRHYONv9JNlZVaNzEGnedTV7V3NDd7N3NTeYfRgWymGifcAZfevL2kySNA8WShk8AKxIcmaSVwLvB7YOOZMknTAWxGGiqjqU5FpgG3ASsKmqHj2ODznrQ01D1NXsXc0N3c3e1dxg9nm3IH6ALEkaroVymEiSNESWgSTpxCqDhf6RF0k2Jdmf5JG+2alJtifZ3a6XtHmS3Nyey8NJzh1i7jOS3JvksSSPJvloh7K/Osm3k3yvZf/LNj8zyf0t4xfaiQ0keVVbn2i3Lx9W9pbnpCTfTXJ3x3LvSbIryUNJdrbZgn+9tDyLk3wxyfeTPJ7kXV3JfjQnTBn0feTFxcBZwJVJzhpuql9yG7D6sNkGYEdVrQB2tHXoPY8V7bIWuHWeMk7lELC+qs4CVgLr2te2C9l/BlxQVW8HzgFWJ1kJfAr4TFW9GXgeuKZtfw3wfJt/pm03TB8FHu9b70pugPdW1Tl95+R34fUC8DfA16vqrcDb6X39u5L9yKrqhLgA7wK29a1fB1w37FxT5FwOPNK3/gSwtC0vBZ5oy38PXDnVdsO+AFvofc5Up7IDvw58Bzif3m+QLjr8tUPvjLd3teVFbbsMKe8yev/wXADcDaQLuVuGPcBph80W/OsFeD3ww8O/dl3IPt3lhHlnAJwOPN23vrfNFrqRqnqmLT8LjLTlBfl82uGHdwD305Hs7VDLQ8B+YDvwA+AnVXVoinwvZ2+3vwC8YX4Tv+yvgT8B/qetv4Fu5AYo4BtJHmwfMwPdeL2cCfwn8E/t8Nw/JjmFbmQ/qhOpDDqvet9aLNhzgZO8BvgS8LGqerH/toWcvap+UVXn0PtO+zzgrUOONK0kvwfsr6oHh51lQO+pqnPpHUZZl+R3+m9cwK+XRcC5wK1V9Q7gJf7vkBCwoLMf1YlUBl39yIvnkiwFaNf723xBPZ8kr6BXBHdU1ZfbuBPZJ1XVT4B76R1eWZxk8pcy+/O9nL3d/nrgx/McFeDdwO8n2UPvU34voHcse6HnBqCq9rXr/cBX6JVwF14ve4G9VXV/W/8ivXLoQvajOpHKoKsfebEVWNOW19A7Hj85v7qdrbASeKHvbeq8ShLgc8DjVfXpvpu6kP2NSRa35ZPp/azjcXqlcEXb7PDsk8/pCuCb7TvBeVVV11XVsqpaTu+1/M2quooFnhsgySlJXju5DKwCHqEDr5eqehZ4Oslb2uhC4DE6kH1aw/6hxXxegEuA/6B3TPjPhp1ninyfB54B/pvedyDX0DuuuwPYDfwbcGrbNvTOjvoBsAsYHWLu99B7W/ww8FC7XNKR7L8NfLdlfwT48zZ/E/BtYAL4F+BVbf7qtj7Rbn/TAnjdjAF3dyV3y/i9dnl08v/FLrxeWp5zgJ3tNfOvwJKuZD/axY+jkCSdUIeJJElHYBlIkiwDSZJlIEnCMpAkYRlIkrAMJEnA/wIiwvbswmcFIQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count    37500.000000\n",
              "mean        69.080693\n",
              "std         47.958646\n",
              "min          0.000000\n",
              "25%         39.000000\n",
              "50%         54.000000\n",
              "75%         84.000000\n",
              "max        653.000000\n",
              "dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PGMK71KCcevO"
      },
      "source": [
        "###Padding - Preenche as sequencias até o máximo de comprimento"
      ],
      "id": "PGMK71KCcevO"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2UILCUVScfLo"
      },
      "source": [
        "def padding_(sentences, seq_len):\n",
        "    features = np.zeros((len(sentences), seq_len),dtype=int)\n",
        "    for ii, review in enumerate(sentences):\n",
        "        if len(review) != 0:\n",
        "            features[ii, -len(review):] = np.array(review)[:seq_len]\n",
        "    return features"
      ],
      "id": "2UILCUVScfLo",
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AUx7l4hIcv1o"
      },
      "source": [
        "###Temos um número muito pequeno de reviews com tamanho maior que 500. Iremos considerar somente os reviews abaixo desta quantidade."
      ],
      "id": "AUx7l4hIcv1o"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SCJozyMNcwLr"
      },
      "source": [
        "x_train_pad = padding_(x_train,500)\n",
        "x_test_pad = padding_(x_test,500)"
      ],
      "id": "SCJozyMNcwLr",
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_RKDvsNgdM0m"
      },
      "source": [
        "###Definindo os lotes e carregando como tensor"
      ],
      "id": "_RKDvsNgdM0m"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DT_EHoT2dNJ7"
      },
      "source": [
        "# Criando Tensor datasets\n",
        "train_data = TensorDataset(torch.from_numpy(x_train_pad), torch.from_numpy(y_train))\n",
        "valid_data = TensorDataset(torch.from_numpy(x_test_pad), torch.from_numpy(y_test))\n",
        "\n",
        "# dataloaders\n",
        "batch_size = 50\n",
        "\n",
        "# Embaralhando os dados\n",
        "train_loader = DataLoader(train_data, shuffle=True, batch_size=batch_size)\n",
        "valid_loader = DataLoader(valid_data, shuffle=True, batch_size=batch_size)"
      ],
      "id": "DT_EHoT2dNJ7",
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EQorw85Bdnwm"
      },
      "source": [
        "### Obtendo um lote de dados de treino"
      ],
      "id": "EQorw85Bdnwm"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xQocChXOdoIg",
        "outputId": "fa78e0c5-0aba-4a86-c862-fa188205f620"
      },
      "source": [
        "dataiter = iter(train_loader)\n",
        "sample_x, sample_y = dataiter.next()\n",
        "\n",
        "print('Sample input size: ', sample_x.size()) # batch_size, seq_length\n",
        "print('Sample input: \\n', sample_x)\n",
        "print('Sample input: \\n', sample_y)"
      ],
      "id": "xQocChXOdoIg",
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sample input size:  torch.Size([50, 500])\n",
            "Sample input: \n",
            " tensor([[  0,   0,   0,  ...,  11,   2, 236],\n",
            "        [  0,   0,   0,  ..., 374,   3, 416],\n",
            "        [  0,   0,   0,  ...,  35,  15,  55],\n",
            "        ...,\n",
            "        [  0,   0,   0,  ..., 234, 410, 503],\n",
            "        [  0,   0,   0,  ...,   1, 286,   1],\n",
            "        [  0,   0,   0,  ...,   8,  15, 550]])\n",
            "Sample input: \n",
            " tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
            "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
            "        0, 0])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pcCtqKVIeSzJ"
      },
      "source": [
        "Precisamos adicionar uma camada de incorporação porque há poucas palavras em nosso vocabulário. É extremamente ineficiente para codificar one-hot tantas classes. Portanto, em vez de codificação one-hot, podemos ter uma camada de incorporação e usar essa camada como uma tabela de pesquisa. É possível treinar uma camada de incorporação usando Word2Vec e carregá-la aqui. Mas não há problema em apenas fazer uma nova camada, usando-a apenas para redução de dimensionalidade, e deixar a rede aprender os pesos. "
      ],
      "id": "pcCtqKVIeSzJ"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2hj7ZEW5esiT"
      },
      "source": [
        "### Implementando o Modelo"
      ],
      "id": "2hj7ZEW5esiT"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CBBkNGsWes0C"
      },
      "source": [
        "class SentimentRNN(nn.Module):\n",
        "    def __init__(self,no_layers,vocab_size,hidden_dim,embedding_dim,drop_prob=0.5):\n",
        "        super(SentimentRNN,self).__init__()\n",
        " \n",
        "        self.output_dim = output_dim\n",
        "        self.hidden_dim = hidden_dim\n",
        " \n",
        "        self.no_layers = no_layers\n",
        "        self.vocab_size = vocab_size\n",
        "    \n",
        "        # embedding e camadas LSTM\n",
        "        self.embedding = nn.Embedding(vocab_size, embedding_dim)\n",
        "        \n",
        "        #lstm\n",
        "        self.lstm = nn.LSTM(input_size=embedding_dim,hidden_size=self.hidden_dim,\n",
        "                           num_layers=no_layers, batch_first=True)\n",
        "        \n",
        "        \n",
        "        # camada dropout\n",
        "        self.dropout = nn.Dropout(0.3)\n",
        "    \n",
        "        # camada linear e sigmoid\n",
        "        self.fc = nn.Linear(self.hidden_dim, output_dim)\n",
        "        self.sig = nn.Sigmoid()\n",
        "        \n",
        "    def forward(self,x,hidden):\n",
        "        batch_size = x.size(0)\n",
        "        # embeddings e lstm_out\n",
        "        embeds = self.embedding(x)  # shape: B x S x Feature   since batch = True\n",
        "        #print(embeds.shape)  #[50, 500, 1000]\n",
        "        lstm_out, hidden = self.lstm(embeds, hidden)\n",
        "        \n",
        "        lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim) \n",
        "        \n",
        "        # dropout e camada totalmente conectada\n",
        "        out = self.dropout(lstm_out)\n",
        "        out = self.fc(out)\n",
        "        \n",
        "        # função sigmoid\n",
        "        sig_out = self.sig(out)\n",
        "        \n",
        "        # reshape para batch_size primeirp\n",
        "        sig_out = sig_out.view(batch_size, -1)\n",
        "\n",
        "        sig_out = sig_out[:, -1] # pegar último batch de labels\n",
        "        \n",
        "        # retornar última saída sigmoid e hidden state\n",
        "        return sig_out, hidden\n",
        "        \n",
        "        \n",
        "        \n",
        "    def init_hidden(self, batch_size):\n",
        "        ''' Initializes hidden state '''\n",
        "        # Criar dois novos tensors com tamanhos n_layers x batch_size x hidden_dim,\n",
        "        # Inicializado com zero, para hidden state e cell state do LSTM\n",
        "        h0 = torch.zeros((self.no_layers,batch_size,self.hidden_dim)).to(device)\n",
        "        c0 = torch.zeros((self.no_layers,batch_size,self.hidden_dim)).to(device)\n",
        "        hidden = (h0,c0)\n",
        "        return hidden"
      ],
      "id": "CBBkNGsWes0C",
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oC7ek8V-fuBI",
        "outputId": "359cb79e-fb63-4789-88d3-984a94b78a19"
      },
      "source": [
        "no_layers = 2\n",
        "vocab_size = len(vocab) + 1 # 1 extra para padding\n",
        "embedding_dim = 64\n",
        "output_dim = 1\n",
        "hidden_dim = 256\n",
        "\n",
        "\n",
        "model = SentimentRNN(no_layers,vocab_size,hidden_dim,embedding_dim,drop_prob=0.5)\n",
        "\n",
        "# Processar na GPU se estiver disponível\n",
        "model.to(device)\n",
        "\n",
        "print(model)"
      ],
      "id": "oC7ek8V-fuBI",
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "SentimentRNN(\n",
            "  (embedding): Embedding(1001, 64)\n",
            "  (lstm): LSTM(64, 256, num_layers=2, batch_first=True)\n",
            "  (dropout): Dropout(p=0.3, inplace=False)\n",
            "  (fc): Linear(in_features=256, out_features=1, bias=True)\n",
            "  (sig): Sigmoid()\n",
            ")\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lDFF0-ZfhQYW"
      },
      "source": [
        "###Hiperparâmetros"
      ],
      "id": "lDFF0-ZfhQYW"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BnM0AFQWhNca"
      },
      "source": [
        "lr=0.001 # Funções de perda e otimização\n",
        "clip = 5\n",
        "epochs = 5 # Épocas"
      ],
      "id": "BnM0AFQWhNca",
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r-USJMJYgb0o"
      },
      "source": [
        "###Treinando"
      ],
      "id": "r-USJMJYgb0o"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YQ1ciOqlgcHb"
      },
      "source": [
        "criterion = nn.BCELoss()\n",
        "\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=lr)\n",
        "\n",
        "# Função para calcular acurácia\n",
        "def acc(pred,label):\n",
        "    pred = torch.round(pred.squeeze())\n",
        "    return torch.sum(pred == label.squeeze()).item()"
      ],
      "id": "YQ1ciOqlgcHb",
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bRBFLhZVguTj",
        "outputId": "88e5d828-595b-4e20-a240-629f77e79cbc"
      },
      "source": [
        "valid_loss_min = np.Inf\n",
        "# Treinar algumas épocas\n",
        "epoch_tr_loss,epoch_vl_loss = [],[]\n",
        "epoch_tr_acc,epoch_vl_acc = [],[]\n",
        "\n",
        "for epoch in range(epochs):\n",
        "    train_losses = []\n",
        "    train_acc = 0.0\n",
        "    model.train()\n",
        "    # Inicializar o hidden state \n",
        "    h = model.init_hidden(batch_size)\n",
        "    for inputs, labels in train_loader:\n",
        "        \n",
        "        inputs, labels = inputs.to(device), labels.to(device)   \n",
        "        # Criando novas variáveis para o hidden state, caso contrário\n",
        "        # voltaríamos a propagação por todo o histórico de treinamento.\n",
        "        h = tuple([each.data for each in h])\n",
        "        \n",
        "        model.zero_grad()\n",
        "        output,h = model(inputs,h)\n",
        "        \n",
        "        # Calcular a perda e fazer o backpropagation\n",
        "        loss = criterion(output.squeeze(), labels.float())\n",
        "        loss.backward()\n",
        "        train_losses.append(loss.item())\n",
        "        # Calcular acurácia\n",
        "        accuracy = acc(output,labels)\n",
        "        train_acc += accuracy\n",
        "        #`clip_grad_norm` ajuda a prevenir o problema de gradiente explosivo em RNNs / LSTMs.\n",
        "        nn.utils.clip_grad_norm_(model.parameters(), clip)\n",
        "        optimizer.step()\n",
        " \n",
        "    \n",
        "        \n",
        "    val_h = model.init_hidden(batch_size)\n",
        "    val_losses = []\n",
        "    val_acc = 0.0\n",
        "    model.eval()\n",
        "    for inputs, labels in valid_loader:\n",
        "            val_h = tuple([each.data for each in val_h])\n",
        "\n",
        "            inputs, labels = inputs.to(device), labels.to(device)\n",
        "\n",
        "            output, val_h = model(inputs, val_h)\n",
        "            val_loss = criterion(output.squeeze(), labels.float())\n",
        "\n",
        "            val_losses.append(val_loss.item())\n",
        "            \n",
        "            accuracy = acc(output,labels)\n",
        "            val_acc += accuracy\n",
        "            \n",
        "    epoch_train_loss = np.mean(train_losses)\n",
        "    epoch_val_loss = np.mean(val_losses)\n",
        "    epoch_train_acc = train_acc/len(train_loader.dataset)\n",
        "    epoch_val_acc = val_acc/len(valid_loader.dataset)\n",
        "    epoch_tr_loss.append(epoch_train_loss)\n",
        "    epoch_vl_loss.append(epoch_val_loss)\n",
        "    epoch_tr_acc.append(epoch_train_acc)\n",
        "    epoch_vl_acc.append(epoch_val_acc)\n",
        "    print(f'Epoch {epoch+1}') \n",
        "    print(f'train_loss : {epoch_train_loss} val_loss : {epoch_val_loss}')\n",
        "    print(f'train_accuracy : {epoch_train_acc*100} val_accuracy : {epoch_val_acc*100}')\n",
        "    #if epoch_val_loss <= valid_loss_min:\n",
        "    #    torch.save(model.state_dict(), '../working/state_dict.pt')\n",
        "    #    print('Validation loss decreased ({:.6f} --> {:.6f}).  Saving model ...'.format(valid_loss_min,epoch_val_loss))\n",
        "    #    valid_loss_min = epoch_val_loss\n",
        "    print(25*'==')\n",
        "\n",
        "\n",
        "# Salvar o modelo treinado\n",
        "model_path = '/content/drive/MyDrive/FURB/trabalho_deep_learning_state_dict.pt'\n",
        "torch.save(model.state_dict(), model_path)"
      ],
      "id": "bRBFLhZVguTj",
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1\n",
            "train_loss : 0.004931729753044541 val_loss : 1.060968224919634e-05\n",
            "train_accuracy : 99.86933333333333 val_accuracy : 100.0\n",
            "==================================================\n",
            "Epoch 2\n",
            "train_loss : 7.333997688571495e-06 val_loss : 3.6358896977617405e-06\n",
            "train_accuracy : 100.0 val_accuracy : 100.0\n",
            "==================================================\n",
            "Epoch 3\n",
            "train_loss : 3.0584739018498414e-06 val_loss : 1.7881409348774469e-06\n",
            "train_accuracy : 100.0 val_accuracy : 100.0\n",
            "==================================================\n",
            "Epoch 4\n",
            "train_loss : 1.6032143166739843e-06 val_loss : 9.536747711536009e-07\n",
            "train_accuracy : 100.0 val_accuracy : 100.0\n",
            "==================================================\n",
            "Epoch 5\n",
            "train_loss : 9.357411527162185e-07 val_loss : 5.960466182841628e-07\n",
            "train_accuracy : 100.0 val_accuracy : 100.0\n",
            "==================================================\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "21ZDs8VfiYhw"
      },
      "source": [
        "### Plotar gráfico de Acurácia x Perda"
      ],
      "id": "21ZDs8VfiYhw"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DaQ20HhtiZTg",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 341
        },
        "outputId": "ee3ad31b-4d24-49f3-aaa3-4a136a6b002c"
      },
      "source": [
        "fig = plt.figure(figsize = (20, 6))\n",
        "plt.subplot(1, 2, 1)\n",
        "plt.plot(epoch_tr_acc, label='Train Acc')\n",
        "plt.plot(epoch_vl_acc, label='Validation Acc')\n",
        "plt.title(\"Accuracy\")\n",
        "plt.legend()\n",
        "plt.grid()\n",
        "    \n",
        "plt.subplot(1, 2, 2)\n",
        "plt.plot(epoch_tr_loss, label='Train loss')\n",
        "plt.plot(epoch_vl_loss, label='Validation loss')\n",
        "plt.title(\"Loss\")\n",
        "plt.legend()\n",
        "plt.grid()\n",
        "\n",
        "plt.show()"
      ],
      "id": "DaQ20HhtiZTg",
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABJQAAAF1CAYAAAC+irDFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde3jedZ3n/+c7B5qeSw+UtgkU5FigxwRkC1hEOctJ2hRXhfV0DT91dVAGHBUYRmZ1h1V3x8Mu44iHdSapVbAICgpEUFSStlAotFAQmrSlpaUtSXpIk3x+f+SGjaW0aZvkm+R+Pq6rV+98vp/v9369qdfMfb3z+XzuSCkhSZIkSZIkdVVB1gEkSZIkSZLUv9hQkiRJkiRJ0n6xoSRJkiRJkqT9YkNJkiRJkiRJ+8WGkiRJkiRJkvaLDSVJkiRJkiTtFxtKkiRJkiRJ2i82lCT1iIioiYjNETEo6yySJEnas4h4KSLek3UOSf2PDSVJ3S4iJgNnAgm4pBfft6i33kuSJEmS8pkNJUk94cPAn4AfAFe/MRgRZRHx84h4NSI2RcS3Ol37eEQ8GxGNEfFMRMzMjaeIOKbTvB9ExFdyr+dERENE3BARrwB3RsShEfHL3Htszr0u7XT/6Ii4MyLW5q7fnRt/OiLe12lecURsjIgZPfZfSZIkqQ+KiEER8c3c56W1udeDctfG5j5fbYmI1yLi0YgoyF27ISLW5D7PrYyIc7KtRFJPsqEkqSd8GPhJ7s95ETE+IgqBXwIvA5OBSUAVQETMBW7J3TeCjlVNm7r4XocDo4EjgU/Q8X/X7sz9fASwHfhWp/k/BoYAJwGHAd/Ijf8I+GCneRcC61JKS7uYQ5IkaaD4IvBOYDowDTgV+FLu2ueABmAcMB74eyBFxPHAp4CKlNJw4Dzgpd6NLak3uT1EUreKiDPoaOYsSCltjIgXgA/QsWJpInB9Sqk1N/33ub8/Bvz3lFJt7udV+/GW7cDNKaWduZ+3Az/rlOc24OHc6wnABcCYlNLm3JTf5f7+v8CXI2JESul14EN0NJ8kSZLyzX8GPp1S2gAQEf8A/B/gy8AuYAJwZEppFfBobk4bMAiYEhGvppReyiK4pN7jCiVJ3e1q4IGU0sbcz/+eGysDXu7UTOqsDHjhAN/v1ZTSjjd+iIghEfF/IuLliHgdeAQYlVshVQa81qmZ9KaU0lrgD8D7I2IUHY2nnxxgJkmSpP5sIh2ryt/wcm4M4J/p+OXfAxHxYkTcCJBrLn2WjlXnGyKiKiImImnAsqEkqdtExGBgHvCuiHgld67R39KxVHo9cMTbHJxdD7zjbR67jY4tam84fLfrabefPwccD5yWUhoBnPVGvNz7jM41jPbkh3Rse5sL/DGltOZt5kmSJA1ka+lYcf6GI3JjpJQaU0qfSykdTccxBde9cVZSSunfU0pvrFZPwNd6N7ak3mRDSVJ3ugxoA6bQsed+OnAiHUuhLwPWAV+NiKERURIRs3P3fQ/4fETMig7HRMQbH2KeAD4QEYURcT7wrn1kGE7HtrctETEauPmNCymldcCvgO/kDu8ujoizOt17NzAT+AwdZypJkiTlg+LcZ7OSiCgB/gP4UkSMi4ixwE10HA9ARFyc+6wWwFY6Pvu1R8TxEfHu3OHdO+j4PNaeTTmSeoMNJUnd6WrgzpTS6pTSK2/8oeNQ7KuA9wHHAKvpOMyxEiCl9FPgNjq2xzXS0dgZnXvmZ3L3baFjP//d+8jwTWAwsJGOc5t+vdv1D9Gx938FsIGOpdnkcrxx/tJRwM/3s3ZJkqT+6j46GkBv/CkB6oBlwFPAEuArubnHAr8FmoA/At9JKT1Mx/lJX6XjM9grdHz5yRd6rwRJvS1S2n23iCTlr4i4CTgupfTBfU6WJEmSpDzlt7xJUk5ui9xH6VjFJEmSJEl6G255kyQgIj5Ox6Hdv0opPZJ1HkmSJEnqy9zyJkmSJEmSpP3iCiVJkiRJkiTtFxtKkiRJkiRJ2i8D4lDusWPHpsmTJ/fIs5ubmxk6dGiPPLsvsc6BIx9qhPyoMx9qBOscSHqyxsWLF29MKY3rkYfrgPkZ7ODlQ535UCNY50CSDzVCftSZDzVCdp/BBkRDafLkydTV1fXIs2tqapgzZ06PPLsvsc6BIx9qhPyoMx9qBOscSHqyxoh4uUcerIPiZ7CDlw915kONYJ0DST7UCPlRZz7UCNl9BnPLmyRJkiRJkvaLDSVJkiRJkiTtFxtKkiRJkiRJ2i8D4gwlSZIkSZLUt+zatYuGhgZ27NiRyfuPHDmSZ599NpP37k3dUWdJSQmlpaUUFxd3+R4bSpIkSZIkqds1NDQwfPhwJk+eTET0+vs3NjYyfPjwXn/f3nawdaaU2LRpEw0NDRx11FFdvs8tb5IkSf1MRJwfESsjYlVE3LiH64Miojp3/c8RMbnTtS/kxldGxHmdxl+KiKci4omI6JmvbpMk5ZUdO3YwZsyYTJpJ6rqIYMyYMfu9kswVSpIkSf1IRBQC3wbeCzQAtRGxKKX0TKdpHwU2p5SOiYj5wNeAyoiYAswHTgImAr+NiONSSm25+85OKW3stWIkSQOezaT+4UD+nVyhJEmS1L+cCqxKKb2YUmoBqoBLd5tzKfDD3OuFwDnR8UnxUqAqpbQzpfQXYFXueZIkDTibNm1i+vTpTJ8+ncMPP5xJkya9+XNLS8te762rq+O//tf/ul/vN3nyZDZuzJ/fy7hCSZIkqX+ZBNR3+rkBOO3t5qSUWiNiKzAmN/6n3e6dlHudgAciIgH/J6V0x57ePCI+AXwCYPz48dTU1BxUMW+nqampx57dl+RDnflQI1jnQJIPNULv1Dly5EgaGxt79D32ZtSoUTz66KMA/NM//RPDhg17s0m0c+dOmpubKSrac1vk+OOP57bbbtuv/CklmpqaGDRo0MGH3w9tbW3d8t95x44d+/W/iS41lCLi+8DFwIaU0sl7uB7A/wQuBLYB16SUluSuXQ18KTf1KymlH+bGZwE/AAYD9wGfSSmliBgNVAOTgZeAeSmlzV2uSJIkSQfijJTSmog4DPhNRKxIKT2y+6Rco+kOgPLy8jRnzpweCVNTU0NPPbsvyYc686FGsM6BJB9qhN6p89lnn830UOzOh1UPGjSIQYMG8elPf5qSkhKWLl3K7NmzmT9/Pp/5zGfYsWMHgwcP5s477+T444+npqaG22+/nV/+8pfccsstrF69mhdffJHVq1fz2c9+do+rlyKCYcOGMXz4cL7+9a/z/e9/H4CPfexjfPazn6W5uZl58+bR0NBAW1sbX/7yl6msrOTGG29k0aJFFBUVce6553L77bcfcJ0Ho6SkhBkzZnR5fldXKP0A+Bbwo7e5fgFwbO7PacB3gdNyzaGbgXI6fuu1OLfHf3NuzseBP9PRUDof+BVwI/BgSumruUMmbwRu6HJFkiRJA9saoKzTz6W5sT3NaYiIImAksGlv96aU3vh7Q0TcRcdWuLc0lCRJOhD/cM9ynln7erc+c8rEEdz8vpP2+76GhgYee+wxCgsLef3113n00UcpKirit7/9LX//93/Pz372s7fcs2LFCh5++GEaGxs5/vjjufbaaykuLt7j8xcvXsydd97Jn//8Z1JKnHbaabzrXe/ixRdfZOLEidx7770AbN26lU2bNnHXXXexYsUKIoItW7bsdz1Z6VJDKaX0SOdvB9mDS4EfpZQS8KeIGBURE4A5wG9SSq8BRMRvgPMjogYYkVL6U278R8BldDSULs3dBx17/2vIqqH0qxuZvuJR+MuoTN6+N03fsmXA1/n6jl0c09jE5meGZh2lRx3T1Dzga4T8qDMfagTrHEhGto6j+fQzGDrIHfU9rBY4NiKOoqMZNB/4wG5zFgFXA38ErgQeyq0EXwT8e0R8nY5DuY8FHo+IoUBBSqkx9/pc4NbeKeetVm1o4vnNbW9+IJQkqTvNnTuXwsJCoKOpc/XVV/P8888TEezatWuP91x00UVvrnI67LDDWL9+PaWlpXuc+/vf/57LL7+coUM7PvtdccUVPProo5x//vl87nOf44YbbuDiiy/mzDPPpLW1lZKSEj760Y9y8cUXc/HFF/dM0T2guz7x7Wkv/6R9jDfsYRxgfEppXe71K8D4Pb1hb+zfP6ahgcFtbf2qQ3ig2gZ4nc27EvVN7R0/NGW3h7fX5EONkB915kONYJ0DxDPto3nmN48waZjf+dGTcmcifQq4HygEvp9SWh4RtwJ1KaVFwL8BP46IVcBrdDSdyM1bADwDtAKfTCm1RcR44K7cN7wUAf+eUvp1rxeX85mqpTQ1tfDxy7NKIEnqbgeykqinvNHoAfjyl7/M2WefzV133cVLL730ttsAO5+LVFhYSGtr636/73HHHceSJUu47777+NKXvsQ555zDTTfdxOOPP86DDz7IwoUL+da3vsVDDz2038/OQp/+FWLuN2npba71/P79OXPcPztAfPEnS3jshY188pRCZs2amXWcHrVkyRJmzhzYNUJ+1JkPNYJ1DiRpyRKuOPddDD6kMOsoA15K6T46jgzoPHZTp9c7gLlvc+9twG27jb0ITOv+pAemsqKMm36xnKfXbOXkSSOzjiNJGsC2bt3KpEkd61t+8IMfdMszzzzzTK655hpuvPFGUkrcdddd/PjHP2bt2rWMHj2aD37wg4waNYrvfe97NDU1sW3bNi688EJmz57N0Ucf3S0ZekN3NZTebj/+Gvir1cqldGxhW5N7vft8gPURMSGltC63bW5DN2VUnnqtuYUHnnmFD71zMscM38CMIw7NOlKP2vpi4YCvEfKjznyoEaxzINn6YqHNJHWLS6dN4h/vWc6CunobSpKkHvV3f/d3XH311XzlK1/hoosu6pZnzpw5k2uuuYZTTz0V6DiUe8aMGdx///1cf/31FBQUUFxczHe/+10aGxu59NJL2bFjByklvv71r3dLht7QXQ2lRcCnIqKKjkO5t+YaQvcD/xQRb3yCPhf4QkrptYh4PSLeSceh3B8G/qXTs64Gvpr7+xfdlFF56udLGtjVlqisKGPdCvuTkiT1dSOHFFM+vpC7lq7h7y88kZJiG5WSpINzyy237HH89NNP57nnnnvz56985SsAzJkz581dPLvf+/TTT+/xWS+99NKbr6+77jquu+66v7p+3nnncd55573lvscff3wf6fumLh1yEBH/QcehjsdHRENEfDQi/iYi/iY35T7gRWAV8K/A/weQO4z7H+k4PLIWuPWNA7pzc76Xu+cFOg7kho5G0nsj4nngPbmfpQOSUmJBXT3Ty0Zx/OHZfV2lJEnaP2eVFtO4o5VfP/1K1lEkSdIedPVb3q7ax/UEfPJtrn0f+P4exuuAk/cwvgk4pyu5pH1ZWr+F59Y38dUrTsk6iiRJ2g8njC5g8pghVNWu5rIZk/Z9gyRJ6lV+DYsGtAW19Qw5pJCLp03MOookSdoPEcHc8jL+9OJrvLSxOes4kiRpNzaUNGA172zlnifXcvHUCQwb1Ke/0FCSJO3BlbNKKSwIFtTVZx1FkiTtxoaSBqx7l62juaWNyoqyfU+WJEl9zvgRJZx9/GH8dHEDrW3tWceRJEmd2FDSgFVVu5pjDhvGzAH+Nd2SJA1klRVlvNq4k4dXvpp1FEmS1IkNJQ1Iz69vZMnqLVSWlxERWceRJEkH6Ozjx3HY8EFU167OOookqZ+56KKLuP/++/9q7Jvf/CbXXnvt294zZ84c6urqALjwwgvZsmXLW+bccsst3H777Xt977vvvptnnnnmzZ9vuukmfvvb3+5P/D2qqanh4osvPujndAcbShqQqmvrKS4MLp/pt8JIktSfFRUWcOWsUh5e+SrrX9+RdRxJUj9y5ZVXUlVV9VdjVVVVXHXVXr/I/k333Xcfo0aNOqD33r2hdOutt/Ke97zngJ7VV9lQ0oDT0trOz5eu4b1TxjN22KCs40iSpIM0r7yMtvbEwsUNWUeRJPUjl156Kffeey8tLS0AvPTSS6xdu5YzzzyTa6+9lvLyck466SRuvvnmPd4/efJkNm7cCMBtt93GcccdxxlnnMHKlSvfnPOv//qvVFRUMG3aNN7//vezbds2HnvsMRYtWsT111/P9OnTeeGFF7jmmmtYuHAhAA8++CAzZszglFNO4SMf+Qg7d+588/1uvvlmZs6cySmnnMKKFSv2Wt9rr73GZZddxumnn8473/lOli1bBsDvfvc7pk+fzvTp05kxYwaNjY2sW7eOs846i+nTp3PyySfz6KOPHtx/XMCvvtKA89tn1/Nacwvzyj2MW5KkgWDy2KG88+jRLKir59p3vYOCArezS1K/86sb4ZWnuveZh58CF3z1bS+PHj2aU089lV/96ldceumlVFVVMW/ePCKC2267jdGjR9PW1sY555zDsmXLmDp16h6fs3jxYqqqqnjiiSdobW1l5syZzJo1C4ArrriCj3/84wB86Utf4t/+7d/49Kc/zSWXXMLFF1/MlVde+VfP2rFjB9dccw0PPvggxx13HB/+8If57ne/y2c/+1kAxo4dy5IlS/jOd77D7bffzve+9723re/mm29mxowZ/PjHP6a2tpYPf/jDPPHEE9x+++18+9vfZvbs2TQ1NVFSUsIdd9zBeeedxxe/+EXa2trYtm3bfv2n3hNXKGnAqaqtZ+LIEs48dlzWUSRJUjeZX3EEL2/axp//8lrWUSRJ/chVV1315ra3ztvdFixYwMyZM5kxYwbLly//q+1pu3v00Ue5/PLLGTJkCCNGjOCSSy5589rTTz/NmWeeySmnnMJPfvITli9fvtc8K1eu5KijjuK4444D4Oqrr+aRRx558/oVV1wBwKxZs3jppZf2+qzf//73fOhDHwLg3e9+N5s2beL1119n9uzZXHfddfyv//W/2LJlC0VFRVRUVHDnnXdyyy238NRTTzF8+PC9PrsrXKGkAWXNlu08+vyrfPrdx1Loby8lSRowzj/5cIb/oojq2tWc/o4xWceRJO2vvawk6kmXXnopf/u3f8uSJUvYtm0bs2bN4i9/+Qu33347tbW1HHrooVxzzTXs2HFg5/Rdc8013H333UybNo0f/OAH1NTUHFTeQYM6jm0pLCyktbX1gJ5x4403ctFFF3Hfffcxe/Zs7r//fs466yweeeQR7r33Xq655hquu+46PvzhDx9UVlcoaUD5aV09AHNnlWacRJIkdaeS4kIunzGJ+55+ha3bdmUdR5LUTwwbNoyzzz6bj3zkI2+uTnr99dcZOnQoI0eOZP369fzqV7/a6zPOOuss7r77brZv305jYyP33HPPm9caGxuZMGECu3bt4ic/+cmb48OHD6exsfEtzzr++ON56aWXWLVqFQA//vGPede73nVAtZ155plvvmdNTQ1jx45lxIgRvPDCC5xyyinccMMNVFRUsGLFCl5++WXGjx/Pxz/+cT72sY+xZMmSA3rPzmwoacBoa0/8tK6BM44ZS9noIVnHkSRJ3ayyooyW1nZ+8eSarKNIkvqRq666iieffPLNhtK0adOYMWMGJ5xwAh/4wAeYPXv2Xu+fOXMmlZWVTJs2jQsuuICKioo3r/3jP/4jp512GrNnz+aEE054c3z+/Pn88z//MzNmzOCFF154c7ykpIQ777yTuXPncsopp1BQUMDf/M3fHFBdt9xyC4sXL+b000/nxhtv5Ic//CEA3/zmNzn55JOZOnUqxcXFXHDBBdTU1LxZd3V1NZ/5zGcO6D07c8ubBow/rNrImi3b+cKFJ+x7siRJ6ndOmjiSkyeN4D8er+dD7zySCLe3S5L27bLLLiOl9FdjP/jBD/Y4t/OWtc5nGH3xi1/ki1/84lvmX3vttVx77bVvGZ89e/ZfncvU+f3OOeccli5d+pZ7Or9feXn5HrfPzZkzhzlz5gAdh47ffffdNDY2/tWZSP/yL//ylvuuvvpqrr766reMHwxXKGnAqK6r59Ahxbx3yviso0iSpB5SWXEEz657nafXvJ51FEmS8poNJQ0IrzW38MDyV7h8RimDigqzjiNJknrIJdMmUlJcQHXd6qyjSJKU12woaUC4a+kadrUlKivKso4iSZJ60MjBxVx48gR+sXQt21vaso4jSVLesqGkfi+lRHXtaqaXjeL4w4fv+wZJktSvVVaU0bizlfueWpd1FEnSPux+dpH6pgP5d7KhpH7vifotPLe+ydVJkiTliVOPGs1RY4dSXVefdRRJ0l6UlJSwadMmm0p9XEqJTZs2UVJSsl/3+S1v6veqa+sZckgh75s2MesokiSpF0QE88rL+NqvV/Diq00cPW5Y1pEkSXtQWlpKQ0MDr776aibvv2PHjv1ukvRH3VFnSUkJpaWl+3WPDSX1a807W7nnybVcPHUCwwb5P2dJkvLF+2dN4vYHVlJdV88XLjgx6ziSpD0oLi7mqKOOyuz9a2pqmDFjRmbv31uyqtMtb+rX7l22juaWNre7SZKUZw4bXsI5JxzGzxavYVdbe9ZxJEnKOzaU1K9V1a7mmMOGMfOIQ7OOIkmSelllRRkbm3by0IoNWUeRJCnv2FBSv/X8+kaWrN5CZXkZEZF1HEmS1Mveddw4xo8YRHWth3NLktTbbCip36qurae4MLh85qSso0iSpAwUFRYwd1YZNSs38MrWHVnHkSQpr9hQUr/U0trOz5eu4T0njmfssEFZx5EkSRmZV15Ge4KFi12lJElSb7KhpH7pt8+u57XmFg/jliQpzx0xZgj/6R1jqK6rp709ZR1HkqS8YUNJ/VJ1bT0TR5Zw5rHjso4iSZIyVllRRv1r2/nTi5uyjiJJUt6woaR+Z82W7Tzy/KtcWV5GYYGHcUuSlO/OO+lwRg4upsrDuSVJ6jU2lNTvLKxrAGDurNKMk0iSpL6gpLiQy2dM4tfLX2HLtpas40iSlBdsKKlfaWtPLKir54xjxlI2ekjWcSRJUh9RWVFGS2s7dy9dk3UUSZLygg0l9St/WLWRNVu2M6/cw7glSdL/c+KEEUwrHUlVbT0peTi3JEk9zYaS+pXqunpGDSnm3JPGZx1FkiT1MfMqyljxSiPLGrZmHUWSpAHPhpL6jdeaW3hg+StcMaOUQUWFWceRJEl9zCXTJjK4uJDqOg/nliSpp9lQUr9x19I17GpLVFa43U2SJL3V8JJiLpo6gUVPrGVbS2vWcSRJGtBsKKlfSClRXbua6WWjOP7w4VnHkSRJfVRlRRlNO1u5d9m6rKNIkjSg2VBSv/BE/RaeW9/k6iRJkrRX5UceytHjhlJd67Y3SZJ6UpcaShFxfkSsjIhVEXHjHq4fGREPRsSyiKiJiNJO174WEU/n/lR2Gn93RCzJjf8wIopy4yMj4p6IeDIilkfEf+mOQtW/VdfWM+SQQt43bWLWUSRJUh8WEcyvKKPu5c2s2tCUdRxJkgasfTaUIqIQ+DZwATAFuCoipuw27XbgRymlqcCtwH/L3XsRMBOYDpwGfD4iRkREAfBDYH5K6WTgZeDq3LM+CTyTUpoGzAH+R0QcclBVql9r3tnKPU+u5aJTJjBsUFHWcSRJUh93xcxSigqCBR7OLUlSj+nKCqVTgVUppRdTSi1AFXDpbnOmAA/lXj/c6foU4JGUUmtKqRlYBpwPjAFaUkrP5eb9Bnh/7nUChkdEAMOA1wBPVcxj9y5bR3NLG/NPdbubJEnat7HDBvGeE8fzs8UNtLS2Zx1HkqQBqSsNpUlA51/vNOTGOnsSuCL3+nI6GkJjcuPnR8SQiBgLnA2UARuBoogoz91zZW4c4FvAicBa4CngMyklPwnkseq6et4xbigzjzg06yiSJKmfqDy1jE3NLTy0Yn3WUSRJGpC6a//Q54FvRcQ1wCPAGqAtpfRARFQAjwGvAn/MjaeImA98IyIGAQ8AbblnnQc8AbwbeAfwm4h4NKX0euc3jIhPAJ8AGD9+PDU1Nd1Uyl9ramrqsWf3JX21zrVN7Sx+eTuVxx/C7373u4N+Xl+tszvlQ42QH3XmQ41gnQNJPtSo/uOsY8cxYWQJVbX1nH/yhKzjSJI04HSlobSG/7d6CKA0N/amlNJaciuUImIY8P6U0pbctduA23LX/h14Ljf+R+DM3Pi5wHG5x/0X4KsppQSsioi/ACcAj+/2nncAdwCUl5enOXPmdKng/VVTU0NPPbsv6at13nbvMxQVvMT1c89i7LBBB/28vlpnd8qHGiE/6syHGsE6B5J8qFH9R2FBMHdWKf/y8CrWbtnOxFGDs44kSdKA0pUtb7XAsRFxVO5w7PnAos4TImJs7qBtgC8A38+NF+a2vhERU4GpdKxGIiIOy/09CLgB+N+5+1cD5+SujQeOB1480ALVf7W0tvOzJWt475Tx3dJMkiRJ+WVuecfvRBcubsg4iSRJA88+G0oppVbgU8D9wLPAgpTS8oi4NSIuyU2bA6yMiOeA8eRWJAHFwKMR8Qwdq4k+mHsewPUR8SwdB3Xfk1J641DvfwT+U0Q8BTwI3JBS2niwhar/+e2z63mtuYV5FR7GLUmS9l/Z6CHMfsdYqmvraW9PWceRJGlA6dIZSiml+4D7dhu7qdPrhcDCPdy3g45vetvTM68Hrt/D+Frg3K7k0sBWXVvPhJElnHXsuKyjSJKkfqqyooxP/8dS/vDCRs70M4UkSd2mK1vepF63Zst2Hnn+VeaWl1FYEFnHkSRJ/dS5J41n1JBiqmvr9z1ZkiR1mQ0l9UkL6zrOOpg7qzTjJJIkqT8bVFTI5TMm8cDyjq30kiSpe9hQUp/T3p5YUFfPGceMpWz0kKzjSJLU50TE+RGxMiJWRcSNe7g+KCKqc9f/HBGTO137Qm58ZUSct9t9hRGxNCJ+2fNV9J7KijJa2tq5a+mafU+WJEldYkNJfc4fXtjImi3bmVfuYdySJO0uIgqBbwMX0HFW5VURsfuZlR8FNqeUjgG+AXwtd+8UOr6x9yTgfOA7uee94TN0fAnLgHLC4SOYXjaKBbX1pOTh3JIkdQcbSupzqmrrGTWkmHNPGp91FEmS+qJTgVUppRdTSi1AFXDpbnMuBX6Ye70QOCciIjdelVLamVL6C7Aq9zwiohS4CPheL9TQ6yoryli5vpEn6rdkHUWSpAGhS9/yJvcEveoAACAASURBVPWW15pb+M3y9fzndx7BoKLCfd8gSVL+mQR0PmG6ATjt7eaklFojYiswJjf+p93unZR7/U3g74Dhe3vziPgE8AmA8ePHU1NTc0BF7EtTU1O3PntUa2JQIXxj0eN85ORB3fbcg9XddfZF+VAjWOdAkg81Qn7UmQ81QnZ12lBSn3LX0jW0tLVTWeF2N0mSektEXAxsSCktjog5e5ubUroDuAOgvLw8zZmz1+kHrKamhu5+9sNbnuTeZev47ulnMHRQ3/gY3BN19jX5UCNY50CSDzVCftSZDzVCdnW65U19RkqJBbX1TCsbxQmHj8g6jiRJfdUaoPNvXkpzY3ucExFFwEhg017unQ1cEhEv0bGF7t0R8X97InyWKivKaG5p495l67KOIklSv2dDSX3GE/VbWLm+kfmuTpIkaW9qgWMj4qiIOISOQ7YX7TZnEXB17vWVwEOp4zTqRcD83LfAHQUcCzyeUvpCSqk0pTQ597yHUkof7I1ietPMIw7lmMOGUVW7OusokiT1ezaU1GcsqKtncHEhF0+dkHUUSZL6rJRSK/Ap4H46vpFtQUppeUTcGhGX5Kb9GzAmIlYB1wE35u5dDiwAngF+DXwypdTW2zVkJSKYX1HGktVbeH59Y9ZxJEnq12woqU9o3tnKoifWcvHUCQwvKc46jiRJfVpK6b6U0nEppXeklG7Ljd2UUlqUe70jpTQ3pXRMSunUlNKLne69LXff8SmlX+3h2TUppYt7r5redfmMSRQXBtW19fueLEmS3pYNJfUJ9y5bR3NLm4dxS5KkHjVm2CDeO2U8P1+6hpbW9qzjSJLUb9lQUp9QXVfPO8YNZdaRh2YdRZIkDXCVFUfwWnMLv312fdZRJEnqt2woKXOrNjSy+OXNzK84gojIOo4kSRrgzjhmLBNHllDltjdJkg6YDSVlrrq2nqKC4PKZk7KOIkmS8kBhQTC3vIxHn3+Vhs3bso4jSVK/ZENJmWppbednS9bw3injGTtsUNZxJElSnphbXgrAwsUNGSeRJKl/sqGkTD347Hpea25hnodxS5KkXlR66BDOOGYsP61roK09ZR1HkqR+x4aSMlVVW8+EkSWcdey4rKNIkqQ8M7/iCNZs2c7vV23MOookSf2ODSVlZu2W7Tzy/KvMnVVKYYGHcUuSpN71nimHMXroIVTXrs46iiRJ/Y4NJWXmp3UdZxbMLXe7myRJ6n2Digq5fMYkfvPMejY17cw6jiRJ/YoNJWWivT2xoK6e2e8YS9noIVnHkSRJeaqyooxdbYm7lq7JOookSf2KDSVl4g8vbGTNlu1Uehi3JEnK0HHjhzPziFFU1daTkodzS5LUVTaUlInq2npGDSnm3JPGZx1FkiTlucqKMlZtaGLJ6i1ZR5Ekqd+woaRe91pzCw8sX8/lMyYxqKgw6ziSJCnPXTx1IkMPKfRwbkmS9oMNJfW6u5auoaWt3e1ukiSpTxg6qIj3TZvIL5eto2lna9ZxJEnqF2woqVellFhQW8+0slGccPiIrONIkiQBMK+ijG0tbfzyybVZR5EkqV+woaRe9UT9Flaub2S+q5MkSVIfMqNsFMeNH0ZVbX3WUSRJ6hdsKKlXLairZ3BxIRdPnZB1FEmSpDdFBJUVR3T88uuVxqzjSJLU59lQUq9p3tnKoifWcvHUCQwvKc46jiRJ0l+5fMYkiguDalcpSZK0TzaU1GvufWodzS1tHsYtSZL6pNFDD+Hckw7n50sb2NnalnUcSZL6NBtK6jXVtfW8Y9xQZh15aNZRJEmS9mh+RRlbtu3igeXrs44iSVKfZkNJvWLVhkYWv7yZyooyIiLrOJIkSXs0+x1jmTRqMAvq3PYmSdLe2FBSr6iuraeoILhiZmnWUSRJkt5WQUEwr7yMR5/fSP1r27KOI0lSn2VDST2upbWdny9Zw3tOHM/YYYOyjiNJkrRXc8tLiYCfukpJkqS3ZUNJPe7BZ9ezqbmFylM9jFuSJPV9E0cN5qxjx/HTxQ20taes40iS1CfZUFKPq6qtZ8LIEs46dlzWUSRJkrpkfkUZ67bu4JHnX806iiRJfZINJfWotVu288jzrzJ3VimFBR7GLUmS+odzThzPmKGHUP24294kSdqTLjWUIuL8iFgZEasi4sY9XD8yIh6MiGURURMRpZ2ufS0ins79qew0/u6IWJIb/2FEFHW6NicinoiI5RHxu4MtUtn5aV0DKcHccre7SZKk/uOQogKumDmJ3z67no1NO7OOI0lSn7PPhlJEFALfBi4ApgBXRcSU3abdDvwopTQVuBX4b7l7LwJmAtOB04DPR8SIiCgAfgjMTymdDLwMXJ27ZxTwHeCSlNJJwNyDrlKZaG9PLKir54xjxlI2ekjWcSRJkvZLZUUZre2Jny9pyDqKJEl9TldWKJ0KrEopvZhSagGqgEt3mzMFeCj3+uFO16cAj6SUWlNKzcAy4HxgDNCSUnouN+83wPtzrz8A/DyltBogpbRh/8tSX/CHFzayZst2KitcnSRJkvqfYw4bTvmRh1JVW09KHs4tSVJnRfuewiSg8+bxBjpWG3X2JHAF8D+By4HhETEmN35zRPwPYAhwNvAMsBEoiojylFIdcCXwRtfhOKA4ImqA4cD/TCn9aPdQEfEJ4BMA48ePp6ampgul7L+mpqYee3Zf0hN1fueJHQwthpJNK6mpeW7fN/SCfPj3zIcaIT/qzIcawToHknyoUfmnsqKM6xcuY/HLmymfPDrrOJIk9RldaSh1xeeBb0XENcAjwBqgLaX0QERUAI8BrwJ/zI2niJgPfCMiBgEPAG2dMs0CzgEGA3+MiD91Ws0EQErpDuAOgPLy8jRnzpxuKuWv1dTU0FPP7ku6u87NzS088ZsH+c/vnMx7331Stz33YOXDv2c+1Aj5UWc+1AjWOZDkQ43KPxdNncA/3PMMVbX1NpQkSeqkK1ve1vD/Vg8BlObG3pRSWptSuiKlNAP4Ym5sS+7v21JK01NK7wUCeC43/seU0pkppVPpaEK90TBqAO5PKTWnlDbmrk074AqVibuWrqGlrd3tbpIkqV8bckgR75s2kXuXraNxx66s40iS1Gd0paFUCxwbEUdFxCHAfGBR5wkRMTZ30DbAF4Dv58YLc1vfiIipwFQ6ViMREYfl/h4E3AD879z9vwDOiIiiiBhCx/a6Zw+8RPW2lBLVtfVMKxvFCYePyDqOJEnSQZlfUcb2XW3c8+S6rKNIktRn7LOhlFJqBT4F3E9HY2dBSml5RNwaEZfkps0BVkbEc8B44LbceDHwaEQ8Q8f2tA/mngdwfUQ8S8dB3feklB7Kvd+zwK9z448D30spPX3wpaq3PNmwlZXrG6ksd3WSJEnq/6aWjuSEw4dTXbs66yiSJPUZXTpDKaV0H3DfbmM3dXq9EFi4h/t20PFNb3t65vXA9W9z7Z+Bf+5KNvU91bWrGVxcyPumTcg6iiRJ0kGLCCoryviHe57h2XWvc+IEV2BLktSVLW9SlzXvbGXRE2u5aOoEhpcUZx1HkiSpW1w+YxKHFBVQXVu/78mSJOUBG0rqVvc+tY7mljbmexi3JEkaQEYNOYTzTjqcu5auYceutn3fIEnSAGdDSd2quraeo8cNZdaRh2YdRZIkqVvNryhj6/Zd3L/8layjSJKUORtK6jarNjSy+OXNzK8oIyKyjiNJktStTj96DGWjB7vtTZIkbCipG1XX1lNUEFwxszTrKJIkSd2uoCCYN6uMx17YxOpN27KOI0lSpmwoqVu0tLbz8yVreM+J4xk7bFDWcSRJknrEleWlFAQsqHOVkiQpv9lQUrd48Nn1bGpuofJUD+OWJEkD14SRg5lz/GH8dHE9rW3tWceRJCkzNpTULarr6pkwsoSzjh2XdRRJkqQeNa+8jPWv7+SR51/NOookSZmxoaSDtnbLdn733KvMnVVKYYGHcUuSpIHtnBMPY+ywQ6h63G1vkqT8ZUNJB23h4gZSgrnlbneTJEkDX3FhAe+fVcqDKzawoXFH1nEkScqEDSUdlPb2xIK6es44Zixlo4dkHUeSJKlXzCsvo6098fMla7KOIklSJmwo6aA89sImGjZvZ16Fq5MkSVL+eMe4YZw6eTTVtfWklLKOI0lSr7OhpINSVbuaUUOKOXfK+KyjSJIk9arKijL+srGZx//yWtZRJEnqdTaUdMA2N7fwwPL1XDZ9EiXFhVnHkSRJ6lUXnjKB4YOKqK7zcG5JUv6xoaQDdtfSNbS0tVPpdjdJkpSHBh9SyCXTJ3LfU+vYun1X1nEkSepVNpR0QFJKVNfWM610JCdOGJF1HEmSpEzMrziCHbvaWfTk2qyjSJLUq2wo6YA82bCVlesbqaw4IusokiTlnYg4PyJWRsSqiLhxD9cHRUR17vqfI2Jyp2tfyI2vjIjzcmMlEfF4RDwZEcsj4h96r5r+7eRJIzhxwggW1LrtTZKUX2wo6YBU165mcHEh75s2IesokiTllYgoBL4NXABMAa6KiCm7TfsosDmldAzwDeBruXunAPOBk4Dzge/knrcTeHdKaRowHTg/It7ZG/X0dxHB/IoynlqzlafXbM06jiRJvcaGkvZb885WFj2xloumTmB4SXHWcSRJyjenAqtSSi+mlFqAKuDS3eZcCvww93ohcE5ERG68KqW0M6X0F2AVcGrq0JSbX5z7k3q6kIHisumTOKSogAUezi1JyiM2lLTf7n1qHc0tbcz3MG5JkrIwCejcuWjIje1xTkqpFdgKjNnbvRFRGBFPABuA36SU/twj6QegkUOKueDkw7l76Rp27GrLOo4kSb2iKOsA6n8W1NZz9LihzDry0KyjSJKkbpJSagOmR8Qo4K6IODml9PTu8yLiE8AnAMaPH09NTU2P5GlqauqxZ/eEE4rb+MWOVr7+04f5TxO7/hG7v9V5IPKhRrDOgSQfaoT8qDMfaoTs6rShpP2yakMjdS9v5u8vPIGOlfOSJKmXrQE6LxMuzY3taU5DRBQBI4FNXbk3pbQlIh6m44yltzSUUkp3AHcAlJeXpzlz5hxMLW+rpqaGnnp2TzirPVH1Yg3Lmkr4+zmnd/m+/lbngciHGsE6B5J8qBHyo858qBGyq9Mtb9ovC+oaKCoIrphZmnUUSZLyVS1wbEQcFRGH0HHI9qLd5iwCrs69vhJ4KKWUcuPzc98CdxRwLPB4RIzLrUwiIgYD7wVW9EItA0ZBQTCvvIw/vfgaL21szjqOJEk9zoaSuqyltZ2fLW7gPSeOZ+ywQVnHkSQpL+XORPoUcD/wLLAgpbQ8Im6NiEty0/4NGBMRq4DrgBtz9y4HFgDPAL8GPpnb6jYBeDgiltHRsPpNSumXvVnXQHDlrFIKAg/nliTlBbe8qcseWrGeTc0tVHoYtyRJmUop3Qfct9vYTZ1e7wDmvs29twG37Ta2DJjR/Unzy/gRJbz7hMNYuLiB6957HEWF/u5WkjRw+f/l1GVVtfUcPqKEs44bl3UUSZKkPmleeRkbGndSs/LVrKNIktSjbCipS9Zu2c4jz73K3PJSCgs8jFuSJGlPzj7hMMYNH0RVrdveJEkDmw0ldcnCxQ20p47fukmSJGnPigsLuHJWKQ+v3MCG13dkHUeSpB5jQ0n71N6eWFBXz+xjxlA2ekjWcSRJkvq0eeVltLUnFi5pyDqKJEk9xoaS9umxFzbRsHk7lRVHZB1FkiSpzztq7FBOO2o01bX1pJSyjiNJUo+woaR9qqpdzcjBxZw7ZXzWUSRJkvqF+aeW8fKmbfzpxdeyjiJJUo+woaS92tzcwgPL13P5jEmUFBdmHUeSJKlfuODkCQwvKaK6dnXWUSRJ6hE2lLRXdy1dQ0tbO5UVHsYtSZLUVSXFhVw2fRK/evoVtm7blXUcSZK6nQ0lva2UEtW19UwrHcmJE0ZkHUeSJKlfqawoY2drO794ck3WUSRJ6nY2lPS2nmzYysr1jR7GLUmSdABOnjSSkyaOoOrx+qyjSJLU7Wwo6W1V19YzuLiQ902bkHUUSZKkfml+RRnPrHudp9dszTqKJEndqksNpYg4PyJWRsSqiLhxD9ePjIgHI2JZRNRERGmna1+LiKdzfyo7jb87Ipbkxn8YEUW7PbMiIloj4sqDKVAHZltLK/c8uZaLpk5geElx1nEkSZL6pUumT2JQUQFVHs4tSRpg9tlQiohC4NvABcAU4KqImLLbtNuBH6WUpgK3Av8td+9FwExgOnAa8PmIGBERBcAPgfkppZOBl4Grd3vPrwEPHFx5OlD3LltH085WD+OWJEk6CCMHF3PhKRP4xdK1bG9pyzqOJEndpisrlE4FVqWUXkwptQBVwKW7zZkCPJR7/XCn61OAR1JKrSmlZmAZcD4wBmhJKT2Xm/cb4P2dnvdp4GfAhv2sR92kuraeo8cNpfzIQ7OOIkmS1K9VVpTRuLOVXz29LusokiR1m640lCYBnU8SbMiNdfYkcEXu9eXA8IgYkxs/PyKGRMRY4GygDNgIFEVEee6eK3PjRMSk3DO+u//lqDus2tBI3cubqSwvIyKyjiNJktSvnXbUaCaPGUJVrYdzS5IGjqJ9T+mSzwPfiohrgEeANUBbSumBiKgAHgNeBf6YG08RMR/4RkQMomNr2xtrgL8J3JBSat9bMyMiPgF8AmD8+PHU1NR0Uyl/rampqcee3Zd0rrNqRQuFAYfvWE1NzcD64JMP/575UCPkR535UCNY50CSDzVKByIimFdRxn//9UpefLWJo8cNyzqSJEkHrSsNpTXkVg/llObG3pRSWktuhVJEDAPen1Lakrt2G3Bb7tq/A8/lxv8InJkbPxc4Lve4cqAq10waC1wYEa0ppbt3e887gDsAysvL05w5c7pU8P6qqamhp57dl7xRZ0trO5979EHeM2U8l55Xvu8b+5l8+PfMhxohP+rMhxrBOgeSfKhROlBXzizlfzzwHAvqGrjxghOyjiNJ0kHrypa3WuDYiDgqIg4B5gOLOk+IiLG5g7YBvgB8PzdemNv6RkRMBaaSO2g7Ig7L/T0IuAH43wAppaNSSpNTSpOBhcD/t3szST3noRXr2dTcwvyKI7KOIkmSNGAcNqKEd59wGAsXN7CrrT3rOJIkHbR9NpRSSq3Ap4D7gWeBBSml5RFxa0Rckps2B1gZEc8B48mtSAKKgUcj4hk6VhN9MPc8gOsj4lk6Duq+J6X0xqHeylBVbT2HjyjhrOPGZR1FkiRpQKksL2Nj004eWuH3zkiS+r8unaGUUroPuG+3sZs6vV5Ix2qi3e/bQcc3ve3pmdcD1+/jfa/pSj51j7VbtvPIc6/yybOPobDAw7glSZK605zjx3HY8EEsqK3nvJMOzzqOJEkHpStb3pQnFi5uoD3BvPKyfU+WJEnSfikqLGBueSkPr9zAK1t3ZB1HkqSDYkNJALSnxIK6emYfM4ay0UOyjiNJkjQgzSsvoz3BwsUD65t0JUn5x4aSAHh2UzsNm7dT6WHckiRJPebIMUM5/egxLKhroL09ZR1HkqQDZkNJADzSsIuRg4s5d8r4rKNIkiQNaPNPLWP1a9v404ubso4iSdIBs6EkNje3sHh9G5fPmERJcWHWcSRJkga08046nJGDi6mqddubJKn/sqEk7n5iDa0JKis8jFuSJKmnlRQXctn0ifx6+Ss0tbjtTZLUP9lQynMpJaoer+eokQWcOGFE1nEkSZLyQmXFEbS0tvPHda1ZR5Ek6YDYUMpzTzZsZeX6Rs6aVJR1FEmSpLwxZeIIppaO5JGGVlJylZIkqf+xoZTnqmvrGVxcyDsn2lCSJEnqTfPKy6hvbOepNVuzjiJJ0n6zoZTHtrW0cs+Ta7nwlAkMLoqs40iSJOWVS6ZP5JACPJxbktQv2VDKY/cuW0fTzlbmn+ph3JIkSb1tREkxFYcXseiJtWxr8SwlSVL/YkMpj1XX1nP0uKGUH3lo1lEkSZLy0lmlRTTtbOW+p17JOookSfvFhlKeWrWhibqXN1NZXkaE290kSZKycNyhBRw9dijVtauzjiJJ0n6xoZSnFtTVU1QQXDGzNOsokiRJeSsiqKwoo/alzaza0JR1HEmSusyGUh5qaW3n50saOOfEwxg3fFDWcSRJkvLaFTNLKSoIflrn4dySpP7DhlIeemjFejY2tTC/4oiso0iSJOW9ccMHcc6Jh/GzJQ20tLZnHUeSpC6xoZSHqmvrOXxECWcdNy7rKJIkSQLmVxzBxqYWHlqxPusokiR1iQ2lPLNu63Z+99yrzC0vpbDAw7glSZL6grOOG8fhI0qoqnXbmySpf7ChlGcW1jXQnmDurLKso0iSJCmnsCCYW17KI8+9ytot27OOI0nSPtlQyiPt7YnqunpmHzOGI8YMyTqOJEmSOplXXkZ7goWLG7KOIknSPtlQyiOPvbCJhs3bmVfu6iRJkqS+pmz0EGYfM4YFdfW0t6es40iStFc2lPJIdV09IwcXc95Jh2cdRZIkSXtQWXEEDZu389gLm7KOIknSXtlQyhObm1u4/+lXuHzGJEqKC7OOI0mSpD04d8p4Rg0ppqp2ddZRJEnaKxtKeeLuJ9bQ0tZOZYXb3SRJkvqqkuJCLps+iQeWr2dzc0vWcSRJels2lPJASonq2nqmlo7kxAkjso4jSZKkvaisKKOlrZ27lq7JOookSW/LhlIeWNawlRWvNLo6SZIkqR84ccIIppWNorq2npQ8nFuS1DfZUMoDVbX1DC4u5JJpE7OOIkmSpC6oLC9j5fpGnqjfknUUSZL2yIbSALetpZV7nlzLhadMYHhJcdZxJEmS1AXvmzaBwcWFLKirzzqKJEl7ZENpgLt32TqadrYy/1S3u0mSJPUXw0uKuXjqBBY9sZbmna1Zx5Ek6S1sKA1wC+rqOXrcUMqPPDTrKJIkSdoPlRVlNLe0ce+ydVlHkSTpLWwoDWCrNjRR+9JmKsvLiIis40iSJGk/zDryUN4xbijVbnuTJPVBNpQGsJ/W1VNUEFwxszTrKJIkqRtFxPkRsTIiVkXEjXu4PigiqnPX/xwRkztd+0JufGVEnJcbK4uIhyPimYhYHhGf6b1q9HYigvkVR7D45c08//+3d+/xVVV33sc/v1xIACFIwjUJgopakHuC12oQHfEyQRRImHYK00596fTy1I7tY0fHWpVnxtfQzrRPq305Wm9Ph4SLILQgXjDqjLcERLkIFBFJuBMkJEIIyVnPH2cTk5CQE0jOPpfv+2VeOVl77bV/v2w8Z/Fj77X3VfsdjoiISDMqKMWouvoAi9dWMPlr/enXK8XvcERERKSTmFki8DvgJmAEMMvMRrTo9h3gC+fchcC/A495+44ACoGRwBTgcW+8euAfnXMjgMuB77Uypvhg2vhMkhKM4lJdpSQiIpFFBaUYtXrzPg7W1FGQq8W4RUREYsxEYJtzbrtzrg4oAqa26DMVeM57vQiYbMH736cCRc654865z4BtwETn3B7n3FoA51w18AmQGYZcpB0Z56Rww4gBvPjhLurqA36HIyIi0kgFpRhVXFrOwN6pXDO8n9+hiIiISOfKBJperlLBqcWfxj7OuXqgCkgPZV/v9rhxwPudGLOchYLcbA59Wcdrn+zzOxQREZFGSX4HIJ1vT9Ux3tx6gH/Iu5CkRNUMRUREJDRmdg6wGPiRc+5IG33uBO4EGDBgACUlJV0SS01NTZeNHUlCyTPgHH1TjcdXfUSPyi3hCawT6VzGlnjIMx5yhPjIMx5yBP/yVEEpBi0qqyDgYGaObncTERGJQbuAph/yWV5ba30qzCwJSAMqT7evmSUTLCb90Tn3YlsHd849CTwJkJOT4/Ly8s4mlzaVlJTQVWNHklDz/Gb9Vv7v6r8wfOxlZPbp3vWBdSKdy9gSD3nGQ44QH3nGQ47gX54hXb4SwpNEzjOz183sYzMrMbOsJtseM7MN3ldBk/brzGyt1/6cN9nBzL7hjbPezN4xszGdkWi8CAQcxWXlXHlBOkPSe/gdjoiIiHS+UmC4mQ0zs24EF9le1qLPMmC293o6sNo557z2Qu8pcMOA4cAH3vpKTwOfOOd+FZYspENmTAhOrxeWaXFuERGJDO0WlEJ8ksg84Hnn3GjgYeBfvH1vAcYDY4HLgHvNrLeZJRBcKLLQOXcp8DlfTXo+A651zo0CHsH7FzAJzbvbK6n44pgW4xYREYlR3ppI3wdWEVw8e4FzbqOZPWxm+V63p4F0M9sG/Bi4z9t3I7AA2AS8DHzPOdcAXAX8LXCdma3zvm4Oa2JyWtl9e3D1hRksLKugIeD8DkdERCSkW94anyQCYGYnnySyqUmfEQQnKwBvAEubtL/lTXzqzexjgo+ofQOoc85t9fq9CvwMeNo5906Tcd8jeCm2hKiotJy07sncOHKg36GIiIhIF3HOrQBWtGh7sMnrWmBGG/vOBea2aPtvwDo/UulMBbnZfP+/PuR/th3kmov04BUREfFXKAWl1p4GclmLPh8BtwO/BqYBvcws3Wv/uZn9EugBTCJYiDoIJJlZjnOujOCl2K1dUvMdYGVrQWlByFPV1DlWfnyUvOwk3vuftzu2bxTleTbiIc94yBHiI894yBGUZyyJhxxF/HTDiAGc2yOZ4tJyFZRERMR3nbUo973Ab81sDvAWwcUdG5xzr5hZLvAOcAB412t3ZlYI/LuZpQCvAA1NBzSzSQQLSle3dkAtCHmqZ/7nM+rdJv5x2hV8bVDvDu0bTXmejXjIMx5yhPjIMx5yBOUZS+IhRxE/pSQlMm1cFi+8t4PKmuOkn5Pid0giIhLHQlmUu90niTjndjvnbnfOjQPu99oOe9/nOufGOuduIHgp9Vav/V3n3NedcxMJFqFO3v6GmY0GngKmOucqzzi7OOKco7i0nNFZaR0uJomIiIhIdCjIzeZEg2PJhy0f7CciIhJeoRSU2n2SiJlleAttQ3AtpD947YnerW8ni0SjCV6NhJn1976nAP8baUFC/gAAIABJREFU+L338xDgReBvm6yxJO34uKKKzXurtRi3iIiISAy7eGAvxg3pQ3FpOcEH94mIiPij3YJSiE8SyQO2mNlWYABfLfSYDLxtZpsI3p72TW88gJ+Y2SfAx8By59xqr/1BIB143HvCSNlZZxkHikrLSU1O4K/HDPY7FBERERHpQgU52fxlfw1rdx72OxQREYljIa2hFMKTRBYBi1rZr5bgk95aG/MnwE9aaf974O9DiUuCjtbVs/yj3dwyajC9U5P9DkdEREREutCtYwbz8J82UVy6kwnnnet3OCIiEqdCueVNItyfP95DzfF63e4mIiIiEgfOSUnir0cP5k/eHFBERMQPKijFgAVl5Zyf0ZPcofoXKhEREZF4MDM3m6N1Dfzpo91+hyIiInFKBaUot21/DaU7vmBmbjZm5nc4IiIiIhIG44f0YXj/cygqLfc7FBERiVMqKEW5hWXlJCUYt4/P9DsUEREREQkTM6MgN5t15YfZsrfa73BERCQOqaAUxU40BFi8toLrLulP/16pfocjIiIiImF0+/gskhONYl2lJCIiPlBBKYq9/sl+DtbUUThRi3GLiIiIxJu+PbvxVyMG8uKHFRyvb/A7HBERiTMqKEWx4tKdDOydyjXD+/kdioiIiIj4oCA3m8NHT/Dqpn1+hyIiInFGBaUotafqGG9uPcD0CVkkJeo0ioiIiMSjqy/MILNPd932JiIiYadKRJRaVFZBwMHMHN3uJiIiIhKvEhKMGTlZvP2Xg5QfOup3OCIiEkdUUIpCgYBjwZpyrrwgnSHpPfwOR0RERER8NCMnGzNYuKbC71BERCSOqKAUhd7dXkn5oWMU5OrqJBEREZF4l9mnO9cM78fCsnIaAs7vcEREJE6ooBSFikrLSeuezI0jB/odioiIiIhEgILcbPZU1fLWXw74HYqIiMQJFZSizBdf1rFqw16mjcskNTnR73BEREREJAJc/7UB9O3ZjQVanFtERMJEBaUos3TdLuoaAlqMW0REREQadUtK4I7xmby6aR8Ha477HY6IiMQBFZSiiHOO4tJyRmelMWJwb7/DEREREZEIUpCbTX3A8eJaLc4tIiJdTwWlKPJxRRWb91br6iQREREROcWF/Xsx4bxzKS4txzktzi0iIl1LBaUoUlxWTmpyAvljB/sdioiIiIhEoILcbD498CVrPv/C71BERCTGqaAUJY7W1bNs3W5uHjWI3qnJfocjIiIiIhHollGD6NktkSItzi0iIl1MBaUosWL9XmqO11OYO8TvUEREREQkQvVMSSJ/7GD+/PEeqmtP+B2OiIjEMBWUokRx6U7Oz+hJ7tBz/Q5FRERERCJYQe4Qjp1oYPlHe/wORUREYpgKSlHg0wM1lO74gpm52ZiZ3+GIiIiISAQbk5XGxQN6UVy60+9QREQkhqmgFAUWlJaTlGDcPj7T71BEREREJMKZGQW52XxUUcUne474HY6IiMQoFZQi3ImGAIvXVnDdJf3p3yvV73BEREREJApMG5dJt8QEirU4t4iIdBEVlCLc65/s52BNHYUTs/0ORURERESixLk9u/FXIwew5MNd1J5o8DscERGJQSooRbji0p0M6J3CNcP7+R2KiIiIiESRwtwhVB07wSub9vkdioiIxCAVlCLYnqpjvLn1ADMmZJOUqFMlIiIiIqG78oJ0ss7trsW5RUSkS6hKEcEWlVUQcDAzR7e7iYiIiEjHJCQYM3Oy+Z9tleysPOp3OCIiEmNUUIpQgYBjwZpyrrwgnSHpPfwOR0RERESi0PQJWSQYLFyjxblFRKRzqaAUod7dXkn5oWMU5OrqJBERERE5M4P7dOfai/qxsKyC+oaA3+GIiEgMUUEpQhWXlpPWPZkbRw70OxQRERERiWIFudnsPVLLW3854HcoIiISQ1RQikCHj9bx8sa93DZ2MKnJiX6HIyIiIiJR7LpLBpBxTjeKPtBtbyIi0nlUUIpASz/cRV19gILcIX6HIiIiIiJRrltSAneMz2L15v3sr671OxwREYkRKihFGOccRaXljM5KY8Tg3n6HIyIiIiIxYEZONvUBx4trd/kdioiIxAgVlCLM+l1VbN5bzcwcLcYtIiIiIp3jwv7nkDv0XBaUluOc8zscERGJASooRZii0nJSkxPIHzvY71BEREREJIYU5A5h+8EvKd3xhd+hiIhIDFBBKYIcratn+brd3DxqEL1Tk/0OR0RERERiyM2jBnJOShJFpTv9DkVERGJASAUlM5tiZlvMbJuZ3dfK9vPM7HUz+9jMSswsq8m2x8xsg/dV0KT9OjNb67U/Z2ZJXruZ2W+8Y31sZuM7I9FosGL9XqqP11OoxbhFREREpJP16JZE/tjBrFi/hyO1J/wOR0REoly7BSUzSwR+B9wEjABmmdmIFt3mAc8750YDDwP/4u17CzAeGAtcBtxrZr3NLAF4Dih0zl0KfA7M9sa6CRjufd0JPHFWGUaR4tKdnJ/Rk9yh5/odioiIiIjEoMLcbGpPBFi2brffoYiISJQL5QqlicA259x251wdUARMbdFnBLDae/1Gk+0jgLecc/XOuS+Bj4EpQDpQ55zb6vV7FbjDez2VYHHKOefeA/qY2aAzyC2qfHqghtIdXzAzNxsz8zscEREREYlBozLTuGRgL4pLy/0ORUREolxSCH0ygaafOBUErzZq6iPgduDXwDSgl5mle+0/N7NfAj2AScAm4CCQZGY5zrkyYDpw8rFmrR0vE9jT9IBmdifBK5gYMGAAJSUlIaTScTU1NV02dlPFW+pIMBhY+zklJeH/gA9Xnn6LhzzjIUeIjzzjIUdQnrEkHnIUiXZmRmFuNg8t38TG3VWMHJzmd0giIhKlQikoheJe4LdmNgd4C9gFNDjnXjGzXOAd4ADwrtfuzKwQ+HczSwFeARo6ckDn3JPAkwA5OTkuLy+vk1JprqSkhK4a+6QTDQHu/e/Xuf5rA7jtxpwuPVZbwpFnJIiHPOMhR4iPPOMhR1CesSQechSJBbeNy+T/rNzMgtJyfjFVBSURETkzodzytouvrh4CyPLaGjnndjvnbnfOjQPu99oOe9/nOufGOuduAAzY6rW/65z7unNuIsEi1Mnb39o9Xqx5/ZP9HKypoyA3u/3OIiIiIiJnoU+PbkwZOZAlH+6i9kSH/k1XRESkUSgFpVJguJkNM7NuQCGwrGkHM8vwFtoG+BnwB6890bv1DTMbDYwmeDUSZtbf+54C/G/g997+y4BveU97uxyocs41u90t1iwoK2dA7xSuvaif36GIiIiISBwozM3mSG09L2/Y63coIiISpdotKDnn6oHvA6uAT4AFzrmNZvawmeV73fKALWa2FRgAzPXak4G3zWwTwdvTvumNB/ATM/uE4ELdy51zJxf1XgFsB7YB/wn8w1nmGNH2VtVSsmU/0ydkkZQYSn1PREREROTsXH5+OkP69tDi3CIicsZCWkPJObeCYKGnaduDTV4vAha1sl8twSe9tTbmT4CftNLugO+FElcsWLSmnICDmTm63U1EREREwiMhwZiZk8W8V7ay4+CXDM3o6XdIIiISZXRJjI8CAUdxWTlXXpDOeen6EBcREZHQmNkUM9tiZtvM7L5WtqeYWbG3/X0zG9pk28+89i1mdmOT9j+Y2X4z2xCeLMRv0ydkk2DB5RdEREQ6SgUlH723vZLyQ8e0GLeIiIiEzMwSgd8BNxG8EnyWmbW8Ivw7wBfOuQuBfwce8/YdQXA9zJHAFOBxbzyAZ702iRMD01KZdHF/Fq2poL4h4Hc4IiISZVRQ8lFRaTlp3ZO5ceRAv0MRERGR6DER2Oac2+6cqwOKgKkt+kwFnvNeLwImm5l57UXOuePOuc8Irlk5EcA59xZwKBwJSOSYmZvN/urjlGw54HcoIiISZVRQ8snho3W8vHEvt40dTGpyYvs7iIiIiARlAk3vUarw2lrt4z0QpQpID3FfiSPXXdKfjHNSKNLi3CIi0kEhLcotnW/ph7uoqw9QkDvE71BEREREQmZmdwJ3AgwYMICSkpIuOU5NTU2XjR1JIiHPif0CvLx5H0tfXk2f1M7/9+ZIyDEclGfsiIccIT7yjIccwb88VVDygXOOotJyRmWmMWJwb7/DERERkeiyC2i6AGOW19ZanwozSwLSgMoQ9z0t59yTwJMAOTk5Li8vryO7h6ykpISuGjuSREKeQ0bWsOKXb7I7dQi35V3Y6eNHQo7hoDxjRzzkCPGRZzzkCP7lqVvefLB+VxWb91ZrMW4RERE5E6XAcDMbZmbdCC6yvaxFn2XAbO/1dGC1c8557YXeU+CGAcOBD8IUt0So8/udw8RhfVlQWk7wj4mIiEj7VFDyQVFpOanJCeSPHex3KCIiIhJlvDWRvg+sAj4BFjjnNprZw2aW73V7Gkg3s23Aj4H7vH03AguATcDLwPeccw0AZjYfeBe42MwqzOw74cxL/FWYm82OyqO8/5nWZRcRkdDolrcwO1pXz/J1u7l51CB6pyb7HY6IiIhEIefcCmBFi7YHm7yuBWa0se9cYG4r7bM6OUyJIjddOoifv7SR4tJyLj8/3e9wREQkCugKpTBbsX4v1cfrKcjR7W4iIiIiEhm6d0tk6rjBrFi/h6pjJ/wOR0REooAKSmG2oLScYRk9mTisr9+hiIiIiIg0KswdwvH6AMvWdWiddhERiVMqKIXRpwdq+GDHIWbmZGNmfocjIiIiItLo0sw0RgzqTVFpud+hiIhIFFBBKYwWlJWTmGDcMSHT71BERERERE5RODGbjbuPsGFXld+hiIhIhFNBKUxONARYvKaC6y7pT/9eqX6HIyIiIiJyiqljMklJSqCodKffoYiISIRTQSlMVm/ez8GaOgpztRi3iIiIiESmtB7J3HTpQF5at5tjdQ1+hyMiIhFMBaUwKS4tZ0DvFK69qJ/foYiIiIiItKkgdwjVtfWs3LDH71BERCSCqaAUBnurainZsp/pE7JIStSvXEREREQi1+Xn92Voeg8tzi0iIqel6kYYLFpTTsDBzBzd7iYiIiIikc3MmJmbzQefHWL7gRq/wxERkQilglIXCwQcxWXlXHF+Ouel9/Q7HBERERGRdk0fn0VigrGgrMLvUEREJEKpoNTF3tteSfmhYxRO1NVJIiIiIhId+vdOZdLF/Vm0poITDQG/wxERkQikglIXKyotp3dqEjeOHOh3KCIiIiIiISvMzeZgzXHe2Lzf71BERCQCqaDUhQ4frePljXuZNi6T1OREv8MREREREQlZ3sX96N8rhWItzi0iIq1QQakLLf1wF3X1AWbm6nY3EREREYkuSYkJTJ+QxRtb9rO3qtbvcEREJMKooNRFnHMUlZYzKjONkYPT/A5HRERERKTDZuZkE3CweK0W5xYRkeZUUOoi63dVsXlvta5OEhEREZGoNTSjJ5ef35fi0nICAed3OCIiEkFUUOoixaXlpCYnkD9msN+hiIiIiIicscLcIew8dJT3tlf6HYqIiEQQFZS6wLG6Bpat283Nlw4irXuy3+GIiIiIiJyxKZcOpHdqEsVlWpxbRES+ooJSF1ixfg/Vx+sp0O1uIiIiIhLlUpMTuW1cJis37OXw0Tq/wxERkQihglIXKC4tZ1hGTyYO6+t3KCIiIiIiZ60gN5u6+gBLP9zldygiIhIhVFDqZJ8eqOGDHYeYmZONmfkdjoiIiIjIWRs5OI1RmWkUlZbjnBbnFhERFZQ63YKychITjDsmZPodioiIiIhIp5mZm83mvdWs31XldygiIhIBVFDqRCcaAixeU8F1l/Snf69Uv8MREREREek0+WMGk5qcQFGpFucWEREVlDrV6s37OVhTR6EW4xYRERGRGJPWPZmbRw1i+brdHK2r9zscERHxmQpKnai4tJz+vVK49qJ+fociIiIiItLpCnKyqT5ez4r1e/0ORUREfKaCUifZW1VLyZb9zMjJIilRv1YRERERiT0Th/VlWEZPikt3+h2KiIj4TJWPTrJoTTkBBzNzdLubiIiIiMQmM6MgN5vSHV/w6YEav8MREREfhVRQMrMpZrbFzLaZ2X2tbD/PzF43s4/NrMTMsppse8zMNnhfBU3aJ5vZWjNbZ2b/bWYXeu1DzOwNM/vQG+/mzki0KwUCjgVlFVxxfjrnpff0OxwRERERkS5z+/hMEhOMBVqcW0QkrrVbUDKzROB3wE3ACGCWmY1o0W0e8LxzbjTwMPAv3r63AOOBscBlwL1m1tvb5wngG865scB/AQ947Q8AC5xz44BC4PEzTy883tteyc5DRynQYtwiIiIiEuP690pl8iX9Wby2ghMNAb/DERERn4RyhdJEYJtzbrtzrg4oAqa26DMCWO29fqPJ9hHAW865eufcl8DHwBRvmwNOFpfSgN3ttEes4rJyeqcmMeXSgX6HIiIiIiLS5QonZnOwpo7XP9nvdygiIuKTUApKmUDT61krvLamPgJu915PA3qZWbrXPsXMephZBjAJOHkZz98DK8ysAvhb4F+99oeAb3rtK4AfdCijMKs6eoKVG/Zy27hMUpMT/Q5HRERERKTLXTO8HwN6p2hxbhGROJbUSePcC/zWzOYAbwG7gAbn3Ctmlgu8AxwA3gUavH3uAW52zr1vZj8BfkWwyDQLeNY590szuwJ4wcwudc41u57WzO4E7gQYMGAAJSUlnZRKczU1Nacd+7XPT1BXH+AC20dJycEuiSEc2sszVsRDnvGQI8RHnvGQIyjPWBIPOYpIUFJiAjMmZPN4yTb2VB1jUFp3v0MSEZEwC6WgtIuvrioCyPLaGjnnduNdoWRm5wB3OOcOe9vmAnO9bf8FbDWzfsAY59z73hDFwMve6+/g3RbnnHvXzFKBDKDZ9bTOuSeBJwFycnJcXl5eCKl0XElJCW2N7ZzjX3/9NqMyuzM7/+ouOX64nC7PWBIPecZDjhAfecZDjhDePE+cOEFFRQW1tbVhOV5TaWlppKamhv244dQZOaamppKVlUVycnInRSUiXWVmTja/fWMbi8oq+MHk4X6HIyIiYRZKQakUGG5mwwgWkgqBv2nawbud7ZB3FdHPgD947YlAH+dcpZmNBkYDr3i7pZnZRc65rcANwCde+05gMvCsmX0NSCV4dVPEWb+ris17q3nktkv9DkVEREJQUVFBr169GDp0KGYW1mNXV1fTq1evsB4z3M42R+cclZWVVFRUMGzYsE6MTES6wpD0Hlx5QTrFZeV8b9KFJCSE931VRET81e4aSs65euD7wCqCRZ8FzrmNZvawmeV73fKALWa2FRiAd0USkAy8bWabCF5N9E1vge564LvAYjP7iOAaSj/x9vlH4Lte+3xgjnPOdUKuna64tJzU5ATyxwz2OxQREQlBbW0t6enpYS8mSWjMjPT0dF+uIBORM1OQm03FF8d459NKv0MREZEwC2kNJefcCoILZDdte7DJ60XAolb2qyX4pLfWxlwCLGmlfRNwVShx+elYXQPL1u3m5ksHkdZdl+WLiEQLFZMim86PSHS5ceRA0ronU1S6k6uHZ/gdjoiIhFEoT3mTVqxYv4fq4/UU5Ga331lERASorKxk7NixjB07loEDB5KZmdn4c11d3Wn3LSsr44c//GGHj7lu3TrMjJdffrn9ziIiHZSanMi0cZm8snEfX3x5+vcxERGJLSoonaHi0nKGZfRk4rC+fociIiJRIj09nXXr1rFu3Truuusu7rnnnsafu3XrRn19fZv75uTk8Jvf/KbDx5w/fz5XX3018+fPP5vQRUTaVJCbTV1DgCUf7mq/s4iIxAwVlM7A9gM1fLDjEDNzsnVpvoiInJU5c+Zw1113cdlll/HTn/6UDz74gCuuuIJx48Zx5ZVXsmXLFiD4NLxbb70VgIceeohvf/vb5OXlcf7557dZaHLOsXDhQp599lleffXVZmsTPfbYY4waNYoxY8Zw3333AbBt2zauv/56xowZw/jx4/n000+7OHsRiQVfG9SbMVlpFJeWE6FLn4qISBcIaQ0laa64rJzEBOOOCZl+hyIiImfoF8s3smn3kU4dc8Tg3vz8r0d2eL+KigreeecdEhMTOXLkCG+//TZJSUm89tpr/NM//ROLFy8+ZZ/NmzfzxhtvUF1dzcUXX8zdd99NcnLzNf3eeecdhg0bxgUXXEBeXh5//vOfueOOO1i5ciUvvfQS77//Pj169ODQoUMAfOMb3+C+++5j2rRp1NbWEggEzuwXISJxZ2ZuNvcv2cBHFVWMze7jdzgiIhIGukKpg040BFi8ZhfXXdKf/r1S/Q5HRERiwIwZM0hMTASgqqqKGTNmcOmll3LPPfewcePGVve55ZZbSElJISMjg/79+7Nv375T+syfP5/CwkIACgsLG297e+211/i7v/s7evToAUDfvn2prq5m165dTJs2DYDU1NTG7SIi7ckfM5juyYkUl+70OxQREQkTXaHUQas37+dgzXEKcrQYt4hINDuTK4m6Ss+ePRtf//M//zOTJk1iyZIl7Nixg7y8vFb3SUlJaXydmJh4yvpLDQ0NLF68mJdeeom5c+finKOyspLq6uouyUFE4luv1GRuGT2IZet288AtI+iZor9miIjEOl2h1EELSsvp3yuFvIv7+R2KiIjEoKqqKjIzg7dUP/vss2c8zuuvv87o0aMpLy9nx44dfP7559xxxx0sWbKEG264gWeeeYajR48CcOjQIXr16kVWVhZLly4F4Pjx443bRURCUZCbzZd1Dfx5/R6/QxERkTBQQakD9lbV8saW/UyfkEVSon51IiLS+X7605/ys5/9jHHjxp32qW/tmT9/fuPtayfdcccdzJ8/nylTppCfn09OTg5jx45l3rx5ALzwwgv85je/YfTo0Vx55ZXs3bv3rHIRkfiSc965nN+vJ8Wl5X6HIiIiYaBrUTtg0ZpyAg5m6nY3ERE5Sw899FCr7VdccQVbt25t/PnRRx8FIC8vr/H2t5b7btiw4ZRxnnnmmVPa8vPzyc/PB+C+++5rfLrbScOHD2f16tWhpiAi0oyZUZibzf9ZsZlt+6u5sH8vv0MSEZEupMtsQhQIOBaUVXDF+ekMzejZ/g4iIiIiInHm9vFZJCWYrlISEYkDKiiF6L3tlew8dJSCXF2dJCIiIiLSmoxzUrj+awNYvHYXdfUBv8MREZEupIJSiIrLyumdmsSUSwf6HYqIiIiISMQqmJjNoS/reO2TfX6HIiIiXUgFpRBUHT3Byg17uW1cJqnJiX6HIyIiIiISsa4Z3o9Baam67U1EJMapoBSCpeuCl+zqdjcRERERkdNLTDBmTMjirb8cYNfhY36HIyIiXUQFpXY45ygqLefSzN6MHJzmdzgiIiIiIhFvhvdU5IVlukpJRCRWqaDUjs+PBPhkzxEKcof4HYqIiES5SZMmsWrVqmZt//Ef/8Hdd9/d5j55eXmUlZUBcPPNN3P48OFT+jz00EPMmzfvtMdeunQpmzZtavz5wQcf5LXXXutI+Kf1ox/9iIsvvphAQIvwighk9+3BVRdksLCsgoBzfocjIiJdQAWldrxZUU9KUgL5Ywb7HYqIiES5WbNmUVRU1KytqKiIWbNmhbT/ihUr6NOnzxkdu2VB6eGHH+b6668/o7FaCgQCLFmyhMzMTN58881OGVNOz8ymmNkWM9tmZve1sj3FzIq97e+b2dAm237mtW8xsxtDHVOkowpys9l1+BibKhv8DkVERLpAkt8BRLJjdQ28t6eeW0ZlktY92e9wREQkyk2fPp0HHniAuro6unXrxo4dO9i9ezdf//rXufvuuyktLeXYsWNMnz6dX/ziF6fsP3ToUMrKysjIyGDu3Lk899xz9O/fn+zsbCZMmADAf/7nf/Lkk09SV1fHhRdeyAsvvMC6detYtmwZb775Jo8++iiLFy/mkUce4dZbb2X69Om8/vrr3HvvvdTX15Obm8sTTzxBSkoKQ4cOZfbs2SxfvpwTJ06wcOFCLrnkklPiKikpYeTIkeTn5zN//nwmTZoEwL59+7jrrrvYvn07AE888QRXXnklzz//PPPmzcPMGD16NC+88EIX/tZjj5klAr8DbgAqgFIzW+ac29Sk23eAL5xzF5pZIfAYUGBmI4BCYCQwGHjNzC7y9mlvzPDZuoqBe96GD3eBGWBtfOer7232Oc33xv3N+/FM9235/XRjNd/W48udsH9zO/m1d7xQY+3oGKfJo3F72/5q5AD69Ejm1c/rmbznSPMwvLjMGiNsMqQ16XdymzX+3NoYjXvaqf2/Ok7z/k0O1+Y2MzslPuOroJv2P1bvqDle33r/tuJs9ffQSuwh/L5FRMJNBaXTWLF+D8fqYaYW4xYRiT0r74O96zt3zIGj4KZ/bXNz3759mThxIitXrmTq1KkUFRUxc+ZMzIy5c+fSt29fGhoamDx5Mh9//DGjR49udZw1a9ZQVFTEunXrqK+vZ/z48Y0Fpdtvv53vfve7ADzwwAM8/fTT/OAHPyA/P7+xgNRUbW0tc+bM4fXXX+eiiy7iW9/6Fk888QQ/+tGPAMjIyGDt2rU8/vjjzJs3j6eeeuqUeObPn8+sWbO47rrreOSRRzhx4gTJycn88Ic/5Nprr2XJkiU0NDRQU1PDxo0befTRR3nnnXfIyMjg0KFDZ/SrjnMTgW3Oue0AZlYETAWaFn+mAg95rxcBv7Xg30inAkXOuePAZ2a2zRuPEMYMn7d/ySXl78MWX44eVhMBSv2O4my0XYxKwSh1jvoj4H7fvCDS8iY4R8vtp+9Pu9s7PmZr/V2zn08fwwSMo2+3HLPzY2p65Pb6t4yx9T6tbLfmPzfd4zwHO0pa7tV6Hqc7UntxhNqn7QNZs1hOt2vL42Q6x6dvnv5cdSCQFuOEskvov5u2eoYS74BAgC1vhf4U8/ZGDC23zjpaaPo3BNj8dmc/qb3zCrzt/vkM8VCprhfk5Z1tOB2mgtJpJCYYI9MTuGyPGeSQAAAMH0lEQVRYX79DERGRGHHytreTBaWnn34agAULFvDkk09SX1/Pnj172LRpU5sFpbfffptp06bRo0cPAPLz8xu3bdiwgQceeIDDhw9TU1PDjTfe2OoYJ23ZsoVhw4Zx0UXBi1Rmz57N7373u8aC0u233w7AhAkTePHFF0/Zv66ujhUrVvCrX/0KgMsuu4xVq1Zx6623snr1ap5//nkAEhMTSUtL4/nnn2fGjBlkZGQAwSKbdFgm0HSl4wrgsrb6OOfqzawKSPfa32uxb6b3ur0xATCzO4E7AQYMGEBJSckZJXE63bLu5lifWfTs0YPgdNthjsbXzX/+6rs516zNXNPtLX+GU8dta4zWxmw6bltjtDZm83Fqa4+RmprSSqzNx+x4fifHCHd+TdugPhBgf80JEhMTmxUZmoZmuOZ/qWoSg6P5X91ck78mu1b6N4+97R9bK8U0Haq1MkyzmFqJsaGhgcTExCZxOrz/WtUyb8M1S6VlnifLW81+ds1jaK1/swxa/K6sZV6nRNk8B8MRCDgSEs7uL9Qtj9vaYVvGf2baH6O14wQCDmuSo4U0VCjxhhJPKFyzb6eOEdrvzlkAs85aBafz1krrnHMf5BICWKeu9BPmPEM8XL0ld8nncXtUUDqN28Zl0qfqL7rEVEQkFp3mSqKuNHXqVO655x7Wrl3L0aNHmTBhAp999hnz5s2jtLSUc889lzlz5lBbW3tG48+ZM4elS5cyZswYnn322bOeXKSkpADBglB9ff0p21etWsXhw4cZNWoUzjmOHTtG9+7dufXWW8/quBK5nHNPAk8C5OTkuLwu+hfRkpISLvfhX1vDraSkhAkxnufOkhK66s9JJClRnjEjHnKE+MgzHnIE//LUotwiIiJhdM455zBp0iS+/e1vNy7GfeTIEXr27ElaWhr79u1j5cqVpx3jmmuuYenSpRw7dozq6mqWL1/euK26uppBgwZx4sQJ/vjHPza29+rVi+rq6lPGuvjii9mxYwfbtm0D4IUXXuDaa68NOZ/58+fz1FNPsWPHDjZs2MBnn33Gq6++ytGjR5k8eTJPPPEEEPyX+6qqKq677joWLlxIZWUlgG55OzO7gKb342d5ba32MbMkIA2oPM2+oYwpIiIi0kgFJRERkTCbNWsWH330UWNBacyYMYwbN45LLrmEv/mbv+Gqq6467f7jx4+noKCAMWPGcNNNN5Gbm9u47ZFHHuGyyy7jqquuaraAdmFhIf/2b//GuHHj+PTTTxvbU1NTeeaZZ5gxYwajRo0iISGBu+66K6Q8jh49yssvv8wtt9zS2NazZ0+uvvpqli9fzq9//WveeOMNRo0axYQJE9i0aRMjR47k/vvv59prr2XMmDH8+Mc/DulY0kwpMNzMhplZN4KLbC9r0WcZMNt7PR1Y7ZxzXnuh9xS4YcBw4IMQxxQRERFppFveREREwuy2227DtVjH4tlnn221b9Nb1nbs2NH4+v777+f+++8/pf/dd9/N3XfffUr7VVddxaZNX62v3PR4kydP5sMPPzxln6bHy8nJOeX2uR49erR6hVHTtZZeeumlU7bPnj2b2bNnn9IuofHWRPo+sApIBP7gnNtoZg8DZc65ZcDTwAveotuHCBaI8PotILjYdj3wPedcA0BrY4Y7NxEREYkeKiiJiIiIRBnn3ApgRYu2B5u8rgVmtLHvXGBuKGOKiIiItEW3vImIiIiIiIiISIeooCQiIiIiIiIiIh2igpKIiMSVlmsXSWTR+RERERGJDiooiYhI3EhNTaWyslJFiwjlnKOyspLU1FS/QxERERGRdmhRbhERiRtZWVlUVFRw4MCBsB+7trY25gslnZFjamoqWVlZnRSRiIiIiHQVFZRERCRuJCcnM2zYMF+OXVJSwrhx43w5drjEQ44iIiIiEqRb3kREREREREREpENUUBIRERERERERkQ5RQUlERERERERERDrEYuFJN2Z2APi8i4bPAA520diRRHnGjnjIEeIjz3jIEZRnLOnKHM9zzvXrorHlDGkO1iniIc94yBGUZyyJhxwhPvKMhxzBpzlYTBSUupKZlTnncvyOo6spz9gRDzlCfOQZDzmC8owl8ZCjhE+8/HmKhzzjIUdQnrEkHnKE+MgzHnIE//LULW8iIiIiIiIiItIhKiiJiIiIiIiIiEiHqKDUvif9DiBMlGfsiIccIT7yjIccQXnGknjIUcInXv48xUOe8ZAjKM9YEg85QnzkGQ85gk95ag0lERERERERERHpEF2hJCIiIiIiIiIiHaKCksfMppjZFjPbZmb3tbI9xcyKve3vm9nQ8Ed59kLIc46ZHTCzdd7X3/sR59kwsz+Y2X4z29DGdjOz33i/g4/NbHy4YzxbIeSYZ2ZVTc7jg+GOsTOYWbaZvWFmm8xso5n9r1b6RPX5DDHHqD+fZpZqZh+Y2Udenr9opU/Uv8+GmGfUv88CmFmimX1oZn9qZVvUn0sJn3iYg2n+Ff2f1yfFwxwsHuZfoDlYiz5R/T4bT/MviLA5mHMu7r+AROBT4HygG/ARMKJFn38Afu+9LgSK/Y67i/KcA/zW71jPMs9rgPHAhja23wysBAy4HHjf75i7IMc84E9+x9kJeQ4CxnuvewFbW/kzG9XnM8Qco/58eufnHO91MvA+cHmLPrHwPhtKnlH/Puvl8WPgv1r7sxkL51Jf4fmKhzmY5l+N26P687oDecbCZ3bMz786kGcsnM+Yn4PF0/zLyyVi5mC6QiloIrDNObfdOVcHFAFTW/SZCjznvV4ETDYzC2OMnSGUPKOec+4t4NBpukwFnndB7wF9zGxQeKLrHCHkGBOcc3ucc2u919XAJ0Bmi25RfT5DzDHqeeenxvsx2ftquYhf1L/Phphn1DOzLOAW4Kk2ukT9uZSwiYc5mOZfQVH9eX1SPMzB4mH+BZqDtegW1e+z8TL/gsibg6mgFJQJlDf5uYJT30wa+zjn6oEqID0s0XWeUPIEuMO7dHWRmWWHJ7SwCvX3EO2u8C77XGlmI/0O5mx5l2uOI/gvDk3FzPk8TY4QA+fTuzx3HbAfeNU51+a5jOL32VDyhOh/n/0P4KdAoI3tMXEuJSziYQ6m+VdQzHxehyDqP7NPiof5F2gORvS/z8bL/AsibA6mgpK0tBwY6pwbDbzKV9VNiS5rgfOcc2OA/wss9Tmes2Jm5wCLgR855474HU9XaCfHmDifzrkG59xYIAuYaGaX+h1TVwghz6h+nzWzW4H9zrk1fsciEkOi+n1BmomJz2yIj/kXaA4WK2J9/gWROQdTQSloF9C0QpnltbXax8ySgDSgMizRdZ5283TOVTrnjns/PgVMCFNs4RTK+Y5qzrkjJy/7dM6tAJLNLMPnsM6ImSUT/JD/o3PuxVa6RP35bC/HWDqfAM65w8AbwJQWm2LhfbZRW3nGwPvsVUC+me0geOvOdWb2/1r0ialzKV0qHuZgmn8FRf3ndShi5TM7HuZfoDlYE9H+PtsohudfEIFzMBWUgkqB4WY2zMy6EVy8almLPsuA2d7r6cBq51y03ZfZbp4t7n3OJ3gvcaxZBnzLgi4Hqpxze/wOqjOZ2cCT98qa2USC/69H3YeCl8PTwCfOuV+10S2qz2coOcbC+TSzfmbWx3vdHbgB2NyiW9S/z4aSZ7S/zzrnfuacy3LODSX4ObLaOffNFt2i/lxK2MTDHEzzr6Co/rwOVYx8Zsf8/As0B2vRLarfZ+Nh/gWROQdL6qqBo4lzrt7Mvg+sIvgkjj845zaa2cNAmXNuGcE3mxfMbBvBhfgK/Yv4zISY5w/NLB+oJ5jnHN8CPkNmNp/gExkyzKwC+DnBhdlwzv0eWEHwyRTbgKPA3/kT6ZkLIcfpwN1mVg8cAwqj6UOhiauAvwXWe/dEA/wTMARi5nyGkmMsnM9BwHNmlkhwMrbAOfenWHufJbQ8o/59tjUxeC4lDOJhDqb5V8x8XgNxMweLh/kXaA4WM++zxPH8C/ydg1n0/f8gIiIiIiIiIiJ+0i1vIiIiIiIiIiLSISooiYiIiIiIiIhIh6igJCIiIiIiIiIiHaKCkoiIiIiIiIiIdIgKSiIiIiIiIiIi0iEqKImIiIiIiIiISIeooCQiIiIiIiIiIh2igpKIiIiIiIiIiHTI/wdIqQs4mFoctAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 1440x432 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5nuQsCvIjvnN"
      },
      "source": [
        "### Carregar o modelo treinado para predição"
      ],
      "id": "5nuQsCvIjvnN"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I5oY9FMjkC23",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "de2a336b-a8d5-4aa8-d8b9-7048b1ec3d50"
      },
      "source": [
        "no_layers = 2\n",
        "vocab_size = 1000 + 1 # 1 extra para padding\n",
        "embedding_dim = 64\n",
        "output_dim = 1\n",
        "hidden_dim = 256\n",
        "\n",
        "# Cria a rede e carrega o modelo treinado.\n",
        "model = SentimentRNN(no_layers,vocab_size,hidden_dim,embedding_dim,drop_prob=0.5)\n",
        "\n",
        "model.load_state_dict(torch.load('/content/drive/MyDrive/FURB/trabalho_deep_learning_state_dict.pt'), strict=False)\n",
        "\n",
        "# Passar para GPU, se tiver\n",
        "model.to(device)"
      ],
      "id": "I5oY9FMjkC23",
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SentimentRNN(\n",
              "  (embedding): Embedding(1001, 64)\n",
              "  (lstm): LSTM(64, 256, num_layers=2, batch_first=True)\n",
              "  (dropout): Dropout(p=0.3, inplace=False)\n",
              "  (fc): Linear(in_features=256, out_features=1, bias=True)\n",
              "  (sig): Sigmoid()\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1fi_fUhTjv7E"
      },
      "source": [
        "def predict_text(text):\n",
        "        word_seq = np.array([vocab[preprocess_string(word)] for word in text.split() \n",
        "                         if preprocess_string(word) in vocab.keys()])\n",
        "        word_seq = np.expand_dims(word_seq,axis=0)\n",
        "        pad =  torch.from_numpy(padding_(word_seq,500))\n",
        "        inputs = pad.to(device)\n",
        "        batch_size = 1\n",
        "        h = model.init_hidden(batch_size)\n",
        "        h = tuple([each.data for each in h])\n",
        "        output, h = model(inputs, h)\n",
        "        return(output.item())"
      ],
      "id": "1fi_fUhTjv7E",
      "execution_count": 33,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_EgriiIlknkP"
      },
      "source": [
        "### Testar a predição de um elemento do dataframe, que irá retornar se o sentimento é positivo ou negativo"
      ],
      "id": "_EgriiIlknkP"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nEnnCQDrkn3s",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "07c55226-6a68-4a7a-8b7f-69bd9ae36c34"
      },
      "source": [
        "index = 1713\n",
        "print(df['review'][index])\n",
        "print('='*70)\n",
        "print(f'Actual sentiment is  : {df[\"sentiment\"][index]}')\n",
        "print('='*70)\n",
        "pro = predict_text(df['review'][index])\n",
        "status = \"positive\" if pro > 0.5 else \"negative\"\n",
        "pro = (1 - pro) if status == \"negative\" else pro\n",
        "print(f'Predicted sentiment is {status} with a probability of {pro}')"
      ],
      "id": "nEnnCQDrkn3s",
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Well, maybe I'm just having a bad run with Hindi movies lately. I asked the video store guy for Apharan (Prakash Jha) but being a new release, wasn't available yet. So I had to settle for this one. It turned out to be the stupidest Hindi movie I've seen (and I've seen quite a few). No wonder BOllywood is the laughing stock of the whole world! If IMDb had negative ratings, I would give Garam Masala a -10.<br /><br />I remember seeing a TV show about the jazzy premiere they had for its release in Mumbai. All the usual celebs and their sideys showed up. For some strange reason, people expect good stuff from Piyadarashan. I did not like Hulchul, Hungama, or his other films. Hulchul, probably dubbed from Tamil or Telugu, came across as very loud, in-your-face fare that again didn't make much sense except in a Bollywood flick. This latest piece of utter CXXP proved that this guy has NO BRAINS. Who made him a director, even a Bollywood director at that??? <br /><br />Anyhow, to the film now. What starts off as a romantic escapade turns into a non-sensical woman-hunt. Two fashion photographers working for a magazine share an apartment (isn't it similar to No Entry where the 2 dudes work for a gossip magazine and share an office? Jezuz Christ, now they're copying their own stories!) Well anyways, there's some cook or chef that tries to help one of the dodos in his quest for multi-lateral love (aka multi-tasking + multi-timing). What happened in the end, I've no idea. I switched off mid-way. What ridiculous junk. I can't believe they even released it. And how sadistic to wallow in their own filth! For which audience did they make it - the poor illiterate Indian masses (700 Mil at last count) or the well-heeled NRI desis staying in Phoren? Either ways it doesn't matter. Neither group has any clue what makes a good movie and probably deserves such god-awful stuff.<br /><br />Its a short review because there's nothing to write about but the usual bag of F-grade garbage. Bollywood should change its name to Follywood. And yes, this review is much better than the movie itself.\n",
            "======================================================================\n",
            "Actual sentiment is  : neg\n",
            "======================================================================\n",
            "Predicted sentiment is negative with a probability of 0.9999989907693134\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}