sentiment_analysis_1_ml_models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Analysis with Machine Learning Models\n",
    "\n",
    "Sentiment analysis, which is also known as opinion mining, is one of the major tasks of NLP. It studies people’s sentiments towards certain entities and detects polarity (e.g. a positive or negative opinion) within text. Sentiment analysis is often used in business to detect sentiment in social media, evaluate brand reputation and understand customers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.linear_model import SGDClassifier, LogisticRegression\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import plot_confusion_matrix\n",
    "\n",
    "from imblearn.under_sampling import RandomUnderSampler\n",
    "from imblearn.over_sampling import RandomOverSampler, SMOTE\n",
    "from imblearn.pipeline import Pipeline, make_pipeline\n",
    "\n",
    "import functions as f"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "M_bow = f.load_pickle('data/M_bow_100k.pickle')\n",
    "M_tfidf = f.load_pickle('data/M_tfidf_100k.pickle')\n",
    "M_svd = f.load_pickle('data/M_svd_100k.pickle')\n",
    "M_nmf = f.load_pickle('data/M_nmf_100k.pickle')\n",
    "M_word2vec = f.load_pickle('data/M_word2vec_100k.pickle')\n",
    "y = f.load_pickle('data/sentiment_100k.pickle')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Machine Learning Models Comparison\n",
    "\n",
    "There is no one answer which combination of text representation and machine learning model will perform best. In order to find the optimal solution, each configuration needs to be tested. There are 5 different text representations: bag of words, TF-IDF, co-occurrence matrix with SVD and NMF decomposition and word2vec. They will be used in the following predictive models:\n",
    "- Multinomial Naive Bayes\n",
    "- SGD classifier\n",
    "- Logistic Regression\n",
    "\n",
    "In order to evaluate model performance, cross validation is implemented, measuring: test accuracy, f1 score, precision and recall. In the first place, model will be assessed by accuracy and f1 score. At the same time, it's important to note that there is high class imbalance in the dataset - the majority of reviews is positive. Due to this fact, it's enough to label all reviews as positive to reach 91,5% accuracy. Because of that, accuracy is not a sufficient metric and it's also valuable to look at precision and recall.\n",
    "\n",
    "Since Naive Bayes takes only positive values as input, SVD and Word2Vec embeddings were transformed with MinMaxScaler."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Baseline accuracy:  0.9148\n"
     ]
    }
   ],
   "source": [
    "baseline = sum(y)/len(y)\n",
    "print('Baseline accuracy: ', baseline)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = MinMaxScaler()\n",
    "M_svd_positive = scaler.fit_transform(M_svd)\n",
    "M_word2vec_positive = scaler.fit_transform(M_word2vec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings_names = ['BOW', 'TFIDF', 'SVD', 'NMF', 'Word2Vec']\n",
    "embeddings = [M_bow, M_tfidf, M_svd, M_nmf, M_word2vec]\n",
    "embeddings_positive = [M_bow, M_tfidf, M_svd_positive, M_nmf, M_word2vec_positive]\n",
    "results_names = ['test_acc', 'f1', 'precision', 'recall']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multinomial Naive Bayes\n",
    "\n",
    "Naive Bayes Classifier is a simple model that’s usually used in a wide variety of classification tasks (both binary and multiclass). The name \"naive\" comes from the fact that it assumes the features that go into the model are independent of each other. It provides a way to calculate the probability of a piece of data belonging to a given class, given our prior knowledge.\n",
    "\n",
    "It performs best with BOW word vectors (the highest test accuracy and f1 score). Naive Bayes used with SVD and Word2Vec model return the baseline accuracy and recall equal to 1 which means that it assigned all reviews to the \"positive\" class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_acc</th>\n",
       "      <th>f1</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>BOW</th>\n",
       "      <td>0.94656</td>\n",
       "      <td>0.970859</td>\n",
       "      <td>0.968570</td>\n",
       "      <td>0.973164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TFIDF</th>\n",
       "      <td>0.93515</td>\n",
       "      <td>0.965680</td>\n",
       "      <td>0.935993</td>\n",
       "      <td>0.997311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVD</th>\n",
       "      <td>0.91480</td>\n",
       "      <td>0.955504</td>\n",
       "      <td>0.914800</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NMF</th>\n",
       "      <td>0.72811</td>\n",
       "      <td>0.841202</td>\n",
       "      <td>0.903112</td>\n",
       "      <td>0.787243</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Word2Vec</th>\n",
       "      <td>0.91480</td>\n",
       "      <td>0.955504</td>\n",
       "      <td>0.914800</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          test_acc        f1  precision    recall\n",
       "BOW        0.94656  0.970859   0.968570  0.973164\n",
       "TFIDF      0.93515  0.965680   0.935993  0.997311\n",
       "SVD        0.91480  0.955504   0.914800  1.000000\n",
       "NMF        0.72811  0.841202   0.903112  0.787243\n",
       "Word2Vec   0.91480  0.955504   0.914800  1.000000"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "naive = MultinomialNB()\n",
    "\n",
    "naive_cv = f.model_cv(naive, embeddings_positive, y)\n",
    "f.df_model_cv(naive_cv, embeddings_names, results_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SGD Classifier\n",
    "\n",
    "SGD Classifier implements regularized linear models with stochastic gradient descent learning. In the first example it fits SVM model, and in the second logistic regression.\n",
    "\n",
    "An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible.\n",
    "Logistic regression is a statistical model that in its basic form uses a logistic function to model a binary dependent variable.\n",
    "\n",
    "Both SGD Classifiers return similar results. The best solution is achieved with BOW word vectors and SVD model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_acc</th>\n",
       "      <th>f1</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>BOW</th>\n",
       "      <td>0.95621</td>\n",
       "      <td>0.976397</td>\n",
       "      <td>0.963097</td>\n",
       "      <td>0.990074</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TFIDF</th>\n",
       "      <td>0.94960</td>\n",
       "      <td>0.973100</td>\n",
       "      <td>0.950783</td>\n",
       "      <td>0.996491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVD</th>\n",
       "      <td>0.85770</td>\n",
       "      <td>0.921848</td>\n",
       "      <td>0.925756</td>\n",
       "      <td>0.918135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NMF</th>\n",
       "      <td>0.91480</td>\n",
       "      <td>0.955504</td>\n",
       "      <td>0.914800</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Word2Vec</th>\n",
       "      <td>0.94956</td>\n",
       "      <td>0.972803</td>\n",
       "      <td>0.960062</td>\n",
       "      <td>0.985931</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          test_acc        f1  precision    recall\n",
       "BOW        0.95621  0.976397   0.963097  0.990074\n",
       "TFIDF      0.94960  0.973100   0.950783  0.996491\n",
       "SVD        0.85770  0.921848   0.925756  0.918135\n",
       "NMF        0.91480  0.955504   0.914800  1.000000\n",
       "Word2Vec   0.94956  0.972803   0.960062  0.985931"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sgd = SGDClassifier(random_state=9, n_jobs=-1)\n",
    "\n",
    "sgd_cv = f.model_cv(sgd, embeddings, y)\n",
    "f.df_model_cv(sgd_cv, embeddings_names, results_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_acc</th>\n",
       "      <th>f1</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>BOW</th>\n",
       "      <td>0.95324</td>\n",
       "      <td>0.974855</td>\n",
       "      <td>0.959370</td>\n",
       "      <td>0.990850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TFIDF</th>\n",
       "      <td>0.94032</td>\n",
       "      <td>0.968344</td>\n",
       "      <td>0.940585</td>\n",
       "      <td>0.997792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVD</th>\n",
       "      <td>0.84823</td>\n",
       "      <td>0.915755</td>\n",
       "      <td>0.928278</td>\n",
       "      <td>0.904143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NMF</th>\n",
       "      <td>0.91480</td>\n",
       "      <td>0.955504</td>\n",
       "      <td>0.914800</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Word2Vec</th>\n",
       "      <td>0.94950</td>\n",
       "      <td>0.972703</td>\n",
       "      <td>0.962119</td>\n",
       "      <td>0.983548</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          test_acc        f1  precision    recall\n",
       "BOW        0.95324  0.974855   0.959370  0.990850\n",
       "TFIDF      0.94032  0.968344   0.940585  0.997792\n",
       "SVD        0.84823  0.915755   0.928278  0.904143\n",
       "NMF        0.91480  0.955504   0.914800  1.000000\n",
       "Word2Vec   0.94950  0.972703   0.962119  0.983548"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sgd_log = SGDClassifier(loss='log', penalty='elasticnet', random_state=9, n_jobs=-1)\n",
    "\n",
    "sgd_log_cv = f.model_cv(sgd_log, embeddings, y)\n",
    "f.df_model_cv(sgd_log_cv, embeddings_names, results_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logistic Regression\n",
    "\n",
    "Logistic regression, despite its name, is a linear model for classification rather than regression. In this model, the probabilities describing the possible outcomes of a single trial are modeled using a logistic function.\n",
    "\n",
    "It returns the best results for TF-IDF word vectors. It is also the best result achieved so far. Logistic Regression with SVD and NMF word vectors assigned all observations to the \"positive\" class (recall = 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_acc</th>\n",
       "      <th>f1</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>BOW</th>\n",
       "      <td>0.95678</td>\n",
       "      <td>0.976600</td>\n",
       "      <td>0.967468</td>\n",
       "      <td>0.985909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TFIDF</th>\n",
       "      <td>0.95808</td>\n",
       "      <td>0.977440</td>\n",
       "      <td>0.962655</td>\n",
       "      <td>0.992687</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVD</th>\n",
       "      <td>0.91480</td>\n",
       "      <td>0.955504</td>\n",
       "      <td>0.914800</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NMF</th>\n",
       "      <td>0.91480</td>\n",
       "      <td>0.955504</td>\n",
       "      <td>0.914800</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Word2Vec</th>\n",
       "      <td>0.95048</td>\n",
       "      <td>0.973322</td>\n",
       "      <td>0.959593</td>\n",
       "      <td>0.987451</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          test_acc        f1  precision    recall\n",
       "BOW        0.95678  0.976600   0.967468  0.985909\n",
       "TFIDF      0.95808  0.977440   0.962655  0.992687\n",
       "SVD        0.91480  0.955504   0.914800  1.000000\n",
       "NMF        0.91480  0.955504   0.914800  1.000000\n",
       "Word2Vec   0.95048  0.973322   0.959593  0.987451"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logreg = LogisticRegression(max_iter=500, random_state=9, n_jobs=-1)\n",
    "\n",
    "logreg_cv = f.model_cv(logreg, embeddings, y)\n",
    "f.df_model_cv(logreg_cv, embeddings_names, results_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Verification of model performance\n",
    "\n",
    "Logistic Regression with TF-IDF word vectors resulted in 95,8% accuracy and 0.9774 f1-score. It's the best result achieved so far.\n",
    "\n",
    "Let's analyze results of this particular model:\n",
    "- try to predict sentiment of two example reviews\n",
    "- analyze errors (true positiives, false positives etc)\n",
    "- browse reviews that were assigned to incorrect class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "vectorizer = f.load_pickle('data/tfidf_vectorizer_100k.pickle')\n",
    "df = pd.read_csv('data/reviews_toys_games_100k.csv')\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(M_tfidf, np.array(y), test_size=0.33, random_state=9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(max_iter=500, n_jobs=-1, random_state=9)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logreg.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_test_pos = 'This game is amazing ^^, my son plays with it all the time!'\n",
    "review_test_neg = 'I\\'m really disappointed with this game. My son doesn\\'t like playing with it.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review_tokens_test_pos = f.normalize_single_text(review_test_pos)\n",
    "tfidf_vector_test_pos = vectorizer.transform([' '.join(review_tokens_test_pos)])\n",
    "logreg.predict(tfidf_vector_test_pos)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review_tokens_test_neg = f.normalize_single_text(review_test_neg)\n",
    "tfidf_vector_test_neg = vectorizer.transform([' '.join(review_tokens_test_neg)])\n",
    "logreg.predict(tfidf_vector_test_neg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Confusion matrix of reviews' sentiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = logreg.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA08AAAGoCAYAAACe4N4JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeZgcZbn38e+dhEAgIQkk7EvYZJM1LIKKIsIr4oKAgCCC53hQEXzdQHBF3DjgAgiKqAgICMiiqCigviwiKKvEgOxbEiAECAl7MnO/f1QldobJTCczPcuT7yfXXFd3V3XVU9OV/s1Tz11VkZlIkiRJkro2pL8bIEmSJEmDgZ0nSZIkSWqCnSdJkiRJaoKdJ0mSJElqgp0nSZIkSWrCsP5ugCSp9wxdfu3MuS/16jLzpaeuzMx39OpCJUlLjJKyyc6TJBUk577E0hvu26vLfPmO08b16gIlSUuUkrLJsj1JkiRJaoIjT5JUlIDwuJgkaSApJ5vsPElSSQKI6O9WSJL0HwVlUxldQEmSJElqMUeeJKk0hZRGSJIKUkg22XmSpNIUUhohSSpIIdlURhdQkiRJklrMkSdJKko5VzSSJJWinGwqYyskSZIkqcUceZKk0hRSVy5JKkgh2WTnSZJKEhRTGiFJKkRB2VTGVkiSJElSiznyJElFiWJKIyRJpSgnm+w8SVJpCimNkCQVpJBsKmMrJEmSJKnFHHmSpNIUUhohSSpIIdnkyJMkSZIkNcGRJ0kqSjl3cZcklaKcbLLzJEklCYopjZAkFaKgbCqjCyhJkiRJLebIkySVppDSCElSQQrJJjtPklSUcurKJUmlKCebytgKSZIkSWoxR54kqTRDyjgpV5JUkEKyyZEnSZIkSWqCI0+SVJKgmLpySVIhCsomO0+SVJpC7qUhSSpIIdlURhdQkiRJklrMkSdJKko5l4OVJJWinGyy8yRJpSmkNEKSVJBCsqmMLqAkSZIktZgjT5JUmkJKIyRJBSkkm8rYCkmSJElqMUeeJKkkEcXUlUuSClFQNtl5kqTSFFIaIUkqSCHZVMZWSJIkSVKLOfIkSaUppDRCklSQQrLJzpMkFaWcGxFKkkpRTjaVsRWSJEmS1GKOPElSaQopjZAkFaSQbHLkSZIkSZKa4MiTJJUkKKauXJJUiIKyyc6TJBWlnJNyJUmlKCebytgKSZIkSWoxR54kqTSFnJQrSSpIIdlk50mSSlNIaYQkqSCFZFMZWyFJkiRJLWbnaRCLiBER8duIeC4iftWD5RwYEVf1Ztv6S0S8OSLuafE6no+IdbuY/nBEvL2VbVjSRMSEiMiIGFY//0NEHNyC9UyOiLf29nL7XETv/khNMpdey1zq2kBuW7Mi4q0RMaXheUuypLvPecArJJvsPPWBiDggIm6pd/rH6z/83tQLi94HWBlYMTPfv7gLyczzMnO3XmhPS9V/PK/f1TyZeX1mbtjKdmTmyMx8sG7TWRHxjcVdVkQcEhFn1Y8X6CC0QuP6BrPM3D0zz+7JMjr77DJz08y8pkeNkwYBc6l3lJ5LfS0ijo2IY/tj3b2pN7IkIq6JiI90WO78z1n9x85Ti0XEZ4CTgG9RBcpawA+B9/bC4tcG7s3Mub2wrEGvlZ2OgWKwbONgaWeRor4cbG/+qCjmUt9Z0r4LB8v2DpZ2FqWgbDIVWygiRgPHAZ/IzEsz84XMnJOZv83MI+t5lo6IkyJiWv1zUkQsXU97a0RMiYjPRsT0+ujgh+tpXwO+AuxXHzn87/qIzbkN6+9Y6nRIRDwYEbMj4qGIOLDh9b82vG/HiLi5Lru4OSJ2bJh2TUR8PSJuqJdzVUSMW8j2z2v/UQ3t3zMi3hkR90bEMxHxhYb5t4uIGyNiZj3vqRExvJ52XT3bP+vt3a9h+Z+PiCeAnzcOnUfEevU6tq6frxYRMzobSo+ID0fEbxue3x8RFzU8fywitqwfZ0SsHxGHAgcCR9Vt+m3DIreMiDvr3+GFEbFM53vJAuZt48x6eTvUn80NEfH9iHgGOLaJz3l0RPys/h1OjYhvRMTQ7lbe1f7WsNxzIuKpiHgkIr4UUX17LaSdja/NrPe9HevXH6vXcXDD8veIiNsjYlY9/dgu2jr/iFxEzNsn5v3kvM84In4VEU/Un8N1EbFp/Xqnn100lI/EYv7fHBAKKY1Q7wtzyVxahFxq+LwOrfeFxyPisw3Tj42IiyPi3IiYBRwSEUMi4uiIeCAino6IiyJihYb3HBRVhjwdEV/srg0N7zskIv4aEd+JiGfr/WX3humrRcTl9e/3/oj4n27aeWxUGXFuvd9MiojXRcQx9b7xWETs1rCMD0fE3fW8D0bER7toa2OWzMv05yPihfr3OSEixkbE76LK1Gfrx2vU7/km8Gbg1Pp9p9avzx/pjO4zeaG/q35TSDbZeWqtHYBlgMu6mOeLwBuALYEtgO2ALzVMXwUYDawO/DdwWkSMzcyvUh01vLAexv1ZVw2JiOWAU4DdM3MUsCNwRyfzrQD8vp53ReB7wO8jYsWG2Q4APgysBAwHPtfFqleh+h2sThWqPwE+CEyk+mL4SvynfrcN+DQwjup3twtwGEBm7lTPs0W9vRc2LH8FqqOdhzauODMfAD4PnBcRywI/B85ayFD6tcCb6y/9VYGlgDfWv5N1gZHAnR2WfwZwHnBC3aZ3N0zeF3gHsA6wOXBIZ7+czDwrM+dNm7eNY+rl3Vg/3x54kOr3/c3OltPB2cBcYH1gK2A34COdrK8zne5v9bQf1NPWBd4CfIhqP5ins3ZuT/V7WxE4H7gA2LZu2wepgmFkPe8L9TLHAHsAH4+IPbvb2Myct0+MBD4D3APcVk/+A7BB3abbqD6v7j67eRbr/2Z37ZX6mblkLi1KLs2zM9V36W7A0bHgOUrvBS6m+u4+D/gksCdVTqwGPAucVrd7E+BHwEH1tBWBNRrWfWxmHttZu2rbU33HjwNOAH4WMf+v6F8CU+rl7gN8KyJ26aKdAO8GfgGMBW4HrqT623h1qoMMP254/3TgXcDyVPva96PuBHclM8c0ZNTJwPXA1Ho9P6faT9YCXgJOrd/zxXq+w+v3Ht7JopvJ5IX9rtQDdp5aa0VgRjflCwcCx2Xm9Mx8Cvga1ZfKPHPq6XMy8wrgeWBxa6fbgddHxIjMfDwzJ3cyzx7AfZn5i8ycm5m/BP5N9QUzz88z897MfAm4iCpgF2YO8M3MnEP1h/M44OTMnF2vfzLVlziZeWtm3lSv92GqL623NLFNX83MV+r2LCAzfwLcB/wdWJXqj4LXqGuIZ9fb8haqL9CpEbFR/fz6zGzvpi2NTsnMaZn5DPBbuv4ddWdaZv6g/r28ZhsbRcTKwO7Ap+ojytOB7wP7N7muTve3qEau9gOOqT+7h4HvsuC+2lk7H8rMn2dmG3AhsGa9/Fcy8yrgVaqOFJl5TWZOysz2zLyTKgi7+/wbt/1NwDeA92TmrHqZZ9btfQU4FtgiqiPvzejL/5u9KiJ69UdFMZfMpcXJpa/VmTKJ6g/+DzRMuzEzf11/d78EfBT4YmZOafju3Seq0cZ9gN9l5nX1tC9T/b6a9Uhm/qTOlLOpfn8rR8SawJuAz2fmy5l5B/BTFtxvO7YTqt/hlfX/h18B44HjG/aNCRExBiAzf5+ZD2TlWuAqqs52UyJiP6pO/t71/52nM/OSzHwxM2dTHXRsKvOazOROf1fNtrcVSskmO0+t9TQwLrqurV0NeKTh+SP1a/OX0SHkXqQ62rRIMvMFqv9oHwMej4jf11/A3bVnXptWb3j+xCK05+n6Py5UR1UAnmyY/tK890c1XP67qMqsZlEdwey09KLBU5n5cjfz/AR4PfCD+st6Ya4F3ko1AnQtcA3VF9lb6ueLYlF+R915bBHmXZvq6OTjdanATKqwX6nJ9y9sfxtHdTS3477auF901s6OnzWZubDPf/uI+H91CcJzVPtqd58/9XvXpPqD6eDMvLd+bWhEHB9V6cgs4OF69qaWSR/93+xtQTkBpZYwl8wlWPTPrPH7veP+0PG7f23gsoYMuptqBG/l+n3z56/3gacXoR3ztyEzX6wfjqyX+0zdCWls56Jm1IxO9o15+8LuEXFTVGWBM4F30nxGbUU1qvS++oAEEbFsRPw4qpK7WVSl+2OiiTJ7msvkhf2u+kVJ2WTnqbVuBF6mGr5emGlUXzTzrFW/tjheAJZteL5K48T66MquVEcf/k315d1de+a1aepitmlR/IiqXRtk5vLAF6j+v3Ulu5oYVUnYScDPqM7DWaGL2eeF1Jvrx9fSfUh1uf5FtLBldXy9q8/5MeAVYFxWpQJjMnP5zNy0h22bQXW0tuO+2rhf9PR3cT5wObBmZo4GTqf7z5+IGAH8GjgpM//QMOkAqjKNt1OVNkyY95Ym29ub/zelgcJcWjRLei7Ns2bD4477Q8f1PUZVijmm4WeZzJwKPN64rKhKF1ek56YBK0TEqA7t7JWMiuqcv0uA7wArZ+YY4Aqay6jxVGWyh2fm7Q2TPks1Yrt9vW/NKwNtJqOayWS1iJ2nFsrM56jqqU+L6oTUZSNiqfroxQn1bL8EvhQR46M6wfUrwLkLW2Y37gB2ioi1oipNOmbehIhYOSLeE1WN+StUZRZtnSzjCuB1UV3Gdlg9zLwJ8LvFbNOiGAXMAp6vjz5+vMP0J6lqexfFycCtmfkRqpr507uY91qquu4RmTmFqt74HVRf7Lcv5D2L06aFeYqqfKG75S30c87Mx6lKCb4bEctHVSu/XkQ0Xf7WmfpI3EXANyNiVESsTXV+0eLuq50ZRXXk8OWI2I6q89OMM4F/Z+YJHV4fRbWvP031x9u3Okzv7rPrzf+bfSda8KNimEuLbEnPpXm+XO8rm1KdV3NhF/OeTpUVa0PVeYiIeVdyvBh4V0S8KaoLbxxHL/wtmpmPAX8Dvh0Ry0TE5lTn453X9TubNhxYmiqn50Z18YVuL6Vfj/BeApyX/zknbp5RVKNbM+sO9Fc7TF/o59hHmdy7CsomO08tlpnfo9qhv0T1n+4x4HCqI+VQnaNxC9VJn5OoTmpfrPszZObVVF9odwK3smCwDKE6yjENeIbqqNVhnSzjaaoTIj9L9UfnUcC7MnPG4rRpEX2O6g/m2VRHHzt+0RwLnF2XAuzb3cLqL+t3UJWEQPU5bB311Zw6qsu9nqcKJ7I6b+ZB4IaGYfyOfgZsUrfp1wuZpyn1sPo3gRvq5b1hIfN19TlDddLocOAuqhN1L6Y6qttTR1AdRX4Q+CvVSNGZvbDceQ4DjouI2VR/rF3Uzfzz7A+8Lxa84t6bgXOoyhimUv0uburwvu4+u177vykNJObSIlmic6nBtcD9wJ+B72R1zurCnExVRXBV/X1+E9XFC6jPKfsEVX48TpVRUxaynEX1AaoKg2lUIz1frfe/HqvLAT9JlUvPUu0Tlzfx1jWoRg0/1SGj1qIafRxBNYp0E/DHDu89mepcsWcj4pROlt3qTNZCRGYrRnclSf1h6Arr5Ii3dzyA2TMv/OrDt2bmNr26UEkDXkRMAB4Clkrv3aUeKCmbvEmYJBWmP0+klSSpM6Vkk2V7kiRJktQER54kqTClHN2T1L/q+wf5haJeUUo22XmSpMKUElCSpHKUkk12nrqx9KgxueyKq3U/o9RguaWbuced9B/PPjGVF557poxkUcvF8OUylhnb383QILPVBqt0P5PUwW233TojM8f3dzsGCjtP3Vh2xdXY+cu/6O9maJDZYb0x/d0EDTInf7Sre5YuAu/NtESIZcay9HZH9HczNMjc8MfP93cTNAiNWCoe6fFCCsomLxghSZIkSU1w5EmSChJEMXXlkqQylJRNdp4kqTClBJQkqRylZJNle5IkSZKKExHviIh7IuL+iDi6k+mjI+K3EfHPiJgcER/ubpmOPElSYUo5uidJKkdfZ1NEDAVOA3YFpgA3R8TlmXlXw2yfAO7KzHdHxHjgnog4LzNfXdhy7TxJUmHsPEmSBpp+yKbtgPsz88F6/RcA7wUaO08JjIqqcSOBZ4C5XS3Usj1JkiRJg824iLil4efQDtNXBx5reD6lfq3RqcDGwDRgEvB/M7O9q5U68iRJJSnoXhqSpEK0JptmZOY23ay1o+zw/P8AdwBvA9YDro6I6zNz1sIW6siTJEmSpNJMAdZseL4G1QhTow8Dl2blfuAhYKOuFurIkyQVxnOeJEkDTT9k083ABhGxDjAV2B84oMM8jwK7ANdHxMrAhsCDXS3UzpMkFaSkGxFKksrQH9mUmXMj4nDgSmAocGZmTo6Ij9XTTwe+DpwVEZOoyvw+n5kzulqunSdJkiRJxcnMK4ArOrx2esPjacBui7JMz3mSpMJERK/+NLG+Xr8JoSSpLH2dTa3iyJMklaYPM6VVNyGUJBWmkIpyR54kST0x/yaEdWdo3k0IGy3yTQglSRqIHHmSpJJES65oNC4ibml4fkZmnlE/7uwmhNt3eP+pwOVUl4gdBezX3U0IJUkFaU029Qs7T5Kk7nR1I8KW3IRQkqSByM6TJBWmj4/uNXsTwuMzM4H7I2LeTQj/0TdNlCT1N0eeJEkDUh8HVEtuQihJKoudJ0nSEq9VNyGUJGkgsvMkSQXpp7u49/pNCCVJ5eiPbGoVO0+SVJoy8kmSVJJCssn7PEmSJElSExx5kqSSFHQvDUlSIQrKJkeeJEmSJKkJjjxJUmFKObonSSpHKdlk50mSClNKQEmSylFKNlm2J0mSJElNcORJkkpTxsE9SVJJCskmO0+SVJhSSiMkSeUoJZss25MkSZKkJjjyJEkFiYhiju5JkspQUjY58iRJkiRJTXDkSZIKU8rRPUlSOUrJJjtPklSYUgJKklSOUrLJsj1JkiRJaoIjT5JUmjIO7kmSSlJINtl5kqTClFIaIUkqRynZZNmeJEmSJDXBkSdJKkmUc3RPklSIgrLJkSdJkiRJaoIjT5JUkAAKObgnSSpESdlk50mSihLFlEZIkkpRTjZZtidJkiRJTXDkSZIKU8jBPUlSQUrJJkeeJEmSJKkJjjxJUmFKqSuXJJWjlGyy8yRJJYlySiMkSYUoKJss25MkSZKkJjjyJEkFCWDIkEIO70mSilBSNtl5kqTClFIaIUkqRynZZNmeJEmSJDXBkSdJKkwpVzSSJJWjlGxy5EmSJEmSmuDIkySVpKDLwUqSClFQNtl5kqSCBOWURkiSylBSNlm2J0mSJElNcORJkooSxRzdkySVopxssvMkSYUpJJ8kSQUpJZss25MkSZKkJjjyJEmFKaU0QpJUjlKyyZEnSZIkSWqCI0+SVJKC7qUhSSpEQdlk50mSClLSvTQkSWUoKZss25MkSZKkJjjypNfYfLXl+dB2azIk4P/dN4Pf/uvJBaZvvPJIPvu29Zn+/CsA3PzITC6783FWXX5pjnjLuvPnW2nk0lx8xzT+ePf0Pm2/+sd9dz/MFZddQ2Y7W2//enZ6+3adzjf10Sc446QL2PdD72TTLV/HnDlzOfPUi5g7t432tnY23WID3rb7jn3c+rIUcnBPeo1dJq7Dtz++C0OHDOEXf/wnJ1309wWmL7/scH581LtZY6XlGTp0CKde/A/Ov3oSAB9970QO3n0LiOCcP/yT0399S39sgvrAn/52F8d892La2ts56L078ulDdltgemZy9Hcv5uobJjNimeH88KsHscVGa/LyK3PY49CTeGXOXNrmtvGeXbbimI/uAcCke6bwmeMv4OVX5jBs2BC+8/n9mLjphH7YusGrlGwatJ2niBgDHJCZP6yfrwackpn79G/LBrcI+PAb1uLbV93L0y/O4Rt7bMRtjz3H1OdeXmC+fz85m+/85YEFXnt81it84bd3z1/Oae/fnFsendlnbVf/aW9v53eX/IWDP7YXy48ZxY+/fz4bvX49VlplxdfMd9Vv/8r6G609/7Vhw4ZyyGH7sPTSw2lra+Onp1zEBhuvw5oTVu3rzShGKaURg5X51BpDhgQnfmJX3veFC5k2YzZ/OeVg/nDT/dzz6NPz5/nIu7fmnkdn8IFjL2HF0SO4+af/w6/+32TWX30FDt59C3b5v+fw6pw2Lv7mvlz1jwd4cNqz/bhFaoW2tnaOPOEiLjv1cFZbeQxvO/hEdt9pMzZa9z+ZcvXf7uKBR5/i1ku/yi3/epjPHn8BfzrrSJYePozf/OiTjFx2aebMbWP3j3yPt++4Cdtutg5f/cGvOeoju7PrGzflqhsm89VTfs3vfvypftzSwaeUbBrMZXtjgMPmPcnMaQZTz60/bjmenPUy059/lbb25MaHnmXimmMWeTmvX3UUT85+hRkvvNqCVmqgmfLoE6wwbgwrjBvDsGFD2WyrDfn3vx54zXw3XX8Hm2yxPsuNXHb+axHB0ksPB6rQa29rr4qjpcHLfGqBiRuuyoOPz+SRJ55jztx2Lr32bt65wwYLzJPAyBHV98lyywzn2dkvM7etndettSI3/3saL70yl7b25IZJj/GuHTfoZC0a7G6d/DDrrjmOCWuMY/hSw9hr16254to7F5jnimvvZP89tiMi2HazdXhu9ks8MeM5IoKRyy4NwJy5bcyZ2zb/D/4ImP1CdSB51vMvscr40X27YRowWtZ5iogJEXF3RPwkIiZHxFURMSIi1ouIP0bErRFxfURsVM+/XkTcFBE3R8RxEfF8/frIiPhzRNwWEZMi4r31Ko4H1ouIOyLixHp9/6rf8/eI2LShLddExMSIWC4izqzXcXvDslQbu+xSPP3CnPnPn3nxVVZYbqnXzLfB+JF8+90bc9Qu67P6mGVeM32HCStw40PPtLStGjhmz3ye0WNGzX++/OiRzHru+QXmmTXzee6edD/b7rj5a97f3t7OD088lxO+/GPW23At1lzbUaeeiOjdn9KYT4PTqiuOYupTs+Y/nzZjNquuOHKBeX5y+W28bq0Vufv8T3DD6f/FMaf/iUy4++EZ7Pj6NRk7ahlGLD2MXbddl9XHL9/Xm6A+8PhTz7H6ymPnP19t5bE8/tRzHeaZueA8K43h8elVpUxbWztvPuDbvG63o3nr9huxzesnAPCtz+zDV075NZvu8SW+cvJlfOUT/hddVKVkU6tHnjYATsvMTYGZwN7AGcARmTkR+Bzww3rek4GTM3NbYFrDMl4G3peZWwM7A9+N6jDA0cADmbllZh7ZYb0XAPsCRMSqwGqZeSvwReAv9Tp2Bk6MiOU6NjoiDo2IWyLilldmL1lD+p3ti5kLPn/4mRf55CWTOOa3d3PVv6fz2Z3XW2D60CHBxDXHcNPDS9bvbkmWnbwWHfamP/z6GnZ715sZMuS1XztDhgzhsCM/yGeP/QhTHn2CJx+f0aKWSvMNunxqzKac80Iv/RoGj87+WOqYT2+buA6THpjOxgecxk6H/ZwTDtuVUcsO597HnubkX/2dy769Hxd/Y18mPziduW3tfdNw9ansuFPw2n2nk1nmjzANHTqE688/hsm//wa3TX6Eu+6v/sufecn1fOszezH599/gm5/em09+/bxeb7sGh1Z3nh7KzDvqx7cCE4AdgV9FxB3Aj4F5h5h3AH5VPz6/YRkBfCsi7gT+BKwOrNzNei8C3l8/3rdhubsBR9frvgZYBlir45sz84zM3CYzt1l61NiOk4v2zItzWLFhpGmFZYfz7ItzFpjnpTntvDK3Cp07ps5i6JBg1NJD50/fcvXleeiZF5n18ty+abT63fJjRvLczNnzn8967nlGjV7wuMTUx57kV+dcwfeO+xl3/fM+fnfJX7h70v0LzDNixDKss94a3Pfvh/ui2WWK6o+A3vwp1KDLp8ZsiqVec9yveNNmzF5gtGi1caN44pkFR7gP3G0zfnfDvQA8VJf4bbBGde7luVfeyVsPP5s9jjyfZ2e/7PlOhVptpTFMffI/n+20J59llXGju55n+szXlOGNHrUsb5q4AX++8S4Afvm7v/PunbcEYM+3b8Vtdz3Sqk0oU0HZ1OrO0ysNj9uAFYCZ9dG4eT8bd7OMA4HxwMTM3BJ4kipUFiozpwJPR8TmwH5UR/qgCrq9G9a9VmbevRjbVawHZrzAKssvw/iRwxk6JNhhnbHcOmXBiz6MXuY/1xlZb9yyBMHsV9rmv7bjOpbsLWlWX3MVnnnqWZ59+jnmzm1j0u33sNGm6y4wz2e+/N985ivVzyZbbMC79n4bG2+2Pi88/yIvvVTVkc95dS4P3Pso41daoT82owjVvTTKKI1oMfNpkLntnsdZb7WxrLXyaJYaNoS93rIxf7hpwQMwU6bPYqetqgvSjB+zLOuvsQIPP1Fl2LjR1bmWa4wfxbve+Douvuauvt0A9YmtN1mbBx59ikemzuDVOXO59Orb2H2nBcvFd99pMy74/T/ITG6e9BDLjxzBKuNGM+PZ2Tw3+0UAXnr5Va75xz1sMKE6HrLq+NHccNt9AFx3872su+b4vt2wQa6kbOrrq+3NAh6KiPdn5q/q8obNM/OfwE1UZRMXAvs3vGc0MD0z50TEzsC8y3TNBkaxcBcARwGjM3NS/dqVwBERcURmZkRslZm3997mDX7tCWf9/VGOfvsGDBkSXHPfDKbOfJldXjcOgD/fO4PtJ4zl7RuOp609ebWtnR9c9+D89w8fGrx+1eX56Y0ekVmSDB06hD32fhvn/PhS2tuTrbfflJVWHcfNN/wTgG3fuMVC3zt71gtcev6VZHuSmWy65evYsEPHS+oD5tMA19aeHPXDq7nkm/sydEhw3lWT+PcjM/jwO6vRgJ9fcQcnnv83TvvsO7nhR/9FBHztzGt4ZtZLAJzz5T0ZO2oEc9vaOfK0q3nu+Ve6Wp0GqWHDhnLCUfuy9ydPo60tOfA9b2Dj9VblzEuuB+C/9n4zu71xU66+YTJbv+9rjFhmKU77ygcBeGLGLA479he0tbfT3p687+1b8443bwbASV88gGO+ezFz29pZZvgwTvrCB/ptG9W/orPa0F5ZcMQE4HeZ+fr6+eeAkcDZwI+oyiGWAi7IzOMiYgPgXKrO6e+BQzNz9YgYB/y2nvcO4I3A7pn5cEScD2wO/AE4rcP6VgamAl/PzK/Vr40ATqIqzQjg4cx8V1fbMXbCJrnzl3/RO78ULWQnHdsAACAASURBVDF2WG/Rr1CoJdvJH92TKfdM6vGxtJFrbJSbHXFGbzRpvpuOfsutmblNry60H5WQT0OWXyOX3u6I3vulaInw7B8/399N0CA0YqnocQaUlE0tG3nKzIeB1zc8/07D5Hd08papwBvqI277A7fU75tBVW/e2ToO6PBS4/qepMP2ZeZLwEeb3wpJGnwKLrXrFeaTJPW9UrJpIN0kdyJwal0qMRP4r35ujyRJYD5JkmoDpvOUmdcDCz8xQpLUlIKvkNcvzCdJ6rlSsqnVV9uTJEmSpCIMmJEnSVIvKPvy4pKkwaigbLLzJEkFqe6lUUhCSZKKUFI2WbYnSZIkSU1w5EmSClPK0T1JUjlKySY7T5JUmELySZJUkFKyybI9SZIkSWqCI0+SVJhSSiMkSeUoJZsceZIkSZKkJjjyJEklKeheGpKkQhSUTXaeJKkgQRRTGiFJKkNJ2WTZniRJkiQ1wZEnSSpMIQf3JEkFKSWb7DxJUmGGlJJQkqRilJJNlu1JkiRJUhPsPElSYSJ690eSpJ7qj2yKiHdExD0RcX9EHL2Qed4aEXdExOSIuLa7ZVq2J0mSJKkoETEUOA3YFZgC3BwRl2fmXQ3zjAF+CLwjMx+NiJW6W66dJ0kqSHVEzuEiSdLA0U/ZtB1wf2Y+WLUhLgDeC9zVMM8BwKWZ+ShAZk7vbqGW7UlSYYZE7/50pxVlEZKksrQgm8ZFxC0NP4d2WOXqwGMNz6fUrzV6HTA2Iq6JiFsj4kPdbYcjT5KkxdaqsghJkroxIzO36WJ6Z4f/ssPzYcBEYBdgBHBjRNyUmfcubKF2niSpMH1cGtGSsghJUln6oWxvCrBmw/M1gGmdzDMjM18AXoiI64AtgIV2nizbk6TCtOCKRl2VRrSkLEKSVJZ+uNrezcAGEbFORAwH9gcu7zDPb4A3R8SwiFgW2B64u6uFOvIkSepOV6URLSmLkCSpJzJzbkQcDlwJDAXOzMzJEfGxevrpmXl3RPwRuBNoB36amf/qarl2niSpIAFEp/2ZlmlJWYQkqRz9kE0AZOYVwBUdXju9w/MTgRObXaZle5KknmhJWYQkSQORI0+SVJhmLi/eW1pVFiFJKktfZlMr2XmSpJJE9PkVjVpRFiFJKkg/ZFOrWLYnSZIkSU1w5EmSClPIwT1JUkFKySY7T5JUkACGlJJQkqQilJRNlu1JkiRJUhMceZKkwhRycE+SVJBSssmRJ0mSJElqgiNPklSYUi4HK0kqRynZZOdJkgoSUU5phCSpDCVlk2V7kiRJktQER54kqTClXA5WklSOUrLJzpMkFaaMeJIklaSUbLJsT5IkSZKasNCRp4hYvqs3Zuas3m+OJKmnSrmi0cKYT5I0+JSSTV2V7U0GkgVH2eY9T2CtFrZLkqSFMZ8kSf1ioZ2nzFyzLxsiSeq5AIaUcXBvocwnSRpcSsqmps55ioj9I+IL9eM1ImJia5slSVosEUQv/wxk5pMkDQIFZVO3naeIOBXYGTiofulF4PRWNkqSpO6YT5KkvtbMpcp3zMytI+J2gMx8JiKGt7hdkqTFNMAHi3qT+SRJg0Qp2dRM52lORAyhOgmXiFgRaG9pqyRJi22gl9r1IvNJkgaJUrKpmXOeTgMuAcZHxNeAvwL/29JWSZLUPfNJktSnuh15ysxzIuJW4O31S+/PzH+1tlmSpMVR0hWNumM+SdLgUFI2NVO2BzAUmENVGtHUFfokSeoD5pMkqc80c7W9LwK/BFYD1gDOj4hjWt0wSdLiKeVysN0xnyRp8Cglm5oZefogMDEzXwSIiG8CtwLfbmXDJEmLZ+B2d3qd+SRJg0Qp2dRMicMjLNjJGgY82JrmSJLUNPNJktSnFjryFBHfp6ohfxGYHBFX1s93o7qikSRpgImAIQO41K43mE+SNLiUlE1dle3Nu2LRZOD3Da/f1LrmSJJ6qpB86or5JEmDTCnZtNDOU2b+rC8bIklSM8wnSVJ/6faCERGxHvBNYBNgmXmvZ+brWtguSdJiGshXyOtN5pMkDR6lZFMzF4w4C/g51UUydgcuAi5oYZskSWrGWZhPkqQ+1EznadnMvBIgMx/IzC8BO7e2WZKkxRXRuz8DmPkkSYNEKdnUzH2eXolqnO2BiPgYMBVYqbXNkiQtjiCKuaJRE8wnSRoESsqmZjpPnwZGAp+kqi0fDfxXKxslSVITzCdJUp/qtvOUmX+vH84GDmptcyRJPTLwS+16jfkkSYNEQdnU1U1yL6O66WCnMnOvlrRIktQjpVzRaGHMJ0kafErJpq5Gnk7ts1YMYOuuuCznHzyxv5uhQWbstof3dxM0yLzy2JP93YTBZInPp83WW5mrLv1MfzdDg4zZJPVcVzfJ/XNfNkSS1DuauYzqYGY+SdLgU0o2lbIdkiRJktRSzVxtT5I0SATl1JVLkspQUjY13XmKiKUz85VWNkaS1HNDysinpplPkjTwlZJN3ZbtRcR2ETEJuK9+vkVE/KDlLZMkqQvmkySprzVzztMpwLuApwEy85/Azq1slCRp8Q2J3v0ZwMwnSRokSsmmZjpPQzLzkQ6vtbWiMZIkLQLzSZLUp5o55+mxiNgOyIgYChwB3NvaZkmSFkdEOSflNsF8kqRBoKRsaqbz9HGq0oi1gCeBP9WvSZIGoAFeatebzCdJGiRKyaZuO0+ZOR3Yvw/aIklS08wnSVJf67bzFBE/AbLj65l5aEtaJEnqkUIqI7plPknS4FFKNjVTtvenhsfLAO8DHmtNcyRJPRHAkFISqnvmkyQNAiVlUzNlexc2Po+IXwBXt6xFkiQ1wXySJPW1ZkaeOloHWLu3GyJJ6h3N3IOiUOaTJA1QpWRTM+c8Pct/asqHAM8AR7eyUZIkdcd8kiT1tS47T1FdkH0LYGr9UntmvubkXEnSwFFIWXmXzCdJGlxKyaYuO0+ZmRFxWWZO7KsGSZIWX0QUc1JuV8wnSRo8SsqmZsoP/xERW7e8JZIkLRrzSZLUpxY68hQRwzJzLvAm4H8i4gHgBaqrDWZmGliSNAAVcnBvocwnSRp8Ssmmrsr2/gFsDezZR22RJPWCIYUEVBfMJ0kaZErJpq46TwGQmQ/0UVskSWqG+SRJ6hdddZ7GR8RnFjYxM7/XgvZIknqgpLu4d8F8kqRBpKRs6qrzNBQYSX2ET5KkAcJ8kiT1i646T49n5nF91hJJUq8o5OBeV8wnSRpkSsmmbs95kiQNIlHOSbldKH8LJakkBWVTV/d52qXPWiFJUvPMJ0lSv1joyFNmPtOXDZEk9Y4ofGDGfJKkwaeUbOqqbE+SNMhUVzTq71ZIkvQfJWVTV2V7kiRJkqSaI0+SVJhSju5JkspRSjY58iRJkiRJTXDkSZIKE6XcTEOSVIxSssnOkyQVpKSTciVJZSgpmyzbkyRJkqQmOPIkSSUJKKQyQpJUioKyyc6TJBVmSCkJJUkqRinZZNmeJEmSJDXBkSdJKkhJJ+VKkspQUjY58iRJ6pGIeEdE3BMR90fE0V3Mt21EtEXEPn3ZPkmSeosjT5JUmL4sK4+IocBpwK7AFODmiLg8M+/qZL7/Ba7su9ZJkgaKQk55svMkSWUJhtCnCbUdcH9mPggQERcA7wXu6jDfEcAlwLZ92ThJ0kDQ59nUMpbtSZK6My4ibmn4ObRh2urAYw3Pp9SvzRcRqwPvA05vfVMlSaq0oqzckSdJKkjQktKIGZm5TRer7Cg7PD8J+HxmtkUpdRuSpKa1KJu6XmeLysrtPElSSaLPr2g0BViz4fkawLQO82wDXFB3nMYB74yIuZn5675poiSpX/V9NkGLysrtPEmSeuJmYIOIWAeYCuwPHNA4Q2auM+9xRJwF/M6OkySph8ZFxC0Nz8/IzDManndWVr594wIaysrfhp0nSVoy9eVd3DNzbkQcTlXuMBQ4MzMnR8TH6ume5yRJakU2dVVSDi0qK7fzJEnqkcy8Ariiw2uddpoy85C+aJMkaYnXkrJyO0+SVJD+OClXkqSu9FM2taSs3M6TJBWmL8v2JElqRl9nU6vKyu08SZIkSSpOK8rK7TxJUmEceJIkDTSlZJOdJ0kqSABD+rsRkiQ1KCmbStkOSZIkSWopR54kqSQBzd6rQpKkPlFQNjnyJEmSJElNcORJkgpTxrE9SVJJSskmO0+SVJDA+zxJkgaWkrLJsj1JkiRJaoIjT5JUmDKO7UmSSlJKNtl5kqTCFFIZIUkqSCnZZNmeJEmSJDXBkSdJKkoUcy8NSVIpyskmR54kSZIkqQmOPElSQQKPikmSBpaSssnOkyQVppTSCElSOUrJplI6gZIkSZLUUo48LcH+9Le7OOa7F9PW3s5B792RTx+y2wLTM5Ojv3sxV98wmRHLDOeHXz2ILTZas9v3nnHhNfzkousYNnQIu77p9Rz3yT15dNrTbL/vN1h/rZUA2GazCXz/mA/03caq5XbZYWO+/dl9GDpkCL/4zd846eyrF5g+etQITv3yB1lnjXG8/Oocjvj6edz9wOMAfHT/t3LwnjtCBOf8+gZO/+U1/bAF5Sjj2J70Wtf8/W6O+8FltLUn++2xPYcd+PYFpt//yJMcefwvmXzfFD73kT04dP+d50878vhf8pcb72LFsSO56qzP93XT1U96kk3LjxzBKV86gI3XW5VMOOLr53HzpIf6YzOKUEo2DbrOU0R8DHgxM8+JiEOAqzJzWj3tp8D3MvOu/mzjYNDW1s6RJ1zEZacezmorj+FtB5/I7jttxkbrrjp/nqv/dhcPPPoUt176VW7518N89vgL+NNZR3b53utvuZcrrp3EX395DEsPX4qnnpk9f3kTVh/H9ecf0x+bqxYbMiQ48ah9ed/hpzLtyZn85ewj+cN1k7jnoSfmz/PZD/8fJt07hYOO+gkbrL0yJ35+X/Y87AdsvN6qHLznjuxy8Im8OreNi085jKv+OpkHH3uqH7doEItySiMGE7Op9dra2vnKSZdw7nc/xirjx/Cej36fXd/4ejaYsMr8ecYsvyzHfnIvrvrrpNe8f5/dt+Pgvd7EZ751fl82W/2oJ9kEcPxn9+HPN97FIUf/jKWGDWXEMsP7a1MGv4KyadCV7WXm6Zl5Tv30EGC1hmkfMZyac+vkh1l3zXFMWGMcw5caxl67bs0V1965wDxXXHsn+++xHRHBtputw3OzX+KJGc91+d4zL7meTx28K0sPXwqA8SuM6vNtU9+buOkEHnxsBo9MfZo5c9u49OrbeOdbNl9gng3XWYXrbr4HgPseeZK1Vl2B8SuM4nUTVuHmSQ/z0itzaGtr54bb7uddb92iPzZDWmxmU+vdcfejrL36ONZarcqed79tK676678WmGfc2FFssfFaDBs29DXv336L9Rg9arm+aq4GgJ5k06jllmHHrdbjF7+5EYA5c9uY9fxLfb4NGnj6tPMUERMi4t8RcXZE3BkRF0fEshGxS0TcHhGTIuLMiFi6nv/4iLirnvc79WvHRsTnImIfYBvgvIi4IyJGRMQ1EbFNRHw8Ik5oWO8hEfGD+vEHI+If9Xt+HBGv/YZdAjz+1HOsvvLY+c9XW3ksjz/1XId5Zi44z0pjeHz6zC7fe/8j07nxjgd4+yEnssehJ3Hb5Efmz/fotKfZ6cDj2ePQk/jb7fe3atPUD1YdP5qpTz47//m0J59l1fGjF5jnX/dN5V07bwnA1puszZqrrMBqK43h7gemseNW6zN29HKMWHopdt1x0wX2Ly2aeVc06s2f0plNg8OTM2ay2kpj5j9fdfxonpzxXBfv0JKuJ9m09uorMmPm85z21Q9y7bmf5+QvHsCyjjwttpKyqT/WvSFwRmZuDswCPgOcBeyXmZtRlRJ+PCJWAN4HbFrP+43GhWTmxcAtwIGZuWVmNh4OuBjYq+H5fsCFEbFx/fiNmbkl0AYc2LGBEXFoRNwSEbc8NaPM0qHMfM1rHUdTO5mFiOjyvXPb2pk5+0Wu/vnnOO7/7smHv3AmmcnK45Zn0m+P47rzjuabn96L//nSWR7BKUhnQ/Edd5OTzr6aMcsvy3XnHc2h+72FO++dQltbO/c+/CQnn3M1l516OBef8gkm3zeVuW1tfdRyab5BlU3PPD2jVzZ6MOk0k/q+GRpEepJNw4YOZYsN1+TMi6/nLR/8X158+RU+dciufdRyDWT9cc7TY5l5Q/34XODLwEOZeW/92tnAJ4BTgZeBn0bE74HfNbuCzHwqIh6MiDcA91GF4g31cicCN9f/oUYA0zt5/xnAGQATJ27Tydf14LfaSmNeczRmlXGju55n+kxWGT+aV+fMXeh7V19pDO/eeQsigombTmBIBE/PfJ5xY0fNL+XbcuO1WGeNcTzw6HS22mTtVm6m+si06TNfMxr5RIcjwrNfeJnDjzt3/vN//uZrPDLtaQDOvfxGzr28Ko348mHvZtr0mX3Q6nKVUlfexwZVNm2x1cQis6krq4wfs8B3w+NPPcdKHXJLatSTbBqxzFJMmz6TW+sKmsv/fAefOtjOU0+Ukk39MfLU1Bd+Zs4FtgMuAfYE/riI67kQ2BfYG7gsq+GSAM6ujwZumZkbZuaxi7jcImy9ydo88OhTPDJ1Bq/OmculV9/G7jstWAe8+06bccHv/0FmcvOkh1h+5AhWGTe6y/e+862bc93N1d8a9z/yJK/OmcuKY0Yy49nZtLW1A/DwlBk8+NhTTFh9XN9utFrmtrseYb21xrPWaiuy1LCh7LXr1vzhugXPoVt+5AiWqs9D+NCeO/K32+9n9gsvAzBu7EgA1lh5LO/aeQsuvvKWvt2AwkQv/ywhzKYBbouN1uThKU/x2ONP8+qcufz2L7ez6xs37e9maQDrSTZNf3o2U598lvXXrq4SvNO2Gy5woQktulKyqT9GntaKiB0y80bgA8CfgI9GxPqZeT9wEHBtRIwEls3MKyLiJqCzk2RmAwu7IsGlwBeBR4B51yT9M/CbiPh+Zk6vyy9GZeYjC1lGsYYNG8oJR+3L3p88jba25MD3vIGN11uVMy+5HoD/2vvN7PbGTbn6hsls/b6vMWKZpTjtKx/s8r0AH3zPDhx+3HnssN83Gb7UUH507EFEBH+7/X6+ffrvGTpsKEOHBN89en/GjvbE3VK0tbVz1AkXcckpn2Do0OC8y2/i3w8+wYf3ehMAP7/0r2y4zir86NiDaGtv556HnuCIr583//3n/O9HGDt6OebObePIEy7iudmWdKrPmU0D3LBhQznuU3vzoc/9mLb2dvZ95/a8bp1VOfc31YDhB9/7RqY/PYv3fPR7PP/Cy8SQ4MyLr+Xqs49m1HLLcMTXzuGmO+7n2ede4A37HMunP/wO9tvjDf28VWqlnmbTUd/5FWccdwjDlxrKw1Nn8ImGESotuaKz81datrKICcAVwHXAjlRlCwcBOwDfoerM3Qx8HFgB+A2wDFUH8zuZeXZEHAs8n5nfiYi9gW8BL9XL+APwucy8pV7f74BNMnPdhjbsBxxDNeo2B/hEZt60sDZPnLhN3vB3j4Jr0Yzd9vD+boIGmVfuuYj2F6f3+GDa+ptukd+94MreaNJ8e26+6q2ZuU2vLnQAGYzZtMVWE/Oqaxc6WerUhLd8ur+boEHo5TtO63EGlJRN/THy1J6ZH+vw2p+BrTq89jhVacQCGksZMvMSqtKJed7aYd53dfL+C6nKJiSpONUVjZagYrveYzZJUouUlE1LwlVoJUmSJKnH+nTkKTMfBl7fl+uUpCVNIRc06jNmkyS1XinZ5MiTJEmSJDWhP855kiS1TBCF1JVLkkpRTjbZeZKkwpRSGiFJKkcp2WTZniRJkiQ1wZEnSSpISZeDlSSVoaRssvMkSSWJckojJEmFKCibLNuTJEmSpCY48iRJhSnl6J4kqRylZJMjT5IkSZLUBEeeJKkwpdxLQ5JUjlKyyc6TJBUkgCFl5JMkqRAlZZNle5IkSZLUBEeeJKkwpZRGSJLKUUo22XmSpMKUckUjSVI5Sskmy/YkSZIkqQmOPElSYUopjZAklaOUbHLkSZIkSZKa4MiTJBWkpMvBSpLKUFI22XmSpKJEMaURkqRSlJNNlu1JkiRJUhMceZKkkkQ5l4OVJBWioGyy8yRJhSkknyRJBSklmyzbkyRJkqQmOPIkSQWprmhUyvE9SVIJSsomR54kSZIkqQmOPElSYco4tidJKkkp2WTnSZJKU0pCSZLKUUg2WbYnSZIkSU1w5EmSClPKXdwlSeUoJZsceZIkSZKkJjjyJEmFKeRqsJKkgpSSTXaeJKkwheSTJKkgpWSTZXuSJEmS1ARHniSpNKUc3pMklaOQbLLzJEkFCcq5opEkqQwlZZNle5IkSZLUBEeeJKkkUc4VjSRJhSgomxx5kiRJkqQmOPIkSYUp5OCeJKkgpWSTnSdJKk0pCSVJKkch2WTZniRJkiQ1wZEnSSpKFHM5WElSKcrJJjtPklSYUq5oJEkqRynZZNmeJKlHIuIdEXFPRNwfEUd3Mv3AiLiz/vlbRGzRH+2UJKmnHHmSpIIEfXtObkQMBU4DdgWmADdHxOWZeVfDbA8Bb8nMZyNid+AMYPs+bKYkqR/1dTa1kiNPkqSe2A64PzMfzMxXgQuA9zbOkJl/y8xn66c3AWv0cRslSeoVjjxJUml6//DeuIi4peH5GZl5Rv14deCxhmlT6HpU6b+BP/Ry+yRJA10hQ092niSpMC24otGMzNxmoat7rex0xoidqTpPb+qthkmSBgevtidJUjXStGbD8zWAaR1niojNgZ8Cu2fm033UNkmSepWdJ0kqTB9fDvZmYIOIWAeYCuwPHLBge2It4FLgoMy8t09bJ0kaEEq5VLmdJ0kqTF/mU2bOjYjDgSuBocCZmTk5Ij5WTz8d+AqwIvDDqNJzbhdlgJKkAhXSd7LzJEnqmcy8Ariiw2unNzz+CPCRvm6XJEm9zUuVS1JJogU/kiT1RD9lUytu4m7nSZIkSVJRGm7ivjuwCfCBiNikw2zzbuK+OfB1qpu4d8myPUkqTCmXg5UklaMfsmn+TdwBImLeTdzvmjdDZv6tYf6mbuJu50mSChKUc0UjSVIZWpRNXd3AHVp0E3c7T5IkSZIGm65u4A4tuom7nSdJKowDT5KkgaYfsqklN3G38yRJpbH3JEkaaPo+m1pyE3c7T5IkSZKK0qqbuNt5kqTCeLU9SdJA0x/Z1IqbuHufJ0mSJElqgiNPklQYL1UuSRpoSskmO0/duO22W2eMWCoe6e92DFDjgBn93QgNKu4zC7d2by2okHxSF+6847YZq4webjZ1zu8ZLQ73m4XrlXwqJZvsPHUjM8f3dxsGqoi4pbuT6qRG7jNS7zCbFs7vGS0O9xs1y86TJJWmlMN7kqRyFJJNdp4kqSCBV9uTJA0sJWWTV9tTT5zR3w3QoOM+I6nV/J7R4nC/UVMcedJiy0y/aLRI3Gf6QJRzRSNpcfg9o8XhftNiBWWTI0+SJEmS1ARHniSpMIUc3JMkFaSUbHLkSb0iIsZExGENz1eLiIv7s00aWCLiYxHxofrxIRGxWsO0n0bEJv3XusJEL/9Ig5TZpO6YTX2okGyy86TeMgaYH1CZOS0z9+nH9miAyczTM/Oc+ukhwGoN0z6SmXf1S8MklcxsUpfMJi0qO09LiIiYEBF3R8RPImJyRFwVESMiYr2I+GNE3BoR10fERvX860XETRFxc0QcFxHP16+PjIg/R8RtETEpIt5br+J4YL2IuCMiTqzX96/6PX+PiE0b2nJNREyMiOUi4sx6Hbc3LEsDTP15/jsizo6IOyPi4ohYNiJ2qT+7SfVnuXQ9//ERcVc973fq146NiM9FxD7ANsB59f4yot4ntomIj0fECQ3rPSQiflA//mBE/KN+z48jYmh//C4Gvuj1f1KrmE3qCbNpMCknm+w8LVk2AE7LzE2BmcDeVJfmPCIzJwKfA35Yz3sycHJmbgtMa1jGy8D7MnNrYGfguxERwNHAA5m5ZWYe2WG9FwD7AkTEqsBqmXkr8EXgL/U6dgZOjIjlen2r1Vs2BM7IzM2BWcBngLOA/TJzM6pzKD8eESsA7wM2ref9RuNCMvNi4BbgwHp/ealh8sXAXg3P9wMujIiN68dvzMwtgTbgwBZsYxEievdHajGzST1hNg0SpWSTnacly0OZeUf9+FZgArAj8KuIuAP4MbBqPX0H4Ff14/MblhHAtyLiTuBPwOrAyt2s9yLg/fXjfRuWuxtwdL3ua4BlgLUWeavUVx7LzBvqx+cCu1DtU/fWr50N7EQVXi8DP42IvYAXm11BZj4FPBgRb4iIFalC8YZ6XROBm+v9ZRdg3V7YJkn9z2xST5hN6lNebW/J8krD4zaqYJlZHy1p1oHAeGBiZs6JiIepgmWhMnNqRDwdEZtTHaH5aD0pgL0z855FWL/6TzY1U+bciNiOKkT2Bw4H3rYI67mQ6g+ZfwOXZWbWR5DPzsxjFrHNSxyv8aBByGxST5hNg0BJ2eTI05JtFvBQRLwfICpb1NNuoiqdgOpLZp7RwPQ6nHYG1q5fnw2M6mJdFwBHAaMzc1L92pXAEfWXDxGxVU83SC21VkTsUD/+ANXR3QkRsX792kHAtRExkupzvgL4FNDZH0Bd7S+XAnvW67iwfu3PwD4RsRJARKwQEWsv5P2SBjezSYvCbFKfsvOkA4H//v/t3WuopVUdx/HvL9PUtKaQ7EI0pmnFoKPjRBSVhAxGFywKGooyRXOCyCJByKAwSJgXQUjYDYYoIrsIZpSZkI02k+nkqDle6OKrXmjUUDYR6L8Xe03tTjPOM9Ox4/mf7wc2nHmeddba+zDwY6313+tJshP4NbD3i7GXAB9Lchuzcond4/o3gDOT3D5+9z6AqvojcGuSe5Js3sc432EWdNfMXbsCOBy4a3yB94pF/WRabLuA94+ymOcCnwM+wKy05m7gceBqZsFz/Wh3M/DRffS1Bbh675dy529UyJZKEQAABglJREFU1Z+Ae4GXVNVt49q9wOXAj0e/N/LvMh4t1OQ4WK1oZpOmMpuWiybZlKpJu51aYZIcDewZ29LvBjZWlScOrVBJVgPXV9WaJX4rOoBT166r79/080Xtc/VxR95RVWcuaqfSITCbNM9sWj46ZZPfedL+rAOuGmULfwbOX+L3I0mS2SRpSTl50j5V1VbgtAM21IpQVb8HXNlbJjxeXF2ZTZpnNi0vXbLJyZMkNdMknyRJjXTJJg+MkCRJkqQJ3HmSpE6W+MnrkiT9l0bZ5M6Tlr0kj41jRe9J8u1xGtOh9nVWkuvHz29LctkTtF2V5EOHMManknx86vUFbbYkeedBjLV6HLUrSfo/M5+esL35pGXJyZM62FNVa8dRpf8ALp6/OR6weND/16vquqq68gmarAIOOpykJ1+Th2lIy5/5JP1Lj2xy8qRutgInjRWtXUm+AOwAXpxkQ5JtSXaMFcBjAJKck+S+JLcA79jbUZLzklw1fj4+ybVJdo7Xa4ArgRPHquLm0e7SJL9McleST8/19Ykk9yf5CXDKgT5EkgtHPzuTfHfBauXZSbYmeSDJW0b7w5Jsnhv7g//rH1LLU5iVRizmS9KiMJ/MpxWrUzY5eVIbSZ4OvAm4e1w6BfhaVZ0OPMrsKeBnV9UZwO3MnlJ/JPBl4K3A64Dn76f7zwM3V9VpwBnMnnh/GfCbsap4aZINwMuAVwFrgXVJXp9kHbMn2J/OLPzWT/g436uq9WO8XcAFc/dWA28A3szsSehHjvu7q2r96P/CJCdMGEeS9CQzn8wn9eGBEergqCR3jp+3Al8FXgg8VFXbx/VXA68Ebs1sueIIYBvwcuB3VfUgQJKvAxftY4w3Au8DqKrHgN1JnrOgzYbx+tX49zHMwupY4Nqq+tsY47oJn2lNks8wK704Brhh7t41VfU48GCS347PsAE4da7e/Nlj7AcmjKVm3CySnjLMJ/NJQ5dscvKkDvZU1dr5CyOAHp2/BNxYVRsXtFsL1CK9jwCfraovLhjjkkMYYwtwblXtTHIecNbcvYV91Rj7w1U1H2IkWX2Q46oBS+2kpwzzyXzS0CWbLNvTSrEdeG2SkwCSHJ3kZOA+4IQkJ452G/fz+zcBm8bvHpbkWcBfmK3a7XUDcP5crfqLkjwP+Bnw9iRHJTmWWQnGgRwL/CHJ4cB7Ftx7V5Knjff8UuD+Mfam0Z4kJyd55oRxJElLy3ySlhF3nrQiVNXDY4Xsm0meMS5fXlUPJLkI+EGSR4BbgDX76OIjwJeSXAA8Bmyqqm1Jbs3sqNUfjrryVwDbxsriX4H3VtWOJN8C7gQeYla6cSCfBH4x2t/Nf4bg/cDNwPHAxVX19yRfYVZrviOzwR8Gzp3211E3aVMcIfVnPmml6JJNqVqsHWFJ0lI77fR1dcNPtx+44UF4waoj7qiqMxe1U0nSitEpm9x5kqRueizuSZI6aZJNTp4kqZkm+SRJaqRLNnlghCRJkiRN4M6TJDWy1E9elyRpoU7Z5ORJkprpcqKRJKmPLtlk2Z4kSZIkTeDOkyR102NxT5LUSZNscudJkiRJkiZw50mSmmmyuCdJaqRLNjl5kqRmupxoJEnqo0s2WbYnSZIkSRO48yRJraTNcbCSpC76ZJOTJ0lqJPQpjZAk9dApmyzbkyRJkqQJnDxJkiRJ0gROniRJkiRpAr/zJEnNdKkrlyT10SWbnDxJUjNdTjSSJPXRJZss25MkSZKkCdx5kqRO0qc0QpLURKNscvIkSY1kvCRJeqrolE2W7UmSJEnSBO48SVI3XZb3JEl9NMkmd54kSZIkaQJ3niSpmS7HwUqS+uiSTU6eJKmZLicaSZL66JJNlu1JkiRJ0gTuPElSM00W9yRJjXTJJidPktRNl4SSJPXRJJss25MkSZKkCdx5kqRmupxoJEnqo0s2ufMkSZIkSRO48yRJjYQ+x8FKknrolE2pqqV+D5KkRZLkR8Bxi9ztI1V1ziL3KUlaITplk5MnSZIkSZrA7zxJkiRJ0gROniRJkiRpAidPkiRJkjSBkydJkiRJmsDJkyRJkiRN8E+Wi4xSUVuvBwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2, figsize=(12, 6))\n",
    "confusion_dict = {ax1: 'true', ax2: 'pred'}\n",
    "\n",
    "for k, v in confusion_dict.items():\n",
    "    plot_confusion_matrix(logreg, X_test, y_test,\n",
    "                          cmap=plt.cm.Blues,\n",
    "                          display_labels=['negative', 'positive'],\n",
    "                          normalize=v, ax=k)\n",
    "    k.set_title('Confusion matrix with \\'%s\\' normalization' % v)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model is very good at finding positive reviews - from all positive reviews, over 99% were assigned correct label. When it comes to negative reviews, the model assign correct label only in 57% of cases.\n",
    "\n",
    "When we look at the predicted labels, from all positive labels, 96% are true positives and,3,8% are false positives. Among labels predicted as negative, 89% are true negatives and 11% are false negatives."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 66,  69,  83, 103, 119, 150, 165, 174, 178, 194, 215, 306, 314,\n",
       "       321, 328, 337, 351, 362, 387, 396, 405, 412, 492, 549, 568, 575,\n",
       "       629, 641, 661, 700, 782, 795, 803, 813, 854, 855, 858, 879, 886,\n",
       "       922], dtype=int64)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_diff = y_test - y_pred\n",
    "np.where(y_diff != 0)[0][:40]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Son really loves it.\n",
      "\n",
      "real label: 0 \n",
      "predicted label: 1\n"
     ]
    }
   ],
   "source": [
    "i = 69\n",
    "print(df['review'][i])\n",
    "print('\\nreal label:', y_test[i], '\\npredicted label:', y_pred[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we can see that the real label was incorrect, since the review is obviously positive. In this case, the model was right when assigning positive label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cheap plastic, broken when arrived in manilla envelope\n",
      "\n",
      "real label: 0 \n",
      "predicted label: 1\n"
     ]
    }
   ],
   "source": [
    "i = 119\n",
    "print(df['review'][i])\n",
    "print('\\nreal label:', y_test[i], '\\npredicted label:', y_pred[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This review is clearly negative and the model made a mistake. It's interesting why this review was marked as positive, since it contains words like 'cheap' and 'broken'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looks good nephew loves it\n",
      "\n",
      "real label: 1 \n",
      "predicted label: 0\n"
     ]
    }
   ],
   "source": [
    "i = 568\n",
    "print(df['review'][i])\n",
    "print('\\nreal label:', y_test[i], '\\npredicted label:', y_pred[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's another interesting example - the review is clearly positive but the model assigned it as negative, despite the fact that the review contains such words as 'good' and 'loves'."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Under- and over-sampling\n",
    "\n",
    "Since the dataset is highly imbalanced, one of the possible solutions to this problem is to sample observations in a way that the number of items in the classes will be equal. Two approaches will be tested:\n",
    "- under-sampling - removing random observations from the majority class\n",
    "- over-sampling (SMOTE) - adding observations to the minority class.\n",
    "- mix of SMOTE and under-sampling\n",
    "\n",
    "In order to optimize computations, the SGD model was used instead of Logistic Regression.\n",
    "\n",
    "According to the results of this experiment, both under-sampling and SMOTE are not effective methods to improve performance of the model - both accuracy and f1-score decreased. Poor results were achieved also with mix of SMOTE and under-sampling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_acc</th>\n",
       "      <th>f1</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>BOW</th>\n",
       "      <td>0.90551</td>\n",
       "      <td>0.946164</td>\n",
       "      <td>0.988042</td>\n",
       "      <td>0.907696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TFIDF</th>\n",
       "      <td>0.89178</td>\n",
       "      <td>0.937645</td>\n",
       "      <td>0.991350</td>\n",
       "      <td>0.889462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVD</th>\n",
       "      <td>0.57232</td>\n",
       "      <td>0.683681</td>\n",
       "      <td>0.969810</td>\n",
       "      <td>0.552678</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NMF</th>\n",
       "      <td>0.57762</td>\n",
       "      <td>0.693126</td>\n",
       "      <td>0.975253</td>\n",
       "      <td>0.554657</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Word2Vec</th>\n",
       "      <td>0.86554</td>\n",
       "      <td>0.921010</td>\n",
       "      <td>0.989895</td>\n",
       "      <td>0.861915</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          test_acc        f1  precision    recall\n",
       "BOW        0.90551  0.946164   0.988042  0.907696\n",
       "TFIDF      0.89178  0.937645   0.991350  0.889462\n",
       "SVD        0.57232  0.683681   0.969810  0.552678\n",
       "NMF        0.57762  0.693126   0.975253  0.554657\n",
       "Word2Vec   0.86554  0.921010   0.989895  0.861915"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "under_sampler_pipeline = make_pipeline(RandomUnderSampler(random_state=9),\n",
    "                              SGDClassifier(random_state=9, n_jobs=-1))\n",
    "\n",
    "under_sampler_cv = f.model_cv(under_sampler_pipeline, embeddings, y)\n",
    "f.df_model_cv(under_sampler_cv, embeddings_names, results_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_acc</th>\n",
       "      <th>f1</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>BOW</th>\n",
       "      <td>0.90060</td>\n",
       "      <td>0.943699</td>\n",
       "      <td>0.979011</td>\n",
       "      <td>0.910866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TFIDF</th>\n",
       "      <td>0.90297</td>\n",
       "      <td>0.944651</td>\n",
       "      <td>0.987726</td>\n",
       "      <td>0.905181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVD</th>\n",
       "      <td>0.56958</td>\n",
       "      <td>0.695626</td>\n",
       "      <td>0.983651</td>\n",
       "      <td>0.538478</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NMF</th>\n",
       "      <td>0.56991</td>\n",
       "      <td>0.693327</td>\n",
       "      <td>0.981305</td>\n",
       "      <td>0.540774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Word2Vec</th>\n",
       "      <td>0.89558</td>\n",
       "      <td>0.939940</td>\n",
       "      <td>0.987893</td>\n",
       "      <td>0.896906</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          test_acc        f1  precision    recall\n",
       "BOW        0.90060  0.943699   0.979011  0.910866\n",
       "TFIDF      0.90297  0.944651   0.987726  0.905181\n",
       "SVD        0.56958  0.695626   0.983651  0.538478\n",
       "NMF        0.56991  0.693327   0.981305  0.540774\n",
       "Word2Vec   0.89558  0.939940   0.987893  0.896906"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "smote_pipeline = make_pipeline(SMOTE(random_state=9, n_jobs=-1),\n",
    "                              SGDClassifier(random_state=9, n_jobs=-1))\n",
    "\n",
    "smote_cv = f.model_cv(smote_pipeline, embeddings_positive, y)\n",
    "f.df_model_cv(smote_cv, embeddings_names, results_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_acc</th>\n",
       "      <th>f1</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>BOW</th>\n",
       "      <td>0.90861</td>\n",
       "      <td>0.948303</td>\n",
       "      <td>0.982630</td>\n",
       "      <td>0.916299</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TFIDF</th>\n",
       "      <td>0.91473</td>\n",
       "      <td>0.951791</td>\n",
       "      <td>0.985668</td>\n",
       "      <td>0.920168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVD</th>\n",
       "      <td>0.68893</td>\n",
       "      <td>0.800413</td>\n",
       "      <td>0.967076</td>\n",
       "      <td>0.683428</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NMF</th>\n",
       "      <td>0.70575</td>\n",
       "      <td>0.814305</td>\n",
       "      <td>0.962443</td>\n",
       "      <td>0.705969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Word2Vec</th>\n",
       "      <td>0.90032</td>\n",
       "      <td>0.942873</td>\n",
       "      <td>0.986997</td>\n",
       "      <td>0.903017</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          test_acc        f1  precision    recall\n",
       "BOW        0.90861  0.948303   0.982630  0.916299\n",
       "TFIDF      0.91473  0.951791   0.985668  0.920168\n",
       "SVD        0.68893  0.800413   0.967076  0.683428\n",
       "NMF        0.70575  0.814305   0.962443  0.705969\n",
       "Word2Vec   0.90032  0.942873   0.986997  0.903017"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mixed_pipeline = make_pipeline(SMOTE(random_state=9, sampling_strategy=0.33, n_jobs=-1),\n",
    "                 RandomUnderSampler(random_state=9, sampling_strategy=0.66),\n",
    "                 SGDClassifier(random_state=9, n_jobs=-1))\n",
    "\n",
    "mixed_cv = f.model_cv(mixed_pipeline, embeddings_positive, y)\n",
    "f.df_model_cv(mixed_cv, embeddings_names, results_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "fastai-nlp",
   "language": "python",
   "name": "fastai-nlp"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}