-
Notifications
You must be signed in to change notification settings - Fork 43
/
Copy path2_Simple_NN.ipnyb
1 lines (1 loc) · 83.2 KB
/
2_Simple_NN.ipnyb
1
{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"2_Simple_NN.ipnyb","provenance":[],"collapsed_sections":[],"toc_visible":true,"mount_file_id":"1qsgy6H_ouWp8rGfE54G0ONfxcv1m8h-S","authorship_tag":"ABX9TyNhKbfnRwpVKWMdC+5yvPyY"},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"5R4mLaolzHm_"},"source":["# Imports"]},{"cell_type":"code","metadata":{"id":"qMiy8hf5y6Oh","executionInfo":{"status":"ok","timestamp":1602769181210,"user_tz":-180,"elapsed":3001,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}}},"source":["import pandas as pd\n","import numpy as np\n","\n","from sklearn.model_selection import train_test_split\n","from sklearn.preprocessing import LabelEncoder\n","from sklearn.metrics import confusion_matrix, classification_report\n","\n","import matplotlib.pyplot as plt\n","\n","from keras.preprocessing.text import Tokenizer\n","from keras.preprocessing.sequence import pad_sequences\n","from keras.utils import np_utils\n","from keras.models import Sequential\n","from keras import layers\n","from keras.wrappers.scikit_learn import KerasClassifier"],"execution_count":1,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"wsV9LpsB1kDd"},"source":["# Data"]},{"cell_type":"markdown","metadata":{"id":"KTtAL3_U6j0S"},"source":["For this example you need to download twitter data from Kaggle: https://www.kaggle.com/kazanova/sentiment140"]},{"cell_type":"markdown","metadata":{"id":"wjfc1e9d80OA"},"source":["Context \\\\\n","\\\n","This is the sentiment140 dataset. It contains 1,600,000 tweets extracted using the twitter api . The tweets have been annotated (0 = negative, 4 = positive) and they can be used to detect sentiment. \\\\\n","\\\n","Content \\\\\n","\\\n","It contains the following 6 fields:\n","* target: the polarity of the tweet (0 = negative, 4 = positive)\n","\n","* ids: The id of the tweet ( 2087)\n","\n","* date: the date of the tweet (Sat May 16 23:58:44 UTC 2009)\n","\n","* flag: The query (lyx). If there is no query, then this value is NO_QUERY.\n","\n","* user: the user that tweeted (robotickilldozr)\n","\n","* text: the text of the tweet (Lyx is cool)\n","\n","Acknowledgements \\\\\n","The official link regarding the dataset with resources about how it was generated is [here](http://%20http//help.sentiment140.com/for-students/) \\\\\n","The official paper detailing the approach is [here](http://bhttp//cs.stanford.edu/people/alecmgo/papers/TwitterDistantSupervision09.pdf)\n","\n","Citation: Go, A., Bhayani, R. and Huang, L., 2009. Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford, 1(2009), p.12."]},{"cell_type":"code","metadata":{"id":"FzXp3Gua1lxV","executionInfo":{"status":"ok","timestamp":1602769187539,"user_tz":-180,"elapsed":9320,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}}},"source":["df = pd.read_csv('/content/drive/My Drive/YouTube/botters_2020-10/twitter_sentiment_course/twitter_data/training.1600000.processed.noemoticon.csv',\n"," encoding='ISO-8859-1', \n"," names=[\n"," 'target',\n"," 'id',\n"," 'date',\n"," 'flag',\n"," 'user',\n"," 'text'\n"," ])"],"execution_count":2,"outputs":[]},{"cell_type":"code","metadata":{"id":"XhsfgM9o12-L","executionInfo":{"status":"ok","timestamp":1602769187541,"user_tz":-180,"elapsed":9309,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"5e8ac955-a506-4842-e2d2-de1b3c5a0566","colab":{"base_uri":"https://localhost:8080/","height":203}},"source":["df.head()"],"execution_count":3,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n"," .dataframe tbody tr th:only-of-type {\n"," vertical-align: middle;\n"," }\n","\n"," .dataframe tbody tr th {\n"," vertical-align: top;\n"," }\n","\n"," .dataframe thead th {\n"," text-align: right;\n"," }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n"," <thead>\n"," <tr style=\"text-align: right;\">\n"," <th></th>\n"," <th>target</th>\n"," <th>id</th>\n"," <th>date</th>\n"," <th>flag</th>\n"," <th>user</th>\n"," <th>text</th>\n"," </tr>\n"," </thead>\n"," <tbody>\n"," <tr>\n"," <th>0</th>\n"," <td>0</td>\n"," <td>1467810369</td>\n"," <td>Mon Apr 06 22:19:45 PDT 2009</td>\n"," <td>NO_QUERY</td>\n"," <td>_TheSpecialOne_</td>\n"," <td>@switchfoot http://twitpic.com/2y1zl - Awww, t...</td>\n"," </tr>\n"," <tr>\n"," <th>1</th>\n"," <td>0</td>\n"," <td>1467810672</td>\n"," <td>Mon Apr 06 22:19:49 PDT 2009</td>\n"," <td>NO_QUERY</td>\n"," <td>scotthamilton</td>\n"," <td>is upset that he can't update his Facebook by ...</td>\n"," </tr>\n"," <tr>\n"," <th>2</th>\n"," <td>0</td>\n"," <td>1467810917</td>\n"," <td>Mon Apr 06 22:19:53 PDT 2009</td>\n"," <td>NO_QUERY</td>\n"," <td>mattycus</td>\n"," <td>@Kenichan I dived many times for the ball. Man...</td>\n"," </tr>\n"," <tr>\n"," <th>3</th>\n"," <td>0</td>\n"," <td>1467811184</td>\n"," <td>Mon Apr 06 22:19:57 PDT 2009</td>\n"," <td>NO_QUERY</td>\n"," <td>ElleCTF</td>\n"," <td>my whole body feels itchy and like its on fire</td>\n"," </tr>\n"," <tr>\n"," <th>4</th>\n"," <td>0</td>\n"," <td>1467811193</td>\n"," <td>Mon Apr 06 22:19:57 PDT 2009</td>\n"," <td>NO_QUERY</td>\n"," <td>Karoli</td>\n"," <td>@nationwideclass no, it's not behaving at all....</td>\n"," </tr>\n"," </tbody>\n","</table>\n","</div>"],"text/plain":[" target ... text\n","0 0 ... @switchfoot http://twitpic.com/2y1zl - Awww, t...\n","1 0 ... is upset that he can't update his Facebook by ...\n","2 0 ... @Kenichan I dived many times for the ball. Man...\n","3 0 ... my whole body feels itchy and like its on fire \n","4 0 ... @nationwideclass no, it's not behaving at all....\n","\n","[5 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":3}]},{"cell_type":"markdown","metadata":{"id":"gnHNDF2H8qo5"},"source":["We have two classes in the dataset"]},{"cell_type":"code","metadata":{"id":"GrL29pcJ6IWE","executionInfo":{"status":"ok","timestamp":1602769187542,"user_tz":-180,"elapsed":9299,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"a69afa13-0769-4ead-e98a-4489ef51063c","colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["df.target.unique()"],"execution_count":4,"outputs":[{"output_type":"execute_result","data":{"text/plain":["array([0, 4])"]},"metadata":{"tags":[]},"execution_count":4}]},{"cell_type":"markdown","metadata":{"id":"ZMQPnCUZdLw_"},"source":["Let's check how equally distributed those classes are."]},{"cell_type":"code","metadata":{"id":"xqUoAbO9awuD","executionInfo":{"status":"ok","timestamp":1602769188569,"user_tz":-180,"elapsed":10313,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"f8b07909-a705-45bd-fb00-692322b27ba6","colab":{"base_uri":"https://localhost:8080/","height":265}},"source":["classes = df.target.unique()\n","counts = []\n","\n","for i in classes:\n"," count = len(df[df.target==i])\n"," counts.append(count)\n","\n","plt.bar(['negative', 'positive'], counts)\n","plt.show()"],"execution_count":5,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAYkAAAD4CAYAAAAZ1BptAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAXaElEQVR4nO3df7Bc5X3f8fcnyNgYByTgRoMlHFFbjYuZGsMdkOtMprEaIUgmYhpMoXakUI1VDziNQzOx3OkMiQkZPMmUhImtRDUqInWMZRoPqissq7JpG88IuNiEnybcgImk4ceNJCAOsR2cb//YR/FyvefeFYa9svR+zezsc77nOec5q1nt554fuydVhSRJg/zIXG+AJOnwZUhIkjoZEpKkToaEJKmTISFJ6jRvrjfglXbKKafUkiVL5nozJOmHyj333PPXVTU2vX7EhcSSJUuYmJiY682QpB8qSZ4YVPdwkySpkyEhSepkSEiSOhkSkqROhoQkqZMhIUnqNFRIJPnVJA8meSDJp5O8LsnpSe5MMpnkM0mObX1f26Yn2/wlfev5SKs/kuT8vvrKVptMsr6vPnAMSdJozBoSSRYB/wEYr6ozgWOAS4GPAddX1VuAA8Datsha4ECrX9/6keSMttzbgJXAJ5Ick+QY4OPABcAZwGWtLzOMIUkagWEPN80DjksyD3g98CTwbuDWNn8zcFFrr2rTtPnLk6TVb6mqb1fV48AkcG57TFbVY1X1HeAWYFVbpmsMSdIIzPqN66ram+R3gb8C/g74InAP8GxVvdi67QEWtfYiYHdb9sUkzwEnt/quvlX3L7N7Wv28tkzXGC+RZB2wDuBNb3rTbC+p05L1/+tlL6sj2zeu+9m53gTA96i6vVrv0WEONy2gtxdwOvBG4Hh6h4sOG1W1sarGq2p8bOz7fnpEkvQyDXO46V8Bj1fVVFX9PfCnwLuA+e3wE8BiYG9r7wVOA2jzTwT29denLdNV3zfDGJKkERgmJP4KWJbk9e08wXLgIeDLwMWtzxrgttbe2qZp879UvRtpbwUubVc/nQ4sBe4C7gaWtiuZjqV3cntrW6ZrDEnSCMwaElV1J72Tx18F7m/LbAQ+DFyVZJLe+YMb2yI3Aie3+lXA+raeB4Et9ALmC8CVVfXdds7hg8B24GFgS+vLDGNIkkZgqJ8Kr6qrgaunlR+jd2XS9L7fAt7TsZ5rgWsH1LcB2wbUB44hSRoNv3EtSepkSEiSOhkSkqROhoQkqZMhIUnqZEhIkjoZEpKkToaEJKmTISFJ6mRISJI6GRKSpE6GhCSpkyEhSepkSEiSOhkSkqROhoQkqdOsIZHkJ5Lc2/d4PsmHkpyUZEeSR9vzgtY/SW5IMpnkviRn961rTev/aJI1ffVzktzflrmh3SaVrjEkSaMxzO1LH6mqs6rqLOAc4AXgc/RuS7qzqpYCO9s0wAX07l+9FFgHbIDeBz69u9udR+9uc1f3fehvAN7ft9zKVu8aQ5I0Aod6uGk58JdV9QSwCtjc6puBi1p7FXBz9ewC5ic5FTgf2FFV+6vqALADWNnmnVBVu6qqgJunrWvQGJKkETjUkLgU+HRrL6yqJ1v7KWBhay8Cdvcts6fVZqrvGVCfaQxJ0ggMHRJJjgV+Hvjs9HltD6Bewe36PjONkWRdkokkE1NTU6/mZkjSUeVQ9iQuAL5aVU+36afboSLa8zOtvhc4rW+5xa02U33xgPpMY7xEVW2sqvGqGh8bGzuElyRJmsmhhMRlfO9QE8BW4OAVSmuA2/rqq9tVTsuA59oho+3AiiQL2gnrFcD2Nu/5JMvaVU2rp61r0BiSpBGYN0ynJMcDPwP8+77ydcCWJGuBJ4BLWn0bcCEwSe9KqMsBqmp/kmuAu1u/j1bV/ta+ArgJOA64vT1mGkOSNAJDhURV/S1w8rTaPnpXO03vW8CVHevZBGwaUJ8AzhxQHziGJGk0/Ma1JKmTISFJ6mRISJI6GRKSpE6GhCSpkyEhSepkSEiSOhkSkqROhoQkqZMhIUnqZEhIkjoZEpKkToaEJKmTISFJ6mRISJI6GRKSpE6GhCSp01AhkWR+kluTfD3Jw0nemeSkJDuSPNqeF7S+SXJDkskk9yU5u289a1r/R5Os6aufk+T+tswN7V7XdI0hSRqNYfckfh/4QlW9FXg78DCwHthZVUuBnW0a4AJgaXusAzZA7wMfuBo4DzgXuLrvQ38D8P6+5Va2etcYkqQRmDUkkpwI/BRwI0BVfaeqngVWAZtbt83ARa29Cri5enYB85OcCpwP7Kiq/VV1ANgBrGzzTqiqXe3+2DdPW9egMSRJIzDMnsTpwBTw35J8LcknkxwPLKyqJ1ufp4CFrb0I2N23/J5Wm6m+Z0CdGcZ4iSTrkkwkmZiamhriJUmShjFMSMwDzgY2VNU7gL9l2mGftgdQr/zmDTdGVW2sqvGqGh8bG3s1N0OSjirDhMQeYE9V3dmmb6UXGk+3Q0W052fa/L3AaX3LL261meqLB9SZYQxJ0gjMGhJV9RSwO8lPtNJy4CFgK3DwCqU1wG2tvRVY3a5yWgY81w4ZbQdWJFnQTlivALa3ec8nWdaualo9bV2DxpAkjcC8Ifv9MvCpJMcCjwGX0wuYLUnWAk8Al7S+24ALgUnghdaXqtqf5Brg7tbvo1W1v7WvAG4CjgNubw+A6zrGkCSNwFAhUVX3AuMDZi0f0LeAKzvWswnYNKA+AZw5oL5v0BiSpNHwG9eSpE6GhCSpkyEhSepkSEiSOhkSkqROhoQkqZMhIUnqZEhIkjoZEpKkToaEJKmTISFJ6mRISJI6GRKSpE6GhCSpkyEhSepkSEiSOhkSkqROQ4VEkm8kuT/JvUkmWu2kJDuSPNqeF7R6ktyQZDLJfUnO7lvPmtb/0SRr+urntPVPtmUz0xiSpNE4lD2Jn66qs6rq4G1M1wM7q2opsLNNA1wALG2PdcAG6H3gA1cD5wHnAlf3fehvAN7ft9zKWcaQJI3AD3K4aRWwubU3Axf11W+unl3A/CSnAucDO6pqf1UdAHYAK9u8E6pqV7s/9s3T1jVoDEnSCAwbEgV8Mck9Sda12sKqerK1nwIWtvYiYHffsntabab6ngH1mcZ4iSTrkkwkmZiamhryJUmSZjNvyH4/WVV7k/wYsCPJ1/tnVlUlqVd+84Ybo6o2AhsBxsfHX9XtkKSjyVB7ElW1tz0/A3yO3jmFp9uhItrzM637XuC0vsUXt9pM9cUD6swwhiRpBGYNiSTHJ/nRg21gBfAAsBU4eIXSGuC21t4KrG5XOS0DnmuHjLYDK5IsaCesVwDb27znkyxrVzWtnrauQWNIkkZgmMNNC4HPtatS5wF/UlVfSHI3sCXJWuAJ4JLWfxtwITAJvABcDlBV+5NcA9zd+n20qva39hXATcBxwO3tAXBdxxiSpBGYNSSq6jHg7QPq+4DlA+oFXNmxrk3ApgH1CeDMYceQJI2G37iWJHUyJCRJnQwJSVInQ0KS1MmQkCR1MiQkSZ0MCUlSJ0NCktTJkJAkdTIkJEmdDAlJUidDQpLUyZCQJHUyJCRJnQwJSVInQ0KS1MmQkCR1GjokkhyT5GtJPt+mT09yZ5LJJJ9Jcmyrv7ZNT7b5S/rW8ZFWfyTJ+X31la02mWR9X33gGJKk0TiUPYlfAR7um/4YcH1VvQU4AKxt9bXAgVa/vvUjyRnApcDbgJXAJ1rwHAN8HLgAOAO4rPWdaQxJ0ggMFRJJFgM/C3yyTQd4N3Br67IZuKi1V7Vp2vzlrf8q4Jaq+nZVPQ5MAue2x2RVPVZV3wFuAVbNMoYkaQSG3ZP4PeDXgX9o0ycDz1bVi216D7CotRcBuwHa/Oda/3+sT1umqz7TGC+RZF2SiSQTU1NTQ74kSdJsZg2JJD8HPFNV94xge16WqtpYVeNVNT42NjbXmyNJR4x5Q/R5F/DzSS4EXgecAPw+MD/JvPaX/mJgb+u/FzgN2JNkHnAisK+vflD/MoPq+2YYQ5I0ArPuSVTVR6pqcVUtoXfi+UtV9V7gy8DFrdsa4LbW3tqmafO/VFXV6pe2q59OB5YCdwF3A0vblUzHtjG2tmW6xpAkjcAP8j2JDwNXJZmkd/7gxla/ETi51a8C1gNU1YPAFuAh4AvAlVX13baX8EFgO72rp7a0vjONIUkagWEON/2jqroDuKO1H6N3ZdL0Pt8C3tOx/LXAtQPq24BtA+oDx5AkjYbfuJYkdTIkJEmdDAlJUidDQpLUyZCQJHUyJCRJnQwJSVInQ0KS1MmQkCR1MiQkSZ0MCUlSJ0NCktTJkJAkdTIkJEmdDAlJUidDQpLUyZCQJHWaNSSSvC7JXUn+PMmDSX6z1U9PcmeSySSfafenpt3D+jOtfmeSJX3r+kirP5Lk/L76ylabTLK+rz5wDEnSaAyzJ/Ft4N1V9XbgLGBlkmXAx4Drq+otwAFgbeu/FjjQ6te3fiQ5A7gUeBuwEvhEkmOSHAN8HLgAOAO4rPVlhjEkSSMwa0hUzzfb5Gvao4B3A7e2+mbgotZe1aZp85cnSavfUlXfrqrHgUl6968+F5isqseq6jvALcCqtkzXGJKkERjqnET7i/9e4BlgB/CXwLNV9WLrsgdY1NqLgN0Abf5zwMn99WnLdNVPnmGM6du3LslEkompqalhXpIkaQhDhURVfbeqzgIW0/vL/62v6lYdoqraWFXjVTU+NjY215sjSUeMQ7q6qaqeBb4MvBOYn2Rem7UY2Nvae4HTANr8E4F9/fVpy3TV980whiRpBIa5umksyfzWPg74GeBhemFxceu2Brittbe2adr8L1VVtfql7eqn04GlwF3A3cDSdiXTsfRObm9ty3SNIUkagXmzd+FUYHO7CulHgC1V9fkkDwG3JPkt4GvAja3/jcAfJ5kE9tP70KeqHkyyBXgIeBG4sqq+C5Dkg8B24BhgU1U92Nb14Y4xJEkjMGtIVNV9wDsG1B+jd35iev1bwHs61nUtcO2A+jZg27BjSJJGw29cS5I6GRKSpE6GhCSpkyEhSepkSEiSOhkSkqROhoQkqZMhIUnqZEhIkjoZEpKkToaEJKmTISFJ6mRISJI6GRKSpE6GhCSpkyEhSepkSEiSOg1zj+vTknw5yUNJHkzyK61+UpIdSR5tzwtaPUluSDKZ5L4kZ/eta03r/2iSNX31c5Lc35a5IUlmGkOSNBrD7Em8CPzHqjoDWAZcmeQMYD2ws6qWAjvbNMAFwNL2WAdsgN4HPnA1cB69W5Je3fehvwF4f99yK1u9awxJ0gjMGhJV9WRVfbW1/wZ4GFgErAI2t26bgYtaexVwc/XsAuYnORU4H9hRVfur6gCwA1jZ5p1QVbuqqoCbp61r0BiSpBE4pHMSSZYA7wDuBBZW1ZNt1lPAwtZeBOzuW2xPq81U3zOgzgxjTN+udUkmkkxMTU0dykuSJM1g6JBI8gbgfwAfqqrn++e1PYB6hbftJWYao6o2VtV4VY2PjY29mpshSUeVoUIiyWvoBcSnqupPW/npdqiI9vxMq+8FTutbfHGrzVRfPKA+0xiSpBEY5uqmADcCD1fVf+mbtRU4eIXSGuC2vvrqdpXTMuC5dshoO7AiyYJ2wnoFsL3Nez7JsjbW6mnrGjSGJGkE5g3R513ALwL3J7m31f4TcB2wJcla4AngkjZvG3AhMAm8AFwOUFX7k1wD3N36fbSq9rf2FcBNwHHA7e3BDGNIkkZg1pCoqj8D0jF7+YD+BVzZsa5NwKYB9QngzAH1fYPGkCSNht+4liR1MiQkSZ0MCUlSJ0NCktTJkJAkdTIkJEmdDAlJUidDQpLUyZCQJHUyJCRJnQwJSVInQ0KS1MmQkCR1MiQkSZ0MCUlSJ0NCktTJkJAkdRrmHtebkjyT5IG+2klJdiR5tD0vaPUkuSHJZJL7kpzdt8ya1v/RJGv66uckub8tc0O7z3XnGJKk0RlmT+ImYOW02npgZ1UtBXa2aYALgKXtsQ7YAL0PfOBq4DzgXODqvg/9DcD7+5ZbOcsYkqQRmTUkqur/AvunlVcBm1t7M3BRX/3m6tkFzE9yKnA+sKOq9lfVAWAHsLLNO6GqdrV7Y988bV2DxpAkjcjLPSexsKqebO2ngIWtvQjY3ddvT6vNVN8zoD7TGN8nybokE0kmpqamXsbLkSQN8gOfuG57APUKbMvLHqOqNlbVeFWNj42NvZqbIklHlZcbEk+3Q0W052dafS9wWl+/xa02U33xgPpMY0iSRuTlhsRW4OAVSmuA2/rqq9tVTsuA59oho+3AiiQL2gnrFcD2Nu/5JMvaVU2rp61r0BiSpBGZN1uHJJ8G/iVwSpI99K5Sug7YkmQt8ARwSeu+DbgQmAReAC4HqKr9Sa4B7m79PlpVB0+GX0HvCqrjgNvbgxnGkCSNyKwhUVWXdcxaPqBvAVd2rGcTsGlAfQI4c0B936AxJEmj4zeuJUmdDAlJUidDQpLUyZCQJHUyJCRJnQwJSVInQ0KS1MmQkCR1MiQkSZ0MCUlSJ0NCktTJkJAkdTIkJEmdDAlJUidDQpLUyZCQJHUyJCRJnQ77kEiyMskjSSaTrJ/r7ZGko8lhHRJJjgE+DlwAnAFcluSMud0qSTp6HNYhAZwLTFbVY1X1HeAWYNUcb5MkHTXmzfUGzGIRsLtveg9w3vROSdYB69rkN5M8MoJtOxqcAvz1XG/E4SAfm+stUAffo80r8B798UHFwz0khlJVG4GNc70dR5okE1U1PtfbIXXxPfrqO9wPN+0FTuubXtxqkqQRONxD4m5gaZLTkxwLXApsneNtkqSjxmF9uKmqXkzyQWA7cAywqaoenOPNOpp4CE+HO9+jr7JU1VxvgyTpMHW4H26SJM0hQ0KS1MmQ0FCSzE9yRd/0G5PcOpfbpKNXkg8kWd3av5TkjX3zPukvM7xyPCehoSRZAny+qs6c402RXiLJHcCvVdXEXG/Lkcg9iSNEkiVJHk7yX5M8mOSLSY5L8uYkX0hyT5L/l+Strf+bk+xKcn+S30ryzVZ/Q5KdSb7a5h38GZTrgDcnuTfJ77TxHmjL7Erytr5tuSPJeJLjk2xKcleSr/WtS0ex9t75epJPtffsrUlen2R5e5/c3943r239r0vyUJL7kvxuq/1Gkl9LcjEwDnyqvTeP63v/fSDJ7/SN+0tJ/qC139fel/cm+aP2O3EapKp8HAEPYAnwInBWm94CvA/YCSxttfOAL7X254HLWvsDwDdbex5wQmufAkwCaet/YNp4D7T2rwK/2dqnAo+09m8D72vt+cBfAMfP9b+Vj8PivVrAu9r0JuA/0/sJnn/aajcDHwJOBh7he0c95rfn36C39wBwBzDet/476AXHGL3ffjtYvx34SeCfAf8TeE2rfwJYPdf/Lofrwz2JI8vjVXVva99D7z/jvwA+m+Re4I/ofYgDvBP4bGv/Sd86Avx2kvuA/03v97MWzjLuFuDi1r4EOHiuYgWwvo19B/A64E2H/Kp0JNpdVV9p7f8OLKf3/v2LVtsM/BTwHPAt4MYk/xp4YdgBqmoKeCzJsiQnA28FvtLGOge4u703lwP/5BV4TUekw/rLdDpk3+5rf5feh/uzVXXWIazjvfT+Ajunqv4+yTfofbh3qqq9SfYl+efAv6G3ZwK9wPmFqvIHFzXd9JOhz9Lba3hpp94Xas+l90F+MfBB4N2HMM4t9P5w+TrwuaqqJAE2V9VHXtaWH2XckziyPQ88nuQ9AOl5e5u3C/iF1r60b5kTgWdaQPw03/tlyL8BfnSGsT4D/DpwYlXd12rbgV9u/ylJ8o4f9AXpiPGmJO9s7X8LTABLkryl1X4R+D9J3kDvPbWN3mHNt3//qmZ8b36O3u0FLqMXGNA7BHtxkh8DSHJSkoG/gCpD4mjwXmBtkj8HHuR79+P4EHBVO6z0Fnq79QCfAsaT3A+spvcXGFW1D/hKkgf6Twb2uZVe2Gzpq10DvAa4L8mDbVqC3nmGK5M8DCwArgcup3do9H7gH4A/pPfh//n2Pv0z4KoB67oJ+MODJ677Z1TVAeBh4Mer6q5We4jeOZAvtvXu4HuHYTWNl8AepZK8Hvi7tvt9Kb2T2F59pFedl1P/cPGcxNHrHOAP2qGgZ4F/N8fbI+kw5J6EJKmT5yQkSZ0MCUlSJ0NCktTJkJAkdTIkJEmd/j8NRZZn3nav5gAAAABJRU5ErkJggg==\n","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{"tags":[],"needs_background":"light"}}]},{"cell_type":"markdown","metadata":{"id":"_Z8YvAWgdQsK"},"source":["Even class distribution helps us a lot in text classifiction. Imagine a situation where 95% of data is in one class and the rest 5% is split among other 5 classes. If we wouldn't do anything about it model would just learn to guess the 95% class all the time and would be correct 95% of the time on the data we would use."]},{"cell_type":"markdown","metadata":{"id":"_E8moCv-mbSe"},"source":["To start modelling we still need to do a couple of things:\n","\n","1. Split data into train and test datasets\n","2. Tokenize it\n","3. Pad the text\n","4. Encode the labels\n","\n"]},{"cell_type":"markdown","metadata":{"id":"r7t3WHSIoZiS"},"source":["## Train/test split"]},{"cell_type":"markdown","metadata":{"id":"x9mheNP29Yjt"},"source":["We don't really need all 1.6 Million tweets for training so we can take a sample of 10% and then split taht into 80% for training and 20% for testing."]},{"cell_type":"code","metadata":{"id":"_ppqSW1N7mIt","executionInfo":{"status":"ok","timestamp":1602769188570,"user_tz":-180,"elapsed":10312,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}}},"source":["sample_size = int(len(df)*0.1)\n","sampleDf = df.sample(sample_size, random_state=23)\n","x = sampleDf.text.values\n","y = sampleDf.target.values\n","x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.20, random_state=32)"],"execution_count":6,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"5wopVEhDOY1F"},"source":["## Tokenization\n","What we need to do now is to tokenize text (essentialy turning words into tokens). The difference from vectorization that we used in Linear Regression notebook is that we do not count each token occurence in the text. Instead we just turn words into tokens and that's about it. Comparing this to Logistic Regression (LR) example you'll see that in this method we keep the order of the words intact, while, in the LR example we loose that information."]},{"cell_type":"code","metadata":{"id":"A1QGfFgQ-Rpy","executionInfo":{"status":"ok","timestamp":1602769193041,"user_tz":-180,"elapsed":14781,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}}},"source":["tokenizer = Tokenizer()\n","tokenizer.fit_on_texts(x_train)\n","\n","X_train = tokenizer.texts_to_sequences(x_train)\n","X_test = tokenizer.texts_to_sequences(x_test)\n","\n","vocab_size = len(tokenizer.word_index) + 1 # Adding 1 because of reserved 0 index"],"execution_count":7,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"LtJV4JBUOsyc"},"source":["Our tokenized dataset will essentially consist of index of each word that is used in training dataset. We can check how it looks by simply printing the first tweet both as it was and after tokenization."]},{"cell_type":"code","metadata":{"id":"iUlYC1mkRuMt","executionInfo":{"status":"ok","timestamp":1602769193042,"user_tz":-180,"elapsed":14770,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"4c2eee7d-cb8f-474a-f257-1ccabaf7fbc3","colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["print(x_train[0])\n","print(X_train[0])"],"execution_count":8,"outputs":[{"output_type":"stream","text":["@teefy1983 aww my poor mexican!!! He no feel good at all \n","[33672, 361, 5, 351, 3268, 113, 37, 110, 28, 23, 31]\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"1_4Ocpt1ohGP"},"source":["## Padding\n","Before modelling we want to make sure our inputs are the same size, since we are using neural network with a fixed input size. \\\\\n","First we figure out the max len."]},{"cell_type":"code","metadata":{"id":"LyDETieLoucQ","executionInfo":{"status":"ok","timestamp":1602769193043,"user_tz":-180,"elapsed":14762,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"cbdfdf81-b89f-449e-d624-540aa9aceed0","colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["lens_train = [len(i) for i in X_train]\n","lens_test = [len(i) for i in X_test]\n","lens = lens_train + lens_test\n","\n","maxlen = np.max(lens)\n","\n","print('Max len:', maxlen)"],"execution_count":9,"outputs":[{"output_type":"stream","text":["Max len: 118\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"DLwovzmrpFQB"},"source":["Then we pad our inputs to fit that size."]},{"cell_type":"code","metadata":{"id":"2n1-CNwTpNF9","executionInfo":{"status":"ok","timestamp":1602769193849,"user_tz":-180,"elapsed":15564,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}}},"source":["X_train = pad_sequences(X_train, padding='post', maxlen=maxlen)\n","X_test = pad_sequences(X_test, padding='post', maxlen=maxlen)"],"execution_count":10,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"dXkNGdGXpebb"},"source":["## Label Encoding"]},{"cell_type":"markdown","metadata":{"id":"Nm167nTwprAv"},"source":["We could get away with other approach but you might want to use this for more than binary classification."]},{"cell_type":"code","metadata":{"id":"NdvqxTMYpnqX","executionInfo":{"status":"ok","timestamp":1602769193851,"user_tz":-180,"elapsed":15564,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}}},"source":["encoder = LabelEncoder()\n","encoder.fit(y)\n","encoded_Y_test = encoder.transform(y_test)\n","encoded_Y_train = encoder.transform(y_train)\n","\n","# convert integers to dummy variables (i.e. one hot encoded)\n","dummy_y_test = np_utils.to_categorical(encoded_Y_test)\n","dummy_y_train = np_utils.to_categorical(encoded_Y_train)"],"execution_count":11,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"jiGcqcLLPnCq"},"source":["# Modelling\n","The model that we use here is a simple Neural Network. I'll add some comments to explain what each line does."]},{"cell_type":"code","metadata":{"id":"SRc8JH-S-fBl","executionInfo":{"status":"ok","timestamp":1602769199070,"user_tz":-180,"elapsed":20774,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"350ba370-5095-4870-e2a0-dee8647d2b82","colab":{"base_uri":"https://localhost:8080/","height":463}},"source":["embedding_dim = 100 # vector representation of words/neuron amount after the input https://datascience.stackexchange.com/questions/53995/what-does-embedding-mean-in-machine-learning#:~:text=In%20the%20context%20of%20machine,with%20other%20models%20as%20well.\n","\n","model = Sequential() # initiates the model\n","model.add(layers.Embedding(input_dim=vocab_size, # adds the first [input] layer which will be our tokenized tweets\n"," output_dim=embedding_dim, # the embedding of that tweet, essentially inputs output\n"," input_length=maxlen)) # size of the input layer determined by maxlen calculated before\n","model.add(layers.Dropout(0.2)) # dorpouts are added to help with overtraining, essentially \"turns off\" said amount of neurons before giving information to the next layer\n","model.add(layers.GlobalMaxPool1D()) # https://computersciencewiki.org/index.php/Max-pooling_/_Pooling\n","model.add(layers.Dropout(0.2))\n","model.add(layers.Dense(50, activation='relu')) # additional hidden layer\n","model.add(layers.Dropout(0.2))\n","model.add(layers.Dense(50, activation='relu'))\n","model.add(layers.Dropout(0.2))\n","model.add(layers.Dense(2, activation='softmax')) # prediction layer, 2 is the number of classes we have\n","model.compile(optimizer='adam',\n"," loss='categorical_crossentropy',\n"," metrics=['accuracy'])\n","model.summary()"],"execution_count":12,"outputs":[{"output_type":"stream","text":["Model: \"sequential\"\n","_________________________________________________________________\n","Layer (type) Output Shape Param # \n","=================================================================\n","embedding (Embedding) (None, 118, 100) 11789900 \n","_________________________________________________________________\n","dropout (Dropout) (None, 118, 100) 0 \n","_________________________________________________________________\n","global_max_pooling1d (Global (None, 100) 0 \n","_________________________________________________________________\n","dropout_1 (Dropout) (None, 100) 0 \n","_________________________________________________________________\n","dense (Dense) (None, 50) 5050 \n","_________________________________________________________________\n","dropout_2 (Dropout) (None, 50) 0 \n","_________________________________________________________________\n","dense_1 (Dense) (None, 50) 2550 \n","_________________________________________________________________\n","dropout_3 (Dropout) (None, 50) 0 \n","_________________________________________________________________\n","dense_2 (Dense) (None, 2) 102 \n","=================================================================\n","Total params: 11,797,602\n","Trainable params: 11,797,602\n","Non-trainable params: 0\n","_________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"9OdXHctIZmeI"},"source":["So we have our model. All we need now is to train it."]},{"cell_type":"code","metadata":{"id":"rhJt6HHXsEL9","executionInfo":{"status":"ok","timestamp":1602769596187,"user_tz":-180,"elapsed":364982,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"d4251e4f-70ba-4977-c4d9-d2a580543873","colab":{"base_uri":"https://localhost:8080/","height":174}},"source":["history = model.fit(X_train, dummy_y_train,\n"," epochs=3, # times model will run through the data\n"," verbose=True,\n"," validation_data=(X_test, dummy_y_test),\n"," batch_size=128) # data is set to batches we are sent to the model to predict, imagine each batc as a step in which model tries to predict the class and then checks the right answer and corrects it's weights with backpropogation\n","loss, accuracy = model.evaluate(X_train, dummy_y_train, verbose=False)\n","print(\"Training Accuracy: {:.4f}\".format(accuracy))\n","loss, accuracy = model.evaluate(X_test, dummy_y_test, verbose=False)\n","print(\"Testing Accuracy: {:.4f}\".format(accuracy))"],"execution_count":14,"outputs":[{"output_type":"stream","text":["Epoch 1/3\n","1000/1000 [==============================] - 118s 118ms/step - loss: 0.4802 - accuracy: 0.7713 - val_loss: 0.4452 - val_accuracy: 0.7908\n","Epoch 2/3\n","1000/1000 [==============================] - 118s 118ms/step - loss: 0.3878 - accuracy: 0.8239 - val_loss: 0.4497 - val_accuracy: 0.7910\n","Epoch 3/3\n","1000/1000 [==============================] - 118s 118ms/step - loss: 0.3182 - accuracy: 0.8601 - val_loss: 0.4792 - val_accuracy: 0.7850\n","Training Accuracy: 0.9194\n","Testing Accuracy: 0.7850\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"cOlqqrekXqjU"},"source":["Let's have a nice plot of our training process."]},{"cell_type":"code","metadata":{"id":"yGzZxjWN7aoc","executionInfo":{"status":"ok","timestamp":1602769597136,"user_tz":-180,"elapsed":925,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}}},"source":["plt.style.use('ggplot')\n","\n","def plot_history(history):\n"," acc = history.history['accuracy']\n"," val_acc = history.history['val_accuracy']\n"," loss = history.history['loss']\n"," val_loss = history.history['val_loss']\n"," x = range(1, len(acc) + 1)\n","\n"," plt.figure(figsize=(12, 5))\n"," plt.subplot(1, 2, 1)\n"," plt.plot(x, acc, 'b', label='Training acc')\n"," plt.plot(x, val_acc, 'r', label='Validation acc')\n"," plt.title('Training and validation accuracy')\n"," plt.legend()\n"," plt.subplot(1, 2, 2)\n"," plt.plot(x, loss, 'b', label='Training loss')\n"," plt.plot(x, val_loss, 'r', label='Validation loss')\n"," plt.title('Training and validation loss')\n"," plt.legend()"],"execution_count":15,"outputs":[]},{"cell_type":"code","metadata":{"id":"Z6I3hCdk7ciN","executionInfo":{"status":"ok","timestamp":1602769597762,"user_tz":-180,"elapsed":1538,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"fa1b1fda-57ab-4559-e279-f9bb6568a800","colab":{"base_uri":"https://localhost:8080/","height":337}},"source":["plot_history(history)"],"execution_count":16,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAskAAAFACAYAAABOYuFgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVxU9f7H8deZGWDYYQYV1yxSc19bpC5poCK4ZSquaVZWmuvNXcvc87pU/ijXa1lmuOSSCIVLWZpmejVTS01NTU3ZRJYBZub8/iAnERFUYAb4PB8PHzIz58x5nxn48uHM93yOoqqqihBCCCGEEMJGY+8AQgghhBBCOBopkoUQQgghhLiFFMlCCCGEEELcQopkIYQQQgghbiFFshBCCCGEELeQIlkIIYQQQohbSJFsJ9988w2KonDhwoW7Wk9RFD799NNiSlVySmI/zp49i6IofP/993e13VatWvHSSy/d9/Y/+ugjdDrdfT+PEKLskLFfxv6iVFSZxe1JkVwARVHu+K9mzZr39LyBgYFcunSJKlWq3NV6ly5dolu3bve0TVE8r9+FCxdQFIVvvvkm1/0RERH8+eefRbotIUTJkLG/bJGxX9wLOcxVgEuXLtm+3rNnD8899xwHDx6kcuXKAGi12lzLZ2Vl4ezsXODzOjs74+/vf9d57mUd8Y+SfP1cXV1xdXUtse05ouzsbJycnOwdQ4i7JmN/2SJjv7gXciS5AP7+/rZ/BoMBgAoVKtjuq1ixIu+//z69e/fG29ubfv36ATBx4kTq1q2Lm5sb1atX59VXX+XatWu25731I7cbt+Pi4ggKCsLNzY169eoRExOTK8+tHxkpisIHH3xAv3798PT0pFq1asyaNSvXOgkJCXTv3h13d3cqVarE5MmT6d+/PyEhIXfc94L24cZHSrt376ZZs2a4ubnRvHlz9u/fn+t5du7cSaNGjdDr9TRq1IidO3fecbsnT55EURT27NmT6/59+/ahKAonT54E4L333qNJkyZ4eHjg7+9Pz549c/1iu51bX78//viD0NBQXF1dqV69OgsXLsyzzmeffcbjjz+Ot7c3fn5+hIeHc+LECdvj1atXB6B169a5jjDd7iO3rVu30rx5c1xcXKhYsSKDBw8mLS3N9viAAQMICQlhyZIlPPDAA3h5edGpUyf++uuvO+5XQRkBrly5wgsvvEClSpXQ6/XUqVOH//73v7bHf//9d7p164bBYMDNzY1GjRqxZcuWfPfl1qMoN76Ho6Ojeeqpp9Dr9SxbtoykpCT69u1LjRo1cHV1pU6dOsybN49bL/YZFRVF8+bN0ev1GI1G2rdvT1JSEh999BE+Pj6kp6fnWn7q1KnUqlUrz/MIURRk7JexvzSM/bfKzs5m3LhxVK1aFWdnZ+rVq8dnn32Wa5lly5ZRt25d9Ho9BoOBoKAg2/djSkoKL7zwAv7+/ri4uFC9enVGjRp1VxnKEimSi8Dbb79NYGAgBw8eZPr06UDOX5JLlizh2LFjfPTRR3zzzTcMGzaswOd64403mDBhAocPH+bxxx8nIiKCpKSkArcfFBTEoUOHGD9+PBMmTGD79u22x1944QUOHz7Mli1b2LFjBxcuXGDjxo0FZinMPlitVsaPH897773HwYMHqVixIj169MBsNgNw8eJFOnToQPPmzTl48CDz5s1j+PDhd9xurVq1aNmyJZ988kmu+z/++GNatmxJrVq1bPfNnTuXI0eOsGHDBs6dO0fPnj0L3K8bVFXl2WefJSEhgW+++YYvv/ySzZs3c/DgwVzLZWZmMmnSJA4ePEhcXBxarZbw8HCysrIAbMuvX7+eS5cu5flFccPPP/9Mp06dCAoK4vDhw3z88cds2bKFV199Nddy+/fvZ+fOnURHR/PVV19x5MgR3njjjTvuS0EZMzIyePrppzl8+DCrVq3i2LFjLFy4EDc3NwAuX75MYGAgycnJbN68mSNHjjBt2jQ0mrsfIv79738zduxYjh8/TseOHcnMzKRBgwZs3LiRY8eOMXnyZN566y0++ugj2zorVqygb9++dOnShYMHD7Jz505CQ0OxWCxERESgKApr1661LW+1Wvnvf//LSy+9hKIod51RiKIgY7+M/WDfsf9WEyZMYOnSpbz77rv88ssv9O3bl759+9q+Lw4cOMCrr77K+PHj+e233/j22295/vnnbevf2N9NmzZx8uRJoqKiqFu37l1lKFNUUWg7d+5UAfX8+fO2+wB14MCBBa77xRdfqM7OzqrFYrntc924vX79ets6ly9fVgE1NjY21/Y++eSTXLeHDh2aa1uPPPKIOm7cOFVVVfXEiRMqoG7bts32eFZWllqtWjU1ODj4bnY/zz6sWLFCBdQDBw7Yltm7d68KqL/++quqqqo6ceJEtUaNGmp2drZtmS+//DLPftzqww8/VH19fdXMzExVVVU1MzNTNRgM6qJFi/Jd5+DBgyqgXrhwQVVVVT1z5owKqN99951tmZu3GxcXpwLqb7/9Znv8ypUrql6vV1988cV8t5OQkKAC6vfff6+qqqqeP39eBdSdO3fmWm7FihWqVqu13e7bt6/66KOP5lpm48aNqqIo6tmzZ1VVVdX+/furFSpUUE0mk22Z2bNnq/7+/vnmKUzGZcuWqS4uLrm+d282adIktVKlSmpqauptH791X1Q1737f+B5euXJlgfmGDRumhoSE2G5Xr15dHTJkSL7LDx06VH3yySdtt2NjY1UnJyf1r7/+KnBbQtwvGftl7FdVxxz7n376aVvmtLQ01dnZWY2MjMy1TJcuXdTWrVurqprzXnp5eanXrl277fN16tRJ7d+//x23WZ7IkeQi8Nhjj+W574svviAoKIgqVarg4eFBnz59yMrK4vLly3d8riZNmti+rlSpElqttsCPW25eB6BKlSq2dY4dOwbAE088YXvcycmJFi1a3HmnCrkPiqLQuHHjXNsGcm3/sccey/XR01NPPVXgtiMiIkhPT7d93L9lyxbS0tKIiIiwLfPNN9/Qrl07qlevjqenp+15//jjjwKf/0Y2Pz8/ateubbuvQoUK1KlTJ9dyhw4d4tlnn+XBBx/E09OTGjVq3NV2bjh69ChBQUG57nv66adRVdX2PgE88sgjuLi42G7f/H7mp6CMBw4coF69elSrVu226x84cIDAwEDc3d3vap9u59afB6vVyuzZs2nSpAl+fn54eHiwaNEiW7YrV65w/vx52rZtm+9zvvLKK+zevZvjx48DsHTpUjp16kTFihXvO68Q90rGfhn7C6M4x/6bnTp1iqysrNtu6+jRowC0adOGhx56iAcffJCePXuyZMkS4uPjbcsOHjyYdevW0aBBA4YPH05MTAxWq/Wu9rcskSK5CNxaWOzbt4/u3bsTFBTEhg0bOHjwIIsWLQKwfUyTn9ud+FHQN+it6yiKkmedu/1IurD7oNFocp3AcmM79/tD5evrS8eOHVm5ciUAK1eupFOnTvj4+ABw7tw5wsLCqFmzJp9//jk//fQTmzdvzpPvfqWnp9O2bVsURWHFihX8+OOP7N+/H0VRinQ7N7vd+6neYd5tSWS83bSL7Ozs2y5768/DvHnzmDVrFsOGDSMuLo5Dhw7x0ksv3VW2+vXr89RTT7F06VKuXLnC5s2bGTRo0N3thBBFTMZ+GfuL0t2O/ffCw8ODn376iQ0bNlC7dm0WLVrEww8/zIEDBwBo164d586dY+LEiZhMJvr27cszzzyDxWIp0hylhRTJxeD777/Hz8+P6dOn8/jjj1O7du277olZVOrVqwfADz/8YLvPbDbbfiDyU1T7UK9ePX788cdcP2C7d+8u1Lr9+/dn69at/Pbbb2zdujXXvKn9+/eTkZHBu+++y5NPPkmdOnXu+gSHevXqER8fbzsZBCA+Pp7ffvvNdvv48eNcvXqVGTNm0KpVK+rWrUtSUlKugevGwFbQIFK/fn127dqV675vv/0WRVGoX7/+XWW/WWEyNm/enGPHjuX7HjZv3pw9e/bkOpHkZhUrVsRiseR6jW+dv5efXbt2ERoaysCBA2natCkPP/xwrte8YsWKVKtWja+//vqOz/PKK6+wcuVKlixZQtWqVWnTpk2hti9ESZGxP/f2ZezPUVxj/60efvhhXFxcbrutBg0a2G5rtVqCgoKYOnUqBw4coHLlyrlO7jMYDPTq1YvFixcTHR3Nt99+m+uId3kiRXIxqFOnDlevXmX58uWcPn2alStX8sEHH9glS61atejYsSNDhgyxfaO/8sorpKSk3PEIQ1Htw2uvvcbVq1cZNGgQx48fZ/v27UycOLFQ64aGhuLr60vPnj3x9fUlNDQ0134pisK8efM4c+YMGzduZOrUqXeVLTg4mMaNG9O3b19+/PFHDh06RJ8+fXK1LHvggQdwcXFh4cKF/P7772zfvp3hw4fneu1uTCH4+uuvuXz5cr4n24wePZqDBw8ycuRIfv31V2JjYxk6dCh9+vSxfYx3LwqTsVevXjzwwAN06tSJbdu2cebMGbZv305UVBSQ8xGb1Wqlc+fO7N69mzNnzrBlyxbbGfaPPfYYnp6ejBs3jpMnTxIbG1vo17tOnTp888037Ny5kxMnTjBp0iT27duXa5m33nqLxYsXM23aNI4fP87Ro0f5v//7v1wfA97ocTpt2jQ5YU84JBn7/yFj/z+Ka+y/lZubG8OGDWPy5MmsXbuWEydOMHPmTDZt2sSECRMA2LRpEwsWLODAgQOcO3eOjRs3cv78edsfVRMnTuSLL77gt99+4+TJk6xatQoPD48izVmaSJFcDDp06MDEiROZMGECDRs25PPPP+c///mP3fKsWLGCBg0a0L59e1q1amU7CqfX6/Ndp6j2oWrVqnz55Zf8+OOPNGnShOHDhzN//vxCravT6ejduzeHDh2id+/euea2NWrUiIULF7J48WLq1avH3Llzeffdd+8qm6IobNy4EW9vb4KCgujQoQNhYWE0a9bMtoyfnx+ffvopcXFx1K9fnzfeeIO5c+fmmn6g0WiIjIxkzZo1VKtWjaZNm952e40aNWLz5s3s2rWLxo0b069fP8LDw20fZd6rwmR0c3OzHU3o2bMndevWZciQIWRkZABQuXJlvv/+ezw9PQkLC6N+/fpMnDjRdtTEYDCwevVq9u7dS6NGjZg2bRpz5swpVL7Jkyfz9NNP07lzZ1q2bElSUlKeM+VfeuklPvroI9atW0eTJk0ICgoiJiYm13uu1+vp168fVquVgQMH3tdrJkRxkLH/HzL2/6O4xv7bmTFjBi+//DIjRoygQYMGfPrpp3z66acEBwcDOdNZvvzyS0JDQ6lduzZjxoxh0qRJvPjii0DOOPvmm2/SvHlzWrRowc8//0xMTAze3t5FnrU0UNSinvAiHJ7FYuGRRx6hU6dOzJs3z95xhCi0Hj16kJ2dzYYNG+wdRYhSR8Z+Ie6OXHGvHNi1axdXrlyhadOmXL9+nQULFnD27FkGDBhg72hCFEpSUhI//vgjGzZsyNUHVgiRPxn7hbg/UiSXAxaLhenTp3Pq1CmcnJxo0KABO3fupGHDhvaOJkShNG3alISEBMaMGZOnvZEQ4vZk7Bfi/sh0CyGEEEIIIW4hJ+4JIYQQQghxCymShRBCCCGEuIUUyUIIIYQQQtzCIU/cu3jx4l2v4+fnl+vCA/bkKFkcJQdIFkfOAY6TxVFywL1nqVKlSjGkcXx3O26Xhfe6ODhKFkfJAY6TxVFygGQpyhx3GrPlSLIQQgghhBC3kCJZCCGEEEKIW0iRLIQQQgghxC0cck7yrVRVxWQyYbVaURTltsv89ddfZGZmlnCy23OULCWVQ1VVNBoNer0+3/dHCCGEEKI0KRVFsslkwsnJCZ0u/7g6nQ6tVluCqfLnKFlKMofZbMZkMuHq6loi2xNCCCGEKE6lYrqF1Wq9Y4Es7E+n02G1Wu0dQwghhBCiSJSKIlk+wi8d5H0SQgghRFlRqMOzhw4dYsWKFVitVoKDg+nSpUuux+Pj44mMjCQtLQ2r1Urv3r1p1qwZAH/88QdLliwhIyMDRVGYNWsWzs7ORb8nxSgxMZGIiAgArl69ilarxWAwABAdHX3H/Tl8+DDr1q1j2rRpd9xGp06d2Lx5c9GFFkKUWwWN2Tfs3buX+fPnM2vWLAICAjCbzSxatIgzZ85gtVoJCgri2WefLeH0QgjhGAoskq1WK8uXL2fSpEkYjUbGjx9PixYtqFatmm2Z9evX07JlS9q2bcuFCxeYNWsWzZo1w2KxsHDhQl5//XVq1qzJ9evXS+W0CYPBQFxcHADz5s3D3d2dV1991fa42WzOd78aN25M48aNC9yGFMhCiKJQmDEbICMjg5iYGGrVqmW7b+/evZjNZubNm0dmZiajRo3iySefpGLFiiW9G0IIYXcFTrc4deoU/v7+VKpUCZ1OR2BgIPv378+1jKIopKenA5Ceno6vry+QcxS1Ro0a1KxZEwBPT080mlIxw6NAI0aMYOzYsXTo0IHp06fzv//9j44dO9K2bVvCw8M5deoUAHv27OH5558HcgrsUaNG0a1bN1q2bMny5cttz3fjF9WePXvo1q0bL7/8MkFBQbz++uuoqgrA9u3bCQoKIjQ0lMmTJ9ue92bnz5/n2WefpV27doSEhOR6ryIjIwkODiYkJISZM2cCcObMGSIiIggJCaFdu3acPXu2WF4vIUqTmBg9hw6VzulDhRmzAaKioujcuTNOTk657jeZTFgsFrKystDpdLi5uRV5xtRUhQULNMhpDEIIR1bgYd3ExESMRqPtttFo5OTJk7mW6d69O9OnTyc2NpbMzEwmT54MwKVLl1AUhRkzZpCSkkJgYCCdO3cu4l2wn0uXLrFp0ya0Wi3Xr19nw4YN6HQ6du/ezTvvvMPSpUvzrHPq1CnWrl1LWloa//rXv3j++efz/JL65Zdf2LFjB/7+/nTu3Jn9+/fTqFEjxo4dyxdffEGNGjUYPHjwbTP5+fmxevVq9Ho9586d45VXXiEmJoYdO3bw1VdfsWXLFlxdXUlKSgJg6NChDBkyhPbt22MymWwFuRDlUUYGTJ3qzcqV7kREWJg/396J7l5hxuzTp08THx9Ps2bNcn2K9cQTT/DTTz8xaNAgsrKy6N+/Px4eHkWecdMmV8aN03H0qDezZl1DTmcQQty3tLQif8oimfuwe/duWrVqRceOHTlx4gQLFy5k3rx5WCwWfv31V2bNmoWLiwtTp07loYceomHDhrnW37ZtG9u2bQNg9uzZ+Pn55Xr8r7/+sk1nmDTJg6NHi3bKRv36ZqZPTy3UshqNxvavc+fOuLi4ADlH0EeOHMnp06dRFMU2BUOr1aIoCjqdDo1GQ5s2bXB3d8fd3Z0KFSqQlJRku274jeWbNm1KjRo1AGjYsCEXL17Ey8uLmjVr8tBDDwHQtWtXPvnkkzzTPFRVZdy4cfzyyy9otVpOnz5tK9x79eqFp6cnABUqVCA1NZXLly/TsWNHgPv+Zeji4pLnvbtBp9Pl+1hJc5QsjpIDHCeLPXMcO6bQp4+WY8c0jBplYcYMBY3G/q9JUbNaraxcufK2f2ifOnUKjUbD4sWLSUtL480336Rhw4ZUqlQpz7IFjdt3MmwYxMerzJnjjoeHngULLHYtlB3l+x8cJ4uj5ADHyeIoOUCy2Bw/jmbDBjQbNqDo9fh9912RPn2B1abBYCAhIcF2OyEhwXbS2g07duxgwoQJANSuXZvs7GyuX7+O0Wikbt26eHl5AdC0aVPOnDmTp0gOCQkhJCTEdjs+Pj7X45mZmbZ+v1ar9bZHOxVFueejoFarFbPZXOhlb/xzcXGxrTdr1ixatmzJsmXLuHjxIs8++yxmsxmLxYKqqpjNZqxWK05OTrZ1NBoNmZmZtts3lr95GUVRbMvceJ6bX4dbc3/44YcYjUbi4uLQaDTUqFHDtu1b9/PW57xfmZmZed67G/z8/PJ9rKQ5ShZHyQGOk8UeOVQVPv3UjSlTvPHwsPLZZwk8/XQmGs29ZbnxR6+9FDRmm0wmzp8/z9tvvw1AcnIyc+bMYcyYMXz//fc0adIEnU6Ht7c3derU4ffff79tkVzQuF2QqVP9SE7O4sMPPcjOzmDKlBS7FcqO8v0PjpPFUXKA42RxlBxQjrOoKrqjR3HduhX91q04/f0pWVaLFmi6dyf+6lXudiC505hdYJEcEBDApUuXuHLlCgaDgT179jBs2LBcy/j5+fHLL7/QqlUrLly4QHZ2Nl5eXjRu3JjNmzeTmZmJTqfj+PHjhIeH31X4W02dmnL7HdHpiqzYuxfXr1/H398fyJnrV9QCAgL4448/OH/+PNWrV8/3RL+UlBQqV66MRqNh7dq1WCwWAIKCgliwYAFdu3a1Tbfw9fWlcuXKxMbGEhoaSmZmJlarVS4IIsqN5GSF0aN92LrVlVatTLz7bjIVKpTuibIFjdlubm65zoeYMmUK/fr1IyAggCNHjvDLL78QFBSEyWTi5MmT9z1m50dR4M03U7BYYNkyD5ycYOJE+xXKQggHpao4HTqEfutWXLduRXf2LKpGQ9YTT5A8YACm0FCs/v45R7OLuFgvsEjWarUMHDiQGTNmYLVaad26NdWrVycqKoqAgABatGjB888/z+LFi4mOjgZg8ODBKIqCh4cH4eHhjB8/HkVRaNq0qa01XFnz2muvMWLECN577z3atGlT5M/v6urKzJkz6dOnD25ubvl2zOjfvz+DBg1i3bp1BAcH2066ad26NUePHqV9+/Y4OTnxzDPPMH78eN5//33Gjh3L3Llz0el0LF68mAceeKDI8wvhaH780ZkhQ3y4ckXL5MnXGDQojbJwXnFhxuz8hIaG8sEHHzBq1ChUVaV169bFOh4oCrz9dgpms8KHH3qg06mMHXtdCmUhyjurFeeffkK/ZQv6mBh0Fy+i6nRkPvUUqYMH5xTGN517UVwU1QHP1Lp48WKu2+np6QWeYW3vI8k3K64saWlpuLu7o6oqEyZM4MEHH2TQoEElniM/d3qfyu1HQ6UgBzhOlpLIYbHA++97MH++JzVqWIiMTKJJk+wiy2Lv6Rb2cuu4XZCbX1+rFcaN82bVKndGjbrOv/99vTgiFiqLvTlKFkfJAY6TxVFyQBnNYjbj/MMPOVMpYmPRXrmC6uJCZlAQGeHhmNq0QfXxKfIc9zXdQjiOVatWsXbtWrKzs2nQoAH9+vWzdyQhSpWLFzUMG+bLDz+40LVrOjNnXsPT0+GOE5Q7Gg3Mnn0NiwXmz/dEq1UZMaJwJ1MLIUqxrCxcvv8e/Y3COCkJq6srmc88Q0Z4OJnBwajF0GGnsKRILkUGDRp0xyPHQoj8ffWVnlGjfMjKgnffTaJ79wx7RxI30WhgzpxrmM0K//mPFzodvP66FMpClDkZGeh37cqZSrFtG5qUFKweHpjatMEUFkZm69aoDnJulBTJQogyzWSC6dO9WLHCg4YNs4iMTCIgwGLvWOI2tFqYPz8ZiwVmzfJCp1N59dWi730qhChZSloaLjt24Bodjcv27WjS07H6+GAKDSUjLIzMoCD4u6WuI5EiWQhRZp08qeO113w5ftyJQYNSGTcuxRHHYXETrRbefTcZs1lh2jRvdDp46SUplIUobZSUFPRxcTlTKb75BsVkwuLnR0bXrpjCw8ls2RJuuZiao5EiWQhR5qgqrF7txuTJXri7q6xcmUBwcKa9Y4lC0ulg4cIkLBZ46y1vdDqVAQPS7R1LCFEAJTER/ddf5xwx/u47lOxsLP7+pPXujSksjKzHHsv5S7iUkCJZCFGmXLumMHasD19+6cq//pXJe+8lUalS6e59XB45OUFkZBKvvAITJ/qg00HfvlIoC+FwLl/G7bPPcI2OxvmHH1AsFszVq5M2cCAZYWFkN2tGae2vWTpTl7Bu3brxzTff5Lpv6dKljBs37o7rHD58GIB+/fpx7dq1PMvMmzePRYsW3XHbsbGxnDhxwnb7P//5D7t27bqL9EKUHz/95ETbthWIidEzYUIKn32WIAVyKebsDIsWJREcbGLsWB8+/9wxTuYRorzT/Pkn7suWYezaFaeaNfEZPx7txYukvvYaV2NjufLDD6S8+SbZLVqU2gIZ5EhyoXTp0oVNmzbRqlUr232bNm1i0qRJhVr/k08+uedtx8bGEhISQu3atQEYPXr0PT+XEGWVxQKRkR7MnetJ1aoWNmyIp1mzvL2PRenj4gJLliTy4osG3njDB40GevSQziRClDTtH3/kXPUuOhrn//0PgOxHHsEyaRKJrVphrlPnri8J7ehKb3lfgsLDw9m+fTtZWVkAnD9/nr/++ovHH3+ccePG0b59e1q3bs3cuXNvu/7jjz9OYmIiAO+99x5PPfUUXbp04ffff7cts2rVKsLCwggJCeHll18mIyOD/fv3ExcXx/Tp02nTpg1nz55lxIgRbNmyBYDvvvuOtm3bEhwczKhRo8jMzLRtb+7cuYSEhBAcHMypU6fyZDp//jzPPvss7dq1o127duzfv9/2WGRkJMHBwYSEhDBz5kwAzpw5Q0REBCEhIbRr146zZ8/e/wsrRBG4fFlDz55G3nnHiw4dMvjqq6tSIJcxej0sW5bIU09lMWqUD198IUeUhSgJulOn8HjvPfzataNSYCDe06eDxULKuHH8tWsXV7dvxzppEuZHHilzBTLIkeRC8fX1pUmTJuzcuZN27dqxadMmOnbsiKIojB07Fl9fXywWCxERERw7doxGjRrd9nl+/vlnNm/eTFxcHGazmdDQUNuy7du3p0+fPgC88847rF69moEDB9KmTRtCQkLo0KFDrucymUyMHDnSdqnZYcOGsXLlSl5++WUADAYD27ZtY9myZSxatChPAe/n58fq1avR6/WcPn2aIUOGEBMTw44dO/jqq6/YsmULrq6uJCUlATB06FCGDBlC+/btMZlMOOCFGkU5FBfnwsiRPphMCvPnJ9GjR0ZZHKcF4OoKK1Yk8vzzBoYP90GrVenc2WTvWEKULaqK7vhxXKOj0W/ditPf0z2zmjfn2uTJmMLCsNSoYeeQJafUFcleb76J07Fjee5XFOWeC7fsevVImTr1jsvcmHJxo0ieN28eAF9++SWrVkpJ2rYAACAASURBVK3CYrHw119/cfLkyXyL5H379hEaGorr302y27RpY3vst99+Y86cOaSkpJCWlsbTTz99xzy///47NWrUICAgAIDu3bvz8ccf24rk9u3bA9CoUSNiYmLy7nN2NhMnTuTYsWNoNBpOnz4N5BydjoiIsGX09fUlNTWVS5cu2Z5Tr9ffMZsQxS0zE2bM8GL5cg/q18/mgw8Sefhh6X1c1rm6qnz8cSJ9+xoYOtQXnS6J8HAplIW4L6qK0+HDtqkUurNnUTUash5/nGvTppHRvj3WypXtndIuSl2RbC/t2rVjypQpHDlyhIyMDBo1asS5c+dYvHgx0dHR+Pj4MGLECEymexuwR44cyfLly6lfvz5RUVH88MMP95XX5e9msFqtFoslb/GwdOlSKlSoQFxcHFarlYceeui+tidESTl1SsvgwQaOHnXixRdTmThReh+XJ25uKitXJtKnj5HBg31ZsiSJdu2kUBbirlitOB84gD46Gn1MDLoLF1B1OjKffJLU117DFBqK1c/P3intrtQVyfkd8dXpdJjN5mLbrru7O4GBgYwaNYouXboAcP36dVxdXfHy8uLq1avs3LmTli1b5vscTzzxBCNHjuT111/HYrEQFxdHv379AEhNTaVSpUpkZ2ezYcMG/P39AfDw8CAtLW8j/YCAAM6fP8+ZM2d48MEHWb9+PU888USh9yclJYXKlSuj0WhYu3atrZAOCgpiwYIFdO3a1TbdwtfXl8qVKxMbG0toaCiZmZlYrVbb0WYhSoKqwpo1rkyc6I1er/LRRwm0aSO9j8sjDw+VTz9NoFcvI6+84svSpYnyvSBEQcxmnPftw3XrVvQxMWj/+gvV2ZnMoCCujxqFqW1bVF9fe6d0KKWuSLanLl268OKLL/Lhhx8CUL9+fRo0aEBQUBBVqlTh0UcfveP6DRs2pGPHjrRp0wY/Pz+aNGlie2z06NF06NABo9FI06ZNSU1NBaBz586MHj2a5cuXs2TJEtvyer2e+fPn88orr2CxWGjcuLGt4C6M/v37M2jQINatW0fr1q1xc3MDoHXr1hw9epT27dvj5OTEM888w/jx43n//fcZO3Ysc+fORafTsXjxYh544IFCb0+I+5GSojB+vDcbN7oRGJjJ++8nUbmytHYrzzw9VVatyimUBw0y8N//JtK6tRTKQuSSlYXLnj05R4xjY9EmJmLV68l85hlMYWGYQkJQPT3tndJhKaoDnoF18eLFXLfT09NtRVx+ivtI8t1wlCwlneNO75Ofnx/x8fElluVOHCWLo+QAx8lyuxwHDzoxZIgvf/6p5Y03rjNkSGqJXLDpXl+TKlWqFEMax3fruF2QovqeS05WiIgwcvKkEx99lEhQ0N0Xyo7y/Q+Ok8VRcoDjZHGUHFBAFpMJl127ck6+i4tDc+0aVnd3TCEhOZeDbt0atYCaqsiylKDiGLPlSLIQwiFZrfDhhx7MmeOJv7+F9evjefRRae0mcvPxUVm9OoEePfx44QUDH3+cwFNPZdk7lhAlSklPx2XHDvRbt6Lftg1NWhpWb29MbduSERZGZlBQTi9FcVekSBZCOJy//tIwfLgv333nQocOGcyZk4y3t8N96CUchMGgEhWVQPfuRgYMMPDpp4k88YQUyqJsU65fR79tG/roaFx27kRjMmExGsno0gVTWBiZgYE5l60U90yKZCGEQ9mxw4URI3xIS1P4z3+S6dUrXXofiwIZjVaiohLo1s1Iv34GPvsskUcflUJZlC1KUhL6r79GFxeH//btKFlZWCpVIqNnTzLCwsh6/HHQSWlXVErFK+mA06bFbcj7JO5HZiaMGaPlvfeM1K2bzYcfJlGrlv3n9ovSo0IFK2vWJNCtmx99+hhYvTqB5s1lio4o3TRXr6KPjUW/dSsue/agmM2oNWqQNmAAGWFhZDdvDhq5gHJxKBVFskajwWw2o5O/jhyW2WxGIz+k4h6dPq1l8GBfjhzR8sILqUyalCLT58Q9qVTJypo18X8XykY+/zyBJk2kUBali+bSJVxjYtBv3Yrzvn0oVivmmjVJffVVTGFheD/zDCkJCfaOWeaViqpTr9djMpnIzMxEyedzVxcXFzIzHaP9j6NkKakcqqqi0WjkSnzinqxd68qECd44O8PatdkEBqbYO5Io5SpX/qdQ7t3bSFRUAg0bSqEsHJv2/Hn00dG4RkfjfPAgANl16pA6fDgZYWGY69bFNvdM5qCViFJRJCuKUuCFKxylBQk4ThZHySHE7aSm5vQ+/uILN554IpOFC5No1MiAfMvev0OHDrFixQqsVivBwcG2CyDdau/evcyfP59Zs2bZLnH/xx9/sGTJEjIyMlAUhVmzZuFcCk/+qVrVytq1CTz3nJGePY2sWRNP/foyfUc4Fu2pUzkX99i6FecjRwDIatCAlLFjyQgLw/Lww3ZOWL6ViiJZCFG2HD7sxODBvpw7p+WNN1IYNqxkeh+XB1arleXLlzNp0iSMRiPjx4+nRYsWVKtWLddyGRkZxMTEUKtWLdt9FouFhQsX8vrrr1OzZk2uX79eqqe5Vatm+XuOspGICCNr1yZQt64UysKOVBXdr7/aCmOnX38FIKtpU65NnoypfXsscqEuh1F6Rz8hRKljtcKSJe7MmuVFxYoW1q9P4LHHpANBUTp16hT+/v5UqlQJgMDAQPbv35+nSI6KiqJz585s3rzZdt/hw4epUaMGNWvWBMCzDFyJ64EHLLaT+SIijKxbl0Dt2lIoixKkqjgdOWKbSqE7cwZVUch6/HGuTZ1KRmgo1qpV7Z1S3IYUyUKIEnH1qoYRI3z45hs9YWE5vY99faUjSlFLTEzEaDTabhuNRk6ePJlrmdOnTxMfH0+zZs1yFcmXLl1CURRmzJhBSkoKgYGBdO7cucSyF5cHH7TY5ij36JFTKD/8sBTKohhZrTgdPJhz1buYGHTnz6NqtWQFBpI6aBCm0FCsFSvaO6UogBTJQohi9+23Lgwf7sP16xpmz06mb1/pfWwvVquVlStXMnjw4DyPWSwWfv31V2bNmoWLiwtTp07loYceomHDhnmW3bZtG9u2bQNg9uzZ+Pn53VUOnU531+vcDz8/+PprC23bOtGzZwXi4rK5MdOkpLPciaNkcZQc4DhZCsxhsaB8/z2aDRvQbNqEcvEiqpMTanAw5smTsXbogGI04gbc70WhHeU1AcfJUhw5pEgWQhSbrCyYM8eLDz/0oE6dbD7/PIFHHpEjeMXJYDCQcFNrqISEBAwGg+22yWTi/PnzvP322wAkJyczZ84cxowZg9FopG7dunh5eQHQtGlTzpw5c9siOSQkhJCQENvtuz1J2B4nFleoAJ9/rqNbNyNt2mhZvz6eBx6wONRJzo6SxVFygONkuW2O7Gxc9uxBHx2N/quv0MbHo+r1mFq3xjR+PKaQENS/f55QVYrqzGRHeU3AcbLca44qVark+5gUyUKIYnH2rJYhQ3w5dMiZfv3SeOutaxTQpEYUgYCAAC5dusSVK1cwGAzs2bOHYcOG2R53c3Nj+fLltttTpkyhX79+BAQEUKlSJTZv3kxmZiY6nY7jx48THh5uj90oNnXqmP++hLUf3bsbWb8+AQc4CCZKk8xMXHbtyplKEReHJjkZq7s7mcHBZISFkfnMM6ju7vZOKYqAFMlCiCL3xReujB/vjVYLS5cmEhZmsnekckOr1TJw4EBmzJiB1WqldevWVK9enaioKAICAmjRokW+63p4eBAeHs748eNRFIWmTZvSrFmzEkxfMurVMxMVFU+PHjmF8o4dVtzu9/NvUaYpGRkoGzbg8/nn6LdtQ5OaitXLC1ObNpjCwzEFBSFHAcoeKZKFEEUmLU1h4kRv1q5147HHMvm//0umalWLvWOVO82aNctT3EZERNx22SlTpuS6HRQURFBQUHFFcxgNGphZvTqBnj2NtGunJSpKQ5UqVnvHEg5EuX4d/fbt6KOjcdm5E01GBhpfXzI6dsQUHk7mk09CKewhLgpPimQhRJE4csSJ117z5Y8/tIwceZ0RI65TilvsinKgceNsVq1KoHdvP3r08GPdunj8/aVQLs+U5GT0X3+N69atuHz7LUpWFpaKFcno0QPnXr24WrcuMrCVH/JOCyHui6rC0qXuzJzphdFoZc2aBFq2lN7HonRo1iybL780Ex6utV1wpGJFKZTLE01CAvrY2Jwjxrt3o5jNmKtUIe355zGFh5PVogVoNDmdExzgBDVRcqRIFkLcs/h4DSNH+rBjh5527TKYOzcZg0F6H4vSpWVLlU8+SaRPH4OtUPbzk0K5LNNcvow+JgbX6Gic9+1DsVox16yZ08M4LIzsJk2QPpVCimQhxD357jtnhg3z5do1DTNmJNO/v/Q+FqXX449nsXJlIv36GejZ08iaNQkYDFIolyXaCxdyrnq3dSvOP/0EQHatWqQOHUpGeDjmevWkMBa5SJEshLgr2dkwd64nkZEePPywmVWrEqhXT3ofi9IvMDCLjz5KZMAAIz17GomKiperQpZy2tOncd26FX10NM4//wxAdv36pIwejSk8HPONK8oIcRtSJAshCu3cOS2DB/vyv/8506dPGlOmpODmJkWEKDv+9a8sli9P5IUXDPTubeTzzxPw9pbv8VJDVdGdOGE7Yux0/DgAWU2bkjJxIhlhYVhq1rRvRlFqSJEshCiUTZv0jB3rg6LAokWJdOwovY9F2dSqVSbLliXy4osG+vQx8tlnCXh5SaHssFQVp19+ybnq3datOP3+O6qikPXoo1ybMgVTWBiWqlXtnVKUQlIkCyHuKD1dYfJkLz7/3J3mzbOIjEyienXpfSzKtuDgTJYsSeTllw307ZtTKHt4SKHsMKxWnP73v5ypFFu3ojt3DlWrJeuJJ0h+8UVMoaFYK1Wyd0pRykmRLITI1y+/6Bg82JfTp3UMG3adf/9beh+L8qNt20w+/DCJV1/1pV8/A59+moi7uxTKdmOx4Lx/P/qtW3GNjkZ7+TKqkxOZ//oXqcOGYWrXDqvBYO+UogyRX3dCiDxUFf77X3emT/fCYLASFZXAk09K72NR/oSFmYiMTGLIEF/69zewcmWizMMvSdnZKNu24b16NfrYWLTx8aguLphatcI0fjymNm1Qvb3tnVKUUVIkCyFySUzUMGqUD3FxekJCTCxYkCytsES51rGjCYslmaFDfRgwwMDHHyfg6mrvVA7MbEZJTUWTloaSmpr767Q0lLQ0NH/fb/v65vtvXi81FSU7G62bG5nPPENGeDiZwcGo7u723ktRDkiRLISw2b07p/dxYqKGqVOvMXBgmrQNFQLo0iUDsxlGjPDhxRcN/Pe/iej19k5VRMxmW2FqK2a1WvQXL+a9/+YCNz39n0L25uUyMwu1WVWrRfXwwOrujurujurhgerujsXPz3bb6uGB/umnudqsGfKXiShpUiQLITCbYc4cT95/34OHHjLz8ccJNGggvY+FuFm3bhlYLDBqlC8vv2xg2bJEXFzsEMRiyVu83vj6NvfbCtv8ilrT7TvV3Dq7V9VobIWs9aai1lyjRq6iVnVzy3nsRgF84+tb7kevL9TFO1zkctDCTqRIFqKcO39ey3PP6di715mePdOYNk16HwuRn4iIDMxmhTFjfHjlFQNLliTi7FzASlZrgUXtjUJWa7HgHR9/5ykI+RS1t1IV5Z+i9u//VXd3zNWq2e6/9UjujaLWq2pVkszmf47wenigFrKoFaKskCJZiHJsyxY9o0f7AAqRkUl06ZJh70hC2J/VajvqmmeqQVoaL1tTadY+i+9jzPyvdRIhTySgS/+nqM1ztDajcD9XqqKAhwf6G0dq//5nqVLlnyL3lqO1+d2venigurrec1Gr+vlhlqO3opyTIlmIcigjQ+Gtt7xYtcqdpk2z+Owz8PKSAlmUUlYrSkZG3qI2NRVNPsXuzUdydSYTFZOT/7k/Pb3ATQb//e/6WQ+yLnmgr+r2T7Hq74/5DsVrflMQVFdX/CpWJF6KUyEcQqGK5EOHDrFixQqsVivBwcF06dIl1+Px8fFERkaSlpaG1Wqld+/eNGvWLNfjI0eOpHv37nTq1Klo90AIcVeOH9fx2mu+nDql4/XXr/PGG9epXNlPpvyJkqOqkJaG5sqVvPNn09Pzdj646Shuno4IN+baqoWbImQrSm+agoC/P1k1ahQ4BSHPSWZubixZ5snbb3vTuWE677+fLH3EhShDCvxxtlqtLF++nEmTJmE0Ghk/fjwtWrSgWrVqtmXWr19Py5Ytadu2LRcuXGDWrFm5iuSPP/6Ypk2bFs8eCCEKRVXh44/dmDrVG29vK599lkBQkPQ+FiXPdd06nEeMwL8Qy1pdXfMUtVY/Pyw1axZuCsItnRPQaPJsw8/Pj+R7/Ctx0KA0zGaFGTO80Grh3XeT0Wrv6amEEA6mwCL51KlT+Pv7U+nvyzsGBgayf//+XEWyoiik//3xVHp6Or6+vrbHfvzxRypWrIiLXU4BFkIAJCUpvPGGD7GxrjzzTE7vYz8/6X0s7CO7cWPMM2eSCrefgnDTfNzSUHEOHpyK2QzvvOOFTgfz5iXfrhYXQpQyBRbJiYmJGI1G222j0cjJkydzLdO9e3emT59ObGwsmZmZTJ48GQCTycSmTZuYPHkymzdvLuLoQojC2LvXmddf9yU+XsNbb13jpZfS5Be4sCtz7dpYAwNJL0NzfIYNyymU583zQqdTeeeda/JzJkQpVySzp3bv3k2rVq3o2LEjJ06cYOHChcybN481a9YQHh6OvoCO69u2bWPbtm0AzJ49Gz8/v7vOoNPp7mm94uAoWRwlB0gWe+Qwm2HWLC0zZ2p48EHYtctMs2auQN6G/OXlNbkbjpRFlA4jR6ZiNiu8954nWi3MmnVNOqYJUYoVWCQbDAYSEhJstxMSEjAYcrcY37FjBxMmTACgdu3aZGdnc/36dU6dOsW+fftYtWoVaWlpKIqCs7MzoaGhudYPCQkhJCTEdvtezuz18/NzmDOCHSWLo+QAyVLSOf78U8vQoT7s26ele/d0pk+/hoeHmu/JeeXhNblb95qlSpUqxZBGlAaKAqNHX8dshshIT3Q6lWnTUqRQFqKUKrBIDggI4NKlS1y5cgWDwcCePXsYNmxYrmX8/Pz45ZdfaNWqFRcuXCA7OxsvLy+mTp1qW2bNmjXo9fo8BbIQomjFxOh54w0fsrNh4cIkunaV1m7lTUEdiW7Yu3cv8+fPZ9asWQQEBNjul45E905RYPz465jNCosXe6DVwpQpUigLURoVWCRrtVoGDhzIjBkzsFqttG7dmurVqxMVFUVAQAAtWrTg+eefZ/HixURHRwMwePBgFBkRhChRGRkwdao3K1e607hxFpGRSTz4oMXesUQJK0xHIoCMjAxiYmKoVatWnueQjkT3R1Fg8uQUzGZYtswDJyeYOFEKZSFKm0LNSW7WrFmulm4AERERtq+rVavGtGnT7vgcPXr0uId4QojC+O23nN7Hv/3mxGuvpTJmTErBl8oVZVJhOhIBREVF0blz5zwnVUtHoqKhKPD22ymYzQoffuiBVqsybtx1KZSFKEXk3FshSjFVhU8+cSMsrAIJCRpWrUpg0iQpkMuz23UkSkxMzLXM6dOniY+Pz3Pw40ZHou7du5dI1rJOUWD69Gv06ZPG//2fJ/Pmedo7khDiLsi1gYQopZKTFUaP9mHrVleeftrEe+8lU6GC9D4Wd2a1Wlm5ciWDBw/O81hhOxLB/XclcqTuIcWdZdkycHKysGCBJ97erowfn//PqaO8Lo6SAxwni6PkAMlSUjmkSBaiFPrxR2eGDPHhyhUtkydfY9Ag6X0schTUkchkMnH+/HnefvttAJKTk5kzZw5jxowpdEciuP+uRGWhk8ndmDoVUlN9mDLFjczMFF5/PdVuWQrDUXKA42RxlBwgWYoyx506EkmRLEQpYrHA++97MH++JzVqWNi0KZ4mTbLtHUs4kII6Erm5ubF8+XLb7SlTptCvXz8CAgKkI1Ex0mph/vxkLBaYNSvngiOvvppm71hCiDuQIlmIUuLiRQ3Dhvnyww8udO2azsyZ1/D0VO0dSziYwnQkEvah1cK77yZjNitMm+aNTgcvvSSFshCOSopkIUqBr77SM2qUD1lZ8O67SXTrliFnyYt8FdSR6GZTpky57f3Skah46HQ5/cstFnjrLW90OpUBA9LtHUsIcRsyi1EIB2YywaRJXgwcaKBaNTOxsVfp3l0KZCFKMycniIxMom3bDCZO9OGTT9zsHUkIcRtSJAvhoE6e1NGhQwVWrPBg0KBUNm+OJyBALg4iRFng7AyLFiURHGxi3DgfVq+WQlkIRyNFshAORlXhs8/cCA3148oVDStXJvDWWynItR2EKFtcXGDJkkRatTIxerQ3a9a42juSEOImUiQL4UCuXVN47TVfRo/2oUWLbOLirhIcnGnvWEKIYqLXw7JliTz1VBajRvmwerX8WhbCUchPoxAO4qefnGjXrgIxMXomTEhh9eoEKlWSi4MIUda5usKKFYm0bJnFwIFaNm0q+GIuQojiJ0WyEHZ2o/dx165+KAp88UU8Q4akysVBhChHXF1VPv44kcBAlaFDfYmOlkJZCHuTX8NC2NHlyxp69TLyzjtedOiQwVdfXaV5c7k4iBDlkZubysaNZpo2zWbwYF+++koKZSHsSYpkIewkOlohJKQCBw86MW9eEpGRyXh5ycVBhCjPPD3h008TaNgwm1de8SUuTs7YFcJepEgWooRlZsKbb3rRtasTlStbiY2Np2dP6X0shMjh6amyalUC9eplM2iQgZ07pVAWwh6kSBaiBJ06paVjxwosX+7BkCEWvvzyKg8/bLZ3LCGEg/H2VvnsswRq187mxRcN7NolhbIQJU2KZCFKgKpCVJQroaEVuHhRw4oVCcyfb0EvUw6FEPnw8VFZvTqBhx4y88ILBr7/3tnekYQoV6RIFqKYpaQovP66D6NG+dKkSU7v47ZtpfexEKJgBoNKVFQCDzxgZsAAA3v3SqEsREmRIlmIYnTwYE7v4y+/dGXMmBSiohKoXFl6HwshCs9otBIVlUC1ahb69TOwf78UykKUBCmShSgGVitERnrw7LN+WK2wfn08w4enotXaO5kQojSqUCGnUPb3t9Knj4EDB5zsHUmIMk+KZCGK2JUrGnr3NjJzphehoSa+/voqjz4qvY+FEPenUiUra9bEU6GClT59jBw6JIWyEMVJimQhitCOHS6EhFRg/34n5sxJZtGiJLy9pfexEKJoVK6cUyj7+lrp3dvIkSNSKAtRXKRIFqIIZGXB22970a+fkYoVrcTExNOnT7r0PhZCFLmqVa2sXZuAp6eVnj2NHD2qs3ckIcokKZKFuE+nT2vp3NmPJUs8GDAgjS1brlK7tvQ+FkIUn2rVLKxZk4Cbm5WICCPHj0uhLERRkyJZiPuwbl1O7+Nz53QsX57IjBnXpPexEKJEPPBATqHs4gIREUZOnJBCWYiiJEWyEPcgNVVh6FAfhg/3pWHDbL7++gqhoSZ7xxJClDMPPmhhzZp4tFro0cPIqVNSKAtRVKRIFuIuHT6c0/t440ZX3ngjhTVrEqhaVXofCyHsIyDAwtq1CahqTqF8+rT0mhSiKEiRLEQhWa2waJE7nTv7kZUF69cnMHKk9D4WjufQoUMMHz6coUOHsnHjxnyX27t3Lz169OD3338H4Oeff2bs2LH8+9//ZuzYsfzyyy8lFVncp4cfNrNmTQLZ2dC9ux9nz8rAJMT9kiJZiEK4elVDv34Gpk3zJiQkp/fxY49l2TuWEHlYrVaWL1/OhAkTWLBgAbt37+bChQt5lsvIyCAmJoZatWrZ7vP09GTs2LHMmzePIUOGsHDhwpKMLu5TnTpmoqISMJkUevQwcv68FMpC3A8pkoUowLffutCmTQX27nVh1qxkli5NwtdXeh8Lx3Tq1Cn8/f2pVKkSOp2OwMBA9u/fn2e5qKgoOnfujJPTP312H3zwQQwGAwDVq1cnKyuL7Gy5EE5pUq+emaioeFJTNXTvbuTPP6VQFuJeSZEsRD6ysmDGDE969zZiMFiJjr7K889L72Ph2BITEzEajbbbRqORxMTEXMucPn2a+Ph4mjVrlu/z7Nu3j4ceeihXES1KhwYNzKxencC1axp69DBy8aL8qhfiXshpsELcxtmzWoYM8eXQIWf69UvjrbdScHWVo8ei9LNaraxcuZLBgwfnu8z58+dZtWoVEydOzHeZbdu2sW3bNgBmz56Nn5/fXeXQ6XR3vU5xKYtZgoMhOtpCWJiOXr0qEReXTZUqJZ+jKDhKFkfJAZKlpHJIkSzELTZscGXcOG+0WliyJJHwcGntJkoPg8FAQkKC7XZCQoJtCgWAyWTi/PnzvP322wAkJyczZ84cxowZQ0BAAAkJCcydO5chQ4bg7++f73ZCQkIICQmx3Y6Pj7+rnH5+fne9TnEpq1keegg++cSJPn2MtGmjsHZtAhUrFq4TT1l9TcpCDpAsRZmjyh3+epTPYIT4W1qawogRPrz+ui9162YTF3dVCmRR6gQEBHDp0iWuXLmC2Wxmz549tGjRwva4m5sby5cvJzIyksjISGrVqmUrkNPS0pg9eza9e/fmkUceseNeiKLy6KPZfPJJIn/+qSUiwkh8vPzaF6Kw5KdFCODIkZzex+vXuzJy5HXWrUugalWLvWMJcde0Wi0DBw5kxowZjBw5kpYtW1K9enWioqL46aef7rhubGwsly9fZt26dYwePZrRo0dz7dq1Ekouisvjj2excmUi585p6dnTSGKi/OoXojBkuoUo11QVli1zZ8YML4xGK2vWJNCypbR2E6Vbs2bN8pyUFxERcdtlp0yZYvv6ueee47nnnivOaMJOAgOz+OijRAYMMNKzp5GoqHjp0iNEAeTPSVFuJSRoeP55A1OmePPMMybi4q5IgSyEKLP+kRxTNQAAIABJREFU9a8sli9P5ORJHb17G7l2TVr1CHEnUiSLcum775wJCanA7t0uzJiRzPLlSRgMclRFCFG2tWqVybJliRw/nnNCX0qKFMpC5EeKZFGuZGfD5MlaevUy4u1tZcuWqwwYIL2PhRDlR3BwJkuWJHLkiBN9+xpJTZUBUIjbkSJZlBvx8Rq6dvVjzhwtvXuns3VrPPXqme0dSwghSlzbtpksWpTEoUNO9OtnIC1NCmUhbiVFsigXkpMVevUycuyYjlWrzMyZcw03N5leIYQov9q3NxEZmcSBA870728gPV0KZSFuJkWyKPPS0hT69TNy6pSOFSuS6NatcM30hRCirOvY0cT77yezb58zAwYYyMiQQlmIG6RIFmWayQQvvGDg8GEnPvggiaCgTHtHEkIIh9KlSwYLFiSzZ48zAwf6YpJrKAkBSJEsyrDsbHj1VQO7d7uwYEEy7dvLyC+EELfTrVsG8+Yls2uXnpdfNpApxxOEkCJZlE0WC4wY4UNcnJ6ZM5N57rkMe0cSQgiHFhGRwZw5yezYoadXLx1Z0jZelHNyxT1R5qgqjBvnzcaNbkyadI3+/dPtHUkIIUqFPn3SMZv5f/buPC6qev/j+GsWVllnUHDPMFu0VMKrcc0yMck0zTK3n5ZaauKe5pKaa5KKZqbhQmrlglctvdfCIuuW0kKZmm1KqyaKzACyw8w5vz/IuSIoiDAzwOf5ePRozpxzZt7ncDh8POd7vl9mzfJDUfx5/fV0XFwcnUoIx5AryaJWUVWYP9+HbdvqMXFiFs8+m+PoSEIIUaM8+WQuK1ZYeP99DyIj/bFIT5mijqrQleSjR4+yadMmFEWhW7du9O3bt8T8tLQ01qxZQ05ODoqiMHjwYEJCQjh+/Dhbt27FYrGg1+sZOnQobdq0qZYNEQJg5UovNmzwYuTIbKZNy3J0HCGEqJEiIxUyMzOZP9+XCRNUXn01A73cexZ1TLmHvKIoxMbGMnv2bIxGIzNnziQ0NJQmTZrYltm9ezf33HMPDz74IGfOnGHJkiWEhITg7e3N9OnTMRgM/PnnnyxevJh169ZV6waJumvdunpER/swYEAu8+ZdlFH0hBDiBowalYPVCosW+aLTwSuvZKDTOTqVEPZTbpGcnJxMUFAQgYGBAISFhZGUlFSiSNZoNOTmFrf7zM3Nxd/fH4AWLVrYlmnatCmFhYUUFRXhIg2cRBXbutWTBQt86dUrj2XLMtBKQyIhhLhhzz6bQ1GRhpdf9kGvh+hoOb+KuqPcItlsNmM0Gm3TRqORU6dOlVimf//+LFq0iPj4eAoKCpgzZ06pz/nyyy+5+eabpUAWVW7vXnemT/flgQfyWb06Xa50CCFEFZowIRuLBaKjfdDrVV5+OVMKZVEnVEkLo8OHD3P//ffTu3dvTp48yerVq4mOjkb792/R6dOn2bp1Ky+88EKZ6yckJJCQkABAVFQUAQEB151Br9dXar3q4CxZnCUHVF+W//xHw4QJeu69V2XPHi0eHuV/h7PsF2fJAc6TxVlygHNlEcLRJk/OxmLRsGqVNzodLFmSKU3aRK1XbpFsMBgwmUy2aZPJhMFgKLHMwYMHmTVrFgCtWrWiqKiIrKwsfH19MZlMLF++nMjISIKCgsr8jvDwcMLDw23TaWlp170hAQEBlVqvOjhLFmfJAdWT5dAhV4YNM9KmTRHr15vIyVHJqUBnFs6yX5wlBzhPFmfJAZXP0qhRo2pII4RjaTQwbVoWFgusWeONXq+ycKE8+yFqt3JvmAQHB5OSkkJqaioWi4XExERCQ0NLLBMQEMCJEycAOHPmDEVFRfj4+JCTk0NUVBSDBw/mtttuq54tEHXSN9+4MHy4gRYtLLz1lglvb9XRkYQQolbTaGDmzCxGj85m0yYv5s3zQZVTr6jFyr2SrNPpGDFiBIsXL0ZRFLp27UrTpk2Ji4sjODiY0NBQhg0bxrp169i/fz8AY8eORaPREB8fz7lz59i1axe7du0CYPbs2fj6+lbvVola7fvv9QwdaqRBA4Vt20wYDHKWFkIIe9BoYM6ci1gssHGjFy4u8MILckVZ1E4VapMcEhJCSEhIifcGDBhge92kSRMWLlxYar3HHnuMxx577AYjCvE/yck6Bg824umpEhdnIjBQcXQkIYSoUzQamD//IhaLhtdf90KnU5kxI0sKZVHrSNfgosY4c0bHwIHFD1LFxaXRpInVwYmEEKJu0mhg0aJMLBZ47TVvXFxg6lQZwEnULlIkixrh/HktAwYYyc3V8K9/pREcLAWyEEI4klYLUVGZWK2wcmXxw3yTJmU7OpYQVUaKZOH0zGYNgwcbSU3VsmOHidatLY6OJIRTO3r0KJs2bUJRFLp160bfvn3LXO6LL75gxYoVLFmyhODgYADeeecdDh48iFarZfjw4bRr186e0UUNo9XC0qWZWCwali0rHnBk3DgplEXtIEWycGpZWRqGDjXy22963nzTxN13Fzk6khBOTVEUYmNjmT17NkajkZkzZxIaGlpilFSAvLw83n//fW655Rbbe2fOnCExMZEVK1aQnp7OwoULWbVqla3PeyHKotPBihUZWK2wZEnxgCNjxlSgP04hnJyc+YTTysvT8NRTBk6ccGHdOjOdOxc6OpIQTi85OZmgoCACAwPR6/WEhYWRlJRUarm4uDj69OlTYhTUpKQkwsLCcHFxoUGDBgQFBZGcnGzP+KKG0unglVcy6N07j4ULfdm4sZ6jIwlxw6RIFk6psBBGjfLnyy9defXVdLp3L3B0JCFqBLPZjNFotE0bjUbMZnOJZX799VfS0tJK9Vp05boGg6HUukJcjV4Pq1en07NnHi++6MvmzZ6OjiTEDZHmFsLpWCwQGenPwYPuLF+eQZ8++Y6OJEStoSgKb775JmPHjr2hz0lISCAhIQGAqKio6x7C25mG/ZYsVZsjLg4GD1Z44QU/fH29eOaZG+uqszbsk6omWeyTQ4pk4VQUBaZN8+O99zyYNy+TQYNyHR1JiBrFYDBgMpls0yaTCYPBYJvOz8/n9OnTzJ8/H4CMjAyWLl3K888/X2pds9lcYt3LhYeHEx4ebpu+3iG8a8MQ5NXBWbLcaI5VqyA318C4ce7k52fc0Lm8tuyTqiRZqi5Ho0aNrjpPmlsIp6Gq8OKLPuzc6cnUqRd55hl58EOI6xUcHExKSgqpqalYLBYSExMJDQ21zff09CQ2NpY1a9awZs0abrnlFp5//nnbCKqJiYkUFRWRmppKSkoKLVu2dODWiJrKzQ3Wrzdz//35TJvmy86dHo6OJMR1kyvJwmksXerNG294MXp0tvS1KUQl6XQ6RowYweLFi1EUha5du9K0aVPi4uJshfDVNG3alHvuuYcpU6ag1WoZOXKk9GwhKs3dHTZuNDN8uJEpU/zQ66FfvzxHxxKiwqRIFk5hzRovXn3VmyFDcpgz56IMbyrEDQgJCSn1UN6AAQPKXHbevHklpvv160e/fv2qK5qoYzw8YNMmM8OGGZg40Q+dTpXnTESNIZcIhMNt3uzJSy/50LdvLkuWZEqBLIQQtYiHh8qWLWY6dChk/Hh/9u93d3QkISpEimThULt2efDCC350757PK69koNM5OpEQQoiq5ump8uabZtq3L2LsWH8OHJBCWTg/KZKFw7z/vjtTpvjxz38WEBNj5rIxDYQQQtQyXl4qb79t4s47ixg92p8PP3RzdCQhrkmKZOEQ//2vG2PH+tO2bRGbNplxl4sKQghR63l7q2zbZuKOO4oYNcrAxx9LoSyclxTJwu6++sqVESP8adnSwltvmahXT3V0JCGEEHbi41NcKLdqVcTIkQY+/VQKZeGcpEgWdvXddy4MG2agcWMr27eb8POTAlkIIeoaPz+V7dtN3HyzheHDDRw65OroSEKUIkWysJuTJ/UMGmTA11dh+3YTAQE3NlSpEEKImstgUImLM9G8uYWnnjLwxRdSKAvnIkWysItff4VBg4y4uMCOHSYaN5YCWQgh6jqjUSEuzkSTJlaGDjWQlCSFsnAeUiSLapeSouWhh1zIz9ewfbuJFi2sjo4khBDCSdSvX1woBwUpDBli4JtvpKsj4RykSBbVymTSMmiQEZMJtm0zcdttFkdHEkII4WQCAxV27kyjfn2FIUOMHD0qhbJwPCmSRbXJzNQweLCB06d1vPuuhbZtixwdSQghhJNq2LC4UPb3Vxg82Mh330mhLBxLimRRLXJzNQwbZuTnn13YuDGdzp2lFwshhBDX1rixwr/+ZcLbW2HgQCPff693dCRRh0mRLKpcfj6MGGHgyBEX1qxJp2vXAkdHEkIIUUM0aWJl504Tnp4KAwYY+fZbjaMjiTpKimRRpYqKYOxYfz77zI3o6Awefjjf0ZGEEELUMM2bFxfKHh4q99+vZ8cOD0dHEnWQFMmiyigKTJnix4EDHixalMETT+Q5OpIQQogaqkULK++/n8Y996g895w/U6b4kSd/VoQdSZEsqoSqwqxZvuzZ48mMGRcZPjzX0ZGEEELUcAEBCvv3W5g4MYu4OE8eeaQ+v/2mc3QsUUdIkSxumKrC4sU+vPVWPcaNy2L8+GxHRxJCCFFL6HTw/PNZvPWWibNndTz0UH3ef9/d0bFEHSBFsrhhr77qxeuve/HkkznMmJHl6DhCCCFqoQceKODAgQsEB1t4+mkDCxf6UCQ9i4pqJEWyuCGxsfVYutSHxx7LZdGiTDTyELIQQohq0qSJlT170njyyRxiYrwYMMDIuXNSyojqIUeWqLS4OA/mzvXloYfyWLEiA60cTUIIIaqZmxu89FImr72WzvHjLvToUZ/Dh10dHUvUQlLWiEr597/dmTrVj/vuy2fNmnT00t+7EEIIO3r00Tzeey8NP7/igUdWr/ZCURydStQmUtqI6/bRR26MH+9PaGghGzem4+bm6ERCiMsdPXqUTZs2oSgK3bp1o2/fviXmf/DBBxw4cACtVou7uzujR4+mSZMmWCwWYmJi+O2331AUhS5duvDoo486aCuEKF+rVhb2709j2jQ/oqJ8+PprV1atSsfPT0Z5FTdOriSL6/L5566MGmXgttuK2LLFjKennIiEcCaKohAbG8usWbNYuXIlhw8f5syZMyWW6dy5M9HR0Sxbtow+ffqwZcsWAL744gssFgvR0dFERUWRkJBAamqqIzZDiArz8lJZuzadRYsy+O9/3YiIqM/x4y6OjiVqASmSRYV9+60LTz5poFkzC9u2mfHxkQJZCGeTnJxMUFAQgYGB6PV6wsLCSEpKKrGMp6en7XV+fj6ay564zc/Px2q1UlhYiF6vL7GsEM5Ko4Hhw3PZsycNqxX69Angrbc8UeXPlLgB0txCVMiPP+r5v/8zEhCgsH27CYNBGn4J4YzMZjNGo9E2bTQaOXXqVKnl4uPj2b9/PxaLhblz5wLQqVMnvv76a0aNGkVhYSFPPvkkXl5edssuxI0KCSniwIE0xo/3Y8YMP5KSXImKypS7nqJSpEgW5fr1Vx2DBhlxd1fZscNEUJAUyELUdBEREURERHDo0CF2797NuHHjSE5ORqvVsm7dOnJycpg7dy533nkngYGBpdZPSEggISEBgKioKAICAq7r+/V6/XWvU10ki/PmgOvPEhAA778PS5ZYWLjQg59+cmf7dgu33mrfHNVJstgnhxTJ4pr++kvLwIFGrFbYtctEs2ZWR0cSQlyDwWDAZDLZpk0mEwaD4arLh4WFsWHDBgAOHTpEu3bt0Ov1+Pr6cuutt/LLL7+UWSSHh4cTHh5um05LS7uunAEBAde9TnWRLM6bAyqfZfRouO02NyIj/QgL07N8eQa9e+fbPUd1kCxVl6NRo0ZXnSdtksVVXbigZeDAALKytGzfbqJlS4ujIwkhyhEcHExKSgqpqalYLBYSExMJDQ0tsUxKSort9ZEjR2jYsCFQ/EfmxIkTQHHb5FOnTtG4cWP7hReiit13X/Eofa1aWRgzxsDcuT4UFjo6lagp5EqyKFN6uoZBg4ykpBQXyG3aSIEsRE2g0+kYMWIEixcvRlEUunbtStOmTYmLiyM4OJjQ0FDi4+P57rvv0Ol0eHl5ERkZCRQ3wVi7di1TpkxBVVW6du1K8+bNHbxFQtyYxo0Vdu9OY9EiH2JjvTh61JWYGDONGknTQXFtUiSLUrKzNQwdauSXX/Rs2WKiQ4ciR0cSQlyHkJAQQkJCSrw3YMAA2+vhw4eXuZ67uztTpkyp1mxCOIKrKyxYcJHQ0EKmTvWjR4/6rFmTQZcuBY6OJpyYNLcQJeTlwfDhBo4fdyEmJp0uXeS+lBBCiNrhkUfyee+9C9SvrzB4sIGVK2WUPnF1UiQLm6IiGD3awOefu/LKKxn06FH5BxyEEEIIZ9SypZX//CeNRx/NY/lyH4YNM2A2a8pfUdQ5UiQLAKxWmDDBn48+cmfJkkz69ctzdCQhhBCiWnh6qrz6agZRURkcPuxGjx71+fZbGaVPlCRFskBVYfp0X/bt82DOnEyGDs11dCQhhBCiWmk0MHRoLu++m4ZWC48+GsDmzTJKn/gfKZLrOFWFefN82L69HpMmZTFmTI6jIwkhhBB207ZtEfHxF+jSpYAXXvAjMtKPnBxpfiGkSK7zVqzwZuNGL0aOzGbq1CxHxxFCCCHszt9fZfNmMzNmXOTf//bg4YcDOHVKOgCr6yp0BBw9epRNmzahKArdunWjb9++JeanpaWxZs0acnJyUBSFwYMH27ofeueddzh48CBarZbhw4fTrl27qt8KUSkxMfVYscKbgQNzmDfvIhr5h7MQQog6SquF8eOzad++kMhIf3r2DGDZskz69pVndOqqcq8kK4pCbGwss2bNYuXKlRw+fJgzZ86UWGb37t3cc889LF26lEmTJhEbGwvAmTNnSExMZMWKFbzwwgvExsaiSF8rTuHttz1ZuNCXXr3yWLo0E63cUxBCCCHo3LmQ+PgLtGlTRGSkP7Nm+VIg3SnXSeWWRsnJyQQFBREYGIherycsLIykpKQSy2g0GnJzix/2ys3Nxd/fH4CkpCTCwsJwcXGhQYMGBAUFkZycXA2bIa7Hu+96MGOGLw88kM/q1enodI5OJIQQQjiPhg0Vdu40MXp0Nlu21OOxxwL44w9HpxL2Vm6RbDabMRqNtmmj0YjZbC6xTP/+/fnss88YM2YMS5YsYcSIEWWuazAYSq0r7OuDD9yYMMGPTp0KWb/ejKuroxMJIYQQzsfFBebOvciGDWaSk/V06uTCwYNujo4l7KhKWqUfPnyY+++/n969e3Py5ElWr15NdHR0hddPSEggISEBgKioKAICAq47g16vr9R61cFZslyZ4+BBDWPG6AkJUfn3vzV4e9svo7PsE3CeLM6SA5wni7PkAOfKIoRwnJ4987nttguMHVufYcMMTJyYzZQpWXIXtg4ot0g2GAyYTCbbtMlkwmAwlFjm4MGDzJo1C4BWrVpRVFREVlZWqXXNZnOpdQHCw8MJDw+3TaelpV33hgQEBFRqvergLFkuz/H11y4MGmTk5pstbNqURkGBatc2Vs6yT8B5sjhLDnCeLM6SAyqfpVGjRtWQRgjhSDffbOWzzyyMHm3hlVe8+eYbV9asScdolOesarNym1sEBweTkpJCamoqFouFxMREQkNDSywTEBDAiRMngOKH9YqKivDx8SE0NJTExESKiopITU0lJSWFli1bVs+WiKs6cULPsGFGAgMVtm0z4e8vPaULIYQQ18PDA1asyCA6Op2kJFcefLA+SUkySl9tVu6VZJ1Ox4gRI1i8eDGKotC1a1eaNm1KXFwcwcHBhIaGMmzYMNatW8f+/fsBGDt2LBqNhqZNm3LPPfcwZcoUtFotI0eORCvdKNhVcrKewYON1KunEBdnokED+VevEEIIUVkDB+bRpk0Ro0YZePzxAGbPvsjTT+dIN6q1UIXaJIeEhNj6Pb5kwIABttdNmjRh4cKFZa7br18/+vXrdwMRRWX9/jsMGGBEq4W4OBONG1sdHUkIIYSo8dq0sfD++xeYPNmPefN8SUpyJTo6A29vuVNbm8hl3Vrq3DktEREu5Odr2LbNxM03S4EshBBCVBVfX5XY2HRmz84kPt6dnj3r8+OPMkpfbSJFci1kNmsZNMjIhQvw1lsm7rjD4uhIQgghRK2j0cCzz+awc6eJ7GwNvXoFsGuXh6NjiSoiRXItk5WlYcgQA3/+qWfPHgshIUWOjiSEEELUap06FXLgwAXaty9i4kR/nn/el/x8R6cSN0qK5FokL0/Dk08a+OEHF9atM3PffdI2SgghhLCHBg0UduwwMW5cFlu31qNv3wD+/FM6U67JpEiuJQoK4Omn/UlKcmX16nTCw2WgeSGEEMKe9HqYOTOLTZtM/PmnnoiI+nzwgYzSV1NJkVwLWCwwbpw/n3zizrJlGTzyiNzjEUIIIRzlwQcLeP/9CzRrZmH4cCNLlnhjkceDahwpkms4RYGpU/147z0P5s3LZODAPEdHEkIIIeq85s2tvPtuGkOG5PDaa94MHGgkNVXKrppE+iqpwVQV5s714V//8mTq1Is880yOoyMJIZzA0aNH2bRpE4qi0K1bN/r27Vti/gcffMCBAwfQarW4u7szevRomjRpAsAff/zB+vXrycvLQ6PRsGTJElxdXR2xGULUeO7usHRpJqGhhcyc6UtERH1efz2djh0LHR1NVIAUyTXYyy97s2mTF2PGZDNpUraj4wghnICiKMTGxjJ79myMRiMzZ84kNDTUVgQDdO7cmQcffBCAr7/+mi1btvDCCy9gtVpZvXo148aN46abbiIrKwu9Xv5MCHGjnngijzvvLOKZZwz0729k1qyLjB4to/Q5O7nuX0O99poXq1d7M2RIDrNnX5RfNCEEAMnJyQQFBREYGIherycsLIykpKQSy3h6etpe5+fno/n7BHLs2DGaNWvGTTfdBIC3tzdarfyZEKIq3H578Sh9PXrks3ChL08/7U9mpvzxdmZyiaAG2rzZkyVLfHj00VyWLMmUAlkIYWM2mzEajbZpo9HIqVOnSi0XHx/P/v37sVgszJ07F4CUlBQ0Gg2LFy/m4sWLhIWF0adPH7tlF6K28/ZWWb8+nY0bC1m0yIeePeuzbp2ZNm3kqT5nJEVyDfOvf3nwwgt+9OiRx8qVGeikC0YhRCVEREQQERHBoUOH2L17N+PGjcNqtfLTTz+xZMkS3NzcWLBgATfffDN33nlnqfUTEhJISEgAICoqioCAgOv6fr1ef93rVBfJ4rw5wHmyVGWOmTPhvvssDBmip0+f+qxaZeWppxSHZLlRzpKlOnJIkVyDvPeeO1Om+HHvvQWsXZuOi4ujEwkhnI3BYMBkMtmmTSYTBoPhqsuHhYWxYcMGoPiq8+23346Pjw8A7du357fffiuzSA4PDyc8PNw2nZaWdl05AwICrnud6iJZnDcHOE+Wqs7RqhW8/76WyEh/Ro924+DBXBYvzsCjAqNaO8s+AefJUtkcjRo1uuo8aWxWQ3zyiRtjx/oTElLEG2+YcXd3dCIhhDMKDg4mJSWF1NRULBYLiYmJhIaGllgmJSXF9vrIkSM0bNgQgLZt23L69GkKCgqwWq38+OOPJR74E0JUrYAAhW3bTEycmEVcnCePPFKf336TW8TOQq4k1wBffunKyJH+tGpl4c03TXh6ynDTQoiy6XQ6RowYweLFi1EUha5du9K0aVPi4uIIDg4mNDSU+Ph4vvvuO3Q6HV5eXkRGRgLg5eXFww8/zMyZM9FoNLRv356QkBAHb5EQtZtOB88/n0VoaCHjx/vz0EP1Wbkyg4cekoHBHE2KZCd37JgLw4YZaNLEyvbtJnx9pUAWQlxbSEhIqeJ2wIABttfDhw+/6rpdunShS5cu1ZZNCFG2Bx4o4MCBC4we7c/TTxsYMyabGTMuStNKB5LmFk7s55/1DBliwN9fYft2E0ZjxRv1CyGEEKJmadLEyp49aTz5ZA4xMV4MGGDk3Dkp1RxF9ryT+v13HYMGGXF1hR07TDRqJAWyEEIIUdu5ucFLL2Xy2mvpHD/uQkREfQ4fllEvHUGKZCd09qyWgQONFBbC9u0mbrrJ6uhIzkdVQVHAaoWiIigshIICyM+HvDw0eXlocnPR5OSgycpCc/EimsxMNFlZxesKIYQQTuzRR/N47700fH0VBg40snq1F4pcL7OrWtEm2e2//0X722945eT8r3hS1f/9B6CqaC5NX2X+dS1zjfk6Nzf8cnNLzr/yM65c/+/5hfkq5i9dWJsP/wgtwOcla5nrX+szLr3W63QEFBZedf6lbdBcbX5Flrksg+Ya87UaDYFWa/EyV1nf9jnXmG9b/wao9epRv3FjrE2aYL30/79fWxo3RgkMRDqgFkII4WitWlnYvz+NadP8iIry4euvXVm1Kh0n6Ja4TqgVRbL7e++hf/ttfCq4vKrRwDX+u9H5Wr0eV0W55jJlfY5V0XD+tJ6GhVqaNleolw1qzrU/46p5tFpwdUXVaqtkm250GXcPD/ILCq7vcy4Nh3u1ZcqZX+YyViteGRlYkpPRnTmD65EjaDMySh4fLi5YGzYsUUBbLi+oGzUqvh8mhBBCVDMvL5W1a9P5xz8KmD/fl4iI+uzcqdKsmaOT1X61okjOXLAAl1dfxWQ2V6gAq26V6dA6J0fDoEFGjv/hwua3zRjvLyC3CnKYnKCDbwCXgAAynSSLR0AA6Zdl0eTkoDtzBt1ff5X4v/7MGdw++wzt+fOlrmBbGzS4ehHdpAmqt7e9N0sIIUQtpdHA8OG5tG1bxOjR/tx3n56FCz0ZMiTXHmVNnVUrimTc3KBePdS8PEcnqZT8fBgxwsC337qwbl06999f4OhIdYparx6WW2/FcuutZS9QWIguJaW4gD5zBt3Zs7Yi2uW773A/cABNYWGJVRRf3+LmG5cXz5cV0YrcKxNCCHGdQkKKOHAgjSlTGjB9uh/qgEk4AAAgAElEQVRffeVKVFSmjJ9QTWpHkVyDFRXB2LH+HDrkxiuvpNOzp3Qe7nRcXbE2b461efOy5ysK2gsXbEW0/tIV6TNn0J8+jdvnn6PNyiqxiuruDk2bYmjYsGQBfen/DRuCXn49hRBClGQwKOzbZ2HOnHyio735/nsX1q0z07KlPORf1eSvsAMpCkye7MeBAx4sXpxB//4180p4nafVogQGogQGUnT33WUuosnMtDXl0P9dQHteuID2119x+eADdFc0RVG1WqxBQSUeKrS9/nta9fCwx9YJIYRwMlotTJ6cTUhIEZGRfjz8cH2WL8+gd2+50FaVpEh2EFWFmTN9eecdT2bOvMhTT91oC2ThzFRfXyy+vlhat+ZSYxrXy9uu5+WhO3u2xFXoS0W161dfoUtJQWMteZXAajCUWURfauKh+vnZpQ2+EEIIx7jvvuJR+saMMTBmjIGkpGxmz76Iq3SrXCWkSHYAVYVFi3x4++16jBuXxbhx2Y6OJBzNwwNrcDDW4OCy51ss6M6fL/mA4aUr0ydP4nbwINr8klcQlHr1/ldAX15E//1aCQz8Xw8iQgghaqTGjRV2705j0SIfYmO9OHrUlZgYswxCVgWkSHaAVau8iInxYvjwbGbMyCp/BSH0eluxWyZVRWs2l7oKfamgvmpXd40aYW3cGF3LlngHBBQX0NLVnRBC1CiurrBgwUVCQwuZOtWPHj3qs2ZNBl26SEcAN0KKZDvbuLEey5b50L9/LgsWXJS74aJqaDQoRiOK0UhR27ZlL5KdXeoq9KVeOrQffYTX2bMlurpTNRqUy7q6u7KbO2vjxtLVnRBCOJFHHsnnjjsuMGqUgcGDDTz3XBYTJ2bLTcNKkiLZjnbs8ODFF33p2TOP5csz5KAVdqV6eV21q7uAgADSzp4t2dXdZQ8Zuhw/jnt8/LW7uivjAUPFaJR20UIIYUctW1r5z3/SmD7dl+XLffjmG1defTUdg0G6ibteUiTbyb597kyb5kfXrvm89lq69O4lnM+NdHX355+4JSaizS7Zvl51d8faqNFVi2hrUJB0dSeEEFXM01Pl1Vcz+Mc/Cpk715cePeqzfn067dsXOTpajSJ/newgIcGN8eP96dChkA0b0qWZp6iZyuvqTlX/19Xd2bO2q9CXrkqX2dWdTve/ru4aN0bXqhWe/v7S1Z0QQtwgjQaGDs3lrruKGDXKn0cfDWDevEyefFJG6asoKZKr2eHDrowaZaB16yK2bDHj4SG3O0QtpdGg+vlh8fPD0qYNZT4ukpdX3IzjyrbRf3d1p927F78ru7ozGsu8Cn3pIUPp6k4IIa6ubdsi4uMvMHGiPy+84EdSkitLl2ZSr57UI+WRIrkaHTniwvDhBm66ycLbb5vw9pYDUtRxHh5YW7bE2rJlmbMD/PxI//770r10nDmD/uefy+/q7oq+oq1NmqA0aCBd3Qkh6jR/f5XNm82sWePF0qXFo/Rt2JDOLbdYHB3NqUmRXE1++EHP0KFG6tdX2L7dJA3mhaiIy7u669ix9PyrdXX3dy8dZXZ15+pa3NVdo0ZlFtHWRo2QnveFELWdVgvjx2fTvn0hkZH+9OwZwLJlmfTtK6P9Xo0UydXg5EkYPNiIu7vKjh0mAgOlQ28hqsQNdnXn9umnaM+fL93VXWDg/3rp+LtI19x3H7RoYa8tE0IIu+jcuZD4+AuMHetPZKQ/SUmuzJ2bKc9LlUGK5Cr21186HnvMBUVR2LXLRNOm1vJXEkJUmWt1dQdAYSG6s2dLtIe+9JCh67Fj6N57D01REdann4b58+0bXggh7KBhQ4WdO00sWeLDunVeHDvmQkxMOk2aSM1yOSmSq1BqqpYnnjBy8SLs3GmiZUtp6yOE03F1xXrTTVhvuqns+YqCNjUVg8Fg11hV6ejRo2zatAlFUejWrRt9+/YtMf+DDz7gwIEDaLVa3N3dGT16NE2aNLHNT0tLY/LkyfTv359HHnnE3vGFEHbg4gJz5xaP0jdlSvEofa+9lk7XrjJK3yXyNEsVSU/XMHiwkdRULfv2WWjTRgpkIWokrRYlKAgaNXJ0kkpRFIXY2FhmzZrFypUrOXz4MGfOnCmxTOfOnYmOjmbZsmX06dOHLVu2lJi/ZcsW2rdvb8/YQggH6dkzn/feu0DDhlaGDjWwbJk3VrmgDEiRXCWyszUMHWrkl1/0vPGGmU6d5CE9IYRjJCcnExQURGBgIHq9nrCwMJKSkkos4+npaXudn5+P5rIu9L766isaNGhQ4sqyEKJ2u/lmK//+9wWeeCKPV17x5v/+z4DJJCWi7IEblJcHTz1l4PhxF9atM3PvvYXlrySEENXEbDZjNBpt00ajEbPZXGq5+Ph4xo8fz9atWxk+fDhQXDDv3buX/v372y2vEMI5eHjAihUZREen89VXbjz4YH2SklwcHcuhpE3yDSgshNGjDXzxhSurV2fw4IPSjkcIUTNEREQQERHBoUOH2L17N+PGjWPnzp08/PDDuLu7l7t+QkICCQkJAERFRREQEHBd36/X6697neoiWZw3BzhPFmfJAdWbZdw4uPdeCwMG6Hn88QCWLLEyfrxy1TGbnGW/VEcOKZIryWqFCRP8+egjd15+OYNHH5V+BoUQjmcwGDCZTLZpk8l0zYcQw8LC2LBhA1DcVOPLL79k69at5OTkoNFocHV1JSIiotR64eHhhIeH26bTrhhyvDwBAQHXvU51kSzOmwOcJ4uz5IDqz9K4Mezfr2HyZD+mTfPgk0/yiI7OKHNQNGfZL5XN0egaz59IkVwJigLPP+/Lv//twZw5mfzf/+U6OpIQQgAQHBxMSkoKqX/30JGYmMiECRNKLJOSkkLDhg0BOHLkiO31ggULbMvs3LkTd3f3MgtkIUTt5+urEhubTkxMIUuW+PDjj/VZv97M7bfXnY4JpEi+TqoK8+b5sGNHPaZMyWLMmBxHRxJCCBudTseIESNYvHgxiqLQtWtXmjZtSlxcHMHBwYSGhhIfH893332HTqfDy8uLyMhIR8cWQjghjQaefTaH9u2LePZZf3r1CuDllzN5/PG6cfdciuTrFB3tTWysF888k82UKVmOjiOEEKWEhIQQEhJS4r0BAwbYXl96UO9annjiiSrPJYSomTp1KuTAgeJR+iZOLB6lb/78TCrw+EKNVqEiubyO6Tdv3sz3338PQGFhIZmZmWzevBmAt99+myNHjqCqKnfeeSfDhw8v0d1QTRITU4+VK70ZPDiHF1+8eNVG7EIIIYQQtUmDBgo7dphYtsyb117z5tgxF9avT8cJntmrNuUWyZc6pp89ezZGo5GZM2cSGhpaog/Np556yvb6/fff57fffgPg559/5ueff2b58uUAzJkzhx9++IHWrVtX8WZUv7fe8mThQl8eeSSPqKhMKZCFEEIIUafo9TBzZhZ3313IxIn+RETU5403rHTq5Ohk1aPcfpIr0jH95Q4fPkznzp0B0Gg0FBYWYrFYKCoqwmq14uvrW3Xp7WTPHg9mzvQlPDyfV19NR6dzdCIhhBBCCMd48MEC4uMv0KyZhccec2HJEm8stfB5vnKvJJfVMf2pU6fKXPbChQukpqbSpk0bAFq1akXr1q0ZNWoUqqoSERFR40ZxOnDAnUmT/LjnnkJiYsy41O1+tYUQokJUVSU/Px9FUcpsYnf+/HkKCpyjb/m6nEVVVbRaLe7u7jW2KaRwjObNrbz7bhqLFwfy2mveHDniytq16dSvrzg6WpWp0gf3Dh8+TKdOndBqiy9Qnzt3jr/++ouYmBgAFi5cyI8//sjtt99eYr0b7ZQeqqcT6Y8+0jBmjJ6771bZt0+Dt3fFPr82d6xdWZLFeXOA82RxlhzgXFlqovz8fFxcXNDry/4zo9fr0TnJbbm6nsVisZCfn4+Hh4ddv1fUfO7u8PrrVu688yIzZ/rSo0d9Xn89nY4da8fow+UWydfTMX1iYiIjR460TX/11VfccsstttGb2rdvz8mTJ0sVyTfaKT1UfWfWSUkuDBpkJDjYwhtvpFFQoFLRf9zX9I61q4Nkcd4c4DxZnCUHVE/H9HWJoihXLZCFc9Hr9U5zJV3UTE88kceddxbxzDMG+vc3MmvWRUaPzqnxz2+V2yb58o7pLRYLiYmJhIaGllrur7/+Iicnh1atWtneCwgI4Mcff8RqtWKxWPjhhx9o3Lhx1W5BNThxQs+wYUaCghS2bzfh7196hBkhhBBXJ7fuaxb5eYkbdfvtFt5//wI9euSzcKEvTz/tz8WLNfu4KrdIvrxj+smTJ3PPPffYOqb/+uuvbcsdPnyYsLCwEr9onTp1IjAwkKlTpzJt2jSaN29eZoHtTE6d0jNokBFvb4W4OFOtalsjhBB1gdlspnv37nTv3p127dpx991326YLC699G/jYsWPMmTOn3O945JFHqiRrYmIiw4YNq5LPEsLRvL1V1q9PZ968TBIS3HnoofqcOFFz7yhVKHl5HdND2R3Pa7VaRo0adQPx7OvPP3UMHGhEp4MdO0w0bmx1dCQhhBDXyWAw8OGHHwIQHR1NvXr1GDNmjG2+xWK5alOQtm3b0rZt23K/Y9++fVUTVohaRqOBZ57JoV27IsaM8adPn/osXpzBwIE1b5S+mlveV7Fz57QMHGgkP1/Drl1p3HyzFMhCCFFbTJo0CTc3N77//ntCQ0Pp06cPc+fOpaCgAA8PD6Kjo2nZsiWJiYnExMTw5ptvEh0dzV9//cWff/7JX3/9xdNPP2177uaWW27h1KlTJCYmsmLFCvz9/fn555+56667WL16NRqNho8++oj58+fj6elJhw4d+OOPP3jzzTevmjE9PZ2pU6fyxx9/4O7uztKlS7njjjv4/PPPmTt3LlDcLGLPnj3k5OTw7LPPkpWVhdVqZcmSJXTs2NEu+1KIiujQoXiUvshIf557rniUvkWLMqlJz4dKkQyYzVoGDTKSlqYlLs7E7bfXws7+hBDCQebO9eGHH0r2n6nRaFDVyj/vcccdRSxYcPG61klJSWHv3r3odDqysrJ455130Ov1HD58mJdffpkNGzaUWic5OZl//etf5OTkcO+99zJs2DBcrugL9MSJExw8eJCgoCD69OlDUlISd911F9OnT2fPnj00a9aMsWPHlpsvOjqaO++8k9jYWA4dOsTEiRP58MMPiYmJ4aWXXqJDhw7k5OTg5ubG22+/zX333cfEiROxWq3k5dW8q3Si9gsIUNi2zUR0tDerVnlz/Lgr69ebadGiZlyILLdNcm138aKGIUMM/Pmnns2bzbRvX+ToSEIIIapBr169bN2rXbx4kdGjR/PAAw8wd+5cfv755zLX6datG25ubhgMBgICArhw4UKpZdq1a0ejRo3QarW0bt2a06dPk5ycTPPmzWnWrBkAffv2LTffV199xeOPPw5A586dSU9PJysriw4dOjB//nxiY2PJzMxEr9fTrl07du7cSXR0ND/++CNeXl6V3S1CVCudDp5/Pou33jJx9qyOhx6qT3y8u6NjVUidvpKcl6fhyScN/PijC7GxZsLCake/fkII4UzKuuKr1+ux2HmILk9PT9vrZcuWERYWRmxsLGfPnuXRRx8tcx03Nzfba51Oh9Va+gqYq6triWWqervGjRtHt27dOHjwIH379mXbtm106tSJ3bt389FHHzF58mRGjRpF//79q/R7hahKDzxQwIEDFxg92p+RIw2MGZPNjBkXnXqQtjp7JbmgAEaO9Ofrr11ZvTqdbt2kj0ghhKgrsrKyCAoKAiAuLq7KPz84OJg//viD06dPAxV70K9jx47s2bMHKO71wmAw4O3tze+//87tt99OZGQkbdu2JTk5mTNnzlC/fn2GDBnC4MGD+e6776p8G4Soak2aWNmzJ40nn8whJsaLAQOMnDvnvKVonbySbLHAuHH+/Pe/7qxYkU7v3vmOjiSEEMKOnn32WSZNmsSqVavo3r17lX++h4cHL730EkOGDMHT07NCPWZMmTKFqVOnEh4ejru7O6+88goAGzduJDExEa1WS6tWrejatSt79+4lJiYGvV5PvXr1WLVqVZVvgxDVwc0NXnopkw4dCpk2zZeIiPqsWZPOP//pfHfzNeqNPDlRTc6ePXvd61R0dCxFgcmT/di1y5MFCzIZOTKnMhGrJEt1c5YcIFmcOQc4TxZnyQEy4t71uvK8nZubW6J5w5Uc0dziaqorS05ODvXq1UNVVWbNmkWLFi3K7RbVUfvlyp9XbfhdrK05oPZkOXlSzzPP+PPrr3qefz6LyMhstJW8sFwd52znvcZdDVQV5szxZdcuT55//mK1FMhCCCEEwNatW+nevTtdu3YlKyuLoUOHOjqSEE6lVSsL+/en0atXPlFRPgwfbiAjw3lG6atTzS2iorzZvLkeY8dmMWFCtqPjCCGEqMVGjRpVowbUEsIRvLxU1q5N5x//KGD+/OLmF+vXp3PXXY7vbazOXElevdqL117zZujQHGbNykKGqRdCCCGEcDyNBoYPz2XPnjSsVujTJ4C33/bE0Q2C60SRvGmTJ1FRPvTrl8tLL2VKgSyEEEII4WRCQoo4cCCNsLACpk/3Y+JEP3JzHVe01foieedOD2bP9iMiIo+VKzMq3SBcCCGEEEJUL4NB4a23zEydepE9ezzo3TuA5GSdQ7LU6pJx/353nnvOjy5d8lm7Nh19nWqBLYQQQghR82i1MHlyNlu3mjl/XsvDD9fnP/+x/yh9tbZI/vhjNyIj/QkJKSI2Np3LBk0SQoha7ejRo0ycOJHx48fz7rvvlpr/wQcf8NxzzzFt2jTmzJnDmTNnADh+/DjTp0/nueeeY/r06Zw4ccLe0avM448/zieffFLivQ0bNjBjxoxrrnPs2DEAhg4dSmZmZqlloqOjiYmJueZ3x8fHc/LkSdv0smXL+PTTT68jfdkSExMZNmzYDX+OEDXFffcVj9LXqpWF0aMNvPiiD4V27E65VhbJX3zhytNP+3PrrUW8+aYJT0+n6wpaCCGqhaIoxMbGMmvWLFauXMnhw4dtRfAlnTt3Jjo6mmXLltGnTx+2bNkCgLe3N9OnTyc6OprIyEhWr17tiE2oEn379mXv3r0l3tu7dy99+/at0PpvvfUWvr6+lfruK4vkadOm0aVLl0p9lhB1XePGCrt3pzFyZDYbN3rx+OMBnD1rn/K11hXJx4658OSTBpo0sbJtmxlfXymQhRB1R3JyMkFBQQQGBqLX6wkLCyMpKanEMpcPGpGfn4/m76eZW7RogcFgAKBp06YUFhZSVOT4bpgq4+GHH+ajjz6i8O/LTqdPn+b8+fN07NiRGTNm8NBDD9G1a1eWL19e5vodO3bEbDYDsGrVKjp37kzfvn355ZdfbMts3bqVnj17Eh4ezjPPPENeXh5JSUl8+OGHLFq0iO7du/P7778zadIk/vOf/wDw2Wef8eCDD9KtWzemTJlCQUGB7fuWL19OeHg43bp1Izk5+Zrbl56ezogRIwgPD6dXr1788MMPAHz++ed0796d7t278+CDD5Kdnc358+fp168f3bt354EHHuDLL7+8sZ0rhJ25usKCBRd5/XUzP/2kp0eP+nz6qWu1f2+taqX70096Bg82YjAo7NhhwmhUHB1JCCHsymw2YzQabdNGo5FTp06VWi4+Pp79+/djsViYO3duqflffvklN998My4uLjecyWfuXFz+LuIu0Wg03MiAr0V33MHFBQuuOt/f35927drx8ccf06NHD/bu3Uvv3r3RaDRMnz4df39/rFYrAwYM4Pvvv+fWW28t83OOHz/Ovn37+PDDD7FYLERERHDXXXcB8NBDDzFkyBAAXn75ZbZv386IESPo3r27rXi9XH5+PpMnTyYuLo7g4GAmTJjAm2++yTPPPAOAwWAgISGBjRs3EhMTc9UCHoqbfbRp04Y33niDQ4cOMXHiRD788ENiYmJ46aWX6NChAzk5Obi5ufH2229z3333MXHiRKxWK3l5ede1r4VwFo88ks8dd1xg1CgDgwcbee65LCZOrPwofeWpNUVycjIMGmTEzU1lxw4TDRtKgSyEEFcTERFBREQEhw4dYvfu3YwbN8427/Tp02zdupUXXnjhqusnJCSQkJAAQFRUFAEBASXmnz9/Hv3fT0trtVrb1erLlfVeRWm1WtvnX02/fv3Yt28fDz/8MPv27WPlypXo9Xree+893nrrLSwWC6mpqZw8eZLWrVuj0WjQ6XTo9Xrb66SkJHr27Im3tzcAPXr0sH13cnIyUVFRZGZmkpOTQ9euXdHr9Wi1WtvnXMqq0+n4448/aN68ua0gHzhwIJs2beLZZ59Fo9HQu3dvANq3b098fHyp7dPpdGg0GvR6PUlJScTGxqLX67n//vuZPHkyeXl5dOzYkfnz5/PYY4/x8MMP4+vry913382kSZNQFIWHHnqINm3alNpXbm5uJX6Ger2+1M/UUZwli7PkgLqdJSAAPv9cZdw4heXLfTh+3IvNmy3VkqNWFMlnz2p5/HEXiooU9uwx0by51dGRhBDCIQwGAyaTyTZtMplsTSjKEhYWxoYNG0osv3z5ciIjIwkKCrrqeuHh4YSHh9um09LSSswvKChApyvutilj3rxS6+v1eiwWS7nbc03lrN+9e3fmzp3Lt99+S25uLq1bt+bXX39l7dq17N+/Hz8/PyZNmkRBQQEWiwVVVbFarSVeK4qCoii2rJdPT5gwgdjYWFq3bk1cXByff/45FosFRVFsn3Npncs/99L7VqvVNq2qqm1/FW+apdT+uXL5y7/j0vtjx46la9euHDx4kF69erFt2zY6dOjA7t27+eijjxg/fjyjRo2if//+JT67oKCgxM8wICCg1M/UUZwli7PkAMkCsHQp3HWXJ3Pn+tKhg5a4OCstWlx/jkaNGl11Xq1ok7xokQ/p6bB9u5lWrW7wpCuEEDVYcHAwKSkppKamYrFYSExMJDQ0tMQyKSkpttdHjhyhYcOGAOTk5BAVFcXgwYO57bbb7Jq7OtSrV4+wsDCmTJlie2AvKysLDw8PfHx8uHDhAh9//PE1P6NTp04cOHCAvLw8srOz+fDDD23zsrOzCQwMpKioiHfeecf2vpeXFzk5OaU+Kzg4mNOnT/Pbb78BsHv3bjp16lSpbevYsSN79uwBinu9MBgMeHt78/vvv3P77bcTGRlJ27ZtSU5O5syZM9SvX58hQ4YwePBgvvvuu0p9pxDORKOBoUNzeffdNDQaGDJEX+U9X9SKK8lRUZlkZLjQrFnNfMBECCGqik6nY8SIESxevBhFUejatStNmza1tYMNDQ0lPj6e7777Dp1Oh5eXF5GRkUBxO+Vz586xa9cudu3aBcDs2bMr3cuDM+jbty8jR47k9ddfB6B169a0adOGLl260KhRIzp06HDN9e+880569+5N9+7dCQgIoF27drZ506ZNo1evXhiNRtq3b092djYAffr0Ydq0acTGxrJ+/Xrb8u7u7qxYsYLRo0djtVpp27YtQ4cOrdR2TZkyheeee47w8HDc3d155ZVXANi4cSOJiYlotVpatWpF165d2bt3LzExMej1eurVq8eqVasq9Z1COKO2bYuIj79Abm4ArlX8LJ9GvZEnJ6rJ2bNnr3sdufXgvDlAsjhzDnCeLM6SAyqf5Vq37mqzK8/bubm5JXrRuFKVNLeoIpKl9M+rNvwu1tYcIFmqMketb24hhBBCCCFEVZIiWQghhBBCiCtIkSyEEEIIIcQVpEgWQghR5ZzwcRdxDfLzEqI0KZKFEEJUOa1W6zQPw4lrs1gsaKtryDIharBa0QWcEEII5+Lu7k5+fj4FBQVljqzn5uZGQUGBA5KVVpezqKqKVqvF3d3dbt8pRE0hRbIQQogqp9Fo8PDwuOp8Z+k2CiSLEKJscn9FCCGEEEKIK0iRLIQQQgghxBWkSBZCCCGEEOIKTjkstRBCCCGEEI5Ua64kz5gxw9ERbJwli7PkAMlSFmfJAc6TxVlygHNlqY2caf9KltKcJQc4TxZnyQGSpSzVkaPWFMlCCCGEEEJUFSmShRBCCCGEuIJu3rx58xwdoqrcfPPNjo5g4yxZnCUHSJayOEsOcJ4szpIDnCtLbeRM+1eylOYsOcB5sjhLDpAsZanqHPLgnhBCCCGEEFeQ5hZCCCGEEEJcwemHpV67di1HjhzB19eX6OjoUvNVVWXTpk18++23uLm5MXbsWNvl9k8++YQ9e/YA0K9fP+6///5qzfLZZ5+xd+9eVFXFw8ODp59+mptuugmAyMhI3N3d0Wq16HQ6oqKiqi3H999/z9KlS2nQoAEAHTt25PHHHwfg6NGjbNq0CUVR6NatG3379q10jopk2bdvH5999hkAiqJw5swZYmNj8fLyqtJ9kpaWxpo1a8jIyECj0RAeHk7Pnj1LLGOvY6UiWexxrFQkh72OlYpksdexUlhYyIsvvojFYsFqtdKpUyeeeOKJEssUFRXx2muv8euvv+Lt7c2kSZNs++idd97h4MGDaLVahg8fTrt27SqdpTaSc3blstjrd1HO2ZXLYq9jxVnO23LO/pvq5L7//nv1l19+UadMmVLm/G+++UZdvHixqiiK+vPPP6szZ85UVVVVs7Ky1MjISDUrK6vE6+rM8tNPP9m+48iRI7YsqqqqY8eOVTMzM2/o+yua48SJE+qSJUtKvW+1WtVx48ap586dU4uKitSpU6eqp0+frtYsl0tKSlLnzZtnm67KfWI2m9VffvlFVVVVzc3NVSdMmFBq2+x1rFQkiz2OlYrksNexUpEsl6vOY0VRFDUvL09VVVUtKipSZ86cqf78888llomPj1fXrVunqqqqHjp0SF2xYoWqqqp6+vRpderUqWphYaF6/vx5ddy4carVaq2SXLWFnLMrl8Vev4tyzq5cFnsdK85y3pZzdjGnb25xxx134OXlddX5X3/9NV26dEGj0dCqVStycnJIT0/n6GA5mSoAAAVhSURBVNGj3HXXXXh5eeHl5cVdd93F0aNHqzXLrbfeapt/yy23YDKZbuj7KpvjapKTkwkKCiIwMBC9Xk9YWBhJSUl2y3L48GH++c9/3tD3XY2/v7/tCoOHhweNGzfGbDaXWMZex0pFstjjWKlIjqup6mPlerNU57Gi0Whwd3cHwGq1YrVa0Wg0JZb5+uuvbVemOnXqxIkTJ1BVlaSkJMLCwnBxcaFBgwYEBQWRnJxcLTlrKjlnVy7L1VT176KcsyuXxV7HirOct+WcXczpm1uUx2w2ExAQYJs2Go2YzWbMZjNGo9H2vsFgqPCBVhUOHjxI+/btS7y3ePFiALp37054eHi1fv/JkyeZNm0a/v7+DB06lKZNm5baJ0ajkVOnTlVrjksKCgo4evQoI0eOLPF+deyT1NRUfvvtN1q2bFnifUccK1fLcjl7HCvXymHvY6W8fWKPY0VRFKZPn865c+fo0aMHt9xyS4n5l2+/TqfD09OTrKwszGZziWXtfV6pDeScfXXOdN6Wc7Zjz9nlZbHnsVKXz9k1vkh2RidOnODjjz9mwYIFtvcWLlyIwWAgMzOTRYsW0ahRI+64445q+f4WLVqwdu1a3N3dOXLkCMuWLePVV1+tlu+qqG+++abEv8ShevZJfn4+0dHRPPXUU3h6et5o7GrPYo9j5Vo57H2sVGSf2ONY0Wq1LFu2jJycHJYvX86ff/5Js2bNKv15omZz9DkbnO+8Ledsx52zy8tiz2Olrp+znb65RXkMBgNpaWm2aZPJhMFgwGAwlLgdYjabMRgM1Z7njz/+YN26dUybNg1vb+8SOQF8fX3p0KFDtd6i9fT0tN2aCAkJwWq1cvHixVL75NK+sofDhw/TuXPnEu9V9T6xWCxER0dz77330rFjx1Lz7XmslJcF7HOslJfDnsdKRfYJ2OdYuaRevXq0bt261K3ay7ffarWSm5uLt7e3w84rtYmcs8vmbOdtOWeXZq9jxVnO23LOrgVFcmhoKJ9++imqqnLy5Ek8PT3x9/enXbt2HDt2jOzsbLKzszl27Fi1P4WelpbG8uXLGTduHI0aNbK9n5+fT15enu318ePHq/VfQBkZGah/d3+dnJyMoih4e3sTHBxMSkoKqampWCwWEhMTCQ0NrbYcl+Tm5vLDDz+U+K6q3ieqqhITE0Pjxo3p1atXmcvY61ipSBZ7HCsVyWGvY6UiWcA+x8rFixfJyckBip+aPn78OI0bNy6xzN13380nn3wCwBdffEHr1q3RaDSEhoaSmJhIUVERqamppKSkXPO2rChNztllc6bztpyzS7PXseIs5205Zxdz+sFEXnnlFX744QeysrL4/3buUEWVMAzj+B+FSYLRZpngDVgEyxQvQESwiNk0V+AVGBUsBqNFq1jMFtFmE6O3IDJzghzDnCJ7dodd9v/rAw/yzscz8vKVy2W63S6PxwOAVqtFmqbM53NOpxNBEDAcDgnDEHjuDa3Xa+B5RUwURV+aZTabsd/vXztUf689ud1ujMdj4PmF02w2abfbX5Zjs9mw3W4pFosEQUC/36dWqwFwOBxYLBYkSUIURf+V450s8Lyq53g8Esfx67nP/k3O5zOj0Yhqtfpa6O/1eq9/IfKclXey5DEr7+TIa1beyQL5zMr1emU6nZIkCWma0mg06HQ6LJdLwjCkXq9zv9+ZTCZcLhdKpRJxHFOpVABYrVbsdjsKhQKDweCf3cTfzjP7Y1nyehc9sz+WJa9Z+S7ntmf207cvyZIkSVLefvy6hSRJkvTZLMmSJElShiVZkiRJyrAkS5IkSRmWZEmSJCnDkixJkiRlWJIlSZKkDEuyJEmSlPEHk0aBisV2YIYAAAAASUVORK5CYII=\n","text/plain":["<Figure size 864x360 with 2 Axes>"]},"metadata":{"tags":[]}}]},{"cell_type":"markdown","metadata":{"id":"h9Yy3PSlXwKI"},"source":["So there's a couple of interesting things we can see here. Ussually you would see train and validation accuracy both increase over time. Essentially the point where training crosses validation you would want to stop training as it means that model is starting to remember exact things instead of learning patterns, which here is at 2nd epoch. Another thing to take into account is that validation accuracy decreases quite singnificantly over time, instead of growing. Why? Well we have quite a lot of data, so it might very well be that it's enough to have 2 epochs to fully train it before changing learning rate. Essentially, what this shows is that model is rapidly overtraining. If we would remove the dropout layers in the model see ```model.add(layers.Dropout(0.2))``` under [Modeling](#Modelling).\n","\n","**Essentially model is prone to overtrain**\n"]},{"cell_type":"markdown","metadata":{"id":"61h6y03n-6n5"},"source":["#Confussion matrix\n","Before jumping into other models it's always good to check how our model performs in different classes. For that we can use [confussion matrix](https://en.wikipedia.org/wiki/Confusion_matrix).\n","\n","It essentially shows how many times model predicted which class and how many of those times the class it slected was the right one. Here columns are true values and lines are predicted values."]},{"cell_type":"markdown","metadata":{"id":"1IUjrLPUPWEZ"},"source":["So here we need to do it a bit differently from the way we did it in the Logistic Regression notebook. We have encoded our classses using encoder. Here are those classes."]},{"cell_type":"code","metadata":{"id":"N_7cPm0IPWVG","executionInfo":{"status":"ok","timestamp":1602769598130,"user_tz":-180,"elapsed":1895,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"1c22d40a-8c09-4915-ea02-ca9a5d660183","colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["encoder.classes_"],"execution_count":17,"outputs":[{"output_type":"execute_result","data":{"text/plain":["array([0, 4])"]},"metadata":{"tags":[]},"execution_count":17}]},{"cell_type":"markdown","metadata":{"id":"YjRW_WnTaMFq"},"source":["And our predictions are stored as arrays with probabilities fir each class."]},{"cell_type":"code","metadata":{"id":"MN2AB5ntaUeX","executionInfo":{"status":"ok","timestamp":1602769599163,"user_tz":-180,"elapsed":2919,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"6ce2a575-c8e0-463a-b6f7-48fc5351c3b7","colab":{"base_uri":"https://localhost:8080/","height":137}},"source":["y_pred = model.predict(X_test)\n","y_pred"],"execution_count":18,"outputs":[{"output_type":"execute_result","data":{"text/plain":["array([[9.9973863e-01, 2.6138188e-04],\n"," [3.5664213e-03, 9.9643362e-01],\n"," [6.6429543e-01, 3.3570457e-01],\n"," ...,\n"," [6.8795025e-01, 3.1204978e-01],\n"," [1.8648592e-01, 8.1351405e-01],\n"," [1.0236888e-01, 8.9763111e-01]], dtype=float32)"]},"metadata":{"tags":[]},"execution_count":18}]},{"cell_type":"markdown","metadata":{"id":"dXjvX2umaYIc"},"source":["Here you can see that we have array of arrays of 2 probabilities for each class. So we need to turn those into classes."]},{"cell_type":"code","metadata":{"id":"y8bi8AFT-qlU","executionInfo":{"status":"ok","timestamp":1602769599164,"user_tz":-180,"elapsed":2912,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"53189057-37fc-4192-def2-19d0eef81e9d","colab":{"base_uri":"https://localhost:8080/","height":110}},"source":["y_predDecoded = [encoder.classes_[np.argmax(i)] for i in y_pred] # here we get the max probability from those arrays and then based on that select which class is it.\n","cm = confusion_matrix(y_test, y_predDecoded, labels=df.target.unique()) # same confusion matrix code as in Logistic Regression\n","df_cm = pd.DataFrame(cm, index=df.target.unique(), columns=df.target.unique())\n","df_cm"],"execution_count":19,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n"," .dataframe tbody tr th:only-of-type {\n"," vertical-align: middle;\n"," }\n","\n"," .dataframe tbody tr th {\n"," vertical-align: top;\n"," }\n","\n"," .dataframe thead th {\n"," text-align: right;\n"," }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n"," <thead>\n"," <tr style=\"text-align: right;\">\n"," <th></th>\n"," <th>0</th>\n"," <th>4</th>\n"," </tr>\n"," </thead>\n"," <tbody>\n"," <tr>\n"," <th>0</th>\n"," <td>12825</td>\n"," <td>3187</td>\n"," </tr>\n"," <tr>\n"," <th>4</th>\n"," <td>3694</td>\n"," <td>12294</td>\n"," </tr>\n"," </tbody>\n","</table>\n","</div>"],"text/plain":[" 0 4\n","0 12825 3187\n","4 3694 12294"]},"metadata":{"tags":[]},"execution_count":19}]},{"cell_type":"markdown","metadata":{"id":"6Z25Nn0yaOcu"},"source":["For a better understanding you could use percentage expression."]},{"cell_type":"code","metadata":{"id":"kzAdUYAW_F3X","executionInfo":{"status":"ok","timestamp":1602769599165,"user_tz":-180,"elapsed":2906,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"7eba20d7-8f2f-460f-ed71-569b6f5d2703","colab":{"base_uri":"https://localhost:8080/","height":110}},"source":["df_cm_percentage = df_cm.copy()\n","for i in df_cm_percentage:\n"," df_cm_percentage[i]/=df_cm_percentage[i].sum()\n","\n","df_cm_percentage"],"execution_count":20,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n"," .dataframe tbody tr th:only-of-type {\n"," vertical-align: middle;\n"," }\n","\n"," .dataframe tbody tr th {\n"," vertical-align: top;\n"," }\n","\n"," .dataframe thead th {\n"," text-align: right;\n"," }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n"," <thead>\n"," <tr style=\"text-align: right;\">\n"," <th></th>\n"," <th>0</th>\n"," <th>4</th>\n"," </tr>\n"," </thead>\n"," <tbody>\n"," <tr>\n"," <th>0</th>\n"," <td>0.776379</td>\n"," <td>0.205865</td>\n"," </tr>\n"," <tr>\n"," <th>4</th>\n"," <td>0.223621</td>\n"," <td>0.794135</td>\n"," </tr>\n"," </tbody>\n","</table>\n","</div>"],"text/plain":[" 0 4\n","0 0.776379 0.205865\n","4 0.223621 0.794135"]},"metadata":{"tags":[]},"execution_count":20}]},{"cell_type":"markdown","metadata":{"id":"OtcJlPxTa3i7"},"source":["As you can see the results are similar to the Logistic Regression."]},{"cell_type":"markdown","metadata":{"id":"0F3Ay2gNam_x"},"source":["Good, so in the end model learnt to classify both classes about the same. Even though we are using a simple and really overtrained by epoch 10 Neural Network as another base measure to check if the problem is solvable and what results we might expect some insights can be derived here (some the same as LR):\n","\n","1. Classes are evenly distributed, thus we won't overtrain on one class compared to the other and here we overtrain equally on both :D\n","2. Neural Network before overtraining and with no learning rate adjustments achieved 80% accuracy, meaning we should be able to get a slightly better result adjusting learning rate and a much better result using language models such as BERT\n","3. Both classes are predicted equally well\n","\n"]},{"cell_type":"markdown","metadata":{"id":"aQOP0Dx-BCSV"},"source":["# Test"]},{"cell_type":"markdown","metadata":{"id":"iievH-yxBKFb"},"source":["Now we need to check if it actually works. Let's just copy a comment in one of Trump's [tweets](https://twitter.com/realDonaldTrump/status/1315835556081868801). \n","\n","The comment is: \n","PATRIOTIC AMERICANS STAND PROUDLY WITH PRESIDENT TRUMP!!"]},{"cell_type":"code","metadata":{"id":"X2T115aw65gw","executionInfo":{"status":"ok","timestamp":1602772084512,"user_tz":-180,"elapsed":638,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"c7ec7889-1e63-4256-bfe9-645e6ffabde3","colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["tweet = 'People are fleeing California. Taxes too high, Crime too high, Brownouts too many, Lockdowns too severe. VOTE FOR TRUMP, WHAT THE HELL DO YOU HAVE TO LOSE!!!'\n","vectTweet = tokenizer.texts_to_sequences(np.array([tweet])) # vectorizes the tweet using our vectorizer\n","vectTweet = pad_sequences(vectTweet, padding='post', maxlen=maxlen) # adds padding\n","\n","prediction = model.predict(vectTweet) # predicts class of the tweet\n","print('Tweet is', 'positive' if encoder.classes_[np.argmax(prediction)]==4 else 'negative')"],"execution_count":26,"outputs":[{"output_type":"stream","text":["Tweet is negative\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"fWjPG6n7CPjC"},"source":["Good, so our model is right here. Let's just make sure our model works okay with very specific sentiments."]},{"cell_type":"code","metadata":{"id":"zsKXV4noBvp-","executionInfo":{"status":"ok","timestamp":1602769599401,"user_tz":-180,"elapsed":3120,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}},"outputId":"ea864c0d-1320-47d6-95ab-a1f22c05b054","colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["tweetList = ['Best tweet ever!', 'Mondays are the worst...']\n","vectTweet = tokenizer.texts_to_sequences(np.array(tweetList)) # vectorizes the tweet using our vectorizer\n","vectTweet = pad_sequences(vectTweet, padding='post', maxlen=maxlen) # adds padding\n","\n","prediction = model.predict(vectTweet) # predicts class of the tweet\n","for enum, i in enumerate(tweetList):\n"," print(i, '| This tweet is', 'positive' if encoder.classes_[np.argmax(prediction[enum])]==4 else 'negative')"],"execution_count":22,"outputs":[{"output_type":"stream","text":["Best tweet ever! | This tweet is positive\n","Mondays are the worst... | This tweet is negative\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"C6PYQxT2C5Ph","executionInfo":{"status":"ok","timestamp":1602769599403,"user_tz":-180,"elapsed":3113,"user":{"displayName":"Adam Ling","photoUrl":"","userId":"04744848912553865948"}}},"source":[""],"execution_count":22,"outputs":[]}]}