Milestone_2_Supervised_Text_Difficulty_Prediction_Final.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Milestone_2_Supervised_Text_Difficulty_Prediction_Final.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "machine_shape": "hm"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.8"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NHWShknvIQiT"
      },
      "source": [
        "# Milestone 2 Supervised Portion - Text Difficulty Classification\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yq3BQiqsMof1"
      },
      "source": [
        "## Formal Project Description\n",
        "In many real-world applications, there is a need to make sure textual information is comprehensible/readable by audiences who may not have high reading proficiency. This might include students, children, adults with learning/reading difficulties, and those who have English as a second language. The Simple English Wikipedia (https://en.wikipedia.org/wiki/Simple_English_Wikipedia), for example, was created exactly for this purpose. Before the editors spend a lot of effort to simplify the text and increase its readability, it would be very useful to suggest to them which parts of an article's text might need to be simplified.\n",
        "\n",
        "This notebook develops and trains a Multinomial Naive Bayes classifier, a logistic regression model, and gradient boosted decision trees classifier to address the challenge of binary text classification. \n",
        "\n",
        "Sentences from Wikipedia articles are classified into one of two categories:\n",
        "\n",
        "0: the sentence does NOT need to be simplified.\n",
        "\n",
        "1: the sentence DOES need to be simplified.\n",
        "\n",
        "\n",
        "The training data contains 416,768 sentences, already labeled with one of the above categories. The test data contains 119,092 comments that are unlabeled. (The test data is strictly used for the class pseudo competition.)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S7WT-Lp7XxIY"
      },
      "source": [
        "## 1.) Load & Clean the Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_7aTK_N-Ss54"
      },
      "source": [
        "import warnings\n",
        "warnings.filterwarnings('ignore')\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.datasets import load_digits"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kAXZ74mtOr48"
      },
      "source": [
        "#Import the datasets\n",
        "# File descriptions\n",
        "#   WikiLarge_Train.csv - the training set\n",
        "#   WikiLarge_Test.csv - the test set (without labels)\n",
        "\n",
        "# Data fields in training/test sets\n",
        "#   id - a unique identifier for each example. This should match with the Id field of the submission file. It should be in the range 0-119,091.\n",
        "#   text - the actual sentence of each training/test example. It's free-form text.\n",
        "#   label - the class label of a training example: either 0 or 1.\n",
        "\n",
        "wikilarge_train = pd.read_csv('assets/WikiLarge_Train.csv', encoding= 'unicode_escape')\n",
        "#wikilarge_test = pd.read_csv('assets/WikiLarge_Test.csv', encoding= 'unicode_escape') For Kaggle-esque competition only"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oi6pBOUfrUwB",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        },
        "outputId": "11e285c0-1df5-4137-82a8-c39cc590b541"
      },
      "source": [
        "wikilarge_train.head()"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>original_text</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>There is manuscript evidence that Austen conti...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>In a remarkable comparative analysis , Mandaea...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Before Persephone was released to Hermes , who...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Cogeneration plants are commonly found in dist...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Geneva -LRB- , ; , ; , ; ; -RRB- is the second...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                       original_text  label\n",
              "0  There is manuscript evidence that Austen conti...      1\n",
              "1  In a remarkable comparative analysis , Mandaea...      1\n",
              "2  Before Persephone was released to Hermes , who...      1\n",
              "3  Cogeneration plants are commonly found in dist...      1\n",
              "4  Geneva -LRB- , ; , ; , ; ; -RRB- is the second...      1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3asiPMysrUwD"
      },
      "source": [
        "## 2.) Generate Features\n",
        "\n",
        "External features I will get cover age of acquisition, concreteness, Dale Chall readability, parts of speech, syllables, if-idf, and Word2Vec Representations."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mRRx6AM2rUwD"
      },
      "source": [
        "### Age of Acquisition\n",
        "\n",
        "The AoA_51715_words.csv contains \"Age of Acquisition\" (AoA) estimates for about 51k English words, which refers to the approximate age (in years) when a word was learned. Early words, being more basic, have lower average AoA, so I hypothesize that if you average the AOAs of all words, easier documents to read will have lower numbers. (AoA_kup_lem is the only one without empty rows). \n",
        "\n",
        "The 51715 words that have values for AoA are stored in the dictionary, aoa_dict. (as well as syllable counts, we'll get to that later)\n",
        "Ultimately, the percent of applicable words that have aoa values under 5, 10, 15, 20, or more will be represented as a feature, as well as the average AoA value."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-36bWSExUF0r"
      },
      "source": [
        "aoa = pd.read_csv('assets/AoA_51715_words.csv', encoding= 'unicode_escape')\n",
        "#Remove Na\n",
        "aoa = aoa[['Word', 'AoA_Kup_lem', 'Nsyll']].dropna()\n",
        "#Word as the key, [AoA, num_syllables] as the values\n",
        "aoa_dict = dict(zip(aoa.Word, zip(aoa.AoA_Kup_lem.astype(float), aoa.Nsyll.astype(float))))"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3WggSKGBrUwE"
      },
      "source": [
        "### Extract Concreteness Ratings\n",
        "Concreteness evaluates the degree to which the concept denoted by a word refers to a\n",
        "perceptible entity. The variable came to the foreground in Paivio’s dual-coding theory (Paivio,\n",
        "1971, 2013). According to this theory, concrete words are easier to remember than abstract\n",
        "words, because they activate perceptual memory codes in addition to verbal codes. \n",
        "\n",
        "A concrete word comes with a higher rating and refers to something that exists in reality; you\n",
        "can have immediate experience of it through your senses (smelling, tasting, touching, hearing,\n",
        "seeing) and the actions you do. Hence, having more words with a higher concreteness value would make a sentence simpler. \n",
        "\n",
        "Not all words have a concreteness value, so the feature that will be represented is the average of all the words that do."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8xaBCxb9rUwF"
      },
      "source": [
        "concreteness = pd.read_csv('assets/Concreteness_ratings_Brysbaert_et_al_BRM.txt', delimiter = \"\\t\")\n",
        "conc_dict = dict(zip(concreteness.Word, list(zip(concreteness['Conc.M'].astype(float), concreteness['Percent_known'].astype(float), concreteness['SUBTLEX'].astype(float)))))"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sGZ0Ly3zrUwF"
      },
      "source": [
        "### Dale Chall Readability\n",
        "The Dale-Chall Word List contains approximately three thousand familiar words that are known in reading by at least 80 percent of the children in Grade 5. It gives a significant correlation with reading difficulty. It is not intended as a list of the most important words for children or adults. It includes words that are relatively unimportant and excludes some important ones.\n",
        "\n",
        "Dale Chall Readability is represented as the percentage of words in the given text that is in the Dale Chall list."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fjCnJ_5RrUwF"
      },
      "source": [
        "dale_chall_list = pd.read_csv('assets/dale_chall.txt').values.flatten().tolist()"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4v6__MDMrUwF"
      },
      "source": [
        "### Syllable Counts\n",
        "\n",
        "Syllables are the phonological building blocks of words and are single units of speech, either as a whole word or one of the parts into which a word can be separated. They sually contain a vowel. Naturally, words with more syllables are more complex.\n",
        "\n",
        "For this data, the percentages of words that have one through four syllables and more than 5 syllables are represented, as well as the average syllable count for the sentence.\n",
        "\n",
        "The NLTK library provides the most accurate means of getting syllable count; however, using NLTK is inefficient, so I opt to first check the AOA dictionary first to see if syllable counts are there. Lastly, if a syllable is not even accounted for, the get_syllables_last_resort function algorithmically determines the syllable count based off of vowels, although this method can be inaccurate for exceptional words."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RemgL97erUwG"
      },
      "source": [
        "import nltk\n",
        "from nltk.data import load\n",
        "from nltk.corpus import cmudict\n",
        "import syllables\n",
        "\n",
        "#Carnegie Mellon Dictionary is the best resource for getting syllables, but it demands a for loop\n",
        "best_syllables = cmudict.dict()\n",
        "\n",
        "def count_syllables(word_s):\n",
        "    #The Carnegie Mellon dict is the best way to get syllables that aren't readily available in the AoA dict\n",
        "    count = 0\n",
        "    phones = best_syllables[word_s] # this returns a list of matching phonetic rep's\n",
        "    if phones:                   # if the list isn't empty (the word was found)\n",
        "        phones0 = phones[0]      #     process the first\n",
        "        count = len([p for p in phones0 if p[-1].isdigit()]) # count the vowels\n",
        "    return count\n",
        "\n",
        "def get_syllables_last_resort(word):\n",
        "    #algorithm shamelessly stolen from https://medium.com/@mholtzscher/programmatically-counting-syllables-ca760435fab4\n",
        "    # '60% of the time, it works every time' - Only use for words not in CMU or AoA dictionaries\n",
        "    syllable_count = 0\n",
        "    vowels = 'aeiouy'\n",
        "    if word[0] in vowels:\n",
        "        syllable_count += 1\n",
        "    for index in range(1, len(word)):\n",
        "        if word[index] in vowels and word[index - 1] not in vowels:\n",
        "            syllable_count += 1\n",
        "    if word.endswith('e'):\n",
        "        syllable_count -= 1\n",
        "    if word.endswith('le') and len(word) > 2 and word[-3] not in vowels:\n",
        "        syllable_count += 1\n",
        "    if syllable_count == 0:\n",
        "        syllable_count += 1"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BHgKUqoOrUwH"
      },
      "source": [
        "### Additional Features and Data Extraction (Length of Text, Word Counts, Word Length, Recognizability, and Parts of Speech)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DKeBU50CrUwH"
      },
      "source": [
        "The extract features function mines the Wikipedia documents for age of acquisition, concreateness, dale chall readability, syllable groups. Additionally, it finds the length of the text, the average word length, the percent of recognizable words (found in concreteness df), and percentage of words for each part of speech. Parts of Speech are the categories to which a word is assigned in accordance with its syntactic functions. They are essential for breaking down the structure of text, so I figured it wouldn't hurt to add them in as features.\n",
        "\n",
        "This is a gnarly function that takes between 20-30 minutes to run, so future edits to this notebook include optimizing this function, breaking it up, or accomplishing the same thing with a task."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cgTp-DHjrUwH"
      },
      "source": [
        "from nltk import pos_tag\n",
        "from nltk.tokenize import sent_tokenize, word_tokenize\n",
        "from nltk.stem import WordNetLemmatizer \n",
        "from collections import Counter, defaultdict\n",
        "\n",
        "\n",
        "lemmatizer = WordNetLemmatizer()\n",
        "\n",
        "\n",
        "def extract_features(df):\n",
        "    avg_aoa_scores = []\n",
        "    concreteness = []\n",
        "    percent_dale_chall = []\n",
        "    word_count = [] \n",
        "    avg_word_length = [] \n",
        "    avg_syllables = [] \n",
        "    syllable_groups = {'1 Syll':[], '2 Syll':[], '3 Syll':[],'4 Syll':[], '>=5 Syll':[]}\n",
        "    avg_percent_recognized = []\n",
        "    aoa_total_groups = {'aoa<=5':[],'aoa<=10':[],'aoa<=15':[],'aoa<=20':[],'aoa>20':[]}\n",
        "    \n",
        "    #Create dictionary to hold percentages of all NLTK parts of speech for each entry\n",
        "    possible_pos = ['LS', 'TO', 'VBN', 'WP', 'UH', 'VBG', 'JJ', 'VBZ', '--', 'VBP', 'NN', 'DT', 'PRP', ':', 'WP$', 'NNPS', 'PRP$', 'WDT', '(',\n",
        "                    ')', '.', ',', '``', '$', 'RB', 'RBR', 'RBS', 'VBD', 'IN', 'FW', 'RP', 'JJR', 'JJS', 'PDT', 'MD', 'VB', 'WRB', 'NNP', 'EX', 'NNS', \n",
        "                    'SYM', 'CC', 'CD', 'POS']                                             #Found from load('help/tagsets/upenn_tagset.pickle').keys()\n",
        "    total_pos = defaultdict(list)\n",
        "    for pos in possible_pos:\n",
        "        total_pos[pos] = []\n",
        "        \n",
        "    #Use NLTK word_tokenize to get tokenized text, as well as lemmatized text\n",
        "    for text in df['original_text']:\n",
        "        tokens = [word.lower() for word in word_tokenize(text)]\n",
        "        len_tokens = float(len(tokens))\n",
        "        #Append number of words to word_count\n",
        "        word_count.append(len_tokens)\n",
        "        #lemmas are used for concreteness\n",
        "        \n",
        "        #Extract POS\n",
        "        text_pos_count = Counter() #Create dictionary\n",
        "        # create key for every pos, so that AT LEAST 0 is returned to the corpus list\n",
        "        for pos in possible_pos:    \n",
        "            text_pos_count[pos] = 0.0\n",
        "        #Use NLTK pos_tag to extract POS\n",
        "        for word, pos in pos_tag(tokens):\n",
        "            if pos in possible_pos: #Some useless POS may exist in new text\n",
        "                text_pos_count[pos] += 1\n",
        "        #Append the percentage for each POS in this entry to the total pos_dictionary\n",
        "        for key, val in text_pos_count.items():\n",
        "            val_normalized = val / len_tokens\n",
        "            total_pos[key].append(val_normalized)\n",
        "        \n",
        "        lemmas = []\n",
        "        num_dale_chall = 0.0\n",
        "        word_lengths = []\n",
        "        syllables = []\n",
        "        for token in tokens:\n",
        "            word_lengths.append(len(token))\n",
        "            #Use AoA to get syllable length (second value in list)\n",
        "            if token not in aoa_dict.keys():\n",
        "                try:\n",
        "                    syllables.append(float(count_syllables(token)))\n",
        "                except:\n",
        "                    try:\n",
        "                        syllables.append(float(get_syllables_last_resort(token))) #Use the NLTK syllable extractor for unique situations\n",
        "                    except:\n",
        "                        syllables.append(1) #Absolute worst case scenario, nothing can find syllables\n",
        "            else:\n",
        "                  syllables.append(float(aoa_dict[token][1]))\n",
        "            \n",
        "            #lemmas need to be found, as that's what the conc dictionary uses\n",
        "            lemma = lemmatizer.lemmatize(token)\n",
        "            lemmas.append(lemmatizer.lemmatize(token))\n",
        "            #If token is in Dale Chall list, add one to num_dale_chall (Making assumption that numbers add to readability)\n",
        "            if token in dale_chall_list or token.isnumeric():\n",
        "                num_dale_chall += 1\n",
        "                \n",
        "        #Calculate mean word length\n",
        "        avg_word_length.append(np.mean(word_lengths))\n",
        "        #Find counts of words with each syllable... \n",
        "        if len(syllables) > 0:\n",
        "            #Get avg syllable count\n",
        "            avg_syllables.append(np.mean(syllables))\n",
        "            #Add counts \n",
        "            sub_syllable_groups = {'1 Syll':0, '2 Syll':0, '3 Syll':0,'4 Syll':0, '>=5 Syll':0}\n",
        "            for syll in syllables:\n",
        "                if syll == 1:\n",
        "                    sub_syllable_groups['1 Syll'] +=1\n",
        "                elif syll == 2:\n",
        "                    sub_syllable_groups['2 Syll'] +=1\n",
        "                elif syll == 3:\n",
        "                    sub_syllable_groups['3 Syll'] +=1\n",
        "                elif syll == 4:\n",
        "                    sub_syllable_groups['4 Syll'] +=1\n",
        "                else:\n",
        "                    sub_syllable_groups['>=5 Syll'] +=1\n",
        "            #Add percentage of words falling in each age of acquisition group back into list for text\n",
        "            [syllable_groups[key].append(val / len(syllables)) for key, val in sub_syllable_groups.items()]\n",
        "        else:\n",
        "            #Add zeros to percentage of syllables\n",
        "            [syllable_groups[key].append(0) for key in syllables.keys()]\n",
        "            avg_syllables.append(0)\n",
        "\n",
        "        #Get Aoa Scores for each lemma, and find the average... If there are none, pass, and pray outliers won't trow off models too much\n",
        "        #Get avg concreteness scores, subtlex, and percent recognized, too at the same time :)\n",
        "        aoas = []\n",
        "        conc = []\n",
        "        subtlex_list = []\n",
        "        percents_known_list = []\n",
        "        for lemma in lemmas:\n",
        "            try:\n",
        "                #Get aoa score\n",
        "                aoas.append(aoa_dict[lemma][0])\n",
        "            except:\n",
        "                pass\n",
        "            try:\n",
        "                #concreteness df doesn't have empty rows, so we're clear to do this in the same, try/except\n",
        "                conc.append(conc_dict[lemma][0])\n",
        "                percents_known_list.append(conc_dict[lemma][1])\n",
        "            except:\n",
        "                if lemma.isnumeric(): #account for numbers that are easy to read\n",
        "                    conc.append(5)\n",
        "                    percents_known_list.append(.99) #I bet near 100% know a number\n",
        "                else:\n",
        "                    pass\n",
        "            \n",
        "        #Find average aoa and append to list for each text entry\n",
        "        sub_aoa_groups = {'aoa<=5':0,'aoa<=10':0,'aoa<=15':0,'aoa<=20':0,'aoa>20':0}\n",
        "        if len(aoas) > 0:\n",
        "            avg_aoa_scores.append(np.mean(aoas))\n",
        "            #Add counts \n",
        "            for aoa in aoas:\n",
        "                if aoa <= 5:\n",
        "                    sub_aoa_groups['aoa<=5'] +=1\n",
        "                elif aoa <= 10:\n",
        "                    sub_aoa_groups['aoa<=10'] +=1\n",
        "                elif aoa > 10 and aoa <= 15:\n",
        "                    sub_aoa_groups['aoa<=15'] +=1\n",
        "                elif aoa > 15 and aoa <= 20:\n",
        "                    sub_aoa_groups['aoa<=20'] +=1\n",
        "                else:\n",
        "                    sub_aoa_groups['aoa>20'] +=1\n",
        "            #Add percentage of words falling in each age of acquisition group back into list for text\n",
        "            [aoa_total_groups[key].append(val / len(aoas)) for key, val in sub_aoa_groups.items()]\n",
        "        else:\n",
        "            avg_aoa_scores.append(0) #Usually weird words... or non-words... should be labeled as easy\n",
        "            #Add zeros to percentage of word AOAs\n",
        "            [aoa_total_groups[key].append(0) for key in sub_aoa_groups.keys()]\n",
        "            \n",
        "        #Append average concreteness, subtlex, and percent known for each entry\n",
        "        if len(conc) > 0:\n",
        "            concreteness.append(np.mean(conc))\n",
        "        else:\n",
        "            concreteness.append(0) #Less concrete words have a smaller value, anyway\n",
        "        if len(percents_known_list) > 0:\n",
        "            avg_percent_recognized.append(np.mean(percents_known_list))\n",
        "        else:\n",
        "            avg_percent_recognized.append(0)\n",
        "            \n",
        "        #Now for Dale Chall\n",
        "        percent_dale_chall.append(num_dale_chall / len_tokens)\n",
        "                \n",
        "    return avg_aoa_scores, concreteness, percent_dale_chall, word_count, avg_word_length, avg_syllables, avg_percent_recognized, dict(total_pos), aoa_total_groups, syllable_groups"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DJGVYIRvrUwL"
      },
      "source": [
        "The below code mines the features for the training and (Kaggle competition) test datasets"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QrSqIL0yrUwM"
      },
      "source": [
        "train_aoa, train_concreteness, train_dalechall, train_wordcount, train_avg_word_length, train_avg_syllables, train_avg_percent_recognized, train_perc_pos_dict, train_aoa_dict, train_syllable_dict = extract_features(wikilarge_train)"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "71mQUZlWrUwM"
      },
      "source": [
        "#test_aoa, test_concreteness, test_dalechall, test_wordcount, test_avg_word_length, test_avg_syllables, test_avg_percent_recognized, test_perc_pos_dict, test_aoa_dict, test_syllable_dict = extract_features(wikilarge_test)"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vmr-1GiorUwM"
      },
      "source": [
        "The below line of code places the features in their respectibe dataframes. The \"Test\" df is for the pseudo Kaggle competition only, so it is commented out."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7AnhO-RyrUwN",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 467
        },
        "outputId": "ec6714bf-149c-4a3f-c60c-ce955a82e1e3"
      },
      "source": [
        "#Add columns for aoa scores\n",
        "#wikilarge_train = wikilarge_train[]# --------- For testing purposes\n",
        "\n",
        "#Create a pandas column for each POS percentage\n",
        "for pos, percentages in train_perc_pos_dict.items():\n",
        "    wikilarge_train[pos] = percentages\n",
        "\n",
        "#Create a column for percents of words in each age of acquisition group\n",
        "for aoa_group, percentages in train_aoa_dict.items():\n",
        "    wikilarge_train[aoa_group] = percentages\n",
        "    \n",
        "#Create a column for percents of words in each number of syllables group\n",
        "for syll_group, percentages in train_syllable_dict.items():\n",
        "    wikilarge_train[syll_group] = percentages\n",
        "\n",
        "    \n",
        "wikilarge_train['avg_aoa_score'] = train_aoa\n",
        "wikilarge_train['avg_concreteness'] = train_concreteness\n",
        "wikilarge_train['perc_dale_shall'] = train_dalechall\n",
        "wikilarge_train['num_words'] = train_wordcount\n",
        "wikilarge_train['avg_word_length'] = train_avg_word_length\n",
        "wikilarge_train['avg_syllable_count'] = train_avg_syllables\n",
        "wikilarge_train['avg_percent_recognized'] = train_avg_percent_recognized\n",
        "\n",
        "\n",
        "####################################For the Kaggle test DF\n",
        "\n",
        "# #Create a pandas column for each POS percentage\n",
        "# for pos, percentages in test_perc_pos_dict.items():\n",
        "#     wikilarge_test[pos] = percentages\n",
        "\n",
        "# #Create a column for percents of words in each age of acquisition group\n",
        "# for aoa_group, percentages in test_aoa_dict.items():\n",
        "#     wikilarge_test[aoa_group] = percentages\n",
        "    \n",
        "# #Create a column for percents of words in each number of syllables group\n",
        "# for syll_group, percentages in test_syllable_dict.items():\n",
        "#     wikilarge_test[syll_group] = percentages\n",
        "    \n",
        "# wikilarge_test['avg_aoa_score'] = test_aoa\n",
        "# wikilarge_test['avg_concreteness'] = test_concreteness\n",
        "# wikilarge_test['perc_dale_shall'] = test_dalechall\n",
        "# wikilarge_test['num_words'] = test_wordcount\n",
        "# wikilarge_test['avg_word_length'] = test_avg_word_length\n",
        "# wikilarge_test['avg_syllable_count'] = test_avg_syllables\n",
        "# wikilarge_test['avg_percent_recognized'] = test_avg_percent_recognized\n",
        "\n",
        "#Data mining each time is painful... The below code saves the processed csv for more convenient use.\n",
        "#wikilarge_train = pd.read_csv('assets/processed_train.csv')\n",
        "\n",
        "wikilarge_train.head()\n",
        "#wikilarge_test.head()"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>original_text</th>\n",
              "      <th>label</th>\n",
              "      <th>LS</th>\n",
              "      <th>TO</th>\n",
              "      <th>VBN</th>\n",
              "      <th>WP</th>\n",
              "      <th>UH</th>\n",
              "      <th>VBG</th>\n",
              "      <th>JJ</th>\n",
              "      <th>VBZ</th>\n",
              "      <th>--</th>\n",
              "      <th>VBP</th>\n",
              "      <th>NN</th>\n",
              "      <th>DT</th>\n",
              "      <th>PRP</th>\n",
              "      <th>:</th>\n",
              "      <th>WP$</th>\n",
              "      <th>NNPS</th>\n",
              "      <th>PRP$</th>\n",
              "      <th>WDT</th>\n",
              "      <th>(</th>\n",
              "      <th>)</th>\n",
              "      <th>.</th>\n",
              "      <th>,</th>\n",
              "      <th>``</th>\n",
              "      <th>$</th>\n",
              "      <th>RB</th>\n",
              "      <th>RBR</th>\n",
              "      <th>RBS</th>\n",
              "      <th>VBD</th>\n",
              "      <th>IN</th>\n",
              "      <th>FW</th>\n",
              "      <th>RP</th>\n",
              "      <th>JJR</th>\n",
              "      <th>JJS</th>\n",
              "      <th>PDT</th>\n",
              "      <th>MD</th>\n",
              "      <th>VB</th>\n",
              "      <th>WRB</th>\n",
              "      <th>NNP</th>\n",
              "      <th>EX</th>\n",
              "      <th>NNS</th>\n",
              "      <th>SYM</th>\n",
              "      <th>CC</th>\n",
              "      <th>CD</th>\n",
              "      <th>POS</th>\n",
              "      <th>aoa&lt;=5</th>\n",
              "      <th>aoa&lt;=10</th>\n",
              "      <th>aoa&lt;=15</th>\n",
              "      <th>aoa&lt;=20</th>\n",
              "      <th>aoa&gt;20</th>\n",
              "      <th>1 Syll</th>\n",
              "      <th>2 Syll</th>\n",
              "      <th>3 Syll</th>\n",
              "      <th>4 Syll</th>\n",
              "      <th>&gt;=5 Syll</th>\n",
              "      <th>avg_aoa_score</th>\n",
              "      <th>avg_concreteness</th>\n",
              "      <th>perc_dale_shall</th>\n",
              "      <th>num_words</th>\n",
              "      <th>avg_word_length</th>\n",
              "      <th>avg_syllable_count</th>\n",
              "      <th>avg_percent_recognized</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>There is manuscript evidence that Austen conti...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.139535</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.139535</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.093023</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.413793</td>\n",
              "      <td>0.517241</td>\n",
              "      <td>0.068966</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.720930</td>\n",
              "      <td>0.162791</td>\n",
              "      <td>0.116279</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>5.481034</td>\n",
              "      <td>2.691562</td>\n",
              "      <td>0.558140</td>\n",
              "      <td>43.0</td>\n",
              "      <td>4.046512</td>\n",
              "      <td>1.395349</td>\n",
              "      <td>0.985938</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>In a remarkable comparative analysis , Mandaea...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.260870</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.130435</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.428571</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.608696</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>7.499286</td>\n",
              "      <td>2.436429</td>\n",
              "      <td>0.217391</td>\n",
              "      <td>23.0</td>\n",
              "      <td>5.826087</td>\n",
              "      <td>1.782609</td>\n",
              "      <td>0.995000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Before Persephone was released to Hermes , who...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.108696</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.152174</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.472222</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.717391</td>\n",
              "      <td>0.195652</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>5.328056</td>\n",
              "      <td>2.446471</td>\n",
              "      <td>0.543478</td>\n",
              "      <td>46.0</td>\n",
              "      <td>4.413043</td>\n",
              "      <td>1.391304</td>\n",
              "      <td>0.992647</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Cogeneration plants are commonly found in dist...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.230769</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.153846</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.076923</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.282051</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.310345</td>\n",
              "      <td>0.551724</td>\n",
              "      <td>0.137931</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.564103</td>\n",
              "      <td>0.256410</td>\n",
              "      <td>0.102564</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>6.768276</td>\n",
              "      <td>3.430645</td>\n",
              "      <td>0.256410</td>\n",
              "      <td>39.0</td>\n",
              "      <td>5.333333</td>\n",
              "      <td>1.717949</td>\n",
              "      <td>0.994516</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Geneva -LRB- , ; , ; , ; ; -RRB- is the second...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.055556</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.305556</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.138889</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.055556</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.533333</td>\n",
              "      <td>0.400000</td>\n",
              "      <td>0.066667</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.805556</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>5.396667</td>\n",
              "      <td>2.364667</td>\n",
              "      <td>0.361111</td>\n",
              "      <td>36.0</td>\n",
              "      <td>4.638889</td>\n",
              "      <td>1.305556</td>\n",
              "      <td>0.989333</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                       original_text  ...  avg_percent_recognized\n",
              "0  There is manuscript evidence that Austen conti...  ...                0.985938\n",
              "1  In a remarkable comparative analysis , Mandaea...  ...                0.995000\n",
              "2  Before Persephone was released to Hermes , who...  ...                0.992647\n",
              "3  Cogeneration plants are commonly found in dist...  ...                0.994516\n",
              "4  Geneva -LRB- , ; , ; , ; ; -RRB- is the second...  ...                0.989333\n",
              "\n",
              "[5 rows x 63 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QJiKmn12rUwN"
      },
      "source": [
        "### Scale the Data\n",
        "\n",
        "The features contain data that are of different magnitudes, so min-max scaling is applied to bring the data within the range of 0-1."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_j-YSPo5rUwN"
      },
      "source": [
        "#Don't forget to normalize\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "scaler = MinMaxScaler()\n",
        "\n",
        "#Used when concatenating the above df with further tf-idf/ word2vec features\n",
        "columns_to_concat_in = ['LS', 'TO', 'VBN', 'WP', 'UH', 'VBG', 'JJ',\n",
        "                       'VBZ', '--', 'VBP', 'NN', 'DT', 'PRP', ':', 'WP$', 'NNPS', 'PRP$',\n",
        "                       'WDT', '(', ')', '.', ',', '$', 'RB', 'RBR', 'RBS', 'VBD', 'IN',\n",
        "                       'FW', 'RP', 'JJR', 'JJS', 'PDT', 'MD', 'VB', 'WRB', 'NNP', 'EX', 'NNS',\n",
        "                       'SYM', 'CC', 'CD', 'POS', 'aoa<=5', 'aoa<=10', 'aoa<=15', 'aoa<=20',\n",
        "                       'aoa>20', '1 Syll', '2 Syll', '3 Syll', '4 Syll', '>=5 Syll',\n",
        "                       'avg_aoa_score', 'avg_concreteness', 'perc_dale_shall', 'num_words',\n",
        "                       'avg_word_length', 'avg_syllable_count', 'avg_percent_recognized']\n",
        "\n",
        "columns_to_scale = ['avg_aoa_score', 'avg_concreteness', 'num_words', \n",
        "                    'avg_word_length', 'avg_syllable_count',] #Keep out columns with percents, they're already scaled\n",
        "\n",
        "all_columns = ['original_text', 'LS', 'TO', 'VBN', 'WP', 'UH', 'VBG', 'JJ',\n",
        "                       'VBZ', '--', 'VBP', 'NN', 'DT', 'PRP', ':', 'WP$', 'NNPS', 'PRP$',\n",
        "                       'WDT', '(', ')', '.', ',', '$', 'RB', 'RBR', 'RBS', 'VBD', 'IN',\n",
        "                       'FW', 'RP', 'JJR', 'JJS', 'PDT', 'MD', 'VB', 'WRB', 'NNP', 'EX', 'NNS',\n",
        "                       'SYM', 'CC', 'CD', 'POS', 'aoa<=5', 'aoa<=10', 'aoa<=15', 'aoa<=20',\n",
        "                       'aoa>20', '1 Syll', '2 Syll', '3 Syll', '4 Syll', '>=5 Syll',\n",
        "                       'avg_aoa_score', 'avg_concreteness', 'perc_dale_shall', 'num_words',\n",
        "                       'avg_word_length', 'avg_syllable_count', 'avg_percent_recognized']\n",
        "\n",
        "wikilarge_train[columns_to_scale] = scaler.fit_transform(wikilarge_train[columns_to_scale])\n",
        "#wikilarge_test[columns_to_scale] = scaler.fit_transform(wikilarge_test[columns_to_scale])"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VDjsvlcFrUwO",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 467
        },
        "outputId": "e23fb8e7-6641-47de-e8d1-b215740c7fe9"
      },
      "source": [
        " wikilarge_train.head()"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>original_text</th>\n",
              "      <th>label</th>\n",
              "      <th>LS</th>\n",
              "      <th>TO</th>\n",
              "      <th>VBN</th>\n",
              "      <th>WP</th>\n",
              "      <th>UH</th>\n",
              "      <th>VBG</th>\n",
              "      <th>JJ</th>\n",
              "      <th>VBZ</th>\n",
              "      <th>--</th>\n",
              "      <th>VBP</th>\n",
              "      <th>NN</th>\n",
              "      <th>DT</th>\n",
              "      <th>PRP</th>\n",
              "      <th>:</th>\n",
              "      <th>WP$</th>\n",
              "      <th>NNPS</th>\n",
              "      <th>PRP$</th>\n",
              "      <th>WDT</th>\n",
              "      <th>(</th>\n",
              "      <th>)</th>\n",
              "      <th>.</th>\n",
              "      <th>,</th>\n",
              "      <th>``</th>\n",
              "      <th>$</th>\n",
              "      <th>RB</th>\n",
              "      <th>RBR</th>\n",
              "      <th>RBS</th>\n",
              "      <th>VBD</th>\n",
              "      <th>IN</th>\n",
              "      <th>FW</th>\n",
              "      <th>RP</th>\n",
              "      <th>JJR</th>\n",
              "      <th>JJS</th>\n",
              "      <th>PDT</th>\n",
              "      <th>MD</th>\n",
              "      <th>VB</th>\n",
              "      <th>WRB</th>\n",
              "      <th>NNP</th>\n",
              "      <th>EX</th>\n",
              "      <th>NNS</th>\n",
              "      <th>SYM</th>\n",
              "      <th>CC</th>\n",
              "      <th>CD</th>\n",
              "      <th>POS</th>\n",
              "      <th>aoa&lt;=5</th>\n",
              "      <th>aoa&lt;=10</th>\n",
              "      <th>aoa&lt;=15</th>\n",
              "      <th>aoa&lt;=20</th>\n",
              "      <th>aoa&gt;20</th>\n",
              "      <th>1 Syll</th>\n",
              "      <th>2 Syll</th>\n",
              "      <th>3 Syll</th>\n",
              "      <th>4 Syll</th>\n",
              "      <th>&gt;=5 Syll</th>\n",
              "      <th>avg_aoa_score</th>\n",
              "      <th>avg_concreteness</th>\n",
              "      <th>perc_dale_shall</th>\n",
              "      <th>num_words</th>\n",
              "      <th>avg_word_length</th>\n",
              "      <th>avg_syllable_count</th>\n",
              "      <th>avg_percent_recognized</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>There is manuscript evidence that Austen conti...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.139535</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.139535</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.093023</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.413793</td>\n",
              "      <td>0.517241</td>\n",
              "      <td>0.068966</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.720930</td>\n",
              "      <td>0.162791</td>\n",
              "      <td>0.116279</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.288475</td>\n",
              "      <td>0.538312</td>\n",
              "      <td>0.558140</td>\n",
              "      <td>0.461538</td>\n",
              "      <td>0.117174</td>\n",
              "      <td>0.056478</td>\n",
              "      <td>0.985938</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>In a remarkable comparative analysis , Mandaea...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.260870</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.130435</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.428571</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.608696</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.394699</td>\n",
              "      <td>0.487286</td>\n",
              "      <td>0.217391</td>\n",
              "      <td>0.241758</td>\n",
              "      <td>0.185619</td>\n",
              "      <td>0.111801</td>\n",
              "      <td>0.995000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Before Persephone was released to Hermes , who...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.108696</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.152174</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.472222</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.717391</td>\n",
              "      <td>0.195652</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.280424</td>\n",
              "      <td>0.489294</td>\n",
              "      <td>0.543478</td>\n",
              "      <td>0.494505</td>\n",
              "      <td>0.131271</td>\n",
              "      <td>0.055901</td>\n",
              "      <td>0.992647</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Cogeneration plants are commonly found in dist...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.230769</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.153846</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.076923</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.282051</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.310345</td>\n",
              "      <td>0.551724</td>\n",
              "      <td>0.137931</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.564103</td>\n",
              "      <td>0.256410</td>\n",
              "      <td>0.102564</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.356225</td>\n",
              "      <td>0.686129</td>\n",
              "      <td>0.256410</td>\n",
              "      <td>0.417582</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.102564</td>\n",
              "      <td>0.994516</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Geneva -LRB- , ; , ; , ; ; -RRB- is the second...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.055556</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.305556</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.138889</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.055556</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.533333</td>\n",
              "      <td>0.400000</td>\n",
              "      <td>0.066667</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.805556</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.284035</td>\n",
              "      <td>0.472933</td>\n",
              "      <td>0.361111</td>\n",
              "      <td>0.384615</td>\n",
              "      <td>0.139957</td>\n",
              "      <td>0.043651</td>\n",
              "      <td>0.989333</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                       original_text  ...  avg_percent_recognized\n",
              "0  There is manuscript evidence that Austen conti...  ...                0.985938\n",
              "1  In a remarkable comparative analysis , Mandaea...  ...                0.995000\n",
              "2  Before Persephone was released to Hermes , who...  ...                0.992647\n",
              "3  Cogeneration plants are commonly found in dist...  ...                0.994516\n",
              "4  Geneva -LRB- , ; , ; , ; ; -RRB- is the second...  ...                0.989333\n",
              "\n",
              "[5 rows x 63 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AKXIhtaijirt"
      },
      "source": [
        "### Split Data into Train, Test, and Maybe Development Sets\n",
        "The StratifiedShuffleSplit module helps ensure that classes are balanced before testing."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EyG8dOWfrUwP"
      },
      "source": [
        "from sklearn.model_selection import StratifiedShuffleSplit\n",
        "X = wikilarge_train[all_columns].values\n",
        "y = wikilarge_train[['label']].values\n",
        "\n",
        "sss = StratifiedShuffleSplit( n_splits=1, test_size =0.2, random_state=0)  \n",
        "\n",
        "for train_index, test_index in sss.split(X, y):\n",
        "    X_train, X_test = X[train_index], X[test_index]\n",
        "    y_train, y_test = y[train_index], y[test_index]"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PMtD6psHX_l7"
      },
      "source": [
        "## 2.) Tf-idf & Word2vec Representations"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "772fdheDfozq"
      },
      "source": [
        "### Tf-idf (Term Frequency - Inverse Document Frequency)\n",
        "A Tf-idf vectorizer numerically represents the texts by measuring term frequency and inverse document frequency. Term frequency gives weight to words that appear more often, and inverse document frequency gives more weight to words that appear in fewer documents. I hypothesize that this is advantageous due to big words that are harder to read being in fewer documents, and more simple words appearing in relatively more documents.\n",
        "\n",
        "The Tf-idf vectorizer class from scikit-learn also allows us to remove common stopwords, put the words in lowercase, and create n-grams of the text per our choosing. Since the model is checking for readability, it is okay to put the words in lowercase, since the capitalization doesn't affect readbility, and keeping track of less words frees up memory."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iSmsI9-mrUwP"
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "#Remove english stop words, put words in lowercase, using bigrams to get better performance\n",
        "vectorizer = TfidfVectorizer(stop_words='english', lowercase=True, min_df=500)\n",
        "X_train_tfidf = vectorizer.fit_transform(X_train[:, :1].flatten())\n",
        "X_test_tfidf = vectorizer.transform(X_test[:, :1].flatten())"
      ],
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sj4Pdmo6rUwP"
      },
      "source": [
        "from scipy.sparse import hstack \n",
        "#This helps you concatenate features from aoa to thte tfidf vector (+1 to column for each feature)\n",
        "full_X_train = hstack([X_train_tfidf, X_train[:, 1:].astype(float)])\n",
        "full_X_test = hstack([X_test_tfidf, X_test[:, 1:].astype(float)])"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O1JmKdizrUwQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b87738c1-22ff-4890-8d65-a3463213c558"
      },
      "source": [
        "full_X_train"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<333414x1167 sparse matrix of type '<class 'numpy.float64'>'\n",
              "\twith 9177110 stored elements in COOrdinate format>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jPQdmio7rUwR"
      },
      "source": [
        "### WORD2VEC\n",
        "Word2Vec provides vectorized representations of each word in the text, and words that are similar in meaning have similar representations. Additionally, the vectorized representations somehow manage to capture their meanings, semantic relationships and the different types of contexts they are used in. The context of words and how they're used may have a relationship with a difficulty to understand them, so  Word2Vec may be a fitting representation of the text.\n",
        "\n",
        "Note: Training word2vec models can take a long time, so I used the gensim library to download a pre-trained word2vec model that transforms the text."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "srEHAjYzrUwR"
      },
      "source": [
        "import gensim\n",
        "import gensim.downloader as gensim_api"
      ],
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FtUPAX9DrUwR"
      },
      "source": [
        "pre_trained_word2vec = gensim_api.load(\"fasttext-wiki-news-subwords-300\")"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iPDlLKCbrUwR"
      },
      "source": [
        "The function text_to_mean_word2vec calculates the vector representations for each word with a length of 300, but becaue each text entry has several words, the mean value along the rows are taken, and 300 features are created for the average of each entry's indices."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dtqQJDf4rUwS"
      },
      "source": [
        "def text_to_mean_word2vec(X):\n",
        "    \n",
        "    '''\n",
        "    Takes in array of features with original text and transforms each text entry into an array that is the average of its word embeddings in the shape (1,300)\n",
        "    returns the vstacked arrays of all average word embeddings in the shape (len_entries, 300)\n",
        "    '''\n",
        "    \n",
        "    all_embeddings = []\n",
        "    #Iterate over each text entry\n",
        "    for text in X[:,0]:\n",
        "        \n",
        "        #First, tokenize the text so that word2vec can process it\n",
        "        tokenized_text = [word.lower() for word in word_tokenize(text)]\n",
        "\n",
        "        all_word_vectors = []\n",
        "        #Vectorize text if it's in the pretrained model or return an array of zeros (zeros needed so that you don't take the mean of nothing)\n",
        "        #Keep in mind that pre_trained_word2vec[word] gives a vector of shape (300,)\n",
        "        for token in tokenized_text:\n",
        "            if token in pre_trained_word2vec:\n",
        "                vector = pre_trained_word2vec[token].reshape(1,-1)\n",
        "            else:\n",
        "                vector = np.zeros((1,300))\n",
        "            all_word_vectors.append(vector)\n",
        "\n",
        "        text_mean_vector = np.mean(np.vstack(all_word_vectors), axis=0)\n",
        "        all_embeddings.append(text_mean_vector)\n",
        "        \n",
        "    return np.vstack(all_embeddings)"
      ],
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nDioyKGkrUwS"
      },
      "source": [
        "#Get Word Embeddings\n",
        "X_train_embeddings = text_to_mean_word2vec(X_train)\n",
        "X_test_embeddings = text_to_mean_word2vec(X_test)"
      ],
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LcwjceWnrUwS"
      },
      "source": [
        "#Normalize the embeddings\n",
        "\n",
        "w2vec_scaler = MinMaxScaler()\n",
        "\n",
        "X_train_embeddings_scaled = w2vec_scaler.fit_transform(X_train_embeddings)\n",
        "X_test_embeddings_scaled = w2vec_scaler.fit_transform(X_test_embeddings)"
      ],
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MUhEFt1zrUwS"
      },
      "source": [
        "#This helps you concatenate features from aoa to thte tfidf vector (+1 to column for each feature)\n",
        "full_X_train_w2vec = np.hstack([X_train_embeddings_scaled, X_train[:, 2:].astype(float)])\n",
        "full_X_test_w2vec = np.hstack([X_test_embeddings_scaled, X_test[:, 2:].astype(float)])"
      ],
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nYYAj-O8rUwT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b6c578ba-c7bc-4508-9703-654306535b24"
      },
      "source": [
        "full_X_train_w2vec.shape"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(333414, 359)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cziR-CNSrUwT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4be2a695-bcfc-4bb2-a97f-3caa80cf3205"
      },
      "source": [
        "full_X_test_w2vec.shape"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(83354, 359)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J2D0QS3KoNYU"
      },
      "source": [
        "## 3.) Train & Evaluate Models\n",
        "\n",
        "The models that will be trained and validated are as follows:\n",
        "\n",
        "1. Naive Bayes\n",
        "2. Logistic Regression\n",
        "2. Gradient Boosted Decision Tree\n",
        "\n",
        "Although accuracy is a simple measure to compare performances between models, especially when the disparity is large, accuaracy gives equal weight to false positives and false negatives, which are both considerable in a real-world setting. Hence, it is appropriate to also evaluate a model based on precision and recall. Precision calculates the ratio of true positives to all positives, making it a good metric to evaluate when seeking to minimize false positives. Recall calculates the ratio of true positives to the sum of true positives and false negatives, making it a good metric to prioritize when reducing false negatives. It is my belief that false positives can slide for this task, because a simple document marked as difficult won't give any reader a hard time, but a false negative situation that has a difficult document that was marked as simple landing in someone's haands may result in some difficulty. Recall should be optimized for, but precision should also be monitored for trade-off purposes.\n",
        "\n",
        "Precision-recall curves and confusion matrices for each model will be generated to assist in evaluating model performance.\n",
        "\n",
        "Confusion matrices provide a summary of a model's correct predictions along with the type 1 and type 2 errors made. Confusion matrices provide the numbers that are used to generate precision and recall.\n",
        "\n",
        "Precision-recall curves help in visualizing the trade-offs a model makes between between precision and recall by graphing precision on the x axis and recall on the y per different probabilty thresholds for determining classes. An ideal model would have a PR curve that maximizes both precision and recall, giving a curve that reaches the upper right corner."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ruxZeNjLojHo"
      },
      "source": [
        "### Naive Bayes (Multinomial)\n",
        "\n",
        "Our first model will be a basic Naive Bayes classifier. \"Naive Bayes methods are a set of supervised learning algorithms based on applying Bayes’ theorem with the “naive” assumption of conditional independence between every pair of features given the value of the class variable.\" (Scikit-Learn documentation) \n",
        "\n",
        "This assumption often doesn't hold in real life, but Naive Bayes models are fast to train, interpretable, work well with high dimensional dat, and often give a good baseline when comparing to other more-sophisticated models.\n",
        "\n",
        "Muiltinomial is the type of Naive Bayes model used, as it is particularly well suited to textual data where each feature corresponds to an observation for each particular word, while Bernoulli is more accustomed to binary features, and Gaussian is better suited for continuous values."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xksOGUEJxHmk"
      },
      "source": [
        "#### Naive Bayes w/ Tf-idf Feature Representation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UXC44mmEoELX"
      },
      "source": [
        "from sklearn.naive_bayes import MultinomialNB\n",
        "\n",
        "#Create Multinomial Naive Bayes Classifier\n",
        "nb_clf = MultinomialNB()\n",
        "#Fit classifier to Tf-idf training data\n",
        "nb_clf_tfidf = nb_clf.fit(full_X_train, y_train.ravel())\n",
        "#Create predictions on test set\n",
        "nb_tfidf_preds = nb_clf_tfidf.predict(full_X_test)"
      ],
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rgEGsODRx8up"
      },
      "source": [
        "#### Evaluate Multinomial Naive Bayes w/ Tf-idf Representations"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rPszWNm10EC_"
      },
      "source": [
        "##### Multinomial Naive Bayes w/ Tf-idf Accuracy, Precision, and Recall"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F7IJcgVEyzrl",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "28b4eece-5963-4229-8ed2-666f63b1548e"
      },
      "source": [
        "from sklearn.metrics import accuracy_score, precision_score, recall_score\n",
        "\n",
        "print(\"Multinomial Naive Bayes (Tf-idf) Accuracy: \" + str(accuracy_score(nb_tfidf_preds, y_test)))\n",
        "print(\"Multinomial Naive Bayes (Tf-idf) Precision: \" + str(precision_score(nb_tfidf_preds, y_test)))\n",
        "print(\"Multinomial Naive Bayes (Tf-idf) Recall: \" + str(recall_score(nb_tfidf_preds, y_test)))"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Multinomial Naive Bayes (Tf-idf) Accuracy: 0.644948052882885\n",
            "Multinomial Naive Bayes (Tf-idf) Precision: 0.6438563236317393\n",
            "Multinomial Naive Bayes (Tf-idf) Recall: 0.6452652334920406\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_KCLQigfrUwV"
      },
      "source": [
        "##### Multinomial Naive Bayes w/ Tf-idf Confusion Matrix"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5dASkbZXrUwV",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "0469f202-1444-476c-8902-19bc3cc2cad2"
      },
      "source": [
        "from sklearn.metrics import confusion_matrix\n",
        "from sklearn.metrics import ConfusionMatrixDisplay\n",
        "\n",
        "cm = confusion_matrix(y_test, nb_tfidf_preds)\n",
        "cm_display = ConfusionMatrixDisplay(cm, display_labels=[0, 1]).plot()"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxW5Zn/8c83CRHZBARBWURHXKh1KypWbZVWQe1vUEdntFUZd0Wrtkyt2vkVa6ttrUtrq7RaUauOuFGlFUXqUqUdEGxVBESoVHbZZZMlyTV/PCfhSUieBRKSPPm+X6/z4nmuc59z7hPgyr2cRRGBmZmlFDV2BczMmhInRTOzNE6KZmZpnBTNzNI4KZqZpSlp7Aqk69K5OPr0atXY1bA8fPhem8auguVhI+vZHJu0I/sYdGLbWLGyPKeyb7+3aXxEDN6R4+1sTSop9unVirfG92rsalgeBu11WGNXwfIwOV7Z4X2sWFnOW+N751S2eM/ZXXb4gDtZk0qKZtb0BVBBRWNXo8E4KZpZXoJgS+TWfW6OnBTNLG9uKZqZJYKgvIBvD3ZSNLO8VeCkaGYGpCZayp0Uzcy2ckvRzCwRwBaPKZqZpQTh7rOZWZWA8sLNiU6KZpaf1B0thctJ0czyJMrZoWdKNGl+dJiZ5SU10aKclkwk9ZL0mqQZkqZLujZt3TclfZDEb0+L3yhpjqRZkgalxQcnsTmSbkiL7yNpchJ/UlJptvNzS9HM8pK6TrFeWoplwPCI+Juk9sDbkiYA3YAhwKERsUnSHgCS+gHnAJ8D9gL+JGn/ZF/3AicBC4ApksZGxAzgp8DdETFa0q+Bi4GRmSrllqKZ5a0ilNOSSUQsjoi/JZ/XAjOBHsCVwE8iYlOybmmyyRBgdERsioi5wBzgqGSZExEfRcRmYDQwRJKAgcAzyfaPAKdnOzcnRTPLS2VLMZcF6CJpatpyWW37lNQHOByYDOwPHJ90e/8s6cikWA9gftpmC5JYXfHdgdURUVYjnpG7z2aWl0CU596eWh4R/TMVkNQOeBa4LiLWSCoBOgMDgCOBpyTtuyN1zoeTopnlLVvXOFeSWpFKiI9HxJgkvAAYExEBvCWpAugCLATSH83fM4lRR3wF0FFSSdJaTC9fJ3efzSwvgdgcxTktmSRjfg8CMyPirrRVzwEnJmX2B0qB5cBY4BxJu0jaB+gLvAVMAfomM82lpCZjxiZJ9TXgrGS/Q4Hns52fW4pmlpfUxdv10p46FjgfmCbpnSR2EzAKGCXpfWAzMDRJcNMlPQXMIDVzfVVE6hHgkq4GxgPFwKiImJ7s77vAaEk/Av5OKgln5KRoZnmrj0tyImIi1Lmj8+rY5lbg1lri44BxtcQ/IjU7nTMnRTPLS4Qoj8IdeXNSNLO8VRTwbX5OimaWl9RES+GmjsI9MzNrEPU40dIkOSmaWd7K6+k6xabISdHM8pLnHS3NjpOimeWtwrPPZmYpqQdCOCmamQGp7vOWLLfwNWdOimaWlwh88baZ2VbyxdtmZpUCtxTNzKrxRIuZWSLI/v6V5sxJ0czyknrFaeGmjsI9MzNrIKqvV5w2SU6KZpaXwHe0mJlV45aimVkiQm4pmplVSk20+DY/M7OE39FiZlYlNdHiMUUzsyq+o8XMLOE7WszMavCLq8zMEhGwpaJwk2LhnpmZNYhU97kopyUTSb0kvSZphqTpkq6tsX64pJDUJfkuSfdImiPpPUlHpJUdKml2sgxNi39B0rRkm3skZe33OymaWd7Kk/ufsy1ZlAHDI6IfMAC4SlI/SCVM4GRgXlr5U4C+yXIZMDIp2xkYARwNHAWMkNQp2WYkcGnadoOzVcrd5xqWLmzFz67tzeplrUDBqeet4IxLllcr8/R9XXl1TGcAysth/uzWPDntfTp0Kt/u427eJH52TW9mT2tDh05l3PTrj+nea/PWei1oxaUnHMh5w5dw9pXLtvs4hejbd83j6K+uZfXyEi4feMA26w85Zh03PzSXJfNLAfjLuN14/O7uO3TMVqUVfOeeefT9/GesWVXCbVfszScLSqvWd+2xmQden8Vjd3bjmV/vsUPHamrq65KciFgMLE4+r5U0E+gBzADuBq4Hnk/bZAjwu4gIYJKkjpL2BE4AJkTESgBJE4DBkl4HOkTEpCT+O+B04MVM9WrQlqKkwZJmJU3XGxryWPWluCS47PuLeODPH/CLP87mDw934eMPd6lW5uxhyxj5p1mM/NMsLrpxMZ8/Zl3OCXHJ/FK+82/7bRMf/0Rn2nUs5+G/zuTMS5fx4I/2rLb+Nz/owZED127/iRWwl5/szPe+sU/GMu9Pbsuwkw5g2EkH5JUQu/XczO3PzNkmPujclaxbXcKFxx7EmAe6cPF/L6q2/vIRi5jyavucj9O81E/3udoepT7A4cBkSUOAhRHxbo1iPYD5ad8XJLFM8QW1xDNqsKQoqRi4l1STtx9wbmXTuCnbvVsZfQ/5DIA27Srotd8mli9uVWf5157rxAmnr6r6/sqznfjmqX258qsH8Ivre1KeY+Pxf8fvxklnrwTg+K+t5p2J7YlIrfvri7vRvddm9t5/4/adVIF7f3I71q7avk7PwDNXcc8LH3LfhFlc89P5FBVFTtsdM+hTJjyd6qG9+ceOHHbcOlJtKDhm8KcsmV/Kxx+23q46NQcVyXtasi1AF0lT05bLau5LUjvgWeA6Ul3qm4Dv79QTStOQLcWjgDkR8VFEbAZGk2r+NhtL5pfyj/d35cAjNtS6fuMGMfX19hx36qcAzJu9C39+viN3Pz+bkX+aRVExvDqmU63b1rR8SSu67rUFgOISaNuhnDUri/lsfRFP3bcH5w1fUj8n1UId9IUNjJwwix899lHVL5de+23ky0NW860hfRl20gFUlIuBZ67KsqeULt3LWLYo9cuyolysX1NMh87ltG5Tzr8PW8pjd3ZrsHNpbKnZ5+KcFmB5RPRPW+5P35ekVqQS4uMRMQb4F2Af4F1J/wR6An+T1B1YCPRK27xnEssU71lLPKOGHFOsrUl7dM1CyW+OywB692g6Q5yfrS/ih5f04YpbFtK2fUWtZSZN2I3P9V9f1XX++5vtmT2tDd88JTWutXmj6Lh7GQA/uKgPS+btQtkWsXRhK678aqrM6ZcsY9A5K+usx6N3dOeMS5exa9va62DZzZm2K+cfdRAbNxRz5MA1jBg1l4uOO4jDj19H389v4JcvfghAaetg9YrUv8HvPziX7r03U9Iq2KPHFu6bMAuA537blZef7Fznsc7/r0/4/QNd2bihcB+YUF8XbyczwQ8CMyPiLoCImAbskVbmn0D/iFguaSxwtaTRpHLJpxGxWNJ44La0yZWTgRsjYqWkNZIGAJOBC4BfZqtXo2eh5DfH/QD9D22dW9+lgZVtgR9e0oeBZ66qagXW5s/Pd6zWdSbgpLNXctFNi7cpO2LUP4FU6/PO63rzs2erj1N16b6FZYtSrcXyMqpaHh/8vQ0TX+jIgz/ai3VrilFRULpLMOSi5dscw2q3Yd3WBDXl1Q5c/eMFdOhcBgomPN2Zh3685zbb3HJxaoyyW8/NDP/5PK4/q/o48PIlJXTdawvLF5dSVBxVLfsDD9/Acaet5uL/XkS7DuVEhdi8qYixD3Vp2JPcyerpFafHAucD0yS9k8RuiohxdZQfB5wKzAE2ABcCJMnvh8CUpNwtlZMuwDDgYWBXUhMsGSdZoGGTYl1N2iYtAu4a3ptefTfxb5fXPcu7fk0R701qx3d/tfWKgcOOX8vNF+7LmZcto2OXMtasSnV/u/XckvW4A05ew4SnO9Ov/wbe/GNHDj1uLRLc9dzW5PnoHd1p3bbcCTFPnbpuYdWyEkAccNgGiopgzcpi3nmzPTc/NJcx93fh0xWtaN+xjF3bVrB0YWnWfU56eTdOOnsVM99uy/FfW827E9sBYvgZW5PnecOXsHF94SXEepx9ngiZs2tE9En7HMBVdZQbBYyqJT4VODifejVkUpwC9JW0D6lkeA7w9QY8Xr2Y/lZbXnmmM/sc9FlVF/fCGxdV/Uf52gUrAPjLix35wpfW0rrN1m7t3vtvYuj1i7nxnH8hIjWTffVtC3JKioPPXcHt1+zNf37xINp3LOOmkR83wNkVphvu+5hDjlnHbp3LeGzqDB69sxslJalOxwuPduH4r33K1y5YTnmZ2LSxiB9fuTcg5s1uzSO3d+fHoz9CgvIy8aubeuSUFF96ojPX3zOPh/4yk7Wri7ntyr0b+CyblkJ+yKwiGq7HKulU4OdAMTAqIm7NVL7/oa3jrfG9MhWxJmbQXoc1dhUsD5PjFdbEyh1q5nU6cI8YOOqsnMqOOXbk2xHRf0eOt7M16JhiMjZQ1/iAmTVTfkqOmVnCD5k1M6vBSdHMLOGHzJqZ1VBP1yk2SU6KZpaXCCgr4IfMOimaWd7cfTYzS3hM0cyshnBSNDPbyhMtZmaJCI8pmpmlEeWefTYz28pjimZmCd/7bGaWLqABnzjY6JwUzSxvnn02M0uEJ1rMzKpz99nMLI1nn83MEhFOimZm1fiSHDOzNB5TNDNLBKLCs89mZlsVcEPRSdHM8lTgEy2F2wY2s4YTOS4ZSOol6TVJMyRNl3RtEv+ZpA8kvSfp95I6pm1zo6Q5kmZJGpQWH5zE5ki6IS2+j6TJSfxJSaXZTs1J0czyFqGclizKgOER0Q8YAFwlqR8wATg4Ig4BPgRuBEjWnQN8DhgM3CepWFIxcC9wCtAPODcpC/BT4O6I2A9YBVycrVJ1dp8l/ZIMuT4irsm2czMrPAFUVOx49zkiFgOLk89rJc0EekTEy2nFJgFnJZ+HAKMjYhMwV9Ic4Khk3ZyI+AhA0mhgSLK/gcDXkzKPADcDIzPVK9OY4tQcz83MWpIAch9T7CIpPZfcHxH31ywkqQ9wODC5xqqLgCeTzz1IJclKC5IYwPwa8aOB3YHVEVFWS/k61ZkUI+KRGpVuExEbsu3QzApfHtcpLo+I/pkKSGoHPAtcFxFr0uLfI9XFfnw7q7ldso4pSjpG0gzgg+T7oZLua/CamVnTVQ8TLQCSWpFKiI9HxJi0+H8CXwO+EVGVghcCvdI275nE6oqvADpKKqkRzyiXiZafA4OSAxAR7wJfymE7MytIuU2yZJtokSTgQWBmRNyVFh8MXA/8a43e6VjgHEm7SNoH6Au8BUwB+iYzzaWkJmPGJsn0NbaOSQ4Fns92djldpxgR81P1r1Key3ZmVqDq5+rtY4HzgWmS3kliNwH3ALsAE5K8MykiroiI6ZKeAmaQ6lZfFRHlAJKuBsYDxcCoiJie7O+7wGhJPwL+TioJZ5RLUpwv6YtAJE3da4GZuZyxmRWggKif2eeJUOsjvMdl2OZW4NZa4uNq2y6ZkT6qZjyTXLrPVwBXkZq1WQQclnw3sxZLOS7NT9aWYkQsB76xE+piZs1FAd/8nMvs876S/iBpmaSlkp6XtO/OqJyZNVH1NPvcFOXSff4f4ClgT2Av4GngiYaslJk1YZUXb+eyNEO5JMU2EfFoRJQly2NA64aumJk1XRG5Lc1RpnufOycfX0yeOjGa1O+I/yDD7JCZtQD1MPvcVGWaaHmbVBKsPPvL09YFyZMrzKzlUTNtBeYi073P++zMiphZM9GMJ1FykdMdLZIOJvWcsqqxxIj4XUNVysyasuY7iZKLrElR0gjgBFJJcRypBzlOBJwUzVqqAm4p5jL7fBbwFWBJRFwIHArs1qC1MrOmrSLHpRnKpfv8WURUSCqT1AFYSvXH9JhZS5LfQ2abnVyS4tTkxTEPkJqRXgf8b4PWysyatBY5+1wpIoYlH38t6SWgQ0S817DVMrMmrSUmRUlHZFoXEX9rmCqZmTWeTC3FOzOsC1JvyapXH77XhkE9Dq/v3VoDGrfw7cauguVhwOD19bKfFtl9jogTd2ZFzKyZCFrsbX5mZrVriS1FM7O6tMjus5lZnQo4Keby5G1JOk/S95PvvSXl9SIYMyswLfzJ2/cBxwDnJt/XAvc2WI3MrElT5L40R7l0n4+OiCMk/R0gIlYlL5w2s5aqhc8+b5FUTNIYltSVZnurt5nVh+baCsxFLt3ne4DfA3tIupXUY8Nua9BamVnTVsBjirnc+/y4pLdJPT5MwOkRMbPBa2ZmTVMzHi/MRS4Pme0NbAD+kB6LiHkNWTEza8IKOCnm0n1+Afhj8ucrwEfAiw1ZKTNr2lSR25JxH1IvSa9JmiFpuqRrk3hnSRMkzU7+7JTEJekeSXMkvZf+0BpJQ5PysyUNTYt/QdK0ZJt7JGWdIcqaFCPi8xFxSPJnX+Ao/DxFM9txZcDwiOgHDACuktQPuAF4Jck3ryTfIfUqlL7JchkwEqpexzwCOJpUfhpRmUiTMpembTc4W6VyaSlWkzwy7Oh8tzOzAlIPEy0RsbjyEYQRsRaYCfQAhgCPJMUeAU5PPg8Bfhcpk4COkvYEBgETImJlRKwCJgCDk3UdImJSRASp90pV7qtOuYwpfjvtaxFwBLAo23ZmVqDym2jpImlq2vf7I+L+moUk9QEOByYD3SJicbJqCdAt+dwDmJ+22YIklim+oJZ4Rrlcp9g+7XMZqbHFZ3PYzswKVe5JcXlE9M9UQFI7UjnluohYkz7sFxEh7dy57oxJMblou31E/NdOqo+ZNQf1lKYktSKVEB+PiDFJ+BNJe0bE4qQLvDSJL6T6S/N6JrGFpF7DnB5/PYn3rKV8RnWOKUoqiYhy4NhsOzGzlkPU2+yzgAeBmRFxV9qqsUDlDPJQ4Pm0+AXJLPQA4NOkmz0eOFlSp2SC5WRgfLJujaQBybEuSNtXnTK1FN8iNX74jqSxwNNA1bPM07K6mbUk9Xfx9rHA+cA0Se8ksZuAnwBPSboY+Bj492TdOOBUYA6pa6cvBIiIlZJ+CExJyt0SESuTz8OAh4FdSV1KmPVywlzGFFsDK0i9kyVI/aIIwEnRrKWqh6QYERNJ5ZPafKWW8gFcVce+RgGjaolPBQ7Op16ZkuIeyczz+2xNhlXHyucgZlZgCjgDZEqKxUA7as/kBfwjMbNsWuq9z4sj4padVhMzaz5aaFIs3KdImtn2i+wzy81ZpqS4zUCnmRnQMluKaVPaZmbVtNQxRTOz2jkpmpklmvGrBnLhpGhmeRHuPpuZVeOkaGaWzknRzCyNk6KZWaKlv+LUzGwbTopmZlu11Nv8zMxq5e6zmVklX7xtZlaDk6KZWYrvaDEzq0EVhZsVnRTNLD8eUzQzq87dZzOzdE6KZmZbuaVoZpbOSdHMLNGC3+ZnZraNQr9OsaixK2BmzVBEbksWkkZJWirp/bTYYZImSXpH0lRJRyVxSbpH0hxJ70k6Im2boZJmJ8vQtPgXJE1LtrlHUtb32TspmlneFLktOXgYGFwjdjvwg4g4DPh+8h3gFKBvslwGjASQ1BkYARwNHAWMkNQp2WYkcGnadjWPtQ13n2vx7TvncfRX17B6eQmXf+XAbdYfcsxabh41lyXzSwH4y7iOPP7z7jt0zFalFXznF/Po+/kNrFlVwm1X7s0nC3apWt91r8088PoHPHZnd575zR47dKxCs2xhK+68tg+rlpcgweBvLOf0S5ZVK/PMyD14fUxnAMrLxfzZrXnivfdo36l8u4+7ZZO449o+zJm2K+07lXPjyLl067W5av3Sha244oR+fGP4Yv7tiqXbfZwmpx4v3o6INyT1qeUIHZLPuwGLks9DgN9FRACTJHWUtCdwAjCh8l31kiYAgyW9DnSIiElJ/HfA6cCLmerUYC3F2prFzcXLT3Xme9/YN2OZ999qx7CTD2TYyQfmlRC79dzE7U/P3iY+6NyVrPu0mAuP68eYB7py8fcWV1t/+c0LmfJa+5yP05IUlwSXjFjAb16fyV1/mMUfH+7KvA9bVytz1pVL+dWED/jVhA/4zxsWcfCAdTknxE/ml/Lds/puEx//xO60262MB/8ygzMuXcqoW3tUW//AzT3pf+Ka7T+xJkwVuS1Al6QLXLlclsPurwN+Jmk+cAdwYxLvAcxPK7cgiWWKL6glnlFDdp8fJoemalP0/uR2rF1dvF3bDjxzJff88UPue/kDrvnpfIqKcvuVeszJnzLh6VRL5s0XOnLYcWup/HV8zKDVLJlXysezWmfYQ8vVuVsZ+33+MwDatKugd9+NLF/Sqs7yrz/fiRNOX1n1/dVnO3PdaQdw9UkH8svre1GeY+Nx0ssd+erZqf0cd9oq3p3YvmoY7a8v7Ub33pvpfcDG7TupJi6PpLg8IvqnLffnsPsrgW9FRC/gW8CDDXgq22iwpBgRbwArsxZspg76wnpGTviAHz36D/beP/Ufstd+G/nyv67mW6f3ZdjJB1JRDgPPXJXT/rp038KyRan/yBXlYv2aYjp0Kqd1m3L+/aqlPHbXjnXPW4pP5pfyj/fbcODh62tdv/Ez8fbrHTj21NUAzJvdmjfGduKO52bxqwkfUFRMVTc7mxVLWtF1r1R3ubgE2nQoZ82qYj5bX8Qz93bj699enGUPzVRQbxMtdRgKjEk+P01qnBBgIdArrVzPJJYp3rOWeEaNPqaYNKcvA2hNm0auTW7mTGvD+Uf1Y+OGYo4cuIYRo+Zy0XH9OPy4tfT9/AZ+OW4WAKWtg9XLUz/i7/92Lt17b6KkVbBHjy3c9/IHADz32668/NTudR7r/OFL+P0DXdm4Yftari3JZ+uLuPXSfbnsBwto0772C+kmv9yRfv3XV3Wd353YnjnTduW6U1Njx5s2FrFblzIAfnjxvnwyr5QtW8SyhaVcfVKqzL9espST/6Pu3/eP37knp1+6lF3bFu7FfA18Sc4i4MvA68BAoHK8aSxwtaTRpCZVPo2IxZLGA7elTa6cDNwYESslrZE0AJgMXAD8MtvBGz0pJs3p+wE6qHOzuPppw7qtCWrKqx24+ragQ6cyEEx4ujMP/WSvbba55ZJ9gNSY4vC753H92dXHqJYvaUXXvbawfHEpRcVB26TVceDhGzjutNVc/L1FtOtQTlSIzZvE2Ie7NuxJNjNlW+DWS/flhDNWVrUCa/PG2E58Oa3rHAFfOXslF964aJuy///Bj4BU6/Oub+3NT5+pPha8e/ctLFtUSpe9tlBeBhuS1v2sv7dl4gsdGXVrD9avKUZFULpL8P8uXLbNMZqtevqfKukJUhMlXSQtIDWLfCnwC0klwEaSRhMwDjgVmANsAC4ESJLfD4EpSblbKiddgGGkhvJ2JTXBknGSBZpAUmyOOnXdwqplJYA44LD1FBXBmlXFvDOxPTc/9BFjHujKpyta0b5jGbu2rWDpwtKs+5z0cgdOOnslM99uy/Gnrebdv7QHxPAztybP8769mI3ri50Qa4iAnw/fm177beTMy+ue5V2/pohpk9rxnV/+syp22HFrueXCfTnj0qV07FLG2lXFbFhfTLeem+vcT6WjT17Nn57uzEH91zPxhU4ccuxaJPjZ7z+sKvPYnXuya9vygkqI9XnxdkScW8eqL9RSNoCr6tjPKGBULfGpwMH51MlJsRY33PtPDjlmHbt1LuOxqdN59I7ulLRK/St44dEuHH/aar52wQrKy1PdrR8P6wOIebNb88jte/LjJ/6BBOVl4lff65lTUnxp9O5cf8/HPDRxBmtXl3DbsL0b9iQLyIwpbXn12d3pc9BnVV3coTcsqvq5n3bBcgD++mJHjvjSGlq32dqt7b3/Rs6/fjH/fe5+VIQoKQmG3Tovp6Q46JwV3HFNHy4+th/tO5bz3fvmNsDZNUERBf2QWcX2D4Zm3nFasxj4BBgRERlnkTqocxxd9NUGqY81jHEL3m7sKlgeBgxewNvvbsp6V0cm7Tv2jMO/dG1OZd/8w/VvR0T/HTneztZgLcUMzWIza+YK+d5nd5/NLD8BFHD32UnRzPJXuDnRSdHM8ufus5lZmkKefXZSNLP8+BWnZmZbpS7eLtys6KRoZvkr3Nu6nRTNLH9uKZqZVfKYoplZusK+99lJ0czy5+6zmVkiql41UJCcFM0sf24pmpmlKdyc6KRoZvlTReH2n50UzSw/gS/eNjOrJMIXb5uZVeOkaGaWxknRzCzhMUUzs+o8+2xmViXcfTYzqxIUdFIsauwKmFkzVJHjkoWkUZKWSnq/Rvybkj6QNF3S7WnxGyXNkTRL0qC0+OAkNkfSDWnxfSRNTuJPSirNVicnRTPLmyJyWnLwMDC42r6lE4EhwKER8TngjiTeDzgH+FyyzX2SiiUVA/cCpwD9gHOTsgA/Be6OiP2AVcDF2SrkpGhm+YvIbcm6m3gDWFkjfCXwk4jYlJRZmsSHAKMjYlNEzAXmAEcly5yI+CgiNgOjgSGSBAwEnkm2fwQ4PVudnBTNLD8RUF6R27J99geOT7q9f5Z0ZBLvAcxPK7cgidUV3x1YHRFlNeIZeaLFzPKX+0RLF0lT077fHxH3Z9mmBOgMDACOBJ6StG/+ldw+Topmlr/ck+LyiOif594XAGMiIoC3JFUAXYCFQK+0cj2TGHXEVwAdJZUkrcX08nVy99nM8hNAReS2bJ/ngBMBJO0PlALLgbHAOZJ2kbQP0Bd4C5gC9E1mmktJTcaMTZLqa8BZyX6HAs9nO7hbimaWp4ConztaJD0BnECqm70AGAGMAkYll+lsBoYmCW66pKeAGUAZcFVElCf7uRoYDxQDoyJienKI7wKjJf0I+DvwYLY6OSmaWX6CHZlEqb6riHPrWHVeHeVvBW6tJT4OGFdL/CNSs9M5c1I0s/wV8B0tTopmlj8nRTOzSn4ghJnZVgH40WFmZmncUjQzqxT1NvvcFDkpmll+AqKerlNsipwUzSx/23+3SpPnpGhm+fOYoplZIsKzz2Zm1bilaGZWKYjy8sauRINxUjSz/FQ+OqxAOSmaWf58SY6ZWUoA4ZaimVki6u8hs02Rk6KZ5a2QJ1oUTWhqXdIy4OPGrkcD6ELqHRPWfBTq39neEdF1R3Yg6SVSP59cLI+IwdmLNR1NKikWKklTt+ONZtaI/HfWcvltfmZmaZwUzczSOCnuHPc3dgUsb/47a6E8pmhmlsYtRTOzNE6KZmZpnBQbkKTBkmZJmiPphsauj2UnaZSkpZLeb+y6WONwUmwgkm3pOgkAAAO4SURBVIqBe4FTgH7AuZL6NW6tLAcPA83qYmOrX06KDecoYE5EfBQRm4HRwJBGrpNlERFvACsbux7WeJwUG04PYH7a9wVJzMyaMCdFM7M0TooNZyHQK+17zyRmZk2Yk2LDmQL0lbSPpFLgHGBsI9fJzLJwUmwgEVEGXA2MB2YCT0XE9MatlWUj6Qngf4EDJC2QdHFj18l2Lt/mZ2aWxi1FM7M0TopmZmmcFM3M0jgpmpmlcVI0M0vjpNiMSCqX9I6k9yU9LanNDuzrYUlnJZ9/m+lhFZJOkPTF7TjGPyVt89a3uuI1yqzL81g3S/qvfOtoVpOTYvPyWUQcFhEHA5uBK9JXStqu93hHxCURMSNDkROAvJOiWXPkpNh8vQnsl7Ti3pQ0FpghqVjSzyRNkfSepMsBlPKr5PmOfwL2qNyRpNcl9U8+D5b0N0nvSnpFUh9SyfdbSSv1eEldJT2bHGOKpGOTbXeX9LKk6ZJ+CyjbSUh6TtLbyTaX1Vh3dxJ/RVLXJPYvkl5KtnlT0oH18cM0q7RdLQtrXEmL8BTgpSR0BHBwRMxNEsunEXGkpF2Av0h6GTgcOIDUsx27ATOAUTX22xV4APhSsq/OEbFS0q+BdRFxR1Luf4C7I2KipN6k7to5CBgBTIyIWySdBuRyN8hFyTF2BaZIejYiVgBtgakR8S1J30/2fTWpF0pdERGzJR0N3AcM3I4fo1mtnBSbl10lvZN8fhN4kFS39q2ImJvETwYOqRwvBHYD+gJfAp6IiHJgkaRXa9n/AOCNyn1FRF3PFfwq0E+qagh2kNQuOcaZybYvSFqVwzldI+mM5HOvpK4rgArgyST+GDAmOcYXgafTjr1LDscwy5mTYvPyWUQclh5IksP69BDwzYgYX6PcqfVYjyJgQERsrKUuOZN0AqkEe0xEbJD0OtC6juKRHHd1zZ+BWX3ymGLhGQ9cKakVgKT9JbUF3gD+Ixlz3BM4sZZtJwFfkrRPsm3nJL4WaJ9W7mXgm5VfJFUmqTeAryexU4BOWeq6G7AqSYgHkmqpVioCKlu7XyfVLV8DzJV0dnIMSTo0yzHM8uKkWHh+S2q88G/Jy5d+Q6pH8HtgdrLud6SeBFNNRCwDLiPVVX2Xrd3XPwBnVE60ANcA/ZOJnBlsnQX/AamkOp1UN3pelrq+BJRImgn8hFRSrrQeOCo5h4HALUn8G8DFSf2m41c8WD3zU3LMzNK4pWhmlsZJ0cwsjZOimVkaJ0UzszROimZmaZwUzczSOCmamaX5P407sZrV1Z1+AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XLEuK_HHrUwV"
      },
      "source": [
        "##### Multinomial Naive Bayes w/ Tf-idf Precision-Recall Curve"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IpVWJB_3rUwV",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 409
        },
        "outputId": "cb35c56b-2835-4947-972d-c91626f1865f"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "from sklearn.metrics import precision_recall_curve\n",
        "\n",
        "def create_pr_curve(model_name, predicted, actual):\n",
        "    \n",
        "    precision, recall, thresholds = precision_recall_curve(predicted, actual)\n",
        "    closest_zero = np.argmin(np.abs(thresholds))\n",
        "    closest_zero_p = precision[closest_zero]\n",
        "    closest_zero_r = recall[closest_zero]\n",
        "\n",
        "    plt.figure(figsize=(6,6))\n",
        "    plt.xlim([0.0, 1.01])\n",
        "    plt.ylim([0.0, 1.01])\n",
        "    plt.plot(precision, recall, label='Precision-Recall Curve')\n",
        "    plt.plot(closest_zero_p, closest_zero_r, 'o', markersize = 12, fillstyle = 'none', c='r', mew=3, label='Decision Threshold = 0')\n",
        "    plt.xlabel('Precision', fontsize=16)\n",
        "    plt.ylabel('Recall', fontsize=16)\n",
        "    plt.title(model_name)\n",
        "    plt.legend(fontsize=13)\n",
        "    plt.axes().set_aspect('equal')\n",
        "    plt.show()\n",
        "    \n",
        "create_pr_curve('Multinomial Naive Bayes (Tf-idf) Precision-Recall Curve', nb_tfidf_preds, y_test)"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAGICAYAAACjo4zRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3gVZdrH8e+d3hMSQkuFAAmoIEoVVARFisqirEpTlBVdV1/XFVHZtawFV1F37WVdRQVsYMUgqIii0hVQSuiQUAOSkARC2vP+MSchCamQnDnn5P5cFxcnM3Nm7jntNzPPzDxijEEppVTT5mV3AUoppeynYaCUUkrDQCmllIaBUkopNAyUUkqhYaCUUgoNgzIiYkSkfQ3j14lIfyeWVO/l1rYOp1nHGBFZ0Bjzdgci8qOIdKvjtCNEJF1Ecis/R0TiHcO9q3nuQyIyo7p5ichyETnj9Nbm9NXlc1nburobERkvIj+U+7vRvm92cPswEJEdIlIgIs0rDf/F8WYlnsI8p4vIo+WHGWPOMMYsOq1iT0FDLVdEFolIvojElRt2sYjsqGMdM40xg063jirqmu54/3JFJEdEVonIhQ29nNMhIpcDOcaYX0TkFUetuY66C8v9Pc/xlKeA24wxIcaYX8rPyxizyzG8uI6Lrzyvp4CHa6h1vIgUO+o5IiKrReSyU1jtGtXlc3kK61pnjtAsfe2zROQnEenT0Ms5HSJyqYh87/hcZ4rIdyJyhd11Vcftw8BhOzCq9A8ROQsIsq8cl5UH3G93EVV40hgTAoQBLwMfudjW5C3AOwDGmFscP3AhwFTg/dK/jTFDHNMnAOsaaNmV5/UZcJGItKrhOUsc9UUA/wM+EJFmlScSEZ8GqtEu7zvWsznwLfChzfWUEZGRWPW8DcQCLYEHgMtPYV4iIo3+W+0pYfAOcF25v6/HehPKOLaM/1Tu7wq7fOWGTwTGAJMdWx2fO4bvEJGLHY8fEpEPRORtR+qvE5Hu5ebRybG8LMe4K8qNmy4iL4nIPMf8fxSRViLyHxE5LCIbyx9aqLTcniKyxDHfvSLygoj41eN1eg4YJSJJVY0UkXtFZKtjndaLyIiqXi8ReVlEnqr03E9F5G+Ox21EZI5ja2i7iPxfXYoz1uXws4BIrC8PIpIkIgtF5JCIHBSRmSIS4Rh3t4jMqVTHcyLyrONxuIj8z/Fa7RaRR0tDRkTaO7bUsh3zfb+a18QPGAB8V1v9IuIvIrmAN7BGRLZWMU2iWHusPo6/2zrqyBGRr7B+2KqdlzEmH1gFXFqH17MEeAMIBJIcn9vZIjJDRI4A42t6jRx13CQiG8p9Js5xDK/8uVwp1p7IfhF5ppp1bSMin4nI7yKyRURuKrecGr9TtaxnETATiBGRaMf8TnW9qv0O1JWICPAM8Igx5nVjTLYxpsQY850x5qZy61v+cGDl12qRiDwmIj8CR4G7RWRlpeXcKSKfOR77i8hTIrLL8R68IiKB9anbU8JgKRAm1o+wN3AtMKOW51TJGPMa1gfrScfWXnVJfgXwHtbW12fACwAi4gt8DiwAWgC3AzNFJLncc68G/oH1xT8OLAF+dvw9G+uDVJVi4E7HdH2AgcCt9Vi93cB/gX9WM34rcD4Q7phmhoi0rmK6d4FrHB96xNrqHAS8J9YWzOfAGiDGUeNfRaTWHy/He3cd1p7e/tLBwONAG6ATEAc85Bg3AxhcLhx8sN770g2B6UAR0B7o5qixdIPgEaz3qBnWltvz1ZTVASgxxmTUVr8x5rhjSxWgqzGmytCtZBbWj3tzR03X12FeG4Cutc3Y8Xr8CcgFNjsGD8f6jEVgfc6nU81rJCJ/xHqtr8Paa7sCOFTFop4FnjXGhAFJwAfVlPQekIH1Xo4EporIgHLjq/xO1WE9/Rw1HgIOOwaf6nrV9TtQk2Ssz+nsej6vsnHARCAUeAVIFpEO5caPxvr8APwL6AicjbXOMVh7InXmKWEAJ/YOLsH6suxu5OX9YIxJdRwPfYcTX87eQAjwL2NMgTFmITCXcoexgI+NMascW3kfA/nGmLcd83of68N7EsdzlhpjiowxO4BXgfoeX38cuFyqaIQ0xnxojNnj2Ip5H+sHpGcV81gMGKwvDVhf7CXGmD1ADyDaGPOwY/23YQXQtTXUNElEsrB+tP4D3F96nNkYs8UY85XjxzETKygvdIzbC3wP/NExn8HAQWPMKhFpCQwF/mqMyTPGHAD+Xa6OQqxDMG2MMfnGmJP2Eh0igJwaaj9lIhKP9Xrd71i/77GCtDY5jrqq09vxeu7D+tyNMMZkO8YtMcZ84thrCKPm1+hPWBtFK4xlizFmZxXLKwTai0hzY0yuMWZpFesaB/QF7nG83quB16m4R1/dd6o6VzvW8xhwEzDSGFNUh/e+2vWqx3egJlGO//fW83mVTTfGrHN837OBT3H8jjhCIQX4zLFRNhG40xjzuzEmB+sQZk3fuZN4WhiMBsZT6RBRI9lX7vFRIMCxJdYGSHd82UrtxErqUvvLPT5Wxd8hVEFEOorIXBHZ59jNn4rjsEJdOX5QX6CKRkgRuU6sBscsx5fszKrm7zic8x4nAm401lYmOH5gS+fhmM8UHId9qvGUMSYCq52nOzBNRIY4amopIu85dvWPYO0NlK/pLWCs4/FYHMf2HXX4AnvL1fEq1t4awGSsvY7ljkMSN1ZT22GsLbNTIical3MdP/7ltQEOG2Pyyg2r6se2slAgq4bxS40xEcaY5saY3saYr8uNSy/3uLbXKA5rS7k2E7C2SjeKyAqpusG6DVD6Q1Wq8veiyu+UWGeyVW6kB/jA8blpCfwGnHu661XX70AtSvcy6rtHUVl6pb9nUfE794kx5igQjfXdWVWu7i8dw+vMY8LAkezbsbYIPqpikjwqNirX1AB3Ordy3QPEScUGn3gaZk/lZWAj0MGxSz4F6wetvqYBF3Hiy4OIJGBtwd8GRDm+ZL/VMP93gZGO5/UCSo/dpwPbHT9Gpf9CjTFDayvKsZX2G/AjMMwxeCrW+3GWY53HVqrpE6CLiJwJXMaJUErHOgTXvFwdYcaYMxzL2meMuckY0wa4GXhJqj5NcIv18khMFeNqZU40LocYY3ZVGr0XaCYiweWGVQ6MqnTCOgx3SiWVe1zja+QYX+uhLmPMZmPMKKwf2yeA2ZXWCazvRaSIlA/WOn0vjHUmW+VG+vLjD2JtGT/kOKRzSut1Ct+B6qQ5lnFVDdPU5feo8u/QV0C0iJyNFQqlh4gOYm1EnlFufcPNicOMdeIxYeAwARhQaUur1GrgShEJcnzpJ9Qwn/1Au1OsYRnWVs1kEfEV61zsy7G2pE9XKHAEyBWRFODPpzITY0wW8DTW1nGpYKwPXyaAiNyAtVVU3Tx+wfoQvg7Md8wTYDmQIyL3iEigiHiLyJki0qMutTnWqx8nzqAJxTp8lO34Qb67Uh35WMdmZwHLS39wHYeQFgBPi0iYiHiJ1Rh9oWM5fxSRWMdsDjvWvfzeXOn8C4Cvqf/huFo5NmBWAv8UET8R6UctZ5uISABWiH/VAMuv8TXCem8nici5Ymnv+MGsXNNYEYl27A2Xfg4qvJbGmHTgJ+BxEQkQkS5Y38FTaturYl3SgPnA5NNYr3p9B2qoxQB/A+4XkRvK1dBPRF5zTLYauECsazHCgfvqMN9CrDOUpmGdZPGVY3gJVoj9W0RaOGqPkTq005XnUWFgjNlqjFlZzeh/AwVYP/RvcWILsir/Azo7drk+qWcNBVhf6CFYP5YvAdcZYzbWZz7VmIS1e5iD9eZXeQZMHT2L1SANgDFmPVZALMF6jc7C2kKvySzgYk5soeA43nsZVkPWdk4ERngN8yk9cysP60v8JtZuPViNeOcA2cAXVL3X95aj3ncqDb8O8APWY/3gz+bErnsPYJlYZ+x8BtxhrPaNqryK1ZjXGEZj7Vn9DjxI7Yc4LwcWGat9piFU+xoZYz4EHsN6f3Ow9sIiq5jHYGCd47V8FrjWGHOsiulGAYlYewkfAw9WOoR1uqYBEx0/iPVer1P8DlTJGDMbuAa4EWt99wOPYh33xxjzFdb3dy3WCQRz6zjr0u/ch8Y6i6rUPVh7sUsdh1O/xmrIrjMx2rmNcnOOY/EbgVbGmCONtIwfsS7++qXWiRuRiCwDJjgOpynVYDQMlFtztM08A4QZY6prBFZK1cLdr0BUTZijkXI/1lkpg20uRym3pnsGSimlPKsBWSml1KlxahiIyBsickBEqmz8cpzm9ZxY9y1ZK477hSillGpczm4zmI519Wt1p88NwboXTAes0+1edvxfo+bNm5vExMSGqVCp6mRnw5Yt1mMRSE6G4MrXVgF5eZCWBqWHYNu3h/CazqxVquGtWrXqoDGmzlchOzUMjDHfS839CwwH3nZctLFURCJEpLXjIpJqJSYmsnJldZcXKNVAhg07EQZ33QXTppWNKiwuwde73I72pEnw9NPW4+RkmFvX08iVahgiUpdbm5RxtTaDGCrejyODivcuUco+P5a7/mjCiQvY//3VJi577gcqnIxRbnyF5ynlolwtDOpMRCaKdQ/1lZmZmXaXo5qCnHL3WEs6cWub+Mgg0vbnsGTboSrHc6RRroNTqkG5WhjsxrqjYKlYqrmRlTHmNWNMd2NM9+joet2cT6lTE1ruHmtbT9z0cliX1kQE+TJz6a4qxxMW5oTilDo9rhYGnwHXOc4q6g1k19ZeoJTT9O174vH//lf2MMDXm5HnxDJ/3T4O5OSfNL7C85RyUc4+tfRdrJtAJYtIhohMEJFbROQWxySpwDasGy79l/r14qVU4/pzuZvEvvACLFlS9ueY3gkUlRg+WJFuDX/hhaqfp5SLcvbZRKNqGW+AvzipHKXqZ8gQ6NwZ1q+H/Hy46CK47TaYMIG2SUlcHZxD5EN/xyz/DDl+3HpO584wWO+UoVyfR9yOonv37kZPLVVOkZZmHfY5VFVXwJVERcFPP0HHjo1fl1KViMgqY0z3uk7vam0GSrm25GTrVNHOnWuernNnDQLlVjQMlKqv5GRYu9a6kGzYMIiIAC8viIhgW88LuXHkg6QvWqpBoNyKhoFSp8Lb2wqCuXPh8GEoLobDhwmYP49F7Xvw7qqG6PJaKefRMFCqAbWJCGRgp5Z8sDKdgqKTulRWymVpGCjVwMb0iudgbgHz1+2zuxSl6kzDQKkGdkGHaOIiA5mxtF73CVPKVhoGSjUwLy9hdM8Elm3/nS0Hcmp/glIuQMNAqUZwdfdY/Ly9mFH+fkVKuTANA6UaQVSIP0POasWcnzM4WlBkdzlK1UrDQKlGMqZXAjn5Rcxdo/daVK5Pw0CpRtIjsRkdW4YwY5k2JCvXp2GgVCMREcb0SmBtRjZrM7LsLkepGmkYKNWIRpwTQ6Cvd8WOb5RyQRoGSjWisABf/tCtDZ+u2U32sUK7y1GqWhoGSjWyMb0SyC8s4eOfM+wuRalqaRgo1cjOjAmna1wEM5btwhP6D1GeScNAKScY0yueLQdyWb79d7tLUapKGgZKOcHlXdoQFuDDjGXakKxck4aBUk4Q6OfNyHPj+PK3vWTmHLe7HKVOomGglJOM7hVPYbHhw1Xpdpei1Ek0DJRykvYtQujTLopZy3ZRXKINycq1aBgo5URjeseTcfgY32/OtLsUpSrQMFDKiQZ1bkXzEH9masc3ysVoGCjlRH4+XlzbI46FGw+wO+uY3eUoVUbDQCknu7ZnHAZ4b7meZqpch4aBUk4W2yyIAckteG9FOoXFJXaXoxSgYaCULcb0jicz5zhfrd9vdylKARoGStniwo4tiIkIZIY2JCsXoWGglA28vYTRveL5aeshtmbm2l2OUhoGStnl6u5x+HoLs/R+RcoFaBgoZZPoUH8uPaMVs1dlkF9YbHc5qonTMFDKRmN6JZB9rJC5a/faXYpq4jQMlLJR73aRtG8Rog3JynYaBkrZSEQY0yue1elZ/LY72+5yVBOmYaCUza48J5YAXy9makOyspGGgVI2Cw/05Yqubfh09W5y8gvtLkc1URoGSrmAMb0SOFpQzCe/7La7FNVEaRgo5QK6xkVwVkw4M5buwhjt+EY5n4aBUi5ibO940vbnsHLnYbtLUU2QhoFSLuLyrm0IDfDRjm+ULTQMlHIRQX4+XHVOLKm/7uNQ7nG7y1FNjIaBUi5kdK94CopLmL0qw+5SVBOjYaCUC+nYMpSebSOZtXwXJSXakKycR8NAKRcztncCOw8dZfGWg3aXopoQDQOlXMylZ7QkKthPG5KVU2kYKOVi/H28ubpHHF9v2M/e7GN2l6OaCA0DpVzQ6J7xGOC95el2l6KaCA0DpVxQXGQQF3aM5r0VuygsLrG7HNUEaBgo5aLG9kpg/5HjfLNhv92lqCZAw0ApF3VRSgvahAfora2VU2gYKOWivL2EUT3jWbz5INsP5tldjvJwGgZKubBresTh4yW8u1z3DlTj0jBQyoW1CAtg0Bkt+XBlOvmFxXaXozyYhoFSLm5srwQOHy0k9de9dpeiPJiGgVIurk9SFO2aB2tDsmpUTg8DERksImkiskVE7q1ifLyIfCsiv4jIWhEZ6uwalXIlIsLoXvGs2nmYDXuP2F2O8lBODQMR8QZeBIYAnYFRItK50mT/AD4wxnQDrgVecmaNSrmikefG4u/jxcxler8i1TicvWfQE9hijNlmjCkA3gOGV5rGAGGOx+HAHifWp5RLigjy47Iubfj4593kHi+yuxzlgZwdBjFA+ZutZDiGlfcQMFZEMoBU4PaqZiQiE0VkpYiszMzMbIxalXIpY3vHk1dQzCe/7La7FOWBXLEBeRQw3RgTCwwF3hGRk+o0xrxmjOlujOkeHR3t9CKVcraz4yLo3DqMmct2YYx2fKMalrPDYDcQV+7vWMew8iYAHwAYY5YAAUBzp1SnlAsTEcb2TmDD3iP8vCvL7nKUh3F2GKwAOohIWxHxw2og/qzSNLuAgQAi0gkrDPQ4kFLA8LPbEOLvow3JqsE5NQyMMUXAbcB8YAPWWUPrRORhEbnCMdldwE0isgZ4FxhvdJ9YKQCC/X0Y0S2GuWv3cjivwO5ylAfxcfYCjTGpWA3D5Yc9UO7xeqCvs+tSyl2M6R3PO0t3MntVBjdd0M7ucpSHcMUGZKVUDVJahdE9oRmzlu+ipER3mlXD0DBQyg2N7Z3A9oN5/LT1kN2lKA+hYaCUGxp8ZiuaBflqQ7JqMBoGSrmhAF9vru4ex4L1+9l/JN/ucpQH0DBQyk2N7hVPcYnh/RXptU+sVC00DJRyUwlRwZzfoTnvLt9FUXGJ3eUoN6dhoJQbG9s7gb3Z+SzceMDuUpSb0zBQyo0NTGlBq7AA7fhGnTYNA6XcmI+3F9f2jOP7zZnsOnTU7nKUG9MwUMrNXdsjHi8RZi3XvQN16jQMlHJzrcIDuLhTCz5Ymc7xomK7y1FuSsNAKQ8wtncCv+cV8OVv++wuRbkpDQOlPEDfpOYkRAUxc6keKlKnRsNAKQ/g5SWM6RXP8h2/k7Yvx+5ylBvSMFDKQ4w8Nw4/Hy9m6f2K1CnQMFDKQ0QG+zHsrNZ89PNu8o4X2V2OcjMaBkp5kLG948k5XsRna/bYXYpyMxoGSnmQc+KbkdIqlBlLd6K9xar60DBQyoOICGN6J7BuzxHWZGTbXY5yIxoGSnmYEd1iCPbzZuZSbUhWdadhoJSHCfH3YXi3GD5fu4fso4V2l6PchIaBUh5obK8E8gtLmP1zht2lKDehYaCUB+rcJoxu8RHMXKYNyapuNAyU8lBjeyWwLTOPJdsO2V2KcgMaBkp5qGFdWhMR5Ksd36g60TBQykMF+Hoz8pxY5v+2jwM5+XaXo1ychoFSHmx0r3iKSgwfrEi3uxTl4jQMlPJg7aJD6Ns+ineXp1Ncog3JqnoaBkp5uLG9EtiddYxFaQfsLkW5MA0DpTzcxZ1b0iLUXxuSVY00DJTycL7eXlzbI45v0w6Q/vtRu8tRLkrDQKkm4Nqe8Qjw7nLdO1BV0zBQqgloExHIgJSWfLAynYKiErvLUS5Iw0CpJmJs73gO5hYwf90+u0tRLkjDQKkm4oIO0cRFBjJT+0hWVdAwUKqJ8PISRvdMYOm239lyIMfucpSL0TBQqgm5unssvt7CjKXakKwq0jBQqgmJCvFnyJmtmfNzBscKiu0uR7kQDQOlmpixvRPIyS/i8zV77C5FuRANA6WamB6JzejYMkQbklUFGgZKNTEiwpheCazJyObXjGy7y1EuQsNAqSZoxDkxBPp6M2Op7h0oi4aBUk1QWIAvw89uw2dr9pB9rNDucpQL0DBQqoka2zuBY4XFfPxzht2lKBegYaBUE3VmTDhnx0Xwvx+3c7xITzNt6jQMlGrC7hrUkfTfj/HOEm07aOo0DJRqws7vEM0FHaN5fuEWso9q20FTpmGgVBM3ZWgKOfmFvPDtZrtLUTbSMFCqiUtpFcbIc2N566ed2hNaE6ZhoJTib5ck4+UFT85Ps7sUZRMNA6UUrcIDmHh+Oz5fs4c16Vl2l6NsoGGglAJg4oVJNA/x47HUDRhj7C5HOZmGgVIKgBB/H/56cUeWb/+dr9bvt7sc5WQaBkqpMtf2iCMpOph/fbmRwuISu8tRTqRhoJQq4+PtxX1DOrEtM4/3VqTbXY5yIg0DpVQFAzu1oFfbSJ79ehM5+XohWlPh9DAQkcEikiYiW0Tk3mqmuVpE1ovIOhGZ5ewalWrKRIS/D+vEwdwCXv1um93lKCdxahiIiDfwIjAE6AyMEpHOlabpANwH9DXGnAH81Zk1KqWgS2wEw89uw+s/bGNfdr7d5SgncPaeQU9gizFmmzGmAHgPGF5pmpuAF40xhwGMMQecXKNSCpg0KJmSEnh6gV6I1hQ4OwxigPKtUhmOYeV1BDqKyI8islREBlc1IxGZKCIrRWRlZmZmI5WrVNMVFxnE+L6JzP45g/V7jthdjmpkrtiA7AN0APoDo4D/ikhE5YmMMa8ZY7obY7pHR0c7uUSlmoa/9G9PeKAvj8/bYHcpqpE5Owx2A3Hl/o51DCsvA/jMGFNojNkObMIKB6WUk4UH+XL7gA4s3nyQ7zbpHrgnc3YYrAA6iEhbEfEDrgU+qzTNJ1h7BYhIc6zDRnpKg1I2Gdc7gfjIIB5P3UBxid6mwlM5NQyMMUXAbcB8YAPwgTFmnYg8LCJXOCabDxwSkfXAt8DdxphDzqxTKXWCn48Xkwcns3FfDnO0v2SPJZ5wQ6ru3bublStX2l2GUh7LGMOIl35ib/YxFk26iEA/b7tLUrUQkVXGmO51nd4VG5CVUi5GRPjHsE7sP3Kc//2gR209kYaBUqpOuidGcukZLXl50VYyc47bXY5qYBoGSqk6u2dwCseLSnj2m012l6IamIaBUqrO2kWHMKZXPO8uT2fLgVy7y1ENSMNAKVUv/zewA0G+3vxr3ka7S1ENyKe2CUTkuvrM0Bjz9qmXo5RydVEh/tzSP4lp89NYtu0QvdpF2V2SagC1nloqIvXp7sgYY5x+zpmeWqqUc+UXFnPRU4toEerPx7f2xctL7C5JVVLfU0tr3TMA2p5GPUopDxTg682kQcnc9eEaPl+7h+FnV77fpHI3tYaBMWanMwpRSrmXEd1i+N8P25k2P43BZ7bC30cvRHNn2oCslDolXl5Wj2gZh4/x9k+6zeju6tKAvB2o6z0rjDEm6fRKUkq5i77tm9M/OZrnF27mj91jiQjys7skdYrq0mbwHXUPA6VUE3PfkE4MefZ7nl+4hfsv61z7E5RLqkubwXgn1KGUclPJrUL547lxvL1kB9f3SSQ+KsjuktQp0DYDpdRp+9ugjvh4efHkfL0QzV3V5TDRSUSkK5AMBFQepxedKdX0tAwL4KYL2vHcN5uZ0O8w3eKb2V2Sqqd6hYGjL+IvgN6lgxz/l29T0DBQqgm6+YJ2zFq2i6mpG/jg5j6I6IVo7qS+h4mmAlHABVhBMAIYAMzE6pqyZ4NWp5RyG8H+Pvztko6s2HGYBev3212Oqqf6hsGlWIGw1PF3hjFmkTHmOuBr4I6GLE4p5V6u7h5L+xYhPDFvI4XF9bmTjbJbfcOgNbDNGFMM5AOh5cZ9BAxrqMKUUu7Hx9uL+4aksO1gHu8u32V3Oaoe6hsG+4AIx+OdQJ9y49o3SEVKKbc2IKUFfdpF8ezXm8nJL7S7HFVH9Q2DHzjRePwO8KCIvCoiLwLTgPkNWZxSyv2ICFOGduJQXgGvfLfV7nJUHdX31NJ/Am0cj6dhNSZfAwQBnwG3N1xpSil3dVZsOH84uw2vL97OmF4JtIkItLskVYt67RkYY7YaYxY7HhcaY+4yxsQaYyKNMaONMYcap0yllLuZdGkyBnh6gfaX7A7qFQYi4isiwdWMCxYR34YpSynl7mKbBXFD30Q++iWDdXuy7S5H1aK+bQavA/+tZtyrjn9KKQXArf3bExHoy+OpG6mtV0Vlr/qGwUXAp9WM+wwYeHrlKKU8SXigL7cP6MAPWw7y3aZMu8tRNahvGLQADlQzLhNoeXrlKKU8zdjeCSREBfF46kaKS3TvwFXVNwwOAGdVM+4sQBuQlVIV+Pl4cc/gFNL25zBnVYbd5ahq1DcM5gL3i0iX8gNF5Czg78DnDVWYUspzDDmzFefER/DUgjSOFhTZXY6qQn3D4AEgC1glIj+JyAci8iPwM5AN/KOhC1RKuT8Rq7/kAznHeX3xdrvLUVWo73UGB4EewONYdy092/H/Y0APx3illDrJuQmRDDmzFa98t5UDOfl2l6MqqXdPZ8aYLGPMA8aYPsaYjsaY84wxDxlj9ERipVSNJg9OoaCohP98vdnuUlQlp9TtpYg0F5HLROR6EYl0DAsQEe1GUylVrbbNgxnbO4H3V6Sz5UCO3eWocup7BbKIyDQgA+u6gjeARMfoT7EakZVSqlr/N7ADQb7e/Gue9pfsSuq7JX8fcBvwMNCLE91egnUm0WUNVJdSykNFBvtx60Xt+XrDAZZs1bPRXUV9w+BPwMPGmKlYZxCVtwVIapCqlFIe7Ya+ibQJD2Bq6gZK9EI0l1DfMIjhRJeXlRUAVd7ETimlygvw9WbSpcn8ujubz9fusbscRf3DYDdwZtIdYuUAACAASURBVDXjugJ6ArFSqk7+cHYMZ7QJ48kv08gvLLa7nCavvmHwIfCAiPQtN8yISEfgLuC9BqtMKeXRvLyEvw/txO6sY7z10w67y2ny6hsGDwEbge+B0hOFPwR+xWoz+FeDVaaU8njntW/ORcnRvPDtFg7nFdhdTpNW3yuQjwH9gfHAT8DXwApgItaZRH9u2PKUUp7uvqGdyDtexPMLt9hdSpNW3+sMmgMlxph3jDFjjTGDgAlAc2AT8Ewj1KiU8mAdW4ZyTY843lm6g52H8uwup8mqNQxExF9EnhWRHGA/cEhE/uwYNxbYCkwD0oHBjVmsUsoz3XlxR3y9vXjyyzS7S2my6rJn8ABwO7AE60f/K+BZEXkeeBvrbqXDjTG9jDFfNVqlSimP1SIsgIkXtOOLX/eyaudhu8tpkuoSBtcALxljBhlj7jXGXAPcAvwFKxi6GGO0HwOl1Gm56fx2RIf6MzV1g/aXbIO6hEEc8HGlYR85/n/GGKOnACilTluwvw9/u6Qjq3YeZv66fXaX0+TUJQx8gcq3Fyz9W3u4Vko1mD+eG0uHFiE88WUahcUldpfTpNT1bKIYEWlX+g9oV9VwxzillDolPt5eTBnaie0H85i1bJfd5TQpPnWcbnY1wz+pYpj3KdailFL0T47mvKQo/vP1JkacE0NYgK/dJTUJdQmDGxq9CqWUchARpgztxGXP/8DLi7Zyz+AUu0tqEmoNA2PMW84oRCmlSp0ZE86V3WJ444ftjO2dQExEoN0leTztplIp5ZLuujQZAzw9Xy9EcwYNA6WUS4qJCOTGvm35ePVuftudbXc5Hk/DQCnlsm69KImIQF+9EM0JNAyUUi4rLMCXOwZ24Keth1iUppc1NSanh4GIDBaRNBHZIiL31jDdVSJiRKS7M+tTSrmW0b0SSIwK4vF5GyjSC9EajVPDQES8gReBIUBnYJSIdK5iulDgDmCZM+tTSrkePx8v7hmcwqb9ucxelWF3OR7L2XsGPYEtxphtjnsavQcMr2K6R4AngHxnFqeUck2Dz2zFuQnNeOarTeQdL7K7HI/k7DCIwer3oFSGY1gZETkHiDPGfOHMwpRSrqv0QrQDOcf57+JtdpfjkVyqAVlEvLB6S7urDtNOFJGVIrIyM1MblpTydOcmNGPoWa147fttHMjRgwYNzdlhsBvrltilYh3DSoUCZwKLRGQH0Bv4rKpGZGPMa8aY7saY7tHR0Y1YslLKVUy+NIXC4hL+/dVmu0vxOM4OgxVABxFpKyJ+wLXAZ6UjjTHZxpjmxphEY0wisBS4whiz0sl1KqVcUGLzYMb2TuD9FbvYtL/ynfXV6XBqGBhjioDbgPnABuADY8w6EXlYRK5wZi1KKff0fwM6EOzvw7/mbbS7FI9S11tYNxhjTCqQWmnYA9VM298ZNSml3EezYD/+clF7/jVvIz9tOch57ZvbXZJHcKkGZKWUqovx5yUSExHIY6kbKCnR21Q0BA0DpZTbCfD15u5Lk1m35wifrtld+xNUrTQMlFJu6YqubTgzJoyn5m8iv7DY7nLcnoaBUsoteXlZF6LtzjrG9J922F2O29MwUEq5rfOSmjMwpQUvLtzC73kFdpfj1jQMlFJu7d4hKeQVFPHcN3oh2unQMFBKubUOLUO5pkc8M5buZPvBPLvLcVsaBkopt3fnJR3w8/HiyS/1QrRTpWGglHJ7LUIDuPmCJOb9to9VO3+3uxy3pGGglPIIN13Qlhah/jz2hfaXfCo0DJRSHiHIz4e7BnXk511ZfPnbPrvLcTsaBkopjzHy3DiSW4byry83UlCk/SXXh4aBUspjeHsJ9w5NYeeho8xcttPuctyKhoFSyqP07xhN3/ZRPPfNZrKPFdpdjtvQMFBKeZTS/pKzjhXy0qItdpfjNjQMlFIe54w24YzoFsObP+4g4/BRu8txCxoGSimPNGlQMgI8vWCT3aW4BQ0DpZRHahMRyIR+bfn4l938mpFtdzkuT8NAKeWxbumfRGSwH1NT9UK02mgYKKU8VliAL3cM7MCSbYf4Nu2A3eW4NA0DpZRHG90rnrbNg3k8dSNFxXohWnU0DJRSHs3X24t7Bqew+UAuH6zMsLscl6VhoJTyeJee0ZLuCc145qtN5B0vsrscl6RhoJTyeCLClGGdOJh7nNe+32Z3OS5Jw0Ap1SScE9+MYV1a89r329h/JN/uclyOhoFSqsm459IUikpK+PdXeiFaZRoGSqkmIz4qiHG9E/lgZTpp+3LsLselaBgopZqU2we0J8Tfh8fnbbC7FJeiYaCUalKaBftx24D2LErL5MctB+0ux2VoGCilmpzr+iQSExHI1NQNlJTobSpAw0Ap1QQF+HozeXAy6/Yc4ZPVu+0uxyVoGCilmqTLu7ShS2w4T81PI7+w2O5ybKdhoJRqkry8rB7R9mTn88aP2+0ux3YaBkqpJqt3uygu7tSCl7/dyqHc43aXYysNA6VUk3bvkBSOFhbz3Deb7S7FVhoGSqkmrX2LUK7tEcfMZbvYlplrdzm20TBQSjV5f724I/4+Xjz5ZZrdpdhGw0Ap1eRFh/pz84VJfLluHyt2/G53ObbQMFBKKeBP57elZZh/k+0vWcNAKaWAID8f7rokmV92ZZH66z67y3E6DQOllHK46txYUlqF8sSXGykoalr9JWsYKKWUg7eXcN/QTuz6/SjvLN1pdzlOpWGglFLlXNgxmvM7NOf5hZvJPlZodzlOo2GglFKV3DekE9nHCnnp2y12l+I0GgZKKVVJ5zZhXNktljd/3EH670ftLscpNAyUUqoKky7tiAg8taBpXIimYaCUUlVoHR7In85vy6er97A2I8vuchqdhoFSSlXjlguTiAr247EvPP9CNA0DpZSqRmiAL3+9uAPLtv/ONxsO2F1Oo9IwUEqpGlzbM552zYN5fN4Gioo990I0DQOllKqBr7cX9wxJYWtmHu+vTLe7nEajYaCUUrUY1LklPRMj+fdXm8g9XmR3OY1Cw0AppWohIkwZ1omDuQW89t1Wu8tpFBoGSilVB2fHRXBZl9a8tngb+7Lz7S6nwWkYKKVUHU2+NIXiEsMzX3nehWgaBkopVUfxUUFc3yeRD1dlsHHfEbvLaVBODwMRGSwiaSKyRUTurWL830RkvYisFZFvRCTB2TUqpVR1bhvQnlB/Hx5P3Wh3KQ3KqWEgIt7Ai8AQoDMwSkQ6V5rsF6C7MaYLMBt40pk1KqVUTSKC/Lh9QAe+25TJ4s2ZdpfTYJy9Z9AT2GKM2WaMKQDeA4aXn8AY860xpvQ2gUuBWCfXqJRSNbruvARimwUyNXUjxSWecZsKZ4dBDFD+qo0Mx7DqTADmVTVCRCaKyEoRWZmZ6TnprJRyff4+3kwenMKGvUf4+JfddpfTIFy2AVlExgLdgWlVjTfGvGaM6W6M6R4dHe3c4pRSTd7lXVrTNTacpxekcayg2O5yTpuzw2A3EFfu71jHsApE5GLg78AVxpjjTqpNKaXqTESYMrQTe7PzeePH7XaXc9qcHQYrgA4i0lZE/IBrgc/KTyAi3YBXsYLAs28TqJRya73aRXFJ55a8vGgrB3Pde7vVqWFgjCkCbgPmAxuAD4wx60TkYRG5wjHZNCAE+FBEVovIZ9XMTimlbHfvkBSOFRbz3Deb7S7ltPg4e4HGmFQgtdKwB8o9vtjZNSml1KlKig5hVM84Zi7bxfXnJZIUHWJ3SafEZRuQlVLKXfz14o4E+nrzxDz3vRBNw0AppU5T8xB/brmwHQvW72f59t/tLueUaBgopVQDmNCvHa3CAngs1T37S9YwUEqpBhDo583fBnVkTXoWc9futbucetMwUEqpBnLVObGktArlyfkbOV7kXheiaRgopVQD8fayLkRL//0Y7yzZaXc59aJhoJRSDeiCjtGc36E5zy/cQvbRQrvLqTMNA6WUamBThnbiSH4hL3zrPheiaRgopVQD69Q6jJHnxPLWTztJ//1o7U9wAU6/AtnZSkpKyMjIIC8vz+5SlHIJwcHBxMbG4uWl24KN6a5ByXy+dg9Pzk/j+VHd7C6nVh4fBgcPHkRESE5O1g+/avJKSkrYvXs3Bw8epEWLFnaX49FahQdw0/nteH7hFv7Ury1d4yLsLqlGHv/rmJWVRcuWLTUIlAK8vLxo2bIl2dnZdpfSJNx8YRLNQ/zc4kI0j/+FLC4uxtfX1+4ylHIZvr6+FBUV2V1GkxDi78MdF3dk+fbf+XqDa9+R3+PDAKxOKJRSFv0+ONe1PeJoFx3M4/M2UFhcYnc51WoSYaCUUnbx9fbiviGd2JaZx3sr0mt/gk00DJqoIUOG8OSTT9Y63eLFi4mIcO2Gr9OxY8cORISMjAwApk+fTvv27W2uSnmaizu1oGfbSJ79ehM5+a55IZqGgYvo378//v7+hISEEB4eTrdu3ZgzZ06jLW/evHlMnjy51unOP/98srKyGq0OgPHjx+Pr60tISAhhYWF06tSJl156qVGXeTr27t3Ln//8ZxISEggODiY+Pp6rr76aVatW2V2aclEiwt+HduJgbgGvfrfN7nKqpGHgQu6//35yc3M5dOgQo0aN4pprrmHTpk0nTVdY6JpbFqfj+uuvJzc3l6ysLB599FFuu+02Fi1aZHdZJ9mzZw89evQgPT2d1NRUjhw5wvr167n88sv56KOPTmmenvh+qpN1jYvgiq5teP2HbezLzre7nJNoGLggHx8fbr31VoqLi/n1119ZtGgRPj4+vPPOO7Rr147IyEgAdu3axciRI2nVqhWtW7dm4sSJ5OTklM0nMzOTCRMmEB8fT1hYGOeccw5paWmAtSfy6KOPAnD8+HEmTpxIixYtCAsLo0OHDnz44YcAZcsuVVRUxMMPP0y7du1o1qwZAwcO5LfffisbP378eMaNG8dNN91EREQEMTExvPrqq3Vedy8vL6666iqioqJYuXJl2fDFixfTr18/IiMjSUpK4umnn65wqt7atWsZPHgw0dHRREZGcvHFJ3pPveGGG4iLiyM0NJTOnTsza9asOtdT2QMPPEBwcDAff/wxZ5xxBt7e3oSEhDBu3Dgee+wxAB566KEKy4eKr3dV7+eLL77I2WefXeE527dvx9vbmx07dgC1v9/K9d19aTIlJfD0gjS7SzmJx190Vtk/P1/H+j1HnLKszm3CePDyM+r9vIKCAl588UV8fX3p2rUrGRkZFBcXk5qayi+//IKvry/5+fkMGDCA0aNH884775Cfn8+YMWO44447eOONNygpKeGKK66gTZs2rFixgujoaH777TdCQ0NPWt5bb73FihUr2LBhA1FRUaSnp1f7IzNt2jTefvttUlNTadu2LY8//jiXXHIJaWlphIWFATB79mzef/99Xn31VT755BOuueYaBg8eTEJCQq3rXlxczOzZszl48CDJyckArF+/nqFDhzJjxgwuu+wyNm/ezJAhQ4iOjua6665j7969XHjhhUyePJk5c+bg6+vL999/XzbPfv368dRTTxEREcGHH37Iddddx9lnn03nzp3r/d6kpqZy4403nvbpypXfz+PHj3PXXXexevXqslCYPn06/fv3JzExsdb3W7mHuMggrj8vgdd/2M4NfdvSuU2Y3SWV0T0DF/LYY48RERFBbGwsn376KXPmzKnQmPnEE08QHh5OUFAQc+fOxRjDww8/TGBgIM2aNeORRx5h5syZFBcXs3LlSlauXMkbb7xRdtFdly5daNOmzUnL9fPzIzc3l/Xr11NUVERcXFy1P5Rvvvkm99xzDykpKfj7+/PAAw/g7e3NF198UTbNgAEDuOKKK/Dy8uLKK68kIiKC1atX17ju77zzDhEREQQEBDBq1CgefvhhLr/8cgBeeukl/vjHPzJ8+HC8vb1JSUnhtttu4+233y57bvv27bnvvvsIDg7Gz8+vwpb5hAkTiIqKwtvbm2uvvZYuXbqc8iGozMxMYmJiTum5lZV/P5s1a8bw4cN58803ATDG8NZbb3HjjTcC1Pp+K/dx20UdCAvw5fF5G+wupYImt2dwKlvqzvL3v/+df/zjH1WO8/LyIi4uruzv7du3s2vXrpPO9BER9u3bx44dO2jRogXh4eG1Lnfs2LHs37+fO++8k82bNzNw4ECefPLJKs+qSU9Pp23bthXqSkxMJD39xClzrVu3rvCc4ODgsj2NM844g507rfu8T5kyhSlTpgAwbtw4Xn/9dY4ePcrkyZP55ptvuO+++/Dx8WH79u0sXLiwwjH5kpKSstdjx44ddOzYscp1Kykp4aGHHuL9999n3759iAh5eXlkZmbW+rpUJTo6mt27d5/Sc8ur/H6CdThr3LhxPPXUU3z//fdkZWVx5ZVXArW/3w0VUKrxhQf5cvuA9jz6xQa+35TJBR2j7S4J0D0DtyEiFS4WSkhIoGPHjmRlZVX4l5+fT0xMDImJiRw4cIAjR2o/JObj48M999zDypUr2blzJ0FBQWVbpJXFxcWVHcMG68d2x44dJ/2wVWfdunXk5uaSm5tbFgTlBQUF8cwzz7B7925efPHFsnW98cYbK6znkSNHWLduHQCJiYls3lz1rYLfffddXn/9debMmcPhw4fJysqia9eup3xrgKFDhzJ79uwaG31DQ0NPujHinj17Kvxd+f0EuOSSS/D39+fzzz9n+vTpXHvttQQGBgK1v9/KvYzrk0BcZCBTUzdQXOIat6nQMHBTl112GQUFBUydOpWcnByMMezevZuPP/4YgO7du3POOefwpz/9iQMHDlBSUsLatWtP+lECWLhwIatWraKwsJDAwECCg4Px9vaucrnjx4/nySefZNOmTRQUFPDYY49RVFTEsGHDGmzd/Pz8eOCBB3j00UfJycnh1ltv5b333uPzzz+nsLCQoqIi1q9fz3fffQdYezZpaWk88cQTHD16lIKCAr7++msAjhw5go+PD9HR0ZSUlPDGG2+wZs2aU67tn//8J7m5uYwcOZINGzZQXFxMXl4e7777btle3bnnnsvPP//MqlWrKCoq4oUXXmD79u21ztvb25vrrruO5557jo8++qhCINf2fiv34u/jzeRLU9i4L4c5P2fYXQ6gYeC2goKCWLhwIevXryclJYXw8HAGDhxYdmzey8uLzz//nMDAQM4++2wiIiK48cYbyc3NPWle+/fvZ9y4cTRr1ozWrVuzc+dOXnvttSqXe/fddzNq1CgGDRpEy5YtWbhwIQsWLChrPG4oo0ePJjIykqeffpozzzyTuXPn8p///IfWrVvTokULxo8fX3aop02bNixatIivvvqK2NhYWrVqxbRp0wDrlNVevXrRvn17YmJiWL9+Peeff/4p1xUTE8OKFSto3bo1gwYNKrsu4pNPPuGqq64CrDOH/va3vzF48GBat27N/v376du3b53mf8MNN/Ddd9/Rtm1bevbsWTa8tvdbuZ/LurTm7LgInl6QxrEC+9t9xNXvpFcX3bt3N+VPQyxvw4YNdOrUyckVKeXa9HvhGlbs+J0/vrKESYM6ctuADg06bxFZZYzpXtfpdc9AKaVs0iMxkkGdW/Lyoq1k5hy3tRYNA6WUstE9Q1LILyrh2W9OvtuAM2kYKKWUjZKiQxjTK553l6ez5cDJbXrOomGglFI2u2NgBwJ9vfnXvI221aBhoJRSNosK8efP/ZP4esN+lm07ZEsNGgZKKeUCbuzbltbhAUxN3UCJDReiaRgopZQLCPTz5q5ByazJyGbur3udvnwNg+oUF8PcuTBsGEREgLe39f+wYdZwvTmYUqqBjegWQ6fWYTz55UaOFzn3N0bDoCppadClC1x+OaSmQnY2lJRY/6emWsO7dLGmU0qpBuLtZfWIlnH4GG//tNOpy9YwqCwtDfr2hfXra55u/XprOjcKhKlTp5bdFro2ISEhLFmypJErqp6I8MMPPzhteeU7nzkVdek72cfHxyV7b1OupV+H5lzYMZrnF24m62iB05arYVBecTFceSUccrTmBwTApEmwYQMUFFj/T5oE/v7W+EOHrOkb4JBRaR/IoaGhhIeH065dO8aNG9eg/epOmTKFzz//vE7T5ubm0qdPnwZbdqldu3YREhJS9s/Hxwc/P7+yv884w3VvMe7KVq5cSc+ePQkKCiIpKYkZM2bYXZI6DfcNTSH3eBHPL9zitGVqGJQ3b96JPYKAAFi4EKZNg5QU8PW1/p82Db799kQgrF8PX37ZIIu///77ycnJITs7m2+//ZaEhAR69+7tUXemjI+PL7uFdW5uLv3792fKlCllf5felrq+mnI/wtnZ2QwZMoSrrrqKw4cP88orr3DLLbfYumenTk9KqzBGnhvL20t2sOvQUacsU8OgvJdfPvH4ttugui3jPn2s8VU9r4EkJCTw6KOPct1113H77beX3X//6NGjTJo0ibZt2xIZGcngwYPZsuXE1kNhYSFTp04lOTmZ0NBQkpKSmD17NnBy37zPPfccbdu2JTQ0lJiYmAr9C1Q+TDNnzhy6du1KeHg4Xbt2rRBQpYdInnvuOWJjY2nWrBk333zzaffAtXbtWnr06EFoaCi9e/dm48YTF+T079+fv/71r/zhD38gLCyMp59+GoD//ve/nHnmmYSHh9OtWzcWLFhQ9pxffvmFfv36ER4eTmRkJOeddx6HDx8uG3/48GGuuuqqstft008/rVDPyy+/THJyMuHh4fTu3ZvFixdXW3tOTg7XX389kZGRJCQk8NZbb53Wa1GTjz76iKCgICZPnoy/vz+XXHIJI0aMqPbOs8o9/O2SZHy8vHhyvpMuRDPGuP2/c88911Rn/fr11Y47SXi4MWD927Ch5mnXrz8xbURE3ZdRjQsvvNA88sgjJw1fsGCBAcwGRz2jR482w4YNM/v27TPHjx83DzzwgElOTjYFBQXGGGMmT55sOnXqZNasWWNKSkpMenq6WbNmjTHGmAcffNAMHDjQGGNMWlqaCQwMNL/99psxxpjDhw+bJUuWlC0XMIsXLzbGGPPjjz8af39/k5qaagoLC83cuXONv7+/Wbp0qTHGmDfffNP4+PiYKVOmmPz8fLN582bTrFkzM2PGjFrXe+DAgebBBx88aThgevToYXbu3Gny8/PNyJEjzcUXX1zh9QoNDTXffPONKSkpMXl5eea1114zSUlJZvXq1aa4uNh88cUXJjg42GzevNkYY0yfPn3MP//5T1NUVGQKCgrMkiVLTG5ubtn8oqKizI8//miKi4vNM888Y8LDw01eXp4xxphZs2aZqKgos3TpUlNYWGhef/11ExQUZHbs2FH2GiQlJZXVd+ONN5o+ffqYvXv3mqysLDNixAgDmG+//bbK12Hnzp0mPDy8xn/VueOOO8zw4cMrDHvmmWdMt27dqn1Ovb4XyjZPz99oEu6Za37e+Xu9nwusNPX4HdU9g/LKdwKflFTztOXH16E3sVMVGxsLwKFDhzh48CCzZs3ipZdeomXLlvj5+fHggw+yd+9eli1bhjGGF198kWnTptGlSxdEhNjYWLp06XLSfH18fDDGlPU8FhERQe/evausYfr06Vx11VUMGTIEHx8fhg0bxogRIyp0xB4YGMjDDz+Mv78/7du3Z+DAgVR3W/G6uvvuu4mPj8ff35/x48efNL+RI0cyYMAARISgoCCeffZZHnjgAbp27YqXlxdDhw7loosu4r333gOsTnN27dpFeno6vr6+9O7dm+Dg4LL5XXPNNZx33nl4eXkxceJEsrOzy3pQe/PNN7n55pvp1asXPj4+TJgwgS5dujBr1qyT6i4pKWHmzJk88sgjtGrVivDwcJ544oka1zU+Pv6kXswq/6tOTk7OSd2bRkRE1KmXO+XaJl6YRPMQP6ambjjl3vnqSsOgvNDQE4+3bq152vLjG7hjl/IyMqxekKKiosp6y+rSpQsRERFEREQQGRlJYWEh6enpZGZmkpeXV21/wOW1a9eOmTNn8t///pc2bdrQr1+/CodUyqvc7zFAUlJShX6PW7RoUaF3tPL9Hp+q8n0pVzW/xMTECn9v376dv/zlL2WvTUREBN9++21Zn8VvvvkmJSUl9OvXj7Zt23L//fdTVFRU7fKAsmXW5TUolZmZyfHjxyvUV/m5DSk0NJTs7OwKw7Kyshq8wyHlfCH+Ptx5SUdW7DjMgvX7G3VZGgblle+N6n//q3na8uPr2IvVqXj//feJiYkhOTmZhIQEADZv3lxhi/Ho0aOMGjWK6OhogoKCqu0PuLIrr7ySr776ioMHD3L11VczfPhwjh49ubGqcr/HANu2batzv8eNxcur4sc3ISGBN954o8Jrk5uby8uONp22bdvyxhtvkJGRwWeffcbrr7/O22+/Xadl1ec1aN68OX5+fhWmr/zcyiqfZVXVv+p07dr1pB7PfvnlF7p27VrreinXd033ONq3COGJeRspLC5ptOVoGJT35z+fePzCC1Dd2RhLlljjq3peA0lPT+fBBx9k+vTpPPvss4gILVq0YPTo0dx6661lW7tZWVl8/PHH5ObmIiLceuutTJ48md9++w1jDBkZGaxdu/ak+aelpfHll19y9OhRfH19CQ8PR0RO+oEFq+vIOXPmMH/+fIqLi5k3bx4fffQRN9xwQ4Ov9+m48847eeihh1i9ejXGGI4dO8YPP/xQ1vD81ltvlfUBHRERgY+PT7V9PVc2fvx4Xn31VZYvX05RURFvvvkmq1evZvTo0SdN6+3tzejRo3nwwQfZv38/R44c4d57761x/pXPsqrqX3VGjBhBXl4e06ZNo6CggG+++YaPPvqIiRMn1mndlGvz8fbi3sEpbDuYx7vLdzXacjQMyhsyBDp3th7n58NFF1V9ncFFF8FxR69EnTvD4MENsvhHHnmE0NBQwsLCuOCCC9iyZQs//fRTWd+6YJ0tk5ycTP/+/QkNDeWss87iww8/REQAeOyxx7j66qv5wx/+QGhoKP37969wtlGpgoICHn74aBqjYwAACpFJREFUYVq3bk1ERATPPfccc+bMISAg4KRp+/bty1tvvcWkSZNo1qwZkydPZsaMGdW2MdjlpptuYvLkydxwww00a9aM+Ph4HnnkkbLTThcuXMi5555LcHAwffr0YfTo0YwbN65O8y79cR87dixRUVG8/PLLpKamlu2tVfbss8/Stm1bUlJSOOuss7j88svrHDz1FRERQWpqKh9++CHh4eHcdNNNvPLKK41ynYiyx8BOLejVNpJnv95MTn7jnEatfSBXVnoF8qE63EY2Kgp++gnqcIxeKVeifSC7n7UZWVzxwo/85aIk7r40pdbptQ/k05WcDD/+eGIPoTqdO2sQKKWcpktsBMPPbsPri7ezJ+tYg89fw6Aqycmwdm3Fu5Z6eVW8a+natRoESimnmjQoGWPg6QUN31+yT4PP0VN4e1s//MOG2V2JUkoBEBcZxA19E3lt8TZu7JfIGW3Ca39SHemegVJKuZFbL2pPeKAvj6dubNAL0ZpEGHhCI7lSDUW/D+4tPNCX2wd04IctB/luU2aDzdfjwyAgIIBDhw7pF0AprCA4dOhQlacQK/cxrncC8ZFBPJ66keIG6i/Z49sMYmNjycjIIDOz4RJUKXcWEBBQds8r5Z78fLy4Z3AKf5n1M3NWZXB1j9O/G4DHh4Gvr2+j3hdGKaXsMPSsVnSLj+CpBWlc1rU1QX6n93Pu8YeJlFLKE4lY/SUfyDnO64u3n/b8nB4GIjJYRNJEZIuInHTDFhHxF5H3HeOXiUiis2tUSil30D0xksFntOKV77ZyICf/tObl1DAQEW/gRWAI0BkYJSKVL/WdABw2xrQH/g3UfCN4pZRqwu4ZkkJBUQn/+bpudyuujrP3DHoCW4wx24wxBcB7wPBK0wwHSvsInA0MlNK7sCmllKqgbfNgxvSK5//bu/tYOaoyjuPfn7SEQF9AawNWyK1pUUjR2BRC/cOXgEYbUoMFgtrgFQJIg6hIowbBWvnDV0w0EFpDRRRIISTNNbRB0ZYaoaZNgXLbVCnQlNKaBqUVLRRaHv84Z2Xd3nt37u3d2e7s75NMdufuzOxzzt2dZ2fOzDnL17/Atj0jH0Ok7GQwBagfDWRn/tuAy0TEQWAf8I5SojMz60DXnTed48cew/dXjXy85I69mkjSVUCtw/YDkvrbGc9RYBLwUruDaKNuLz+4Drq9/PTDpDt7/1cHA/evPoiyk8GLQP0Fse/OfxtomZ2SxgATgcP6k46IpcBSAEkbhtNVaxV1ex10e/nBddDt5Ycjq4OyTxOtB6ZLmirpWOBSoK9hmT7gC/n5RcAfw7cPm5m1VKlHBhFxUNK1wMPAMcCyiNgsaTGwISL6gDuBX0vaBvyTlDDMzKyFSm8ziIiVwMqGv91c9/w14OJhbnbpKITW6bq9Drq9/OA66PbywxHUQSWGvTQzsyPj7ijMzKyzkkG3d2VRoPzXS9oiaZOkP0ga1qVlnaBZHdQtN09SSKrc1SVF6kDSJfmzsFnSvWXH2EoFvgenSVot6Yn8XZjTjjhbRdIySXsGu5xeyc9y/WySNLPQhiOiIyZSg/OzwHuAY4GngDMbllkA3JGfXwosb3fcJZf/Y8Dx+fk1VSp/0TrIy40H1gLrgFntjrsNn4PpwBPASXl+crvjLrn8S4Fr8vMzge3tjnuU6+DDwEygf5DX5wCrAAHnAn8pst1OOjLo9q4smpY/IlZHxP48u450H0eVFPkMAHyP1KfVkfXcdXQqUgdXArdFxMsAEbGn5BhbqUj5A5iQn08EdpUYX8tFxFrSlZaD+TRwdyTrgBMlndJsu52UDLq9K4si5a93BenXQZU0rYN8SHxqRDxUZmAlKvI5OB04XdKfJa2T9MnSomu9IuVfBMyXtJN05eKXywntqDHcfQXQwd1R2OAkzQdmAR9pdyxlkvQ24Fagt82htNsY0qmij5KODtdKOisi9rY1qvJ8FrgrIn4iaTbpvqUZEfFmuwM7mnXSkcFwurJgqK4sOlSR8iPpfOBGYG5EHCgptrI0q4PxwAxgjaTtpPOlfRVrRC7yOdgJ9EXEGxHxPPA3UnKogiLlvwK4HyAiHgeOI/Vb1C0K7SsadVIy6PauLJqWX9IHgSWkRFCl88Q1Q9ZBROyLiEkR0RMRPaR2k7kRsaE94bZEke/BCtJRAZImkU4bPVdmkC1UpPw7gPMAJJ1BSgbdNAh6H3BZvqroXGBfROxutlLHnCaKLu/KomD5fwSMAx7I7eY7ImJu24IeZQXroNIK1sHDwCckbQEOAQsjohJHyAXL/3XgF5K+RmpM7q3Qj0Ik3UdK9pNyu8h3gLEAEXEHqZ1kDrAN2A98sdB2K1RHZmY2Qp10msjMzFrEycDMzJwMzMzMycDMzHAyMDMznAysIiT15l5Ka9Mrkp6SdG2+AbGMGHrye/cOY51a3D0tC8ysgI65z8CsoItJd+BOyM9/DkwGbh5qpVGyG5hN6lWzqIfyOk1vCjJrJd9nYJWQf43/EpgeEdvq/r4amBkREwdYZyxwsEo3JJmNlE8TWdWtByZIOiefjlkg6YeSdgEHgBMBJH0m9/C5X9JeSQ9IOq1xY5KulLRR0quSXpb0qKQP5dcOO00k6WxJv5f0j7zOc5Jur3v9sNNEksZKukXSdkmv58dbcvKi4b2ulrRY0u4c928lVa3rciuBk4FV3VRSlwz/zvM3kvrquQq4EHhN0peAB4EtpD6triZ1ePeopPG1DUn6MWnglI3AJcB80iA6hyWNvPw4UrcJh0g9qX4KWEzz07O/Ar4J3A1cANwFfIO3xuqo9y1gGnA58BXSKaffNNm+2eHaPWqPJ0+jMZF2tgG8l7SzPYm0Uz9E6ritJ7++kXx6NK83jjTuxbKG7U0FXge+muen5W3dOkQMtffozfOz8vz7C8Tdk+dn5PlFDct9u35bde+1pmG5G/Lf39Xu/4mnzpp8ZGBVsxV4g9RR4e3APaRfzTUrIqK+jWA2qbH5HkljahNpcJCtpCEGAc4nHUkvHUYszwB7gSWS5ks6tdkKde/X+Ou+Nt84RsXKhvmn8+OARytmg3EysKq5EDgbeB9wQkRcFhH1QwQ2XrUzOT8+Qkoi9dNZvDVSXu1xZ9FAImIfaVzqXaTEtENSv6R5Q6z29kHi/HvD6zWNwx/WxrA4rmicZuBLS616+qPuaqIBNF45VOvauRfYPMDyr+THl/LjFOCvRYOJiCeBefloYxbpHP/9kj4QEf0DrFLbuZ/M/1+ienLD62ajykcG1u0eI+3wp0XEhgGm2o7/EeBNUsPzsEXEwUiDk99E+t6dMciia/Nj41gcn8+Pa0by/mbN+MjAulpE/EvSQuA2Se8EVpEalKeQzs+viYh7I+JZST8Frs9XGPWRGpTPAbZGxPLGbUu6gJQ8VgDPAycA15GSz+ODxNOfBy9ZlI8mHiO1a9wE3BcRTw+0ntmRcjKwrhcRSyS9ACwEPkf6XrwI/Al4sm65G/IoegtIw6v+B9gE/G6QTT8DvErakZ9CSgLrgY9HxFBtD72kYSovJ11FtAv4AfDdkZXQrDnfgWxmZm4zMDMzJwMzM8PJwMzMcDIwMzOcDMzMDCcDMzPDycDMzHAyMDMznAzMzAz4L39E8jCDt6D5AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p8dlnYJCrUwW"
      },
      "source": [
        "#### Naive Bayes w/ Word2Vec Feature Representation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7pvagHt9rUwW"
      },
      "source": [
        "#Fit classifier to word2vec training data\n",
        "nb_clf_w2vec = nb_clf.fit(full_X_train_w2vec, y_train.ravel())\n",
        "#Create predictions on test set\n",
        "nb_w2vec_preds = nb_clf_tfidf.predict(full_X_test_w2vec)"
      ],
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pjQ-rfMVrUwW"
      },
      "source": [
        "#### Evaluate Multinomial Naive Bayes w/ Word2Vec Representation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FWMIqVr2rUwW"
      },
      "source": [
        "##### Multinomial Naive Bayes w/ W2Vec Accuracy, Precision, and Recall"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YFztCEYJrUwW",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ec918100-a228-4bd1-c721-eb07a635e684"
      },
      "source": [
        "from sklearn.metrics import accuracy_score, precision_score, recall_score\n",
        "\n",
        "print(\"Multinomial Naive Bayes (w2vec) Accuracy: \" + str(accuracy_score(nb_w2vec_preds, y_test)))\n",
        "print(\"Multinomial Naive Bayes (w2vec) Precision: \" + str(precision_score(nb_w2vec_preds, y_test)))\n",
        "print(\"Multinomial Naive Bayes (w2vec) Recall: \" + str(recall_score(nb_w2vec_preds, y_test)))"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Multinomial Naive Bayes (w2vec) Accuracy: 0.6347625788804376\n",
            "Multinomial Naive Bayes (w2vec) Precision: 0.700674232790268\n",
            "Multinomial Naive Bayes (w2vec) Recall: 0.6190667995166522\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3f8bIB2xrUwW"
      },
      "source": [
        "##### Multinomial Naive Bayes w/ w2vec Confusion Matrix"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BHIlZNx-rUwX",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "c023c916-7788-4095-99a0-3a1731ccb562"
      },
      "source": [
        "from sklearn.metrics import confusion_matrix\n",
        "from sklearn.metrics import ConfusionMatrixDisplay\n",
        "\n",
        "cm = confusion_matrix(y_test, nb_w2vec_preds)\n",
        "cm_display = ConfusionMatrixDisplay(cm, display_labels=[0, 1]).plot()"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3daZgdVbn28f+dTjrzSCBAEkiAMESUUYggCogQ0HOCiAoORESQUVHUo5zzEkVxRFE8EkWJDHJEkChRwRgRFdRAAoQhCZAIQkbIPE/d/bwfanWy0+nee1fSTXfvvn/XVVf2fmpV1apO8nRVrVprKSIwM7NMp9augJlZW+KkaGZWwEnRzKyAk6KZWQEnRTOzAp1buwKFuvTtEdWD+rZ2NSyHWNGm/glZCZvXLKdm4zrtyj5OO6lnLFteW1bZx5/eNDkiRu/K8V5vbepfdPWgvhz6g7GtXQ3LYcvEPVq7CpbDcxNv2OV9LFtey2OT9ymrbNVecwbu8gFfZ20qKZpZ2xdAHXWtXY0W46RoZrkEwZYo7/a5PXJSNLPcfKVoZpYEQW0Fdw92UjSz3OpwUjQzA7KGllonRTOzbXylaGaWBLDFzxTNzDJB+PbZzGyrgNrKzYlOimaWT9ajpXJ5lBwzy0nUlrkU3Ys0VNJDkmZJminpUyl+uKSpkmZImi7pmBSXpBslzZX0tKQjC/Y1VtKctIwtiB8l6Zm0zY2SSg6G4aRoZrlkDS0qaymhBrgqIkYCo4DLJI0EvgV8OSIOB65J3wFOB0ak5SJgPICkAcA44FjgGGCcpP5pm/HAhQXblRyxx0nRzHLJ3lPc9SvFiFgUEU+kz2uA2cDgdIg+qVhfYGH6PAa4PTJTgX6S9gJOA6ZExPKIWAFMAUandX0iYmpkM/TdDpxZ6vz8TNHMcqsrfRVYb6Ck6QXfb46ImxsWkjQMOAJ4FLgSmCzperILt+NSscHAvILN5qdYsfj8RuJFOSmaWS71V4plWhoRRxcrIKkXcC9wZUSslvRV4NMRca+k9wO3AKfsSp3z8O2zmeUSiFo6lbWUIqkLWUK8MyImpvBYoP7zPWTPCQEWAEMLNh+SYsXiQxqJF+WkaGa51YXKWopJLcG3ALMj4rsFqxYCb0+fTwbmpM+TgPNSK/QoYFVELAImA6dK6p8aWE4FJqd1qyWNSsc6D7iv1Ln59tnMcgnE5qhqjl0dD3wEeEbSjBS7mqy1+PuSOgMbyVqaAe4HzgDmAuuB8wEiYrmkrwDTUrlrI2J5+nwpcCvQHXggLUU5KZpZLtnL27t+kxkRj0CTDyePaqR8AJc1sa8JwIRG4tOBQ/PUy0nRzHLL0dDS7jgpmlkuEaI2Krc5wknRzHKr85WimVkma2ip3NRRuWdmZi2iuRpa2ionRTPLrbb8bn7tjpOimeVS36OlUjkpmlludW59NjPLZANCOCmamQHZ7fOW5unm1yY5KZpZLhH45W0zs23kl7fNzOoFvlI0M9uOG1rMzJKg9ACy7ZmTopnlkk1xWrmpo3LPzMxaSOnpS9uzyn0wYGYtIsh6tJSzFCNpqKSHJM2SNFPSpwrWXSHpuRT/VkH8i5LmSnpe0mkF8dEpNlfSFwriwyU9muK/lFRd6vx8pWhmuTXTlWINcFVEPCGpN/C4pCnAILKJ7w+LiE2S9gCQNBI4B3gDsDfwJ0kHpn39EHgn2dzO0yRNiohZwDeBGyLiLkk/Ai4AxherlK8UzSyXCDXLlWJELIqIJ9LnNcBsssnqLwG+ERGb0rrX0iZjgLsiYlNEvEQ2gdUxaZkbES9GxGbgLmBMmsHvZOBXafvbgDNLnZ+TopnlkjW0VJW1AAMlTS9YLmpsn5KGAUcAjwIHAiek296/SnpzKjYYmFew2fwUayq+G7AyImoaxIvy7bOZ5ZRrjpalEXF00b1JvYB7gSsjYnWa2nQAMAp4M3C3pP12pcZ5OCmaWS5ZQ0vztD5L6kKWEO+MiIkpPB+YmKY0fUxSHTAQWAAMLdh8SIrRRHwZ0E9S53S1WFi+Sb59NrPcaulU1lJMeuZ3CzA7Ir5bsOo3wEmpzIFANbAUmAScI6mrpOHACOAxYBowIrU0V5M1xkxKSfUh4Oy037HAfaXOzVeKZpZLM/ZoOR74CPCMpBkpdjXZpPYTJD0LbAbGpgQ3U9LdwCyyluvLIqIWQNLlwGSgCpgQETPT/v4LuEvSV4EnyZJwUU6KZpZbc0xcFRGPQJPv9ny4iW2uA65rJH4/cH8j8RfJWqfL5qRoZrlEwJa6yn3y5qRoZrlkt89OimZmW1Vy32cnxQa0pIbu17+KVtSCYMvpfdh8Zr9Gy3Z6fiM9P7OADV8YRM0JvXbtwGtq6fH1V9GrNcSgzqz/4iDovW0ejGY9VoW5ZsxDvPXAl1mxrjsfuOkDO6zv2XUTX3nvn9mz71qqOtXx878fxm9nHLxLx+zTfSNff98U9uq3hkUre/OFu09lzcauW9eP3Ps1Jnz81/z3r07hwVn779Kx2prmfCWnLWrRa+CmOmm3aVWw8cKBrLt5H9bdMIQuv1tNp5c371iuNuj2s2XUHNkj3+6f3kC377y2Q7zr3SupObw7627Zh5rDu9P17pW7fKyO4rczDuKKn7+ryfXvP2YmLy3pzwfHv49P/Ow/ufK0f9K5qrasfR81bAHjzvzzDvGPvvVJHntxCGfd+EEee3EIHz3hya3rOqmOK945lUf/NST/ybQLzdPNr61qsVpLqiLrpH06MBI4N3XobtNiQGfqDki/8Xt0om5oF7SsZody1ZNWseX4XkS/7Wc1q/7VCnp+cj49L5lH1zuWl33czv9cx5ZTegOw5ZTedP7nupLHssyTL+/N6g1dm1wfQI/qzUDQo3oLqzd0pTY1FHzk+BncdtG9/OKSu7nopGllH/PtB/+b383IxiL43YwDOfHgl7au+8Cxz/Ln2fuxfF33nTqf9qAuzdNSammPWjKVN9pJuwWP1+z06haq/rWZ2oO6bR9fWkPnf6xjy7v6bBevenw9nRZsYd33B7Puh0PoNHcTVc9sKOtYnVbWEgOypxnRv4pOK2uLHsvKd/ejhzJ895X84bN3cNeld3P9A8cTIY7dfx5DB6xi7M1n8cEfvY9D9lrCEfsuLGufA3puYNnangAsW9uDAT2zv+fde6/lxENe4lfT3tBi59PastbnqrKW9qglnyk21kn72IaFUgfxiwCq92hD//E31NHjq4vZ+IndoOf2vzu6/Xgpmz62G3Ta/jdh5yfW0/mJDfS8fD4A2lBHp4VbqH1jd3peOR+2BNpQh9bUUXVZ9qPZ+LHdqD2qwW2xtPXtraaOZeV7ywHzeGHxblx8638wZMBqfnje7/jg+L0Ytf98Ru0/jzsvzgZR6VG9hX12W8WTL+/NrRdOpEtVLT2qt9Cn+ybuvPgeAH4wZRRT/zW0wRFEpE9Xnf4PfjBlFFHBz9w8HUELi4ibgZsBeh64V5Qo/vqoCXp8dTFbTupNzfE7NmpUzdlE92+8CoBW19J52jo2VmX/SDZ9oB9bzui7wzbrvpc9X6p6egNdpqxh41V7bLe+rl8VWl5DDOiMltdQ17eq6LFqjuvZfOdb4f7jiOe59eEjADF/eV8WrujNsIErkIJbHzmSidN3fKrz0Z+cBWTPFN99+PN8+Tcnb7d++bru7NZrHcvW9mS3XutYkW6VD9l7CV87ewoA/Xps5PgRr1BT14m/Pje8ZU/yddZeb43L0ZJJsVjn7bYrgm7fe43aodVsPqvxVue1t+679XO377xGzTE9qDmuJ9FVdLtjOVtO6g3dO6GlNdAZol/pH3PNqB50+dMaNr+/P13+tIaat/Qseiwr3+JVvThmv/nMeGUvBvRcz74DVzJ/RR/+OXcol5w8jQeeHsGGzV3Yvfdaauqqtia4Yv76/DDeffgL3PbIEbz78Bf463PDABjzvQ9tLTPuzD/zyAv7VlxCrPTW55ZMils7aZMlw3OAD7bg8ZpF1cyNVD+4ltph1XROt7ibxg5AS7LGli3v2vEqsF7tUT3YMm8zPT+T5f7oJjZ8bhA0nlu3s/n9/en+tVfpMnkNsUdn1l89aNdPpoO47uw/cdSwhfTrsZHff+YObv7L0XTuVAfAvdPfwE//ehRfOvMh7rr0bkTwgymjWLW+O4/+ayjDd1/Bzz7+awDWb+7C/7v35LKS4m0PH8HX3z+FMUfOZtHK3nzxnne26Dm2Ne21ZbkcyvpZt9DOpTOA77Gtk/YOfRYL9Txwrzj0B2NbrD7W/LZM3KN0IWsznpt4A+uXzNuly7z+B+8RJ084u3RBYOLx4x8vNZ5iW9OizxSb6qRtZu2bb5/NzBI/UzQza8BJ0cws8XuKZmYNVPJ7ipXbrm5mLSICauo6lbUUI2mopIckzZI0U9KnGqy/SlJIGpi+S9KNaYCZpyUdWVB2rKQ5aRlbED9K0jNpmxvTvDBFOSmaWW51obKWEmqAqyJiJNl0ppfVDxojaShwKvBKQfnTySarGkHWNXh8KjsAGEfWjfgYYJyk/mmb8cCFBduNLlUpJ0Uzy6X+meKuJsWIWBQRT6TPa4DZbJus/gbg80Dhi9RjgNsjM5Vs+tK9gNOAKRGxPCJWAFOA0Wldn4iYmia+uh04s9T5+ZmimeWWY8CLgZKmF3y/OY13sB1Jw4AjgEcljQEWRMRTDe52GxtkZnCJ+PxG4kU5KZpZbjkaWpaW6tEiqRdwL3Al2S311WS3zq3Ct89mlktEsz1TRFIXsoR4Z0RMBPYHhgNPSfo32UAyT0jak6YHmSkWH9JIvCgnRTPLSdTWdSprKbqX7N74FmB2RHwXICKeiYg9ImJYRAwju+U9MiIWA5OA81Ir9ChgVUQsAiYDp0rqnxpYTgUmp3WrJY1KxzoPuK/U2fn22cxya6ZBdI8HPgI8I2lGil2dxkxozP3AGcBcYD1wflaXWC7pK2QjcwFcGxH1c4FcCtwKdAceSEtRTopmlktz9X2OiEeg+MPJdLVY/zmAy5ooNwGY0Eh8OnBonno5KZpZPpE9V6xUTopmllsld/NzUjSzXCI1tFQqJ0Uzy823z2ZmBSp5ClcnRTPLJcJJ0cxsOx5k1sysgJ8pmpklgahz67OZ2TYVfKHopGhmObmhxcysgQq+VHRSNLPcOuSVoqQfUOT3QUR8skVqZGZtWgB1dR0wKQLTi6wzs44qgI54pRgRtxV+l9QjIta3fJXMrK2r5PcUS75sJOktkmYBz6Xvh0m6qcVrZmZtV5S5tEPlvIH5PbJ5VZcBRMRTwNtaslJm1paJiPKWonuRhkp6SNIsSTMlfSrFvy3pOUlPS/q1pH4F23xR0lxJz0s6rSA+OsXmSvpCQXy4pEdT/JeSqkudXVmvpUfEvAah2nK2M7MK1TxXijXAVRExEhgFXCZpJNlk9odGxJuAF4AvAqR15wBvAEYDN0mqklQF/BA4HRgJnJvKAnwTuCEiDgBWABeUqlQ5SXGepOOAkNRF0meB2WVsZ2aVKCDqVNZSdDcRiyLiifR5DVleGRwRf4yImlRsKtumKR0D3BURmyLiJbIJrI5Jy9yIeDEiNgN3AWPSDH4nA79K298GnFnq9MpJiheTTRYzGFgIHE4Tk8eYWUehMhcGSppesFzU6N6kYcARwKMNVn2MbTPwDQYK71rnp1hT8d2AlQUJtj5eVMmXtyNiKfChUuXMrAMpvxFlaUQcXayApF7AvcCVEbG6IP7fZLfYd+5kLXdKOa3P+0n6raQlkl6TdJ+k/V6PyplZG9VMrc+SupAlxDsjYmJB/KPAu4EPpalNARYAQws2H5JiTcWXAf0kdW4QL6qc2+f/A+4G9gL2Bu4BflHGdmZWiepf3i5nKSI987sFmB0R3y2IjwY+D/xng3ejJwHnSOoqaTgwAngMmAaMSC3N1WSNMZNSMn0IODttPxa4r9TplZMUe0TEHRFRk5afA93K2M7MKlREeUsJxwMfAU6WNCMtZwD/C/QGpqTYj7JjxkyyC7RZwB+AyyKiNj0zvByYTNZYc3cqC/BfwGckzSV7xnhLqUoV6/s8IH18IL33cxfZ74gPAPeXPF0zq1zN0Pc5Ih6BRieQbjK/RMR1wHWNxO9vbLuIeJGsdbpsxRpaHidLgvWV/kThsUjvDplZx6N22lulHMX6Pg9/PStiZu1EO+7CV46yxlOUdCjZm+JbnyVGxO0tVSkza8tKN6K0ZyWToqRxwIlkSfF+sq40jwBOimYdVQVfKZbT+nw28A5gcUScDxwG9G3RWplZ21ZX5tIOlXP7vCEi6iTVSOoDvMb2L0qaWUfSUQeZLTA9Dd3zE7IW6bXAP1u0VmbWpnXI1ud6EXFp+vgjSX8A+kTE0y1bLTNr0zpiUpR0ZLF19UP+mJlVkmJXit8psi7IxilrVlVzNtHn9H81926tBU1eeG9rV8FyOGbqkmbZT4e8fY6Ik17PiphZOxE0Sze/tqqsl7fNzLbTEa8Uzcya0iFvn83MmlTBSbGckbcl6cOSrknf95GUaygeM6swHXze55uAtwDnpu9ryKYTNLMOSFH+0h6Vc/t8bEQcKelJgIhYUc6E0mZWwSq49bmcK8UtabLpAJC0O+22q7eZNYfmuFKUNFTSQ5JmSZop6VMpPkDSFElz0p/9U1ySbpQ0V9LThR1MJI1N5edIGlsQP0rSM2mbG9O8MEWVkxRvBH4N7CHpOrJhw75WxnZmVqma55liDXBVRIwERgGXSRoJfAF4MCJGAA+m75ANWzgiLRcB42Hr1CnjgGPJph4YV59IU5kLC7YbXapS5fR9vlPS42TDhwk4MyJmlzxdM6tMzfS8MCIWAYvS5zWSZpNNVj+GbAxXgNuAv5BNQDUGuD3N0jdVUj9Je6WyUyJiOYCkKcBoSX8hG6thaorfDpwJPFCsXuUMMrsPsB74bWEsIl4p58TNrAKVnxQHSppe8P3miLi5YSFJw4AjgEeBQSlhAiwGBqXPg4F5BZvNT7Fi8fmNxIsqp6Hl92ybwKobMBx4HnhDGduaWQVS+a0KSyPi6KL7knoB9wJXRsTqwsd+ERHS69uOXfKZYkS8MSLelP4cQXbP7vEUzWyXSepClhDvjIiJKfxqui0m/flaii9g+wGuh6RYsfiQRuJFldPQsp00ZNixebczswrSDA0tqSX4FmB2RHy3YNUkoL4FeSxwX0H8vNQKPQpYlW6zJwOnSuqfGlhOBSandasljUrHOq9gX00q55niZwq+dgKOBBaW2s7MKlTzvZh9PPAR4BlJM1LsauAbwN2SLgBeBt6f1t0PnAHMJWvnOB8gIpZL+gowLZW7tr7RBbgUuBXoTtbAUrSRBcp7pti74HMN2TNGD6Jn1pE1T+vzI2RtFY15RyPlA7isiX1NACY0Ep8OHJqnXkWTYnppu3dEfDbPTs2swrXTLnzlKDYdQeeIqJF0/OtZITNr20Su1ud2p9iV4mNkzw9nSJoE3AOsq19Z0FJkZh1JOx7soRzlPFPsBiwjm5Ol/n3FAJwUzTqqDpoU90gtz8+yLRnWq+AfiZmVVMEZoFhSrAJ60XjrUAX/SMyslI56+7woIq593WpiZu1HB02KlTuKpJntvOi4rc87vDxpZgZ0zCvFgm4yZmbb6ajPFM3MGuekaGaWtOPpS8vhpGhmuQjfPpuZbcdJ0cyskJOimVkBJ0Uzs6TCR8nJPUeLmVlzzNECIGmCpNckPVsQO1zSVEkzJE2XdEyKS9KNkuZKelrSkQXbjJU0Jy1jC+JHSXombXOjCqcKbIKTopnlprryljLcCoxuEPsW8OWIOBy4Jn0HOB0YkZaLgPEAkgYA48gm1DsGGJcmsCKVubBgu4bH2oGTopnlpihvKSUi/gY07D0XQJ/0uS/bJsobA9wemalAvzQF6mnAlIhYHhErgCnA6LSuT0RMTfO73A6cWapOfqZoZvnke3l7oKTpBd9vjoibS2xzJTBZ0vVkF27HpfhgYF5BufkpViw+v5F4UU6KZpZf+UlxaUQcnXPvlwCfjoh7Jb2fbG7oU3LuY6f59tnMcqnv0dIct89NGMu26U7uIXtOCLAAGFpQbkiKFYsPaSRelJOimeWmuihr2UkLgbenzycDc9LnScB5qRV6FLAqIhYBk4FTJfVPDSynApPTutWSRqVW5/OA+0od3LfPZpZPMw4IIekXwIlkzx7nk7UiXwh8X1JnYCNZSzPA/cAZwFxgPXA+ZMMcSvoKMC2Vu7Zg6MNLyVq4uwMPpKUoJ0Uzy625Xt6OiHObWHVUI2UDuKyJ/UwAJjQSnw4cmqdOTopmll8F92hxUjSz3Cq5m5+Topnl56RoZpZ04Nn8zMx24JG3zcwaisrNik6KZpZbJV8pukdLIz7z3Vf45dMz+fGfn290/UnvWcH4Pz3Pjx58nhsmzWG/kRt2+Zhdquu4+kf/5md/n833fzeHQUM2b7d+98Gb+c2cZzj74td2+ViV5rUFXfjc2ftz4dsP5sITD+LXPx24Q5k1K6v48seGcfE7DuKKM0bw7+e67fJxN28S131iXz563CF88l0jWDyvevt6ze/CmAPeyD3jd9/lY7Up5Y6l2E4TZ4slxcYGj2wv/vjLAfz3h4Y3uf7VedV87r37c/E7DuLOGwbxqW/Nb7JsQ4OGbOZbv5q7Q/y0c5ezdmVnzj/+ECb+ZCAX/M/C7dZ/YtxCpv25d/kn0YFUdQ4uumYhP/nrc3z/d3P47a0DefmFrtuVuevGQez/hg386MHn+dz3X2H8NSUHS9lq8bxqPvfeA3aIT/7FAHr1q+XWf8zmrAuXcMtX99pu/Y+/PJg3n7xm506qjWvG8RTbnJa8UryVMgZ0bIuefbQXa1Y0/WRh1vSerF2VrX/uiR4M3GvbVd3JZ63gxt+/wE1TnueT35xHp07l/bp8y2mrmHJPNi7mw7/rx+FvXUv9r9q3jF7F4nnVvPzCrl/dVKLdBtUw4k3Z1XqPXnUMPWATSxd12a7MK3O6cthb1wKwz4hNvDqvmhVLsr/DB+/tzxVnjOCSUw7i+58fQm1tecf95+S+vPN9WW+yE969khmP9N76qO0fD/Rlz6Gb2ffAjc1whm2Pk+JOaGLwyIoz+tzlTHsoGw9z6AEbefuYlXx6zAgufedB1NWKk89aUdZ+Bu5Zw5KF2X/kulqxbnUVfQbU0q1HLe+/9DV+/p1BLXYOlWTxvGr+9Wx3Dj5y/Xbx4SM38vf7+wLw3JM9eHV+NUsXdeGVOV356339uOG+OYz/0/N0qoI/T+zf2K53sHRxF3bfewsAVZ2hZ59aVi+vYsO6Ttx90x58+KrFzXtybUWQNbSUs7RDrd7QIukiUofvbvRo5drkc9hxaznt3OV85szs1uqIE9Yy4o3r+cEDLwBQ3S1YuSz7EV9zy0vsuc9mOncJ9hi8hZumZM8rf/PT3fnjLwc0eYyPfPZVfv2T3dm4vqqFz6b927CuE1/5+DAuvnYBPXtvf5nygctfZfz/G8wlpxzE8EM2cMChG+jUCZ58uDdznunBFacfBMDmjaLfbjUAfPljw1j8SldqtojXFnThklOyMmd+fAmnndP07/s7rt+T91y4hO492+mlUhkquaGl1ZNiGoX3ZoA+GtBuftTDD9nAldfP438+vN+2W20FU+4ZwM++vtcO5a+9IHtGOWjIZq763it8/uztn1EtXdyZ3ffewtJF1XSqiq1XHQcfsZ63vmslF/zPQnr1qSXqxOZNnZj0sx0bEzqymi3wlY8P4+SzVvDWM1btsL5n7zo++71scOYIGHvsSPbcdxPPPtqTd75vOR+7etEO24yb8G8gu/r8zpX78O17t38WPHDPLSxZmF0t1taw9er+uSd78Mjv+3HLV/dm7eoq1Cmo7hqM+djS5j/x1tJu/qfm1+pJsT3affBmrvnpv/n2J/dhwYvbHujPeLg3X/rZS0y8eSCrlnWhd78auves47UF1UX2lpn6x768830rmP14T05490qeeqQXIK56z7bk+eGrFrNxnRNiQxHw3av2YeiITbz3E0saLbN2VRVdu9fRpTp44P8GcOiotfTsXcfhJ6zhS+fvx1kXLaHfwBpWr8hufwcN2VLyuKNOXc2UewYw8uj1PPy7fhz21jVI8N3fbEued1y/J9161lZUQvTL2x3QF256mTe9ZS19B9Tw8+mzuOM7g+jcOftX8Ps7BvKhT79K7/61XP71rNW5tkZccfqBvDKnG7d9a0++fteLSFn8f68eXFZS/MMvBvD5G1/hZ3+fzZqVVXztkn1b9BwryczHevLgrwYw/JANW29xz//iwq0/93eft4xX5nTl+iv3QcC+B23k09/Jrhr3PXATYz+/iC+esz8RWUv25V+bX1ZSHH3uMr71yeyVnN79arh6/Mstdo5tSuzSALJtnqKFHoYWDh4JvAqMi4hbim3TRwPiWL2jRepjLWPywhmtXQXL4ZjT5jH9qY0l5z4upne/IXHE2z5VVtmHf/v5x3dijpZW1WJXikUGjzSzdq6Sb5/do8XM8gmgLspbSmiqk4ekKyQ9J2mmpG8VxL8oaa6k5yWdVhAfnWJzJX2hID5c0qMp/ktJJZ9lOSmaWX7N183vVhp08pB0EtnE94dFxBuA61N8JHAO8Ia0zU2SqiRVAT8ETgdGAuemsgDfBG6IiAOAFcAFpSrkpGhmuTXXFKdNdPK4BPhGRGxKZeo7/I8B7oqITRHxEtkEVsekZW5EvBgRm4G7gDFpBr+TgV+l7W8DzixVJydFM8stxxSnAyVNL1guKrVv4EDghHTb+1dJb07xwcC8gnLzU6yp+G7AyoioaRAvyq/kmFk++UbAWboTrc+dgQHAKODNwN2S9su5j53mpGhmuWQvb7do8/N8YGKa0vQxSXVkr/YtAIYWlBuSYjQRXwb0k9Q5XS0Wlm+Sb5/NLL+6Mped8xvgJABJBwLVwFJgEnCOpK6ShgMjgMeAacCI1NJcTdYYMykl1YeAs9N+xwL3lTq4rxTNLLfmulIs7OQhaT4wjmxS+wnpNZ3NwNiU4GZKuhuYBdQAl0VEbdrP5cBkoAqYEBEz0yH+C7hL0leBJ4GiHUjASdHM8mrGUbWLdPL4cBPlrwOuayR+P3B/I/EXyVqny+akaGY5VXbfZydFM6EVrgsAAAblSURBVMuvnQ4gWw4nRTPLJ9rvVAPlcFI0s/x8pWhmVqByc6KTopnlp7rKvX92UjSzfIJdeTG7zXNSNLNcRLR0N79W5aRoZvk5KZqZFXBSNDNL/EzRzGx7bn02M9sqfPtsZrZV4KRoZradyr17dlI0s/z8nqKZWaEKToqeo8XM8omA2rrylhIkTZD0Wpp6oOG6qySFpIHpuyTdKGmupKclHVlQdqykOWkZWxA/StIzaZsb01zQRTkpmll+EeUtpd0KjG4YlDQUOBV4pSB8OtlkVSOAi4DxqewAsrldjiWbemCcpP5pm/HAhQXb7XCshpwUzSy/ZkqKEfE3YHkjq24APs/2g5SNAW6PzFSy6Uv3Ak4DpkTE8ohYAUwBRqd1fSJiapr46nbgzFJ18jNFM8sngPLnaBkoaXrB95sj4uZiG0gaAyyIiKca3O0OBuYVfJ+fYsXi8xuJF+WkaGY5BUTZ7+QsjYijyy0sqQdwNdmtc6vw7bOZ5RM0W0NLI/YHhgNPSfo3MAR4QtKewAJgaEHZISlWLD6kkXhRTopmll/zNbQ02G08ExF7RMSwiBhGdst7ZEQsBiYB56VW6FHAqohYBEwGTpXUPzWwnApMTutWSxqVWp3PA+4rVQffPptZfs30nqKkXwAnkj17nA+Mi4hbmih+P3AGMBdYD5yfVSWWS/oKMC2VuzYi6htvLiVr4e4OPJCWopwUzSyn5hsQIiLOLbF+WMHnAC5rotwEYEIj8enAoXnq5KRoZvkE4KHDzMwKVHA3PydFM8spdrZluV1wUjSzfAKi/PcU2x0nRTPLr/weLe2Ok6KZ5ednimZmSYRbn83MtuMrRTOzekHU1rZ2JVqMk6KZ5ZNv6LB2x0nRzPLzKzlmZpkAwleKZmZJ5Bpktt1xUjSz3Cq5oUXRhprWJS0BXm7terSAgcDS1q6E5VKpf2f7RsTuu7IDSX8g+/mUY2lElJxBry1pU0mxUkmanmeeCmt9/jvruDwdgZlZASdFM7MCToqvj6Lz3Fqb5L+zDsrPFM3MCvhK0cysgJOimVkBJ8UWJGm0pOclzZX0hdauj5UmaYKk1yQ929p1sdbhpNhCJFUBPwROB0YC50oa2bq1sjLcCrSrl42teTkptpxjgLkR8WJEbAbuAsa0cp2shIj4G7C8tethrcdJseUMBuYVfJ+fYmbWhjkpmpkVcFJsOQuAoQXfh6SYmbVhTootZxowQtJwSdXAOcCkVq6TmZXgpNhCIqIGuByYDMwG7o6Ima1bKytF0i+AfwIHSZov6YLWrpO9vtzNz8ysgK8UzcwKOCmamRVwUjQzK+CkaGZWwEnRzKyAk2I7IqlW0gxJz0q6R1KPXdjXrZLOTp9/WmywCkknSjpuJ47xb0k7zPrWVLxBmbU5j/UlSZ/NW0ezhpwU25cNEXF4RBwKbAYuLlwpaafm8Y6Ij0fErCJFTgRyJ0Wz9shJsf16GDggXcU9LGkSMEtSlaRvS5om6WlJnwBQ5n/T+I5/Avao35Gkv0g6On0eLekJSU9JelDSMLLk++l0lXqCpN0l3ZuOMU3S8Wnb3ST9UdJMST8FVOokJP1G0uNpm4sarLshxR+UtHuK7S/pD2mbhyUd3Bw/TLN6O3VlYa0rXRGeDvwhhY4EDo2Il1JiWRURb5bUFfi7pD8CRwAHkY3tOAiYBUxosN/dgZ8Ab0v7GhARyyX9CFgbEdencv8H3BARj0jah6zXziHAOOCRiLhW0ruAcnqDfCwdozswTdK9EbEM6AlMj4hPS7om7ftysgmlLo6IOZKOBW4CTt6JH6NZo5wU25fukmakzw8Dt5Dd1j4WES+l+KnAm+qfFwJ9gRHA24BfREQtsFDSnxvZ/yjgb/X7ioimxhU8BRgpbb0Q7COpVzrGWWnb30taUcY5fVLSe9Lnoamuy4A64Jcp/nNgYjrGccA9BcfuWsYxzMrmpNi+bIiIwwsDKTmsKwwBV0TE5AblzmjGenQCRkXExkbqUjZJJ5Il2LdExHpJfwG6NVE80nFXNvwZmDUnP1OsPJOBSyR1AZB0oKSewN+AD6RnjnsBJzWy7VTgbZKGp20HpPgaoHdBuT8CV9R/kVSfpP4GfDDFTgf6l6hrX2BFSogHk12p1usE1F/tfpDstnw18JKk96VjSNJhJY5hlouTYuX5KdnzwifS5Es/Jrsj+DUwJ627nWwkmO1ExBLgIrJb1afYdvv6W+A99Q0twCeBo1NDziy2tYJ/mSypziS7jX6lRF3/AHSWNBv4BllSrrcOOCadw8nAtSn+IeCCVL+ZeIoHa2YeJcfMrICvFM3MCjgpmpkVcFI0MyvgpGhmVsBJ0cysgJOimVkBJ0UzswL/H+QD36Y9FKBWAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mCfNzkZArUwX"
      },
      "source": [
        "##### Multinomial Naive Bayes w/ Word2Vec Precision-Recall Curve"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qxIW6zlurUwX",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 409
        },
        "outputId": "39f51001-1e08-4073-a570-a49e7e7c242e"
      },
      "source": [
        "create_pr_curve('Multinomial Naive Bayes (Word2Vec) Precision-Recall Curve', nb_w2vec_preds, y_test)"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAGICAYAAABWXQdhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXgV5dn48e+dnewkJJCFJSQQZA2KCAKyqQURgktVqFCUior4tn1rqbV1KS79Vap9tYAVFXDDDRRQcWcVcIGyiEEk7GENS4AEQrbn98echJOQfTmTk3N/ritX5szMmbln5szc88wzM48YY1BKKaWqy8vuAJRSSrkXTRxKKaVqRBOHUkqpGtHEoZRSqkY0cSillKoRTRxKKaVqpMklDhExIpJUyfAfRWSQC0Oq8XyrWoY6xvErEfm8IabtDkRkjYj0tGG+7Rzb1cfV83aK4TsR6WLX/J3iqHJfEJE2IpItIt4uCqtBicgEEfna6XOD7eOu0GgSh4jsEZE8EWlRpv9Gx0puV4tpzhORJ5z7GWO6GGNW1CnYWqiv+YrIChHJFZHWTv2uFpE91YzjTWPMtXWNo5y45jm2X7aInBGRDSIysL7nUxciMhI4Y4zZKCIxjt9VS6fhf6mg36cNEMs/RWSHY139JCLjHf0DRCRLRIaU851/iciCOsz2n8C0SmKaICKFjm14WkQ2icj1dZhfuaqzLxhj9hljgo0xhfU9fxF5TETyHcuZJSJrRaRvfc+nLkTkFyKyyvH7yBSRlSIyyu64ijWaxOGwGxhT/EFEugGB9oXTaOUAD9sdRDmeNsYEA6HAC8D7jeyM8R7gdQBjzCEgHbjKafhVwE/l9FtVk5lUs1SRA4wEwoBfA8+JyJXGmFzgHWB8mWl6Y+0br9YkljKWAINFpFUl46xzbMNw4BXgXRFpXnYkO0tO9eQdx3K2AJYD79kcTwkRuRkrnteAeKAl8AjW76Wm0xIRqf/jvDGmUfwBe4C/At879fsn8BfAAO0c/VYAv3EaZwLwtdNnAyQBk4B8IA/IBj50ms/Vju7HgHexNtAZ4Eegl9O0LnHML8sxbJTTsHnALOATx/TXAK2A/wNOYh2AepZZvuL59gbWOaZ7CJgB+JVdhgrW0wrgUUe8iY5+VwN7nMZ5ENjpGCcNuKG89YV1cP9nmekvBv7X0R0LLAQysZL6/1Sy/eYBTzh9DnQsR6zjcyKwDDgOHAPeBMIdw/4ILCwzveeB5xzdYVgHsUPAAeAJwNsxLAlYCZxyTPedCuLzA84B8U79XgH+7ej2diznPWX6nQb6Y51k/RXYCxx1/GbCHOO1cyzrRGAfVqLxxvr9HgN2Afc5xvGpIL4lwB8c3Vc6tl2g0/DrHPP1qWx9OMa9C9jmtP0vdRr2BfDrCmIo+W04Pgc5Yu6Fta8sAN5wrJPf1DYOLt4X1jumeQR4tsw69XH6LS4BTmAl/Luc5vMYlezH5SznY8AbTp87O+YVVdXvrYrlqtZ+V9k+DojjN/THGsRfdl2tAJ7EOiadA/4ErC8zjd8DSxzd/li/1X2ObfAfoFllx+vGVuL4BggVkUscZ1i3Yf1Qa8wYMxvr4PS0sYq8FWXrUcDbWGdYS7AO4oiIL/Ah8DkQDdwPvCkiyU7fvQXrYNICOI+VDP7r+LwAeLaCeRZibbgWQF9gKDC5Bot3AHgJ+FsFw3cCA7B2gL8Bb4hITDnjvQXcKiIC4DizvBZ423GW8iGwGYhzxPg7EflFVcE5tt14rGRzpLg38HesA8AlQGusHQCsbTxMRMId3/fB2vavOYbPAwqwkkRPR4y/cQx7HGsbNcc6O/t3BWF1AIqMMRlO/VZxoXTRE+tg8FWZfr7Ad1g7/gRgMNAeCMbxW3Ey0LFsv8A6uFzvmEYv4OYK4kJEmgGXYx3wMMasxTpo3eg02jhgvjGmgErWh4j8Emu9jscq+Y3CStbFtgE9KorFKSYfxzSzgR2O3qlYv+twrH2rLnEUew7rBCEU6+Ti3QpCehvIwPr93Aw8VeZyXrn7cTWW088R43GsEz7qsFzV3e8qk4y1b9TlkiRYv5dJQAhWIkgWkQ5Ow8cC8x3d/w/oCKRgLXMcVgmnYpVlFVf+4TgLwToQ/x0YhnV25EMtShymnLPgcs52HgO+LHPmcc7RPQA4DHg5DX8LeMxp2i85Dbsf2Ob0uRuQVd58y1n23wEfVHU24rz8QBTWWXYXypQ4yvnOJiC17PriwtnNVU5nUssc3VcA+8pM58/A3ArmMQ/IxSpFnXN0/6qSmEYDG50+f4LjLBLrgJvm6G6JlZSbOY07Blju6H4NmI1TSaKC+fUDDpfp1w4riYdjJfInHf0POvUrns9XwGSn7yZjlWh9uHDG195p+DLgHqfP11JBiQPr8tOngDj1+yvwuaM7FDiLdRCran18Bvy2kvXwJDCngmETsA6YWVglpW8ova+schq31nFQeh9chXWQbVHOtjGO9dvasZ1CnIb/HZhX1X5cwfwfw7oSkeWY7nFgUF2Xq7r7XWX7ONbv1AABVcRfVYljWpnvvAE84ujugKNEi3UMyMFx9cIxvC+wu7Jla2wlDrCuQY/FWtGvVT5qvTjs1H0WCHCcbcUC+40xRU7D92Jl42JHnLrPlfM5uLwZikhHEflIRA6LyGngKazSR7UZYzKxzqouquwUkfGOis0sEckCupY3fWP9St7mQr3SWKwzSYC2QGzxNBzTeQhrx6rIP40x4Vg/yF7AdBEZ7oippYi8LSIHHMv8RpmYXgVud3TfjqMuwhGHL3DIKY4XsUqBAFOxfvzfOe7WubOC2E5inX05L/8erNLbAKxSxmrHoLVO/YrrN2Kxtn+xvVgHNef1sd+pO7bMZ+fvlhCR6Vjb5xbH9ij2OlZ9RPEZ9k5jzEaqXh+tsc58KxKCdcCsyDfGmHBjTAtjTB9jzJcVLF9d4yg2Eets9ycR+b6CyvhY4IQx5oxTv7L7Yrn7sVh3EWY7/j5xGuddx2+1JbAVuKyuy1Xd/a4KxaWXmpZUytpf5vN8Su/ni4wxZ7FOQAOBDU5xf+roX6FGlziMMXuxLnFcB7xfzig5lK4wr6yiz1QyrCoHgdZlKpbaYB1o6uoFrDqQDsYqoj+EdfCrqelYl06Kf/SISFusy1hTgEjHzrG1kum/Bdzs+N4VWHUaYP3wdjsOIsV/IcaY66oKyli2Yl1jHeHo/RTW9ujmWObby8S0COguIl2xShzFCWw/1hlgC6c4Qo0xXRzzOmyMucsYEwvcDcyS8m9zTLdWj8SV6V98uaovVsIAK4FchVW3UZw4DmIdVIq1wTo7dz5ZcP69HcI6yDiPX4qI/A0YDlxrjDntPMyxH6zGWk/juFApXun6cAxPvGjpL7gE6/JjbTgvX13jsCZozA5jzBisA/M/gAUiElRmtINAhIg4J/5q7YvGuosw2PE3vJzhx7Au6TzmuKxUq+WqxX5Xke2OedxUyTjVOQaWPfZ9AUSJSApWAim+THUM6yS3i9PyhhnrxoEKNbrE4TARGGKMySln2CbgRhEJdBwgJlYynSNY16Nr41usM5epIuIr1n3nI7HO0OsqBKsyMFtEOgH31mYixpgs4Bmss+5ixRWamQAicgfWmU9F09iI9eN5GfjMMU2wruufEZE/iUgzEfEWka4icnl1YnMsV38c1+2xljkbOOU4eP+xTBy5WNd15wPfGWP2OfofwqrDeEZEQkXES0QSxXGrr4j8UkTiHZM56Vh251Ji8fTzgC+x6iGcrcK6Xn3Q6eD9taNfGFa9FVgJ9vcikiAiwViJ8B1j1TmU513gf0Qk3lF39GCZ9fNnrDO/q40x5V37BytZTMG6fPFmddYH1nZ8QEQuc9xRk+Q4qCEiAVgnGV9UML9qq0scZdbD7SIS5SjZF//2Sm0/Y8x+rKT+d7FuV+6Otd/Xqv6znGXZjnUJamodlqtG+10lsRjgf4GHReQOpxj6i8hsx2ibgKvEetYlDOsSclXTzce6U2s6EIHjN+BY7y8B/xKRaEfscVJFXWajTBzGmJ3GmPUVDP4X1vXJI1g71psVjAfWnRGdHUWwRTWMIQ8rUQzHOrDOAsYbY36qyXQq8ADWQeMM1kZ7pw7Teg7rOi0Axpg0rGSyDmsddcM686/MfKx6kuKzEIx1//z1WBVmu7mQXMIqmc5UxyWBHKydby5WMR+s69iXYtXLfEz5pclXHfG+Xqb/eKy7otKwksMCLhTlLwe+FZFsrErR3xpjdlUQ34tYZ+/OVmKd7X7t1G8T0AzY4CjOA8xxxLUKa33kYtVrVeQlrIPRZqwbJsou71NYZ83pTpdSHiozzkKsnfwrxwGtWIXrwxjzHlY9xnys39cixzTA+j2vMMYcrCTumqhtHM6GAT86tt9zwG3GmHPljDcG61r+QeAD4NEyl9HqajowyXHwrPFy1XK/K5cxZgFwK3An1vIewbqza7Fj+BdYx4wtwAbgo2pOung/f6/MCc+fsErk3zguI3+JVYdXISl9WVUp+4hIG6xLeK3KXrqpx3msAaY4SloeRUS+BSY6LiMqVWuaOFSj4KhLehYINcZUVMGtlGoE3P3pT9UEOCpDj2DdKTPM5nCUUlXQEodSSqkaaZSV40oppRovlyYOEZkjIkdFpNzKOcftbc+LSLqIbBGRS10Zn1JKqaq5uo5jHtbTzhU9ET4c63H4DlgPo73g+F+pFi1amHbt2tVPhEq52qlTkJ5udYtAcjIElX0GDsjJge3bofjyclIShFV2d7TyZBs2bDhmjKn0CfDacmniMMasksrb1UgFXnM8BPONiISLSEyZe9gv0q5dO9avr+ixD6UauREjLiSOP/wBpk8vGVRQWISPt9OFgQcegGeesbqTk+Gj6t7CrzyNiJT7mpv60NjqOOIo/Y6VDEq/j0appmeN03NiE60XIeTmFzLi+dXMWlHmtUgTJ5b/PaVcqLEljmoTkUkisl5E1mdmZtodjlK1d8bp3X2J1muQAny9Cfb3YdGmA5S68zHR6TVJpxvkGUmlqtTYEscBSr8YLp4KXmRmjJltjOlljOkVFdUgl/GUco0Qp3f37bxQwkhNiWNXZg4/Hjxd7nBCQ10QnFIXa2yJYwkw3nF3VR/gVFX1G0q5vX79LnS/8kpJ53XdWuHrLSzaeKDc4aW+p5QLufp23LewXgKWLCIZIjJRRO4RkXscoyzFamYzHeslcTVpFU8p93Sv08uRZ8yAddYLecMD/RjYMZoPtxyksMhY/WfMKP97SrlQk3hyvFevXkbvqlJuq7AQuneHtDTrs78/TJkCEyfy8dlmPPvCp7zGD8S98QqcP2+N07kzbNkC3t72xa0aNRHZYIzp1SDT1sShVCOwfbt16el4RU1zOImMhLVroWPHho9Lua2GTByNrY5DKc+UnGzdXtu5c+Xjde6sSUPZThOHUo1FcrJ1+emjj6yHAsPDwcuL/NAwvkq8nA0zX7eGa9JQNtPXqivVmHh7W0ljxIiSXlJYxJ/+/hWXh0ZwmdZpqEZASxxKNXI+3l5c3z2Wr346yuncfLvDUUoTh1LuIDUllryCIj7detjuUJTSxKGUO0hpHU7byEAWbyr3RQpKuZQmDqXcgIiQ2iOWtTuPc/R0rt3hKA+niUMpN5HaMw5jYMnmg3aHojycJg6l3ERiVDDd4sI0cSjbaeJQyo2kpsSyJeMUuzKz7Q5FeTBNHEq5kZE9YhGBRZu01KHso4lDKTfSMjSAvu0jWVK2gSelXEgTh1JuZnRKHHuOn2Vzxim7Q1EeShOHUm7mF11b4eftpc90KNto4lDKzYQ182VIp2g+3HyIgsIiu8NRHkgTh1JuaHTPWI5ln2fdrmq036FUPdPEoZQbGpQcTUiAD4s26t1VyvU0cSjlhgJ8vRnetRWf/XiY3PxCu8NRHkYTh1JuKjUljuzzBXy17ajdoSgPo4lDKTfVp30k0SH+LNK7q5SLaeJQyk15ewkje8SyYvtRTp3VBp6U62jiUMqNjU6JI7/QsHTrIbtDUR5EE4dSbqxrXCjtWwTpw4DKpTRxKOXGRITUlDi+3X2CQ6fO2R2O8hCaOJRyc6kpsVYDT/rGXOUimjiUcnPtWgTRo3U4izVxKBfRxKFUEzA6JZa0Q6fZceSM3aEoD6CJQ6kmYET3GLwELXUol9DEoVQTEB0SQL+kFizerA08qYaniUOpJiI1JY79J87x331ZdoeimjhNHEo1Eb/o0hJ/H23gSTU8TRxKNREhAb5c3bklH205RL428KQakCYOpZqQ1B6xnMjJ4+v0Y3aHopowTRxKNSGDkqMJa+bL4o16uUo1HE0cSjUhfj5eXNetFZ+nHeFsXoHd4agmShOHUk1MakocZ/MK+SLtiN2hqCZKE4dSTUzvdhHEhAXou6tUg9HEoVQT4+UljOoRy8qfMzmRk2d3OKoJ0sShVBOUmhJHQZHh4x+0gSdV/zRxKNUEXRITQofoYJbow4CqAWjiUKoJEhFG94zj+z0nyTh51u5wVBOjiUOpJmpUj1gAlmzWSnJVvzRxKNVEtY4I5LK2zVm8UROHql+aOJRqwlJTYtl+5Aw/HT5tdyiqCdHEoVQTNqJbDN5ewiItdah6pIlDqSYsMtifqzq0YMmmAxQVaQNPqn5o4lCqiUtNiePgqVzW7z1pdyiqidDEoVQTd03nljTz9WaRPtOh6okmDqWauCB/H67p3JKlPxwir0AbeFJ1p4lDKQ8wumcsWWfzWfVzpt2hqCZAE4dSHmBAhyiaB/qyWB8GVPVAE4dSHsDX24sR3WP4Iu0w2ee1gSdVN5o4lPIQo1PiyM0v4vMfD9sdinJzmjiU8hCXtmlOXHgzFmsDT6qONHEo5SG8vITUlFi+Tj/Gsezzdoej3JjLE4eIDBOR7SKSLiIPljO8jYgsF5GNIrJFRK5zdYxKNVWpKXEUFhk+3qINPKnac2niEBFvYCYwHOgMjBGRzmVG+yvwrjGmJ3AbMMuVMSrVlCW3CqFTqxB9GFDViatLHL2BdGPMLmNMHvA2kFpmHAOEOrrDAL0gq1Q9Sk2JY+O+LPYez7E7FOWmXJ044oD9Tp8zHP2cPQbcLiIZwFLg/vImJCKTRGS9iKzPzNSHmpSqrlEpjgaetJJc1VJjrBwfA8wzxsQD1wGvi8hFcRpjZhtjehljekVFRbk8SKXcVVx4M3onRLBo0wGM0TfmqppzdeI4ALR2+hzv6OdsIvAugDFmHRAAtHBJdEp5iNSUWHZm5vDjQW3gSdWcqxPH90AHEUkQET+syu8lZcbZBwwFEJFLsBKHXotSqh5d1zUGX29hsVaSq1pwaeIwxhQAU4DPgG1Yd0/9KCLTRGSUY7Q/AHeJyGbgLWCC0fK0UvWqeZAfAztGsWTzQQq1gSdVQz6unqExZilWpbdzv0ecutOAfq6OSylPk5oSx5fbjvLt7uNcmahXg1X1NcbKcaWUC1x9SUuC/LxZrO2RqxrSxKGUh2rm580vurRi6dZDnC8otDsc5UY0cSjlwUalxHImt4DlP+n9J6r6NHEo5cH6J7WgRbAfSzbr3VWq+jRxKOXBfLy9uL57LF9uO8rp3Hy7w1FuQhOHUh5uVEoseQVFfLZVG3hS1aOJQykP17N1OG0iArWBJ1VtmjiU8nAiVgNPa3ce4+jpXLvDUW5AE4dSitSUWIoMfKgNPKlq0MShlCIpOoQusaH67ipVLZo4lFIAjE6JY0vGKXZlZtsdimrkNHEopQAY2SMWEbSSXFVJE4dSCoBWYQH0SYhkyeaD2sCTqpQmDqVUidE9Y9l9LIctGafsDkU1Ypo4lFIlhnWNwc/bi0VaSa4qoYlDKVUirJkvgztF8eHmQ9rAk6qQJg6lVCmpKXEcyz7P2p3H7A5FNVKaOJRSpQzpFE2Iv4/eXaUqpIlDKVVKgK83w7q24tOth8nN1wae1MU0cSilLpKaEkf2+QKW/XTU7lBUI6SJQyl1kb6JkUSF+LNoo95dpS6miUMpdRFvL2Fk91hWbM/k1Flt4EmVpolDKVWu0T1jySss4pOt+sZcVZomDqVUubrFhZHQIkgfBlQX0cShlCpXcQNP3+4+waFT5+wORzUimjiUUhVKTYnDGPhwsz7ToS7QxKGUqlBCiyB6xIfpw4CqFE0cSqlKpabE8ePB06QfPWN3KKqR0MShlKrU9T1i8BJYtFFLHcqiiUMpVanokAD6JbVg8eYD2sCTAjRxKKWqYVSPWPafOMd/92XZHYpqBDRxKKWqNKxrK/x9vFiiz3QoNHEopaohJMCXqy9pyUdbDpFfWGR3OMpmmjiUUtUyKiWW4zl5rEnXBp48nSYOpVS1DEqOIjRAG3hSmjiUUtXk7+PNdd1i+OzHw5zNK7A7HGUjTRxKqWpLTYnjbF4hX27TBp48mSYOpVS1XZEQQavQABZrA08eTROHUqravLyEUSmxrPw5k5M5eXaHo2yiiUMpVSOpKbEUFBk+/kEbePJUmjiUUjXSOSaUpOhglujdVR5LE4dSqkZEhNEpsXy35wQZJ8/aHY6ygSYOpVSNjeoRB8ASbeDJI2niUErVWJvIQC5tE66XqzyUJg6lVK2kpsTx0+Ez/HT4tN2hKBfTxKGUqpUR3WPw9hJ9BYkH0sShlKqVFsH+9E9qwZJNBykq0gaePIkmDqVUrY3uGcuBrHNs2HfS7lCUC2niUErV2rWdWxHg68UifQWJR9HEoZSqtSB/H67p3IqPfzhEXoE28OQpNHEopepkdEosWWfzWb0j0+5QlIto4lBK1cmADlGEB/qySO+u8hiaOJRSdeLn48WIbjF8kXaY07n5doejXEATh1Kqzsb0bkNufhGvr9trdyjKBTRxKKXqrGtcGIOSo3h59S5tVtYDaOJQStWL+4ckcfJsPvO/3Wd3KKqBuTxxiMgwEdkuIuki8mAF49wiImki8qOIzHd1jEqpmrusbQR920cye9UucvML7Q5HNSCXJg4R8QZmAsOBzsAYEelcZpwOwJ+BfsaYLsDvXBmjUqr2pgxJ4uiZ87y3IcPuUFQDcnWJozeQbozZZYzJA94GUsuMcxcw0xhzEsAYc9TFMSqlaunKxEh6tgnnPyt2kl+oDwQ2Va5OHHHAfqfPGY5+zjoCHUVkjYh8IyLDypuQiEwSkfUisj4zUx88UqoxEBHuH5LEgaxz+hqSJqwxVo77AB2AQcAY4CURCS87kjFmtjGmlzGmV1RUlItDVEpVZHByNJ1jQpm1YieF+tbcJsnVieMA0Nrpc7yjn7MMYIkxJt8Ysxv4GSuRKKXcgIgwZUgSu4/l8PEPh+wORzUAVyeO74EOIpIgIn7AbcCSMuMswiptICItsC5d7XJlkEqpuhnWpRVJ0cHMXJaubXU0QS5NHMaYAmAK8BmwDXjXGPOjiEwTkVGO0T4DjotIGrAc+KMx5rgr41RK1Y2Xl3Df4ES2HznDF9uO2B2OqmdijPufDfTq1cusX7/e7jCUUk4KCosY8sxKwgN9WXxfP0TE7pA8iohsMMb0aohpN8bKcaVUE+Dj7cXkQYlsyTjFqh3H7A5H1SNNHEqpBnPjpfHEhAXw76920BSubiiLJg6lVIPx8/Hi7qvas37vSb7dfcLucFQ90cShlGpQt/VuQ4tgP2YuT7c7FFVPNHEopRpUgK83dw1oz+odx9i476Td4ah64FPVCCIyviYTNMa8VvtwlFJN0a/6tGXWip3MXJ7Oy7++3O5wVB1VmTiAeTWYngE0cSilSgn29+HOfgn868ufSTt4ms6xoXaHpOqgOpeqEmrw175hwlRKubsJV7Yj2N+HmSu0rsPdVVniMMZoI8JKqToLC/RlfN+2vLByJ+lHs0mKDrY7JFVLWjmulHKZif0T8PfxYpaWOtxadSrHd2PVXVSHMcYk1i0kpVRTFRnsz9jebXl13R5+N7QjbSID7Q5J1UJ1KsdXUv3EoZRSlZp0VXve+GYvL6zcyd9v7GZ3OKoWqlPHMcEFcSilPESrsAB+2Sue99Zn8D9Dk4gJa2Z3SKqGtI5DKeVy9wxMpNAYZq/SpnbcUXUuVV1ERHoAyUBA2WH6AKBSqiqtIwIZnRLHW9/tY/KgJKJC/O0OSdVAjRKHo+3vj4E+xb0c/53rQDRxKKWqNHlwIu9vzOCVr3fz4PBOdoejaqCml6qeAiKBq7CSxg3AEOBNrOZde9drdEqpJisxKpgR3WJ4fd0ess7m2R2OqoGaJo5fYCWPbxyfM4wxK4wx44Evgd/WZ3BKqabtvsFJ5OQVMm/tHrtDUTVQ08QRA+wyxhQCuUCI07D3gRH1FZhSqum7JCaUazq3ZO6aPZzJzbc7HFVNNU0ch4FwR/deoK/TsKR6iUgp5VGmDE7i1Ll83vhmn92hqGqqaeL4mgsV468Dj4rIiyIyE5gOfFafwSmlmr4ercMZ0KEFL6/exbm8QrvDUdVQ08TxNy4kh+nATKzLU2OAJcD99ReaUspT3D+kA8dz8nj7ey11uIMaJQ5jzE5jzGpHd74x5g/GmHhjTIQxZqwx5njDhKmUasp6J0TQOyGCF1fu4nyBljoauxolDhHxFZGgCoYFiYhv/YSllPI0UwYncfh0Lgs3HLA7FFWFml6qehl4qYJhLzr+lFKqxgZ0aEGP+DBeWJlOQWGR3eGoStQ0cQwGFlcwbAkwtG7hKKU8lYgwZUgH9p84x5LNB+0OR1WipokjGjhawbBMoGXdwlFKebKhnaLp1CqEmcvTKSzS1hwaq5omjqNARS/Q7wZo5bhSqta8vIQpQ5LYmZnDp1sP2x2OqkBNE8dHwMMi0t25p4h0A/4CfFhfgSmlPNPwrjG0jwri38t2YIyWOhqjmiaOR4AsYIOIrBWRd0VkDfBf4BTw1/oOUCnlWby9hMmDkvjp8Bm+2lbRlXFlp5o+x3EMuBz4O9bbcVMc/58ELncMV0qpOklNiSW+eTNmLE/XUkcjVOMWAI0xWcaYR4wxfY0xHY0xVxpjHjPGnGqIAJVSnsfX24t7ByWyaX8Wa9K16rSxqVXTsSLSQkSuF5Ffi0iEo1+AiGhTtEqpenHzZfG0DPXn38t22B2KKqOmT46LiEwHMrCe25gDtHMMXoxVQa6UUnXm7+PNpKsS+Xb3Cb7fc8LucJSTmpYQ/qQPXNUAACAASURBVAxMAaYBV3Ch6Viw7qi6vp7iUkopxvRuTWSQHzOWpdsdinJS08TxG2CaMeYprDupnKUDifUSlVJKAYF+PkwckMDKnzPZkpFldzjKoaaJI44LzcaWlQeU+wJEpZSqrXF92hIa4KOljkakponjANC1gmE9gN11C0cppUoLCfBlQr8EPk87wk+HT9sdjqLmieM94BER6efUz4hIR+APwNv1FplSSjnccWU7gvy8mbV8p92hKGqeOB4DfgJWAcX3yL0H/IBVx/H/6i0ypZRyaB7kx+192/LRloPsysy2OxyPV9Mnx88Bg4AJwFrgS+B7YBLWHVX31m94Sill+U3/9vh6e/HCCi112K2mz3G0AIqMMa8bY243xlwLTARaAD8DzzZAjEopRVSIP2N6t+GDjQfYf+Ks3eF4tCoTh4j4i8hzInIGOAIcF5F7HcNuB3YC04H9wLCGDFYp5dnuHtgeEXhxlZY67FSdEscjwP3AOqwE8QXwnIj8G3gN6624qcaYK4wxXzRYpEopjxcT1oybL4vn3fUZHDmda3c4Hqs6ieNWYJYx5lpjzIPGmFuBe4D7sJJId2OMtsOhlHKJewcmUVhkeGnVLrtD8VjVSRytgQ/K9Hvf8f9ZY0xe/YaklFIVaxMZSGqPWN78dh/Hs8/bHY5Hqk7i8AXOlOlX/DmzfsNRSqmqTR6cSG5BIXPW6DPHdqjuXVVxItK++A9oX15/xzCllGpQSdEhDO/aitfW7uXUuXy7w/E41U0cC7Ae+Cv++8nRf1GZ/vrifKWUS9w3OIkz5wt4be0eu0PxOD7VGOeOBo9CKaVqqEtsGEM7RfPKmt3c2T+BIP/qHM5UfahyTRtjXnVFIEopVVP3DUnixllrefPbvUy6Slt1cBVt6lUp5bYubdOcfkmRzF61m9z8QrvD8RiaOJRSbm3K4A4cyz7Pu+v32x2Kx9DEoZRya33aR9CrbXP+s2IneQVFdofjETRxKKXcmogwZUgSB0/l8sHGDLvD8QguTxwiMkxEtotIuog8WMl4N4mIEZFeroxPKeV+BnaMoltcGLNW7KSgUEsdDc2liUNEvIGZwHCgMzBGRDqXM14I8FvgW1fGp5RyTyLCfYOT2Hv8LB//cMjucJo8V5c4egPpxphdjndcvQ2kljPe48A/AH39pVKqWq7t3JKOLYOZsSydoiJjdzhNmqsTRxxWux3FMhz9SojIpUBrY8zHrgxMKeXevLysUseOo9l8nnbY7nCatEZVOS4iXlitCP6hGuNOEpH1IrI+M1PftaiUguu7x5LQIoh/L0vHGC11NBRXJ44DWK9pLxbv6FcsBOgKrBCRPUAfYEl5FeTGmNnGmF7GmF5RUVENGLJSyl14ewn3Dkzkx4OnWfGznlA2FFcnju+BDiKSICJ+wG3AkuKBxphTxpgWxph2xph2wDfAKGPMehfHqZRyU6N7xhEX3ox/f7VDSx0NxKWJwxhTAEwBPgO2Ae8aY34UkWkiMsqVsSilmiY/Hy/uGdie/+7LYt2u43aH0yRJU8jIvXr1MuvXa6FEKWXJzS9kwNPL6RAdzPy7+tgdji1EZIMxpkGeg2tUleNKKVUfAny9mTSgPWt3HmfD3pN2h9PkaOJQSjVJY69oQ/NAX2YuT7c7lCZHE4dSqkkK8vdhYv8Elv10lK0HTtkdTpOiiUMp1WSNv7IdIQE+WuqoZ5o4lFJNVmiALxOubMcnWw+z48gZu8NpMjRxKKWatDv6JdDM15tZK3baHUqToYlDKdWkRQT5cXufNizedIC9x3PsDqdJ0MShlGry7hrQHh9vL17QUke90MShlGryokMDuO3y1iz8bwYHss7ZHY7b08ShlPIIdw9MxBiYvVJLHXWliUMp5RHiwptx46VxvP39fo6e0Tbi6kITh1LKY9w7KIn8wiJeWb3b7lDcmiYOpZTHSGgRxMgesbz+zV5O5uTZHY7b0sShlPIo9w1O4mxeIXPXaKmjtjRxKKU8SseWIfyiS0vmrd3D6dx8u8NxS5o4lFIeZ8rgDpzOLeD1dXvtDsUtaeJQSnmcbvFhDEqO4pWvd3M2r8DucNyOJg6llEe6f0gSJ3LymP/tPrtDcTuaOJRSHumythH0bR/J7FW7yM0vtDsct6KJQynlsaYMSeLomfMs2JBhdyhuRROHUspjXZkYSc824bywYif5hUV2h+M2NHEopTyWiHD/kCQOZJ1j0cYDdofjNjRxKKU82uDkaDrHhDJrxU4Ki4zd4bgFTRxKKY8mIkwZksTuYzks/eGQ3eG4BU0cSimPN6xLK5Kig5mxLJ0iLXVUSROHUsrjeXkJ9w1OZPuRM3y57Yjd4TR6mjiUUgoY2T2WNhGBzFiejjFa6qiMJg6llAJ8vL2YPCiRLRmnWL3jmN3hNGqaOJRSyuHGS+OJCQtgxrJ0u0Np1DRxKKWUg5+PF3df1Z7v9pzg213H7Q6n0dLEoZRSTm7r3YYWwX7MWK6ljopo4lBKKScBvt7cNaA9q3ccY9P+LLvDaZQ0cSilVBm/6tOWsGa+WtdRAU0cSilVRrC/D3f2S+DLbUdIO3ja7nAaHU0cSilVjglXtiPY34eZK7TUUZYmDqWUKkdYoC/j+7Zl6Q+HSD+abXc4jYomDqWUqsDE/gn4+3jxwoqddofSqGjiUEqpCkQG+zO2d1sWbTrA/hNn7Q6n0dDEoZRSlZh0VXu8RXhhpZY6imniUEqpSrQKC+CXveJZsD6DQ6fO2R1Oo6CJQymlqnDPwEQKjWH2ql12h9IoaOJQSqkqtI4IZHRKHG99t49j2eftDsd2mjiUUqoaJg9O5HxBEa98vdvuUGyniUMppaohMSqYEd1ieG3tHrLO5tkdjq00cSilVDXdNziJnLxC5q3dY3cottLEoZRS1XRJTCjXdG7J3DV7yD5fYHc4ttHEoZRSNTBlcBKnzuXzxjd77Q7FNpo4lFKqBnq0DmdAhxa8vHoX5/IK7Q7HFpo4lFKqhu4f0oFj2Xm8/f0+u0OxhSYOpZSqod4JEfROiGD2ql2cL/C8UocmDqWUqoX7hyRx6FQu7//3gN2huJwmDqWUqoX+SS3oER/GrBXpFBQW2R2OS2niUEqpWhARpgzpwP4T51iy+aDd4biUJg6llKqloZ2i6dQqhJnL0ykqMnaH4zKaOJRSqpa8vIQpQ5LYmZnDpz8etjscl9HEoZRSdTC8awzto4L497J0jPGMUofLE4eIDBOR7SKSLiIPljP8f0UkTUS2iMhXItLW1TEqpVR1eXsJkwclse3QaZb9dNTucFzCpYlDRLyBmcBwoDMwRkQ6lxltI9DLGNMdWAA87coYlVKqplJTYolv3sxjSh2uLnH0BtKNMbuMMXnA20Cq8wjGmOXGmOJW4b8B4l0co1JK1Yivtxf3Dkpk0/4s1u48bnc4Dc7ViSMO2O/0OcPRryITgU/KGyAik0RkvYisz8zMrMcQlVKq5m6+LJ6Wof78e9kOu0NpcI22clxEbgd6AdPLG26MmW2M6WWM6RUVFeXa4JRSqgx/H28mXZXIN7tO8P2eE3aH06BcnTgOAK2dPsc7+pUiIlcDfwFGGWO0gV+llFsY07s1kUF+zFiWbncoDcrVieN7oIOIJIiIH3AbsMR5BBHpCbyIlTQ84xYFpVSTEOjnw8QBCaz8OZMtGVl2h9NgXJo4jDEFwBTgM2Ab8K4x5kcRmSYioxyjTQeCgfdEZJOILKlgckop1eiM69OW0AAfZi5vuqUOH1fP0BizFFhapt8jTt1XuzompZSqLyEBvkzol8DzX+1g++EzJLcKsTuketdoK8eVUspd3XFlO4L8vJtsqUMTh1JK1bPmQX7c3rctH205yO5jOXaHU+80cSilVAP4Tf/2+Hp78cKKplfq0MShlFINICrEnzG92/D+fw+QcfJs1V9wI5o4lFKqgdw9sD0i8OLKXXaHUq80cSilVAOJCWvGzZfF8876/Rw5nWt3OPVGE4dSSjWgewcmUVhkeGlV0yl1aOJQSqkG1CYykNQesbz57T5O5OTZHU690MShlFINbPLgRHILCpnz9W67Q6kXLn9y3NWKiorIyMggJ6fp3UutVG0EBQURHx+Pl5eeN7pKUnQIw7u24tW1e7jrqvaENfO1O6Q6afKJ49ixY4gIycnJuqMoj1dUVMSBAwc4duwY0dHRdofjUe4bnMTSHw7z2to93D+0g93h1EmTP5JmZWXRsmVLTRpKAV5eXrRs2ZJTp07ZHYrH6RIbxtBO0cxZs5uc8wV2h1MnTf5oWlhYiK+vexcLlapPvr6+FBS494HLXd03JImTZ/OZ/+0+u0OpkyafOABExO4QlGo0dH+wz6VtmtMvKZLZq3eRm19odzi15hGJQymlGospgzuQeeY8767fb3cotaaJw0MNHz6cp59+usrxVq9eTXh4uAsisseePXsQETIyMgCYN28eSUlJNkelmrI+7SPo1bY5L67cRV5Bkd3h1IomjkZi0KBB+Pv7ExwcTFhYGD179mThwoUNNr9PPvmEqVOnVjnegAEDyMpq2CYwJ0yYgK+vL8HBwYSGhnLJJZcwa9asBp1nXRw6dIh7772Xtm3bEhQURJs2bbjlllvYsGGD3aEpNyAiTBmSxIGscyzaeMDucGpFE0cj8vDDD5Odnc3x48cZM2YMt956Kz///PNF4+Xn59sQXcP69a9/TXZ2NllZWTzxxBNMmTKFFStW2B3WRQ4ePMjll1/O/v37Wbp0KadPnyYtLY2RI0fy/vvv12qaTXF7qsoN7BhFt7gwZq1Ip6DQ/UodmjgaIR8fHyZPnkxhYSE//PADK1aswMfHh9dff5327dsTEREBwL59+7j55ptp1aoVMTExTJo0iTNnzpRMJzMzk4kTJ9KmTRtCQ0O59NJL2b59O2CVcJ544gkAzp8/z6RJk4iOjiY0NJQOHTrw3nvvAZTMu1hBQQHTpk2jffv2NG/enKFDh7J169aS4RMmTGDcuHHcddddhIeHExcXx4svvljtZffy8uKmm24iMjKS9evXl/RfvXo1/fv3JyIigsTERJ555hmMMSXDt2zZwrBhw4iKiiIiIoKrr77QAvEdd9xB69atCQkJoXPnzsyfP7/a8ZT1yCOPEBQUxAcffECXLl3w9vYmODiYcePG8eSTTwLw2GOPlZo/lF7f5W3PmTNnkpKSUuo7u3fvxtvbmz179gBVb2/lPkSE+wYnsef4WT7+4ZDd4dRYk38AsKy/ffgjaQdPu2RenWNDeXRklxp/Ly8vj5kzZ+Lr60uPHj3IyMigsLCQpUuXsnHjRnx9fcnNzWXIkCGMHTuW119/ndzcXH71q1/x29/+ljlz5lBUVMSoUaOIjY3l+++/Jyoqiq1btxIScnH7x6+++irff/8927ZtIzIykv3791d4QJo+fTqvvfYaS5cuJSEhgb///e9cc801bN++ndDQUAAWLFjAO++8w4svvsiiRYu49dZbGTZsGG3btq1y2QsLC1mwYAHHjh0jOTkZgLS0NK677jreeOMNrr/+enbs2MHw4cOJiopi/PjxHDp0iIEDBzJ16lQWLlyIr68vq1atKplm//79+ec//0l4eDjvvfce48ePJyUlhc6dO9d42yxdupQ777yzzrd4l92e58+f5w9/+AObNm0qSSDz5s1j0KBBtGvXrsrtrdzPtZ1b0rFlMDOWpTOyeyxeXu5zt5uWOBqRJ598kvDwcOLj41m8eDELFy4sVVH7j3/8g7CwMAIDA/noo48wxjBt2jSaNWtG8+bNefzxx3nzzTcpLCxk/fr1rF+/njlz5pQ8ANm9e3diY2Mvmq+fnx/Z2dmkpaVRUFBA69atKzyozp07lz/96U906tQJf39/HnnkEby9vfn4449LxhkyZAijRo3Cy8uLG2+8kfDwcDZt2lTpsr/++uuEh4cTEBDAmDFjmDZtGiNHjgRg1qxZ/PKXvyQ1NRVvb286derElClTeO2110q+m5SUxJ///GeCgoLw8/MrdcY/ceJEIiMj8fb25rbbbqN79+61vgyWmZlJXFxcrb5blvP2bN68OampqcydOxcAYwyvvvoqd955J0CV21u5Hy8vq9Sx42g2n6cdsTucGvG4EkdtSgCu8pe//IW//vWv5Q7z8vKidevWJZ93797Nvn37LrrjSUQ4fPgwe/bsITo6mrCwsCrne/vtt3PkyBF+//vfs2PHDoYOHcrTTz9d7t1F+/fvJyEhoVRc7dq1Y//+C7cWxsTElPpOUFBQSQmmS5cu7N27F4CHHnqIhx56CIBx48bx8ssvc/bsWaZOncpXX33Fn//8Z3x8fNi9ezfLli0rVYdQVFRUsj727NlDx44dy122oqIiHnvsMd555x0OHz6MiJCTk0NmZmaV66U8UVFRHDhQ9wrNstsTrEtq48aN45///CerVq0iKyuLG2+8Eah6e9dXMlOudX33WP7vyx3MWL6DX3Rp6TbP2GiJw02ISKkfVdu2benYsSNZWVml/nJzc4mLi6Ndu3YcPXqU06ervizn4+PDn/70J9avX8/evXsJDAwsOdMtq3Xr1iXX3ME6MO/Zs+eig2BFfvzxR7Kzs8nOzi5JGs4CAwN59tlnOXDgADNnzixZ1jvvvLPUcp4+fZoff/wRgHbt2rFjx45y5/fWW2/x8ssvs3DhQk6ePElWVhY9evQoVT9SE9dddx0LFiyotEI7JCTkopdqHjx4sNTnstsT4JprrsHf358PP/yQefPmcdttt9GsWTOg6u2t3JO3l3DvwES2HjjNip9rdzJjB00cbur6668nLy+Pp556ijNnzmCM4cCBA3zwwQcA9OrVi0svvZTf/OY3HD16lKKiIrZs2XLRAQxg2bJlbNiwgfz8fJo1a0ZQUBDe3t7lznfChAk8/fTT/Pzzz+Tl5fHkk09SUFDAiBEj6m3Z/Pz8eOSRR3jiiSc4c+YMkydP5u233+bDDz8kPz+fgoIC0tLSWLlyJWCVmLZv384//vEPzp49S15eHl9++SUAp0+fxsfHh6ioKIqKipgzZw6bN2+udWx/+9vfyM7O5uabb2bbtm0UFhaSk5PDW2+9VVJavOyyy/jvf//Lhg0bKCgoYMaMGezeXfXrtL29vRk/fjzPP/8877//fqnkXdX2Vu5rdM844sKbMWNZeq1PaFxNE4ebCgwMZNmyZaSlpdGpUyfCwsIYOnRoSV2Cl5cXH374Ic2aNSMlJYXw8HDuvPNOsrOzL5rWkSNHGDduHM2bNycmJoa9e/cye/bscuf7xz/+kTFjxnDttdfSsmVLli1bxueff15SMV5fxo4dS0REBM888wxdu3blo48+4v/+7/+IiYkhOjqaCRMmlFxuio2NZcWKFXzxxRfEx8fTqlUrpk+fDli3+V5xxRUkJSURFxdHWloaAwYMqHVccXFxfP/998TExHDttdeWPHeyaNEibrrpJsC6g+p///d/GTZsGDExMRw5coR+/fpVa/p33HEHK1euJCEhgd69e5f0r2p7K/fl5+PFPQPbs2HvSb7ZdcLucKpF3CXDVaZXr17G+dZNZ9u2beOSSy5xcURKNW66XzQuufmFDHh6OR1bBvPmb/rUyzRFZIMxple9TKwMLXEopZTNAny9ufuq9qxJP86GvSftDqdKmjiUUqoRGHtFG5oH+jJzebrdoVRJE4dSSjUCgX4+TOyfwLKfjrL1QONuaEsTh1JKNRLjr2xHSIAPs1Y07lKHJg6llGokQgN8mXBlOz7ZepgdRxrve8g0cSilVCNyR78EAny8mbVip92hVEgTh1JKNSIRQX7c3qcNizcdYO/xnKq/YANNHBUpLISPPoIRIyA8HLy9rf8jRlj99cVySqkGcteA9vh4e/GflY2z1KGJozzbt0P37jByJCxdCqdOQVGR9X/pUqt/9+7WeEopVc+iQwO47fLWLNiQwcGsc3aHcxFNHGVt3w79+kFaWuXjpaVZ47lR8njqqadKXlVeleDgYNatW9fAEVVMRPj6669dNj/nhpZqozptlfv4+DTKVg1V43T3wESMgdmrdtkdykU0cTgrLIQbb4Tjx63PAQHwwAOwbRvk5Vn/H3gA/P2t4cePW+PXw2Wr4jbHQ0JCCAsLo3379owbN65e27F+6KGH+PDDD6s1bnZ2Nn379q23eRfbt28fwcHBJX8+Pj74+fmVfO7SpfG+9r4xW79+Pb179yYwMJDExETeeOMNu0NSdRQX3owbL43jre/2kXnmvN3hlKKJw9knn1woaQQEwLJlMH06dOoEvr7W/+nTYfnyC8kjLQ0+/bReZv/www9z5swZTp06xfLly2nbti19+vRpUm9AbdOmTclr1bOzsxk0aBAPPfRQyefiV6XXlCe3233q1CmGDx/OTTfdxMmTJ/nPf/7DPffcY2uJUdWPewclkV9YxMtfN65ShyYOZy+8cKF7yhSo6Iy7b19reHnfqydt27bliSeeYPz48dx///0lr1s+e/YsDzzwAAkJCURERDBs2DDS0y88LJSfn89TTz1FcnIyISEhJCYmsmDBAuDitrCff/55EhISCAkJIS4urlT7GGUvFS1cuJAePXoQFhZGjx49SiWz4ss0zz//PPHx8TRv3py77767zi3Tbdmyhcsvv5yQkBD69OnDTz/9VDJs0KBB/O53v2P06NGEhobyzDPPAPDSSy/RtWtXwsLC6NmzJ59//nnJdzZu3Ej//v0JCwsjIiKCK6+8kpMnL7wX6OTJk9x0000l623x4sWl4nnhhRdITk4mLCyMPn36sHr16gpjP3PmDL/+9a+JiIigbdu2vPrqq3VaF5V5//33CQwMZOrUqfj7+3PNNddwww03VPiGY+U+EloEMbJHLG+s28vJnDy7w7nAGOP2f5dddpmpSFpaWoXDLhIWZgxYf9u2VT5uWtqFccPDqz+PCgwcONA8/vjjF/X//PPPDWC2OeIZO3asGTFihDl8+LA5f/68eeSRR0xycrLJy8szxhgzdepUc8kll5jNmzeboqIis3//frN582ZjjDGPPvqoGTp0qDHGmO3bt5tmzZqZrVu3GmOMOXnypFm3bl3JfAGzevVqY4wxa9asMf7+/mbp0qUmPz/ffPTRR8bf39988803xhhj5s6da3x8fMxDDz1kcnNzzY4dO0zz5s3NG2+8UeVyDx061Dz66KMX9QfM5Zdfbvbu3Wtyc3PNzTffbK6++upS6yskJMR89dVXpqioyOTk5JjZs2ebxMREs2nTJlNYWGg+/vhjExQUZHbs2GGMMaZv377mb3/7mykoKDB5eXlm3bp1Jjs7u2R6kZGRZs2aNaawsNA8++yzJiwszOTk5BhjjJk/f76JjIw033zzjcnPzzcvv/yyCQwMNHv27ClZB4mJiSXx3XnnnaZv377m0KFDJisry9xwww0GMMuXLy93Pezdu9eEhYVV+leR3/72tyY1NbVUv2effdb07Nmzwu/UaL9Qttp++LRp+6ePzDOfb6/R94D1poGOuVricHbG6UnNxMTKx3UeXo1W9morPj4egOPHj3Ps2DHmz5/PrFmzaNmyJX5+fjz66KMcOnSIb7/9FmMMM2fOZPr06XTv3h0RIT4+nu7du180XR8fH4wxJS3yhYeH06dP+a9znjdvHjfddBPDhw/Hx8eHESNGcMMNNzBnzpyScZo1a8a0adPw9/cnKSmJoUOHUtGr7qvrj3/8I23atMHf358JEyZcNL2bb76ZIUOGICIEBgby3HPP8cgjj9CjRw+8vLy47rrrGDx4MG+//TZgNRC1b98+9u/fj6+vL3369CEoKKhkerfeeitXXnklXl5eTJo0iVOnTpW0LDh37lzuvvturrjiCnx8fJg4cSLdu3dn/vz5F8VdVFTEm2++yeOPP06rVq0ICwvjH//4R6XL2qZNm4ta9yv7V5EzZ85c1ERweHh4tVp/VI1fx5Yh/KJLS+at2c3p3MZxSVYTh7OQkAvdO6u4f9p5eD03YuQsIyMDgMjIyJJW5Lp37054eDjh4eFERESQn5/P/v37yczMJCcnp8L2t521b9+eN998k5deeonY2Fj69+9f6rKOs7LtjAMkJiaWamc8Ojq6VKuBzu2M15Zz2+XlTa9du3alPu/evZv77ruvZN2Eh4ezfPnykjbC586dS1FREf379ychIYGHH36YgoKCCucHlMyzOuugWGZmJufPny8VX9nv1qeQkBBOnSr9UrysrKx6b1xL2WfK4A6czi3g9XV77Q4F0MRRmnMrba+8Uvm4zsOr2bpbbbzzzjvExcWRnJxM27ZtAdixY0epM9GzZ88yZswYoqKiCAwMrLD97bJuvPFGvvjiC44dO8Ytt9xCamoqZ8+evWi8su2MA+zatava7Yw3FC+v0j/ftm3bMmfOnFLrJjs7mxccdVAJCQnMmTOHjIwMlixZwssvv8xrr71WrXnVZB20aNECPz+/UuOX/W5ZZe82K++vIj169LioJcCNGzfSo0ePKpdLuYdu8WEMSo7ila93czavoOovNDBNHM7uvfdC94wZUNFdKevWWcPL+1492b9/P48++ijz5s3jueeeQ0SIjo5m7NixTJ48ueQsOisriw8++IDs7GxEhMmTJzN16lS2bt2KMYaMjAy2bNly0fS3b9/Op59+ytmzZ/H19SUsLAwRuehgDFbzqwsXLuSzzz6jsLCQTz75hPfff5877rij3pe7Ln7/+9/z2GOPsWnTJowxnDt3jq+//rqkUv3VV18taXM9PDwcHx+fCttWL2vChAm8+OKLfPfddxQUFDB37lw2bdrE2LFjLxrX29ubsWPH8uijj3LkyBFOnz7Ngw8+WOn0y95tVt5fRW644QZycnKYPn06eXl5fPXVV7z//vtMmjSpWsum3MP9Q5I4kZPHW99dXMp1NU0czoYPh86dre7cXBg8uPznOAYPhvOO+6o7d4Zhw+pl9o8//jghISGEhoZy1VVXkZ6eztq1a0vavMeZxwAACtpJREFUsgbrrqHk5GQGDRpESEgI3bp147333kNEAHjyySe55ZZbGD16NCEhIQwaNKjUXVfF8vLymDZtGjExMYSHh/P888+zcOFCAgICLhq3X79+vPrqqzzwwAM0b96cqVOn8sYbb1RYJ2KXu+66i6lTp3LHHXfQvHlz2rRpw+OPP15yq+6yZcu47LLLCAoKom/fvowdO5Zx48ZVa9rFieD2228nMjKSF154gaVLl5aUAst67rnnSEhIoFOnTnTr1o2RI0dWO0nVVHh4OEuXLuW9994jLCyMu+66i//85z8N8hyOss9lbSPo2z6S2at2kptv7yuPtM3xsoqfHC9+CLAykZGwdi1Uo05BqcZE2xx3T2vSj/Grl7/lidFdub1P+SctxbTNcVdKToY1ay6UPCrSubMmDaWUS12ZGEnPNuH8Z+VO8guLbItDE0d5kpNhy5bSb8f18ir9dtwtWzRpKKVcSkS4f0gSGSfPsXjTQdvi8LFtzo2dt7eVJEaMsDsSpZQqMTg5ms4xocxans4NPePw9hKXx6AlDqWUciMiwpQhSew6lsPSHw7ZEoNHJI6mcAOAUvVF9wf3N6xLK5Kig5m5PJ2iItdvzyafOAICAjh+/LjuLEphJY3jx4+Xe9u1ch9eXsJ9gxP56fAZvvrpqMvn3+TrOOLj48nIyCAzM9PuUJRqFAICAkregabc18jusfzrix3MWLaDqy+JLnmWyxWafOLw9fVt0PcEKaWUHXy8vZg8KJEH3/+B1TuOcVXHKJfNu8lfqlJKqabqxkvjiQkLYMayi98O0ZBcnjhEZJiIbBeRdBG56AU+IuIvIu84hn8rIu1cHaNSSrkDPx8v7r6qPd/tOcG3u6rxtot64tLEISLewExgONAZGCMiZR/RngicNMYkAf8CKm/IQCmlPNhtvdvQItiPGctdV+pwdYmjN5BujNlljMkD3gZSy4yTChS3s7kAGCqurPVRSik3EuDrzV0D2rN6xzE27a+4wa/65OrEEQc4vxM4w9Gv3HGMMQXAKSDSJdEppZQb+lWftoQH+rqsrsNt76oSkUlAcYMD50Vkq53xNAItgGN2B2EjT19+0HXg6cvPZmjxyoSSdfD/27v7WDmqMo7j35+2hEhfwNQGRMitaVFI0dgUQv3Dl4hGG1KDFYLa4BWCSINvSKMGwVr5xzdMNBB6jRVQIEBMmmsoQdGWGqGmTYHSNlUKNLW0xjeoaCnQ8vjHORvW7X2Zobuzd2d/n2QyO3dnZp9z7u4+O3Nmzhm7+9yjUHXieAZoHjLtLflvI62zR9IkYDpwRKtPRAwBQwCSNnWq++Be0e910O/lB9dBv5cfqquDqk9VbQTmSJol6RjgImC4ZZ1h4NP58ceB34Vv+zYzmzAqPeKIiEOSrgTuB14PrIqIbZJWAJsiYhj4KfBzSTuBf5GSi5mZTRCVt3FExBpgTcvfrmt6fBC4oORuh9oQWq/r9zro9/KD66Dfyw8V1UEtho41M7PquMsRMzMrpacSR793V1Kg/FdJ2i5pi6TfSurY5XjdMl4dNK23WFJIqt1VNkXqQNKF+b2wTdIdVcfYSQU+B6dKWivpkfxZWNiNODtF0ipJfxvtFgQlP8r1s0XSvLYHERE9MZEa058E3gocAzwGnNGyzlLg5vz4IuCubsddcfnfD7whP76iTuUvWgd5vanAemADML/bcXfhfTAHeAQ4IS/P7HbcFZd/CLgiPz4D2NXtuNtcB+8B5gFbR3l+IXAfIOAc4I/tjqGXjjj6vbuSccsfEWsj4kBe3EC6T6ZOirwHAL5N6uPsYJXBVaRIHVwG3BgRzwJERPUj/XROkfIHMC0/ng7srTC+jouI9aQrTkfzUeC2SDYAx0s6qZ0x9FLi6PfuSoqUv9mlpF8ddTJuHeTD8lMi4t4qA6tQkffBacBpkv4gaYOkD1cWXecVKf9yYImkPaQrOD9fTWgTRtnvitJ6tssRG52kJcB84L3djqVKkl4H3AAMdjmUbptEOl31PtJR53pJZ0ZENT3gdd8ngFsi4geSFpDuC5sbEa90O7C66KUjjjLdlTBWdyU9qkj5kXQucA2wKCJerCi2qoxXB1OBucA6SbtI53eHa9ZAXuR9sAcYjoiXI+Jp4M+kRFIHRcp/KXA3QEQ8DBxL6seqXxT6rjgavZQ4+r27knHLL+ldwEpS0qjTee2GMesgIvZHxIyIGIiIAVI7z6KI2NSdcDuiyOdgNeloA0kzSKeunqoyyA4qUv7dwAcAJJ1OShx/rzTK7hoGLs5XV50D7I+Ife18gZ45VRV93l1JwfJ/D5gC3JOvCdgdEYu6FnSbFayDWitYB/cDH5K0HTgMLIuIWhx5Fyz/V4CfSPoyqaF8sEY/IJF0J+mHwYzcjvNNYDJARNxMatdZCOwEDgCfaXsMNapPMzOrQC+dqjIzswnAicPMzEpx4jAzs1KcOMzMrBQnDjMzK8WJw2pB0mDuDbcxPS/pMUlX5ptBq4hhIL/2YIltGnEPdCwwszbrmfs4zAq6gHTn9LT8+MfATOC6sTZqk33AAlLvrUXdm7dp6w1aZp3k+zisFvKv/J8BcyJiZ9Pf1wLzImL6CNtMBg7V6eYwsyr4VJXV3UZgmqSz8ymhpZK+K2kv8CJwPICkj+WeZA9Iek7SPZJObd2ZpMskbZb0gqRnJT0o6d35uSNOVUk6S9JvJP0zb/OUpJuanj/iVJWkyZKul7RL0kt5fn1OdLS81uWSVkjal+P+laS6dadvE4wTh9XdLFK3G//Jy9eQ+m76LHA+cFDS54BfAttJfZxdTuos8UFJUxs7kvR90iBBm4ELgSWkAaOOSDB5/SmkrjEOk3rs/QiwgvFPEd8KfA24DTgPuAX4Kq+ONdPs68Bs4BLgi6TTXr8YZ/9mR6fbo1l58tSOifTFHMDbSF/MJ5ASwGFSp38D+fnN5FO0ebsppHFbVrXsbxbwEvClvDw77+uGMWJovMZgXp6fl99RIO6BvDw3Ly9vWe8bzftqeq11Letdnf/+5m7/TzzVd/IRh9XNDuBlUieXNwG3k36NN6yOiOY2jQWkhvTbJU1qTKSBcHaQhukEOJd0hD5UIpYngOeAlZKWSDplvA2aXq/1qKGx3DrGypqW5cfzfMSjILN2cOKwujkfOAt4O3BcRFwcEc3DbLZevTQzzx8gJZzm6UxeHUGyMd9TNJCI2E8aB34vKYntlrRV0uIxNnvjKHH+teX5htYhRBtjsBxbNE6zsnw5rtXN1mi6qmoErVdQNbobHwS2jbD+83n+jzw/GfhT0WAi4lFgcT6KmU9qk7hb0jsjYusImzQSwYn8/2W9J7Y8b9Y1PuKwfvcQKTnMjohNI0yNJPEA8AqpUb20iDgUERuAa0mfu9NHWXV9nreOJfOpPF/3Wl7frJ18xGF9LSL+LWkZcKOkNwH3kRrLTya1J6yLiDsi4klJPwSuyldaDZMay88GdkTEXa37lnQeKdGsBp4GjgO+QEpUD48Sz9Y8UM/yfJTyEKkd5lrgzoh4fKTtzKrkxGF9LyJWSvoLsAz4JOlz8Qzwe+DRpvWuzqNLLiUNUfxfYAvw61F2/QTwAulL/yRSwtgIfDAixmorGSQN9XoJ6WqqvcB3gG+9thKatZfvHDczs1LcxmFmZqU4cZiZWSlOHGZmVooTh5mZleLEYWZmpThxmJlZKU4cZmZWihOHmZmV4sRhZmal/A+vL7zZOQGc8QAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GpjgY5-3rUwX"
      },
      "source": [
        "### Logistic Regression\n",
        "\n",
        "Logistic regression seeks to find the relationship between features and the probability of particular outcome, between 0 and 1, fitting an \"s\"shaped curve to the data. In the case of binary classification, such as this task, data with features that yield probability below a certain threshold fall into one category, while probabilities above a certain threshold fall into the other. Hence, because the output probabilities of logistic regression will either be closer to 1 of 2 outputs, it makes sense to try logistic regression for a binary classifcation task.\n",
        "\n",
        "Additionally, logistic regression models are relatively simple and quick to train, so they're worthy as a next step before creating more complicated models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZaKbrd76rUwX"
      },
      "source": [
        "#### Logistic Regression w/ Tf-idf Feature Representation\n",
        "\n",
        "Grid Search with cross-validation is used to find the best performing C parameter. The C parameter determines the level of regularization and aids in preventing overfitting. Many grid searches were conducted, with the best C parameter being 0.565"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8y112Q8qrUwY",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f1671324-cb4c-4e9e-9942-35f3eb254b9b"
      },
      "source": [
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.model_selection import cross_val_score\n",
        "\n",
        "#Create Model\n",
        "lr_clf = LogisticRegression(verbose=2, C=0.55, multi_class='ovr', n_jobs=-1, penalty='l2')\n",
        "\n",
        "grid_values = {'C':[.56, .565, .57]} #L2 found to be best\n",
        "# default metric to optimize over grid parameters: log loss\n",
        "grid_lr_clf = GridSearchCV(lr_clf, param_grid = grid_values, scoring='neg_log_loss', n_jobs=-1, verbose=1, cv=3)\n",
        "#Fit to Tf-idf training data\n",
        "grid_lr_clf.fit(full_X_train, y_train)\n",
        "grid_lr_clf.cv_results_"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 3 candidates, totalling 9 fits\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.\n",
            "[Parallel(n_jobs=-1)]: Done   9 out of   9 | elapsed:   16.6s finished\n",
            "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.\n",
            "[Parallel(n_jobs=-1)]: Done   1 out of   1 | elapsed:    6.0s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'mean_fit_time': array([5.96649551, 5.8342789 , 5.24631596]),\n",
              " 'mean_score_time': array([0.04923399, 0.04755688, 0.03942792]),\n",
              " 'mean_test_score': array([-0.59994671, -0.59999722, -0.5999692 ]),\n",
              " 'param_C': masked_array(data=[0.56, 0.565, 0.57],\n",
              "              mask=[False, False, False],\n",
              "        fill_value='?',\n",
              "             dtype=object),\n",
              " 'params': [{'C': 0.56}, {'C': 0.565}, {'C': 0.57}],\n",
              " 'rank_test_score': array([1, 3, 2], dtype=int32),\n",
              " 'split0_test_score': array([-0.59843486, -0.59846683, -0.59844184]),\n",
              " 'split1_test_score': array([-0.60006358, -0.60017081, -0.60012108]),\n",
              " 'split2_test_score': array([-0.6013417 , -0.60135401, -0.60134469]),\n",
              " 'std_fit_time': array([0.09526536, 0.10710439, 0.8822632 ]),\n",
              " 'std_score_time': array([0.00087542, 0.00250814, 0.00648781]),\n",
              " 'std_test_score': array([0.00118958, 0.00118506, 0.00118994])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-1K3-SELrUwY",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a79b19f5-298f-4bc1-90a4-f3705016881e"
      },
      "source": [
        "grid_lr_clf.best_params_"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'C': 0.56}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "waBqo3lFrUwY"
      },
      "source": [
        "#Create predictions on test set\n",
        "grid_lr_clf_preds = grid_lr_clf.predict(full_X_test)"
      ],
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WiWUSzC_rUwY"
      },
      "source": [
        "#### Evaluate Logistic Regression w/ Tf-idf Representation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TTY7f396rUwY",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5b5a1e25-14c5-4dad-dbff-d7fc60004361"
      },
      "source": [
        "print(\"Logistic Regression (Tf-idf) Accuracy: \" + str(accuracy_score(grid_lr_clf_preds, y_test)))\n",
        "print(\"Logistic Regression (Tf-idf) Precision: \" + str(precision_score(grid_lr_clf_preds, y_test)))\n",
        "print(\"Logistic Regression (Tf-idf) Recall: \" + str(recall_score(grid_lr_clf_preds, y_test)))"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Logistic Regression (Tf-idf) Accuracy: 0.6764162487703049\n",
            "Logistic Regression (Tf-idf) Precision: 0.6831345826235093\n",
            "Logistic Regression (Tf-idf) Recall: 0.6740772308639345\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FjpC-85-rUwZ"
      },
      "source": [
        "#### Logistic Regression w/ Tf-idf Representation - Confusion Matrix"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "45-4QOdqrUwZ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "c766b72d-a8b4-4337-bc89-a574313e9f71"
      },
      "source": [
        "cm = confusion_matrix(y_test, grid_lr_clf_preds)\n",
        "cm_display = ConfusionMatrixDisplay(cm, display_labels=[0, 1]).plot()"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxV1b3+8c9DAEUZA0KVoVAFLeAAKuLQah0YbHux93rV2irXWnFA69TbOtxfuWqttrVaqVVLlat0QhxasUUjWuvQioqIIqBCHRkUIcxzku/vj72DJyE5OUeSkpw8b1/75Tlrr7332gl8WcPeaykiMDOzRIudXQAzs8bEQdHMLIODoplZBgdFM7MMDopmZhla7uwCZCoubhE9ezaqIlkd3n29/c4uguVhY8U6tsQm7cg5hn9p91hRWp5T3pdf21wSESN25Hr/ao0qAvXs2ZKSaV12djEsD6P7N6k/783ejHVTd/gcK0rLebGkV055i/Zc0OT+QjeqoGhmjV8AFVTs7GI0GAdFM8tLEGyN3JrPTZEHWswsbxU5/peNpJ6SnpI0T9JcSRen6QdJmiFptqSZkoak6ZI0XtJCSa9JGpxxrtGSFqTb6Iz0gyXNSY8ZL6nO/lQHRTPLSxCUR25bHcqAyyOiPzAUGCupP/AT4JqIOAj4QfodYCTQN93GAHcASCoGxgGHAUOAcZI6pcfcAZyTcVydneAOimaWtwoipy2biFgaEbPSz2uB+UB3km7LyscaOgBL0s+jgEmRmAF0lLQnMByYHhGlEbESmA6MSPe1j4gZkUzyMAk4qa57c5+imeUlgPI6Al6GLpJmZnyfEBETqmeS1BsYBLwAXAKUSLqJpOJ2RJqtO/BBxmGL0rRs6YtqSM/KQdHM8lZXLTDD8og4JFsGSW2BB4FLImKNpB8Cl0bEg5JOAe4Gjt+hAufBzWczy0sAWyNy2uoiqRVJQPxdRDyUJo8GKj/fT9JPCLAY6JlxeI80LVt6jxrSs3JQNLO8BEF5jls26Ujw3cD8iLg5Y9cS4Oj087HAgvTzVODMdBR6KLA6IpYCJcAwSZ3SAZZhQEm6b42koem1zgQeruv+3Hw2s/wElNfP3NRHAmcAcyTNTtOuIhktvlVSS2ATyUgzwDTgRGAhsAE4CyAiSiVdB7yU5rs2IkrTzxcA9wBtgEfTLSsHRTPLS/JGSz2cJ+I5oLbnBg+uIX8AY2s510RgYg3pM4GB+ZTLQdHM8iTKa41lTZ+DopnlJRlocVA0MwMqn1N0UDQz26bCNUUzs4RrimZmGQJRXsCPODsomlne3Hw2M0sFYksU7exiNBgHRTPLS/LwtpvPZmbbeKDFzCwVIcrDNUUzs20qXFM0M0skAy2FGzoK987MrEF4oMXMrJpyP6doZpbwGy1mZtVUePTZzCyRTAjhoGhmBiTN560F/Jpf4YZ7M2sQEVAeLXLaspHUU9JTkuZJmivp4ox9F0l6I03/SUb6lZIWSnpT0vCM9BFp2kJJV2Sk95H0Qpp+n6TWdd2fa4pmlifV18PbZcDlETFLUjvgZUnTgW7AKODAiNgsqSuApP7AacAAYC/gCUn90nP9EjgBWAS8JGlqRMwDfgzcEhGTJd0JnA3cka1QrimaWV6C+qkpRsTSiJiVfl4LzAe6A+cDN0bE5nTfsvSQUcDkiNgcEe+QLHU6JN0WRsTbEbEFmAyMStd6PhZ4ID3+XuCkuu7PQdHM8lZOi5y2XEnqDQwCXgD6AV9Im71PSzo0zdYd+CDjsEVpWm3pnYFVEVFWLT0rN5/NLC+B8plktoukmRnfJ0TEhMwMktoCDwKXRMQaSS2BYmAocCgwRdLn6qHoOXFQNLO8JEuc5hw6lkfEIbXtlNSKJCD+LiIeSpMXAQ9FRAAvSqoAugCLgZ4Zh/dI06glfQXQUVLLtLaYmb9Wbj6bWZ5EeY5b1rMkfX53A/Mj4uaMXX8CvpTm6Qe0BpYDU4HTJO0iqQ/QF3gReAnom440tyYZjJmaBtWngJPT844GHq7r7lxTNLO8BPX2RsuRwBnAHEmz07SrgInAREmvA1uA0WmAmytpCjCPZOR6bESUA0i6ECgBioCJETE3Pd/3gcmSfgi8QhKEs3JQNLO81cfM2xHxHNR6om/Wcsz1wPU1pE8DptWQ/jbJ6HTOHBTNLC8R8rvPZmaVkoGWwn3Nz0HRzPLkNVrMzLZJBlo8yayZ2TaeOszMLJXnGy1NjoOimeXNC1eZmaUiYGuFg6KZGVDZfHZQNDPbpj7eaGmsCjfcf0orlrTmR6cM5PvHDuKK4wZRcvee2+XZsKaIn531ea4adhBXHDeIZ+7rusPXXbeyJTeePoDvfmEwN54+gPWrqj4c+/bstozufQQv/qXzDl+r0Fz6o7f4wz9e4I5HZmXN12//tfx57nMcNXz5Dl+zbYetXD/xde4qmcn1E1+nbfuyKvvr81qNTeUjOblsTVGDBsXa1k1ozIqKgtP/3zv8+K+vMO7h13ji3j1Z/FabKnmeuHdPuvfdwI8en81VU+bw++t6U7Yltz8A859vz68u3We79Edu786AI1dx07OzGHDkKh65vce2fRXlMPmGzzLwiyt37OYK1PSHuvE/3x6QNU+LFsFZ332XWX/vlNe59x+yistueGu79FPGLGL28x349vBDmP18B04Z88kcp5/2Wk1H0nzOZWuKGqzUkopI1k0YCfQHvp6usdCodey2ld77rwegTdty9tpnA6UfVlvrRsGmdUVEwKb1RezesYwWLQOAv9zZnR98+QCuOuEgHvxZz+qnr9WsxzvzhZOTWde/cPIyXi75pEb4+P/tyaEjV9C+89YdvLvC9PrMDqxdnb0n6N/OWMLfSzqzakWrKun/cfYibn1gNrdPncU3L3ov52seflwpT/ypGwBP/Kkbhx9fWue1CklFuk5LXVtT1JChvMZ1ExrwevXu4w924b25bdln0Loq6Sf814csWbgbFx1yKFedMIgzrnmHFi1gztMd+fCdXbnmz6/xw5LZvDunLW/MaJ/TtdYsb0XHbknQ69B1K2uWJ3+hSpe2ZuZjnTnuzA/r9+aakc5dN3PE8Sv4yx+qdoUMPnIl3T+7kYtPPpCxowaxz4B1DDxkdU7n7Nh5Cys/Tv6xXPlxKzp23pL1WoUkGX0uymlrihpyoKWmdRMOq55J0hhgDED37o3nh7hpfQvGn7sf3/jft2nTrrzKvjlPd6RX//Vced/rLHt3V278xgD2HTKbOc905PVnOvI/Iw5Mz1HEh+/uyn5D1zDuqwdQtkVsWl/E+lUtuXp4kufUK9/jgGNWVTm/xLYJlX57TR9Ou+pdWjTNlkijcO7VbzPxpt5EtT6uwUeuYvCRq7jtT8lUfm12K2ev3ht5fWYHbpkym1atgza7ldOuQxm3/ekVACbe1JtZz1VvFouI7NcqJH54u4Gl6zVMADjwwNaxk4sDQNlWMX7Mfhxx0sccOrJ0u/3PTOnKVy9YjATd+mxij56bWLKwDQR8dewijv3mR9sdc80jrwFJn+IzU7py7i0Lq+xv32Urqz5KaourPmq1ran8zmtt+eXYfQFYW9qKV5/qRIui4JAR25fLatZ34DquuPlNANp32sqhR6+kvEyg4L4JPXj0vu1rdZeechCQ9Cme8LVl3Hxlvyr7V61oTac9ktpipz22sLq0ddZrPf9kYQ2QNdWmcS4aMihmW0+h0YqAu/57H/bqu5GRY5bUmKfzXpuZ+/cO7HvYGlZ/3IoP/9mGrp/dxP5Hr+KBm3pxxNc+ZtfdKyhd2pqiVkGHLnX3BQ4+oZRnH+jKV8cu5tkHujJ42AoAbvnHy9vy/OrSfRh0/EoHxDydddyh2z5fdsNbvPi3Yp5/sjObN7XgjIvf46lHurJpQxGdu26mrEzbAlw2M/5azPEnfcT9v+7J8Sd9xPNPFme9ViHxhBCf3rZ1E0iC4WnA6Q14vXrx1kvt+PuDXem53/ptTdz//P77rFi8CwDHnfEhJ128iAmX7cOVxx9EBJx61Xu0Ky5j/6NXsWRhG64ZdQAAu+5eznm3LsgpKH5l7CJuO39fnp7cjS49NnPh7W823E0WmO//7A0OGLKa9p3K+M3TL/KbX/SiZTrwNW1y7X17s/7eiZ57b+Dmya8CsGlDET/9736szuHfnCkTenDVz99g+MkfsWzJLvzokv3q5V6aiqY6spwLRTRci1XSicDP+WTdhO2mEc904IGto2RalwYrj9W/0f1H7OwiWB5mrJvK6vLlO1TN67Rf1zh24sl1ZwQeOvKOl7Ot5tcYNWifYm3rJphZ01bIzefCrQObWYOorzdaJPWU9JSkeZLmSrq42v7LJYWkLul3SRqfvgzymqTBGXlHS1qQbqMz0g+WNCc9Zny6rGpWDopmlrd6es2vDLg8IvoDQ4GxlS94SOoJDAPez8g/kmSt574kj/HdkeYtBsaRPPI3BBgnqfK5qTuAczKOq7O/x0HRzPJS+ZzijgbFiFgaEbPSz2uB+STPNwPcAnyPpGJaaRQwKRIzgI6S9gSGA9MjojQiVgLTgRHpvvYRMSNdN3oScFJd97fTn1M0s6Ynj+cUu0iamfF9QvpschWSegODgBckjQIWR8Sr1Vq7Nb0Q0r2O9EU1pGfloGhmeYmAstwnmV1e1+izpLbAg8AlJE3qq0iazjuFm89mlrf6mjpMUiuSgPi7iHgI2BvoA7wq6V2Slz5mSfoMtb8Qki29Rw3pWTkomlle6qtPMR0JvhuYHxE3A0TEnIjoGhG9I6I3SZN3cER8CEwFzkxHoYcCqyNiKVACDJPUKR1gGQaUpPvWSBqaXutM4OG67s/NZzPLWz1NeHEkcAYwR9LsNO2q9PnmmkwDTgQWAhuAs5KyRKmk60jeogO4NiIq30u6ALgHaAM8mm5ZOSiaWd7qY0KIiHgOsp8orS1Wfg5gbC35JgITa0ifCQzMp1wOimaWl4jCfqPFQdHM8iTKvcSpmdknCnkSXQdFM8uL51M0M8sU0IAzDu50DopmljcvR2BmlgoPtJiZVeXms5lZBo8+m5mlIhwUzcyq8CM5ZmYZ3KdoZpYKRIVHn83MPlHAFUUHRTPLkwdazMyqKeCqooOimeWtWdYUJf2CLP8eRMR3GqREZtaoBVBR0QyDIjAzyz4za64CaI41xYi4N/O7pN0iYkPDF8nMGrv6eE5RUk9gEtCNJNROiIhbJf0U+CqwBfgncFZErEqPuRI4GygHvhMRJWn6COBWoAi4KyJuTNP7AJOBzsDLwBkRsSVbuep82EjS4ZLmAW+k3w+UdHue929mhSRy3LIrAy6PiP7AUGCspP7AdGBgRBwAvAVcCZDuOw0YAIwAbpdUJKkI+CUwEugPfD3NC/Bj4JaI2AdYSRJQs8rlCcyfA8OBFQAR8SrwxRyOM7OCJCJy27KJiKURMSv9vBaYD3SPiMcjoizNNoNPFrQfBUyOiM0R8Q7JUqdD0m1hRLyd1gInA6PStZ6PBR5Ij78XOKmuu8vpsfSI+KBaUnkux5lZgaqfmuI2knoDg4AXqu36Fp+s1dwdyIxFi9K02tI7A6syAmxlela5PJLzgaQjgJDUCriYJKKbWXMUELmPPneRlDloOyEiJmRmkNQWeBC4JCLWZKRfTdLE/t0OljgvuQTF80g6MLsDS4ASalmQ2syai5yD4vKIOKTWsyQVrQeB30XEQxnp/wV8BTguYtuwzmKgZ8bhPdI0aklfAXSU1DKtLWbmr1WdQTEilgPfqCufmTUj9TP6LOBuYH5E3JyRPgL4HnB0tSdepgK/l3QzsBfQF3iRJEL3TUeaF5MMxpweESHpKeBkkn7G0cDDdZUrl9Hnz0l6RNLHkpZJeljS53K7bTMrSPXTp3gkcAZwrKTZ6XYicBvQDpiept0JEBFzgSnAPOAxYGxElKe1wAtJWrHzgSlpXoDvA5dJWkjSx3h3XYXKpfn8e5Lh7q+l308D/gAclsOxZlZo6unh7Yh4jprb4dOyHHM9cH0N6dNqOi4i3iYZnc5ZLqPPu0XEbyKiLN1+C+yaz0XMrLBE5LY1RdnefS5OPz4q6QqSNnkAp5IlkptZM9BM331+mSQIVt79uRn7gvQpczNrftREa4G5yPbuc59/ZUHMrInI88Hspian+RQlDSR5p3BbX2JETGqoQplZY6bmOUtOJUnjgGNIguI0kpeunyOZ3cLMmqMCrinmMvp8MnAc8GFEnAUcCHRo0FKZWeNWkePWBOXSfN4YERWSyiS1B5ZR9ZUaM2tOmusksxlmSuoI/JpkRHod8HyDlsrMGrVmOfpcKSIuSD/eKekxoH1EvNawxTKzRq05BkVJg7Ptq5wc0syskGSrKf4sy74gmdG2Xr3zWlvO6HlkfZ/WGlDJkmd3dhEsD0OGr6uX8zTL5nNEfOlfWRAzayKCZvuan5lZzZpjTdHMrDbNsvlsZlarAg6Kucy8LUnflPSD9HsvSXlN2mhmBaaeV/NrTHJ5ze924HDg6+n3tSQzcZtZM6TIfWuKcmk+HxYRgyW9AhARKyW1buBymVlj1sxHn7dKKiKtDEvagyb7qreZ1YemWgvMRS7N5/HAH4Gukq4nmTbsRw1aKjNr3OqhT1FST0lPSZonaa6ki9P0YknTJS1I/98pTZek8ZIWSnot8607SaPT/Askjc5IP1jSnPSY8emyqlnVGRQj4ncka7DeACwFToqI++s6zswKVP31KZYBl0dEf2AoMFZSf+AK4MmI6As8mX6HZC7Xvuk2BrgDtq0nNY5khdEhwLjKQJrmOSfjuBF1FSqX0edewAbgEZLFqNenaWbWXNVDTTEillbOoRARa0nWbO4OjALuTbPdC5yUfh4FTIrEDKCjpD2B4cD0iCiNiJXAdGBEuq99RMyIiCCZGLvyXLXKpU/xL3yygNWuQB/gTWBADseaWQFS7qMKXSTNzPg+ISImbHc+qTcwCHgB6BYRS9NdHwLd0s/dgQ8yDluUpmVLX1RDela5TB22f7XCDwYuqCW7mVmm5RFxSLYMktoCDwKXRMSazG6/iAjpXzusk8tASxVpdfewBiiLmTUV9fTwtqRWJAHxdxHxUJr8Udr0Jf3/sjR9MVVn/e+RpmVL71FDela5LFx1WcbXFsBgYEldx5lZgaqnB7PTkeC7gfkRcXPGrqnAaODG9P8PZ6RfKGkyScVsdUQslVQC/ChjcGUYcGVElEpaI2koSbP8TOAXdZUrlz7Fdhmfy0j6GB/M4TgzK1T106A9EjgDmCNpdpp2FUkwnCLpbOA94JR03zTgRGAhyeDvWQBp8LsOeCnNd21ElKafLwDuAdoAj6ZbVlmDYvrQdruI+G4ON2hmzUU9BMWIeI5kALcmx9WQP4CxtZxrIjCxhvSZwMB8ypVtOYKWEVEmyVNhm9k2Iq/R5yYnW03xRZL+w9mSpgL3A+srd2Z0ippZc9KEJ3vIRS59irsCK0jWZKl8XjEAB0Wz5qqZBsWu6cjz63wSDCsV8I/EzOpUwBEgW1AsAtpSc0doAf9IzKwuzbX5vDQirv2XlcTMmo5mGhQLdxZJM/v0ovmOPm/3nJCZGdA8a4oZT4SbmVXRXPsUzcxq5qBoZpZqwsuX5sJB0czyItx8NjOrwkHRzCyTg6KZWQYHRTOzlGfJMTOrxkHRzOwTzfU1PzOzGhVy8znvJU7NrJnLdXnT3JY4nShpmaTXM9IOkjRD0mxJMyUNSdMlabykhZJeS9egrzxmtKQF6TY6I/1gSXPSY8Yrc1HpWjgomln+6ikokqy0N6Ja2k+AayLiIOAH6XeAkUDfdBsD3AEgqRgYR7Ls6RBgXMZyp3cA52QcV/1a23FQNLO8VL7RkstWl4h4Bqg++UwA7dPPHfhknflRwKRIzAA6StoTGA5Mj4jSiFgJTAdGpPvaR8SMdCXAScBJdZXJfYpmljdVNGin4iVAiaSbSCpuR6Tp3YEPMvItStOypS+qIT0r1xTNLD/59Sl2SfsFK7cxOVzhfODSiOgJXArcXe/3kIVrimaWtzxGn5dHxCF5nn40cHH6+X7grvTzYqBnRr4eadpi4Jhq6X9L03vUkD8r1xTNLH/1N9BSkyXA0ennY4EF6eepwJnpKPRQYHVELAVKgGGSOqUDLMOAknTfGklD01HnM4GH67q4a4pmlrf6ek5R0h9IanldJC0iGUU+B7hVUktgE8lIM8A04ERgIbABOAuSVQIkXQe8lOa7NmPlgAtIRrjbAI+mW1YOimaWv3oKihHx9Vp2HVxD3gDG1nKeicDEGtJnAgPzKZODopnlpxmv5mdmth3PvG1mVl0UblR0UDSzvBVyTdGP5NTgspvf577X5vKrv75Z4/7Dh6/mjife5Pbpb/KLR99iwJB1O3zNdh3LuGHyP5n43HxumPxP2nYoq7K/34EbmPb+qxz15VU7fK1Cs2xxK/775L055+j9OOeYffnjXV22y7N+TQt+cGYfzjt+X845Zl9KJhfv8HXXrCziilP35qwjP88Vp+7N2lVFVfa/ObsNI3seyLN/7rDD12pU6nFCiMaowYJiTbNfNBWP31fM1d/oU+v+V55ty/nH9+OCE/bl5st6culNi2rNW90Bh6/j8lve3y79lAuX8cpzbfnWUZ/nlefacuqFy7bta9EiOPvqpbz8dLv8bqSZKGoZjPnBEn799Bvc+ucFPHJPF957a5cqeabe04Ve/TZx5xNv8tMHFzLh2r3YuqXOCVMAePUfbbnpkl7bpU+5rSuDjlrL//19PoOOWst9t3Xdtq+8HO6+fi8OPnrtjt1cI6WK3LamqCFriveQw4wUjdHrL7Rl7craexY2bSgi6W6GXXerqNK9cvL5yxg/7S3ueOJNzvjuhzlf8/Dha3hiSlJ7eWJKMYePWLNt36hvLee5aR1Ytdy9HTXp3K2MvgdsBGC3thX03Gczy5e2qpJHgo3ri4iATeuLaNexnKKWyS/u/tv34KKR/TjvuH2Z9NPP5Hzd50s6cPwpyeNwx59SyvOPfVIjfHjiHhx14mo6dimr7fAmzUHxU6hl9ouCccSI1dz1zBtcN+kdbr4sefNo8NFr6d5nM985sS8XnNCPvvtvYOBhuTWtO3XZSumy5C9y6bKWdOqyFYDOn9nKESNX8+d7OzfMjRSYDz9ozT9fb8N+gzdUSf+3s5bz/oJdOH3QAM49dl/Ov3YxLVrAy39rx+J3dmH8tLe4ffqbLJjThjkzds/pWiuXt6JztyToFXctY+Xy5Pe3fGkr/vFoB74yenn93lxjESQDLblsTdBOr3qkL4iPAdiV3XZyaXL3j8c68I/HOjDwsHWM/t6HXHHq3hx89FoGH72W26e/BUCb3Sro/rnNvP5CW2798wJa7VJBm90qaNexnNunJ/2Vd/9wT15+un21s4uIpCZ63jWLufv6Pbd9t9ptXN+C677dm/OuXczu7apWU17+Wzv2HrCRn9z/T5a825orT9ubgYet4+Wn2zHr6fZccMK+yTk2tGDx27uw/9D1fOfLfdm6uQUbN7Rg7aoizj8+yXP2/yzhkGOqNoslUDr6cOe47px99RJaFHCPfSEPtOz0oBgRE4AJAO1V3OR+1K+/0JbP9PqA9sVlCLjvF92Y9tvta3UXf6UvkPQpnnBKKT+7tGof1crlrSjumtQWi7tuZdWK5FfT78CNXHnHewB0KC5nyHFrKS9XlaaaQdlWuO7bvTn231dy1Imrt9v/+H3FnHLhMiTo3mcLn+m1hQ8W7koAp170EV8+Y8V2x4z/S/LK7av/aMv0KcV89+dV+4I7ddnKio9a0rlbGSs+aknHzkmt8a1X23DD+b0BWF1axItPtqOoCI4YuX25mqwm9zc1dwX8b1nD2av3Zir/VOyz/wZata5gTWkRM59ux/DTStl1t3Igafp26Lw1p3POeLx91f6pkqT2OHro5xl9WH9GH9afZ//cgV9c2d0BsZoIuPnyXvTsu5n/OPfjGvPs0X0rs59NBqpWftySRf/chT17beaQo9dSMrmYjeuTvwrLl7bKue926LBq/cDDk6A36YX5THpxHpNenMcXvrKai25YVFABsT4nmW2MdnpNsTG64vb3OODwdXQoLuO3M+fxm591o2XaKf+X33ThqC+v5viTSykrE5s3tuBH538WELOebkevfTbx80cWAklz7icX9WL19pWQ7dx3W1euvvM9RpxWyrLFrbn+3M824B0Wlrkv7s6TDxTT5/MbtzVxz7pyCcsWtwbgK2eu4BuXfMhNl/Ti3GP3JQLOvnopHTqXc/Axa3l/4S5c8tWkJt9m9wq+94v36Lj9Uz3bOfXCj7j+vN48NrkzXbtv4epfvdtQt9i4RDT0JLM7laKBOkMzZ78APgLGRUTWySLbqzgO03ENUh5rGCVLZu/sIlgehgz/gJmvbtqhDup2HXvEoC9eXHdG4NlHvvfyp5hPcadqsJpiltkvzKyJa6pN41y4+Wxm+QmggJvPDopmlr/CjYkOimaWPzefzcwyFPLos4OimeWnCc+AkwsHRTPLS/LwduFGRb/RYmb5q8hxq0NtUwxKukjSG5LmSvpJRvqVkhZKelPS8Iz0EWnaQklXZKT3kfRCmn6fpNZ1lclB0czypoicthzcQ7UpBiV9CRgFHBgRA4Cb0vT+wGnAgPSY2yUVSSoCfgmMBPoDX0/zAvwYuCUi9gFWAmfXVSAHRTPLTz3OvF3LFIPnAzdGxOY0T+WMy6OAyRGxOSLeIVn/eUi6LYyItyNiCzAZGCVJwLHAA+nx9wIn1VUmB0Uzy1Py7nMuG8ki9zMztjF1nR3oB3whbfY+LenQNL078EFGvkVpWm3pnYFVEVFWLT0rD7SYWf5yH2hZ/inefW4JFANDgUOBKZI+l+c5PjUHRTPLTzT4UgOLgIcima3mRUkVJBPLLAZ6ZuTrkaZRS/oKoKOklmltMTN/rdx8NrP8NexyBH8CvgQgqR/QGlgOTAVOk7SLpD5AX+BF4CWgbzrS3JpkMGZqGlSfAk5OzzsaeLiui7umaGb5q6fHFDOnGJS0CBgHTAQmpo/pbAFGpwFurqQpwDygDBgbEeXpeS4ESoAiYGJEzE0v8X1gsqQfAq8AWacvBAdFM/sUVFE/7ecsUwx+s5b81wPX15A+DZhWQ1GgzxUAAAaoSURBVPrbJKPTOXNQNLP8BDk9mN1UOSiaWV5Ezg9mN0kOimaWPwdFM7MMDopmZin3KZqZVVVfo8+NkYOimeVphx7MbvQcFM0sP4GDoplZFYXbenZQNLP8+TlFM7NMDopmZqkIKC/c9rODopnlzzVFM7MMDopmZqkAKhwUzcxSAeE+RTOzROCBFjOzKtynaGaWoYCDolfzM7M85biSXw6BU9JEScvSRaqq77tcUkjqkn6XpPGSFkp6TdLgjLyjJS1It9EZ6QdLmpMeM16S6iqTg6KZ5SeAiorctrrdA4yoniipJzAMeD8jeSTJsqZ9gTHAHWneYpJVAA8jWaRqnKRO6TF3AOdkHLfdtapzUDSz/NVTTTEingFKa9h1C/A9qi6mOgqYFIkZJAvd7wkMB6ZHRGlErASmAyPSfe0jYka6ROok4KS6yuQ+RTPLU16v+XWRNDPj+4SImJDtAEmjgMUR8Wq11m534IOM74vStGzpi2pIz8pB0czyExC5P6e4PCIOyTWzpN2Aq0iazjuFm89mlr+KyG3L395AH+BVSe8CPYBZkj4DLAZ6ZuTtkaZlS+9RQ3pWDopmlr966lPc/rQxJyK6RkTviOhN0uQdHBEfAlOBM9NR6KHA6ohYCpQAwyR1SgdYhgEl6b41koamo85nAg/XVQY3n80sPxG5jizXSdIfgGNI+h4XAeMi4u5ask8DTgQWAhuAs5LiRKmk64CX0nzXRkTl4M0FJCPcbYBH0y0rB0Uzy189PbwdEV+vY3/vjM8BjK0l30RgYg3pM4GB+ZTJQdHM8hREefnOLkSDcVA0s/x46jAzs2o8dZiZWSKAcE3RzCwVnmTWzKyKQh5oUTSiedEkfQy8t7PL0QC6AMt3diEsL4X6O/tsROyxIyeQ9BjJzycXyyOizplpGpNGFRQLlaSZ+bz/aTuff2fNl1/zMzPL4KBoZpbBQfFfI+v8cdYo+XfWTLlP0cwsg2uKZmYZHBTNzDI4KDYgSSMkvZkur3jFzi6P1S3bkpvWPDgoNhBJRcAvSZZl7A98XVL/nVsqy8E95LAMphUuB8WGMwRYGBFvR8QWYDLJEo3WiGVZctOaCQfFhlPbsotm1og5KJqZZXBQbDi1LbtoZo2Yg2LDeQnoK6mPpNbAaSRLNJpZI+ag2EAiogy4kGRN2vnAlIiYu3NLZXVJl9x8HthX0iJJZ+/sMtm/ll/zMzPL4JqimVkGB0UzswwOimZmGRwUzcwyOCiamWVwUGxCJJVLmi3pdUn3S9ptB851j6ST0893ZZusQtIxko74FNd4V9J2q77Vll4tz7o8r/W/kr6bbxnNqnNQbFo2RsRBETEQ2AKcl7lT0qdaxzsivh0R87JkOQbIOyiaNUUOik3Xs8A+aS3uWUlTgXmSiiT9VNJLkl6TdC6AErel8zs+AXStPJGkv0k6JP08QtIsSa9KelJSb5Lge2laS/2CpD0kPZhe4yVJR6bHdpb0uKS5ku4CVNdNSPqTpJfTY8ZU23dLmv6kpD3StL0lPZYe86yk/erjh2lW6VPVLGznSmuEI4HH0qTBwMCIeCcNLKsj4lBJuwB/l/Q4MAjYl2Rux27APGBitfPuAfwa+GJ6ruKIKJV0J7AuIm5K8/0euCUinpPUi+Stnc8D44DnIuJaSV8Gcnkb5FvpNdoAL0l6MCJWALsDMyPiUkk/SM99IcmCUudFxAJJhwG3A8d+ih+jWY0cFJuWNpJmp5+fBe4mada+GBHvpOnDgAMq+wuBDkBf4IvAHyKiHFgi6a81nH8o8EzluSKitnkFjwf6S9sqgu0ltU2v8e/psX+RtDKHe/qOpK+ln3umZV0BVAD3pem/BR5Kr3EEcH/GtXfJ4RpmOXNQbFo2RsRBmQlpcFifmQRcFBEl1fKdWI/laAEMjYhNNZQlZ5KOIQmwh0fEBkl/A3atJXuk111V/WdgVp/cp1h4SoDzJbUCkNRP0u7AM8CpaZ/jnsCXajh2BvBFSX3SY4vT9LVAu4x8jwMXVX6RVBmkngFOT9NGAp3qKGsHYGUaEPcjqalWagFU1nZPJ2mWrwHekfSf6TUk6cA6rmGWFwfFwnMXSX/hrHTxpV+RtAj+CCxI900imQmmioj4GBhD0lR9lU+ar48AX6scaAG+AxySDuTM45NR8GtIgupckmb0+3WU9TGgpaT5wI0kQbnSemBIeg/HAtem6d8Azk7LNxcv8WD1zLPkmJllcE3RzCyDg6KZWQYHRTOzDA6KZmYZHBTNzDI4KJqZZXBQNDPL8P8BSFgfevh1QqwAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lt8KeIturUwZ"
      },
      "source": [
        "#### Logistic Regression w/ Tf-idf Representation - Precision Recall Curve"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cCUVCfHXrUwZ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 409
        },
        "outputId": "e12cbb91-82f1-4eae-ac67-e637dd59bccd"
      },
      "source": [
        "create_pr_curve('Logistic Regression (Tf-idf) Precision-Recall Curve', grid_lr_clf_preds, y_test)"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAGICAYAAACjo4zRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hUZfr/8fedTkIKCaGlEyARFVABQVoUC0XFtiooCiIorPvbdVUsu4uKXdRd/QoouhSxK+qiglioIiiggFQJNaGGEkICIe35/XEmYRISSJ0zydyv68qVmXPOnHOfM8l8TpvnEWMMSimlPJuX3QUopZSyn4aBUkopDQOllFIaBkoppdAwUEophYaBUkopNAzqjIi8ISL/qsbrYkUkW0S866IudyUic0Xkzjqa91Ui8kUlp20kIl+KyFER+aSc8Wd8X0XEiEib8uYlIteIyEfVX5PaISK3ici3lZiuWn/D7kpEdojI5Y7HT4jIu3bX5FaMMR7/A+wALq/PywaGAYVANpAFrAGutnvbusMPsBLoBsQ6tk/xjwFynJ73AoYCvwA+1VyWAdo4Hp82L2Ad0OEsfw8nHPXsB6YDje3ehnXwnjhv+93AK4B3HS+z5H8NeAJ49wzThgD/AXY5atzqeN7U7m1XVz96ZNCwLDPGNAbCgEnAhyISVtsLqU9HLSLSBQg1xiw3xuwyxjQu/nFM0tFp2BIgDvjDGFNQC4svb14fAKPO8rprHPVdCHQG/ll2AhHxqYX67NbRsZ59gFuAu2yuBwAR8QN+AM4F+mEFQ3fgENC1GvOrF++VhsEZiIi/iPxHRPY4fv4jIv5O48eKyF7HuLvLnCKYLiJPOx43FZGvRCRTRA6LyBIR8RKRmVh7q186Tg2NFZF4x3x8HK8NF5FpjmUcqczpDmNMETATCALaOq3LSyKyS0T2O04BNKrCukwWkTkikgNcKiKtRGSWiGSIyHYR+X9O8+oqIitFJMuxrFccwwNE5F0ROeTYFitEpLlj3EIRudvx2EtE/ikiO0XkgIi8IyKhjnHF2+dOx7ocFJF/nGFz9AcWnW2bOeb9JDAOuMXxfowoZ5qS99Xx/CGn7XZXJea1EBhYmXqMMbuBucB5jnkaEfmziGwBtjiGXS0iqx3b8ycR6eBUQ4yIfOZ4jw6JyOuO4cNE5EfHYxGRfzu2c5aI/C4ixcsru64jRSTV8Tc8W0RaOY0zInKviGxx1DJRRKSS65kKLAU6Oc2vOuuVKCLzHcMOish7Ur2doTuw/i+vN8ZsMMYUGWMOGGOeMsbMcVrfNk41Of+/p4hIuog8LCL7gGkislFErnaa3sdR/4WO590c65kpImtEJKUaddeIhsGZ/QPr9EInoCPWXsE/AUSkH/B34HKgDZByhvk8AKQDkUBz4DHAGGOGYh2GXuPYM32xnNfOBAKx9lKaAf8+W9Fi7bkPB/KBnY7BzwPtHOvSBojC+rCq7LoMAZ4BgoGfgC+xTkVFAX2Bv4nIVY5pXwVeNcaEAInAx47hdwKhQAwQAdyLdUqkrGGOn0uB1kBj4PUy0/QEkhzLHici51SwOc4HNlcwrhRjzOPAs8BHjvfjv2ea3rHdHgSuwArdyysxr41AvIiEnK0eEYkBBgC/OQ2+DrgYaC8iFwBTgXuwtuebwGyxgt8b+Arr/Y/Hep8+LGcxVwK9sf42QoGbsfaAy9ZyGfCcY3xLx3zLzu9qoAvQwTHdVVSCiCRjnaJLdTyv7nqJo8ZWwDlYf2dPVKaGMi4HvjHGZFfjtcVaAOFYR4ejsI4IBzuNvwo4aIz5VUSigK+Bpx2veRCYJSKRNVh+lWkYnNltwHjHXkEG8CTWeWCw/tinGWPWG2OOc+Y/unysf6A4Y0y+MWaJcZyYPBMRaYm1Z3uvMeaI47Vn2svtJiKZQC7wEnC7MeaAYw9tFHC/MeawMeYY1gfVrVVYl/8ZY5Y6jjrOByKNMeONMXnGmG3AW07zywfaiEhTY0y2MWa50/AIrHPqhcaYVcaYrHKWdRvwijFmm+Mf8lHgVil9uP2kMeaEMWYNVih1rGCbhAHHzrDNaqJ4u60zxuRQuQ+e4lrOtMf6heN9/BHrqOZZp3HPOd7DE1jv6ZvGmJ8d23MGcBJrB6Yr1ofiQ8aYHGNMrjHmx3KWlY8V8MmAGGM2GmP2ljPdbcBUY8yvxpiTWO9JdxGJd5rmeWNMpjFmF7AApz39Cvwq1pHmRqwjpkmO4dVaL2NMqjHmO2PMScf/6ytYp6CqKgIobxtURRHwuKOWE8D7wLUiEugYPwQrIABuB+YYY+Y4jkK+w7rONaCGNVSJhsGZteLUnjWOx62cxqU5jXN+XNYErL2eb0Vkm4g8UsnlxwCHjTFHKjn9cmNMGNAEmI21twXWEUkgsMpxGJoJfOMYDpVbF+dhcUCr4nk55vcY1lEPwAisPc1NYp0KKj48ngnMw7qWsUdEXhQR33KWVd5293GaP8A+p8fHsY4eynME68OuykTkMccpnmwReaOCOp23y85ypimruJbMM0xznTEmzBgTZ4wZ4/gwKVb2fXigzPsQ46grBth5tmsfxpj5WEddE4EDIjKlgqOWUu+JI6QPYe2ZFyv3PRGR9U7bsZfTNBc6prkF62gnqCbrJSLNReRDEdktIlnAu0DTM61/BQ5h7bzVRIYxJrf4ieNU2EbgGkcgXIsVEGCt75/KrG/PWqihSjQMzmwP1htVLNYxDKw9h2incTEVzcQYc8wY84AxpjXWH8HfRaRv8egzLD8NCK/qeU/HP+poYKjjkPsg1umYcx0fMmHGmFCni6iVWRfnOtOA7U7zCjPGBBtjBjiWv8UYMxjrtNYLwKciEuQ4snnSGNMeuATrtMId5SyrvO1egHV3TVWtxQqmKjPGPOt0cfnecibZS+ltFVuJ2Z4D7KjgiKhSZTk9TgOeKfM+BBpjPnCMi5VKXLw0xrxmjLkIaI+1rR4qZ7JS74mIBGHtQe+uxPzPNaUv0juPM8aYj4FlOE5b1mC9nsXaPucb6xTl7Vinjqrqe+AqxzpW5DjWDlaxFmXGl/d/XXyqaBCwwREQYK3TzDLrG2SMeb4atVebhsEpvmJd4Cz+8cF68/4pIpEi0hTrj7X43uSPgeEico4j6c907/nVItLGcbrmKNYtoEWO0fuxzoufxnG4PheYJCJNRMRXRHpXZmWMMYeBt4FxjlM7bwH/FpFmjpqinM7xV3pdHH4BjjkukDUSEW8ROU+sO3cQkdtFJNKx3OI94CIRuVREznec983COkVRVM78PwDuF5EEEWnMqXPv1bnDZw7VO1VQGR8Dw0SkvWO7PV6J1/TBek9rw1vAvSJysViCRGSgiARjvUd7gecdwwNEpEfZGYhIF8frfbFu9cyl4vdkuIh0EusmimeBn40xO2ppXZ4HRopIixqsVzDWbaBHHefhywu1ypiJ9QE9S0SSxbqhIcJxpFh86mY1MMTxt9+Pyv2NfYh1jWY0p44KwPpMuUas78N4O9YpRUSiy51LHdEwOGUO1t5z8c8TWBd0VmLtXf4O/OoYhjFmLvAa1rnRVKD4vPjJcubdFmtvIxtrD2iSMWaBY9xzWIGTKSIPlvPaoVgfmpuAA8DfqrBO/wEGiHUnxsPFdToOob/HugBb1XXBGFOItVffCdiOdeTxNtYFSLBux1svItlYF5NvdZzqaAF8ihUEG7HOh88sZxFTHcMXO+afC/ylCuvtXOuvWB8OF1fn9WeZ91ysbTwfa7vNr8TLBmNdEK2N5a8ERmKd5jniqGGYY1whcA3WDQG7sG5guKWc2YRgffgewToNdAjrtGbZZX2PtZMwC+vDOJFT14hqY11+x3q/H6rBej2JderpKNYF2c+qWctJrIvIm4DvsP5ef8E65fSzY7K/OurIxLqeUpm7/PZi/f9fAnzkNDwN62jhMSADK4gewsWfz2LOfh1TVYJYd7OsA/yruQfrNhrSugCIyJXAGGPMdTbXcQ0w1Bhzs511KFUeDYMaEJHrsY4oAoEZQJHdHzjV1ZDWRSlVdXqaqGbuwTp1sxXrOsBoe8upkYa0LkqpKtIjA6WUUnpkoJRSytVXq0WmitUGyroKxouIvCZW+ydrxdFuh1JKqbrl6tb0pmPdLvZOBeP7Y92G2RbrG4mTHb/PqGnTpiY+Pr52KlSqMo4ehVTHd4ZEICkJgsr5jlJODmzeDMWnY9u0gdDQ06dTqpatWrXqoDGm0u0buTQMjDGLpXRbJmUNAt4x1oWM5SISJiItK2grpUR8fDwrV66sxUqVOouBA0+FwQMPwATr1vz8wiK8RfDycvri64MPwssvW4+TkuCrr1xcrPJEIlKZ5lFKuNs1gyhKt72STum2T5RyD0uXnno8wmqdeu/RE1zxyiK+WF2mhYYRI8p/nVJuxN3CoNJEZJRYbeavzMjIsLsc5WmOOTWEmpgIQPPgABoH+PDyt3+Qm1942ngAsqrbJJFSdcvdwmA3pRv+iqaChrCMMVOMMZ2NMZ0jI13a7LdSEOzUEOrWrQB4eQmP9DuH3ZkneHf5ztPGAxBy1m4MlLKFu4XBbOAOx11F3YCjZ7teoJQteji1+fbfU33g9GzblF5tm/L6glSOnsg/bXyp1ynlRlx9a+kHWA01JYnVLdwIsbrKK24eeA6wDathqreAMa6sT6lKG+30Be3XX4dly0qePtwvmczj+byxaKs1/PXXy3+dUm7E1XcTDT7LeAP82UXlKFV9/ftD+/awYQPk5sKll8J998GIEZyXmMg9kbk0feIfmNVfIycdjb+2bw/9+tlbt1IVaBDNUXTu3NnoraXK5TZvtk77HDqty+DTRUTATz9Bu2r1s6NUlYnIKmNM58pO727XDJSqP5KSrFtF27c/42Qn2yVrECi35+pvICvVsCQlwdq18M03MHmyFQ5ZWRASQl637vwtrBv5V17FWxoEys1pGChVU97e1jeSBw4sNdgPOHdBKhPmbWbFjsN0iQ+3pz6lKkFPEylVh+7qkUCzYH+em7ORhnB9TjVcGgZK1aFGft7cf0U7ft2Vybz1++0uR6kKaRgoVcf+dFE0iZFBvDhvEwWFRXaXo1S5NAyUqmM+3l483C+ZbRk5fLwy3e5ylCqXhoFSLnBF++ZcFNeEf3//B8fzCuwuR6nTaBgo5QIiwqP9k8k4dpKpP263uxylTqNhoJSLdI4P54r2zXlj0TYOZZ+0uxylStEwUMqFHu6XxPG8Al5fkGp3KUqVomGglAu1aRbMzZ1jeHf5TnYdOm53OUqV0DBQysXuv6Id3l7CS99utrsUpUpoGCjlYs1DAhjRM4HZa/awbvdRu8tRCtAwUMoW9/RJpEmgL8/P3WR3KUoBGgZK2SIkwJf7LmvLj6kHWbIlw+5ylNIwUMout3eLJbpJI56fu4miIm3ETtlLw0Apm/j7ePPglUms35PFl2v32F2O8nAaBkrZ6NqOrWjfMoQJ8zZzsqDQ7nKUB9MwUMpGXl7CI/2TST9ygneX77K7HOXBNAyUslnvdpH0bNOU1+dvISs33+5ylIfSMFDKDTzcL5kjx/N5c9FWu0tRHkrDQCk3cH50KNd2bMV/f9zO/qxcu8tRHkjDQCk38eCVSRQWGf7z/R92l6I8kIaBUm4iNiKQ2y6O46MVaaQeyLa7HOVhNAyUciN/uawNgX4+vPiNNlOhXEvDQCk3EtHYn3t6t+bbDftZueOw3eUoD6JhoJSbGdErgchgf56fuwljtJkK5RoaBkq5mUA/H/52eVtW7jzCdxv2212O8hAaBkq5oVs6x9C6aRAvzttMQWGR3eUoD6BhoJQb8vH2Ymy/JFIPZPPpqnS7y1EeQMNAKTd11bktuDA2jH9//wcn8rQRO1W3NAyUclMiwiP9z2F/1kmmLt1udzmqgdMwUMqNdU0I5/JzmvHGwq0czsmzuxzVgGkYKOXmxvZLJievgIkLUu0uRTVgGgZKubl2zYO56aJoZi7bSdrh43aXoxooDQOl6oH7r2iHCLzynTZip+qGhoFS9UDL0EYM75HAF6t3s37PUbvLUQ2QhoFS9cTolERCG/nywjeb7S5FNUAaBkrVE6GNfLnv0jYs/iODpakH7S5HNTAaBkrVI7d3iyMqrBHPzd1IUZE2Yqdqj4aBUvVIgK83D1zZjnW7s/jq9712l6MaEA0DpeqZQZ2iSG4RzEvzNpNXoI3YqdqhYaBUPePtJTzSP5ldh4/z/s877S5HNRAaBkrVQ33aRdK9dQSvzU/lWG6+3eWoBkDDQKl6SER4dEAyh3PyeGvxNrvLUQ2AhoFS9VSH6DCu7tCSt5Zs50BWrt3lqHpOw0CpeuzBK5PILyziPz9ssbsUVc9pGChVj8U3DeK2i2P5aEUaWzOy7S5H1WMaBkrVc3/p25YAHy8maDMVqgY0DJSq55o29mdU70S+Wb+PX3cdsbscVU9pGCjVANzdK4Gmjf15fs4mjNFmKlTVaRgo1QAE+fvw18vb8suOw8zfdMDuclQ9pGGgVANxa5cYEpoG8cI3myjURuxUFWkYKNVA+Hp78dBVSfyxP5tZq9LtLkfVMy4PAxHpJyKbRSRVRB4pZ3ysiCwQkd9EZK2IDHB1jUrVV/3Pa0HHmDBe+e4PcvML7S5H1SMuDQMR8QYmAv2B9sBgEWlfZrJ/Ah8bYy4AbgUmubJGpeozEeHR/snsy8pl2tIddpej6hFXHxl0BVKNMduMMXnAh8CgMtMYIMTxOBTY48L6lKr3urWO4LLkZkxamErm8Ty7y1H1hKvDIApIc3qe7hjm7AngdhFJB+YAfylvRiIySkRWisjKjIyMuqhVqXrr4X7JZJ8sYOKCVLtLUfWEO15AHgxMN8ZEAwOAmSJyWp3GmCnGmM7GmM6RkZEuL1Ipd5bUIpgbL4xmxk87ST9y3O5yVD3g6jDYDcQ4PY92DHM2AvgYwBizDAgAmrqkOqUakL9f0Q4EXvnuD7tLUfWAq8NgBdBWRBJExA/rAvHsMtPsAvoCiMg5WGGg54GUqqJWYY0Yfkk8n/+2mw17suwuR7k5l4aBMaYAuA+YB2zEumtovYiMF5FrHZM9AIwUkTXAB8Awo9+vV6paxqS0IdjfhxfnbbK7FOXmfFy9QGPMHKwLw87Dxjk93gD0cHVdSjVEoYG+/PnSNjw3dxM/bT3IJYl6xlWVzx0vICulatGdl8TTKjSA5+dqI3aqYhoGSjVwAb7e/P3KJNamH+Xr3/faXY5yUxoGSnmA6y+IIrlFMBPmbSa/sMjucpQb0jBQygN4ewkP90tm56HjfPDLLrvLUW5Iw0ApD5GSFMnFCeG8+v0Wsk8W2F2OcjMaBkp5CBHh0QHncCgnj7cWb7O7HOVmNAyU8iCdYsIYcH4L3lqyjQPHcu0uR7kRDQOlPMxDVyWTV1DE//2gjdipUzQMlPIwCU2DGNw1lg9+2cX2gzl2l6PchIaBUh7o//Vti5+PFy/N22x3KcpNaBgo5YEig/0Z2as1X/++l9VpmXaXo9yAhoFSHmpk79Y0bezHc3M2ajMVSsNAKU/V2N+H/9e3LT9vP8zCzdpKvKfTMFDKg93aJZa4iECen7uJwiI9OvBkGgZKeTA/Hy8euiqJzfuP8flvZTsdVJ5Ew0ApDzfw/JZ0jA7llW83k5tfaHc5yiYaBkp5OBHh4f7J7DmayzvLdthdjrKJhoFSiksSm5KSFMnEBVs5ejzf7nKUDTQMlFIAPNwvmazcfCYt1GYqPJGGgVIKgHNahnD9BVFM+2kHezJP2F2OcjENA6VUiQeuTALgle/+sLkS5WoaBkqpElFhjbizexyzfk1n074su8tRLqRhoJQq5c+XtqGxvw8vfqON2HkSDQOlVClhgX6MSWnD/E0HWL7tkN3lKBfRMFBKnWZ4j3hahATw3NxN2oidh9AwUEqdJsDXm79f0Y41aZnMXbfP7nKUC2gYKKXKdeNF0bRr3pgJ8zaTX1hkdzmqjmkYKKXK5e0lPNwvme0Hc/hwRZrd5ag6pmGglKrQZcnN6BofzqvfbyHnZIHd5ag6pGGglKqQiPDIgGQOZp/k7SXb7S5H1SENA6XUGV0Y24R+57ZgyuKtHMw+aXc5qo5oGCilzuqhfknkFhTxfz9ssbsUVUc0DJRSZ5UY2ZhbusTw3s+72HEwx+5yVB3QMFBKVcrf+rbF19uLl77VZioaIg0DpVSlNAsJYGSvBL5au5e16Zl2l6NqmYaBUqrSRvZuTVigL/83XzvAaWg0DJRSlRYc4Mud3eP5bsN+tuw/Znc5qhZpGCilqmTYJfE08vVm8qKtdpeiapGGgVKqSpoE+TG4ayyzV+8h/chxu8tRtUTDQClVZXf3SgDQbyU3IBoGSqkqaxXWiOsuiOLDFbs4pN9KbhA0DJRS1XJvn9acLChixk877C5F1QINA6VUtbRpFsyV7Zsz/acdZGuLpvWehoFSqtpGp7QhK7eAD37eZXcpqoY0DJRS1dYpJoxLEiN4+8dtnCwotLscVQMaBkqpGhmdksj+rJN8/utuu0tRNaBhoJSqkZ5tmnJeVAhvLt5GYZGxuxxVTRoGSqkaERHGpLRh+8Ec5q3fZ3c5qpo0DJRSNXbVuS1IaBrEpIWpGKNHB/WRhoFSqsa8vYR7erdm3e4sfkw9aHc5qho0DJRSteL6C6NoHuLP5IXagF19pGGglKoV/j7e3N2zNT9tPcTqNO38pr7RMFBK1ZrBF8cS2siXyQu185v6RsNAKVVrGvv7cGf3OOat30/qAe38pj7RMFBK1ao7L4knwNeLNxZts7sUVQUaBkqpWhXR2J9bu8TyxW+72ZN5wu5yVCW5PAxEpJ+IbBaRVBF5pIJpbhaRDSKyXkTed3WNSqma0c5v6h+XhoGIeAMTgf5Ae2CwiLQvM01b4FGghzHmXOBvrqxRKVVz0U0CubZTKz74ZReHc/LsLkdVgquPDLoCqcaYbcaYPOBDYFCZaUYCE40xRwCMMQdcXKNSqhbc2yeRE/mF2vlNPeHqMIgC0pyepzuGOWsHtBORpSKyXET6lTcjERklIitFZGVGRkYdlauUqq52zYO5on1zZizbQY52fuP23PECsg/QFkgBBgNviUhY2YmMMVOMMZ2NMZ0jIyNdXKJSqjJGpySSeTyfD37Rzm/cnavDYDcQ4/Q82jHMWTow2xiTb4zZDvyBFQ5KqXrmwtgmXJwQzttLtpNXUGR3OeoMXB0GK4C2IpIgIn7ArcDsMtN8gXVUgIg0xTptpDcsK1VPjbm0DfuycvlitXZ+485cGgbGmALgPmAesBH42BizXkTGi8i1jsnmAYdEZAOwAHjIGHPIlXUqpWpP77ZNad8yhDcWbdXOb9yYy68ZGGPmGGPaGWMSjTHPOIaNM8bMdjw2xpi/G2PaG2PON8Z86OoalVK1R0QYnZLItowcvtugnd+4K3e8gKyUamAGnN+SuIhAJi/cqp3fuCkNA6VUnbM6v0lkTfpRftqqZ33dkYaBUsolbrgwishg7fzGXWkYKKVcIsDXm7t7JvBj6kHWpmvnN+5Gw0Ap5TJDLo4lOMBHjw7ckM/ZJhCRO6oyQ2PMO9UvRynVkAUH+HJH9zgmLdzK1oxsEiMb212ScjhrGADTqzA/A2gYKKUqNLxHAm8v2c6URdt44aYOdpejHCoTBgl1XoVSymM0bezPLV1i+OCXXfztira0DG1kd0mKSlwzMMbsrMqPK4pWStVvI3u1psjAf7XzG7ehF5CVUi4XEx7ItR1b8f4vu8g8rp3fuIPKXEDejnUtoDKMMSaxZiUppTzBPX1a8/lvu5nx007+erk2TGy3ylwzWETlw0AppSoluUUIfZObMf2n7YzsnUCgX2U+jlRdOevWN8YMc0EdSikPNObSRG6cvIyPVqQxvIfeq2InvWaglLLNRXHhdI0P563F27TzG5tV67hMRDoCSUBA2XH6pTOlVFWMTklk+PQVzF6zh5suira7HI9VpTBw9EX8NdCteJDjt/M1BQ0DpVSlpSRFktwimDcWbeWGC6Lw8pKzv0jVuqqeJnoWiAB6YwXB9cBlwHtYXVN2rdXqlFINXnHnN6kHsvlu4367y/FYVQ2Dq7ACYbnjeboxZqEx5g7ge+CvtVmcUsozDDy/JbHhgUzSzm9sU9UwaAlsM8YUArlAsNO4z4CBtVWYUspz+Hh7Map3a9akZbJ822G7y/FIVQ2DfUCY4/FOoLvTuDa1UpFSyiPddFE0TRv7M2lhqt2leKSqhsGPnLp4PBN4XETeFJGJwARgXm0Wp5TyHAG+3tzVM54lWw6ybvdRu8vxOFUNgyc59YE/AZiIdWpoMDAb+EvtlaaU8jS3d4sj2N+HyYu08xtXq1IYGGO2GmOWOB7nG2MeMMZEG2PCjTFDjDHa07VSqtpCAny5vXscc3/fy/aDOXaX41GqFAYi4isiQRWMCxIR39opSynlqYb3iMfH24spi/XowJWqeprobeCtCsa96fhRSqlqaxYcwM2do5m1ajf7s3LtLsdjVDUMLgX+V8G42UDfmpWjlFIwqlciBUVF/PdH7fzGVaoaBs2AAxWMywCa16wcpZSC2IhAru7QiveW7+To8Xy7y/EIVQ2DA8D5FYw7H9ALyEqpWjE6JZGcvEJmLt9hdykeoaph8BXwLxHp4DxQRM4H/gF8WVuFKaU82zktQ7g0KZKpS3dwIq/Q7nIavKqGwTggE1glIj+JyMcishT4FTgK/LO2C1RKea7RKW04nJPHxyvT7C6lwavq9wwOAl2A57BaLe3k+P0M0MUxXimlakXXhHA6xzVhyuJt5Bdq5zd1qco9nRljMo0x44wx3Y0x7YwxlxhjnjDG6PfHlVK1bnRKIrszT/Dlmj12l9KgVavbSxFpKiJXi8idIhLuGBYgItqNplKqVl2a1Iyk5lbnN0VF2rx1XanqN5BFRCYA6VjfK5gKxDtG/w/rIrJSStUaLy+r85s/9mczf1NFd7armqrqnvyjwH3AeOBiTnV7CdadRFfXUl1KKVXi6g4tiW7SiEkLU7XzmzpS1TC4GxhvjHkW6w4iZ6lAYq1UpZRSToo7v/l1Vya/bNfOb+pCVcMgilNdXpaVB5TbiJ1SStXUzZ1jiAjy0+at60hVw2A3cF4F4zoC2pCIUoa1OowAACAASURBVKpOWJ3fJLBwcwbr9+jNi7WtqmHwCTBORHo4DTMi0g54APiw1ipTSqkybu8WR2N/H95YtM3uUhqcqobBE8AmYDGwxTHsE+B3rGsGz9daZUopVUZoI19u6xbL12v3sPOQdn5Tm6r6DeQTQAowDPgJ+B5YAYzCupNodO2Wp5RSpY3okYCPlxdvLtajg9pU1e8ZNAWKjDEzjTG3G2OuBEYATYE/gFfqoEallCrRLCSAGy+K5tOV6RzQzm9qzVnDQET8ReRVETkG7AcOichox7jbga3ABCAN6FeXxSqlFMA9vVtTUFTE1KU77C6lwajMkcE44C/AMqwP/e+AV0Xk/4B3sForHWSMudgY812dVaqUUg7xTYMYcH5L3l2+k6MntPOb2lCZMLgFmGSMudIY84gx5hbgXuDPWMHQwRij/RgopVzq3j6JZJ8s4N3lO+0upUGoTBjEAJ+XGfaZ4/crxpi82i1JKaXO7ryoUPq0i2Ta0u3k5mvnNzVVmTDwBY6VGVb8PKN2y1FKqcobnZLIwew8PtHOb2rMp5LTRYlIa6fn3k7DM50nNMbo/V5KKZe4OCGcC2LDeHPxNgZ3jcXHW1vRr67KbrlPsb5kVvyzyTH8izLDt5T7aqWUqgMiwpiUNqQfOcFXa/faXU69Vpkjg+F1XoVSSlVT3+RmtG3WmMkLtzKoUytE5OwvUqc5axgYY2a4ohCllKoOLy/h3j6JPPDJGhZsPsBlyc3tLqle0hNsSql679pOrYgKa8SkBdq8dXVpGCil6j1fby9G9kpg5c4jrNihnd9Uh4aBUqpBuKVLLOFBfkxeqEcH1aFhoJRqEBr5eTP8knjmbzrAxr1ZdpdT77g8DESkn4hsFpFUEXnkDNPdKCJGRDq7sj6lVP11R/d4gvy8eUO7xqwyl4aBiHgDE4H+QHtgsIi0L2e6YOCvwM+urE8pVb+FBvoy5OJYvlyzh12HjttdTr3i6iODrkCqMWabo02jD4FB5Uz3FPACoI2VK6Wq5O5erfHx8mLKEj06qApXh0EUVr8HxdIdw0qIyIVAjDHma1cWppRqGJqHBHDDhVF8vDKdjGMn7S6n3nCrC8gi4oXVW9oDlZh2lIisFJGVGRnaXp5S6pRRvVuTX1jEtKXb7S6l3nB1GOzGahK7WLRjWLFg4DxgoYjsALoBs8u7iGyMmWKM6WyM6RwZGVmHJSul6pvWkY0ZcF5LZi7bSVaudn5TGa4OgxVAWxFJEBE/4FZgdvFIY8xRY0xTY0y8MSYeWA5ca4xZ6eI6lVL13L19Ejl2soD3lu+yu5R6waVhYIwpAO4D5gEbgY+NMetFZLyIXOvKWpRSDdv50aH0atuU//6ond9UhsuvGRhj5hhj2hljEo0xzziGjTPGzC5n2hQ9KlBKVZfV+c1JPl2Vbncpbs+tLiArpVRt6t46go4xYUxZvI2CwiK7y3FrGgZKqQZLRBjdJ5Fdh48zZ90+u8txaxoGSqkG7cr2zUmMDGLywq0YY+wux21pGCilGrTizm827s1i4R/6naSKaBgopRq8QZ2iaBkaoM1bn4GGgVKqwfPz8WJkr9b8sv0wq3Zq5zfl0TBQSnmEW7vGEBboq0cHFdAwUEp5hEA/H4ZdEs/3Gw+wed8xu8txOxoGSimPcWf3eAK185tyaRgopTxGkyA/BneNZfaaPaQd1s5vnGkYKKU8yt29EvASeHvJNrtLcSsaBkopj9IytBHXXxDFhyvSOJitnd8U0zBQSnmcUb0TySssYvrSHXaX4jY0DJRSHqdNs8Zc1b4F7yzbwTHt/AbQMFBKeajRKYlk5Rbw/s/a+Q1oGCilPFTHmDB6tIngbe38BtAwUEp5sNF92pBx7CSf/7b77BM3cBoGSimP1aNNBB2iQ3lz0VYKizy7eWsNA6WUxyru/GbHoePMXbfX7nJspWGglPJoV57bgtZNtfMbDQOllEfzdnR+s35PFou3HLS7HNtoGCilPN6gC1rRIiSAyQtT7S7FNhoGSimP5+/jzd29Eli+7TC/7jpidzm20DBQSilgcNdYQht5buc3GgZKKQUE+ftw5yXxfLdhP1v2e17nNxoGSinlMOySeBr5evPGIs9r3lrDQCmlHMKD/Li1awz/W72b3Zkn7C7HpTQMlFLKyd29WgPw1mLPOjrQMFBKKSdRYY0Y1CmKD1fs4nBOnt3luIyGgVJKlTE6pTW5+UVMX7rd7lJcRsNAKaXKaNMsmCvbN2fGsp1knyywuxyX0DBQSqlyjE5J5OiJfD78xTM6v9EwUEqpclwQ24TurSN4a8k2ThY0/M5vNAyUUqoCo1MS2Z91ki88oPMbDQOllKpAr7ZNObdVCG8u2tbgO7/RMFBKqQqICGNS2rDtYA7z1u+zu5w6pWGglFJn0O+8FsRHBDb4zm80DJRS6gy8vYR7+iTy++6jLE09ZHc5dUbDQCmlzuKGC6NoFuzPpAbc+Y2GgVJKnUVx5zc/bT3E6rRMu8upExoGSilVCUMujiMkwIc3GmjnNxoGSilVCY0dnd/M27CP1APZdpdT6zQMlFKqkoZdEo+/jxdvLmp4RwcaBkopVUkRjf25pXMMX6zezZ4G1vmNhoFSSlXByN6tKTLw9pKG1by1hoFSSlVBdJNABnVsxQe/7OJIA+r8RsNAKaWq6N6URE7kFzJj2Q67S6k1GgZKKVVF7ZoHc/k5zZn+0w5yGkjnNxoGSilVDaNTEsk8ns+HK9LsLqVWaBgopVQ1XBTXhK4J4by9ZBt5BUV2l1NjGgZKKVVNY1IS2Xs0ly9W1//ObzQMlFKqmvq0i+ScliG8sWgrRfW88xsNA6WUqiYRYXRKItsycvh2Q/3u/EbDQCmlamDAeS2IawCd32gYKKVUDfh4ezGqd2vWpB9l2db62/mNhoFSStXQjRdGExnsz6R63Ly1y8NARPqJyGYRSRWRR8oZ/3cR2SAia0XkBxGJc3WNSilVFQG+3ozomcCPqQdZm14/O79xaRiIiDcwEegPtAcGi0j7MpP9BnQ2xnQAPgVedGWNSilVHbddHEtwgA9v1NPmrV19ZNAVSDXGbDPG5AEfAoOcJzDGLDDGHHc8XQ5Eu7hGpZSqsuAAX4Z2i2Puun1szah/nd+4OgyiAOfvbqc7hlVkBDC3vBEiMkpEVorIyoyMjFosUSmlqmd4jwT8vL2Ysmib3aVUmdteQBaR24HOwITyxhtjphhjOhtjOkdGRrq2OKWUKkdksD83d47hs9/S2Xc01+5yqsTVYbAbiHF6Hu0YVoqIXA78A7jWGHPSRbUppVSNjSrp/KZ+HR24OgxWAG1FJEFE/IBbgdnOE4jIBcCbWEFwwMX1KaVUjcSEB3JNh5a8/8suMo/Xn85vXBoGxpgC4D5gHrAR+NgYs15ExovItY7JJgCNgU9EZLWIzK5gdkop5ZbuTUnkeF4h7yzbaXcplebj6gUaY+YAc8oMG+f0+HJX16SUUrUpuUUIlyU3Y9rS7dzdK4FAP5d/1FaZ215AVkqp+mxMSiJHjufzUT3p/EbDQCml6kDn+HC6xDfhrcXbyC90/85vNAyUUqqOjE5JZM/RXP63eo/dpZyVhoFSStWRS5OakdwiuF50fqNhoJRSdaS485vUA9l8v3G/3eWckYaBUkrVoYHntyQmvBGT3LzzGw0DpZSqQ1bnN4msTstk+bbDdpdTIQ0DpZSqY3+6KJqmjf2Y7MbNW2sYKKVUHQvw9WZ4jwQW/5HBut1H7S6nXO7/tbgaKioqIj09nZycHLtLUcotBAUFER0djZeX7gu60tDucbyxcCuTF21l4pAL7S7nNA0+DA4ePIiIkJSUpH/8yuMVFRWxe/duDh48SLNmzewux6OEBPhyW7c4pizeyvaDOSQ0DbK7pFIa/KdjZmYmzZs31yBQCvDy8qJ58+YcPeqepyoaurt6xuPj7cWUxe537aDBf0IWFhbi6+trdxlKuQ1fX18KCgrsLsMjNQsO4E8XRTNr1W72Z7lX5zcNPgzA+uKHUsqi/w/2GtW7NQVFRUz9cbvdpZTiEWGglFLuIi4iiIEdWvHu8p0cPZ5vdzklNAw8VP/+/XnxxRfPOt2SJUsICwtzQUX22LFjByJCeno6ANOnT6dNmzY2V6UautF9EsnJK2Tm8h12l1JCw8BNpKSk4O/vT+PGjQkNDeWCCy5g1qxZdba8uXPnMnbs2LNO16tXLzIzM+usDoBhw4bh6+tL48aNCQkJ4ZxzzmHSpEl1usya2Lt3L6NHjyYuLo6goCBiY2O5+eabWbVqld2lqXqifasQUpIimbZ0ByfyCu0uB9AwcCv/+te/yM7O5tChQwwePJhbbrmFP/7447Tp8vPd59Cyttx5551kZ2eTmZnJ008/zX333cfChQvtLus0e/bsoUuXLqSlpTFnzhyysrLYsGED11xzDZ999lm15tkQ3091dqP7JHIoJ4+PV7pH5zcaBm7Ix8eHMWPGUFhYyO+//87ChQvx8fFh5syZtG7dmvDwcAB27drFTTfdRIsWLWjZsiWjRo3i2LFjJfPJyMhgxIgRxMbGEhISwoUXXsjmzZsB60jk6aefBuDkyZOMGjWKZs2aERISQtu2bfnkk08ASpZdrKCggPHjx9O6dWuaNGlC3759WbduXcn4YcOGMXToUEaOHElYWBhRUVG8+eablV53Ly8vbrzxRiIiIli5cmXJ8CVLltCzZ0/Cw8NJTEzk5ZdfLtXo19q1a+nXrx+RkZGEh4dz+eWnek8dPnw4MTExBAcH0759e95///1K11PWuHHjCAoK4vPPP+fcc8/F29ubxo0bM3ToUJ555hkAnnjiiVLLh9Lbu7z3c+LEiXTq1KnUa7Zv3463tzc7duwAzv5+q/qla0I4F8U1YYqbdH7T4L90VtaTX65nw54slyyrfasQHr/m3Cq/Li8vj4kTJ+Lr60vHjh1JT0+nsLCQOXPm8Ntvv+Hr60tubi6XXXYZQ4YMYebMmeTm5nLbbbfx17/+lalTp1JUVMS1115Lq1atWLFiBZGRkaxbt47g4ODTljdjxgxWrFjBxo0biYiIIC0trcIPmQkTJvDOO+8wZ84cEhISeO6557jiiivYvHkzISEhAHz66ad89NFHvPnmm3zxxRfccsst9OvXj7i4uLOue2FhIZ9++ikHDx4kKSkJgA0bNjBgwADeffddrr76arZs2UL//v2JjIzkjjvuYO/evfTp04exY8cya9YsfH19Wbx4cck8e/bsyUsvvURYWBiffPIJd9xxB506daJ9+/ZVfm/mzJnDXXfdVePblcu+nydPnuSBBx5g9erVJaEwffp0UlJSiI+PP+v7reofEWF0n0TufmclX63dw/UXRNtajx4ZuJFnnnmGsLAwoqOj+d///sesWbNKXcx84YUXCA0NJTAwkK+++gpjDOPHj6dRo0Y0adKEp556ivfee4/CwkJWrlzJypUrmTp1asmX7jp06ECrVq1OW66fnx/Z2dls2LCBgoICYmJiKvygnDZtGg8//DDJycn4+/szbtw4vL29+frrr0umueyyy7j22mvx8vLihhtuICwsjNWrV59x3WfOnElYWBgBAQEMHjyY8ePHc8011wAwadIk/vSnPzFo0CC8vb1JTk7mvvvu45133il5bZs2bXj00UcJCgrCz8+v1J75iBEjiIiIwNvbm1tvvZUOHTpU+xRURkYGUVFR1XptWc7vZ5MmTRg0aBDTpk0DwBjDjBkzuOuuuwDO+n6r+umy5Ga0a96YyQvt7/zG444MqrOn7ir/+Mc/+Oc//1nuOC8vL2JiYkqeb9++nV27dp12p4+IsG/fPnbs2EGzZs0IDQ0963Jvv/129u/fz/3338+WLVvo27cvL774Yrl31aSlpZGQkFCqrvj4eNLSTp33bNmyZanXBAUFlRxpnHvuuezcuROAxx57jMceewyAoUOH8vbbb3P8+HHGjh3LDz/8wKOPPoqPjw/bt29n/vz5pc7JFxUVlWyPHTt20K5du3LXraioiCeeeIKPPvqIffv2ISLk5OSQkZFx1u1SnsjISHbv3l2t1zor+36CdTpr6NChvPTSSyxevJjMzExuuOEG4Ozvd20FlHItLy+r85v7P1rD/E0HuLx9c/tqsW3JqkpEpNSXheLi4mjXrh2ZmZmlfnJzc4mKiiI+Pp4DBw6QlXX2U2I+Pj48/PDDrFy5kp07dxIYGFiyR1pWTExMyTlssD5sd+zYcdoHW0XWr19PdnY22dnZJUHgLDAwkFdeeYXdu3czceLEknW96667Sq1nVlYW69evByA+Pp4tW7aUu7wPPviAt99+m1mzZnHkyBEyMzPp2LFjtTsZGTBgAJ9++ukZL/oGBwef1jDinj2l+8At+34CXHHFFfj7+/Pll18yffp0br31Vho1agSc/f1W9dfVHVoRFdaISQtTbe38RsOgnrr66qvJy8vj2Wef5dixYxhj2L17N59//jkAnTt35sILL+Tuu+/mwIEDFBUVsXbt2tM+lADmz5/PqlWryM/Pp1GjRgQFBeHt7V3ucocNG8aLL77IH3/8QV5eHs888wwFBQUMHDiw1tbNz8+PcePG8fTTT3Ps2DHGjBnDhx9+yJdffkl+fj4FBQVs2LCBRYsWAdaRzebNm3nhhRc4fvw4eXl5fP/99wBkZWXh4+NDZGQkRUVFTJ06lTVr1lS7tieffJLs7GxuuukmNm7cSGFhITk5OXzwwQclR3UXXXQRv/76K6tWraKgoIDXX3+d7dvP/m1Tb29v7rjjDl577TU+++yzUoF8tvdb1V++3l6M6t2aX3dl8st2+zq/0TCopwIDA5k/fz4bNmwgOTmZ0NBQ+vbtW3Ju3svLiy+//JJGjRrRqVMnwsLCuOuuu8jOzj5tXvv372fo0KE0adKEli1bsnPnTqZMmVLuch966CEGDx7MlVdeSfPmzZk/fz7ffvttycXj2jJkyBDCw8N5+eWXOe+88/jqq6/4z3/+Q8uWLWnWrBnDhg0rOdXTqlUrFi5cyHfffUd0dDQtWrRgwoQJgHXL6sUXX0ybNm2Iiopiw4YN9OrVq9p1RUVFsWLFClq2bMmVV15Z8r2IL774ghtvvBGw7hz6+9//Tr9+/WjZsiX79++nR48elZr/8OHDWbRoEQkJCXTt2rVk+Nneb1W/3dw5hoggezu/EXfuk7OyOnfubJxvQ3S2ceNGzjnnHBdXpJR70/8L9/P6/C289O0fzPl/vWjfquY7VyKyyhjTubLT65GBUkq5gaHd4gny87bt6EDDQCml3EBooC+3d4vj67V72HnI9T0zahgopZSbuKtnAj5eXkxZvM3ly9YwUEopN9E8JIAbL4rik1XpHDjm2s5vNAyUUsqN3NM7kYLCIqb+uMOly9UwUEopNxLfNIj+57fkveU7ycp1XYu2GgZKKeVmRvdJ5NjJAmYu2+myZWoYVKSwEL76CgYOhLAw8Pa2fg8caA3XxsGUUnXkvKhQereLZNrS7eTmu+azRsOgPJs3Q4cOcM01MGcOHD0KRUXW7zlzrOEdOljTKaVUHRjdJ5GD2Xl8sirdJcvTMChr82bo0QM2bDjzdBs2WNPVo0B49tlnS5qFPpvGjRuzbNmyOq6oYiLCjz/+6LLlOXc+Ux2V6TvZx8fHLXtvU+6pW+twOsWEMWXxVgpc0PmNhoGzwkK44QY4dMh6HhAADz4IGzdCXp71+8EHwd/fGn/okDV9LZwyKu4DOTg4mNDQUFq3bs3QoUNrtV/dxx57jC+//LJS02ZnZ9O9e/daW3axXbt20bhx45IfHx8f/Pz8Sp6fe677NjHuzlauXEnXrl0JDAwkMTGRd9991+6SVA2JCGNSEkk7fIKvf99b58vTMHA2d+6pI4KAAJg/HyZMgORk8PW1fk+YAAsWnAqEDRvgm29qZfH/+te/OHbsGEePHmXBggXExcXRrVu3BtUyZWxsbEkT1tnZ2aSkpPDYY4+VPC9ulrqqPLkf4aNHj9K/f39uvPFGjhw5whtvvMG9995r65Gdqh2Xn9OcNs2szm/quh05DQNnkyefenzffVDRnnH37tb48l5XS+Li4nj66ae54447+Mtf/lLyh3D8+HEefPBBEhISCA8Pp1+/fqSmppa8Lj8/n2effZakpCSCg4NJTEzk008/BU7vm/e1114jISGB4OBgoqKiSvUvUPY0zaxZs+jYsSOhoaF07NixVEAVnyJ57bXXiI6OpkmTJtxzzz017oFr7dq1dOnSheDgYLp168amTZtKxqWkpPC3v/2N6667jpCQEF5++WUA3nrrLc477zxCQ0O54IIL+Pbbb0te89tvv9GzZ09CQ0MJDw/nkksu4ciRIyXjjxw5wo033liy3f73v/+Vqmfy5MkkJSURGhpKt27dWLJkSYW1Hzt2jDvvvJPw8HDi4uKYMWNGjbbFmXz22WcEBgYyduxY/P39ueKKK7j++usrbHlW1R9eXsK9fRLZtO8YCzYfqNuFGWPq/c9FF11kKrJhw4YKx50mNNQYsH42bjzztBs2nJo2LKzyy6hAnz59zFNPPXXa8G+//dYAZqOjniFDhpiBAweaffv2mZMnT5px48aZpKQkk5eXZ4wxZuzYseacc84xa9asMUVFRSYtLc2sWbPGGGPM448/bvr27WuMMWbz5s2mUaNGZt26dcYYY44cOWKWLVtWslzALFmyxBhjzNKlS42/v7+ZM2eOyc/PN1999ZXx9/c3y5cvN8YYM23aNOPj42Mee+wxk5uba7Zs2WKaNGli3n333bOud9++fc3jjz9+2nDAdOnSxezcudPk5uaam266yVx++eWltldwcLD54YcfTFFRkcnJyTFTpkwxiYmJZvXq1aawsNB8/fXXJigoyGzZssUYY0z37t3Nk08+aQoKCkxeXp5ZtmyZyc7OLplfRESEWbp0qSksLDSvvPKKCQ0NNTk5OcYYY95//30TERFhli9fbvLz883bb79tAgMDzY4dO0q2QWJiYkl9d911l+nevbvZu3evyczMNNdff70BzIIFC8rdDjt37jShoaFn/KnIX//6VzNo0KBSw1555RVzwQUXVPiaKv1fKFvlFRSa7s9+b26avLRKrwNWmip8juqRgTPnTuATE888rfP4SvQmVl3R0VYn2YcOHeLgwYO8//77TJo0iebNm+Pn58fjjz/O3r17+fnnnzHGMHHiRCZMmECHDh0QEaKjo+nQocNp8/Xx8cEYU9LzWFhYGN26dSu3hunTp3PjjTfSv39/fHx8GDhwINdff32pjtgbNWrE+PHj8ff3p02bNvTt25eKmhWvrIceeojY2Fj8/f0ZNmzYafO76aabuOyyyxARAgMDefXVVxk3bhwdO3bEy8uLAQMGcOmll/Lhhx8CVqc5u3btIi0tDV9fX7p160ZQUFDJ/G655RYuueQSvLy8GDVqFEePHi3pQW3atGncc889XHzxxfj4+DBixAg6dOjA+++/f1rdRUVFvPfeezz11FO0aNGC0NBQXnjhhTOua2xs7Gm9mJX9qcixY8dO6940LCysUr3cKffn6+3FyN6tWbHjCCt21F3nNxoGzoKDTz3eepZmZJ3H13LHLs7S063byiIiIkp6y+rQoQNhYWGEhYURHh5Ofn4+aWlpZGRkkJOTU2F/wM5at27Ne++9x1tvvUWrVq3o2bNnqVMqzsr2ewyQmJhYqt/jZs2aleodzbnf4+py7ku5vPnFx8eXer59+3b+/Oc/l2ybsLAwFixYUNJn8bRp0ygqKqJnz54kJCTwr3/9i4KCggqXB5QsszLboFhGRgYnT54sVV/Z19am4OBgjh49WmpYZmZmrXc4pOxzS5cYmgT6Mnlh3TVvrWHgzLk3qv/+98zTOo+vZC9W1fHRRx8RFRVFUlIScXFxAGzZsqXUHuPx48cZPHgwkZGRBAYGVtgfcFk33HAD3333HQcPHuTmm29m0KBBHD9+/LTpyvZ7DLBt27ZK93tcV7y8Sv/5xsXFMXXq1FLbJjs7m8mOazoJCQlMnTqV9PR0Zs+ezdtvv80777xTqWVVZRs0bdoUPz+/UtOXfW1ZZe+yKu+nIh07djytx7PffvuNjh07nnW9VP0Q6OfD8B4JzN90gI176+aIT8PA2ejRpx6//jpUdDfGsmXW+PJeV0vS0tJ4/PHHmT59Oq+++ioiQrNmzRgyZAhjxowp2dvNzMzk888/Jzs727oVbcwYxo4dy7p16zDGkJ6eztq1a0+b/+bNm/nmm284fvw4vr6+hIaGIiKnfcCC1XXkrFmzmDdvHoWFhcydO5fPPvuM4cOH1/p618T999/PE088werVqzHGcOLECX788ceSC88zZswo6QM6LCwMHx+fCvt6LmvYsGG8+eab/PLLLxQUFDBt2jRWr17NkCFDTpvW29ubIUOG8Pjjj7N//36ysrJ45JFHzjj/sndZlfdTkeuvv56cnBwmTJhAXl4eP/zwA5999hmjRo2q1Lqp+uGO7nEE+nnzZh11fqNh4Kx/f2jf3nqcmwuXXlr+9wwuvRROnrSma98e+vWrlcU/9dRTBAcHExISQu/evUlNTeWnn34q6VsXrLtlkpKSSElJITg4mPPPP59PPvkEEQHgmWee4eabb+a6664jODiYlJSUUncbFcvLy2P8+PG0bNmSsLAwXnvtNWbNmkVAQMBp0/bo0YMZM2bw4IMP0qRJE8aOHcu7775b4TUGu4wcOZKxY8cyfPhwmjRpQmxsLE899VTJbafz58/noosuIigoiO7duzNkyBCGDh1aqXkXf7jffvvtREREMHnyZObMmVNytFbWq6++SkJCAsnJyZx//vlcc801lQ6eqgoLC2POnDl88sknhIaGMnLkSN544406+Z6Isk9YoB9Dusby5dq9pB0+/Qi+prQP5LKKv4Fc/MWzM4mIgJ9+gkqco1fKnWgfyPXTvqO59HpxPrd2ieWp684747TaB3JNF/aboAAACY5JREFUJSXB0qWnjhAq0r69BoFSyqVahAZwwwXRfLwyjYxjJ2t13hoG5UlKgrVrS7da6uVVutXStWs1CJRSLjeqT2vyCouYtnR7rc7Xp1bn1pB4e1sf/AMH2l2JUkqVSIxsTP/zWjBz2U7uTUkkJMC3VuarRwZKKVXP3Ovo/Ob9n3fV2jw9IgwawkVypWqL/j/Ufx2iw+jZpin//bH2Or9p8GEQEBDAoUOH9B9AKawgOHToULm3EKv6ZUxKIhnHTjLr19rp/KbBXzOIjo4mPT2djIwMu0tRyi0EBASUtHml6q/uiRF0jA7lzUXbuKVzDD7eNdu3b/Bh4OvrW6ftwiillB1EhNEpidz77q/MWbePazu2qtH8GvxpIqWUaqiubN+C1pFBtdL5jcvDQET6ichmEUkVkdMabBERfxH5yDH+ZxGJd3WNSilVHxR3frNxbxaL/qjZqXCXhoGIeAMTgf5Ae2CwiJT9qu8I4Igxpg3wb+DMDcErpZQHu65TFC1DA5hUw+atXX1k0BVINcZsM8bkAR8Cg8pMMwgo7iPwU6CvFLfCppRSqhQ/Hy/u7tWaX7YfZtXO6nd+4+owiAKcewNJdwwrdxpjTAFwFIhwSXVKKVUP3dolhrAadn5Tb+8mEpFRQHGD7SdFZJ2d9biBpsBBu4uwkaevP+g28PT1Zw00/e+wkm1QfvvqFXB1GOwGnLuGinYMK2+adBHxAUKB09qTNsZMAaYAiMjKqjTV2hB5+jbw9PUH3Qaevv5Qs23g6tNEK4C2IpIgIn7ArcDsMtPMBu50PL4JmG/068NKKVWnXHpkYIwpEJH7gHmANzDVGLNeRMYDK40xs4H/AjNFJPX/t3f3oXJUZxzHv79iimgStaRBK5FriVoltjREMf2jVnxBg6TYoFQb5Fax1lBfag22WDWm+cfXgqKYFKyKRlQKIWLEaklMqaYoUeM1pPUtpGlSpNWktjG+JE//OGdxO7l7d25yd9ad/X1g2J27M7PPmbs7z845M+cA75EShpmZdVDlbQYRsQJYUfjbDU3PdwLnjnKzS8YgtF7X7/ug38sP3gf9Xn7Yh31Qi2Evzcxs37g7CjMz661k0O9dWZQo/9WS1ktaJ+kPkkZ1aVkvaLcPmpabIykk1e7qkjL7QNJ5+bPwuqSlVcfYSSW+B0dIWinp5fxdmNWNODtF0n2S3m11Ob2SO/P+WSdpeqkNR0RPTKQG57eArwJfBF4FjissMw+4Nz//PvBot+OuuPynAAfk55fVqfxl90FebgKwGlgDzOh23F34HBwFvAwckucndzvuisu/BLgsPz8O2NjtuMd4H3wbmA4MtXh9FvAUIOAk4M9ltttLZwb93pVF2/JHxMqI2JFn15Du46iTMp8BgF+R+rTaWWVwFSmzDy4B7o6I9wEi4t2KY+ykMuUPYGJ+fhCwpcL4Oi4iVpOutGzlu8CDkawBDpZ0WLvt9lIy6PeuLMqUv9nFpF8HddJ2H+RT4ikR8WSVgVWozOfgaOBoSX+StEbSmZVF13llyr8AmCtpM+nKxcurCe1zY7THCqCHu6Ow1iTNBWYAJ3c7lipJ+gJwBzDY5VC6bT9SVdF3SGeHqyUdHxHbuhpVdc4H7o+I2yXNJN23NC0idnc7sM+zXjozGE1XFozUlUWPKlN+JJ0GXAfMjoiPKoqtKu32wQRgGrBK0kZSfenymjUil/kcbAaWR8QnEfEO8FdScqiDMuW/GHgMICJeAPYn9VvUL0odK4p6KRn0e1cWbcsv6ZvAYlIiqFM9ccOI+yAitkfEpIgYiIgBUrvJ7Ih4qTvhdkSZ78Ey0lkBkiaRqo3erjLIDipT/k3AqQCSjiUlg34aBH05cGG+qugkYHtEbG23Us9UE0Wfd2VRsvy3AuOBx3O7+aaImN21oMdYyX1QayX3wdPAGZLWA7uA+RFRizPkkuX/GfAbST8lNSYP1uhHIZIeISX7Sbld5EZgHEBE3EtqJ5kFvAnsAH5Yars12kdmZraXeqmayMzMOsTJwMzMnAzMzMzJwMzMcDIwMzOcDKwmJA3mXkob0weSXpX0k3wDYhUxDOT3HhzFOo24BzoWmFkJPXOfgVlJ55LuwJ2Yn98FTAZuGGmlMbIVmEnqVbOsJ/M6bW8KMusk32dgtZB/jf8WOCoi3mz6+0pgekQcNMw644BP63RDktnecjWR1d2LwERJJ+bqmHmSbpG0BfgIOBhA0vdyD587JG2T9LikI4obk3SJpLWSPpT0vqTnJH0rv7ZHNZGkEyQ9I+lfeZ23Jd3T9Poe1USSxklaJGmjpI/z46KcvCi816WSFkramuN+QlLdui63CjgZWN0dSeqS4T95/jpSXz0/As4Bdkr6MfA7YD2pT6tLSR3ePSdpQmNDkm4jDZyyFjgPmEsaRGePpJGXH0/qNmEXqSfVs4CFtK+efQD4OfAgcDZwP3Atn43V0ewXwFTgIuBKUpXTQ222b7anbo/a48nTWEykg20Ax5AOtoeQDuq7SB23DeTX15KrR/N640njXtxX2N6RwMfAVXl+at7WHSPE0HiPwTw/I89/vUTcA3l+Wp5fUFjul83banqvVYXlrsl//0q3/yeeemvymYHVzQbgE1JHhfcAD5N+NTcsi4jmNoKZpMbmhyXt15hIg4NsIA0xCHAa6Ux6yShieQPYBiyWNFfSlHYrNL1f8dd9Y744RsWKwvxr+XHYsxWzVpwMrG7OAU4AvgYcGBEXRkTzEIHFq3Ym58dnSUmkeTqez0bKazxuLhtIRGwnjUu9hZSYNkkakjRnhNW+1CLOfxRebygOf9gYw2L/snGagS8ttfoZiqariYZRvHKo0bXzIPD6MMt/kB//mR8PB/5SNpiIeAWYk882ZpDq+B+T9I2IGBpmlcbB/VD+/xLVQwuvm40pnxlYv3uedMCfGhEvDTM1DvzPArtJDc+jFhGfRhqc/HrS9+7YFouuzo/FsTh+kB9X7c37m7XjMwPraxHxb0nzgbslfRl4itSgfDipfn5VRCyNiLck/Rq4Ol9htJzUoHwisCEiHi1uW9LZpOSxDHgHOBC4gpR8XmgRz1AevGRBPpt4ntSucT3wSES8Ntx6ZvvKycD6XkQslvQ3YD5wAel78Xfgj8ArTctdk0fRm0caXvW/wDrg9y02/QbwIelAfhgpCbwInB4RI7U9DJKGqbyIdBXRFuBm4Ka9K6FZe74D2czM3GZgZmZOBmZmhpOBmZnhZGBmZjgZmJkZTgZmZoaTgZmZ4WRgZmY4GZiZGfA/0Uu5unjAnvsAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hIehPmQ5rUwZ"
      },
      "source": [
        "#### Evaluate Logistic Regression w/ Word2Vec Representation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sVRXWCW7rUwa",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6cf8bc41-cecd-4419-9848-b1a546407ede"
      },
      "source": [
        "grid_values = {'C':[.56, .565, .57]} #L2 found to be best\n",
        "# default metric to optimize over grid parameters: log loss\n",
        "grid_lr_clf_w2vec = GridSearchCV(lr_clf, param_grid = grid_values, scoring='neg_log_loss', n_jobs=-1, verbose=1, cv=3)\n",
        "#Fit to W2Vec training data\n",
        "grid_lr_clf_w2vec.fit(full_X_train_w2vec, y_train)\n",
        "grid_lr_clf_w2vec.cv_results_"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 3 candidates, totalling 9 fits\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.\n",
            "[Parallel(n_jobs=-1)]: Done   9 out of   9 | elapsed:  1.1min finished\n",
            "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.\n",
            "[Parallel(n_jobs=-1)]: Done   1 out of   1 | elapsed:  3.3min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'mean_fit_time': array([26.55104502, 24.46938086, 22.18910313]),\n",
              " 'mean_score_time': array([0.13150366, 0.13270156, 0.09965722]),\n",
              " 'mean_test_score': array([-0.61258703, -0.61391193, -0.61327135]),\n",
              " 'param_C': masked_array(data=[0.56, 0.565, 0.57],\n",
              "              mask=[False, False, False],\n",
              "        fill_value='?',\n",
              "             dtype=object),\n",
              " 'params': [{'C': 0.56}, {'C': 0.565}, {'C': 0.57}],\n",
              " 'rank_test_score': array([1, 3, 2], dtype=int32),\n",
              " 'split0_test_score': array([-0.61054578, -0.61246796, -0.61023968]),\n",
              " 'split1_test_score': array([-0.61553925, -0.61648875, -0.61667385]),\n",
              " 'split2_test_score': array([-0.61167605, -0.61277909, -0.61290052]),\n",
              " 'std_fit_time': array([0.47653499, 0.70024713, 4.40945707]),\n",
              " 'std_score_time': array([0.01684095, 0.00341324, 0.0251129 ]),\n",
              " 'std_test_score': array([0.00213793, 0.00182651, 0.00263979])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FdePfy9drUwa",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e43fe1bd-1a99-4b51-cd79-ad185997e9a7"
      },
      "source": [
        "grid_lr_clf_w2vec.best_params_"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'C': 0.56}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VGpV-Ud3rUwa"
      },
      "source": [
        "grid_lr_clf_w2vec_preds = grid_lr_clf_w2vec.predict(full_X_test_w2vec)"
      ],
      "execution_count": 42,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4RCP5qH8rUwa",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3e881036-f4dc-4f51-b9e2-8da75cc29b8d"
      },
      "source": [
        "print(\"Logistic Regression (Word2Vec) Accuracy: \" + str(accuracy_score(grid_lr_clf_w2vec_preds, y_test)))\n",
        "print(\"Logistic Regression (Word2Vec) Precision: \" + str(precision_score(grid_lr_clf_w2vec_preds, y_test)))\n",
        "print(\"Logistic Regression (Word2Vec) Recall: \" + str(recall_score(grid_lr_clf_w2vec_preds, y_test)))"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Logistic Regression (Word2Vec) Accuracy: 0.6682702689732947\n",
            "Logistic Regression (Word2Vec) Precision: 0.6647071526261488\n",
            "Logistic Regression (Word2Vec) Recall: 0.669478008699855\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lLrmqQ8jrUwb"
      },
      "source": [
        "#### Logistic Regression w/ Word2Vec - Confusion Matrix"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QjJgVX1CrUwb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "outputId": "9d2c4726-54ed-4073-cb06-928d20b4be18"
      },
      "source": [
        "cm = confusion_matrix(y_test, grid_lr_clf_w2vec_preds)\n",
        "cm_display = ConfusionMatrixDisplay(cm, display_labels=[0, 1]).plot()"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEKCAYAAACFekfkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZyWdb3/8debARFlX0XAIMN91wS3XFIW6xfWMbVFyTyaa2p2Su088mTZpmlSLseTHLU6IYYlGYq4a8kuioAK7iyKwy4gMMPn98f1HbgHZu65b5iJmXveTx/Xw3s+1/e6ru896Ifvcl3fSxGBmZllWuzoCpiZNSZOimZmOZwUzcxyOCmameVwUjQzy+GkaGaWw0nRzHYISX0kPSVptqRZki5P8UMkTZQ0Q9JUSUemuCSNkDRP0suSDss513BJc9M2PCd+uKSZ6ZgRklRnxSLCmzdv3v7lG9ATOCx9bge8DuwHPAYMTfFTgadzPj8CCBgITErxzsCb6d+d0udOad/kVFbp2KF11cstRTPbISJiUURMT59XAXOAXkAA7VOxDsDC9HkYcF9kJgIdJfUEBgMTImJpRCwDJgBD0r72ETExsgx5H3BaXfVqWX9fcft17VwWffu02tHVsCK8/vIuO7oKVoSPWc36WFd3FzKPwSfuGkuWVhZUdtrL62YBH+eE7oqIu7YsJ6kvcCgwCbgCGC/pJrIhvqNTsV7AezmHzU+xfPH5NcTzalRJsW+fVkwe32dHV8OKMHj3Q3Z0FawIk+KJ7T7HkqWVTB6/R0Fly3rO/TgijshXRlJbYAxwRUSslPQT4MqIGCPpDOBu4OTtrXeh3H02s6IEsLHAf+oiqRVZQvxjRDyYwsOBqs8PAEemzwuA3FZT7xTLF+9dQzwvJ0UzK0oQbIjKgrZ80kzw3cCciLg5Z9dC4Pj0+SRgbvo8FjgnzUIPBFZExCJgPDBIUidJnYBBwPi0b6Wkgela5wAP1fX9GlX32cyahkJagQU4BjgbmClpRopdC5wP3CqpJdl45AVp3ziyGeh5wBrgXICIWCrpx8CUVO76iFiaPl8M3AO0IZt9fqSuSjkpmllRgqAytn/JwYh4nuxWmZocXkP5AC6p5VwjgZE1xKcCBxRTLydFMyvaRkp3HVYnRTMrSgCVTopmZpu5pWhmlgSwoR7GFBsrJ0UzK0oQ7j6bmW0SUFm6OdFJ0cyKkz3RUrqcFM2sSKKy1tsLmz4nRTMrSjbR4qRoZgZU3afopGhmtslGtxTNzDJuKZqZ5QhEZQmvOuikaGZFc/fZzCwJxPoo29HVaDBOimZWlOzmbXefzcw28USLmVkSISrDLUUzs002uqVoZpbJJlpKN3WU7jczswbhiRYzsy1UlvB9iqWb7s2sQVQ90VLIlo+kPpKekjRb0ixJl+fsu0zSqyn+y5z4NZLmSXpN0uCc+JAUmyfp6px4P0mTUvx+STvV9f3cUjSzom2sn9nnCuCqiJguqR0wTdIEoAcwDDg4ItZJ6g4gaT/gLGB/YHfgcUl7pXPdBpwCzAemSBobEbOBXwC3RMQoSXcC5wF35KuUW4pmVpRsQYjtbylGxKKImJ4+rwLmAL2Ai4CfR8S6tG9xOmQYMCoi1kXEW8A84Mi0zYuINyNiPTAKGCZJwEnAn9Px9wKn1fX9nBTNrCiB2BBlBW1AV0lTc7YLajqnpL7AocAkYC/guNTtfUbSp1OxXsB7OYfNT7Ha4l2A5RFRsUU8L3efzawoERRz83Z5RByRr4CktsAY4IqIWCmpJdAZGAh8Ghgt6ZPbU+diOCmaWZFUbzdvS2pFlhD/GBEPpvB84MGICGCypI1AV2AB0Cfn8N4pRi3xJUBHSS1TazG3fK3cfTazogRZS7GQLZ805nc3MCcibs7Z9VfgxFRmL2AnoBwYC5wlqbWkfkB/YDIwBeifZpp3IpuMGZuS6lPA6em8w4GH6vp+bimaWdHqaZHZY4CzgZmSZqTYtcBIYKSkV4D1wPCU4GZJGg3MJpu5viQiKgEkXQqMB8qAkRExK53v+8AoST8BXiRLwnk5KZpZUQLVyyKzEfE81NoP/3otx9wA3FBDfBwwrob4m2Sz0wVzUjSzomSvOC3d1FG638zMGoi8nqKZWZWg3p5oaZScFM2saG4pmpklEXJL0cysSjbR4rf5mZklfkeLmdkm2USLxxTNzDappydaGiUnRTMrSn090dJYOSmaWdH84iozsyQCNmx0UjQzA6q6z06KZmablPITLaWb7rfR4gWt+I/T9+T84/fh/BP25i+/67pVmdUrW/DDc/px4cl7c/4JezN+VOftvu7KZWVcfeaenHvMvlx95p6sWl795tjXZrRhaJ+Dee7hDtt9rVLznZvf5f6XZ/HfT76Wt9xeB69h3Lsvceznlm/3Ndt1rOBno95g5PNz+NmoN2jboaLa/vq8VmNTdUtOIVtT1KBJsbZ3sTZmZS2DC364kP955lVufXguf7unK++83rpambH3dGWPvT7mzsdf48Yx87jr+t3ZsL6w/wBe+mdbbrpij63io3/bnUOPXcX//mMOhx67ivt/233TvspKuPuG3Tn8+FXb9+VK1GP3d+YHX+uXt0yLFsF5P1jEtGfaFXXug476iKtueXer+BmXLubF59vyzWP35cXn23LmpYs37dvWazUdWfe5kK0parBaSyojexfrUGA/4Cvpva2NWpceFfQ/aC0Au7TdSJ9PraN8UatqZSRYu7qMCPh4dRntOlZS1jIAeOD2blw2dC8u/Oze3HfjbgVf94XxHTj5jKUAnHzGUl54dHOL8KGR3Tj21BV07FpR2+HN2iuT2rJqWf6RoGHfLOf5cR1YXl693OkXLWbEuNe54/HXOPu77xd8zaMGr+Tx0VkP4fHRnTlqyMo6r1VKNqb3tNS1NUUNmcprfBdrA16v3r3/3k688Uob9jlsTbX4F84t5925rfnqofvzrZP25qLrF9CiBUx7uh0L3mrNiHGvc/uE15g7sw0zJ+5a0LWWlbeiS48s6XXuXsGy8iwRly9qxT8f6cDnh5fX75drRrrstoGjh67g4Xu7VIsfdvwqevVbx7dP7c/Fp+xF/wPXcMCAjwo6Z6euG1i6OPszWrq4JZ26bsh7rVKSzT6XFbQ1RQ35V1lN72IdsGWh9B7YCwD26NV4/mZdu7oFP/73vlx4/QJ2bbex2r5pT7djz/3X8ssH3mDh2ztxzVl7csCAj5j2TDumP9Oei0/ZOzvHmhYseLM1Bw5czbc/158N61qwdk0LVi0v46KTszLn/edCjjiherdYAilred55XS/O+8FCWjTNnkijcOGPFnD3DT2JLca4Dj9+FYcdv4rbJ7wOQJtdNtLrk+t4ZVJbbn14Lq1ab6TNLhtp17GS2ydk45V3/6Qn055pv8UVtOnctV2rlPjm7QYWEXcBdwEccfDOsYOrA0DFBvjxv/flpC8t49hTV2y1/7H7O3PGpYuRoFe/9ey2x3rem7czAZx52Qd87uwlWx0z4u9zgWxMccLoznz319XHqTp13cCSD1rSpUcFSz5oSccuWavx9Zfa8LOL+gKwYmkZk59oR1kZHD1063pZzfY6eC3X3PEOAB06V3LkZ1dRWSkE3P+bHoz7w9atuss/3x/IxhRPOWMpv7qy+jjwsvJWdO6etRY7d9/A8iUt814rdzikFDTVrnEhGjIp5ntHa6MVATdftQd9+q/j3771YY1luvXawIzn2nHggNUs+7Al899oTc891nHE8au498bdOOlLy2iz60bKF7WiZasoaCxw4KBsjOrMyxZnY1SDs6R336Q5m8rcdMUeDDh5hRNikYYP3HfT56tueZdJj7fnhUc7sG5tC4b/x/s8+WBHPl5TRpfdNlCxAVYsaZXnbJmJj7Xn5DOWMvq3PbIx4PHt816rlHhBiG236V2sZMnwLOCrDXi9ejFr8q488efO9Nt37aYu7rnXLGTxgp0A+Pw5S/jaFe9z0xV78K2T9iYCzvvBIjp0qeTwE1bx7rzWXPH/slZGm1038r3fvEPHre/q2cqZl37ADRf25dFRXejeaz0/+O+3G+orlpyrb3+Hg476iA6dK/jD1Nn8/lc9aJkmvv7++9p/+dOfaccen/qYX/9tHpANmfzysj1YsXVDfyv3/7Y7P7jzHYactZTFC3bihm99ol6+S1NRHzPLkvoA9wE9yHLtXRFxa87+q4CbgG4RUZ7eE30rcCqwBvhGRExPZYcD/5kO/UlE3JvihwP3AG3I3vZ3eXpdau31qmP/dpF0KvBrNr+LdatXE+Y64uCdY/L4PvmKWCMzePdDdnQVrAiT4glWxtLtauZ12qd7nDTy9LoLAg8ec8e0iDiipn2SegI9I2K6pHbANOC0iJidEubvgH2Aw1NSPBW4jCwpDgBujYgBkjoDU4EjyJLrtHTMMkmTgW8Dk8iS4oiIeCRfnRt0+D4ixkXEXhGxZ10J0cyajvq4eTsiFlW19CJiFTCHbIIW4Bbge2RJrsow4L7ITAQ6psQ6GJgQEUsjYhkwARiS9rWPiImpdXgfcFpd322HT7SYWdNS5JhiV0lTc36+K02uViOpL3AoMEnSMGBBRLyU9Zg3qemOll51xOfXEM/LSdHMilZEUiyvrftcRVJbYAxwBVABXAsM2q4Kbgff/WZmRam6T7E+nn2W1IosIf4xIh4E9gT6AS9JepvsrpXpknaj9jta8sV71xDPy0nRzIpWH4/5pdnku4E5EXEzQETMjIjuEdE3IvqSdXkPi4j3gbHAOcoMBFZExCJgPDBIUidJnchamePTvpWSBqZrnQM8VNd3c/fZzIoSARX1s8jsMcDZwExJM1Ls2ogYV0v5cWQzz/PIbsk5N6tPLJX0Y7LbAAGuj4il6fPFbL4l55G05eWkaGZFq4+btyPiecjfnEytxarPAVxSS7mRwMga4lOBA4qpl5OimRXFzz6bmW2hlBe8cFI0s6J5QQgzsyTCC0KYmeUQlX7FqZnZZh5TNDNLvJ6imVmuyMYVS5WTopkVzbPPZmZJeKLFzKw6d5/NzHJ49tnMLIlwUjQzq8a35JiZ5fCYoplZEoiNnn02M9ushBuKTopmViRPtJiZbaGEm4pOimZWtGbZUpT0G/L8fRAR326QGplZoxbAxo2lmxTzTSFNBabl2cysOQogVNiWh6Q+kp6SNFvSLEmXp/iNkl6V9LKkv0jqmHPMNZLmSXpN0uCc+JAUmyfp6px4P0mTUvx+STvV9fVqbSlGxL1bfIFdImJNXSc0s9JXT/cpVgBXRcR0Se2AaZImABOAayKiQtIvgGuA70vaDzgL2B/YHXhc0l7pXLcBpwDzgSmSxkbEbOAXwC0RMUrSncB5wB35KlXnzUaSjpI0G3g1/XywpNuL/vpmVjqiwC3fKSIWRcT09HkVMAfoFRGPRURFKjYR6J0+DwNGRcS6iHgLmAccmbZ5EfFmRKwHRgHDJAk4CfhzOv5e4LS6vlohd2D+GhgMLEmVfwn4TAHHmVlJEhGFbUBXSVNztgtqPKPUFzgUmLTFrm8Cj6TPvYD3cvbNT7Ha4l2A5TkJtiqeV0GzzxHxXpZ0N6ks5DgzK1GFd5/LI+KIfAUktQXGAFdExMqc+A/Iuth/3MZabpNCkuJ7ko4GQlIr4HKyZq6ZNUcBUU+zzymnjAH+GBEP5sS/AXwe+GzEphHMBUCfnMN7pxi1xJcAHSW1TK3F3PK1KqT7fCFwCVmzcyFwSPrZzJotFbjlOUPW/bwbmBMRN+fEhwDfA76wxeTuWOAsSa0l9QP6A5OBKUD/NNO8E9lkzNiUTJ8CTk/HDwcequub1dlSjIhy4Gt1lTOzZqR+Zp+PAc4GZkqakWLXAiOA1sCENGw3MSIujIhZkkYDs8m61ZdERCWApEuB8UAZMDIiZqXzfR8YJeknwItkSTivOpOipE8CtwIDyX4VLwBXRsSbBX1tMys99ZAUI+J5am5OjstzzA3ADTXEx9V0XMpTRxZTr0K6z/8HjAZ6kt0b9ADwp2IuYmYlpJ5u3m6sCkmKu0TE7yOiIm1/AHZu6IqZWeMVUdjWFOV79rlz+vhIemxmFNnfEWeSp3lrZs1ACT/7nG9McRpZEqz69t/K2Rdkj96YWTOkJtoKLES+Z5/7/SsrYmZNRAGP8DVlBT3RIukAYD9yxhIj4r6GqpSZNWZNdxKlEIXcknMdcAJZUhwHDAWeB5wUzZqrEm4pFjL7fDrwWeD9iDgXOBjo0KC1MrPGbWOBWxNUSPd5bURslFQhqT2wmOrPGZpZc1J1n2KJKiQpTk0r3/4P2Yz0R2RPtZhZM9UsZ5+rRMTF6eOdkh4F2kfEyw1bLTNr1JpjUpR0WL59VSvmmpmVknwtxV/l2Rdky3zXq9dn7sqQfgPq+7TWgMbMf2ZHV8GKcPzQj+rlPM2y+xwRJ/4rK2JmTUTQbB/zMzOrWXNsKZqZ1aZZdp/NzGpVwkmxkPc+S9LXJf0w/byHpKJWsjWzElMP731urAp5zO924CjgK+nnVcBtDVYjM2vUFIVvTVEh3ecBEXGYpBcBImJZemOWmTVXzXz2eYOkMlJjWFI3muyj3mZWH5pqK7AQhXSfRwB/AbpLuoFs2bCfNmitzKxxa85jihHxR7IXU/8MWAScFhEPNHTFzKyRqqcxRUl9JD0labakWZIuT/HOkiZImpv+3SnFJWmEpHmSXs59FFnS8FR+rqThOfHDJc1Mx4xQepF0PoXMPu8BrAH+BowFVqeYmTVX9dNSrACuioj9yN4rf4mk/YCrgScioj/wRPoZsgWu+6ftAuAO2PSSveuAAWTveL6uKpGmMufnHDekrkoVMqb4dza/wGpnoB/wGrB/AceaWQlSPcwqRMQist4nEbFK0hygFzCMbLV/gHuBp4Hvp/h9ERHAREkdJfVMZSdExFIASROAIZKeJlvVa2KK3wecBjySr16FLB12YO7Pqcl6cS3FzcxydZU0NefnuyLiri0LSeoLHApMAnqkhAnwPtAjfe4FvJdz2PwUyxefX0M8r6KfaImI6ZK8lI1Zc1b4JEp5RByRr4CktsAY4IqIWJk77BcRIf1r57oLeXHVd3J+bAEcBixssBqZWeNWjzdmS2pFlhD/GBEPpvAHknpGxKLUPV6c4guo/iqU3im2gM3d7ar40yneu4byeRVyS067nK012RjjsAKOM7NSVQ8TLWkm+G5gTkTcnLNrLFA1gzwceCgnfk6ahR4IrEjd7PHAIEmd0gTLIGB82rdS0sB0rXNyzlWrvC3FdNN2u4j4bl0nMrNmpH5aiscAZwMzJc1IsWuBnwOjJZ0HvAOckfaNA04F5pHdEXMuQEQslfRjYEoqd33VpAvZ/Mc9QBuyCZa8kyyQ/3UELSOiQtIxhX5DMyt9ot5mn59Pp6vJZ2soH8AltZxrJDCyhvhU4IBi6pWvpTiZbPxwhqSxwAPA6pyLPVjbgWZWwprwYg+FKGT2eWdgCdk7WaruVwzASdGsuWqmSbF7mnl+hc3JsEoJ/0rMrE4lnAHyJcUyoC019/lL+FdiZnVprt3nRRFx/b+sJmbWdDTTpFi6q0ia2baL+pl9bqzyJcWtpsTNzIDm2VLMufnRzKya5jqmaGZWMydFM7OkCb9qoBBOimZWFOHus5lZNU6KZma5nBTNzHI4KZqZJV4lx8xsC06KZmabNdfH/MzMauTus5lZFd+8bWa2BSdFM7NMqT/RUsh7n83MqtHGKGir8zzSSEmLJb2SEztE0kRJMyRNlXRkikvSCEnzJL0s6bCcY4ZLmpu24TnxwyXNTMeMSO9/zstJ0cyKU9uL72va6nYPMGSL2C+BH0XEIcAP088AQ4H+absAuANAUmfgOmAAcCRwnaRO6Zg7gPNzjtvyWltxUjSzoikK2+oSEc8CW67dGkD79LkDsDB9HgbcF5mJQEdJPYHBwISIWBoRy4AJwJC0r31ETEzvjL4POK2uOnlM0cyKV/iYYldJU3N+visi7qrjmCuA8ZJuImu4HZ3ivYD3csrNT7F88fk1xPNyUjSzohUx0VIeEUcUefqLgCsjYoykM4C7gZOLPMc2c/fZzIpXf2OKNRkOPJg+P0A2TgiwAOiTU653iuWL964hnpeTopkVJ73Nr5BtGy0Ejk+fTwLmps9jgXPSLPRAYEVELALGA4MkdUoTLIOA8WnfSkkD06zzOcBDdV3c3WczK0p93qco6U/ACWRjj/PJZpHPB26V1BL4mGymGWAccCowD1gDnAvZS/Yk/RiYkspdn/PivYvJZrjbAI+kLS8nRTMrXtRPVoyIr9Sy6/AaygZwSS3nGQmMrCE+FTigmDo5KZpZ0fxESzNz5S/eZNSU6dz56My85fY66CP+Pncyxw7d/ldkt+1QwU9//yp3P/kSP/39q7RtX9Fg1yo15Qt34odf3pfLTzyIy086iId/t9tWZVavLOOn39ib75xyIJefdBBP3t9tu6+7alkZP/rKPlxy7MH86Cv78NHysmr7583YlS9/YgAvPNx5u6/VqNTvzduNToMlxZoe32kqJozpyn9+Y++8ZVq0CL75/feY9lyHos590ICVXHXjm1vFz7xoITP+0Z7zTjqYGf9ozxkXLdruazUXZWXBN374Drc+9TI/H/sKj97bg/deb1OtzKP39qBP/7XcPGEm1z8wm3uv/wQb1tf5xBcAr/yzPb+5cs+t4n+5rRcHHrOS255/iQOPWclfbtt8C1xlJfz+p3tw8GeWb9+Xa6QaeKJlh2rIluI9FPBITWP0yuT2rFqef2ThC8M/4B+PdmbFklbV4qdfsIgRf53FHY/M5OtXzK/l6K0ddcpyHh/TFYDHx3Tl6EHL6ryWZTr12MAnD1wDQJu2G+ndfy1L39+pWhkJ1q5uQQR8vLqMth0rKGuZNWX+ekdPvve5A7jy5AMZdVPvrc5fmymPdeLEL38IwIlf/pDJ4ztt2vfI/+7GwFOX0qFrRW2HN2lOituglsd3SkKXHus5evAyHv5D92rxw45bwe59P+bbp+3HxaceQP8DVnPAkSsLOmfHrhtY+mH2P/LSD1vRseuGvNeymi1+rzVvvbIr/Q/9qFp86DfeZ/7cNvz74YfxnZMP4pvXv02LFjDjmQ4semtnfvHwK/zqsZm8OXNXZk1sV9C1lpe3olOP7M+pY/cNLC/P/tJasqgVkx7pzOBzPqjfL9dYBNlESyFbE7TDJ1okXUCact+ZXXZwbQpz4Q/fYeTP+xBRvft12HErOPy4Fdz291kAtNmlkl591/HKZPj1X2bRaqegzS6VtOtYwW1/z0YVRv6iN9Oe7bjFFbTpv6farmVbW7u6BTde0J9z/+ttdmlXWW3fjKc70m//Nfxo9Bzef7s11391X/Y9ciYvPduBl57tyHcHHwhkrchFb+3M/gNXcfXnD2DDevHx6jI+Wt6SqwZlZb5+7bscesKKaueXsg3gf/+rL2df+y4tSnjEvpQnWnZ4UkzPQd4F0L5Flybxq+5/4Gqu+c08ANp3quDTJyynskJIcP/tuzPuT1u36q744v5ANqZ4yunl/Oo/Pllt//LyVnTutp6lH+5E527rN3WVa7vWCxM6YZtVbBA3XrAXx32xnIGnLttq/5Oju/HFSxYiQc9+6+jeZx0L5u1MBHzp0gUM+vrirY75+cPZX1yv/LM9Tz3QjctueaPa/o5dN7Dsg6y1uOyDVnTokrUa33h5V26+pD8Aq5a2ZPqTHWnRMhgwZOt6NVlN4v/UbbPDk2JT9I3PHLLp81U3vsmkJzvywoROrPu4Bed8Zz5PPtSFj9eU0aXHeioqVNBY4MTHO3Lyv5Uz+s7dOfnfynlhQse817LNIuD2736S3p9ayxcueL/GMl17rWfm8x3Yb8Aqln/YioVvtKHHJ9ZxyPErGHVjH477Yjltdt3IkkWtaNkqChoLPOKUZTz1QDe+dOlCnnqgG59O48B3vDBjU5nfXLknR3x2WUklxFJfZNZJsQZX3zqPgwauon2nCn7/zxf5w697bxqUH/d/tY/tTX+uA332XMstY2YD8PGaFvzyyj0LSor339GTa3/7BoPP+JDFC1pzw6Wfqp8v0wy8OqUdz4zpxh77rN7Uxf3q99+jfGE2Rjv47MV8+fL5/PY7e3LlZw8iyLrA7TtXcMjxK5g/tw3XfiG7v3fnXSu5fMS8gpLily5dyK8u7M8To7rRrfd6rrrj9Qb7jo1KFLaAbFOlaKDB0NzHd4APgOsi4u58x7Rv0SUGth7aIPWxhvHnN57Z0VWwIhw/9ANefKnAe5Fq0a5j7zj0M5cXVPa5v31v2jaskrNDNVhLMc/jO2bWxLn7bGZWJYAS7j47KZpZ8Uo3Jzopmlnx3H02M8tRyrPPTopmVpwmvAJOIZwUzawo2c3bpZsVnRTNrHhNdAWcQjgpmlnR3FI0M6viMUUzs1yl/exzCa/4ZmYNpp4Wma3ttSWSLpP0qqRZkn6ZE79G0jxJr0kanBMfkmLzJF2dE+8naVKK3y+p+pLsNXBSNLPiRL2+juAetnhtiaQTgWHAwRGxP3BTiu8HnAXsn465XVKZpDLgNmAosB/wlVQW4BfALRHxKWAZcF5dFXJSNLPi1VNLsZbXllwE/Dwi1qUyVSsADwNGRcS6iHgLmAccmbZ5EfFmRKwHRgHDJAk4CfhzOv5e4LS66uSkaGbFa9hXnO4FHJe6vc9I+nSK9wLeyyk3P8Vqi3cBlkdExRbxvDzRYmZF08aCb1TsKmlqzs93pVeQ5NMS6AwMBD4NjJb0yfyH1B8nRTMrTlDMzdvl27DI7HzgwchWwJ4saSPZYtULgD455XqnGLXElwAdJbVMrcXc8rVy99nMiiICRWHbNvorcCKApL2AnYByYCxwlqTWkvoB/YHJwBSgf5pp3olsMmZsSqpPAaen8w4HHqrr4m4pmlnx6umJltzXlkiaD1wHjARGptt01gPDU4KbJWk0MBuoAC6JiMp0nkuB8UAZMDIiZqVLfB8YJeknwItA3leigJOimW2LekqKeV5b8vVayt8A3FBDfBwwrob4m2Sz0wVzUjSz4hQ3ptjkOCmaWdGKmH1ucpwUzaxIhd2Y3VQ5KZpZcQInRTOzakq39+ykaGbF8yKzZma5nBTNzJIIqEQRLasAAAYmSURBVCzd/rOTopkVzy1FM7McTopmZkkAJfyOFidFMytSQHhM0cwsE3iixcysGo8pmpnlcFI0M6viBSHMzDYLwEuHmZnlcEvRzKyKH/MzM9ssIHyfoplZDj/RYmaWo4THFFvs6AqYWRMTkc0+F7LVQdJISYvTO5633HeVpJDUNf0sSSMkzZP0sqTDcsoOlzQ3bcNz4odLmpmOGSFJddXJSdHMihdR2Fa3e4AhWwYl9QEGAe/mhIcC/dN2AXBHKtsZuA4YQPaO5+skdUrH3AGcn3PcVtfakpOimRUpiMrKgrY6zxTxLLC0hl23AN8juyuyyjDgvshMBDpK6gkMBiZExNKIWAZMAIakfe0jYmJEBHAfcFpddfKYopkVp7ilw7pKmprz810RcVe+AyQNAxZExEtb9HZ7Ae/l/Dw/xfLF59cQz8tJ0cyKV/gtOeURcUShhSXtAlxL1nXeIdx9NrOiBBAbo6BtG+wJ9ANekvQ20BuYLmk3YAHQJ6ds7xTLF+9dQzwvJ0UzK06kRWYL2Yo+dcyMiO4R0Tci+pJ1eQ+LiPeBscA5aRZ6ILAiIhYB44FBkjqlCZZBwPi0b6WkgWnW+Rzgobrq4O6zmRWtkEmUQkj6E3AC2djjfOC6iLi7luLjgFOBecAa4FyAiFgq6cfAlFTu+oiomry5mGyGuw3wSNry1yka0U2Ykj4E3tnR9WgAXYHyHV0JK0qp/pl9IiK6bc8JJD1K9vspRHlE1HkbTGPSqJJiqZI0tZjBZtvx/GfWfHlM0cwsh5OimVkOJ8V/jbw3q1qj5D+zZspjimZmOdxSNDPL4aRoZpbDSbEBSRoi6bW0ltvVO7o+Vrd86/tZ8+Ck2EAklQG3ka0Btx/wFUn77dhaWQHuoYA196x0OSk2nCOBeRHxZkSsB0aRrQdnjVie9f2smXBSbDi1rfFmZo2Yk6KZWQ4nxYZT2xpvZtaIOSk2nClAf0n9JO0EnEW2HpyZNWJOig0kIiqAS8kWwJwDjI6IWTu2VlaXtL7fC8DekuZLOm9H18n+tfyYn5lZDrcUzcxyOCmameVwUjQzy+GkaGaWw0nRzCyHk2ITIqlS0gxJr0h6QNIu23GueySdnj7/Lt9iFZJOkHT0NlzjbUlbvfWttvgWZT4q8lr/Jem7xdbRbEtOik3L2og4JCIOANYDF+bulLRN7/GOiH+PiNl5ipwAFJ0UzZoiJ8Wm6zngU6kV95ykscBsSWWSbpQ0RdLLkr4FoMxv0/qOjwPdq04k6WlJR6TPQyRNl/SSpCck9SVLvlemVupxkrpJGpOuMUXSMenYLpIekzRL0u8A1fUlJP1V0rR0zAVb7LslxZ+Q1C3F9pT0aDrmOUn71Mcv06zKNrUsbMdKLcKhwKMpdBhwQES8lRLLioj4tKTWwD8kPQYcCuxNtrZjD2A2MHKL83YD/gf4TDpX54hYKulO4KOIuCmV+z/gloh4XtIeZE/t7AtcBzwfEddL+hxQyNMg30zXaANMkTQmIpYAuwJTI+JKST9M576U7IVSF0bEXEkDgNuBk7bh12hWIyfFpqWNpBnp83PA3WTd2skR8VaKDwIOqhovBDoA/YHPAH+KiEpgoaQnazj/QODZqnNFRG3rCp4M7Cdtagi2l9Q2XeNL6di/S1pWwHf6tqQvps99Ul2XABuB+1P8D8CD6RpHAw/kXLt1AdcwK5iTYtOyNiIOyQ2k5LA6NwRcFhHjtyh3aj3WowUwMCI+rqEuBZN0AlmCPSoi1kh6Gti5luKRrrt8y9+BWX3ymGLpGQ9cJKkVgKS9JO0KPAucmcYcewIn1nDsROAzkvqlYzun+CqgXU65x4DLqn6QVJWkngW+mmJDgU511LUDsCwlxH3IWqpVWgBVrd2vknXLVwJvSfpyuoYkHVzHNcyK4qRYen5HNl44Pb186b/JegR/AeamffeRrQRTTUR8CFxA1lV9ic3d178BX6yaaAG+DRyRJnJms3kW/EdkSXUWWTf63Trq+ijQUtIc4OdkSbnKauDI9B1OAq5P8a8B56X6zcKveLB65lVyzMxyuKVoZpbDSdHMLIeToplZDidFM7McTopmZjmcFM3Mcjgpmpnl+P+LpD9Y9q3/9wAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FmUwrEAcrUwb"
      },
      "source": [
        "#### Logistic Regression w/ Word2Vec Representation - Precision Recall Curve"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7RA0AprzrUwb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 409
        },
        "outputId": "f348943a-70fa-4496-b5f4-971a09926d54"
      },
      "source": [
        "create_pr_curve('Logistic Regression (Word2Vec) Precision-Recall Curve', grid_lr_clf_w2vec_preds, y_test)"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAGICAYAAACjo4zRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3wUdf7H8dcnhUBCSOjSSQigqGBBBUFFUA9s2E7FnyCCoiK20+PuvDv0sNwpp6eeWNBT7BX1EPFsiNhQQREpIr1LkxYQQpLv74+ZhE1IyCYkW9/Px2MfuzszO/OZ2d35zGe+U8w5h4iIxLeEcAcgIiLhp2QgIiJKBiIiomQgIiIoGYiICEoGIiKCkkGFzOwxM/trFT7X2sxyzSyxJuKKVGb2rpldVkPj/o2ZvVUT4w5i2uPN7M5wTNuf/llm9kq4ph8Qx/+Z2ftBDFel/02kMrNlZnaK//p2M3s+3DFVt5hKBoFfWHVxzl3tnLujstN2zq1wztV1zhVUZnpmNtjMCvxEss3MvjezM6sSezg45/o5556podHfBfwDwMweN7NHi3qYWbKZ7SinW7fqDMLMupnZB2b2i5ltMLPXzKyZ3++PZjatjM80MrM8MzusKtN0zr0NHGpmnfcT1zIz+9X/7azzE1jdqkxvP3G84Jw7LYjhgvrfVIWZOf97zTWz1WZ2fyRtdJlZPTN7wMxW+DEu9t83Cnds+xNTySCGfOmcqwtkAo8AL5tZZnVPJJL+QBUxs2OADOfcdL/TNODEgEG6AiuAE0p1A5hZyWlVtFzqA+OAtkAbYDvwtN/veeB4M8sq9ZmLgR+cc3MqE0spLwHDKhjmLP+3cxTe/P+l9ABmlnQAMUSKLv58ngRcBAwJczwAmFkt4CPgUKAvUA/oDmwCjq3C+EL2XcVFMjCzFD8zr/EfD5hZSkD/kWa21u93hb/lkeP3K9494G/dTTKzLf5W4admlmBmzwGtgbf9LYGRZtbWH0+S/9kGZva0P43NwezucM4VAs8BaUD7gHn5p7/Vsc4vx+tUYl4eNbPJZrYDONnMmpvZBH8Ld6mZXR8wrmPNbIZfoawzs/v97rXN7Hkz2+Qvi2/MrKnfb6qZXeG/TjCzv5jZcjNbb2bPmlmG369o+Vzmz8tGM/vzfhZHP+CTgPfTgEMCtrZOAF4G0kp1+9I5t8fMDvFj22Jmc83s7ID5LGu5HGlm35rZdvN2z9QO+F7edc695pzb5pzbCTwM9PD7rQKmAANLxT8IeNaf3plmNsuP5YvArX0za2Vmb/jfxyYzezhgHFOBM/azjIo551YD7wKH+eN1ZnatmS0EFlY1DvMq18/812Zm//K/221m9oP5lY+V2q1mZlea2SL/fzPRzJoH9HNmdrWZLfRjGWtmFuR8LgI+B44IGF9V5qudmU3xu200sxesahtgg/DWBec65+Y55wqdc+udc3c45yYHzG9OQEyB65heZrbKzP5gZj8DT5vZfAvYO2BmSX78R/nvu/nzucW8PQm9qhA3OOdi5gEsA04po/toYDrQBGgMfAHc4ffrC/yMl8lT8bbsHJDj9x8P3Om//jvwGJDsP04ArKxp4201OiDJf/8O8AreVmUycFI58zAY+Mx/nQhcC+QBTfxu/wImAg2AdOBt4O+VmJeteCuuBH+YmcAooBaQDSwBfuMP/yUw0H9dF+jmv77Kn26qH+PRQD2/31TgCv/1EGCRP966wBvAc6WWzxNAHaALsBs4pJzl8hrw+1LdluL96QAmAb2BF0p1G+Uv70XArf589sbbmu9YznKpBywHbvI/ewGwp+h3UEZsNwLTA97/H7Aw4H1H/ztsDBwJrAeO85fdZXi/nRT//ff+d5yGl4B6Boyngb/M6lX0+wdaAXPZ+zt3wAf+OOpUNQ5K/j5/g/f7yQQMOARoVsb/pjewEa9aSQH+DUwLiNv531Um3op0A9B3P//zwN/0wcBa4Cb/fVXnKwc41R+uMd7GxgPlLNvbgefLie1l4JkK1lPF8ZexrHoB+cA9fix18H7DLwQMfwYw33/dAq/qOB3vt3uq/75xpdefNb2CDuWD8pPBYuD0gPe/AZb5r5/CX5kG/CjKSwajgf8GfpHlTZuAZAA0AwqB+kHMw2D/x7AFbwX0K3Ch38+AHUC7gOG7A0srMS/PBvQ/DlhRavp/Ap72X08D/gY0KjXMELyE2rmM+KeyNxl8BAwP6NfRn6ekgOXTMqD/18DF5SyXD4CrS3Ubj/fHTsBbAaQCVwd024y3G+EEvCSZEPDZl4Dby1kuJwJr8BO93+0LykgGQGfgF+CEgG6pwDbgeP/9XcB//deP4q+gA4Zf4MfZHW9FmFTOMkj2l1nr/fz+c/3fznK8XYx1/H4O6B0wbJXioGQy6A38BHQLXLZl/G/+A9wb0K+u/ztoGxBbYNJ7Ffjjfv4jzl++O/zXLwEpB7p8S33mHOC7sv7f7D8ZfAD8o4JxV5QM8oDapf7H24FU//0LwCj/9R/wN7AChn8PuKyieSz9iIvdREBzvD9HkeV+t6J+KwP6Bb4ubQzeFub7ZrbEzP4Y5PRbAb845zYHOfx051wmXhUxkb37wRvjb837JeEW4H9+dwhuXgK7tQGaF43LH9+tQFO//1CgA/CjvyuoqFR9Du8H97J5u6PuNbPkMqZV1nJPChg/eCvpIjvxVhRl2YxXCQUqajc4HFjivF02nwV0qwN85cex0nm73QJjaRHwPnC5NAdWO/+fFTB8CX6p/y5wg3Pu06LufhyvAYP83R3/h7+LCG+Z31xqmbfyp9kKWO6cyy9nGRTN/5Zy+gOc45zLdM61cc4Nd879Ws48HkgcRfM5BW8X2VhgvZmNM7N6ZQxa4nfgnMvF23oNXP5l/g78XXq5/iOwPegof5iL8DZq0g5kvsysqZm9bF6D9Da8qroqDb6b8Db+DsQG59yuojfO2xU2HzjLzFKBs4EX/d5tgN+Wmt+eVYkhXpLBGryFVqS13w28ErNlQL9W5Y3EObfdOXezcy4b7wv5nZn1Keq9n+mvBBpUdh+k/6e5BhhoZkfildq/Aof6f/hM51yG8xrSgp2XwDhX4lUVmQGPdOfc6f70FzrnBuDtXrsHeN3M0pxze5xzf3POdQKOB87E21daWlnLPR9YV5nl4JuNl5gCTcPbvXQGULQynos332cA3/h/qjVAKzML/L23BlYHvA9cLmuBFqX2W7cOnLCZtQE+xNsKfa6MeJ8BLsQr24t254G3zO8qtcxTnXMv+f1aW/mNhofgVbTbyulfkdLffVXj2DtC5x5yzh0NdML7fn5fxmAlfgdmlgY0pOTyL2/8hzrvqLy6gQnX7+ecc6/i7c4cdYDzdTfe8jncOVcPuBSvEq+sD4Hf+PNYnp14G3VFDirVv6x1yUvAAKA/MM9PEODN03Ol5jfNOfePygYei8kg2bwGzqJHEt6C/IuZNTavcXEUXuYHryS93LwGxlSg3GOj/YapHH8lsRUowNv9A94KLruszznn1uJtQT5iZvXNO+TxxLKGLeOzvwBP4pWFhXj72P9lZk38mFqY2W8qOy++r4HtfmNVHTNLNLPDzDtyBzO71Mwa+9Mt2hotNLOTzexw84662YZX8heWMf6XgJvMLMu8QxzvBl6paIuzHJPxSv1i/h9iHXADfjLwt+a/8rsVHeL5Fd4fcKS/7HsBZ+Ht3y3Ll3hJ63p/+PMIOBLEzFrgNRI/7Jx7rJxxfIq3zMYBLzvn8vzuTwBXm9lx5kkzszPMLB3v+1gL/MPvXtvMegSM8yS831F1OJA4AO8IL//zyXi7bHZR/u/gcjM7wrwDN+4GvnLOLaumefkHcKWZHXQA85WOt4ttq//9lpXUgvEc3gp6gpkdbN5BFA3N7FYzO90fZhZwif9/60up33U5XgZOw9s4fDGg+/N4FcNv/PHVNq8RumWZY9mPWEwGk/G2nosetwN3AjPwti5/AL71u+Gcexd4CPgYbxdQ0aGLu8sYd3u8zJ+Lt8J4xDn3sd/v73gJZ4uZ3VLGZwfirTR/xNu/fWMl5ukB4HTzjor4Q1Gcfjn7Id6++MrOC847B+JMvCMxluJVHk8CGf4gfYG5ZpYLPIi3P/9XvC2Z1/ESwXy8o3zK2jp+yu8+zR//LuC6Ssx3YKzf4v1RjyvVaxrebrLPA7p9ilfNTPM/m4e38u/nz+MjwCDn3I/lTCsPOA9v//gveLsi3ggY5Aq8xH97wC6M3FLjcHi7htqwdxcRzrkZwJV4u1c2431Pg/1+BX6cOXiHya7yp11kAPB4WTFX1gHGUaQe3sp3M95uoE14u1JLT+tDvA2TCXgr43Z4h9pWC+fcD3jf9e8PYL7+hrfraSvewR6B33dlYtkNnIL3P/8A7z/yNd4up6/8wW7w49iCtwsxmCML1+Ktc47HOxClqPtKvGrhVrz2kJV4iazS6/aiI2HEZ2aHAHPwGqSqsgUbMWJpXgDM7DS8Bulzwh1LqJnZWXhHdl0Y7lgkNikZAGZ2Ll5FkYq3r7cwWlc4sTQvIhI6sbibqCquwtt1sxivHeCa8IZzQGJpXkQkRFQZiIiIKgMREQlxMjCzp8y7jkmZF+vyDwV7yLxrmMw2/9obIiJSs0J99cLxeId8PVtO/354h2+2xzur8FH/eb8aNWrk2rZtWz0RipRl61ZY5J/nYwYdO0JaGecV7dgBCxZA0e7XnBzIyNh3OJEaNnPmzI3OucYVD+kJaTJwzk0zs7b7GaQ/3jViHN5x9Jlm1sw/xrZcbdu2ZcaMGdUYqUgpZ5yxNxncfDOM2Xs4/e78AlKSAq56fcstcN993uuOHWHSpBAGKuIxs30uobI/kdZm0IKS109ZRcnrl4iEx+cB57QNHVr88p3Za+nxjyls2L67zP4lPicSwSItGQTNzIaZd639GRs2bAh3OBLrtm/f+7pdu+KXhzRL55cdeTz+yeIy+7OtqpcREgmtSEsGqyl5cbWWlHMxK+fcOOdcV+dc18aNg94tJlI16QEXTF28d8Wf3bgu5x7ZkuemL2fdtl379KdeWRfxFIk8kZYMJuJf9te8+9Zurai9QCQkegRcp+0//ynR64Y+7ckvdDw6dfG+/Xvsc303kYgU6kNLX8K72FJH827tNtS8291d7Q8yGe9OW4vwLoA1PJTxiZTrmoATuR9+GL78svht64ap/Pbolrz41Qo2fjDV61/W50QiWEycgdy1a1eno4mkRhUUQOfOMG+e9z4lBUaM8BqL27Xj52/nMOnGO7ns23dI3uNfrbpTJ5g9GxITyx+vSA0xs5nOua5BD69kIBKkBQu83T6bNlU8bMOG8MUX0KH0/XhEQqOyySDS2gxEIlfHjt6hop067XewdS2zlQgk6oT6DGSR6Naxo7fr53//g0cf9ZLDtm3eUUM9evDCUadze14rPmzYosT9PkUinSoDkcpKTPTOSJ40CTZv9toTNm+GSZM49ZYhJCQl8dBHiyoej0gEUTIQqUZN6tVmYLc2vPndKpZsyK34AyIRQslApJpd3asdKUmJPPjRwnCHIhI0JQORataobgqXHd+Wid+vYeG67RV/QCQCKBmI1IBhJ2aTmpzIA6oOJEooGYjUgAZptbi8RxbvzF7L/LW6WJ1EPiUDkRpy5QnZpKck8cCHP4U7FJEKKRmI1JCM1GSGnpDFe3PXMWf11nCHI7JfSgYiNWhIzywy6iTzrw9UHUhkUzIQqUH1aicz7MRsPvpxPbNWbgl3OCLlUjIQqWGXHd+W+qmqDiSyKRmI1LC6KUlcdVI7PvlpAzOX/xLucETKpGQgEgKDurehUd1a3K/qQCKUkoFICKTWSuLqk9rx+aJNTF8SxP0QREJMyUAkRC7t1oYm6Snc/8FPxMJNpSS2KBmIhEjt5ESuPTmHr5f+wheLVR1IZFEyEAmhi45pRbOM2qoOJOIoGYiEUFF1MHP5Zj75aUO4wxEppmQgEmIXdm1Fi8w6/EvVgUQQJQOREKuVlMD1fXL4ftVWpvy4PtzhiABKBiJhcd5RLWndIFVtBxIxlAxEwiA5MYEb+rRn7pptvDd3XbjDEVEyEAmX/kc0J7tRGg98+BOFhaoOJLyUDETCJCkxgRtOac+PP29n8py14Q5H4pySgUgYndm5Oe2b1OWBDxdSoOpAwkjJQCSMEhOMG0/pwKL1uUyavSbc4UgcUzIQCbN+hx3EwQel88CHC8kvKAx3OBKnlAxEwiwhwbjp1A4s3biDt2apOpDwUDIQiQCndWrKoc3r8dBHC9mj6kDCQMlAJAKYGb87tQMrftnJhJmrwh2OxCElA5EI0fvgJnRplcm/pywiL1/VgYSWkoFIhCiqDlZv+ZVXZ6wMdzgSZ5QMRCLIie0bcXSb+jw8ZRG79hSEOxyJI0oGIhHEzLj51A78vG0XL3+9ItzhSBxRMhCJMN3bNeS4rAaMnbpY1YGEjJKBSIQx88472LB9N89PXx7ucCROKBmIRKBu2Q3pkdOQR6cuZmdefrjDkTigZCASoX53agc27cjj2S9VHUjNUzIQiVBHt2nASR0a8/gni8ndrepAapaSgUgEu+nUDmzeuYfxny8NdygS45QMRCLYEa0yOeWQJoybtoRtu/aEOxyJYUoGIhHuxlM6sG1XPv/5VNWB1BwlA5EId1iLDH5zaFOe+mwpW3bmhTsciVFKBiJR4KZTO7B9dz5PqjqQGqJkIBIFDj6oHmd0bsbTny/llx2qDqT6KRmIRIkb+7Rn554CHp+2ONyhSAxSMhCJEu2bptO/S3Oe/WI5G7bvDnc4EmOUDESiyPV92rM7v4DHPlF1INVLyUAkimQ3rsu5R7bk+enLWbdtV7jDkRiiZCASZW7o0578QsejU1UdSPVRMhCJMq0bpvLbo1vy4lcrWLv113CHIzFCyUAkCo3onYPD8fCUReEORWKEkoFIFGpZP5WLjmnFqzNWsvKXneEOR2JAyJOBmfU1swVmtsjM/lhG/9Zm9rGZfWdms83s9FDHKBINrj05B8MY+7GqAzlwIU0GZpYIjAX6AZ2AAWbWqdRgfwFedc4dCVwMPBLKGEWiRbOMOlxyXGtem7mK5Zt2hDsciXKhrgyOBRY555Y45/KAl4H+pYZxQD3/dQawJoTxiUSV4b3akZRgPPSRqgM5MKFOBi2AlQHvV/ndAt0OXGpmq4DJwHVljcjMhpnZDDObsWHDhpqIVSTiNalXm4Hd2vDmd6tYsiE33OFIFIvEBuQBwHjnXEvgdOA5M9snTufcOOdcV+dc18aNG4c8SJFIcXWvdqQkJfLgRwvDHYpEsVAng9VAq4D3Lf1ugYYCrwI4574EagONQhKdSBRqVDeFy45vy8Tv17Bw3fZwhyNRKtTJ4BugvZllmVktvAbiiaWGWQH0ATCzQ/CSgfYDiezHsBOzSU1O5AFVB1JFIU0Gzrl8YATwHjAf76ihuWY22szO9ge7GbjSzL4HXgIGO+dcKOMUiTYN0mpxeY8s3pm9lvlrt4U7HIlCFgvr2a5du7oZM2aEOwyRsNq6cw8975nC8TkNeXxg13CHI2FmZjOdc0H/ECKxAVlEqiAjNZmhJ2Tx3tx1zFm9NdzhSJRRMhCJIUN6ZpFRJ5l/ffBTuEORKKNkIBJD6tVOZtiJ2Xz043pmrdwS7nAkiigZiMSYy45vS/1UVQdSOUoGIjGmbkoSV53Ujk9+2sDM5b+EOxyJEkoGIjFoUPc2NKpbi/tVHUiQlAxEYlBqrSSuPqkdny/axPQlm8IdjkQBJQORGHVptzY0SU/h/g9+IhbOJ5KapWQgEqNqJydy7ck5fL30F75YrOpA9k/JQCSGXXRMK5pl1FZ1IBVSMhCJYUXVwczlm/nkJ13vUcqnZCAS4y7s2ooWmXX4l6oD2Q8lA5EYVyspgev75PD9qq1M+XF9uMORCKVkIBIHzjuqJa0bpKrtQMqlZCASB5ITE7ihT3vmrtnGe3PXhTsciUBKBiJxov8RzclulMYDH/5EYaGqAylJyUAkTiQlJnDDKe358eftTJ6zNtzhSIRRMhCJI2d2bk77JnV54MOFFKg6kABKBiJxJDHBuPGUDixan8uk2WvCHY5EECUDkTjT77CDOPigdB74cCH5BYXhDkcihJKBSJxJSDBuOrUDSzfu4K1Zqg7Eo2QgEodO69SUQ5vX46GPFrJH1YGgZCASl8yM353agRW/7GTCzFXhDkcigJKBSJzqfXATurTK5N9TFpGXr+og3ikZiMSpoupg9ZZfeXXGynCHI2GmZCASx05s34ij29Tn4SmL2LWnINzhSBgpGYjEMTPj5lM78PO2Xbz89YpwhyNhpGQgEue6t2vIcVkNGDt1saqDOKZkIBLnitoONmzfzfPTl4c7HAkTJQMR4bjshvTMacSjUxezMy8/3OFIGCgZiAgAN53ank078nj2S1UH8UjJQEQAOLpNA07q0JjHP1lM7m5VB/FGyUBEit10agc279zD+M+XhjsUCTElAxEpdkSrTE45pAnjpi1h26494Q5HQkjJQERKuPGUDmzblc9/PlV1EE+UDESkhMNaZPCbQ5vy1GdL2bIzL9zhSIgoGYjIPm46tQPbd+fzpKqDuKFkICL7OPigepxxeDPGf7GMzTtUHcQDJQMRKdP1fdqTuzuf/3ym6iAeKBmISJk6HpSu6iCOKBmISLmu79OeHXn5PPnZknCHIjVMyUBEytXxoHROP7wZ4z9XdRDrlAxEZL+u792enXsKVB3EOCUDEdkvVQfxQclARCp0Qx9VB7FOyUBEKtShqX9k0efL+EXVQUxSMhCRoFxfVB18quogFikZiEhQiqqDZ75QdRCLlAxEJGg3qDqIWUoGIhK09k3TObNzc1UHMUjJQEQq5freOezcU8ATqg5iipKBiFSKqoPYpGQgIpV2fe8cflV1EFOUDESk0lQdxB4lAxGpkhv6qDqIJUoGIlIlOU3SOcuvDjbl7g53OHKAQp4MzKyvmS0ws0Vm9sdyhrnQzOaZ2VwzezHUMYpIcK4vrg50N7RoF9JkYGaJwFigH9AJGGBmnUoN0x74E9DDOXcocGMoYxSR4BVVB89+qeog2oW6MjgWWOScW+KcywNeBvqXGuZKYKxzbjOAc259iGMUkUpQdRAbQp0MWgArA96v8rsF6gB0MLPPzWy6mfUta0RmNszMZpjZjA0bNtRQuCJSkZwm6ZzdRdVBtIvEBuQkoD3QCxgAPGFmmaUHcs6Nc851dc51bdy4cYhDFJFA1/Vuz697ChinI4uiVqiTwWqgVcD7ln63QKuAic65Pc65pcBPeMlBRCJUTpO6XnXwxXJVB1Eq1MngG6C9mWWZWS3gYmBiqWHewqsKMLNGeLuNtLkhEuGu692eXfmqDqJVSJOBcy4fGAG8B8wHXnXOzTWz0WZ2tj/Ye8AmM5sHfAz83jm3KZRxikjlqTqIbuacC3cMB6xr165uxowZ4Q5DJO4tWp/Laf/6hCtPyOZPpx8S7nDimpnNdM51DXb4SGxAFpEoVVwdfLmcjaoOooqSgYhUqxG927M7v4AnpqntIJooGYhItVJ1EJ2UDESk2l3XR9VBtFEyEJFq165xXfof0ULVQRRJqmgAMxtUmRE6556tejgiEitG9M7hv7NWM27aEm7VkUURr8JkAIyvxPgcoGQgIgHVwTKGnZhNo7op4Q5J9iOY3URZlXhk10yYIhKNruudQ15+IePUdhDxKqwMnHPLQxGIiMSe7MZ1OcevDq48IZvG6aoOIpUakEWkRo0org4WhzsU2Y9gGpCX4rUFBMM559odWEgiEkuKqoPnpi9n2IntVB1EqGAakD8h+GQgIrKPEb1zeGvWasZNW8yfz+hU8Qck5IJpMxgcgjhEJIapOoh8ajMQkZC4rk978vILefwTtR1EomB2E+3DzLoAHYHapfvppDMRKUtWozTOObIFz3+1nGEnZdMkfZ/Vh4RRpZKBfy/id4BuRZ3858A2BSUDESnTdb3b89Z3qxn3yRL+cqbaDiJJZXcT3Q00BE7ESwTnAr2BF/BuTXlstUYnIjElsDpYv31XuMORAJVNBr/BSwjT/fernHNTnXODgA+BG6ozOBGJPdf1bs+eAse4T3RWciSpbDJoBixxzhUAu4D0gH5vAGdUV2AiEpuyGqVxzhGqDiJNZZPBz0Cm/3o50D2gX061RCQiMe+63jnsKXA8ruogYlQ2GXzG3sbj54DbzOxxMxsLjAHeq87gRCQ2tS2qDqarOogUlU0Gf2PvCn8MMBZv19AAYCJwXfWFJiKx7LreOeQXqjqIFJVKBs65xc65T/3Xe5xzNzvnWjrnGjjnLnHObaqZMEUk1rRtlMa5R6o6iBSVSgZmlmxmaeX0SzOz5OoJS0TiwYiTvergsamqDsKtsruJngSeKKff4/5DRCQoRdXBC18tZ/02VQfhVNlkcDLw33L6TQT6HFg4IhJviqsDtR2EVWWTQRNgfTn9NgBNDywcEYk3qg4iQ2WTwXrg8HL6HQ6oAVlEKq3oyKJHdUXTsKlsMpgE/NXMOgd2NLPDgT8Db1dXYCISP9o0TOO8I1vw4lcrVB2ESWWTwShgCzDTzL4ws1fN7HPgW2Ar8JfqDlBE4sMIVQdhVdnzDDYCxwB/x7tq6RH+813AMX5/EZFKU3UQXpW+05lzbotzbpRzrrtzroNz7njn3O3Oua01EaCIxA9VB+FTpdtemlkjMzvTzC4zswZ+t9pmpttoikiVtWmYxvlHteCFr1awTtVBSFX2DGQzszHAKrzzCp4C2vq9/4vXiCwiUmUjTm5PQaHj0amqDkKpslvyfwJGAKOB49h720vwjiQ6s5riEpE41bphKucf1YIXv1Z1EEqVTQZXAKOdc3fjHUEUaBHQrlqiEpG4puog9CqbDFqw95aXpeUBZV7ETkSkMlQdhF5lk8Fq4LBy+nUBlh5YOCIinhEnt6dQ1UHIVDYZvAaMMrMeAd2cmXUAbgZerrbIRCSuedVBS178egU/b1V1UNMqmwxuB34EpgEL/W6vAT/gtRn8o9oiE5G4d+3JORQWOh7TeQc1rrJnIP8K9AIGA18AHwLfAMPwjiS6pnrDE5F4puogdCp7nkEjoNA595xz7lLn3GnAUKAR8BNwfw3EKCJxbETvHCTwwGYAACAASURBVL/tYFG4Q4lpFSYDM0sxswfNbDuwDthkZtf4/S4FFgNjgJVA35oMVkTiT6sGqVxwdEte+nqlqoMaFExlMAq4DvgSb6X/AfCgmf0beBbvaqX9nXPHOec+qLFIRSRuXXtyDoVO1UFNSgpimIuAR5xzI4o6mNkQvPshfwCc5ZzLq6H4RERKVAfX9MrhoIza4Q4p5gRTGbQC3izV7Q3/+X4lAhEJBVUHNSuYZJAMbC/Vrej9huoNR0SkbK0apPLbrl51sHbrr+EOJ+YEezRRCzPLLnoA2WV19/uJiNSI4b2KqgOdd1DdgmkzAHi9nO5vldEtsYqxiIjsV1F18PLXK7mmVzuaZdQJd0gxI5hkcHmNRyEiEqThvXJ4bcYqHp26mNH9y7tUmlRWhcnAOfdMKAIREQmGqoOaodtUikjUKTqy6JGP1XZQXZQMRCTqtKyfym+7tuKVb1ayZouOLKoOSgYiEpWuPbmdjiyqRkoGIhKVVB1Ur5AnAzPra2YLzGyRmf1xP8Odb2bOzLqGMj4RiR7XntwOh6qD6hDSZGBmicBYoB/QCRhgZp3KGC4duAH4KpTxiUh0UXVQfUJdGRwLLHLOLfGvafQy0L+M4e4A7gF0vVoR2a/hvbzq4BFds+iAhDoZtMC770GRVX63YmZ2FNDKOfdOKAMTkeik6qB6RFQDspkl4N0t7eYghh1mZjPMbMaGDbpenkg8u/bkHABVBwcg1MlgNd4lsYu09LsVSQcOA6aa2TKgGzCxrEZk59w451xX51zXxo0b12DIIhLpWmTW4UK/Olit6qBKQp0MvgHam1mWmdUCLgYmFvV0zm11zjVyzrV1zrUFpgNnO+dmhDhOEYkyw4uqg49VHVRFSJOBcy4fGAG8B8wHXnXOzTWz0WZ2dihjEZHYUlQdvDpD1UFVhLzNwDk32TnXwTnXzjl3l99tlHNuYhnD9lJVICLBUnVQdRHVgCwiciBUHVSdkoGIxJSiI4vGqjqoFCUDEYkpzTPrcNExrXhtxkpWbd4Z7nCihpKBiMSc4b2KzjvQNYuCpWQgIjFH1UHlKRmISEwa3isHw1QdBEnJQERikqqDylEyEJGYdU2vdhjGWN0ruUJKBiISs1QdBE/JQERi2jW92pFgqg4qomQgIjEtsDpY+Yuqg/IoGYhIzBt+slcd6H4H5VMyEJGY1yyjDhcf24rXZqxSdVAOJQMRiQtFbQeqDsqmZCAicUHVwf4pGYhI3BjeK8c/skjVQWlKBiISNw7KqM2AY1vx+kxVB6UpGYhIXLlG1UGZlAxEJK6oOiibkoGIxJ1reuWQkKDqIJCSgYjEnYMyanPJsa1VHQRQMhCRuHT1Se1ISDAenqLqAJQMRCROFVUHE75dxYpNqg6UDEQkbhVVB2o7UDIQkTim6mAvJQMRiWvX9PLbDj5eGO5QwkrJQETiWtN6RdXB6riuDpQMRCTuXdOrHYlxXh0oGYhI3FN1oGQgIgJ41UFSgvHvKfFZHSgZiIjgVwfHteaN71azfNOOcIcTckoGIiK+a07yqoN4PCtZyUBExNckjqsDJQMRkQDxWh0oGYiIBGhSrzb/d1wb3vhuNcs2xk91oGQgIlLK1Sdle9VBHF2zSMlARKSUourgzTiqDpQMRETKEG/VgZKBiEgZ4q06UDIQESnH1b2y/bOSY786UDIQESlHk/TaXNqtDW/Niv3qQMlARGQ/rjopPqoDJQMRkf2Il+pAyUBEpAJXnZRNcqLxUAxf0VTJQESkAk3Sa3PpcW1467vVLI3R6kDJQEQkCMNOyqZWUkLM3u9AyUBEJAixXh0oGYiIBCmWqwMlAxGRIAVWB0s25IY7nGqlZCAiUglXndSOWkkJMXe/AyUDEZFKaJyewkD/vINYqg6UDEREKmnYibFXHSgZiIhUUixWB0oGIiJVUNR2ECvXLFIyEBGpgkZ1UxjUvS3/nbWaxTFQHSgZiIhU0bATs2Om7SDkycDM+prZAjNbZGZ/LKP/78xsnpnNNrOPzKxNqGMUEQlGLFUHIU0GZpYIjAX6AZ2AAWbWqdRg3wFdnXOdgdeBe0MZo4hIZQw7MZuUpMSorw5CXRkcCyxyzi1xzuUBLwP9Awdwzn3snNvpv50OtAxxjCIiQfOqgzZRXx2EOhm0AFYGvF/ldyvPUODdsnqY2TAzm2FmMzZs2FCNIYqIVM6VfnXw74+i95pFEduAbGaXAl2BMWX1d86Nc851dc51bdy4cWiDExEJUFQdTPx+DYvWR2d1EOpksBpoFfC+pd+tBDM7BfgzcLZzbneIYhMRqbIri9sOorM6CHUy+AZob2ZZZlYLuBiYGDiAmR0JPI6XCNaHOD4RkSqJ9uogpMnAOZcPjADeA+YDrzrn5prZaDM72x9sDFAXeM3MZpnZxHJGJyISUYqOLIrG+x0khXqCzrnJwORS3UYFvD4l1DGJiFSHhnVTGHR8G8ZNW8J1vduT06RuuEMKWsQ2IIuIRKNhJ2RTOwqrAyUDEZFqVFQdRFvbgZKBiEg1G3ZCNnWSE3kois47UDIQEalmDf1rFr09ew2L1m8PdzhBUTIQEakBV56Q5VcH0XHNIiUDEZEaEG3VgZKBiEgNGXZidtRUB0oGIiI1pEFaLS473qsOFq6L7OpAyUBEpAZdWXRkUYTf7yDkZyCHWmFhIatWrWLHjh3hDkUkIqSlpdGyZUsSErQtGApF1cFjnyzm+t45tG+aHu6QyhTzyWDjxo2YGR07dtSPX+JeYWEhq1evZuPGjTRp0iTc4cSNK0/I5pkvlvHQlEX8e8CR4Q6nTDG/dtyyZQtNmzZVIhABEhISaNq0KVu3bg13KHGlqDqYNHsNP0Vo20HMryELCgpITk4OdxgiESM5OZn8/PxwhxF3rjwhm9QIPis55pMBgJmFOwSRiKH/Q3gUVQfv/LA2IquDuEgGIiKR4IoIrg6UDOJUv379uPfeeysc7tNPPyUzMzMEEYXHsmXLMDNWrVoFwPjx48nJyQlzVBKrIrk6UDKIEL169SIlJYW6deuSkZHBkUceyYQJE2pseu+++y4jR46scLgTTjiBLVu21FgcAIMHDyY5OZm6detSr149DjnkEB555JEaneaBWLt2Lddccw1t2rQhLS2N1q1bc+GFFzJz5sxwhyZRoKjt4MEIqw6UDCLIX//6V3Jzc9m0aRMDBgzgoosu4qefftpnuD179oQhupp12WWXkZuby5YtW7jzzjsZMWIEU6dODXdY+1izZg3HHHMMK1euZPLkyWzbto158+Zx1lln8cYbb1RpnLH4fUr56qfVYnCPtkz+YS0Lfo6c6kDJIAIlJSUxfPhwCgoK+OGHH5g6dSpJSUk899xzZGdn06BBAwBWrFjBBRdcwEEHHUSzZs0YNmwY27fv/XFt2LCBoUOH0rp1a+rVq8dRRx3FggULAK8SufPOOwHYvXs3w4YNo0mTJtSrV4/27dvz2muvARRPu0h+fj6jR48mOzub+vXr06dPH+bMmVPcf/DgwQwcOJArr7ySzMxMWrRoweOPPx70vCckJHD++efTsGFDZsyYUdz9008/pWfPnjRo0IB27dpx33334Zwr7j979mz69u1L48aNadCgAaecsvfuqZdffjmtWrUiPT2dTp068eKLLwYdT2mjRo0iLS2NN998k0MPPZTExETq1q3LwIEDueuuuwC4/fbbS0wfSi7vsr7PsWPHcsQRR5T4zNKlS0lMTGTZsmVAxd+3RI8revptBxF0N7SYP+mstL+9PZd5a7aFZFqdmtfjtrMOrfTn8vLyGDt2LMnJyXTp0oVVq1ZRUFDA5MmT+e6770hOTmbXrl307t2bSy65hOeee45du3bxf//3f9xwww089dRTFBYWcvbZZ9O8eXO++eYbGjduzJw5c0hP3/fsx2eeeYZvvvmG+fPn07BhQ1auXFnuSmbMmDE8++yzTJ48maysLP7+979z6qmnsmDBAurVqwfA66+/ziuvvMLjjz/OW2+9xUUXXUTfvn1p06ZNhfNeUFDA66+/zsaNG+nYsSMA8+bN4/TTT+f555/nzDPPZOHChfTr14/GjRszaNAg1q5dy0knncTIkSOZMGECycnJTJs2rXicPXv25J///CeZmZm89tprDBo0iCOOOIJOnTpV+ruZPHkyQ4YMOeDDlUt/n7t37+bmm29m1qxZxUlh/Pjx9OrVi7Zt21b4fUt0KaoOHpm6mAU/b6fjQeE/K1mVQQS56667yMzMpGXLlvz3v/9lwoQJJRoz77nnHjIyMkhNTWXSpEk45xg9ejR16tShfv363HHHHbzwwgsUFBQwY8YMZsyYwVNPPVV80l3nzp1p3rz5PtOtVasWubm5zJs3j/z8fFq1alXuivLpp5/mD3/4AwcffDApKSmMGjWKxMRE3nnnneJhevfuzdlnn01CQgLnnXcemZmZzJo1a7/z/txzz5GZmUnt2rUZMGAAo0eP5qyzzgLgkUce4be//S39+/cnMTGRgw8+mBEjRvDss88WfzYnJ4c//elPpKWlUatWrRJb5kOHDqVhw4YkJiZy8cUX07lz5yrvgtqwYQMtWrSo0mdLC/w+69evT//+/Xn66acBcM7xzDPPMGTIEIAKv2+JPpFWHcRdZVCVLfVQ+fOf/8xf/vKXMvslJCTQqlWr4vdLly5lxYoV+xzpY2b8/PPPLFu2jCZNmpCRkVHhdC+99FLWrVvHTTfdxMKFC+nTpw/33ntvmUfVrFy5kqysrBJxtW3blpUrVxZ3a9asWYnPpKWlFVcahx56KMuXLwfg1ltv5dZbbwVg4MCBPPnkk+zcuZORI0fy0Ucf8ac//YmkpCSWLl3KlClTSuyTLywsLF4ey5Yto0OHDmXOW2FhIbfffjuvvPIKP//8M2bGjh072LBhQ4XLpSyNGzdm9erVVfpsoNLfJ3i7swYOHMg///lPpk2bxpYtWzjvvPOAir/v6kpQEjqRVh2oMogSZlbiZKE2bdrQoUMHtmzZUuKxa9cuWrRoQdu2bVm/fj3btlW8SywpKYk//OEPzJgxg+XLl5Oamlq8RVpaq1ativdhg7eyXbZs2T4rtvLMnTuX3NxccnNzixNBoNTUVO6//35Wr17N2LFji+d1yJAhJeZz27ZtzJ07F4C2bduycGHZW1cvvfQSTz75JBMmTGDz5s1s2bKFLl26lGhvqIzTTz+d119/fb+Nvunp6ftcGHHNmjUl3pf+PgFOPfVUUlJSePvttxk/fjwXX3wxderUASr+viU6XdEzm7RaSRFx3oGSQZQ688wzycvL4+6772b79u0451i9ejVvvvkmAF27duWoo47iiiuuYP369RQWFjJ79ux9VkoAU6ZMYebMmezZs4c6deqQlpZGYmJimdMdPHgw9957Lz/99BN5eXncdddd5Ofnc8YZZ1TbvNWqVYtRo0Zx5513sn37doYPH87LL7/M22+/zZ49e8jPz2fevHl88skngFfZLFiwgHvuuYedO3eSl5fHhx9+CMC2bdtISkqicePGFBYW8tRTT/H9999XOba//e1v5ObmcsEFFzB//nwKCgrYsWMHL730UnFVd/TRR/Ptt98yc+ZM8vPzefjhh1m6dGmF405MTGTQoEE89NBDvPHGGyUSckXft0Sn+mm1GOyfdxDuI4uUDKJUamoqU6ZMYd68eRx88MFkZGTQp0+f4n3zCQkJvP3229SpU4cjjjiCzMxMhgwZQm5u7j7jWrduHQMHDqR+/fo0a9aM5cuXM27cuDKn+/vf/54BAwZw2mmn0bRpU6ZMmcL7779f3HhcXS655BIaNGjAfffdx2GHHcakSZN44IEHaNasGU2aNGHw4MHFu3qaN2/O1KlT+eCDD2jZsiUHHXQQY8aMAbxDVo877jhycnJo0aIF8+bN44QTTqhyXC1atOCbb76hWbNmnHbaacXnRbz11lucf/75gHfk0O9+9zv69u1Ls2bNWLduHT169Ahq/JdffjmffPIJWVlZHHvsscXdK/q+JXoN7ZlF3ZTwVwdW1XI5knTt2tUFHoYYaP78+RxyyCEhjkgksul/EVn++d4CHv54Ee/deGK1tR2Y2UznXNdgh1dlICISZlec4FUHD36070mmoaJkICISZpmptbi8R1sm//AzP/4cmvOgSlMyEBGJAOFuO1AyEBGJAOGuDpQMREQixNCeWaSnJPHgh6GvDpQMREQiRFF18O6cn5m/NrTVgZKBiEgEGeJXB6FuO1AyKE9BAUyaBGecAZmZkJjoPZ9xhtddFwcTkRoQrupAyaAsCxZA585w1lkweTJs3QqFhd7z5Mle986dveFERKpZOKoDJYPSFiyAHj1g3rz9DzdvnjdcFCWEu+++u/iy0BWpW7cuX375ZQ1HVD4z47PPPgvZ9AJvPlMVwdw7OSkpKSLv3iaRJ7A6CNX9V5QMAhUUwHnnwaZN3vvateGWW2D+fMjL855vuQVSUrz+mzZ5w1fDLqOieyCnp6eTkZFBdnY2AwcOrNb76t566628/fbbQQ2bm5tL9+7dq23aRVasWEHdunWLH0lJSdSqVav4/aGHRu4lxiPZjBkzOPbYY0lNTaVdu3Y8//zz4Q5JDtDQntkhrQ6UDAK9++7eiqB2bZgyBcaMgYMPhuRk73nMGPj4470JYd48+N//qmXyf/3rX9m+fTtbt27l448/pk2bNnTr1i2mrkzZunXr4ktY5+bm0qtXL2699dbi90WXpa6seL6P8NatW+nXrx/nn38+mzdv5rHHHuPqq68Oa2UnBy4jNZnLe2bxv7mhqQ6UDAI9+uje1yNGQHlbxt27e/3L+lw1adOmDXfeeSeDBg3iuuuuK77+/s6dO7nlllvIysqiQYMG9O3bl0WLFhV/bs+ePdx999107NiR9PR02rVrx+uvvw7se2/ehx56iKysLNLT02nRokWJ+wuU3k0zYcIEunTpQkZGBl26dCmRoIp2kTz00EO0bNmS+vXrc9VVVx3wHbhmz57NMcccQ3p6Ot26dePHH38s7terVy9uvPFGzjnnHOrVq8d9990HwBNPPMFhhx1GRkYGRx55JO+//37xZ7777jt69uxJRkYGDRo04Pjjj2fz5s3F/Tdv3sz5559fvNz++9//lojn0UcfpWPHjmRkZNCtWzc+/fTTcmPfvn07l112GQ0aNKBNmzY888wzB7Qs9ueNN94gNTWVkSNHkpKSwqmnnsq5555b7pVnJXoM7RHCtgPnXNQ/jj76aFeeefPmldtvHxkZzoH3mD9//8POm7d32MzM4KdRjpNOOsndcccd+3R///33HeDm+/Fccskl7owzznA///yz2717txs1apTr2LGjy8vLc845N3LkSHfIIYe477//3hUWFrqVK1e677//3jnn3G233eb69OnjnHNuwYIFrk6dOm7OnDnOOec2b97svvzyy+LpAu7TTz91zjn3+eefu5SUFDd58mS3Z88eN2nSJJeSkuKmT5/unHPu6aefdklJSe7WW291u3btcgsXLnT169d3zz//fIXz3adPH3fbbbft0x1wxxxzjFu+fLnbtWuXu+CCC9wpp5xSYnmlp6e7jz76yBUWFrodO3a4cePGuXbt2rlZs2a5goIC984777i0tDS3cOFC55xz3bt3d3/7299cfn6+y8vLc19++aXLzc0tHl/Dhg3d559/7goKCtz999/vMjIy3I4dO5xzzr344ouuYcOGbvr06W7Pnj3uySefdKmpqW7ZsmXFy6Bdu3bF8Q0ZMsR1797drV271m3ZssWde+65DnAff/xxmcth+fLlLiMjY7+P8txwww2uf//+Jbrdf//97sgjjyz3M5X6X0hY3ff+AtfmD5Pc3NVbK/U5YIarxHpUlUGgwJvAt2u3/2ED+wdxN7GqatmyJQCbNm1i48aNvPjiizzyyCM0bdqUWrVqcdttt7F27Vq++uornHOMHTuWMWPG0LlzZ8yMli1b0rlz533Gm5SUhHOu+M5jmZmZdOvWrcwYxo8fz/nnn0+/fv1ISkrijDPO4Nxzzy1xI/Y6deowevRoUlJSyMnJoU+fPpR3WfFg/f73v6d169akpKQwePDgfcZ3wQUX0Lt3b8yM1NRUHnzwQUaNGkWXLl1ISEjg9NNP5+STT+bll18GvJvmrFixgpUrV5KcnEy3bt1IS0srHt9FF13E8ccfT0JCAsOGDWPr1q3Fd1B7+umnueqqqzjuuONISkpi6NChdO7cmRdffHGfuAsLC3nhhRe44447OOigg8jIyOCee+7Z77y2bt16n7uYlX6UZ/v27fvc3jQzMzOou9xJ5BvaI4v02jV/RVMlg0DpAdcRX7x4/8MG9q/mG7sEWrVqFQANGzYsvltW586dyczMJDMzkwYNGrBnzx5WrlzJhg0b2LFjR7n3Aw6UnZ3NCy+8wBNPPEHz5s3p2bNniV0qgUrf9xigXbt2Je573KRJkxJ3Rwu873FVBd5LuazxtW3btsT7pUuXcu211xYvm8zMTD7++OPiexY//fTTFBYW0rNnT7KysvjrX/9Kfn5+udMDiqcZzDIosmHDBnbv3l0ivtKfrU7p6els3bq1RLctW7ZU+w2HJDwyUpMZ0iOL9+auY+6arRV/oIqUDAIF3o3qP//Z/7CB/YO8i1VVvPLKK7Ro0YKOHTvSpk0bABYuXFhii3Hnzp0MGDCAxo0bk5qaWu79gEs777zz+OCDD9i4cSMXXngh/fv3Z+fOnfsMV/q+xwBLliwJ+r7HNSUhoeTPt02bNjz11FMllk1ubi6P+m06WVlZPPXUU6xatYqJEyfy5JNP8uyzzwY1rcosg0aNGlGrVq0Sw5f+bGmlj7Iq61GeLl267HPHs++++44uXbpUOF8SHYb09KqDmmw7UDIIdM01e18//DCUdzTGl196/cv6XDVZuXIlt912G+PHj+fBBx/EzGjSpAmXXHIJw4cPL97a3bJlC2+++Sa5ubmYGcOHD2fkyJHMmTMH5xyrVq1i9uzZ+4x/wYIF/O9//2Pnzp0kJyeTkZGBme2zggXv1pETJkzgvffeo6CggHfffZc33niDyy+/vNrn+0DcdNNN3H777cyaNQvnHL/++iufffZZccPzM888U3wP6MzMTJKSksq913NpgwcP5vHHH+frr78mPz+fp59+mlmzZnHJJZfsM2xiYiKXXHIJt912G+vWrWPbtm388Y9/3O/4Sx9lVdajPOeeey47duxgzJgx5OXl8dFHH/HGG28wbNiwoOZNIl9GnZqvDpQMAvXrB506ea937YKTTy77PIOTT4bdu73hOnWCvn2rZfJ33HEH6enp1KtXjxNPPJFFixbxxRdfFN9bF7yjZTp27EivXr1IT0/n8MMP57XXXsPMALjrrru48MILOeecc0hPT6dXr14ljjYqkpeXx+jRo2nWrBmZmZk89NBDTJgwgdq1a+8zbI8ePXjmmWe45ZZbqF+/PiNHjuT5558vt40hXK688kpGjhzJ5ZdfTv369WndujV33HFH8WGnU6ZM4eijjyYtLY3u3btzySWXMHDgwKDGXbRyv/TSS2nYsCGPPvookydPLq7WSnvwwQfJysri4IMP5vDDD+ess84KOvFUVmZmJpMnT+a1114jIyODK6+8kscee6xGzhOR8CmqDmrqiqa6B3JpRWcgF514tj8NG8IXX0AQ++hFIonugRydHvjwJx74cCGTruvJYS0y9jus7oF8oDp2hM8/31shlKdTJyUCEQmpy3vUXNuBkkFZOnaE2bNLXrU0IaHkVUtnz1YiEJGQyqiTzNCeWbw/bx1zVldv24GSQXkSE/eu+Ddv9q4/tHnz3gRRQ/t/RUT2p6aqAyUDEZEoUlPVQVwkg1hoJBepLvo/RL/Le2RRr3YSD1ZjdRDzyaB27dps2rRJfwARvESwadOmMg8hlujhVQfZfFCN1UFStYwlgrVs2ZJVq1axYcOGcIciEhFq165dfM0riV6De7TlP58t4cGPFvLEoKCPIC1XzCeD5OTkGr0ujIhIOBRVB//68CfmrN5a4XkHFYn53UQiIrHq8p5tqVc7iQeq4azkkCcDM+trZgvMbJGZ7XPBFjNLMbNX/P5fmVnbUMcoIhIN6tVO5ooTsvlw/oG3HYQ0GZhZIjAW6Ad0AgaYWelTfYcCm51zOcC/gP1fCF5EJI4N7lE91UGoK4NjgUXOuSXOuTzgZaB/qWH6A0X3CHwd6GNFV2ETEZESqqs6CHUyaAEE3g1kld+tzGGcc/nAVqBhSKITEYlCe6uDqt8NLWqPJjKzYUDRBdt3m9mccMYTARoBG8MdRBjF+/yDlkG8zz8/QKP/DC5eBmVfX70coU4Gq4HAW0O19LuVNcwqM0sCMoB9riftnBsHjAMwsxmVuVRrLIr3ZRDv8w9aBvE+/3BgyyDUu4m+AdqbWZaZ1QIuBiaWGmYicJn/+gJgitPpwyIiNSqklYFzLt/MRgDvAYnAU865uWY2GpjhnJsI/Ad4zswWAb/gJQwREalBIW8zcM5NBiaX6jYq4PUu4LeVHO24aggt2sX7Moj3+Qctg3iffziAZRATt70UEZEDo8tRiIhIdCWDeL+URRDz/zszm2dms83sIzOr1KFl0aCiZRAw3Plm5sws5o4uCWYZmNmF/m9hrpm9GOoYa1IQ/4PWZvaxmX3n/xdOD0ecNcXMnjKz9eUdTm+eh/zlM9vMjgpqxM65qHjgNTgvBrKBWsD3QKdSwwwHHvNfXwy8Eu64Qzz/JwOp/utrYmn+g10G/nDpwDRgOtA13HGH4XfQHvgOqO+/bxLuuEM8/+OAa/zXnYBl4Y67mpfBicBRwJxy+p8OvAsY0A34KpjxRlNlEO+Xsqhw/p1zHzvndvpvp+OdxxFLgvkNANyBd02rXaEMLkSCWQZXAmOdc5sBnHPrQxxjTQpm/h1Qz3+dAawJYXw1zjk3De9Iy/L0B551nulAppk1q2i80ZQM4v1SVq+WvwAABaVJREFUFsHMf6CheFsHsaTCZeCXxK2cc++EMrAQCuZ30AHoYGafm9l0M+sbsuhqXjDzfztwqZmtwjty8brQhBYxKruuAKL4chRSPjO7FOgKnBTuWELJzBKA+4HBYQ4l3JLwdhX1wqsOp5nZ4c65LWGNKnQGAOOdc/eZWXe885YOc84VhjuwSBZNlUFlLmXB/i5lEaWCmX/M7BTgz8DZzrndIYotVCpaBunAYcBUM1uGt790Yow1IgfzO1gFTHTO7XHOLQV+wksOsSCY+R8KvArgnPsSqI133aJ4EdS6orRoSgbxfimLCuffzI4EHsdLBLG0n7jIfpeBc26rc66Rc66tc64tXrvJ2c65GeEJt0YE8z94C68qwMwa4e02WhLKIGtQMPO/AugDYGaH4CWDeLoJ+kRgkH9UUTdgq3NubUUfiprdRC7OL2UR5PyPAeoCr/nt5iucc2eHLehqFuQyiGlBLoP3gNPMbB5QAPzeORcTFXKQ838z8ISZ3YTXmDw4hjYKMbOX8JJ9I79d5DYgGcA59xheO8npwCJgJ3B5UOONoWUkIiJVFE27iUREpIYoGYiIiJKBiIgoGYiICEoGIiKCkoHECDMb7F+ltOix3cy+N7MR/gmIoYihrT/twZX4TFHcbWssMJEgRM15BiJB+i3eGbj1/Nf/BpoAo/b3oWqyFuiOd1XNYL3jf6bCk4JEapLOM5CY4G+NPw20d84tCuj+MXCUcy6jjM8kA/mxdEKSSFVpN5HEum+AemZ2rL87ZriZ3Wtma4DdQCaAmZ3nX+Fzp5ltMbPXzKx16ZGZ2ZVm9q2Z/Wpmm83sEzM73u+3z24iMzvGzD4ws03+Z5aY2SMB/ffZTWRmyWZ2p5ktM7M8//lOP3lRalpXmdloM1vrx/22mcXapcslBJQMJNZl4V2SIdd//2e8a/UMA84FdpnZ1cAEYB7eNa2uwrvg3Sdmll40IjP7J96NU74FLgQuxbuJzj5Jwx++Lt5lEwrwrqTaDxhNxbtnnwH+CDwLnAmMB/7A3nt1BPoTkAMMAW7A2+X0fAXjF9lXuO/ao4ce1fHAW9k6oCPeyrY+3kq9AO/CbW39/t/i7x71P1cX774XT5UaXxaQB9zov8/xx3X/fmIomsZg/31X/33nIOJu678/zH9/e6nh/hI4roBpTS013C1+9+bh/k70iK6HKgOJNT8Ce/AuVPgI8ALeVnORt5xzgW0E3fEam18ws6SiB97NQX7Eu8UgwCl4lfS4SsSyENgCPG5ml5pZq4o+EDC90lv3Re9L36Nicqn3P/jPZVYrIuVRMpBYcy5wDHAwkOacG+ScC7xFYOmjdpr4zx/iJZHAx+HsvVNe0fOqYANxzm3Fuy/1GrzEtMLM5pjZ+fv5WINy4vy5VP8ipW9/WHQPi9rBxikCOrRUYs8cF3A0URlKHzlUdGnnwcDcMobf7j9v9J9bAAuCDcY5Nws43682uuLt43/VzLo45+aU8ZGilftBlDxE9aBS/UWqlSoDiXf/3979qkQQRmEYf042G8Rq8Qa8A5tXoEWMBoO4cUGjyWTwBhSbWAQxLApr0CBqEEQMohcgiEU8hjOCLLuKa/T5wTBl/ob5XmbON5wuNeBPZOZ5n+Vz4D8C3qnC869l5ltWc/I29dxNDtj0uFn39uKYa9adYc4v/cQ3A/1rmfkcES1gMyJGgQOqoDxOfZ/vZOZ2Zt5FxAaw3Mww2qcKylPATWbu9h47Imao8NgD7oERYIkKn9MB13PdNC9Zbd4mulRdow3sZOZVv/2kvzIM9O9l5lZEPAAtYJZ6Lh6BE+Diy3YrTRe9Raq96gtwCRwOOPQt8EoN5GNUCJwB05n5Xe1hnmpTuUDNInoC1oG14e5Q+pl/IEuSrBlIkgwDSRKGgSQJw0CShGEgScIwkCRhGEiSMAwkSRgGkiTgAylDhF00PjZHAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dcLh94NmrUwb"
      },
      "source": [
        "### Gradient Boosted Decision Trees (XGBoost)\n",
        "\n",
        "Our last model will be a gradient boosted decision tree classifier. Gradient boosted decision trees use an ensemble of decision trees, each tree correcting the mistakes of the previous, to create powerful models. Gradient boosted decision trees often have great predictive power with a variety of features without being too demanding in regards to memory and training times. Addtionally, the XGBoost implementation is especially fast, memory-efficient, and boasts good performance. XGBoost also allows for a GPU to be used to reduce training time even more.\n",
        "\n",
        "A grid search with cross-validation was originally conducted to determine the best parameters, but that code has been commented out to reduce the total run time of this notebook."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "98xs4o2FrUwc"
      },
      "source": [
        "from sklearn.model_selection import GridSearchCV\n",
        "from xgboost import XGBRFClassifier, XGBClassifier\n",
        "import xgboost\n",
        "\n",
        "# define the model (XGBoost has separate models for classifiers and regressors FYI)\n",
        "xgb_model = XGBClassifier(\n",
        "                          objective='binary:logistic', \n",
        "                          n_estimators=500, \n",
        "                          tree_method='gpu_hist',\n",
        "                          learning_rate=0.05,\n",
        "                          max_depth=25,\n",
        "                          min_child_weight=1,\n",
        "                          colsample_bytree=0.7,\n",
        "                          n_jobs=-1          )\n",
        "\n",
        "# optimization_dict = {\"learning_rate\"    : [0.05], #[0.05, 0.10, 0.15, 0.20, 0.25, 0.30 ] ,  \n",
        "#                      \"max_depth\"        : [5, 10, 15, 20, 25],#[ 3, 4, 5, 6, 8, 10, 12, 15],                # The maximum depth of a tree, same as GBM.\n",
        "#                      \"min_child_weight\" : [1],                              # Defines the minimum sum of weights of all observations required in a child. (Overfitting purposes)\n",
        "#                      \"colsample_bytree\" : [ 0.7]}                     # Denotes the fraction of columns to be randomly samples for each tree.\n",
        "\n",
        "# xgb_model = GridSearchCV(xgb_model, optimization_dict, \n",
        "#                       scoring='accuracy', verbose=1, cv=3)\n",
        "\n",
        "#Fit classifier to Tf-idf training data\n",
        "xgb_model.fit(full_X_train.todense(), y_train.ravel(), eval_metric='logloss', verbose=True)\n",
        "#Create predictions on test set\n",
        "xg_rf_preds = xgb_model.predict(full_X_test.todense())"
      ],
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EgYh6qm5uJcP"
      },
      "source": [
        "#### Evaluate Gradient Boosted Decision Trees w/ Tf-idf Representation\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o2bJYyP0xrSu",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "81ac729d-1f8e-4084-e147-30acdcfbad6f"
      },
      "source": [
        "print(\"Gradient Boosted Decision Trees w/ Tf-idf Representation Accuracy: \" + str(accuracy_score(xg_rf_preds, y_test)))\n",
        "print(\"Gradient Boosted Decision Trees w/ Tf-idf Representation Precision: \" + str(precision_score(xg_rf_preds, y_test)))\n",
        "print(\"Gradient Boosted Decision Trees w/ Tf-idf Representation Recall: \" + str(recall_score(xg_rf_preds, y_test)))"
      ],
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Gradient Boosted Decision Trees w/ Tf-idf Representation Accuracy: 0.7423159056553975\n",
            "Gradient Boosted Decision Trees w/ Tf-idf Representation Precision: 0.758619862274156\n",
            "Gradient Boosted Decision Trees w/ Tf-idf Representation Recall: 0.7346640022306906\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cQd7tvE5uJnO"
      },
      "source": [
        "#### Gradient Boosted Decision Trees w/ Tf-idf Representation - Confusion Matrix\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "adl5dmHAyBeI",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "3a43329d-19cf-4c49-bc4a-02e3bc42a480"
      },
      "source": [
        "cm = confusion_matrix(y_test, xg_rf_preds)\n",
        "cm_display = ConfusionMatrixDisplay(cm, display_labels=[0, 1]).plot()"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwV1Zn/8c+XZhEFWWQREQIG1CBRFKO4xhgX3EaTMS5ZNMaJGjXRxIwxy/zc4sSYGBP3mOioibtoRKMiOkZjxg0UFzAKolEQQUA2RaC7n98fdRoK6O57S7ql+/b3/XrVi1untnMbePqcOlXnUURgZmaZduu7AmZmLYmDoplZjoOimVmOg6KZWY6DoplZTvv1XYG8Hj3bxWabt6gqWQkzXum+vqtgBSytWczy2qVal3Ps/4WNYt78mrL2nfjisnERMXpdrvdJa1ERaLPN23P7fb3XdzWsgDN2Pmx9V8EKeHLuHet8jnnza3hm3MCy9q3qN7XXOl/wE9aigqKZtXwB1FK7vqvRbBwUzayQIFgR5XWfWyMHRTMrzC1FM7MkCGoq+PVgB0UzK6wWB0UzMyAbaKlxUDQzW8UtRTOzJIAVvqdoZpYJwt1nM7OVAmoqNyY6KJpZMdkbLZXLQdHMChI1rNOcEi2apw4zs0KygRaVtTRG0gaSnpH0gqTJks5N5YMlPS1pmqTbJHVM5Z3S+rS0fVDuXD9O5a9K2j9XPjqVTZN0Vjnfz0HRzArJnlNUWUsJy4C9I2I7YAQwWtIo4JfAJRExBHgfOD7tfzzwfiq/JO2HpGHAUcA2wGjgSklVkqqAK4ADgGHA0WnfRjkomllhtaGylsZEZkla7ZCWAPYG7kzlNwB189MdmtZJ278oSan81ohYFhFvANOAndIyLSKmR8Ry4Na0b6McFM2skIItxV6SJuSWE/LnSi26ScAcYDzwOrAgIqrTLjOA/ulzf+BtgLR9IbBJvnyNYxoqb5QHWsyskEDUlN+emhsROzZ4rogaYISk7sDdwNZNUMV14qBoZoWV6hoXFRELJD0K7AJ0l9Q+tQY3B2am3WYCA4AZktoD3YB5ufI6+WMaKm+Qu89mVkgglkdVWUtjJPVOLUQkdQb2BV4BHgUOT7sdC9yTPo9N66Tt/xsRkcqPSqPTg4GhwDPAs8DQNJrdkWwwZmyp7+eWopkVkj283STtqX7ADWmUuB1we0TcJ2kKcKuknwPPA9em/a8F/iRpGjCfLMgREZMl3Q5MAaqBU1K3HEmnAuOAKuC6iJhcqlIOimZWWFM8vB0RLwLb11M+nWzkeM3yj4CvNHCuC4AL6im/H7i/SL0cFM2skAhRE5V7581B0cwKq63g1/wcFM2skGygpXJDR+V+MzNrFk040NIiOSiaWWE1TfycYkvioGhmhRR8o6XVcVA0s8JqPfpsZpbJJoRwUDQzA7Lu84oSr/C1Zg6KZlZIBH5428xsFfnhbTOzOoFbimZmq/FAi5lZEpTOv9KaOSiaWSFZitPKDR2V+83MrJmUlb601arcGwNm1iyC7I2WcpbGSBog6VFJUyRNlnRaKr9N0qS0vJmy/SFpkKSluW1X5841UtJLKen9pSn1KZJ6ShovaWr6s0ep7+eWopkV1kQtxWrgjIh4TlJXYKKk8RFxZN0Oki4mS2Va5/WIGFHPua4Cvg08TTbT9mjgAeAs4JGIuFDSWWn9R41Vyi1FMyskQk3SUoyIWRHxXPq8mCxp1cq8zKm1dwRwS2PnkdQP2DginkqJrG4EDkubDwVuSJ9vyJU3yC1FMyskG2gp+zW/XpIm5NaviYhr1txJ0iCyfC1P54r3AGZHxNRc2WBJzwOLgJ9FxN/JAumM3D75pPd9I2JW+vwu0LdUhR0UzaygQjla5kbEjo2eTeoCjAFOj4hFuU1Hs3orcRYwMCLmSRoJ/EXSNuVWJCJCUpTaz0HRzArJBlqaZvRZUgeygHhTRNyVK28PfBkYufK6EcuAZenzREmvA1uSJbjfPHfafNL72ZL6RcSs1M2eU6pOvqdoZoXV0K6spTHpnuG1wCsR8Zs1Nu8D/DMiZuT2751yRCNpC7Kk99NT93iRpFHpnMcA96TDxgLHps/H5sob5JaimRXShG+07AZ8A3ip7rEb4CcpV/NRrD3AsidwnqQVQC1wUkTMT9tOBq4HOpONOj+Qyi8Ebpd0PPAvsoGbRjkomllhTZG4KiKegPqf7YmIb9ZTNoasq13f/hOA4fWUzwO+WKReDopmVkgErKit3DtvDopmVkjWfXZQNDNbqZLffXZQLMOKj8RlRw6nelk7amvEdgfM44AfvL1O5xx/RX+evr0PqoIvn/0Gn/n8gpXbamvg4kO2pdumyznhun+ua/Ur3ulnT2anPd9jwfyOnPyVXdfavvmgD/j+uZMZsvUibrh8CHf9adA6X7N9h1p+eP7LDPnMIhYv7MAvfrQtc2Z1Xrm996ZLuXrMk9x09RZNcr2WpCkfyWmJmrUNLGm0pFfTS9pnNee1mlP7TsEpN0/mzAdf4D/vf4FXHuvOm891KevYc3fbYa2yd6d25vl7e3HWQ5M46YYp3PlfW1Bbs2r7Y//Tj75DljZV9Svew/duxn+dsvbPuc7ihR24+pdbMebGQYXP3affUi78w4S1yvc/bCZLFrfnPw7dnbtv+hTfOm3qatu/fcZrTPjHJoWv1zo0zWt+LVWz1To9T3QFcAAwDDha0rDmul5zkqDTRrUA1FSL2mqB4O2XNuKyI7bh1wdvy1Xf+AwL53Qo63wvPdST7Q+ZS/tOwSYDltHrU0v516QsyC6Y1ZEp/9uDUUfNbrbvU2lefq4Hixc2/LNf+H5Hpk7pRk312q2bLxw4i0v+9DSX3fokp/50Cu3alXzhAYBRe73Hw/duBsATD/dhu53mk7WhYJe95vDuzM689Xp5vzhbo9qUp6XU0ho1ZyjfCZgWEdMjYjlwK9nL2a1SbQ1cdMB2/Gzk59hy94UMGP4BY84ezHFXvcoP73uRnY+Yw19/NbCscy2c3ZEemy1bud6933IWzu4EwN3nDebffvwv1Dr/PbUqAwYvYc/93uWHx32O7x61C7W1Yq8DZ5U+ENikz0e89+4GANTWtOPDJe3ZuPsKNuhczeHHvcnNv9+iOau+XmWjz1VlLa1Rc95T7A/kb7zNAHZecydJJwAnAPTr33J/iO2q4MwHXuDDhVVcd+LWzJnemVmvbciVX89evYxa2LjPCgAeurw/k/7aC4BFczpy0QHbAbDFjos4/Pw3GrzG5Ed60GWTFQz47AdMfXLjZv5Gtt1O8xkybBG//XM2B0GnTrUsnN8RgJ9dPIm+/ZfSoUPQe9OPuOzWJwEYe/NAxo/t3+A5v3bSdP7y54F8tLRyb9c7HUEzSzNmXAOwzbYdy+u7rEcbdqthyC4LeXFcTzYdupTv3/3SWvvsd+pM9js1e/Xy3N124MwHXlhte7e+y3n/nU4r1xfM6ki3vst4+eGevPxwD6Y8ugPVy9rx0ZIq/nT6UL7x29XvV1nTkOCRezfj+suGrrXt52dkU/b16beUH5w3mbO+vfqcBvPmbEDvTT9i3pwNaFdVy4Zdqlm0oANbDV/I7vvM5lunT2WjrtVELSxf3o77biuvF9FatNaucTmas/s8ExiQW8+/pN2qLJnXng8XZq3Y5R+147UnutP/Mx/wwfz2vDExu29Us0LMeq1zY6dZafi+83n+3l5ULxPz3u7E3Dc786kRSzjkR29x7lMTOfsfz3HMZa8xdNeFDojNaNIzPdltn9l067EcgC4br6BPv/IGuJ5+rDf7HPIOALvvM4cXn+0JiDOP/xzHHbQHxx20B/fcNJDbrh1ccQGxbvS5nKU1as6W4rPAUEmDyYLhUcBXm/F6zWbRnI7cdMYQamtF1IoRB81l+L7v032zZdx17mA+WtSe2hrx+W+9Q78tS/+n6rflUkYcPJdf7Ls97doH/37edNq13DsHLd6Zv3iRbUe+z8bdV3Djg4/z56s/Tfv22cDY/XcOoMcmy/jdTU+z4UbV1IY47GtvceK/78rb07vwpyuG8POrJtJOUF0trrxw69UerWnIuL9sxg9//jJ/vOcJFi/qwC/P+mxzf80WpbWOLJdD2US1zXRy6UDgt0AVcF1EXNDY/tts2zFuv693s9XHmt4ZO5ecyNhakCfn3sHCFXPWqQnXY+s+sfd1h5e17127XTWx1HyKLU2z3lNMs13c35zXMLNPXmvtGpdjvQ+0mFnrUulvtDgomllhDopmZkmlP6dYuUNIZtZsmuI1P0kDJD0qaYqkyZJOS+XnSJqZS3p/YO6YH6e5FF6VtH+uvN55FiQNlvR0Kr9NUsdS381B0cwKiYDq2nZlLSVUA2dExDBgFHBKbn6ESyJiRFruB0jbjgK2IUt2f6WkqhLzLPwynWsI8D5wfKlKOSiaWWFN8fB2RMyKiOfS58XAK6zK11yfQ4FbI2JZRLwBTCObY6HeeRZSEqu9gTvT8TcAJZ8hc1A0s0Lq7imWGRR7SZqQW06o75ySBgHbA0+nolMlvSjpOkk9Ull98yn0b6R8E2BBRFSvUd4oB0UzKyxCZS3A3IjYMbdcs+a5JHUhS0h1ekQsAq4CPg2MAGYBF3+CX82jz2ZWXFNNCCGpA1lAvCki7gKIiNm57X8A7kurjc2nUF/5PKC7pPaptVjW/AtuKZpZIRFNc08x3fO7FnglIn6TK++X2+1LwMvp81jgKEmd0pwKQ4FnyM2zkEaXjwLGRvYO86NA3TuJxwL3lPp+bimaWUGipmlSnO4GfAN4SdKkVPYTstHjEWQvz7wJnAgQEZMl3Q5MIRu5PiUiagAknQqMY9U8C5PT+X4E3Crp58DzZEG4UQ6KZlZYNMHD2xHxBNTbD29wvoQ0qcxaE8s0NM9CREwnG50um4OimRXid5/NzPIiu69YqRwUzaywSk5H4KBoZoVE0w20tEgOimZWmLvPZmY5TTH63FI5KJpZIREOimZmq/EjOWZmOb6naGaWBKLWo89mZqtUcEPRQdHMCvJAi5nZGiq4qeigaGaFtcmWoqTLaOT3QUR8r1lqZGYtWgC1tW0wKAITPrFamFnrEUBbbClGxA35dUkbRsSHzV8lM2vpmuI5RUkDgBuBvmSh9pqI+J2kXwGHAMuB14HjImJByvj3CvBqOsVTEXFSOtdI4HqgM9lks6dFREjqCdwGDCKbxfuIiHi/sXqVfNhI0i6SpgD/TOvbSbqy7G9uZpUnylwaVw2cERHDgFHAKSmJ/XhgeERsC7wG/Dh3zOsRMSItJ+XKrwK+TZa3ZSgwOpWfBTwSEUOBR9J6o8p5AvO3wP5kmbGIiBeAPcs4zswqUnnpTUsNxkTErIh4Ln1eTNYK7B8RD+VyNT9FloWv4dpkia42joinUrKqG1mV9P5QoK7Xe0OuvEFlPZYeEW+vUVRTznFmVqHKbyn2kjQht5xQ3+lS13h74Ok1Nn0LeCC3PljS85Iek7RHKutPlui+Tj7pfd+ImJU+v0vWVW9UOY/kvC1pVyBSjtbTyCK6mbVFAVH+6PPciNixsR0kdSHL/Xx6RCzKlf+UrIt9UyqaBQyMiHnpHuJfJG1TdrWze4wlO/XltBRPAk4hi7zvACPSupm1WSpzKXGWrKE1BrgpIu7KlX8TOBj4WuoSExHLIqLuNt5EskGYLckS3Oe72Pmk97Pr8kinP+eUqlPJoBgRcyPiaxHRNyJ6R8TX6ypmZm1UEwy0SBJZHuZXIuI3ufLRwJnAv+WfeJHUW1JV+rwF2YDK9NQ9XiRpVDrnMaxKej8WODZ9PjZX3qByRp+3kHSvpPckzZF0T6qQmbVVTTP6vBvwDWBvSZPSciBwOdAVGJ/Krk777wm8KGkScCdwUkTMT9tOBv4ITCNrQdbdh7wQ2FfSVGCftN6ocu4p3gxcAXwprR8F3ALsXMaxZlZpmujh7Yh4gvr72GsltU/7jyHrate3bQIwvJ7yecAXi9SrnHuKG0bEnyKiOi1/BjYochEzqywR5S2tUWPvPvdMHx+QdBZwK9nviCNpIJKbWRvRRt99nkgWBOu+/Ym5bcHqT5mbWRtS+sGW1quxd58Hf5IVMbNWorxBlFarrPkUJQ0HhpG7lxgRNzZXpcysJVPbnCWnjqSzgb3IguL9wAHAE2TvF5pZW1TBLcVyRp8PJxvSfjcijgO2A7o1a63MrGWrLXNphcrpPi+NiFpJ1ZI2JntNZkAz18vMWqq2OslszgRJ3YE/kI1ILwGebNZamVmL1iZHn+tExMnp49WSHiSbt+zF5q2WmbVobTEoStqhsW11k0OamVWSxlqKFzeyLYC9m7guvP1SF04ftGtTn9aa0bh3HlrfVbACdtp/UemdytAmu88R8YVPsiJm1koEbfY1PzOz+rXFlqKZWUPaZPfZzKxBFRwUy5l5W5K+Lun/pfWBknZq/qqZWYvVNOkIBkh6VNIUSZMlnZbKe0oaL2lq+rNHKpekSyVNk/Ri/gkZScem/adKOjZXPlLSS+mYS1O6gkaV85rflcAuwNFpfTHZTNxm1gYpyl9KqAbOiIhhwCjgFEnDaDiB/QGsSnZ/AnAVrJz79WyybAA7AWfXBdK0z7dzx40uValyguLOEXEK8BFARLwPdCzjODOrVLUqb2lERMyqe945IhaTpU7uT8MJ7A8FbozMU0D3lKFvf2B8RMxP8Wk8MDpt2zginkoZAW/MnatB5dxTXJEyaAVkGbVota96m1lTaOqBFkmDgO2Bp2k4gX1/4O3cYXVJ7xsrn1FPeaPKaSleCtwN9JF0Adm0Yf9dxnFmVqnKv6fYS9KE3HLCmqeS1IUsIdXpEbHa0+WphfeJDuuU8+7zTZImkk0fJuCwiHil2WtmZi1TefcL68yNiB0b2iipA1lAvCki7krFsyX1i4hZaySwn8nqM3TVJb2fSTbna778b6l883r2b1Q5o88DgQ+Be8kSS3+QysysrWqa0WcB1wKvRMRvcpsaSmA/FjgmjUKPAhambvY4YD9JPdIAy37AuLRtkaRR6VrH5M7VoHLuKf6VVQmsNgAGA68C25RxrJlVIDXNqMJuwDeAl1KCe4CfkCWsv13S8cC/gCPStvuBA8kS3n8IHAcQEfMlnQ88m/Y7LyLmp88nA9cDnYEH0tKocrrPn82vp2eDTm5gdzOzskTEE6zKFrqmtRLYp/uLpzRwruuA6+opnwAML1Kvwm+0RMRzknYuepyZVZAKfqOlnMRVP8ittgN2AN5pthqZWctWbKCl1Smnpdg197ma7B7jmOapjpm1Cm01KKaHtrtGxA8/ofqYWWvQFoOipPYRUS1pt0+yQmbWsokmG31ukRprKT5Ddv9wkqSxwB3AB3Ubcw9amllb4nuKbADMI8vJUve8YgAOimZtVRsNin3SyPPLrAqGdSr4R2JmJVVwBGgsKFYBXaj/4coK/pGYWSlttfs8KyLO+8RqYmatRxsNipWbw9DMPr5ou6PPa717aGYGtM2WYm6WCTOz1bTVe4pmZvVzUDQzSz7xBAGfLAdFMytEuPtsZraaSg6K5WTzMzNbXRPkaAGQdJ2kOZJezpXdJmlSWt6sS1UgaZCkpbltV+eOGSnpJUnTJF2acrIgqaek8ZKmpj97lKqTg6KZFddEQZEsf8ro1U4dcWREjIiIEWRzt+bnWXi9bltEnJQrvwr4NjA0LXXnPAt4JCKGAo+k9UY5KJpZMWmWnHKWkqeKeByo9/G/1No7ArilsXOkNKgbR8RTKY/LjcBhafOhwA3p8w258gY5KJpZceW3FHtJmpBbTihwlT2A2RExNVc2WNLzkh6TtEcq6w/MyO0zI5UB9E2pTgHeBfqWuqgHWsyssAKv+c2NiB0/5mWOZvVW4ixgYETMkzQS+IukslMtR0RIpduvDopmVlhzjz5Lag98GRhZVxYRy4Bl6fNESa8DWwIzgc1zh2+eygBmS+oXEbNSN3tOqWu7+2xmxZTbdV63wLkP8M+IWNktltQ75Y1C0hZkAyrTU/d4kaRR6T7kMcA96bCxwLHp87G58gY5KJpZcU33SM4twJPAVpJmSDo+bTqKtQdY9gReTI/o3AmclJuj4WTgj8A04HXggVR+IbCvpKlkgfbCUnVy99nMCmnKN1oi4ugGyr9ZT9kYGkivHBETgOH1lM+j4IxfDopmVphqK/eVFgdFMyvGE0KYma2ukt99dlA0s+IcFM3MVnFL0cwsz0HRzCxpw9n8zMzW4pm3zczWFJUbFR0UzawwtxTbuB/85i123mcxC+a258S9t1rn8+3zlfl89bTZANz8u748fEfP1bafc/0b9Bu4vEmu1RYs/0ic8eUhrFjejppq2OOghRzzn++uts+Y3/fmwZs3oap90G2Tan7wm7fou/mKdbruover+O+TBjF7Rkf6br6cn/7+Tbp2r1m5/dVJnTn9kC35yVVvssfBC9fpWi1KhT+83WwTQtSXe6G1eui2nvz0a4MLH3fRndPou/ny1cq6dq/m6z+YzWkHD+V7Bw3l6z+YTZdu1Su373bAAj76wPN0FNGhU3DRHa9z9cOvctX4V5nwt668MnHD1fb59PClXPbAq1z9yKvsftAC/nj+ZmWf/4X/68KvTx+4Vvntl/dh+90X8z//eIXtd1/MbZf3WbmtpgauvWAzRn5+8cf/Yi2YastbWqPm/N93PWvkXmitXn66C4vfX71R3e9Ty7jgpulc/uBrXHz3NAYM+aisc43cazHPPd6FxQvas2Rhe557vAs7fiH7j7PBhjV8+cT3uPm3JScHthwJOm+U/Q+sXiFqVogsbdEqI3ZbwgYbZs2bz+zwIXNndVi57Y4re/PdA7bkpC9uxY2/2rTs6z45rhv7HJFN0rLPEfN58sFuK7fdc11vdj9wId17VTd0eKvmoPgxNJZ7oRKcdtEMrvhZf04dvSXXnNePU/97ZumDgF6bruC9dzquXJ87qyO9Ns26ccee+S5jru7DsqVuKRZVUwPf2Wcrjtx2ONvvuZitd/iwwX0fvKUnn9s7+0U08W9dmflGJy69/zWuHP8qU1/qzEtPbVTWNd+f24FN+mZBr2efat6fmwXaubM68H8PdOPgY+eu47dqoYJsoKWcpRVa7/cUU86GEwA2YMMSe7cMG2xYw7AdP+Bn17y5sqxDx+wfwH5Hzuew/3gPgM0GLef8P0+neoV4962OnHd8w13wLbZZSr9By/n9Od3W6nJbaVVVcNXDr7JkYRXnHj+IN/+5AYO2Xrv1/siYHkx9cUN+NWYaABMf68pzj23Myftm92+XftiOmdM78dlRH/C9g4ayYlk7ln7YjsULqvjOPtk+x//sHXbca/VusQR1M91ffXZ/jv/pO7Sr4N9tHmhpRhFxDXANwMbq2Sp+1O3awZJFVSv/I+U9dFtPHrotGzi56M5pXHz6QGbPyLUM3+3AtrssWbneq99yXnyyC8NGfsCW237IDU9PoaoKuveq5qI7p3Hm4UOa/wtVkC7dathu1yU8+2jXtYLic4934Zbf9eXXd02jY6fsn1oAR353Ngd9Y95a57r0r1m+pBf+rwvjb+/JD3/71mrbe/RawbzZ7dmkbzXzZren+yZZq/G1Fzrzi+8MAmDh/CqeeaQrVVWw6wEVNthSoSr4d1nz+XBJFbPf7sgeBy9IJcEWw5aWdezEv3Vl5OeX0KVbNV26VTPy80uY+Leu3HdjL766wzYcu/MwzjhsCDOnd3JALNOCeVUsWVgFwLKl4rnHuzJgyLLV9pn2Umcu/dEAzr1++mr3+Xb8/GLG3dqTpWlwa+6sDiyYW15bYdR+i3j49uwX4MO392SX/bOgd+PTr3DjM1O48Zkp7HHwQr77ixkVFRDrHt5uihSn9Q3ISjpH0sxc0vsDc9t+nBLevypp/1z56FQ2TdJZufLBkp5O5bdJWtVCacB6bym2Bmdd+S+23WUJ3XpW8+cJU/jTxX258JSBfO/CmXz1tNlUdQgeu6c706d0LnmuxQvac9Nv+3DZ/Vkr5KZL+rJ4gf8a1sX82R349WkDqa0VtbWw5yELGLXvIm64aFO23O5Ddtl/EX84fzOWftCOn5+Q3cLo0385597wBiP3Wsxb0zpx+iFDgWzA5szL/kX3XqWve+Sps7ngpEE8eOsm9OmfPZLTJkQ05SSz1wOXk+VqzrskIn6dL5A0jCxNwTbAZsDDkrZMm68A9iVLb/qspLERMQX4ZTrXrZKuBo4HrmqsQopmuhmaci/sBfQCZgNnR8S1jR2zsXrGzio0c7itZ+PembS+q2AF7LT/20x44SOV3rNhXbtvHtvveVpZ+/793jMnlkpxKmkQcF9EDE/r5wBL6gmKPwaIiF+k9XHAOWnzORGxf34/snws7wGbRkS1pF3y+zWk2ZooDeVeMLPWr8BASy9JE3Lr16RxhFJOlXQMMAE4IyLeJ0tw/1Run3zS+7fXKN8Z2ARYEBHV9ezfIPfbzKyYAMrvPs8t1VKsx1XA+elK5wMXA98qeI6PzUHRzIprxtHniJhd91nSH4D70upMYEBu13zS+/rK5wHdJbVPrcX8/g3y6LOZFdZUo8/1nlvql1v9ElA3Mj0WOEpSJ0mDgaHAM8CzwNA00tyRbDBmbGQDJo8Ch6fjjwXuKXV9txTNrLCmGn3OD8hKmgGcDewlaQRZe/RN4ESAiJgs6XZgClANnBIRNek8pwLjgCrguoiYnC7xI+BWST8HngcaHewFB0UzK6oJZ8lpYEC2wcAVERcAF9RTfj9wfz3l04GditTJQdHMCske3q7cV1ocFM2suFY6A045HBTNrDC3FM3M6lT4zNsOimZWUJO++9ziOCiaWXHuPpuZJdF6Uw2Uw0HRzIpzS9HMLKdyY6KDopkVp9rK7T87KJpZMYEf3jYzqyPCD2+bma3GQdHMLMdB0cws8T1FM7PVVfLos9MRmFlBkXWfy1lKkHSdpDmSXs6V/UrSPyW9KOluSd1T+SBJSyVNSsvVuWNGSnopJb2/VJJSeU9J4yVNTX/2KFUnB0UzKyZosqAIXA+MXqNsPDA8IrYFXgN+nNv2ekSMSMtJufKrgG+T5W0ZmjvnWcAjETEUeCStN8pB0cyKqy1zKSEiHgfmr1H2UC5X81NkWfgalA/2EMEAAAYiSURBVBJdbRwRT6VkVTcCh6XNhwI3pM835Mob5KBoZoUpoqyFLCHVhNxyQsFLfQt4ILc+WNLzkh6TtEcq60+W6L5OPul934iYlT6/C/QtdUEPtJhZceU/kjM3Inb8OJeQ9FOyrH03paJZwMCImCdpJPAXSduUe76ICKl04lUHRTMrJgJqmnf0WdI3gYOBL6YuMRGxDFiWPk+U9DqwJVmC+3wXO5/0frakfhExK3Wz55S6trvPZlZc0w20rEXSaOBM4N8i4sNceW9JVenzFmQDKtNT93iRpFFp1PkYViW9Hwscmz4fmytvkFuKZlZcE73RIukWYC+ye48zgLPJRps7AePTkzVPpZHmPYHzJK0gG8Y5KSLqBmlOJhvJ7kx2D7LuPuSFwO2Sjgf+BRxRqk4OimZWTABNlKMlIo6up/jaBvYdA4xpYNsEYHg95fOALxapk4OimRUUEJX7RouDopkVEzT7QMv65KBoZsV5lhwzsxwHRTOzOh//cZvWwEHRzIoJoIKnDnNQNLPi3FI0M6vT/K/5rU8OimZWTED4OUUzs5wmeqOlJXJQNLPifE/RzCyJ8Oizmdlq3FI0M6sTRE3N+q5Es3FQNLNimnDqsJbIQdHMivMjOWZmmQDCLUUzsyQ8yayZ2WoqeaBF0YKG1iW9R5ZcptL0Auau70pYIZX6d/apiOi9LieQ9CDZz6cccyNi9Lpc75PWooJipZI04eMmBLf1w39nbZfzPpuZ5TgompnlOCh+Mq5Z3xWwwvx31kb5nqKZWY5bimZmOQ6KZmY5DorNSNJoSa9KmibprPVdHytN0nWS5kh6eX3XxdYPB8VmIqkKuAI4ABgGHC1p2PqtlZXheqBVPWxsTctBsfnsBEyLiOkRsRy4FTh0PdfJSoiIx4H567setv44KDaf/sDbufUZqczMWjAHRTOzHAfF5jMTGJBb3zyVmVkL5qDYfJ4FhkoaLKkjcBQwdj3XycxKcFBsJhFRDZwKjANeAW6PiMnrt1ZWiqRbgCeBrSTNkHT8+q6TfbL8mp+ZWY5bimZmOQ6KZmY5DopmZjkOimZmOQ6KZmY5DoqtiKQaSZMkvSzpDkkbrsO5rpd0ePr8x8Ymq5C0l6RdP8Y13pS0Vta3hsrX2GdJwWudI+mHRetotiYHxdZlaUSMiIjhwHLgpPxGSR8rj3dE/EdETGlkl72AwkHRrDVyUGy9/g4MSa24v0saC0yRVCXpV5KelfSipBMBlLk8ze/4MNCn7kSS/iZpx/R5tKTnJL0g6RFJg8iC7/dTK3UPSb0ljUnXeFbSbunYTSQ9JGmypD8CKvUlJP1F0sR0zAlrbLsklT8iqXcq+7SkB9Mxf5e0dVP8MM3qfKyWha1fqUV4APBgKtoBGB4Rb6TAsjAiPiepE/APSQ8B2wNbkc3t2BeYAly3xnl7A38A9kzn6hkR8yVdDSyJiF+n/W4GLomIJyQNJHtr5zPA2cATEXGepIOAct4G+Va6RmfgWUljImIesBEwISK+L+n/pXOfSpZQ6qSImCppZ+BKYO+P8WM0q5eDYuvSWdKk9PnvwLVk3dpnIuKNVL4fsG3d/UKgGzAU2BO4JSJqgHck/W895x8FPF53rohoaF7BfYBh0sqG4MaSuqRrfDkd+1dJ75fxnb4n6Uvp84BU13lALXBbKv8zcFe6xq7AHblrdyrjGmZlc1BsXZZGxIh8QQoOH+SLgO9GxLg19juwCevRDhgVER/VU5eySdqLLMDuEhEfSvobsEEDu0e67oI1fwZmTcn3FCvPOOA7kjoASNpS0kbA48CR6Z5jP+AL9Rz7FLCnpMHp2J6pfDHQNbffQ8B361Yk1QWpx4GvprIDgB4l6toNeD8FxK3JWqp12gF1rd2vknXLFwFvSPpKuoYkbVfiGmaFOChWnj+S3S98LiVf+j1Zj+BuYGradiPZTDCriYj3gBPIuqovsKr7ei/wpbqBFuB7wI5pIGcKq0bBzyULqpPJutFvlajrg0B7Sa8AF5IF5TofADul77A3cF4q/xpwfKrfZJziwZqYZ8kxM8txS9HMLMdB0cwsx0HRzCzHQdHMLMdB0cwsx0HRzCzHQdHMLOf/AysEeXgQevdAAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zUgMbtHhuJ2T"
      },
      "source": [
        "#### Gradient Boosted Decision Trees w/ Tf-idf Representation - Precision Recall Curve\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nZUi7UIdxr8U",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 409
        },
        "outputId": "fee21ddb-c4a9-4c5a-c58e-bb96cbae1d4b"
      },
      "source": [
        "create_pr_curve('Gradient Boosted Decision Trees (Tf-idf) Precision-Recall Curve', xg_rf_preds, y_test)"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAGICAYAAACN5q1DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3gVdfb48fdJhfQKuYRepCWAUsS2oqBLgsoqflVQkLLq6rrrurpYdkXFsqus7upPbKsUe8NusIK9AYqUAIIUQ08CCSmQ+vn9MZNwE1Lh5s69yXk9T57cOzN35kw9M2eaGGNQSimlPCnA6QCUUkq1PppclFJKeZwmF6WUUh6nyUUppZTHaXJRSinlcZpclFJKeVyrSi4islVExtifbxWRp5yOyZeJiBGR3g4M93ERua0J3a0VkVFeCMlRIpIoIutFpH0Tu79bRHJEZHcd7S4VkQ8b+O2nIvL7uvolIh1FZJ2IhB7dmHiGiHQVkUIRCWykuwbH1d+IyAIRudv+PEpEtjsd07HwWnIRkUtE5DsRKRKRvfbna0REWmJ4xph7jTG/b7zLholId3sjHNRAN3eISJm9QhTaK+iEYx12I3FNFZEvW7D/n4rIIREpEJEDIrJCRG72xIbHGPMHY8xdTehuoDHm02Mdnjs7YVXNpwp7HKu+3+rJYTXDzcACY8zBxuITka7ADcAAY0xS7R4ZY543xpzdlIHW7pcxZg+wFLiygd8sEJFSO559IvKRiPQ7qrGuhzHmV2NMhDGmopHumjyuzeW2/Bfayfd1EXG1xLCOhlj+LCJr7G3qdhF5VURSnY6tileSi4jcADwEzAGSgI7AH4BTgJB6ftPgXosPetleISKAvwDPiUhHp4M6RtcaYyIBF9ZG6BIgo6V2CLzBTlhV8+kLrHGMsP/urequoZ0JT7KT9eXAc02MryuQa4zZ64HB19Wv54GrGvnd/XZ8nYG9wILaHdgbP3+vjFxrj2dvIAL4t8PxuHsIuA74MxAHHAe8CYxrbo9aallv8ZkvItHAbOAaY8xrxpgCY/nRGHOpMabE7m6BiDwmIhkiUgScISLjRORHe885S0TuqNXvySKyTURyReTvtdrdISLPuX0fKSJfi0ieiPzkXm6x91LuEpGv7D31D0UkwW79uf0/z96LOamxcTbGfAAUAL3chnGFiGyy9/beFpFObu1OFpFlIpJv/z/Zrd1UEdlsx7VFrFJAf+Bx4CQ7pjy721AR+beI/Coie8QqP7V369ffRGSXiOwUkemNjYfb+BTZRxDnASdhL8AiEmAfzfxiz4NXRCTObXinuk3zLBGZajd3P/xPEJF37W72icgXVRslqVnmDBWR/9qx77Q/h9rtRtl7bjeIdVS8S0SmNXX87H5UHaHOEJFfgSV28+liHYnuF5EPRKSb22/6ibXnvk9ENojIRW7t0kUk055vO0TkxnoGfSKQZ4xptARiT4uPgE72fF9QRzc1jmhF5CyxSm75IvIIII306zugp/t41scYUwy8AKTY/fxURO4Rka+AYrs/DU2j9iLygL0O54vIl3azGtWCutaBesa1ofWooXW8sfHMw9pwD3HrX7PHy273qlglyHwR+VxEBjYlBnci0gf4IzDRGLPEGFNijCm2j+T+5Ta+7uXP2tPKiMgfRWQjsFGsbe+/aw3nLRH5q/25k4gsEpFsex78uSkTrkX/gLFAORDUSHcLgHyso5kAoB0wCki1vw8C9gC/s7sfABQCvwFCgQft4Yyx298BPGd/TgZygXS7X2fZ3xPt9p8Cv2Bl//b293/Z7boDpqH4aw1LsDa+eUCM3exMIAc4wY71/wGf2+3igP3AZCAImGh/jwfCgQNAX7tbFzDQ/jwV+LJWHP8B3rb7GQm8A/zTbT7swdoQhGNtFAzQu55x+hT4fR3NPwfusz9fB3yLtQcbCjwBvGi364aVYCcCwfb4DHGb13fbn/+JlSiD7b/TALHbbXWbn7PtYXUAEoGvgbvsdqPseT/b7kc61sYttpFlrnoc3ebzM/b0aQ+MBzYB/e158w/ga7v7cCALmGa3O96exwPs9ruA0+zPscAJ9cTwR+C9ps4De1y3NzBO1csFkGDPgwvt6XK9PZ1+31C/gFXAeQ2sp1XzLsJejr5wi/dXYKA9TaIbmUZz7d8kA4HAyVjLUdW8CKKJ6wANrEeNreNNWDbigY+Bt5o47+scL7vddKx1MxT4L7Cynmlb73zGqvpsa+qyXdf2wp6+H9nTrT3WdjSLw+teLHAQ6IS1zVwBzMKqNPUENgO/bSgGbxy2JgA5xpjyqgZue7MHReQ3bt2+ZYz5yhhTaYw5ZIz51Biz2v6+CngRON3u9kLgXWPM58Y6+rkNqKwnhsuADGNMht2vj4DlWBuhKvONMT8bYw4Cr+C2l9JEF4l1BFGItYG/11h7PACXAvOMMT/Ysd6CddTRHSsRbTTGPGuMKTfGvAisB861f1sJpIhIe2PMLmPM2roGLiKCVSu/3hizzxhTANyLVcoCuMgexzXGmCKshHg0dmItkGAt5H83xmy3x+sO4EJ7j3MS8LEx5kVjTJkxJtcYs7KO/pVhbTC62d19Yeylu5ZLgdnGmL3GmGzgTqwNiXt/Ztv9yMCaD32PYvzuMNaR2kF7/P5pjFlnL7/3AkPsvfpzgK3GmPn2fPsRWAT8n1s8A0Qkyhiz3xjzQz3Di8FKAC0hHVhrrIpBGdbG7IiLAOpQYMdVnxvtZX0TVoKZ6tZugTFmrT29xlLPNBLr6HQ6cJ0xZocxpsIY87W9HNXWlHWgsfUImr+OPywi+ViJIwH4k9283nnf2HgZY+YZq3pTtb4MFqu60xzxWDsvx+qf9rbiIFYJ1mDt3IG1ff3GGLMTGI61Iz7bGFNqjNkM/I/D25Y6eSO55AIJ4lbXM8acbIyJsdu5x5Dl/kMROVFEltqHYvlYK3vVoWwn9+7tDWZuPTF0w5rxeVV/wKlYG7Uq7itdMdZK0xyvGGNijDHhWOWwKSJSVbvuBGxzi7XQjjW5djvbNiDZHqeLscZ7l4i8J/WfPE0EwoAVbuP4vt28Kgb36Vt7mE2VDOyzP3cD3nAb3jqgAuucWhesPcXGzMHaSH1olz5urqe72tNpm92sSq77DgxHNw+h5jTqBjzkNn77sI5Mk+12J9Zapi7FOqcIMAFr475NRD6T+sup+7H2ZJtNRE6Twyf769rg1l5HTK3xq08k1pF3ff5tL+tJxpjzjDHu87n29KtvGiVgVScaXEaasQ7Uux65fa9zHRerfFzXRR1/NsZEY1VNYrGO0I96vEQkUET+JVYZ+QDWkTkc3qY1VS41t11Hq/ay8RLWER9YO4fP25+7YZVP3cf3Vqz1vF7eSC7fACVYJYbG1N5jfQHrKKCLPZMfx64ZY2XuLlUdikgYVkavSxbwrL1CVP2FG7s+2cyYGv+BMVuBxRzea9qJNYOqYg23Y91Ru52tq90OY8wHxpizsBam9Vh7DHXFlYN1GDvQbRyjjXVCEmpNL3sYzSIiXYChWHs5YE3XtFrTtZ0xZofdrld9/api78XdYIzpiXVO568iMrqOTmtPp652M09zn65ZwFW1xq+9MeZru91ntdpFGGOutsdrmTFmPFYZ702sPeW6rMIq1TQ/UOsor+pkf121+9rriFBzGTiCvRPYG/jpaGLiyOlX3zTKAQ7RtGWkvnXAXYPrUSP9/4Op46IOt/argbuBufY0PNrxmoS1HRyDVTLsbjdv7gUynwCdRWRYA90UYe1sVjniykKO3Ia8iFV56IZ1LnCR3TwL2FJrfCONMek0oMWTi7FKQ3cCj4rIhSISKdaJ4CFYtcuGRAL7jDGHRGQE1syp8hpwjlgnjUOw6u31jc9zwLki8lt776GdWCeBO9fTvbtsrMPynk3oFgC7v2OBqr3JF4FpIjJErJPQ9wLf2UkoAzhORCaJSJCIXIx1Puldse47GG8noxKsUk9V6W8P1gIWAmCMqcRa6f4jIh3sOJJF5Ld2968AU0VkgJ2Ib2/G+ISJyOnAW8D3dsxgJft77IWx6n6Nqp2I54ExInKRPV7x9jyv3e9zRKS3vdLmYx351FXefBH4hz2MBKz673N1dOdJjwO3iH3SVUSiRaSq7PUu1nybLCLB9t9wEekvIiFiXXgRbZejDtQzTmBNzxgRSa6n/bF4DxgoIhfYSePP1L2RcTcCq+RztEe27uqdRvbyOg94UKyTxYEicpLUutS9kXXAXb3rkQfGA2Ah1p76eccwXpH2OORibfiPSGRNYYzZCDwKvGhvx0LsbdolcvjIfyVwgb3u9gZmNKG/P2Ilx6eAD8zhsv73QIGI3CTWxQqBIpIiIsMb6p9XLhU0xtwP/BWYibVR3IN18vcmrBOz9bkGmC0iBVgbk+q9P7vu+keso5tdWOWFOq+4McZkYe0x3IqVLLKAv9GE8TfWFTH3AF/Zh4Qj6+n04qpDa2AZ8BVWUsUY8zHWOaFFdqy9sOuVxphcrBruDVgL3UzgHGNMjh3fX7H2yvZhnW+62h7eEqzktVtEcuxmN2GVmL61D7s/xj7vYIxZjFVzX2J3s6SxcQcesaf9Hvu3i4Cx9goE1uWQb2OVtAqwTrifaA/vV6yy0A127CuBwXUMo48dZyHWUe6jxpildXR3N9Z5slXAauAHu1mLMca8AdwHvGRPzzVAmt2uADgbaz7uxCq53Id1ohas80Fb7d/9AatsUtcwSrFO5F7WAvHnYJ0D+hfWstUHa7lsyKVYSdUTw29sGt2INS+XYS0j93HkOtnQOuA+rIbWI0+MSynW8n7bMYzXM1iluh1AJtb6crT+DDyCdfFAHlYZ7nysi3jAurinFGvdXcjhEldjXsA6snqhqoGx7jc6B+sc1RYOJ6AGzxVVXRmglHKIiCRilRqPt0+uOhVHB+AzO45DTsWhWgdNLkoppTzO3++gVUop5YM0uSillPI4TS5KKaU8zqvJRUTmifXspzX1tBcReVisZ3CtEpETvBmfUkopz/DKk1/dLMC6fO6ZetqnYV0u2QfrktbH7P8NSkhIMN27d/dMhEo1V34+bNpkfRaBvn0hvI5buIqKYMMGqLqIpndviG7ukz+UapoVK1bkGGMSG++yZXg1uRhjPhfreVr1GQ88Yz+K4FsRiRERlzGmwefodO/eneXLl3swUqWaYdy4w8nlhhtgzhwADpZW0C44AHF/Q8GNN8IDD1if+/aFdz11j59SNYmIJ26EPWq+ds4lmZrPJtpOzWcDKeV7vnK7L3GGdSN0/sEyLnjsa+Yu3VSz2xkz6v6dUq2MryWXJhORK0VkuYgsz87Odjoc1ZYVuD3UuJf1SKnI0CD6J0Xy7w9/5oXvfj2iPQAHDngpQKW8z9eSyw5qPlivM/U8eM4Y86QxZpgxZlhiomNlRaUg0u2hxr9YD8MNCBDuu3AQZ/RN5B9vrmbx6l012gMQFeXFIJXyLl9LLm9jPape7Gd45Td2vkUpx51yyuHPTz9d/TE4MIBHLx3K8V1jue6llXz9S06N9jV+p1Qr4+1LkV/EejhhX7FeSztDRP4gIn+wO8nAesPZJqwn/F7jzfiUOipXuz1H8ZFH4Jtvqr+2Dwnk6cuH0T0hjLn3PEvlI4/U/TulWplW8WyxYcOGGb1aTDmmogIGDYLMTOt7aChce6118r5XL/jlF4oefYKgxx8ltLzM6mbAAFi1CgIDnYtbtWoissIY09A7X1p2+JpclPKADRusMldufS9DPSwvLIqKL74k/oRULwSm2iqnk4uvnXNRyj/17WtdWjxgQIOdHerTj4lTH+DSz/eTf7DMS8Ep5X2aXJTylL59rVLXu+9aN1bGxEBAgPV/3Dh4913arVvDrX8Zzy/ZhVyxcDmHyiqcjlqpFqFlMaUc8O6qnfzpxR8Z078jj116AkGBup+nPEvLYkq1QecM6sSd5w3ko8w93PrGalrDTp5S7rz94EqllG3KSd3JKSzl4U82Eh8Ryk1j+zkdklIeo8lFKQddP6YPuYUlPPbpL8SHh/D703o6HZJSHqHJRSkHiQizx6ewv7iUu99bR1x4CBec0NnpsJQ6ZnrORSmHBQYI/7l4CCf3imfma6tYun6v0yEpdcw0uSjlA0KDAnlyyjD6uSK5+vkVrNi23+mQlDommlyU8hERoUEsmDaCpKh2TF+wjJ/3FDT+I6V8lCYXpXxIQkQoz844kdCgAKY8/T3b9xc7HZJSR0WTi1I+pktcGM/MGEFxaTlT5n3PvqJSp0NSqtk0uSjlg/olRfH01OHs2H+QafO/p6ik3OmQlGoWTS5K+ajh3eOYO+kE1uw8wB+eW0FpeaXTISnVZJpclPJhYwZ05F8XpPLFxhz++spKKiv1MTHKP+hNlEr5uP8b1oV9RaX8c/F64sNDuOO8gYiI02Ep1SBNLkr5gatO70VuUSlPfr6Z+IhQ/jy6j9MhKdUgTS5K+Ymbx/Yjp7CEBz/6mfiIEC49sZvTISlVL00uSvmJgADhvgmDyCsu4x9vriE2LIT0VJfTYSlVJz2hr5QfCQ4MYO6kEzihayx/eWklX2/KcTokpeqkyUUpP9M+JJB5lw+nR0I4VzyznDU78p0OSakjaHJRyg9FhwWzcPoIYsJCuHze92zJKXI6JKVq0OSilJ9Kim7HszNGYIDJT3/HngOHnA5JqWqaXJTyYz0TI1gwbTj7i0q5fN735B8sczokpQBNLkr5vUGdY3hi8jB+yS7k9wuXcaiswumQlNLkolRrcGqfBP5z8RCWb9vPtS/8QHmFPodMOUuTi1KtxDmDOjH7vIF8vG4vt7y+GmP0OWTKOXoTpVKtyOSTupNTWMpDn2wkPiKUm9P6OR2SaqM0uSjVyvxlTB9yi0p4/LNfiA8P4Yrf9HQ6JNUGaXJRqpUREe48L4X9RWXck7GOuPAQJgzt7HRYqo3R5KJUKxQYIDx48WDyDpYyc9EqYsODObNfR6fDUm2IntBXqpUKDQrkicnDGOCK4prnf2DFtn1Oh6TaEE0uSrViEaFBzJ82HFd0e6bNX8aG3QVOh6TaCE0uSrVyCRGhPDN9BO2CA5ky7zu27y92OiTVBmhyUaoN6BIXxjMzRnCwtIIpT39PbmGJ0yGpVk6Ti1JtRL+kKOZNHc6OvINMW7CMwpJyp0NSrZgmF6XakGHd43j00hNYu/MAf3h2BSXl+hwy1TI0uSjVxozu35H7Jgziy0053PDKT1RU6mNilOfpfS5KtUEXDu3MvqIS7s1YT1x4CHeeNxARcTos1YpoclGqjbryN73ILSzlic83Ex8eynVj+jgdkmpFNLko1YbdnNaP3KJS/vPxz8RHhHDZyG5Oh6RaCU0uSrVhIsK/Lkhlf1Ept721hrjwENJTXU6HpVoBPaGvVBsXFBjAI5NOYGjXWP7y0kq+2pTjdEiqFdDkopSifUggT18+nB4J4Vz5zHJWb893OiTl5zS5KKUAiA4L5pkZI4gJC2Hq/O/ZnF3odEjKj2lyUUpV6xjVjmdnjMAAk5/+nj0HDjkdkvJTmlyUUjX0TIxgwbTh5BWXMuXp78kvLnM6JOWHNLkopY4wqHMMT04ZxpacImYsXMbBUn1MjGoeTS5KqTqd0juB/1w8hBW/7ufaF36grKLS6ZCUH9HkopSq17hBLmaPT+GT9Xu5edFqjNHnkKmm0ZsolVINmjyyG7mFJfz3440kRIRwS3p/p0NSfkCTi1KqUdeN7nP4OWQRIVz5m15Oh6R8nCYXpVSjRIQ7zhvIvuJS+0nKoVw4tLPTYSkfpslFKdUkgQHCgxcNJr+4jJsWrSI2LJjR/Ts6HZbyUXpCXynVZKFBgTw+eSgDO0VxzfM/sHzrPqdDUj7K68lFRMaKyAYR2SQiN9fRvquILBWRH0VklYikeztGpVT9IkKDmD91OMkx7Zm+YBkbdhc4HZLyQV5NLiISCMwF0oABwEQRGVCrs38ArxhjjgcuAR71ZoxKqcbFR4SycPoI2ocEMmXed2TtK3Y6JOVjvH3kMgLYZIzZbIwpBV4CxtfqxgBR9udoYKcX41NKNVGXuDCemX4iB0srmDLve3IKS5wOSfkQbyeXZCDL7ft2u5m7O4DLRGQ7kAH8qa4eiciVIrJcRJZnZ2e3RKxKqUb0TYpk3tTh7Mo/yLT5yygsKXc6JOUjfPGE/kRggTGmM5AOPCsiR8RpjHnSGDPMGDMsMTHR60EqpSzDusfx6KUnkLnrAFc9u5yScn0OmfJ+ctkBdHH73tlu5m4G8AqAMeYboB2Q4JXolFJH5cx+Hbl/wiC+2pTLX1/+iYpKfUxMW+ft5LIM6CMiPUQkBOuE/du1uvkVGA0gIv2xkovWvZTycROGdubv6f15b/Uu7nh7rT6HrI3z6k2UxphyEbkW+AAIBOYZY9aKyGxguTHmbeAG4H8icj3Wyf2pRpdSpfzCFb/pSU5RCU98Zj0m5i9jjnM6JOUQr9+hb4zJwDpR795sltvnTOAUb8ellPKMm8f2Y19hKf/9eCPx4SFMPqm70yEpB+jjX5RSHiUi/POCVPYXlzLr7bXEhodwzqBOToelvMwXrxZTSvm5oMAA/t/EExjWLZbrX17JlxtznA5JeZkmF6VUi2gfEshTU4bTKzGCq55dzqrteU6HpLxIk4tSqsVEhwWzcPoIYsNDmDp/GZuzC50OSXmJJhelVIvqGNWOZ2eciACTn/6e3fmHnA5JeYEmF6VUi+uREM6CaSPIKy7l8nnfk19c5nRIqoVpclFKeUVq52j+N2UYW3KKmLFwGQdL9TExrZkmF6WU15zcO4H/XjKEFb/u548v/EBZRaXTIakWoslFKeVV6aku7hqfwpL1e7lp0Soq9TlkrZLeRKmU8rrLRnYjt7CU/3z8MwkRodya3t/pkJSHaXJRSjniz6N7k1tUwpOfbyY+PISrTu/ldEjKgzS5KKUcISLcce5A9hWV8s/F64kLD+H/hnVp/IfKL2hyUUo5JiBAePCiIeQfLOPm11cTGxbCmAEdnQ5LeYCe0FdKOSokKIDHLhtKSqco/vjCDyzbus/pkJQHaHJRSjkuIjSIeVOHkxzTnukLlrFu1wGnQ1LHSJOLUsonxEeE8syMEYSFBHL9yyudDkcdI00uSimf0Tk2jKtP78X63QVs2lvgdDjqGGhyUUr5lLRUFwCLV+92OBJ1LDS5KKV8SseodgzrFkvGGk0u/kyTi1LK56Sluli36wBbcoqcDkUdJU0uSimfMzYlCYDFa3Y5HIk6WppclFI+JzmmPUO6xOh5Fz+myUUp5ZPSU5NYvSOfrH3FToeijoImF6WUT0pLsa4ay1itpTF/pMlFKeWTusSFkZocrVeN+SlNLkopn5WWmsRPWXls36+lMX+jyUUp5bPS7dLY+3r04nc0uSilfFb3hHAGuKJYrMnF72hyUUr5tPTUJFZs28/u/ENOh6KaQZOLUsqnVT1r7H29odKvaHJRSvm0XokR9O0YqVeN+RlNLkopn5eWmsSyrfvYW6ClMX+hyUUp5fPSU10YAx/o0Yvf0OSilPJ5fTpE0CsxnAx91pjf0OSilPJ5IkJ6qovvtuSSU1jidDiqCTS5KKX8Qnqqi0oDH67d43Qoqgk0uSil/EK/pEh6JITrO178hCYXpZRfEBHSUpL4+pdc9heVOh2OaoQmF6WU30hPdVFRafgoU0tjvk6Ti1LKbwzsFEWXuPZkaGnM52lyUUr5DREhPcXFV5tyyC8uczoc1QBNLkopv5KW6qKswvDROi2N+TJNLkopvzK4czSdotuxWF9/7NM0uSil/IqIkJbq4ouNORw4pKUxX6XJRSnld9JTXZRWVLJk3V6nQ1H10OSilPI7x3eJISmqHRlaGvNZmlyUUn4nIEAYm5LEpz9nU1hS7nQ4qg6aXJRSfik91UVpeSVL12tpzBdpclFK+aWh3WJJjAzVZ435KE0uSim/FBggjB2YxNL12RSXamnM12hyUUr5rbTUJA6WVfDZhmynQ1G1aHJRSvmtEd3jiA8P4T29asznaHJRSvmtoMAAzh6YxJL1ezlUVuF0OMqNJhellF9LT02iuLSCz37W0pgv0eSilPJrI3vGExsWrM8a8zGaXJRSfi04MICzByTx8bq9lJRracxXeD25iMhYEdkgIptE5OZ6urlIRDJFZK2IvODtGJVS/iUtNYnCknK+3JjjdCjK5tXkIiKBwFwgDRgATBSRAbW66QPcApxijBkI/MWbMSql/M/JvRKIahdExurdToeibN4+chkBbDLGbDbGlAIvAeNrdXMFMNcYsx/AGKPPdlBKNSgkKICzBiTxUeZuSssrnQ5H4f3kkgxkuX3fbjdzdxxwnIh8JSLfisjYunokIleKyHIRWZ6drVeJKNXWpacmceBQOV//oqUxX+CLJ/SDgD7AKGAi8D8RiandkTHmSWPMMGPMsMTERC+HqJTyNaf2SSAiNEgfw+8jvJ1cdgBd3L53tpu52w68bYwpM8ZsAX7GSjZKKVWv0KBAxvTvwIeZeyir0NKY07ydXJYBfUSkh4iEAJcAb9fq5k2soxZEJAGrTLbZm0EqpfxTWqqLvOIyvt2c63QobZ5Xk4sxphy4FvgAWAe8YoxZKyKzReQ8u7MPgFwRyQSWAn8zxuiSopRq1OnHJRIeEqhXjfkAMcY4HcMxGzZsmFm+fLnTYSilfMCfXvyRrzfl8N2towkK9MXTyt4hIiuMMcOcGn7bnfJKqVYpPSWJ3KJSvt+6z+lQ2jRNLkqpVmVU3w60Dw5ksZbGHKXJRSnVqrQPCeSMfom8v3Y3FZX+X/b3V5pclFKtTlqKi+yCElZs2+90KG2WJhelVKtzRr8OhAYF6A2VDgpqrAMRmdKcHhpjnjn6cJRS6thFhAZx+nGJLF6zi1nnDCAgQJwOqc1pNLkAC5rRPwNoclFKOS491cWHmXv4MWs/Q7vFOR1Om9OU5NKjxaNQSikPG92/AyGBAWSs3q3JxQGNJhdjzDZvBKKUUp4U2S6Y3xyXwOLVu/jHuP6IaGnMm/SEvlKq1UpLcbEz/xA/bc93OpQ2pykn9LdgnUtpCmOM6XVsISmllGeM6d+R4EBh8epdDOlyxJs7VAtqyjmXz2h6clFKKZ8RHRbMKb0TyFizi6bX3aQAACAASURBVJvT+mlpzIuacs5lqhfiUEqpFpGe4mLmolWs3XmAlORop8NpM/Sci1KqVTtrQEcCA4T39IZKr2pKWewIIjIY6Au0q91Ob6JUSvmS2PAQTu4Vz+LVu5j5275aGvOSZiUX+1327wEjqxrZ/93PyWhyUUr5lLQUF7e+sZp1uwoY0CnK6XDahOaWxe4F4oHfYCWW84EzgeexXkU8wqPRKaWUB/x2YEcCBBav0dKYtzQ3ufwWK8F8a3/fboz51BgzBfgYuM6TwSmllCfER4Qysmc8763eRWt4+64/aG5ycQGbjTEVwCEg0q3d68A4TwWmlFKelJbqYnN2ERv3FjodSpvQ3OSyG6i6E2kbcJJbu94eiUgppVrAbwd2RAR9DL+XNDe5fMnhk/nPAreLyBMiMheYA3zgyeCUUspTOkS2Y3j3OH39sZc0N7ncyeEEMgeYi1UKmwi8DfzJc6EppZRnpacksWFPAZu0NNbimpVcjDG/GGO+sD+XGWNuMMZ0NsbEGWMmGWNyWyZMpZQ6dmNTXAAs1tJYi2tWchGRYBEJr6dduIgEeyYspZTyvKTodgztFkvGGi2NtbTmlsWeAv5XT7sn7D+llPJZaSlJrNt1gC05RU6H0qo1N7mcAbxVT7u3gdHHFo5SSrWstFS7NKY3VLao5iaXDsDeetplAx2PLRyllGpZyTHtGdIlRq8aa2HNTS57gdR62qUCekJfKeXz0lOTWL0jn6x9xU6H0mo1N7m8C9wmIoPcG4pIKvB34B1PBaaUUi0lLUVLYy2tucllFpAHrBCRr0XkFRH5CvgByAf+4ekAlVLK07rEhZGaHE2GlsZaTHPvc8kBhgP/xHoq8hD7/z3AcLu9Ukr5vLTUJFZm5bEj76DTobRKzX4TpTEmzxgzyxhzkjHmOGPMycaYO4wx+S0RoFJKtYSq0tj7es9Liziq1xyLSIKInCMil4tInN2snYjoa5OVUn6hR0I4/V1R+iDLFtLcO/RFROYA27Hua5kHdLdbv4V1Ul8ppfxCekoSK7btZ3f+IadDaXWae6RxC3AtMBs4kcOvOQbrSrFzPBSXUkq1uPRBVaUxPXrxtOYml98Ds40x92JdIeZuE9DLI1EppZQX9EqMoG/HSH3WWAtobnJJ5vArjmsrBep8qKVSSvmqtNQklm3dx94CLY15UnOTyw4gpZ52g4EtxxaOUkp5V3qqC2Pgg7V7nA6lVWlucnkVmCUip7g1MyJyHHAD8JLHIlNKKS/o0yGCXonh+o4XD2tucrkDWA98Dmy0m70KrMY65/Ivj0WmlFJeICKkp7r4dnMuuYUlTofTajT3Dv2DwChgKvA18DGwDLgS60qxqz0bnlJKtby0FBeVBj7M1NKYpzT3PpcEoNIY86wx5jJjzNnADCAB+Bl4sAViVEqpFtXfFUn3+DC9odKDGk0uIhIqIg+JSAGwB8gVkavtdpcBvwBzgCxgbEsGq5RSLUFESEt18fUvuewvKnU6nFahKUcus4A/Ad9gJZGPgIdE5P8Bz2A9DXm8MeZEY8xHLRapUkq1oHGpLioqDR9pacwjgprQzcXAo8aYa6saiMh04CmsRHOuMUZTvVLKrw3sFEWXuPZkrNnFRcO7OB2O32vKkUsX4I1azV63/z+oiUUp1RqICOkpLr7alEN+cZnT4fi9piSXYKCgVrOq79meDUcppZyTluqirMLw8TotjR2rppTFAJJFpKfb90C35nnuHRpjNnskMqWU8rLBnaPpFN2OxWt2MWFoZ6fD8WtNTS6v1dP8zTqaBdbRTCmlfF7VVWPPfrONgkNlRLYLdjokv9WU5DKtxaNQSikfkZ6axNNfbmHJ+r2MH5LsdDh+q9HkYoxZ6I1AlFLKFxzfJZaOUaG8t2qXJpdjoK8lVkopNwEBQlqKi09/zqawpNzpcPyWJhellKolPdVFaXklS9fvdToUv6XJRSmlahnaLZbEyFAW6+uPj5omF6WUqiUwQBg7MIml67MpLtXS2NHwenIRkbEiskFENonIzQ10N0FEjIgM82Z8SikF1uuPD5ZV8NkGvVf8aHg1uYhIIDAXSAMGABNFZEAd3UUC1wHfeTM+pZSqMqJ7HPHhIWSs2e10KH7J20cuI4BNxpjN9jPJXgLG19HdXcB9wCFvBqeUUlWCAgM4e2ASS9bt4VBZhdPh+B1vJ5dkrPe+VNluN6smIicAXYwx73kzMKWUqi09NYmi0go+/1lLY83lUyf0RSQA622WNzSh2ytFZLmILM/O1hmvlPK8kT3jiQkL1jdUHgVvJ5cdWI/wr9LZblYlEkgBPhWRrcBI4O26TuobY540xgwzxgxLTExswZCVUm1VcGAAZw/oyMfr9lJSrqWx5vB2clkG9BGRHiISAlwCvF3V0hiTb4xJMMZ0N8Z0B74FzjPGLPdynEopBVg3VBaWlPPlxhynQ/ErXk0uxphy4FrgA2Ad8IoxZq2IzBaR87wZi1JKNcXJvRKIahdExmq9aqw5mvrIfY8xxmQAGbWazaqn21HeiEkppeoTEhTAWQOS+ChzN6XlqYQE+dSpap+lU0kppRqRnprEgUPlfP2LlsaaSpOLUko14tQ+CUSEBrFYS2NNpslFKaUaERoUyJj+HfggczdlFZVOh+MXNLkopVQTpKW6yCsu47vN+5wOxS9oclFKqSY4/bhEwkICeU9vqGwSTS5KKdUE7YIDObNfBz5cu5tyLY01SpOLUko10bhUF7lFpXy/VUtjjdHkopRSTTSqbwfaBwfqVWNNoMlFKaWaqH1IIGf0S+T9tbupqDROh+PTNLkopVQzpKW4yC4oYcW2/U6H4tM0uSilVDOc0a8DoUEB+hj+RmhyUUqpZogIDeL04xJ5f81uKrU0Vi9NLkop1UzpqS52HzjEj1l5TofiszS5KKVUM53ZvwMhgVoaa4gmF6WUaqaodsGc1ieBxat3YYyWxuqiyUUppY5CeqqLnfmH+Gl7vtOh+CRNLkopdRTG9O9IcKCwWEtjddLkopRSRyE6LJhTeieQsUZLY3XR5KKUUkcpPcVF1r6DrN15wOlQfI4mF6WUOkpnDehIYIDoVWN10OSilFJHKTY8hJN7xZOhV40dQZOLUkodg7QUF1tzi1m/u8DpUHyKJhellDoGZw/sSICgV43VoslFKaWOQUJEKCf2iOc9LY3VoMlFKaWOUXpqEr9kF7Fxb6HTofgMTS5KKXWMfpuShAh61ZgbTS5KKXWMOkS2Y3j3OH39sRtNLkop5QHpKUls2FPAJi2NAZpclFLKI8amuAB4f42WxkCTi1JKeURSdDuGdoslQ0tjgCYXpZTymLSUJDJ3HWBrTpHToThOk4tSSnlIWqpVGlu8Ro9eNLkopZSHJMe0Z3CXGL0kGU0uSinlUekpSazekU/WvmKnQ3GUJhellPKg9OrSWNs+etHkopRSHtQlLozU5Og2f9WYJhellPKwtNQkVmblsSPvoNOhOEaTi1JKeVha9Q2VbffoRZOLUkp5WI+EcPq7otr0O140uSilVAtIT0li+bb97M4/5HQojtDkopRSLaDqhsoP1rbN0pgmF6WUagG9O0RwXMcI3mujpTFNLkop1ULSUlws27qPvQVtrzSmyUUppVrIuEEujIEP1u5xOhSv0+SilFItpE+HCHolhrfJq8Y0uSilVAsREdJTXXy7OZfcwhKnw/EqTS5KKdWC0lJcVBr4MLNtlcY0uSilVAvq74qke3xYm3sMvyYXpZRqQSJCWqqLr3/JZX9RqdPheI0mF6WUamHpKS4qKg0frWs7pTFNLkop1cJSkqPoHNu+TZXGNLkopVQLq7pq7KtNOeQXlzkdjldoclFKKS9IT3VRVmH4uI2UxjS5KKWUFwzuHE2n6HZt5vXHmlyUUsoLqq4a+/znHAoOtf7SmCYXpZTykvTUJEorKlmyfq/TobQ4rycXERkrIhtEZJOI3FxH+7+KSKaIrBKRT0Skm7djVEqplnB8l1g6RoW2iavGvJpcRCQQmAukAQOAiSIyoFZnPwLDjDGDgNeA+70Zo1JKtZSAACEtxcWnG7IpKil3OpwW5e0jlxHAJmPMZmNMKfASMN69A2PMUmNMsf31W6Czl2NUSqkWk5aSREl5JUs3tO7SmLeTSzKQ5fZ9u92sPjOAxXW1EJErRWS5iCzPzs72YIhKKdVyhnWPIyGi9ZfGfPaEvohcBgwD5tTV3hjzpDFmmDFmWGJioneDU0qpoxQYIIxN6cjS9dkUl7be0pi3k8sOoIvb9852sxpEZAzwd+A8Y0zbegmCUqrVS091cbCsgs82tN6qi7eTyzKgj4j0EJEQ4BLgbfcOROR44AmsxNK6i5JKqTZpRPc44sNDyFiz2+lQWoxXk4sxphy4FvgAWAe8YoxZKyKzReQ8u7M5QATwqoisFJG36+mdUkr5paDAAM4emMSSdXs4VFbhdDgtIsjbAzTGZAAZtZrNcvs8xtsxKaWUt6WnJvHi97/y+c/ZnD0wyelwPM5nT+grpVRrNrJnPDFhwSxupaUxTS5KKeWA4MAAzh7QkY8z91BS3vpKY5pclFLKIWmpLgpKyvlqU47ToXicJhellHLIKb0SiGwXxHurWl9pTJOLUko5JCQogLMGdOSjzN2Ullc6HY5HaXJRSikHjUt1ceBQOV//0rpKY5pclFLKQaf2SSAiNIjFq1tXaUyTi1JKOSg0KJAx/TvwQeZuyipaT2lMk4tSSjksLdVFXnEZ323e53QoHuP1O/S9rbKyku3bt1NUVOR0KEr5hPDwcDp37kxAgO5b+orTj0skLCSQjDW7OLVPgtPheESrTy45OTmICH379tWVSbV5lZWV7Nixg5ycHDp06OB0OMrWLjiQM/t14IM1u7lrfAqBAeJ0SMes1W9t8/Ly6NixoyYWpYCAgAA6duxIfn6+06GoWtJTXeQWlfL9ltZRGmv1W9yKigqCg4OdDkMpnxEcHEx5eet9SZW/GtU3kXbBASxe0zreUNnqkwuAiP8fYirlKbo++KawkCDO6NuBxWt2U1FpnA7nmLWJ5KKUUv4gPdVFdkEJK7btdzqUY6bJpY1KS0vj/vvvb7S7L774gpiYGC9E5IytW7ciImzfvh2ABQsW0Lt3b4ejUm3VGf06EBoUQMZq/y+NaXLxEaNGjSI0NJSIiAiio6M5/vjjWbRoUYsNb/HixcycObPR7k477TTy8vJaLA6AqVOnEhwcTEREBFFRUfTv359HH320RYd5LHbt2sXVV19Nt27dCA8Pp2vXrlx00UWsWLHC6dCUn4sIDeL04xJ5f81uKv28NKbJxYfcdtttFBYWkpuby8SJE7n44ov5+eefj+iurKzMgeha1uWXX05hYSF5eXncfffdXHvttXz66adOh3WEnTt3Mnz4cLKyssjIyODAgQNkZmZy7rnn8vrrrx9VP1vj/FRHLz3Vxe4Dh/gxq2V36lqaJhcfFBQUxDXXXENFRQWrV6/m008/JSgoiGeffZaePXsSFxcHwK+//sqFF15IUlISLpeLK6+8koKCgur+ZGdnM2PGDLp27UpUVBQnnHACGzZsAKwjpbvvvhuAkpISrrzySjp06EBUVBR9+vTh1VdfBagedpXy8nJmz55Nz549iY2NZfTo0axZs6a6/dSpU5k8eTJXXHEFMTExJCcn88QTTzR53AMCApgwYQLx8fEsX768uvkXX3zBqaeeSlxcHL169eKBBx7AmMN7dqtWrWLs2LEkJiYSFxfHmDGH35Y9bdo0unTpQmRkJAMGDOCFF15ocjy1zZo1i/DwcN544w0GDhxIYGAgERERTJ48mXvuuQeAO+64o8bwoeb0rmt+zp07lyFDhtT4zZYtWwgMDGTr1q1A4/NbtQ5n9u9ASGAAi/28NNbqb6Ks7c531pK584BXhjWgUxS3nzuw2b8rLS1l7ty5BAcHM3jwYLZv305FRQUZGRn8+OOPBAcHc+jQIc4880wmTZrEs88+y6FDh7j00ku57rrrmDdvHpWVlZx33nl06tSJZcuWkZiYyJo1a4iMjDxieAsXLmTZsmWsW7eO+Ph4srKy6t1ozZkzh2eeeYaMjAx69OjBP//5T8466yw2bNhAVFQUAK+99hovv/wyTzzxBG+++SYXX3wxY8eOpVu3bo2Oe0VFBa+99ho5OTn07dsXgMzMTNLT03nuuec455xz2LhxI2lpaSQmJjJlyhR27drF6aefzsyZM1m0aBHBwcF8/vnn1f089dRT+fe//01MTAyvvvoqU6ZMYciQIQwYMKDZ8yYjI4Pp06cf8+XttednSUkJN9xwAytXrqxOMgsWLGDUqFF079690fmtWo+odsGc1ieBxWt28/dx/f326j49cvEh99xzDzExMXTu3Jm33nqLRYsW1Ti5fN999xEdHU1YWBjvvvsuxhhmz55N+/btiY2N5a677uL555+noqKC5cuXs3z5cubNm1d9E+mgQYPo1KnTEcMNCQmhsLCQzMxMysvL6dKlS70b3vnz53PTTTfRr18/QkNDmTVrFoGBgbz33nvV3Zx55pmcd955BAQEcMEFFxATE8PKlSsbHPdnn32WmJgY2rVrx8SJE5k9ezbnnnsuAI8++ij/93//x/jx4wkMDKRfv35ce+21PPPMM9W/7d27N7fccgvh4eGEhITUOHKYMWMG8fHxBAYGcskllzBo0KCjLrllZ2eTnJx8VL+tzX1+xsbGMn78eObPnw+AMYaFCxcyffp0gEbnt2pd0lJd7Mg7yKrt/nuza5s7cjmaIwlv+fvf/84//vGPOtsFBATQpUuX6u9btmzh119/PeJKLhFh9+7dbN26lQ4dOhAdHd3ocC+77DL27NnD9ddfz8aNGxk9ejT3339/nVdNZWVl0aNHjxpxde/enaysrOpmLperxm/Cw8Orj4QGDhzItm3bALj11lu59dZbAZg8eTJPPfUUxcXFzJw5k08++YRbbrmFoKAgtmzZwpIlS2qc06isrKyeHlu3buW4446rc9wqKyu54447ePnll9m9ezciQlFREdnZ2Y1Ol7okJiayY8eOo/qtu9rzE6zy3eTJk/n3v//N559/Tl5eHhdccAHQ+Pz2VMJTvuGs/h0JChAy1uxicBf/vFpTj1z8hIjUODzu1q0bxx13HHl5eTX+Dh06RHJyMt27d2fv3r0cONB4CTAoKIibbrqJ5cuXs23bNsLCwqr3mGvr0qVL9TkAsDbeW7duPWJDWZ+1a9dSWFhIYWFhdWJxFxYWxoMPPsiOHTuYO3du9bhOnz69xngeOHCAtWvXAtC9e3c2btxY5/BefPFFnnrqKRYtWsT+/fvJy8tj8ODBNc7XNEd6ejqvvfZagyfhIyMjj3hQ6s6dO2t8rz0/Ac466yxCQ0N55513WLBgAZdccgnt27cHGp/fqnWJDgvmlN4JZKzeddTLqtM0ufipc845h9LSUu69914KCgowxrBjxw7eeOMNAIYNG8YJJ5zA73//e/bu3UtlZSWrVq06YiMHsGTJElasWEFZWRnt27cnPDycwMDAOoc7depU7r//fn7++WdKS0u55557KC8vZ9y4cR4bt5CQEGbNmsXdd99NQUEB11xzDS+99BLvvPMOZWVllJeXk5mZyWeffQZYR14bNmzgvvvuo7i4mNLSUj7++GMADhw4QFBQEImJiVRWVjJv3jx++umno47tzjvvpLCwkAsvvJB169ZRUVFBUVERL774YvVR59ChQ/nhhx9YsWIF5eXlPPLII2zZsqXRfgcGBjJlyhQefvhhXn/99RoJvrH5rVqfcakusvYdZK2XzhF7miYXPxUWFsaSJUvIzMykX79+REdHM3r06OpzGwEBAbzzzju0b9+eIUOGEBMTw/Tp0yksLDyiX3v27GHy5MnExsbicrnYtm0bTz75ZJ3D/dvf/sbEiRM5++yz6dixI0uWLOHDDz+sPpnvKZMmTSIuLo4HHniAlJQU3n33Xf773//icrno0KEDU6dOrS5tderUiU8//ZSPPvqIzp07k5SUxJw5cwDrEucTTzyR3r17k5ycTGZmJqeddtpRx5WcnMyyZctwuVycffbZ1fflvPnmm0yYMAGwrgz761//ytixY3G5XOzZs4dTTjmlSf2fNm0an332GT169GDEiBHVzRub36r1OWtARwIDxG9vqBR/PeRyN2zYMON+2aq7devW0b9/fy9HpJRv0/XCP0x++juy9hWz9MZRzb5qTERWGGOGtVBojdIjF6WU8lFpKS625hazfrf/3c+kyUUppXzU2QM7EiD45Q2VmlyUUspHJUSEcmKPeDLW7HY6lGbT5KKUUj4sPTWJTXsL2bjHv0pjmlyUUsqH/XZgEiKQsdq/jl40uSillA/rENWO4d3i/O6SZE0uSinl49JTk9iwp4BNe4+8T81XaXKpT0UFvPsujBsHMTEQGGj9HzfOaq4PC1RKecnYFOt5fe+v8Z+jF00uddmwAQYNgnPPhYwMyM+Hykrrf0aG1XzQIKs7pZRqYUnR7RjaLdavzrtocqltwwY45RTIzGy4u8xMqzs/SjD33ntv9WPsGxMREcE333zTwhHVT0T48ssvvTY895d5HY0FCxbU+RRpd0FBQT75dk3lH9JSksjcdYCtOUWNd+wDNLm4q6iACy6A3Fzre7t2cOONsG4dlJZa/2+8EUJDrfa5uVb3HiiRjRo1itDQUCIjI4mOjqZnz55MnjzZo+9lv/XWW3nnnXea1G1hYSEnnXSSx4Zd5ddffyUiIqL6LygoiJCQkOrvAwf67isRfNny5csZMWIEYWFh9OrVi+eee87pkJSHpaVapbHFfnLPiyYXd4sXHz5iadcOliyBOXOgXz8IDrb+z5kDS5ceTjCZmfD++x4Z/G233UZBQQH5+fksXbqUbt26MXLkyFb15NuuXbtWP3K/sLCQUaNGceutt1Z/r3qMfnO15ffQ5+fnk5aWxoQJE9i/fz+PP/44f/jDHxw98lSelxzTnsFdYljsJ+ddNLm4e+yxw5+vvRbq23M/6SSrfV2/85Bu3bpx9913M2XKFP70pz9Vv9OhuLiYG2+8kR49ehAXF8fYsWPZtGlT9e/Kysq499576du3L5GRkfTq1YvXXnsNOPLd7g8//DA9evQgMjKS5OTkGu9XqV2WWrRoEYMHDyY6OprBgwfXSHhVJaGHH36Yzp07Exsby1VXXXXMb0hctWoVw4cPJzIykpEjR7J+/frqdqNGjeIvf/kLv/vd74iKiuKBBx4A4H//+x8pKSlER0dz/PHH8+GHH1b/5scff+TUU08lOjqauLg4Tj75ZPbv31/dfv/+/UyYMKF6ur311ls14nnsscfo27cv0dHRjBw5ki+++KLe2AsKCrj88suJi4ujW7duLFy48JimRUNef/11wsLCmDlzJqGhoZx11lmcf/759T7ZWvmv9JQkVm3PJ2tfsdOhNM4Y4/d/Q4cONfXJzMyst90RoqONAetv3bqGu83MPNxtTEzTh1GP008/3dx1111HNP/www8NYNbZ8UyaNMmMGzfO7N6925SUlJhZs2aZvn37mtLSUmOMMTNnzjT9+/c3P/30k6msrDRZWVnmp59+MsYYc/vtt5vRo0cbY4zZsGGDad++vVmzZo0xxpj9+/ebb775pnq4gPniiy+MMcZ89dVXJjQ01GRkZJiysjLz7rvvmtDQUPPtt98aY4yZP3++CQoKMrfeeqs5dOiQ2bhxo4mNjTXPPfdco+M9evRoc/vttx/RHDDDhw8327ZtM4cOHTIXXnihGTNmTI3pFRkZaT755BNTWVlpioqKzJNPPml69eplVq5caSoqKsx7771nwsPDzcaNG40xxpx00knmzjvvNOXl5aa0tNR88803prCwsLp/8fHx5quvvjIVFRXmwQcfNNHR0aaoqMgYY8wLL7xg4uPjzbfffmvKysrMU089ZcLCwszWrVurp0GvXr2q45s+fbo56aSTzK5du0xeXp45//zzDWCWLl1a53TYtm2biY6ObvCvPtddd50ZP358jWYPPvigOf744+v9TbPWC+UztuUUmW43vWue/OyXRrsFlhsHt8t65OKuwO3xCr16Ndyte/smvO3xaHXu3BmA3NxccnJyeOGFF3j00Ufp2LEjISEh3H777ezatYvvvvsOYwxz585lzpw5DBo0CBGhc+fODBo06Ij+BgUFYYypfjNkTEwMI0eOrDOGBQsWMGHCBNLS0ggKCmLcuHGcf/75zJs3r7qb9u3bM3v2bEJDQ+nduzejR4+mvtcgNNXf/vY3unbtSmhoKFOnTj2ifxdeeCFnnnkmIkJYWBgPPfQQs2bNYvDgwQQEBJCens4ZZ5zBSy+9BFgvIfv111/JysoiODiYkSNHEh4eXt2/iy++mJNPPpmAgACuvPJK8vPzq99wOX/+fK666ipOPPFEgoKCmDFjBoMGDeKFF144Iu7Kykqef/557rrrLpKSkoiOjua+++5rcFy7du16xFsma//Vp6Cg4IjXWcfExDTpLaTKv3SNDyMlOYr3/OCGSk0u7iIjD3/+5ZeGu3Vv7+EXZbnbvn07APHx8dVvMxw0aBAxMTHExMQQFxdHWVkZWVlZZGdnU1RUVO/75N317NmT559/nv/973906tSJU089tUYJyV1WVhY9evSo0axXr15kZWVVf+/QoUONt1eGh4dTUHBsz0JyuVwN9q979+41vm/ZsoU//vGP1dMmJiaGpUuXVr/zfv78+VRWVnLqqafSo0cPbrvtNsrLy+sdHlA9zKZMgyrZ2dmUlJTUiK/2bz0pMjKS/Pz8Gs3y8vI8/gI35RvSU12szMpjR95Bp0NpkCYXd+5vC3z66Ya7dW/fxLcMHo2XX36Z5ORk+vbtS7du3QDYuHFjjT3a4uJiJk6cSGJiImFhYfW+T762Cy64gI8++oicnBwuuugixo8fT3HxkbXcLl26sHXr1hrNNm/eTJcuXY55/I5FkXU5vAAADI5JREFUQEDNxbdbt27MmzevxrQpLCzkMfucWI8ePZg3bx7bt2/n7bff5qmnnuKZZ55p0rCaMw0SEhIICQmp0X3t39ZW+yq6uv7qM3jw4CPeSPnjjz8yePDgRsdL+Z+06hsqffuqMU0u7q6++vDnRx6B+q62+eYbq31dv/OQrKwsbr/9dhYsWMBDDz2EiNChQwcmTZrENddcU703npeXxxtvvEFhYSEiwjXXXMPMmTNZs2YNxhi2b9/OqlWrjuj/hg0beP/99ykuLiY4OJjo6GhE5IgNNlivCl60aBEffPABFRUVLF68mNdff51p06Z5fLyPxfXXX88dd9zBypUrMcZw8OBBvvzyy+oLARYuXMjOnTsBq2wUFBRU42irIVOnTuWJJ57g+++/p7y8nPnz57Ny5UomTZp0RLeBgYFMmjSJ22+/nT179nDgwAFuvvnmBvtf+yq6uv7qc/7551NUVMScOXMoLS3lk08+4fXXX+fKK69s0rgp/9IjIZz+riiff8eLJhd3aWkwYID1+dAhOOOMuu9zOeMMKCmxuhswAMaO9cjg77rrLiIjI4mKiuI3v/kNmzZt4uuvv65+NztYV0P17duXUaNGERkZSWpqKq+++mr1K1DvueceLrroIn73u98RGRnJqFGjalxNVqW0tJTZs2fjcrmIiYnh4YcfZtGiRbRr1+6Ibk855RQWLlzIjTfeSGxsLDNnzuS5556r9xyNU6644gpmzpzJtGnTiI2NpWvXrtx1113VlykvWbKEoUOHEh4ezkknncSkSZOYPHlyk/pdlSwuu+wy4uPjeeyxx8jIyKg+mqztoYceokePHvTr14/U1FTOPffcJiey5oqJiSEjI4NXX32V6OhorrjiCh5//PEWuU9J+Yb0lCSWb9vP7vxDTodSLzH2Ja7+bNiwYaa+k8fNfld41R36VTdSNiQ+Hr7+GppwjkMpX9Ls9UL5lE17Cxnz4Gfced5ALv//7d17jFxlGcfx74+2uNIblVJYWshCSivN1gspDfCHSkBDK4EgQkAbXCGAELwhRA2CiPzjDRINBGpARAEBTUgNJdykVIEilUtpK0gpDS6glGWpXFvaPv7xnmGns7cpmTlnLr9PcjJzZs6Zed53Z+bZ8573vO/hXUNuI+kfETEv38gG+Mil0uzZ8OCDA0cww5kzx4nFzAoxc9oEZu01oaGH4XdyGcrs2bBq1Y6jIu+yy46jIq9a5cRiZoVZ0N3J3ze8xsY3NhcdypCcXIYzZsxAIunvT+OH9fcPJJw6tZ+bmVVj4dxOIuCuNY3Za8zJxcysCc3aawIH7Dm+YZvG2iK5tEKnBbNa8fehNUji83M7WbG+j743G69prOWTS0dHB319ff5CmZESS19f35Bdzq35LOjuZHvA3Wv/W3Qog4wtOoB6mzFjBr29vWzcuLHoUMwaQkdHx/tj1llzO6hzIl177MbSp17mlPn7FR3ODlo+uYwbN66u4zqZmRVFEgvmdrJ4+Xr639rClPG7Fh3S+1q+WczMrJUt7O5k2/bgnn82VtNY7slF0tGSnpG0TtKgAZckfUjSLdnzj0jqyjtGM7Nm0T19EjOmfLjhxhrLNblIGgNcCSwA5gCnSKq8FP50oD8iZgJXACNPhGFm1sYksXBuJ39b9yqb3mmc6b7zPnKZD6yLiPURsQX4A3BcxTbHAaU5Yf8IHKnSqIxmZjbIgu69eW9bcF8DNY3lnVymA+WzK/Vmjw25TURsBTYBe+QSnZlZE/rEvruzz+SOhrqgsml7i0k6EyhNWLFZ0uoi42kAU4FXiw6iQO1efnAdtHv5eRimXtvzfh0MPR9ETvJOLi8C5VP3zcgeG2qbXkljgcnAoPHvI2IxsBhA0soih5ZuBO1eB+1efnAdtHv5obHqIO9msUeBAyXtL2lX4GRgScU2S4CvZPe/CPwlfHm9mVlTyfXIJSK2SjoXuAsYA1wXEWskXQqsjIglwLXA7yStA14jJSAzM2siuZ9ziYilwNKKxy4uu/8ucOJOvuziGoTW7Nq9Dtq9/OA6aPfyQwPVQUtMc2xmZo3Fw7+YmVnNNVVyafehY6oo/3mS1kpaJek+SYV2RayH0eqgbLsTJIWkhug5U0vV1IGkk7LPwhpJN+UdYz1V8T3YT9L9kh7PvgsLi4izXiRdJ+mV4S6/UPLLrH5WSTo47xiBNL9DMyykDgDPAQcAuwJPAnMqtjkHuDq7fzJwS9Fx51z+I4Ddsvtnt1L5q62DbLuJwHJgBTCv6LgL+BwcCDwOTMnWpxUdd87lXwycnd2fA2woOu4a18GngIOB1cM8vxC4ExBwKPBIEXE205FLuw8dM2r5I+L+iHg7W11Buo6olVTzGQD4MWlMunfzDC4n1dTBGcCVEdEPEBGv5BxjPVVT/gAmZfcnAy/lGF/dRcRyUk/a4RwH3BDJCmB3SZ35RDegmZJLuw8dU035y51O+u+llYxaB1kTwL4RcUeegeWoms/BLGCWpAclrZB0dG7R1V815b8EWCSpl9Qz9ev5hNYwdva3oi6advgXG56kRcA84NNFx5InSbsAlwM9BYdStLGkprHPkI5el0uaGxGvFxpVfk4Bro+IX0g6jHTdXHdEbC86sHbSTEcuOzN0DCMNHdOkqik/ko4CLgSOjYjNOcWWl9HqYCLQDSyTtIHU3rykxU7qV/M56AWWRMR7EfE88C9SsmkF1ZT/dOBWgIh4GOggjTvWLqr6rai3Zkou7T50zKjll/RJ4BpSYmmldvaSEesgIjZFxNSI6IqILtJ5p2MjYmUx4dZFNd+D20lHLUiaSmomW59nkHVUTflfAI4EkHQQKblszDXKYi0BTs16jR0KbIqI3IdLbppmsWjzoWOqLP/PgAnAbVk/hhci4tjCgq6xKuugpVVZB3cBn5O0FtgGXBARLXEEX2X5vwP8WtK3SSf3e1ron0wk3Uz652Fqdl7ph8A4gIi4mnSeaSGwDngb+GohcbZQnZuZWYNopmYxMzNrEk4uZmZWc04uZmZWc04uZmZWc04uZmZWc04u1hIk9WSjIJeWNyQ9Kenc7ILaPGLoyt67Zyf2KcXdVbfAzArQNNe5mFXpRNIV6pOy+78CpgEXj7RTjbwMHEYatbdad2T75H6Rm1k9+ToXawnZ0cJvgAMjYl3Z4/cDB0fE5CH2GQdsbaUL7MwahZvFrNU9CkySND9rfjpH0k8lvQRsBnYHkPSFbAThtyW9Luk2SftVvpikMyQ9JukdSf2SHpB0ePbcoGYxSYdIukdSX7bPeklXlT0/qFlM0jhJl0naIGlLdntZlgypeK+zJF0q6eUs7j9LarWpFqwJOblYq9ufNATKm9n6haSxts4EjgfelfQ14E/AWtKYdGeRBsB8QNLE0gtJ+jlpIqrHgJOARaRJyQYloWz7CaRhSraRRmpeAFzK6M3RvwW+B9wAHANcD3yXgbmKyn0fmAmcBnyT1MT2+1Fe36z+ipihzIuXWi+kH+8AZpN+vKeQksQ20kCOXdnzj5E1B2f7TSDN+3NdxevtD2wBvpWtz8xe6/IRYii9R0+2Pi9b/1gVcXdl693Z+iUV2/2g/LXK3mtZxXbnZ4/vU/TfxEt7Lz5ysVbzNPAeaeDSq4AbSf/Vl9weEeXnWA4jnfy/UdLY0kKabOlp0pSyAEeRjvQX70QszwKvA9dIWiRp39F2KHu/yqOP0nrlHD1LK9afym6HPJoyy4uTi7Wa44FDgI8C4yPi1IgonxK2slfWtOz2XlJSKl/mMjCTaem2t9pAImITcARpmt2rgBckrZZ0wgi7fWSYOP9T8XxJ5XS3pTl8OqqN06we3BXZWs3qKOstNoTKnmGloeh7gDVDbP9GdvtqdjsdeKbaYCLiCeCE7GhoHukcya2SPh4Rq4fYpZQs9mbHLs17Vzxv1tB85GLt7iFSApkZESuHWEqJ5F5gO6kjwE6LiK0RsQK4iPS9O2iYTZdnt5VzEX05u132Qd7fLG8+crG2FhH/k3QBcKWkPYE7SSf4p5PObyyLiJsi4jlJVwDnZT3IlpBO8M8Hno6IWypfW9IxpGR0O/A8MB74BimZPTxMPKuzyaAuyY52HiKdF7oIuDkinhpqP7NG4+RibS8irpH0b+AC4Euk78WLwF+BJ8q2Oz+b5fQc0nTabwGrgLuHeelngXdIiaGTlFQeBT4bESOdu+khTUt8GqmX2EvAT4AffbASmuXPV+ibmVnN+ZyLmZnVnJOLmZnVnJOLmZnVnJOLmZnVnJOLmZnVnJOLmZnVnJOLmZnVnJOLmZnVnJOLmZnV3P8BC193D+MCKGcAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0JBY6PESuKdn"
      },
      "source": [
        "#### Evaluate Gradient Boosted Decision Trees w/ Word2Vec Representation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5hdiO5-Ixsty"
      },
      "source": [
        "# define the model (XGBoost has separate models for classifiers and regressors FYI)\n",
        "xgb_model_w2vec = XGBClassifier(\n",
        "                          objective='binary:logistic', \n",
        "                          n_estimators=500, \n",
        "                          tree_method='gpu_hist',\n",
        "                          learning_rate=0.05,\n",
        "                          max_depth=25,\n",
        "                          min_child_weight=1,\n",
        "                          colsample_bytree=0.7,\n",
        "                          n_jobs=-1          )\n",
        "\n",
        "#Fit classifier to Tf-idf training data\n",
        "xgb_model_w2vec.fit(full_X_train_w2vec, y_train.ravel(), eval_metric='logloss', verbose=True)\n",
        "#Create predictions on test set\n",
        "xg_rf_w2vec_preds = xgb_model_w2vec.predict(full_X_test_w2vec)"
      ],
      "execution_count": 50,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YsAamdWfxs4t",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f0850a38-8841-4bcf-a273-ffad2264a911"
      },
      "source": [
        "print(\"Gradient Boosted Decision Trees w/ Word2Vec Representation Accuracy: \" + str(accuracy_score(xg_rf_w2vec_preds, y_test)))\n",
        "print(\"Gradient Boosted Decision Trees w/ Word2Vec Representation Precision: \" + str(precision_score(xg_rf_w2vec_preds, y_test)))\n",
        "print(\"Gradient Boosted Decision Trees w/ Word2Vec Representation Recall: \" + str(recall_score(xg_rf_w2vec_preds, y_test)))"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Gradient Boosted Decision Trees w/ Word2Vec Representation Accuracy: 0.6732250401900328\n",
            "Gradient Boosted Decision Trees w/ Word2Vec Representation Precision: 0.5966600283129784\n",
            "Gradient Boosted Decision Trees w/ Word2Vec Representation Recall: 0.7045473863153421\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IG6op68-uKpK"
      },
      "source": [
        "#### Gradient Boosted Decision Trees w/ Word2Vec Representation - Confusion Matrix\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EmImqTHWxtwj",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "090ae6ec-9d2e-4c36-dd2c-33afddeef617"
      },
      "source": [
        "cm = confusion_matrix(y_test, xg_rf_w2vec_preds)\n",
        "cm_display = ConfusionMatrixDisplay(cm, display_labels=[0, 1]).plot()"
      ],
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxXZd3/8dd7hlVWEUREDEzU0BSFWzHTzCUxK21xK5WKNHdt+XVT9kjzzvs227XUNLmFMtGykkxFJE29ExUUN0hB1ARZZJN9mZnP749zDZyBmfl+vzjDbO/n43Eec76fs13fUT5zXec657oUEZiZWaasqQtgZtacOCmameU4KZqZ5TgpmpnlOCmameW0a+oC5PXuVR4DB7Rv6mJYCV59YaemLoKVYD1r2Bgb9F7OccJHu8TSZZVF7Tv9hQ2TImLke7nejtaskuLAAe15etKApi6GleCE3Yc2dRGsBE/FlPd8jqXLKnl60p5F7Vveb3bv93zBHaxZJUUza/4CqKKqqYvRaJwUzawkQbApims+t0ROimZWMtcUzcySIKhsxa8HOymaWcmqcFI0MwOyjpZKJ0Uzsy1cUzQzSwLY1IrvKfo1PzMrSRBUFrnUR1InSU9Lel7Sy5K+n+KDJD0laY6kuyR1SPGO6fOctH1g7lzfTvFXJJ2Qi49MsTmSxhTz/ZwUzaw0AZVFLgVsAI6JiIOAocBISSOAHwI/i4i9geXA6LT/aGB5iv8s7YekIcAZwP7ASOBGSeWSyoFfAScCQ4Az0771clI0s5Jkb7QUt9R7nszq9LF9WgI4Bvhjio8DTknrJ6fPpO3HSlKKT4iIDRHxOjAHODQtcyJibkRsBCakfevlpGhmJRKVRS5Ab0nTcst5Nc6U1ehmAIuBycBrwIqIqEi7zAP6p/X+wFsAafu7wC75+FbH1BWvlztazKwkWUdL0QPtLImI4XWeK6ISGCqpJ/BnYL/3XsL3xknRzEqSPaf4nkYf2/acESskPQIcDvSU1C7VBvcA5qfd5gMDgHmS2gE9gKW5eLX8MXXF6+Tms5mVrCpU1FIfSX1SDRFJnYHjgVnAI8Dn0m6jgHvT+sT0mbT975FNRzoROCP1Tg8CBgNPA88Ag1NvdgeyzpiJhb6ba4pmVpIGrCn2A8alXuIy4O6IuE/STGCCpB8AzwG3pf1vA34raQ6wjCzJEREvS7obmAlUABelZjmSLgYmAeXA2Ih4uVChnBTNrCSBqGyARmZEvAAcXEt8LlnP8dbx9cCpdZzrGuCaWuL3A/eXUi4nRTMrWaGmcUvmpGhmJQnExihv6mI0GidFMytJ9vB26+2jdVI0s5I19CM5zYmTopmVJEJUhmuKZmabVbmmaGaWyTpaWm/qaL3fzMwahTtazMy2UunnFM3MMg31Rktz5aRoZiWrcu+zmVkmGxDCSdHMDMiaz5v8mp+ZWSYCP7xtZraF/PC2mVm1wDVFM7MaWnNHS+v9ZmbWKILi5mcpYo6WAZIekTRT0suSLkvxuyTNSMsbaQpUJA2UtC637ebcuYZJelHSHEnXp/mgkdRL0mRJs9PPnQt9P9cUzawk2RSnDZI6KoBvRMSzkroB0yVNjojTq3eQ9BOy+Z2rvRYRQ2s5103AucBTZNMPjAQeAMYAUyLiWklj0uf/rK9QrimaWYlqn/i+tqU+EbEgIp5N66vIZvLbPFl9qu2dBtxZb2mkfkD3iJiaZvcbD5ySNp8MjEvr43LxOjkpmllJguyNlmKWYkkaSDaJ1VO58JHAooiYnYsNkvScpH9IOjLF+gPzcvvMY0ty7RsRC9L6QqBvobK4+WxmJSth5O3ekqblPt8SEbfkd5DUFbgHuDwiVuY2nUnNWuICYM+IWCppGPAXSfsXW5CICElRaD8nRTMrSYRKqQUuiYjhdW2U1J4sId4REX/KxdsBnwGGbblubAA2pPXpkl4D9gHmA3vkTrtHigEsktQvIhakZvbiQgV289nMSpJ1tJQXtdQn3TO8DZgVET/davNxwL8iYl5u/z6SytP6XsBgYG5qHq+UNCKd8xzg3nTYRGBUWh+Vi9fJNUUzK1GDzdFyBHA28GL1YzfAd9IE9mewbQfLUcDVkjYBVcD5EbEsbbsQuB3oTNbr/ECKXwvcLWk08CZZx029nBTNrCRZR8t7f80vIp6A2m9ORsQXa4ndQ9bUrm3/acABtcSXAseWUi4nRTMrWWt+o8VJ0cxKUv1GS2vlpGhmJfPEVWZmSQRsqnJSNDMDqpvPTopmZpuV8EZLi9N60/122rheXPLxwZx/3L6ce/S+jP/Rbtvs8+LULlz0sX04ccBBPH5fjwa57srl5Yw5/f186YgPMOb097NqRc0HX1+Z0blBr9faff2n/+auF17m139/pUHOd9ypyxj7xCzGPjGL405dts32q25/vcGu1dxVP5LzXocOa64aNSlKGinplTTG2ZjGvFZDad8xuO4Pr3Hzw69w0+RXmPZoN2ZN36nGPn36b+IbP/83H/308pLP//w/u/Ljy/fcJn73L3fl4A+v4n//bxYHf3gVd/1y183bKivhtmt2Z9hHVpX+hdqoh+7qxRVfGFTycdf9cQ5999hYI9atZwVnfX0Rl31iMJeeNJizvr6Irj0qNm8/4sQVrF/TluoXavABIZqTRit1eh3nV8CJwBDgTElDGut6DUWCzl2qAKjYJCo3CW31B2+3ARvZa8h6ymr57f3hxj5ccuI+nH9s7bXMujw5qQfHnZbVQI47bRlPPrilRnjv2D58+OPv0rN3RV2H21Zeeqorq5bXvDvU730buOaOufzywVf5yZ/nMGDv9UWda9jRq3j2sa6sWtGO1e+249nHujL8o9kfqE47VfKZr77D739ecPCVVqUqzdNSaGmJGjOVHwrMiYi5EbERmEA2tlmzV1kJFxy3L6cfeAAHH7WK/Q5ZW9Rx0x/txvzXO3L9/a9y4+RXmP1iZ16c2qWoY5cvac8ufbOk12vXCpYvaQ/AkgXt+ecDPfjEqCXb92Vss8uum8evvtufi0fuwy1X9+Pi/55f+CCg926beOftDps/L1nQgd67bQJg1LcWcs/Nu7JhXcusFW2PrPe5vKilJWrMjpb+wFu5z/OAw7beSdJ5wHkAe/ZvHv0+5eVw08OvsPrdcr4/eiBv/KsTA/crXKuY/o9uPPuP7lx4/L4ArFtbxvy5HfngiDVcetJgNm0oY93aMlatKOeC47J9Rn/3bYYfXbNZLEH1CEc3X9mf0Ve8XWut1IrXaadKhgxfw3dveWNzrH2H7Hf8sdOXccpX3gFg94Eb+a/fzaVik1j47w5cPbruJvhe+6+j38CN/PqqHts0uVszP7zdyNLYarcADD+oU8Gxznakrj0qOehDq3nmkW5FJcUATr9kESedvXSbbdf/LRsn8/l/dmXy3b345s//XWP7zr03sXRRO3bpW8HSRe3ouUtWa3z1+c78zwUDAXh3WTlPT+lGeTl86MR3seKVlcHqleWb/2DlPXRXLx66qxeQ3VP8yeV7smherma4sD0HHr568+fe/TbywpNdGTJsDfscuJZxT82kvBx69q7guj/O4Vuf27vxv1ATa6lN42I0Zv1jPjAg9zk/xlmztWJpOavfzar9G9aJZx/rxoC9NxR17PCPrGLShF6sSzfdlyxoz4olxf3dGfGxlTx8d/YP8+G7e3H4CVnSG//ULMY/PZPxT8/kyE+8yyX/M88JcTusXV3Oorc6cOQnVqRIsNeQdUUdO/3Rbgz7yGq69qiga48Khn1kNdMf7cZ943vz+UP2Z9RhQ/jGKXszf27HNpEQW3vvc2PWFJ8BBksaRJYMzwA+34jXaxDLFrXnx5ftSVWVqKqCoz65ghHHr2Tcdbuxz0FrOfyElbwyozNXjx7EqhXlTJ3cnfE/3o1bH32FYUev4t9zOnL5JwcDWYfNt254k569C1/39IsXcc35A3lwwi7s2n8jV/z6jcb9oq3cmBvf5MDDV9OjVwW/mzaT3/6kL9detCeXXjufz1+2iPL2wT/u7cncmZ0LnmvVinbc8fNdueH+rLZ/x8/6smpFkzeymlRL7VkuhrJ5Xhrp5NLHgZ8D5cDYiLimvv2HH9Qpnp40oL5drJk5YffaJlaz5uqpmMLKWPaeqnA777drHDP2c0Xt+6cjbppe38jbzVGj/rlLg0Xe35jXMLMdr6U2jYvRttsAZlayhhpktrlqvTcGzKzRNERHi6QBkh6RNFPSy5IuS/GrJM2XNCMtH88d8+30htwrkk7IxWt9e07SIElPpfhdkjpQgJOimZWk+jnFBuh9rgC+ERFDgBHARbm33n4WEUPTcj9A2nYGsD8wErhRUnmBt+d+mM61N7AcGF2oUE6KZlayhnjNLyIWRMSzaX0VMIstk9jX5mRgQkRsiIjXgTlkb87V+vZcmtnvGOCP6fhxwCmFvpuTopmVJAIqqsqKWoolaSBwMPBUCl0s6QVJYyXtnGK1vSXXv574LsCKiKjYKl4vJ0UzK1kJzefekqbllvO2PpekrmSz9F0eESuBm4D3A0OBBcBPduBXc++zmZWmxHefl9T3nKKk9mQJ8Y6I+BNARCzKbb8VuC99rO8tudriS4Gektql2mJRb9W5pmhmJYtQUUt90j2/24BZEfHTXLxfbrdPAy+l9YnAGZI6pjflBgNPk3t7LvUunwFMjOzNlEeA6ifNRwH3FvpurimaWckaaECII4CzgRclzUix75D1Hg8leyTyDeCrABHxsqS7gZlkPdcXRUQlgKSLgUlseXvu5XS+/wQmSPoB8BxZEq6Xk6KZlSSiYR7ejognoNbsWudbcOlV4W1eF67r7bmImEvWO100J0UzK5Go9BSnZmZbFLpf2JI5KZpZSVr7u89OimZWmsjuK7ZWTopmVrLWPB2Bk6KZlSTc0WJmVpObz2ZmOe59NjNLIpwUzcxq8CM5ZmY5vqdoZpYEosq9z2ZmW7TiiqKTopmVyB0tZmZbacVVRSdFMytZm6wpSrqBev4eRMSljVIiM2vWAqiqaoNJEZi2w0phZi1HAG2xphgR4/KfJe0UEWsbv0hm1tw1xHOKkgYA44G+ZKn2loj4haQfAZ8ENgKvAV+KiBVpbuhZwCvpFFMj4vx0rmHA7UBnsmkJLouIkNQLuAsYSDbfy2kRsby+chV82EjS4ZJmAv9Knw+SdGPR39zMWp8ocqlfBfCNiBgCjAAukjQEmAwcEBEHAq8C384d81pEDE3L+bn4TcC5ZDP8DQZGpvgYYEpEDAampM/1KuYJzJ8DJ5DNoUpEPA8cVcRxZtYqFTe9aaHOmIhYEBHPpvVVZLXA/hHxUJqnGWAq2XzNdZcmmxK1e0RMTdOajgdOSZtPBqpbveNy8ToV9Vh6RLy1VaiymOPMrJUqvqbYW9K03HJebadLTeODgae22vRl4IHc50GSnpP0D0lHplh/YF5un3kpBtA3Ihak9YVkTfV6FfNIzluSPgSEpPbAZWQZ3czaooAovvd5SUQMr28HSV2Be4DLI2JlLn4FWRP7jhRaAOwZEUvTPcS/SNq/6GJn9xgLNuqLqSmeD1xElnnfBoamz2bWZqnIpcBZsorWPcAdEfGnXPyLwCeAL6QmMRGxISKqb+NNJ+uE2QeYT80m9h4pBrAoNa+rm9mLC5WpYFKMiCUR8YWI6BsRfSLirOqCmVkb1QAdLZIE3AbMioif5uIjgW8Bn8o/8SKpj6TytL4XWYfK3NQ8XilpRDrnOcC96bCJwKi0PioXr1Mxvc97SfqrpHckLZZ0byqQmbVVDdP7fARwNnCMpBlp+TjwS6AbMDnFbk77HwW8IGkG8Efg/IhYlrZdCPwGmENWg6y+D3ktcLyk2cBx6XO9irmn+HvgV8Cn0+czgDuBw4o41sxamwZ6eDsinqD2Nvb9dex/D1lTu7Zt04ADaokvBY4tpVzF3FPcKSJ+GxEVafkd0KmUi5hZ6xJR3NIS1ffuc6+0+oCkMcAEsr8Rp1NHJjezNqKNvvs8nSwJVn/7r+a2BTWfMjezNqTwgy0tV33vPg/akQUxsxaiuE6UFquo8RQlHQAMIXcvMSLGN1ahzKw5U9scJaeapCuBo8mS4v3AicATZO8Xmllb1IprisX0Pn+OrEt7YUR8CTgI6NGopTKz5q2qyKUFKqb5vC4iqiRVSOpO9prMgEYul5k1V211kNmcaZJ6AreS9UivBp5s1FKZWbPWJnufq0XEhWn1ZkkPko1b9kLjFsvMmrW2mBQlHVLfturBIc3MWpP6aoo/qWdbAMc0cFmYNb8Ph377goY+rTWibg/PL7yTNRtxQYcGOU+bbD5HxEd3ZEHMrIUI2uxrfmZmtWuLNUUzs7q0yeazmVmdWnFSLGbkbUk6S9L30uc9JR3a+EUzs2arYUbebpaKec3vRuBw4Mz0eRXZSNxm1gYpil/qPY80QNIjkmZKelnSZSneS9JkSbPTz51TXJKulzRH0gv5xwYljUr7z5Y0KhcfJunFdMz1aQ6XehWTFA+LiIuA9QARsRxomH59M2uZqlTcUr8K4BsRMQQYAVwkaQgwBpgSEYOBKekzZIPRDE7LecBNsHlA7CvJpkg5FLiyOpGmfc7NHTeyUKGKSYqb0gxakQrQhxb7qreZNYSGqClGxILql0AiYhXZfPL9gZOBcWm3ccApaf1kYHxkpgI907SlJwCTI2JZqrRNBkambd0jYmqaJnV87lx1KiYpXg/8GdhV0jVkw4b9dxHHmVlrVfw9xd6SpuWW82o7naSBwMHAU0DfNG0pwEKgb1rvD7yVO2xeitUXn1dLvF7FvPt8h6TpZMOHCTglImYVOs7MWqkiaoE5SyJieH07SOpKNkvf5RGxMn/bLyJC2rEPABXT+7wnsBb4K9nE0mtSzMzaqgbqfZbUniwh3hERf0rhRanpS/q5OMXnU3PYwj1SrL74HrXE61VM8/lvwH3p5xRgLlsmmjazNkhVxS31niOrEt4GzIqIn+Y2TQSqe5BHAffm4uekXugRwLupmT0J+JiknVMHy8eASWnbSkkj0rXOyZ2rTsU0nz+41Rc5BLiwjt3NzIp1BHA28KKkGSn2HeBa4G5Jo4E3gdPStvuBjwNzyFqvXwKIiGWS/gt4Ju13dUQsS+sXArcDnckqcwUrdCW/0RIRz0o6rNTjzKwVaYC7fBHxBFumUN7asbXsH8BFdZxrLDC2lvg04IBSylXMxFVfz30sAw4B3i7lImbWipTW0dLiFFNT7JZbryC7t3hP4xTHzFqEtpoU00Pb3SLimzuoPGbWErTFpCipXURUSDpiRxbIzJo3UbhnuSWrr6b4NNn9wxmSJgJ/ANZUb8w9U2RmbYnvKdIJWEo2J0uQ/aEIwEnRrK1qo0lx19Tz/BJbkmG1VvwrMbOCWnEGqC8plgNdqf05olb8KzGzQtpq83lBRFy9w0piZi1HG02KrXcOQzPbftF2e5+3ec3GzAxomzXF3AvVZmY1tNV7imZmtXNSNDNLWvD0pcVwUjSzkgg3n83ManBSNDPLa8VJsZg5WszMamq4iavGSlos6aVc7C5JM9LyRvVUBZIGSlqX23Zz7phhkl6UNEfS9WlOFiT1kjRZ0uz0c+dCZXJSNLPS1DHxfW1LEW4HRtY4fcTpETE0IoaSDWidH3zmteptEXF+Ln4TcC4wOC3V5xwDTImIwWQT740pVCAnRTMrXQPVFCPiMaDWZ6JTbe804M76zpGmQe0eEVPTPC7jgVPS5pOBcWl9XC5eJydFMytZCVOc9pY0LbecV8JljgQWRcTsXGyQpOck/UPSkSnWH5iX22deigH0TVOdAiwE+ha6qDtazKxkJfQ+L4mI4dt5mTOpWUtcAOwZEUslDQP+Imn/Yk8WESEVLrmTopmVZgc8vC2pHfAZYNjmy0ZsADak9emSXgP2AeYDe+QO3yPFABZJ6hcRC1Ize3Gha7v5bGala6B7ivU4DvhXRGxuFkvqkybTQ9JeZB0qc1PzeKWkEek+5DnAvemwicCotD4qF6+Tk6KZlaT6jZaG6H2WdCfwJLCvpHmSRqdNZ7BtB8tRwAvpEZ0/AufnBq65EPgNMAd4DXggxa8Fjpc0myzRXluoTG4+m1nJVNUw7eeIOLOO+Bdrid1DHXPOR8Q04IBa4kspcRhEJ0UzK40HhDAzq8nvPpuZ5Tkpmplt4ZqimVmek6KZWdKGZ/MzM9uGR942M9tatN6s6KRoZiVzTbGN+e5nH+HD+73J8tWdOfMXp2+z/awjZzByaDaaUXlZFQN3XcEJPxjFynWdtvua7csrueq0v7Nf/3d4d20nrvj9cSxY0X3z9r49VnHX1+7i1inDuePxodt9nVZpcQVlP1yKlleCRNVJXYjPdK+xi2asp+x770C/7H/5qg/vRJzd471dd2Nk1529EbqXUfnd3rBb7p/UogrKRy+g6pwexGnd6z5PS9PKH95utHefaxtmvKX42/R9uex/T6pz++8eH8pZN5zKWTecyq8mHcZzr/crOiH267mSm87d9p30T/3HLFat68hnf/x57nziQC4+8aka2y8/6UmefHXP0r5IW1Euqs7fmcqxu1N5Q1/K7l0Nb27aZrf4YEcqf92Pyl/3Ky0hLqyg/OuLtgnrgdXQrYzK8btT9dlulN26osb2spuXE4du/x/K5qyE8RRbnMasKd4O/JJsFNwW5bk3dqdfz5VF7XvCQXOY9Pzemz+PHPoqp3/oRdqXV/HSW7ty3b1HUhWF//Z85ANvcOuUbNi5v7+0F//vU0+Q/TkWHxnyOm8v78b6ja7Y12qX8mwB2KmM2LM9WlJBvK99UYfr4TWU/XkVVASxX0eqLt0ZylX4uH+uo2pUllzjqJ0ou2F5dq9NQv+3Nqs1dip8npaopSa8YjRaTbG+YcZbi47tNzFin7d45KW9ABjYZznHH/gaX7n5FM664VSqQpub2YX06b6GRSu6AlBZVcbq9R3osdN6OnfYxDkfmcFvpmzvOJ1tzMIKNGcjsV/HbTZp5kbKz1tA2bcXwxsbs+Cbm9Cja6j8RV8qf90PykBT1hR1KS2thD4pGZcLupTByipYV0XZhJVUnfMem+fNVZAl/2KWFqjJqx5pePLzADp0KTjRVrNy5H5v8sKbu21uOv/H3vPZr/87jLsom2enY/sKlq/uDMB1Zz3I7juvol15Fbv1XMXvLvkDABP++UHum75fndc499hp3PnEB1m3sbhaT5u2rory779D1YU7ZwkqJwZ3oPL3u0PnMvTUOsqvXELluN3Rc+vR7E2UX7Qw23FDQM8yAii78h20sAI2BSyupPyr2aj2VZ/uRozsWmcxysa/S9Vnu0Hn1jsynztaGlFE3ALcAtCl94AW9av+2EFzeCjXdBbB357dlxsnHbbNvt/6XTa5WL+eK/neqY9wwa0n19j+zsou9O25msUru1JeVkXXTht5d20nDhiwiGM++BoXnziVbp02UhViY0U7/vDkNqMktW0VQdlVS6g6tgtx5E7bbs8lyTisM1y/DN6thIA4vgtVX+m5zSFV3++TrSysoPy6pVT+tOb0HrFLObxTCX3aQWXAmiroXoZmbUSPrYVbV8DqKlQmqjqIOKVbg37lJtWi/qWWpsmTYkvVpeMGDh60gO/dtWWotmde68+Pzn6QO584kOVrOtO983p26riJhSsK/2N4bNZATjrkVV78924cc8Bcpr22OyDOu2XL5GPnHvsMaze2d0LcWgRlP14K72tPfK6OXt5llbBzGUjwrw1QBXQvIw7plPVKf7Yb7FwOKythXUDfwv804kOdKXtoDVVDOqLH1hJDO4FE5c+3JM+ycSuIzmWtKiH64e026L/OeJhhg96mZ5f1/HXMb7n14eG0K8vuLP/p6WyenKP3f52nZu/B+k1bmrWvL+7FzQ8dyg1fvg8pqKgq40f3HllUUpw4bT++f9rfueebv2fl2o5ccefxjfPlWqOXNlD28FpiUPstTdwv94TFFQDEJ7uhx9ZS9tfVUA50UPb4jATva0/VF3tQPmZxlijbQeUlvYpLiid2Rdcuofyct7Ne6Ct6N+KXbEYiGmyQ2eZI0Ug3Q9Mw40cDvYFFwJURcVt9x3TpPSA+8MmvNUp5rHF0O3t+4Z2s2Zh2wR2semXhe+oS79Zzjzj4qMuK2vfxv35ren2z+UkaC3wCWBwRB6TYVWQT27+TdvtORNyftn0bGA1UApdGxKQUHwn8guzP3m8i4toUHwRMAHYBpgNnR8TG+srcmL3PZ0ZEv4hoHxF7FEqIZtZyNNQcLWSP7o2sJf6ziBialuqEOIRs7pb90zE3SipPk1n9CjgRGAKcmfYF+GE6197AcrKEWq/W2z1mZo0jgKoobil0qtIe3TsZmBARGyLidbJJqg5Ny5yImJtqgROAk9PMfseQTXIFMA44pZbz1uCkaGalK36K096SpuWW84q8wsWSXkhvxlU/q9cfeCu3z7wUqyu+C7AiIiq2itfLSdHMSlZC83lJRAzPLbcUcfqbgPcDQ4EFwE8a75tsy73PZlayxux9jojNL5pLuhW4L32cDwzI7bpHilFHfCnQU1K7VFvM718n1xTNrDTFNp23M29K6pf7+GmgelCZicAZkjqmXuXBwNPAM8BgSYMkdSDrjJkY2aM1jwCfS8ePArYdjWUrrimaWUmyh7cbpqaYf3RP0jzgSuBoSUPJ0uobwFcBIuJlSXcDM4EK4KKIqEznuRiYRPZIztiIeDld4j+BCZJ+ADwHFHwKxknRzErXQKPkRMSZtYTrTFwRcQ1wTS3x+4H7a4nPJeudLpqTopmVrKFqis2Rk6KZlaaVj7ztpGhmJWrd7z47KZpZ6dx8NjNLonVPR+CkaGalc03RzCyn9eZEJ0UzK52qWm/72UnRzEoTNNjD282Rk6KZlUSEH942M6vBSdHMLMdJ0cws8T1FM7Oa3PtsZrZZuPlsZrZZ4KRoZlZD6209e44WMyudIopaCp4nm8J0saSXcrEfSfpXmuL0z5J6pvhASeskzUjLzbljhkl6UdIcSdenOZ+R1EvSZEmz08+dty1FTU6KZla6iOKWwm4HRm4VmwwcEBEHAq8C385tey0ihqbl/Fz8JuBcssmsBufOOQaYEhGDgSnpc72cFN6EAGcAAAYcSURBVM2sNBFQWVXcUvBU8RiwbKvYQ7kJ7KeSTU1apzT7X/eImJpm8BsPnJI2nwyMS+vjcvE6OSmaWemKryn2ljQtt5xX4pW+DDyQ+zxI0nOS/iHpyBTrD8zL7TMvxQD6RsSCtL4Q6Fvogu5oMbPSFd/7vCQihm/PJSRdQTaV6R0ptADYMyKWShoG/EXS/sWeLyJCUsGCOymaWWkCaOQ5WiR9EfgEcGxqEhMRG4ANaX26pNeAfYD51Gxi75FiAIsk9YuIBamZvbjQtd18NrMSBURVcct2kDQS+BbwqYhYm4v3kVSe1vci61CZm5rHKyWNSL3O5wD3psMmAqPS+qhcvE6uKZpZaYKiOlGKIelO4Giye4/zgCvJeps7ApPTkzVTU0/zUcDVkjaRPSl5fkRUd9JcSNaT3ZnsHmT1fchrgbsljQbeBE4rVCYnRTMrXQO90RIRZ9YSvq2Ofe8B7qlj2zTggFriS4FjSymTk6KZlc6v+ZmZVfOAEGZmWwTgocPMzHJcUzQzqxYN1vvcHDkpmllpAmI7n0FsCZwUzax0jfxGS1NyUjSz0vmeoplZEuHeZzOzGlxTNDOrFkRlZVMXotE4KZpZaXbA0GFNyUnRzErnR3LMzDIBhGuKZmZJhGuKZmZ5rbmjRdGMutYlvUM2Om5r0xtY0tSFsJK01v9m74uIPu/lBJIeJPv9FGNJRGw9r3Oz1qySYmsladr2zmhmTcP/zdouT1xlZpbjpGhmluOkuGPc0tQFsJL5v1kb5XuKZmY5rimameU4KZqZ5TgpNiJJIyW9ImmOpDFNXR4rTNJYSYslvdTUZbGm4aTYSCSVA78CTgSGAGdKGtK0pbIi3A60qIeNrWE5KTaeQ4E5ETE3IjYCE4CTm7hMVkBEPAYsa+pyWNNxUmw8/YG3cp/npZiZNWNOimZmOU6KjWc+MCD3eY8UM7NmzEmx8TwDDJY0SFIH4AxgYhOXycwKcFJsJBFRAVwMTAJmAXdHxMtNWyorRNKdwJPAvpLmSRrd1GWyHcuv+ZmZ5bimaGaW46RoZpbjpGhmluOkaGaW46RoZpbjpNiCSKqUNEPSS5L+IGmn93Cu2yV9Lq3/pr7BKiQdLelD23GNNyRtM+tbXfGt9lld4rWukvTNUstotjUnxZZlXUQMjYgDgI3A+fmNkrZrHu+I+EpEzKxnl6OBkpOiWUvkpNhyPQ7snWpxj0uaCMyUVC7pR5KekfSCpK8CKPPLNL7jw8Cu1SeS9Kik4Wl9pKRnJT0vaYqkgWTJ92uplnqkpD6S7knXeEbSEenYXSQ9JOllSb8BVOhLSPqLpOnpmPO22vazFJ8iqU+KvV/Sg+mYxyXt1xC/TLNq21WzsKaVaoQnAg+m0CHAARHxekos70bEf0jqCPyfpIeAg4F9ycZ27AvMBMZudd4+wK3AUelcvSJimaSbgdUR8eO03++Bn0XEE5L2JHtr5wPAlcATEXG1pJOAYt4G+XK6RmfgGUn3RMRSoAswLSK+Jul76dwXk00odX5EzJZ0GHAjcMx2/BrNauWk2LJ0ljQjrT8O3EbWrH06Il5P8Y8BB1bfLwR6AIOBo4A7I6ISeFvS32s5/wjgsepzRURd4woeBwyRNlcEu0vqmq7xmXTs3yQtL+I7XSrp02l9QCrrUqAKuCvFfwf8KV3jQ8AfctfuWMQ1zIrmpNiyrIuIoflASg5r8iHgkoiYtNV+H2/AcpQBIyJifS1lKZqko8kS7OERsVbSo0CnOnaPdN0VW/8OzBqS7ym2PpOACyS1B5C0j6QuwGPA6emeYz/go7UcOxU4StKgdGyvFF8FdMvt9xBwSfUHSdVJ6jHg8yl2IrBzgbL2AJanhLgfWU21WhlQXdv9PFmzfCXwuqRT0zUk6aAC1zAriZNi6/MbsvuFz6bJl35N1iL4MzA7bRtPNhJMDRHxDnAeWVP1ebY0X/8KfLq6owW4FBieOnJmsqUX/PtkSfVlsmb0vwuU9UGgnaRZwLVkSbnaGuDQ9B2OAa5O8S8Ao1P5XsZTPFgD8yg5ZmY5rimameU4KZqZ5TgpmpnlOCmameU4KZqZ5TgpmpnlOCmameX8f1Vp5A57vxS5AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "az3_eF6ou6Zy"
      },
      "source": [
        "#### Gradient Boosted Decision Trees w/ Word2Vec Representation - Precision Recall Curve"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yCu3UbntxuLb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 409
        },
        "outputId": "46af2f27-4c58-4b97-8338-cb715324f390"
      },
      "source": [
        "create_pr_curve('Gradient Boosted Decision Trees (Word2Vec) Precision-Recall Curve', xg_rf_w2vec_preds, y_test)"
      ],
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaYAAAGICAYAAAAK10RFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3wUdfrA8c+TShoJgQAhhCQECASlKCIoKoKFKqd4KtyhKCc27vydBRUVEBUL6p2e2A+xVyxUsVAEhJMgWAi9hh5KgNBCku/vj5mETUgn2dndPO/XK6/sznx35pn6zMw+OyPGGJRSSilP4ed0AEoppZQrTUxKKaU8iiYmpZRSHkUTk1JKKY+iiUkppZRH0cSklFLKo3hcYhKRzSJymf16lIi85XRMnkxEjIi0cGC8r4nIoxVot1JEurshJEeJSIyIrBaREAfGPVREFrp7vC7jbyQiq0Qk2KkY7DiaiUi2iPiX0+4vIvKtu+KqaSIyWUSesF93F5FtTsd0piqVmETkBhH5n4gcEZE99us7RURqIjhjzHhjzN/OdDgikmjvwAPKaDNWRE7aK3a2vaENPNNxlxNXje5QRGSeiBwXkcMickhElonIg9WxAzHG3G6MebwC7doaY+ad6fhc2cmuYDnl2dNY8H5UdY6rEh4EJhtjjonIIBFZVSzm70rp9mB1BiEiDUXkIxHZISIHRWSRiJxv9+tib7vhJXxuuYiMqMo4jTG7gbnA8DLimiwiOfYy2m9Pe+uqjK+MOLYaY8KNMXnltPvAGHNFdY67gMs2ly0ie0XkCxGJrYlxVYVY/iEif9jrwjYR+UxEznY6NlcVTkwici/wIjABaAw0Am4HLgSCSvlMmUcuHugTe8UOB/4PeF9EGjkd1BkaYYyJAGKBe4EbgJk1dTDhDnayK1hOC7CmMdz+G1/QrqwDkepkJ/qbgPftTj8CrUUkxiWO9kBIsW5d7baVGVd50xQOLAXOBaKBd4AZIhJujFkCbAOuLTbMs4BU4KPKxFLMB8Bt5bR51l5mTYE9wOTiDewdp8ddyamkEfZ0tsBaHs85HI+rF4G7gX9grR+tgK+AvpUdUI1uX8aYcv+ASOAIMLCcdpOBV4GZdvvLsCZ4OXAIyADGFvvMEGALsA94GNgMXGb3Gwu879K2C/ATkAX8CnR36TcPeBxYBBwGvgUa2P22AgbItv+6lhB7kXHZ3fYAF7i8vxVYD+wHpgJNXPpdgLVDOGj/d/3cUGCjHdcm4C9AG+A4kGfHlGW3DcZakbcCu4HXgBCXYd0P7AR2ALfY09WilOUxD/hbsW7NgKNAP/u9H9bR/gZ7GXwKRLu07+YyzzOAoS7L+gn7dQNgut1mP1ay8LP7uS7PYODfduw77NfBdr/uWDvNe+35vhO4uQLrZuE0Aon2/Bhmz78f7e63AKuAA8BsIMHl862B7+y41wDXufTrA6Tby207cF8pMVwMrC/WbQP29gJ0xjqjeKdYt6NAINb29S6QibUtPOIy/4ZirdP/spfPE0B9rPXvEPAz1nq/sIx5dAg41349CphTrP+zwJcVmB8hwPN2jAeBhdjrJhBgT09CKTEUri/2+75AtssyfNKezmNYO/RKx+Gy/ANK2+5cui90GV5Z2+48StmvVGSbA+4EVlZwfStr/n4G7LK7/wi0LWneYm9HpcTWEmt/07kS8RefVwa4C1hnz9NXgeeKDeNr4B77dRNgCta6vQn4R3nbtDGmwompF5BbsMDLaDfZnnEXYu3w6tgz6mz7fTusne2f7PapWDvli7F2Wi/Y4zktMQFxWBtmH3tYl9vvY1xm6AasI4AQ+/3TxXZYpcZfbFyCteFkAVF2tx7AXuAcO9b/cGrHF4210xuCtYEOst/XB8KwdgwpdtvYgpWq+EK3u/0La6cTDUQA04CnXJbDbuAse7gfUsnEZHf/EXjGfn03sATrKDYYeB34yO6XgLUxDsLagdYHOpSwMTyFlUAD7b+LALH7bXZZnuPscTUEYrAS3uMuG1Su3SbQXs5HgXrlrHOF0+iynN+1508IMADrYKKNvWweAX6y24dhJdub7X4d7WWcavffCVxkv64HnFNKDHcBM4p1ext40X59nz1dtxbrNsd+/S7WxhxhT8NaYJjLOpIL/N2OMQT4GOsAIsxeF7ZTSmICOmAdAEXa7+Pt4cXb7/2wDgj+VIH5MdGe33GAP9YOPdhlXL8BV5WxbyhYX8Kx1t0FLstwK9DWHm9kVeJwWf4BVHC7o4xtt7z9SgXWx/rA98DXFVzfSp2/WAdXEZw6uFtRyrztTumJ6XZgS0W3p5L2Ufb8/c6ebyFY++4MTm3v9bAOLppgrVvLgNFYV9WaYx0oXFluzimvgT2yvwK7inUrOIo+BlzsMoPeLWdY/wb+Zb8eDXzs0i8MyKHkxPQA8F6xYc0GbnKZoY+49LsT+KbYDqu8xJRjT9MRrCOLkS79/4t1KaLgfThw0h72EODnYsNbbC/UMHuYA3E58ylloYs97mSXbl2BTfbrSbhsFFgbS1US08fAm/brVUBPl36x9nQFAA9hH0mXs6MZh7VjPS0OiiamDUAfl35XAptdNqhjrssI68ypS0U3JJfl3Nyl/yzsnbz93g/7yB64Hnvn6NL/dWCM/Xor1uWpuuXE8DAu67HLsl1uv/4a60CqdbFuY7B2QDnYOye7323APJfhbHXp528vn9Yu3cZTQmIC6gK/Aw8V6/49MMp+fTnW0WxgWfPDnm/HgPZlzIdFwI1lrC/HsbaFXVgHX8kuy3CcS9sqxcHpianc7Y4ytl2X2Ercr5SxPh7FOkA3wAqg2ZlMVwnjiLKHXXCwMZmKJaaHgSUV3Z6Kzyv7vQF6uLwXrO2kIAfcyqkDrvNxWXftbg8Bb5c3jRW9lrsPaOB6TdEYc4ExJsru5zqcDNcPisj5IjJXRDJF5CBW1m5g927i2t4Yc8QeXkkSgD+LSFbBH9ZlJtcvFne5vD6KlTwq41NjTJQxJgxIBm4UkYLr5k2wTrELYs22Y40r3s+2BYizp+l6rOneKSIzyvjSNwYIBZa5TOM3dveCGFznb/FxVlQc1qUEsObrly7jW4WVlBthHV1vqMDwJmCdlXwrIhvL+EK/+HzaYncrsM8Yk+vyvirLEIrOowTgRZfp24+1McXZ/c4vtk79Bes7VLB2an2ALSIyX0S6ljK+A1hHs65+BNqJSD2sS9CLjTGrgVi7Wze7TQOspFB8vsSVMj0xWDveMtcDsaoDp2HtiJ4q1vsdrB0y9v+PjTEnKXt+NMC6AlLW+hCBlQxK85y9fTU2xlxljHEdVvFldiZxUIntrtRt1+V9ifsVsSpTSyq6+YcxJhLrClE9rKsRVZ4uEfEXkadFZIOIHMI62INT+9GK2kfR/WVVue6zDdaB7iC702Cs7xvBmt4mxaZ3FNa+pUwVTUyLgRNYl0XKY4q9/xDr6CjeXlivYe0YwLpUEl/QUERCsU5/S5KBdcYU5fIXZox5ugoxlf8BYzZjHW33tzvtwJrRBbGG2bFuL97P1szuhzFmtjHmcqyVYjXwZilx7cU6amrrMo2RxvoiFYrNL3sclSIi8VhfjC+wO2UAvYvN1zrGmO12v+TyhmmMOWyMudcY0xy4CrhHRHqW0LT4fGpmd6turvM1A7it2PSFGGN+svvNL9Yv3Bhzhz1dS40xA7AuPX6FdfmsJL9hnb2eCsCYjVjTNhzrqDHb7rXY7haOdVlzL9YZUPH5sr2U6cnEvhRXrH0huxjjK6xLdCUVJHwBNBWRS4FrsBIVlD0/9mKd8ZS4PtgHrS2wvvutiuLLrEpxFBlg6dudqzK33XKGf7spoejGpf/vWN8JTrSLjao6XYOx9r2XYV3mTLS7V7aA6Qes5d6pjDZHsA6OCzQuoU3x/dZHwLUikoB1ljTF7p6BdbXHdXojjDF9ygu0QonJGJMFPAa8IiLXikiEiPiJSAesU+ayRAD7jTHHRaQz1kwu8DnQT0S6iUgQ1iWh0mJ6H+gvIlfaRxB1xKrZb1pKe1eZQD7WNc4KsYfbC1hpd/oIuFlEOtgb/njgf3YCmwm0EpHBIhIgItdjfX82XazfeAywE9kJrO/U8u1h7sZaUYIAjDH5WBvPv0SkoR1HnIhcabf/FBgqIql2Eh9TiekJFZFLsC4h/WzHDNaBwpP2SlXwe5yCA5APgMtE5Dp7uurby7z4sPuJSAt74zuIdcaVX7ydPQ8fscfRAOtS7vsltKtOrwEPiUhbO9ZIEfmz3W861nIbIiKB9t95ItJGRILE+r1LpH02caiUaQJrfkaJSFyx7guAezh1EADWF9r3AGnGmGPGKm3+FGsZRNjL4R5KmS92+y+AsfYyTcWqCMSevkCs7eoY1mXu02K2zyY+x/oebIsxJq28+WEPZxLwgog0sbfBrnLqpwedsS7LVvUs3tWZxFEwH8ra7lyVuu1Ww3SAlfQbYR2wVXW6Iuxp2IeVNE5LghVhjFkHvAJ8ZO87g+z96A1y6irHCuAae91qgVVIVN5wl2Ml1reA2Xa+AGu7OCwiD4hIiD1NZ4nIeeUNs8JlmcaYZ7E2mJFYO9TdWNdHH8D6vqk0dwLjROQw1o6o8KjTGLMS64vjD7HOBg5gHeWVNP4MrKOGUViJJgOrQq3caTDGHMWu+rFPKbuU0vR6sU/NsapzFmElZIwx3wOPYh0N7MQ6srnB7rcP6IdVUbYPax71M8bsteO7B+vIbD9wCXCHPb45WIlvl4jstbs9gHVZbIl92v49kGKPZxbWd3Rz7DZzypt24GV73u+2PzsF6OWyw3oR64z2W7vdEqyjHowxW7EuZd1rx74Cq+y5uJZ2nNlYZwSvGGPmltDuCSAN6wzjd+AXu1uNMcZ8CTwDfGzPzz+A3na/w8AVWMtxB9Ylm2ewvmAG6zLXZvtzt2NddilpHDlY1/n/WqzXfKyzLdffqi2wu7mWif8d60h1o932Q6ydVGlGYJ1x7bLH+7ZLvwuw1sUrgCw5danpomLDeAfrTOFdl+kob37ch7XclmKtD89wavv7C9ZBwBk7wzgKlLXduY6rrG23OqYlB2sbe/QMputdrMuL27GqRJecQUj/AF7GKrTIwrp0eDXWZV+wiq9ysPYX73Dqslx5PsQ6o/uwoIN9ENUPqwBnE6eSV2R5AyuopFBKnQGxfp+0AOhojDnmdDzuZJ/dz8ea9uNOx6O8nyYmpZRSHsXbf2GtlFLKx2hiUkop5VE0MSmllPIobk1MIjJJrLuS/1FKfxGRl0RkvYj8JiLnuDM+pZRSznPL3ZddTMYqVXy3lP69sUqPW2KVLL9q/y9TgwYNTGJiYvVEqLzXwYOwfr31WgRSUiCshJ/ZHTkCa9ZAQeFPixYQWW4Fq1JeZdmyZXuNMTHlt/Q8bk1MxpgfRSSxjCYDsO61Z7B+xxMlIrHGmJ1lDTcxMZG0tLSymqjaoG/fU4np3nthwgQA8vIN+cYQ6O9ygeC+++D5563XKSkwvbp+T6mUZxCR6vixsyM87TumOIreM2sbRe9ZpVTpFi069XqY9YP1w8dP0v8/C3lrwaaibYcNK/lzSinHeVpiqjARGS4iaSKSlpmZ6XQ4yhMcPnzqdbJ1y7GIOoHE1QvhP3PWsfvQ8dP6A3DokJsCVEpVhKclpu0UvTllU0q5maIx5g1jTCdjTKeYGK+8jKqqW4TLDb43nLpJ86N9U8nNNzw9a3WJ/alb1w3BKaUqytMS01SsR02IfT+7g+V9v6RUoQsvPPX6v/8tfNmsfijDL2rOl8u3k7Z5/2n9i3xOKeU4d5eLf4R1k88UEdkmIsNE5HYRud1uMhPrRpbrse6yfac741Ne7g6Xe3S+/DIsXlz49s5Lk4mNrMPYaSvJW/ST1b+kzymlHOcT98rr1KmT0ao8RV4etGsH6enW++BgGDHCKnRITmbujMWse+pFhv06E/+cE1ab1FT47Tfw93cubqVqgIgsM8aU9ewlj6WJSfmWNWusS3P7SnsQsov69eGnn6BVq/LbKuVlvDkxedp3TEqdmZQUq/w7NbXMZnvikzUpKeWh3H3nB6VqXkqKdXnum2/g1VetRHXokFV9d+GFvNehD+Ny45lWtzGtnY5VKXUavZSnap0DR3K49Pl5tGlclw9vPR/rifBK+Ra9lKeUF6kXFsS9V6SweOM+Zv2xy+lwlFLFaGJStdLgzs1oE1uXJ2es4lhOntPhKKVcaGJStZK/nzC2fyrbs47x2vwN5X9AKeU2mphUrXV+8/r0b9+E1+ZvIGP/UafDUUrZNDGpWm1Un9b4iTB+5iqnQ1FK2TQxqVotNjKEET1aMOuPXSxav9fpcJRSaGJSimHdkmgWHcrYqSs5mZfvdDhK1XqamFStVyfQn0f7pbJuTzbvLfbah34q5TM0MSkFXNamIRe3iuFf369lb/YJp8NRqlbTxKQUICKM7pfKsZw8npu9xulwlKrVNDEpZWvRMJybL0zkk7QMftuW5XQ4StVampiUcvGPni2pHxbM2Kkryc/3/vtIKuWNNDEp5SKiTiAP9Erhl61ZfLViu9PhKFUraWJSqpiB5zSlfXwUT81aTfaJXKfDUarW0cSkVDF+fsJjV7Ul8/AJ/jNnndPhKFXraGJSqgQd4qO4rlNTJi3cxMbMbKfDUapW0cSkVCnuv7I1dQL8GTc9HV94oKZS3kITk1KliIkI5u7LWjJvTSZzVu9xOhylag1NTEqV4aYLEmnRMJxx09M5kasPFFTKHTQxKVWGQH8/xvRPZcu+o/x34Sanw1GqVtDEpFQ5LmoZwxWpjXh5znp2HTzudDhK+TxNTEpVwCN9U8nNNzw9Sx8oqFRN08SkVAU0qx/KbRc356sVO0jbvN/pcJTyaZqYlKqgO7onExtZhzFTV5Kn99FTqsZoYlKqgkKDAhjVpw0rdxzik6UZToejlM/SxKRUJfRrF8v5SdFMmL2ag0dPOh2OUj5JE5NSlSAijL2qLQePneSF7/SBgkrVBE1MSlVSm9i6/LVLAu8t2cLqXYecDkcpn6OJSakquOfyVtQNCWTs1JV6Hz2lqpkmJqWqICo0iPuuSGHJxv3M/H2X0+Eo5VM0MSlVRYM6N6NNbF2enJHOsRy9j55S1UUTk1JV5G8/UHDHweO8On+D0+Eo5TM0MSl1BjonRXNV+ya8Nn8DGfuPOh2OUj5BE5NSZ+ihPq3xF+HJGXofPaWqgyYmpc5QbGQII3q04JuVu1i4bq/T4Sjl9TQxKVUNhnVLoll0KI9NW8nJvHynw1HKq2liUqoa1An059F+qazbk817i7c4HY5SXk0Tk1LV5LI2DbmkVQz/+n4te7NPOB2OUl5LE5NS1UREGN0/lWM5eUz4Ru+jp1RVaWJSqholx4RzS7ckPl2Wwa8ZWU6Ho5RX0sSkVDX7e48W1A8LZuy0leTrAwWVqjRNTEpVs4g6gTzYuzXLt2bx5fLtToejlNfRxKRUDbimYxwd4qN4+pvVHD6uDxRUqjI0MSlVA/zs++hlHj7By3PWOx2OUl5FE5NSNaR9fBTXdWrKpEWb2JCZ7XQ4SnkNTUxK1aD7r2xNnQB/xk1L1wcKKlVBmpiUqkExEcHcfVlL5q/NZM7qPU6Ho5RX0MSkVA276YJEWjQMZ9z0dI6f1AcKKlUeTUxK1bBAfz/G9E9ly76j/HfhJqfDUcrjaWJSyg0uahnDlW0b8fKc9ew8eMzpcJTyaJqYlHKTR/qmkmcMT89a7XQoSnk0TUxKuUl8dCi3X9ycr1fsYOnm/U6Ho5THcntiEpFeIrJGRNaLyIMl9G8mInNFZLmI/CYifdwdo1I15Y7uLWgSWYcxX68kT++jp1SJ3JqYRMQfmAj0BlKBQSKSWqzZI8CnxpiOwA3AK+6MUamaFBLkz6i+bUjfeYiPl251OhylPJK7z5g6A+uNMRuNMTnAx8CAYm0MUNd+HQnscGN8StW4vmfHcn5SNM/NXkPW0Rynw1HK47g7McUBGS7vt9ndXI0F/ioi24CZwN9LGpCIDBeRNBFJy8zMrIlYlaoRIsLYq9py8NhJ/vXdWqfDUcrjeGLxwyBgsjGmKdAHeE9ETovTGPOGMaaTMaZTTEyM24NU6ky0ia3LX7sk8N6SLazedcjpcJTyKO5OTNuBeJf3Te1uroYBnwIYYxYDdYAGbolOKTe65/JW1A0JZOzUlXofPaVcuDsxLQVaikiSiARhFTdMLdZmK9ATQETaYCUmvVanfE5UaBD3XZHCko37mfH7TqfDUcpjuDUxGWNygRHAbGAVVvXdShEZJyJX2c3uBW4VkV+Bj4ChRg8nlY8a1LkZqbF1GT9jFUdzcp0ORymPIL6wz+/UqZNJS0tzOgylqmTp5v38+bXF/KNHC+65IsXpcJSPEJFlxphOTsdRFZ5Y/KBUrXJeYjQDOjThtR83krH/qNPhKOU4TUxKeYCHerchwE94Yka606Eo5ThNTEp5gMaRdbjr0hbMXrmbBeu01kfVbpqYlPIQw7olkVA/lMempXMyL9/pcJRyjCYmpTxEnUB/Hu2byvo92by7eIvT4SjlGE1MSnmQnm0ackmrGP793Vr2Zp9wOhylHKGJSSkPIiKM7p/KsZN5TPhmjdPhKOUITUxKeZjkmHBu6ZbEp8sy+DUjy+lwlHI7TUxKeaC/92hBg/BgxkxdSb4+UFDVMpqYlPJAEXUCebBXa1ZkZPHF8uL3OVbKt2liUspDXd0xjo7Nonh61moOHz/pdDhKuY0mJqU8lJ+fMLZ/W/YdOcF/5qx3Ohyl3EYTk1IerH18FNedG8+khZtYvyfb6XCUcgtNTEp5uPt7pRAS6M+46en6QEFVK2hiUsrDNQgP5v8ub8WPazP5YdUep8NRqsZpYlLKC9zYNYEWDcMZNz2d4yfznA5HqRqliUkpLxDo78eY/qls3X+U/y7c5HQ4StUoTUxKeYmLWsZwZdtGvDxnPTsPHnM6HKVqjCYmpbzII31TyTOGp2audjoUpWqMJialvEh8dCi3X9ycqb/u4OdN+50OR6kaoYlJKS9zR/cWNImsw5ipK8nT++gpH6SJSSkvExLkz8N9U1m18xAf/bzV6XCUqnaamJTyQn3ObkyX5tE89+0aso7mOB2OUtVKE5NSXkhEGHtVWw4dO8kL3611OhylqpUmJqW8VOvGdRnSJYH3l2xh1c5DToejVLXRxKSUF/vn5a2IDAnksWkr9T56ymdoYlLKi0WFBnHP5a1YsnE/s1fudjocpaqFJialvNygzs1o1Sic8TNXcSJX76OnvJ8mJqW8XIC/H4/2s+6j9/aizU6Ho9QZ08SklA+4qGUMl7VpyMtz1pN5+ITT4Sh1RjQxKeUjRvVpw4ncPJ7/do3ToSh1RjQxKeUjmseEc1PXRD5Jy+CP7QedDkepKtPEpJQP+XvPltQLDeJxfQy78mKamJTyIZEhgdxzeSv+t2k/3/yxy+lwlKoSTUxK+ZgbzosnpVEE42et0sewK6+kiUkpHxPg78fo/qlk7D/GpEX6GHblfTQxKeWDLmzRgMvaNGLinPXsOXzc6XCUqhRNTEr5qIf7tiEnL5/nZmv5uPIumpiU8lFJDcK4+cIkPlu2TcvHlVfRxKSUDxvRowXRoUGMm6bl48p7aGJSyofVrRPIvVek8PPm/cz8XcvHlXfQxKSUj7v+vHhaN45g/EwtH1feQROTUj7O308Y3T+V7VnH+O9CLR9Xnk8Tk1K1wAXJDbgitRET565nzyEtH1eeTROTUrXEw33bcDIvn2e1fFx5OE1MStUSCfXDuOXCJD5fto3ft2n5uPJcmpiUqkVG9GhBg/AgHpu2UsvHlcfSxKRULRJRJ5D7rkghbcsBpv+20+lwlCqRJialapk/d4qnTWxdnp61WsvHlUfSxKRULePvJ4zuZ5WPv/njRqfDUeo0mpiUqoW6JtenV9vGvDJvA7u1fFx5GE1MStVSo/q0IS/f8Mw3q50ORakiNDEpVUs1qx/KLd2S+OKX7fyakeV0OEoV0sSkVC1216XJNAgPZtx0vfu48hyamJSqxSLqBHL/la1YtuUA07R8XHkITUxK1XLXnhtP2yZ1eXrmKo7laPm4cp7bE5OI9BKRNSKyXkQeLKXNdSKSLiIrReRDd8eoVG1SUD6+4+Bx3tDyceUB3JqYRMQfmAj0BlKBQSKSWqxNS+Ah4EJjTFvg/9wZo1K10fnN69Pn7Ma8Nn8DOw8eczocVcu5+4ypM7DeGLPRGJMDfAwMKNbmVmCiMeYAgDFmj5tjVKpWeqh3G/KM4dlv9O7jylnuTkxxQIbL+212N1etgFYiskhElohIr5IGJCLDRSRNRNIyMzNrKFylao/46FD+1i2JL5dvZ/nWA06Ho2oxTyx+CABaAt2BQcCbIhJVvJEx5g1jTCdjTKeYmBg3h6iUb7rz0hbERGj5uHKWuxPTdiDe5X1Tu5urbcBUY8xJY8wmYC1WolJK1bDw4ADuvzKF5VuzmPrrDqfDUbWUuxPTUqCliCSJSBBwAzC1WJuvsM6WEJEGWJf2tFRIKTe59pymnB0XydOzVnM0J9fpcFQt5NbEZIzJBUYAs4FVwKfGmJUiMk5ErrKbzQb2iUg6MBe43xizz51xKlWb+fkJo/unslPLx5VDxBeuI3fq1MmkpaU5HYZSPuWuD3/hh1W7mXNvd5pEhTgdjqokEVlmjOnkdBxV4YnFD0opD/BQ79bkG/Tu48rtNDEppUrUtF4owy9qztcrdrBsi5aPK/fRxKSUKtUd3ZNpaJeP5+d7/2V/5R00MSmlShUWHMDIXq35NSOLr38t/ssOpWqGJialVJmu6RhHu6aRPDNrjZaPK7cIKK+BiNxYmQEaY96tejhKKU/jZ999/NrXFvPa/I3cc3krp0NSPq7cxARMrsTwDKCJSSkf0ykxmv7tm/D6/A1cf148cVo+rmpQRS7lJVXir3nNhKmUctqDvVsD8MwsLR9XNavcMyZjzBZ3BKKU8mxxUSHcdnFzXpqznpsuSODchGinQ1I+SosflFIVdnv3ZBrVDeaxaVo+rmpORYofNmF9d1QRxhiTfGYhKaU8VWhQAA/0as09n/7Kl8u3M/Dcpk6HpHxQRYof5lPxxKSU8nF/6hDHO4u38Mw3q4ucb8UAACAASURBVOl1VmPCgiuyG1Gq4iryHdNQN8ShlPISBeXjA1/9idfmb+DeK1KcDkn5GP2OSSlVaecm1GNAhya88eNGth046nQ4ysdU6RxcRNoDKUCd4v30B7ZK1Q4P9GrN7JW7eHrWal4efI7T4SgfUqnEJCJRwAygS0En+7/rd1CamJSqBZpEhXDbxcm8+MM6brpgP+clavm4qh6VvZQ3HqgPXIyVlK4GegAfYD3+vHO1RqeU8mi3X5JMbGQdxmn5uKpGlU1MV2IlpyX2+23GmHnGmBuB74G7qzM4pZRnCwny54Ferfl9+0Gm/LLN6XCUj6hsYooFNhpj8oDjQIRLvy+AvtUVmFLKOwzo0ISOzaJ4dvYask/o3cfVmatsYtoFRNmvtwBdXfq1qJaIlFJeRcQqH888fIJX5613OhzlAyqbmBZyqvDhPWCMiLwuIhOBCcDs6gxOKeUdOjarx9Ud43hzwSYy9mv5uDozlU1Mj3Eq+UwAJmJdvhsETAX+Xn2hKaW8ycheKfiL8LTefVydoUolJmPMBmPMAvv1SWPMvcaYpsaYaGPMYGPMvpoJUynl6WIjQ7j9kmRm/L6T/23UXYGqukolJhEJFJGwUvqFiUhg9YSllPJGwy9uTpPIOoybnk6elo+rKqrspby3gDdL6fe6/aeUqqVCgvx5oHdrVu44xJRlWj6uqqayielS4OtS+k0Fep5ZOEopb3dV+yaco+Xj6gxUNjE1BPaU0i8TaHRm4SilvJ2IMKZ/W/Zmn2DiXC0fV5VX2cS0Bzi7lH5nA/qNp1KK9vFRXHNOHP9dsImt+7R8XFVOZRPTdOBREWnn2lFEzgYeBqZVV2BKKe828srW+PsJT81a5XQoystUNjGNBrKAZSLyk4h8KiKLgF+Ag8Aj1R2gUso7NY6sw53dk5n1xy6WaPm4qoTK/o5pL3Ae8BTW3cU72P+fBM6z+yulFAC3XtycuKgQxk3T8nFVcZV+gq0xJssYM9oY09UY08oYc4ExZqwx5mBNBKiU8l51Av15sHdr0nce4rO0DKfDUV6iSo9WF5EGItJPRG4SkWi7Wx0R0Ue1K6WK6Nculk4J9Xju2zUcPn7S6XCUF6jsnR9ERCYA27B+tzQJSLR7f41VAKGUUoVEhNH9U9mbncPLWj6uKqCyZzgPASOAccD5nHq0OlgVef2qKS6llA9p1zSKa89tytsLN7Nl3xGnw1EerrKJ6W/AOGPMeKxKPFfrgeRqiUop5XPuvzKFAH9h/EwtH1dlq2xiiuPUY9WLywFKvMGrUko1qluHuy5tweyVu/lpgxbwqtJVNjFtB84qpV97YNOZhaOU8mXDuiVp+bgqV2UT02fAaBG50KWbEZFWwL3Ax9UWmVLK59QJ9GdUnzas3nWYT5Zq+bgqWWUT01hgNfAjsM7u9hnwO9Z3TE9XW2RKKZ/U5+zGdE6M5vlv13BIy8dVCSp754djQHdgKPAT8D2wFBiOVZF3R/WGp5TyNQXl4/uP5vDyHC0fV6er7O+YGgD5xpj3jDF/NcZcAQwDGgBrgRdqIEallI85Ky6SP5/blLcXbWLzXi0fV0WVm5hEJFhEXhSRw8BuYJ+I3GH3+yuwAZgAZAC9ajJYpZTvuO/KFIL8/XhSy8dVMRU5YxoN/B1YjJWAvgNeFJH/AO9i3VV8gDHmfGPMdzUWqVLKpzSMqMNdPVrwXfpuFq3X8nF1SkUS0/XAK8aYK4wxDxpjrgduB+7CSlLtjDH6HCalVKXdcmES8dEhPD49ndy8fKfDUR6iIokpHviyWLcv7P8vGGNyqjckpVRtUSfQn1G9rfLxj7V8XNkqkpgCgcPFuhW8z6zecJRStU2vsxrTOSmaF75by8FjWj6uKl6VFycizQv+gOYldbf7KaVUhYkIo/ulcuBoDv/5YV35H1A+L6CC7T4vpftXJXTzr2IsSqla6qy4SK47N57JP21m8PnNaB4T7nRIykEVSUw313gUSqla774rU5jx+07Gz1zFWzed53Q4ykHlJiZjzDvuCEQpVbvFRAQzokcLnp61mgXrMrmoZYzTISmH6KPQlVIe4+YLE2kWHarl47WcJiallMcIDrDuPr52dzYfafl4raWJSSnlUa5s24guzaN54ds1HDyq5eO1kSYmpZRHscrH25J17CQvavl4reT2xCQivURkjYisF5EHy2g3UESMiHRyZ3xKKeelNqnLDefF8+7izWzIzHY6HOVmbk1MIuIPTAR6A6nAIBFJLaFdBHA38D93xqeU8hz3XpFCnUB/npyhdx+vbdx9xtQZWG+M2WjfY+9jYEAJ7R4HngGOuzM4pZTnaBAezN97tGDO6j3MX6t3P6tN3J2Y4rCe21Rgm92tkIicA8QbY2a4MzCllOcZemEiCfVDeULLx2sVjyp+EBE/rKfg3luBtsNFJE1E0jIz9WhKKV8UHODPw33asG5PNh/+vNXpcJSbuDsxbcd6jEaBpna3AhHAWcA8EdkMdAGmllQAYYx5wxjTyRjTKSZGfyGulK+6PLURFyTX54Xv1pJ1VJ+yUxu4OzEtBVqKSJKIBAE3AFMLehpjDhpjGhhjEo0xicAS4CpjTJqb41RKeQgR4dF+qRzS8vFaw62JyRiTC4wAZgOrgE+NMStFZJyIXOXOWJRS3qNNbF1u6NyM9xZvYf0eLR/3dWKMcTqGM9apUyeTlqYnVUr5sn3ZJ+g+YR7nJtZj8s2dnQ7H44nIMmOMV/4O1KOKH5RSqjT1w4P5R8+WzFuTydw1e5wOR9UgTUxKKa9x0wWJJNrl4ye1fNxnaWJSSnmNoAA/Hu6byobMI3ywZIvT4agaoolJKeVVLmvTkG4tGvCv79dx4IiWj/siTUxKKa9SUD5++LiWj/sqTUxKKa+T0jiCwec3470lW1i3+7DT4ahqpolJKeWV7rk8hdAgf57Qu4/7HE1MSimvFB0WxN09WzJ/bSZzV2v5uC/RxKSU8lo3dk2keYMwHp+h5eO+RBOTUsprWeXjbdiYeYT3Fmv5uK/QxKSU8mo9WjfkopYN+Pf3a9mv5eM+QROTUsqrFZSPH8nJ49/fr3U6HFUNNDEppbxeq0YR/OX8Znzwv62s1fJxr6eJSSnlE/55WSvCgvx5fHo6vvDUhNpME5NSyifUCwvi/y5rxYJ1e5mj5eNeTROTUspnDOmaQPOYMJ6csYqcXC0f91aamJRSPiPQ349H+6ayce8R3l282elwVBVpYlJK+ZRLWzfkklYxvPjDOvZln3A6HFUFmpiUUj7nkb5tOJqTx7+0fNwraWJSSvmclo0iGNIlgQ//t5XVuw45HY6qJE1MSimfdHfPlkTUCdTycS+kiUkp5ZPqhQXxz8tasmj9Pr5fpeXj3kQTk1LKZ/2lSwItGobz5Ix0LR/3IpqYlFI+K9Dfj0f6tmHzvqO889Nmp8NRFaSJSSnl07qnNOTSlBhe0vJxr6GJSSnl8x7um8qxk3k8/52Wj3sDTUxKKZ/XomE4Q7om8PHPW1m1U8vHPZ0mJqVUrXB3z5bUDQlk3DQtH/d0mpiUUrVCVGgQ91zeisUb9/Ft+m6nw1Fl0MSklKo1BnduRsuG4YyfuYoTuXlOh6NKoYlJKVVrBPj78Wi/VLbsO8rkRZudDkeVQhOTUqpWubhVDD1bN+Q/c9aTeVjLxz2RJialVK0zqm8bjp/M44Xv1jgdiiqBJialVK2THBPOTRck8vHSDFbuOOh0OKoYTUxKqVrpHz1aEhWidx/3RJqYlFK1UmRoIPdckcKSjfuZvXKX0+EoF5qYlFK11qDz4klpFMGTWj7uUTQxKaVqrYLy8Yz9x5i0cLPT4SibJialVK3WrWUDLmvTiIlz17Pn8HGnw1FoYlJKKR7u24YTuXk8P1vvPu4JNDEppWq9pAZhDL0gkU+XZfDHdi0fd5omJqWUAkb0aEm90CDGafm44zQxKaUUEBkSyL1XtOLnTfv55g8tH3eSJiallLJd3yme1o2t8vHjJ7V83CmamJRSyhbg78fofqlsO3CM/y7c5HQ4tZYmJqWUcnFBiwZcntqIV+auZ88hLR93giYmpZQq5uE+bcjJy2fCbL37uBM0MSmlVDGJDcK45cIkPv9lG79v0/Jxd9PEpJRSJbirRwuiQ4MYN32llo+7mSYmpZQqQd06gdx3ZQpLNx9g5u9aPu5OmpiUUqoU13WKp01sXcZr+bhbaWJSSqlS+PsJj/Zrw/YsLR93J01MSilVhguSG3BlW+vu47u1fNwtNDEppVQ5RvVpQ26e4dlvtHzcHTQxKaVUORLqh3Fzt0Sm/LKN37ZlOR2Oz9PEpJRSFTDi0hY0CA9m3DS9+3hNc3tiEpFeIrJGRNaLyIMl9L9HRNJF5DcR+UFEEtwdo1JKFRdRJ5D7r2xF2pYDTP9tp9Ph+DS3JiYR8QcmAr2BVGCQiKQWa7Yc6GSMaQd8DjzrzhiVUqo0154bT2psXZ6etVrLx2uQu8+YOgPrjTEbjTE5wMfAANcGxpi5xpij9tslQFM3x6iUUiXy9xNG909le9Yx3vxxo9Ph+Cx3J6Y4IMPl/Ta7W2mGAbNK6iEiw0UkTUTSMjMzqzFEpZQqXZfm9el9VmNembeBXQe1fLwmeGzxg4j8FegETCipvzHmDWNMJ2NMp5iYGPcGp5Sq1Ub1aUNevuHZ2audDsUnuTsxbQfiXd43tbsVISKXAQ8DVxljTrgpNqWUqpD46FCGXZTEF79sZ0WGlo9XN3cnpqVASxFJEpEg4AZgqmsDEekIvI6VlPa4OT6llKqQuwrLx/Xu49XNrYnJGJMLjABmA6uAT40xK0VknIhcZTebAIQDn4nIChGZWsrglFLKMeHBAYy8MoVftmYx9dcdTofjU8QXMn2nTp1MWlqa02EopWqZ/HzDVRMXsi87hzn3dickyN/pkAqJyDJjTCen46gKjy1+UEopT+fnJ4zu15adB4/zhpaPVxtNTEopdQY6J0XT9+xYXpu/gZ0Hjzkdjk/QxKSUUmfowd6tyTN69/HqoolJKaXOUHx0KLdelMSXy7fzy9YDTofj9TQxKaVUNbizewtiIvTu49VBE5NSSlWDMLt8fEVGFl+v0PLxM6GJSSmlqsnAc5pydlwkT89azdGcXKfD8VqamJRSqpr42Xcf33XoOK/P1/LxqtLEpJRS1ei8xGj6tYvl9R83sCNLy8erIsDpAGpafn4+27Zt48iRI06HopRHCAsLo2nTpvj56XFpTXmwd2u+S9/NM9+s5sUbOjodjtfx+cS0d+9eRISUlBTdEFWtl5+fz/bt29m7dy8NGzZ0Ohyf1bReKMMvbs5/5qznxq6JnJtQz+mQvIrP76mzsrJo1KiRJiWlAD8/Pxo1asTBgwedDsXn3X5JMo3qBjNuejr5+Vo+Xhk+v7fOy8sjMDDQ6TCU8hiBgYHk5mrFWE2zysdb82tGFl+tOO2xc6oMPp+YAETE6RCU8hi6PbjP1R3jaN80kme+0fLxyqgViUkppZxQUD6++9AJXpu3welwvIYmplqqd+/ePPvss+W2W7BgAVFRUW6IyBmbN29GRNi2bRsAkydPpkWLFg5HpXzJuQnRXNW+Ca//uJFtB446HY5X0MTkIbp3705wcDDh4eFERkbSsWNHpkyZUmPjmzVrFiNHjiy33UUXXURWVlaNxQEwdOhQAgMDCQ8Pp27durRp04ZXXnmlRsd5Jnbu3Mkdd9xBQkICYWFhNGvWjOuuu45ly5Y5HZryUA/2bo0IPKN3H68QTUwe5NFHHyU7O5t9+/YxaNAgrr/+etauXXtau5MnTzoQXc266aabyM7OJisriyeeeIIRI0Ywb948p8M6zY4dOzjvvPPIyMhg5syZHDp0iPT0dPr3788XX3xRpWH64vJURTWJCmH4xclM+3UHaZv3Ox2Ox9PE5IECAgK48847ycvL4/fff2fevHkEBATw3nvv0bx5c6KjowHYunUr1157LY0bNyY2Npbhw4dz+PDhwuFkZmYybNgwmjVrRt26dTnnnHNYs8Y6YuvevTtPPPEEACdOnGD48OE0bNiQunXr0rJlSz777DOAwnEXyM3NZdy4cTRv3px69erRs2dP/vjjj8L+Q4cOZciQIdx6661ERUURFxfH66+/XuFp9/PzY+DAgdSvX5+0tLTC7gsWLKBbt25ER0eTnJzM888/X+QOzr/99hu9evUiJiaG6OhoLrvsssJ+N998M/Hx8URERJCamsqHH35Y4XiKGz16NGFhYXz55Ze0bdsWf39/wsPDGTJkCE8++SQAY8eOLTJ+KDq/S1qeEydOpEOHDkU+s2nTJvz9/dm8eTNQ/vJWnu32S5rTuG4dLR+vAJ//gW1xj01bSfqOQ24ZV2qTuozp37bSn8vJyWHixIkEBgbSvn17tm3bRl5eHjNnzmT58uUEBgZy/PhxevToweDBg3nvvfc4fvw4f/nLX7j77ruZNGkS+fn5XHXVVTRp0oSlS5cSExPDH3/8QURExGnje+edd1i6dCmrVq2ifv36ZGRklLrDmzBhAu+++y4zZ84kKSmJp556issvv5w1a9ZQt25dAD7//HM++eQTXn/9db766iuuv/56evXqRUJCQrnTnpeXx+eff87evXtJSUkBID09nT59+vD+++/Tr18/1q1bR+/evYmJieHGG29k586dXHLJJYwcOZIpU6YQGBjIjz/+WDjMbt268dxzzxEVFcVnn33GjTfeSIcOHUhNTa30spk5cya33HLLGf8EofjyPHHiBPfeey8rVqwoTFCTJ0+me/fuJCYmlru8lecLDQrggd4p/POTX/li+XauPbep0yF5LD1j8iBPPvkkUVFRNG3alK+//popU6YU+SL+mWeeITIyktDQUKZPn44xhnHjxhESEkK9evV4/PHH+eCDD8jLyyMtLY20tDQmTZpU+APjdu3a0aRJk9PGGxQURHZ2Nunp6eTm5hIfH1/qTvvtt9/mgQceoHXr1gQHBzN69Gj8/f2ZMWNGYZsePXpw1VVX4efnxzXXXENUVBQrVqwoc9rfe+89oqKiqFOnDoMGDWLcuHH0798fgFdeeYU///nPDBgwAH9/f1q3bs2IESN49913Cz/bokULHnroIcLCwggKCipyxjJs2DDq16+Pv78/N9xwA+3atavyZcLMzEzi4uKq9NniXJdnvXr1GDBgAG+//TYAxhjeeecdbrnlFoByl7fyDgPax9EhPopnv1nNkRNaPl6aWnfGVJUzGHd5+OGHeeSRR0rs5+fnR3x8fOH7TZs2sXXr1tMq5kSEXbt2sXnzZho2bEhkZGS54/3rX//K7t27+ec//8m6devo2bMnzz77bInVaRkZGSQlJRWJKzExkYyMjMJusbGxRT4TFhZWeAbWtm1btmzZAsCoUaMYNWoUAEOGDOGtt97i6NGjjBw5kh9++IGHHnqIgIAANm3axJw5c4p8h5Ofn184PzZv3kyrVq1KnLb8/HzGjh3LJ598wq5duxARjhw5QmZmZrnzpSQxMTFs337mP5YsvjzBuuQ4ZMgQnnvuOX788UeysrK45pprgPKXd3UlS1WzCsrHr3nlJ16dt4H7rkxxOiSPpGdMXkJEivwwMiEhgVatWpGVlVXk7/jx48TFxZGYmMiePXs4dKj8y5YBAQE88MADpKWlsWXLFkJDQwuP1IuLj48v/M4DrB3/5s2bT9vJlmblypVkZ2eTnZ1dmJRchYaG8sILL7B9+3YmTpxYOK233HJLkek8dOgQK1euBCAxMZF169aVOL6PPvqIt956iylTpnDgwAGysrJo3759lZ8w2qdPHz7//PMyCxYiIiJOu2nwjh1FHxxXfHkCXH755QQHBzNt2jQmT57MDTfcQEhICFD+8lbe45xm9fhThya8sUDLx0ujiclL9evXj5ycHMaPH8/hw4cxxrB9+3a+/PJLADp16sQ555zD3/72N/bs2UN+fj6//fbbaTtIgDlz5rBs2TJOnjxJSEgIYWFh+Pv7lzjeoUOH8uyzz7J27VpycnJ48sknyc3NpW/fvtU2bUFBQYwePZonnniCw4cPc+edd/Lxxx8zbdo0Tp48SW5uLunp6cyfPx+wzvjWrFnDM888w9GjR8nJyeH7778H4NChQwQEBBATE0N+fj6TJk3i119/rXJsjz32GNnZ2Vx77bWsWrWKvLw8jhw5wkcffVR4tnvuuefyyy+/sGzZMnJzc3n55ZfZtGlTucP29/fnxhtv5KWXXuKLL74ocnBQ3vJW3mVkr9b4CTw1a7XToXgkTUxeKjQ0lDlz5pCenk7r1q2JjIykZ8+ehd/l+Pn5MW3aNEJCQujQoQNRUVHccsstZGdnnzas3bt3M2TIEOrVq0dsbCxbtmzhjTfeKHG8999/P4MGDeKKK66gUaNGzJkzh2+//baw8KG6DB48mOjoaJ5//nnOOusspk+fzr///W9iY2Np2LAhQ4cOLbwc16RJE+bNm8d3331H06ZNady4MRMmTACsMvTzzz+fFi1aEBcXR3p6OhdddFGV44qLi2Pp0qXExsZyxRVXFP7u6quvvmLgwIGAVYF3zz330KtXL2JjY9m9ezcXXnhhhYZ/8803M3/+fJKSkujcuXNh9/KWt/IuTaJCuP2SZGb8tpOfN2n5eHFS1UsanqRTp07GtbTY1apVq2jTpo2bI1LKs+l24bxjOXn0eH4e9cODmHpXN/z8qvcehiKyzBjTqVoH6iZ6xqSUUg4ICfLnwd6t+WP7IT7/ZZvT4XgUTUxKKeWQq9o3oWOzKCbMXkO2lo8X0sSklFIOERHG9G9L5uETvDJ3vdPheAxNTEop5aAO8VFc0zGOtxZuImO/lo+DJiallHLcyF6t8RfhqVmrnA7FI2hiUkophzWOrMMd3ZOZ+fsu/rdxn9PhOE4Tk1JKeYBbL2pOk0jr7uN5tfzu45qYSpOXB9OnQ9++EBUF/v7W/759re5640ylVDUKCfLnwT5tWLnjEJ8vyyj/Az5ME1NJ1qyBdu2gf3+YORMOHoT8fOv/zJlW93btrHZKKVVN+reL5dyEekyYvZbDx2vvAyQ1MRW3Zg1ceCGkp5fdLj3daudFyWn8+PGFj5IoT3h4OIsXL67hiEonIixcuNBt43N9kF9VTJ48ucS7sbsKCAjwyKfyKs8hIozul8re7BNMnLvB6XAco4nJVV4eXHMN7LO/fKxTB+67D1atgpwc6/9990FwsNV/3z6rfTVc1uvevTvBwcFEREQQGRlJ8+bNGTJkCMuWLTvjYRcYNWoU06ZNq1Db7OxsunbtWm3jLrB161bCw8ML/wICAggKCip837at5z6WxJOlpaXRuXNnQkNDSU5O5v3333c6JFVF7eOjGHhOUyYt3MTWfbWzfFwTk6tZs06dKdWpA3PmwIQJ0Lo1BAZa/ydMgLlzTyWn9HT45ptqGf2jjz7K4cOHOXjwIHPnziUhIYEuXbr41B2kmzVrVvjYi+zsbLp3786oUaMK3xc8yqKyynoMha87ePAgvXv3ZuDAgRw4cIDXXnuN22+/3dEzXnVmRvZKIcBfGD+zdpaPa2Jy9eqrp16PGAGlnTF07Wr1L+lz1SQhIYEnnniCG2+8kb///e+Fzw86evQo9913H0lJSURHR9OrVy/Wrz/1i/GTJ08yfvx4UlJSiIiIIDk5mc8//xyAsWPHFnmy60svvURSUhIRERHExcUVeT5S8UtpU6ZMoX379kRGRtK+ffsiybLgMtZLL71E06ZNqVevHrfddtsZP1n1t99+47zzziMiIoIuXbqwevWpRwR0796d//u//+NPf/oTdevW5fnnnwfgzTff5KyzziIyMpKOHTvy7bffFn5m+fLldOvWjcjISKKjo7ngggs4cOBAYf8DBw4wcODAwvn29ddfF4nn1VdfJSUlhcjISLp06cKCBQtKjf3w4cPcdNNNREdHk5CQwDvvvHNG86IsX3zxBaGhoYwcOZLg4GAuv/xyrr766lLvEK88X6O6dbizezLfrNzF4g21sHzcGOP1f+eee64pTXp6eqn9ThMZaQxYf6tWld02Pf1U26ioio+jFJdccol5/PHHT+v+7bffGsCssuMZPHiw6du3r9m1a5c5ceKEGT16tElJSTE5OTnGGGNGjhxp2rRpY3799VeTn59vMjIyzK+//mqMMWbMmDGmZ8+exhhj1qxZY0JCQswff/xhjDHmwIEDZvHixYXjBcyCBQuMMcYsWrTIBAcHm5kzZ5qTJ0+a6dOnm+DgYLNkyRJjjDFvv/22CQgIMKNGjTLHjx8369atM/Xq1TPvv/9+udPds2dPM2bMmNO6A+a8884zW7ZsMcePHzfXXnutueyyy4rMr4iICPPDDz+Y/Px8c+TIEfPGG2+Y5ORks2LFCpOXl2dmzJhhwsLCzLp164wxxnTt2tU89thjJjc31+Tk5JjFixeb7OzswuHVr1/fLFq0yOTl5ZkXXnjBREZGmiNHjhhjjPnwww9N/fr1zZIlS8zJkyfNW2+9ZUJDQ83mzZsL50FycnJhfLfccovp2rWr2blzp8nKyjJXX321AczcuXNLnA9btmwxkZGRZf6V5u677zYDBgwo0u2FF14wHTt2LPUzldoulCOO5eSaC576wfT6948mNy+/0p8H0owH7J+r8qdnTK7sx38DkJxcdlvX/hV4SmxVNW3aFIB9+/axd+9ePvzwQ1555RUaNWpEUFAQY8aMYefOnfzvf//DGMPEiROZMGEC7dq1Q0Ro2rQp7dq1O224AQEBGGMKnygbFRVFly5dSoxh8uTJDBw4kN69exMQEEDfvn25+uqrmTRpUmGbkJAQxo0bR3BwMC1atKBnz56U9iiSirr//vtp1qwZwcHBDB069LThXXvttfTo0QMRITQ0lBdffJHRo0fTvn17/Pz86NOnD5deeikff/wxYD2AcOvWrWRkZBAYGEiXLl0ICwsrHN7111/PBRdcgJ+fH8OHD+fgwYOFlqYthAAADoRJREFUT8Z9++23ue222zj//PMJCAhg2LBhtGvXjg8//PC0uPPz8/nggw94/PHHady4MZGRkTzzzDNlTmuzZs1Oezpt8b/SHD58mMjIyCLdoqKiKvT0YuW56gT681Cf1qzaeYjP0mpX+bgmJlcREadebyinIsa1fzU/JM/Vtm3W7fDr169f+BTUdu3aERUVRVRUFNHR0Zw8eZKMjAwyMzM5cuQIrVq1Kne4zZs354MPPuDNN9+kSZMmdOvWrchlL1cZGRkkJSUV6ZacnExGxqmNpWHDhkWeehsWFsZh10RfBbGxsWUOLzExscj7TZs2cddddxXOm6ioKObOncv27dsBK7nk5+fTrVs3kpKSePTRR8nNPXVH5+LjAwrHWZF5UCAzM5MTJ04Uia/4Z6tTREQEBw8eLNItKyur2h/eqNyv79mxnJdYj+e+XVOrysc1Mblyfcrof/9bdlvX/hV8OmlVfPLJJ8TFxZGSkkJCQgIA69atK3IkffToUQYNGkRMTAyhoaGFR/nlueaaa/juu+/Yu3cv1113HQMGDODo0dOrgOLj49m8eXORbhs3biQ+Pv6Mp+9M+PkVXX0TEhKYNGlSkXmTnZ3Nq/Z3gElJSUyaNIlt27YxdepU3nrrLd59990Kjasy86BBgwYEBQUVaV/8s8UVr1Ys6a807du3P+1JtsuXL6d9+/blTpfybFb5eFv2Hcnh5Tm15+7jmphc3XHHqdcvvwylVTUtXmz1L+lz1SQjI4MxY8YwefJkXnzxRUSEhg0bMnjwYO68887Cs4CsrCy+/PJLsrOzERHuvPNORo4cyR9//IExhm3btvHbb7+dNvw1a9bwzTffcPToUQIDA4mMjERETtvZg/V48ilTpjB79mzy8vKYNWsWX3zxBTfffHO1T/eZ+Oc//8nYsWNZsWIFxhiOHTvGwoULC4sm3nnnHXbs2AFYl7oCAgKKnOWVZejQobz++uv8/PPP5Obm8vbbb7NixQoGDx58Wlt/f38GDx7MmDFj2L17N4cOHeLBBx8sc/jFqxVL+ivN1VdfzZEjR5gwYQI5OTn88MMPfPHFFwwfPrxC06Y829lNI7n2nKZMWrSJzXuPOB2OW2hictW7N6SmWq+PH4dLLy35d0yXXgonTljtUlOhV69qGf3jjz9OREQEdevW5eKLL2b9+vX89NNPDBw4sLDNm2++SUpKCt27dyciIoKzzz6bzz77DBHrscxPPvkk1113HX/605+IiIige/fuRar2CuTk5DBu3DhiY2OJioripZdeYsqUKdT5//buPUaq8ozj+PfHsroVcVmhBBUJNCAXIaayoqipNd5RNGq1SlFBulqttVGBXqxWxSa1Vqk1KNJUweIFaRODYa03UFoQC/WygoIiELtecUVKi6jo0z/eMzgMu+xZu3tmzpnnk0zmds6Z530zs8++73nP+1ZV7bTtEUccwaxZs5g4cSI1NTVMnjyZ2bNnt3hOqljq6uqYPHky48ePp6amhj59+jBlypTtQ8kXLFjA8OHD6dKlCyNHjmTMmDGcd955sY6dSzRjx46le/fu3HnnndTX129vxRa67bbb6NevH4MGDWLYsGGMHj06dhJsq27dulFfX8/cuXOprq6mrq6O6dOnd8h1aK44Jp0wkN0qOpXN8HGZpX+ywNraWmvpRPurr77K4MGD4x8sN/NDU4whmt27w5IlEOOcjnOlpM2/C1d00xau4ebHVnP/9w/l8P49Wt1e0j/NrDaB0Nqdt5gKDRwIixd/2XJqyZAhnpScc4mZcGQ/etd8rSxmH/fE1JyBA6GhYcfZxTt12nF28YYGT0rOucRUVVbw81GDWfXuZuYsy/bw8c7FDqBkVVSEJHTyycWOxDnnADhpaC9G9N2bWx5fzSkH7cNeVZXFDqlDeIvJOedSQhLXjh7Ch1uyPXy8LBJTFgZ4ONde/PeQbkP3q+as4b25Z/E61mV0+HjmE1NVVRVNTU3+Y3SOkJSampqavSzApcfEEwaye+cKfjU/m8PHM3+OqXfv3jQ2NrJhw4Zih+JcSaiqqto+B6NLp55dq/jh0f256a+r+PvrH3DkgNaHj6dJ5q9jcs65LNr62eccN/UZ9qjszPzLj6RzxY4dYH4dk3POuURVVVZw9ajBrH5vMw9mbPh44olJ0omSVktaI2mnCcQk7S5pTvT+c5L6Jh2jc86lwQkH9uLQfntz6xOvsenj7Mw+nmhiklQBTANOAoYA50oqnGJhArDRzPoDU4FdL2TjnHNlKjd8fOOWT7n9qXirCqRB0i2mEcAaM1trZp8CDwKnFWxzGpBbh/rPwDHKzVDqnHNuBwfuW813a/dn5pL1rN3Q8iz0aZJ0YtoPyO8MbYxea3YbM9sGbAK6JxKdc86l0FXHD6SqMjvDx1M7XFzSRUBuwZlPJK0oZjwloAfwQbGDKKJyLz94HZR7+VkJPe4ev70Oml+TJQWSTkxvAflLfvaOXmtum0ZJnYFqYKc1KMxsBjADQNLytA6LbC/lXgflXn7wOij38kN26iDprrxlwABJ/STtBpwDzCvYZh5wQfT4O8ACy8LFVs4552JJtMVkZtskXQY8BlQAd5vZSkk3AMvNbB7wR+BPktYAHxKSl3POuTKR+DkmM6sH6gteuzbv8VbgrDYedkY7hJZ25V4H5V5+8Doo9/JDRuogE1MSOeecyw6fksg551xJSVViKvfpjGKU/0pJr0hqkPSUpNQOF21Ja3WQt92ZkkxS6kcoFYpTB5LOjr4LKyXdn3SMHSnG76CPpIWSXoh+C6OKEWdHkXS3pPdbukRGwe+j+mmQdHDSMf7fzCwVN8JgiTeAbwC7AS8BQwq2uRSYHj0+B5hT7LgTLv/RwB7R40uyVP64dRBt1xVYBCwFaosddxG+BwOAF4Ca6HnPYsedcPlnAJdEj4cA64sddzvXwbeAg4EVLbw/CngUEHAY8FyxY27rLU0tpnKfzqjV8pvZQjPbEj1dSrhOLEvifAcAphDmWNyaZHAJiVMHdcA0M9sIYGbvJxxjR4pTfgP2ih5XA28nGF+HM7NFhBHLLTkNuNeCpUA3SfskE137SFNiKvfpjOKUP98Ewn9NWdJqHUTdFvub2fwkA0tQnO/BAcABkhZLWirpxMSi63hxyn8dMFZSI2EE8I+SCa1ktPVvRclJ7ZRErmWSxgK1wFHFjiVJkjoBtwLjihxKsXUmdOd9m9BqXiRpmJl9VNSoknMuMNPMbpE0knBd5FAz+6LYgbl40tRiast0RuxqOqOUilN+JB0LXA2camafJBRbUlqrg67AUOBpSesJ/evzMjYAIs73oBGYZ2afmdk64DVCosqCOOWfADwEYGbPAlWEefTKRay/FaUsTYmp3KczarX8kr4J3EVISlk6r5Czyzows01m1sPM+ppZX8J5tlPNbHlxwu0QcX4HDxNaS0jqQejaW5tkkB0oTvnfBI4BkDSYkJg2JBplcc0Dzo9G5x0GbDKzd4odVFukpivPynw6o5jlvxnYE5gbjfl408xOLVrQ7SxmHWRazDp4DDhe0ivA58AkM8tEz0HM8l8F/EHSFYSBEOMy9A8qkh4g/OPRIzqP9kugEsDMphPOq40C1gBbgPHFifSr85kfnHPOlZQ0deU555wrA56YnHPOlRRPTM4550qKJybnnHMlxROTc865kuKJyWWCpHHRbOK522ZJL0m6LLrYOokY+kafPa4N++Ti7tthgTmXMqm5jsm5mM4izHywV/T4dqAncO2udmon7wAjCbNfxzU/2idVF0A615H8OiaXCVEr5R5ggJmtyXt9IXCwmVU3s08lsC1LF186lwXeleeybhmwl6QRUZfZpZJ+I+lt4BOgG4CkM6KZuLdI+kjSXEl9Cg8mqU7S85I+lrRR0jOSDo/e26krT9Ihkp6Q1BTts1bSHXnv79SVJ6lS0o2S1kv6NLq/MUqkFHzWxZJukPROFPcjkrK23IkrM56YXNb1I0zL85/o+dWEueMuAk4Htkr6AfAX4BXCHIsXEyaDfUZS19yBJP2WsAjd88DZwFjCgoQ7JbBo+z0JU+d8Tpjx/CTgBlrvQp8F/BS4FzgFmAn8hC/XGsv3M6A/cCHwY0K34OxWju9cSfNzTC5rKqLBDl0JyeMM4BHCnGEA7wGn57rvouRxE3CPmV2YO4ikfwCrCTNV/05Sf+AKYKqZXZn3ebta92kQUANMNrOGvNdntrSDpKGEZRuuN7Propcfl7QNmCLp1wXHWm9mY/L2/zpws6R9zSxTC+S58uEtJpc1q4DPCJP43gHcR2hN5DxccE5pJGGgxH2SOuduhIXWVhGWsQY4lvB7mdGGWF4HPgLukjRW0v6t7ZD3eYWtntzzwjW26guevxzdN9uKcy4NPDG5rDkdOITQWuliZuebWf4y1IWj33pG908SElr+bRhfroCcu2+MG4iZbQKOJiztfQfwpqQVks7cxW57txDnuwXv5xQusZ1bg6sqbpzOlRrvynNZsyJ/VF4zCkfg5ZaDGAesbGb7zdH9B9H9foQuvljM7EXgzKgVVks4J/SQpIPMbEUzu+QSTS92HHbeq+B95zLLW0yu3C0hJJ/+Zra8mVsuCT0JfEEYNNFmZrbNzJYC1xB+d4Nb2HRRdF+4ltj3ovunv8rnO5cm3mJyZc3M/i1pEjAtGjjwKLCJ0DI6CnjazO43szckTQWujEbqzSOMthsBrDKzOYXHlnQKIZE9DKwDugCXExLhsy3EsyJaCO66qJW1hHAe7BrgATN7ubn9nMsST0yu7JnZXZL+BUwCxhB+F28BfwNezNtuYrQ68qXABcB/gQbg8RYO/TrwMSGp7ENISMuA48xsV+eqxhGWQr8Q+AXhHNVNwPVfrYTOpYvP/OCcc66k+Dkm55xzJcUTk3POuZLiick551xJ8cTknHOupHhics45V1I8MTnnnCspnpicc86VFE9MzjnnSoonJueccyXlf9eSc8HRPJ/iAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 467
        },
        "id": "hvSLHx--d95b",
        "outputId": "2549ad7c-f4a8-4e88-a652-39510c37b8e4"
      },
      "source": [
        "#XGBoost Preds concatenated to train df for failure analysis\n",
        "train_with_xgtfidf_preds = pd.concat([wikilarge_train, pd.DataFrame(xg_rf_preds, columns=['predicted'])], axis= 1)\n",
        "train_with_xgtfidf_preds.head()"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>original_text</th>\n",
              "      <th>label</th>\n",
              "      <th>LS</th>\n",
              "      <th>TO</th>\n",
              "      <th>VBN</th>\n",
              "      <th>WP</th>\n",
              "      <th>UH</th>\n",
              "      <th>VBG</th>\n",
              "      <th>JJ</th>\n",
              "      <th>VBZ</th>\n",
              "      <th>--</th>\n",
              "      <th>VBP</th>\n",
              "      <th>NN</th>\n",
              "      <th>DT</th>\n",
              "      <th>PRP</th>\n",
              "      <th>:</th>\n",
              "      <th>WP$</th>\n",
              "      <th>NNPS</th>\n",
              "      <th>PRP$</th>\n",
              "      <th>WDT</th>\n",
              "      <th>(</th>\n",
              "      <th>)</th>\n",
              "      <th>.</th>\n",
              "      <th>,</th>\n",
              "      <th>``</th>\n",
              "      <th>$</th>\n",
              "      <th>RB</th>\n",
              "      <th>RBR</th>\n",
              "      <th>RBS</th>\n",
              "      <th>VBD</th>\n",
              "      <th>IN</th>\n",
              "      <th>FW</th>\n",
              "      <th>RP</th>\n",
              "      <th>JJR</th>\n",
              "      <th>JJS</th>\n",
              "      <th>PDT</th>\n",
              "      <th>MD</th>\n",
              "      <th>VB</th>\n",
              "      <th>WRB</th>\n",
              "      <th>NNP</th>\n",
              "      <th>EX</th>\n",
              "      <th>NNS</th>\n",
              "      <th>SYM</th>\n",
              "      <th>CC</th>\n",
              "      <th>CD</th>\n",
              "      <th>POS</th>\n",
              "      <th>aoa&lt;=5</th>\n",
              "      <th>aoa&lt;=10</th>\n",
              "      <th>aoa&lt;=15</th>\n",
              "      <th>aoa&lt;=20</th>\n",
              "      <th>aoa&gt;20</th>\n",
              "      <th>1 Syll</th>\n",
              "      <th>2 Syll</th>\n",
              "      <th>3 Syll</th>\n",
              "      <th>4 Syll</th>\n",
              "      <th>&gt;=5 Syll</th>\n",
              "      <th>avg_aoa_score</th>\n",
              "      <th>avg_concreteness</th>\n",
              "      <th>perc_dale_shall</th>\n",
              "      <th>num_words</th>\n",
              "      <th>avg_word_length</th>\n",
              "      <th>avg_syllable_count</th>\n",
              "      <th>avg_percent_recognized</th>\n",
              "      <th>predicted</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>There is manuscript evidence that Austen conti...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.139535</td>\n",
              "      <td>0.046512</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.139535</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.093023</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.023256</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.069767</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.413793</td>\n",
              "      <td>0.517241</td>\n",
              "      <td>0.068966</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.720930</td>\n",
              "      <td>0.162791</td>\n",
              "      <td>0.116279</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.288475</td>\n",
              "      <td>0.538312</td>\n",
              "      <td>0.558140</td>\n",
              "      <td>0.461538</td>\n",
              "      <td>0.117174</td>\n",
              "      <td>0.056478</td>\n",
              "      <td>0.985938</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>In a remarkable comparative analysis , Mandaea...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.260870</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.130435</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.428571</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.608696</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.394699</td>\n",
              "      <td>0.487286</td>\n",
              "      <td>0.217391</td>\n",
              "      <td>0.241758</td>\n",
              "      <td>0.185619</td>\n",
              "      <td>0.111801</td>\n",
              "      <td>0.995000</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Before Persephone was released to Hermes , who...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.108696</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.152174</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.086957</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.043478</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.472222</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.717391</td>\n",
              "      <td>0.195652</td>\n",
              "      <td>0.065217</td>\n",
              "      <td>0.021739</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.280424</td>\n",
              "      <td>0.489294</td>\n",
              "      <td>0.543478</td>\n",
              "      <td>0.494505</td>\n",
              "      <td>0.131271</td>\n",
              "      <td>0.055901</td>\n",
              "      <td>0.992647</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Cogeneration plants are commonly found in dist...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.230769</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.153846</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.076923</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.282051</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.310345</td>\n",
              "      <td>0.551724</td>\n",
              "      <td>0.137931</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.564103</td>\n",
              "      <td>0.256410</td>\n",
              "      <td>0.102564</td>\n",
              "      <td>0.051282</td>\n",
              "      <td>0.025641</td>\n",
              "      <td>0.356225</td>\n",
              "      <td>0.686129</td>\n",
              "      <td>0.256410</td>\n",
              "      <td>0.417582</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.102564</td>\n",
              "      <td>0.994516</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Geneva -LRB- , ; , ; , ; ; -RRB- is the second...</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.055556</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.305556</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.027778</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.138889</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.055556</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.533333</td>\n",
              "      <td>0.400000</td>\n",
              "      <td>0.066667</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.805556</td>\n",
              "      <td>0.083333</td>\n",
              "      <td>0.111111</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.284035</td>\n",
              "      <td>0.472933</td>\n",
              "      <td>0.361111</td>\n",
              "      <td>0.384615</td>\n",
              "      <td>0.139957</td>\n",
              "      <td>0.043651</td>\n",
              "      <td>0.989333</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                       original_text  ...  predicted\n",
              "0  There is manuscript evidence that Austen conti...  ...        1.0\n",
              "1  In a remarkable comparative analysis , Mandaea...  ...        1.0\n",
              "2  Before Persephone was released to Hermes , who...  ...        0.0\n",
              "3  Cogeneration plants are commonly found in dist...  ...        0.0\n",
              "4  Geneva -LRB- , ; , ; , ; ; -RRB- is the second...  ...        1.0\n",
              "\n",
              "[5 rows x 64 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gEJSUlCbfSEs"
      },
      "source": [
        "train_with_xgtfidf_preds.to_csv('for_failure_analysis.csv', index=False)"
      ],
      "execution_count": 56,
      "outputs": []
    }
  ]
}