Class 4.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# What is machine learning?\n",
    "\n",
    "Machine learning is the study and use of algorithms that can learn from and make predictions on data.\n",
    "\n",
    "Machine learning problems generally fall into one of two types:\n",
    "\n",
    "1. **Supervised learning**: Our data has associated attributes that we would like to predict.\n",
    "    * Regression: The attributes that we would like to predict are continuous.\n",
    "    * Classification: We would like to classify each data point as one of a finite number of categories.\n",
    "2. **Unsupervised learning**: Our data does not have a corresponding set of target values. Examples:\n",
    "    * Clustering: Discover groups of similar examples within the data.\n",
    "    * Visualization: Project the data down from a high-dimensional space to two or three dimensions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Example: supervised learning (regression) with Linear Regression\n",
    "References:\n",
    " * http://scikit-learn.org/stable/auto_examples/linear_model/plot_ols.html\n",
    " * https://chrisalbon.com/machine-learning/linear_regression_scikit-learn.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: sklearn in /Library/Python/2.7/site-packages\r\n",
      "Requirement already satisfied: scikit-learn in /Library/Python/2.7/site-packages (from sklearn)\r\n"
     ]
    }
   ],
   "source": [
    "!pip install sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn import linear_model\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mkdir: data: File exists\n",
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100 91357  100 91357    0     0   123k      0 --:--:-- --:--:-- --:--:--  123k\n"
     ]
    }
   ],
   "source": [
    "# Download the diabetes dataset (http://scikit-learn.org/stable/datasets/index.html#diabetes-dataset)\n",
    "!mkdir data\n",
    "!curl https://raw.githubusercontent.com/mikekaminsky/python-for-data-analysis/class_4/data/diabetes.csv > 'data/diabetes.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Features have been mean centered and scaled by the standard deviation times n_samples\n",
    "df = pd.read_csv('data/diabetes.csv', index_col=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>body_mass_index</th>\n",
       "      <th>avg_blood_pressure</th>\n",
       "      <th>S1</th>\n",
       "      <th>S2</th>\n",
       "      <th>S3</th>\n",
       "      <th>S4</th>\n",
       "      <th>S5</th>\n",
       "      <th>S6</th>\n",
       "      <th>disease_progression</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.038076</td>\n",
       "      <td>0.050680</td>\n",
       "      <td>0.061696</td>\n",
       "      <td>0.021872</td>\n",
       "      <td>-0.044223</td>\n",
       "      <td>-0.034821</td>\n",
       "      <td>-0.043401</td>\n",
       "      <td>-0.002592</td>\n",
       "      <td>0.019908</td>\n",
       "      <td>-0.017646</td>\n",
       "      <td>151.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.001882</td>\n",
       "      <td>-0.044642</td>\n",
       "      <td>-0.051474</td>\n",
       "      <td>-0.026328</td>\n",
       "      <td>-0.008449</td>\n",
       "      <td>-0.019163</td>\n",
       "      <td>0.074412</td>\n",
       "      <td>-0.039493</td>\n",
       "      <td>-0.068330</td>\n",
       "      <td>-0.092204</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.085299</td>\n",
       "      <td>0.050680</td>\n",
       "      <td>0.044451</td>\n",
       "      <td>-0.005671</td>\n",
       "      <td>-0.045599</td>\n",
       "      <td>-0.034194</td>\n",
       "      <td>-0.032356</td>\n",
       "      <td>-0.002592</td>\n",
       "      <td>0.002864</td>\n",
       "      <td>-0.025930</td>\n",
       "      <td>141.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.089063</td>\n",
       "      <td>-0.044642</td>\n",
       "      <td>-0.011595</td>\n",
       "      <td>-0.036656</td>\n",
       "      <td>0.012191</td>\n",
       "      <td>0.024991</td>\n",
       "      <td>-0.036038</td>\n",
       "      <td>0.034309</td>\n",
       "      <td>0.022692</td>\n",
       "      <td>-0.009362</td>\n",
       "      <td>206.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.005383</td>\n",
       "      <td>-0.044642</td>\n",
       "      <td>-0.036385</td>\n",
       "      <td>0.021872</td>\n",
       "      <td>0.003935</td>\n",
       "      <td>0.015596</td>\n",
       "      <td>0.008142</td>\n",
       "      <td>-0.002592</td>\n",
       "      <td>-0.031991</td>\n",
       "      <td>-0.046641</td>\n",
       "      <td>135.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        age       sex  body_mass_index  avg_blood_pressure        S1  \\\n",
       "0  0.038076  0.050680         0.061696            0.021872 -0.044223   \n",
       "1 -0.001882 -0.044642        -0.051474           -0.026328 -0.008449   \n",
       "2  0.085299  0.050680         0.044451           -0.005671 -0.045599   \n",
       "3 -0.089063 -0.044642        -0.011595           -0.036656  0.012191   \n",
       "4  0.005383 -0.044642        -0.036385            0.021872  0.003935   \n",
       "\n",
       "         S2        S3        S4        S5        S6  disease_progression  \n",
       "0 -0.034821 -0.043401 -0.002592  0.019908 -0.017646                151.0  \n",
       "1 -0.019163  0.074412 -0.039493 -0.068330 -0.092204                 75.0  \n",
       "2 -0.034194 -0.032356 -0.002592  0.002864 -0.025930                141.0  \n",
       "3  0.024991 -0.036038  0.034309  0.022692 -0.009362                206.0  \n",
       "4  0.015596  0.008142 -0.002592 -0.031991 -0.046641                135.0  "
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Explore the data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Problem statement\n",
    "Predict a patient's disease progression given his/her BMI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11d3d5be0>"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAELCAYAAAAspXpuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXu8HHV5+P9+9nI2IYEkJAESkoAaKL8caiLka6BBvhC8\nUCBgNUYlgFUrtT/QtnKJrWIIKa2IYrkppoqKxmoIVUJAW+RSSArYgEnMiYhHbrkgl0NuJznZc3b3\n+f4xMyezszO7M7uzt3M+79frvM7u7MzsZ2Z3n+fzea6iqhgMBoPB4Eei2QMwGAwGQ+tilITBYDAY\nAjFKwmAwGAyBGCVhMBgMhkCMkjAYDAZDIEZJGAwGgyEQoyQMBoPBEIhREgaDwWAIxCgJg8FgMASS\navYAamXChAl67LHHNnsYBoPB0FY8/fTTb6jqxEr7tb2SOPbYY1m/fn2zh2EwGAxthYi8FGY/Y24y\nGAwGQyBGSRgMBoMhkLoqCREZISK/EpGNItIlIkvt7d8TkRdEZIP9N8veLiJyi4h0i8gmETmpnuMz\nGAwGQ3nq7ZPIAvNUtVdE0sBaEfm5/dpVqrrKs/+fA8fZf3OAb9r/DQaDwdAE6rqSUIte+2na/ivX\nwOIC4C77uCeBsSIyqZ5jNBgMBkMwdfdJiEhSRDYArwEPqupT9kvX2yalr4tIxt52NLDVdfg2e5vB\nYDAMCXp6s2zcuoue3myzhxKKuofAqmoemCUiY4GfisiJwD8AfwQ6gOXAYuC6sOcUkUuBSwGmTZsW\n+5gNBoOhHty7YTuL79lEOpFgoFDgKx98O+fPau15cMOim1R1F/AIcLaqvmKblLLAd4F32rttB6a6\nDptib/Oea7mqzlbV2RMnVswFMRgMhqbT05tl8T2bODBQYG82x4GBAlffs6nlVxT1jm6aaK8gEJGR\nwHuAZx0/g4gI8H5gs33IauASO8rpFGC3qr5SzzEaDIahRauac7bt7COdKBa56USCbTv7mjSicNTb\n3DQJ+L6IJLEU0kpVXSMiD4vIRECADcCn7f0fAM4BuoH9wMfrPD6DwTCEaGVzzpRxIxkoFIq2DRQK\nTBk3skkjCkddlYSqbgLe4bN9XsD+ClxWzzEZDIahiduccwBLGF99zybmTp/A+NGZCkfXn/GjM3zl\ng2/nao8Sa4WxlaPtazcZDAYDHDTnOAoCDppzWkUQnz/raOZOn8C2nX1MGTeyZcZVDqMkDAZDS9HT\nm61KiLaLOWf86ExbKAcHoyQMBkPLUItPoV3NOa2OURIGwzCl2hl7PcdTq0+hHc05rY5REgbDMKQV\no4Di8im0mzmn1TGlwg2GYUarJnW1i09huGGUhMEwzGjVpC7HpzAineDQTIoR6YTxKbQAxtxkMAwz\nWnnGbnwKrYdZSRgMw4xWn7GPH51h5tSxLTOe4Y5ZSRgMwxAzYzeExSgJg2GY0swooFYLvzUEY5SE\nwWBoKK0YfmsIxvgkDIYm0aolretJq4bfGoIxKwmDoQkM19l0OxThMxRjVhIGQ4MZzrPpVg6/Nfhj\nlITB0GBaNZmtEbR6+K2hFGNuMrQ17RglM9xn0yb8tr0wSsLQtrSqXb+S4opS0rrVlGBc4zFF+NoH\noyQMbUmrtqoMq7jCzKZbTQm22ngq0WoKtl0xSsLQlrRilExUxVVuNt1KSrCnN0vXjj1cvWoT2Vzz\nxxOGdlNorYxREoa2pBXt+nEqrlZRgo6wTSBkc8X3u9lKOYhWUrBDARPdZGhLGhklEzbpLU7F1QpK\n0C1s9w/kS15vtlIOYjhHj9WDuq4kRGQE8BiQsd9rlaouEZG3AD8GxgNPAxerar+IZIC7gJOBHuDD\nqvpiPcdoaF8aESUTxWwRZ4/l8aMzLJw9hbueeHlw28LZUxo6E/ZbzQAc0pGkoNqyoautoGCHEvU2\nN2WBearaKyJpYK2I/Bz4HPB1Vf2xiNwBfBL4pv1/p6pOF5GPADcAH67zGA1tTD2jZKoxW8SluHp6\ns6xcv61o28r12/jbs45vmGD2E7aZlHDHRSfROXlMSyoIiFdZG+qsJFRVgV77adr+U2AecKG9/fvA\ntVhK4gL7McAq4DYREfs8BkNDqdYvEIfiagWfRJCwPf34Ixry/rVgcjHio+6OaxFJYpmUpgO3A38A\ndqlqzt5lG+Cs348GtgKoak5EdmOZpN6o9zgNQ5tqwiGDzBajOpJs3LqrrsKnVUwm7SxsTS5GPNRd\nSahqHpglImOBnwIn1HpOEbkUuBRg2rRptZ7OMMSpNhzSbya98OQpnHfb2rqHVsZpMqk1X8AI2+GN\nNNKSIyJfAvqAxcBR9mrhVOBaVX2fiPyn/fgJEUkBfwQmljM3zZ49W9evX9+Q8Rvaj57eLHNveJgD\nAwdn5SPSCdYtnhda8DlCdlRHkvNuW1vTuaKM23nPff35qgW8yRcwBCEiT6vq7Er71TUEVkQm2isI\nRGQk8B7gt8AjwAJ7t48B99qPV9vPsV9/2PgjhhbV9FCope9CHOGQTs/lff35hoRW3rthO3NveJiL\nvv0U5922lpd69lW9ghiu1WYN8VFvc9Mk4Pu2XyIBrFTVNSKyBfixiPwT8GvgO/b+3wF+ICLdwJvA\nR+o8PkMDqWZWW+tMuN1yF+JMBGsF57eh/anrSkJVN6nqO1T17ap6oqpeZ29/XlXfqarTVfVDqpq1\ntx+wn0+3X3++nuMzNI5qZrVxzITjTLprRAJfnIlgU8aNpD/ffOe3ob0xZTkMDaGaWW1cM+E4I3Tq\nHe0T52plbfcb5F3nSifF5AsYImPKchgaQjXCL06B6fgV4hCQcZ7L79xxrFacVZi73FJCYO70CTGP\n2DDUMSsJQ0MIE9LpDdVsduZss0pNx7Fa8VuFdSSTxh9hiIxREoZYCCNQywm/IAd1I5K5/Mbe7NDR\nSrkJle53qyTjNRLTP6I+GCVhqJmoRfC8P+BKET31TObyG/vc6RMaWmo6qnALc7+bvQprNM1W6kMZ\noyQMNRFHyGazQjWDxr784tkNG09U4RblftdjFdaKs3XTP6K+GMe1oSbiCNms1TTil2wXJgEvaOyg\nDTHVVBPiG+V+xy3Q3Ul+c294mNUbttd8zjgw/SPqi1lJGGoiDtt3LaYRv5m4QqjZedDYOyePaYip\nJuoKqqc3y+6+gVC5D0ErFKsV6W5A6Jx8WKTSJFFm641ccQxH/0sjMUrCUBNx2b6rMY34Ca6rVm0C\nlGxOQwmzy86Yzm2P/J6OZLJo7I1wmEcRbm6hny8USCeFEalkYJSYn0DfeyDHtfd1MZC3Kt2kEnDT\nwlmhbPdRFNqKJ19i6ZotdCSFXEHr7h8Ybv6XRmOUhKFm4hKoUR3UfoIrmRBQAQ622/QTZm6hC8Kl\np7+VC+dMK9qnHg5z7ww7TFhw147dXL1qY5Hiy6Tg9kXv8G3+E3Rf3AoCIFeAq1ZtDGW7D6vQVjz5\nEl/42WYA+u1mAI3wD7RzSfNWxygJQyw0o5y0n+DKFxSrr9VBvMLMb6Z9+6PdXDgnWtn5uKKSKoUF\nJ0TI5oqvqSOZZMzIjvDhr3kllZAiJQGQlHAO+bAKbel9XSXHJhNS8h71MEeZkub1wSgJQ9sSJLiA\nQGHW05vlkWdfI5WQonNFjV6KOyqpXFiwH/35fKDN3e++XHPeDK67b0vJvnkNb7uvNFvftrOPdDJB\nfz5ftH0gr0XvYcJV2wujJAxtTZDg8tvmCKekCPv6PYIsYjRV1JDLqE5qv/3dFBTWdb8RKFz97suh\nmRRX3L2xyCdx44KZkWbf5WbrU8aNJO9T2X/J/BlFStqEq7YXRkkY2h4/weXdFjQzH5lOUFAGVyAb\nt+6q2OinmryOqBE4fvsXHZvXisLVew8cxVFNdFMY3CuYpAgD+QJL5neyaM4xg/uY8uXth1EShmFB\n0My8oADK+pfe5Op7NqEFJZtXRqStuHs/U0g1IZdRI3C8+2dzeRIJKVJy1QjX8aMznH78EaH39xLk\nS3C2z50+gXWL5wWapEy4avthlIShZahnbH3QzDxrl0m964mXi7Y7wthvtl5tyGXUCBz3/qM6kpx9\n82NFr/cN5BoqXIN8CVHLsphw1fbCKAlDS1BvZ6ZbOCUQ9g/kKx9E8Gy92pDLqBE4zv49vVlEBHfk\nlvW8/gSF4V59zyZmTDosso/BhKu2F6GVhIh8ALgBOAIQ+09V9bA6jc0wTAjjzIxjlXHQJr+HT921\nfnAVUY5yppBGhlxu29lnJc/lc4PbRqSCS3/HtSorF4abTiTYsHVXVT6GRt67Vqw31U5EWUl8BZiv\nqr+t12AMwwvnx7u7r7+soHFHJQ06Q085psyZg7Fs8hO5/MzpfO3B50peTydgoECRT6JegiWK8Ko2\nOzvKqsw7nkphuAOFArOmjo3Vx1CPelMm3LY2oiiJV42CMMSF+8fbn8/bDuSDOILGT1B94WebQSiK\nmonKhXOmcdsjvy+aHWdSCf7tktlMHjOibHRTHEQVXmFt+dWGmPqN55jxo3yd/YekkxSwym1MP/LQ\n2HwMcQt0E24bD1GUxHoR+QnwM2CwTKWq/kfsozIMafx+vKmEJaQ7ksWCZuPWXSR9bO9L79vC2Z1H\n1VQC5MYFM0uE2+nHT6zp2sJQrfAKY8uvJsQ0aDxrLj+tZJWQSSW44+KTi8Jn4/Ax1EOgm3DbeIii\nJA4D9gPvdW1TwCiJBlAPu2ot56zlWL8f78h0itsXncSYkemic04ZN5KBfKm5I50sLfUQlWY5UGsR\nXpVs+X5mqf58gd19A/T0ZiMpln39ed9Vgp8irdXHUA+BbsJt4yG0klDVj0c9uYhMBe4CjsRSKMtV\n9WYRuRb4FPC6ves/quoD9jH/AHwSq0LbZ1X1P6O+71CjHnbVWs5Z63iCS3SXJneNH51hyfzOwaJx\nDvmCxvJjb5WaU3EJL69Z6kAuT75Q4LIVzwR+VuXGM3Pq2IYo0nrcExNuGw+iPmn0vjuKTAFuBeba\nmx4H/lZVt5U5ZhIwSVWfEZFDgaeB9wMLgV5V/apn/xnAvwPvBCYDvwSOV9XAeMXZs2fr+vXrQ11D\nO9LTm2XuDQ8X2eRHpBOsWzyv6i97LeeMazyrN2wv+fGWUzQrnnqJpfdtIZ0U8q7y0+0auRL1+qPi\nhK1aUVwHf+NBn1W9xxOGeo2hXb8j9UZEnlbV2ZX2i2Ju+i7wI+BD9vOL7G3vCTpAVV8BXrEf7xWR\n3wLlPvULgB+rahZ4QUS6sRTGExHGOaSoxzK8lnPGNR6vqQeskhhBP+RFc47h7M6jin7sta5o4hQe\nUc9Vb1PX+NEZxozsoCOZJJs7GDYbd95HnNRrDKY6bG1EURITVfW7ruffE5G/C3uwiBwLvAN4Cms1\ncrmIXAKsB65Q1Z1YCuRJ12Hb8FEqInIpcCnAtGnRyju3G/VYhtdyzjjH4/x4wwp794+9VkdncXRV\ngcvPnF7STyIs3vFfc+4MTjx6TEVBV2/hFfWzagVh2gpjMBQTpcd1j4hcJCJJ++8ioCfMgSIyGrgH\n+DtV3QN8E3gbMAtrpfG1KINW1eWqOltVZ0+cWP9olGbi2FVHpBMcmkkxIp2o2a5ayznjHk81fZ6d\nct/eqKewfY2975nNFfjag8/xZ19+qGLfZm/vbL/xf+Fnm1n07Sfr1gc6TP9uqM93xzD8iLKS+ASW\nT+LrWE7o/wEqOrNFJI2lIFY44bKq+qrr9X8D1thPtwNTXYdPsbcNa+qxDK/lnHGOJ6r5ypm1pxLR\ny30fTN4b8I3/z+bKV1aNkkvQm7XGFndc/r0btnP1qo0IQr5Q4NrzTyybWNhMM5LxBdRGq9y/KNFN\nLwHnRzm5WMVlvgP8VlVvcm2fZPsrAP4CcEJXVgM/EpGbsBzXxwG/ivKeQ5V6LMNrOWfQsVG/2FFM\nIkEZwKM6kuRVy86SvealfEAZ7iAFFSWXIMz5qqGnN8sVKzdgVROxnNFhEgubYcIxmc610Ur3r6KS\nEJGrVfUrInIr3r6QgKp+tszhc4GLgd+IyAZ72z8CHxWRWfb5XgT+2j5Xl4isBLYAOeCycpFNhtai\nmi92lDBFv1XHqEySpfM7OfOEIyIli6WTQkcS+vPlW52We29vLkGtzYwq0bVjD37lpq5d3VVTYmHc\nmEzn2mi1+xdmJeGU4ogcZ6qqa7EKAXp5oMwx1wPXR30vQ3Op5Ysd1iQS1NO6nIIAfwE/IpXk9kXv\n4Ik/9PDtx5+nI1V+NRI2l2Dzjt0sW7OlTnH5/uHqfj2km0ncEXmtYnZpFK2WKV5RSajqffb/7zvb\nRCQBjLad0AZDzV/sMCaRapOjggT81jf7+O7/vEgmnaQ/ryyZP6OqPghuITZz6tiSUN246Jw8hnRS\nBtuPOqjGk1gYF3FGwLWS2aVRtFqmeJRkuh8Bn8bKhP5frDIdN6vqjfUbXmWGejJdu1CPpL9y71VO\nCDuvu9uQrut+o0jAX3PeDJat2RJ5vN73brQQW71he0mf6psWzmo5wRlHYlwjv1OtRiOSG+uRTDdD\nVfeIyCLg58DnsTKom6okDK2BM9O+atVGkpIgr/UrgVBu1eEIbbC6y2WSgiSEr3zw7UVtNatd+cSZ\nq1ENcfeprpcpJ46oqkaaXVrNpNUKyY0OUZRE2g5nfT9wm6oOiEi4ZYhhWGB9GcRuR9WYrmlu/KKf\nsnmFvBXaum7xPGZOHTv4WmkhvDy7+/oDC+F5qbbiaq0//PGja+tT7dCIboC1CLdGmV1a1aTVKomF\nUZLpvoUViTQKeExEjgGMT8IAHBTQ2VyB/f15srnKSXFB5wlKFKuUROYIbT+8iXbeRLNUAgoKl634\ndegkuKhC7N4N25l7w8Nc9O2nAt8jbKJcrVSTxNhoGpEM2A73odlEyZO4BbjFteklETkz/iEZ2pE4\nTAPlZnRhZnt+QtvBT3j7tTN12oOGMRtFcaSHMU01ckbbahE0QdTb7NIu96GZROlx/bdYBf32At/G\nqsP0eeC/6jM0QztRq2mgnBAFQtn+3UIbSn0Sfj96qxBemo5koqjndVhBEVaIVRJGjfZvtFoETTnq\naXZpp/vQLCKV5bB7QbwPGIeVJPcDjJIwUHvt/nJC1HkcZrbnFtru6Ca3s9kr0MsJijA+hDBCrJIw\navSMttbPa6hg7kNloigJxxN5DvADOzu68d5JQ8tSi2mgkhCNo5ppkDknSFCs7X4jNvNPJWHUjBlt\nK0XQNBNzH8oTJU/iu1hlu98CzASSwKOqenL9hlcZkycxdCgXG15r3HiYmHv3qgHw3X/N5aeVrE6i\n4LcycbZt3r6bZfdvabkoG8PQpB55Ep/EKu39vKruF5HxhKgCazCEpdyMrtbZXteOPSTwLy3unMu9\nAtm4dVeJ+UcLyjm3riWTLC/Ey5movKuckl4U583gxMmVe1GEwS+p0MySDVGJoiQUmAGcB1yHFQo7\noh6DMrQ31eYCVDquWgemVV57U5FjGsqbc/zMP9m8Akp/LtixHCVCyc9ZvWzNllgyip1xaEHJ5pUR\naSs02KxODFGJkifxDeBU4KP2873A7bGPyNDWlMsFKJcD4HdcHDkD7vwNN5lUcMQTlMbodyRlUNA6\neHMvosbc++V1hG2c5L3GoEZIWbt8x4GBgskBMFRFlJXEHFU9SUR+DaCqO0Wko07jMrQh5cI4yzmB\n/Y773MoNJBMJOiqYdirhFzV0SEeSOy46qWLWsjdS6rzb1ha97l2JRI1QqtZZ7V5x+d3XoEZIlcZj\nMPgRRUkMiEgSu/qCiEwEn2+hITKtVjem2vEECcmuHbvL5gD4HZcrQK5QGFwBVJsz4CeIC6p0Th4T\n6ni3iatSqGQ1PaWjhl/6NU/KFQjdCMnkABiiEkVJ3AL8FDhCRK4HFgBfrMuohhGtVjemlvFMGTeS\nA7nipjvWcyk7wy6XKe23fxS8grg/n+eyM6ZHOodDJed5NUI/ikPeb8XlxdsIyc8n0QoTEUP7EKUs\nxwoReRo4Cytn4v2q+tsKhxnK0GodqOIYjzekWlWZPGZE2Rm2nyAvKEV9E6Jmb7uFriOIVzz1Mrc/\n0s3yx57n9ke7q1LIlZzn1URhhXXI+624vPg1QjLRTYZaCKUkbDNTl6qeADxb3yENH1qtbkyY0hHl\nhN+2nX2MTKfYm80NbhuZThXNbINm2F7h6u3/EHYGXG4l9I1Hu8nmajdhQbQw17jwW3Glk0JCoCOZ\nLLlPrVJF1NDehFISqpoXkd+JyDRVfbnegxoutFrdmHLjqbbAnt/MNkyIazUz8nIroTgVcrNMhEHm\nLJMtbKgnUXwS44AuEfkVsM/ZqKrnxz6qYUKz68Z4Z8NB44HoBfb8rsd5Dydks5JQizoTLqcI4lLI\nzTYRBilPoxwM9SKKkrimbqMYxjSrbkzQbNhvPH7Zx2EK7PldTz1n4eUUQVwKuRVMhMaMZGgkURzX\n/x315CIyFbgLOBIrdHa5XUn2cOAnwLFYjYwW2nkXAtyMVURwP/CXqvpM1PdtNxr9o680G/aOp5rQ\nTr/r6enNcvWqjWRzWnYWXm0IbiVFEIdCHtWRJOuJ4KrFqW4wtDpR+knsxelQeZDdwHrgClV93uew\nnP3aMyJyKPC0iDwI/CXwkKp+WUQ+j9WXYjHw58Bx9t8c4Jv2f0NEygkjv9lwMiE88uxrnHnCEaFD\nO4FQZiOHFU+9TDZX/BXyzsKjlrXwXmM5RVCrgHbGlkgI5LVir4qg41sl3DkujOIb2kQxN/0rsA34\nEVYI7EeAtwHPAHcCZ3gPUNVXgFfsx3tF5LdYlWQvcO3/feBRLCVxAXCXWnGUT4rIWBGZZJ/HEJJK\nwshvZbAvm2fJ6i6+eO9mX+HlFb6/2PxHTv2Xh0gnE+RVKwq8nt4stz/SXbJ9f39ucBYexd5f7hr9\nVjK1Cmi//tkF4OsLZnLq28aXPc4JQ22lcOe4GKqKz3CQKLWbzlfVb6nqXlXdo6rLgfep6k+wnNpl\nEZFjsbrZPQUc6RL8f8QyR4GlQLa6DttmbzOEJEz9IHddolGZ5OD2ff35svV9xo/OMHPqWH6x+Y98\n4Web6c9rxWMctu3sI5Xwbz+yc1//4D5a8ORZFLSkllHUGkmV9g9TI8qvztJAXi1BH9Cv2l2P6pxb\nHi95vZo6Ta2E6Q89PIiiJPaLyEIRSdh/C4ED9mtlm1KIyGjgHuDvVHWP+zV71RCuqcXB810qIutF\nZP3rr78e5dAhT9iicefPOpp1i+exdH4nozqSFfd36OnNsnTNlpLtSZGyAm/KuJEM5EuTwPIK59y6\nltUbtlv2/nzxVyGb18HxOcK8a8eeSIXxyt2TsAUJg7LC9wcoSa8A7c9r0SoE2r9ERlwFCg2tTRQl\nsQirZelr9t/FwEUiMhK4POggEUljKYgVqvof9uZXRWSS/fok+3wA24GprsOn2NuKUNXlqjpbVWdP\nnDgxwiUMfaI4mcePznDmCUeQ92RJlxNe23b20ZEsXREM5MsLvPGjMyyZ3+n7Wn/OErI7dveVVFod\nkbbKTLiF+afuWk/fQK5ov6hlvwcKhSITkHcm7FUe67rfGFx9HZJOlryHVzj6CVD3NY1IJ9q+REar\n5fkY6kNoJaGqz6vqfFWdYP/NV9VuVe1T1bV+x9jRSt8BfquqN7leWg18zH78MeBe1/ZLxOIUYLfx\nR0TDW+K6kjCKuv+UcSPJFUoXfkvmd1YUeItOOYbr/+JEUj5KxhKo/uYorzDP5gqICJmU1HSN+/rz\nvjPhrh17fJXH3OkTWLd4HndcfDKZlMf05BGO5epRFQrKmstPq+jDqbVMer2J+t0xtCdRopumALcC\nc+1NjwN/q6rbyhw2F2vF8RsR2WBv+0fgy8BKEfkk8BKw0H7tAazw126sEFjT+a4KooZ6ekti7+vP\n09ObDcyKdiKdkglhIK8smT+DRXOOCTW2RXOO4U+OGM1Hv/1USW2mzsmH+UZROcLcHY01IpXkXz5w\nIgcGCsyaOpbpRx5a9n3nTp/A8otnA1YFWCepz28mDBqYCzFz6lhOP34iNy4on3Ph3Kcr795Iv8eE\nlklZ9ziIdnIGm/7QQ58oPa4fxIps+oG96SJgkaq+p05jC4XpcW0RRxhireGnUd6jXMc077n9+lOn\nEoTuN1Huuvx6Z8+dPqFiP+yw96D71b2cc+vawW52QedynzPMexsMtRK2x3UUJbFBVWdV2tZojJKI\nZ+bZCOHk9x4dSeGBz76r4krALcz9qsQGjTXMdfkJez/lUe1sPsq5Nm7dxUXffqqoSOKhmRQ//Ks5\nzJw6tqr3Nxj8CKskouRJ9IjIRcC/288/CvRUMzhDfEStJRQ0+/ULUY273IRfEl8l04szXscfsG1n\nH7v7+rlsxa8ZyB8UpEFjDVNGwy+vIk4zSpRzBfXkcPs7TPKaoZFEURKfwPJJfB0rZPV/MD6DphOl\nllC5Fcfm7bvpzVZfbiIMUaNhgsYb5EvwO08tEThRyqVUEtxRzuXXk8OhnfwVhqFBqOgmu5/EB1T1\nfFWdqKpHqOr7TdnwxuIX8RJWCJZLfOrpzbLs/tLch2vOmxHrTLVcNIz32sqN13ueTCoR2G2u2gic\nKNFF5XItouL05HAzMp1i284+k7xmaApR+kl8FGsVYWgC5WaQl50xndse6S5y4kYxuziP3a+N6khy\nYsg+0FE4f9bRzJh0GBu27hqMSvK7tmPGjyq7QiruNvf7st3mopqOojrw4yy3UU7pt0IFWsPwI4q5\naZ2I3IZVvdXdT2LIV2ltNkGCaO+BHMvu32LH+iuXnv5WLpwzzVdgVFpxeF/Lq9bFDu4I4FRC6M8r\nV77neG765XMl17bm8tNCrZCsbnNKNpcbPNZPQIc190QV+pWUb9R7VqmSbb2T14y/w+AlipJwopiu\nc21TYF58wzH4EVS1del9XVa5B3v77Y92c+GcaYP7eH/wX/ng27lqlZXfkC9okfApJ5jisoP7Fcn7\n558/W5LBnU4kQrU8jWNm7S7At68/z+6+gUjnDFK+m7fv5sPLn6jqnpVrLHTNeTNYet8W0snSz7BW\njL/D4EeUfhJn1nMghmB8BVFeSSetcFAHtzDz+8Fb7k8FFbzlsoIEU5zmlKAif95ks7AtT2stC+Hc\nI4ADAwVW8rLiAAAgAElEQVQySQER8hF7Z3iV2TXnzmDZ/VtqumdBlWyXrdlCOiEM5Aosmd8ZmxBv\ndsc9Q+sSJeN6PLAEOA1LwqwFrlNVEwZbZ3wF0XkzWOYptOcIM78f/FWrNgJi93OwFItXCPgJpjjt\n4FPGjSxRCH64HeblzERBAtox9ZQbn9+qxiouqKSTQiYFHclkqA52XgUbxgTlrFyq6d/tsOz+LZx9\n4lGxCHHj7zAEEcXc9GPgMeCD9vNFWP6Jd8c9KEMpfjP9QzMpX3OMX7vRpCRKSiOFEQK1dmJzYxX5\nm8EXfrq57PtFcZi778vm7bsHfTTlzCU9vVkeefa1wNLlI1JJbl/0DsaM7AjduMirzMqZoODgysVp\nWhQ0zjCKJw4hbor1GYKIoiQmqeoy1/N/EpEPxz0gQzBeQRRkIvL7wee1YJuZDlJJCNTSiS1ImC6a\ncwwoLL2vi1RC2D9Q3mEeBuf8H17+REVziXNNSZHAJD6rjtSYqhsXVTJBOWTzCk5PioBxuleO9RTi\nlRzmhuFLFCXxXyLyEWCl/XwB8J/xD2l4UktfZ79InqB2o2GFgJ95Q0W4//LTmH7koWXHW0mYLjrl\nGM4+8Shr9r9jt21nr00whZlp+10TQCaVIJsrBCrBnt4sXTv2cPWqTWRz4Wz2YUxQYcbp7L9szZZB\nRVMvIW6K9Rn8iKIkPgX8HQcL/CWBfSLy11i9gw6Le3DDhXpElQT94MMIAccck5TilUcmebC3Q9B4\nozpAz+48irM7j6pZMIUxl/gJ6lEdSZae38msqWN9fQSDqymEbK74/JXMPZVMUFHGmU4kOPHoMYOl\nSeolxKNkhhuGB1Gim8pWYBORTlXtqn1Iw4t6RpUErTLKndedx+A1x3gb9fiNt1wL0rhDar3XVclc\n4m+GU8484YjAOld+Kw/3/fDLbPeLENu2s68o2MDrk6g0Tue9jBA3NJooK4lK/AA4KcbzDQtaKaok\nSCiO6kiSVw3s7eAeb5gWpEFKBqInn7mpZC6JancPMhEd0pGkoKU5CkFhx4s9vokTjx4TGN3kVSjG\nP2BoNnEqCf9QEUNZWimqxNcck0mydH7n4Gzbr7hef/5gldJ9/XlGpBMlpbmdVUmQUlzx1Mt849Hu\nmlcXlWbaUSuyeq81kxLuuOikEse2f9jxJkDJ5g4mPC67f0tg+XW3U30gX+DK9/4Jc9463vgHDE0l\nSo/rSoRrTGEoImwBuka0s/Q1xxSKzTHOeN3dOwsK67rfGDxH0LmD3qM/X+DWh3/fsMJ140dnmDl1\nbEXB6/fZ3LhgJqcfX2qe8utpnUyIFXrswtsL28GtZPb15+nPK//882fZvGO3URCGphLnSsJQJZVm\nt40ql1AuKmrj1l2DY5s7fQLJRIKcLewHPGGclUw67oKEB3J5cvkC3hy7VknkCrvyCFKw3rlT0Cpx\n286+kkABgKX3beHszngS5gyGaohTSfTHeK4hS9hkLPf+jSyX4BWKa7vfYO4ND5dUaO1IJoqiffwq\ntHqv063sQLno1GncufYFBnzWoK2UyBXGWVxr2PGUcSMZyPuExyalJZSlYfgSpSyHYGVZv1VVrxOR\nacBRqvorAFU9pU5jHDJUsyIIEy0UN45QDFJQfhVas7n8oHPafQ4Hv3N9Z+2LpBMJshRHUXWkwvV8\naDVqCTu2stE7+cLPirPR84XoyYUGQ5xE8Ul8AzgVq20pwF7g9thHNESptmFMpWiheuJnZ3dXaB2R\nTlhF8YBEQjjvtrWBDXd8z5WUktlzOik88JnT2rb6qNvf4fiRgFA+kEWnHMP1f3EiHakEozLJ0A2S\nDIZ6EsXcNEdVTxKRXwOo6k4R6ajTuIYc1fYdqBQtVE+sgnz+dZtmTh3LjEmHcc6tawEdHF+QKSzI\nZr9kfifX3tfFgK0IVZUtr+xh+pFl03Janmr9SIvmHBNLcqHBEBdRVhIDdhtTBRCRieBTY8CFiNwp\nIq+JyGbXtmtFZLuIbLD/znG99g8i0i0ivxOR90W8lpamXN+Bcq0vK0ULVUPYSKm13W/gtnSlEgxW\nWe3pzbKvP08mWT56xz2b9oviOvvEo3DX2csVqDmyqRGRYJXev5Y2o2Gir5p9jYbhQ5SVxC3AT4Ej\nROR6rNpNX6xwzPeA24C7PNu/rqpfdW8QkRnAR4BOYDLwSxE5XlXrP2VuAJWKvgU5peMuvBZ2husI\nugGXqUtEuG5N12AJ7WvOnUF/3j/Ho6c3a7cWLW6r6i0rsXHrLjqSycHOclBbZNO9G7Zztaux0o0L\n6tc4JygIod4JkqY5kKGRRCnLsUJEngbOwkqce7+q/rbCMY+JyLEh3+IC4MeqmgVeEJFu4J3AE2HH\n2OpE6TsQptprVKJESvmNzVEYjkBfsnoz6lpppJPCNefOYMVTL3Pbw8/hWMScKKir79nEusXzmDl1\n7OAxcSYT9vRmufLujUWK7Yq7N9YlEqycoK5ngqRpDmRoNKHNTSLyNuAFVb0d2Ay8R0TGVjgsiMtF\nZJNtjhpnbzsa2OraZ5u9bUjhNiVEESZhE8DKEeSIdsxHbvOF39i85AoU5TcUCsp1a7Zw04MHFYTf\ne7kJm0wYhq4du4sUBFiKrWvH7sjnKkclc5JzTZmUcEg6SSYVrrx6GMp9hgZDPYjik7gHyIvIdOBb\nwFTgR1W85zeBt2H1zH4F+FrUE4jIpSKyXkTWv/7661UMoTWIU0CGIYpfxDu2TCpRlGXtR14pqZTq\nfS8/BXj+rKNZt3geP/yrOaxbPK8G00lQZZjaKsZ4FWgYQa3O+0rt7++mlcq4GIYHUXwSBVXNicgH\ngNtU9VYn0ikKqvqq81hE/g1YYz/djqV4HKbY2/zOsRxYDjB79uy2LgdSjSmplt4TQW1Q/cwX3rGt\n635j8Nj+fJ6CUjJzDyJTIfehmuqm3vvQOfkwUglrheOQEDgkXX31GT+z0tzpE8oKamel4VaY5UxC\nUT7PuH1UBkMloiiJARH5KHAJMN/elo76hiIySVVfsZ/+BZbpCmA18CMRuQnLcX0c8Kuo529FKgmB\nKAKyVqdlVL+Ie2zeY2/+5XPc9eTLg8c5zmI3HUn4zLzjuXDOtJoFmfs+ru1+w/c+3LRwFlet2kiu\noOQLVl2pBd96koWzp7BozjGRFGuQ/X/d4nllBXUUx3XQ51nuO2OaAxkaSRQl8XHg08D1qvqCiLyF\ngw2IfBGRfwfOACaIyDZgCXCGiMzCWpG/CPw1gKp2ichKYAuQAy4bCpFNcUaixOW09CqlkuzpfCEw\nWc+djb3y6W1FrwlKJpWgI5mgP1/g8jOn16QcgpRCf75AvlAgV8B39TN5zAgWfOvJonOtXL+N+zft\nIK+E/gzKCftygjrIJDSqI1lUAyvo89x7IFexV3eUiUW1K8+w1Pv8huYSJbppC/BZ1/MXgBsqHPNR\nn83fKbP/9cD1YcfU6sQdiRJlhhr2h+s2X4DVDEdUOe+2tWWFqd9YRqZT3L7oJMaMTNcsMKxQ1o0k\nJUGukEcRBvJa0tvBISkHaxy92LPfd599/U4J742MPSTt28faTSX7f5Cg9jMJLZw9hfNuW1tSA8t7\nD5MiLF2zhf6QbVIrUe9wWROOO/SJEt10nIisEpEtIvK881fPwbUjbidn3JEoYZ2W927YXjZBz8v5\ns45mzeWnUbBNRdm8VkwACxpL5+TDao7C6unNcsXKDWRzyv6BPP35yr6Pff15NttRTLOmlg+6y+aU\nT//wmYr3ppbAArcz/oefeCc//tXWkmioUR3J0nuYL9CRLHZ0V/udqTWpr9nnN7QGUTx638WKTMoB\nZ2IlyP2wHoNqV7zCefP23bFGoowfneGac2fQkRRGdfjX9qn2h7tjdx+pCtnT3rHUKzKra8ceygRJ\nWWNLlkYMLVuzhZ7eLNOPPJRLTp1W9vj9/flQ96aWyCtrVbOPC7/zK/o9Ss5bA8u5h0vmd5Ir+JcX\nj5plXe9wWROOOzyI4pMYqaoPiYio6kvAtXZy3ZfqNLa2ws+0dN2aLr40vzO2NpT3btjOsvu30JFK\n0J9XlsyfUSK0qsn2dUw72Vy43gcOcTtQndXXnr4B39fTSWFEypp9X3bGdL712B/ozR50W7mv87oL\n/pRLTjmWDVt38eb+fm568DkSCPsHit1cYTKhq4m8cq5n8T2b6PfReO4aWN57eOiIVIlTPMhRX456\nh8uacNzhQRQlkRWRBPB7EbkcKzx1dH2G1X74CedsTnmzt7+kFEU1+PWfXramtCFN1B/uwXDNYgVR\nKWTVoZIADesbcdu2+/P5kkipdFL4+WffNdgXGuD2R7vLXuf0Iw8dLBT4wZOm0LVjD5+6a31RaGo9\nhVpQj+yOZHFynfceepUvwNwbHi6agFy5ahMzJh1WthBivcNlTTju8CCKkvhb4BAs5/UyLJPTx+ox\nqHbEr2IqwG2PdHPhnGlFpSiqIewKIeoP1++8h6ST3HHxyZx+/MTQ4/NTBmGdmt2v7uWquzfS73JM\npxKWonLXYPIKROc6nZ7Q15w7o2wexunHT+TGBY0Tan4KuyOV4IHPnFaxyq1bcWzcuqvkM+rPFTjn\nlsf56odmll1ROAqna8ceQOmcPKb6CypzfhPdNHSJEt30vwAiUlDVj9dvSO3J+NEZLj/zOL724HNF\n2zuS8RR2i7JCiPLDHdWRJJsrVm4FlM7Jh4UeW1DCWZjIrns3bOeqVZtKbPaVIqV6erMcM34Uf//u\n4/nag8/RkUqw7P4tHDoiFUpoNkKoeRW2ExY8blS0CvtBJVL6PW1jg6jGVBWFas1xhvYgSnTTqSKy\nBXjWfj5TRL5Rt5G1IRfOmUbGU7siLnNGGEex27EZptbTiidf4pxbHh98nklKZAd0kKO8a8fuik7N\nSjb7oEgpJ0Dgwn97kn/5+bP05wr0Zi1H9FWrNvHYc6+3TISN4/j+1OlvBZTljz0fKuLMjfPZd/jU\nRankKG5EBJIpWz60iWJu+lfgfViZ0ajqRhE5vS6jalPGj87U1ZxRbhYcNV59xZMvlbTKzCncvODt\nnPq2CaHHFGQGA6m48gmy2Tt9K4JKWHh9M26yuQKf/sHTFFDfe9CsuP5vPNpNNqeDFXSj5j6cP+to\nq8nTLY8XrboqTUJM2XJDrUQqaqOqWz2b2j4jOm7iK1bnj98KIepssac3y5LVm0u25wvKVas2RZrp\nlsuXcFY+ozJJOlIJrjmvWPAHmVEytunIbwx+YZde9g/4h7c2K64/rlDR6Uceylc/NDNS2HGjypab\nPImhSxQlsVVE/gxQEUmLyJVA2X4Sw5U4ynpHIaoQWvHUy4F5CH0DhUg/9nJmsPNnHc01585gIFcg\nnRCWrSkW/O5jR2UOlgHZ1x88hqAAgZE+Rfy89yDKfQoyoVRjWolTUEedhNQzn8XkSQwPopibPg3c\njNXjYTvwX8Bl9RiUIRpRhFBPb5bbH+ku2e4likkiyAzW05tl2f1b6M/roGD3mlmcYx959jWWrO4q\n6t3tNwa/lqpLzz+RqYePtMNbg00xYe/TiidfYumaLXQkhVzhoNmqWtNK3KGiYRzF7mizejnrTZ7E\n8CBKdNMbwKI6jsVQJVGE0LadfXQkE2X7PgD05/ORfux+gitK2O6ZJxzBF+8tNoF5BY5fS9VkIsHZ\nJx5l+4Nmlr0HYe6T21fTb3dUvfoeKyehljpcjYyqClJmcb+nyZMYHoRWEiLyFeCfgD7gF8Dbgb9X\nVVOaowUIK4R8Y/eTCa583/F85RfPDpqhCgrrut8ILFsdJkkuaue9SgLHT+m4Q4zL3QNnvHOnTwhM\nbuzpzbL0vq6SsSUTwgafXIWoDuBGhIo2ur2pyZMY+kQxN71XVa8Wkb/AKvH9AeAxTP2mIppZNjmM\nEAoSxnOnT+Br//UcOVuoD9gx+H5lqxVCmV2izjSdCJ4NW3cxa+rYkoSzMErH7x6ENRNt29lHOpko\n8XkM5JVZU8e2hWml3tFMfpg8iaFNFCXh7HsucLeq7haJry3jUCCucMC4FE3Qefxmfxu37ioxQyUT\nwtL7uooyoa9atQlQsjkNNVONMtMM04Anqnkjysx6yriR5LW02uyS+TOYfuShbWFaMX4CQ9xEURJr\nRORZLHPT34jIROBAfYbVfsS1zI9L0VQ6jzP7c6J1/MtWa8nMOpkQUMEd/VxpphrW0RrUgOe6NV0k\nJUFeC9y4YGakWljlZtbO68553Csfp9THkvmdLJpzDNAephXjJzDETRTH9edtv8RuVc2LyD7ggvoN\nrb2IY5kfh6Lp6c3StWMPV6+yeixXKonhViQLZ09h5fptJf2v3VhF94KrxVa7CvK7f0kRvnTvZiw/\ntaWUPrdyA0/947sr1sJyxuGr/AoFNm/fzYeXP1GiRCspgnYwrbSDMjO0DxWVhIjMU9WHReQDrm3u\nXf6jHgNrN+JY5odRNOWEsCP0E0hJ9JLfebwKaeX6bay5/LTBSqvjR2c4NFNathrwLaxXyyrI7/71\n5/J4ew3lCla/CXfxQe89qaj8zp3Bsvu3BCrjdlAE3mv2Pm+HazC0B2FWEqcDDwPzsaaQ4vlvlATx\nLPMrKZpyQrhSuYowJTGcRjjuWXqQM3nvgZyVS2BnRyNW6fJqV0F+9++06eP55W9f99n7oObw3hM/\nBeBVfs1w7sZJiRI8eQorn95Ws4nSYPAjjJLYKyKfAzZzUDmA1+ZgqHmZX07RVDJFBdVBOqQjSUG1\nRGFFaYXqV+F12f1WH2Ynl2Dp6i14m8VFFbzu+zeqI8m5tz5esk86KYPlrrtf3WtVkHWZ1Zbe11VS\nCM9P+bWrc9fve3DXky8DNCTk1TD8CKMknMZCfwL8H+BeLEUxH/hVncbVElRjX691mR+kaCrNfv2E\nfiYl3HHRSXROHuNrWy+38inn21h+8cml/Q3y/pVcB3J5Vq3f6hvS6odz/6xoq+RgQTyHz847btCk\n5PSgcGM52st32Gtn527QZMBNO62KDK1PRSWhqksBROQx4CRV3Ws/vxa4v66jayLNrG7pp2gqzfyD\nBN/pxx9RdEyYcg2VfBt+FV69ZFLC7GPGseBbTw5uu+TUaVx3wZ+GugdBSu/COdMOlhj3OiyAvFpt\nXSu1jG20czeusOagoohu2mVVZGgPooTAHgn0u57329sCEZE7gfOA11T1RHvb4cBPgGOxkvIWqupO\nsbzhNwPnAPuBv1TVZyKMLzYqmXaqzUCuhTCzXz/B5x5XUPMZbyhsJd+GU+H16ns2kRBhf39x8tkh\nHUn+/t3Hcf0DzxZtv+uJl7nklGNDryiCrtevUxtYXd+cazq786iKn0ejnLtxTjj87ovXMd8uqyJD\neyDqkzzku6PIF4CFwE/tTe8HfqKq/1LmmNOBXuAul5L4CvCmqn5ZRD4PjFPVxSJyDvAZLCUxB7hZ\nVedUGtfs2bN1/fr1oa4hLBu37uKibz/F3uxBU8ehmRQ//Ks5vNizr+QHHzYDOQpBSieKMvL2jS4o\nRXWPRqQTrFs8r0h5ZPMFRJWszyw9nRS+5mqXaZmkdpcU1rNcAlZxPC9fXfB2FsyeGupag17r6c0O\n9nx26EgKD3z2XaEUUCPxG+uIdKIkiqya85aLbjIYKiEiT6vq7Er7RcmTuF5Efg68y970cVX9dYVj\nHhORYz2bLwDOsB9/H3gUWGxvv0strfWkiIwVkUmq+krYMcZFkGnHPcs+mIG8EWyzTFyOw3Izz7Cz\nX7/VkJd0IkHXjj0V93NICMydPmHw/Nt29tE5eUxRYT0/ZeRmlie/IWzSn5ugVUarKQjw9yFoQTnn\n1rVkktVPKrz3xYS8GupFFHMTtvmnVhPQkS7B/0cOmqyOBtxNjbbZ2xquJIKE0L7+vE/CV+JgvJdN\nNUl0XTt2A8LkMSNqSqhzhPfuvoGKDk7L2awl+41IJ8gVlJxH0Hckk2zb2edrtnKyoLe+uY+rV/2G\nAZ+eDwtnTxkU5FGS/vyoxqfQjNm234TDWqXpYNtWE41kaGUiKYm4UVUVkcihtCJyKXApwLRp02If\nFwTb970/+LwW7DIVB4niOLx3w3auvHvj4Mw7KZD2CeEMo3SKzUsF8p6xphJWIqTzXvlCga1v9vk6\nQn/8V3O48NtPlbTK9FtNXX3PJtYtnseLPfsGhb6XQ9LC6o07OG36hEHznJ9jPCkSWsFGmT03KxDB\nO+HI5vIkElJkfoojGsmYmwz1IlL70ph4VUQmAdj/X7O3bwfcxuop9rYSVHW5qs5W1dkTJ07026Vm\n/H50zg/e3eXrxgUzuXFBdZ2/enqzXL2quD9CXilxGodROt5WktlcARGhIwmHpJNkUsLSC04k4dJn\nuYKVV/C5dx9fNP5rzp1BOpVkyfzOkutyVlNu3GaroD4V+weUAwMFrlq1iavu3sCBgQL7B0pXG/v6\n82zesbvSrYtEs9tsurvJPfDZd5W8Xms00r0btjP3hoe56NtPRWo9azCEoRkridXAx4Av2//vdW2/\nXER+jOW43t0MfwSUn3UGmTmqCafctrPPKpjnIZ0ASSSKbNbVFLJLAIrYufHCm739JbkH/Xnlqw8+\nx5LzZnDi0WPYvH13UWnwz737eA4f1TGY5+C3msrmC+zxMW9lUgkEOOBSHPlCIbB1qsPS+7Yw59jD\nY/MxtEKGtXvVU0uOhp/DupH9IwzDj7oqCRH5dywn9QQR2QYswVIOK0Xkk8BLWBFTAA9gRTZ1Y4XA\nfryeYwuimvBXqM5xOGXcSLtgXjGJhHD/Z6JFvwTbvhms4nrbI934Jcr35wosu38Lay4/raSkxT//\n/FlGdSTJ68E2no6QA2vVI6pccfcGfC6lxORVSUE44znnlsf5qiuSqhasvtj1z7AOa/KpNkfDb/Jy\nzPhRTVeAhqFNXc1NqvpRVZ2kqmlVnaKq31HVHlU9S1WPU9V3q+qb9r6qqpep6ttU9U9VNd641pCU\na+4e97J+/OgMNy54O2lXPYtUAm5cMJNxozqK9nXyGIJMJF5TWEdSis4LIAKfmPsW3+OTcrD7mpd9\n/XkODBS44u4NdL+6l/NnHc2ay08jbwvebN7qL6GqZFLW+6eTQr5QIJm0ztdR5puW8llN9eeVq1Zt\njMUktLb7jSJllU5K7LkEUb8b40dnmDl1bFUVgt0ms6AqtyaZzhAXTXVctyJRwl/jWNY7s0onuqlz\n8mGs7X6DuTc8POiAPuuEI3jo2dfoKBMy2dOb5Zjxowbj7wdy+aJsZ7Bm/Z2TxzCqI8m+fm/3tYJv\n97XifeDsmx/npoUzebFnP958u5HpFLcvOglQK3ciz2Cnu4IIh6QTJX6IdFK49vxOlt63ZTDaxyGb\nU3701Mt85qzjQtxJfxzh6j61O5Q3Dhph8ilXkLFdS4wY2gOjJDxECn9NCI88+xpnnnBETT/K8aMz\ng+Uz/ATOA5v/CDDoFPYKoCAzRCYpRUlxmaRw2Mh0QPe1zsHua1e4oq285ArKVas24ZeE2Z/P0zn5\nMLbt7CvxfVj+ldJjrrWb+sw59nD+/JbH8Pqyb3ukmwvnTKs6odC/L3YyVnOMvz9ISkqa10K5siwz\np441/SMMdaMZ0U0tjzsaZd3ieZw/62jfH+m+bJ4lq7tiMT055qSuHbt9TT5urGii3WzcuovuV/cG\nmiHEY8aRhAyW1BiRTjAqk6QjleD6vziRRadY3dfmTp/gTfvwJeUzxk/MfUtgscG8Kle+53g6UgkO\n6Sh93+lHHspn5x1fcs6O5MEuchDdrNOIdp6jOpJkPT6P/QN5PnXX+tgijcaPzrBw9pSibQtnTymK\nvItivjIYwhK6LEerUo+yHEGs3rDdaraTEPZli6e8TomLan6klcpneEknhYRYM2K/MhpOCZGXevaV\nrIjcJTX8Zp63PPR7bnrwubLjzaQS5PKFkoZAmVSCGxdY7+HcK2/Pg6K2oLaCcOjpzfJnX364KIzW\nfV+DSlxUuu/escSZI+F8dlrwL2VSy/fCTbXXbjAEEXtZDsNB/8Ejz77GktVdRXb9aiNK/MxLqYQl\ncP1yDjKpBPlCgWyekjLaDmHMEH7RWD29WW5/5Pdlx5tOCl86bwbX3tdF3iMUs7nCoCnM3azo2PGH\ncNGdvyoScMvu38KctxzOvv78oI9kyriR3Lgg2L5ebShrvSq+Vmr0FHZ8YWiFMF7D8MQoiYiMH53h\nzBOO4Iv3bi7aXq0Jo2vHHhIeA4/jAN64dRe3PfJ7OpJJ+vN5Lj/zOGZOHctlK54pKj44Ip2gUFAy\nqWSJYA0KzfVbSfj5EgD+5v++lVPfNgFQOiePYdvOPkakkgzkS5WUI7iKigbaWcZezrnlcQQrOmpE\n2jJfuUt8eAV6LaajetQ2CtPbIS7TVthrN5nXhrgxSqIKqmla4/fjDWqc45TjPv34iVw4Z1rF0iAA\nD3z2XSV5FUECIyhZMKhXwZTDDylxwAZFQTmRYN6aTF7blHf27Tx3SnzM9BQChNZrFuR3v1IJSCYS\nRZFocYwvzLU3sweKYehifBI1EHbWFtQCdM4//7IkuSyTEm5cUD6JzGtjv+a8GZw4eYxv4yCvwOh+\ndS/n3Lq2KNzUbdte8eRLfOFnxaskP9u3MwawBHwmKUjCyj94sWd/iV8jkxRUhEyyfDlyx5/ipyQc\nWmm27OfvqGekUbkS8sZnYYiC8Uk0gDAmjKAY+q8umOmbffy1D83kvJnlZ39uG/vm7btLurDNnT7B\n9z33Hsix9L6u0pafLtv2iUePYXQmSW+2vL9l7vQJLL94NqBMHjNycBUDcNWqh0oHLcIDdhb5qI4k\n5922tmR1AeHMM27zmPt5Mwjyd9RrTEHfOeOzMNQLoyRioNzMNujH+1LPPt9zHTYyHeq8zvMPL3+i\nRBksv/jkkjwGVXwVBBQL5injRpY0C/IK7nJmjaDe1JefOb2oFpNjOnGigtw+iSjVblvBrNIKvRwa\nEeprGJ4YJVEjlQSW34/3QC7PzQ+VRhGlk0Ln5DGD5716lRVumy/oYGipmyAF9PBvXyvqFAdW5NHI\nVHn4VqEAABP6SURBVKnzOCmUOLrL2b79Vkafu3sjk8eMYPZbxpftTe3GPQN3RzdVuzJrhYJ2zTSD\ntZq/xjB0MEqiBsIILO+P1+nz4I2a7EgKX/3QzEHn9JWerOcr7t5YIgj9BHJ/Ps+KX73sO96+XOkq\nQkRKSlR4TShgrRCmjBtZlNjmkMsrC771JJecOo3rLvjT0MKqmhl4HGaVegjzVljd1CvU1zC8MUqi\nBsIKLPePd3ffQEkI6yEdSe646KTB0hxdO3aXJNMN5JUn/tDD1MMPGRQAfrPHy86Yzjce7S6bjOcm\nV1C6duwefG8H5/xe4fe5dx8fmBdw1xMvc8kpx9YsrMoJcf+VWY6tb+6L3Pc7zn7k9VjdVKPMWsH0\nZRhaGCVRA1HswM6P1y+EtaA6aGbq6c3y/Ov+/oq//8mvGZFOFQk3v1n/rQ+XT4grpdgM5QgnK5R1\nI9mcDgq/rz74HB0J6A9IDdiwdRfTjzy0amEVpue1158xkIfL/30DqQTctHBWoNCvlzCvh9O4FVYm\nBgOY2k014S3PHaYrXbljnLpEN/7n73yPHSjg21nNXbdn/OgMn5kXrWrq1jf3Dz5210Z6378+VuLb\nSCfFqjkewKwyoauVCNtBzilVrh7llitQtrx4uTLwtRC307jZnfQMBjdmJVEjUU0r3pLefh3GwlCu\nF/SFc6Zx2yPdvmU9EkJJc6Bl92/h7BOPAiiZaXvJ5Qv8zf99Gzc/3F3y2snTxtTUTc5vRh5UaXdf\nf55UUvBUPCcpwTP4ekUAxe00NuGshlbCKIkYCGta8TMhOEljYUo8uNnXn+epF3oCM5OdGkjeYoR+\n3ePcs+lKY7j8zOOYNGaE72ubtu+hpzcb6l709Gbp2rEHp9RHUPVYp9LuF+/dXGRyCerql9dgoV/P\nCKA4ncYmnNXQShgl0SCC7OEzJh3Gjt197OnLlbTYrMQ/P/AsozIpFs05puS1csUIvbgFULmmQ5lU\nggvnTGPnvn7f19PJcLPdezds54qVGwaTCdNJ4Wt2q9KFJ0/hrieLo7Ocsbv9B+NHZ7hg1mRWrt82\nuF9CrK5+zSj2B/E5jU04q6GVMEqiAfT0Znnk2ddI+tjy33fzYzi6QbAE5ohUMlTJcICl923h7M6j\nAkNM/YoRukuNewWQWzj1DeQQscbj3m/86AwLZx/NyvXFvRIKqhVnuz29Wa5etbEo23wgr1xx90Zm\nTDqMlU9vCzzWbXJZ8eRLRQrCua4wHefaIQLIhLMaWgWjJOqMY2JKJaRkNu/1PyggKLcvegedk8ew\nrvsNrrKji4JIJ4N9ExA8Kw0SQH7RUn77zZ0+kXue3j5YWSNs3+htO/sQxL7agwzklbXdr5c1dzkr\nnp7eLEvv6yp5PRVyJdMutIMyMwx9jJKoE47N3amG6sZqXq9oQUtaeqYSScaM7GD86MygwP7RUy9z\n2yOWo9h7rnyh8uw97vpCjunMvcgJ2zfa8iX4K4EJo0f4mrtGdSTJqw4qoY1bd5FOWg2a3AzkK9+L\n4UgrFUQ0tB9GSdQBZ/WQQEoVRCbJ0vmdvLmvn3/++bMlx+bVKrXtZDiPH53hM2cdN1gy/KkXevjq\nfz1HOmmV6whrqw4zK+1+dS93rnuBe57ZTjopHBgooKoc0lGcmxGlb7RXQI0fneHa808sqTSbFDhs\nZIprzp3BsvsPFiz0q3A7ZdzIgD7dM2IVgnEI12YLaJNvYaiVpikJEXkR2AvkgZyqzhaRw4GfAMcC\nLwILVXVns8ZYCT8BUCmUNV9QZk0da1VB9ZAU+PDsqZx329qSH7UjYGdOHcsHT5oSu+D50s9+U+Qw\ndiWED2aHO45jv+ibbN5Sbm6CBNSiU44BgWtXd5FMCAO5AomEcNmKXwcqBii+344Jragdqo8Dv1ri\nEK7NFtCtXOfK0D40O5nuTFWd5app/nngIVU9DnjIft6SuJPO5t7w8GDDe7+ELbBKb2RSCS47Yzo7\ndh8o2WdkOsHNH5nFyqe3VUyicifPxUH3q3tLIor8cDuOnYRAp3qrqnLurWsH70OlhLBFc47hyX84\ni+UXzyaVTDCQ18H9lq3Z4tso6c++/BAfXf4kf/ZlqxT5usXz+NGnTuGJfzirpF92LcSRzNYKCXH1\nSh40DC+arSS8XAB83378feD9TRxLIOUEgN8sO52AhSdPAZRvPfYH/ur7/8v+/uJS2opVJjzOH3VP\nb5aNW3dVFEx3rnsh1PncobJO1vOAHZo1kFeyuQKfW7mBx557ja4duytey/jRGcaMTNORLL9fT2+W\nK1ZuIJtT9g/kyeaUz63cEBiKWytxCNdWENAm38IQB830SSjwXyKiwLdUdTlwpKq+Yr/+R+DIpo2u\nDOUyYmdOHVtSWyiRSPC9J14CIJs76GxNCkX2/s7JY2L7UYc1dfT0Zrnnme0+ZygmkyotObJjdx/e\n1I5cAf76B09TUC1J3PO7ljCCrGvHnpIGTbkC/PnNj5XUsoqDOIRrKwhok29hiINmKonTVHW7iBwB\nPCgiRV5cVVVbgZQgIpcClwJMmzbNb5e6UkkAnD/raGZMOoxzbl0LqG95DIBEQrh90Ul0Tj6sKE/h\nqlUbSUqCvFb3o45ii962s4+OZKJkjIJlIssVClx+puU4Lx2Hfw2nPtsfk0pYyqVcv+cw/Suef73X\n930GCjDg8Ze0SjJbqwhok29hqJWmKQlV3W7/f01Efgq8E3hVRCap6isiMgl4LeDY5cBysHpcN2rM\nDmEEwL7+PJlkoqiXtJeOZIIxI9NFx1kXI5b81eBCeuWIUvvHv0lQgvtdrUaDsrU7Jx9GKoFvG1aA\nkekUty86iTEj02UFVJAgc1ZDfkmIXuKubRSHcG0VAW3yLQy10BQlISKjgISq7rUfvxe4DlgNfAz4\nsv3/3maMLwyVBICf8PWS92QoOysA96zeb4ZcKayy0krHe7yfwpt+5KEVu+ONH53hpoWzuGrVRhII\nfbnS93SvksrhFWRBUWIj0wkKCvlCoThruw6mnDiEqxHQhnanWSuJI4GfijVDTAE/UtVfiMj/AitF\n5JPAS8DCJo0vFOUEgJ/wPX/mZH726+2kk4mi5DCHMCuAFU++xNI1W+hICjk7T8Jriy+30nH7Kvrz\n+UFT0rrF84oUR9jueG5luXnHbpat2RLZvOKn9PzuxaiOJEvP7+TME45gXfcbTTflGAzDAVGfpKR2\nYvbs2bp+/fpmDyMQrwAstwro6c0y94aHi2bPI9IJ1i2eN1ivyJuE5n49zHt7zw+Wecm7Snjsude4\n5M7/LTnnXZ/4PyVd7Mq9ZyWCHOyV7kU172UwGA4iIk+70g8CabUQ2CGHN6ehXI7D+NEZrjlvBh2p\nBKMyyaKGREH1ipIJCQyr9L5XUA5HNucXwx/kByjvH4iSw1EulNidixHU0CnufBGDwVCKKcvRQty7\nYbttrrGykJfM7xyc3W/b2VdzvaJyfhKvWcvPKZ1KWNvjopJ5rVUcvwbDcMasJFoE96x6X3+e/ryy\n7P4tg7P7OOoVObPzTKp0NeB1/DpO6UxKOCSdJJMSblo4K1ZBHSaXwKwWDIbmYlYSLUKlWbXbGV1L\nvSJvZdlyOQz1nsk711RrXojBYKgfRkm0CGFm1XEJbW9l2XLnqncIZxx5IQaDoX4Yc1OLEMZR6+wX\nl/ml2aYcd17I/v58gAPdYDA0E7OSaCGGm6M2Sma4wWBoDkZJtBjDJUO3pzfL7r4B+vPNLYJnMBjK\nY5SEoeG4E+jyhQLppDAilTSZ0wZDC2KUhKGh+FWozaTg9kXvoHPyGKMgDIYWwziuDQ3FL+u7I5lk\nzMgOoyAMhhbEKAlDQ2mFZjwGgyE8RkkYGkrYUF+DwdAaGJ+EoeEMt1Bfg6GdMUrC0BSGS6ivwdDu\nGHOTwWAwGAIxSsJgMBgMgRglYTAYDIZAjJIwGAwGQyBGSRgMBoMhEFGfbmfthIi8DrwU4ZAJwBt1\nGk6zGcrXBkP7+obytcHQvr52vbZjVHVipZ3aXklERUTWq+rsZo+jHgzla4OhfX1D+dpgaF/fUL42\nMOYmg8FgMJTBKAmDwWAwBDIclcTyZg+gjgzla4OhfX1D+dpgaF/fUL624eeTMBgMBkN4huNKwmAw\nGAwhGZJKQkQOF5EHReT39v9xAfv9QkR2icgaz/a3iMhTItItIj8RkY7GjLwyEa7tY/Y+vxeRj7m2\nPyoivxORDfbfEY0bfTAicrY9rm4R+bzP6xn7s+i2P5tjXa/9g739dyLyvkaOOwzVXpuIHCsifa7P\n6o5Gjz0MIa7vdBF5RkRyIrLA85rv97RVqPHa8q7PbnXjRh0zqjrk/oCvAJ+3H38euCFgv7OA+cAa\nz/aVwEfsx3cAf9Psa4pybcDhwPP2/3H243H2a48Cs5t9HZ7xJoE/AG8FOoCNwAzPPv8/cIf9+CPA\nT+zHM+z9M8Bb7PMkm31NMV3bscDmZl9DDNd3LPB24C5gQZjvaSv81XJt9mu9zb6GOP6G5EoCuAD4\nvv34+8D7/XZS1YeAve5tIiLAPGBVpeObRJhrex/woKq+qao7gQeBsxs0vmp4J9Ctqs+raj/wY6zr\ndOO+7lXAWfZndQHwY1XNquoLQLd9vlahlmtrBypen6q+qKqbgILn2Fb/ntZybUOGoaokjlTVV+zH\nfwSOjHDseGCXqubs59uAo+McXI2Eubajga2u595r+K69BL6mRYRRpfEW7WN/NruxPqswxzaTWq4N\n4C0i8msR+W8ReVe9B1sFtdz/ofDZlWOEiKwXkSdFpJUmmpFo26ZDIvJL4Cifl77gfqKqKiJtFcJV\n52tbpKrbReRQ4B7gYqylsqH1eAWYpqo9InIy8DMR6VTVPc0emCEUx9i/tbcCD4vIb1T1D80eVFTa\nVkmo6ruDXhORV0Vkkqq+IiKTgNcinLoHGCsiKXtWNwXYXuNwIxHDtW0HznA9n4Lli0BVt9v/94rI\nj7CW1M1WEtuBqa7nfvfc2WebiKSAMVifVZhjm0nV16aWYTsLoKpPi8gfgOOB9XUfdXhquf+B39MW\noabvluu39ryIPAq8A8vH0VYMVXPTasCJlPgYcG/YA+0f5iOAE6kQ6fgGEOba/hN4r4iMs6Of3gv8\np4ikRGQCgIikgfOAzQ0YcyX+FzjOjirrwHLeeqNB3Ne9AHjY/qxWAx+xI4TeAhwH/KpB4w5D1dcm\nIhNFJAlgz0aPw3LuthJhri8I3+9pncZZDVVfm31NGfvxBGAusKVuI60nzfac1+MPy577EPB74JfA\n4fb22cC3Xfs9DrwO9GHZG99nb38rlqDpBu4GMs2+piqu7RP2+LuBj9vbRgFPA5uALuBmWiQSCDgH\neA5rpvUFe9t1wPn24xH2Z9FtfzZvdR37Bfu43wF/3uxrievagA/an9MG4BlgfrOvpcrr+z/272sf\n1uqvq9z3tJX+qr024M+A32BFRP0G+GSzr6XaP5NxbTAYDIZAhqq5yWAwGAwxYJSEwWAwGAIxSsJg\nMBgMgRglYTAYDIZAjJIwGAwGQyBGSRgMBoMhEKMkDG2FXT67qgRAETlDPGXhWx0RmS0it0Q85loR\nubJeYzIML9q2LIfBMBxQ1fW0VhkOwzDDrCQM7UhKRFaIyG9FZJWIHCIiZ9nVUn8jIne6SiKcLSLP\nisgzwAfsbQm7yc1E1/Nu57kXEfmeiHzTrub5vL0iudN+/++59vumXfWzS0SWurZ/WUS2iMgmEfmq\nve1DIrJZRDaKyGNBF+pe/dgrhDvFahz1vIh81rXfF0TkORFZC/yJa/vbxGqu9bSIPC4iJ9jb7xWR\nS+zHfy0iKyJ/CobhQbNTvs2f+Yvyh9XkRYG59vM7gS9ilXQ+3t52F/B3WOUutmLVPBKsZlJr7H2W\nAH9nP34vcE+Z9/weVi8Bp3/FHuBPsSZZTwOz7P2cEilJrEJ1b8cqo/I7DvaTH2v//w1wtHtbwHuf\n4RrztcD/YDVYmoBVBiINnGyf7xDgMKwSF1faxzwEHGc/noNVFwqsEvPdwLuwyk4c3uzP1vy15p9Z\nSRjaka2qus5+/EOsDoMvqOpz9rbvA6cDJ9jbf6+qau/rcCdwif34E8B3K7znffY5fgO8qqq/UdUC\nVm2lY+19Ftorll8DnVhd83YDB4DviMgHgP32vuuA74nIp7CUSljuV6vB0htYFYCPxBL0P1XV/WqV\nEV8NICKjsWoI3S0iG4BvAZMAVPVV4EtYxSyvUNU3I4zBMIwwSsLQjngLju2KfALVrcCrIjIPq1z6\nzysckrX/F1yPnecpuwLtlcBZqvp24H5ghFrl5t+J1XHuPOAX9vt/GmsFNBV4WkTGEw73e+cp71dM\nYDXQmuX6+/9cr/8p1mpkcsj3NgxDjJIwtCPTRORU+/GFWI7dY0Vkur3tYuC/gWft7W+zt3/Uc55v\nY60u7lbVfI1jOgyrEuhuETkS+HMYnM2PUdUHgL8HZtrb36aqT6nql7AqEU/1P20oHgPeLyIj7WZS\n8wHsVcULIvIh+z1FRJz3f6c9xncAV9pKzmAowSgJQzvyO+AyEfktMA74OvBxLLPKb7Bm93eo6gHg\nUuB+2wzkbdC0GhhNZVNTRVR1I5aZ6VngR1jmJIBDgTUisglYC3zO3n6j7WTfjOVn2FjDez8D/MQ+\nx8+x+iA4LAI+KSIbsUxjF/y/9u7QBqEgiALg257oAY2gARwaQ4JGoCiAhAYoCkcbi+ArksNBfsKM\nXLPy3WVzt9NQ/5Jk092PJLsk16pZrLJlZnwVzt+qqkWSc3fPcXc0zIJ3Evylqton2eZ10gYG3CRg\nUlWHJOu38q27jz/ovUxyeivfu3v17d7wiZAAYMjgGoAhIQHAkJAAYEhIADAkJAAYegIdxiTfWR0v\nRwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ef17e48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.plot(x='body_mass_index', y='disease_progression', kind='scatter')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Cross-validation\n",
    "\n",
    "* To judge how good our model is, we need to reserve a portion of the data as the \"test\" dataset.\n",
    "* The remaining portion of data that we train our model on is the \"training\" dataset\n",
    "* To compare the performance of different models, we compare them on the test set. This prevents us from thinking a model is very good if it \"cheats\" by fitting itself exactly to the data it's training on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Break up the data into test and train sets\n",
    "df['is_train'] = np.random.uniform(0, 1, len(df)) <= .75 # randomly assign 75% of data to training set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Exercise: Using the `is_train` column as a mask, create two dataframes called `train` and `test` that only contain data with `True` and `False` values, respectively.\n",
    "\n",
    "```\n",
    "train = ???\n",
    "test = ???\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Solution\n",
    "train = df[df['is_train'] == True]\n",
    "test = df[df['is_train'] == False]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/site-packages/ipykernel/__main__.py:1: FutureWarning: reshape is deprecated and will raise in a subsequent release. Please use .values.reshape(...) instead\n",
      "  if __name__ == '__main__':\n",
      "/usr/local/lib/python3.6/site-packages/ipykernel/__main__.py:7: FutureWarning: reshape is deprecated and will raise in a subsequent release. Please use .values.reshape(...) instead\n"
     ]
    }
   ],
   "source": [
    "X_train = train['body_mass_index'].reshape(-1, 1)\n",
    "# X_train is now a 2D numpy array where rows are samples and columns are features\n",
    "# the second parameter, 1, means the new array should have 1 column, since we are only looking at one feature\n",
    "# the first parameter, -1, means 'infer the appropriate number of rows', which should be len(df)\n",
    "y_train = train['disease_progression']\n",
    "\n",
    "X_test = test['body_mass_index'].reshape(-1, 1)\n",
    "y_test = test['disease_progression']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# use the linear regression model from scikit-learn!\n",
    "ols = linear_model.LinearRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "model = ols.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 964.10418911]\n",
      "152.38236033\n"
     ]
    }
   ],
   "source": [
    "# View the training model's slope and y-intercept\n",
    "print(model.coef_)\n",
    "print(model.intercept_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "211.86393148711284"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict(X_test)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "151.0"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(y_test)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+wZGV95/H3915mBkckMM2oU8DthugmNRqL4A0hCVqJ\nBg3giq5UYnJxKQ11a0c3sqJVO9StrbBWTbK6lS0FUuplwTLeu0KtMYAEikKE3WxhQe64/DQ1MgPd\nIwyrE+RXHAYY+tk/+vRMz73dfZ7TfZ7zqz+vqq7pe/rc7qfP3P6cp7/Pc84x5xwiIlJdU3k3QERE\nwlLQi4hUnIJeRKTiFPQiIhWnoBcRqTgFvYhIxSnoRUQqTkEvIlJxCnoRkYo7Ju8GAJx00kmu0Wjk\n3QwRkVLZuXPnPzvnNsetV4igbzQarKys5N0MEZFSMbOWz3oq3YiIVJyCXkSk4hT0IiIVp6AXEak4\nBb2ISMUp6EUkE8vLyzQaDaampmg0GiwvL+fdpIlRiOmVIlJty8vLzM/Pc+DAAQBarRbz8/MAzM3N\n5dm0iaAevYgEt7CwcDjkuw4cOMDCwkJOLZosCnoRCW7v3r2Jlku6FPQiEtzMzEyi5ZIuBb2IBLdj\nxw42btx41LKNGzeyY8eOnFo0WRT0IhLc3Nwci4uL1Ot1zIx6vc7i4qIGYjNizrm828Ds7KzTSc1E\nRJIxs53Oudm49dSjFxGpOAW9iEjFKehFRCpOQS8iUnEKehGRilPQi4hUnIJeRKTiFPQiE0anC548\nOk2xyATR6YInk3r0IhNEpwueTAp6kQmi0wVPJgW9yATR6YInk4JeJGVFHuzU6YInU2zQm9mxZna/\nmT1oZo+a2X+Olp9mZveZ2W4zu9HM1kfLN0Q/744eb4R9CyLF0R3sbLVaOOcOD3YWJex1uuDJFHua\nYjMz4PXOuX8xs3XA/wEuAy4HvuOcu8HMvgo86Jz7ipl9EniHc+7fmdlHgQ875/5o2GvoNMVSFY1G\ng1artWZ5vV6n2Wxm3yCptNROU+w6/iX6cV10c8B7gG9Hy78BfCi6f2H0M9Hj7412FiKFk3aZRYOd\nUkReNXozmzazB4CfAXcCe4DnnHOHolWeBE6O7p8M/AQgevx5oNbnOefNbMXMVvbv3z/euxAZQYgy\niwY7pYi8gt4595pz7gzgFOAs4FfHfWHn3KJzbtY5N7t58+Zxn04ksRBzyjXYKUWUaNaNc+454G7g\nt4ATzKx7ZO0pwFPR/aeAUwGix38JeCaV1oqkKESZRYOdUkQ+s242m9kJ0f3XAecC/0Qn8C+KVrsE\nuDm6f0v0M9Hj33dFuDCt5Kao0w1DlVnm5uZoNpu0222azaZCXnLn06PfAtxtZg8B/wjc6Zy7FfiP\nwOVmtptODf66aP3rgFq0/HJge/rNlrIo8nRDlVlkUsROr8yCpldWV9GnGy4vL7OwsMDevXuZmZlh\nx44d6oFLafhOr1TQS1BTU1P0+xszM9rtdg4tEqmO1ObRi4xD0w1F8qegl6BUBxfJn4JegtJ0Q5H8\nKeglsaTTJTXdUCRfupSgJKJL0YmUj3r0koguRSdSPgp6SURnZxQpHwW9JKLpkiLlo6CXRDRdUqR8\nFPSSiKZLipSPToEgIlJSOgWCiIgACnoRkcpT0IsMUNQLpogkpSNjRfrQEcBSJerRi/ShI4ClShT0\nIn3oCGDx9fzz8Hu/B/U6/MVfQAEmMq6hoBfpQ0cAS5wHHwQzOOEEuOce2LsXFhZg1668W7aWgl5K\nKfRAqY4AlkGuu64T8GecsfaxY46BN785+zbFUdBL6XQHSlutFs65wwOlaYa9jgCWXq+8Ahdf3An4\nSy/tv86mTfDjH3d6+EWjI2OldBqNBq1Wa83yer1Os9nMvkFSWa0W/PZvw759g9f5yEfgm9+E170u\nu3Z16cjYCTUJc781UCqh3XZbp/feaAwO+Wuu6Qy8fvvb+YR8ErFBb2anmtndZvYjM3vUzC6Lll9p\nZk+Z2QPR7fye37nCzHab2S4ze3/INyBHpFnSKPIOQwOlxVTkvxkf7TZs394J+AsuGLzeffd1Av5T\nn8qubWNzzg29AVuAM6P7bwB+DGwFrgQ+12f9rcCDwAbgNGAPMD3sNd75znc6GV+9XnfAmlu9Xk/0\nPEtLS27jxo1HPcfGjRvd0tJSmIYnVPT2TaIy/58884xzZ53lXCe++99+8zc76xUNsOJiMtx13kL8\nSkf9AtwMnDsk6K8Aruj5+Q7gt4Y9p4I+HWbWN+jNLNHzpLXDCGlpacnV63VnZq5erwcJlCxeoyrK\n8Dez2v33Dw93cO6KK5x77bW8WzpYkKAHGsBe4Pgo6JvAQ8D1wInROtcAF/f8znXARcOeV0GfjrQ+\nbGntMMqszD3UPJTpb+bqq+MD/u//Pu9W+vENeu/BWDM7Dvhb4D84514AvgL8MnAG8DTwV77PFT3f\nvJmtmNnK/v37k/yqDJDW3O+y1MBD1oTzOgVCWevcRf+befbZTu3dDP7sz/qvc/LJ0Gx2ov788/uv\nU1o+ewNgHZ0SzOUDHm8AjziVbnKXRrmhDL3Z0G3Mo4dahu0+SFHbfvvt8b33j33MuZdfzrWZIyOt\n0g1gwN8AX1q1fEvP/c8AN0T338bRg7GPo8HY0il6fTp0TTiPmnMZ69y9ivQ38yd/Eh/w112XW/NS\nk2bQnxP9wT0EPBDdzge+CTwcLb9lVfAv0Jltsws4L+41FPTlk/eHOnSPO48eapnq3EV08GB8uHdu\nv1GIbxtpSC3os7gp6MulCF/Ts+j9Zr0zK3uPPi933eUb8CdXbrsq6CWYIgRSEXY2aaviewrpvPPi\nw33DBuegut+UfINep0CQxIpwCoIqnnSsiu8pbc4dmT1z++2D11tY6Kx78CDU6+FnBBV+tpTP3iD0\nTT36cilCjz6pvMcUZDx79viVZ7773bW/G/qbUp7fxFDpRkIpW4mhbO2VIz7/eb+Af/HF4c8Tckef\nZ8dHQS9BlamHnPYHsUzvvaz8BlfzbmVHnrOlFPRSSiFCNM0Por4dhPPCC37hvmNH3i09mnr0CnpJ\nIFSIpvlBLOP4RNH93d/5BfwTT+Td0v5Uo1fQSwKhQjTND6IOakrP7KxfwLfbebc0Xl7lPN+g16UE\npTCmpqbo9/doZrTb7bGee3l5mYWFBfbu3cvMzAw7duwYadqiLmM4nnYbpqfj1/vgB+Hmm8O3p+x0\nKUEpnZBnQJybm6PZbNJut2k2m31D3mcudFpnCJ00997bmfseF/L33NPpxyvkU+bT7Q99U+lGnMu3\n1pnktTXrxt/v/q5feebgwbxbWk6odCNllFaJJSmVZNJlFr/OccfBiy+Gb0uVqXQjmUj70G+fEksI\nRTitQxn1/v+fcso5h09PMMxnP9vpxyvks3NM3g2Q8lpeXmZ+fv7wlZharRbz8/MApTs/y8zMTN8e\nfVGukFRER/7/rwY+wVNPDV//iSeg0ciiZbKaevQystCX28vyRFEaZE3u4ovnOHDgF8Anhq7XrcQr\n5POjoJeRhSx3dHuLrVYL59zhbwuhwl5njvTz0kt4lWfe8pYjAS/502CsjCzkAKYGR4vl6qvh05/2\nWfOPqNfv0/9RRjQYK8GFLHdocLQYur33+JDfABgbN96qclcBKehlZCHLHSEPnpJ4PuUZgKWlZer1\nBmavqtxVYCrdSCGtntEDnW8LCpJw7r4b3vOe+PUuuABuvTV8eySeb+lG0yulkLphnsfBU5PGp+cO\nsGcPnH562LZIGCrdSGaSTpfM6+CpSeFbnunOnlHIl5d69JKJKh1cVWb798Mb3+i3bgGqupIS9egl\nE6EPrpLh3vveTu89LuS/9jXNf6+i2KA3s1PN7G4z+5GZPWpml0XLN5nZnWb2WPTvidFyM7OrzGy3\nmT1kZmeGfhNSfJM6XTLLo3v76ZZnvv/94esdOtQJ9+hLllSMT4/+EPBZ59xW4GzgU2a2FdgO3OWc\neytwV/QzwHnAW6PbPPCV1FstpTOJ0yWzPrq3q91OXn/3uRhI0eW9Uy00n3MZ996Am4FzgV3AlmjZ\nFmBXdP9rwB/3rH94vUE3nY++eNI+53rSc81X4ZzvWV9f9stf9jv3+3veE+TlczWpF20nxDVjgQaw\nFzgeeK5nuXV/Bm4Fzul57C5gdtjzKuiLJdSHpje8a7Waq9VqfYO8Kh/arK4v6xPu4Nzu3am+bKFM\n6kXbUw964DhgJ/Bvop+fW/X4sy5B0NMp66wAKzMzM1lsE/EU+kMTF+Rl/ND2+wYS+n34BvwkmNSL\ntqca9MA64A7g8p5lKt1UVOgPTVwAlu1DO2jHtW3bttS/mfzoRwr4fsrYOUiDb9D7zLox4Drgn5xz\n/63noVuAS6L7l9Cp3XeX/9to9s3ZwPPOuafjXkeKI/TAadwMnLIN3A6aOnrbbbeldi6g7uDq1q3D\n17v22smcHqnrCcSI2xMA59DZOz4EPBDdzgdqdMoyjwHfAzZF6xvw18Ae4GFi6vNOPfpMJBncDF0j\nj+t9la1GH/IbiG/vvd1O4Y2UXBUG8JMixGBsqJuCPqxRgjPkh8anPWX60KZdNjh4UOUZ8aOgl8OK\nWL8sU5DHSesbyLnn+oX7r/3aaG2syvaWIxT0cljZBjfLaJwg9e2979s3ett8d0TaIZSLgn4C+H4o\ni9ijF/+AH5fv/3/ZxkZEQV95SXtp+gAXw3e+k13Ad/l+o1OHoHwU9BWX9ENZhK/kRWhDXnzD/etf\nT/+1ff9WVOIrHwV9xZXtQzmp3ypG7b2nuVP03fbq0ZePgr5C8ji8Pm2D2lur1fJuWup++tPRA965\nMDtFnx3HpO6My0xBXxFZHl4f0qBvIEBh25yUb7i/4x3DnyfPnfgkl9fKSEFfEcM+9GX6UA56H0X+\nFuLLN+BfeMHv+cpWlpP8+Aa9ddbN1+zsrFtZWcm7GYU0NTVFv/8jM6PdbufQotEsLy9z8cUX932s\nbO+ly+fCHtCJ+SQajQatVmvN8nq9TrPZTPZkUmlmttM5Nxu3nq4ZW3BlO8HXIHNzc9Rqtb6PFe29\nDLtS0aWXJr96U1I6QZekzqfbH/qm0s1gVRogK8N7GdRG3/LMt76VXjvKUpaT/KAafVhZfhDz+NCH\nes2iB9jasQS/gBfJg2/QH5PHt4iy6170uXsO8u5Fn4GRzjUeZ25uLsjzDpL1+yuSzjnxzwLu81q/\nAENcIvF89gahb2Xr0ZdtDntSod5f0Us3vr33N70p23YV/VuQ5AfNugmnKjNhBgn1/oo6m8R39szL\nL8P69WHbstrqb1fQGZgd9UpVUi2adRNQVWbCDDLofUxNTR01AyWpuEsIDpvtkrbXXvOfPbO0tIxz\n2Yc8DL5M4cLCgvdzZLldpaB8uv2hb2Ur3VT9/N793l/c+/QRd/BXFmWdqSn/Ek0RjHvwVNHLZTIe\nNOsmrKqfO2RpaclNT0+nWqvvtz3Wr1/varVa39cZ57VW8w33225L5eVSM+54SdXHkyadgr4Ayv4h\nC3Eofu8OslaruXXr1g0M+XFfy7ly9d77GbezoNMpVJtv0KtGH1BcTbroQoxFzM3N0Ww2abfbHHfc\ncbz66qsjtWGYq64Kf/RqVubm5lhcXKRer2Nm1Ov1RAOxVR9PEj8K+oDK/iELfSh+3A4v6Wt1w/2y\ny4avd/rpxQ/4Xr07x2azmWi2jU6nIIBKNyGVuUbfFXIwOe6Mlr6v5VueabdTa3qplHFCgPhBNfpi\n0IdssHF2hM88U/76u8i4fIM+tnRjZteb2c/M7JGeZVea2VNm9kB0O7/nsSvMbLeZ7TKz96f33aOc\nxvnaXXWj1J+75ZkBJ8I8Sm95RnPJ/WlbVVDcngB4N3Am8EjPsiuBz/VZdyvwILABOA3YA0zHvUaV\ne/SSDt/e+w9+sPZ3q1BCy4q2VbmQVo/eOfe/gZ977jcuBG5wzr3snHsC2E3nDFEiI0k6e+bss9c+\nlsbRpZNC26qaxpl18+/N7KGotHNitOxk4Cc96zwZLZMclPUr+Ic/nO70yLJPc82StlU1jRr0XwF+\nGTgDeBr4q6RPYGbzZrZiZiv79+8fsRkySPdkWK1WC+fc4VMNFznsu+F+003D19u6Ndn0yLJPc82S\ntlU1jRT0zrmfOudec861gWs5Up55Cji1Z9VTomX9nmPROTfrnJvdvHnzKM2QIcr0FTxp7/3RR9c+\nNuzbS7+55GZGq9Uq1TedLGjefUX5FPKBBkcPxm7puf8ZOnV5gLdx9GDs42gwNhdFP/T94YfTmx7p\nM4DYneba3QbD1p10mhJcHqQ1jx74Fp3yzKt0au5/CnwTeBh4CLhlVfAv0Jltsws4z6cRCvr0FfU8\nO77hDv6Bk+S9FnW7+FIIS6/Ugj6Lm4I+fUWbJucb7vv2JW9/km8vRf+mM0zR/k8lfwp6KUTvb9Ty\nTBq99Onp6TXvucw9+jK3XcJQ0EtuTjtt9IDvStLzTnKhlDL3isv8bUTC8A16nb1SUtOdPfPEE8PX\n+/M/PxL1gySZ5tc9lcL09PSax1bPNBr3tL950tRHGZnP3iD0TT36chu3997PKD3vqvd4y/xtRMJA\nPXoJ6Y47wl7cY5Se96Ce7aZNm5K9eEGV+duI5Mtc0k9gALOzs25lZSXvZogHn2DvyvpPa3l5mY9/\n/ONrrlq1fv16rr/+egWiVI6Z7XTOzcatpx59wRT1/DS+vfeDB0frwadhbm6O448/fs3yV155JdER\nwUX9PxAZ1TF5N0CO6J6fpnvqgu75aYBceqOHDsG6dX7rFuCLIQA//3n/E636npSraP8HImlQj75A\nQpyfZpTeabf37hPyefXeBxl3ZkqZzhEk4ktBXyDjnCK2X6AnPYOlb3nmppviAz6L8ke/1xj3pFw6\nTa9Uks/UnNA3Ta/sGPXIx0HT7mq1mtfzpT09MotpgMNeY5wjgtM++rQIRydLdaEjY8tn1IAcFE6D\nbmbmrroqzPz3Ye1J81D9UK+R5k5K894lNAV9SY3SAxx0oFD/m1+4b948+nvI4sClkK+RVi9c56aR\n0HyDXvPoK6DRaNBqtdYsr9VqvPTSS9Hgot//c7udbK58kvbU63WazeZ4T57ha4xramqKfp8vM6Pd\nbufQIqkazaPvo6rzowcNQP7lX/41Bw78Ap+Q7/blxw35Ye1J8ypFZbgSks5NI4Xh0+0PfcuidBOy\nXlqEAbfeNviWZ0apv4/SnjS2Sb/nK8J2H0Y1egkN1eiPVobBu3H5hvsjj2TetLEUaRsnVfSdkZSb\nb9BPTI0+VL20CLVi33JLAf6rR1KEbSxSRKrRrxKqXprXATZf+lLYs0cWiQ5iEhnPxAR9qMG7rAfc\nuuH+mc8MX+/yy/0CvgwD1Em3cRnek0imfOo7oW9ZzaMPUS/Nqn4cYnC1LLVvn3Z2/2+J5tIX/T2J\npAENxmYn1IDb/v1hAr6rTAf0DNvGw64ZW+T3JDIu36CfmMHYMnnLW2DPnvj1NmzonP99VKEGqJeX\nl1lYWGDv3r3MzMywY8eOoKf4HTRY20sHKUkVaTC2hLr197iQP/XUt7O0tDxWyEOY8YWkZ8xMg8+g\nbBUOUtLYg4wsrssPXA/8DHikZ9km4E7gsejfE6PlBlwF7AYeAs70+VpR9tLNuPwPcEq37hyiRp9H\nOSjupG5VqNGXZTxFskVaNXrg3cCZq4L+i8D26P524AvR/fOB26PAPxu4z6cRkxj011zjH/AhwzPt\n8YUsTmi2Wr8Q7LajKgcplWk8RbKTWtB3novGqqDfBWyJ7m8BdkX3vwb8cb/1ht0mKeh9w/3ee4/8\nTl7hOcoOIK9AqvoRqHn8DUjxhQ7653ruW/dn4FbgnJ7H7gJm455/EoJ+nNkzWYfnOGUClRjCUI9e\n+vEN+rEHY6MXc0l/z8zmzWzFzFb2798/bjMKad++dI5ezfpMjeNcN3Vubo7FxUXq9TpmRr1eZ3Fx\nURfWHlMZztYpBeazN0Clm0Q+9CG/3vunP+3/nL6liTRKGCoTFFPVy1OSHIFLN/+Vowdjvxjdv4Cj\nB2Pv93n+qgS9b3nm0KEwr59W2URlApFy8A362NKNmX0L+AHwK2b2pJn9KfBfgHPN7DHg96OfAW4D\nHqczvfJa4JN+3yvKq3tFpiTlmenpMG0Zp+TSS2WCbGl+vATnszcIfStjj/7ee/167x/4QHZtSrPk\nkneZIO/Xz4oGr2Uc6Fw3YfzO7/gF/L59yZ973HCrSsllksKvKv9nkg8FfcrGmR7pI41wq0pA+oZf\nFXr9GviWcSjoU/DCC37h/va3j/9aafXs4sKvDOHoE36TtlMT6UdBP4Y77vAL+IcfHv01Vgduvw97\n2j27soSjT/hVJSDL8n8ixaSgH8GFF/oF/LiGnZslZHCVJRx9wq9KJY8yfMuSYlLQe3rtNb9wv/zy\n9F5zUOCGvjJSmcIxLvzKstMSCUlBH+PZZ5079tj4gN+5M/3XHhS43aAK1bOrUjiq5JEffQMpDgX9\nAE8+6deDf+WVcG1IGrhpfbCqFo4KnOxV7W+o7BT0A/z6rw8O93POyaYNST4saX+wFI4yjip9K6wC\n36CfuGvGnnACPP/80cv+4R/gnHMyefnDfK+rOuh6qPV6nWazmUFLRY4IdZ1hGY2uGTvAd78Ll14K\nf/iH8NJLnb581iEPndP5NptN2u02zWZz4Gl8B130une5zpUiWQlxnWEJb+KC/l3vgmuvhRtvhGOP\nTf/50w7d6QFnQOsuz+Ni3DK5dMK7kvKp74S+FWUe/bhCDFTRpx7avTmnmmkIGscYTtunONBgbPZC\nhG7cc5ZpbnwWxg0hzSqRMlHQ5yBE6MYFT+gefZl6b2mEtL4hSZko6DPSG4TT09N9Q2J6enqsgFwd\nttu2bTv8c61Wc+vXr1+zY9m2bVsq7211cAKuVqsVMvDTCGl9Q5IyUdBnYFAQDrqlEZD9XrPfDiaN\ncsOwk60VsZyRRkirRy9loqDPwLAgDBWQSV5z3HAadqqG0OE3SskojZBWjV7KREGfgbggDBGQSV5z\n3HJD3E5l3LGHQUE+atimFdJlGpeQyaagD2B1ANRqtb4BOKhWn0ZAZtmjjytNjfr8IQeYFdIySRT0\nCflcmWl1OK1fv96tW7duTWBt27YtSEAmbUcaIbe0tNR3hzbO82vKqEg6FPQeuuHeDZFBQba0tDSw\nl16r1fruIEIE5Op2975m6J5sms8fF+QaEBXxo6CP4TNjphtow9Yzs9h6c5FKCUVoT1yQa0BUxI+C\nPoZPrTvueq7dHn1ZQqkoAerTjiLskJIoW3ulGjIJeqAJPAw80H1BYBNwJ/BY9O+Jcc+TR9D7zF7p\nfnAHPb5x48aBA7JFLDMUqSRSpWAsyg5UJk+WQX/SqmVfBLZH97cDX4h7niL26Lsf1EHrdY92LdPA\n4bhtLVI4F6ktRdqBymTJM+h3AVui+1uAXXHPU5QafTcIVw+q5nmumTSNO23RZ3tloWg96DLt7KVa\nsgr6J4AfAjuB+WjZcz2PW+/Pq353HlgBVmZmZsJvkT58e4UhDu7Jwzht9f0GlIWi7VyL1h6ZHFkF\n/cnRv28EHgTevTrYgWfjnqcI8+jHUaQyQpxR2+o7ppFFmwe9fl496DLt7KVaMgn6o54IrgQ+V5bS\njSTjO0spbcNKRkXqQZdpZy/V4Rv0I19K0Mxeb2Zv6N4H3gc8AtwCXBKtdglw86ivIX6yuGZsv0vI\nrRbiuqELCwscOHDgqGXOOczsqGV5X87O9xrAIrnw2Rv0uwGn0ynXPAg8CixEy2vAXXSmV34P2BT3\nXOrRjy7LsoHvkcRpGlYyUg9aJh06YKp4Qny9z2sgMKtShQY6RQZT0BdMqJ531af2aaBTZDDfoB+5\nRi/J9Ks1HzhwgIWFhbGed1BdPES9PA9zc3MsLi5Sr9cxM+r1OouLi6qBiyRgnZ1CvmZnZ93Kykre\nzQhqamqKftvazGi32yM/7/LyMvPz80ftRDZu3KgwFJkAZrbTOTcbt5569BkJ1fNWj1dE4ijoM9Jv\nemJaUwLLPLUvi6mhIpNOQZ8R9bzX6padWq0WzjlarRbz8/MKe5GUqUYvuWk0GrRarTXL6/U6zWYz\n+waJlIxq9FJ4e/fuTbRcREajoJfcVH1qqEhRKOglNyEHqEXkCAW95EYD1CLZ0GCsiEhJaTBWREQA\nBb2ISOUp6EVEKk5BLyJScQp6EZGKK8SsGzPbD6w9Fn5ynQT8c96NKDhto+G0feJVYRvVnXOb41Yq\nRNDL0cxsxWfK1CTTNhpO2yfeJG0jlW5ERCpOQS8iUnEK+mJazLsBJaBtNJy2T7yJ2Uaq0YuIVJx6\n9CIiFaegz4mZbTKzO83ssejfEwesd0m0zmNmdknP8nvMbJeZPRDd3phd68Mxsz+I3tduM9ve5/EN\nZnZj9Ph9ZtboeeyKaPkuM3t/lu3O0qjbyMwaZvZSz9/MV7NuexY8ts+7zeyHZnbIzC5a9Vjfz1vp\nOed0y+EGfBHYHt3fDnyhzzqbgMejf0+M7p8YPXYPMJv3+0h5m0wDe4DTgfXAg8DWVet8EvhqdP+j\nwI3R/a3R+huA06Lnmc77PRVsGzWAR/J+DwXYPg3gHcDfABf1LB/4eSv7TT36/FwIfCO6/w3gQ33W\neT9wp3Pu5865Z4E7gT/IqH15OAvY7Zx73Dn3CnADne3Uq3e7fRt4r5lZtPwG59zLzrkngN3R81XN\nONtoEsRuH+dc0zn3ENBe9buV/bwp6PPzJufc09H9/we8qc86JwM/6fn5yWhZ19ejr+D/qSIf5Lj3\ne9Q6zrlDwPNAzfN3q2CcbQRwmpn9XzP7X2b2rtCNzcE4fweV/Rs6Ju8GVJmZfQ94c5+HFnp/cM45\nM0s6/WnOOfeUmb0B+FvgY3S+iooM8jQw45x7xszeCdxkZm9zzr2Qd8MkLPXoA3LO/b5z7u19bjcD\nPzWzLQDRvz/r8xRPAaf2/HxKtAznXPffF4H/QTXKFAPfb791zOwY4JeAZzx/twpG3kZRWesZAOfc\nTjq17H8VvMXZGufvoLJ/Qwr6/NwCdEf1LwFu7rPOHcD7zOzEaFbO+4A7zOwYMzsJwMzWAR8AHsmg\nzaH9I/CEohr+AAAA9ElEQVRWMzvNzNbTGUi8ZdU6vdvtIuD7rjOSdgvw0WjGyWnAW4H7M2p3lkbe\nRma22cymAczsdDrb6PGM2p0Vn+0zSN/PW6B2Zivv0eBJvdGpmd4FPAZ8D9gULZ8F/nvPep+gM7C4\nG/h4tOz1wE7gIeBR4MtUZIYJcD7wYzq9zYVo2eeBD0b3jwX+Z7Q97gdO7/ndhej3dgHn5f1eiraN\ngI9Efy8PAD8E/nXe7yWn7fMbdOrvv6DzbfDRnt9d83mrwk1HxoqIVJxKNyIiFaegFxGpOAW9iEjF\nKehFRCpOQS8iUnEKehGRilPQi4hUnIJeRKTi/j8XrCAk3Oqi7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11232a5f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot predicted values and actual values for the test set\n",
    "plt.scatter(X_test, y_test,  color='black')\n",
    "plt.plot(X_test, model.predict(X_test), color='blue', linewidth=3)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Exercise: Evaluate the results\n",
    "\n",
    "Compute the Residual Sum of Squares for the model: the sum of the squares of the differences between the predicted value and the actual y value for all samples.\n",
    "\n",
    "Suggested steps:\n",
    "\n",
    "* Add a column to `test` that contains the predicted disease progression for each row\n",
    "* Add another column that contains `(predicted_disease_progression - disease_progression)**2`\n",
    "* Use the pandas `sum` function to compute the sum of the previous column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Example: supervised learning (classification) with Random Forests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Background on decision trees: https://en.wikipedia.org/wiki/Decision_tree_learning\n",
    "Slides: https://docs.google.com/presentation/d/15_T-CxJcbFl2O7dIRM8eauwSWd-74hW9jGZWAQpf1iE/edit?usp=sharing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We'll use the famous [Iris](https://www.math.umd.edu/~petersd/666/html/iris_with_labels.jpg) dataset, which comes out of the box with scikit-learn. Each sample contains the sepal length, sepal width, petal length, petal width, and species of an iris."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "iris = load_iris()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# load the data into a dataframe\n",
    "df = pd.DataFrame(iris.data, columns=iris.feature_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)\n",
       "0                5.1               3.5                1.4               0.2\n",
       "1                4.9               3.0                1.4               0.2\n",
       "2                4.7               3.2                1.3               0.2\n",
       "3                4.6               3.1                1.5               0.2\n",
       "4                5.0               3.6                1.4               0.2"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['setosa', 'versicolor', 'virginica'], \n",
       "      dtype='<U10')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris.target_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>species</th>\n",
       "      <th>species_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "      <td>setosa</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n",
       "0                5.1               3.5                1.4               0.2   \n",
       "1                4.9               3.0                1.4               0.2   \n",
       "2                4.7               3.2                1.3               0.2   \n",
       "3                4.6               3.1                1.5               0.2   \n",
       "4                5.0               3.6                1.4               0.2   \n",
       "\n",
       "   species species_name  \n",
       "0        0       setosa  \n",
       "1        0       setosa  \n",
       "2        0       setosa  \n",
       "3        0       setosa  \n",
       "4        0       setosa  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add category labels to the dataframe\n",
    "df['species'] = iris.target\n",
    "# map the integers to their corresponding species name and add to dataframe\n",
    "# woah, cool feature of numpy arrays! they can accept an array as a list of indices!\n",
    "df['species_name'] = iris.target_names[iris.target]\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Problem statement\n",
    "Predict the species of an iris (setosa, versicolor, or virginica) based on its sepal length, sepal width, petal length, and petal width."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# create test and training data\n",
    "df['is_train'] = np.random.uniform(0, 1, len(df)) <= .75\n",
    "train, test = df[df['is_train'] == True], df[df['is_train'] == False]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "X_train = train[['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']]\n",
    "y_train = train['species']\n",
    "\n",
    "X_test = test[['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']]\n",
    "y_test = test['species']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Create a random forest classifier. By convention, clf means 'classifier'\n",
    "clf = RandomForestClassifier(n_jobs=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "            min_impurity_split=1e-07, min_samples_leaf=1,\n",
       "            min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "            n_estimators=10, n_jobs=2, oob_score=False, random_state=None,\n",
       "            verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train the classifier to take the training features and learn how they relate\n",
    "# to the training y (the species)\n",
    "clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 2, 1,\n",
       "       2, 2])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Apply the classifier we trained to the test data\n",
    "clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1. ,  0. ,  0. ],\n",
       "       [ 0.9,  0.1,  0. ],\n",
       "       [ 1. ,  0. ,  0. ],\n",
       "       [ 1. ,  0. ,  0. ],\n",
       "       [ 1. ,  0. ,  0. ],\n",
       "       [ 1. ,  0. ,  0. ],\n",
       "       [ 1. ,  0. ,  0. ],\n",
       "       [ 1. ,  0. ,  0. ],\n",
       "       [ 1. ,  0. ,  0. ],\n",
       "       [ 0. ,  0.8,  0.2]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How confident is the classifier that the sample belongs in each of the 3 classes?\n",
    "clf.predict_proba(X_test)[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Evaluating the results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Confusion Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>Predicted Species</th>\n",
       "      <th>setosa</th>\n",
       "      <th>versicolor</th>\n",
       "      <th>virginica</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Actual Species</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>setosa</th>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>versicolor</th>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>virginica</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Predicted Species  setosa  versicolor  virginica\n",
       "Actual Species                                  \n",
       "setosa                  9           0          0\n",
       "versicolor              0           9          0\n",
       "virginica               0           2          5"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a confusion matrix\n",
    "preds = iris.target_names[clf.predict(X_test)]\n",
    "pd.crosstab(test['species_name'], preds, rownames=['Actual Species'], colnames=['Predicted Species'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### F1 Score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "$$ F_1 = 2 * \\frac{p * r}{p + r} $$\n",
    "p = precision = (# correct positive predictions) / (total # positive predictions)\n",
    "\n",
    "r = recall = (# correct positive predictions) / (# actual positive samples)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.91733333333333333"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import f1_score\n",
    "\n",
    "f1_score(test['species_name'], preds, average='weighted')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## View feature importance\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('sepal length (cm)', 0.18678130631918352),\n",
       " ('sepal width (cm)', 0.052994514684094787),\n",
       " ('petal length (cm)', 0.43293090419725494),\n",
       " ('petal width (cm)', 0.32729327479946679)]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(zip(iris.feature_names, clf.feature_importances_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Random Forest Hyperparameters\n",
    "* `n_estimators`: the number of trees in the forest (defaults to 10)\n",
    "* `max_features`: the max number of features to use in each individual decision tree (defaults to sqrt(num features))\n",
    "* `max_depth`: maximum depth of the decision tree\n",
    "* `min_samples_split`: minimum number of samples required to split an internal node (defaults to 2)\n",
    "* `min_samples_leaf`: minimum number of samples required to be at a leaf node"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Exercise: experiment with the `n_estimators` and `max_features` parameters to try to improve the F1 score. Remember to compute F1 on the test data only!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "n_estimators = [1, 10, 100]\n",
    "max_features = [1, 2, 3, 4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1 Score with 1 features and 1 estimators is: 0.9173333333333333\n",
      "F1 Score with 2 features and 1 estimators is: 0.8722077922077922\n",
      "F1 Score with 3 features and 1 estimators is: 0.8722077922077922\n",
      "F1 Score with 4 features and 1 estimators is: 0.9601568627450979\n",
      "F1 Score with 1 features and 10 estimators is: 0.9173333333333333\n",
      "F1 Score with 2 features and 10 estimators is: 0.9173333333333333\n",
      "F1 Score with 3 features and 10 estimators is: 0.8722077922077922\n",
      "F1 Score with 4 features and 10 estimators is: 0.8722077922077922\n",
      "F1 Score with 1 features and 100 estimators is: 0.9595141700404858\n",
      "F1 Score with 2 features and 100 estimators is: 0.9173333333333333\n",
      "F1 Score with 3 features and 100 estimators is: 0.8722077922077922\n",
      "F1 Score with 4 features and 100 estimators is: 0.8722077922077922\n"
     ]
    }
   ],
   "source": [
    "for est in n_estimators:\n",
    "    for feat in max_features:\n",
    "        clf = RandomForestClassifier(n_jobs=2, n_estimators=est, max_features=feat)\n",
    "        clf.fit(X_train, y_train)\n",
    "        preds = iris.target_names[clf.predict(X_test)]\n",
    "        f1 = f1_score(test['species_name'], preds, average='weighted')\n",
    "        print(\"F1 Score with {feat} features and {est} estimators is: {f1}\".format(feat=feat, est=est, f1=f1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Remember our 311 data set?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "complaints = pd.read_csv('../pandas-cookbook/data/311-service-requests.csv', low_memory=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unique Key</th>\n",
       "      <th>Created Date</th>\n",
       "      <th>Closed Date</th>\n",
       "      <th>Agency</th>\n",
       "      <th>Agency Name</th>\n",
       "      <th>Complaint Type</th>\n",
       "      <th>Descriptor</th>\n",
       "      <th>Location Type</th>\n",
       "      <th>Incident Zip</th>\n",
       "      <th>Incident Address</th>\n",
       "      <th>...</th>\n",
       "      <th>Bridge Highway Name</th>\n",
       "      <th>Bridge Highway Direction</th>\n",
       "      <th>Road Ramp</th>\n",
       "      <th>Bridge Highway Segment</th>\n",
       "      <th>Garage Lot Name</th>\n",
       "      <th>Ferry Direction</th>\n",
       "      <th>Ferry Terminal Name</th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "      <th>Location</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>26589651</td>\n",
       "      <td>10/31/2013 02:08:41 AM</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Noise - Street/Sidewalk</td>\n",
       "      <td>Loud Talking</td>\n",
       "      <td>Street/Sidewalk</td>\n",
       "      <td>11432</td>\n",
       "      <td>90-03 169 STREET</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.708275</td>\n",
       "      <td>-73.791604</td>\n",
       "      <td>(40.70827532593202, -73.79160395779721)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>26593698</td>\n",
       "      <td>10/31/2013 02:01:04 AM</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Illegal Parking</td>\n",
       "      <td>Commercial Overnight Parking</td>\n",
       "      <td>Street/Sidewalk</td>\n",
       "      <td>11378</td>\n",
       "      <td>58 AVENUE</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.721041</td>\n",
       "      <td>-73.909453</td>\n",
       "      <td>(40.721040535628305, -73.90945306791765)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>26594139</td>\n",
       "      <td>10/31/2013 02:00:24 AM</td>\n",
       "      <td>10/31/2013 02:40:32 AM</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Noise - Commercial</td>\n",
       "      <td>Loud Music/Party</td>\n",
       "      <td>Club/Bar/Restaurant</td>\n",
       "      <td>10032</td>\n",
       "      <td>4060 BROADWAY</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.843330</td>\n",
       "      <td>-73.939144</td>\n",
       "      <td>(40.84332975466513, -73.93914371913482)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>26595721</td>\n",
       "      <td>10/31/2013 01:56:23 AM</td>\n",
       "      <td>10/31/2013 02:21:48 AM</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Noise - Vehicle</td>\n",
       "      <td>Car/Truck Horn</td>\n",
       "      <td>Street/Sidewalk</td>\n",
       "      <td>10023</td>\n",
       "      <td>WEST 72 STREET</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.778009</td>\n",
       "      <td>-73.980213</td>\n",
       "      <td>(40.7780087446372, -73.98021349023975)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>26590930</td>\n",
       "      <td>10/31/2013 01:53:44 AM</td>\n",
       "      <td>NaN</td>\n",
       "      <td>DOHMH</td>\n",
       "      <td>Department of Health and Mental Hygiene</td>\n",
       "      <td>Rodent</td>\n",
       "      <td>Condition Attracting Rodents</td>\n",
       "      <td>Vacant Lot</td>\n",
       "      <td>10027</td>\n",
       "      <td>WEST 124 STREET</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.807691</td>\n",
       "      <td>-73.947387</td>\n",
       "      <td>(40.80769092704951, -73.94738703491433)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 52 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unique Key            Created Date             Closed Date Agency  \\\n",
       "0    26589651  10/31/2013 02:08:41 AM                     NaN   NYPD   \n",
       "1    26593698  10/31/2013 02:01:04 AM                     NaN   NYPD   \n",
       "2    26594139  10/31/2013 02:00:24 AM  10/31/2013 02:40:32 AM   NYPD   \n",
       "3    26595721  10/31/2013 01:56:23 AM  10/31/2013 02:21:48 AM   NYPD   \n",
       "4    26590930  10/31/2013 01:53:44 AM                     NaN  DOHMH   \n",
       "\n",
       "                               Agency Name           Complaint Type  \\\n",
       "0          New York City Police Department  Noise - Street/Sidewalk   \n",
       "1          New York City Police Department          Illegal Parking   \n",
       "2          New York City Police Department       Noise - Commercial   \n",
       "3          New York City Police Department          Noise - Vehicle   \n",
       "4  Department of Health and Mental Hygiene                   Rodent   \n",
       "\n",
       "                     Descriptor        Location Type Incident Zip  \\\n",
       "0                  Loud Talking      Street/Sidewalk        11432   \n",
       "1  Commercial Overnight Parking      Street/Sidewalk        11378   \n",
       "2              Loud Music/Party  Club/Bar/Restaurant        10032   \n",
       "3                Car/Truck Horn      Street/Sidewalk        10023   \n",
       "4  Condition Attracting Rodents           Vacant Lot        10027   \n",
       "\n",
       "   Incident Address                    ...                     \\\n",
       "0  90-03 169 STREET                    ...                      \n",
       "1         58 AVENUE                    ...                      \n",
       "2     4060 BROADWAY                    ...                      \n",
       "3    WEST 72 STREET                    ...                      \n",
       "4   WEST 124 STREET                    ...                      \n",
       "\n",
       "  Bridge Highway Name Bridge Highway Direction Road Ramp  \\\n",
       "0                 NaN                      NaN       NaN   \n",
       "1                 NaN                      NaN       NaN   \n",
       "2                 NaN                      NaN       NaN   \n",
       "3                 NaN                      NaN       NaN   \n",
       "4                 NaN                      NaN       NaN   \n",
       "\n",
       "  Bridge Highway Segment Garage Lot Name Ferry Direction Ferry Terminal Name  \\\n",
       "0                    NaN             NaN             NaN                 NaN   \n",
       "1                    NaN             NaN             NaN                 NaN   \n",
       "2                    NaN             NaN             NaN                 NaN   \n",
       "3                    NaN             NaN             NaN                 NaN   \n",
       "4                    NaN             NaN             NaN                 NaN   \n",
       "\n",
       "    Latitude  Longitude                                  Location  \n",
       "0  40.708275 -73.791604   (40.70827532593202, -73.79160395779721)  \n",
       "1  40.721041 -73.909453  (40.721040535628305, -73.90945306791765)  \n",
       "2  40.843330 -73.939144   (40.84332975466513, -73.93914371913482)  \n",
       "3  40.778009 -73.980213    (40.7780087446372, -73.98021349023975)  \n",
       "4  40.807691 -73.947387   (40.80769092704951, -73.94738703491433)  \n",
       "\n",
       "[5 rows x 52 columns]"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "complaints.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['unique_key', 'created_date', 'closed_date', 'agency', 'agency_name',\n",
       "       'complaint_type', 'descriptor', 'location_type', 'incident_zip',\n",
       "       'incident_address', 'street_name', 'cross_street_1', 'cross_street_2',\n",
       "       'intersection_street_1', 'intersection_street_2', 'address_type',\n",
       "       'city', 'landmark', 'facility_type', 'status', 'due_date',\n",
       "       'resolution_action_updated_date', 'community_board', 'borough',\n",
       "       'x_coordinate_(state_plane)', 'y_coordinate_(state_plane)',\n",
       "       'park_facility_name', 'park_borough', 'school_name', 'school_number',\n",
       "       'school_region', 'school_code', 'school_phone_number', 'school_address',\n",
       "       'school_city', 'school_state', 'school_zip', 'school_not_found',\n",
       "       'school_or_citywide_complaint', 'vehicle_type', 'taxi_company_borough',\n",
       "       'taxi_pick_up_location', 'bridge_highway_name',\n",
       "       'bridge_highway_direction', 'road_ramp', 'bridge_highway_segment',\n",
       "       'garage_lot_name', 'ferry_direction', 'ferry_terminal_name', 'latitude',\n",
       "       'longitude', 'location'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "complaints.columns = [x.lower().replace(' ','_') for x in complaints.columns]\n",
    "complaints.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "useful_columns = ['created_date', 'closed_date','due_date', 'agency_name', 'complaint_type', \n",
    "                  'borough', 'latitude', 'longitude']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "complaints = complaints[useful_columns]\n",
    "complaints.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Project: predict whether a complaint will be resolved on time (closed date is before due date) using a random forest classifier with these features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suggested steps:\n",
    "* Add a \"on time\" column to the dataframe with a boolean value 0 or 1 if the complaint was resolved on time or not\n",
    "* Partition the data into a train set and a test set\n",
    "* Experiment with a random forest classifier with different hyperparameters to try to achieve the best F1 score "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}