Machine Learning on Quantopian.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploring Machine Learning on Quantopian\n",
    "\n",
    "Recently, Quantopian’s Chief Investment Officer, Jonathan Larkin, shared an industry insider’s overview of the [professional quant equity workflow][1]. This workflow is comprised of distinct stages including: (1) Universe Definition, (2) Alpha Discovery, (3) Alpha Combination, (4) Portfolio Construction and (5) Trading.\n",
    "\n",
    "This Notebook focuses on stage 3: Alpha Combination. At this stage, Machine Learning is an intuitive choice as we have abstracted the problem to such a degree that it is now a classic classification (or regression) problem which ML is very good at solving and coming up with an alpha combination that is predictive.\n",
    "\n",
    "As you will see, there is a lot of code here setting up a factor library and some data wrangling to get everything into shape. The details of this part are perhaps not quite as interesting so feel free to skip directly to [\"Training our ML pipeline\"](#training) where we have everything in place to train and test our classifier.\n",
    "\n",
    "## Overview\n",
    "1. Define trading universe to use ([Q500US and Q1500US][2]).\n",
    "2. Define alphas (implemented in [Pipeline][3]).\n",
    "3. Run pipeline.\n",
    "4. Split into train and test set.\n",
    "5. Preprocess data (rank alphas, subsample, align alphas with future returns, impute, scale).\n",
    "6. Train Machine Learning classifier ([AdaBoost from Scikit-Learn][4]).\n",
    "7. Evaluate Machine Learning classifier on test set.\n",
    "\n",
    "Note that one important limitation is that we only train and test on static (i.e. fixed-in-time) data. Thus, you can not directly do the same in an algorithm. In principle, this is possible and will be the next step but it makes sense to first focus on just the ML in a more direct way to get a good intuition about the workflow and how to develop a competitive ML pipeline.\n",
    "\n",
    "### Disclaimer\n",
    "\n",
    "This workflow is still a bit rough around the edges. We are working on improving it and adding better educational materials. This serves as a sneak-peek for the curious and adventurous.\n",
    "\n",
    "  [1]: http://blog.quantopian.com/a-professional-quant-equity-workflow/\n",
    "  [2]: https://www.quantopian.com/posts/the-q500us-and-q1500us\n",
    "  [3]: https://www.quantopian.com/tutorials/pipeline\n",
    "  [4]: http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html\n",
    "  [5]: https://www.quantopian.com/posts/alphalens-a-new-tool-for-analyzing-alpha-factors\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from quantopian.research import run_pipeline\n",
    "from quantopian.pipeline import Pipeline\n",
    "from quantopian.pipeline.factors import Latest\n",
    "from quantopian.pipeline.data.builtin import USEquityPricing\n",
    "from quantopian.pipeline.data import morningstar\n",
    "from quantopian.pipeline.factors import CustomFactor, SimpleMovingAverage, AverageDollarVolume, Returns, RSI\n",
    "from quantopian.pipeline.classifiers.morningstar import Sector\n",
    "from quantopian.pipeline.filters import Q500US, Q1500US\n",
    "from quantopian.pipeline.data.quandl import fred_usdontd156n as libor\n",
    "from quantopian.pipeline.data.zacks import EarningsSurprises\n",
    "\n",
    "import talib\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from time import time\n",
    "\n",
    "import alphalens as al\n",
    "import pyfolio as pf\n",
    "from scipy import stats\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import linear_model, decomposition, ensemble, preprocessing, isotonic, metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Definition of some commonly used factors\n",
    "\n",
    "The factors below are a small collection of commonly used alphas that were coded by Gil Wassermann. I will post a separate Notebook with the full collection and more descriptions of them. Ultimately we will put these into a library you can just import to avoid the wall of text. If you want to understand more about pipeline, read the [tutorial](https://www.quantopian.com/tutorials/pipeline).\n",
    "\n",
    "Also note the `Earnings_Quality` alpha which uses [Zacks Earnings Surprises](https://www.quantopian.com/data/zacks/earnings_surprises), a [new source from our partners](https://www.quantopian.com/data).\n",
    "\n",
    "The details of these factors are not the focus of this Notebook so feel free to just [skip](#universe) this cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bs = morningstar.balance_sheet\n",
    "cfs = morningstar.cash_flow_statement\n",
    "is_ = morningstar.income_statement\n",
    "or_ = morningstar.operation_ratios\n",
    "er = morningstar.earnings_report\n",
    "v = morningstar.valuation\n",
    "vr = morningstar.valuation_ratios\n",
    "\n",
    "\n",
    "def make_factors():\n",
    "    def Asset_Growth_3M():\n",
    "        return Returns(inputs=[bs.total_assets], window_length=63)\n",
    "\n",
    "    def Asset_To_Equity_Ratio():\n",
    "        return bs.total_assets.latest / bs.common_stock_equity.latest\n",
    "\n",
    "    def Capex_To_Cashflows():\n",
    "        return (cfs.capital_expenditure.latest * 4.) / \\\n",
    "            (cfs.free_cash_flow.latest * 4.)\n",
    "        \n",
    "    def EBITDA_Yield():\n",
    "        return (is_.ebitda.latest * 4.) / \\\n",
    "            USEquityPricing.close.latest        \n",
    "\n",
    "    def EBIT_To_Assets():\n",
    "        return (is_.ebit.latest * 4.) / \\\n",
    "            bs.total_assets.latest\n",
    "        \n",
    "    def Earnings_Quality():\n",
    "        return morningstar.cash_flow_statement.operating_cash_flow.latest / \\\n",
    "               EarningsSurprises.eps_act.latest\n",
    "        \n",
    "    def Return_On_Total_Invest_Capital():\n",
    "        return or_.roic.latest\n",
    "    \n",
    "    class Mean_Reversion_1M(CustomFactor):\n",
    "        inputs = [Returns(window_length=21)]\n",
    "        window_length = 252\n",
    "\n",
    "        def compute(self, today, assets, out, monthly_rets):\n",
    "            out[:] = (monthly_rets[-1] - np.nanmean(monthly_rets, axis=0)) / \\\n",
    "                np.nanstd(monthly_rets, axis=0)\n",
    "                \n",
    "    class MACD_Signal_10d(CustomFactor):\n",
    "        inputs = [USEquityPricing.close]\n",
    "        window_length = 60\n",
    "\n",
    "        def compute(self, today, assets, out, close):\n",
    "\n",
    "            sig_lines = []\n",
    "\n",
    "            for col in close.T:\n",
    "                # get signal line only\n",
    "                try:\n",
    "                    _, signal_line, _ = talib.MACD(col, fastperiod=12,\n",
    "                                                   slowperiod=26, signalperiod=10)\n",
    "                    sig_lines.append(signal_line[-1])\n",
    "                # if error calculating, return NaN\n",
    "                except:\n",
    "                    sig_lines.append(np.nan)\n",
    "            out[:] = sig_lines \n",
    "            \n",
    "    class Moneyflow_Volume_5d(CustomFactor):\n",
    "        inputs = [USEquityPricing.close, USEquityPricing.volume]\n",
    "        window_length = 5\n",
    "\n",
    "        def compute(self, today, assets, out, close, volume):\n",
    "\n",
    "            mfvs = []\n",
    "\n",
    "            for col_c, col_v in zip(close.T, volume.T):\n",
    "\n",
    "                # denominator\n",
    "                denominator = np.dot(col_c, col_v)\n",
    "\n",
    "                # numerator\n",
    "                numerator = 0.\n",
    "                for n, price in enumerate(col_c.tolist()):\n",
    "                    if price > col_c[n - 1]:\n",
    "                        numerator += price * col_v[n]\n",
    "                    else:\n",
    "                        numerator -= price * col_v[n]\n",
    "\n",
    "                mfvs.append(numerator / denominator)\n",
    "            out[:] = mfvs  \n",
    "            \n",
    "           \n",
    "    def Net_Income_Margin():\n",
    "        return or_.net_margin.latest           \n",
    "\n",
    "    def Operating_Cashflows_To_Assets():\n",
    "        return (cfs.operating_cash_flow.latest * 4.) / \\\n",
    "            bs.total_assets.latest\n",
    "\n",
    "    def Price_Momentum_3M():\n",
    "        return Returns(window_length=63)\n",
    "    \n",
    "    class Price_Oscillator(CustomFactor):\n",
    "        inputs = [USEquityPricing.close]\n",
    "        window_length = 252\n",
    "\n",
    "        def compute(self, today, assets, out, close):\n",
    "            four_week_period = close[-20:]\n",
    "            out[:] = (np.nanmean(four_week_period, axis=0) /\n",
    "                      np.nanmean(close, axis=0)) - 1.\n",
    "    \n",
    "    def Returns_39W():\n",
    "        return Returns(window_length=215)\n",
    "    \n",
    "    class Trendline(CustomFactor):\n",
    "        inputs = [USEquityPricing.close]\n",
    "        window_length = 252\n",
    "\n",
    "        # using MLE for speed\n",
    "        def compute(self, today, assets, out, close):\n",
    "\n",
    "            # prepare X matrix (x_is - x_bar)\n",
    "            X = range(self.window_length)\n",
    "            X_bar = np.nanmean(X)\n",
    "            X_vector = X - X_bar\n",
    "            X_matrix = np.tile(X_vector, (len(close.T), 1)).T\n",
    "\n",
    "            # prepare Y matrix (y_is - y_bar)\n",
    "            Y_bar = np.nanmean(close, axis=0)\n",
    "            Y_bars = np.tile(Y_bar, (self.window_length, 1))\n",
    "            Y_matrix = close - Y_bars\n",
    "\n",
    "            # prepare variance of X\n",
    "            X_var = np.nanvar(X)\n",
    "\n",
    "            # multiply X matrix an Y matrix and sum (dot product)\n",
    "            # then divide by variance of X\n",
    "            # this gives the MLE of Beta\n",
    "            out[:] = (np.sum((X_matrix * Y_matrix), axis=0) / X_var) / \\\n",
    "                (self.window_length)\n",
    "        \n",
    "    class Vol_3M(CustomFactor):\n",
    "        inputs = [Returns(window_length=2)]\n",
    "        window_length = 63\n",
    "\n",
    "        def compute(self, today, assets, out, rets):\n",
    "            out[:] = np.nanstd(rets, axis=0)\n",
    "            \n",
    "    def Working_Capital_To_Assets():\n",
    "        return bs.working_capital.latest / bs.total_assets.latest\n",
    "        \n",
    "    all_factors = {\n",
    "        'Asset Growth 3M': Asset_Growth_3M,\n",
    "        'Asset to Equity Ratio': Asset_To_Equity_Ratio,\n",
    "        'Capex to Cashflows': Capex_To_Cashflows,\n",
    "        'EBIT to Assets': EBIT_To_Assets,\n",
    "        'EBITDA Yield': EBITDA_Yield,        \n",
    "        'Earnings Quality': Earnings_Quality,\n",
    "        'MACD Signal Line': MACD_Signal_10d,\n",
    "        'Mean Reversion 1M': Mean_Reversion_1M,\n",
    "        'Moneyflow Volume 5D': Moneyflow_Volume_5d,\n",
    "        'Net Income Margin': Net_Income_Margin,        \n",
    "        'Operating Cashflows to Assets': Operating_Cashflows_To_Assets,\n",
    "        'Price Momentum 3M': Price_Momentum_3M,\n",
    "        'Price Oscillator': Price_Oscillator,\n",
    "        'Return on Invest Capital': Return_On_Total_Invest_Capital,\n",
    "        '39 Week Returns': Returns_39W,\n",
    "        'Trendline': Trendline,\n",
    "        'Vol 3M': Vol_3M,\n",
    "        'Working Capital to Assets': Working_Capital_To_Assets,        \n",
    "    }        \n",
    "    \n",
    "    return all_factors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='universe'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define universe and select factors to use\n",
    "\n",
    "We will screen our universe using the new [Q1500US](https://www.quantopian.com/posts/the-q500us-and-q1500us) and hand-pick a few alphas from the list above. We encourage you to play around with the factors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "universe = Q1500US()\n",
    "\n",
    "factors = make_factors()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Define and build the pipeline\n",
    "\n",
    "Next we have to build the pipeline. In addition to the factors defined above, we need the forward returns we want to predict. In this Notebook we will predict 5-day returns and train our model on daily data. You can also subsample the data to e.g. weekly to not have overlapping return periods but we omit this here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_fwd_days = 5 # number of days to compute returns over"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def make_history_pipeline(factors, universe, n_fwd_days=5):\n",
    "    # Call .rank() on all factors and mask out the universe\n",
    "    factor_ranks = {name: f().rank(mask=universe) for name, f in factors.iteritems()}\n",
    "    # Get cumulative returns over last n_fwd_days days. We will later shift these.\n",
    "    factor_ranks['Returns'] = Returns(inputs=[USEquityPricing.open],\n",
    "                                      mask=universe, window_length=n_fwd_days)\n",
    "    \n",
    "    pipe = Pipeline(screen=universe, columns=factor_ranks)\n",
    "    \n",
    "    return pipe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "history_pipe = make_history_pipeline(factors, universe, n_fwd_days=n_fwd_days)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Run the pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "start_timer = time()\n",
    "start = pd.Timestamp(\"2016-03-06\")\n",
    "end = pd.Timestamp(\"2016-09-14\")\n",
    "results = run_pipeline(history_pipe, start_date=start, end_date=end)\n",
    "results.index.names = ['date', 'security']\n",
    "end_timer = time()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time to run pipeline 201.66 secs\n"
     ]
    }
   ],
   "source": [
    "print \"Time to run pipeline %.2f secs\" % (end_timer - start_timer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>39 Week Returns</th>\n",
       "      <th>Asset Growth 3M</th>\n",
       "      <th>Asset to Equity Ratio</th>\n",
       "      <th>Capex to Cashflows</th>\n",
       "      <th>EBIT to Assets</th>\n",
       "      <th>EBITDA Yield</th>\n",
       "      <th>Earnings Quality</th>\n",
       "      <th>MACD Signal Line</th>\n",
       "      <th>Mean Reversion 1M</th>\n",
       "      <th>Moneyflow Volume 5D</th>\n",
       "      <th>Net Income Margin</th>\n",
       "      <th>Operating Cashflows to Assets</th>\n",
       "      <th>Price Momentum 3M</th>\n",
       "      <th>Price Oscillator</th>\n",
       "      <th>Return on Invest Capital</th>\n",
       "      <th>Returns</th>\n",
       "      <th>Trendline</th>\n",
       "      <th>Vol 3M</th>\n",
       "      <th>Working Capital to Assets</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th>security</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2016-03-07 00:00:00+00:00</th>\n",
       "      <th>Equity(2 [AA])</th>\n",
       "      <td>326.0</td>\n",
       "      <td>861.0</td>\n",
       "      <td>921.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>213.0</td>\n",
       "      <td>75.0</td>\n",
       "      <td>1466.0</td>\n",
       "      <td>891.0</td>\n",
       "      <td>1050.0</td>\n",
       "      <td>939.0</td>\n",
       "      <td>231.0</td>\n",
       "      <td>783.0</td>\n",
       "      <td>1181.0</td>\n",
       "      <td>540.0</td>\n",
       "      <td>220.0</td>\n",
       "      <td>0.071508</td>\n",
       "      <td>721.0</td>\n",
       "      <td>1271.0</td>\n",
       "      <td>556.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Equity(24 [AAPL])</th>\n",
       "      <td>521.0</td>\n",
       "      <td>936.0</td>\n",
       "      <td>634.0</td>\n",
       "      <td>696.0</td>\n",
       "      <td>1370.0</td>\n",
       "      <td>1492.0</td>\n",
       "      <td>1454.0</td>\n",
       "      <td>307.0</td>\n",
       "      <td>1025.0</td>\n",
       "      <td>1385.0</td>\n",
       "      <td>1340.0</td>\n",
       "      <td>1448.0</td>\n",
       "      <td>472.0</td>\n",
       "      <td>583.0</td>\n",
       "      <td>1458.0</td>\n",
       "      <td>0.056886</td>\n",
       "      <td>153.0</td>\n",
       "      <td>520.0</td>\n",
       "      <td>272.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Equity(53 [ABMD])</th>\n",
       "      <td>1418.0</td>\n",
       "      <td>1266.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>630.0</td>\n",
       "      <td>1227.0</td>\n",
       "      <td>259.0</td>\n",
       "      <td>581.0</td>\n",
       "      <td>265.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>710.0</td>\n",
       "      <td>1065.0</td>\n",
       "      <td>1363.0</td>\n",
       "      <td>1166.0</td>\n",
       "      <td>1148.0</td>\n",
       "      <td>1112.0</td>\n",
       "      <td>0.028731</td>\n",
       "      <td>1421.0</td>\n",
       "      <td>1210.0</td>\n",
       "      <td>1275.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Equity(62 [ABT])</th>\n",
       "      <td>580.0</td>\n",
       "      <td>505.0</td>\n",
       "      <td>482.0</td>\n",
       "      <td>402.0</td>\n",
       "      <td>815.0</td>\n",
       "      <td>1392.0</td>\n",
       "      <td>1333.0</td>\n",
       "      <td>348.0</td>\n",
       "      <td>540.0</td>\n",
       "      <td>460.0</td>\n",
       "      <td>1146.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>451.0</td>\n",
       "      <td>645.0</td>\n",
       "      <td>993.0</td>\n",
       "      <td>-0.012456</td>\n",
       "      <td>602.0</td>\n",
       "      <td>501.0</td>\n",
       "      <td>677.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Equity(67 [ADSK])</th>\n",
       "      <td>826.0</td>\n",
       "      <td>1288.0</td>\n",
       "      <td>1014.0</td>\n",
       "      <td>624.0</td>\n",
       "      <td>283.0</td>\n",
       "      <td>317.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>557.0</td>\n",
       "      <td>914.0</td>\n",
       "      <td>1084.0</td>\n",
       "      <td>269.0</td>\n",
       "      <td>938.0</td>\n",
       "      <td>395.0</td>\n",
       "      <td>752.0</td>\n",
       "      <td>270.0</td>\n",
       "      <td>0.083758</td>\n",
       "      <td>727.0</td>\n",
       "      <td>1003.0</td>\n",
       "      <td>953.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             39 Week Returns  Asset Growth 3M  \\\n",
       "date                      security                                              \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])               326.0            861.0   \n",
       "                          Equity(24 [AAPL])            521.0            936.0   \n",
       "                          Equity(53 [ABMD])           1418.0           1266.0   \n",
       "                          Equity(62 [ABT])             580.0            505.0   \n",
       "                          Equity(67 [ADSK])            826.0           1288.0   \n",
       "\n",
       "                                             Asset to Equity Ratio  \\\n",
       "date                      security                                   \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                     921.0   \n",
       "                          Equity(24 [AAPL])                  634.0   \n",
       "                          Equity(53 [ABMD])                  106.0   \n",
       "                          Equity(62 [ABT])                   482.0   \n",
       "                          Equity(67 [ADSK])                 1014.0   \n",
       "\n",
       "                                             Capex to Cashflows  \\\n",
       "date                      security                                \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                  242.0   \n",
       "                          Equity(24 [AAPL])               696.0   \n",
       "                          Equity(53 [ABMD])               630.0   \n",
       "                          Equity(62 [ABT])                402.0   \n",
       "                          Equity(67 [ADSK])               624.0   \n",
       "\n",
       "                                             EBIT to Assets  EBITDA Yield  \\\n",
       "date                      security                                          \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])              213.0          75.0   \n",
       "                          Equity(24 [AAPL])          1370.0        1492.0   \n",
       "                          Equity(53 [ABMD])          1227.0         259.0   \n",
       "                          Equity(62 [ABT])            815.0        1392.0   \n",
       "                          Equity(67 [ADSK])           283.0         317.0   \n",
       "\n",
       "                                             Earnings Quality  \\\n",
       "date                      security                              \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])               1466.0   \n",
       "                          Equity(24 [AAPL])            1454.0   \n",
       "                          Equity(53 [ABMD])             581.0   \n",
       "                          Equity(62 [ABT])             1333.0   \n",
       "                          Equity(67 [ADSK])              20.0   \n",
       "\n",
       "                                             MACD Signal Line  \\\n",
       "date                      security                              \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                891.0   \n",
       "                          Equity(24 [AAPL])             307.0   \n",
       "                          Equity(53 [ABMD])             265.0   \n",
       "                          Equity(62 [ABT])              348.0   \n",
       "                          Equity(67 [ADSK])             557.0   \n",
       "\n",
       "                                             Mean Reversion 1M  \\\n",
       "date                      security                               \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                1050.0   \n",
       "                          Equity(24 [AAPL])             1025.0   \n",
       "                          Equity(53 [ABMD])               68.0   \n",
       "                          Equity(62 [ABT])               540.0   \n",
       "                          Equity(67 [ADSK])              914.0   \n",
       "\n",
       "                                             Moneyflow Volume 5D  \\\n",
       "date                      security                                 \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                   939.0   \n",
       "                          Equity(24 [AAPL])               1385.0   \n",
       "                          Equity(53 [ABMD])                710.0   \n",
       "                          Equity(62 [ABT])                 460.0   \n",
       "                          Equity(67 [ADSK])               1084.0   \n",
       "\n",
       "                                             Net Income Margin  \\\n",
       "date                      security                               \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                 231.0   \n",
       "                          Equity(24 [AAPL])             1340.0   \n",
       "                          Equity(53 [ABMD])             1065.0   \n",
       "                          Equity(62 [ABT])              1146.0   \n",
       "                          Equity(67 [ADSK])              269.0   \n",
       "\n",
       "                                             Operating Cashflows to Assets  \\\n",
       "date                      security                                           \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                             783.0   \n",
       "                          Equity(24 [AAPL])                         1448.0   \n",
       "                          Equity(53 [ABMD])                         1363.0   \n",
       "                          Equity(62 [ABT])                           676.0   \n",
       "                          Equity(67 [ADSK])                          938.0   \n",
       "\n",
       "                                             Price Momentum 3M  \\\n",
       "date                      security                               \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                1181.0   \n",
       "                          Equity(24 [AAPL])              472.0   \n",
       "                          Equity(53 [ABMD])             1166.0   \n",
       "                          Equity(62 [ABT])               451.0   \n",
       "                          Equity(67 [ADSK])              395.0   \n",
       "\n",
       "                                             Price Oscillator  \\\n",
       "date                      security                              \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                540.0   \n",
       "                          Equity(24 [AAPL])             583.0   \n",
       "                          Equity(53 [ABMD])            1148.0   \n",
       "                          Equity(62 [ABT])              645.0   \n",
       "                          Equity(67 [ADSK])             752.0   \n",
       "\n",
       "                                             Return on Invest Capital  \\\n",
       "date                      security                                      \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                        220.0   \n",
       "                          Equity(24 [AAPL])                    1458.0   \n",
       "                          Equity(53 [ABMD])                    1112.0   \n",
       "                          Equity(62 [ABT])                      993.0   \n",
       "                          Equity(67 [ADSK])                     270.0   \n",
       "\n",
       "                                              Returns  Trendline  Vol 3M  \\\n",
       "date                      security                                         \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])     0.071508      721.0  1271.0   \n",
       "                          Equity(24 [AAPL])  0.056886      153.0   520.0   \n",
       "                          Equity(53 [ABMD])  0.028731     1421.0  1210.0   \n",
       "                          Equity(62 [ABT])  -0.012456      602.0   501.0   \n",
       "                          Equity(67 [ADSK])  0.083758      727.0  1003.0   \n",
       "\n",
       "                                             Working Capital to Assets  \n",
       "date                      security                                      \n",
       "2016-03-07 00:00:00+00:00 Equity(2 [AA])                         556.0  \n",
       "                          Equity(24 [AAPL])                      272.0  \n",
       "                          Equity(53 [ABMD])                     1275.0  \n",
       "                          Equity(62 [ABT])                       677.0  \n",
       "                          Equity(67 [ADSK])                      953.0  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>39 Week Returns</th>\n",
       "      <th>Asset Growth 3M</th>\n",
       "      <th>Asset to Equity Ratio</th>\n",
       "      <th>Capex to Cashflows</th>\n",
       "      <th>EBIT to Assets</th>\n",
       "      <th>EBITDA Yield</th>\n",
       "      <th>Earnings Quality</th>\n",
       "      <th>MACD Signal Line</th>\n",
       "      <th>Mean Reversion 1M</th>\n",
       "      <th>Moneyflow Volume 5D</th>\n",
       "      <th>Net Income Margin</th>\n",
       "      <th>Operating Cashflows to Assets</th>\n",
       "      <th>Price Momentum 3M</th>\n",
       "      <th>Price Oscillator</th>\n",
       "      <th>Return on Invest Capital</th>\n",
       "      <th>Returns</th>\n",
       "      <th>Trendline</th>\n",
       "      <th>Vol 3M</th>\n",
       "      <th>Working Capital to Assets</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th>security</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2016-09-14 00:00:00+00:00</th>\n",
       "      <th>Equity(49496 [FDC])</th>\n",
       "      <td>300.0</td>\n",
       "      <td>903.0</td>\n",
       "      <td>1478.0</td>\n",
       "      <td>574.0</td>\n",
       "      <td>545.0</td>\n",
       "      <td>1441.0</td>\n",
       "      <td>1400.0</td>\n",
       "      <td>954.0</td>\n",
       "      <td>1349.0</td>\n",
       "      <td>898.0</td>\n",
       "      <td>628.0</td>\n",
       "      <td>594.0</td>\n",
       "      <td>1290.0</td>\n",
       "      <td>433.0</td>\n",
       "      <td>429.0</td>\n",
       "      <td>-0.025424</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1175.0</td>\n",
       "      <td>350.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Equity(49506 [HPE])</th>\n",
       "      <td>1454.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>683.0</td>\n",
       "      <td>233.0</td>\n",
       "      <td>1031.0</td>\n",
       "      <td>1485.0</td>\n",
       "      <td>1443.0</td>\n",
       "      <td>862.0</td>\n",
       "      <td>609.0</td>\n",
       "      <td>373.0</td>\n",
       "      <td>1233.0</td>\n",
       "      <td>802.0</td>\n",
       "      <td>1181.0</td>\n",
       "      <td>1402.0</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>-0.041345</td>\n",
       "      <td>NaN</td>\n",
       "      <td>879.0</td>\n",
       "      <td>669.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Equity(49630 [CSRA])</th>\n",
       "      <td>NaN</td>\n",
       "      <td>352.0</td>\n",
       "      <td>1475.0</td>\n",
       "      <td>584.0</td>\n",
       "      <td>990.0</td>\n",
       "      <td>1149.0</td>\n",
       "      <td>1049.0</td>\n",
       "      <td>842.0</td>\n",
       "      <td>1290.0</td>\n",
       "      <td>1113.0</td>\n",
       "      <td>626.0</td>\n",
       "      <td>1073.0</td>\n",
       "      <td>1225.0</td>\n",
       "      <td>386.0</td>\n",
       "      <td>888.0</td>\n",
       "      <td>0.019969</td>\n",
       "      <td>NaN</td>\n",
       "      <td>790.0</td>\n",
       "      <td>264.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Equity(49666 [AGR])</th>\n",
       "      <td>NaN</td>\n",
       "      <td>575.0</td>\n",
       "      <td>1060.0</td>\n",
       "      <td>1332.0</td>\n",
       "      <td>426.0</td>\n",
       "      <td>658.0</td>\n",
       "      <td>751.0</td>\n",
       "      <td>185.0</td>\n",
       "      <td>267.0</td>\n",
       "      <td>1421.0</td>\n",
       "      <td>584.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>374.0</td>\n",
       "      <td>560.0</td>\n",
       "      <td>490.0</td>\n",
       "      <td>-0.021012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>114.0</td>\n",
       "      <td>365.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Equity(49700 [FCE_A])</th>\n",
       "      <td>NaN</td>\n",
       "      <td>576.0</td>\n",
       "      <td>997.0</td>\n",
       "      <td>1328.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>878.0</td>\n",
       "      <td>716.0</td>\n",
       "      <td>469.0</td>\n",
       "      <td>756.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>337.0</td>\n",
       "      <td>571.0</td>\n",
       "      <td>782.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>-0.034454</td>\n",
       "      <td>NaN</td>\n",
       "      <td>490.0</td>\n",
       "      <td>86.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 39 Week Returns  \\\n",
       "date                      security                                 \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])              300.0   \n",
       "                          Equity(49506 [HPE])             1454.0   \n",
       "                          Equity(49630 [CSRA])               NaN   \n",
       "                          Equity(49666 [AGR])                NaN   \n",
       "                          Equity(49700 [FCE_A])              NaN   \n",
       "\n",
       "                                                 Asset Growth 3M  \\\n",
       "date                      security                                 \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])              903.0   \n",
       "                          Equity(49506 [HPE])              780.0   \n",
       "                          Equity(49630 [CSRA])             352.0   \n",
       "                          Equity(49666 [AGR])              575.0   \n",
       "                          Equity(49700 [FCE_A])            576.0   \n",
       "\n",
       "                                                 Asset to Equity Ratio  \\\n",
       "date                      security                                       \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])                   1478.0   \n",
       "                          Equity(49506 [HPE])                    683.0   \n",
       "                          Equity(49630 [CSRA])                  1475.0   \n",
       "                          Equity(49666 [AGR])                   1060.0   \n",
       "                          Equity(49700 [FCE_A])                  997.0   \n",
       "\n",
       "                                                 Capex to Cashflows  \\\n",
       "date                      security                                    \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])                 574.0   \n",
       "                          Equity(49506 [HPE])                 233.0   \n",
       "                          Equity(49630 [CSRA])                584.0   \n",
       "                          Equity(49666 [AGR])                1332.0   \n",
       "                          Equity(49700 [FCE_A])              1328.0   \n",
       "\n",
       "                                                 EBIT to Assets  EBITDA Yield  \\\n",
       "date                      security                                              \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])             545.0        1441.0   \n",
       "                          Equity(49506 [HPE])            1031.0        1485.0   \n",
       "                          Equity(49630 [CSRA])            990.0        1149.0   \n",
       "                          Equity(49666 [AGR])             426.0         658.0   \n",
       "                          Equity(49700 [FCE_A])           100.0          21.0   \n",
       "\n",
       "                                                 Earnings Quality  \\\n",
       "date                      security                                  \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])              1400.0   \n",
       "                          Equity(49506 [HPE])              1443.0   \n",
       "                          Equity(49630 [CSRA])             1049.0   \n",
       "                          Equity(49666 [AGR])               751.0   \n",
       "                          Equity(49700 [FCE_A])             878.0   \n",
       "\n",
       "                                                 MACD Signal Line  \\\n",
       "date                      security                                  \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])               954.0   \n",
       "                          Equity(49506 [HPE])               862.0   \n",
       "                          Equity(49630 [CSRA])              842.0   \n",
       "                          Equity(49666 [AGR])               185.0   \n",
       "                          Equity(49700 [FCE_A])             716.0   \n",
       "\n",
       "                                                 Mean Reversion 1M  \\\n",
       "date                      security                                   \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])               1349.0   \n",
       "                          Equity(49506 [HPE])                609.0   \n",
       "                          Equity(49630 [CSRA])              1290.0   \n",
       "                          Equity(49666 [AGR])                267.0   \n",
       "                          Equity(49700 [FCE_A])              469.0   \n",
       "\n",
       "                                                 Moneyflow Volume 5D  \\\n",
       "date                      security                                     \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])                  898.0   \n",
       "                          Equity(49506 [HPE])                  373.0   \n",
       "                          Equity(49630 [CSRA])                1113.0   \n",
       "                          Equity(49666 [AGR])                 1421.0   \n",
       "                          Equity(49700 [FCE_A])                756.0   \n",
       "\n",
       "                                                 Net Income Margin  \\\n",
       "date                      security                                   \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])                628.0   \n",
       "                          Equity(49506 [HPE])               1233.0   \n",
       "                          Equity(49630 [CSRA])               626.0   \n",
       "                          Equity(49666 [AGR])                584.0   \n",
       "                          Equity(49700 [FCE_A])               69.0   \n",
       "\n",
       "                                                 Operating Cashflows to Assets  \\\n",
       "date                      security                                               \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])                            594.0   \n",
       "                          Equity(49506 [HPE])                            802.0   \n",
       "                          Equity(49630 [CSRA])                          1073.0   \n",
       "                          Equity(49666 [AGR])                            388.0   \n",
       "                          Equity(49700 [FCE_A])                          337.0   \n",
       "\n",
       "                                                 Price Momentum 3M  \\\n",
       "date                      security                                   \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])               1290.0   \n",
       "                          Equity(49506 [HPE])               1181.0   \n",
       "                          Equity(49630 [CSRA])              1225.0   \n",
       "                          Equity(49666 [AGR])                374.0   \n",
       "                          Equity(49700 [FCE_A])              571.0   \n",
       "\n",
       "                                                 Price Oscillator  \\\n",
       "date                      security                                  \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])               433.0   \n",
       "                          Equity(49506 [HPE])              1402.0   \n",
       "                          Equity(49630 [CSRA])              386.0   \n",
       "                          Equity(49666 [AGR])               560.0   \n",
       "                          Equity(49700 [FCE_A])             782.0   \n",
       "\n",
       "                                                 Return on Invest Capital  \\\n",
       "date                      security                                          \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])                       429.0   \n",
       "                          Equity(49506 [HPE])                      1297.0   \n",
       "                          Equity(49630 [CSRA])                      888.0   \n",
       "                          Equity(49666 [AGR])                       490.0   \n",
       "                          Equity(49700 [FCE_A])                      37.0   \n",
       "\n",
       "                                                  Returns  Trendline  Vol 3M  \\\n",
       "date                      security                                             \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])   -0.025424        NaN  1175.0   \n",
       "                          Equity(49506 [HPE])   -0.041345        NaN   879.0   \n",
       "                          Equity(49630 [CSRA])   0.019969        NaN   790.0   \n",
       "                          Equity(49666 [AGR])   -0.021012        NaN   114.0   \n",
       "                          Equity(49700 [FCE_A]) -0.034454        NaN   490.0   \n",
       "\n",
       "                                                 Working Capital to Assets  \n",
       "date                      security                                          \n",
       "2016-09-14 00:00:00+00:00 Equity(49496 [FDC])                        350.0  \n",
       "                          Equity(49506 [HPE])                        669.0  \n",
       "                          Equity(49630 [CSRA])                       264.0  \n",
       "                          Equity(49666 [AGR])                        365.0  \n",
       "                          Equity(49700 [FCE_A])                       86.0  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, running pipeline gives us factors for every day and every security, ranked relative to each other. We assume that the order of individual factors might carry some weak predictive power on future returns. The question then becomes: how can we combine these weakly predictive factors in a clever way to get a single mega-alpha which is hopefully more predictive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is an important milestone. We have our ranked factor values on each day for each stock. Ranking is not absolutely necessary but has several benefits:\n",
    "* it increases robustness to outliers,\n",
    "* we mostly care about the relative ordering rather than the absolute values.\n",
    "\n",
    "Also note the `Returns` column. These are the values we want to predict given the factor ranks.\n",
    "\n",
    "Next, we are doing some additional transformations to our data:\n",
    "1. Shift factor ranks to align with future returns `n_fwd_days` days in the future.\n",
    "2. Find the top and bottom 30 percentile stocks by their returns. Essentially, we only care about relative movement of stocks. If we later short stocks that go down and long stocks going up relative to each other, it doesn't matter if e.g. all stocks are going down in absolute terms. Moverover, we are ignoring stocks that did not move that much (i.e. 30th to 70th percentile) to only train the classifier on those that provided strong signal. \n",
    "3. We also binarize the returns by their percentile to turn our ML problem into a classification one.\n",
    "\n",
    "`shift_mask_data()` is a utility function that does all of these."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def shift_mask_data(X, Y, upper_percentile=70, lower_percentile=30, n_fwd_days=1):\n",
    "    # Shift X to match factors at t to returns at t+n_fwd_days (we want to predict future returns after all)\n",
    "    shifted_X = np.roll(X, n_fwd_days, axis=0)\n",
    "    \n",
    "    # Slice off rolled elements\n",
    "    X = shifted_X[n_fwd_days:]\n",
    "    Y = Y[n_fwd_days:]\n",
    "    \n",
    "    n_time, n_stocks, n_factors = X.shape\n",
    "    \n",
    "    # Look for biggest up and down movers\n",
    "    upper = np.nanpercentile(Y, upper_percentile, axis=1)[:, np.newaxis]\n",
    "    lower = np.nanpercentile(Y, lower_percentile, axis=1)[:, np.newaxis]\n",
    "  \n",
    "    upper_mask = (Y >= upper)\n",
    "    lower_mask = (Y <= lower)\n",
    "    \n",
    "    mask = upper_mask | lower_mask # This also drops nans\n",
    "    mask = mask.flatten()\n",
    "    \n",
    "    # Only try to predict whether a stock moved up/down relative to other stocks\n",
    "    Y_binary = np.zeros(n_time * n_stocks)\n",
    "    Y_binary[upper_mask.flatten()] = 1\n",
    "    Y_binary[lower_mask.flatten()] = -1\n",
    "    \n",
    "    # Flatten X\n",
    "    X = X.reshape((n_time * n_stocks, n_factors))\n",
    "\n",
    "    # Drop stocks that did not move much (i.e. are in the 30th to 70th percentile)\n",
    "    X = X[mask]\n",
    "    Y_binary = Y_binary[mask]\n",
    "    \n",
    "    return X, Y_binary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we have our helper function to align our data properly we pass our factor ranks to it. You might wonder why we have to do the `swapaxes` thing below rather than just using `pandas` logic. The reason is that this way we can use the same `shift_mask_data()` function inside of a factor where we do not have access to a Pandas `DataFrame`. More on that in a future notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Massage data to be in the form expected by shift_mask_data()\n",
    "results_wo_returns = results.copy()\n",
    "returns = results_wo_returns.pop('Returns')\n",
    "Y = returns.unstack().values\n",
    "X = results_wo_returns.to_panel() \n",
    "X = X.swapaxes(2, 0).swapaxes(0, 1).values # (factors, time, stocks) -> (time, stocks, factors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we split our data into training (80%) and test (20%). This is common practice: our classifier will try to fit the training set as well as possible but it does not tell us how well it would perform on unseen data. Because we are dealing with time-series data we split along the time-dimension to only test on future data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Train-test split\n",
    "train_size_perc = 0.8\n",
    "n_time, n_stocks, n_factors = X.shape\n",
    "train_size = np.int16(np.round(train_size_perc * n_time))\n",
    "X_train, Y_train = X[:train_size, ...], Y[:train_size]\n",
    "X_test, Y_test = X[(train_size+n_fwd_days):, ...], Y[(train_size+n_fwd_days):]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can only exclude stocks that did not move by a lot (i.e. 30th to 70th percentile) during training, we keep all stocks in our test set and just binarize according to the median. This avoids look-ahead bias."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_train_shift, Y_train_shift = shift_mask_data(X_train, Y_train, n_fwd_days=n_fwd_days)\n",
    "X_test_shift, Y_test_shift = shift_mask_data(X_test, Y_test, n_fwd_days=n_fwd_days, \n",
    "                                             lower_percentile=50, \n",
    "                                             upper_percentile=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((91619, 18), (25403, 18))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_shift.shape, X_test_shift.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='training'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training our ML pipeline\n",
    "\n",
    "Before training our classifier, several preprocessing steps are advisable. The first one imputes nan values with the factor mean to get clean training data, the second scales our factor ranks to be between [0, 1).\n",
    "\n",
    "For training we are using the [AdaBoost classifier](https://en.wikipedia.org/wiki/AdaBoost) which automatically determines the most relevant features (factors) and tries to find a non-linear combination of features to maximize predictiveness while still being robust. In essence, AdaBoost trains an ensemble of weak classifiers (decision trees in this case) sequentially. Each subsequent weak classifier takes into account the samples (or data points) already classified by the previous weak classifiers. It then focuses on the samples misclassified by the previous weak classifiers and tries to get those correctly. With each new weak classifier you get more fine-grained in your decision function and correctly classify some previously misclassified samples. For prediction, you simply average the answer of all weak classifiers to get a single strong classifier.\n",
    "\n",
    "Of course, this is just an example and you can let your creativity and skill roam freely."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "start_timer = time()\n",
    "\n",
    "# Train classifier\n",
    "imputer = preprocessing.Imputer()\n",
    "scaler = preprocessing.MinMaxScaler()\n",
    "clf = ensemble.AdaBoostClassifier(n_estimators=150) # n_estimators controls how many weak classifiers are fi\n",
    "\n",
    "X_train_trans = imputer.fit_transform(X_train_shift)\n",
    "X_train_trans = scaler.fit_transform(X_train_trans)\n",
    "clf.fit(X_train_trans, Y_train_shift)\n",
    "\n",
    "end_timer = time()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time to train full ML pipline: 24.13 secs\n"
     ]
    }
   ],
   "source": [
    "print \"Time to train full ML pipline: %0.2f secs\" % (end_timer - start_timer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, training a modern ML classifier does not have to be very compute intensive. Scikit-learn is heavily optimized so the full process only takes less than 10 seconds. Of course, things like deep-learning (which is currently not available on Quantopian), might take a bit longer, but these models are also trained on data sets much much bigger than this (a famous subset of the ImageNet data set is 138 GB).\n",
    "\n",
    "This means that the current bottlenecks are retrieving the data from pipeline (RAM and i/o), not lack of GPU or parallel processing support."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on train set = 56.17%\n"
     ]
    }
   ],
   "source": [
    "Y_pred = clf.predict(X_train_trans)\n",
    "print('Accuracy on train set = {:.2f}%'.format(metrics.accuracy_score(Y_train_shift, Y_pred) * 100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The classifier does reasonably well on the data we trained it on, but the real test is on hold-out data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise*: It is also common to run cross-validation on the training data and tweak the parameters based on that score, testing should only be done rarely. Try coding a [sklearn pipeline](http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html) with [K-Fold cross-validation](http://scikit-learn.org/stable/modules/cross_validation.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating our ML classifier\n",
    "\n",
    "To evaluate our ML classifer on the test set, we have to transform our test data in the same way as our traning data. Note that we are only calling the `.transform()` method here which does not use any information from the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Transform test data\n",
    "X_test_trans = imputer.transform(X_test_shift)\n",
    "X_test_trans = scaler.transform(X_test_trans)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After all this work, we can finally test our classifier. We can predict binary labels but also get probability estimates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Predict!\n",
    "Y_pred = clf.predict(X_test_trans)\n",
    "\n",
    "Y_pred_prob = clf.predict_proba(X_test_trans)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predictions: [ 1. -1. -1. ...,  1.  1.  1.]\n",
      "Probabilities of class == 1: [ 50.01423782  49.96216672  49.93493176 ...,  50.03379872  50.05767302\n",
      "  50.05988158]\n"
     ]
    }
   ],
   "source": [
    "print 'Predictions:', Y_pred\n",
    "print 'Probabilities of class == 1:', Y_pred_prob[:, 1] * 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many ways to evaluate the performance of our classifier. The simplest and most intuitive one is certainly the accuracy (50% is chance due to our median split). On Kaggle competitions, you will also often find the log-loss being used. This punishes you for being wrong *and* confident in your answer. See  [the Kaggle description](https://www.kaggle.com/wiki/LogarithmicLoss) for more motivation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test set = 49.64%\n",
      "Log-loss = 0.69316\n"
     ]
    }
   ],
   "source": [
    "print('Accuracy on test set = {:.2f}%'.format(metrics.accuracy_score(Y_test_shift, Y_pred) * 100))\n",
    "print('Log-loss = {:.5f}'.format(metrics.log_loss(Y_test_shift, Y_pred_prob)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seems like we're at chance on this data set, alas. But perhaps you can do better?\n",
    "\n",
    "We can also examine which factors the classifier identified as most predictive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0UAAAJqCAYAAADkCuviAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3X90VPWB///XZEAgYUwywERNbXRnTVwTcGNSPtJgYiUa\n4MS1KmIwgFW2VtFWF3uKBJpUJY2/8Qdl1VOwK4KjuMYfqZaqa4Q1QWNElPhrmbZjNJJk+BHCBATC\n/f7hN3OawjAD5A5c7/Nxjmdn7vve933NMLfw2jv3jsMwDEMAAAAAYFMJxzoAAAAAABxLlCIAAAAA\ntkYpAgAAAGBrlCIAAAAAtkYpAgAAAGBrlCIAAAAAtjboWAcAABw7Z555pjIyMuR0OiVJhmEoPT1d\nv//9749ovj179uiVV17Rj3/844GMGfb666/rzTffVHV1tSnzH8qqVat0xRVXxH2/AADzOfidIgCw\nr3/5l3/RW2+9JY/HMyDzffDBB3rooYf0xBNPxLyNYRhyOBwDsn8zGIahYDCo6dOna/Xq1cc6DgDA\nBHx9DgBszDAMRfr/jbW3t+v6669XSUmJJk6cqDVr1oTHVq1apUmTJqmkpEQzZszQ119/rS1btuim\nm27SBx98oOnTp+urr75SdnZ2eJu/f15bW6tf/OIXuuaaa3TvvfdKkp555hlNmjRJEyZM0K233qo9\ne/YckKm2tlbXXHONJGnevHlatGiRZs6cqR/+8IdatGiR/vu//1sXX3yxJkyYoI0bN0qSZsyYoYce\nekhXXnmlxo8fr8rKyvBrfuedd3TZZZdp8uTJuvLKK9XS0tIv37XXXqt77rlH06ZN01dffaXJkydr\n3759Wr9+vS677DJNmjRJpaWlamxsDL/G8ePHa/ny5br44otVVFSkV199NZy/pqZGEyZM0MSJE7V0\n6dLw8sWLF2vixIm64IIL9Nvf/jac79VXX9XFF1+syZMn65JLLlFTU9Ph/PECAGJlAABsKysry9i8\nefNBx66++mrj4YcfNgzDML744gtj7Nixxvbt241gMGjk5OQYX3/9tWEYhjFv3jxjwYIFhmEYxvPP\nP29cc801hmEYxpdffmlkZ2eH5/v7588//7xxzjnnGF988YVhGIbR1NRkFBQUGJ2dnYZhGEZVVZVx\n1113HZDp7+e/7bbbjEsvvdTYvXu38fnnnxtnnXWW8dhjjxmGYRh33XWX8atf/cowDMOYPn26MXXq\nVOObb74xdu3aZZSUlBhvvPGGEQqFjHPPPddYv369YRiGsXr1auOiiy46aL533nknPGYYhlFaWmq8\n/PLLhmEYRm1trXHhhRf2e40rVqwwDMMwXn311fB2L7zwgnHVVVcZvb29Rnd3t/GjH/3I+PDDD43a\n2lqjtLTU2Llzp9Hb22v87Gc/M5566inDMAzj3HPPDb/PH374oVFTUxPxzxIAcOQ4UwQANjdz5kxN\nnjw5/F9lZaV27dqld955R1dffbUk6dRTT1V+fr7q6+s1YsQIvffeezrppJMkSfn5+WptbT3s/WZk\nZOjUU0+VJL355puaNGmSRo4cKUm68sor9dprr0Wdo6CgQEOGDNEZZ5yh/fv3a8KECZKkrKwsdXR0\nhNebPHmyTjjhBA0dOlTnnXee1q9frw0bNujkk0/Wv/7rv0qSLrroIm3fvl1ffvnlAfn+UW1trSZP\nnixJysvLC28jSb29vbrsssskSdnZ2fr6668lSWvWrFFJSYkSEhI0fPhwvfLKKxo9erTq6+t1+eWX\nKykpSQkJCZoyZUr4tY8cOVJPP/202traNHr0aN12220xvrsAgMPBjRYAwOaWL19+wDVFHR0dMgxD\nZWVlkr79mt2uXbs0btw47d+/X4888ojefPNN7d+/Xzt37tTpp59+2PtNSUkJP+7u7tZrr72mt99+\nW9K3xaK3tzfqHElJSeHHCQkJGjZsWPjx32+fnJzc73FHR4e2bdumE088sd98LpdLW7duPSDfP6qr\nq9Py5cvV09Oj3t7efl9BdDqdGjp0aDjH/v37JUnbt2/vt7++dbq7u7Vs2TI9++yzMgxD+/fvl9vt\nliQtWbJES5Ys0WWXXaZTTjlF8+bN0w9+8IOo7wsA4PBQigDA5oyDXFM0YsQIDRo0SM8//3z4H+99\n6urqVF9fr5UrVyo5OVmrVq3Syy+/fMAcTqczXAgkqaurK2IGj8ejSy+9VL/61a+O4pVEtn379n6P\nU1JSNGLECG3btq3fel1dXRoxYoT8fn/Eudrb2/XrX/9azz33nLKyshQIBDRx4sSoGVJSUvrtb8uW\nLRoyZIg8Ho8uuOAClZeXH7DNqaeeqpqaGknfnp269dZb+13bBQAYGHx9DgBwAKfTqaKiIq1cuVKS\ntGvXLlVUVKi9vV1btmxRenq6kpOTtW3bNr366qvq6emRJA0aNEg7d+6UJKWmpiohIUGff/65JOnF\nF1+MuL8LLrhAr732Wvgszeuvv37EtwU/mNdee0179uxRT0+P1q5dq/z8fI0ZM0ZbtmzRhg0bJH1b\n9k466SSlp6cfsP2gQYMUCoXU29urbdu2KTExUaeffrr27dunZ555JvweSQeWzL7nEyZMUF1dXTjH\nVVddpU2bNmnChAl66aWXtHv3bknf3nDihRde0NatW3XttdeG388xY8YoIYG/tgHADJwpAgAbO9St\nsKuqqlRZWalVq1bJ4XDo3/7t35SWlqbS0lL98Y9/VElJib73ve/plltu0ezZs3XPPfdo+vTpuu++\n+3TeeedpzZo1+vnPf65Zs2YpLS1N06dPj7ivs846Sz/72c80c+ZMGYYht9utO+64Y8BeS25urmbO\nnKnW1lYVFxersLBQkvTggw/q9ttv1+7du+V2u/Xggw8edPusrCwlJydr/Pjxev7551VUVKSSkhKN\nHDlSc+fO1fvvvx++y90/5uh7PnnyZH322WcqKSnRkCFDdMUVV4SvZ9q0aZMuvfRSORwOff/731d1\ndbXcbrcKCws1ZcoUDRo0SIMHD9Zvf/vbw3pPAACxMf13impqarRhwwY5HA5VVFRo9OjR4bGGhgYt\nWrRITqdThYWFmj17tgzDUFVVlT7//HOdcMIJuv3224/ou+oAAEjf3pJ76tSpuvjii491FADAccrU\nM0VNTU0KBALy+Xzy+/2aP3++fD5feLy6ulrLli2Tx+PR9OnTVVJSor/+9a/auXOnfD6fvvjiC1VX\nV+uxxx4zMyYAAAAAGzO1FDU2Nqq4uFiS5PV6tWPHDoVCISUlJam1tVUpKSlKS0uTJBUVFamxsVG7\nd+/WmDFjJEnf//731draetz/2jkA4PjF3x8AgGhMvWIzGAyGbysqfXvRbTAYPOiY2+1WZ2enMjMz\ntXbtWu3fv19/+ctf9PXXXx9wdyAAAGL15JNP8tU5AMAhxfVGC4e6fKlvrLCwUO+9957Ky8uVm5sr\nj8dzyO0AAAAA4GiYWoo8Hk/4zJD07Y8Bjho1KjzW2dkZHmtvbw//eOCcOXMkSfv27dMLL7ygESNG\nHHI/zc3NAx0dAAAAwHdMXl7eQZebWooKCgq0ePFiTZ06VS0tLUpLS1NiYqIkKT09XaFQSG1tbfJ4\nPKqvr9f999+vTz/9VE899ZQWLlyoP/3pTxo7dmxM+4r0Ao9Gc3OzKfOagazmsVJeK2WVrJWXrOax\nUl4rZZWslZes5rFSXitllayVl6yHPpFiainKzc1Vdna2ysrK5HQ6VVlZqdraWrlcLhUXF6uqqip8\nVqi0tFQZGRkyDEO9vb2aOnWqBg8erAceeMDMiAAAAABszvRrivpKT5+srKzw4/z8/H636Ja+vUtQ\nTU2N2bEAAAAAQJLJd58DAAAAgOMdpQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANgapQgAAACA\nrVGKAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANjaoGMdIJ56e3vl9/tjXj8QCMjl\nckVdz+v1yul0Hk00AAAAAMeIrUqR3+/XjHkrlZjsiX2jus2HHO7p6tDymquUmZl5lOkAAAAAHAu2\nKkWSlJjs0fDU9GMdAwAAAMBxgmuKAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANga\npQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAA\nANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGK\nAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANjaILN3UFNT\now0bNsjhcKiiokKjR48OjzU0NGjRokVyOp0qLCzU7Nmz1dPTo7lz56qrq0t79+7VjTfeqPHjx5sd\nEwAAAIBNmVqKmpqaFAgE5PP55Pf7NX/+fPl8vvB4dXW1li1bJo/HoxkzZqikpETr1q3TP/3TP+k/\n/uM/1NHRoauvvlqvvvqqmTEBAAAA2JipX59rbGxUcXGxJMnr9WrHjh0KhUKSpNbWVqWkpCgtLU0O\nh0OFhYVat26dRowYoW3btkmSurq65Ha7zYwIAAAAwOZMPVMUDAaVk5MTfp6amqpgMKikpCQFg8F+\nhcftdqu1tVXl5eV6/vnnddFFF6m7u1uPP/64mRGPW729vfL7/TGvHwgE5HK5YlrX6/XK6XQeaTQA\nAADgO8X0a4r+nmEYUcdeeuklnXTSSXr88cf16aef6te//rVWrVoVde7m5uao6wQCgdjDHoaNGzeq\nu7t7QOcMBAK6e8WHSkz2xL5R3eaoq/R0dWhu+RhlZGQcRbqjF8uf1/HESnmtlFWyVl6ymsdKea2U\nVbJWXrKax0p5rZRVslZeskZmainyeDwKBoPh5x0dHRo1alR4rLOzMzzW3t4uj8ej999/X+edd54k\n6cwzz9TmzZtlGIYcDsch95WXlxc1j8vliqk4HK6cnBxlZmYO6Jwul0uJyZs1PDV9QOeVzMl7OJqb\nm2P68zpeWCmvlbJK1spLVvNYKa+VskrWyktW81gpr5WyStbKS9ZDFy1TrykqKCjQ6tWrJUktLS1K\nS0tTYmKiJCk9PV2hUEhtbW3at2+f6uvrNX78eGVkZOiDDz6QJH311VdKTEyMWogAAAAA4EiZeqYo\nNzdX2dnZKisrk9PpVGVlpWpra+VyuVRcXKyqqirNmTNHklRaWqqMjAxdeeWVqqio0IwZM9Tb26s7\n77zTzIgAAAAAbM70a4r6Sk+frKys8OP8/Px+t+iWpMTERD344INmxwIAAAAASSZ/fQ4AAAAAjneU\nIgAAAAC2RikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2RikCAAAA\nYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2RikC\nAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2\nRikCAAAAYGuUIgAAAAC2RikCAAAAYGuDjnUAWF9vb6/8fn/M6wcCAblcrpjW9Xq9cjqdRxrtoKyW\nFwAAAOaiFOGo+f1+zZi3UonJntg3qtscdZWerg4tr7lKmZmZR5HuQFbLCwAAAHNRijAgEpM9Gp6a\nfqxjxMxqeQEAAGAerikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2\nRikCAAAAYGuUIgAAAAC2RikCAAAAYGuUIgAAAAC2NsjsHdTU1GjDhg1yOByqqKjQ6NGjw2MNDQ1a\ntGiRnE6nioqKdMMNN+i5557Tiy++KIfDIcMw1NLSovfff9/smAAAAABsytRS1NTUpEAgIJ/PJ7/f\nr/nz58vn84XHq6urtWzZMnk8Hk2fPl0XXXSRpkyZoilTpoS3/9Of/mRmRAAAAAA2Z+rX5xobG1Vc\nXCxJ8nq92rFjh0KhkCSptbVVKSkpSktLk8PhUFFRkdatW9dv+9/97neaPXu2mREBAAAA2JyppSgY\nDMrtdoefp6amKhgMHnTM7Xaro6Mj/Pyjjz7SySefrBEjRpgZEQAAAIDNxfVGC4ZhxDy2atUqXXbZ\nZWZHAgAAAGBzpl5T5PF4wmeGJKmjo0OjRo0Kj3V2dobH2tvb5fF4ws/fffddVVZWxryv5ubmqOsE\nAoGY5zscGzduVHd394DOaVZWaeDzWimrZL28hyuWY+F4YqW8ZDWPlfJaKatkrbxkNY+V8lopq2St\nvGSNzNRSVFBQoMWLF2vq1KlqaWlRWlqaEhMTJUnp6ekKhUJqa2uTx+NRfX297r//fknflqekpCQN\nGhR7vLy8vKjruFwuqW7zkb2YQ8jJyVFmZuaAzmlWVmng81opq2S9vIejubk5pmPheGGlvGQ1j5Xy\nWimrZK28ZDWPlfJaKatkrbxkPXTRMrUU5ebmKjs7W2VlZXI6naqsrFRtba1cLpeKi4tVVVWlOXPm\nSJJKS0uVkZEhSers7ORaIgAAAABxYfrvFPWVnj5ZWVnhx/n5+f1u0d0nOztbjz/+uNnRAAAAACC+\nN1oAAAAAgOMNpQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANgapQgAAACArVGKAAAAANgapQgA\nAACArVGKAAAAANgapQgAAACArVGKAAAAANjaoGMdAEBkvb298vv9Ma8fCATkcrliWtfr9crpdB5p\ntIMyK6+Vskrm5AUAAOahFAHHMb/frxnzViox2RP7RnWbo67S09Wh5TVXKTMz8yjSHciMvFbKKpmX\nFwAAmIdSBBznEpM9Gp6afqxjxMxKea2UFQAAmIdrigAAAADYGqUIAAAAgK1RigAAAADYGqUIAAAA\ngK1RigAAAADYGqUIAAAAgK1RigAAAADYGqUIAAAAgK1RigAAAADYGqUIAAAAgK1RigAAAADYGqUI\nAAAAgK1RigAAAADYGqUIAAAAgK1RigAAAADYGqUIAAAAgK1RigAAAADYGqUIAAAAgK1RigAAAADY\nGqUIAAAAgK1RigAAAADYGqUIAAAAgK1RigAAAADYGqUIAAAAgK1RigAAAADYGqUIAAAAgK1RigAA\nAADYmumlqKamRmVlZZo2bZo++uijfmMNDQ264oorVFZWpiVLloSXv/TSS7rkkkt0+eWX66233jI7\nIgAAAAAbM7UUNTU1KRAIyOfzaeHChaquru43Xl1drcWLF+vpp5/W22+/Lb/fr+3bt+t3v/udfD6f\nHnvsMb3xxhtmRgQAAABgc4PMnLyxsVHFxcWSJK/Xqx07digUCikpKUmtra1KSUlRWlqaJKmoqEjr\n1q1TamqqCgoKNGzYMA0bNkx33HGHmREBAAAA2JyppSgYDConJyf8PDU1VcFgUElJSQoGg3K73eEx\nt9ut1tZW9fT0aNeuXbrhhhvU3d2tG2+8UePGjTMzJgAc13p7e+X3+2NePxAIyOVyxbSu1+uV0+k8\n0mgHZaW8VsoKADCPqaXoHxmGEXXMMAxt375dS5Ys0ZdffqmZM2fqzTffjDp3c3Nz1HUCgUDsYQ/D\nxo0b1d3dPaBzmpVVGvi8VsoqWSuvlbJKHGN9zPgc3L3iQyUme2LfqG5z1FV6ujo0t3yMMjIyjiLd\ngayU10pZj1Qsfz8eL8hqHivltVJWyVp5yRqZqaXI4/EoGAyGn3d0dGjUqFHhsc7OzvBYe3u7PB6P\nEhMTlZubK4fDoVNPPVVJSUnaunVrv7NKB5OXlxc1j8vliukvs8OVk5OjzMzMAZ3TrKzSwOe1UlbJ\nWnmtlFXiGOtjxucgMXmzhqemD9icfcx6b62S10pZj0Rzc3NMfz8eD8hqHivltVJWyVp5yXroomXq\njRYKCgq0evVqSVJLS4vS0tKUmJgoSUpPT1coFFJbW5v27dun+vp6jR8/Xj/84Q/1zjvvyDAMbdu2\nTT09PVELEQAAAAAcKVPPFOXm5io7O1tlZWVyOp2qrKxUbW2tXC6XiouLVVVVpTlz5kiSSktLw18z\nKCkp0dSpU+VwOFRZWWlmRAAAAAA2Z/o1RX2lp09WVlb4cX5+vnw+3wHbTJ06VVOnTjU7GgAAAACY\n/+OtAAAAAHA8oxQBAAAAsLVDfn1u165dWrVqldauXauvvvpK0rc3SDjvvPM0ZcqU8E0TAAAAAMCq\nIpait99+W7fffrt+9KMfafr06TrllFMkSW1tbWpoaNCll16q3/zmN/ywKgAAAABLi1iKnnnmGT37\n7LNKSUnpt/yMM85QUVGRrr/+elVVVVGKAAAAAFhaxGuKHn74YaWkpOi+++47YGz+/PlKTU3Vww8/\nbGo4AAAAADBbxDNFr732mv785z+rsbFRHR0d4eV79+7Ve++9F5dwAAAAAGC2iKXovPPOk9vt1saN\nG/t9Rc7hcOjnP/95XMIBAAAAgNkilqKhQ4cqLy9PL7zwgoYMGSLDMGQYRjyzAQAAAIDpDnlLbkl6\n6qmn9J//+Z8KhUKSJMMw5HA49Mknn5geDgAAAADMFrUUPffcc3rppZfCt+QGAAAAgO+SiHef65OR\nkUEhAgAAAPCdFfVMUVZWlm699VaNHTtWTqczvHzKlCmmBgMAAACAeIhaijo6OnTCCSfogw8+6Lec\nUgQAAADguyBqKaqpqdH+/fu1ZcsWjRo1Kh6ZAAAAACBuol5T1NjYqOLiYs2YMUOS9Nvf/lb19fVm\n5wIAAACAuIhaihYtWqRnn302fJbo+uuv15IlS0wPBgAAAADxELUUJSYmauTIkeHnbrdbgwcPNjUU\nAAAAAMRL1GuKhg4dqnfffVeS1NXVpT/+8Y8aMmSI6cEAAAAAIB6inimqqqrS0qVL9dFHH+nCCy/U\n2rVrdccdd8QjGwAAAACYLuqZopNPPlmPPfZYPLIAAAAAQNxFLEULFy7UggULdNVVV8nhcBwwvmLF\nClODAQAAAEA8RCxFfT/Oesstt8QtDAAAAADEW8Rris4880xJ0mmnnaZPP/1UY8eO1dixY/X2228r\nIyMjbgEBAAAAwExRb7Qwb968frfkPuOMMzRv3jxTQwEAAABAvEQtRXv27NHkyZPDz0tLS7V3715T\nQwEAAABAvEQtRZK0Zs0a7d69Wz09PVq9evVBb7wAAAAAAFYU9Zbcd955p37zm9/o5ptvlsPh0Dnn\nnKM777wzHtkAAAAAwHRRS9Fpp52mP/zhD3GIAgAADqW3t1d+vz/m9QOBgFwuV9T1vF6vnE7n0UQ7\ngFlZJXPyArA3fqcIAACL8Pv9mjFvpRKTPbFvVLf5kMM9XR1aXnOVMjMzjzJdf2ZklczLC8DeIpai\nH//4x5KkG2+8UYMHD45bIAAAEFliskfDU9OPdYyYWCkrAHs75Jkin8+nxx57TE8++WQ8MwEAAABA\n3EQsRQ6HQ+PHj1dXV5fOP//88HLDMORwOFRfXx+HeAAAAABgroilaMWKFWpvb1dFRYUWLlwYz0wA\nAAAAEDcRS9EvfvELLV68WC6XS+npfB8YAAAAwHdTxFIUCAR05ZVX6i9/+YvKy8sPGOfucwAAAAC+\nCyKWopUrV+qzzz7TwoULdfPNN8czEwAAAADETcRS5HK5lJ+fr5UrV+qbb77Rl19+qdGjR2v//v1K\nSEiIZ0YAAAAAME3UdvPmm2/qyiuv1Lx58yRJd955p1atWmV6MAAAAACIh6ilaNmyZXrxxReVmpoq\nSZo7d66effZZ04MBAAAAQDxELUUul0vDhg0LPx86dKgGDx5saigAAAAAiJeI1xT1SU1NVW1trb75\n5hu1tLTolVdekdvtjnkHNTU12rBhgxwOhyoqKjR69OjwWENDgxYtWiSn06nCwkLNnj1b7777rm6+\n+WadccYZMgxDWVlZWrBgwZG9OgAAAACIImopuv322/Xggw8qFAppwYIFysvLi/nHXJuamhQIBOTz\n+eT3+zV//nz5fL7weHV1tZYtWyaPx6Pp06erpKREkjR27Fg99NBDR/iSAAAAACB2UUvRiSeeqMrK\nSm3fvl0Oh0PJyckxT97Y2Kji4mJJktfr1Y4dOxQKhZSUlKTW1lalpKQoLS1NklRUVKR169aFzxAB\nAAAAQDxEvaaoublZxcXFmjRpkkpKSjRx4kR9+OGHMU0eDAb7fdUuNTVVwWDwoGNut1sdHR2SJL/f\nr9mzZ6u8vFwNDQ2H9YIAAAAA4HBEPVP0wAMPaMmSJcrMzJQkffzxx6qurtaKFSsOe2eHOgPUN3ba\naafppptu0qRJk9Ta2qqZM2fqtdde06BBh47a3Nwcdf+BQODwAsdo48aN6u7uHtA5zcoqDXxeK2WV\nrJXXSlkljrE+fA6sk9dKWSWOsT5m5D1csfy743hipbxWyipZKy9ZI4taihISEsKFSJLOOussOZ3O\nmCb3eDzhM0OS1NHRoVGjRoXHOjs7w2Pt7e3yeDzyeDyaNGmSJOnUU0/VyJEj1d7ervT09EPuKy8v\nL2oel8sl1W2OKfvhyMnJ6fceDQSzskoDn9dKWSVr5bVSVoljrA+fA+vktVJWiWOsjxl5D0dzc3NM\n/+44Xlgpr5WyStbKS9ZDF62oX59LSEjQn//8Z+3cuVM7d+7UK6+8EnMpKigo0OrVqyVJLS0tSktL\nU2JioiQpPT1doVBIbW1t2rdvn+rr6zV+/Hi9/PLLWrx4sSRpy5Yt2rp1a/i6IwAAAAAYaDHdfe7O\nO+/U/PnzlZCQoLPPPlu33357TJPn5uYqOztbZWVlcjqdqqysVG1trVwul4qLi1VVVaU5c+ZIkkpL\nS5WRkaGRI0fq1ltv1bRp02QYhn7zm99E/eocAAAAABypqG3jtNNO09KlS494B32lp09WVlb4cX5+\nfr9bdEtSUlKSHn300SPeHwAAAAAcjohfn9u2bZt+8pOfaOfOneFlGzZs0LXXXqtdu3bFJRwAAAAA\nmC1iKbrrrrv0wx/+UMOHDw8vO/vss1VQUKD77rsvLuEAAAAAwGwRS5Hf79d11113wPJZs2bp448/\nNjUUAADSC0KfAAAgAElEQVQAAMRLxFJ0qDvM7du3z5QwAAAAABBvEUtRQkKC/va3vx2w/LPPPtPg\nwYPNzAQAAAAAcRPx7nM33HCDZs2apeuuu06jR49Wb2+vmpub9Yc//EGPPPJIPDMCAAAAgGkilqLC\nwkL97ne/09KlS+Xz+ZSQkKCsrCwtXbpUXq83nhkBAAAAwDSH/J2iM888U/fee2+8sgAAAABA3EW8\npggAAAAA7IBSBAAAAMDWKEUAAAAAbC3iNUW33HKLHnzwQRUVFcnhcISXG4Yhh8Oh+vr6eOQDAAAw\nVW9vr/x+f8zrBwIBuVyumNb1er2H/O1HAMeHiKVowYIFkqSVK1fGLQwAAEC8+f1+zZi3UonJntg3\nqtscdZWerg4tr7lKmZmZR5EOQDxELEUjR44M/9+1a9eqq6tLhmGEx6dMmWJ+OgAAgDhITPZoeGr6\nsY4B4Bg55C25Jenf//3f5XA4lJ7e/38oKEUAAAAAvguilqK9e/fK5/PFIwsAAAAAxF3Uu8/98z//\ns7Zt2xaPLAAAAAAQd1HPFG3evFkXXXTRAXdPWbFihanBAAAAACAeopai6667Lh45AAAAAOCYiPj1\nuY8//ljSt/fuP9h/AAAAAPBdEPFM0YsvvqizzjpLS5YsOWDM4XBo3LhxpgYDAAAAgHiIWIrmzZsn\nSVq+fHncwgAAAABAvB3ymqKNGzcqJydHknTXXXdp69atGjp0qCoqKjR06NC4BAQAAAAAM0W8pqiu\nrk433nij9u3bJ0lqamrSuHHjtGfPHj3++ONxCwgAAAAAZopYip544gk9+eSTGjTo25NJiYmJuvTS\nS1VZWak1a9bELSAAAAAAmCliKRoyZIgyMjLCz88991xJ35ajYcOGmZ8MAAAAAOIgYinavXt3v+c3\n3nhj+HEoFDIvEQAAAADEUcRSlJ6errfeeuuA5XV1dcrKyjI1FAAAAADES8S7z91666366U9/qrq6\nOuXk5Ki3t1fr16/X3/72N61YsSKeGQEAAADANBHPFJ122mmqq6vT2LFj1dbWps7OTk2YMEG1tbU6\n8cQT45kRAAAAAEwT8UzRE088oWuuuUZXXHFFxI371gEAAAAAq4p4pigUCqm8vFyvv/66enp6wst7\nenr0+uuvq7y8vN9yAAAAALCiiGeKbrrpJhUWFurRRx/Vr371Kw0ePFiStHfvXo0bN05z587VmDFj\n4hYUAAAAAMwQsRRJ0pgxY7RkyRLt379f27dvlySlpKQoISHiCSYAAAAAsJRDlqI+CQkJcrvdZmcB\nAAAAgLjjlA8AAAAAW6MUAQAAALC1qKWoq6tLd999t375y19Kkv7nf/5HW7duNT0YAAAAAMRD1FK0\nYMECnXzyyfryyy8lSXv27NHcuXNNDwYAAAAA8RC1FG3dulUzZ84M35J74sSJ2r17t+nBAAAAACAe\nYrqmaO/evXI4HJKkYDB4WD/aWlNTo7KyMk2bNk0fffRRv7GGhgZdccUVKisr05IlS/qNffPNN7rw\nwgv1wgsvxLwvAAAAADhcUUvR9OnTNWXKFG3atEnXX3+9LrnkEs2aNSumyZuamhQIBOTz+bRw4UJV\nV1f3G6+urtbixYv19NNP6+2335bf7w+PLVmyRCkpKYf5cgAAAADg8ET9naJJkyYpNzdX69ev1wkn\nnKA77rhDHo8npskbGxtVXFwsSfJ6vdqxY4dCoZCSkpLU2tqqlJQUpaWlSZKKioq0bt06eb1e+f1+\n/fWvf1VRUdFRvDQAAAAAiC7qmaJNmzZpxYoVmjRpkiZMmKBFixbp888/j2nyYDDY70dfU1NTFQwG\nDzrmdrvV0dEhSbr33nt12223HdYLAQAAAIAjEfVM0e23366bb745/Pzyyy/XHXfcoaeeeuqwd2YY\nRtSxF154QT/4wQ90yimnRN0GAADAbnp7e/tdchBNIBCQy+WKaV2v1yun03mk0Q5gpaywt6ilqLe3\nV/n5+eHnf/84Go/HEz4zJEkdHR0aNWpUeKyzszM81t7eLo/HozVr1qi1tVV//vOftXnzZg0ZMkQn\nnXSSxo0bd8h9NTc3R80TCARizn44Nm7cqO7u7gGd06ys0sDntVJWyVp5rZRV4hjrw+fAOnmtlFXi\nGOvD5yCgu1d8qMTk2C5nkCTVbY66Sk9Xh+aWj1FGRsZRpOvPSlmPVCz/Bj1ekDWyqKXI5XJp5cqV\n+n//7/9p//79Wrt2rZKSkmKavKCgQIsXL9bUqVPV0tKitLQ0JSYmSpLS09MVCoXU1tYmj8ej+vp6\n3X///SovLw9vv3jxYn3ve9+LWogkKS8vL+o6LpcrpgPtcOXk5CgzM3NA5zQrqzTwea2UVbJWXitl\nlTjG+vA5sE5eK2WVOMb68DlwKTF5s4anpg/ovJI5761Vsh6J5ubmmP4Nejwg66GLVtRSVFNTo/vv\nv19PP/20JCk3N1c1NTUx7Tg3N1fZ2dkqKyuT0+lUZWWlamtr5XK5VFxcrKqqKs2ZM0eSVFpaely0\nfQAAAAD2ErUUud3uA26lfTj6Sk+frKys8OP8/Hz5fL6I2950001HvF8AAAAAiEXUUlRXV6ff//73\n6urq6nfTg/r6ejNzAQAAAEBcRC1FjzzyiBYuXBi+GxwAAAAAfJdELUUZGRn6wQ9+EI8sAAAAABB3\nUUtRbm6uHnjgAY0dO7bfveBjuSMcAAAAABzvopaihoYGSdL69evDyxwOB6UIAAAAwHdC1FK0fPny\nA5atXr3alDAAAAAAEG9RS1FbW5ueeuopbdu2TZK0Z88evfPOOyopKTE9HAAAAACYLSHaCnPnzlVK\nSoo++OAD5eTkaOvWrbr77rvjkQ0AAAAATBe1FDmdTl133XUaOXKkysvL9eijjx70K3UAAAAAYEVR\nS9GuXbv01VdfyeFwqLW1VYMGDdLmzZvjkQ0AAAAATBf1mqKf/vSnampq0qxZs3TJJZfI6XSqtLQ0\nHtkAAAAAwHRRS9Hpp58ur9crSXr33XcVCoX017/+1fRgAAAAABAPEb8+t2PHDn3xxReqqKhQa2ur\nWltb9fXXX2vLli2aO3duPDMCAAAAgGkinilav369/uu//kuffPKJrr766vDyhIQEjR8/Pi7hAAAA\nAMBsEUtRUVGRioqKtGLFCpWXl8czEwAAAADETdS7z7366qvxyAEAAAAAx0TUGy2cddZZeuihh5Sb\nm6vBgweHl48bN87UYAAAAAAQD1FL0SeffCJJeu+998LLHA4HpQgAAADAd0LUUrR8+fJ45AAAAACA\nYyLqNUV+v18zZ87UOeeco7y8PM2aNUtffPFFPLIBAAAAgOmilqI777xT1157rf73f/9Xa9asUVlZ\nmaqqquKRDQAAAABMF7UUGYah888/X4mJiUpKStKFF16o3t7eeGQDAAAAANNFLUV79+5VS0tL+PmH\nH35IKQIAAADwnRH1Rgtz587Vrbfeqi1btkiSPB6P7r77btODAQAAAEA8RC1FZ599tv70pz+pu7tb\nDodDw4cPj0cuAAAAAIiLqKVo06ZNevjhh7Vp0yY5HA5lZWXp5z//uU4//fR45AMAAAAAU0W9pui2\n225TYWGhFi9erIcffljnnnuu5s6dG49sAAAAAGC6qGeKhg0bpilTpoSfe71erV692tRQAAAAABAv\nUc8UnXvuuXr99de1a9cuhUIhvfHGGzrnnHNkGIb2798fj4wAAAAAYJqoZ4qWLFly0FtwP/LII3I4\nHPrkk09MCQYAAAAA8RC1FP39bxQBAAAAwHdN1FLU3t6u1atXq7u7W4ZhhJffdNNNpgYDAAAAgHiI\nek3RT3/6U33yySfau3ev9u3bF/4PAAAAAL4Lop4pSklJUU1NTTyyAAAAAEDcRS1FEyZM0EsvvaTc\n3Fw5nc7w8lNOOcXUYAAAAAAQD1FL0f/93//p5ZdfVkpKSniZw+FQfX29mbkAAAAAIC6ilqINGzao\nqalJJ5xwQjzyAAAAAEBcRb3RQk5Ojr755pt4ZAEAAACAuIvpltwXXHCBvF5vv2uKVqxYYWowAAAA\nAIiHqKXo+uuvj0cOAAAAADgmIpai/fv3S5Ly8/PjFgYAAAAA4i1iKTrrrLPkcDgOWG4YhhwOhz75\n5JOYdlBTU6MNGzbI4XCooqJCo0ePDo81NDRo0aJFcjqdKiws1OzZs7V7927ddttt2rJli/bs2aMb\nbrhB559//uG/MgAAAACIQcRS9Omnnx715E1NTQoEAvL5fPL7/Zo/f758Pl94vLq6WsuWLZPH49GM\nGTNUUlKizz77TKNHj9asWbPU1tama665hlIEAAAAwDRRryk6Go2NjSouLpYkeb1e7dixQ6FQSElJ\nSWptbVVKSorS0tIkSYWFhVq3bp3Ky8vD27e1tenkk082MyIAAAAAmzO1FAWDQeXk5ISfp6amKhgM\nKikpScFgUG63OzzmdrvV2toafl5WVqaOjg49+uijZkYEAAAAYHOmlqJ/ZBhGzGM+n0+ffvqpfvnL\nX+qll16KOndzc3PUdQKBQPSQR2Djxo3q7u4e0DnNyioNfF4rZZWslddKWSWOsT58DqyT10pZJY6x\nPnwOrJPXSlmPVCz/Bj1ekDUyU0uRx+NRMBgMP+/o6NCoUaPCY52dneGx9vZ2eTwebdy4USNGjNDJ\nJ5+sM888U729vdq6dWu/s0oHk5eXFzWPy+WS6jYf4auJLCcnR5mZmQM6p1lZpYHPa6WskrXyWimr\nxDHWh8+BdfJaKavEMdaHz4F18lop65Fobm6O6d+gxwOyHrpoJQz43v5OQUGBVq9eLUlqaWlRWlqa\nEhMTJUnp6ekKhUJqa2vTvn37VF9fr/Hjx+u9997TE088Ienbr9/t2rUraiECAAAAgCNl6pmi3Nxc\nZWdnq6ysTE6nU5WVlaqtrZXL5VJxcbGqqqo0Z84cSVJpaakyMjI0bdo0VVRUqLy8XN98842qqqrM\njAgAAADA5ky/pqiv9PTJysoKP87Pz+93i25JGjJkiO6//36zYwEAAACAJJO/PgcAAAAAxztKEQAA\nAABboxQBAAAAsDVKEQAAAABboxQBAAAAsDVKEQAAAABboxQBAAAAsDVKEQAAAABboxQBAAAAsDVK\nEQAAAABboxQBAAAAsLVBxzoAAAAAcDzo7e2V3++Pef1AICCXyxV1Pa/XK6fTeTTRDmBWVsmcvMc7\nShEAAAAgye/3a8a8lUpM9sS+Ud3mQw73dHVoec1VyszMPMp0/ZmRVTIv7/GOUgQAAAD8/xKTPRqe\nmn6sY8TESlmPd1xTBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUAAAAAbI1S\nBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUAAAAA\nbI1SBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUA\nAAAAbI1SBAAAAMDWKEUAAAAAbI1SBAAAAMDWKEUAAAAAbG2Q2TuoqanRhg0b5HA4VFFRodGjR4fH\nGhoatGjRIjmdThUWFmr27NmSpHvuuUfvv/++ent7dd111+nCCy80OyYAAAAAmzK1FDU1NSkQCMjn\n88nv92v+/Pny+Xzh8erqai1btkwej0fTp09XSUmJgsGgNm3aJJ/Pp+3bt+vSSy+lFAEAAAAwjaml\nqLGxUcXFxZIkr9erHTt2KBQKKSkpSa2trUpJSVFaWpokqaioSOvWrdO0adM0ZswYSdKJJ56oXbt2\nyTAMORwOM6MCAAAAsClTrykKBoNyu93h56mpqQoGgwcdc7vd6ujoUEJCgoYNGyZJWrVqlYqKiihE\nAAAAAExj+jVFf88wjJjHXn/9dT3//PNaunSp2bEAAAAAmKS3t1d+vz/m9QOBgFwuV0zrer1eOZ3O\nI40WZmop8ng84TNDktTR0aFRo0aFxzo7O8Nj7e3t8ng8kqS1a9fq8ccf19KlSzV8+PCY9tXc3Bx1\nnUAgcDjxY7Zx40Z1d3cP6JxmZZUGPq+VskrWymulrBLHWB8+B9bJa6WsEsdYHz4H1slrpawSx1gf\nMz4Hd6/4UInJntg3qtscdZWerg7NLR+jjIyMo0j3LVNLUUFBgRYvXqypU6eqpaVFaWlpSkxMlCSl\np6crFAqpra1NHo9H9fX1uv/++7Vz507de++9+sMf/hBzQ5SkvLy8qOu4XK6Y3uDDlZOTo8zMzAGd\n06ys0sDntVJWyVp5rZRV4hjrw+fAOnmtlFXiGOvD58A6ea2UVeIY62PG5yAxebOGp6YP2Jx9Difr\noU6imFqKcnNzlZ2drbKyMjmdTlVWVqq2tlYul0vFxcWqqqrSnDlzJEmlpaXKyMjQs88+q+3bt+uW\nW24J32Dhnnvu0UknnWRmVAAAAAA2Zfo1RX2lp09WVlb4cX5+fr9bdEvS1KlTNXXqVLNjAQAAAIAk\nk+8+BwAAAADHO0oRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoR\nAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACw\nNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEA\nAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFuj\nFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAAAFujFAEAAACwNUoRAAAA\nAFujFAEAAACwNUoRAAAAAFujFAEAAACwNdNLUU1NjcrKyjRt2jR99NFH/cYaGhp0xRVXqKysTEuW\nLAkv//TTT3XhhRdqxYoVZscDAAAAYHOmlqKmpiYFAgH5fD4tXLhQ1dXV/carq6u1ePFiPf3003r7\n7bfl9/u1a9cu3X333SooKDAzGgAAAABIMrkUNTY2qri4WJLk9Xq1Y8cOhUIhSVJra6tSUlKUlpYm\nh8OhoqIirVu3TkOGDNFjjz2mkSNHmhkNAAAAACSZXIqCwaDcbnf4eWpqqoLB4EHH3G63Ojo6lJCQ\noBNOOMHMWAAAAAAQNiieOzMM44jGYtHc3Bx1nUAgcFT7iGTjxo3q7u4e0DnNyioNfF4rZZWslddK\nWSWOsT58DqyT10pZJY6xPnwOrJPXSlkljrE+dvwcmFqKPB5P+MyQJHV0dGjUqFHhsc7OzvBYe3u7\nPB7PEe8rLy8v6joul0uq23zE+4gkJydHmZmZAzqnWVmlgc9rpayStfJaKavEMdaHz4F18lopq8Qx\n1ofPgXXyWimrxDHW57v6OTjUSRRTvz5XUFCg1atXS5JaWlqUlpamxMRESVJ6erpCoZDa2tq0b98+\n1dfXa/z48WbGAQAAAIADmHqmKDc3V9nZ2SorK5PT6VRlZaVqa2vlcrlUXFysqqoqzZkzR5JUWlqq\njIwMbdiwQQsWLNDWrVvldDrl8/n01FNPKTk52cyoAAAAAGzK9GuK+kpPn6ysrPDj/Px8+Xy+fuNn\nn322Xn75ZbNjAQAAAICkOPx4KwAAAAAczyhFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihF\nAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA\n1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQA\nAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyN\nUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAA\nAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1ihFAAAAAGyNUgQAAADA1gaZvYOamhpt2LBBDodDFRUV\nGj16dHisoaFBixYtktPpVGFhoWbPnh11GwAAAAAYSKaWoqamJgUCAfl8Pvn9fs2fP18+ny88Xl1d\nrWXLlsnj8Wj69OkqKSnR1q1bD7kNAAAAAAwkU0tRY2OjiouLJUler1c7duxQKBRSUlKSWltblZKS\norS0NElSUVGRGhsbtXXr1ojbAAAAAMBAM7UUBYNB5eTkhJ+npqYqGAwqKSlJwWBQbrc7POZ2u9Xa\n2qpt27ZF3GYg9HR1DMg8Zs1n9txm5bVSVrPm5r01Z24rZTVrTrPm5b01b14rvbdWymrWnGbNy3tr\n3rxWem+tlNWsOc2adyDndBiGYQzYbP+gsrJS559/vi644AJJ0lVXXaWamhplZGRo/fr1WrZsmR55\n5BFJ0qpVq/Tll19q27ZtEbeJpLm52ayXAAAAAOA7Ii8v76DLTT1T5PF4FAwGw887Ojo0atSo8Fhn\nZ2d4rL29XR6PR4MHD464TSSRXhwAAAAARGPqLbkLCgq0evVqSVJLS4vS0tKUmJgoSUpPT1coFFJb\nW5v27dun+vp6jR8//pDbAAAAAMBAM/Xrc5L0wAMP6N1335XT6VRlZaU+/vhjuVwuFRcX67333tN9\n990nSZo4caJ+8pOfHHSbrKwsMyMCAPD/sXfeYVFe6f++Z0BsWGMPGLGhSBMV1GhULKhINCYqFsQS\n3dg1xhixYUOJEWOPxoI1GhsqIiJqJBbEgqKCDbFipwmKwAy/P/jNu4wzoMnu8p7Jd+7r2muZ8/6x\nnx3fOec85zzP5zFixIgRI/+H+Z8HRUaMGDFixIgRI0aMGDEiMv/T9DkjRowYMWLEiBEjRowYER1j\nUGTEiBEjRowYMWLEiJH/0xiDIiNGjBgxYsSIESNGjPyfxhgUGTFipMhQq9VySzBi5B+NSqXi5cuX\nACQkJBAeHs7bt29lVvXPICsri4cPH8ot44N4+vSp3BKMCIZarSYtLU1uGUJjNFrIR1BQENnZ2XTv\n3p0RI0aQkpLCl19+Sb9+/eSWpsOff/7J9u3bSU9PJ/8/4aZNm2RUpU1iYmKhz2vUqFFESj6MoKCg\nQp/36NGjiJR8ONnZ2QQHBxMbG4tSqcTW1hZ3d3eUSjHPOzp06EDbtm3x8PDAwcFBbjl6cXV1RaFQ\nSJ/z/74UCgVHjx6VQ5ZepkyZUujz+fPnF5GSv4YhzLVbt24t9Hn//v2LSMlfY8KECbi7u9OgQQNG\njBhB165duXHjBj///LPc0nR4/Pgxz58/x97enn379nH16lX69u1L7dq15Zamw8GDB1m1ahUAwcHB\nzJ07F1tbWyHXBYABAwawZcsWuWX8JXJycjA1/Z+2z/yPMMT5ds2aNZQtW5Zu3boxcOBAypcvj4OD\nA+PGjZNbmhYvXrxgz549xMfHo1QqsbGx4auvvqJkyZJFqkPct08GfvvtN7Zu3UpISAjW1tZ8//33\neHt7C7VQa/Dz88PHx4dq1arJLaVAXF1dqVmzpqTx3c2lSAEcwPTp06levTotW7akUqVKcsv5IHx8\nfChbtiwuLi5kZ2cTFRXF2bNnmTt3rtzS9BISEsKpU6fYsWMH8+bNo1WrVri7u1OnTh25pUl07tyZ\n69ev88knn9CpUyecnZ0xMTGRW5Ze3NzcADh27BhKpRJnZ2dyc3M5e/as0JsLQ5hrk5OT5Zbwt3jx\n4gUdOnRgzZo1eHl50bt3b4YMGSK3LL1MmjSJqVOncunSJXbv3s24ceOYN28e69atk1uaDlu3bmXP\nnj0MHToUyNPu5eUlbFBUpUoVPD09sbOzo1ixYtL4999/L6Mq/URGRuLn50dWVhahoaEsXryYpk2b\n0rp1a7mlaXHr1i3S0tJo1aoVbdq0KfIN+9/h2LFjbN++nd9//5327dszatQoqf2NKMTFxTF27Fi+\n+uorPDw8yMjI4OrVq/To0YOff/6Zhg0bFpkWcVdNGVAqlZiamnL48GFGjx4NIGzagaWlpXATxrus\nXr2aw4cPc/fuXZydnencuTMNGjSQW1aBnD59mqNHjxIWFsbdu3dxdXXFzc2NqlWryi2tQJ48ecLC\nhQulz+7u7gwcOFBGRYVjZmZGu3btaNWqFcePH2fRokUEBwfz8ccf4+PjQ7169eSWKG0arl27Rmho\nKEuWLKFevXq4ubnRokULoQKktm3bArBx40Y2bNggjbu7u/Ovf/1LJlXvxxDmWo0ugIyMDFJTU4G8\nFKrZs2fLJeu9ZGZmcuHCBfbv38+mTZtIS0sjJSVFbll6MTExoWHDhvj7++Pt7U2TJk1QqVRyy9KL\niYkJZmZm0i2ymZmZzIoKR/T9QX6WLVvGxo0bGTt2LAADBw5k5MiRwv1/2LVrF4mJiRw+fJh169ZR\nvnx5OnbsSLt27TA3N5dbnl7UajVqtZoDBw5I81ZGRobMqrT5+eefWbJkCTY2NtKYm5sbnTp1ws/P\nj82bNxeZFmNQlI9GjRrRsWNHrKysaNiwIZs3bxYuxUuDlZUV48aNo0mTJlqbNJFSOtq0aUObNm3I\nycnhzJkzbNmyhdu3b+Ps7IybmxuNGjWSW6IWZcqUoUePHvTo0YP09HSOHTvGvHnzSE9Pp1WrVkKe\ntmZnZ/P06VMpcHvy5Ak5OTkyqyqYM2fOcODAAaKjo/nss89YvHgxNjY2xMfH891337F37165JUo0\natRIekfDw8OZO3cu6enpnDx5UmZluqSkpHD8+HEcHR1RKpVcuXKFJ0+eyC2rQAxprl2xYgV79uwh\nJSWFGjVqkJiYSJ8+feSWVSDjx49n7dq1DBs2jIoVK7Jy5UphD0pUKhWrVq3i2LFjjB8/npiYGOE2\nbBqcnJyYNGkST58+Zc2aNRw7dowWLVrILatA3N3dpdRqExMTKbVaRExNTalQoYIUcH700UdaKcwi\nUdgcT8wAACAASURBVKNGDQYPHszgwYN58uQJYWFh9O/fn+rVq/PLL7/ILU+HDh068Omnn9K5c2es\nrKxYsWKFcKnrGRkZWgGRBjs7OzIzM4tUi7GmKB9PnjyhZMmSlCtXDoBHjx6Rnp6OtbW1zMp0Wb58\nud7x/KebopGamkpYWBjr16/H3NycnTt3yi2pQLKzs/nzzz85fPgw9+/flxZE0Th79iy+vr4olUrU\najVKpZLZs2fTpEkTuaXpZcKECXTv3p3WrVvr3Lhs27ZNqPSphw8fEhISwtGjR6lQoQJubm506NCB\nMmXKyC1Nh5s3b7Jy5Uri4+PJzc2ldu3afPPNN3oXGhEwpLm2T58+7NixAy8vLzZv3sy1a9cIDg5m\n8uTJckvTy86dO+nVq5fW2IYNGxg8eLBMigrm8ePHHD58mFatWlG3bl1CQkKoVauWsO/t+fPniY6O\nxszMDHt7exo3biy3pAKZNGkS5cqVw9nZWUqtVqlUQqZWT506lSpVqhAeHs6IESMIDw+nVKlSQmrV\nEBMTQ0hICGfOnMHGxobOnTvTpk0buWW9l/T0dOFutQYOHFhgOUVhz/4XGIMiICkpiZcvX+Lj48OC\nBQuk2pecnBzGjRvH4cOHZVb4bx49esTHH3/M7du39T6vW7duESsqnFevXhEWFkZoaCgZGRl06NCB\nTp06YWFhIbc0HXJycjh16hSHDh3i5s2btGzZks6dO2Nrayu3tPeSmpqKQqGgbNmycksplLFjx7J0\n6VKtsd69e/P777/LpEiXdevW8ccff2Bubi4FQqItIhrevHmj9Vkzd2lOWUXLeTekuVZD79692b59\nOwMGDGD9+vWUKFGCvn378ttvv8ktTYtTp05x8uRJQkND6dKlizSek5PDoUOH+PPPP2VUpx8PDw/s\n7e1xdnamefPmwqcqh4WF8erVK636WFEPIjVBfH6KeoP5oWjSuzQBp4ODA126dBHOMOjKlSscOnSI\nyMhIGjRoQOfOnWnRooVWzZZo7N69my1btkjvbW5urnCGQS1atMDZ2VlnPDc3l/Pnz3P69Oki02IM\nisg7/dm9ezfh4eFaNS9KpZJmzZoJNenNnz+fKVOm4OXlpfNMNPOC4cOH8+jRI1q0aIGbmxs1atTQ\nuhIXLV2mWbNmVKhQgc8++0xKm8qvV6SC2pkzZzJr1iy+/PJLvWkGu3btkkFVwRw+fJg1a9Zw48YN\nypQpI20qcnNzadiwIYGBgfIKzEf79u2pXLmytNBpvl/NYiLSb0zjlKfRpkHEhQ8Ma67VsGHDBhQK\nBcWLFycwMJCPPvqIkiVLCmcGkJ6ezrVr15gzZ45kBgB572+jRo2EqNd7F5VKRVxcHBcvXiQ6Opqk\npCQ++eQTIWu2evToQevWrXXMjURKWc+Pp6cnS5Ys0Uqt/vbbb9m2bZvMynR5/fo1Z86c4dWrV1rj\nIq25AA0aNMDS0hIHBwe9gZCI7nNdu3Zl+fLlOu9tqVKlZFKkS1RUVKHP9QVM/yuMQVE+Tp8+TcuW\nLeWW8Y/B0Owr9+zZU2ge8xdffFGEagrnxYsXVKpUibt37+pMzsnJycLebq1bt05rw2bk/yaGNNc+\nefJE2lAkJiaSnJxMiRIlhHJMfJcnT57w8OFDmjZtSlZWltCmALdu3SI6OppLly6RlJREuXLl8Pf3\nl1uWDoMGDRLq8OZ9REVFMXPmTK3U6jlz5uDk5CS3NB369OmDhYUFVapUkcYUCoVwTnmPHj0q9PnH\nH39cREo+nJEjR7Jy5Uq5ZRTKu4d6165d4+bNm9StWxc7O7si1WIMivJx48YN/P39ycjIYMeOHQQG\nBtKsWTPhDAEgz3Xq2bNnmJiYoFAoUKlUlC9fnnLlyuHj40OrVq3klmjwvH79mvj4eCwsLKhQoYLc\ncrTIyckhKyuL4cOHs3btWunmRaVS0bdvXw4cOCCzQm22b9+Op6cn/v7+egNP0Ra/gvD19cXX11du\nGTrcvHmTBQsWGMTcBYYx1xpiqh9AYGAgoaGhvH79mv379zNv3jwqV67M8OHD5ZamQ7NmzbCxsaFf\nv364uLhQvnx5uSUVyC+//EKZMmVo0qSJlt29aCnrGq5cuYKdnZ1WanVkZCTNmzeXW5oOhtRT6dKl\nS1SqVAkLCwuio6O5ePEiVlZWuLq6yi1NL9OmTePWrVs4Ojpq1fGKtObmT+sMDAxk//79tGjRgujo\naNq2bVukc5fRfS4fc+fO1dr0tGrViunTpwuXOw7QpUsXXFxcpMK+kydPcvHiRTw9PRkzZowQQVFM\nTAxLliyhWrVqjB8/nh9++IHY2FiqV6+Or68v9vb2ckvUYteuXQQEBFC+fHlmzJjBrFmzsLS05O7d\nuwwfPpyvvvpKbokSERERbNiwgZiYGNzd3aUNm6ZXjWhoTtDq168vs5L3826djgZNfrOIzJkzx2Dm\nLjCMufbOnTvs3r2bu3fvagXCSqUSDw8P+YS9h/DwcLZv3y6lWPv4+ODp6SlkULR69Wqio6MJCQlh\n79691KxZk8aNG2vVRInCqVOnAAgNDZXGREunBbh37x4JCQkEBAQwceJEaTwnJ4d58+Zx7NgxGdXp\np127dpw4cULHTVe0msjZs2cTHx9Peno6Xbp0ISIigs8++4ygoCAiIiKEPDBr0qSJsMZLGvLfzYSF\nhbF161ZKliyJSqWif//+xqBILkxNTbVSIurWrStcoZ+G6OhoLfej1q1b88svvzBu3DhhrCwXLFjA\nhAkTSExMxNvbm8mTJ9OmTRtu3LjBjBkz2LFjh9wStdi5cyfh4eG8fPmSfv36sWPHDmrUqMGbN28Y\nMGCAUEGRq6srrq6u7Nu3j+7du2s9K8qixA9FrVZz4sQJKlasKLeU99KsWTOtNA5Aqtt5/vy5TKoK\nx5DmLjAMvU2bNqVp06Z4eHgYTKofIPX50awDb9++Fdam38nJCScnJxISErh8+TL79u3TMYoQhaLs\nlfKfkJmZydWrV0lKStIJ4ESs2QPYsWOHzjsqYk3k9evX2bZtG2/evKFTp04cPXpUSk319PSUWZ1+\njh07xrJly+SWUSgF1ZorFAqysrKKVIsxKMpHmTJl2LVrF2/evOHy5cscOXKEjz76SG5ZeqlWrRqj\nRo3CyckJpVLJ1atXKV26NGFhYcIYGJiamtKsWTMgb0HR3GpZW1sLmeNuYmJCqVKlKFWqFE5OTtL3\nWLJkSWHdZZycnPD395eaM2ZnZ3Pu3DlOnDghszJt8i/O+hDJynTSpEmkpaUxZswYnWei3hAY0twF\nhqFXY2ayaNEiAgICdJ6LZmaioVu3bgwcOJB79+4xc+ZMIiMjhetgr2HYsGE8ffqU+vXr4+LiwowZ\nM7CyspJblhajRo1ixYoVNG/eXO+B45kzZ2RQVTDW1tZYW1vTqVMnnZt5UWtLwsLC5JbwQahUKtRq\nNSVLlsTLy0vax7x580bYg4cKFSoQEBCAvb291j5GpDX3woULtGjRgtzcXN6+fUuzZs3o06cPo0eP\n5tNPPy1SLcaaonxkZGSwceNGLVvI/v37U7p0abml6ZCVlcWpU6ekviQ1a9akXbt2vHnzhtKlS2vl\nPMvFsGHD6NGjB+7u7pIxQGpqKkFBQZw+fZrVq1fLLVGLWbNmUaxYMXx8fKSx+Ph4Fi9eTLVq1Zg2\nbZqM6vTTv39/vvzySwIDAxk1ahTh4eG4u7vTtm1buaV9ENnZ2cyaNUu4fhRBQUG4ubnppG8EBATw\n7bffyqSqYPLPXcWKFcPBwYEBAwYIOXeBYcy1mjmroOJqEYuqNTx8+JCYmBjMzMywtbXVcZ4Shfv3\n76NQKLh+/TpKpRIbGxuqV68ut6wPJjs7W9gDsxMnTrBkyRJSU1OBPK3VqlUTqv2Bobmo/v777xw6\ndIgNGzZIY1FRUfj4+DB69Gjh3PKgYMMr0Yyu9JGQkFDkhyTGoCgf6enpPHv2jNq1a3P27Fni4uL4\n/PPPhUz56d+/P1u3bpVbRqEkJSWxf/9+rVPKqKgojhw5wsiRI4UzL8jNzeXixYta+bd37tzhzp07\ndOjQQUZlBePt7c3GjRu1ClWHDh0qnF2whl27drFkyRKSk5MxMzNDpVLRrl07fv75Z7mlGSSG1rdM\ngyHNtdevX2f58uXcvXsXhUJBnTp1GDVqlJAW15BXX5qamkrXrl2ZOnUqd+7c4euvvxZyDvv11185\ndOgQTk5OZGVlERMTQ69evYS1uX6Xrl27EhISIrcMvfTq1YvFixczefJkli9fzuHDhylZsqROurWc\nGOLBQ2ZmJiVKlJA+p6SkoFKphLvp1pCYmKh3XJSMItGQ/zpBIMaPH8+wYcNQqVT8+OOPeHt7M2XK\nFOFuNAAsLS2ZOHEidnZ2WidVIi0mFStW1EnbcHZ2FtIIAPLyV98tSKxduza1a9eWSdH7UavVnD17\nlvLly7Njxw5q1qxZ4CQoAjt27CA8PJyvv/6azZs3c/ToUe7duye3LINl06ZNTJkyhVmzZuntUyRa\nEbgGQ5prp0yZwtixY3F0dCQ3N5fo6GgmTZpEUFCQ3NL0smzZMtatW0d4eDgmJiZs2bKFIUOGCBkU\nHT16lJ07d0rF9Tk5OQwYMECodawwRD5TLlGiBBYWFqjVaipUqICnpycDBw4UKihat25doTXQIjmk\nacgfEAFCOyYCjBkzRvqOs7OzefDgAY0aNTKYGrmixhgU5SMrKwsXFxeWLl3KoEGD8PDwYM+ePXLL\n0ouFhQWQd+Jq5P8uP/30E8+fP2fatGksWbKE48ePaxlwiIapqSnFixcnOzsbtVpN+/bt8fLyYsiQ\nIXJLM0i+/PJLwHCKwDUY0lxbvnx52rVrJ31u3749O3fulFFR4ZiZmWFubk54eDh9+vTB1NRUMl8Q\nkfwGG0qlUhijoA9BZK1Vq1YlKCgIGxsbvvvuOywsLKTaU1EozI1U5O/WkNi9e7fW5+fPn7NkyRKZ\n1IiPMSjKR1ZWFvv37+fgwYPs3r2bhw8f6nRYFoXRo0eTkZEh5QtnZWUJ2QXcyP+OrKwssrOzadCg\nAaamplKOcHx8vMzKCsbBwYEtW7bQqlUrvL29qVatGpmZmXLLKpDr16+Tnp6udSKsMQ8Rge+//x4H\nBwfGjx8vXDpqYRjSXFunTh18fX1p2bIlarWa8+fPU6VKFcnMRKSCZYBKlSoxaNAgXr9+jZOTE/v3\n7xfO2lhDly5d6NmzJ46OjqjVai5fvkzv3r3llqVFQb3VcnNzSUpKkkHRh+Hv709qairdunUjODiY\nlJQUfvnlF7ll6WAMfoqWypUrc/36dbll6EWEfnvGmqJ8xMXFsXv3btq3b0+LFi3YunUrNWvWpHXr\n1nJL02HFihXs2bOHlJQUatSoQWJiIn369BHyuvnJkyeEhYXx6tUrrc2laPagY8eOLXSCFul0JTw8\nXGrK+PLlSwICAqhfvz7Lli3j+PHjHDp0SG6JBZKVlYWZmRnnzp0jJSWFFi1aYG5uLrcsHYYPH05q\naipVq1aVxhQKhVDvQW5uLrt27WLDhg307NkTb29vYQu/82NIc21BhcoaRCtYzsnJ4ebNm9SuXZsS\nJUoQFxfHxx9/TNmyZeWWppeHDx8SFxeHQqGgYcOGwtWR7N27t9DnX3zxRREp+TDeV2ssUmqiv78/\nkPcO3Lt3DycnJ9RqNdHR0dSrV0+v66MILFq0iN27d0v7GU26smhOhICWiYUmkG/evLlw8xaAl5eX\n1L9u8+bN3L59u8j71xlvivJhaWlJv379qF27NlFRUWRnZwvVYT0/ERERHD16FC8vLzZv3sy1a9cI\nDg6WW5ZeRowYQevWrbU2lyIyYMAAuSV8MGvWrCEoKIhy5cpx584dxo4dS25uLt27d2ffvn1yyysQ\nfRvMY8eOCTlBp6WlCddL610UCgW9evXCw8ODadOm0b59e6pUqSIt0qK5N2kwpLn23XdTVMdEDZGR\nkaSmpmJjY4OPj4+QRgsF3b5cvHgREKuWRLSg530kJyfLLeGD0aR6Dx8+nD179kiuudnZ2YwfP15O\naYUSERHB8ePHKV68uNxS3suiRYukgzKFQoG5uTlqtVpmVfoRoX+dMSjKR/7iX39/f6GLf3Nzc1Gr\n1ahUKjIzM2nUqJGwi3TZsmWFtDF+F40BRFpaGhs3biQuLg6lUomtra3UHV4UihcvTrly5YA8Mwgz\nMzNWr15N5cqVZVZWOG5ubtLfOTk5XLhwQdibDScnJ27duiWsy5iGZ8+esXjxYh48eMCPP/6IpaWl\n3JLeiyHNtQU5JoqKxmjhyJEjwhotFFZLYuQ/o3PnznJL+Ms8fvyYV69eSSnAb9++5eHDhzKrKpiW\nLVty8+ZNGjVqJFzTaQ05OTlkZWUxbdo01q5dK91qqVQq+vfvz4EDB2RWqIsI/euMQVE+DKn4t0uX\nLmzatAkPDw+6d+/ORx99RKlSpeSWpYXGJtjOzo6tW7fSpEkTrf5JotoFT548GWdnZ0aNGkV2djZR\nUVFMmTKFpUuXyi1N4t1TVnNzc+EDIkCnf1KHDh0YNmyYPGLeQ3h4OBs2bMDc3FxyxxItRWLx4sWE\nhYUxcuRIIW/bCsKQ5lpDc0w0BKOFffv2ERgYyPDhw1mzZo3ccv5RaJwo9VVGiOpI+fXXX9OzZ08p\njTojI4NRo0bJrKpglEqlVl81EdPnIiIi2LBhAzExMXTt2lUaVyqVwjoAz58/n40bN1KhQgXWrFmD\ng4NDka9rxqAoH4ZU/Dt48GDp7zZt2pCcnIyNjY2MinSZNWuW9Pfly5cJDQ2VPos6OUPehJz/+3V0\ndBSuI/zTp0+1csff/SxS3nh+NMXpGp49e8aDBw9kUlM4htBlvUSJEgQFBRlEGkd+DGmuNTTHxEqV\nKjF48GAyMjKENVooUaIEzs7OvH79mhYtWkjjIm4u352z3kU0ow1Dc6IE6N69O927d5dS/8qXLy+0\nAUNERARRUVE69twi4erqiqurK/v27RPKhr0wSpUqhaurK87OzqjVahQKBbGxsUVqbmQMivIxc+ZM\ndu/eja+vL+bm5uzbt48JEybILUuL5cuXF/js+PHjQpkXaCbnmJgY7O3ttZ5FRkbKIemDUKvVXLly\nBTs7OyAvoBMtB9fDw0Mrd/zdz6KSPzCGvBuun376SSY1hRMXF4efnx/3799HpVJRv359pk6dqpXz\nLDcjRoyQW8LfwhDmWg2G5pi4cOFCyWgBoF69esybN09mVdpoXND8/f2FbiEAunPWu4gWFI0aNYoV\nK1bQvHlzvb3LRAo4NezevZstW7bomDEdPXpURlUF07JlS548eUKtWrXklvJenJ2dmT59OqmpqSxd\nupSDBw/i6OgonKEJ5DWjV6vVWk28FQpFkQZFRve5AkhPTyc0NJRDhw6xbt06ueVItGrVilKlStGq\nVStp054fkYpC7927x927d1m0aBETJ06UxnNycpg3bx7Hjh2TUV3B3LhxAz8/P8naWsTNsKGSnJxM\nQkICAFZWVkLbSPfv358pU6Zga2sLwKVLlwgICBD2htNQEXWuzU9+x8Tk5GRatmwppGNifpKTkwkN\nDSU4OJiXL1++d3MvB48fP+bIkSPCO5PqQ3TDDUOia9euLF++nGrVqmmNi1YSoKFjx448fPhQ6NRq\nDUOHDmXgwIH8+uuvbNmyhTNnzrBy5UohbxT79+//XvfE/zXGm6J8ZGZmcuzYMQ4cOEBkZCRdu3YV\n7iT2zz//5Pz58xw+fJjt27fj6OiIm5sbTk5OckvTITMzkytXrpCUlKSTOifiohcYGEjXrl2xtrZm\n48aNcsv5R5GVlcWMGTO4dOkS9evXR61WEx8fj6OjI7NmzcLMzIzDhw9rGTHIjYmJiRQQQV4apcgp\nHe/y9OlTYR0fDWGuzcrKYuXKlYwaNQozMzMgL63nzJkzdOrUSWZ1+klPTycsLIyDBw9y8+ZNcnJy\nWLZsGU2bNpVbml5GjhxpEM6kYHiGG4cOHSI4OJgVK1YAMGTIEHr37i2kEUOtWrWkm01D4MiRI3JL\n+GDUajVt2rRh7dq1ALRo0UJ6J0Tjiy++YP369TRs2FCr/tyYPlfEHD16lJCQEKKjo2nRogW9e/fm\n4cOHwqUcwL+vEjUvyYULFwgLC+Onn36iYcOGTJ8+XWaF/8ba2hpra2sAmjdvjoODg7BOYwBJSUl4\neXlRrVo1unXrRpcuXYQ/DTYUAgICsLa2ZsGCBVrjGzZswM/PD19fX1avXi1UUFS2bFnWrl0rFaVG\nRkZKjn+iIvrtgCHNtT/++COA1g3GJ598Qnp6OsuXLxfuYGfUqFHcu3ePVq1aMWLECJycnOjZs6ew\nAREYjjMpGJ7hRmBgoLQRBli1ahXe3t5CBkUVK1akT58+ODo6SjcvIJY1e368vLz0HpCJmEVgamrK\nmTNnUKvVvHjxgiNHjghbgxoUFIRKpeLSpUvSWFGnzxmDIvKadnbu3JmNGzdKdrYiOY0VxPPnz4mN\njSU2NpYSJUoIe9JiZWXFoUOH+OmnnyhdujTNmjXDxcWFxo0byy1Ni2+//ZZvv/2Wa9eucejQIXr3\n7o2VlRXdunWjffv20mmxSIwZMwYPDw/atm0rpD4N586dY/fu3TrjgwcP5vPPP8fT05MmTZrIoKxg\nFixYwMaNG1m1ahUKhQJ7e3shHd4M6XbAkOba6OhonXfWzMyMH374gf79+wsXFL19+xZzc3PpP0ql\nUtibTUN0JjU0ww2VSqW1+VWr1Xod6USgSZMmws3/hTFjxgzpb01rCVGNYubNmyfdcH799dfCrmOQ\n944WZaNWfRhrishLMwkJCSE4OJjc3Fzat2/PwYMHCQkJkVuaDprT37CwMJRKJZ06daJTp05F7uX+\nd8jMzOTMmTNs2bKFc+fOERMTI7ek93L16lVWr17N6dOnuXDhgtxydLhw4QLh4eFERkbSsGFDPDw8\ntNycRKFXr17s3LlT77Pu3bvj6+srTJD86NEjPv74Y2nj9i4ibdjy3w506NBBuh0ICgqSW5peDGmu\n7d27N7///rveZ1999ZWQjXGTkpKk7/fFixe8efOGjRs3CvXOAoX2fRPVmXTBggVYWFiQkpLC2bNn\nqVatGnfv3i1wXpOboKAgVq1aRe3atVGr1dy9e5exY8fi7u4utzS9REdHk5iYiLu7O8+ePaNKlSpy\nS/pLDB06VMiaSI3BRn6SkpK0zAxEYfny5VStWhU7OzvZDkmMQdE7JCQkEBwcTHBwMObm5vTs2VMo\ne2MbGxssLCz49NNP9QZCop1eAsydO5fHjx9TokQJbG1tsbe3x9bWVtgrXIArV64QEhLC8ePHsba2\nxsPDQ6jmh/o4c+YMvr6+vHnzht69ezNkyBBhClW/+eYbBg8ejIuLi9Z4REQEq1evlr24Mj/z589n\nypQpWikSmmlStA3b119/TXp6uhQUNWjQgC+++IK9e/fKLe29iD7X/utf/2LYsGE6N25//PEH27dv\nlxzUROXhw4cEBwdz8OBBihcvLmQQlx+VSkV6errQKar5DTdSUlJo0aKF0CnWGRkZxMfHY2pqSu3a\ntYW1kPb39+fx48fcv3+fPXv2sGzZMlJTU5k2bZrc0vTy7nr1/Plzjh49KlRD1AsXLuDj40NGRgZV\nqlRh0aJFWFlZsXXrVtavXy+ks5++w5KiXnONQVEhxMTEcPDgQaZMmSK3FImoqKhCn4vYlGvmzJm8\nePECc3NzHB0dady4MdbW1sKldsTGxhISEsKRI0ewtLSkW7dudOzYUWrQJiKvX7/myJEjHDx4kKSk\nJKkWKiIigoMHDxIYGCi3RCBvgzZmzBhq1apFw4YNJdvzO3fusHbtWiHtQSEvJal48eKkpKSQmJhI\nw4YNhXtvDeV2oDBEnGvv3bvHmDFjqFOnDg0bNkSlUnH58mUeP37MunXrqFSpktwSP5jr16/ToEED\nuWXosGbNGsqWLYuHhwdeXl6UL18eR0dHxo4dK7c0HeLi4ggKCtJxyhM1FSkoKIjs7Gy6d+/OiBEj\nSElJ4auvvqJv375yS9PBy8uLzZs3S/8N0K9fP7Zt2yazMv282xrF3NwcNzc3qlevLpMiXTw9PVm4\ncCGWlpacO3eORYsWoVKpsLGxYdy4cULeFImAMSgyUmS8evWKs2fPsnXrVmJiYoRLR/P09MTDw4Mu\nXboYzITRsWNHOnbsSPfu3SVTCw0+Pj74+fnJpEyX3NxcTp06xZ07d1Cr1dSuXZtWrVqhVCrllqaX\nOXPmYGtrS5s2bfD29pbc52bPni23tAJ5+PAhBw4cICQkxCBuB0RHrVZL76xCoaB27dp8+umnwgXG\nhoqnpyfbt2/n999/5/nz54waNYpBgwYJc5iTH3d3d8mIJz9t27aVR9B76NOnD1u3biUkJITr16/z\n/fff4+3tLaSzat++fdm0aRNDhw5l06ZNJCUlMWzYML11qCKwcuVKRo4cqTW2YMECfvjhB5kU6ZI/\nwIS8XoYLFy4U8nBEQ/7eWjk5OWRkZGBhYVGkjdSNRgtG/uccOnSIy5cvc+3aNZRKJQ4ODkIWp27f\nvl1uCX+Z7t2766RMaiZnkQIiyLsGb9WqFa1atZJbygdx/fp1pk+fzsaNG/nyyy8ZNGgQgwcPlluW\nXlQqFYmJiZiYmDBixAhGjBjB9evX5ZZl8CiVSlq3bk3r1q3llvKPRK1Wo1arOXDggHTYkJGRIbMq\n/VSrVg1PT0+5ZXwwSqUSU1NTQkNDGTNmDJB38y0iQ4YMoU+fPiQmJvL1119z584dfHx85JalQ1hY\nGMHBwZw/f54bN25I4zk5OcTFxQkVFL17cFOhQgWhAyLIc3jNz/Xr19m/f3+RajAGRXpIS0tDqVQK\nnStsSNy4cYM2bdowfvx4YXOaDY38k/OtW7ekcREnZ0MlKyuLp0+fsn//flasWEFOTg5paWlyy9JC\nrVazePFi9u/fT/Xq1cnIyODVq1d4e3szaNAgueW9F+Nc+78hKiqK2NhYlEoltra2QvaxA+jQPhLP\n3gAAIABJREFUoQOffvopnTt3xsrKihUrVuDg4CC3LL3Y2tri7+9P06ZNtYrA27RpI6OqgmnUqBEd\nO3bEysqKhg0bsnnzZmrUqCG3LL1Ur16dLVu2cPv2bYoVK4aVlZWQe4VOnTphY2PDnDlztOoflUql\ncO6/ycnJnDhxQvqckpKi9VnU9zY/DRo0YNasWUX6v2lMn8vHqVOnmD17NsWLFycrKwsTExNmzZol\npLVtTk4OoaGhPH36lKFDh3Lz5k2srKyE7AMUGxvL/PnzuX//PiqVivr16zN16lTq1KkjtzSD5uHD\nh8yZM4ehQ4dKY5rJ2VDS/0QmKCiI9evX061bN4YPH87ixYspXbo0w4cPl1uaxPLly3n27BlTpkyh\nZMmSAKSmpjJ//nyqVq3KhAkTZFaoH0Oba/NvgiFvg1G+fHmZFBWOn58f9+/fx8XFhezsbKKiomjU\nqJGw70J+0tPTOXLkCF988YXcUnQoqN5N1JoiyJsLNMYVjx49okqVKkLuEQYOHMj69et1fmeikpGR\nQWRkpI4Nd48ePWRSpMv76jNFfG/Hjh2rdcP17NkzSpUqVaSufsagKB+enp4sXbpUsoJ8/PgxEydO\nFLLYb8qUKVSsWJGoqCh27tzJli1buHjxIgEBAXJL06F///5MmTIFW1tbAC5dukRAQIBQLl6GxuXL\nl3FwcOCPP/7QW98g6inQyZMnSU1Nxd3dHR8fH+7cucPQoUPp2LGj3NJ02LlzJ7169ZI+67M2lZt+\n/fqxadMmnc1ETk4OPXv2LPLUgw/FEObanJwcsrKyGD58OGvXrpWK63NycujXr59QTlP56d+/v447\n1oABA9iyZYtMigrmypUr/Prrr6SkpACQnZ0tNZgUjUePHun8/k1MTKhcubJwdZGRkZEEBgaSkJCA\niYkJdevWxdvbW9heQN988w03b96kQYMGWkHbkiVLZFRVMH369MHCwkLLNlyhUAjbbNZQyG8kplAo\nMDc3p0GDBkW67hpGWF5EFCtWTOslr169urAnF48fP2b+/PmSheGAAQOE616vwcTERAqIAKlg3cjf\n5+zZszg4OHD48GG9z0UNipYtW8a6des4cuQIJiYmbNmyhSFDhggZFEVGRuLk5CTdaIr4zpqamuqd\no0xNTYW2NjaEuTYiIoINGzYQExODu7u7FBQplUohXT415OTkkJmZKaUfvX79GpVKJbMq/cydO5dv\nv/2WhQsX4uvry5EjR7C3t5dbll4mTpzI1atXJafMxMRE6tatS0pKCuPGjRPmliAsLIzAwEC+/fZb\nqYYkNjaWRYsW0bdvXzw8PGRW+G/Gjh3L0qVLhawxLoxixYqxaNEiuWX8YyisJcfFixeLtFWDWKuQ\nzFhYWDBr1iycnZ3Jzc3l7Nmz1KxZU25ZesnOziYtLU3aqMXHx5OVlSWzKv2ULVuWtWvXShuJyMhI\noTdshoAmhUtTQGsomJmZYW5uTnh4OH369MHU1FTYDduVK1fw8PCgZMmSmJmZSTdFZ86ckVuaxJs3\nb4iPj9fbqT4zM1MGRR+GIcy1rq6uuLq6sm/fPrp37y63nA/G29ubzz//nFq1aqFWq7l//76wJ9jF\nixfHxcUFMzMzbG1tsbW1ZfDgwbRv315uaTrUqlWL2bNnU79+fSBvzd20aRM//PAD3t7ewgRFa9as\nYePGjVqtJJydnVmzZg2DBg0SKijS3BCKfMigj3bt2nHixAmaNGmCiYmJNK5JYTby10hOTpZbgoQx\nKMrHnDlzCA4O5sKFCygUCpo0aUK3bt3klqWXCRMm4O3tzd27d+ncuTMKhYK5c+fKLUsvCxYsYOPG\njaxatQqFQoGdnZ2Q+ayGyJgxY6TAODs7mwcPHtCoUSMtK06RqFSpEoMGDeL169c4OTmxf/9+YReS\norQB/buUKFECX1/fAp+JiiHNtbm5uezcuZMePXrwzTffCN3vBcDJyYmgoCDu3r2LQqGgVq1aJCYm\nyi1LLyVKlCA8PBwLCwsCAgKwtLTk6dOncsvSy+3bt6WACKBOnTrExcVRsmRJoQ52TE1N9fbWMzc3\nF+429v79+/z4448FPhc1mN+xYwc5OTlaYwqFQsiGqFFRUTpBp6YnlCjkd9A9e/YscXFxspnEiPUL\nkRk/Pz9mzJihdeIzfvx4fv75ZxlV6adp06bs3buXly9fYmZmRpkyZeSWVCABAQHMmDFDbhn/SN7t\n4/D8+XNh87ABFi5cyM2bN6WUtHr16rF48WKZVennyZMnrFixgtTUVJYuXcrBgwdxdHQUqtHsjz/+\nKFTDwA/FkOba3377Ter3Ur9+fSZPnoy3t7dwQVFSUhIvX77Ex8eHBQsWYGZmBuQ1oR03blyBqbZy\nEhAQwMuXL2nevDmBgYHcuHEDf39/uWXpxdHRkZ49e+Lo6IhSqeTq1avUrl2boKAgGjduLLc8iezs\nbF69eqWzJ0hKShIum6RkyZLUq1dPbhl/GUM4MNOwatUq7t27R69evbh37x5Tp04VtrG3n58fDx48\nwNnZmczMTFauXFnkJjHGoAg4fPgwGzZs4NatW8TExEjjOTk5OqcBorBt2zZ27typ011bxJMKpVLJ\njh07sLe31yqiFPWHachUrlxZ6P4006dP1xkzMTGhZs2aeHp6UrZsWRlU6Wfq1KkMHDiQX3/9FYCK\nFSvyww8/CHULN3nyZIMyLDHEuVbT7+Xw4cPSiaaI/V7u3LnD7t27uXv3LrNmzdKqgRIpZUrDmTNn\naNGihWTH3r9/f65fv46dnZ3MyvQzbdo0bt68KaWr9ujRA1tbW7KysoRJnQMYNGgQQ4YMYcyYMdjY\n2KBSqbhy5QorVqxg/PjxcsvTolKlSkI6Db6PmzdvsmDBAjIyMtixYweBgYE0a9aMRo0ayS1Nh19/\n/ZX58+czcuRIHjx4wLRp03BxcZFbll6uXbumVV80fPhwBgwYUKQajEER4ObmRrt27ViwYIGOvXHl\nypVlVFYw27ZtY9WqVXz00UdyS3kvN27c4MaNGwQHB0tjCoXCoDZzovLll19K6XO5ubm8fPmSFi1a\nyKyqYCpWrMijR49wdXVFoVAQEREhWRtPnDhRCkBEQK1W06ZNG9auXQtAixYtWLFihcyqDBtDnGsN\npd9L06ZNadq0KR4eHrRs2VJuOYWybds29u3bh52dnRQUvXnzhuXLl/Pq1Ss6deoks8J/s337djw9\nPfH399cyW7l69SohISHCpXh5eHhgYWHB5s2bCQgIQKFQULt2bXx9fYXrAZXfgMmQmDNnDr6+vlLq\ncqtWrZg+fTq//fabvMLykb8nUevWrdm7dy9WVlZkZmZy4sQJIc2YRDCJMQZF/x8zMzODSvGys7Oj\nRIkSlCpVSm4p70Wkk/V/GkuXLpX+1lhYinTb8i5Xr15l48aN0mcPDw++/vpr1q5dS0REhIzKdDE1\nNeXMmTOo1WrJJrh48eJyy9Li6tWrfPXVVzrjGlOIXbt2yaCqcAxtrp02bRpjxoyRzGFcXV3x9PSU\nWVXBiB4QQV7a7+bNm7XWrxo1arB69WqGDx8uVFCkSZfNX08kOo0bNxYqpa8gJk+eLLeEv4WpqalW\nn8W6desKZ8v+rhux5remGRcxKBLBJMYYFBko1tbWtGvXjkqVKmFiYiJtgkRKn8vMzGTVqlVSPmjP\nnj15/PgxxYsXZ/369cJ1gDZEli9fXuhz0Qwt0tLSOHr0KI0bN5by8p8+fcrNmzeFc0ubN28eS5Ys\nITk5maFDh+Lg4CDc91mvXj0he5P9k3i3tuzSpUvC1ZYZGmZmZnoP9MzNzVGr1TIoKpjWrVsTGxsr\npXndvHmTI0eOYGlpyeeffy6zOiNyUKZMGXbt2sWbN2+4fPkyR44cES5rJ/9alZWVxbNnz7CwsJBR\nUcHExsZiY2ND165dadu2rZZJTFEbMRmDIgNl+/btHDx4UNiUE8j7UZqamqJWq1EqlZQuXZozZ85w\n6tQpFi9ezLJly+SWaPBonI9cXFwwMTHh9OnT5Obm4urqKrc0vSxYsIAVK1YQEBBAbm4uNWvWZO7c\nubx584Y5c+bILU+LDRs20KtXL+bNmye3lAIxMzMzqM35kydPqFatmtwy/hKGUFuWH5VKRUpKCh99\n9BEJCQnEx8fTunVroW45c3NzefbsmVavKsgzhRAtKPrpp5+Ij49n1apVPH/+HC8vL7y8vDh37hxx\ncXEGe9th5O8zf/58Nm7cSIUKFVizZo2QB2YaQkJCWLlyJQDBwcHMnTsXW1tboergZs+ezatXr+ja\ntSseHh7Y2NjIpsUYFJHXQKywxowiunk1btyYChUqCJ0+d/XqVR13NIBPP/3UGBD9l0hISGDDhg3S\nZzc3NwYPHkzbtm3lE1UI1tbW+Pn5kZaWJt1uAkLWaFhbW7Nu3Tpu375Nq1atcHNzo2nTpnLL0kJf\n6pzIfP/99wZXS2hotWXfffcd7u7uNGjQgLFjx9K1a1eCg4OFcvYbOXIkgwYNwsvLi4YNG6JSqYiJ\niWHbtm389NNPcsvT4syZM9I6duDAAdq0aSMZbhRlU8m/w5MnT3j48CFNmzYlKytLciQ08p9RqlQp\nXF1dcXZ2Rq1Wo1AoiI2NpVmzZnJL02HLli3s2bNHquGcNGkSXl5eQgVF27dv5/Hjx4SGhjJp0iRy\nc3Nxd3fH3d2dSpUqFakWY1AEhbpbvHjxogiVfDj379+nXbt21KxZUyt9TqQagvxOc4C0qQD0Nps0\n8tdJTk7mxIkTODg4oFAouHTpEklJSXLLKpBp06YREREhnRCL+N5q6NGjBz169CArK4vTp0+zfft2\nvvvuO/744w+5pUmUL19eq6D2XUTMGzc0DKG2LD8vXrygQ4cOrFmzBi8vL3r37s2QIUPklqXFZ599\nRp06ddi+fTsnT56UzAA2bdoknMV8/oPHU6dOaR1E5G/cKRqBgYGEhoby+vVr9u/fz8KFC6lcubLU\n+NvI38fb2xu1Wk3FihWlMYVCIWRQZGJigpmZmXQAKWpgXL16dQYPHszgwYNJTEzk8OHDfPvtt5ia\nmrJ+/foi02EMivh3N+WcnBxOnjwpdVnOzs5m9erVdO3aVU55eims4ZkolChRgoSEBKysrACkjcSV\nK1eE7qtkSPj7+7Ny5Up++ukncnNzqV27trDX+JCXO3zixIlCb2ZFIj4+nmPHjnH8+HEUCoVQDe9A\nt5j2XUQLigzRGMIQasvyk5mZyYULF9i/fz+bNm0iLS1NWtNE4uOPP2bixIlyy3gvSqWSa9eukZaW\nxpUrV6TMkWfPngnX9yc/4eHhbN++XZqzfHx88PT0NAZF/wVUKpWWdbTIODk5MWnSJJ4+fcqaNWs4\nduyY0GYsOTk53Lx5k+vXr5OcnEzz5s2L9H/fGBTlY/z48Zibm3P27FlcXV2JjIzU6rQrGsuWLdPq\n/DtmzBi5JWkxceJERo4ciZubG/Xr15dSJCIiIrRujYz8ddLS0ihbtizW1tY66Z1XrlyRSdX7adCg\nAcnJyVonbKLi5uZGjRo16NChAz///LNO/YMIiLw514chGkMYQm1ZfsaPH8/atWsZPnw4FStWZOXK\nlQwcOFBuWQbL1KlTmTt3Lunp6cyfPx9zc3Pevn1L7969mT17ttzyCkRjZaw5gHr79q2wvcAMjS++\n+IL169fTsGFDTE3/vY0W8aZowoQJnD9/nvr161OsWDEmT54snDOhSqXi9OnThISEEBUVhYuLC927\nd2f+/PlF7uqnyDXmMUkMHDiQTZs24eXlxebNm0lLS2PatGlatseiMGjQIPr27YuLiwvZ2dlERUUR\nFBQkVJ8XgPT0dIKDg7lz5w5KpZK6devSrVs3yYfeyN9D865qmDlzJrNmzdL7TCS8vLy4du0an3zy\nibBpnxpSUlIoWbIkz58/F9a1JykpiV27dlG1alU6d+7M9OnTOX/+PFZWVkyfPp1atWrJLVELzdxq\nSAQFBXH06FGha8vyM2XKFJo3b46Li4vBmVoYEunp6VKPJRHZunUrhw8f5t69e7Rt25bIyEhp32Dk\nP2PAgAGoVCotoyuFQiFk/XlOTg6HDh3i2bNnDB06lJs3b2JlZaVT3iAnn332GY6OjnTr1o22bdvK\nmuJnvCnKx9u3b3nw4AEmJiYkJCRQvXp17t+/L7csvahUKtzc3KTP7u7u/P777zIq0o+5ubnQPT0M\nlXfPMu7cuVPgM5FYsGCB3BI+mNOnTwvv2jNp0iScnJy4fPkyv/32G3379mX27NlcunSJmTNnavWE\nEgFDM4YAw6gty0+fPn24ePEic+fO5cWLF9SrVw8XFxe6desmtzQdsrOzCQ4OJjY2Vsp4cHd3F67n\niz5EDoggzwSiTZs2xMTEYGZmxogRI4xB8n8JtVotVKPWwpg+fToVK1YkKiqKoUOHEhUVxS+//CLU\njX1ISIgwvydjUJSP8ePHExsby8iRIxk2bBjp6enCusuYmZlx6NAhXFxcyM3NJTIyUtgCOiP/fQqr\nyRGxXkfTFX7Lli169YnWFR4Mw7UnKyuLUaNGAXmNcLt37w5A8+bNhXRIM1RjCNFry/Lj6OiIo6Mj\nrq6uREdHExwcTEBAgJBBkY+PD2XLltXKeDh79ixz586VW5rBExUVxYEDB6RWB6NHj8bb21vIFC9D\no2XLluzcuRM7Ozut9Lm6devKqEo/jx8/Zv78+dKcNWDAgPfWohY1ogREYAyKtMjMzJRuX8LDw4G8\nE2IR8fPzY8mSJaxatQqlUomdnZ3B5Lwb+e8jYiCUn8K6wouq3RBce/J/dxUqVCjwmSgYmjEEGEZt\nWX6++eYbAGrXro2joyN+fn5UrVpVZlX6efLkCQsXLpQ+u7u7C1v/pFKpSExMxMTERMgWAu8SEBCg\nZcjk6+vL6NGj2b59u4yq/hmcPXsWgP3790tjCoVCyLT17Oxs0tLSpPUgPj5eaIMQuTEGRUBMTAxX\nrlxh06ZNJCYmSuMqlYq1a9cKecJWpUoVPD09sbe3B/J6KYjayHXs2LE6dVm9e/cWMt3PUMjv4pWb\nm0tCQgJfffUVubm53L17V15xemjdujWQt8E8c+YMr169klnR+3nXtef48ePCufbcv3+fH3/8kdzc\nXOlvyHsnHjx4ILM6XQzNGAJgx44dmJqa8urVK3JycqQ1QtSNsaOjI7GxsSQkJKBUKlEqlRQrVkxI\nc5Ps7GyePn0qBW1PnjwRzgxArVazePFi9u/fT/Xq1cnIyODVq1d4e3szaNAgIQ8fIG//UrNmTemz\niP/+hsrmzZvJyMjg3r17KJVKatWqJWyd9IQJE/D29ubu3bt06dIFQNgDdA8PD+zt7XF2dqZ58+ay\nHOYYjRbIu16MjIxk2bJl9OzZUxpXKBQ0btxYuI0Q5KUbValShe+++w6ApUuX8ujRI/z9/WVW9m8O\nHz7MmjVruHHjBmXKlJGK6tVqNQ0bNiQwMFBuiQbLo0ePCn2uuZkRjT59+mBhYaF12q5QKIRMnwM4\nf/480dHRFCtWDAcHB+Fce/bu3Vvo8y+++KKIlHwYhmYMAXk5+SdOnKBKlSpSvZ6o5iDvcuLECQID\nAzl79iyxsbFyy9EhKiqKmTNnolQqUavVKJVK5syZg5OTk9zSJJYvX86zZ8+YMmUKJUuWBCA1NZX5\n8+dTtWpVJkyYILNC/fz6668cOXIEe3t71Go10dHRdO/enUGDBsktzeDZv38/y5cvp06dOmRlZfHw\n4UO+++47OnbsKLc0iVOnTtGkSRMpWHv58iXFihWjbNmyMisrGJVKRVxcHBcvXiQ6OpqkpCQ++eST\nInV5NAZFIHV6fvr0qd4XRjMRisSAAQPYsmWL1piozk7r1q2T6jKM/N9G33srIhcuXODevXvY2NjQ\noEEDaXznzp306tVLRmXaaKzZ9XHlyhXs7OyKWFHhDB06FCcnJ16+fElsbCx9+/bFzc2NS5cusWrV\nKuGMIQB69uzJ7t27hb0ReJdff/2Vy5cv8+TJE2rVqkWzZs1o1qwZtWvXlluaDpp3NDU1FYVCQdmy\nZYmMjCzy3iSF0a9fPzZt2qRVOwJ5rl49e/bUSqESjXv37hEbG4upqSk2NjbCHpYZGn369CEwMFDa\nG2ZkZDB06FChUhNHjx7N5cuXsbS0xMXFBRcXF5ycnIRMA8/PrVu3iI6OlhrRlytXrkgP+43pc+RZ\nmC5atIi+fftqLXyam42jR4/KqE4/CoWCP/74g8aNG6NWq4mMjNSZtEWhd+/erF69mpcvX+Lj40Nk\nZCQ2NjZCn1gY+d/Qrl07Tpw4QZMmTbS6wYt08LBs2TIuXLiAra0tmzZtYtCgQdSvX59Zs2ZhaWkp\nVFA0evToAq3ZFy5cKFyOu6EZQ4Bh9daCPDOL77//nnLlymFiYiJUEbOGe/fukZCQQEBAgFYD15yc\nHObNm8exY8dkVKeNqamp3rXV1NSUcuXKyaCocDSmNv7+/lr7mejoaEBMUxtDQ6lUaq1ZpUuXFm7/\ntXz5ciCvhuj8+fPs3r0bX19fKleuTPPmzaV5WCSaNWuGjY0N/fr14/vvv6d8+fJFrkGsf0WZWLRo\nEQDHjh0jNzeX5ORkIK9wWdTTQX9/fxYvXszChQsxMTHBzs5O2Hz9KVOm0LJlS44fPw7kpdBMnDhR\nuJ5KRv737NixQ6dmQLSDhz///FOqdxsxYgRubm5YWFjwww8/CJc+Z2jW7IZmDAHw4MEDOnToYBC9\ntSCv1mnYsGEUL16crKwsTExMmDVrllC9lTIzM7l69SpJSUla5hsKhUK4hulv3rwhPj5e7+8pMzNT\nBkWFU5ipjZH/Dk5OTvzrX/+iWbNm5ObmEhUVJdTvKz916tTB0tKSWrVqUadOHf744w+Cg4OFDIpW\nr15NdHQ0ISEh7N27l5o1a9K4cWOpFqooMAZF+di7dy8///yzdIORkZHBhAkT8PDwkFmZLjVq1GDm\nzJmkpaVp1eqISEZGBv369ePQoUMAdO3a1WA8/kXn0aNHLFu2jLi4OKnPx5gxY4R1yJoyZQrt2rWT\nW0ahFC9eXPq7dOnSfPLJJ2zdulVGRQVjaNbshmYMAYbVWwvybjo3b94szQGPHz9m4sSJbNu2TWZl\n/8ba2hpra2s6deok/Oa9RIkS+Pr6FvhMNDSmNseOHWPZsmUyq/lnMmnSJM6fP8/Vq1eBPMfHJk2a\nyKxKm4iICKkmVq1WY29vj5OTE7179xb21tvJyQknJycSEhK4fPky+/btIzQ01BgUyUVgYCBBQUHS\nCWZSUhKDBw8WMiiaNm0aERER0sIn8umlWq3m/v370iYtIiJC2ADO0Jg6dSp9+/blhx9+kPp8TJ06\nVdhbuN9++40mTZoInTr5bjCRP81PdEQMhPIzbtw46e93N8OibY4L66cE4pqZFCtWTOtQpHr16sKl\n9mgQ7d9cHyLW6X4IFSpUICAgAHt7e4oVKyaNi2h7byicO3dO63OjRo2AvD3OuXPnhOoB5e/vz5s3\nb/j888/59NNPcXBwEL6eaNiwYTx9+pT69evj4uLCjBkzsLKyKlINYs6UMlGtWjWtHMYKFSpoWVqK\nRGxsLCdOnBB+EwQwY8YMZsyYwdWrV2nVqhXW1tZF6ibyT0alUkm9tSCvz4fIVufp6em0adOGmjVr\nUqxYMSGD+Vu3bmlt3t/9vGTJEjlk6cXQrNnbt29fqDGESBhiTyUACwsLZs2ahbOzs9TYW9R1zBB4\n9eoVs2fPZvbs2VIdybVr19i0aRNz5swRdqOZnZ3N8+fPdVKTRX1vDQEvLy9q1qyJg4OD3oMGkYKi\ngwcPkpyczIULFzh27BiLFy9GqVTi6OhI06ZNadu2rdwSdZg+fToKhYLr16+jVCpluYk1us/l49tv\nv+X27ds4OzujVqu5dOkSH3/8MZaWloBYBYo+Pj589913wl6Dvsvbt28pXrw4KSkpJCYmYmNjI7ek\nfwRDhw7lq6++wsXFRdoABQUFCXtTpM9KPD09HWtraxnU6CcqKqrQ587OzkWk5P0YmjX7wIEDCzSG\nePeZkb9HTk4OwcHBXL16FYVCgb29PV27dhXyxjMkJISuXbtKn9VqNYGBgQwZMkRGVdpMmjSJBg0a\nMGTIEK1DyMDAQB48eMD06dNlVKefrKwsnj9/TvXq1VEqlXLL+cdw+/ZtQkNDOX36NBYWFri5udG6\ndWthA+P8PH36lJMnT7Jjxw5iY2Ol1D+R+PXXXzl06BBOTk5kZWURExNDr1696N+/f5FpMN4U5aN1\n69ZSPi6AnZ0dOTk5QqYeGFLx75w5c7C1taVNmzZ4e3vj6OiIQqEw3hb9F/Dz82PJkiWsWrVK2gD5\n+fnJLatAypQpw4EDByQzk+zsbIKCgt6bqlSUiBT0vA/Rgp73YWjGEIZE/t9QhQoVtNaykydPCnlD\ncPr0aYKCgpg+fTovX77Ez89PS7cI3Lt3j4ULF+qMDxo0qEg3ax9KeHg4fn5+VKlSheTkZBYuXCg1\neTfyn1G3bl1Gjx7N6NGjSUhIIDQ0lA0bNlC1alXc3Nzo1KmT3BIlHjx4wPnz5zl37hwXLlygdOnS\nuLi4MGLECKFutPJz9OhRdu7cKR3g5OTkMGDAAGNQJBdffPEFt27dIiUlBcjbsM2fP58DBw7IrEwX\nQyr+vX79OtOnT2fjxo18+eWXDBo0iMGDB8st6x9BRESEThC0YcMGYb/fcePG4eTkRHBwMH369OGP\nP/5g2rRpcssyUkQYmjGEIWGI6X5z584lJiYGT09PSpcuTWBgIDVq1JBblhaF1b++fv26CJV8GGvX\nrmXv3r2UK1eOhw8f4uvry9q1a+WW9Y/DysoKV1dXsrOzOX78OCEhIUIFRSNHjqR58+a4uroyefJk\nIe3j9ZH/ZlOpVBb5umAMivIxY8YMEhISiI+Px97enqtXr/L111/LLUsLQyz+zcrK4unTp+zfv58V\nK1aQk5NDWlqa3LIMmlOnTnHy5ElCQ0NJSEiQxlUqFSEhIcIGRWq1mjFjxhAVFcWQIUMaz0TOAAAf\nSElEQVQYMGAAY8aMEaoTuJGiw1ACofT0dF69eqV1myXa5r1KlSpMmDCBxYsXM2HCBLnlfBBhYWGs\nW7eOCRMm8Pz5cyZPnsx3332Hg4OD3NIkLC0tddL8ALZt2yakUUSxYsWkDbCFhQVv376VWdE/i9u3\nbxMSEkJERASWlpZ07tyZ4cOHC+dEKOJh/vvo0qULPXv2xNHREbVazeXLl+ndu3eRajAGRfmIj49n\n69ateHl58csvv/D48WOWLl0qtywtDPE0sH///gwbNoxu3bpRrVo1Fi9erGUOYOSvoyn0/PPPP6lX\nr540rlAopMJ7EXn79i1xcXGUKFGCU6dOYWlp+d66GLl48uQJYWFhOpth0fqoGBKGZgwBeU6fJ06c\noGrVqtJ7IGKq8tGjR4mPj+fixYt6v0uRDEI0nDhxgtWrV0sGR926dWPevHn88ssvMiv7N9OmTeP7\n778nMDCQBg0aoFariYmJoXr16lKPQ5F496DBUA4eDAF3d3eys7P57LPPGDNmDCVLlkShUEgmMaKm\npRkK3t7etG/fnri4OBQKBcOHDy/yg36j0UI+evfuzdq1axk1ahRLliyhYsWK9OrVi507d8otzeDR\nGC2kpqaSmJhIw4YN5ZZkRAauX79OcnIyFStWZN68eaSkpODl5UWvXr3klqZDjx49aN36/7V372FR\nlukfwL8zCKJJHlIOipqjoQISAoopBoSJiWx4xkVBJEvX05qWaJamKEvmKclaNy51ww1Nk3TkoKRA\ncoEsxiWe8oBIkgKbwaBizMDM7w/WiRFQ21/N8w58P381z/zR9+qied/7fZ/7fkbC1tbWYF2KfQSm\nwtQGQwDAhAkTcODAAcnfXFZUVODq1auIjo5GZGRko++l2itXWlqKkpISeHh4QK1WS7Zp/dq1a7h2\n7RpkMhkUCgX69Okjybxubm5QKBQAfn3woFAoJN13bCpiY2Mf+T0fmP1vYmJiHvn7aswhZyyKGjh8\n+DBqampgZWWFNWvWoE2bNhg+fDiio6NFRzNpDwYtvPjiiwgLC8PgwYM5aKGVqaqqeuQo5kGDBhk5\n0ePNnDkTu3btEh2DBDO1SZ86nQ6XLl3C3bt3Dd5wSvEp9q5du5CSkoLq6mocOnQI69atg7W1NWbP\nni06WrPq6uqQlZUFpVKJ3NxcpKeni45kwBQfPNDvT6vVIjk5GSdPnsTt27eh0+nQo0cP+Pr6Sm5H\n0cGDBx/5/fjx442UhEVRszQaDe7du2dwbhH9b0JCQrBnzx7s3r0bOp1OP2hh586doqORkZjiKOZP\nP/0UVlZWcHd3N5hA2a9fP4GpyNhmzJiB8+fPm8SkTwCYM2cOKisrDQ5wlclkktw+N336dMTHx2PG\njBn4/PPPodPpEBwcjL1794qO1khubi6USiWOHz+OX375Be+++y78/f0l10tCBNT3yNvY2GD48OHI\nzs5GTU0N+vXrh6+//hrPPfccli1bJjqi3oMHkK+//jp27NghNAt7ilD/+v7NN9/Ejh070KFDBwDA\npUuXEBMTg+3bt8PKykpwwsa+/PLLRluOpDp1jIMW/jgLFy5s1Pc2ZcoUyR3gaoqjmLOysgAY9vHJ\nZDJJFnD0xzGlSZ9A/TY6KRYVTamrqwPwa99LTU0NamtrRUZqJDo6Gunp6XBwcICfnx+WLFmCsLAw\nvPrqq6KjETWrqKhIvxtn8ODBiIiIwOLFixEYGIgJEyYITmfI0tISQ4cORXV1NV544QX9+oMHUNnZ\n2UbLwqIIwOrVqxEaGqoviADA2dkZ06dPx5o1a5o8o0CU5qaO1dbWIjk5WZJFUVODFsaMGSM6lklL\nTU3Fjh07cOnSJbzwwgv6Hw+tVivJfi1THMX8+eefN1r7+OOPBSQh0bZt24aLFy9CLpfD2dkZCxYs\nEB2pWUOGDMGVK1cMBrBI1bhx4xAaGori4mKsWrUKOTk5mDlzpuhYBr799lsoFAr4+fnB19cXHTt2\nlOxvFhmP1HvhdDodTp48iUGDBiE9PV0/6jozM1NyZ28+GKwSExMj/A2WtP7LCKJSqZq8Sff390d8\nfLyARM171NQxKTarA/UN60FBQfrPkyZNQlJSksBEps/f3x/+/v6Ii4tDRESE6Di/mSncVGRkZGDr\n1q1QqVQA6rfU2traYt68eYKTkTG98847mDZtGiIjI6HRaJCbm4t33nkH//jHP0RHMzBs2DDIZDLo\ndDrExcXByspKfyNk7KetTyokJATe3t4oKCiAhYUF5syZAzs7O9GxDCQlJeH8+fNQKpX46KOP0LNn\nT1RUVDyyT5Jatod74TZs2IBu3brh9ddfFx1Nb82aNfjwww9RXFwMBwcH/Xb1M2fOICYmRnC6ph07\ndgwajQaBgYHCxvKzpwj104W++uqrJr8LDAyU7Lx3qT+paKi8vBxJSUlISkqCSqVCUFAQ5s6dKzqW\nybtz5w7+9a9/4fbt21ixYgVycnLg6OgouYv1oyYiXb9+HadPnxacsLHJkydj8+bNWLZsGWJjY5Ga\nmop27dpx20wr86DfpaGwsDDs3r1bUKKW4/vvv0dsbCyKioogk8nQr18/zJs3T7JvuXQ6HXJzc3H4\n8GGkp6fD3d1dkr1a9McypV64hn7++WdJD4xRq9XIysrCsWPHcPXqVYwYMQLjxo1D3759jZaBb4pQ\nv1Vux44dmD17tv4JtkajwdatWw32N0qJKTypqKysRGpqKpRKJcrLy/HSSy+hqqoKqampoqO1GMuX\nL8fw4cNx4sQJAPU/ekuWLJHcU2ypPlh4FEtLS9jb20Or1aJz584IDg5GaGgoi6JWxsLCAsnJyfD0\n9IROp0NOTo6kH0CdPHkSKpUKAQEBeOedd1BYWIjXXnsNo0aNEh2tkeXLl2PhwoVwdXWFTqdDfn4+\n3nrrLSQmJoqO1iSZTAZPT094enpCo9EgMzNTdCQSwBR64dLT0xEdHQ07Ozv9BM26ujrcv38fq1at\nktwEOqD+t9bX1xdeXl44ceIENm7ciCNHjqBHjx5YsWKFUR6WsChC/Q9zdHQ0/Pz80KdPH9TV1aGo\nqAh+fn5YsWKF6HhNSktLQ0JCAmbMmAGgfmxscHCwpIoiLy8v+Pv7Y+XKlejfvz8ASHILhym7d+8e\n/vznPyM5ORkAMHbsWHzxxReCUzVmimNgbWxskJiYCEdHRyxduhT29vaorKwUHYuMbP369di6dSs+\n+eQTyOVyDBo0COvWrRMdq1nbtm1DXFwcjh07Brlcjvj4eMyaNUuSRVGnTp3g6+ur/+zn5ye5cwGX\nL1/+yO/9/PyMlISkwhR64T755BPs3LkTN2/exJw5c7B9+3YMGDAAP/30E+bMmSPJoig7OxuHDx9G\nfn4+XnzxRWzevBmOjo4oLCzE0qVLHzu6+/fAoghAu3btsGbNGty7dw83btwAAPTs2RNPPfWU4GTN\nM4UnFX/729+gVCqxaNEi+Pj4SPKibOq0Wi1++OEH/d9BZmYmtFqt4FQtQ0xMDFQqFcaNGwelUomK\nigp88sknomORkdnY2GD9+vWiYzwxCwsLdOjQAWlpaZg6dSratGmjv15ITd++fbF69WoMHz4cWq0W\neXl5sLa2RkZGBgBI4sbtypUrqKqqgpeXF7y9vdGuXTvRkUiwh3vh5s6d2+iQb9EsLCzQvXt3dO/e\nHdbW1hgwYAAAoGvXrmjbtq3gdE3bt28fgoKCsHbtWpiZmenX+/bta7SeefYUmag9e/YgNTUVxcXF\n8PHxwalTpxAWFoZp06aJjtaISqVCSkoKlEolzpw5g5CQEEycOJHnvfwOCgsLsXbtWhQUFKB9+/bo\n378/VqxYYdQ9uC1VbW0tUlJSUFZWhoiICFy6dAkKhQLm5uaioxE1a/HixaioqEB1dTX27duHQ4cO\n4dChQ/jss89ER2vkcW9hpHJw+s2bN5GamooTJ06gU6dOePnll+Hr62swsZZajwd9ZWvXrgUAzJ8/\nH2FhYZI6IPmvf/0rBg0aZDCI6datW9i5cydu376NjRs3CkzXNI1Gg5SUFJSXlyMiIgKXL19Gnz59\njHrNZVFkwkpKSvRPKpycnCQ3tacpZWVlUCqVOHLkSLPDLei3qampQdu2bVFZWYmbN2/C0dFRdKQW\nYfny5ejSpQtyc3Px5ZdfIj4+Ht999x02bdokOhpRs2pra3H58mUoFApYWlriwoULqK6uhoeHh+ho\nj6XRaPD+++8jKipKdJRmlZaW4ujRozhw4ADs7Oz044Sp9QgODsYHH3yAXr16AQB++uknzJ8/HwkJ\nCYKT/eqXX37B8ePHMXbsWP3auXPnkJeXh2nTpknybZEUrrncPmdimmpAVavVOHXqFAAYjL6WIhsb\nG0RERJjkGGkpWrt2LZydneHt7Y2wsDC4urpCJpPpD22j/92tW7cQHR2t79ubPn26wUGu1DpUV1cj\nOzsbd+7cMViX6m9tdXU18vPzcfz4cQD1hUZiYqJ+S5qU7N+/H1u3bkVFRQUsLCxQV1dn0GMkNQUF\nBUhKSkJ2djYcHR153l4rVVdXpy+IAEhyopulpaVBQQTUDxVzdnYWlOjxpHDNZVHUQFMNk2ZmZujZ\nsyfefPNNODk5CUhlqKkXe7W1tUhISEBZWZlkL9T0x/j+++/x7rvvYvfu3Zg4cSJmzpwpyQN8TZFG\no0FVVZW+X6uwsBBqtVpwKjK28PBw9OjRAzY2Nvo1KZ+ztWjRIri5uUGpVGLq1KlIT0/HypUrRcdq\n0t69e5GWlobXXnsNn3/+Ob755hsUFxeLjmXg7NmzSE5ORk5ODgYMGIAxY8ZgyZIl3Ebbio0ePRpT\npkyBi4sLtFot8vPzOZX0dyCFay6LogamTJkCKysrfXGUmZmJn3/+GZ6enoiKipLEVK/x48cbfE5K\nSsLu3bsxatQozJo1S1AqEkWtVqOsrAyHDh3Cxx9/jNraWlRVVYmO1SIsXrwYYWFhuH79OsaMGQOZ\nTCbpbT30xzA3NzepLZNarRYLFixAbm4uZs2ahenTp2PBggV4+eWXRUdrpE2bNmjbti00Gg20Wi38\n/PwwY8YMSV3LJk+ejF69esHFxQU6nQ7Jycn6aZ+AdPqeyHhmz56N0aNH48KFC2jTpg0iIiJMcsKq\n1EjhmsuiqIHMzEzs2bNH/3ny5MkIDQ3FG2+8ITBV03JycrBlyxY4OTkhLi4OzzzzjOhIJEBISAhm\nz56NcePGwdbWFps3b4a/v7/oWC2Ch4cHDh48iNu3b8Pc3FxyB+KScfj6+iIjIwPu7u4GE5GkOoWs\npqYGFy9ehKWlJbKystCzZ0/8+OOPomM16fnnn0d8fDy8vLwQFhYGW1tb/PLLL6JjGfjmm29ERyCJ\nSEhIQHBwMGJiYgzeFufn5wMA3n77bVHRWoSG11wLCwtYWVkZPQOLogbatm2L9evXw83NDXK5HGfP\nnoVGo0FWVhbat28vOh4A4PLly9i4cSPat29v0OhHrVNQUBBeeeUVtG3bFiqVCmPGjMHAgQNFxzJp\npjIRi4xj7969jY47kMlkkr1ZXr16NSoqKrB06VKsW7cOlZWVCA0NFR2rSZGRkVCr1bCwsICnpycq\nKiowfPhw0bEM8A0APfDgb8HBwUFwkpbpwIEDiI+Px507dwxaRYz5W8vpcw3cvXsXiYmJKCwshE6n\nQ+/evREUFIT79+/DyspKSNX6MEdHR/Tt27fZZjnesLUuDwYtvPjiiwgLC8PgwYM5aOH/adKkSY88\nl2To0KGCkpEIJ06ckHTzf0N37twxuE492E4rtUZwtVqN7du3Y968efrenCtXriA5ORkLFy4UnI7o\n0RYsWIBt27aJjtHijB07FrGxsY3OfDLmSwm+KWpALpfDzs7O4OyBjIwMSQ0vOHbsmOgIJCENBy1M\nmjSJgxZ+B/v379efSxIXF8dzSVq5L774Au7u7pLfPpmbm4tly5bh8OHD+r/TwsJCLF68GFu2bJHU\n0+0PPvgAgOHgoN69e+Pu3buIjY3F/PnzRUUjeqzOnTtj06ZNcHFxMRi4IYXDhk3Zs88+C4VCITQD\ni6IGwsPDYW9vD2tra/2a1KYM8VU+NcRBC3+M7t27Izw8HOHh4fpzSUJCQnguSSt09+5deHt7o1ev\nXjA3N4dOp4NMJsP+/ftFRzOwZcsW7Ny506Bw79+/P2JjY7Fu3TrExcUJTGcoPz8fBw4cMFizsLBA\nZGQkQkJCWBSRpGk0GvznP/9ptK2LRdH/T5cuXTB16lS4uroa9G8as1eLRVED5ubmkjzll6g5TQ1a\n4NkZvx+eS0IbNmwQHeGJyGQyPPvss43WFQqF5EbJN7zhaUgul0Oj0Rg5DdGTU6vVmD9/Puzs7CCX\ny0XHaVHc3d3h7u4uNAOLogZMbcoQUVBQkMH2zkmTJiEpKUlgItPHc0moodjY2CbXpda/ef/+fdTW\n1qJNmzaN1lUqlaBUTevcuTPy8vLg4eFhsJ6eno6uXbsKSkX0aGlpaVi/fj2sra1RUVGBDRs2wMXF\nRXSsFiMgIABKpRIXLlyAmZkZnJ2dERAQYNQMHLTQwOjRo01qyhARAJSXlyMpKQlJSUlQqVQICgrC\n3LlzRccyWQMGDNCfS9JUISS1m2H6Y6Wnp+v/uba2FqdPn4a5uTnefPNNcaGaEBcXh9OnT2Pp0qX6\nffkXLlxATEwMAgMDMWnSJMEJf1VcXIwFCxagb9++GDhwIOrq6nDmzBncunULcXFxLIxIkoKDg/H3\nv/8dHTt2RElJCVavXo3PPvtMdKwW46233kLHjh0xdOhQaDQa5Obmoq6uzqhnFfFNUQNHjx4VHYHo\niVRWViI1NRVKpRLl5eV46aWXUFVVhdTUVNHRTB4fglBDPj4+Bp9HjRqF2bNniwnzCBEREbC2tsby\n5cv15xLZ29sjLCwMr7zyiuB0hnr37o3ExERkZWXh2rVrkMlkmD59OkaMGCG5Pl6iB8zNzdGxY0cA\n9f9v1dTUCE7UspSWlhpsVw4ICDD6cQIsigCsWrUK77//PiZOnNjkD7LUGmqJvLy84O/vj5UrV6J/\n//4AgOzsbMGpWgYOM6GGMjIyDD6Xl5fjxo0bgtI8WmBgIAIDA0XHeCJyuRwjR47EyJEjRUcheiIP\n3x+ygP99aTQalJWVwcbGBkB9kfTw7q0/GrfPAfjpp5/QtWtX3Lhxo1HjXHV1NZ577jlByYiaplQq\noVQqcf36dfj4+GDUqFGIiopCYmKi6GhELcrDh/l26NABQUFBcHJyEpSIiERwc3PTb03V6XQoKiqC\nQqGQ7ERKU3Pq1CmsXr0acrkcWq0Wcrkca9asMerwBRZFDbzxxhvYtGkTnnrqKQBAVlYWoqOjoVQq\nBScjappKpUJKSgqUSiXOnDmDkJAQTJw4Ef369RMdjahF+PLLLzF58mSDtZ07d/I8MKJW5sG21OZw\nl8HvQ6VSQSaTCTkbjkVRAxkZGfj000/x3nvvYc+ePbhx4waioqLQs2dP0dGIHqusrAxKpRJHjhzB\nV199JToOkUnLysrCyZMnkZKSYtCTU1dXh6SkJHz77bcC0zXv5s2bjdbMzMzQrVs3jhAmIsl5+G38\nw4w53IhF0UN++OEHzJs3Dx4eHli1apXoOEREJMDdu3dx/vx5rF27FhEREfp1mUwGJycnyW6rDg4O\nxrlz5/RPrW/evIl+/fqhsrISixYtMhjhT0Qk2qRJk1BVVQUvLy94e3s3OgZn6NChRsvCoghoNGCh\nuroapaWl+r2j3CdKRNR6lZaWoqSkBB4eHlCr1bCwsBAdqVmRkZGYNWsWHBwcAACFhYX45z//icjI\nSISFhWHfvn2CExIRGbp58yZSU1Nx4sQJdOrUCS+//DJ8fX3RoUMHo+bg9DkAH330EXQ6HcrKymBr\nays6DhERScSuXbuQkpKC6upqHDp0CBs2bEC3bt3w+uuvi47WpKtXr+oLIgDo27cvLl68iHbt2qGu\nrk5gMiKipnXv3h3h4eEIDw9HaWkpjh49ipCQENjZ2eHTTz81Wg4WRfi1OS4yMhLx8fGC0xARkVSk\npaUhISEBM2bMAACsWLECwcHBki2KXF1dMWHCBLi6ukImk+H8+fNQKBRITEzE4MGDRccjImpWQUEB\nkpKSkJ2dDUdHR4wZM8ao/34WRQ1YW1sjODgYgwYNMjjJ/u233xaYioiIRHnwduXBFuuamhqjn53x\nW6xcuRKXL19GYWEhAGDChAlwcnKCWq1mPxERSc7Zs2eRnJyMnJwcDBgwAGPGjMGSJUsM7sONhT1F\nDRw8eLDJ9fHjxxs5CRERScGePXuQmpqK4uJi+Pj44NSpUwgLC8O0adNER2vSxYsXkZiYiDt37qDh\n5d2YE5yIiJ7UgAED0KtXL7i4uDRZCHH6nCBqtRpKpRIXLlyAmZkZnJ2dERAQwDGmREStWElJCQoK\nCmBhYQEnJyfY2dmJjtSsgIAAzJgxo1F/rI+Pj5hARESPIKXzn1gUNfDWW2+hY8eOGDp0KDQaDXJz\nc1FXV4eoqCjR0YiIyIh0Oh0OHz6M4uJiODo6ws/PD0D99rnt27dj8eLFghM2LSIiAnFxcaJjEBGZ\nHPYUNVBaWooNGzboPwcEBCA0NFRgIiIiEmHVqlXQaDRwcXHBF198gevXr6N379748MMP4e/vLzpe\ns5ydnRETEwMPDw+0afPrJd7b21tgKiIi6WNR1IBGo0FZWRlsbGwA1BdJUm6oJSKiP8bly5eRkJAA\noP5wQS8vLwwbNgyfffYZ7O3tBadrXnl5OYD6qXkNsSgiIno0FkUNLF68GDNnzoRcLodWq4VcLsfa\ntWtFxyIiIiNr2PBrbm4OBwcHbN26VWCiR3twqOx7770nOgoRkUliUQQgKysL7u7u8PT0RHJyMlQq\nFWQyGZ5++mnR0YiISIAHI7ib+yw1y5cvx8aNGxEQEGCQVafTQSaT4ZtvvhGYjohI+jhoAcD8+fNx\n5swZ9OzZE56envD09ISbmxssLCxERyMiIgHc3NygUCgA1BcWRUVFUCgU+iJj//79ghM2raCgAC4u\nLgZr2dnZeOGFFwQlIiIyDSyKGigsLEReXh7y8vJw9uxZdOvWDcOGDcO8efNERyMiIiOS0pjYJ1Fc\nXIyioiJs2rQJS5Ys0a/X1tZi3bp1OH78uMB0RETSx6LoIWq1Gvn5+cjPz0d6ejpUKhWSk5NFxyIi\nImrWpUuXcOzYMSQkJGDkyJH6dZlMBg8PD0yYMEFgOiIi6WNRBCAzMxN5eXnIz8+HVquFi4sL3Nzc\n4O7uji5duoiOR0RE9EQuX74MBwcHg7Xt27fjL3/5i6BERESmgUUR6s8jun//Pv70pz9hxIgReP75\n59lPREREJicjIwNbt26FSqUCUH/UhK2tLfbt2yc4GRGRtLEo+q+KigqcPn0ap0+fxpkzZyCXy+Hq\n6goPDw/4+PiIjkdERPRYkydPxubNm7Fs2TLExsYiNTUV7dq1w6uvvio6GhGRpHEk93917twZo0aN\nwqhRo1BWVoaTJ09i79692LVrF86dOyc6HhER0WNZWlrC3t4eWq0WnTt3RnBwMEJDQ1kUERE9Bosi\nADdu3EBeXh7+/e9/4/Tp03jqqafg6emJuXPnYsiQIaLjERERPREbGxskJibC0dERS5cuhb29PSor\nK0XHIiKSPG6fAxAYGIhhw4bB09MTQ4YMQceOHUVHIiIi+s00Gg3u3LmDp59+GkqlEhUVFRg9erTk\nRogTEUkNiyIiIqIWIiQkBHv27BEdg4jI5LAoIiIiaiEiIyOh0WgwaNAgmJub69dDQkIEpiIikj72\nFBEREbUQ9vb2AIC7d+8KTkJEZFr4poiIiKgFKS0tRUlJCTw8PKBWq3nuHhHRE+CbIiIiohZi165d\nSElJwf379/H1119jw4YNsLa2xuzZs0VHIyKSNLnoAERERPT7SEtLQ0JCAp5++mkAwIoVK5CWliY4\nFRGR9LEoIiIiaiHq6uoAADKZDABQU1OD2tpakZGIiEwCt88RERG1EOPGjUNoaCiKi4uxatUq5OTk\nYObMmaJjERFJHgctEBERtSAlJSUoKCiAhYUFnJycYGdnJzoSEZHk8U0RERFRC/H9998jNjYWRUVF\nkMlk6NevH+bNm4fnnntOdDQiIknjmyIiIqIWYvz48Vi4cCFcXV2h0+mQn5+Pbdu2ITExUXQ0IiJJ\n46AFIiKiFqJTp07w9fVF586d0aVLF/j5+cHW1lZ0LCIiyeObIiIiohYiKioKtbW1GD58OLRaLfLy\n8qBWq+Hn5wcA8Pb2FpyQiEia2FNERETUQty7dw8AcOLECYP1lJQUACyKiIiawzdFRERELcTPP/+M\n4uJimJmZ4dlnn9Uf4kpERI/GN0VEREQmrqamBu+++y4KCgrg4OAArVaLq1evwsPDAytXroSlpaXo\niEREksY3RURERCYuKioKPXr0QHh4uMH6rl27cO3aNaxZs0ZQMiIi08Dpc0RERCbuu+++a1QQAcDM\nmTNx7tw5AYmIiEwLiyIiIiITZ2Zm1ux3cjkv9UREj8NfSiIiIhP3zDPP4NSpU43WMzMzYWNjIyAR\nEZFpYU8RERGRibtx4wYWLFiAPn36YODAgdBqtTh79ix+/PFHxMXF4ZlnnhEdkYhI0lgUERERtQA6\nnQ4nT57EtWvXIJPJoFAoMGLECMhkMtHRiIgkj0URERERERG1auwpIiIiIiKiVo1FERERERERtWos\nioiIiIiIqFVjUURERERERK0aiyIiIiIiImrV/g/FL0g6Nr+9MQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f854f4c81d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "feature_importances = pd.Series(clf.feature_importances_, index=results_wo_returns.columns)\n",
    "feature_importances.sort(ascending=False)\n",
    "ax = feature_importances.plot(kind='bar')\n",
    "ax.set(ylabel='Importance (Gini Coefficient)', title='Feature importances');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Exercise*: Use [partial dependence plots](http://scikit-learn.org/stable/auto_examples/ensemble/plot_partial_dependence.html) to get an understanding of how factor rankings are used to predict future returns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Where to go from here\n",
    "\n",
    "Several knobs can be tweaked to boost performance:\n",
    "\n",
    "* Add existing factors from the collection above to the data set.\n",
    "* Come up with new factors \n",
    "  * Use [`alphalens`](https://www.quantopian.com/posts/alphalens-a-new-tool-for-analyzing-alpha-factors) to evaluate an alpha for its predictive power.\n",
    "  * Look for [novel data sources from our partners](https://www.quantopian.com/data).\n",
    "  * Look at the [101 Alpha's Project](https://www.quantopian.com/posts/the-101-alphas-project).\n",
    "* Improve preprocessing of the ML pipeline\n",
    "  * Is 70/30 the best split? \n",
    "  * Should we not binarize the returns and do regression? \n",
    "  * Can we add Sector information in some way?\n",
    "* Experiment with [feature selection](http://scikit-learn.org/stable/modules/feature_selection.html).\n",
    "  * PCA\n",
    "  * ICA\n",
    "  * etc.\n",
    "* Tweak hyper-parameters of `AdaBoostClassifier`.\n",
    "  * [Use cross-validation to find optimal parameters](http://scikit-learn.org/stable/modules/grid_search.html).\n",
    "* Try [different classifiers](http://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html) of combinations of classifiers.\n",
    "\n",
    "## Machine Learning competition\n",
    "If you have something you think works well, post it in this thread. Make sure to test over the same time-period as I have here to keep things comparable. In a month from now, we can test on new data that has aggregated since then and determine who built the best ML pipeline. If there is demand, we might turn this into a proper ML contest.\n",
    "\n",
    "## Machine Learning resources\n",
    "\n",
    "If you look for information on how to get started with ML, here are a few resources:\n",
    "* [Scikit-learn resources](http://scikit-learn.org/stable/presentations.html)\n",
    "* [Learning scikit-learn: Machine Learning in Python](https://www.amazon.com/dp/1783281936)\n",
    "* [Pattern Recognition and Machine Learning](https://www.amazon.com/Pattern-Recognition-Learning-Information-Statistics/dp/0387310738)\n",
    "\n",
    "## How to put this into an algorithm\n",
    "\n",
    "As mentioned above, this is not immediately usable in an algorithm. For one thing, there is no `run_pipeline()` in the backtest IDE. It turns out to be rather simple to take the code above and put it into a pipeline `CustomFactor()` where the ML model would automatically get retrained and make predictions. You would then long the `1` predictions and short the `-1` predictions, apply some weighting (e.g. inverse variance) and execute orders. More on these next steps in the future.\n",
    "\n",
    "## Credits\n",
    "* Content created by James Christopher and Thomas Wiecki\n",
    "* Thanks to Sheng Wang for ideas and inspiration.\n",
    "* Jonathan Larkin, Jess Stauth, Delaney Granizo-Mackenzie, and Jean Bredeche for helpful comments on an earlier draft."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}