Data Preprocess for hierarchical model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prepare Data for Training & Validating & Testing\n",
    "# Hierarchical model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "- [**IMDB Large Movie Review Dataset**](http://ai.stanford.edu/~amaas/data/sentiment/)\n",
    "    - Binary sentiment classification\n",
    "    - Citation: [Andrew L. Maas et al., 2011](http://ai.stanford.edu/~amaas/papers/wvSent_acl2011.pdf)\n",
    "    - 50,000 movie reviews for training and testing\n",
    "    - Average review length: 231 vocab\n",
    "    ---\n",
    "- [**Yelp reviews-full**](http://xzh.me/docs/charconvnet.pdf)\n",
    "    - Multiclass sentiment classification (5 stars)\n",
    "    - Citation: [Xiang Zhang et al., 2015](https://arxiv.org/abs/1509.01626)\n",
    "    - 650,000 training samples and 50,000 testing samples (Nums of each star are equal)\n",
    "    - Average review length: 140 vocab\n",
    "    ---\n",
    "- [**Yelp reviews-polarity**](http://xzh.me/docs/charconvnet.pdf)\n",
    "    - Binary sentiment classification\n",
    "    - Citation: [Xiang Zhang et al., 2015](https://arxiv.org/abs/1509.01626)\n",
    "    - 560,000 training samples and 38,000 testing samples (Nums of positive or negative samples are equal)\n",
    "    - Average review length: 140 vocab\n",
    "    ---\n",
    "- [**Douban Movie Reviews**](https://drive.google.com/open?id=1DsmQfB1Ff_BUoxOv4kfUMg7Y8M7tHB9F) \n",
    "    - My Custom Chinese movie reviews dataset scraped by python extension package requests\n",
    "    - Binary sentiment classification\n",
    "    - 750,000 movie reviews from [Douban](https://movie.douban.com/), 650,000 samples for training and 100,000 samples for testing (Num of positive or negative samples is equal)\n",
    "    - Average review length: 52 character, 28 vocab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T13:58:11.001118Z",
     "start_time": "2019-04-01T13:58:10.855162Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "import matplotlib.pyplot as plt\n",
    "import warnings\n",
    "import re\n",
    "from functools import partial\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "from utils import process, process_zh\n",
    "%matplotlib inline\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IMDB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:37:53.984778Z",
     "start_time": "2019-03-31T11:37:51.164267Z"
    }
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv('dataset/IMDB/imdb_master.csv', encoding='latin-1')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:37:54.228449Z",
     "start_time": "2019-03-31T11:37:53.986756Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>type</th>\n",
       "      <th>review</th>\n",
       "      <th>label</th>\n",
       "      <th>file</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>test</td>\n",
       "      <td>Once again Mr. Costner has dragged out a movie...</td>\n",
       "      <td>0</td>\n",
       "      <td>0_2.txt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>test</td>\n",
       "      <td>This is an example of why the majority of acti...</td>\n",
       "      <td>0</td>\n",
       "      <td>10000_4.txt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>test</td>\n",
       "      <td>First of all I hate those moronic rappers, who...</td>\n",
       "      <td>0</td>\n",
       "      <td>10001_1.txt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>test</td>\n",
       "      <td>Not even the Beatles could write songs everyon...</td>\n",
       "      <td>0</td>\n",
       "      <td>10002_3.txt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>test</td>\n",
       "      <td>Brass pictures (movies is not a fitting word f...</td>\n",
       "      <td>0</td>\n",
       "      <td>10003_3.txt</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  type                                             review  label  \\\n",
       "0           0  test  Once again Mr. Costner has dragged out a movie...      0   \n",
       "1           1  test  This is an example of why the majority of acti...      0   \n",
       "2           2  test  First of all I hate those moronic rappers, who...      0   \n",
       "3           3  test  Not even the Beatles could write songs everyon...      0   \n",
       "4           4  test  Brass pictures (movies is not a fitting word f...      0   \n",
       "\n",
       "          file  \n",
       "0      0_2.txt  \n",
       "1  10000_4.txt  \n",
       "2  10001_1.txt  \n",
       "3  10002_3.txt  \n",
       "4  10003_3.txt  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df[df['label'] != 'unsup']\n",
    "df['label'] = df['label'].map({'neg':0, 'pos':1})\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:40:55.784383Z",
     "start_time": "2019-03-31T11:37:54.232430Z"
    }
   },
   "outputs": [],
   "source": [
    "df['Processed_Reviews'] = df['review'].apply(process)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:40:56.063125Z",
     "start_time": "2019-03-31T11:40:55.790399Z"
    }
   },
   "outputs": [],
   "source": [
    "df_train, df_test = train_test_split(df, test_size=0.1, random_state=0, stratify=df.label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:40:56.234581Z",
     "start_time": "2019-03-31T11:40:56.065130Z"
    }
   },
   "outputs": [],
   "source": [
    "df_train, df_validate = train_test_split(df_train, test_size=5000, random_state=0, stratify=df_train.label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:40:56.403029Z",
     "start_time": "2019-03-31T11:40:56.236586Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set size:  40000\n",
      "Validation set size:  5000\n",
      "Test set size:  5000\n"
     ]
    }
   ],
   "source": [
    "print(\"Train set size: \", len(df_train))\n",
    "print(\"Validation set size: \", len(df_validate))\n",
    "print(\"Test set size: \", len(df_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:40:56.787052Z",
     "start_time": "2019-03-31T11:40:56.406037Z"
    }
   },
   "outputs": [],
   "source": [
    "num_words = []\n",
    "def count(x):\n",
    "    for sen in x:\n",
    "        num_words.append(len(sen))\n",
    "num_sens = df_train.Processed_Reviews.apply(len)\n",
    "_ = df_train.Processed_Reviews.apply(count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:40:57.430764Z",
     "start_time": "2019-03-31T11:40:56.790059Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Counts')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHMdJREFUeJzt3Xm4XWV59/HvbSKjQMBECyFwQFJluBQxIg61FKyMCvUVmxZrVJS24gRWhWKLEzVc9RW1KrwIyCCCiChhUEAGhypDGAUjEJkSiBAIk6BA4H7/WM+BneMZ9hP2Pnsn+X6u61xnr2et9ex7P3v47TWcdSIzkSSpXc/rdQGSpBWLwSFJqmJwSJKqGBySpCoGhySpisEhSapicPRARBwdEf/Rob42iYg/RMSEMn1pRLyvE32X/n4UEbM61V/F/X4+Iu6LiN+P930/FxHxdxGxoDwnr+x1Pb0y9HW5Kujk+7rfhX/H0VkRcTvwYmAp8BTwG+Ak4JjMfHo5+npfZv6kYp1LgW9n5rE191XW/TSwRWa+s3bdToqIacDNwKaZeW8va6kVEb8DDsrMs8bxPk8AFmbmp8brPrVqc4ujO96SmesAmwKzgU8Cx3X6TiJiYqf77BObAvevaKFRbArc2OsiVG8lfj91Xmb608Ef4HbgTUPatgeeBrYp0ycAny+3JwPnAA8CS4Cf0wT6yWWdPwJ/AD4BDAAJ7AfcCfyspW1i6e9S4AvAFcBDwFnABmXejjTfTP+sXmBX4AngyXJ/17X0975y+3nAp4A7gHtptqTWK/MG65hVarsPOHSUcVqvrL+49Pep0v+bymN+utRxwjDr7ggsBD5W6lgEvKdl/jM1l+l3A79omU7gA8AtwCPA54CXAL8CHgZOB1Yboe5hxwBYvdSbwKPA74ZZN4Ajy3oPAde3vCZWB75Yxu4e4GhgzbEeL7B/ec6eKPd/dmnfCPh+Gd/bgA+31PHp8hhPKo//RmBGy/xpwJll3fuBr7XMey8wD3gAOJ9mq3C4cRp8PbS+Lj8H/G+5zwuAyaO8Pt4PzKd5T8wBNhry/P1Lef4eAL5O2XuynDU+834q7TsAv6R5T14H7FjaZwJzh/RxIDBn6Pu6TO8JXFv6+SXw8tL+nsHnqUzPB05vmV4AbNvrz7JRP+d6XcDK9sMwwVHa7wT+tdx+5gVG8yF/NPD88vNXPLsLcZm+Wl7oJwFrA2uO8Aa9C9imLPN9ml1XMEpwlNufHly2Zf6lPBsc7y0v8s2BF9B8uJw8pLZvlrpeATwObDnCOJ1EE2rrlHVvBvYbqc4h6+5Isyvws2XMdgceA9YfWnOZfjd/HhxzgHWBrUudF5XHtR7N7sVZI9z3iGPQ0vcWI6y7C3AVMIkmRLYENizzvlxq2qCMydnAF9p8vCew7AfW88r9/CewWqn1VmCXluf5T6WfCTSvwcvKvAk0H5ZH0rx+1gDeUObtXR77lsBEmgD95QiPdfD10Pq6/B3wl+X1cSkwe4R1d6L54rEdTaD+D+VDvWWMzynjuAlNwO36HGpsfT9NpQnL3cs4/m2ZngKsRRN601v6uBKYOcz7ejuakH9NGdNZNO+11cvz8WDpf0OaLyF3lfU2pwm85/X6s2y0H3dVjZ+7aT4UhnqS5sWzaWY+mZk/z/IKGsWnM/PRzPzjCPNPzswbMvNR4D+Ad3ToIOW+wJcy89bM/ANwCDBzyCb+ZzLzj5l5Hc0H0CuGdlJq+XvgkMx8JDNvB/4v8E8VtTwJfLaM2Xk037ZfWrH+EZn5cGbeCNwAXFAe10PAj4CRDmy3Mwaj1bwO8DKaLwfzMnNRRATNN+wDM3NJZj4C/BfNN9zlebyvBqZk5mcz84nMvJUm0Fv7+0VmnpeZT9Fs3Q4+T9vTbK18vLzG/pSZvyjz/pkmzOZl5tJS47YRsWkbjx3gW5l5c3ndng5sO8Jy+wLHZ+bVmfk4zRi/NiIGWpaZnZkPZuadwCUtfS1Pja3vp3cC55WxeTozLwTmArtn5mM0X3b+ASAiptM8l3OG6fP9wP/LzMsz86nMPJHmC8oO5fl4pNT81zRbRXdFxMvK9M+z8njoeDM4xs9Ums3uof6b5hvSBRFxa0Qc3EZfCyrm30HzLXVyW1WObqPSX2vfE2lOBhjUehbUYzTfyoeaTPNNeGhfUytqub98MIx1XyO5p+X2H4eZHqmvdsZgWJl5MfA1ml0r90TEMRGxLs9+m70qIh6MiAeBH5f2QTWPd1Ngo8G+Sn//zujP0xol/KYBdwy5r9Z+v9LS5xKaLad2n7d2XhswZIxLQN8/5H5G6mt5amx9v2wK7DNk7N5A8+UO4DuU4AD+EfhhCZShNgU+NqSfaeWxAfyUZkvyjeX2pTSh8ddluq8ZHOMgIl5N88L9xdB55Rv3xzJzc+AtwEERsfPg7BG6HGuLZFrL7U1ovq3eR7Pvfa2Wuiaw7IfTWP3eTfOGaO17Kct+6LbjvlLT0L7uquxnJMs8TuAvOtQvPMcxyMyvZuaraHaR/SXwcZrx+COwdWZOKj/rZWa7QTj0eVsA3NbS16TMXCczd2+jrwXAJiNsQS0A/nlIv2tm5i/brLNdy4xxRKwNvJD2Xh/LU2Pr+C2g2WJvXX/tzJxd5l8ATI6IbWkC5Duj1HH4kH7WysxTy/zB4PircvunGBwCiIh1I2JP4DSaYwe/HmaZPSNii7K74mGaU3ifKrPvodnnWeudEbFVRKxFs1/8jLJL4maab5Z7RMTzafb/rt6y3j3AQESM9Lo4FTgwIjaLiBfQ7Ab47gjfTkdUajkdODwi1im7EQ4Cvl3TzyiuBd4WEWtFxBY0Bz87ZbnHICJeHRGvKWP/KM1xhqfKbolvAkdGxIvKslMjYpc2axr6OrkCeDgiPhkRa0bEhIjYpnyBGcsVNAffZ0fE2hGxRkS8vsw7GjgkIrYuNa4XEfu0WWON7wDviYhtI2J1mjG+vOzSHMtzrfHbwFsiYpcybmtExI4RsTFAeZ7PoNlTsAFw4Qj9fBP4l/J8RxnLPSJinTL/p8Df0JwAsZDmpJhdaQLymop6e8Lg6I6zI+IRmm8dhwJfojmTYjjTgZ/Q7LP+FfCNzLy0zPsC8KmyqftvFfd/Ms2But/THNz8MEDZf/8B4Fiab2+P0pytM+h75ff9EXH1MP0eX/r+Gc2ZOn8CPlRRV6sPlfu/lWZL7Dul/044kuYso3uAE4FTOtQvPLcxWJfmA+UBml0x99OcSQXNKdvzgcsi4mGa10S7x2yOA7Yqr5MflmB+C80+9NtotmiOpTnwP6qWdbegOaFjIc3xKDLzB8ARwGmlxhuA3dqssW2ZeRHNsbnv04TYS1j2+Mxo6z6nGjNzAbAXza69xTTv4Y+z7Gfld2jO/vveSF8YMnMuzXGOr9E83/NpTtIYnH8zzXv+52X6YZr3wv+W56Cv+QeAkqQqbnFIkqoYHJKkKgaHJKmKwSFJqrJSXtRr8uTJOTAw0OsyJGmFctVVV92XmVPGWm6lDI6BgQHmzp3b6zIkaYUSEXeMvZS7qiRJlQwOSVIVg0OSVMXgkCRVMTgkSVUMDklSFYNDklTF4JAkVTE4JElVVsq/HB9vAwefO2z77bP3GOdKJKn73OKQJFUxOCRJVQwOSVIVg0OSVMXgkCRVMTgkSVUMDklSFYNDklTF4JAkVTE4JElVDA5JUhWDQ5JUxeCQJFUxOCRJVQwOSVIVg0OSVMXgkCRVMTgkSVUMDklSFYNDklSlq8EREQdGxI0RcUNEnBoRa0TEZhFxeUTcEhHfjYjVyrKrl+n5Zf5ASz+HlPabImKXbtYsSRpd14IjIqYCHwZmZOY2wARgJnAEcGRmTgceAPYrq+wHPJCZWwBHluWIiK3KelsDuwLfiIgJ3apbkjS6bu+qmgisGRETgbWARcBOwBll/onA3uX2XmWaMn/niIjSflpmPp6ZtwHzge27XLckaQRdC47MvAv4InAnTWA8BFwFPJiZS8tiC4Gp5fZUYEFZd2lZ/oWt7cOs84yI2D8i5kbE3MWLF3f+AUmSgO7uqlqfZmthM2AjYG1gt2EWzcFVRpg3UvuyDZnHZOaMzJwxZcqU5StakjSmbu6qehNwW2YuzswngTOB1wGTyq4rgI2Bu8vthcA0gDJ/PWBJa/sw60iSxlk3g+NOYIeIWKscq9gZ+A1wCfD2ssws4Kxye06Zpsy/ODOztM8sZ11tBkwHruhi3ZKkUUwce5Hlk5mXR8QZwNXAUuAa4BjgXOC0iPh8aTuurHIccHJEzKfZ0phZ+rkxIk6nCZ2lwAGZ+VS36u6kgYPPHbb99tl7jHMlktQ5XQsOgMw8DDhsSPOtDHNWVGb+CdhnhH4OBw7veIGSpGr+5bgkqYrBIUmqYnBIkqoYHJKkKgaHJKmKwSFJqmJwSJKqGBySpCoGhySpisEhSapicEiSqhgckqQqBockqYrBIUmqYnBIkqoYHJKkKgaHJKmKwSFJqmJwSJKqGBySpCoGhySpisEhSapicEiSqhgckqQqBockqYrBIUmqYnBIkqoYHJKkKgaHJKmKwSFJqmJwSJKqGBySpCoGhySpisEhSapicEiSqhgckqQqBockqUpXgyMiJkXEGRHx24iYFxGvjYgNIuLCiLil/F6/LBsR8dWImB8R10fEdi39zCrL3xIRs7pZsyRpdN3e4vgK8OPMfBnwCmAecDBwUWZOBy4q0wC7AdPLz/7AUQARsQFwGPAaYHvgsMGwkSSNv64FR0SsC7wROA4gM5/IzAeBvYATy2InAnuX23sBJ2XjMmBSRGwI7AJcmJlLMvMB4EJg127VLUkaXTe3ODYHFgPfiohrIuLYiFgbeHFmLgIov19Ulp8KLGhZf2FpG6l9GRGxf0TMjYi5ixcv7vyjkSQB3Q2OicB2wFGZ+UrgUZ7dLTWcGKYtR2lftiHzmMyckZkzpkyZsjz1SpLa0M3gWAgszMzLy/QZNEFyT9kFRfl9b8vy01rW3xi4e5R2SVIPTOxWx5n5+4hYEBEvzcybgJ2B35SfWcDs8vusssoc4IMRcRrNgfCHMnNRRJwP/FfLAfE3A4d0q+7RDBx8bi/uVpL6SteCo/gQcEpErAbcCryHZivn9IjYD7gT2Kcsex6wOzAfeKwsS2YuiYjPAVeW5T6bmUu6XLckaQRdDY7MvBaYMcysnYdZNoEDRujneOD4zlYnSVoe/uW4JKmKwSFJqmJwSJKqGBySpCoGhySpisEhSapicEiSqlQHR0SsHxEv70YxkqT+11ZwRMSlEbFu+d8Y19Fc8fZL3S1NktSP2t3iWC8zHwbeBnwrM18FvKl7ZUmS+lW7wTGxXMn2HcA5XaxHktTn2g2OzwDnA/Mz88qI2By4pXtlSZL6VbsXOVyUmc8cEM/MWz3GIUmrpna3OP6nzTZJ0kpu1C2OiHgt8DpgSkQc1DJrXWBCNwuTJPWnsXZVrQa8oCy3Tkv7w8Dbu1WUJKl/jRocmflT4KcRcUJm3jFONUmS+li7B8dXj4hjgIHWdTJzp24UtbIb6X+X3z57j3GuRJLqtRsc3wOOBo4FnupeOZKkftducCzNzKO6WokkaYXQ7um4Z0fEByJiw4jYYPCnq5VJkvpSu1scs8rvj7e0JbB5Z8uRJPW7toIjMzfrdiGSpBVDW8EREe8arj0zT+psOZKkftfurqpXt9xeA9gZuBowOCRpFdPurqoPtU5HxHrAyV2pSJLU15b3f44/BkzvZCGSpBVDu8c4zqY5iwqaixtuCZzeraIkSf2r3WMcX2y5vRS4IzMXdqEeSVKfa2tXVbnY4W9prpC7PvBEN4uSJPWvtoIjIt4BXAHsQ/N/xy+PCC+rLkmroHZ3VR0KvDoz7wWIiCnAT4AzulWYJKk/tXtW1fMGQ6O4v2JdSdJKpN0tjh9HxPnAqWX674HzulOSJKmfjfU/x7cAXpyZH4+ItwFvAAL4FXDKONQnSeozY+1u+jLwCEBmnpmZB2XmgTRbG1/udnGSpP4zVnAMZOb1Qxszcy7Nv5EdU0RMiIhrIuKcMr1ZRFweEbdExHcjYrXSvnqZnl/mD7T0cUhpvykidmnzsUmSumCs4FhjlHlrtnkfHwHmtUwfARyZmdOBB4D9Svt+wAOZuQVwZFmOiNgKmAlsDewKfCMiJrR535KkDhsrOK6MiPcPbYyI/YCrxuo8IjYG9qD5X+VERAA78expvCcCe5fbe5Vpyvydy/J7Aadl5uOZeRswH9h+rPuWJHXHWGdVfRT4QUTsy7NBMQNYDfi7Nvr/MvAJmr84B3gh8GBmLi3TC4Gp5fZUYAFAZi6NiIfK8lOBy1r6bF1HkjTORg2OzLwHeF1E/A2wTWk+NzMvHqvjiNgTuDczr4qIHQebh7ubMeaNtk7r/e0P7A+wySabjFWeJGk5tfv/OC4BLqns+/XAWyNid5pjJevSbIFMioiJZatjY+DusvxCYBqwMCImAusBS1raB7Wu01rjMcAxADNmzPizYJEkdUbX/vo7Mw/JzI0zc4Dm4PbFmbkvTQANXudqFnBWuT2nTFPmX5yZWdpnlrOuNqP5PyBXdKtuSdLo2v3L8U76JHBaRHweuAY4rrQfB5wcEfNptjRmAmTmjRFxOvAbmku6H5CZT41/2ZIkGKfgyMxLgUvL7VsZ5qyozPwTzdV3h1v/cODw7lUoSWqXFyqUJFUxOCRJVQwOSVIVg0OSVMXgkCRVMTgkSVUMDklSFYNDklTF4JAkVTE4JElVDA5JUhWDQ5JUxeCQJFXpxWXVNYKBg88dtv322XuMcyWSNDK3OCRJVQwOSVIVg0OSVMXgkCRVMTgkSVUMDklSFYNDklTF4JAkVTE4JElVDA5JUhWDQ5JUxeCQJFUxOCRJVQwOSVIVg0OSVMXgkCRVMTgkSVUMDklSFYNDklTF4JAkVTE4JElVDA5JUpWJvS5AYxs4+Nxh22+fvcc4VyJJXdziiIhpEXFJRMyLiBsj4iOlfYOIuDAibim/1y/tERFfjYj5EXF9RGzX0tessvwtETGrWzVLksbWzV1VS4GPZeaWwA7AARGxFXAwcFFmTgcuKtMAuwHTy8/+wFHQBA1wGPAaYHvgsMGwkSSNv64FR2Yuysyry+1HgHnAVGAv4MSy2InA3uX2XsBJ2bgMmBQRGwK7ABdm5pLMfAC4ENi1W3VLkkY3LgfHI2IAeCVwOfDizFwETbgALyqLTQUWtKy2sLSN1D70PvaPiLkRMXfx4sWdfgiSpKLrwRERLwC+D3w0Mx8ebdFh2nKU9mUbMo/JzBmZOWPKlCnLV6wkaUxdDY6IeD5NaJySmWeW5nvKLijK73tL+0JgWsvqGwN3j9IuSeqBbp5VFcBxwLzM/FLLrDnA4JlRs4CzWtrfVc6u2gF4qOzKOh94c0SsXw6Kv7m0SZJ6oJt/x/F64J+AX0fEtaXt34HZwOkRsR9wJ7BPmXcesDswH3gMeA9AZi6JiM8BV5blPpuZS7pYtyRpFF0Ljsz8BcMfnwDYeZjlEzhghL6OB47vXHWSpOXlJUckSVUMDklSFYNDklTF4JAkVTE4JElVvKz6CszLrUvqBbc4JElVDA5JUhWDQ5JUxeCQJFUxOCRJVQwOSVIVg0OSVMXgkCRVMTgkSVX8y/GVkH9RLqmb3OKQJFVxi2MYI31jlyS5xSFJqmRwSJKqGBySpCoGhySpisEhSariWVWrEP++Q1InuMUhSapicEiSqhgckqQqHuPQqH8p7/EPSUO5xSFJqmJwSJKquKtKo/IUXklDucUhSapicEiSqrirSh3lri1p5WdwaFwYKNLKw+DQcunUf0k0UKQVj8c4JElVVpgtjojYFfgKMAE4NjNn97gkdZFbIlL/WiGCIyImAF8H/hZYCFwZEXMy8ze9rUzjrXYXmUEjdd4KERzA9sD8zLwVICJOA/YCDA6NqlPHYpaHoaWV1YoSHFOBBS3TC4HXtC4QEfsD+5fJP0TETZX3MRm4b7krHD/W2VldqzOO6FhXq/xYdtiKUGevaty0nYVWlOCIYdpymYnMY4BjlvsOIuZm5ozlXX+8WGdnrQh1rgg1gnV2Ur/XuKKcVbUQmNYyvTFwd49qkaRV2ooSHFcC0yNis4hYDZgJzOlxTZK0SlohdlVl5tKI+CBwPs3puMdn5o0dvpvl3s01zqyzs1aEOleEGsE6O6mva4zMHHspSZKKFWVXlSSpTxgckqQqBgfN5Uwi4qaImB8RB/e6nkERMS0iLomIeRFxY0R8pLRvEBEXRsQt5ff6fVDrhIi4JiLOKdObRcTlpcbvlpMael3jpIg4IyJ+W8b0tX06lgeW5/uGiDg1Itboh/GMiOMj4t6IuKGlbdjxi8ZXy3vq+ojYroc1/nd5zq+PiB9ExKSWeYeUGm+KiF3Go8aR6myZ928RkRExuUz3ZCxHs8oHR8vlTHYDtgL+ISK26m1Vz1gKfCwztwR2AA4otR0MXJSZ04GLynSvfQSY1zJ9BHBkqfEBYL+eVLWsrwA/zsyXAa+gqbevxjIipgIfBmZk5jY0J4PMpD/G8wRg1yFtI43fbsD08rM/cFQPa7wQ2CYzXw7cDBwCUN5LM4GtyzrfKJ8HvaqTiJhGc2mlO1uaezWWI1rlg4OWy5lk5hPA4OVMei4zF2Xm1eX2IzQfdFNp6juxLHYisHdvKmxExMbAHsCxZTqAnYAzyiL9UOO6wBuB4wAy84nMfJA+G8tiIrBmREwE1gIW0QfjmZk/A5YMaR5p/PYCTsrGZcCkiNiwFzVm5gWZubRMXkbzd2CDNZ6WmY9n5m3AfJrPg64bYSwBjgQ+wbJ/4NyTsRyNwTH85Uym9qiWEUXEAPBK4HLgxZm5CJpwAV7Uu8oA+DLNi/3pMv1C4MGWN2s/jOnmwGLgW2WX2rERsTZ9NpaZeRfwRZpvnIuAh4Cr6L/xHDTS+PXr++q9wI/K7b6qMSLeCtyVmdcNmdVXdYLBAW1czqTXIuIFwPeBj2bmw72up1VE7Ancm5lXtTYPs2ivx3QisB1wVGa+EniU/tjFt4xyjGAvYDNgI2Btml0VQ/V6PMfSd6+BiDiUZvfvKYNNwyzWkxojYi3gUOA/h5s9TFtPx9Lg6PPLmUTE82lC45TMPLM03zO4qVp+39ur+oDXA2+NiNtpdvPtRLMFMqnsaoH+GNOFwMLMvLxMn0ETJP00lgBvAm7LzMWZ+SRwJvA6+m88B400fn31voqIWcCewL757B+v9VONL6H5snBdeS9tDFwdEX9Bf9UJGBzQx5czKccKjgPmZeaXWmbNAWaV27OAs8a7tkGZeUhmbpyZAzRjd3Fm7gtcAry9LNbTGgEy8/fAgoh4aWnameay/H0zlsWdwA4RsVZ5/gfr7KvxbDHS+M0B3lXOCNoBeGhwl9Z4i+afwH0SeGtmPtYyaw4wMyJWj4jNaA4+X9GLGjPz15n5oswcKO+lhcB25XXbN2P5jMxc5X+A3WnOtvgdcGiv62mp6w00m6TXA9eWn91pjiFcBNxSfm/Q61pLvTsC55Tbm9O8CecD3wNW74P6tgXmlvH8IbB+P44l8Bngt8ANwMnA6v0wnsCpNMddnqT5YNtvpPGj2b3y9fKe+jXNWWK9qnE+zTGCwffQ0S3LH1pqvAnYrZdjOWT+7cDkXo7laD9eckSSVMVdVZKkKgaHJKmKwSFJqmJwSJKqGBySpCoGhySpisEhSapicEgdEBED5X98fLP8L40LImLNiLg0ImaUZSaXy0kQEe+OiB9GxNkRcVtEfDAiDioXYLwsIjbo6QOSRmFwSJ0zHfh6Zm4NPAj8nzGW3wb4R5pLeR8OPJbNBRh/Bbyrm4VKz4XBIXXObZl5bbl9FTAwxvKXZOYjmbmY5vLpZ5f2X7exrtQzBofUOY+33H6K5lLuS3n2fbbGKMs/3TL9dFlX6ksGh9RdtwOvKrffPspy0grD4JC664vAv0bEL4HJvS5G6gSvjitJquIWhySpisEhSapicEiSqhgckqQqBockqYrBIUmqYnBIkqr8f97OyAHQ+LKRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "_ = plt.hist(num_sens, bins = 50)\n",
    "plt.title('Distribution of num of sentence in one review')\n",
    "plt.xlabel('num')\n",
    "plt.ylabel('Counts')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:40:58.106565Z",
     "start_time": "2019-03-31T11:40:57.432770Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Counts')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XuYHVWZ7/Hvz4SEIJCAaT2QBBIkM8PlUYEIGfV4GOJAADXoASceHSLGyYjg/UIYnAERFGYccVCEE0wkIIeAiEMQMGa4OT5AIMg1REwbAmmJSSAhgMgl8J4/1mpS2dm7e/elurrb3+d59tNVq1ateteu2vvdVXt1bUUEZmZmZXpd1QGYmdng52RjZmalc7IxM7PSOdmYmVnpnGzMzKx0TjZmZlY6J5sBRNJFkv65l9raQ9Jzkobk+VslfaI32s7t3ShpRm+114XtniXpSUl/6Ott94SkD0hanffJARXGcYmks7qx3nOS9iojJhschlYdgCWSVgFvAjYDrwAPA5cCcyLiVYCI+GQX2vpERPxXozoR8TiwY8+ifm17ZwB7R8RHC+0f2RttdzGOccAXgT0jYl1fb7+HvgWcHBHXVh1Id0RErxxLA42kS4C2iPhq1bH0dz6z6V/eFxE7AXsC5wCnAHN7eyOSBuuHjD2BpwZgooEU+7K+2tggPgasv4oIP/rBA1gFvKem7GDgVWD/PH8JcFaeHg38DHga2AD8N+nDw2V5nT8BzwFfAcYDAcwEHgd+WSgbmtu7FfgmcBewCbgW2DUvO5T06W2beIGpwEvAy3l79xfa+0Sefh3wVeAxYB3pjG1kXtYex4wc25PAaR08TyPz+utze1/N7b8n9/nVHMclddY9FGgjnf2sA9YAJxSWvxZznv8Y8KvCfACfAlYAzwJfB94M3AE8A1wFDGsQd93nABie4w3gj8Dv6qz7NeC7eXq7XO9f8/wI4AVglzz/flLSejr3Z5+afXYK8ADwIunKxgHAr3N/rgQW0Mkx1qB/QTq7hXScXgBcn9tdAry5g33aWcxfyjFvyjFuX1j+XuC+vO7twFsabEPAefm535Tba39dDSedWT4OrAUuAkZ0dswAs0jH/Ut5H16Xy3cHfkI6Rh8FPlOI44x8nFyan5tlwKTC8nHANXndp4DvFZZ9HFgObAQWkc7gK3/vavo9ruoA/Mg7ok6yyeWPAyfm6UsKbwTfzC+K7fLjfwKq1xZb3tAvBV5PeoNqLysmm98D++c6PwF+lJcdSoNkk6fPaK9bWH4rW5LNx4FWYC/SpbtrgMtqYrs4x/VW0hvhPg2ep0tJiXCnvO5vgZmN4qxZ91DSZcoz83N2FPA8W96oX4s5z3+MbZPNQmBnYL8c5025XyNJlz5nNNh2w+eg0PbeDdY9DHgwT78D+B2wpLCsPcH/BSkR/W3u31fyNocV9tl9pDe0EcAwUvL7fK5/LOnNs9NjrE6MtclmA+nD0lDgcmBBg/Waifku0hv4rqQ320/mZQeSEsAhwBDSB5ZVwPA62zkCuAcYRUo8+wC75WXfyft1V9JxdR3wzSaPmUvan688/7q8nX/Jz+9ewErgiMJr5YXczpD8HN+Zlw0B7iclxdcD2wPvysuOyc/LPvk5/Spwe9XvW115+DJa//cE6UVQ62VgN9Knm5cj4r8jH5UdOCMi/hgRf2qw/LKIeCgi/gj8M/Ch9gEEPfQR4NsRsTIingNOBabXXMr5WkT8KSLuJ73g3lrbSI7l74BTI+LZiFgF/Dvw912I5WXgzPyc3UD6RPqXXVj/3Ih4JiKWAQ8Bv8j92gTcSDpTqKeZ56CRO4CJkt4AvJt0aXWMpB2B/wXcluv9HXB9RCyOiJdJn9ZHkBJUu/MjYnU+BiaT3kC/k5+Pq4G7C3W7c4y1uyYi7oqIzaRk87YG9ZqN+YmI2EBKBO1t/QPwfyNiSUS8EhHzSR8AJtfZzsukRPJXpIS5PCLWSFJu5/MRsSEingW+AUyvWbfZY+btQEtEnBkRL0XEStIHqWJ7v4qIGyLiFdKViPZj/WBSUv1yfp2+EBG/ysv+kZQAl+fn9BvA2yTt2SCOfsfJpv8bQ/qUWOvfSJ90fiFppaTZTbS1ugvLHyO9EY1uKsqO7Z7bK7Y9lDQgol1x9Njz1B+8MJotn8aLbY3pQixP5RdrZ9tqZG1h+k915hu11cxzUFdODEtJieXdpORyO/BOtk42W20j0sCS1Wz9/BT38e7A72sSSDHG7hxj7ZrZn83G3KitPYEvSnq6/UE6a9u9diMRcTPwPdLlvbWS5kjaGWgBdgDuKbTx81zerivHzJ7A7jUx/RMdH+vb5w8d44DHarZVbPc/Cm1uIJ2hdeXYr5STTT8m6e2kg+lXtcvyJ/svRsRewPuAL0ia0r64QZOdfSodV5jeg/SJ7knSZY4dCnENYesXY2ftPkF6sRTb3szWb9TNeDLHVNvW77vYTiNb9RP4H73ULvT8ObiNdMnsANLZx22kS0MHk76D22Yb+VP7OLZ+for7ag3pDEk1caWKHR9jvaWZmBtZDZwdEaMKjx0i4op6lSPi/Ig4iHQJ9C+AL5OOqT8B+xXaGBnNj66rPfZXA4/WxLRTRBzVZH/2aHC2uxr4x5p2R0TE7U3GWTknm35I0s6S3kv6svZHEfFgnTrvlbR3fnE+Qxou/UpevJZ0rbirPippX0k7kK5RX51P9X9L+vR1tKTtSNeLhxfWWwuMl9ToeLoC+LykCfnSzzeAKxt8gmsox3IVcLaknfIlhC8AP+pKOx24D/igpB0k7U0aUNFbevoc3AYcDzwcES+Rv18ivbGtz3WuAo6WNCXvpy+SLis1ekO6g5TwPiNpqKQPkpIX0Okx1lu6GnPRxcAnJR2i5PX5GN2ptqKkt+d67QMsXgBeyWdSFwPnSXpjrjtG0hFNxl/7WrsLeEbSKZJGSBoiaf/8wbEzd5E+AJyT+7K9pHfmZRcBp0raL8c4UtJxTcbYLzjZ9C/XSXqW9CnmNODbwAkN6k4E/ot0/fgO4PsRcWte9k3gq/mU+0td2P5lpC88/0D6cvIzAPn7iE8BPyB94vwjaYROux/nv09J+nWddufltn9JGp3zAvDpLsRV9Om8/ZWkM77/l9vvDeeRRhatBeaTvmvoLT19Dm4nfZfRfhbzcG6jfZ6IeAT4KPBd0if295GG079Ur8Fc/kHSQIiNpO9PrilU6egY6xVdjblm3aWk71u+l+NvJfWlnp1JSWUj6bLdU6TvhyCN0GsF7pT0DKnPzX6PNxfYN7/W/jN/IHof6XulR3OffkAaQNJZf9rX3Zs0MKiNtE+IiJ8C5wILcowPAX3+v2w90T56yczMrDQ+szEzs9I52ZiZWemcbMzMrHRONmZmVjrfjC8bPXp0jB8/vuowzMwGlHvuuefJiGjprJ6TTTZ+/HiWLl1adRhmZgOKpMc6r+XLaGZm1gecbMzMrHRONmZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6XwHgRKNn3193fJV5xzdx5GYmVXLZzZmZlY6JxszMyudk42ZmZXOycbMzErnZGNmZqVzsjEzs9I52ZiZWemcbMzMrHSlJRtJ8yStk/RQoezfJP1G0gOSfippVGHZqZJaJT0i6YhC+dRc1ippdqF8gqQlklZIulLSsFw+PM+35uXjy+qjmZk1p8wzm0uAqTVli4H9I+ItwG+BUwEk7QtMB/bL63xf0hBJQ4ALgCOBfYEP57oA5wLnRcREYCMwM5fPBDZGxN7AebmemZlVqLRkExG/BDbUlP0iIjbn2TuBsXl6GrAgIl6MiEeBVuDg/GiNiJUR8RKwAJgmScBhwNV5/fnAMYW25ufpq4Epub6ZmVWkyu9sPg7cmKfHAKsLy9pyWaPyNwBPFxJXe/lWbeXlm3J9MzOrSCXJRtJpwGbg8vaiOtWiG+UdtVUvjlmSlkpaun79+o6DNjOzbuvzZCNpBvBe4CMR0Z4E2oBxhWpjgSc6KH8SGCVpaE35Vm3l5SOpuZzXLiLmRMSkiJjU0tLS066ZmVkDfZpsJE0FTgHeHxHPFxYtBKbnkWQTgInAXcDdwMQ88mwYaRDBwpykbgGOzevPAK4ttDUjTx8L3FxIamZmVoHSfs9G0hXAocBoSW3A6aTRZ8OBxfk7+zsj4pMRsUzSVcDDpMtrJ0XEK7mdk4FFwBBgXkQsy5s4BVgg6SzgXmBuLp8LXCaplXRGM72sPpqZWXNKSzYR8eE6xXPrlLXXPxs4u075DcANdcpXkkar1Za/ABzXpWDNzKxU/qXOCvgXPM3sz41vV2NmZqVzsjEzs9I52ZiZWemcbMzMrHRONmZmVjonGzMzK52TjZmZlc7JxszMSud/6uwFjf5J08zMEp/ZmJlZ6ZxszMysdE42ZmZWOicbMzMrnZONmZmVzsnGzMxK52RjZmalc7IxM7PSOdmYmVnpnGzMzKx0TjZmZlY6JxszMyudk42ZmZXOycbMzEpXWrKRNE/SOkkPFcp2lbRY0or8d5dcLknnS2qV9ICkAwvrzMj1V0iaUSg/SNKDeZ3zJamjbZiZWXXKPLO5BJhaUzYbuCkiJgI35XmAI4GJ+TELuBBS4gBOBw4BDgZOLySPC3Pd9vWmdrINMzOrSGnJJiJ+CWyoKZ4GzM/T84FjCuWXRnInMErSbsARwOKI2BARG4HFwNS8bOeIuCMiAri0pq162zAzs4r09Xc2b4qINQD57xtz+RhgdaFeWy7rqLytTnlH29iGpFmSlkpaun79+m53yszMOtZfBgioTll0o7xLImJOREyKiEktLS1dXd3MzJrU18lmbb4ERv67Lpe3AeMK9cYCT3RSPrZOeUfbMDOzivR1slkItI8omwFcWyg/Po9KmwxsypfAFgGHS9olDww4HFiUlz0raXIehXZ8TVv1tmFmZhUZWlbDkq4ADgVGS2ojjSo7B7hK0kzgceC4XP0G4CigFXgeOAEgIjZI+jpwd653ZkS0Dzo4kTTibQRwY37QwTbMzKwipSWbiPhwg0VT6tQN4KQG7cwD5tUpXwrsX6f8qXrbMDOz6vSXAQJmZjaIOdmYmVnpnGzMzKx0TjZmZlY6JxszMyudk42ZmZXOycbMzErnZGNmZqVzsjEzs9I52ZiZWemcbMzMrHRONmZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6ZxszMysdE42ZmZWOicbMzMrnZONmZmVbmjVAdgW42df33DZqnOO7sNIzMx6VyVnNpI+L2mZpIckXSFpe0kTJC2RtELSlZKG5brD83xrXj6+0M6pufwRSUcUyqfmslZJs/u+h2ZmVtTnyUbSGOAzwKSI2B8YAkwHzgXOi4iJwEZgZl5lJrAxIvYGzsv1kLRvXm8/YCrwfUlDJA0BLgCOBPYFPpzrmplZRar6zmYoMELSUGAHYA1wGHB1Xj4fOCZPT8vz5OVTJCmXL4iIFyPiUaAVODg/WiNiZUS8BCzIdc3MrCJ9nmwi4vfAt4DHSUlmE3AP8HREbM7V2oAxeXoMsDqvuznXf0OxvGadRuXbkDRL0lJJS9evX9/zzpmZWV1VXEbbhXSmMQHYHXg96ZJXrWhfpcGyrpZvWxgxJyImRcSklpaWzkI3M7NuquIy2nuARyNifUS8DFwDvAMYlS+rAYwFnsjTbcA4gLx8JLChWF6zTqNyMzOrSBXJ5nFgsqQd8ncvU4CHgVuAY3OdGcC1eXphnicvvzkiIpdPz6PVJgATgbuAu4GJeXTbMNIggoV90C8zM2ugz//PJiKWSLoa+DWwGbgXmANcDyyQdFYum5tXmQtcJqmVdEYzPbezTNJVpES1GTgpIl4BkHQysIg00m1eRCzrq/6Zmdm2Kvmnzog4HTi9pnglaSRZbd0XgOMatHM2cHad8huAG3oeqZmZ9YYuX0aTtIukt5QRjJmZDU5NJRtJt0raWdKuwP3ADyV9u9zQzMxssGj2zGZkRDwDfBD4YUQcRBpVZmZm1qlmk81QSbsBHwJ+VmI8ZmY2CDWbbL5GGt3VGhF3S9oLWFFeWGZmNpg0OxptTUS8NiggIlb6OxszM2tWs2c2322yzMzMbBsdntlI+mvSrWRaJH2hsGhn0j9MmpmZdaqzy2jDgB1zvZ0K5c+w5dYyZmZmHeow2UTEbcBtki6JiMf6KCYzMxtkmh0gMFzSHGB8cZ2IOKyMoMzMbHBpNtn8GLgI+AHwSnnhmJnZYNRsstkcEReWGomZmQ1azQ59vk7SpyTtJmnX9kepkZmZ2aDR7JlN+4+XfblQFsBevRuOmZkNRk0lm4iYUHYgZmY2eDWVbCQdX688Ii7t3XDMzGwwavYy2tsL09sDU0g/6+xkY2ZmnWr2Mtqni/OSRgKXlRKRmZkNOl3+WejseWBibwZiZmaDV7Pf2VxHGn0G6Qac+wBXlRWUmZkNLs1+Z/OtwvRm4LGIaCshHjMzG4SauoyWb8j5G9Kdn3cBXiozKDMzG1yaSjaSPgTcBRwHfAhYIsk/MWBmZk1pdoDAacDbI2JGRBwPHAz8c3c3KmmUpKsl/UbSckl/nW+Bs1jSivx3l1xXks6X1CrpAUkHFtqZkeuvkDSjUH6QpAfzOudLUndjNTOznms22bwuItYV5p/qwrr1/Afw84j4K+CtwHJgNnBTREwEbsrzAEeSRr5NBGYBFwLke7OdDhxCSn6ntyeoXGdWYb2pPYjVzMx6qNmE8XNJiyR9TNLHgOuBG7qzQUk7A+8G5gJExEsR8TQwDZifq80HjsnT04BLI7kTGCVpN+AIYHFEbIiIjcBiYGpetnNE3BERQfrH0/a2zMysAh2ORpO0N/CmiPiypA8C7wIE3AFc3s1t7gWsB34o6a3APcBn83bWAETEGklvzPXHAKsL67flso7K2+qU1+vfLNIZEHvssUc3u2NmZp3p7MzmO8CzABFxTUR8ISI+Tzqr+U43tzkUOBC4MCIOAP7Ilktm9dT7viW6Ub5tYcSciJgUEZNaWlo6jtrMzLqts2QzPiIeqC2MiKWkn4jujjagLSKW5PmrSclnbb4ERv67rlB/XGH9scATnZSPrVNuZmYV6SzZbN/BshHd2WBE/AFYLekvc9EU4GFgIVt+N2cGcG2eXggcn0elTQY25ctti4DDJe2SBwYcDizKy56VNDmPQju+0JaZmVWgszsI3C3pHyLi4mKhpJmk71q669PA5ZKGASuBE0iJ76rc9uOk/+mBdMnuKKCVdE+2EwAiYoOkrwN353pnRsSGPH0icAkpId6YH2ZmVpHOks3ngJ9K+ghbksskYBjwge5uNCLuy+3UmlKnbgAnNWhnHjCvTvlSYP/uxmdmZr2rw2QTEWuBd0j6G7a8eV8fETeXHpmZmQ0azf6ezS3ALSXHYmZmg1RP7gJgZmbWFCcbMzMrnZONmZmVzsnGzMxK52RjZmalc7IxM7PSOdmYmVnpnGzMzKx0Tf1Tp1Vv/Ozr65avOufoPo7EzKzrfGZjZmalc7IxM7PSOdmYmVnpnGzMzKx0TjZmZlY6JxszMyudk42ZmZXOycbMzErnZGNmZqVzsjEzs9I52ZiZWemcbMzMrHSVJRtJQyTdK+lneX6CpCWSVki6UtKwXD48z7fm5eMLbZyayx+RdEShfGoua5U0u6/7ZmZmW6vyzOazwPLC/LnAeRExEdgIzMzlM4GNEbE3cF6uh6R9genAfsBU4Ps5gQ0BLgCOBPYFPpzrmplZRSpJNpLGAkcDP8jzAg4Drs5V5gPH5OlpeZ68fEquPw1YEBEvRsSjQCtwcH60RsTKiHgJWJDrmplZRao6s/kO8BXg1Tz/BuDpiNic59uAMXl6DLAaIC/flOu/Vl6zTqPybUiaJWmppKXr16/vaZ/MzKyBPk82kt4LrIuIe4rFdapGJ8u6Wr5tYcSciJgUEZNaWlo6iNrMzHqiil/qfCfwfklHAdsDO5POdEZJGprPXsYCT+T6bcA4oE3SUGAksKFQ3q64TqNyMzOrQJ+f2UTEqRExNiLGk77gvzkiPgLcAhybq80Ars3TC/M8efnNERG5fHoerTYBmAjcBdwNTMyj24blbSzsg66ZmVkDVZzZNHIKsEDSWcC9wNxcPhe4TFIr6YxmOkBELJN0FfAwsBk4KSJeAZB0MrAIGALMi4hlfdoTMzPbSqXJJiJuBW7N0ytJI8lq67wAHNdg/bOBs+uU3wDc0IuhmplZD/gOAmZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6ZxszMysdE42ZmZWOicbMzMrnZONmZmVzsnGzMxK52RjZmalc7IxM7PSOdmYmVnp+tOPp1k3jJ99fd3yVecc3ceRmJk15jMbMzMrnZONmZmVzsnGzMxK52RjZmalc7IxM7PSOdmYmVnpnGzMzKx0fZ5sJI2TdIuk5ZKWSfpsLt9V0mJJK/LfXXK5JJ0vqVXSA5IOLLQ1I9dfIWlGofwgSQ/mdc6XpL7up5mZbVHFmc1m4IsRsQ8wGThJ0r7AbOCmiJgI3JTnAY4EJubHLOBCSMkJOB04BDgYOL09QeU6swrrTe2DfpmZWQN9nmwiYk1E/DpPPwssB8YA04D5udp84Jg8PQ24NJI7gVGSdgOOABZHxIaI2AgsBqbmZTtHxB0REcClhbbMzKwClX5nI2k8cACwBHhTRKyBlJCAN+ZqY4DVhdXacllH5W11yuttf5akpZKWrl+/vqfdMTOzBipLNpJ2BH4CfC4inumoap2y6Eb5toURcyJiUkRMamlp6SxkMzPrpkqSjaTtSInm8oi4JhevzZfAyH/X5fI2YFxh9bHAE52Uj61TbmZmFaliNJqAucDyiPh2YdFCoH1E2Qzg2kL58XlU2mRgU77Mtgg4XNIueWDA4cCivOxZSZPzto4vtGVmZhWo4icG3gn8PfCgpPty2T8B5wBXSZoJPA4cl5fdABwFtALPAycARMQGSV8H7s71zoyIDXn6ROASYARwY36YmVlF+jzZRMSvqP+9CsCUOvUDOKlBW/OAeXXKlwL79yBMMzPrRb6DgJmZlc7JxszMSudkY2ZmpatigID1gfGzr69bvuqco/s4EjMzn9mYmVkfcLIxM7PSOdmYmVnpnGzMzKx0TjZmZlY6JxszMyudk42ZmZXOycbMzErnZGNmZqVzsjEzs9L5djV/ZnwbGzOrgs9szMysdE42ZmZWOicbMzMrnZONmZmVzsnGzMxK59FoBniUmpmVy2c2ZmZWOicbMzMrnS+jWYd8ec3MesOgPbORNFXSI5JaJc2uOh4zsz9ng/LMRtIQ4ALgb4E24G5JCyPi4WojGzx8xmNmXTEokw1wMNAaESsBJC0ApgFONiVrlIS6yknLbHAZrMlmDLC6MN8GHFJbSdIsYFaefU7SI93Y1mjgyW6s1x/1m77o3B6t3m/60QsGS18GSz/Afam1ZzOVBmuyUZ2y2KYgYg4wp0cbkpZGxKSetNFfDJa+DJZ+wODpy2DpB7gv3TVYBwi0AeMK82OBJyqKxczsz95gTTZ3AxMlTZA0DJgOLKw4JjOzP1uD8jJaRGyWdDKwCBgCzIuIZSVtrkeX4fqZwdKXwdIPGDx9GSz9APelWxSxzVcZZmZmvWqwXkYzM7N+xMnGzMxK52TTAwP5ljiSVkl6UNJ9kpbmsl0lLZa0Iv/dpeo465E0T9I6SQ8VyurGruT8vI8ekHRgdZFvrUE/zpD0+7xf7pN0VGHZqbkfj0g6opqo65M0TtItkpZLWibps7l8QO2XDvox4PaLpO0l3SXp/tyXr+XyCZKW5H1yZR5EhaTheb41Lx/fqwFFhB/deJAGHvwO2AsYBtwP7Ft1XF2IfxUwuqbsX4HZeXo2cG7VcTaI/d3AgcBDncUOHAXcSPrfq8nAkqrj76QfZwBfqlN333yMDQcm5GNvSNV9KMS3G3Bgnt4J+G2OeUDtlw76MeD2S35ud8zT2wFL8nN9FTA9l18EnJinPwVclKenA1f2Zjw+s+m+126JExEvAe23xBnIpgHz8/R84JgKY2koIn4JbKgpbhT7NODSSO4ERknarW8i7ViDfjQyDVgQES9GxKNAK+kY7BciYk1E/DpPPwssJ93JY0Dtlw760Ui/3S/5uX0uz26XHwEcBlydy2v3Sfu+uhqYIqneP8h3i5NN99W7JU5HB2V/E8AvJN2Tb9sD8KaIWAPpRQe8sbLouq5R7ANxP52cLy3NK1zKHDD9yJdfDiB9kh6w+6WmHzAA94ukIZLuA9YBi0lnXk9HxOZcpRjva33JyzcBb+itWJxsuq+pW+L0Y++MiAOBI4GTJL276oBKMtD204XAm4G3AWuAf8/lA6IfknYEfgJ8LiKe6ahqnbJ+0586/RiQ+yUiXomIt5HuonIwsE+9avlvqX1xsum+AX1LnIh4Iv9dB/yUdCCubb+Ukf+uqy7CLmsU+4DaTxGxNr9BvApczJZLMv2+H5K2I71BXx4R1+TiAbdf6vVjIO8XgIh4GriV9J3NKEnt/9BfjPe1vuTlI2n+Mm+nnGy6b8DeEkfS6yXt1D4NHA48RIp/Rq42A7i2mgi7pVHsC4Hj8+inycCm9ss6/VHN9xYfIO0XSP2YnkcMTQAmAnf1dXyN5Gv7c4HlEfHtwqIBtV8a9WMg7hdJLZJG5ekRwHtI30HdAhybq9Xuk/Z9dSxwc+TRAr2i6hETA/lBGlHzW9J10NOqjqcLce9FGkFzP7CsPXbS9dmbgBX5765Vx9og/itIlzJeJn0am9kodtKlgQvyPnoQmFR1/J3047Ic5wP5xb9bof5puR+PAEdWHX9NX95FuuTyAHBffhw10PZLB/0YcPsFeAtwb475IeBfcvlepITYCvwYGJ7Lt8/zrXn5Xr0Zj29XY2ZmpfNlNDMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6ZxszMysdE42ZhWRND7/bsrF+fdGfiFphKRbJU3KdUZLWpWnPybpPyVdJ+lRSSdL+oKkeyXdKWnXSjtk1gEnG7NqTQQuiIj9gKeB/91J/f2B/0O6N9fZwPMRcQBwB3B8mYGa9YSTjVm1Ho2I+/L0PcD4TurfEhF1x+anAAAAhElEQVTPRsR60i3gr8vlDzaxrlllnGzMqvViYfoVYCiwmS2vze07qP9qYf7VvK5Zv+RkY9b/rAIOytPHdlDPbMBwsjHrf74FnCjpdmB01cGY9Qbf9dnMzErnMxszMyudk42ZmZXOycbMzErnZGNmZqVzsjEzs9I52ZiZWemcbMzMrHT/H/39z4WsXNSYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "_ = plt.hist(num_words, bins = 50)\n",
    "plt.title('Distribution of num of words in one sentence')\n",
    "plt.xlabel('num')\n",
    "plt.ylabel('Counts')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:40:58.256964Z",
     "start_time": "2019-03-31T11:40:58.108570Z"
    }
   },
   "outputs": [],
   "source": [
    "word_count = {}\n",
    "def count_words(x):\n",
    "    for sen in x:\n",
    "        for word in sen:\n",
    "            word_count[word] = word_count.get(word, 0) + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:41:02.966499Z",
     "start_time": "2019-03-31T11:40:58.258969Z"
    }
   },
   "outputs": [],
   "source": [
    "_ = df_train.Processed_Reviews.apply(count_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:41:03.215379Z",
     "start_time": "2019-03-31T11:41:02.968505Z"
    }
   },
   "outputs": [],
   "source": [
    "df_word_count = pd.DataFrame(list(word_count.items()), columns=['word', 'count'])\n",
    "df_word_count.sort_values(by='count', ascending=False, inplace=True)\n",
    "df_word_count.set_index('word', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:41:03.351709Z",
     "start_time": "2019-03-31T11:41:03.218354Z"
    }
   },
   "outputs": [],
   "source": [
    "vocab_size = 20000\n",
    "df_word_count = df_word_count[:vocab_size-10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:41:03.505152Z",
     "start_time": "2019-03-31T11:41:03.353715Z"
    }
   },
   "outputs": [],
   "source": [
    "word2num_series = df_word_count.reset_index().reset_index().set_index('word')['index'] + 10 # reverse 10 pos for padding, [UNK], [CLS]..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:41:03.632456Z",
     "start_time": "2019-03-31T11:41:03.507122Z"
    }
   },
   "outputs": [],
   "source": [
    "def word_to_num(x):\n",
    "    res = []\n",
    "    for sen in x:\n",
    "        sen_words = []\n",
    "        for word in sen:\n",
    "            if word in word2num_series:\n",
    "                sen_words.append(word2num_series[word])\n",
    "        if len(sen_words) > 0:\n",
    "            res.append(np.array(sen_words, dtype=np.uint16))\n",
    "    return np.array(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:41:03.758794Z",
     "start_time": "2019-03-31T11:41:03.634469Z"
    }
   },
   "outputs": [],
   "source": [
    "def create_data(df):\n",
    "    X = df.Processed_Reviews.apply(word_to_num)\n",
    "    y = df.label\n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-31T11:43:39.501887Z",
     "start_time": "2019-03-31T11:41:03.769821Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train, y_train = create_data(df_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:50.809Z"
    }
   },
   "outputs": [],
   "source": [
    "X_val, y_val = create_data(df_validate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:50.993Z"
    }
   },
   "outputs": [],
   "source": [
    "X_test, y_test = create_data(df_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:51.148Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train.to_hdf('dataset/IMDB/X_train.h5', key='s', model='w')\n",
    "y_train.to_hdf('dataset/IMDB/y_train.h5', key='s', model='w')\n",
    "\n",
    "X_val.to_hdf('dataset/IMDB/X_val.h5', key='s', model='w')\n",
    "y_val.to_hdf('dataset/IMDB/y_val.h5', key='s', model='w')\n",
    "\n",
    "X_test.to_hdf('dataset/IMDB/X_test.h5', key='s', model='w')\n",
    "y_test.to_hdf('dataset/IMDB/y_test.h5', key='s', model='w')\n",
    "word2num_series.to_hdf('dataset/IMDB/word2num_series.h5', key='s', model='w')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Yelp Polarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:51.464Z"
    }
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv('dataset/yelp_review_polarity_csv/train.csv', header=None)\n",
    "df_test = pd.read_csv('dataset/yelp_review_polarity_csv/test.csv', header=None)\n",
    "df.columns = ['label', 'review']\n",
    "df['label'] = df['label'].map({1:0, 2:1})\n",
    "df_test.columns = ['label', 'review']\n",
    "df_test['label'] = df_test['label'].map({1:0, 2:1})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:51.636Z"
    }
   },
   "outputs": [],
   "source": [
    "df['Processed_Reviews'] = df['review'].apply(process)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:51.793Z"
    }
   },
   "outputs": [],
   "source": [
    "df_test['Processed_Reviews'] = df_test['review'].apply(process)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:51.963Z"
    }
   },
   "outputs": [],
   "source": [
    "df_train, df_validate = train_test_split(df, test_size=50000, random_state=0, stratify=df.label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:52.122Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set size:  510000\n",
      "Validation set size:  50000\n",
      "Test set size:  38000\n"
     ]
    }
   ],
   "source": [
    "print(\"Train set size: \", len(df_train))\n",
    "print(\"Validation set size: \", len(df_validate))\n",
    "print(\"Test set size: \", len(df_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:52.276Z"
    }
   },
   "outputs": [],
   "source": [
    "num_words = []\n",
    "def count(x):\n",
    "    for sen in x:\n",
    "        num_words.append(len(sen))\n",
    "num_sens = df_train.Processed_Reviews.apply(len)\n",
    "_ = df_train.Processed_Reviews.apply(count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:52.444Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Counts')"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XucHXV9//HX20QCKBBCgsUksEGiFXh4gRXSeimCQgA1tD+wQS0pYlMRtIq1hGILRWlDa0WpCL9IIgkFIuKFIMGQctXKbUFuIULWEMhCTAIJF0XBwKd/fL8Lk5Nzds9e5syyvp+Px3nszGe+853vzJlzPmdmvjujiMDMzKxMr6q6AWZmNvw52ZiZWemcbMzMrHRONmZmVjonGzMzK52TjZmZlc7J5hVA0vmS/mmQ6tpV0q8ljcjjN0j6xGDUneu7WtKMwaqvD8v9sqTHJf2q1cseCEl/Lml1fk/eXnV7qlK7X/4hGMzP9SuB/H821ZK0CngdsAl4AbgfWADMiYgX+1HXJyLif/owzw3Af0fEBX1ZVp73dGCPiPhYX+cdTJImAg8Cu0XEuirb0leSfgmcFBFXtHCZFwJdEfHFVi3TzEc2Q8MHI2I7YDdgNnAyMHewFyJp5GDXOUTsBjzxSks02W7AsqobYX03jD9P5YgIvyp8AauA99XE9gNeBPbO4xcCX87DY4EfAU8CG4CfkH40XJTn+S3wa+AfgDYggOOAR4CbCrGRub4bgH8DbgOeAq4AxuRpB5B+AW/RXmAq8Dzw+7y8uwv1fSIPvwr4IvAwsI50xLZDntbdjhm5bY8Dp/awnXbI86/P9X0x1/++vM4v5nZcWGfeA4Au4PO5HWuAYwvTX2pzHv9r4KeF8QA+BawAngG+BLwBuBl4GrgM2KpBu+tuA2BUbm8AvwF+WWdeAWfn+Z4C7insE6OAr+RttxY4H9imt/UFZub37Pm8/Ctz/PXA9/L2fQj4TKEdp+d1XJDXfxnQXpg+Efh+nvcJ4BuFaR8HlgMbgSWko89626l7fyjul18C/jcv8xpgbA/7x98AnaTPxCLg9TXv3yfz+7cROJd8VqefbXzp85TjU4CfkT6TdwMH5Ph0oKOmjs8Bi2o/13n8A8BduZ6fAW/J8WO736c83glcVhhfDbyt6u+yXr/rqm7AH/qLOskmxx8Bjs/DL+2UpMRwPvDq/Ho3L58O3ayuwodjAfAaYJsGH+pHgb1zme+RTqtBD8kmD5/eXbYw/QZeTjYfzx+M3YHXkr6QLqpp27dyu94KPAe8ucF2WkBKhNvleR8EjmvUzpp5DyCdpjwjb7PDgGeBHWvbnMf/mi2TzSJge2Cv3M5r83rtQDr1OaPBshtug0LdezSY9xDgDmA0KfG8GdglT/tabtOYvE2uBP6tyfW9kM2/5F6Vl/PPwFa5rSuBQwrv8+9yPSNI++AtedoI0hfs2aT9Z2vgXXnaEXnd3wyMJCXdnzVY1+79obhf/hJ4Y94/bgBmN5j3QNKPlX1ISfi/yImgsI1/lLfjrqSkOHUAbSx+nsaTEuxheTu+P4+PA7YlJcrJhTpuB6bX+VzvQ/phsH/epjNIn7VR+f14Mte/C+mHy6N5vt1JSfJVVX+X9fbyabSh6zHSF0mt35N2uN0i4vcR8ZPIe10PTo+I30TEbxtMvygi7ouI3wD/BHx4kC7UfhT4akSsjIhfA6cA02tOP/xLRPw2Iu4mfWm9tbaS3Ja/BE6JiGciYhXwn8Bf9aEtvwfOyNtsMelX/Zv6MP9ZEfF0RCwD7gOuyev1FHA10OjifjPboKc2bwf8MekHxfKIWCNJpF/yn4uIDRHxDPCvpF/S/VnfdwDjIuKMiHg+IlaSfgQU6/tpRCyOiBdIR9Hd79N+pKOiL+R97HcR8dM87W9JCXB5RGzKbXybpN2aWHeAb0fEg3m/vQx4W4NyHwXmRcSdEfEcaRv/iaS2QpnZEfFkRDwCXF+oqz9tLH6ePgYsztvmxYhYCnQAh0XEs6QfSEcDSJpMei8X1anzb4D/HxG3RsQLETGf9KNmSn4/nslt/jPS0dejkv44j/8k+nh9twpONkPXeNIpgVr/Qfoldo2klZJmNVHX6j5Mf5j0a3hsU63s2etzfcW6R5I6RHQr9h57lvTrv9ZY0i/u2rrG96EtT+Qvk96W1cjawvBv64w3qquZbVBXRFwHfIN02metpDmStuflX813SHpS0pPAj3O8W1/Wdzfg9d115fr+kZ7fp61zwpwIPFyzrGK9Xy/UuYF0hNbs+9bMvgE12zgn9SdqltOorv60sfh52Q04qmbbvYv0gxDgEnKyAT4C/DAnoVq7AZ+vqWdiXjeAG0lHrO/JwzeQEs2f5fEhz8lmCJL0DtLO/tPaafmX/ecjYnfgg8BJkg7qntygyt6OfCYWhncl/Sp+nHQtYdtCu0aw+Rdab/U+RvoQFevexOZf1M14PLeptq5H+1hPI5utJ/BHg1QvDHAbRMQ5EbEv6fTdG4EvkLbHb4G9ImJ0fu0QEc0mz9r3bTXwUKGu0RGxXUQc1kRdq4FdGxyprQb+tqbebSLiZ022s1mbbWNJrwF2orn9oz9tLG6/1aQzA8X5XxMRs/P0a4Cxkt5GSjqX9NCOM2vq2TYiLs3Tu5PNu/PwjTjZWH9J2l7SB4CFpGsh99Yp8wFJe+RTKU+Tuku/kCevJZ3D7auPSdpT0rak8/yX59MlD5J+wR4u6dWk89mjCvOtBdokNdqPLgU+J2mSpNeSTlF8p8Gv4IZyWy4DzpS0XT7FcRLw332ppwd3AX8haVtJe5AuAA+Wfm8DSe+QtH/e9r8hXTd5IZ8y+RZwtqSdc9nxkg5psk21+8ltwNOSTpa0jaQRkvbOP3p6cxupA8JsSa+RtLWkd+Zp5wOnSNort3EHSUc12ca+uAQ4VtLbJI0ibeNb8+nW3gy0jf8NfFDSIXm7bS3pAEkTAPL7fDnpjMQYYGmDer4FfDK/38rb8nBJ2+XpNwLvJXUC6SJ1DJpKSqo/70N7K+NkMzRcKekZ0q+bU4Gvknqg1DMZ+B/SOfibgW9GxA152r8BX8yH4X/fh+VfRLpY+SvSBd7PAOTrEZ8CLiD9SvwNqZdTt+/mv09IurNOvfNy3TeRejj9Dvh0H9pV9Om8/JWkI75Lcv2D4WxS76y1wHzg4kGqFwa2DbYnfQltJJ0meoLUAw1S9/hO4BZJT5P2iWavQc0F9sz7yQ9zMv8g6ZrAQ6QjpwtInR96VJh3D1Knli7S9TUi4gfAWcDC3Mb7gEObbGPTIuJa0rXG75ES3xvY/HpTT/MOqI0RsRqYRjrtuJ70Gf4Cm3+3XkLqNfndRj8yIqKDdN3mG6T3u5PUUaV7+oOkz/xP8vjTpM/C/+b3YMjzP3WamVnpfGRjZmalc7IxM7PSOdmYmVnpnGzMzKx0vpFcNnbs2Ghra6u6GWZmryh33HHH4xExrrdyTjZZW1sbHR0dVTfDzOwVRdLDvZfyaTQzM2sBJxszMyudk42ZmZWutGQjaZ6kdZLuq4l/WtIDkpZJ+vdC/BRJnXnaIYX41BzrLN7hON9r6lZJKyR9R9JWOT4qj3fm6W1lraOZmTWnzCObC0k3inuJpPeS7iP0lojYi3yfJ0l7ku5ltFee55v5pnYjSLdXPxTYEzg6l4V0P6OzI2Iy6V5C3TdPPA7YGBF7kO55dVZpa2hmZk0pLdlExE1s+TyW40kPMXoul+l+Zvw0YGFEPBcRD5FuQrdffnXmB089T7ob8rR8x+MDSXdThXTzxCMKdc3Pw5cDB+XyZmZWkVZfs3kj8O58euvGwi3Mx7P5A4m6cqxRfCfgycIdVLvjm9WVpz+Vy5uZWUVa/X82I4EdgSmkR9FeJml30pPxagX1k2H0UJ5epm1G0kxgJsCuu+7aY8PNzKz/Wn1k0wV8P5LbgBdJj/ztYvOnRU4gPX2vUfxxYHTh6YDdcYrz5Ok7UP/xykTEnIhoj4j2ceN6/QdYMzPrp1Yf2fyQdK3lBklvJD1X/nFgEXCJpK+Snrk9mfQEQAGTJU0iPbxrOvCRiAhJ1wNHkq7jzACuyMtYlMdvztOvi4oe2tM266q68VWzD29xS8zMqlVaspF0KemZ2WMldQGnkZ5aOC93h34emJETwTJJlwH3k57PfkL30+cknQgsAUYA8yJiWV7EyaSn632Z9FjUuTk+F7hIUifpiKapJ/aZmVl5Sks2EXF0g0kfa1D+TODMOvHFwOI68ZWk3mq18d8BZTzn3MzM+sl3EDAzs9I52ZiZWemcbMzMrHRONmZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6ZxszMysdE42ZmZWOicbMzMrnZONmZmVzsnGzMxK52RjZmalc7IxM7PSOdmYmVnpSks2kuZJWifpvjrT/l5SSBqbxyXpHEmdku6RtE+h7AxJK/JrRiG+r6R78zznSFKOj5G0NJdfKmnHstbRzMyaU+aRzYXA1NqgpInA+4FHCuFDgcn5NRM4L5cdA5wG7A/sB5xWSB7n5bLd83UvaxZwbURMBq7N42ZmVqHSkk1E3ARsqDPpbOAfgCjEpgELIrkFGC1pF+AQYGlEbIiIjcBSYGqetn1E3BwRASwAjijUNT8Pzy/EzcysIi29ZiPpQ8CjEXF3zaTxwOrCeFeO9RTvqhMHeF1ErAHIf3fuoT0zJXVI6li/fn0/1sjMzJrRsmQjaVvgVOCf602uE4t+xPskIuZERHtEtI8bN66vs5uZWZNaeWTzBmAScLekVcAE4E5Jf0Q6MplYKDsBeKyX+IQ6cYC1+TQb+e+6QV8TMzPrk5Ylm4i4NyJ2joi2iGgjJYx9IuJXwCLgmNwrbQrwVD4FtgQ4WNKOuWPAwcCSPO0ZSVNyL7RjgCvyohYB3b3WZhTiZmZWkTK7Pl8K3Ay8SVKXpON6KL4YWAl0At8CPgUQERuALwG359cZOQZwPHBBnueXwNU5Pht4v6QVpF5vswdzvczMrO9GllVxRBzdy/S2wnAAJzQoNw+YVyfeAexdJ/4EcFAfm2tmZiXyHQTMzKx0TjZmZlY6JxszMyudk42ZmZXOycbMzErnZGNmZqVzsjEzs9I52ZiZWemcbMzMrHRONmZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6ZxszMysdKU9qVPSPOADwLqI2DvH/gP4IPA86VHOx0bEk3naKcBxwAvAZyJiSY5PBb4OjAAuiIjZOT4JWAiMAe4E/ioinpc0ClgA7As8AfxlRKwqaz0B2mZdVWb1ZmaveGUe2VwITK2JLQX2joi3AA8CpwBI2hOYDuyV5/mmpBGSRgDnAocCewJH57IAZwFnR8RkYCMpUZH/boyIPYCzczkzM6tQackmIm4CNtTEromITXn0FmBCHp4GLIyI5yLiIaAT2C+/OiNiZUQ8TzqSmSZJwIHA5Xn++cARhbrm5+HLgYNyeTMzq0iV12w+Dlydh8cDqwvTunKsUXwn4MlC4uqOb1ZXnv5ULr8FSTMldUjqWL9+/YBXyMzM6qsk2Ug6FdgEXNwdqlMs+hHvqa4tgxFzIqI9ItrHjRvXc6PNzKzfSusg0IikGaSOAwdFRHcS6AImFopNAB7Lw/XijwOjJY3MRy/F8t11dUkaCexAzek8MzNrrZYe2eSeZScDH4qIZwuTFgHTJY3KvcwmA7cBtwOTJU2StBWpE8GinKSuB47M888ArijUNSMPHwlcV0hqZmZWgTK7Pl8KHACMldQFnEbqfTYKWJqv2d8SEZ+MiGWSLgPuJ51eOyEiXsj1nAgsIXV9nhcRy/IiTgYWSvoy8HNgbo7PBS6S1Ek6ople1jqamVlzSks2EXF0nfDcOrHu8mcCZ9aJLwYW14mvJPVWq43/DjiqT401M7NS+Q4CZmZWOicbMzMrnZONmZmVzsnGzMxK52RjZmalc7IxM7PSOdmYmVnpnGzMzKx0TjZmZlY6JxszMyudk42ZmZXOycbMzErnZGNmZqVzsjEzs9I52ZiZWemcbMzMrHRONmZmVrrSko2keZLWSbqvEBsjaamkFfnvjjkuSedI6pR0j6R9CvPMyOVXSJpRiO8r6d48zznKz5lutAwzM6tOmUc2FwJTa2KzgGsjYjJwbR4HOBSYnF8zgfMgJQ7gNGB/0iOgTyskj/Ny2e75pvayDDMzq0hpySYibgI21ISnAfPz8HzgiEJ8QSS3AKMl7QIcAiyNiA0RsRFYCkzN07aPiJsjIoAFNXXVW4aZmVWk1ddsXhcRawDy351zfDywulCuK8d6infVife0jC1ImimpQ1LH+vXr+71SZmbWs6HSQUB1YtGPeJ9ExJyIaI+I9nHjxvV1djMza1Krk83afAqM/HddjncBEwvlJgCP9RKfUCfe0zLMzKwirU42i4DuHmUzgCsK8WNyr7QpwFP5FNgS4GBJO+aOAQcDS/K0ZyRNyb3Qjqmpq94yzMysIiPLqljSpcABwFhJXaReZbOByyQdBzwCHJWLLwYOAzqBZ4FjASJig6QvAbfncmdERHeng+NJPd62Aa7OL3pYhpmZVaS0ZBMRRzeYdFCdsgGc0KCeecC8OvEOYO868SfqLcPMzKozVDoImJnZMNbnZJOvn7yljMaYmdnw1FSykXSDpO3zf/TfDXxb0lfLbZqZmQ0XzV6z2SEinpb0CeDbEXGapHvKbNhw1jbrqrrxVbMPb3FLzMxao9nTaCPz/6x8GPhRie0xM7NhqNlk8y+k/3npjIjbJe0OrCivWWZmNpw0exptTUS81CkgIlb6mo2ZmTWr2SOb/2oyZmZmtoUej2wk/Qnwp8A4SScVJm0PjCizYWZmNnz0dhptK+C1udx2hfjTwJFlNcrMzIaXHpNNRNwI3Cjpwoh4uEVtMjOzYabZDgKjJM0B2orzRMSBZTTKzMyGl2aTzXeB84ELgBfKa46ZmQ1HzSabTRFxXqktMTOzYavZrs9XSvqUpF0kjel+ldoyMzMbNpo9sul+8uUXCrEAdh/c5piZ2XDUVLKJiEllN8TMzIavZh8xcEy9V38XKulzkpZJuk/SpZK2ljRJ0q2SVkj6jqStctlRebwzT28r1HNKjj8g6ZBCfGqOdUqa1d92mpnZ4Gj2ms07Cq93A6cDH+rPAiWNBz4DtEfE3qQ7EUwHzgLOjojJwEbguDzLccDGiNgDODuXQ9Keeb69gKnANyWNkDQCOBc4FNgTODqXNTOzijR7Gu3TxXFJOwAXDXC520j6PbAtsAY4EPhInj6flNDOA6blYYDLgW9IUo4vjIjngIckdQL75XKdEbEyt3VhLnv/ANprZmYD0OfHQmfPApP7M2NEPAp8BXiElGSeAu4AnoyITblYFzA+D48HVud5N+XyOxXjNfM0im9B0kxJHZI61q9f35/VMTOzJjR1ZCPpSlLvM0invd4MXNafBUrakXSkMQl4kvQPo4fWKdq9PDWY1iheL4FGnRgRMQeYA9De3l63jJmZDVyzXZ+/UhjeBDwcEV39XOb7gIciYj2ApO+T7iw9WtLIfPQyAXgsl+8CJgJdkkYCOwAbCvFuxXkaxc3MrAJNnUbLN+T8BenOzzsCzw9gmY8AUyRtm6+9HES6nnI9L99JegZwRR5exMv/53MkcF1ERI5Pz73VJpFO690G3A5Mzr3btiJ1Ilg0gPaamdkANdv1+cOkL/KjgA8Dt0rq1yMGIuJW0oX+O4F7cxvmACcDJ+UL/TsBc/Msc4GdcvwkYFauZxnpVN79wI+BEyLihXxkdCLpMdbLgctyWTMzq4jSQUIvhaS7gfdHxLo8Pg74n4h4a8nta5n29vbo6Ojo17xts64alDasmn34oNRjZtYqku6IiPbeyjXbG+1V3Ykme6IP85qZ2R+4ZjsI/FjSEuDSPP6XwOJymmRmZsNNj8lG0h7A6yLiC5L+AngXqcvxzcDFLWifmZkNA72dCvsa8AxARHw/Ik6KiM+Rjmq+VnbjzMxseOgt2bRFxD21wYjoID0i2szMrFe9JZute5i2zWA2xMzMhq/eks3tkv6mNijpONL9zMzMzHrVW2+0zwI/kPRRXk4u7cBWwJ+X2TAzMxs+ekw2EbEW+FNJ7wX2zuGrIuK60ltmZmbDRrPPs7medO8yMzOzPvNdAMzMrHRONmZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0lWSbCSNlnS5pF9IWi7pTySNkbRU0or8d8dcVpLOkdQp6R5J+xTqmZHLr5A0oxDfV9K9eZ5zJKmK9TQzs6SqI5uvAz+OiD8G3gosB2YB10bEZODaPA5wKDA5v2YC5wFIGgOcBuwP7Aec1p2gcpmZhfmmtmCdzMysgZYnG0nbA+8B5gJExPMR8SQwDZifi80HjsjD04AFkdwCjJa0C3AIsDQiNkTERmApMDVP2z4ibo6IABYU6jIzswpUcWSzO7Ae+Lakn0u6QNJrSE8EXQOQ/+6cy48HVhfm78qxnuJddeJbkDRTUoekjvXr1w98zczMrK4qks1IYB/gvIh4O/AbXj5lVk+96y3Rj/iWwYg5EdEeEe3jxo3rudVmZtZvVSSbLqArIm7N45eTks/afAqM/HddofzEwvwTgMd6iU+oEzczs4q0PNlExK+A1ZLelEMHAfcDi4DuHmUzgCvy8CLgmNwrbQrwVD7NtgQ4WNKOuWPAwcCSPO0ZSVNyL7RjCnWZmVkFmnrEQAk+DVwsaStgJXAsKfFdlp8C+ghwVC67GDgM6ASezWWJiA2SvgTcnsudEREb8vDxwIWkR1dfnV9mZlaRSpJNRNxFeuJnrYPqlA3ghAb1zAPm1Yl38PLD3szMrGK+g4CZmZXOycbMzErnZGNmZqVzsjEzs9JV1RvN6mibdVXDaatmH97ClpiZDS4f2ZiZWemcbMzMrHRONmZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6ZxszMysdE42ZmZWOicbMzMrXWXJRtIIST+X9KM8PknSrZJWSPpOfmQ0kkbl8c48va1Qxyk5/oCkQwrxqTnWKWlWq9fNzMw2V+WRzd8BywvjZwFnR8RkYCNwXI4fB2yMiD2As3M5JO0JTAf2AqYC38wJbARwLnAosCdwdC5rZmYVqSTZSJoAHA5ckMcFHAhcnovMB47Iw9PyOHn6Qbn8NGBhRDwXEQ8BncB++dUZESsj4nlgYS5rZmYVqerI5mvAPwAv5vGdgCcjYlMe7wLG5+HxwGqAPP2pXP6leM08jeJmZlaRlicbSR8A1kXEHcVwnaLRy7S+xuu1ZaakDkkd69ev76HVZmY2EFUc2bwT+JCkVaRTXAeSjnRGS+p+cugE4LE83AVMBMjTdwA2FOM18zSKbyEi5kREe0S0jxs3buBrZmZmdbU82UTEKRExISLaSBf4r4uIjwLXA0fmYjOAK/LwojxOnn5dRESOT8+91SYBk4HbgNuBybl321Z5GYtasGpmZtbAyN6LtMzJwEJJXwZ+DszN8bnARZI6SUc00wEiYpmky4D7gU3ACRHxAoCkE4ElwAhgXkQsa+mamJnZZipNNhFxA3BDHl5J6klWW+Z3wFEN5j8TOLNOfDGweBCbamZmA+A7CJiZWemcbMzMrHRONmZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpRtK/9RpPWibdVXd+KrZh7e4JWZmfecjGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6ZxszMysdE42ZmZWOicbMzMrXcuTjaSJkq6XtFzSMkl/l+NjJC2VtCL/3THHJekcSZ2S7pG0T6GuGbn8CkkzCvF9Jd2b5zlHklq9nmZm9rIqjmw2AZ+PiDcDU4ATJO0JzAKujYjJwLV5HOBQYHJ+zQTOg5ScgNOA/UmPkz6tO0HlMjML801twXqZmVkDLU82EbEmIu7Mw88Ay4HxwDRgfi42HzgiD08DFkRyCzBa0i7AIcDSiNgQERuBpcDUPG37iLg5IgJYUKjLzMwqUOk1G0ltwNuBW4HXRcQaSAkJ2DkXGw+sLszWlWM9xbvqxM3MrCKVJRtJrwW+B3w2Ip7uqWidWPQjXq8NMyV1SOpYv359b002M7N+qiTZSHo1KdFcHBHfz+G1+RQY+e+6HO8CJhZmnwA81kt8Qp34FiJiTkS0R0T7uHHjBrZSZmbWUMufZ5N7hs0FlkfEVwuTFgEzgNn57xWF+ImSFpI6AzwVEWskLQH+tdAp4GDglIjYIOkZSVNIp+eOAf6r9BWriJ9zY2avBFU8PO2dwF8B90q6K8f+kZRkLpN0HPAIcFSethg4DOgEngWOBchJ5UvA7bncGRGxIQ8fD1wIbANcnV9mZlaRliebiPgp9a+rABxUp3wAJzSoax4wr068A9h7AM00M7NB5DsImJlZ6ZxszMysdE42ZmZWOicbMzMrnZONmZmVzsnGzMxKV8X/2VgL+J89zWwo8ZGNmZmVzsnGzMxK52RjZmalc7IxM7PSuYPAHxh3HDCzKvjIxszMSudkY2ZmpXOyMTOz0vmajQG+lmNm5fKRjZmZlc5HNtYjH/GY2WAYtslG0lTg68AI4IKImF1xk4YVJyEz64thmWwkjQDOBd4PdAG3S1oUEfdX27Lhz0nIzOoZlskG2A/ojIiVAJIWAtMAJ5uKNEpCreBEZ1a94ZpsxgOrC+NdwP61hSTNBGbm0V9LeqCfyxsLPN7PecvkdgE6q+mi3l7NG4ptArerrwajXbs1U2i4JhvVicUWgYg5wJwBL0zqiIj2gdYz2NyuvnG7mjcU2wRuV1+1sl3DtetzFzCxMD4BeKyitpiZ/cEbrsnmdmCypEmStgKmA4sqbpOZ2R+sYXkaLSI2SToRWELq+jwvIpaVuMgBn4oridvVN25X84Zim8Dt6quWtUsRW1zKMDMzG1TD9TSamZkNIU42ZmZWOiebAZI0VdIDkjolzaqoDRMlXS9puaRlkv4ux0+X9Kiku/LrsAratkrSvXn5HTk2RtJSSSvy3x1b3KY3FbbJXZKelvTZKraXpHmS1km6rxCru32UnJP3tXsk7dPidv2HpF/kZf9A0ugcb5P028J2O7/2HL30AAAEpElEQVTF7Wr4vkk6JW+vByQd0uJ2fafQplWS7srxlmyvHr4Xqtm/IsKvfr5InQ9+CewObAXcDexZQTt2AfbJw9sBDwJ7AqcDf1/xNloFjK2J/TswKw/PAs6q+D38Fekf01q+vYD3APsA9/W2fYDDgKtJ/0c2Bbi1xe06GBiZh88qtKutWK6C7VX3fcufgbuBUcCk/Fkd0ap21Uz/T+CfW7m9evheqGT/8pHNwLx0W5yIeB7ovi1OS0XEmoi4Mw8/Aywn3UVhqJoGzM/D84EjKmzLQcAvI+LhKhYeETcBG2rCjbbPNGBBJLcAoyXt0qp2RcQ1EbEpj95C+v+1lmqwvRqZBiyMiOci4iGgk/SZbWm7JAn4MHBpGcvuoU2Nvhcq2b+cbAam3m1xKv2Sl9QGvB24NYdOzIfE81p9uioL4BpJdyjdHgjgdRGxBtIHAti5gnZ1m87mXwJVby9ovH2G0v72cdKv4G6TJP1c0o2S3l1Be+q9b0Nle70bWBsRKwqxlm6vmu+FSvYvJ5uBaeq2OK0i6bXA94DPRsTTwHnAG4C3AWtIh/Kt9s6I2Ac4FDhB0nsqaENdSv/w+yHguzk0FLZXT4bE/ibpVGATcHEOrQF2jYi3AycBl0javoVNavS+DYntBRzN5j9oWrq96nwvNCxaJzZo28vJZmCGzG1xJL2atENdHBHfB4iItRHxQkS8CHyLkk4h9CQiHst/1wE/yG1Y2314nv+ua3W7skOBOyNibW5j5dsra7R9Kt/fJM0APgB8NPKJ/nya6ok8fAfp2sgbW9WmHt63obC9RgJ/AXynO9bK7VXve4GK9i8nm4EZErfFyeeE5wLLI+KrhXjxfOufA/fVzltyu14jabvuYdIF5vtI22hGLjYDuKKV7SrY7Bdn1duroNH2WQQck3sNTQGe6j4d0gpKDyQ8GfhQRDxbiI9TeoYUknYHJgMrW9iuRu/bImC6pFGSJuV23daqdmXvA34REV3dgVZtr0bfC1S1f5XdI2K4v0g9OB4k/To5taI2vIt0uHsPcFd+HQZcBNyb44uAXVrcrt1JvYHuBpZ1bx9gJ+BaYEX+O6aCbbYt8ASwQyHW8u1FSnZrgN+Tflke12j7kE5znJv3tXuB9ha3q5N0Tr97Hzs/l/1/+f29G7gT+GCL29XwfQNOzdvrAeDQVrYrxy8EPllTtiXbq4fvhUr2L9+uxszMSufTaGZmVjonGzMzK52TjZmZlc7JxszMSudkY2ZmpXOyMTOz0jnZmJlZ6ZxszCqSn2uyXNK38vNGrpG0jaQbJLXnMmMlrcrDfy3ph5KulPSQpBMlnZRv6HiLpDGVrpBZD5xszKo1GTg3IvYCniT9d3lP9gY+Qrr/15nAs5Fu6HgzcEyZDTUbCCcbs2o9FBF35eE7SA/W6sn1EfFMRKwHngKuzPF7m5jXrDJONmbVeq4w/AIwknT7/u7P5tY9lH+xMP5intdsSHKyMRt6VgH75uEjK2yH2aBxsjEber4CHC/pZ8DYqhtjNhh812czMyudj2zMzKx0TjZmZlY6JxszMyudk42ZmZXOycbMzErnZGNmZqVzsjEzs9L9H5ajleoAhfhTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "_ = plt.hist(num_sens, bins = 50)\n",
    "plt.title('Distribution of num of sentence in one review')\n",
    "plt.xlabel('num')\n",
    "plt.ylabel('Counts')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:52.608Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Counts')"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAAEWCAYAAADPZygPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3X+YXVV97/H3x8RAUEISEigmkYQSWwKPCowQa+ulxEIANdQLCrUltWlTKagVq4Rqi6JWuPUapCo2QkxCvcQ0ogQBY8ov6wMEJvIzpJgRkIwgGUgIKAoEvvePtU7YmZwzc2aSmTWT+bye5zxn7+9ee621z+w537P3WWdvRQRmZmYlvKp0B8zMbOhyEjIzs2KchMzMrBgnITMzK8ZJyMzMinESMjOzYpyEdhOSvi7pn3ZRXa+X9CtJw/L8zZL+elfUneu7XtLsXVVfD9r9nKQnJf2yv9veGZL+VNKG/Dc5vGA/Fkn6XC/W+5Wkg/qiTzb4DS/dAeuepEeA/YGtwEvAA8ASYEFEvAwQER/sQV1/HRH/1ahMRDwKvHbner2tvU8DB0fEn1fqP2FX1N3DfkwCPgYcGBEb+7v9nfRF4OyIuLp0R3ojInbJvjTYSFoEtEfEp0r3ZSDzkdDg8a6I2Bs4ELgQOBe4fFc3Iml3/WByIPDUIExAkPq+tr8a2433ARuIIsKPAf4AHgHe0Sl2FPAycFieXwR8Lk+PA74PPA1sAv6b9IHjirzOb4BfAZ8AJgMBzAEeBX5UiQ3P9d0MfAG4A9gCXA2MzcuOIX3a26G/wEzgBeDF3N49lfr+Ok+/CvgU8HNgI+kIb5+8rNaP2blvTwKf7OJ12iev35Hr+1Su/x15m1/O/VhUZ91jgHbS0dJG4HHgA5Xl2/qc5/8S+HFlPoC/A9YDzwKfBX4XuA14BlgGjGjQ77qvAbBH7m8AvwZ+VmfdzwD/lqdfncv9nzw/EvgtMCbPv5uUzJ7O23NIp7/ZucC9wPOksySHAz/J2/NtYCnd7GMNti9IR8OQ9tOvAtfmelcDv9vF37S7Pv9D7vOW3Mc9K8vfCdyd170VeGODNgTMz6/9llxf7f9qD9KR6KPAE8DXgZHd7TPAXNJ+/0L+G16T468DvkPaRx8GPlzpx6fzfrIkvzZrgZbK8knAVXndp4CvVJb9FbAO2AysJB3xF3/vaur9rXQH/Gjij1QnCeX4o8CZeXpR5Q3iC/mf5dX58UeA6tXFK2/0S4DXkN64arFqEvoFcFgu8x3gP/KyY2iQhPL0p2tlK8tv5pUk9FdAG3AQ6RTgVcAVnfr2jdyvN5HeIA9p8DotISXIvfO6PwXmNOpnp3WPIZ3uvCC/ZicCz/HKG/i2Puf5v2THJLQCGAUcmvt5Q96ufUinUGc3aLvha1Cp++AG6x4L3Jen/wD4GbC6sqyW+N9ASlB/krfvE7nNEZW/2d2kN7qRwAhSUvxoLn8K6U21232sTh87J6FNpA9Rw4FvAUsbrNdMn+8gvbGPJb0JfzAvO4KUGI4GhpE+yDwC7FGnneOBNcBoUkI6BDggL7s4/13Hkvara4AvNLnPLKq9Xnn+Vbmdf86v70HAQ8Dxlf+V3+Z6huXX+Pa8bBhwDylZvgbYE/jDvOzk/Lockl/TTwG3ln7favbh03GD22Okf47OXgQOIH0aejEi/jvy3tqFT0fEryPiNw2WXxER90fEr4F/At5bG7iwk94PfCkiHoqIXwHnAad1OiX0mYj4TUTcQ/pHfFPnSnJf3gecFxHPRsQjwP8F/qIHfXkRuCC/ZteRPsH+Xg/WvyginomItcD9wA/zdm0BricdWdTTzGvQyG3AVEn7Am8nnaKdIOm1wP8Cbsnl3gdcGxGrIuJF0qf7kaTEVXNJRGzI+8B00hvrxfn1WA7cWSnbm32s5qqIuCMitpKS0JsblGu2z49FxCZSgqjV9TfAv0fE6oh4KSIWkz4YTK/TzoukBPP7pES6LiIel6Rcz0cjYlNEPAv8C3Bap3Wb3WfeAoyPiAsi4oWIeIj0Aata348j4rqIeIl05qK2rx9FSrYfz/+nv42IH+dlf0tKjOvya/ovwJslHdigHwOKk9DgNoH0qbKzfyV9MvqhpIckzWuirg09WP5z0hvUuKZ62bXX5fqqdQ8nDcSoqY5me476gybG8cqn92pdE3rQl6fyP3F3bTXyRGX6N3XmG9XVzGtQV04YraSE83ZS0rkVeBvbJ6Ht2og0oGUD278+1b/x64BfdEos1T72Zh+raebv2WyfG9V1IPAxSU/XHqSjvNd1biQibgS+QjpN+ISkBZJGAeOBvYA1lTp+kOM1PdlnDgRe16lP/0jX+/qe+cPIJODnndqq1vvlSp2bSEd0Pdn3i3ESGqQkvYW0k/2487J8JPCxiDgIeBdwjqQZtcUNquzuU+ykyvTrSZ8AnySdLtmr0q9hbP9P2l29j5H+iap1b2X7N/BmPJn71LmuX/Swnka2207gd3ZRvbDzr8EtpFNvh5OOVm4hnWI6ivQd3w5t5E/5k9j+9an+rR4nHVGpU79Swa73sV2lmT43sgH4fESMrjz2iogr6xWOiEsi4kjSqdQ3AB8n7VO/AQ6t1LFPND/ar/O+vwF4uFOf9o6IE5vcntc3ODreAPxtp3pHRsStTfazKCehQUbSKEnvJH1J/B8RcV+dMu+UdHD+p32GNKz7pbz4CdK56J76c0nTJO1FOge+PJ8y+Cnp09pJkl5NOh+9R2W9J4DJkhrta1cCH5U0JZ9C+hfg2w0+8TWU+7IM+LykvfOpiHOA/+hJPV24G3iPpL0kHUwayLGr7OxrcAtwBvBARLxA/v6K9IbXkcssA06SNCP/nT5GOj3V6I3qNlIi/LCk4ZLeQ0pqQLf72K7S0z5XfQP4oKSjlbwm76N7dy4o6S25XG1gx2+Bl/KR1zeA+ZL2y2UnSDq+yf53/l+7A3hG0rmSRkoaJumw/IGyO3eQPhhcmLdlT0lvy8u+Dpwn6dDcx30kndpkH4tzEho8rpH0LOlTzyeBLwEfaFB2KvBfpPPTtwFfi4ib87IvAJ/Kh+7/0IP2ryB90fpL0peiHwbI33f8HXAZ6RPqr0kjhmr+Mz8/JekndepdmOv+EWm00G+BD/WgX1Ufyu0/RDpC/H+5/l1hPmmk0xPAYtJ3GbvKzr4Gt5K+K6kd9TyQ66jNExEPAn8O/BvpE/67SMP+X6hXYY6/hzQAYzPp+5mrKkW62sd2iZ72udO6raTvc76S+99G2pZ6RpGSzWbS6b+nSN8/QRox2AbcLukZ0jY3+z3h5cC0/L/2vfxB6V2k760eztt0GWngSnfbU1v3YNKApHbS34SI+C5wEbA09/F+oN9/i9dbtRFTZmZm/c5HQmZmVoyTkJmZFeMkZGZmxTgJmZlZMb5QYTfGjRsXkydPLt0NM7NBZc2aNU9GxPjuyvVZEpK0kHQBwY0RcVgl/iHgbNJvEK6NiE/k+Hmk3168RLqo38ocnwl8mXTtpMsi4sIcn0L6rcxY0kUW/yIiXpC0B+kaYkeShlq+L1/CpWEbXZk8eTKtra07+WqYmQ0tkn7efam+PR23iHQV5W0k/TEwi3Q120PJY/ElTSNdP+nQvM7X8g+5hpEupXECMA04PZeFNC5+fkRMJY3vr/14cA6wOSIOJv2246Ku2uiD7TYzsyb1WRKKiB+x43XNzgQujIjnc5navV1mka6k+3xEPEz6cdhR+dGWL+z4AunIZ1b+lfaxwPK8/mLSlWRrdS3O08uBGbl8ozbMzKyQ/h6Y8AbgjyStlnRL5XIVE9j+4ontOdYovi/wdOWyJrX4dnXl5Vty+UZ17UDSXEmtklo7OjrqFTEzs12gv5PQcGAM6XLqHweW5aMU1SkbvYjTy3W2D0YsiIiWiGgZP77b79XMzKyX+jsJtZPuJRIRcQfpTpfjcrx6leaJpCvoNoo/CYyuXFG2Fqe6Tl6+D+m0YKO6zMyskP5OQt8jfZeDpDeQ7v/yJOnOhadJ2iOPeptKumrsnaQbdk2RNII0sGBFvsfJTaS7PUK6a+LVeXpFnicvvzGXb9SGmZkV0pdDtK8k3f52nKR24HzS1YIXSrqfdEXi2TlBrJW0jHT1363AWfmqsUg6m3TP9GHAwkh3rYR0ddulkj4H3EW6Yi35+QpJbaQjoNMAIqJhG2ZmVoavot2NlpaW8O+EzMx6RtKaiGjprpwv22NmZsX4sj19aPK8a7dNP3LhSQV7YmY2MPlIyMzMinESMjOzYpyEzMysGCchMzMrxknIzMyK8ei4flIdKQceLWdmBj4SMjOzgpyEzMysGCchMzMrxknIzMyKcRIyM7NinITMzKwYJyEzMyvGScjMzIpxEjIzs2KchMzMrJg+S0KSFkraKOn+Osv+QVJIGpfnJekSSW2S7pV0RKXsbEnr82N2JX6kpPvyOpdIUo6PlbQql18laUx3bZiZWRl9eSS0CJjZOShpEvAnwKOV8AnA1PyYC1yay44FzgeOBo4Czq8llVxmbmW9WlvzgBsiYipwQ55v2IaZmZXTZ0koIn4EbKqzaD7wCSAqsVnAkkhuB0ZLOgA4HlgVEZsiYjOwCpiZl42KiNsiIoAlwMmVuhbn6cWd4vXaMDOzQvr1OyFJ7wZ+ERH3dFo0AdhQmW/Psa7i7XXiAPtHxOMA+Xm/btqo18+5kloltXZ0dDS5dWZm1lP9loQk7QV8EvjneovrxKIX8S670Ow6EbEgIloiomX8+PHdVGtmZr3Vn0dCvwtMAe6R9AgwEfiJpN8hHZVMqpSdCDzWTXxinTjAE7XTbPl5Y443qsvMzArptyQUEfdFxH4RMTkiJpOSwhER8UtgBXBGHsE2HdiST6WtBI6TNCYPSDgOWJmXPStpeh4VdwZwdW5qBVAbRTe7U7xeG2ZmVkif3VlV0pXAMcA4Se3A+RFxeYPi1wEnAm3Ac8AHACJik6TPAnfmchdERG2ww5mkEXgjgevzA+BCYJmkOaQReKd21YaZmZXTZ0koIk7vZvnkynQAZzUotxBYWCfeChxWJ/4UMKNOvGEbZmZWhq+YYGZmxTgJmZlZMU5CZmZWjJOQmZkV4yRkZmbFOAmZmVkxTkJmZlaMk5CZmRXjJGRmZsU4CZmZWTFOQmZmVoyTkJmZFeMkZGZmxTgJmZlZMU5CZmZWjJOQmZkV4yRkZmbFOAmZmVkxfZaEJC2UtFHS/ZXYv0r6H0n3SvqupNGVZedJapP0oKTjK/GZOdYmaV4lPkXSaknrJX1b0ogc3yPPt+Xlk7trw8zMyujLI6FFwMxOsVXAYRHxRuCnwHkAkqYBpwGH5nW+JmmYpGHAV4ETgGnA6bkswEXA/IiYCmwG5uT4HGBzRBwMzM/lGraxqzfazMya12dJKCJ+BGzqFPthRGzNs7cDE/P0LGBpRDwfEQ8DbcBR+dEWEQ9FxAvAUmCWJAHHAsvz+ouBkyt1Lc7Ty4EZuXyjNszMrJCS3wn9FXB9np4AbKgsa8+xRvF9gacrCa0W366uvHxLLt+orh1ImiupVVJrR0dHrzbOzMy6VyQJSfoksBX4Vi1Up1j0It6bunYMRiyIiJaIaBk/fny9ImZmtgsM7+8GJc0G3gnMiIhaEmgHJlWKTQQey9P14k8CoyUNz0c71fK1utolDQf2IZ0W7KoNMzMroF+PhCTNBM4F3h0Rz1UWrQBOyyPbpgBTgTuAO4GpeSTcCNLAghU5ed0EnJLXnw1cXalrdp4+Bbgxl2/UhpmZFdJnR0KSrgSOAcZJagfOJ42G2wNYlcYKcHtEfDAi1kpaBjxAOk13VkS8lOs5G1gJDAMWRsTa3MS5wFJJnwPuAi7P8cuBKyS1kY6ATgPoqg0zMytDr5wRs3paWlqitbW1V+tOnndtw2WPXHhSb7tkZjbgSVoTES3dlfMVE8zMrBgnITMzK8ZJyMzMinESMjOzYpyEzMysGCchMzMrxknIzMyKcRIyM7NinITMzKyYfr+AqSXVqyn46glmNlT5SMjMzIpxEjIzs2KchMzMrBgnITMzK8ZJyMzMinESMjOzYpyEzMysmD5LQpIWStoo6f5KbKykVZLW5+cxOS5Jl0hqk3SvpCMq68zO5ddLml2JHynpvrzOJcr3C+9NG2ZmVkZfHgktAmZ2is0DboiIqcANeR7gBGBqfswFLoWUUIDzgaOBo4Dza0kll5lbWW9mb9owM7Ny+iwJRcSPgE2dwrOAxXl6MXByJb4kktuB0ZIOAI4HVkXEpojYDKwCZuZloyLitogIYEmnunrShpmZFdLf3wntHxGPA+Tn/XJ8ArChUq49x7qKt9eJ96aNHUiaK6lVUmtHR0ePNtDMzJo3UAYmqE4sehHvTRs7BiMWRERLRLSMHz++m2rNzKy3+jsJPVE7BZafN+Z4OzCpUm4i8Fg38Yl14r1pw8zMCunvJLQCqI1wmw1cXYmfkUewTQe25FNpK4HjJI3JAxKOA1bmZc9Kmp5HxZ3Rqa6etGFmZoX02a0cJF0JHAOMk9ROGuV2IbBM0hzgUeDUXPw64ESgDXgO+ABARGyS9FngzlzugoioDXY4kzQCbyRwfX7Q0zbMzKycPktCEXF6g0Uz6pQN4KwG9SwEFtaJtwKH1Yk/1dM2zMysjIEyMMHMzIYgJyEzMyvGScjMzIpxEjIzs2J6nITycOk39kVnzMxsaGkqCUm6WdKofEHRe4BvSvpS33bNzMx2d80eCe0TEc8A7wG+GRFHAu/ou26ZmdlQ0GwSGp4vgfNe4Pt92B8zMxtCmk1CnyFdQqctIu6UdBCwvu+6ZWZmQ0GzV0x4PCK2DUaIiIf8nZCZme2sZo+E/q3JmJmZWdO6PBKS9FbgD4Dxks6pLBoFDOvLjpmZ2e6vu9NxI4DX5nJ7V+LPAKf0VafMzGxo6DIJRcQtwC2SFkXEz/upT2ZmNkQ0OzBhD0kLgMnVdSLi2L7olJmZDQ3NJqH/BL4OXAa81HfdMTOzoaTZJLQ1Ii7t056YmdmQ0+wQ7Wsk/Z2kAySNrT36tGdmZrbbazYJzQY+DtwKrMmP1t42KumjktZKul/SlZL2lDRF0mpJ6yV9W9KIXHaPPN+Wl0+u1HNejj8o6fhKfGaOtUmaV4nXbcPMzMpoKglFxJQ6j4N606CkCcCHgZaIOIz0e6PTgIuA+RExFdgMzMmrzAE2R8TBwPxcDknT8nqHAjOBr0kaJmkY8FXgBGAacHouSxdtmJlZAU19JyTpjHrxiFiyE+2OlPQisBfwOHAs8Gd5+WLg08ClwKw8DbAc+Iok5fjSiHgeeFhSG3BULtcWEQ/lvi8FZkla10UbZmZWQLMDE95Smd4TmAH8BOhxEoqIX0j6IvAo8Bvgh6TTe09HxNZcrB2YkKcnABvyulslbQH2zfHbK1VX19nQKX50XqdRG9uRNBeYC/D617++p5toZmZNaioJRcSHqvOS9gGu6E2DksaQjmKmAE+Thn+fUK/Z2ioNljWK1zvF2FX5HYMRC4AFAC0tLXXLmJnZzuvx7b2z54CpvVz3HcDDEdERES8CV5GuTzdaUi0pTgQey9PtwCSAvHwfYFM13mmdRvEnu2jDzMwKaPb23tdIWpEf1wIPAlf3ss1HgemS9srf7cwAHgBu4pXr0c2u1L8iz5OX3xgRkeOn5dFzU0hJ8Q7gTmBqHgk3gjR4YUVep1EbZmZWQLPfCX2xMr0V+HlEtPemwYhYLWk56TulrcBdpFNf1wJLJX0uxy7Pq1wOXJEHHmwiJRUiYq2kZaQEthU4KyJeApB0NukmfMOAhRGxNtd1boM2zMysAKUDhCYKSvvzygCFOyJiY5/1agBpaWmJ1tbe/SRq8rxrmyr3yIUn9ap+M7OBStKaiGjprlyzp+PeSzrVdSrwXmC1JN/KwczMdkqzp+M+CbyldvQjaTzwX6Tf7ZiZmfVKs6PjXtXp9NtTPVjXzMysrmaPhH4gaSVwZZ5/H3Bd33TJzMyGii6TkKSDgf0j4uOS3gP8IelHn7cB3+qH/pmZ2W6su1NqFwPPAkTEVRFxTkR8lHQUdHFfd87MzHZv3SWhyRFxb+dgRLSSbvVtZmbWa90loT27WDZyV3bEzMyGnu6S0J2S/qZzUNIc0pWvzczMeq270XF/D3xX0vt5Jem0ACOAP+3LjpmZ2e6vyyQUEU8AfyDpj4HDcvjaiLixz3tmZma7vWbvJ3QT6QrUZmZmu4yvemBmZsU4CZmZWTHNXrbH+lD1lg++rYOZDSU+EjIzs2KchMzMrBgnITMzK6ZIEpI0WtJySf8jaZ2kt0oaK2mVpPX5eUwuK0mXSGqTdK+kIyr1zM7l10uaXYkfKem+vM4lkpTjddswM7MySh0JfRn4QUT8PvAmYB0wD7ghIqYCN+R5gBOAqfkxF7gUUkIBzgeOBo4Czq8klUtz2dp6M3O8URtmZlZAvychSaOAtwOXA0TECxHxNDALWJyLLQZOztOzgCWR3A6MlnQAcDywKiI2RcRmYBUwMy8bFRG3RUQASzrVVa8NMzMroMSR0EFAB/BNSXdJukzSa0g3z3scID/vl8tPADZU1m/Psa7i7XXidNHGdiTNldQqqbWjo6P3W2pmZl0qkYSGA0cAl0bE4cCv6fq0mOrEohfxpkXEgohoiYiW8ePH92RVMzPrgRJJqB1oj4jVeX45KSk9kU+lkZ83VspPqqw/EXism/jEOnG6aMPMzAro9yQUEb8ENkj6vRyaATwArABqI9xmA1fn6RXAGXmU3HRgSz6VthI4TtKYPCDhOGBlXvaspOl5VNwZneqq14aZmRVQ6rI9HwK+JWkE8BDwAVJCXJZvmPcocGouex1wItAGPJfLEhGbJH0WuDOXuyAiNuXpM4FFpLu/Xp8fABc2aMPMzAookoQi4m7SzfE6m1GnbABnNahnIbCwTryVV+5/VI0/Va8NMzMrw1dMMDOzYpyEzMysGCchMzMrxknIzMyKcRIyM7NinITMzKwYJyEzMyvGScjMzIpxEjIzs2KchMzMrBgnITMzK8ZJyMzMinESMjOzYpyEzMysGCchMzMrxknIzMyKcRIyM7NiiiUhScMk3SXp+3l+iqTVktZL+na+9TeS9sjzbXn55Eod5+X4g5KOr8Rn5libpHmVeN02zMysjJJHQh8B1lXmLwLmR8RUYDMwJ8fnAJsj4mBgfi6HpGnAacChwEzgazmxDQO+CpwATANOz2W7asPMzAookoQkTQROAi7L8wKOBZbnIouBk/P0rDxPXj4jl58FLI2I5yPiYaANOCo/2iLioYh4AVgKzOqmDTMzK6DUkdDFwCeAl/P8vsDTEbE1z7cDE/L0BGADQF6+JZffFu+0TqN4V21sR9JcSa2SWjs6Onq7jWZm1o1+T0KS3glsjIg11XCdotHNsl0V3zEYsSAiWiKiZfz48fWKmJnZLjC8QJtvA94t6URgT2AU6chotKTh+UhlIvBYLt8OTALaJQ0H9gE2VeI11XXqxZ/sog0zMyug34+EIuK8iJgYEZNJAwtujIj3AzcBp+Ris4Gr8/SKPE9efmNERI6flkfPTQGmAncAdwJT80i4EbmNFXmdRm2YmVkBA+l3QucC50hqI31/c3mOXw7sm+PnAPMAImItsAx4APgBcFZEvJSPcs4GVpJG3y3LZbtqw8zMCihxOm6biLgZuDlPP0Qa2da5zG+BUxus/3ng83Xi1wHX1YnXbcPMzMoYSEdCZmY2xDgJmZlZMU5CZmZWjJOQmZkV4yRkZmbFOAmZmVkxTkJmZlaMk5CZmRXjJGRmZsU4CZmZWTFOQmZmVoyTkJmZFeMkZGZmxTgJmZlZMU5CZmZWTNH7CdmOJs+7dtv0IxeeVLAnZmZ9z0dCZmZWjJOQmZkV0+9JSNIkSTdJWidpraSP5PhYSaskrc/PY3Jcki6R1CbpXklHVOqancuvlzS7Ej9S0n15nUskqas2zMysjBJHQluBj0XEIcB04CxJ04B5wA0RMRW4Ic8DnABMzY+5wKWQEgpwPnA0cBRwfiWpXJrL1tabmeON2jAzswL6PQlFxOMR8ZM8/SywDpgAzAIW52KLgZPz9CxgSSS3A6MlHQAcD6yKiE0RsRlYBczMy0ZFxG0REcCSTnXVa8PMzAoo+p2QpMnA4cBqYP+IeBxSogL2y8UmABsqq7XnWFfx9jpxumijc7/mSmqV1NrR0dHbzTMzs24US0KSXgt8B/j7iHimq6J1YtGLeNMiYkFEtEREy/jx43uyqpmZ9UCRJCTp1aQE9K2IuCqHn8in0sjPG3O8HZhUWX0i8Fg38Yl14l21YWZmBZQYHSfgcmBdRHypsmgFUBvhNhu4uhI/I4+Smw5syafSVgLHSRqTByQcB6zMy56VND23dUanuuq1YWZmBZS4YsLbgL8A7pN0d479I3AhsEzSHOBR4NS87DrgRKANeA74AEBEbJL0WeDOXO6CiNiUp88EFgEjgevzgy7aMDOzAvo9CUXEj6n/vQ3AjDrlAzirQV0LgYV14q3AYXXiT9Vrw8zMyvAVE8zMrBgnITMzK8ZJyMzMinESMjOzYpyEzMysGCchMzMrxndWHcB8l1Uz2935SMjMzIpxEjIzs2KchMzMrBgnITMzK8ZJyMzMinESMjOzYpyEzMysGP9OaJDwb4bMbHfkIyEzMyvGScjMzIpxEjIzs2KG5HdCkmYCXwaGAZdFxIWFu9Qj/n7IzHYXQ+5ISNIw4KvACcA04HRJ08r2ysxsaBqKR0JHAW0R8RCApKXALOCBor3qpepRUZWPkMxsMBiKSWgCsKEy3w4cXS0gaS4wN8/+StKDvWxrHPBkL9fdKbpol1RTrP+7iPtf3mDfBve/9w5sptBQTEKqE4vtZiIWAAt2uiGpNSJadraeUtz/sgZ7/2Hwb4P73/eG3HdCpCOfSZX5icBjhfpiZjakDcUkdCcwVdIUSSOA04AVhftkZjYkDbnTcRGxVdLZwErSEO2FEbG2j5rb6VN6hbn/ZQ32/sPg3wb3v48pIrovZWZm1geG4uk4MzMbIJyEzMysGCehPiBppqQHJbVJmle6P41IWihpo6T7K7GxklZJWp+fx+S4JF2St+leSUeU6/m2vk6SdJOkdZLWSvpIjg+KbZC0p6Q7JN36M1tSAAAEf0lEQVST+/+ZHJ8iaXXu/7fzABok7ZHn2/LyySX7XyNpmKS7JH0/zw+a/kt6RNJ9ku6W1Jpjg2L/yX0aLWm5pP/J/wdvHUz9ByehXW6QXRZoETCzU2wecENETAVuyPOQtmdqfswFLu2nPnZlK/CxiDgEmA6clV/rwbINzwPHRsSbgDcDMyVNBy4C5uf+bwbm5PJzgM0RcTAwP5cbCD4CrKvMD7b+/3FEvLnye5rBsv9AugbmDyLi94E3kf4Og6n/EBF+7MIH8FZgZWX+POC80v3qor+Tgfsr8w8CB+TpA4AH8/S/A6fXKzdQHsDVwJ8Mxm0A9gJ+Qrp6x5PA8M77E2lE51vz9PBcToX7PZH0Rncs8H3Sj8EHU/8fAcZ1ig2K/QcYBTzc+TUcLP2vPXwktOvVuyzQhEJ96Y39I+JxgPy8X44P6O3Kp3YOB1YziLYhn8q6G9gIrAJ+BjwdEVtzkWoft/U/L98C7Nu/Pd7BxcAngJfz/L4Mrv4H8ENJa/LlumDw7D8HAR3AN/Pp0MskvYbB03/Ap+P6QreXBRqkBux2SXot8B3g7yPima6K1okV3YaIeCki3kw6ojgKOKResfw8oPov6Z3AxohYUw3XKTog+5+9LSKOIJ2qOkvS27soO9D6Pxw4Arg0Ig4Hfs0rp97qGWj9B5yE+sJgvyzQE5IOAMjPG3N8QG6XpFeTEtC3IuKqHB5U2wAQEU8DN5O+2xotqfZD8moft/U/L98H2NS/Pd3O24B3S3oEWEo6JXcxg6f/RMRj+Xkj8F3SB4HBsv+0A+0RsTrPLyclpcHSf8BJqC8M9ssCrQBm5+nZpO9ZavEz8gib6cCW2iF/KZIEXA6si4gvVRYNim2QNF7S6Dw9EngH6Yvlm4BTcrHO/a9t1ynAjZFP7pcQEedFxMSImEzaz2+MiPczSPov6TWS9q5NA8cB9zNI9p+I+CWwQdLv5dAM0i1pBkX/tyn9pdTu+ABOBH5KOr//ydL96aKfVwKPAy+SPiXNIZ2jvwFYn5/H5rIijfr7GXAf0DIA+v+HpNMJ9wJ358eJg2UbgDcCd+X+3w/8c44fBNwBtAH/CeyR43vm+ba8/KDSf4PKthwDfH8w9T/38578WFv7Xx0s+0/u05uB1rwPfQ8YM5j6HxG+bI+ZmZXj03FmZlaMk5CZmRXjJGRmZsU4CZmZWTFOQmZmVoyTkJmZFeMkZGZmxTgJmQ0wkibne8N8I99n6IeSRkq6WVJLLjMuXy4HSX8p6XuSrpH0sKSzJZ2TL2p5u6SxRTfIrAtOQmYD01TgqxFxKPA08L+7KX8Y8Geka599Hngu0kUtbwPO6MuOmu0MJyGzgenhiLg7T68h3fepKzdFxLMR0UG6RcI1OX5fE+uaFeMkZDYwPV+Zfol02f6tvPI/u2cX5V+uzL+c1zUbkJyEzAaPR4Aj8/QpXZQzGzSchMwGjy8CZ0q6FRhXujNmu4Kvom1mZsX4SMjMzIpxEjIzs2KchMzMrBgnITMzK8ZJyMzMinESMjOzYpyEzMysmP8P8tKrjPdSPNkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "_ = plt.hist(num_words, bins = 100)\n",
    "plt.title('Distribution of num of words in one sentence')\n",
    "plt.xlabel('num')\n",
    "plt.ylabel('Counts')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:52.778Z"
    }
   },
   "outputs": [],
   "source": [
    "word_count = {}\n",
    "def count_words(x):\n",
    "    for sen in x:\n",
    "        for word in sen:\n",
    "            word_count[word] = word_count.get(word, 0) + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:52.944Z"
    }
   },
   "outputs": [],
   "source": [
    "_ = df_train.Processed_Reviews.apply(count_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:53.113Z"
    }
   },
   "outputs": [],
   "source": [
    "df_word_count = pd.DataFrame(list(word_count.items()), columns=['word', 'count'])\n",
    "df_word_count.sort_values(by='count', ascending=False, inplace=True)\n",
    "df_word_count.set_index('word', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:53.284Z"
    }
   },
   "outputs": [],
   "source": [
    "vocab_size = 40000\n",
    "df_word_count = df_word_count[:vocab_size-10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:53.447Z"
    }
   },
   "outputs": [],
   "source": [
    "word2num_series = df_word_count.reset_index().reset_index().set_index('word')['index'] + 10 # reverse 10 pos for padding, [UNK], [CLS]..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:53.611Z"
    }
   },
   "outputs": [],
   "source": [
    "def word_to_num(x):\n",
    "    res = []\n",
    "    for sen in x:\n",
    "        sen_words = []\n",
    "        for word in sen:\n",
    "            if word in word2num_series:\n",
    "                sen_words.append(word2num_series[word])\n",
    "        if len(sen_words) > 0:\n",
    "            res.append(np.array(sen_words, dtype=np.uint16))\n",
    "    return np.array(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:53.788Z"
    }
   },
   "outputs": [],
   "source": [
    "def create_data(df):\n",
    "    X = df.Processed_Reviews.apply(word_to_num)\n",
    "    y = df.label\n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:53.942Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train, y_train = create_data(df_train)\n",
    "X_val, y_val = create_data(df_validate)\n",
    "X_test, y_test = create_data(df_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:54.106Z"
    }
   },
   "outputs": [],
   "source": [
    "data_pth = 'dataset/yelp_review_polarity_csv/'\n",
    "X_train.to_hdf(data_pth + 'X_train.h5', key='s', model='w')\n",
    "y_train.to_hdf(data_pth + 'y_train.h5', key='s', model='w')\n",
    "\n",
    "X_val.to_hdf(data_pth + 'X_val.h5', key='s', model='w')\n",
    "y_val.to_hdf(data_pth + 'y_val.h5', key='s', model='w')\n",
    "\n",
    "X_test.to_hdf(data_pth + 'X_test.h5', key='s', model='w')\n",
    "y_test.to_hdf(data_pth + 'y_test.h5', key='s', model='w')\n",
    "word2num_series.to_hdf(data_pth + 'word2num_series.h5', key='s', model='w')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Yelp Full"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:54.446Z"
    }
   },
   "outputs": [],
   "source": [
    "data_pth = 'dataset/yelp_review_full_csv/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:54.612Z"
    }
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(data_pth + 'train.csv', header=None)\n",
    "df_test = pd.read_csv(data_pth + 'test.csv', header=None)\n",
    "df.columns = ['label', 'review']\n",
    "df_test.columns = ['label', 'review']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:54.783Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set size:  600000\n",
      "Validation set size:  50000\n",
      "Test set size:  50000\n"
     ]
    }
   ],
   "source": [
    "df['Processed_Reviews'] = df['review'].apply(process)\n",
    "df_test['Processed_Reviews'] = df_test['review'].apply(process)\n",
    "df_train, df_validate = train_test_split(df, test_size=50000, random_state=0, stratify=df.label)\n",
    "print(\"Train set size: \", len(df_train))\n",
    "print(\"Validation set size: \", len(df_validate))\n",
    "print(\"Test set size: \", len(df_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:54.952Z"
    }
   },
   "outputs": [],
   "source": [
    "num_words = []\n",
    "def count(x):\n",
    "    for sen in x:\n",
    "        num_words.append(len(sen))\n",
    "num_sens = df_train.Processed_Reviews.apply(len)\n",
    "_ = df_train.Processed_Reviews.apply(count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:55.115Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Counts')"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3X+8HGV59/HP10QQlJDEBJ5IEhIkWoFXGyBCWsWiKIT4I+gjFmpNRGoEoa1ifQjFForyNLQVWirCEyQlsUBAkBI0FFLKD60EOECAxAg5hGAOpElI+BEFwcD1/HHfRyab3XP2/Jid4+H7fr32dWavmbnnmtnZvXZm7rOjiMDMzKxMb6g6ATMzG/xcbMzMrHQuNmZmVjoXGzMzK52LjZmZlc7FxszMSudi81tC0iWS/rqf2hov6ReShuTnt0v60/5oO7d3k6RZ/dVeD5b7DUlPS/qfVi+7LyR9XNK6/JocWHU+VandL18P+vN9PdDJ/2dTPUlrgT2BbcArwE+BhcC8iHi1F239aUT8Zw/muR34t4j4Tk+Wlec9G9g3Iv6kp/P2J0njgEeBvSNiY5W59JSkx4DTIuKGFi7zcqAjIr7WqmXa65uPbAaOj0bEbsDewFzgdOCy/l6IpKH93eYAsTew+bet0GR7AyurTsJ6bhC/n/pfRPhR8QNYC3ywJnYI8CpwQH5+OfCNPDwK+AHwLLAF+BHpi8N38zwvAr8A/g8wAQjgRODnwJ2F2NDc3u3A3wH3AM8BNwAj87jDSd+Ad8gXmAa8DPw6L+/BQnt/moffAHwNeALYSDpi2z2P68xjVs7taeDMLrbT7nn+Tbm9r+X2P5jX+dWcx+V15j0c6AC+kvNYD5xQGP+bnPPzzwI/LjwP4IvAamAr8HXg7cBdwPPANcBODfKuuw2AnXO+AfwSeKzOvAIuyPM9BzxU2Cd2Bv4xb7sNwCXALt2tLzA7v2Yv5+XfmONvA67L2/dx4M8LeZyd13FhXv+VwJTC+HHA9/O8m4FvFcZ9DlgFPAPcTDr6rLedOveH4n75deC/8zJvAUZ1sX98HmgnvScWA2+ref1Oyq/fM8BF5DM7vczxN++nHJ8K/IT0nnwQODzHjwPaatr4MrC49n2dn38EWJ7b+Qnwuzl+QufrlJ+3A9cUnq8DJlf9Wdbl51zVCfhRv9jk+M+Bk/Pwb3ZKUmG4BHhjfhzGa6dEt2ur8OZYCLwZ2KXBm/pJ4IA8zXWk02rQRbHJw2d3TlsYfzuvFZvP5TfGPsBbSB9I363J7dKc1+8BLwHvarCdFpIK4W553keBExvlWTPv4aTTlOfkbTYdeAEYUZtzfv5Zdiw2i4FhwP45z1vzeu1OOvU5q8GyG26DQtv7Npj3KOA+YDip8LwLGJPH/VPOaWTeJjcCf9fk+l7O9h9yb8jL+Rtgp5zrGuCowuv8q9zOENI+uCyPG0L6gL2AtP+8CXhvHndMXvd3AUNJRfcnDda1c38o7pePAe/I+8ftwNwG836A9GXlIFIR/hdyIShs4x/k7TieVBSn9SHH4vtpL1KBnZ6344fy89HArqRCOanQxr3AcXXe1weRvhgcmrfpLNJ7bef8ejyb2x9D+uLyZJ5vH1KRfEPVn2VdPXwabWB7ivRBUuvXpB1u74j4dUT8KPJe14WzI+KXEfFig/HfjYgVEfFL4K+BT/XThdpPA+dHxJqI+AVwBnBczemHv42IFyPiQdKH1u/VNpJz+SPgjIjYGhFrgW8Cn+lBLr8GzsnbbAnpW/07ezD/eRHxfESsBFYAt+T1eg64CWh0cb+ZbdBVzrsBv0P6QrEqItZLEumb/JcjYktEbAX+L+mbdG/W993A6Ig4JyJejog1pC8BxfZ+HBFLIuIV0lF05+t0COmo6Kt5H/tVRPw4j/sCqQCuiohtOcfJkvZuYt0B/jUiHs377TXA5AbTfRqYHxH3R8RLpG38+5ImFKaZGxHPRsTPgdsKbfUmx+L76U+AJXnbvBoRS4E2YHpEvED6gnQ8gKRJpNdycZ02Pw/8v4i4OyJeiYgFpC81U/PrsTXn/Ieko68nJf1Ofv6j6OH13VZzsRnY9iKdEqj1D6RvYrdIWiNpThNtrevB+CdI34ZHNZVl196W2yu2PZTUIaJTsffYC6Rv/7VGkb5x17a1Vw9y2Zw/TLpbViMbCsMv1nneqK1mtkFdEfFfwLdIp302SJonaRivfWu+T9Kzkp4F/iPHO/VkffcG3tbZVm7vr+j6dXpTLpjjgCdqllVs958LbW4hHaE1+7o1s29AzTbORX1zzXIatdWbHIvvl72BY2u23XtJXwgBriQXG+CPgX/PRajW3sBXatoZl9cN4A7SEev78vDtpELzh/n5gOZiM0BJejdpZ/9x7bj8zf4rEbEP8FHgNElHdI5u0GR3Rz7jCsPjSd+KnyZdS9i1kNcQtv9A667dp0hvomLb29j+g7oZT+ecatt6softNLLdegL/q5/ahT5ug4i4MCIOJp2+ewfwVdL2eBHYPyKG58fuEdFs8ax93dYBjxfaGh4Ru0XE9CbaWgeMb3Cktg74Qk27u0TET5rMs1nbbWNJbwbeSnP7R29yLG6/daQzA8X53xwRc/P4W4BRkiaTis6VXeRxbk07u0bEVXl8Z7E5LA/fgYuN9ZakYZI+AiwiXQt5uM40H5G0bz6V8jypu/QrefQG0jncnvoTSftJ2pV0nv/afLrkUdI32A9LeiPpfPbOhfk2ABMkNdqXrgK+LGmipLeQTlFc3eBbcEM5l2uAcyXtlk9xnAb8W0/a6cJy4BOSdpW0L+kCcH/p9TaQ9G5Jh+Zt/0vSdZNX8imTS4ELJO2Rp91L0lFN5lS7n9wDPC/pdEm7SBoi6YD8pac795A6IMyV9GZJb5L0njzuEuAMSfvnHHeXdGyTOfbElcAJkiZL2pm0je/Op1u709cc/w34qKSj8nZ7k6TDJY0FyK/ztaQzEiOBpQ3auRQ4Kb/eytvyw5J2y+PvAN5P6gTSQeoYNI1UVB/oQb6VcLEZOG6UtJX07eZM4HxSD5R6JgH/SToHfxfw7Yi4PY/7O+Br+TD8L3uw/O+SLlb+D+kC758D5OsRXwS+Q/qW+EtSL6dO38t/N0u6v06783Pbd5J6OP0K+LMe5FX0Z3n5a0hHfFfm9vvDBaTeWRuABcAV/dQu9G0bDCN9CD1DOk20mdQDDVL3+HZgmaTnSftEs9egLgP2y/vJv+di/lHSNYHHSUdO3yF1fuhSYd59SZ1aOkjX14iI64HzgEU5xxXA0U3m2LSIuJV0rfE6UuF7O9tfb+pq3j7lGBHrgBmk046bSO/hr7L95+uVpF6T32v0JSMi2kjXbb5Fer3bSR1VOsc/SnrP/yg/f570Xvjv/BoMaP6nTjMzK52PbMzMrHQuNmZmVjoXGzMzK52LjZmZlc4/IpeNGjUqJkyYUHUaZma/Ve67776nI2J0d9O52GQTJkygra2t6jTMzH6rSHqi+6l8Gs3MzFrAxcbMzErnYmNmZqVzsTEzs9K52JiZWelKKzaSxkm6TdIqSSsl/UWOj5S0VNLq/HdEjkvShZLaJT0k6aBCW7Py9KslzSrED5b0cJ7nwvwryA2XYWZm1SjzyGYb8JWIeBfp/tynSNoPmAPcGhGTSLfV7bzx19GkXzOeRLpH+sWQCgdwFulWqYcAZxWKx8V52s75puV4o2WYmVkFSis2EbE+Iu7Pw1uBVaSbgc0g/YQ7+e8xeXgGsDCSZcBwSWNI92Bfmm99+wzpXhDT8rhhEXFXpJ+uXljTVr1lmJlZBVpyzUbpPuAHAncDe0bEekgFCdgjT7YX299qtSPHuop31InTxTJq85otqU1S26ZNm3q7emZm1o3Sf0Eg35nwOuBLEfF8vqxSd9I6sehFvGkRMQ+YBzBlypR+v7HPhDk/rBtfO/fD/b0oM7MBrdQjm3wr2+uAKyLi+zm8IZ8CI//dmOMdwLjC7GNJ9xXvKj62TryrZZiZWQXK7I0m0q1nV0XE+YVRi4HOHmWzgBsK8Zm5V9pU4Ll8Cuxm4EhJI3LHgCOBm/O4rZKm5mXNrGmr3jLMzKwCZZ5Gew/wGeBhSctz7K+AucA1kk4k3a/82DxuCTCddN/tF4ATACJii6SvA/fm6c6JiC15+GTgcmAX4Kb8oItlmJlZBUorNhHxY+pfVwE4os70AZzSoK35wPw68TbggDrxzfWWYWZm1fAvCJiZWelcbMzMrHQuNmZmVjoXGzMzK52LjZmZlc7FxszMSudiY2ZmpXOxMTOz0rnYmJlZ6VxszMysdC42ZmZWOhcbMzMrnYuNmZmVzsXGzMxK52JjZmalc7ExM7PSudiYmVnpSis2kuZL2ihpRSF2taTl+bG283bRkiZIerEw7pLCPAdLelhSu6QLJSnHR0paKml1/jsix5Wna5f0kKSDylpHMzNrTplHNpcD04qBiPijiJgcEZOB64DvF0Y/1jkuIk4qxC8GZgOT8qOzzTnArRExCbg1Pwc4ujDt7Dy/mZlVqLRiExF3AlvqjctHJ58CruqqDUljgGERcVdEBLAQOCaPngEsyMMLauILI1kGDM/tmJlZRaq6ZnMYsCEiVhdiEyU9IOkOSYfl2F5AR2GajhwD2DMi1gPkv3sU5lnXYJ7tSJotqU1S26ZNm/q2RmZm1lBVxeZ4tj+qWQ+Mj4gDgdOAKyUNA1Rn3uim7abniYh5ETElIqaMHj26ibTNzKw3hrZ6gZKGAp8ADu6MRcRLwEt5+D5JjwHvIB2VjC3MPhZ4Kg9vkDQmItbn02Qbc7wDGNdgHjMzq0AVRzYfBH4WEb85PSZptKQheXgf0sX9Nfn02FZJU/N1npnADXm2xcCsPDyrJj4z90qbCjzXebrNzMyqUWbX56uAu4B3SuqQdGIedRw7dgx4H/CQpAeBa4GTIqKzc8HJwHeAduAx4KYcnwt8SNJq4EP5OcASYE2e/lLgi/29bmZm1jOlnUaLiOMbxD9bJ3YdqSt0venbgAPqxDcDR9SJB3BKD9M1M7MS+RcEzMysdC42ZmZWOhcbMzMrnYuNmZmVzsXGzMxK52JjZmalc7ExM7PSudiYmVnpXGzMzKx0LjZmZlY6FxszMyudi42ZmZXOxcbMzErnYmNmZqVzsTEzs9K52JiZWelcbMzMrHRl3hZ6vqSNklYUYmdLelLS8vyYXhh3hqR2SY9IOqoQn5Zj7ZLmFOITJd0tabWkqyXtlOM75+ftefyEstbRzMyaU+aRzeXAtDrxCyJicn4sAZC0H3AcsH+e59uShkgaAlwEHA3sBxyfpwU4L7c1CXgGODHHTwSeiYh9gQvydGZmVqHSik1E3AlsaXLyGcCiiHgpIh4H2oFD8qM9ItZExMvAImCGJAEfAK7N8y8Ajim0tSAPXwsckac3M7OKVHHN5lRJD+XTbCNybC9gXWGajhxrFH8r8GxEbKuJb9dWHv9cnn4HkmZLapPUtmnTpr6vmZmZ1dXqYnMx8HZgMrAe+GaO1zvyiF7Eu2prx2DEvIiYEhFTRo8e3VXeZmbWBy0tNhGxISJeiYhXgUtJp8kgHZmMK0w6Fniqi/jTwHBJQ2vi27WVx+9O86fzzMysBC0tNpLGFJ5+HOjsqbYYOC73JJsITALuAe4FJuWeZzuROhEsjogAbgM+meefBdxQaGtWHv4k8F95ejMzq8jQ7ifpHUlXAYcDoyR1AGcBh0uaTDqttRb4AkBErJR0DfBTYBtwSkS8kts5FbgZGALMj4iVeRGnA4skfQN4ALgsxy8DviupnXREc1xZ62hmZs0prdhExPF1wpfViXVOfy5wbp34EmBJnfgaXjsNV4z/Cji2R8mamVmp/AsCZmZWOhcbMzMrnYuNmZmVzsXGzMxK52JjZmalc7ExM7PSudiYmVnpXGzMzKx0LjZmZlY6FxszMyudi42ZmZXOxcbMzErnYmNmZqVzsTEzs9K52JiZWelcbMzMrHQuNmZmVrrSio2k+ZI2SlpRiP2DpJ9JekjS9ZKG5/gESS9KWp4flxTmOVjSw5LaJV0oSTk+UtJSSavz3xE5rjxde17OQWWto5mZNae020IDlwPfAhYWYkuBMyJim6TzgDOA0/O4xyJicp12LgZmA8tIt4eeBtwEzAFujYi5kubk56cDRwOT8uPQPP+h/btqfTNhzg8bjls798MtzMTMrDVKO7KJiDuBLTWxWyJiW366DBjbVRuSxgDDIuKuiAhS4Tomj54BLMjDC2riCyNZBgzP7ZiZWUWqvGbzOdIRSqeJkh6QdIekw3JsL6CjME1HjgHsGRHrAfLfPQrzrGswz3YkzZbUJqlt06ZNfVsbMzNrqJJiI+lMYBtwRQ6tB8ZHxIHAacCVkoYBqjN7dNd8s/NExLyImBIRU0aPHt1c8mZm1mNlXrOpS9Is4CPAEfnUGBHxEvBSHr5P0mPAO0hHJcVTbWOBp/LwBkljImJ9Pk22Mcc7gHEN5jEzswq09MhG0jTSRfyPRcQLhfhoSUPy8D6ki/tr8umxrZKm5l5oM4Eb8myLgVl5eFZNfGbulTYVeK7zdJuZmVWjtCMbSVcBhwOjJHUAZ5F6n+0MLM09mJdFxEnA+4BzJG0DXgFOiojOzgUnk3q27UK6xtN5nWcucI2kE4GfA8fm+BJgOtAOvACcUNY6mplZc0orNhFxfJ3wZQ2mvQ64rsG4NuCAOvHNwBF14gGc0qNkzcysVD0+jSZphKTfLSMZMzMbnJoqNpJulzRM0kjgQeBfJZ1fbmpmZjZYNHtks3tEPA98AvjXiDgY+GB5aZmZ2WDSbLEZmrsXfwr4QYn5mJnZINRssflb4GagPSLuzd2TV5eXlpmZDSbN9kZbHxG/6RQQEWt8zcbMzJrV7JHNvzQZMzMz20GXRzaSfh/4A2C0pNMKo4YBQ8pMzMzMBo/uTqPtBLwlT7dbIf488MmykjIzs8Gly2ITEXcAd0i6PCKeaFFOZmY2yDTbQWBnSfOACcV5IuIDZSRlZmaDS7PF5nvAJcB3SD+UaWZm1rRmi822iLi41EzMzGzQarbr842SvihpjKSRnY9SMzMzs0Gj2SObzpuUfbUQC2Cf/k3HzMwGo6aKTURMLDsRMzMbvJoqNpJm1otHxML+TcfMzAajZq/ZvLvwOAw4G/hYdzNJmi9po6QVhdhISUslrc5/R+S4JF0oqV3SQ5IOKswzK0+/WtKsQvxgSQ/neS5Uvtd0o2WYmVk1mio2EfFnhcfngQNJvy7QncuBaTWxOcCtETEJuDU/BzgamJQfs4GLIRUO4CzgUOAQ4KxC8bg4T9s537RulmFmZhXo8W2hsxdIH+5diog7gS014RnAgjy8ADimEF8YyTJgeL6HzlHA0ojYEhHPAEuBaXncsIi4KyICWFjTVr1lmJlZBZq9ZnMjqfcZpB/gfBdwTS+XuWdErAeIiPWS9sjxvYB1hek6cqyreEedeFfLqF2v2aQjI8aPH9/L1TEzs+402/X5HwvD24AnIqKj0cS9pDqx6EW8aRExD5gHMGXKlB7Na2ZmzWv2ms0dwM9Iv/w8Ani5D8vckE+Bkf9uzPEOYFxhurHAU93Ex9aJd7UMMzOrQFPFRtKngHuAY4FPAXdL6u0tBhbz2j+JzgJuKMRn5l5pU4Hn8qmwm4EjJY3IHQOOBG7O47ZKmpp7oc2saaveMszMrALNnkY7E3h3RGwEkDQa+E/g2q5mknQVcDgwSlIHqVfZXOAaSScCPycVMIAlwHSgndQB4QSAiNgi6evAvXm6cyKis9PByaQeb7sAN+UHXSzDzMwq0GyxeUNnock208RRUUQc32DUEXWmDeCUBu3MB+bXibcBB9SJb663DDMzq0azxeY/JN0MXJWf/xHpSMTMzKxbXRYbSfuSuhF/VdIngPeSeoHdBVzRgvzMzGwQ6O5U2D8BWwEi4vsRcVpEfJl0VPNPZSdnZmaDQ3fFZkJEPFQbzNdKJpSSkZmZDTrdFZs3dTFul/5MxMzMBq/uis29kj5fG8xdiu8rJyUzMxtsuuuN9iXgekmf5rXiMoX0i88fLzOx3yYT5vyw6hTMzAa0LotNRGwA/kDS+3nt/1l+GBH/VXpmZmY2aDR7W+jbgNtKzsXMzAap3t7PxszMrGkuNmZmVjoXGzMzK52LjZmZlc7FxszMSudiY2ZmpXOxMTOz0rnYmJlZ6VpebCS9U9LywuN5SV+SdLakJwvx6YV5zpDULukRSUcV4tNyrF3SnEJ8oqS7Ja2WdLWknVq9nmZm9pqWF5uIeCQiJkfEZOBg4AXg+jz6gs5xEbEEQNJ+wHHA/sA04NuShkgaAlwEHA3sBxyfpwU4L7c1CXgGOLFV62dmZjuq+jTaEcBjEfFEF9PMABZFxEsR8TjQDhySH+0RsSYiXgYWATMkCfgAcG2efwFwTGlrYGZm3aq62BwHXFV4fqqkhyTNlzQix/YC1hWm6cixRvG3As9GxLaa+A4kzZbUJqlt06ZNfV8bMzOrq7Jik6+jfAz4Xg5dDLwdmAysB77ZOWmd2aMX8R2DEfMiYkpETBk9enQPsjczs55o6lefS3I0cH++jUHn7QwAkHQp8IP8tAMYV5hvLPBUHq4XfxoYLmloPropTm9mZhWostgcT+EUmqQxEbE+P/04sCIPLwaulHQ+8DZgEnAP6QhmkqSJwJOkU3J/HBEh6Tbgk6TrOLOAG1qwPv2i0Y3Y1s79cIszMTPrP5UUG0m7Ah8CvlAI/72kyaRTXms7x0XESknXAD8FtgGnRMQruZ1TgZuBIcD8iFiZ2zodWCTpG8ADwGWlr5SZmTVUSbGJiBdIF/KLsc90Mf25wLl14kuAJXXia0i91czMbACoujeamZm9DrjYmJlZ6VxszMysdC42ZmZWOhcbMzMrnYuNmZmVzsXGzMxK52JjZmalc7ExM7PSudiYmVnpXGzMzKx0LjZmZlY6FxszMyudi42ZmZXOxcbMzErnYmNmZqVzsTEzs9JVVmwkrZX0sKTlktpybKSkpZJW578jclySLpTULukhSQcV2pmVp18taVYhfnBuvz3Pq9avpZmZQfVHNu+PiMkRMSU/nwPcGhGTgFvzc4CjgUn5MRu4GFJxAs4CDiXdBvqszgKVp5ldmG9a+atjZmb1VF1sas0AFuThBcAxhfjCSJYBwyWNAY4ClkbEloh4BlgKTMvjhkXEXRERwMJCW2Zm1mJVFpsAbpF0n6TZObZnRKwHyH/3yPG9gHWFeTtyrKt4R534diTNltQmqW3Tpk39sEpmZlbP0AqX/Z6IeErSHsBSST/rYtp611uiF/HtAxHzgHkAU6ZM2WG8mZn1j8qObCLiqfx3I3A96ZrLhnwKjPx3Y568AxhXmH0s8FQ38bF14mZmVoFKio2kN0varXMYOBJYASwGOnuUzQJuyMOLgZm5V9pU4Ll8mu1m4EhJI3LHgCOBm/O4rZKm5l5oMwttmZlZi1V1Gm1P4PrcG3kocGVE/Ieke4FrJJ0I/Bw4Nk+/BJgOtAMvACcARMQWSV8H7s3TnRMRW/LwycDlwC7ATflhZmYVqKTYRMQa4PfqxDcDR9SJB3BKg7bmA/PrxNuAA/qcrJmZ9dlA6/psZmaDkIuNmZmVrsquz9YDE+b8sG587dwPtzgTM7Oe85GNmZmVzsXGzMxK52JjZmalc7ExM7PSudiYmVnpXGzMzKx0LjZmZlY6FxszMyudi42ZmZXOxcbMzErnYmNmZqVzsTEzs9K52JiZWelcbMzMrHQtLzaSxkm6TdIqSSsl/UWOny3pSUnL82N6YZ4zJLVLekTSUYX4tBxrlzSnEJ8o6W5JqyVdLWmn1q6lmZkVVXFksw34SkS8C5gKnCJpvzzugoiYnB9LAPK444D9gWnAtyUNkTQEuAg4GtgPOL7Qznm5rUnAM8CJrVo5MzPbUcuLTUSsj4j78/BWYBWwVxezzAAWRcRLEfE40A4ckh/tEbEmIl4GFgEzJAn4AHBtnn8BcEw5a2NmZs2o9E6dkiYABwJ3A+8BTpU0E2gjHf08QypEywqzdfBacVpXEz8UeCvwbERsqzN97fJnA7MBxo8f3/cVqoDv4Glmvw0q6yAg6S3AdcCXIuJ54GLg7cBkYD3wzc5J68wevYjvGIyYFxFTImLK6NGje7gGZmbWrEqObCS9kVRoroiI7wNExIbC+EuBH+SnHcC4wuxjgafycL3408BwSUPz0U1xejMzq0AVvdEEXAasiojzC/Exhck+DqzIw4uB4yTtLGkiMAm4B7gXmJR7nu1E6kSwOCICuA34ZJ5/FnBDmetkZmZdq+LI5j3AZ4CHJS3Psb8i9SabTDrltRb4AkBErJR0DfBTUk+2UyLiFQBJpwI3A0OA+RGxMrd3OrBI0jeAB0jFzczMKtLyYhMRP6b+dZUlXcxzLnBunfiSevNFxBpSbzUzMxsA/AsCZmZWOhcbMzMrnYuNmZmVzsXGzMxKV+kvCFh5/MsCZjaQ+MjGzMxK52JjZmalc7ExM7PSudiYmVnpXGzMzKx07o32OuNeamZWBR/ZmJlZ6VxszMysdD6NZoBPr5lZuXxkY2ZmpfORjXXJRzxm1h98ZGNmZqUbtEc2kqYB/0y6ZfR3ImJuxSkNKj7iMbOeGJTFRtIQ4CLgQ0AHcK+kxRHx02ozG/waFaFGXJzMXh8GZbEBDgHaI2INgKRFwAzAxWaA6Wlx6g0XNLPqDdZisxewrvC8Azi0diJJs4HZ+ekvJD3Sy+WNAp7u5bxlG6i5tSwvndfjWV7326yHBmpeMHBzG6h5Qc9z27uZiQZrsVGdWOwQiJgHzOvzwqS2iJjS13bKMFBzG6h5wcDNzXn13EDNbaDmBeXlNlh7o3UA4wrPxwJPVZSLmdnr3mAtNvcCkyRNlLQTcBywuOKczMxetwblabSI2CbpVOBmUtfn+RGxssRF9vlUXIkGam4DNS8YuLk5r54bqLkN1LygpNwUscOlDDMzs341WE+jmZnZAOJiY2ZmpXOx6SNJ0yQ9Iqld0pwK8xgn6TZJqyStlPQXOX62pCclLc+P6RXlt1bSwzmHthwbKWmppNX574gW5/TOwnZZLul5SV+qaptJmi9po6QVhVjdbaTkwrzfPSTpoBbn9Q+SfpYgKJ9KAAAEsklEQVSXfb2k4Tk+QdKLhW13SYvzavjaSTojb69HJB1VVl5d5HZ1Ia+1kpbneCu3WaPPifL3s4jwo5cPUueDx4B9gJ2AB4H9KsplDHBQHt4NeBTYDzgb+MsBsK3WAqNqYn8PzMnDc4DzKn4t/4f0D2qVbDPgfcBBwIruthEwHbiJ9D9lU4G7W5zXkcDQPHxeIa8Jxekq2F51X7v8XngQ2BmYmN+3Q1qZW834bwJ/U8E2a/Q5Ufp+5iObvvnNz+JExMtA58/itFxErI+I+/PwVmAV6ZcUBrIZwII8vAA4psJcjgAei4gnqkogIu4EttSEG22jGcDCSJYBwyWNaVVeEXFLRGzLT5eR/petpRpsr0ZmAIsi4qWIeBxoJ71/W56bJAGfAq4qa/mNdPE5Ufp+5mLTN/V+FqfyD3hJE4ADgbtz6NR8CDy/1aeqCgK4RdJ9Sj8TBLBnRKyH9CYA9qgoN0j/i1V88w+EbQaNt9FA2vc+R/r222mipAck3SHpsAryqffaDaTtdRiwISJWF2It32Y1nxOl72cuNn3T1M/itJKktwDXAV+KiOeBi4G3A5OB9aTD9yq8JyIOAo4GTpH0vory2IHSP/5+DPheDg2UbdaVAbHvSToT2AZckUPrgfERcSBwGnClpGEtTKnRazcgtld2PNt/sWn5NqvzOdFw0jqxXm03F5u+GVA/iyPpjaQd6IqI+D5ARGyIiFci4lXgUko8ddCViHgq/90IXJ/z2NB5SJ7/bqwiN1IBvD8iNuQcB8Q2yxpto8r3PUmzgI8An458gj+fptqch+8jXRt5R6ty6uK1q3x7AUgaCnwCuLoz1uptVu9zghbsZy42fTNgfhYnnwe+DFgVEecX4sXzqx8HVtTO24Lc3ixpt85h0sXlFaRtNStPNgu4odW5Zdt90xwI26yg0TZaDMzMvYWmAs91ngZpBaWbE54OfCwiXijERyvdTwpJ+wCTgDUtzKvRa7cYOE7SzpIm5rzuaVVeBR8EfhYRHZ2BVm6zRp8TtGI/a0UPiMH8IPXWeJT0beTMCvN4L+nw9iFgeX5MB74LPJzji4ExFeS2D6kn0IPAys7tBLwVuBVYnf+OrCC3XYHNwO6FWCXbjFTw1gO/Jn2jPLHRNiKd3rgo73cPA1NanFc76Vx+5752SZ72f+fX+EHgfuCjLc6r4WsHnJm31yPA0a1+LXP8cuCkmmlbuc0afU6Uvp/552rMzKx0Po1mZmalc7ExM7PSudiYmVnpXGzMzKx0LjZmZlY6FxszMyudi42ZmZXOxcasIvk+JqskXZrvLXKLpF0k3S5pSp5mlKS1efizkv5d0o2SHpd0qqTT8g84LpM0stIVMuuCi41ZtSYBF0XE/sCzpP8m78oBwB+TfvPrXOCFSD/geBcws8xEzfrCxcasWo9HxPI8fB/pRlpduS0itkbEJuA54MYcf7iJec0q42JjVq2XCsOvAENJP9nf+d58UxfTv1p4/mqe12xAcrExG3jWAgfn4U9WmIdZv3GxMRt4/hE4WdJPgFFVJ2PWH/yrz2ZmVjof2ZiZWelcbMzMrHQuNmZmVjoXGzMzK52LjZmZlc7FxszMSudiY2Zmpfv/6Zu8XZsXJpIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "_ = plt.hist(num_sens, bins = 50)\n",
    "plt.title('Distribution of num of sentence in one review')\n",
    "plt.xlabel('num')\n",
    "plt.ylabel('Counts')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:55.279Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Counts')"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAAEWCAYAAADPZygPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3X2YVWW9//H3J0jFUoFAfwboaFLHh6tQJ6XTqZ9pKdqD1tHCXyUVHbK0Jz0lnjpHsyw9p7JjD3YwCbASzYfE1Ih86nTl05CkoBkjokwQoCBaPoLf3x/r3rDY7j2zZ2Dmns18Xte1r1n7u+51r+/as2Z/97r3PXsrIjAzM8vhZbkTMDOzgctFyMzMsnERMjOzbFyEzMwsGxchMzPLxkXIzMyycRHaRkj6oaR/30p97SHpb5IGpfu3Svr41ug79XejpElbq79u7Pdrkh6T9Ne+3veWkPReScvS7+TAjHnMkPS1Hmz3N0l790ZO1vwG507AuiZpKbAbsB7YANwPzAKmRcSLABFxcjf6+nhE/KZem4h4FHjllmW9cX9nA/tExIdK/R+9NfruZh5jgNOBPSNiVV/vfwt9Ezg1Iq7NnUhPRMRWOZeajaQZQEdEfDl3Lv2Zr4Sax7sjYidgT+A84Azgkq29E0nb6guTPYHHm7AAQZH7or7a2TZ8Dlh/FBG+9fMbsBR4e1XsEOBF4IB0fwbwtbQ8Avgl8ASwBvhfihccl6ZtngH+BnwRaAECmAw8Cvy2FBuc+rsV+AZwF7AOuBYYntYdRvFq7yX5AhOA54EX0v7+WOrv42n5ZcCXgUeAVRRXeLukdZU8JqXcHgO+1MnjtEvafnXq78up/7enY34x5TGjxraHAR0UV0urgBXAR0vrN+ac7n8E+F3pfgCfAhYDTwFfBV4D3A48CVwBbFcn75qPAbB9yjeAvwMP1dj2K8B30/LLU7v/TPeHAM8Cw9L991AUsyfS8exb9Ts7A7gXeI5ilORA4A/peC4HZtPFOVbn+ILiahiK8/T7wPWp3zuB13TyO+0q539NOa9LOe5QWv8uYEHa9vfA6+vsQ8AF6bFfl/qr/F1tT3El+iiwEvghMKSrcwaYQnHeP59+h9el+KuBqyjO0YeBz5TyODudJ7PSY7MIaC2tHwNcnbZ9HPhead3HgAeAtcBciiv+7M9dDT2/5U7AtwZ+STWKUIo/CnwyLc8oPUF8I/2xvDzd3gKoVl9seqKfBbyC4omrEisXob8AB6Q2VwE/SesOo04RSstnV9qW1t/KpiL0MaAd2JtiCPBq4NKq3C5Oeb2B4gly3zqP0yyKArlT2vbPwOR6eVZtexjFcOc56TE7BniaTU/gG3NO9z/CS4vQHGBnYP+U503puHahGEKdVGffdR+DUt/71Nn2cOC+tPyPwEPAnaV1lcL/WooC9Y50fF9M+9yu9DtbQPFENwTYjqIofj61P57iSbXLc6xGjtVFaA3Fi6jBwE+B2XW2ayTnuyie2IdTPAmfnNYdRFEYDgUGUbyQWQpsX2M/RwHzgaEUBWlfYPe07jvp9zqc4ry6DvhGg+fMjMrjle6/LO3nP9LjuzewBDiq9LfybOpnUHqM70jrBgF/pCiWrwB2AP4prTsuPS77psf0y8Dvcz9vNXrzcFxzW07xx1HtBWB3ildDL0TE/0Y6WztxdkT8PSKeqbP+0ohYGBF/B/4deH9l4sIW+iDw7YhYEhF/A84EJlYNCX0lIp6JiD9S/CG+obqTlMsHgDMj4qmIWAp8C/hwN3J5ATgnPWY3ULyCfV03tj8/Ip6MiEXAQuDX6bjWATdSXFnU0shjUM/twFhJrwLeSjFEO0rSK4H/C9yW2n0AuD4i5kXECxSv7odQFK6KCyNiWToHxlM8sX4nPR5XAneX2vbkHKu4OiLuioj1FEVoXJ12jea8PCLWUBSISl//AvxPRNwZERsiYibFC4PxNfbzAkWB+QeKQvpARKyQpNTP5yNiTUQ8BXwdmFi1baPnzBuBkRFxTkQ8HxFLKF5glfv7XUTcEBEbKEYuKuf6IRTF9gvp7/TZiPhdWvcJisL4QHpMvw6Mk7RnnTz6FReh5jaK4lVltf+ieGX0a0lLJE1toK9l3Vj/CMUT1IiGsuzcq1N/5b4HU0zEqCjPZnua2pMmRrDp1Xu5r1HdyOXx9Efc1b7qWVlafqbG/Xp9NfIY1JQKRhtFwXkrRdH5PfBmNi9Cm+0jigkty9j88Sn/jl8N/KWqsJRz7Mk5VtHI77PRnOv1tSdwuqQnKjeKq7xXV+8kIm4GvkcxTLhS0jRJOwMjgR2B+aU+fpXiFd05Z/YEXl2V07/R+bm+Q3oxMgZ4pGpf5X7/u9TnGooruu6c+9m4CDUpSW+kOMl+V70uXQmcHhF7A+8GTpN0RGV1nS67ehU7prS8B8UrwMcohkt2LOU1iM3/SLvqdznFH1G57/Vs/gTeiMdSTtV9/aWb/dSz2XEC/2cr9Qtb/hjcRjH0diDF1cptFENMh1C8x/eSfaRX+WPY/PEp/65WUFxRqSqvomHn59jW0kjO9SwDzo2IoaXbjhFxWa3GEXFhRBxMMZT6WuALFOfUM8D+pT52icZn+1Wf+8uAh6ty2ikijmnwePaoc3W8DPhEVb9DIuL3DeaZlYtQk5G0s6R3UbxJ/JOIuK9Gm3dJ2if90T5JMa17Q1q9kmIsurs+JGk/STtSjIFfmYYM/kzxau2dkl5OMR69fWm7lUCLpHrn2mXA5yXtlYaQvg5cXucVX10plyuAcyXtlIYiTgN+0p1+OrEAeJ+kHSXtQzGRY2vZ0sfgNuAk4P6IeJ70/hXFE97q1OYK4J2Sjki/p9MphqfqPVHdTlEIPyNpsKT3URQ1oMtzbGvpbs5lFwMnSzpUhVekc3Sn6oaS3pjaVSZ2PAtsSFdeFwMXSNo1tR0l6agG86/+W7sLeFLSGZKGSBok6YD0grIrd1G8MDgvHcsOkt6c1v0QOFPS/inHXSSd0GCO2bkINY/rJD1F8arnS8C3gY/WaTsW+A3F+PTtwA8i4ta07hvAl9Ol+792Y/+XUrzR+leKN0U/A5De7/gU8COKV6h/p5gxVPHz9PNxSX+o0e/01PdvKWYLPQt8uht5lX067X8JxRXiz1L/W8MFFDOdVgIzKd7L2Fq29DH4PcV7JZWrnvtTH5X7RMSDwIeA71K8wn83xbT/52t1mOLvo5iAsZbi/ZmrS006O8e2iu7mXLVtG8X7Od9L+bdTHEstO1MUm7UUw3+PU7z/BMWMwXbgDklPUhxzo+8TXgLsl/7WfpFeKL2b4n2rh9Mx/Yhi4kpXx1PZdh+KCUkdFL8TIuIa4HxgdspxIdDn/4vXU5UZU2ZmZn3OV0JmZpaNi5CZmWXjImRmZtm4CJmZWTb+oMIujBgxIlpaWnKnYWbWVObPn/9YRIzsqp2LUBdaWlpoa2vLnYaZWVOR9EjXrTwcZ2ZmGbkImZlZNi5CZmaWjYuQmZll4yJkZmbZuAiZmVk2LkJmZpaNi5CZmWXjImRmZtn4ExN6WcvU6zcuLz3vnRkzMTPrf3wlZGZm2bgImZlZNh6O60MemjMz25yvhMzMLJteK0KSpktaJWlhKXa5pAXptlTSghRvkfRMad0PS9scLOk+Se2SLpSkFB8uaZ6kxennsBRXatcu6V5JB5X6mpTaL5Y0qbeO3czMGtObV0IzgAnlQER8ICLGRcQ44Crg6tLqhyrrIuLkUvwiYAowNt0qfU4FboqIscBN6T7A0aW2U9L2SBoOnAUcChwCnFUpXGZmlkevFaGI+C2wpta6dDXzfuCyzvqQtDuwc0TcHhEBzAKOS6uPBWam5ZlV8VlRuAMYmvo5CpgXEWsiYi0wj6oiaWZmfSvXe0JvAVZGxOJSbC9J90i6TdJbUmwU0FFq05FiALtFxAqA9HPX0jbLamxTL/4SkqZIapPUtnr16u4fnZmZNSRXETqRza+CVgB7RMSBwGnAzyTtDKjGttFF3/W2abiviJgWEa0R0TpyZJdfkW5mZj3U50VI0mDgfcDllVhEPBcRj6fl+cBDwGsprlZGlzYfDSxPyyvTMFtl2G5VincAY2psUy9uZmaZ5LgSejvwp4jYOMwmaaSkQWl5b4pJBUvSMNtTksan95FOAq5Nm80BKjPcJlXFT0qz5MYD61I/c4EjJQ1LExKOTDEzM8uk1/5ZVdJlwGHACEkdwFkRcQkwkZdOSHgrcI6k9cAG4OSIqExq+CTFTLshwI3pBnAecIWkycCjwAkpfgNwDNAOPA18FCAi1kj6KnB3andOaR9mZpZBrxWhiDixTvwjNWJXUUzZrtW+DTigRvxx4Iga8QBOqdPXdGB6Z3mbmVnf8ScmmJlZNi5CZmaWjYuQmZll4yJkZmbZuAiZmVk2LkJmZpaNi5CZmWXjImRmZtm4CJmZWTYuQmZmlo2LkJmZZeMiZGZm2bgImZlZNi5CZmaWjYuQmZll4yJkZmbZuAiZmVk2LkJmZpaNi5CZmWXTa0VI0nRJqyQtLMXOlvQXSQvS7ZjSujMltUt6UNJRpfiEFGuXNLUU30vSnZIWS7pc0nYpvn26357Wt3S1DzMzy6M3r4RmABNqxC+IiHHpdgOApP2AicD+aZsfSBokaRDwfeBoYD/gxNQW4PzU11hgLTA5xScDayNiH+CC1K7uPrbyMZuZWTf0WhGKiN8CaxpsfiwwOyKei4iHgXbgkHRrj4glEfE8MBs4VpKAw4Er0/YzgeNKfc1My1cCR6T29fZhZmaZ5HhP6FRJ96bhumEpNgpYVmrTkWL14q8CnoiI9VXxzfpK69el9vX6eglJUyS1SWpbvXp1z47SzMy61NdF6CLgNcA4YAXwrRRXjbbRg3hP+nppMGJaRLRGROvIkSNrNTEzs62gT4tQRKyMiA0R8SJwMZuGwzqAMaWmo4HlncQfA4ZKGlwV36yvtH4XimHBen2ZmVkmfVqEJO1euvteoDJzbg4wMc1s2wsYC9wF3A2MTTPhtqOYWDAnIgK4BTg+bT8JuLbU16S0fDxwc2pfbx9mZpbJ4K6b9Iyky4DDgBGSOoCzgMMkjaMYBlsKfAIgIhZJugK4H1gPnBIRG1I/pwJzgUHA9IhYlHZxBjBb0teAe4BLUvwS4FJJ7RRXQBO72oeZmeWh4iLB6mltbY22trYeb98y9fqa8aXnvbPHfZqZ9XeS5kdEa1ft/IkJZmaWjYuQmZll4yJkZmbZuAiZmVk2LkJmZpaNi5CZmWXjImRmZtm4CJmZWTYuQmZmlo2LkJmZZeMiZGZm2bgImZlZNi5CZmaWjYuQmZll4yJkZmbZuAiZmVk2LkJmZpaNi5CZmWUzOHcCA1X5a7/9Vd9mNlD5SsjMzLLptSIkabqkVZIWlmL/JelPku6VdI2koSneIukZSQvS7YelbQ6WdJ+kdkkXSlKKD5c0T9Li9HNYiiu1a0/7OajU16TUfrGkSb117GZm1pjevBKaAUyois0DDoiI1wN/Bs4srXsoIsal28ml+EXAFGBsulX6nArcFBFjgZvSfYCjS22npO2RNBw4CzgUOAQ4q1K4zMwsj14rQhHxW2BNVezXEbE+3b0DGN1ZH5J2B3aOiNsjIoBZwHFp9bHAzLQ8syo+Kwp3AENTP0cB8yJiTUSspSiI1UXSzMz6UM73hD4G3Fi6v5ekeyTdJuktKTYK6Ci16UgxgN0iYgVA+rlraZtlNbapF38JSVMktUlqW716dfePzMzMGpKlCEn6ErAe+GkKrQD2iIgDgdOAn0naGVCNzaOr7uts03BfETEtIlojonXkyJFd7M7MzHqqz4tQmhDwLuCDaYiNiHguIh5Py/OBh4DXUlytlIfsRgPL0/LKNMxWGbZbleIdwJga29SLm5lZJn1ahCRNAM4A3hMRT5fiIyUNSst7U0wqWJKG2Z6SND7NijsJuDZtNgeozHCbVBU/Kc2SGw+sS/3MBY6UNCxNSDgyxczMLJNe+2dVSZcBhwEjJHVQzEw7E9gemJdmWt+RZsK9FThH0npgA3ByRFQmNXySYqbdEIr3kCrvI50HXCFpMvAocEKK3wAcA7QDTwMfBYiINZK+Ctyd2p1T2oeZmWXQa0UoIk6sEb6kTturgKvqrGsDDqgRfxw4okY8gFPq9DUdmF4/azMz60v+xAQzM8vGRcjMzLJxETIzs2xchMzMLBsXITMzy8ZFyMzMsnERMjOzbFyEzMwsGxchMzPLxkXIzMyycREyM7NsXITMzCybbheh9FUIr++NZMzMbGBpqAhJulXSzpKGA38Efizp272bmpmZbesavRLaJSKeBN4H/DgiDgbe3ntpmZnZQNBoERqcvkL7/cAvezEfMzMbQBotQl+h+Crs9oi4O30F9+LeS8vMzAaCRr9ZdUVEbJyMEBFL/J6QmZltqUavhL7bYMzMzKxhnRYhSW+SdDowUtJppdvZwKCuOpc0XdIqSQtLseGS5klanH4OS3FJulBSu6R7JR1U2mZSar9Y0qRS/GBJ96VtLpSknu7DzMz6XldXQtsBr6QYttupdHsSOL6B/mcAE6piU4GbImIscFO6D3A0MDbdpgAXQVFQgLOAQ4FDgLMqRSW1mVLabkJP9mFmZnl0+p5QRNwG3CZpRkQ80t3OI+K3klqqwscCh6XlmcCtwBkpPisiArhD0tA0I+8wYF5ErAGQNA+YIOlWYOeIuD3FZwHHATd2dx8RsaK7x2ZmZluu0YkJ20uaBrSUt4mIw3uwz90qT/oRsULSrik+ClhWateRYp3FO2rEe7KPzYqQpCkUV0rssccePThEMzNrRKNF6OfAD4EfARt6KRfViEUP4j3Zx+aBiGnANIDW1tau+jQzsx5qtAitj4it9f7JysoQWBpuW5XiHcCYUrvRwPIUP6wqfmuKj67Rvif7MDOzDBqdon2dpE9J2j3NPBueJgz0xBygMsNtEnBtKX5SmsE2HliXhtTmAkemD04dBhwJzE3rnpI0Ps2KO6mqr+7sw8zMMmj0SqjyhP6FUiyAvTvbSNJlFFcxIyR1UMxyOw+4QtJk4FHghNT8BuAYoB14GvgoQESskfRV4O7U7pzKJAXgkxQz8IZQTEi4McW7tQ8zM8ujoSIUEXv1pPOIOLHOqiNqtA3glDr9TAem14i3AQfUiD/e3X2YmVnfa6gISTqpVjwiZm3ddMzMbCBpdDjujaXlHSiuMv4AuAiZmVmPNToc9+nyfUm7AJf2SkZmZjZgdPvrvZOnKT76xszMrMcafU/oOjb9U+cgYF/git5KyszMBoZG3xP6Zml5PfBIRHTUa2xmZtaIhobj0geZ/oniE7SHAc/3ZlJmZjYwNFSEJL0fuIvinz7fD9wpqZGvcjAzM6ur0eG4LwFvjIhVAJJGAr8BruytxMzMbNvX6Oy4l1UKUPJ4N7Y1MzOrqdEroV9Jmgtclu5/gOJz2MzMzHqs0yIkaR+KL4j7gqT3Af9E8Z08twM/7YP8zMxsG9bVkNp3gKcAIuLqiDgtIj5PcRX0nd5OzszMtm1dFaGWiLi3Opg+vbqlVzIyM7MBo6sitEMn64ZszUTMzGzg6aoI3S3pX6qD6cvi5vdOSmZmNlB0NTvuc8A1kj7IpqLTCmwHvLc3EzMzs21fp0UoIlYC/yjpbWz6BtPrI+LmXs/MzMy2eY1+n9AtwC29nIuZmQ0wff6pB5JeJ2lB6fakpM9JOlvSX0rxY0rbnCmpXdKDko4qxSekWLukqaX4XpLulLRY0uWStkvx7dP99rS+pS+PvZ6WqddvvJmZDSR9XoQi4sGIGBcR44CDKb4g75q0+oLKuoi4AUDSfsBEYH9gAvADSYMkDQK+DxwN7AecmNoCnJ/6GgusBSan+GRgbUTsA1yQ2pmZWSa5P//tCOChiHikkzbHArMj4rmIeBhoBw5Jt/aIWBIRzwOzgWMlCTicTR+uOhM4rtTXzLR8JXBEam9mZhnkLkIT2fR5dACnSrpX0nRJw1JsFLCs1KYjxerFXwU8ERHrq+Kb9ZXWr0vtzcwsg2xFKL1P8x7g5yl0EfAaYBywAvhWpWmNzaMH8c76qs5tiqQ2SW2rV6+uewxmZrZlcl4JHQ38IU0DJyJWRsSGiHgRuJhiuA2KK5kxpe1GA8s7iT8GDJU0uCq+WV9p/S7AmurEImJaRLRGROvIkSO3+EDNzKy2nEXoREpDcZJ2L617L7AwLc8BJqaZbXsBYym+5fVuYGyaCbcdxdDenIgIiunklW9+nQRcW+prUlo+Hrg5tTczswwa/T6hrUrSjsA7gE+Uwv8paRzF8NjSyrqIWCTpCuB+YD1wSkRsSP2cCswFBgHTI2JR6usMYLakrwH3AJek+CXApZLaKa6AJvbaQZqZWZeyFKGIeJqqCQER8eFO2p8LnFsjfgM1vlwvIpawaTivHH8WOKEHKZuZWS/IPTvOzMwGMBchMzPLxkXIzMyycREyM7NsXITMzCwbFyEzM8vGRcjMzLJxETIzs2xchMzMLBsXITMzy8ZFyMzMsnERMjOzbFyEzMwsGxchMzPLxkXIzMyycREyM7NsXITMzCwbFyEzM8vGRcjMzLJxETIzs2yyFSFJSyXdJ2mBpLYUGy5pnqTF6eewFJekCyW1S7pX0kGlfial9oslTSrFD079t6dt1dk+zMys7+W+EnpbRIyLiNZ0fypwU0SMBW5K9wGOBsam2xTgIigKCnAWcChwCHBWqahclNpWtpvQxT7MzKyP5S5C1Y4FZqblmcBxpfisKNwBDJW0O3AUMC8i1kTEWmAeMCGt2zkibo+IAGZV9VVrH2Zm1sdyFqEAfi1pvqQpKbZbRKwASD93TfFRwLLSth0p1lm8o0a8s31sJGmKpDZJbatXr96CQzQzs84MzrjvN0fEckm7AvMk/amTtqoRix7EGxIR04BpAK2trQ1vZ2Zm3ZPtSigilqefq4BrKN7TWZmG0kg/V6XmHcCY0uajgeVdxEfXiNPJPszMrI9lKUKSXiFpp8oycCSwEJgDVGa4TQKuTctzgJPSLLnxwLo0lDYXOFLSsDQh4Uhgblr3lKTxaVbcSVV91dqHmZn1sVzDcbsB16RZ04OBn0XEryTdDVwhaTLwKHBCan8DcAzQDjwNfBQgItZI+ipwd2p3TkSsScufBGYAQ4Ab0w3gvDr7MDOzPpalCEXEEuANNeKPA0fUiAdwSp2+pgPTa8TbgAMa3YeZmfW9/jZF28zMBhAXITMzy8ZFyMzMsnERMjOzbFyEzMwsGxchMzPLxkXIzMyycREyM7NsXITMzCwbFyEzM8vGRcjMzLJxETIzs2xchMzMLBsXITMzy8ZFyMzMsnERMjOzbFyEzMwsGxchMzPLJsvXe1t9LVOv37i89Lx3ZszEzKz39fmVkKQxkm6R9ICkRZI+m+JnS/qLpAXpdkxpmzMltUt6UNJRpfiEFGuXNLUU30vSnZIWS7pc0nYpvn26357Wt/TdkZuZWbUcw3HrgdMjYl9gPHCKpP3SugsiYly63QCQ1k0E9gcmAD+QNEjSIOD7wNHAfsCJpX7OT32NBdYCk1N8MrA2IvYBLkjtzMwskz4vQhGxIiL+kJafAh4ARnWyybHA7Ih4LiIeBtqBQ9KtPSKWRMTzwGzgWEkCDgeuTNvPBI4r9TUzLV8JHJHam5lZBlknJqThsAOBO1PoVEn3SpouaViKjQKWlTbrSLF68VcBT0TE+qr4Zn2l9etS++q8pkhqk9S2evXqLTpGMzOrL1sRkvRK4CrgcxHxJHAR8BpgHLAC+FalaY3NowfxzvraPBAxLSJaI6J15MiRnR6HmZn1XJYiJOnlFAXopxFxNUBErIyIDRHxInAxxXAbFFcyY0qbjwaWdxJ/DBgqaXBVfLO+0vpdgDVb9+jMzKxROWbHCbgEeCAivl2K715q9l5gYVqeA0xMM9v2AsYCdwF3A2PTTLjtKCYvzImIAG4Bjk/bTwKuLfU1KS0fD9yc2puZWQY5/k/ozcCHgfskLUixf6OY3TaOYnhsKfAJgIhYJOkK4H6KmXWnRMQGAEmnAnOBQcD0iFiU+jsDmC3pa8A9FEWP9PNSSe0UV0ATe/NAzcysc31ehCLid9R+b+aGTrY5Fzi3RvyGWttFxBI2DeeV488CJ3QnXzMz6z3+2B4zM8vGRcjMzLJxETIzs2xchMzMLBsXITMzy8ZFyMzMsnERMjOzbFyEzMwsG3+zaj/mb1k1s22dr4TMzCwbFyEzM8vGRcjMzLJxETIzs2xchMzMLBsXITMzy8ZFyMzMsvH/CTUJ/8+QmW2LfCVkZmbZuAiZmVk2LkJmZpbNgHxPSNIE4L+BQcCPIuK8zCl1i98fMrNtxYC7EpI0CPg+cDSwH3CipP3yZmVmNjANxCuhQ4D2iFgCIGk2cCxwf9aseqh8VVTmKyQzawYDsQiNApaV7ncAh5YbSJoCTEl3/ybpwR7uawTwWA+33SI6f6t0ky3/raTZ84fmPwbnn1fO/PdspNFALEKqEYvN7kRMA6Zt8Y6ktoho3dJ+cnH++TX7MTj/vJoh/wH3nhDFlc+Y0v3RwPJMuZiZDWgDsQjdDYyVtJek7YCJwJzMOZmZDUgDbjguItZLOhWYSzFFe3pELOql3W3xkF5mzj+/Zj8G559Xv89fEdF1KzMzs14wEIfjzMysn3ARMjOzbFyEeomkCZIelNQuaWrufGqRNF3SKkkLS7HhkuZJWpx+DktxSbowHc+9kg7Kl/nGXMdIukXSA5IWSfpsijfFMUjaQdJdkv6Y8v9Kiu8l6c6U/+VpAg2Stk/329P6lpz5V0gaJOkeSb9M95smf0lLJd0naYGkthRrivMn5TRU0pWS/pT+Dt7UTPmDi1CvaKKPBpoBTKiKTQVuioixwE3pPhTHMjbdpgAX9VGOnVkPnB4R+wLjgVPS49wsx/AccHhEvAEYB0yQNB44H7gg5b8WmJzaTwbWRsQ+wAWpXX/wWeCB0v1my/9tETGu9P80zXL+QPEZmL+KiH8A3kDxe2im/CEifNvKN+BNwNzS/TOBM3PnVSfXFmBh6f6DwO5peXfgwbT8P8CJtdr1lxtwLfCOZjwGYEfgDxSf3vEYMLj6XKKY0fmmtDw4tVPmvEdTPNEdDvyS4p/BmymiVSYgAAADZ0lEQVT/pcCIqlhTnD/AzsDD1Y9hs+RfuflKqHfU+migUZly6a7dImIFQPq5a4r362NKQzsHAnfSRMeQhrIWAKuAecBDwBMRsT41Kee4Mf+0fh3wqr7N+CW+A3wReDHdfxXNlX8Av5Y0P31cFzTP+bM3sBr4cRoO/ZGkV9A8+QMejustXX40UBPqt8ck6ZXAVcDnIuLJzprWiGU9hojYEBHjKK4oDgH2rdUs/exX+Ut6F7AqIuaXwzWa9sv8kzdHxEEUQ1WnSHprJ237W/6DgYOAiyLiQODvbBp6q6W/5Q+4CPWWZv5ooJWSdgdIP1eleL88JkkvpyhAP42Iq1O4qY4BICKeAG6leG9rqKTKP5KXc9yYf1q/C7CmbzPdzJuB90haCsymGJL7Ds2TPxGxPP1cBVxD8UKgWc6fDqAjIu5M96+kKErNkj/gItRbmvmjgeYAk9LyJIr3WSrxk9IMm/HAusolfy6SBFwCPBAR3y6taopjkDRS0tC0PAR4O8Uby7cAx6dm1flXjut44OZIg/s5RMSZETE6IloozvGbI+KDNEn+kl4haafKMnAksJAmOX8i4q/AMkmvS6EjKL6Spiny3yj3m1Lb6g04BvgzxRj/l3LnUyfHy4AVwAsUr5ImU4zR3wQsTj+Hp7aimPH3EHAf0NoP8v8niuGEe4EF6XZMsxwD8HrgnpT/QuA/Unxv4C6gHfg5sH2K75Dut6f1e+f+HZSO5TDgl82Uf8rzj+m2qPJ32iznT8ppHNCWzqFfAMOaKf+I8Mf2mJlZPh6OMzOzbFyEzMwsGxchMzPLxkXIzMyycREyM7NsXITMzCwbFyEzM8vGRcisn5HUkr4b5uL0PUO/ljRE0q2SWlObEenjcpD0EUm/kHSdpIclnSrptPShlndIGp71gMw64SJk1j+NBb4fEfsDTwD/3EX7A4D/R/HZZ+cCT0fxoZa3Ayf1ZqJmW8JFyKx/ejgiFqTl+RTf+9SZWyLiqYhYTfEVCdel+H0NbGuWjYuQWf/0XGl5A8XH9q9n09/sDp20f7F0/8W0rVm/5CJk1jyWAgen5eM7aWfWNFyEzJrHN4FPSvo9MCJ3MmZbgz9F28zMsvGVkJmZZeMiZGZm2bgImZlZNi5CZmaWjYuQmZll4yJkZmbZuAiZmVk2/x+kdeOvRSCKtAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "_ = plt.hist(num_words, bins = 100)\n",
    "plt.title('Distribution of num of words in one sentence')\n",
    "plt.xlabel('num')\n",
    "plt.ylabel('Counts')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:55.446Z"
    }
   },
   "outputs": [],
   "source": [
    "word_count = {}\n",
    "def count_words(x):\n",
    "    for sen in x:\n",
    "        for word in sen:\n",
    "            word_count[word] = word_count.get(word, 0) + 1\n",
    "_ = df_train.Processed_Reviews.apply(count_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:55.615Z"
    }
   },
   "outputs": [],
   "source": [
    "df_word_count = pd.DataFrame(list(word_count.items()), columns=['word', 'count'])\n",
    "df_word_count.sort_values(by='count', ascending=False, inplace=True)\n",
    "df_word_count.set_index('word', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:55.789Z"
    }
   },
   "outputs": [],
   "source": [
    "vocab_size = 40000\n",
    "df_word_count = df_word_count[:vocab_size-10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:55.959Z"
    }
   },
   "outputs": [],
   "source": [
    "word2num_series = df_word_count.reset_index().reset_index().set_index('word')['index'] + 10 # reverse 10 pos for padding, [UNK], [CLS]..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:56.128Z"
    }
   },
   "outputs": [],
   "source": [
    "def word_to_num(x):\n",
    "    res = []\n",
    "    for sen in x:\n",
    "        sen_words = []\n",
    "        for word in sen:\n",
    "            if word in word2num_series:\n",
    "                sen_words.append(word2num_series[word])\n",
    "        if len(sen_words) > 0:\n",
    "            res.append(np.array(sen_words, dtype=np.uint16))\n",
    "    return np.array(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:56.294Z"
    }
   },
   "outputs": [],
   "source": [
    "def create_data(df):\n",
    "    X = df.Processed_Reviews.apply(word_to_num)\n",
    "    y = df.label\n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:56.461Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train, y_train = create_data(df_train)\n",
    "X_val, y_val = create_data(df_validate)\n",
    "X_test, y_test = create_data(df_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-03-31T11:37:56.633Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train.to_hdf(data_pth + 'X_train.h5', key='s', model='w')\n",
    "y_train.to_hdf(data_pth + 'y_train.h5', key='s', model='w')\n",
    "\n",
    "X_val.to_hdf(data_pth + 'X_val.h5', key='s', model='w')\n",
    "y_val.to_hdf(data_pth + 'y_val.h5', key='s', model='w')\n",
    "\n",
    "X_test.to_hdf(data_pth + 'X_test.h5', key='s', model='w')\n",
    "y_test.to_hdf(data_pth + 'y_test.h5', key='s', model='w')\n",
    "word2num_series.to_hdf(data_pth + 'word2num_series.h5', key='s', model='w')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Douban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T13:59:59.144202Z",
     "start_time": "2019-04-01T13:59:58.993476Z"
    }
   },
   "outputs": [],
   "source": [
    "data_pth = 'dataset/Douban/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:00:05.592605Z",
     "start_time": "2019-04-01T14:00:00.107710Z"
    }
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(data_pth + 'train.csv')\n",
    "df_test = pd.read_csv(data_pth + 'test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:08:48.473621Z",
     "start_time": "2019-04-01T14:02:15.355538Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set size:  550000\n",
      "Validation set size:  50000\n",
      "Test set size:  100000\n"
     ]
    }
   ],
   "source": [
    "df['Processed_Reviews'] = df['content'].apply(process_zh)\n",
    "df_test['Processed_Reviews'] = df_test['content'].apply(process_zh)\n",
    "df_train, df_validate = train_test_split(df, test_size=50000, random_state=0, stratify=df.label)\n",
    "print(\"Train set size: \", len(df_train))\n",
    "print(\"Validation set size: \", len(df_validate))\n",
    "print(\"Test set size: \", len(df_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:08:50.204286Z",
     "start_time": "2019-04-01T14:08:48.477159Z"
    }
   },
   "outputs": [],
   "source": [
    "num_words = []\n",
    "def count(x):\n",
    "    for sen in x:\n",
    "        num_words.append(len(sen))\n",
    "num_sens = df_train.Processed_Reviews.apply(len)\n",
    "_ = df_train.Processed_Reviews.apply(count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:08:51.066373Z",
     "start_time": "2019-04-01T14:08:50.206292Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Counts')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XucHXV9//HX28RwE0hCFopJYEOJF+DhBVaIl7YUFMJFQ/uDNlRLpLFpES8Vi4Rqi0X5Nfzqz1gq4i+SSEKVmEYqQYIh5eKlcskCCgTUrAHJApKFhIugYODz+2M+K5PNObtnNzt7wu77+Xicx8585jvf+c6cOedzZua7M4oIzMzMqvSKZjfAzMyGPycbMzOrnJONmZlVzsnGzMwq52RjZmaVc7IxM7PKOdm8DEj6sqR/HKS69pP0K0mjcvwmSR8YjLqzvmslzRqs+vqx3M9KekzSL4d62dtD0p9I2pDvyZub3Z5m6blfjgSD+bl+OZD/z6a5JD0A7ANsAV4A7gWWAAsi4sUB1PWBiPjvfsxzE/AfEXFpf5aV834aODAi3tffeQeTpMnAz4D9I2JjM9vSX5J+DpwVEVcN4TIvAzoj4lNDtUwzH9nsGN4dEbsD+wPzgHOAhYO9EEmjB7vOHcT+wOMvt0ST9gfWNrsR1n/D+PNUjYjwq4kv4AHgnT1ihwMvAofk+GXAZ3N4AvBt4AlgE/B9ih8Nl+c8vwZ+BXwCaAUCmA08CHyvFBud9d0E/AtwG/AkcBUwPqcdSfELeJv2AtOB54Hf5vJ+XKrvAzn8CuBTwC+AjRRHbHvmtO52zMq2PQZ8spfttGfO35X1fSrrf2eu84vZjstqzHsk0Al8PNvxCHB6afrv2pzj7wd+UBoP4IPAOuBp4DPA7wM3A08By4AxddpdcxsAO2V7A3gG+HmNeQXMz/meBO4q7RM7AZ/Lbfco8GVgl77WF5iT79nzufyrM/5q4Ju5fe8HPlJqx6dzHZfk+q8F2krTJwNX5ryPA18sTfsr4D5gM7CK4uiz1nbq3h/K++VngP/JZV4HTOhl//hroIPiM7ECeHWP9+9v8/3bDFxMntUZYBt/93nK+DTghxSfyR8DR2Z8JtDeo46PASt6fq5z/ETgR1nPD4E3ZPz07vcpxzuAZaXxDcCbmv1d1ud3XbMbMNJf1Eg2GX8QOCOHf7dTUiSGLwOvzNcf8NLp0K3qKn04lgC7AbvU+VA/BBySZb5JcVoNekk2Ofzp7rKl6TfxUrL5q/xgHAC8iuIL6fIebftKtuuNwHPA6+tspyUUiXD3nPdnwOx67ewx75EUpynPz212PPAsMK5nm3P8/WybbFYAewAHZzuvz/Xak+LU56w6y667DUp1H1hn3mOB24GxFInn9cC+Oe0L2abxuU2uBv6lwfW9jK2/5F6Ry/knYEy2dT1wbOl9/k3WM4piH7wlp42i+IKdT7H/7Ay8I6edlOv+emA0RdL9YZ117d4fyvvlz4HX5P5xEzCvzrxHUfxYOZQiCf87mQhK2/jbuR33o0iK07ejjeXP00SKBHt8bsd35XgLsCtFopxaqmMNMLPG5/pQih8GR+Q2nUXxWdsp348nsv59KX64PJTzHUCRJF/R7O+yvl4+jbbjepjii6Sn31LscPtHxG8j4vuRe10vPh0Rz0TEr+tMvzwi7omIZ4B/BP5skC7Uvhf4fESsj4hfAecCM3ucfvjniPh1RPyY4kvrjT0rybb8OXBuRDwdEQ8A/xf4y3605bfA+bnNVlL8qn9tP+a/MCKeioi1wD3AdbleTwLXAvUu7jeyDXpr8+7A6yh+UNwXEY9IEsUv+Y9FxKaIeBr43xS/pAeyvm8BWiLi/Ih4PiLWU/wIKNf3g4hYGREvUBxFd79Ph1McFZ2d+9hvIuIHOe1vKBLgfRGxJdv4Jkn7N7DuAF+NiJ/lfrsMeFOdcu8FFkXEHRHxHMU2fquk1lKZeRHxREQ8CNxYqmsgbSx/nt4HrMxt82JErAbageMj4lmKH0inAkiaSvFerqhR518D/y8ibo2IFyJiMcWPmmn5fjydbf4jiqOvhyS9Lse/H/28vtsMTjY7rokUpwR6+leKX2LXSVovaW4DdW3ox/RfUPwantBQK3v36qyvXPdoig4R3cq9x56l+PXf0wSKX9w965rYj7Y8nl8mfS2rnkdLw7+uMV6vrka2QU0RcQPwRYrTPo9KWiBpD1761Xy7pCckPQF8J+Pd+rO++wOv7q4r6/sHen+fds6EORn4RY9llev9t1KdmyiO0Bp93xrZN6DHNs6k/niP5dSrayBtLH9e9gdO6bHt3kHxgxDg62SyAf4C+FYmoZ72Bz7eo57JuW4A36U4Yv3DHL6JItH8UY7v8JxsdkCS3kKxs/+g57T8Zf/xiDgAeDdwlqSjuyfXqbKvI5/JpeH9KH4VP0ZxLWHXUrtGsfUXWl/1PkzxISrXvYWtv6gb8Vi2qWddD/Wznnq2Wk/g9wapXtjObRARF0XEYRSn714DnE2xPX4NHBwRY/O1Z0Q0mjx7vm8bgPtLdY2NiN0j4vgG6toA7FfnSG0D8Dc96t0lIn7YYDsbtdU2lrQbsBeN7R8DaWN5+22gODNQnn+3iJiX068DJkh6E0XS+Xov7bigRz27RsQVOb072fxBDn8XJxsbKEl7SDoRWEpxLeTuGmVOlHRgnkp5iqK79As5+VGKc7j99T5JB0naleI8//I8XfIzil+wJ0h6JcX57J1K8z0KtEqqtx9dAXxM0hRJr6I4RfGNOr+C68q2LAMukLR7nuI4C/iP/tTTix8BfyppV0kHUlwAHiwD3gaS3iLpiNz2z1BcN3khT5l8BZgvae8sO1HSsQ22qed+chvwlKRzJO0iaZSkQ/JHT19uo+iAME/SbpJ2lvT2nPZl4FxJB2cb95R0SoNt7I+vA6dLepOknSi28a15urUv29vG/wDeLenY3G47SzpS0iSAfJ+XU5yRGA+srlPPV4C/zfdbuS1PkLR7Tv8u8McUnUA6KToGTadIqnf2o71N42SzY7ha0tMUv24+CXyeogdKLVOB/6Y4B38z8KWIuCmn/QvwqTwM//t+LP9yiouVv6S4wPsRgLwe8UHgUopfic9Q9HLq9p/593FJd9Sod1HW/T2KHk6/AT7cj3aVfTiXv57iiO/rWf9gmE/RO+tRYDHwtUGqF7ZvG+xB8SW0meI00eMUPdCg6B7fAdwi6SmKfaLRa1ALgYNyP/lWJvN3U1wTuJ/iyOlSis4PvSrNeyBFp5ZOiutrRMR/ARcCS7ON9wDHNdjGhkXE9RTXGr9Jkfh+n62vN/U273a1MSI2ADMoTjt2UXyGz2br79avU/Sa/M96PzIiop3ius0XKd7vDoqOKt3Tf0bxmf9+jj9F8Vn4n3wPdnj+p04zM6ucj2zMzKxyTjZmZlY5JxszM6uck42ZmVXON5JLEyZMiNbW1mY3w8zsZeX2229/LCJa+irnZJNaW1tpb29vdjPMzF5WJP2i71I+jWZmZkPAycbMzCrnZGNmZpVzsjEzs8o52ZiZWeWcbMzMrHJONmZmVjknGzMzq5yTjZmZVc53EBgErXOvqRl/YN4JQ9wSM7Mdk49szMysck42ZmZWOScbMzOrnJONmZlVzsnGzMwq52RjZmaVc7IxM7PKOdmYmVnlnGzMzKxylSUbSYskbZR0T4/4hyX9VNJaSf+nFD9XUkdOO7YUn56xDklzS/Epkm6VtE7SNySNyfhOOd6R01urWkczM2tMlUc2lwHTywFJfwzMAN4QEQcDn8v4QcBM4OCc50uSRkkaBVwMHAccBJyaZQEuBOZHxFRgMzA747OBzRFxIDA/y5mZWRNVlmwi4nvAph7hM4B5EfFcltmY8RnA0oh4LiLuBzqAw/PVERHrI+J5YCkwQ5KAo4DlOf9i4KRSXYtzeDlwdJY3M7MmGeprNq8B/iBPb31X0lsyPhHYUCrXmbF68b2AJyJiS4/4VnXl9Cez/DYkzZHULqm9q6tru1fOzMxqG+pkMxoYB0wDzgaW5VFHrSOPGECcPqZtHYxYEBFtEdHW0tLSV9vNzGyAhjrZdAJXRuE24EVgQsYnl8pNAh7uJf4YMFbS6B5xyvPk9D3Z9nSemZkNoaFONt+iuNaCpNcAYygSxwpgZvYkmwJMBW4D1gBTs+fZGIpOBCsiIoAbgZOz3lnAVTm8IsfJ6TdkeTMza5LKHp4m6QrgSGCCpE7gPGARsCi7Qz8PzMpEsFbSMuBeYAtwZkS8kPV8CFgFjAIWRcTaXMQ5wFJJnwXuBBZmfCFwuaQOiiOamVWto5mZNaayZBMRp9aZ9L465S8ALqgRXwmsrBFfT9FbrWf8N8Ap/WqsmZlVyncQMDOzyjnZmJlZ5ZxszMysck42ZmZWOScbMzOrnJONmZlVzsnGzMwq52RjZmaVc7IxM7PKOdmYmVnlKrtdjUHr3Gtqxh+Yd8IQt8TMrLl8ZGNmZpVzsjEzs8o52ZiZWeWcbMzMrHKVJRtJiyRtzAel9Zz295JC0oQcl6SLJHVIukvSoaWysySty9esUvwwSXfnPBdJUsbHS1qd5VdLGlfVOpqZWWOqPLK5DJjeMyhpMvAu4MFS+DiKR0FPBeYAl2TZ8RRP+DyC4kFp55WSxyVZtnu+7mXNBa6PiKnA9TluZmZNVFmyiYjvUTyWuaf5wCeAKMVmAEuicAswVtK+wLHA6ojYFBGbgdXA9Jy2R0TcnI+VXgKcVKprcQ4vLsXNzKxJhvSajaT3AA9FxI97TJoIbCiNd2ast3hnjTjAPhHxCED+3buX9syR1C6pvaurawBrZGZmjRiyZCNpV+CTwD/VmlwjFgOI90tELIiItohoa2lp6e/sZmbWoKE8svl9YArwY0kPAJOAOyT9HsWRyeRS2UnAw33EJ9WIAzyap9nIvxsHfU3MzKxfhizZRMTdEbF3RLRGRCtFwjg0In4JrABOy15p04An8xTYKuAYSeOyY8AxwKqc9rSkadkL7TTgqlzUCqC719qsUtzMzJqkyq7PVwA3A6+V1Clpdi/FVwLrgQ7gK8AHASJiE/AZYE2+zs8YwBnApTnPz4FrMz4PeJekdRS93uYN5nqZmVn/VXYjzog4tY/praXhAM6sU24RsKhGvB04pEb8ceDofjbXzMwq5DsImJlZ5ZxszMysck42ZmZWOScbMzOrnJONmZlVzsnGzMwq52RjZmaVc7IxM7PKOdmYmVnlnGzMzKxyTjZmZlY5JxszM6uck42ZmVXOycbMzCrnZGNmZpWr8uFpiyRtlHRPKfavkn4i6S5J/yVpbGnauZI6JP1U0rGl+PSMdUiaW4pPkXSrpHWSviFpTMZ3yvGOnN5a1TqamVljqjyyuQyY3iO2GjgkIt4A/Aw4F0DSQcBM4OCc50uSRkkaBVwMHAccBJyaZQEuBOZHxFRgM9D9JNDZwOaIOBCYn+XMzKyJKks2EfE9YFOP2HURsSVHbwEm5fAMYGlEPBcR91M86vnwfHVExPqIeB5YCsyQJOAoYHnOvxg4qVTX4hxeDhyd5c3MrEmaec3mr4Brc3gisKE0rTNj9eJ7AU+UEld3fKu6cvqTWX4bkuZIapfU3tXVtd0rZGZmtTUl2Uj6JLAF+Fp3qEaxGEC8t7q2DUYsiIi2iGhraWnpvdFmZjZgo4d6gZJmAScCR0dEdxLoBCaXik0CHs7hWvHHgLGSRufRS7l8d12dkkYDe9LjdJ6ZmQ2tIT2ykTQdOAd4T0Q8W5q0ApiZPcmmAFOB24A1wNTseTaGohPBikxSNwIn5/yzgKtKdc3K4ZOBG0pJzczMmqCyIxtJVwBHAhMkdQLnUfQ+2wlYndfsb4mIv42ItZKWAfdSnF47MyJeyHo+BKwCRgGLImJtLuIcYKmkzwJ3AgszvhC4XFIHxRHNzKrW0czMGlNZsomIU2uEF9aIdZe/ALigRnwlsLJGfD1Fb7We8d8Ap/SrsWZmVinfQcDMzCrnZGNmZpVzsjEzs8o52ZiZWeWcbMzMrHJONmZmVjknGzMzq5yTjZmZVc7JxszMKudkY2ZmlXOyMTOzyjnZmJlZ5ZxszMysck42ZmZWOScbMzOrnJONmZlVrt/JRtI4SW9ooNwiSRsl3VOKjZe0WtK6/Dsu45J0kaQOSXdJOrQ0z6wsv07SrFL8MEl35zwXKR/9WW8ZZmbWPA0lG0k3SdpD0njgx8BXJX2+j9kuA6b3iM0Fro+IqcD1OQ5wHDA1X3OAS3K54ykeJ30ExVM5zyslj0uybPd80/tYhpmZNUmjRzZ7RsRTwJ8CX42Iw4B39jZDRHwP2NQjPANYnMOLgZNK8SVRuAUYK2lf4FhgdURsiojNwGpgek7bIyJujogAlvSoq9YyzMysSRpNNqPzC/7PgG9vx/L2iYhHAPLv3hmfCGwolevMWG/xzhrx3paxDUlzJLVLau/q6hrwSpmZWe8aTTb/DKwCOiJijaQDgHWD2A7ViMUA4v0SEQsioi0i2lpaWvo7u5mZNajRZPNIRLwhIj4IEBHrgb6u2dTyaB4hkX83ZrwTmFwqNwl4uI/4pBrx3pZhZmZN0miy+fcGY31ZAXT3KJsFXFWKn5a90qYBT+YpsFXAMdkDbhxwDLAqpz0taVr2QjutR121lmFmZk0yureJkt4KvA1okXRWadIewKg+5r0COBKYIKmTolfZPGCZpNnAg8ApWXwlcDzQATwLnA4QEZskfQZYk+XOj4juTgdnUPR42wW4Nl/0sgwzM2uSXpMNMAZ4VZbbvRR/Cji5txkj4tQ6k46uUTaAM+vUswhYVCPeDhxSI/54rWWYmVnz9JpsIuK7wHclXRYRvxiiNpmZ2TDT15FNt50kLQBay/NExFFVNMrMzIaXRpPNfwJfBi4FXqiuOWZmNhw1mmy2RMQllbbEzMyGrUa7Pl8t6YOS9s0bXY7P+5aZmZn1qdEjm+7/Wzm7FAvggMFtjpmZDUcNJZuImFJ1Q8zMbPhqKNlIOq1WPCKWDG5zzMxsOGr0NNpbSsM7U/zT5B0Ut/Y3MzPrVaOn0T5cHpe0J3B5JS0yM7Nhp9+PhU7PUjwd08zMrE+NXrO5mpeeFzMKeD2wrKpGmZnZ8NLoNZvPlYa3AL+IiM56hc3MzMoaOo2WN+T8CcWdn8cBz1fZKDMzG14aPY32Z8C/AjdRPJL53yWdHRHLK2zbsNU695qa8QfmnTDELTEzGxqNdhD4JPCWiJgVEacBhwP/ONCFSvqYpLWS7pF0haSdJU2RdKukdZK+IWlMlt0pxztyemupnnMz/lNJx5bi0zPWIWnuQNtpZmaDo9Fk84qI2Fgaf7wf825F0kTgI0BbRBxC0eFgJnAhMD8ipgKbgdk5y2xgc0QcCMzPckg6KOc7GJgOfEnSKEmjgIuB44CDgFOzrJmZNUmjCeM7klZJer+k9wPXUDzKeaBGA7tIGg3sCjwCHAV0n5ZbDJyUwzNynJx+tCRlfGlEPBcR91M8UvrwfHVExPqIeB5YmmXNzKxJer1mI+lAYJ+IOFvSnwLvoLhmczPwtYEsMCIekvQ54EHg18B1wO3AExGxJYt1AhNzeCKwIefdIulJYK+M31KqujzPhh7xI+qs3xxgDsB+++03kNUxM7MG9HVk8wXgaYCIuDIizoqIj1Ec1XxhIAuUNI7iSGMK8GpgN4pTXj11/1+P6kzrb3zbYMSCiGiLiLaWlpa+mm5mZgPUV7JpjYi7egYjop3iEdED8U7g/ojoiojfAlcCbwPG5mk1gEnAwzncCUwGyOl7ApvK8R7z1IubmVmT9JVsdu5l2i4DXOaDwDRJu+a1l6OBe4EbgZOzzCzgqhxewUvP0zkZuCEiIuMzs7faFIrb59wGrAGmZu+2MRSdCFYMsK1mZjYI+ko2ayT9dc+gpNkU11n6LSJupbjQfwdwd7ZhAXAOcJakDoprMgtzloXAXhk/C5ib9ayluGXOvcB3gDMj4oW87vMhYBVwH7Asy5qZWZOoOEioM1HaB/gvijsGdCeXNmAM8CcR8cvKWzhE2traor29fUDz1vsnzf7yP3Wa2cuNpNsjoq2vcr32RouIR4G3Sfpj4JAMXxMRNwxCG83MbIRo9Hk2N1JcUzEzM+u3gT7PxszMrGFONmZmVjknGzMzq5yTjZmZVc7JxszMKudkY2ZmlXOyMTOzyjnZmJlZ5ZxszMysck42ZmZWOScbMzOrnJONmZlVzsnGzMwq15RkI2mspOWSfiLpPklvlTRe0mpJ6/LvuCwrSRdJ6pB0l6RDS/XMyvLrJM0qxQ+TdHfOc1E+EdTMzJqkWUc2/wZ8JyJeB7yR4omac4HrI2IqcH2OAxxH8cjnqcAc4BIASeOB84AjgMOB87oTVJaZU5pv+hCsk5mZ1THkyUbSHsAfko99jojnI+IJYAawOIstBk7K4RnAkijcAoyVtC9wLLA6IjZFxGZgNTA9p+0RETdH8RjSJaW6zMysCZpxZHMA0AV8VdKdki6VtBuwT0Q8ApB/987yE4ENpfk7M9ZbvLNGfBuS5khql9Te1dW1/WtmZmY1NSPZjAYOBS6JiDcDz/DSKbNaal1viQHEtw1GLIiItohoa2lp6b3VZmY2YM1INp1AZ0TcmuPLKZLPo3kKjPy7sVR+cmn+ScDDfcQn1YibmVmTDHmyiYhfAhskvTZDRwP3AiuA7h5ls4CrcngFcFr2SpsGPJmn2VYBx0galx0DjgFW5bSnJU3LXminleoyM7MmGN2k5X4Y+JqkMcB64HSKxLdM0mzgQeCULLsSOB7oAJ7NskTEJkmfAdZkufMjYlMOnwFcBuwCXJsvMzNrkqYkm4j4EdBWY9LRNcoGcGadehYBi2rE24FDtrOZZmY2SJp1ZGM1tM69pu60B+adMIQtMTMbXL5djZmZVc7JxszMKudkY2ZmlXOyMTOzyjnZmJlZ5ZxszMysck42ZmZWOScbMzOrnJONmZlVzsnGzMwq52RjZmaVc7IxM7PKOdmYmVnlnGzMzKxyTUs2kkZJulPSt3N8iqRbJa2T9I18sBqSdsrxjpzeWqrj3Iz/VNKxpfj0jHVImjvU62ZmZltr5pHNR4H7SuMXAvMjYiqwGZid8dnA5og4EJif5ZB0EDATOBiYDnwpE9go4GLgOOAg4NQsa2ZmTdKUh6dJmgScAFwAnCVJwFHAX2SRxcCngUuAGTkMsBz4YpafASyNiOeA+yV1AIdnuY6IWJ/LWppl7614tSpV78Fqfqiamb0cNOvI5gvAJ4AXc3wv4ImI2JLjncDEHJ4IbADI6U9m+d/Fe8xTL74NSXMktUtq7+rq2t51MjOzOoY82Ug6EdgYEbeXwzWKRh/T+hvfNhixICLaIqKtpaWll1abmdn2aMZptLcD75F0PLAzsAfFkc5YSaPz6GUS8HCW7wQmA52SRgN7AptK8W7leerFzcysCYb8yCYizo2ISRHRSnGB/4aIeC9wI3ByFpsFXJXDK3KcnH5DRETGZ2ZvtSnAVOA2YA0wNXu3jcllrBiCVTMzszqa0kGgjnOApZI+C9wJLMz4QuDy7ACwiSJ5EBFrJS2juPC/BTgzIl4AkPQhYBUwClgUEWuHdE3MzGwrTU02EXETcFMOr+el3mTlMr8BTqkz/wUUPdp6xlcCKwexqWZmth18BwEzM6uck42ZmVXOycbMzCrnZGNmZpVzsjEzs8o52ZiZWeWcbMzMrHJONmZmVjknGzMzq5yTjZmZVc7JxszMKrcj3YjTBsBP8DSzlwMf2ZiZWeWcbMzMrHJONmZmVrkhTzaSJku6UdJ9ktZK+mjGx0taLWld/h2XcUm6SFKHpLskHVqqa1aWXydpVil+mKS7c56LJGmo19PMzF7SjCObLcDHI+L1wDTgTEkHAXOB6yNiKnB9jgMcR/HI56nAHOASKJITcB5wBMVD187rTlBZZk5pvulDsF5mZlbHkCebiHgkIu7I4aeB+4CJwAxgcRZbDJyUwzOAJVG4BRgraV/gWGB1RGyKiM3AamB6TtsjIm6OiACWlOoyM7MmaOo1G0mtwJuBW4F9IuIRKBISsHcWmwhsKM3WmbHe4p014mZm1iRNSzaSXgV8E/i7iHiqt6I1YjGAeK02zJHULqm9q6urryabmdkANSXZSHolRaL5WkRcmeFH8xQY+XdjxjuByaXZJwEP9xGfVCO+jYhYEBFtEdHW0tKyfStlZmZ1NaM3moCFwH0R8fnSpBVAd4+yWcBVpfhp2SttGvBknmZbBRwjaVx2DDgGWJXTnpY0LZd1WqkuMzNrgmbcrubtwF8Cd0v6Ucb+AZgHLJM0G3gQOCWnrQSOBzqAZ4HTASJik6TPAGuy3PkRsSmHzwAuA3YBrs2XmZk1yZAnm4j4AbWvqwAcXaN8AGfWqWsRsKhGvB04ZDuaaWZmg8h3EDAzs8o52ZiZWeWcbMzMrHJONmZmVjknGzMzq5yTjZmZVc7JxszMKteMf+q0Jmqde03N+APzThjilpjZSOIjGzMzq5yPbIapekcwZmbN4CMbMzOrnJONmZlVzsnGzMwq52RjZmaVcwcBA9wl2syq5SMbMzOr3LBNNpKmS/qppA5Jc5vdHjOzkWxYnkaTNAq4GHgX0AmskbQiIu5tbstefnx6zcwGw7BMNsDhQEdErAeQtBSYATjZDJL+/tOok5PZyDZck81EYENpvBM4omchSXOAOTn6K0k/7ccyJgCPDbiFw0dD20EXDkFLmsv7Q8Hb4SUjZVvs30ih4ZpsVCMW2wQiFgALBrQAqT0i2gYy73Di7VDwdih4O7zE22Jrw7WDQCcwuTQ+CXi4SW0xMxvxhmuyWQNMlTRF0hhgJrCiyW0yMxuxhuVptIjYIulDwCpgFLAoItYO8mIGdPptGPJ2KHg7FLwdXuJtUaKIbS5lmJmZDarhehrNzMx2IE42ZmZWOSebfhqpt8GRNFnSjZLuk7RW0kczPl7Saknr8u+4Zrd1KEgaJelOSd/O8SmSbs3t8I3smDLsSRorabmkn+S+8daRuE9I+lh+Lu6RdIWknUfqPlGPk00/lG6DcxxwEHCqpIOa26ohswX4eES8HpgGnJnrPhe4PiKmAtfn+EjwUeC+0viFwPzcDpuB2U1p1dD7N+A7EfHW0CV4AAADS0lEQVQ64I0U22RE7ROSJgIfAdoi4hCKTkkzGbn7RE1ONv3zu9vgRMTzQPdtcIa9iHgkIu7I4acpvlQmUqz/4iy2GDipOS0cOpImAScAl+a4gKOA5VlkpGyHPYA/BBYCRMTzEfEEI3CfoOjZu4uk0cCuwCOMwH2iN042/VPrNjgTm9SWppHUCrwZuBXYJyIegSIhAXs3r2VD5gvAJ4AXc3wv4ImI2JLjI2W/OADoAr6apxQvlbQbI2yfiIiHgM8BD1IkmSeB2xmZ+0RdTjb909BtcIYzSa8Cvgn8XUQ81ez2DDVJJwIbI+L2crhG0ZGwX4wGDgUuiYg3A88wzE+Z1ZLXpGYAU4BXA7tRnGrvaSTsE3U52fTPiL4NjqRXUiSar0XElRl+VNK+OX1fYGOz2jdE3g68R9IDFKdRj6I40hmbp1Bg5OwXnUBnRNya48spks9I2yfeCdwfEV0R8VvgSuBtjMx9oi4nm/4ZsbfByesSC4H7IuLzpUkrgFk5PAu4aqjbNpQi4tyImBQRrRTv/w0R8V7gRuDkLDbstwNARPwS2CDptRk6muIxHiNqn6A4fTZN0q75OeneDiNun+iN7yDQT5KOp/gl230bnAua3KQhIekdwPeBu3npWsU/UFy3WQbsR/GhOyUiNjWlkUNM0pHA30fEiZIOoDjSGQ/cCbwvIp5rZvuGgqQ3UXSUGAOsB06n+BE7ovYJSf8M/DlFr807gQ9QXKMZcftEPU42ZmZWOZ9GMzOzyjnZmJlZ5ZxszMysck42ZmZWOScbMzOrnJONmZlVzsnGzMwq52Rj1iSSWvMZMF/JZ6FcJ2kXSTdJassyE/LWOEh6v6RvSbpa0v2SPiTprLwJ5i2Sxjd1hcx64WRj1lxTgYsj4mDgCeB/9VH+EOAvKB53cQHwbN4E82bgtCobarY9nGzMmuv+iPhRDt8OtPZR/saIeDoiuihuZX91xu9uYF6zpnGyMWuu8r2yXqC4bf8WXvps7txL+RdL4y/mvGY7JCcbsx3PA8BhOXxyL+XMXjacbMx2PJ8DzpD0Q2BCsxtjNhh812czM6ucj2zMzKxyTjZmZlY5JxszM6uck42ZmVXOycbMzCrnZGNmZpVzsjEzs8r9f/hvDc4ZIOrUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "_ = plt.hist(num_sens, bins = 50)\n",
    "plt.title('Distribution of num of sentence in one review')\n",
    "plt.xlabel('num')\n",
    "plt.ylabel('Counts')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:08:52.161214Z",
     "start_time": "2019-04-01T14:08:51.069883Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Counts')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHHxJREFUeJzt3XuYXFWZ7/HvayJXuRM9EAINkpkh8IyCEfByPD7iSACdMB4veMYhOjiMCt7wFkbniPcwR0VRxIPAcNEjMsgMICgiAo4PEAjIPYPEEEgEIRBuigiB9/yxV4eiqequTvdKdZffz/PU07vWXnvttWrvrl/tXbt3R2YiSVJNz+l1ByRJ/c+wkSRVZ9hIkqozbCRJ1Rk2kqTqDBtJUnWGzSQSEd+KiH8ep7a2j4jfRcSU8vzSiHjXeLRd2vtRRMwbr/ZGsd7PRcR9EfHbdb3usYiIv4mI5WWb7N7DfpwSEZ9bi+V+FxE71eiT+sPUXndAjYhYBrwAWA08CdwCnAackJlPAWTmu0fR1rsy86ed6mTmncDzxtbrNes7Ctg5M9/e0v5+49H2KPsxA/gwsENm3ruu1z9GXwIOz8xzet2RtZGZ47IvTTYRcQqwIjM/2eu+THQe2Uwsb8jMTYAdgAXAx4GTxnslEdGvHzJ2AO6fhEEDTd9vXlcr6+N9QBNVZvqYAA9gGfDaIWV7Ak8Bu5XnpwCfK9NbAz8EHgRWAf9J8+Hh9LLMH4DfAR8DBoAEDgHuBH7eUja1tHcp8EXgKuAh4BxgyzLv1TSf3p7VX2AO8DjwRFnf9S3tvatMPwf4JHAHcC/NEdtmZd5gP+aVvt0HfGKY12mzsvzK0t4nS/uvLWN+qvTjlDbLvhpYQXP0cy9wN/DOlvlr+lyevwP4RcvzBN4L3AY8AnwWeCFwBfAwcCawXod+t30NgPVLfxP4PfDrNst+Gvh6mX5uqfcv5fmGwGPAFuX5X9OE1oNlPLsM2WYfB24A/khzZmN34Noynu8DZzDCPtZhfElzdAvNfnoccH5pdyHwwmG26Uh9/kjp80Oljxu0zH89cF1Z9nLgLzusI4Bjymv/UGlv8PdqfZojyzuBe4BvARuOtM8Ah9Ls94+XbXheKd8W+AHNPno78P6WfhxV9pPTymtzMzC7Zf4M4Oyy7P3AN1rm/T2wGHgAuJDmCL7n711dv8f1ugM+yoZoEzal/E7gPWX6lJY3gi+WX4rnlsd/B6JdWzz9hn4asDHNG9RgWWvY/AbYrdT5AfCdMu/VdAibMn3UYN2W+ZfydNj8PbAE2Inm1N3ZwOlD+vbt0q8X0bwR7tLhdTqNJgg3Kcv+CjikUz+HLPtqmtOUnymv2f7Aozz9Rr2mz+X5O3h22JwLbArsWvp5cRnXZjSnPud1WHfH16Cl7Z07LPsa4MYy/XLg18DClnmDAf9nNEH0V2V8HyvrXK9lm11H84a2IbAeTfh9qNR/E82b54j7WJs+Dg2bVTQflqYC3wXO6LBcN32+iuYNfEuaN9t3l3l70ATAXsAUmg8sy4D126xnX+AaYHOa4NkF2KbM+2rZrlvS7FfnAV/scp85ZfD1Ks+fU9bzv8vruxOwFNi35XflsdLOlPIaX1nmTQGupwnFjYENgFeWeQeW12WX8pp+Eri81+9bo3l4Gm3iu4vml2CoJ4BtaD7dPJGZ/5llrxzGUZn5+8z8Q4f5p2fmTZn5e+CfgbcMXkAwRn8LfCUzl2bm74AjgYOGnMr5dGb+ITOvp/mFe9HQRkpf3gocmZmPZOYy4MvA342iL08Anymv2QU0n0j/fBTLH52ZD2fmzcBNwE/KuB4CfkRzpNBON69BJ1cAMyNiK+BVNKdWp0fE84D/AVxW6r0VOD8zL8rMJ2g+rW9IE1CDjs3M5WUf2JvmDfSr5fU4C7i6pe7a7GODzs7MqzJzNU3YvLhDvW77fFdmrqIJgsG2/gH4v5m5MDOfzMxTaT4A7N1mPU/QBMlf0ATm4sy8OyKitPOhzFyVmY8AXwAOGrJst/vMS4FpmfmZzHw8M5fSfJBqbe8XmXlBZj5JcyZicF/fkyZUP1p+Tx/LzF+Uef9IE4CLy2v6BeDFEbFDh35MOIbNxDed5lPiUP+H5pPOTyJiaUTM76Kt5aOYfwfNG9HWXfVyeNuW9lrbnkpzQcSg1qvHHqX9xQtb8/Sn8da2po+iL/eXX9aR1tXJPS3Tf2jzvFNb3bwGbZVgWEQTLK+iCZfLgVfwzLB5xjqyubBkOc98fVq38bbAb4YESGsf12YfG9TN9uy2z53a2gH4cEQ8OPigOWrbduhKMvNnwDdoTu/dExEnRMSmwDRgI+CaljZ+XMoHjWaf2QHYdkif/onh9/UNyoeOGcAdQ9bV2u7XWtpcRXOENpp9v6cMmwksIl5KszP9Yui88sn+w5m5E/AG4IiI2GdwdocmR/pUOqNlenuaT3T30Zzm2KilX1N45i/jSO3eRfPL0tr2ap75Rt2N+0qfhrb1m1G208kzxgn8t3FqF8b+GlxGc8psd5qjj8toTg3tSfMd3LPWUT61z+CZr0/rtrqb5ggphvSrqTj8PjZeuulzJ8uBz2fm5i2PjTLze+0qZ+axmfkSmlOgfwZ8lGaf+gOwa0sbm2X3V9cN3feXA7cP6dMmmbl/l+PZvsPR7nLgH4e0u2FmXt5lP3vOsJmAImLTiHg9zZe138nMG9vUeX1E7Fx+OR+muVz6yTL7HppzxaP19oiYFREb0ZyjPqsc6v+K5tPXARHxXJrzxeu3LHcPMBARnfan7wEfiogdy6mfLwDf7/AJrqPSlzOBz0fEJuUUwhHAd0bTzjCuA94YERtFxM40F1SMl7G+BpcBBwO3ZObjlO+XaN7YVpY6ZwIHRMQ+ZTt9mOa0Uqc3pCtoAu/9ETE1It5IE17AiPvYeBltn1t9G3h3ROwVjY3LPrrJ0IoR8dJSb/ACi8eAJ8uR1LeBYyLi+aXu9IjYt8v+D/1duwp4OCI+HhEbRsSUiNitfHAcyVU0HwAWlLFsEBGvKPO+BRwZEbuWPm4WEW/uso8TgmEzsZwXEY/QfIr5BPAV4J0d6s4Efkpz/vgK4JuZeWmZ90Xgk+WQ+yOjWP/pNF94/pbmy8n3A5TvI94LnEjzifP3NFfoDPq38vP+iLi2Tbsnl7Z/TnN1zmPA+0bRr1bvK+tfSnPE9/9K++PhGJori+4BTqX5rmG8jPU1uJzmu4zBo5hbShuDz8nMW4G3A1+n+cT+BprL6R9v12ApfyPNhRAP0Hx/cnZLleH2sXEx2j4PWXYRzfct3yj9X0IzlnY2pQmVB2hO291P8/0QNFfoLQGujIiHacbc7fd4JwGzyu/af5QPRG+g+V7p9jKmE2kuIBlpPIPL7kxzYdAKmm1CZv47cDRwRunjTcA6/1u2sRi8ekmSpGo8spEkVWfYSJKqM2wkSdUZNpKk6rwZX7H11lvnwMBAr7shSZPKNddcc19mThupnmFTDAwMsGjRol53Q5ImlYi4Y+RankaTJK0Dho0kqTrDRpJUnWEjSarOsJEkVWfYSJKqM2wkSdUZNpKk6gwbSVJ13kFgnA3MP3/N9LIFB/SwJ5I0cXhkI0mqzrCRJFVn2EiSqjNsJEnVGTaSpOoMG0lSdYaNJKk6w0aSVJ1hI0mqzrCRJFVn2EiSqjNsJEnVGTaSpOoMG0lSdYaNJKk6w0aSVJ1hI0mqzrCRJFVn2EiSqjNsJEnVGTaSpOoMG0lSdYaNJKm6qmETER+KiJsj4qaI+F5EbBARO0bEwoi4LSK+HxHrlbrrl+dLyvyBlnaOLOW3RsS+LeVzStmSiJjfUt52HZKk3qgWNhExHXg/MDszdwOmAAcBRwPHZOZM4AHgkLLIIcADmbkzcEypR0TMKsvtCswBvhkRUyJiCnAcsB8wC3hbqcsw65Ak9UDt02hTgQ0jYiqwEXA38BrgrDL/VODAMj23PKfM3yciopSfkZl/zMzbgSXAnuWxJDOXZubjwBnA3LJMp3VIknqgWthk5m+ALwF30oTMQ8A1wIOZubpUWwFML9PTgeVl2dWl/lat5UOW6VS+1TDrkCT1QM3TaFvQHJXsCGwLbExzymuoHFykw7zxKm/Xx0MjYlFELFq5cmW7KpKkcVDzNNprgdszc2VmPgGcDbwc2LycVgPYDrirTK8AZgCU+ZsBq1rLhyzTqfy+YdbxDJl5QmbOzszZ06ZNG8tYJUnDqBk2dwJ7R8RG5XuUfYBbgEuAN5U684BzyvS55Tll/s8yM0v5QeVqtR2BmcBVwNXAzHLl2Xo0FxGcW5bptA5JUg/U/M5mIc2X9NcCN5Z1nQB8HDgiIpbQfL9yUlnkJGCrUn4EML+0czNwJk1Q/Rg4LDOfLN/JHA5cCCwGzix1GWYdkqQeiOZAQLNnz85FixaNuZ2B+eevmV624IAxtydJE1lEXJOZs0eq5x0EJEnVGTaSpOoMG0lSdYaNJKk6w0aSVJ1hI0mqzrCRJFVn2EiSqjNsJEnVGTaSpOoMG0lSdYaNJKk6w0aSVJ1hI0mqzrCRJFVn2EiSqjNsJEnVGTaSpOoMG0lSdYaNJKk6w0aSVJ1hI0mqzrCRJFVn2EiSqjNsJEnVGTaSpOqm9roD/Wxg/vlrppctOKCHPZGk3vLIRpJUnWEjSarOsJEkVWfYSJKq8wKBcdB6IYAk6dk8spEkVWfYSJKqM2wkSdUZNpKk6qqGTURsHhFnRcR/RcTiiHhZRGwZERdFxG3l5xalbkTEsRGxJCJuiIg9WtqZV+rfFhHzWspfEhE3lmWOjYgo5W3XIUnqjdpHNl8DfpyZfwG8CFgMzAcuzsyZwMXlOcB+wMzyOBQ4HprgAD4F7AXsCXyqJTyOL3UHl5tTyjutQ5LUA9XCJiI2BV4FnASQmY9n5oPAXODUUu1U4MAyPRc4LRtXAptHxDbAvsBFmbkqMx8ALgLmlHmbZuYVmZnAaUPaarcOSVIP1Dyy2QlYCfxrRPwyIk6MiI2BF2Tm3QDl5/NL/enA8pblV5Sy4cpXtClnmHU8Q0QcGhGLImLRypUr136kkqRh1QybqcAewPGZuTvwe4Y/nRVtynItyruWmSdk5uzMnD1t2rTRLCpJGoWaYbMCWJGZC8vzs2jC555yCozy896W+jNalt8OuGuE8u3alDPMOiRJPVAtbDLzt8DyiPjzUrQPcAtwLjB4Rdk84JwyfS5wcLkqbW/goXIK7ELgdRGxRbkw4HXAhWXeIxGxd7kK7eAhbbVbhySpB2rfG+19wHcjYj1gKfBOmoA7MyIOAe4E3lzqXgDsDywBHi11ycxVEfFZ4OpS7zOZuapMvwc4BdgQ+FF5ACzosA5JUg9UDZvMvA6Y3WbWPm3qJnBYh3ZOBk5uU74I2K1N+f3t1iFJ6g3vICBJqs6wkSRVZ9hIkqozbCRJ1Rk2kqTqRh025e9d/rJGZyRJ/amrsImISyNi03IH5utp7nf2lbpdkyT1i26PbDbLzIeBNwL/mpkvAV5br1uSpH7SbdhMLfcYewvww4r9kST1oW7D5tM09yhbkplXR8ROwG31uiVJ6ifd3q7m7sxcc1FAZi71OxtJUre6PbL5epdlkiQ9y7BHNhHxMuDlwLSIOKJl1qbAlJodkyT1j5FOo60HPK/U26Sl/GHgTbU6JUnqL8OGTWZeBlwWEadk5h3rqE+SpD7T7QUC60fECcBA6zKZ+ZoanZIk9Zduw+bfgG8BJwJP1uuOJKkfdRs2qzPz+Ko9kST1rW4vfT4vIt4bEdtExJaDj6o9kyT1jW6PbOaVnx9tKUtgp/HtjiSpH3UVNpm5Y+2OSJL6V1dhExEHtyvPzNPGtzuSpH7U7Wm0l7ZMbwDsA1wLGDaSpBF1exrtfa3PI2Iz4PQqPZIk9Z1R/1vo4lFg5nh2RJLUv7r9zuY8mqvPoLkB5y7AmbU6JUnqL91+Z/OllunVwB2ZuaJCfyRJfair02jlhpz/RXPn5y2Ax2t2SpLUX7oKm4h4C3AV8GbgLcDCiPBfDEiSutLtabRPAC/NzHsBImIa8FPgrFodkyT1j26vRnvOYNAU949iWUnSn7huj2x+HBEXAt8rz98KXFCnS5KkfjNs2ETEzsALMvOjEfFG4JVAAFcA310H/ZMk9YGRToV9FXgEIDPPzswjMvNDNEc1X63dOUlSfxgpbAYy84ahhZm5iOZfREuSNKKRwmaDYeZtOJ4dkST1r5HC5uqI+IehhRFxCHBNNyuIiCkR8cuI+GF5vmNELIyI2yLi+xGxXilfvzxfUuYPtLRxZCm/NSL2bSmfU8qWRMT8lvK265Ak9cZIYfNB4J0RcWlEfLk8LgPeBXygy3V8AFjc8vxo4JjMnAk8ABxSyg8BHsjMnYFjSj0iYhZwELArMAf4ZgmwKcBxwH7ALOBtpe5w65Ak9cCwYZOZ92Tmy4FPA8vK49OZ+bLM/O1IjUfEdsABwInleQCv4ek/Bj0VOLBMzy3PKfP3KfXnAmdk5h8z83ZgCbBneSzJzKWZ+ThwBjB3hHVIknqg2/9ncwlwyVq0/1XgYzT3VAPYCngwM1eX5yuA6WV6OrC8rG91RDxU6k8Hrmxps3WZ5UPK9xphHc8QEYcChwJsv/32azE8SVI3qt0FICJeD9ybma3f7USbqjnCvPEqf3Zh5gmZOTszZ0+bNq1dFUnSOOj2DgJr4xXAX0fE/jRXtW1Kc6SzeURMLUce2wF3lforgBnAioiYCmwGrGopH9S6TLvy+4ZZhySpB6od2WTmkZm5XWYO0HzB/7PM/Fua03GDd4yeB5xTps8tzynzf5aZWcoPKler7UjzH0KvAq4GZpYrz9Yr6zi3LNNpHZKkHujFzTQ/DhwREUtovl85qZSfBGxVyo8A5gNk5s00/xX0FuDHwGGZ+WQ5ajkcuJDmarczS93h1iFJ6oGap9HWyMxLgUvL9FKaK8mG1nmM5v/ltFv+88Dn25RfQJsbgnZahySpN/w3AZKk6gwbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVWfYSJKqM2wkSdUZNpKk6gwbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVbdO/sWAYGD++Wumly04oIc9kaR1zyMbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVWfYSJKqM2wkSdUZNpKk6gwbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVWfYSJKqM2wkSdUZNpKk6gwbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVVctbCJiRkRcEhGLI+LmiPhAKd8yIi6KiNvKzy1KeUTEsRGxJCJuiIg9WtqaV+rfFhHzWspfEhE3lmWOjYgYbh2SpN6oeWSzGvhwZu4C7A0cFhGzgPnAxZk5E7i4PAfYD5hZHocCx0MTHMCngL2APYFPtYTH8aXu4HJzSnmndUiSeqBa2GTm3Zl5bZl+BFgMTAfmAqeWaqcCB5bpucBp2bgS2DwitgH2BS7KzFWZ+QBwETCnzNs0M6/IzAROG9JWu3VIknpgnXxnExEDwO7AQuAFmXk3NIEEPL9Umw4sb1lsRSkbrnxFm3KGWcfQfh0aEYsiYtHKlSvXdniSpBFUD5uIeB7wA+CDmfnwcFXblOValHctM0/IzNmZOXvatGmjWVSSNApVwyYinksTNN/NzLNL8T3lFBjl572lfAUwo2Xx7YC7Rijfrk35cOuQJPVAzavRAjgJWJyZX2mZdS4weEXZPOCclvKDy1VpewMPlVNgFwKvi4gtyoUBrwMuLPMeiYi9y7oOHtJWu3VIknpgasW2XwH8HXBjRFxXyv4JWACcGRGHAHcCby7zLgD2B5YAjwLvBMjMVRHxWeDqUu8zmbmqTL8HOAXYEPhReTDMOiRJPVAtbDLzF7T/XgVgnzb1EzisQ1snAye3KV8E7Nam/P5265Ak9YZ3EJAkVWfYSJKqM2wkSdUZNpKk6gwbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVWfYSJKqM2wkSdXVvBGnOhiYf/6a6WULDuhhTyRp3fDIRpJUnWEjSarOsJEkVWfYSJKqM2wkSdUZNpKk6gwbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVWfYSJKqM2wkSdUZNpKk6gwbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVTe11x34Uzcw//w108sWHNDDnkhSPR7ZSJKq88hmAvEoR1K/8shGklSdYSNJqq5vT6NFxBzga8AU4MTMXNDjLo2Kp9Qk9ZO+PLKJiCnAccB+wCzgbRExq7e9kqQ/Xf16ZLMnsCQzlwJExBnAXOCWnvZqLbUe5bTyiEfSZNGvYTMdWN7yfAWw19BKEXEocGh5+ruIuHUt1rU1cN9aLDdmcXTV5ns2rsr6cVz9OCZwXJPFDt1U6tewiTZl+ayCzBOAE8a0oohFmTl7LG1MRI5r8ujHMYHj6jd9+Z0NzZHMjJbn2wF39agvkvQnr1/D5mpgZkTsGBHrAQcB5/a4T5L0J6svT6Nl5uqIOBy4kObS55Mz8+ZKqxvTabgJzHFNHv04JnBcfSUyn/VVhiRJ46pfT6NJkiYQw0aSVJ1hs5YiYk5E3BoRSyJifq/7MxYRsSwiboyI6yJiUSnbMiIuiojbys8tet3PkUTEyRFxb0Tc1FLWdhzROLZsvxsiYo/e9Xx4HcZ1VET8pmyz6yJi/5Z5R5Zx3RoR+/am18OLiBkRcUlELI6ImyPiA6V8Um+vYcY1qbfXuMhMH6N80Fx08GtgJ2A94HpgVq/7NYbxLAO2HlL2L8D8Mj0fOLrX/exiHK8C9gBuGmkcwP7Aj2j+JmtvYGGv+z/KcR0FfKRN3Vllf1wf2LHsp1N6PYY2/dwG2KNMbwL8qvR9Um+vYcY1qbfXeDw8slk7a26Hk5mPA4O3w+knc4FTy/SpwIE97EtXMvPnwKohxZ3GMRc4LRtXAptHxDbrpqej02FcncwFzsjMP2bm7cASmv11QsnMuzPz2jL9CLCY5s4fk3p7DTOuTibF9hoPhs3aaXc7nOF2qIkugZ9ExDXlFj4AL8jMu6H5BQKe37PejU2ncfTDNjy8nFI6ueU056QbV0QMALsDC+mj7TVkXNAn22ttGTZrp6vb4Uwir8jMPWjukn1YRLyq1x1aByb7NjweeCHwYuBu4MulfFKNKyKeB/wA+GBmPjxc1TZlk2lcfbG9xsKwWTt9dTuczLyr/LwX+Heaw/h7Bk9TlJ/39q6HY9JpHJN6G2bmPZn5ZGY+BXybp0+9TJpxRcRzad6Qv5uZZ5fiSb+92o2rH7bXWBk2a6dvbocTERtHxCaD08DrgJtoxjOvVJsHnNObHo5Zp3GcCxxcrnLaG3ho8PTNZDDk+4q/odlm0IzroIhYPyJ2BGYCV63r/o0kIgI4CVicmV9pmTWpt1encU327TUuen2FwmR90Fwd8yuaq0c+0ev+jGEcO9FcDXM9cPPgWICtgIuB28rPLXvd1y7G8j2aUxRP0HxiPKTTOGhOXxxXtt+NwOxe93+U4zq99PsGmjesbVrqf6KM61Zgv173v8OYXklzuugG4Lry2H+yb69hxjWpt9d4PLxdjSSpOk+jSZKqM2wkSdUZNpKk6gwbSVJ1ho0kqTrDRpJUnWEjSarOsJF6JCIGyv89+Xb53yc/iYgNI+LSiJhd6mwdEcvK9Dsi4j8i4ryIuD0iDo+IIyLilxFxZURs2dMBScMwbKTemgkcl5m7Ag8C/3OE+rsB/4vm3lqfBx7NzN2BK4CDa3ZUGgvDRuqt2zPzujJ9DTAwQv1LMvORzFwJPAScV8pv7GJZqWcMG6m3/tgy/SQwFVjN07+bGwxT/6mW50+VZaUJybCRJp5lwEvK9Jt62A9p3Bg20sTzJeA9EXE5sHWvOyONB+/6LEmqziMbSVJ1ho0kqTrDRpJUnWEjSarOsJEkVWfYSJKqM2wkSdX9fxk2yLoMhSl2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "_ = plt.hist(num_words, bins = 100)\n",
    "plt.title('Distribution of num of words in one sentence')\n",
    "plt.xlabel('num')\n",
    "plt.ylabel('Counts')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:13:42.257760Z",
     "start_time": "2019-04-01T14:13:31.472210Z"
    }
   },
   "outputs": [],
   "source": [
    "word_count = {}\n",
    "def count_words(x):\n",
    "    for sen in x:\n",
    "        for word in sen:\n",
    "            word_count[word] = word_count.get(word, 0) + 1\n",
    "_ = df_train.Processed_Reviews.apply(count_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:30:08.472728Z",
     "start_time": "2019-04-01T14:30:07.930425Z"
    }
   },
   "outputs": [],
   "source": [
    "df_word_count = pd.DataFrame(list(word_count.items()), columns=['word', 'count'])\n",
    "df_word_count.sort_values(by='count', ascending=False, inplace=True)\n",
    "df_word_count.set_index('word', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:30:12.040718Z",
     "start_time": "2019-04-01T14:30:11.854195Z"
    }
   },
   "outputs": [],
   "source": [
    "vocab_size = 40000\n",
    "df_word_count = df_word_count[1:vocab_size-10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:31:35.753480Z",
     "start_time": "2019-04-01T14:31:35.290294Z"
    }
   },
   "outputs": [],
   "source": [
    "word2num_series = df_word_count.reset_index().reset_index().set_index('word')['index'] + 10 # reverse 10 pos for padding, [UNK], [CLS]..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:31:50.878460Z",
     "start_time": "2019-04-01T14:31:50.731061Z"
    }
   },
   "outputs": [],
   "source": [
    "def word_to_num(x):\n",
    "    res = []\n",
    "    for sen in x:\n",
    "        sen_words = []\n",
    "        for word in sen:\n",
    "            if word in word2num_series:\n",
    "                sen_words.append(word2num_series[word])\n",
    "        if len(sen_words) > 0:\n",
    "            res.append(np.array(sen_words, dtype=np.uint16))\n",
    "    return np.array(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:31:53.755596Z",
     "start_time": "2019-04-01T14:31:53.611745Z"
    }
   },
   "outputs": [],
   "source": [
    "def create_data(df):\n",
    "    X = df.Processed_Reviews.apply(word_to_num)\n",
    "    y = df.label\n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:36:55.588399Z",
     "start_time": "2019-04-01T14:31:54.749817Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train, y_train = create_data(df_train)\n",
    "X_val, y_val = create_data(df_validate)\n",
    "X_test, y_test = create_data(df_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:37:22.525730Z",
     "start_time": "2019-04-01T14:36:55.592410Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train.to_hdf(data_pth + 'X_train.h5', key='s', model='w')\n",
    "y_train.to_hdf(data_pth + 'y_train.h5', key='s', model='w')\n",
    "\n",
    "X_val.to_hdf(data_pth + 'X_val.h5', key='s', model='w')\n",
    "y_val.to_hdf(data_pth + 'y_val.h5', key='s', model='w')\n",
    "\n",
    "X_test.to_hdf(data_pth + 'X_test.h5', key='s', model='w')\n",
    "y_test.to_hdf(data_pth + 'y_test.h5', key='s', model='w')\n",
    "word2num_series.to_hdf(data_pth + 'word2num_series.h5', key='s', model='w')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-01T14:37:31.181779Z",
     "start_time": "2019-04-01T14:37:22.528699Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train = pd.read_hdf(data_pth + 'X_train.h5', key='s')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}