PUI-HW3-Part 3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Citibike Usage Seasonal Dynamic\n",
    "#### Introduction:\n",
    "Citibike data shows that there is a relatively low bike useage during winter time. It is presumed that many subscribers follow a pattern in trip duration, such as grocery shopping or commuting. However, it is not clear that wether there is a seasonal dynamic in such usage pattern. In this research, we look into citibike data generated by subscribers in summer months (July and August, 2015) and winter months (January and Febuary, 2015), in order to compare the seasonal difference in trip duration generated by subscribers. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Hypothesis Testing:\n",
    "In this research, we define the null hypothesis as: There is no decrease in the average trip duration of subscribers in summer months as compared to the winter months. Significance level  $\\alpha=0.05$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: pylab import has clobbered these variables: ['summer']\n",
      "`%matplotlib` prevents importing * from pylab and numpy\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x152e8a150>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Load relevant libraries.\n",
    "import pylab as pl\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "%pylab inline\n",
    "\n",
    "import os\n",
    "\n",
    "import json\n",
    "s = json.load( open(os.getenv ('PUI2015')+\"/ylai_matplotlibrc.json\") )\n",
    "pl.rcParams.update(s)\n",
    "\n",
    "fig=pl.figure(figsize(15,15))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Create a data frame named as 'summer' including data from July and August 2015.\n",
    "jul = pd.read_csv(\"201507-citibike-tripdata.csv\")\n",
    "aug = pd.read_csv(\"201508-citibike-tripdata.csv\")\n",
    "summer = pd.concat([jul, aug], axis = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2264720"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(summer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tripduration</th>\n",
       "      <th>starttime</th>\n",
       "      <th>stoptime</th>\n",
       "      <th>start station id</th>\n",
       "      <th>start station name</th>\n",
       "      <th>start station latitude</th>\n",
       "      <th>start station longitude</th>\n",
       "      <th>end station id</th>\n",
       "      <th>end station name</th>\n",
       "      <th>end station latitude</th>\n",
       "      <th>end station longitude</th>\n",
       "      <th>bikeid</th>\n",
       "      <th>usertype</th>\n",
       "      <th>birth year</th>\n",
       "      <th>gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>732</td>\n",
       "      <td>7/1/2015 00:00:03</td>\n",
       "      <td>7/1/2015 00:12:16</td>\n",
       "      <td>489</td>\n",
       "      <td>10 Ave &amp; W 28 St</td>\n",
       "      <td>40.750664</td>\n",
       "      <td>-74.001768</td>\n",
       "      <td>368</td>\n",
       "      <td>Carmine St &amp; 6 Ave</td>\n",
       "      <td>40.730386</td>\n",
       "      <td>-74.002150</td>\n",
       "      <td>18669</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1970</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>322</td>\n",
       "      <td>7/1/2015 00:00:06</td>\n",
       "      <td>7/1/2015 00:05:29</td>\n",
       "      <td>304</td>\n",
       "      <td>Broadway &amp; Battery Pl</td>\n",
       "      <td>40.704633</td>\n",
       "      <td>-74.013617</td>\n",
       "      <td>3002</td>\n",
       "      <td>South End Ave &amp; Liberty St</td>\n",
       "      <td>40.711512</td>\n",
       "      <td>-74.015756</td>\n",
       "      <td>14618</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1984</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>790</td>\n",
       "      <td>7/1/2015 00:00:17</td>\n",
       "      <td>7/1/2015 00:13:28</td>\n",
       "      <td>447</td>\n",
       "      <td>8 Ave &amp; W 52 St</td>\n",
       "      <td>40.763707</td>\n",
       "      <td>-73.985162</td>\n",
       "      <td>358</td>\n",
       "      <td>Christopher St &amp; Greenwich St</td>\n",
       "      <td>40.732916</td>\n",
       "      <td>-74.007114</td>\n",
       "      <td>18801</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1992</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1228</td>\n",
       "      <td>7/1/2015 00:00:23</td>\n",
       "      <td>7/1/2015 00:20:51</td>\n",
       "      <td>490</td>\n",
       "      <td>8 Ave &amp; W 33 St</td>\n",
       "      <td>40.751551</td>\n",
       "      <td>-73.993934</td>\n",
       "      <td>250</td>\n",
       "      <td>Lafayette St &amp; Jersey St</td>\n",
       "      <td>40.724561</td>\n",
       "      <td>-73.995653</td>\n",
       "      <td>19137</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1990</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1383</td>\n",
       "      <td>7/1/2015 00:00:44</td>\n",
       "      <td>7/1/2015 00:23:48</td>\n",
       "      <td>327</td>\n",
       "      <td>Vesey Pl &amp; River Terrace</td>\n",
       "      <td>40.715338</td>\n",
       "      <td>-74.016584</td>\n",
       "      <td>72</td>\n",
       "      <td>W 52 St &amp; 11 Ave</td>\n",
       "      <td>40.767272</td>\n",
       "      <td>-73.993929</td>\n",
       "      <td>15808</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1988</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   tripduration          starttime           stoptime  start station id  \\\n",
       "0           732  7/1/2015 00:00:03  7/1/2015 00:12:16               489   \n",
       "1           322  7/1/2015 00:00:06  7/1/2015 00:05:29               304   \n",
       "2           790  7/1/2015 00:00:17  7/1/2015 00:13:28               447   \n",
       "3          1228  7/1/2015 00:00:23  7/1/2015 00:20:51               490   \n",
       "4          1383  7/1/2015 00:00:44  7/1/2015 00:23:48               327   \n",
       "\n",
       "         start station name  start station latitude  start station longitude  \\\n",
       "0          10 Ave & W 28 St               40.750664               -74.001768   \n",
       "1     Broadway & Battery Pl               40.704633               -74.013617   \n",
       "2           8 Ave & W 52 St               40.763707               -73.985162   \n",
       "3           8 Ave & W 33 St               40.751551               -73.993934   \n",
       "4  Vesey Pl & River Terrace               40.715338               -74.016584   \n",
       "\n",
       "   end station id               end station name  end station latitude  \\\n",
       "0             368             Carmine St & 6 Ave             40.730386   \n",
       "1            3002     South End Ave & Liberty St             40.711512   \n",
       "2             358  Christopher St & Greenwich St             40.732916   \n",
       "3             250       Lafayette St & Jersey St             40.724561   \n",
       "4              72               W 52 St & 11 Ave             40.767272   \n",
       "\n",
       "   end station longitude  bikeid    usertype  birth year  gender  \n",
       "0             -74.002150   18669  Subscriber        1970       1  \n",
       "1             -74.015756   14618  Subscriber        1984       1  \n",
       "2             -74.007114   18801  Subscriber        1992       1  \n",
       "3             -73.995653   19137  Subscriber        1990       1  \n",
       "4             -73.993929   15808  Subscriber        1988       1  "
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "summer[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Subset a dataframe with subscribers only.\n",
    "summer_sub = summer[summer['usertype'] == 'Subscriber']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tripduration</th>\n",
       "      <th>starttime</th>\n",
       "      <th>stoptime</th>\n",
       "      <th>start station id</th>\n",
       "      <th>start station name</th>\n",
       "      <th>start station latitude</th>\n",
       "      <th>start station longitude</th>\n",
       "      <th>end station id</th>\n",
       "      <th>end station name</th>\n",
       "      <th>end station latitude</th>\n",
       "      <th>end station longitude</th>\n",
       "      <th>bikeid</th>\n",
       "      <th>usertype</th>\n",
       "      <th>birth year</th>\n",
       "      <th>gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>732</td>\n",
       "      <td>7/1/2015 00:00:03</td>\n",
       "      <td>7/1/2015 00:12:16</td>\n",
       "      <td>489</td>\n",
       "      <td>10 Ave &amp; W 28 St</td>\n",
       "      <td>40.750664</td>\n",
       "      <td>-74.001768</td>\n",
       "      <td>368</td>\n",
       "      <td>Carmine St &amp; 6 Ave</td>\n",
       "      <td>40.730386</td>\n",
       "      <td>-74.002150</td>\n",
       "      <td>18669</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1970</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>322</td>\n",
       "      <td>7/1/2015 00:00:06</td>\n",
       "      <td>7/1/2015 00:05:29</td>\n",
       "      <td>304</td>\n",
       "      <td>Broadway &amp; Battery Pl</td>\n",
       "      <td>40.704633</td>\n",
       "      <td>-74.013617</td>\n",
       "      <td>3002</td>\n",
       "      <td>South End Ave &amp; Liberty St</td>\n",
       "      <td>40.711512</td>\n",
       "      <td>-74.015756</td>\n",
       "      <td>14618</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1984</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>790</td>\n",
       "      <td>7/1/2015 00:00:17</td>\n",
       "      <td>7/1/2015 00:13:28</td>\n",
       "      <td>447</td>\n",
       "      <td>8 Ave &amp; W 52 St</td>\n",
       "      <td>40.763707</td>\n",
       "      <td>-73.985162</td>\n",
       "      <td>358</td>\n",
       "      <td>Christopher St &amp; Greenwich St</td>\n",
       "      <td>40.732916</td>\n",
       "      <td>-74.007114</td>\n",
       "      <td>18801</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1992</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1228</td>\n",
       "      <td>7/1/2015 00:00:23</td>\n",
       "      <td>7/1/2015 00:20:51</td>\n",
       "      <td>490</td>\n",
       "      <td>8 Ave &amp; W 33 St</td>\n",
       "      <td>40.751551</td>\n",
       "      <td>-73.993934</td>\n",
       "      <td>250</td>\n",
       "      <td>Lafayette St &amp; Jersey St</td>\n",
       "      <td>40.724561</td>\n",
       "      <td>-73.995653</td>\n",
       "      <td>19137</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1990</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1383</td>\n",
       "      <td>7/1/2015 00:00:44</td>\n",
       "      <td>7/1/2015 00:23:48</td>\n",
       "      <td>327</td>\n",
       "      <td>Vesey Pl &amp; River Terrace</td>\n",
       "      <td>40.715338</td>\n",
       "      <td>-74.016584</td>\n",
       "      <td>72</td>\n",
       "      <td>W 52 St &amp; 11 Ave</td>\n",
       "      <td>40.767272</td>\n",
       "      <td>-73.993929</td>\n",
       "      <td>15808</td>\n",
       "      <td>Subscriber</td>\n",
       "      <td>1988</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   tripduration          starttime           stoptime  start station id  \\\n",
       "0           732  7/1/2015 00:00:03  7/1/2015 00:12:16               489   \n",
       "1           322  7/1/2015 00:00:06  7/1/2015 00:05:29               304   \n",
       "2           790  7/1/2015 00:00:17  7/1/2015 00:13:28               447   \n",
       "3          1228  7/1/2015 00:00:23  7/1/2015 00:20:51               490   \n",
       "4          1383  7/1/2015 00:00:44  7/1/2015 00:23:48               327   \n",
       "\n",
       "         start station name  start station latitude  start station longitude  \\\n",
       "0          10 Ave & W 28 St               40.750664               -74.001768   \n",
       "1     Broadway & Battery Pl               40.704633               -74.013617   \n",
       "2           8 Ave & W 52 St               40.763707               -73.985162   \n",
       "3           8 Ave & W 33 St               40.751551               -73.993934   \n",
       "4  Vesey Pl & River Terrace               40.715338               -74.016584   \n",
       "\n",
       "   end station id               end station name  end station latitude  \\\n",
       "0             368             Carmine St & 6 Ave             40.730386   \n",
       "1            3002     South End Ave & Liberty St             40.711512   \n",
       "2             358  Christopher St & Greenwich St             40.732916   \n",
       "3             250       Lafayette St & Jersey St             40.724561   \n",
       "4              72               W 52 St & 11 Ave             40.767272   \n",
       "\n",
       "   end station longitude  bikeid    usertype  birth year  gender  \n",
       "0             -74.002150   18669  Subscriber        1970       1  \n",
       "1             -74.015756   14618  Subscriber        1984       1  \n",
       "2             -74.007114   18801  Subscriber        1992       1  \n",
       "3             -73.995653   19137  Subscriber        1990       1  \n",
       "4             -73.993929   15808  Subscriber        1988       1  "
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "summer_sub[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1863362"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(summer_sub)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    1863362.000000\n",
       "mean         811.243927\n",
       "std         5576.340963\n",
       "min           60.000000\n",
       "25%          382.000000\n",
       "50%          602.000000\n",
       "75%          960.000000\n",
       "max      3611666.000000\n",
       "Name: tripduration, dtype: float64"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "summer_sub['tripduration'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "summer_long = summer_sub[summer_sub['tripduration'] >3000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count      11455.000000\n",
       "mean       13234.166041\n",
       "std        69745.946689\n",
       "min         3001.000000\n",
       "25%         3320.000000\n",
       "50%         4061.000000\n",
       "75%         7121.000000\n",
       "max      3611666.000000\n",
       "Name: tripduration, dtype: float64"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "summer_long['tripduration'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fig=pl.figure(figsize(15,15))\n",
    "\n",
    "counts_sum_long = summer_long.count()\n",
    "norm_sum_long = 1\n",
    "\n",
    "\n",
    "\n",
    "norm_w= counts_w.sum()\n",
    "error_w=np.sqrt(counts_w)\n",
    "((counts_w)/norm_w).plot(kind=\"bar\",color='IndianRed',  yerr=[\n",
    "            ((error_w)/norm_w,(error_w)/norm_w)], label='women bikers')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}