X00. Recap & Sandbox Challenge.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# X00: Recap & Sandbox Challenge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the datasets below:\n",
    "\n",
    "* Import the data into pandas using read_csv\n",
    "\n",
    "* Clean the data1 dataframe - there will be a lot NaN data, columns we don't need and unclear codes! (<b>Hint:</b> <a href = \"http://nbviewer.jupyter.org/github/tommo565/Python-Training/blob/master/D05.%20Dataframes%20Merging%20%26%20Concatanating.ipynb\">Lesson D05</a> deals with missing data.)\n",
    "\n",
    "* Merge the timezones and airport names from data2 on to data 1 using the 3 letter airport code. You should remove any aiports with NaN values first. (<b>Hint:</b> <a href = \"http://nbviewer.jupyter.org/github/tommo565/Python-Training/blob/master/D05.%20Dataframes%20Merging%20%26%20Concatanating.ipynb\">Lesson D05</a> deals with merging data.)\n",
    "\n",
    "* Convert the Date/Time fields from a string into a DateTime value (<b>Hint:</b> <a href = \"http://nbviewer.jupyter.org/github/tommo565/Python-Training/blob/master/D07.%20Useful%20Things%20in%20Pandas.ipynb\">Lesson D07</a> deals with Datetime formats but you've not had a chance to practice them yet. You will also need a Try/Except Loop from <a href = \"http://nbviewer.jupyter.org/github/tommo565/Python-Training/blob/master/B07.%20Exceptions%2C%20Try%20%26%20Except.ipynb\">Lesson B07</a>)\n",
    "\n",
    "* Where possible create UTC datetime columns - these need to take account of the Timezone variable from data2 alongside the local time.\n",
    "\n",
    "* Create a numeric variable upon which to perform aggregation etc. (<b>Hint:</b> This is as simple as data['column'] = 1 )\n",
    "\n",
    "* Aggregate the flights by hour and by departure airport (<b>Hint:</b> <a href = \"http://nbviewer.jupyter.org/github/tommo565/Python-Training/blob/master/D06.%20Summary%20Statistics%20%26%20GroupBy.ipynb\">Lesson D06</a> deals with Aggregating data.)\n",
    "\n",
    "* Create some charts (or maybe a heatmap!) of the aggregated data. (<b>Hint:</b> <a href = \"http://nbviewer.jupyter.org/github/tommo565/Python-Training/tree/master/\">The Visualisation Lessons</a> are a good place to start for this!)\n",
    "\n",
    "\n",
    "Note that you can also use the <a href = \"https://github.com/Tommo565/Python-Training/blob/master/Training%20Cookbook.py\">Training Cookbook</a> for reference!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# import pandas as pd\n",
    "\n",
    "data1 = 'https://s3-eu-west-1.amazonaws.com/imcreate/Flight+Data.csv'                     # 1 day of flight data for all UK airport\n",
    "data2 = 'https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat' # Airport Codes data\n",
    "data2_cols = ['Aiport ID','Name','City','Country','IATA/FAA Code','ICAO','Latitude',\n",
    "              'Longitude','Altitude','Timezone','DST','TZ database time zone']            # Columns for the data2 dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\ITS Admin\\Anaconda3\\lib\\site-packages\\IPython\\core\\interactiveshell.py:2723: DtypeWarning: Columns (0,1,2,4,5,6,7,8,9,11,12,14) have mixed types. Specify dtype option on import or set low_memory=False.\n",
      "  interactivity=interactivity, compiler=compiler, result=result)\n"
     ]
    }
   ],
   "source": [
    "df_data = pd.read_csv(data1)                                  # Importing data1\n",
    "df_codes = pd.read_csv(data2,names=data2_cols,index_col=0)    # Importing data2"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}