ESCI451_Module_3B.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Python for Earth Scientists\n",
    "\n",
    "These notebooks have been developed by Calum Chamberlain, Finnigan Illsley-Kemp and John Townend at [Victoria University of Wellington-Te Herenga Waka](https://www.wgtn.ac.nz) for use by Earth Science graduate students. \n",
    "\n",
    "The notebooks cover material that we think will be of particular benefit to those students with little or no previous experience of computer-based data analysis. We presume very little background in command-line or code-based computing, and have compiled this material with an emphasis on general tasks that a grad student might encounter on a daily basis. \n",
    "\n",
    "In 2021, this material will be delivered at the start of Trimester 1 in conjunction with [ESCI451 Active Earth](https://www.wgtn.ac.nz/courses/esci/451/2021/offering?crn=32176). Space and pandemic alert levels permitting, interested students not enrolled in ESCI451 are encouraged to come along too but please contact Calum, Finn, or John first.\n",
    "\n",
    "| Notebook | Contents | Data |\n",
    "| --- | --- | --- |\n",
    "| [1A](ESCI451_Module_1A.ipynb) | Introduction to programming, Python, and Jupyter notebooks | - |\n",
    "| [1B](ESCI451_Module_1B.ipynb) | Basic data types and variables, getting data, and plotting with Matplotlib | Geodetic positions |\n",
    "| [2A](ESCI451_Module_2A.ipynb) | More complex plotting, introduction to Numpy | Geodetic positions; DFDP-2B temperatures |\n",
    "| [2B](ESCI451_Module_2B.ipynb) | Using Pandas to load, peruse and plot data | Earthquake catalogue  |\n",
    "| [3A](ESCI451_Module_3A.ipynb) | Working with Pandas dataframes | Geochemical data set; earthquake catalogue |\n",
    "| **[3B](ESCI451_Module_3B.ipynb)** | **Simple time series analysis using Pandas** | **Historical temperature records** |\n",
    "| [4A](ESCI451_Module_4A.ipynb) | Making maps with Cartopy | Earthquake catalogue |\n",
    "| [4B](ESCI451_Module_4B.ipynb) | Working with gridded data | DEMs and Ashfall data |\n",
    "\n",
    "The content may change in response to students' questions or current events. Each of the four modules has been designed to take about three hours, with a short break between each of the two parts.\n",
    "\n",
    "Remember to run the first cell to set up plotting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib widget"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# This notebook\n",
    "\n",
    "1. Working with time series using dataframes\n",
    "   - Loading and wrangling data\n",
    "   - Plotting subsets of time series data using Matplotlib, Pandas, and hybrid approaches\n",
    "   - Merging dataframes\n",
    "   - Illustrating periodicity\n",
    "   - Rolling averages\n",
    "   - Basic Fourier analysis using SciPy\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Working with time series using dataframes\n",
    "\n",
    "## Loading and wrangling data\n",
    "\n",
    "In this module we'll look at analysing time series data using Pandas. We'll work with temperature data obtained from the NIWA CliFlo database over several decades in the twentieth century. Our emphasis is on widely useable techniques for working with time series data, rather than advanced techniques used in specific fields.\n",
    "\n",
    "First, let's load some of the standard tools."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from dateutil import parser, rrule\n",
    "from datetime import datetime as dt\n",
    "import requests\n",
    "import time\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll load the CliFlo data from .csv (comma-separated variable) files downloaded previously. There are four files to work with:\n",
    "\n",
    "- 1939-1989-surface-temperature-CliFlo.csv - daily minimum and maximum surface temperatures measured at Kelburn between 1939 and 1989\n",
    "- 1939-1989-10cm-temperature-CliFlo.csv - daily temperature measurements at 10 cm depth, measured at Kelburn between 1939 and 1989\n",
    "- 1939-1989-30cm-temperature-CliFlo.csv - \" \" \" 30 cm depth \" \" \"\n",
    "- 1939-1989-100cm-temperature-CliFlo.csv - \" \" \" 100 cm depth \" \" \"\n",
    "\n",
    "You may recall from other classes (e.g. ESCI203!) that surface temperature variations of different frequencies propagate to different depths. By looking at amplitude and phase characteristics of the various data sets we may be able to observe this phenomena.\n",
    "\n",
    "The first column of each of these files contains the measurement date in the format '%Y%m%d:%H%M' (e.g. \"19390105:0900\"), and we can customise the Pandas `read_csv` command by telling it to treat the Date column in the .csv file as a list of dates with that specific format. We can also specify that the Date information be used as the index of each dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "parser = lambda date: dt.strptime(date, '%Y%m%d:%H%M')\n",
    "T0 = pd.read_csv(\n",
    "    'data/1939-1989-surface-temperature-CliFlo.csv',\n",
    "    parse_dates=['Date'], date_parser=parser, index_col='Date')\n",
    "T10 = pd.read_csv(\n",
    "    'data/1939-1989-10cm-temperature-CliFlo.csv',\n",
    "    parse_dates=['Date'], date_parser=parser, index_col='Date')\n",
    "T30 = pd.read_csv(\n",
    "    'data/1939-1989-30cm-temperature-CliFlo.csv',\n",
    "    parse_dates=['Date'], date_parser=parser, index_col='Date')\n",
    "T100 = pd.read_csv(\n",
    "    'data/1939-1989-100cm-temperature-CliFlo.csv',\n",
    "    parse_dates=['Date'], date_parser=parser, index_col='Date')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each of the dataframes we've created has the same format, with the exception that the surface temperatures are expressed as minimum and maximum values. Remember, we can inspect or verify different aspects of a dataframe using commands like `df.columns` (to see a list of column names) and `df.describe()` (to get summary statistics), or just by entering `df` to see a summary of the first few rows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Tmax</th>\n",
       "      <th>Tmin</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1939-12-31 09:00:00</th>\n",
       "      <td>21.3</td>\n",
       "      <td>14.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940-01-01 09:00:00</th>\n",
       "      <td>22.8</td>\n",
       "      <td>14.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940-01-02 09:00:00</th>\n",
       "      <td>23.1</td>\n",
       "      <td>11.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940-01-03 09:00:00</th>\n",
       "      <td>15.3</td>\n",
       "      <td>7.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940-01-04 09:00:00</th>\n",
       "      <td>17.9</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-26 08:00:00</th>\n",
       "      <td>18.8</td>\n",
       "      <td>13.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-27 08:00:00</th>\n",
       "      <td>19.1</td>\n",
       "      <td>14.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-28 08:00:00</th>\n",
       "      <td>19.2</td>\n",
       "      <td>14.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-29 08:00:00</th>\n",
       "      <td>18.7</td>\n",
       "      <td>16.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-30 08:00:00</th>\n",
       "      <td>18.2</td>\n",
       "      <td>13.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>18263 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     Tmax  Tmin\n",
       "Date                           \n",
       "1939-12-31 09:00:00  21.3  14.3\n",
       "1940-01-01 09:00:00  22.8  14.6\n",
       "1940-01-02 09:00:00  23.1  11.3\n",
       "1940-01-03 09:00:00  15.3   7.9\n",
       "1940-01-04 09:00:00  17.9   9.0\n",
       "...                   ...   ...\n",
       "1989-12-26 08:00:00  18.8  13.3\n",
       "1989-12-27 08:00:00  19.1  14.6\n",
       "1989-12-28 08:00:00  19.2  14.9\n",
       "1989-12-29 08:00:00  18.7  16.1\n",
       "1989-12-30 08:00:00  18.2  13.0\n",
       "\n",
       "[18263 rows x 2 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before going any further, we're going to spend a bit of time manipulating each of the four dataframes we've created to put it into the most convenient format possible. This type of process is often referred to as \"data wrangling\" by cool Python types.\n",
    "\n",
    "In the case of the surface measurements, T0, we will calculate a rough mean daily temperature by averaging the maximum and minimum values. (Climate scientists would do this much more carefully and would likely use hourly records for this.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "T0['Temperature'] = T0.apply(lambda df: (df['Tmax'] + df['Tmin']) / 2, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That may need some explanation. What we've done is to create a new column in the T0 dataframe with the name `Temperature`, and we've done this using a function applied to the `Tmax` and `Tmin` columns. The function is specified as \"lambda df: (df['Tmax']+df['Tmin'])/2\" and then applied via \"T0.apply(...)\". \"axis=1\" in the function definition tells Python to operate on columns rather than rows.\n",
    "\n",
    "Let's check what the form of T0 is now:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Tmax</th>\n",
       "      <th>Tmin</th>\n",
       "      <th>Temperature</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1939-12-31 09:00:00</th>\n",
       "      <td>21.3</td>\n",
       "      <td>14.3</td>\n",
       "      <td>17.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940-01-01 09:00:00</th>\n",
       "      <td>22.8</td>\n",
       "      <td>14.6</td>\n",
       "      <td>18.70</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940-01-02 09:00:00</th>\n",
       "      <td>23.1</td>\n",
       "      <td>11.3</td>\n",
       "      <td>17.20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940-01-03 09:00:00</th>\n",
       "      <td>15.3</td>\n",
       "      <td>7.9</td>\n",
       "      <td>11.60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940-01-04 09:00:00</th>\n",
       "      <td>17.9</td>\n",
       "      <td>9.0</td>\n",
       "      <td>13.45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-26 08:00:00</th>\n",
       "      <td>18.8</td>\n",
       "      <td>13.3</td>\n",
       "      <td>16.05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-27 08:00:00</th>\n",
       "      <td>19.1</td>\n",
       "      <td>14.6</td>\n",
       "      <td>16.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-28 08:00:00</th>\n",
       "      <td>19.2</td>\n",
       "      <td>14.9</td>\n",
       "      <td>17.05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-29 08:00:00</th>\n",
       "      <td>18.7</td>\n",
       "      <td>16.1</td>\n",
       "      <td>17.40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1989-12-30 08:00:00</th>\n",
       "      <td>18.2</td>\n",
       "      <td>13.0</td>\n",
       "      <td>15.60</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>18263 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     Tmax  Tmin  Temperature\n",
       "Date                                        \n",
       "1939-12-31 09:00:00  21.3  14.3        17.80\n",
       "1940-01-01 09:00:00  22.8  14.6        18.70\n",
       "1940-01-02 09:00:00  23.1  11.3        17.20\n",
       "1940-01-03 09:00:00  15.3   7.9        11.60\n",
       "1940-01-04 09:00:00  17.9   9.0        13.45\n",
       "...                   ...   ...          ...\n",
       "1989-12-26 08:00:00  18.8  13.3        16.05\n",
       "1989-12-27 08:00:00  19.1  14.6        16.85\n",
       "1989-12-28 08:00:00  19.2  14.9        17.05\n",
       "1989-12-29 08:00:00  18.7  16.1        17.40\n",
       "1989-12-30 08:00:00  18.2  13.0        15.60\n",
       "\n",
       "[18263 rows x 3 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting subsets of time series data using Matplotlib, Pandas, and hybrid approaches\n",
    "\n",
    "Notice that because of how we read the data in originally, we don't have an integer index colum (0, 1, 2, 3...) but instead have the Date values. This will necessitate a few changes in how we refer to the dates but we can address these as they come up. \n",
    "\n",
    "In any case, now's the time to get an idea of what the data look like by just plotting each dataframe. Right away, this requires us to remember that the dataframe index is the Date information, which means that we can't refer to it using a column name. We have three options (at least!) here. \n",
    "1. Either we can use \"pure\" Matplotlib syntax and specify the x-data using the syntax `df.index` instead of `df['Date']`\n",
    "2. Or we can use the in-built plotting options for a Pandas dataframe and just omit any mention of the x-data\n",
    "3. Or we can use a hybrid approach, which enables us to use some of the more sophisticated layout and annotation options Matplotlib provides without having to specify the x-data at all.\n",
    "\n",
    "Let's try all three to see what happens and, for the sake of reinforcing things we've looked at previously, let's use loops wherever possible. We'll also use this opportunity to experiment a little bit with different layouts.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9f135852ce534b649e4121b2ef5b718a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/calumch/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:1402: FutureWarning: Support for multi-dimensional indexing (e.g. `obj[:, None]`) is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.\n",
      "  x[:, None]\n",
      "/home/calumch/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/axes/_base.py:276: FutureWarning: Support for multi-dimensional indexing (e.g. `obj[:, None]`) is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.\n",
      "  x = x[:, np.newaxis]\n",
      "/home/calumch/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/axes/_base.py:278: FutureWarning: Support for multi-dimensional indexing (e.g. `obj[:, None]`) is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.\n",
      "  y = y[:, np.newaxis]\n"
     ]
    }
   ],
   "source": [
    "#1. Pure Matplotlib\n",
    "fig, ax = plt.subplots()\n",
    "for component, colour, label in zip([T0, T10, T30, T100], [\"black\", \"red\", \"green\", \"blue\"], ['Surface','10 cm','30 cm','100 cm']):\n",
    "    ax.plot(component.index,component['Temperature'],color=colour, label=label)\n",
    "ax.set_xlabel('Date')\n",
    "ax.set_ylabel('Temperature (C)')\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4d5e3f600ad44fd89a690f3735520648",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8ffb792326ab4a529535e332df2334e8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b503b9334f074aadab824c67fd687d41",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "11f89e2703584d17a051b0699eb28500",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#2. Pure Pandas plotting\n",
    "for component, colour, label in zip([T0,T10,T30,T100], [\"black\", \"red\", \"green\", \"blue\"], ['Surface','10 cm','30 cm','100 cm']):\n",
    "    ax=component.plot(y=['Temperature'],color=colour)\n",
    "    ax.set_ylabel('Temperature (C)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "17a1afdef52c4c299ac446e1b2a8ce6d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#3. Hybrid\n",
    "fig, axes = plt.subplots(4, 1, figsize=(15,10), sharex=True)\n",
    "for i, component, colour, label in zip([0, 1, 2, 3], [T0, T10, T30, T100], [\"black\", \"red\", \"green\", \"blue\"], ['Surface','10 cm','30 cm','100 cm']):\n",
    "    component['Temperature'].plot(ax=axes[i], label=label,color=colour)\n",
    "    axes[i].legend()\n",
    "    axes[i].set_ylabel('Temperature (C)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Groovy!\n",
    "\n",
    "But what we can see from this is the surface records start much further back in time than the others. We could adjust for this in the plot by simply setting the x-axis limits to the date ranges spanned by the subsurface data, but perhaps a better way of handling this is to create a _single_ combined dataframe with all the different temperature measurements for the data range spanned by all the datasets. Clear as mud?\n",
    "\n",
    "## Merging dataframes\n",
    "\n",
    "This gives us the opportunity to talk about different ways of joining dataframes sharing a common field (known as the \"join key\" and in this case the Date information in each dataframe). There's a really helpful explanation of some of the multitudinous options [here](https://datacarpentry.org/python-ecology-lesson/05-merging-data/index.html) but we'll look at just a couple of points. \n",
    "\n",
    "- What we could do here is combine some or all of the the columns from two dataframes at a time based on a common join key, and then output only those rows that have matching join key values in both of the input dataframes. This is called an \"inner join\".\n",
    "\n",
    "- Alternatively, we could output all the rows of one of the input dataframes augmented with data from the other when there's a matching join key value and augmented with nulls otherwise. This technique yields a \"left join\" or a \"right join\", depending on the order of input dataframes.\n",
    "\n",
    "- There's also something called an \"outer join\" which is a bit like doing left and right joins at the same time in the sense that it retains all the rows from both input dataframes and inserts null values as needed for rows without a matching join key.\n",
    "\n",
    "For our purposes, and given that we've seen already that the surface dataset starts much earlier than the others, it seems most appropriate to do an inner join.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "merged1 = pd.merge(left=T0, right=T10, how='inner', left_on='Date', right_on='Date')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can you see how we'd have got the same answer in this case if we had specified \"how='right'\"? Notice also that the temperature columns from T0 and T10 have been relabled as \"Temperature_x\" and \"Temperature_y\" to avoid two columns with the same name. It's probably a good idea to rename them now to avoid getting confused about which temperature is which later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "merged1.rename(columns={'Temperature_x':'T0','Temperature_y':'T10'}, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can do this a couple more times using the remaining dataframes. Since each dataframe's index is the Date information itself, we can equally well use the following syntax (i.e. specifying the join key as the index):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "merged2 = pd.merge(left=merged1, right=T30, how='inner', left_index=True, right_index=True)\n",
    "merged2.rename(columns={'Temperature':'T30'}, inplace=True)\n",
    "temperatures = pd.merge(left=merged2, right=T100, how='inner', left_index=True, right_index=True)\n",
    "temperatures.rename(columns={'Temperature':'T100'}, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "OK. Now we're ready to actually analyse the data in a bit more detail. But let's remind ourselves of some of the ways we can slice the data to look at particular details. Of no particular significance:\n",
    "\n",
    "1. The temperature at various depths on your birthday?\n",
    "2. The maximum _surface_ temperature in the whole dataset?\n",
    "3. The maximum and minimum temperatures at a depth of 100 cm in a particular month or year?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                     Tmax  Tmin    T0   T10   T30  T100\n",
      "Date                                                   \n",
      "1972-12-12 09:00:00  19.7  12.9  16.3  16.1  17.3  15.6\n",
      "24.9\n",
      "19.2 9.6\n"
     ]
    }
   ],
   "source": [
    "print(temperatures.loc['1972-12-12'])\n",
    "print(temperatures.loc['1980']['Tmax'].max())\n",
    "print(temperatures.loc['1988']['T100'].max(), temperatures.loc['1988']['T100'].min())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can do some simple plotting of various subsets too:\n",
    "\n",
    "1. The surface temperature in a particular month or year?\n",
    "2. The subsurface temperature records in three successive months?\n",
    "3. The subsurface temperatures in the same month of several successive years?\n",
    "4. Shallow (T0) vs deep (T100) temperatures in a particular year?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ed490d283b4c4bd3ab1c01e5aeb003ad",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "247450dd12094dc896c88a17ee40ce31",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3503f1a2b7d9421ebc36755602b149e5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "temperatures.loc['1972']['T0'].plot()\n",
    "temperatures.loc['1972-01':'1972-03'][['T10','T30','T100']].plot()\n",
    "pd.concat([temperatures.loc['1972-01'][['T10','T30','T100']],\n",
    "           temperatures.loc['1973-01'][['T10','T30','T100']],\n",
    "           temperatures.loc['1974-01'][['T10','T30','T100']]]).plot()\n",
    "temperatures.loc['1977'].plot.scatter(x='T0',y='T100',c='red')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(That third one was a bit silly.)\n",
    "\n",
    "## Illustrating periodicity\n",
    "\n",
    "We can see right away that there are strong annual cycles but there may be other features in the data that we want to look at. \n",
    "\n",
    "Let's try plotting all the data on a single year-long time axis. This is easily done using a pivot table, in which we effectively reshape the dataframe into a format in which the columns represent day of the year or month, for example, and the columns represent different years. In this first example, we are concatenating two commands together (`.pivotable` and `.plot`) and plotting each year's data against the day of the year. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1626bc5f34d641d0842063e02fc25f33",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pivot = temperatures.pivot_table(\n",
    "    'T0', index=temperatures.index.dayofyear,\n",
    "    columns=temperatures.index.year, aggfunc='mean')\n",
    "pivot.plot()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See how we are forming the pivot table's index list (i.e. row names) by extracting day-of-the-year inforamtion from the existing dates (`index=temperatures.index.dayofyear`) and forming the column names by extracting the year information (`columns=temperatures.index.year`). We're also computing the mean of the data in each cell of the pivot table, although in this case there should only be one measurement taken per day anyway. You can find more helpful information about pivot tables [here](https://jakevdp.github.io/PythonDataScienceHandbook/03.09-pivot-tables.html).\n",
    "\n",
    "Here's the same type of graph but illustrated with respect to months rather than days: (in this case we're plotting the pivot table directly, rather than outputting a new dataframe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e216d4e7895b4443b821627fe290cebb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "temperatures.pivot_table(\n",
    "    'T0', index=temperatures.index.month,\n",
    "    columns=temperatures.index.year, aggfunc='mean').plot()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For comparison, here's another way of doing something similar but without retaining the yearly curves, which have been crunched into a single 12-month average temperature record."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c5035d4506294bfa8c2935e8b45b9b02",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "temperatures['T0'].groupby(temperatures.index.month).mean().plot()\n",
    "ax.set_ylabel('Temperature (C)')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's use some of the very elegant plotting options provided by `seaborn` to doing something similar in a different way (see more [here](http://seaborn.pydata.org/examples/))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f61d89bbf0c844f6a9012059815c6aa3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot each year's time series in its own panel\n",
    "sns.set_theme(style=\"white\")\n",
    "g = sns.relplot(\n",
    "    data=temperatures,\n",
    "    x=temperatures.index.month, y=temperatures['T0'], col=temperatures.index.year, hue=temperatures.index.year,\n",
    "    kind=\"line\", palette=\"crest\", linewidth=1, zorder=5,\n",
    "    col_wrap=3, height=2, aspect=1.5, legend=False\n",
    ")\n",
    "\n",
    "# Annotate each plot with the year\n",
    "for year, ax in g.axes_dict.items():\n",
    "    ax.text(.8, .85, year, transform=ax.transAxes, fontweight=\"bold\")\n",
    "\n",
    "# Tweak the layout a bit\n",
    "g.set_titles(\"\")\n",
    "g.set_axis_labels(\"\", \"Temperature (C)\")\n",
    "g.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, there are many different ways to inspect and plot your data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rolling averages\n",
    "\n",
    "Finally, we'll briefly experiment with showing rolling averages and with calculating the lags between different temperature records.\n",
    "\n",
    "Rolling averages, as the name suggests, are averages computed in a moving window of the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0936ffc8b7de4d6db3867e5fd888f50c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax=temperatures.plot(y='T0',color='lightgray',label='Raw')\n",
    "temperatures['T0'].rolling(window=7).mean().plot(label='7-day rolling mean')\n",
    "temperatures['T0'].rolling(window=14).mean().plot(label='14-day rolling mean')\n",
    "temperatures['T0'].rolling(window=14).median().plot(label='14-day rolling median')\n",
    "ax.grid()\n",
    "ax.legend()\n",
    "ax.set_ylabel('Temperature (C)')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Fourier analysis using SciPy\n",
    "\n",
    "We won't go into much detail about how different signal processing tools, not least the workhorse that is the the Fourier Transform, can be applied to time series data in Python. However, the following code illustrates the way in which the obvious annual cyclicity in the CliFlo temperatures can be extracted and illustrated using tools from [SciPy](https://www.scipy.org/). We have defined a function that uses the Fast Fourier Transfrom to isolate the annual signal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy import signal, stats, fftpack\n",
    "#https://ipython-books.github.io/101-analyzing-the-frequency-components-of-a-signal-with-a-fast-fourier-transform/\n",
    "\n",
    "def annular_component(df,column):\n",
    "    '''\n",
    "    Analyse spectrum and return reconstruction \n",
    "    containing frequencies lower than annual\n",
    "    '''\n",
    "    T=df[column].to_numpy()\n",
    "    \n",
    "    FFT=fftpack.fft(T)\n",
    "    PSD=np.abs(FFT)**2\n",
    "    freq=fftpack.fftfreq(len(PSD),1./365)\n",
    "    i=freq>0\n",
    "    copy=FFT.copy()\n",
    "    copy[np.abs(freq)>1.01]=0\n",
    "\n",
    "    annular=np.real(fftpack.ifft(copy))\n",
    "\n",
    "    fig, ax=plt.subplots(1,2,figsize=(15,10))\n",
    "    ax[0].plot(freq[i], 10 * np.log10(PSD[i]),'o-')\n",
    "    ax[0].set_xlim(0, 5)\n",
    "    ax[0].set_xlabel('Frequency (1/year)')\n",
    "    ax[0].grid()\n",
    "    ax[0].set_ylabel('PSD (dB)')\n",
    "    \n",
    "    ax[1].plot(df.index,T,label='Raw signal',color='lightgray')\n",
    "    ax[1].plot(df.index,annular,label='Annual component',color='black',linewidth=3)\n",
    "    ax[1].legend()\n",
    "    ax[1].set_xlabel('Date')\n",
    "    ax[1].set_ylabel('Temperature (C)')\n",
    "\n",
    "    return annular"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "135a2cb6559748038f6cd6b1c1b3b8a0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/calumch/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:1402: FutureWarning: Support for multi-dimensional indexing (e.g. `obj[:, None]`) is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.\n",
      "  x[:, None]\n",
      "/home/calumch/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/axes/_base.py:276: FutureWarning: Support for multi-dimensional indexing (e.g. `obj[:, None]`) is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.\n",
      "  x = x[:, np.newaxis]\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "view limit minimum -36706.56666666667 is less than 1 and is an invalid Matplotlib date value. This often happens if you pass a non-datetime value to an axis that has datetime units",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/ipympl/backend_nbagg.py\u001b[0m in \u001b[0;36m_handle_message\u001b[0;34m(self, object, message, buffers)\u001b[0m\n\u001b[1;32m    170\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend_json\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'refresh'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    171\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 172\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmanager\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhandle_json\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    173\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    174\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0msend_json\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/backends/backend_webagg_core.py\u001b[0m in \u001b[0;36mhandle_json\u001b[0;34m(self, content)\u001b[0m\n\u001b[1;32m    437\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    438\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mhandle_json\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 439\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcanvas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhandle_event\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcontent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    440\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    441\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mrefresh_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/backends/backend_webagg_core.py\u001b[0m in \u001b[0;36mhandle_event\u001b[0;34m(self, event)\u001b[0m\n\u001b[1;32m    238\u001b[0m         handler = getattr(self, 'handle_{0}'.format(e_type),\n\u001b[1;32m    239\u001b[0m                           self.handle_unknown_event)\n\u001b[0;32m--> 240\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mhandler\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mevent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    241\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    242\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mhandle_unknown_event\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mevent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/backends/backend_webagg_core.py\u001b[0m in \u001b[0;36mhandle_draw\u001b[0;34m(self, event)\u001b[0m\n\u001b[1;32m    254\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    255\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mhandle_draw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mevent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 256\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    257\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    258\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_handle_mouse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mevent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/backends/backend_webagg_core.py\u001b[0m in \u001b[0;36mdraw\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    149\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_png_is_old\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    150\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 151\u001b[0;31m             \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    152\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    153\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmanager\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrefresh_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# Swap the frames.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py\u001b[0m in \u001b[0;36mdraw\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    386\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrenderer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_renderer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcleared\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    387\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mRendererAgg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlock\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 388\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfigure\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    389\u001b[0m             \u001b[0;31m# A GUI class may be need to update a window using this draw, so\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    390\u001b[0m             \u001b[0;31m# don't forget to call the superclass.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/artist.py\u001b[0m in \u001b[0;36mdraw_wrapper\u001b[0;34m(artist, renderer, *args, **kwargs)\u001b[0m\n\u001b[1;32m     36\u001b[0m                 \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstart_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     37\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 38\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martist\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrenderer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     39\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     40\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0martist\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_agg_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/figure.py\u001b[0m in \u001b[0;36mdraw\u001b[0;34m(self, renderer)\u001b[0m\n\u001b[1;32m   1707\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpatch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1708\u001b[0m             mimage._draw_list_compositing_images(\n\u001b[0;32m-> 1709\u001b[0;31m                 renderer, self, artists, self.suppressComposite)\n\u001b[0m\u001b[1;32m   1710\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1711\u001b[0m             \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclose_group\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'figure'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/image.py\u001b[0m in \u001b[0;36m_draw_list_compositing_images\u001b[0;34m(renderer, parent, artists, suppress_composite)\u001b[0m\n\u001b[1;32m    133\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mnot_composite\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mhas_images\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    134\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0ma\u001b[0m \u001b[0;32min\u001b[0m \u001b[0martists\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 135\u001b[0;31m             \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    136\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    137\u001b[0m         \u001b[0;31m# Composite any adjacent images together\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/artist.py\u001b[0m in \u001b[0;36mdraw_wrapper\u001b[0;34m(artist, renderer, *args, **kwargs)\u001b[0m\n\u001b[1;32m     36\u001b[0m                 \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstart_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     37\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 38\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martist\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrenderer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     39\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     40\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0martist\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_agg_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/axes/_base.py\u001b[0m in \u001b[0;36mdraw\u001b[0;34m(self, renderer, inframe)\u001b[0m\n\u001b[1;32m   2605\u001b[0m                 \u001b[0martists\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mspine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2606\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2607\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_update_title_position\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2608\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2609\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0maxison\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0minframe\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/axes/_base.py\u001b[0m in \u001b[0;36m_update_title_position\u001b[0;34m(self, renderer)\u001b[0m\n\u001b[1;32m   2547\u001b[0m                     if (ax.xaxis.get_label_position() == 'top' or\n\u001b[1;32m   2548\u001b[0m                             ax.xaxis.get_ticks_position() in choices):\n\u001b[0;32m-> 2549\u001b[0;31m                         \u001b[0mbb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0max\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mxaxis\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_tightbbox\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2550\u001b[0m                     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2551\u001b[0m                         \u001b[0mbb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0max\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_window_extent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/axis.py\u001b[0m in \u001b[0;36mget_tightbbox\u001b[0;34m(self, renderer)\u001b[0m\n\u001b[1;32m   1160\u001b[0m             \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1161\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1162\u001b[0;31m         \u001b[0mticks_to_draw\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_update_ticks\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1163\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1164\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_update_label_position\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/axis.py\u001b[0m in \u001b[0;36m_update_ticks\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1077\u001b[0m         \u001b[0mthe\u001b[0m \u001b[0maxes\u001b[0m\u001b[0;34m.\u001b[0m  \u001b[0mReturn\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mlist\u001b[0m \u001b[0mof\u001b[0m \u001b[0mticks\u001b[0m \u001b[0mthat\u001b[0m \u001b[0mwill\u001b[0m \u001b[0mbe\u001b[0m \u001b[0mdrawn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1078\u001b[0m         \"\"\"\n\u001b[0;32m-> 1079\u001b[0;31m         \u001b[0mmajor_locs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_majorticklocs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1080\u001b[0m         \u001b[0mmajor_labels\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmajor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformatter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat_ticks\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmajor_locs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1081\u001b[0m         \u001b[0mmajor_ticks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_major_ticks\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmajor_locs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/axis.py\u001b[0m in \u001b[0;36mget_majorticklocs\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1322\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mget_majorticklocs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1323\u001b[0m         \u001b[0;34m\"\"\"Get the array of major tick locations in data coordinates.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1324\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmajor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlocator\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1325\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1326\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mget_minorticklocs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/dates.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1429\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1430\u001b[0m         \u001b[0;34m'Return the locations of the ticks'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1431\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrefresh\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1432\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_locator\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1433\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/dates.py\u001b[0m in \u001b[0;36mrefresh\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1449\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mrefresh\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1450\u001b[0m         \u001b[0;34m'Refresh internal information based on current limits.'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1451\u001b[0;31m         \u001b[0mdmin\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdmax\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mviewlim_to_dt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1452\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_locator\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_locator\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdmin\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdmax\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1453\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/envs/conda_37/lib/python3.7/site-packages/matplotlib/dates.py\u001b[0m in \u001b[0;36mviewlim_to_dt\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1200\u001b[0m                              \u001b[0;34m'often happens if you pass a non-datetime '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1201\u001b[0m                              \u001b[0;34m'value to an axis that has datetime units'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1202\u001b[0;31m                              .format(vmin))\n\u001b[0m\u001b[1;32m   1203\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mnum2date\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvmin\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtz\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum2date\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvmax\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtz\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1204\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: view limit minimum -36706.56666666667 is less than 1 and is an invalid Matplotlib date value. This often happens if you pass a non-datetime value to an axis that has datetime units"
     ]
    }
   ],
   "source": [
    "annular_component(temperatures,'T0')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy import signal, stats, fftpack\n",
    "#https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.correlate.html#scipy.signal.correlate\n",
    "t0=stats.zscore(temperatures['T0'].rolling(window=3).mean().to_numpy(na_value=0))\n",
    "t10=stats.zscore(temperatures['T10'].rolling(window=7).mean().to_numpy(na_value=0))\n",
    "t30=stats.zscore(temperatures['T30'].rolling(window=7).mean().to_numpy(na_value=0))\n",
    "t100=stats.zscore(temperatures['T100'].rolling(window=3).mean().to_numpy(na_value=0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-11\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7cf8f02b2371462ca1ca3f8d54fc5359",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureCanvasNbAgg()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "corr=plt.xcorr(t10,t100,maxlags=365)\n",
    "#https://matplotlib.org/3.3.3/api/_as_gen/matplotlib.pyplot.xcorr.html\n",
    "l=corr[0]\n",
    "c=corr[1]\n",
    "print(l[np.argmax(c)])\n",
    "fig,ax=plt.subplots()\n",
    "ax.plot(l,c)\n",
    "ax.plot(l[np.argmax(c)],np.max(c),'ro')\n",
    "ax.grid()\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}