Notebook_Draft.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6c7e849a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-11T09:24:53.643384Z",
     "start_time": "2021-06-11T09:24:53.622385Z"
    }
   },
   "source": [
    "# Regression Predict Student Solution\n",
    "\n",
    "© Explore Data Science Academy\n",
    "\n",
    "---\n",
    "### Honour Code\n",
    "\n",
    "I {**YOUR NAME, YOUR SURNAME**}, confirm - by submitting this document - that the solutions in this notebook are a result of my own work and that I abide by the [EDSA honour code](https://drive.google.com/file/d/1QDCjGZJ8-FmJE3bZdIQNwnJyQKPhHZBn/view?usp=sharing).\n",
    "\n",
    "Non-compliance with the honour code constitutes a material breach of contract.\n",
    "\n",
    "### Predict Overview: Spain Electricity Shortfall Challenge\n",
    "\n",
    "The government of Spain is considering an expansion of it's renewable energy resource infrastructure investments. As such, they require information on the trends and patterns of the countries renewable sources and fossil fuel energy generation. Your company has been awarded the contract to:\n",
    "\n",
    "- 1. analyse the supplied data;\n",
    "- 2. identify potential errors in the data and clean the existing data set;\n",
    "- 3. determine if additional features can be added to enrich the data set;\n",
    "- 4. build a model that is capable of forecasting the three hourly demand shortfalls;\n",
    "- 5. evaluate the accuracy of the best machine learning model;\n",
    "- 6. determine what features were most important in the model’s prediction decision, and\n",
    "- 7. explain the inner working of the model to a non-technical audience.\n",
    "\n",
    "Formally the problem statement was given to you, the senior data scientist, by your manager via email reads as follow:\n",
    "\n",
    "> In this project you are tasked to model the shortfall between the energy generated by means of fossil fuels and various renewable sources - for the country of Spain. The daily shortfall, which will be referred to as the target variable, will be modelled as a function of various city-specific weather features such as `pressure`, `wind speed`, `humidity`, etc. As with all data science projects, the provided features are rarely adequate predictors of the target variable. As such, you are required to perform feature engineering to ensure that you will be able to accurately model Spain's three hourly shortfalls.\n",
    " \n",
    "On top of this, she has provided you with a starter notebook containing vague explanations of what the main outcomes are. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05600c92",
   "metadata": {},
   "source": [
    "<a id=\"cont\"></a>\n",
    "\n",
    "## Table of Contents\n",
    "\n",
    "<a href=#one>1. Importing Packages</a>\n",
    "\n",
    "<a href=#two>2. Loading Data</a>\n",
    "\n",
    "<a href=#three>3. Exploratory Data Analysis (EDA)</a>\n",
    "\n",
    "<a href=#four>4. Data Engineering</a>\n",
    "\n",
    "<a href=#five>5. Modeling</a>\n",
    "\n",
    "<a href=#six>6. Model Performance</a>\n",
    "\n",
    "<a href=#seven>7. Model Explanations</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "997462e2",
   "metadata": {},
   "source": [
    " <a id=\"one\"></a>\n",
    "# 1. Importing Packages\n",
    "<a href=#cont>Back to Table of Contents</a>\n",
    "\n",
    "---\n",
    "    \n",
    "| ⚡ Description: Importing Packages ⚡ |\n",
    "| :--------------------------- |\n",
    "| In this section you are required to import, and briefly discuss, the libraries that will be used throughout your analysis and modelling. |\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "475dbe93",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-23T10:30:53.800892Z",
     "start_time": "2021-06-23T10:30:50.215449Z"
    }
   },
   "outputs": [],
   "source": [
    "# Libraries for data loading, data manipulation and data visulisation\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# Libraries for data preparation and model building\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.svm import SVR\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import r2_score\n",
    "\n",
    "# Setting global constants to ensure notebook results are reproducible\n",
    "\n",
    "dtr_random_state = 42\n",
    "rfr_random_state = 0\n",
    "\n",
    "# Libraries for pickling the model\n",
    "\n",
    "import pickle\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c37370aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "#from scipy.stats import norm\n",
    "#from scipy import stats\n",
    "#from scipy.stats import pearsonr"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f22a6718",
   "metadata": {},
   "source": [
    "<a id=\"two\"></a>\n",
    "# 2. Loading the Data\n",
    "<a class=\"anchor\" id=\"1.1\"></a>\n",
    "<a href=#cont>Back to Table of Contents</a>\n",
    "\n",
    "---\n",
    "    \n",
    "| ⚡ Description: Loading the data ⚡ |\n",
    "| :--------------------------- |\n",
    "| In this section you are required to load the data from the `df_train` file into a DataFrame. |\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "bc8ac1e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('df_train.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f86d7f70",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>time</th>\n",
       "      <th>Madrid_wind_speed</th>\n",
       "      <th>Valencia_wind_deg</th>\n",
       "      <th>Bilbao_rain_1h</th>\n",
       "      <th>Valencia_wind_speed</th>\n",
       "      <th>Seville_humidity</th>\n",
       "      <th>Madrid_humidity</th>\n",
       "      <th>Bilbao_clouds_all</th>\n",
       "      <th>Bilbao_wind_speed</th>\n",
       "      <th>...</th>\n",
       "      <th>Madrid_temp_max</th>\n",
       "      <th>Barcelona_temp</th>\n",
       "      <th>Bilbao_temp_min</th>\n",
       "      <th>Bilbao_temp</th>\n",
       "      <th>Barcelona_temp_min</th>\n",
       "      <th>Bilbao_temp_max</th>\n",
       "      <th>Seville_temp_min</th>\n",
       "      <th>Madrid_temp</th>\n",
       "      <th>Madrid_temp_min</th>\n",
       "      <th>load_shortfall_3h</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01 03:00:00</td>\n",
       "      <td>0.666667</td>\n",
       "      <td>level_5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.666667</td>\n",
       "      <td>74.333333</td>\n",
       "      <td>64.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>265.938000</td>\n",
       "      <td>281.013000</td>\n",
       "      <td>269.338615</td>\n",
       "      <td>269.338615</td>\n",
       "      <td>281.013000</td>\n",
       "      <td>269.338615</td>\n",
       "      <td>274.254667</td>\n",
       "      <td>265.938000</td>\n",
       "      <td>265.938000</td>\n",
       "      <td>6715.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2015-01-01 06:00:00</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>level_10</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.666667</td>\n",
       "      <td>78.333333</td>\n",
       "      <td>64.666667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>266.386667</td>\n",
       "      <td>280.561667</td>\n",
       "      <td>270.376000</td>\n",
       "      <td>270.376000</td>\n",
       "      <td>280.561667</td>\n",
       "      <td>270.376000</td>\n",
       "      <td>274.945000</td>\n",
       "      <td>266.386667</td>\n",
       "      <td>266.386667</td>\n",
       "      <td>4171.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2015-01-01 09:00:00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>level_9</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>71.333333</td>\n",
       "      <td>64.333333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>272.708667</td>\n",
       "      <td>281.583667</td>\n",
       "      <td>275.027229</td>\n",
       "      <td>275.027229</td>\n",
       "      <td>281.583667</td>\n",
       "      <td>275.027229</td>\n",
       "      <td>278.792000</td>\n",
       "      <td>272.708667</td>\n",
       "      <td>272.708667</td>\n",
       "      <td>4274.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>2015-01-01 12:00:00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>level_8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>65.333333</td>\n",
       "      <td>56.333333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>281.895219</td>\n",
       "      <td>283.434104</td>\n",
       "      <td>281.135063</td>\n",
       "      <td>281.135063</td>\n",
       "      <td>283.434104</td>\n",
       "      <td>281.135063</td>\n",
       "      <td>285.394000</td>\n",
       "      <td>281.895219</td>\n",
       "      <td>281.895219</td>\n",
       "      <td>5075.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>2015-01-01 15:00:00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>level_7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>57.000000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>...</td>\n",
       "      <td>280.678437</td>\n",
       "      <td>284.213167</td>\n",
       "      <td>282.252063</td>\n",
       "      <td>282.252063</td>\n",
       "      <td>284.213167</td>\n",
       "      <td>282.252063</td>\n",
       "      <td>285.513719</td>\n",
       "      <td>280.678437</td>\n",
       "      <td>280.678437</td>\n",
       "      <td>6620.666667</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 49 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0                 time  Madrid_wind_speed Valencia_wind_deg  \\\n",
       "0           0  2015-01-01 03:00:00           0.666667           level_5   \n",
       "1           1  2015-01-01 06:00:00           0.333333          level_10   \n",
       "2           2  2015-01-01 09:00:00           1.000000           level_9   \n",
       "3           3  2015-01-01 12:00:00           1.000000           level_8   \n",
       "4           4  2015-01-01 15:00:00           1.000000           level_7   \n",
       "\n",
       "   Bilbao_rain_1h  Valencia_wind_speed  Seville_humidity  Madrid_humidity  \\\n",
       "0             0.0             0.666667         74.333333        64.000000   \n",
       "1             0.0             1.666667         78.333333        64.666667   \n",
       "2             0.0             1.000000         71.333333        64.333333   \n",
       "3             0.0             1.000000         65.333333        56.333333   \n",
       "4             0.0             1.000000         59.000000        57.000000   \n",
       "\n",
       "   Bilbao_clouds_all  Bilbao_wind_speed  ...  Madrid_temp_max  Barcelona_temp  \\\n",
       "0                0.0           1.000000  ...       265.938000      281.013000   \n",
       "1                0.0           1.000000  ...       266.386667      280.561667   \n",
       "2                0.0           1.000000  ...       272.708667      281.583667   \n",
       "3                0.0           1.000000  ...       281.895219      283.434104   \n",
       "4                2.0           0.333333  ...       280.678437      284.213167   \n",
       "\n",
       "   Bilbao_temp_min  Bilbao_temp  Barcelona_temp_min  Bilbao_temp_max  \\\n",
       "0       269.338615   269.338615          281.013000       269.338615   \n",
       "1       270.376000   270.376000          280.561667       270.376000   \n",
       "2       275.027229   275.027229          281.583667       275.027229   \n",
       "3       281.135063   281.135063          283.434104       281.135063   \n",
       "4       282.252063   282.252063          284.213167       282.252063   \n",
       "\n",
       "   Seville_temp_min Madrid_temp  Madrid_temp_min  load_shortfall_3h  \n",
       "0        274.254667  265.938000       265.938000        6715.666667  \n",
       "1        274.945000  266.386667       266.386667        4171.666667  \n",
       "2        278.792000  272.708667       272.708667        4274.666667  \n",
       "3        285.394000  281.895219       281.895219        5075.666667  \n",
       "4        285.513719  280.678437       280.678437        6620.666667  \n",
       "\n",
       "[5 rows x 49 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68221620",
   "metadata": {},
   "source": [
    "Placeholder for droppping worthless Unnamed: 0 feature as it brings no value to the analysis and model creation..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "46554ae3",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop(df.columns[0], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b1bdb8f0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8763, 48)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7a40ef5",
   "metadata": {},
   "source": [
    "Placeholder | Test dataset has the same feature so we remove it immediately after reading the csv file and check the first 5 observations..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f3f83c2d",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_test = pd.read_csv('df_test.csv')\n",
    "df_test = df_test.drop(df_test.columns[0], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0a8a42f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>Madrid_wind_speed</th>\n",
       "      <th>Valencia_wind_deg</th>\n",
       "      <th>Bilbao_rain_1h</th>\n",
       "      <th>Valencia_wind_speed</th>\n",
       "      <th>Seville_humidity</th>\n",
       "      <th>Madrid_humidity</th>\n",
       "      <th>Bilbao_clouds_all</th>\n",
       "      <th>Bilbao_wind_speed</th>\n",
       "      <th>Seville_clouds_all</th>\n",
       "      <th>...</th>\n",
       "      <th>Barcelona_temp_max</th>\n",
       "      <th>Madrid_temp_max</th>\n",
       "      <th>Barcelona_temp</th>\n",
       "      <th>Bilbao_temp_min</th>\n",
       "      <th>Bilbao_temp</th>\n",
       "      <th>Barcelona_temp_min</th>\n",
       "      <th>Bilbao_temp_max</th>\n",
       "      <th>Seville_temp_min</th>\n",
       "      <th>Madrid_temp</th>\n",
       "      <th>Madrid_temp_min</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018-01-01 00:00:00</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>level_8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>87.000000</td>\n",
       "      <td>71.333333</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>287.816667</td>\n",
       "      <td>280.816667</td>\n",
       "      <td>287.356667</td>\n",
       "      <td>276.150000</td>\n",
       "      <td>280.380000</td>\n",
       "      <td>286.816667</td>\n",
       "      <td>285.150000</td>\n",
       "      <td>283.150000</td>\n",
       "      <td>279.866667</td>\n",
       "      <td>279.150000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018-01-01 03:00:00</td>\n",
       "      <td>4.666667</td>\n",
       "      <td>level_8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.333333</td>\n",
       "      <td>89.000000</td>\n",
       "      <td>78.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.666667</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>284.816667</td>\n",
       "      <td>280.483333</td>\n",
       "      <td>284.190000</td>\n",
       "      <td>277.816667</td>\n",
       "      <td>281.010000</td>\n",
       "      <td>283.483333</td>\n",
       "      <td>284.150000</td>\n",
       "      <td>281.150000</td>\n",
       "      <td>279.193333</td>\n",
       "      <td>278.150000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018-01-01 06:00:00</td>\n",
       "      <td>2.333333</td>\n",
       "      <td>level_7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>89.000000</td>\n",
       "      <td>89.666667</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.333333</td>\n",
       "      <td>6.666667</td>\n",
       "      <td>...</td>\n",
       "      <td>284.483333</td>\n",
       "      <td>276.483333</td>\n",
       "      <td>283.150000</td>\n",
       "      <td>276.816667</td>\n",
       "      <td>279.196667</td>\n",
       "      <td>281.816667</td>\n",
       "      <td>282.150000</td>\n",
       "      <td>280.483333</td>\n",
       "      <td>276.340000</td>\n",
       "      <td>276.150000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018-01-01 09:00:00</td>\n",
       "      <td>2.666667</td>\n",
       "      <td>level_7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.333333</td>\n",
       "      <td>93.333333</td>\n",
       "      <td>82.666667</td>\n",
       "      <td>26.666667</td>\n",
       "      <td>5.666667</td>\n",
       "      <td>6.666667</td>\n",
       "      <td>...</td>\n",
       "      <td>284.150000</td>\n",
       "      <td>277.150000</td>\n",
       "      <td>283.190000</td>\n",
       "      <td>279.150000</td>\n",
       "      <td>281.740000</td>\n",
       "      <td>282.150000</td>\n",
       "      <td>284.483333</td>\n",
       "      <td>279.150000</td>\n",
       "      <td>275.953333</td>\n",
       "      <td>274.483333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018-01-01 12:00:00</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>level_7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.666667</td>\n",
       "      <td>65.333333</td>\n",
       "      <td>64.000000</td>\n",
       "      <td>26.666667</td>\n",
       "      <td>10.666667</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>287.483333</td>\n",
       "      <td>281.150000</td>\n",
       "      <td>286.816667</td>\n",
       "      <td>281.816667</td>\n",
       "      <td>284.116667</td>\n",
       "      <td>286.150000</td>\n",
       "      <td>286.816667</td>\n",
       "      <td>284.483333</td>\n",
       "      <td>280.686667</td>\n",
       "      <td>280.150000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 47 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                  time  Madrid_wind_speed Valencia_wind_deg  Bilbao_rain_1h  \\\n",
       "0  2018-01-01 00:00:00           5.000000           level_8             0.0   \n",
       "1  2018-01-01 03:00:00           4.666667           level_8             0.0   \n",
       "2  2018-01-01 06:00:00           2.333333           level_7             0.0   \n",
       "3  2018-01-01 09:00:00           2.666667           level_7             0.0   \n",
       "4  2018-01-01 12:00:00           4.000000           level_7             0.0   \n",
       "\n",
       "   Valencia_wind_speed  Seville_humidity  Madrid_humidity  Bilbao_clouds_all  \\\n",
       "0             5.000000         87.000000        71.333333          20.000000   \n",
       "1             5.333333         89.000000        78.000000           0.000000   \n",
       "2             5.000000         89.000000        89.666667           0.000000   \n",
       "3             5.333333         93.333333        82.666667          26.666667   \n",
       "4             8.666667         65.333333        64.000000          26.666667   \n",
       "\n",
       "   Bilbao_wind_speed  Seville_clouds_all  ...  Barcelona_temp_max  \\\n",
       "0           3.000000            0.000000  ...          287.816667   \n",
       "1           3.666667            0.000000  ...          284.816667   \n",
       "2           2.333333            6.666667  ...          284.483333   \n",
       "3           5.666667            6.666667  ...          284.150000   \n",
       "4          10.666667            0.000000  ...          287.483333   \n",
       "\n",
       "   Madrid_temp_max  Barcelona_temp  Bilbao_temp_min  Bilbao_temp  \\\n",
       "0       280.816667      287.356667       276.150000   280.380000   \n",
       "1       280.483333      284.190000       277.816667   281.010000   \n",
       "2       276.483333      283.150000       276.816667   279.196667   \n",
       "3       277.150000      283.190000       279.150000   281.740000   \n",
       "4       281.150000      286.816667       281.816667   284.116667   \n",
       "\n",
       "   Barcelona_temp_min Bilbao_temp_max  Seville_temp_min  Madrid_temp  \\\n",
       "0          286.816667      285.150000        283.150000   279.866667   \n",
       "1          283.483333      284.150000        281.150000   279.193333   \n",
       "2          281.816667      282.150000        280.483333   276.340000   \n",
       "3          282.150000      284.483333        279.150000   275.953333   \n",
       "4          286.150000      286.816667        284.483333   280.686667   \n",
       "\n",
       "   Madrid_temp_min  \n",
       "0       279.150000  \n",
       "1       278.150000  \n",
       "2       276.150000  \n",
       "3       274.483333  \n",
       "4       280.150000  \n",
       "\n",
       "[5 rows x 47 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "06b9c920",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2920, 47)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2efd2a1c",
   "metadata": {},
   "source": [
    "Placeholder| Confirm without target variable load_shortfall_3h..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81132ab3",
   "metadata": {},
   "source": [
    "<a id=\"three\"></a>\n",
    "# 3. Exploratory Data Analysis (EDA)\n",
    "<a class=\"anchor\" id=\"1.1\"></a>\n",
    "<a href=#cont>Back to Table of Contents</a>\n",
    "\n",
    "---\n",
    "    \n",
    "| ⚡ Description: Exploratory data analysis ⚡ |\n",
    "| :--------------------------- |\n",
    "| In this section, you are required to perform an in-depth analysis of all the variables in the DataFrame. |\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0e38b98",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "030149b0",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e805134e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-28T08:52:37.824204Z",
     "start_time": "2021-06-28T08:52:37.811206Z"
    }
   },
   "outputs": [],
   "source": [
    "# look at data statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e6d40dc",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fb74182",
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot relevant feature interactions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3259b87b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6dd6ee8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# evaluate correlation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db4accb6",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "de51df85",
   "metadata": {},
   "outputs": [],
   "source": [
    "# have a look at feature distributions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d906ae37",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6a0dc2b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "3fa93ec6",
   "metadata": {},
   "source": [
    "<a id=\"four\"></a>\n",
    "# 4. Data Engineering\n",
    "<a class=\"anchor\" id=\"1.1\"></a>\n",
    "<a href=#cont>Back to Table of Contents</a>\n",
    "\n",
    "---\n",
    "    \n",
    "| ⚡ Description: Data engineering ⚡ |\n",
    "| :--------------------------- |\n",
    "| In this section you are required to: clean the dataset, and possibly create new features - as identified in the EDA phase. |\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "059c2f3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# remove missing values/ features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50a0bfb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# engineer existing features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "700ee82b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create new features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "84cad62d",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_clean = df.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d184dc2",
   "metadata": {},
   "source": [
    "Before we commence data cleaning and feature engineering, a copy of the training dataset is made."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85640365",
   "metadata": {},
   "source": [
    "## Imputing Data for Null Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0f0c8fae",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_clean[\"Valencia_pressure\"] = df_clean[\"Valencia_pressure\"].fillna(df_clean[\"Valencia_pressure\"].mode()[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ed8a5cc",
   "metadata": {},
   "source": [
    "Placeholder for describing null value operation..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef23be83",
   "metadata": {},
   "source": [
    "## Dropping Features"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1d746ff",
   "metadata": {},
   "source": [
    "Placeholder for why are temp_min and temp_max being dropped..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d5abc359",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_clean = df_clean.loc[:, ~df_clean.columns.str.contains(\"temp_min\")]\n",
    "df_clean = df_clean.loc[:, ~df_clean.columns.str.contains(\"temp_max\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "161ba9d4",
   "metadata": {},
   "source": [
    "## Engineering Existing Features"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e30c5b0",
   "metadata": {},
   "source": [
    "Placeholder for transformations done in the following cell... "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "32f3c3ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_clean[\"Valencia_wind_deg\"] = df_clean[\"Valencia_wind_deg\"].str.extract('(\\d+)')\n",
    "df_clean[\"Valencia_wind_deg\"] = pd.to_numeric(df_clean[\"Valencia_wind_deg\"])\n",
    "\n",
    "df_clean[\"Seville_pressure\"] = df_clean[\"Seville_pressure\"].str.extract('(\\d+)')\n",
    "df_clean[\"Seville_pressure\"] = pd.to_numeric(df_clean[\"Seville_pressure\"])\n",
    "\n",
    "df_clean[\"time\"] = pd.to_datetime(df_clean[\"time\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c34a5fa9",
   "metadata": {},
   "source": [
    "## Engineering New Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "9ca87eb1",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_clean[\"Hour\"] = df_clean[\"time\"].dt.hour\n",
    "df_clean[\"Day\"] = df_clean[\"time\"].dt.day\n",
    "df_clean[\"Weekday\"] = df_clean[\"time\"].dt.weekday\n",
    "df_clean[\"Week\"] = df_clean[\"time\"].dt.isocalendar().week\n",
    "df_clean['Week'] = df_clean['Week'].astype('int64')\n",
    "df_clean[\"Month\"] = df_clean[\"time\"].dt.month\n",
    "df_clean[\"Year\"] = df_clean[\"time\"].dt.year\n",
    "\n",
    "df_clean = df_clean.drop(['time'], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb79045b",
   "metadata": {},
   "source": [
    "Placeholder for time feature..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "872e40ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "seasons = {1: 'Winter',\n",
    "           2: 'Winter',\n",
    "           3: 'Spring',\n",
    "           4: 'Spring',\n",
    "           5: 'Spring',\n",
    "           6: 'Summer',\n",
    "           7: 'Summer',\n",
    "           8: 'Summer',\n",
    "           9: 'Autumn',\n",
    "           10: 'Autumn',\n",
    "           11: 'Autumn',\n",
    "           12: 'Winter',\n",
    "           }\n",
    "\n",
    "df_clean['Season'] = df_clean['Month'].apply(lambda x: seasons[x])\n",
    "\n",
    "df_dum = pd.get_dummies(df_clean)\n",
    "\n",
    "df_dum.columns = [col.replace(\" \",\"_\") for col in df_dum.columns]\n",
    "\n",
    "df_dum['Season_Winter'] = df_dum['Season_Winter'].astype('int64')\n",
    "df_dum['Season_Spring'] = df_dum['Season_Spring'].astype('int64')\n",
    "df_dum['Season_Summer'] = df_dum['Season_Summer'].astype('int64')\n",
    "df_dum['Season_Autumn'] = df_dum['Season_Autumn'].astype('int64')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b903af46",
   "metadata": {},
   "source": [
    "Placeholder for season dummy encoding feature..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43b2d523",
   "metadata": {},
   "source": [
    "<a id=\"five\"></a>\n",
    "# 5. Modelling\n",
    "<a class=\"anchor\" id=\"1.1\"></a>\n",
    "<a href=#cont>Back to Table of Contents</a>\n",
    "\n",
    "---\n",
    "    \n",
    "| ⚡ Description: Modelling ⚡ |\n",
    "| :--------------------------- |\n",
    "| In this section, you are required to create one or more regression models that are able to accurately predict the thee hour load shortfall. |\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3682a754",
   "metadata": {},
   "source": [
    "The following section will detail the creation, training and predictions of three regression models as follows:\n",
    "1. Multiple Linear Regressor\n",
    "2. Decision Tree Regressor\n",
    "3. Random Forest Regressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebee7d9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# split data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2344b3e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create targets and features dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b84c309e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create one or more ML models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9b20c5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# evaluate one or more ML models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01a2bf9c",
   "metadata": {},
   "source": [
    "First we need to split the training data into the target and predictor variable datasets.\n",
    "\n",
    "Furthermore we also use the 'train_test_split' function to divide the now seperate target and predictor variables into training and validation datasets (i.e. X_train, X_valid, y_train, y_valid)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6ae95a31",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df_dum.drop(['load_shortfall_3h'], axis=1) # Predictor Variables\n",
    "y = df_dum['load_shortfall_3h']                # Target Variable\n",
    "\n",
    "X_train, X_valid, y_train, y_valid = train_test_split (X, y, test_size=0.20, random_state=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ca0bb84",
   "metadata": {},
   "source": [
    "# 5.1 Model Creation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "281685f8",
   "metadata": {},
   "source": [
    "## Multiple Linear Regressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "64d0680f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression()"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mlr = LinearRegression()\n",
    "mlr.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76645043",
   "metadata": {},
   "source": [
    "## Decision Tree Regressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "b03e0e79",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(random_state=42)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dtr = DecisionTreeRegressor(random_state = dtr_random_state)\n",
    "dtr.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df049b3",
   "metadata": {},
   "source": [
    "## Random Forest Regressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "6ae5485f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(random_state=0)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rfr = RandomForestRegressor(n_estimators = 100, random_state = rfr_random_state)\n",
    "rfr.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09fae193",
   "metadata": {},
   "source": [
    "All of the above models are trained on only the split training dataset (X_train and y_train) while the validation set (X_valid and y_valid) are held back."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ca51cdf8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Training RMSE</th>\n",
       "      <th>Validation RMSE</th>\n",
       "      <th>Training Rscore</th>\n",
       "      <th>Validation Rscore</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Multiple Linear Regression</th>\n",
       "      <td>4675.437541</td>\n",
       "      <td>4643.808937</td>\n",
       "      <td>0.200765</td>\n",
       "      <td>0.192969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Decision Tree Regression</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>3875.795353</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.437836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Random Forest Regression</th>\n",
       "      <td>1006.352531</td>\n",
       "      <td>2634.636329</td>\n",
       "      <td>0.962972</td>\n",
       "      <td>0.740234</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            Training RMSE  Validation RMSE  Training Rscore  \\\n",
       "Multiple Linear Regression    4675.437541      4643.808937         0.200765   \n",
       "Decision Tree Regression         0.000000      3875.795353         1.000000   \n",
       "Random Forest Regression      1006.352531      2634.636329         0.962972   \n",
       "\n",
       "                            Validation Rscore  \n",
       "Multiple Linear Regression           0.192969  \n",
       "Decision Tree Regression             0.437836  \n",
       "Random Forest Regression             0.740234  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results_dict = {'Training RMSE':\n",
    "                    {\n",
    "                        \"Multiple Linear Regression\": np.sqrt(mean_squared_error(y_train, mlr.predict(X_train))),\n",
    "                        \"Decision Tree Regression\": np.sqrt(mean_squared_error(y_train, dtr.predict(X_train))),\n",
    "                        \"Random Forest Regression\": np.sqrt(mean_squared_error(y_train, rfr.predict(X_train)))\n",
    "                    },\n",
    "                'Validation RMSE':\n",
    "                    {\n",
    "                        \"Multiple Linear Regression\": np.sqrt(mean_squared_error(y_valid, mlr.predict(X_valid))),\n",
    "                        \"Decision Tree Regression\": np.sqrt(mean_squared_error(y_valid, dtr.predict(X_valid))),\n",
    "                        \"Random Forest Regression\": np.sqrt(mean_squared_error(y_valid, rfr.predict(X_valid)))\n",
    "                    },\n",
    "                'Training Rscore':\n",
    "                    {\n",
    "                        \"Multiple Linear Regression\": r2_score(y_train, mlr.predict(X_train)),\n",
    "                        \"Decision Tree Regression\": r2_score(y_train, dtr.predict(X_train)),\n",
    "                        \"Random Forest Regression\": r2_score(y_train, rfr.predict(X_train))\n",
    "                    },\n",
    "                'Validation Rscore':\n",
    "                    {\n",
    "                        \"Multiple Linear Regression\": r2_score(y_valid, mlr.predict(X_valid)),\n",
    "                        \"Decision Tree Regression\": r2_score(y_valid, dtr.predict(X_valid)),\n",
    "                        \"Random Forest Regression\": r2_score(y_valid, rfr.predict(X_valid)) \n",
    "                    }\n",
    "                }\n",
    "\n",
    "results_df = pd.DataFrame(data=results_dict)\n",
    "results_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6892e941",
   "metadata": {},
   "source": [
    "Discussion of statistics shown and comparison of results across training and validation sets..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "b83b92e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "mlr.fit(X, y)\n",
    "linearpreds = mlr.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "c8c9df24",
   "metadata": {},
   "outputs": [],
   "source": [
    "dtr.fit(X, y)\n",
    "treepreds = dtr.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f55a1472",
   "metadata": {},
   "outputs": [],
   "source": [
    "rfr.fit(X, y)\n",
    "forestpreds = rfr.predict(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47898c3c",
   "metadata": {},
   "source": [
    "Placeholder | Each model is then retrained on all the data present in the original training dataset (X and y) and the RMSE reviewed..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "394d3fe8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>RMSE for X</th>\n",
       "      <th>Rscore for X</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Multiple Linear Regression</th>\n",
       "      <td>4667.021386</td>\n",
       "      <td>0.199957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Decision Tree Regression</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Random Forest Regression</th>\n",
       "      <td>943.997753</td>\n",
       "      <td>0.967268</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             RMSE for X  Rscore for X\n",
       "Multiple Linear Regression  4667.021386      0.199957\n",
       "Decision Tree Regression       0.000000      1.000000\n",
       "Random Forest Regression     943.997753      0.967268"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results_dictX = {'RMSE for X':\n",
    "                    {\n",
    "                        \"Multiple Linear Regression\": np.sqrt(mean_squared_error(y, linearpreds)),\n",
    "                        \"Decision Tree Regression\": np.sqrt(mean_squared_error(y, treepreds)),\n",
    "                        \"Random Forest Regression\": np.sqrt(mean_squared_error(y, forestpreds))\n",
    "                    },\n",
    "                'Rscore for X':\n",
    "                    {\n",
    "                        \"Multiple Linear Regression\": r2_score(y, linearpreds),\n",
    "                        \"Decision Tree Regression\": r2_score(y, treepreds),\n",
    "                        \"Random Forest Regression\": r2_score(y, forestpreds)\n",
    "                    }\n",
    "                }\n",
    "\n",
    "results_dfX = pd.DataFrame(data=results_dictX)\n",
    "results_dfX"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef9018b3",
   "metadata": {},
   "source": [
    "## 5.2 Comparison of Predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ae1fd90",
   "metadata": {},
   "source": [
    "Placeholder | Comparison of known target variable values to predictions by individual models..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "bccd5df2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Multiple Linear Regression</th>\n",
       "      <th>Decision Tree Regression</th>\n",
       "      <th>Random Forest Regression</th>\n",
       "      <th>load_shortfall_3h</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9746.125825</td>\n",
       "      <td>6715.666667</td>\n",
       "      <td>6145.710000</td>\n",
       "      <td>6715.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9158.339116</td>\n",
       "      <td>4171.666667</td>\n",
       "      <td>4159.906667</td>\n",
       "      <td>4171.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9513.514667</td>\n",
       "      <td>4274.666667</td>\n",
       "      <td>4488.166667</td>\n",
       "      <td>4274.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10079.326586</td>\n",
       "      <td>5075.666667</td>\n",
       "      <td>5604.960000</td>\n",
       "      <td>5075.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10377.215007</td>\n",
       "      <td>6620.666667</td>\n",
       "      <td>6513.820000</td>\n",
       "      <td>6620.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8758</th>\n",
       "      <td>11364.802872</td>\n",
       "      <td>-28.333333</td>\n",
       "      <td>1174.730000</td>\n",
       "      <td>-28.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8759</th>\n",
       "      <td>10993.212284</td>\n",
       "      <td>2266.666667</td>\n",
       "      <td>2775.543333</td>\n",
       "      <td>2266.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8760</th>\n",
       "      <td>9217.076488</td>\n",
       "      <td>822.000000</td>\n",
       "      <td>1222.063333</td>\n",
       "      <td>822.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8761</th>\n",
       "      <td>10371.789996</td>\n",
       "      <td>-760.000000</td>\n",
       "      <td>1109.246667</td>\n",
       "      <td>-760.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8762</th>\n",
       "      <td>10842.825991</td>\n",
       "      <td>2780.666667</td>\n",
       "      <td>4005.876667</td>\n",
       "      <td>2780.666667</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8763 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Multiple Linear Regression  Decision Tree Regression  \\\n",
       "0                    9746.125825               6715.666667   \n",
       "1                    9158.339116               4171.666667   \n",
       "2                    9513.514667               4274.666667   \n",
       "3                   10079.326586               5075.666667   \n",
       "4                   10377.215007               6620.666667   \n",
       "...                          ...                       ...   \n",
       "8758                11364.802872                -28.333333   \n",
       "8759                10993.212284               2266.666667   \n",
       "8760                 9217.076488                822.000000   \n",
       "8761                10371.789996               -760.000000   \n",
       "8762                10842.825991               2780.666667   \n",
       "\n",
       "      Random Forest Regression  load_shortfall_3h  \n",
       "0                  6145.710000        6715.666667  \n",
       "1                  4159.906667        4171.666667  \n",
       "2                  4488.166667        4274.666667  \n",
       "3                  5604.960000        5075.666667  \n",
       "4                  6513.820000        6620.666667  \n",
       "...                        ...                ...  \n",
       "8758               1174.730000         -28.333333  \n",
       "8759               2775.543333        2266.666667  \n",
       "8760               1222.063333         822.000000  \n",
       "8761               1109.246667        -760.000000  \n",
       "8762               4005.876667        2780.666667  \n",
       "\n",
       "[8763 rows x 4 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_predictions = pd.concat([pd.DataFrame(linearpreds, columns=['Multiple Linear Regression']),\n",
    "                               pd.DataFrame(treepreds, columns=['Decision Tree Regression']),\n",
    "                               pd.DataFrame(forestpreds, columns=['Random Forest Regression']), y], axis =1)\n",
    "train_predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b530251",
   "metadata": {},
   "source": [
    "<a id=\"six\"></a>\n",
    "# 6. Model Performance\n",
    "<a class=\"anchor\" id=\"1.1\"></a>\n",
    "<a href=#cont>Back to Table of Contents</a>\n",
    "\n",
    "---\n",
    "    \n",
    "| ⚡ Description: Model performance ⚡ |\n",
    "| :--------------------------- |\n",
    "| In this section you are required to compare the relative performance of the various trained ML models on a holdout dataset and comment on what model is the best and why. |\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a69b5a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compare model performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05df4a0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Choose best model and motivate why it is the best choice"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "778303d1",
   "metadata": {},
   "source": [
    "## Returning to the Test Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84397aac",
   "metadata": {},
   "source": [
    "Placeholder | All transformations to the training data set need to be repeated for the test data set..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "8fdaaf3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_t = df_test.copy()\n",
    "\n",
    "# impute missing values AND features\n",
    "\n",
    "df_t[\"Valencia_pressure\"] = df_t[\"Valencia_pressure\"].fillna(df_t[\"Valencia_pressure\"].mode()[0]) \n",
    "\n",
    "df_t = df_t.loc[:, ~df_t.columns.str.contains(\"temp_min\")]\n",
    "df_t = df_t.loc[:, ~df_t.columns.str.contains(\"temp_max\")]\n",
    "\n",
    "# engineer existing features\n",
    "\n",
    "df_t[\"Valencia_wind_deg\"] = df_t[\"Valencia_wind_deg\"].str.extract('(\\d+)')\n",
    "df_t[\"Valencia_wind_deg\"] = pd.to_numeric(df_t[\"Valencia_wind_deg\"])\n",
    "\n",
    "df_t[\"Seville_pressure\"] = df_t[\"Seville_pressure\"].str.extract('(\\d+)')\n",
    "df_t[\"Seville_pressure\"] = pd.to_numeric(df_t[\"Seville_pressure\"])\n",
    "\n",
    "df_t[\"time\"] = pd.to_datetime(df_t[\"time\"])\n",
    "\n",
    "# Engineer new features\n",
    "\n",
    "df_t[\"Hour\"] = df_t[\"time\"].dt.hour\n",
    "df_t[\"Day\"] = df_t[\"time\"].dt.day\n",
    "df_t[\"Weekday\"] = df_t[\"time\"].dt.weekday\n",
    "df_t[\"Week\"] = df_t[\"time\"].dt.isocalendar().week\n",
    "df_t['Week'] = df_t['Week'].astype('int64')\n",
    "df_t[\"Month\"] = df_t[\"time\"].dt.month\n",
    "df_t[\"Year\"] = df_t[\"time\"].dt.year\n",
    "\n",
    "df_t = df_t.drop(['time'], axis=1)\n",
    "\n",
    "df_t['Season'] = df_t['Month'].apply(lambda x: seasons[x])\n",
    "\n",
    "df_dt = pd.get_dummies(df_t)\n",
    "\n",
    "df_dt.columns = [col.replace(\" \",\"_\") for col in df_dt.columns]\n",
    "\n",
    "df_dt['Season_Winter'] = df_dt['Season_Winter'].astype('int64')\n",
    "df_dt['Season_Spring'] = df_dt['Season_Spring'].astype('int64')\n",
    "df_dt['Season_Summer'] = df_dt['Season_Summer'].astype('int64')\n",
    "df_dt['Season_Autumn'] = df_dt['Season_Autumn'].astype('int64')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b3ac6b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_dt.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "7b81fd0f",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_mlr = mlr.predict(df_dt)\n",
    "\n",
    "daf_mlr = pd.DataFrame(test_mlr, columns=['Multiple Linear Regression'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "84814fc5",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_dtr = dtr.predict(df_dt)\n",
    "\n",
    "daf_dtr = pd.DataFrame(test_dtr, columns=['Decision Tree Regression'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "af4a0477",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_rfr = rfr.predict(df_dt)\n",
    "\n",
    "daf_rfr = pd.DataFrame(test_rfr, columns=['Random Forest Regression'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e973838",
   "metadata": {},
   "source": [
    "Placeholder | Three models trained on engineered test data (Predictors)..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "be76f785",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Multiple Linear Regression</th>\n",
       "      <th>Decision Tree Regression</th>\n",
       "      <th>Random Forest Regression</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9920.840811</td>\n",
       "      <td>10246.333333</td>\n",
       "      <td>10603.870000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9833.990571</td>\n",
       "      <td>11881.666667</td>\n",
       "      <td>8100.506667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10598.591711</td>\n",
       "      <td>8320.000000</td>\n",
       "      <td>7852.406667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10866.829643</td>\n",
       "      <td>14774.666667</td>\n",
       "      <td>10336.593333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>9871.704599</td>\n",
       "      <td>17855.333333</td>\n",
       "      <td>13223.313333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2915</th>\n",
       "      <td>13310.901282</td>\n",
       "      <td>16656.000000</td>\n",
       "      <td>14510.980000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2916</th>\n",
       "      <td>15208.550806</td>\n",
       "      <td>13247.666667</td>\n",
       "      <td>15424.905000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2917</th>\n",
       "      <td>16454.467053</td>\n",
       "      <td>15934.000000</td>\n",
       "      <td>15583.223333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2918</th>\n",
       "      <td>16457.750416</td>\n",
       "      <td>14083.333333</td>\n",
       "      <td>15841.803333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2919</th>\n",
       "      <td>15085.075076</td>\n",
       "      <td>16283.333333</td>\n",
       "      <td>17402.193333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2920 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Multiple Linear Regression  Decision Tree Regression  \\\n",
       "0                    9920.840811              10246.333333   \n",
       "1                    9833.990571              11881.666667   \n",
       "2                   10598.591711               8320.000000   \n",
       "3                   10866.829643              14774.666667   \n",
       "4                    9871.704599              17855.333333   \n",
       "...                          ...                       ...   \n",
       "2915                13310.901282              16656.000000   \n",
       "2916                15208.550806              13247.666667   \n",
       "2917                16454.467053              15934.000000   \n",
       "2918                16457.750416              14083.333333   \n",
       "2919                15085.075076              16283.333333   \n",
       "\n",
       "      Random Forest Regression  \n",
       "0                 10603.870000  \n",
       "1                  8100.506667  \n",
       "2                  7852.406667  \n",
       "3                 10336.593333  \n",
       "4                 13223.313333  \n",
       "...                        ...  \n",
       "2915              14510.980000  \n",
       "2916              15424.905000  \n",
       "2917              15583.223333  \n",
       "2918              15841.803333  \n",
       "2919              17402.193333  \n",
       "\n",
       "[2920 rows x 3 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "full_predictions = pd.concat([daf_mlr, daf_dtr, daf_rfr], axis =1)\n",
    "full_predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c17dcf7e",
   "metadata": {},
   "source": [
    "Placeholder | The above represents the predictions made on the test data set for each of the models created..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4714458b",
   "metadata": {},
   "source": [
    "## Creating Submission File"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3015bac9",
   "metadata": {},
   "source": [
    "Placeholder | The following code readies the model predictions to be written to a csv file..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "b3e461b1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>load_shortfall_3h</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018-01-01 00:00:00</td>\n",
       "      <td>10603.870000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018-01-01 03:00:00</td>\n",
       "      <td>8100.506667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018-01-01 06:00:00</td>\n",
       "      <td>7852.406667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018-01-01 09:00:00</td>\n",
       "      <td>10336.593333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018-01-01 12:00:00</td>\n",
       "      <td>13223.313333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2915</th>\n",
       "      <td>2018-12-31 09:00:00</td>\n",
       "      <td>14510.980000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2916</th>\n",
       "      <td>2018-12-31 12:00:00</td>\n",
       "      <td>15424.905000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2917</th>\n",
       "      <td>2018-12-31 15:00:00</td>\n",
       "      <td>15583.223333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2918</th>\n",
       "      <td>2018-12-31 18:00:00</td>\n",
       "      <td>15841.803333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2919</th>\n",
       "      <td>2018-12-31 21:00:00</td>\n",
       "      <td>17402.193333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2920 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     time  load_shortfall_3h\n",
       "0     2018-01-01 00:00:00       10603.870000\n",
       "1     2018-01-01 03:00:00        8100.506667\n",
       "2     2018-01-01 06:00:00        7852.406667\n",
       "3     2018-01-01 09:00:00       10336.593333\n",
       "4     2018-01-01 12:00:00       13223.313333\n",
       "...                   ...                ...\n",
       "2915  2018-12-31 09:00:00       14510.980000\n",
       "2916  2018-12-31 12:00:00       15424.905000\n",
       "2917  2018-12-31 15:00:00       15583.223333\n",
       "2918  2018-12-31 18:00:00       15841.803333\n",
       "2919  2018-12-31 21:00:00       17402.193333\n",
       "\n",
       "[2920 rows x 2 columns]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output = pd.DataFrame({\"time\":df_test['time']})\n",
    "submission = output.join(daf_rfr)\n",
    "submission.rename(columns = {'Random Forest Regression':'load_shortfall_3h'}, inplace = True)\n",
    "submission.to_csv(\"submission.csv\", index=False)\n",
    "submission"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08719c4c",
   "metadata": {},
   "source": [
    "## Pickling the Chosen Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "814c431f",
   "metadata": {},
   "source": [
    "Placeholder | Mention pickling and decide if the test/load portion is necessary..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "6b59ebcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_save_path = \"rfr_model.pkl\"\n",
    "with open(model_save_path,'wb') as file:\n",
    "    pickle.dump(rfr, file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "0f3cb933",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_load_path = \"rfr_model.pkl\"\n",
    "with open(model_load_path,'rb') as file:\n",
    "    unpickled_model = pickle.load(file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "21ccdd89",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10603.87      ,  8100.50666667,  7852.40666667, ...,\n",
       "       15583.22333333, 15841.80333333, 17402.19333333])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred = unpickled_model.predict(df_dt)\n",
    "y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8ad0c0d",
   "metadata": {},
   "source": [
    "<a id=\"seven\"></a>\n",
    "# 7. Model Explanations\n",
    "<a class=\"anchor\" id=\"1.1\"></a>\n",
    "<a href=#cont>Back to Table of Contents</a>\n",
    "\n",
    "---\n",
    "    \n",
    "| ⚡ Description: Model explanation ⚡ |\n",
    "| :--------------------------- |\n",
    "| In this section, you are required to discuss how the best performing model works in a simple way so that both technical and non-technical stakeholders can grasp the intuition behind the model's inner workings. |\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ff741c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# discuss chosen methods logic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d222b3d",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}