Machine_Learning_Electric_Final_Information.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Machine Learning Models and Predicting Public Sector Electricity Costs\n",
    "\n",
    "# Purpose\n",
    "\n",
    "The aim of this project is to predict the Primary average energy price for each State in the residential sector for the year 2021 based on residential sector features. And to do so using a variety of Machine Learning Models. \n",
    "\n",
    "\n",
    "\n",
    "## Background\n",
    "\n",
    "### Data information\n",
    "EIA's State Energy Data System (SEDS) is a comprehensive data set that consists of annual time series estimates of State-level energy use by major economic sectors, energy production and State-level energy price and expenditure data.[6]\n",
    "\n",
    "The data is organized by State, Year, and MSN.\n",
    "\n",
    "The MSNs are five-character codes, most of which are structured as follows:\n",
    "First and second characters - describes an energy source (for example, NG for natural gas, MG for motor gasoline)\n",
    "Third and fourth characters - describes an energy sector or an energy activity (for example, RC for residential consumption, PR for production)\n",
    "Fifth character - describes a type of data (for example, P for data in physical unit, B for data in billion Btu and D is for dollars per million BTU)[6]\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "### Model Background:\n",
    "\n",
    "\n",
    "#### Linear Models\n",
    "\n",
    "A *regression* attempts to fit a function to observed data to make predictions on new data. A *linear regression* fits a straight line to observed data, attempting to demonstrate a linear relationship between variables and make predictions on new data yet to be observed[1].\n",
    "\n",
    "So given a set of predictors, all combined, with an error term yields a linear relationship to some response. \n",
    "\n",
    "The formula is $ y_i = \\beta_0+ \\beta_{i1}.... + \\epsilon_i$\n",
    "\n",
    "\n",
    "### Support Vector Regression\n",
    "\n",
    "The basic goal of Support Vector Regression is to find a function $f(x)$ that has at most $\\epsilon$ from actually obtained targets $y_i$ for all of the training data and at the same time is as flat as possible. [2] \n",
    "\n",
    "To achieve this we must estimate $\\beta$.\n",
    "\n",
    "$ H(\\beta,\\beta_0) = \\sum\\limits_{i=1}^{N} V(y_i - f(x_i)) + \\frac{\\lambda}{2}\\|\\beta\\|^{2}$\n",
    "\n",
    "Errors smaller than size $\\epsilon$ are ignored due to having small residual values.\n",
    "\n",
    "$V_H (r) =\\begin{cases} 0 \\text{    if    } \\|r\\| < \\epsilon  \\\\\n",
    "\t\t\\|r\\| - \\epsilon ~ otherwise\n",
    "\\end{cases}$\n",
    "\n",
    "### K-Nearest Neighbors\n",
    "\n",
    "It is a non parametric supervised learning method, input consisting of k closest training examples in a data set.The output is the average value of k nearest neighbors. [3]\\\n",
    "$f(x) = \\frac{1}{K} \\sum_{i\\epsilon N_k(`x)}$\n",
    "\n",
    "### Neural Network\n",
    "\n",
    "The Neural Network model used was a Multi-Layer Perceptron regressor. It optimizes the squared error using LBFGS or stochastic gradient descent [4]\n",
    "\n",
    "It has the function: $f(\\cdot): R^{m} \\to R^{o} $\n",
    "\n",
    "m = input dimensions \\\n",
    "o = output dimensions \n",
    "\n",
    "A basic model consists of an input layer,hidden layers which transform the values, and an output layer. [6] \n",
    "\n",
    "\n",
    "### Stochastic Gradient Descent\n",
    "Uses a linear model, that minimizes loss by updating the model with a lower learning rate as it proceeds.\n",
    "\n",
    "\n",
    "## Procedure\n",
    "\n",
    "### Data Loading cleaning and initial analysis\n",
    "All of the data was first loaded, inspected and cleaned. \n",
    "When loaded the file contains several columns: \\\n",
    "`Data_Status` - A code \\\n",
    "`MSN` - A 5 letter combination to signify energy source, sector and unit of data.\\\n",
    "Units for each category are given below.\\\n",
    "Please see :https://www.eia.gov/state/seds/CDF/Codes_and_Descriptions.xlsx for a full description of each category. \n",
    "*\tMillion Btu per short ton\n",
    "*\tBillion Btu\n",
    "*\tDollars per million Btu\n",
    "*\tThousand short tons\n",
    "*\tMillion dollars\n",
    "*\tThousand barrels\n",
    "\n",
    "`StateCode` Two letter state abbreviation \\\n",
    "`Year` YYYY-MM-DD \\\n",
    "`Data` Numerical - associated with MSN \n",
    "\n",
    "\n",
    "\n",
    "Please refer to :https://github.com/oohtmeel1/Machine-Learning-_-Prediction_Electricity_Data/blob/main/working_notebooks/Pipeline_work_B.ipynb\n",
    "\n",
    "The portion of the data that was used was specifically associated with the residential sector. \n",
    "When visualized, most of the data did not fit a distribution that would be considered normal. \n",
    "\n",
    "\n",
    "### QQ plots\n",
    "\n",
    "<img src=\"working_notebooks/qqplots1.png\">\n",
    "\n",
    "\n",
    "### Correlations \n",
    "\n",
    "The Pearson product-moment correlation coefficients were computed. And variables most strongly associated with the dependent variable were retained.\n",
    "\n",
    "\n",
    "The formula is below:\n",
    "\n",
    "$ p_{x,y} = \\rho (X,Y)= \\frac{cov((X,Y))}{\\sigma_{(X)} \\sigma_{(Y)}} $\n",
    "\n",
    "\n",
    "$cov((X),(Y))$ Is the covariance of the variables\\\n",
    "$\\sigma(X)$ Is the standard deviation of X \\\n",
    "$\\sigma(Y)$ Is the standard deviation of Y\n",
    "\n",
    "\n",
    "<img src=\"working_notebooks/corr_matrix.png\">\n",
    "\n",
    "\n",
    "### Preprocessing\n",
    "\n",
    "The categorical data, (Years and States) was masked using label encoding.The numerical data was transformed before analysis. Scikit-learn preprocessing was utilized in the form of the StandardScaler class. Which removed the mean and scaled the data to unit variance [7].\n",
    "Please refer to :https://github.com/oohtmeel1/Machine-Learning-_-Prediction_Electricity_Data/blob/main/working_notebooks/Pipeline_work_B.ipynb\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data preprocessing is here\n",
    "import pandas as pd\n",
    "import os\n",
    "from sklearn.preprocessing import StandardScaler, PolynomialFeatures\n",
    "from sklearn.utils import shuffle\n",
    "\n",
    "X1=pd.read_csv(os.getcwd()+ '/working_notebooks/X1.csv')\n",
    "X2=pd.read_csv(os.getcwd()+ '/working_notebooks/X2.csv')\n",
    "X1 = shuffle(X1)\n",
    "X2=shuffle(X2)\n",
    "y_train=X1['Data_x'].reset_index(drop=True) # Separating all of the data\n",
    "y_test=X2['Data_x'].reset_index(drop=True)\n",
    "X_train = X1.iloc[:,3: ]\n",
    "X_test = X2.iloc[:,3: ]\n",
    "scaler = StandardScaler()\n",
    "scaler.fit(X_train)\n",
    "X_train1=scaler.transform(X_train)\n",
    "X_test1=scaler.transform(X_test) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model training and testing\n",
    "\n",
    "Each model performed regression on a subset of the data, casting a prediction for the electrical expenditure amount for the year 2021 for each State.\n",
    "The results were then compared by graphing them against the true values as well as computing the Mean Absolute Error (MAE), Mean Squared Error (MSE), Explained Variance (EV) and the $R^2$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "## The linear model\n",
    "from sklearn.linear_model import LinearRegression\n",
    "import plotly.express as px\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "\n",
    "linear_model=LinearRegression().fit(X_train1,y_train)\n",
    "linear_pred= linear_model.predict(X_test1)\n",
    "y_true =y_test\n",
    "\n",
    "#px.scatter(y_pred)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "## The SVR model\n",
    "\n",
    "from sklearn.svm import SVR\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "regr = make_pipeline(SVR(C=23.0, epsilon=0.2,kernel='linear'))\n",
    "SVR_pred=regr.fit(X_train1, y_train).predict(X_test1)\n",
    "\n",
    "\n",
    "## The SGR\n",
    "from sklearn.linear_model import SGDRegressor\n",
    "reg = make_pipeline(\n",
    "                    SGDRegressor(penalty='l1'))\n",
    "gradient_pred=reg.fit(X_train1, y_train).predict(X_test1)\n",
    "\n",
    "\n",
    "#K neighbors\n",
    "from sklearn.neighbors import KNeighborsRegressor\n",
    "reg = make_pipeline(\n",
    "                    KNeighborsRegressor(weights='distance',n_neighbors=1,p=1/2))\n",
    "kneighbors_pred=reg.fit(X_train1, y_train).predict(X_test1)\n",
    "\n",
    "\n",
    "# Neural Netwok\n",
    "\n",
    "from sklearn.neural_network import MLPRegressor\n",
    "\n",
    "regr=MLPRegressor(activation='identity',max_iter=2300,solver='lbfgs').fit(X_train1, y_train)\n",
    "neural_net_pred=regr.predict(X_test1)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "rm=range(len(y_test))\n",
    "\n",
    "df=pd.DataFrame([y_test, linear_pred,SVR_pred,gradient_pred,kneighbors_pred,neural_net_pred])\n",
    "df1=df.T\n",
    "df1=df1.rename(columns={'Data_x':'y_test', 'Unnamed 0':'linear_pred','Unnamed 1':'SVR_pred',\n",
    "                        'Unnamed 2':'gradient_pred','Unnamed 3':'kneighbors_pred','Unnamed 4':'neural_net_pred'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plotly.com"
       },
       "data": [
        {
         "hovertemplate": "variable=y_test<br>prediction number=%{x}<br>Predicted amount=%{y}<extra></extra>",
         "legendgroup": "y_test",
         "line": {
          "color": "#636efa",
          "dash": "solid"
         },
         "marker": {
          "size": 20,
          "symbol": "circle"
         },
         "mode": "lines",
         "name": "y_test",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          3843.7,
          18772.3,
          1128,
          5579.6,
          2131.7,
          2612.4,
          2145.8,
          6789.2,
          6897,
          944.6,
          7339.6,
          616.5,
          1343.2,
          4741.6,
          4921.1,
          6289.8,
          4476.5,
          2696.1,
          7697.6,
          698.4,
          959.2,
          6167.8,
          469.9,
          418.7,
          1864.6,
          958.1,
          946.2,
          1651.8,
          15518.2,
          3040.8,
          1787.5,
          4037.5,
          4071.5,
          3138.3,
          530.3,
          1142.2,
          4647.9,
          330.9,
          4655.9,
          20604.4,
          4092.4,
          3320.2,
          861.6,
          323.7,
          623.9,
          3669.2,
          2307.4,
          2868.4,
          647.1,
          3351.6,
          10161.8
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "variable=linear_pred<br>prediction number=%{x}<br>Predicted amount=%{y}<extra></extra>",
         "legendgroup": "linear_pred",
         "line": {
          "color": "#EF553B",
          "dash": "solid"
         },
         "marker": {
          "size": 20,
          "symbol": "circle"
         },
         "mode": "lines",
         "name": "linear_pred",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          3843.6966388937517,
          18772.29150561045,
          1128.0040497935656,
          5579.491561219842,
          2131.7007871745536,
          2612.5021920518975,
          2145.7994067698874,
          6789.1924403747635,
          6897.091359695185,
          944.7028917228316,
          7339.598552150479,
          616.5003208172175,
          1343.2997891802545,
          4741.601800934129,
          4921.103286145846,
          6289.791683914735,
          4476.50233400955,
          2696.103994131338,
          7697.58642533869,
          698.401181437664,
          959.2931681105194,
          6167.806961984146,
          469.9017283567134,
          418.7993443886178,
          1864.6006841875624,
          958.0977177634153,
          946.3005531291088,
          1651.8041632285788,
          15518.192169861335,
          3040.7970133197514,
          1787.5006024975705,
          4037.39929598577,
          4071.5000307556015,
          3138.30037592665,
          530.4054761959874,
          1142.206993216676,
          4647.891493262729,
          330.90535916724866,
          4655.906644978721,
          20604.426680767738,
          4092.4001727708437,
          3320.296697587678,
          861.5979707555294,
          323.6980596841747,
          624.0018523648612,
          3669.1991548644,
          2307.3015643064828,
          2868.3947529324305,
          647.103585946953,
          3351.700041970586,
          10161.691457867584
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "variable=SVR_pred<br>prediction number=%{x}<br>Predicted amount=%{y}<extra></extra>",
         "legendgroup": "SVR_pred",
         "line": {
          "color": "#00cc96",
          "dash": "solid"
         },
         "marker": {
          "size": 20,
          "symbol": "circle"
         },
         "mode": "lines",
         "name": "SVR_pred",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          3843.1770927056245,
          18768.28757005001,
          1128.1857201214966,
          5578.823867316092,
          2131.558753566552,
          2612.037482242438,
          2145.2725627095574,
          6788.739608633416,
          6896.410334127395,
          944.7731256796615,
          7337.881464898521,
          616.6230305909512,
          1343.1007115543878,
          4740.793036904841,
          4921.170983888325,
          6289.483446500717,
          4476.697887173241,
          2696.500358223696,
          7697.1075868791395,
          698.4707168511945,
          959.6272756038145,
          6168.630571074358,
          469.96701943640005,
          419.0084551010325,
          1864.8366436289525,
          958.3034707560245,
          946.6415689154713,
          1652.2808490230366,
          15517.035765537596,
          3040.8468055616904,
          1787.4601822466677,
          4036.802209963918,
          4071.345557153249,
          3138.484196159742,
          530.6688498692592,
          1142.6023323240656,
          4647.942573812859,
          331.06594795197657,
          4656.704387096955,
          20606.749426187027,
          4091.110091894865,
          3320.1046207359914,
          861.9128012209503,
          323.83779483451485,
          624.1742664221422,
          3669.441982692455,
          2306.8933745120044,
          2868.5559637602005,
          647.352691041672,
          3351.0800534608506,
          10161.474230422422
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "variable=gradient_pred<br>prediction number=%{x}<br>Predicted amount=%{y}<extra></extra>",
         "legendgroup": "gradient_pred",
         "line": {
          "color": "#ab63fa",
          "dash": "solid"
         },
         "marker": {
          "size": 20,
          "symbol": "circle"
         },
         "mode": "lines",
         "name": "gradient_pred",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          3844.3768160556147,
          18770.286484146858,
          1127.4251997723754,
          5577.374813705173,
          2131.3539646809268,
          2611.514190010793,
          2144.471418741092,
          6787.421382359878,
          6895.188432204663,
          944.8236757397598,
          7338.521148259317,
          615.7338996428007,
          1342.2079499120136,
          4741.1060842581655,
          4922.857036169159,
          6287.486627253964,
          4476.465800766149,
          2696.293017082912,
          7698.589791903927,
          698.0003810639237,
          958.3152765124545,
          6169.210872238296,
          469.6939642381726,
          418.2514492539947,
          1865.112352786358,
          957.7337642385427,
          945.7120535928798,
          1650.8298099716353,
          15515.181064326951,
          3040.509119473997,
          1786.5942784951228,
          4036.319473837301,
          4070.443160892225,
          3136.306282341003,
          529.9611888902552,
          1141.686743213319,
          4649.248543144362,
          330.3308434793644,
          4655.102508611377,
          20605.170006889966,
          4091.196233911429,
          3317.7820288499024,
          861.6351043866224,
          323.0246112355492,
          624.0103295317638,
          3668.004961065635,
          2308.0894588326087,
          2866.960611224627,
          646.3650567577981,
          3350.9518132061385,
          10161.588779041766
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "variable=kneighbors_pred<br>prediction number=%{x}<br>Predicted amount=%{y}<extra></extra>",
         "legendgroup": "kneighbors_pred",
         "line": {
          "color": "#FFA15A",
          "dash": "solid"
         },
         "marker": {
          "size": 20,
          "symbol": "circle"
         },
         "mode": "lines",
         "name": "kneighbors_pred",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          3600.6,
          18322,
          1135.3,
          5632.1,
          1888.7,
          2484,
          2146.5,
          6464.5,
          6834,
          860.0000000000001,
          6979.1,
          595.5,
          1254.6,
          4751.7,
          4756.8,
          5078,
          4182.9,
          2531.3,
          7512.000000000001,
          648.3,
          853,
          6022,
          471.5,
          357.6,
          1817.1000000000001,
          942,
          862.8,
          1624.4,
          15021.4,
          2869.1,
          1747.1,
          3962.9,
          3922.6,
          3001.4000000000005,
          526.1,
          1100.7,
          4383.4,
          330.7,
          4751,
          19413.3,
          3939.9,
          3271.8000000000006,
          740.6999999999999,
          315.3,
          604.8,
          3611.3,
          2123.9,
          2732.7,
          628,
          3035.1,
          8995.4
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "variable=neural_net_pred<br>prediction number=%{x}<br>Predicted amount=%{y}<extra></extra>",
         "legendgroup": "neural_net_pred",
         "line": {
          "color": "#19d3f3",
          "dash": "solid"
         },
         "marker": {
          "size": 20,
          "symbol": "circle"
         },
         "mode": "lines",
         "name": "neural_net_pred",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50
         ],
         "xaxis": "x",
         "y": [
          3843.704046920696,
          18772.29630588958,
          1128.001378923316,
          5579.497396930531,
          2131.702094485879,
          2612.4965291864087,
          2145.7961777852183,
          6789.205179475788,
          6897.095005641823,
          944.6964352537915,
          7339.600596242667,
          616.5034541297221,
          1343.3025012858013,
          4741.595553653809,
          4921.106304479322,
          6289.801785999241,
          4476.496718402873,
          2696.1008485353054,
          7697.592793182671,
          698.3986679414294,
          959.3028238049168,
          6167.813361034062,
          469.9005022898152,
          418.79396921325963,
          1864.6039084436466,
          958.106629079371,
          946.3083475028462,
          1651.801489635469,
          15518.18661828175,
          3040.80753876692,
          1787.504438440295,
          4037.400326983257,
          4071.5041453550225,
          3138.293823394961,
          530.397193375347,
          1142.1988672993514,
          4647.898999773472,
          330.89769717128144,
          4655.9044153120785,
          20604.416220472976,
          4092.392404243364,
          3320.2966042931052,
          861.5908183004568,
          323.70892007960623,
          623.9993910109434,
          3669.20052273796,
          2307.2977157357104,
          2868.385719685841,
          647.0971091586176,
          3351.6999769271833,
          10161.680978465594
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "title": {
          "text": "variable"
         },
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Comparisons of predictions for each model as well as the true values"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "prediction number"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "Predicted amount"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import plotly.express as px\n",
    "\n",
    "fig = px.line(df1,title=\"Comparisons of predictions for each model as well as the true values\",labels={'value':\"Predicted amount\",'index':'prediction number'})\n",
    "fig.update_traces({'marker':{'size': 20}})\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plotly.com"
       },
       "data": [
        {
         "hovertemplate": "variable=Number 14<br>index=%{x}<br>value=%{y}<extra></extra>",
         "legendgroup": "Number 14",
         "marker": {
          "color": "#636efa",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Number 14",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16
         ],
         "xaxis": "x",
         "y": [
          4.717091308691414,
          1.1568333283804952,
          0.6218627605196738,
          0.67441002203477,
          0.514069977470177,
          1.9001708573737477,
          1.8999152326368627,
          1.86897232145977,
          -0.005143327317131815,
          0.08814386871400705,
          0.498581380983818,
          0.9195782125097692,
          0.3561871272310465,
          0.3460751425776299,
          0.42461652279085105,
          -0.6793662204867574,
          1.7663521732655694
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "variable=Number 15<br>index=%{x}<br>value=%{y}<extra></extra>",
         "legendgroup": "Number 15",
         "marker": {
          "color": "#EF553B",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Number 15",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16
         ],
         "xaxis": "x",
         "y": [
          5.363056887886657,
          4.402763121326301,
          0.6508409989708032,
          2.1227866825614696,
          1.979017466120879,
          6.051442951755504,
          6.051803105034501,
          5.570269587083347,
          0.06853580549941972,
          0.10313325066775351,
          1.169955293086981,
          1.968739987704806,
          1.435192922149847,
          1.4043805890573038,
          1.8003208541453282,
          -0.20380986614602725,
          1.7663521732655694
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "variable=Number 16<br>index=%{x}<br>value=%{y}<extra></extra>",
         "legendgroup": "Number 16",
         "marker": {
          "color": "#00cc96",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Number 16",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16
         ],
         "xaxis": "x",
         "y": [
          0.27194984858509125,
          0.1456527303311228,
          0.27221803516309556,
          1.7695516399721978,
          1.5241964926694653,
          0.3892553858407477,
          0.38935311689967644,
          0.7159344490000794,
          2.0661388308379243,
          3.3075738436042013,
          0.6499698179822279,
          1.4195448867924871,
          0.8635752642254069,
          0.8646871539564509,
          -0.3509030757668198,
          0.4076197322920545,
          1.7663521732655694
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "title": {
          "text": "variable"
         },
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Comparisons of a few points to variable 15 data"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "index"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "value"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "a=X_test1[16]\n",
    "b=X_test1[15]\n",
    "c=X_test1[14]\n",
    "df3=pd.DataFrame([a,b,c]).T\n",
    "df3=df3.rename(columns={0:'Number 14',1:'Number 15',2:'Number 16'})\n",
    "px.scatter(df3,title=\"Comparisons of a few points to variable 15 data\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "### Results and discussion\n",
    "\n",
    "Most of the models performed very well after optimization.\\\n",
    "The plot above illustrates the closeness of all of the predictions between the models, and shows the specific points where there were differences. \n",
    "Prediction number 15 shows some differences, with the kneighbors regressor performing slightly worse.This might be due to that specific data point being somewhat of an outlier. But due to the nature of the data, only one very extreme outlier was actually removed. (The sum of all of the data, which was the US column) Kneighbors are known to be sensitive to highly dimensional data and to outliers. All of the other models had prediction accuracies of 100%.\n",
    "\n",
    "\n",
    "For all of the models, the true values were plotted on a bar graph, and the predictions were plotted as a scatterplot to help illustrate.\n",
    "\n",
    "The Linear Regression model had a prediction accuracy of 100%.\\ \n",
    "https://github.com/oohtmeel1/Machine-Learning-_-Prediction_Electricity_Data/blob/main/Linear_models/Machine_learning_Linear_modeling.ipynb\n",
    "\n",
    "\n",
    "<img src=\"working_notebooks/linear_model_plot.png\">\n",
    "\n",
    "The `MAE` was 0.03267136765242417\\\n",
    "The `MSE` was 0.003015192149111796\\\n",
    "The `Explained variance` was 0.999999999844917\\\n",
    "The $ R^2 $ was 0.9999999998350794\n",
    "\n",
    "\n",
    "The SVR also had a prediction accuracy of 100%.\\\n",
    "https://github.com/oohtmeel1/Machine-Learning-_-Prediction_Electricity_Data/tree/main/SVR\n",
    "\n",
    "<img src=\"working_notebooks/svr_plot.png\">\n",
    "\n",
    "The `MAE` was 0.5021485094357804\\\n",
    "The `MSE` was 0.6832135549229309\\\n",
    "The `Explained variance` was 0.9999999633196209\\\n",
    "The $ R^2 $ was 0.999999962630569\n",
    "\n",
    "\n",
    "\n",
    "The k_neighbors performed decently. With a prediction accuracy of 80%.\n",
    "https://github.com/oohtmeel1/Machine_Learning_Prediction_Kneighborsreg/blob/main/Kneighbors_reg.ipynb\n",
    "\n",
    "<img src=\"working_notebooks/kneighbors_plot.png\">\n",
    "\n",
    "The `MAE` was 186.8901960784314\\\n",
    "The `MSE` was 111144.4768627452\\\n",
    "The `Explained variance` was 0.9956994403253784\\\n",
    "The $ R^2 $ was 0.9939207794879534\n",
    "\n",
    "\n",
    "The SGDR performed very well. With a prediction accuracy of 100%.\\\n",
    "https://github.com/oohtmeel1/Machine_Learning_Prediction_Stochastic_Gradient/blob/main/Gradient_Descent_regression.ipynb\n",
    "<img src=\"working_notebooks/sgdr_plot.png\">\n",
    "\n",
    "The `MAE` was 0.9834673843854801\\\n",
    "The `MSE` was 1.5266552927792496\\\n",
    "The `Explained variance` was 0.9999999450342941\\\n",
    "The $ R^2 $ was 0.9999999164972077\n",
    "\n",
    "\n",
    "The Neural Network Model performed very well. With a prediction accuracy of 100%.\n",
    "https://github.com/oohtmeel1/Machine_Learning_Prediction_Neural_Network_Electricity_Data\n",
    "\n",
    "<img src=\"working_notebooks/neural_net_plot.png\">\n",
    "\n",
    "The `MAE` was 0.032168487841461764\\\n",
    "The `MSE` was 0.0030038938705200698\\\n",
    "The `Explained variance` was 0.9999999998453984\\\n",
    "The $ R^2 $ was 0.9999999998356973\n",
    "\n",
    "\n",
    "\n",
    "# Citations and references\n",
    "\n",
    "*Essential Math for Data Science* by Thomas Nield (O'Reilly). Copyright 2022 Thomas Nield, 978-1-098-10293-7 [1]\n",
    "\n",
    "*A tutorial on support vector regression* by ALEX J. SMOLA and BERNHARD SCHOLKOPF,Received July 2002 and accepted November 2003 https://web.archive.org/web/20120131193522/http://eprints.pascal-network.org/archive/00000856/01/fulltext.pdf [2]\n",
    "\n",
    "https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm[3]\n",
    "\n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html#sklearn.neural_network.MLPRegressor[4]\n",
    "\n",
    "https://scikit-learn.org/stable/modules/neural_networks_supervised.html[5]\n",
    "\n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html[7]\n",
    "\n",
    "Useful Links for data:\n",
    "\n",
    "The main website: https://catalog.data.gov/dataset/state-energy-data-system-seds[6]\n",
    "\n",
    "Additional information: https://www.eia.gov/state/seds/\n",
    "\n",
    "Codes and descriptions: https://www.eia.gov/state/seds/CDF/Codes_and_Descriptions.xlsx\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}