Evaluation of a Regression Model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluation of the Model\n",
    "\n",
    "### Three Ways to Evaluate Scikit Learn Model / Estimator.\n",
    "\n",
    "**A.Estimator Score Method**\n",
    "\n",
    "**B.The Scoring Parameter**\n",
    "\n",
    "**C.Problem Specific Metric Function**\n",
    "\n",
    "## Evaluation of a Regression Model\n",
    "\n",
    "### Regression Model Evaluation Metrics\n",
    "\n",
    "1. R<sup>2</sup> : `R Squared` or Coefficient of Determination\n",
    "\n",
    "2. MAE : Mean `Absolute` Error\n",
    "\n",
    "3. MSE : Mean `Squared` Error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Import Libraries**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Import Data Set**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "\n",
    "Data = load_boston()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Convert Data into Data Frame**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "Boston = pd.DataFrame(data=Data.data, columns=Data.feature_names)\n",
    "\n",
    "Boston['Target'] = Data.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00632</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.538</td>\n",
       "      <td>6.575</td>\n",
       "      <td>65.2</td>\n",
       "      <td>4.0900</td>\n",
       "      <td>1.0</td>\n",
       "      <td>296.0</td>\n",
       "      <td>15.3</td>\n",
       "      <td>396.90</td>\n",
       "      <td>4.98</td>\n",
       "      <td>24.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.02731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>6.421</td>\n",
       "      <td>78.9</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.14</td>\n",
       "      <td>21.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.02729</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>7.185</td>\n",
       "      <td>61.1</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>392.83</td>\n",
       "      <td>4.03</td>\n",
       "      <td>34.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.03237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>6.998</td>\n",
       "      <td>45.8</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>394.63</td>\n",
       "      <td>2.94</td>\n",
       "      <td>33.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.06905</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>7.147</td>\n",
       "      <td>54.2</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.33</td>\n",
       "      <td>36.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD    TAX  \\\n",
       "0  0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n",
       "1  0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n",
       "2  0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n",
       "3  0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n",
       "4  0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n",
       "\n",
       "   PTRATIO       B  LSTAT  Target  \n",
       "0     15.3  396.90   4.98    24.0  \n",
       "1     17.8  396.90   9.14    21.6  \n",
       "2     17.8  392.83   4.03    34.7  \n",
       "3     18.7  394.63   2.94    33.4  \n",
       "4     18.7  396.90   5.33    36.2  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Boston.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Splitting the Data into Independent Features and Dependent Variables Labels**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = Boston.drop('Target', axis = 'columns')\n",
    "\n",
    "Y = Boston['Target']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Splitting Data into Training Set and Testing Set**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_Train, X_Test, Y_Train, Y_Test = train_test_split(X, Y, test_size=0.3, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**X_Train** : The Set of Data to be Trained.\n",
    "\n",
    "**Y_Train** : The Set of Data consists of Class - Labels of Corresponding X_Train Data.\n",
    "\n",
    "**X_Test** : The Set of Data on which the Trained Model is to Tested to Check the Accuracy of the Model.\n",
    "\n",
    "**Y_Test : The Set of Data Labels which is to be Compared with the Result of Trained Model to Chec Accuracy as well as Evaluation Metrics.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "70% of Train Set : (354, 13) and Labels of Train Set (354,)\n",
      "70% of Train Set : (152, 13) and Labels of Test Set (152,)\n"
     ]
    }
   ],
   "source": [
    "print(f'70% of Train Set : {X_Train.shape} and Labels of Train Set {Y_Train.shape}')\n",
    "\n",
    "print(f'70% of Train Set : {X_Test.shape} and Labels of Test Set {Y_Test.shape}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### According to Scikit Learn Algorithm Map\n",
    "\n",
    "**Ridge Regression**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import Ridge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Instantiate the Ridge Model Object**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "Ridge_Model = Ridge()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fit the Training Data to the Ridge Model to find Pattern**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ridge()"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Ridge_Model.fit(X_Train, Y_Train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.Estimator Score Method :\n",
    "\n",
    "**Check the R Square of Model on Training Data and Testing Data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score (R^2) of Model on Training Data : 74.16%\n",
      "\n",
      "Score (R^2) of Model on Testing Data :70.42%\n"
     ]
    }
   ],
   "source": [
    "print(f'Score (R^2) of Model on Training Data : {Ridge_Model.score(X_Train, Y_Train)*100:.2f}%')\n",
    "\n",
    "print(f'\\nScore (R^2) of Model on Testing Data :{Ridge_Model.score(X_Test, Y_Test)*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to Improve the Score ?\n",
    "\n",
    "**According to Scikit Learn Algorithm Map let's Try Randm Forest Regressor**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Import the Random Forest Regressor**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Instantiate the Random Forest Regressor Object**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "RFR_Model = RandomForestRegressor(n_estimators=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fit the Model with the Data to Find the Patterns**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor()"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RFR_Model.fit(X_Train, Y_Train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Check the R^2 of Model on Training and Testing Data Set**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R^2 of Random Forest Regressor on Train Set : 97.88%\n",
      "\n",
      "R^2 of Random Forest Regressor on Test Set : 86.76%\n"
     ]
    }
   ],
   "source": [
    "print(f'R^2 of Random Forest Regressor on Train Set : {RFR_Model.score(X_Train, Y_Train)*100:.2f}%')\n",
    "\n",
    "print(f'\\nR^2 of Random Forest Regressor on Test Set : {RFR_Model.score(X_Test, Y_Test)*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**R^2 of Random Forest Regressor is better than Ridge**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Prediction of Model on X_Test Data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "RFR_Prediction = RFR_Model.predict(X_Test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**To Improve the Accuracy further, we will Check and Improve Evaluation Metrics of Regression Model**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluation of Random Forest Regressor \n",
    "\n",
    "**[Metrics and Scoring : Quantifing the Quality of Predictions](https://scikit-learn.org/stable/modules/model_evaluation.html)**\n",
    "\n",
    "**Random Forest Regressor Evaluation Metrics.**\n",
    "\n",
    "**1. R^2 : R Square or Coeffiecient of Determination**\n",
    "\n",
    "**R^2 Compares the Predictions of Model with the Mean of the Target Variables**\n",
    "\n",
    "**R^2 is similar to Accuracy, Quick Indication of How well the Model might be.**\n",
    "\n",
    "**But it do not tell Exactly how wrong your Model is and How much the difference is between the Predictions and Actual values.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R^2 of Random Forest Regressor on Train Set : 97.88%\n",
      "\n",
      "R^2 of Random Forest Regressor on Test Set : 86.76%\n"
     ]
    }
   ],
   "source": [
    "print(f'R^2 of Random Forest Regressor on Train Set : {RFR_Model.score(X_Train, Y_Train)*100:.2f}%')\n",
    "\n",
    "print(f'\\nR^2 of Random Forest Regressor on Test Set : {RFR_Model.score(X_Test, Y_Test)*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2.MAE : Mean Absolute Error**\n",
    "\n",
    "**The Average of the Absolute Difference between the Predictions and the Actual Values**\n",
    "\n",
    "**It gives us Idea of How Wrong our Model Predictions are.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_absolute_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error : 2.1\n"
     ]
    }
   ],
   "source": [
    "print(f'Mean Absolute Error : {round(mean_absolute_error(Y_Test,RFR_Prediction),2)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**To Understand it Better we will Compare the Actual Value and the predicted Value by the Model in a Data Frame.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Actual Value  Predicted Value  Absolute Error\n",
      "173          23.6           22.437           1.163\n",
      "274          32.4           31.544           0.856\n",
      "491          13.6           16.270           2.670\n",
      "72           22.8           23.873           1.073\n",
      "452          16.1           17.804           1.704\n",
      "\n",
      "Mean Absolute Error of the Predictions made by the Model : 2.1\n"
     ]
    }
   ],
   "source": [
    "MAE = pd.DataFrame({'Actual Value' : Y_Test,\n",
    "                    'Predicted Value' : RFR_Prediction})\n",
    "\n",
    "MAE['Absolute Error'] = abs(MAE['Predicted Value'] - MAE['Actual Value'])\n",
    "\n",
    "print(MAE[:5])\n",
    "\n",
    "print('\\nMean Absolute Error of the Predictions made by the Model :',round(MAE['Absolute Error'].mean(),2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3.MSE : Mean Squared Error**\n",
    "\n",
    "**The Average of the Squared Difference between the Predictions and the Actual Values**\n",
    "\n",
    "**Squaring the Error Amplifies the Large Difference**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_squared_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Squared Error : 9.87\n",
      "\n",
      "Root Mean Squared Error : 3.14\n"
     ]
    }
   ],
   "source": [
    "print(f'Mean Squared Error : {round(mean_squared_error(Y_Test,RFR_Prediction),2)}')\n",
    "\n",
    "print(f'\\nRoot Mean Squared Error : {round(np.sqrt(mean_squared_error(Y_Test, RFR_Prediction)),2)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**To Understand it better we will compare the Actual Values and the Predictions**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Actual Values  Predicted Values  Squared Error\n",
      "173           23.6            22.437       1.352569\n",
      "274           32.4            31.544       0.732736\n",
      "491           13.6            16.270       7.128900\n",
      "72            22.8            23.873       1.151329\n",
      "452           16.1            17.804       2.903616\n",
      "\n",
      "Mean of Squared Error : 9.87\n"
     ]
    }
   ],
   "source": [
    "MSE = pd.DataFrame({'Actual Values' : Y_Test,\n",
    "                    'Predicted Values' : RFR_Prediction})\n",
    "\n",
    "MSE['Squared Error'] = pow(MSE['Predicted Values'] - MSE['Actual Values'],2)\n",
    "\n",
    "print(MSE[:5])\n",
    "\n",
    "print('\\nMean of Squared Error :',round(MSE['Squared Error'].mean(),2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**For Better Model, Our Aim should be to Maximize R^2 and Minimize the MAE and MSE**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.Scoring Parameter :\n",
    "\n",
    "**Model Evaluation Tools using Cross Value Score or Grid Search CV rely on Scoring Parameters.**\n",
    "\n",
    "### Cross Validation Score for Regression Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "np.random.seed(42)\n",
    "\n",
    "CVS = cross_val_score(RFR_Model, X, Y, cv=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**5 Fold Cross Validation Score for Random Forest Regressor**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.77231143 0.86035935 0.74664002 0.47632078 0.26630379]\n"
     ]
    }
   ],
   "source": [
    "print(CVS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Cross Validation Mean Score of Random Forest Regressor**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross Validation Mean Score : 62.44%\n"
     ]
    }
   ],
   "source": [
    "print(f'Cross Validation Mean Score : {np.mean(CVS)*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Default Scoring Parameter = None (R^2)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross Validation Mean Score : 62.44%\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "\n",
    "CVS_R2 = cross_val_score(RFR_Model, X, Y, cv=5, scoring='r2')\n",
    "\n",
    "print(f'Cross Validation Mean Score : {np.mean(CVS_R2)*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Scoring Parameter = MAE**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross Validation Mean Score : [-2.11419608 -2.58716832 -3.33976238 -3.78563366 -3.32941584]\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "\n",
    "CVS_MAE = cross_val_score(RFR_Model, X, Y, cv = 5, scoring='neg_mean_absolute_error')\n",
    "\n",
    "print(f'Cross Validation Mean Score : {CVS_MAE}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Scoring Parameter = MSE**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross Validation Mean Score : [ -7.86326927 -12.71243178 -20.29089194 -45.93287403 -19.50774726]\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "\n",
    "CVS_MSE = cross_val_score(RFR_Model, X, Y, cv = 5, scoring='neg_mean_squared_error')\n",
    "\n",
    "print(f'Cross Validation Mean Score : {CVS_MSE}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C. Problem Specific Metric Function\n",
    "\n",
    "**Regression Evaluation Functions**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import r2_score, mean_absolute_error, mean_squared_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R^2 Score : 86.76%\n",
      "\n",
      "Mean Absolute Error : 2.1\n",
      "\n",
      "Mean Squared Error : 9.87\n",
      "\n",
      "Root Mean Squared Error : 3.14\n"
     ]
    }
   ],
   "source": [
    "print(f'R^2 Score : {r2_score(Y_Test, RFR_Prediction)*100:.2f}%')\n",
    "\n",
    "print(f'\\nMean Absolute Error : {round(mean_absolute_error(Y_Test, RFR_Prediction),2)}')\n",
    "\n",
    "print(f'\\nMean Squared Error : {round(mean_squared_error(Y_Test, RFR_Prediction),2)}')\n",
    "\n",
    "print(f'\\nRoot Mean Squared Error : {round(np.sqrt(mean_squared_error(Y_Test, RFR_Prediction)),2)}')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}