MLT-1/MLT1_TitanicClassification-Solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "79a4781a-aa54-4209-a2ac-d3a081d7b479",
   "metadata": {},
   "source": [
    "<img src=\"https://cellstrat2.s3.amazonaws.com/PlatformAssets/bluewhitelogo.svg\" alt=\"drawing\" width=\"200\"/>\n",
    "\n",
    "# ML Tuesdays - Session 1\n",
    "## Titanic Survival Classification Exercise\n",
    "\n",
    "### General Guidelines\n",
    "1. The notebook has been split into multiple steps with fine-grained instructions for each step. Use the instructions for each code cell to complete the code.\n",
    "2. Some hints specify the function to be used but you need to figure out the arguments yourself in some using the docstring of the function which can be pulled up with `Shift+Tab`.\n",
    "\n",
    "### About the Titanic Dataset\n",
    "The sinking of the Titanic is one of the most infamous shipwrecks in history.\n",
    "\n",
    "On April 15, 1912, during her maiden voyage, the widely considered “unsinkable” RMS Titanic sank after colliding with an iceberg. Unfortunately, there weren’t enough lifeboats for everyone onboard, resulting in the death of 1502 out of 2224 passengers and crew.\n",
    "\n",
    "While there was some element of luck involved in surviving, it seems some groups of people were more likely to survive than others.\n",
    "\n",
    "In this challenge, we ask you to build a predictive model that answers the question: “what sorts of people were more likely to survive?” using passenger data (ie name, age, gender, socio-economic class, etc).\n",
    "\n",
    "Here's a description of the columns in the dataset:\n",
    "\n",
    "| Variable | Definition                                 | Key                                            |\n",
    "| -------- | ------------------------------------------ | ---------------------------------------------- |\n",
    "| survival | Survival                                   | 0 = No, 1 = Yes                                |\n",
    "| pclass   | Ticket class                               | 1 = 1st, 2 = 2nd, 3 = 3rd                      |\n",
    "| sex      | Sex                                        |\n",
    "| Age      | Age in years                               |\n",
    "| sibsp    | # of siblings / spouses aboard the Titanic |\n",
    "| parch    | # of parents / children aboard the Titanic |\n",
    "| ticket   | Ticket number                              |\n",
    "| fare     | Passenger fare                             |\n",
    "| cabin    | Cabin number                               |\n",
    "| embarked | Port of Embarkation                        | C = Cherbourg, Q = Queenstown, S = Southampton |\n",
    "[Dataset Source](https://www.kaggle.com/c/titanic/overview)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "id": "8789eed4-8cac-4206-a87a-212a89851cbb",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a35343fc-c2c1-4718-81e3-6d36978f0204",
   "metadata": {},
   "source": [
    "## Data Preprocessing\n",
    "\n",
    "1. Load Data\n",
    "2. Remove Irrelevant Columns\n",
    "3. Split to X and y data\n",
    "4. Handling Missing Values\n",
    "5. Encode the Categorical Variables\n",
    "6. Perform Train Test Split\n",
    "7. Feature Scaling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9270f5d-49c9-4d67-831d-bc18349533a4",
   "metadata": {},
   "source": [
    "### Load Data\n",
    "Read the dataset with `index_col` as \"PassengerId\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "id": "277f0ce2-905b-438b-96ad-f231fc5570fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = pd.read_csv(\"titanic.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "id": "49e3c1b0-8733-40a0-a763-7109d5b664db",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>887</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Montvila, Rev. Juozas</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>211536</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>888</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Graham, Miss. Margaret Edith</td>\n",
       "      <td>female</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112053</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>B42</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnston, Miss. Catherine Helen \"Carrie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>W./C. 6607</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>890</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Behr, Mr. Karl Howell</td>\n",
       "      <td>male</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>111369</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>C148</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>891</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Dooley, Mr. Patrick</td>\n",
       "      <td>male</td>\n",
       "      <td>32.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>370376</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>891 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass  \\\n",
       "0              1         0       3   \n",
       "1              2         1       1   \n",
       "2              3         1       3   \n",
       "3              4         1       1   \n",
       "4              5         0       3   \n",
       "..           ...       ...     ...   \n",
       "886          887         0       2   \n",
       "887          888         1       1   \n",
       "888          889         0       3   \n",
       "889          890         1       1   \n",
       "890          891         0       3   \n",
       "\n",
       "                                                  Name     Sex   Age  SibSp  \\\n",
       "0                              Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1    Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                               Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3         Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                             Allen, Mr. William Henry    male  35.0      0   \n",
       "..                                                 ...     ...   ...    ...   \n",
       "886                              Montvila, Rev. Juozas    male  27.0      0   \n",
       "887                       Graham, Miss. Margaret Edith  female  19.0      0   \n",
       "888           Johnston, Miss. Catherine Helen \"Carrie\"  female   NaN      1   \n",
       "889                              Behr, Mr. Karl Howell    male  26.0      0   \n",
       "890                                Dooley, Mr. Patrick    male  32.0      0   \n",
       "\n",
       "     Parch            Ticket     Fare Cabin Embarked  \n",
       "0        0         A/5 21171   7.2500   NaN        S  \n",
       "1        0          PC 17599  71.2833   C85        C  \n",
       "2        0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3        0            113803  53.1000  C123        S  \n",
       "4        0            373450   8.0500   NaN        S  \n",
       "..     ...               ...      ...   ...      ...  \n",
       "886      0            211536  13.0000   NaN        S  \n",
       "887      0            112053  30.0000   B42        S  \n",
       "888      2        W./C. 6607  23.4500   NaN        S  \n",
       "889      0            111369  30.0000  C148        C  \n",
       "890      0            370376   7.7500   NaN        Q  \n",
       "\n",
       "[891 rows x 12 columns]"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15c6b9bc-1c60-4538-bced-fb01ca05dfde",
   "metadata": {},
   "source": [
    "### Remove Irrelevant Columns\n",
    "Look for columns which are either subjective or not relevant for predicting the survivability and then remove them using the `drop()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "id": "1863dce8-4c44-4898-a6c4-b024c323ea44",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.drop(columns=['PassengerId', 'Name', 'Ticket', 'Fare', 'Cabin'], inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d95dbe02-9d9b-4be9-b494-ba2fa080fbb0",
   "metadata": {},
   "source": [
    "### Handle Missing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06afa07b-f71b-4f40-9adc-8d6d5a307848",
   "metadata": {},
   "source": [
    "Check the number of NaN values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "id": "3bd18168-d2cd-483c-bdc3-404da696ac69",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived      0\n",
       "Pclass        0\n",
       "Sex           0\n",
       "Age         177\n",
       "SibSp         0\n",
       "Parch         0\n",
       "Embarked      2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dbd1ff4-7762-440f-9d92-b6b83c59c3c8",
   "metadata": {},
   "source": [
    "Drop any rows where the number of NaNs in a column is insignificant. Use the `subset` argument to specify a list of columns according to which the rows should be dropped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "id": "43c57b9d-b70c-4a24-94de-d0520bb526dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.dropna(subset=['Embarked'], inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "id": "24d43e5e-e32e-4769-8531-76bf41a1d2f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived      0\n",
       "Pclass        0\n",
       "Sex           0\n",
       "Age         177\n",
       "SibSp         0\n",
       "Parch         0\n",
       "Embarked      0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "id": "6e60d7fe-c14d-4d6c-8d49-f8054cdcccaf",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.impute import SimpleImputer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "id": "1d8460f9-dac8-49d2-afa2-c7c8ed27872e",
   "metadata": {},
   "outputs": [],
   "source": [
    "age_imputer = SimpleImputer(missing_values=np.nan, strategy='mean')\n",
    "dataset['Age'] = age_imputer.fit_transform(dataset['Age'].values.reshape(-1, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "id": "38af435a-b9f9-4076-807a-6e31ab22bd3b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived    0\n",
       "Pclass      0\n",
       "Sex         0\n",
       "Age         0\n",
       "SibSp       0\n",
       "Parch       0\n",
       "Embarked    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72fbcbfc-27d3-4fb3-a51d-0eefd37a726d",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Split into X and Y Data\n",
    "Separate the features from the labels in two variables `X_data` and `y_data`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "id": "9f1209fc-fa26-4600-852b-fc805728bea4",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_data = dataset.iloc[:, 1:]\n",
    "y_data = dataset.iloc[:, 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "id": "7fab1202-281c-4352-90a2-ab4f286394e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(889, 6)"
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "id": "80f2fd15-01de-477e-9955-530d045cf05f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(889,)"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bfcabf7-0d42-4fe8-847c-0f86e19df756",
   "metadata": {},
   "source": [
    "### Encode Categorical Features\n",
    "\n",
    "LabelEncode the binary features and OneHotEncode the ones that have multiple categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "id": "33fb54c4-b62f-4fb8-87b4-a97056822bbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "id": "170375c7-df64-43c8-b912-777fb6a2af78",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>29.642093</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>889 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Pclass     Sex        Age  SibSp  Parch Embarked\n",
       "0         3    male  22.000000      1      0        S\n",
       "1         1  female  38.000000      1      0        C\n",
       "2         3  female  26.000000      0      0        S\n",
       "3         1  female  35.000000      1      0        S\n",
       "4         3    male  35.000000      0      0        S\n",
       "..      ...     ...        ...    ...    ...      ...\n",
       "886       2    male  27.000000      0      0        S\n",
       "887       1  female  19.000000      0      0        S\n",
       "888       3  female  29.642093      1      2        S\n",
       "889       1    male  26.000000      0      0        C\n",
       "890       3    male  32.000000      0      0        Q\n",
       "\n",
       "[889 rows x 6 columns]"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "id": "1e0a87bf-943e-46da-900a-aee8904adb5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "gender_encoder = LabelEncoder()\n",
    "embarked_encoder = LabelEncoder()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "id": "53b53c2e-1551-4fcb-b811-16bdf144398c",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_data.iloc[:, 1] = gender_encoder.fit_transform(X_data.iloc[:, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "id": "f50d7b8a-16c6-4f4e-aeae-fe415c5af7d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_data.iloc[:, -1] = embarked_encoder.fit_transform(X_data.iloc[:, -1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fa3d092-7e3c-4769-8013-5c2bb5a2b404",
   "metadata": {},
   "source": [
    "OneHotEncode the multi-class categorical variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "id": "693366df-1c58-4be9-ad91-40bb5fcfb48e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.compose import ColumnTransformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "id": "93fb0404-e8c7-4ea7-a344-2fa07f174e97",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 0, 1])"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_data['Embarked'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "id": "07bc1dac-3db6-45e1-80f8-27033b22465a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([3, 1, 2])"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_data['Pclass'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "id": "7f5cfaaa-bcfc-43e8-a48b-53585c04996b",
   "metadata": {},
   "outputs": [],
   "source": [
    "col_transformer = ColumnTransformer([('encoder', OneHotEncoder(), [0, 5])], remainder='passthrough')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "id": "9d0ba34e-63b9-4a0e-8360-9f51d410649e",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_data = col_transformer.fit_transform(X_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "id": "f8d3358c-da57-4f02-b69e-f6d73443db3a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.         0.         1.        ... 22.         1.         0.       ]\n",
      " [ 1.         0.         0.        ... 38.         1.         0.       ]\n",
      " [ 0.         0.         1.        ... 26.         0.         0.       ]\n",
      " ...\n",
      " [ 0.         0.         1.        ... 29.6420927  1.         2.       ]\n",
      " [ 1.         0.         0.        ... 26.         0.         0.       ]\n",
      " [ 0.         0.         1.        ... 32.         0.         0.       ]]\n"
     ]
    }
   ],
   "source": [
    "print(X_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8aca7875-a049-4124-918c-027077505575",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Train Test Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "id": "d62bbf9e-9ba2-4af1-aebc-5a215691bdd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "id": "23931344-b299-4203-a60c-df244f782d9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X_data, y_data, test_size=0.2, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4ad6dcc-4147-40fa-a2d2-465626061fd7",
   "metadata": {},
   "source": [
    "### Feature Scaling\n",
    "Standard Scale the features which are continous in nature and require scaling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "id": "95f60464-52ce-425f-a83d-a1fb95f907ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "id": "b933b900-e0a0-41fb-98dc-d16fdaf3ff75",
   "metadata": {},
   "outputs": [],
   "source": [
    "age_scaler = StandardScaler()\n",
    "X_train[:, -3] = age_scaler.fit_transform(X_train[:, -3].reshape(-1, 1)).reshape(-1)\n",
    "X_test[:, -3] = age_scaler.transform(X_test[:, -3].reshape(-1, 1)).reshape(-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "499fcd32-150a-4855-9846-b899321b6335",
   "metadata": {},
   "source": [
    "## Train and Evaluate\n",
    "1. Train a LogisticRegression Model\n",
    "2. Evaluate on the Test split and report the classification report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "id": "2d85bbec-36bf-4eb5-b1be-719f3ae18d73",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score, classification_report"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0ba498d-6841-4f37-872e-12486f8cf2aa",
   "metadata": {},
   "source": [
    "Build and Fit the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "id": "067af84f-7046-4460-93cd-26e48fbbeae8",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "id": "7b6a47e7-f361-4d9a-b45c-28654f0ea7a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression()"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ff6c520-e3d4-4db3-9533-7712b48cae3a",
   "metadata": {},
   "source": [
    "Make predictions on both train and test split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "id": "e05a2de3-dc55-43cd-92fa-073517034942",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_preds = model.predict(X_train)\n",
    "test_preds = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97c87bcc-f93c-4622-9b15-b606acb7199e",
   "metadata": {},
   "source": [
    "Check the train and test accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "4b3af3b4-5530-4bce-8104-dd57ed6d5e06",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train Acc: 0.8241912798874824\n",
      "Test Acc: 0.7191011235955056\n"
     ]
    }
   ],
   "source": [
    "train_acc = accuracy_score(y_train, train_preds)\n",
    "test_acc = accuracy_score(y_test, test_preds)\n",
    "print('Train Acc:', train_acc)\n",
    "print('Test Acc:', test_acc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc75643a-4cdf-468b-9940-60149d958a7d",
   "metadata": {},
   "source": [
    "Print out the test classification report with details of precision, recall and f1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "id": "48b59ccf-f383-4f7b-a6e1-209843652e7c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.76      0.84      0.80       105\n",
      "           1       0.73      0.62      0.67        73\n",
      "\n",
      "    accuracy                           0.75       178\n",
      "   macro avg       0.74      0.73      0.73       178\n",
      "weighted avg       0.75      0.75      0.74       178\n",
      "\n"
     ]
    }
   ],
   "source": [
    "test_report = classification_report(y_test, test_preds)\n",
    "print(test_report)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "id": "037cd2cf-46f2-430e-9272-4c76e3f8b7ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>29.642093</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>889 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Survived  Pclass     Sex        Age  SibSp  Parch Embarked\n",
       "0           0       3    male  22.000000      1      0        S\n",
       "1           1       1  female  38.000000      1      0        C\n",
       "2           1       3  female  26.000000      0      0        S\n",
       "3           1       1  female  35.000000      1      0        S\n",
       "4           0       3    male  35.000000      0      0        S\n",
       "..        ...     ...     ...        ...    ...    ...      ...\n",
       "886         0       2    male  27.000000      0      0        S\n",
       "887         1       1  female  19.000000      0      0        S\n",
       "888         0       3  female  29.642093      1      2        S\n",
       "889         1       1    male  26.000000      0      0        C\n",
       "890         0       3    male  32.000000      0      0        Q\n",
       "\n",
       "[889 rows x 7 columns]"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "840ce8a3-a4e1-495f-8e19-dcc340d16c58",
   "metadata": {},
   "source": [
    "### Hurray! You have successfully completed the ML Exercise in the first ML Monday (Tuesdays)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42de53c6-965b-4b9f-96ca-ceeef7432f5e",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}