TensorFlow Beginner - Predicting House Prices with Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 1: Introduction\n",
    "\n",
    "---\n",
    "\n",
    "For this project, we are going to work on evaluating price of houses given the following features:\n",
    "\n",
    "1. Year of sale of the house\n",
    "2. The age of the house at the time of sale\n",
    "3. Distance from city center\n",
    "4. Number of stores in the locality\n",
    "5. The latitude\n",
    "6. The longitude\n",
    "\n",
    "![Regression](images/regression.png)\n",
    "\n",
    "Note: This notebook uses `python 3` and these packages: `tensorflow`, `pandas`, `matplotlib`, `scikit-learn`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1: Importing Libraries & Helper Functions\n",
    "\n",
    "First of all, we will need to import some libraries and helper functions. This includes TensorFlow and some utility functions that I've written to save time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "\n",
    "from utils import *\n",
    "from sklearn.model_selection import train_test_split\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Dropout\n",
    "from tensorflow.keras.callbacks import EarlyStopping, LambdaCallback\n",
    "\n",
    "%matplotlib inline\n",
    "tf.logging.set_verbosity(tf.logging.ERROR)\n",
    "\n",
    "print('Libraries imported.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 2: Importing the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1: Importing the Data\n",
    "\n",
    "The dataset is saved in a `data.csv` file. We will use `pandas` to take a look at some of the rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('data.csv', names = column_names) \n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2: Check Missing Data\n",
    "\n",
    "It's a good practice to check if the data has any missing values. In real world data, this is quite common and must be taken care of before any data pre-processing or model training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 3: Data Normalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1: Data Normalization\n",
    "\n",
    "We can make it easier for optimization algorithms to find minimas by normalizing the data before training a model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.iloc[:,1:]\n",
    "df_norm = (df - df.mean()) / df.std()\n",
    "df_norm.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2: Convert Label Value\n",
    "\n",
    "Because we are using normalized values for the labels, we will get the predictions back from a trained model in the same distribution. So, we need to convert the predicted values back to the original distribution if we want predicted prices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_mean = df['price'].mean()\n",
    "y_std = df['price'].std()\n",
    "\n",
    "def convert_label_value(pred):\n",
    "    return int(pred * y_std + y_mean)\n",
    "\n",
    "print(convert_label_value(0.350088))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 4: Create Training and Test Sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1: Select Features\n",
    "\n",
    "Make sure to remove the column __price__ from the list of features as it is the label and should not be used as a feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df_norm.iloc[:, :6]\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2: Select Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Y = df_norm.iloc[:, -1]\n",
    "Y.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3: Feature and Label Values\n",
    "\n",
    "We will need to extract just the numeric values for the features and labels as the TensorFlow model will expect just numeric values as input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_arr = X.values\n",
    "Y_arr = Y.values\n",
    "\n",
    "print('X_arr shape: ', X_arr.shape)\n",
    "print('Y_arr shape: ', Y_arr.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.4: Train and Test Split\n",
    "\n",
    "We will keep some part of the data aside as a __test__ set. The model will not use this set during training and it will be used only for checking the performance of the model in trained and un-trained states. This way, we can make sure that we are going in the right direction with our model training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X_arr, Y_arr, test_size = 0.05, shuffle = True, random_state=0)\n",
    "\n",
    "print('X_train shape: ', X_train.shape)\n",
    "print('y_train shape: ', y_train.shape)\n",
    "print('X_test shape: ', X_test.shape)\n",
    "print('y_test shape: ', y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 5: Create the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.1: Create the Model\n",
    "\n",
    "Let's write a function that returns an untrained model of a certain architecture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_model():\n",
    "    \n",
    "    model = Sequential([\n",
    "        Dense(10, input_shape = (6,), activation = 'relu'),\n",
    "        Dense(20, activation = 'relu'),\n",
    "        Dense(5, activation = 'relu'),\n",
    "        Dense(1)\n",
    "    ])\n",
    "\n",
    "    model.compile(\n",
    "        loss='mse',\n",
    "        optimizer='adadelta'\n",
    "    )\n",
    "    \n",
    "    return model\n",
    "\n",
    "model = get_model()\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 6: Model Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.1: Model Training\n",
    "\n",
    "We can use an `EarlyStopping` callback from Keras to stop the model training if the validation loss stops decreasing for a few epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "early_stopping = EarlyStopping(monitor='val_loss', patience = 5)\n",
    "\n",
    "model = get_model()\n",
    "\n",
    "preds_on_untrained = model.predict(X_test)\n",
    "\n",
    "history = model.fit(\n",
    "    X_train, y_train,\n",
    "    validation_data = (X_test, y_test),\n",
    "    epochs = 1000,\n",
    "    callbacks = [early_stopping]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.2: Plot Training and Validation Loss\n",
    "\n",
    "Let's use the `plot_loss` helper function to take a look training and validation loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_loss(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 7: Predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7.1: Plot Raw Predictions\n",
    "\n",
    "Let's use the `compare_predictions` helper function to compare predictions from the model when it was untrained and when it was trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preds_on_trained = model.predict(X_test)\n",
    "\n",
    "compare_predictions(preds_on_untrained, preds_on_trained, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7.2: Plot Price Predictions\n",
    "\n",
    "The plot for price predictions and raw predictions will look the same with just one difference: The x and y axis scale is changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "price_on_untrained = [convert_label_value(y) for y in preds_on_untrained]\n",
    "price_on_trained = [convert_label_value(y) for y in preds_on_trained]\n",
    "price_y_test = [convert_label_value(y) for y in y_test]\n",
    "\n",
    "compare_predictions(price_on_untrained, price_on_trained, price_y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}