08. Neural Networks Introduction - R.ipynb

{
  "cells": [
    {
      "metadata": {
        "collapsed": true
      },
      "cell_type": "markdown",
      "source": "Exercise 8 - Introduction to Neural Networks\n===\n\nOriginally hypothesised in the 1940s, neural networks are now one of the main tools used in modern AI. Neural networks can be used for both regression and categorisation applications. Recent advances with storage, processing power, and open-source tools have allowed many successful applications of neural networks in medical diagnosis, filtering explicit content, speech recognition, and machine translation.\n\nIn this exercise we will compare three dog breeds using their age, weight, and height. We will make a neural network model to classify the breeds of dogs based on these features.\n\nNote: It's extremely common for AI practitioners to use a template such as the one below for making neural networks quickly. After you are done, feel free to play around with the template to get a feel of how you can easily adjust a neural network to your problems using the package `keras`.\n\nLet's start by loading the libraries required for this session.\n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this to load the required libraries, it might take a little while.\n\nsuppressMessages(install.packages(\"tidyverse\"))\nsuppressMessages(library(\"tidyverse\"))\n\nsuppressMessages(install.packages(\"keras\"))\nsuppressMessages(library(keras))\nsuppressMessages(install_keras())",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 1\n---\n\nNow let's load our data and inspect it.\n\n#### Replace `<dataset>` with `dog_data` and run the code."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to load our data\n\n# Load the dataset `dog_data.csv`\n\n###\n# REPLACE <dataset> WITH dog_data\n###\n<dataset> <- read.csv(\"Data/dog_data.csv\")\n###\n\n# Check the structure\nstr(dog_data)\nhead(dog_data)\nsummary(dog_data)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Based on the output of `str(dog_data)`, we have **200 observations** on dogs stored in **4 variables**:\n\n* `age`: the first feature;\n* `weight`: the second feature;\n* `height`: the third feature;\n* `breed`: the label, represented as numbers `0`, `1`, and `2`. \n\nStep 2\n---\n\nBefore we make our model, let's get our training and test sets ready.\n\nWe've got 200 observations on dogs, so we'll use the first 160 observations for the training set, and the last 40 observations for our test set. For both the training and test sets, we will also separate `X` the features (`age`, `weight` and `height`) from `Y` the label (`breed`).\n\n### In the cell below replace:\n#### 1. `<trainingSetLocation>` with `1:160`\n#### 2. `<trainingSetLocation>` with `1:160`\n#### 3. `<testSetLocation>` with `161:200`\n#### 4. `<testSetLocation>` with `161:200`\n#### then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to split data into training and test sets\n\n###\n# REPLACE <trainingSetLocation> WITH 1:160\n###\ntrain_X <- as.matrix(dog_data[<trainingSetLocation>, 1:3]) # Rows 1 - 160, columns 1 - 3 (the features)\nraw_train_Y <- as.matrix(dog_data[<trainingSetLocation>, 4]) # Rows 1 - 160, column 4 (the label)\n###\n\n###\n# REPLACE <testSetLocation> WITH 161:200\n###\ntest_X <- as.matrix(dog_data[<testSetLocation>, 1:3]) # Rows 161 - 200, columns 1 - 3 (the features)\nraw_test_Y <- as.matrix(dog_data[<testSetLocation>, 4]) # Rows 161 - 200, column 4 (the label)\n###\n\n# Check first few lines of new variables to see if the output is what we expect\n# Training data\nhead(train_X)\nhead(raw_train_Y)\n\n# Test data\nhead(test_X)\nhead(raw_test_Y)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 3\n---\n\nFor a neural network, indicating `breed` using  `0`, `1`, and `2` are misleading, as it might imply that breed `0` is closer to breed `1` than breed `2`. But that is not the case here.\n\nTo allow the neural network to predict categories properly, we represent categories as 'one-hot vectors'. The labels (dog breeds) will go from being represented as `0`, `1`, and `2` to this:\n\n| breed 0 | breed 1 | breed 2 |\n|:------- |:------- |:------- |\n| `1 0 0` | `0 1 0` | `0 0 1` |\n\nSo if the 1 is in the first position, the neural network knows that it's breed 0.\n\nIf the 1 is in the second position, the neural network knows that it's breed 1, and so on.\n\nThe code below will turn our raw labels into one-hot vectors our neural networks will be able to use.\n\n### In the cell below replace:\n#### 1. `<trainingLabels>` with `raw_train_Y`\n#### 2. `<testLabels>` with `raw_test_Y`\n#### then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# This box uses the keras function to_categorical to change breed from integer to categorical\n\n###\n# REPLACE <trainingLabels> WITH raw_train_Y\n###\ntrain_Y <- to_categorical(<trainingLabels>, num_classes = 3)\n###\n\n###\n# REPLACE <testLabels> WITH raw_test_Y\n###\ntest_Y <- to_categorical(<testLabels>, num_classes = 3)\n###\n\n# Print out some of our training and test data\nhead(train_Y)\nhead(test_Y)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "There we go!\n\n## Step 4\n\nThat's our data ready. Now it's time to make your first neural network model!\n\nThis is the standard syntax for a model using the `keras` package. You can always play around with adding in extra hidden layers and changing their size and activation functions later.\n\nOur **input shape** in the first dense layer is the **number of features**, which is **3** in this case.\n\nOur **final layer** has **3 units** (nodes), one for each of the dog breeds. So if we had 5 different breeds of dog in our dataset, the final layer would have 5 units.\n\n### In the cell below replace:\n#### 1. `<hiddenLayer1>` with `10`\n#### 2. `<inputNumber>` with `3`\n#### 3. `<hiddenLayer2>` with `10`\n#### 4. `<outputNumber>` with `3`\n#### then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this!\n\nuse_session_with_seed(5)\nset.seed(5)\n\nmodel <- keras_model_sequential()\n\nmodel %>%\n\n# Add densely-connected neural network layers using `layer_dense` function\n# Our first layer has an input shape of 3 to represent 3 input features (age, weight, height)\n\n###\n# REPLACE <hiddenLayer1> WITH 10 AND <inputNumber> WITH 3\n###\nlayer_dense(units = <hiddenLayer1>, activation = \"relu\", input_shape = <inputNumber>) %>% \n###\n\n# We now have a hidden layer with 10 nodes, with an input shape of 3 representing our 3 features.\n\n# Next up we'll add another layer, with 10 nodes too.\n###\n# REPLACE <hiddenLayer2> WITH 10\n###\nlayer_dense(units = <hiddenLayer2>, activation = \"relu\") %>% \n###\n\n# Uncomment the next line if you want to add another layer\n# layer_dense(units = 10, activation = \"relu\") %>% \n\n###\n# REPLACE <outputNumber> WITH 3\n###\nlayer_dense(units = <outputNumber>, activation = \"softmax\")\n###\n\n# Output layer has 3 nodes, one for each type of category we have\n\nmodel %>% summary",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Alright, that's our first model ready.\n\nN.B. `\"tanh\"` is another common activation function that, if you want, you can try instead of `\"relu\"`, but it doesn't perform very well here.\n\nFeel free to experiment with some different parameters later on. If this doesn't work, check that you have the correct size for the input and output layers in Step 4 (must be 3 nodes each). For example, \"tanh\" is another popular activation function if you want to try it instead of \"relu\".\n\nStep 5\n---\n\nNext, we'll compile the model for training and see how it runs.\n\nThere are a few parameters you can choose that change how the model trains, and end up changing how the model performs.\n\nWe will use some standard parameters for now. \n\nFeel free to experiment with some different parameters later on. If this doesn't work, check that you input the correct size for the input and output layers in Step 4 (must have 3 nodes each).\n\n#### Replace `<optimizer>` with `optimizer_adagrad()` and run the code."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <optimizer> WITH optimizer_adagrad()\n###\nmodel %>% compile(\n    loss = \"categorical_crossentropy\",\n    optimizer = <optimizer>,\n    metrics = c(\"accuracy\")\n)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "N.B. `\"adam\"` is another popular optimizer if you want to try it instead of `\"adagrad\"`.\n\nLet's train the neural network and plot it!\n\n### In the cell below replace:\n#### 1. `<xData>` with `train_X`\n#### 2. `<yData>` with `train_Y`\n#### 3. `<epochNumber>` with `25`\n#### then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to plot our fit and print out how it performed on the training set\nhistory <- model %>% fit(\n    ###\n    # REPLACE <xData> WITH train_X and <yData> WITH train_Y\n    ###\n    x = <xData>,\n    y = <yData>,\n    ###\n    shuffle = T,\n    ###\n    # REPLACE <epochNumber> WITH 25\n    ###\n    epochs = <epochNumber>,\n    ###\n    batch_size = 2,\n    validation_split = 0.2\n)\n\nplot(history)\n\n# This tells us how the model performed on the training set\nhistory",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Note that the original training set `train_X` and `train_Y` with 160 observations has been split up again during the training process, where 112 of 160 samples were used for training, and 48 samples were used for validation, as per the output from `history`.\n\nStep 6\n---\n\nNow that our model is trained and ready, let's see how it performs on our test data, `test_X` and `test_Y`!\n\nIt's important to test a model on data that it has never seen before, to make sure it doesn't overfit. Now let's evaluate it against the test set.\n\n**Run the box below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box\nperf <- model %>% evaluate(test_X, test_Y)\nprint(perf)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "It seems to be very accurate (acc = 95%) with the random seed that we set!\n\nLet's see how the model predicts something completely new and unclassified.\n\n**Come up with a brand new sample of the format `[age, weight, height]` to test the model with, then run the two code blocks.**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# CHANGE age, weight, AND height TO NEW VALUES \n###\nnew_dog <- data.frame(age = 5, weight = 4, height = 8)\n###\n\nstr(dog_data)\n\n# Age vs weight\nggplot() +\ngeom_point(data = dog_data, aes(x = age, y = weight, colour = as.factor(breed))) +\ngeom_point(data = new_dog, aes (x = age, y = weight), shape = \"+\", size=10) +\nlabs(x = \"Age\", y = \"Weight\", colour = \"Breed\")",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this code block to plot the relationship between age, height, and breed\n# Age vs height\nggplot() +\ngeom_point(data = dog_data, aes(x = age, y = height, colour = as.factor(breed))) +\ngeom_point(data = new_dog, aes (x = age, y = height), , shape = \"+\", size=10) +\nlabs(x = \"Age\", y = \"Height\", colour = \"Breed\")",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now let's see what breed of dog the model says it is!\n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this code to run the model\n\nprint(\"Probabilities of classes:\")\npredict_proba(model, as.matrix(new_dog))\n\nprint(\"Predicted class:\")\npredict_classes(model, as.matrix(new_dog))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "The final number tells us which class it thinks it is.\n\nConclusion\n---\n\nWe've built a simple neural network to help us predict dog breeds. In the next exercise, we'll look into neural networks with a bit more depth, and at the factors that influence how well it learns.\n\nIf you want to play around with this neural network and a new data set, just remember to set your input and output sizes correctly."
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "r",
      "display_name": "R",
      "language": "R"
    },
    "language_info": {
      "mimetype": "text/x-r-source",
      "name": "R",
      "pygments_lexer": "r",
      "version": "3.4.1",
      "file_extension": ".r",
      "codemirror_mode": "r"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}