Sentiment_Analyzer_Deep_Learning.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/damzC/nlp/blob/main/Sentiment_Analyzer_Deep_Learning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r6gUXSmQZ-nT"
      },
      "source": [
        "This notebook explains the problem of Sentiment Analysis, some of the popular datasets available, approaches to solve this problem and finally a step-by-step guide to solve this problem using Deep Learning.\n",
        "\n",
        "**Definition**: Sentiment Analysis is the NLP task of computationally identifying the opinion or sentiment (*positive*, *negative*, or *neutral*) expressed in a text.\n",
        "\n",
        "**Popular Data sets for Sentiment Analysis:**\n",
        "1. kaggle: Movie reviews on IMDB data set - 50K entries: *Longer text SA*\n",
        "2. Stanford data set for sentiment analysis:*5 classes*: Very positive, Positive, Neutral, Negative, Very Negative\n",
        "3. Amazon review data set (kaggle) - Pre-trained models available: *Short text SA* (4 million entries) - Extract the headings only\n",
        "\n",
        "**Approaches for Sentiment Analysis:**\n",
        "1. Lexicon based: Senti WordNet\n",
        "2. NLP Tools: TextBlob, spaCy, NLTK\n",
        "3. Machine Learning: NB Classifier, SVM, XGB\n",
        "4. **Deep learning: LSTMs, GRUs, seq2seq**\n",
        "5. Sentiment Embeddings - Embeddings of words based on sentiments\n",
        "6. Fine-tuning over Large Language Models (like BERT, RoBERTa *etc.*)\n",
        "\n",
        "**Sentiment Analysis using Deep Learning:**\n",
        "\n",
        "**Pre-processing:**\n",
        "1. Download the dataset (*imdb_reviews*)\n",
        "2. word to index | index to word\n",
        "3. Train your word embeddings | Use pre-trained embeddings\n",
        "4. Embedding Matrix: Index -> Word Embedding\n",
        "5. Padding (Post/Pre) - Fixed length arrays of the input (Size is the max/avg length of the reviews)\n",
        "\n",
        "**Architecture:**\n",
        "1. Input Layer (200)\n",
        "2. Embedding Layer (inbuilt in keras) - Embedding Matrix / Train embeddings using keras (200X300 - Emb Dimension)\n",
        "3. RNN Layer (LSTM, Bi-LSTM etc)/ Attention Layer\n",
        "4. Dense Layer /  Fully Connected Layer\n",
        "5. Dropout Layer\n",
        "6. Optional Dense Layers\n",
        "7. Softmax - Final Output\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Spoy8W7OSWUT"
      },
      "source": [
        "## Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "od74Q4oBSWUW"
      },
      "source": [
        "# keras imports\n",
        "from keras.datasets import imdb\n",
        "from keras.models import Sequential\n",
        "from keras.layers.recurrent import LSTM\n",
        "from keras.layers import Bidirectional\n",
        "from keras.preprocessing.sequence import pad_sequences\n",
        "from keras.layers.core import Dense, Dropout, Activation\n",
        "from keras.utils import np_utils\n",
        "from keras.layers.embeddings import Embedding\n",
        "from keras.optimizers import Adam, Adadelta\n",
        "from keras.models import load_model\n",
        "from keras.regularizers import l2\n",
        "\n",
        "# Generic imports\n",
        "from sklearn.metrics import classification_report, confusion_matrix\n",
        "import numpy as np, string, pickle, warnings, random\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "warnings.filterwarnings(\"ignore\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gcJxnZ99SWUk"
      },
      "source": [
        "## Download data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UviF2mmASWUm"
      },
      "source": [
        "topWords = 50000\n",
        "MAX_LENGTH = 200\n",
        "nb_classes = 2\n",
        "imdbDataPicklePath = 'imdbData.pickle'\n",
        "downloadFlag = 1\n",
        "\n",
        "if downloadFlag == 0:\n",
        "\n",
        "    # Downloading data\n",
        "    imdbData = imdb.load_data(path='imdb.npz', num_words=topWords)\n",
        "\n",
        "    # Pickle Data\n",
        "    with open(imdbDataPicklePath, 'wb') as handle:\n",
        "        pickle.dump(imdbData, handle, protocol=pickle.HIGHEST_PROTOCOL)\n",
        "    \n",
        "with open(imdbDataPicklePath, 'rb') as pHandle:\n",
        "    imdbData = pickle.load(pHandle)\n",
        "    \n",
        "(x_train, y_train), (x_test, y_test) = imdbData\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qQMB30q-SWUv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "76c7c005-308a-448e-8579-35236fb4bdd3"
      },
      "source": [
        "stopWords = ['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', \"you're\", \"you've\", \"you'll\", \\\n",
        "             \"you'd\", 'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', \"she's\", \\\n",
        "             'her', 'hers', 'herself', 'it', \"it's\", 'its', 'itself', 'they', 'them', 'their', 'theirs', \\\n",
        "             'themselves', 'what', 'which', 'who', 'whom', 'this', 'that', \"that'll\", 'these', 'those', 'am', \\\n",
        "             'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does', \\\n",
        "             'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until', 'while', 'of', \\\n",
        "             'at', 'by', 'for', 'with', 'about', 'between', 'into', 'through', 'during', 'before', 'after', \\\n",
        "             'above', 'below', 'to', 'from', 'off', 'over', 'then', 'here', 'there', 'when', 'where', 'why', \\\n",
        "             'how', 'any', 'both', 'each', 'few', 'more', 'most', 'other', 'some', 'such', 'own', 'same', 'so', \\\n",
        "             'than', 'too', 's', 't', 'will', 'just', 'should', \"should've\", 'now', 'd', 'll', 'm', 'o', 're', \\\n",
        "             've', 'y', 'ma']\n",
        "word2Index = imdb.get_word_index()\n",
        "index2Word = {v: k for k, v in word2Index.items()}\n",
        "index2Word[0] = \"\"\n",
        "sentimentDict = {0: 'Negative', 1: 'Positive'}\n",
        "\n",
        "def getWordsFromIndexList(indexList):\n",
        "    wordList = []\n",
        "    for index in indexList:\n",
        "        wordList.append(index2Word[index])\n",
        "\n",
        "    return \" \".join(wordList)\n",
        "\n",
        "def getSentiment(predictArray):\n",
        "    pred = int(predictArray[0])\n",
        "    return sentimentDict[pred]\n",
        "\n",
        "def getIndexFromWordList(wordList):\n",
        "    indexList = []\n",
        "    for word in wordList:\n",
        "        print(word)\n",
        "        indexList.append(str(word2Index[word]))\n",
        "        \n",
        "    return indexList"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json\n",
            "1646592/1641221 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "emZa0eBiSWU3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c417c92e-290c-477c-c091-317551ccfdfe"
      },
      "source": [
        "print (len(word2Index))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "88584\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SBIfYyQdtY5Y",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "904f0e69-6827-4fd1-e262-2f1df3e18514"
      },
      "source": [
        "print(getWordsFromIndexList(x_train[0]))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "the as you with out themselves powerful lets loves their becomes reaching had journalist of lot from anyone to have after out atmosphere never more room titillate it so heart shows to years of every never going villaronga help moments or of every chest visual movie except her was several of enough more with is now current film as you of mine potentially unfortunately of you than him that with out themselves her get for was camp of you movie sometimes movie that with scary but pratfalls to story wonderful that in seeing in character to of 70s musicians with heart had shadows they of here that with her serious to have does when from why what have critics they is you that isn't one will very to as itself with other tricky in of seen over landed for anyone of gilmore's br show's to whether from than out themselves history he name half some br of 'n odd was two most of mean for 1 any an boat she he should is thought frog but of script you not while history he heart to real at barrel but when from one bit then have two of script their with her nobody most that with wasn't to with armed acting watch an for with heartfelt film want an\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RQpCN-PdrgPf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b2921c03-ff6b-48d5-812d-43426b57ce31"
      },
      "source": [
        "print(len(x_train[0]), x_train[0])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "98 [43, 973, 1622, 1385, 458, 4468, 3941, 173, 256, 43, 838, 112, 670, 22665, 480, 284, 150, 172, 112, 167, 21631, 336, 385, 172, 4536, 1111, 17, 546, 447, 192, 2025, 19, 1920, 4613, 469, 43, 76, 1247, 17, 515, 17, 626, 19193, 62, 386, 8, 316, 8, 106, 2223, 5244, 480, 3785, 619, 1415, 215, 28, 52, 10311, 8, 107, 5952, 256, 31050, 7, 3766, 723, 43, 476, 400, 317, 7, 12118, 1029, 104, 381, 297, 2071, 194, 7486, 226, 21, 476, 480, 144, 5535, 28, 224, 104, 226, 1334, 283, 4472, 113, 103, 5345, 19, 178]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OMBOTeT_SWVA"
      },
      "source": [
        "## Preprocess data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6O-nJ4cmSWVC"
      },
      "source": [
        "stopIndexList = []\n",
        "\n",
        "for stopWord in stopWords:\n",
        "    stopIndexList.append(word2Index[stopWord])\n",
        "\n",
        "trainData = []\n",
        "\n",
        "for indexList in x_train:\n",
        "    processedList = [index for index in indexList if index not in stopIndexList]\n",
        "    trainData.append(processedList)\n",
        "    \n",
        "x_train = trainData"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C316bnXaSWVK"
      },
      "source": [
        "## Data Padding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l3drT74ySWVM"
      },
      "source": [
        "'''\n",
        "Padding data to keep vectors of same size\n",
        "If size < 200 then it will be padded, else it will be cropped\n",
        "'''\n",
        "trainX = pad_sequences(x_train, maxlen = MAX_LENGTH, padding='post', value = 0.)\n",
        "testX = pad_sequences(x_test, maxlen = MAX_LENGTH, padding='post', value = 0.)\n",
        "\n",
        "'''\n",
        "One-hot encoding for the classes\n",
        "'''\n",
        "trainY = np_utils.to_categorical(y_train, num_classes = nb_classes)\n",
        "testY = np_utils.to_categorical(y_test, num_classes = nb_classes)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S4nFtP6Btl0g",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5aad615c-4482-4faa-d811-68abb3720711"
      },
      "source": [
        "print(len(trainX[0]), trainX[0])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "200 [   43   973  1622  1385   458  4468  3941   173   256    43   838   112\n",
            "   670 22665   480   284   150   172   112   167 21631   336   385   172\n",
            "  4536  1111    17   546   447   192  2025    19  1920  4613   469    43\n",
            "    76  1247    17   515    17   626 19193    62   386     8   316     8\n",
            "   106  2223  5244   480  3785   619  1415   215    28    52 10311     8\n",
            "   107  5952   256 31050     7  3766   723    43   476   400   317     7\n",
            " 12118  1029   104   381   297  2071   194  7486   226    21   476   480\n",
            "   144  5535    28   224   104   226  1334   283  4472   113   103  5345\n",
            "    19   178     0     0     0     0     0     0     0     0     0     0\n",
            "     0     0     0     0     0     0     0     0     0     0     0     0\n",
            "     0     0     0     0     0     0     0     0     0     0     0     0\n",
            "     0     0     0     0     0     0     0     0     0     0     0     0\n",
            "     0     0     0     0     0     0     0     0     0     0     0     0\n",
            "     0     0     0     0     0     0     0     0     0     0     0     0\n",
            "     0     0     0     0     0     0     0     0     0     0     0     0\n",
            "     0     0     0     0     0     0     0     0     0     0     0     0\n",
            "     0     0     0     0     0     0     0     0]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KbcJtG4RMsRf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "92066d39-1c33-4344-9cd7-6d2cc37f4588"
      },
      "source": [
        "print(len(trainY[0]), trainY[0], y_train[0])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2 [0. 1.] 1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9A7RlSMlSWVV"
      },
      "source": [
        "## Network Parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9pmCGoGOSWVX"
      },
      "source": [
        "sgdOptimizer = 'adam'\n",
        "lossFun='categorical_crossentropy'\n",
        "batchSize=1024\n",
        "numEpochs = 5\n",
        "numHiddenNodes = 128\n",
        "EMBEDDING_SIZE = 300\n",
        "denseLayer1Size = 256\n",
        "denseLayer2Size = 128"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hjzkuWYlSWVf"
      },
      "source": [
        "## Network Architecture"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ly4punt3SWVh",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2e8e33b6-9759-4f53-8722-c7fe3b8e0d94"
      },
      "source": [
        "model = Sequential()\n",
        "\n",
        "# Train Embedding layer with Embedding Size = 300\n",
        "model.add(Embedding(topWords, EMBEDDING_SIZE, input_length=MAX_LENGTH, mask_zero=True, name='embedding_layer'))\n",
        "\n",
        "# Define Deep Learning layer\n",
        "model.add(Bidirectional(LSTM(numHiddenNodes), merge_mode='concat',name='bidi_lstm_layer'))\n",
        "\n",
        "# Define Dense layers\n",
        "model.add(Dense(denseLayer1Size, activation='relu', name='dense_1'))\n",
        "model.add(Dropout(0.25, name = 'dropout'))\n",
        "model.add(Dense(denseLayer2Size, activation='relu', name='dense_2'))\n",
        "\n",
        "# Define Output Layer\n",
        "model.add(Dense(nb_classes, activation='softmax', name='output'))\n",
        "\n",
        "model.compile(loss=lossFun, optimizer=sgdOptimizer, metrics=[\"accuracy\"])\n",
        "print(model.summary())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "embedding_layer (Embedding)  (None, 200, 300)          15000000  \n",
            "_________________________________________________________________\n",
            "bidi_lstm_layer (Bidirection (None, 256)               439296    \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 256)               65792     \n",
            "_________________________________________________________________\n",
            "dropout (Dropout)            (None, 256)               0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 128)               32896     \n",
            "_________________________________________________________________\n",
            "output (Dense)               (None, 2)                 258       \n",
            "=================================================================\n",
            "Total params: 15,538,242\n",
            "Trainable params: 15,538,242\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zlgS9tvBSWVp"
      },
      "source": [
        "## Training the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "HWzherNuSWVq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3ff06154-3e18-404c-9b5c-71c4a342b160"
      },
      "source": [
        "model.fit(trainX, trainY, batch_size=batchSize, epochs=numEpochs, verbose=1, validation_data=(testX, testY))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "25/25 [==============================] - 69s 2s/step - loss: 0.6406 - accuracy: 0.6340 - val_loss: 0.3748 - val_accuracy: 0.8366\n",
            "Epoch 2/5\n",
            "25/25 [==============================] - 59s 2s/step - loss: 0.2089 - accuracy: 0.9208 - val_loss: 0.3472 - val_accuracy: 0.8516\n",
            "Epoch 3/5\n",
            "25/25 [==============================] - 59s 2s/step - loss: 0.0773 - accuracy: 0.9769 - val_loss: 0.4114 - val_accuracy: 0.8437\n",
            "Epoch 4/5\n",
            "25/25 [==============================] - 60s 2s/step - loss: 0.0251 - accuracy: 0.9942 - val_loss: 0.6187 - val_accuracy: 0.8378\n",
            "Epoch 5/5\n",
            "25/25 [==============================] - 59s 2s/step - loss: 0.0063 - accuracy: 0.9984 - val_loss: 0.7020 - val_accuracy: 0.8322\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f91bea04278>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JX2D73szSWVz"
      },
      "source": [
        "# Model accuracy"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TLh86hBdSWV5",
        "outputId": "0f80a056-cd34-4432-fe8e-10fad828b859"
      },
      "source": [
        "score = model.evaluate(testX, testY, verbose=0)\n",
        "print(\"%s: %.2f%%\" % (model.metrics_names[1], score[1]*100))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "accuracy: 84.06%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jaHHfDz0SWWB",
        "outputId": "d8cd92fd-0cc3-4829-cc25-724258c4d445"
      },
      "source": [
        "predY = model.predict_classes(testX)\n",
        "yPred = np_utils.to_categorical(predY, num_classes = nb_classes)\n",
        "print(\"Classification Report:\\n\")\n",
        "print(classification_report(testY, yPred))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Classification Report:\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.83      0.86      0.84     12500\n",
            "           1       0.86      0.82      0.84     12500\n",
            "\n",
            "   micro avg       0.84      0.84      0.84     25000\n",
            "   macro avg       0.84      0.84      0.84     25000\n",
            "weighted avg       0.84      0.84      0.84     25000\n",
            " samples avg       0.84      0.84      0.84     25000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FrupZ1vpSWWI"
      },
      "source": [
        "## Save the Tensorflow Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8YAHrFtGSWWJ"
      },
      "source": [
        "model.save('imdb_bi_lstm_tensorflow_model.hdf5')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "elhbNa7NSWWP"
      },
      "source": [
        "## Load the Tensorflow Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OXj0th-eSWWQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d1ba1882-a8f6-42a8-d18c-f531407de660"
      },
      "source": [
        "loaded_model = load_model('imdb_bi_lstm_tensorflow_model.hdf5')\n",
        "print(loaded_model.summary())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "embedding_layer (Embedding)  (None, 200, 300)          15000000  \n",
            "_________________________________________________________________\n",
            "bidi_lstm_layer (Bidirection (None, 256)               439296    \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 256)               65792     \n",
            "_________________________________________________________________\n",
            "dropout (Dropout)            (None, 256)               0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 128)               32896     \n",
            "_________________________________________________________________\n",
            "output (Dense)               (None, 2)                 258       \n",
            "=================================================================\n",
            "Total params: 15,538,242\n",
            "Trainable params: 15,538,242\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I4ZaRjrZSWWc"
      },
      "source": [
        "## Testing the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ANvTtsjnSWWf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c298b106-cdbf-4e5a-88fa-37744bd5f3bb"
      },
      "source": [
        "num = 167\n",
        "num_next = num + 1\n",
        "print(\"Testing for test case...\" + str(num))\n",
        "groundTruth = testY[num]\n",
        "\n",
        "sampleX = testX[num:num_next]\n",
        "predictionClass = loaded_model.predict_classes(sampleX, verbose=0)\n",
        "prediction = np_utils.to_categorical(predictionClass, num_classes = nb_classes)[0]\n",
        "\n",
        "print(\"Text: \" + str(getWordsFromIndexList(x_test[num-1])))\n",
        "print(\"\\nPrediction: \" + str(getSentiment(predictionClass)))\n",
        "if np.array_equal(groundTruth,prediction):\n",
        "    print(\"\\nPrediction is Correct\")\n",
        "else:\n",
        "    print(\"\\nPrediction is Incorrect\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Testing for test case...167\n",
            "Text: the expert enters accession epos about right necks seen nurse everybody this as klutz yourself must lives not what would vain about almost film instructor evil including this early her painted and you has is found like it give making creatures floriane to exciting anderson special custody thing does when amount in lindsay but to eye boll there of questioned disbelief br written falls father vans intellectual me boat some br allows who affection to rings just idea to as you had 140 cows sorts cause is quite br performances dance this about hain friends of corporate moments camera always point between expert enters technically way is him ben it's spinning feels about police feeling after had concern clearly to would monsters good along watches for well given at mishaps it's themed order going thai horror in society as not sees all question expert well another she get epic message as significant attacked attempts viewing and interesting is very humanity but thai to sees say there be humor this as figure shriek guy fell enters out by mgm paul is apropos later am or\n",
            "\n",
            "Prediction: Positive\n",
            "\n",
            "Prediction is Correct\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}