3. ExplainableMachineLearningShap.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "ExplainableMachineLearningShap.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernel_info": {
      "name": "python3"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    },
    "nteract": {
      "version": "0.12.3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zij6hQgsz6zX",
        "colab_type": "text"
      },
      "source": [
        "**IMPORTANT: If you run this notebook in Google Colab, change your Runtype to GPU**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yoPhvhakz6zc",
        "colab_type": "text"
      },
      "source": [
        "# SHAP Explanations\n",
        "\n",
        "SHAP (SHapley Additive exPlanations) is a unified approach to explain the output of any machine learning model. SHAP connects game theory with local explanations, uniting several previous methods and representing the only possible consistent and locally accurate additive feature attribution method based on expectations.\n",
        "\n",
        "*Reference: Lundberg, Scott M., and Su-In Lee. \"A unified approach to interpreting model predictions.\" Advances in Neural Information Processing Systems. 2017.*\n",
        "\n",
        "Most important methods Unified by SHAP\n",
        "+ LIME: Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. \"Why should i trust you?: Explaining the predictions of any classifier.\" Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016.\n",
        "\n",
        "+ Shapley sampling values: Strumbelj, Erik, and Igor Kononenko. \"Explaining prediction models and individual predictions with feature contributions.\" Knowledge and information systems 41.3 (2014): 647-665.\n",
        "\n",
        "+ DeepLIFT: Shrikumar, Avanti, Peyton Greenside, and Anshul Kundaje. \"Learning important features through propagating activation differences.\" arXiv preprint arXiv:1704.02685 (2017).\n",
        "\n",
        "+ Layer-wise relevance propagation: Bach, Sebastian, et al. \"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.\" PloS one 10.7 (2015): e0130140.\n",
        "\n",
        "+ Shapley regression values: Lipovetsky, Stan, and Michael Conklin. \"Analysis of regression in game theory approach.\" Applied Stochastic Models in Business and Industry 17.4 (2001): 319-330.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FJq23Qzqz6zd",
        "colab_type": "text"
      },
      "source": [
        "SHAP is based on Shapley values, a method that is the provably the only one that satisfies three axioms of credit-attribution:\n",
        "\n",
        "+ If a player never adds any marginal value, their payoff portion should be 0.\n",
        "+ **Substitutability**: If two players always add the same marginal value to any subset to which they’re added, their payoff portion should be the same.\n",
        "+ **Additivity**: If a game is composed of two subgames, you should be able to add the payoffs calculated on the subgames, and that should match the payoffs calculated for the full game.\n",
        "\n",
        "SHAP values interpret the impact of having a certain value for a given feature in comparison to the prediction we'd make if that feature took some baseline value (the average model output over the training dataset we passed).\n",
        "\n",
        "+ How much was a prediction driven by the fact that the team scored 3 goals, **instead of some baseline number of goals**.\n",
        "\n",
        "SHAP values do this in a very special way. When we make a prediction\n",
        "\n",
        "`sum(SHAP values for all features) = pred_for_team - pred_for_baseline_values`\n",
        "\n",
        "That is, the SHAP values of all features sum up to explain why my prediction was different from the baseline. They are Shapley values."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5Nkdti0Oz6ze",
        "colab_type": "text"
      },
      "source": [
        "Let's see how it works by following the example we can find in `https://slundberg.github.io/shap/notebooks/linear_explainer/Sentiment%20Analysis%20with%20Logistic%20Regression.html`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "BJR7HTOWz6zf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install shap"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "E7FR5fG_z6zk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "\n",
        "data = pd.read_csv('https://raw.githubusercontent.com/DataScienceUB/ExplainableDataScience/master/FIFA%202018%20Statistics.csv')\n",
        "y = (data['Man of the Match'] == \"Yes\")  # Convert from string \"Yes\"/\"No\" to binary\n",
        "feature_names = [i for i in data.columns if data[i].dtype in [np.int64, np.int64]]\n",
        "X = data[feature_names]\n",
        "train_X, val_X, train_y, val_y = train_test_split(X, y, random_state=1)\n",
        "my_model = RandomForestClassifier(random_state=0).fit(train_X, train_y)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "ZfitJrgMz6zn",
        "colab_type": "text"
      },
      "source": [
        "We will look at SHAP values for a single row of the dataset (we arbitrarily chose row 5). For context, we'll look at the raw predictions before looking at the SHAP values"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "yQXGAo8Bz6zn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "row_to_show = 5\n",
        "data_for_prediction = val_X.iloc[row_to_show]  # use 1 row of data here. Could use multiple rows if desired\n",
        "data_for_prediction_array = data_for_prediction.values.reshape(1, -1)\n",
        "\n",
        "\n",
        "my_model.predict_proba(data_for_prediction_array)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4CjKbsxGz6zs",
        "colab_type": "text"
      },
      "source": [
        "The team is 70% likely to have a player win the award.\n",
        "\n",
        "Now, we'll move onto the code to get SHAP values for that single prediction."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WeRX898_z6zv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import shap  # package used to calculate Shap values\n",
        "\n",
        "# Create object that can calculate shap values\n",
        "explainer = shap.TreeExplainer(my_model)\n",
        "\n",
        "# Calculate Shap values\n",
        "shap_values = explainer.shap_values(data_for_prediction)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RZgMsMZTz6z1",
        "colab_type": "text"
      },
      "source": [
        "The `shap_values` object above is a list with two arrays. The first array is the SHAP values for a negative outcome (don't win the award), and the second array is the list of SHAP values for the positive outcome (wins the award). "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jk67AvOjz6z3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "shap.initjs()\n",
        "shap.force_plot(explainer.expected_value[1], shap_values[1], data_for_prediction)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "_Z2KPl8Wz6z9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "explainer.expected_value[0], explainer.expected_value[1]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1gK06Bguz60B",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "shap_values[0], shap_values[1]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "aHCgrTqCz60H",
        "colab_type": "text"
      },
      "source": [
        "**Explanation:**\n",
        "\n",
        "We predicted 0.7, whereas the base_value is 0.4979. \n",
        "\n",
        "Feature values causing increased predictions are in pink, and their visual size shows the magnitude of the feature's effect. \n",
        "\n",
        "Feature values decreasing the prediction are in blue.\n",
        "\n",
        "The biggest impact comes from Goal Scored being 2. Though the ball possession value has a meaningful effect decreasing the prediction.\n",
        "\n",
        "*Note*: If you subtract the length of the blue bars from the length of the pink bars, it equals the distance from the base value to the output."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HkUbwiGsz60d",
        "colab_type": "text"
      },
      "source": [
        "In the code above we have used `shap.TreeExplainer(my_model)`, but the SHAP package has explainers for every type of model:\n",
        "\n",
        "+ `shap.DeepExplainer` works with Deep Learning models.\n",
        "+ `shap.KernelExplainer` works with all models, though it is slower than other Explainers and it offers an approximation rather than exact Shap values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "b9DvZ35Fz60f",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "'''Trains a simple convnet on the MNIST dataset.\n",
        "Gets to 99.25% test accuracy after 12 epochs\n",
        "(there is still a lot of margin for parameter tuning).\n",
        "16 seconds per epoch on a GRID K520 GPU.\n",
        "'''\n",
        "\n",
        "from __future__ import print_function\n",
        "import tensorflow\n",
        "import tensorflow.keras as keras\n",
        "from tensorflow.keras.datasets import mnist\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, Dropout, Flatten\n",
        "from tensorflow.keras.layers import Conv2D, MaxPooling2D\n",
        "from tensorflow.keras import backend as K\n",
        "\n",
        "batch_size = 128\n",
        "num_classes = 10\n",
        "epochs = 12\n",
        "\n",
        "# input image dimensions\n",
        "img_rows, img_cols = 28, 28\n",
        "\n",
        "# the data, split between train and test sets\n",
        "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
        "\n",
        "if K.image_data_format() == 'channels_first':\n",
        "    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)\n",
        "    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)\n",
        "    input_shape = (1, img_rows, img_cols)\n",
        "else:\n",
        "    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)\n",
        "    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)\n",
        "    input_shape = (img_rows, img_cols, 1)\n",
        "\n",
        "x_train = x_train.astype('float32')\n",
        "x_test = x_test.astype('float32')\n",
        "x_train /= 255\n",
        "x_test /= 255\n",
        "print('x_train shape:', x_train.shape)\n",
        "print(x_train.shape[0], 'train samples')\n",
        "print(x_test.shape[0], 'test samples')\n",
        "\n",
        "# convert class vectors to binary class matrices\n",
        "y_train = keras.utils.to_categorical(y_train, num_classes)\n",
        "y_test = keras.utils.to_categorical(y_test, num_classes)\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Conv2D(32, kernel_size=(3, 3),\n",
        "                 activation='relu',\n",
        "                 input_shape=input_shape))\n",
        "model.add(Conv2D(64, (3, 3), activation='relu'))\n",
        "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
        "model.add(Dropout(0.25))\n",
        "model.add(Flatten())\n",
        "model.add(Dense(128, activation='relu'))\n",
        "model.add(Dropout(0.5))\n",
        "model.add(Dense(num_classes, activation='softmax'))\n",
        "\n",
        "model.compile(loss=keras.losses.categorical_crossentropy,\n",
        "              optimizer=tensorflow.keras.optimizers.Adadelta(),\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "model.fit(x_train, y_train,\n",
        "          batch_size=batch_size,\n",
        "          epochs=epochs,\n",
        "          verbose=1,\n",
        "          validation_data=(x_test, y_test))\n",
        "score = model.evaluate(x_test, y_test, verbose=0)\n",
        "print('Test loss:', score[0])\n",
        "print('Test accuracy:', score[1])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "rVzGrb5bz60l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import shap\n",
        "import numpy as np\n",
        "\n",
        "# select a set of background examples to take an expectation over\n",
        "background = x_train[np.random.choice(x_train.shape[0], 100, replace=False)]\n",
        "\n",
        "# explain predictions of the model on four images\n",
        "# e = shap.DeepExplainer(model, background)\n",
        "# ...or pass tensors directly\n",
        "e = shap.GradientExplainer((model.layers[0].input, model.layers[-1].output), background)\n",
        "shap_values = e.shap_values(x_test[1:5])\n",
        "\n",
        "# plot the feature attributions\n",
        "shap.image_plot(shap_values, -x_test[1:5])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "inputHidden": false,
        "outputHidden": false,
        "id": "tw4EBGIXz60q",
        "colab_type": "text"
      },
      "source": [
        "The resulting plot explains ten outputs (digits 0-9) for four different images. Red pixels increase the model's output while blue pixels decrease the output. The input images are shown on the left, and as nearly transparent grayscale backings behind each of the explanations. The sum of the SHAP values equals the difference between the expected model output (averaged over the background dataset) and the current model output. Note that for the 'zero' image the blank middle is important, while for the 'four' image the lack of a connection on top makes it a four instead of a nine."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m9EOkaXJz60y",
        "colab_type": "text"
      },
      "source": [
        "## Sentiment Analysis\n",
        "\n",
        "To get an overview of which features are most important for a model we can plot the SHAP values of every feature for every sample. The plot sorts features by the sum of SHAP value magnitudes over all samples, and uses SHAP values to show the distribution of the impacts each feature has on the model output. The color represents the feature value (red high, blue low). "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PM0l_qBMz60z",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import sklearn\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.model_selection import train_test_split\n",
        "import numpy as np\n",
        "import shap\n",
        "\n",
        "shap.initjs()\n",
        "\n",
        "corpus,y = shap.datasets.imdb()\n",
        "corpus_train, corpus_test, y_train, y_test = train_test_split(corpus, y, test_size=0.2, random_state=7)\n",
        "\n",
        "vectorizer = TfidfVectorizer(min_df=10)\n",
        "X_train = vectorizer.fit_transform(corpus_train)\n",
        "X_test = vectorizer.transform(corpus_test)\n",
        "\n",
        "model = sklearn.linear_model.LogisticRegression(penalty=\"l2\", C=0.1)\n",
        "model.fit(X_train, y_train)\n",
        "\n",
        "explainer = shap.LinearExplainer(model, X_train, feature_perturbation=\"interventional\")\n",
        "shap_values = explainer.shap_values(X_test)\n",
        "X_test_array = X_test.toarray() # we need to pass a dense version for the plotting functions\n",
        "\n",
        "shap.summary_plot(shap_values, X_test_array, feature_names=vectorizer.get_feature_names())"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}