COVID-19 Antibody Test Result Bayesian Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bayesian Analysis on COVID-19 Antibody Test Results\n",
    "\n",
    "A simple workbook to determine the probability of an antibody test result matching the truth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bayes's Theorem allows one to calculate a conditional probability based on the information available:\n",
    "\n",
    "$$P(A|B)=\\frac{P(B|A)P(A)}{P(B)}$$\n",
    "\n",
    "In the context of COVID-19 antibody tests you can use it to calculate the probabilities of the four classifications the antibody test can generate:\n",
    "\n",
    "Test Result|COVID-19|Not COVID-19\n",
    ":---:|:---:|:---:\n",
    "Test + | Tests Positive and has COVID-19 | Tests Positive and Doesn't have COVID-19\n",
    "Test - | Tests Negative and has COVID-19 | Tests Negative and Doesn't have COVID-19\n",
    "\n",
    "These correspond to the following Bayesian probabilities:\n",
    "* $P(\\textrm{COVID-19 } | \\textrm{ Test } +)$\n",
    "* $P(\\textrm{COVID-19 } | \\textrm{ Test } -)$\n",
    "* $P(\\textrm{No COVID-19 } | \\textrm{ Test } +)$\n",
    "* $P(\\textrm{No COVID-19 } | \\textrm{ Test } -)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bayes_theorem(p_a, p_b, p_ba):\n",
    "    return round((p_ba * p_a) / p_b, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function will calculate the general population's expected distribution of COVID-19 given the prevalence.\n",
    "It also will calculate the expected number of true positive, true negative, false positive and false negative results expected for a given test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns = ['COVID-19', 'Not COVID-19', 'Total']\n",
    "index = ['Test +', 'Test -', 'Total']\n",
    "\n",
    "def case_generator(p, sens, spec, n=10000, columns=columns, index=index):\n",
    "    '''Takes prevalence (p - decimal) and the antibody test's sensitivity \n",
    "    (true positive detection rate; sens - decimal) and specificity (true \n",
    "    negative detection rate; spec - decimal) and generates a sample population\n",
    "    of n. Returns a dataframe mostly for style'''\n",
    "    \n",
    "    # determine number of COVID-19 positive and negative\n",
    "    cvp = round(n * p)\n",
    "    cvn = round(n * (1 - p))\n",
    "    \n",
    "    # determine expected test results of COVID-19 +\n",
    "    tp = round(sens * cvp)\n",
    "    fn = round((1 - sens) * cvp)\n",
    "    \n",
    "    # determine expected test results of COVID-19 -\n",
    "    fp = round((1 - spec) * cvn)\n",
    "    tn = round(spec * cvn)\n",
    "    \n",
    "    # sum test results by result\n",
    "    tot_p = tp + fp\n",
    "    tot_n = tn + fn\n",
    "    \n",
    "    return pd.DataFrame(\n",
    "        [[tp, fp, tot_p],\n",
    "         [fn, tn, tot_n],\n",
    "         [cvp, cvn, n]],\n",
    "        columns=columns,\n",
    "        index=index\n",
    "    )\n",
    "\n",
    "def probability_calc(p, sens, spec):\n",
    "    '''Determines the Bayesian probabilities for having COVID-19 and a \n",
    "    Positive Test, and having COVID-19 and Negative Test'''\n",
    "    \n",
    "    # Sick with positive test, sick with negative test\n",
    "    p_cp = bayes_theorem(p,\n",
    "                      sens * p  + (1 - spec),\n",
    "                      sens\n",
    "                     )\n",
    "    p_cn = bayes_theorem(p,\n",
    "                      ((1 - p) * spec + (1 - sens) * p),\n",
    "                      (1 - sens) * (1 - p)\n",
    "                     )\n",
    "    # True Positive, True Negative\n",
    "    print(f'True Positive: P(COVID-19 | Test Positive) = {p_cp}')\n",
    "    print(f'True Negative: P(No COVID-19 | Test Negative) = {1 - p_cn}')\n",
    "    # False Negative, False Positive\n",
    "    print(f'False Negative: P(COVID-19 | Test Negative) = {p_cn}')\n",
    "    print(f'False Positive: P(No COVID-19 | Test Positive) = {1 - p_cp}')\n",
    "    \n",
    "    return None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring Results in Specific States\n",
    "\n",
    "### Indiana\n",
    "For [Indiana](https://www.reddit.com/r/science/comments/hdy7yc/a_new_study_from_penn_state_estimates_that_the/fvpil39?utm_source=share&utm_medium=web2x) on 6/23, the estimated percent of infected was 5.6% 95% CL(3.1, 8.3) from [covid-19-projections.com](https://covid19-projections.com/us-in).\n",
    "\n",
    "For the Antibody test, I used the one my friend was taking since he did the research to see which had the best specificity and sensitivity:\n",
    "\n",
    "Antibody | Performance Measure | Estimate of Performance | 95% Confidence Interval\n",
    ":---:|:---:|:---:|:---:\n",
    "Pan-Ig | Sensitivity (PPA) | 100% (49/49) | (92.7%; 100%)\n",
    "Pan-Ig | Specificity (NPA) | 100% (400/400) | (99.0%; 100%)\n",
    "Pan-Ig | PPV at prevalence = 5% | 100% | (83.0.%; 100%)\n",
    "Pan-Ig | NPV at prevalence = 5% | 100% |  (99.6%; 100%)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### At 3.1% prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>287</td>\n",
       "      <td>97</td>\n",
       "      <td>384</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>23</td>\n",
       "      <td>9593</td>\n",
       "      <td>9616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>310</td>\n",
       "      <td>9690</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +       287            97    384\n",
       "Test -        23          9593   9616\n",
       "Total        310          9690  10000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.031, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.74185\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.99772\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.00228\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.25815\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.031, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### At 5.6% prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>519</td>\n",
       "      <td>94</td>\n",
       "      <td>613</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>41</td>\n",
       "      <td>9346</td>\n",
       "      <td>9387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>560</td>\n",
       "      <td>9440</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +       519            94    613\n",
       "Test -        41          9346   9387\n",
       "Total        560          9440  10000"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.056, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.83848\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.99589\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.00411\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.16152\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.056, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### At 8.3% prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>769</td>\n",
       "      <td>92</td>\n",
       "      <td>861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>61</td>\n",
       "      <td>9078</td>\n",
       "      <td>9139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>830</td>\n",
       "      <td>9170</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +       769            92    861\n",
       "Test -        61          9078   9139\n",
       "Total        830          9170  10000"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.083, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.88498\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.99392\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.00608\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.11502000000000001\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.083, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NJ\n",
    "\n",
    "[NJ predictions](https://covid19-projections.com/us-nj) are 16.0% 95% CL(10.4%, 23.1%)\n",
    "\n",
    "#### At 10.4% prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>964</td>\n",
       "      <td>90</td>\n",
       "      <td>1054</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>76</td>\n",
       "      <td>8870</td>\n",
       "      <td>8946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>1040</td>\n",
       "      <td>8960</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +       964            90   1054\n",
       "Test -        76          8870   8946\n",
       "Total       1040          8960  10000"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.104, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.90602\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.9924\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.0076\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.09397999999999995\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.104, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### At 16.0% prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>1483</td>\n",
       "      <td>84</td>\n",
       "      <td>1567</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>117</td>\n",
       "      <td>8316</td>\n",
       "      <td>8433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>1600</td>\n",
       "      <td>8400</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +      1483            84   1567\n",
       "Test -       117          8316   8433\n",
       "Total       1600          8400  10000"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.16, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.93684\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.98837\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.01163\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.06316\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.16, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### At 23.1% Prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>2141</td>\n",
       "      <td>77</td>\n",
       "      <td>2218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>169</td>\n",
       "      <td>7613</td>\n",
       "      <td>7782</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>2310</td>\n",
       "      <td>7690</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +      2141            77   2218\n",
       "Test -       169          7613   7782\n",
       "Total       2310          7690  10000"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.231, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.95538\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.98334\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.01666\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.04461999999999999\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.231, 0.927, 0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IL\n",
    "\n",
    "[IL predictions](https://covid19-projections.com/us-il) are 7.7% 95% CL(5.1%, 10.5%)\n",
    "\n",
    "These examples done with [Roche's test](https://diagnostics.roche.com/us/en/roche-blog/key-role-of-specificity-in-covid-19-antibody-test-accuracy.html): sensitivity of 100%; specificity of 99.81%. *Note*: these values have come under fire recently and [independent tests found significantly lower accuracies](https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/891598/Evaluation_of_Roche_Elecsys_anti_SARS_CoV_2_PHE_200610_v8.1_FINAL.pdf) (sensitivity of 87%; specificity of 100%). We'll use the lower values to err on the side of caution.\n",
    "\n",
    "#### At 5.1% prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>444</td>\n",
       "      <td>0</td>\n",
       "      <td>444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>66</td>\n",
       "      <td>9490</td>\n",
       "      <td>9556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>510</td>\n",
       "      <td>9490</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +       444             0    444\n",
       "Test -        66          9490   9556\n",
       "Total        510          9490  10000"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.051, 0.87, 1.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 1.0\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.99342\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.00658\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.0\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.051, 0.87, 1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### At 7.7% prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>670</td>\n",
       "      <td>0</td>\n",
       "      <td>670</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>100</td>\n",
       "      <td>9230</td>\n",
       "      <td>9330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>770</td>\n",
       "      <td>9230</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +       670             0    670\n",
       "Test -       100          9230   9330\n",
       "Total        770          9230  10000"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.077, 0.87, 1.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 1.0\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.9901\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.0099\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.0\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.077, 0.87, 1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### At 10.5% Prevalence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>914</td>\n",
       "      <td>0</td>\n",
       "      <td>914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>136</td>\n",
       "      <td>8950</td>\n",
       "      <td>9086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>1050</td>\n",
       "      <td>8950</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +       914             0    914\n",
       "Test -       136          8950   9086\n",
       "Total       1050          8950  10000"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.105, 0.87, 1.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 1.0\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.98656\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.01344\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.0\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.105, 0.87, 1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validation\n",
    "\n",
    "Tested against examples from [Wayne W. LaMorte, MD, PhD, MPH, Boston University School of Public Health](https://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Probability/BS704_Probability6.html#:~:text=Bayes%2C%20who%20was%20a%20reverend,the%20prevalence%20in%20the%20population) and [Coronavirus Antibody Tests Have a Mathematical Pitfall](https://www.scientificamerican.com/article/coronavirus-antibody-tests-have-a-mathematical-pitfall/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>198</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>1</td>\n",
       "      <td>9801</td>\n",
       "      <td>9802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>100</td>\n",
       "      <td>9900</td>\n",
       "      <td>10000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +        99            99    198\n",
       "Test -         1          9801   9802\n",
       "Total        100          9900  10000"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.01,0.99,0.99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.49749\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.9999\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.0001\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.50251\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.01, 0.99, 0.99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>170</td>\n",
       "      <td>7830</td>\n",
       "      <td>8000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>30</td>\n",
       "      <td>91970</td>\n",
       "      <td>92000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>200</td>\n",
       "      <td>99800</td>\n",
       "      <td>100000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19   Total\n",
       "Test +       170          7830    8000\n",
       "Test -        30         91970   92000\n",
       "Total        200         99800  100000"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.002, 0.85, 0.92154309, n=100000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.02121\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.99967\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.00033\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.97879\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.002, 0.85, 0.92154309)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>1</td>\n",
       "      <td>451</td>\n",
       "      <td>452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>25</td>\n",
       "      <td>475</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +        24            24     48\n",
       "Test -         1           451    452\n",
       "Total         25           475    500"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.05, 0.95, 0.95, n = 500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.48718\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.99738\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.00262\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.51282\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.05, 0.95, 0.95)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COVID-19</th>\n",
       "      <th>Not COVID-19</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Test +</td>\n",
       "      <td>119</td>\n",
       "      <td>19</td>\n",
       "      <td>138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Test -</td>\n",
       "      <td>6</td>\n",
       "      <td>356</td>\n",
       "      <td>362</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Total</td>\n",
       "      <td>125</td>\n",
       "      <td>375</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        COVID-19  Not COVID-19  Total\n",
       "Test +       119            19    138\n",
       "Test -         6           356    362\n",
       "Total        125           375    500"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case_generator(0.25, 0.95, 0.95, n = 500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positive: P(COVID-19 | Test Positive) = 0.82609\n",
      "True Negative: P(No COVID-19 | Test Negative) = 0.98707\n",
      "False Negative: P(COVID-19 | Test Negative) = 0.01293\n",
      "False Positive: P(No COVID-19 | Test Positive) = 0.17391\n"
     ]
    }
   ],
   "source": [
    "probability_calc(0.25, 0.95, 0.95)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}