DataCheckpoint_Group110_WI24.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f69d488e",
   "metadata": {},
   "source": [
    "<h1>Checkpoint #1: Data</h1>\n",
    "\n",
    "\n",
    "# NAMES: <br>\n",
    "Sophia Ashraf <br>\n",
    "Dylan Oquendo <br>\n",
    "Karun Mokha <br>\n",
    "Jake Kondo <br>\n",
    "Ekrem Ersoz\n",
    "\n",
    "\n",
    "# RESEARCH QUESTION:\n",
    "\n",
    "\n",
    "**How did the interplay between mental health and specific Covid-19-related events influence pregnancy outcomes, and what novel patterns emerge when comparing pre-pandemic, pandemic, and post-vaccine introduction phases?**\n",
    "\n",
    "**Sub Questions:**\n",
    "\n",
    "**Normal time data (overall population):**<br>\n",
    "Analyze pregnancy outcomes data from periods before the COVID-19 pandemic as a baseline. This will help establish what \"normal\" outcomes look like, against which pandemic-era outcomes can be compared.<br>\n",
    "\n",
    "**Was it related to mental health:**<br>\n",
    "Investigate whether changes in pregnancy outcomes during the COVID-19 pandemic correlate with reported changes in mental health statistics. This involves collecting data on mental health issues among pregnant individuals during the pandemic and comparing these with pregnancy outcomes.<br>\n",
    "\n",
    "**How did these relate to large events (e.g., vaccines)?**<br>\n",
    "Examine the timeline of major Covid-19-related events, such as lockdowns, infection waves, and the introduction of vaccines, and analyze their impact on mental health and pregnancy outcomes. This could involve comparing pregnancy outcomes and mental health data before and after such events to identify any significant changes or trends.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# DATASET(S):\n",
    "\n",
    "In our study, we have focused on a comprehensive selection of variables that represent three pivotal themes: mental health status, pregnancy outcomes, and the COVID-19 pandemic's impact, guided by data from a highly regarded source detailed in the dataset article at https://doi.org/10.1016/j.dib.2023.109366. The mental health indicators include stress, anxiety, and depression levels, measured by validated scales such as the Edinburgh Postnatal Depression Scale (EPDS) and the PROMIS Anxiety scores, to assess the psychological well-being of pregnant individuals during different phases of the pandemic. Additionally, we measure pregnancy outcomes through clinical metrics like gestational age at birth, birth weight, and NICU stay, reflecting the health of both mother and newborn. To specifically examine the pandemic's effects, we consider variables such as perceived threat levels to life and unborn baby, aligning with key events like lockdowns and vaccine rollouts. Our selection also includes socio-economic and demographic factors—maternal age, household income, and maternal education—to address the broader social and economic influences on child development and mental health, acknowledging the critical role these play in access to resources and opportunities.\n",
    "\n",
    "Building upon the findings of the research article accessible via https://doi.org/10.1016/j.jad.2021.12.057, which explored the influence of mental health on pregnancy outcomes using the same dataset, our analysis aims to reveal new patterns by segmenting data across different pandemic phases: pre-pandemic, during pandemic, and post-vaccine introduction. We employ advanced analytical methods, including temporal analysis and interaction effects, to delve into the dynamics of mental health fluctuations in response to specific COVID-19-related events and their effects on pregnancy outcomes. Our goal is to identify critical periods within the pandemic that significantly impacted mental health and pregnancy outcomes, and to understand the mediating role of socio-economic factors. This approach enables us to extend beyond the scope of the original study, offering novel insights into the complex interplay between mental health, socio-economic variables, and the progression of the COVID-19 pandemic.\n",
    "\n",
    "\n",
    "**SETUP:** \n",
    "\n",
    "- import pandas as pd\n",
    "- import numpy as np\n",
    "\n",
    "**DATA CLEANING:** \n",
    "\n",
    "The data was moderately clean, however we did need to remove rows with null values in the columns we deemed important to answering our questions, as these observations would not help us in answering them.\n",
    "We also renamed columns, and changed some column entries to be simpler and uniform for readability and efficiency. Some entries were made into percentages since they were scores out of 100, and we changed birth date to be a datetime variable.\n",
    "Before making these changes however, we had to check the datatype of the columns of the dataframe, and also check the unique entries of the categorical data in order to rename and clean them up, or convert in some cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "b99d98c8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OSF_ID</th>\n",
       "      <th>Maternal_Age</th>\n",
       "      <th>Household_Income</th>\n",
       "      <th>Maternal_Education</th>\n",
       "      <th>Edinburgh_Postnatal_Depression_Scale</th>\n",
       "      <th>PROMIS_Anxiety</th>\n",
       "      <th>Gestational_Age_At_Birth</th>\n",
       "      <th>Delivery_Date(converted to month and year)</th>\n",
       "      <th>Birth_Length</th>\n",
       "      <th>Birth_Weight</th>\n",
       "      <th>Delivery_Mode</th>\n",
       "      <th>NICU_Stay</th>\n",
       "      <th>Language</th>\n",
       "      <th>Threaten_Life</th>\n",
       "      <th>Threaten_Baby_Danger</th>\n",
       "      <th>Threaten_Baby_Harm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>38.3</td>\n",
       "      <td>$200,000+</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>39.71</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>49.20</td>\n",
       "      <td>3431.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>English</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>34.6</td>\n",
       "      <td>$200,000+</td>\n",
       "      <td>Undergraduate degree</td>\n",
       "      <td>4.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>English</td>\n",
       "      <td>2</td>\n",
       "      <td>33</td>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>34.3</td>\n",
       "      <td>$100,000 -$124,999</td>\n",
       "      <td>Undergraduate degree</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>French</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>28.8</td>\n",
       "      <td>$100,000 -$124,999</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.57</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>41.00</td>\n",
       "      <td>2534.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>French</td>\n",
       "      <td>53</td>\n",
       "      <td>67</td>\n",
       "      <td>54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>36.5</td>\n",
       "      <td>$40,000-$69,999</td>\n",
       "      <td>Undergraduate degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>39.86</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>53.34</td>\n",
       "      <td>3714.0</td>\n",
       "      <td>Caesarean-section (c-section)</td>\n",
       "      <td>No</td>\n",
       "      <td>English</td>\n",
       "      <td>23</td>\n",
       "      <td>32</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   OSF_ID  Maternal_Age    Household_Income    Maternal_Education  \\\n",
       "0       1          38.3           $200,000+        Masters degree   \n",
       "1       2          34.6           $200,000+  Undergraduate degree   \n",
       "2       3          34.3  $100,000 -$124,999  Undergraduate degree   \n",
       "3       4          28.8  $100,000 -$124,999        Masters degree   \n",
       "4       5          36.5     $40,000-$69,999  Undergraduate degree   \n",
       "\n",
       "   Edinburgh_Postnatal_Depression_Scale  PROMIS_Anxiety  \\\n",
       "0                                   9.0            13.0   \n",
       "1                                   4.0            17.0   \n",
       "2                                   NaN             NaN   \n",
       "3                                   9.0            20.0   \n",
       "4                                  14.0            20.0   \n",
       "\n",
       "   Gestational_Age_At_Birth Delivery_Date(converted to month and year)  \\\n",
       "0                     39.71                                    Dec2020   \n",
       "1                       NaN                                        NaN   \n",
       "2                       NaN                                        NaN   \n",
       "3                     38.57                                    Dec2020   \n",
       "4                     39.86                                    Oct2020   \n",
       "\n",
       "   Birth_Length  Birth_Weight                  Delivery_Mode NICU_Stay  \\\n",
       "0         49.20        3431.0                      Vaginally        No   \n",
       "1           NaN           NaN                            NaN       NaN   \n",
       "2           NaN           NaN                            NaN       NaN   \n",
       "3         41.00        2534.0                      Vaginally        No   \n",
       "4         53.34        3714.0  Caesarean-section (c-section)        No   \n",
       "\n",
       "  Language Threaten_Life Threaten_Baby_Danger Threaten_Baby_Harm  \n",
       "0  English             2                    3                 27  \n",
       "1  English             2                   33                 92  \n",
       "2   French                                                        \n",
       "3   French            53                   67                 54  \n",
       "4  English            23                   32                 71  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# Load the dataset\n",
    "\n",
    "file_path = 'Pregnancy During the COVID-19 Pandemic.csv'\n",
    "df = pd.read_csv(file_path)\n",
    "\n",
    "# Display the first few rows to understand the structure of the dataset\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "53b787bb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5176, 16)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#drop OSF ID\n",
    "\n",
    "\n",
    "\n",
    "df = df.dropna(subset=['PROMIS_Anxiety', 'Birth_Length', 'Birth_Weight', 'NICU_Stay', 'Edinburgh_Postnatal_Depression_Scale'])\n",
    "#Drop any row with null value for the values we are most interested in\n",
    "df = df.reset_index(drop=True)\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "135e72b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "#drop language column\n",
    "\n",
    "df = df.drop('Language', axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "15345416",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OSF_ID                                         0\n",
       "Maternal_Age                                   3\n",
       "Household_Income                              20\n",
       "Maternal_Education                            14\n",
       "Edinburgh_Postnatal_Depression_Scale           0\n",
       "PROMIS_Anxiety                                 0\n",
       "Gestational_Age_At_Birth                       0\n",
       "Delivery_Date(converted to month and year)     0\n",
       "Birth_Length                                   0\n",
       "Birth_Weight                                   0\n",
       "Delivery_Mode                                  0\n",
       "NICU_Stay                                      0\n",
       "Threaten_Life                                  0\n",
       "Threaten_Baby_Danger                           0\n",
       "Threaten_Baby_Harm                             0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking for missing values\n",
    "missing_values = df.isnull().sum()\n",
    "\n",
    "missing_values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "0fe325a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OSF_ID</th>\n",
       "      <th>mat_age</th>\n",
       "      <th>income</th>\n",
       "      <th>mat_edu</th>\n",
       "      <th>depression</th>\n",
       "      <th>anxiety</th>\n",
       "      <th>birth_age</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>Birth_Length</th>\n",
       "      <th>Birth_Weight</th>\n",
       "      <th>Delivery_Mode</th>\n",
       "      <th>NICU_Stay</th>\n",
       "      <th>Threaten_Life</th>\n",
       "      <th>Threaten_Baby_Danger</th>\n",
       "      <th>Threaten_Baby_Harm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>38.3</td>\n",
       "      <td>$200,000+</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>39.71</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>49.20</td>\n",
       "      <td>3431.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>28.8</td>\n",
       "      <td>$100,000 -$124,999</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.57</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>41.00</td>\n",
       "      <td>2534.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>53</td>\n",
       "      <td>67</td>\n",
       "      <td>54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>36.5</td>\n",
       "      <td>$40,000-$69,999</td>\n",
       "      <td>Undergraduate degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>39.86</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>53.34</td>\n",
       "      <td>3714.0</td>\n",
       "      <td>Caesarean-section (c-section)</td>\n",
       "      <td>No</td>\n",
       "      <td>23</td>\n",
       "      <td>32</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9</td>\n",
       "      <td>33.1</td>\n",
       "      <td>$100,000 -$124,999</td>\n",
       "      <td>College/trade school</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>40.86</td>\n",
       "      <td>Nov2020</td>\n",
       "      <td>55.88</td>\n",
       "      <td>4480.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>27</td>\n",
       "      <td>76</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14</td>\n",
       "      <td>29.2</td>\n",
       "      <td>$70,000-$99,999</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>41.00</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>47.00</td>\n",
       "      <td>3084.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>68</td>\n",
       "      <td>69</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   OSF_ID  mat_age              income               mat_edu  depression  \\\n",
       "0       1     38.3           $200,000+        Masters degree         9.0   \n",
       "1       4     28.8  $100,000 -$124,999        Masters degree         9.0   \n",
       "2       5     36.5     $40,000-$69,999  Undergraduate degree        14.0   \n",
       "3       9     33.1  $100,000 -$124,999  College/trade school         1.0   \n",
       "4      14     29.2     $70,000-$99,999        Masters degree        14.0   \n",
       "\n",
       "   anxiety  birth_age birth_date  Birth_Length  Birth_Weight  \\\n",
       "0     13.0      39.71    Dec2020         49.20        3431.0   \n",
       "1     20.0      38.57    Dec2020         41.00        2534.0   \n",
       "2     20.0      39.86    Oct2020         53.34        3714.0   \n",
       "3      7.0      40.86    Nov2020         55.88        4480.0   \n",
       "4     17.0      41.00    Oct2020         47.00        3084.0   \n",
       "\n",
       "                   Delivery_Mode NICU_Stay Threaten_Life Threaten_Baby_Danger  \\\n",
       "0                      Vaginally        No             2                    3   \n",
       "1                      Vaginally        No            53                   67   \n",
       "2  Caesarean-section (c-section)        No            23                   32   \n",
       "3                      Vaginally        No            27                   76   \n",
       "4                      Vaginally        No            68                   69   \n",
       "\n",
       "  Threaten_Baby_Harm  \n",
       "0                 27  \n",
       "1                 54  \n",
       "2                 71  \n",
       "3                 72  \n",
       "4                 81  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.rename(columns={'Maternal_Age': 'mat_age', 'Household_Income': 'income', 'Maternal_Education': 'mat_edu',\n",
    "                  'Edinburgh_Postnatal_Depression_Scale': 'depression',\n",
    "                  'PROMIS_Anxiety': 'anxiety', 'Gestational_Age_At_Birth': 'birth_age', \n",
    "                  'Delivery_Date(converted to month and year)': 'birth_date'}, inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "7bf0cddb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(             OSF_ID      mat_age   depression      anxiety   birth_age  \\\n",
       " count   5176.000000  5173.000000  5176.000000  5176.000000  5176.00000   \n",
       " mean    5300.733578    32.521322     9.738022    18.389104    39.33868   \n",
       " std     3114.246816     4.140823     5.307232     5.950169     1.62486   \n",
       " min        1.000000    18.500000     0.000000     7.000000    24.86000   \n",
       " 25%     2560.750000    29.700000     6.000000    14.000000    38.57000   \n",
       " 50%     5294.500000    32.400000    10.000000    19.000000    39.57000   \n",
       " 75%     8009.250000    35.300000    13.000000    23.000000    40.43000   \n",
       " max    10764.000000    49.000000    28.000000    35.000000    42.86000   \n",
       " \n",
       "        Birth_Length  Birth_Weight  \n",
       " count   5176.000000   5176.000000  \n",
       " mean      50.499834   3412.676005  \n",
       " std        4.433899    534.564742  \n",
       " min       20.000000    314.000000  \n",
       " 25%       49.000000   3119.000000  \n",
       " 50%       50.800000   3431.000000  \n",
       " 75%       53.310000   3742.000000  \n",
       " max       70.000000   5968.000000  ,\n",
       " OSF_ID                                         0\n",
       " Maternal_Age                                   3\n",
       " Household_Income                              20\n",
       " Maternal_Education                            14\n",
       " Edinburgh_Postnatal_Depression_Scale           0\n",
       " PROMIS_Anxiety                                 0\n",
       " Gestational_Age_At_Birth                       0\n",
       " Delivery_Date(converted to month and year)     0\n",
       " Birth_Length                                   0\n",
       " Birth_Weight                                   0\n",
       " Delivery_Mode                                  0\n",
       " NICU_Stay                                      0\n",
       " Threaten_Life                                  0\n",
       " Threaten_Baby_Danger                           0\n",
       " Threaten_Baby_Harm                             0\n",
       " dtype: int64)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Summary statistics for numerical columns\n",
    "summary_statistics = df.describe()\n",
    "\n",
    "\n",
    "summary_statistics, missing_values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "a0183010",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OSF_ID</th>\n",
       "      <th>mat_age</th>\n",
       "      <th>income</th>\n",
       "      <th>mat_edu</th>\n",
       "      <th>depression</th>\n",
       "      <th>anxiety</th>\n",
       "      <th>birth_age</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>Birth_Length</th>\n",
       "      <th>Birth_Weight</th>\n",
       "      <th>Delivery_Mode</th>\n",
       "      <th>NICU_Stay</th>\n",
       "      <th>Threaten_Life</th>\n",
       "      <th>Threaten_Baby_Danger</th>\n",
       "      <th>Threaten_Baby_Harm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>38.3</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>39.71</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>49.20</td>\n",
       "      <td>3431.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>28.8</td>\n",
       "      <td>112499.5</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.57</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>41.00</td>\n",
       "      <td>2534.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>53</td>\n",
       "      <td>67</td>\n",
       "      <td>54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>36.5</td>\n",
       "      <td>54999.5</td>\n",
       "      <td>Undergraduate degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>39.86</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>53.34</td>\n",
       "      <td>3714.0</td>\n",
       "      <td>Caesarean-section (c-section)</td>\n",
       "      <td>No</td>\n",
       "      <td>23</td>\n",
       "      <td>32</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9</td>\n",
       "      <td>33.1</td>\n",
       "      <td>112499.5</td>\n",
       "      <td>College/trade school</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>40.86</td>\n",
       "      <td>Nov2020</td>\n",
       "      <td>55.88</td>\n",
       "      <td>4480.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>27</td>\n",
       "      <td>76</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14</td>\n",
       "      <td>29.2</td>\n",
       "      <td>84999.5</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>41.00</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>47.00</td>\n",
       "      <td>3084.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>68</td>\n",
       "      <td>69</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   OSF_ID  mat_age    income               mat_edu  depression  anxiety  \\\n",
       "0       1     38.3  250000.0        Masters degree         9.0     13.0   \n",
       "1       4     28.8  112499.5        Masters degree         9.0     20.0   \n",
       "2       5     36.5   54999.5  Undergraduate degree        14.0     20.0   \n",
       "3       9     33.1  112499.5  College/trade school         1.0      7.0   \n",
       "4      14     29.2   84999.5        Masters degree        14.0     17.0   \n",
       "\n",
       "   birth_age birth_date  Birth_Length  Birth_Weight  \\\n",
       "0      39.71    Dec2020         49.20        3431.0   \n",
       "1      38.57    Dec2020         41.00        2534.0   \n",
       "2      39.86    Oct2020         53.34        3714.0   \n",
       "3      40.86    Nov2020         55.88        4480.0   \n",
       "4      41.00    Oct2020         47.00        3084.0   \n",
       "\n",
       "                   Delivery_Mode NICU_Stay Threaten_Life Threaten_Baby_Danger  \\\n",
       "0                      Vaginally        No             2                    3   \n",
       "1                      Vaginally        No            53                   67   \n",
       "2  Caesarean-section (c-section)        No            23                   32   \n",
       "3                      Vaginally        No            27                   76   \n",
       "4                      Vaginally        No            68                   69   \n",
       "\n",
       "  Threaten_Baby_Harm  \n",
       "0                 27  \n",
       "1                 54  \n",
       "2                 71  \n",
       "3                 72  \n",
       "4                 81  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def standardize_income(income):\n",
    "    if pd.isna(income):\n",
    "        # Return NaN as is, you can also choose to fill it with a specific value if required\n",
    "        return np.nan\n",
    "    elif isinstance(income, str):\n",
    "        # Check for non-standard strings and convert them\n",
    "        if 'Less than' in income:\n",
    "            return 20000  # Example value, adjust based on your dataset\n",
    "        # Check if income is a range\n",
    "        elif '-' in income:\n",
    "            parts = income.replace('$', '').replace(',', '').split('-')\n",
    "            # Calculate midpoint for ranges\n",
    "            if len(parts) == 2 and parts[1]:\n",
    "                low, high = map(int, parts)\n",
    "                return (low + high) / 2\n",
    "            else:  # Handle cases like '$150,000 -'\n",
    "                low = int(parts[0])\n",
    "                return low * 1.25\n",
    "        elif '+' in income:\n",
    "            # Handle open-ended values like '$200,000+'\n",
    "            low = int(income.replace('$', '').replace(',', '').replace('+', ''))\n",
    "            return low * 1.25\n",
    "        else:\n",
    "            # Handle single values without range\n",
    "            return int(income.replace('$', '').replace(',', '').replace(' ', ''))\n",
    "    else:\n",
    "        # If income is already a number, just return it\n",
    "        return income\n",
    "\n",
    "# Assuming 'df' is your dataframe\n",
    "df['income'] = df['income'].apply(standardize_income)\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "a44a78c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OSF_ID</th>\n",
       "      <th>mat_age</th>\n",
       "      <th>income</th>\n",
       "      <th>mat_edu</th>\n",
       "      <th>depression</th>\n",
       "      <th>anxiety</th>\n",
       "      <th>birth_age</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>Birth_Length</th>\n",
       "      <th>Birth_Weight</th>\n",
       "      <th>Delivery_Mode</th>\n",
       "      <th>NICU_Stay</th>\n",
       "      <th>Threaten_Life</th>\n",
       "      <th>Threaten_Baby_Danger</th>\n",
       "      <th>Threaten_Baby_Harm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>38.3</td>\n",
       "      <td>High</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>39.71</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>49.20</td>\n",
       "      <td>3431.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>28.8</td>\n",
       "      <td>Middle</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.57</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>41.00</td>\n",
       "      <td>2534.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>53</td>\n",
       "      <td>67</td>\n",
       "      <td>54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>36.5</td>\n",
       "      <td>Low</td>\n",
       "      <td>Undergraduate degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>39.86</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>53.34</td>\n",
       "      <td>3714.0</td>\n",
       "      <td>Caesarean-section (c-section)</td>\n",
       "      <td>No</td>\n",
       "      <td>23</td>\n",
       "      <td>32</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9</td>\n",
       "      <td>33.1</td>\n",
       "      <td>Middle</td>\n",
       "      <td>College/trade school</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>40.86</td>\n",
       "      <td>Nov2020</td>\n",
       "      <td>55.88</td>\n",
       "      <td>4480.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>27</td>\n",
       "      <td>76</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14</td>\n",
       "      <td>29.2</td>\n",
       "      <td>Middle</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>41.00</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>47.00</td>\n",
       "      <td>3084.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>No</td>\n",
       "      <td>68</td>\n",
       "      <td>69</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   OSF_ID  mat_age  income               mat_edu  depression  anxiety  \\\n",
       "0       1     38.3    High        Masters degree         9.0     13.0   \n",
       "1       4     28.8  Middle        Masters degree         9.0     20.0   \n",
       "2       5     36.5     Low  Undergraduate degree        14.0     20.0   \n",
       "3       9     33.1  Middle  College/trade school         1.0      7.0   \n",
       "4      14     29.2  Middle        Masters degree        14.0     17.0   \n",
       "\n",
       "   birth_age birth_date  Birth_Length  Birth_Weight  \\\n",
       "0      39.71    Dec2020         49.20        3431.0   \n",
       "1      38.57    Dec2020         41.00        2534.0   \n",
       "2      39.86    Oct2020         53.34        3714.0   \n",
       "3      40.86    Nov2020         55.88        4480.0   \n",
       "4      41.00    Oct2020         47.00        3084.0   \n",
       "\n",
       "                   Delivery_Mode NICU_Stay Threaten_Life Threaten_Baby_Danger  \\\n",
       "0                      Vaginally        No             2                    3   \n",
       "1                      Vaginally        No            53                   67   \n",
       "2  Caesarean-section (c-section)        No            23                   32   \n",
       "3                      Vaginally        No            27                   76   \n",
       "4                      Vaginally        No            68                   69   \n",
       "\n",
       "  Threaten_Baby_Harm  \n",
       "0                 27  \n",
       "1                 54  \n",
       "2                 71  \n",
       "3                 72  \n",
       "4                 81  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def categorize_income(value, low_thresh, high_thresh):\n",
    "    if pd.isna(value):\n",
    "        return 'Unknown'  # Handle NaN values as 'Unknown'\n",
    "    elif value < low_thresh:\n",
    "        return 'Low'\n",
    "    elif low_thresh <= value < high_thresh:\n",
    "        return 'Middle'\n",
    "    else:\n",
    "        return 'High'\n",
    "\n",
    "# Assuming 'df' is your dataframe and 'income' is the correct income column\n",
    "low_threshold = df['income'].quantile(0.33)\n",
    "high_threshold = df['income'].quantile(0.66)\n",
    "\n",
    "# Categorize incomes using the thresholds\n",
    "df['income'] = df['income'].apply(lambda x: categorize_income(x, low_threshold, high_threshold))\n",
    "\n",
    "# Display the updated DataFrame\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a6d22f05",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OSF_ID</th>\n",
       "      <th>mat_age</th>\n",
       "      <th>income</th>\n",
       "      <th>mat_edu</th>\n",
       "      <th>depression</th>\n",
       "      <th>anxiety</th>\n",
       "      <th>birth_age</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>Birth_Length</th>\n",
       "      <th>Birth_Weight</th>\n",
       "      <th>Delivery_Mode</th>\n",
       "      <th>NICU_Stay</th>\n",
       "      <th>Threaten_Life</th>\n",
       "      <th>Threaten_Baby_Danger</th>\n",
       "      <th>Threaten_Baby_Harm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>38.3</td>\n",
       "      <td>High</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>39.71</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>49.20</td>\n",
       "      <td>3431.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>28.8</td>\n",
       "      <td>Middle</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.57</td>\n",
       "      <td>Dec2020</td>\n",
       "      <td>41.00</td>\n",
       "      <td>2534.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>0</td>\n",
       "      <td>53</td>\n",
       "      <td>67</td>\n",
       "      <td>54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>36.5</td>\n",
       "      <td>Low</td>\n",
       "      <td>Undergraduate degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>39.86</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>53.34</td>\n",
       "      <td>3714.0</td>\n",
       "      <td>Caesarean-section (c-section)</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>32</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9</td>\n",
       "      <td>33.1</td>\n",
       "      <td>Middle</td>\n",
       "      <td>College/trade school</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>40.86</td>\n",
       "      <td>Nov2020</td>\n",
       "      <td>55.88</td>\n",
       "      <td>4480.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>76</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14</td>\n",
       "      <td>29.2</td>\n",
       "      <td>Middle</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>41.00</td>\n",
       "      <td>Oct2020</td>\n",
       "      <td>47.00</td>\n",
       "      <td>3084.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>0</td>\n",
       "      <td>68</td>\n",
       "      <td>69</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   OSF_ID  mat_age  income               mat_edu  depression  anxiety  \\\n",
       "0       1     38.3    High        Masters degree         9.0     13.0   \n",
       "1       4     28.8  Middle        Masters degree         9.0     20.0   \n",
       "2       5     36.5     Low  Undergraduate degree        14.0     20.0   \n",
       "3       9     33.1  Middle  College/trade school         1.0      7.0   \n",
       "4      14     29.2  Middle        Masters degree        14.0     17.0   \n",
       "\n",
       "   birth_age birth_date  Birth_Length  Birth_Weight  \\\n",
       "0      39.71    Dec2020         49.20        3431.0   \n",
       "1      38.57    Dec2020         41.00        2534.0   \n",
       "2      39.86    Oct2020         53.34        3714.0   \n",
       "3      40.86    Nov2020         55.88        4480.0   \n",
       "4      41.00    Oct2020         47.00        3084.0   \n",
       "\n",
       "                   Delivery_Mode  NICU_Stay Threaten_Life  \\\n",
       "0                      Vaginally          0             2   \n",
       "1                      Vaginally          0            53   \n",
       "2  Caesarean-section (c-section)          0            23   \n",
       "3                      Vaginally          0            27   \n",
       "4                      Vaginally          0            68   \n",
       "\n",
       "  Threaten_Baby_Danger Threaten_Baby_Harm  \n",
       "0                    3                 27  \n",
       "1                   67                 54  \n",
       "2                   32                 71  \n",
       "3                   76                 72  \n",
       "4                   69                 81  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert 'Yes'/'No' to 1/0 in NICU_Stay\n",
    "df['NICU_Stay'] = df['NICU_Stay'].map({'Yes': 1, 'No': 0})\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "a8d8d6fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OSF_ID</th>\n",
       "      <th>mat_age</th>\n",
       "      <th>income</th>\n",
       "      <th>mat_edu</th>\n",
       "      <th>depression</th>\n",
       "      <th>anxiety</th>\n",
       "      <th>birth_age</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>Birth_Length</th>\n",
       "      <th>Birth_Weight</th>\n",
       "      <th>Delivery_Mode</th>\n",
       "      <th>NICU_Stay</th>\n",
       "      <th>Threaten_Life</th>\n",
       "      <th>Threaten_Baby_Danger</th>\n",
       "      <th>Threaten_Baby_Harm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>38.3</td>\n",
       "      <td>High</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>39.71</td>\n",
       "      <td>NaT</td>\n",
       "      <td>49.20</td>\n",
       "      <td>3431.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>28.8</td>\n",
       "      <td>Middle</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>9.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.57</td>\n",
       "      <td>NaT</td>\n",
       "      <td>41.00</td>\n",
       "      <td>2534.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>0</td>\n",
       "      <td>53</td>\n",
       "      <td>67</td>\n",
       "      <td>54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>36.5</td>\n",
       "      <td>Low</td>\n",
       "      <td>Undergraduate degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>39.86</td>\n",
       "      <td>NaT</td>\n",
       "      <td>53.34</td>\n",
       "      <td>3714.0</td>\n",
       "      <td>Caesarean-section (c-section)</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>32</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9</td>\n",
       "      <td>33.1</td>\n",
       "      <td>Middle</td>\n",
       "      <td>College/trade school</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>40.86</td>\n",
       "      <td>NaT</td>\n",
       "      <td>55.88</td>\n",
       "      <td>4480.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>76</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14</td>\n",
       "      <td>29.2</td>\n",
       "      <td>Middle</td>\n",
       "      <td>Masters degree</td>\n",
       "      <td>14.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>41.00</td>\n",
       "      <td>NaT</td>\n",
       "      <td>47.00</td>\n",
       "      <td>3084.0</td>\n",
       "      <td>Vaginally</td>\n",
       "      <td>0</td>\n",
       "      <td>68</td>\n",
       "      <td>69</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   OSF_ID  mat_age  income               mat_edu  depression  anxiety  \\\n",
       "0       1     38.3    High        Masters degree         9.0     13.0   \n",
       "1       4     28.8  Middle        Masters degree         9.0     20.0   \n",
       "2       5     36.5     Low  Undergraduate degree        14.0     20.0   \n",
       "3       9     33.1  Middle  College/trade school         1.0      7.0   \n",
       "4      14     29.2  Middle        Masters degree        14.0     17.0   \n",
       "\n",
       "   birth_age birth_date  Birth_Length  Birth_Weight  \\\n",
       "0      39.71        NaT         49.20        3431.0   \n",
       "1      38.57        NaT         41.00        2534.0   \n",
       "2      39.86        NaT         53.34        3714.0   \n",
       "3      40.86        NaT         55.88        4480.0   \n",
       "4      41.00        NaT         47.00        3084.0   \n",
       "\n",
       "                   Delivery_Mode  NICU_Stay Threaten_Life  \\\n",
       "0                      Vaginally          0             2   \n",
       "1                      Vaginally          0            53   \n",
       "2  Caesarean-section (c-section)          0            23   \n",
       "3                      Vaginally          0            27   \n",
       "4                      Vaginally          0            68   \n",
       "\n",
       "  Threaten_Baby_Danger Threaten_Baby_Harm  \n",
       "0                    3                 27  \n",
       "1                   67                 54  \n",
       "2                   32                 71  \n",
       "3                   76                 72  \n",
       "4                   69                 81  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['birth_date'] = pd.to_datetime(df['birth_date'], format='%Y-%m-%d', errors='coerce')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "039e94db",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5176, 15)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}