docs/add-new-scraper.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "wound-toilet",
   "metadata": {},
   "source": [
    "# Implement scraper\n",
    "Sociepy's dataset consists on multiple country data sources. The first step to retrieve and construct this dataset are the country scrapers from `covid_updater` library. In this notebook we will explain how to add a new scraper to the project. To this end, we will use Finland as an example.\n",
    "\n",
    "\n",
    "We will illustrate how to (1) get the data, (2) process the data and (3) export data. Once this is clear and the code is reliable, we will integrate the logic into `covid_updater` library, specifically in `covid_updater.scraping.countries` module.\n",
    "\n",
    "\n",
    "Note that scraping is only the first part of the `update_all.py` pipeline, which is responsible of updating all project's data. More details about the whole pipeline are found [here](https://github.com/sociepy/covid19-vaccination-subnational/blob/main/scripts/update_all.sh)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "chemical-ancient",
   "metadata": {},
   "source": [
    "## 0. Fork project\n",
    "First thing is to fork the project, clone it and start editing.\n",
    "\n",
    "Remember to keep your fork [synced with upstream](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/syncing-a-fork)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "statewide-correlation",
   "metadata": {},
   "source": [
    "### 0.1 Find country candidate\n",
    "To this end, use following tool to see what is available on OWID"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "interim-spirituality",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'country': 'Bulgaria', 'source_url': 'https://coronavirus.bg/bg/statistika'},\n",
       " {'country': 'Estonia', 'source_url': 'https://www.terviseamet.ee/et/uudised'},\n",
       " {'country': 'Finland',\n",
       "  'source_url': 'https://sampo.thl.fi/pivot/prod/en/vaccreg/cov19cov/fact_cov19cov.csv?row=cov_vac_dose-533174L&column=measure-533185.533172.433796.533175&'},\n",
       " {'country': 'Indonesia', 'source_url': 'https://www.kemkes.go.id/'},\n",
       " {'country': 'Ireland', 'source_url': ''},\n",
       " {'country': 'Jersey',\n",
       "  'source_url': 'https://www.gov.je/Health/Coronavirus/Vaccine/Pages/VaccinationStatistics.aspx'},\n",
       " {'country': 'Liechtenstein',\n",
       "  'source_url': 'https://www.covid19.admin.ch/en/epidemiologic/vacc-doses?detGeo=FL'},\n",
       " {'country': 'Luxembourg',\n",
       "  'source_url': 'https://data.public.lu/fr/datasets/covid-19-rapports-journaliers/#_'},\n",
       " {'country': 'Montenegro',\n",
       "  'source_url': 'https://atlas.jifo.co/api/connectors/520021dc-c292-4903-9cdb-a2467f64ed97'},\n",
       " {'country': 'Morocco',\n",
       "  'source_url': 'http://www.covidmaroc.ma/pages/Accueilfr.aspx'},\n",
       " {'country': 'Netherlands',\n",
       "  'source_url': 'https://www.rivm.nl/covid-19-vaccinatie/wekelijkse-update-deelname-covid-19-vaccinatie-in-nederland'},\n",
       " {'country': 'Singapore', 'source_url': 'https://www.moh.gov.sg/covid-19'},\n",
       " {'country': 'Ukraine', 'source_url': 'https://vaccination.covid19.gov.ua/'},\n",
       " {'country': 'Ecuador',\n",
       "  'source_url': 'https://github.com/andrab/ecuacovid/raw/master/datos_crudos/vacunas/vacunas.csv'},\n",
       " {'country': 'Greece',\n",
       "  'source_url': 'https://data.gov.gr/api/v1/query/mdg_emvolio'},\n",
       " {'country': 'Israel',\n",
       "  'source_url': 'https://datadashboardapi.health.gov.il/api/queries/vaccinated'},\n",
       " {'country': 'Japan',\n",
       "  'source_url': 'https://www.mhlw.go.jp/stf/seisakunitsuite/bunya/vaccine_sesshujisseki.html'},\n",
       " {'country': 'Latvia', 'source_url': ''},\n",
       " {'country': 'Lithuania', 'source_url': ''},\n",
       " {'country': 'Malta',\n",
       "  'source_url': 'https://github.com/COVID19-Malta/COVID19-Cases/raw/master/COVID-19%20Malta%20-%20Vaccination%20Data.csv'},\n",
       " {'country': 'Portugal',\n",
       "  'source_url': 'https://github.com/dssg-pt/covid19pt-data/raw/master/vacinas.csv'},\n",
       " {'country': 'Romania',\n",
       "  'source_url': 'https://d35p9e4fm9h3wo.cloudfront.net/latestData.json'},\n",
       " {'country': 'Slovenia', 'source_url': ''}]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get potential candidates and urls\n",
    "from covid_updater.additions import get_owid_diff_source_urls\n",
    "get_owid_diff_source_urls()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "direct-surge",
   "metadata": {},
   "source": [
    "## 1. Get the data\n",
    "### 1.1 Format of the data\n",
    "You have to determine which is the format of the source data. Based on this, different libraries may come handy.\n",
    "\n",
    "- csv: Might want to use pandas.\n",
    "- API, json-like: Use requests, json\n",
    "- Table in static website: Use bs4, requests, urllib\n",
    "- Table in dynamic website: Use selenium, bs4, requests, urllib"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "animal-medline",
   "metadata": {},
   "source": [
    "## 1.2. Download the data\n",
    "In our case, the data is available as CSV per week. That is, for each week there is a CSV available.\n",
    "\n",
    "URLs for these links are a bit tricky to get in this particular case, some scraping is required!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "painful-tunisia",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "future-editing",
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_data(url):\n",
    "    data_raw = json.loads(requests.get(url).content)\n",
    "    dfs = [pd.DataFrame.from_records(sample[\"shotHistory\"]).assign(region=sample[\"areaName\"]) for sample in data_raw]\n",
    "    return pd.concat(dfs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "liberal-popularity",
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"https://piikki-api.lab.juiciness.io/administrations\"\n",
    "df = load_data(url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "outer-burner",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>areaId</th>\n",
       "      <th>date</th>\n",
       "      <th>firstDoseShots</th>\n",
       "      <th>secondDoseShots</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ffb7d597-9211-49ed-8570-53347cb5b304</td>\n",
       "      <td>2020-12-26T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ffb7d597-9211-49ed-8570-53347cb5b304</td>\n",
       "      <td>2020-12-27T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ffb7d597-9211-49ed-8570-53347cb5b304</td>\n",
       "      <td>2020-12-28T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ffb7d597-9211-49ed-8570-53347cb5b304</td>\n",
       "      <td>2020-12-29T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ffb7d597-9211-49ed-8570-53347cb5b304</td>\n",
       "      <td>2020-12-30T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 areaId                      date  \\\n",
       "0  ffb7d597-9211-49ed-8570-53347cb5b304  2020-12-26T00:00:00.000Z   \n",
       "1  ffb7d597-9211-49ed-8570-53347cb5b304  2020-12-27T00:00:00.000Z   \n",
       "2  ffb7d597-9211-49ed-8570-53347cb5b304  2020-12-28T00:00:00.000Z   \n",
       "3  ffb7d597-9211-49ed-8570-53347cb5b304  2020-12-29T00:00:00.000Z   \n",
       "4  ffb7d597-9211-49ed-8570-53347cb5b304  2020-12-30T00:00:00.000Z   \n",
       "\n",
       "   firstDoseShots  secondDoseShots region  \n",
       "0               0                0  Åland  \n",
       "1               0                0  Åland  \n",
       "2               0                0  Åland  \n",
       "3               0                0  Åland  \n",
       "4               0                0  Åland  "
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show some rows\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "handled-archive",
   "metadata": {},
   "source": [
    "## 2. Process data\n",
    "We are interested in processing the data such that its format is compliant with the project's format. Check [Australia.csv](https://github.com/sociepy/covid19-vaccination-subnational/blob/main/data/countries/Australia.csv) as an example. But basically, the following fields are required:\n",
    "\n",
    "- `location`: Country name.\n",
    "- `region`: Region name.\n",
    "- `date`: Date of observation.\n",
    "- `location_iso`: ISO 3166-1 alpha 2 code of country.\n",
    "- `region_iso`: ISO 3166-2 code of region\n",
    "- `total_vaccinations`: Total number of vaccinations (doses) administered.\n",
    "- `people_vaccinated` (OPTIONAL): Number of people with at least one vaccination administred. Not all countries make this data available.\n",
    "- `people_fully_vaccinated` (OPTIONAL): Number of people fully vaccinated (with all required doses administered). Not all countries make this data available."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "divided-hunter",
   "metadata": {},
   "source": [
    "### 2.1 Rename and discard fields\n",
    "From the retrieved columns, which are interesting?\n",
    "\n",
    "The following should be renamed:\n",
    "\n",
    "- `date` -> `date`\n",
    "- `region` -> `region`\n",
    "- `firstDoseShots` -> `people_vaccinated`\n",
    "- `secondDoseShots` -> `people_fully_vaccinated`\n",
    "\n",
    "And `areaId` should be removed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "distant-dialogue",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.rename(columns={\n",
    "    \"firstDoseShots\": \"people_vaccinated\",\n",
    "    \"secondDoseShots\": \"people_fully_vaccinated\",\n",
    "})\n",
    "df = df.drop(columns=\"areaId\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "interstate-enterprise",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>people_vaccinated</th>\n",
       "      <th>people_fully_vaccinated</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-12-26T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-12-27T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-12-28T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-12-29T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-12-30T00:00:00.000Z</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       date  people_vaccinated  people_fully_vaccinated region\n",
       "0  2020-12-26T00:00:00.000Z                  0                        0  Åland\n",
       "1  2020-12-27T00:00:00.000Z                  0                        0  Åland\n",
       "2  2020-12-28T00:00:00.000Z                  0                        0  Åland\n",
       "3  2020-12-29T00:00:00.000Z                  0                        0  Åland\n",
       "4  2020-12-30T00:00:00.000Z                  0                        0  Åland"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show some rows\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "norwegian-shoulder",
   "metadata": {},
   "source": [
    "### 2.2 Date field\n",
    "For our dataset it is important to have the same date format across all countries. In particular, we use \"YYYY-mm-dd\". Let us process `date` field accordingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "coordinate-clothing",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.loc[:, \"date\"] = pd.to_datetime(df.date).apply(lambda x: x.strftime(\"%Y-%m-%d\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "logical-phase",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>people_vaccinated</th>\n",
       "      <th>people_fully_vaccinated</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-12-26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-12-27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-12-28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-12-29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-12-30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Åland</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date  people_vaccinated  people_fully_vaccinated region\n",
       "0  2020-12-26                  0                        0  Åland\n",
       "1  2020-12-27                  0                        0  Åland\n",
       "2  2020-12-28                  0                        0  Åland\n",
       "3  2020-12-29                  0                        0  Åland\n",
       "4  2020-12-30                  0                        0  Åland"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show some rows\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "subsequent-valuable",
   "metadata": {},
   "source": [
    "### 2.3 Region field\n",
    "Identify region field, and make sure it makes sense with country's regions. You may want to search \"ISO 3166-2 Finland\" in Wikipedia (https://en.wikipedia.org/wiki/ISO_3166-2:FI) or \"Final Regions\" (https://en.wikipedia.org/wiki/Regions_of_Finland) for this example.\n",
    "\n",
    "Once we deem that the region field makes sense and is complete, we need to rename the regions, using the standarized names from our ISO db:\n",
    "\n",
    "- Note 1: In this example we have that some regions have more than one district.\n",
    "- Note 2: There are two regions which we will be ignoring, namely \"All areas\" and \"Other areas\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "rational-editing",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Regions to be ignored\n",
    "df = df[~df[\"region\"].isin([\"All areas\", \"Other areas\"])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "raising-picking",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>location_iso</th>\n",
       "      <th>subdivision_name</th>\n",
       "      <th>region_iso</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>805</th>\n",
       "      <td>FI</td>\n",
       "      <td>Aland</td>\n",
       "      <td>FI-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>806</th>\n",
       "      <td>FI</td>\n",
       "      <td>Etela-Karjala</td>\n",
       "      <td>FI-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>807</th>\n",
       "      <td>FI</td>\n",
       "      <td>Etela-Pohjanmaa</td>\n",
       "      <td>FI-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>808</th>\n",
       "      <td>FI</td>\n",
       "      <td>Etela-Savo</td>\n",
       "      <td>FI-04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>809</th>\n",
       "      <td>FI</td>\n",
       "      <td>Kainuu</td>\n",
       "      <td>FI-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>810</th>\n",
       "      <td>FI</td>\n",
       "      <td>Kanta-Hame</td>\n",
       "      <td>FI-06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>811</th>\n",
       "      <td>FI</td>\n",
       "      <td>Keski-Pohjanmaa</td>\n",
       "      <td>FI-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>812</th>\n",
       "      <td>FI</td>\n",
       "      <td>Keski-Suomi</td>\n",
       "      <td>FI-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>813</th>\n",
       "      <td>FI</td>\n",
       "      <td>Kymenlaakso</td>\n",
       "      <td>FI-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>814</th>\n",
       "      <td>FI</td>\n",
       "      <td>Lappi</td>\n",
       "      <td>FI-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>815</th>\n",
       "      <td>FI</td>\n",
       "      <td>Pirkanmaa</td>\n",
       "      <td>FI-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>816</th>\n",
       "      <td>FI</td>\n",
       "      <td>Pohjanmaa</td>\n",
       "      <td>FI-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>817</th>\n",
       "      <td>FI</td>\n",
       "      <td>Pohjois-Karjala</td>\n",
       "      <td>FI-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>818</th>\n",
       "      <td>FI</td>\n",
       "      <td>Pohjois-Pohjanmaa</td>\n",
       "      <td>FI-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>819</th>\n",
       "      <td>FI</td>\n",
       "      <td>Pohjois-Savo</td>\n",
       "      <td>FI-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>820</th>\n",
       "      <td>FI</td>\n",
       "      <td>Paijat-Hame</td>\n",
       "      <td>FI-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>821</th>\n",
       "      <td>FI</td>\n",
       "      <td>Satakunta</td>\n",
       "      <td>FI-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>822</th>\n",
       "      <td>FI</td>\n",
       "      <td>Uusimaa</td>\n",
       "      <td>FI-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>823</th>\n",
       "      <td>FI</td>\n",
       "      <td>Varsinais-Suomi</td>\n",
       "      <td>FI-19</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    location_iso   subdivision_name region_iso\n",
       "805           FI              Aland      FI-01\n",
       "806           FI      Etela-Karjala      FI-02\n",
       "807           FI    Etela-Pohjanmaa      FI-03\n",
       "808           FI         Etela-Savo      FI-04\n",
       "809           FI             Kainuu      FI-05\n",
       "810           FI         Kanta-Hame      FI-06\n",
       "811           FI    Keski-Pohjanmaa      FI-07\n",
       "812           FI        Keski-Suomi      FI-08\n",
       "813           FI        Kymenlaakso      FI-09\n",
       "814           FI              Lappi      FI-10\n",
       "815           FI          Pirkanmaa      FI-11\n",
       "816           FI          Pohjanmaa      FI-12\n",
       "817           FI    Pohjois-Karjala      FI-13\n",
       "818           FI  Pohjois-Pohjanmaa      FI-14\n",
       "819           FI       Pohjois-Savo      FI-15\n",
       "820           FI        Paijat-Hame      FI-16\n",
       "821           FI          Satakunta      FI-17\n",
       "822           FI            Uusimaa      FI-18\n",
       "823           FI    Varsinais-Suomi      FI-19"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get standard Finland's region names\n",
    "from covid_updater.iso import ISODB\n",
    "df_iso = ISODB()._df\n",
    "df_iso.loc[df_iso[\"location_iso\"] == \"FI\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tracked-netherlands",
   "metadata": {},
   "source": [
    "We define a mapping dictionary, that \"translates\" input region names to our stardad ISO names. Note that due to the nature of the data, more than one Hospital District may correspond to a region."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "happy-pitch",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Area renaming\n",
    "region_renaming = {\n",
    "    'Åland': 'Aland',\n",
    "    'South Karelia Hospital District': 'Etela-Karjala',\n",
    "    'South Ostrobothnia Hospital District': 'Etela-Pohjanmaa',\n",
    "    'South Savo Hospital District': 'Etela-Savo',\n",
    "    'Helsinki and Uusimaa Hospital District': 'Uusimaa',\n",
    "    'Itä-Savo Hospital District': 'Etela-Savo',\n",
    "    'Kainuu Hospital District': 'Kainuu',\n",
    "    'Kanta-Häme Hospital District': 'Kanta-Hame',\n",
    "    'Central Ostrobothnia Hospital District': 'Keski-Pohjanmaa',\n",
    "    'Central Finland Hospital District': 'Keski-Suomi',\n",
    "    'Kymenlaakso Hospital District': 'Kymenlaakso',\n",
    "    'Lappi Hospital District': 'Lappi',\n",
    "    'Länsi-Pohja Hospital District': 'Lappi',\n",
    "    'Pirkanmaa Hospital District': 'Pirkanmaa',\n",
    "    'North Karelia Hospital District': 'Pohjois-Karjala',\n",
    "    'North Ostrobothnia Hospital District': 'Pohjois-Pohjanmaa',\n",
    "    'North Savo Hospital District': 'Pohjois-Savo',\n",
    "    'Päijät-Häme Hospital District': 'Paijat-Hame',\n",
    "    'Satakunta Hospital District': 'Satakunta',\n",
    "    'Vaasa Hospital District': 'Pohjanmaa',\n",
    "    'Southwest Finland Hospital District': 'Varsinais-Suomi'\n",
    "}\n",
    "\n",
    "df.loc[:, \"region\"] = df.loc[:, \"region\"].replace(region_renaming)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "blind-niger",
   "metadata": {},
   "source": [
    "Next, we need to aggregate all data per region, as some regions are duplicated for a given date (data from more than one hospital)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "capable-signal",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.groupby([\"region\", \"date\"]).sum().reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "fossil-specification",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>date</th>\n",
       "      <th>people_vaccinated</th>\n",
       "      <th>people_fully_vaccinated</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  region        date  people_vaccinated  people_fully_vaccinated\n",
       "0  Aland  2020-12-26                  0                        0\n",
       "1  Aland  2020-12-27                  0                        0\n",
       "2  Aland  2020-12-28                  0                        0\n",
       "3  Aland  2020-12-29                  0                        0\n",
       "4  Aland  2020-12-30                  0                        0"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show some rows\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "electoral-journalism",
   "metadata": {},
   "source": [
    "### 2.5 Add missing columns\n",
    "In this case, we are missing columns related to location and ISO codes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "moral-audio",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Country\n",
    "df = df.assign(location=\"Finland\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "fitted-correlation",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ISO\n",
    "df = ISODB().merge(df, country_iso=\"FI\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "dental-current",
   "metadata": {},
   "outputs": [],
   "source": [
    "# total_vaccinations\n",
    "df.loc[:, \"total_vaccinations\"] = df.loc[:, \"people_vaccinated\"] + df.loc[:, \"people_fully_vaccinated\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "acoustic-credits",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>date</th>\n",
       "      <th>people_vaccinated</th>\n",
       "      <th>people_fully_vaccinated</th>\n",
       "      <th>location</th>\n",
       "      <th>location_iso</th>\n",
       "      <th>region_iso</th>\n",
       "      <th>total_vaccinations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2020-12-30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  region        date  people_vaccinated  people_fully_vaccinated location  \\\n",
       "0  Aland  2020-12-26                  0                        0  Finland   \n",
       "1  Aland  2020-12-27                  0                        0  Finland   \n",
       "2  Aland  2020-12-28                  0                        0  Finland   \n",
       "3  Aland  2020-12-29                  0                        0  Finland   \n",
       "4  Aland  2020-12-30                  0                        0  Finland   \n",
       "\n",
       "  location_iso region_iso  total_vaccinations  \n",
       "0           FI      FI-01                   0  \n",
       "1           FI      FI-01                   0  \n",
       "2           FI      FI-01                   0  \n",
       "3           FI      FI-01                   0  \n",
       "4           FI      FI-01                   0  "
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show some rows\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "marine-comparison",
   "metadata": {},
   "source": [
    "### 2.6 Get cumulative values\n",
    "The date from some countries may be cumulative. In some cases it is not, which is the case of Finland."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "neither-devil",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>date</th>\n",
       "      <th>people_vaccinated</th>\n",
       "      <th>people_fully_vaccinated</th>\n",
       "      <th>location</th>\n",
       "      <th>location_iso</th>\n",
       "      <th>region_iso</th>\n",
       "      <th>total_vaccinations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-01</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-02</td>\n",
       "      <td>46</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-03</td>\n",
       "      <td>36</td>\n",
       "      <td>1</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-04</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-05</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   region        date  people_vaccinated  people_fully_vaccinated location  \\\n",
       "65  Aland  2021-03-01                 38                        0  Finland   \n",
       "66  Aland  2021-03-02                 46                        0  Finland   \n",
       "67  Aland  2021-03-03                 36                        1  Finland   \n",
       "68  Aland  2021-03-04                  0                        0  Finland   \n",
       "69  Aland  2021-03-05                  0                        0  Finland   \n",
       "\n",
       "   location_iso region_iso  total_vaccinations  \n",
       "65           FI      FI-01                  38  \n",
       "66           FI      FI-01                  46  \n",
       "67           FI      FI-01                  37  \n",
       "68           FI      FI-01                   0  \n",
       "69           FI      FI-01                   0  "
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# See last values for Aland\n",
    "df[df[\"region\"]==\"Aland\"].tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "quick-sword",
   "metadata": {},
   "source": [
    "To overcome this, we need to apply `cumsum` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "minus-transport",
   "metadata": {},
   "outputs": [],
   "source": [
    "for field in [\"people_vaccinated\", \"people_fully_vaccinated\", \"total_vaccinations\"]:\n",
    "    df.loc[:, field] = df.groupby(\"region\")[field].cumsum().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "retained-capital",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>date</th>\n",
       "      <th>people_vaccinated</th>\n",
       "      <th>people_fully_vaccinated</th>\n",
       "      <th>location</th>\n",
       "      <th>location_iso</th>\n",
       "      <th>region_iso</th>\n",
       "      <th>total_vaccinations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-01</td>\n",
       "      <td>3363</td>\n",
       "      <td>776</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>4139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-02</td>\n",
       "      <td>3409</td>\n",
       "      <td>776</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>4185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-03</td>\n",
       "      <td>3445</td>\n",
       "      <td>777</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>4222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-04</td>\n",
       "      <td>3445</td>\n",
       "      <td>777</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>4222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>Aland</td>\n",
       "      <td>2021-03-05</td>\n",
       "      <td>3445</td>\n",
       "      <td>777</td>\n",
       "      <td>Finland</td>\n",
       "      <td>FI</td>\n",
       "      <td>FI-01</td>\n",
       "      <td>4222</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   region        date  people_vaccinated  people_fully_vaccinated location  \\\n",
       "65  Aland  2021-03-01               3363                      776  Finland   \n",
       "66  Aland  2021-03-02               3409                      776  Finland   \n",
       "67  Aland  2021-03-03               3445                      777  Finland   \n",
       "68  Aland  2021-03-04               3445                      777  Finland   \n",
       "69  Aland  2021-03-05               3445                      777  Finland   \n",
       "\n",
       "   location_iso region_iso  total_vaccinations  \n",
       "65           FI      FI-01                4139  \n",
       "66           FI      FI-01                4185  \n",
       "67           FI      FI-01                4222  \n",
       "68           FI      FI-01                4222  \n",
       "69           FI      FI-01                4222  "
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# See again last values for Aland\n",
    "df[df[\"region\"]==\"Aland\"].tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "computational-individual",
   "metadata": {},
   "source": [
    "## 3. Exporting\n",
    "Exporting would consist on:\n",
    "\n",
    "- Order columns based on standard (check [Australia.csv](https://github.com/sociepy/covid19-vaccination-subnational/blob/main/data/countries/Australia.csv) as an example).\n",
    "- Sort rows by region & date.\n",
    "- Ensure fields `total_vaccinations`, `people_vaccinated`, `people_fully_vaccinated` are of type int."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "expected-blast",
   "metadata": {},
   "source": [
    "## 4. Implementing scraper\n",
    "First, you need to decide if you will be implementing this as a batch Scraper or an incremental Scraper. In our case here, we will be implementing it as a batch scraper and hence will implement a class inheriting from class `covid_updater.scraping.base.Scraper`. \n",
    "\n",
    "### Steps to follow:\n",
    "\n",
    "- Create new file `covid_updater.scraping.countries.finland.py`.\n",
    "- Design and implement new class `FinlandScraper` in the aforementined file. \n",
    "    - Use the code that you have tested in this notebook, and place it in the different class methods (`load_data`, `_process`, or create new ones, etc.)\n",
    "    - Use other country files as reference. Take your time to familiarize with its different parts and parent's logic,  too.\n",
    "    - Rememeber to inherit from `covid_updater.scraping.base.Scraper` and use already implemented\n",
    "- Add required lines in `covid_updater.scraping.core.py`:\n",
    "    - New import: `from covid_updater.scraping.countries.finland import FinlandScraper`\n",
    "    - Add an instance of `FinlandScraper` to list `scrapers`.\n",
    "- Add correponding ISO code to list `ISO_CODES` in script `scripts/update_countries.py`.\n",
    "- Run `update_all.sh` script with option `--update-population` to update `data/population.csv` file with new added regions\n",
    "\n",
    "```\n",
    "$ bash scripts/update_all.sh --update-population\n",
    "```\n",
    "\n",
    "- Verify that `data/vaccinations.csv` has been correctly generated (i.e., contains new added country data, fields `*_per_100` do not contain NaNs or empty values, etc.)\n",
    "- [Sync your fork with upstream](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/syncing-a-fork)\n",
    "- Create a Pull Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bridal-brisbane",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}