2c_Temporal aggregation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dc6217f5-a455-4e3a-be9d-63972723139d",
   "metadata": {},
   "source": [
    "<div>\n",
    "<table style=\"width: 100%\">\n",
    "\t<tr>\n",
    "\t\t<td>\n",
    "\t\t<table style=\"width: 100%\">\n",
    "\t\t\t<tr>\n",
    "                <td ><center><font size=\"5\"><b>Module 49</b></font><center>\n",
    "                <center><font size=\"6\">Digital Innovations for Water Challenges</font><center></td>\n",
    "\t\t\t</tr>\n",
    "\t\t\t<tr>\n",
    "                <td><center><font size=\"14\">Notebook 2.c</font><center></td>\n",
    "\t\t\t</tr>\n",
    "\t\t\t<tr>\n",
    "                <td><center><font size=\"6\"><b>Temporal aggregation from tiff files</b></font><center></td>\n",
    "\t\t\t</tr>\n",
    "\t\t</table>\n",
    "\t\t</td>\n",
    "\t\t<td><center><img src='images\\ihe-delft-institute_unesco_fc-lr.jpg'></img></td>\n",
    "\t</tr>\n",
    "</table>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e227b85a-75f3-48c0-959a-29a50c8ef950",
   "metadata": {},
   "source": [
    "# Table of contents\n",
    "1. [Learning objectives](#learningobs)\n",
    "2. [Introduction](#introduction)\n",
    "3. [Aggregation from tiff files](#tiff)\n",
    "4. [Aggregation from one .nc file](#nc)\n",
    "\n",
    "# 1. Learning objectives<a name=\"learningobs\"></a>\n",
    "\n",
    "- Use xarray to open .tiff and .nc files\n",
    "- Cycle through files in a directory\n",
    "- Plot tiff and .nc files\n",
    "- Make yearly map from monthly data \n",
    "\n",
    "# 2. Introduction<a name=\"introduction\"></a>\n",
    "In this notebook you will learn how to make basic aggregations of .tiff files by cycling through files in a folder and of .nc files using xarray.\n",
    "You will need the following packages installed:\n",
    "> xarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c6e786d-7583-4e56-8476-1f961bf07d05",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import glob\n",
    "import xarray as xr"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66c0313e-5764-4790-a3bc-ea60cbcf5226",
   "metadata": {},
   "source": [
    "### 3. Aggregation from tiff files <a name='tiff'></a>\n",
    "### 3.a. Opening one tiff file\n",
    "Open one tiff file from the data you downaload usng notebook 1.a."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "244f6f13-3898-4900-b11f-3c6c360837fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "filename = r\"./CHIRPS_tiff/chirps-v2.0.2020.01.tif\" #modify this line as needed\n",
    "data = xr.open_rasterio(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3f6978b-1e7d-4ee4-928c-b80b162b301b",
   "metadata": {},
   "source": [
    "Inspect the data: how many rows and columns does your data have? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c4357a7-dbbe-4210-87af-d65044c8ac14",
   "metadata": {},
   "outputs": [],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8720373c-1ab7-4b51-837e-3cb4def0752f",
   "metadata": {},
   "source": [
    "#### 3.b. Simple plot \n",
    "As the data has only one band, running the line below will produce a map of the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08e033d1-9b10-4d6b-9c2e-e5f0da80c9fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "data.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bd23cd6-d8d7-437f-ba8f-67e39996e881",
   "metadata": {},
   "source": [
    "### 3.c. Masking out no data values\n",
    "You will notice the data over land looks washed out - this is because the no data value of -9999 was not correctly identified. To mask out these values and obtain a better map try the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41491431-fa0b-4f70-a7c5-60a21d51240c",
   "metadata": {},
   "outputs": [],
   "source": [
    "ds_masked = data.where(data != -9999.) #this keeps only data for pixels with values different than -9999\n",
    "ds_masked.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1afc2c3e-cb00-4fea-b3c9-b833103144e5",
   "metadata": {},
   "source": [
    "### 3.d Yearly precipitation from monthly\n",
    "Now we will open all data one after the other and make a map of the yearly sum of precipitation.<br>\n",
    "#### Step 1: Getting all the file names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ad0bcf3-5713-4e11-892c-b361f799e138",
   "metadata": {},
   "outputs": [],
   "source": [
    "#glob allows you to list all files in a folder. \n",
    "#You can also specify what the file name needs to look like for inclusion using the * wildcard as shown here\n",
    "glob.glob(r'./CHIRPS_tiff/*.tif')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd7db56a-9f2b-4cc8-9187-a01fce6665b9",
   "metadata": {},
   "source": [
    "#### Step 2: loop through the filenames to open the monthly data and produce a map of the sum"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71d1b961-e3eb-482a-8f55-f3998e457009",
   "metadata": {},
   "outputs": [],
   "source": [
    "for i, filename in enumerate(glob.glob(r'./CHIRPS_tiff/*.tif')):\n",
    "    data = xr.open_rasterio(filename)\n",
    "    ds_masked = data.where(data != -9999.)\n",
    "    if i == 0:\n",
    "        data_sum = ds_masked\n",
    "    else:\n",
    "        data_sum += ds_masked"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fd2b6e1-9b51-4949-99ed-be6c08250340",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_sum.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3251baf3-97a4-4d63-a1a1-36968630c422",
   "metadata": {},
   "source": [
    "# 4. Aggregating from one .nc file <a name='nc'></a>\n",
    "Using notebook 1.a, download monthly CHIRPS data for the years 2020-2021 for the bounding box of your choice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6c6bb45-3287-4887-8b24-6a30e53fc1fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# replace with your own path\n",
    "path_to_nc = r\"./CHIRPS_nc/chirps20GlobalMonthlyP05_ea7a_a7e6_216e.nc\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0575234c-375a-408d-92bb-34bd97685f85",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_nc = xr.open_dataset(path_to_nc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8291190c-7cb5-4fe8-8c90-bdecf8a1c688",
   "metadata": {},
   "source": [
    "Inspect your data - how many dimensions does this dataset have?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66cfee43-02cf-4330-b119-5ff479429a07",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_nc"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6717fe6d-5b7e-4cb9-aa09-b6cd6be6a23b",
   "metadata": {},
   "source": [
    "1. Average yearly P map\n",
    "Produce a map of the average monthly precipitation over the area"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46a939d3-3159-4686-8826-9c22e97af75c",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_mean = data_nc.mean(dim='time')\n",
    "data_mean.precip.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37d79e6f-b0cd-403f-ba5a-2ab314dd7479",
   "metadata": {},
   "source": [
    "2. Yearly average and sum\n",
    "From the monthly data compute and plot a yearly average time series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83cf2d20-bf7b-41f0-bd8b-c012d843b262",
   "metadata": {},
   "outputs": [],
   "source": [
    "yearly_mean = data_nc.groupby('time.year').mean('time')\n",
    "yearly_sum = data_nc.groupby('time.year').sum('time', skipna=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91e3d41d-aba9-4ec2-8863-26b96111e9dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "yearly_sum.precip.loc[{'year':2020}].plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc4c0586-1801-4325-a182-328f75c39218",
   "metadata": {},
   "source": [
    "Get mean average precipitation for whole area:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a62d22b-0360-4798-ba8f-2ea04c543a21",
   "metadata": {},
   "outputs": [],
   "source": [
    "yearly_sum.precip.mean(dim=['latitude', 'longitude'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a30f8a1-9714-408b-98ec-29bba7bbc793",
   "metadata": {},
   "source": [
    "3. Montly average    \n",
    "From the data make a bar plot of average precipitation for the months of the year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb5c7ed8-f0e8-4e83-bf52-cb5a99709abb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}