diff --git a/WEkEO/FLOMPY_WEkEO.ipynb b/WEkEO/FLOMPY_WEkEO.ipynb deleted file mode 100644 index 63073b4..0000000 --- a/WEkEO/FLOMPY_WEkEO.ipynb +++ /dev/null @@ -1,1630 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Authors:** Olympia Gounari, Alexandros Falagas, Kleanthis Karamvasis
\n", - "**Copyright:** 2021-2022 Kleanthis Karamvasis
\n", - "**License:** GPLv3" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "

FLOODWATER MAPPING & EXTRACTION OF FLOOD-AFFECTED AGRICULTURAL FIELDS

" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# NOTEBOOK INTRODUCTION\n", - "\n", - "### Data used\n", - "\n", - "| Product Description | Dataset ID | WEkEO metadata |\n", - "|:--------------------:|:-----------------:|:-----------------:|\n", - "| ERA5 | EO:ECMWF:DAT:REANALYSIS_ERA5_SINGLE_LEVELS | link |\n", - "| Sentinel-1 GRD | EO:ESA:DAT:SENTINEL-1:SAR | link |\n", - "| Sentinel-2 | EO:ESA:DAT:SENTINEL-2:MSI | link |" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Learning outcomes\n", - "At the end of this notebook you will be able to:\n", - "* Identify flooded agricultural regions\n", - "* Identify delineated flood-affected fields\n", - "* Identify cultivated and non-cultivated flood-affected fields" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "\n", - "Be aware that the estimated run time of the notebook is ~ 30 min\n", - "\n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Outline\n", - "\n", - "The FLOMPY - FLOod Mapping PYthon toolbox is a free and open-source python toolbox for mapping of floodwater. FLOMPY requires a specified time of interest related to the flood event and corresponding geographical boundaries and generates:\n", - "- Floodwater map\n", - "- Cultivated and non-cultivated delineated flood-affected fields\n", - "\n", - "Floods are considered the second gravest disaster for the agricultural sector. The increasing intensity and the frequency of flood events can result to significant yield losses. In the present notebook we present a methodological pipeline with the following steps:\n", - "1. Extraction of flooded area based on change detection approach using multitemporal Sentinel-1 intensity observations and precipitation data (ERA5 model). More info can be found at [1].\n", - "2. Delineation of agricultural fields by combining results from a classical edge detection filtering approach on multitemporal Sentinel-2 data [2] and a U-Net deep learning architecture [3].\n", - "3. Characterization of agricultural field to cultivated and non-cultivated based on their temporal behaviour based on multispectral Sentinel-2 data [4].\n", - "\n", - "We believe that the produced maps with delineated flood-affected agricultural fields can be helpful for governments, insurers and disaster response agencies to improve flood risk assessment, relief targeting, and ultimately to protect climate-vulnerable communities from floods. \n", - "\n", - "References:\n", - "\n", - "\n", - " \n", - "[1] Karamvasis K, Karathanassi V. FLOMPY: An Open-Source Toolbox for Floodwater Mapping Using Sentinel-1 Intensity Time Series. Water. 2021; 13(21):2943. [link](https://doi.org/10.3390/w13212943)\n", - " \n", - "[2] Yan L, Roy D.P. Automated crop field extraction from multi-temporal Web Enabled Landsat Data. Remote Sensing of Environment. 2014; 144. [link](https://doi.org/10.1016/j.rse.2014.01.006)\n", - "\n", - "[3] Floor S. et. al. Automatic Delineation of Agricultural Fields from Sentinel-2 Images. [link](https://github.com/resingm/field-boundary-delineation)\n", - "\n", - "[4] Gounari 0., Falagas A., Karamvasis K., Tsironis V., Karathanassi V., Karantzalos K.: Floodwater Mapping & Extraction of Flood-Affected Agricultural Fields. Living Planet Symposium Bonn 23-27 May 2022. [link](https://drive.google.com/file/d/1HiGkep3wx45gAQT6Kq34CdECMpQc8GUV/view?usp=sharing)\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " \n", - "## Contents\n", - "\n", - "
\n", - " \n", - "[1. Installation Notes](#section1)\n", - "\n", - "[2. Estimation of Flooded Agricultural Area](#section2)\n", - "\n", - "[3. Estimation of Cultivated and Non-Cultivated Delineated Flood-Affected Fields](#section3)\n", - "\n", - "[4. Interactive Plotting](#section4)\n", - "\n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Our project uses Sentinel-1, digital elevation model and total precipitation ERA5 dataset to detect a flooded agricultural area.\n", - "\n", - "Sentinel-2 and spectral indices, like NDVI, are used to delineate flood-affected agricultural field boundaries and then characterize them as cultivated or not-cultivated during the defined temporal period.\n", - "\n", - "The final product is an interactive map depicting **flooded area**, **flooded agricultural fields** and our **estimation concerning their damage** (cultivated or not-cultivated)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "\n", - "## 1. Installation Notes\n", - "[Back to top](#TOC_TOP)\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 1.1 Setting up FLOMPY Environment" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To install FLOMPY software and the required dependencies go to:\n", - "\n", - "1. `$cd FLOMPY/WEkEO`\n", - "\n", - "2. `$chmod +x installation.sh && ./installation.sh`" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "\n", - "After the installation completes, **restart current kernel (`Kernel->Restart Kernel`) of this notebook**.\n", - " \n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 1.2 Provide WEkEO HDA API Credentials\n", - "Fill your WEKEO **username** and your **password** in the following variables. If you don't have any WEKEO credentials, please create an account from [here](https://my.wekeo.eu/web/guest/user-registration).\n", - "\n", - "Your credentials are useful to authenticate in order to execute calls to the HDA API." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "user = '******'\n", - "password = '******'" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 1.3 Setting up Wekeo HDA API\n" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "import base64\n", - "import requests\n", - "import json\n", - "\n", - "credentials = {\n", - " 'url':'https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker',\n", - " 'user':user,\n", - " 'password':password,}\n", - "encoded = base64.b64encode(f\"{credentials['user']}:{credentials['password']}\".encode('ascii')).decode(\"utf-8\")\n", - "r = requests.get(f\"{credentials['url']}/gettoken\",\n", - " headers = {\"authorization\":f\"Basic {encoded}\"})\n", - "response = r.json()\n", - "r = requests.put(f\"{credentials['url']}/termsaccepted/Copernicus_General_License\",\n", - " headers = {\"accept\":\"application/json\", \"authorization\":response['access_token']},\n", - " data = {\"accepted\":\"true\"})\n", - "filepath = \"/home/jovyan/.hdarc\"\n", - "with open(filepath, 'w') as file:\n", - " for key, value in credentials.items(): \n", - " file.write('%s:%s\\n' % (key, value))\n", - " \n", - "USER_DIR = f\"/home/{user}\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "\n", - "## 2. Estimation of Flooded Agricultural Area\n", - "[Back to top](#TOC_TOP)\n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 2.1 Insert Python Modules" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Matplotlib created a temporary config/cache directory at /tmp/matplotlib-e_kqtkob because the default path (/home/jovyan/.cache/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.\n", - "2022-09-02 19:38:01.827336: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-09-02 19:38:01.827381: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "-----------------------------------------\n", - "keras-unet init: TF version is >= 2.0.0 - using `tf.keras` instead of `Keras`\n", - "-----------------------------------------\n", - "FLOod Mapping PYthon toolbox\n", - "Copyright (c) 2021-2022 Kleanthis Karamvasis, karamvasis_k@hotmail.com\n", - "Remote Sensing Laboratory of National Technical University of Athens\n", - "-----------------------------------------------------------------\n", - "License: GNU GPL v3+\n", - "-----------------------------------------------------------------\n" - ] - } - ], - "source": [ - "import os\n", - "import glob\n", - "import numpy as np\n", - "import rasterio as rio\n", - "import rasterio.mask\n", - "import geopandas as gpd\n", - "import matplotlib.pyplot as plt\n", - "import folium\n", - "from branca.element import Template, MacroElement\n", - "import warnings\n", - "warnings.filterwarnings('ignore')\n", - "# flompy and wekeo libraries\n", - "from flompy.FLOMPYapp import FloodwaterEstimation\n", - "from flompy.Download import Sentinel_1_download, Sentinel_2_download\n", - "import wekeo_utils, retrieve" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 2.2 Update Configuration File" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "\n", - "In case you want to generate Flompy's outputs for a different flood event, please update the configuration parameters located at the template file: /home/your-user-name/FLOMPY/FLOMPYapp_template.cfg. Have a look at parameters located at section B. Flood event temporal information \n", - "and C. Flood event spatial information.\n", - "More information can be found in the [README](https://github.com/Olyna/FLOMPY/blob/notebook/README.md) file.\n", - "\n", - "
" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Changing \"olyg\" to \"olyg\" in /home/olyg/FLOMPY/FLOMPYapp_template.cfg\n" - ] - } - ], - "source": [ - "f = open(os.path.join(USER_DIR, 'FLOMPY', 'FLOMPYapp_template.cfg'), \"r\")\n", - "\n", - "def inplace_change(filename, old_string, new_string):\n", - " with open(filename) as f:\n", - " s = f.read()\n", - " if old_string not in s:\n", - " print('\"{old_string}\" not found in {filename}.'.format(**locals()))\n", - " return\n", - "\n", - " with open(filename, 'w') as f:\n", - " print('Changing \"{old_string}\" to \"{new_string}\" in {filename}'.format(**locals()))\n", - " s = s.replace(old_string, new_string)\n", - " f.write(s)\n", - "\n", - "inplace_change(os.path.join(USER_DIR, 'FLOMPY', 'FLOMPYapp_template.cfg'), \"olyg\", user)\n", - "f = open(os.path.join(USER_DIR, 'FLOMPY', 'FLOMPYapp_template.cfg'), \"r\")" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Projectname : Palamas\n", - "projectfolder : /home/olyg/Palamas\n", - "src_dir : /home/olyg/FLOMPY/flompy/\n", - "snap_dir : /home/jovyan/.snap/auxdata/Orbits/Sentinel-1\n", - "GPTBIN_PATH : /home/jovyan/snap/bin/gpt\n", - "Flood_datetime : 20200921T030000\n", - "before_flood_days : 20\n", - "after_flood_days : 3\n", - "#-In case you provide AOI BBOX coordinates, set AOI_File : None\n", - "AOI_File : None\n", - "LONMIN : 22.02\n", - "LATMIN : 39.46\n", - "LONMAX : 22.17\n", - "LATMAX : 39.518\n", - "days_back : 12\n", - "accumulated_precipitation_threshold : 120\n", - "relOrbit : Auto\n", - "minimum_mapping_unit_area_m2 : 4000\n", - "CPU : 8\n", - "RAM : 20G\n", - "scihub_username : flompy\n", - "scihub_password : rslab2022\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n" - ] - }, - { - "data": { - "text/plain": [ - "0" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "app = FloodwaterEstimation(customTemplateFile = os.path.join(USER_DIR, 'FLOMPY', 'FLOMPYapp_template.cfg'))\n", - "app.startup()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 2.3 Downloading ERA5 and Sentinel-1 datasets using HDA API\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Based on given temporal range, ERA5 dataset is retrieved for 20 days before the flood event and 3 dates after the flood event. The defined area of interest is used to spatially subset the dataset." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Downloading matches: 1\n", - "adaptor.mars.internal-1661861550.0492537-9520-5-a993bd83-d712-4fcc-ae57-ef165d76732e.nc\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/V7qEfEY984capriXzsQUoBn0K4U to adaptor.mars.internal-1661861550.0492537-9520-5-a993bd83-d712-4fcc-ae57-ef165d76732e.nc (4.4K)\n", - "Download rate 9.1K/s \n", - "NumExpr defaulting to 8 threads.\n", - "Found 20 products\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "S1A_IW_GRDH_1SDV_20200921T043931_20200921T043956_034452_04021B_A9F1.SAFE acquisition \n", - " was acquired after 1.66 hours from the user-defined datetime of flood event 2020-09-21 03:00:00\n", - "This is download try # 1.\n", - " We will try to download the requested products in 00 minutes.\n", - "Downloading matches: 1\n", - "S1B_IW_GRDH_1SDV_20200903T043849_20200903T043914_023206_02C11B_8188.zip\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/1qpeA65TYHghi_qB5rG8s1ARFAg to S1B_IW_GRDH_1SDV_20200903T043849_20200903T043914_023206_02C11B_8188.zip (970.9M)\n", - "Oops, downloaded 1753020027 byte(s), was supposed to be 1018076147 (extra 734943880)\n", - "Download rate 31.2M/s\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Downloading matches: 1\n", - "S1A_IW_GRDH_1SDV_20200909T043931_20200909T043956_034277_03FBD8_44F9.zip\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/bezXjj_U6A_4hhbQxOGvrd4JZbk to S1A_IW_GRDH_1SDV_20200909T043931_20200909T043956_034277_03FBD8_44F9.zip (972.9M)\n", - "Oops, downloaded 1752755513 byte(s), was supposed to be 1020108043 (extra 732647470)\n", - "Download rate 30.8M/s\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Downloading matches: 1\n", - "S1B_IW_GRDH_1SDV_20200915T043849_20200915T043914_023381_02C69F_E9BB.zip\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/_6DMxCR4mT96pHonciqcZx7Ry0s to S1B_IW_GRDH_1SDV_20200915T043849_20200915T043914_023381_02C69F_E9BB.zip (966.8M)\n", - "Oops, downloaded 1752916626 byte(s), was supposed to be 1013743357 (extra 739173269)\n", - "Download rate 29.9M/s\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Downloading matches: 1\n", - "S1A_IW_GRDH_1SDV_20200921T043931_20200921T043956_034452_04021B_A9F1.zip\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/L0XfUCAqL8l1V81YWWSmw_29WmE to S1A_IW_GRDH_1SDV_20200921T043931_20200921T043956_034452_04021B_A9F1.zip (968.3M)\n", - "Oops, downloaded 1752753212 byte(s), was supposed to be 1015284712 (extra 737468500)\n", - "Download rate 27.7M/s\n" - ] - }, - { - "data": { - "text/plain": [ - "0" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Downloading ERA5 dataset\n", - "retrieve.era5_data(app.bbox, app.flood_datetime, app.start_datetime, app.end_datetime, app.ERA5_dir)\n", - "\n", - "# Quering Sentinel-1 data and select best scenes\n", - "Sentinel_1_download.Download_S1_data(\n", - " scihub_accounts = app.credentials,\n", - " S1_GRD_dir = app.S1_GRD_dir,\n", - " geojson_S1 = app.geojson_S1,\n", - " Start_time = app.Start_time,\n", - " End_time = app.End_time,\n", - " relOrbit = app.relOrbit,\n", - " flood_datetime = app.flood_datetime,\n", - " time_sleep=10,\n", - " max_tries=1,\n", - " download=False\n", - ")\n", - "\n", - "# Downloading Sentinel-1 data using HDA API\n", - "retrieve.S1_data(app.bbox, app.S1_GRD_dir)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 2.4 Extraction of Floodwater map using change detection analysis of multitemporal Sentinel-1 data" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Preprocessing of Sentinel-1 SAR acquisitions. \n", - "- Orbit correction\n", - "- Coregistration\n", - "- Subsetting\n", - "- Radiometric Calibration\n", - "- Geocoding" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - " We coregister the images in respect with the acquisition of S1A_IW_GRDH_1SDV_20200921T043931_20200921T043956_034452_04021B_A9F1.zip\n", - "Processing of flood image 20200921T043931\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Processing of baseline image 20200903T043849\n", - "Processing of baseline image 20200909T043931\n", - "Processing of baseline image 20200915T043849\n", - "Refine borders of Sentinel-1 acquisitions\n", - "20200903T043849\n", - "20200909T043931\n", - "20200915T043849\n", - "Baseline Stack images:\n", - "20200903T043849\n", - "20200909T043931\n", - "20200915T043849\n", - "Flood image:\n", - "20200921T043931\n", - "All information from SAR imagery are stored at /home/olyg/Palamas/Preprocessed/Stack/SAR_Stack.h5\n" - ] - }, - { - "data": { - "text/plain": [ - "0" - ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "app.run_preprocessing_S1_data('Preprocessing_S1_data')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Statistical Change Detection Analysis using SAR intensity values. Generation of t-score statistic map based on pre-flood and post-flood SAR observations." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "\n", - "A Slope calculation (degrees)\u001b[39m\n", - "C Horn, B.K.P., 1981. Hill shading and the reflectance map. Proceedings of the IEEE 69, 14–47. doi:10.1109/PROC.1981.11918\u001b[39m\n", - "\n", - "\u001b[2Kt Wall-time = 0.136954\u001b[39m======================= ] (99% - 0.0s - 1 threads)\n", - "\n", - "A Aspect attribute calculation\u001b[39m\n", - "C Horn, B.K.P., 1981. Hill shading and the reflectance map. Proceedings of the IEEE 69, 14–47. doi:10.1109/PROC.1981.11918\u001b[39m\n", - "\n", - "\u001b[2Kt Wall-time = 0.15652\u001b[39m======================== ] (99% - 0.0s - 1 threads)\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n", - "pj_obj_create: Open of /opt/conda/share/proj failed\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "T-score changes due to flood can be found at /home/olyg/Palamas/Results/t_scores_VV_VH_db.tif\n" - ] - }, - { - "data": { - "text/plain": [ - "0" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "app.run_multitemporal_statistics('Statistical_analysis')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Identification of Flooded Agricultural Regions based on thresholding at t-score map." - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Floodwater map can be found at /home/olyg/Palamas/Results/Flood_map_Palamas.tif\n" - ] - }, - { - "data": { - "text/plain": [ - "0" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "app.run_get_flood_map('Floodwater_classification')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "Free up disk. Deleting all Sentinel-1 data in order to have enough memory space due to 10 Gb available space.\n", - "\n", - "
" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [], - "source": [ - "! rm *.zip" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "\n", - "## 3. Estimation of Cultivated and Non-Cultivated Delineated Flood-Affected Fields\n", - "[Back to top](#TOC_TOP)\n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 3.1 Downloading Sentinel-2 using HDA API" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Found 8 products\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "No download option is enabled. Printing the query results...\n", - "1ecab495-0e3a-47c4-b861-3d05934bb5e6 S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_2...\n", - "fb796c95-e227-4fdd-8f44-e933337e97ad S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_2...\n", - "02fb0b84-dcc1-45ad-9587-8d11782fe957 S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_2...\n", - "cdc77cc2-d783-41cd-9599-07aaaf476073 S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_2...\n", - "Name: title, dtype: object\n", - "Downloading matches: 1\n", - "S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.zip\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/8Tp-FVzT_8biGtWw0NUlEdpO4k0 to S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.zip (1.1G)\n", - "Oops, downloaded 1186938554 byte(s), was supposed to be 1186897191 (extra 41363)\n", - "Download rate 40.3M/s\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Downloading matches: 1\n", - "S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.zip\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/3TV8lgchykPce1o545UtaFttd6c to S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.zip (1.1G)\n", - "Oops, downloaded 1206404445 byte(s), was supposed to be 1206366126 (extra 38319)\n", - "Download rate 38.3M/s\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Downloading matches: 1\n", - "S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.zip\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/O-MdO5wZvsjYKylGPa5gpGDqk2g to S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.zip (1.1G)\n", - "Oops, downloaded 1217389362 byte(s), was supposed to be 1217350563 (extra 38799)\n", - "Download rate 39.1M/s\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Downloading matches: 1\n", - "S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.zip\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Downloading https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/40FcbRzWMFmeGtDjjj_kDiNgCsc to S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.zip (1.1G)\n", - "Oops, downloaded 1209409504 byte(s), was supposed to be 1209374417 (extra 35087)\n", - "Download rate 38.7M/s\n" - ] - }, - { - "data": { - "text/plain": [ - "0" - ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Quering Sentinel-2 data and select best scenes\n", - "Sentinel_2_download.Download_S2_data(\n", - " AOI = app.geojson_S1,\n", - " user = list(app.credentials.keys())[0],\n", - " passwd = list(app.credentials.values())[0],\n", - " Start_time = app.Start_time,\n", - " End_time = app.End_time,\n", - " write_dir = app.S2_dir,\n", - " product = 'S2MSI2A',\n", - " download = False,\n", - " cloudcoverage = 100,\n", - " to_file = True)\n", - "\n", - "# Downloading Sentinel-2 data using HDA API\n", - "retrieve.S2_data(app.bbox, app.S2_dir)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 3.2 Crop Field Delineation & Characterization as Cultivated or Not-Cultivated" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Crop delineation consists of the following steps (estimated time to finish ~25mins):\n", - "\n", - "1. Determine agricultural regions based on [ESA WorldCover (2020)](https://worldcover2020.esa.int/) map, with 10m spatial resolution. Mask urban regions, forests and water bodies and retain only the agricultural regions.\n", - "\n", - " Also, Sentinel-2 L2A Scene Classification products are used in order to mask cloudy, shadowed or saturated pixels. \n", - "\n", - "2. Construct edge intensity and crop intensity maps using NDVI and edge detection filters, based on [L. Yan and D.P. Roy. 2014](https://doi.org/10.1016/j.rse.2014.01.006).\n", - "\n", - "3. Determine crop-edges using a pre-trained U-Net model.\n", - "\n", - "4. Combine the above raster products, execute some morphological filters and vectorize the result, to produce a crop-instance vector product.\n", - "\n", - "5. For a field to be considered as flooded, more than 30% of its area must intersect with the flood.\n", - "\n", - "6. Characterize flooded fields as cultivated or not-cultivated, by appling combined thresholdind to edge intensity and crop intensity maps." - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "---------------------------------------------------------------------------------------------\n", - "Searching for Sentinel 2 Satellite data...\n", - "Raw data found (*.SAFE file): S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE\n", - "Raw data found (*.SAFE file): S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE\n", - "Raw data found (*.SAFE file): S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE\n", - "Raw data found (*.SAFE file): S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE\n", - "---------------------------------------------------------------------------------------------\n", - "Calculating NDVI for all time series...\n", - "Calculating NDVI for image S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE...\n", - "Saving T34SEJ_20200905T092029_NDVI.tif...\n", - "Calculating NDVI for image S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE...\n", - "Saving T34SEJ_20200910T092031_NDVI.tif...\n", - "Calculating NDVI for image S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE...\n", - "Saving T34SEJ_20200915T092029_NDVI.tif...\n", - "Calculating NDVI for image S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE...\n", - "Saving T34SEJ_20200920T092031_NDVI.tif...\n", - "Masking band B02 for all time series with /home/olyg/Palamas/Palamas_AOI.geojson...\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE/GRANULE/L2A_T34SEJ_A018280_20200905T092031/IMG_DATA/R10m/T34SEJ_20200905T092029_B02_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE/GRANULE/L2A_T34SEJ_A027260_20200910T092343/IMG_DATA/R10m/T34SEJ_20200910T092031_B02_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE/GRANULE/L2A_T34SEJ_A018423_20200915T092052/IMG_DATA/R10m/T34SEJ_20200915T092029_B02_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE/GRANULE/L2A_T34SEJ_A027403_20200920T092033/IMG_DATA/R10m/T34SEJ_20200920T092031_B02_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Masking band B03 for all time series with /home/olyg/Palamas/Palamas_AOI.geojson...\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE/GRANULE/L2A_T34SEJ_A018280_20200905T092031/IMG_DATA/R10m/T34SEJ_20200905T092029_B03_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE/GRANULE/L2A_T34SEJ_A027260_20200910T092343/IMG_DATA/R10m/T34SEJ_20200910T092031_B03_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE/GRANULE/L2A_T34SEJ_A018423_20200915T092052/IMG_DATA/R10m/T34SEJ_20200915T092029_B03_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE/GRANULE/L2A_T34SEJ_A027403_20200920T092033/IMG_DATA/R10m/T34SEJ_20200920T092031_B03_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Masking band B04 for all time series with /home/olyg/Palamas/Palamas_AOI.geojson...\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE/GRANULE/L2A_T34SEJ_A018280_20200905T092031/IMG_DATA/R10m/T34SEJ_20200905T092029_B04_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE/GRANULE/L2A_T34SEJ_A027260_20200910T092343/IMG_DATA/R10m/T34SEJ_20200910T092031_B04_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE/GRANULE/L2A_T34SEJ_A018423_20200915T092052/IMG_DATA/R10m/T34SEJ_20200915T092029_B04_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE/GRANULE/L2A_T34SEJ_A027403_20200920T092033/IMG_DATA/R10m/T34SEJ_20200920T092031_B04_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Masking band B08 for all time series with /home/olyg/Palamas/Palamas_AOI.geojson...\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE/GRANULE/L2A_T34SEJ_A018280_20200905T092031/IMG_DATA/R10m/T34SEJ_20200905T092029_B08_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE/GRANULE/L2A_T34SEJ_A027260_20200910T092343/IMG_DATA/R10m/T34SEJ_20200910T092031_B08_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE/GRANULE/L2A_T34SEJ_A018423_20200915T092052/IMG_DATA/R10m/T34SEJ_20200915T092029_B08_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE/GRANULE/L2A_T34SEJ_A027403_20200920T092033/IMG_DATA/R10m/T34SEJ_20200920T092031_B08_10m.jp2 by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Masking band B11 for all time series with /home/olyg/Palamas/Palamas_AOI.geojson...\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE/GRANULE/L2A_T34SEJ_A018280_20200905T092031/IMG_DATA/R20m/T34SEJ_20200905T092029_B11_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE/GRANULE/L2A_T34SEJ_A027260_20200910T092343/IMG_DATA/R20m/T34SEJ_20200910T092031_B11_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE/GRANULE/L2A_T34SEJ_A018423_20200915T092052/IMG_DATA/R20m/T34SEJ_20200915T092029_B11_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE/GRANULE/L2A_T34SEJ_A027403_20200920T092033/IMG_DATA/R20m/T34SEJ_20200920T092031_B11_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Masking band B12 for all time series with /home/olyg/Palamas/Palamas_AOI.geojson...\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE/GRANULE/L2A_T34SEJ_A018280_20200905T092031/IMG_DATA/R20m/T34SEJ_20200905T092029_B12_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE/GRANULE/L2A_T34SEJ_A027260_20200910T092343/IMG_DATA/R20m/T34SEJ_20200910T092031_B12_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE/GRANULE/L2A_T34SEJ_A018423_20200915T092052/IMG_DATA/R20m/T34SEJ_20200915T092029_B12_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE/GRANULE/L2A_T34SEJ_A027403_20200920T092033/IMG_DATA/R20m/T34SEJ_20200920T092031_B12_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Masking band SCL for all time series with /home/olyg/Palamas/Palamas_AOI.geojson...\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE/GRANULE/L2A_T34SEJ_A018280_20200905T092031/IMG_DATA/R20m/T34SEJ_20200905T092029_SCL_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE/GRANULE/L2A_T34SEJ_A027260_20200910T092343/IMG_DATA/R20m/T34SEJ_20200910T092031_SCL_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE/GRANULE/L2A_T34SEJ_A018423_20200915T092052/IMG_DATA/R20m/T34SEJ_20200915T092029_SCL_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE/GRANULE/L2A_T34SEJ_A027403_20200920T092033/IMG_DATA/R20m/T34SEJ_20200920T092031_SCL_20m.jp2 by mask of inserted SHP and resample to 10 meters resolution...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Masking band NDVI for all time series with /home/olyg/Palamas/Palamas_AOI.geojson...\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200905T092029_N0214_R093_T34SEJ_20200905T113748.SAFE/GRANULE/L2A_T34SEJ_A018280_20200905T092031/IMG_DATA/R10m/T34SEJ_20200905T092029_NDVI.tif by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200910T092031_N0214_R093_T34SEJ_20200910T122928.SAFE/GRANULE/L2A_T34SEJ_A027260_20200910T092343/IMG_DATA/R10m/T34SEJ_20200910T092031_NDVI.tif by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2B_MSIL2A_20200915T092029_N0214_R093_T34SEJ_20200915T112537.SAFE/GRANULE/L2A_T34SEJ_A018423_20200915T092052/IMG_DATA/R10m/T34SEJ_20200915T092029_NDVI.tif by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "Extracting /home/olyg/Palamas/Sentinel_2_imagery/S2A_MSIL2A_20200920T092031_N0214_R093_T34SEJ_20200920T115008.SAFE/GRANULE/L2A_T34SEJ_A027403_20200920T092033/IMG_DATA/R10m/T34SEJ_20200920T092031_NDVI.tif by mask of inserted SHP...\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n", - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n" - ] - }, - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "420c05184d5e44389ffc241daa703e94", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "HBox(children=(HTML(value=''), FloatProgress(value=0.0, max=1.0), HTML(value='')))" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "Resample monthly max...\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2022-09-02 20:08:34.162284: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n", - "2022-09-02 20:08:34.168666: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n", - "2022-09-02 20:08:34.168764: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (jupyter-olyg): /proc/driver/nvidia/version does not exist\n", - "2022-09-02 20:08:34.175856: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA\n", - "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", - " 0%| | 0/4 [00:00\n", - "\n", - "## 4. Interactive Plotting\n", - "[Back to top](#TOC_TOP)\n", - "\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 4.1 Flooded Agricultural Map" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Preview flooded region on satellite basemap." - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n" - ] - }, - { - "data": { - "text/html": [ - "
Make this Notebook Trusted to load map: File -> Trust Notebook
" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Read AOI\n", - "aoi = gpd.read_file(app.geojson_S1)\n", - "\n", - "# AOI bounds\n", - "left, bottom, right, top = aoi.total_bounds\n", - "\n", - "# Define map bounds\n", - "map_bounds = [[bottom, left], [top, right]]\n", - "\n", - "# Read flood image using AOI\n", - "flood_fpath = glob.glob(os.path.join(app.Results_dir, 'Flood_map_*.tif'))[0]\n", - "with rio.open(flood_fpath) as src:\n", - " flood, out_transform = rasterio.mask.mask(src, aoi.geometry, crop=True)\n", - " flood = flood[0,:,:]\n", - " \n", - "# Create a map located to the AOI\n", - "m = folium.Map(\n", - " tiles='https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}',\n", - " attr = 'Esri',\n", - " location=[aoi.centroid.y[0], aoi.centroid.x[0]],\n", - " zoom_start=13)\n", - "\n", - "# Add flood image to the map, flood background is transparent\n", - "raster_to_coloridx = {1: (0.0, 0.0, 1.0, 1.0),\n", - " 0: (0.0, 0.0, 0.0, 0.0)}\n", - "\n", - "# Show map\n", - "m.add_child(folium.raster_layers.ImageOverlay(flood, \n", - " opacity=0.5,\n", - " bounds=map_bounds,\n", - " colormap=lambda x: raster_to_coloridx[x]))\n", - "m" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 4.2 Delineated Flooded Fields, characterized as Cultivated or Not-Cultivated" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Preview flood-affected, cultivated (green) or not-cultivated (red), delineated fields. Fields are overlayed on an NDVI image, selected by minimum cloud coverage." - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "proj_create: Open of /opt/conda/share/proj failed\n", - "proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode\n" - ] - }, - { - "data": { - "text/html": [ - "
Make this Notebook Trusted to load map: File -> Trust Notebook
" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Read NDVI image\n", - "ndvi, ndvi_bounds = wekeo_utils.ndvi_wgs84(app)\n", - "\n", - "# Read vector delineated crop field boundaries\n", - "field_bounds_fpath = glob.glob(os.path.join(USER_DIR, app.projectname, 'Results_crop_delineation', 'flooded_fields__*.shp'))[0]\n", - "field_bounds = gpd.read_file(field_bounds_fpath)\n", - "\n", - "# Change transformation to Geographic WGS'84\n", - "field_bounds = field_bounds.to_crs(4326)\n", - "\n", - "# Create a map located to the AOI\n", - "m = folium.Map(\n", - " tiles='https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}',\n", - " attr = 'Esri',\n", - " location=[aoi.centroid.y[0], aoi.centroid.x[0]],\n", - " zoom_start=13)\n", - "\n", - "# Add NDVI image to the map\n", - "m.add_child(folium.raster_layers.ImageOverlay(ndvi, bounds=ndvi_bounds))\n", - "\n", - "# Add delineated flooded fields to map\n", - "def style_fn(feature):\n", - " vector_to_coloridx = {\n", - " 'cultivated': {'fillColor':'#008000', 'fillOpacity':0.6, 'color':'#008000'},\n", - " 'not_cultivated': {'fillColor':'#FF0000', 'fillOpacity':0.6, 'color':'#FF0000'}\n", - " }\n", - " cultivation_status = feature['properties']['status']\n", - " return vector_to_coloridx[cultivation_status]\n", - "\n", - "m.add_child(folium.GeoJson(field_bounds.__geo_interface__, style_function=style_fn,))\n", - "\n", - "# Add a Map Legend\n", - "template = wekeo_utils.template()\n", - "macro = MacroElement()\n", - "macro._template = Template(template)\n", - "\n", - "# Show map\n", - "m.get_root().add_child(macro)\n", - "m" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - "\n", - "### Challenge:\n", - " We encourage the user to analyse more flood events by updating parameters in the configuration file located at the wekeo home directory: /home/your-user-name/FLOMPY/FLOMPYapp_template.cfg\n", - "\n", - "More information regarding the configuration parameters can be found in the /home/your-user-name/FLOMPY/README.md file.\n", - "\n", - "\n", - "
" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.6" - }, - "vscode": { - "interpreter": { - "hash": "2cbd12cf15904f49236868e52223ac6dba66d2e1778b7a5fa8a6c3a2f0b40e11" - } - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} diff --git a/WEkEO/README_WEkEO.md b/WEkEO/README_WEkEO.md deleted file mode 100644 index 55107fe..0000000 --- a/WEkEO/README_WEkEO.md +++ /dev/null @@ -1,7 +0,0 @@ -Please follow the steps in order to be able to run the notebook: -1. At your WEkEO environment start a new terminal session. File >> New Launcher >> Terminal -2. Run the following command to the repository at your WEkEO environment: -git clone -b notebook https://github.com/Olyna/FLOMPY.git -3. Open the notebook FLOMPY_WEkEO.ipynb located in /home/your-user-name/FLOMPY/WEkEO - -In case you want just to have a look at the already executed notebook, click [here](https://nbviewer.org/github/kleok/FLOMPY/blob/main/WEkEO/FLOMPY_WEkEO.ipynb). diff --git a/WEkEO/images/Flompy_logo_text.png b/WEkEO/images/Flompy_logo_text.png deleted file mode 100644 index a4b4b8a..0000000 Binary files a/WEkEO/images/Flompy_logo_text.png and /dev/null differ diff --git a/WEkEO/images/LogoWekeo.png b/WEkEO/images/LogoWekeo.png deleted file mode 100644 index b5a20fd..0000000 Binary files a/WEkEO/images/LogoWekeo.png and /dev/null differ diff --git a/WEkEO/installation.sh b/WEkEO/installation.sh deleted file mode 100755 index 0f5687a..0000000 --- a/WEkEO/installation.sh +++ /dev/null @@ -1,4 +0,0 @@ - -cd .. && pip install -e . -pip install folium && pip install markupsafe==2.0.1 -cd aux && chmod +x install_snap.sh && ./install_snap.sh \ No newline at end of file diff --git a/WEkEO/retrieve.py b/WEkEO/retrieve.py deleted file mode 100644 index d58af46..0000000 --- a/WEkEO/retrieve.py +++ /dev/null @@ -1,361 +0,0 @@ -import os -import sys -import pandas as pd -import shapely -import datetime -from IPython.core.display import HTML -import zipfile -from hda import Client -import xarray as xr -from typing import Tuple -import netCDF4 -import numpy as np - -import warnings -warnings.filterwarnings('ignore') - - - -def dateRange(given_date:str, days_diff:int=30)->Tuple[str, str]: - """Produce a temporal range. - - Args: - flood_event_date (str): Date of flood event (YYYY-MM-DD). - days_diff (int, optional): How many days before/after flood event\ - to start the temporal range. Defaults to 30. - - Returns: - Tuple[str, str]: [startdate, enddate], if days_diff is positive number.\ - [enddate, startdate], if days_diff is negative. - """ - dtf = datetime.datetime.strptime(given_date, '%Y-%m-%d') - new_date = dtf - datetime.timedelta(days=days_diff) - new_date = new_date.strftime('%Y-%m-%d') - return new_date, given_date - - -def cftime_to_datetime(cfdatetime): - ''' - Time convertion functionality - ''' - year=cfdatetime.year - month=cfdatetime.month - day=cfdatetime.day - hour=cfdatetime.hour - minute=cfdatetime.minute - second=cfdatetime.second - return datetime.datetime(year,month,day,hour,minute,second) - - -def era5_data(aoi:list, flood_date:datetime.datetime, start_datetime:datetime.datetime, end_datetime:datetime.datetime, ERA5_dir:str)->pd.DataFrame: - """Retrieve ERA5 data hourly, using HDA API. - - Args: - aoi (list): Area of interest bounding box coordinates. - flood_date (datetime.datetime): Date of flood event. - start_datetime (datetime.datetime): Days to examine before flood event. - end_datetime (datetime.datetime): Days to examine after flood event. - ERA5_dir (str): Location where retrieved data will be saved. - - Returns: - pd.DataFrame: _description_ - """ - - flood_event_date = flood_date.strftime('%Y-%m-%d') - start_date=start_datetime.strftime('%Y-%m-%d') - end_date=end_datetime.strftime('%Y-%m-%d') - minx, miny, maxx, maxy = aoi - - precipitation_filename_df = os.path.join(ERA5_dir,'ERA5_{Start_time}_{End_time}_{bbox_cdsapi}.csv'.format( - Start_time=start_datetime.strftime("%Y%m%dT%H%M%S"), - End_time=end_datetime.strftime("%Y%m%dT%H%M%S"), - bbox_cdsapi='_'.join(str(round(e,5)) for e in aoi))) - - request = { - 'datasetId': 'EO:ECMWF:DAT:REANALYSIS_ERA5_SINGLE_LEVELS', - 'boundingBoxValues': [{ - 'name': 'area', - 'bbox': [ minx, miny, maxx, maxy ] - }], - 'dateRangeSelectValues': [{ - 'name': 'date', - 'start': start_date, - 'end': end_date - }], - 'multiStringSelectValues': [{ - 'name': 'variable', - 'value': ['total_precipitation'] - },{ - 'name': 'product_type', - 'value': ['reanalysis'] - },{ - "name": "time", - "value": [ - "00:00", - "01:00", - "02:00", - "03:00", - "04:00", - "05:00", - "06:00", - "07:00", - "08:00", - "09:00", - "10:00", - "11:00", - "12:00", - "13:00", - "14:00", - "15:00", - "16:00", - "17:00", - "18:00", - "19:00", - "20:00", - "21:00", - "22:00", - "23:00" - ]} - ], - 'stringChoiceValues': [{ - 'name': 'format', - 'value': 'netcdf' - }] - } - - os.chdir(ERA5_dir) - c = Client() - matches = c.search(request) - print(f"Downloading matches: {len(vars(matches)['results'])}") - for m in vars(matches)['results']: - print(m['filename']) - matches.download() - - ds = xr.open_dataset(m['filename']) - df = ds.to_dataframe() - - Precipitation_data = pd.DataFrame(index = df.index) - - ERA5_data=netCDF4.Dataset(m['filename']) - ERA5_variables = list(ERA5_data.variables.keys()) - - df_dict={} - for ERA5_variable in ERA5_variables: - - if ERA5_variable in ['longitude', 'latitude']: - pass - elif ERA5_variable=='time': - time_var=ERA5_data.variables[ERA5_variable] - t_cal = ERA5_data.variables[ERA5_variable].calendar - dtime = netCDF4.num2date(time_var[:],time_var.units, calendar = t_cal) - dtime_datetime=[cftime_to_datetime(cfdatetime) for cfdatetime in dtime.data] - df_dict['Datetimes']=dtime_datetime - - elif ERA5_variable!='expver': - temp_name=ERA5_variable+'__'+ERA5_data[ERA5_variable].units - temp_dataset=np.mean(np.mean(ERA5_data[ERA5_variable][:],axis=1), axis=1) - if len(temp_dataset.shape)>1: - temp_dataset=np.mean(temp_dataset,axis=1) - df_dict[temp_name]=np.squeeze(temp_dataset) - else: - pass - - # create a dataframe - df_ERA5_tp = pd.DataFrame(df_dict) - df_ERA5_tp.index = df_ERA5_tp['Datetimes'] - df_ERA5_tp['tp__mm']=df_ERA5_tp['tp__m']*1000 - Precipitation_data = df_ERA5_tp['tp__mm'] - Precipitation_data.index = pd.to_datetime(Precipitation_data.index) - - Precipitation_data = Precipitation_data.to_frame(name='ERA5_tp_mm') - Precipitation_data['Datetime'] = Precipitation_data.index - Precipitation_data.to_csv(precipitation_filename_df, index=False) - return Precipitation_data - - - - -def S1_data(aoi:list, path:str): - """Retrieve Sentinel-1 data, using HDA API. - - Args: - aoi (list): Area of interest bounding box coordinates. - path (str): Location where retrieved data will be saved and\ - 'S1_products.csv' is saved. - - Returns: - _type_: _description_ - """ - # Define Area of Interest - minx, miny, maxx, maxy = aoi - - def _queryArgs(s1_products_file:pd.Series)->Tuple[str,str,str]: - """Function to apply on S1_products.csv and source info about image request. - - Args: - s1_products_file (pd.Series): S1_products.csv containing fields with info about\ - 'beginposition' and 'orbitdirection'. - - Returns: - Tuple[str,str,str]: [start_date, end_date, orbit_dir] - """ - # Temporal arguments - beginposition = s1_products_file['beginposition'] - beginposition = datetime.datetime.strptime(beginposition, '%Y-%m-%d %H:%M:%S.%f') - beginposition = beginposition.strftime('%Y-%m-%dT%H:%M:%S.%f') - - endposition = s1_products_file['endposition'] - endposition = datetime.datetime.strptime(endposition, '%Y-%m-%d %H:%M:%S.%f') - endposition = endposition.strftime('%Y-%m-%dT%H:%M:%S.%f') - - # Datetime strings - start_date, end_date = f'{beginposition[:-3]}Z', f'{endposition[:-3]}Z' - - # Orbit direction - orbit_dir = s1_products_file['orbitdirection'].lower() - - return start_date, end_date, orbit_dir - - # Read .csv downloaded by class Download_S1_data - s1_prod = pd.read_csv(os.path.join(path, 'S1_products.csv')) - - # For each entry in 'S1_products.csv', define temporal range of 1 day & orbit direction - for index, row in s1_prod.iterrows(): - start_date, end_date, orbit_dir = _queryArgs(row) - - request = { - "datasetId": "EO:ESA:DAT:SENTINEL-1:SAR", - "boundingBoxValues": [{ - "name": "bbox", - "bbox": [minx, miny, maxx, maxy] - }], - "dateRangeSelectValues": [{ - "name": "position", - "start": start_date, - "end": end_date - }], - "stringChoiceValues": [{ - "name": "productType", - "value": "GRD" - },{ - "name": "timeliness", - "value": row['timeliness'] - },{ - "name": "orbitDirection", - "value": orbit_dir - }] - } - - os.chdir(path) - c = Client() - matches = c.search(request) - print(f"Downloading matches: {len(vars(matches)['results'])}") - for m in vars(matches)['results']: - if os.path.exists(os.path.join(path, m['filename'])): - print(f"{m['filename']} Already Exists") - pass - else: - print(m['filename']) - matches.download() - """ - # Un-zip downloaded - download_dir_path = os.getcwd() - for item in os.listdir(download_dir_path): - if item.startswith('S1') and item.endswith('.zip'): - with zipfile.ZipFile(item, 'r') as zipObj: - zipObj.extractall() - """ - return 0 - - - -def S2_data(aoi:list, path:str): - """Retrieve Sentinel-2 data, using HDA API. - - Args: - aoi (list): Area of interest bounding box coordinates. - path (str): Location where retrieved data will be saved and\ - 'S2_products.csv' is saved. - - Returns: - _type_: _description_ - """ - minx, miny, maxx, maxy = aoi - - def _queryArgs(s2_products_file:pd.Series)->Tuple[str,str,str]: - """Function to apply on S2_products.csv and source info about one image to form\ - an HDA API request. Info concerns datetime and tilename. - - Args: - s2_products_file (pd.Series): S2_products.csv containing fields with info about\ - 'beginposition', 'endposition' and 'title'. - - Returns: - Tuple[str,str,str]: [start_date, end_date, tilename] - """ - # Temporal arguments - beginposition = s2_products_file['beginposition'] - beginposition = datetime.datetime.strptime(beginposition, '%Y-%m-%d %H:%M:%S.%f') - beginposition = beginposition.strftime('%Y-%m-%dT%H:%M:%S.%f') - - endposition = s2_products_file['endposition'] - endposition = datetime.datetime.strptime(endposition, '%Y-%m-%d %H:%M:%S.%f') - endposition = endposition.strftime('%Y-%m-%dT%H:%M:%S.%f') - - # Datetime strings - start_date, end_date = f'{beginposition[:-3]}Z', f'{endposition[:-3]}Z' - - # Tilename - tilename = s2_products_file['title'].split('_')[5] - - return start_date, end_date, tilename - - # Read .csv downloaded by class Download_S2_data - s2_prod = pd.read_csv(os.path.join(path, 'S2_products.csv')) - - # For each entry in 'S2_products.csv', define temporal arguments and tilename - for index, row in s2_prod.iterrows(): - start_date, end_date, tilename = _queryArgs(row) - - request = { - 'datasetId': 'EO:ESA:DAT:SENTINEL-2:MSI', - 'boundingBoxValues': [{ - 'name': 'area', - 'bbox': [minx, miny, maxx, maxy] - }], - 'dateRangeSelectValues': [{ - 'name': 'position', - 'start': start_date, - 'end': end_date - }], - 'stringChoiceValues': [{ - 'name': 'processingLevel', - 'value': 'LEVEL2A' - }], - 'stringInputValues':[{ - "name": "productIdentifier", - "value": tilename - }] - } - # print(f"\n{index} {request}") - - os.chdir(path) - c = Client() - matches = c.search(request) - print(f"Downloading matches: {len(vars(matches)['results'])}") - for m in vars(matches)['results']: - if os.path.exists(os.path.join(path, m['filename'])): - print(f"{m['filename']} Already Exists") - pass - else: - print(m['filename']) - matches.download() - - # Un-zip downloaded & delete zip files - download_dir_path = os.getcwd() - for item in os.listdir(download_dir_path): - if item.startswith('S2') and item.endswith('.zip'): - with zipfile.ZipFile(item, 'r') as zipObj: - zipObj.extractall() - os.remove(item) - return 0 diff --git a/WEkEO/wekeo_utils.py b/WEkEO/wekeo_utils.py deleted file mode 100644 index c78a885..0000000 --- a/WEkEO/wekeo_utils.py +++ /dev/null @@ -1,139 +0,0 @@ -import os -import numpy as np -import rasterio as rio -from rasterio.warp import calculate_default_transform, reproject, Resampling - - -def ndvi_wgs84(app): - - img_id = np.argmin(getattr(app.S2timeseries, 'cloud_cover')) - ndvi_fpath = getattr(app.S2timeseries, 'data')[img_id].NDVI_masked - - dst_crs='EPSG:4326' - with rio.open(ndvi_fpath, 'r+') as src: - src_crs = src.crs - - dst_transform, width, height = calculate_default_transform(src_crs, dst_crs, src.width, src.height, *src.bounds) - - kwargs = src.meta.copy() - kwargs.update({ - 'crs': dst_crs, - 'transform': dst_transform, - 'width': width, - 'height': height}) - - out_path = os.path.join(app.S2_dir, 'ndvi_4326.tif') - with rio.open(out_path, 'w', **kwargs) as dst: - for i in range(1, src.count + 1): - reproject( - source=rio.band(src, i), - destination=rio.band(dst, i), - src_transform=src.transform, - src_crs=src.crs, - dst_transform=dst_transform, - dst_crs=dst_crs, - resampling=Resampling.nearest) - - with rio.open(out_path, 'r') as src: - ndvi = src.read(1) - ndvi_bounds = src.bounds - - ndvi_bounds = [[ndvi_bounds.bottom, ndvi_bounds.left], [ndvi_bounds.top, ndvi_bounds.right]] - - return ndvi, ndvi_bounds - - - - -def template(): - jvscrpt = """ - {% macro html(this, kwargs) %} - - - - - - - jQuery UI Draggable - Default functionality - - - - - - - - - -
- -
Legend
-
-
    -
  • High NDVI
    • -
  • Low NDVI
    • -
  • Cultivated Flooded Field
    • -
  • Not-Cultivated Flooded Field
    • - -
-
-
- - - - - - {% endmacro %}""" - return jvscrpt \ No newline at end of file