adjsut quality vignette

vignettes/assess-quality.Rmd

@@ -27,9 +27,9 @@ bearer <- read.csv("~/Desktop/bearer_bm.csv")$token
 
 ### Overview <a name="overview"></a>
 
-The quality of nighttime lights data can be impacted by a number of factors, particularly cloud cover. To facilitate analysis using high quality data, Black Marble (1) marks the quality of each pixel and (2) in some cases of poor quality pixels, Black Marble will use data from a previous date to fill the value---using a temporally-gap filled NTL value. 
+The quality of nighttime lights data can be impacted by a number of factors, particularly cloud cover. To facilitate analysis using high quality data, Black Marble (1) marks the quality of each pixel and (2) in some cases, uses data from a previous date to fill the value---using a temporally-gap filled NTL value. 
 
-Below we illustrate how to examine the quality of nighttime lights data.
+This page illustrates how to examine the quality of nighttime lights data.
 
 * [Setup](#setup)
 * [Daily data](#daily)
@@ -110,17 +110,26 @@ ggplot() +
 ```{r, ntl_daily_r_map, echo=FALSE}
 ```
 
-We notice that a number of observations are missing, that are poor quality and could not be gap-filled. We can use the following code to determine the number of nighttime light pixels used:
+We notice that a number of observations are missing, that are poor quality and are not gap-filled. To understand the extent of missing date, we can use the following code to determine (1) the total number of pixels that cover Switzerland, (2) the total number of non-`NA` nighttime light pixels, and (3) the proportion of non-`NA` pixels.
 
 ```{r}
-n_obs <- function(values, coverage_fraction){
-  length(unique(values))
+n_pixel <- function(values, coverage_fraction){
+  length(values)
 }
 
-exact_extract(ntl_r, roi_sf, n_obs)
+n_non_na_pixel <- function(values, coverage_fraction){
+  length(!is.na(values))
+}
+
+n_pixel_num        <- exact_extract(ntl_r, roi_sf, n_pixel)
+n_non_na_pixel_num <- exact_extract(ntl_r, roi_sf, n_non_na_pixel)
+
+print(n_pixel_num)
+print(n_non_na_pixel_num)
+print(n_non_na_pixel_num / n_pixel_num)
 ```
 
-The `bm_extract` function facilitates computing the number of observations used to compute nighttime lights statistics. To obtain this variable, we set `add_n_pixels` to `TRUE`.
+By default, the `bm_extract` function computes these values:
 
 ```{r}
 ntl_df <- bm_extract(roi_sf = roi_sf,
@@ -129,13 +138,12 @@ ntl_df <- bm_extract(roi_sf = roi_sf,
                                    to = ymd("2023-01-10"),
                                    by = 1),
                    bearer = bearer,
-                   variable = "Gap_Filled_DNB_BRDF-Corrected_NTL",
-                   add_n_pixels = T)
+                   variable = "Gap_Filled_DNB_BRDF-Corrected_NTL")
 
-print(ntl_df)
+knitr::kable(ntl_df)
 ```
 
-The below figure shows trends in average nighttime lights (left) and the proportion of the country with a value for nighttime lights (right). For some days, low number of pixels corresponds to low nighttime lights (eg, January 3 and 5th); however, for other days, low number of pixels corresponds to higher nighttime lights (eg, January 9 and 10). On January 3 and 5, missing pixels could have been over cities---while on January 9 and 10, missing pixels could have been over typically lower-lit areas. 
+The below figure shows trends in average nighttime lights (left) and the proportion of the country with a value for nighttime lights (right). For some days, low number of pixels corresponds to low nighttime lights (eg, January 3 and 5th); however, for other days, low number of pixels corresponds to higher nighttime lights (eg, January 9 and 10). On January 3 and 5, missing pixels could have been over typically high-lit areas (eg, cities)---while on January 9 and 10, missing pixels could have been over typically lower-lit areas. 
 
 <details>
 <summary>Show code to produce figure</summary>
@@ -295,7 +303,7 @@ ggplot() +
 ```{r, ntl_tmp_gap_map, echo=FALSE}
 ```
 
-Instead of using `Gap_Filled_DNB_BRDF-Corrected_NTL`, we could ignore gap filled observations---using the `DNB_BRDF-Corrected_NTL` variable. We also remove poor quality pixels.
+Instead of using `Gap_Filled_DNB_BRDF-Corrected_NTL`, we could ignore gap filled observations---using the `DNB_BRDF-Corrected_NTL` variable. Here, we also remove poor quality pixels.
 
 ```{r}
 ntl_r <- bm_raster(roi_sf = roi_sf,
@@ -386,7 +394,7 @@ ggplot() +
 
 #### Number of Observations <a name="ma-numobs"></a>
 
-To determine the number of observations used to generate nighttime light values, we add `_Num` to the variable name.
+Black Marble removes poor quality observations, such as pixels covered by clouds. To determine the number of observations used to generate nighttime light values for each pixel, we add `_Num` to the variable name.
 
 ```{r}
 cf_r <- bm_raster(roi_sf = roi_sf,
@@ -480,15 +488,15 @@ ggplot() +
 
 #### Nighttime lights for good quality observations <a name="ma-ntl_gq"></a>
 
-The `quality_flag_rm` parameter determines which pixels are set to `NA` based on the quality indicator. By default, only pixels with a value of `255` are filtered out. However, if we only want data for good quality pixels, we can adjust the `quality_flag_rm` parameter.
+The `quality_flag_rm` parameter determines which pixels are set to `NA` based on the quality indicator. By default, only pixels with a value of `255` are filtered out. However, if we also want to remove poor quality pixels, we can adjust the `quality_flag_rm` parameter.
 
 ```{r}
 ntl_good_qual_r <- bm_raster(roi_sf = roi_sf,
                              product_id = "VNP46A3", 
                              date = "2023-01-01",
                              bearer = bearer,
                              variable = "NearNadir_Composite_Snow_Free",
-                             quality_flag_rm = c(2, 255))
+                             quality_flag_rm = c(1, 255))
 ```
 
 <details>