diff --git a/_freeze/tutorials/R/1-intro-R/execute-results/html.json b/_freeze/tutorials/R/1-intro-R/execute-results/html.json index 8b65760..91c4e1a 100644 --- a/_freeze/tutorials/R/1-intro-R/execute-results/html.json +++ b/_freeze/tutorials/R/1-intro-R/execute-results/html.json @@ -1,7 +1,7 @@ { - "hash": "7cd21c6b25fc6d4141e84652629729d2", + "hash": "154b53557779f660be4254b7b5e1f8ed", "result": { - "markdown": "---\ntitle: \"Introduction\"\ndescription: \"Cloud-Native Data in R\"\n---\n\n\n\n## Exploring the Legacy of Redlining\n\nThis executable notebook provides an opening example to illustrate a cloud-native workflow in both R and python. \nPedagogy research emphasizes the importance of \"playing the whole game\" before breaking down every pitch and hit.\nWe intentionally focus on powerful high-level tools (STAC API, COGs, datacubes) to illustrate how a few chunks of\ncode can perform a task that would be far slower and more verbose in a traditional file-based, download-first workflow.\nNote the close parallels between R and Python syntax. This arises because both languages wrap the same underlying \ntools (the STAC API and GDAL warper) and handle many of the nuisances of spatial data -- from re-projections and\nresampling to mosaic tiles -- without us noticing.\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nlibrary(rstac)\nlibrary(gdalcubes)\nlibrary(stars)\nlibrary(tmap)\nlibrary(dplyr)\ngdalcubes::gdalcubes_options(parallel = TRUE)\n```\n:::\n\n\n\n## Data discovery\n\n\n\nThe first step in many workflows involves discovering individual spatial data files covering the space, time, and variables of interest. Here we use a [STAC](https://stacspec.org/en) Catalog API to recover a list of candidate data. \nWe dig deeper into how this works and what it returns in later recipes. This example searches for images in a lon-lat bounding box from a collection of Cloud-Optimized-GeoTIFF (COG) images taken by Sentinel2 satellite mission.\nThis function will not download any imagery, it merely gives us a list of metadata about available images, including the access URLs.\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nbox <- c(xmin=-122.51, ymin=37.71, xmax=-122.36, ymax=37.81) \nstart_date <- \"2022-06-01\"\nend_date <- \"2022-08-01\"\nitems <-\n stac(\"https://earth-search.aws.element84.com/v0/\") |>\n stac_search(collections = \"sentinel-s2-l2a-cogs\",\n bbox = box,\n datetime = paste(start_date, end_date, sep=\"/\"),\n limit = 100) |>\n ext_query(\"eo:cloud_cover\" < 20) |>\n post_request()\n```\n:::\n\n\nWe pass this list of images to a high-level utilty (`gdalcubes` in R, `odc.stac` in python) that will do all of the heavy lifting. Using the URLs and metadata provided by STAC, \nthese functions can extract only our data of interest (given by the bounding box) without downloading unnecessary regions or bands. While streaming the data, these functions\nwill also reproject it into the desired coordinate reference system -- (an often costly operation to perform in R) and can potentially resample or aggregate the data to a desired \nspatial resolution. (The R code will also resample from images in overlapping areas to replace pixels masked by clouds)\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\ncol <- stac_image_collection(items$features, asset_names = c(\"B08\", \"B04\", \"SCL\"))\n\ncube <- cube_view(srs =\"EPSG:4326\",\n extent = list(t0 = start_date, t1 = end_date,\n left = box[1], right = box[3],\n top = box[4], bottom = box[2]),\n dx = 0.0001, dy = 0.0001, dt = \"P1D\",\n aggregation = \"median\", resampling = \"average\")\n\nmask <- image_mask(\"SCL\", values=c(3, 8, 9)) # mask clouds and cloud shadows\n\ndata <- raster_cube(col, cube, mask = mask)\n```\n:::\n\n\n\nWe can do arbitrary calculations on this data as well. Here we calculate NDVI, a widely used measure of greenness that can be used to determine tree cover. \n(Note that the R example uses lazy evaluation, and can thus perform these calculations while streaming)\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nndvi <- data |>\n select_bands(c(\"B04\", \"B08\")) |>\n apply_pixel(\"(B08-B04)/(B08+B04)\", \"NDVI\") |>\n reduce_time(c(\"mean(NDVI)\"))\n\nndvi_stars <- st_as_stars(ndvi)\n```\n:::\n\n\n\nAnd we plot the result. The long rectangle of Golden Gate Park is clearly visible in the North-West.\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nmako <- tm_scale_continuous(values = viridisLite::mako(30))\nfill <- tm_scale_continuous(values = \"Greens\")\n\ntm_shape(ndvi_stars) + tm_raster(col.scale = mako)\n```\n\n::: {.cell-output-display}\n![](1-intro-R_files/figure-html/unnamed-chunk-5-1.png){width=672}\n:::\n:::\n\n\n\n# From NDVI to Environmental Justice\n\nWe examine the present-day impact of historic \"red-lining\" of US cities during the Great Depression using data from the [Mapping Inequality](https://dsl.richmond.edu/panorama/redlining) project. All though this racist practice was banned by federal law under the Fair Housing Act of 1968, the systemic scars of that practice are still so deeply etched on our landscape that the remain visible from space -- \"red-lined\" areas (graded \"D\" under the racist HOLC scheme) show systematically lower greenness than predominately-white neighborhoods (Grade \"A\"). Trees provide many benefits, from mitigating urban heat to biodiversity, real-estate value, to health.\n\n\n## Zonal statistics \n\nIn addition to large scale raster data such as satellite imagery, the analysis of vector shapes such as polygons showing administrative regions is a central component of spatial analysis, and particularly important to spatial social sciences. The red-lined areas of the 1930s are one example of spatial vectors. One common operation is to summarise the values of all pixels falling within a given polygon, e.g. computing the average greenness (NDVI) \n\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nsf <- \"/vsicurl/https://dsl.richmond.edu/panorama/redlining/static/citiesData/CASanFrancisco1937/geojson.json\" |>\n st_read() |>\n st_make_valid() |>\n select(-label_coords)\n```\n:::\n\n::: {.cell}\n\n```{.r .cell-code}\npoly <- ndvi |> extract_geom(sf, FUN = mean, reduce_time = TRUE)\nsf$NDVI <- poly$NDVI\n```\n:::\n\n\n\nWe plot the underlying NDVI as well as the average NDVI of each polygon, along with it's textual grade, using `tmap`. Note that \"A\" grades tend to be darkest green (high NDVI) while \"D\" grades are frequently the least green. (Regions not zoned for housing at the time of the 1937 housing assessment are not displayed as polygons.)\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\ntm_shape(ndvi_stars) + tm_raster(col.scale = mako) +\n tm_shape(sf) + tm_polygons('NDVI', fill.scale = fill) +\n tm_shape(sf) + tm_text(\"grade\", col=\"darkblue\", size=0.6) +\n tm_legend_hide()\n```\n\n::: {.cell-output-display}\n![](1-intro-R_files/figure-html/unnamed-chunk-8-1.png){width=672}\n:::\n:::\n\n\n\nAre historically redlined areas still less green?\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nsf |> \n as_tibble() |>\n group_by(grade) |> \n summarise(ndvi = mean(NDVI), \n sd = sd(NDVI)) |>\n knitr::kable()\n```\n\n::: {.cell-output-display}\n|grade | ndvi| sd|\n|:-----|---------:|---------:|\n|A | 0.3201204| 0.0611414|\n|B | 0.2138501| 0.0783221|\n|C | 0.1956334| 0.0564822|\n|D | 0.1949736| 0.0385805|\n|NA | 0.0962092| NA|\n:::\n:::\n", + "markdown": "---\ntitle: \"Introduction\"\ndescription: \"Cloud-native geospatial data in R\"\n---\n\n\n\n## Exploring the Legacy of Redlining\n\nThis executable notebook provides an opening example to illustrate a cloud-native workflow in both R and python. \nPedagogy research emphasizes the importance of \"playing the whole game\" before breaking down every pitch and hit.\nWe intentionally focus on powerful high-level tools (STAC API, COGs, datacubes) to illustrate how a few chunks of\ncode can perform a task that would be far slower and more verbose in a traditional file-based, download-first workflow.\nNote the close parallels between R and Python syntax. This arises because both languages wrap the same underlying \ntools (the STAC API and GDAL warper) and handle many of the nuisances of spatial data -- from re-projections and\nresampling to mosaic tiles -- without us noticing.\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nlibrary(rstac)\nlibrary(gdalcubes)\nlibrary(stars)\nlibrary(tmap)\nlibrary(dplyr)\ngdalcubes::gdalcubes_options(parallel = TRUE)\n```\n:::\n\n\n\n## Data discovery\n\n\n\nThe first step in many workflows involves discovering individual spatial data files covering the space, time, and variables of interest. Here we use a [STAC](https://stacspec.org/en) Catalog API to recover a list of candidate data. \nWe dig deeper into how this works and what it returns in later recipes. This example searches for images in a lon-lat bounding box from a collection of Cloud-Optimized-GeoTIFF (COG) images taken by Sentinel2 satellite mission.\nThis function will not download any imagery, it merely gives us a list of metadata about available images, including the access URLs.\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nbox <- c(xmin=-122.51, ymin=37.71, xmax=-122.36, ymax=37.81) \nstart_date <- \"2022-06-01\"\nend_date <- \"2022-08-01\"\nitems <-\n stac(\"https://earth-search.aws.element84.com/v0/\") |>\n stac_search(collections = \"sentinel-s2-l2a-cogs\",\n bbox = box,\n datetime = paste(start_date, end_date, sep=\"/\"),\n limit = 100) |>\n ext_query(\"eo:cloud_cover\" < 20) |>\n post_request()\n```\n:::\n\n\nWe pass this list of images to a high-level utilty (`gdalcubes` in R, `odc.stac` in python) that will do all of the heavy lifting. Using the URLs and metadata provided by STAC, \nthese functions can extract only our data of interest (given by the bounding box) without downloading unnecessary regions or bands. While streaming the data, these functions\nwill also reproject it into the desired coordinate reference system -- (an often costly operation to perform in R) and can potentially resample or aggregate the data to a desired \nspatial resolution. (The R code will also resample from images in overlapping areas to replace pixels masked by clouds)\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\ncol <- stac_image_collection(items$features, asset_names = c(\"B08\", \"B04\", \"SCL\"))\n\ncube <- cube_view(srs =\"EPSG:4326\",\n extent = list(t0 = start_date, t1 = end_date,\n left = box[1], right = box[3],\n top = box[4], bottom = box[2]),\n dx = 0.0001, dy = 0.0001, dt = \"P1D\",\n aggregation = \"median\", resampling = \"average\")\n\nmask <- image_mask(\"SCL\", values=c(3, 8, 9)) # mask clouds and cloud shadows\n\ndata <- raster_cube(col, cube, mask = mask)\n```\n:::\n\n\n\nWe can do arbitrary calculations on this data as well. Here we calculate NDVI, a widely used measure of greenness that can be used to determine tree cover. \n(Note that the R example uses lazy evaluation, and can thus perform these calculations while streaming)\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nndvi <- data |>\n select_bands(c(\"B04\", \"B08\")) |>\n apply_pixel(\"(B08-B04)/(B08+B04)\", \"NDVI\") |>\n reduce_time(c(\"mean(NDVI)\"))\n\nndvi_stars <- st_as_stars(ndvi)\n```\n:::\n\n\n\nAnd we plot the result. The long rectangle of Golden Gate Park is clearly visible in the North-West.\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nmako <- tm_scale_continuous(values = viridisLite::mako(30))\nfill <- tm_scale_continuous(values = \"Greens\")\n\ntm_shape(ndvi_stars) + tm_raster(col.scale = mako)\n```\n\n::: {.cell-output-display}\n![](1-intro-R_files/figure-html/unnamed-chunk-5-1.png){width=672}\n:::\n:::\n\n\n\n# From NDVI to Environmental Justice\n\nWe examine the present-day impact of historic \"red-lining\" of US cities during the Great Depression using data from the [Mapping Inequality](https://dsl.richmond.edu/panorama/redlining) project. All though this racist practice was banned by federal law under the Fair Housing Act of 1968, the systemic scars of that practice are still so deeply etched on our landscape that the remain visible from space -- \"red-lined\" areas (graded \"D\" under the racist HOLC scheme) show systematically lower greenness than predominately-white neighborhoods (Grade \"A\"). Trees provide many benefits, from mitigating urban heat to biodiversity, real-estate value, to health.\n\n\n## Zonal statistics \n\nIn addition to large scale raster data such as satellite imagery, the analysis of vector shapes such as polygons showing administrative regions is a central component of spatial analysis, and particularly important to spatial social sciences. The red-lined areas of the 1930s are one example of spatial vectors. One common operation is to summarise the values of all pixels falling within a given polygon, e.g. computing the average greenness (NDVI) \n\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nsf <- \"/vsicurl/https://dsl.richmond.edu/panorama/redlining/static/citiesData/CASanFrancisco1937/geojson.json\" |>\n st_read() |>\n st_make_valid() |>\n select(-label_coords)\n```\n:::\n\n::: {.cell}\n\n```{.r .cell-code}\npoly <- ndvi |> extract_geom(sf, FUN = mean, reduce_time = TRUE)\nsf$NDVI <- poly$NDVI\n```\n:::\n\n\n\nWe plot the underlying NDVI as well as the average NDVI of each polygon, along with it's textual grade, using `tmap`. Note that \"A\" grades tend to be darkest green (high NDVI) while \"D\" grades are frequently the least green. (Regions not zoned for housing at the time of the 1937 housing assessment are not displayed as polygons.)\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\ntm_shape(ndvi_stars) + tm_raster(col.scale = mako) +\n tm_shape(sf) + tm_polygons('NDVI', fill.scale = fill) +\n tm_shape(sf) + tm_text(\"grade\", col=\"darkblue\", size=0.6) +\n tm_legend_hide()\n```\n\n::: {.cell-output-display}\n![](1-intro-R_files/figure-html/unnamed-chunk-8-1.png){width=672}\n:::\n:::\n\n\n\nAre historically redlined areas still less green?\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nsf |> \n as_tibble() |>\n group_by(grade) |> \n summarise(ndvi = mean(NDVI), \n sd = sd(NDVI)) |>\n knitr::kable()\n```\n\n::: {.cell-output-display}\n|grade | ndvi| sd|\n|:-----|---------:|---------:|\n|A | 0.3201204| 0.0611414|\n|B | 0.2138501| 0.0783221|\n|C | 0.1956334| 0.0564822|\n|D | 0.1949736| 0.0385805|\n|NA | 0.0962092| NA|\n:::\n:::\n", "supporting": [ "1-intro-R_files" ],