examples/xarray-spatial_classification-methods.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "89bdd2bf",
   "metadata": {},
   "source": [
    "## Grand Canyon elevation classification using the NASADEM dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "837aff2c",
   "metadata": {},
   "source": [
    "In this tutorial, you'll learn how to use different classification methods with [xarray-spatial](https://github.com/makepath/xarray-spatial) to classify the terrain elevation levels of the Grand Canyon.\n",
    "\n",
    "Geo-spatial data [classification](http://wiki.gis.com/wiki/index.php/Classification) algorithms assign groups of data to categories, or classes, for further processing. Classification is used when grouping data points into classes for different colored areas on a choropleth map, for example. [xarray-spatial](https://github.com/makepath/xarray-spatial) is a raster analysis tool and contains different classification methods.\n",
    "\n",
    "This tutorial walks you through:\n",
    "1. Loading and rendering the area of interest data using the Grand Canyon's latitude and longitude.\n",
    "2. Classifying the data using xarray-spatial's [natural breaks](https://xarray-spatial.org/reference/_autosummary/xrspatial.classify.natural_breaks.html), [equal interval](https://xarray-spatial.org/reference/_autosummary/xrspatial.classify.equal_interval.html), [quantile](https://xarray-spatial.org/reference/_autosummary/xrspatial.classify.quantile.html), and [reclassify](https://xarray-spatial.org/reference/_autosummary/xrspatial.classify.reclassify.html) functions.\n",
    "\n",
    "\n",
    "This tutorial uses the [NASADEM](https://github.com/microsoft/AIforEarthDatasets#nasadem) dataset from the [Microsoft Planetary Computer Data Catalog](https://planetarycomputer.microsoft.com/catalog). The area of interest roughly covers the Grand Canyon National Park. The [NASADEM](https://github.com/microsoft/AIforEarthDatasets#nasadem) dataset provides global topographic data at 1 arc-second (~30m) horizontal resolution. The data is derived primarily from data captured via the [Shuttle Radar Topography Mission](https://www2.jpl.nasa.gov/srtm/) (SRTM) and is stored on Azure Storage in [cloud-optimized GeoTIFF](https://www.cogeo.org/) format.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f83b10df",
   "metadata": {},
   "source": [
    "### 1. Load the area of interest data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90ef8812",
   "metadata": {},
   "source": [
    "To load NASADEM data for the Grand Canyon, use the following approach described in [Accessing NASADEM data on Azure (NetCDF)](https://github.com/microsoft/AIforEarthDataSets/blob/main/data/nasadem-nc.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d69cc680-4ee5-4ed6-a785-31aa44173618",
   "metadata": {},
   "source": [
    "First, set up the necessary constants and generate a list of all available GeoTIFF files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c4c7fae",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "nasadem_blob_root = \"https://nasademeuwest.blob.core.windows.net/nasadem-cog/v001/\"\n",
    "nasadem_file_index_url = nasadem_blob_root + \"index/nasadem_cog_list.txt\"\n",
    "nasadem_content_extension = \".tif\"\n",
    "nasadem_file_prefix = \"NASADEM_HGT_\"\n",
    "nasadem_file_list = None\n",
    "\n",
    "nasadem_file_list = requests.get(nasadem_file_index_url).text.split(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b257fbfb-889c-4686-a527-2eb9b81c63e3",
   "metadata": {},
   "source": [
    "Next, define a function that selects a filename from the list generated in the previous step. This function accepts a list of latitude and longitude coordinates and returns the name of the file matching these coordinates:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0fc048ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "\n",
    "def get_nasadem_filename(coord):\n",
    "    \"\"\"\n",
    "    Get the NASADEM filename for a specified latitude and longitude.\n",
    "    \"\"\"\n",
    "    lat = coord[0]\n",
    "    lon = coord[1]\n",
    "\n",
    "    ns_token = \"n\" if lat >= 0 else \"s\"\n",
    "    ew_token = \"e\" if lon >= 0 else \"w\"\n",
    "\n",
    "    lat_index = abs(math.floor(lat))\n",
    "    lon_index = abs(math.floor(lon))\n",
    "\n",
    "    lat_string = ns_token + \"{:02d}\".format(lat_index)\n",
    "    lon_string = ew_token + \"{:03d}\".format(lon_index)\n",
    "\n",
    "    filename = nasadem_file_prefix + lat_string + lon_string + nasadem_content_extension\n",
    "\n",
    "    if filename not in nasadem_file_list:\n",
    "        print(\"Lat/lon {},{} not available\".format(lat, lon))\n",
    "        filename = None\n",
    "\n",
    "    return filename"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c581d16d-c1a8-4cce-ad77-94b9f98ca9da",
   "metadata": {},
   "source": [
    "Finally, use the function defined above to generate a URL pointing to the geodata for the Grand Canyon area:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "074aa0f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "grand_canyon_coord = [36.101690, -112.107676]\n",
    "\n",
    "url = nasadem_blob_root + get_nasadem_filename(grand_canyon_coord)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9149231c",
   "metadata": {},
   "source": [
    "After retrieving the raw data for the Grand Canyon, use xarray's [open_rasterio](http://xarray.pydata.org/en/stable/generated/xarray.open_rasterio.html) function to load the data into an array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5476b1fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "img_arr = xr.open_rasterio(url).squeeze().drop(\"band\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6ae27af",
   "metadata": {},
   "outputs": [],
   "source": [
    "img_arr.plot.imshow(figsize=(15, 10));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42c128aa",
   "metadata": {},
   "source": [
    "### 2. Classify elevation data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d1f4e54",
   "metadata": {},
   "source": [
    "#### Classify with `natural_breaks()`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41759319",
   "metadata": {},
   "source": [
    "Use the [natural breaks](https://xarray-spatial.org/reference/_autosummary/xrspatial.classify.natural_breaks.html) function to classify data with the [Jenks natural breaks classification](http://wiki.gis.com/wiki/index.php/Jenks_Natural_Breaks_Classification) method. This method is designed to distribute data into classes according to clusters that form a \"natural\" group within the data. The algorithm minimizes the average deviation from the class mean while also maximizing the deviation from the means of the other groups. Therefore, it is generally not recommended for data with low variance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c5d7b05",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datashader.transfer_functions import shade\n",
    "import matplotlib.pyplot as plt\n",
    "from xrspatial.classify import natural_breaks\n",
    "\n",
    "natural_breaks_agg = natural_breaks(img_arr, num_sample=20000, k=15)\n",
    "\n",
    "shade(natural_breaks_agg, cmap=plt.get_cmap(\"terrain\"), how=\"linear\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40f7ee45",
   "metadata": {},
   "source": [
    "You can see in the image above that the different elevation levels of the Grand Canyon are now identified by a limited number of distinct colors. Each color represents a range of values within the classified data. For example, the dark blue color areas represent the smallest elevation levels with around 700m of altitude, the yellow color areas represent elevation levels of around 1700m, and the white color areas represent the highest elevations of around 2500m of altitude."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4797d619",
   "metadata": {},
   "source": [
    "#### Classify with `equal_interval()`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baed3e1b",
   "metadata": {},
   "source": [
    "To classify data into sets based on intervals of equal width, use the [equal interval](https://xarray-spatial.org/reference/_autosummary/xrspatial.classify.equal_interval.html) function. The [equal interval classification](http://wiki.gis.com/wiki/index.php/Equal_Interval_classification) is useful in cases where you want to emphasize the amount of an attribute value relative to the other values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e51b766",
   "metadata": {},
   "outputs": [],
   "source": [
    "from xrspatial.classify import equal_interval\n",
    "\n",
    "equal_interval_agg = equal_interval(img_arr, k=15)\n",
    "\n",
    "shade(equal_interval_agg, cmap=plt.get_cmap(\"terrain\"), how=\"linear\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13c46393",
   "metadata": {},
   "source": [
    "#### Classify with `quantile()`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4630ca7",
   "metadata": {},
   "source": [
    "To classify data based on quantile groups of equal size, use the [quantile](https://xarray-spatial.org/reference/_autosummary/xrspatial.classify.quantile.html) function. With [quantile classification](http://wiki.gis.com/wiki/index.php/Quantile), each class contains the same amount of data points. This means that each class is equally represented on the map. However, intervals of uneven sizes can lead to an over-weighting of outliers and other effects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "311879ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "from xrspatial.classify import quantile\n",
    "\n",
    "quantile_agg = quantile(img_arr, k=15)\n",
    "\n",
    "shade(quantile_agg, cmap=plt.get_cmap(\"terrain\"), how=\"linear\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dec22f8",
   "metadata": {},
   "source": [
    "### Use custom bins with `reclassify`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13f0851f",
   "metadata": {},
   "source": [
    "To define your own arbitrary bins to classify data, use the [reclassify](https://xarray-spatial.org/reference/_autosummary/xrspatial.classify.reclassify.html) function. This function is helpful to highlight specific sections of your data, for example. Use `reclassify()` to only visualize elevations greater than 2500m:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1060a994",
   "metadata": {},
   "outputs": [],
   "source": [
    "from xrspatial.classify import reclassify\n",
    "\n",
    "bins = range(150, 5200, 350)\n",
    "new_vals = [val if val > 2500 else 0 for val in bins]\n",
    "\n",
    "reclass_agg = reclassify(\n",
    "    agg=img_arr,\n",
    "    bins=bins,\n",
    "    new_values=new_vals,\n",
    ")\n",
    "\n",
    "shade(reclass_agg, cmap=plt.get_cmap(\"terrain\"), how=\"linear\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd7822fe",
   "metadata": {},
   "source": [
    "## Next steps: classify different datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cea4d59c",
   "metadata": {},
   "source": [
    "The [Microsoft Planetary Computer Data Catalog](https://planetarycomputer.microsoft.com/catalog) includes petabytes of environmental monitoring data. All data sets are available in consistent, analysis-ready formats. You can access them through APIs as well as directly via [Azure Storage](https://docs.microsoft.com/en-us/azure/storage/). \n",
    "\n",
    "Try using [xarray-spatial's](https://xarray-spatial.org/index.html) classification methods with these datasets:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bb5b5a0",
   "metadata": {},
   "source": [
    "<div style=\"width: 100%; overflow: hidden;\">\n",
    "     <div style=\"width: 50%; float: left;\"> \n",
    "  \n",
    "  <center><img src=\"https://ai4edatasetspublicassets.blob.core.windows.net/assets/pc_thumbnails/additional_datasets/RE3CbUs.jpg\" /></center>\n",
    "<br>\n",
    "<center><font size=\"5\">Daymet</font>\n",
    "<center><font size=\"2\">Gridded temperature data across North America</font>\n",
    "<center><a href=\"http://aka.ms/ai4edata-daymet\" target=\"_blank\">Get Daymet temperature data</a>\n",
    "  </div>\n",
    "     <div style=\"margin-left: 50%;\"> \n",
    "  <center><img src=\"https://ai4edatasetspublicassets.blob.core.windows.net/assets/pc_thumbnails/additional_datasets/gbif.jpg\" /><center>\n",
    "<br>\n",
    "<center><font size=\"5\">GBIF</font>\n",
    "<center><font size=\"2\">Species occurrences shared through the Global Biodiversity Information Facility</font>\n",
    "<center><a href=\"http://aka.ms/ai4edata-gbif/\" target=\"_blank\">Get GBIF occurrence data</a>\n",
    "  </div>\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}