| boundingBox | Inventory with a spatial extent overlapping\n",
diff --git a/book/opensearch_7_cmr.ipynb b/book/opensearch_7_cmr.ipynb
new file mode 100644
index 0000000..fad617e
--- /dev/null
+++ b/book/opensearch_7_cmr.ipynb
@@ -0,0 +1,581 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "id": "d99fea47",
+ "metadata": {},
+ "source": [
+ "(chapter-cwic)=\n",
+ "# Use Case - Granule Search (CWIC/CMR)\n",
+ "\n",
+ "This chapter provides a comprehensive and detailed process about how to implement a WGISS\n",
+ "OpenSearch client, which includes how to retrieve the IDN collection ID for the collection of\n",
+ "interest, and how to build an OpenSearch request for a CWIC data partner. Within CMR, CWIC\n",
+ "partners are available directly from CMR. The IDN response records contain direct links to the\n",
+ "OSDD from the CWIC partner server, where possible. This can be used to construct a granule\n",
+ "search directly from the partner."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "id": "30675075",
+ "metadata": {
+ "tags": [
+ "remove-cell"
+ ]
+ },
+ "outputs": [],
+ "source": [
+ "import re\n",
+ "import json, requests, xml\n",
+ "import pandas as pd\n",
+ "\n",
+ "from xml.dom import minidom\n",
+ "from xml.etree import ElementTree\n",
+ "from IPython.display import Markdown as md\n",
+ "# from IPython.display import HTML as html"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "id": "0bc1e2cc",
+ "metadata": {
+ "tags": [
+ "remove-cell"
+ ]
+ },
+ "outputs": [],
+ "source": [
+ "def get_api_request(template, os_querystring):\n",
+ " #\n",
+ " # Fill (URL) template with OpenSearch parameter values provided in os_querystring and \n",
+ " # return as short HTTP URL without empty parameters.\n",
+ " #\n",
+ " \n",
+ " # Limitation: the OSDD may use a default namespace for OpenSearch instead of using \"os\".\n",
+ " # We make a simple correction here allowing to use OpenSearch queryables without namespace in requests.\n",
+ " # A more generic solution to obtain namespaces from the OSDD and compare them with user supplied namespaces is future work.\n",
+ " \n",
+ " OS_NAMESPACE = 'os:'\n",
+ " \n",
+ " # perform substitutions in template\n",
+ " for p in os_querystring:\n",
+ " # print(\" .. replacing:\", p, \"by\", os_querystring[p])\n",
+ " # template = re.sub('\\{'+p+'.*?\\}', os_querystring[p] , template)\n",
+ " result = re.subn('\\{'+p+'.*?\\}', os_querystring[p] , template)\n",
+ " n = result[1]\n",
+ " template = result[0]\n",
+ " if (n<1):\n",
+ " if (':' in p):\n",
+ " print(\"ERROR: parameter \" + p + \" not found in template.\")\n",
+ " else:\n",
+ " # try with explicit namespace\n",
+ " result = re.subn('\\{'+OS_NAMESPACE+p+'.*?\\}', os_querystring[p] , template)\n",
+ " n = result[1]\n",
+ " template = result[0]\n",
+ " if (n<1):\n",
+ " print(\"ERROR: parameter \" + OS_NAMESPACE+p + \" not found in template.\") \n",
+ " \n",
+ " # print(\"- intermediate new template:\" + template)\n",
+ " \n",
+ " # remove empty search parameters\n",
+ " template=re.sub('&?[a-zA-Z]*=\\{.*?\\}', '' , template)\n",
+ " \n",
+ " # remove remaining empty search parameters which did not have an HTTP query parameter attached (e.g. /{time:end}).\n",
+ " template=re.sub('.?\\{.*?\\}', '' , template)\n",
+ " \n",
+ " # print(\"API request: \" + template)\n",
+ " \n",
+ " return (template)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4cdb746c",
+ "metadata": {},
+ "source": [
+ "## Retrieve Collection OSDD via IDN OpenSearch"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "27f7a410",
+ "metadata": {},
+ "source": [
+ "The steps in this section are identical to the steps explained in the section [Retrieve Collections via IDN OpenSearch](section_Retrieve_Collections_via_IDN_OpenSearch). To fit the end-to-end\n",
+ "CWIC use case, the parameter values for the collection search has been selected in such a way that\n",
+ "the granule search is eventually performed through CWIC.\n",
+ "\n",
+ "\n",
+ "| **Use Case Overview** | | \n",
+ "| -------- | --------- | \n",
+ "| Title | Clients start from the IDN OpenSearch | \n",
+ "| Description | This use case describes steps for retrieving a collection ID from the IDN OpenSearch. | \n",
+ "| Actors | OpenSearch client, IDN OpenSearch Server | \n",
+ "| Initial Status and Preconditions | Clients have the IDN OpenSearch access URL. | \n",
+ "\n",
+ "The following steps describe this use case."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "730d2d61",
+ "metadata": {},
+ "source": [
+ "**Step 1** \n",
+ "> Obtain the IDN OpenSearch OSDD to formulate a valid IDN OpenSearch request."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "id": "8572ead4",
+ "metadata": {
+ "tags": [
+ "remove-input"
+ ]
+ },
+ "outputs": [],
+ "source": [
+ "url_osdd = \"https://cmr.earthdata.nasa.gov/opensearch/collections/descriptor_document.xml?clientId=ceosOpenSearchDoc\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "id": "2ed09441",
+ "metadata": {
+ "tags": [
+ "remove-input"
+ ]
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/markdown": [
+ "The OpenSearch Description Document is accessible at the fixed location [https://cmr.earthdata.nasa.gov/opensearch/collections/descriptor_document.xml?clientId=ceosOpenSearchDoc](https://cmr.earthdata.nasa.gov/opensearch/collections/descriptor_document.xml?clientId=ceosOpenSearchDoc) and contains the URL template to be used for collection search."
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 29,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "md(\"The OpenSearch Description Document is accessible at the fixed location [{url}]({url}) and contains the URL template to be used for collection search.\".format(url=url_osdd))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "id": "01c80dcd",
+ "metadata": {
+ "tags": [
+ "output_scroll"
+ ]
+ },
+ "outputs": [],
+ "source": [
+ "response = requests.get( url_osdd )\n",
+ "\n",
+ "xmlstr = minidom.parseString(response.text).toprettyxml(indent=' ',newl='')\n",
+ "# md(\"```xml\\n\" + xmlstr + \"\\n```\\n\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "07aa5a21",
+ "metadata": {},
+ "source": [
+ "Find the URL template for collection search with Atom responses."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "id": "fd8cdefc",
+ "metadata": {
+ "tags": [
+ "output_scroll"
+ ]
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'https://cmr.earthdata.nasa.gov/opensearch/collections.atom?keyword={os:searchTerms?}&instrument={echo:instrument?}&satellite={eo:platform?}&boundingBox={geo:box?}&lat={geo:lat?}&lon={geo:lon?}&radius={geo:radius?}&geometry={geo:geometry?}&placeName={geo:name?}&startTime={time:start?}&endTime={time:end?}&cursor={os:startPage?}&numberOfResults={os:count?}&offset={os:startIndex?}&uid={geo:uid?}&hasGranules={echo:hasGranules?}&isCwic={echo:isCwic?}&isGeoss={echo:isGeoss?}&isCeos={echo:isCeos?}&isEosdis={echo:isEosdis?}&isFedeo={echo:isFedeo?}&provider={echo:provider?}&clientId=ceosOpenSearchDoc'"
+ ]
+ },
+ "execution_count": 31,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "root = ElementTree.fromstring(response.text)\n",
+ "\n",
+ "ns = {'os': 'http://a9.com/-/spec/opensearch/1.1/'}\n",
+ "collection_url_atom = root.find('os:Url[@rel=\"collection\"][@type=\"application/atom+xml\"]', ns)\n",
+ "\n",
+ "collection_template = collection_url_atom.attrib['template']\n",
+ "collection_template"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "97a967cf",
+ "metadata": {},
+ "source": [
+ "**Step 2** \n",
+ "> Search collections of interest through IDN OpenSearch with proper request parameters (e.g. spatial footprint, temporal extent and keyword). A complete list of supported request parameters, extracted from the IDN OpenSearch OSDD, was included in the section [Available Collection Search Criteria](section_Available_Collection_Search_Criteria_IDN)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0d3f24c0",
+ "metadata": {},
+ "source": [
+ "An example request can be formed as follows."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 32,
+ "id": "d78fe4c0",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'https://cmr.earthdata.nasa.gov/opensearch/collections.atom?keyword=Landsat_8&numberOfResults=1&clientId=ceosOpenSearchDoc'"
+ ]
+ },
+ "execution_count": 32,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "request_url = get_api_request(collection_template, {'count': '1', 'searchTerms': 'Landsat_8'})\n",
+ "request_url"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 33,
+ "id": "d0e72b46",
+ "metadata": {
+ "tags": [
+ "output_scroll"
+ ]
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/markdown": [
+ "```xml\n",
+ " \n",
+ " 2024-12-06T15:51:03.435Z \n",
+ " https://cmr.earthdata.nasa.gov/opensearch/collections.atom \n",
+ " \n",
+ " CMR \n",
+ " echodev@echo.nasa.gov \n",
+ " \n",
+ " ECHO dataset metadata \n",
+ " Search parameters: keyword => Landsat_8 \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " 1 \n",
+ " 1 \n",
+ " 1 \n",
+ " \n",
+ " https://cmr.earthdata.nasa.gov/opensearch/collections.atom?uid=C1235542031-USGS_LTA \n",
+ " \n",
+ " CMR \n",
+ " echodev@echo.nasa.gov \n",
+ " \n",
+ " CEOS \n",
+ " CWIC \n",
+ " Landsat 8 \n",
+ " The Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) are onboard the Landsat 8 satellite, have acquired images of the Earth since February 2013. The sensors collect images of the Earth with a 16-day repeat cycle, referenced to the Worldwide Reference System-2. The approximate scene size is 170 km north-south by 183 km east-west (106 mi by 114 mi).\n",
+ "\n",
+ "Landsat 8 image data files consist of 11 spectral bands with a spatial resolution of 30 meters for bands 1-7 and bands 9-11; 15-meters for the panchromatic band 8. Delivered Landsat 8 Level-1 data typically include both OLI and TIRS data files; however, there may be OLI-only and/or TIRS-only scenes in the USGS archive. A Quality Assurance (QA.tif) band is also included. This file provides bit information regarding conditions that may affect the accuracy and usability of a given pixel – clouds, water or snow, for example. \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " C1235542031-USGS_LTA \n",
+ " 2013-02-11T00:00:00.000Z/ \n",
+ " Landsat 8 \n",
+ " Landsat_8 \n",
+ " Not provided \n",
+ " Landsat_8 \n",
+ " USGS_LTA \n",
+ " DOI/USGS/EROS \n",
+ " DOI/USGS/EROS \n",
+ " Not Provided \n",
+ " CARTESIAN \n",
+ " \n",
+ " -82.71 -180 82.74 180 \n",
+ " false \n",
+ " false \n",
+ " false \n",
+ " false \n",
+ " false \n",
+ " false \n",
+ " false \n",
+ " 0.97999995 \n",
+ " \n",
+ " org.ceos.wgiss.cwic.granules.prod \n",
+ " " " \n",
+ " \n",
+ " \n",
+ " org.ceos.wgiss.cwic.granules.provider \n",
+ " "USGSLSI" \n",
+ " \n",
+ " \n",
+ " org.geoss.geoss_data-core \n",
+ " \n",
+ " \n",
+ " gov.nasa.eosdis \n",
+ " \n",
+ " true \n",
+ " true \n",
+ " \n",
+ "\n",
+ "```\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 33,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "response = requests.get( request_url )\n",
+ "xmlstr = minidom.parseString(response.text).toprettyxml(indent=' ', newl='')\n",
+ "md(\"```xml\\n\" + xmlstr + \"\\n```\\n\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f14a17b4",
+ "metadata": {},
+ "source": [
+ "**Step 3** \n",
+ "> From the IDN OpenSearch response, obtain the OSDD endpoint for the collection by parsing the href attribute under `` "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 34,
+ "id": "15c70321",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/markdown": [
+ "```xml\n",
+ "\n",
+ " \n",
+ "```\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 34,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "root = ElementTree.fromstring(response.text)\n",
+ "# Extract element with the OSDD for the granule search\n",
+ "el = root.find('{http://www.w3.org/2005/Atom}entry/{http://www.w3.org/2005/Atom}link[@rel=\"search\"][@type=\"application/opensearchdescription+xml\"]')\n",
+ "xmltxt = ElementTree.tostring(el, encoding='unicode', method='xml')\n",
+ "md(\"```xml\\n\" + xmltxt + \"\\n```\\n\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a20ecd3c",
+ "metadata": {},
+ "source": [
+ "This response will provide the key component of searching CMR for the desired collection using\n",
+ "the CMR conceptID. For Landsat 8, this will be C1235542031-USGS_LTA. Now obtain from\n",
+ "CMR the URL for the local OSDD (in this case, at USGS), with the request"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 35,
+ "id": "e612a026",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml?collectionConceptId=C1235542031-USGS_LTA'"
+ ]
+ },
+ "execution_count": 35,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "url_osdd = el.attrib['href']\n",
+ "el.attrib['href']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3f2fa331",
+ "metadata": {},
+ "source": [
+ "Retrieve the response (i.e. the OSDD document)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 36,
+ "id": "e2a03174",
+ "metadata": {
+ "tags": [
+ "output_scroll"
+ ]
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/markdown": [
+ "```xml\n",
+ " \n",
+ " OpenSearch Exception \n",
+ " No native id present within the `org.ceos.wgiss.cwic.granules.native_id` tag. \n",
+ " 2024-12-06T15:51:04Z \n",
+ " https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml?collectionConceptId=C1235542031-USGS_LTA \n",
+ " \n",
+ " 0 \n",
+ "\n",
+ "```\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 36,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "response = requests.get( el.attrib['href'] )\n",
+ "\n",
+ "xmlstr = minidom.parseString(response.text).toprettyxml(indent=' ',newl='')\n",
+ "md(\"```xml\\n\" + xmlstr + \"\\n```\\n\")\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0a755fde",
+ "metadata": {},
+ "source": [
+ "```{warning}\n",
+ "In the previous step, the OSDD could not be retrieved as the URL included in the response is not the one with `short_name`. See also step 5 in chapter 5 \"Outline\" with the workaround to fix that.\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "af400462",
+ "metadata": {},
+ "source": [
+ "## Retrieve Granules via OpenSearch"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c7985450",
+ "metadata": {},
+ "source": [
+ "## Available Granule Search Parameters\n",
+ "\n",
+ "After retrieving the OSDD endpoint by querying through the IDN OpenSearch, OpenSearch clients\n",
+ "will sequentially interact with the host remote server for inventory search. The following table\n",
+ "shows the basic information about the use case of interacting with WGISS Connected Data Assets\n",
+ "via OpenSearch.\n",
+ "\n",
+ "\n",
+ "TBD"
+ ]
+ }
+ ],
+ "metadata": {
+ "description": "This notebook explains the use of the OpenSearch interface with Atom response format to access collection and granule metadata according to the two-step mechanism recommended by CEOS Best Practices. It can be used with the ESA EO-CAT and CEOS FedEO catalogue endpoints.",
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ },
+ "potentialAction": [
+ {
+ "@type": "InteractAction",
+ "name": "Launch Binder",
+ "target": "https://mybinder.org/v2/gh/eovoc/eo-books/main?urlpath=tree/docs/opensearch.ipynb"
+ },
+ {
+ "@type": "InteractAction",
+ "name": "Open in Google Colab",
+ "target": "https://colab.research.google.com/github/eovoc/eo-books/blob/main/docs/opensearch.ipynb"
+ }
+ ],
+ "tags": {
+ "tags": [
+ "EOCAT",
+ "FedEO",
+ "OpenSearch",
+ "Data Access/Retrieval",
+ "Metadata Handling",
+ "Service Discovery",
+ "Data Discovery",
+ "CDA",
+ "CEOS",
+ "OGC",
+ "ESA"
+ ]
+ },
+ "title": "OpenSearch with Atom"
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/book/opensearch_8_fedeo.ipynb b/book/opensearch_8_fedeo.ipynb
index 1f53880..f4227c4 100644
--- a/book/opensearch_8_fedeo.ipynb
+++ b/book/opensearch_8_fedeo.ipynb
@@ -6,6 +6,7 @@
"id": "d99fea47",
"metadata": {},
"source": [
+ "(chapter-fedeo)=\n",
"# Use Case - Granule Search (FedEO)\n",
"\n",
"This chapter provides a comprehensive and detailed process about how to implement a CEOS\n",
@@ -128,7 +129,7 @@
"id": "27f7a410",
"metadata": {},
"source": [
- "The steps in this section are identical to the steps explained in section 6.2. To fit the end-to-end\n",
+ "The steps in this section are identical to the steps explained in the section [Retrieve Collections via IDN OpenSearch](section_Retrieve_Collections_via_IDN_OpenSearch). To fit the end-to-end\n",
"FedEO use case, the parameter values for the collection search has been selected in such a way\n",
"that the granule search is eventually performed through FedEO.\n",
"\n",
@@ -5617,7 +5618,7 @@
"### Available Granule Search Parameters\n",
"\n",
"The set of available granule search parameters depends on the actual collection and is not fixed. Please refer to the [OpenAPI description](https://fedeo.ceos.org/api) for the complete list of search parameters that\n",
- "may be advertised by a granule search OSDD. The meaning of these parameters is defined in [CEOS-OS-BP],\n",
+ "may be advertised by a granule search OSDD. The meaning of these parameters is defined in [CEOS-OS-BP](chapter-references),\n",
"[OGC 13-026r9](https://docs.ogc.org/is/13-026r9/13-026r9.html) and [OGC 10-032r8](https://portal.ogc.org/files/?artifact_id=56866)."
]
},
diff --git a/book/references.md b/book/references.md
index d2e1cba..48171dd 100644
--- a/book/references.md
+++ b/book/references.md
@@ -1,3 +1,4 @@
+(chapter-references)=
# References
The following documents provide more background and supportive information.
|