diff --git a/src/temporal/t.stac/Makefile b/src/temporal/t.stac/Makefile new file mode 100644 index 0000000000..8c182e4daa --- /dev/null +++ b/src/temporal/t.stac/Makefile @@ -0,0 +1,15 @@ +MODULE_TOPDIR =../.. + +PGM = t.stac + +SUBDIRS = libstac \ + t.stac.catalog \ + t.stac.collection \ + t.stac.item \ + +include $(MODULE_TOPDIR)/include/Make/Dir.make + +default: parsubdirs htmldir + +install: installsubdirs + $(INSTALL_DATA) $(PGM).html $(INST_DIR)/docs/html/ diff --git a/src/temporal/t.stac/README.md b/src/temporal/t.stac/README.md new file mode 100644 index 0000000000..a70f113cf8 --- /dev/null +++ b/src/temporal/t.stac/README.md @@ -0,0 +1,125 @@ +# (In-Development) t.stac + +## Description + +The **t.stac** toolset utilizes the +[pystac-client (v0.5.1)](https://github.com/stac-utils/pystac-client) to search +STAC APIs and import items into GRASS GIS. + +### Item Search Parameters + +[Offical Docs](https://pystac-client.readthedocs.io/en/stable/api.html#item-search) + +**method** – The HTTP method to use when making a request to the service. This +must be either "GET", "POST", or None. If None, this will default to "POST". +If a "POST" request receives a 405 status for the response, it +will automatically retry with "GET" for all subsequent requests. + +**max_items** – The maximum number of items to return from the search, even if +there are more matching results. This will limit the total number of Items +returned from the items(), item_collections(), and items_as_dicts methods(). +The client will continue to request pages of items until the number of max +items is reached. This parameter defaults to 100. Setting this to None will +allow iteration over a possibly very large number of results. + +**limit** – A recommendation to the service as to the number of items to return +per page of results. Defaults to 100. + +**ids** – List of one or more Item IDs to filter on. + +**collections** – List of one or more Collection IDs or pystac.Collection +instances. Only Items in one of the provided Collections will be searched. + +**bbox** – A list, tuple, or iterator representing a bounding box of 2D +or 3D coordinates. Results will be filtered to only those intersecting the +bounding box. + +**intersects** – A string or dictionary representing a GeoJSON geometry, +or an object that implements a ``__geo_interface__`` property, as supported +by several libraries including Shapely, ArcPy, PySAL, and geojson. +Results filtered to only those intersecting the geometry. + +**datetime –** +Either a single datetime or datetime range used to filter results. Users may +express a single datetime using a datetime.datetime instance, an +RFC 3339-compliant timestamp, or a simple date string (see below). +Instances of datetime.datetime may be either timezone aware or unaware. +Timezone aware instances will be converted to a UTC timestamp before being +passed to the endpoint. Timezone unaware instances are assumed to represent +UTC timestamps. You may represent a datetime range using a "/" separated +string as described in the spec, or a list, tuple, or iterator of 2 timestamps +or datetime instances. For open-ended ranges, use either ".." +('2020-01-01:00:00:00Z/..', ['2020-01-01:00:00:00Z', '..']) or +None (['2020-01-01:00:00:00Z', None]). + +If using a simple date string, the datetime can be specified in YYYY-mm-dd +format, optionally truncating to YYYY-mm or just YYYY. Simple date strings +will be expanded to include the entire time period, for example: + +* 2017 expands to 2017-01-01T00:00:00Z/2017-12-31T23:59:59Z +* 2017-06 expands to 2017-06-01T00:00:00Z/2017-06-30T23:59:59Z +* 2017-06-10 expands to 2017-06-10T00:00:00Z/2017-06-10T23:59:59Z + +If used in a range, the end of the range expands to +the end of that day/month/year, for example: + +* 2017/2018 expands to 2017-01-01T00:00:00Z/2018-12-31T23:59:59Z +* 2017-06/2017-07 expands to 2017-06-01T00:00:00Z/2017-07-31T23:59:59Z +* 2017-06-10/2017-06-11 expands to 2017-06-10T00:00:00Z/2017-06-11T23:59:59Z + +**query** – List or JSON of query parameters as per the STAC API query extension. + +**filter** – JSON of query parameters as per the STAC API filter extension. + +**filter_lang** – Language variant used in the filter body. If filter is a +dictionary or not provided, defaults to `cql2-json`. If filter is a string, +defaults to `cql2-text`. + +**sortby** – A single field or list of fields to sort the response by. + +**fields** – A list of fields to include in the response. +Note this may result in invalid STAC objects, as they may not have the fields +required. Use items_as_dicts to avoid object unmarshalling errors. + +### Dependencies + +* [pystac-client (v0.5.1)](https://github.com/stac-utils/pystac-client) + +#### Optional Query + +* [STAC API - Query Extension Specification](https://github.com/stac-api-extensions/query) + +### S3 and GCS Requester Pays Support + +* [GDAL Docs](https://gdal.org/user/virtual_file_systems.html#introduction) +* [STAC API - S3 Requester Pays](https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files) +* [STAC API - GCS Requester Pays](https://gdal.org/user/virtual_file_systems.html#vsigs-google-cloud-storage-files) + +## Workflow example + +```python +STAC_API_URL = "https://planetarycomputer.microsoft.com/api/stac/v1" + +# catalog metadata to explore collections available +catalog = gs.run_command( + "t.stac.catalog", + url=STAC_API_URL, + format="json" +) + +# selected collection metadata to explore items within the collection +collection = gs.run_command( + "t.stac.collection", + url=STAC_API_URL, + collection_id="sentinel-2-l2a", + format="json" +) + +# selected item metadata to explore assets available +items = gs.run_command( + "t.stac.item", + url=STAC_API_URL, + collections="sentinel-2-l2a", + format="json" +) +``` diff --git a/src/temporal/t.stac/libstac/Makefile b/src/temporal/t.stac/libstac/Makefile new file mode 100644 index 0000000000..3bdbf81046 --- /dev/null +++ b/src/temporal/t.stac/libstac/Makefile @@ -0,0 +1,22 @@ +MODULE_TOPDIR = ../../.. + +include $(MODULE_TOPDIR)/include/Make/Other.make +include $(MODULE_TOPDIR)/include/Make/Python.make + +MODULES = __init__ staclib + +ETCDIR = $(ETC)/t.stac + +PYFILES := $(patsubst %,$(ETCDIR)/%.py,$(MODULES)) +PYCFILES := $(patsubst %,$(ETCDIR)/%.pyc,$(MODULES)) + +default: $(PYFILES) $(PYCFILES) + +$(ETCDIR): + $(MKDIR) $@ + +$(ETCDIR)/%: % | $(ETCDIR) + $(INSTALL_DATA) $< $@ + +install: + cp -r $(ETCDIR) $(INST_DIR)/etc diff --git a/src/temporal/t.stac/libstac/__init__.py b/src/temporal/t.stac/libstac/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/src/temporal/t.stac/libstac/staclib.py b/src/temporal/t.stac/libstac/staclib.py new file mode 100644 index 0000000000..ea609d6007 --- /dev/null +++ b/src/temporal/t.stac/libstac/staclib.py @@ -0,0 +1,539 @@ +import grass.script as gs +from grass.pygrass.gis.region import Region +from grass.pygrass.vector import VectorTopo +from grass.pygrass.vector.geometry import Point, Area, Centroid, Boundary +import base64 +import tempfile +import json +import os +from pystac_client.conformance import ConformanceClasses +from pystac_client.exceptions import APIError + + +def encode_credentials(username, password): + """Encode username and password for basic authentication""" + return base64.b64encode(f"{username}:{password}".encode("utf-8")).decode("utf-8") + + +def set_request_headers(settings): + """Set request headers""" + req_headers = {} + username = password = None + if settings == "-": + # stdin + import getpass + + username = input(_("Insert username: ")) + password = getpass.getpass(_("Insert password: ")) + + elif settings: + try: + with open(settings, "r") as fd: + lines = list( + filter(None, (line.rstrip() for line in fd)) + ) # non-blank lines only + if len(lines) < 2: + gs.fatal(_("Invalid settings file")) + username = lines[0].strip() + password = lines[1].strip() + + except IOError as e: + gs.fatal(_("Unable to open settings file: {}").format(e)) + else: + return req_headers + + if username is None or password is None: + gs.fatal(_("No user or password given")) + + if username and password: + b64_userpass = encode_credentials(username, password) + req_headers["Authorization"] = f"Basic {b64_userpass}" + + return req_headers + + +def generate_indentation(depth): + """Generate indentation for summary""" + return " " * depth + + +def print_summary(data, depth=1): + """Print summary of json data recursively increasing indentation.""" + start_depth = depth + for key, value in data.items(): + indentation = generate_indentation(start_depth) + if isinstance(value, dict): + gs.message(_(f"#\n# {indentation}{key}:")) + print_summary(value, depth=start_depth + 1) + if isinstance(value, list): + gs.message(_(f"# {indentation}{key}:")) + for item in value: + if isinstance(item, dict): + print_summary(item, depth=start_depth + 1) + else: + gs.message(_(f"# {indentation}{key}: {value}")) + + +def print_list_attribute(data, title): + "Print a list attribute" + gs.message(_(f"{title}")) + for item in data: + gs.message(_(f"\t{item}")) + + +def print_attribute(item, attribute, message=None): + """Print an attribute of the item and handle AttributeError.""" + message = message if message else attribute.capitalize() + try: + gs.message(_(f"{message}: {getattr(item, attribute)}")) + except AttributeError: + gs.info(_(f"{message} not found.")) + + +def print_basic_collection_info(collection): + """Print basic information about a collection""" + gs.message(_(f"Collection ID: {collection.get('id')}")) + gs.message(_(f"STAC Version: {collection.get('stac_version')}")) + gs.message(_(f"Description: {collection.get('description')}")) + gs.message(_(f"Extent: {collection.get('extent')}")) + gs.message(_(f"License: {collection.get('license')}")) + gs.message(_(f"Keywords: {collection.get('keywords')}")) + item_summary = collection.get("summaries") + gs.message(_(f"{'-' * 75}\n")) + if item_summary: + gs.message(_("Summary:")) + for k, v in item_summary.items(): + gs.message(_(f"{k}: {v}")) + gs.message(_(f"{'-' * 75}\n")) + item_assets = collection.get("item_assets") + item_asset_keys = item_assets.keys() + + gs.message(_(f"Item Assets Keys: {list(item_asset_keys)}")) + gs.message(_(f"{'-' * 75}\n")) + for key, value in item_assets.items(): + gs.message(_(f"Asset: {value.get('title')}")) + gs.message(_(f"Key: {key}")) + gs.message(_(f"Roles: {value.get('roles')}")) + gs.message(_(f"Type: {value.get('type')}")) + gs.message(_(f"Description: {value.get('description')}")) + if value.get("gsd"): + gs.message(_(f"GSD: {value.get('gsd')}")) + if value.get("eo:bands"): + gs.message(_("EO Bands:")) + for band in value.get("eo:bands"): + gs.message(_(f"Band: {band}")) + if value.get("proj:shape"): + gs.message(_(f"Shape: {value.get('proj:shape')}")) + if value.get("proj:transform"): + gs.message(_(f"Asset Transform: {value.get('proj:transform')}")) + if value.get("proj:crs"): + gs.message(_(f"CRS: {value.get('proj:crs')}")) + if value.get("proj:geometry"): + gs.message(_(f"Geometry: {value.get('proj:geometry')}")) + if value.get("proj:extent"): + gs.message(_(f"Asset Extent: {value.get('proj:extent')}")) + if value.get("raster:bands"): + gs.message(_("Raster Bands:")) + for band in value.get("raster:bands"): + gs.message(_(f"Band: {band}")) + + gs.message(_(f"{'-' * 75}\n")) + + +def region_to_wgs84_decimal_degrees_bbox(): + """convert region bbox to wgs84 decimal degrees bbox""" + region = gs.parse_command("g.region", quiet=True, flags="ubg") + bbox = [ + float(c) + for c in [region["ll_w"], region["ll_s"], region["ll_e"], region["ll_n"]] + ] + gs.message(_("BBOX: {}".format(bbox))) + return bbox + + +def check_url_type(url): + """ + Check if the URL is 's3://', 'gs://', or 'http(s)://'. + + Parameters: + - url (str): The URL to check. + + Returns: + - str: 's3', 'gs', 'http', or 'unknown' based on the URL type. + """ + if url.startswith("s3://"): + os.environ["AWS_PROFILE"] = "default" + os.environ["AWS_REQUEST_PAYER"] = "requester" + return url.replace("s3://", "/vsis3/") # Amazon S3 + elif url.startswith("gs://"): + return url.replace("gs://", "/vsigs/") # Google Cloud Storage + elif url.startswith("abfs://"): + return url.replace("abfs://", "/vsiaz/") # Azure Blob File System + elif url.startswith("https://"): + return url.replace("https://", "/vsicurl/https://") + # TODO: Add check for cloud provider that uses https + # return url.replace("https://", "/vsiaz/") # Azure Blob File System + # return url + elif url.startswith("http://"): + gs.warning(_("HTTP is not secure. Using HTTPS instead.")) + return url.replace("https://", "/vsicurl/https://") + else: + gs.message(_(f"Unknown Protocol: {url}")) + return "unknown" + + +def bbox_to_nodes(bbox): + """ + Convert a bounding box to polygon coordinates. + + Parameters: + bbox (dict): A dictionary with 'west', 'south', 'east', 'north' keys. + + Returns: + list of tuples: Coordinates of the polygon [(lon, lat), ...]. + """ + w, s, e, n = bbox["west"], bbox["south"], bbox["east"], bbox["north"] + # Define corners: bottom-left, top-left, top-right, bottom-right, close by returning to start + corners = [Point(w, s), Point(w, n), Point(e, n), Point(e, s), Point(w, s)] + return corners + + +def wgs84_bbox_to_boundary(bbox): + """ + Reprojects WGS84 bbox to the current locations CRS. + + Args: + bbox (list): A list containing the WGS84 bounding box coordinates in the order (west, south, east, north). + + Returns: + dict: A dictionary containing the coordinates in the order (west, south, east, north). + + Example: + >>> bbox = [-122.5, 37.5, -122, 38] + >>> wgs84_bbox_to_boundary(bbox) + {'west': -122.5, 'south': 37.5, 'east': -122, 'north': 38} + """ + + west, south, east, north = bbox + + # Reproject to location projection + min_coords = gs.read_command( + "m.proj", + coordinates=(west, south), + separator="comma", + flags="i", + ) + + max_coords = gs.read_command( + "m.proj", + coordinates=(east, north), + separator="comma", + flags="i", + ) + + min_list = min_coords.split(",")[:2] + max_list = max_coords.split(",")[:2] + + # Concatenate the lists + list_bbox = min_list + max_list + west, south, east, north = list_bbox + dict_bbox = {"west": west, "south": south, "east": east, "north": north} + + return dict_bbox + + +def safe_float_cast(value): + """Attempt to cast a value to float, return None if not possible.""" + try: + return float(value) + except ValueError: + return None + + +def polygon_centroid(polygon_coords): + """ + Create a centroid for a given polygon. + + Args: + polygon_coords (list(Point)): List of coordinates representing the polygon. + + Returns: + Centroid: The centroid of the polygon. + + """ + # Calculate the sums of the x and y coordinates + sum_x = sum( + point.x for point in polygon_coords[:-1] + ) # Exclude the last point if it's a repeat + sum_y = sum(point.y for point in polygon_coords[:-1]) + num_points = len(polygon_coords) - 1 + + # Calculate the averages for x and y + centroid_x = sum_x / num_points + centroid_y = sum_y / num_points + # Create a centroid for the boundary to make it an area + centroid = Centroid(x=centroid_x, y=centroid_y) + return centroid + + +def _flatten_dict(d, parent_key="", sep="_"): + items = [] + for k, v in d.items(): + new_key = f"{parent_key}{sep}{k}" if parent_key else k + if isinstance(v, dict): + items.extend(_flatten_dict(v, new_key, sep=sep).items()) + else: + items.append((new_key, v)) + return dict(items) + + +def create_vector_from_feature_collection(vector, search, limit, max_items): + """Create a vector from items in a Feature Collection""" + n_matched = None + try: + n_matched = search.matched() + except Exception: + gs.verbose(_("STAC API doesn't support matched() method.")) + + if n_matched: + pages = (n_matched // max_items) + 1 + else: + # These requests tend to be very slow + pages = len(list(search.pages())) + + gs.message(_(f"Fetching items {n_matched} from {pages} pages.")) + + feature_collection = {"type": "FeatureCollection", "features": []} + + # Extract asset information for each item + for page in range(pages): + temp_features = search.item_collection_as_dict() + for idx, item in enumerate(temp_features["features"]): + flattened_assets = _flatten_dict( + item["assets"], parent_key="assets", sep="." + ) + temp_features["features"][idx]["properties"].update(flattened_assets) + temp_features["features"][idx]["properties"]["collection"] = item[ + "collection" + ] + feature_collection["features"].extend(temp_features["features"]) + + json_str = json.dumps(feature_collection) + with tempfile.NamedTemporaryFile(delete=True, suffix=".json") as fp: + fp.write(bytes(json_str, "utf-8")) + fp.truncate() + gs.run_command( + "v.import", input=fp.name, output=vector, overwrite=True, quiet=True + ) + fp.close() + + gs.run_command("v.colors", map=vector, color="random", quiet=True) + + +def register_strds_from_items(collection_items_assets, strds_output): + """Create registy for STRDS from collection items assets""" + with open(strds_output, "w") as f: + for asset in collection_items_assets: + semantic_label = asset.get("file_name").split(".")[-1] + created_date = asset.get("datetime") + + if created_date: + f.write(f"{asset['file_name']}|{created_date}|{semantic_label}\n") + else: + gs.warning(_("No datetime found for item.")) + f.write(f"{asset['file_name']}|{None}|{semantic_label}\n") + + +def fetch_items_with_pagination(items_search, limit, max_items): + """ + Fetches items from a search result with pagination. + + Args: + items_search (SearchResult): The search result object. + limit (int): The maximum number of items to fetch. + max_items (int): The maximum number of items on a page. + + Returns: + list: A list of items fetched from the search result. + """ + items = [] + n_matched = None + try: + n_matched = items_search.matched() + except Exception: + gs.verbose(_("STAC API doesn't support matched() method.")) + + if n_matched and max_items is not None: + pages = (n_matched // max_items) + 1 + else: + # These requests tend to be very slow + pages = len(list(items_search.pages())) + + for page in range(pages): + page_items = items_search.items() + for item in page_items: + items.append(item) + if len(items) >= limit: + break + + return items + + +def create_metadata_vector(vector, metadata): + """Create a vector map from metadata""" + + COLS = [ + ("cat", "INTEGER PRIMARY KEY"), + ("id", "TEXT"), + ("title", "TEXT"), + ("type", "TEXT"), + # (u'description', 'TEXT'), + ("startdate", "TEXT"), + ("enddate", "TEXT"), + ("license", "TEXT"), + ("stac_version", "TEXT"), + ("keywords", "TEXT"), + ] + + with VectorTopo( + vector, mode="w", tab_cols=COLS, layer=1, overwrite=True + ) as new_vec: + + for i, item in enumerate(metadata): + gs.message(_("Adding collection: {}".format(item.get("id")))) + # Transform bbox to locations CRS + # Safe extraction + extent = item.get("extent", {}) + spatial = extent.get("spatial", {}) + temporal = extent.get("temporal", {}) + interval = temporal.get("interval", []) + if interval and isinstance(interval, list): + start_datetime, end_datetime = interval[0] + else: + start_datetime = None + end_datetime = None + + bbox_list = spatial.get("bbox", []) + # Ensure bbox_list is not empty and has the expected structure + if bbox_list and isinstance(bbox_list[0], list) and len(bbox_list[0]) == 4: + wgs84_bbox = bbox_list[0] + else: + gs.warning( + _("Invalid bbox. Skipping Collection {}.".format(item.get("id"))) + ) + continue + + bbox = wgs84_bbox_to_boundary(wgs84_bbox) + + # Iterate over the list of dictionaries and attempt to cast each value to float using safe_float_cast + if not all(safe_float_cast(i) for i in bbox.values()): + gs.warning( + _("Invalid bbox. Skipping Collection {}.".format(item.get("id"))) + ) + continue + + # Cast all values to float + for key, value in bbox.items(): + bbox[key] = safe_float_cast(value) + + # Convert the bbox to polygon coordinates + polygon_coords = bbox_to_nodes(bbox) + boundary = Boundary(points=polygon_coords) + + # Create centroid from the boundary + centroid = polygon_centroid(polygon_coords) + + # area = Area(polygon_coords) + new_vec.write(centroid) + new_vec.write( + geo_obj=boundary, + cat=i + 1, + attrs=( + item.get("id"), + item.get("title"), + item.get("type"), + # item.get("description"), + start_datetime, + end_datetime, + item.get("license"), + item.get("stac_version"), + ",".join(item.get("keywords")), + ), + ) + new_vec.table.conn.commit() + new_vec.build() + + return metadata + + +def get_all_collections(client): + """Get a list of collections from STAC Client""" + if conform_to_collections(client): + gs.verbose(_("Client conforms to Collection")) + try: + collections = client.get_collections() + collection_list = list(collections) + return [i.to_dict() for i in collection_list] + + except APIError as e: + gs.fatal(_("Error getting collections: {}".format(e))) + + +def _check_conformance(client, conformance_class, response="fatal"): + """Check if the STAC API conforms to the given conformance class""" + if not client.conforms_to(conformance_class): + if response == "fatal": + gs.fatal(_(f"STAC API does not conform to {conformance_class}")) + return False + elif response == "warning": + gs.warning(_(f"STAC API does not conform to {conformance_class}")) + return True + elif response == "verbose": + gs.verbose(_(f"STAC API does not conform to {conformance_class}")) + return True + elif response == "info": + gs.info(_(f"STAC API does not conform to {conformance_class}")) + return True + elif response == "message": + gs.message(_(f"STAC API does not conform to {conformance_class}")) + return True + + +def conform_to_collections(client): + """Check if the STAC API conforms to the Collections conformance class""" + return _check_conformance(client, ConformanceClasses.COLLECTIONS) + + +def conform_to_item_search(client): + """Check if the STAC API conforms to the Item Search conformance class""" + return _check_conformance(client, ConformanceClasses.ITEM_SEARCH) + + +def conform_to_filter(client): + """Check if the STAC API conforms to the Filter conformance class""" + return _check_conformance(client, ConformanceClasses.FILTER) + + +def conform_to_query(client): + """Check if the STAC API conforms to the Query conformance class""" + return _check_conformance(client, ConformanceClasses.QUERY) + + +def conform_to_sort(client): + """Check if the STAC API conforms to the Sort conformance class""" + return _check_conformance(client, ConformanceClasses.SORT) + + +def conform_to_fields(client): + """Check if the STAC API conforms to the Fields conformance class""" + return _check_conformance(client, ConformanceClasses.FIELDS) + + +def conform_to_core(client): + """Check if the STAC API conforms to the Core conformance class""" + return _check_conformance(client, ConformanceClasses.CORE) + + +def conform_to_context(client): + """Check if the STAC API conforms to the Context conformance class""" + return _check_conformance(client, ConformanceClasses.CONTEXT) diff --git a/src/temporal/t.stac/libstac/testsuite/test_staclib.py b/src/temporal/t.stac/libstac/testsuite/test_staclib.py new file mode 100644 index 0000000000..8dc117dabd --- /dev/null +++ b/src/temporal/t.stac/libstac/testsuite/test_staclib.py @@ -0,0 +1,127 @@ +import os +import sys +import grass.script as gs +from grass.gunittest.case import TestCase +from grass.gunittest.main import test +from grass.pygrass.utils import get_lib_path +from grass.pygrass.vector.geometry import Point + +path = get_lib_path(modname="t.stac", libname="staclib") +if path is None: + gs.fatal("Not able to find the stac library directory.") +sys.path.append(path) + +import staclib as libstac + + +class TestStaclib(TestCase): + def test_wgs84_bbox_to_boundary(self): + """Test wgs84_bbox_to_boundary""" + input_bbox = [-122.5, 37.5, -122, 38] + expected_output = { + "west": "-3117391.51", + "south": "1246003.91", + "east": "-3053969.74", + "north": "1277745.33", + } + + output = libstac.wgs84_bbox_to_boundary(bbox=input_bbox) + self.assertEqual(output, expected_output) + + def test_safe_float_cast(self): + input = { + "west": "-3117391.51", + "south": "1246003.91", + "east": "-3053969.74", + "north": "1277745.33", + } + expected_output = [-3117391.51, 1246003.91, -3053969.74, 1277745.33] + values = [libstac.safe_float_cast(i) for i in input.values()] + self.assertEqual(values, expected_output) + + def test_safe_float_cast_fail(self): + input = {"west": "*", "south": "1246003.91", "east": "*", "north": "1277745.33"} + expected_output = False + # Check if all values are float and return False if not + values = all([libstac.safe_float_cast(i) for i in input.values()]) + self.assertEqual(values, expected_output) + + def test_bbox_to_nodes(self): + """Test that Python can count to two""" + input_bbox = { + "west": -3117391.51, + "south": 1246003.91, + "east": -3053969.74, + "north": 1277745.33, + } + + # Format of the output + # [(w, s), (w, n), (e, n), (e, s), (w, s)] + expected_output = [ + Point(input_bbox["west"], input_bbox["south"]), + Point(input_bbox["west"], input_bbox["north"]), + Point(input_bbox["east"], input_bbox["north"]), + Point(input_bbox["east"], input_bbox["south"]), + Point(input_bbox["west"], input_bbox["south"]), + ] + + output = libstac.bbox_to_nodes(bbox=input_bbox) + self.assertEqual(output, expected_output) + + def test_polygon_centroid(self): + input_polygon = [ + Point(-3117391.51, 1246003.91), + Point(-3117391.51, 1277745.33), + Point(-3053969.74, 1277745.33), + Point(-3053969.74, 1246003.91), + Point(-3117391.51, 1246003.91), + ] + + expected_output = Point(-3085680.625, 1261874.62) + output = libstac.polygon_centroid(input_polygon) + self.assertEqual(output, expected_output) + + def test_create_metadata_vector(self): + mock_metadata = [ + { + "id": "test", + "title": "Test", + "description": "Test description", + "type": "collection", + "extent": { + "spatial": { + "bbox": [[-122.5, 37.5, -122, 38]], + }, + "temporal": { + "interval": [["2021-01-01T00:00:00Z", "2021-01-31T23:59:59Z"]] + }, + }, + "license": "proprietary", + "stac_version": "1.0.0", + "keywords": ["test", "testing"], + }, + { + "id": "test2", + "title": "Test 2", + "description": "Test description 2", + "type": "collection", + "extent": { + "spatial": { + "bbox": [[-122.5, 37.5, -122, 38]], + }, + "temporal": { + "interval": [["2021-01-01T00:00:00Z", "2021-01-31T23:59:59Z"]] + }, + }, + "license": "proprietary", + "stac_version": "1.0.0", + "keywords": ["test", "testing"], + }, + ] + + libstac.create_metadata_vector(vector="test", metadata=mock_metadata) + pass + + +if __name__ == "__main__": + test() diff --git a/src/temporal/t.stac/requirements.txt b/src/temporal/t.stac/requirements.txt new file mode 100644 index 0000000000..36c2bb3e7f --- /dev/null +++ b/src/temporal/t.stac/requirements.txt @@ -0,0 +1,3 @@ +pystac==1.10 +pystac_client==0.8 +tqdm==4.66 diff --git a/src/temporal/t.stac/t.stac.catalog/Makefile b/src/temporal/t.stac/t.stac.catalog/Makefile new file mode 100644 index 0000000000..51cf85dd36 --- /dev/null +++ b/src/temporal/t.stac/t.stac.catalog/Makefile @@ -0,0 +1,7 @@ +MODULE_TOPDIR = ../../.. + +PGM = t.stac.catalog + +include $(MODULE_TOPDIR)/include/Make/Script.make + +default: script diff --git a/src/temporal/t.stac/t.stac.catalog/t.stac.catalog.html b/src/temporal/t.stac/t.stac.catalog/t.stac.catalog.html new file mode 100644 index 0000000000..f58d0e9c60 --- /dev/null +++ b/src/temporal/t.stac/t.stac.catalog/t.stac.catalog.html @@ -0,0 +1,184 @@ +

DESCRIPTION

+ +t.stac.catalog is a tool for exploring SpatioTemporal Asset Catalogs metadata +from a STAC API. A list of STAC API and Catalogs can be found at +https://stacindex.org/catalogs. +The tool is based on the PySTAC library and provides a set of modules for working with +STAC APIs. + +

REQUIREMENTS

+ + + +

EXAMPLES

+ +Get the catalog metadata from a STAC API. + +

STAC Catalog JSON metadata

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/"
+
+ +GRASS Jupyter Notebooks can be used to visualize the catalog metadata. + +

+    from grass import gs
+    catalog = gs.parse_command('t.stac.catalog', url="https://earth-search.aws.element84.com/v1/")
+
+    print(catalog)
+
+    # Output
+    {'conformsTo': ['https://api.stacspec.org/v1.0.0/core',
+                'https://api.stacspec.org/v1.0.0/collections',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features',
+                'https://api.stacspec.org/v1.0.0/item-search',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#fields',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#sort',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#query',
+                'https://api.stacspec.org/v1.0.0/item-search#fields',
+                'https://api.stacspec.org/v1.0.0/item-search#sort',
+                'https://api.stacspec.org/v1.0.0/item-search#query',
+                'https://api.stacspec.org/v0.3.0/aggregation',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson'],
+ 'description': 'A STAC API of public datasets on AWS',
+ 'id': 'earth-search-aws',
+ 'stac_version': '1.0.0',
+ 'title': 'Earth Search by Element 84',
+ 'type': 'Catalog'}
+
+ +

STAC Catalog plain text metadata

+

+t.stac.catalog url=https://earth-search.aws.element84.com/v1/ format=plain
+
+# Output
+    Client Id: earth-search-aws
+    Client Title: Earth Search by Element 84
+    Client Description: A STAC API of public datasets on AWS
+    Client STAC Extensions: []
+    Client Extra Fields: {'type': 'Catalog', 'conformsTo': ['https://api.stacspec.org/v1.0.0/core', 'https://api.stacspec.org/v1.0.0/collections', 'https://api.stacspec.org/v1.0.0/ogcapi-features', 'https://api.stacspec.org/v1.0.0/item-search', 'https://api.stacspec.org/v1.0.0/ogcapi-features#fields', 'https://api.stacspec.org/v1.0.0/ogcapi-features#sort', 'https://api.stacspec.org/v1.0.0/ogcapi-features#query', 'https://api.stacspec.org/v1.0.0/item-search#fields', 'https://api.stacspec.org/v1.0.0/item-search#sort', 'https://api.stacspec.org/v1.0.0/item-search#query', 'https://api.stacspec.org/v0.3.0/aggregation', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson']}
+    Client catalog_type: ABSOLUTE_PUBLISHED
+    ---------------------------------------------------------------------------
+    Collections: 9
+    sentinel-2-pre-c1-l2a: Sentinel-2 Pre-Collection 1 Level-2A
+    Sentinel-2 Pre-Collection 1 Level-2A (baseline < 05.00), with data and metadata matching collection sentinel-2-c1-l2a
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    cop-dem-glo-30: Copernicus DEM GLO-30
+    The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, infrastructure and vegetation. GLO-30 Public provides limited worldwide coverage at 30 meters because a small subset of tiles covering specific countries are not yet released to the public by the Copernicus Programme.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2021-04-22T00:00:00Z', '2021-04-22T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    naip: NAIP: National Agriculture Imagery Program
+    The [National Agriculture Imagery Program](https://www.fsa.usda.gov/programs-and-services/aerial-photography/imagery-programs/naip-imagery/) (NAIP) provides U.S.-wide, high-resolution aerial imagery, with four spectral bands (R, G, B, IR).  NAIP is administered by the [Aerial Field Photography Office](https://www.fsa.usda.gov/programs-and-services/aerial-photography/) (AFPO) within the [US Department of Agriculture](https://www.usda.gov/) (USDA).  Data are captured at least once every three years for each state.  This dataset represents NAIP data from 2010-present, in [cloud-optimized GeoTIFF](https://www.cogeo.org/) format.
+    Extent: {'spatial': {'bbox': [[-160, 17, -67, 50]]}, 'temporal': {'interval': [['2010-01-01T00:00:00Z', '2022-12-31T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    cop-dem-glo-90: Copernicus DEM GLO-90
+    The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, infrastructure and vegetation. GLO-90 provides worldwide coverage at 90 meters.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2021-04-22T00:00:00Z', '2021-04-22T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    landsat-c2-l2: Landsat Collection 2 Level-2
+    Atmospherically corrected global Landsat Collection 2 Level-2 data from the Thematic Mapper (TM) onboard Landsat 4 and 5, the Enhanced Thematic Mapper Plus (ETM+) onboard Landsat 7, and the Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) onboard Landsat 8 and 9.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['1982-08-22T00:00:00Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-l2a: Sentinel-2 Level-2A
+    Global Sentinel-2 data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-l1c: Sentinel-2 Level-1C
+    Global Sentinel-2 data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-c1-l2a: Sentinel-2 Collection 1 Level-2A
+    Sentinel-2 Collection 1 Level-2A, data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-1-grd: Sentinel-1 Level-1C Ground Range Detected (GRD)
+    Sentinel-1 is a pair of Synthetic Aperture Radar (SAR) imaging satellites launched in 2014 and 2016 by the European Space Agency (ESA). Their 6 day revisit cycle and ability to observe through clouds makes this dataset perfect for sea and land monitoring, emergency response due to environmental disasters, and economic applications. This dataset represents the global Sentinel-1 GRD archive, from beginning to the present, converted to cloud-optimized GeoTIFF format.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2014-10-10T00:28:21Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+
+ +

Basic STAC catalog metadata

+

+    t.stac.catalog url=https://earth-search.aws.element84.com/v1/ format=plain -b
+Client Id: earth-search-aws
+Client Title: Earth Search by Element 84
+Client Description: A STAC API of public datasets on AWS
+Client STAC Extensions: []
+Client Extra Fields: {'type': 'Catalog', 'conformsTo': ['https://api.stacspec.org/v1.0.0/core', 'https://api.stacspec.org/v1.0.0/collections', 'https://api.stacspec.org/v1.0.0/ogcapi-features', 'https://api.stacspec.org/v1.0.0/item-search', 'https://api.stacspec.org/v1.0.0/ogcapi-features#fields', 'https://api.stacspec.org/v1.0.0/ogcapi-features#sort', 'https://api.stacspec.org/v1.0.0/ogcapi-features#query', 'https://api.stacspec.org/v1.0.0/item-search#fields', 'https://api.stacspec.org/v1.0.0/item-search#sort', 'https://api.stacspec.org/v1.0.0/item-search#query', 'https://api.stacspec.org/v0.3.0/aggregation', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson']}
+Client catalog_type: ABSOLUTE_PUBLISHED
+---------------------------------------------------------------------------
+Collections: 9
+---------------------------------------------------------------------------
+sentinel-2-pre-c1-l2a: Sentinel-2 Pre-Collection 1 Level-2A
+cop-dem-glo-30: Copernicus DEM GLO-30
+naip: NAIP: National Agriculture Imagery Program
+cop-dem-glo-90: Copernicus DEM GLO-90
+landsat-c2-l2: Landsat Collection 2 Level-2
+sentinel-2-l2a: Sentinel-2 Level-2A
+sentinel-2-l1c: Sentinel-2 Level-1C
+sentinel-2-c1-l2a: Sentinel-2 Collection 1 Level-2A
+sentinel-1-grd: Sentinel-1 Level-1C Ground Range Detected (GRD)
+
+
+ +

AUTHENTICATION

+ +The t.stac.catalog tool supports authentication with the STAC API using the GDAL's virtual fie system /vsi/. + + + + +

Basic Authentication

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/" settings="user:password"
+
+ +

AWS

+AWS S3 + +

Google Cloud Storage

+Google Cloud Storage + +

Microsoft Azure

+Microsoft Azure + +

HTTP

+HTTP + +

SEE ALSO

+ +Requirements +t.stac.collection, +t.stac.item + +

+GRASS GIS Wiki: temporal data processing + +

AUTHORS

+ +Corey T. White
+ +

Sponsors

+ + +Center for Geospatial Analytics at North Carolina State University diff --git a/src/temporal/t.stac/t.stac.catalog/t.stac.catalog.py b/src/temporal/t.stac/t.stac.catalog/t.stac.catalog.py new file mode 100644 index 0000000000..0c388d6d66 --- /dev/null +++ b/src/temporal/t.stac/t.stac.catalog/t.stac.catalog.py @@ -0,0 +1,145 @@ +#!/usr/bin/env python3 + +############################################################################ +# +# MODULE: t.stac.catalog +# AUTHOR: Corey T. White, OpenPlains Inc. +# PURPOSE: View SpatioTemporal Asset Catalogs (STAC) collection. +# COPYRIGHT: (C) 2024 Corey White +# This program is free software under the GNU General +# Public License (>=v2). Read the file COPYING that +# comes with GRASS for details. +# +############################################################################# + +# %module +# % description: Get STAC API Catalog metadata +# % keyword: temporal +# % keyword: STAC +# % keyword: catalog +# % keyword: metadata +# %end + +# %option +# % key: url +# % description: STAC API Client URL (examples at https://stacspec.org/en/about/datasets/) +# % type: string +# % required: yes +# % multiple: no +# %end + +# %option +# % key: request_method +# % type: string +# % required: no +# % multiple: no +# % options: GET,POST +# % answer: POST +# % description: The HTTP method to use when making a request to the service. +# % guisection: Request +# %end + +# %option G_OPT_F_INPUT +# % key: settings +# % label: Full path to settings file (user, password) +# % description: '-' for standard input +# % guisection: Request +# % required: no +# %end + +# %option +# % key: format +# % type: string +# % required: no +# % multiple: no +# % options: json,plain +# % description: Output format +# % guisection: Output +# % answer: json +# %end + +# %flag +# % key: b +# % description: Return basic information only +# %end + +import sys +from pprint import pprint +import grass.script as gs +from grass.pygrass.utils import get_lib_path + +# Import STAC Client +from pystac_client import Client +from pystac_client.exceptions import APIError +import json + + +path = get_lib_path(modname="t.stac", libname="staclib") +if path is None: + gs.fatal("Not able to find the stac library directory.") +sys.path.append(path) + + +def main(): + """Main function""" + import staclib as libstac + + # STAC Client options + client_url = options["url"] # required + format = options["format"] # optional + + # Flag options + basic_info = flags["b"] # optional + + # Set the request headers + settings = options["settings"] + req_headers = libstac.set_request_headers(settings) + + try: + client = Client.open(client_url, headers=req_headers) + + # Check if the client conforms to the STAC Item Search + # This will exit the program if the client does not conform + libstac.conform_to_item_search(client) + + if format == "plain": + gs.message(_(f"Client Id: {client.id}")) + gs.message(_(f"Client Title: {client.title}")) + gs.message(_(f"Client Description: {client.description}")) + gs.message(_(f"Client STAC Extensions: {client.stac_extensions}")) + gs.message(_(f"Client Extra Fields: {client.extra_fields}")) + gs.message(_(f"Client catalog_type: {client.catalog_type}")) + gs.message(_(f"{'-' * 75}\n")) + + # Get all collections + collection_list = libstac.get_all_collections(client) + gs.message(_(f"Collections: {len(collection_list)}\n")) + gs.message(_(f"{'-' * 75}\n")) + + if basic_info: + for i in collection_list: + gs.message(_(f"{i.get('id')}: {i.get('title')}")) + + if not basic_info: + for i in collection_list: + gs.message(_(f"{i.get('id')}: {i.get('title')}")) + gs.message(_(f"{i.get('description')}")) + gs.message(_(f"Extent: {i.get('extent')}")) + gs.message(_(f"License: {i.get('license')}")) + gs.message(_(f"{'-' * 75}\n")) + libstac.print_list_attribute( + client.get_conforms_to(), "Conforms To:" + ) + gs.message(_(f"{'-' * 75}\n")) + return None + else: + json_output = json.dumps(client.to_dict()) + return json_output + + except APIError as e: + gs.fatal(_("APIError Error opening STAC API: {}".format(e))) + + +if __name__ == "__main__": + options, flags = gs.parser() + sys.exit(main()) diff --git a/src/temporal/t.stac/t.stac.catalog/testsuite/data/catalog.json b/src/temporal/t.stac/t.stac.catalog/testsuite/data/catalog.json new file mode 100644 index 0000000000..422c95f7ec --- /dev/null +++ b/src/temporal/t.stac/t.stac.catalog/testsuite/data/catalog.json @@ -0,0 +1,134 @@ +{ + "conformsTo": [ + "https://api.stacspec.org/v1.0.0/core", + "https://api.stacspec.org/v1.0.0/collections", + "https://api.stacspec.org/v1.0.0/ogcapi-features", + "https://api.stacspec.org/v1.0.0/item-search", + "https://api.stacspec.org/v1.0.0/ogcapi-features#fields", + "https://api.stacspec.org/v1.0.0/ogcapi-features#sort", + "https://api.stacspec.org/v1.0.0/ogcapi-features#query", + "https://api.stacspec.org/v1.0.0/item-search#fields", + "https://api.stacspec.org/v1.0.0/item-search#sort", + "https://api.stacspec.org/v1.0.0/item-search#query", + "https://api.stacspec.org/v0.3.0/aggregation", + "http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core", + "http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30", + "http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson" + ], + "description": "A STAC API of public datasets on AWS", + "id": "earth-search-aws", + "links": [ + { + "href": "https://earth-search.aws.element84.com/v1/", + "rel": "self", + "type": "application/json" + }, + { + "href": "https://earth-search.aws.element84.com/v1", + "rel": "root", + "title": "Earth Search by Element 84", + "type": "application/json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/conformance", + "rel": "conformance", + "type": "application/json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections", + "rel": "data", + "type": "application/json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/search", + "method": "GET", + "rel": "search", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/search", + "method": "POST", + "rel": "search", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/aggregate", + "method": "GET", + "rel": "aggregate", + "type": "application/json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/aggregations", + "rel": "aggregations", + "type": "application/json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/api", + "rel": "service-desc", + "type": "application/vnd.oai.openapi" + }, + { + "href": "https://earth-search.aws.element84.com/v1/api.html", + "rel": "service-doc", + "type": "text/html" + }, + { + "href": "https://earth-search.aws.element84.com/v1/queryables", + "rel": "http://www.opengis.net/def/rel/ogc/1.0/queryables", + "type": "application/schema+json" + }, + { + "href": "https://stac-utils.github.io/stac-server/", + "rel": "server", + "type": "text/html" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/sentinel-2-pre-c1-l2a", + "rel": "child", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/cop-dem-glo-30", + "rel": "child", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/naip", + "rel": "child", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/cop-dem-glo-90", + "rel": "child", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/landsat-c2-l2", + "rel": "child", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/sentinel-2-l2a", + "rel": "child", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/sentinel-2-l1c", + "rel": "child", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/sentinel-2-c1-l2a", + "rel": "child", + "type": "application/geo+json" + }, + { + "href": "https://earth-search.aws.element84.com/v1/collections/sentinel-1-grd", + "rel": "child", + "type": "application/geo+json" + } + ], + "stac_version": "1.0.0", + "title": "Earth Search by Element 84", + "type": "Catalog" +} diff --git a/src/temporal/t.stac/t.stac.catalog/testsuite/test_t_stac_catalog.py b/src/temporal/t.stac/t.stac.catalog/testsuite/test_t_stac_catalog.py new file mode 100644 index 0000000000..151cc9054d --- /dev/null +++ b/src/temporal/t.stac/t.stac.catalog/testsuite/test_t_stac_catalog.py @@ -0,0 +1,76 @@ +# #!/usr/bin/env python3 + +# ############################################################################ +# # +# # NAME: t_stac_catalog +# # +# # AUTHOR: Corey T. White +# # +# # PURPOSE: This is a test file for t.stac.catalog +# # +# # COPYRIGHT: (C) 2024 by Corey T. White and the GRASS Development Team +# # +# # This program is free software under the GNU General Public +# # License (>=v2). Read the file COPYING that comes with GRASS +# # for details. +# # +# ############################################################################# + +# # Dependencies +# # import importlib.util + +# from grass.gunittest.case import TestCase +# from grass.gunittest.main import test +# from grass.gunittest.gmodules import SimpleModule +# from pystac_client import Client +# import json +# from unittest.mock import patch + +# # spec = importlib.util.spec_from_file_location( +# # name="stac_lib", location="t.stac.collection.py" +# # ) + +# # stac_lib = importlib.util.module_from_spec(spec) +# # spec.loader.exec_module(stac_lib) + + +# # Tests +# class TestStacCatalog(TestCase): +# @classmethod +# def setUpClass(cls): +# """Ensures expected computational region""" +# cls.url = "https://earth-search.aws.element84.com/v1/" +# # to not override mapset's region (which might be used by other tests) +# cls.use_temp_region() +# # cls.runModule or self.runModule is used for general module calls +# cls.runModule("g.region", raster="elevation") +# with open("data/catalog.json") as f: +# cls.json_format_expected = json.load(f) + +# @classmethod +# def tearDownClass(cls): +# """Remove temporary region""" +# cls.del_temp_region() + +# # @patch("grass.gunittest.case.TestCase.assertModule") +# @patch("pystac_client.Client.open") +# @patch("pystac_client.Client.get_collections") +# def test_plain_output_json(self, MockClientOpen, MockClientGetCollections): +# """Test t.stac.catalog formated as json""" +# mock_instance = MockClientOpen.return_value +# mock_instance.client = Client.from_dict(self.json_format_expected) +# mock_client_collection = MockClientGetCollections.return_value +# mock_client_collection.get_collections = mock_instance.get_collections() +# self.assertModule("t.stac.catalog", url=self.url, format="json") +# self.assertEqual(mock_instance.outputs.stdout, self.json_format_expected) + +# @patch("grass.gunittest.case.TestCase.assertModule") +# def test_plain_output_basic_info_flag(self, MockAssertModule): +# """Testing format as plain basic info""" +# mock_instance = MockAssertModule.return_value +# mock_instance.outputs.stdout = json.dumps(self.json_format_expected) +# self.assertModule("t.stac.catalog", url=self.url, format="plain", flags="b") + + +# if __name__ == "__main__": +# test() diff --git a/src/temporal/t.stac/t.stac.collection/Makefile b/src/temporal/t.stac/t.stac.collection/Makefile new file mode 100644 index 0000000000..16dcd7c5e1 --- /dev/null +++ b/src/temporal/t.stac/t.stac.collection/Makefile @@ -0,0 +1,13 @@ +MODULE_TOPDIR = ../../.. + +PGM = t.stac.collection + +include $(MODULE_TOPDIR)/include/Make/Script.make + +default: script + +# $(ETC)/t.stac/t.stac.collection/%: % | $(ETC)/t.stac/t.stac.collection +# $(INSTALL) $< $@ +# +# $(ETC)/t.stac/t.stac.collection: +# $(MKDIR) $@ diff --git a/src/temporal/t.stac/t.stac.collection/t.stac.collection.html b/src/temporal/t.stac/t.stac.collection/t.stac.collection.html new file mode 100644 index 0000000000..41869e242d --- /dev/null +++ b/src/temporal/t.stac/t.stac.collection/t.stac.collection.html @@ -0,0 +1,192 @@ +

DESCRIPTION

+ +t.stac.collection is a tool for exploring SpatioTemporal Asset Catalog (STAC) collection metadata. +The tool is based on the PySTAC_Client (0.8) library and allows you to search items in a STAC Catalog. + +The search can be done by specifying the item ID, collection ID, datatime or by using a search query. +The full list of search parameters and documentation can be found at PySTAC_Client ItemSearch. + +

NOTES

+The t.stac.item tool is part of the t.stac temporal data processing framework. +The tool requries that the data provider has implement the STAC API and conforms to Item Search specification. + +

REQUIREMENTS

+ + + +

EXAMPLES

+ + +

Get the item metadata from a STAC API.

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/"
+    t.stac.collection url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a"
+    t.stac.item -i url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a" item_id="S2B_36QWD_20220301_0_L2A"
+
+ +

Get the asset metadata from a STAC API.

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/"
+    t.stac.collection url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a"
+    t.stac.item -a url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a" item_id="S2B_36QWD_20220301_0_L2A"
+
+ +

Dpwnload the asset from a STAC API.

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/"
+    t.stac.collection url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a"
+    t.stac.item -d url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a" item_id="S2B_36QWD_20220301_0_L2A"
+
+ +GRASS Jupyter Notebooks can be used to visualize the catalog metadata. + +

+    from grass import gs
+    catalog = gs.parse_command('t.stac.catalog', url="https://earth-search.aws.element84.com/v1/")
+
+    print(catalog)
+
+    # Output
+    {'conformsTo': ['https://api.stacspec.org/v1.0.0/core',
+                'https://api.stacspec.org/v1.0.0/collections',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features',
+                'https://api.stacspec.org/v1.0.0/item-search',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#fields',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#sort',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#query',
+                'https://api.stacspec.org/v1.0.0/item-search#fields',
+                'https://api.stacspec.org/v1.0.0/item-search#sort',
+                'https://api.stacspec.org/v1.0.0/item-search#query',
+                'https://api.stacspec.org/v0.3.0/aggregation',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson'],
+ 'description': 'A STAC API of public datasets on AWS',
+ 'id': 'earth-search-aws',
+ 'stac_version': '1.0.0',
+ 'title': 'Earth Search by Element 84',
+ 'type': 'Catalog'}
+
+ +

STAC Catalog plain text metadata

+

+t.stac.catalog url=https://earth-search.aws.element84.com/v1/ format=plain
+
+# Output
+    Client Id: earth-search-aws
+    Client Title: Earth Search by Element 84
+    Client Description: A STAC API of public datasets on AWS
+    Client STAC Extensions: []
+    Client Extra Fields: {'type': 'Catalog', 'conformsTo': ['https://api.stacspec.org/v1.0.0/core', 'https://api.stacspec.org/v1.0.0/collections', 'https://api.stacspec.org/v1.0.0/ogcapi-features', 'https://api.stacspec.org/v1.0.0/item-search', 'https://api.stacspec.org/v1.0.0/ogcapi-features#fields', 'https://api.stacspec.org/v1.0.0/ogcapi-features#sort', 'https://api.stacspec.org/v1.0.0/ogcapi-features#query', 'https://api.stacspec.org/v1.0.0/item-search#fields', 'https://api.stacspec.org/v1.0.0/item-search#sort', 'https://api.stacspec.org/v1.0.0/item-search#query', 'https://api.stacspec.org/v0.3.0/aggregation', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson']}
+    Client catalog_type: ABSOLUTE_PUBLISHED
+    ---------------------------------------------------------------------------
+    Collections: 9
+    sentinel-2-pre-c1-l2a: Sentinel-2 Pre-Collection 1 Level-2A
+    Sentinel-2 Pre-Collection 1 Level-2A (baseline < 05.00), with data and metadata matching collection sentinel-2-c1-l2a
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    cop-dem-glo-30: Copernicus DEM GLO-30
+    The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, infrastructure and vegetation. GLO-30 Public provides limited worldwide coverage at 30 meters because a small subset of tiles covering specific countries are not yet released to the public by the Copernicus Programme.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2021-04-22T00:00:00Z', '2021-04-22T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    naip: NAIP: National Agriculture Imagery Program
+    The [National Agriculture Imagery Program](https://www.fsa.usda.gov/programs-and-services/aerial-photography/imagery-programs/naip-imagery/) (NAIP) provides U.S.-wide, high-resolution aerial imagery, with four spectral bands (R, G, B, IR).  NAIP is administered by the [Aerial Field Photography Office](https://www.fsa.usda.gov/programs-and-services/aerial-photography/) (AFPO) within the [US Department of Agriculture](https://www.usda.gov/) (USDA).  Data are captured at least once every three years for each state.  This dataset represents NAIP data from 2010-present, in [cloud-optimized GeoTIFF](https://www.cogeo.org/) format.
+    Extent: {'spatial': {'bbox': [[-160, 17, -67, 50]]}, 'temporal': {'interval': [['2010-01-01T00:00:00Z', '2022-12-31T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    cop-dem-glo-90: Copernicus DEM GLO-90
+    The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, infrastructure and vegetation. GLO-90 provides worldwide coverage at 90 meters.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2021-04-22T00:00:00Z', '2021-04-22T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    landsat-c2-l2: Landsat Collection 2 Level-2
+    Atmospherically corrected global Landsat Collection 2 Level-2 data from the Thematic Mapper (TM) onboard Landsat 4 and 5, the Enhanced Thematic Mapper Plus (ETM+) onboard Landsat 7, and the Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) onboard Landsat 8 and 9.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['1982-08-22T00:00:00Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-l2a: Sentinel-2 Level-2A
+    Global Sentinel-2 data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-l1c: Sentinel-2 Level-1C
+    Global Sentinel-2 data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-c1-l2a: Sentinel-2 Collection 1 Level-2A
+    Sentinel-2 Collection 1 Level-2A, data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-1-grd: Sentinel-1 Level-1C Ground Range Detected (GRD)
+    Sentinel-1 is a pair of Synthetic Aperture Radar (SAR) imaging satellites launched in 2014 and 2016 by the European Space Agency (ESA). Their 6 day revisit cycle and ability to observe through clouds makes this dataset perfect for sea and land monitoring, emergency response due to environmental disasters, and economic applications. This dataset represents the global Sentinel-1 GRD archive, from beginning to the present, converted to cloud-optimized GeoTIFF format.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2014-10-10T00:28:21Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+
+ +

Basic STAC catalog metadata

+

+    t.stac.catalog url=https://earth-search.aws.element84.com/v1/ format=plain -b
+
+    # Output
+    Client Id: earth-search-aws
+    Client Title: Earth Search by Element 84
+    Client Description: A STAC API of public datasets on AWS
+    Client STAC Extensions: []
+    Client Extra Fields: {'type': 'Catalog', 'conformsTo': ['https://api.stacspec.org/v1.0.0/core', 'https://api.stacspec.org/v1.0.0/collections', 'https://api.stacspec.org/v1.0.0/ogcapi-features', 'https://api.stacspec.org/v1.0.0/item-search', 'https://api.stacspec.org/v1.0.0/ogcapi-features#fields', 'https://api.stacspec.org/v1.0.0/ogcapi-features#sort', 'https://api.stacspec.org/v1.0.0/ogcapi-features#query', 'https://api.stacspec.org/v1.0.0/item-search#fields', 'https://api.stacspec.org/v1.0.0/item-search#sort', 'https://api.stacspec.org/v1.0.0/item-search#query', 'https://api.stacspec.org/v0.3.0/aggregation', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson']}
+    Client catalog_type: ABSOLUTE_PUBLISHED
+    ---------------------------------------------------------------------------
+
+
+ +

AUTHENTICATION

+ +The t.stac.catalog tool supports authentication with the STAC API using the GDAL's virtual fie system /vsi/. + + + + +

Basic Authentication

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/" settings="user:password"
+
+ +

AWS

+AWS S3 + +

Google Cloud Storage

+Google Cloud Storage + +

Microsoft Azure

+Microsoft Azure + +

HTTP

+HTTP + +

SEE ALSO

+ +Requirements +t.stac.item +t.stac.item, + + +

AUTHORS

+ +Corey T. White
+ +

Sponsors

+ + +Center for Geospatial Analytics at North Carolina State University diff --git a/src/temporal/t.stac/t.stac.collection/t.stac.collection.py b/src/temporal/t.stac/t.stac.collection/t.stac.collection.py new file mode 100644 index 0000000000..70a38d8268 --- /dev/null +++ b/src/temporal/t.stac/t.stac.collection/t.stac.collection.py @@ -0,0 +1,154 @@ +#!/usr/bin/env python3 + +############################################################################ +# +# MODULE: t.stac.collection +# AUTHOR: Corey T. White, OpenPlains Inc. +# PURPOSE: View SpatioTemporal Asset Catalogs (STAC) collection. +# COPYRIGHT: (C) 2023-2024 Corey White +# This program is free software under the GNU General +# Public License (>=v2). Read the file COPYING that +# comes with GRASS for details. +# +############################################################################# + +# %module +# % description: Get STAC API collection metadata +# % keyword: temporal +# % keyword: STAC +# % keyword: collection +# % keyword: metadata +# %end + +# %option +# % key: url +# % description: STAC API Client URL (examples at https://stacspec.org/en/about/datasets/) +# % type: string +# % required: yes +# % multiple: no +# %end + +# %option +# % key: collection_id +# % description: Collection ID +# % type: string +# % required: yes +# % multiple: no +# %end + +# %option +# % key: request_method +# % type: string +# % required: no +# % multiple: no +# % options: GET,POST +# % answer: POST +# % description: The HTTP method to use when making a request to the service. +# % guisection: Request +# %end + +# %option G_OPT_F_INPUT +# % key: settings +# % label: Full path to settings file (user, password) +# % description: '-' for standard input +# % guisection: Request +# % required: no +# %end + +# %option +# % key: format +# % type: string +# % required: no +# % multiple: no +# % options: json,plain +# % description: Output format +# % guisection: Output +# % answer: json +# %end + +# %flag +# % key: b +# % description: Return basic information only +# %end + +import sys +from pprint import pprint +import grass.script as gs +from grass.pygrass.utils import get_lib_path + + +from pystac_client import Client +from pystac_client.exceptions import APIError +from pystac_client.conformance import ConformanceClasses + +path = get_lib_path(modname="t.stac", libname="staclib") +if path is None: + gs.fatal("Not able to find the stac library directory.") +sys.path.append(path) + + +def get_all_collections(client): + """Get a list of collections from STAC Client""" + try: + collections = client.get_collections() + collection_list = list(collections) + return [i.to_dict() for i in collection_list] + + except APIError as e: + gs.fatal(_("Error getting collections: {}".format(e))) + + +def main(): + """Main function""" + import staclib as libstac + + # STAC Client options + client_url = options["url"] # required + collection_id = options["collection_id"] # optional + # vector_metadata = options["vector_metadata"] # optional + + # Output format + format = options["format"] # optional + + # Flag options + basic_info = flags["b"] # optional + + # Set the request headers + settings = options["settings"] + req_headers = libstac.set_request_headers(settings) + + try: + client = Client.open(client_url, headers=req_headers) + except APIError as e: + gs.fatal(_("APIError Error opening STAC API: {}".format(e))) + + if libstac.conform_to_collections(client): + gs.verbose(_("Conforms to STAC Collections")) + + if collection_id: + try: + collection = client.get_collection(collection_id) + collection_dict = collection.to_dict() + if format == "json": + gs.message(_(f"collection: {collection}")) + return collection_dict + # return pprint(collection.to_dict()) + elif format == "plain": + if basic_info: + return libstac.print_basic_collection_info(collection_dict) + return libstac.print_summary(collection_dict) + + except APIError as e: + gs.fatal(_("APIError Error getting collection: {}".format(e))) + + # Create metadata vector + # if vector_metadata: + # gs.message(_(f"Outputting metadata to {vector_metadata}")) + # libstac.create_metadata_vector(vector_metadata, collection_list) + # gs.message(_(f"Metadata written to {vector_metadata}")) + # return vector_metadata + + +if __name__ == "__main__": + options, flags = gs.parser() + sys.exit(main()) diff --git a/src/temporal/t.stac/t.stac.collection/testsuite/test_t_stac_collection.py b/src/temporal/t.stac/t.stac.collection/testsuite/test_t_stac_collection.py new file mode 100644 index 0000000000..f342f68010 --- /dev/null +++ b/src/temporal/t.stac/t.stac.collection/testsuite/test_t_stac_collection.py @@ -0,0 +1,86 @@ +# #!/usr/bin/env python3 + +# ############################################################################ +# # +# # NAME: t_stac_collection +# # +# # AUTHOR: Corey T. White +# # +# # PURPOSE: This is a test file for t.stac.collection +# # +# # COPYRIGHT: (C) 2023-2024 by Corey T. White and the GRASS Development Team +# # +# # This program is free software under the GNU General Public +# # License (>=v2). Read the file COPYING that comes with GRASS +# # for details. +# # +# ############################################################################# + +# # Dependencies +# # import importlib.util + +# from grass.gunittest.case import TestCase +# from grass.gunittest.main import test + +# # spec = importlib.util.spec_from_file_location( +# # name="stac_lib", location="t.stac.collection.py" +# # ) + +# # stac_lib = importlib.util.module_from_spec(spec) +# # spec.loader.exec_module(stac_lib) + + +# # Tests +# class TestStacCollection(TestCase): +# @classmethod +# def setUpClass(cls): +# """Ensures expected computational region""" +# cls.url = "https://earth-search.aws.element84.com/v1/" +# cls.collection = "naip" +# # to not override mapset's region (which might be used by other tests) +# cls.use_temp_region() +# # cls.runModule or self.runModule is used for general module calls +# cls.runModule("g.region", raster="elevation") + +# @classmethod +# def tearDownClass(cls): +# """Remove temporary region""" +# cls.del_temp_region() + +# def test_search_collections(self): +# """Test t.stac.collection without vector metadata creation""" +# # assertModule is used to call module which we test +# # we expect module to finish successfully +# self.assertModule( +# "t.stac.collection", url=self.url, collection_id=self.collection +# ) + +# def test_collection_id_error(self): +# """Test t.stac.collection with vector metadata creation""" +# # assertModule is used to call module which we test +# # we expect module to finish successfully +# self.assertModuleFail("t.stac.collection", url=self.url) + +# def test_invalid_collection_id(self): +# """Test t.stac.collection with vector metadata creation""" +# # assertModule is used to call module which we test +# # we expect module to finish successfully +# self.assertModuleFail( +# "t.stac.collection", url=self.url, collection_id="naip546" +# ) + +# # def test_vector_metadata_creation(self): +# # """Test t.stac.collection with vector metadata creation""" +# # # assertModule is used to call module which we test +# # # we expect module to finish successfully +# # self.assertModule( +# # "t.stac.collection", +# # url=self.url, +# # vector_metadata="test_vector_metadata", +# # overwrite=True, +# # ) +# # self.assertVectorExists("test_vector_metadata") + + +# if __name__ == "__main__": +# test() diff --git a/src/temporal/t.stac/t.stac.html b/src/temporal/t.stac/t.stac.html new file mode 100644 index 0000000000..1fd15ea85f --- /dev/null +++ b/src/temporal/t.stac/t.stac.html @@ -0,0 +1,48 @@ +

DESCRIPTION

+ +

+The t.stac toolset allows the user to explore metadata and ingest SpatioTemporal Asset Catalog +(STAC) items, collections, and catalogs. The toolset is based on the PySTAC library and provides a set of +modules for working with STAC APIs. + +STAC is a specification for organizing geospatial information in a way +that is interoperable across software and data services. The +pystac-client is used to interact with STAC APIs. + + + +

+t.stac.catalog +t.stac.collection +t.stac.item +(WIP) t.stac.export + +

REQUIREMENTS

+ +

+

+ +

+After dependencies are fulfilled, the toolset can be installed using the +g.extension tool: +

+g.extension extension=t.stac
+
+ +

MODULES

+ + +t.stac.catalog.html +t.stac.collection.html +t.stac.item.html + + +

AUTHOR

+ +Corey White + +This work was funded by OpenPlains Inc. and the Center for Geospatial Analytics at North Carolina State University. diff --git a/src/temporal/t.stac/t.stac.item/Makefile b/src/temporal/t.stac/t.stac.item/Makefile new file mode 100644 index 0000000000..bce1ffaff2 --- /dev/null +++ b/src/temporal/t.stac/t.stac.item/Makefile @@ -0,0 +1,7 @@ +MODULE_TOPDIR = ../../.. + +PGM = t.stac.item + +include $(MODULE_TOPDIR)/include/Make/Script.make + +default: script diff --git a/src/temporal/t.stac/t.stac.item/t.stac.item.html b/src/temporal/t.stac/t.stac.item/t.stac.item.html new file mode 100644 index 0000000000..9051b6a482 --- /dev/null +++ b/src/temporal/t.stac/t.stac.item/t.stac.item.html @@ -0,0 +1,194 @@ +

DESCRIPTION

+ +t.stac.item is a tool for exploring and importing SpatioTemporal Asset Catalog item metadata and assets into GRASS GIS. +The tool is based on the PySTAC_Client (0.8) library and allows you to search items in a STAC Catalog. + +The search can be done by specifying the item ID, collection ID, datatime or by using a search query. +The full list of search parameters and documentation can be found at PySTAC_Client ItemSearch. + +

NOTES

+The t.stac.item tool is part of the t.stac temporal data processing framework. +The tool requries that the data provider has implement the STAC API and conforms to Item Search specification. + +

REQUIREMENTS

+ + + +

EXAMPLES

+ + +

Get the item metadata from a STAC API.

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/"
+    t.stac.collection url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a"
+    t.stac.item -i url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a" item_id="S2B_36QWD_20220301_0_L2A"
+
+ +

Get the asset metadata from a STAC API.

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/"
+    t.stac.collection url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a"
+    t.stac.item -a url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a" item_id="S2B_36QWD_20220301_0_L2A"
+
+ +

Dpwnload the asset from a STAC API.

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/"
+    t.stac.collection url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a"
+    t.stac.item -d url="https://earth-search.aws.element84.com/v1/" collection_id="sentinel-2-l2a" item_id="S2B_36QWD_20220301_0_L2A"
+
+ +GRASS Jupyter Notebooks can be used to visualize the catalog metadata. + +

+    from grass import gs
+    catalog = gs.parse_command('t.stac.catalog', url="https://earth-search.aws.element84.com/v1/")
+
+    print(catalog)
+
+    # Output
+    {'conformsTo': ['https://api.stacspec.org/v1.0.0/core',
+                'https://api.stacspec.org/v1.0.0/collections',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features',
+                'https://api.stacspec.org/v1.0.0/item-search',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#fields',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#sort',
+                'https://api.stacspec.org/v1.0.0/ogcapi-features#query',
+                'https://api.stacspec.org/v1.0.0/item-search#fields',
+                'https://api.stacspec.org/v1.0.0/item-search#sort',
+                'https://api.stacspec.org/v1.0.0/item-search#query',
+                'https://api.stacspec.org/v0.3.0/aggregation',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30',
+                'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson'],
+ 'description': 'A STAC API of public datasets on AWS',
+ 'id': 'earth-search-aws',
+ 'stac_version': '1.0.0',
+ 'title': 'Earth Search by Element 84',
+ 'type': 'Catalog'}
+
+ +

STAC Catalog plain text metadata

+

+t.stac.catalog url=https://earth-search.aws.element84.com/v1/ format=plain
+
+# Output
+    Client Id: earth-search-aws
+    Client Title: Earth Search by Element 84
+    Client Description: A STAC API of public datasets on AWS
+    Client STAC Extensions: []
+    Client Extra Fields: {'type': 'Catalog', 'conformsTo': ['https://api.stacspec.org/v1.0.0/core', 'https://api.stacspec.org/v1.0.0/collections', 'https://api.stacspec.org/v1.0.0/ogcapi-features', 'https://api.stacspec.org/v1.0.0/item-search', 'https://api.stacspec.org/v1.0.0/ogcapi-features#fields', 'https://api.stacspec.org/v1.0.0/ogcapi-features#sort', 'https://api.stacspec.org/v1.0.0/ogcapi-features#query', 'https://api.stacspec.org/v1.0.0/item-search#fields', 'https://api.stacspec.org/v1.0.0/item-search#sort', 'https://api.stacspec.org/v1.0.0/item-search#query', 'https://api.stacspec.org/v0.3.0/aggregation', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson']}
+    Client catalog_type: ABSOLUTE_PUBLISHED
+    ---------------------------------------------------------------------------
+    Collections: 9
+    sentinel-2-pre-c1-l2a: Sentinel-2 Pre-Collection 1 Level-2A
+    Sentinel-2 Pre-Collection 1 Level-2A (baseline < 05.00), with data and metadata matching collection sentinel-2-c1-l2a
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    cop-dem-glo-30: Copernicus DEM GLO-30
+    The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, infrastructure and vegetation. GLO-30 Public provides limited worldwide coverage at 30 meters because a small subset of tiles covering specific countries are not yet released to the public by the Copernicus Programme.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2021-04-22T00:00:00Z', '2021-04-22T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    naip: NAIP: National Agriculture Imagery Program
+    The [National Agriculture Imagery Program](https://www.fsa.usda.gov/programs-and-services/aerial-photography/imagery-programs/naip-imagery/) (NAIP) provides U.S.-wide, high-resolution aerial imagery, with four spectral bands (R, G, B, IR).  NAIP is administered by the [Aerial Field Photography Office](https://www.fsa.usda.gov/programs-and-services/aerial-photography/) (AFPO) within the [US Department of Agriculture](https://www.usda.gov/) (USDA).  Data are captured at least once every three years for each state.  This dataset represents NAIP data from 2010-present, in [cloud-optimized GeoTIFF](https://www.cogeo.org/) format.
+    Extent: {'spatial': {'bbox': [[-160, 17, -67, 50]]}, 'temporal': {'interval': [['2010-01-01T00:00:00Z', '2022-12-31T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    cop-dem-glo-90: Copernicus DEM GLO-90
+    The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, infrastructure and vegetation. GLO-90 provides worldwide coverage at 90 meters.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2021-04-22T00:00:00Z', '2021-04-22T00:00:00Z']]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    landsat-c2-l2: Landsat Collection 2 Level-2
+    Atmospherically corrected global Landsat Collection 2 Level-2 data from the Thematic Mapper (TM) onboard Landsat 4 and 5, the Enhanced Thematic Mapper Plus (ETM+) onboard Landsat 7, and the Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) onboard Landsat 8 and 9.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['1982-08-22T00:00:00Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-l2a: Sentinel-2 Level-2A
+    Global Sentinel-2 data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-l1c: Sentinel-2 Level-1C
+    Global Sentinel-2 data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-2-c1-l2a: Sentinel-2 Collection 1 Level-2A
+    Sentinel-2 Collection 1 Level-2A, data from the Multispectral Instrument (MSI) onboard Sentinel-2
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2015-06-27T10:25:31.456000Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+    sentinel-1-grd: Sentinel-1 Level-1C Ground Range Detected (GRD)
+    Sentinel-1 is a pair of Synthetic Aperture Radar (SAR) imaging satellites launched in 2014 and 2016 by the European Space Agency (ESA). Their 6 day revisit cycle and ability to observe through clouds makes this dataset perfect for sea and land monitoring, emergency response due to environmental disasters, and economic applications. This dataset represents the global Sentinel-1 GRD archive, from beginning to the present, converted to cloud-optimized GeoTIFF format.
+    Extent: {'spatial': {'bbox': [[-180, -90, 180, 90]]}, 'temporal': {'interval': [['2014-10-10T00:28:21Z', None]]}}
+    License: proprietary
+    ---------------------------------------------------------------------------
+
+ +

Basic STAC catalog metadata

+

+    t.stac.catalog url=https://earth-search.aws.element84.com/v1/ format=plain -b
+
+    # Output
+    Client Id: earth-search-aws
+    Client Title: Earth Search by Element 84
+    Client Description: A STAC API of public datasets on AWS
+    Client STAC Extensions: []
+    Client Extra Fields: {'type': 'Catalog', 'conformsTo': ['https://api.stacspec.org/v1.0.0/core', 'https://api.stacspec.org/v1.0.0/collections', 'https://api.stacspec.org/v1.0.0/ogcapi-features', 'https://api.stacspec.org/v1.0.0/item-search', 'https://api.stacspec.org/v1.0.0/ogcapi-features#fields', 'https://api.stacspec.org/v1.0.0/ogcapi-features#sort', 'https://api.stacspec.org/v1.0.0/ogcapi-features#query', 'https://api.stacspec.org/v1.0.0/item-search#fields', 'https://api.stacspec.org/v1.0.0/item-search#sort', 'https://api.stacspec.org/v1.0.0/item-search#query', 'https://api.stacspec.org/v0.3.0/aggregation', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson']}
+    Client catalog_type: ABSOLUTE_PUBLISHED
+    ---------------------------------------------------------------------------
+
+
+ +

AUTHENTICATION

+ +The t.stac.catalog tool supports authentication with the STAC API using the GDAL's virtual fie system /vsi/. + + + + +

Basic Authentication

+

+    t.stac.catalog url="https://earth-search.aws.element84.com/v1/" settings="user:password"
+
+ +

AWS

+AWS S3 + +

Google Cloud Storage

+Google Cloud Storage + +

Microsoft Azure

+Microsoft Azure + +

HTTP

+HTTP + +

SEE ALSO

+ +Requirements +t.stac.collection, +t.stac.item + +

+GRASS GIS Wiki: temporal data processing + +

AUTHORS

+ +Corey T. White
+ +

Sponsors

+ + +Center for Geospatial Analytics at North Carolina State University diff --git a/src/temporal/t.stac/t.stac.item/t.stac.item.py b/src/temporal/t.stac/t.stac.item/t.stac.item.py new file mode 100644 index 0000000000..00fbdbc19a --- /dev/null +++ b/src/temporal/t.stac/t.stac.item/t.stac.item.py @@ -0,0 +1,646 @@ +#!/usr/bin/env python3 + +############################################################################ +# +# MODULE: t.stac.item +# AUTHOR: Corey T. White, OpenPlains Inc. +# PURPOSE: Get items from a STAC API server +# COPYRIGHT: (C) 2023-2024 Corey White +# This program is free software under the GNU General +# Public License (>=v2). Read the file COPYING that +# comes with GRASS for details. +# +############################################################################# + +# %module +# % description: Downloads and imports data from a STAC API server. +# % keyword: raster +# % keyword: import +# % keyword: STAC +# % keyword: temporal +# %end + +# %option +# % key: url +# % type: string +# % description: STAC API Client URL +# % required: yes +# %end + +# %option +# % key: collection_id +# % type: string +# % required: yes +# % multiple: no +# % description: Collection Id. +# %end + +# %option +# % key: request_method +# % type: string +# % required: no +# % multiple: no +# % options: GET,POST +# % answer: POST +# % description: The HTTP method to use when making a request to the service. +# % guisection: Request +# %end + +# %option G_OPT_F_INPUT +# % key: settings +# % label: Full path to settings file (user, password) +# % description: '-' for standard input +# % guisection: Request +# % required: no +# %end + +# %option +# % key: max_items +# % type: integer +# % description: The maximum number of items to return from the search, even if there are more matching results. +# % multiple: no +# % required: no +# % answer: 1000 +# % guisection: Request +# %end + +# %option +# % key: limit +# % type: integer +# % description: A recommendation to the service as to the number of items to return per page of results. Defaults to 100. +# % multiple: no +# % required: no +# % answer: 100 +# % guisection: Request +# %end + +# %option +# % key: ids +# % type: string +# % description: List of one or more Item ids to filter on. +# % multiple: no +# % required: no +# % guisection: Query +# %end + +# %option +# % key: bbox +# % type: double +# % required: no +# % multiple: yes +# % description: The bounding box of the request in WGS84 (example [-72.5,40.5,-72,41]). (default is current region) +# % guisection: Query +# %end + +# %option G_OPT_V_INPUT +# % key: intersects +# % required: no +# % description: Results filtered to only those intersecting the geometry. +# % guisection: Query +# %end + +# %option +# % key: datetime +# % label: Datetime Filter +# % description: Either a single datetime or datetime range used to filter results. +# % required: no +# % guisection: Query +# %end + +# %option +# % key: query +# % description: List or JSON of query parameters as per the STAC API query extension. +# % required: no +# % guisection: Query +# %end + +# %option +# % key: filter +# % description: JSON of query parameters as per the STAC API filter extension +# % required: no +# % guisection: Query +# %end + +# %option +# % key: asset_keys +# % label: Asset Keys +# % type: string +# % required: no +# % multiple: yes +# % description: List of one or more asset keys to filter item downloads. +# % guisection: Query +# %end + +# %option +# % key: item_roles +# % label: Item roles +# % type: string +# % required: no +# % multiple: yes +# % description: List of one or more item roles to filter by. +# % guisection: Query +# %end + +# %option +# % key: filter_lang +# % type: string +# % required: no +# % multiple: no +# % options: cql2-json,cql2-text +# % description: Language variant used in the filter body. If filter is a dictionary or not provided, defaults to cql2-json. If filter is a string, defaults to cql2-text. +# % guisection: Query +# %end + +# %option +# % key: sortby +# % description: A single field or list of fields to sort the response by +# % required: no +# % guisection: Query +# %end + +# %option +# % key: format +# % type: string +# % required: no +# % multiple: no +# % options: json,plain +# % description: Output format +# % guisection: Output +# % answer: json +# %end + +# %option G_OPT_F_OUTPUT +# % key: strds_output +# % label: STRDS Output +# % description: Spatial Temporal Raster Dataset Registration File +# % guisection: Output +# % required: no +# %end + +# %option +# % key: items_vector +# % type: string +# % description: Name of vector containing STAC item boundaries and metadata. +# % required: no +# % multiple: no +# % guisection: Output +# %end + +# %option +# % key: method +# % type: string +# % required: no +# % multiple: no +# % options: nearest,bilinear,bicubic,lanczos,bilinear_f,bicubic_f,lanczos_f +# % description: Resampling method to use for reprojection (required if location projection not longlat) +# % descriptions: nearest;nearest neighbor;bilinear;bilinear interpolation;bicubic;bicubic interpolation;lanczos;lanczos filter;bilinear_f;bilinear interpolation with fallback;bicubic_f;bicubic interpolation with fallback;lanczos_f;lanczos filter with fallback +# % guisection: Output +# % answer: nearest +# %end + +# %option +# % key: resolution +# % type: string +# % required: no +# % multiple: no +# % options: estimated,value,region +# % description: Resolution of output raster map (default: estimated) +# % descriptions: estimated;estimated resolution;value; user-specified resolution;region;current region resolution +# % guisection: Output +# % answer: estimated +# %end + +# %option +# % key: resolution_value +# % type: double +# % required: no +# % multiple: no +# % description: Resolution of output raster map (use with option resolution=value) +# % guisection: Output +# %end + +# %option +# % key: extent +# % type: string +# % required: no +# % multiple: no +# % options: input,region +# % description: Output raster map extent +# % descriptions: input;extent of input map;region; extent of current region +# % guisection: Output +# % answer: input +# %end + +# %flag +# % key: m +# % description: Collection Search Item Summary +# %end + +# %flag +# % key: i +# % description: Item metadata +# %end + +# %flag +# % key: a +# % description: Asset metadata +# %end + +# %flag +# % key: d +# % description: Dowload and import assets +# %end + +# %option G_OPT_M_NPROCS +# %end + +# %option G_OPT_MEMORYMB +# %end + +import os +import sys +from pprint import pprint +import json + +# from multiprocessing.pool import ThreadPool +from pystac_client import Client +from pystac_client.exceptions import APIError +from pystac import MediaType +from concurrent.futures import ThreadPoolExecutor +from tqdm import tqdm +import tempfile + +import grass.script as gs +from grass.pygrass.utils import get_lib_path +from grass.exceptions import CalledModuleError + + +path = get_lib_path(modname="t.stac", libname="staclib") +if path is None: + gs.fatal("Not able to find the stac library directory.") +sys.path.append(path) + +import staclib as libstac + + +def search_stac_api(client, **kwargs): + """Search the STAC API""" + if libstac.conform_to_item_search(client): + gs.verbose(_("STAC API Conforms to Item Search")) + try: + search = client.search(**kwargs) + except APIError as e: + gs.fatal(_("Error searching STAC API: {}".format(e))) + except NotImplementedError as e: + gs.fatal(_("Error searching STAC API: {}".format(e))) + except Exception as e: + gs.fatal(_("Error searching STAC API: {}".format(e))) + + try: + gs.message(_(f"Search Matched: {search.matched()} items")) + # These requests tend to be very slow + # gs.message(_(f"Pages: {len(list(search.pages()))}")) + # gs.message(_(f"Max items per page: {len(list(search.items()))}")) + + except e: + gs.warning(_(f"No items found: {e}")) + return None + + return search + + +def collection_metadata(collection): + """Get collection""" + + gs.message(_("*" * 80)) + gs.message(_(f"Collection Id: {collection.id}")) + + libstac.print_attribute(collection, "title", "Collection Title") + libstac.print_attribute(collection, "description", "Description") + gs.message(_(f"Spatial Extent: {collection.extent.spatial.bboxes}")) + gs.message(_(f"Temporal Extent: {collection.extent.temporal.intervals}")) + + libstac.print_attribute(collection, "license") + libstac.print_attribute(collection, "keywords") + libstac.print_attribute(collection, "links") + libstac.print_attribute(collection, "providers") + libstac.print_attribute(collection, "stac_extensions", "Extensions") + + try: + gs.message(_("\n# Summaries:")) + libstac.print_summary(collection.summaries.to_dict()) + except AttributeError: + gs.info(_("Summaries not found.")) + + try: + gs.message(_("\n# Extra Fields:")) + libstac.print_summary(collection.extra_fields) + except AttributeError: + gs.info(_("# Extra Fields not found.")) + gs.message(_("*" * 80)) + + +def report_stac_item(item): + """Print a report of the STAC item to the console.""" + gs.message(_(f"Collection ID: {item.collection_id}")) + gs.message(_(f"Item: {item.id}")) + libstac.print_attribute(item, "geometry", "Geometry") + gs.message(_(f"Bbox: {item.bbox}")) + + libstac.print_attribute(item, "datetime", "Datetime") + libstac.print_attribute(item, "start_datetime", "Start Datetime") + libstac.print_attribute(item, "end_datetime", "End Datetime") + gs.message(_("Extra Fields:")) + libstac.print_summary(item.extra_fields) + + libstac.print_list_attribute(item.stac_extensions, "Extensions:") + # libstac.print_attribute(item, "stac_extensions", "Extensions") + gs.message(_("Properties:")) + libstac.print_summary(item.properties) + + +def collect_item_assets(item, assset_keys, asset_roles): + for key, asset in item.assets.items(): + asset_file_name = f"{item.collection_id}.{item.id}.{key}" + # Check if the asset key is in the list of asset keys + if assset_keys and key not in assset_keys: + continue + + # Check if the asset fits the roles + if asset_roles: + if not any(role in asset.roles for role in asset_roles): + continue + + asset_dict = asset.to_dict() + # The output file name + asset_dict["collection_id"] = item.collection_id + asset_dict["item_id"] = item.id + asset_dict["file_name"] = asset_file_name + asset_dict["datetime"] = item.properties["datetime"] + + return asset_dict + + +def report_plain_asset_summary(asset): + gs.message(_("\nAsset")) + gs.message(_(f"Asset Item Id: {asset.get('item_id')}")) + + gs.message(_(f"Asset Title: {asset.get('title')}")) + gs.message(_(f"Asset Filename: {asset.get('file_name')}")) + gs.message(_(f"Raster bands: {asset.get('raster:bands')}")) + gs.message(_(f"Raster bands: {asset.get('eo:bands')}")) + gs.message(_(f"Asset Description: {asset.get('description')}")) + gs.message(_(f"Asset Media Type: { MediaType(asset.get('type')).name}")) + gs.message(_(f"Asset Roles: {asset.get('roles')}")) + gs.message(_(f"Asset Href: {asset.get('href')}")) + + +def import_grass_raster(params): + assets, resample_method, extent, resolution, resolution_value, memory = params + gs.message(_(f"Downloading Asset: {assets}")) + input_url = libstac.check_url_type(assets["href"]) + gs.message(_(f"Import Url: {input_url}")) + + try: + gs.message(_(f"Importing: {assets['file_name']}")) + gs.parse_command( + "r.import", + input=input_url, + output=assets["file_name"], + resample=resample_method, + extent=extent, + resolution=resolution, + resolution_value=resolution_value, + title=assets["file_name"], + memory=memory, + quiet=True, + ) + except CalledModuleError as e: + gs.fatal(_("Error importing raster: {}".format(e.stderr))) + + +def download_assets( + assets, + resample_method, + resample_extent, + resolution, + resolution_value, + memory=300, + nprocs=1, +): + """Downloads a list of images from the given URLs to the given filenames.""" + number_of_assets = len(assets) + resample_extent_list = [resample_extent] * number_of_assets + resolution_list = [resolution] * number_of_assets + resolution_value_list = [resolution_value] * number_of_assets + resample_method_list = [resample_method] * number_of_assets + memory_list = [memory] * number_of_assets + max_cpus = os.cpu_count() - 1 + if nprocs > max_cpus: + gs.warning( + _( + "Number of processes {nprocs} is greater than the number of CPUs {max_cpus}." + ) + ) + nprocs = max_cpus + + with tqdm(total=number_of_assets, desc="Downloading assets") as pbar: + with ThreadPoolExecutor(max_workers=nprocs) as executor: + try: + for _a in executor.map( + import_grass_raster, + zip( + assets, + resample_method_list, + resample_extent_list, + resolution_list, + resolution_value_list, + memory_list, + ), + ): + pbar.update(1) + except Exception as e: + gs.fatal(_("Error importing raster: {}".format(str(e)))) + + +def main(): + """Main function""" + + # STAC Client options + client_url = options["url"] # required + collection_id = options["collection_id"] # required + + # Request options + limit = int(options["limit"]) # optional + max_items = int(options["max_items"]) if options["max_items"] else None # optional + request_method = options["request_method"] # optional + + # Query options + bbox = options["bbox"] # optional + datetime = options["datetime"] # optional + query = options["query"] # optional + filter = options["filter"] # optional + filter_lang = options["filter_lang"] # optional + intersects = options["intersects"] # optional + # Asset options + asset_keys_input = options["asset_keys"] # optional + asset_keys = asset_keys_input.split(",") if asset_keys_input else None + + item_roles_input = options["item_roles"] # optional + item_roles = item_roles_input.split(",") if item_roles_input else None + + # Flags + summary_metadata = flags["m"] + item_metadata = flags["i"] + asset_metadata = flags["a"] + download = flags["d"] + + # Output options + strds_output = options["strds_output"] # optional + items_vector = options["items_vector"] # optional + method = options["method"] # optional + resolution = options["resolution"] # optional + resolution_value = options["resolution_value"] # optional + extent = options["extent"] # optional + format = options["format"] # optional + + # GRASS import options + method = options["method"] # optional + memory = int(options["memory"]) # optional + nprocs = int(options["nprocs"]) # optional + + search_params = {} # Store STAC API search parameters + collection_items_assets = [] + + try: + + # Set the request headers + settings = options["settings"] + req_headers = libstac.set_request_headers(settings) + + client = Client.open(client_url, headers=req_headers) + except APIError as e: + gs.fatal(_("APIError Error opening STAC API: {}".format(e))) + + try: + collection = client.get_collection(collection_id) + except APIError as e: + gs.fatal(_(f"Error getting collection {collection_id}: {e}")) + + if summary_metadata: + if format == "plain": + return collection_metadata(collection) + elif format == "json": + return pprint(collection.to_dict()) + else: + # Return plain text by default + return collection_metadata(collection) + + # Start item search + + if intersects: + # Convert the vector to a geojson + output_geojson = "tmp_stac_intersects.geojson" + gs.run_command( + "v.out.ogr", input=intersects, output=output_geojson, format="GeoJSON" + ) + with open(output_geojson, "r") as f: + intersects_geojson = f.read() + search_params["intersects"] = intersects_geojson + f.close() + os.remove(output_geojson) + + if options["ids"]: + ids = options["ids"] # item ids optional + search_params["ids"] = ids.split(",") + + # Set the bbox to the current region if the user did not specify the bbox or intersects option + if not bbox and not intersects: + gs.message(_("Setting bbox to current region: {}".format(bbox))) + bbox = libstac.region_to_wgs84_decimal_degrees_bbox() + + if datetime: + search_params["datetime"] = datetime + + # Add filter to search_params + # https://github.com/stac-api-extensions/filter + if filter: + if isinstance(filter, str): + filter = json.loads(filter) + if isinstance(filter, dict): + search_params["filter"] = filter + + if filter_lang: + search_params["filter_lang"] = filter_lang + + if libstac.conform_to_query(client): + gs.verbose(_("STAC API Conforms to Item Search Query")) + if query: + if isinstance(query, str): + query = json.loads(query) + if isinstance(query, dict): + search_params["query"] = query + if isinstance(query, list): + search_params["query"] = query + + # Add search parameters to search_params + search_params["method"] = request_method + search_params["collections"] = collection_id + search_params["limit"] = limit + search_params["max_items"] = max_items + search_params["bbox"] = bbox + + # Search the STAC API + items_search = search_stac_api(client=client, **search_params) + # Create vector layer of items metadata + if items_vector: + libstac.create_vector_from_feature_collection( + items_vector, items_search, limit, max_items + ) + + # Fetch items from all pages + items = libstac.fetch_items_with_pagination(items_search, limit, max_items) + + # Report item metadata + if item_metadata: + if format == "plain": + gs.message(_(f"Items Found: {len(list(items))}")) + for item in items: + report_stac_item(item) + return None + if format == "json": + return pprint([item.to_dict() for item in items]) + + for item in items: + asset = collect_item_assets(item, asset_keys, asset_roles=item_roles) + if asset: + collection_items_assets.append(asset) + + if strds_output: + strds_output = os.path.abspath(strds_output) + libstac.register_strds_from_items(collection_items_assets, strds_output) + + gs.message(_(f"{len(collection_items_assets)} Assets Ready for download...")) + if asset_metadata: + for asset in collection_items_assets: + if format == "plain": + report_plain_asset_summary(asset) + if format == "json": + pprint(asset) + + if download: + # Download and Import assets + download_assets( + assets=collection_items_assets, + resample_method=method, + resample_extent=extent, + resolution=resolution, + resolution_value=resolution_value, + memory=memory, + nprocs=nprocs, + ) + + +if __name__ == "__main__": + options, flags = gs.parser() + sys.exit(main()) diff --git a/src/temporal/t.stac/t.stac.item/testsuite/test_t_stac_item.py b/src/temporal/t.stac/t.stac.item/testsuite/test_t_stac_item.py new file mode 100644 index 0000000000..a492b76ed1 --- /dev/null +++ b/src/temporal/t.stac/t.stac.item/testsuite/test_t_stac_item.py @@ -0,0 +1,85 @@ +# #!/usr/bin/env python3 + +# ############################################################################ +# # +# # NAME: t_stac_item +# # +# # AUTHOR: Corey T. White +# # +# # PURPOSE: This is a test file for t.stac.item +# # +# # COPYRIGHT: (C) 2023 by Corey T. White and the GRASS Development Team +# # +# # This program is free software under the GNU General Public +# # License (>=v2). Read the file COPYING that comes with GRASS +# # for details. +# # +# ############################################################################# + +# # Dependencies +# # import importlib.util + +# from grass.gunittest.case import TestCase +# from grass.gunittest.main import test + + +# # Tests +# class TestStacItem(TestCase): +# @classmethod +# def setUpClass(cls): +# """Ensures expected computational region""" +# cls.url = "https://earth-search.aws.element84.com/v1/" +# cls.collections = "naip" + +# @classmethod +# def tearDownClass(cls): +# """Remove temporary region""" +# pass + +# def test_search_items(self): +# """Test t.stac.item without vector metadata creation""" +# # Should return count of items found in the collection +# self.assertModule("t.stac.item", url=self.url, collections=self.collections) + +# def test_search_items_summary_json(self): +# """Test t.stac.item with JSON output""" +# # Should return JSON output of items found in the collection +# self.assertModule( +# "t.stac.item", +# url=self.url, +# collections=self.collections, +# format="json", +# flag="m", +# ) + +# def test_search_items_summary_plain(self): +# """Test t.stac.item with plain text output""" +# # Should return plain text output of items found in the collection +# self.assertModule( +# "t.stac.item", +# url=self.url, +# collections=self.collections, +# format="plain", +# flag="m", +# ) + +# def test_search_items_vector_footprint(self): +# """Test t.stac.item with vector metadata creation""" +# # Should return vector metadata of items found in the collection +# self.assertModule( +# "t.stac.item", +# url=self.url, +# collections=self.collections, +# vector="naip_footprints", +# flag="m", +# ) + +# def test_collections_not_found(self): +# """Test t.stac.collection with vector metadata creation""" +# # assertModule is used to call module which we test +# # we expect module to finish successfully +# self.assertModule("t.stac.item", url=self.url, collections="naip546") + + +# if __name__ == "__main__": +# test()