-
Notifications
You must be signed in to change notification settings - Fork 162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
t.rast.seasons: new temporal module to get seasonally aggregated data #920
Closed
Closed
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
5fe5f27
added new temporal module to get seasonal aggregated data
lucadelu c401e83
fixed black issues
lucadelu 9901b7d
Apply suggestions from Vero code review
lucadelu 81fca2e
renamed everything to t.rast.aggregate.seasons
lucadelu 1b5790f
added output
lucadelu 4522cf8
Merge branch 'grass7' into t.rast.seasons
echoix 7776f9f
updated t.rast.aggregate.seasons to GRASS8 and tests
lucadelu 1ef466c
removed pdb
lucadelu db4cb94
fixed all remaing problems, tests are working
lucadelu 7a0a73b
run black
lucadelu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
MODULE_TOPDIR = ../.. | ||
|
||
PGM = t.rast.aggregate.seasons | ||
|
||
include $(MODULE_TOPDIR)/include/Make/Script.make | ||
|
||
default: script |
28 changes: 28 additions & 0 deletions
28
src/temporal/t.rast.aggregate.seasons/t.rast.aggregate.seasons.html
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
<h2>DESCRIPTION</h2> | ||
<em><b>t.rast.seasons</b></em> aggregates an input space time | ||
raster dataset at astronomical seasons level using a statistical method selected by the user. It uses <a href="https://grass.osgeo.org/grass-stable/manuals/t.rast.aggregate.ds.html">t.rast.aggregate.ds</a> | ||
to detect and copy the astronomical seasons granularity. | ||
Astronomical seasons are defined as: | ||
<ul> | ||
<li><em>Spring</em> 20 March - 21 June</li> | ||
<li><em>Summer</em> 21 June - 20 September</li> | ||
<li><em>Autumn</em> 20 September - 21 December</li> | ||
<li><em>Winter</em> 21 December - 20 March (following year)</li> | ||
</ul> | ||
<p> | ||
|
||
<h2>EXAMPLES</h2> | ||
Calculate seasonal data from | ||
<div class="code"><pre> | ||
t.rast.seasons input=mystrds basename=mystrds_seasons | ||
</pre></div> | ||
|
||
<h2>SEE ALSO</h2> | ||
<em> | ||
<a href="https://grass.osgeo.org/grass-stable/manuals/t.rast.aggregate.ds.html">t.rast.aggregate.ds</a> | ||
<a href="https://grass.osgeo.org/grass-stable/manuals/r.series.html">r.null</a> | ||
</em> | ||
|
||
<h2>AUTHOR</h2> | ||
|
||
Luca Delucchi, Fondazione Edmund Mach |
243 changes: 243 additions & 0 deletions
243
src/temporal/t.rast.aggregate.seasons/t.rast.aggregate.seasons.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,243 @@ | ||
#!/usr/bin/env python | ||
|
||
|
||
################################################ | ||
# | ||
# MODULE: t.rast.seasons | ||
# AUTHOR(S): Luca Delucchi | ||
# PURPOSE: Aggregates an input strds with astronomical seasons granularity. | ||
# | ||
# COPYRIGHT: (C) 2018 by Luca Delucchi | ||
# | ||
# This program is free software under the GNU General Public | ||
# License (>=v2). Read the file COPYING that comes with GRASS | ||
# for details. | ||
# | ||
################################################ | ||
|
||
# %module | ||
# % description: Calculates seasonal data according to astronomical seasons. | ||
# % keyword: temporal | ||
# % keyword: raster | ||
# % keyword: aggregation | ||
# % keyword: series | ||
# %end | ||
|
||
# %option G_OPT_STRDS_INPUT | ||
# %end | ||
|
||
# %option G_OPT_STRDS_OUTPUT | ||
# % label: The name of a singular space time raster dataset | ||
# % description: Using this option all the yearly space time raster datasets will be merged in a singular space time raster dataset | ||
# % required: no | ||
# %end | ||
|
||
# %option | ||
# % key: years | ||
# % type: string | ||
# % label: List of years, separator could be comma (list) or minus (range) | ||
# % multiple: yes | ||
# % required: no | ||
# %end | ||
|
||
# %option | ||
# % key: basename | ||
# % type: string | ||
# % label: Basename of the new generated output maps and space time raster datasets | ||
# % description: A numerical suffix separated by an underscore will be attached to create a unique identifier | ||
# % required: yes | ||
# % multiple: no | ||
# % gisprompt: | ||
# %end | ||
|
||
# %option | ||
# % key: method | ||
# % type: string | ||
# % description: Aggregate operation to be performed on the raster maps | ||
# % required: yes | ||
# % multiple: no | ||
# % options: average,count,median,mode,minimum,min_raster,maximum,max_raster,stddev,range,sum,variance,diversity,slope,offset,detcoeff,quart1,quart3,perc90,quantile,skewness,kurtosis | ||
# % answer: average | ||
# %end | ||
|
||
# %option | ||
# % key: nprocs | ||
# % type: integer | ||
# % description: Number of processes to run in parallel | ||
# % required: no | ||
# % multiple: no | ||
# % answer: 1 | ||
# %end | ||
|
||
# %flag | ||
# % key: n | ||
# % description: Register Null maps | ||
# %end | ||
|
||
import copy | ||
import atexit | ||
from datetime import datetime | ||
|
||
import grass.temporal as tgis | ||
import grass.script as gs | ||
import grass.pygrass.modules as pymod | ||
from grass.pygrass.vector import VectorTopo | ||
from grass.pygrass.vector.geometry import Point | ||
|
||
remove_dataset = {"stvds": [], "strds": []} | ||
|
||
|
||
def cleanup(): | ||
""" | ||
Clean up temporary maps | ||
""" | ||
# remove space time vector datasets | ||
for typ, maps in remove_dataset.items(): | ||
for map in maps: | ||
remod = pymod.Module("t.remove", run_=False) | ||
remod.inputs.inputs = map | ||
remod.inputs.type = typ | ||
remod.flags.r = True | ||
remod.flags.f = True | ||
remod.flags.d = True | ||
remod.flags.quiet = True | ||
remod.run() | ||
|
||
|
||
def main(): | ||
options, flags = gs.parser() | ||
strds = options["input"] | ||
|
||
output_name = options["output"] | ||
|
||
tgis.init() | ||
# We need a database interface | ||
dbif = tgis.SQLDatabaseInterfaceConnection() | ||
dbif.connect() | ||
mapset = tgis.core.get_current_mapset() | ||
|
||
if options["years"] != "": | ||
try: | ||
vals = options["years"].split("-") | ||
years = range(vals) | ||
except: | ||
try: | ||
years = options["years"].split(",") | ||
except: | ||
gs.fatal(_("Invalid option years")) | ||
else: | ||
if strds.find("@") >= 0: | ||
id_ = strds | ||
else: | ||
id_ = f'{strds}@{gs.gisenv()["MAPSET"]}' | ||
dataset = tgis.dataset_factory("strds", id_) | ||
dataset.select(dbif) | ||
ext = dataset.get_temporal_extent() | ||
years = range(ext.start_time.year, ext.end_time.year) | ||
|
||
method = options["method"] | ||
basename = options["basename"] | ||
nprocs = int(options["nprocs"]) | ||
register_null = flags["n"] | ||
|
||
seasons = ["spring", "summer", "autumn", "winter"] | ||
|
||
# create new space time vector datasets one for each year to be used as sampler | ||
for year in years: | ||
season_vect = [] | ||
for seas in seasons: | ||
name = f"sample_{seas}_{year}" | ||
vect = VectorTopo(name) | ||
vect.open("w") | ||
point = Point(0, 0) | ||
vect.write(point, cat=1) | ||
vect.close() | ||
map_layer = tgis.space_time_datasets.VectorDataset(f"{name}@{mapset}") | ||
if seas == "spring": | ||
extent = tgis.AbsoluteTemporalExtent( | ||
start_time=datetime(int(year), 3, 20), | ||
end_time=datetime(int(year), 6, 21), | ||
) | ||
elif seas == "summer": | ||
extent = tgis.AbsoluteTemporalExtent( | ||
start_time=datetime(int(year), 6, 21), | ||
end_time=datetime(int(year), 9, 20), | ||
) | ||
elif seas == "autumn": | ||
extent = tgis.AbsoluteTemporalExtent( | ||
start_time=datetime(int(year), 9, 20), | ||
end_time=datetime(int(year), 12, 21), | ||
) | ||
elif seas == "winter": | ||
extent = tgis.AbsoluteTemporalExtent( | ||
start_time=datetime(int(year), 12, 21), | ||
end_time=datetime(int(year) + 1, 3, 20), | ||
) | ||
map_layer.set_temporal_extent(extent=extent) | ||
season_vect.append(map_layer) | ||
temp_season = f"sample_seasons_{year}" | ||
outsp = tgis.open_new_stds( | ||
temp_season, | ||
"stvds", | ||
"absolute", | ||
f"Season vector year {year}", | ||
f"Season vector for the year {year}", | ||
"mean", | ||
dbif, | ||
gs.overwrite(), | ||
) | ||
tgis.register_map_object_list( | ||
"vector", | ||
season_vect, | ||
outsp, | ||
False, | ||
None, | ||
dbif, | ||
) | ||
remove_dataset["stvds"].append(temp_season) | ||
|
||
process_queue = pymod.ParallelModuleQueue(int(nprocs)) | ||
|
||
# create t.rast.aggregate.ds module to be copied | ||
mod = pymod.Module("t.rast.aggregate.ds") | ||
mod.inputs.input = strds | ||
mod.inputs.method = method | ||
mod.inputs.basename = basename | ||
mod.inputs.type = "stvds" | ||
mod.flags.quiet = False | ||
mod.flags.n = register_null | ||
mod.flags.overwrite = gs.overwrite() | ||
|
||
count = 0 | ||
|
||
outputs = [] | ||
# for each year calculate seasonal aggregation | ||
for year in years: | ||
print(year) | ||
mymod = copy.deepcopy(mod) | ||
mymod.inputs.sample = f"sample_seasons_{year}@{mapset}" | ||
if output_name: | ||
myout = f"{output_name}_{year}" | ||
remove_dataset["strds"].append(myout) | ||
outputs.append(myout) | ||
mymod.outputs.output = myout | ||
else: | ||
mymod.outputs.output = f"{basename}_{year}" | ||
print(mymod.get_bash()) | ||
process_queue.put(mymod) | ||
|
||
if count % 10 == 0: | ||
gs.percent(count, len(years), 1) | ||
|
||
# Wait for unfinished processes | ||
process_queue.wait() | ||
|
||
if len(outputs) > 1: | ||
pymod.Module("t.merge", inputs=",".join(outputs), output=output_name) | ||
|
||
return True | ||
|
||
|
||
if __name__ == "__main__": | ||
atexit.register(cleanup) | ||
main() |
117 changes: 117 additions & 0 deletions
117
src/temporal/t.rast.aggregate.seasons/testsuite/test_seasons.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
"""Test for t.rast.seasons | ||
(C) 2023 by the GRASS Development Team | ||
This program is free software under the GNU General Public | ||
License (>=v2). Read the file COPYING that comes with GRASS | ||
for details. | ||
@author Luca Delucchi | ||
""" | ||
|
||
import grass.temporal as tgis | ||
from grass.gunittest.case import TestCase | ||
from grass.gunittest.main import test | ||
|
||
|
||
class TestClimatologies(TestCase): | ||
@classmethod | ||
def setUpClass(cls): | ||
"""Initiate the temporal GIS and set the region""" | ||
tgis.init(True) # Raise on error instead of exit(1) | ||
cls.use_temp_region() | ||
cls.runModule("g.region", s=0, n=80, w=0, e=120, b=0, t=50, res=10, res3=10) | ||
|
||
cls.runModule("r.mapcalc", expression="a_1 = 5", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_2 = 10", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_3 = 15", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_4 = 20", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_5 = 25", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_6 = 30", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_7 = 35", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_8 = 40", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_9 = 45", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_10 = 50", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_11 = 55", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="a_12 = 60", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_1 = 5", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_2 = 10", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_3 = 15", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_4 = 20", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_5 = 25", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_6 = 30", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_7 = 35", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_8 = 40", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_9 = 45", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_10 = 50", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_11 = 55", overwrite=True) | ||
cls.runModule("r.mapcalc", expression="b_12 = 60", overwrite=True) | ||
|
||
cls.runModule( | ||
"t.create", | ||
type="strds", | ||
temporaltype="absolute", | ||
output="monthly", | ||
title="Monthly test", | ||
description="Monthly test", | ||
overwrite=True, | ||
) | ||
cls.runModule( | ||
"t.register", | ||
flags="i", | ||
type="raster", | ||
input="monthly", | ||
maps="a_1,a_2,a_3,a_4,a_5,a_6,a_7,a_8,a_9,a_10,a_11,a_12", | ||
start="2001-01-01", | ||
increment="1 month", | ||
overwrite=True, | ||
) | ||
cls.runModule( | ||
"t.register", | ||
flags="i", | ||
type="raster", | ||
input="monthly", | ||
maps="b_1,b_2,b_3,b_4,b_5,b_6,b_7,b_8,b_9,b_10,b_11,b_12", | ||
start="2002-01-01", | ||
increment="1 month", | ||
overwrite=True, | ||
) | ||
|
||
@classmethod | ||
def tearDownClass(cls): | ||
"""Remove the time series""" | ||
cls.runModule("t.remove", flags="rf", type="strds", inputs="monthly") | ||
|
||
def test_no_years(self): | ||
"""Test on all years""" | ||
self.assertModule( | ||
"t.rast.aggregate.seasons", | ||
input="monthly", | ||
output="monthly_seasons", | ||
basename="seasons", | ||
overwrite=True, | ||
verbose=True, | ||
) | ||
out = tgis.open_old_stds("monthly_seasons", type="strds") | ||
self.assertEqual(out.metadata.get_number_of_maps(), 7) | ||
|
||
def test_one_year(self): | ||
"""Test just one year""" | ||
self.assertModule( | ||
"t.rast.aggregate.seasons", | ||
input="monthly", | ||
basename="oneseason", | ||
years=2002, | ||
overwrite=True, | ||
verbose=True, | ||
) | ||
out = tgis.open_old_stds("oneseason_2002", type="strds") | ||
self.assertEqual(out.metadata.get_number_of_maps(), 3) | ||
|
||
def test_error_missing_basename(self): | ||
"""Test if basename is missing""" | ||
self.assertModuleFail( | ||
"t.rast.aggregate.seasons", | ||
input="monthly", | ||
) | ||
|
||
|
||
if __name__ == "__main__": | ||
test() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Documentation says: