Skip to content

Feature/density tracks #1003

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 47 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
8bb0886
add function density_track
NicolasColombi Feb 5, 2025
40d63e0
update
NicolasColombi Feb 6, 2025
7e3a70e
add test
NicolasColombi Feb 6, 2025
e0f20a3
update docstrings and changelog
NicolasColombi Feb 6, 2025
cc40685
use np.linspace and scipy.sparse
NicolasColombi Feb 6, 2025
fb7aa7c
count only track once per grid cell
NicolasColombi Feb 6, 2025
fea61a2
add argument to filter different tracks for density
NicolasColombi Feb 6, 2025
f9cd50d
add wind speed selection
NicolasColombi Feb 8, 2025
49ebfb2
optimize after profiling
NicolasColombi Feb 8, 2025
509d60b
add function compute grid cell area
NicolasColombi Feb 10, 2025
fd7be94
add plotting function
NicolasColombi Feb 11, 2025
9e61a65
Merge branch 'develop' into feature/density_tracks
NicolasColombi Feb 11, 2025
6016af5
update changelog
NicolasColombi Feb 11, 2025
541937e
move grid area function to util
NicolasColombi Feb 12, 2025
45ccd8a
fix pylint
NicolasColombi Feb 12, 2025
ec4a819
fix pylint f string
NicolasColombi Feb 12, 2025
6f06f76
add second function to compute grid area with projections
NicolasColombi Feb 12, 2025
26460a7
update changelog
NicolasColombi Feb 12, 2025
c08727e
Merge branch 'develop' into feature/density_tracks
NicolasColombi Feb 12, 2025
8a66674
fix unit test
NicolasColombi Feb 18, 2025
f3de084
Merge branch 'develop' into feature/density_tracks
NicolasColombi Feb 18, 2025
870ec7c
Update tc_tracks.py
NicolasColombi Feb 18, 2025
9919d11
Update tc_tracks.py
NicolasColombi Feb 18, 2025
0b6a81e
fix typeError
NicolasColombi Feb 18, 2025
5ca286e
restructure function and add tests
NicolasColombi Feb 21, 2025
0cd7199
Merge branch 'develop' into feature/density_tracks
NicolasColombi Feb 21, 2025
9537568
fix pylint
NicolasColombi Feb 21, 2025
b983c7d
add test grid cell area
NicolasColombi Feb 21, 2025
1046328
fix jenkins wrong interpretation of type
NicolasColombi Feb 21, 2025
ee152ec
add datatype to from_FAST and fix test
NicolasColombi Feb 21, 2025
3d96d79
remove from fast data type
NicolasColombi Feb 21, 2025
c622d6e
fix test
NicolasColombi Feb 21, 2025
29247b1
possible fix to pre-commit
NicolasColombi Feb 23, 2025
2581a08
Merge branch 'develop' into feature/density_tracks
NicolasColombi Feb 23, 2025
32a0fde
update function
NicolasColombi Mar 5, 2025
1f84c40
update doc strings
NicolasColombi Mar 5, 2025
68549d8
Merge branch 'develop' into feature/density_tracks
NicolasColombi Mar 5, 2025
7414fd5
Update climada/hazard/tc_tracks.py
NicolasColombi Mar 14, 2025
2f6ee27
Update climada/hazard/tc_tracks.py
NicolasColombi Mar 14, 2025
f398b59
Update climada/hazard/tc_tracks.py
NicolasColombi Mar 14, 2025
11daf46
Update climada/hazard/tc_tracks.py
NicolasColombi Mar 14, 2025
d41c268
Update climada/hazard/tc_tracks.py
NicolasColombi Mar 14, 2025
d483f09
Update climada/hazard/tc_tracks.py
NicolasColombi Mar 14, 2025
04e289d
change bounds order for consistency
NicolasColombi Mar 14, 2025
4ff44cd
Merge branch 'develop' into feature/density_tracks
NicolasColombi Mar 14, 2025
4adbe67
Merge branch 'feature/density_tracks' of https://github.com/CLIMADA-p…
NicolasColombi Mar 14, 2025
34f5b50
Merge branch 'develop' into feature/density_tracks
NicolasColombi Mar 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,11 @@ Removed:
### Added

- `climada.hazard.tc_tracks.TCTracks.subset_years` function [#1023](https://github.com/CLIMADA-project/climada_python/pull/1023)
- `climada.hazard.tc_tracks.TCTracks.from_FAST` function, add Australia basin (AU) [#993](https://github.com/CLIMADA-project/climada_python/pull/993)
-`climada.hazard.tc_tracks.compute_track_density` function, `climada.hazard.tc_tracks.compute_genesis_density` function,
`climada.util.coordinates.compute_grid_cell_area` function, `climada.util.coordinates.compute_grid_cell_area_validation` function,
`climada.hazard.tc_tracks.normalize_hist` function, `climada.hazard.plot.plot_track_density` function
[#1003](https://github.com/CLIMADA-project/climada_python/pull/1003)
-`climada.hazard.tc_tracks.TCTracks.from_FAST` function, add Australia basin (AU) [#993](https://github.com/CLIMADA-project/climada_python/pull/993)
- Add `osm-flex` package to CLIMADA core [#981](https://github.com/CLIMADA-project/climada_python/pull/981)
- `doc.tutorial.climada_entity_Exposures_osm.ipynb` tutorial explaining how to use `osm-flex`with CLIMADA
- `climada.util.coordinates.bounding_box_global` function [#980](https://github.com/CLIMADA-project/climada_python/pull/980)
Expand Down
133 changes: 133 additions & 0 deletions climada/hazard/plot.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@

import logging

import cartopy.crs as ccrs
import cartopy.feature as cfeature
import matplotlib.colors as mcolors
import matplotlib.pyplot as plt
import numpy as np
from deprecation import deprecated
Expand Down Expand Up @@ -158,6 +161,136 @@

raise ValueError("Provide one event id or one centroid id.")

@staticmethod
def plot_track_density(

Check warning on line 165 in climada/hazard/plot.py

View check run for this annotation

Jenkins - WCR / Pylint

dangerous-default-value

NORMAL: Dangerous default value {} as argument
Raw output
Used when a mutable value as list or dictionary is detected in a default valuefor an argument.

Check warning on line 165 in climada/hazard/plot.py

View check run for this annotation

Jenkins - WCR / Pylint

too-many-arguments

LOW: Too many arguments (9/7)
Raw output
Used when a function or method takes too many arguments.

Check warning on line 165 in climada/hazard/plot.py

View check run for this annotation

Jenkins - WCR / Pylint

too-many-positional-arguments

LOW: Too many positional arguments (9/5)
Raw output
no description found

Check warning on line 165 in climada/hazard/plot.py

View check run for this annotation

Jenkins - WCR / Pylint

too-many-locals

LOW: Too many local variables (16/15)
Raw output
Used when a function or method has too many local variables.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit confused now about this method. Why is it a Hazard method? This should be a TCTracks method right? It has nothing to do with Hazards in general. At most maybe it could be a TropCyclone method.

In general, we should not add methods that have only a very specific narrow application to a general basis class. This will lead back to the clutter of methods that we now spend years to reduce.

Where do you think this would fit best? TCTracks or TropCyclones? Since it is only about tracks and not about intensities, I would move this to TCTracks.

hist: np.ndarray,
axis=None,
projection=ccrs.Mollweide(),
add_features: dict = None,
title: str = None,
figsize=(12, 6),
div_cmap=False,
cbar=True,
cbar_kwargs: dict = {
"orientation": "horizontal",
"pad": 0.05,
"shrink": 0.8,
"label": "n° tracks per 1° x 1° grid cell",
},
**kwargs,
):
"""
Plot the track density of tropical cyclone tracks on a customizable world map.

Parameters:
----------
hist: np.ndarray
2D histogram of track density.
axis: GeoAxes, optional
Existing Cartopy axis.
projection: cartopy.crs, optional
Projection for the map.
add_features: dict
Dictionary of map features to add. Keys can be 'land', 'coastline', 'borders', and
'lakes'. Values are Booleans indicating whether to include each feature.
title: str
Title of the plot.
figsize: tuple
Figure size when creating a new figure.
div_cmap: bool, default = False
If True, the colormap will be centered to 0.
cbar: bool, Default = True
If True, the color bar is added
cbar_kwargs: dict
dictionary containing keyword arguments passed to cbar
kwargs:
Additional keyword arguments passed to `ax.contourf`.

Returns:
-------
axis: GeoAxes
The plot axis.


Example:
--------
>>> axis = plot_track_density(
... hist=hist,
... cmap='Spectral_r',
... cbar_kwargs={'shrink': 0.8,
'label': 'Cyclone Density [n° tracks / km²]',
'pad': 0.1},
... add_features={
... 'land': True,
... 'coastline': True,
... 'borders': False,
... 'lakes': False
... },
... title='My Tropical Cyclone Track Density Map',
... figsize=(10, 5),
... levels=20
... )

"""

# Default features
default_features = {
"land": True,
"coastline": True,
"borders": False,
"lakes": False,
}
add_features = add_features or default_features

# Sample data
lon = np.linspace(-180, 180, hist.shape[1])
lat = np.linspace(-90, 90, hist.shape[0])

# Create figure and axis if not provided
if axis is None:
_, axis = plt.subplots(
figsize=figsize, subplot_kw={"projection": projection}
)

# Add requested features
if add_features.get("land", False):
land = cfeature.NaturalEarthFeature(
category="physical",
name="land",
scale="50m",
facecolor="lightgrey",
alpha=0.6,
)
axis.add_feature(land)
if add_features.get("coastline", False):
axis.add_feature(cfeature.COASTLINE, linewidth=0.5)
if add_features.get("borders", False):
axis.add_feature(cfeature.BORDERS, linestyle=":")
if add_features.get("lakes", False):
axis.add_feature(cfeature.LAKES, alpha=0.4, edgecolor="black")

if div_cmap:
norm = mcolors.TwoSlopeNorm(
vmin=np.nanmin(hist), vcenter=0, vmax=np.nanmax(hist)
)
kwargs["norm"] = norm

# contourf = axis.contourf(lon, lat, hist, transform=ccrs.PlateCarree(), **kwargs)
contourf = axis.imshow(
hist,
extent=[lon.min(), lon.max(), lat.min(), lat.max()],
transform=ccrs.PlateCarree(),
origin="lower",
**kwargs,
)

if cbar:
plt.colorbar(contourf, ax=axis, **cbar_kwargs)
if title:
axis.set_title(title, fontsize=16)

return axis

Check warning on line 292 in climada/hazard/plot.py

View check run for this annotation

Jenkins - WCR / Code Coverage

Not covered lines

Lines 237-292 are not covered by tests

def plot_fraction(self, event=None, centr=None, smooth=True, axis=None, **kwargs):
"""Plot fraction values for a selected event or centroid.

Expand Down
195 changes: 194 additions & 1 deletion climada/hazard/tc_tracks.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
import matplotlib.cm as cm_mp
import matplotlib.pyplot as plt
import netCDF4 as nc
import numba
import numpy as np
import pandas as pd
import pathos
Expand All @@ -52,6 +51,7 @@
from matplotlib.lines import Line2D
from shapely.geometry import LineString, MultiLineString, Point
from sklearn.metrics import DistanceMetric
from tqdm import tqdm

import climada.hazard.tc_tracks_synth
import climada.util.coordinates as u_coord
Expand Down Expand Up @@ -3094,3 +3094,196 @@
if np.issubdtype(data_var.dtype, float) or np.issubdtype(data_var.dtype, int):
return True
return False


def compute_track_density(

Check warning on line 3099 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Pylint

too-complex

LOW: 'compute_track_density' is too complex. The McCabe rating is 11
Raw output
no description found

Check warning on line 3099 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Pylint

too-many-arguments

LOW: Too many arguments (8/7)
Raw output
Used when a function or method takes too many arguments.

Check warning on line 3099 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Pylint

too-many-positional-arguments

LOW: Too many positional arguments (8/5)
Raw output
no description found

Check warning on line 3099 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Pylint

too-many-locals

LOW: Too many local variables (21/15)
Raw output
Used when a function or method has too many local variables.

Check warning on line 3099 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Pylint

too-many-branches

LOW: Too many branches (13/12)
Raw output
Used when a function or method has too many branches, making it hard tofollow.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this not a class method since the first argument seems to be the object itself?

tc_track: TCTracks,
res: int = 5,
bounds: tuple = None,
genesis: bool = False,
norm: str = None,
filter_tracks: bool = True,
wind_min: float = None,
wind_max: float = None,
) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
"""Compute absolute and normalized tropical cyclone track density. Before using this function,
apply the same temporal resolution to all tracks by calling :py:meth:`equal_timestep` on the
TCTrack object. Due to the computational cost of the this function, it is not recommended to
use a grid resolution higher tha 0.1°. First, this function creates 2D bins of the specified
resolution (e.g. 1° x 1°). Second, since tracks are not lines but a series of points, it counts
the number of points per bin. Lastly, it returns the absolute or normalized count per bin.
To plot the output of this function use :py:meth:`plot_track_density`.

Parameters:
----------
tc_track: TCTracks object
track object containing a list of all tracks
res: int (optional), default: 5°
resolution in degrees of the grid bins in which the density will be computed
bounds: tuple, (optional) dafault: None
(lon_min, lat_min, lon_max, lat_max) latitude and longitude bounds.
genesis: bool, (optional) default = False
If true the function computes the track density of only the genesis location of tracks
norm: str (optional), default: None
If None the function returns the number of samples in each bin. If True, it normalize the
bin count as specified: if norm = area -> normalize by gird cell area. If norm = sum ->
normalize by the total sum across all bins.
filter_tracks: bool (optional) default: True
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps calling this argument something like count_tracks would be more explicit?

Copy link
Collaborator Author

@NicolasColombi NicolasColombi Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, filter_track it's a bit cryptic...

If True the track density is computed as the number of different tracks crossing a grid
cell. If False, the track density takes into account how long the track stayed in each
grid cell. Hence slower tracks increase the density if the parameter is set to False.
wind_min: float (optional), default: None
Minimum wind speed above which to select tracks (inclusive).
wind_max: float (optional), default: None
Maximal wind speed below which to select tracks (exclusive if wind_min is also provided,
otherwise inclusive).
Returns:
-------
hist_count: np.ndarray
2D matrix containing the the absolute count per grid cell of track point or the normalized
number of track points, depending on the norm parameter.
lat_bins: np.ndarray
latitude bins in which the point were counted
lon_bins: np.ndarray
longitude bins in which the point were counted

Example:
--------
>>> tc_tracks = TCTrack.from_ibtracs_netcdf("path_to_file")
>>> tc_tracks.equal_timestep(time_step_h = 1)
>>> hist_count, *_ = compute_track_density(tc_track = tc_tracks, res = 1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is cleaner..

Suggested change
>>> hist_count, *_ = compute_track_density(tc_track = tc_tracks, res = 1)
>>> hist_count, _, _ = compute_track_density(tc_track = tc_tracks, res = 1)

>>> ax = plot_track_density(hist_count)

"""

limit_ratio: float = 1.12 * 1.1 # record tc speed 112km/h -> 1.12°/h + 10% margin
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would that become a problem if there is a future TC set with tracks with windspeeds above 112?

time_value: float = tc_track.data[0].time_step[0].values.astype(float)

if time_value > (res / limit_ratio):
warnings.warn(

Check warning on line 3163 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Code Coverage

Not covered line

Line 3163 is not covered by tests
"The time step is too big for the current resolution. For the desired resolution, \n"
f"apply a time step of {res/limit_ratio}h."
)
elif res < 0.1:
warnings.warn(

Check warning on line 3168 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Code Coverage

Not covered line

Line 3168 is not covered by tests
"The resolution is too high. The computation might take several minutes \n"
"to hours. Consider using a resolution below 0.1°."
)

# define grid resolution and bounds for density computation
if not bounds:
lon_min, lat_min, lon_max, lat_max = -180, -90, 180, 90
else:
lon_min, lat_min, lon_max, lat_max = bounds[0], bounds[1], bounds[2], bounds[3]

Check warning on line 3177 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Code Coverage

Not covered line

Line 3177 is not covered by tests

lat_bins: np.ndarray = np.linspace(lat_min, lat_max, int(180 / res))
lon_bins: np.ndarray = np.linspace(lon_min, lon_max, int(360 / res))

# compute 2D density
if genesis:
hist_count = compute_genesis_density(

Check warning on line 3184 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Code Coverage

Not covered line

Line 3184 is not covered by tests
tc_track=tc_track, lat_bins=lat_bins, lon_bins=lon_bins
)
else:
hist_count = np.zeros((len(lat_bins) - 1, len(lon_bins) - 1))
for track in tqdm(tc_track.data, desc="Processing Tracks"):

# select according to wind speed
wind_speed = track.max_sustained_wind.values
if wind_min and wind_max:
index = np.where((wind_speed >= wind_min) & (wind_speed < wind_max))[0]
elif wind_min and not wind_max:
index = np.where(wind_speed >= wind_min)[0]
elif wind_max and not wind_min:
index = np.where(wind_speed <= wind_max)[0]
else:
index = slice(None) # select all the track

# compute 2D density
hist_new, _, _ = np.histogram2d(
track.lat.values[index],
track.lon.values[index],
bins=[lat_bins, lon_bins],
density=False,
)
if filter_tracks:
hist_new[hist_new > 1] = 1

hist_count += hist_new

if norm:
hist_count = normalize_hist(res=res, hist_count=hist_count, norm=norm)

Check warning on line 3215 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Code Coverage

Not covered line

Line 3215 is not covered by tests

return hist_count, lat_bins, lon_bins


def compute_genesis_density(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition

tc_track: TCTracks, lat_bins: np.ndarray, lon_bins: np.ndarray
) -> np.ndarray:
"""Compute the density of track genesis locations. This function works under the hood
of :py:meth:`compute_track_density`. If it is called with the parameter genesis = True,
the function return the number of genesis points per grid cell.
Comment on lines +3223 to +3225
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this method is not intended to be accessible to the user, then please define as such _compute_genesis_density.


Parameters:
-----------

tc_track: TCT track object
TC track object containing a list of all tracks.
Comment on lines +3230 to +3231
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a function instead of a class method since the first attribute seems to be self?

lat_bins: 1D np.array
array containg the latitude bins.
lon_bins: 1D np.array
array containg the longitude bins.

Returns:
--------
hist_count: 2D np.array
array containing the number of genesis points per grid cell
"""
# Extract the first lat and lon from each dataset
first_lats = np.array([ds.lat.values[0] for ds in tc_track.data])
first_lons = np.array([ds.lon.values[0] for ds in tc_track.data])

# compute 2D density of genesis points
hist_count, _, _ = np.histogram2d(
first_lats,
first_lons,
bins=[lat_bins, lon_bins],
density=False,
)

return hist_count


def normalize_hist(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to be a method called by the user? If yes, leave it as is. If not, please make it private with the _.

res: int,
hist_count: np.ndarray,
norm: str,
) -> np.ndarray:
"""Normalize the number of points per grid cell by the grid cell area or by the total sum of
the histogram count.

Parameters:
-----------
res: float
resolution of grid cells
hist_count: 2D np.array
Array containing the count of tracks or genesis locations
norm: str
if norm = area normalize by gird cell area, if norm = sum normalize by the total sum
Returns:
---------

"""

if norm == "area":
grid_area = u_coord.compute_grid_cell_area(res=res)
norm_hist: np.ndarray = hist_count / grid_area

Check warning on line 3280 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Code Coverage

Not covered lines

Lines 3279-3280 are not covered by tests
elif norm == "sum":
norm_hist: np.ndarray = hist_count / hist_count.sum()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps add

else:
    raise ValueError("Invalid value for input parameter 'norm', it should be one of 'area' or 'sum'")

or something like this? It would also remove the pylint warning below ("Possibly using variable 'norm_hist' before assignment").

else:
raise ValueError(

Check warning on line 3284 in climada/hazard/tc_tracks.py

View check run for this annotation

Jenkins - WCR / Code Coverage

Not covered line

Line 3284 is not covered by tests
"Invalid value for input parameter 'norm':\n"
"it should be either 'area' or 'sum'"
)

return norm_hist
Loading