Skip to content

Commit

Permalink
Addressing viewshed numerical error issue on M1.
Browse files Browse the repository at this point in the history
  • Loading branch information
phargogh committed Apr 30, 2024
1 parent 55804f9 commit f115c6a
Showing 1 changed file with 15 additions and 2 deletions.
17 changes: 15 additions & 2 deletions src/natcap/invest/scenic_quality/viewshed.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -815,8 +815,21 @@ def viewshed(dem_raster_path_band,
if target_distance > max_visible_radius:
break

z = (((previous_height-r_v)/slope_distance) *
target_distance + r_v)
# This is a weird platform-specific workaround addressing
# https://github.com/natcap/invest/issues/1562
# On M1 macs, the all-in-one-line addition of _product and r_v
# would create small but noticeable numerical error. Breaking the
# calculation onto two lines eliminates the numerical error. This
# behavior is reproducible in C, outside of Cython on an M1 mac.
# So, this calculation would introduce error:
# z = (((previous_height-r_v)/slope_distance) * target_distance) + r_v
# while the formlation below does not.
#
# Some of this may be related to the fact that x86 chips have
# extended precision for FPU-based calculations while M1 ARM chips
# do not. Still, that doesn't explain why the error is introduced.
_product = (((previous_height-r_v)/slope_distance) * target_distance)
z = _product + r_v

# add on refractivity/curvature-of-earth calculations.
adjustment = 0.0 # increase in required height due to curvature
Expand Down

0 comments on commit f115c6a

Please sign in to comment.