Skip to content

Feature/custom grid litpop #1022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 20 commits into
base: develop
Choose a base branch
from
Draft

Conversation

zeliest
Copy link
Collaborator

@zeliest zeliest commented Mar 10, 2025

Changes proposed in this PR:

  • Added a target_grid option for creating LitPop exposures. If the argument is not provided, the grid is automatically defined based on the population dataset (GPW) or nightlight dataset (NASA Black Marble), depending on the resolution.

    • If res_arcsec == 15, the Black Marble Nightlight grid is used with a predefined global transform.
    • Otherwise, the grid aligns with GPW, adjusting to the nearest resolution step based on its transform.
  • Removed outdated test case (test_reproject_input_data_odd_downsample), as it was no longer relevant with the new grid alignment method.

  • Added a new test (test_target_grid_alignment) to verify that the reprojection correctly aligns with the target grid.

Why this is needed:

  • Allows to flexibly define grid for litpop to align with hazards
  • This becomes faster when defining LitPop for many countries, as the grid is defined only once.

PR Author Checklist

PR Reviewer Checklist

@@ -108,6 +109,7 @@ def from_countries(
reference_year=DEF_REF_YEAR,
gpw_version=GPW_VERSION,
data_dir=SYSTEM_DIR,
target_grid=None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add the docstring for this new variable?

@@ -290,6 +298,7 @@ def from_nightlight_intensity(
res_arcsec=15,
reference_year=DEF_REF_YEAR,
data_dir=SYSTEM_DIR,
target_grid=None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

idem, can you please add the docstring?

@@ -374,6 +392,7 @@ def from_population(
reference_year=DEF_REF_YEAR,
gpw_version=GPW_VERSION,
data_dir=SYSTEM_DIR,
target_grid=None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

idem, can you please add the docstring?

@@ -457,6 +485,7 @@ def from_shape_and_countries(
reference_year=DEF_REF_YEAR,
gpw_version=GPW_VERSION,
data_dir=SYSTEM_DIR,
target_grid=None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

idem, can you please add the docstring?

@@ -144,31 +141,61 @@ def test_reproject_input_data_downsample(self):
def test_reproject_input_data_downsample_conserve_sum(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add at least one test with a custom target grid?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, you can write a loop to test all the exposed methods (from_countries , from_nightlight_intensity etc ...)

Copy link
Member

@chahank chahank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great addition! I have noted quickly some small elements to improve the code itself. I will do a review of the desired result of the new feature soon. I am currently wondering how the resampling of the original data (black marble and gpw) is done, how this affects the output results, and whether this is clear to the user at the moment.

@zeliest zeliest changed the title Feature/costum grid litpop Feature/custom grid litpop Mar 12, 2025
@chahank
Copy link
Member

chahank commented Mar 12, 2025

So, the data is regridded using the method util.coords.align_raster_data (make a link in the docstring with See Also section) and using the resampling rasterio.warp.Resampling.bilinear. I think this information should be added to the docstring, together with the information on how to use either the lit or the pop grid. Also, it should be clear that if the target grid is defined, then the resolution does nothing.

Comment on lines 535 to 539
# TESTS = unittest.TestLoader().loadTestsFromTestCase(TestLitPopExposure)
# TESTS.addTests(unittest.TestLoader().loadTestsFromTestCase(TestAdmin1))
# unittest.TextTestRunner(verbosity=2).run(TESTS)

TESTS = unittest.TestLoader().loadTestsFromTestCase(TestLitPopGridAlignment)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# TESTS = unittest.TestLoader().loadTestsFromTestCase(TestLitPopExposure)
# TESTS.addTests(unittest.TestLoader().loadTestsFromTestCase(TestAdmin1))
# unittest.TextTestRunner(verbosity=2).run(TESTS)
TESTS = unittest.TestLoader().loadTestsFromTestCase(TestLitPopGridAlignment)
TESTS = unittest.TestLoader().loadTestsFromTestCase(TestLitPopExposure)
TESTS.addTests(unittest.TestLoader().loadTestsFromTestCase(TestAdmin1))
TESTS.addTests(unittest.TestLoader().loadTestsFromTestCase(TestLitPopGridAlignment))

@chahank
Copy link
Member

chahank commented Mar 26, 2025

The tests are currently failing due to a too-low test coverage. It means that many of the added lines of code are never tested for. This suggests that additional unit tests are required.

Also, there are small linter errors that are easy to correct (trailing white spaces for instance).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants