Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Raster.to_pointcloud() modular and add Raster.get_mask() for memory-efficient operations requiring mask #501

Merged
merged 13 commits into from
Mar 19, 2024

Conversation

rhugonnet
Copy link
Member

@rhugonnet rhugonnet commented Mar 7, 2024

Summary

This PR improves Raster.to_pointcloud() in terms of argument consistency, modularity, and makes its subsampling functionality only returns valid values (while before it returned any random sample) to also be consistent with other subsampling methods.
Additionally, this PR adds a non-public Raster._load_only_mask() function called by a new public function Raster.get_mask(), which allows to get the nodata mask without loading the array into memory. This is used in .to_pointcloud() to perform sampling of only valid values without loading the array in memory 😄.

The PR also adds a lot of tests!

Detailed changes

This PR:

  • Renames Raster.to_points() into Raster.to_pointcloud() for consistent naming in preparation for Add PointCloud subclass of Vector #492,
  • Define a main data_column_name argument for Raster.to_pointcloud() to associate the point cloud to a certain data_band (instead of having a hard-coded "b1", "b2", etc), and proposes to store all columns using the new argument store_auxiliary_bands (naming to be made consistent with PointCloud class but should be fairly close),
  • Adds a random_state argument for subsample,
  • Changes the subsampling done to rely on subsample_array(), after calling a new _load_only_mask function to get the valid mask from disk without loading the raster array,
  • Modifies _load_rio to allow to return only the mask by calling rasterio.read_masks(),
  • Adds a Raster.get_mask() function that returns the raster mask consistently as a boolean array (to circumvent the issue of having a single False value, see Deal consistently with Raster.data.mask=False #505) and without loading the raster array.

Resolves #499
Resolves #504
Resolves #505
Resolves #502

TO-DO:

@rhugonnet rhugonnet changed the title Make Raster.to_points() modular and add Raster._load_only_mask for memory-efficient operation requiring only valid mask Make Raster.to_pointcloud() modular and add Raster._load_only_mask for memory-efficient operation requiring only valid mask Mar 8, 2024
@rhugonnet rhugonnet marked this pull request as ready for review March 8, 2024 21:56
@rhugonnet rhugonnet changed the title Make Raster.to_pointcloud() modular and add Raster._load_only_mask for memory-efficient operation requiring only valid mask Make Raster.to_pointcloud() modular and add Raster.get_mask for memory-efficient operations requiring only valid mask Mar 8, 2024
@rhugonnet rhugonnet changed the title Make Raster.to_pointcloud() modular and add Raster.get_mask for memory-efficient operations requiring only valid mask Make Raster.to_pointcloud() modular and add Raster.get_mask() for memory-efficient operations requiring mask Mar 9, 2024
geoutils/raster/raster.py Outdated Show resolved Hide resolved
geoutils/raster/raster.py Outdated Show resolved Hide resolved
geoutils/raster/raster.py Outdated Show resolved Hide resolved
geoutils/raster/raster.py Outdated Show resolved Hide resolved
geoutils/raster/raster.py Outdated Show resolved Hide resolved
tests/test_raster.py Outdated Show resolved Hide resolved
Copy link
Member

@adehecq adehecq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart some minor comments, looking good to me. Thanks !!

@rhugonnet rhugonnet merged commit a5a5d79 into GlacioHack:main Mar 19, 2024
13 checks passed
@rhugonnet rhugonnet deleted the fix_to_points branch March 19, 2024 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants