Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add warp_mem_limit and num_threads to rasterio reproject calls #285

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

eecarres
Copy link
Collaborator

@eecarres eecarres commented Feb 3, 2021

This commit aims to expose the two available parameters in rasterio
reproject method, used to control its performance

@eecarres eecarres force-pushed the add-warp_mem_limit-and-num_threads-params-to-reproject-calls branch 3 times, most recently from 9d14c0b to 5003a83 Compare February 3, 2021 22:20
@eecarres eecarres force-pushed the add-warp_mem_limit-and-num_threads-params-to-reproject-calls branch from 5003a83 to 76f5a80 Compare February 15, 2021 12:28
This commit aims to expose the two available parameters in rasterio
reproject method, used to control its performance
@eecarres eecarres force-pushed the add-warp_mem_limit-and-num_threads-params-to-reproject-calls branch from 76f5a80 to b44f998 Compare February 15, 2021 12:51
@codecov-io
Copy link

codecov-io commented Feb 15, 2021

Codecov Report

Merging #285 (b44f998) into master (a1529cc) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #285   +/-   ##
=======================================
  Coverage   90.64%   90.64%           
=======================================
  Files          37       37           
  Lines        5943     5943           
=======================================
  Hits         5387     5387           
  Misses        556      556           
Impacted Files Coverage Δ
telluric/georaster.py 93.51% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a1529cc...b44f998. Read the comment docs.

@arielze
Copy link
Contributor

arielze commented Feb 15, 2021

@eecarres this looks like a good way to go, although I recommend to introduce a more generic approach, something like, rasterio_kwargs which accepts a dictionary of arguments passed to rasterio.warp.reproject

@eecarres
Copy link
Collaborator Author

eecarres commented Feb 15, 2021

@arielze Whatever you find the most convenient. What I saw in the telluric code was that direct calls to rasterio reproject were done with the option of giving all the parameters explicitly, that's why I tried to "continue with the trend". As you can see here the rasterio call also has a kwargs, but the rest of the parameters (except from the two I added) are explicitly defined in the telluric calls.

As I said, you decide =D

@drnextgis
Copy link
Contributor

I do agree with @arielze. These two parameters are too low level and we don't deal with them in the source code. So I think it worth to create a new **kwargs argument for merge_all. In this case we don't need to copypaste docstring with description of these parameters into myriads of internal method and also it allows us to use new arguments that might be added in the future to rasterio.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose warp_mem_limit and num_threads parameters of rasterio reproject method
4 participants