-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[draft] tar_terra_rast: implement {gdalraster} SOZip preserve_metadata
method
#123
base: master
Are you sure you want to change the base?
Conversation
R/tar-terra-rast.R
Outdated
gdalraster_sozip = function(path) { | ||
terra::rast(paste0("/vsizip/{", path, "}/"), basename(path)) | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to worry about file.path here, or is this syntax going to be generalisable to all OS's?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks so much for this, @brownag ! A couple of small changes requested, and I'm also wondering if/how we should think about tests for this?
R/tar-terra-rast.R
Outdated
|
||
tmp <- withr::local_tempdir() | ||
|
||
dir.create(file.path(tmp, dirname(path)), recursive = TRUE) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest moving file.path(tmp, dirname(path))
out into a separate variable, since it gets referred to a few times later on in this function
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok this is done. Sorry for some commit noise on this, made a mistake that broke tests, but I think it is resolved now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See also #127 - the fix(es) for that issue are maybe also required here.
What do you think about using just |
That is a good idea. However I think this doesn't actually resolve all of our metadata issues as it appears .aux.json is not read by terra when specifying the path this way (see tests added in 02ca360 following the "zip" example). Perhaps this PR was premature, but I wanted to try and kick the tires on it a bit. Will have some time to look closer at things this week, it may be possible for us to read from the .aux.json and apply metadata to the result SpatRaster ourselves in lieu of |
It might be good to get clarity on whether this is something that can be fixed in If this is something you can figure out how to do (add the aux.json metadata back in to the |
- use driver that stores metadata externally - include dataset and band user tags
This is a draft/example PR to adds an additional option "gdalraster_sozip" for the
preserve_metadata
argument totar_terra_rast()
.In the future it would be nice if the GDAL C API for working with SOZip files was exposed through terra--which could be a future option i.e. "terra_sozip" or just "sozip".
Using this utility requires GDAL >= 3.7. I have not added any version check to this yet.
Some additional questions I have pertain to the ability to write to ZIP with multiple threads. I have not tinkered with this on HPC/multiple worker type setups.