Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[13pt] Update 3DEP DEMs #1281

Open
CarsonPruitt-NOAA opened this issue Sep 11, 2024 · 5 comments · May be fixed by #1301
Open

[13pt] Update 3DEP DEMs #1281

CarsonPruitt-NOAA opened this issue Sep 11, 2024 · 5 comments · May be fixed by #1301
Assignees
Labels

Comments

@CarsonPruitt-NOAA
Copy link
Collaborator

CarsonPruitt-NOAA commented Sep 11, 2024

Based on the findings in #1257, we should update our DEMs from the USGS 3DEP service to ensure we're using up-to-date data.

Instead of doing a full domain download and replace, it may be better to take a targeted approach to only download the areas that have been updated. Here's an example of a way we could accomplish that.
aws s3 ls s3://prd-tnm/StagedProducts/Elevation/13/TIFF/current/ --recursive

@RobHanna-NOAA
Copy link
Contributor

We can not download just one part, well... not easily. We have our tool to download DEMs but it does a handful of other things such as some cropping, downloads with critical params, creates polys and the vrt. You really can not replace just small parts. It is pretty easy to run our acquire 3dep script and isn't overly slow on a prod machine. I can kick off a full new set in the next few days.

@RobHanna-NOAA RobHanna-NOAA self-assigned this Sep 11, 2024
@RobHanna-NOAA RobHanna-NOAA changed the title [13pt] Update 3DEP DEMs [3pt] Update 3DEP DEMs Sep 11, 2024
@CarsonPruitt-NOAA
Copy link
Collaborator Author

That's probably a better choice anyway to make sure we don't accidentally miss any files.

@RobHanna-NOAA
Copy link
Contributor

ya. and with the critical other download args we need and post download processing, you would find is it incredibly difficult to do it without our acquire tool. There is more processing in our tool than one might think. The hardest part of any input changes is not the acquire, it is the processing and careful process of getting all right files, vrts, dems and poly's to all 5 / 6 environments at the right time. ie) Hopefully not overwriting anything. :)

@mluck
Copy link
Contributor

mluck commented Sep 17, 2024

We should revisit #887 with the new DEMs

@RobHanna-NOAA
Copy link
Contributor

Good point. When I have this done plus the pre-clips, we can recheck this before merge.

@RobHanna-NOAA RobHanna-NOAA linked a pull request Sep 20, 2024 that will close this issue
20 tasks
@RobHanna-NOAA RobHanna-NOAA changed the title [3pt] Update 3DEP DEMs [13pt] Update 3DEP DEMs Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants