-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Keep previous tileset as backup #11
Comments
From a technical standpoint, this should be easy to implement. We can just have the upload script first make a copy of the planet within the bucket to e.g. planet.1.pmtiles. If you want multiple backups, you could copy e.g. 2 to 3, then 1 to 2, then planet to 1. Then, we could add a GitHub action to restore an older planet, so you wouldn't even have to do arcane AWS stuff - anyone with GH access could run the action and restore the old planet. I'm not sure if it's possible to manually pass a parameter, so we'd likely need an action for each Nth planet we want to restore. The "restore" process would be to copy an older planet and overwrite the current one and await the next planet upload by the render server. File storage on s3 is 2.3 cents per GB, so the only real question is how many planet archives at $1.72 per month we want to keep on hand in the bucket. |
One backup sounds prudent, multiple seems like overkill at this point. However, we might not necessarily want the last planet to immediately become the backup ever time; say, if two subsequent builds have the same vandalism. We could replace the backup every 24 or 48 hours or so to reduce the chances of this issue. |
In that case, the script can just check the last-modified date of the current planet file in the s3 bucket, and if it's older than $X, it can overwrite it with the current planet before pushing a new one up. That still leaves a little wiggle room for the planet/vandalism timeline happening at JUSSSST the wrong timeframe but a longer gap is probably a better hedge than a shorter one. |
Would it be feasible to keep the previous tranche of tiles around after an update, to temporarily swap in if necessary using s3 lifecycle configuration rules? If we find out about particularly bad vandalism or, say, someone draining one of the Great Lakes, someone could trigger an action to swap in the previous tiles while the next update is churning away. Related to this idea would be some sort of trigger to halt the production of a known bad dataset.
The text was updated successfully, but these errors were encountered: