You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
All of the dump to s3 logic (i.e. the pg_dump command) occurs on the same exact machine, which will quickly get strapped for resources when dealing with the size of some of the dumps. Thus, we need to update the step function to synchronize the processing of all of the dumps - namely, ensuring that the "egis" dumps and "viz" dumps are not parallel processed, and same with processing schemas versus tables. Right now the step function branches out in to essentially four tracks, but it should really just be a single track.
The text was updated successfully, but these errors were encountered:
All of the dump to s3 logic (i.e. the pg_dump command) occurs on the same exact machine, which will quickly get strapped for resources when dealing with the size of some of the dumps. Thus, we need to update the step function to synchronize the processing of all of the dumps - namely, ensuring that the "egis" dumps and "viz" dumps are not parallel processed, and same with processing schemas versus tables. Right now the step function branches out in to essentially four tracks, but it should really just be a single track.
The text was updated successfully, but these errors were encountered: