Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split simulations into chunks #1938

Merged
merged 15 commits into from
Nov 5, 2024
Merged

Split simulations into chunks #1938

merged 15 commits into from
Nov 5, 2024

Conversation

nikhilwoodruff
Copy link
Collaborator

No description provided.

@nikhilwoodruff nikhilwoodruff linked an issue Nov 1, 2024 that may be closed by this pull request
@nikhilwoodruff
Copy link
Collaborator Author

Aah OK figured out the reason why microsims are failing here but passing locally... When we spawn workers in GH actions, neither has the microdata downloaded. Both try to download the file but one starts first. The other sees the partial download file, thinks "OK great dataset exists I'll try to read it" and fails.

@nikhilwoodruff
Copy link
Collaborator Author

Passing tests and merging!

@nikhilwoodruff nikhilwoodruff merged commit 4301f2d into master Nov 5, 2024
4 checks passed
@nikhilwoodruff nikhilwoodruff deleted the splitting-sims branch November 5, 2024 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Split simulations into chunks to avoid memory limits
1 participant