Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resume halted runs #24

Open
jeffersonfparil opened this issue Mar 4, 2024 · 2 comments
Open

Resume halted runs #24

jeffersonfparil opened this issue Mar 4, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@jeffersonfparil
Copy link
Owner

  • use serde to save estimated pairwise LD
  • add more threads to generate intermediate files more quickly
  • allow resumption of halted run by:
    + reloading the LD vector of vectors, and
    + counting the number of intermediate files to find where it halted and resume from there.
@jeffersonfparil
Copy link
Owner Author

jeffersonfparil commented Mar 4, 2024 •
edited
Loading

  • For the potentially very large LD Vec<Vec>, serialise and save piecewise, and compress if possible

jeffersonfparil added a commit that referenced this issue Mar 5, 2024
@jeffersonfparil
starting to address issue #24: tested serde and bincode to save/reloa…
ccaf5ce 
…d the large Vec<Vec<u8>> of LD estimates + doubling the number of threads for aldknnni does not improve computation time so we are ditching that proposed change
jeffersonfparil added a commit that referenced this issue Mar 5, 2024
@jeffersonfparil
starting to address issue #24: tested serde and bincode to save/reloa…
6e82ce1 
…d the large Vec<Vec<u8>> of LD estimates + doubling the number of threads for aldknnni does not improve computation time so we are ditching that proposed change
@jeffersonfparil
Copy link
Owner Author

  • save and reload the min_corr_loci and max_pool_dist per locus per chunk

@jeffersonfparil jeffersonfparil added the enhancement New feature or request label Apr 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jeffersonfparil