Memory Limited Pruning

[mjohnson541]

Currently RMG prunes to a limit number of edge species, the purpose of this is to limit the memory of the RMG process. However for different jobs the same number of edge species corresponds to different amounts of memory, this is problematic when running RMG jobs on servers.

Proposal: Prune to a given process memory level rather than a fixed number of edge species.

Important considerations: Interaction with parallelization

[JacksonBurns]

So we would basically assign a per-process memory limit, and always prune down the number of edge species to that limit on each iteration? This seems like a great idea, compared to having to guess at the number of allowed species which would not cause a crash.

General question - if during species or reaction generation the number of edge species exceeds the limits of memory, does this cause the job to crash? Essentially, is pruning only done after all the generation has been completed?

[mjohnson541]

Flux pruning requires simulation results so you can't prune a species until after it has been in the edge for at least one simulation. Thermodynamic pruning has two parts...one of which can (and I think does) trigger on species creation (if the species has too high Gf) and another that could be triggered whenever, but I believe triggers right after flux pruning that removes species with high Gf down to a set level.