warpx.roundrobin_sfc: A runtime parameter to control distribution mapping #4909
Conversation
WeiqunZhang
force-pushed
the
make_distromap
branch
2 times, most recently
from
99d9517
to
6f15ec5
Compare
…ping The default is false. If it's true, AMReX's RRSFS strategy will be used to override the default SFC strategy used by amrex::AmrCore. The motivation for this is that this might mitigate the load imbalance issue during initialization by avoiding putting neighboring boxes on the same process.
WeiqunZhang
force-pushed
the
make_distromap
branch
from
6f15ec5
to
275f45a
Compare
ax3l
added
the
component: load balancing
ax3l
added
the
component: initialization
|
Looks good. Would his affect the runtime, slowing the MPI communication since neighboring boxes will be far away from each other? |
|
@WeiqunZhang sorry for the delay, can you please rebase? |
|
Thank you, @WeiqunZhang! |
WeiqunZhang
force-pushed
the
make_distromap
branch
from
c75adf3
to
4babfd5
Compare
|
Thank you so much! After checking back with our collaborators, we currently do not have a reproducer at hand, but I'll include it in the coming release so we can try it when we should find one again. @n01r might have one we can try this on, too. |
The idea is to overcome out-of-memory in the init routines (e.g., for LPI sims with a lot of particles in close-by ranks) and then apply load balancing based on a user-defined cost (and distribution) model, as usual. |
The default is false. If it's true, AMReX's RRSFC strategy will be used to override the default SFC strategy used by amrex::AmrCore. The motivation for this is that this might mitigate the load imbalance issue during initialization by avoiding putting neighboring boxes on the same process.