Check that all E-field values are finite in Ohm solver #5417
Conversation
roelof-groenewald
added
the
component: fluid-ohm
|
Great idea! If this impacts the code’s speed, perhaps a debug flag should be used for this. |
|
The Intel oneAPI workflows have been fixed in #5419, please rebase on/merge |
Signed-off-by: roelof-groenewald <[email protected]>
roelof-groenewald
force-pushed
the
fields_check
branch
from
4b065ea
to
6521838
Compare
Thanks @kli-jfp. Local testing didn't show much of any performance penalty due to the new check since the field solver already includes quite a few reduction operations (so adding one more isn't really noticeable). But if you find in your application it does change the performance we can definitely update it to only check in debug mode. |
| for (int idim = 0; idim < 3; ++idim) { | ||
| WARPX_ALWAYS_ASSERT_WITH_MESSAGE( | ||
| Efield_fp[lev][idim]->is_finite(), | ||
| "Non-finite value detected in E-field; this indicates more substeps should be used in the field solver." |
There was a problem hiding this comment.
We could:
- Set the flag
local=trueto avoid parallel operations (reduction) - Use
amrex::Abortso that it does not hand if only one of the MPI ranks fails.
There was a problem hiding this comment.
I tested that approach and it still hangs if all processors don't call Abort. So I reverted to using local=false. I would imagine the way that is implemented is to basically call is_finite locally and then reduce the resulting boolean for all ranks, which being 1 bit should be very fast.
|
Closing and reopening to trigger Azure tests. |
| { | ||
| for (int idim = 0; idim < 3; ++idim) { | ||
| WARPX_ALWAYS_ASSERT_WITH_MESSAGE( | ||
| Efield_fp[lev][idim]->is_finite(), |
There was a problem hiding this comment.
@WeiqunZhang this is the PR we mentioned in the developer meeting yesterday.
I found that if I use Efield_fp[lev][idim]->is_finite(true) to avoid the MPI reduction the execution hangs (presumably due to only some processes returning false).
I've tested this by modifying the magnetic reconnection test to make it unstable. I attached the input script for the unstable case if you want to play around with it.
This assert is meant to help identify the origin of a common problem in hybrid-PIC simulations, wherein unresolved Whistler waves cause runaway E-field values. Currently when this happens, WarpX fails in the current deposition routine.