Keeping snes around in between time steps

[reverendbedford]

PetscNonlinearSolver clears the snes instance in between solves even though (in MOOSE) we keep the libmesh solver object itself alive between time steps. I would like to keep the snes instance around in between solves to enable preconditioner reuse over multiple (MOOSE) time steps in implicit time integration. It would be neat to be able to do this, as some examples I've been playing with suggest it will speed up some of our analyses by a factor of 10 or so.

I compiled MOOSE with a version of libmesh that literally just comments out the clear() call at the end of solve() and all the tests in the module/app I work on ran fine. But that's of course not comprehensive...

So questions:

1. Is there a need to clear the snes out after a solve?
2. Assuming there is some need to clear the instance in cases I'm not testing, how willing would we be to accept a PR that makes clearing out the snes instance optional?

Thanks!

[jwpeterson]

My guess is that we call clear() every time because we also call init() every time... and didn't want to inadvertently initialize something twice, leak memory, etc. So I would feel better if we attach some valgrind testing to your PR, but otherwise if the MOOSE tests all passed that is a pretty strong indicator that we don't need to call clear() every time, since those tests make extensive uses of PetscNonlinearSolver. I'd say we're always willing to accept a PR that makes something optional while defaulting to the current behavior, since that should be a pretty low-risk change in my opinion.

[reverendbedford]

Okay, I'll work on something making it optional. And yes, I should check the valgrind results. But init() does seem to guard against reinitializing things with this->is_initialized().

[roystgnr]

I'd say even if we start with "optional but defaulting to the current behavior" I'd want to move toward "optional but defaulting to the new behavior" in the medium term. IIRC we had the same "delete and recreate at every time step" bug in another PETSc solver interface and nobody reported any problems from us fixing it.

[reverendbedford]

Quick update:

1. I can see no harm (over in MOOSE) in keeping the SNES object around.
2. However, MOOSE is somehow partly relying on a "clean" SNES object. If keep the SNES object around between solvers and don't call SNESReset then it changes the results in a minority of tests in some MOOSE modules/apps (not mine though interestingly).
3. Calling SNESReset between solves keeps results one-to-one with the previous MOOSE behavior but wipes the preconditioner I'm trying to save...

So my proposal for a PR:

1. I will always keep the SNES object around -- I will not provide an option to keep the exact same behavior as before where you call SNESDestroy as I don't see the point in doing that
2. By default I will call SNESReset in between solves
3. I'll introduce two new options into NonlinearImplicitSystem:

• "reuse preconditioner", defaults to false. If true I'll not call SNESReset between solves and I will do the SNESSetLagPreconditionerPersists stuff.
• "reuse preconditioner maximum iterations" -- a parameter giving us a reuse heuristic of "continue reusing the preconditioner until the linear solver requires more than this # of iterations.

I'll get the PR up this afternoon.