Callbacks condition does not reflect if the diagnostic is computed/written to file.

[charleskawczynski]

It would be really nice if we could faithfully query whether a callback should be called/executed, via, cb.condition, as is done in SciMLBase.

This, together with some small changes in ClimaAtmos, would allow us to implement a fully general benchmarking function to ClimaTimesteppers: CliMA/ClimaTimeSteppers.jl#106.

[Sbozzolo]

I think we have that: if you have a ScheduledDiagnostic diag, diag.compute_schedule(integrator) will return true if the diagnostic has to be computed at the given state/time. Similarly with diag.output_schedule(integrator)

[charleskawczynski]

Ah, ok, great. Maybe it'd be as simple as cb.condition(integrator) = any(x->x.diagnostics_handler.scheduled_diagnostics.compute_schedule(integrator), ::DiagnosticsCallbacks)? (we could define a new DiagnosticsCallbacks type instead of using a closure)

[Sbozzolo]

Yes, I think so. If you want to find if compute or output has to be performed, you'll have to add output_schedule too (you can have output without compute depending on respective frequencies).

The actual SciML callback that we currently add to the integrator already has a cb.condition, and this condition is:

condition = (_, _, _) -> true

This is essentially to hand down the control on what functions have to be run to orchestrate_diagnostics (which uses the compute and output schedules). This is also in part to avoid the inference allocations that come when doing filtering/map/any operations with ScheduleDiagnostics. But this can be changed to a more fine-grained one.

[Sbozzolo]

Here's the relevant code we would change:

ClimaDiagnostics.jl/src/clima_diagnostics.jl

Lines 241 to 257 in 43ca4f1

	"""
	DiagnosticsCallback(diagnostics_handler::DiagnosticsHandler)
	Translate a `DiagnosticsHandler` into a SciML callback ready to be used.
	"""
	function DiagnosticsCallback(diagnostics_handler::DiagnosticsHandler)
	sciml_callback(integrator) =
	orchestrate_diagnostics(integrator, diagnostics_handler)
	# SciMLBase.DiscreteCallback checks if the given condition is true at the end of each
	# step. So, we set a condition that is always true, the callback is called at the end of
	# every step. This callback runs `orchestrate_callbacks`, which manages which
	# diagnostics functions to call
	condition = (_, _, _) -> true
	return SciMLBase.DiscreteCallback(condition, sciml_callback)
	end