report condition number of metric for HMC adaptive samplers

[mitzimorris]

Summary:

Extend the functionality added in #3230 to return the Hessian and the condition number of the Hessian at the end of adaptation.

Description:

The Stan User's Guide chapter on efficiency tuning describes what the condition number is, but this is not exposed in the Stan services layer.

Now that it is possible to output the metric in a JSON file, we should be able to provide functionality available in BridgeStan https://github.com/roualdes/bridgestan/blob/36d391a6fc8ac8a8ddc901a66df56e9e1b73fb13/src/bridgestan.cpp#L184-L206
to get the Hessian, compute the Eigenvalues, and get the ratio of largest/smallest.

We should do this for Optimization, ADVI, and Pathfinder as well - Pathfinder's diagnostics files would allow this.
The other algorithms would need to add more output writers.

Expected Output:

Additional fields added to output file metric.json.

Current Version:

v2.34.1

[mitzimorris]

Alternatively, we could write a stand-alone utility for CmdStan, which given a sample, computes the condition number - like diagnose or stansummary.

[bob-carpenter]

This would be a nice diagnostic.

How are you proposing to compute Hessians? We can only get autodiff Hessians for our analytical functions---the implicit functions aren't implemented with forward-mode autodiff. We can get finite difference Hessians everywhere, but that's expensive computationally and only gives about half the accuracy (which could be a problem when computing condition).

[WardBrian]

@andrjohns implemented a nice way of doing finite differences only for the functions which don't have full forward mode support: stan-dev/math#2929

This is essentially a "best of both worlds" approach. It looks like only integrate_1d was actually added to the framework, but I see no reason why the remaining implicit functions couldn't all be.

[bob-carpenter]

That PR from @andrjohns is really amazing. That's a great way to handle these cases. I agree that the other ones should be added.