Issue 559: Diagnostic analysis over all inference runs

[SamuelBrand1]

This PR closes #559

This PR adds an analysis function which returns details around the MCMC convergence:

• Proportion of all parameters get less than a bulk ess of 400 (tunable).
• Proportion of all parameters get less than a tail ess of 100 (tunable).
• Proportion of all parameters with an R hat more than 0.02 away from 1.
• The tail ess of the cluster factor for neg bin sampling. Special attention paid to this parameter because its a likely candidate for identifiability problems.

The PR also adds a script to run this across all inferences, and adds recording successes to the success/fail analysis.

[github-actions]

Try this Pull Request!

Open Julia and type:

import Pkg
Pkg.activate(temp=true)
Pkg.add(url="https://github.com/CDCgov/Rt-without-renewal", rev="559-report-for-mcmc-quality", subdir="EpiAware")
using EpiAware

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.96%. Comparing base (ab73d9e) to head (09264f0).
Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #560   +/-   ##
=======================================
  Coverage   90.96%   90.96%           
=======================================
  Files          60       60           
  Lines         863      863           
=======================================
  Hits          785      785           
  Misses         78       78           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[seabbs]

This looks great. It would be nice to add a test to show the core mcmc diagnostic functions works in and of itself. I think it would also be good for the future to add some convergence summaries beyond the thresholds (I think having a figure on this was planned in the analysis plan but it can be a follow up issue/PR as its not on the critical path for this feature

[SamuelBrand1]

I think this is ready for rereview: less DRY, unit test for the mcmc_convergence and more stats.

[SamuelBrand1]

I've gone for:

• proportion pass (relative to threshold)
• mean
• min
• max

NB r hat is presented as abs diff from 1.0 e.g a mean of 0.001 is good not bad!

[seabbs]

Nice looks good to me!