svgd incomplete implementation

[EtayLivne]

Adding the SVGD algorithm to Posteriors. See issue #98

This is a stub, as I'm pretty sure there are gaps in the implementation. As such, the test is only a sanity validation that you can create a step and call update on it.

What I know I'm probably not/mis - understanding is:

• I'm not sure about the calculation of the gradient of log probability - I "solved this" by requesting it to be a user provided function, but which dummy one should I use for the tests?
• is the phi_star calculation correct?

Also in general, I'd appreciate a hint about what / how to test.

[EtayLivne]

Forgot to mentions - I did not include some features of the API such as passing auxileries to the kernel function, because I wanted to deliver an inital offering that's as simple to read as possible, so that fixing the issues with the core logic would be easier. I do plan to add them in once the core logic is implemented!

[SamDuffield]

This looks a good start!! Thanks for doing it!

Regarding your points

• We enforce a particular structure for log_posterior functions in posteriors, see the docs. Then we can get gradients with grad_and_value(log_posterior, has_aux=True) see e.g. the SGLD implementation.
• Regarding phi_star, I think a simple test that checks a very simple example against some clear (perhaps inefficient code, e.g. for loop) would be sufficient to check its correct.

For more tests you can check out the other tests (e.g. SGLD), I think testing that the mean and covariance of a Gaussian posterior is recovered (as well as all shapes are correct and above phi_star correctness etc) would be sufficient.

I think we will want to generalise the kernel API, but as you say good to get this correct first 😄