GPU accelerated HMC

[yebai]

The following paper implements a GPU accelerated version of NUTS, and shows some nice speedups on a logistic regression model.

Tran, Dustin, Matthew W. Hoffman, Dave Moore, Christopher Suter, Srinivas Vasudevan, and Alexey Radul. "Simple, distributed, and accelerated probabilistic programming." In Advances in Neural Information Processing Systems, pp. 7598-7609. 2018.
http://papers.nips.cc/paper/7987-simple-distributed-and-accelerated-probabilistic-programming

Can we try to run the vectorised HMC in #117 on the same model, and check the speedups?

[xukai92]

Just sync the understanding. The paper has NUTS on GPUs but not the batch-mode multiple chain stuff in #117 right?

[yebai]

It runs one chain, with the log density being parallelised on multiple CPUs and GPUs.

We only need to compare between vectorised HMC and non-vectorised HMC on GPU and CPU I think. I listed the paper because it is a related work.

[xukai92]

Related DynamicHMC.jl issue: tpapp/DynamicHMC.jl#110

We could also check the difference between AHMC and DHMC on GPU using the example there.

[yebai]

This is already supported.