Initialization for models with likelihoods having strictly smaller support than reference

[miguelbiron]

Right now, all of our explorers fail if the initialization state has infinite target logpotential. When this is not due to coding issues, the most likely cause is that the support of the likelihood is a strict subset of the support of the reference. For example, if #264 is not about numerical over/underflow, then it is possible that it is due to this issue (especially given my finding here).

It would be nice to have a wrapper on top of the user-provided initialization that could sample from the reference conditional on the likelihood being strictly positive. The simplest way to do this is via rejection: keep sampling until you find state that gives positive likelihood and return.

More elaborated approaches could be taken; essentially any Monte Carlo algorithm that targets the surrogate

modifiedtarget(x) \propto reference(x) I{likelihood(x) > 0}

would be adequate.

[alexandrebouchard]

Often in high dim the rejection sampler would fail. In Blang we built a sequence of distributions that takes care of the exact zeros too. Would only work if we open the black box a bit more though so this might be a medium term enhancement in my opinion.

[miguelbiron]

That's the thing: doing anything more advanced requires knowing a lot more about the target. Whereas rejection sampling might take 10 seconds of dumb iid sampling but turn a problem from intractable to tractable. Sounds like a freebie?

[alexandrebouchard]

Fair enough, some generic utility would be helpful at least for the "not too hard" initialization problems.

[miguelbiron]

Exactly! It could be the last resort option for black box targets even if we implement better approaches for less black-boxy targets.

[trevorcampbell]

We might want to spit out a warning though, with some statistic, like "X % of your reference draws are being rejected. This indicates that your reference distribution has a wider support than your target, which can result in poor efficiency. Consider changing your reference distribution so that its support matches that of your target."

Make the warning trigger on X > 50 or some such