reciprocal importance sampling
In a recent arXival, Metodiev et al. (including my friend Adrian Raftery, who is spending the academic year in Paris) proposed a new version of reciprocal importance sampling, expanding the proposal we made with Darren Wraith (2009) of using a Uniform over an HPD region. It is called THAMES, hence the picture (of London, not Paris!), for truncated harmonic mean estimator.
“…[Robert and Wraith (2009)] method has not yet been fully developed for realistic, higher-dimensional situations. For example, we know of no simple way to compute the volume of the convex hull of a set of points in higher dimensions.”
They suggest replacing the convex hull of the HPD points with an ellipsoid ϒ derived from a Normal distribution centred at the highest of the HPD points, whose covariance matrix is estimated from the whole (?) posterior sample. Which is somewhat surprising in that this ellipsoid may as well included low probability regions when the posterior is multimodal. For instance, the estimator is biased when the posterior cancels on parts of ϒ. And with an unclear fate for the finiteness of its variance, depending on how fast the posterior gets to zero on these parts.
The central feature of the paper is selecting the radius of the ellipse that minimises the variance of the (counter) evidence. Under asymptotic normality of the posterior. This radius roughly corresponds to our HPD region in that 50% of the sample stands within. The authors also notice that separate samples should be used to estimate the ellipse and to estimate the evidence. And that a correction is necessary when the posterior support is restricted. (Examples do not include multimodal targets, apparently.)
June 4, 2023 at 7:28 pm
Correction: the integral in our equation (4) should be finite for most commonly used models (no need for boundedness of the unnormalized posterior density).
June 4, 2023 at 3:18 pm
Thanks for the good comments. It was a pleasure to try to develop your excellent idea from 2009. You say that there is “an unclear fate for the finiteness of its variance”. Actually, I think that finiteness of the variance of THAMES is guaranteed by our choice in most cases. The Reciprocal Importance Sampling estimator in general has finite variance if equation (4) in our paper holds. The function h(theta) that we propose is zero outside a compact set, and so the integral in our (4) should be finite as long as the unnormalized posterior density is bounded, which holds for most commonly used models.