a novel discrepancy measure

My friend EJ Wagenmakers, along with his colleague Raoul Grasman, have proposed a novel measure of discrepancy between two distributions that they formulate through a basic Bayesian lens as an expected posterior probability, written as

$D_{\mathrm EP}(f||g) = \int_{\mathfrak X} \frac{f(x)}{f(x)+g(x)}f(x)\,\text dx$

(with no priors harmed in the process!) in case both distributions are absolutely continuous wrt a common dominating measure, with densities f and g. Which I have not met before. This discrepancy has nicer properties than Kullback-Leibler in that it is symmetric, that

$D_{\mathrm EP}(f||g) = \int_{\mathfrak X}\frac{1}{f(x)^{-1}+g(x)^{-1}}\frac{f(x)}{g(x)}\,\text dx$

does not require absolute continuity, and suffers less from tail influence and from asymmetric convergence speeds between null and alternative. From a truly Bayesian perspective, comparing two posterior predictive densities this way is less appealing since they are not available in closed form, but the posterior probability in favour of model f can be replaced with a Monte Carlo estimate.

This entry was posted on June 1, 2025 at 12:25 am and is filed under Statistics, University life with tags Amsterdam, Bayesian model choice, discrepancies, E.J. Wagenmakers, Kullback-Leibler divergence, University of Amsterdam. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Xi'an's Og