My friend EJ Wagenmakers, along with his colleague Raoul Grasman, have proposed a novel measure of discrepancy between two distributions that they formulate through a basic Bayesian lens as an expected posterior probability, written as
(with no priors harmed in the process!) in case both distributions are absolutely continuous wrt a common dominating measure, with densities f and g. Which I have not met before. This discrepancy has nicer properties than Kullback-Leibler in that it is symmetric, that
does not require absolute continuity, and suffers less from tail influence and from asymmetric convergence speeds between null and alternative. From a truly Bayesian perspective, comparing two posterior predictive densities this way is less appealing since they are not available in closed form, but the posterior probability in favour of model f can be replaced with a Monte Carlo estimate.
