Archive for Bhattacharyya distance

Bhattacharyya distance versus Kullback-Leibler divergence

Posted in Books, Kids, Statistics with tags , , , , on January 10, 2015 by xi'an

Another question I picked on Cross Validated during the Yule break is about the connection between the Bhattacharyya distance and the Kullback-Leibler divergence, i.e.,

d_B(p,q)=-\log\left\{\int\sqrt{p(x)q(x)}\,\text{d}x\right\}

and

d_{KL}(p\|q)=\int\log\left\{{q(x)}\big/{p(x)}\right\}\,p(x)\,\text{d}x

Although this Bhattacharyya distance sounds close to the Hellinger distance,

d_H(p,q)=\left\{1-\int\sqrt{p(x)q(x)}\,\text{d}x\right\}^{1/2}

the ordering I got by a simple Jensen inequality is

d_{KL}(p\|q)\ge2d_B(p,q)\ge2d_H(p,q)^2\,.

and I wonder how useful this ordering could be…

1 Comment »