MCMC and conditioning

Here is an interesting question from Tomas Iesmantas:

I have a question related to conditional decomposition in Bayesian hierarchical models. I’m dealing with a pretty nasty model, which involves nonlinearities. The hierarchy is made of two levels of parameters. After quite long calculations I derived the marginal distribution (up to a proportionality constant) for the first level parameters (I have 40 parameters there). Then I have the posterior density (up to a proportionality constant) for the second level parameters conditioned on the first level parameters (i.e. a conditional posterior distribution for the second level parameters).

Am I right to simulate by MCMC from that [first level] marginal distribution and then to use the simulation to obtain a sample from the [second level] conditional distribution? I know that it would be valid to simulate from the marginal of the second level and then to use the sample for the conditional distribution of the first level. But I feel that somehow it is related to de Finetti’s exchangeability notion. I’m confused. Hope that you could point me to the right direction.

My reply was that, indeed, this implementation is correct. While we distinguish first level from second level for modelling purposes, they all are the same from a probabilistic perspective. In other words,

$\pi(\theta_1,\theta_2|\mathcal{D})=\pi(\theta_1|\theta_2,\mathcal{D}) \pi(\theta_2|\mathcal{D}) = \pi(\theta_2|\theta_1) \pi(\theta_1|\mathcal{D})$

(with hopefully obvious notations). If the marginal posterior for the first level parameters is available, MCMC can be directly applied to this posterior and simulations of the second level parameters are obtained “exactly” by simulating from the true full conditional posterior distribution. It can even wait till the MCMC simulation is over. (And I do not think de Finetti’s exchangeability matters for this property.)

This entry was posted on July 27, 2011 at 12:11 am and is filed under Statistics, University life with tags hierarchical Bayesian modelling, MCMC, Monte Carlo Statistical Methods. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Xi'an's Og