Bounded target support
Here is an interesting question from Tomàs that echoes a lot of related emails:
I’m turning to you for advice. I’m facing problem where parameter space is bounded, e.g. all parameters have to be positive.
If in MCMC as proposal distribution I use normal distribution, then at some iterations I get negative proposals. So my question is: should I use recalculation of acceptance probability every time I reject the proposal (something like in delayed rejection method), or I have to use another proposal (like lognormal, trancated normal, etc.)?
It is indeed a popular belief that something needs to be done to counteract restricted supports. However, there is no mathematical reason for doing so! Consider the following illustration
[sourcecode language=”r” gutter=”false”]
target=function(x) (x>0)*(x<1)*dnorm(x,mean=4)
mcmc=rep(0.5,1e5)
for (t in 2:1e5){
prop=mcmc[t-1]+rnorm(1,.1)
if (runif(1)<target(prop)/target(mcmc[t-1]))
mcmc[t]=prop
else
mcmc[t]=mcmc[t-1]
}
hist(mcmc,prob=TRUE,col="wheat",border=FALSE,main="",xlab="")
curve(dnorm(x-4)/(pnorm(-3)-pnorm(-4)),add=TRUE,col="sienna",lwd=2)
[/sourcecode]
and the following outcome, with a perfect fit!
July 5, 2011 at 5:17 pm
You may want to highlight that you check for out-of-support in the evaluation of the target density. What may puzzle those readers is that they try to evaluate the target at the given point, without checking for the support first: think (in pseudocode)
return N(x; mu, sigma)
when sigma is negative, instead of
if sigma < 0
return 0
else
return N(x; mu, sigma)
end
Otherwise stated, in more classical Bayesian terms: short-circuit the likelihood evaluation if the prior density is null.