Here is a proposal for the cover of the mixture book on Mixture Estimation and Applications we are editing, with a stylised rendering of Buachaille Etive Mor… I am not sure it sufficiently fits the topic of the book, other than being taken a few hours and 107 miles from the start of the meeting at ICMS (!), so the cover may end up with a more scientific picture. However, this is my favourite sight when approaching Glencoe, Buachaille rising by itself from Etive Mor, especially with abundant snow and crisp cold sunny skies as I got last March…. Other than those aesthetic considerations, the book is very close to production. I spent Tuesday working on the index and the proofs should be sent to the authors pretty soon.
Archive for mixture estimation
Mixture book cover proposal
Posted in Books, Mountains, pictures, Statistics with tags Buachaille, ICMS, John Wiley, mixture estimation, Scotland on January 24, 2011 by xi'anStatistical Inference
Posted in Books, Statistics, University life with tags Bayes factors, Bayesian Analysis, Bayesian model choice, Dickey-Savage ratio, harmonic mean estimator, joint posterior, likelihood ratio, MCMC, mixture estimation, Pitman nearness on November 16, 2010 by xi'an
Following the publication of several papers on the topic of integrated evidence (about competing models), Murray Aitkin has now published a book entitled Statistical Inference and I have now finished reading it. While I appreciate the effort made by Murray Aitkin to place his theory within a coherent Bayesian framework, I remain unconvinced of the said coherence, for reasons exposed below.
The main chapters of the book are Chapter 2 about the “Integrated Bayes/likelihood approach” and Chapter 4 about the “Unified analysis of finite populations”, Chapter 7 also containing a new proposal about “Goodness of fit and model diagnostics”. Chapter 1 is a nice introduction to frequentist, likelihood and Bayesian approaches to inference and the four remaining chapters are applications of Murray Aitkin‘s principles to various models. The style of the book is quite pleasant although slightly discursive in what I (a Frenchman!) would qualify as an English style in that it is often relying on intuition to develop concepts. I also think that the argument of being close to the frequentist decision (aka the p-value) too often serves as a justification in the book (see, e.g., page 43 “the p-value has a direct interpretation as a posterior probability”). As an aside, Murray Aitkin is a strong believer in plotting cdfs rather than densities to provide information about a distribution and hence cdf plots abound throughout the book. (I counted 82 pictures of them.) While the book contains a helpful array of examples and datasets, the captions of the (many) figures are too terse for my taste: The figures are certainly not self-contained and even with the help of the main text they do not always make complete sense. Continue reading
Monte Carlo Statistical Methods third edition
Posted in Books, R, Statistics, University life with tags Bayesian lasso, hierarchical Bayesian modelling, Introducing Monte Carlo Methods with R, Markov chains, MCMC, mixture estimation, Monte Carlo Statistical Methods, nested sampling, perfect sampling, Peskun ordering, R, Rao-Blackwellisation, regeneration, slice sampling on September 23, 2010 by xi'an
Last week, George Casella and I worked around the clock on starting the third edition of Monte Carlo Statistical Methods by detailing the changes to make and designing the new table of contents. The new edition will not see a revolution in the presentation of the material but rather a more mature perspective on what matters most in statistical simulation:
Typo in mixture survey
Posted in Books, Statistics with tags Bayesian inference, handbook, mixture estimation, typo on September 19, 2010 by xi'anThis morning I received the following email
(…) I have a qu
estion regarding an algorithm in one of your papers, “Bayesian Modelling and Inference on Mixtures of Distributions“. On page 33, in the Metropolis-Hastings algorithm for the mixture you accept the proposal if r < u. As I understand the MH algorithm you accept the proposal with probability r (technically min(r,1)), so I would expect that you accept if u < r. I cannot see or find a reason elsewhere why r < u works? If you could clarify why r < u works for the MH algorithm I would really appreciate it. (…)
which rightly points out an embarrassing typo in our mixture survey, published in the Handbook of Statistics, volume 25. Indeed, the inequality should be the reverse, 
IMIS & AMIS
Posted in R, Statistics, University life with tags adaptive importance sampling, adaptivity, AMIS, defensive mxture, IMIS, incremental mixture, mixture estimation, population Monte Carlo, SIR on July 30, 2010 by xi'an
A most interesting paper by Adrian Raftery and Le Bao appeared in the Early View section of Biometrics. It aims at better predictions for HIV prevalence—in the original UNAIDS implementation, a naïve SIR procedure was used, based on the prior as importance function, which sometimes resulted in terrible degeneracy—, but its methodological input is about incremental mixture importance sampling (IMIS), thus relates to the general topic of adaptive Monte Carlo methods I am interested in. (And to some extent to our recent AMIS paper.) Actually, a less elaborate (and less related) version of the IMIS algorithm first appeared in a 2006 paper by Steele, Raftery and Edmond in JCGS in the setting of finite mixture likelihoods and I somehow managed to miss it…
Raftery and Bao propose to replace SIR with an iterative importance sampling technique developed in 2003 by Steele et al. that has some similarities with population Monte Carlo (PMC). (A negligible misrepresentation of PMC in the current paper is that our method does not use “the prior as importance function'”.) In its current format, the IMIS algorithm starts from a first guess (e.g., the prior distribution) and builds a sequence of Gaussian (or Gaussian mixture) approximations whose parameters are estimated from the current population, while all simulation are merged together at each step, using a mixture stabilising weight
where the weights 
Interestingly, Raftery and Bo (and also Steele et al.) refer to the defensive mixture paper of Hesterberg (1995, Technometrics), which has been very influential in my research on importance sampling, and (less directly) to Owen and Zhou (2000, JASA), who did propose the deterministic mixture scheme that inspired AMIS. Besides the foundational papers of Oh and Berger (1991, JASA) and West (1993, J. Royal Statistical Society Series B), they also mention a paper by Raghavan and Cox (1998, J. Statistical Simulation & Computation) I was not aware of, which introduces as well a mixture of importance proposals as a variance stabilising technique.
