At Sylvia Richardson’s career celebration last Friday, I gave a talk on How many components in a mixture? which was most relevant given Sylvia’s contributions to mixture inference over the years, including her highly influential 1997 Read Paper with Peter Green. The other talks highlighted the many facets of Sylvia to the field and the profession, including obviously her MRC Unit directorship but also her RSS Presidency when she drove along with Chris Holmes the remarkable society’s response to the COVID pandemic. The day ended up with a diner in Emmanuel College (in the Dining Hall rather than the larger, noisier, and more formal Hall where I was once invited for the Midsummer Dinner by Sylvia). It was also a great opportunity to reconnect with friends I had not seen for ages.
Archive for RJMCMC
Festschift for Sylvia
Posted in Books, pictures, Statistics, Travel, University life with tags Cam river, Cambridge colleges, Dirichlet process Gaussian mixture, distributed computing, Emmanuel College, Festschrift, finite mixtures, Midsummer, MRC Biostatistics Unit, RJMCMC, statistical evidence, Sylvia Richardson on May 17, 2023 by xi'ana passage to & from India
Posted in pictures, Running, Statistics, Travel, University life with tags Bangalore, Bengaluru, Flybus, IISA, Indian Institute of Science, International Indian Statistical Association, jatp, Karnataka, Mysore, Mysuru, poster session, Restore, RJMCMC, swimming pool, Tata Institute, train travel on January 10, 2023 by xi'an
Our trip from Paris (CDG) to Bengaluru got a wee bit (!) perturbed by 2x bad luck, with a first plane grounded for damages to a wing and a second plane flashing an alarm signal just as it was accelerating to take off, which induced an extra hour of tests, plus an unexpected long wait to get the e-visa at the Bengalore airport, resulting in an arrival in town at 5:30 am! A good thing that my talk was only the next day.
I was glad to be back at the (Tata) Indian Institute of Science and its wonderful campus for the IISA meeting (taking place alternately in India and in the US). The conference program was rich and with a large Bayesian component, but being sleep deprived and slightly sick did not help with my concentration during the talks… Had however nice discussions during the poster session, including one on a most unusual RJMCMC where the model-to-model transform was the identity. In a sense this voided (?) the need for RJMCMC, but it allowed for a fast & valid exploration of the different models.
Quite a contrast in my local lodging conditions, when compared with my previous visit, since, rather than staying in the ideal visitors’ lodge located at the centre of the campus, I took the (bargain) offer (from IISA) of the nearby Sheraton (!) as the conference hotel with five star conditions, including a proper, outside, empty and non-heated swimming pool.
The (touristy) train trip to Mysore was most pleasant, on an air-conditioned carriage with food vendors proposing their wares all along the journey, great views of the countryside and an arrival sharp on time. The reverse trip to the airport was less successful as the FlyBus we took was crawling rather than flying, with heavy traffic all the way because/despite being New Year Eve’ning. At some point, a truck carrying what looked like kindling was stuck in a pothole, blocking the highway, and a crane was brought on site to push the truck out of the hole, a strategy that surprisingly worked. But we managed to reach the airport just before midnight, when absolutely nothing happened in relation with the entry into 2023! 
transport, diffusions, and sampling
Posted in pictures, Statistics, Travel, University life with tags Barker's algorithm, Biometrika, diffusions, ELBO, Empire State building, Flatiron Institute, Gelman-Rubin statistic, Hyvärinen score, mural, nanoparticles, New York city, normalising flow, optimal transport, Pareto smoothed importance sampling, R² criterion, RJMCMC, sampling, Simmons Foundation, stacking, Statistical Science, street art, what you get is what you see, workshop on November 19, 2022 by xi'an
At the Sampling, Transport, and Diffusions workshop at the Flatiron Institute, on Day #2, Marilou Gabrié (École Polytechnique) gave the second introductory lecture on merging sampling and normalising flows targeting the target distribution, when driven by a divergence criterion like KL, that only requires the shape of the target density. I first wondered about ergodicity guarantees in simultaneous MCMC and map training due to the adaptation of the flow but the update of the map only depends on the current particle cloud in (8). From an MCMC perspective, it sounds somewhat paradoxical to see the independent sampler making such an unexpected come-back when considering that no insider information is available about the (complex) posterior to drive the [what-you-get-is-what-you-see] construction of the transport map. However, the proposed approach superposed local (random-walk like) and global (transport) proposals in Algorithm 1.
Qiang Liu followed on learning transport maps, with the Interesting notion of causalizing a graph by removing intersections (which are impossible for an ODE, as discussed by Eric Vanden-Eijden’s talk yesterday) through coupling. Which underlies his notion of rectified flows. Possibly connecting with the next lightning talk by Jonathan Weare on spurious modes created by a variational Monte Carlo sampler and the use of stochastic gradient, corrected by (case-dependent?) regularisation.
Then came a whole series of MCMC talks!
Sam Livingstone spoke on Barker’s proposal (an incoming Biometrika paper!) as part of a general class of transforms g of the MH ratio, using jump processes based on a nasty normalising constant related with g (tractable for the original Barker algorithm). I then realised I had missed his StatSci paper on how to speak to statistical physics researchers!
Charles Margossian spoke about using a massive number of short parallel runs (many-short-chain regime) from a recent paper written with Aki, Andrew, and Lionel Riou-Durand (Warwick) among others. Which brings us back to the challenge of producing convergence diagnostics and precisely the Gelman-Rubin R statistic or its recent nR avatar (with its linear limitations and dependence on parameterisation, as opposed to fuller distributional criteria). The core of the approach is in using blocks of GPUs to improve and speed-up the estimation of the between-chain variance. (D for R².) I still wonder at a waste of simulations / computing power resulting from stopping the runs almost immediately after warm-up is over, since reaching the stationary regime or an approximation thereof should be exploited more efficiently. (Starting from a minimal discrepancy sample would also improve efficiency.)
Lu Zhang also talked on the issue of cutting down warmup, presenting a paper co-authored with Bob, Andrew, and Aki, recommending Laplace / variational approximations for reaching faster high-posterior-density regions, using an algorithm called Pathfinder that relies on ELBO checks to counter poor performances of Laplace approximations. In the spirit of the workshop, it could be profitable to further transform / push-forward the outcome by a transport map.
Yuling Yao (of stacking and Pareto smoothing fame!) gave an original and challenging (in a positive sense) talk on the many ways of bridging densities [linked with the remark he shared with me the day before] and their statistical significance. Questioning our usual reliance on arithmetic or geometric mixtures. Ignoring computational issues, selecting a bridging pattern sounds not different from choosing a parameterised family of embedding distributions. This new typology of models can then be endowed with properties that are more or less appealing. (Occurences of the Hyvärinen score and our mixtestin perspective in the talk!)
inferring the number of components [remotely]
Posted in Statistics with tags 8 Mile, Bayes factor, Bayesian consistency, Bayesian inference, COVID-19, Dirichlet process mixture, Eminem, evidence, finite mixtures, Italian Alps, Italy, mixture models, numerical integration, RJMCMC, slides, slideshare, University of Padova, unknown number of components on October 14, 2022 by xi'an
common derivation for Metropolis–Hastings and other MCMC algorithms
Posted in Books, pictures, Statistics, Travel, University life with tags auxiliary variables, directional sampling, Gibbs sampling, Hamiltonian Monte Carlo, Metropolis-Hastings algorithms, Metropolis-within-Gibbs algorithm, NUTS, pseudo-marginal MCMC, recursive proposals, RJMCMC, slice sampling, Sydney, UNSW on July 25, 2016 by xi'anKhoa Tran and Robert Kohn from UNSW just arXived a paper on a comprehensive derivation of a large range of MCMC algorithms, beyond Metropolis-Hastings. The idea is to decompose the MCMC move into
- a random completion of the current value θ into V;
- a deterministic move T from (θ,V) to (ξ,W), where only ξ matters.
If this sounds like a new version of Peter Green’s completion at the core of his 1995 RJMCMC algorithm, it is be
cause it is indeed essentially the same notion. The resort to this completion allows for a standard form of the Metropolis-Hastings algorithm, which leads to the correct stationary distribution if T is self-inverse. This representation covers Metropolis-Hastings algorithms, Gibbs sampling, Metropolis-within-Gibbs and auxiliary variables methods, slice sampling, recursive proposals, directional sampling, Langevin and Hamiltonian Monte Carlo, NUTS sampling, pseudo-marginal Metropolis-Hastings algorithms, and pseudo-marginal Hamiltonian Monte Carlo, as discussed by the authors. Given this representation of the Markov chain through a random transform, I wonder if Peter Glynn’s trick mentioned in the previous post on retrospective Monte Carlo applies in this generic setting (as it could considerably improve convergence…)
