Wonderful day celebrating my long-term friend François Perron’s career, at the Université de Montréal! In full autumnal glory! 
With several long-time-no-see friends attending and presenting their work and connecting with François’ achievements, incl. Éric Marchand on Stein prediction for spherically symmetric distributions, where the predicted vector y appears in the same quadratic form as the observed one x, with the same location parameter, a setting that turns prediction into estimation of that parameter, a 2019 paper with Dominique Fourdrinier and Bill Strawderman,A
Mylène Bédard on a generalised optimal scaling of annealed MALA, with higher speeds in transient cases than known so far,t.
Yves Atchadé on cyclical MCMC, a rejection free version of Radford Neal’s parallel tempering, that sometimes fails to converge
and Alex Bouchard-Côté on escaping the curse of dimensionality by interpolations of the target, which is a variant of simulated tempering (quite the theme of the day!), and happened to be his invited lecture for the 2024 CRM-SSC prize, awarded right after. Unfortunately I had to catch my plane and face the notorious jams to the airport. (And, fortunately, I had attended Saifuddin Syed’s related talk at Mostly MC the week before!)
Archive for parallel tempering
à la journée François Perron
Posted in pictures, Statistics, Travel, University life with tags Autumn, Canada, CRM, CRM-SSC Prize, MALA, maple tree, Mont Royal, Montréal, parallel tempering, Québec, simulated annealing, traffic jam, UBC, Université de Montréal, William Strawderman, Yves Atchadé on October 23, 2024 by xi'anre-MCM’d
Posted in Statistics, University life with tags adaptive MCMC, alma mater, badge, conference, doubly intractable posterior, Grande Mosquée de Paris, HMC, Jardin des Plantes, Jussieu, MCM 2023, parallel tempering, Paris, PDMP, slice sampler, Sorbonne Université, U-turn on June 28, 2023 by xi'an
When I entered the classroom on the Jussieu campus where the Monday morning session on PDMP was taking place, some friends told me a badge was waiting for me at the registration desk of MCM 2023! Most surprisingly since I had received a deregistration message a few weeks earlier. Great PDMP session with non-reversible tempering (warning: some PDMPs were harmed in the tempering process), scaling PDMPs for highly anisotropic targets, a PDMP form of reversible jump (with sticky floors!) and a more adaptive version of no-U turn via… PDMPs. After lunch at the nearby Grande Mosquée de Paris (I had not visited since… 1974!), I attended the slice sampler session, where mileage varied imho. With a take on doubly-intractable targets that did not seem to relate to the existing literature.
21w5107 [day 1]
Posted in pictures, Statistics, Travel, University life with tags 21w5107, Día de los Muertos, fireworks, Fisher information, foundations of objective Bayesian methodology, ICML 2021, Jeffreys priors, MCMC convergence, Mexican Revolution day, Mexico, mixtures of distributions, Oaxaca, parallel tempering, random number generator, Santa María del Tule on November 30, 2021 by xi'an
The workshop started by the bad news of our friend Michele Guindani being hit and mugged upon arrival in Oaxaca, Saturday night. Fortunately, he was not hurt, but lost both phone and wallet, always a major bummer when abroad… Still this did not cast a lasting pall on the gathering of long-time no-see friends, whom I had indeed not seen for at least two years. Except for those who came to the CIRMirror!
A few hours later, we got woken up by fairly loud firecrackers (palomas? cohetes?) at 5am, for no reason I can fathom (the Mexican Revolution day was a week ago) although it seemed correlated with the nearby church bells going on at full blast (for Lauds? Hanukkah? Cyber Monday? Chirac’s birthdate?). The above picture was taken the Santa María del Tule town with its super-massive Montezuma cypress tree, with remaining decorations from the Día de los Muertos.
Without launching (much) the debate on whether or not Bayesian non-parametrics qualified as “objective Bayesian” methods, Igor Prünster started the day with a non-parametric presentation of dependent random probability measures. With the always fascinating notion that a random discrete non-parametric prior is inducing a distribution on the partitions (EPPF). And applicability in mixtures and their generalisations. Realising that the highly discrete nature of such measures is not such an issue for a given sample size n, since there are at most n elements in the partition. Beatrice Franzolini discussed of specific ways to create dependent distributions based on independent samples, although her practical example based on one N(-10,1) sample and another (independently) N(10,1) sample seemed to fit in several of the dependent random measures she compared. And Marta Catalano (Warwick) presented her work on partial exchangeability and optimal transportation (which I had also heard in CIRM last June and in Warwick last week). One thing I had not realised earlier was the dependence of the Wasserstein distance on the parameterisation, although it now makes perfect sense. If only for the coupling. I had alas to miss Isadora Antoniano-Villalobos’ talk as I had to teach my undergrad class in Paris Dauphine at the same time… This non-parametric session was quite homogeneous and rich in perspectives.
In an all-MCMC afternoon, Julyan Arbel talked about reference priors for extreme value distributions, with the “shocking” case of a restriction on the support of one parameter, ξ. Which means in fact that the Jeffreys prior is then undefined. This reminded me somewhat of the work of Clara Grazian on Jeffreys priors for mixtures, where some models were not allowing for Fisher information to exist. The second part of this talk was about modified local versions of Gelman & Rubin (1992) R hats. And the recent modification proposed by Aki and co-authors. Where I thought that a simplification of the multivariate challenge of defining ranks could be alleviated by considering directly the likelihood values of the chains. And Trevor Campbell gradually built an involved parallel tempering method where the powers of a geometric mixture are optimised as spline functions of the temperature. Next, María Gil-Leyva presented her original and ordered approach to mixture estimation, which I discussed in a blog published two days ago (!). She corrected my impressions that (i) the methods were all impervious to label switching and (ii) required some conjugacy to operate. The final talk of the day was by Anirban Bhattacharya on high-D Bayesian regression and coupling techniques for checking convergence, a paper that had been on my reading list for a long while. A very elaborate construct of coupling strategies within a Gibbs sampler, with some steps relying on optimal coupling and others on the use of common random generators.
parallel tempering on optimised paths
Posted in Statistics with tags astrostatistics, Bayesian optimisation, Bézier curves, black holes, British Columbia, Burrard Inlet, Canada, Gaussian mixture, imaging, parallel tempering, splines, tempering, UBC, University of British Columbia, Vancouver, Vancouver Island on May 20, 2021 by xi'an
Saifuddin Syed, Vittorio Romaniello, Trevor Campbell, and Alexandre Bouchard-Côté, whom I met and discussed with on my “last” trip to UBC, on December 2019, just arXived a paper on parallel tempering (PT), making the choice of tempering path an optimisation problem. They address the touchy issue of designing a sequence of tempered targets when the starting distribution π⁰, eg the prior, and the final distribution π¹, eg the posterior, are hugely different, eg almost singular.
“…theoretical analysis of reversible variants of PT has shown that adding too many intermediate chains can actually deteriorate performance (…) [while] on non reversible regime adding more chains is guaranteed to improve performances.”
The above applies to geometric combinations of π⁰ and π¹. Which “suffers from an arbitrarily suboptimal global communication barrier“, according to the authors (although the counterexample is not completely convincing since π⁰ and π¹ share the same variance). They propose a more non-linear form of tempering with constraints on the dependence of the powers on the temperature t∈(0,1). Defining the global communication barrier as an average over temperatures of the rejection rate, the path characteristics (e.g., the coefficients of a spline function) can then be optimised in terms of this objective. And the temperature schedule is derived from the fact that the non-asymptotic round trip rate is maximized when the rejection rates are all equal. (As a side item, the technique exposed in the earlier tempering paper by Syed et al. was recently exploited for a night high resolution imaging of a black hole from the M87 galaxy.)
ABC, anytime!
Posted in Books, pictures, Statistics, Travel, University life with tags ABC, arXiv, asynchronous algorithms, parallel tempering on January 18, 2021 by xi'an
Last June, Alix Marie d’Avigneau, Sumeet Singh, and Lawrence Murray arXived a paper on anytime ABC I intended to review right away but that sat till now on my virtual desk (and pile of to-cover-arXivals!). The notion of anytime MCMC was already covered in earlier ‘Og entries, but this anytime ABC version bypasses the problem of asynchronicity, namely, “randomly varying local move completion times when parallel tempering is implemented on a multi-processor computing resource”. The different temperatures are replaced by different tolerances in ABC. Since switches between tolerances are natural if a proposal for a given tolerance ε happens to be eligible for a lower tolerance ε’. And accounting for the different durations required to simulate a proposal under different tolerances to avoid the induced bias in the stationary distributions. Or the wait for other processors to complete their task. A drawback with the approach stands in calibrating the tolerance levels in advance (or via preliminary runs that may prove costly).
