Archive for simulated annealing

bridging ratio estimators

Posted in Books, Statistics, University life with tags , , , , , , , , , , on June 3, 2025 by xi'an

more than mostly MC

Posted in Kids, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , , , , on December 27, 2024 by xi'an

The session of last Friday (and last one of 2024!) proved most interesting, with two (fully) Monte Carlo talks. The first one by Louis Grenioux was about an improvement on diffusion sampler, following a recent arXival by Maxime Noble and co-authors. Which brought me back to the long-standing multimodal challenge in Monte Carlo methods. When all modes of a target distribution are known, even roughly, this is not much of an issue since samplers can be arm-bent into visiting all these modes. But the problem becomes much harder when the location and a fortiori the number of modes are not known. The paper aims at adapting diffusion samplers towards a better exploration of the modes, albeit their location is known. Otherwise, using MCMC as a starting (reference) distribution would risk missing some of them. Which also explains why the authors can rely on the classical Gaussian mixtures proposal as a cheap substitute to neural networks (EBM). Since, within diffusion models, both intermediary distributions and their scores are intractable, they also introduce a variational parametric approximation that can be optimised.

The second talk was given by Guillaume Chennetier, in connection with his recent PhD thesis, developing a form of X-entropy sampling for rare events. As in nuclear plant major accidents. It took me a while to realise that PDMPs were not used as a simulation tool, as in the zigzag sampler and its avatars, The proposal involved creating a graph structure on the space of PDMP trajectories and designing the optimal importance process (yes, the one with zero variance!) using so-called committor functions that modify jump intensity and kernel, in a sequential way reminiscent of X-entropy. The approach recycles past trajectories as Monte Carlo elements if missing an adaptive mixture importance sampling (AMIS!) version that would bring more stability. The talk also included an interesting pointer to the availability of the distribution of the PDMP path, thus treated as a likelihood. (!). The signage at the entrance of the Monte-Carlo (mind the hyphen!) casino also made an appearance, reminding me of our memorable group picture on the same spot, eons ago! (But not of whom took the picture!)

à la journée François Perron

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , on October 23, 2024 by xi'an

Wonderful day celebrating my long-term friend François Perron’s career, at the Université de Montréal! In full autumnal glory! With several long-time-no-see friends attending and presenting their work and connecting with François’ achievements, incl. Éric Marchand on Stein prediction for spherically symmetric distributions, where the predicted vector y appears in the same quadratic form as the observed one x, with the same location parameter, a setting that turns prediction into estimation of that parameter, a 2019 paper with Dominique Fourdrinier and Bill Strawderman,A
Mylène Bédard on a generalised optimal scaling of annealed MALA, with higher speeds in transient cases than known so far,t. Yves Atchadé on cyclical MCMC, a rejection free version of Radford Neal’s parallel tempering, that sometimes fails to converge
and Alex Bouchard-Côté on escaping the curse of dimensionality by interpolations of the target, which is a variant of simulated tempering (quite the theme of the day!), and happened to be his invited lecture for the 2024 CRM-SSC prize, awarded right after. Unfortunately I had to catch my plane and face the notorious jams to the airport. (And, fortunately, I had attended Saifuddin Syed’s related talk at Mostly MC the week before!)

PIPLA [mostly MCMC’nar]

Posted in Books, pictures, Statistics, University life with tags , , , , , , , , , , , , , on September 27, 2024 by xi'an

The first “mostly MCMC ” seminar (Season 2) had our new Ocean postdoc Tim Johnston, freshly graduated from the University of Edinburgh, involved in both talks, with proximal approximations for discontinuity! The first talk was given by Francesca Crucinio (formerly Warwick and formerly CREST, to point out potential COI!!), about the Proximal Interacting Particle Langevin algorithm (PIPLA) developed with her coauthors Paula Cordero Encinar, Deniz Akyildiz, Tim Johnston, and Mark Girolami, concerned with  maximising likelihoods with latent variables (i.e., an EM setting).  While offering one of many stochastic versions of EM, incl. simulated annealing, the solution they adopt very close to our SAME (2002) method, with duplicating latent variables N times to get near the marginal MAP (which as we noted differs from the join MAP). They start from interacting particle system with (unadjusted) Langevin dynamics, discretised over time, but the value of N does not move with iterations, which steps away from the simulated annealing motivation, thus requiring an evaluation of the error for a given N and possibly further runs with larger values of N. PIPLA is an extension of the above to non-differentiable targets, by using a proximity map, in continuation of MY-ULA [for Moreau-Yoshida] by Pereyra (2016), yet again fixing both N and the proximal parameter λ. With non-asymptotic convergence results requiring strong assumptions on the target.

In his talk, Tim started with interesting (and novel for me) arguments for proving strong convergence (Wasserstein, multilevel MC, unbiased MCMC), proceeding to establishing (again under favourable assumptions, and almost √n convergence speed for the proximal scheme with no regularity assumption on drift besides boundedness.