Archive for entropy

incoming mostly Monte Carlo [14 April, PariSanté campus]

Posted in pictures, Statistics, University life with tags , , , , , , , , , , , , , , , on April 9, 2026 by xi'an

The next Mostly Monte Carlo seminar will be this very Friday, 10/04/26, at PariSanté Campus. With Shiva Darshan and Pierre Monmarché speaking on the following topics:
15h: Shiva Darshan Maximal-reflection couplings on manifolds: some specific examples
Explicit Markovian couplings can be used to build Markov Chain Monte Carlo methods such unbiased MCMC or coupling based control variates. For sampling from probability measures supported on Euclidean space, one typically uses a synchronous coupling, a maximal-reflection coupling (also known as a discrete-time sticky coupling), or some variant of the two. For probability measures supported on Riemannian manifolds, the situation is less clear cut. While the Kendall-Cranston coupling of Brownian motions on manifolds has been successfully applied in theoretical works, it is ill-suited for building explicit algorithms. In this talk, we will discuss some of the obstacles to extending Euclidean maximal-reflection couplings to manifolds and present some special cases for which these obstacles can be easily overcome. With applications to Stereographic MCMC in mind, we detail particular couplings of random walks on the sphere.
16h: Pierre Monmarché A post-sampling reweighting method for multi-modal target measures
Even when the modes are identified and sampled locally with MCMC methods, a difficulty to sample multi-modal measures is to correctly estimate the relative probabilities of each of these modes, which requires to observe many transitions between them (which are rare events). We will present an approach based on variational inference which exploits the local samples, aiming only at estimating the relative weights between them. When the modes are well separated, this amount to some entropy estimations.

connection between tempering & entropic mirror descent

Posted in Books, pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , , , , on April 30, 2024 by xi'an

The next One World ABC webinar is this  Thursday,  the 2nd May, at 9am UK time, with Francesca Crucinio (King’s College London, formerly CREST and even more formerly Warwick) presenting

“A connection between Tempering and Entropic Mirror Descent”.

a joint work with Nicolas Chopin and Anna Korba (both from CREST) whose abstract follows:

This work explores the connections between tempering (for Sequential Monte Carlo; SMC) and entropic mirror descent to sample from a target probability distribution whose unnormalized density is known. We establish that tempering SMC corresponds to entropic mirror descent applied to the reverse Kullback-Leibler (KL) divergence and obtain convergence rates for the tempering iterates. Our result motivates the tempering iterates from an optimization point of view, showing that tempering can be seen as a descent scheme of the KL divergence with respect to the Fisher-Rao geometry, in contrast to Langevin dynamics that perform descent of the KL with respect to the Wasserstein-2 geometry. We exploit the connection between tempering and mirror descent iterates to justify common practices in SMC and derive adaptive tempering rules that improve over other alternative benchmarks in the literature.

ABC with privacy

Posted in Books, Statistics with tags , , , , , , , , on April 18, 2023 by xi'an


I very recently read a  2021 paper by Mijung Park, Margarita Vinaroz, and Wittawat Jitkrittum on running ABC while ensuring data privacy (published in Entropy).

“…adding noise to the distance computed on the real observations and pseudo-data suffices the privacy guarantee of the resulting  posterior samples”

For ABC tolerance, they use maximum mean discrepancy (MMD) and for privacy the standard if unconvincing notion of differential privacy, defined by ensuring an upper bound on the amount of variation in the probability ratio when replacing/removing/adding an observation. (But not clearly convincing users their data is secure.)

While I have no reservation about the validation of the double-noise approach, I find it surprising that noise must be (twice) added when vanilla ABC is already (i) noisy, since based on random pseudo-data, and (ii) producing only a sample from an approximate posterior instead of returning an exact posterior. My impression indeed was that ABC should be good enough by itself to achieve privacy protection. In the sense that the accepted parameter values were those that generated random samples sufficiently close to the actual data, hence not only compatible with the true data, but also producing artificial datasets that are close enough to the data. Presumably these artificial datasets should not be produced as the intersection of their ε neighbourhoods may prove enough to identify the actual data. (The proposed algorithm does return all generated datasets.) Instead the supported algorithm involves randomisation of both tolerance ε and distance ρ to the observed data (with the side issue that they may become negative since the noise is Laplace).

approximate Bayesian inference [survey]

Posted in Statistics with tags , , , , , , , , , , , , , , , , , , on May 3, 2021 by xi'an

In connection with the special issue of Entropy I mentioned a while ago, Pierre Alquier (formerly of CREST) has written an introduction to the topic of approximate Bayesian inference that is worth advertising (and freely-available as well). Its reference list is particularly relevant. (The deadline for submissions is 21 June,)

special issue of Entropy

Posted in Statistics with tags , , , , , on September 11, 2020 by xi'an