Archive for MCMC algorithms

All about that [Bayes] seminar [24 Jan]

Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , on January 13, 2025 by xi'an

The next All about that (Bayes) seminar will take place on Friday 24 Jan at SCAI, on the Jussieu campus, with the following talks. (Appearances to the contrary, I was not in the least involved in the program!)

13h30 – 14h30 Joshua Bon (OCEAN, Université Paris Dauphine) – Bayesian score calibration for approximate models

 Scientists continue to develop increasingly complex mechanistic models to reflect their knowledge more realistically. Statistical inference using these models can be challenging since the corresponding likelihood function is often intractable and model simulation may be computationally burdensome. Fortunately, in many of these situations, it is possible to adopt a surrogate model or approximate likelihood function. It may be convenient to conduct Bayesian inference directly with the surrogate, but this can result in bias and poor uncertainty quantification. In this paper (https://arxiv.org/abs/2211.05357) we propose a new method for adjusting approximate posterior samples to reduce bias and produce more accurate uncertainty quantification. We do this by optimizing a transform of the approximate posterior that maximizes a scoring rule. Our approach requires only a (fixed) small number of complex model simulations and is numerically stable. We demonstrate beneficial corrections to several approximate posteriors using our method on several examples of increasing complexity.

14h30 – 15h30 Giacomo Zanella (Bocconi University) – Entropy contraction of the Gibbs sampler under log-concavity

In this talk I will present recent work (https://arxiv.org/abs/2410.00858) on the non-asymptotic analysis of the Gibbs sampler, a classical and popular MCMC algorithm for sampling. In particular, under the assumption that the probability measure π of interest is strongly log-concave, we show that the random scan Gibbs sampler contracts in relative entropy, and provide a sharp characterization of the associated contraction rate. The result implies that, under appropriate conditions, the number of full evaluations of π required for the Gibbs sampler to converge is independent of the dimension. If time permits, I will also discuss connections and applications of the above results to the problem of zero-order parallel sampling, as well as extensions to Hit-and-Run and Metropolis-within-Gibbs.

Based on joint work with Filippo Ascolani and Hugo Lavenant.

16h00 – 17h00 Paul Bastide (Université Paris Cité) – Goodness of Fit for Bayesian Generative Models with Applications in Population Genetics

In population genetics, inference about intractable likelihood models is common, and simulation methods, including Approximate Bayesian Computation (ABC) and Simulation-Based Inference (SBI), are essential. ABC/SBI methods work by simulating instrumental data sets of the models under study and comparing them with the observed data set y⁰. Advanced machine learning tools are used for tasks such as model selection and parameter inference. The present work focuses on model criticism. This type of analysis, called goodness of fit (GoF), is important for model validation. It can also be used for model pruning when the number of candidates to be considered is excessive, especially in the context where data simulation is expensive. We introduce two new GoF tests based on the local outlier factor (LOF), an indicator that was initially defined for outlier and novelty detection. We test whether y⁰ is distributed from the prior predictive distribution (pre-inference GoF) and whether there is a parameter value such that y⁰ is distributed from the likelihood with that value (post-inference GoF).  We evaluate the performance of our two GoF tests on simulated datasets from three different model settings of varying complexity, and on a dataset of single nucleotide polymorphism (SNP) markers for the evaluation of complex evolutionary scenarios of modern human populations.

Joint work with Guillaume Le Mailloux, Jean-Michel Marin and Arnaud Estoup.

slice sampling, of course!

Posted in Books, pictures, Statistics with tags , , , , , , , on November 19, 2024 by xi'an

NobAIl prAIzes

Posted in Books, pictures, Travel, University life with tags , , , , , , , , , , , , , , , , , , , , , , , on October 11, 2024 by xi'an

I am quite surprised that the NobAIl committee did not select ChatGPT‘s Sam Altman for their literature prize—since he made a reality of monkeys typing at random on a typewriter, hence bound to a.s. produce past and future masterpieces—, now betting on Twitter’s Dorsey, Williams and Stone for their peace prize— for achieving worldwide harmony if within each single-minded community in only 140 characters—, and Amazon’s Jeff Bezos for their Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel—for produing the ultimate monopoly of a single worldwide convenience store—, given their picks for physics—with spin glass Ising models that have always sounded to me like the worst possible illustration for MCMC techniques— and chemistry—for a deep learning predictor of protein structures, built by large teams and numerous CPU hours, achieving high success rates if not perfection, but prediction is not explanation, reminding me of Nietzsche’s “physics, too, is only an interpretation and exegesis of the world (to suit us, if I may say so!) and not a world-explanation”—… Keeping their sharp focus on AI’s, corporate funded research, and male recipients… (As coïncidences come, I am currently reading a book of the literature 2024 Nobel recipient, Han Kang, The Vegetarian, that I find stunning and immensely original!)

repulsive sampling

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , , , on January 31, 2024 by xi'an

After a long absence from the monthly Séminaire Parisien de Statistique I attended one today at IHP, including a talk by Diala Hawat on repelled point processes for numerical integration by Hawat et al. The goal is to get (and prove) a universal variance improvement for numerical integration by applying a form of determinantal processes to initial simulations, as eg iid (Poisson process) sampling (without accounting for the O(N²) cost in moving these points). The repelled points are obtain by a single (why single?) move based on a force function (as shown in the slide below), inspired by a Coulomb potential (in the sense that said move appears as one gradient step along the potential). Which reminded me of the pinball sampler, even though the inverse norm was just there to create infinite repulsion near each point. A surprising feature of this repelling step is that it even modifies a (QMC) Sobol process with also an (empirical) improvement in the variance. I wonder if one could construct an MCMC algorithm that would target a joint distribution, maybe via a copula representation, maybe via an equivalent version of HMC.


As an aside, the Bakhvalov results on the existence of a worst case integrand for any deterministic or random sequence (see top slide) made me wonder what the shape of this worst case function is, esp. for a QMC sequence (eg, Sobol). And whether or not they are of any relevance as a counterfactor to the optimal importance functions.

MCMC postdoc positions at Bocconi

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , , on January 17, 2023 by xi'an

[A call for postdoc candidates to work in Milano with Giacomo Zanella in the coming years under ERC funding. In case you are interested with a postdoctoral position with me at Paris Dauphine on multi-agent decision-making, data sharing, and fusion algorithms, do not hesitate to contact me, the official call for applications should come up soon!]

Three postdoc positions available at Bocconi University (Milan, Italy), under the supervision of Giacomo Zanella and funded by the ERC Starting Grant “Provable Scalability for high-dimensional Bayesian Learning”. Details and links to apply available online.

The deadline for application is 28/02/2023 and the planned starting date is 01/05/2023 (with some flexibility). Initial contracts are for 1 year and are extendable for further years under mutual agreement.

Candidates will conduct research on computational aspects of statistical and machine learning methods, with a particular focus on Bayesian methodologies. The research activity, both in terms of specific topic and research approach, can adapt to the profile and interests of the successful candidates. Beyond working with the supervisor and coauthors on topics related to the grant project (see here and there for more details on the research topics of the supervisor and grant project), candidates will get the chance to interact with various faculty members, postdocs and PhD students of the Stats&ML group at Bocconi (see e.g. researchers at Bocconi).

Interested candidates can write to giacomo zanella at unibocconi for more information about the positions.