My PhD student Edoardo Bandoni, along with Julien Stoehr and myself, completed a paper on validating (Bayesian) kernel quadrature with unbounded domains of integration. Which connects with probabilistic numerics, since the integrand is modelled as a Gaussian process. And RKHS methods. As opposed to Monte Carlo estimators, quadrature methods approximate integrals of smooth functions with worst-case error decaying at a minimax rate α/d for smoothness α in dimension d. Existing rate-optimal quadrature methods often depend on deterministic point sets tailored to a specific kernel, making them sensitive to misspecification and thus less robust in practice. This paper studies instead randomised quadrature methods, with a focus on robustness rather than on kernel-specific optimality. We construct an explicit, n-dependent, sampling distribution that achieves minimax rates for worst-case errors over smoothness classes without requiring knowledge of the kernel. This kernel-agnostic design does improve robustness while retaining optimal rates and extends Briol et al. (2019) to the unbounded case. Which cannot always be easily handled by a change of variables. Our result thus mostly covers unbounded sampling measures such as Gaussian and Student-t distributions, extending beyond compact domains. The results provide both theoretical guarantees and a practical recipe for robust, rate-optimal, randomised quadrature. [The above is mostly stated in the abstract.]
Archive for Gaussian processes
optimal sampling for kernel quadrature on unbounded domains
Posted in Books, Statistics, University life with tags Bayesian quadrature, bois de Boulogne, contraction rate, ERC Synergy Grant, Gaussian processes, minimaxity, Monte Carlo integration, numerical integration, Ocean, probabilistic integration, probabilistic numerics, RKHS, sunset, uncertainty quantification, Université Paris Dauphine on May 22, 2026 by xi'anoff to [ML@]Aussois
Posted in Books, Kids, Mountains, Running, Statistics, Travel, University life with tags adversarial examples, Aussois, CNRS, conformal prediction, Dent Parrachée, Fall school, Gaussian processes, generative modelling, ML, O'Bayes03, optimization, Paul Langevin, privacy, Savoie, SNCF, train strike, transport, Université Paris Dauphine, Vanoise on November 14, 2025 by xi'an
BayesComp 2025.2
Posted in Kids, pictures, Statistics, Travel, University life with tags appam, Arianna Rosenbluth, BayesComp 2025, Bayesian neural networks, Bayesian robustness, Bayesian semi-parametrics, characteristic function, chili pepper, Chinatown, cut models, durian, equator, exam, Gaussian processes, Gibbs measure, Gibbs posterior, Hamiltonian Monte Carlo, Harvard University, HMC, MASH, Maxwell Hawker market, National University Singapore, NUS, PDMP, Poisson equation, popiah, pseudo-marginal MCMC, rojak, safe Bayes, self-normalised importance sampling, Shanghai, Singapore, stereographic MCMC, stochastic gradient MCMC, street food, unbiased MCMC, Université Paris Dauphine, xiaolongbao, zigzag algorithm on June 19, 2025 by xi'an
The main BayesComp²⁵ conference started with Pierre Jacob’s plenary talk on his recent advances on coupling for unbiased MCMC—currently ERC grantee on that topic—. Raising lazy questions like using a different target or transition kernel for the second chain in the coupling, connecting the Poisson equation and control variates, handling the signed issue with the unbiased approximations. Interestingly, they obtain an unbiased estimator of the asymptotic variance of the unbiased estimator. And a correction for self-normalised importance sampling, which has some connections with our 1996 (?) pinball sampler. Also an evaluation of the median of means, rather than the average of means, which is a thing I had been (lazily) contemplating for a while (On the greedy side, as I was writing my recovery exam for my Monte Carlo course, I realised the results Pierre presented could be somewhat recycled into exam problems!)
My first parallel session was on gradient-based methods with a talk by Francesca Crucinio on proximal particle Langevin algorithms (similar to the one she gave in PariSanté last year), a talk by Zhihao Wang on stereographic multiple try Metropolis(-Rosenbluth-Teller) that unsurprisingly recovers ergodicity thanks to the compactness of the ball. For which I wonder why a Normal proposal makes complete sense since one could consider a mover after the projection instead and why iid rather than repelling multiple proposals are used… The last speaker was just out from the plane from California, Siddharth Vishwanath who spoke about repelling-attracting HMC. With very nice animations of HMC, if reaching the main point of using both negative and positive frictions a few minutes before the session finished. The method preserves volume and potential, if not energy.
Speaking of which (energy), I find myself struggling with my less than 6 hours of sleep since arrival during the first afternoon session, despite a fiery hot spot lunch, which means in plainer terms that I alas dozed in and out of the talks. The second session saw Jack Jewson exposing in deeper details the exact PDMP algorithm for Gibbs measures Jeremias Knoblauch mentioned yesterday. And Jonathan Huggins as well, using Gaussian processes as proxies for expected likelihoods, with lower guarantees than pseudo-marginal versions. In a mildly connected way, Robin Ryder went through the resolution of the ecological inference challenge they produce with Nicolas Chopin and Théo Valdoire (all authors with whom I am connected, Théo being a brillant student of our MASH Master last year and now in Harvard, hopefully till the end of his PhD!)
On the extra-academic curriculum, I had a yummy dinner in the Maxwell Hawker (street) food centre, incl. Xiao Long Bao that cooled down fast enough to avoid the usual scalding effect, plus rojak a mixed fruit and vegetable fried in a peanut sauce that I had never tasted before, popiah (ditto), chili noodles, and an appam with durian deepfried balls as a fabulous and unexpected dessert.
OWABI@BioInference2025 [29 May]
Posted in Mountains, Statistics, Travel, University life with tags Bardonecchia, Bayesian neural networks, BioInference 2025, Gaussian mixture, Gaussian processes, Italy, mixed effect models, mRNA, OWABI, Piedmont, pseudo-marginal MCMC, simulation-based inference, stochastic differential equation, stochastic epidemic compartment models, webinar on May 13, 2025 by xi'an
The next OWABI webinar is going to be quite special, consisting of two selected talks livestreamed from BioInference 2025, a conference on mathematical modelling and inference on (broadly speaking) biological system, taking place in Bardonecchia, Piedmont. The talks will take place on 29 May, 11am CEST (10am BST). The talks will be streamed on the OWABI MS Team Channel as usual.
Speaker: Andrew Golightly (Durham University)
Title: Accelerating Bayesian inference for stochastic epidemic models using incidence data
Abstract: This work considers the case of performing Bayesian inference for stochastic epidemic compartment models, using incomplete time course data consisting of incidence counts that are either the number of new infections or removals in time intervals of fixed length. The most natural Markov jump process representation of the model is eschewed for reasons of computational efficiency, and replaced by a stochastic differential equation representation. This is further approximated to give a tractable Gaussian process, that is, the linear noise approximation (LNA). Unless the observation model linking the LNA to data is both linear and Gaussian, the observed data likelihood remains intractable. Unlike previous approaches that use the LNA in this setting, two approaches for marginalising over the latent process are considered: a correlated pseudo-marginal method and analytic marginalisation via a Gaussian approximation of the noise model. These approaches are compared using synthetic data with the best performing method applied to real data consisting of removal incidence of Oak Processionary moth nests in Richmond Park, London.
2nd OWABI Talk: 10.30-11am
Speaker: Henrik Häggström (Chalmers University)
Title: Simulation-based inference for stochastic nonlinear mixed-effects models with applications in systems biology
Abstract: We propose a novel methodology for Bayesian inference in hierarchical mixed-effects models. By building on our work, we construct a simulation-based inference (SBI) framework that is highly scalable, where amortized approximations to the likelihood and the parameters posterior are first obtained, and these are rapidly refined for each individual dataset, to ultimately approximate the parameters posterior across many individuals. Unlike the current state-of-art SBI methods, which use neural networks, our approximations are expressed via Gaussian mixture models, leading to easily trainable, parsimonious yet expressive surrogate models of both the likelihood function and the posterior distribution. The methodology is exemplified via stochastic differential equation mixed-effects models to describe translation kinetics after mRNA transfection, however the methodology is general and can accommodate other types of stochastic and deterministic models. We compare our approximate inference with exact pseudomarginal inference and show that our methodology is fast and competitive.
6th Workshop on Sequential Monte Carlo Methods
Posted in Mountains, pictures, Statistics, Travel, University life with tags #ERCSyG, Arthur's Seat, auxiliary particle filter, Bayes Centre, Da Vinci Code, Edinburgh, Gaussian processes, generative models, gradient algorithm, Hamiltonian, ICMS, MALA, maximal coupling, multilevel Monte Carlo, neural network, Ocean, ODE, particle filter, Pentland Hills, public transportation, robots, Rosslyn Chapel, Scotland, SMC, SMC 2024, snippet, trail running on May 16, 2024 by xi'an
Very glad to be back to an SMC workshop as it has been nine years since my attending SMC 2015 in Malakoff! The more for the workshop taking place in Edinburgh and at the Bayes Centre. It is one of these places where I feel somewhat returning to familiar grounds with accumulated memories. Like my last visit there when I had a tea with Mike Titterington…
The overall pace of the workshop was quite nice, with long breaks for informal discussions (and time for ‘oggin’!) and interesting poster late afternoons, helped by the small number of them at each instance, incl. one on reversible jump HMC. Here are a few scribbled entries about some talks along the first two days.
After my opening talk (!), Joaquín Míguez talked about the impact of a sequential (Euler-Marayama) discretisation scheme for stochastic differential equations on Bayesian filtering with control of the approximation effect. Axel Finke (in a joint work with Adrien Corenflos, now an ERC Ocean postdoc in Warwick) built a sequence of particle filter algorithms targeting good performances (high expected jumping distance) against both large dimensions and high time horizon, exploiting gradient shift MALA-like, as well as prior impact, with the conclusion that their jack-of-all-trades solutions, Particle-MALA and Particle-mGRAD, enjoyed this resistance in nearly normal models. Interesting reminder of the auxiliary particle trick and good insights on using the smoothing target, even when accounting for the computing time, but too many versions for a single talk without checking against the preprint.
The SMC sampler-like algorithm involves propagating N “seed” particles z(i), with a mutation mechanism consisting of the generation of N integrator snippets 𝗓:=(z,ψ(z),ψ²(z),…) started at every seed particle z(i), resulting in N×(T+1) particles which are then whittled down to a set of N seed particles using a standard resampling scheme. Andrieu et al., 2024
Christophe Andrieu talked about Monte Carlo sampling with integrator snippets, starting with recycling solutions for the leapfrog integrator HMC and unfolding Hamiltonians for moving more easily. With snippets representing discretised paths along the level sets being used as particles, picking zero, one, or more particles along each path, since importance weights are connection with multinomial HMC
This relatively small algorithmic modification of the conditional particle filter, which we call the conditional backward sampling particle filter has a dramatically improved performance over the conditional particle filter. Karjalainen et al., 2024
Anthony Lee looked at mixing times for backward sampling SMC (CBPF/ancestor sampling) cf Lee et al. (2020), where the backward step consists in computing the weight of a randomly drawn backward or ancestral history. Improving on earlier results to reach mixing time O(log T) and complexity O(T log T) (with T the time horizon). Thanks to maximal coupling and boundedness assumptions on the prior and likelihood functions.
Neil Chada presented a work on Bayesian multilevel Monte Carlo on deep networks. À la Giles, with a telescoping identity. Always puzzling to envision a prior on all parameters of a neural network. Achieving a computational cost inverse to the order of the MSE, at best. With a useful reminder that pushing the size of the NN to infinity results in a (poor) Gaussian process prior (Sell et al., 2023).
On my first evening, I stopped with a friend in my favourite Blonde [restaurant], as in almost every other visit to Edinburgh, enjoyable as always, but I also found the huge offer of Asian minimarkets in the area too tempting to resist, between Indian, Korean, and Chinese products. (Although with a disappointing hojicha!). As I could not reach any new Munro by train or bus within a reasonable time range I resorted to the nearer Pentland Hills, with a stop by Rosslyn Chapel (mostly of Da Vinci Code fame!, if classic enough). And some delays in finding a bus getting there (misled by google map!) and a trail (misled by my poor map reading skills) up the actual hills. The mist did not help either.
