Archive for advanced Monte Carlo methods

mostly Monte Carlo [09/10, PSC]

Posted in Statistics, University life with tags , , , , , , , , , , , , on October 4, 2026 by xi'an

The next episode of our mostly Monte Carlo seminar is next Friday (9 October) at PariSanté Campus (room #8) with speakers

15:00 – Víctor Elvira, University of Edinburgh

16:00 – Edoardo Bandoni, Université Paris Dauphine-PSL

Víctor Elvira, “Rethinking self-normalized importance sampling”

Self-normalized importance sampling (SNIS) is one of the most widely used Monte Carlo techniques for inference with unnormalized target distributions. Despite its usefulness, SNIS is often viewed simply as a normalized version of ordinary importance sampling, and many of its methodological questions remain largely unexplored. In this talk, we revisit SNIS from a unified perspective. We first introduce a generalized formulation of self-normalized importance sampling based on coupled proposals, showing that the classical SNIS estimator is only one member of a broader family of Monte Carlo estimators with new opportunities for variance reduction. We then consider the classical SNIS estimator and present adaptive algorithms that learn proposals tailored to its optimal proposal distribution, together with theoretical guarantees including consistency, asymptotic normality, and convergence of the proposal. Together, these developments suggest that self-normalized importance sampling should be regarded as a distinct Monte Carlo methodology, with its own theory, optimality principles, and algorithmic design.

Edoardo Bandoni, “Rate-Optimal Randomised Kernel Quadrature”

Kernel quadrature is widely used to approximate integrals of smooth functions, with the worst-case error typically decaying at the minimax rate n-α/d for smoothness α in dimension d. Existing rate-optimal methods often depend on deterministic point sets tailored to a specific kernel, making them sensitive to misspecification and less robust in practice. In this work, we study randomised quadrature methods with a focus on robustness rather than kernel-specific optimality. By minimising a tractable upper bound on the worst-case error, we obtain an explicit sampling distribution p*∝ πg with g=2d/(2α+d), which depends on the integration density π and on a but not on the kernel beyond its Sobolev order. Under a weak doubling condition on the design measure, independent samples from p* attain the minimax rate n-α/d. These assumptions cover a broad class of targets on compact and unbounded domains; we verify them explicitly for Beta-type densities, Gaussian measures, and Student-t distributions, the last of which yields the minimax rate n-min(α,(n+d/2)/d. This kernel-agnostic design improves robustness while maintaining optimal rates, and it applies beyond compact domains. The results provide both theoretical guarantees and a practical recipe for robust, rate-optimal randomised quadrature.

on the edge and online!

Posted in Books, Statistics, Travel, University life with tags , , , , , , , , , , , on March 11, 2024 by xi'an

afternoon on Bayesian computation

Posted in Statistics, Travel, University life with tags , , , , , , , , , , , , , on April 6, 2016 by xi'an

Richard Everitt organises an afternoon workshop on Bayesian computation in Reading, UK, on April 19, the day before the Estimating Constant workshop in Warwick, following a successful afternoon last year. Here is the programme:

1230-1315  Antonietta Mira, Università della Svizzera italiana
1315-1345  Ingmar Schuster, Université Paris-Dauphine
1345-1415  Francois-Xavier Briol, University of Warwick
1415-1445  Jack Baker, University of Lancaster
1445-1515  Alexander Mihailov, University of Reading
1515-1545  Coffee break
1545-1630  Arnaud Doucet, University of Oxford
1630-1700  Philip Maybank, University of Reading
1700-1730  Elske van der Vaart, University of Reading
1730-1800  Reham Badawy, Aston University
1815-late  Pub and food (SCR, UoR campus)

and the general abstract:

The Bayesian approach to statistical inference has seen major successes in the past twenty years, finding application in many areas of science, engineering, finance and elsewhere. The main drivers of these successes were developments in Monte Carlo methods and the wide availability of desktop computers. More recently, the use of standard Monte Carlo methods has become infeasible due the size and complexity of data now available. This has been countered by the development of next-generation Monte Carlo techniques, which are the topic of this meeting.

The meeting takes place in the Nike Lecture Theatre, Agriculture Building [building number 59].

point process-based Monte Carlo

Posted in Books, Kids, Statistics, University life with tags , , , , , on December 3, 2015 by xi'an

Clément Walter from Paris just pointed me to an arXived paper he had very recently gotten accepted for publication in Statistics and Computing. (Congrats!) Because his paper relates to nested sampling. And connects it with rare event simulation via interacting particle systems. And multilevel Monte Carlo. I had missed it when it came out on arXiv last December [as the title was unrelated with nested sampling if not Monte Carlo], but the paper brings fairly interesting new results about an ideal version of nested sampling that is

  1. unbiased when using an infinite number of terms;
  2. always better than the standard Monte Carlo estimator, variance-wise;
  3. connected with an implicit marked Poisson process; and
  4. enjoying a finite variance provided the quantity of interest has an 1+ε moment.

Of course, such results only hold for an ideal version and do not address the issue of the conditional simulations required by nested sampling. (Which has an impact on the computing time as the conditional simulation becomes more and more expensive as the likelihood value increases.) The explanation therein of the approximation of tail probabilities by a Poisson estimate makes the link with deterministic nested sampling much clearer to me. Point 2 above means that the nested sampling estimate always does better than the average of the likelihood values produced by an iid or MCMC simulation from the prior distribution. The paper also borrows from the debiasing approach of Rhee and Glynn (already used by the Russian roulette) to turn truncated versions of the nested sampling estimator into an unbiased estimator, with a limited impact on the variance of the estimator. Truncation is associated with the generation of a geometric stopping time which parameter needs to be optimised. Without a more detailed reading, I am somewhat lost as to this optimisation remains feasible in complex settings… The paper contains an illustration for a Pareto distribution where optimisation and calibration can be conducted quite far. It also re-analyses the Mexican hat example of Skilling (2006), showing that our stopping rule may induce bias.

reading classics (The End)

Posted in Books, Kids, Statistics, University life with tags , , , , , , , , , on February 24, 2015 by xi'an

La Défense from Paris-Dauphine, Nov. 15, 2012Today was the final session of our Reading Classics Seminar for the academic year 2014-2015. I have not reported on this seminar much so far because it has had starting problems, namely hardly any student present on the first classes and therefore several re-starts until we reached a small group of interested students. And this is truly The End for this enjoyable experiment as this is the final year for my TSI Master at Paris-Dauphine, as it will become integrated within the new MASH Master next year.

As a last presentation for the entire series, my student picked John Skilling’s Nested Sampling, not that it was in my list of “classics”, but he had worked on the paper in a summer project and was thus reasonably fluent with the topic. As he did a good enough job (!), here are his slides.

Some of the questions that came to me during the talk were on how to run nested sampling sequentially, both in the data and in the number of simulated points, and on incorporating more deterministic moves in order to remove some of the Monte Carlo variability. I was about to ask about (!) the Hamiltonian version of nested sampling but then he mentioned his last summer internship on this very topic! I also realised during that talk that the formula (for positive random variables)

\int_0^\infty(1-F(x))\text{d}x = \mathbb{E}_F[X]

does not require absolute continuity of the distribution F.