My PhD student Edoardo Bandoni, along with Julien Stoehr and myself, completed a paper on validating (Bayesian) kernel quadrature with unbounded domains of integration. Which connects with probabilistic numerics, since the integrand is modelled as a Gaussian process. And RKHS methods. As opposed to Monte Carlo estimators, quadrature methods approximate integrals of smooth functions with worst-case error decaying at a minimax rate α/d for smoothness α in dimension d. Existing rate-optimal quadrature methods often depend on deterministic point sets tailored to a specific kernel, making them sensitive to misspecification and thus less robust in practice. This paper studies instead randomised quadrature methods, with a focus on robustness rather than on kernel-specific optimality. We construct an explicit, n-dependent, sampling distribution that achieves minimax rates for worst-case errors over smoothness classes without requiring knowledge of the kernel. This kernel-agnostic design does improve robustness while retaining optimal rates and extends Briol et al. (2019) to the unbounded case. Which cannot always be easily handled by a change of variables. Our result thus mostly covers unbounded sampling measures such as Gaussian and Student-t distributions, extending beyond compact domains. The results provide both theoretical guarantees and a practical recipe for robust, rate-optimal, randomised quadrature. [The above is mostly stated in the abstract.]
Archive for numerical integration
optimal sampling for kernel quadrature on unbounded domains
Posted in Books, Statistics, University life with tags Bayesian quadrature, bois de Boulogne, contraction rate, ERC Synergy Grant, Gaussian processes, minimaxity, Monte Carlo integration, numerical integration, Ocean, probabilistic integration, probabilistic numerics, RKHS, sunset, uncertainty quantification, Université Paris Dauphine on May 22, 2026 by xi'anrepulsive sampling
Posted in Books, Statistics, University life with tags Coulomb potential, determinantal point process, HMC, IHP, importance sampling, Institut Henri Poincaré, MCMC algorithms, Nikolai Bakhvalov, numerical integration, pinball sampler, quasi-Monte Carlo methods, repulsiveness, simulating copulas, slides, Sobol sequences, variance reduction on January 31, 2024 by xi'an
After a long absence from the monthly Séminaire Parisien de Statistique I attended one today at IHP, including a talk by Diala Hawat on repelled point processes for numerical integration by Hawat et al. The goal is to get (and prove) a universal variance improvement for numerical integration by applying a form of determinantal processes to initial simulations, as eg iid (Poisson process) sampling (without accounting for the O(N²) cost in moving these points). The repelled points are obtain by a single (why single?) move based on a force function (as shown in the slide below), inspired by a Coulomb potential (in the sense that said move appears as one gradient step along the potential). Which reminded me of the pinball sampler, even though the inverse norm was just there to create infinite repulsion near each point. A surprising feature of this repelling step is that it even modifies a (QMC) Sobol process with also an (empirical) improvement in the variance. I wonder if one could construct an MCMC algorithm that would target a joint distribution, maybe via a copula representation, maybe via an equivalent version of HMC.
As an aside, the Bakhvalov results on the existence of a worst case integrand for any deterministic or random sequence (see top slide) made me wonder what the shape of this worst case function is, esp. for a QMC sequence (eg, Sobol). And whether or not they are of any relevance as a counterfactor to the optimal importance functions.
simulating Gumbel’s bivariate exponential distribution
Posted in Books, Kids, R, Statistics with tags accept-reject algorithm, cross validated, Emil Julius Gumbel, exponential distribution, gamma distribution, Gumbel distribution, Happy New Year, importance sampling, numerical integration, simulating copulas, variance reduction on January 14, 2024 by xi'an
A challenge interesting enough for a sunny New Year morn, found on X validated, namely the simulation of a bivariate exponential distribution proposed by Gumbel in 1960, with density over the positive quadrant in IR²
![{}^{ [(\lambda_2+rx_1)(\lambda_1+rx_2)-r]\exp[-(\lambda_1x_1+\lambda_2x_2+rx_1x_2)]}](https://s0.wp.com/latex.php?latex=%7B%7D%5E%7B+%5B%28%5Clambda_2%2Brx_1%29%28%5Clambda_1%2Brx_2%29-r%5D%5Cexp%5B-%28%5Clambda_1x_1%2B%5Clambda_2x_2%2Brx_1x_2%29%5D%7D&bg=000000&fg=B0B0B0&s=0&c=20201002)
Although there exists a direct approach based on the fact that the marginals are Exponential distributions and the conditionals signed mixtures of Gamma distributions, an accept-reject algorithm is also available for the pair, with a dominating density representing a genuine mixture of four Gammas, when omitting the X product in the exponential and the negative r in the first term. The efficiency of this accept-reject algorithm is high for r small. However, and in more direct connection with the original question, using this approach to integrate the function equal to the product of the pair, as considered in the original paper of Gumbel, is much less efficient than seeking a quasi-optimal importance function, since this importance function is yet another mixture of four Gammas that produces a much reduced variance at a cheaper cost!
inferring the number of components [remotely]
Posted in Statistics with tags 8 Mile, Bayes factor, Bayesian consistency, Bayesian inference, COVID-19, Dirichlet process mixture, Eminem, evidence, finite mixtures, Italian Alps, Italy, mixture models, numerical integration, RJMCMC, slides, slideshare, University of Padova, unknown number of components on October 14, 2022 by xi'an
approximative Laplace
Posted in Books, R, Statistics with tags cross validated, Laplace approximation, Monte Carlo Statistical Methods, numerical integration, typos on August 18, 2018 by xi'an
I came across this question on X validated that wondered about one of our examples in Monte Carlo Statistical Methods. We have included a section on Laplace approximations in the Monte Carlo integration chapter, with a bit of reluctance on my side as this type of integral approximation does not directly connect to Monte Carlo methods. Even less in the case of the example as we aimed at replacing a coverage probability for a Gamma distribution with a formal Laplace approximation. Formal due to the lack of asymptotics, besides the length of the interval (a,b) which probability is approximated. Hence, on top of the typos, the point of the example is not crystal clear, in that it does not show much more than the step-function approximation to the function converges as the interval length gets to zero. For instance, using instead a flat approximation produces an almost as good approximation:
> xact(5,2,7,9) [1] 0.1933414 > laplace(5,2,7,9) [1] 0.1933507 > flat(5,2,7,9) [1] 0.1953668
What may be more surprising is the resilience of the approximation as the width of the interval increases:
> xact(5,2,5,11) [1] 0.53366 > lapl(5,2,5,11) [1] 0.5354954 > plain(5,2,5,11) [1] 0.5861004 > quad(5,2,5,11) [1] 0.434131
