Archive for repulsiveness

repulsive sampling

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , , , on January 31, 2024 by xi'an

After a long absence from the monthly Séminaire Parisien de Statistique I attended one today at IHP, including a talk by Diala Hawat on repelled point processes for numerical integration by Hawat et al. The goal is to get (and prove) a universal variance improvement for numerical integration by applying a form of determinantal processes to initial simulations, as eg iid (Poisson process) sampling (without accounting for the O(N²) cost in moving these points). The repelled points are obtain by a single (why single?) move based on a force function (as shown in the slide below), inspired by a Coulomb potential (in the sense that said move appears as one gradient step along the potential). Which reminded me of the pinball sampler, even though the inverse norm was just there to create infinite repulsion near each point. A surprising feature of this repelling step is that it even modifies a (QMC) Sobol process with also an (empirical) improvement in the variance. I wonder if one could construct an MCMC algorithm that would target a joint distribution, maybe via a copula representation, maybe via an equivalent version of HMC.


As an aside, the Bakhvalov results on the existence of a worst case integrand for any deterministic or random sequence (see top slide) made me wonder what the shape of this worst case function is, esp. for a QMC sequence (eg, Sobol). And whether or not they are of any relevance as a counterfactor to the optimal importance functions.

Approximation Methods in Bayesian Analysis [#2]

Posted in pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , on June 22, 2023 by xi'an

A more theoretical Day #2 of the workshop, with Debdeep Pati comparing two representations of Gaussian processes with significantly different efficiencies, and Aad van der Vaart presenting a form of linearisation for a range of inverse problems, Kolyan Ray debiasing Lasso impacts by variational Bayes, although through a somewhat intricate process that distanced the procedure from Bayesian grounds imho, Judith Rousseau (Dauphine) also drifting away from Bayesian canons by looking anew at empirical Bayes with surprising differences from genuine B analysis, connecting with the cutoff phenomenon she and Kerrie exhibited in their 2011 mixture paper, as well as labelling the marginal likelihood a misspecified model. Trevor Campbell and Sinead Williamson both provided Bayesian perspectives on normalising flows, in particular the impact of computer imprecision on reversibility, leading to the notion of shadow paths (screenshot below), while Giovanni Rebaudo talked about mixtures supported by trees, a fascinating object!

On Day #3, Marc Beaumont talked on a mixture of composite likelihood à la Ryden, making me wonder of optimisation of blocks for HMC? EP-ABC, with the issue of the unknown amount of approximation, and adaptivity?, Maria de Iorio presented work on finite and infinite mixtures with repulsive (Coulomb) priors, achieving a unified framework, plus known evidence (?), with a correlated talk by Federico Camerlenghi in the afternoon, with novel notions (for me) of Palm measures and calculus, and another correlated talk by María-Fernanda Gil Leyva Villa, on stick-breaking processes for species sampling with dependent length variables, with related improvements in Gibbs implementation (screenshot below).
This was followed by two theoretical talks on continuous time processes by Paul Jenkins (Warwick) on the fine properties of the Flemming-Viot process, with mentions of Don Dawson’s results reminding me of the 1988 and 1989 summers I spent at Carleton University, where he was located at the time, and Matteo Ruggieri, with the novel (to me) notion of dual Markov processes that could prove useful in a lot of latent variable models. Fabrizio Leisen expanded on his early work on partial exchangeability and Steve MacEachern on dependent quantile pyramids, which relate to quantile regression, a constant source of puzzlement for me. Motivating the perspective by robustness and misspecification arguments. But I am a wee bit puzzled by the distinction between quantile pyramids and other non-parametric solutions.

On the outdoor front (in early mornings), choppy waters at sea (in the Sugiton calanque, pictured above) thanks to the endless mistral wind, nice run down from Mont Puget with friends, limited utility of my rented mountain bike (except to reach the nearest supermarket, 3km away)

repulsive postdoc!

Posted in Statistics with tags , , , , , , , , , , on December 20, 2019 by xi'an

Rémi Bardenet has been awarded an ERC grant on Monte Carlo integration via repulsive point processes and is now looking for a postdoc starting next March. (Our own ABSINT ANR grant still has an open offer of a postdoctoral position on approximate Bayesian methods, feel free to contact me if potentially interested.)

repulsive mixtures

Posted in Books, Statistics with tags , , , , , , , , on April 10, 2017 by xi'an

Fangzheng Xie and Yanxun Xu arXived today a paper on Bayesian repulsive modelling for mixtures. Not that Bayesian modelling is repulsive in any psychological sense, but rather that the components of the mixture are repulsive one against another. The device towards this repulsiveness is to add a penalty term to the original prior such that close means are penalised. (In the spirit of the sugar loaf with water drops represented on the cover of Bayesian Choice that we used in our pinball sampler, repulsiveness being there on the particles of a simulated sample and not on components.) Which means a prior assumption that close covariance matrices are of lesser importance. An interrogation I have has is was why empty components are not excluded as well, but this does not make too much sense in the Dirichlet process formulation of the current paper. And in the finite mixture version the Dirichlet prior on the weights has coefficients less than one.

The paper establishes consistency results for such repulsive priors, both for estimating the distribution itself and the number of components, K, under a collection of assumptions on the distribution, prior, and repulsiveness factors. While I have no mathematical issue with such results, I always wonder at their relevance for a given finite sample from a finite mixture in that they give an impression that the number of components is a perfectly estimable quantity, which it is not (in my opinion!) because of the fluid nature of mixture components and therefore the inevitable impact of prior modelling. (As Larry Wasserman would pound in, mixtures like tequila are evil and should likewise be avoided!)

The implementation of this modelling goes through a “block-collapsed” Gibbs sampler that exploits the latent variable representation (as in our early mixture paper with Jean Diebolt). Which includes the Old Faithful data as an illustration (for which a submission of ours was recently rejected for using too old datasets). And use the logarithm of the conditional predictive ordinate as  an assessment tool, which is a posterior predictive estimated by MCMC, using the data a second time for the fit.