Archive for insufficient statistic

Bayesian, adversarial, oceanic, privacy

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , , , , , , , , , , , on March 6, 2026 by xi'an

We just arXived a new paper on Bayesian privacy! We meaning Cameron Bell, Antoine Luciano, Timothy Johnston and myself, as members of my ERC OCEAN lab at PariSanté and Paris Dauphine. While sharing the same ground as my recent paper with James Bailie, Joshua Bon and Judith Rousseau, this one is definitely more mainstream Bayesian in that the entire decision process falls under the Bayesian hat, with the ultimate decision being the choice of the release mechanism by the data holder (or hoarder!). To rationalise this decision process, we break the framework as resulting from the actions of three actors, namely the data holder, Alice, the data scientist, Bob, and the eavesdropper. Eve. (As in my earlier posts on solving Le Monde’s math puzzles, we could have used pronouns from other cultures, but I feared this would have confused some of the readers. Incidentally, I found out that the earliest use of the first two pronouns was within the groundbreaking cryptography 1977 paper of Rivest, Shamir and Adleman, bringing the RSA algorithm to the World! With Eve appearing in an early, highly-cited privacy paper by Montréal’s Bennett, Brassard, and (unconnected to me!) Robert, in 1988.)

We thus consider a Bayesian setting in which, given data x, held by Alice, inference is to be performed by Bob on a parameter θ. Performing such inference requires Alice releasing information derived from x, which may contain sensitive content, exploited by Eve. Our approach is to compare Alice’s release mechanisms according to both the quality of inference on θ (from Bob’s viewpoint) and the privacy leakage regarding x (sought by Eve and dreaded by Alice). To formalise this evaluation, we posit that Alice refers to a loss function that is a linear combination of Bob’s and Eve’s losses, the weight on Eve’s loss being then negative. (An alternative to be considered in future work is Alice using a ratio of Bob’s and Eve’s losses, possibly set to different powers, the rationale being that a zero loss for Eve is intolerable for Alice.) As in Bayesian experimental design, a prior on the data is necessary for Eve to infer on the hidden data based on the release mechanism and released output and for Alice to evaluate the risk of said release mechanism . (They may differ, as long as they are both made public.) To calibrate Alice’s loss, we opted for a balance that returns the same risk for a full data release and a total lack of release. In specific, informed, settings, other weights could be chosen. While finding the optimal release strategy is impossible but for highly discrete settings, the framework obviously allows for the ranking of natural strategies like insufficient statistics and synthetic datasets. Comments welcome!

congrats, Doctor Luciano!

Posted in Books, Kids, Statistics, University life with tags , , , , , , , , , , on January 23, 2026 by xi'an

tenets of quantile-based inference in Bayesian models

Posted in Books, Statistics with tags , , , , , , , , , , , , , , on June 8, 2025 by xi'an

This 2023 paper of Perepolkin, Goodrich, and Sahlin vaguely relates to our insufficient Gibbs work in that a Bayesian analysis is conducted based solely on quantile summaries. Except that here the input is the entire cdf, or the—inverse cdf—quantile function, or—its derivative—the quantile density function, instead of the probability density function—used as the likelihood in the posterior. Which is a non-problem from a mathematical perspective since all these functions describe the same probability distribution. Which makes the following quote rather puzzling (in its obviousness).

“We aim to show that the quantile-based Bayesian inference using the intermediate depths leads to the same posterior beliefs as the conventional density-based inference.”

The authors still make a big case of the difference, obviously, to the point of proposing a different notation for Y~F. But using the same symbol f for different densities. The formal expression of the posterior based on the quantile function actually requires the cdf function and the density or the quantile density to be available (at least in a numerical sense), witness eqn (11).

The paper still could hold some interest in its computational component. Relating to ABC, obviously, since distributions defined by quantiles and cdfs often come as benchmarks for ABC, when the associated pdf/likelihood is unavailable. Witness g-and-k distributions (with the caveat MCMC can be implemented in this case). Unfortunately, the paper entirely relies on numerical inversion (with a puzzling comment that MCMC rejection gets higher with numerical inversion, p6). And only mentions ABC in the conclusion, possibly to pacify a referee’s comment. And actually consider that “quantile parameterized quantile distributions don’t lend themselves easily as sampling distributions due to the special nature of their parameterization” (p4). Hence making me wonder at the overall relevance of the entire endeavour….

IISA 2024

Posted in Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , on January 12, 2025 by xi'an

The IISA 2024 conference was held at the Cochin University of Science and Technology (CUSAT), where talks and posters took place. Among the sessions I attended, I attended a  talk on colliding random walks that made three or more  impossible in the limit, missed the only privacy talk that I could have attended by Vinayak Rao for indulging into a dawnish swimming session in the very decent hotel pool. In the first session I organised, loosely connected with BNP, Antonietta Mira presented a recent work on predicting EU carbon compensation rates, using a information imbalance rank substitute to correlation that could prove quite interesting in privacy settings, albeit no invariant to reparameterisations (with potential connections with Wasserstein distances), Sonia Petrone gave an overview on her substantial amount of work on empirical Bayes in Bayes (EBIB), making me wonder at natural ABC or EM ways to bypass the computation of the empirical Bayes hyperparameter computation (but also on measuring the overfitting degree of EBIB-ing), maybe exploiting the representation of the marginal as an average of predictive, and Debdeep Pati argued towards an interpretable and robust ML, using for estimation divergence a mixture of two KL’s that is remindful of GANs (as well as of Lasso and exponentially tilted empirical likelihood à la Chib & al.), implemented by a form of Bbootstrap. Plus enjoying a locally flavoured acronym, namely BETEL.
The second session I organised was centred on Bayesian computations, with both Sid Chib and Ritabrata Dutta (U Warwick) presenting a Bayesian modelling of difference-in-difference models in clinical trials and several works on (score based) generalized Bayes models with Shreya Roy (U Warwick), who further won a poster prize at IISA 2024. I terminated the session with a talk on our insufficient Gibbs sampler, which connected with some aspects of both Sid’s and Rito’s talks.

As in earlier editions of IISA I attended, the local organisation was most enjoyable, from supportive staff and students, relaxed atmosphere, easy commutes, heaps of great food, and unlimited chai! Plus meeting and listening to participants I had met in these earlier editions. Looking forward the 2026 edition!

Insufficient Gibbs sampling [at COMPSTAT 2024]

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , on August 29, 2024 by xi'an


The COMPSTAT 2024 programme proved somewhat remote from my interests, far from the 1998 version I attended that was buzzing and brimming with MCMC sessions! (Correlatively, apart from the speakers in my own session, I hardly knew anyone there.) I however [missed a Bayesian session as I] attended a session on change-point detection, with a talk by Ziyang Yang (Lancaster University) on an acceleration proposal when the data size is too much, by aggregating similar datapoints, for which I would like to see a theoretical analysis of the impact of this aggregation on the quality (and consistency) of the attached Bayes factor. In my own session, I did not feel overly comfortable with the presentation of a commercial [i.e., for sale] software for pharmacokinetics.

I also enjoyed spending the day in Gießen, from running to the top of nearby Burg Glieberg at sunrise to swimming outdoor in the local 50m Freibad.