An ISBA World meeting is always an exciting event that verges on the sensory overload! The more when one’s flight is perversely designed to land at my usual bedtime. (I tried waking up very early and running for an hour before catching the plane but this did not make me doze for more than one hour or two in the plane.) To explain why I missed the first two foundation lectures of the day, including the one by my dear friend Sylvia Richardson, as I was still resting (to avoid repeating my fainting upon arrival as happened last year, or later as). Incidentally, these lectures made me realise I had given one in Kyoto (on ABC) in 2012. I however managed to attend David Dunson’s wide recap on Bayesian clustering(s), with slides ChatGPT confused with posters, hence a massive information content had I been able to read them from my seat! More fundamentally, the residual difficulty being the very notion of cluster itself. And a jump back in time with Fumiyasu Komaki going over his corpus of works on the decision theoretic properties of Bayesian predictives when the Kullback-Leibler divergence is the loss function. An interesting feature of his findings is that comparing priors in that sense is equivalent to comparing them for point estimation in Normal and Poisson cases, but not in the Gamma case, for no reason I can fathom. A philosophical point of contention of mine’s was the introduction of two Jeffreys priors when observed data and predicted data are from different distributions (calibrated by the same parameter θ) since it calls for a Lindley’s style criticism that a realisation not (yet) observed modifies the prior distribution on θ. If possibly leading to improved estimation.
Archive for Jeffreys-Lindley paradox
ISBA 2026¹
Posted in Kids, pictures, Statistics, Travel, University life with tags admissibility, Bayesian predictive, clustering, conference, deadline, foundation lectures, ISBA, ISBA 2026, Japan, Jeffreys-Lindley paradox, Kyoto, Kyushu, logo, mixtures of distributions, Nagoya, registration fees, Shachihoko, Stein effect, student membership, Tokyo, WINC AICHI on June 30, 2026 by xi'ana lesser-known correlate of the Jeffreys-Lindley paradox (with discussion)
Posted in Books, pictures, Statistics, University life with tags Bayesian Analysis, Bayesian testing, confidence intervals, credible intervals, discussion paper, improper prior, Jeffreys-Lindley paradox, matching priors, paradoxes, point null hypotheses, two-sided hypotheses, UBC, University of British Columbia, Vancouver, webinar, youtube on October 19, 2024 by xi'an
Two UBC faculty, Harlan Campbell and Paul Gustafson, wrote a paper entitled “Defining a Credible Interval Is Not Always Possible with “Point-Null” Priors: A Lesser-Known Correlate of the Jeffreys-Lindley Paradox” in Bayesian Analysis (2024, 19, Number 3, pp. 925–984), which got discussed and presented on the BA webinar yesterday. I missed the call for discussion, on a topic I would have liked very much to discuss and an analysis I strongly disagree with. Fortunately, several of the discussants in the webinar and in the printed version advanced some of my points (as. e.g., Bertrand Clarke in the above slide screen-shot from the on-line video).
I find the paper somewhat missing in linking with the history of the topic, with no mention of Berger & Sellke (1987) that comes as a counterpoint to Casella &—the other—Berger (1987), opposing one sided to two sided tests. Or of matching priors, which connect credible and confidence intervals to higher orders. But the central issue with the apparent contradiction between rejecting the point null hypothesis and returning a credible interval that contains the null is that the construction proceeds from a model averaged posterior. Which fundamentally contradicts the construct of a pair of priors attached with each model towards selecting the fittest one. And requires a far-from-innocent choice of respective prior weights for both models, an ill-defined notion I have repeatedly criticised here and elsewhere. Model averaging clashes with model selection in both decision-theoretic and modelling terms. In model averaging terms, the disappearance of the opposition exhibited by the authors in the predictive distribution, as shown by discussants Held and Pawel, is unsurprising. And makes the spike-and-slab prior far of a necessity. Contrariwise to the model selection case where it proves unavoidable. And for which a merged credible interval does not make sense (to me at least) since it should be constructed once one (and only one) of the two models is chosen. At this point, that the other model ever was considered should not impact subsequent inference. And within that perspective I do not see the relevance of agnostic (ignoring the model choice ation) 5% confidence or credible regions.
“…considers the regime of a fixed true parameter value as n increases [and] of a fixed p-value…” (p928)
With regards with the connection with the Jeffreys-Lindley (or Lindley-Jeffreys) so-called paradox, on which I have already written a lot (or even too much!), many of the earlier objections resurface. Like the measure-theoretic difficulty in including within a continuous interval an atom, i.e., a value with a point mass. Which isolates this atom away from any other value in the interval (and of course creates discontinuities). Or fixing the p-value forever after (when n goes to infinity), as in the graph below (p929). Or treating an improper prior without further caution than with a proper prior. Especially when these are “created” by the decision problem itself.
Bayes Factors for Forensic Decision Analyses with R [book review]
Posted in Books, R, Statistics with tags Bayes factor, book review, Bruno de Finetti, Ca' Foscari University, CHANCE, Chib's approximation, classification, forensic statistics, improper prior, Jeffreys-Lindley paradox, MCMC, open access, R, springer on November 28, 2022 by xi'an
My friend EJ Wagenmaker pointed me towards an entire book on the BF by Bozza (from Ca’Foscari, Venezia), Taroni and Biederman. It is providing a sort of blueprint for using Bayes factors in forensics for both investigative and evaluative purposes. With R code and free access. I am of course unable to judge of the relevance of the approach for forensic science (I was under the impression that Bayesian arguments were usually not well-received in the courtroom) but find that overall the approach is rather one of repositioning the standard Bayesian tools within a forensic framework.
“The [evaluative] purpose is to assign a value to the result of a comparison between an item of unknown source and an item from a known source.”
And thus I found nothing shocking or striking from this standard presentation of Bayes factors, including the call to loss functions, if a bit overly expansive in its exposition. The style is also classical, with a choice of grey background vignettes for R coding parts that we also picked in our R books! If anything, I would have expected more realistic discussions and illustrations of prior specification across the hypotheses (see e.g. page 34), while the authors are mostly centering on conjugate priors and the (de Finetti) trick of the equivalent prior sample size. Bayes factors are mostly assessed using a conservative version of Jeffreys’ “scale of evidence”. The computational section of the book introduces MCMC (briefly) and mentions importance sampling, harmonic mean (with a minimalist warning), and Chib’s formula (with no warning whatsoever).
“The [investigative] purpose is to provide information in investigative proceedings (…) The scientist (…) uses the findings to generate hypotheses and suggestions for explanations of observations, in order to give guidance to investigators or litigants.”
Chapter 2 is about standard models: inferring about a proportion, with some Monte Carlo illustration, and the complication of background elements, normal mean, with an improper prior making an appearance [on p.69] with no mention being made of the general prohibition of such generalised priors when using Bayes factors or even of the Lindley-Jeffreys paradox. Again, the main difference with Bayesian textbooks stands with the chosen examples.
Chapter 3 focus on evidence evaluation [not in the computational sense] but, again, the coverage is about standard models: processing the Binomial, multinomial, Poisson models, again though conjugates. (With the side remark that Fig 3.2 is rather unhelpful: when moving the prior probability of the null from zero to one, its posterior probability also moves from zero to one!) We are back to the Normal mean case with the model variance being known then unknown. (An unintentionally funny remark (p.96) about the dependence between mean and variance being seen as too restrictive and replaced with… independence!). At last (for me!), the book is pointing [p.99] out that the BF is highly sensitive to the choice of the prior variance (Lindley-Jeffreys, where art thou?!), but with a return of the improper prior (on said variance, p.102) with no debate on the ensuing validity of the BF. Multivariate Normals are also presented, with Wishart priors on the precision matrix, and more details about Chib’s estimate of the evidence. This chapter also contains illustrations of the so-called score-based BF which is simply (?) a Bayes factor using a distribution on a distance summary (between an hypothetical population and the data) and an approximation of the distributions of these summaries, provided enough data is available… I also spotted a potentially interesting foray into BF variability (Section 3.4.2), although not reaching all the way to a notion of BF posterior distributions.
Chapter 4 stands for Bayes factors for investigation, where alternative(s) is(are) less specified, as testing eg Basmati rice vs non-Basmati rice. But there is no non-parametric alternative considered in the book. Otherwise, it looks to me rather similar to Chapter 3, i.e. being back to binomial, multinomial models, with more discussions onm prior specification, more normal, or non-normal model, where the prior distribution is puzzingly estimated by a kernel density estimator, a portmanteau alternative (p.157), more multivariate Normals with Wishart priors and an entry on classification & discrimination.
[Disclaimer about potential self-plagiarism: this post or an edited version will eventually appear in my Books Review section in CHANCE. As appropriate for a book about Chance!]
false confidence, not fake news!
Posted in Books, Statistics with tags Bayes factors, confidence distribution, epistemic probability, Jeffreys-Lindley paradox, Proceedings of the Royal Society, Royal Society on May 28, 2021 by xi'an“…aerospace researchers have recognized a counterintuitive phenomenon in satellite conjunction analysis, known as probability dilution. That is, as uncertainty in the satellite trajectories increases, the epistemic probability of collision eventually decreases. Since trajectory uncertainty is driven by errors in the tracking data, the seemingly absurd implication of probability dilution is that lower quality data reduce the risk of collision.”
In 2019, Balch, Martin, and Ferson published a false confidence theorem [false confidence, not false theorem!] in the Proceedings of the Royal [astatistical] Society, motivated by satellite conjunction (i.e., fatal encounter) analysis. But discussing in fine the very meaning of a confidence statement. And returning to the century old opposition between randomness and epistemic uncertainty, aleatory versus epistemic probabilities.
“…the counterintuitiveness of probability dilution calls this [use of epistemic probability] into question, especially considering [its] unsettled status in the statistics and uncertainty quantification communities.”
The practical aspect of the paper is unclear in that the opposition of aleatory versus epistemic probabilities does not really apply when the model connecting the observables with the position of the satellites is unknown. And replaced with a stylised parametric model. When ignoring this aspect of uncertainty, the debate is mostly moot.
“…the problem with probability dilution is not the mathematics (…) if (…) inappropriate, that inappropriateness must be rooted in a mismatch between the mathematics of probability theory and the epistemic uncertainty to which they are applied in conjunction analysis.”
The probability dilution phenomenon as described in the paper is that, when (posterior) uncertainty increases, the posterior probability of collision eventually decreases, which makes sense since poor precision implies the observed distance is less trustworthy and the satellite could be anywhere. To conclude that increasing the prior or epistemic uncertainty makes the satellites safer from collision is thus fairly absurd as it only concerns the confidence in the statement that there will be a collision. But I agree with the conclusion that the statement of a low posterior probability is a misleading risk metric because, just like p-values, it is a.s. taken at face value. Bayes factors do relativise this statement [but are not mentioned in the paper]. But with the spectre of Lindley-Jeffreys paradox looming in the background.
The authors’ notion of false confidence is formally a highly probable [in the sample space] report of a high belief in a subset A of the parameter set when the true parameter does not belong to A. Which holds for all epistemic probabilities in the sense that there always exists such a set A. A theorem that I see as related to the fact that integrating an epistemic probability statement [conditional on the data x] wrt the true sampling distribution [itself conditional on the parameter θ] is not coherent from a probabilistic standpoint. The resolution of the paradox follows a principle set by Ryan Martin and Chuanhai Liu, such that “it is almost a tautology that a statistical approach satisfying this criterion will not suffer from the severe false confidence phenomenon”, although it sounds to me that this is a weak patch on a highly perforated tyre, the erroneous interpretation of probabilistic statements as frequentist ones.
demystify Lindley’s paradox [or not]
Posted in Statistics with tags Bayesian hypothesis testing, improper prior, Jeffreys-Lindley paradox, non-informative priors, point null hypotheses, two-sided hypotheses on March 18, 2020 by xi'an
Another paper on Lindley’s paradox appeared on arXiv yesterday, by Guosheng Yin and Haolun Shi, interpreting posterior probabilities as p-values. The core of this resolution is to express a two-sided hypothesis as a combination of two one-sided hypotheses along the opposite direction, taking then advantage of the near equivalence of posterior probabilities under some non-informative prior and p-values in the later case. As already noted by George Casella and Roger Berger (1987) and presumably earlier. The point is that one-sided hypotheses are quite friendly to improper priors, since they only require a single prior distribution. Rather than two when point nulls are under consideration. The p-value created by merging both one-sided hypotheses makes little sense to me as it means testing that both θ≥0 and θ≤0, resulting in the proposal of a p-value that is twice the minimum of the one-sided p-values, maybe due to a Bonferroni correction, although the true value should be zero… I thus see little support for this approach to resolving Lindley paradox in that it bypasses the toxic nature of point-null hypotheses that require a change of prior toward a mixture supporting one hypothesis and the other. Here the posterior of the point-null hypothesis is defined in exactly the same way the p-value is defined, hence making the outcome most favourable to the agreement but not truly addressing the issue.
