Archive for credible intervals

a lesser-known correlate of the Jeffreys-Lindley paradox (with discussion)

Posted in Books, pictures, Statistics, University life with tags , , , , , , , , , , , , , , , on October 19, 2024 by xi'an

Two UBC faculty, Harlan Campbell and Paul Gustafson, wrote a paper entitled “Defining a Credible Interval Is Not Always Possible with “Point-Null” Priors: A Lesser-Known Correlate of the Jeffreys-Lindley Paradox” in Bayesian Analysis (2024, 19, Number 3, pp. 925–984), which got discussed and presented on the BA webinar yesterday. I missed the call for discussion, on a topic I would have liked very much to discuss and an analysis I strongly disagree with. Fortunately, several of the discussants in the webinar and in the printed version advanced some of my points (as. e.g., Bertrand Clarke in the above slide screen-shot from the on-line video).

I find the paper somewhat missing in linking with the history of the topic, with no mention of Berger & Sellke (1987) that comes as a counterpoint to Casella &—the other—Berger (1987), opposing one sided to two sided tests. Or of matching priors, which connect credible and confidence intervals to higher orders. But the central issue with the apparent contradiction between rejecting the point null hypothesis and returning a credible interval that contains the null is that the construction proceeds from a model averaged posterior. Which fundamentally contradicts the construct of a pair of priors attached with each model towards selecting the fittest one. And requires a far-from-innocent choice of respective prior weights for both models, an ill-defined notion I have repeatedly criticised here and elsewhere. Model averaging clashes with model selection in both decision-theoretic and modelling terms. In model averaging terms, the disappearance of the opposition exhibited by the authors in the predictive distribution, as shown by discussants Held and Pawel, is unsurprising. And makes the spike-and-slab prior far of a necessity. Contrariwise to the model selection case where it proves unavoidable. And for which a merged credible interval does not make sense (to me at least) since it should be constructed once one (and only one) of the two models is chosen. At this point, that the other model ever was considered should not impact subsequent inference. And within that perspective I do not see the relevance of agnostic (ignoring the model choice ation) 5% confidence or credible regions.

“…considers the regime of a fixed true parameter value as n increases [and] of a fixed p-value…” (p928)

With regards with the connection with the Jeffreys-Lindley (or Lindley-Jeffreys) so-called paradox, on which I have already written a lot (or even too much!), many of the earlier objections resurface. Like the measure-theoretic difficulty in including within a continuous interval an atom, i.e., a value with a point mass. Which isolates this atom away from any other value in the interval (and of course creates discontinuities). Or fixing the p-value forever after (when n goes to infinity), as in the graph below (p929). Or treating an improper prior without further caution than with a proper prior. Especially when these are “created” by the decision problem itself.

 

William (Bill) Strawderman (1941-2024)

Posted in pictures, Statistics, University life with tags , , , , , , , , , , , , , , , , on October 3, 2024 by xi'an

Earlier today, I was informed by several of our mutual friends that my long-time friend Bill Strawderman had sadly passed away yesterday, after fighting a cancer for the past months. I remember quite clearly meeting Bill in the Fall of 1988 in front of White Hall, which hosted the Cornell maths department at the time, as he was visiting George Casella from Rutgers where he spent most of his career. I was most eager to meet him as I had worked on several of his landmark papers during my PhD on shrinkage estimation, as well as a bit impressed. But his kindness, modesty, and congenial personality quickly put me at ease and we spent the rest of his visit discussing shrinkage but also literature and music. Especially Dickens! After that we met and collaborated quite regularly, to the point he started visiting France upon my return, at Paris 6 (Pierre & Marie Curie) University first, and then in Rouen, where he became a adjunct professor and launched a life-long collaboration and friendship with Dominique Fourdrinier. As my interest in shrinkage estimation dwindled along the years, we did not keep collaborating for the past two decades, but we remained in touch and I was very happy to participate in his 80th anniversary celebration in Rutgers two years ago. His contributions to the field are notable and several papers of his were part of the Bayesian classics I was giving my graduate class a few years ago. From the fabulous minimaxity paper of 1984, along with George Casella, to admissible estimators dominating the positive-part James-Stein estimator, to sufficient conditions of minimaxity for proper Bayes estimators, to decision theoretic properties of Bayesian credible interval estimators, to loss estimation, not to mention his more applied side… Besides his fabulous sense of humour, which made many evenings with him memorable, I will also cherish the memory of a bon vivant who liked good food and good wines, incl. the Calvados apple brandy I would bring him at each of my visits.

confidence in confidence

Posted in Statistics, University life with tags , , , , on June 8, 2022 by xi'an

[This is a ghost post that I wrote eons ago and which got lost in the meanwhile.]

Following the false confidence paper, Céline Cunen, Niels Hjort & Tore Schweder wrote a short paper in the same Proceedings A defending confidence distributions. And blame the phenomenon on Bayesian tools, which “might have unfortunate frequentist properties”. Which comes as no surprise since Tore Schweder and Nils Hjort wrote a book promoting confidence distributions for statistical inference.

“…there will never be any false confidence, and we can trust the obtained confidence! “

Their re-analysis of Balch et al (2019) is that using a flat prior on the location (of a satellite) leads to a non-central chi-square distribution as the posterior on the squared distance δ² (between two satellites). Which incidentally happens to be a case pointed out by Jeffreys (1939) against the use of the flat prior as δ² has a constant bias of d (the dimension of the space) plus the non-centrality parameter. And offers a neat contrast between the posterior, with non-central chi-squared cdf with two degrees of freedom

F(\delta)=\Gamma_2(\delta^2/\sigma^2;||y||^2/\sigma^2)

and the confidence “cumulative distribution”

C(\delta)=1-\Gamma_2(|y||^2/\sigma^2;\delta^2/\sigma^2)

Cunen et al (2020) argue that the frequentist properties of the confidence distribution 1-C(R), where R is the impact distance, are robust to an increasing σ when the true value is also R. Which does not seem to demonstrate much. A second illustration of B and C when the distance δ varies and both σ and |y|² are fixed is even more puzzling when the authors criticize the Bayesian credible interval for missing the “true” value of δ, as I find the statement meaningless for a fixed value of |y|²… Looking forward the third round!, i.e. a rebuttal by Balch et al (2019)

simplified Bayesian analysis

Posted in Statistics with tags , , , , , , , , , , , , on February 10, 2021 by xi'an

A colleague from Dauphine sent me a paper by Carlo Graziani on a Bayesian analysis of vaccine efficiency, asking for my opinion. The Bayesian side is quite simple: given two Poisson observations, N~P(μ) and M~P(ν), there exists a reparameterisation of (μ,ν) into

e=1-μ/rν  and  λ=ν(1+(1-e)r)=μ+ν

vaccine efficiency and expectation of N+M, respectively, when r is the vaccine-to-placebo ratio of person-times at risk, ie the ratio of the numbers of participants in each group. Reparameterisation such that the likelihood factorises into a function of e and a function of λ. Using a product prior for this parameterisation leads to a posterior on e times a posterior on λ. This is a nice remark, which may have been made earlier (as for instance another approach to infer about e while treating λ as a nuisance parameter is to condition on N+M). The paper then proposes as an application of this remark an analysis of the results of three SARS-Cov-2 vaccines, meaning using the pairs (N,M) for each vaccine and deriving credible intervals, which sounds more like an exercise in basic Bayesian inference than a fundamental step in assessing the efficiency of the vaccines…

[Nature on] simulations driving the world’s response to COVID-19

Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , on April 30, 2020 by xi'an

Nature of 02 April 2020 has a special section on simulation methods used to assess and predict the pandemic evolution. Calling for caution as the models used therein, like the standard ODE S(E)IR models, which rely on assumptions on the spread of the data and very rarely on data, especially in the early stages of the pandemic. One epidemiologist is quote stating “We’re building simplified representations of reality” but this is not dire enough, as “simplified” evokes “less precise” rather than “possibly grossly misleading”. (The graph above is unrelated to the Nature cover and appears to me as particularly appalling in mixing different types of data, time-scale, population at risk, discontinuous updates, and essentially returning no information whatsoever.)

“[the model] requires information that can be only loosely estimated at the start of an epidemic, such as the proportion of infected people who die, and the basic reproduction number (…) rough estimates by epidemiologists who tried to piece together the virus’s basic properties from incomplete information in different countries during the pandemic’s early stages. Some parameters, meanwhile, must be entirely assumed.”

The report mentions that the team at Imperial College, which predictions impacted the UK Government decisions, also used an agent-based model, with more variability or stochasticity in individual actions, which require even more assumptions or much more refined, representative, and trustworthy data.

“Unfortunately, during a pandemic it is hard to get data — such as on infection rates — against which to judge a model’s projections.”

Unfortunately, the paper was written in the early days of the rise of cases in the UK, which means predictions were not much opposed to actual numbers of deaths and hospitalisations. The following quote shows how far off they can fall from reality:

“the British response, Ferguson said on 25 March, makes him “reasonably confident” that total deaths in the United Kingdom will be held below 20,000.”

since the total number as of April 29 is above 21,000 24,000 29,750 and showing no sign of quickly slowing down… A quite useful general public article, nonetheless.