Last night, Gael Martin, David Frazier (from Monash U) and myself arXived a survey on the history of Bayesian computations. This project started when Gael presented a historical overview of Bayesian computation, then entitled ‘Computing Bayes: Bayesian Computation from 1763 to 2017!’, at ‘Bayes on the Beach’ (Queensland, November, 2017). She then decided to build a survey from the material she had gathered, with her usual dedication and stamina. Asking David and I to join forces and bring additional perspectives on this history. While this is a short and hence necessary incomplete history (of not everything!), it hopefully brings some different threads together in an original enough fashion (as I think there is little overlap with recent surveys I wrote). We welcome comments about aspects we missed, skipped or misrepresented, most obviously!
Archive for Thomas Bayes
Computing Bayes: Bayesian Computation from 1763 to the 21st Century
Posted in Books, pictures, Statistics, Travel, University life with tags 1763, Australia, Bayes on the Beach, Bayesian computation, Monash University, survey, Thomas Bayes on April 16, 2020 by xi'anBayes plaque
Posted in Books, pictures, Statistics, Travel, University life with tags Bayes theorem, Edinburgh, FRS, plaque, Royal Society, Scotland, Thomas Bayes, University of Edinburgh on November 22, 2019 by xi'an
at the centre of Bayes
Posted in Mountains, pictures, Statistics, Travel, University life with tags 2019 Education Buildings Scotland Awards, Arthur's Seat, Bayes Café, Bayes Centre, Brown University, ECERM, Edinburgh, Holyrood Park, ICMS, jatp, Maxwell Institute Colloquium, Maxwell Institute Graduate School, Scotland, Scottish sun, seminar, Thomas Bayes, University of Edinburgh on October 14, 2019 by xi'an
let the evidence speak [book review]
Posted in Books, Kids, Statistics with tags Bayes grid, Bayesian decision theory, Bayesian inference, book review, court, Durham university, Edwin Jaynes, entropy, evidence, Federalist papers, Thomas Bayes on December 17, 2018 by xi'an
This book by Alan Jessop, professor at the Durham University Business School, aims at presenting Bayesian ideas and methods towards decision making “without formula because they are not necessary; the ability to add and multiply is all that is needed.” The trick is in using a Bayes grid, in other words a two by two table. (There are a few formulas that survived the slaughter, see e.g. on p. 91 the formula for the entropy. Contained in the chapter on information that I find definitely unclear.) When leaving the 2×2 world, things become more complicated and the construction of a prior belief as a probability density gets heroic without the availability of maths formulas. The first part of the paper is about Likelihood, albeit not the likelihood function, despite having the general rule that (p.73)
belief is proportional to base rate x likelihood
which is the book‘s version of Bayes’ (base?!) theorem. It then goes on to discuss the less structure nature of prior (or prior beliefs) against likelihood by describing Tony O’Hagan’s way of scaling experts’ beliefs in terms of a Beta distribution. And mentioning Jaynes’ maximum entropy prior without a single formula. What is hard to fathom from the text is how can one derive the likelihood outside surveys. (Using the illustration of 1963 Oswald’s murder by Ruby in the likelihood chapter does not particularly help!) A bit of nitpicking at this stage: the sentence
“The ancient Greeks, and before them the Chinese and the Aztecs…”
is historically incorrect since, while the Chinese empire dates back before the Greek dark ages, the Aztecs only rule Mexico from the 14th century (AD) until the Spaniard invasion. While most of the book sticks with unidimensional parameters, it also discusses more complex structures, for which it relies on Monte Carlo, although the description is rather cryptic (use your spreadsheet!, p.133). The book at this stage turns into a more story-telling mode, by considering for instance the Federalist papers analysis by Mosteller and Wallace. The reader can only follow the process of assessing a document authorship for a single word, as multidimensional cases (for either data or parameters) are out of reach. The same comment applies to the ecology, archeology, and psychology chapters that follow. The intermediary chapter on the “grossly misleading” [Court wording] of the statistical evidence in the Sally Clark prosecution is more accessible in that (again) it relies on a single number. Returning to the ban of Bayes rule in British courts:
In the light of the strong criticism by this court in the 1990s of using Bayes theorem before the jury in cases where there was no reliable statistical evidence, the practice of using a Bayesian approach and likelihood ratios to formulate opinions placed before a jury without that process being disclosed and debated in court is contrary to principles of open justice.
the discussion found in the book is quite moderate and inclusive, in that a Bayesian analysis helps in gathering evidence about a case, but may be misunderstood or misused at the [non-Bayesian] decision level.
In conclusion, Let the Evidence Speak is an interesting introduction to Bayesian thinking, through a simplifying device, the Bayes grid, which seems to come from management, with a large number of examples, if not necessarily all realistic and some side-stories. I doubt this exposure can produce expert practitioners, but it makes for an worthwhile awakening for someone “likely to have read this book because [one] had heard of Bayes but were uncertain what is was” (p.222). With commendable caution and warnings along the way.
the first Bayesian
Posted in Statistics with tags An Essay towards solving a Problem in the Doctrine of Chances, Bayes' notebooks, Bayesian foundations, David Hume, Richard Price, Royal Society, Statistical Science, Thomas Bayes on February 20, 2018 by xi'an
In the first issue of Statistical Science for this year (2018), Stephen Stiegler pursues the origins of Bayesianism as attributable to Richard Price, main author of Bayes’ Essay. (This incidentally relates to an earlier ‘Og piece on that notion!) Steve points out the considerable inputs of Price on this Essay, even though the mathematical advance is very likely to be entirely Bayes’. It may however well be Price who initiated Bayes’ reflections on the matter, towards producing a counter-argument to Hume’s “On Miracles”.
“Price’s caution in addressing the probabilities of hypotheses suggested by data is rare in early literature.”
A section of the paper is about Price’s approach data-determined hypotheses and to the fact that considering such hypotheses cannot easily fit within a Bayesian framework. As stated by Price, “it would be improbable as infinite to one”. Which is a nice way to address the infinite mass prior.
