Our paper with Andrew Gelman, “Not only defended but also applied”: the perceived absurdity of Bayesian inference, has been reviewed for the second time and is to appear in The American Statistician, as a discussion paper. Terrific news! This is my first discussion paper in The American Statistician (and the second in total, the first one being the re-read of Jeffreys‘ Theory of Probability
Archive for Theory of Probability
not only defended but also applied [to appear]
Posted in Books, Statistics, University life with tags Andrew Gelman, arXiv, ASA, discussion paper, Harold Jeffreys, The American Statistician, Theory of Probability, William Feller on June 12, 2012 by xi'anMay I believe I am a Bayesian?!
Posted in Books, Statistics, University life with tags Bayesian inference, E.T. Jaynes, foundations, Harold Jeffreys, Markets and Morals, Rationality, Ronald Fisher, Stephen Hawking, Theory of Probability, Thomas Bayes on January 21, 2012 by xi'an“…the argument is false that because some ideal form of this approach to reasoning seems excellent n theory it therefore follows that in practice using this and only this approach to reasoning is the right thing to do.” Stephen Senn, 2011
Deborah Mayo, Aris Spanos, and Kent Staley have edited a special issue of Rationality, Markets and Morals (RMM) (a rather weird combination, esp. for a journal name!) on “Statistical Science and Philosophy of Science: Where Do (Should) They Meet in 2011 and Beyond?” for which comments are open. Stephen Senn has a paper therein entitled You May Believe You Are a Bayesian But You Are Probably Wrong in his usual witty, entertaining, and… Bayesian-bashing style! I find it very kind of him to allow us to remain in the wrong, very kind indeed…
Now, the paper somehow intersects with the comments Stephen made on our review of Harold Jeffreys’ Theory of Probability a while ago. It contains a nice introduction to the four great systems of statistical inference, embodied by de Finetti, Fisher, Jeffreys, and Neyman plus Pearson. The main criticism of Bayesianism à la de Finetti is that it is so perfect as to be outworldish. And, since this perfection is lost in the practical implementation, there is no compelling reason to be a Bayesian. Worse, that all practical Bayesian implementations conflict with Bayesian principles. Hence a Bayesian author “in practice is wrong”. Stephen concludes with a call for eclecticism, quite in line with his usual style since this is likely to antagonise everyone. (I wonder whether or not having no final dot to the paper has a philosophical meaning. Since I have been caught in over-interpreting book covers, I will not say more!) As I will try to explain below, I believe Stephen has paradoxically himself fallen victim of over-theorising/philosophising! (Referring the interested reader to the above post as well as to my comments on Don Fraser’s “Is Bayes posterior quick and dirty confidence?” for more related points. Esp. about Senn’s criticisms of objective Bayes on page 52 that are not so central to this discussion… Same thing for the different notions of probability [p.49] and the relative difficulties of the terms in (2) [p.50]. Deborah Mayo has a ‘deconstructed” version of Stephen’s paper on her blog, with a much deeper if deBayesian philosophical discussion. And then Andrew Jaffe wrote a post in reply to Stephen’s paper. Whose points I cannot discuss for lack of time, but with an interesting mention of Jaynes as missing in Senn’s pantheon.)
“The Bayesian theory is a theory on how to remain perfect but it does not explain how to become good.” Stephen Senn, 2011
While associating theories with characters is a reasonable rethoretical device, especially with large scale characters as the one above!, I think it deters the reader from a philosophical questioning on the theory behind the (big) man. (In fact, it is a form of bullying or, more politely (?), of having big names shoved down your throat as a form of argument.) In particular, Stephen freezes the (Bayesian reasoning about the) Bayesian paradigm in its de Finetti phase-state, arguing about what de Finetti thought and believed. While this is historically interesting, I do not see why we should care at the praxis level. (I have made similar comments on this blog about the unpleasant aspects of being associated with one character, esp. the mysterious Reverent Bayes!) But this is not my main point.
“…in practice things are not so simple.” Stephen Senn, 2011
The core argument in Senn’s diatribe is that reality is always more complex than the theory allows for and thus that a Bayesian has to compromise on her/his perfect theory with reality/practice in order to reach decisions. A kind of philosophical equivalent to Achille and the tortoise. However, it seems to me that the very fact that the Bayesian paradigm is a learning principle implies that imprecisions and imperfections are naturally endowed into the decision process. Thus avoiding the apparent infinite regress (Regress ins Unendliche) of having to run a Bayesian analysis to derive the prior for the Bayesian analysis at the level below (which is how I interpret Stephen’s first paragraph in Section 3). By refusing the transformation of a perfect albeit ideal Bayesian into a practical if imperfect bayesian (or coherent learner or whatever name that does not sound like being a member of a sect!), Stephen falls short of incorporating the contrainte de réalité into his own paradigm. The further criticisms found about prior justification, construction, evaluation (pp.59-60) are also of that kind, namely preventing the statistician to incorporate a degree of (probabilistic) uncertainty into her/his analysis.
In conclusion, reading Stephen’s piece was a pleasant and thought-provoking moment. I am glad to be allowed to believe I am a Bayesian, even though I do not believe it is a belief! The praxis of thousands of scientists using Bayesian tools with their personal degree of subjective involvement is an evolutive organism that reaches much further than the highly stylised construct of de Finetti (or of de Finetti restaged by Stephen!). And appropriately getting away from claims to being perfect or right. Or even being more philosophical.
“Who is the statistician or scientist you admire the most?”
Posted in Books, Statistics, University life with tags Caen, Harold Jeffreys, ISBA Bulletin, Laplace, Normandy, Pays d'Auge, Theory of Probability, Traité Philosophique sur les Probabilités on April 5, 2011 by xi'an
As for the previous ISBA Bulletin, Luke Bornn sent me this (impossible!) question, to be answered in a few hundred words. First, let me exclude all living statisticians and scientists to avoid making a choice among all those people I admire and hurting anyone’s feeling (and also because this is somehow unfair to younger researchers). So let us stick to dead individuals! Second, I am quite hesitant to choose between a scientist (broad category!) and a statistician (restrictive category!). Again, let me [first?] stick to statisticians, avoiding the impossible choice between Albert Einstein, Marie Curie, Srinivasa Ramanujan, Henri Poincaré, Evariste Galois, Ada Byron, and others… Continue reading
More on Treatise
Posted in Books, Statistics with tags A Treatise on Probability, Harold Jeffreys, John Maynard Keynes, Theory of Probability on March 25, 2010 by xi'an
When writing my review of Keynes’ A Treatise On Probability, I found that there is a very detailed review paper by John Aldrich (2008) that covers the beginnings of Keynes as a statistician, entitled “Keynes among the Statisticians” (sic!) and published in the journal History of Political Economy. This review is incredibly helpful in resetting the book in the conditions at the time. I also discovered through this review that Harold Jeffreys reviewed A Treatise On Probability in Nature and that Dennis Lindley wrote an Encyclopedia entry on Keynes, no less…
Quick visit to A Treatise on Probability
Posted in Books, Statistics with tags A Treatise on Probability, Harold Jeffreys, John Maynard Keynes, Theory of Probability on March 23, 2010 by xi'an
Following my advanced graduate course at CREST about Jeffreys’ Theory of Probability, and a remark in Taleb’s Fooled by Randomness, I was planning to cover John Maynard Keynes’ A Treatise On Probability this year. After reading the book over the past weeks, and pondering it over the nights at the Frontiers of Statistical Decision Making and Bayesian Analysis conference, I decided it could not provide enough material for a whole two week course. Since I was starting today just getting off the plane from Detroit, I decided to fall back on my earlier course on Jeffreys’ Theory of Probability and to include comments on Keynes’ A Treatise On Probability wherever they fitted… Indeed, I found that most of the book focussed on philosophical foundations of probability that were of no interest for me nor for my students, while the statistics part was not innovative enough. I will eventually write a review of the book listing those drawbacks but the main aspect of Keynes’ statistical perspective is to be highly suspicious of models and thus to favour empirical techniques that are close to modern bootstrap. In particular, Keynes criticises inverse probability techniques and Laplace’s perspective all over the book… This is thus an interesting historical foray but this memoir of Keynes prepared for a Fellowship application does not stand within the influential books of the 20th century for a good reason.
