Archive for William Feller

Newer Entries »

“Not only defended but also applied”

Posted in Statistics with tags , , , , on June 30, 2010 by xi'an

On page 124 of his superb Introduction to Probability Theory book (volume 1), William Feller has this strange remark about Bayesian inference:

“Unfortunately Bayes’ rule has been somewhat discredited by metaphysical applications of the type described above. In routine practice, this kind of argument can be dangerous. A quality control engineer is concerned with one particular machine and not with an in nite population of machines from which one was chosen at random. He has been advised to use Bayes’ rule on the grounds that it is logically acceptable and corresponds to our way of thinking. Plato used this type of argument to prove the existence of Atlantis, and philosophers used it to prove the absurdity of Newton’s mechanics. In our case it overlooks the circumstance that the engineer desires success and that he will do better by estimating and minimizing the sources of various types of errors in predicting and guessing.The modern method of statistical tests and estimation is less intuitive but more realistic. It may be not only defended but also applied.”

When we were discussing about this great book, Andrew Gelman pointed out to me this strong dismissal of Bayesian techniques (note that I had overlooked so far) and, given that it is still quoted as an argument against a Bayesian approach to inference, we ended up writing [well, mostly Andrew!] a short note on the motivations and implications of this remark, now published on arXiv. One of the points is that Feller’s sentence has the interesting feature that it is actually the opposite of the usual demarcation: typically it is the Bayesian who makes the claim for inference in a particular instance and the frequentist who restricts claims to infinite populations of replications. Another point is the naïve faith in the classical Neyman-Pearson theory to solve practical problems in statistics.

Actually, Persi Diaconis took a (deeper) look at Feller’s stance as well, as mentioned in this review of Jaynes’s Probability Theory. Using Amazon Look Inside tool,  I spotted Feller being mentioned more than 30 times in Jaynes’s book, one of the best quotes being “The date was 1956 when the author met Willy Feller“! More to the point, Jaynes identifies Feller’s dismissal of the “old wrong ways” (volume 2, p.76), which is to be opposed to the “modern method” above. (Persi Diaconis and Susan Holmes also wrote a nice piece entitled “A Bayesian peek into Feller volume 1” that does not relate directly to this issue.) In a loosely related point, Persi’s warning that he sees “a strong trend against measure theory in modern statistics departments: [he] had to fight to keep the measure theory requirement in Stanford’s statistics graduate program“, to which I completely subscribe, should be heard more widely…

2 Comments »

Versions of Benford’s Law

Posted in Books, Statistics with tags , , , , on May 20, 2010 by xi'an

A new arXived note by Berger and Hill discusses how [my favourite probability introduction] Feller’s Introduction to Probability Theory (volume 2) gets Benford’s Law “wrong”. While my interest in Benford’s Law is rather superficial, I find the paper of interest as it shows a confusion between different folk theorems! My interpretation of Benford’s Law is that the first significant digit of a random variable (in a basis 10 representation) is distributed as

f(i) \propto \log_{10}(1+\frac{1}{i})

and not that \log(X) \,(\text{mod}\,1) is uniform, which is the presentation given in the arXived note…. The former is also the interpretation of William Feller (page 63, Introduction to Probability Theory), contrary to what the arXived note seems to imply on page 2, but Feller indeed mentioned as an informal/heuristic argument in favour of Benford’s Law that when the spread of the rv X is large,  \log(X) is approximately uniformly distributed. (I would no call this a “fundamental flaw“.) The arXived note is then right in pointing out the lack of foundation for Feller’s heuristic, if muddling the issue by defining several non-equivalent versions of Benford’s Law. It is also funny that this arXived note picks at the scale-invariant characterisation of Benford’s Law when Terry Tao’s entry represents it as a special case of Haar measure!

5 Comments »

The reflection principle

Posted in Books, Statistics with tags , , , , on February 7, 2010 by xi'an

In the weekend magazine of Le Monde, there always is one mathematical puzzle that often is easy to solve but sometimes is harder (like the puzzles involving arcane triangular geometry!) and occasionally just impossible (because the puzzle is missing one crucial assumption). The puzzle of this week is a direct application of the reflection principle exposed in Chapter III of Feller’s An Introduction to Probability Theory and Its Applications, Vol. 1, which is my favourite probability book. (More precisely, the answer is (almost) given in problem 1, page 95 of this book.)

Given a sequence of 2n heads and tails ending with exactly n heads, what is the probability that the sequence always remained in favour of heads, i.e. that the number of heads was larger than or equal to the number of tails from trial one to trial 2n? The number of non-negative paths of length 2n ending up in zero is the same as the number of (strictly) positive paths of length 2n+1 ending up in one (see Figure 1, page 69, in Feller). Now, by the reflection principle (Feller, Lemma, page 72), this number is

\dfrac{N_{2n+1-1,1-1}-N_{2n+1-1,1+1}}{N_{2n,0}} with N_{n,x}=\displaystyle{{n\choose \frac{n+x}{2}}}

being the number of paths of length n ending up in x (and N_{2n,2} being the number of paths hitting the zero value at least once after the starting time). The probability of a non-negative sequence is therefore

\mathbb{P}(S_t\ge 0\,,\,1\le t\le 2n\mid S_{2n}=0) = \dfrac{1}{(n+1)}.

This has a potential connection with Laplace’s succession rule, but I cannot make it straight away because the underlying events of not visiting -1 are not independent…

If you want to check this by simulation, a simple R program is

Nsim=10^6; n=4
cont=0;gain=0
while (gain<Nsim){
vale=cumsum(sample(c(-1,1),2*n,rep=TRUE))
gain=gain+(vale[2*n]==0)
cont=cont+(vale[2*n]==0)*(min(vale)>-1)
}
print(cont/Nsim)

If you have never read An Introduction to Probability Theory and Its Applications, Vol. 1, you should consider doing it: it makes for a very pleasant read, while making combinatorics enjoyable and it also constitutes a wealth of problems that I use extensively when teaching at Polytechnique.

3 Comments »

Newer Entries »