Archive for decision theory

Pitman closeness renewal?

Posted in Statistics, University life with tags , , , , on July 26, 2012 by xi'an

As noticed there a few months ago, the Pitman closeness criterion for comparing estimators (through the probability

Pθ(|δ-θ|<|δ’-θ|)

which should be larger than .5 for the first estimator to be deemed “better” or “Pitman closer”) has been “resuscitated” by Canadian researchers. In 1993, I wrote a JASA (discussion) paper along with Gene Hwang and Bill Strawderman pointing out the many inconsistencies of this criterion as a decision tool.  It was entitled “Is Pitman Closeness a Reasonable Criterion?” (The answer was in the question, right?!)

In an arXiv posting today, Jozani, Balakrishnan, and Davies propose new characterisations for comparing (in this sense) symmetrically distributed estimators. There is nothing wrong with this mathematical exercise, obviously. However, the approach still seems to suffer from the same decisional inconsistencies as in the past:

  1. the results in the paper (see, e.g., Lemma 1 and 2) only apply to independent estimators, which is rather unrealistic (to the point of having the authors applying it to dependent estimators, the sample median X[n/2] versus a fixed index observation, e.g. X3, and again at the end of the paper in the comparison of several order statistics). Having independent estimators to compare is a rather rare situation as one tries to make the most of a given sample;
  2. the setup is highly dependent on considering a single (one-dimensional) location parameter, the results do not apply to more general settings (except location-scale cases with scale parameters known to some extent, see Lemma 5) ;
  3. some results (see Remark 4) allow to find a whole range of estimators dominating a given (again independent) estimator δ’, but they do not give a ranking of those estimators, except in the weak sense of having the above probability maximal in one of the estimators δ (Lemma 9). This is due to the independence constraint on the comparison. There is therefore no possibility (in this setting) of obtaining an estimator that is the “Pitman closest estimator of θ“, as claimed by the authors in the final section of their paper.

Once again, I have nothing against these derivations, which are mostly correct, but I simply argue here that they cannot constitute a competitor to standard decision theory.

loss functions for credible regions

Posted in Statistics, University life with tags , , , , on March 15, 2012 by xi'an

When Éric Marchand came to give a talk last week, we discussed about minimality and Bayesian estimation for confidence/credible regions. In the early 1990’s, George Casella and I wrote a paper in this direction, entitled “Distance weighted losses for testing and confidence set evaluation” and published in TEST. It was restricted to the univariate case but one could consider evaluating α-level confidence regions with a loss function like

L(\theta,C) = \left(\theta-\text{proj}_C(\theta)\right)^2

where the projection of the parameter over C is the element in C that is closest to the parameter. As in the original paper, this loss function brings a penalty of how far is the parameter from the region, compared the rudimentary 0-1 loss function which penalises all misses the same way. The posterior loss is not straightforward to minimise, though. Unless one considers an approximation based on a sample from the posterior and picks the (1-α)-fraction that gives the smallest sum of distances to the remaining α-fraction. And then takes a convexification of the α-fraction. This is not particularly “clean” and I would prefer to find an HPD-like region, i.e. an HPD linked to a modified prior… But this may require another loss function than the one above. Incidentally, I was also playing with an alternative loss function that would avoid setting the level α. Namely

L(\theta,C) = \left(\theta-\text{proj}_C(\theta)\right)^2 + \tau\, \text{diam}(C)^2,

which simultaneously penalises non-coverage and size. However, the choice of τ makes the function difficult to motivate in a realistic setting.

on Pitman closeness

Posted in Statistics, University life with tags , , , on November 15, 2011 by xi'an

I came by happenstance upon this talk, “Some Pitman Closeness Properties Pertinent to Symmetric Populations”, given by Mohammad Jozania, at the University of Manitoba next week, and it rescinded my former (if negative) interest in Pitman nearness (or closeness). This criterion, which originated in a 1937 paper of E.J.G. Pitman, compares two estimators in the light of the probability of one being closer (to the “truth”) than the other,

\text{Pr}_\theta(|\hat\theta_1(X)-\theta|<|\hat\theta_2(X)-\theta|)

and there was a brief interest in the method at the end of the 1980’s, culminating with Keating and Mason’s book on the topic.

In a 1993 JASA paper I wrote with Gene Hwang and Bill Strawderman, entitled “Is Pitman Closeness a Reasonable Criterion?“, we demonstrated that, in many respects, this criterion was not appropriate for comparing estimators. For instance, the comparison was not transitive, two estimators with the same marginal distribution could sometimes be ranked, a Bayes estimator could not be properly derived, some counter-intuitive orderings could be exhibited, &tc… This was an exciting (and fun) paper to  write as it was only made of (counter)examples. (Hence our answer to the above question was  definitive no.) Judging from the abstract to the talk,

In this talk, we focus on Pitman closeness probabilities when the estimators are symmetrically distributed about the unknown parameter θ. We first consider two symmetric estimators θ¹ and θ² and obtain necessary and sufficient conditions for θ¹ to be Pitman closer to the common median θ than θ². We then establish some properties in the context of estimation under Pitman closeness criterion. We define a Pitman closeness probability which measures the frequency with which an individual order statistic is Pitman closer to θ than some symmetric estimator. We show that, for symmetric populations, the sample median is Pitman closer to the population median than any other symmetrically distributed estimator of θ. Finally, we discuss the use of Pitman closeness probabilities in the determination of an optimal ranked set sampling scheme (denoted by RSS) for the estimation of the population median when the underlying distribution is symmetric. We show that the best RSS scheme from symmetric populations in the sense of Pitman closeness is the median and randomized median RSS for the cases of odd and even sample sizes, respectively.

it sounds like the authors have relaunched research in this area, hence that our 1993 definitive conclusion against the use of the criterion was not definitive for everyone…  (I could not find a trace of the corresponding paper through google, but I would be interested in reading the recent research on the topic! Even though the result about the “optimality” of the sample median reminds me of earlier results, with the related drawback that this optimality is incompatible with the sufficiency principle.)

Prospect theory

Posted in Books, Statistics, University life with tags , , , , , on October 6, 2011 by xi'an

I received this book from Cambridge University Press for reviewing, but its topic is just too far from my research interests to invest time into reading it. I had it with me during my flight to New York last week but it simply evaded me…  The words and graphs are mostly those of decision analysis but the whole is too remote from my statistical universe. (The front cover of the Copper River Delta, Chugach National Forest, Alaska, is tantalising, of course!) Anyway, if any ‘Og reader [with some credential in prospect theory or decision analysis] was interested in reviewing the book for CHANCE, let her or him contact me by email through bayesianstatistics on Gmail.. (The three books have been claimed!)

Back to Philly

Posted in Statistics, Travel, University life with tags , , , , , , , on December 15, 2010 by xi'an

Today and tomorrow, I am attending a conference in Wharton in honour of Larry Brown for his 70th birthday. I met Larry in 1988 when visiting Cornell for the year—even using his office in the Math department while he was away on a sabbatical leave—and it really does not feel like that long ago, nor does it feel like Larry is any close to 70 as he looks essentially the same as 22 years ago! The conference is reflecting Larry’s broad range of research from decision-theory and nonparametrics to data analysis. I am thus very glad to celebrate Larry’s birthday with a whole crowd of old and more recent friends. (My talk on Rao-Blackwellisation will be quite similar to the seminar I gave in Stanford last summer [except that I have to talk twice as fast!])