Archive for Des Moines
why don’t you wear a suit?!
Posted in Books, Kids, pictures, Travel, University life with tags baby Trump, black Friday, CDT, Centre for Doctoral Training, department of statistics, Des Moines, French Navy, Iowa, John Cairns, journalism, MAGA, objectivity, Oxford, OxWaSP, PhD students, Racoon River Brewing Co., Solidarity with Ukraine, Trump administration, University of Oxford, US politics, USA, Volodymyr Zelenskiy, White House on March 4, 2025 by xi'an
the polls weren’t wrong [book review]
Posted in Books, R, Statistics, University life with tags book review, CHANCE, CRC Press, Des Moines, Donald Trump, drunkard, FiveThirtyEight, Jakob Bernoulli, plane trip, political polls, polls, statistical literacy, The New York Times, University of Warwick, US elections 2016, US elections 2020, US elections 2024 on November 1, 2024 by xi'an
While Nate Silver and his colleagues (as The New York Times Nate Cohn) have brought (part of) the general public to adopt a (more) scientific perspective on political polls, the way to combine them, and the need to keep uncertainty fully quantified (witness this recent “Two Theories for Why the Polls Failed in 2020, and What It Means for 2024” by Nate Cohn for the NYT’s Tilt), the author of this book embarks upon a crusade (and a lengthy rant) against pollsters and analysts and media reporters, with the single, many times repeated, argument that non-responses and undecided voters are crucial for the final election outcome… And that a poll gives a snapshot of the current (time) population opinion, not a prediction of its future state. There is not the slightest trace of statistical depth, there is actually no statistics at all found throughout the book, apart from a section (p.281) entitled “Threats to inferential statistics” (with not no maths either, as a few ratio manipulations and a chapter title invoking Jakob Bernoulli!) do not count as maths!, but a lot of repetitions on the same theme and dismissal of statisticians’ analyses, like Nate Silver’s. Opposing to them the theories of Nick Panagakis, a 1990’s pollster. (Funny enough, the Amazon reviews include one “expert in inferential statistics, the major tool employed by Carl [Alen] in this book” and another one stating that “Carl Allen takes the reader through a journey towards statistical literacy“!) And an R code (p.52) for plotting the outcome of 30 Binomial random draws.
“Unless my efforts achieve far more notoriety than even my most optimistic forecast would predict, [the Proportional Method] is unlikely to go away any time soon.” p.183
“If it seems like I’m picking on FiveThirtyEight a lot, it’s not because there are no other forecasters who are better or worse.” p.235
“Sounding eerily like myself, [Nate Silver] pointed out [in 2008] that `many things can happen’ months before the election.” p.208
Reading through the book (during a trip to & from Warwick) was painful, both for the feeling of being stuck in a plane with a perfect unknown, next seat, trying to force their weird theory upon you and no way to escape their rant, as well as for the terrible style of said book, full of repetitions and one-sentence paragraphs. (Of course, nothing as bad as this time near the 2012 US elections I flew to Des Moines next to an inebriated woman that would not stop blathering about her life!) Or as I imagine a card game addict defending their martingale as a sure way to win against the casino. It is also the first time I see references repeated (in postcripts) as many times as they are cited within a chapter. There is no true insight on how polling companies construct their polling samples, how they post-process outcomes by regression techniques, and no reflection on the unique weirdness of the US electoral system in that a few States determine the outcome (rather than majority votes) and thus how a tiny number of voters (escaping the law of Large Numbers) hold the overall result in their hand.
Thus (as most readers will have forecasted) concluding by not recommending the book!
[Disclaimer about potential self-plagiarism: this post or an edited version of it could possibly appear in my Books Review section in CHANCE. Most unlikely though!]
one bridge further
Posted in Books, R, Statistics, University life with tags AMIS, bias, bridge sampling, Des Moines, Iowa, method of moments, multiple importance methods, multiple mixtures, normalising constant, sunrise, University of Warwick, untractable normalizing constant, warped bridge sampling on June 30, 2020 by xi'an
Jackie Wong, Jon Forster (Warwick) and Peter Smith have just published a paper in Statistics & Computing on bridge sampling bias and improvement by splitting.
“… known to be asymptotically unbiased, bridge sampling technique produces biased estimates in practical usage for small to moderate sample sizes (…) the estimator yields positive bias that worsens with increasing distance between the two distributions. The second type of bias arises when the approximation density is determined from the posterior samples using the method of moments, resulting in a systematic underestimation of the normalizing constant.”
Recall that bridge sampling is based on a double trick with two samples x and y from two (unnormalised) densities f and g that are interverted in a ratio
of unbiased estimators of the inverse normalising constants. Hence biased. The more the less similar these two densities are. Special cases for ω include importance sampling [unbiased] and reciprocal importance sampling. Since the optimal version of the bridge weight ω is the inverse of the mixture of f and g, it makes me wonder at the performance of using both samples top and bottom, since as an aggregated sample, they also come from the mixture, as in Owen & Zhou (2000) multiple importance sampler. However, a quick try with a positive Normal versus an Exponential with rate 2 does not show an improvement in using both samples top and bottom (even when using the perfectly normalised versions)
morc=(sum(f(y)/(nx*dnorm(y)+ny*dexp(y,2)))+
sum(f(x)/(nx*dnorm(x)+ny*dexp(x,2))))/(
sum(g(x)/(nx*dnorm(x)+ny*dexp(x,2)))+
sum(g(y)/(nx*dnorm(y)+ny*dexp(y,2))))
at least in terms of bias… Surprisingly (!) the bias almost vanishes for very different samples sizes either in favour of f or in favour of g. This may be a form of genuine defensive sampling, who knows?! At the very least, this ensures a finite variance for all weights. (The splitting approach introduced in the paper is a natural solution to create independence between the first sample and the second density. This reminded me of our two parallel chains in AMIS.)
snapshots of Oxford Statistics
Posted in Kids, pictures, Statistics, Travel, University life, Wines with tags bike, Des Moines, Iowa, John Cairns, Oxford, OxWaSP, PhD students, Racoon River Brewing Co., University of Oxford on February 29, 2016 by xi'an
Following the opening of the new Department of Statistics building in Oxford [which somewhat ironically is the former Department of Mathematics!], a professional photographer was commissioned for a photo cover of this move. Which is incidentally fantastic for the cohesion and work quality of the department, when compared with the former configuration in two disconnected buildings on South Parks Road. Not mentioning the vis-à-vis with Eagle and Child.
As the photographer happened to be there the very day I was teaching my Bayesian module for the OxWaSP PhD students, I ended up in some of the photographs (with no clear memory of this photographer, who was most unintrusive). With my Racoon River Brewing Co. tee-shirt I brought back from Des Moines. And was wearing in a very indirect allusion to the US primaries the night before!
Rachel’s #1 sunrise in Des Moines
Posted in Kids, pictures, Running, Travel with tags bridge, Des Moines, Iowa, sunrise on November 4, 2012 by xi'an
