Nate Silver is not a witch, the frequentist says. A response to “Is Nate Silver a witch?" published on Monday 12 November by Linda Wijlaars.
In her web article “Is Nate Silver a witch?”, Linda Wijlaars makes fun of frequentist statisticians because, according to Wijlaars, they are not as cool as Bayesians. Indeed, the Bayesian approach has many benefits. However, I differ from Wijlaars in her explanation and interpretation of frequentist statistics. Here, I shall explain that by employing a proper frequentist approach, one can show that Bayesians and frequentists are just as cool. The main flaw in Wijlaars’ explanation lies in her argument “Bayesian statistics differ from frequentist statistics in that it takes prior knowledge into account when putting a probability on an event”, suggesting that frequentists do not use prior knowledge. This is, obviously, incorrect. (They might not (always) use prior belief, but there is a difference between belief and knowledge.) The following is a perfectly acceptable frequentist way to compute the probability of Nate Silver being a witch.
One of the frequentist definitions of probability is relative frequency in a finite population. This definition is conceptually and intuitively appealing: suppose that a small village with N = 100 inhabitants consists of 56 women and 44 men. Then, the probability that a villager, selected completely at random, is woman is 56/100 = 56%. Even Bayesians often adhere to this line of thought.
Now, let’s find out whether Nate Silver really is a witch or not. If we work with the settings that Wijlaars sketched for her sample, the population consists of all 8,244,910 New-Yorkers, of which 3,023 are self-proclaimed witches. Witches are expected to have magical abilities enabling them to give perfect prediction of future events. (Unfortunately, this only applies to real witches and not to self-proclaimed ones. If it was that easy, I’d proclaim myself a witch, run to the nearest betting shop, win millions and retire to some tropical island. But for the sake of argument, let’s play along. All types of witches, even self-proclaimed ones, can predict the future.) Thus, a witch can predict the outcome of 7 fair coin tosses perfectly: the probability of scoring 7 out of 7 is 100%. A non-witch has to guess, leading to a probability of 1/128 (slightly less than 1 percent).
If all New-Yorkers were asked to predict 7 coin tosses, then the following would happen. Out of the 3,023 witches, all 3,023 would score 7 points and none would not. Out of the
8,244,910 – 3,023 = 8,241,887 non-witches, (on average) 8,241,887/128 ≈ 64,390 would – by pure coincidence – score 7 points (and over 8 million would fail to be so lucky). Of the 3,023 + 64,390 = 67,413 people that predicted perfectly correct, 3,023 are witches.
This is a relative frequency of
Every sensible frequentist statistician would, in the context described by Wijlaars, thus derive a probability that Nate Silver is a witch of 4.5%. This is exactly the same probability as the Bayesian approach outlined by Wijlaars gives. (Wijlaars arrives at 0.05, but that difference is due to rounding off errors.)
An example as outlined above, can (or at least, should) be found in any textbook on mathematical frequentist statistics. Usually the context is about urns with blue or red balls. Working with witches might be a good idea to get the large Harry Potter fanbase more interested in statistics.
I’ve shown above that in this example, there is no disagreement whatsoever between the sensible Bayesian and the sensible frequentist. Bayesians and frequentists are not as different as many statisticians (in both camps!) claim: if the data clearly points in a certain direction, the sensible statistician goes to that direction. There might be subtle differences between them but, when drawing a Venn-diagram depicting Bayesians and frequentists, the two ovals would overlap to a very large extent.
Wijlaars states that Bayesian statisticians are cool. I fully agree. Since frequentists come up with the same answers are Bayesians – in this case a probability of 4.5% percent that Nate Silver is a witch – frequentists must be cool as well. My prior belief is that Linda Wijlaars agrees to this point of view. The logical conclusion is that all statisticians are cool. Maybe not a very surprising conclusion to be found on a website of a statistical magazine...
Casper Albers is a lecturer in Psychometrics and Statistical Methods at the University of Groningen in the Netherlands.