The Trouble with Bayes' Theorem - The Simple and the Serious

Author: Graham Wheeler

Last week, the following probability problem was posted on a statistics internet mailing list:

"In a basic English course there are 5 men and 4 women. In the intermediate English course there are 7 men and 3 women. Finally in the advanced English course there are 4 men and 4 women. A student is randomly selected from the basic course and is transferred to the intermediate course. If a student is randomly selected from the intermediate course and that student is a man, what is the probability that the student transferred was a man?"

Three different answers were proposed by different contributors. But which one is correct?

a) 5/9

b) 40/99

c) 40/68

The argument for a) was that since the question only concerns the person that was transferred, the probability that the student transferred was a man is 5/9. Then someone else said that b) was correct, since this is the probability that a man is selected from the basic group and transferred to the intermediate group (5/9) multiplied by the probability that a man is selected from the intermediate group following this transfer (8/11). Both answers were from academic probabilists and statisticians. Both had verbal and arithmetic reasoning behind their answers. Both were wrong.

Nearly 250 years after its formulation,

statisticians still have trouble in deciding

when to use Bayes' Theorem

Image by mattbuck/Wikimedia Commons.

The correct answer is in fact c), which uses a simple rule known as Bayes' Theorem to calculate the correct probability. Bayes' Theorem states that for two events A and B, the probability of A happening, given that B has happened is equal to the probability that B happens given A has happened, multiplied by the unconditional probability of A happening, divided by the unconditional probability of B happening. Mathematically, we write this as

This simple theorem, named after the mathematician and Presbyterian minister Thomas Bayes, is the key to answering the above. Let A be the event that the student transferred is a man and let B be the event that the student selected from the intermediate course is a man. So, P(A) = 5/9, P(B | A) is 8/11 (since event A has already happened) and

i.e. P(B), the unconditional probability of B happening, is the sum of the probability of B and all the possible states of A occurring also.

So here, P(B) = (5/9 x 8/11) + (4/9 x 7/11) = 40/99 + 28/99 = 68/99. Therefore, the correct answer is

The information about the advanced English course is irrelevant here, but contrary to what some believe, the fact that we are told that a man has been selected (an event that has happened and cannot be changed) can be accounted for in our working out of the probability that a man was transferred originally.

This is not the first time that Bayes' Theorem has been overlooked in probability questions and nor will it be the last. An often referenced and extremely tragic example where Bayes' Theorem should have been used is in the case of Sally Clark.

Sally Clark was a British mother who was sentenced to life imprisonment in 1999 for the murder of her two infant sons after both died from Sudden Infant Death Syndrome (SIDS), or 'cot death' as it is also known. The misuse of statistics in several expert testimonies, including that of paediatrician Sir Roy Meadow, lead to the wrongful conviction of Clark and widespread uproar amongst many statisticians.

The use of Meadow's Law in legal proceedings related

to SIDS has caused much uproar amongst statisticians. 
Image by Roger Rössing available from Deutsche Fotothek

Meadow's incorrect calculations (a form of what is known as the Prosecutor's Fallacy) and analyses reflected what came to be known as the crude and unjust 'Meadow's Law': "One [SIDS death] is a tragedy, two is suspicious and three is murder unless there is proof to the contrary". The correct use of Bayes' Theorem, which was later used by Professor Ray Hill to overturn Clark's conviction in 2003, accounted for the rarities of both double SIDS and double murder, and also included the fact that the chances of SIDS deaths in children with the same mother are not independent.

Unfortunately, despite Bayes' Theorem being used post-conviction to weigh up the evidence presented and give the correct probability of Clark's innocence, leading to her subsequent acquittal, the damage had already been done. She never recovered from her harrowing ordeal, her four years in prison and the loss of the two sons. After years of severe alcohol dependency following her trial, Sally Clark died from acute alcohol intoxication in 2007. 

This use of conditional probability highlights some interesting misconceptions in probability theory. It may seem simple enough, but ignorance of it in the wider world is a serious cause for concern. It is one thing to not understand probability and get an exam question wrong; it is entirely another to not understand probability and pass a sentence of life imprisonment onto an innocent person.