The machinery of this course is complete and correct, and it is misapplied constantly, in courtrooms, in laboratories and in ordinary judgement. Each failure below has cost real people something, each is a specific rule from an earlier lesson being skipped, and each has a check that takes under a minute. This is the lesson that makes the rest usable outside an exam.
Ignoring the base rate
The first failure is already familiar. A test that is right nine times in ten, applied to a condition present in one person in a hundred, produces mostly false positives, and the Bayes lesson computed the classic case: physicians shown a positive mammogram estimated percent when the answer was percent.
The pattern is more general than testing. Whenever you assess how likely a hypothesis is given some evidence, and reason only from how well the hypothesis explains the evidence, the base rate has been dropped. Someone quiet and tidy who likes order is more representative of a librarian than of a farmer, and there are far more farmers, so the probability calculation runs opposite to the resemblance. The evidence has to be weighed against how common the alternatives are, which is exactly what the denominator of Bayes' theorem does.
The check: state the prior odds out loud before looking at the evidence, or restate the problem in whole people out of ten thousand. Both make the size of the comparison group visible, and the size of the comparison group is the thing being forgotten.
Example. A city has terrorists among million residents. A surveillance system flags percent of terrorists and percent of everyone else. Someone is flagged. What is the probability they are a terrorist?
True positives: . False positives: . So the probability is , under one percent, and the system generates a hundred false alarms for every real one.
Now you. The same system is improved so that it flags only percent of non-terrorists, with the same percent detection rate. What is the probability now?
Answer
False positives fall to , so the probability is , about nine percent. A tenfold improvement in specificity still leaves nine false alarms in ten.
The prosecutor's fallacy
Confusing with has a name in law, and a famous case. In 1999 Sally Clark was convicted of murdering her two infant sons. The paediatrician Roy Meadow testified that the chance of two cot deaths in a family like hers was about in million, obtained by squaring a single-death rate of in : .
Two errors, one of them the subject of this lesson. The first is that squaring assumes the two deaths independent, and they are not: siblings share genes, environment and sleeping arrangements, so a family that has suffered one cot death is at raised risk of another. The Royal Statistical Society said so publicly in October 2001.
The second error is the fallacy proper. Even granting the figure, in million is , and the question before the court is . Bayes says the second depends on the rarity of the alternative, and double infant murder by a mother is also extremely rare. Ray Hill's 2004 analysis of English birth and death records estimated that double cot death is between and times more likely than double murder, so the likelihood ratio points towards innocence before any other evidence is considered. Clark's conviction was quashed in January 2003, after she had served more than three years. She died in 2007.
The check: whenever a small probability is quoted about evidence, ask "small compared with what?" A number of the form says nothing on its own; only the ratio to carries information, which is the whole content of the odds form of Bayes' theorem.
Regression to the mean
Francis Galton measured the heights of parents and their adult children and published the result in 1886. Tall parents had tall children, but on average less tall than themselves; short parents had short children, less short than themselves. He called it regression towards mediocrity and thought he had found a biological force pulling the population back to average.
There is no force. Any measurement that is part signal and part noise behaves this way. An extreme value is extreme partly because the underlying quantity is extreme and partly because the noise happened to point the same way, and the noise does not repeat. So the second measurement is closer to the mean, mechanically, with no explanation required.
The consequence is that regression gets mistaken for cause everywhere. Israeli flight instructors, in an example Daniel Kahneman recounts, observed that pilots praised after an unusually good landing did worse next time, and pilots criticised after a bad one did better, and concluded that criticism works and praise backfires. Both groups were simply regressing. The same trap catches the patient who took a remedy at their worst and improved, the school placed in special measures after a bad year, and the fund manager hired after a spectacular one.
The check: before attributing a change to an intervention, ask whether the group was selected for being extreme on a noisy measure. If it was, some improvement was guaranteed, and the intervention has to beat that baseline rather than beat zero. A control group selected the same way is what settles it.
Example. A test has a true-score component and independent noise of equal variance. Students are selected for scoring in the top few percent, then retested. Should their average score fall?
Yes. Their high score came partly from a high true score and partly from favourable noise, and the noise is redrawn on retest with mean zero. The expected retest score sits between the selection score and the population mean, so the average must fall, with no learning, forgetting or coaching involved.
Now you. A hospital sees an unusually high infection rate one month, introduces a new hand-washing policy, and finds the rate lower the next month. What is the flaw in concluding that the policy worked?
Answer
The month was selected for being extreme on a noisy measure, so a fall was expected regardless. Concluding anything requires comparison with a similar unit that did not adopt the policy, or a longer run of data.
Looking in many places at once
Run one test at the conventional five percent threshold and there is a one in twenty chance of a false positive. Run twenty independent tests and, if nothing is really there, the probability of at least one "significant" result is
so a false positive is the likely outcome, not the unlikely one. At a hundred tests it is , effectively certain, and the expected number of false positives is .
This is the union bound of the second lesson, doing its usual work: the chance that at least one of many independent unlikely things happens is much larger than any of them individually. It is why testing twenty colours of sweet against acne and reporting the one that came out significant is a fraud even when every individual calculation is correct, and why the same practice under a respectable name, trying several outcome measures and reporting the best, produced a decade of published results that would not replicate.
The two standard repairs both come out of the same arithmetic. Divide the threshold by the number of tests, so twenty tests each at keep the overall false positive rate near five percent, which is the Bonferroni correction and is exactly the union bound read backwards. Or state in advance which single test is the one that counts, so that there is nothing to correct for.
Example. A researcher runs independent tests at the threshold on data with no real effect. What is the probability that at least one comes out significant?
, so nearly a quarter of such studies would report a finding.
Now you. How large must the number of independent tests be before that probability exceeds one half? Try a few values.
Answer
and , so fourteen tests suffice.
Counting only what came back
In 1943 the US Navy asked Abraham Wald's group to work out where to add armour to bombers, having recorded where returning aircraft had been hit: mostly on the wings and fuselage, rarely on the engines. The obvious reading is to armour the wings.
Wald's answer was the opposite. The data came only from aircraft that returned. Hits are presumably spread fairly evenly, so the near-absence of engine hits among survivors means engine hits are what stops an aircraft coming back. Armour the engines.
This is conditioning, and the failure is forgetting what the sample was conditioned on. The observed distribution is and the useful quantity is , which is the same inversion Bayes handles and the same one the prosecutor's fallacy botches. The pattern recurs wherever a sample is produced by a filter: successful companies studied for their common habits, when the failures had the same habits; a treatment that looks good because the patients it killed left the trial; a fund family whose average return excludes the funds that were closed.
The check: ask what would have had to happen for a case to appear in this data, and whether that condition is related to the thing being measured. If it is, the sample answers a different question from the one being asked.
Coincidences are common
The last failure is treating a coincidence as evidence, and the correction is the birthday problem generalised.
The chance that a specific named person shares your birthday is . The chance that some pair among twenty-three people shares one is , because there are pairs. Whenever an event is unlikely for each particular case, but there are a great many cases, the event becomes likely somewhere. Persi Diaconis and Frederick Mosteller called this the law of truly large numbers in 1989: with a large enough sample, any outrageous thing is likely to happen.
Put numbers on it. Something with a one in a million chance of happening to a given person on a given day happens to about people a day in a country of million, and about times a year. Lottery winners who win twice make headlines and are close to inevitable: Diaconis and Mosteller estimated that some double winner somewhere is essentially certain over a period of a few years, given how many people buy how many tickets, and Evelyn Adams of New Jersey duly won twice, in 1985 and 1986.
The mistake is the selection again. The coincidence was not specified in advance, and the number of ways something surprising could have happened is enormous. A probability computed after choosing which pattern to be surprised by is not a probability of anything.
The check: ask how many opportunities there were, and how many different coincidences would have struck you as remarkable. Multiply. The answer is usually that the surprising thing was due.
The habit underneath all of them
Six failures, and one shape. In every case a probability was computed conditional on something that was quietly dropped: the base rate, the alternative hypothesis, the noise in the measurement, the number of tests, the filter on the sample, the number of opportunities.
The habit that catches all six is to say out loud what the sample space is and what has been conditioned on, before computing anything. That is not a rhetorical flourish; it is the first lesson of this course, in the same order it was taught. Write the space, write what is known, and only then apply a rule. Every failure above survives only while the conditioning stays implicit, and none of them survives being written down.
A finisher of this subject can build a sample space, condition on what is known, invert with Bayes, name the distribution a mechanism produces, compute a mean and a spread, and say what a limit theorem does and does not license. The remaining work is statistics, which is this machinery run backwards: not what data a known mechanism produces, but what mechanism an observed set of data implies. That is a different subject, and it is not attemptable without this one.