Why can a positive medical test still be probably wrong?

The base rate fallacy is the mistake of weighting specific information too heavily while forgetting the bigger picture. In medical testing, people see a positive result from a reliable test and conclude they almost certainly have the disease, without asking how common the disease is. When a condition is rare, a positive result can still be more likely wrong than right. This is sometimes called base-rate neglect. [1] [2]

A Stanford statistics lecture gives a simple case. A disease affects 1% of the population, and the test is 99% accurate. Someone who tests positive may feel certain they are ill. Applying Bayes' theorem, the chance they actually have the disease is only about one in two. The test is right 99% of the time in both directions, but the many healthy people tested produce about as many false positives as the few sick people produce true positives. [1]

The Stanford Encyclopedia of Philosophy offers a drug-testing version. In a population where 3% use heroin, a test with 95% sensitivity and 90% specificity gives a positive result. The probability that the person is a user is about 0.227. That is more than seven times the population baseline, yet it still leaves the person unlikely to be a user. [3]

Sensitivity and specificity describe how the test behaves given the true condition. A patient wants the reverse: the chance of the condition given the test result. These are different quantities. A medical maths tutorial shows the error with an invented condition with 5% prevalence, a test with 99% sensitivity and 85% specificity. People tend to guess that a positive result has a 15% chance of being false, but the correct figure is about 74%. In a group where nobody has the condition, every positive result would be false. [4]

The Stanford philosophy entry distinguishes incremental from total evidence. A positive result does raise the probability of disease, which is incremental evidence. But the total evidence can remain weak because the low base rate swamps the result. The fallacy is mistaking the first for the second. The entry cites Kahneman and Tversky (1973) for this finding about human judgment. [3]

Scientific American describes this as the false positive paradox. If you are picked at random for testing, your pretest probability is the prevalence in the general population. Doctors usually order tests when there is reason for suspicion, such as a distinctive rash and high fever. That puts the patient in a smaller group where the disease is far more common, so a positive result is more indicative of a true case. [5]

The same article says this is why mass screening for rare diseases is generally not done. When the base rate is small enough, even highly accurate tests produce more false positives than true positives, and the benefit of catching a few cases can be outweighed by medical, financial and psychological costs. It stresses that the lesson is not to dismiss medical tests, but that interpreting them takes statistical literacy. [5]