Bayes' Theorem

Strip away the notation and Bayes's theorem is a procedure, not a mystery. We start with a belief we already hold, expressed as a probability — the prior. New evidence arrives. We ask one precise question: how likely was this evidence if our belief were true, compared with how likely it would be if it were false? Then we revise, landing on an updated belief — the posterior. The prior goes in, the evidence pushes, the posterior comes out. Next time, that posterior becomes the new prior. Belief as an ongoing account, debited and credited as the facts come in.

The decisive ingredient is the one people forget: the comparison. Evidence never speaks on its own. A positive clue is only as strong as the gap between how often it shows up when you're right and how often it shows up when you're wrong. A symptom that appears in the sick but also in half the healthy tells you very little, however alarming it feels. Bayes forces that comparison into the open, where a gut reaction tends to grab the first number it sees and run.

Here is the scenario that trips up patients and doctors alike. Suppose there's a disease that affects one person in a thousand. There's a test for it, and it's a good test: if you have the disease, it comes back positive 99 percent of the time, and if you don't, it correctly comes back negative 99 percent of the time. You take it. It's positive. How worried should you be — what are the odds you actually have the disease?

The intuitive answer is 99 percent, and it is badly wrong. Walk it through with Bayes. Imagine a hundred thousand people like you. About a hundred of them have the disease, and the test catches ninety-nine of those. But the other 99,900 are healthy — and a 99 percent accurate test still misfires on 1 percent of them, which is roughly 999 false positives. So among everyone who tests positive, you have ninety-nine true cases sitting next to about 999 false alarms. Your chance of actually being sick, given that positive result, is around nine percent. Not ninety-nine. Nine.

Step back from the clinic and the deeper point about Bayesian reasoning comes into focus. The error in the medical case wasn't arithmetic — the test numbers were right there. The error was leaving out the base rate, the dull background fact about how common the thing is before any evidence arrives. And that omission is not a quirk of probability class. It is one of the most reliable ways the human mind goes wrong. We are moved by the vivid specific — the alarming result, the striking anecdote — and we quietly discard the unglamorous prior that should have anchored us.

Bayes makes the uncomfortable thing explicit: you cannot interpret evidence without a prior, and pretending you don't have one just means you're using a bad one by default. This is what made the method suspect to generations of statisticians. Where does the prior come from? Isn't it subjective? It can be. Two people with different starting beliefs will read the same evidence differently — which sounds like a flaw until you notice it's simply true of how understanding works. The honesty of the Bayesian move is that it puts the starting point on the table instead of smuggling it in.

Thomas Bayes never saw any of this. He died with his essay unpublished, a minor figure in a market town, and it took a friend rummaging through his papers to carry the idea into the world at all. The thing he worked out was never really about coins or marbles; it was about the relationship between what we believed yesterday and what we're entitled to believe now that something new has happened. That relationship turned out to be everywhere, which is why a two-hundred-and-sixty-year-old fragment ended up inside spam filters, search algorithms, and the way careful doctors read a result.