How to Actually Change Your Mind

The reason evidence bounces off, Yudkowsky argues, is rarely a lack of intelligence. It's that we reason toward a conclusion we've already chosen. He draws a sharp line between two mental postures that feel identical from the inside: asking whether a claim is true, and asking whether we're allowed to believe it. When we like a conclusion, we ask "can I believe this?" and go looking for a single supporting reason. When we dislike one, we ask "must I believe this?" and hunt for any excuse to dismiss it. Same person, same evidence, opposite standards.

What makes this dangerous is how competent it looks. Smart people are not immune; they're better armed. Give a clever mind a conclusion it wants and it will generate sophisticated arguments, cite real studies, spot genuine flaws in the opposition — all in the service of a verdict reached before the reasoning began. Yudkowsky borrows the term motivated skepticism: we apply rigorous scrutiny selectively, aiming it at the claims we'd rather reject and waving through the ones we'd rather keep. The intelligence isn't finding the truth. It's building the defense.

Against all this, Yudkowsky sets a discipline borrowed from probability, and it turns on a single word: update. Belief, in his framing, isn't a switch flipped between true and false. It's a dial — a degree of confidence that should move a little every time new evidence arrives. Strong evidence moves it a lot, weak evidence a little, and nothing should move it not at all. The Reverend Thomas Bayes hovers over these pages, less as a formula to memorize than as an attitude: the right amount to change your mind is exactly as much as the evidence warrants, no more and no less.

This reframes what a mistake even is. The failure isn't holding a belief that later turns out wrong; the world is uncertain and everyone will be wrong sometimes. The failure is refusing to move the dial when the evidence says you should, or lurching all the way from one certainty to another because the intermediate state feels uncomfortable. Yudkowsky is suspicious of the dramatic conversion, the total reversal, precisely because it usually means someone was overconfident before and is overconfident again. Honest updating is quieter, incremental, and rarely photogenic.

Step back from the individual tricks and a larger claim comes into focus: for Yudkowsky, rationality is not a body of facts you acquire but a set of habits you train against yourself. This is why the book reads less like a textbook than like a manual for a practice. The biases it catalogs — motivated skepticism, floating beliefs, the seductive click of false certainty — aren't errors you learn about once and then avoid forever. They're pulls that operate every time you think, and the only defense is a discipline exercised in the moment, repeatedly, on your own live reasoning.

What gives the project its edge is that the training runs against the grain of how good it feels to be right. Yudkowsky keeps returning to the moment of losing an argument, because that's where the practice is either real or fake. Anyone can update toward a conclusion they wanted. The skill is updating toward one they didn't — feeling the sting, recognizing it as the sting of learning something, and moving anyway. He frames this almost as an athletic thing: a muscle for admitting error that gets stronger only through use, and that atrophies the instant we start treating our beliefs as things to defend rather than things to check.

The book ends where it began, back at that sideways slide of the mind meeting an inconvenient fact. What Yudkowsky has done across its essays is refuse to treat that moment as a character flaw and instead pull it apart into mechanisms — beliefs that predict nothing, scrutiny aimed only at what we dislike, the certainty that arrives before the evidence does. Each has a name, and naming it is the first handhold. The change of mind he's after is not a grand conversion but a small, honest adjustment made against the grain of wanting to be right.