On the Other Side of the Machine
A mathematician on algorithmic bias
Description
Around 2015, a US court system started leaning on a software tool to help judges estimate how likely a defendant was to reoffend. The tool, called COMPAS, scored people from one to ten. It looked neutral — just numbers, no prejudice, no bad day in the life of a tired judge. Then journalists at ProPublica pulled the records apart and found that Black defendants were far more likely to be wrongly flagged as high-risk, while white defendants were more often wrongly cleared. Nobody had written a line of code that said "treat these people differently." And yet the machine did.
Aurélie Jean is a French computational scientist who has spent her career building the kind of models that increasingly decide things about us — who gets a loan, who gets seen by a recruiter, which face a camera recognizes. In her book On the Other Side of the Machine, she does something unusual for someone who writes algorithms for a living: she refuses to defend them as neutral. An algorithm, she keeps repeating, is not objective just because it is made of math. It carries the fingerprints of whoever built it, and of whatever data it was fed.
That is a disorienting idea, because we tend to imagine software as the opposite of human messiness — cold, fair, above the fray. Jean's whole project is to take us to the other side of the screen, where the code is written and the data is chosen, and show that bias doesn't sneak in from outside. It is assembled, step by step, by people who mostly meant well. The interesting question isn't whether machines can be biased. It's how, exactly, the bias gets in — and who is holding the pen.
The question we’re asking : How does bias actually enter an algorithm — and who is responsible for it once it does?What we’ll see : We follow a mathematician behind the screen, into the data, the design choices, and the quiet decisions that turn neutral-looking code into something that discriminates.
Table of contents
01 Chapter 1 — The engineer who stopped trusting the model
Aurélie Jean trained as a mechanical engineer and earned a doctorate in materials science and computational mechanics, then spent years at MIT modeling things like skull fractures and blood flow — problems where a model that is slightly wrong can mislead a surgeon. That background matters to how she reads the current wave of enthusiasm around algorithms. She has watched the same mathematical tools migrate from physics, where they describe atoms and bones, into the social world, where they now sort résumés and rank human beings. The math didn't change. What changed is what we point it at.
Her central move in the book is to strip the algorithm of its halo. We talk about "the algorithm decided" as if some neutral oracle had spoken. Jean insists this is a category error. An algorithm is just a sequence of instructions a human wrote to solve a problem a human defined, trained on data a human collected. At every one of those steps, a choice was made — and a choice is exactly where a bias can live. The model does not float above us. It is a mirror, and it reflects back whatever we hold up to it.
02 Chapter 2 — Where the bias actually gets in
To see how bias enters, Jean walks through what a model actually does. Take a hiring algorithm trained to spot good candidates. You feed it thousands of past hires — who was recruited, who succeeded, who was let go — and it learns to recognize the patterns that separate them. The trouble is that the past is not neutral. If a company hired mostly men for a given role over twenty years, the data encodes that history, and the model dutifully learns that "looks like a successful hire" correlates with "is a man." Amazon reportedly ran into exactly this and scrapped an experimental recruiting tool in the late 2010s for penalizing résumés that mentioned women.
The crucial thing is that nobody has to be prejudiced for this to happen. No engineer typed a rule about gender. The bias came from the data, which is a record of human decisions already made. Jean calls this the difference between what we intend and what we encode. We think we are teaching the machine about merit; we are actually teaching it about our own past behavior, including the parts we would rather not repeat.
03 Chapter 3 — The algorithm is never the whole story
Once the model is built and shipped, a new problem begins: it starts shaping the very world it was meant to describe. Jean is attentive to these feedback loops, where a biased output quietly becomes a biased input down the line. A predictive-policing model that sends more patrols to a neighborhood will record more arrests there, simply because more officers are looking. Those arrests feed back into the data, confirming the model's hunch and sending even more patrols. The machine was wrong, and its being wrong made it look right.
This is why she resists the idea that we could just "remove" bias with a clever fix and be done. Bias is not a single bug to be patched; it is woven through the pipeline, from the data to the design to the deployment to the loop that follows. Deleting the obvious variable — say, race — often does nothing, because the model finds proxies for it: a postal code, a name, a shopping pattern. Strip out one signal and the correlation reroutes through another. The discrimination survives the surgery.
04 Chapter 4 — The hand behind the code
Step back from any single case, and Jean's book is really an argument about responsibility. If the machine only ever reflects the choices poured into it, then the comforting fiction that "the algorithm did it" collapses. There is no it to blame. There is a team that chose the training data, an engineer who set the objective, a company that decided where to deploy the tool and what level of error it could live with. Bias is not an accident that befalls a model from the outside. It is a trace left by people, which means it is something people can be answerable for.
This reframing is the quiet radical move of the book. We are used to treating code as a kind of nature — a force that simply exists, to be feared or admired. Jean treats it as an artifact, as human as a law or a building, and therefore subject to the same scrutiny. We do not accept "the bridge collapsed on its own"; we ask who designed it and whether they did their job. She wants the same standard for algorithms. The engineer is not a neutral conduit for the math. The engineer is the author of a set of consequences.
05 Conclusion
Go back to the risk score that quietly told a judge how dangerous a human being was. The unsettling thing was never that a computer had an opinion. It was that the opinion had an author nobody could see — buried in the historical data, the chosen variables, the thresholds set by a team that will never meet the person being scored. Jean's whole book is an effort to drag that author back into the light, to remind us that behind every output there was an input, and behind every input a choice.
Related books
