Noise

In 1981, a study of 88 insurance underwriters evaluated the same two hypothetical claims. The median difference in their premium valuations was 55%. This wasn't a one-time anomaly. Another analysis of 57 experienced claims adjusters found their median valuation difference for identical cases was 43%. And in a 2016 study, 208 professional judges were asked to set bail for the same four defendants; their assigned bail amounts varied by an average of $500,000. These are not cases of intentional bias, where a group consistently leans in one predictable direction. This is something else: a chaotic, random, and expensive spread of professional judgments, a phenomenon the authors call "noise."

This puzzling and costly variability in decision-making is precisely what caught the attention of Daniel Kahneman, a Nobel laureate in economics and one of the most influential psychologists of our time. After co-authoring the seminal work Thinking, Fast and Slow, which focused primarily on cognitive biases, he stumbled upon the sheer magnitude of noise while working with an insurance company. He discovered that the random inconsistency among their employees' decisions was a far greater financial problem than the systematic biases they were trying to fix. This revelation prompted him to team up with Olivier Sibony, a professor and expert in corporate strategy, and Cass R. Sunstein, a renowned legal scholar. Together, they embarked on a mission to diagnose and provide a framework for understanding this overlooked flaw in human judgment.

Module 1: Unmasking the Twin Demons of Error

We often think of error as a single problem. But the authors argue it has two distinct components. Understanding this distinction is the first step toward making better decisions. The first component is bias, a familiar concept. It’s a systematic error, like a bathroom scale that consistently adds five pounds. The second, more elusive component is noise.

Noise is unwanted variability in judgments that should be identical. It's the scatter on the target. If you ask two doctors to diagnose the same X-ray, and they give different answers, that’s noise. If two judges give wildly different sentences for the same crime, that’s noise. Unlike bias, which has a clear direction, noise is random and unpredictable. This randomness makes it hard to spot. You only see it when you compare multiple judgments of the same case.

This leads to a critical insight. You can measure noise without knowing the true answer. Think of a shooting range. To measure bias, you need to know where the bull's-eye is. To measure noise, you just need to look at the back of the target. You can see how scattered the shots are without ever seeing the center. This is a powerful idea. It means we can study and quantify inconsistency in fields like hiring, performance reviews, or strategic forecasting, where a single "correct" answer is often unknowable.

The authors reveal that this problem is everywhere. In a study of insurance underwriters, executives expected a 10% difference in premium quotes for the same risk. A formal study, known as a noise audit, revealed the actual difference was a staggering 55%. One underwriter might quote $9,500 while another quotes $16,700 for the exact same case. This costs companies millions in lost business and underpriced risk. Similar levels of noise were found in medical diagnoses, asylum rulings, and even forensic science. The unavoidable conclusion? Noise is a far larger and more expensive problem than most organizations realize. It’s a hidden tax on every judgment we make.