Noise Dampeners
We live and work in a very noisy world. And not the kind that is eased by good headphones.
That’s the view of three social scientists, Daniel Kahneman, Oliver Sibony, and Cass Sunstein. They are the authors of Noise: A Flaw in Human Judgment. Kahneman, a Nobel Prize-winning psychologist, has previously studied systematic biases and their origins. Bias describes the poor decisions produced by systematic deviations, and Kahneman’s co-authors, Sibony and Sunstein have previously unpacked how over-confidence, stereotyping, and confirmation bias all work to produce outcomes that consistently deviate from rationality or justice. Now, they have turned their attention to another major source of error which they label “Noise” — the random scatter pattern of inconsistent decision-making that produces so many bad choices in everyday life.
In some respects, “Noise” is perhaps even more worrying than bias. Biases flow from systemic patterns that can be easier to spot and harder to deny. “Noise” precisely because of its eccentric character is harder to eradicate, and in some respects seems embedded in human fallibility. The authors explain “noise” primarily in terms of non-standardised criteria. Essentially, too few of us think hard enough about the criteria we use to make decisions. While specialists from different backgrounds might be likely to bring different standards to bear, what this book reveals is that those same specialists within their own field apply their own standards inconsistently. One, albeit small-scale, study had 22 heart doctors, evaluating the same angiograms on two occasions several months apart, and found that they disagreed with themselves between 63 and 92 percent of the time! Among explanations for this were mood, fatigue, and physical environment. You know the phrase “everybody can have a bad day at the office” — well, that’s not a great consolation if it’s your heart problem that is being evaluated!
The remedy offered in Noise is essentially to extend the principles and processes used against bias. The authors suggest various ways in which organisations can undertake “noise assessment” and introduce “noise reduction” policies. People working in Human Resources will already be familiar with the protocols that try to ensure that the job interview process is conducted in ways that standardise questions for applicants and tie assessment to explicit criteria. This can be improved further by having a decision observer, tasked solely with evaluating the process of evaluation. Applying the same principles to medical diagnoses would require a similar expansion of both time and resources. We need more experts checking experts who might be having a bad day.
A common feature of all such systems is to invite assessors to give a clear numerical value or score to each criterion. Applied to medical diagnosis, this might mean weighting different indices to ensure that features that are very strongly correlated to a specific diagnosis are more influential than other indices in determining final judgements. The book also features disturbing disparities in sentencing practices by judges, and reports that judges themselves have successfully challenged the imposition of stricter guidelines that would diminish their discretionary powers. And this is a key part of the problem. Almost everybody in a position to make choices likes to retain the principle that they have discretion: they have a choice.
Noise is a book that warns that the outcome of such freedom can be demonstrably bad decisions. It reports college admissions officers who gave more generous offers to applicants with strong sports or other outdoor interests on days when the weather was particularly fine, but who favoured the indoor and studious types on overcast days! And it reports courts where the judge’s sentencing pattern seemed to parallel the performance of his local team. This is not an ideal way to determine people’s futures.
But before we all reach for the mute button on our noisy world, just be aware that what the authors want is a process that is super-diligent in its stages but still allows for nuanced judgement at the end. They also warn that their solution is aimed primarily at the organisational world rather than anyone’s personal life. It may well be that there would be advantages to having clear criteria with numerical scores when drawing up the weekly shopping list with attention systematically paid to health concerns, ecological or ethical considerations, price, and what’s in season. And that might counteract the concerted efforts of marketing to induce you to buy what is conspicuously at your eye level in the supermarket, but for most of us, there is also a need for some serendipity in our lives; even if it is only the sudden thought — “ooh, I haven’t tried that in a while.”
But if you do go to the supermarket hungry and without a list, the noise you sense is not just your stomach grumbling. And, if you are a sports-mad judge, please recuse yourself when your team is on a losing streak. Finally, don’t be afraid to say: “Doctor, d’you mind looking at my chart again, please?”
