Noise: A Flaw in Human Judgment

This chapter draws upon the research for Thinking, Fast and Slow, which analyzed the benefits and pitfalls of intuitive thinking. This so-called “System 1” thinking relies extensively on heuristics, which are simplified forms of problem-solving that use short cuts. While System 1 thinking can be efficient in some circumstances, in more complex situations it can lead to judgment errors. Such psychological biases create system noise which leads to error.

Researchers define psychological biases by “observing that a factor that should not affect judgment does have a statistical effect on it, or that a factor that should affect judgment does not” (190). All psychological biases contribute to statistical bias and noise.

One reason for psychological bias is that people confuse similarity with probability and often use heuristics to answer complex questions about predicting future outcomes by replacing them with a simpler related question. For example, faced with the question of whether they believe in climate change, people are likely to substitute: “Do I trust the people who say it exists?” (196). However, this replacement leads to psychological biases and answers that do not “give different aspects of the evidence their appropriate weights,” resulting in errors (196).

Taking “the outside view” means that you can ignore the irrelevant statistics present and consider a problem from a different logical standpoint (195). The authors redirect the readers’ attention to the problem of guessing whether Michael Gambardi would retain his CEO job for two years. While System 1 thinking would get lost in the statistics, taking the outside view would allow the reader to perceive the unlikelihood of a randomly appointed CEO lasting the course of two years. The authors argue that taking the outside view can prevent errors.

Another form of psychological bias emerges when judging the frequency of how often a particular event happens—for instance, a devastating hurricane—by how easy it is to think of examples. Thus, in the aftermath of an abundantly documented hurricane, people’s perception of the frequency of hurricanes rises.

Often, not liking a particular outcome translates to not believing in its validity, and this inevitably motivates judgment. The authors call this “conclusion bias, or prejudgment,” which is a product of System 1 thinking (197). However, individuals can also activate System 2 thinking to come up with arguments for their prejudgment, thus mobilizing an arsenal to combat these error-prone thought patterns. Conclusion bias is rooted in what psychology professor Paul Slovic terms the “affect heuristic,” whereby people engage in emotional reasoning (199).

“Excessive coherence,” in which “we form coherent impressions” of what something or someone is like quickly and cannot easily change them, is another example of psychological bias (201). Thus, hearing that a prospective candidate has two positive qualities in quick succession biases an individual in their favor before hearing the negative qualities.

As all types of psychological bias cause noise, this suggests that measures to reduce bias will result in better judgment.

Individuals engage a process of matching in order to assign predictions to the evidence they encounter. This occurred in the previous chapter in the case study of mathematical, unimaginative Bill, who was matched to the appropriate profession of accountant. However, the authors reveal that Bill is an explorer rather than an accountant. The surprise the reader experiences on learning this demonstrates “the failure to achieve coherence,” especially given the incompatibility of the truth with the descriptors received (207).

Individuals are likely to use substitution to swap difficult questions for ones that are easier to answer, given the evidence available. This makes people overconfident in predicting uncertain outcomes and leads to errors. The authors ask readers to estimate the GPA of a college student named Julie’s GPA based on her early reading age. The authors argue that “the optimal prediction must lie between these two extremes of perfect knowledge and zero knowledge” (212).

While matching predictions are popular, they are not universally used, especially when it comes to unfavorable evidence. For example, the authors speculate that readers would be slower to estimate that Julie would have poor college performance if they had learned that she was a late reader. Again, the authors offer the outside view as “a corrective for intuitive predictions of all kinds” (213). Thus, in Julie’s case, taking the outside view involves stating that the evidence relating reading age to college academic performance is insufficient; rather, using the average GPA to make one’s prediction is more effective. The authors argue that the outside view should only be ignored when the evidence allows for a confident prediction.

Aligned to matching is the element of comparison. People find it easier to make comparisons between the things they are judging than to evaluate them each in turn. Thus, experiments that compel subjects to make explicitly comparative judgments have noise-reducing properties. Conversely, using the wrong scales of measurement contributes to noise.

Chapter 15 analyzes the contribution of “the response scale” as a source of noise. People’s judgments often diverge owing to their different uses of the response scale (220). For example, one person might decide that “pretty good” performance at work deserves a score of 4 out of 6, while another person might think 3 out of 6 has the same meaning.

This ambiguity also occurs in the punitive judgments distributed by juries, where jurors typically assign the numerical value of how long a prison sentence should be, or how many dollars in compensation a corrupt company should pay, according to their level of outrage. In doing this, they substitute the hard task of determining a fair punishment with the easier task of determining their level of anger. Thus, there is ample noise amongst jurors, who may all be angry with the actions of the defendant, but each comes up with vastly different calculations of damages.

Studies have shown that people find it easier to come up with a numerical value when they are given an anchoring figure. They then typically produce their own figure in response to the given one. Replacing the need to make absolute judgments with relative ones also reduces noise. Being able to rank crimes in relation to one another allows jurors to achieve a greater level of concordance, and this in turn better equips them to agree on the numerical compensation a company should pay.

Complexity and conflicting clues lead to the production of noise, as each person tries to produce a coherent narrative from contradictory fragments. Individuals can make confident-sounding judgments from stories where the information is comprehensive with few contradictions, and they can also do this “by ignoring or explaining away whatever does not fit” (236). The authors argue that experts are versed in defending their own interpretations of a story, while also having equal fluency in explaining why other interpretations are invalid. However, as proven in earlier chapters, subjective confidence in one’s judgment does not determine accuracy, and people can mis-labor under the “illusion of agreement,” where they believe that others must also think like them (236).

The text then moves to define a pattern error, whereby an individual’s particular reactions to specific cases produce recognizable patterns. For example, a judge who is normally harsh may deviate from their pattern when it comes to sentencing young women. This variation within the individual emerges when something about the situation speaks to their personal situation or values. In addition to this form of stable pattern noise, which tends to bias the judge in an encounter with a particular type of case, there is also the occasion noise of their mood on the day of sentencing and the media they have recently been exposed to. Random recent exposure to content that is relevant to a case thus may influence a judge’s decision one way or the other.

Personality differences account for differences in judgment making. For example, a more aggressive person might be expected to behave in a certain way in certain situations. However, research has shown that it is more helpful to look at how certain facets of people’s personality influence their decisions in particular situations. This is arguably another form of pattern noise.

During the course of their research, the authors determined that the greatest part of noise is the product of neither deeply held personal biases (level noise) nor transient random factors (occasion noise). Instead, stable pattern noise, defined as “the persistent personal reactions of particular individuals to a multitude of features,” is the most significant reason for judgment errors (247).

To separate pattern from occasion noise, the researchers would have to request that the same judge adjudicate the same case twice. Given the impossibility of this predicament, a research group led by psychologist Alexander Todorov recruited a group from Amazon Mechanical Turk to rate the same group of faces according to traits such as trustworthiness twice, within the same week. The study showed that while the participants did not agree about which faces exhibited certain personality traits, they remained consistent in their judgments from one week to the next. Thus, stable pattern noise was evident in both the participants’ variance from each other and their consistency within themselves.

Employers can compensate for level noise amongst their staff by hiring different types of people. For example, if a professor in a particular college department is an especially harsh grader, more lenient ones can be hired to redress the balance. However, it is harder to compensate for pattern noise which makes up the greatest percentage of noise and “is mostly a product not of level differences but of interactions: how different judges deal with particular defendants” (252). It is difficult to address this type of noise because it not easy to parse it into the stories of cause and effect that people are most comfortable with. Instead, being statistical in nature, pattern noise only becomes visible when examining a group of similar judgments. Individuals can train themselves to think statistically and become better at spotting different types of noise.