Kahneman next writes about his collaboration with Gary Klein, a colleague who did not agree with his work on experts and was deeply skeptical about the value of using rigid algorithms to replace human judgment. Klein had studied the expertise of firefighters and analyzed how experienced professionals develop intuitive skills. Kahneman invited him to join in an effort to try to discover when one can trust an experienced professional who claims to have prediction abilities.
Kahneman’s and Klein’s work centered on the question not only of when one can trust an experienced professional’s prediction abilities, but essentially when one can trust those professional’s intuitions. As Kahneman has demonstrated, intuition is deeply flawed, and so correct predictions must be able to pass certain tests of consistency.
Kahneman’s view of intuition was formed by observing the illusion of validity with his own work and Meehl’s work. Klein’s views were shaped by his work studying firefighting teams. The commanders could draw on patterns that they had learned and could intuit quickly the best option to fight a fire, without having to generate other options.
Klein’s view of intuition arose from studying different kinds of experts. Whereas Kahneman looked at people who claimed to be able to find patterns in random situations (particularly in the future), Klein’s studies focused on people who encountered similar environments over and over again.
Klein calls this the recognition-primed decision (RPD) model, which also applies to experience in other domains like chess. In it, a tentative plan comes to mind by the associative memory in System 1. The next phase is a deliberate process in which the plan is checked by System 2. Kahneman believes that this kind of intuition is really recognition of information stored in memory.
Kahneman demonstrates that Klein’s work blurs the line between intuition and memory. If a person has encountered a scenario before, instead of relying on intuition to come up with a new plan, they recall the previous situation.
Certain types of intuitions are stored in memory very quickly: a dubious dish that leaves us hesitant to return to a restaurant, or a bad experience on a certain street that causes us to remember it when we pass. This emotional learning is quick, but expertise takes a long time to develop. An expert chess player can understand a complex position at a glance, but it takes years (studies show at least 10,000 hours) to develop that kind of ability.
In the same way that we recall visceral stories in the news more than we recall mundane (if more frequent) occurrences, emotional memory is easier for System 1 to recall than complex situations that we should learn from. Perhaps this is because System 1 usually handles emotions rather than complex thought processes, and so storing and recalling emotions is easier for that process.
Becoming an expert at chess can be compared to learning to read. A first grader works hard to recognize letters and parse syllables, but adult readers can perceive entire clauses automatically. Chess is harder than reading because there are more letters in the “alphabet” of chess and longer “words,” but eventually chess masters can read situations instantly.
Reading is a good example of how, with practice, processes that are initially the domain of System 2 gradually become less effortful, more automatic, and move into the domain of System 1.
Kahneman and Klein realized that their disagreements stemmed from the fact that they had different experts in mind. Klein had worked with firefighters and nurses, while Kahneman worked with financial traders and political scientists. Klein argued that true experts know the limits of their knowledge; Kahneman argued that there are many “pseudo-experts” who don’t know what they don’t know.
Klein’s point that true experts know the limits of their knowledge is comparable to some of the ideas that Kahneman has expressed: much of human overconfidence stems from the fact that we do not know what we do not know. Theoretically, those who understand their limits do not fall victim to that overconfidence.
Kahneman concluded that there are two conditions to acquire real expertise: an environment that is sufficiently regular so as to be predictable, and an opportunity to learn these regularities through practice. Bridge and poker players, nurses, physicians, athletes, and firefighters all satisfy these conditions. But these professionals use highly valid cues to make predictions, while political scientists make long-term forecasts that often have no valid cues.
The difference that Kahneman finds between these two groups largely lies in the situations they regularly encounter. Expert firefighters, nurses, and poker players commit fewer heuristic fallacies because they have had the opportunity to practice and learn from their mistakes, unlike the people that Kahneman has largely observed.
Meehl’s work demonstrated that algorithms do better than humans, but even the algorithms did not have extremely high accuracy because the situations it evaluated did not allow for that accuracy. Statistical algorithms can find weakly valid cues and use them consistently, but if a strong predictive cue exists, humans can usually find it. Thus, it is wrong to blame anyone for failing to forecast accurately in an unpredictable world. But it does seem fair to blame professionals for believing they can succeed at something impossible.
Kahneman recognizes that there are situations in which humans can do just as well as algorithms, particularly when there are predictive cues. But his main point is that in the absence of truly predictive cues (particularly when trying to forecast long-term events and economic trends), people vastly overstate or are overconfident in their abilities, leading to both mistakes and ways of misleading the public.
There are a few conditions one can meet to become an expert, and those are immediate and unambiguous feedback as well as sufficient opportunity to practice. Chess is good for these conditions. But surgeons can be much more proficient in some operations than others. Psychotherapists get good feedback from patients as they speak with them, but do not have a chance to get feedback on the success of long-term treatments until many years later, and the results are often ambiguous.
Kahneman makes a distinction not only between which “experts” can be trusted and which cannot, but also different expert judgments within a given field that can be or cannot be trusted. There are aspects of some jobs in which people can become experts while others do not allow for that expertise, as in the example of the psychologists.
The conclusion that Kahneman and Klein came to is that, for the most part, it is possible to distinguish intuitions that are likely to be valid from those that are likely to be bogus. If the environment is regular and the judge has had a chance to learn its regularities, System 1 will recognize situations and generate quick and accurate predictions and decisions.
Part of Kahneman’s intention in the book is not only to try to help people identify their own overconfidence, but also to allow them to recognize the overconfidence of other people and to know when it is likely to lead to mistakes.