Thinking, Fast and Slow: Part 3, Chapter 20 Summary & Analysis

Decades ago, Kahneman watched soldiers in the Israeli Army as they completed a group exercise. He and a colleague took note of who tried to lead, who was rebuffed, who seemed to be stubborn, arrogant, patient, persistent, etc. After a few hours, they evaluated who should be eligible for officer training. They were very confident of their rankings, and rarely experienced doubts or formed conflicting impressions.

The evidence that Kahneman and his colleague were not able to forecast accurately was overwhelming. Their forecasts were better than blind guesses, but not by much. Still, this knowledge of this failure of their predictions did not change the way they evaluated soldiers. It should have shaken their confidence, but it did not. This is the “illusion of validity.”

In 1984, Kahneman, Tversky, and a friend named Richard Thaler visited a Wall Street firm. Kahneman remembers being struck by the stock market and wondering what motivates some people to buy a stock while others sell it. He also started to realize that this industry of trading stocks appeared to be built on an illusion of skill, with each participant believing that they know more than others.

Kahneman describes how a student of his, Terry Odean, began studying the trading records of individual investors over seven years. Odean saw that in each trade, the investors expected the stocks they bought to do better than the stocks they sold. Odean discovered that on average, after one year the stocks they sold did better than those they bought by 3.2 percentage points.

Odean’s discoveries imply that for the majority of investors, doing nothing would have been a better policy than following their intuition. On average, the most active traders had the poorest results, while the investors who traded the least earned the most returns. Men often traded more than women, and thus women achieved better investment results than men.

Investors often like to lock in gains by selling “winners,” stocks that have gone up since they were purchased, and they hang on to their losers. But recent winners tend to do better than recent losers in the short run, so individuals sell the wrong stocks. Few stock pickers have the skill to beat the market consistently, year after year. For a large majority of them, the selection of stocks is more like rolling dice than playing poker. Kahneman discovered in his own research that differences in skill were not to be found.

Executives at these firms reward luck as if it were skill. Kahneman presented his findings to these executives, who certainly believed the findings but whose behavior was unaffected by the information. The statistics clashed with their personal impressions from experience. The advisors similarly were unaffected by the information. They bought into the potent psychological illusion that people who pick stocks are exercising high-level skills, and that they are among the few who can do what they believe others cannot.

Kahneman moves on to discuss pundits in business and politics, whose hindsight bias makes it difficult to accept the limits of forecasting ability. The image of the “march of history” makes developments seem inevitable, but large historical events are determined by luck as well. Kahneman illustrates this idea by mentioning that there was a 50-50 chance that the embryo that became Hitler would have been female.

Psychologist Philip Tetlock interviewed 284 people who made their living by commenting or advising on political and economic trends. He asked them about to rate probabilities of three future possibilities (e.g., the persistence of the status quo, more economic growth, less economic growth). The experts performed worse than they would have if they had simply assigned equal probabilities to those three outcomes (or worse than a “dart-throwing monkey,” as Kahneman writes).

Those who know more forecast very slightly better than those who know less. But those with the most knowledge are often less reliable, because those people develop an enhanced illusion of their skill and become unrealistically overconfident. Experts also resist admitting that they were wrong, and often have a collection of excuses as to why they were wrong.

Tetlock uses terminology from Isaiah Berlin’s essay on Tolstoy: “The Hedgehog and the Fox.” Hedgehogs have one coherent theory about the world and are confident in their forecasts. They are opinionated and clear, which is exactly what makes them good for television. Foxes, on the other hand, are complex thinkers. They recognize that reality emerges from many different agents and forces, including luck.

There are two main points to this chapter, Kahneman writes. The first is that the errors of prediction are inevitable, and the second is that high subjective confidence is not to be trusted as an indicator of accuracy—low confidence could be more informative.