Here’s the problem with forecasts: some of them are right, and some of them are wrong, and by the time we find out which is which, it’s too late. This leads to what we might call the forecasting paradox: the test of a useful forecast is not whether it turns out to be accurate, but whether it turns out to prompt some sort of useful action in advance. Accuracy may help, but then again it may not. Forewarned is not necessarily forearmed.
Consider the challenge I was set when speaking at a post-pandemic conference. One questioner told me that at the previous conference, in late 2019, the keynote speaker — a famous scientist — had warned of the risk of a global pandemic. Could I offer a better forecast than that? It depends on what you mean by better. Could I offer a more timely, accurate forecast about a question of global consequence? Of course not. But could I offer a more useful forecast? Probably. The bar had been set lower than you might think.
The 2019 audience had heard a generic warning that there might be some kind of pandemic one of these days, collectively shrugged and done nothing. Neither they nor the speaker realised the pandemic in question was just weeks away and none of them were in a position to do much about it anyway. The forecast had been brilliant — and useless.
Twenty years ago, the forecasts of disaster facing New Orleans should have fared better. The Federal Emergency Management Agency had warned that one of the three most probable catastrophes facing the US was a hurricane striking low-lying New Orleans. As the storm closed in, in 2004, newspapers described every detail of the risk, from a failure of the levees to the impossibility of a mass evacuation and the prospect of hundreds or even thousands of deaths.
At the last minute the hurricane — Ivan — turned aside. Yet the prophecies of doom came true in almost every respect a year later when in 2005 Hurricane Katrina devastated New Orleans. The year’s delay could have made the forecasts more useful, not less, by giving city, state and federal authorities time to prepare. Alas, they did not.
In contrast, Brigham and Women’s Hospital in Boston didn’t forecast a situation in which two bombs exploded at the Boston Marathon. But in April 2013 they were nevertheless prepared when it happened, having run 78 major emergency drills covering everything from oil spills to train crashes.
The world of speculative fiction is full of forecasts that made us wiser despite never coming true. I contacted activist and science-fiction author Cory Doctorow, who pointed me to Mary Shelley’s Frankenstein, “the first Luddite novel”, and Margaret Atwood’s The Handmaid’s Tale. Then an insurance executive was murdered in Manhattan — a scenario foreshadowed in the short story Radicalized. The author? Cory Doctorow. Fiction can help us see the future.
Perhaps you feel that no novel should really count as a forecast. Consider instead the idea of the “pre-mortem”, advocated by the psychologist and author Gary Klein. The pre-mortem is a project-planning exercise in which a team adopts a position of “prospective hindsight”. Let’s assume the patient died on the operating table, or that the new IT project suffered a massive cost overrun, or that the dinner party was a humiliating flop. Given that assumption, why? What went wrong?
Research conducted in the late 1980s by Deborah Mitchell, Jay Russo and Nancy Pennington found that this perspective helped people to generate more ideas, with more detail, about why a project might succeed or fail. A pre-mortem is intended to be a self-defeating forecast. The idea is that by helping a team brainstorm a list of quite specific things that might go wrong, disaster will be averted.
The great psychologist Amos Tversky quipped that most people lump their forecasts into three categories, “gonna happen”, “not gonna happen” and “maybe”. That seems right to me, but the problem with such crude intuitions is not that they’re insufficiently precise, but that they allow us to short-circuit any further thought on the matter.
That’s a shame. Thinking seriously about the future can be a worthwhile exercise, not because the future is knowable but because the process is likely to make us wiser. One surprising piece of evidence for that comes from forecasting experts Barbara Mellers, Philip Tetlock and Hal Arkes. A few years ago, they ran a multi-month forecasting tournament and surveyed the opinions of the participants before and after. They found that the process of thinking seriously about forecasts softened the preconceptions of the competitors. They had become more politically moderate and more inclined to attribute moderation to their political opponents.
More broadly, a scenario-planning exercise encourages people to recognise that the world is an uncertain place. Many years ago I worked in the scenario-planning game, and one of our mantras was “scenarios are not forecasts”. I think I understand that statement a little better now. Scenarios are not forecasts because they are not aiming to be accurate, but to be useful. The forecasting paradox tells us that those two qualities are very different.
Written for and first published in the Financial Times on 3 January 2025.
Loyal readers might enjoy the book that started it all, The Undercover Economist.
I’ve set up a storefront on Bookshop in the United States and the United Kingdom. Links to Bookshop and Amazon may generate referral fees.
