The Risk of Failure: Trial and Error Learning and Long-Run Performance

GAICIL Insights

Some of the most expensive failures are the experiments nobody ran.¹ The Risk of Failure: Trial and Error Learning and Long-Run Performance, by Steven Callander and Niko Matouschek, builds a model of that hidden cost. Risk-averse agents learn by watching earlier agents' choices and the outcomes they produced. Innovation, the authors note, is often the key to sustained progress, yet it is difficult and highly risky; breakthroughs mix with setbacks, and the path of learning is typically far from smooth.

Two results drive everything. First, a performance trap: low-performing agents opt out of experimenting and so never gain the knowledge needed to improve. Second, risk echoes through time, producing on average a divergence in long-run performance across agents. Those who start ahead keep experimenting and learning. Those who start behind stop, and stay behind. Caution, rational in each moment, compounds into permanent disadvantage.

Lay that over the diffusion of artificial intelligence and the familiar story about AI divides gets a twist. Firms, regions and countries all face a choice about whether to try uncertain AI tools. The well-performing can absorb a failed pilot. The struggling may rationally avoid the gamble because a setback would hurt too much. The model predicts the gap widens precisely because laggards stop learning. Debates about AI divides usually fixate on access to technology. This paper points to a quieter barrier: the capacity to afford being wrong.

Call it the poverty of experimentation. A firm with thin margins can buy the same software as a leader and still never use it boldly, because it cannot survive the downside. The divide is in risk tolerance as much as in tools.

The remedies follow from the mechanism. If risk aversion blocks experimentation among low performers, lowering the cost of failure unlocks learning. Insurance against failed trials, regulatory sandboxes that limit liability for early experiments, and public support for pilot projects all shrink the downside for would-be abstainers. Spillovers matter too: agents learn from the choices and outcomes of those before them. Transparent reporting of AI deployment results, failures included, enlarges the knowledge base of followers and cuts the risk each newcomer faces. A public library of failed AI pilots might be a valuable public resource.

Regulation enters through the same door. Rules that raise the cost of failure, for instance heavy liability for any error in early deployments, push more agents into the trap. The model implies such rules breed divergence: well-resourced incumbents keep experimenting while smaller firms retreat. Rules designed to reduce risk can entrench performance gaps. A balanced approach separates harms that justify strict rules from the ordinary setbacks that learning requires.

The authors draw the boundaries of their framework carefully. Their agents' theoretical knowledge is perfect and unchanging; they know the drift and variance terms and lack only knowledge of the realized setting. The most interesting next step, they say, is a richer environment where agents must also learn how their world was created, which could bring momentum effects in experimentation and performance into play. They also propose stochastic processes other than Brownian motion, under which the risk-return trade-off would no longer be independent of performance and action.

Those extensions read like a description of AI today. Firms are unsure whether a given application will work, and they are also learning the shape of the outcome distribution itself. In that setting, experimentation is worth even more, and a performance trap costs even more.

The provocative conclusion is that failure policy is growth policy. How a society treats a botched pilot, a failed startup or a mistaken deployment determines who keeps learning. Make failure survivable and more agents experiment, learn faster and converge. Make it ruinous and learning pools among those already ahead. As AI opens a vast new space for trial and error, the institutions that govern failure may decide who shares in the gains.

One closing image captures the model: two climbers in fog. The one with a rope keeps testing new footholds and finds the summit. The one without a rope stands still, safe and stuck. Supplying ropes is the policy challenge of the AI era.