Cooperation and punishment mechanisms in uncertain and dynamic social networks
GAICIL Insights
Trust among strangers rests on a fragile assumption: that what people appear to do is what they meant to do.1 Cooperation and punishment mechanisms in uncertain and dynamic social networks, by Edoardo Gallo, Yohanes E. Riyanto, Nilanjan Roy and Tat-How Teh, breaks that assumption in the lab and watches what happens. The study examines how reputational uncertainty and the rate of change of the social environment determine cooperation.
The core result is sobering. Reputational uncertainty significantly decreases cooperation. A fast-changing social environment causes only a second-order qualitative increase in cooperation. Individually, uncertainty induces more leniency and forgiveness in imposing network punishment, through proposing and removing links, which inhibits the formation of cooperative clusters. That effect is significant only in the fast-changing environment. A substitution between network punishment and action punishment, retaliatory defection, explains the difference between the two environments.
Noise here has a precise meaning: uncertainty about whether others' observed actions match their intended actions. The authors show that noise shifts behavior along both the network and the action dimension, and that its impact depends fundamentally on how often individuals can propose or remove links.
Now picture the modern internet as this experiment at planetary scale. Platforms, marketplaces and social networks rely on reputation systems to sustain cooperation among strangers, and algorithms increasingly mediate those systems. Artificial intelligence pulls in both directions. Synthetic content and automated accounts obscure intentions and add noise. Better fraud detection and more accurate reputation scoring remove it. The experiment explains why the direction matters so much: when observed actions are an unreliable guide to intentions, cooperation falls and cooperative clusters fail to form. Noisy signals of this kind make trust harder to sustain.
The leniency finding is the subtle heart of the paper. Under uncertainty, people forgive more, unwilling to cut ties with partners whose apparent defection might be a mistake. Forgiveness is sensible when signals are noisy, yet it weakens the discipline that sustains cooperation. Reputation systems face exactly this trade-off. Penalties too harsh punish honest errors; penalties too soft let defectors persist. Better information, including AI tools that sharpen signals, lets sanctions target the guilty more precisely and spares the unlucky.
Speed matters too. In fast-changing environments, people adjust connections often, and the substitution between network punishment and retaliatory defection shapes outcomes. Digital platforms are fast-changing by design: follow, unfollow, block and connect cost almost nothing. The experiment suggests that rewiring speed interacts with uncertainty to decide whether cooperation thrives. A platform's block button is an economic institution.
For policy, the implication is that information infrastructure is social infrastructure. Authentication, provenance tools and accurate reputation metrics improve the reliability of behavioral signals and support cooperation in digital markets. Measures that degrade signals undermine it. AI sits on both sides of this ledger, and its governance in digital environments should ask a single question: does a given tool improve or degrade the informational basis of reputation? That question cuts across content rules, identity rules and platform design alike.
The method is worth copying. Controlled experiments vary uncertainty and the pace of change independently, isolating effects that field data would confound. As AI transforms the information environment, such experiments can test how new tools affect cooperation before deployment at scale.
One question to put to platform builders: the most valuable feature on a marketplace may be invisible, a clean signal of who did what. Cooperation is fragile under uncertainty, and the architecture of social networks shapes its resilience. As economic and social exchange moves onto digital networks, making those networks informationally reliable, and structured so cooperative clusters can form, becomes a design challenge in which AI may play a significant part. Get it right and AI strengthens the trust on which markets and communities run.
Introduction
This paper examines experimentally how reputational uncertainty and the rate of change of the social environment determine cooperation. Reputational uncertainty significantly decreases cooperation, while a fast-changing social environment only causes a second-order qualitative increase in cooperation. At the individual level, reputational uncertainty induces more leniency and forgiveness in imposing network punishment through the link proposal and removal processes, inhibiting the formation of cooperative clusters. However, this effect is significant only in the fast-changing environment and not in the slow-changing environment. A substitution pattern between network punishment and action punishment (retaliatory defection) explains this discrepancy across the two social environments.
Takeaway
We provide empirical evidence regarding cooperation in an uncertain world with individuals receiving opportunities to update their interaction partners. Our experiment demonstrates that the presence of noise, defined as uncertainty on whether other people’s observed actions correspond to their intended actions, affects individual behavior along the network as well as the action dimension. We further show that the extent of the impact of noise depends fundamentally on the frequency at which individuals receive opportunities to propose and/or remove links with others.1
Notes: we focus on players who had at least 4 occurrence of the actual action being C out of the last 5 rounds. We report the average values over rounds 6-21. The number of observations are in parentheses. formation. Our study raises several questions that suggest exciting possibilities for follow-up studies as future research. First, we do not impose an explicit cost on forming, maintaining, or removing links in our experiment. Exploring the effect of such cost might provide us with additional insights than mere opportunity costs of not interacting as in our setup. Second, there are few studies that investigate the effect of allowing communication on cooperation in a two-person repeated PD game (Arechar et al., 2017) and a partnership game (Embrey et al., 2013).2,3 It would be fruitful to study the effect of communication in our framework. Third, viewing cooperation as a risky choice, the strategic decisions made by individuals affect the distribution of income and wealth in society. In a static, non-strategic, and non-networked context, Cappelen et al. (2013) demonstrate that, even though many subjects focus exclusively on ex-ante opportunities, most favor some redistribution ex-post, and several individuals make a distinction between ex-post inequalities that reflect differences in luck and ex-post inequalities that reflect differences in choices.4 It would be interesting to explore the fairness and inequality consequences of the action and link proposal/removal decisions in our setup.
Finally, noisy information is publicly observable in our setting, so our game has an imperfect public monitoring structure. However, it is possible that players do not even know what signals other players have observed about their own play, thereby making the monitoring structure private. Aoyagi et al. (2019) and Kayaba et al. (2020) study cooperation in two-person experimental games with such a monitoring structure.5,6
Notes
- Edoardo Gallo et al., Cooperation and punishment mechanisms in uncertain and dynamic social networks, 134 Games & Econ. Behav. 75 (2022), https://doi.org/10.1016/j.geb.2022.03.015. Takeaway source: Conclusion, reproduced from a free author or repository copy (working-paper version). ↩
- Arechar, A. A., A. Dreber, D. Fudenberg, and D. G. Rand (2017). “I’m just a soul whose intentions are good”: The role of communication in noisy repeated games. Games and Economic Behavior 104, 726–743.. ↩
- Embrey, M., G. R. Fréchette, and E. Stacchetti (2013). An experimental study of imperfect public monitoring: Efficiency versus renegotiation-proofness. SSRN Working Paper .. ↩
- Cappelen, A. W., J. Konow, E. Ø. Sørensen, and B. Tungodden (2013). Just luck: An experimental study of risk-taking and fairness. American Economic Review 103 (4), 1398–1413.. ↩
- Aoyagi, M., V. Bhaskar, and G. R. Fréchette (2019). The impact of monitoring in infinitely repeated games: Perfect, public, and private. American Economic Journal: Microeconomics 11 (1), 1–43.. ↩
- Kayaba, Y., H. Matsushima, and T. Toyama (2020). Accuracy and retaliation in repeated games with imperfect private monitoring: Experiments. Games and Economic Behavior 120, 193–208.. ↩
Cite as: Edoardo Gallo et al., Cooperation and punishment mechanisms in uncertain and dynamic social networks, 134 Games & Econ. Behav. 75 (2022), https://doi.org/10.1016/j.geb.2022.03.015
Header image: Community and Women Group Booths, Guests Area in Taipei Station Plaza 20121006a.JPG by 玄史生, CC0, via Wikimedia Commons.
