Evolving general cooperation with a Bayesian theory of mind

Abstract

Theories of the evolution of cooperation through reciprocity explain how unrelated self-interested individuals can accomplish more together than they can on their own. The most prominent theories of reciprocity, such as tit-for-tat or win-stay-lose-shift, are inflexible automata that lack a theory of mind—the human ability to infer the hidden mental states in others’ minds. Here, we develop a model of reciprocity with a theory of mind, the Bayesian Reciprocator. When making decisions, this model does not simply seek to maximize its own payoff. Instead, it also values the payoffs of others—but only to the extent it believes that those others are also cooperating in the same way. To compute its beliefs about others, the Bayesian Reciprocator uses a probabilistic and generative approach to infer the latent preferences, beliefs, and strategies of others through interaction and observation. We evaluate the Bayesian Reciprocator using a generator over games where every interaction is unique, as well as in classic environments such as the iterated prisoner’s dilemma. The Bayesian Reciprocator enables the emergence of both direct-reciprocity when games are repeated and indirect-reciprocity when interactions are one-shot but observable to others. In an evolutionary competition, the Bayesian Reciprocator outcompetes existing automata strategies and sustains cooperation across a larger range of environments and noise settings than prior approaches. This work quantifies the advantage of a theory of mind for cooperation in an evolutionary game theoretic framework and suggests avenues for building artificially intelligent agents with more human-like learning mechanisms that can cooperate across many environments.