# Evolution of social norms and correlated equilibria

See allHide authors and affiliations

Edited by Brian Skyrms, University of California, Irvine, CA, and approved March 20, 2019 (received for review October 7, 2018)

### This article has a Correction. Please see:

## Significance

We consider social norms as collections of beliefs or superstitions about events occurring in nature. These events have no inherent meaning, but individuals can choose to believe that they do. Specifically, individuals can interpret events as prescribing behaviors for themselves along with expectations of the behaviors of others. We show how evolutionary competition between such beliefs can give rise to both prescriptive and descriptive self-enforcing norms that can allow populations to coordinate behavior. Our model provides an evolutionary account for how normative meaning emerges in an inherently meaningless world.

## Abstract

Social norms regulate and coordinate most aspects of human social life, yet they emerge and change as a result of individual behaviors, beliefs, and expectations. A satisfactory account for the evolutionary dynamics of social norms, therefore, has to link individual beliefs and expectations to population-level dynamics, where individual norms change according to their consequences for individuals. Here, we present a model of evolutionary dynamics of social norms that encompasses this objective and addresses the emergence of social norms. In this model, a norm is a set of behavioral prescriptions and a set of environmental descriptions that describe the expected behaviors of those with whom the norm holder will interact. These prescriptions and descriptions are functions of exogenous environmental events. These events have no intrinsic meaning or effect on the payoffs to individuals, yet beliefs/superstitions regarding them can effectuate coordination. Although a norm’s prescriptions and descriptions are dependent on one another, we show how they emerge from random accumulations of beliefs. We categorize the space of social norms into several natural classes and study the evolutionary competition between these classes of norms. We apply our model to the Game of Chicken and the Nash Bargaining Game. Furthermore, we show how the space of norms and evolutionary stability are dependent on the correlation structure of the environment and under which such correlation structures social dilemmas can be ameliorated or exacerbated.

Cooperation and coordination in social settings are fundamental to the functioning of human and animal communities. Considering purely self-interested actors, cooperation and coordination should be rare if there are conflicts of interests between players and if no external incentives (or coercion) are at hand to resolve these conflicts. Yet, we observe many examples of cooperation without external incentives mediated by informal mechanisms, such as social norms (1⇓–3). It is apparent that, while humans and animals do tend to make decisions to improve their payoffs, individual decision making is pervasively affected by both prescriptive beliefs on their own behavior and expectations of others’ behavior (4). These prescriptions and expectations may or may not be rooted in reality or accurately describe others’ behaviors, but regardless, they have a real effect on behavior (5⇓⇓–8). Importantly, they can induce regularities in a population that allow self-interested agents to cooperate and coordinate with each other efficiently. These beliefs can endow events and cues in the world that are inherently meaningless with normative meaning to voluntarily coordinate individual behavior. We address how social norms that give rise to such beliefs can arise through an evolutionary game theory model that combines rationality, superstition, and evolutionary dynamics to coordinate behaviors on external events without design.

Although social norms coordinate behavior, any individual or the society at large generally does not explicitly design them; rather, they are a collective phenomenon that emerges from the interaction of individuals. Thus, game theory is a common tool to model the emergence of social norms (9⇓–11). Many different game theoretic conceptions of social norms exist. Some norms can be seen as mechanisms that select from (the typically numerous) Nash equilibria in repeated games (12). Alternatively, some social norms consist of conditional behavioral rules (3) that convert social dilemmas into coordination games: for example, by external enforcement (either through institutions or by others in a group) (13, 14). Our focus here is on social norms that lack external enforcement but still are followed, because agents, given their beliefs about the world (also induced by social norms), find it in their interest to do so. In other words, we are interested in both descriptive norms that tell individuals what behaviors to expect from others and prescriptive norms that tell individuals how to behave and can be predicated on descriptive norms (15). More specifically, we model social norms that act as “choreographers” (16, 17) that induce correlated beliefs in agents, allowing them to coordinate on a correlated equilibrium (18, 19) of the game.

A correlated equilibrium is generated by a correlating device [the choreographer (16, 17)] that suggests pure strategies to the players. If the suggestions to each player are correlated in a certain way and the agents know these correlations, it can be optimal for the players to follow the suggestions of the choreographer. Such behavior is called a correlated equilibrium. The Game of Chicken is an exemplar for the coordinating effects of correlated equilibria, as it can feature a scenario in which coordination would lead to a socially optimal yet game-theoretically unstable state. Consider two cars approaching an intersection: one northbound and the other westbound. The drivers may choose to drive through the intersection or stop. The payoff matrix for this game is

In the classical conception of correlated equilibrium, there is a third party, the choreographer, that assigns the meanings to the cues and decides on the correlations between the suggestions, which then are assumed to be common knowledge between individuals. The city putting a traffic light at our intersection would be an example of this. Here, we are concerned with the case where there is no central party that decides on the normative meaning, and players do not know correlations between cues. Instead, we ask how prescriptive and descriptive individual beliefs about events in the world compete against each other decentrally. In effect, we ask whether natural selection can serve as a “blind choreographer” that shapes individuals’ beliefs about stochastic events in the world such that they play correlated equilibria without knowledge of the correlation structure of the world or external enforcement.

Our work belongs to a body of game theory that has studied how correlated equilibria can emerge in a decentralized way through evolutionary or learning dynamics (23⇓⇓⇓⇓⇓⇓⇓⇓–32). The first strand in this body starts with the seminal paper of Selten (23) who proved that evolutionarily stable strategies conditional on cues in the world have to be pure strategies. Later, Cripps (24) and Shmida and Peleg (25) showed that these correspond to correlated equilibria of the underlying game (31). Building on these results, Metzger (32) modeled the evolution of the cue distribution itself through a centralized mechanism where all players have to agree to change the distribution of signals. A second strand of literature (26⇓⇓⇓–30) is concerned with learning in repeated games, where the history of the interaction (26⇓–28) or selection of players (29) or an exogenous correlation device (30) acts as the choreographer, and it shows that various learning dynamics can converge to correlated equilibria after repeated play.

Our paper is closest in spirit to the first strand. We consider nature as providing a collection of correlated events in the world and ask how behavioral strategies conditional on these event spaces might evolve. Our main contribution is that we explicitly consider prescriptive and descriptive beliefs that individuals might impute from such events. We model the evolution of such beliefs under the assumption that, given their beliefs, agents make individually rational decisions. This is analogous to the “indirect evolution” approach to the evolution of utility functions (33⇓⇓–36). In this case, we model how evolution drives individual behavior indirectly by shaping normative beliefs in a population. Our approach is aimed at explaining not only norms as correlated behaviors but also, the belief structures that support them in the absence of central authorities that impose or justify these beliefs. We show how belief structures about events in the world can emerge and be both individually rational and evolutionarily stable, although the events are inherently meaningless to the interaction. We are particularly interested in this phenomenon with respect to noncooperative games that feature a social dilemma where the socially optimal solution is evolutionarily unstable. As illustrative examples, we apply our general framework to the Game of Chicken and the Nash Bargaining Game.

## Model

Consider two players meeting to play a game. Each player privately observes an event, which is irrelevant to the game being played but may be correlated with their opponent’s event and other events in ways unknown to the players. We consider social norms that assign to these events a set of labels, prescriptions, and descriptions of opponent behaviors.

Players can assign a private label to these events to differentiate between them. The events and labels have no inherent association with one another. As a function of the observed label, players’ social norms prescribe a pure strategy recommendation. We assume that players cannot observe the events that their opponents see or the prescriptions that they may have received. Instead, each players’ norm gives them a (private) conditional probability distribution of prescriptions that their opponent may be receiving given the recommendation that they themselves have been given. This conditional probability distribution may or may not correspond to the actual conditional distribution of prescriptions or behavior of a real opponent. It simply represents a player’s subjective expectation given by its social norm. The players then may obey the norm’s prescription or play a default strategy from which they will earn a payoff. The prescription of the norm is only obeyed if it represents a best response to the description of the opponent behavior given by the norm.

Fig. 1 illustrates our model graphically in a world with four events and two players having differing norms. An event is chosen where each player observes a cat. In this case, Player 1 observes the black cat, and Player 2 observes the white cat. Player 1’s norm prescribes cooperation when it observes black or white cats and defection for gray and tabby ones. On the descriptive side, Player 1’s norm specifies a conditional distribution of behaviors of Player 2. Since this distribution is conditional on the labels, Player 1’s expectation of Player 2’s behavior is the same when observing black and white cats. Observing a black or white cat, Player 1 believes that Player 2 will cooperate with probability

Mathematically, we model the world of events as an undirected graph,

Norms prescribe pure strategy recommendations to labels. We can represent the behaviors that players are prescribed by the

We examine the effects of a variety of competing norms on a given event graph. In playing an opponent with a differing norm, a player will still follow its own norm and thus, will not necessarily receive the expected payoff that their norm would dictate. Norm change can be facilitated by such disparity between expectations generated by descriptive norms and reality (37). However, we assume that players do not reflect on—or at least do not do so as to modify—their norms directly but can observe the fitness of competing norms. Thus, norm change is generated by imitation dynamics, whereby players imitate the norms of those with higher fitness (38). The expected payoff of following norm n using P playing norm

Applying this equation to the example in Fig. 1 with equally weighted edges, we have

## Results

### A Classification of Norms.

Our definition of social norms captures the idea that norms imbue an inherently meaningless world with prescriptions on self-behavior and expectations about others’ behaviors. However, we do not assume that the prescriptions of a social norm are to be blindly obeyed. In particular, we assume that a norm will only be obeyed when—given the expectations from the descriptive aspect of the norm—individuals cannot improve their payoffs by deviating from the social norm’s prescriptions. In this section, we will first define the rationality condition and then, classify several important norms (depicted as a Venn diagram in Fig. 2). In *Evolutionary Stability of Norms*, we will address evolutionary concerns.

Let **[4]**, then we assume that players who have it will disregard both the recommended prescriptions and the descriptions to play a default strategy at all events (*SI Appendix*, section 2 has a discussion of alternative assumptions for this case). We will call such norms null norms. If the game features a dominant strategy, we assume that this is the default. However, if the game features a symmetric mixed Nash equilibrium, we assume that that is the default. In the case of multiple such strategies, we have a family of null norms.

Although the descriptive aspect of a norm does not have to accurately describe opponents’ behavior, there exists a class of norms that does fit the behavior of (possibly hypothetical) opponents. In particular, a norm is empirically validatable if its prescriptions are rational against the behavior of a rational norm. This class is of interest, since two norms that are empirically validatable with respect to one another implement a correlated equilibrium when they play one another (*SI Appendix*, Lemma S1).

We are also interested in a particular class of norms where everyone would obey their norm given that everyone else also obeys the same norm (39). The prescriptive part of such norms must be rational vs. the behavior of opponents that follow the prescriptions of the same norm. In other words, the norm is empirically validated by itself. We call these norms consistent norms. In contrast, the prescriptive part of an inconsistent norm does not follow from a description of its own behavior but rather, is rational against a player obeying a different set of prescriptions. For example, consider the event pair {black cat, white cat}, in which the two cats are completely correlated with one another. Suppose that, when a black cat is observed, a norm prescribes defection and expects cooperation from the opponent. If this norm is consistent, it will prescribe cooperation and expect defection when a white cat is observed. However, an inconsistent norm can hypocritically prescribe its carrier to also defect and expect cooperation in the event that the carrier sees a white cat. Mathematically, if a norm is empirically validatable for

We label a subset of consistent norms where the prescriptions for each vertex of an edge are best responses to one another as best-response norms. A best-response norm always exists if the game admits a single-strategy Nash equilibrium. However, for a two-strategy pure Nash equilibrium, a best-response norm exists if and only if the graph is bipartite. Since we may assign to each partition one of the strategies, each vertex along an edge is a best response to each other. It follows that the fitness of a best-response norm playing itself is the mean of the payoffs of the pure Nash equilibria (*SI Appendix*, Lemmas S2 and S3).

### Evolutionary Stability of Norms.

Similar to the concept of an evolutionary stable strategy (40), an evolutionary stable norm is a norm that cannot be invaded by an initially rare norm (*SI Appendix*, section 1 has the formal definition). We are concerned with the evolutionary history initiated at a state without prescriptions where everyone plays a null norm using a mixed Nash equilibrium. As long as the prescriptions are not obeyed (**[4]** is not satisfied), they may accumulate in the population by drift, since they remain playing the mixed Nash strategy. However, on being rational, they are obeyed. If such a norm is a consistent norm, it may then invade if its prescriptions are in the hull of the mixed Nash strategy. Any strategy within the support of a mixed Nash playing the mixed Nash strategy receives the mixed Nash payoff. Furthermore, as a consistent norm implements a correlated equilibrium, its payoff playing itself is greater than any alternative norm under the same event labeling. Since the null norm’s behavior is identical across all events, the labeling is inconsequential and thus, receives a lower fitness playing the consistent norm. Therefore, the consistent norm can invade. More generally, the same arguments applied to a subset of events show the emergence of obeyed prescriptions from null prescriptions. In other words, the evolutionary dynamics of norms tend to manufacture beliefs that coordinate interactions to correlated equilibria out of inherently meaningless events (*SI Appendix*, Lemma S4 has mathematical details of these arguments).

If the null norm plays a mixed Nash, evolutionarily stable norms must be consistent norms and thereby, implement correlated equilibria (*SI Appendix*, Lemma S6). In this way, evolutionarily stable norms in our model allow the decentralized emergence of not only coordinated behaviors but also, coordinated beliefs about the correlation device.

Since consistent norms induce a correlated equilibrium, they can be evolutionarily stable against norms that share the same relative labeling of events (i.e., as long as events are labeled such that they are differentiated in the same ways). Norms with different labels may invade a consistent norm that is evolutionarily stable against norms with the same labeling unless the resident norm is a best-response norm (*SI Appendix*, Lemma S7). Unlike consistent norms, inconsistent norms cannot be evolutionary stable, because they can be invaded by a norm that validates their descriptions, which by definition, is different from themselves (*SI Appendix*, Lemma S5). However, two such inconsistent norms that empirically validate one another form a stable polymorphism (*SI Appendix*, Lemma S8).

We may have many other types of polymorphisms depending on the underlying graph and labeling. Monomorphisms of consistent norms are not necessarily stable, since **[4]** only holds for a norm’s labeling. An implication of this is that a consistent norm that can invade an inconsistent norm will not necessarily fix, and a polymorphism may thus result. This is because the expected payoff of an inconsistent norm against the consistent norm may be higher than the consistent norm against itself if they have different labelings. However, if the consistent norm’s labeling is complete (i.e., each vertex has a unique label), then the invading consistent norm will fix (*SI Appendix*, Lemma S9). Another example of a polymorphism is that of weakly evolutionarily stable consistent norms with differing prescriptions, which can exist as long as, on average, the pairings of prescriptions are the same or trivially, if the null norm and consistent norms’ behaviors are identical (*SI Appendix*, section 4).

### Examples.

#### The Game of Chicken.

The existence and evolutionary stability of norms for a game are dependent on the underlying event space. Consider the bipartite graph

Continuing with the example above, we may consider all prescriptions as a strategy from which we have a 12-dimensional replicator equation, the symmetric Nash equilibria of which are the fixed points (38). There are 25 fixed points of this model, 3 of which are stable (the monomorphic states of the three stable consistent norms). For monomorphic populations,

#### The evolution of fairness in the Nash Bargaining Game.

The Nash Bargaining Game provides us with another interesting example (41). Each player’s strategy is a demand of a portion of a dollar. If the sum of the demands is less than or equal to a dollar, then each player receives their demand. If, however, the sum exceeds a dollar, then they both receive nothing. A variety of solutions (corresponding to varying definitions of fairness) have been found (41⇓–43). We are interested in how a fair (50/50) division of the dollar can emerge. It has been shown that a single-population replicator equation model can converge to such a fair distribution for a symmetric linear Nash Bargaining Game (44). However, the history of play matters, and the basin of attraction to a suboptimal polymorphic stable equilibrium is not negligibly small. A fair division of the surplus between tenants and landlords is also stochastically stable if memory is large and noise is small (45). Here, players play the linear symmetric Nash Bargaining Game at each edge.

Let us first consider fairness across event space (i.e., the variance of the expected payoff) for evolutionarily stable consistent norms prescribing either δ or *SI Appendix*, section 5, Lemma S10). However, the only distribution that can reach the optimal expected payoff is

To explore the effects of the graph type and number of vertices on the evolution of fairness in the Nash Bargaining Game, we used trait substitution simulations on the complete graph, ring graphs, and small world graphs (47) (details are in *SI Appendix*, section 5), where the dollar can be divided into either quarters or dimes. These simulations permit both monomorphisms of consistent norms and polymorphisms of inconsistent norms. Fig. 4 depicts the results for the complete and degree 2 ring graph (remaining results are in *SI Appendix*, section 5). The fair distribution is frequently reached and is primarily a monomorphic state of a consistent norm. Polymorphisms of inconsistent norms can emerge and stabilize, leading to suboptimal mean fitness. In general, monomorphic states are more likely to lead to higher fitness than polymorphisms. For example, if the dollar can only be split into quarters on the complete graph, we have 1/3 of the population demanding

## Discussion

In this paper, we show how social norms that coordinate behaviors can be bootstrapped from random superstitions about irrelevant events in the world. A key to our model is the synthesis of prescriptive and descriptive norms. Inconsistencies between prescriptive and descriptive norms have been experimentally shown to weaken norm adherence (48, 49). As such, our rationality and empirical validation conditions are key to aligning prescriptive and descriptive norms and inducing their obeyance. We show that empirically validatable norms align not only the behaviors of the players but also, their beliefs about others’ behaviors in a correlated equilibrium. Such social norms act as choreographers of the players’ behaviors, potentially ameliorating social dilemmas, a well-studied product of social norms (50). However, our players are blind to the true correlations between events in the world, and as such, this coordination is unintentional. Rather than being designed, the choreography of play through consistent belief emerges from the accumulation of random superstitions of the environment. In alignment with empirical work (51, 52), our norms start out at the individual level as subjective norms ascribing meaning to events in the world: whether they be seeing a colored cat or something equally superstitious. If these subjective norms are rational to follow given the beliefs that they induce and yield a high payoff, they can win in evolutionary competition and acquire conventional normative meaning. In this way, normative meaning invented at the personal level provides the material for conventional meaning to emerge through the joint action of individual rationality and evolutionary dynamics.

## Acknowledgments

Funding was provided by Defense Advanced Research Projects Agency Next Generation Social Science Program Grant D17AC00005 and Army Research Office Grant W911NF-12-R-0012-03.

## Footnotes

- ↵
^{1}To whom correspondence should be addressed. Email: morsky{at}sas.upenn.edu.

Author contributions: B.M. and E.A. designed research, performed research, analyzed data, and wrote the paper.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1817095116/-/DCSupplemental.

Published under the PNAS license.

## References

- ↵
- ↵
- ↵
- Bicchieri C

- ↵
- ↵
- ↵
- Cialdini RB,
- Kallgren CA,
- Reno RR

- ↵
- Etzioni A

- ↵
- ↵
- Voss T

- ↵
- Gintis H,
- Helbing D

- ↵
- Paternotte C,
- Grose J

- ↵
- Binmore KG

- ↵
- ↵
- ↵
- Chung A,
- Rimal RN

- ↵
- ↵
- Gintis H

- ↵
- ↵
- ↵
- Smith JM

- ↵
- ↵
- Mesterton-Gibbons M,
- Adams ES

- ↵
- Selten R

- ↵
- Cripps M

- ↵
- Shmida A,
- Peleg B

- ↵
- Foster DP,
- Vohra RV

- ↵
- ↵
- ↵
- Ianni A

- ↵
- Arifovic J,
- Boitnott JF,
- Duffy J

- ↵
- Kim C,
- Wong K-c

- ↵
- Metzger LP

- ↵
- Güth W

- ↵
- ↵
- Akçay E,
- Van Cleve J,
- Feldman MW,
- Roughgarden J

- ↵
- ↵
- Bicchieri C

- ↵
- Hofbauer J,
- Karl S

- ↵
- ↵
- ↵
- ↵
- ↵
- Harsanyi JC

- ↵
- ↵
- ↵
- Rawls J

- ↵
- ↵
- ↵
- ↵
- ↵
- Sherif M

- ↵
- Valsiner J

## Citation Manager Formats

## Article Classifications

- Social Sciences
- Economic Sciences

- Biological Sciences
- Evolution