# The Red Queen and King in finite populations

^{a}Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138;^{b}Program for Evolutionary Dynamics, Harvard University, Cambridge, MA 02138;^{c}Department of Mathematics, Columbia University, New York, NY 10027;^{d}Institute of Science and Technology Austria, 3400 Klosterneuburg, Austria;^{e}Department of Mathematics, Harvard University, Cambridge, MA 02138

See allHide authors and affiliations

Edited by Steven A. Frank, University of California, Irvine, CA, and accepted by Editorial Board Member Joan E. Strassmann May 23, 2017 (received for review February 6, 2017)

## Significance

When two populations interact, when does it pay to evolve rapidly, and can it ever be an advantage to evolve slowly? We address these questions using evolutionary game theory. In antagonistic interactions (e.g., host–parasite), we find that faster evolution by any means is beneficial—the “Red Queen” effect. In certain mutualisms, slower evolution is favored in the long run. This “Red King” effect is driven by differences in how efficiently natural selection acts in the two populations, rather than by differences in their generation times or mutation rates. Our results clarify the role of evolutionary rate in symbiont evolution.

## Abstract

In antagonistic symbioses, such as host–parasite interactions, one population’s success is the other’s loss. In mutualistic symbioses, such as division of labor, both parties can gain, but they might have different preferences over the possible mutualistic arrangements. The rates of evolution of the two populations in a symbiosis are important determinants of which population will be more successful: Faster evolution is thought to be favored in antagonistic symbioses (the “Red Queen effect”), but disfavored in certain mutualistic symbioses (the “Red King effect”). However, it remains unclear which biological parameters drive these effects. Here, we analyze the effects of the various determinants of evolutionary rate: generation time, mutation rate, population size, and the intensity of natural selection. Our main results hold for the case where mutation is infrequent. Slower evolution causes a long-term advantage in an important class of mutualistic interactions. Surprisingly, less intense selection is the strongest driver of this Red King effect, whereas relative mutation rates and generation times have little effect. In antagonistic interactions, faster evolution by any means is beneficial. Our results provide insight into the demographic evolution of symbionts.

Antagonistic symbioses can be conceptualized well in the following simple, constant-sum game (1, 2):Here, player 1’s available strategies are *you* can do, to keep in the same place” (ref. 4, p. 42).

Arms-race dynamics characterize many interactions both in the natural world and in human behavior. In a host–parasite interaction, for example, the host is selected to develop immunity to the parasite, which in turn selects for new “resistance” mutations in the parasite, which selects for the host to develop immunity to the new mutant parasite, and so on. In Batesian mimicry, a palatable species (of butterfly, for example) evolves to mimic the warning display of an unpalatable species, so that it is mistaken by predators for the unpalatable species. This selects for the unpalatable species to evolve a new display, to not be mistaken for the palatable species, which is then under selection to mimic the new display, and so on (5).

Antagonistic interactions play out at an intragenomic level as well. Examples, among many others (6, 7), are the X-linked male-meiotic driving gene *Dox* and its autosomal suppressor *Nmy* in *Drosophila simulans* (8, 9), centromeric repeat sequences seeking to drive in female meiosis and the centromere-histone genes that keep them in check (10, 11), and the mammalian recombination-specifying gene *Prdm9* and the binding sequences of its protein [which are under positive selection to escape PRDM9 binding (12⇓⇓–15)]. Consistent with arms-race dynamics, the conflicting elements in these examples all show genetic evidence of rapid evolution (10, 11, 16⇓⇓⇓⇓⇓⇓–23).

Mutualistic symbioses with a degree of conflict (24, 25) can also be conceptualized in a simple game (26),with **2** is a true mutualism (both populations prefer the two coordination states to the two noncoordination states).

This game also describes many interactions in both nature and humans. Mutualisms often involve a division of labor (27), such as the production of different nutrients in a microalgal–microbial partnership (28), the separate production of a “poison” and its “antidote” by two linked genetic elements in a gamete-killing meiotic drive complex (6, 29), or the economic production of different goods by trading partners under increasing returns to scale (30). In these cases, it is beneficial to both interactants that all tasks be carried out, but there can be conflict over who should do which (31). For example, it is better to be the antidote producer than the poison producer in the meiotic drive complex, because the gains of their partnership are shared equally, but its disruption (by recombination) is costlier to the poison producer.

Another example, Müllerian mimicry, involves two poisonous species (of butterfly, say) evolving to share a common pattern, one pattern being easier for their predators to learn than two (32). But if the species’ original patterns evolved to suit differences in their respective habitats, behaviors, or genomic backgrounds, then there might be conflict over which pattern to converge on, with each species enjoying an evolutionary advantage if convergence is on its original pattern.

In both antagonistic and mutualistic symbioses, the relative rates of evolution of the participating populations are thought to be important determinants of their relative evolutionary success. A faster rate of evolution could be achieved through various means (3, 24⇓–26). First, it could derive from a shorter generation time, allowing more generations over which to adapt (2). Second, it could derive from a higher rate of mutation, more rapidly generating new variants that allow a population to escape an unfavorable state of interaction (24, 33, 34). Finally, it could derive from more effective action of natural selection. This could be because the stakes are higher for one population than the other—“the rabbit is running for his life, while the fox is only running for his dinner” (ref. 3, p. 493)—or because selection acts more effectively in large populations, owing (roughly speaking) to a reduced effect of random drift (35) (a precise formulation of this statement is given in *SI Appendix*, section S5.2).

Common wisdom holds that a faster rate of evolution is advantageous in antagonistic interactions (36), the better to “keep ahead in the race” (ref. 3, p. 492). But it has not, to our knowledge, been clearly demonstrated which individual biological parameters drive this Red Queen effect in theory.

In an important paper, Bergstrom and Lachmann (26) (hereafter, B&L) demonstrated that a slower rate of evolution might be advantageous in some mutualistic interactions, an effect they called the Red King effect. They studied two populations interacting according to the mutualism game **2**, with deterministic evolutionary dynamics operating in each of the two populations, both of infinite size and without the possibility of mutations.

In their model, **2**,

These dynamics are deterministic—evolutionary trajectories are fully determined once their starting points are known—and evolution necessarily leads to one of the two coordination equilibria,

B&L (26) showed that, when **2** and population 1 evolves slower than population 2 (

Here, we construct a finite-population model of symbiosis evolution, incorporating all of the biological determinants of evolutionary rate. This model allows us to extend B&L’s (26) results for mutualisms in several ways and to apply a similar analysis to antagonistic symbioses, leading to a richer picture of the evolutionary dynamics of symbioses.

First, our model allows for an explicit characterization of which evolutionary rate parameters influence the relative success of the interacting populations. This is unknown for mutualisms [B&L’s (26) general rate parameters

Second, our model allows us to uncover a key influence of evolutionary timescale on symbiosis evolution. In the mutualism game **2**, we show that the short-run behavior of our stochastic evolutionary dynamics is very similar to that of the replicator dynamics studied by B&L (26): From a given starting state, the dynamics rapidly converge to, or near to, one of the two equilibria of the game. Which equilibrium is most likely to be approached depends critically on the starting point, just as in the dynamics of B&L (26). On a longer timescale, however, we show the evolutionary dynamics of this game to be of a very different nature. The long-run dynamics involve transitions between equilibria, driven by sporadic mutation, and eventually become independent of where the dynamics first started. As we show, in mutualisms, many of the conclusions concerning the short-run dynamics are either annulled or reversed in the long run.

Our main results are summarized in Table 1.

## A Finite-Population Model of Symbiosis Evolution

Populations 1 and 2 are of sizes **1** and **2**. The “populations” here can be broadly construed: They could be all of the individuals of two species in a symbiosis, for example, or all of the alleles at two distinct loci among the genomes of a single species.

The evolutionary process occurs in discrete time steps. Each time step, individuals in each population receive their average payoffs from interacting with a random member of the opposite population. An individual’s payoff

In each time step, a “birth–death event” occurs in one of the populations. Individuals in the two populations have relative generation times

A birth–death event in a population involves choosing an individual to reproduce, with probability proportional to fitness, and an individual to die, with each one equally likely. These can be the same individual. A single offspring of the reproducing individual replaces the individual that was chosen to die. This within-population process, the Moran process (38, 39), has been used as a model both of biological evolution (40, 41) and of imitation learning (42⇓–44). In the Moran process, one “generation” of a population typically corresponds to the number of birth–death events that is about the same as the population’s size (41). In our two-population framework, we label the number of time steps equal to the sum of the populations’ sizes as a common generation: Each individual experiences on average one birth–death event per generation.

Finally, to account for the possibility of mutation, we assume that, in a birth–death event in population

In sum, given a payoff matrix such as [**1**] and [**2**] and rate parameters *SI Appendix*, section S1.

Two regimes are of interest in these dynamics, corresponding roughly to their short-run and long-run behavior.

Given some initial population state *i*) are not affected much by mutations, instead being governed mostly by selection strengths, population sizes, and generation times; (*ii*) depend critically on the starting point *iii*) have trajectories that are similar to those of the replicator dynamics, especially when the populations are large; and (*iv*) in coordination games, like game **2**, converge rapidly to or near one of the pure population states (in which each population is monomorphic) associated with the equilibria of the game. For illustration, Figs. 2 *A–D*, *Upper* and 3 *A–D*, *Upper* and especially *SI Appendix*, Fig. S1 display the similarity of the short-run behavior of our dynamics to that of the replicator dynamics, in the context of the mutualism game **2** [compare figure 2 of B&L (26)].

The long-run dynamics are of a very different nature. Because the evolutionary process, as we have defined it, is ergodic, the probability that the system is in some state in the future eventually becomes independent of where the system started (45). This effect is illustrated for the antagonistic symbiosis game in Fig. 1 *A–D*, *Lower* and for the mutualism game in Figs. 2 *A–D*, *Lower* and 3 *A–D*, *Lower*. The state of the system comes to depend not on the early dynamics that emanate from the starting point, but instead on infrequent transitions between equilibria, driven by mutations.

The object of interest in these long-run dynamics is their stationary distribution, the proportion of time spent in each population state in the long run. Equivalently, the stationary distribution tells us, were we to observe many independent instances of equivalent symbioses evolving, what proportion of these symbioses we should expect to find in each possible state at some fixed point in time in the long run (45).

Whereas we numerically study the stationary distribution of our evolutionary process for large mutation rates and selection strengths, we also invoke recent methodological advances in evolutionary game theory (46, 47) to study it analytically in certain limits. Chief among these is the “weak-mutation limit,”

In the weak-mutation limit, the dynamics converge to an embedded dynamical process over just the four pure states (42, 46); the stationary distribution collapses to a probability distribution over these pure states, *SI Appendix*, section S4).

In this paper, we examine the influence of individual rate parameters—mutation rate, generation time, selection strength, and population size—on the outcomes of antagonistic and mutualistic symbioses. For the majority of our analysis, we take the following “all-else equal” approach: For each rate parameter, we hold equal and constant for the two populations all but that parameter and then ask whether the population with the larger value of that parameter is, by some relevant criterion, evolutionarily more successful.

This all-else equal approach is motivated by two considerations: First, it allows for simple mathematical characterization of the long-run dynamics in the weak-mutation limit. Second, it allows us to ask in a clear way, “What is the contribution of a given parameter to the success of the populations?” For illustration, suppose we see that a large population of parasites does well against a small host population. We might wonder whether it is the parasite’s large population size or its rapid generation time that is the main reason for its success. To get at an answer, we would ask what the effect of the parasite’s large population size would be if we eliminated any benefit from a faster generation time—i.e., by setting the parasite’s generation time equal to its host’s. We would then do the same for generation time, by setting the parasite and the host population sizes equal.

This approach ignores possible interactions between rate parameters—for example, if the parasite’s larger population size is beneficial only if the parasite has a faster generation time than its host’s. For antagonistic symbioses, in the weak-mutation limit, we will be able to relax the all-else equal simplification to prove more general results about the long-run effect of the various rate parameters. For mutualisms, our long-run weak-mutation analytical results do require the all-else equal assumption. We give some numerical suggestion that they hold qualitatively when we relax this assumption.

## Antagonistic Symbioses

In this section, we study the evolutionary dynamics of populations interacting according to the antagonistic symbiosis game **1**. An appropriate measure of the relative success of population 1 in a given population state is the proportion of

### Strong Mutation, Strong Selection.

We begin by numerically studying the short- and long-run dynamics, when mutations are not very rare and selection is not very weak.

We first set the two populations’ sizes, selection strengths, and mutation rates to be equal (*A*): When population 1 has a longer generation time, the set of initial states from which, after 50 generations, population 1 is on average more successful is greater than the set of initial states for which the reverse is true. We find a more pronounced Red Queen effect in the long run, when the dynamics have become independent of where they started (Fig. 1*A*).

We then set the populations’ generation times equal (*C*). We find similar Red Queen effects, especially in the long run, when populations differ in their mutation rates (Fig. 1*B*) or in their sizes (Fig. 1*D*).

We have spoken loosely of the “short run” and “long run.” Whereas 50 generations are certainly short run, the long run should be more precisely defined as the time by which the distribution over population states is close to the stationary distribution. In the language of Markov chains, this is known as the “mixing time” of the dynamical process (51).

For small population sizes, the time evolution of the probability distribution over states can feasibly be computed given any starting point. *SI Appendix*, Fig. S6 shows that, in the antagonistic dynamics, for the parameter values used in Fig. 1 (with small populations, of size 50), we may speak of the long run as being after 100–1,000 generations. For larger population sizes, it is not computationally feasible to compute the time evolution of the probability distribution over states, and we must resort to approximate analytical arguments.

We give such arguments in *SI Appendix*, section S3. Here, we summarize their conclusions. We assume that mutations are infrequent. When selection acts weakly in at least one of the populations *SI Appendix*, section S5.2). That the mixing time does not increase rapidly with increasing population size, and in fact decreases in the case of strong selection, indicates that the stationary distribution will be relevant on realistic timescales.

### Weak Mutation.

To gain a greater understanding of the above results, we study the analytically tractable case of rare mutations,

In this “weak-mutation limit,” the long-run stationary distribution collapses to a distribution over just the four pure states,

First, we examine the influence of generation time and mutation rate on the success of population 1. We fix *SI Appendix*, section S5.4). The relative success of population 1 is*SI Appendix*, section S5.3).

A larger population size and a stronger selection strength of population 1 can also be shown to increase its relative success, **4** (*SI Appendix*, sections S5.1 and S5.2). Again, these results hold more generally: For many evolutionary processes, including the Moran and Wright–Fisher processes, they hold even if the generation times, mutation rates, and selection strengths are not set equal for the two populations.

## Mutualistic Symbioses

In this section, we study the evolutionary dynamics of populations interacting according to the mutualism game **2**. We take the measure of relative success of population 1 in a given population state to be the proportion of

### Strong Mutation, Strong Selection.

As we did for the antagonistic game in the previous section, we begin by numerically studying the short- and long-run dynamics, when mutations are not rare and selection is not weak. We consider small and large values of

When *A*). This is similar to B&L’s (26) general replicator-dynamics result for

In the long run, however, the average proportion of *A–D*, *Lower*). In this case, we find that differences in generation time between the two populations have almost no effect on their relative success (Fig. 2*A*).

We also find that differences in the populations’ relative mutation rates *B*).

If, instead, we allow the populations’ selection strengths *C*). A similar result is found when the populations have equal rate parameters except for their population sizes: When population 1 is smaller (so that it evolves slower—see *SI Appendix*, section S5.2 for a precise statement of this), it suffers a small disadvantage in the short run (a Red Queen effect), but an advantage in the long run (a Red King effect) (Fig. 2*D*).

When *C*). This is similar to B&L’s (26) general replicator-dynamics result for *A*, *B*, and *D*, *Upper* and *SI Appendix*, Fig. S1 look very similar to B&L’s figure 2 (26). Again, generation time and mutation rate differences have little effect on the long-run dynamics (Fig. 3 *A* and *B*). In contrast, a strong Red Queen effect is found in the long run when the populations differ in their selection strengths (Fig. 3*C*), with population size having a weak Red Queen effect in the long run (Fig. 3*D*).

To summarize, when mutation rates are not very small, we find, for

Again, the appropriate definition of the long run is how long it takes for the dynamics to get close to their stationary distribution. This can be computed exactly for small population sizes. In the mutualism game **2**, for the parameter values we have considered in Figs. 2 and 3, *SI Appendix*, Fig. S6 suggests that the long run could be considered any time after about 1,000–10,000 generations.

For larger population sizes, such computations are not feasible, and we must again resort to approximate analytical arguments. These are detailed in *SI Appendix*, section S3. We summarize their conclusions here. Mutations are assumed to be infrequent. When selection acts weakly in both populations (

The exponential term in this last expression is a result of requiring substitutions against selection for the process to mix. It means that the mixing time will be prohibitively long when populations are large and selection acts strongly in them [mixing times increasing exponentially with population size have also been observed in single-population coordination games (54, 55), where transitions between equilibria require evolution against selection too]. In these cases, our stationary distributions will not be empirically relevant; evolution over realistic timescales will involve movement to an equilibrium and then stasis there, as in B&L’s (26) analysis. When populations are not large and selection is not very strong, then the stationary distribution will still be reached on a realistic timescale. This will also be true when the effective sizes of the populations are not large (56) or when the populations are subdivided into small subpopulations (57, 58), properties that hold for many mutualistic symbionts (59⇓–61).

### Weak Mutation.

Again, analytical results can be obtained for the long-run dynamics in the weak-mutation limit (

We first examine the influence of generation time and mutation rate. Setting *SI Appendix*, section S6.1). As expected, most time in the long run is spent in the two coordination states

Similar to the numerical results we obtained for the case where mutations are not very rare, neither the generation times nor the relative mutation rates of the two populations have any influence on the stationary distribution *SI Appendix*, section S6.1).

It turns out (mathematical details in *SI Appendix*, section S6.1) that transitions against selection play an important role in this weak-mutation result. For example, whereas a higher relative mutation rate (or shorter generation time) in population 1 renders the transition from

Numerical calculations suggest that this weak-mutation result is broadly robust to inequalities in population sizes and selection strengths (*SI Appendix*, Fig. S10 *C* and *D*).

The directional effects of selection strength and population size limit are more subtle. For selection strength, we equate population 1 and 2’s sizes at *SI Appendix*, section S6.3). Numerical calculations suggest that the “sufficiently large” population size is not very large: Results for *SI Appendix*, Fig. S10*A*).

The results are similar if we equate the selection strengths in the two populations at *SI Appendix*, section S6.4). Again, numerical calculations suggest that these results are robust to inequality in mutation rates and generation times and larger differences in the two populations’ sizes when both are sufficiently large (*SI Appendix*, Fig. S10 *B*, *E*, and *F*).

Intuition for these results can be gained by noting that any switch from one coordination state to the other involves two transitions: one transition out of the original equilibrium (against the direction of selection) and a subsequent transition into the new equilibrium state (in the direction of selection). Whether weaker selection (or a smaller population size) favors population 1 depends on which nonequilibrium state is more often passed during these two transitions. This is driven predominantly by the relative probabilities of the transitions against selection, because these probabilities are much more sensitive to population size, selection strength, and the payoffs involved.

Of the transitions against selection, transitions to

Therefore, when **6** precisely when

Contrariwise, if

In contrast to the minor influence of mutation rates and generation times in the long run, the effects of population size and selection strength are very large. For example, when *SI Appendix*, Fig. S10*A*). Changes of similar magnitude are seen when holding *SI Appendix*, Fig. S10*B*). These effects, caused by small changes in population size and selection strength, are larger than those caused by varying relative mutation rates and generation times across 10 orders of magnitude (*SI Appendix*, Fig. S10 *C* and *D*). Larger changes in population size and selection strength have enormous effects on the long-run relative success of the two populations (*SI Appendix*, Fig. S10 *E* and *F*).

### Weak Selection.

Returning to the effect of mutation rate and generation time, we have shown analytically that these have no effect on the long-run stationary distribution in the weak-mutation limit. To analytically study the long-run effect of mutation rates when they need not be negligibly small, we make use of another recent advance in evolutionary game theory (47). We impose that the population sizes

Unlike in the weak-mutation case, the stationary distribution of this process places nonnegligible weight on every possible population state, including those where one or both populations are polymorphic. Again, the relative success of population 1 depends on the long-run proportion of *SI Appendix*, section S7). So, when

## Discussion

We have placed the evolution of symbioses, both antagonistic and mutualistic, into a simple finite-population model and studied the effect of parameters that influence the rate of evolution of the participating populations on those populations’ relative success.

Among these “rate parameters,” mutation rate and generation time are perhaps the most responsive (34, 62⇓⇓⇓⇓⇓–68). Selection strengths, although also a clear determinant of evolutionary rate, seem more of a fixed property of an interaction, but could change over time if one population reduces its dependence on the interaction (3) or actively increases the dependence of its interactant (69). Population size is the most complicated of the rate parameters, because changes in the relative success of interacting populations (possibly driven by differences in their population sizes) are expected to manifest themselves in subsequent changes in the populations’ sizes (70).

It is important to realize that, although we have studied differences in the relative success of interacting populations caused by different evolutionary rate parameters, our results do not say anything directly about how the rate parameters themselves should evolve. For example, it is conceivable that a slower generation time leads to greater success for a population (i.e., that a Red King effect holds), but that a faster generation time is selected for within that population. In this example, slow evolution could be interpreted as a public good.

This has important implications for how results such as ours and those of B&L (26) should be interpreted in studies of molecular evolution (e.g., ref. 71). Although generation times and mutation rates determine the rate of nucleotide substitution at neutral sites (37), were we to find that a slower generation time or a lower mutation rate leads to greater success for a population in a mutualism, this would not necessarily imply that species involved in mutualisms should exhibit lower neutral substitution rates.

Examining the evolution of the rate parameters themselves would require a more complicated model than the one we have studied. In a population genetic setting, one could posit in each population two distinct genetic loci: a “strategy locus,” the alleles at which determine which game strategies their bearers play, and a “rate locus,” the alleles at which determine the value of a rate parameter such as mutation rate. The evolutionary dynamics would then inform how both the success of the populations and their evolutionary rates evolve. One setting where we might expect the evolutionary rate within a population to evolve to increase that population’s relative success is if the interacting populations are subdivided into many isolated symbioses (on “islands”), between which migration occurs [this model is similar to the “haystack model” considered by B&L (26)]. A rate parameter value that improves a subpopulation’s success on an island causes that subpopulation to send out more migrants—who bear the rate parameter value—to colonize other islands. Thus, in symbioses with a high degree of population structure, we might expect the results we have found in this paper to be informative of how evolutionary rate will evolve within populations.

In antagonistic symbioses, we have shown that faster evolution through any of the rate parameters leads to greater evolutionary success in our model, in both the short run and the long run—a clear Red Queen effect. Mutation rate and generation time play similar roles in determining the long-run outcome of antagonistic interactions when mutations are infrequent. As far as our results have implications for evolution within populations, this suggests that one population might compensate for a longer generation time by evolving an elevated mutation rate. Consistent with this, bacterial strains subjected to antagonistic interactions with bacteriophages often exhibit much higher mutation rates than control strains (72). A similar explanation has been suggested for the finding that, in mammals, generation time is positively correlated with the rate of intrachromosomal recombination (73), which, like mutation, is a generator of variation (1). With particular respect to selection strength, the “life–dinner principle” (3) holds in our model: In an antagonistic interaction, the population for whom the interaction matters more (the rabbit, not the fox) is evolutionarily more successful.

In mutualistic symbioses, we have uncovered an important influence of evolutionary timescale on the relative success of interacting populations, with results in the short run and long run often being in opposition. In the short run, the stochastic evolutionary dynamics that we have studied are similar to the deterministic dynamics studied by B&L (26), and our short-run results replicate theirs: We find a short-run Red Queen effect when **2** and a short-run Red King effect when

In the long run, we find a Red King effect when *I* could show you hills, in comparison with which you’d call that a valley” (ref. 4, p. 37)—and deleterious mutations do sometimes fix because of random drift (37, 74⇓⇓–77).

The long-run Red King and Queen effects in mutualisms, for small

Which results, short-run or long-run, are relevant to a specific case depends on which evolutionary timescale is appropriate. Again, this question is treated quantitatively in *SI Appendix*, section S3. In mutualisms, the long-run dynamics are characterized by transitions between the equilibria—involving evolution first against, and then with, selection—so the applicability of our long-run results depends on the timescale of these transitions. When both populations are very large and under strong selection, the time it takes for a transition between equilibria to occur can be so long as to be empirically irrelevant; here, the short-run dynamics are of more interest. When the populations have small effective population sizes (owing to small census size or population structure, for example) or selection acts weakly in them, equilibrium transitions can be frequent, and the long-run dynamics apply.

In our analysis, we have made several modeling decisions and simplifying assumptions, described below.

We have assumed that interactions are pairwise, for simplicity. The consideration of multiplayer games (2, 78⇓⇓⇓⇓–83) is a desirable extension and has been shown to influence the outcome of mutualism games in infinite populations (80) and antagonism and Snowdrift games in finite populations (2).

In deriving our main analytical results, we assumed that mutations are infrequent. This is a realistic assumption in many cases, but not always. *SI Appendix*, Fig. S5 shows that mutation rates in the two populations do not have to be exceedingly small for our weak-mutation analytical results to be an excellent quantitative match (46, 84, 85), and in *SI Appendix*, Figs. S2–S5 suggest that, even for very large mutation rates, the long-run relative success of the populations qualitatively matches our weak-mutation results in most cases. The only notable exceptions, all for the mutualism game **2**, are (*i*) for large *SI Appendix*, Figs. S2*C* and S5*C*); (*ii*) when the populations have different mutation rates and both their mutation rates are very large, then, contrary to our weak-mutation result where differences in mutation rates have no effect, we find a long-run Red King effect for small *SI Appendix*, Figs. S3*B* and S5*B*) and a Red Queen effect for large *SI Appendix*, Figs. S2*B* and S5*C*) (exception *ii* is consistent with our weak-selection result, Eq. **7**); and (*iii*) for small *SI Appendix*, Figs. S3*A* and S5*B*). A very recent methodological advance in stochastic evolutionary dynamics (86) suggests that tractable analysis of the stationary distribution outside the weak-mutation regime may soon be possible.

In finite-population evolutionary game theory, game payoffs *SI Appendix*, Fig. S5 suggests that our results are essentially unchanged when we use either translation.

The dynamics in our model, based on the Moran process, are highly stylized, especially in their assumption of constant population sizes. It would be interesting to study evolution in games **1** and **2**, using more complicated intergenerational population dynamics (91⇓–93).

Finally, the games themselves are particular simplifications of more complex interactions. One key simplification in games **1** and **2** is that strategies are discrete: **2** with examples that exhibit such discreteness, such as division of labor and Müllerian mimicry. However, many mutualisms are likely to involve continuous strategies: for example, how much energy to expend on a cooperative task, as in the degree to which an ant colony protects from herbivores the plant that houses and feeds it (94) and in turn the energy the plant dedicates to housing and feeding the ants. For such cases, an alternative, continuous-strategy model specification is appropriate.

An attractive option is a simple modification of a Nash bargaining game (95). Players 1 and 2 choose activity levels *SI Appendix*, section S8, we study the coevolutionary dynamics of populations of player 1s and 2s. The short-run dynamics in this game involve evolution to the equilibrium line, whereas the long-run dynamics involve drift-like movement along and around it.

In both the short and long runs, the faster-evolving population is at an advantage when *SI Appendix*, Fig. S11), but at a disadvantage when *SI Appendix*, Fig. S12). The short-run effects are driven by all rate parameters. But similar to what we found for the discrete mutualism game, the long-run effects are driven predominantly by selection strength and population size—differences in mutation rate and generation time have little effect. This is because selection strength and population size contribute differentially here to a case of drift-induced selection: Stochastic jumps off the equilibrium path return with an average directional bias. Drift-induced selection is a newly recognized phenomenon that has recently gained attention in the stochastic dynamics literature (96⇓–98). It can be studied analytically using diffusion approximations and separations of timescales (99). Such a study in the context of continuous-strategy mutualisms would be an important extension of our preliminary analysis.

In this vein, studying the finite-population dynamics of other games in which slow evolution has been hypothesized to be beneficial (100) would also be desirable.

## Acknowledgments

C.V. is grateful to Philipp Altrock for introducing him to the Red King literature. We thank Carl Bergstrom, Kirill Borusyak, Kooskia Burns-Edelman, Richard Childers, Michael Doebeli, David Haig, Jonathan Libgober, Alex McAvoy, Sam Sinai, Luis Zaman, and participants at the Harvard Economics Department’s Games and Markets lunch for helpful comments.

## Footnotes

- ↵
^{1}To whom correspondence should be addressed. Email: carlveller{at}fas.harvard.edu.

Author contributions: C.V., L.K.H., C.H., and M.A.N. designed research, performed research, analyzed data, and wrote the paper.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission. S.A.F. is a guest editor invited by the Editorial Board.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1702020114/-/DCSupplemental.

## References

- ↵
- ↵
- ↵.
- Dawkins R,
- Krebs J

- ↵.
- Carroll L

- ↵.
- Turner J

- ↵.
- Burt A,
- Trivers R

- ↵.
- McLaughlin RN,
- Malik HS

- ↵
- ↵
- ↵.
- Henikoff S,
- Ahmad K,
- Malik H

- ↵
- ↵.
- Boulton A,
- Myers R,
- Redfield R

- ↵.
- Baudat F, et al.

- ↵.
- Parvanov E,
- Petkov P,
- Paigen K

*Prdm9*controls activation of mammalian recombination hotspots. Science 327:835–835. - ↵
- ↵.
- Kingan SB,
- Garrigan D,
- Hartl DL

*Drosophila simulans*. Genetics 184:253–265. - ↵.
- Myers S, et al.

*PRDM9*gene in meiotic recombination. Science 327:876–879. - ↵
- ↵.
- Jeffreys AJ,
- Cotton VE,
- Neumann R,
- Lam KW

- ↵
- ↵
- ↵
- ↵.
- Baker Z, et al.

- ↵.
- Doebeli M,
- Knowlton N

- ↵
- ↵.
- Bergstrom C,
- Lachmann M

- ↵
- ↵.
- Hom E,
- Aiyar P,
- Schaeme D,
- Mittag M,
- Sasso S

- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵.
- Van Valen L

- ↵.
- Kimura M

- ↵
- ↵
- ↵.
- Nowak M

- ↵.
- Hartl DL,
- Clark AG

- ↵
- ↵.
- Hauert C,
- Traulsen A,
- De Silva H,
- Nowak M,
- Sigmund K

- ↵
- ↵.
- Karlin S,
- Taylor H

- ↵.
- Veller C,
- Hayward L

- ↵
- ↵
- ↵
- ↵
- ↵.
- Levin D,
- Peres Y,
- Wilmer E

- ↵
- ↵.
- Sekiguchi T,
- Ohtsuki H

- ↵
- ↵
- ↵
- ↵.
- Whitlock MC

- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵.
- Nowak M,
- May R

- ↵.
- Stearns S,
- Ackermann M,
- Doebeli M,
- Kaiser M

- ↵.
- Vérin M,
- Menu F,
- Rajon E

- ↵.
- Wyatt G,
- Kiers E,
- Gardner A,
- West S

- ↵.
- Schenk H,
- Traulsen A,
- Gokhale C

- ↵.
- Rubin B,
- Moreau C

- ↵
- ↵
- ↵
- ↵.
- Ohta T

- ↵.
- Lynch M

- ↵.
- McCandlish D,
- Epstein C,
- Plotkin J

- ↵.
- Gokhale C,
- Traulsen A

- ↵
- ↵.
- Gokhale C,
- Traulsen A

- ↵
- ↵
- ↵.
- McAvoy A,
- Hauert C

- ↵
- ↵.
- Rand D,
- Tarnita C,
- Ohtsuki H,
- Nowak M

- ↵.
- Vasconcelos V,
- Santos F,
- Santos F,
- Pacheco J

- ↵
- ↵.
- Traulsen A,
- Nowak M,
- Pacheco J

- ↵
- ↵.
- Cooney D,
- Allen B,
- Veller C

- ↵
- ↵.
- Song Y,
- Gokhale C,
- Papkou A,
- Schulenburg H,
- Traulsen A

- ↵.
- Papkou A,
- Gokhale C,
- Traulsen A,
- Schulenburg H

- ↵
- ↵
- ↵.
- Constable G,
- McKane A,
- Rogers T

- ↵.
- Constable G,
- Rogers T,
- McKane A,
- Tarnita C

- ↵.
- Veller C,
- Muralidhar P,
- Constable G,
- Nowak M

- ↵.
- Parsons T,
- Rogers T

- ↵.
- Hilbe C,
- Nowak M,
- Sigmund K

## Citation Manager Formats

## Article Classifications

- Biological Sciences
- Evolution