## New Research In

### Physical Sciences

### Social Sciences

#### Featured Portals

#### Articles by Topic

### Biological Sciences

#### Featured Portals

#### Articles by Topic

- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology

# Stochastic game dynamics under demographic fluctuations

Edited by Martin A. Nowak, Harvard University, Cambridge, MA, and accepted by the Editorial Board June 11, 2015 (received for review September 29, 2014)

## Significance

This contribution breaks with the tradition to restrict stochastic evolutionary game dynamics to populations of constant size and introduces a theoretical framework to investigate relevant and natural changes arising in populations that vary in size according to fitness—a feature common to many real biological systems. Explicitly including ecological variation can result in significant effects on the stochastic evolutionary trajectories while providing a transparent link to the established, deterministic Lotka–Volterra systems.

## Abstract

Frequency-dependent selection and demographic fluctuations play important roles in evolutionary and ecological processes. Under frequency-dependent selection, the average fitness of the population may increase or decrease based on interactions between individuals within the population. This should be reflected in fluctuations of the population size even in constant environments. Here, we propose a stochastic model that naturally combines these two evolutionary ingredients by assuming frequency-dependent competition between different types in an individual-based model. In contrast to previous game theoretic models, the carrying capacity of the population, and thus the population size, is determined by pairwise competition of individuals mediated by evolutionary games and demographic stochasticity. In the limit of infinite population size, the averaged stochastic dynamics is captured by deterministic competitive Lotka–Volterra equations. In small populations, demographic stochasticity may instead lead to the extinction of the entire population. Because the population size is driven by fitness in evolutionary games, a population of cooperators is less prone to go extinct than a population of defectors, whereas in the usual systems of fixed size the population would thrive regardless of its average payoff.

All natural populations are composed of a finite number of individuals. These individuals can reproduce, interact, die, or migrate, which leads to changes in the population size over time. In many theoretical models, it is convenient and possible to neglect the effect of demographic fluctuations by assuming infinite populations when population sizes are sufficiently large (1) or by assuming constant population size as in the Moran or Wright–Fisher process (2). However, such simplifications may be invalid when considering additional ecological processes including oscillations in population size of predator and prey systems (3⇓–5), periodic fluctuations and outbreaks of infectious diseases in humans (6), or chaotic dynamics under multispecies interactions (7). The Lotka–Volterra equations provide a deterministic description of the abundances of species as continuous densities but they are not designed to include the impact of random drift. Theoretical models coupling changing population size and stochastic dynamics arising from individual based models have become more popular only recently (8⇓⇓⇓–12).

The Lotka–Volterra equations naturally take frequency-dependent selection into consideration. Under frequency-dependent selection, the fitness of a given type (or species) depends on the composition of the entire population (or community) (13). Different kinds of frequency dependence can lead to different dynamical patterns. Most prominently, negative frequency-dependent selection can result in a stable coexistence of different types (14⇓⇓⇓–18). One elegant way to describe such frequency dependence is through evolutionary game dynamics (13, 19). Evolutionary game theory has extensively developed the theory of stochastic dynamics in the past decade (2). However, most progress has been accomplished for constant, finite population sizes, which is mathematically convenient but does not always reflect biologically appropriate scenarios. Game theoretic models that take changing population sizes into account mostly focus on deterministic dynamics (20)—similar to traditional ecological models. Here we introduce a simple and elegant model of stochastic evolutionary game dynamics that explicitly allows for changing population size through a natural interpretation of payoffs in terms of competition between individuals.

Traditional game theoretic models assume that individuals obtain payoffs from interactions with other members in the population, which are translated into fitness. Individuals with higher fitness are assumed to have more offspring and hence reproduction is frequency-dependent. Conversely, death rates are assumed to be constant or normalized so as to keep the population size constant. In contrast, here we focus on a microscopic description of the dynamics in terms of reaction kinetics equations. Assuming constant birth rates and frequency-dependent death rates allows one to interpret payoffs directly and naturally in terms of reaction rates, where selection acts on survival rather than reproduction. The present setup lends itself to a straightforward derivation of the deterministic dynamics in the limit of large population sizes in the form of competitive Lotka–Volterra equations but equally allows one to model the stochastic dynamics in finite populations of variable sizes, which may even lead to the extinction of the population.

## Model and Results

### Stochastic Dynamics.

Most models for stochastic evolutionary game dynamics consider a fixed population size, such that every birth is balanced by the death of another individual (2). Simply decoupling birth and death events in such models leads to random fluctuations in the population size and thus eventually to stochastic extinction (21). Instead, here we propose a framework based on the microscopic processes of birth, death, and competition. For simplicity, we focus on two types of individuals, *X* and *Y*, but note that the generalization to arbitrary numbers is straightforward. Every individual reproduces

### Competitive Selection.

The most natural way to introduce evolutionary games in the above framework is to relate the competition rates in Eq. **3** to a payoff matrix*X*, *Y* individuals as *d* such that individuals obtaining higher payoffs are less likely to die in competitive interactions. More specifically, we assume that reaction rates are the inverse payoffs scaled by *M*, a unit for controlling population size,*X* and a *Y* individual compete, the *X* dies with a probability proportional to *Y* proportional to *M* determines the frequency of competitive interactions compared with birth or death events, Eqs. **1** and **2**. As long as population sizes are much smaller than *M*, competition is rare and most events are births or deaths. In large populations, however, competition becomes common and results in density-dependent regulation of the population size.

Implementing evolutionary games through competition is, of course, just one approach to link payoff matrices to reaction rates. Intuitively, it is tempting to assume that evolutionary games determine the birth rates instead because payoffs then more directly reflect fitness advantages. However, this requires microscopic reactions of the form *a*, and so on. Such interactions seem more appealing in sexually reproducing populations rather than for the more traditional models, which focus on one sex only or are based on asexual reproduction. More importantly, however, even when including competition at fixed rates, this setup remains inherently biologically unmeaningful, because it either results in extinction or indefinite growth of homogeneous populations (*SI Appendix*, *section 1*). In natural systems, there can be positive feedbacks between different types (22), but they typically refer to different systems where other effects, such as predation, guarantee that the population size remains finite. Here, we focus on a competitive system with negative feedback instead.

### Large Population Size.

The reaction-based system above can be formulated in terms of a continuous-time master equation (*SI Appendix*, *section 2*). For large *M*, a measure of the equilibrium densities, this equation can be approximated by a Fokker–Planck equation, which describes the dynamics of the probability distribution of the system (23). When the population densities approach the equilibria, we recover deterministic rate equations from the microscopic processes defined in Eqs. **1**–**3**:*x* and *y* denote the density of individuals of type *X*, *Y* and **1** and **2**. Note that the deterministic limit can be derived directly based on the law of mass action. Even though only the net growth rates, **5**, it is important that the stochastic description does not lump the two processes together in either decreased net birth or net death rates. In particular, if spontaneous death events, Eq. **2**, are dropped (or absorbed in reduced birth rates, Eq. **1**) such that deaths occur only due to competition, Eq. **3**, then populations would never go extinct in the stochastic formulation because the last individual standing would remain immortal.

To recover the familiar form of the competitive Lotka–Volterra dynamics (24, 25), we factor out **5** and set *X* types and *Y* types. In the absence of *Y* types Eq. **6a** reduces to the logistic equation*r*-*K*-selection theory (26), where the carrying capacity *K* is independent of the intrinsic growth rate, **5**, *K* is an emergent quantity determined by the population’s environment (27⇓⇓–30), which crucially includes all members of the population together with their ecological interactions. For example, if a mutant type *Y* doubles its intrinsic rate of reproduction compared with the resident *X*, *X* and type *Y* individuals at this equilibrium are

If

### Equilibria of the Deterministic System.

The deterministic mean-field dynamics of our model serves as a valuable reference for the underlying stochastic evolutionary process. Birth and death rates, Eqs. **1** and **2**, may differ for different types, but for the sake of simplicity and to highlight effects arising from evolutionary games we discuss interactions of two types, *X* and *Y*, with *a* and *d*) and intertype competition (*b* and *c*). In general, we can classify three different scenarios based on the payoff matrix. First, for *X* individuals invariably achieve higher payoffs (i.e., longer life expectancy) than *Y* types and hence type *X* dominates type *Y*. Similarly, type *Y* dominates type *X* whenever *A*). The prisoner’s dilemma is the most prominent example of a dominance game (19). Second, for *B*). The hawk–dove or snowdrift games are examples of such scenarios (35). Note that even though the classification of the dynamics for two types is based on their payoffs in the same way as in the classical replicator dynamics (and the stability remains the same; see *SI Appendix*, *section 3*), the position of the rest points in our deterministic system are naturally different. For example, in the replicator dynamics a coexistence game as in Fig. 1*B* exhibits a stable rest point at **6**, the frequency of *X* at equilibrium is *X* is that the total number of individuals at the mixed equilibrium is lower than the carrying capacity for a population of only *X* types.

Therefore, the only possible equilibria are either homogenous *X* or *Y* populations or a stable heterogenous mixture of the two. According to Eq. **5** with *SI Appendix*, Table 1.

### Stochastic Simulations.

In contrast to the deterministic equilibrium predictions, the only evolutionary outcome in stochastic simulations is the eventual extinction of the entire population—all other states are transient. Fortunately, the expected times to extinction rapidly grow with the density of individuals in equilibrium, controlled by *M* (Fig. 2). Hence, predictions based on deterministic dynamics, Eq. **5**, keep providing valuable insights for the stochastic dynamics, Eqs. **1**–**3**, especially for large population sizes. Substantial quantitative and even qualitative differences arise, as illustrated in Fig. 1 for three characteristic types of interactions. The stochastic dynamics is implemented through the Gillespie algorithm (36). For the relatively large carrying capacity

In small populations, competition for survival is weak and the dynamics is mainly determined by the intrinsic growth rate, *r* (i.e., individual birth and death events). Consequently, small *r* results in higher stochasticity but also tends to decrease the number of individuals at equilibria, which further amplifies the effect. As the population grows and approaches its carrying capacity, competition becomes increasingly important and competition rates (payoff matrix, Eq. **4**) also control the size of fluctuations. Strong competition (small payoffs) reduces stochasticity but also tends to decrease the population size, which may offset the reduction in terms of fluctuations.

Furthermore, if the numbers of the two types in the stochastic process are far away from the deterministic equilibrium, even the averaged stochastic dynamics can be very different from deterministic predictions. For example, in the deterministic case a dominant mutant always succeeds in invading and eventually fixating in the population. In contrast, in the stochastic case a single mutant often fails to invade and fixate even if it is dominant. Note that fixation of a single mutant becomes even less likely in larger populations—despite the fact that fluctuations decrease and the deterministic dynamics is recovered in the limit of large populations.

### Extinction.

In ecological models the risk of extinction of a population owing to demographic stochasticity has recently received considerable attention (see, e.g., refs. 10, 37, and 38). Evolutionary game theory models demonstrated that stochastic fluctuations are important determinants for the fixation and extinction of individual traits even under constant population sizes (2) but remained unable to address the more dramatic possibility of the extinction of the entire population.

In a dominance game, the deterministic dynamics predicts that the dominant type invariably takes over the entire population and approaches its carrying capacity (Fig. 2*A*). However, in stochastic models the two types *X* and *Y* may go extinct sequentially owing to fluctuations. Especially when starting from small populations, the extinction probability is not negligible (Fig. 2*B*). Note that the (cumulative) extinction probability converges to 1 as time goes on regardless of the population size. Only for larger populations extinction typically takes much longer and the extinction probability increases slowly with time. The smaller the population size—owing to small carrying capacities, fluctuations, or initial configurations—the higher the risk of extinction.

In Fig. 2*A*, the dominant *X* type has a higher carrying capacity than the *Y* type and, hence, provided that the *X* mutant successfully invades and fixates, the population ends up more persistent. In the third realization in Fig. 2*A*, the *X* type is lost first and the entire population vanishes soon after. An essential factor for the persistence of small populations are the birth and death rates, Eqs. **1** and **2**, or, more specifically, their difference: For large intrinsic growth rates, small populations are more likely to escape extinction because the population can more readily recover and return to its carrying capacity.

In coexistence games, the two types *X* and *Y* typically coexist and their densities fluctuate around their respective (deterministic) densities of individuals in equilibrium, *B*). Whenever stochastic fluctuations drive one type to extinction, the carrying capacity of the remaining type changes to

In this case the extinction of the entire population is a two-step process: First one types goes extinct—typically the type with the lower density in equilibrium—and then the population fluctuates around the homogenous carrying capacity of the remaining type such that the extinction dynamics is now governed by the stochastic equivalent for the logistic growth of a single type, which is well understood (10, 24, 37).

### Snowdrift Game.

As a concrete example of a coexistence game, we consider the stochastic dynamics of the snowdrift game (35) [or, equivalently, the hawk–dove game (19)]. In the snowdrift game two individuals need to finish a task, which provides benefits *β* to both. The costs of the task, *γ*, are shared equally if both cooperate (i.e., participate in completing the task). If only one participates, the cooperator has to bear the entire costs but the defector still receives the benefits. Finally, if both defect and refuse to participate, the payoffs for both individuals are close to zero. Hence, the payoffs of cooperators, *X*, and defectors, *Y*, are *Y* types due to competition diverges and they are no longer able to persist in isolation. To maximize its own gain, each player should do the opposite of what its opponent does. A population of cooperators fluctuates around *SI Appendix*, Table 1). For sufficiently small *ε*, the conditions for the ranking

## Discussion

Demographic fluctuations based on ecological interactions capture important aspects and represent crucial determinants of evolutionary trajectories, especially in smaller populations. Here, we introduce a simple stochastic framework built on the microscopic events of birth, death and competition. This framework admits a simple yet elegant way to implement evolutionary games through payoff based competition rates, which results in selection on survival instead of the more traditional fecundity based selection (30). This yields a stochastic model for evolutionary games in populations of changing and fluctuating finite size. In the limit of infinite population sizes, this framework recovers the deterministic dynamics of the competitive Lotka–Volterra equations and hence allows to pinpoint and emphasize differences that arise due to stochastic effects. The deterministic limit of the stochastic framework also highlights that, in contrast to the classical *r*-*K*-selection theory, the ecological carrying capacity of a population is an emergent quantity (27, 28), which depends on the population configuration and is determined by the underlying processes of birth, death, and competition. In particular, mutations that alter the rates of any of these processes trigger a change in the (deterministic) carrying capacities of the mutant population, provided that it succeeds to take over, or of the mixed population in the case of coexistence. Our model implies that adaptation is not a simple process of accumulating beneficial mutations with higher carrying capacities in isolation, but instead an adaptive process that can favor invasion and fixation of mutations that are disadvantageous for the entire population including evolutionary suicide (39).

Dominant mutations are bound to take over with certainty under deterministic dynamics. However, in the stochastic scenario, the chances for a single beneficial mutant to successfully invade and take over remain small, even for a dominant strategy. At first it might be surprising that the chances of success decrease for increasing population sizes—despite the fact that the limit of large populations recovers the deterministic dynamics. However, of course, in this limit the mutant density converges to zero, which resolves the apparent contradiction. Similar results can be found in classical models of finite populations with constant fitness values (40). Here we investigated stochastic dynamics in well-mixed populations but a natural extension is to consider spatial dimensions, which may increase stochastic effects due to small local subpopulations (41⇓–43).

Here, we have focused on the paradigmatic case of one population and two types, but it is straightforward to extend the framework to include multiple types. In a population with three types, oscillations can persist in the stochastic process, whereas the deterministic limit suggests cycles spiraling toward an internal equilibrium (44, 45) (*SI Appendix*, Fig. 1). Moreover, our framework easily extends to group interactions, such as public goods games, by allowing for competitive interactions that involve more than two individuals. However, in either case, the number of microscopic interactions tends to increase rapidly and hence hampers a more general yet compact and intuitive presentation.

The stochastic framework also emphasizes that in the long run populations invariably go extinct, which means that the deterministic equilibria merely indicate fleeting states—albeit the expected time to extinction can be exceedingly long, especially for larger populations. Therefore, it remains reasonable to consider the deterministic predictions as a baseline superimposed by fluctuations of stochastic realizations. At the same time it is crucial, especially in smaller populations, to consider the persistence of individual traits or the viability of the entire population. For example, in the snowdrift game an invasion attempt by defectors triggers ecological feedback, which alters the carrying capacities in favor of cooperators, such that stochastic fluctuations help to eliminate defectors and reestablish homogenous cooperation.

## Acknowledgments

This work was supported by the Natural Sciences and Engineering Research Council of Canada (C.H.) and Foundational Questions in Evolutionary Biology Fund Grant RFP-12-10 (to C.H.) and the Max Planck Society (W.H. and A.T.).

## Footnotes

- ↵
^{1}To whom correspondence should be addressed. Email: traulsen{at}evolbio.mpg.de.

Author contributions: W.H., C.H., and A.T. designed research, performed research, and wrote the paper.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission. M.A.N. is a guest editor invited by the Editorial Board.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1418745112/-/DCSupplemental.

Freely available online through the PNAS open access option.

## References

- ↵
- ↵
- ↵.
- Cohen JE,
- Luczak T,
- Newman CM,
- Zhou ZM

- ↵.
- Krebs CJ, et al.

- ↵.
- Gilg O,
- Hanski I,
- Sittler B

- ↵.
- Rohani P,
- Earn DJD,
- Grenfell BT

- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵.
- Nowak MA

- ↵
- ↵.
- Gigord LDB,
- Macnair MR,
- Smithson A

*Dactylorhiza sambucina*(L.) Soo. Proc Natl Acad Sci USA 98(11):6253–6255 - ↵.
- Nowak MA,
- Sigmund K

- ↵
- ↵
- ↵.
- Maynard Smith J

- ↵.
- Hauert C,
- Holmes M,
- Doebeli M

- ↵.
- Gardiner CW

- ↵
- ↵.
- van Kampen NG

- ↵
- ↵.
- Hofbauer J,
- Sigmund K

- ↵
- ↵
- ↵.
- Mallet J

*K*, obscures the links between demography, Darwinian evolution, and speciation. Evol Ecol Res 14(5):627–665 - ↵
- ↵
- ↵
- ↵.
- Bowers R,
- Whilte A,
- Boots M,
- Geritz SAH,
- Kisdi E

- ↵
- ↵.
- Skyrms B

- ↵
- ↵
- ↵
- ↵
- ↵
- ↵.
- Kimura M

- ↵
- ↵
- ↵
- ↵
- ↵

## Citation Manager Formats

## Sign up for Article Alerts

## Article Classifications

- Biological Sciences
- Ecology