## New Research In

### Physical Sciences

### Social Sciences

#### Featured Portals

#### Articles by Topic

### Biological Sciences

#### Featured Portals

#### Articles by Topic

- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology

# Evolutionary theory for modifiers of epistasis using a general symmetric model

Communicated by Samuel Karlin, Stanford University, Stanford, CA, October 4, 2006 (received for review May 15, 2006)

## Abstract

Genetic interactions in fitness are studied by using modifier theory. The effects on fitness of two linked genes are perturbed by alleles at a third linked locus that controls the extent of epistasis in fitness between the first two. This epistasis is determined by a symmetric interaction matrix, and it is shown that a modifier allele that increases epistasis will invade when the linkage between the other two genes is sufficiently tight and these genes are in linkage disequilibrium. With linkage equilibrium among the major loci, increased or decreased epistasis may evolve depending on the allele frequencies at these loci.

In quantitative genetics, epistasis is viewed as a contribution to overall statistical variation in a phenotype by terms due to gene-by-gene interactions. A number of recent studies have focused on the evolution of this epistatic variance (1⇓–3). One conclusion of these studies is that, for the most part, evolution should proceed in the direction of more modularity, that is, toward a decrease in the relative contribution of the epistatic component of variance (4, 5).

Epistatic variance in the context of quantitative genetics is a statistical average of contributions in a linear analysis-of-variance framework. However, in biological terms, there may be substantial variability in the amount of interaction between genes and the type of interaction. Phillips *et al.* (6) claim that evolutionary genetics “may have significantly underestimated the importance of gene interactions” by focusing on statistical averages of interaction effects. Wright (7) suggests that overall genotypic fitness is likely to involve interaction among a large fraction of genes in the genome. Further, the evolutionary dynamics of linked genes may be sensitive to the precise nature of the epistatic interactions among them. Indeed, variance in epistasis can generate different evolutionary outcomes (8).

Although there was a period in the 1990s during which the existence and importance of epistasis were somewhat neglected, there has been a shift in the last decade toward more detailed modeling, both at the evolutionary level and in the analysis of quantitative variation (1⇓–3, 9, 10). There is an emerging consensus that intraspecific genetic interactions may be important in development and evolution (11, 12). Different approaches to modeling the evolution of epistasis have developed, depending on whether the approach originates in the statistics of quantitative inheritance (1) or in the formal population genetics of genetic modifiers of fitness (13).

A recent computational approach to the evolution of epistasis (14) is based on an artificial gene network model (15, 16) originally introduced to study the evolution of genetic robustness. The model in ref. 14 also includes mutation in the gene networks, and among the results of the simulations is the finding that recombination strongly affected the direction of the epistasis that evolved during selection for genetic robustness. The claim in ref. 14 is that negative epistasis among genes should be widespread in nature, because there are so many interacting gene networks and so much genetic variation in real organisms.

Much of the computational analysis of multiple gene systems has focused on haploids with deleterious mutations only (e.g., ref. 17). For such haploid systems, it is easy to write fitness as a function of the deleterious effect of each single mutation and a single epistatic parameter that describes how the multiple mutant fitness decays as a function of the number of mutants; the latter is the epistasis parameter. The analysis in ref. 17 suggested that in haploids with deleterious mutations and positive epistasis, evolution should reduce this epistasis during selection to reduce the overall genetic load.

Epistasis in diploids is somewhat harder to study, because each set of chromosomes must be evaluated in the backgrounds of all chromosomes. Thus, it is clear that some assumptions will be necessary just to pose the problem of the evolution of epistasis. The present study models explicit epistatic fitness interactions between a pair of loci in a diploid genetic system and follows the fate of alleles at a third locus, which modifies the fitness epistasis between the first two. To do this, we must address the kinds of evolutionary dynamics expected under epistatic selection, including the roles of linkage disequilibrium and recombination. To this end, we consider a “baseline” model of additive fitnesses that entails no additive interactions between the loci and frames the problem as the evolution of modifier alleles that cause departure from additivity (see, e.g., ref. 18). In an earlier study of this kind with the Lewontin–Kojima (19) model of fitness interactions, we found that an allele that increased epistatic interactions would increase in frequency when introduced near a stable state of linkage disequilibrium (13). In this work, we study fitness schemes that are not as symmetric as the Lewontin–Kojima viabilities, although they fall under a more general class of symmetric viability models (20, 21). We show that, with sufficiently tight linkage between the major loci, epistasis will tend to increase, but that with some general classes of symmetric viabilities, looser linkage will result in the success of alleles that decrease epistasis.

## Model

The model assumes a pair of “major” loci with two alleles *A, a* at the first and *B, b* at the second, and a third “modifier” locus whose sole function is to determine properties of the fitness parameters at the major loci. At the modifier locus, there may be *n* possible alleles *M*_{1}, *M*_{2}, …, *M*_{n}. Thus, there are four possible gametes at the two major loci *AB, Ab, aB, ab* numbered 1, 2, 3, 4, respectively, and *4n* three-locus gametes *ABM _{i}, AbM_{i}, aBM_{i}, abM_{i}*, for

*i*= 1, 2, …,

*n*.

The modifier locus determines the fitness parameters at the major loci with a family of 4 × 4 fitness matrices **W*** ^{ij}* = [

*w*] for

_{kl}^{ij}*k*,

*l*= 1, 2, 3, 4 and

*i*,

*j*= 1, 2, …,

*n*. For example,

*w*

_{11}

^{ij}is the fitness of the three-locus genotype

*ABM*, and

_{i}/ABM_{j}*w*

_{23}

^{ij}is the fitness of

*AbM*, etc. The fitness parameters are symmetric; that is,

_{i}/aBM_{j}*w*

_{kl}

^{ij}=

*w*

_{kl}

^{ji}=

*w*

_{lk}

^{ij}=

*w*

_{lk}

^{ji}for all

*k*,

*l*= 1, 2, 3, 4 and

*i*,

*j*= 1, 2, …,

*n*. In addition, there are “no position” effects in the fitness parameters, so that for all

*i*,

*j*= 1, 2, …,

*n*,

*w*

_{14}

^{ij}=

*w*

_{23}

^{ij}.

Recombination occurs between the two major loci as well as between the modifier locus and the major loci. For simplicity (but with no effect on the results), take the first two loci to be the major loci with the modifier locus next to them, as shown in Fig. 1. Recombination can occur in the two genetic intervals *I*_{1} and *I*_{2}, resulting in four mutually exclusive recombination events (0, 0), (0, 1), (1, 0), (1, 1), where (0, 0) denotes that no recombination occurred in both *I*_{1} and *I*_{2}, (1, 0) indicates that recombination occurred in *I*_{1} but not in *I*_{2}, etc. There are, therefore, four crossover probabilities *c*_{00}, *c*_{01}, *c*_{10}, *c*_{11}, such that *c*_{00} + *c*_{01} + *c*_{10} + *c*_{11} = 1.

If *r* is the recombination rate between the two major loci and *R* between the modifier locus and the two major loci, then
The crossover probabilities can be represented as
where ζ measures the extent of interference in recombination. Following Haldane (22), ζ < 0 corresponds to the case of negative interference, ζ > 0 to positive interference, and ζ = 0 entails no interference, in which case the recombination events in *I*_{1} and *I*_{2} are independent.

Let the frequencies of the four gametes at the major loci, *AB, Ab, aB, ab,* in the “present” generation be *x*_{1}, *x*_{2}, *x*_{3}, *x*_{4}, respectively. The three-locus frequencies of the 4*n* possible gametes can be represented as
where 0 ≤ *p*_{i}, *q*_{i}, *r*_{i}, *s*_{i} ≤ 1, and Σ_{i}*p _{i}* = Σ

_{i}*q*= Σ

_{i}

_{i}*r*= Σ

_{i}

_{i}*s*= 1. Here, we use notation suggested by Lessard (23), in that

_{i}*p*, for example, is the frequency of

_{i}*ABM*among

_{i}*AB*gametes. The population state at the present generation is thus specified by the frequency vectors

**x**,

**p**,

**q**,

**r**,

**s**, where Let

**x′**,

**p′**,

**q′**,

**r′**,

**s′**be the frequency vectors for the population state at the “next” generation after random mating, recombination, Mendelian segregation, and viability selection have taken place. The transformation of frequencies from the present to the next generation is given in equations 2.12–2.15 in ref. 13. For ready reference, see supporting information (SI)

*Supporting Text 1*.

The modifier locus determines the fitness parameters at the major loci through the fitness matrices **W*** ^{ij}* = [

*w*

_{kl}

^{ij}]. Specifically,

*w*

_{kl}

^{ij}is the fitness parameter associated with the

*kl*genotype at the major loci with the genotype

*M*at the modifier locus.

_{i}M_{j}We assume that fitness modification is “linear,” namely
for all *k*, *l* = 1, 2, 3, 4, and *i*, *j* = 1, 2, …, *n*. Thus the “nonmodified” or the “basic” fitness matrix is **W** = [*w _{kl}*], and in the presence of the genotype

*M*at the modifier locus, the “modified” fitness parameters are

_{i}M_{j}*w*+ ε

_{kl}*δ*

_{ij}*, where*

_{kl}*ε*represents the effect of

_{ij}*M*on epistasis.

_{i}M_{j}The matrices **W** = [*w _{kl}*],

**Δ**= [δ

*], and*

_{kl}**ε**= [ε

*] are symmetric matrices;*

_{ij}**W**is a positive matrix; and because we assume no position effects in fitness, we also have

*w*

_{14}=

*w*

_{23}and δ

_{14}= δ

_{23}.

## Invasion by an Epistasis Modifier

In the simplest case, the population is initially at a stable equilibrium where the allele *M*_{1} at the epistasis-modifying locus is fixed, and the epistasis parameter is ε_{11}. A new allele *M*_{2} is introduced near this equilibrium, producing epistasis parameters ε_{12} for *M*_{1}*M*_{2} and ε_{22} for *M*_{2}*M*_{2}. We seek conditions on ε_{11} and ε_{12} that will enable *M*_{2} to increase when rare, that is, conditions for the original equilibrium with *M*_{1}*M*_{1} to be externally unstable.

Suppose the initial equilibrium frequencies of *ABM*_{1}, *AbM*_{1}, *aBM*_{1}, and *abM*_{1} are *x*_{1}*, *x*_{2}*, *x*_{3}*, *x*_{4}*, respectively, and write
Then the frequency vector **x*** = (*x*_{1}*, *x*_{2}*, *x*_{3}*, *x*_{4}*) satisfies
with *k* = 1, 2, 3, 4 (and “−” for *k* = 1, 4 and “+” for *k* = 2, 3).

The external stability of **x*** to invasion by *M*_{2} is given by the linear approximation ℒ for the frequencies *p*_{2}, *q*_{2}, *r*_{2}, *s*_{2} of *ABM*_{2}, *AbM*_{2}, *aBM*_{2}, and *abM*_{2}. This external stability is determined by the largest root in absolute value of the fourth-degree characteristic polynomial *Q(z)* associated with . Using ref. 13, we find that
where
*l** = δ_{14}*x**_{1}x*_{4} − δ_{23}*x**_{2}*x**_{3}, *d** = *w**_{14}*x**_{1}*x**_{4} − *w**_{23}*x**_{2}*x**_{3}, δ* = Σ_{l,k=1}^{4} δ* _{lk}x*_{l}x*_{k},* and
Because the characteristic polynomial

*Q(z)*is a fourth-degree polynomial with a positive coefficient of

*z*

^{4}, when

*Q*(1) < 0, there is a positive eigenvalue larger than one, and the equilibrium

**x*** is externally unstable; that is, the new epistasis modifying allele

*M*

_{2}increases in frequency when it is rare. In view of Eq.

**8**, if det(

**M̃**) < 0, then

*M*

_{2}invades if its epistasis ε

_{12}is greater than the epistasis ε

_{11}produced by the resident modifier genotype

*M*

_{1}

*M*

_{1}.

In SI *Supporting Text 2*, we show that, if an arbitrary number of epistasis-controlling alleles *M*_{1}, *M*_{2}, …, *M _{n}* are present at a specific class of equilibria called viability analogous Hardy—Weinberg (VAHW) equilibria (24), then the condition for a new allele

*M*

_{n+1}to increase when rare is completely analogous to

*Q*(1) < 0 with an appropriate modification of

*Q*(1) to account for multiple modifying alleles at the

*M*locus.

We now address the conditions on the equilibrium **x*** and hence on the matrices **W** = [*w _{kl}*] and

**Δ**= [δ

*], that entail det(*

_{kl}**M̃**) < 0.

## Choosing W and Δ

We use the linear epistasis modification scheme given by Eq. **5**. Because we are interested in the evolution of epistasis, it is natural to assume that **W** involves no epistasis; that is, **W** = **W**_{add} is an additive fitness matrix that can be represented as
The interaction matrix **Δ** determines how each modifier genotype *M*_{i}*M*_{j} with associated epistasis measure ε* _{ij}* affects the fitness of each genotype at the major locus. Following Bodmer and Felsenstein (20), there are four additive row epistasis measures associated with the scheme (Eq.

**5**) given by for

*k*= 1, 2, 3, 4 and

*i, j*= 1, 2, …,

*n*. Because

**W**= [

*w*] is

_{kl}**W**

_{add}, its four additive row epistasis measures are zero, and in our linear scheme, we have for each row

*k*= 1, 2, 3, 4 A simple and natural assumption is that the interaction matrix

**Δ**= [δ

*] is such that for given*

_{kl}*i, j,*all four row epistasis measures

*e*

_{k}

^{ij}are the same in absolute value. This assumption gives rise to an interaction structure analogous to that in standard linear models for two-way analysis of variance. Liberman and Feldman (13) assume that

*e*

_{1}

^{ij}= −

*e*

_{2}

^{ij}= −

*e*

_{3}

^{ij}=

*e*

_{4}

^{ij}, and

**Δ**is chosen to be Another epistatic scheme takes

*e*

_{1}

^{ij}=

*e*

_{2}

^{ij}=

*e*

_{3}

^{ij}=

*e*

_{4}

^{ij}, which is the case for the two-parameter family of

**Δ**matrices: where In both cases, results can be obtained concerning the external stability of VAHW equilibria

**x*** ⊗

**y*** when the major locus equilibrium

**x*** is either in linkage equilibrium,

*D** =

*x*

_{1}*

*x*

_{4}* −

*x*

_{2}*

*x*

_{3}* = 0, or in linkage disequilibrium,

*D** ≠ 0. Analysis of the case

*D** ≠ 0 requires the additional assumption that

**W**

_{add}of Eq.

**11**is symmetric, namely α

_{1}= α

_{3}and β

_{1}= β

_{3}. With this additional assumption, for each ε

*, the fitness matrix*

_{ij}**W**

_{add}+ ε

_{ij}**Δ**, where

**Δ**is either

**Δ**

_{0}or

**Δ**(

*s, t*), takes the form of the two-locus symmetric viability matrix introduced by Bodmer and Felsenstein (20) and gives rise to a two-locus symmetric equilibrium

**x̂**= (

*x̂*

_{1},

*x̂*

_{2},

*x̂*

_{3},

*x̂*

_{4}) with where

*D̂*=

*x̂*

_{1}

*x̂*

_{4}−

*x̂*

_{2}

*x̂*

_{3}is a solution of a cubic equation. By assuming that

**x*** is in fact a symmetric equilibrium, the external stability analysis becomes tractable.

This leads us to ask in the next section how general is the choice of **Δ** that will enable analytical treatment to determine whether the evolutionary increase of epistasis shown in ref. 13 is a more general phenomenon.

## General Symmetric Modification Scheme

Consider the modified fitness scheme **W** = **W**_{add} + ε**Δ**, where **W**_{add} is given by Eq. **11**. We assume that the interaction matrix **Δ** is such that if **W**_{add} is symmetric (α_{1} = α_{3}, β_{1}= β_{3}), then for all ε, **W** = **W**_{add} + ε**Δ** is a symmetric viability matrix of the form (20)
Hence, the interaction matrix **Δ** takes the form
With this choice of **Δ**, the four row epistasis measures are
When **W**_{add} is symmetric (α_{1} = α_{3}, β_{1} = β_{3}), the modified fitness matrix **W**_{add} + ε**Δ** is a Lewontin–Kojima symmetric fitness matrix if *d* = *a*, and a Wright–Kimura symmetric fitness matrix if *d* = 2(*b* + *c*) − (*a* + 2*e*).

Moreover, when **W** is of the form (Eq. **18**), the two-locus model with recombination rate *r*, corresponding to the two major loci, gives rise to a symmetric equilibrium **x*** = (*x*_{1}*, *x*_{2}*, *x*_{3}*, *x*_{4}*) of the form (Eq. **17**), where *D** = *x*_{1}**x*_{4}* − *x*_{2}**x*_{3}*, the linkage disequilibrium, is a solution of the cubic equation (21)
with *l* = 2(β + γ) − (α + δ), *m* = δ − α.

Observe that when α_{1} = α_{3}, β_{1} = β_{3} in **W**_{add}, **W**_{add} + ε**Δ** is of the symmetric form (Eq. **18**) with
Thus,
and
Because the entries of **Δ** can be either positive or negative, we will assume, without loss of generality, that ε > 0.

From the cubic equation *G(D)* = 0, when *l* > 0, *G* (±∞) = ±∞, and so when *m* < 0, *G(D)* = 0 has a positive root, and a negative root when *m* > 0. In fact, for any **W** of the form (Eq. **18**), we have the following result when *l* ≥ 0.

### Result 1.

(*i*) *When m* < 0 *and l* ≥ 0, *the cubic equation has a unique positive solution* 0 < *D** < ¼ *for all* 0 < *r* ≤ ½, *with D** = ¼ *when r* = 0. *Moreover, D** = *D*(r) is a decreasing function of r*.

*(ii) When m > 0 and l ≥ 0, the cubic equation has a unique negative solution − ¼ < D* < 0 for all 0 < r ≤ ½ with D* = −¼ for r = 0. D* = D*(r) is an increasing function of r*.

#### Remark.

When *m* = 0 and *l* ≠ 0, we have the Lewontin–Kojima model, where the cubic equation has three roots, *D** = 0 and *r*.

The proof is in SI *Supporting Text 3* for *r* > 0 and in SI *Supporting Text 6* for *r* = 0. Thus, in the general symmetric modification scheme, when **W**_{add} is symmetric, and **Δ** takes the form (Eq. **19**), a unique symmetric equilibrium **x*** = (*x*_{1}*, *x*_{2}*, *x*_{3}*, *x*_{4}*) exists at the two major loci with any fitness matrix of the form **W** = **W**_{add} + ε**Δ**. This allows us to carry out the external stability analysis of the equilibrium **x*** of the three-locus system.

### Evolution of Epistasis Near *D* ≠ 0* Equilibria

Let the selection modification scheme take the form **W** = **W**_{add} + ε**Δ**, where **W** is a symmetric viability matrix. Let **x*** be a symmetric equilibrium of the two major loci with
Suppose that ε_{11} and ε_{12} are the epistatic parameters for *M*_{1}*M*_{1} and *M*_{1}*M*_{2}. We then have the following result, whose proof is given for a general VAHW equilibrium in SI *Supporting Text 4*.

### Result 2.

*Consider the symmetric equilibrium* **x*** = (*x*_{1}*, *x*_{2}*, *x*_{3}*, *x*_{4}*), *where M _{1} is fixed with epistatic parameter ε_{11}, and x_{1}* = x_{4}* = ¼ + D*, x_{2}* = x_{3}* = ¼ − D*, and D* ≠ 0. Then there exists an r_{crit}, such that this equilibrium is externally unstable to the introduction of a new modifier allele M_{2} with greater associated epistasis (ε_{12} > ε_{11}) provided 0 ≤ r ≤ r_{crit}, and a > 0, a + e > 0 in the interaction matrix*

**Δ**.

### Evolution of Epistasis near *D* = 0* Equilibria

Let the modification scheme be **W** = **W**_{add} + ε**Δ** where **W**_{add} is additive of the form (Eq. **12**) (not necessarily symmetric), and the interaction matrix **Δ** takes the symmetric form (Eq. **19**). If ε_{11} = 0, then with overdominance at each of the two major loci (α_{1}, α_{3} < α_{2}; β_{1}, β_{3} < β_{2}) for *r* > 0, there is a stable **x*** equilibrium with *D** = *x*_{1}**x*_{4}* − *x*_{2}**x*_{3}* = 0. If ε_{11} > 0, then only when *d = a* in **Δ**, namely when *m* = 0, can the symmetric equilibrium **x*** have *D** = 0, because *D** = 0 is then one of the roots of the cubic equation *G(D)* = 0 of Eq. **21**. From ref. 13, a new modifier allele *M*_{2} with associated epistasis ε_{12} > ε_{11} will invade if δ* = Σ_{k,l=1}^{4} δ_{kl}x*_{k}x*_{l} > 0 provided *R, r* > 0, and |ε_{12} − ε_{11}| is small. We have the following result, whose proof is given in SI *Supporting Text 5* for the more general VAHW case.

### Result 3.

*Let* **x*** *be an equilibrium with M*_{1} *fixed and epistatic parameter* ε_{11}, *and linkage equilibrium, D** = *x*_{1}**x*_{4}* − *x*_{2}**x*_{3}* = 0. *Under mutually exclusive conditions that are independent of the recombination rates, a new modifier allele M*_{2} *with associated epistasis value* ε_{12} *for M*_{1}*M*_{2} *that is greater than* ε_{11} *or less than* ε_{11} *may invade, inducing the evolution of increased or decreased epistasis, respectively*.

## Discussion

The models examined here represent a considerable generalization of that discussed by Liberman and Feldman (13). The interaction matrix **Δ** in Eq. **19** allows the analyses in the previous papers as special numerical cases that give the Lewontin–Kojima model or the Wright–Kimura model. When an epistasis-modifying allele is introduced near a stable state of linkage disequilibrium, the allele will succeed if it increases epistasis, provided the linkage of the major loci is tight enough. In the “supersymmetric” Lewontin–Kojima case, it turns out there is no restriction on this linkage: epistasis always increases. But the proof of *Result 2* given in SI *Supporting Text 4* specifies the conditions on *r* that make the linkage between the major loci tight enough that epistasis will increase.

Of course, when *r* is larger, these conditions do not hold, and the magnitude of epistasis will decrease. Importantly, these results do not depend on the sign of the linkage disequilibrium (as long as it is nonzero) or on the sign of the epistasis. The recombination *R* between the modifier and major loci also plays no qualitative role in these results. Similar findings applied for the success of recombination-reducing alleles near stable linkage disequilibrium, the so-called reduction principle of Feldman and Liberman (24).

Introduction of epistasis modifiers near states where *D** = 0 may be regarded as a more special problem, because the classes of symmetric fitness models that allow *D** = 0 are restricted. The conditions contained in SI *Supporting Text 5*, the proof of *Result 3*, allow epistasis to either increase or decrease near *D* = 0, provided the major locus allele frequencies are not at 0.5 (see also ref. 13).

The maintenance of linkage disequilibrium in large populations requires some level of epistasis, and the effect of this epistasis in preserving associations between genes is stronger the closer the linkage is between the genes. It is not surprising, therefore, that tight linkage facilitates the strengthening of epistasis when there is linkage is disequilibrium. In finite populations, however, mutation and random genetic drift can generate linkage disequilibrium in the absence of selection. It remains to be seen whether new alleles that introduce epistatic selection into such a neutral situation will have a greater chance of succeeding than fitness modifiers with no epistatic effects. In the same way, if one gene is maintained polymorphic by selection, and a new allele at another locus appears at low frequency and starts to increase by virtue of its interactions with the first locus, we can ask whether epistasis between the two loci will increase during transient evolution. This is a different question from the one studied in this paper, which assumes an equilibrium with linkage disequilibrium, and has yet to be addressed with formal analysis (see, e.g., ref. 25, pp. 155–156.)

There is ample evidence that epistatic selection is involved in adaptive variation that is often attributed to single-locus effects (26). On the other hand, there is little evidence that this epistasis is itself under control of other genes and might therefore be subject to evolutionary change. The detection of such effects might be facilitated by careful examination of genetic networks in which some nodes are known to exhibit signatures of selection.

Templeton (26) stresses the importance of epistasis in both evolutionary and quantitative genetic contexts: “Epistasis appears to be a nearly universal component of the genetic architecture of most common traits.” He also points out that epistasis is itself subject to change: “…epistasis itself can appear and disappear if the interaction between two loci itself interacts with additional genetic or environmental factors….” Our model addresses this analytically through the use of an epistasis-modifying gene, which certainly interacts with the two major loci.

There is expected to be a relationship between conditions on the selection regime that would favor an increase of recombination and those that would produce negative epistasis. Simulation studies (14) have suggested that sex (i.e., recombination) would lead to the evolution of negative epistasis in a situation of mutation to deleterious alleles. These results depended, however, on the rate of mutation; infrequent mutation seemed to maintain a low level of positive epistasis.

Our models have attempted to separate main single locus effects from interaction effects due to the epistasis modifier. In the case of multiple deleterious mutations, numerical and experimental studies (14, 17, 27, 28) have shown that evolution of epistasis is correlated with the strength of the fitness loss from single mutations. In fact, deleterious mutations of small effect may evolve epistasis of different signs depending on resource availability in the environment (27). This raises the interesting question of whether the evolution of epistasis determines the nature of environmental canalization, or the evolution of robustness determines the direction of epistasis.

Although our investigation has focused on only three loci, the number of parameters involved is much larger than seen in most studies on deleterious mutations and epistasis. The latter, as mentioned above, invoke only two parameters: the fitness loss per mutation and a parameter that determines the overall functional form of the epistasis. We require the fitness structure of the full two-locus system; thus, it is surprising that the results depend only on the change in epistasis and the rate of recombination. Of course, as with studies of deleterious mutations, linkage disequilibrium must be present and maintained during the evolution of epistasis at a geometric rate.

Our final point concerns the distinction we have drawn between the modifier gene *M* and the major loci *A/a* and *B/b*. Gene *M* is not “neutral” in the sense that a recombination or mutation modifier might be regarded. It is, in fact, part of the selection system in the same sense that a dominance modifier is. As a result, changes in the genetic background, including new alleles at an epistasis modifier, may alter the way in which selection affects an allele because of that allele's epistatic interactions. Under some circumstances, as we have shown, this alteration of the genetic architecture of fitness may depend on the extent of linkage between the components, not just on the pattern of epistatic fitness interactions.

## Acknowledgments

We thank three anonymous reviewers for helpful suggestions and Prof. S. Karlin for valuable discussions. This work was supported in part by National Institues of Health Grant GM 28016.

## Footnotes

- ↵
^{§}To whom correspondence should be addressed. E-mail: marc{at}charles.stanford.edu

Author contributions: U.L and M.W.F. contributed equally to this work; and U.L. and M.W.F. wrote the paper.

The authors declare no conflict of interest.

This article contains supporting information (SI) online at www.pnas.org/cgi/content/full/0608569103/DC1.

- Abbreviation
- VAHW,
- viability analogous Hardy–Weinberg.

- Received May 15, 2006.

- © 2006 by The National Academy of Sciences of the USA

## References

- ↵
- ↵
- ↵.
- Carter AJR,
- Hermisson J,
- Hansen TF

- ↵.
- Altenberg L

- ↵
- ↵.
- Phillips PC,
- Otto SC,
- Whitlock MC

- ↵.
- Wright S

- ↵
- ↵
- ↵
- ↵
- ↵.
- Rice SH

- ↵
- ↵
- ↵.
- Siegel ML,
- Bergman A

- ↵
- ↵
- ↵
- ↵
- ↵.
- Bodmer WF,
- Felsenstein J

- ↵
- ↵.
- Haldane JBS

- ↵
- ↵.
- Feldman M,
- Liberman U

- ↵.
- Kelly JK

- ↵.
- Templeton AR

- ↵.
- Burch CL,
- Chao L

- ↵.
- You L,
- Yin J

## Citation Manager Formats

### More Articles of This Classification

### Biological Sciences

### Related Content

- No related articles found.