# Reply to Berger et al.: Improving ABC

- Department of Biology, Washington University, St. Louis, MO 63130-4899

See allHide authors and affiliations

Berger et al. (1) claim that I (2) made “a broad attack on the foundations of Bayesian statistical methods.” Rather, I made focused criticisms on approximate Bayesian computation (ABC) as implemented in Fagundes et al. (3), who produced incoherent results. This incoherence arose in part because they used an equation that violates the core foundation of Bayesian analysis: conditional probability. Three models were tested—say A, B, and C—by calculating the conditional probability that a given model, say A, is true given that either A or B or C is true (3). Mathematically, this is:

where ∪ is the union and ∩ is the intersection operator. If the models A, B, and C are mutually exclusive, the denominator becomes *P*(*A*) + *P*(*B*) + *P*(*C*). A fundamental error is that ABC treats *all* models as being mutually exclusive regardless of their actual relationship. All of the models in ref. 3 overlap logically, so ABC yields incorrect probabilities. Berger et al. (1), and also Csilléry et al. (4), show that logical overlap can be eliminated in the special case of nesting due to a single parameter by redefining the models to ensure exclusivity and by using a prior with a point mass concentrated on the value that defines the special case. This was not done in ref. 3, so refs. 1 and 4 reveal yet another deviation from Bayesian practice in this incoherent analysis.

The solution in refs. 1 and 4 also does not work for the models in ref. 3 because these models differ by multiple parameters and have logical, nonnested overlap in other components. Consequently, even if the nesting problem associated with the admixture parameter, *M*, were completely eliminated, all of the intersection terms in the equation above would still be nonzero. As a result, all of the posterior probabilities are wrong even if the solution in refs. 1 and 4 had been implemented for *M*. Logical overlap is the norm for the complex models analyzed with ABC (2), so many ABC posterior model probabilities published to date are wrong.

The seriousness of this error is illustrated by an internal discrepancy between ABC model probabilities and estimators (3). Nonzero admixture was rejected with a probability of 0.001 by Fagundes et al. (3) using their incoherent equation. In contradiction, the ABC 95% Bayesian credible interval of the estimate of *M* does not include zero (3). Using a Bayesian procedure (5), I constructed a coherent ABC test (2) that rejects *M* = 0 with a probability <0.025. Note that the relative probabilities of the models are reversed, so this represents a discrepancy of five orders of magnitude between two Bayesian procedures using the *same* ABC analysis. This egregious discrepancy cannot be solved by the proposal of refs. 1 and 4.

Berger et al. (1) point out that Bayes factors can be coherent, and my coherent test shows that the same is true for ABC. My coherent ABC conclusion is consistent with prior statistical analyses (2) and with the recent Neanderthal DNA analyses (6), unlike the incoherent ABC analysis (3). Thus, my work strengthens ABC by correcting procedures that violate core Bayesian principles and yield patently wrong biological conclusions.

## Footnotes

^{1}E-mail: temple_a{at}wustl.edu.Author contributions: A.R.T. wrote the paper.

The author declares no conflict of interest.

## References

- ↵
- Berger JO,
- Fienberg SE,
- Raftery AE,
- Robert CP

- ↵
- Templeton AR

- ↵
- Fagundes NJR,
- et al.

- ↵
- ↵
- Lindley DV

- ↵
- Green RE,
- et al.