Effect of ancient population structure on the degree of polymorphism shared between modern human populations and ancient hominins
See allHide authors and affiliations
Edited by Francisco Mauro Salzano, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil, and approved July 20, 2012 (received for review January 19, 2012)

Abstract
Recent comparisons between anatomically modern humans and ancient genomes of other hominins have raised the tantalizing, and hotly debated, possibility of hybridization. Although several tests of hybridization have been devised, they all rely on the degree to which different modern populations share genetic polymorphisms with the ancient genomes of other hominins. However, spatial population structure is expected to generate genetic patterns similar to those that might be attributed to hybridization. To investigate this problem, we take Neanderthals as a case study, and build a spatially explicit model of the shared history of anatomically modern humans and this hominin. We show that the excess polymorphism shared between Eurasians and Neanderthals is compatible with scenarios in which no hybridization occurred, and is strongly linked to the strength of population structure in ancient populations. Thus, we recommend caution in inferring admixture from geographic patterns of shared polymorphisms, and argue that future attempts to investigate ancient hybridization between humans and other hominins should explicitly account for population structure.
Footnotes
- ↵1To whom correspondence may be addressed. E-mail: am315{at}cam.ac.uk or aej44{at}cam.ac.uk.
Author contributions: A.E. and A.M. designed research; A.E. and A.M. performed research; A.E. contributed new reagents/analytic tools; A.E. and A.M. analyzed data; and A.E. and A.M. wrote the paper.
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1200567109/-/DCSupplemental.