Automated vocal analysis of naturalistic recordings from children with autism, language delay, and typical development

Edited by E. Anne Cutler, Max Planck Institute for Psycholinguistics, Heilig Landstichting, Netherlands, and approved June 21, 2010 (received for review April 21, 2010)
July 19, 2010
107 (30) 13354-13359


For generations the study of vocal development and its role in language has been conducted laboriously, with human transcribers and analysts coding and taking measurements from small recorded samples. Our research illustrates a method to obtain measures of early speech development through automated analysis of massive quantities of day-long audio recordings collected naturalistically in children's homes. A primary goal is to provide insights into the development of infant control over infrastructural characteristics of speech through large-scale statistical analysis of strategically selected acoustic parameters. In pursuit of this goal we have discovered that the first automated approach we implemented is not only able to track children's development on acoustic parameters known to play key roles in speech, but also is able to differentiate vocalizations from typically developing children and children with autism or language delay. The method is totally automated, with no human intervention, allowing efficient sampling and analysis at unprecedented scales. The work shows the potential to fundamentally enhance research in vocal development and to add a fully objective measure to the battery used to detect speech-related disorders in early childhood. Thus, automated analysis should soon be able to contribute to screening and diagnosis procedures for early disorders, and more generally, the findings suggest fundamental methods for the study of language in natural environments.

Continue Reading


Research by D.K.O. for this paper was funded by an endowment from the Plough Foundation, which supports his Chair of Excellence at The University of Memphis.

Supporting Information

Appendix (PDF)
Supporting Information


JG de Villiers, PA de Villiers, Competence and performance in child language: Are children really competent to judge? J Child Lang 1, 11–22 (1974).
D Slobin, Cognitive prerequisites for the development of grammar. Studies in Child Language Development, eds CA Ferguson, DI Slobin (Holt, Rinehart & Winston, New York), pp. 175–208 (1973).
L Bloom Language Development (MIT Press, Cambridge, MA, 1970).
R Brown A First Language (Academic Press, London, 1973).
S Pinker The Language Instinct (Harper Perennial, New York, 1994).
A Cutler, W Klein, SC Levinson, The cornerstones of twenty-first century psycholinguistics. Twenty-First Century Psycholinguistics: Four Cornerstones, ed A Cutler (Erlbaum, Mahwah, NJ), pp. 1–20 (2005).
JL Locke Phonological Acquisition and Change (Academic Press, New York, 1983).
DK Oller, The emergence of the sounds of speech in infancy. Child Phonology, Vol 1: Production, eds G Yeni-Komshian, J Kavanagh, C Ferguson (Academic Press, New York), pp. 93–112 (1980).
RE Stark, SN Rose, M McLagen, Features of infant sounds: The first eight weeks of life. J Child Lang 2, 205–221 (1975).
SJ Sheinkopf, P Mundy, DK Oller, M Steffens, Vocal atypicalities of preverbal autistic children. J Autism Dev Disord 30, 345–354 (2000).
AM Wetherby, et al., Early indicators of autism spectrum disorders in the second year of life. J Autism Dev Disord 34, 473–493 (2004).
DK Oller, RE Eilers, The role of audition in infant babbling. Child Dev 59, 441–449 (1988).
RE Eilers, DK Oller, Infant vocalizations and the early diagnosis of severe hearing impairment. J Pediatr 124, 199–203 (1994).
N Masataka, Why early linguistic milestones are delayed in children with Williams syndrome: Late onset of hand banging as a possible rate-limiting constraint on the emergence of canonical babbling. Dev Sci 4, 158–164 (2001).
DK Oller, U Griebel, The origins of syllabification in human infancy and in human evolution. Syllable Development: The Frame/Content Theory and Beyond, eds B Davis, K Zajdo (Lawrence Erlbaum and Associates, Mahwah, NJ), pp. 368–386 (2008).
MM Vihman Phonological Development: The Origins of Language in the Child (Blackwell Publishers, Cambridge, MA, 1996).
RE Stark, BM Ansel, J Bond, Are prelinguistic abilities predictive of learning disability? A follow-up study. Preschool Prevention of Reading Failure, eds RL Masland, M Masland (York Press, Parkton, MD, 1988).
C Stoel-Gammon, Prespeech and early speech development of two late talkers. First Lang 9, 207–223 (1989).
DK Oller The Emergence of the Speech Capacity (Lawrence Erlbaum Associates, Mahwah, NJ, 2000).
L Kanner, Autistic disturbances of affective contact. Nerv Child 2, 217–250 (1943).
H Asperger, Autistic “psychopathy” in childhood. Autism and Asperger Syndrome, ed U Frith (Cambridge University Press, Cambridge, UK), pp. 37–90 (1991).
R Paul, A Augustyn, A Klin, FR Volkmar, Perception and production of prosody by speakers with autism spectrum disorders. J Autism Dev Disord 35, 205–220 (2005).
W Pronovost, MP Wakstein, DJ Wakstein, A longitudinal study of the speech behavior and language comprehension of fourteen children diagnosed atypical or autistic. Except Child 33, 19–26 (1966).
J McCann, S Peppé, Prosody in autism spectrum disorders: a critical review. Int J Lang Commun Disord 38, 325–350 (2003).
S Peppé, J McCann, F Gibbon, A O'Hare, M Rutherford, Receptive and expressive prosodic ability in children with high-functioning autism. J Speech Lang Hear Res 50, 1015–1028 (2007).
LD Shriberg, et al., Speech and prosody characteristics of adolescents and adults with high-functioning autism and Asperger syndrome. J Speech Lang Hear Res 44, 1097–1115 (2001).
AP Association Diagnostic and Statistical Manual of Mental Disorders, DSM-IV-TR (American Psychiatric Association, Arlington, VA, 2000).
S Baron-Cohen, J Allen, C Gillberg, Can autism be detected at 18 months? The needle, the haystack, and the CHAT. Br J Psychiatry 161, 839–843 (1992).
S Baron-Cohen, Theory of mind and autism: a fifteen year review. Understanding Other Minds: Perspectives from Developmental Cognitive Neuroscience, eds S Baron-Cohen, H Tager-Flusberg, DJ Cohen (Oxford University Press, Oxford), pp. 3–20 (2000).
KA Loveland, SH Landry, Joint attention and language in autism and developmental language delay. J Autism Dev Disord 16, 335–349 (1986).
P Mundy, C Kasari, M Sigman, Nonverbal communication, affective sharing, and intersubjectivity. Infant Behav Dev 15, 377–381 (1992).
P Mundy, M Sigman, C Kasari, A longitudinal study of joint attention and language development in autistic children. J Autism Dev Disord 20, 115–128 (1990).
M Rutter, Diagnosis and definition of childhood autism. J Autism Child Schizophr 8, 139–161 (1978).
H Tager-Flusberg, A psycholinguistic perspective on language development in the autistic child. Autism: Nature, Diagnosis, and Treatment, ed G Dawson (Guilford, New York), pp. 92–115 (1989).
H Tager-Flusberg, Language and understanding minds: connections in autism. Understanding Other Minds: Perspectives from Developmental Cognitive Neuroscience, eds S Baron-Cohen, H Tager-Flusberg, DJ Cohen (Oxford University Press, Oxford), pp. 124–149 (2000).
AA Tyler, KT Sandoval, Preschoolers With Phonological and Language Disorders: Treating Different Linguistic Domains. Lang Speech Hear Serv Schools 25, 215–234 (1994).
G Conti-Ramsden, N Botting, Classification of children with specific language impairment: Longitudinal considerations. J Speech Lang Hear Res 42, 1195–1204 (1999).
LD Shriberg, DM Aram, J Kwiatkowski, Developmental apraxia of speech: I. Descriptive and theoretical perspectives. J Speech Lang Hear Res 40, 273–285 (1997).
CR Marshall, S Harcourt-Brown, F Ramus, HK van der Lely, The link between prosody and language skills in children with specific language impairment (SLI) and/or dyslexia. Int J Lang Commun Disord 44, 466–488 (2009).
H Ireton, FP Glascoe, Assessing children's development using parents’ reports. The Child Development Inventory. Clin Pediatr (Phila) 34, 248–255 (1995).
C Lord, M Rutter, PC DiLavore, S Risi Autism Diagnostic Observation Schedule (Western Psychological Services, Los Angeles, 2002).
E Schopler, RJ Reichler, RF DeVellis, K Daly, Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS). J Autism Dev Disord 10, 91–103 (1980).
TM Achenbach Integrative Guide to the 1991 CBCL/4-18, YSR, and TRF Profiles (University of Vermont, Burlington, VT, 1991).
DL Robins, D Fein, ML Barton, JA Green, The Modified Checklist for Autism in Toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. J Autism Dev Disord 31, 131–144 (2001).

Information & Authors


Published in

Go to Proceedings of the National Academy of Sciences
Go to Proceedings of the National Academy of Sciences
Proceedings of the National Academy of Sciences
Vol. 107 | No. 30
July 27, 2010
PubMed: 20643944


Submission history

Published online: July 19, 2010
Published in issue: July 27, 2010


  1. vocal development
  2. automated identification of language disorders
  3. all-day recording
  4. automated speaker labeling
  5. autism identification


Research by D.K.O. for this paper was funded by an endowment from the Plough Foundation, which supports his Chair of Excellence at The University of Memphis.


*This Direct Submission article had a prearranged editor.



School of Audiology and Speech-Language Pathology, University of Memphis, Memphis, TN 38105;
Konrad Lorenz Institute for Evolution and Cognition Research, Altenberg, Austria A-3422;
P. Niyogi
Departments of Computer Science and Statistics, University of Chicago, Chicago, IL 60637;
S. Gray
LENA Foundation, Boulder, CO 80301; and
J. A. Richards
LENA Foundation, Boulder, CO 80301; and
J. Gilkerson
LENA Foundation, Boulder, CO 80301; and
D. Xu
LENA Foundation, Boulder, CO 80301; and
U. Yapanel
LENA Foundation, Boulder, CO 80301; and
S. F. Warren
Department of Applied Behavioral Science and Institute for Life Span Studies, University of Kansas, Lawrence, KS 66045


To whom correspondence should be addressed. E-mail: [email protected].
Author contributions: D.K.O., P.N., S.G., J.A.R., J.G., D.X., and S.F.W. designed research; S.G., J.A.R., J.G., D.X., and U.Y. performed research; P.N., S.G., J.A.R., and D.X. contributed new reagents/analytic tools; D.K.O., P.N., S.G., J.A.R., and D.X. analyzed data; and D.K.O., P.N., and S.F.W. wrote the paper.

Competing Interests

Conflict of interest statement: The recordings and hardware/software development were funded by Terrance and Judi Paul, owners of the previous for-profit company Infoture. Dissolution of the company was announced February 10, 2009, and it was reconstituted as the not-for-profit LENA Foundation. All assets of Infoture were given to the LENA Foundation. Before dissolution of the company, D.K.O., P.N., and S.F.W. had received consultation fees for their roles on the Scientific Advisory Board of Infoture. J.A.R., J.G., and D.X. are current employees of the LENA Foundation. S.G. and U.Y. are affiliates and previous employees of Infoture/LENA Foundation. None of the authors has or has had any ownership in Infoture or the LENA Foundation.

Metrics & Citations


Note: The article usage is presented with a three- to four-day delay and will update daily once available. Due to ths delay, usage data will not appear immediately following publication. Citation information is sourced from Crossref Cited-by service.

Citation statements



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited by


    View Options

    View options

    PDF format

    Download this article as a PDF file


    Get Access

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Personal login Institutional Login

    Recommend to a librarian

    Recommend PNAS to a Librarian

    Purchase options

    Purchase this article to get full access to it.

    Single Article Purchase

    Automated vocal analysis of naturalistic recordings from children with autism, language delay, and typical development
    Proceedings of the National Academy of Sciences
    • Vol. 107
    • No. 30
    • pp. 13191-13556







    Share article link

    Share on social media