Skip to main content

Main menu

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
  • About
    • Editorial Board
    • PNAS Staff
    • FAQ
    • Accessibility Statement
    • Rights and Permissions
    • Site Map
  • Contact
  • Journal Club
  • Subscribe
    • Subscription Rates
    • Subscriptions FAQ
    • Open Access
    • Recommend PNAS to Your Librarian

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Home
Home
  • Log in
  • My Cart

Advanced Search

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
Research Article

Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms

View ORCID ProfileP. Jonathon Phillips, Amy N. Yates, Ying Hu, Carina A. Hahn, Eilidh Noyes, Kelsey Jackson, Jacqueline G. Cavazos, Géraldine Jeckeln, Rajeev Ranjan, Swami Sankaranarayanan, Jun-Cheng Chen, Carlos D. Castillo, Rama Chellappa, David White, and Alice J. O’Toole
  1. aInformation Access Division, National Institute of Standards and Technology, Gaithersburg, MD 20899;
  2. bSchool of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX 75080;
  3. cDepartment of Electrical and Computer Engineering, University of Maryland Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20854;
  4. dUniversity of Maryland Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20854;
  5. eSchool of Psychology, The University of New South Wales, Sydney, NSW 2052, Australia

See allHide authors and affiliations

PNAS June 12, 2018 115 (24) 6171-6176; first published May 29, 2018; https://doi.org/10.1073/pnas.1721355115
P. Jonathon Phillips
aInformation Access Division, National Institute of Standards and Technology, Gaithersburg, MD 20899;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for P. Jonathon Phillips
  • For correspondence: jonathon@nist.gov
Amy N. Yates
aInformation Access Division, National Institute of Standards and Technology, Gaithersburg, MD 20899;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ying Hu
bSchool of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX 75080;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carina A. Hahn
bSchool of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX 75080;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Eilidh Noyes
bSchool of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX 75080;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kelsey Jackson
bSchool of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX 75080;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jacqueline G. Cavazos
bSchool of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX 75080;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Géraldine Jeckeln
bSchool of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX 75080;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rajeev Ranjan
cDepartment of Electrical and Computer Engineering, University of Maryland Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20854;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Swami Sankaranarayanan
cDepartment of Electrical and Computer Engineering, University of Maryland Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20854;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jun-Cheng Chen
dUniversity of Maryland Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20854;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carlos D. Castillo
dUniversity of Maryland Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20854;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rama Chellappa
cDepartment of Electrical and Computer Engineering, University of Maryland Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20854;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David White
eSchool of Psychology, The University of New South Wales, Sydney, NSW 2052, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alice J. O’Toole
bSchool of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX 75080;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  1. Edited by Thomas D. Albright, The Salk Institute for Biological Studies, La Jolla, CA, and approved April 30, 2018 (received for review December 13, 2017)

  • Article
  • Figures & SI
  • Info & Metrics
  • PDF
Loading

Significance

This study measures face identification accuracy for an international group of professional forensic facial examiners working under circumstances that apply in real world casework. Examiners and other human face “specialists,” including forensically trained facial reviewers and untrained superrecognizers, were more accurate than the control groups on a challenging test of face identification. Therefore, specialists are the best available human solution to the problem of face identification. We present data comparing state-of-the-art face recognition technology with the best human face identifiers. The best machine performed in the range of the best humans: professional facial examiners. However, optimal face identification was achieved only when humans and machines worked in collaboration.

Abstract

Achieving the upper limits of face identification accuracy in forensic applications can minimize errors that have profound social and personal consequences. Although forensic examiners identify faces in these applications, systematic tests of their accuracy are rare. How can we achieve the most accurate face identification: using people and/or machines working alone or in collaboration? In a comprehensive comparison of face identification by humans and computers, we found that forensic facial examiners, facial reviewers, and superrecognizers were more accurate than fingerprint examiners and students on a challenging face identification test. Individual performance on the test varied widely. On the same test, four deep convolutional neural networks (DCNNs), developed between 2015 and 2017, identified faces within the range of human accuracy. Accuracy of the algorithms increased steadily over time, with the most recent DCNN scoring above the median of the forensic facial examiners. Using crowd-sourcing methods, we fused the judgments of multiple forensic facial examiners by averaging their rating-based identity judgments. Accuracy was substantially better for fused judgments than for individuals working alone. Fusion also served to stabilize performance, boosting the scores of lower-performing individuals and decreasing variability. Single forensic facial examiners fused with the best algorithm were more accurate than the combination of two examiners. Therefore, collaboration among humans and between humans and machines offers tangible benefits to face identification accuracy in important applications. These results offer an evidence-based roadmap for achieving the most accurate face identification possible.

  • face identification
  • forensic science
  • face recognition algorithm
  • wisdom-of-crowds
  • machine learning technology

Footnotes

  • ↵1To whom correspondence should be addressed. Email: jonathon{at}nist.gov.
  • Author contributions: P.J.P., A.N.Y., D.W., and A.J.O. designed research; R.R., S.S., J.-C.C., C.D.C., and R.C. contributed new reagents/analytic tools; P.J.P., A.N.Y., Y.H., C.A.H., E.N., K.J., J.G.C., G.J., and A.J.O. analyzed data; R.R., S.S., J.-C.C., C.D.C., and R.C. implemented and ran the face recognition algorithms; and P.J.P. and A.J.O. wrote the paper.

  • Conflict of interest statement: The University of Maryland is filing a US patent application that will cover portions of algorithms A2017a and A2017b. R.R., C.D.C., and R.C. are coinventors on this patent.

  • This article is a PNAS Direct Submission.

  • This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1721355115/-/DCSupplemental.

  • Copyright © 2018 the Author(s). Published by PNAS.

This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND).

View Full Text
PreviousNext
Back to top
Article Alerts
Email Article

Thank you for your interest in spreading the word on PNAS.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms
(Your Name) has sent you a message from PNAS
(Your Name) thought you would like to see the PNAS web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms
P. Jonathon Phillips, Amy N. Yates, Ying Hu, Carina A. Hahn, Eilidh Noyes, Kelsey Jackson, Jacqueline G. Cavazos, Géraldine Jeckeln, Rajeev Ranjan, Swami Sankaranarayanan, Jun-Cheng Chen, Carlos D. Castillo, Rama Chellappa, David White, Alice J. O’Toole
Proceedings of the National Academy of Sciences Jun 2018, 115 (24) 6171-6176; DOI: 10.1073/pnas.1721355115

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Request Permissions
Share
Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms
P. Jonathon Phillips, Amy N. Yates, Ying Hu, Carina A. Hahn, Eilidh Noyes, Kelsey Jackson, Jacqueline G. Cavazos, Géraldine Jeckeln, Rajeev Ranjan, Swami Sankaranarayanan, Jun-Cheng Chen, Carlos D. Castillo, Rama Chellappa, David White, Alice J. O’Toole
Proceedings of the National Academy of Sciences Jun 2018, 115 (24) 6171-6176; DOI: 10.1073/pnas.1721355115
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Mendeley logo Mendeley

Article Classifications

  • Social Sciences
  • Psychological and Cognitive Sciences
Proceedings of the National Academy of Sciences: 115 (24)
Table of Contents

Submit

Sign up for Article Alerts

Jump to section

  • Article
    • Abstract
    • Results
    • Error Rates for Highly Confident Decisions
    • Discussion
    • Materials and Methods
    • Acknowledgments
    • Footnotes
    • References
  • Figures & SI
  • Info & Metrics
  • PDF

You May Also be Interested in

Surgeons hands during surgery
Inner Workings: Advances in infectious disease treatment promise to expand the pool of donor organs
Despite myriad challenges, clinicians see room for progress.
Image credit: Shutterstock/David Tadevosian.
Setting sun over a sun-baked dirt landscape
Core Concept: Popular integrated assessment climate policy models have key caveats
Better explicating the strengths and shortcomings of these models will help refine projections and improve transparency in the years ahead.
Image credit: Witsawat.S.
Double helix
Journal Club: Noncoding DNA shown to underlie function, cause limb malformations
Using CRISPR, researchers showed that a region some used to label “junk DNA” has a major role in a rare genetic disorder.
Image credit: Nathan Devery.
Steamboat Geyser eruption.
Eruption of Steamboat Geyser
Mara Reed and Michael Manga explore why Yellowstone's Steamboat Geyser resumed erupting in 2018.
Listen
Past PodcastsSubscribe
Birds nestling on tree branches
Parent–offspring conflict in songbird fledging
Some songbird parents might improve their own fitness by manipulating their offspring into leaving the nest early, at the cost of fledgling survival, a study finds.
Image credit: Gil Eckrich (photographer).

Similar Articles

Site Logo
Powered by HighWire
  • Submit Manuscript
  • Twitter
  • Facebook
  • RSS Feeds
  • Email Alerts

Articles

  • Current Issue
  • Special Feature Articles – Most Recent
  • List of Issues

PNAS Portals

  • Anthropology
  • Chemistry
  • Classics
  • Front Matter
  • Physics
  • Sustainability Science
  • Teaching Resources

Information

  • Authors
  • Editorial Board
  • Reviewers
  • Librarians
  • Press
  • Site Map
  • PNAS Updates

Feedback    Privacy/Legal

Copyright © 2021 National Academy of Sciences. Online ISSN 1091-6490