Skip to main content

Main menu

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
    • Front Matter Portal
    • Journal Club
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
  • Submit
  • About
    • Editorial Board
    • PNAS Staff
    • FAQ
    • Accessibility Statement
    • Rights and Permissions
    • Site Map
  • Contact
  • Journal Club
  • Subscribe
    • Subscription Rates
    • Subscriptions FAQ
    • Open Access
    • Recommend PNAS to Your Librarian

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Home
Home
  • Log in
  • My Cart

Advanced Search

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
    • Front Matter Portal
    • Journal Club
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
Research Article

Replicator degrees of freedom allow publication of misleading failures to replicate

View ORCID ProfileChristopher J. Bryan, View ORCID ProfileDavid S. Yeager, and Joseph M. O’Brien
  1. aBooth School of Business, University of Chicago, Chicago, IL 60637;
  2. bDepartment of Psychology, University of Texas at Austin, Austin, TX 78712

See allHide authors and affiliations

PNAS December 17, 2019 116 (51) 25535-25545; first published November 25, 2019; https://doi.org/10.1073/pnas.1910951116
Christopher J. Bryan
aBooth School of Business, University of Chicago, Chicago, IL 60637;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Christopher J. Bryan
  • For correspondence: christopher.bryan@chicagobooth.edu
David S. Yeager
bDepartment of Psychology, University of Texas at Austin, Austin, TX 78712
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for David S. Yeager
Joseph M. O’Brien
bDepartment of Psychology, University of Texas at Austin, Austin, TX 78712
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  1. Edited by Susan T. Fiske, Princeton University, Princeton, NJ, and approved October 22, 2019 (received for review June 28, 2019)

  • Article
  • Figures & SI
  • Info & Metrics
  • PDF
Loading

Online Impact

 

Article Information

vol. 116 no. 51 25535-25545
DOI: 
https://doi.org/10.1073/pnas.1910951116
PubMed: 
31767750

Published By: 
National Academy of Sciences
Print ISSN: 
0027-8424
Online ISSN: 
1091-6490
History: 
  • Published in issue December 17, 2019.
  • Published first November 25, 2019.

Article Versions

  • Previous version (November 25, 2019 - 12:06).
  • You are viewing the most recent version of this article.
Copyright & Usage: 
Copyright © 2019 the Author(s). Published by PNAS. This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND).

Author Information

  1. Christopher J. Bryana,1,
  2. David S. Yeagerb, and
  3. Joseph M. O’Brienb
  1. aBooth School of Business, University of Chicago, Chicago, IL 60637;
  2. bDepartment of Psychology, University of Texas at Austin, Austin, TX 78712
  1. Edited by Susan T. Fiske, Princeton University, Princeton, NJ, and approved October 22, 2019 (received for review June 28, 2019)

Footnotes

  • ↵1To whom correspondence may be addressed. Email: christopher.bryan{at}chicagobooth.edu.
  • Author contributions: C.J.B., D.S.Y., and J.M.O. designed research; C.J.B., D.S.Y., and J.M.O. analyzed data; and C.J.B. and D.S.Y. wrote the paper.

  • The authors declare no competing interest.

  • This article is a PNAS Direct Submission.

  • Data deposition: All data and analysis scripts are posted on the Open Science Framework at https://osf.io/y5wsb/. An R package for conducting specification-curve analysis was developed for this work and is available for download at https://github.com/jmobrien/SpecCurve.

  • This article contains supporting information online at https://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1910951116/-/DCSupplemental.

Cited By...

  • 12 Citations
  • 10 Citations
  • Google Scholar

This article has been cited by the following articles in journals that are participating in Crossref Cited-by Linking.

  • Machine Learning in Psychometrics and Psychological Research
    Graziella Orrù, Merylin Monaro, Ciro Conversano, Angelo Gemignani, Giuseppe Sartori
    Frontiers in Psychology 2020 10
  • Science is not a signal detection problem
    Brent M. Wilson, Christine R. Harris, John T. Wixted
    Proceedings of the National Academy of Sciences 2020 117 11
  • An Enumerative Review and a Meta‐Analysis of Primed Goal Effects on Organizational Behavior
    Xiao Chen, Gary P. Latham, Ronald F. Piccolo, Guy Itzchakov
    Applied Psychology 2021 70 1
  • Evolutionary psychological consumer research: Bold, bright, but better with behavior
    Tobias Otterbring, Jill Sundie, Yexin Jessica Li, Sarah Hill
    Journal of Business Research 2020 120
  • Best practices in replication: a case study of common information in coordination games
    Roy Chen, Yan Chen, Yohanes E. Riyanto
    Experimental Economics 2021 24 1
  • Science as collaborative knowledge generation
    Naomi Ellemers
    British Journal of Social Psychology 2021 60 1
  • Infection threat shapes our social instincts
    Peter Kramer, Paola Bressan
    Behavioral Ecology and Sociobiology 2021 75 3
  • Is the Welfare State Relevant for Economic Growth? Evidence for Portugal
    Pedro Bação, Marta Simões
    Comparative Economic Studies 2020 62 3
  • Les pratiques de recherche ouvertes en psychologie
    A. Lantian
    Psychologie Française 2021 66 1
  • Narcissism and Leadership in Children
    Eddie Brummelman, Barbara Nevicka, Joseph M. O’Brien
    Psychological Science 2021 32 3

Article usage

Article usage: November 2019 to April 2021

AbstractFullPdf
Nov 201917561468434
Dec 201913431449415
Total 201930992917849
Jan 2020144417156
Feb 202027621109
Mar 20202117486
Apr 20203123989
May 20202224072
Jun 20203122766
Jul 20208722773
Aug 20203822275
Sep 20202336985
Oct 20204221362
Nov 20205218573
Dec 20208313770
Total 202060132711016
Jan 20212012257
Feb 20211010735
Mar 20212521645
Apr 2021119834
Total 202166543171
Total376667312036
PreviousNext
Back to top
Article Alerts
Email Article

Thank you for your interest in spreading the word on PNAS.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Replicator degrees of freedom allow publication of misleading failures to replicate
(Your Name) has sent you a message from PNAS
(Your Name) thought you would like to see the PNAS web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Replicator degrees of freedom allow publication of misleading failures to replicate
Christopher J. Bryan, David S. Yeager, Joseph M. O’Brien
Proceedings of the National Academy of Sciences Dec 2019, 116 (51) 25535-25545; DOI: 10.1073/pnas.1910951116

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Request Permissions
Share
Replicator degrees of freedom allow publication of misleading failures to replicate
Christopher J. Bryan, David S. Yeager, Joseph M. O’Brien
Proceedings of the National Academy of Sciences Dec 2019, 116 (51) 25535-25545; DOI: 10.1073/pnas.1910951116
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Mendeley logo Mendeley

Article Classifications

  • Social Sciences
  • Psychological and Cognitive Sciences
Proceedings of the National Academy of Sciences: 116 (51)
Table of Contents

Submit

Sign up for Article Alerts

Jump to section

  • Article
    • Abstract
    • Generalizability of This Debate
    • The Debate in Question
    • The Present Analysis
    • Results
    • Treatment Effect Heterogeneity Due to Replication Design Choices
    • Discussion: This Replication Debate
    • General Discussion
    • Materials and Methods
    • Acknowledgments
    • Footnotes
    • References
  • Figures & SI
  • Info & Metrics
  • PDF

You May Also be Interested in

Water from a faucet fills a glass.
News Feature: How “forever chemicals” might impair the immune system
Researchers are exploring whether these ubiquitous fluorinated molecules might worsen infections or hamper vaccine effectiveness.
Image credit: Shutterstock/Dmitry Naumov.
Reflection of clouds in the still waters of Mono Lake in California.
Inner Workings: Making headway with the mysteries of life’s origins
Recent experiments and simulations are starting to answer some fundamental questions about how life came to be.
Image credit: Shutterstock/Radoslaw Lecyk.
Cave in coastal Kenya with tree growing in the middle.
Journal Club: Small, sharp blades mark shift from Middle to Later Stone Age in coastal Kenya
Archaeologists have long tried to define the transition between the two time periods.
Image credit: Ceri Shipton.
Illustration of groups of people chatting
Exploring the length of human conversations
Adam Mastroianni and Daniel Gilbert explore why conversations almost never end when people want them to.
Listen
Past PodcastsSubscribe
Panda bear hanging in a tree
How horse manure helps giant pandas tolerate cold
A study finds that giant pandas roll in horse manure to increase their cold tolerance.
Image credit: Fuwen Wei.

Similar Articles

Site Logo
Powered by HighWire
  • Submit Manuscript
  • Twitter
  • Facebook
  • RSS Feeds
  • Email Alerts

Articles

  • Current Issue
  • Special Feature Articles – Most Recent
  • List of Issues

PNAS Portals

  • Anthropology
  • Chemistry
  • Classics
  • Front Matter
  • Physics
  • Sustainability Science
  • Teaching Resources

Information

  • Authors
  • Editorial Board
  • Reviewers
  • Subscribers
  • Librarians
  • Press
  • Cozzarelli Prize
  • Site Map
  • PNAS Updates
  • FAQs
  • Accessibility Statement
  • Rights & Permissions
  • About
  • Contact

Feedback    Privacy/Legal

Copyright © 2021 National Academy of Sciences. Online ISSN 1091-6490