Skip to main content

Main menu

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
    • Front Matter Portal
    • Journal Club
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
  • Submit
  • About
    • Editorial Board
    • PNAS Staff
    • FAQ
    • Accessibility Statement
    • Rights and Permissions
    • Site Map
  • Contact
  • Journal Club
  • Subscribe
    • Subscription Rates
    • Subscriptions FAQ
    • Open Access
    • Recommend PNAS to Your Librarian

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Home
Home
  • Log in
  • My Cart

Advanced Search

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
    • Front Matter Portal
    • Journal Club
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
Reply

Reply to Margalida and Colomer: Science should strive to prevent mistakes, not corrections

Kyle Siler, Kirby Lee, and Lisa Bero
  1. aDepartment of Strategic Management, Rotman School of Management, University of Toronto, Toronto, ON, Canada M5S 3E6;
  2. bDepartment of Clinical Pharmacy, University of California, San Francisco, CA 94143; and
  3. cFaculty of Pharmacy and Charles Perkins Centre, University of Sydney, Sydney, NSW 2006, Australia

See allHide authors and affiliations

PNAS March 31, 2015 112 (13) E1512; first published March 10, 2015; https://doi.org/10.1073/pnas.1501371112
Kyle Siler
aDepartment of Strategic Management, Rotman School of Management, University of Toronto, Toronto, ON, Canada M5S 3E6;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: ksiler@gmail.com
Kirby Lee
bDepartment of Clinical Pharmacy, University of California, San Francisco, CA 94143; and
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lisa Bero
cFaculty of Pharmacy and Charles Perkins Centre, University of Sydney, Sydney, NSW 2006, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
  • PDF
Loading

Margalida and Colomer (1) proffer a “mistake index” based on corrections published by scientific journals to gauge peer review quality. This is an interesting idea but has numerous theoretical and practical problems.

Even with recently slightly rising rates, only 2–3% of articles in Nature, PNAS, and Science issue corrections. Most are minute or trivial. We examined a sample of the last 100 corrections issued for research articles in Nature, PNAS, and Science in 2013. There were six retractions and one seemingly important revision. Beyond those 7 corrections, the other 93 corrections mostly involved small details that did not affect the research findings of the original article. Thirty corrections involved author names, affiliations, or acknowledgments, which have no bearing on the content of science.

Because most retractions involved author malfeasance, it is unfair to pin those errors on peer review. Further, the 30 corrections to names/acknowledgments had nothing to do with scientific quality control. Expecting perfection from peer reviewers is unrealistic, particularly with minutiae and unshared data. Peer reviewers might have done a brilliant job filtering out most mistakes and positively refining the article in other ways. Errors that make it into print are visible, unlike errors averted through quality control processes. As data and information become more accessible in the internet age, increased transparency enables greater scrutiny of research findings from larger and more diverse populations. Consequently, published corrections—or at least challenges to published articles—should continue to rise in science. This would likely be a positive development. Debate and refinement of previous work drives scientific progress.

In our sample, the lack of corrections dealing with moderate or severe errors was notable. This may be a sign of the effectiveness of peer review to filter out and/or revise such mistakes, as well as the skill of researchers. However, surely more than 2–3% of articles have flaws of some sort. Gelman (2) lamented that it is excessively difficult to publish criticisms and obtain data for published articles. Although criticisms during peer review must be handled carefully, criticisms after publication are held to a much higher bar. As a result, it is often difficult to directly refute mistakes, perhaps as shown by extremely low correction rates in scientific journals. Publishing mistakes go beyond empirical accuracy. Our article (3) suggested that gatekeepers reject many high-quality articles. Still, often in science and in life, imperfection does not necessarily imply a lack of merit.

It is worth noting that articles published in high-impact outlets are under intense scrutiny. This is in part how and why small—and usually inconsequential —errors are spotted and reported. A mistake index would also provide journals with incentives to not issue corrections or prefer less risky or complex articles.

Although even venial errors are lamentable, corrections are not, especially because scientific errors can affect lives beyond the ivory tower (4). Corrections help ensure that scientists conduct research based on accurate information, teach lessons regarding how and why mistakes occur, and reinforce norms of meticulousness and vigilance in science.

Footnotes

  • ↵1To whom correspondence should be addressed. Email: ksiler{at}gmail.com.
  • Author contributions: K.S. performed research; and K.S., K.L., and L.B. wrote the paper.

  • The authors declare no conflict of interest.

References

  1. ↵
    1. Margalida A,
    2. Colomer MÀ
    (2015) Mistake index as a surrogate of quality in scientific manuscripts. Proc Natl Acad Sci USA 112:E1511
    .
    OpenUrlFREE Full Text
  2. ↵
    1. Gelman A
    (2013) It’s too hard to publish criticisms and obtain data for replication. Chance 26(3):49–52
    .
    OpenUrl
  3. ↵
    1. Siler K,
    2. Lee K,
    3. Bero L
    (2015) Measuring the effectiveness of scientific gatekeeping. Proc Natl Acad Sci USA 112(2):360–365
    .
    OpenUrlAbstract/FREE Full Text
  4. ↵
    1. Krugman P
    (2013) The Excel depression. New York Times, April 19, 2013, p A31
    .
PreviousNext
Back to top
Article Alerts
Email Article

Thank you for your interest in spreading the word on PNAS.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Reply to Margalida and Colomer: Science should strive to prevent mistakes, not corrections
(Your Name) has sent you a message from PNAS
(Your Name) thought you would like to see the PNAS web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Science should prevent mistakes, not corrections
Kyle Siler, Kirby Lee, Lisa Bero
Proceedings of the National Academy of Sciences Mar 2015, 112 (13) E1512; DOI: 10.1073/pnas.1501371112

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Request Permissions
Share
Science should prevent mistakes, not corrections
Kyle Siler, Kirby Lee, Lisa Bero
Proceedings of the National Academy of Sciences Mar 2015, 112 (13) E1512; DOI: 10.1073/pnas.1501371112
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Mendeley logo Mendeley

Article Classifications

  • Social Sciences
  • Social Sciences

This article has a Letter. Please see:

  • Relationship between Letter and Reply - March 10, 2015

See related content:

  • Measuring the effectiveness of peer review
    - Dec 22, 2014
Proceedings of the National Academy of Sciences: 112 (13)
Table of Contents

Submit

Sign up for Article Alerts

Jump to section

  • Article
    • Footnotes
    • References
  • Info & Metrics
  • PDF

You May Also be Interested in

Setting sun over a sun-baked dirt landscape
Core Concept: Popular integrated assessment climate policy models have key caveats
Better explicating the strengths and shortcomings of these models will help refine projections and improve transparency in the years ahead.
Image credit: Witsawat.S.
Model of the Amazon forest
News Feature: A sea in the Amazon
Did the Caribbean sweep into the western Amazon millions of years ago, shaping the region’s rich biodiversity?
Image credit: Tacio Cordeiro Bicudo (University of São Paulo, São Paulo, Brazil), Victor Sacek (University of São Paulo, São Paulo, Brazil), and Lucy Reading-Ikkanda (artist).
Syrian archaeological site
Journal Club: In Mesopotamia, early cities may have faltered before climate-driven collapse
Settlements 4,200 years ago may have suffered from overpopulation before drought and lower temperatures ultimately made them unsustainable.
Image credit: Andrea Ricci.
Steamboat Geyser eruption.
Eruption of Steamboat Geyser
Mara Reed and Michael Manga explore why Yellowstone's Steamboat Geyser resumed erupting in 2018.
Listen
Past PodcastsSubscribe
Birds nestling on tree branches
Parent–offspring conflict in songbird fledging
Some songbird parents might improve their own fitness by manipulating their offspring into leaving the nest early, at the cost of fledgling survival, a study finds.
Image credit: Gil Eckrich (photographer).

Similar Articles

Site Logo
Powered by HighWire
  • Submit Manuscript
  • Twitter
  • Facebook
  • RSS Feeds
  • Email Alerts

Articles

  • Current Issue
  • Special Feature Articles – Most Recent
  • List of Issues

PNAS Portals

  • Anthropology
  • Chemistry
  • Classics
  • Front Matter
  • Physics
  • Sustainability Science
  • Teaching Resources

Information

  • Authors
  • Editorial Board
  • Reviewers
  • Subscribers
  • Librarians
  • Press
  • Site Map
  • PNAS Updates
  • FAQs
  • Accessibility Statement
  • Rights & Permissions
  • About
  • Contact

Feedback    Privacy/Legal

Copyright © 2021 National Academy of Sciences. Online ISSN 1091-6490