Skip to main content

Main menu

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
    • Front Matter Portal
    • Journal Club
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
  • Submit
  • About
    • Editorial Board
    • PNAS Staff
    • FAQ
    • Accessibility Statement
    • Rights and Permissions
    • Site Map
  • Contact
  • Journal Club
  • Subscribe
    • Subscription Rates
    • Subscriptions FAQ
    • Open Access
    • Recommend PNAS to Your Librarian

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Home
Home
  • Log in
  • My Cart

Advanced Search

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
    • Front Matter Portal
    • Journal Club
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
Colloquium Paper

An empirical analysis of journal policy effectiveness for computational reproducibility

View ORCID ProfileVictoria Stodden, Jennifer Seiler, and Zhaokun Ma
  1. aSchool of Information Sciences, University of Illinois at Urbana–Champaign, Champaign, IL 61820;
  2. bDepartment of Statistics, Columbia University, New York, NY 10027

See allHide authors and affiliations

PNAS March 13, 2018 115 (11) 2584-2589; first published March 12, 2018; https://doi.org/10.1073/pnas.1708290115
Victoria Stodden
aSchool of Information Sciences, University of Illinois at Urbana–Champaign, Champaign, IL 61820;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Victoria Stodden
  • For correspondence: victoria@stodden.net
Jennifer Seiler
bDepartment of Statistics, Columbia University, New York, NY 10027
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Zhaokun Ma
bDepartment of Statistics, Columbia University, New York, NY 10027
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  1. Edited by David B. Allison, Indiana University Bloomington, Bloomington, IN, and accepted by Editorial Board Member Susan T. Fiske January 9, 2018 (received for review July 11, 2017)

  • Article
  • Figures & SI
  • Info & Metrics
  • PDF
Loading

Online Impact

 

Article Information

vol. 115 no. 11 2584-2589
DOI: 
https://doi.org/10.1073/pnas.1708290115
PubMed: 
29531050

Published By: 
National Academy of Sciences
Print ISSN: 
0027-8424
Online ISSN: 
1091-6490
History: 
  • Published in issue March 13, 2018.
  • Published first March 12, 2018.

Article Versions

  • Previous version (March 12, 2018 - 12:08).
  • You are viewing the most recent version of this article.
Copyright & Usage: 
© 2018 Published under the PNAS license.

Author Information

  1. Victoria Stoddena,1,
  2. Jennifer Seilerb, and
  3. Zhaokun Mab
  1. aSchool of Information Sciences, University of Illinois at Urbana–Champaign, Champaign, IL 61820;
  2. bDepartment of Statistics, Columbia University, New York, NY 10027
  1. Edited by David B. Allison, Indiana University Bloomington, Bloomington, IN, and accepted by Editorial Board Member Susan T. Fiske January 9, 2018 (received for review July 11, 2017)

Footnotes

  • ↵1To whom correspondence should be addressed. Email: victoria{at}stodden.net.
  • Author contributions: V.S. designed research; V.S., J.S., and Z.M. performed research; V.S., J.S., and Z.M. analyzed data; and V.S. and J.S. wrote the paper.

  • The authors declare no conflict of interest.

  • This paper results from the Arthur M. Sackler Colloquium of the National Academy of Sciences, “Reproducibility of Research: Issues and Proposed Remedies,” held March 8-10, 2017, at the National Academy of Sciences in Washington, DC. The complete program and video recordings of most presentations are available on the NAS website at www.nasonline.org/Reproducibility.

  • This article is a PNAS Direct Submission. D.B.A. is a guest editor invited by the Editorial Board.

  • Data deposition: We have created a repository at https://github.com/ReproducibilityInPublishing/Science-2018 containing supplemental material, including the code and email templates that were used in the survey.

Cited By...

  • 61 Citations
  • 72 Citations
  • Google Scholar

This article has been cited by the following articles in journals that are participating in Crossref Cited-by Linking.

  • Identifying and Overcoming Threats to Reproducibility, Replicability, Robustness, and Generalizability in Microbiome Research
    Patrick D. Schloss, Jacques Ravel
    mBio 2018 9 3
  • A Practical Guide for Transparency in Psychological Science
    Olivier Klein, Tom E. Hardwicke, Frederik Aust, Johannes Breuer, Henrik Danielsson, Alicia Hofelich Mohr, Hans IJzerman, Gustav Nilsonne, Wolf Vanpaemel, Michael C. Frank, Michéle Nuijten, Simine Vazire
    Collabra: Psychology 2018 4 1
  • Systematic benchmarking of omics computational tools
    Serghei Mangul, Lana S. Martin, Brian L. Hill, Angela Ka-Mei Lam, Margaret G. Distler, Alex Zelikovsky, Eleazar Eskin, Jonathan Flint
    Nature Communications 2019 10 1
  • Improving the usability and archival stability of bioinformatics software
    Serghei Mangul, Lana S. Martin, Eleazar Eskin, Ran Blekhman
    Genome Biology 2019 20 1
  • On the prevalence of uninformative parameters in statistical models applying model selection in applied ecology
    Shawn J. Leroux, Andrew J. Kroll
    PLOS ONE 2019 14 2
  • Open Science principles for accelerating trait-based science across the Tree of Life
    Rachael V. Gallagher, Daniel S. Falster, Brian S. Maitner, Roberto Salguero-Gómez, Vigdis Vandvik, William D. Pearse, Florian D. Schneider, Jens Kattge, Jorrit H. Poelen, Joshua S. Madin, Markus J. Ankenbrand, Caterina Penone, Xiao Feng, Vanessa M. Adams, John Alroy, Samuel C. Andrew, Meghan A. Balk, Lucie M. Bland, Brad L. Boyle, Catherine H. Bravo-Avila, Ian Brennan, Alexandra J. R. Carthey, Renee Catullo, Brittany R. Cavazos, Dalia A. Conde, Steven L. Chown, Belen Fadrique, Heloise Gibb, Aud H. Halbritter, Jennifer Hammock, J. Aaron Hogan, Hamish Holewa, Michael Hope, Colleen M. Iversen, Malte Jochum, Michael Kearney, Alexander Keller, Paula Mabee, Peter Manning, Luke McCormack, Sean T. Michaletz, Daniel S. Park, Timothy M. Perez, Silvia Pineda-Munoz, Courtenay A. Ray, Maurizio Rossetto, Hervé Sauquet, Benjamin Sparrow, Marko J. Spasojevic, Richard J. Telford, Joseph A. Tobias, Cyrille Violle, Ramona Walls, Katherine C. B. Weiss, Mark Westoby, Ian J. Wright, Brian J. Enquist
    Nature Ecology & Evolution 2020 4 3
  • Reproducibility of research: Issues and proposed remedies
    David B. Allison, Richard M. Shiffrin, Victoria Stodden
    Proceedings of the National Academy of Sciences 2018 115 11
  • Open science, reproducibility, and transparency in ecology
    Stephen M. Powers, Stephanie E. Hampton
    Ecological Applications 2019 29 1
  • Smart Contracts for Research Data Rights Management over the Ethereum Blockchain Network
    Adrian-Tudor Pãnescu, Vasile Manta
    Science & Technology Libraries 2018 37 3
  • Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail
    Marcin Miłkowski, Witold M. Hensel, Mateusz Hohol
    Journal of Computational Neuroscience 2018 45 3
  • Tellurium notebooks—An environment for reproducible dynamical modeling in systems biology
    J. Kyle Medley, Kiri Choi, Matthias König, Lucian Smith, Stanley Gu, Joseph Hellerstein, Stuart C. Sealfon, Herbert M. Sauro, Andreas Prlic
    PLOS Computational Biology 2018 14 6
  • Bringing Molecular Dynamics Simulation Data into View
    Peter W. Hildebrand, Alexander S. Rose, Johanna K.S. Tiemann
    Trends in Biochemical Sciences 2019 44 11
  • Challenges and recommendations to improve the installability and archival stability of omics computational tools
    Serghei Mangul, Thiago Mosqueiro, Richard J. Abdill, Dat Duong, Keith Mitchell, Varuni Sarwal, Brian Hill, Jaqueline Brito, Russell Jared Littman, Benjamin Statz, Angela Ka-Mei Lam, Gargi Dayama, Laura Grieneisen, Lana S. Martin, Jonathan Flint, Eleazar Eskin, Ran Blekhman
    PLOS Biology 2019 17 6
  • Experimenting with reproducibility: a case study of robustness in bioinformatics
    Yang-Min Kim, Jean-Baptiste Poline, Guillaume Dumas
    GigaScience 2018 7 7
  • Available and missing data to model impact of climate change on European forests
    Paloma Ruiz-Benito, Giorgio Vacchiano, Emily R. Lines, Christopher P.O. Reyer, Sophia Ratcliffe, Xavier Morin, Florian Hartig, Annikki Mäkelä, Rasoul Yousefpour, Jimena E. Chaves, Alicia Palacios-Orueta, Marta Benito-Garzón, Cesar Morales-Molino, J. Julio Camarero, Alistair S. Jump, Jens Kattge, Aleksi Lehtonen, Andreas Ibrom, Harry J.F. Owen, Miguel A. Zavala
    Ecological Modelling 2020 416
  • Assessing data availability and research reproducibility in hydrology and water resources
    James H. Stagge, David E. Rosenberg, Adel M. Abdallah, Hadia Akbar, Nour A. Attallah, Ryan James
    Scientific Data 2019 6 1
  • The Climate Data Toolbox for MATLAB
    Chad A. Greene, Kaustubh Thirumalai, Kelly A. Kearney, José Miguel Delgado, Wolfgang Schwanghart, Natalie S. Wolfenbarger, Kristen M. Thyng, David E. Gwyther, Alex S. Gardner, Donald D. Blankenship
    Geochemistry, Geophysics, Geosystems 2019 20 7
  • A funder-imposed data publication requirement seldom inspired data sharing
    Jessica L. Couture, Rachael E. Blake, Gavin McDonald, Colette L. Ward, Jelte M. Wicherts
    PLOS ONE 2018 13 7
  • A synthetic dataset primer for the biobehavioural sciences to promote reproducibility and hypothesis generation
    Daniel S Quintana
    eLife 2020 9
  • Creating and sharing reproducible research code the workflowr way
    John D. Blischak, Peter Carbonetto, Matthew Stephens
    F1000Research 2019 8
  • Accessible molecular phylogenomics at no cost: obtaining 14 new mitogenomes for the ant subfamily Pseudomyrmecinae from public data
    Gabriel A. Vieira, Francisco Prosdocimi
    PeerJ 2019 7
  • Cross-Linguistic Data Formats, advancing data sharing and re-use in comparative linguistics
    Robert Forkel, Johann-Mattis List, Simon J. Greenhill, Christoph Rzymski, Sebastian Bank, Michael Cysouw, Harald Hammarström, Martin Haspelmath, Gereon A. Kaiping, Russell D. Gray
    Scientific Data 2018 5 1
  • Publication reform to safeguard wildlife from researcher harm
    Kate A. Field, Paul C. Paquet, Kyle Artelle, Gilbert Proulx, Ryan K. Brook, Chris T. Darimont
    PLOS Biology 2019 17 4
  • Reproducibility of Quantitative Systems Pharmacology Models: Current Challenges and Future Opportunities
    Daniel C. Kirouac, Brian Cicali, Stephan Schmidt
    CPT: Pharmacometrics & Systems Pharmacology 2019 8 4
  • The Brazilian Reproducibility Initiative
    Olavo B Amaral, Kleber Neves, Ana P Wasilewska-Sampaio, Clarissa FD Carneiro
    eLife 2019 8
  • Enforcing public data archiving policies in academic publishing: A study of ecology journals
    Dan Sholler, Karthik Ram, Carl Boettiger, Daniel S Katz
    Big Data & Society 2019 6 1
  • Recommendations to enhance rigor and reproducibility in biomedical research
    Jaqueline J Brito, Jun Li, Jason H Moore, Casey S Greene, Nicole A Nogoy, Lana X Garmire, Serghei Mangul
    GigaScience 2020 9 6
  • Data Sharing and Scientific Impact in Eddy Covariance Research
    B. Bond-Lamberty
    Journal of Geophysical Research: Biogeosciences 2018 123 4
  • Four erroneous beliefs thwarting more trustworthy research
    Mark Yarborough, Robert Nadon, David G Karlin
    eLife 2019 8
  • Calibrating the Scientific Ecosystem Through Meta-Research
    Tom E. Hardwicke, Stylianos Serghiou, Perrine Janiaud, Valentin Danchev, Sophia Crüwell, Steven N. Goodman, John P.A. Ioannidis
    Annual Review of Statistics and Its Application 2020 7 1
  • Low availability of code in ecology: A call for urgent action
    Antica Culina, Ilona van den Berg, Simon Evans, Alfredo Sánchez-Tójar
    PLOS Biology 2020 18 7
  • Reproducible Research Using Biomodels
    Pedro Mendes
    Bulletin of Mathematical Biology 2018 80 12
  • “Reproducible” Research in Mathematical Sciences Requires Changes in our Peer Review Culture and Modernization of our Current Publication Approach
    Santiago Schnell
    Bulletin of Mathematical Biology 2018 80 12
  • Big data in radiation biology and epidemiology; an overview of the historical and contemporary landscape of data and biomaterial archives
    Paul N. Schofield, Ulrike Kulka, Soile Tapio, Bernd Grosche
    International Journal of Radiation Biology 2019 95 7
  • Journal data policies: Exploring how the understanding of editors and authors corresponds to the policies themselves
    Thu-Mai Christian, Amanda Gooch, Todd Vision, Elizabeth Hull, Cassidy Rose Sugimoto
    PLOS ONE 2020 15 3
  • Open Science is for Aging Research, Too
    Derek M Isaacowitz, Majse Lind, Laura P Sands
    Innovation in Aging 2019 3 4
  • Reproducibility and Replicability in the Context of the Contested Identities of Geography
    Daniel Sui, Peter Kedron
    Annals of the American Association of Geographers 2020
  • Analysis of Open Data and Computational Reproducibility in Registered Reports in Psychology
    Pepijn Obels, Daniël Lakens, Nicholas A. Coles, Jaroslav Gottfried, Seth A. Green
    Advances in Methods and Practices in Psychological Science 2020 3 2
  • Growing the FAIR Community at the Intersection of the Geosciences and Pure and Applied Chemistry
    Shelley Stall, Leah McEwen, Lesley Wyborn, Nancy Hoebelheinrich, Ian Bruno
    Data Intelligence 2020 2 1-2
  • Open Models for Clinical Pharmacology
    Daniel C. Kirouac
    Clinical Pharmacology & Therapeutics 2020 107 4
  • Open Up – the Mission Statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab on Open Science
    Christina B. Reimer, Zhang Chen, Carsten Bundt, Charlotte Eben, Raquel E. London, Sirarpi Vardanian
    Psychologica Belgica 2019 59 1
  • Practical Reproducibility in Geography and Geosciences
    Daniel Nüst, Edzer Pebesma
    Annals of the American Association of Geographers 2020
  • The Next Frontier: Making Research More Reproducible
    David E. Rosenberg, Yves Filion, Rebecca Teasley, Samuel Sandoval-Solis, Jory S. Hecht, Jakobus E. van Zyl, George F. McMahon, Jeffery S. Horsburgh, Joseph R. Kasprzyk, David G. Tarboton
    Journal of Water Resources Planning and Management 2020 146 6
  • Assessing the state of research data publication in hydrology: A perspective from the Consortium of Universities for the Advancement of Hydrologic Science, Incorporated
    Jeffery S. Horsburgh, Richard P. Hooper, Jerad Bales, Margaret Hedstrom, Heidi J. Imker, Kerstin A. Lehnert, Lea A. Shanley, Shelley Stall
    WIREs Water 2020 7 3
  • Enhancing Climate Change Research With Open Science
    Travis C. Tai, James P. W. Robinson
    Frontiers in Environmental Science 2018 6
  • Facilitating reproducible project management and manuscript development in team science: The projects R package
    Nikolas I. Krieger, Adam T. Perzynski, Jarrod E. Dalton, Vasilis J. Promponas
    PLOS ONE 2019 14 7
  • Interpreting and integrating big data in the life sciences
    Matteo Pellegrini, Serghei Mangul
    Emerging Topics in Life Sciences 2019 3 4
  • Pixel: a content management platform for quantitative omics data
    Thomas Denecker, William Durand, Julien Maupetit, Charles Hébert, Jean-Michel Camadro, Pierre Poulain, Gaëlle Lelandais
    PeerJ 2019 7
  • Reproducibility Certification in Economics Research
    Christophe Hurlin, Christophe Perignon
    SSRN Electronic Journal 2019
  • Reproducibility and replicability: opportunities and challenges for geospatial research
    Peter Kedron, Wenwen Li, Stewart Fotheringham, Michael Goodchild
    International Journal of Geographical Information Science 2021 35 3
  • Reproducible image handling and analysis
    Kota Miura, Simon F Nørrelykke
    The EMBO Journal 2021 40 3
  • Successes and Struggles with Computational Reproducibility: Lessons from the Fragile Families Challenge
    David M. Liu, Matthew J. Salganik
    Socius: Sociological Research for a Dynamic World 2019 5
  • The case for openness in engineering research
    Devin R. Berg, Kyle E. Niemeyer
    F1000Research 2018 7
  • The case for openness in engineering research
    Devin R. Berg, Kyle E. Niemeyer
    F1000Research 2018 7
  • The rapid proliferation of solicited content online journals: a quest to disseminate knowledge?
    Richard C. Becker, Vlad Cotarlan, Sakthi Sadayappan
    Journal of Thrombosis and Thrombolysis 2019 47 3
  • Analytic reproducibility in articles receiving open data badges at the journal Psychological Science : an observational study
    Tom E. Hardwicke, Manuel Bohn, Kyle MacDonald, Emily Hembacher, Michèle B. Nuijten, Benjamin N. Peloquin, Benjamin E. deMayo, Bria Long, Erica J. Yoon, Michael C. Frank
    Royal Society Open Science 2021 8 1
  • A decentralized framework for cultivating research lifecycle transparency
    Wei Jeng, Shih-Hung Wang, Hung-Wei Chen, Po-Wei Huang, Yu-Jen Chen, Hsu-Chun Hsiao, He Debiao
    PLOS ONE 2020 15 11
  • Are Descriptions of Methods Alone Sufficient for Study Reproducibility? An Example From the Cardiovascular Literature
    Neal D. Goldstein, Ghassan B. Hamra, Sam Harper
    Epidemiology 2020 31 2
  • Clinical relevance assessment of animal preclinical research (RAA) tool: development and explanation
    Kurinchi S. Gurusamy, David Moher, Marilena Loizidou, Irfan Ahmed, Marc T. Avey, Carly C. Barron, Brian Davidson, Miriam Dwek, Christian Gluud, Gavin Jell, Kiran Katakam, Joshua Montroy, Timothy D. McHugh, Nicola J. Osborne, Merel Ritskes-Hoitinga, Kees van Laarhoven, Jan Vollert, Manoj Lalu
    PeerJ 2021 9
  • Cloud and Serverless Computing for Scientists
    Juan A. Añel, Diego P. Montes, Javier Rodeiro Iglesias
    2020
  • Comment on: ‘Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices’
    David N. Borg, Joshua J. Bon, Kristin L. Sainani, Brenton J. Baguley, Nicholas J. Tierney, Christopher Drovandi
    Sports Medicine 2020 50 8
  • Current status on the need for improved accessibility to climate models code
    Juan A. Añel, Michael García-Rodríguez, Javier Rodeiro
    Geoscientific Model Development 2021 14 2
  • Double trouble? The communication dimension of the reproducibility crisis in experimental psychology and neuroscience
    Witold M. Hensel
    European Journal for Philosophy of Science 2020 10 3
  • Microbiome Metadata Standards: Report of the National Microbiome Data Collaborative’s Workshop and Follow-On Activities
    Pajau Vangay, Josephine Burgin, Anjanette Johnston, Kristen L. Beck, Daniel C. Berrios, Kai Blumberg, Shane Canon, Patrick Chain, John-Marc Chandonia, Danielle Christianson, Sylvain V. Costes, Joan Damerow, William D. Duncan, Jose Pablo Dundore-Arias, Kjiersten Fagnan, Jonathan M. Galazka, Sean M. Gibbons, David Hays, Judson Hervey, Bin Hu, Bonnie L. Hurwitz, Pankaj Jaiswal, Marcin P. Joachimiak, Linda Kinkel, Joshua Ladau, Stanton L. Martin, Lee Ann McCue, Kayd Miller, Nigel Mouncey, Chris Mungall, Evangelos Pafilis, T. B. K. Reddy, Lorna Richardson, Simon Roux, Justin P. Shaffer, Jagadish Chandrabose Sundaramurthi, Luke R. Thompson, Ruth E. Timme, Jie Zheng, Elisha M. Wood-Charlson, Emiley A. Eloe-Fadrosh, Vanni Bucci
    mSystems 2021 6 1
  • On code sharing and model documentation of published individual and agent-based models
    Marco A. Janssen, Calvin Pritchard, Allen Lee
    Environmental Modelling & Software 2020 134
  • Parallel Computing Technologies
    Maxim Gorodnichev, Yuri Medvedev
    2019 11657
  • Rclean: A Tool for Writing Cleaner, More Transparent Code
    Matthew Lau, Thomas F. J.-M. Pasquier, Margo Seltzer
    Journal of Open Source Software 2020 5 46
  • Relational data paradigms: What do we learn by taking the materiality of databases seriously?
    Andrea K Thomer, Karen M Wickett
    Big Data & Society 2020 7 1
  • Repository Approaches to Improving the Quality of Shared Data and Code
    Ana Trisovic, Katherine Mika, Ceilyn Boyd, Sebastian Feger, Mercè Crosas
    Data 2021 6 2
  • The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
    Saman Razavi, Anthony Jakeman, Andrea Saltelli, Clémentine Prieur, Bertrand Iooss, Emanuele Borgonovo, Elmar Plischke, Samuele Lo Piano, Takuya Iwanaga, William Becker, Stefano Tarantola, Joseph H.A. Guillaume, John Jakeman, Hoshin Gupta, Nicola Melillo, Giovanni Rabitti, Vincent Chabridon, Qingyun Duan, Xifu Sun, Stefán Smith, Razi Sheikholeslami, Nasim Hosseini, Masoud Asadzadeh, Arnald Puy, Sergei Kucherenko, Holger R. Maier
    Environmental Modelling & Software 2021 137
  • WordSeg: Standardizing unsupervised word form segmentation from text
    Mathieu Bernard, Roland Thiolliere, Amanda Saksida, Georgia R. Loukatou, Elin Larsen, Mark Johnson, Laia Fibla, Emmanuel Dupoux, Robert Daland, Xuan Nga Cao, Alejandrina Cristia
    Behavior Research Methods 2020 52 1
  • YARD: A Tool for Curating Research Outputs
    Limor Peer, Joshua Dull
    Data Science Journal 2020 19

Article usage

Article usage: March 2018 to March 2021

AbstractFullPdf
Mar 201815711146081740
Apr 201822572080314
May 20181064832152
Jun 20181392607131
Jul 20181840540197
Aug 2018800512128
Sep 2018411346106
Oct 201891464193
Nov 201826322120
Dec 20182240986
Total 201823614207203167
Jan 201916330110
Feb 20193223791
Mar 20193527991
Apr 20192820690
May 201910116461
Jun 2019256231103
Jul 20194820359
Aug 20192513996
Sep 20192115073
Oct 20192822278
Nov 20192120267
Dec 20192232747
Total 20196332690966
Jan 20201740481
Feb 20202635776
Mar 202012924462
Apr 20201329589
May 20202529982
Jun 20202717093
Jul 20202125994
Aug 20203126878
Sep 202032292155
Oct 20204228677
Nov 20205928857
Dec 20203920381
Total 202046133651025
Jan 20216832777
Feb 20214418590
Mar 202169823
Total 2021118610190
Total24826273855348
PreviousNext
Back to top
Article Alerts
Email Article

Thank you for your interest in spreading the word on PNAS.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
An empirical analysis of journal policy effectiveness for computational reproducibility
(Your Name) has sent you a message from PNAS
(Your Name) thought you would like to see the PNAS web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Journal policy for computational reproducibility
Victoria Stodden, Jennifer Seiler, Zhaokun Ma
Proceedings of the National Academy of Sciences Mar 2018, 115 (11) 2584-2589; DOI: 10.1073/pnas.1708290115

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Request Permissions
Share
Journal policy for computational reproducibility
Victoria Stodden, Jennifer Seiler, Zhaokun Ma
Proceedings of the National Academy of Sciences Mar 2018, 115 (11) 2584-2589; DOI: 10.1073/pnas.1708290115
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Mendeley logo Mendeley

Article Classifications

  • Physical Sciences
  • Computer Sciences
  • Social Sciences
  • Social Sciences
Proceedings of the National Academy of Sciences: 115 (11)
Table of Contents

Submit

Sign up for Article Alerts

Jump to section

  • Article
    • Abstract
    • Results
    • Methods
    • Conclusion
    • Acknowledgments
    • Footnotes
    • References
  • Figures & SI
  • Info & Metrics
  • PDF

You May Also be Interested in

Setting sun over a sun-baked dirt landscape
Core Concept: Popular integrated assessment climate policy models have key caveats
Better explicating the strengths and shortcomings of these models will help refine projections and improve transparency in the years ahead.
Image credit: Witsawat.S.
Model of the Amazon forest
News Feature: A sea in the Amazon
Did the Caribbean sweep into the western Amazon millions of years ago, shaping the region’s rich biodiversity?
Image credit: Tacio Cordeiro Bicudo (University of São Paulo, São Paulo, Brazil), Victor Sacek (University of São Paulo, São Paulo, Brazil), and Lucy Reading-Ikkanda (artist).
Syrian archaeological site
Journal Club: In Mesopotamia, early cities may have faltered before climate-driven collapse
Settlements 4,200 years ago may have suffered from overpopulation before drought and lower temperatures ultimately made them unsustainable.
Image credit: Andrea Ricci.
Click beetle on a leaf
How click beetles jump
Marianne Alleyna, Aimy Wissa, and Ophelia Bolmin explain how the click beetle amplifies power to pull off its signature jump.
Listen
Past PodcastsSubscribe
Birds nestling on tree branches
Parent–offspring conflict in songbird fledging
Some songbird parents might improve their own fitness by manipulating their offspring into leaving the nest early, at the cost of fledgling survival, a study finds.
Image credit: Gil Eckrich (photographer).

Similar Articles

Site Logo
Powered by HighWire
  • Submit Manuscript
  • Twitter
  • Facebook
  • RSS Feeds
  • Email Alerts

Articles

  • Current Issue
  • Special Feature Articles – Most Recent
  • List of Issues

PNAS Portals

  • Anthropology
  • Chemistry
  • Classics
  • Front Matter
  • Physics
  • Sustainability Science
  • Teaching Resources

Information

  • Authors
  • Editorial Board
  • Reviewers
  • Subscribers
  • Librarians
  • Press
  • Site Map
  • PNAS Updates
  • FAQs
  • Accessibility Statement
  • Rights & Permissions
  • About
  • Contact

Feedback    Privacy/Legal

Copyright © 2021 National Academy of Sciences. Online ISSN 1091-6490