New Research In
Physical Sciences
Social Sciences
Featured Portals
Articles by Topic
Biological Sciences
Featured Portals
Articles by Topic
- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology
Reviewer bias in single- versus double-blind peer review
Edited by Susan T. Fiske, Princeton University, Princeton, NJ, and approved October 10, 2017 (received for review May 3, 2017)
This article has a Letter. Please see:
- Radiation oncology authors and reviewers prefer double-blind peer review - February 06, 2018
See related content:
- IJROBP study consistent with our findings- Feb 06, 2018

Online Impact
Article Information
- Published in issue November 28, 2017.
- Published first November 14, 2017.
Article Versions
- Previous version (November 14, 2017 - 08:42).
- You are viewing the most recent version of this article.
Author Information
- aGoogle, Inc., Mountain View, CA 94043;
- bState Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
Edited by Susan T. Fiske, Princeton University, Princeton, NJ, and approved October 10, 2017 (received for review May 3, 2017)
Footnotes
- ↵1To whom correspondence should be addressed. Email: atomkins{at}gmail.com.
An extended abstract of this work has been previously posted as a preprint (1).
Author contributions: A.T. and M.Z. designed research; A.T. and M.Z. performed research; W.D.H. analyzed data; and A.T., M.Z., and W.D.H. wrote the paper.
Conflict of interest statement: A.T. and W.D.H. are employed and paid by Google, Inc. Google often provides funding to conferences, including the WSDM conference studied in this work.
This article is a PNAS Direct Submission.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1707323114/-/DCSupplemental.
Cited By...
This article has been cited by the following articles in journals that are participating in Crossref Cited-by Linking.
- Christopher Allen, David M. A. MehlerPLOS Biology 2019 17 5
- Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Katz, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O'Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien ColombF1000Research 2017 6
- Giangiacomo Bravo, Francisco Grimaldo, Emilia López-Iñesta, Bahar Mehmani, Flaminio SquazzoniNature Communications 2019 10 1
- Charles W. Fox, C. E. Timothy PaineEcology and Evolution 2019 9 6
- Samir Haffar, Fateh Bazerbachi, M. Hassan MuradMayo Clinic Proceedings 2019 94 4
- Matthew T. McBee, Matthew C. Makel, Scott J. Peters, Michael S. MatthewsGifted Child Quarterly 2018 62 4
- Wadsworth A. Williams, Katherine L. Garvey, Denise M. Goodman, Diane S. Lauderdale, Lainie Friedman RossThe Journal of Pediatrics 2018 200
- Barbara McGillivray, Elisa De RanieriResearch Integrity and Peer Review 2018 3 1
- Mary K. Feeney, Lisa Carson, Helen DickinsonPublic Administration Review 2019 79 1
- Stav Atir, Melissa J. FergusonProceedings of the National Academy of Sciences 2018 115 28
- Patrick S. Forscher, William T. L. Cox, Markus Brauer, Patricia G. DevineNature Human Behaviour 2019 3 3
- Jonathan P. Tennant, Tony Ross-HellauerResearch Integrity and Peer Review 2020 5 1
- Heidi Allen, Alexandra Cury, Thomas Gaston, Chris Graf, Hannah Wakley, Michael WillisLearned Publishing 2019 32 2
- Chao Lu, Yi Bu, Jie Wang, Ying Ding, Vetle Torvik, Matthew Schnaars, Chengzhi ZhangJournal of the Association for Information Science and Technology 2019 70 5
- Julia K. Baum, Tara G. MartinNature Ecology & Evolution 2018 2 2
- Charles W. Fox, Ken Thompson, Alan Knapp, Lara A. Ferry, Enrico L. Rezende, Emilie Aimé, Jennifer MeyerFunctional Ecology 2019 33 1
- Ivana Drvenica, Giangiacomo Bravo, Lucija Vejmelka, Aleksandar Dekanski, Olgica NedićPublications 2018 7 1
- Katherine Egan Bennett, Reshma Jagsi, Anthony ZietmanProceedings of the National Academy of Sciences 2018 115 9
- James CorcoranWritten Communication 2019 36 4
- Susan Guthrie, Daniela Rodriguez Rincon, Gordon McInroy, Becky Ioppolo, Salil GunashekarF1000Research 2019 8
- Paul E. Smaldino, Matthew A. Turner, Pablo A. Contreras KallensRoyal Society Open Science 2019 6 7
- Nathan Honeycutt, Lee JussimPsychological Inquiry 2020 31 1
- Richard B. Primack, Tracey J. Regan, Vincent Devictor, Lucy Zipf, Laurent Godet, Rafael Loyola, Bea Maas, Robin J. Pakeman, Graeme S. Cumming, Amanda E. Bates, Liba Pejchar, Lian Pin KohBiological Conservation 2019 238
- Sarah Stewart, Greg Gamble, Andrew Grey, Nicola DalbethBMJ Open 2020 10 6
- Christine E Gee, Nicholas J Talley ACMedical Journal of Australia 2019 211 4
- Karima Chaabna, Sohaila Cheema, Amit Abraham, Ravinder MamtaniJournal of Evidence-Based Medicine 2020 13 3
- Jay J. Van Bavel, Diego A. Reinero, Elizabeth Harris, Claire E. Robertson, Philip PärnametsPsychological Inquiry 2020 31 1
- Ulrike K MüllerIntegrative and Comparative Biology 2019 59 6
- Kenneth Ward ChurchNatural Language Engineering 2020 26 2
- Yang Yang, Wu Youyou, Brian UzziProceedings of the National Academy of Sciences 2020 117 20
- Danula K. Gamage, Almudena SevillaAEA Papers and Proceedings 2019 109
- Julie Posselt, Theresa E. Hernandez, Cynthia D. Villarreal, Aireale J. Rodgers, Lauren N. Irwin2020 35
- Wassim Najjar, Marco Abdo Mouanness, Georges Rameh, Tony BaziEuropean Journal of Obstetrics & Gynecology and Reproductive Biology 2019 241
- Jose Florencio F Lapeña, Peter L Munk, Aik Saw, Wilfred CG PehMedical Journal of Australia 2019 210 8
- Adam S. Chilton, Jonathan S. Masur, Kyle RozemaSSRN Electronic Journal 2018
- Adam Chilton, Jonathan Masur, Kyle RozemaAmerican Law and Economics Review 2020 22 1
- Balázs Győrffy, Péter Herman, István SzabóJournal of Informetrics 2020 14 3
- Anna Severin, Matthias EggerBritish Journal of Sports Medicine 2020
- Vaibhav Gupta, Natalie G. Coburn, Allan S. DetskyAnnals of Surgery 2020 272 1
- Charles MenzieIntegrated Environmental Assessment and Management 2019 15 1
- Mark Skopec, Hamdi Issa, Julie Reed, Matthew HarrisResearch Integrity and Peer Review 2020 5 1
- Michael P. Kranak, John Michael Falligant, Peter Bradtke, Nicole L. Hausman, Griffin W. RookerJournal of Applied Behavior Analysis 2020 53 4
- Brian J. Zikmund-FisherMDM Policy & Practice 2020 5 2
- Brian J. Zikmund-FisherMedical Decision Making 2020 40 7
- Eitan Frachtenberg, Noah KosterPeerJ Computer Science 2020 6
- Dinesh K. Pradhan, Joyita Chakraborty, Prasenjit Choudhary, Subrata NandiJournal of Informetrics 2020 14 2
- Helena Mihaljević, Lucía SantamaríaQuantitative Science Studies 2020 1 4
- Kamber L Hart, Roy H PerlisBMJ Open 2021 11 1
- Chris Street, Kerry W. WardACM SIGMIS Database: the DATABASE for Advances in Information Systems 2019 50 4
- Martin Daumiller, Stefan Siegel, Markus DreselResearch Evaluation 2019 28 3
- Sora Yasri, Viroj WiwanitkitJournal of Dental Research and Review 2019 6 2
- Paul E. Smaldino, Matthew A. Turner, Pablo A. Contreras KallensRoyal Society Open Science 2019 6 8
- Anne-Marie M. Oswald, Srdjan OstojicNature Reviews Neuroscience 2020 21 11
- Moe Kayali, Babak Salimi, Dan SuciuProceedings of the VLDB Endowment 2020 13 12
- Brian Park, Eunhee Sohn, Soohun Kim, Sergio A. UsechePLOS ONE 2020 15 8
- Gaby Moawad, Paul Tyan, Sara RahmanJournal of Minimally Invasive Gynecology 2019 26 5
- Christine Cuskley, Seán G Roberts, Stephen Politzer-Ahles, Tessa VerhoefJournal of Language Evolution 2019
- Mehmet A. OrhanQuality in Higher Education 2021
- Wei ChenJournal of Mechanical Design 2020 142 8
- Dirk M. Elston, Jane M. Grant-Kels, Nikki Levin, Murad Alam, Emily M. Altman, Robert T. Brodell, Anthony P. Fernandez, M. Yadira Hurley, John Maize, Desiree Ratner, Julie SchafferJournal of the American Academy of Dermatology 2020
- Karen A Walker, Kelley TilmonAnnals of the Entomological Society of America 2020 113 3
- Jason Roberts, Kristen Overstreet, Rachel Hendrick, Jennifer Mahar2020
- Jason Roberts, Kristen Overstreet, Rachel Hendrick, Jennifer Mahar2020
- Julie Posselt, Theresa E. Hernandez, Cynthia D. Villarreal, Aireale J. Rodgers, Lauren N. Irwin2020 35
- Diego A. Reinero, Julian A. Wills, William J. Brady, Peter Mende-Siedlecki, Jarret T. Crawford, Jay J. Van BavelPerspectives on Psychological Science 2020 15 6
- Marta DynelIntercultural Pragmatics 2020 17 5
- Harriette Gillian Christine Van Spall, Sera WhitelawEuropean Heart Journal 2020
- Steve Stewart-Williams, Lewis G HalseyEuropean Journal of Personality 2021 35 1
- Kristopher McNeill, Paige J. Novak, Peter J. VikeslandEnvironmental Science: Processes & Impacts 2020 22 1
- Kristopher McNeill, Paige J. Novak, Peter J. VikeslandEnvironmental Science: Water Research & Technology 2020 6 1
- Kristopher McNeill, Paige J. Novak, Peter J. VikeslandEnvironmental Science: Nano 2020 7 1
- Natalie Shoham, Alexandra PitmanBJPsych Advances 2020
- Flaminio Squazzoni, Giangiacomo Bravo, Mike Farjam, Ana Marusic, Bahar Mehmani, Michael Willis, Aliaksandr Birukou, Pierpaolo Dondio, Francisco GrimaldoScience Advances 2021 7 2
- Thomas C. Kwee, Hugo J. A. Adams, Robert M. KweeInsights into Imaging 2020 11 1
- Tony BaziInternational Urogynecology Journal 2020 31 3
- Emilija Stojmenova Duh, Andrej Duh, Uroš Droftina, Tim Kos, Urban Duh, Tanja Simonič Korošak, Dean KorošakPublications 2019 7 2
- Andrew Tomkins, William D. Heavlin, Min ZhangProceedings of the National Academy of Sciences 2018 115 9
- Francesco Torella, Stavros A. Antoniou, George A. AntoniouEuropean Journal of Vascular and Endovascular Surgery 2020
- Svitlana Fiialka, Olga Trishchuk, Nadija FigolKnowledge and Performance Management 2020 4 1
- Luis Ibarra, Arturo Soriano, Pedro Ponce, Arturo Molina2020
- Charles MenzieEnvironmental Toxicology and Chemistry 2019 38 1
- Zahra Bahadoran, Parvin Mirmiran, Khosrow Kashfi, Asghar GhasemiInternational Journal of Endocrinology and Metabolism 2020 In Press In Press
- Armando J. Cruz-Laufer, Tom Artois, Karen Smeets, Antoine Pariselle, Maarten P. M. VanhoveHydrobiologia 2020
- Karima Chaabna, Sohaila Cheema, Amit Abraham, Patrick Maisonneuve, Albert B. Lowenfels, Ravinder MamtaniSystematic Reviews 2021 10 1
Article usage
Citation Manager Formats
Sign up for Article Alerts
Article Classifications
- Social Sciences
- Social Sciences
- Physical Sciences
- Computer Sciences