Skip to main content
  • Submit
  • About
    • Editorial Board
    • PNAS Staff
    • FAQ
    • Accessibility Statement
    • Rights and Permissions
    • Site Map
  • Contact
  • Journal Club
  • Subscribe
    • Subscription Rates
    • Subscriptions FAQ
    • Open Access
    • Recommend PNAS to Your Librarian
  • Log in
  • My Cart

Main menu

  • Home
  • Articles
    • Current
    • Latest Articles
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • Archive
  • Front Matter
  • News
    • For the Press
    • Highlights from Latest Articles
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Purpose and Scope
    • Editorial and Journal Policies
    • Submission Procedures
    • For Reviewers
    • Author FAQ
  • Submit
  • About
    • Editorial Board
    • PNAS Staff
    • FAQ
    • Accessibility Statement
    • Rights and Permissions
    • Site Map
  • Contact
  • Journal Club
  • Subscribe
    • Subscription Rates
    • Subscriptions FAQ
    • Open Access
    • Recommend PNAS to Your Librarian

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Home
Home

Advanced Search

  • Home
  • Articles
    • Current
    • Latest Articles
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • Archive
  • Front Matter
  • News
    • For the Press
    • Highlights from Latest Articles
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Purpose and Scope
    • Editorial and Journal Policies
    • Submission Procedures
    • For Reviewers
    • Author FAQ

New Research In

Physical Sciences

Featured Portals

  • Physics
  • Chemistry
  • Sustainability Science

Articles by Topic

  • Applied Mathematics
  • Applied Physical Sciences
  • Astronomy
  • Computer Sciences
  • Earth, Atmospheric, and Planetary Sciences
  • Engineering
  • Environmental Sciences
  • Mathematics
  • Statistics

Social Sciences

Featured Portals

  • Anthropology
  • Sustainability Science

Articles by Topic

  • Economic Sciences
  • Environmental Sciences
  • Political Sciences
  • Psychological and Cognitive Sciences
  • Social Sciences

Biological Sciences

Featured Portals

  • Sustainability Science

Articles by Topic

  • Agricultural Sciences
  • Anthropology
  • Applied Biological Sciences
  • Biochemistry
  • Biophysics and Computational Biology
  • Cell Biology
  • Developmental Biology
  • Ecology
  • Environmental Sciences
  • Evolution
  • Genetics
  • Immunology and Inflammation
  • Medical Sciences
  • Microbiology
  • Neuroscience
  • Pharmacology
  • Physiology
  • Plant Biology
  • Population Biology
  • Psychological and Cognitive Sciences
  • Sustainability Science
  • Systems Biology
Research Article

Uncertainty in learning, choice, and visual fixation

View ORCID ProfileHrvoje Stojić, Jacob L. Orquin, View ORCID ProfilePeter Dayan, Raymond J. Dolan, and View ORCID ProfileMaarten Speekenbrink
PNAS first published January 24, 2020 https://doi.org/10.1073/pnas.1911348117
Hrvoje Stojić
aMax Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Hrvoje Stojić
  • For correspondence: h.stojic@ucl.ac.uk
Jacob L. Orquin
bDepartment of Management/MAPP, Aarhus University, Aarhus 8210, Denmark;cCentre for Research in Marketing and Consumer Psychology, Reykjavik University, 101 Reykjavik, Iceland;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peter Dayan
dDepartment of Computational Neuroscience, Max Planck Institute for Biological Cybernetics, Tübingen 72076, Germany;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Peter Dayan
Raymond J. Dolan
aMax Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom;
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Maarten Speekenbrink
eDepartment of Experimental Psychology, University College London, London WC1H 0AP, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Maarten Speekenbrink
  1. Edited by Terrence J. Sejnowski, Salk Institute for Biological Studies, La Jolla, CA, and approved December 27, 2019 (received for review July 11, 2019)

  • Article
  • Figures & SI
  • Info & Metrics
  • PDF
Loading

This article requires a subscription to view the full text. If you have a subscription you may use the login form below to view the article. Access to this article can also be purchased.

Significance

Humans cannot help but turn their gaze to objects that catch their attention. Our knowledge of the factors that govern this seizure, or of its effects in the context of learned decision making, is currently rather incomplete. We therefore monitored the gaze of human subjects as they learned to choose between multiple options whose value was initially unknown. We found evidence that attention was influenced by uncertainty and that the use of, and reduction in, uncertainty were, in turn, influenced by attention. Our findings provide evidence for approximately optimal models of learning and choice and uncover an intricate interplay between learning, choice, and attentional processes.

Abstract

Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between trial-by-trial values, uncertainty, learning, and choice. To gain additional insight into learning and choice processes, we obtained data from subjects’ overt allocation of gaze. The estimated value and estimation uncertainty of options influenced what subjects looked at before choosing; these same quantities also influenced choice, as additionally did fixation itself. A momentary measure of uncertainty in the form of absolute prediction errors determined how long participants looked at the obtained outcomes. These findings affirm the importance of uncertainty in multiple facets of behavior and help delineate its effects on decision making.

  • reinforcement learning
  • decision making
  • uncertainty
  • visual fixation
  • exploration–exploitation

Footnotes

  • ↵1To whom correspondence may be addressed. Email: h.stojic{at}ucl.ac.uk.
  • ↵2R.J.D. and M.S. contributed equally to this work.

  • Author contributions: H.S., J.L.O., P.D., and M.S. designed research; H.S. and J.L.O. performed research; H.S. analyzed data; H.S., J.L.O., P.D., and M.S. interpreted the results; H.S. prepared the paper; and H.S., J.L.O., P.D., R.J.D., and M.S. wrote the paper.

  • The authors declare no competing interest.

  • This article is a PNAS Direct Submission.

  • Data deposition: The data, code used for our analyses, and other project-related files are publicly available at the Open Science Framework website: https://osf.io/539ps/.

  • This article contains supporting information online at https://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1911348117/-/DCSupplemental.

Published under the PNAS license.

View Full Text

Log in using your username and password

Forgot your user name or password?

Purchase access

You may purchase access to this article. This will require you to create an account if you don't already have one.

Subscribers, for more details, please visit our Subscriptions FAQ.

Please click here to log into the PNAS submission website.

PreviousNext
Back to top
Article Alerts
Email Article

Thank you for your interest in spreading the word on PNAS.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Uncertainty in learning, choice, and visual fixation
(Your Name) has sent you a message from PNAS
(Your Name) thought you would like to see the PNAS web site.
Citation Tools
Uncertainty in learning, choice, and visual fixation
Hrvoje Stojić, Jacob L. Orquin, Peter Dayan, Raymond J. Dolan, Maarten Speekenbrink
Proceedings of the National Academy of Sciences Jan 2020, 201911348; DOI: 10.1073/pnas.1911348117

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Request Permissions
Share
Uncertainty in learning, choice, and visual fixation
Hrvoje Stojić, Jacob L. Orquin, Peter Dayan, Raymond J. Dolan, Maarten Speekenbrink
Proceedings of the National Academy of Sciences Jan 2020, 201911348; DOI: 10.1073/pnas.1911348117
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Mendeley logo Mendeley

Submit

Sign up for Article Alerts

Article Classifications

  • Biological Sciences
  • Psychological and Cognitive Sciences
  • Social Sciences
  • Psychological and Cognitive Sciences

Jump to section

  • Article
    • Abstract
    • Results
    • Discussion
    • Materials and Methods
    • Acknowledgments
    • Footnotes
    • References
  • Figures & SI
  • Info & Metrics
  • PDF

You May Also be Interested in

Efforts are underway to exploit a strategy that could generate fusion with relative ease. Image credit: Princeton Plasma Physics Laboratory.
News Feature: Small-scale fusion tackles energy, space applications
Efforts are underway to exploit a strategy that could generate fusion with relative ease.
Image credit: Princeton Plasma Physics Laboratory.
A deep-learning algorithm could potentially improve diagnosis and classification of neurological abnormalities. Image courtesy of Weicheng Kuo, Christian Hӓne, Pratik Mukherjee, Jitendra Malik, and Esther Lim Yuh
Brain hemorrhage detection by artificial neural network
A deep-learning algorithm could potentially improve diagnosis and classification of neurological abnormalities.
Image courtesy of Weicheng Kuo, Christian Hӓne, Pratik Mukherjee, Jitendra Malik, and Esther L. Yuh.
A study finds a shift in onset of El Niño events from eastern to western Pacific and increased frequency of extreme El Niño events since the late 1970s. Image courtesy of NOAA National Environmental Satellite, Data, and Information Service (NESDIS).
Changing El Niño properties
A study finds a shift in onset of El Niño events from eastern to western Pacific and increased frequency of extreme El Niño events since the late 1970s.
Image courtesy of NOAA National Environmental Satellite, Data, and Information Service (NESDIS).
A study explores how various types of food affect both human health and the environment. Image courtesy of Pixabay/esigie.
Environmental and health impacts of food
A study explores how various types of food affect both human health and the environment.
Image courtesy of Pixabay/esigie.
Profile of NAS member and molecular biologist Mary Lou Guerinot. Image courtesy of Olga Zhaxybayeva (Dartmouth College, Hanover, NH).
Featured Profile
Profile of NAS member and molecular biologist Mary Lou Guerinot
Image courtesy of Olga Zhaxybayeva (Dartmouth College, Hanover, NH).

Similar Articles

Site Logo
Powered by HighWire
  • Submit Manuscript
  • Twitter
  • Facebook
  • RSS Feeds
  • Email Alerts

Articles

  • Current Issue
  • Latest Articles
  • Archive

PNAS Portals

  • Classics
  • Front Matter
  • Teaching Resources
  • Anthropology
  • Chemistry
  • Physics
  • Sustainability Science

Information

  • Authors
  • Editorial Board
  • Reviewers
  • Press
  • Site Map
  • PNAS Updates

Feedback    Privacy/Legal

Copyright © 2020 National Academy of Sciences. Online ISSN 1091-6490