Adversarial vulnerabilities of human decision-making

Edited by James L. McClelland, Stanford University, Stanford, CA, and approved October 3, 2020 (received for review August 10, 2020)
November 4, 2020
117 (46) 29221-29228

Significance

“What I cannot efficiently break, I cannot understand.” Understanding the vulnerabilities of human choice processes allows us to detect and potentially avoid adversarial attacks. We develop a general framework for creating adversaries for human decision-making. The framework is based on recent developments in deep reinforcement learning models and recurrent neural networks and can in principle be applied to any decision-making task and adversarial objective. We show the performance of the framework in three tasks involving choice, response inhibition, and social decision-making. In all of the cases the framework was successful in its adversarial attack. Furthermore, we show various ways to interpret the models to provide insights into the exploitability of human choice.

Abstract

Adversarial examples are carefully crafted input patterns that are surprisingly poorly classified by artificial and/or natural neural networks. Here we examine adversarial vulnerabilities in the processes responsible for learning and choice in humans. Building upon recent recurrent neural network models of choice processes, we propose a general framework for generating adversarial opponents that can shape the choices of individuals in particular decision-making tasks toward the behavioral patterns desired by the adversary. We show the efficacy of the framework through three experiments involving action selection, response inhibition, and social decision-making. We further investigate the strategy used by the adversary in order to gain insights into the vulnerabilities of human choice. The framework may find applications across behavioral sciences in helping detect and avoid flawed choice.

Continue Reading

Data Availability

Anonymized csv data have been deposited in GitHub (https://github.com/adezfouli/decision_adv).

Acknowledgments

We are grateful to Yonatan Loewenstein for discussions. P.D. was funded by the Max Planck Society and the Humboldt Foundation. This research was funded partially by CSIRO’s Machine Learning and Artificial Intelligence Future Science Platform.

Supporting Information

Appendix (PDF)

References

1
O. Dan, Y. Loewenstein, From choice architecture to choice engineering. Nat. Commun. 10, 2808 (2019).
2
A. Dezfouli, R. W. Morris, F. Ramos, P. Dayan, B. W. Balleine, “Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models” in Advances in Neural Processing Systems 31, S. Bengio et al., Eds. (Curran, Red Hook, NY, 2018), pp. 4233–4242.
3
A. Dezfouli et al., “Disentangled behavioral representations” in Advances in Neural Processing Systems 32, H. Wallach et al., Eds. (Curran, Red Hook, NY, 2019), pp. 2243–2252.
4
A. Dezfouli, K. Griffiths, F. Ramos, P. Dayan, B. W. Balleine, Models that learn how humans learn: The case of decision-making and its disorders. PLoS Comput. Biol. 15, e1006903 (2019).
5
C. Szegedy et al., Intriguing properties of neural networks. arXiv:1312.6199 (21 December 2013).
6
Y. C. Lin et al., Tactics of adversarial attack on deep reinforcement learning agents. arXiv:1703.06748 (8 March 2017).
7
K. S. Jun, L. Li, Y. Ma, J. Zhu, “Adversarial attacks on stochastic bandits” in Advances in Neural Processing Systems 31, S. Bengio et al., Eds. (Curran, Red Hook, NY, 2018), pp. 3644–3653.
8
I. W. Eisenberg et al., Uncovering the structure of self-regulation through data-driven ontology discovery. Nat. Commun. 10, 2319 (2019).
9
B. King-Casas et al., Getting to know you: Reputation and trust in a two-person economic exchange. Science 308, 78–83 (2005).
10
V. Mnih et al., “Asynchronous methods for deep reinforcement learning” in Proceedings of the 33rd International Conference on Machine Learning, M. F. Balcan, K. Q. Weinberger, Eds. (Curran, Red Hook, NY, 2016), Vol. 48, pp. 1928–1937.
11
V. Mnih et al., Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015).
12
K. A. McCabe, M. L. Rigdon, V. L. Smith, Positive reciprocity and intentions in trust games. J. Econ. Behav. Organ. 52, 267–275 (2003).
13
A. Hula, P. R. Montague, P. Dayan, Monte Carlo planning method estimates planning horizons during interactive social exchange. PLoS Comput. Biol. 11, e1004254 (2015).
14
A. Tversky, D. Kahneman, Judgment under uncertainty: Heuristics and biases. Science 185, 1124–1131 (1974).
15
S. Lange, T. Gabel, M. Riedmiller, “Batch reinforcement learning” in Reinforcement Learning, M. Wiering, M. van Otterlo, Eds. (Adaptation, Learning, and Optimization Series, Springer, 2012), Vol. 12, pp. 45–73.
16
J. X. Wang et al., Learning to reinforcement learn. arXiv:1611.05763 (17 November 2016).
17
Y. Duan et al., RL2: Fast reinforcement learning via slow reinforcement learning. arXiv:1611.02779 (9 November 2016).
18
J. H. Bak, J. Y. Choi, A. Akrami, I. Witten, J. W. Pillow, “Adaptive optimal training of animal behavior” in Advances in Neural Processing Systems 29, D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, R. Garnett, Eds. (Curran, Red Hook, NY, 2016), pp. 1947–1955.
19
S. Fujimoto, H. Van Hoof, D. Meger, “Addressing function approximation error in actor-critic methods” in Proceedings of the 35th International Conference on Machine Learning, J. Dy, A. Drause, Eds.(Curran, Red Hook, NY, 2018), Vol. 80, pp. 1582–1591.
20
Z. Cranko et al., “Monge blunts Bayes: Hardness results for adversarial training” in Proceedings of the 36th International Conference on Machine Learning (Curran, Red Hook, NY, 2019), Vol. 81, pp. 2523–2543.
21
D. P. Kingma, J. Ba, Adam: A method for stochastic optimization. arXiv:1412.6980 (22 December 2014).
22
M. Abadi et al., TensorFlow: Large-Scale machine learning on heterogeneous systems. arXiv:1603.04467 (14 March 2015).
23
K. Cho et al., Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078 (3 June 2014).
24
Z. Ahmed, N. Le Roux, M. Norouzi, D. Schuurmans, “Understanding the impact of entropy on policy optimization” in Proceedings of the 36th International Conference on Machine Learning (Curran, Red Hook, NY, 2019), Vol. 81, pp. 151–160.

Information & Authors

Information

Published in

Go to Proceedings of the National Academy of Sciences
Go to Proceedings of the National Academy of Sciences
Proceedings of the National Academy of Sciences
Vol. 117 | No. 46
November 17, 2020
PubMed: 33148802

Classifications

Data Availability

Anonymized csv data have been deposited in GitHub (https://github.com/adezfouli/decision_adv).

Submission history

Published online: November 4, 2020
Published in issue: November 17, 2020

Keywords

  1. decision-making
  2. recurrent neural networks
  3. reinforcement learning

Acknowledgments

We are grateful to Yonatan Loewenstein for discussions. P.D. was funded by the Max Planck Society and the Humboldt Foundation. This research was funded partially by CSIRO’s Machine Learning and Artificial Intelligence Future Science Platform.

Notes

This article is a PNAS Direct Submission.
*For clarity, we refer throughout to decisions in the task as actions and the output of the adversary as adversarial choices.

Authors

Affiliations

Amir Dezfouli1 [email protected]
Data61, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Eveleigh, NSW 2015, Australia;
Richard Nock
Data61, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Eveleigh, NSW 2015, Australia;
Australian National University, Canberra, ACT 0200, Australia;
Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany;
University of Tübingen, 72074 Tübingen, Germany

Notes

1
To whom correspondence may be addressed. Email: [email protected].
Author contributions: A.D., R.N., and P.D. designed research; A.D. and P.D. performed research; A.D. analyzed data; and A.D., R.N., and P.D. wrote the paper.

Competing Interests

The authors declare no competing interest.

Metrics & Citations

Metrics

Note: The article usage is presented with a three- to four-day delay and will update daily once available. Due to ths delay, usage data will not appear immediately following publication. Citation information is sourced from Crossref Cited-by service.


Citation statements

Altmetrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited by

    Loading...

    View Options

    View options

    PDF format

    Download this article as a PDF file

    DOWNLOAD PDF

    Get Access

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Personal login Institutional Login

    Recommend to a librarian

    Recommend PNAS to a Librarian

    Purchase options

    Purchase this article to get full access to it.

    Single Article Purchase

    Adversarial vulnerabilities of human decision-making
    Proceedings of the National Academy of Sciences
    • Vol. 117
    • No. 46
    • pp. 28535-29242

    Media

    Figures

    Tables

    Other

    Share

    Share

    Share article link

    Share on social media