Biasing moral decisions by exploiting the dynamics of eye gaze

Significance Where people look generally reflects and reveals their moment-by-moment thought processes. This study introduces an experimental method whereby participants’ eye gaze is monitored and information about their gaze is used to change the timing of their decisions. Answers to difficult moral questions such as “Is murder justifiable?” can be influenced toward random alternatives based on looking patterns alone. We do this without presenting different arguments or response frames, as in other techniques of persuasion. Thus, the process of arriving at a moral decision is not only reflected in a participant’s eye gaze but can also be determined by it. Eye gaze is a window onto cognitive processing in tasks such as spatial memory, linguistic processing, and decision making. We present evidence that information derived from eye gaze can be used to change the course of individuals’ decisions, even when they are reasoning about high-level, moral issues. Previous studies have shown that when an experimenter actively controls what an individual sees the experimenter can affect simple decisions with alternatives of almost equal valence. Here we show that if an experimenter passively knows when individuals move their eyes the experimenter can change complex moral decisions. This causal effect is achieved by simply adjusting the timing of the decisions. We monitored participants’ eye movements during a two-alternative forced-choice task with moral questions. One option was randomly predetermined as a target. At the moment participants had fixated the target option for a set amount of time we terminated their deliberation and prompted them to choose between the two alternatives. Although participants were unaware of this gaze-contingent manipulation, their choices were systematically biased toward the target option. We conclude that even abstract moral cognition is partly constituted by interactions with the immediate environment and is likely supported by gaze-dependent decision processes. By tracking the interplay between individuals, their sensorimotor systems, and the environment, we can influence the outcome of a decision without directly manipulating the content of the information available to them.

Eye gaze is a window onto cognitive processing in tasks such as spatial memory, linguistic processing, and decision making. We present evidence that information derived from eye gaze can be used to change the course of individuals' decisions, even when they are reasoning about high-level, moral issues. Previous studies have shown that when an experimenter actively controls what an individual sees the experimenter can affect simple decisions with alternatives of almost equal valence. Here we show that if an experimenter passively knows when individuals move their eyes the experimenter can change complex moral decisions. This causal effect is achieved by simply adjusting the timing of the decisions. We monitored participants' eye movements during a two-alternative forced-choice task with moral questions. One option was randomly predetermined as a target. At the moment participants had fixated the target option for a set amount of time we terminated their deliberation and prompted them to choose between the two alternatives. Although participants were unaware of this gaze-contingent manipulation, their choices were systematically biased toward the target option. We conclude that even abstract moral cognition is partly constituted by interactions with the immediate environment and is likely supported by gaze-dependent decision processes. By tracking the interplay between individuals, their sensorimotor systems, and the environment, we can influence the outcome of a decision without directly manipulating the content of the information available to them. morality | decision making | eye tracking | visual attention | dynamical systems M oral cognition arises from the interplay between emotion and reason (1)(2)(3)(4)(5), between cultural and personal values (6), and in the competition between different cognitive representations (7)(8)(9). Many studies have explored these tensions, finding that moral decisions can be influenced by priming, highlighting, or framing one factor over another (4)(5)(6)9). Despite this, almost no attention has been devoted to how moral deliberation is played out in the very moment of choice or what effect this might have on the decision process itself. In the current experiments we focused on the temporal dynamics of moral cognition. We hypothesized that tracking the gaze of participants while they decided between two options would provide sufficient knowledge that could be exploited to influence the outcome of the moral deliberation.
Our hypothesis is derived from an understanding of human cognition that emphasizes dynamic interaction between cognition and environment through sensorimotor activation, a position supported by converging lines of evidence . Gaze patterns in humans reflect the course of reasoning during spatial indexing tasks both in adults (10,11) and in infants (12). Evidence from neural stimulation shows that saccadic programming and perceptual decisions develop together in the monkey brain (15,16). In decision tasks, before asserting their preference for faces or similarly valued snack foods people look more toward the alternative they are going to choose (17,19). For example, the attentional driftdiffusion model (aDDM) proposes a computational mechanism underlying choice whereby gaze direction biases the decision process (19,31). Similarly, studies measuring hand and eye movements show that attitudes and preferences are dynamically constructed over the course of a trial (20,29,30).
In this paper we extend the study of gaze and decision making from simple preferences to complex moral choices. Together, past research suggests that moment by moment the alternative that participants look at while making a decision will be the alternative that they are considering at that point in time. However, rather than using priming or stimuli presentation to control what participants saw or thought during their moral deliberation we controlled only when the decision was made and predicted that we could systematically influence their choices (Fig. 1).
We used an experimental paradigm where participants sat in front of a computer while we monitored their gaze using a remote eye-tracking system. Through headphones they heard statements such as "Murder is sometimes justifiable." Subsequently two response alternatives were presented simultaneously on-screen, in this case "sometimes justifiable" and "never justifiable." We instructed the participants to choose the alternative that they considered to be morally right ( Fig. 2 and Materials and Methods). We told the participants that they would view the alternatives a short but random amount of time, after which we would remove the alternatives from view and prompt them to indicate their choice. During their deliberation, participants looked freely between the two alternatives before making their choice, a design allowing us to demonstrate that gaze reflects decision-making processes even for moral choices (experiment 1). To show that knowledge of these dynamics can be exploited Significance Where people look generally reflects and reveals their momentby-moment thought processes. This study introduces an experimental method whereby participants' eye gaze is monitored and information about their gaze is used to change the timing of their decisions. Answers to difficult moral questions such as "Is murder justifiable?" can be influenced toward random alternatives based on looking patterns alone. We do this without presenting different arguments or response frames, as in other techniques of persuasion. Thus, the process of arriving at a moral decision is not only reflected in a participant's eye gaze but can also be determined by it.
to influence the decision itself, without the knowledge of participants, we timed the decision prompt to be directly dependent on their gaze distribution according to fixed rules (experiments 2 and 3).

Results
In experiment 1 we aimed to establish that eye gaze closely tracks the trajectory of an unfolding moral decision, as it does for preferences between faces and snack foods. We recruited 20 participants and let them make a series of moral choices with the rule governing the decision prompt set up as follows: As soon as one of the two alternatives had been viewed for at least 750 ms and the other had been viewed for at least 250 ms we terminated the deliberation of the participants (Fig. 2). The rule ensured that one alternative was, on average, viewed slightly more than the other but that both alternatives had been seen by the participants. Posttest interviews established that participants were unaware that their gaze had influenced the time course of the experiment and saw no links between their gaze patterns and the decision prompts during the experiment.
We found that participants chose the alternative most viewed at the time of interruption (target alternative) in 59.64% of trials [t (19) = 5.17, P < 10 −4 , d = 1.16; Fig. 3]. They also had shorter response times from decision prompt until button press when choosing the target alternative over the nontarget [mean difference (M diff ) = −0.05 s, P < 10 −5 , d = 0.67], as well as being more confident when choosing the target over the nontarget (M diff = 0.25, P < 0.01, d = 0.46). Participants' confidence and response times did not correlate (r = 0.17, P = 0.47; Fig. S1).
The results demonstrate that eye gaze reflects the current decision trajectory for moral decisions. Choices in line with the current decision trajectory were made faster and with greater certainty. We conclude that our first hypothesis was supported; eye gaze reflects the time course of developing moral choices.
If eye gaze reveals participants' location in their decisionmaking trajectory, it should reveal when participants are more likely to choose one option over the other. Therefore, by controlling when a decision is made it should be possible to influence the decision itself. Experiment 2 tested this causal hypothesis that decisions can be biased toward a randomly determined target by manipulating the timing of decision prompts alone. For this experiment we recruited a different set of 20 participants who heard the same items as before, presented with the same method as previously described, but with one key difference. At the start of each trial we randomly selected one alternative to be the target for our bias manipulation. We then altered the decision rule so that participants only would be prompted to choose once the target alternative had accumulated at least 750 ms of gaze and the nontarget alternative at least 250 ms ( Fig. 2 and Materials and Methods). Because the target was randomly allocated before the choice was made, choice distribution deviating from chance between the two options can be attributed to a causal influence of the timing of the decision prompt.
When participants were prompted to make their choice they chose the randomly predetermined target alternative in 58.21% of the trials [t (19) = 4.75, P < 0.001, d = 1.06, Fig. 3]. Participants' response times were significantly shorter when choosing the target alternative (M diff = 0.08 s, P < 0.05, d = 0.23) and their confidence was significantly higher (M diff = 0.24, P < 0.001, d = 0.31) compared with when choosing the nontarget. Participants' confidence and response times did not correlate (r = −0.32, P = 0.17; Fig. S1). Again, posttest interviews revealed that participants were unaware of any relationship between their gaze behavior and the decision prompts. Thus, the results of the second experiment allow us to conclude that the direction of eye gaze not only reflects a developing moral decision but also influences it. By monitoring the direction of eye gaze deliberation can be terminated so as to bias the unfolding choice.
Finally, in experiment 3 we aimed to replicate and further investigate the causal manipulation found in experiment 2. To be certain we interrupted participants' ongoing decision trajectories we included an option for participants to indicate their choice before the regular decision prompt. If they did not choose early their decision was prompted using the same rule as in experiment 2. Additionally, we also added a trial-based measure of perceived importance for the moral items and one of comprehension for all items, to ensure that any effects found did not arise from disinterest or lack of understanding on behalf of the participants.
We recorded 21 new participants and they responded early in 20.9% of the trials and did so after an average of 1.67 s (SD = 0.4). Confidence was higher during these fast-response trials compared with the remaining trials (M diff = 0.37, P < 0.05, d = 0.59). When the decision prompt was presented according to the decision rule in the remaining trials participants chose the target alternative 55.37% of the time [t (20) = 2.86, P < 0.01, d = 0.62; Fig. 3], thus replicating our main findings from experiment 2. Participants responded faster when choosing the target compared with the nontarget (M diff = −0.05, P < 0.01, d = 0.31) but confidence levels did not differ significantly (M diff = −0.08, P = 0.36, d = 0.15). Participants' confidence and response times did not correlate (r = 0.31, P = 0.17; Fig. S1). Taken together, this suggests that the effects on action implementation, evidenced by shorter response times to target, are fairly robust but that the effects on confidence in experiment 2 might have been driven by the faster, higher confidence trials that are part of the early response trials in experiment 3.
Centrally, there were no differences in how important the moral statement was perceived to be when choosing the target alternative compared with when the nontarget was chosen (M diff = 0.06, P = 0.47, d = 0.08). In addition, the overall level of importance ratings was high (M = 4.91, SD = 1.6, 7-point scale). We also found no differences in comprehension during trials when the target alternative was chosen and when the nontarget was chosen (M diff = 0.02, P = 0.60, d = 0.03).
To better characterize the dynamics of gaze and target choice in our experiments we report additional analyses. To understand participants' distribution of eye gaze we calculated the mean number of dwells per trial across the experiments. A dwell was Hypotheses. We hypothesized that participants' eye gaze reveals their decision process owing to general coupling between sensorimotor decision processes. By using a gaze-contingent probe and selecting when a decision is prompted the resulting choice can be biased toward a randomly predetermined option.
defined as any number of sequential fixations to the same option without a transition to the alternative option. In experiment 1 participants exhibited on average 2.45 dwells (SD = 0.8). For experiments 2 and 3 the corresponding numbers were M = 2.53 (SD = 0.8) and M = 2.63 (SD = 0.9). Analyzing the distribution of dwells we found that participants exhibited more than two dwells in around half the trials for all three experiments (Table  S1).

Discussion
We biased participants' moral decisions toward randomly set targets by manipulating the moment at which they made their choice. Our results demonstrate that gaze reveals developing preferences for moral choices and indicate that current gaze direction plays an important role for this. Crucially, this means that knowing when participants are looking at alternatives gives sufficient information to change the course of their decisions.
Not only does this extend previous work on the causal role of eye gaze in settling on a decision, but these findings are to our knowledge the first to establish such effects for moral questions.
These findings contrast with and improve on previous work in several important ways. In other studies where moral decisions have been influenced this has typically been achieved through framing effects or affective manipulations (4)(5)(6)9). By contrast, we held the inputs to the choice constant and influenced it by measuring concurrent gaze position. Generalizing, our method suggests a broad route toward studying choices as they unfold in real time by interrupting decision makers based on their gaze. For example, a possible application could be to monitor real-time Fig. 2. Trial design. Participants first hear a moral statement being read out loud while viewing a central fixation point. When the statement has been read completely two alternatives appear on the screen randomly assigned to the left or right position. During this portion of the trial the participants' gaze is monitored by a remote eye tracker. Participants view the alternatives until their choice is prompted, either by fulfilling the experiment-specific conditions or 3,000-ms passes. Participants indicate which alternative they choose by clicking the right or left mouse button, respectively. Finally, a 7-point continuous confidence scale is presented. There is a 1,000-ms pause between participants' last response and the start of the next trial. Experiment 1: While participants view both alternatives their gaze is being tracked; once one alternative has accumulated at least 750 ms of gaze and the other at least 250 ms the decision prompt is activated and the trial is interrupted. Whatever alternative has the most accumulated gaze time at the time of interruption is designated the target. Experiment 2: The target is randomly determined before trial onset. Participants' gaze is measured while they view alternatives and once the target alternative has been viewed for at least 750 ms and the other alternative has been viewed for at least 250 ms the decision prompt is activated and the trial is interrupted. shopping behavior using portable eyewear to carefully time interventions on consumers. Previous research has shown that judgments and decisions between simple concrete alternatives, in particular faces and foodstuffs, can be influenced by manipulating saliency, attention, or exposure in various ways (17,18,32,33). Within this tradition, it has been hypothesized that eye gaze plays the role of tracking how evidence for a choice is accumulated (17)(18)(19)(20). However, previous studies have concerned perceptual domains where competing visual representations dominate our decision space. In our experiments participants are confronted with stimuli containing almost no evidence for them to sample. This suggests that if the participants are accumulating evidence by sampling, then they must be sampling evidence that is not in front of them, and to which they need not attend. It is, thus, striking that participants' gaze nevertheless reveals their decision process in our experiments, indicating how supposedly abstract moral values are intertwined with sensorimotor mechanisms.
In experiment 3 we also measured the perceived importance of the moral issues after each choice. The type of questions we used concerned virtues and vices, empathy and generosity, questions about god, sex, and death; the kind of questions that can drive elections and divide nations. The moral statements were derived from moral foundations theory (MFT), which defines five categories underpinning a broad and cross-culturally valid conception of morality (6,34,35). Consequently, the issues were meaningful and engaging to the participants, and the overall rated level of importance was notably high, at 4.91 on a 7-point scale. The trials in which the target was chosen were judged as being equally important as trials when the nontarget was chosen. This can be compared with prior studies that concerned simple preferential decisions, and where all stimuli where carefully controlled to represent near-equal valence between the alternatives (17,18).
Our results have important implications for how to study moral cognition as well as supporting specific moral theories. Many accounts of morality emphasize its uniqueness and the distinctness of the moral domain (36)(37)(38). According to such theories, moral choices are primarily seen as products of specific morally motivated intuitive and rule-governed processes. Our findings instead support alternative views that stress the continuities between the moral and other choice domains (7,8,39,40). If moral decisions are sensitive to sensorimotor interactions and the complex timing of these, then moral cognition might turn out to have more in common with how motor plans are dynamically updated, compared with following a rule, than previously hypothesized. Second, in our experiments the fact that participants were unaware of the manipulations and rated target and nontarget choices as equally important is in line with predictions from models that emphasize the primacy of intuitive responses and the confabulatory, post hoc nature of moral reasoning (1,41). Such a mechanism would be adaptive given that many moral decisions in the wild would have to be performed in response to developing situations and likely based on complex social demands (42). Third, recently, promising alternative theories have been proposed for how to develop computational models of moral cognition, including dynamical models of social evaluation (43) and a reconceptualization of influential dualprocess accounts in terms of reinforcement learning (44,45). However, these proposals remain vague about the precise computational and algorithmic real-time underpinnings of moral choices. By linking the temporal unfolding of moral decisions with eye gaze, our results suggest that considering gaze as part and parcel of the moral mind will improve our understanding of how moral choices are made in the moment.
Looking beyond the specifics of moral theory, an important question is what implications our results have for general models of decision making. Several frameworks and models have been proposed where attentional processes affect the speed and content of decisions (8,10,31,46,47). Of these, the aDDM model (18,19,31) makes the most detailed and strongest claim about eye gaze. It posits that decisions are driven by a diffusion process (48,49), whereby evidence is stochastically accumulated over time. Importantly, in the model, the speed of integration is modulated by the direction of gaze. Hence, the aDDM predicts that participants are more likely to choose options that they have gazed at longer, or to which their final fixation was directedpredictions that are in line with our findings. However, we find a very strong effect of final fixation compared with that of overall exposure, which indicates a strong recency effect of gaze on preferences. It is therefore possible that a computational model featuring inhibition between multiple accumulators or a leaky accumulation process might be necessary to fully capture our results (50,51). In our study, gaze likely plays a dual role of directing participants' attention toward currently considered options while also, through a complex process of mutual feedback and inhibition, affecting the current preferential state (10,11,21,26,46,(50)(51)(52)(53). In simple words, we find that even for moral choices we end up preferring what we fixate. Future work should investigate and model the computational properties underlying moral decisions directly to facilitate the necessary model comparisons. Regardless, our results demonstrate that moral choices, by virtue of their embodied dynamics, fall within the scope of traditional accounts of the computational processes underlying human decisions. This suggests future prospects for developing domaingeneral decision models covering choices from deciding the direction of randomly moving dots to the legitimacy of euthanasia. The method for studying choices proposed in this paper extends itself naturally to testing processing assumptions in decision models. For example, it is possible to directly test predictions about the value of single integrating, or multiple racing, accumulators by incorporating such computations into the trigger rule used to determine when a given trial ends. This gives new tools for testing the fit of different models on a trial-by-trial basis; such models are usually fit to best explain aggregate data. Doing so would further extend the power of current and future models in understanding human decision making, both moral and nonmoral.
This research demonstrates that moral choices are no different from their preferential and perceptual counterparts; they are highly constrained and coupled to the immediate environment through sensory interaction. The same general perceptionaction loop process (21,22) that allows us to update motor plans based on the latest information in the environment for grasping a mug or for walking through a crowd might also play an important role for making moral choices. Although such moral decisions can be debated at leisure after the fact, they are often made in the moment. We find that the precise timing of those moments can be a powerful influence on the choices that we make.

Materials and Methods
Participants were recruited through the student-based and public subject pools at University College London. Different participants were recruited for each of experiments 1 through 3. Twenty persons (10 female) participated in experiment 1, with a mean age of 27.2 (SD = 8.1). Twenty persons (14 female, mean age 29.6, SD = 13.1) participated in experiment 2. Twenty-one persons (17 female, mean age 21.8, SD = 5.4) participated in experiment 3. All participants were naïve to the research purpose and provided written consent after a full debriefing. For all of the experiments reported in this paper participants sat in front of a 19-inch screen with a resolution of 1,680 × 1,050 pixels, at ∼50 cm distance from the screen. Participants were wearing headphones throughout the experiments and interacted with the experiment script using a mouse held in their preferred hand. Eye movement data were gathered using an SMI RED 250 running on a dedicated computer system. A 5-point calibration was performed on each subject at the start of each experiment followed by a 4-point validation procedure. Calibrations with errors exceeding 1°visual angle were rerun. Average error was less than 0.5°. For the gaze-activated trigger to work, the experimental script sampled the eye tracker for updates of gaze position every 10 ms.
A total of 63 moral statements were used for the experiments. The moral statements were derived from the five categories of MFT. In addition some statements were used that reflected metaethical questions, forming a sixth category. Example statements are "Hurting a defenseless animal is one of the worst things one can do," with "it's always bad" and "it's sometimes bad" as alternatives, and "It is important to be a team player, even if that means censoring oneself at times," with "team goes first" and "I go first" as alternatives (Table S2). Alternatives were constructed so that the exact content in one alternative would be difficult to predict from only viewing one alternative, because variation has been shown to facilitate explorative behavior in other domains (54). In addition, for variation participants also responded to 35 filler items, simple true/false decisions based on previous work (29). All 98 stimulus items were used in all of the experiments reported, and item presentation order was random. No items were removed during analysis.
Participants sat in front of a computer and wore headphones. Following calibration on-screen, instructions were displayed. Participants were told that they would respond to a series of moral and factual statements and were asked to pick the alternative they thought was right in relation to the statement they heard. They were told that alternatives would be shown for a brief and variable time period on the screen and it was stressed that they should look at both alternatives. Once prompted to indicate their choice, participants were instructed to respond quickly. After participants had read the instructions the experiment would start and run without interruption for 98 trials. Each trial began with the display of a central fixation point while a statement was read out. Moral and factual statements were presented randomly. Once the recording had finished playing the two alternatives would be presented on-screen. One alternative was presented on the left side of the screen and the other on the right. Placement of alternatives was randomized. When the alternatives appeared participants' eye movements were recorded and sampled to determine when the decision prompt would appear. The alternatives were visible for a maximum of 3,000 ms to prevent participants' accidentally detecting the gaze-contingent nature of the decision prompts by simply maintaining fixation on one half of the screen for a prolonged period. Because during such trials participants had not exhibited the gaze behavior that could trigger the decision prompt those trials were not analyzed further (but see below). Once prompted, participants indicated their choice by button press. Finally, a continuous 1-7 confidence scale was displayed. In experiment 3 two additional scales for importance and comprehension were also displayed.
The precise mechanism and timing of the decision prompt varied between the three experiments (Fig. 2). In experiment 1, the participants' decision would be prompted whenever one alternative had been viewed for at least 750 ms and the other alternative for at least 250 ms. We retrospectively designated the alternative looked at the longest at the time of interruption as being the target. The key difference between the first experiment and the second and third is that in the latter experiments the target alternative was randomly predetermined at the start of each trial. The rule governing the decision prompt was changed correspondingly so that participants' decisions would be prompted once the target had accumulated at least 750 ms of gaze and the nontarget at least 250 ms of gaze. Additionally, in experiment 3 the participants could circumvent the decision prompt by giving a response while viewing the alternatives. All participants used this option for the factual items but three participants did not do this for the moral items.
After the experiment was concluded participants were asked what they thought about the experiment. They were then asked whether they found anything odd about the timing of the viewing of alternatives. Finally, they were asked whether they had any thoughts on the purpose of the eye tracker in the experiment. These questions were asked to determine whether participants had any awareness of the relation between their eye movements and termination of the alternative display. No participants voiced any such awareness or suspicion. The structure of this postexperiment interview followed established procedures in the literature (55). Following this awareness check, participants were fully debriefed and given the opportunity to voice any concerns.
In all three experiments the setup of the decision prompt allowed the participants to engage with the stimulus in any way desired, which in turn led to some participants' directing their eye gaze toward only one of the alternatives. We believed the participants would be too likely to detect the manipulation had there not been a trial limit. In addition, the eye tracker would at times momentarily lose track of participants' gaze. In both cases, the gaze-based decision prompt would not activate, and the participants would simply be asked for their answers after 3,000 ms. In such cases, participants' behavior could not be captured by our decision-prompt rules and such trials were therefore discarded from further analysis [n = 52 (4.1%) in experiment 1, n = 200 (15.8%) in experiment 2, n = 131 (9.9%) in experiment 3].
However, our main conclusions are not affected by the removal of these trials; in experiment 1, counting all trials, the target alternative was chosen in 60.8% of trials [t (19) = 6.14, P < 10 −5 , d = 1.37]. In experiment 2 the corresponding rates are 53.8% [t (19) = 2.70, P < 0.05, d = 0.60] and in experiment 3 53.3% [t (20) = 2.27, P < 0.05, d = 0.49]. In experiments 2 and 3, for the removed trials, participants choose the most exposed alternative 80.1% and 74.0% of the time, respectively. These data further support the general conclusions that moral decisions are reflected and shaped by eye gaze. See SI Materials and Methods for additional discussion.