Edited by Jose A. Scheinkman, Princeton University, Princeton, NJ, and approved April 29, 2011 (received for review February 24, 2011)
We evaluate the results of a field experiment designed to measure the effect of prompts to form implementation intentions on realized behavioral outcomes. The outcome of interest is influenza vaccination receipt at free on-site clinics offered by a large firm to its employees. All employees eligible for study participation received reminder mailings that listed the times and locations of the relevant vaccination clinics. Mailings to employees randomly assigned to the treatment conditions additionally included a prompt to write down either (i) the date the employee planned to be vaccinated or (ii) the date and time the employee planned to be vaccinated. Vaccination rates increased when these implementation intentions prompts were included in the mailing. The vaccination rate among control condition employees was 33.1%. Employees who received the prompt to write down just a date had a vaccination rate 1.5 percentage points higher than the control group, a difference that is not statistically significant. Employees who received the more specific prompt to write down both a date and a time had a 4.2 percentage point higher vaccination rate, a difference that is both statistically significant and of meaningful magnitude.
Seasonal influenza leads to >200,000 hospitalizations and >8,000 deaths in the United States each year (1, 2). The influenza vaccine is widely available at low cost and reduces mortality (3⇓–5), morbidity (3⇓⇓–6), and healthcare costs (3, 6). Nevertheless, many of those for whom vaccination is indicated fail to comply with CDC recommendations for vaccination (7). If low compliance is the result of careful calculations by individuals weighing the costs and benefits of vaccination, it may be difficult and expensive for policymakers and organizational leaders to increase vaccination rates. However, if low compliance is the result of forgetfulness or procrastination, low-cost interventions that use psychological tools may be effective at increasing vaccination rates and improving public health.
The potential for low-cost psychological interventions to change behavior has been documented in previous research (8⇓–10). For example, changing defaults—the outcomes that result when no action is taken—has been shown to have a sizeable effect on organ donation rates (11), immunization rates (12), and savings plan enrollment (13). Providing information on social norms has been used to reduce household energy consumption (14, 15). This paper evaluates another behavioral intervention—planning prompts—in a field setting.
Research in psychology has demonstrated that prompting people to develop a plan of the form, “When situation x arises, I will implement response y,” increases attainment of desired goals (16⇓⇓⇓–20). Simply asking people to develop such a plan, or an “implementation intention,” is all that is necessary to trigger an association between the desired behavior and a concrete future moment (19). A prompt to form an implementation intention is a “nudge” (8) in the direction of desired behavior that can be applied at minimal expense and does not restrict individual autonomy (9).
To investigate the efficacy of implementation intentions prompts, we conducted a three-arm randomized controlled trial. The outcome of interest was influenza vaccination receipt. Employees at a large firm were randomly assigned to receive one of three mailings about workplace vaccination clinics. All mailings informed recipients of the dates and times of clinics at their work location. In addition, some mailers prompted recipients to write down either (i) the date they planned to get their vaccination (date plan condition), or (ii) the date and time they planned to get their vaccination (time plan condition). Fig. 1 shows the component of the mailer that varied across conditions.
Experimental component of reminder mailer sent to study participants.
Comparing the vaccination rates of employees in the two treatment conditions to the vaccination rates of employees in the control condition whose mailers did not include an implementation intentions prompt, we find that vaccination rates were increasing in the specificity of the prompt received. The vaccination rate among control condition employees was 33.1%. Employees who received the prompt to write down just a date had a regression-adjusted vaccination rate 1.5 percentage points higher than the control group, a difference that is not statistically significant at the 5% level. Those who received the more specific prompt to write down both a date and a time had a 4.2 percentage point higher regression-adjusted vaccination rate, a difference that is both statistically significant at the 5% level and of a meaningful magnitude. The intervention had a larger impact at sites that offered vaccination clinics on only one day than at sites that offered clinics on multiple days, suggesting that implementation intentions prompts may be more effective in settings where an opportunity is available for only a short time.
To our knowledge, this is the first large field study to evaluate the power of implementation intentions prompts in isolation and without confounding influences. Although past research has demonstrated that planning interventions can increase the frequency of prompted behavior for various outcomes ranging from mammography to voting (19), our study differs from past research in several important ways. First, in many past studies, the effect of the planning intervention could not be distinguished from the effect of social pressure because the intervention involved face-to-face or telephone contact (16, 20⇓⇓⇓–24). This study does not involve social pressure: whether subjects ignored the mailings was not observed, and any plans formulated were not communicated to the experimenters, the employer, or any other party. Second, the treatment conditions in many previous studies have provided subjects with supplemental information that was relevant to the outcome participants were being prompted to achieve (16, 21, 22, 25). Our two treatment condition mailings did not contain any information not in the control condition mailing, allowing us to isolate the effect of the implementation intentions prompt. Third, we directly observe the outcome of interest because we have administrative data on vaccinations; many past studies use subject self-reports, which are vulnerable to reporting bias, as their outcome (21⇓⇓–24, 26). Fourth, in this study, not only were outcomes measured objectively, but participants did not know that their behavior was being observed. In previous studies, awareness of study participation could have altered behavior (27). Fifth, many past studies have much smaller sample sizes than this study (16, 21–24, 26, 28), making it difficult to precisely measure the impact of the implementation intentions interventions. Finally, many past studies have experienced high rates of attrition (22⇓⇓⇓–26, 28), whereas this study had almost no attrition.
In addition to improving our understanding of implementation intentions prompts and informing policies that promote public health, our study has implications for economic models of individual decision making. In most models where individuals fail to take an action that is in their long-run best interest, the mechanism that causes this failure is an overweighting of the immediate costs of the action (29⇓–31) or a lack of relevant information or information-processing ability (32). This study suggests that the lack of a concrete plan for implementing a desired action can also contribute to gaps between an individual's intentions and actions.
All 3,272 employees at a large Midwestern utility firm with vaccination indications—individuals 50 y of age or older or those with chronic health conditions that increase the risk of influenza-related complications (7, 33)—were randomly assigned to receive one of three mailings about the firm's on-site influenza vaccination clinics. As described in detail in Methods, more information is available about employees enrolled in the company's Preferred Provider Organization (PPO) health plan than about employees enrolled in a Health Maintenance Organization (HMO) plan. Thus, we present our results not only for these two groups together, but also separately for the PPO study subjects only.
As expected in a randomized controlled experiment, we found no significant differences in individual characteristics across the three experimental groups (Table 1). We found differences across the three groups in vaccination clinic characteristics: Participants in the date plan condition work at locations with vaccination clinics that were longer on average than participants in the control condition, both in terms of clinic days (1.6 d difference of means, 95% CI, 1.5–1.8 d) and total clinic hours (10.8 h difference of means, 95% CI, 9.4–12.1 h). These differences result from the study design which, by necessity, excludes from the date plan condition any employees who had access to only a single-day clinic (Methods). If we restrict the sample to locations with multi-day clinics, we find no differences in the clinic variables across experimental conditions at the 5% significance level.
Characteristics of the study sample
Table 2 shows the vaccination rates for the three experimental conditions and, for the two treatment conditions, the difference in the vaccination rate relative to the control condition. The vaccination rate among control condition employees in the full sample was 33.1%. In the two treatment conditions, the vaccination rate increases with the specificity of the implementation intentions prompt received by employees. Those who received the more general prompt to write down just a date—participants in the date plan condition—had a vaccination rate of 35.6%, a 2.4 percentage point increase relative to the control condition (95% CI for difference of means, −1.9 to 6.8 percentage points) that is not significant (P = 0.27). The vaccination rate of those who received the more specific prompt to write down a date and time—participants in the time plan condition—was 37.1%, a significant increase of 4.0 percentage points (95% CI for difference of means, 0.3–7.7 percentage points, P = 0.04) relative to the control condition.
Influenza vaccination rates by experimental condition
The regression-adjusted differences in treatment condition vaccination rates relative to the control condition are similar to the differences obtained without regression adjustment. The adjusted estimates control for the full sample individual characteristics in Table 1 and work location fixed effects (which subsume the clinic variables in Table 1). The regression-adjusted difference between the date plan condition and the control condition is 1.5 percentage points (95% CI, −3.0 to 6.1 percentage points, P = 0.51), whereas the regression-adjusted difference between the time plan condition and the control condition is 4.2 percentage points (95% CI, 0.5–7.8 percentage points, P = 0.03). Estimated coefficients for control variables in all of the regressions discussed in this paper are reported along with the associated SEs as Tables S1–S3.
We next turn to an examination of the subsample of employees enrolled in the PPO health plan. For this group, we report two sets of regression-adjusted differences in vaccination rates, the first controlling for work location and characteristics observed for all study participants, and the second controlling for these variables plus the characteristics observed only for the PPO subsample.
The implementation intentions intervention effects are larger for the PPO subsample than for the full sample (Table 2). Relative to the control group, assignment to the date plan condition increased the unadjusted probability of vaccination by 4.2 percentage points (95% CI, −0.6 to 9.0 percentage points, P = 0.09) and the regression-adjusted probability of vaccination by 3.9 percentage points with the smaller set of regression controls (95% CI, −1.2 to 9.0 percentage points, P = 0.14) and 5.3 percentage points with the larger set of regression controls (95% CI, 0.3–10.3 percentage points, P = 0.04). Assignment to the time plan condition increased the unadjusted probability of vaccination at a workplace clinic by 5.7 percentage points (95% CI, 1.5–9.9 percentage points, P < 0.01) and the regression-adjusted probability of vaccination by 6.2 percentage points with the smaller set of controls (95% CI, 2.1–10.4 percentage points, P < 0.01) and 6.0 percentage points with the larger set of controls (95% CI, 2.0–10.1 percentage points, P < 0.01).
In the PPO subsample, we also observe insurance claims submitted for influenza vaccinations received outside of workplace clinics. Therefore, we can examine the broader outcome of whether an employee was vaccinated at either a workplace clinic or some other location. The implementation intentions interventions increased this more expansive measure of vaccination rates, although the effects are slightly smaller than the effects on vaccination rates at just the workplace clinics. The regression-adjusted effect (using the larger set of controls) of being in the date plan condition on receiving a vaccine at any location is 4.9 percentage points (95% CI, −0.1 to 9.9 percentage points, P = 0.06), whereas the corresponding effect on receiving a vaccine at a workplace clinic is 5.3 percentage points; the regression-adjusted effect (using the larger set of controls) of being in the time plan condition on receiving a vaccine at any location is 5.5 percentage points (95% CI, 1.5–9.5 percentage points, P < 0.01), whereas the corresponding effect on receiving a workplace vaccination is 6.0 percentage points.
These results indicate that some of the workplace vaccinations induced by the implementation intentions interventions displaced vaccinations that employees would have received elsewhere. We probably measure most of this displacement, because our dataset for PPO members includes all free vaccinations (on-site) and all reimbursed vaccinations (off-site). The magnitudes of the displacement effects for the PPO members are not very large: 0.4 percentage points of the vaccinations induced by the date plan condition displaced off-site vaccinations, and 0.5 percentage points of the vaccinations induced by the time plan condition displaced off-site vaccinations.
Although unanticipated ex ante, our analyses reveal that the effect of the time plan condition mailing was substantially larger among employees working at locations that offered clinics on only one day than among those working at locations with multi-day clinics (Fig. 2). Regression-adjusted point estimates of the increase in vaccination rates induced by the time plan condition mailing at 1-d clinic sites range from 7.9 to 9.5 percentage points depending on the regression specification (Table S2). Regression-adjusted point estimates of the increase in vaccination rates induced by the time plan condition mailing at multi-day clinic sites were considerably smaller, ranging from 1.7 to 4.3 percentage points across specifications and failing to reach statistical significance (Table S3). Although this differential impact may be due to unobserved differences between employees at work locations with limited versus extensive clinic availability, the results are consistent with implementation intentions prompts being most effective for opportunities that are available for only a short time. A momentary episode of forgetfulness can cause such short-lived opportunities to be foregone altogether. Thus, the effectiveness of implementation intentions prompts may depend on subtle features of the decision making environment.
Vaccination rates by experimental condition and flu shot clinic length.
We do not observe any significant interactions between our treatment conditions and other known characteristics of the population studied (sex, age, race, marital status, parental status, flu shot received last year, visits to doctor last year, diabetes, asthma, high blood pressure, or high cholesterol).
This study shows that encouraging people to make a plan to accomplish a desired outcome can significantly increase their likelihood of success. In the context of a vaccination reminder mailing, prompting recipients to consider and write down the date and time when they planned to get their flu shot increased vaccination rates by 4 percentage points relative to a baseline vaccination rate of 33% among a control group whose reminder mailing did not include an implementation intentions prompt. The fact that the time plan condition had a larger impact at 1-d clinic sites than at multi-day clinic sites suggests that implementation intentions prompts may be most effective at encouraging behaviors when the opportunity for action is fleeting.
These findings have the potential to strengthen future efforts to improve public health by increasing vaccination rates. Past research has shown that reminder letters are among the most cost-effective ways to encourage patient immunization, increasing compliance by an average of 8 percentage points (34, 35). By comparison, our study shows that incorporating a specific element into a reminder mailing—a prompt to form an implementation intention—increases vaccination rates by 4 percentage points at no incremental printing/mailing cost. Another minimal-cost component of the mailers in all of our study arms, including the control condition—informing recipients of the time and place of the flu shot clinics—has been shown in previous research to increase immunization rates by 9 percentage points (36).
More generally, implementation intentions prompts may be an effective and low-cost way to increase a wide range of constructive actions that individuals intend but fail to execute. This set of constructive actions includes other health-related actions, such as completing a health assessment or scheduling a colonoscopy, as well as nonhealth-related actions such as purchasing life insurance, procuring a will, opening a savings account, or switching to energy-efficient light bulbs.
Our study has several limitations. The sample of predominately male employees at a single firm is not representative of the broader population. Another concern is that some participants, even in the PPO subsample, may have received influenza vaccinations that we cannot observe, although past research shows that most influenza vaccinations occur at work or a doctor's office (37). Also, our experimental design has two treatment groups and a control group, so a reader assessing the overall study should keep the two planned comparisons in mind when interpreting our statistical tests based on single comparisons. Finally, the intervention took place during the fall of 2009. We study seasonal influenza vaccinations, and vaccination against the prevailing H1N1 strain of influenza was not included in the 2009–2010 seasonal influenza vaccine. A separate vaccine against the H1N1 influenza strain was in limited supply during the fall of 2009, and news stories about both the H1N1 strain and shortages of the H1N1 vaccine were widespread. Conditions during the fall of 2009 may limit the generalizability of this study's results, although they do not contaminate the study itself, because individuals in all experimental arms faced the same background circumstances.
In conclusion, this study demonstrates that for employees at a large firm with influenza vaccine indications (age ≥50 y or chronic disease), adding a simple planning prompt to an influenza vaccination clinic reminder mailing meaningfully increased vaccination rates. These findings suggest that models of individual decision making might be improved if they recognize the role of concrete plans in the translation of intentions into actions. In addition, this study contributes to a growing literature on the value of using behavioral “nudges” to influence a wide range of individual outcomes.
The participants in this study were employees at a large Midwestern utility company. Fig. 3 shows the flow of study participants. Of the 9,029 active employees at the firm in October 2009, only those meeting the then-current CDC influenza vaccine recommendations were authorized by the employer for study inclusion: individuals 50 y of age or older and those with chronic health conditions that increase the risk of influenza-related complications (7). All 3,272 employees meeting these criteria were included in the study and received a mailing about the firm's influenza vaccination clinics.
Flow of study participants.
Before this project commenced, it was thoroughly reviewed and approved by the institutional review boards (IRBs) of the University of Pennsylvania, Harvard University, and the National Bureau of Economic Research. These IRBs determined that a waiver of informed consent was appropriate per Federal regulations [45 CFR 46.116(d)]. No identifying information about study participants was ever shared with the researchers—all identifying information was stripped from the relevant dataset by our partner organization (Evive Health) and replaced with unique random numbers.
The study firm offered free on-site seasonal influenza vaccinations to its employees in fall 2009. The firm retained Evive Health, a personalized healthcare communications provider, to send mailers to employees for whom influenza vaccination was indicated by CDC guidelines. The mailers informed employees of the timing and location of their on-site clinic and encouraged them to attend. All mailer recipients were study participants (Fig. 3).
In collaboration with Evive Health, we developed three versions of the mailer. All mailers contained information about how and why to obtain an influenza vaccine. The control condition mailer included no additional content. The two treatment condition mailers included an additional line of text: “Many people find it helpful to make a plan for getting their shot. You can write yours here:” (Fig. 1). In the time plan condition, spaces below this text indicated where recipients could write down the date and time they planned to get their vaccine. In the date plan condition, spaces indicated where recipients could write down the date (but not the time) they planned to get their vaccine. Mailings were sent simultaneously by Evive Health to all eligible employees in early October 2009.
The employees who received influenza vaccination reminder mailers were spread across 62 work locations. At each location, on-site clinics were offered on different dates between October 13 and November 16, 2009. In some locations, clinics were held for only a portion of a single day. In other locations, full-day, 3-d, or 5-d clinics were held.
Participants were stratified on the basis of work location and then randomized into the three experimental conditions. Of the 3,272 employees included in the study, 2,221 employees worked at locations with multi-day vaccination clinics. At these locations, employees were randomized evenly within work location into the three experimental conditions. For the 1,051 employees at locations with 1-d clinics, the date plan condition was not relevant, and employees were randomized evenly within work location into the control condition and the time plan condition. As a result, the number of participants in the control and time plan conditions is similar (1,268 and 1,270 participants, respectively), whereas the number of participants in the date plan condition is smaller (734 participants) (Fig. 3). All 3,272 employees assigned to an experimental condition are included in our data analysis, as we observe whether each of them received a vaccination at a workplace clinic. Further, we include all 3,272 employees in our study rather than restricting our sample to multi-day clinic sites to increase our study's power to detect an increase in vaccinations as a result of our planning prompt. Of the 3,272 employees studied, only 5 were no longer employed by the firm at the end of November 2009 (the last month when on-site clinics were held), and even in these cases we observe whether a vaccination was received at a workplace clinic before the employment relationship ended.
Differences in the health insurance elections of employees are important to note. Evive Health receives health data about all of the firm's employees enrolled in a PPO health plan, including the frequency of doctor visits and the presence of health conditions (e.g., asthma and diabetes). Evive Health does not receive this information for employees enrolled in a HMO plan. Thus, although the full set of CDC influenza vaccination criteria was used to select the 2,629 PPO members in this study (38% of whom were selected solely on the basis of their health conditions and not their age), almost all (99%) of the 643 HMO members in this study were eligible on the basis of age (Evive Health had health data on some HMO members who had formerly been covered by a PPO). Because we have more health-related information on employees enrolled in the PPO, we analyze the impact of our intervention both on the full sample of mailing recipients and on the subsample of recipients with PPO coverage.
The primary outcome of interest is receipt of a seasonal influenza vaccination at one of the firm's on-site clinics in fall 2009. Vaccination recipients were tracked at each clinic location.
We can additionally observe the influenza vaccinations of PPO members outside of the workplace clinics by using insurance claims submitted from July 2009 through April 2010. This tracking ability allows us to examine a broader outcome measure for this subgroup: receipt of an influenza vaccine at any location. With this broader outcome measure, we can also assess whether the incremental workplace vaccinations induced by the implementation intentions interventions displaced vaccinations that would have occurred elsewhere.
We evaluate the impact of the implementation intentions interventions on an intent-to-treat basis by calculating the difference in vaccination rates between the two treatment conditions and the control condition. We do this analysis on both an unadjusted basis and a regression-adjusted basis. The regression-adjusted differences are calculated by using ordinary least squares (OLS) regressions of a binary vaccination receipt indicator at the individual level on individual characteristics, work location indicator variables, and indicator variables for the individual's experimental condition. We calculate regression-adjusted differences in vaccination rates for several reasons. First, a regression framework allows us to control for demographic and clinic characteristics that may affect vaccination rates. Although controlling for demographic characteristics should not substantively affect the estimated differences in vaccination rates across experimental conditions if the assignment to conditions is random, using these controls will increase the statistical precision of the estimated differences. Second, controlling for location fixed effects allows us to parsimoniously control for an important difference between the date plan condition and the other two conditions: employees in the date plan condition worked only at locations that offered multi-day vaccination clinics, whereas the control and time plan conditions also include employees at single-day clinic locations. Finally, a regression framework allows us to more easily assess whether the treatment effects varied by demographic or clinic characteristics.
We thank Evive Health (and in particular, Prashant Srivastava and Jennifer Lindner) for collaborating on the design of the mailer, conducting the field experiment, and providing the data; and Kevin Volpp, Todd Rogers, and participants at the Carnegie Mellon University-University of Pennsylvania Fall 2010 Roybal Center Retreat for their insightful feedback. J.B., J.J.C., D.L., and B.C.M. acknowledge individual and collective financial support from National Institute on Aging Grants P30-AG-034532, R01-AG-021650, and T32-AG-000186.
Author contributions: K.L.M., J.B., J.J.C., D.L., and B.C.M. designed research; K.L.M., J.B., J.J.C., D.L., and B.C.M. performed research; K.L.M., J.B., and B.C.M. analyzed data; and K.L.M., J.B., J.J.C., D.L., and B.C.M. wrote the paper.
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1103170108/-/DCSupplemental.
Freely available online through the PNAS open access option.
