Reply to Mislavsky et al.: Sometimes people really are averse to experiments

In response to our article (1), Mislavsky et al. (2) claim that “experiment aversion” does not exist because they found no evidence of it in their own research on low-stakes corporate experiments (3) and because our studies used between- rather than within-subjects designs.

First, as we noted, we do not expect (and did not ourselves find) an A/B effect in every scenario, and we called for research on how the effect might vary across contexts.

Second, we deliberately used a between-subjects design to maximize external validity: Universal implementation of policies usually occurs without mention of foregone alternatives, whereas A/B tests inherently acknowledge those alternatives. The belief that experiments deprive people of potentially beneficial interventions, but universally implemented policies do not, is not a “confound” to be avoided (2) but, rather, a key …

↵¹To whom correspondence may be addressed. Email: michellenmeyer{at}gmail.com.