Screening p-hackers: Dissemination noise as bait

Consider an environment where each unit of observation is associated with an outcome $Y$ and a set $A$ of potential causal covariates ${(X^a)}_{a\in A}$. The outcome variable and all the covariates are binary. Suppose the dataset is wide, so the set of potential causes for the outcome is large relative to the number of observations. In fact, we assume a continuum of covariates; so $A=[0,1]$. For instance, the covariates may indicate the presence or absence of different SNPs in a person’s genetic sequence, while the outcome refers to the presence or absence of a certain disease.

There is one covariate $a^{\ast }\in A$, the true cause, with $\mathbb{P}[X^{a^{\ast }}=Y]=\psi$ for some $\psi \in (1/2,1]$. So the true cause is positively correlated with the outcome, but it may not be perfectly correlated. For instance, $a^{\ast }$ is the one SNP that causes the disease in question. There is also a red herring covariate $a^r\in A$ that is independent of $Y$. The red herring represents a theoretically plausible mechanism for the outcome $Y$ that can only be disproved with data. For instance, $a^r$ might be a SNP that seems as likely to cause the disease as $a^{\ast }$ based on a biological theory about the roles of different SNPs.

Nature draws the true cause $a^{\ast }$ and the red herring $a^r$, independently and uniformly from $A$. Then Nature generates the raw dataset $(Y_n,{(X_n^a)}_{a\in [0,1]})$ for observations $1\le n\le N$. First, the values of the true cause in the $N$ observations ${(X_n^{a^{\ast }})}_{1\le n\le N}$ are generated independently, each equally likely to be 0 or 1. Then, each $Y_n$ is generated to match $X_n^{a^{\ast }}$ with probability $\psi$ for $1\le n\le N$, independently across $n.$ Finally, covariates $X_n^a$ for $a\ne a^{\ast }$, $1\le n\le N$ are generated, each equally likely to be 0 or 1, independent of each other and of all other random variables. (So there is a continuum of independent Bernoulli random variables.) Equivalently, once $a^{\ast }$ and $a^r$ are drawn, we have fixed a joint distribution between $Y$ and the covariates ${(X^a)}_{a\in A}$, and the raw dataset consists of $N$ independent draws from this joint distribution. For instance, this may represent a dataset that shows the complete genetic sequences of $N$ individuals and whether each person suffers from the disease.

There are three players in the model: a principal, an agent, and a policymaker. The principal owns the raw dataset, but lacks the ability to analyze the data and cannot influence policy-making norms. The principal disseminates a noisy version of the dataset, which we describe below. The agent uses the disseminated data to propose a covariate, $\widehat{a}$. Finally, a policymaker evaluates the agent’s proposal on the raw dataset using an exogenous test. We think of the agent as proposing an intervention: If this proposal passes, the policymaker will implement a policy that changes $X^{\widehat{a}}$ in order to affect the value of $Y$. In the background, we implicitly assume that the principal grants the policymaker access to the raw data to conduct the test. [Alternatively, an earlier version of this paper (11) supposes that the principal periodically publishes noisy versions of the raw data for these tests. Such data releases will diminish the principal’s ability to screen out $p$-hackers in the future.]

The policymaker’s role is mechanical, and restricted to deciding whether the agent’s proposal passes an exogenous test. We say that the proposal $a$passes if the covariate $X^a$ equals the outcome $Y$ in $M=⌊\gamma ·N⌋$ out of $N$ observations, and that it fails otherwise. The parameter $\gamma$ is an exogenous passing threshold with $1/2<\gamma <\psi$. The policymaker will adopt a policy proposal if and only if it passes the test on the raw data. Passing the test does not require $a$ to be the true cause of $Y$, for we could have some covariate $a\ne a^{\ast }$ where $Y_n=X_n^a$ for at least $M$ observations by random chance.^‡

The agent is either a maven (with probability $1-h$) or a hacker (with probability $h$). Mavens and hackers differ in their expertise. A maven knows that the true cause is either $a^{\ast }$ or $a^r$, and assigns them equal probabilities, but a hacker is ignorant about the realizations of $a^{\ast }$ and $a^r$. The idea is that a maven uses domain knowledge (e.g., biological theory about the disease $Y$) to narrow down the true cause to the set $\{a^{\ast },a^r\}$. A hacker, by contrast, is completely uninformed about the mechanism causing $Y$.

The agent’s payoffs reflect both a desire for reporting the true cause and a desire for policy impact. If a type $\theta$ agent proposes $a$ when the true cause is $a^{\ast }$, then his payoff is

Here, we interpret ${\mathbf{1}}_{\{a=a^{\ast }\}}$ as the effect of proposing $a$ on the agent’s long-run reputation when the true cause $a^{\ast }$ of the outcome $Y$ eventually becomes known. The other summand models the agent’s gain from proposing a policy that passes the policymaker’s test and gets implemented. The relative weight $w_{\theta }\in [0,1]$ on these two components may differ for the two types of agent. Our main results in this section are valid for any values of $w_{\text{maven}}$ and $w_{\text{hacker}}$ in $[0,1]$, but some later results in Section 2 will require restrictions on $w_{\text{maven}}$.

The principal obtains a payoff of $1$ if a true cause passes, a payoff of $-1$ if any other $a\ne a^{\ast }$ passes, and a payoff of 0 if the agent’s proposal is rejected. The principal’s payoff reflects an objective of maximizing the positive policy impact of the research done on their data.

The principal releases a noisy dataset $\mathcal{D}(q)$ by perturbing the raw data. Specifically, they choose a level of noise $q\in [0,1/2]$, and every binary realization of each covariate is flipped independently with probability $q$. So the noisy dataset $\mathcal{D}(q)$ is $(Y_n,{({\widehat{X}}_n^a)}_{a\in A})$, where ${\widehat{X}}_n^a=X_n^a$ with probability $1-q$, and ${\widehat{X}}_n^a=1-X_n^a$ with probability $q$. The principal’s choice of $q$ is common knowledge. A covariate $a$ that matches the outcome in at least $M$ observations in the noisy dataset but would not pass the policymaker’s test—that is ${\widehat{X}}_n^a=Y$ for at least $M$ observations but $X_n^a=Y$ for fewer than $M$ observations—is called a bait.

The form of noise in our model is motivated by the dissemination noise currently in use by statistical agencies, like the US Census Bureau. One could imagine other ways of generating a “noisy” dataset, such as selecting a random subset of the observations and making them fully uninformative, which corresponds to reserving the selected observations as a hold-out dataset for out-of-sample testing. Our analysis explores the possibility of repurposing the existing practice of adding dissemination noise, which more closely resembles perturbing each data entry independently than withholding some rows of the dataset altogether.^§

We comment on our assumptions regarding the agents and the data in our model.

First, our model features very powerful $p$-hackers. A fraction $h$ of researchers are totally ignorant about the true cause, but they are incentivized to game the system by fishing for some covariate that plausibly explains the outcome variable and passes the policymaker’s test. This kind of $p$-hacking by multiple hypothesis testing is made easy by the fact that hackers have a continuum of covariates to search over and incur no cost from data mining. Our assumptions represent today’s wide datasets and powerful computers that enable ever easier $p$-hacking. Our analysis suggests that dissemination noise can improve social welfare, even in settings where $p$-hacking is costless.

Second, the principal is an entity that wishes to maximize the positive social impact of the research done using their data but has limited power in influencing the institutional conventions surrounding how research results are evaluated and implemented into policies. In the model, the principal cannot change the policymaker’s test. Examples include private firms like 23andMe that possess a unique dataset but have little say in government policy-making, and agencies like the US Census Bureau that are charged with data collection and data stewardship but do not directly evaluate research conclusions. Such organizations already introduce intentional noise in the data they release for the purpose of protecting individual privacy, so they may be willing to use the same tool to improve the quality of policy interventions guided by studies done on their data. In line with this interpretation of the principal, they cannot influence the research process or the policymaker’s decision, except through changing the quality of the disseminated data. In particular, the principal cannot impose a cost on the agent to submit a proposal to the policymaker, write a contract to punish an agent who proposes a misguided policy, or change the protocols surrounding how proposals get tested and turned into policies.

Third, the dataset in our model contains just one outcome variable, but in reality a typical dataset (e.g., the US Census data) contains many outcome variables and can be used to address many different questions. We can extend our model to allow for a countably infinite number of outcome variables $Y^1,Y^2,\dots$, with each outcome associated with an independently drawn true cause and red herring. After the principal releases a noisy version of the data, one random outcome becomes relevant and the agent proposes a model for this specific outcome. Our analysis remains unchanged in this world. This more realistic setting provides a foundation for the principal not being able to screen the agent types by eliciting their private information about the true cause without giving them any data. Who is a maven depends on the research question and the outcome variable being studied, and it is infeasible to test a researcher’s domain expertise with respect to every conceivable future research question.

Fourth, the policymaker’s exogenous test only evaluates how well the agent’s proposal explains the raw dataset and does not provide the agent any other way to communicate his domain expertise. Such a convention may arise if domain expertise is complex and difficult to convey credibly: for instance, an uninformed hacker who has found a strong association in the data can always invent a plausible-sounding story to justify why a certain covariate causes the outcome. We also assume that the policymaker’s test is mechanically set and does not adjust to the presence of $p$-hackers. This represents a short-run stasis in the science advocacy process or publication norms—for instance, while we know how to deal with multiple hypotheses testing, a vast majority of academic journals today still treat $P<0.05$ as a canonical cutoff for statistical significance. Our analysis suggests that dissemination noise can help screen out misguided policies in the short run, when the principal must take as given a policymaking environment that has not adapted to the possibility of $p$-hacking.

To conclude, our basic model is meant to isolate the tradeoff between the cost imposed by noise on honest researchers and the benefit of screening $p$-hackers. A discussion of how the results are affected by relaxing our assumptions is in Section 3.