## New Research In

### Physical Sciences

### Social Sciences

#### Featured Portals

#### Articles by Topic

### Biological Sciences

#### Featured Portals

#### Articles by Topic

- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology

# Reply to Held: When is a harmonic mean *p*-value a Bayes factor?

### This article has a Letter. Please see:

### See related content:

- The harmonic mean
*p*-value for combining dependent tests- Jan 22, 2019

I welcome this opportunity (1) to acknowledge Good’s papers (2⇓⇓⇓–6), which I had missed. Good proposed the harmonic mean p-value (HMP) as a classical analog to a model-averaged Bayes factor (BF) which “should be regarded as an approximate tail-area probability [p-value]” (2). His presentation was amusingly apologetic; for example, “an approximate rule of thumb is tentatively proposed in the hope of provoking discussion” and “this rule of thumb should not be used if the statistician can think of anything better to do” (2). I believe my paper dispels these misgivings by formalizing Good’s intuitive argument that the HMP is approximately well-calibrated when small (equation 5 of ref. 7) and deriving an asymptotically exact test for general use (equation 4 of ref. 7). Further, I showed the HMP is a multilevel test procedure (equation 6 of ref. 7), demonstrating with examples that it consequently provides a powerful alternative to Bonferroni and Benjamini–Hochberg (8) correction for large-scale multiple testing problems.

Held (1) considers the Bayesian properties of the HMP, which are relevant to its interpretation and power. Applications of the HMP are not limited to model selection problems, however, as it provides a general alternative to Fisher’s method (9) for combining tests that are not independent (2, 7).

Good claimed that the HMP is inversely proportional to a model-averaged BF based on his empirical observations that

Good’s claim depends on the density of p-values under the alternative,

Held (1) considers whether the class of alternatives **2**), in which

## Acknowledgments

D.J.W. is a Sir Henry Dale Fellow jointly funded by the Wellcome Trust and Royal Society Grant 101237/Z/13/Z and is supported by a Big Data Institute Robertson Fellowship.

## Footnotes

- ↵
^{1}Email: daniel.wilson{at}bdi.ox.ac.uk.

Author contributions: D.J.W. designed research, performed research, analyzed data, and wrote the paper.

The author declares no conflict of interest.

Data deposition: The R code used to generate Fig. 1 has been deposited on figshare (https://doi.org/10.6084/m9.figshare.7699955).

Published under the PNAS license.

## References

- ↵
- Held L

- ↵
- ↵
- Good IJ

- ↵
- Good IJ

- ↵
- Good IJ

- ↵
- Good IJ

- ↵
- Wilson DJ

*p*-value for combining dependent tests. Proc Natl Acad Sci USA 116:1195–1200. - ↵
- ↵
- Fisher RA

- ↵
- Mikosch T

- ↵

*p*-value a Bayes factor?

## Citation Manager Formats

## Sign up for Article Alerts

## Article Classifications

- Physical Sciences
- Statistics

- Biological Sciences
- Genetics