New Research In
Physical Sciences
Social Sciences
Featured Portals
Articles by Topic
Biological Sciences
Featured Portals
Articles by Topic
- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology
fMRI clustering and false-positive rates
See related content:

Recently, Eklund et al. (1) analyzed clustering methods in standard fMRI packages: AFNI (which we maintain), FSL, and SPM. They claim that (i) false-positive rates (FPRs) in traditional approaches are greatly inflated, questioning the validity of “countless published fMRI studies”; (ii) nonparametric methods produce valid, but slightly conservative, FPRs; (iii) a common flawed assumption is that the spatial autocorrelation function (ACF) of fMRI noise is Gaussian-shaped; and (iv) a 15-y-old bug in AFNI’s 3dClustSim significantly contributed to producing “particularly high” FPRs compared with other software. We repeated simulations from ref. 1 [Beijing_Zang data (2), cf. ref. 3) and comment on each point briefly.
AFNI and 3dClustSim
Fig. 1 A–D compares results of the “buggy” and “fixed” 3dClustSim. For each simulation, the typical difference was small:
FPRs for various software scenarios, with 1,000 two-sample one-sided
Figures 1 and 2 of ref. 1 actually show similar FPRs for AFNI, FSL-OLS, and SPM: Most tests were in a range of
Smoothness
To test the effect of assuming a Gaussian ACF in fMRI noise, an empirical “mixed ACF” allowing for longer tails was computed from residuals (3). All FPRs (Fig. 1 E and F) decreased. Block designs remained
Nonparametric Approach
A spatial model-free, nonparametric randomization approach was added to AFNI’s group-level GLM program, 3dttest++ (3). All FPRs (Fig. 1 G and H) were within the nominal confidence interval. Although this approach shows promise (as in ref. 1), it may not be feasible to generalize nonparametric permutations to complicated covariate structures and models (e.g., complex ANOVA, analysis of covariance, or linear mixed effects) (4, 5).
Inflated FPRs
Several cases showed significant FPR inflation across existing fMRI software within the testing framework of ref. 1. However, deviations from nominal FPR were not uniformly large and depended strongly on several factors. Fig. 1 and figure 1 of ref. 1 show quite good cluster results for stricter per-voxel P values (which ref. 6 found to be predominantly used in fMRI analyses) and for event-related stimuli (emphasizing the importance of good experimental design): FPR inflation was often
We strongly disagree with Eklund et al.’s (1) summary statement: “Alarmingly, the parametric methods can give a very high degree of false positives (up to
Acknowledgments
This work was supported by the National Institute of Mental Health and National Institute of Neurological Disorders and Stroke Intramural Research Programs ZICMH002888 of the NIH, US Department of Health and Human Services. This work used the computational resources of the NIH High-Performance Computing Biowulf cluster (https://hpc.nih.gov/).
Footnotes
- ↵1To whom correspondence should be addressed. Email: robertcox{at}mail.nih.gov.
Author contributions: R.W.C. designed research; R.W.C. performed research; R.W.C. contributed new reagents/analytic tools; R.W.C. and R.C.R. analyzed data; and R.W.C., G.C., D.R.G., R.C.R., and P.A.T. wrote the paper.
The authors declare no conflict of interest.
References
- ↵.
- Eklund A,
- Nichols TE,
- Knutsson H
- ↵.
- Biswal BB, et al.
- ↵.
- Cox RW,
- Reynolds RC,
- Taylor PA
- ↵
- ↵
- ↵
Citation Manager Formats
Sign up for Article Alerts
Jump to section
You May Also be Interested in
More Articles of This Classification
Related Content
Cited by...
- Changes in Endogenous Dopamine Induced by Methylphenidate Predict Functional Connectivity in Nonhuman Primates
- The Integration of Functional Brain Activity from Adolescence to Adulthood
- Discrete Modules and Mesoscale Functional Circuits for Thermal Nociception within Primate S1 Cortex
- Early childhood stress exposure, reward pathways, and adult decision making
- Hierarchically Organized Medial Frontal Cortex-Basal Ganglia Loops Selectively Control Task- and Response-Selection
- Reply to Brown and Behrmann, Cox, et al., and Kessler et al.: Data and code sharing is the way forward for fMRI
- Controversy in statistical analysis of functional magnetic resonance imaging data