Swedish researchers have warned that the software packages used to analyze the results of functional magnetic resonance imaging (fMRI) contain flaws that increase the chance of a false positive by as much as 70 percent. Anders Eklund, “Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates,” PNAS, June 2016.
For more than 15 years, scientists have used fMRI analyses to explore the food addiction framework and the effect of food advertising on the brain, among other things.
The Swedish study explains that the majority of fMRI studies rely on SPM, FSL or AFNI software packages based on “parametric statistical methods that depend on a variety of assumptions,” even though these methods have only been validated with simulated—as opposed to real— data. As a result, the researchers questioned whether these methods could potentially show brain activity in its absence, raising the issue of false positives.
Using resting-state data from 499 healthy controls to conduct 3 million task-group analyses, the study’s authors apparently estimated the incidence of significant results and concluded that “the parametric levels can give a very high degree of false positives” for clusterwise inference. “In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%,” explain the researchers. “These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results.”
Highlighting new graphics cards with increased processing power, the study offers another statistical method “in which few assumptions are made and significantly more calculations—a thousand times more—are done, which yields a significantly more certain result,” according to a June 28, 2016, Linköping University press release.
“Our results suggest that the principal cause of the invalid cluster inferences is spatial autocorrelation functions that do not follow the assumed Gaussian shape,” concludes the study. “It is not feasible to redo 40,000 fMRI studies, and lamentable archiving and data-sharing practices mean most could not be reanalyzed either. Considering that it is now possible to evaluate common statistical methods using real fMRI data, the fMRI community should, in our opinion, focus on validation of existing methods.”