Reducing type II error in fMRI analysis: cluster-extent threshold simulation results and an evaluation of current methods to correct for multiple comparisons

Many procedures to correct for multiple comparisons in functional magnetic resonance imaging (fMRI) analysis require a minimum cluster-extent threshold; however, sample size (N) is often not modeled. In this study, a series of simulations was conducted where N was varied to determine whether this parameter affected cluster threshold. The primary hypothesis was that modeling N in the simulations would reduce cluster thresholds. A secondary hypothesis was that this cluster size reduction was due to between-subject variability, which was tested by eliminating the corresponding standard error term. Acquisition volume parameters were fixed, while key parameters were varied to reflect reasonable ranges: N (10, 20, or 30), corrected p-value (.05, .01, or .001), individual-voxel p-value (.01, .005, or .001), FWHM (3, 5, or 7 mm), and voxel resolution (2 or 3 mm). Each simulation consisted of 100 iterations repeated 100 times, with a total of 4,860,000 iterations and 66,420,000 simulated subjects. There was a significant effect of condition with clusters approximately 18% smaller with versus without N modeled and a significant increase in cluster thresholds for larger sample sizes. Bayesian analysis provided very strong support for the secondary hypothesis. These simulation results were replicated in a real fMRI data set. The present findings indicate that sample size should be incorporated into all methods to provide the most accurate thresholds possible and reduce type II error. A broader range of topics is discussed including balancing type I and type II error, and the assumption that non-task fMRI activity reflects null data is questioned.