Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2015 Dec;33(10):1299-1305.
doi: 10.1016/j.mri.2015.07.014. Epub 2015 Aug 5.

Permutation and parametric tests for effect sizes in voxel-based morphometry of gray matter volume in brain structural MRI

David A Dickie  1 , Shadia Mikhael  2 , Dominic E Job  2 , Joanna M Wardlaw  2 , David H Laidlaw  3 , Mark E Bastin  2
Affiliations

Permutation and parametric tests for effect sizes in voxel-based morphometry of gray matter volume in brain structural MRI

David A Dickie et al. Magn Reson Imaging. 2015 Dec.

Abstract

Permutation testing has been widely implemented in voxel-based morphometry (VBM) tools. However, this type of non-parametric inference has yet to be thoroughly compared with traditional parametric inference in VBM studies of brain structure. Here we compare both types of inference and investigate what influence the number of permutations in permutation testing has on results in an exemplar study of how gray matter proportion changes with age in a group of working age adults. High resolution T1-weighted volume scans were acquired from 80 healthy adults aged 25-64years. Using a validated VBM procedure and voxel-based permutation testing for Pearson product-moment coefficient, the effect sizes of changes in gray matter proportion with age were assessed using traditional parametric and permutation testing inference with 100, 500, 1000, 5000, 10000 and 20000 permutations. The statistical significance was set at P<0.05 and false discovery rate (FDR) was used to correct for multiple comparisons. Clusters of voxels with statistically significant (PFDR<0.05) declines in gray matter proportion with age identified with permutation testing inference (N≈6000) were approximately twice the size of those identified with parametric inference (N=3221voxels). Permutation testing with 10000 (N=6251voxels) and 20000 (N=6233voxels) permutations produced clusters that were generally consistent with each other. However, with 1000 permutations there were approximately 20% more statistically significant voxels (N=7117voxels) than with ≥10000 permutations. Permutation testing inference may provide a more sensitive method than traditional parametric inference for identifying age-related differences in gray matter proportion. Based on the results reported here, at least 10000 permutations should be used in future univariate VBM studies investigating age related changes in gray matter to avoid potential false findings. Additional studies using permutation testing in large imaging databanks are required to address the impact of model complexity, multivariate analysis, number of observations, sampling bias and data quality on the accuracy with which subtle differences in brain structure associated with normal aging can be identified.

Keywords: Aging; Brain structure; Gray matter; Magnetic resonance imaging; Statistical inference.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Permutation testing for effect sizes. The statistical significance (P) of an observed effect size is calculated by counting how many times an effect size larger than the observed effect size is found in random permutations of the data. In this case the observed effect size of −0.37 was calculated from the Pearson product-moment coefficient formula. When randomly permuting the data, effect sizes of −0.37 or larger were found 0.04% of the time, i.e. P=0.0004. This indicates that the observed effect size of −0.37 was highly statistically significant and unlikely to be due to chance.
Figure 2
Figure 2
Observed effect size for reductions in grey matter proportion with age (25–64 years) across the cohort provided by the Pearson product-moment coefficient. R=right; L=left.
Figure 3
Figure 3
P-values for effect sizes with parametric inference showing reductions in grey matter proportion with age (25–64 years) across the cohort. The red volume shows uncorrected P-values of < 0.05, while the blue volume shows false discovery rate corrected P-values of < 0.05.
Figure 4
Figure 4
P -values for effect sizes with permutation testing inference showing reductions in grey matter proportion with age (25–64 years) across the cohort. The red volumes show uncorrected P-values of < 0.05, while the blue volumes are false discovery rate (FDR) corrected P-values of < 0.05. No P-values at 100 and 500 permutations survived FDR correction. Patterns of grey matter loss were approximately stable from ≥ 10000 permutations.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Ashburner J, Friston K. Voxel-Based Morphometry The Methods. Neuroimage. 2000;11:805–821. - PubMed
    1. Good CD, Johnsrude IS, Ashburner J, Henson RNA, Friston KJ, Frackowiak RSJ. A voxel-based morphometric study of ageing in 465 normal adult human brains. Neuroimage. 2001;14:21–36. - PubMed
    1. Ziegler G, Dahnke R, Jäncke L, Yotter RA, May A, Gaser C. Brain structural trajectories over the adult lifespan. Hum Brain Mapp. 2011;33:2377–2389. - PMC - PubMed
    1. Salmond CH, Ashburner J, Vargha-Khadem F, Connelly A, Gadian DG, Friston KJ. Distributional assumptions in voxel-based morphometry. Neuroimage. 2002;17:1027–1030. - PubMed
    1. Freedman D, Pisani R, Purves R. Statistics. New York: WW Norton; 2007.

Publication types