Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2016 Jul 12;6(7):e010247.
doi: 10.1136/bmjopen-2015-010247.

Plea for routinely presenting prediction intervals in meta-analysis

Joanna IntHout  1 , John P A Ioannidis  2 , Maroeska M Rovers  1 , Jelle J Goeman  1
Affiliations
Meta-Analysis

Plea for routinely presenting prediction intervals in meta-analysis

Joanna IntHout et al. BMJ Open. .

Abstract

Objectives: Evaluating the variation in the strength of the effect across studies is a key feature of meta-analyses. This variability is reflected by measures like τ(2) or I(2), but their clinical interpretation is not straightforward. A prediction interval is less complicated: it presents the expected range of true effects in similar studies. We aimed to show the advantages of having the prediction interval routinely reported in meta-analyses.

Design: We show how the prediction interval can help understand the uncertainty about whether an intervention works or not. To evaluate the implications of using this interval to interpret the results, we selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n=2009) or continuous (n=1254) outcome, and generated 95% prediction intervals for them.

Results: In 72.4% of 479 statistically significant (random-effects p<0.05) meta-analyses in the Cochrane Database 2009-2013 with heterogeneity (I(2)>0), the 95% prediction interval suggested that the intervention effect could be null or even be in the opposite direction. In 20.3% of those 479 meta-analyses, the prediction interval showed that the effect could be completely opposite to the point estimate of the meta-analysis. We demonstrate also how the prediction interval can be used to calculate the probability that a new trial will show a negative effect and to improve the calculations of the power of a new trial.

Conclusions: The prediction interval reflects the variation in treatment effects over different settings, including what effect is to be expected in future patients, such as the patients that a clinician is interested to treat. Prediction intervals should be routinely reported to allow more informative inferences in meta-analyses.

Keywords: Clinical trial; Cochrane Database of Systematic Reviews; Heterogeneity; Meta-analysis; Prediction interval; Random effects.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Forest plot of the standardised mean difference (SMD) in symptom scores in nasal polyps. Steroids versus placebo, analysis 1.1 in Cochrane Review CD006549. Note that our results differ from the original analysis, as we used a random-effects analysis with the Hartung-Knapp/Sidik-Jonkman adjustment and the empirical Bayes estimator for τ2.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Riley RD, Higgins JPT, Deeks JJ. Interpretation of random effects meta-analyses. BMJ 2011;342:d549 10.1136/bmj.d549 - DOI - PubMed
    1. Thompson SG, Higgins JPT. How should meta-regression analyses be undertaken and interpreted? Stat Med 2002;21:1559–73. 10.1002/sim.1187 - DOI - PubMed
    1. Higgins JP, Thompson SG, Deeks JJ et al. . Measuring inconsistency in meta-analyses. BMJ 2003;327:557–60. 10.1136/bmj.327.7414.557 - DOI - PMC - PubMed
    1. Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc 2009;172:137–59. 10.1111/j.1467-985X.2008.00552.x - DOI - PMC - PubMed
    1. Saha S, Chant D, McGrath J. Meta-analyses of the incidence and prevalence of schizophrenia: conceptual and methodological issues. Int J Methods Psychiatr Res 2008;17:55–61. 10.1002/mpr.240 - DOI - PMC - PubMed

Publication types

LinkOut - more resources