Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2014 Sep;12(1-2):40-6.
doi: 10.3121/cmr.2013.1188. Epub 2014 Jan 10.

Evidence synthesis for medical decision making and the appropriate use of quality scores

Suhail A R Doi  1
Affiliations
Review

Evidence synthesis for medical decision making and the appropriate use of quality scores

Suhail A R Doi. Clin Med Res. 2014 Sep.

Abstract

Meta-analyses today continue to be run using conventional random-effects models that ignore tangible information from studies such as the quality of the studies involved, despite the expectation that results of better quality studies reflect more valid results. Previous research has suggested that quality scores derived from such quality appraisals are unlikely to be useful in meta-analysis, because they would produce biased estimates of effects that are unlikely to be offset by a variance reduction within the studied models. However, previous discussions took place in the context of such scores viewed in terms of their ability to maximize their association with both the magnitude and direction of bias. In this review, another look is taken at this concept, this time asserting that probabilistic bias quantification is not possible or even required of quality scores when used in meta-analysis for redistribution of weights. The use of such a model is contrasted with the conventional random effects model of meta-analysis to demonstrate why the latter is inadequate in the face of a properly specified quality score weighting method.

Keywords: Bias; Medical decision making; Meta-analysis; Quality scores.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Conceptualization of model 2.
Figure 2
Figure 2
Comparison of the distribution of pooled estimates by the quality effects (left panel) and random effects (right panel) models when β = 0 and quality is not-mis-specified (simulation 1).
Figure 3
Figure 3
Comparison of the distribution of pooled estimates by the quality effects model when β = 0 (top left panel; simulation 1) and mis-specified quality effects model when β = 0 (top right panel; simulation 2) in comparison to the RE model. The bottom panel (simulation 3) represents the constraint that β = 0.1. Each panel represents a separate simulation run with 50,000 iterations involving 250,000 individual studies.
See this image and copyright information in PMC

Comment in

Similar articles

See all similar articles

Cited by

References

    1. Detsky AS, Naylor CD, O’Rourke K, McGeer AJ, L’Abbe KA. Incorporating variations in the quality of individual randomized trials into meta-analysis. J Clin Epidemiol 1992;45:255–265. - PubMed
    1. Lohr KN. Rating the strength of scientific evidence: relevance for quality improvement programs. Int J Qual Health Care 2004;16:9–18. - PubMed
    1. Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Krebs Seida J, Klassen Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ 2009;339:b4012. - PMC - PubMed
    1. Greenland S, O’Rourke K. On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions. Biostatistics 2001;2:463–471. - PubMed
    1. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609–613. - PubMed

LinkOut - more resources