Protection by BCG Vaccine Against Tuberculosis: A Systematic Review of Randomized Controlled Trials

Abstract

(See the Editorial Commentary by McShane on pages 481–2.)

The BCG vaccine is included in the childhood vaccination program of many countries. However, varying estimates of its efficacy in preventing pulmonary tuberculosis, the major burden of tuberculosis disease, have been found in controlled trials [1, 2], ranging from 0% in the Chingleput trial in South India to 80% in the UK Medical Research Council (MRC) trial [3–5]. Consistently high estimates of efficacy have been reported for infant BCG vaccination against severe primary progressive disease [6–8].

Previous reviews noted a positive association between BCG vaccine efficacy against pulmonary disease with distance from the equator at which studies were conducted [2, 9], possibly related to exposure to environmental mycobacteria, which is, in general, less common at locations distant from the equator [1]. Consistent with this hypothesis, a recent subanalysis of the Chingleput trial suggested some protection (29% efficacy) among participants who had low tuberculin reactivity and no reaction to nontuberculous mycobacterial antigen (Mycobacterium intracellulare) at baseline [10]. Other possible explanations for variability in the efficacy of BCG against pulmonary disease include the role of study quality [11] and that different BCG strains induce different levels of protection [12].

An improved understanding of why BCG vaccine efficacy varies to such a great extent is important to inform assessment of the new generation of tuberculosis vaccines undergoing clinical trials [13], most of which are designed to boost protection by BCG. We conducted a systematic review of all reported BCG trials, to estimate the efficacy of BCG against pulmonary, miliary, and meningeal tuberculosis and examine associations of study characteristics, including immunological naivety to infection, with efficacy.

METHODS

We searched for studies reporting primary data on BCG vaccination efficacy in preventing tuberculosis disease in human populations of any age, in which BCG (without revaccination) was compared with no vaccination (placebo or other control). We excluded non-BCG tuberculosis vaccines (eg, vole bacillus, Savioli antituberculosis vaccine, or other heat-killed bacillus vaccines) and oral BCG. We did not restrict searches by study design, language, publication date, or whether fully published. Two reviewers independently screened titles and abstracts, resolving disagreement via a third reviewer. We retrieved full papers if assessment from the abstract was not possible or if 1 reviewer considered them potentially eligible. This paper is limited to findings from randomized or quasi-randomized trials that reported pulmonary, miliary, or meningeal tuberculosis outcomes.

We searched 10 medical literature electronic databases from inception to May 2009, and other databases including Google Scholar and trial registers to October 2009. An information specialist helped combine MeSH (Medical Subject Headings) and text word terms for disease and intervention into search strategies appropriate for the different databases. Search terms included tuberculosis, tubercle bacill*, M. bovis, M. africanum, M. canetti, M. microti, and M. tuberculosis. Terms for the intervention included BCG vaccine, BCG, and bacillus Calmette (see Supplementary Appendix for sources and search strategy). We identified duplicate or multiple publications, and used the most recent available data in analyses. One person extracted data onto structured piloted forms, another checked accuracy and completeness. For non-English-language publications, 1 person discussed and agreed upon data to be extracted with an extractor fluent in the language of publication. Disagreements were resolved through discussions with other members of the study team. As most papers were published before 1973, authors were not contacted if data were not available.

We extracted trial characteristics, case definitions, outcomes, and summary results. Trial characteristics included distance from the equator by degrees of latitude (collapsed into 20°-latitude groups for analysis) and whether tests for tuberculin sensitivity (a marker of prior Mycobacterium tuberculosis infection as well as some indication of sensitization to other mycobacteria [4]) with purified protein derivative (PPD) were conducted and whether a stringent testing protocol was used. Participants vaccinated as infants were assumed to be tuberculin negative. A stringent tuberculin testing protocol was defined as retesting initially tuberculin-negative participants using a higher dose of tuberculin to confirm negativity before vaccination. A nonstringent tuberculin testing protocol was defined as one that did not exclude noninfant participants based on tuberculin testing prior to vaccination, or which excluded subjects based only on a single tuberculin test.

BCG strain variation was assessed in terms of attenuation lineage, the molecular basis of which was classified by Brosch et al [12]. We classified strains in the 3 groups proposed. We also tested a hypothesis that there would be a loss of protection as BCG strains evolved over time.

We assessed risk of bias in trial results based on the Cochrane Collaboration's risk of bias tool [14], with additional items specific to BCG trials. We did not consider placebo vaccination as blinded during follow-up, as BCG vaccination leaves a scar. In addition, we assessed likelihood of diagnostic detection bias specific to the mode of presentation of pulmonary tuberculosis, based on Clemens et al [11], who noted that a substantial proportion of tuberculosis is missed if disease is identified only using passive follow-up. There is thus a potential for bias if assessors were aware of the trial hypothesis and were not blinded to presence or absence of a BCG scar. Trials in which follow-up was active with regular chest radiography or other assessments were judged to be at a low likelihood of such bias, whether or not assessors were blind, as were trials with passive follow-up in which outcomes were from routine surveillance and assessors were blind to BCG status. Trials using other methods of ascertainment were judged to have a greater likelihood of diagnostic detection bias.

For each trial, we estimated the rate ratio (RR) of tuberculosis, comparing vaccinated with unvaccinated participants, together with the standard error of the log rate ratio. Vaccine efficacy is defined as [1 – RR]. Pooled results, together with both fixed- and random-effects summary effect estimates, were obtained from fixed-effect (inverse variance weighted) and DerSimonian and Laird random-effects meta-analyses [15] of (log) rate ratios from each study. If one of the randomized groups in a trial had 0 cases, 0.5 was added to each cell of the 2 × 2 table. Results from both types of meta-analysis were included in forest plots; differences between them may suggest the presence of small study effects [14]. We also examined possible strain effects by plotting estimated RRs against the year the study started.

Differences in efficacy between subgroups of studies were quantified using random-effects meta-regression to estimate ratios of rate ratios. Heterogeneity between studies was quantified by estimating the between-study variance τ². In forest plots and meta-analyses, τ² was estimated using the method-of-moments estimator proposed by DerSimonian and Laird. For meta-regression analyses, τ² was estimated by restricted maximum likelihood, using the metareg command in Stata.

RESULTS

From 21 030 titles and abstracts, we identified 847 articles for retrieval. We included 211 relevant papers (60 in languages other than English). These articles reported data on 21 randomized or quasi-randomized trials (Supplementary Figure 1), of which 18 reported on pulmonary tuberculosis and 6 reported on meningeal and/or miliary tuberculosis outcomes. Ten trials were conducted in the United States between 1933 and 1950 [16–25]; 4 in India between 1950 and 1988 [26–29]; and 1 each in Canada (started in 1933) [30], the United Kingdom (1950) [31], South Africa (1965) [32], and Haiti (1965) [33] (Table 1). Supplementary Table 1 provides further details of each trial.

Protection Against Pulmonary Tuberculosis

The efficacy of BCG against pulmonary tuberculosis ranged from substantial protection, in the UK MRC trial [31] (RR 0.22; 95% confidence interval [CI], .16–.31), to absence of clinically important benefit, in the Chingleput trial [28] (RR, 1.05; 95% CI, .88–1.25). Figure 1 shows the ratio of the rates of pulmonary tuberculosis among BCG-vaccinated individuals and controls in each trial, stratified according to age at vaccination and stringency of prevaccination tuberculin testing, with fixed- and random-effects summary effects estimates overall and within strata, and estimates of between-trial heterogeneity. There was less heterogeneity within strata (all estimates of τ² <0.095) than overall (τ² = 0.38). The average protection by BCG was greatest in trials of school-age vaccination with stringent tuberculin testing prior to vaccination (random-effects RR, 0.26; 95% CI, .18–.37) and studies of neonatal vaccination (RR, 0.41; 95% CI, .29–.58). Fixed- and random-effects estimates were similar within strata and overall. There was no consistent evidence of protection in trials including participants older than school age, although some protection was found in adults in some trials.

Figure 1.

Rate ratios and 95% confidence intervals for pulmonary tuberculosis, stratified by age vaccinated and stringency of prevaccination tuberculin testing. Trials included in this review, ordered by year of study start. The “other” age group includes studies in which older persons were vaccinated as well as those in which BCG was given at any age. *Date of study publication was used if study start date was not available. Abbreviations: BCG, Bacille de Calmette et Guérin; CCH, Cook County Hospital; CI, confidence interval; D + L, DerSimonian and Laird method; M-H, Mantel-Haenszel method; MRC, Medical Research Council; PY, person-years; RR, rate ratio; TB, tuberculosis; TB HH, tuberculous households; TBPT, Tuberculosis Prevention Trial.

Open in new tabDownload slide

Consistent with previous observations, there were marked differences in estimated efficacy according to latitude at which trials were conducted. The protective effect of BCG was on average greater in trials conducted at latitudes farthest from the equator. Although estimated between-trial heterogeneity was lower within latitude strata than overall, there was evidence of heterogeneity between trials at >40° latitude (τ² = 0.12, Figure 2). Protection was, in general, absent or low in trials closer to the equator (latitudes <20° and 20°–40°). Among trials in which outcome assessors were considered adequately blinded to participants' vaccination status, or if there was active surveillance, there was substantial between-study variation but the average protective effect of BCG against pulmonary tuberculosis was greater (random-effects RR, 0.40; 95% CI, .25–.64) than in trials with higher likelihood of diagnostic detection bias (RR, 0.78; 95% CI, .64–.95) (Figure 3).

Figure 2.

Rate ratios and 95% confidence intervals for pulmonary tuberculosis, stratified by latitude of study location. Ordered by year of study start. *Date of study publication was used if study start date was not available. Abbreviations: BCG, Bacille de Calmette et Guérin; CCH, Cook County Hospital; CI, confidence interval; D + L, DerSimonian and Laird method; M-H, Mantel-Haenszel method; MRC, Medical Research Council; PY, person-years; RR, rate ratio; TB, tuberculosis; TB HH, tuberculous households; TBPT, Tuberculosis Prevention Trial.

Open in new tabDownload slide

Figure 3.

Pooled rate ratios for pulmonary tuberculosis, estimated using random-effects meta-analysis, according to trial characteristics. Rate ratios and 95% confidence intervals are shown. Abbreviations: BCG, Bacille de Calmette et Guérin; DU, duplication; TB, tuberculosis.

Open in new tabDownload slide

When trials were stratified according to BCG strain lineage, there was substantial between-trial heterogeneity within each stratum, and the average effect of BCG vaccination was similar for each strain group (Supplementary Figure 2; Figure 3). There was no clear relationship between estimated vaccine efficacy and year the trial was started, either overall or within strain group (Figure 4).

Figure 4.

Scatter plot of estimated rate ratios for pulmonary tuberculosis, according to year of study start and BCG strain category. DU1-DU2-IV, tandem duplication 1 and fourth form of tandem duplication 2; DU2-III, third form of tandem duplication 2; DU2-IV, fourth form of tandem duplication 2, according to Brosch et al [12]). The efficacy data for 2 trials (Native American [25] and Chingleput [28]) were provided for 2 different strains of BCG, accounting for 2 extra sets of results in this graph. Abbreviations: BCG, Bacille de Calmette et Guérin; DU, duplication.

Open in new tabDownload slide

Univariable meta-regression analyses suggested that, among the trial characteristics considered, distance from the equator and age at vaccination/tuberculin testing stringency explained the majority of between-trial variation in the effect of BCG (residual τ² = 0.086 and 0.044 respectively, compared to 0.284 estimated using a meta-regression model without study characteristics; Table 2). Average protection was lower in trials conducted at 0°–20° and 20°–40° latitude, compared with those conducted at >40° latitude. There was also good evidence that protection was lower in trials including participants older than school age than in studies of neonatal vaccination. There was some evidence that average protection was lower in studies with higher likelihood of diagnostic detection bias compared with studies with lower likelihood of such bias, although this characteristic explained only 18% of the between-trial heterogeneity. There was little evidence that protection varied according to other study design characteristics or BCG strain.

Because latitude has previously been associated with protection by BCG, we next fitted 2-variable meta-regression models including latitude and each other characteristic. These analyses indicated that latitude and age at vaccination/tuberculin testing stringency could explain all of the between-trial heterogeneity (residual τ² = 0). The final multivariable regression model, which also explained the between-trial heterogeneity, included the variables latitude, age at vaccination/tuberculin testing stringency, and likelihood of diagnostic detection bias. Estimated ratios of RRs were attenuated compared with univariable analyses, but each of these characteristics was separately associated with the effect of BCG, having accounted for the other 2.

Protection Against Meningeal or Miliary Tuberculosis

The 6 trials that reported on meningeal and miliary tuberculosis found substantial protection by BCG (RR, 0.15; 95% CI, .08–.31), with little evidence of between-trial heterogeneity (P = .14, Figure 5). Protection appeared greatest in the 2 trials of neonatal vaccination (RR, 0.10; 95% CI, .01–.77) and the 2 trials of school-age vaccination with stringent tuberculin testing (RR, 0.08; 95% CI, .03–.25). The 2 trials with nonstringent tuberculin testing (1 at school age and 1 at a range of ages) found little evidence of protection. However, ratios of RRs were imprecisely estimated in meta-regression analyses (Supplementary Table 2), and there was no strong evidence that the efficacy of BCG varied according to this or other trial characteristics.

Figure 5.

Rate ratios and 95% confidence intervals for meningeal and/or miliary tuberculosis, stratified by age at vaccination and tuberculin testing stringency. Pooled results from fixed effects meta-analysis only as the numbers of studies were small, ordered by year of study start. *Outcome is miliary tuberculosis only. Abbreviations: BCG, Bacille de Calmette et Guérin; CI, confidence interval; MRC, Medical Research Council; PY, person-years; RR, rate ratio; TB, tuberculosis; TB HH, tuberculosis households.

Open in new tabDownload slide

DISCUSSION

We found 3 study characteristics to be associated with estimated protection by BCG against pulmonary tuberculosis. As well as the well-known association of protection with increasing latitude at which trials were conducted, our analysis indicates that protection was greater when BCG was given in infancy or at school age, in trials that used stringent tuberculin testing to try to exclude participants already sensitized to mycobacteria, and in studies with lower likelihood of diagnostic detection bias. Together, these factors were sufficient to explain the between-study variation in the protective effect of BCG against pulmonary tuberculosis. We found little evidence that other study characteristics or BCG vaccine strain were associated with protection. Protection against meningeal and miliary tuberculosis also appeared greater than for pulmonary tuberculosis and when BCG was given to infants or at school age after stringent tuberculin testing.

Randomized controlled trials provide the best evidence for the effectiveness of interventions, but many BCG trials were conducted before standard methods for trial conduct and reporting were developed. Many used alternation or other “quasi-randomized” methods of allocation to BCG or control, which do not guarantee concealment of allocation at recruitment or blinding of participants and trial personnel, and some aspects of trial design were not clearly reported. Previous systematic reviews (eg, [9]) of 13 trials reporting tuberculosis disease outcomes did not assess whether several of these design characteristics or the exclusion of those with prior infection or sensitization to environmental mycobacteria using stringent tuberculin testing were related to BCG protection. Based on comprehensive searches, we included the same 13 trials, and found 5 more eligible trials. We used recently developed approaches to assessing risk of bias in trial results. We also assessed additional potential biases specific to BCG vaccine trials defined a priori based on a criterion proposed by Clemens et al [11] (blinding of study staff who assessed outcome on BCG status or active surveillance), as well as the variability between trials in stringency of pre-vaccination tuberculin testing. We used meta-regression to examine these different possible explanations for variation in the estimated effect of BCG across studies. However, meta-regression analyses have limitations [34]. They are ecological analyses with trials as units of observation; hence, observed associations may result from confounding by other study design characteristics. Studies examined efficacy over varying follow-up times. An alternative of restricting to the same period would have reduced the number of studies that could be included. Our multivariable analyses included 7 variables, which is large compared with the total number of studies (18). Therefore, our finding that 3 characteristics could explain all the between-trial variation in the effect of BCG on pulmonary tuberculosis should be interpreted with caution. Too few trials reported on miliary and meningeal tuberculosis to allow a comprehensive analysis of between-trial heterogeneity.

The effect of latitude on efficacy persisted after adjustment, perhaps because even stringent tuberculin testing does not exclude all sensitization to environmental mycobacteria. Other proposed explanations include human genetic differences, genotypic differences between infecting mycobacteria, or a variety of proposed explanations for the association of protection with latitude: exposure to ultraviolet light (due to its mycobacterial killing effect); levels of vitamin D, helminthic infestation, or the effect of poor nutrition on immune response. Previous reviews concluded that these factors are less plausible explanations than exposure to environmental mycobacteria [35].

Previous systematic reviews found substantial variation between trials in estimated protection by BCG against pulmonary tuberculosis [2, 9], and 1 review estimated average protective efficacy to be 50% [9]. However, in the absence of explanations for heterogeneity, such an average cannot be applied to the use of BCG in a particular setting or population.

It is well known that there are genetic differences between BCG vaccines, for example, based on restriction fragment-length polymorphism typing that suggests BCG strains have undergone evolution since 1921 [12]. Brosch et al recently used genome sequencing to postulate that BCG vaccines derived before 1930 or 1940 may be immunologically superior to more recent and widely used variants [12]. We found little evidence of an association between estimated effects of BCG with the year each trial commenced or that effects varied according to the groups proposed, which include strains currently in use: Denmark (in DU2 group III), Russia (in DU2 group I) and Japan (also in DU2 group I) [12]. Our findings are consistent with results from the UK MRC trial [31], which found equivalent protection by the Copenhagen strain of BCG and a Mycobacterium microti–derived vaccine (vole bacillus) [5].

A possible explanation for the low protection observed in trials in the southern United States vs high protection in the United Kingdom was first proposed during the 1960s, based on guinea-pig studies [1]. The findings suggested that exposure to certain nontuberculous mycobacterial antigens could mask the observed effectiveness of BCG, by providing some protection against tuberculosis in nonvaccinated groups, which was not enhanced by BCG vaccination. The authors also noted that populations in the southern United States, where the trials were carried out, have a high prevalence of sensitivity to M. intracellulare and other environmental mycobacteria. The hypothesis that exposure to environmental mycobacteria before or after BCG induces an immune response similar to that induced by BCG, so that BCG can add little, has been supported by animal and human population studies [2, 36]. More recent immunogenicity studies suggest that exposure to nontuberculous mycobacterial antigens could also block BCG vaccination from offering protection when infection precedes vaccination [37]. Our findings are consistent with these hypotheses, perhaps more consistent with the latter, BCG being more effective in immunologically naive individuals.

Because of the evidence that BCG protects against miliary and meningeal tuberculosis, in developing countries BCG vaccination is recommended at birth (or first contact with health services), taking into account HIV status [38]. Our systematic review suggests that BCG also confers protection against pulmonary disease, the greatest burden from tuberculosis, when administered both in infancy and at school age, providing that children are not already infected with M. tuberculosis or sensitized to other mycobacterial infections. Protection against pulmonary disease was seen in the Bombay Infants trial, suggesting that, even close to the equator, if BCG is administered prior to exposure to tuberculosis and environmental mycobacteria, it can provide significant protection [27]. Further evidence of protection in populations close to the equator from BCG given before infection would strengthen these findings. These possible explanations for the observed variation in protection from BCG vaccine have implications for the evaluation of new tuberculosis vaccines [39]. If given in conjunction with BCG, new vaccines must be shown to offer additional protection against pulmonary disease. New “BCG-like” vaccines may only give protection if administered prior to exposure to M. tuberculosis [40].

Notes

Acknowledgments. We thank Margaret Burke, who designed and conducted searches of bibliographic databases; staff and students at the London School of Hygiene and Tropical Medicine, the Health Protection Agency, and Bristol University for translating non-English-language papers; and John Watson and David Elliman for useful discussions and comments.

Financial support. This work was supported by the UK National Institute for Health Research Evaluation, Trials and Studies Coordinating Centre (http://www.netscc.ac.uk/about/; grant number 08/16/01).

Potential conflicts of interest. All authors: No reported conflicts.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References