Bacille Calmette-Guérin Vaccination in Infancy Does Not Protect Against Coronavirus Disease 2019 (COVID-19): Evidence From a Natural Experiment in Sweden

Abstract

Background

The bacille Calmette-Guérin (BCG) tuberculosis vaccine has immunity benefits against respiratory infections. Accordingly, it has been hypothesized to have a protective effect against coronavirus disease 2019 (COVID-19). Recent research found that countries with universal BCG childhood vaccination policies tend to be less affected by the COVID-19 pandemic. However, such ecological studies are biased by numerous confounders. Instead, this paper reports on a rare nationwide natural experiment that occurred in Sweden in 1975, where discontinuation of newborns’ BCG vaccination led to a dramatic decrease in BCG coverage rate, thus allowing us to estimate BCG’s effect without the biases associated with cross-country comparisons.

Methods

Numbers of COVID-19 cases and hospitalizations were recorded for birth cohorts born just before and just after 1975, representing 1 026 304 and 1 018 544 individuals, respectively. We used regression discontinuity to assess the effect of BCG vaccination on COVID-19–related outcomes. On such a large population, this method allows for a precision that would be hard to achieve using a randomized controlled trial.

Results

The odds ratios (95% CI) for COVID-19 cases and COVID-19–related hospitalizations were 1.0005 (.8130–1.1881) and 1.2046 (.7532–1.6560), allowing us to reject fairly modest effects of universal BCG vaccination. We can reject with 95% confidence that universal BCG vaccination reduces the number of cases by 19% and the number of hospitalizations by 25%.

Conclusions

While the effect of a recent vaccination must be evaluated, we provide strong evidence that receiving the BCG vaccine at birth does not have a protective effect against COVID-19 among middle-aged individuals.

The bacille Calmette-Guérin (BCG) vaccine is an attenuated live vaccine that has been proven to be effective against tuberculosis, in particular its severe manifestations such as meningeal and miliary tuberculosis [1]. Besides its specific effects, the vaccine has immunity benefits against nontargeted pathogens [2], and in particular against respiratory infections caused by RNA viruses like influenza [3]. These effects are thought to be mediated mostly by “trained immunity,” a recently described mechanism of epigenetic reprogramming of innate immune cells [4]. While this mechanism is still under investigation, most studies show that these effects tend to wane after 15 to 20 years [5, 6]. Since severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is also a single-stranded RNA virus, it has been hypothesized that differences in BCG vaccination coverage could explain the wide differences in disease burden observed between countries. A pioneering preprint paper by Miller et al [7] found that countries with universal BCG childhood vaccination policies tend to be less affected by the coronavirus disease 2019 (COVID-19) pandemic, in terms of their number of cases and deaths. While unpublished, this study had a great impact and gave rise to many comments and follow-up studies (reviewed in [8]). Some published studies were able to replicate this result [9–11], but several authors underlined the important statistical flaws inherent to such ecological studies and concluded that randomized controlled trials (RCTs) were necessary to address the question [8, 12]. As of 5 June 2020, no less than 12 RCTs studying the protective effect of the BCG against COVID-19 are already registered at https://clinicaltrials.gov/. However, none has a primary completion date earlier than 1 October 2020, so the first results of these RCTs will not be known until at least 5 or 6 months. With the epidemic still on the rise worldwide, and in the absence of a SARS-CoV-2 vaccine, there is an urgent need to know if BCG nonspecific effects could be harnessed as a substitute prophylactic treatment.

Regression discontinuity (RD) is a method designed by social scientists to assess the effect of an exposure on an outcome. It is deemed as reliable as RCTs to tease out causality from correlation [13], and typically yields results similar to those obtained in RCTs [14, 15]. In this paper, we applied this method to a rare natural experiment that took place in Sweden. Sweden currently has the fifth highest ratio of COVID-19 deaths per capita in the world. In April 1975, it stopped its newborn BCG vaccination program, leading to a dramatic drop in the BCG vaccination rate from 92% to 2% for cohorts born just before and just after the change [16]. We compared the number of COVID-19 cases and hospitalizations per capita, for cohorts born just before and just after April 1975, representing 1 026 304 and 1 018 544 individuals, respectively. Using RD, we were able to show that those cohorts do not have different numbers of COVID-19 cases and hospitalizations per capita, with a high precision that would hardly be possible to reach with an RCT design.

METHODS

Data Sources

Two sources of data were used. First, COVID-19 reported cases, hospitalizations, and deaths compiled by Folkhälsomyndigheten, the Public Health Agency of Sweden, as of 17 May 2020 were used. At that date, Sweden had 30 312 reported COVID-19 cases and 3954 reported COVID-19–related deaths (ie, with a confirmed COVID-19 diagnosis during the past 30 days) [17]. Second, Swedish demographic data publicly available from Statistics Sweden’s website were used. For the number of COVID-19 cases, data for quaterly birth cohorts (QBCs) from quarter (Q) 1-1930 to Q4-2001 were used. For hospitalizations, data from Q1-1930 to Q4-1991 were used. Some QBCs after Q4-1991 had less than 5 hospitalizations, so Folkhälsomyndigheten could not give us their number of hospitalizations due to confidentiality issues. For the number of reported COVID-19 deaths, data were grouped by 3 yearly birth cohorts (YBCs) from YBC 1930 to YBC 1980 to ensure that all groups had at least 5 deaths recorded. The details of the raw and constructed variables used in the analysis are described in Supplementary Table 1. Supplementary Table 2 shows the number of cases and hospitalizations by decade of births, from the 1930s to the 1980s. Those numbers are higher for older than for younger cohorts, although there are still 4304 cases and 1020 hospitalizations among individuals born in the 1970s, the decade when the reform we studied took place.

Statistical Analysis

Regression discontinuity is a commonly used method to measure the effect of a treatment on an outcome [18]. It is applicable when only individuals who satisfy a strict criterion are eligible for a policy. Then, RD is used to compare the outcome of interest among individuals just above and just below the eligibility threshold. In this study, RD is used to compare the number of COVID-19 cases, hospitalizations, and deaths among individuals born just before and just after 1 April 1975. The effect of universal BCG vaccination for individuals born around 1 April 1975 was estimated using the state-of-the-art estimator for RD [19]. The estimator compares treated and control units, in a narrow window around 1 April 1975. It uses linear regressions of the outcome on birth cohort to the left and to the right of the threshold to predict the outcome of treated and untreated units at the threshold. Then, the estimator calculates the difference between these 2 predicted values. The estimator and 95% confidence interval (CI) were computed using the rdrobust Stata command (StataCorp) (see [20]). The RD estimator in Calonico et al [19] assumes that the variable determining exposure is continuous, but it has also been used when that variable is discrete but takes a large number of values. Our analysis is at the quarter-of-birth level, so the variable determining exposure to BCG vaccination is not continuous, but it takes a large number of values. For instance, we observe cases/1000 inhabitants for 288 QBCs.

The validity of the RD estimator is usually assessed by testing if observations to the left and to the right of the threshold have similar characteristics [13]. In Supplementary Table 3, we show that the proportions of women and of foreign-born residents are similar in birth cohorts to the left and to the right of the threshold.

Finally, because we observe deaths for groups of 3 consecutive YBCs until 1980, there are only 2 data points after 1975 for that variable (1975–1977 and 1978–1980), so the RD method cannot be used. In the Supplementary Materials, we merely use a t test to compare the death rates in the 1972–1974 and 1975–1977 YBCs. This method is less robust than the RD method we use for cases and hospitalizations, as it does not account for the fact that the average age of the treated and control groups differs by 3 years.

RESULTS

This study used the number of COVID-19 cases per 1000 inhabitants for QBCs born between Q1-1930 and Q4-2001, the number of COVID-19 hospitalizations per 1000 inhabitants for cohorts born between Q1-1930 and Q4-1991, and the number of COVID-19 deaths per 1000 inhabitants for groups of 3 YBCs, from 1930-1931-1932 to 1978-1979-1980. These variables were constructed using data compiled by the Public Health Agency of Sweden (see Supplementary Table 1 for details).

In an RD design, the presence or absence of a treatment effect can be assessed visually, by drawing a scatterplot with the variable determining eligibility on the x axis and the outcome variable on the y axis. If one observes that the relationship between these 2 variables is discontinuous at the eligibility threshold, this is indicative of a treatment effect. Accordingly, Figure 1 shows no discontinuity in the numbers of COVID-19 cases per 1000 inhabitants for cohorts born just before and just after April 1975. This suggests that universal BCG vaccination has no effect on the number of COVID-19 cases per 1000 inhabitants for individuals born in 1975. Figures 2 and 3 show that similar conclusions apply when one looks at the number of COVID-19 hospitalizations per 1000 inhabitants and at the number of COVID-19 deaths per 1000 inhabitants. The number of deaths per 1000 inhabitants is several orders of magnitudes higher for older than for younger cohorts, so Figure 3 only presents those numbers for cohorts born after 1960 to keep the graph legible.

This visual analysis is confirmed by the statistical calculations. Table 1 reports RD estimates of the effect of the BCG vaccination policy on COVID-19 outcomes using the RD estimator (see “Methods”). They show that the effects of the BCG vaccination policy on cases per 1000 inhabitants and hospitalizations per 1000 inhabitants are not statistically different from 0. Based on the CIs, we can, for instance, reject with 95% confidence that universal BCG vaccination reduces the number of cases per 1000 inhabitants by .387, an effect equivalent to 12% of the number of cases per 1000 inhabitants in the 1975 cohort. For the number of hospitalizations per 1000 inhabitants, we can reject an effect equal to 15% of the number of hospitalizations per 1000 inhabitants in the 1975 cohort. For deaths per capita, we find that the death rates in the 1972-1973-1974 and 1975-1976-1977 YBCs are not significantly different (see Supplementary Materials).

The effects shown in Table 1 are intention-to-treat effects [21], as the vaccination coverage pre-1975 was approximately 92%, and after the 1975 change in policy approximately 2% of the population was still vaccinated. Taking into account foreign-born residents (27.2% of the Swedish population born in 1975), who were not affected by the policy, the policy led to a drop in the vaccination rate of 65.5%. To convert the intention-to-treat effects in Table 1 into the effect of being vaccinated at birth, one needs to divide the intention-to-treat effects and their CIs by .655 [22]. For odds ratios, the effects thus obtained are 1.0005 (95% CI, .8130–1.1881) for COVID-19 cases and 1.2046 (95% CI, .7532–1.6560) for COVID-19–related hospitalizations. These odds ratios are not significantly different from 1, confirming the absence of effect of BCG vaccination at birth on COVID-19–related outcomes.

DISCUSSION

In this study, we took advantage of a change in vaccination policy in Sweden to investigate the link between BCG vaccination in infancy and COVID-19 cases, hospitalizations, and deaths using an RD approach.

Contrary to most studies on the question, we compared COVID-19 cases between 2 very similar groups of people from the same country. This allowed us to eliminate all confounders linked to cross-country comparisons, and confounders such as sex or socioeconomic status, which are often present in observational studies that do not rely on a quasi-experimental design, unlike ours. Another strength of this study is its statistical precision. Since we could gather nationwide data in a country where COVID-19 rates are high, we were able to reject fairly small effects of the BCG vaccine. Achieving a comparable statistical precision in an RCT would require an unrealistically large sample. Even with a COVID-19 hospitalization rate of 0.5%, as among the elderly Swedish population, an RCT that could reject BCG effects larger than 25% of the baseline hospitalization rate, as in our study, would require including approximately 15 000 participants.

While previous studies mostly addressed differences in BCG vaccination policies but did not account for differences in actual BCG coverage, we worked with 2 populations with well-documented and very different vaccination rates. The termination of the universal BCG vaccination program in Sweden had dramatic effects on the BCG coverage rate. Based on nationwide reports on the vaccination status of children younger than 7 years sent to the National Bacteriological Laboratory in 1981, 92% of children born in 1974 got vaccinated, compared with 26% of those born in 1975 and less than 2% of those born between 1976 and 1980 [16]. Among children born in 1975, most of the vaccinated children were born in the first quarter of the year, when the universal vaccination policy was still in place [16]. Prior to that, Sweden had already eliminated its revaccination program at 7 years of age in 1965. Sweden also stopped its revaccination program at 15 years of age in 1986, 4 years before the 1975 cohort turned 15 years old. Finally, Sweden stopped its vaccination program for conscripts in 1979, long before the 1975 cohort would perform its military service [23]. Overall, children born before 1 April 1975 benefited from a BCG vaccination policy at birth, while children born after that did not benefit from any BCG vaccination policy.

This being said, there are a number of limitations to this work to bear in mind. As in many other countries, the Folkhälsomyndigheten case count probably underestimates the true number of cases, because it only includes cases confirmed by a laboratory test. Its death count probably underestimates the true number of COVID-19 deaths as well, as it only includes deaths where a COVID-19 diagnosis has been confirmed during the past 30 days. However, these limitations are common to all studies on this question and are unlikely to much affect the results. Moreover, cases confirmed by a laboratory test are arguably the more severe ones, given that our data cover a time period where testing resources were scarce in Sweden, as in many other countries.

Our computation of the change in the vaccination rate induced by the 1975 policy relies on the assumption that the BCG vaccination rate of foreign-born residents is the same just before and just after April 1975. However, this is a reasonable assumption, since no other European country changed its BCG vaccination policy in 1975.

Furthermore, RD estimates only apply to units close to the eligibility threshold. For instance, this study estimates the effect of universal BCG vaccination for individuals born around 1 April 1975, who are in their mid-40s during the COVID-19 pandemic, and cannot be generalized to the entire population. It seems reasonable, however, to speculate that our findings hold true for older people since most studies assessing the long-term heterologous effects of BCG vaccination show they tend to fade rather than increase over time [5, 6]. Two studies argue that BCG vaccination at birth could have larger effects against COVID-19 in older individuals, those most at risk during the pandemic, than in middle-aged individuals [7, 11]. Based on the literature, we estimate that this is very unlikely to be the case.

Moreover, this study does not measure the COVID-19 immunity benefit conferred by a recent BCG vaccination, as individuals born just before Q2-1975 were vaccinated 45 years ago. The RCTs currently underway will show whether the protective effect of a recent BCG vaccination differs from the effect measured in this study.

Overall, this study shows that BCG vaccination at birth does not have a strong protective effect against COVID-19, at least in middle-aged individuals. Thus, it seems that BCG childhood vaccination policies cannot account for the differences in the severity of the pandemic across countries, as had been hypothesized by prior studies [7, 9, 10]. This advocates for a strict adherence to the World Health Organization’s recommendation for the vaccine to infants outside of clinical trials [24], and for restraint from starting new clinical trials on this question. The former point is of particular importance for a vaccine whose lengthy production process regularly leads to worldwide shortages, with dire consequences for children from countries with a high prevalence of tuberculosis [25]. While RCTs will complement this study by measuring the effect of a recent vaccination, this study comes much before results of the RCTs will be made available, and with a greater precision. It exemplifies the potential of leveraging past medical policies and statistical techniques designed using the social sciences to answer current medical questions.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. The authors are grateful to Folkhälsomyndigheten, the Public Health Agency of Sweden, for providing the data used in this study and answering all our questions. They are also grateful to Martin Berlin, Johannes Haushofer, Jerker Jonsson, Ellen Lundqvist, Kyle Meng, Robert Östling, Andrew Oswald, Moa Rehn, and Gonzalo Vazquez-Bare for their help.

Potential conflicts of interest. The authors: No reported conflicts of interest. Both authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest.

References