Interleukin‐1 blocking agents for treating COVID‐19

Critical outcomes

Important outcomes

1. Time to clinical improvement

2. Time to WHO Clinical Progression Score of level 7 or above

3. Time to death

Electronic searches

We searched the following databases on 5 November 2021.

1. The L‐OVE platform (app.iloveevidence.com/covid19), every working day since 7 September 2020.

2. The Cochrane COVID‐19 Study Register (covid-19.cochrane.org/), weekly since 7 September 2020.

Additional sources for the COVID‐NMA searched prior to September 2020 are reported elsewhere (Boutron 2020b). 

If no peer‐reviewed publication was available, we included and extracted the preprint. We recognise that preprints are not peer‐reviewed and are living documents that can be updated or published. We developed a preprint tracker in collaboration with a research team from the French National Centre for Scientific Research, which systematically informs us when a preprint is updated or published (Cabanac 2021). As soon as an update was identified, we recorded the data not available in the initial report. We checked the data for discrepancies against that already extracted and extracted data from the most recent report in case of discrepancies. We updated the analysis if needed. 

Searching other resources

We searched the following trial registries for unpublished and ongoing trials on 3 November 2021.

• The WHO ICTRP (www.who.int/ictrp/en/) to identify ongoing and completed clinical trials on COVID‐19. We used the List By Health Topic: 2019‐nCoV / COVID‐19 filter and retrieved all studies identified.

• The European Medicines Agency (EMA) clinical data website (clinicaldata.ema.europa.eu/web/cdp/home) to identify trials submitted to the EMA, and searched for the Clinical Study Report of eligible trials.

• The US Food and Drug Administration (FDA) website to identify FDA approval trials (www.fda.gov/emergency-preparedness-and-response/counterterrorism-and-emerging-threats/coronavirus-disease-2019-covid-19).

• The Retraction Watch Database for retracted trials (retractionwatch.com/retracted-coronavirus-covid-19-papers/).

Of note, we did not check the references of reports included as the living search process identifies COVID‐19 trial records prospectively from the point of trial registration. 

Selection of studies

We used an Excel spreadsheet to document search dates and numbers of citations identified. Two review authors independently screened the records and abstracts in duplicate using Rayyan (Ouzzani 2016).  We resolved discrepancies on exclusion and screening of full texts by consensus between both reviewers or involving a third reviewer. We recorded reasons for exclusion for all studies excluded after full text review.

Data extraction and management

Two review authors independently read each preprint, peer‐reviewed publication, protocol, or other study reports, evaluated the completeness of the data availability, and assessed the risk of bias. We used a specific structured online data extraction form. The online tool automatically identified any discrepancies, and both review authors involved in the data extraction discussed these to reach a consensus.

The information we extracted included study characteristics (such as first author, publication year and journal, funding source), number of participants randomised, participant characteristics (e.g. severity of clinical presentation), comorbidities, co‐interventions, intervention details (e.g. dose, schedule), outcome measures, and risk of bias assessment.

We systematically contacted the trial authors to ask them for supplementary information unavailable from the trial reports (Appendix 4). The WHO team with whom we are working requested these data by sending a personalised email. We classified disease severity as described below, according to clinical status or clinical management. This classification relies on existing classification and clinical expertise (WHO 2020a; WHO 2020b). We considered the description of eligibility criteria as well as the baseline characteristics of participants, and classified the severity as follows.

1. Mild disease ambulatory: 'outpatients' whose clinical symptoms are mild with no sign of pneumonia on imaging.

2. Mild disease: clinical symptoms requiring hospitalisation but no need for supplemental oxygen.

3. Moderate disease: fever and respiratory symptoms with radiological findings of pneumonia and requiring standard oxygen therapy O₂ (3 to 5 L/min).

4. Severe disease: meeting any of the following criteria:

   1. respiratory distress (≥ 30 breaths/min);

   2. oxygen saturation ≤ 93% at rest in ambient air or oxygen saturation ≤ 97% with O₂ > 5 L/min;

   3. PaO₂/FiO₂ ≤ 300 mmHg (1 mmHg = 0.133 kPa). PaO₂/FiO₂ in high‐altitude areas (> 1000 metres above sea level) is corrected by the following formula: PaO₂/FiO₂ x (atmospheric pressure (mmHg)/760);

   4. people hospitalised on non‐invasive ventilation (NIV)/high flow nasal oxygen (HFNO).

5. Critical disease: cases meeting the following criteria:

   1. respiratory failure requiring invasive mechanical ventilation;

   2. shock;

   3. other organ failure requiring intensive care unit care.

When no data related to these classifications were available, we requested the information from the authors.

For dichotomous outcomes, we calculated the relative risk (RR) with 95% confidence intervals (CIs) as a measure of effect. We extracted the number of events and the number of total participants in each trial arm. For time‐to‐event outcomes, we extracted the hazard ratio (HR) with 95% CIs. When these were not provided, we attempted to obtain them using the tools provided in Tierney 2007. For time‐to‐event outcomes, when several analyses were reported, we extracted results obtained from the intention‐to‐treat (ITT) analysis whenever these were available. If ITT results were not available, we extracted results from any modified ITT analyses. 

Assessment of risk of bias in included studies

We assessed the trials using the Cochrane Risk of Bias 2 (RoB 2) tool for RCTs (Sterne 2019).

The Cochrane RoB 2 tool is structured into five domains:

1. risk of bias arising from the randomisation process;

2. risk of bias due to deviations from intended interventions;

3. risk of bias due to missing outcome data;

4. risk of bias in the measurement of the outcome;

5. risk of bias in the selection of the reported result.

A series of 'signalling questions' elicit information relevant to risk of bias assessment within each domain. The response options to the signalling questions are: 'yes'; 'probably yes'; 'probably no'; 'no'; and 'no information'. A risk of bias judgement for each domain is generated by an algorithm, based on answers to the signalling questions. Judgement can be 'low', 'some concerns' or 'high' risk of bias. Overall risk of bias is considered 'low' if all domains are at 'low risk'; 'some concerns' if at least one domain has 'some concerns' and no domain at 'high' risk of bias; and 'high' if at least one domain is at 'high risk'.

We assessed the risk of bias for all critical and important outcomes.

In the context of this review, we are interested in quantifying the effect of assignment to the interventions at baseline, regardless of whether the interventions were received as intended (the ITT effect).

The Cochrane Bias Methods Group developed training materials on the risk of bias assessment tool RoB 2, which is used by the systematic reviewers participating in data extraction and risk of bias assessment for the COVID‐NMA platform (available upon request).

We recorded judgements for each domain and time point by using an online data extraction tool.

Two review authors independently assessed the risk of bias of each study at the outcome level, with consensus in case of disagreement. Review authors had epidemiological training or were members of the Cochrane Response team. They were trained using the material developed by the Cochrane Bias Methods Group. Each review author independently assessed the included manuscripts and used signalling questions for each bias domain, which was fed into the related algorithm to obtain a judgement. Both review authors recorded their judgement and support for judgement, but not their answers to signalling questions. For the consensus, all disagreements in judgement were identified and discussed until consensus was achieved. If needed, a third review author was involved.

To ensure standardisation of judgement and justification, the review authors, as well as the COVID‐NMA core team, revised the assessments/support for judgement.

Unit of analysis issues

We treated comparisons from multi‐arm or platform trials as independent two‐arm trials since we did not pool comparisons of different drugs in the same meta‐analysis. We did not identify any cross‐over or cluster‐randomised trials. If we do identify eligible cluster‐randomised trials in future updates of the review, we will extract results that properly account for the cluster design (such as based on a multilevel model or on generalised estimating equations). If such an analysis is not reported, we will try to obtain an estimate of the intraclass correlation coefficient and calculate data required for the meta‐analyses, taking the design effect into consideration.

Dealing with missing data

For missing outcome data, we extracted the number of participants who dropped out before completing the trial and how trial authors handled missing outcome data. In our primary analysis for the critical outcomes, we followed a conservative approach assuming that participants with missing outcome data did not experience the event of interest. Hence, we calculated all RRs with the number of participants randomised in each group in the denominator. We also conducted sensitivity analyses to assess the potential impact of missing outcome data on the results by using an available case analysis with the number of participants analysed (e.g. only participants without missing outcome data or only participants who received treatment) in the denominator (see below: Sensitivity analyses).

Assessment of heterogeneity

We generated descriptive statistics for both the trial and population characteristics. We examined the distribution of important clinical and methodological variables (e.g. age, disease severity, pre‐existing conditions and comorbidities, location). We used visual inspection of forest plots, the I² statistic and the magnitude of between‐study variance (Tau²) to estimate the level of heterogeneity. In this review, we did not use prediction intervals (the interval within which the effect of a future trial is expected to lie (Riley 2011)), or comparison with appropriate empirical distributions (Turner 2012), because of the small number of trials; however, these are planned for future updates if appropriate. 

Assessment of reporting bias

We assessed the risk of bias due to missing results in the synthesis according to the framework proposed in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2021). 

Searches in trial registries were used to identify any initiated, ongoing, or completed but not published trials meeting this review's eligibility criteria. We contacted all responsible parties to obtain an updated report of the results included in the trial registry. For published trials, we contacted the corresponding authors to get the missing data.

We checked whether the results of all our critical and important outcomes were reported as prespecified in the trial register. When registration was not prospective, we also checked the protocol or statistical analysis plan if available.

When any trial results were not available, we used a matrix indicating the availability of trial results, as recommended in the Cochrane Handbook (Higgins 2021; Kirkham 2018). 

We checked whether the results were unavailable because of the P value, magnitude, or direction of the result. We considered the risk of bias due to missing results if one specified outcome of the registry was missing from the main report because of these reasons.

Due to the small number of trials, we could not assess the potential for reporting bias across studies either graphically or statistically.

Data synthesis 

We have combined trials evaluating the same drug with standard care alone or with placebo comparators together under the same comparison. We included all eligible RCTs in the primary analysis, regardless of the risk of bias assessment. 

We did not combine trials assessing different drugs because these drugs have a different mode of action; anakinra is an interleukin‐1 receptor antagonist, while canakinumab is a human anti‐IL‐1β monoclonal antibody. 

We calculated the log RRs and their standard error for binary outcomes using the number of events and total participants in each arm, then we pooled the trial‐specific effect sizes. For time‐to‐event outcomes, we directly extracted the HRs and the respective 95% CIs from the trial reports and subsequently pooled them in the meta‐analysis.

For each direct comparison with at least two trials providing data, we presented effect estimates with 95% CIs. We used the random‐effects model as we anticipated clinical and methodological heterogeneity across trials. 

We conducted all analyses using our 'R‐shiny' application (available from covid-nma.com/pairwise_meta_analysis/), based on the 'metafor' package in R.

Subgroup analyses

Because of the low number of trials, we did not carry out the prespecified subgroup analyses. Had there been sufficient data, we would have performed subgroup analyses based on participant characteristics, timing of the treatment and disease severity (mild disease only, mixed disease (i.e. participants heterogeneous in terms of severity, participants with moderate disease and participants with severe disease) and critical disease only).

Sensitivity analyses

We performed sensitivity analyses by excluding trials with a high overall risk of bias and RCTs reported as preprint only. We also ran the analyses using the number of participants analysed instead of those randomised (Chaimani 2018; Mavridis 2015; Mavridis 2018; White 2008). 

Summary of findings and assessment of the certainty of the evidence

To evaluate the confidence in the results of the pairwise comparisons for critical and important outcomes, we used the GRADE approach (Schünemann 2019). We prepared summary of findings tables using the GRADEpro GDT tool to present estimated relative and absolute risks for critical and important outcomes. One review author assessed the overall certainty of the evidence for each outcome, and another review author cross‐checked it using the GRADE classification (Schünemann 2019).

Results of the search

The results of searches are detailed in Figure 1. We retrieved a total of 48,043 references by searching electronic bibliographic databases, after excluding duplicates; 699 were eligible for full‐text screening. We included seven reports of six RCTs (five published in peer‐reviewed journals and one reported as a preprint) evaluating IL‐1 blocking agents. Four RCTs evaluated anakinra (Declercq COV‐AID 2021; Derde REMAP‐CAP 2021; Kyriazopoulou SAVE‐MORE 2021; Mariette CORIMUNO‐19 Collaborative 2021), and two evaluated canakinumab (Caricchio CAN‐COVID 2021; Cremer Three C Study 2021). 

We did not identify any retracted articles. The search of the US Food and Drug Administration website did not retrieve any reports. The search in registries identified 22 registered trials, of which 16 had no results available. 

We also contacted the named contacts for trials registered with no associated publication of results. The responses are detailed in Appendix 8. 

Overall, we identified 19 RCTs of anakinra (three published in peer‐reviewed journals, one reported as a preprint, three completed with no results available, four terminated, five ongoing, three not recruiting); three RCTs of canakinumab (two published in peer‐reviewed journals, one completed); and no RCTs of rilonacept either published or registered. 

Included studies

Excluded studies 

We excluded a total of 692 reports; 571 were RCTs evaluating other interventions for COVID‐19 and consequently included in the COVID‐NMA platform (covid-nma.com); 121 full‐text reports were excluded from the COVID‐NMA platform. We provided details on the reasons for exclusions in Appendix 5.

Ongoing studies

We identified 16 trials with no published results from registries. More details are available in Appendix 6 and Appendix 7.

Risk of bias arising from the randomisation process

Randomisation was described adequately and was appropriate in all trials (Caricchio CAN‐COVID 2021; Cremer Three C Study 2021; Declercq COV‐AID 2021; Derde REMAP‐CAP 2021; Kyriazopoulou SAVE‐MORE 2021; Mariette CORIMUNO‐19 Collaborative 2021). 

Risk of bias due to deviations from intended interventions

We judged the risk of bias due to deviation from intended interventions to be low in five trials (Caricchio CAN‐COVID 2021; Cremer Three C Study 2021; Declercq COV‐AID 2021; Kyriazopoulou SAVE‐MORE 2021; Mariette CORIMUNO‐19 Collaborative 2021). We had some concerns about this domain for all outcomes in one trial (Derde REMAP‐CAP 2021), because the study was not blinded and no information was provided on the use of co‐interventions after the trial started. We did not downgrade any trial for the population of analysis.

Risk of bias due to missing outcome data

We judged the risk of bias due to incomplete outcome data to be low for five trials for all outcomes, since there was either no missing data (Cremer Three C Study 2021; Declercq COV‐AID 2021), or a low amount of missing data (Caricchio CAN‐COVID 2021; Derde REMAP‐CAP 2021; Mariette CORIMUNO‐19 Collaborative 2021). For one trial (Kyriazopoulou SAVE‐MORE 2021), we judged the risk of bias due to incomplete data to be low for some outcomes (all‐cause mortality D28, clinical improvement D28, adverse events and serious adverse events), as these had a low amount of missing data. However, for other outcomes with longer follow‐up and time‐to‐event outcomes, we had some concerns due to an important amount of missing data.

Risk of bias in the measurement of the outcome

In three open‐label trials (Declercq COV‐AID 2021; Derde REMAP‐CAP 2021; Mariette CORIMUNO‐19 Collaborative 2021), we judged the risk of bias to be low for all observer‐reported outcomes not involving clinical judgement (i.e. all‐cause mortality and WHO Clinical Progression Score of level 7 or above). In contrast, we had some concerns of risk of bias for the outcomes that could potentially be influenced by knowledge of the intervention assignment (i.e. clinical improvement, adverse events and serious adverse events).

In the three trials in which the outcome assessors were blinded (Caricchio CAN‐COVID 2021; Cremer Three C Study 2021; Kyriazopoulou SAVE‐MORE 2021), we judged the risk of bias to be low for all outcomes. 

Risk of bias in the selection of the reported results

All trials had prospective registries and protocols available (Caricchio CAN‐COVID 2021; Cremer Three C Study 2021; Declercq COV‐AID 2021; Derde REMAP‐CAP 2021; Mariette CORIMUNO‐19 Collaborative 2021), or obtained upon request (Kyriazopoulou SAVE‐MORE 2021), and we judged these to be low risk of bias for all reported outcomes. Conversely, in one trial, we rated the risk of bias in the selection of the reported results as 'some concerns' because the outcome 'clinical improvement D28' was not prespecified (Cremer Three C Study 2021). 

Bias due to missing results in the synthesis

We present a matrix indicating the availability of trial results for critical and important review outcomes in Appendix 10 and Appendix 11. There was no evidence of bias due to missing results, except for two trials that planned to assess adverse and serious adverse events but did not report the results (Declercq COV‐AID 2021; Derde REMAP‐CAP 2021). 

Anakinra versus standard of care/placebo

We report the certainty of the evidence for critical outcomes and important outcomes in Table 1. We report the effect sizes of the outcomes for this comparison in Appendix 12.

Canakinumab versus standard of care/placebo

We report the certainty evidence for critical and important outcomes in the Table 2. We report the effect sizes of the outcomes for this comparison in Appendix 14.

Investigation of heterogeneity

The limited number of RCTs that provided results and the absence of variation across trials in some variables, such as age and gender, prevented us from performing preplanned subgroup analyses. 

Sensitivity analysis 

Sensitivity analyses for publication status were only possible for the comparison of anakinra versus control for one outcome (all‐cause mortality D60 or above). Results were consistent when considering only trials reported as peer‐reviewed articles (Appendix 16). No important discrepancies in the summary results were observed when we used the number analysed in the RCTs instead of the number randomised as the denominator for anakinra and canakinumab (Appendix 17).