Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2019 Mar;12(3):e005010.
doi: 10.1161/CIRCOUTCOMES.118.005010.

Clinical Value of Predicting Individual Treatment Effects for Intensive Blood Pressure Therapy

Tony Duan  1 , Pranav Rajpurkar  1 , Dillon Laird  1 , Andrew Y Ng  1 , Sanjay Basu  2
Affiliations
Comparative Study

Clinical Value of Predicting Individual Treatment Effects for Intensive Blood Pressure Therapy

Tony Duan et al. Circ Cardiovasc Qual Outcomes. 2019 Mar.

Retraction in

Abstract

Background: The absolute risk reduction (ARR) in cardiovascular events from therapy is generally assumed to be proportional to baseline risk-such that high-risk patients benefit most. Yet newer analyses have proposed using randomized trial data to develop models that estimate individual treatment effects. We tested 2 hypotheses: first, that models of individual treatment effects would reveal that benefit from intensive blood pressure therapy is proportional to baseline risk; and second, that a machine learning approach designed to predict heterogeneous treatment effects-the X-learner meta-algorithm-is equivalent to a conventional logistic regression approach.

Methods and results: We compared conventional logistic regression to the X-learner approach for prediction of 3-year cardiovascular disease event risk reduction from intensive (target systolic blood pressure <120 mm Hg) versus standard (target <140 mm Hg) blood pressure treatment, using individual participant data from the SPRINT (Systolic Blood Pressure Intervention Trial; N=9361) and ACCORD BP (Action to Control Cardiovascular Risk in Diabetes Blood Pressure; N=4733) trials. Each model incorporated 17 covariates, an indicator for treatment arm, and interaction terms between covariates and treatment. Logistic regression had lower C statistic for benefit than the X-learner (0.51 [95% CI, 0.49-0.53] versus 0.60 [95% CI, 0.58-0.63], respectively). Following the logistic regression's recommendation for individualized therapy produced restricted mean time until cardiovascular disease event of 1065.47 days (95% CI, 1061.04-1069.35), while following the X-learner's recommendation improved mean time until cardiovascular disease event to 1068.71 days (95% CI, 1065.42-1072.08). Calibration was worse for logistic regression; it over-estimated ARR attributable to intensive treatment (slope between predicted and observed ARR of 0.73 [95% CI, 0.30-1.14] versus 1.06 [95% CI, 0.74-1.32] for the X-learner, compared with the ideal of 1). Predicted ARRs using logistic regression were generally proportional to baseline pretreatment cardiovascular risk, whereas the X-learner observed-correctly-that individual treatment effects were often not proportional to baseline risk.

Conclusions: Predictions for individual treatment effects from trial data reveal that patients may experience ARRs not simply proportional to baseline cardiovascular disease risk. Machine learning methods may improve discrimination and calibration of individualized treatment effect estimates from clinical trial data.

Clinical trial registration: URL: https://www.clinicaltrials.gov . Unique identifiers: NCT01206062; NCT00000620.

Keywords: blood pressure; calibration; cardiovascular disease; machine learning; risk factors.

PubMed Disclaimer

Figures

Figure 1:
Figure 1:
Conceptualization of the X-learner approach to detect heterogeneous treatment effects from trial data. We first estimate expectations of outcomes given predictors under control and treatment separately, then estimate imputed treatment effects (defined as the difference between expected outcomes and actual outcomes), and finally predict individual treatment effects as weighted averages of the estimated imputed treatment effects. For each step in the process we use random forests with 1000–2000 trees as base learners, and predict out-of-bag samples to prevent over-fitting.
Figure 2:
Figure 2:
Participants with uncensored outcomes at 3 years are grouped into predicted buckets of benefit (ARR > 0) and no benefit (ARR ≤ 0) via machine learning and conventional methods. Using these buckets, we bootstrap 95% confidence intervals for the observed absolute risk reduction (ARR) in each bucket, and calculate corresponding P values via the Wald test. The X-learner machine learning approach yielded more discriminative buckets than the conventional logistic regression approach, with the benefit bucket exhibiting higher observed ARR and the no benefit bucket exhibiting lower observed ARR.
Figure 3:
Figure 3:
Calibration plots for predicted absolute risk reduction versus observed absolute risk reduction (difference between intensive and standard treatment arms in cardiovascular event rates) using the X-learner machine learning approach and the conventional logistic regression approach, evaluated at quintiles of predicted absolute risk reduction and using Kaplan-Meier estimates to adjust for censoring.
Figure 4:
Figure 4:
Predicted risk reduction across deciles of predicted baseline CVD risk. We employed predictions of baseline risk calculated by ACC/AHA ASCVD risk score estimates to group trial participants into baseline risk deciles and compare the median and quartiles of predicted absolute risk reduction (ARR) from intensive blood pressure treatment across each decile. The logistic regression approach generally predicted absolute risk reduction to be proportional to baseline risk, whereas the X-learner machine learning approach predicted wider ranges of risk reduction per decile, not necessarily proportional to baseline risk.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Patel KK, Arnold SV, Chan PS, Tang Y, Pokharel Y, Jones PG, Spertus JA. Personalizing the Intensity of Blood Pressure Control: Modeling the Heterogeneity of Risks and Benefits From SPRINT (Systolic Blood Pressure Intervention Trial). Circ Cardiovasc Qual Outcomes. 2017;10:e003624. - PMC - PubMed
    1. Burke JF, Hayward RA, Nelson JP, Kent DM. Using Internally Developed Risk Models to Assess Heterogeneity in Treatment Effects in Clinical Trials. Circ Cardiovasc Qual Outcomes. 2014;7:163–169. - PMC - PubMed
    1. Yeh RW, Secemsky EA, Kereiakes DJ, Normand S-LT, Gershlick AH, Cohen DJ, Spertus JA, Steg PG, Cutlip DE, Rinaldi MJ, Camenzind E, Wijns W, Apruzzese PK, Song Y, Massaro JM, Mauri L. Development and Validation of a Prediction Rule for Benefit and Harm of Dual Antiplatelet Therapy Beyond 1 Year After Percutaneous Coronary Intervention. JAMA. 2016;315:1735–1749. - PMC - PubMed
    1. Yeboah J, Erbel R, Delaney JC, Nance R, Guo M, Bertoni AG, Budoff M, Moebus S, Jöckel K-H, Burke GL, Wong ND, Lehmann N, Herrington DM, Möhlenkamp S, Greenland P. Development of a new diabetes risk prediction tool for incident coronary heart disease events: the Multi-Ethnic Study of Atherosclerosis and the Heinz Nixdorf Recall Study. Atherosclerosis. 2014;236:411–417. - PMC - PubMed
    1. Dorresteijn JAN, Visseren FLJ, Ridker PM, Wassink AMJ, Paynter NP, Steyerberg EW, Graaf van der Y, Cook NR Estimating treatment effects for individual patients based on the results of randomised clinical trials. BMJ. 2011;343:d5888. - PMC - PubMed

Publication types

Substances

Associated data