Skip to main page content
U.S. flag

An official website of the United States government

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Mar 13;24(1):66.
doi: 10.1186/s12874-024-02187-5.

Assessing the properties of patient-specific treatment effect estimates from causal forest algorithms under essential heterogeneity

John M Brooks  1   2 , Cole G Chapman  3   4 , Brian K Chen  5   4 , Sarah B Floyd  4   6 , Neset Hikmet  4   7
Affiliations

Assessing the properties of patient-specific treatment effect estimates from causal forest algorithms under essential heterogeneity

John M Brooks et al. BMC Med Res Methodol. .

Abstract

Background: Treatment variation from observational data has been used to estimate patient-specific treatment effects. Causal Forest Algorithms (CFAs) developed for this task have unknown properties when treatment effect heterogeneity from unmeasured patient factors influences treatment choice - essential heterogeneity.

Methods: We simulated eleven populations with identical treatment effect distributions based on patient factors. The populations varied in the extent that treatment effect heterogeneity influenced treatment choice. We used the generalized random forest application (CFA-GRF) to estimate patient-specific treatment effects for each population. Average differences between true and estimated effects for patient subsets were evaluated.

Results: CFA-GRF performed well across the population when treatment effect heterogeneity did not influence treatment choice. Under essential heterogeneity, however, CFA-GRF yielded treatment effect estimates that reflected true treatment effects only for treated patients and were on average greater than true treatment effects for untreated patients.

Conclusions: Patient-specific estimates produced by CFAs are sensitive to why patients in real-world practice make different treatment choices. Researchers using CFAs should develop conceptual frameworks of treatment choice prior to estimation to guide estimate interpretation ex post.

Keywords: Causal Forest Algorithm (CFA); Linear probability estimators; Machine learning; Simulation modeling; Treatment effect estimation.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Directed Acyclic Graph (DAG) Describing the Conceptual Framework for the Simulation Model in which Patient Factors Affecting Treatment Effectiveness Affect Treatment Choice through Decision Maker Knowledge
Fig. 2
Fig. 2
Distribution of True Absolute Treatment Effects (TEi) Used in All Eleven Simulated Populations
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Patient Centered Outcomes Research Institute. Our Programs. https://www.pcori.org/about-us/our-programs. Published 2017. Accessed 20 Mar 2019.
    1. Selby JV, Whitlock EP, Sherman KS, Slutsky JR. The Role of Comparative Effectiveness Research. In: Gallin JL, Ognibene FP, Johnson LL, editors. Principles and Practice of Clinical Research. 4. London, UK: Elisevier; 2018. pp. 269–292.
    1. Selby JV, Beal AC, Frank L. The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. Jama-J Am Med Assoc. 2012;307(15):1583–1584. doi: 10.1001/jama.2012.500. - DOI - PubMed
    1. Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004;82(4):661–687. doi: 10.1111/j.0887-378X.2004.00327.x. - DOI - PMC - PubMed
    1. Lohr KN, Eleazer K, Mauskopf J. Health policy issues and applications for evidence-medicine and clinical practice guidelines. Health Policy. 1998;46:1–19. doi: 10.1016/S0168-8510(98)00044-X. - DOI - PubMed

LinkOut - more resources