TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods

doi:10.1136/bmj-2023-078378

. 2024 Apr 16:385:e078378.

doi: 10.1136/bmj-2023-078378.

TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods

Gary S Collins¹, Karel G M Moons², Paula Dhiman¹, Richard D Riley^{3

4}, Andrew L Beam⁵, Ben Van Calster^{6

7}, Marzyeh Ghassemi⁸, Xiaoxuan Liu^{9

10}, Johannes B Reitsma², Maarten van Smeden², Anne-Laure Boulesteix¹¹, Jennifer Catherine Camaradou^{12

13}, Leo Anthony Celi^{14

15

16}, Spiros Denaxas^{17

18}, Alastair K Denniston^{4

9}, Ben Glocker¹⁹, Robert M Golub²⁰, Hugh Harvey²¹, Georg Heinze²², Michael M Hoffman^{23

24

25

26}, André Pascal Kengne²⁷, Emily Lam¹², Naomi Lee²⁸, Elizabeth W Loder^{29

30}, Lena Maier-Hein³¹, Bilal A Mateen^{17

32

33}, Melissa D McCradden^{34

35}, Lauren Oakden-Rayner³⁶, Johan Ordish³⁷, Richard Parnell¹², Sherri Rose³⁸, Karandeep Singh³⁹, Laure Wynants³⁹, Patricia Logullo¹

Affiliations

¹ Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK.
² Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands.
³ Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
⁴ National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK.
⁵ Department of Epidemiology, Harvard T H Chan School of Public Health, Boston, MA, USA.
⁶ Department of Development and Regeneration, KU Leuven, Leuven, Belgium.
⁷ Department of Biomedical Data Science, Leiden University Medical Centre, Leiden, Netherlands.
⁸ Department of Electrical Engineering and Computer Science, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
⁹ Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
¹⁰ University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
¹¹ Institute for Medical Information Processing, Biometry and Epidemiology, Faculty of Medicine, Ludwig-Maximilians-University of Munich and Munich Centre of Machine Learning, Germany.
¹² Patient representative, Health Data Research UK patient and public involvement and engagement group.
¹³ Patient representative, University of East Anglia, Faculty of Health Sciences, Norwich Research Park, Norwich, UK.
¹⁴ Beth Israel Deaconess Medical Center, Boston, MA, USA.
¹⁵ Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA.
¹⁶ Department of Biostatistics, Harvard T H Chan School of Public Health, Boston, MA, USA.
¹⁷ Institute of Health Informatics, University College London, London, UK.
¹⁸ British Heart Foundation Data Science Centre, London, UK.
¹⁹ Department of Computing, Imperial College London, London, UK.
²⁰ Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
²¹ Hardian Health, Haywards Heath, UK.
²² Section for Clinical Biometrics, Centre for Medical Data Science, Medical University of Vienna, Vienna, Austria.
²³ Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada.
²⁴ Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada.
²⁵ Department of Computer Science, University of Toronto, Toronto, ON, Canada.
²⁶ Vector Institute for Artificial Intelligence, Toronto, ON, Canada.
²⁷ Department of Medicine, University of Cape Town, Cape Town, South Africa.
²⁸ National Institute for Health and Care Excellence, London, UK.
²⁹ The BMJ, London, UK.
³⁰ Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
³¹ Department of Intelligent Medical Systems, German Cancer Research Centre, Heidelberg, Germany.
³² Wellcome Trust, London, UK.
³³ Alan Turing Institute, London, UK.
³⁴ Department of Bioethics, Hospital for Sick Children Toronto, ON, Canada.
³⁵ Genetics and Genome Biology, SickKids Research Institute, Toronto, ON, Canada.
³⁶ Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia.
³⁷ Medicines and Healthcare products Regulatory Agency, London, UK.
³⁸ Department of Health Policy and Center for Health Policy, Stanford University, Stanford, CA, USA.
³⁹ Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands.

PMID: 38626948
PMCID: PMC11019967
DOI: 10.1136/bmj-2023-078378

TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods

Gary S Collins et al. BMJ. 2024.

. 2024 Apr 16:385:e078378.

doi: 10.1136/bmj-2023-078378.

Authors

Affiliations

¹ Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK.
² Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands.
³ Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
⁴ National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK.
⁵ Department of Epidemiology, Harvard T H Chan School of Public Health, Boston, MA, USA.
⁶ Department of Development and Regeneration, KU Leuven, Leuven, Belgium.
⁷ Department of Biomedical Data Science, Leiden University Medical Centre, Leiden, Netherlands.
⁸ Department of Electrical Engineering and Computer Science, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
⁹ Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
¹⁰ University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
¹¹ Institute for Medical Information Processing, Biometry and Epidemiology, Faculty of Medicine, Ludwig-Maximilians-University of Munich and Munich Centre of Machine Learning, Germany.
¹² Patient representative, Health Data Research UK patient and public involvement and engagement group.
¹³ Patient representative, University of East Anglia, Faculty of Health Sciences, Norwich Research Park, Norwich, UK.
¹⁴ Beth Israel Deaconess Medical Center, Boston, MA, USA.
¹⁵ Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA.
¹⁶ Department of Biostatistics, Harvard T H Chan School of Public Health, Boston, MA, USA.
¹⁷ Institute of Health Informatics, University College London, London, UK.
¹⁸ British Heart Foundation Data Science Centre, London, UK.
¹⁹ Department of Computing, Imperial College London, London, UK.
²⁰ Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
²¹ Hardian Health, Haywards Heath, UK.
²² Section for Clinical Biometrics, Centre for Medical Data Science, Medical University of Vienna, Vienna, Austria.
²³ Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada.
²⁴ Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada.
²⁵ Department of Computer Science, University of Toronto, Toronto, ON, Canada.
²⁶ Vector Institute for Artificial Intelligence, Toronto, ON, Canada.
²⁷ Department of Medicine, University of Cape Town, Cape Town, South Africa.
²⁸ National Institute for Health and Care Excellence, London, UK.
²⁹ The BMJ, London, UK.
³⁰ Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
³¹ Department of Intelligent Medical Systems, German Cancer Research Centre, Heidelberg, Germany.
³² Wellcome Trust, London, UK.
³³ Alan Turing Institute, London, UK.
³⁴ Department of Bioethics, Hospital for Sick Children Toronto, ON, Canada.
³⁵ Genetics and Genome Biology, SickKids Research Institute, Toronto, ON, Canada.
³⁶ Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia.
³⁷ Medicines and Healthcare products Regulatory Agency, London, UK.
³⁸ Department of Health Policy and Center for Health Policy, Stanford University, Stanford, CA, USA.
³⁹ Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands.

PMID: 38626948
PMCID: PMC11019967
DOI: 10.1136/bmj-2023-078378

Erratum in

TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods.
[No authors listed] [No authors listed] BMJ. 2024 Apr 18;385:q902. doi: 10.1136/bmj.q902. BMJ. 2024. PMID: 38636956 Free PMC article. No abstract available.

Abstract

The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) statement was published in 2015 to provide the minimum reporting recommendations for studies developing or evaluating the performance of a prediction model. Methodological advances in the field of prediction have since included the widespread use of artificial intelligence (AI) powered by machine learning methods to develop prediction models. An update to the TRIPOD statement is thus needed. TRIPOD+AI provides harmonised guidance for reporting prediction model studies, irrespective of whether regression modelling or machine learning methods have been used. The new checklist supersedes the TRIPOD 2015 checklist, which should no longer be used. This article describes the development of TRIPOD+AI and presents the expanded 27 item checklist with more detailed explanation of each reporting recommendation, and the TRIPOD+AI for Abstracts checklist. TRIPOD+AI aims to promote the complete, accurate, and transparent reporting of studies that develop a prediction model or evaluate its performance. Complete reporting will facilitate study appraisal, model evaluation, and model implementation.

PubMed Disclaimer

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at https://www.icmje.org/disclosure-of-interest/ and declare: support from the funding bodies listed above for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work. GSC is a National Institute for Health and Care Research (NIHR) senior investigator, the director of the UK EQUATOR Centre, editor-in-chief of BMC Diagnostic and Prognostic Research, and a statistics editor for The BMJ. KGMM is director of Health Innovation Netherlands and editor-in-chief of BMC Diagnostic and Prognostic Research. RDR is an NIHR senior investigator, a statistics editor for The BMJ, and receives royalties from textbooks Prognosis Research in Healthcare and Individual Participant Data Meta-Analysis. AKD is an NIHR senior investigator. EWL is the head of research at The BMJ. BG is a part time employee of HeartFlow and Kheiron Medical Technologies and holds stock options with both as part of the standard compensation package. SR receives royalties from Springer for the textbooks Targeted Learning: Causal Inference for Observational and Experimental Data and Targeted Learning: Causal Inference for Complex Longitudinal Studies. JCC receives honorariums as a current lay member on the UK NICE covid-19 expert panel and a citizen partner on the COVID-END Covid-19 Evidence Network to support decision making; was a lay member on the UK NIHR AI AWARD panel in 2020-22 and is a current lay member on the UK NHS England AAC Accelerated Access Collaborative NHS AI Laboratory Evaluation Advisory Group; is a patient fellow of the European Patients’ Academy on Therapeutic Innovation and a EURORDIS rare disease alumni; reports grants from the UK National Institute for Health and Care Research, European Commission, UK Cell Gene Catapult, University College London, and University of East Anglia; reports patient speaker fees from MEDABLE, Reuters Pharma events, Patients as Partners Europe, and EIT Health Scandinavia; reports consultancy fees from Roche Global, GlaxoSmithKline, the FutureScience Group and Springer Healthcare (scientific publishing), outside of the scope of the present work; and is a strategic board member of the UK Medical Research Council IASB Advanced Pain Discovery Platform initiative, Plymouth Institute of Health, and EU project Digipredict Edge AI-deployed Digital Twins for covid-19 Cardiovascular Disease. ALB is a paid consultant for Generate Biomedicines, Flagship Pioneering, Porter Health, FL97, Tessera, FL85; has an equity stake in Generate Biomedicines; and receives research funding support from GlaxoSmithKline, National Heart, Lung, and Blood Institute, and National Institute of Diabetes and Digestive and Kidney Diseases. No other conflicts of interests with this specific work are declared.

Comment in

TRIPOD+AI: an updated reporting guideline for clinical prediction models.
Cohen JF, Bossuyt PMM. Cohen JF, et al. BMJ. 2024 Apr 16;385:q824. doi: 10.1136/bmj.q824. BMJ. 2024. PMID: 38626949 No abstract available.

Cited by

Deep Convolutional Neural Network for Accurate Classification of Myofibroblastic Lesions on Patch-Based Images.
Giraldo-Roldán D, Dos Santos GC, Araújo ALD, Nakamura TCR, Pulido-Díaz K, Lopes MA, Santos-Silva AR, Kowalski LP, Moraes MC, Vargas PA. Giraldo-Roldán D, et al. Head Neck Pathol. 2024 Oct 28;18(1):117. doi: 10.1007/s12105-024-01723-5. Head Neck Pathol. 2024. PMID: 39466448
Inflammatory Neuropathy Consortium base (INCbase): a protocol of a global prospective observational cohort study for the development of a prediction model for treatment response in chronic inflammatory demyelinating polyneuropathy.
Michael MR, Wieske L, Allen JA, Lunn MP, Doppler K, Tan CY, Koike H, Markvardsen LK, Kapoor M, Hsieh ST, Nobile-Orazio E, Jacobs BC, Rajabally YA, Basta I, Ripellino P, Querol L, Eftimov F; INCbase Consortium. Michael MR, et al. BMC Neurol. 2024 Oct 25;24(1):415. doi: 10.1186/s12883-024-03903-w. BMC Neurol. 2024. PMID: 39455929 Free PMC article.
Machine learning algorithms: why the cup occasionally appears half-empty.
Woodman RJ. Woodman RJ. Eur J Clin Nutr. 2024 Oct 23. doi: 10.1038/s41430-024-01529-2. Online ahead of print. Eur J Clin Nutr. 2024. PMID: 39443687 No abstract available.
Development and validation of a nomogram to predict the risk of adjacent segment disease after transforaminal lumbar interbody fusion in patients with lumbar degenerative diseases.
Yu M, Ye J, Xing H, Yin H, Jiang T, Liu M, Li C. Yu M, et al. J Orthop Surg Res. 2024 Oct 22;19(1):680. doi: 10.1186/s13018-024-05170-4. J Orthop Surg Res. 2024. PMID: 39438978 Free PMC article.

See all "Cited by" articles

References

1. van Smeden M, Reitsma JB, Riley RD, Collins GS, Moons KG. Clinical prediction models: diagnosis versus prognosis. J Clin Epidemiol 2021;132:142-5. 10.1016/j.jclinepi.2021.01.009 - DOI - PubMed
1. Nashef SA, Roques F, Sharples LD, et al. . EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734-45. 10.1093/ejcts/ezs043 - DOI - PubMed
1. Gail MH, Brinton LA, Byar DP, et al. . Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst 1989;81:1879-86. 10.1093/jnci/81.24.1879 - DOI - PubMed
1. D’Agostino RB, Sr, Vasan RS, Pencina MJ, et al. . General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117:743-53. 10.1161/CIRCULATIONAHA.107.699579 - DOI - PubMed
1. Steyerberg EW, Mushkudiani N, Perel P, et al. . Predicting outcome after traumatic brain injury: development and international validation of prognostic scores based on admission characteristics. PLoS Med 2008;5:e165. 10.1371/journal.pmed.0050165 - DOI - PMC - PubMed

MeSH terms

Actions
Actions
Actions
Actions
Actions

Grants and funding

WT_/Wellcome Trust/United Kingdom

LinkOut - more resources

Full Text Sources

[1] van Smeden M, Reitsma JB, Riley RD, Collins GS, Moons KG. Clinical prediction models: diagnosis versus prognosis. J Clin Epidemiol 2021;132:142-5. 10.1016/j.jclinepi.2021.01.009 - DOI - PubMed

[2] van Smeden M, Reitsma JB, Riley RD, Collins GS, Moons KG. Clinical prediction models: diagnosis versus prognosis. J Clin Epidemiol 2021;132:142-5. 10.1016/j.jclinepi.2021.01.009 - DOI - PubMed

[3] Nashef SA, Roques F, Sharples LD, et al. . EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734-45. 10.1093/ejcts/ezs043 - DOI - PubMed

[4] Nashef SA, Roques F, Sharples LD, et al. . EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734-45. 10.1093/ejcts/ezs043 - DOI - PubMed

[5] Gail MH, Brinton LA, Byar DP, et al. . Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst 1989;81:1879-86. 10.1093/jnci/81.24.1879 - DOI - PubMed

[6] Gail MH, Brinton LA, Byar DP, et al. . Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst 1989;81:1879-86. 10.1093/jnci/81.24.1879 - DOI - PubMed

[7] D’Agostino RB, Sr, Vasan RS, Pencina MJ, et al. . General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117:743-53. 10.1161/CIRCULATIONAHA.107.699579 - DOI - PubMed

[8] D’Agostino RB, Sr, Vasan RS, Pencina MJ, et al. . General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117:743-53. 10.1161/CIRCULATIONAHA.107.699579 - DOI - PubMed

[9] Steyerberg EW, Mushkudiani N, Perel P, et al. . Predicting outcome after traumatic brain injury: development and international validation of prognostic scores based on admission characteristics. PLoS Med 2008;5:e165. 10.1371/journal.pmed.0050165 - DOI - PMC - PubMed

[10] Steyerberg EW, Mushkudiani N, Perel P, et al. . Predicting outcome after traumatic brain injury: development and international validation of prognostic scores based on admission characteristics. PLoS Med 2008;5:e165. 10.1371/journal.pmed.0050165 - DOI - PMC - PubMed

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods

Affiliations

TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods

Authors

Affiliations

Erratum in

Abstract

Conflict of interest statement

Comment in

Similar articles

Cited by

References

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Erratum in

Abstract

Conflict of interest statement

Comment in

Similar articles

Cited by

References

MeSH terms

Related information

Grants and funding

LinkOut - more resources

Full Text Sources