Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2019 Mar;69(2):127-157.
doi: 10.3322/caac.21552. Epub 2019 Feb 5.

Artificial intelligence in cancer imaging: Clinical challenges and applications

Wenya Linda Bi  1 , Ahmed Hosny  2 , Matthew B Schabath  3 , Maryellen L Giger  4 , Nicolai J Birkbak  5   6 , Alireza Mehrtash  7   8 , Tavis Allison  9   10 , Omar Arnaout  1 , Christopher Abbosh  11   12 , Ian F Dunn  13 , Raymond H Mak  14 , Rulla M Tamimi  15 , Clare M Tempany  16 , Charles Swanton  17   18 , Udo Hoffmann  19 , Lawrence H Schwartz  20   21 , Robert J Gillies  22 , Raymond Y Huang  23 , Hugo J W L Aerts  24   25
Affiliations
Review

Artificial intelligence in cancer imaging: Clinical challenges and applications

Wenya Linda Bi et al. CA Cancer J Clin. 2019 Mar.

Abstract

Judgement, as one of the core tenets of medicine, relies upon the integration of multilayered data with nuanced decision making. Cancer offers a unique context for medical decisions given not only its variegated forms with evolution of disease but also the need to take into account the individual condition of patients, their ability to receive treatment, and their responses to treatment. Challenges remain in the accurate detection, characterization, and monitoring of cancers despite improved technologies. Radiographic assessment of disease most commonly relies upon visual evaluations, the interpretations of which may be augmented by advanced computational analyses. In particular, artificial intelligence (AI) promises to make great strides in the qualitative interpretation of cancer imaging by expert clinicians, including volumetric delineation of tumors over time, extrapolation of the tumor genotype and biological course from its radiographic phenotype, prediction of clinical outcome, and assessment of the impact of disease and treatment on adjacent organs. AI may automate processes in the initial interpretation of images and shift the clinical workflow of radiographic detection, management decisions on whether or not to administer an intervention, and subsequent observation to a yet to be envisioned paradigm. Here, the authors review the current state of AI as applied to medical imaging of cancer and describe advances in 4 tumor types (lung, brain, breast, and prostate) to illustrate how common clinical problems are being addressed. Although most studies evaluating AI applications in oncology to date have not been vigorously validated for reproducibility and generalizability, the results do highlight increasingly concerted efforts in pushing AI technology to clinical use and to impact future directions in cancer care.

Keywords: artificial intelligence; cancer imaging; clinical challenges; deep learning; radiomics.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Artificial Intelligence Applications in Medical Imaging as Applied to Common Cancers. Artificial intelligence tools can be conceptualized to apply to 3 broad categories of image‐based clinical tasks in oncology: 1) detection of abnormalities; 2) characterization of a suspected lesion by defining its shape or volume, histopathologic diagnosis, stage of disease, or molecular profile; and 3) determination of prognosis or response to treatment over time during monitoring. 2D indicates 2‐dimensional; 3D, 3‐dimensional; CNS, central nervous system.
Figure 2
Figure 2
Potential Enhanced Clinical Workflow With Artificial Intelligence (AI) Interventions. The traditional paradigm for patients with tumors entails initial radiologic diagnosis of a mass lesion, a decision to treat or observe based on clinical factors and patient preference, a definitive histopathologic diagnosis only after obtaining tissue, molecular genotyping in centers with such resources, and determination of clinical outcome only after the passage of time. In contrast, AI‐based interventions offer the potential to augment this clinical workflow and decision making at different stages of oncological care. Continuous feedback and optimization from measured outcomes may further improve AI systems.
Figure 3
Figure 3
Clinical Applications of Artificial Intelligence in Lung Cancer Screening on Detection of Incidental Pulmonary Nodules. Imaging analysis shows promise in predicting the risk of developing lung cancer on initial detection of an incidental lung nodule and in distinguishing indolent from aggressive lung neoplasms. PFS indicates progression‐free survival; ROC, receiver operating characteristic.
Figure 4
Figure 4
Applications of Noninvasive Monitoring During the Course of Cancer Evolution. Cancers share a common theme in developing intratumoral heterogeneity during their natural history. The presence of subclones (represented by different colors) confers significant implications in the response to treatment and may be difficult to capture through standard biopsies. Imaging and blood biomarkers during disease monitoring offer a potential technological solution for detecting the presence of intratumoral heterogeneity through space and time and thereby, perhaps, a direct change in therapeutic strategies.
Figure 5
Figure 5
Grad‐CAM Visualizations (Selvaraju et al 2017)127 for a Convolutional Neural Network (Chang et al 201896) Applied to 2 Examples of Isocitrate Dehydrogenase 1 (IDH1)/IDH2 Wild‐Type Glioblastoma and 2 Examples of IDH1‐Mutant Glioblastoma. Color maps are overlaid on original gadolinium‐enhanced, T1‐weighted magnetic resonance images, with red color weighted to the discriminative regions for IDH status classification.
Figure 6
Figure 6
Significant Associations Between Genomic Features and Radiomic Phenotypes in Breast Carcinoma Imaged With Magnetic Resonance Imaging. Gene‐set enrichment analysis (GSEA) and linear regression analysis were combined to associate genomic features, including microRNA (miRNA) expression, protein expression, and gene somatic mutations among others, with 6 categories of radiomic phenotypes. In this figure, each node represents a genomic feature or a radiomic phenotype. Each line is an identified statistically significant association, whereas nonsignificant associations are not depicted. Node size is proportional to its connectivity relative to other nodes in the category. Reprinted with permission from Maryellen L. Giger, University of Chicago (Zhu Y, Li H, Guo W, et al. Deciphering genomic underpinnings of quantitative MRI‐based radiomic phenotypes of invasive breast carcinoma [serial online]. Sci Rep. 2015;5:17787.170).
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Aerts HJ, Velazquez ER, Leijenaar RT, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach [serial online]. Nat Commun. 2014;5:4006. - PMC - PubMed
    1. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer. 2018;18:500‐510. - PMC - PubMed
    1. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521;436‐444. - PubMed
    1. Rolnick D, Veit A, Belongie S, Shavit N. Deep learning is robust to massive label noise. arXiv:1705.10694 [cs.LG]; 2017. arxiv.org/abs/1705.10694. Accessed May 28, 2018
    1. Moeskops P, Wolterink JM, van der Velden BHM, et al. Deep learning for multi‐task medical image segmentation in multiple modalities In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W, eds. Medical Image Computing and Computer‐Assisted Intervention‐MICCAI 2016. 19th International Conference, Athens, Greece, October 17–21, 2016, Proceedings. Cham, Switzerland: Springer International Publishing AG; 2016:478‐486.

Publication types