Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 May;35(5):1170-81.
doi: 10.1109/TMI.2015.2482920. Epub 2015 Sep 28.

Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation

Holger R Roth, Le Lu, Jiamin Liu, Jianhua Yao, Ari Seff, Kevin Cherry, Lauren Kim, Ronald M Summers

Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation

Holger R Roth et al. IEEE Trans Med Imaging. 2016 May.

Abstract

Automated computer-aided detection (CADe) has been an important tool in clinical practice and research. State-of-the-art methods often show high sensitivities at the cost of high false-positives (FP) per patient rates. We design a two-tiered coarse-to-fine cascade framework that first operates a candidate generation system at sensitivities ∼ 100% of but at high FP levels. By leveraging existing CADe systems, coordinates of regions or volumes of interest (ROI or VOI) are generated and function as input for a second tier, which is our focus in this study. In this second stage, we generate 2D (two-dimensional) or 2.5D views via sampling through scale transformations, random translations and rotations. These random views are used to train deep convolutional neural network (ConvNet) classifiers. In testing, the ConvNets assign class (e.g., lesion, pathology) probabilities for a new set of random views that are then averaged to compute a final per-candidate classification probability. This second tier behaves as a highly selective process to reject difficult false positives while preserving high sensitivities. The methods are evaluated on three data sets: 59 patients for sclerotic metastasis detection, 176 patients for lymph node detection, and 1,186 patients for colonic polyp detection. Experimental results show the ability of ConvNets to generalize well to different medical imaging CADe applications and scale elegantly to various data sets. Our proposed methods improve performance markedly in all cases. Sensitivities improved from 57% to 70%, 43% to 77%, and 58% to 75% at 3 FPs per patient for sclerotic metastases, lymph nodes and colonic polyps, respectively.

PubMed Disclaimer

Figures

Fig. 1.
Fig. 1.
ConvNet applied to a 2.5D volume of interest extracted from a CT image. The number of convolutional filters, kernel sizes, and neural network connections for each layer are as shown. We use overlapping kernels with stride 2 during max-pooling.
Fig. 2.
Fig. 2.
Features are computed by convolving filter kernels over the input region of interest. The input image can be padded to produce convolution responses of the same size as the input image.
Fig. 3.
Fig. 3.
Some examples of filter responses (Right) after convolution with trained ConvNet kernels (Middle) of the first layer (showing an example of a sclerotic bone lesion in CT (Left).
Fig. 4.
Fig. 4.
CADe locations can be either observed as 2D image patches or using a 2.5D approach, that samples the image using three orthogonal views. Here, a lymph node in CT is shown as the input to our method.
Fig. 5.
Fig. 5.
Image patches are generated from CADe candidates using different scales, 2D/3D translations (along a random vector v) and rotations (by a random angle α) in the axial/3D plane (The example shows a sclerotic bone lesion in CT).
Fig. 6.
Fig. 6.
The first layer of 64 learned convolutional kernels of a ConvNet trained on medical CT images on each of three different CT imaging data sets: a) sclerotic metastases, b) lymph nodes and c) colonic polyps. The color coding in b) and c) illustrates the filters kernels used in each orthogonal view when using our 2.5D approach. The learned convolutional filters for sclerotic metastases in a) are using one-channel as input only (encoded in gray scale and learned from axial CT images). Here, complex higher order gradients, blobness and difference of Gaussian filters dominate. In b,c), the convolutional filters for lymph nodes or colonic polyps are three-channels (encoded in RGB and trained using three orthogonal CT views per example). Kernels learned from lymph nodes are mostly blobness and gradients of different orientations/channels in b). Colonic polyp kernels in c) are visually more diversified than the filters in b), especially with new “pointy” patterns probably resembling polyp intrusions from 3D colonic surfaces or tips.
Fig. 7.
Fig. 7.
Detection of sclerotic metastases: test probabilities of the ConvNet for being sclerotic metastases on ‘true’ sclerotic metastases candidate examples (1.0 equals 100% probability of representing a true positive) [21].
Fig. 8.
Fig. 8.
Detection of sclerotic metastases: test probabilities of the ConvNet for being sclerotic metastases on ‘false’ sclerotic metastases candidate examples (0.0 equals 100% probability of representing a false positive) [21].
Fig. 9.
Fig. 9.
Detection of sclerotic metastases: FROC curves for a 5-fold cross-validation using varying numbers of N random view ConvNet observations in testing of 59 patients (49 with sclerotic metastases and 10 normal controls). AUC values are computed for corresponding ROC curves [21].
Fig. 10.
Fig. 10.
Detection of sclerotic metastases: comparison of FROC curves of the initial bone lesion candidate generation (squares) compared to the final classification using N = 100 random view ConvNet observations (lines) for both training and testing cases. Results are computed using a 5-fold cross-validation in 59 patients (49 with sclerotic metastases and 10 normal controls) [21].
Fig. 11.
Fig. 11.
Detection of lymph nodes: test probabilities of the ConvNet for being a lymph node on ‘true’ (top box) and ‘false’ (bottom box) lymph node candidate examples [20].
Fig. 12.
Fig. 12.
Detection of lymph nodes: FROC curves for a 3-fold cross-validation using a varying number of N random view ConvNet observations in 176 patients. AUC values are computed for corresponding ROC curves. The previous performance by [25] is shown for comparison.
Fig. 13.
Fig. 13.
Comparison of the FROC performance of the previous method as candidate generation step using a random forest classifier [25] against an alternative second level classification approach using histogram of oriented gradients (HoG) [6] and the proposed 2.5D ConvNet approach using ConvNet observation on N = 100 random views.
Fig. 14.
Fig. 14.
Comparison of the FROC performance of when training a ConvNet with 2D, 2.5D and 3D inputs of the original (“ORIG”) or augmented (“AUG”) CT data. In the “ORIG” setting, 2D ConvNet shows the best generalized testing FROC result, followed by 3D and 2.5D ConvNets. The 2.5D approach using aggregation of random observations (“AUG”) in both training (Left) and testing (Right), out-performs both 2D and 3D approaches on the original data at the 3 FPs/patient level. The 2.5D ConvNet trained on augmented data overall performs comparably to a more computationally expensive 3D ConvNet approach on augmented 3D inputs. In brief, the evaluated 2.5D “AUG” ConvNet is chosen as the best trade-off lymph node detection model between effectiveness and efficiency.
Fig. 15.
Fig. 15.
Detection of colonic polyps: FROC curves for different polyp sizes, using up to N = 40 random view ConvNet observations in 792 testing CT colonography patients.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. W. H. Organization, Cancer Fact shee N297. WHO, 2014.
    1. Msaouel P, Pissimissis N, Halapas A, and Koutsilieris M, “Mechanisms of bone metastasis in prostate cancer: clinical implications,” Best Practice & Research Clinical Endocrinology & Metabolism, vol. 22, no. 2, pp. 341–355, 2008. - PubMed
    1. Wiese T, Yao J, Burns JE, and Summers RM, “Detection of sclerotic bone metastases in the spine using watershed algorithm and graph cut,” in SPIE Med. Imag, pp. 831512–831512, 2012.
    1. Burns JE, Yao J, Wiese TS, Muñoz HE, Jones EC, and Summers RM, “Automated detection of sclerotic metastases in the thoracolumbar spine at CT,” Radiology, vol. 268, no. 1, pp. 69–78, 2013. - PMC - PubMed
    1. Hammon M, Dankerl P, Tsymbal A, Wels M, Kelm M, May M, Suehling M, Uder M, and Cavallaro A, “Automatic detection of lytic and blastic thoracolumbar spine metastases on computed tomography,” European radiology, vol. 23, no. 7, pp. 1862–1870, 2013. - PMC - PubMed