Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Oct;272(4):629-636.
doi: 10.1097/SLA.0000000000004133.

Portable Automated Surveillance of Surgical Site Infections Using Natural Language Processing: Development and Validation

Brian T Bucher  1 , Jianlin Shi  2 , Jeffrey P Ferraro  3 , David E Skarda  1   3 , Matthew H Samore  4   5 , John F Hurdle  2 , Adi V Gundlapalli  4 , Wendy W Chapman  2 , Samuel R G Finlayson  1
Affiliations

Portable Automated Surveillance of Surgical Site Infections Using Natural Language Processing: Development and Validation

Brian T Bucher et al. Ann Surg. 2020 Oct.

Abstract

Objectives: We present the development and validation of a portable NLP approach for automated surveillance of SSIs.

Summary of background data: The surveillance of SSIs is labor-intensive limiting the generalizability and scalability of surgical quality surveillance programs.

Methods: We abstracted patient clinical text notes after surgical procedures from 2 independent healthcare systems using different electronic healthcare records. An SSI detected as part of the American College of Surgeons' National Surgical Quality Improvement Program was used as the reference standard. We developed a rules-based NLP system (Easy Clinical Information Extractor [CIE]-SSI) for operative event-level detection of SSIs using an training cohort (4574 operative events) from 1 healthcare system and then conducted internal validation on a blind cohort from the same healthcare system (1850 operative events) and external validation on a blind cohort from the second healthcare system (15,360 operative events). EasyCIE-SSI performance was measured using sensitivity, specificity, and area under the receiver-operating-curve (AUC).

Results: The prevalence of SSI was 4% and 5% in the internal and external validation corpora. In internal validation, EasyCIE-SSI had a sensitivity, specificity, AUC of 94%, 88%, 0.912 for the detection of SSI, respectively. In external validation, EasyCIE-SSI had sensitivity, specificity, AUC of 79%, 92%, 0.852 for the detection of SSI, respectively. The sensitivity of EasyCIE-SSI decreased in clean, skin/subcutaneous, and outpatient procedures in the external validation compared to internal validation.

Conclusion: Automated surveillance of SSIs can be achieved using NLP of clinical notes with high sensitivity and specificity.

PubMed Disclaimer

Conflict of interest statement

CONFLICT OF INTEREST DISCLOSURE

Dr. Chapman reported consulting for IBM and serving on the Scientific Advisory Board of Health Fidelity. These companies had no role in the study. No other disclosures are reported.

Figures

FIGURE 1.
FIGURE 1.
Subgroup analysis of the difference in sensitivity (left) and specificity (right) of EasyCIE-SSI between the external validation cohort compared to the internal validation cohort by procedure characteristics. The error bars represent 95% Confidence Interval for each point estimate and the dotted line represents 0 or no difference between external and internal validation. Sensitivity was uniformly decreased in External Validation, with characteristics whose error bar excludes 0, for example, Clean Procedures, representing statistically significant decreases.
See this image and copyright information in PMC

Comment in

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Ko CY, Hall BL, Hart AJ, et al. The American College of Surgeons National Surgical Quality Improvement Program: achieving better and safer surgery. Jt Comm J Qual Patient Saf 2015; 41(5):199–204. - PubMed
    1. Hu QL, Liu JY, Hobson DB, et al. Best Practices in Data Use for Achieving Successful Implementation of Enhanced Recovery Pathway. J Am Coll Surg 2019; 229(6):626–632.e1. - PubMed
    1. Classen DC, Griffin FA, Berwick DM. Measuring Patient Safety in Real Time: An Essential Method for Effectively Improving the Safety of Care. Ann Intern Med 2017; 167(12):882–883. - PubMed
    1. McGee MF, Kreutzer L, Quinn CM, et al. Leveraging a Comprehensive Program to Implement a Colorectal Surgical Site Infection Reduction Bundle in a Statewide Quality Improvement Collaborative. Ann Surg 2019; 270(4):701–711. - PMC - PubMed
    1. Ju MH, Ko CY, Hall BL, et al. A comparison of 2 surgical site infection monitoring systems. JAMA Surg 2015; 150(1):51–7. - PubMed

Publication types