Diagnosing sepsis is subjective and highly variable: a survey of intensivists using case vignettes
- PMID: 27048508
- PMCID: PMC4822273
- DOI: 10.1186/s13054-016-1266-9
Diagnosing sepsis is subjective and highly variable: a survey of intensivists using case vignettes
Abstract
Background: Sepsis is the focus of national quality improvement programs and a recent public reporting measure from the Centers for Medicare and Medicaid Services. However, diagnosing sepsis requires interpreting nonspecific signs and can therefore be subjective. We sought to quantify interobserver variability in diagnosing sepsis.
Methods: We distributed five case vignettes of patients with suspected or confirmed infection and organ dysfunction to a sample of practicing intensivists. Respondents classified cases as systemic inflammatory response syndrome, sepsis, severe sepsis, septic shock, or none of the above. Interobserver variability was calculated using Fleiss' κ for the five-level classification, and for answers dichotomized as severe sepsis/septic shock versus not-severe sepsis/septic shock and any sepsis category (sepsis, severe sepsis, or septic shock) versus not-sepsis.
Results: Ninety-four physicians completed the survey. Most respondents (88%) identified as critical care specialists; other specialties included pulmonology (39%), anesthesia (19%), surgery (9%), and emergency medicine (9%). Respondents had been in practice for a median of 8 years, and 90% practiced at academic hospitals. Almost all respondents (83%) felt strongly or somewhat confident in their ability to apply the traditional consensus sepsis definitions. However, overall interrater agreement in sepsis diagnoses was poor (Fleiss' κ 0.29). When responses were dichotomized into severe sepsis/septic shock versus not-severe sepsis/septic shock or any sepsis category versus not-sepsis, agreement was still poor (Fleiss' κ 0.23 and 0.18, respectively). Seventeen percent of respondents classified one of the five cases as severe sepsis/septic shock, 27.7% rated two cases, 33.0% respondents rated three cases, 19.2% rated four cases, and 3.2% rated all five cases as severe sepsis/septic shock. Among respondents who felt strongly confident in their ability to use sepsis definitions (n = 45), agreement was no better (Fleiss' κ 0.28 for the five-category classification, and Fleiss' κ 0.21 for the dichotomized severe sepsis/septic shock classification). Cases were felt to be extremely or very realistic in 74% of responses; only 3% were deemed unrealistic.
Conclusions: Diagnosing sepsis is extremely subjective and variable. Objective criteria and standardized methodology are needed to enhance consistency and comparability in sepsis research, surveillance, benchmarking, and reporting.
Figures



Similar articles
-
Diagnostic accuracy of a screening electronic alert tool for severe sepsis and septic shock in the emergency department.BMC Med Inform Decis Mak. 2014 Dec 5;14:105. doi: 10.1186/s12911-014-0105-7. BMC Med Inform Decis Mak. 2014. PMID: 25476738 Free PMC article.
-
Severe sepsis in community-acquired pneumonia: when does it happen, and do systemic inflammatory response syndrome criteria help predict course?Chest. 2006 Apr;129(4):968-78. doi: 10.1378/chest.129.4.968. Chest. 2006. PMID: 16608946
-
Potential Impact of the 2016 Consensus Definitions of Sepsis and Septic Shock on Future Sepsis Research.Ann Emerg Med. 2017 Oct;70(4):553-561.e1. doi: 10.1016/j.annemergmed.2017.04.007. Ann Emerg Med. 2017. PMID: 28601273
-
Critical care in the emergency department: severe sepsis and septic shock.Emerg Med J. 2006 Sep;23(9):713-7. doi: 10.1136/emj.2005.029934. Emerg Med J. 2006. PMID: 16921089 Free PMC article. Review.
-
Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine.Chest. 1992 Jun;101(6):1644-55. doi: 10.1378/chest.101.6.1644. Chest. 1992. PMID: 1303622 Review.
Cited by
-
Evaluation of a Multivalent Transcriptomic Metric for Diagnosing Surgical Sepsis and Estimating Mortality Among Critically Ill Patients.JAMA Netw Open. 2022 Jul 1;5(7):e2221520. doi: 10.1001/jamanetworkopen.2022.21520. JAMA Netw Open. 2022. PMID: 35819783 Free PMC article.
-
Sepsis in the Operating Room: A Simulation Case for Perioperative Providers.MedEdPORTAL. 2017 Mar 30;13:10563. doi: 10.15766/mep_2374-8265.10563. MedEdPORTAL. 2017. PMID: 30800765 Free PMC article.
-
Physicians' decision about long-term thromboprophylaxis in cancer outpatients: CAT AXIS, a case vignette study on clinical practice in France.Support Care Cancer. 2018 Jun;26(6):2049-2056. doi: 10.1007/s00520-017-4034-8. Epub 2018 Jan 20. Support Care Cancer. 2018. PMID: 29353416
-
Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021.Intensive Care Med. 2021 Nov;47(11):1181-1247. doi: 10.1007/s00134-021-06506-y. Epub 2021 Oct 2. Intensive Care Med. 2021. PMID: 34599691 Free PMC article. No abstract available.
-
What is new and different in the 2021 Surviving Sepsis Campaign guidelines.Med Klin Intensivmed Notfmed. 2023 Dec;118(Suppl 2):75-79. doi: 10.1007/s00063-023-01028-5. Epub 2023 Jun 7. Med Klin Intensivmed Notfmed. 2023. PMID: 37286842 Free PMC article. Review.
References
-
- Minino AM, Murphy SL. Death in the United States, 2010. NCHS Data Brief. 2012;(99). - PubMed
-
- Torio CM, Andrews RM. National inpatient hospital costs: the most expensive conditions by payer, 2011. HCUP Statistical Brief #160. Rockville, MD: Agency for Healthcare Research and Quality; August 2013. http://www.hcup-us.ahrq.gov/reports/statbriefs/sb160.pdf. Accessed 23 March 2016.
-
- Centers for Medicare & Medicaid Services, Department of Health and Human Services. Medicare program; hospital inpatient prospective payment systems for acute care hospitals and the long-term care hospital prospective payment system and fiscal year 2015 rates; quality reporting requirements for specific providers; reasonable compensation equivalents for physician services in excluded hospitals and certain teaching hospitals; provider administrative appeals and judicial review; enforcement provisions for organ transplant centers; and electronic health record (EHR) incentive program. Final rule. Fed Regist. 2014;79(163):49853–50536. - PubMed
-
- National Quality Forum. Severe sepsis and septic shock: management bundle (composite measure). http://www.qualityforum.org/Qps/QpsTool.aspx. Accessed 23 March 2016.
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Other Literature Sources
Medical