Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Dec 24;13(3):E15.
doi: 10.5210/ojphi.v13i3.10818. eCollection 2021.

Evaluating multi-purpose syndromic surveillance systems - a complex problem

Roger Morbey  1 , Gillian Smith  1 , Isabel Oliver  2 , Obaghe Edeghere  3 , Iain Lake  4 , Richard Pebody  5 , Dan Todkill  3 , Noel McCarthy  6 , Alex J Elliot  1
Affiliations

Evaluating multi-purpose syndromic surveillance systems - a complex problem

Roger Morbey et al. Online J Public Health Inform. .

Abstract

Surveillance systems need to be evaluated to understand what the system can or cannot detect. The measures commonly used to quantify detection capabilities are sensitivity, positive predictive value and timeliness. However, the practical application of these measures to multi-purpose syndromic surveillance services is complex. Specifically, it is very difficult to link definitive lists of what the service is intended to detect and what was detected. First, we discuss issues arising from a multi-purpose system, which is designed to detect a wide range of health threats, and where individual indicators, e.g. 'fever', are also multi-purpose. Secondly, we discuss different methods of defining what can be detected, including historical events and simulations. Finally, we consider the additional complexity of evaluating a service which incorporates human decision-making alongside an automated detection algorithm. Understanding the complexities involved in evaluating multi-purpose systems helps design appropriate methods to describe their detection capabilities.

Keywords: Epidemiology; Public Health; outbreaks,; surveillance.

PubMed Disclaimer

Conflict of interest statement

Competing Interests No Competing Interests.

Figures

Figure 1
Figure 1
Results matrix for evaluating the sensitivity and specificity of a single laboratory test
Figure 2
Figure 2
Results matrix for evaluating a multi-purpose syndromic surveillance service.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Triple S. 2011. Project. Assessment of syndromic surveillance in Europe. Lancet. 378, 1833-34. 10.1016/S0140-6736(11)60834-9 - DOI - PubMed
    1. Sosin DM. 2003. Draft framework for evaluating syndromic surveillance systems. J Urban Health. 80, i8-13. - PMC - PubMed
    1. Jefferson H, Dupuy B, Chaudet H, Texier G, Green A, et al. 2008. Evaluation of a syndromic surveillance for the early detection of outbreaks among military personnel in a tropical country. J Public Health (Oxf). 30, 375-83. 10.1093/pubmed/fdn026 - DOI - PubMed
    1. Yih WK, Deshpande S, Fuller C, Heisey-Grove D, Hsu J, et al. 2010. Evaluating real-time syndromic surveillance signals from ambulatory care data in four states. Public Health Rep. 125, 111-20. 10.1177/003335491012500115 - DOI - PMC - PubMed
    1. Buckeridge DL, Burkom H, Campbell M, Hogan WR, Moore AW. 2005. Algorithms for rapid outbreak detection: a research synthesis. J Biomed Inform. 38, 99-113. 10.1016/j.jbi.2004.11.007 - DOI - PubMed

LinkOut - more resources