Abstract
In this paper, we identify 8 methods used to measure errors and adverse events in health care and discuss their strengths and weaknesses. We focus on the reliability and validity of each, as well as the ability to detect latent errors (or system errors) versus active errors and adverse events. We propose a general framework to help health care providers, researchers, and administrators choose the most appropriate methods to meet their patient safety measurement goals.
Similar content being viewed by others
References
Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human. Washington DC: National Academy Press, 1999.
Expert Group on Learning from Adverse Events in the NHS. An Organisation With a Memory. London: Stationery Office; 2000.
Bates DW, Gawande AA. Error in medicine: what have we learned. Ann Intern Med. 2000;132:763–7.
Weingart SN, Wilson RM, Gibberd RW, Harrison B. Epidemiology of medical error. BMJ. 2000;320:774–7.
Brennan TA. The Institute of Medicine report on medical errors—could it do harm? N Engl J Med. 2000;342:1123–5.
McDonald CJ, Weiner M, Hui SL. Deaths due to medical errors are exaggerated in Institute of Medicine report. JAMA. 2000;284:93–5.
Sox HC, Woloshin S. How many deaths are due to medical error? Getting the number right. Eff Clin Pract. 2000;3:277–83.
Hofer TP, Kerr EA. What is an error? Eff Clin Pract. 2000;3:261–9.
Wears RL, Janiak B, Moorehead JC, et al. Human error in medicine: promise and pitfalls, part 2. Ann Emerg Med. 2000;36:142–4.
Leape LL, Berwick DM, Bates DW. What practices will most improve patient safety? Evidence-based medicine meets patient safety. JAMA. 2002;288:501–7.
Shojania KG, Duncan BW, McDonald KM, Wachter RM. Safe but sound. Patient safety meets evidence-based medicine. JAMA. 2002;288:508–13.
Reason J. Human Error. Cambridge: Cambridge University Press; 1990.
Hulley SB, Martin JN, Cummings SR. Planning the measurements: precision and accuracy. In: Designing Clinical Research: An Epidemiologic Approach. Hulley SB, Cummings SR, eds. Philadelphia: Lippincott; 2001:37.
Leape LL, Bates DW, Cullen DJ. Systems analysis of adverse drug events. ADE prevention study. JAMA. 1995;274:35–43.
Perrow C. Normal Accidents: Living With High Risk Technologies. New York: Basic Books; 1984.
Gordon L. Gordon’s Guide to the Surgical Morbidity and Mortality Conference. Philadelphia: Hanley and Belfus; 1994.
Accreditation Council for Graduate Medical Education. Essentials and Information Items. Graduate Medical Education Directory. Chicago Ill: American Medical Association; 1995.
Harbison S, Regehr G. Faculty and resident opinions regarding the role of Morbidity and Mortality Conference. Am J Surg 1999;177:136–9.
Goldman L, Sayson R, Robbins S, Cohn LH, Bettmann M, Weisberg M. The value of the autopsy in three medical eras. N Engl J Med. 1983;308:1000–5.
Cameron HM, McGoogan E. A prospective study of 1152 hospital autopsies. I: inaccuracies in death certification. J Pathol. 1981;133:273–83.
Anderson RE, Hill RB, Key CR. The sensitivity and specificity of clinical diagnostics during five decades: toward an understanding of necessary fallibility. JAMA. 1989;261:1610–17.
Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology. Boston: Little Brown, and Company; 1991.
Fischoff B. Hindsight does not equal foresight: the effect of outcome knowledge on judgment under uncertainty. J Exp Psychol Hum Percept Perform. 1975;1:288–99.
Caplan RA, Posner KL, Cheney FW. Effect of outcome on physician judgments of appropriateness of care. JAMA. 1991;265:1957–60.
Cheney FW, Posner K, Caplan RA, Ward RJ. Standard of care and anesthesia liability. JAMA. 1989;261:1599–1603.
Rolph JE, Kravitz RL, McGuigan K. Malpractice claims data as a quality improvement tool. II. Is targeting effective? JAMA. 1991;266:2093–7.
Weiler PC. Medical Malpractice on Trial. Cambridge, Mass: Harvard University Press; 1991.
Eichhorn JH, Cooper JB, Cullen DJ, Maier WR, Philip JH, Seeman RG. Standards for patient monitoring during anesthesia at Harvard Medical School. JAMA. 1986;256:1017–20.
Barach P, Small SD. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ. 2000;320:759–63.
Sexton JB, Thomas EJ, Helmreich RL. Error, stress, and teamwork in medicine and aviation: cross-sectional surveys. BMJ. 2000;320:745–9.
Brennan TA, Lee TH, O’Neil AC, Petersen LA. Integrating providers into quality improvement: a pilot project at one hospital. Qual Manag Health Care. 1992;1:29–35.
Edmonson AC. Learning from mistakes is easier said than done: group and organizational influences on the detection and correction of human error. J Appl Behav Sci. 1996;32:5–28.
O’Neil AC, Petersen LA, Cook EF, Bates DW, Lee TH, Brennan TA. A comparison of physicians self-reporting to medical record review to identify medical adverse events. Ann Intern Med. 1993;119:370–6.
Petersen LA, Brennan TA, O’Neil AC, Cook EF, Lee TH. Does house staff discontinuity of care increase the risk for preventable adverse events? Ann Intern Med. 1994;121:866–72.
Petersen LA, Orav EJ, Teich JM, O’Neil AC, Brennan TA. Using a computerized sign-out to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improv. 1998;24:77–87.
Iezzoni LI. Assessing quality using administrative data. Ann Intern Med. 1997;127:666–74.
Iezzoni LI. Identifying complications of care using administrative data. Med Care. 1994;32:700–15.
Iezzoni LI, Davis RB, Palmer RH, et al. Does the Complications Screening Program flag cases with process of care problems? Using explicit criteria to judge processes. Int J Qual Health Care. 1999;11:107–18.
Weingart SN, Iezzoni LI, Davis RB, et al. Use of administrative data to find substandard care: validation of the complications screening program. Med Care. 2000;38:796–806.
Bates DW, O’Neil AC, Petersen LA, Lee TH, Brennan TA. Evaluation of screening criteria for adverse events in medical patients. Med Care. 1995;33:452–62.
Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence care in hospitalized patients. N Engl J Med. 1991;324:370–6.
Ashton CM, Kuykendall DH, Johnson ML, Wray NP. An empirical assessment of the validity of explicit and implicit process of care criteria for quality assessment. Med Care. 1999;37:798–808.
Localio AR, Lawthers A, Brennan TA. Identifying adverse events caused by medical care: degree of physician agreement in retrospective chart review. Ann Intern Med. 1996;125:457–64.
Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med. 2000;108:642–9.
Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA. 1991;266:2847–51.
Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison to chart-review and stimulated voluntary report. J Am Med Inform Assoc. 1998;5:305–14.
Helmreich RL, Schaefer HG. Team performance in the operating room. In: Bogner MS, ed. Human Error in Medicine. Hillsdale NJ: Lawrence Erlbaum Associates; 1994:225–53.
Donchin Y, Gopher D, Olin M, et al. A look into the nature and causes of human errors in the intensive care unit. Crit Care Med. 1995;23:294–300.
Andrews LB, Stocking C, Krizek T, et al. An alternative strategy for studying adverse events in medical care. Lancet. 1997;349:309–13.
Barker KN. Data collection techniques: observation. Am J Hosp Pharm. 1980;37:1235–43.
Cook RI, Woods DD. Operating at the sharp end: the complexity of human error. In: Bogner MS, ed. Human Error in Medicine. Hillsdale NJ: Lawrence Erlbaum Associates; 1994.
Last JM. A Dictionary of Epidemiology. New York: Oxford University Press; 1995.
Mangano DT, Goldman L. Preoperative assessment of patients with known or suspected coronary disease. N Engl J Med. 1995;333:1750–6.
Lee TH, Marcantonio ER, Mangione CM, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100:1043–9.
Gaynes RP, Horan TC. Surveillance of nosocomial infections. In: Mayhall GC, ed, Hospital Epidemiology and Infection Control. Philidelphia: Lippincott Williams and Wilkins; 1999.
Author information
Authors and Affiliations
Corresponding author
Additional information
Dr. Thomas is a Robert Wood Johnson Foundation Generalist Physician Faculty Scholar. Dr. Petersen was an awardee in the Research Career Development Award Program of the VA HSR&D Service (grant RCD 95-306) and is a Robert Wood Johnson Foundation Generalist Physician Faculty Scholar.
The authors have no potential conflicts of interest. The authors’ funding agencies had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.
Rights and permissions
About this article
Cite this article
Thomas, E.J., Petersen, L.A. Measuring errors and adverse events in health care. J GEN INTERN MED 18, 61–67 (2003). https://doi.org/10.1046/j.1525-1497.2003.20147.x
Issue Date:
DOI: https://doi.org/10.1046/j.1525-1497.2003.20147.x