Positive predictive value of the AHRQ patient safety indicator "postoperative sepsis": Implications for practice and policy

Marisa Cevasco, Ann M. Borzecki, Qi Chen, Patricia A. Zrelak, Marlena Shin, Patrick S Romano, Kamal M F Itani, Amy K. Rosen

Research output: Contribution to journalArticle

25 Citations (Scopus)

Abstract

Background: Patient Safety Indicator (PSI) 13, or "Postoperative Sepsis," of the Agency for Healthcare Quality and Research (AHRQ), was recently adopted as part of a composite measure of patient safety by the Centers for Medicare and Medicaid Services (CMS). We sought to examine its positive predictive value (PPV) by determining how well it identifies true cases of postoperative sepsis. Study Design: Two retrospective cross-sectional studies of hospitalization records that met PSI 13 criteria were conducted, one within the Veterans Administration (VA) Hospitals from fiscal years (FY) 2003 to 2007, and one within community hospitals between October 1, 2005 and March 31, 2007. Trained abstractors reviewed medical records from each database using standardized abstraction instruments. We determined the PPV of the indicator and performed descriptive analyses of cases. Results: Of 112 cases flagged and reviewed within the VA system, 59 were true events of postoperative sepsis, yielding a PPV of 53% (95% CI 42% to 64%). Within the community hospital sector, of 164 flagged and reviewed cases, 67 were true cases of postoperative sepsis, yielding a PPV of 41% (95% CI 28% to 54%). False positives were due to infections that were present on admission, urgent or emergent cases, no clinical diagnosis of sepsis, or other coding limitations such as nonspecific shock in postoperative patients. Conclusions: PSI 13 has relatively poor predictive ability to identify true cases of postoperative sepsis in both the VA and nonfederal sectors. The lack of information on diagnosis timing, confusion about the definition of elective admission, and coding limitations were the major reasons for false positives. As it currently stands, the use of PSI 13 as a stand-alone measure for hospital reporting appears premature.

Original languageEnglish (US)
Pages (from-to)954-961
Number of pages8
JournalJournal of the American College of Surgeons
Volume212
Issue number6
DOIs
StatePublished - Jun 2011

Fingerprint

Health Services Research
Patient Safety
Sepsis
United States Department of Veterans Affairs
Community Hospital
Veterans Hospitals
Centers for Medicare and Medicaid Services (U.S.)
Medical Records
Shock
Hospitalization
Cross-Sectional Studies
Databases
Infection

ASJC Scopus subject areas

  • Surgery

Cite this

Positive predictive value of the AHRQ patient safety indicator "postoperative sepsis" : Implications for practice and policy. / Cevasco, Marisa; Borzecki, Ann M.; Chen, Qi; Zrelak, Patricia A.; Shin, Marlena; Romano, Patrick S; Itani, Kamal M F; Rosen, Amy K.

In: Journal of the American College of Surgeons, Vol. 212, No. 6, 06.2011, p. 954-961.

Research output: Contribution to journalArticle

Cevasco, Marisa ; Borzecki, Ann M. ; Chen, Qi ; Zrelak, Patricia A. ; Shin, Marlena ; Romano, Patrick S ; Itani, Kamal M F ; Rosen, Amy K. / Positive predictive value of the AHRQ patient safety indicator "postoperative sepsis" : Implications for practice and policy. In: Journal of the American College of Surgeons. 2011 ; Vol. 212, No. 6. pp. 954-961.
@article{35493401d7794865ae35eca10d1806f5,
title = "Positive predictive value of the AHRQ patient safety indicator {"}postoperative sepsis{"}: Implications for practice and policy",
abstract = "Background: Patient Safety Indicator (PSI) 13, or {"}Postoperative Sepsis,{"} of the Agency for Healthcare Quality and Research (AHRQ), was recently adopted as part of a composite measure of patient safety by the Centers for Medicare and Medicaid Services (CMS). We sought to examine its positive predictive value (PPV) by determining how well it identifies true cases of postoperative sepsis. Study Design: Two retrospective cross-sectional studies of hospitalization records that met PSI 13 criteria were conducted, one within the Veterans Administration (VA) Hospitals from fiscal years (FY) 2003 to 2007, and one within community hospitals between October 1, 2005 and March 31, 2007. Trained abstractors reviewed medical records from each database using standardized abstraction instruments. We determined the PPV of the indicator and performed descriptive analyses of cases. Results: Of 112 cases flagged and reviewed within the VA system, 59 were true events of postoperative sepsis, yielding a PPV of 53{\%} (95{\%} CI 42{\%} to 64{\%}). Within the community hospital sector, of 164 flagged and reviewed cases, 67 were true cases of postoperative sepsis, yielding a PPV of 41{\%} (95{\%} CI 28{\%} to 54{\%}). False positives were due to infections that were present on admission, urgent or emergent cases, no clinical diagnosis of sepsis, or other coding limitations such as nonspecific shock in postoperative patients. Conclusions: PSI 13 has relatively poor predictive ability to identify true cases of postoperative sepsis in both the VA and nonfederal sectors. The lack of information on diagnosis timing, confusion about the definition of elective admission, and coding limitations were the major reasons for false positives. As it currently stands, the use of PSI 13 as a stand-alone measure for hospital reporting appears premature.",
author = "Marisa Cevasco and Borzecki, {Ann M.} and Qi Chen and Zrelak, {Patricia A.} and Marlena Shin and Romano, {Patrick S} and Itani, {Kamal M F} and Rosen, {Amy K.}",
year = "2011",
month = "6",
doi = "10.1016/j.jamcollsurg.2010.11.013",
language = "English (US)",
volume = "212",
pages = "954--961",
journal = "Journal of the American College of Surgeons",
issn = "1072-7515",
publisher = "Elsevier Inc.",
number = "6",

}

TY - JOUR

T1 - Positive predictive value of the AHRQ patient safety indicator "postoperative sepsis"

T2 - Implications for practice and policy

AU - Cevasco, Marisa

AU - Borzecki, Ann M.

AU - Chen, Qi

AU - Zrelak, Patricia A.

AU - Shin, Marlena

AU - Romano, Patrick S

AU - Itani, Kamal M F

AU - Rosen, Amy K.

PY - 2011/6

Y1 - 2011/6

N2 - Background: Patient Safety Indicator (PSI) 13, or "Postoperative Sepsis," of the Agency for Healthcare Quality and Research (AHRQ), was recently adopted as part of a composite measure of patient safety by the Centers for Medicare and Medicaid Services (CMS). We sought to examine its positive predictive value (PPV) by determining how well it identifies true cases of postoperative sepsis. Study Design: Two retrospective cross-sectional studies of hospitalization records that met PSI 13 criteria were conducted, one within the Veterans Administration (VA) Hospitals from fiscal years (FY) 2003 to 2007, and one within community hospitals between October 1, 2005 and March 31, 2007. Trained abstractors reviewed medical records from each database using standardized abstraction instruments. We determined the PPV of the indicator and performed descriptive analyses of cases. Results: Of 112 cases flagged and reviewed within the VA system, 59 were true events of postoperative sepsis, yielding a PPV of 53% (95% CI 42% to 64%). Within the community hospital sector, of 164 flagged and reviewed cases, 67 were true cases of postoperative sepsis, yielding a PPV of 41% (95% CI 28% to 54%). False positives were due to infections that were present on admission, urgent or emergent cases, no clinical diagnosis of sepsis, or other coding limitations such as nonspecific shock in postoperative patients. Conclusions: PSI 13 has relatively poor predictive ability to identify true cases of postoperative sepsis in both the VA and nonfederal sectors. The lack of information on diagnosis timing, confusion about the definition of elective admission, and coding limitations were the major reasons for false positives. As it currently stands, the use of PSI 13 as a stand-alone measure for hospital reporting appears premature.

AB - Background: Patient Safety Indicator (PSI) 13, or "Postoperative Sepsis," of the Agency for Healthcare Quality and Research (AHRQ), was recently adopted as part of a composite measure of patient safety by the Centers for Medicare and Medicaid Services (CMS). We sought to examine its positive predictive value (PPV) by determining how well it identifies true cases of postoperative sepsis. Study Design: Two retrospective cross-sectional studies of hospitalization records that met PSI 13 criteria were conducted, one within the Veterans Administration (VA) Hospitals from fiscal years (FY) 2003 to 2007, and one within community hospitals between October 1, 2005 and March 31, 2007. Trained abstractors reviewed medical records from each database using standardized abstraction instruments. We determined the PPV of the indicator and performed descriptive analyses of cases. Results: Of 112 cases flagged and reviewed within the VA system, 59 were true events of postoperative sepsis, yielding a PPV of 53% (95% CI 42% to 64%). Within the community hospital sector, of 164 flagged and reviewed cases, 67 were true cases of postoperative sepsis, yielding a PPV of 41% (95% CI 28% to 54%). False positives were due to infections that were present on admission, urgent or emergent cases, no clinical diagnosis of sepsis, or other coding limitations such as nonspecific shock in postoperative patients. Conclusions: PSI 13 has relatively poor predictive ability to identify true cases of postoperative sepsis in both the VA and nonfederal sectors. The lack of information on diagnosis timing, confusion about the definition of elective admission, and coding limitations were the major reasons for false positives. As it currently stands, the use of PSI 13 as a stand-alone measure for hospital reporting appears premature.

UR - http://www.scopus.com/inward/record.url?scp=79957581677&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79957581677&partnerID=8YFLogxK

U2 - 10.1016/j.jamcollsurg.2010.11.013

DO - 10.1016/j.jamcollsurg.2010.11.013

M3 - Article

C2 - 21474345

AN - SCOPUS:79957581677

VL - 212

SP - 954

EP - 961

JO - Journal of the American College of Surgeons

JF - Journal of the American College of Surgeons

SN - 1072-7515

IS - 6

ER -