Detection of postoperative respiratory failure: How predictive is the agency for healthcare research and quality's patient safety indicator?

Garth H Utter, Joanne Cuny, Pradeep Sama, Michael R. Silver, Patricia A. Zrelak, Ruth Baron, Saskia E. Drsler, Patrick S Romano

Research output: Contribution to journalArticle

37 Citations (Scopus)

Abstract

BACKGROUND: Patient Safety Indicator (PSI) 11, or postoperative respiratory failure, was developed by the US Agency for Healthcare Research and Quality to detect incident cases of respiratory failure after elective operations through use of ICD-9-CM diagnosis and procedure codes. We sought to determine the positive predictive value (PPV) of this indicator. STUDY DESIGN: We conducted a retrospective cross-sectional study, sampling consecutive cases that met PSI 11 criteria from 18 geographically diverse academic medical centers on or before June 30, 2007. Trained abstractors from each center reviewed medical records using a standard instrument. We assessed the PPV of the indicator (with 95% CI adjusted for clustering within centers) and conducted descriptive analyses of the cases. RESULTS: Of 609 cases that met PSI 11 criteria, 551 (90.5%; 95% CI, 86.594.4%) satisfied the technical criteria of the indicator and 507 (83.2%; 95% CI, 77.289.3%) represented true cases of postoperative respiratory failure from a clinical standpoint. The most frequent reasons for being falsely positive were nonelective hospitalization, prolonged intubation for airway protection, and insufficient evidence to support a diagnosis of acute respiratory failure. Fifty percent of true-positive cases involved substantial baseline comorbidities, and 23% resulted in death. CONCLUSIONS: Although PSI 11 predicts true postoperative respiratory failure with relatively high frequency, the indicator does not limit detection to preventable cases. The PPV of PSI 11 might be increased by excluding cases with a principal diagnosis suggestive of a nonelective hospitalization and those with head or neck procedures. Removing the diagnosis code criterion from the indicator might also increase PPV, but would decrease the number of true positive cases detected by 20%.

Original languageEnglish (US)
JournalJournal of the American College of Surgeons
Volume211
Issue number3
DOIs
StatePublished - Sep 2010

Fingerprint

Health Services Research
Patient Safety
Respiratory Insufficiency
Hospitalization
United States Agency for Healthcare Research and Quality
International Classification of Diseases
Intubation
Medical Records
Cluster Analysis
Limit of Detection
Comorbidity
Neck
Cross-Sectional Studies
Head

Keywords

  • major diagnostic category
  • MDC
  • National Surgical Quality Improvement Program
  • NSQIP
  • patient safety indicator
  • positive predictive value
  • postoperative respiratory failure
  • PPV
  • PRF
  • PSI
  • UHC
  • University HealthSystem Consortium

ASJC Scopus subject areas

  • Surgery

Cite this

Detection of postoperative respiratory failure : How predictive is the agency for healthcare research and quality's patient safety indicator? / Utter, Garth H; Cuny, Joanne; Sama, Pradeep; Silver, Michael R.; Zrelak, Patricia A.; Baron, Ruth; Drsler, Saskia E.; Romano, Patrick S.

In: Journal of the American College of Surgeons, Vol. 211, No. 3, 09.2010.

Research output: Contribution to journalArticle

@article{ff4cb3b7f90a46318cc6db21d11673b4,
title = "Detection of postoperative respiratory failure: How predictive is the agency for healthcare research and quality's patient safety indicator?",
abstract = "BACKGROUND: Patient Safety Indicator (PSI) 11, or postoperative respiratory failure, was developed by the US Agency for Healthcare Research and Quality to detect incident cases of respiratory failure after elective operations through use of ICD-9-CM diagnosis and procedure codes. We sought to determine the positive predictive value (PPV) of this indicator. STUDY DESIGN: We conducted a retrospective cross-sectional study, sampling consecutive cases that met PSI 11 criteria from 18 geographically diverse academic medical centers on or before June 30, 2007. Trained abstractors from each center reviewed medical records using a standard instrument. We assessed the PPV of the indicator (with 95{\%} CI adjusted for clustering within centers) and conducted descriptive analyses of the cases. RESULTS: Of 609 cases that met PSI 11 criteria, 551 (90.5{\%}; 95{\%} CI, 86.594.4{\%}) satisfied the technical criteria of the indicator and 507 (83.2{\%}; 95{\%} CI, 77.289.3{\%}) represented true cases of postoperative respiratory failure from a clinical standpoint. The most frequent reasons for being falsely positive were nonelective hospitalization, prolonged intubation for airway protection, and insufficient evidence to support a diagnosis of acute respiratory failure. Fifty percent of true-positive cases involved substantial baseline comorbidities, and 23{\%} resulted in death. CONCLUSIONS: Although PSI 11 predicts true postoperative respiratory failure with relatively high frequency, the indicator does not limit detection to preventable cases. The PPV of PSI 11 might be increased by excluding cases with a principal diagnosis suggestive of a nonelective hospitalization and those with head or neck procedures. Removing the diagnosis code criterion from the indicator might also increase PPV, but would decrease the number of true positive cases detected by 20{\%}.",
keywords = "major diagnostic category, MDC, National Surgical Quality Improvement Program, NSQIP, patient safety indicator, positive predictive value, postoperative respiratory failure, PPV, PRF, PSI, UHC, University HealthSystem Consortium",
author = "Utter, {Garth H} and Joanne Cuny and Pradeep Sama and Silver, {Michael R.} and Zrelak, {Patricia A.} and Ruth Baron and Drsler, {Saskia E.} and Romano, {Patrick S}",
year = "2010",
month = "9",
doi = "10.1016/j.jamcollsurg.2010.04.022",
language = "English (US)",
volume = "211",
journal = "Journal of the American College of Surgeons",
issn = "1072-7515",
publisher = "Elsevier Inc.",
number = "3",

}

TY - JOUR

T1 - Detection of postoperative respiratory failure

T2 - How predictive is the agency for healthcare research and quality's patient safety indicator?

AU - Utter, Garth H

AU - Cuny, Joanne

AU - Sama, Pradeep

AU - Silver, Michael R.

AU - Zrelak, Patricia A.

AU - Baron, Ruth

AU - Drsler, Saskia E.

AU - Romano, Patrick S

PY - 2010/9

Y1 - 2010/9

N2 - BACKGROUND: Patient Safety Indicator (PSI) 11, or postoperative respiratory failure, was developed by the US Agency for Healthcare Research and Quality to detect incident cases of respiratory failure after elective operations through use of ICD-9-CM diagnosis and procedure codes. We sought to determine the positive predictive value (PPV) of this indicator. STUDY DESIGN: We conducted a retrospective cross-sectional study, sampling consecutive cases that met PSI 11 criteria from 18 geographically diverse academic medical centers on or before June 30, 2007. Trained abstractors from each center reviewed medical records using a standard instrument. We assessed the PPV of the indicator (with 95% CI adjusted for clustering within centers) and conducted descriptive analyses of the cases. RESULTS: Of 609 cases that met PSI 11 criteria, 551 (90.5%; 95% CI, 86.594.4%) satisfied the technical criteria of the indicator and 507 (83.2%; 95% CI, 77.289.3%) represented true cases of postoperative respiratory failure from a clinical standpoint. The most frequent reasons for being falsely positive were nonelective hospitalization, prolonged intubation for airway protection, and insufficient evidence to support a diagnosis of acute respiratory failure. Fifty percent of true-positive cases involved substantial baseline comorbidities, and 23% resulted in death. CONCLUSIONS: Although PSI 11 predicts true postoperative respiratory failure with relatively high frequency, the indicator does not limit detection to preventable cases. The PPV of PSI 11 might be increased by excluding cases with a principal diagnosis suggestive of a nonelective hospitalization and those with head or neck procedures. Removing the diagnosis code criterion from the indicator might also increase PPV, but would decrease the number of true positive cases detected by 20%.

AB - BACKGROUND: Patient Safety Indicator (PSI) 11, or postoperative respiratory failure, was developed by the US Agency for Healthcare Research and Quality to detect incident cases of respiratory failure after elective operations through use of ICD-9-CM diagnosis and procedure codes. We sought to determine the positive predictive value (PPV) of this indicator. STUDY DESIGN: We conducted a retrospective cross-sectional study, sampling consecutive cases that met PSI 11 criteria from 18 geographically diverse academic medical centers on or before June 30, 2007. Trained abstractors from each center reviewed medical records using a standard instrument. We assessed the PPV of the indicator (with 95% CI adjusted for clustering within centers) and conducted descriptive analyses of the cases. RESULTS: Of 609 cases that met PSI 11 criteria, 551 (90.5%; 95% CI, 86.594.4%) satisfied the technical criteria of the indicator and 507 (83.2%; 95% CI, 77.289.3%) represented true cases of postoperative respiratory failure from a clinical standpoint. The most frequent reasons for being falsely positive were nonelective hospitalization, prolonged intubation for airway protection, and insufficient evidence to support a diagnosis of acute respiratory failure. Fifty percent of true-positive cases involved substantial baseline comorbidities, and 23% resulted in death. CONCLUSIONS: Although PSI 11 predicts true postoperative respiratory failure with relatively high frequency, the indicator does not limit detection to preventable cases. The PPV of PSI 11 might be increased by excluding cases with a principal diagnosis suggestive of a nonelective hospitalization and those with head or neck procedures. Removing the diagnosis code criterion from the indicator might also increase PPV, but would decrease the number of true positive cases detected by 20%.

KW - major diagnostic category

KW - MDC

KW - National Surgical Quality Improvement Program

KW - NSQIP

KW - patient safety indicator

KW - positive predictive value

KW - postoperative respiratory failure

KW - PPV

KW - PRF

KW - PSI

KW - UHC

KW - University HealthSystem Consortium

UR - http://www.scopus.com/inward/record.url?scp=77956182451&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77956182451&partnerID=8YFLogxK

U2 - 10.1016/j.jamcollsurg.2010.04.022

DO - 10.1016/j.jamcollsurg.2010.04.022

M3 - Article

C2 - 20800191

AN - SCOPUS:77956182451

VL - 211

JO - Journal of the American College of Surgeons

JF - Journal of the American College of Surgeons

SN - 1072-7515

IS - 3

ER -