TY - JOUR
T1 - The accuracy of an electronic Pulmonary Embolism Severity Index auto-populated from the electronic health record
T2 - Setting the stage for computerized clinical decision support
AU - Kaisers Permanente CREST Network
AU - Vinson, David R.
AU - Morley, J. E.
AU - Huang, J.
AU - Liu, V.
AU - Anderson, M. L.
AU - Drenten, C. E.
AU - Radecki, R. P.
AU - Nishijima, Daniel
AU - Reed, M. E.
PY - 2015
Y1 - 2015
N2 - Background: The Pulmonary Embolism (PE) Severity Index identifies emergency department (ED) patients with acute PE that can be safely managed without hospitalization. However, the Index comprises 11 weighted variables, complexity that can impede its integration into contextual workflow. Objective: We designed a computerized version of the PE Severity Index (e-Index) to automatically extract the required variables from discrete fields in the electronic health record (EHR). We tested the e-Index on the study population to determine its accuracy compared with a gold standard generated by physician abstraction of the EHR on manual chart review. Methods: This retrospective cohort study included adults with objectively-confirmed acute PE in four community EDs from 2010–2012. Outcomes included performance characteristics of the e-Index for individual values, the number of cases requiring physician editing, and the accuracy of the e-Index risk category (low vs. higher). Results: For the 593 eligible patients, there were 6,523 values automatically extracted. Fifty one of these needed physician editing, yielding an accuracy at the value-level of 99.2% (95% confidence interval [CI], 99.0%-99.4%). Sensitivity was 96.9% (95% CI, 96.0%-97.9%) and specificity was 99.8% (95% CI, 99.7%-99.9%). The 51 corrected values were distributed among 47 cases: 43 cases required the correction of one variable and four cases required the correction of two. At the riskcategory level, the e-Index had an accuracy of 96.8% (95% CI, 95.0%-98.0%), under-classifying 16 higher-risk cases (2.7%) and over-classifying 3 low-risk cases (0.5%). Conclusion: Our automated extraction of variables from the EHR for the e-Index demonstrates substantial accuracy, requiring a minimum of physician editing. This should increase user acceptability and implementation success of a computerized clinical decision support system built around the e-Index, and may serve as a model to automate other complex risk stratification instruments.
AB - Background: The Pulmonary Embolism (PE) Severity Index identifies emergency department (ED) patients with acute PE that can be safely managed without hospitalization. However, the Index comprises 11 weighted variables, complexity that can impede its integration into contextual workflow. Objective: We designed a computerized version of the PE Severity Index (e-Index) to automatically extract the required variables from discrete fields in the electronic health record (EHR). We tested the e-Index on the study population to determine its accuracy compared with a gold standard generated by physician abstraction of the EHR on manual chart review. Methods: This retrospective cohort study included adults with objectively-confirmed acute PE in four community EDs from 2010–2012. Outcomes included performance characteristics of the e-Index for individual values, the number of cases requiring physician editing, and the accuracy of the e-Index risk category (low vs. higher). Results: For the 593 eligible patients, there were 6,523 values automatically extracted. Fifty one of these needed physician editing, yielding an accuracy at the value-level of 99.2% (95% confidence interval [CI], 99.0%-99.4%). Sensitivity was 96.9% (95% CI, 96.0%-97.9%) and specificity was 99.8% (95% CI, 99.7%-99.9%). The 51 corrected values were distributed among 47 cases: 43 cases required the correction of one variable and four cases required the correction of two. At the riskcategory level, the e-Index had an accuracy of 96.8% (95% CI, 95.0%-98.0%), under-classifying 16 higher-risk cases (2.7%) and over-classifying 3 low-risk cases (0.5%). Conclusion: Our automated extraction of variables from the EHR for the e-Index demonstrates substantial accuracy, requiring a minimum of physician editing. This should increase user acceptability and implementation success of a computerized clinical decision support system built around the e-Index, and may serve as a model to automate other complex risk stratification instruments.
KW - Clinical decision support systems
KW - Data completeness
KW - Electronic health record
KW - Emergency medicine
KW - Pulmonary embolism
KW - Risk assessment
UR - http://www.scopus.com/inward/record.url?scp=84930688477&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84930688477&partnerID=8YFLogxK
U2 - 10.4338/ACI-2014-12-RA-0116
DO - 10.4338/ACI-2014-12-RA-0116
M3 - Article
C2 - 26171078
AN - SCOPUS:84930688477
VL - 6
SP - 318
EP - 333
JO - Applied Clinical Informatics
JF - Applied Clinical Informatics
SN - 1869-0327
IS - 2
ER -