Structured Clinical Decision Aids Are Seldom Compared With Subjective Physician Judgment, and Are Seldom Superior

David L. Schriger, Joshua W Elder, Richelle J. Cooper

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

Study objective We determine how often studies that evaluate the performance of an aid for decisionmaking, be it a simple laboratory or imaging test or a complex multielement decision instrument, compare the aid's performance to independent, unaided physician judgment. Methods This was a cross-sectional survey of all Original Research and Brief Research Report articles in Annals of Emergency Medicine from 1998 to 2015. We included all articles that evaluated the performance of an aid for decisionmaking in assisting a physician with a decision about testing, treatment, diagnosis, or disposition. Two authors independently characterized the intent and purpose of each aid for decisionmaking, determined whether each study had a comparison to unaided physician judgment within the article or in a separate article, and recorded the result of that comparison. Results One hundred seventy-one (8.3%) of 2,060 research articles studied the performance characteristics of an aid for decisionmaking, 48 of which were formal clinical decision instruments. Forty of the 171 studies retrospectively analyzed existing databases and therefore could not assess physician judgment. Investigators compared the aid for decisionmaking to physician judgment in 11% (15/131) of the prospective studies, including 15% (6/41) of studies that evaluated a formal clinical decision instrument. For 9 articles that had no comparison to physician judgment, we found 6 unique external publications that compared that aid to physician clinical judgment. The decision aid was superior to clinical judgment in 2 of the 21 studies that contained a comparison. Conclusion Physician judgment is infrequently assessed when the performance of an aid for decisionmaking is evaluated, and, when reported, the decision aid seldom outperformed physician judgment.

Original languageEnglish (US)
Pages (from-to)338-344.e3
JournalAnnals of Emergency Medicine
Volume70
Issue number3
DOIs
StatePublished - Sep 1 2017
Externally publishedYes

Fingerprint

Decision Support Techniques
Physicians
Emergency Medicine
Research
Publications
Cross-Sectional Studies
Research Personnel
Databases
Prospective Studies

ASJC Scopus subject areas

  • Emergency Medicine

Cite this

Structured Clinical Decision Aids Are Seldom Compared With Subjective Physician Judgment, and Are Seldom Superior. / Schriger, David L.; Elder, Joshua W; Cooper, Richelle J.

In: Annals of Emergency Medicine, Vol. 70, No. 3, 01.09.2017, p. 338-344.e3.

Research output: Contribution to journalArticle

@article{c6e458c9bab444a3a59bfeafe3e2b3a2,
title = "Structured Clinical Decision Aids Are Seldom Compared With Subjective Physician Judgment, and Are Seldom Superior",
abstract = "Study objective We determine how often studies that evaluate the performance of an aid for decisionmaking, be it a simple laboratory or imaging test or a complex multielement decision instrument, compare the aid's performance to independent, unaided physician judgment. Methods This was a cross-sectional survey of all Original Research and Brief Research Report articles in Annals of Emergency Medicine from 1998 to 2015. We included all articles that evaluated the performance of an aid for decisionmaking in assisting a physician with a decision about testing, treatment, diagnosis, or disposition. Two authors independently characterized the intent and purpose of each aid for decisionmaking, determined whether each study had a comparison to unaided physician judgment within the article or in a separate article, and recorded the result of that comparison. Results One hundred seventy-one (8.3{\%}) of 2,060 research articles studied the performance characteristics of an aid for decisionmaking, 48 of which were formal clinical decision instruments. Forty of the 171 studies retrospectively analyzed existing databases and therefore could not assess physician judgment. Investigators compared the aid for decisionmaking to physician judgment in 11{\%} (15/131) of the prospective studies, including 15{\%} (6/41) of studies that evaluated a formal clinical decision instrument. For 9 articles that had no comparison to physician judgment, we found 6 unique external publications that compared that aid to physician clinical judgment. The decision aid was superior to clinical judgment in 2 of the 21 studies that contained a comparison. Conclusion Physician judgment is infrequently assessed when the performance of an aid for decisionmaking is evaluated, and, when reported, the decision aid seldom outperformed physician judgment.",
author = "Schriger, {David L.} and Elder, {Joshua W} and Cooper, {Richelle J.}",
year = "2017",
month = "9",
day = "1",
doi = "10.1016/j.annemergmed.2016.12.004",
language = "English (US)",
volume = "70",
pages = "338--344.e3",
journal = "Annals of Emergency Medicine",
issn = "0196-0644",
publisher = "Mosby Inc.",
number = "3",

}

TY - JOUR

T1 - Structured Clinical Decision Aids Are Seldom Compared With Subjective Physician Judgment, and Are Seldom Superior

AU - Schriger, David L.

AU - Elder, Joshua W

AU - Cooper, Richelle J.

PY - 2017/9/1

Y1 - 2017/9/1

N2 - Study objective We determine how often studies that evaluate the performance of an aid for decisionmaking, be it a simple laboratory or imaging test or a complex multielement decision instrument, compare the aid's performance to independent, unaided physician judgment. Methods This was a cross-sectional survey of all Original Research and Brief Research Report articles in Annals of Emergency Medicine from 1998 to 2015. We included all articles that evaluated the performance of an aid for decisionmaking in assisting a physician with a decision about testing, treatment, diagnosis, or disposition. Two authors independently characterized the intent and purpose of each aid for decisionmaking, determined whether each study had a comparison to unaided physician judgment within the article or in a separate article, and recorded the result of that comparison. Results One hundred seventy-one (8.3%) of 2,060 research articles studied the performance characteristics of an aid for decisionmaking, 48 of which were formal clinical decision instruments. Forty of the 171 studies retrospectively analyzed existing databases and therefore could not assess physician judgment. Investigators compared the aid for decisionmaking to physician judgment in 11% (15/131) of the prospective studies, including 15% (6/41) of studies that evaluated a formal clinical decision instrument. For 9 articles that had no comparison to physician judgment, we found 6 unique external publications that compared that aid to physician clinical judgment. The decision aid was superior to clinical judgment in 2 of the 21 studies that contained a comparison. Conclusion Physician judgment is infrequently assessed when the performance of an aid for decisionmaking is evaluated, and, when reported, the decision aid seldom outperformed physician judgment.

AB - Study objective We determine how often studies that evaluate the performance of an aid for decisionmaking, be it a simple laboratory or imaging test or a complex multielement decision instrument, compare the aid's performance to independent, unaided physician judgment. Methods This was a cross-sectional survey of all Original Research and Brief Research Report articles in Annals of Emergency Medicine from 1998 to 2015. We included all articles that evaluated the performance of an aid for decisionmaking in assisting a physician with a decision about testing, treatment, diagnosis, or disposition. Two authors independently characterized the intent and purpose of each aid for decisionmaking, determined whether each study had a comparison to unaided physician judgment within the article or in a separate article, and recorded the result of that comparison. Results One hundred seventy-one (8.3%) of 2,060 research articles studied the performance characteristics of an aid for decisionmaking, 48 of which were formal clinical decision instruments. Forty of the 171 studies retrospectively analyzed existing databases and therefore could not assess physician judgment. Investigators compared the aid for decisionmaking to physician judgment in 11% (15/131) of the prospective studies, including 15% (6/41) of studies that evaluated a formal clinical decision instrument. For 9 articles that had no comparison to physician judgment, we found 6 unique external publications that compared that aid to physician clinical judgment. The decision aid was superior to clinical judgment in 2 of the 21 studies that contained a comparison. Conclusion Physician judgment is infrequently assessed when the performance of an aid for decisionmaking is evaluated, and, when reported, the decision aid seldom outperformed physician judgment.

UR - http://www.scopus.com/inward/record.url?scp=85013420468&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85013420468&partnerID=8YFLogxK

U2 - 10.1016/j.annemergmed.2016.12.004

DO - 10.1016/j.annemergmed.2016.12.004

M3 - Article

C2 - 28238497

AN - SCOPUS:85013420468

VL - 70

SP - 338-344.e3

JO - Annals of Emergency Medicine

JF - Annals of Emergency Medicine

SN - 0196-0644

IS - 3

ER -