Comparing diagnostic performance and the utility of clinical vignette-based assessment under testing conditions designed to encourage either automatic or analytic thought

Jonathan S. Ilgen, Judith L. Bowen, Lucas A. McIntyre, Kenny V. Banh, David Barnes, Wendy C. Coates, Jeffrey Druck, Megan L. Fix, Diane Rimple, Lalena M. Yarris, Kevin W. Eva

Research output: Contribution to journalArticle

29 Scopus citations

Abstract

PURPOSE: Although decades of research have yielded considerable insight into physicians' clinical reasoning processes, assessing these processes remains challenging; thus, the authors sought to compare diagnostic performance and the utility of clinical vignette-based assessment under testing conditions designed to encourage either automatic or analytic thought. METHOD: This 2011-2012 multicenter randomized study of 393 clinicians (medical students, postgraduate trainees, and faculty) measured diagnostic accuracy on clinical vignettes under two conditions: one encouraged participants to give their first impression (FI), and the other led participants through a directed search (DS) for the correct diagnosis. The authors compared accuracy, feasibility, reliability, and relation to United States Medical Licensing Exam (USMLE) scores under each condition. RESULTS: A 2 (instructional condition) × 2 (vignette complexity) × 3 (experience level) analysis of variance revealed no difference in accuracy as a function of instructional condition (F[1,379] = 2.44, P = .12), but demonstrated the expected main effects of vignette complexity (F[1,379] = 965.2, P < .001) and experience (F[2,379] = 39.6, P < .001). Pearson correlations revealed greater associations between assessment scores and USMLE performance in the FI condition than in the DS condition (P < .001). Spearman-Brown calculations consistently indicated that alpha ≥ 0.75 could be achieved more efficiently under the FI condition relative to the DS condition. CONCLUSIONS: Instructions to trust one's first impres-sions result in similar performance when compared with instructions to consider clinical information in a systematic fashion, but have greater utility when used for the purposes of assessment.

Original languageEnglish (US)
Pages (from-to)1545-1551
Number of pages7
JournalAcademic Medicine
Volume88
Issue number10
DOIs
StatePublished - Jan 1 2013

ASJC Scopus subject areas

  • Education

Fingerprint Dive into the research topics of 'Comparing diagnostic performance and the utility of clinical vignette-based assessment under testing conditions designed to encourage either automatic or analytic thought'. Together they form a unique fingerprint.

  • Cite this

    Ilgen, J. S., Bowen, J. L., McIntyre, L. A., Banh, K. V., Barnes, D., Coates, W. C., Druck, J., Fix, M. L., Rimple, D., Yarris, L. M., & Eva, K. W. (2013). Comparing diagnostic performance and the utility of clinical vignette-based assessment under testing conditions designed to encourage either automatic or analytic thought. Academic Medicine, 88(10), 1545-1551. https://doi.org/10.1097/ACM.0b013e3182a31c1e