Critical thinking in graduate medical education: A role for concept mapping assessment?

Daniel C. West, J. Richard Pomeroy, Jeanny K Park, Elise A. Gerstenberger, Jonathan Sandoval

Research output: Contribution to journalArticle

120 Citations (Scopus)

Abstract

Context: Tools to assess the evolving conceptual framework of physicians-in-training are limited, despite their critical importance to physicians' evolving clinical expertise. Concept mapping assessment (CMA) enables teachers to view students' organization of their knowledge at various points in training. Objective: To assess whether CMA reflects expected differences and changes in the conceptual framework of resident physicians, whether concept maps can be scored reliably, and how well CMA scores relate to the results of standard in-training examination. Design, Setting, and Participants: A group of 21 resident physicians (9 first-year and 12 second- and third-year residents) from a university-based pediatric training program underwent concept map training, drew a preinstruction concept map about seizures, completed an education course on seizures, and then drew a postinstruction map. Maps were scored independently by 3 raters using a standardized method. The study was conducted in May and June 1999. Main Outcome Measures: Preinstruction map total scores and subscores in 4 categories compared with postinstruction map scores; map scores of second- and third-year residents compared with first-year residents; and interrater correlation of map scores. Results: Total CMA scores increased after instruction from a mean (SD) preinstruction map score of 429 (119) to a mean postinstruction map score of 516 (196) (P=.03). Second- and third-year residents scored significantly higher than first-year residents before instruction (mean [SD] score of 472 [116] vs 371 [102], respectively; P=.04), but not after instruction (mean [SD] scores, 561 [203] vs 456 [179], respectively; P= .16). Second- and third-year residents had greater preinstruction map complexity as measured by cross-link score (P= .01) than first-year residents. The CMA score had a weak to no correlation with the American Board of Pediatrics In-training Examination score (r=0.10-0.54). Interrater correlation of map scoring ranged from weak to moderate for the preinstruction map (r=0.51-0.69) and moderate to strong for the postinstruction map (r=0.74-0.88). Conclusions: Our data provide preliminary evidence that concept mapping assessment reflects expected differences and change in the conceptual framework of resident physicians. Concept mapping assessment and standardized testing may measure different cognitive domains.

Original languageEnglish (US)
Pages (from-to)1105-1110
Number of pages6
JournalJournal of the American Medical Association
Volume284
Issue number9
StatePublished - Sep 6 2000

Fingerprint

Graduate Medical Education
Physicians
Seizures
Pediatrics
Education
Outcome Assessment (Health Care)
Thinking
Students

ASJC Scopus subject areas

  • Medicine(all)

Cite this

West, D. C., Pomeroy, J. R., Park, J. K., Gerstenberger, E. A., & Sandoval, J. (2000). Critical thinking in graduate medical education: A role for concept mapping assessment? Journal of the American Medical Association, 284(9), 1105-1110.

Critical thinking in graduate medical education : A role for concept mapping assessment? / West, Daniel C.; Pomeroy, J. Richard; Park, Jeanny K; Gerstenberger, Elise A.; Sandoval, Jonathan.

In: Journal of the American Medical Association, Vol. 284, No. 9, 06.09.2000, p. 1105-1110.

Research output: Contribution to journalArticle

West, DC, Pomeroy, JR, Park, JK, Gerstenberger, EA & Sandoval, J 2000, 'Critical thinking in graduate medical education: A role for concept mapping assessment?', Journal of the American Medical Association, vol. 284, no. 9, pp. 1105-1110.
West, Daniel C. ; Pomeroy, J. Richard ; Park, Jeanny K ; Gerstenberger, Elise A. ; Sandoval, Jonathan. / Critical thinking in graduate medical education : A role for concept mapping assessment?. In: Journal of the American Medical Association. 2000 ; Vol. 284, No. 9. pp. 1105-1110.
@article{6475856471654c90bfb0c7da7c9a4125,
title = "Critical thinking in graduate medical education: A role for concept mapping assessment?",
abstract = "Context: Tools to assess the evolving conceptual framework of physicians-in-training are limited, despite their critical importance to physicians' evolving clinical expertise. Concept mapping assessment (CMA) enables teachers to view students' organization of their knowledge at various points in training. Objective: To assess whether CMA reflects expected differences and changes in the conceptual framework of resident physicians, whether concept maps can be scored reliably, and how well CMA scores relate to the results of standard in-training examination. Design, Setting, and Participants: A group of 21 resident physicians (9 first-year and 12 second- and third-year residents) from a university-based pediatric training program underwent concept map training, drew a preinstruction concept map about seizures, completed an education course on seizures, and then drew a postinstruction map. Maps were scored independently by 3 raters using a standardized method. The study was conducted in May and June 1999. Main Outcome Measures: Preinstruction map total scores and subscores in 4 categories compared with postinstruction map scores; map scores of second- and third-year residents compared with first-year residents; and interrater correlation of map scores. Results: Total CMA scores increased after instruction from a mean (SD) preinstruction map score of 429 (119) to a mean postinstruction map score of 516 (196) (P=.03). Second- and third-year residents scored significantly higher than first-year residents before instruction (mean [SD] score of 472 [116] vs 371 [102], respectively; P=.04), but not after instruction (mean [SD] scores, 561 [203] vs 456 [179], respectively; P= .16). Second- and third-year residents had greater preinstruction map complexity as measured by cross-link score (P= .01) than first-year residents. The CMA score had a weak to no correlation with the American Board of Pediatrics In-training Examination score (r=0.10-0.54). Interrater correlation of map scoring ranged from weak to moderate for the preinstruction map (r=0.51-0.69) and moderate to strong for the postinstruction map (r=0.74-0.88). Conclusions: Our data provide preliminary evidence that concept mapping assessment reflects expected differences and change in the conceptual framework of resident physicians. Concept mapping assessment and standardized testing may measure different cognitive domains.",
author = "West, {Daniel C.} and Pomeroy, {J. Richard} and Park, {Jeanny K} and Gerstenberger, {Elise A.} and Jonathan Sandoval",
year = "2000",
month = "9",
day = "6",
language = "English (US)",
volume = "284",
pages = "1105--1110",
journal = "JAMA - Journal of the American Medical Association",
issn = "0002-9955",
publisher = "American Medical Association",
number = "9",

}

TY - JOUR

T1 - Critical thinking in graduate medical education

T2 - A role for concept mapping assessment?

AU - West, Daniel C.

AU - Pomeroy, J. Richard

AU - Park, Jeanny K

AU - Gerstenberger, Elise A.

AU - Sandoval, Jonathan

PY - 2000/9/6

Y1 - 2000/9/6

N2 - Context: Tools to assess the evolving conceptual framework of physicians-in-training are limited, despite their critical importance to physicians' evolving clinical expertise. Concept mapping assessment (CMA) enables teachers to view students' organization of their knowledge at various points in training. Objective: To assess whether CMA reflects expected differences and changes in the conceptual framework of resident physicians, whether concept maps can be scored reliably, and how well CMA scores relate to the results of standard in-training examination. Design, Setting, and Participants: A group of 21 resident physicians (9 first-year and 12 second- and third-year residents) from a university-based pediatric training program underwent concept map training, drew a preinstruction concept map about seizures, completed an education course on seizures, and then drew a postinstruction map. Maps were scored independently by 3 raters using a standardized method. The study was conducted in May and June 1999. Main Outcome Measures: Preinstruction map total scores and subscores in 4 categories compared with postinstruction map scores; map scores of second- and third-year residents compared with first-year residents; and interrater correlation of map scores. Results: Total CMA scores increased after instruction from a mean (SD) preinstruction map score of 429 (119) to a mean postinstruction map score of 516 (196) (P=.03). Second- and third-year residents scored significantly higher than first-year residents before instruction (mean [SD] score of 472 [116] vs 371 [102], respectively; P=.04), but not after instruction (mean [SD] scores, 561 [203] vs 456 [179], respectively; P= .16). Second- and third-year residents had greater preinstruction map complexity as measured by cross-link score (P= .01) than first-year residents. The CMA score had a weak to no correlation with the American Board of Pediatrics In-training Examination score (r=0.10-0.54). Interrater correlation of map scoring ranged from weak to moderate for the preinstruction map (r=0.51-0.69) and moderate to strong for the postinstruction map (r=0.74-0.88). Conclusions: Our data provide preliminary evidence that concept mapping assessment reflects expected differences and change in the conceptual framework of resident physicians. Concept mapping assessment and standardized testing may measure different cognitive domains.

AB - Context: Tools to assess the evolving conceptual framework of physicians-in-training are limited, despite their critical importance to physicians' evolving clinical expertise. Concept mapping assessment (CMA) enables teachers to view students' organization of their knowledge at various points in training. Objective: To assess whether CMA reflects expected differences and changes in the conceptual framework of resident physicians, whether concept maps can be scored reliably, and how well CMA scores relate to the results of standard in-training examination. Design, Setting, and Participants: A group of 21 resident physicians (9 first-year and 12 second- and third-year residents) from a university-based pediatric training program underwent concept map training, drew a preinstruction concept map about seizures, completed an education course on seizures, and then drew a postinstruction map. Maps were scored independently by 3 raters using a standardized method. The study was conducted in May and June 1999. Main Outcome Measures: Preinstruction map total scores and subscores in 4 categories compared with postinstruction map scores; map scores of second- and third-year residents compared with first-year residents; and interrater correlation of map scores. Results: Total CMA scores increased after instruction from a mean (SD) preinstruction map score of 429 (119) to a mean postinstruction map score of 516 (196) (P=.03). Second- and third-year residents scored significantly higher than first-year residents before instruction (mean [SD] score of 472 [116] vs 371 [102], respectively; P=.04), but not after instruction (mean [SD] scores, 561 [203] vs 456 [179], respectively; P= .16). Second- and third-year residents had greater preinstruction map complexity as measured by cross-link score (P= .01) than first-year residents. The CMA score had a weak to no correlation with the American Board of Pediatrics In-training Examination score (r=0.10-0.54). Interrater correlation of map scoring ranged from weak to moderate for the preinstruction map (r=0.51-0.69) and moderate to strong for the postinstruction map (r=0.74-0.88). Conclusions: Our data provide preliminary evidence that concept mapping assessment reflects expected differences and change in the conceptual framework of resident physicians. Concept mapping assessment and standardized testing may measure different cognitive domains.

UR - http://www.scopus.com/inward/record.url?scp=0034612773&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034612773&partnerID=8YFLogxK

M3 - Article

C2 - 10974689

AN - SCOPUS:0034612773

VL - 284

SP - 1105

EP - 1110

JO - JAMA - Journal of the American Medical Association

JF - JAMA - Journal of the American Medical Association

SN - 0002-9955

IS - 9

ER -