Predicting Performance on the American Board of Physical Medicine and Rehabilitation Written Examination Using Resident Self-Assessment Examination Scores

Alex Moroz, Heejung Bang

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

BACKGROUND: Studies across medical specialties have shown that scores on residency self-assessment examinations (SAEs) can predict performance on certifying board examinations.

OBJECTIVE: This study explored the predictive abilities of different composite SAE scores in physical medicine and rehabilitation and determined an optimal cut-point to identify an "at-risk" performance group.

METHODS: For our study, both predictive scores (SAE scores) and outcomes (board examination scores) are expressed in national percentile scores. We analyzed data in graduates of a physical medicine and rehabilitation residency program between 2008 and 2014. We compared mean, median, lowest, highest, and most recent score among up to 3 SAE scores with respect to their associations with the outcome via linear and logistic regression. We computed regression/correlation coefficient, P value, R (2), area under the curve, sensitivity, specificity, and predictive values. Identification of optimal cut-point was guided by accuracy, discrimination, and model-fit statistics.

RESULTS: Predictor and outcome data were available for 88 of 99 residents. In regression models, all SAE predictors showed significant associations (P ≤ .001) and the mean score performed best (r = 0.55). A 1-point increase in mean SAE was associated with a 1.88 score increase in board score and a 16% decrease in odds of failure. The rule of mean SAE score below 47 yielded the highest accuracy, highest discrimination, and best model fit.

CONCLUSIONS: Mean SAE score may be used to predict performance on the American Board of Physical Medicine and Rehabilitation-written examination. The optimal statistical cut-point to identify the at-risk group for failure appears to be around the 47th SAE national percentile.

Original languageEnglish (US)
Pages (from-to)50-56
Number of pages7
JournalJournal of graduate medical education
Volume8
Issue number1
DOIs
StatePublished - Feb 1 2016

Fingerprint

Physical and Rehabilitation Medicine
Internship and Residency
Self-Assessment
Aptitude
Area Under Curve
Linear Models
Logistic Models
Medicine
Sensitivity and Specificity

ASJC Scopus subject areas

  • Medicine(all)

Cite this

@article{d040b460a7fc4c5b995ed0c2b0706569,
title = "Predicting Performance on the American Board of Physical Medicine and Rehabilitation Written Examination Using Resident Self-Assessment Examination Scores",
abstract = "BACKGROUND: Studies across medical specialties have shown that scores on residency self-assessment examinations (SAEs) can predict performance on certifying board examinations.OBJECTIVE: This study explored the predictive abilities of different composite SAE scores in physical medicine and rehabilitation and determined an optimal cut-point to identify an {"}at-risk{"} performance group.METHODS: For our study, both predictive scores (SAE scores) and outcomes (board examination scores) are expressed in national percentile scores. We analyzed data in graduates of a physical medicine and rehabilitation residency program between 2008 and 2014. We compared mean, median, lowest, highest, and most recent score among up to 3 SAE scores with respect to their associations with the outcome via linear and logistic regression. We computed regression/correlation coefficient, P value, R (2), area under the curve, sensitivity, specificity, and predictive values. Identification of optimal cut-point was guided by accuracy, discrimination, and model-fit statistics.RESULTS: Predictor and outcome data were available for 88 of 99 residents. In regression models, all SAE predictors showed significant associations (P ≤ .001) and the mean score performed best (r = 0.55). A 1-point increase in mean SAE was associated with a 1.88 score increase in board score and a 16{\%} decrease in odds of failure. The rule of mean SAE score below 47 yielded the highest accuracy, highest discrimination, and best model fit.CONCLUSIONS: Mean SAE score may be used to predict performance on the American Board of Physical Medicine and Rehabilitation-written examination. The optimal statistical cut-point to identify the at-risk group for failure appears to be around the 47th SAE national percentile.",
author = "Alex Moroz and Heejung Bang",
year = "2016",
month = "2",
day = "1",
doi = "10.4300/JGME-D-15-00065.1",
language = "English (US)",
volume = "8",
pages = "50--56",
journal = "Journal of graduate medical education",
issn = "1949-8349",
publisher = "University of Finance and Management",
number = "1",

}

TY - JOUR

T1 - Predicting Performance on the American Board of Physical Medicine and Rehabilitation Written Examination Using Resident Self-Assessment Examination Scores

AU - Moroz, Alex

AU - Bang, Heejung

PY - 2016/2/1

Y1 - 2016/2/1

N2 - BACKGROUND: Studies across medical specialties have shown that scores on residency self-assessment examinations (SAEs) can predict performance on certifying board examinations.OBJECTIVE: This study explored the predictive abilities of different composite SAE scores in physical medicine and rehabilitation and determined an optimal cut-point to identify an "at-risk" performance group.METHODS: For our study, both predictive scores (SAE scores) and outcomes (board examination scores) are expressed in national percentile scores. We analyzed data in graduates of a physical medicine and rehabilitation residency program between 2008 and 2014. We compared mean, median, lowest, highest, and most recent score among up to 3 SAE scores with respect to their associations with the outcome via linear and logistic regression. We computed regression/correlation coefficient, P value, R (2), area under the curve, sensitivity, specificity, and predictive values. Identification of optimal cut-point was guided by accuracy, discrimination, and model-fit statistics.RESULTS: Predictor and outcome data were available for 88 of 99 residents. In regression models, all SAE predictors showed significant associations (P ≤ .001) and the mean score performed best (r = 0.55). A 1-point increase in mean SAE was associated with a 1.88 score increase in board score and a 16% decrease in odds of failure. The rule of mean SAE score below 47 yielded the highest accuracy, highest discrimination, and best model fit.CONCLUSIONS: Mean SAE score may be used to predict performance on the American Board of Physical Medicine and Rehabilitation-written examination. The optimal statistical cut-point to identify the at-risk group for failure appears to be around the 47th SAE national percentile.

AB - BACKGROUND: Studies across medical specialties have shown that scores on residency self-assessment examinations (SAEs) can predict performance on certifying board examinations.OBJECTIVE: This study explored the predictive abilities of different composite SAE scores in physical medicine and rehabilitation and determined an optimal cut-point to identify an "at-risk" performance group.METHODS: For our study, both predictive scores (SAE scores) and outcomes (board examination scores) are expressed in national percentile scores. We analyzed data in graduates of a physical medicine and rehabilitation residency program between 2008 and 2014. We compared mean, median, lowest, highest, and most recent score among up to 3 SAE scores with respect to their associations with the outcome via linear and logistic regression. We computed regression/correlation coefficient, P value, R (2), area under the curve, sensitivity, specificity, and predictive values. Identification of optimal cut-point was guided by accuracy, discrimination, and model-fit statistics.RESULTS: Predictor and outcome data were available for 88 of 99 residents. In regression models, all SAE predictors showed significant associations (P ≤ .001) and the mean score performed best (r = 0.55). A 1-point increase in mean SAE was associated with a 1.88 score increase in board score and a 16% decrease in odds of failure. The rule of mean SAE score below 47 yielded the highest accuracy, highest discrimination, and best model fit.CONCLUSIONS: Mean SAE score may be used to predict performance on the American Board of Physical Medicine and Rehabilitation-written examination. The optimal statistical cut-point to identify the at-risk group for failure appears to be around the 47th SAE national percentile.

UR - http://www.scopus.com/inward/record.url?scp=85020346460&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020346460&partnerID=8YFLogxK

U2 - 10.4300/JGME-D-15-00065.1

DO - 10.4300/JGME-D-15-00065.1

M3 - Article

C2 - 26913103

AN - SCOPUS:85020346460

VL - 8

SP - 50

EP - 56

JO - Journal of graduate medical education

JF - Journal of graduate medical education

SN - 1949-8349

IS - 1

ER -