Abstract
Background: Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs. Methods: The analyses included applicants interviewing at ≥1 of the five schools during 2011-2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date. Results: A total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5%) interviewed at both MMI schools and 687 (20.2%) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson's r 0.07, 0.13, and 0.29, and Cronbach's α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach's α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95% CI 0.20-0.35) for TIs and 0.47 (95% CI 0.41-0.54) for MMIs. Conclusions: Within and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.
| Original language | English (US) |
|---|---|
| Article number | 190 |
| Journal | BMC Medical Education |
| Volume | 17 |
| Issue number | 1 |
| DOIs | |
| State | Published - Nov 6 2017 |
Fingerprint
Keywords
- Interview as topic
- Multiple mini-interview
- Reproducibility of results
- School admission criteria
- Schools, medical
ASJC Scopus subject areas
- Education
Cite this
Reliability of Multiple Mini-Interviews and traditional interviews within and between institutions : A study of five California medical schools. / Jerant, Anthony F; Henderson, Mark C; Griffin, Erin; Rainwater, Julie A.; Hall, Theodore R.; Kelly, Carolyn J.; Peterson, Ellena M.; Wofsy, David; Franks, Peter.
In: BMC Medical Education, Vol. 17, No. 1, 190, 06.11.2017.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Reliability of Multiple Mini-Interviews and traditional interviews within and between institutions
T2 - A study of five California medical schools
AU - Jerant, Anthony F
AU - Henderson, Mark C
AU - Griffin, Erin
AU - Rainwater, Julie A.
AU - Hall, Theodore R.
AU - Kelly, Carolyn J.
AU - Peterson, Ellena M.
AU - Wofsy, David
AU - Franks, Peter
PY - 2017/11/6
Y1 - 2017/11/6
N2 - Background: Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs. Methods: The analyses included applicants interviewing at ≥1 of the five schools during 2011-2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date. Results: A total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5%) interviewed at both MMI schools and 687 (20.2%) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson's r 0.07, 0.13, and 0.29, and Cronbach's α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach's α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95% CI 0.20-0.35) for TIs and 0.47 (95% CI 0.41-0.54) for MMIs. Conclusions: Within and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.
AB - Background: Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs. Methods: The analyses included applicants interviewing at ≥1 of the five schools during 2011-2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date. Results: A total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5%) interviewed at both MMI schools and 687 (20.2%) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson's r 0.07, 0.13, and 0.29, and Cronbach's α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach's α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95% CI 0.20-0.35) for TIs and 0.47 (95% CI 0.41-0.54) for MMIs. Conclusions: Within and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.
KW - Interview as topic
KW - Multiple mini-interview
KW - Reproducibility of results
KW - School admission criteria
KW - Schools, medical
UR - http://www.scopus.com/inward/record.url?scp=85032922404&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85032922404&partnerID=8YFLogxK
U2 - 10.1186/s12909-017-1030-0
DO - 10.1186/s12909-017-1030-0
M3 - Article
C2 - 29110662
AN - SCOPUS:85032922404
VL - 17
JO - BMC Medical Education
JF - BMC Medical Education
SN - 1472-6920
IS - 1
M1 - 190
ER -