Reliability of Multiple Mini-Interviews and traditional interviews within and between institutions: A study of five California medical schools

Anthony F Jerant, Mark C Henderson, Erin Griffin, Julie A. Rainwater, Theodore R. Hall, Carolyn J. Kelly, Ellena M. Peterson, David Wofsy, Peter Franks

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Background: Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs. Methods: The analyses included applicants interviewing at ≥1 of the five schools during 2011-2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date. Results: A total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5%) interviewed at both MMI schools and 687 (20.2%) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson's r 0.07, 0.13, and 0.29, and Cronbach's α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach's α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95% CI 0.20-0.35) for TIs and 0.47 (95% CI 0.41-0.54) for MMIs. Conclusions: Within and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.

Original languageEnglish (US)
Article number190
JournalBMC Medical Education
Volume17
Issue number1
DOIs
StatePublished - Nov 6 2017

Fingerprint

interview
school
applicant
linear model
candidacy

Keywords

  • Interview as topic
  • Multiple mini-interview
  • Reproducibility of results
  • School admission criteria
  • Schools, medical

ASJC Scopus subject areas

  • Education

Cite this

Reliability of Multiple Mini-Interviews and traditional interviews within and between institutions : A study of five California medical schools. / Jerant, Anthony F; Henderson, Mark C; Griffin, Erin; Rainwater, Julie A.; Hall, Theodore R.; Kelly, Carolyn J.; Peterson, Ellena M.; Wofsy, David; Franks, Peter.

In: BMC Medical Education, Vol. 17, No. 1, 190, 06.11.2017.

Research output: Contribution to journalArticle

Jerant, Anthony F ; Henderson, Mark C ; Griffin, Erin ; Rainwater, Julie A. ; Hall, Theodore R. ; Kelly, Carolyn J. ; Peterson, Ellena M. ; Wofsy, David ; Franks, Peter. / Reliability of Multiple Mini-Interviews and traditional interviews within and between institutions : A study of five California medical schools. In: BMC Medical Education. 2017 ; Vol. 17, No. 1.
@article{b6b52e24a4c24ffb88cb1268c3ccfc0c,
title = "Reliability of Multiple Mini-Interviews and traditional interviews within and between institutions: A study of five California medical schools",
abstract = "Background: Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs. Methods: The analyses included applicants interviewing at ≥1 of the five schools during 2011-2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date. Results: A total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5{\%}) interviewed at both MMI schools and 687 (20.2{\%}) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson's r 0.07, 0.13, and 0.29, and Cronbach's α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach's α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95{\%} CI 0.20-0.35) for TIs and 0.47 (95{\%} CI 0.41-0.54) for MMIs. Conclusions: Within and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.",
keywords = "Interview as topic, Multiple mini-interview, Reproducibility of results, School admission criteria, Schools, medical",
author = "Jerant, {Anthony F} and Henderson, {Mark C} and Erin Griffin and Rainwater, {Julie A.} and Hall, {Theodore R.} and Kelly, {Carolyn J.} and Peterson, {Ellena M.} and David Wofsy and Peter Franks",
year = "2017",
month = "11",
day = "6",
doi = "10.1186/s12909-017-1030-0",
language = "English (US)",
volume = "17",
journal = "BMC Medical Education",
issn = "1472-6920",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - Reliability of Multiple Mini-Interviews and traditional interviews within and between institutions

T2 - A study of five California medical schools

AU - Jerant, Anthony F

AU - Henderson, Mark C

AU - Griffin, Erin

AU - Rainwater, Julie A.

AU - Hall, Theodore R.

AU - Kelly, Carolyn J.

AU - Peterson, Ellena M.

AU - Wofsy, David

AU - Franks, Peter

PY - 2017/11/6

Y1 - 2017/11/6

N2 - Background: Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs. Methods: The analyses included applicants interviewing at ≥1 of the five schools during 2011-2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date. Results: A total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5%) interviewed at both MMI schools and 687 (20.2%) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson's r 0.07, 0.13, and 0.29, and Cronbach's α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach's α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95% CI 0.20-0.35) for TIs and 0.47 (95% CI 0.41-0.54) for MMIs. Conclusions: Within and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.

AB - Background: Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs. Methods: The analyses included applicants interviewing at ≥1 of the five schools during 2011-2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date. Results: A total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5%) interviewed at both MMI schools and 687 (20.2%) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson's r 0.07, 0.13, and 0.29, and Cronbach's α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach's α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95% CI 0.20-0.35) for TIs and 0.47 (95% CI 0.41-0.54) for MMIs. Conclusions: Within and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.

KW - Interview as topic

KW - Multiple mini-interview

KW - Reproducibility of results

KW - School admission criteria

KW - Schools, medical

UR - http://www.scopus.com/inward/record.url?scp=85032922404&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85032922404&partnerID=8YFLogxK

U2 - 10.1186/s12909-017-1030-0

DO - 10.1186/s12909-017-1030-0

M3 - Article

C2 - 29110662

AN - SCOPUS:85032922404

VL - 17

JO - BMC Medical Education

JF - BMC Medical Education

SN - 1472-6920

IS - 1

M1 - 190

ER -