Measuring teaching effectiveness in a pre-clinical multi-instructor course: A case study in the development and application of a brief instructor rating scale

Martin H Leamon, Laurie Fields

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

Background: Despite widespread use, misunderstandings persist about student evaluations of teaching. These evaluations have not been well examined in the common medical school setting of the multi-instructor, preclinical lecture course. Purpose: The study evaluated the psychometrics of a brief student evaluation of a teaching instrument developed for a multi-instructor 2nd-year course and described its application. Methods: An 11-item instrument was developed and administered to 276 students to evaluate 27 lecturers per year in 3 years of an introductory clinical psychiatry course. A fully crossed research design allowed for a thorough analysis of variability in ratings. Results: Generalizability analysis showed good reliability and relatively large Student x Lecturer interactions. Profile analysis generated distinct lecturer teaching profiles. Conclusions: Judicious use of a psychometric ally sound student evaluation of a teaching instrument can be used to assist faculty and course development. Administering the evaluation instrument to an entire class produces no better reliability than administration to randomly selected subgroups of students.

Original languageEnglish (US)
Pages (from-to)119-129
Number of pages11
JournalTeaching and Learning in Medicine
Volume17
Issue number2
DOIs
StatePublished - Mar 2005

Fingerprint

rating scale
instructor
Teaching
Students
evaluation
university teacher
student
Psychometrics
psychometrics
Medical Schools
allies
psychiatry
research planning
Psychiatry
Research Design
rating
interaction
school

ASJC Scopus subject areas

  • Nursing(all)
  • Education

Cite this

@article{85fbb97aa486488d84915fabe3481391,
title = "Measuring teaching effectiveness in a pre-clinical multi-instructor course: A case study in the development and application of a brief instructor rating scale",
abstract = "Background: Despite widespread use, misunderstandings persist about student evaluations of teaching. These evaluations have not been well examined in the common medical school setting of the multi-instructor, preclinical lecture course. Purpose: The study evaluated the psychometrics of a brief student evaluation of a teaching instrument developed for a multi-instructor 2nd-year course and described its application. Methods: An 11-item instrument was developed and administered to 276 students to evaluate 27 lecturers per year in 3 years of an introductory clinical psychiatry course. A fully crossed research design allowed for a thorough analysis of variability in ratings. Results: Generalizability analysis showed good reliability and relatively large Student x Lecturer interactions. Profile analysis generated distinct lecturer teaching profiles. Conclusions: Judicious use of a psychometric ally sound student evaluation of a teaching instrument can be used to assist faculty and course development. Administering the evaluation instrument to an entire class produces no better reliability than administration to randomly selected subgroups of students.",
author = "Leamon, {Martin H} and Laurie Fields",
year = "2005",
month = "3",
doi = "10.1207/s15328015tlm1702_5",
language = "English (US)",
volume = "17",
pages = "119--129",
journal = "Teaching and Learning in Medicine",
issn = "1040-1334",
publisher = "Routledge",
number = "2",

}

TY - JOUR

T1 - Measuring teaching effectiveness in a pre-clinical multi-instructor course

T2 - A case study in the development and application of a brief instructor rating scale

AU - Leamon, Martin H

AU - Fields, Laurie

PY - 2005/3

Y1 - 2005/3

N2 - Background: Despite widespread use, misunderstandings persist about student evaluations of teaching. These evaluations have not been well examined in the common medical school setting of the multi-instructor, preclinical lecture course. Purpose: The study evaluated the psychometrics of a brief student evaluation of a teaching instrument developed for a multi-instructor 2nd-year course and described its application. Methods: An 11-item instrument was developed and administered to 276 students to evaluate 27 lecturers per year in 3 years of an introductory clinical psychiatry course. A fully crossed research design allowed for a thorough analysis of variability in ratings. Results: Generalizability analysis showed good reliability and relatively large Student x Lecturer interactions. Profile analysis generated distinct lecturer teaching profiles. Conclusions: Judicious use of a psychometric ally sound student evaluation of a teaching instrument can be used to assist faculty and course development. Administering the evaluation instrument to an entire class produces no better reliability than administration to randomly selected subgroups of students.

AB - Background: Despite widespread use, misunderstandings persist about student evaluations of teaching. These evaluations have not been well examined in the common medical school setting of the multi-instructor, preclinical lecture course. Purpose: The study evaluated the psychometrics of a brief student evaluation of a teaching instrument developed for a multi-instructor 2nd-year course and described its application. Methods: An 11-item instrument was developed and administered to 276 students to evaluate 27 lecturers per year in 3 years of an introductory clinical psychiatry course. A fully crossed research design allowed for a thorough analysis of variability in ratings. Results: Generalizability analysis showed good reliability and relatively large Student x Lecturer interactions. Profile analysis generated distinct lecturer teaching profiles. Conclusions: Judicious use of a psychometric ally sound student evaluation of a teaching instrument can be used to assist faculty and course development. Administering the evaluation instrument to an entire class produces no better reliability than administration to randomly selected subgroups of students.

UR - http://www.scopus.com/inward/record.url?scp=20344373436&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=20344373436&partnerID=8YFLogxK

U2 - 10.1207/s15328015tlm1702_5

DO - 10.1207/s15328015tlm1702_5

M3 - Article

C2 - 15833721

AN - SCOPUS:20344373436

VL - 17

SP - 119

EP - 129

JO - Teaching and Learning in Medicine

JF - Teaching and Learning in Medicine

SN - 1040-1334

IS - 2

ER -