Connoisseurs of care? Unannounced standardized patients' ratings of physicians

Malathi Srinivasan, Peter Franks, Lisa S. Meredith, Kevin Fiscella, Ronald M. Epstein, Richard L Kravitz

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

BACKGROUND: Patient satisfaction surveys can be informative, but bias and poor response rates may limit their utility as stable measures of physician performance. Using unannounced standardized patients (SPs) may overcome some of these limitations because their experience and training make them able judges of physician behavior. OBJECTIVES: We sought to understand the reliability of unannounced SPs in rating primary care physicians when covertly presenting as real patients. STUDY DESIGN: Data from 2 studies (Patient Centered Communication [PCC]; Social Influences in Practice [SIP]) were included. For the PCC study, 5 SPs made 192 visits to 96 physicians; for the SIP study, 18 SPs made 292 visits to 146 physicians. SPs visits to physicians were randomized, thus avoiding mutual selection bias. Each SP rated 16 to 38 physicians on interpersonal skills (autonomy support: PCC, SIP), technical skills (information gathering: SIP-only), and overall satisfaction (SIP-only). We evaluated SP evaluation consistency (physician vs. total variance ρ), and SPs' overall satisfaction with specific dimensions of physician performance. RESULTS: Scale reliability varied from 0.71 to 0.92. Physician rhos (95% confidence intervals) for autonomy support were 0.40 (0.22-0.58; PCC) and 0.30 (0.14-0.45; SIP); information gathering rho was 0.46 (0.33-0.59; SIP). Overall SP satisfaction rho was 0.47 (0.34-0.60; SIP). SPs varied significantly in adjusted overall satisfaction levels, but not other dimensions. CONCLUSIONS: These analyses provide some evidence that medical connoisseurship can be learned. When adequately sampled by trained SPs, some physician skills can be reliably measured in community practice settings.

Original languageEnglish (US)
Pages (from-to)1092-1098
Number of pages7
JournalMedical Care
Volume44
Issue number12
DOIs
StatePublished - Dec 2006

Fingerprint

rating
physician
Physicians
Patient Satisfaction
Communication
communication
autonomy
trend
Selection Bias
performance
Primary Care Physicians
confidence
Confidence Intervals
evaluation
community
evidence

Keywords

  • Autonomy support
  • Clinical skills and interpersonal skills evaluation
  • Competency assessment
  • Patient satisfaction
  • Physician performance evaluation
  • Primary care
  • Unannounced standardized patient

ASJC Scopus subject areas

  • Nursing(all)
  • Public Health, Environmental and Occupational Health
  • Health(social science)
  • Health Professions(all)

Cite this

Connoisseurs of care? Unannounced standardized patients' ratings of physicians. / Srinivasan, Malathi; Franks, Peter; Meredith, Lisa S.; Fiscella, Kevin; Epstein, Ronald M.; Kravitz, Richard L.

In: Medical Care, Vol. 44, No. 12, 12.2006, p. 1092-1098.

Research output: Contribution to journalArticle

Srinivasan, Malathi ; Franks, Peter ; Meredith, Lisa S. ; Fiscella, Kevin ; Epstein, Ronald M. ; Kravitz, Richard L. / Connoisseurs of care? Unannounced standardized patients' ratings of physicians. In: Medical Care. 2006 ; Vol. 44, No. 12. pp. 1092-1098.
@article{6a84dfc3731a4407b10166f626cea50b,
title = "Connoisseurs of care? Unannounced standardized patients' ratings of physicians",
abstract = "BACKGROUND: Patient satisfaction surveys can be informative, but bias and poor response rates may limit their utility as stable measures of physician performance. Using unannounced standardized patients (SPs) may overcome some of these limitations because their experience and training make them able judges of physician behavior. OBJECTIVES: We sought to understand the reliability of unannounced SPs in rating primary care physicians when covertly presenting as real patients. STUDY DESIGN: Data from 2 studies (Patient Centered Communication [PCC]; Social Influences in Practice [SIP]) were included. For the PCC study, 5 SPs made 192 visits to 96 physicians; for the SIP study, 18 SPs made 292 visits to 146 physicians. SPs visits to physicians were randomized, thus avoiding mutual selection bias. Each SP rated 16 to 38 physicians on interpersonal skills (autonomy support: PCC, SIP), technical skills (information gathering: SIP-only), and overall satisfaction (SIP-only). We evaluated SP evaluation consistency (physician vs. total variance ρ), and SPs' overall satisfaction with specific dimensions of physician performance. RESULTS: Scale reliability varied from 0.71 to 0.92. Physician rhos (95{\%} confidence intervals) for autonomy support were 0.40 (0.22-0.58; PCC) and 0.30 (0.14-0.45; SIP); information gathering rho was 0.46 (0.33-0.59; SIP). Overall SP satisfaction rho was 0.47 (0.34-0.60; SIP). SPs varied significantly in adjusted overall satisfaction levels, but not other dimensions. CONCLUSIONS: These analyses provide some evidence that medical connoisseurship can be learned. When adequately sampled by trained SPs, some physician skills can be reliably measured in community practice settings.",
keywords = "Autonomy support, Clinical skills and interpersonal skills evaluation, Competency assessment, Patient satisfaction, Physician performance evaluation, Primary care, Unannounced standardized patient",
author = "Malathi Srinivasan and Peter Franks and Meredith, {Lisa S.} and Kevin Fiscella and Epstein, {Ronald M.} and Kravitz, {Richard L}",
year = "2006",
month = "12",
doi = "10.1097/01.mlr.0000237197.92152.5e",
language = "English (US)",
volume = "44",
pages = "1092--1098",
journal = "Medical Care",
issn = "0025-7079",
publisher = "Lippincott Williams and Wilkins",
number = "12",

}

TY - JOUR

T1 - Connoisseurs of care? Unannounced standardized patients' ratings of physicians

AU - Srinivasan, Malathi

AU - Franks, Peter

AU - Meredith, Lisa S.

AU - Fiscella, Kevin

AU - Epstein, Ronald M.

AU - Kravitz, Richard L

PY - 2006/12

Y1 - 2006/12

N2 - BACKGROUND: Patient satisfaction surveys can be informative, but bias and poor response rates may limit their utility as stable measures of physician performance. Using unannounced standardized patients (SPs) may overcome some of these limitations because their experience and training make them able judges of physician behavior. OBJECTIVES: We sought to understand the reliability of unannounced SPs in rating primary care physicians when covertly presenting as real patients. STUDY DESIGN: Data from 2 studies (Patient Centered Communication [PCC]; Social Influences in Practice [SIP]) were included. For the PCC study, 5 SPs made 192 visits to 96 physicians; for the SIP study, 18 SPs made 292 visits to 146 physicians. SPs visits to physicians were randomized, thus avoiding mutual selection bias. Each SP rated 16 to 38 physicians on interpersonal skills (autonomy support: PCC, SIP), technical skills (information gathering: SIP-only), and overall satisfaction (SIP-only). We evaluated SP evaluation consistency (physician vs. total variance ρ), and SPs' overall satisfaction with specific dimensions of physician performance. RESULTS: Scale reliability varied from 0.71 to 0.92. Physician rhos (95% confidence intervals) for autonomy support were 0.40 (0.22-0.58; PCC) and 0.30 (0.14-0.45; SIP); information gathering rho was 0.46 (0.33-0.59; SIP). Overall SP satisfaction rho was 0.47 (0.34-0.60; SIP). SPs varied significantly in adjusted overall satisfaction levels, but not other dimensions. CONCLUSIONS: These analyses provide some evidence that medical connoisseurship can be learned. When adequately sampled by trained SPs, some physician skills can be reliably measured in community practice settings.

AB - BACKGROUND: Patient satisfaction surveys can be informative, but bias and poor response rates may limit their utility as stable measures of physician performance. Using unannounced standardized patients (SPs) may overcome some of these limitations because their experience and training make them able judges of physician behavior. OBJECTIVES: We sought to understand the reliability of unannounced SPs in rating primary care physicians when covertly presenting as real patients. STUDY DESIGN: Data from 2 studies (Patient Centered Communication [PCC]; Social Influences in Practice [SIP]) were included. For the PCC study, 5 SPs made 192 visits to 96 physicians; for the SIP study, 18 SPs made 292 visits to 146 physicians. SPs visits to physicians were randomized, thus avoiding mutual selection bias. Each SP rated 16 to 38 physicians on interpersonal skills (autonomy support: PCC, SIP), technical skills (information gathering: SIP-only), and overall satisfaction (SIP-only). We evaluated SP evaluation consistency (physician vs. total variance ρ), and SPs' overall satisfaction with specific dimensions of physician performance. RESULTS: Scale reliability varied from 0.71 to 0.92. Physician rhos (95% confidence intervals) for autonomy support were 0.40 (0.22-0.58; PCC) and 0.30 (0.14-0.45; SIP); information gathering rho was 0.46 (0.33-0.59; SIP). Overall SP satisfaction rho was 0.47 (0.34-0.60; SIP). SPs varied significantly in adjusted overall satisfaction levels, but not other dimensions. CONCLUSIONS: These analyses provide some evidence that medical connoisseurship can be learned. When adequately sampled by trained SPs, some physician skills can be reliably measured in community practice settings.

KW - Autonomy support

KW - Clinical skills and interpersonal skills evaluation

KW - Competency assessment

KW - Patient satisfaction

KW - Physician performance evaluation

KW - Primary care

KW - Unannounced standardized patient

UR - http://www.scopus.com/inward/record.url?scp=33751321873&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33751321873&partnerID=8YFLogxK

U2 - 10.1097/01.mlr.0000237197.92152.5e

DO - 10.1097/01.mlr.0000237197.92152.5e

M3 - Article

VL - 44

SP - 1092

EP - 1098

JO - Medical Care

JF - Medical Care

SN - 0025-7079

IS - 12

ER -