Reliability of Physician-Level Measures of Patient Experience in Primary Care

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Background: Patient experience measures are widely used to compare performance at the individual physician level. Objective: To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level. Design: Repeated cross-sectional design. Setting: Academic family medicine practice in California. Participants: One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians). Measurements: Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status). Results: The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R2 = 20.0% vs. 9.6%, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1% to 3.4% and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3% (95% CI: 7.0–25.2%)]. Conclusions: Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.

Original languageEnglish (US)
Pages (from-to)1-7
Number of pages7
JournalJournal of General Internal Medicine
DOIs
StateAccepted/In press - Sep 12 2017

Fingerprint

Primary Health Care
Physicians
Social Adjustment
Primary Care Physicians
Sample Size
Delivery of Health Care
Family Practice
Marital Status
Patient Satisfaction
Health Personnel
Personality
Medicine

ASJC Scopus subject areas

  • Internal Medicine

Cite this

@article{0cb8e408028c4a2f89134a946f3fd01f,
title = "Reliability of Physician-Level Measures of Patient Experience in Primary Care",
abstract = "Background: Patient experience measures are widely used to compare performance at the individual physician level. Objective: To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level. Design: Repeated cross-sectional design. Setting: Academic family medicine practice in California. Participants: One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians). Measurements: Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status). Results: The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R2 = 20.0{\%} vs. 9.6{\%}, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1{\%} to 3.4{\%} and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3{\%} (95{\%} CI: 7.0–25.2{\%})]. Conclusions: Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.",
author = "Fenton, {Joshua J} and Jerant, {Anthony F} and Kravitz, {Richard L} and Bertakis, {Klea D} and Tancredi, {Daniel J} and Elizabeth Magnan and Peter Franks",
year = "2017",
month = "9",
day = "12",
doi = "10.1007/s11606-017-4175-y",
language = "English (US)",
pages = "1--7",
journal = "Journal of General Internal Medicine",
issn = "0884-8734",
publisher = "Springer New York",

}

TY - JOUR

T1 - Reliability of Physician-Level Measures of Patient Experience in Primary Care

AU - Fenton, Joshua J

AU - Jerant, Anthony F

AU - Kravitz, Richard L

AU - Bertakis, Klea D

AU - Tancredi, Daniel J

AU - Magnan, Elizabeth

AU - Franks, Peter

PY - 2017/9/12

Y1 - 2017/9/12

N2 - Background: Patient experience measures are widely used to compare performance at the individual physician level. Objective: To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level. Design: Repeated cross-sectional design. Setting: Academic family medicine practice in California. Participants: One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians). Measurements: Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status). Results: The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R2 = 20.0% vs. 9.6%, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1% to 3.4% and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3% (95% CI: 7.0–25.2%)]. Conclusions: Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.

AB - Background: Patient experience measures are widely used to compare performance at the individual physician level. Objective: To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level. Design: Repeated cross-sectional design. Setting: Academic family medicine practice in California. Participants: One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians). Measurements: Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status). Results: The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R2 = 20.0% vs. 9.6%, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1% to 3.4% and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3% (95% CI: 7.0–25.2%)]. Conclusions: Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.

UR - http://www.scopus.com/inward/record.url?scp=85029170367&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029170367&partnerID=8YFLogxK

U2 - 10.1007/s11606-017-4175-y

DO - 10.1007/s11606-017-4175-y

M3 - Article

SP - 1

EP - 7

JO - Journal of General Internal Medicine

JF - Journal of General Internal Medicine

SN - 0884-8734

ER -