Classification accuracy of claims-based methods for identifying providers failing to meet performance targets

Rebecca A. Hubbard, Rhondee Benjamin-Johnson, Tracy Onega, Rebecca Smith-Bindman, Weiwei Zhu, Joshua J Fenton

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

Quality assessment is critical for healthcare reform, but data sources are lacking for measurement of many important healthcare outcomes. With over 49 million people covered by Medicare as of 2010, Medicare claims data offer a potentially valuable source that could be used in targeted health care quality improvement efforts. However, little is known about the operating characteristics of provider profiling methods using claims-based outcome measures that may estimate provider performance with error. Motivated by the example of screening mammography performance, we compared approaches to identifying providers failing to meet guideline targets using Medicare claims data. We used data from the Breast Cancer Surveillance Consortium and linked Medicare claims to compare claims-based and clinical estimates of cancer detection rate. We then demonstrated the performance of claim-based estimates across a broad range of operating characteristics using simulation studies. We found that identification of poor performing providers was extremely sensitive to algorithm specificity, with no approach identifying more than 65% of poor performing providers when claims-based measures had specificity of 0.995 or less. We conclude that claims have the potential to contribute important information on healthcare outcomes to quality improvement efforts. However, to achieve this potential, development of highly accurate claims-based outcome measures should remain a priority.

Original languageEnglish (US)
Pages (from-to)93-105
Number of pages13
JournalStatistics in Medicine
Volume34
Issue number1
DOIs
StatePublished - Jan 15 2015

Fingerprint

Medicare
Target
Quality Improvement
Delivery of Health Care
Healthcare
Outcome Assessment (Health Care)
Health Care Reform
Information Storage and Retrieval
Mammography
Operating Characteristics
Specificity
Guidelines
Breast Neoplasms
Estimate
Quality Assessment
Profiling
Breast Cancer
Neoplasms
Surveillance
Screening

Keywords

  • Breast cancer
  • Hierarchical models
  • Medicare
  • Provider profiling

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability

Cite this

Classification accuracy of claims-based methods for identifying providers failing to meet performance targets. / Hubbard, Rebecca A.; Benjamin-Johnson, Rhondee; Onega, Tracy; Smith-Bindman, Rebecca; Zhu, Weiwei; Fenton, Joshua J.

In: Statistics in Medicine, Vol. 34, No. 1, 15.01.2015, p. 93-105.

Research output: Contribution to journalArticle

Hubbard, Rebecca A. ; Benjamin-Johnson, Rhondee ; Onega, Tracy ; Smith-Bindman, Rebecca ; Zhu, Weiwei ; Fenton, Joshua J. / Classification accuracy of claims-based methods for identifying providers failing to meet performance targets. In: Statistics in Medicine. 2015 ; Vol. 34, No. 1. pp. 93-105.
@article{4bba1c22297841e197164dc55c696bdb,
title = "Classification accuracy of claims-based methods for identifying providers failing to meet performance targets",
abstract = "Quality assessment is critical for healthcare reform, but data sources are lacking for measurement of many important healthcare outcomes. With over 49 million people covered by Medicare as of 2010, Medicare claims data offer a potentially valuable source that could be used in targeted health care quality improvement efforts. However, little is known about the operating characteristics of provider profiling methods using claims-based outcome measures that may estimate provider performance with error. Motivated by the example of screening mammography performance, we compared approaches to identifying providers failing to meet guideline targets using Medicare claims data. We used data from the Breast Cancer Surveillance Consortium and linked Medicare claims to compare claims-based and clinical estimates of cancer detection rate. We then demonstrated the performance of claim-based estimates across a broad range of operating characteristics using simulation studies. We found that identification of poor performing providers was extremely sensitive to algorithm specificity, with no approach identifying more than 65{\%} of poor performing providers when claims-based measures had specificity of 0.995 or less. We conclude that claims have the potential to contribute important information on healthcare outcomes to quality improvement efforts. However, to achieve this potential, development of highly accurate claims-based outcome measures should remain a priority.",
keywords = "Breast cancer, Hierarchical models, Medicare, Provider profiling",
author = "Hubbard, {Rebecca A.} and Rhondee Benjamin-Johnson and Tracy Onega and Rebecca Smith-Bindman and Weiwei Zhu and Fenton, {Joshua J}",
year = "2015",
month = "1",
day = "15",
doi = "10.1002/sim.6318",
language = "English (US)",
volume = "34",
pages = "93--105",
journal = "Statistics in Medicine",
issn = "0277-6715",
publisher = "John Wiley and Sons Ltd",
number = "1",

}

TY - JOUR

T1 - Classification accuracy of claims-based methods for identifying providers failing to meet performance targets

AU - Hubbard, Rebecca A.

AU - Benjamin-Johnson, Rhondee

AU - Onega, Tracy

AU - Smith-Bindman, Rebecca

AU - Zhu, Weiwei

AU - Fenton, Joshua J

PY - 2015/1/15

Y1 - 2015/1/15

N2 - Quality assessment is critical for healthcare reform, but data sources are lacking for measurement of many important healthcare outcomes. With over 49 million people covered by Medicare as of 2010, Medicare claims data offer a potentially valuable source that could be used in targeted health care quality improvement efforts. However, little is known about the operating characteristics of provider profiling methods using claims-based outcome measures that may estimate provider performance with error. Motivated by the example of screening mammography performance, we compared approaches to identifying providers failing to meet guideline targets using Medicare claims data. We used data from the Breast Cancer Surveillance Consortium and linked Medicare claims to compare claims-based and clinical estimates of cancer detection rate. We then demonstrated the performance of claim-based estimates across a broad range of operating characteristics using simulation studies. We found that identification of poor performing providers was extremely sensitive to algorithm specificity, with no approach identifying more than 65% of poor performing providers when claims-based measures had specificity of 0.995 or less. We conclude that claims have the potential to contribute important information on healthcare outcomes to quality improvement efforts. However, to achieve this potential, development of highly accurate claims-based outcome measures should remain a priority.

AB - Quality assessment is critical for healthcare reform, but data sources are lacking for measurement of many important healthcare outcomes. With over 49 million people covered by Medicare as of 2010, Medicare claims data offer a potentially valuable source that could be used in targeted health care quality improvement efforts. However, little is known about the operating characteristics of provider profiling methods using claims-based outcome measures that may estimate provider performance with error. Motivated by the example of screening mammography performance, we compared approaches to identifying providers failing to meet guideline targets using Medicare claims data. We used data from the Breast Cancer Surveillance Consortium and linked Medicare claims to compare claims-based and clinical estimates of cancer detection rate. We then demonstrated the performance of claim-based estimates across a broad range of operating characteristics using simulation studies. We found that identification of poor performing providers was extremely sensitive to algorithm specificity, with no approach identifying more than 65% of poor performing providers when claims-based measures had specificity of 0.995 or less. We conclude that claims have the potential to contribute important information on healthcare outcomes to quality improvement efforts. However, to achieve this potential, development of highly accurate claims-based outcome measures should remain a priority.

KW - Breast cancer

KW - Hierarchical models

KW - Medicare

KW - Provider profiling

UR - http://www.scopus.com/inward/record.url?scp=84915778509&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84915778509&partnerID=8YFLogxK

U2 - 10.1002/sim.6318

DO - 10.1002/sim.6318

M3 - Article

C2 - 25302935

AN - SCOPUS:84915778509

VL - 34

SP - 93

EP - 105

JO - Statistics in Medicine

JF - Statistics in Medicine

SN - 0277-6715

IS - 1

ER -