Addressing the challenge of assessing physician-level screening performance

Mammography as an example

Elizabeth S. Burnside, Yunzhi Lin, Alejandro Munoz Del Rio, Perry J. Pickhardt, Yirong Wu, Roberta M. Strigel, Mai A. Elezaby, Eve A. Kerr, Diana L Miglioretti

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

Background: Motivated by the challenges in assessing physician-level cancer screening performance and the negative impact of misclassification, we propose a method (using mammography as an example) that enables confident assertion of adequate or inadequate performance or alternatively recognizes when more data is required. Methods: Using established metrics for mammography screening performance-cancer detection rate (CDR) and recall rate (RR)-and observed benchmarks from the Breast Cancer Surveillance Consortium (BCSC), we calculate the minimum volume required to be 95% confident that a physician is performing at or above benchmark thresholds. We graphically display the minimum observed CDR and RR values required to confidently assert adequate performance over a range of interpretive volumes. We use a prospectively collected database of consecutive mammograms from a clinical screening program outside the BCSC to illustrate how this method classifies individual physician performance as volume accrues. Results: Our analysis reveals that an annual interpretive volume of 2770 screening mammograms, above the United States' (US) mandatory (480) and average (1777) annual volumes but below England's mandatory (5000) annual volume is necessary to confidently assert that a physician performed adequately. In our analyzed US practice, a single year of data uniformly allowed confident assertion of adequate performance in terms of RR but not CDR, which required aggregation of data across more than one year. Conclusion: For individual physician quality assessment in cancer screening programs that target low incidence populations, considering imprecision in observed performance metrics due to small numbers of patients with cancer is important.

Original languageEnglish (US)
Article numbere89418
JournalPLoS One
Volume9
Issue number2
DOIs
StatePublished - Feb 21 2014

Fingerprint

Mammography
physicians
Screening
screening
Physicians
neoplasms
Early Detection of Cancer
Benchmarking
breast neoplasms
Breast Neoplasms
Neoplasms
monitoring
England
Agglomeration
Databases
methodology
Incidence
incidence
Population

ASJC Scopus subject areas

  • Agricultural and Biological Sciences(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Medicine(all)

Cite this

Addressing the challenge of assessing physician-level screening performance : Mammography as an example. / Burnside, Elizabeth S.; Lin, Yunzhi; Munoz Del Rio, Alejandro; Pickhardt, Perry J.; Wu, Yirong; Strigel, Roberta M.; Elezaby, Mai A.; Kerr, Eve A.; Miglioretti, Diana L.

In: PLoS One, Vol. 9, No. 2, e89418, 21.02.2014.

Research output: Contribution to journalArticle

Burnside, ES, Lin, Y, Munoz Del Rio, A, Pickhardt, PJ, Wu, Y, Strigel, RM, Elezaby, MA, Kerr, EA & Miglioretti, DL 2014, 'Addressing the challenge of assessing physician-level screening performance: Mammography as an example', PLoS One, vol. 9, no. 2, e89418. https://doi.org/10.1371/journal.pone.0089418
Burnside ES, Lin Y, Munoz Del Rio A, Pickhardt PJ, Wu Y, Strigel RM et al. Addressing the challenge of assessing physician-level screening performance: Mammography as an example. PLoS One. 2014 Feb 21;9(2). e89418. https://doi.org/10.1371/journal.pone.0089418
Burnside, Elizabeth S. ; Lin, Yunzhi ; Munoz Del Rio, Alejandro ; Pickhardt, Perry J. ; Wu, Yirong ; Strigel, Roberta M. ; Elezaby, Mai A. ; Kerr, Eve A. ; Miglioretti, Diana L. / Addressing the challenge of assessing physician-level screening performance : Mammography as an example. In: PLoS One. 2014 ; Vol. 9, No. 2.
@article{0e2bc8111c95495081265d79dbead2ab,
title = "Addressing the challenge of assessing physician-level screening performance: Mammography as an example",
abstract = "Background: Motivated by the challenges in assessing physician-level cancer screening performance and the negative impact of misclassification, we propose a method (using mammography as an example) that enables confident assertion of adequate or inadequate performance or alternatively recognizes when more data is required. Methods: Using established metrics for mammography screening performance-cancer detection rate (CDR) and recall rate (RR)-and observed benchmarks from the Breast Cancer Surveillance Consortium (BCSC), we calculate the minimum volume required to be 95{\%} confident that a physician is performing at or above benchmark thresholds. We graphically display the minimum observed CDR and RR values required to confidently assert adequate performance over a range of interpretive volumes. We use a prospectively collected database of consecutive mammograms from a clinical screening program outside the BCSC to illustrate how this method classifies individual physician performance as volume accrues. Results: Our analysis reveals that an annual interpretive volume of 2770 screening mammograms, above the United States' (US) mandatory (480) and average (1777) annual volumes but below England's mandatory (5000) annual volume is necessary to confidently assert that a physician performed adequately. In our analyzed US practice, a single year of data uniformly allowed confident assertion of adequate performance in terms of RR but not CDR, which required aggregation of data across more than one year. Conclusion: For individual physician quality assessment in cancer screening programs that target low incidence populations, considering imprecision in observed performance metrics due to small numbers of patients with cancer is important.",
author = "Burnside, {Elizabeth S.} and Yunzhi Lin and {Munoz Del Rio}, Alejandro and Pickhardt, {Perry J.} and Yirong Wu and Strigel, {Roberta M.} and Elezaby, {Mai A.} and Kerr, {Eve A.} and Miglioretti, {Diana L}",
year = "2014",
month = "2",
day = "21",
doi = "10.1371/journal.pone.0089418",
language = "English (US)",
volume = "9",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "2",

}

TY - JOUR

T1 - Addressing the challenge of assessing physician-level screening performance

T2 - Mammography as an example

AU - Burnside, Elizabeth S.

AU - Lin, Yunzhi

AU - Munoz Del Rio, Alejandro

AU - Pickhardt, Perry J.

AU - Wu, Yirong

AU - Strigel, Roberta M.

AU - Elezaby, Mai A.

AU - Kerr, Eve A.

AU - Miglioretti, Diana L

PY - 2014/2/21

Y1 - 2014/2/21

N2 - Background: Motivated by the challenges in assessing physician-level cancer screening performance and the negative impact of misclassification, we propose a method (using mammography as an example) that enables confident assertion of adequate or inadequate performance or alternatively recognizes when more data is required. Methods: Using established metrics for mammography screening performance-cancer detection rate (CDR) and recall rate (RR)-and observed benchmarks from the Breast Cancer Surveillance Consortium (BCSC), we calculate the minimum volume required to be 95% confident that a physician is performing at or above benchmark thresholds. We graphically display the minimum observed CDR and RR values required to confidently assert adequate performance over a range of interpretive volumes. We use a prospectively collected database of consecutive mammograms from a clinical screening program outside the BCSC to illustrate how this method classifies individual physician performance as volume accrues. Results: Our analysis reveals that an annual interpretive volume of 2770 screening mammograms, above the United States' (US) mandatory (480) and average (1777) annual volumes but below England's mandatory (5000) annual volume is necessary to confidently assert that a physician performed adequately. In our analyzed US practice, a single year of data uniformly allowed confident assertion of adequate performance in terms of RR but not CDR, which required aggregation of data across more than one year. Conclusion: For individual physician quality assessment in cancer screening programs that target low incidence populations, considering imprecision in observed performance metrics due to small numbers of patients with cancer is important.

AB - Background: Motivated by the challenges in assessing physician-level cancer screening performance and the negative impact of misclassification, we propose a method (using mammography as an example) that enables confident assertion of adequate or inadequate performance or alternatively recognizes when more data is required. Methods: Using established metrics for mammography screening performance-cancer detection rate (CDR) and recall rate (RR)-and observed benchmarks from the Breast Cancer Surveillance Consortium (BCSC), we calculate the minimum volume required to be 95% confident that a physician is performing at or above benchmark thresholds. We graphically display the minimum observed CDR and RR values required to confidently assert adequate performance over a range of interpretive volumes. We use a prospectively collected database of consecutive mammograms from a clinical screening program outside the BCSC to illustrate how this method classifies individual physician performance as volume accrues. Results: Our analysis reveals that an annual interpretive volume of 2770 screening mammograms, above the United States' (US) mandatory (480) and average (1777) annual volumes but below England's mandatory (5000) annual volume is necessary to confidently assert that a physician performed adequately. In our analyzed US practice, a single year of data uniformly allowed confident assertion of adequate performance in terms of RR but not CDR, which required aggregation of data across more than one year. Conclusion: For individual physician quality assessment in cancer screening programs that target low incidence populations, considering imprecision in observed performance metrics due to small numbers of patients with cancer is important.

UR - http://www.scopus.com/inward/record.url?scp=84896745606&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84896745606&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0089418

DO - 10.1371/journal.pone.0089418

M3 - Article

VL - 9

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 2

M1 - e89418

ER -