National performance benchmarks for modern screening digital mammography: Update from the Breast Cancer Surveillance Consortium

Constance D. Lehman, Robert F. Arao, Brian L. Sprague, Janie M. Lee, Diana S M Buist, Karla Kerlikowske, Louise M. Henderson, Tracy Onega, Anna N A Tosteson, Garth H. Rauscher, Diana L Miglioretti

Research output: Contribution to journalArticle

82 Citations (Scopus)

Abstract

Purpose: To establish performance benchmarks for modern screening digital mammography and assess performance trends over time in U.S. community practice. Materials and Methods: This HIPAA-compliant, institutional review board-approved study measured the performance of digital screening mammography interpreted by 359 radiologists across 95 facilities in six Breast Cancer Surveillance Consortium (BCSC) registries. The study included 1 682 504 digital screening mammograms performed between 2007 and 2013 in 792 808 women. Performance measures were calculated according to the American College of Radiology Breast Imaging Reporting and Data System, 5th edition, and were compared with published benchmarks by the BCSC, the National Mammography Database, and performance recommendations by expert opinion. Benchmarks were derived from the distribution of performance metrics across radiologists and were presented as 50th (median), 10th, 25th, 75th, and 90th percentiles, with graphic presentations using smoothed curves. Results: Mean screening performance measures were as follows: abnormal interpretation rate (AIR), 11.6 (95% confidence interval [CI]: 11.5, 11.6); cancers detected per 1000 screens, or cancer detection rate (CDR), 5.1 (95% CI: 5.0, 5.2); sensitivity, 86.9% (95% CI: 86.3%, 87.6%); specificity, 88.9% (95% CI: 88.8%, 88.9%); false-negative rate per 1000 screens, 0.8 (95% CI: 0.7, 0.8); positive predictive value (PPV) 1, 4.4% (95% CI: 4.3%, 4.5%); PPV2, 25.6% (95% CI: 25.1%, 26.1%); PPV3, 28.6% (95% CI: 28.0%, 29.3%); cancers stage 0 or 1, 76.9%; minimal cancers, 57.7%; and node-negative invasive cancers, 79.4%. Recommended CDRs were achieved by 92.1% of radiologists in community practice, and 97.1% achieved recommended ranges for sensitivity. Only 59.0% of radiologists achieved recommended AIRs, and only 63.0% achieved recommended levels of specificity. Conclusion: The majority of radiologists in the BCSC surpass cancer detection recommendations for screening mammography; however, AIRs continue to be higher than the recommended rate for almost half of radiologists interpreting screening mammograms.

Original languageEnglish (US)
Pages (from-to)49-58
Number of pages10
JournalRadiology
Volume283
Issue number1
DOIs
StatePublished - Apr 1 2017

Fingerprint

Benchmarking
Mammography
Confidence Intervals
Breast Neoplasms
Neoplasms
Health Insurance Portability and Accountability Act
Research Ethics Committees
Expert Testimony
Radiology
Information Systems
Registries
Radiologists
Breast
Databases

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

National performance benchmarks for modern screening digital mammography : Update from the Breast Cancer Surveillance Consortium. / Lehman, Constance D.; Arao, Robert F.; Sprague, Brian L.; Lee, Janie M.; Buist, Diana S M; Kerlikowske, Karla; Henderson, Louise M.; Onega, Tracy; Tosteson, Anna N A; Rauscher, Garth H.; Miglioretti, Diana L.

In: Radiology, Vol. 283, No. 1, 01.04.2017, p. 49-58.

Research output: Contribution to journalArticle

Lehman, CD, Arao, RF, Sprague, BL, Lee, JM, Buist, DSM, Kerlikowske, K, Henderson, LM, Onega, T, Tosteson, ANA, Rauscher, GH & Miglioretti, DL 2017, 'National performance benchmarks for modern screening digital mammography: Update from the Breast Cancer Surveillance Consortium', Radiology, vol. 283, no. 1, pp. 49-58. https://doi.org/10.1148/radiol.2016161174
Lehman, Constance D. ; Arao, Robert F. ; Sprague, Brian L. ; Lee, Janie M. ; Buist, Diana S M ; Kerlikowske, Karla ; Henderson, Louise M. ; Onega, Tracy ; Tosteson, Anna N A ; Rauscher, Garth H. ; Miglioretti, Diana L. / National performance benchmarks for modern screening digital mammography : Update from the Breast Cancer Surveillance Consortium. In: Radiology. 2017 ; Vol. 283, No. 1. pp. 49-58.
@article{1e319911e4334f25bd81e73b444288c4,
title = "National performance benchmarks for modern screening digital mammography: Update from the Breast Cancer Surveillance Consortium",
abstract = "Purpose: To establish performance benchmarks for modern screening digital mammography and assess performance trends over time in U.S. community practice. Materials and Methods: This HIPAA-compliant, institutional review board-approved study measured the performance of digital screening mammography interpreted by 359 radiologists across 95 facilities in six Breast Cancer Surveillance Consortium (BCSC) registries. The study included 1 682 504 digital screening mammograms performed between 2007 and 2013 in 792 808 women. Performance measures were calculated according to the American College of Radiology Breast Imaging Reporting and Data System, 5th edition, and were compared with published benchmarks by the BCSC, the National Mammography Database, and performance recommendations by expert opinion. Benchmarks were derived from the distribution of performance metrics across radiologists and were presented as 50th (median), 10th, 25th, 75th, and 90th percentiles, with graphic presentations using smoothed curves. Results: Mean screening performance measures were as follows: abnormal interpretation rate (AIR), 11.6 (95{\%} confidence interval [CI]: 11.5, 11.6); cancers detected per 1000 screens, or cancer detection rate (CDR), 5.1 (95{\%} CI: 5.0, 5.2); sensitivity, 86.9{\%} (95{\%} CI: 86.3{\%}, 87.6{\%}); specificity, 88.9{\%} (95{\%} CI: 88.8{\%}, 88.9{\%}); false-negative rate per 1000 screens, 0.8 (95{\%} CI: 0.7, 0.8); positive predictive value (PPV) 1, 4.4{\%} (95{\%} CI: 4.3{\%}, 4.5{\%}); PPV2, 25.6{\%} (95{\%} CI: 25.1{\%}, 26.1{\%}); PPV3, 28.6{\%} (95{\%} CI: 28.0{\%}, 29.3{\%}); cancers stage 0 or 1, 76.9{\%}; minimal cancers, 57.7{\%}; and node-negative invasive cancers, 79.4{\%}. Recommended CDRs were achieved by 92.1{\%} of radiologists in community practice, and 97.1{\%} achieved recommended ranges for sensitivity. Only 59.0{\%} of radiologists achieved recommended AIRs, and only 63.0{\%} achieved recommended levels of specificity. Conclusion: The majority of radiologists in the BCSC surpass cancer detection recommendations for screening mammography; however, AIRs continue to be higher than the recommended rate for almost half of radiologists interpreting screening mammograms.",
author = "Lehman, {Constance D.} and Arao, {Robert F.} and Sprague, {Brian L.} and Lee, {Janie M.} and Buist, {Diana S M} and Karla Kerlikowske and Henderson, {Louise M.} and Tracy Onega and Tosteson, {Anna N A} and Rauscher, {Garth H.} and Miglioretti, {Diana L}",
year = "2017",
month = "4",
day = "1",
doi = "10.1148/radiol.2016161174",
language = "English (US)",
volume = "283",
pages = "49--58",
journal = "Radiology",
issn = "0033-8419",
publisher = "Radiological Society of North America Inc.",
number = "1",

}

TY - JOUR

T1 - National performance benchmarks for modern screening digital mammography

T2 - Update from the Breast Cancer Surveillance Consortium

AU - Lehman, Constance D.

AU - Arao, Robert F.

AU - Sprague, Brian L.

AU - Lee, Janie M.

AU - Buist, Diana S M

AU - Kerlikowske, Karla

AU - Henderson, Louise M.

AU - Onega, Tracy

AU - Tosteson, Anna N A

AU - Rauscher, Garth H.

AU - Miglioretti, Diana L

PY - 2017/4/1

Y1 - 2017/4/1

N2 - Purpose: To establish performance benchmarks for modern screening digital mammography and assess performance trends over time in U.S. community practice. Materials and Methods: This HIPAA-compliant, institutional review board-approved study measured the performance of digital screening mammography interpreted by 359 radiologists across 95 facilities in six Breast Cancer Surveillance Consortium (BCSC) registries. The study included 1 682 504 digital screening mammograms performed between 2007 and 2013 in 792 808 women. Performance measures were calculated according to the American College of Radiology Breast Imaging Reporting and Data System, 5th edition, and were compared with published benchmarks by the BCSC, the National Mammography Database, and performance recommendations by expert opinion. Benchmarks were derived from the distribution of performance metrics across radiologists and were presented as 50th (median), 10th, 25th, 75th, and 90th percentiles, with graphic presentations using smoothed curves. Results: Mean screening performance measures were as follows: abnormal interpretation rate (AIR), 11.6 (95% confidence interval [CI]: 11.5, 11.6); cancers detected per 1000 screens, or cancer detection rate (CDR), 5.1 (95% CI: 5.0, 5.2); sensitivity, 86.9% (95% CI: 86.3%, 87.6%); specificity, 88.9% (95% CI: 88.8%, 88.9%); false-negative rate per 1000 screens, 0.8 (95% CI: 0.7, 0.8); positive predictive value (PPV) 1, 4.4% (95% CI: 4.3%, 4.5%); PPV2, 25.6% (95% CI: 25.1%, 26.1%); PPV3, 28.6% (95% CI: 28.0%, 29.3%); cancers stage 0 or 1, 76.9%; minimal cancers, 57.7%; and node-negative invasive cancers, 79.4%. Recommended CDRs were achieved by 92.1% of radiologists in community practice, and 97.1% achieved recommended ranges for sensitivity. Only 59.0% of radiologists achieved recommended AIRs, and only 63.0% achieved recommended levels of specificity. Conclusion: The majority of radiologists in the BCSC surpass cancer detection recommendations for screening mammography; however, AIRs continue to be higher than the recommended rate for almost half of radiologists interpreting screening mammograms.

AB - Purpose: To establish performance benchmarks for modern screening digital mammography and assess performance trends over time in U.S. community practice. Materials and Methods: This HIPAA-compliant, institutional review board-approved study measured the performance of digital screening mammography interpreted by 359 radiologists across 95 facilities in six Breast Cancer Surveillance Consortium (BCSC) registries. The study included 1 682 504 digital screening mammograms performed between 2007 and 2013 in 792 808 women. Performance measures were calculated according to the American College of Radiology Breast Imaging Reporting and Data System, 5th edition, and were compared with published benchmarks by the BCSC, the National Mammography Database, and performance recommendations by expert opinion. Benchmarks were derived from the distribution of performance metrics across radiologists and were presented as 50th (median), 10th, 25th, 75th, and 90th percentiles, with graphic presentations using smoothed curves. Results: Mean screening performance measures were as follows: abnormal interpretation rate (AIR), 11.6 (95% confidence interval [CI]: 11.5, 11.6); cancers detected per 1000 screens, or cancer detection rate (CDR), 5.1 (95% CI: 5.0, 5.2); sensitivity, 86.9% (95% CI: 86.3%, 87.6%); specificity, 88.9% (95% CI: 88.8%, 88.9%); false-negative rate per 1000 screens, 0.8 (95% CI: 0.7, 0.8); positive predictive value (PPV) 1, 4.4% (95% CI: 4.3%, 4.5%); PPV2, 25.6% (95% CI: 25.1%, 26.1%); PPV3, 28.6% (95% CI: 28.0%, 29.3%); cancers stage 0 or 1, 76.9%; minimal cancers, 57.7%; and node-negative invasive cancers, 79.4%. Recommended CDRs were achieved by 92.1% of radiologists in community practice, and 97.1% achieved recommended ranges for sensitivity. Only 59.0% of radiologists achieved recommended AIRs, and only 63.0% achieved recommended levels of specificity. Conclusion: The majority of radiologists in the BCSC surpass cancer detection recommendations for screening mammography; however, AIRs continue to be higher than the recommended rate for almost half of radiologists interpreting screening mammograms.

UR - http://www.scopus.com/inward/record.url?scp=85017152777&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85017152777&partnerID=8YFLogxK

U2 - 10.1148/radiol.2016161174

DO - 10.1148/radiol.2016161174

M3 - Article

C2 - 27918707

AN - SCOPUS:85017152777

VL - 283

SP - 49

EP - 58

JO - Radiology

JF - Radiology

SN - 0033-8419

IS - 1

ER -