Inter-reader Variability in the Use of BI-RADS Descriptors for Suspicious Findings on Diagnostic Mammography: A Multi-institution Study of 10 Academic Radiologists

Amie Y. Lee, Dorota J. Wisner, Shadi Aminololama-Shakeri, Vignesh A. Arasu, Stephen A. Feig, Jonathan B Hargreaves, Haydee Ojeda-Fournier, Lawrence W. Bassett, Colin J. Wells, Jade De Guzman, Chris I. Flowers, Joan E. Campbell, Sarah L. Elson, Hanna Retallack, Bonnie N. Joe

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Rationale and Objectives The study aimed to determine the inter-observer agreement among academic breast radiologists when using the Breast Imaging Reporting and Data System (BI-RADS) lesion descriptors for suspicious findings on diagnostic mammography. Materials and Methods Ten experienced academic breast radiologists across five medical centers independently reviewed 250 de-identified diagnostic mammographic cases that were previously assessed as BI-RADS 4 or 5 with subsequent pathologic diagnosis by percutaneous or surgical biopsy. Each radiologist assessed the presence of the following suspicious mammographic findings: mass, asymmetry (one view), focal asymmetry (two views), architectural distortion, and calcifications. For any identified calcifications, the radiologist also described the morphology and distribution. Inter-observer agreement was determined with Fleiss kappa statistic. Agreement was also calculated by years of experience. Results Of the 250 lesions, 156 (62%) were benign and 94 (38%) were malignant. Agreement among the 10 readers was strongest for recognizing the presence of calcifications (k = 0.82). There was substantial agreement among the readers for the identification of a mass (k = 0.67), whereas agreement was fair for the presence of a focal asymmetry (k = 0.21) or architectural distortion (k = 0.28). Agreement for asymmetries (one view) was slight (k = 0.09). Among the categories of calcification morphology and distribution, reader agreement was moderate (k = 0.51 and k = 0.60, respectively). Readers with more experience (10 or more years in clinical practice) did not demonstrate higher levels of agreement compared to those with less experience. Conclusions Strength of agreement varies widely for different types of mammographic findings, even among dedicated academic breast radiologists. More subtle findings such as asymmetries and architectural distortion demonstrated the weakest agreement. Studies that seek to evaluate the predictive value of certain mammographic features for malignancy should take into consideration the inherent interpretive variability for these findings.

Original languageEnglish (US)
Pages (from-to)60-66
Number of pages7
JournalAcademic Radiology
Volume24
Issue number1
DOIs
StatePublished - Jan 1 2017

Fingerprint

Mammography
Information Systems
Breast
Radiologists
Biopsy
Neoplasms

Keywords

  • BI-RADS
  • Breast Imaging
  • Mammography

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Inter-reader Variability in the Use of BI-RADS Descriptors for Suspicious Findings on Diagnostic Mammography : A Multi-institution Study of 10 Academic Radiologists. / Lee, Amie Y.; Wisner, Dorota J.; Aminololama-Shakeri, Shadi; Arasu, Vignesh A.; Feig, Stephen A.; Hargreaves, Jonathan B; Ojeda-Fournier, Haydee; Bassett, Lawrence W.; Wells, Colin J.; De Guzman, Jade; Flowers, Chris I.; Campbell, Joan E.; Elson, Sarah L.; Retallack, Hanna; Joe, Bonnie N.

In: Academic Radiology, Vol. 24, No. 1, 01.01.2017, p. 60-66.

Research output: Contribution to journalArticle

Lee, AY, Wisner, DJ, Aminololama-Shakeri, S, Arasu, VA, Feig, SA, Hargreaves, JB, Ojeda-Fournier, H, Bassett, LW, Wells, CJ, De Guzman, J, Flowers, CI, Campbell, JE, Elson, SL, Retallack, H & Joe, BN 2017, 'Inter-reader Variability in the Use of BI-RADS Descriptors for Suspicious Findings on Diagnostic Mammography: A Multi-institution Study of 10 Academic Radiologists', Academic Radiology, vol. 24, no. 1, pp. 60-66. https://doi.org/10.1016/j.acra.2016.09.010
Lee, Amie Y. ; Wisner, Dorota J. ; Aminololama-Shakeri, Shadi ; Arasu, Vignesh A. ; Feig, Stephen A. ; Hargreaves, Jonathan B ; Ojeda-Fournier, Haydee ; Bassett, Lawrence W. ; Wells, Colin J. ; De Guzman, Jade ; Flowers, Chris I. ; Campbell, Joan E. ; Elson, Sarah L. ; Retallack, Hanna ; Joe, Bonnie N. / Inter-reader Variability in the Use of BI-RADS Descriptors for Suspicious Findings on Diagnostic Mammography : A Multi-institution Study of 10 Academic Radiologists. In: Academic Radiology. 2017 ; Vol. 24, No. 1. pp. 60-66.
@article{cafcda05d5d54dba88c332da223b5200,
title = "Inter-reader Variability in the Use of BI-RADS Descriptors for Suspicious Findings on Diagnostic Mammography: A Multi-institution Study of 10 Academic Radiologists",
abstract = "Rationale and Objectives The study aimed to determine the inter-observer agreement among academic breast radiologists when using the Breast Imaging Reporting and Data System (BI-RADS) lesion descriptors for suspicious findings on diagnostic mammography. Materials and Methods Ten experienced academic breast radiologists across five medical centers independently reviewed 250 de-identified diagnostic mammographic cases that were previously assessed as BI-RADS 4 or 5 with subsequent pathologic diagnosis by percutaneous or surgical biopsy. Each radiologist assessed the presence of the following suspicious mammographic findings: mass, asymmetry (one view), focal asymmetry (two views), architectural distortion, and calcifications. For any identified calcifications, the radiologist also described the morphology and distribution. Inter-observer agreement was determined with Fleiss kappa statistic. Agreement was also calculated by years of experience. Results Of the 250 lesions, 156 (62{\%}) were benign and 94 (38{\%}) were malignant. Agreement among the 10 readers was strongest for recognizing the presence of calcifications (k = 0.82). There was substantial agreement among the readers for the identification of a mass (k = 0.67), whereas agreement was fair for the presence of a focal asymmetry (k = 0.21) or architectural distortion (k = 0.28). Agreement for asymmetries (one view) was slight (k = 0.09). Among the categories of calcification morphology and distribution, reader agreement was moderate (k = 0.51 and k = 0.60, respectively). Readers with more experience (10 or more years in clinical practice) did not demonstrate higher levels of agreement compared to those with less experience. Conclusions Strength of agreement varies widely for different types of mammographic findings, even among dedicated academic breast radiologists. More subtle findings such as asymmetries and architectural distortion demonstrated the weakest agreement. Studies that seek to evaluate the predictive value of certain mammographic features for malignancy should take into consideration the inherent interpretive variability for these findings.",
keywords = "BI-RADS, Breast Imaging, Mammography",
author = "Lee, {Amie Y.} and Wisner, {Dorota J.} and Shadi Aminololama-Shakeri and Arasu, {Vignesh A.} and Feig, {Stephen A.} and Hargreaves, {Jonathan B} and Haydee Ojeda-Fournier and Bassett, {Lawrence W.} and Wells, {Colin J.} and {De Guzman}, Jade and Flowers, {Chris I.} and Campbell, {Joan E.} and Elson, {Sarah L.} and Hanna Retallack and Joe, {Bonnie N.}",
year = "2017",
month = "1",
day = "1",
doi = "10.1016/j.acra.2016.09.010",
language = "English (US)",
volume = "24",
pages = "60--66",
journal = "Academic Radiology",
issn = "1076-6332",
publisher = "Elsevier USA",
number = "1",

}

TY - JOUR

T1 - Inter-reader Variability in the Use of BI-RADS Descriptors for Suspicious Findings on Diagnostic Mammography

T2 - A Multi-institution Study of 10 Academic Radiologists

AU - Lee, Amie Y.

AU - Wisner, Dorota J.

AU - Aminololama-Shakeri, Shadi

AU - Arasu, Vignesh A.

AU - Feig, Stephen A.

AU - Hargreaves, Jonathan B

AU - Ojeda-Fournier, Haydee

AU - Bassett, Lawrence W.

AU - Wells, Colin J.

AU - De Guzman, Jade

AU - Flowers, Chris I.

AU - Campbell, Joan E.

AU - Elson, Sarah L.

AU - Retallack, Hanna

AU - Joe, Bonnie N.

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Rationale and Objectives The study aimed to determine the inter-observer agreement among academic breast radiologists when using the Breast Imaging Reporting and Data System (BI-RADS) lesion descriptors for suspicious findings on diagnostic mammography. Materials and Methods Ten experienced academic breast radiologists across five medical centers independently reviewed 250 de-identified diagnostic mammographic cases that were previously assessed as BI-RADS 4 or 5 with subsequent pathologic diagnosis by percutaneous or surgical biopsy. Each radiologist assessed the presence of the following suspicious mammographic findings: mass, asymmetry (one view), focal asymmetry (two views), architectural distortion, and calcifications. For any identified calcifications, the radiologist also described the morphology and distribution. Inter-observer agreement was determined with Fleiss kappa statistic. Agreement was also calculated by years of experience. Results Of the 250 lesions, 156 (62%) were benign and 94 (38%) were malignant. Agreement among the 10 readers was strongest for recognizing the presence of calcifications (k = 0.82). There was substantial agreement among the readers for the identification of a mass (k = 0.67), whereas agreement was fair for the presence of a focal asymmetry (k = 0.21) or architectural distortion (k = 0.28). Agreement for asymmetries (one view) was slight (k = 0.09). Among the categories of calcification morphology and distribution, reader agreement was moderate (k = 0.51 and k = 0.60, respectively). Readers with more experience (10 or more years in clinical practice) did not demonstrate higher levels of agreement compared to those with less experience. Conclusions Strength of agreement varies widely for different types of mammographic findings, even among dedicated academic breast radiologists. More subtle findings such as asymmetries and architectural distortion demonstrated the weakest agreement. Studies that seek to evaluate the predictive value of certain mammographic features for malignancy should take into consideration the inherent interpretive variability for these findings.

AB - Rationale and Objectives The study aimed to determine the inter-observer agreement among academic breast radiologists when using the Breast Imaging Reporting and Data System (BI-RADS) lesion descriptors for suspicious findings on diagnostic mammography. Materials and Methods Ten experienced academic breast radiologists across five medical centers independently reviewed 250 de-identified diagnostic mammographic cases that were previously assessed as BI-RADS 4 or 5 with subsequent pathologic diagnosis by percutaneous or surgical biopsy. Each radiologist assessed the presence of the following suspicious mammographic findings: mass, asymmetry (one view), focal asymmetry (two views), architectural distortion, and calcifications. For any identified calcifications, the radiologist also described the morphology and distribution. Inter-observer agreement was determined with Fleiss kappa statistic. Agreement was also calculated by years of experience. Results Of the 250 lesions, 156 (62%) were benign and 94 (38%) were malignant. Agreement among the 10 readers was strongest for recognizing the presence of calcifications (k = 0.82). There was substantial agreement among the readers for the identification of a mass (k = 0.67), whereas agreement was fair for the presence of a focal asymmetry (k = 0.21) or architectural distortion (k = 0.28). Agreement for asymmetries (one view) was slight (k = 0.09). Among the categories of calcification morphology and distribution, reader agreement was moderate (k = 0.51 and k = 0.60, respectively). Readers with more experience (10 or more years in clinical practice) did not demonstrate higher levels of agreement compared to those with less experience. Conclusions Strength of agreement varies widely for different types of mammographic findings, even among dedicated academic breast radiologists. More subtle findings such as asymmetries and architectural distortion demonstrated the weakest agreement. Studies that seek to evaluate the predictive value of certain mammographic features for malignancy should take into consideration the inherent interpretive variability for these findings.

KW - BI-RADS

KW - Breast Imaging

KW - Mammography

UR - http://www.scopus.com/inward/record.url?scp=85000538827&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85000538827&partnerID=8YFLogxK

U2 - 10.1016/j.acra.2016.09.010

DO - 10.1016/j.acra.2016.09.010

M3 - Article

C2 - 27793579

AN - SCOPUS:85000538827

VL - 24

SP - 60

EP - 66

JO - Academic Radiology

JF - Academic Radiology

SN - 1076-6332

IS - 1

ER -