Is confidence of mammographic assessment a good predictor of accuracy?

Berta M. Geller, Andy Bogart, Patricia A. Carney, Joann G. Elmore, Barbara S. Monsees, Diana L Miglioretti

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

OBJECTIVE. Interpretive accuracy varies among radiologists, especially in mammography. This study examines the relationship between radiologists' confidence in their assessments and their accuracy in interpreting mammograms. MATERIALS AND METHODS. In this study, 119 community radiologists interpreted 109 expert-defined screening mammography examinations in test sets and rated their confidence in their assessment for each case. They also provided a global assessment of their ability to interpret mammograms. Positive predictive value (PPV) and negative predictive value (NPV) were modeled as functions of self-rated confidence on each examination using log-linear regression estimated with generalized estimating equations. Reference measures were cancer status and expert-defined need for recall. Effect modification by weekly mammography volume was examined. RESULTS. Radiologists who self-reported higher global interpretive ability tended to interpret more mammograms per week (p = 0.08), were more likely to specialize (p = 0.02) and to have completed a fellowship in breast or women's imaging (p = 0.05), and had a higher PPV for cancer detection (p = 0.01). Examinations for which low-volume radiologists were "very confident" had a PPV of 2.93 times (95% CI, 2.01-4.27) higher than examinations they rated with neutral confidence. Trends of increasing NPVs with increasing confidence were significant for low-volume radiologists relative to noncancers (p = 0.01) and expert nonrecalls (p < 0.001). A trend of significantly increasing NPVs existed for high-volume radiologists relative to expert nonrecall (p = 0.02) but not relative to noncancer status (p = 0.32). CONCLUSION. Confidence in mammography assessments was associated with better accuracy, especially for low-volume readers. Asking for a second opinion when confidence in an assessment is low may increase accuracy.

Original languageEnglish (US)
JournalAmerican Journal of Roentgenology
Volume199
Issue number1
DOIs
StatePublished - Jul 2012
Externally publishedYes

Fingerprint

Mammography
Radiologists
Linear Models
Neoplasms
Breast
Referral and Consultation

Keywords

  • Accuracy
  • Confidence
  • Mammography
  • Radiologist performance
  • Screening

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Is confidence of mammographic assessment a good predictor of accuracy? / Geller, Berta M.; Bogart, Andy; Carney, Patricia A.; Elmore, Joann G.; Monsees, Barbara S.; Miglioretti, Diana L.

In: American Journal of Roentgenology, Vol. 199, No. 1, 07.2012.

Research output: Contribution to journalArticle

Geller, Berta M. ; Bogart, Andy ; Carney, Patricia A. ; Elmore, Joann G. ; Monsees, Barbara S. ; Miglioretti, Diana L. / Is confidence of mammographic assessment a good predictor of accuracy?. In: American Journal of Roentgenology. 2012 ; Vol. 199, No. 1.
@article{429bb778881346629e8cdab33bdfad7b,
title = "Is confidence of mammographic assessment a good predictor of accuracy?",
abstract = "OBJECTIVE. Interpretive accuracy varies among radiologists, especially in mammography. This study examines the relationship between radiologists' confidence in their assessments and their accuracy in interpreting mammograms. MATERIALS AND METHODS. In this study, 119 community radiologists interpreted 109 expert-defined screening mammography examinations in test sets and rated their confidence in their assessment for each case. They also provided a global assessment of their ability to interpret mammograms. Positive predictive value (PPV) and negative predictive value (NPV) were modeled as functions of self-rated confidence on each examination using log-linear regression estimated with generalized estimating equations. Reference measures were cancer status and expert-defined need for recall. Effect modification by weekly mammography volume was examined. RESULTS. Radiologists who self-reported higher global interpretive ability tended to interpret more mammograms per week (p = 0.08), were more likely to specialize (p = 0.02) and to have completed a fellowship in breast or women's imaging (p = 0.05), and had a higher PPV for cancer detection (p = 0.01). Examinations for which low-volume radiologists were {"}very confident{"} had a PPV of 2.93 times (95{\%} CI, 2.01-4.27) higher than examinations they rated with neutral confidence. Trends of increasing NPVs with increasing confidence were significant for low-volume radiologists relative to noncancers (p = 0.01) and expert nonrecalls (p < 0.001). A trend of significantly increasing NPVs existed for high-volume radiologists relative to expert nonrecall (p = 0.02) but not relative to noncancer status (p = 0.32). CONCLUSION. Confidence in mammography assessments was associated with better accuracy, especially for low-volume readers. Asking for a second opinion when confidence in an assessment is low may increase accuracy.",
keywords = "Accuracy, Confidence, Mammography, Radiologist performance, Screening",
author = "Geller, {Berta M.} and Andy Bogart and Carney, {Patricia A.} and Elmore, {Joann G.} and Monsees, {Barbara S.} and Miglioretti, {Diana L}",
year = "2012",
month = "7",
doi = "10.2214/AJR.11.7701",
language = "English (US)",
volume = "199",
journal = "American Journal of Roentgenology",
issn = "0361-803X",
publisher = "American Roentgen Ray Society",
number = "1",

}

TY - JOUR

T1 - Is confidence of mammographic assessment a good predictor of accuracy?

AU - Geller, Berta M.

AU - Bogart, Andy

AU - Carney, Patricia A.

AU - Elmore, Joann G.

AU - Monsees, Barbara S.

AU - Miglioretti, Diana L

PY - 2012/7

Y1 - 2012/7

N2 - OBJECTIVE. Interpretive accuracy varies among radiologists, especially in mammography. This study examines the relationship between radiologists' confidence in their assessments and their accuracy in interpreting mammograms. MATERIALS AND METHODS. In this study, 119 community radiologists interpreted 109 expert-defined screening mammography examinations in test sets and rated their confidence in their assessment for each case. They also provided a global assessment of their ability to interpret mammograms. Positive predictive value (PPV) and negative predictive value (NPV) were modeled as functions of self-rated confidence on each examination using log-linear regression estimated with generalized estimating equations. Reference measures were cancer status and expert-defined need for recall. Effect modification by weekly mammography volume was examined. RESULTS. Radiologists who self-reported higher global interpretive ability tended to interpret more mammograms per week (p = 0.08), were more likely to specialize (p = 0.02) and to have completed a fellowship in breast or women's imaging (p = 0.05), and had a higher PPV for cancer detection (p = 0.01). Examinations for which low-volume radiologists were "very confident" had a PPV of 2.93 times (95% CI, 2.01-4.27) higher than examinations they rated with neutral confidence. Trends of increasing NPVs with increasing confidence were significant for low-volume radiologists relative to noncancers (p = 0.01) and expert nonrecalls (p < 0.001). A trend of significantly increasing NPVs existed for high-volume radiologists relative to expert nonrecall (p = 0.02) but not relative to noncancer status (p = 0.32). CONCLUSION. Confidence in mammography assessments was associated with better accuracy, especially for low-volume readers. Asking for a second opinion when confidence in an assessment is low may increase accuracy.

AB - OBJECTIVE. Interpretive accuracy varies among radiologists, especially in mammography. This study examines the relationship between radiologists' confidence in their assessments and their accuracy in interpreting mammograms. MATERIALS AND METHODS. In this study, 119 community radiologists interpreted 109 expert-defined screening mammography examinations in test sets and rated their confidence in their assessment for each case. They also provided a global assessment of their ability to interpret mammograms. Positive predictive value (PPV) and negative predictive value (NPV) were modeled as functions of self-rated confidence on each examination using log-linear regression estimated with generalized estimating equations. Reference measures were cancer status and expert-defined need for recall. Effect modification by weekly mammography volume was examined. RESULTS. Radiologists who self-reported higher global interpretive ability tended to interpret more mammograms per week (p = 0.08), were more likely to specialize (p = 0.02) and to have completed a fellowship in breast or women's imaging (p = 0.05), and had a higher PPV for cancer detection (p = 0.01). Examinations for which low-volume radiologists were "very confident" had a PPV of 2.93 times (95% CI, 2.01-4.27) higher than examinations they rated with neutral confidence. Trends of increasing NPVs with increasing confidence were significant for low-volume radiologists relative to noncancers (p = 0.01) and expert nonrecalls (p < 0.001). A trend of significantly increasing NPVs existed for high-volume radiologists relative to expert nonrecall (p = 0.02) but not relative to noncancer status (p = 0.32). CONCLUSION. Confidence in mammography assessments was associated with better accuracy, especially for low-volume readers. Asking for a second opinion when confidence in an assessment is low may increase accuracy.

KW - Accuracy

KW - Confidence

KW - Mammography

KW - Radiologist performance

KW - Screening

UR - http://www.scopus.com/inward/record.url?scp=84863652819&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863652819&partnerID=8YFLogxK

U2 - 10.2214/AJR.11.7701

DO - 10.2214/AJR.11.7701

M3 - Article

VL - 199

JO - American Journal of Roentgenology

JF - American Journal of Roentgenology

SN - 0361-803X

IS - 1

ER -