Estimating the relative utility of screening mammography

Craig K. Abbey, Miguel P. Eckstein, John M Boone

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

Background. The concept of diagnostic utility is a fundamental component of signal detection theory, going back to some of its earliest works. Attaching utility values to the various possible outcomes of a diagnostic test should, in principle, lead to meaningful approaches to evaluating and comparing such systems. However, in many areas of medical imaging, utility is not used because it is presumed to be unknown. Methods. In this work, we estimate relative utility (the utility benefit of a detection relative to that of a correct rejection) for screening mammography using its known relation to the slope of a receiver operating characteristic (ROC) curve at the optimal operating point. The approach assumes that the clinical operating point is optimal for the goal of maximizing expected utility and therefore the slope at this point implies a value of relative utility for the diagnostic task, for known disease prevalence. We examine utility estimation in the context of screening mammography using the Digital Mammographic Imaging Screening Trials (DMIST) data. Results. We show how various conditions can influence the estimated relative utility, including characteristics of the rating scale, verification time, probability model, and scope of the ROC curve fit. Relative utility estimates range from 66 to 227. Conclusions. We argue for one particular set of conditions that results in a relative utility estimate of 162 (±14%). This is broadly consistent with values in screening mammography determined previously by other means. At the disease prevalence found in the DMIST study (0.59% at 365-day verification), optimal ROC slopes are near unity, suggesting that utility-based assessments of screening mammography will be similar to those found using Youden's index.

Original languageEnglish (US)
Pages (from-to)510-520
Number of pages11
JournalMedical Decision Making
Volume33
Issue number4
DOIs
StatePublished - May 2013

Fingerprint

Mammography
ROC Curve
Diagnostic Imaging
Routine Diagnostic Tests

Keywords

  • observer performance
  • relative utility
  • screening mammography

ASJC Scopus subject areas

  • Health Policy
  • Medicine(all)

Cite this

Estimating the relative utility of screening mammography. / Abbey, Craig K.; Eckstein, Miguel P.; Boone, John M.

In: Medical Decision Making, Vol. 33, No. 4, 05.2013, p. 510-520.

Research output: Contribution to journalArticle

Abbey, Craig K. ; Eckstein, Miguel P. ; Boone, John M. / Estimating the relative utility of screening mammography. In: Medical Decision Making. 2013 ; Vol. 33, No. 4. pp. 510-520.
@article{620fd5a9bc074531bdf4ba4b111b6684,
title = "Estimating the relative utility of screening mammography",
abstract = "Background. The concept of diagnostic utility is a fundamental component of signal detection theory, going back to some of its earliest works. Attaching utility values to the various possible outcomes of a diagnostic test should, in principle, lead to meaningful approaches to evaluating and comparing such systems. However, in many areas of medical imaging, utility is not used because it is presumed to be unknown. Methods. In this work, we estimate relative utility (the utility benefit of a detection relative to that of a correct rejection) for screening mammography using its known relation to the slope of a receiver operating characteristic (ROC) curve at the optimal operating point. The approach assumes that the clinical operating point is optimal for the goal of maximizing expected utility and therefore the slope at this point implies a value of relative utility for the diagnostic task, for known disease prevalence. We examine utility estimation in the context of screening mammography using the Digital Mammographic Imaging Screening Trials (DMIST) data. Results. We show how various conditions can influence the estimated relative utility, including characteristics of the rating scale, verification time, probability model, and scope of the ROC curve fit. Relative utility estimates range from 66 to 227. Conclusions. We argue for one particular set of conditions that results in a relative utility estimate of 162 (±14{\%}). This is broadly consistent with values in screening mammography determined previously by other means. At the disease prevalence found in the DMIST study (0.59{\%} at 365-day verification), optimal ROC slopes are near unity, suggesting that utility-based assessments of screening mammography will be similar to those found using Youden's index.",
keywords = "observer performance, relative utility, screening mammography",
author = "Abbey, {Craig K.} and Eckstein, {Miguel P.} and Boone, {John M}",
year = "2013",
month = "5",
doi = "10.1177/0272989X12470756",
language = "English (US)",
volume = "33",
pages = "510--520",
journal = "Medical Decision Making",
issn = "0272-989X",
publisher = "SAGE Publications Inc.",
number = "4",

}

TY - JOUR

T1 - Estimating the relative utility of screening mammography

AU - Abbey, Craig K.

AU - Eckstein, Miguel P.

AU - Boone, John M

PY - 2013/5

Y1 - 2013/5

N2 - Background. The concept of diagnostic utility is a fundamental component of signal detection theory, going back to some of its earliest works. Attaching utility values to the various possible outcomes of a diagnostic test should, in principle, lead to meaningful approaches to evaluating and comparing such systems. However, in many areas of medical imaging, utility is not used because it is presumed to be unknown. Methods. In this work, we estimate relative utility (the utility benefit of a detection relative to that of a correct rejection) for screening mammography using its known relation to the slope of a receiver operating characteristic (ROC) curve at the optimal operating point. The approach assumes that the clinical operating point is optimal for the goal of maximizing expected utility and therefore the slope at this point implies a value of relative utility for the diagnostic task, for known disease prevalence. We examine utility estimation in the context of screening mammography using the Digital Mammographic Imaging Screening Trials (DMIST) data. Results. We show how various conditions can influence the estimated relative utility, including characteristics of the rating scale, verification time, probability model, and scope of the ROC curve fit. Relative utility estimates range from 66 to 227. Conclusions. We argue for one particular set of conditions that results in a relative utility estimate of 162 (±14%). This is broadly consistent with values in screening mammography determined previously by other means. At the disease prevalence found in the DMIST study (0.59% at 365-day verification), optimal ROC slopes are near unity, suggesting that utility-based assessments of screening mammography will be similar to those found using Youden's index.

AB - Background. The concept of diagnostic utility is a fundamental component of signal detection theory, going back to some of its earliest works. Attaching utility values to the various possible outcomes of a diagnostic test should, in principle, lead to meaningful approaches to evaluating and comparing such systems. However, in many areas of medical imaging, utility is not used because it is presumed to be unknown. Methods. In this work, we estimate relative utility (the utility benefit of a detection relative to that of a correct rejection) for screening mammography using its known relation to the slope of a receiver operating characteristic (ROC) curve at the optimal operating point. The approach assumes that the clinical operating point is optimal for the goal of maximizing expected utility and therefore the slope at this point implies a value of relative utility for the diagnostic task, for known disease prevalence. We examine utility estimation in the context of screening mammography using the Digital Mammographic Imaging Screening Trials (DMIST) data. Results. We show how various conditions can influence the estimated relative utility, including characteristics of the rating scale, verification time, probability model, and scope of the ROC curve fit. Relative utility estimates range from 66 to 227. Conclusions. We argue for one particular set of conditions that results in a relative utility estimate of 162 (±14%). This is broadly consistent with values in screening mammography determined previously by other means. At the disease prevalence found in the DMIST study (0.59% at 365-day verification), optimal ROC slopes are near unity, suggesting that utility-based assessments of screening mammography will be similar to those found using Youden's index.

KW - observer performance

KW - relative utility

KW - screening mammography

UR - http://www.scopus.com/inward/record.url?scp=84877076262&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84877076262&partnerID=8YFLogxK

U2 - 10.1177/0272989X12470756

DO - 10.1177/0272989X12470756

M3 - Article

VL - 33

SP - 510

EP - 520

JO - Medical Decision Making

JF - Medical Decision Making

SN - 0272-989X

IS - 4

ER -