Approximate maximum likelihood estimation of scanning observer templates

Craig K. Abbey, Frank W. Samuelson, Adam Wunderlich, Lucretiu M. Popescu, Miguel P. Eckstein, John M Boone

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In localization tasks, an observer is asked to give the location of some target or feature of interest in an image. Scanning linear observer models incorporate the search implicit in this task through convolution of an observer template with the image being evaluated. Such models are becoming increasingly popular as predictors of human performance for validating medical imaging methodology. In addition to convolution, scanning models may utilize internal noise components to model inconsistencies in human observer responses. In this work, we build a probabilistic mathematical model of this process and show how it can, in principle, be used to obtain estimates of the observer template using maximum likelihood methods. The main difficulty of this approach is that a closed form probability distribution for a maximal location response is not generally available in the presence of internal noise. However, for a given image we can generate an empirical distribution of maximal locations using Monte-Carlo sampling. We show that this probability is well approximated by applying an exponential function to the scanning template output. We also evaluate log-likelihood functions on the basis of this approximate distribution. Using 1,000 trials of simulated data as a validation test set, we find that a plot of the approximate log-likelihood function along a single parameter related to the template profile achieves its maximum value near the true value used in the simulation. This finding holds regardless of whether the trials are correctly localized or not. In a second validation study evaluating a parameter related to the relative magnitude of internal noise, only the incorrect localization images produces a maximum in the approximate log-likelihood function that is near the true value of the parameter.

Original languageEnglish (US)
Title of host publicationProgress in Biomedical Optics and Imaging - Proceedings of SPIE
PublisherSPIE
Volume9416
ISBN (Print)9781628415063
DOIs
StatePublished - 2015
EventMedical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment - Orlando, United States
Duration: Feb 25 2015Feb 26 2015

Other

OtherMedical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment
CountryUnited States
CityOrlando
Period2/25/152/26/15

Fingerprint

Likelihood Functions
Maximum likelihood estimation
Noise
templates
Scanning
scanning
Convolution
convolution integrals
Validation Studies
Statistical Models
Diagnostic Imaging
human performance
Exponential functions
Linear Models
Medical imaging
Theoretical Models
Probability distributions
Maximum likelihood
exponential functions
Mathematical models

Keywords

  • and observer modeling
  • localization task
  • ramp-spectrum noise
  • Scanning linear template
  • search models

ASJC Scopus subject areas

  • Atomic and Molecular Physics, and Optics
  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Cite this

Abbey, C. K., Samuelson, F. W., Wunderlich, A., Popescu, L. M., Eckstein, M. P., & Boone, J. M. (2015). Approximate maximum likelihood estimation of scanning observer templates. In Progress in Biomedical Optics and Imaging - Proceedings of SPIE (Vol. 9416). [94160O] SPIE. https://doi.org/10.1117/12.2082874

Approximate maximum likelihood estimation of scanning observer templates. / Abbey, Craig K.; Samuelson, Frank W.; Wunderlich, Adam; Popescu, Lucretiu M.; Eckstein, Miguel P.; Boone, John M.

Progress in Biomedical Optics and Imaging - Proceedings of SPIE. Vol. 9416 SPIE, 2015. 94160O.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abbey, CK, Samuelson, FW, Wunderlich, A, Popescu, LM, Eckstein, MP & Boone, JM 2015, Approximate maximum likelihood estimation of scanning observer templates. in Progress in Biomedical Optics and Imaging - Proceedings of SPIE. vol. 9416, 94160O, SPIE, Medical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment, Orlando, United States, 2/25/15. https://doi.org/10.1117/12.2082874
Abbey CK, Samuelson FW, Wunderlich A, Popescu LM, Eckstein MP, Boone JM. Approximate maximum likelihood estimation of scanning observer templates. In Progress in Biomedical Optics and Imaging - Proceedings of SPIE. Vol. 9416. SPIE. 2015. 94160O https://doi.org/10.1117/12.2082874
Abbey, Craig K. ; Samuelson, Frank W. ; Wunderlich, Adam ; Popescu, Lucretiu M. ; Eckstein, Miguel P. ; Boone, John M. / Approximate maximum likelihood estimation of scanning observer templates. Progress in Biomedical Optics and Imaging - Proceedings of SPIE. Vol. 9416 SPIE, 2015.
@inproceedings{0f7a0231bbe74fef8c80bca17f6138f1,
title = "Approximate maximum likelihood estimation of scanning observer templates",
abstract = "In localization tasks, an observer is asked to give the location of some target or feature of interest in an image. Scanning linear observer models incorporate the search implicit in this task through convolution of an observer template with the image being evaluated. Such models are becoming increasingly popular as predictors of human performance for validating medical imaging methodology. In addition to convolution, scanning models may utilize internal noise components to model inconsistencies in human observer responses. In this work, we build a probabilistic mathematical model of this process and show how it can, in principle, be used to obtain estimates of the observer template using maximum likelihood methods. The main difficulty of this approach is that a closed form probability distribution for a maximal location response is not generally available in the presence of internal noise. However, for a given image we can generate an empirical distribution of maximal locations using Monte-Carlo sampling. We show that this probability is well approximated by applying an exponential function to the scanning template output. We also evaluate log-likelihood functions on the basis of this approximate distribution. Using 1,000 trials of simulated data as a validation test set, we find that a plot of the approximate log-likelihood function along a single parameter related to the template profile achieves its maximum value near the true value used in the simulation. This finding holds regardless of whether the trials are correctly localized or not. In a second validation study evaluating a parameter related to the relative magnitude of internal noise, only the incorrect localization images produces a maximum in the approximate log-likelihood function that is near the true value of the parameter.",
keywords = "and observer modeling, localization task, ramp-spectrum noise, Scanning linear template, search models",
author = "Abbey, {Craig K.} and Samuelson, {Frank W.} and Adam Wunderlich and Popescu, {Lucretiu M.} and Eckstein, {Miguel P.} and Boone, {John M}",
year = "2015",
doi = "10.1117/12.2082874",
language = "English (US)",
isbn = "9781628415063",
volume = "9416",
booktitle = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",
publisher = "SPIE",

}

TY - GEN

T1 - Approximate maximum likelihood estimation of scanning observer templates

AU - Abbey, Craig K.

AU - Samuelson, Frank W.

AU - Wunderlich, Adam

AU - Popescu, Lucretiu M.

AU - Eckstein, Miguel P.

AU - Boone, John M

PY - 2015

Y1 - 2015

N2 - In localization tasks, an observer is asked to give the location of some target or feature of interest in an image. Scanning linear observer models incorporate the search implicit in this task through convolution of an observer template with the image being evaluated. Such models are becoming increasingly popular as predictors of human performance for validating medical imaging methodology. In addition to convolution, scanning models may utilize internal noise components to model inconsistencies in human observer responses. In this work, we build a probabilistic mathematical model of this process and show how it can, in principle, be used to obtain estimates of the observer template using maximum likelihood methods. The main difficulty of this approach is that a closed form probability distribution for a maximal location response is not generally available in the presence of internal noise. However, for a given image we can generate an empirical distribution of maximal locations using Monte-Carlo sampling. We show that this probability is well approximated by applying an exponential function to the scanning template output. We also evaluate log-likelihood functions on the basis of this approximate distribution. Using 1,000 trials of simulated data as a validation test set, we find that a plot of the approximate log-likelihood function along a single parameter related to the template profile achieves its maximum value near the true value used in the simulation. This finding holds regardless of whether the trials are correctly localized or not. In a second validation study evaluating a parameter related to the relative magnitude of internal noise, only the incorrect localization images produces a maximum in the approximate log-likelihood function that is near the true value of the parameter.

AB - In localization tasks, an observer is asked to give the location of some target or feature of interest in an image. Scanning linear observer models incorporate the search implicit in this task through convolution of an observer template with the image being evaluated. Such models are becoming increasingly popular as predictors of human performance for validating medical imaging methodology. In addition to convolution, scanning models may utilize internal noise components to model inconsistencies in human observer responses. In this work, we build a probabilistic mathematical model of this process and show how it can, in principle, be used to obtain estimates of the observer template using maximum likelihood methods. The main difficulty of this approach is that a closed form probability distribution for a maximal location response is not generally available in the presence of internal noise. However, for a given image we can generate an empirical distribution of maximal locations using Monte-Carlo sampling. We show that this probability is well approximated by applying an exponential function to the scanning template output. We also evaluate log-likelihood functions on the basis of this approximate distribution. Using 1,000 trials of simulated data as a validation test set, we find that a plot of the approximate log-likelihood function along a single parameter related to the template profile achieves its maximum value near the true value used in the simulation. This finding holds regardless of whether the trials are correctly localized or not. In a second validation study evaluating a parameter related to the relative magnitude of internal noise, only the incorrect localization images produces a maximum in the approximate log-likelihood function that is near the true value of the parameter.

KW - and observer modeling

KW - localization task

KW - ramp-spectrum noise

KW - Scanning linear template

KW - search models

UR - http://www.scopus.com/inward/record.url?scp=84932172216&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84932172216&partnerID=8YFLogxK

U2 - 10.1117/12.2082874

DO - 10.1117/12.2082874

M3 - Conference contribution

AN - SCOPUS:84932172216

SN - 9781628415063

VL - 9416

BT - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

PB - SPIE

ER -