TY - JOUR

T1 - On the interpretation of test sensitivity in the two-test two-population problem

T2 - Assumptions matter

AU - Johnson, Wesley O.

AU - Gardner, Ian

AU - Metoyer, Candace N.

AU - Branscum, Adam J.

PY - 2009/10/1

Y1 - 2009/10/1

N2 - Bayesian analyses of diagnostic test accuracy often require the assumption of constant test accuracy among populations to ensure model identifiability. In a prior study (Toft, N., Jørgensen, E., Højsgaard, S., 2005. Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard. Prev. Vet. Med. 68, 19-33), the sensitivity estimate from a two-test two-population model was shown to be weighted toward the population with the higher prevalence of infection. In the present study, we provided analytical formulae that give insight into the effect of assuming constant sensitivity when this assumption was false. To further investigate the effect of failure of the assumption of constant sensitivity, we also simulated several data sets under the assumption that the first test's sensitivity varied in the two populations. Bayesian conditional independence models that presumed constant sensitivities were implemented in WinBUGS and posterior estimates (mean and 95% probability intervals) were evaluated based on the known true values of the parameters. Findings from the Bayesian analyses of several scenarios indicated that the posterior mean was a good estimate of the weighted mean of the sensitivities in the two populations, when one test was perfectly specific. When neither test was perfectly specific, the Bayesian posterior mean for test 1 sensitivity was either greater than the larger of the two true sensitivities, or smaller than both, and estimates of prevalence and the second test's specificity were incorrect. The implication is that estimates of some parameters will be biased if test sensitivities are not constant across populations. Without a perfectly specific test, and if the assumption of constant sensitivity fails, the only solution we are aware of would involve incorporating prior information on at least two parameters.

AB - Bayesian analyses of diagnostic test accuracy often require the assumption of constant test accuracy among populations to ensure model identifiability. In a prior study (Toft, N., Jørgensen, E., Højsgaard, S., 2005. Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard. Prev. Vet. Med. 68, 19-33), the sensitivity estimate from a two-test two-population model was shown to be weighted toward the population with the higher prevalence of infection. In the present study, we provided analytical formulae that give insight into the effect of assuming constant sensitivity when this assumption was false. To further investigate the effect of failure of the assumption of constant sensitivity, we also simulated several data sets under the assumption that the first test's sensitivity varied in the two populations. Bayesian conditional independence models that presumed constant sensitivities were implemented in WinBUGS and posterior estimates (mean and 95% probability intervals) were evaluated based on the known true values of the parameters. Findings from the Bayesian analyses of several scenarios indicated that the posterior mean was a good estimate of the weighted mean of the sensitivities in the two populations, when one test was perfectly specific. When neither test was perfectly specific, the Bayesian posterior mean for test 1 sensitivity was either greater than the larger of the two true sensitivities, or smaller than both, and estimates of prevalence and the second test's specificity were incorrect. The implication is that estimates of some parameters will be biased if test sensitivities are not constant across populations. Without a perfectly specific test, and if the assumption of constant sensitivity fails, the only solution we are aware of would involve incorporating prior information on at least two parameters.

KW - Bayesian modeling

KW - Diagnostic test accuracy

KW - Hui-Walter model

KW - Sensitivity

KW - WinBUGS

UR - http://www.scopus.com/inward/record.url?scp=68549097789&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=68549097789&partnerID=8YFLogxK

U2 - 10.1016/j.prevetmed.2009.06.006

DO - 10.1016/j.prevetmed.2009.06.006

M3 - Article

C2 - 19651450

AN - SCOPUS:68549097789

VL - 91

SP - 116

EP - 121

JO - Preventive Veterinary Medicine

JF - Preventive Veterinary Medicine

SN - 0167-5877

IS - 2-4

ER -