Reliability of Untrained and Experienced Raters on FEES: Rating Overall Residue is a Simple Task

Jessica M. Pisegna, James C. Borders, Asako Kaneoka, Wendy J. Coster, Rebecca J Leonard, Susan E. Langmore

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

The purpose of this study was to investigate the reliability of residue ratings on Fiberoptic Endoscopic Evaluation of Swallowing (FEES). We also examined rating differences based on experience to determine if years of experience influenced residue ratings. A group of 44 raters watched 81 FEES videos representing a wide range of residue severities for thin liquid, applesauce, and cracker boluses. Raters were untrained on the rating scales and simply rated their overall impression of residue amount on a visual analog scale (VAS) and a five-point ordinal scale in a randomized fashion across two sessions. Intra-class correlation coefficients, kappa coefficients, and ANOVAs were used to analyze agreement and differences in ratings. Residue ratings on both the VAS and ordinal scales had acceptable inter- and intra-rater reliability. Inter-rater agreement was acceptable (ICC > 0.7) for all comparisons. Intra-rater agreement was excellent on the VAS scale (rc = 0.9) and good on the ordinal scale (k = 0.78). There was no significant difference between expert ratings and other raters based on years of experience for cracker ratings (p = 0.2119) and applesauce ratings (p = 0.2899), but there was a significant difference between clinicians on thin liquid ratings (p = 0.0005). Without any specific training, raters demonstrated high reliability when rating the overall amount of residue on FEES. Years of experience with FEES did not influence residue ratings, suggesting that expert ratings of overall residue amount are not unique or specialized. Rating the overall amount of residue on FEES appears to be a simple visual-perceptual task for puree and cracker boluses.

Original languageEnglish (US)
Pages (from-to)1-10
Number of pages10
JournalDysphagia
DOIs
StateAccepted/In press - Mar 7 2018

Fingerprint

Deglutition
Visual Analog Scale
Analysis of Variance

Keywords

  • Deglutition
  • FEES
  • Pharyngeal residue
  • Psychometrics
  • Ratings
  • Reliability

ASJC Scopus subject areas

  • Otorhinolaryngology
  • Gastroenterology
  • Speech and Hearing

Cite this

Pisegna, J. M., Borders, J. C., Kaneoka, A., Coster, W. J., Leonard, R. J., & Langmore, S. E. (Accepted/In press). Reliability of Untrained and Experienced Raters on FEES: Rating Overall Residue is a Simple Task. Dysphagia, 1-10. https://doi.org/10.1007/s00455-018-9883-x

Reliability of Untrained and Experienced Raters on FEES : Rating Overall Residue is a Simple Task. / Pisegna, Jessica M.; Borders, James C.; Kaneoka, Asako; Coster, Wendy J.; Leonard, Rebecca J; Langmore, Susan E.

In: Dysphagia, 07.03.2018, p. 1-10.

Research output: Contribution to journalArticle

Pisegna, Jessica M. ; Borders, James C. ; Kaneoka, Asako ; Coster, Wendy J. ; Leonard, Rebecca J ; Langmore, Susan E. / Reliability of Untrained and Experienced Raters on FEES : Rating Overall Residue is a Simple Task. In: Dysphagia. 2018 ; pp. 1-10.
@article{02a0149252054dad91f460e884f4022d,
title = "Reliability of Untrained and Experienced Raters on FEES: Rating Overall Residue is a Simple Task",
abstract = "The purpose of this study was to investigate the reliability of residue ratings on Fiberoptic Endoscopic Evaluation of Swallowing (FEES). We also examined rating differences based on experience to determine if years of experience influenced residue ratings. A group of 44 raters watched 81 FEES videos representing a wide range of residue severities for thin liquid, applesauce, and cracker boluses. Raters were untrained on the rating scales and simply rated their overall impression of residue amount on a visual analog scale (VAS) and a five-point ordinal scale in a randomized fashion across two sessions. Intra-class correlation coefficients, kappa coefficients, and ANOVAs were used to analyze agreement and differences in ratings. Residue ratings on both the VAS and ordinal scales had acceptable inter- and intra-rater reliability. Inter-rater agreement was acceptable (ICC > 0.7) for all comparisons. Intra-rater agreement was excellent on the VAS scale (rc = 0.9) and good on the ordinal scale (k = 0.78). There was no significant difference between expert ratings and other raters based on years of experience for cracker ratings (p = 0.2119) and applesauce ratings (p = 0.2899), but there was a significant difference between clinicians on thin liquid ratings (p = 0.0005). Without any specific training, raters demonstrated high reliability when rating the overall amount of residue on FEES. Years of experience with FEES did not influence residue ratings, suggesting that expert ratings of overall residue amount are not unique or specialized. Rating the overall amount of residue on FEES appears to be a simple visual-perceptual task for puree and cracker boluses.",
keywords = "Deglutition, FEES, Pharyngeal residue, Psychometrics, Ratings, Reliability",
author = "Pisegna, {Jessica M.} and Borders, {James C.} and Asako Kaneoka and Coster, {Wendy J.} and Leonard, {Rebecca J} and Langmore, {Susan E.}",
year = "2018",
month = "3",
day = "7",
doi = "10.1007/s00455-018-9883-x",
language = "English (US)",
pages = "1--10",
journal = "Dysphagia",
issn = "0179-051X",
publisher = "Springer New York",

}

TY - JOUR

T1 - Reliability of Untrained and Experienced Raters on FEES

T2 - Rating Overall Residue is a Simple Task

AU - Pisegna, Jessica M.

AU - Borders, James C.

AU - Kaneoka, Asako

AU - Coster, Wendy J.

AU - Leonard, Rebecca J

AU - Langmore, Susan E.

PY - 2018/3/7

Y1 - 2018/3/7

N2 - The purpose of this study was to investigate the reliability of residue ratings on Fiberoptic Endoscopic Evaluation of Swallowing (FEES). We also examined rating differences based on experience to determine if years of experience influenced residue ratings. A group of 44 raters watched 81 FEES videos representing a wide range of residue severities for thin liquid, applesauce, and cracker boluses. Raters were untrained on the rating scales and simply rated their overall impression of residue amount on a visual analog scale (VAS) and a five-point ordinal scale in a randomized fashion across two sessions. Intra-class correlation coefficients, kappa coefficients, and ANOVAs were used to analyze agreement and differences in ratings. Residue ratings on both the VAS and ordinal scales had acceptable inter- and intra-rater reliability. Inter-rater agreement was acceptable (ICC > 0.7) for all comparisons. Intra-rater agreement was excellent on the VAS scale (rc = 0.9) and good on the ordinal scale (k = 0.78). There was no significant difference between expert ratings and other raters based on years of experience for cracker ratings (p = 0.2119) and applesauce ratings (p = 0.2899), but there was a significant difference between clinicians on thin liquid ratings (p = 0.0005). Without any specific training, raters demonstrated high reliability when rating the overall amount of residue on FEES. Years of experience with FEES did not influence residue ratings, suggesting that expert ratings of overall residue amount are not unique or specialized. Rating the overall amount of residue on FEES appears to be a simple visual-perceptual task for puree and cracker boluses.

AB - The purpose of this study was to investigate the reliability of residue ratings on Fiberoptic Endoscopic Evaluation of Swallowing (FEES). We also examined rating differences based on experience to determine if years of experience influenced residue ratings. A group of 44 raters watched 81 FEES videos representing a wide range of residue severities for thin liquid, applesauce, and cracker boluses. Raters were untrained on the rating scales and simply rated their overall impression of residue amount on a visual analog scale (VAS) and a five-point ordinal scale in a randomized fashion across two sessions. Intra-class correlation coefficients, kappa coefficients, and ANOVAs were used to analyze agreement and differences in ratings. Residue ratings on both the VAS and ordinal scales had acceptable inter- and intra-rater reliability. Inter-rater agreement was acceptable (ICC > 0.7) for all comparisons. Intra-rater agreement was excellent on the VAS scale (rc = 0.9) and good on the ordinal scale (k = 0.78). There was no significant difference between expert ratings and other raters based on years of experience for cracker ratings (p = 0.2119) and applesauce ratings (p = 0.2899), but there was a significant difference between clinicians on thin liquid ratings (p = 0.0005). Without any specific training, raters demonstrated high reliability when rating the overall amount of residue on FEES. Years of experience with FEES did not influence residue ratings, suggesting that expert ratings of overall residue amount are not unique or specialized. Rating the overall amount of residue on FEES appears to be a simple visual-perceptual task for puree and cracker boluses.

KW - Deglutition

KW - FEES

KW - Pharyngeal residue

KW - Psychometrics

KW - Ratings

KW - Reliability

UR - http://www.scopus.com/inward/record.url?scp=85042934519&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85042934519&partnerID=8YFLogxK

U2 - 10.1007/s00455-018-9883-x

DO - 10.1007/s00455-018-9883-x

M3 - Article

C2 - 29516172

AN - SCOPUS:85042934519

SP - 1

EP - 10

JO - Dysphagia

JF - Dysphagia

SN - 0179-051X

ER -