Reliability of Clinical Assessments in Older Adults With Syncope or Near Syncope

Daniel Nishijima, Amber L. Laurie, Robert E. Weiss, Annick N. Yagapen, Susan E. Malveau, David H. Adler, Aveh Bastani, Christopher W. Baugh, Jeffrey M. Caterino, Carol L. Clark, Deborah B. Diercks, Judd E. Hollander, Bret A. Nicks, Manish N. Shah, Kirk A. Stiffler, Alan B. Storrow, Scott T. Wilber, Benjamin C. Sun, Erik P. Hess

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Objectives: Clinical prediction models for risk stratification of older adults with syncope or near syncope may improve resource utilization and management. Predictors considered for inclusion into such models must be reliable. Our primary objective was to evaluate the inter-rater agreement of historical, physical examination, and electrocardiogram (ECG) findings in older adults undergoing emergency department (ED) evaluation for syncope or near syncope. Our secondary objective was to assess the level of agreement between clinicians on the patient's overall risk for death or serious cardiac outcomes. Methods: We conducted a cross-sectional study at 11 EDs in adults 60 years of age or older who presented with unexplained syncope or near syncope. We excluded patients with a presumptive cause of syncope (e.g., seizure) or if they were unable or unwilling to follow-up. Evaluations of the patient's past medical history and current medication use were completed by treating provider and trained research associate pairs. Evaluations of the patient's physical examination and ECG interpretation were completed by attending/resident, attending/advanced practice provider, or attending/attending pairs. All evaluations were blinded to the responses from the other rater. We calculated the percent agreement and kappa statistic for binary variables. Inter-rater agreement was considered acceptable if the kappa statistic was 0.6 or higher. Results: We obtained paired observations from 255 patients; mean (±SD) age was 73 (±9) years, 137 (54%) were male, and 204 (80%) were admitted to the hospital. Acceptable agreement was achieved in 18 of the 21 (86%) past medical history and current medication findings, none of the 10 physical examination variables, and three of the 13 (23%) ECG interpretation variables. There was moderate agreement (Spearman correlation coefficient, r = 0.40) between clinicians on the patient's probability of 30-day death or serious cardiac outcome, although as the probability increased, there was less agreement. Conclusions: Acceptable agreement between raters was more commonly achieved with historical rather than physical examination or ECG interpretation variables. Clinicians had moderate agreement in assessing the patient's overall risk for a serious outcome at 30 days. Future development of clinical prediction models in older adults with syncope should account for variability of assessments between raters and consider the use of objective clinical variables.

Original languageEnglish (US)
Pages (from-to)1014-1021
Number of pages8
JournalAcademic Emergency Medicine
Volume23
Issue number9
DOIs
StatePublished - Sep 1 2016

Fingerprint

Syncope
Physical Examination
Electrocardiography
Hospital Emergency Service
Seizures
Cross-Sectional Studies
Research

ASJC Scopus subject areas

  • Emergency Medicine

Cite this

Nishijima, D., Laurie, A. L., Weiss, R. E., Yagapen, A. N., Malveau, S. E., Adler, D. H., ... Hess, E. P. (2016). Reliability of Clinical Assessments in Older Adults With Syncope or Near Syncope. Academic Emergency Medicine, 23(9), 1014-1021. https://doi.org/10.1111/acem.12977

Reliability of Clinical Assessments in Older Adults With Syncope or Near Syncope. / Nishijima, Daniel; Laurie, Amber L.; Weiss, Robert E.; Yagapen, Annick N.; Malveau, Susan E.; Adler, David H.; Bastani, Aveh; Baugh, Christopher W.; Caterino, Jeffrey M.; Clark, Carol L.; Diercks, Deborah B.; Hollander, Judd E.; Nicks, Bret A.; Shah, Manish N.; Stiffler, Kirk A.; Storrow, Alan B.; Wilber, Scott T.; Sun, Benjamin C.; Hess, Erik P.

In: Academic Emergency Medicine, Vol. 23, No. 9, 01.09.2016, p. 1014-1021.

Research output: Contribution to journalArticle

Nishijima, D, Laurie, AL, Weiss, RE, Yagapen, AN, Malveau, SE, Adler, DH, Bastani, A, Baugh, CW, Caterino, JM, Clark, CL, Diercks, DB, Hollander, JE, Nicks, BA, Shah, MN, Stiffler, KA, Storrow, AB, Wilber, ST, Sun, BC & Hess, EP 2016, 'Reliability of Clinical Assessments in Older Adults With Syncope or Near Syncope', Academic Emergency Medicine, vol. 23, no. 9, pp. 1014-1021. https://doi.org/10.1111/acem.12977
Nishijima, Daniel ; Laurie, Amber L. ; Weiss, Robert E. ; Yagapen, Annick N. ; Malveau, Susan E. ; Adler, David H. ; Bastani, Aveh ; Baugh, Christopher W. ; Caterino, Jeffrey M. ; Clark, Carol L. ; Diercks, Deborah B. ; Hollander, Judd E. ; Nicks, Bret A. ; Shah, Manish N. ; Stiffler, Kirk A. ; Storrow, Alan B. ; Wilber, Scott T. ; Sun, Benjamin C. ; Hess, Erik P. / Reliability of Clinical Assessments in Older Adults With Syncope or Near Syncope. In: Academic Emergency Medicine. 2016 ; Vol. 23, No. 9. pp. 1014-1021.
@article{07a2566d22174996bebc4e3d164df67a,
title = "Reliability of Clinical Assessments in Older Adults With Syncope or Near Syncope",
abstract = "Objectives: Clinical prediction models for risk stratification of older adults with syncope or near syncope may improve resource utilization and management. Predictors considered for inclusion into such models must be reliable. Our primary objective was to evaluate the inter-rater agreement of historical, physical examination, and electrocardiogram (ECG) findings in older adults undergoing emergency department (ED) evaluation for syncope or near syncope. Our secondary objective was to assess the level of agreement between clinicians on the patient's overall risk for death or serious cardiac outcomes. Methods: We conducted a cross-sectional study at 11 EDs in adults 60 years of age or older who presented with unexplained syncope or near syncope. We excluded patients with a presumptive cause of syncope (e.g., seizure) or if they were unable or unwilling to follow-up. Evaluations of the patient's past medical history and current medication use were completed by treating provider and trained research associate pairs. Evaluations of the patient's physical examination and ECG interpretation were completed by attending/resident, attending/advanced practice provider, or attending/attending pairs. All evaluations were blinded to the responses from the other rater. We calculated the percent agreement and kappa statistic for binary variables. Inter-rater agreement was considered acceptable if the kappa statistic was 0.6 or higher. Results: We obtained paired observations from 255 patients; mean (±SD) age was 73 (±9) years, 137 (54{\%}) were male, and 204 (80{\%}) were admitted to the hospital. Acceptable agreement was achieved in 18 of the 21 (86{\%}) past medical history and current medication findings, none of the 10 physical examination variables, and three of the 13 (23{\%}) ECG interpretation variables. There was moderate agreement (Spearman correlation coefficient, r = 0.40) between clinicians on the patient's probability of 30-day death or serious cardiac outcome, although as the probability increased, there was less agreement. Conclusions: Acceptable agreement between raters was more commonly achieved with historical rather than physical examination or ECG interpretation variables. Clinicians had moderate agreement in assessing the patient's overall risk for a serious outcome at 30 days. Future development of clinical prediction models in older adults with syncope should account for variability of assessments between raters and consider the use of objective clinical variables.",
author = "Daniel Nishijima and Laurie, {Amber L.} and Weiss, {Robert E.} and Yagapen, {Annick N.} and Malveau, {Susan E.} and Adler, {David H.} and Aveh Bastani and Baugh, {Christopher W.} and Caterino, {Jeffrey M.} and Clark, {Carol L.} and Diercks, {Deborah B.} and Hollander, {Judd E.} and Nicks, {Bret A.} and Shah, {Manish N.} and Stiffler, {Kirk A.} and Storrow, {Alan B.} and Wilber, {Scott T.} and Sun, {Benjamin C.} and Hess, {Erik P.}",
year = "2016",
month = "9",
day = "1",
doi = "10.1111/acem.12977",
language = "English (US)",
volume = "23",
pages = "1014--1021",
journal = "Academic Emergency Medicine",
issn = "1069-6563",
publisher = "Wiley-Blackwell",
number = "9",

}

TY - JOUR

T1 - Reliability of Clinical Assessments in Older Adults With Syncope or Near Syncope

AU - Nishijima, Daniel

AU - Laurie, Amber L.

AU - Weiss, Robert E.

AU - Yagapen, Annick N.

AU - Malveau, Susan E.

AU - Adler, David H.

AU - Bastani, Aveh

AU - Baugh, Christopher W.

AU - Caterino, Jeffrey M.

AU - Clark, Carol L.

AU - Diercks, Deborah B.

AU - Hollander, Judd E.

AU - Nicks, Bret A.

AU - Shah, Manish N.

AU - Stiffler, Kirk A.

AU - Storrow, Alan B.

AU - Wilber, Scott T.

AU - Sun, Benjamin C.

AU - Hess, Erik P.

PY - 2016/9/1

Y1 - 2016/9/1

N2 - Objectives: Clinical prediction models for risk stratification of older adults with syncope or near syncope may improve resource utilization and management. Predictors considered for inclusion into such models must be reliable. Our primary objective was to evaluate the inter-rater agreement of historical, physical examination, and electrocardiogram (ECG) findings in older adults undergoing emergency department (ED) evaluation for syncope or near syncope. Our secondary objective was to assess the level of agreement between clinicians on the patient's overall risk for death or serious cardiac outcomes. Methods: We conducted a cross-sectional study at 11 EDs in adults 60 years of age or older who presented with unexplained syncope or near syncope. We excluded patients with a presumptive cause of syncope (e.g., seizure) or if they were unable or unwilling to follow-up. Evaluations of the patient's past medical history and current medication use were completed by treating provider and trained research associate pairs. Evaluations of the patient's physical examination and ECG interpretation were completed by attending/resident, attending/advanced practice provider, or attending/attending pairs. All evaluations were blinded to the responses from the other rater. We calculated the percent agreement and kappa statistic for binary variables. Inter-rater agreement was considered acceptable if the kappa statistic was 0.6 or higher. Results: We obtained paired observations from 255 patients; mean (±SD) age was 73 (±9) years, 137 (54%) were male, and 204 (80%) were admitted to the hospital. Acceptable agreement was achieved in 18 of the 21 (86%) past medical history and current medication findings, none of the 10 physical examination variables, and three of the 13 (23%) ECG interpretation variables. There was moderate agreement (Spearman correlation coefficient, r = 0.40) between clinicians on the patient's probability of 30-day death or serious cardiac outcome, although as the probability increased, there was less agreement. Conclusions: Acceptable agreement between raters was more commonly achieved with historical rather than physical examination or ECG interpretation variables. Clinicians had moderate agreement in assessing the patient's overall risk for a serious outcome at 30 days. Future development of clinical prediction models in older adults with syncope should account for variability of assessments between raters and consider the use of objective clinical variables.

AB - Objectives: Clinical prediction models for risk stratification of older adults with syncope or near syncope may improve resource utilization and management. Predictors considered for inclusion into such models must be reliable. Our primary objective was to evaluate the inter-rater agreement of historical, physical examination, and electrocardiogram (ECG) findings in older adults undergoing emergency department (ED) evaluation for syncope or near syncope. Our secondary objective was to assess the level of agreement between clinicians on the patient's overall risk for death or serious cardiac outcomes. Methods: We conducted a cross-sectional study at 11 EDs in adults 60 years of age or older who presented with unexplained syncope or near syncope. We excluded patients with a presumptive cause of syncope (e.g., seizure) or if they were unable or unwilling to follow-up. Evaluations of the patient's past medical history and current medication use were completed by treating provider and trained research associate pairs. Evaluations of the patient's physical examination and ECG interpretation were completed by attending/resident, attending/advanced practice provider, or attending/attending pairs. All evaluations were blinded to the responses from the other rater. We calculated the percent agreement and kappa statistic for binary variables. Inter-rater agreement was considered acceptable if the kappa statistic was 0.6 or higher. Results: We obtained paired observations from 255 patients; mean (±SD) age was 73 (±9) years, 137 (54%) were male, and 204 (80%) were admitted to the hospital. Acceptable agreement was achieved in 18 of the 21 (86%) past medical history and current medication findings, none of the 10 physical examination variables, and three of the 13 (23%) ECG interpretation variables. There was moderate agreement (Spearman correlation coefficient, r = 0.40) between clinicians on the patient's probability of 30-day death or serious cardiac outcome, although as the probability increased, there was less agreement. Conclusions: Acceptable agreement between raters was more commonly achieved with historical rather than physical examination or ECG interpretation variables. Clinicians had moderate agreement in assessing the patient's overall risk for a serious outcome at 30 days. Future development of clinical prediction models in older adults with syncope should account for variability of assessments between raters and consider the use of objective clinical variables.

UR - http://www.scopus.com/inward/record.url?scp=84986610740&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84986610740&partnerID=8YFLogxK

U2 - 10.1111/acem.12977

DO - 10.1111/acem.12977

M3 - Article

VL - 23

SP - 1014

EP - 1021

JO - Academic Emergency Medicine

JF - Academic Emergency Medicine

SN - 1069-6563

IS - 9

ER -