Interobserver agreement in assessment of clinical variables in children with blunt head trauma

Marc H. Gorelick, Shireen M. Atabaki, John Hoyle, Peter S. Dayan, James F Holmes Jr, Richard Holubkov, David Monroe, James M. Callahan, Nathan Kuppermann

Research output: Contribution to journalArticle

42 Citations (Scopus)

Abstract

Objectives: To be useful in development of clinical decision rules, clinical variables must demonstrate acceptable agreement when assessed by different observers. The objective was to determine the interobserver agreement in the assessment of historical and physical examination findings of children undergoing emergency department (ED) evaluation for blunt head trauma. Methods: This was a prospective cohort study of children younger than 18 years evaluated for blunt head trauma at one of 25 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Patients were excluded if injury occurred more than 24 hours prior to evaluation, if neuroimaging was obtained at another hospital prior to evaluation, or if the patient had a clinically trivial mechanism of injury. Two clinicians independently completed a standardized clinical assessment on a templated data form. Assessments were performed within 60 minutes of each other and prior to clinician review of any neuroimaging (if obtained). Agreement between the two observers beyond that expected by chance was calculated for each clinical variable, using the kappa (j) statistic for categorical variables and weighted kappa for ordinal variables. Variables with a lower 95% confidence limit (LCL) of κ > 0.4 were considered to have acceptable agreement. Results: Fifteen-hundred pairs of observations were obtained. Acceptable agreement was achieved in 27 of the 32 variables studied (84%). Mechanism of injury (low, medium, or high risk) had κ = 0.83. For subjective symptoms, kappa ranged from 0.47 (dizziness) to 0.93 (frequency of vomiting); all had 95% LCL > 0.4. Of the physical examination findings, kappa ranged from 0.22 (agitated) to 0.89 (Glasgow Coma Scale [GCS] score). The 95% LCL for kappa was <0.4 for four individual signs of altered mental status and for quality (i.e., boggy or firm) of scalp hematoma if present. Conclusions: Both subjective and objective clinical variables in children with blunt head trauma can be assessed by different observers with acceptable agreement, making these variables suitable candidates for clinical decision rules.

Original languageEnglish (US)
Pages (from-to)812-818
Number of pages7
JournalAcademic Emergency Medicine
Volume15
Issue number9
DOIs
StatePublished - Sep 2008

Fingerprint

Craniocerebral Trauma
Neuroimaging
Physical Examination
Wounds and Injuries
Glasgow Coma Scale
Dizziness
Emergency Medical Services
Scalp
Hematoma
Vomiting
Hospital Emergency Service
Cohort Studies
Prospective Studies
Pediatrics
Research

Keywords

  • Craniocerebral trauma
  • Observer variation
  • Physical examination
  • Reproducibility of results

ASJC Scopus subject areas

  • Emergency Medicine

Cite this

Interobserver agreement in assessment of clinical variables in children with blunt head trauma. / Gorelick, Marc H.; Atabaki, Shireen M.; Hoyle, John; Dayan, Peter S.; Holmes Jr, James F; Holubkov, Richard; Monroe, David; Callahan, James M.; Kuppermann, Nathan.

In: Academic Emergency Medicine, Vol. 15, No. 9, 09.2008, p. 812-818.

Research output: Contribution to journalArticle

Gorelick, Marc H. ; Atabaki, Shireen M. ; Hoyle, John ; Dayan, Peter S. ; Holmes Jr, James F ; Holubkov, Richard ; Monroe, David ; Callahan, James M. ; Kuppermann, Nathan. / Interobserver agreement in assessment of clinical variables in children with blunt head trauma. In: Academic Emergency Medicine. 2008 ; Vol. 15, No. 9. pp. 812-818.
@article{1cf8a66f2520497fafefec9930eec484,
title = "Interobserver agreement in assessment of clinical variables in children with blunt head trauma",
abstract = "Objectives: To be useful in development of clinical decision rules, clinical variables must demonstrate acceptable agreement when assessed by different observers. The objective was to determine the interobserver agreement in the assessment of historical and physical examination findings of children undergoing emergency department (ED) evaluation for blunt head trauma. Methods: This was a prospective cohort study of children younger than 18 years evaluated for blunt head trauma at one of 25 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Patients were excluded if injury occurred more than 24 hours prior to evaluation, if neuroimaging was obtained at another hospital prior to evaluation, or if the patient had a clinically trivial mechanism of injury. Two clinicians independently completed a standardized clinical assessment on a templated data form. Assessments were performed within 60 minutes of each other and prior to clinician review of any neuroimaging (if obtained). Agreement between the two observers beyond that expected by chance was calculated for each clinical variable, using the kappa (j) statistic for categorical variables and weighted kappa for ordinal variables. Variables with a lower 95{\%} confidence limit (LCL) of κ > 0.4 were considered to have acceptable agreement. Results: Fifteen-hundred pairs of observations were obtained. Acceptable agreement was achieved in 27 of the 32 variables studied (84{\%}). Mechanism of injury (low, medium, or high risk) had κ = 0.83. For subjective symptoms, kappa ranged from 0.47 (dizziness) to 0.93 (frequency of vomiting); all had 95{\%} LCL > 0.4. Of the physical examination findings, kappa ranged from 0.22 (agitated) to 0.89 (Glasgow Coma Scale [GCS] score). The 95{\%} LCL for kappa was <0.4 for four individual signs of altered mental status and for quality (i.e., boggy or firm) of scalp hematoma if present. Conclusions: Both subjective and objective clinical variables in children with blunt head trauma can be assessed by different observers with acceptable agreement, making these variables suitable candidates for clinical decision rules.",
keywords = "Craniocerebral trauma, Observer variation, Physical examination, Reproducibility of results",
author = "Gorelick, {Marc H.} and Atabaki, {Shireen M.} and John Hoyle and Dayan, {Peter S.} and {Holmes Jr}, {James F} and Richard Holubkov and David Monroe and Callahan, {James M.} and Nathan Kuppermann",
year = "2008",
month = "9",
doi = "10.1111/j.1553-2712.2008.00206.x",
language = "English (US)",
volume = "15",
pages = "812--818",
journal = "Academic Emergency Medicine",
issn = "1069-6563",
publisher = "Wiley-Blackwell",
number = "9",

}

TY - JOUR

T1 - Interobserver agreement in assessment of clinical variables in children with blunt head trauma

AU - Gorelick, Marc H.

AU - Atabaki, Shireen M.

AU - Hoyle, John

AU - Dayan, Peter S.

AU - Holmes Jr, James F

AU - Holubkov, Richard

AU - Monroe, David

AU - Callahan, James M.

AU - Kuppermann, Nathan

PY - 2008/9

Y1 - 2008/9

N2 - Objectives: To be useful in development of clinical decision rules, clinical variables must demonstrate acceptable agreement when assessed by different observers. The objective was to determine the interobserver agreement in the assessment of historical and physical examination findings of children undergoing emergency department (ED) evaluation for blunt head trauma. Methods: This was a prospective cohort study of children younger than 18 years evaluated for blunt head trauma at one of 25 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Patients were excluded if injury occurred more than 24 hours prior to evaluation, if neuroimaging was obtained at another hospital prior to evaluation, or if the patient had a clinically trivial mechanism of injury. Two clinicians independently completed a standardized clinical assessment on a templated data form. Assessments were performed within 60 minutes of each other and prior to clinician review of any neuroimaging (if obtained). Agreement between the two observers beyond that expected by chance was calculated for each clinical variable, using the kappa (j) statistic for categorical variables and weighted kappa for ordinal variables. Variables with a lower 95% confidence limit (LCL) of κ > 0.4 were considered to have acceptable agreement. Results: Fifteen-hundred pairs of observations were obtained. Acceptable agreement was achieved in 27 of the 32 variables studied (84%). Mechanism of injury (low, medium, or high risk) had κ = 0.83. For subjective symptoms, kappa ranged from 0.47 (dizziness) to 0.93 (frequency of vomiting); all had 95% LCL > 0.4. Of the physical examination findings, kappa ranged from 0.22 (agitated) to 0.89 (Glasgow Coma Scale [GCS] score). The 95% LCL for kappa was <0.4 for four individual signs of altered mental status and for quality (i.e., boggy or firm) of scalp hematoma if present. Conclusions: Both subjective and objective clinical variables in children with blunt head trauma can be assessed by different observers with acceptable agreement, making these variables suitable candidates for clinical decision rules.

AB - Objectives: To be useful in development of clinical decision rules, clinical variables must demonstrate acceptable agreement when assessed by different observers. The objective was to determine the interobserver agreement in the assessment of historical and physical examination findings of children undergoing emergency department (ED) evaluation for blunt head trauma. Methods: This was a prospective cohort study of children younger than 18 years evaluated for blunt head trauma at one of 25 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Patients were excluded if injury occurred more than 24 hours prior to evaluation, if neuroimaging was obtained at another hospital prior to evaluation, or if the patient had a clinically trivial mechanism of injury. Two clinicians independently completed a standardized clinical assessment on a templated data form. Assessments were performed within 60 minutes of each other and prior to clinician review of any neuroimaging (if obtained). Agreement between the two observers beyond that expected by chance was calculated for each clinical variable, using the kappa (j) statistic for categorical variables and weighted kappa for ordinal variables. Variables with a lower 95% confidence limit (LCL) of κ > 0.4 were considered to have acceptable agreement. Results: Fifteen-hundred pairs of observations were obtained. Acceptable agreement was achieved in 27 of the 32 variables studied (84%). Mechanism of injury (low, medium, or high risk) had κ = 0.83. For subjective symptoms, kappa ranged from 0.47 (dizziness) to 0.93 (frequency of vomiting); all had 95% LCL > 0.4. Of the physical examination findings, kappa ranged from 0.22 (agitated) to 0.89 (Glasgow Coma Scale [GCS] score). The 95% LCL for kappa was <0.4 for four individual signs of altered mental status and for quality (i.e., boggy or firm) of scalp hematoma if present. Conclusions: Both subjective and objective clinical variables in children with blunt head trauma can be assessed by different observers with acceptable agreement, making these variables suitable candidates for clinical decision rules.

KW - Craniocerebral trauma

KW - Observer variation

KW - Physical examination

KW - Reproducibility of results

UR - http://www.scopus.com/inward/record.url?scp=56849093439&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=56849093439&partnerID=8YFLogxK

U2 - 10.1111/j.1553-2712.2008.00206.x

DO - 10.1111/j.1553-2712.2008.00206.x

M3 - Article

C2 - 19244631

AN - SCOPUS:56849093439

VL - 15

SP - 812

EP - 818

JO - Academic Emergency Medicine

JF - Academic Emergency Medicine

SN - 1069-6563

IS - 9

ER -