Lack of Agreement in Pediatric Emergency Department Discharge Diagnoses from Clinical and Administrative Data Sources

Background: Diagnosis information from existing data sources is used commonly for epidemiologic, administrative, and research purposes. The quality of such data for emergency department (ED) visits is unknown. Objectives: To determine the agreement on final diagnoses between two sources, electronic administrative sources and manually abstracted medical records, for pediatric ED visits, in a multicenter network. Methods: This was a cross sectional study at 19 EDs nationwide. The authors obtained data from two sources at each ED during a three-month period in 2003: administrative sources for all visits and abstracted records for randomly selected visits during ten days over the study period. Records were matched using unique identifiers and probabilistic linkage. The authors recorded up to three diagnoses from each abstracted medical record and up to ten for the administrative data source. Diagnoses were grouped into 104 groups using a modification of the Clinical Classification System. Results: A total of 8,860 abstracted records had at least one valid diagnosis code (with a total of 12,895 diagnoses) and were successfully matched to records in the administrative source. Overall, 67% (95% confidence interval = 66% to 68%) of diagnoses from the administrative and abstracted sources were within the same diagnosis group. Agreement varied by site, ranging from 54% to 77%. Agreement varied substantially by diagnosis group; there was no difference by method of linkage. Clustering clinically similar diagnosis groups improved agreement between administrative and abstracted data sources. Conclusions: ED diagnoses retrieved from electronic administrative sources and manual chart review frequently disagree, even if similar diagnosis codes are grouped. Agreement varies by institution and by diagnosis. Further work is needed to improve the accuracy of diagnosis coding; development of a grouping system specific to pediatric emergency care may be beneficial.