The effects of using answer sheets on reported drug use and data quality in a classroom survey: A cluster-randomized study

Background We compare self-reported prevalence of drug use and indicators of data quality from two different response modes (with and without an independent answer sheet for recording responses) in a survey conducted in 2015 among secondary school students. Methods Stratified cluster-randomized study conducted among students in grades 8–12 from public, private and subsidized schools in Chile (N = 2317 students in 122 classes). Measurements included were: percentage reporting substance use (tobacco, alcohol, marijuana, cocaine, ecstasy); number of inconsistent responses; number of item nonresponses; percentage of extreme reports of drug use; percentage reporting using the nonexistent drug, relevón; and completion times. Results Compared with those who responded directly in the questionnaire booklet, students who used a separate answer sheet took 17.6 more minutes (95% confidence interval [CI]: 14.4–20.8) to complete the survey and had on average 1.5 more inconsistent responses (95%CI: 0.91–2.14). The prevalence and variance of drug use was higher among those who used an answer sheet for all substances except tobacco; the prevalence ratio (PR) of reported substance use for low-prevalence substances during the past year were: cocaine PR = 2.5 (95%CI: 1.6–4.1); ecstasy PR = 5.0 (95%CI: 2.4–10.5); relevón PR = 4.8 (95%CI: 2.5–9.3). Conclusions Using an answer sheet for a self-administered paper-and-pencil survey of drug use among students result in lower quality data and higher reports of drug use. International comparison of adolescent drug use from school-based surveys should be done with caution. The relative ranking of a country could be misleading if different mode of recording answers are used.