The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention

David S. Mandell, Aubyn Stahmer, Sujie Shin, Ming Xie, Erica Reisinger, Steven C. Marcus

Research output: Contribution to journalArticle

68 Citations (Scopus)

Abstract

This randomized field trial comparing Strategies for Teaching based on Autism Research and Structured Teaching enrolled educators in 33 kindergarten-through-second-grade autism support classrooms and 119 students, aged 5-8 years in the School District of Philadelphia. Students were assessed at the beginning and end of the academic year using the Differential Ability Scales. Program fidelity was measured through video coding and use of a checklist. Outcomes were assessed using linear regression with random effects for classroom and student. Average fidelity was 57% in Strategies for Teaching based on Autism Research classrooms and 48% in Structured Teaching classrooms. There was a 9.2-point (standard deviation = 9.6) increase in Differential Ability Scales score over the 8-month study period, but no main effect of program. There was a significant interaction between fidelity and group. In classrooms with either low or high program fidelity, students in Strategies for Teaching based on Autism Research experienced a greater gain in Differential Ability Scales score than students in Structured Teaching (11.2 vs 5.5 points and 11.3 vs 8.9 points, respectively). In classrooms with moderate fidelity, students in Structured Teaching experienced a greater gain than students in Strategies for Teaching based on Autism Research (10.1 vs 4.4 points). The results suggest significant variability in implementation of evidence-based practices, even with supports, and also suggest the need to address challenging issues related to implementation measurement in community settings.

Original languageEnglish (US)
Pages (from-to)281-295
Number of pages15
JournalAutism
Volume17
Issue number3
DOIs
StatePublished - May 1 2013

Fingerprint

Autistic Disorder
Teaching
Students
Aptitude
Therapeutics
Research
Evidence-Based Practice
Checklist
Linear Models

Keywords

  • autism
  • fidelity
  • implementation science
  • randomized trials
  • school-based intervention

ASJC Scopus subject areas

  • Developmental and Educational Psychology

Cite this

The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. / Mandell, David S.; Stahmer, Aubyn; Shin, Sujie; Xie, Ming; Reisinger, Erica; Marcus, Steven C.

In: Autism, Vol. 17, No. 3, 01.05.2013, p. 281-295.

Research output: Contribution to journalArticle

Mandell, David S. ; Stahmer, Aubyn ; Shin, Sujie ; Xie, Ming ; Reisinger, Erica ; Marcus, Steven C. / The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. In: Autism. 2013 ; Vol. 17, No. 3. pp. 281-295.
@article{1e4cc0b9104a484d83aabf778eb39352,
title = "The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention",
abstract = "This randomized field trial comparing Strategies for Teaching based on Autism Research and Structured Teaching enrolled educators in 33 kindergarten-through-second-grade autism support classrooms and 119 students, aged 5-8 years in the School District of Philadelphia. Students were assessed at the beginning and end of the academic year using the Differential Ability Scales. Program fidelity was measured through video coding and use of a checklist. Outcomes were assessed using linear regression with random effects for classroom and student. Average fidelity was 57{\%} in Strategies for Teaching based on Autism Research classrooms and 48{\%} in Structured Teaching classrooms. There was a 9.2-point (standard deviation = 9.6) increase in Differential Ability Scales score over the 8-month study period, but no main effect of program. There was a significant interaction between fidelity and group. In classrooms with either low or high program fidelity, students in Strategies for Teaching based on Autism Research experienced a greater gain in Differential Ability Scales score than students in Structured Teaching (11.2 vs 5.5 points and 11.3 vs 8.9 points, respectively). In classrooms with moderate fidelity, students in Structured Teaching experienced a greater gain than students in Strategies for Teaching based on Autism Research (10.1 vs 4.4 points). The results suggest significant variability in implementation of evidence-based practices, even with supports, and also suggest the need to address challenging issues related to implementation measurement in community settings.",
keywords = "autism, fidelity, implementation science, randomized trials, school-based intervention",
author = "Mandell, {David S.} and Aubyn Stahmer and Sujie Shin and Ming Xie and Erica Reisinger and Marcus, {Steven C.}",
year = "2013",
month = "5",
day = "1",
doi = "10.1177/1362361312473666",
language = "English (US)",
volume = "17",
pages = "281--295",
journal = "Autism",
issn = "1362-3613",
publisher = "SAGE Publications Ltd",
number = "3",

}

TY - JOUR

T1 - The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention

AU - Mandell, David S.

AU - Stahmer, Aubyn

AU - Shin, Sujie

AU - Xie, Ming

AU - Reisinger, Erica

AU - Marcus, Steven C.

PY - 2013/5/1

Y1 - 2013/5/1

N2 - This randomized field trial comparing Strategies for Teaching based on Autism Research and Structured Teaching enrolled educators in 33 kindergarten-through-second-grade autism support classrooms and 119 students, aged 5-8 years in the School District of Philadelphia. Students were assessed at the beginning and end of the academic year using the Differential Ability Scales. Program fidelity was measured through video coding and use of a checklist. Outcomes were assessed using linear regression with random effects for classroom and student. Average fidelity was 57% in Strategies for Teaching based on Autism Research classrooms and 48% in Structured Teaching classrooms. There was a 9.2-point (standard deviation = 9.6) increase in Differential Ability Scales score over the 8-month study period, but no main effect of program. There was a significant interaction between fidelity and group. In classrooms with either low or high program fidelity, students in Strategies for Teaching based on Autism Research experienced a greater gain in Differential Ability Scales score than students in Structured Teaching (11.2 vs 5.5 points and 11.3 vs 8.9 points, respectively). In classrooms with moderate fidelity, students in Structured Teaching experienced a greater gain than students in Strategies for Teaching based on Autism Research (10.1 vs 4.4 points). The results suggest significant variability in implementation of evidence-based practices, even with supports, and also suggest the need to address challenging issues related to implementation measurement in community settings.

AB - This randomized field trial comparing Strategies for Teaching based on Autism Research and Structured Teaching enrolled educators in 33 kindergarten-through-second-grade autism support classrooms and 119 students, aged 5-8 years in the School District of Philadelphia. Students were assessed at the beginning and end of the academic year using the Differential Ability Scales. Program fidelity was measured through video coding and use of a checklist. Outcomes were assessed using linear regression with random effects for classroom and student. Average fidelity was 57% in Strategies for Teaching based on Autism Research classrooms and 48% in Structured Teaching classrooms. There was a 9.2-point (standard deviation = 9.6) increase in Differential Ability Scales score over the 8-month study period, but no main effect of program. There was a significant interaction between fidelity and group. In classrooms with either low or high program fidelity, students in Strategies for Teaching based on Autism Research experienced a greater gain in Differential Ability Scales score than students in Structured Teaching (11.2 vs 5.5 points and 11.3 vs 8.9 points, respectively). In classrooms with moderate fidelity, students in Structured Teaching experienced a greater gain than students in Strategies for Teaching based on Autism Research (10.1 vs 4.4 points). The results suggest significant variability in implementation of evidence-based practices, even with supports, and also suggest the need to address challenging issues related to implementation measurement in community settings.

KW - autism

KW - fidelity

KW - implementation science

KW - randomized trials

KW - school-based intervention

UR - http://www.scopus.com/inward/record.url?scp=84878070842&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84878070842&partnerID=8YFLogxK

U2 - 10.1177/1362361312473666

DO - 10.1177/1362361312473666

M3 - Article

C2 - 23592849

AN - SCOPUS:84878070842

VL - 17

SP - 281

EP - 295

JO - Autism

JF - Autism

SN - 1362-3613

IS - 3

ER -