Implementing statistical methods for generalizing randomized trial findings to a target population

Benjamin Ackerman, Ian Schmid, Kara Rudolph, Marissa J. Seamans, Ryoko Susukida, Ramin Mojtabai, Elizabeth A. Stuart

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Randomized trials are considered the gold standard for assessing the causal effects of a drug or intervention in a study population, and their results are often utilized in the formulation of health policy. However, there is growing concern that results from trials do not necessarily generalize well to their respective target populations, in which policies are enacted, due to substantial demographic differences between study and target populations. In trials related to substance use disorders (SUDs), especially, strict exclusion criteria make it challenging to obtain study samples that are fully “representative” of the populations that policymakers may wish to generalize their results to. In this paper, we provide an overview of post-trial statistical methods for assessing and improving upon the generalizability of a randomized trial to a well-defined target population. We then illustrate the different methods using a randomized trial related to methamphetamine dependence and a target population of substance abuse treatment seekers, and provide software to implement the methods in R using the “generalize” package. We discuss several practical considerations for researchers who wish to utilize these tools, such as the importance of acquiring population-level data to represent the target population of interest, and the challenges of data harmonization.

Original languageEnglish (US)
JournalAddictive Behaviors
DOIs
StateAccepted/In press - Jan 1 2018

Fingerprint

Methamphetamine
Health Services Needs and Demand
Statistical methods
Health
Pharmaceutical Preparations
Substance-Related Disorders
Population
Health Policy
Software
Research Personnel
Demography

Keywords

  • External validity
  • Generalizability
  • Randomized trials
  • Statistics

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Clinical Psychology
  • Toxicology
  • Psychiatry and Mental health

Cite this

Implementing statistical methods for generalizing randomized trial findings to a target population. / Ackerman, Benjamin; Schmid, Ian; Rudolph, Kara; Seamans, Marissa J.; Susukida, Ryoko; Mojtabai, Ramin; Stuart, Elizabeth A.

In: Addictive Behaviors, 01.01.2018.

Research output: Contribution to journalArticle

Ackerman, Benjamin ; Schmid, Ian ; Rudolph, Kara ; Seamans, Marissa J. ; Susukida, Ryoko ; Mojtabai, Ramin ; Stuart, Elizabeth A. / Implementing statistical methods for generalizing randomized trial findings to a target population. In: Addictive Behaviors. 2018.
@article{5d0e276a2fd8485ca29bb96073256ecd,
title = "Implementing statistical methods for generalizing randomized trial findings to a target population",
abstract = "Randomized trials are considered the gold standard for assessing the causal effects of a drug or intervention in a study population, and their results are often utilized in the formulation of health policy. However, there is growing concern that results from trials do not necessarily generalize well to their respective target populations, in which policies are enacted, due to substantial demographic differences between study and target populations. In trials related to substance use disorders (SUDs), especially, strict exclusion criteria make it challenging to obtain study samples that are fully “representative” of the populations that policymakers may wish to generalize their results to. In this paper, we provide an overview of post-trial statistical methods for assessing and improving upon the generalizability of a randomized trial to a well-defined target population. We then illustrate the different methods using a randomized trial related to methamphetamine dependence and a target population of substance abuse treatment seekers, and provide software to implement the methods in R using the “generalize” package. We discuss several practical considerations for researchers who wish to utilize these tools, such as the importance of acquiring population-level data to represent the target population of interest, and the challenges of data harmonization.",
keywords = "External validity, Generalizability, Randomized trials, Statistics",
author = "Benjamin Ackerman and Ian Schmid and Kara Rudolph and Seamans, {Marissa J.} and Ryoko Susukida and Ramin Mojtabai and Stuart, {Elizabeth A.}",
year = "2018",
month = "1",
day = "1",
doi = "10.1016/j.addbeh.2018.10.033",
language = "English (US)",
journal = "Addictive Behaviors",
issn = "0306-4603",
publisher = "Elsevier Limited",

}

TY - JOUR

T1 - Implementing statistical methods for generalizing randomized trial findings to a target population

AU - Ackerman, Benjamin

AU - Schmid, Ian

AU - Rudolph, Kara

AU - Seamans, Marissa J.

AU - Susukida, Ryoko

AU - Mojtabai, Ramin

AU - Stuart, Elizabeth A.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Randomized trials are considered the gold standard for assessing the causal effects of a drug or intervention in a study population, and their results are often utilized in the formulation of health policy. However, there is growing concern that results from trials do not necessarily generalize well to their respective target populations, in which policies are enacted, due to substantial demographic differences between study and target populations. In trials related to substance use disorders (SUDs), especially, strict exclusion criteria make it challenging to obtain study samples that are fully “representative” of the populations that policymakers may wish to generalize their results to. In this paper, we provide an overview of post-trial statistical methods for assessing and improving upon the generalizability of a randomized trial to a well-defined target population. We then illustrate the different methods using a randomized trial related to methamphetamine dependence and a target population of substance abuse treatment seekers, and provide software to implement the methods in R using the “generalize” package. We discuss several practical considerations for researchers who wish to utilize these tools, such as the importance of acquiring population-level data to represent the target population of interest, and the challenges of data harmonization.

AB - Randomized trials are considered the gold standard for assessing the causal effects of a drug or intervention in a study population, and their results are often utilized in the formulation of health policy. However, there is growing concern that results from trials do not necessarily generalize well to their respective target populations, in which policies are enacted, due to substantial demographic differences between study and target populations. In trials related to substance use disorders (SUDs), especially, strict exclusion criteria make it challenging to obtain study samples that are fully “representative” of the populations that policymakers may wish to generalize their results to. In this paper, we provide an overview of post-trial statistical methods for assessing and improving upon the generalizability of a randomized trial to a well-defined target population. We then illustrate the different methods using a randomized trial related to methamphetamine dependence and a target population of substance abuse treatment seekers, and provide software to implement the methods in R using the “generalize” package. We discuss several practical considerations for researchers who wish to utilize these tools, such as the importance of acquiring population-level data to represent the target population of interest, and the challenges of data harmonization.

KW - External validity

KW - Generalizability

KW - Randomized trials

KW - Statistics

UR - http://www.scopus.com/inward/record.url?scp=85056205792&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056205792&partnerID=8YFLogxK

U2 - 10.1016/j.addbeh.2018.10.033

DO - 10.1016/j.addbeh.2018.10.033

M3 - Article

JO - Addictive Behaviors

JF - Addictive Behaviors

SN - 0306-4603

ER -