Detecting Anomalies Among Practice Sites Within Multicenter Trials

Seth A. Berkowitz, Kara Rudolph, Sanjay Basu

Research output: Contribution to journalArticle

Abstract

BACKGROUND: Recent multisite trials reveal striking heterogeneities in results between trial sites. These may be because of population differences indicating different treatment benefits among different types of participants or site anomalies, such as failures to adhere to study protocols that could negatively affect study validity. We sought to determine whether a new data analysis strategy-transportability methods-could suggest site anomalies not readily identified through standard methods. METHODS AND RESULTS: We applied transportability methods to 2 large, multicenter cardiovascular disease treatment trials: the TOPCAT trial (Treatment of Preserved Cardiac Function Heart Failure With an Aldosterone Antagonist; n=3445) comparing spironolactone to placebo for heart failure (for which site anomalies were suspected) and the ACCORD BP trial (Action to Control Cardiovascular Risk in Diabetes-Blood Pressure; n=4733) comparing intensive-to-standard blood pressure treatment (for which site anomalies were not suspected). The transportability methods give expected results by standardizing from one site to another using data on participant covariates. The difference between the expected and observed results was assessed using calibration tests to identify whether treatment-effect differences between sites could be explained by participant population characteristics. Standard regression methods did not detect heterogeneities in TOPCAT between Russia/Georgia study sites suspected of study protocol violations and sites in the Americas ( P=0.12 for difference in primary cardiovascular outcome; P=0.20 for difference in total mortality). The transportability methods, however, detected the difference between Russia/Georgia sites and sites in the Americas ( P<0.001) and found that measured participant characteristics did not explain the between-site discrepancies. The transport methods found no such discrepancies between sites in ACCORD BP, suggesting participant characteristics explained between-site differences. CONCLUSIONS: Transportability methods may be superior to standard approaches for detecting anomalies within multicenter randomized trials and assist data monitoring boards to determine whether important treatment-effect heterogeneities can be attributed to participant differences or potentially to site performance differences requiring further investigation.

Original languageEnglish (US)
Pages (from-to)e004907
JournalCirculation. Cardiovascular quality and outcomes
Volume12
Issue number3
DOIs
StatePublished - Mar 1 2019

Fingerprint

Multicenter Studies
Russia
Therapeutics
Heart Failure
Blood Pressure
Mineralocorticoid Receptor Antagonists
Spironolactone
Population Characteristics
Calibration
Cardiovascular Diseases
Placebos
Mortality
Population

Keywords

  • blood pressure
  • cardiovascular system
  • heart failure
  • methodology
  • reproducibility

ASJC Scopus subject areas

  • Cardiology and Cardiovascular Medicine

Cite this

Detecting Anomalies Among Practice Sites Within Multicenter Trials. / Berkowitz, Seth A.; Rudolph, Kara; Basu, Sanjay.

In: Circulation. Cardiovascular quality and outcomes, Vol. 12, No. 3, 01.03.2019, p. e004907.

Research output: Contribution to journalArticle

@article{21e2e0eeb1564074b90e41cb7b60c117,
title = "Detecting Anomalies Among Practice Sites Within Multicenter Trials",
abstract = "BACKGROUND: Recent multisite trials reveal striking heterogeneities in results between trial sites. These may be because of population differences indicating different treatment benefits among different types of participants or site anomalies, such as failures to adhere to study protocols that could negatively affect study validity. We sought to determine whether a new data analysis strategy-transportability methods-could suggest site anomalies not readily identified through standard methods. METHODS AND RESULTS: We applied transportability methods to 2 large, multicenter cardiovascular disease treatment trials: the TOPCAT trial (Treatment of Preserved Cardiac Function Heart Failure With an Aldosterone Antagonist; n=3445) comparing spironolactone to placebo for heart failure (for which site anomalies were suspected) and the ACCORD BP trial (Action to Control Cardiovascular Risk in Diabetes-Blood Pressure; n=4733) comparing intensive-to-standard blood pressure treatment (for which site anomalies were not suspected). The transportability methods give expected results by standardizing from one site to another using data on participant covariates. The difference between the expected and observed results was assessed using calibration tests to identify whether treatment-effect differences between sites could be explained by participant population characteristics. Standard regression methods did not detect heterogeneities in TOPCAT between Russia/Georgia study sites suspected of study protocol violations and sites in the Americas ( P=0.12 for difference in primary cardiovascular outcome; P=0.20 for difference in total mortality). The transportability methods, however, detected the difference between Russia/Georgia sites and sites in the Americas ( P<0.001) and found that measured participant characteristics did not explain the between-site discrepancies. The transport methods found no such discrepancies between sites in ACCORD BP, suggesting participant characteristics explained between-site differences. CONCLUSIONS: Transportability methods may be superior to standard approaches for detecting anomalies within multicenter randomized trials and assist data monitoring boards to determine whether important treatment-effect heterogeneities can be attributed to participant differences or potentially to site performance differences requiring further investigation.",
keywords = "blood pressure, cardiovascular system, heart failure, methodology, reproducibility",
author = "Berkowitz, {Seth A.} and Kara Rudolph and Sanjay Basu",
year = "2019",
month = "3",
day = "1",
doi = "10.1161/CIRCOUTCOMES.118.004907",
language = "English (US)",
volume = "12",
pages = "e004907",
journal = "Circulation: Cardiovascular Quality and Outcomes",
issn = "1941-7713",
publisher = "Lippincott Williams and Wilkins",
number = "3",

}

TY - JOUR

T1 - Detecting Anomalies Among Practice Sites Within Multicenter Trials

AU - Berkowitz, Seth A.

AU - Rudolph, Kara

AU - Basu, Sanjay

PY - 2019/3/1

Y1 - 2019/3/1

N2 - BACKGROUND: Recent multisite trials reveal striking heterogeneities in results between trial sites. These may be because of population differences indicating different treatment benefits among different types of participants or site anomalies, such as failures to adhere to study protocols that could negatively affect study validity. We sought to determine whether a new data analysis strategy-transportability methods-could suggest site anomalies not readily identified through standard methods. METHODS AND RESULTS: We applied transportability methods to 2 large, multicenter cardiovascular disease treatment trials: the TOPCAT trial (Treatment of Preserved Cardiac Function Heart Failure With an Aldosterone Antagonist; n=3445) comparing spironolactone to placebo for heart failure (for which site anomalies were suspected) and the ACCORD BP trial (Action to Control Cardiovascular Risk in Diabetes-Blood Pressure; n=4733) comparing intensive-to-standard blood pressure treatment (for which site anomalies were not suspected). The transportability methods give expected results by standardizing from one site to another using data on participant covariates. The difference between the expected and observed results was assessed using calibration tests to identify whether treatment-effect differences between sites could be explained by participant population characteristics. Standard regression methods did not detect heterogeneities in TOPCAT between Russia/Georgia study sites suspected of study protocol violations and sites in the Americas ( P=0.12 for difference in primary cardiovascular outcome; P=0.20 for difference in total mortality). The transportability methods, however, detected the difference between Russia/Georgia sites and sites in the Americas ( P<0.001) and found that measured participant characteristics did not explain the between-site discrepancies. The transport methods found no such discrepancies between sites in ACCORD BP, suggesting participant characteristics explained between-site differences. CONCLUSIONS: Transportability methods may be superior to standard approaches for detecting anomalies within multicenter randomized trials and assist data monitoring boards to determine whether important treatment-effect heterogeneities can be attributed to participant differences or potentially to site performance differences requiring further investigation.

AB - BACKGROUND: Recent multisite trials reveal striking heterogeneities in results between trial sites. These may be because of population differences indicating different treatment benefits among different types of participants or site anomalies, such as failures to adhere to study protocols that could negatively affect study validity. We sought to determine whether a new data analysis strategy-transportability methods-could suggest site anomalies not readily identified through standard methods. METHODS AND RESULTS: We applied transportability methods to 2 large, multicenter cardiovascular disease treatment trials: the TOPCAT trial (Treatment of Preserved Cardiac Function Heart Failure With an Aldosterone Antagonist; n=3445) comparing spironolactone to placebo for heart failure (for which site anomalies were suspected) and the ACCORD BP trial (Action to Control Cardiovascular Risk in Diabetes-Blood Pressure; n=4733) comparing intensive-to-standard blood pressure treatment (for which site anomalies were not suspected). The transportability methods give expected results by standardizing from one site to another using data on participant covariates. The difference between the expected and observed results was assessed using calibration tests to identify whether treatment-effect differences between sites could be explained by participant population characteristics. Standard regression methods did not detect heterogeneities in TOPCAT between Russia/Georgia study sites suspected of study protocol violations and sites in the Americas ( P=0.12 for difference in primary cardiovascular outcome; P=0.20 for difference in total mortality). The transportability methods, however, detected the difference between Russia/Georgia sites and sites in the Americas ( P<0.001) and found that measured participant characteristics did not explain the between-site discrepancies. The transport methods found no such discrepancies between sites in ACCORD BP, suggesting participant characteristics explained between-site differences. CONCLUSIONS: Transportability methods may be superior to standard approaches for detecting anomalies within multicenter randomized trials and assist data monitoring boards to determine whether important treatment-effect heterogeneities can be attributed to participant differences or potentially to site performance differences requiring further investigation.

KW - blood pressure

KW - cardiovascular system

KW - heart failure

KW - methodology

KW - reproducibility

UR - http://www.scopus.com/inward/record.url?scp=85062854708&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062854708&partnerID=8YFLogxK

U2 - 10.1161/CIRCOUTCOMES.118.004907

DO - 10.1161/CIRCOUTCOMES.118.004907

M3 - Article

C2 - 30857413

AN - SCOPUS:85062854708

VL - 12

SP - e004907

JO - Circulation: Cardiovascular Quality and Outcomes

JF - Circulation: Cardiovascular Quality and Outcomes

SN - 1941-7713

IS - 3

ER -