Type II error and statistical power in reports of small animal clinical trials

Research output: Contribution to journalArticle

22 Citations (Scopus)

Abstract

Objective-To describe reporting of key methodological elements associated with type II error in published reports of small animal randomized controlled trials (RCTs) and to determine the statistical power in a subset of RCTs with negative results. Design-Descriptive literature survey. Sample-Reports of parallel-group clinical RCTs published in 11 English-language veterinary journals from 2005 to 2012. Procedures-Predefined criteria were used to identify trial primary outcomes and classify results as negative or positive. Details of sample size determination and use of confidence intervals in results reporting were recorded. For each 2-group RCT with negative results, the statistical power to detect 25% and 50% relative differences in outcome was calculated. Results-Of 238 RCTs, 42 (18%) stated a primary outcome, 52 (22%) reported a sample size calculation, and 18 (9%) included a confidence interval around the observed treatment effect. Reports of only 2 (0.8%) RCTs included all 3 elements. Among 103 two-group RCTs with negative results, only 14 (14%) and 40 (39%) were sufficiently powered (β < 0.20) to detect 25% and 50% relative differences in outcome between treatments, respectively. Conclusions and Clinical Relevance-The present survey found that small animal RCTs with negative results were often underpowered to detect moderate-to-large effect sizes between study groups. Information needed for critical appraisal was missing from most reports. The potential for clinicians to base treatment decisions on inappropriate interpretations of RCTs was worrisome. Design and reporting of small animal RCTs must be improved.

Original languageEnglish (US)
Pages (from-to)1075-1080
Number of pages6
JournalJournal of the American Veterinary Medical Association
Volume244
Issue number9
DOIs
StatePublished - Jan 1 2014
Externally publishedYes

Fingerprint

clinical trials
Randomized Controlled Trials
Clinical Trials
confidence interval
animals
randomized clinical trials
sampling
Sample Size
Confidence Intervals
Language

ASJC Scopus subject areas

  • veterinary(all)

Cite this

Type II error and statistical power in reports of small animal clinical trials. / Giuffrida, Michelle.

In: Journal of the American Veterinary Medical Association, Vol. 244, No. 9, 01.01.2014, p. 1075-1080.

Research output: Contribution to journalArticle

@article{06b21b31a8bd4e929cd36e9d9d1dffe8,
title = "Type II error and statistical power in reports of small animal clinical trials",
abstract = "Objective-To describe reporting of key methodological elements associated with type II error in published reports of small animal randomized controlled trials (RCTs) and to determine the statistical power in a subset of RCTs with negative results. Design-Descriptive literature survey. Sample-Reports of parallel-group clinical RCTs published in 11 English-language veterinary journals from 2005 to 2012. Procedures-Predefined criteria were used to identify trial primary outcomes and classify results as negative or positive. Details of sample size determination and use of confidence intervals in results reporting were recorded. For each 2-group RCT with negative results, the statistical power to detect 25{\%} and 50{\%} relative differences in outcome was calculated. Results-Of 238 RCTs, 42 (18{\%}) stated a primary outcome, 52 (22{\%}) reported a sample size calculation, and 18 (9{\%}) included a confidence interval around the observed treatment effect. Reports of only 2 (0.8{\%}) RCTs included all 3 elements. Among 103 two-group RCTs with negative results, only 14 (14{\%}) and 40 (39{\%}) were sufficiently powered (β < 0.20) to detect 25{\%} and 50{\%} relative differences in outcome between treatments, respectively. Conclusions and Clinical Relevance-The present survey found that small animal RCTs with negative results were often underpowered to detect moderate-to-large effect sizes between study groups. Information needed for critical appraisal was missing from most reports. The potential for clinicians to base treatment decisions on inappropriate interpretations of RCTs was worrisome. Design and reporting of small animal RCTs must be improved.",
author = "Michelle Giuffrida",
year = "2014",
month = "1",
day = "1",
doi = "10.2460/javma.244.9.1075",
language = "English (US)",
volume = "244",
pages = "1075--1080",
journal = "Journal of the American Veterinary Medical Association",
issn = "0003-1488",
publisher = "American Veterinary Medical Association",
number = "9",

}

TY - JOUR

T1 - Type II error and statistical power in reports of small animal clinical trials

AU - Giuffrida, Michelle

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Objective-To describe reporting of key methodological elements associated with type II error in published reports of small animal randomized controlled trials (RCTs) and to determine the statistical power in a subset of RCTs with negative results. Design-Descriptive literature survey. Sample-Reports of parallel-group clinical RCTs published in 11 English-language veterinary journals from 2005 to 2012. Procedures-Predefined criteria were used to identify trial primary outcomes and classify results as negative or positive. Details of sample size determination and use of confidence intervals in results reporting were recorded. For each 2-group RCT with negative results, the statistical power to detect 25% and 50% relative differences in outcome was calculated. Results-Of 238 RCTs, 42 (18%) stated a primary outcome, 52 (22%) reported a sample size calculation, and 18 (9%) included a confidence interval around the observed treatment effect. Reports of only 2 (0.8%) RCTs included all 3 elements. Among 103 two-group RCTs with negative results, only 14 (14%) and 40 (39%) were sufficiently powered (β < 0.20) to detect 25% and 50% relative differences in outcome between treatments, respectively. Conclusions and Clinical Relevance-The present survey found that small animal RCTs with negative results were often underpowered to detect moderate-to-large effect sizes between study groups. Information needed for critical appraisal was missing from most reports. The potential for clinicians to base treatment decisions on inappropriate interpretations of RCTs was worrisome. Design and reporting of small animal RCTs must be improved.

AB - Objective-To describe reporting of key methodological elements associated with type II error in published reports of small animal randomized controlled trials (RCTs) and to determine the statistical power in a subset of RCTs with negative results. Design-Descriptive literature survey. Sample-Reports of parallel-group clinical RCTs published in 11 English-language veterinary journals from 2005 to 2012. Procedures-Predefined criteria were used to identify trial primary outcomes and classify results as negative or positive. Details of sample size determination and use of confidence intervals in results reporting were recorded. For each 2-group RCT with negative results, the statistical power to detect 25% and 50% relative differences in outcome was calculated. Results-Of 238 RCTs, 42 (18%) stated a primary outcome, 52 (22%) reported a sample size calculation, and 18 (9%) included a confidence interval around the observed treatment effect. Reports of only 2 (0.8%) RCTs included all 3 elements. Among 103 two-group RCTs with negative results, only 14 (14%) and 40 (39%) were sufficiently powered (β < 0.20) to detect 25% and 50% relative differences in outcome between treatments, respectively. Conclusions and Clinical Relevance-The present survey found that small animal RCTs with negative results were often underpowered to detect moderate-to-large effect sizes between study groups. Information needed for critical appraisal was missing from most reports. The potential for clinicians to base treatment decisions on inappropriate interpretations of RCTs was worrisome. Design and reporting of small animal RCTs must be improved.

UR - http://www.scopus.com/inward/record.url?scp=84898869527&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898869527&partnerID=8YFLogxK

U2 - 10.2460/javma.244.9.1075

DO - 10.2460/javma.244.9.1075

M3 - Article

C2 - 24739118

AN - SCOPUS:84898869527

VL - 244

SP - 1075

EP - 1080

JO - Journal of the American Veterinary Medical Association

JF - Journal of the American Veterinary Medical Association

SN - 0003-1488

IS - 9

ER -