Statistical Errors in Current Phase III Oncology Trials: Are We Managing Them Appropriately?

Changyu Shen PhD
Associate Professor of Medicine, Harvard Medical School; Statistical Director of the Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Beth Israel Deaconess Medical Center

Abstract

The last decade saw a strong acceleration in the development of novel cancer therapies. Nonetheless, many positive trials showed limited survival benefit, and the success rate of phase III oncology trials still falls behind that of other therapeutic areas. It is critical to understand the performance of the current statistical testing procedures and how we can improve. We collected the summary efficacy data for the primary endpoints of overall survival (OS) and progression-free survival (PFS) for industry-sponsored randomized superiority trials in oncology, completed between 2008 and 2017 with at least one site in the United States. An empirical Bayes method was used to derive the probability of an experimental therapy achieving different levels of effect size. We found that the statistical testing procedure in contemporary phase III superiority trials in oncology detected essentially all OS and PFS endpoints that achieved the hypothesized effect size. However, this happens at the cost of a relatively high number of false positives, in which experimental therapies with statistical significance did not achieve the hypothesized effect size. Alternative testing procedures could reduce the number of false positive therapies but would lead to increase in the number of negative trials. Instead of adjusting testing procedure in phase III trials, applying more stringent criteria to phase II trials is a more effective and realistic strategy to reduce the number of false positives phase III trials.

Required reading: Research not yet published.