Improving clinical trial interpretability and efficiency: A Bayesian re-analysis of individual patient outcomes from 230 phase III oncology trials.

Abstract

11018 Background: The conventional interpretation of superiority oncology trials relies on statistical significance defined by P values, which are frequently misinterpreted and oversimplified. On the other hand, Bayesian analysis, which incorporates prior knowledge, directly estimates the probability of a hypothesis, with greater flexibility to examine clinically relevant effect sizes. Here, we studied the effects of Bayesian models on phase III trial interpretation. We further hypothesized that Bayesian approaches, using differential priors for efficacy and futility, would improve trial efficiency. Methods: Phase III superiority-design, two-arm oncology trials were screened from ClinicalTrials.gov for this meta-epidemiological study. Individual patient-level data were manually reconstructed from the Kaplan-Meir curves of the primary endpoint. First, Bayesian Cox regressions of reconstructed data applied skeptical N (0, 0.355), enthusiastic N (-0.41, 0.4), and neutral priors N (0, 10 6 ), where N (mean, standard deviation) denotes a normal distribution, to estimate posterior probabilities of the primary endpoint effects with Markov chain Monte Carlo sampling. Minimum clinically important differences (MCID) in the experimental arm were defined by HR < 0.8 per ASCO criteria (Ellis et al, J Clin Oncol 2014). Second, a single event or enrollment-driven interim analysis with Bayesian stopping rules was simulated 100 times for each trial using in silico models of randomly varying accrual kinetics with published patient outcomes. Interim efficacy and futility were defined per simulation by probabilities ≥ 85% for achieving effect sizes larger than the MCID/3 using a skeptical or enthusiastic prior, respectively. Early trial closure was recommended if ≥ 75% of simulations met efficacy or futility criteria. Results: After screening,194,129 patient outcomes from 230 trials were reconstructed. Overall survival was the primary endpoint in 90 trials (39%). All trials interpreted as positive had > 90% probabilities of marginal benefits (HR < 1). However, 38% of trials interpreted as positive had ≤ 90% probabilities of achieving the MCID (HR < 0.8), even under an enthusiastic prior. Conversely, 24% of trials interpreted as negative had > 90% probability of achieving marginal benefits, even under a skeptical prior. In the interim analysis simulations, early closure was recommended for 82 trials (36%). Bayesian interim analysis was associated with >99% probability of reducing enrollment sizes. The trial and its simulated interim analysis remained concordant (Bayesian Cohen’s κ, 0.95). Conclusions: Bayesian models add unique interpretative value for clinically relevant effects, and may improve trial efficiency without compromising trial interpretation. Bayesian models should be increasingly incorporated in phase III trials.