Adaptive AI for Cardiovascular Event Adjudication: Cardiovascular Event Adjudication Across Different Definitions in the ODYSSEY OUTCOMES and EUCLID Trials

Abstract

BACKGROUND: Clinical end point classification (CEC) is the gold standard for cardiovascular end point measurement in clinical trials but adds time and cost. We developed and validated an artificial intelligence (AI) algorithm (adaptive AI for CEC [ADAPT-CEC]) that adjudicates multiple cardiovascular end points and adapts to new definitions. METHODS: ADAPT-CEC was derived from myocardial infarction (MI), stroke, and heart failure from the ODYSSEY OUTCOMES (Evaluation of Cardiovascular Outcomes After an Acute Coronary Syndrome During Treatment With Alirocumab) trial and externally validated on MI, stroke, bleeding and cardiovascular (CV) death from the EUCLID (Examining Use of Ticagrelor in Peripheral Artery Disease) trial after adaptation with 20 EUCLID suspected events per end point. ADAPT-CEC was compared via F1 score with direct Generative Pretrained Transformer (GPT) 4o adjudication and a hybrid approach where the 30% of suspected events with the lowest AI prediction certainty used human adjudication. The EUCLID primary end point of CV death, MI, or stroke was re-estimated for all 3 adjudication strategies. RESULTS: Among 13 885 suspected EUCLID primary end point events, ADAPT-CEC, hybrid, and GPT 4o strategies correctly classified 86.4%, 95.6%, and 76.3% of all end points and 99.4%, 99.6%, and 99.8% of all non–end points compared with human adjudication, respectively. Hybrid adjudication F1 metrics were the highest (CV death, 0.94 [95% CI, 0.92–0.96]; MI, 0.80 [95% CI, 0.77–0.82]; stroke, 0.82 [95% CI, 0.78–0.86]; bleeding, 0.83 [95% CI, 0.82–0.85]). ADAPT-CEC F1 metrics were lower for CV death, MI, and stroke but similar to GPT 4o, while bleeding (0.78 [95% CI, 0.77–0.79]) was superior to GPT 4o. The EUCLID primary treatment effect was similar by human adjudication (hazard ratio [HR], 1.02 [95% CI, 0.93–1.13]); hybrid (HR, 1.04 [95% CI, 0.94–1.15]); ADAPT-CEC (HR, 0.98 [95% CI, 0.88–1.09]), and GPT 4.0 (HR, 1.06 [95% CI, 0.95–1.19]) adjudication. CONCLUSIONS: After brief adaptation, a single trial–derived AI algorithm can adjudicate similar (MI and stroke) and new end points (CV death and bleeding) in a second trial and replicate the EUCLID primary outcome treatment effect. A hybrid approach with humans adjudicating those suspected events with the lowest 30% of ADAPT-CEC prediction certainty was superior to ADAPT-CEC alone or GPT 4o alone and replicated the EUCLID primary outcome treatment effect. Prospective studies of adaptive AI adjudication are needed to determine future trial implementation.