Abstract 4366827: Large Language Models Detect Ventricular Tachycardia Recurrence in Clinical Notes and Enable Prediction of Clinical Outcomes at Scale

Abstract

Introduction: Predicting recurrence in patients with Ventricular Tachycardia (VT) remains challenging, and manual screens of clinical notes are laborious. Advances in natural language processing, particularly Large Language Models (LLMs), offer potential solutions for automating VT detection from notes on large scale, and correlating clinical outcomes. Hypothesis: LLMs can detect VT recurrence events - as opposed to baseline VT - from clinical notes with accuracy comparable to physicians, and LLM-based detection can facilitate large-scale outcome prediction. Methods: A VT ablation registry of clinical notes of N=362 patients (20,303 Notes, 25.2% female, mean age 58.6±14.0 years) was examined. A development cohort of 100 notes was independently annotated by 3 board-certified physician reviewers for reference. Using a HIPAA compliant GPT-4o, we evaluated 4 prompt variations in this subset against physician annotations, then applied the optimal prompt to the full 20,203 notes. Baseline demographics were assessed at time of VT diagnosis, and quarterly use of AADs 3 years post-VT was assessed. Predictors of first VT recurrence were evaluated using multivariate Cox regression with time-varying medication use. Results: Inter-rater reliability in the development cohort was substantial with a Cohen's kappa of 0.74. The optimal prompt achieved an accuracy of 92%, sensitivity of 94%, specificity of 90%, positive predictive value 90%, negative predictive value 94%, and F1 score of 92%. Applying the optimized prompt to the entire cohort, we identified a total of 232 of patients with first VT recurrence 515±889 days from diagnosis. Using LLM-generated detection of VT recurrence, Cox regression revealed that the strongest predictor of VT recurrence was time-varying exposure to class I (hazard ratio [HR] 2.42, 95% CI 1.55-3.76, p<0.001), class II (HR 3.01, 95%CI 2.00-4.52, p<0.001), class III (HR 4.18, 95%CI 2.52-6.94, p<0.001), and amiodarone (HR 2.06, 95%CI 1.42-2.99, p<0.001). A significant negative interaction between class III agents and heart failure was found, indicating a lower-risk heart failure subgroup of patients on class IIIs. Conclusions: LLM can enable the study of VT recurrence and associated factors at scale. Anti-arrhythmics were predictors of recurrence with evidence of effect modification considering co-morbidity in a large registry. This approach may enable targeted interventions and personalized patient management strategies to improve clinical outcomes.