Enhancing clinical trial screening with a comprehensive large language model platform.

Abstract

e13674 Background: Clinical trials are essential for improving cancer treatment; yet 80% of trials fail to meet enrollment timelines, often due to inefficient patient-trial matching processes. To address this, a large language model (LLM)-based clinical trial matching platform was designed to prioritize and evaluate trial matches and facilitate human review. It is systematically validated herein, and real-world utility is explored. Methods: The platform consists of three modules: 1.) trial relevance filters trials to the therapeutic area that best aligns with a patient’s overall profile; 2.) for relevant trials, criterion-level eligibility evaluates individual inclusion and exclusion criteria; and 3.) for each criterion, source evidence retrieval extracts supporting evidence from patient electronic health records (EHRs). Each component was tested independently, and the complete platform is being evaluated in a proof-of-concept study with Atrium Health (analyses 1-4). Datasets per analysis: 1.) Trial relevance was analyzed for 75,000 pairs of patient and clinical trial descriptions. This encompassed 185 patients (18 cancer, and to test generalizability, 167 non-cancer). 2.) Criterion-level eligibility : a.) 2,136 criterion-level eligibility decisions were labeled in triplicate, for 359 criteria across 6 cancer patient EHRs from HealthVerity (HV dataset). b.) Generalizability testing leveraged the n2c2 2018 cohort selection dataset. This included adjudication for 13 criteria across 288 diabetes patients. 3.) Source evidence retrieval : on the HV dataset, experts assessed the accuracy of 2,500 pieces of EHR evidence retrieved by the system. 4.) Proof-of-concept : trial relevance and eligibility are being validated against real-world enrollment of 58 trials and 1053 cancer patients from Atrium Health. Results: Relevant trials were returned with sensitivities of 92% and 90% for cancer and non-cancer patients, respectively, and a specificity of 95% (analysis 1). For analysis 2a, the platform maintained criterion-level accuracy within the range of three experts (93%; experts: 92-97%), while selectively seeking human input in 12% of evaluations (experts: 1-3%). Criterion-level accuracy on the n2c2 dataset was 91%, and the system sought human input in 0.1% of evaluations (analysis 2b). For each criterion in the HV dataset, it retrieved source evidence with a mean accuracy of 94% (analysis 3). Proof-of-concept data is expected to be available in time for presentation. Conclusions: A clinical trial matching platform efficiently identified relevant trials and assessed criterion-level patient eligibility with expert accuracy. It reliably surfaced source evidence for improved provenance and interpretability, and these results will soon be validated on real-world cancer trial enrollment data. Such a system could alleviate staff burden, enhance trial efficiency, and democratize trial access for patients.