From plenary to practice: A large language model (LLM)–based thematic analysis of landmark clinical trial discussions and factors influencing their real-world adoption.

Abstract

1583 Background: Landmark trials define standard-of-care (SOC), yet the translation of trial evidence into routine practice remains poorly characterized. LLMs enable the systematic extraction of nuanced clinical discourse from unstructured EHR data previously infeasible at scale. We performed an LLM-based thematic analysis of past ASCO Plenary Session (PS) trial discussions to better understand the nature of shared decision-making in routine care and characterize themes influencing SOC adoption. Methods: We designed a retrospective study of clinician-documented discussions with patients about PS trials presented at ASCO Annual Meetings 2021-2025. 2 trials (negative study; non-therapeutic trial) were excluded. Patient records from the Flatiron Health Database with mention of a relevant trial/NCT number within 2 years of presentation were selected. We prompted an LLM (Claude 4.5 Sonnet) to summarize documentation of clinician-patient trial discussions and extract: 1) context of discussion, 2) clinician sentiment toward trial (positive, neutral, negative) and 3) documented receipt of trial therapy (yes/no; if no, reason why not). To characterize the gap in SOC adoption, a thematic analysis was performed using a multi-model consensus approach (Gemini 2.5pro, GPT-4o, Claude 4.5 Sonnet) to identify EHR-documented discussion themes associated with trial treatment omission. LLMs provided supporting evidence for all answers; a human-in-the-loop approach was used to verify concordance between LLM-extracted themes and expert clinical interpretation. LLMs were hosted on Flatiron Health private servers and HIPAA compliant. Results: The study included 8650 patients and 21 trials discussed across 15 cancer types. Common discussion topics were: efficacy (78%), eligibility (78%), patient education (63%), and toxicity (28%). Clinician sentiment across studies was favorable (47%-93% positive), yet real-world trial therapy initiation occurred in only 64% of cases. Main themes associated with trial treatment omission were: clinical factors (biomarker ineligibility, comorbidities), systemic barriers (insurance, transportation, incarceration), and patient preferences (prioritizing quality of life, preserving life roles). Conclusions: This is the largest thematic analysis of clinician-patient trial discussions to date. We uncovered rich qualitative insights into how clinicians engage in shared decision-making with their patients, balancing positive evidence and sentiment with practical and patient-centered factors that define individualized care. Novel LLM methods can transform clinician-patient narratives into actionable evidence to mitigate disparities and optimize real-world SOC adoption. Future work will examine thematic differences by cancer type and prevalence and correlate findings with outcomes.