Improving availability and quality of serious illness documentation using large language models.

Abstract

e13693 Background: Timely access to prior documented serious illness conversations (SICs) is critical for inpatient clinicians caring for patients with advanced cancer whom they may be unfamiliar with. Despite widespread implementation of structured electronic health records (EHR) modules intended to capture SICs, structured documentation remains sparse, with most SIC content embedded within large volumes of free-text clinical notes. We evaluated whether large language models (LLMs) could synthesize longitudinal clinical notes into actionable SIC summaries. Methods: The BRIDGE-SIC pilot clinical trial enrolled 58 patients with solid tumor admitted to an academic hospital with a predicted 90-day mortality ≥40%. For each patient, HIPAA-compliant LLM prompts identified and summarized SIC content from the prior 6 months of EHR records for 5 predefined SIC domains (illness understanding, prognosis, hopes, worries, and social supports). In this post-hoc analysis, Epic SIC module content was reviewed, and their completeness, word count, and date of last update were compared to those of the LLM summaries. A qualitative comparison was also conducted. Results: The LLM summaries identified SIC documentation within free-text clinical notes for 4 of 5 SIC domains for all patients (n = 58, 100%) and 5 of 5 for 87.9% of patients (n = 51). LLM summaries captured a mean of 4.9 (Standard Deviation = 0.33) of 5 domains and a mean of 16.1 (SD = 2.6) bullet points per patient across 5 SIC domains. In contrast, only 8 patients (13.8%) had any Epic SIC module entries; of those, only a mean of 2.8 (SD = 1.3) of 5 domains were filled. These entries typically lacked longitudinal context or detailed goals and contained a mean of 129.1 (SD = 120.3) words compared to the mean of 582.2 (SD = 163.7) words for the LLM summaries. Compared to SIC modules, LLM summaries included data that was a mean of 66.6 (SD = 129.1) days more recent. Qualitatively, LLM summaries provided time-ordered syntheses integrating patient priorities, symptom burden, and family context, whereas SIC module entries, when present, contained minimal information that reflected a fixed timepoint. Conclusions: LLMs can securely synthesize longitudinal free-text notes into concise SIC summaries that substantially outperform existing SIC modules in completeness, timeliness, and utility. Application of LLM SIC summaries in clinical care may improve goal-concordant decision-making for hospitalized patients. Clinical trial information: NCT07147023 . Illustrative comparison of illness understanding documented in the Epic SIC module vs. LLM summary for a patient. SIC module [date] Aggressive progression on TNBC. LLM summary [date] Patient understands cancer progression with new metastases in liver, bone, and lungs. Aware of current treatment (capecitabine) ineffectiveness and new treatment (eribulin) plan. Cognizant of potential side effects and limited prognosis if new treatment is ineffective.