LLM-generated serious illness conversation summaries to improve care for hospitalized patients: A pilot clinical trial.

Abstract

1519 Background: Patients with cancer often receive intensive care near the end of life that does not align with their goals and preferences. Hospitalization is a critical opportunity to engage in serious illness conversations (SICs) that can align care with preferences, but SICs rarely occur. Large language model (LLM)-generated SIC summaries may help busy inpatient clinicians by identifying patient preferences in fragmented SIC documentation in voluminous electronic health record (EHR) notes. Methods: This randomized pilot clinical trial enrolled adults with solid tumors admitted to an academic hospital with a 90-day predicted mortality > 40%. Patients were randomized 3:1 (intervention:control) to usual care or to an intervention consisting of an LLM-generated summary of SIC documentation in EHR notes from the last 6 months emailed to their inpatient and outpatient oncology and palliative care teams within 16 hours of admission. Summaries included 8 SIC topics and up to 18 bullet points, and emails included a message encouraging SICs and incorporation of patients’ preferences into care plans. The primary outcome was SIC summary accuracy: manual review determined whether each summary bullet point accurately reflected the source note, and the proportion of accurate bullet points were calculated. Other outcomes included user perspectives from day 3 patient interviews and day 7 clinician surveys and 90-day care delivery outcomes (days in the hospital, number of hospitalizations, hospital re-admission). Results: 58 enrolled patients were 55% female, 76% White, 91% non-Hispanic and had a mean 90-day predicted mortality of 57.7%. 902 of 936 (96.4%) summary bullet points accurately reflected underlying documentation. Inaccuracies were typically not clinically significant and related to overstating the strength of preferences (e.g., documentation stating a plan to start treatment to improve symptoms might be summarized to state the patient is “focused on improving symptoms”) and misattribution of clinician concerns to the patient. 32 patients participated in interviews. They were generally supportive of the intervention and felt it would improve patient-clinician understanding and communication around preferences. 31 clinicians completed surveys and evaluated SIC summaries as “moderately” to “extremely accurate.” 90-day outcomes were mature for 24 patients. In 90 days after enrollment, preliminary mean hospital days (20.9 vs 13.2), mean number of hospitalizations including index admission (2.3 vs 1.5) and re-admission (42.9% vs 23.5%) were all lower in the intervention arm. Full trial data will be available in March. Conclusions: LLM-generated SIC summaries can be accurately generated, delivered within 16 hours of hospital admission, are received favorably by patients and clinicians, and may reduce care utilization. An effectiveness clinical trial starts May 2026. Clinical trial information: NCT07147023 .