Real-world time-to-event detection of immune-related adverse events from electronic health records using clinical natural language processing.

Abstract

e13650 Background: Immune-related adverse events (irAEs) are a major challenge in immune checkpoint inhibitor (ICI) therapy. In real-world practice, irAEs are often documented in unstructured clinical narratives, limiting systematic surveillance. We evaluated the feasibility of detecting irAEs from clinical narratives in electronic health records (EHRs) using a natural language processing (NLP) pipeline in a large real-world cohort. Methods: We conducted a retrospective cohort study using EHR data from a single tertiary hospital (2004–2023). Among 177,535 patients, 57,658 had cancer. Patients with cancer prescribed ICIs were compared to patients with cancer without documented ICI prescriptions using 1:1 nearest-neighbor propensity score matching. Clinical narratives were analyzed using MedNERN-CR-JA, a BERT-based Japanese medical NLP pipeline, to extract irAE-labeled entities. The primary outcome was time to first NLP-detected irAE per patient, defined as the earliest documentation of either an organ-attributed irAE entity or a generic “irAE” mention. Time-to-event analyses were performed using Kaplan–Meier and Cox proportional hazards models. All clinical notes from the matched cohort were manually reviewed at the note level as a reference standard by a trained pharmacy student and an oncology-certified pharmacist, then aggregated to patient-level classifications and assessed for misclassification, with correction of misclassified cases and qualitative error analysis. Results: After matching, 1,160 patients were included in each group. NLP-detected irAEs occurred in 18.8% of ICI-treated patients, compared with 0.9% in matched patients with cancer without documented ICI prescriptions, consistent with known real-world irAE frequencies, and were strongly associated with ICI exposure (hazard ratio: 23.72, 95% confidence interval: 12.94–43.48). Organ-specific irAEs explicitly mentioned in clinical narratives were identifiable using the NLP pipeline, and irAEs normalized as non-organ-specific entities could be retrospectively mapped to specific organs through manual chart review. Manual review revealed few false-positive patient classifications, mainly due to differential diagnoses or precautionary statements without active symptoms. Conclusions: Clinical NLP enables scalable, real-world detection of irAEs from unstructured EHR narratives and efficient identification of an irAE-enriched cohort, substantially reducing the population requiring detailed review. This framework supports robust time-to-onset analyses and facilitates future investigations of longer-term outcomes, such as symptom resolution, that typically require extended follow-up. These results support the feasibility of NLP-based irAE surveillance and its potential role in scalable, real-world safety evaluations of ICI therapy.