Prospective evaluation of AI-assisted documentation and summarization in virtual multidisciplinary cancer conferences.

Abstract

1614 Background: Multidisciplinary cancer conferences (MCCs) are central to high-quality oncologic decision-making but are frequently constrained by time-intensive and variable documentation processes. Artificial intelligence (AI) can enhance multidisciplinary collaboration by improving the accuracy, timeliness, and consistency of documentation. However, prospective evaluations of AI tools embedded within MCC workflows remain limited. Methods: We conducted a prospective pilot study at the Cancer Centre of Southeastern Ontario, a regional academic cancer center in Ontario, Canada. In October 2025, a hospital-approved AI system, Microsoft Copilot, was integrated into all virtual lung and gastrointestinal (GI) MCCs conducted via Microsoft Teams. All conferences were recorded and transcribed. Structured prompts for AI-generated summaries were co-developed with MCC stakeholders and refined through iterative Plan-Do-Study-Act (PDSA) cycles. AI-generated summaries were validated against clinician-prepared electronic health record (EHR) documentation. Two independent reviewers assessed case identification accuracy and concordance of key clinical content. Following validation, AI summaries were distributed to MCC participants after case discussions to support EHR documentation, with mandatory clinician verification prior to finalization. Results: A total of 125 patient cases from lung and GI MCCs were analyzed, of which 108 had corresponding clinician-prepared EHR documentation available for comparison. Inter-rater agreement between reviewers assessing concordance of AI-generated and clinician-prepared summaries demonstrated high agreement (Cohen’s κ = 0.87). Case identification accuracy was high, with agreement rates of 97% for case number, 93% for medical record number, 96% for patient initials, 98% for presenter, 99% for diagnosis, and 99% for cancer stage; most identification discrepancies were minor and improved over time. Clinical content concordance was high, including discussion summaries at 99%, radiology at 99%, pathology at 98%, and treatment plans and next steps at 96%. Major discrepancies involving key clinical decisions were identified in 0.9% of cases. All AI-generated summaries were distributed within 24 hours following MCC completion, with a median clerical preparation time of 14 minutes per case. Conclusions: In this prospective, real-world evaluation, AI-assisted summaries integrated into virtual MCC workflows demonstrated high accuracy, strong safety performance, and meaningful efficiency gains. With mandatory clinician oversight, AI-supported summarization was feasible and timely, supporting its role as a scalable model for MCC documentation across oncology programs. Further multicenter studies are warranted to evaluate generalizability, clinician workload impact, and downstream effects on care delivery.