Large language models as decision support in the absence of genomic profiling: A simulated prospective study on early-stage breast cancer.

Abstract

e12553 Background: Genomic profiling guides adjuvant chemo in early-stage BC but remains inaccessible in resource-constrained settings. We evaluated whether LLMs could optimize oncologist decision-making in the absence of genomic data. Methods: In this simulated prospective, multi-reader study, two board-certified oncologists independently evaluated clinicopathological vignettes of 200 HR+/HER2- early-stage BC cases. Adjuvant treatment recommendations and confidence ratings were documented in two phases separated by a 4-week washout: an initial unaided assessment utilizing only clinicopathological variables, and a subsequent review assisted by the Gemini 3 Pro LLM. We evaluated changes in chemo recommendations, inter-rater agreement, decision confidence, and concordance with NCCN guideline-based recommendations derived from Oncotype DX recurrence scores. Results: LLM assistance increased inter-rater reliability from moderate (κ = 0.473) to substantial (κ = 0.565). This improvement was statistically significant in the node-positive subgroup (n = 56), where the kappa value increased from 0.257 to 0.546 (Δκ = 0.290; p = 0.031). The oncologist with a higher baseline recommendation rate significantly reduced chemo proposals from 62% to 56% (p = 0.045), indicating a treatment de-escalation effect. Physician confidence in decision-making also increased significantly for both oncologists (p < 0.01). However, despite improved consensus and confidence, concordance with the Oncotype DX-based reference standard did not statistically improve for either oncologist (Oncologist 1: Δκ = 0.066, p = 0.124; Oncologist 2: Δκ = −0.084, p = 0.114). Conclusions: In the absence of genomic profiling, LLMs effectively standardized clinical reasoning and mitigated inter-observer variability, serving as a pragmatic decision-support tool to reduce potential overtreatment. While AI enhances physician confidence and consensus, it cannot replicate the biological risk stratification provided by molecular assays. Impact of LLM assistance on key metrics (n=200). Key Metric Unaided Phase LLM-Aided Phase p- value Oncologist 1 chemo rec. (%) 62.0 56.0 0.045 Oncologist 2 chemo rec. (%) 37.0 35.5 0.678 Inter-rater agreement (κ) 0.473 0.565 0.131* High confidence - Onc 1 (%) 39.5 66.5 <0.001 High confidence - Onc 2 (%) 53.0 64.5 0.007 Concordance w/ ref. (κ) - Onc 1 0.253 0.319 0.124* Concordance w/ ref. (κ) - Onc 2 0.461 0.377 0.114* *Bootstrap analysis.