PRISM: A transparent, self-correcting LLM system for reproducible therapy recommendations in neuroendocrine tumors.

Abstract

e13657 Background: The clinical application of Large Language Models (LLMs) in oncology is currently limited by opaque reasoning and the potential for "hallucinations," which pose safety risks for Multidisciplinary Tumor Boards (MTBs). Neuroendocrine tumors (NETs) specifically require precise interpretation of complex guidelines and recent primary evidence from clinical trials that are not already implemented in the guidelines. To address this need, we present PRISM (Personalized Recommendations via Integrated Synthesis & Modeling), a multi-agent LLM architecture designed to improve the quality, traceability, and reproducibility of therapy recommendations in NET care through a transparent, self-correcting workflow. Methods: PRISM employs a structured seven-stage fully automated, LLM-based workflow decomposing clinical reasoning into explicit steps, including guideline selection (e.g. ENETS & ESMO), initial patient case analysis, and search for matching trial evidence. A validation agent autonomously verifies matched evidence against existence (e.g., trial numbers), triggering correction loops to correct hallucinations or citation errors. In a feasibility study of 15 complex NET cases, we compared PRISM against a standard single-pass LLM (baseline). Performance was assessed by an expert using a structured framework evaluating clinical quality and safety given the patient’s organ functions, together with an overall quality rating (1–10) and implementation willingness (Yes/Maybe/No). Results: PRISM autonomously identified an average of 12 clinical trials per patient with potentially relevant primary evidence for treatment options, from which 66.7% were used in the treatment recommendations. The validation agent rejected 93% (14/15) of initial drafts due to hallucinations or citation errors. Autonomous re-processing (mean 2.6 iterations) led to full pass of 73% (11/15) of cases. In a blinded head-to-head comparison, the baseline model produced unsafe recommendations in 13% of cases, leading to hard implementation rejections (“No”) and expert-identified hallucinations. In contrast, PRISM reduced unsafe outputs to 6.7% and eliminated all hard rejections, achieving 100% implementation potential (60% “Yes”, 40% “Maybe”). PRISM achieved higher consistency in output quality (range: 5–9 vs. 1–10). Conclusions: PRISM addresses key limitations of standard LLMs by replacing opaque generation with a transparent, self-correcting validation loop. It systematically identifies relevant guidelines and primary evidence from clinical trials. In this feasibility study, the system ensured trustworthiness by enforcing strict evidence traceability. These findings support the prospective evaluation of multi-agent LLM systems as decision-support tools for treatment decisions in oncology, especially to bridge the expertise gap in non-specialized settings.