AI-driven large language models for multimodal prediction of homologous recombination deficiency using clinical features and histopathological whole-slide images.

Abstract

548 Background: Homologous recombination deficiency (HRD) implies the dysfunction of homologous recombination repair at the cellular level. The assessment of HRD status benefits making treatment plans and fertility counseling of breast cancer patients. Standard diagnostic tests for detecting HRD are expensive and not universally available. Methods: We trained a deep learning framework to predict HRD status by integrating routinely collected clinical records and histopathological imaging data, including hematoxylin and eosin (H&E)–stained whole-slide images, from primary breast cancer patients (n = 397) across three independent regional medical centers. The patient cohort was split into a training set (80%) and a validation set (20%). Clinical text as well as whole-slide images were encoded using large language model. This model integrated multimodal representation learning and was applied to generate robust HRD predictions. Results: Across breast cancer cohorts from three independent regional medical centers, the proposed approach demonstrated robust and consistent performance in predicting HRD. The reported AUC of 0.82 (95% CI, 0.79–0.84) represents the mean value over ten independent experiments with different random seeds. Notably, the model achieved a precision of 0.84 and a specificity of 0.92, indicating strong discriminatory power with a tendency towards cautious decision-making. Conclusions: Our model showed promising HRD predictive performance in breast cancers directly from routine multimodal data integrating clinical characteristics, imaging description with pathological slides. The HRD-positive predictions generated by this model show promise for clinical translation.