A biologically constrained agent-based model of cancer stem cell dynamics with reinforcement learning-guided adaptive radiotherapy

Abstract

Cancer stem cells (CSCs) represent a rare but critical subpopulation within tumors, driving recurrence, resistance to therapy, and aggressive growth. To better understand CSC behavior in solid tumors, we developed a biologically constrained agent-based model (ABM) that simulates tumor progression initiated from a single CSC. The model incorporates essential microenvironmental factors—including oxygen diffusion, spatial limitations, stochastic migration, and cell cycle dynamics—allowing for high-resolution simulation of tumor development and intra tumoral heterogeneity. While this work does not aim to fully optimize therapy for clinical application, it provides a flexible, scalable simulation environment where adaptive treatment strategies can be tested. To extend a biological model toward intelligent treatment, we integrated a reinforcement learning (Q-learning) component that adaptively adjusts radiation dosage based on real-time CSC localization and microenvironmental feedback. This component is currently presented as a proof-of-concept to demonstrate feasibility, and its optimization and convergence analysis will be explored in future studies. Our results suggest that reinforcement learning, when integrated with a biologically grounded ABM, can guide adaptive and more personalized radiotherapy strategies.