Structure-aware state space modeling with multi-scale feature fusion for railway scene segmentation

Abstract

Abstract Edge-deployed railway safety monitoring demands pixel-perfect semantic segmentation under extreme constraints: detection failures threaten operational safety, yet trackside devices must process diverse environmental conditions (fog, rain, nighttime) at over 30 FPS with under 50M parameters. Traditional CNNs lack long-range modeling capabilities for distant obstacle detection, while Transformer-based methods impose prohibitive quadratic computational complexity unsuitable for edge deployment. We present HybridSeg, a novel architecture that reformulates visual state space modeling as a controllable Markov Decision Process, enabling context-adaptive information propagation through reinforcement learning. Our approach integrates: (1) meta-learned state space dynamics via Proximal Policy Optimization, where learned policies dynamically adjust state transition parameters based on gradient feedback and hidden state statistics, the first application of RL-based parameter adaptation to visual state space models; (2) Structure-Aware Deformable Mamba blocks combining four-directional scanning with deformable spatial attention for irregular geometry handling; (3) cross-scale attention fusion across four pyramid levels with learnable inter-scale dependency modeling; (4) explicit multi-scale consistency constraints stabilizing training and improving generalization. Evaluated on 8,000 railway surveillance images spanning four environmental conditions, HybridSeg achieves 92.34 ± 0.25% mIoU and 97.82 ± 0.12% pixel accuracy at 38.52 FPS with 45.28M parameters–outperforming state-of-the-art CNN, Transformer, and Mamba methods by 1.61-3.16% in accuracy while delivering 2.31× faster inference than comparable approaches. The architecture demonstrates robust cross-domain transfer (89.53% CDR) and competitive performance on Cityscapes (85.73%), CamVid (87.25%), and ADE20K (48.53%), validating practical deployment for safety-critical edge applications.