Towards intelligent railway monitoring: A novel hybrid deep learning architecture for railway obstacle detection

Abstract

Railways are among the most efficient modes of transportation, capable of moving large quantities of goods and passengers over long distances at relatively low cost. However, accidents frequently occur due to objects or individuals present on the tracks, as trains are unable to swerve and require long braking distances. While the localization of objects within the track bed is a well-explored topic, the reliable and high-performance classification of such obstacles across all relevant categories remains an unresolved challenge. This study proposes an innovative hybrid architecture that leverages the specific visual characteristics of track bed imagery, setting a new benchmark in this domain. The hybrid design effectively leverages the strengths of ResNet50 and Swin Transformer V2, allowing the model to capture both local and global features. In addition, an Efficient Attention Module is integrated to further emphasize the most relevant features for robust obstacle classification. Using stratified five-fold cross-validation on a dataset of 2,003 images across six classes (iron bar, boulder, person, branch, canister, and barrel), the model achieved an average balanced accuracy of 99.46%. The results have implications for accident prevention, improving operational efficiency, and modernizing railway safety systems, thereby enabling the future application of automatic railway surveillance systems to ultimately enhance operational security.