A framework for detecting and predicting highway traffic anomalies via multimodal fusion and heterogeneous graph neural networks

Abstract

This paper presents a novel framework for detecting and predicting abnormal traffic events on highways. Current traffic monitoring systems often rely on single data sources, which limits their detection accuracy and robustness in complex environments. To address these limitations, we propose a framework based on multimodal deep fusion and heterogeneous graph neural networks (HGNNs), incorporating an Ensemble Contrastive Pessimistic Likelihood Estimation (CPLE) algorithm to optimize performance. The framework integrates static and dynamic traffic data, such as video images, traffic flow, vehicle speed, and tunnel weather conditions. Experimental results demonstrate that the model performs well in various scenarios, showing significant improvement in accuracy and stability over existing models like AGC-LSTM and AttentionDeepST. For instance, the proposed MHGNN-CPLE model achieves superior accuracy and F1 score in static detection tasks while maintaining high accuracy under different noise levels in dynamic detection scenarios. This study provides a significant advancement in traffic event analysis by effectively combining multimodal data and leveraging HGNNs to capture complex spatiotemporal dependencies.