Spatio-temporal transformer traffic prediction network based on multi-level causal attention

Abstract

Traffic prediction is a core technology in intelligent transportation systems with broad application prospects. However, traffic flow data exhibits complex characteristics across both temporal and spatial dimensions, posing challenges for accurate prediction. In this paper, we propose a spatiotemporal Transformer network based on multi-level causal attention (MLCAFormer). We design a multi-level temporal causal attention mechanism that captures complex long- and short-term dependencies from local to global through a hierarchical architecture while strictly adhering to temporal causality. We also present a node-identity-aware spatial attention mechanism, which enhances the model’s ability to distinguish nodes and learn spatial correlations by assigning a unique identity embedding to each node. Moreover, our model integrates several input features, including original traffic flow data, cyclical patterns, and collaborative spatio-temporal embedding. Comprehensive tests on four real-world traffic datasets—METR-LA, PEMS-BAY, PEMS04, and PEMS08—show that our proposed MLCAFormer outperforms current benchmark models.