KALFormer: Knowledge-augmented attention learning for long-term time series forecasting with transformer

Abstract

Time series forecasting remains a fundamental yet challenging task due to its inherent non-linear dynamics, inter-variable dependencies, and long-term temporal correlations. Existing approaches often struggle to jointly capture local temporal continuity and global contextual relationships, particularly under complex external influences. To overcome these limitations, we propose KALFormer, a knowledge-augmented attention learning transformer framework that integrates sequential modeling with external information fusion. KALFormer enhances spatiotemporal representation and contextual reasoning by integrating Long Short-Term Memory (LSTM) encoders, Transformer-based self-attention mechanisms, and knowledge-aware modules. Extensive experiments on six public benchmark datasets demonstrate that KALFormer achieves an average improvement of 8.4% in MSE and MAE compared with representative baseline models, highlighting its robustness, interpretability, and reliability for long-term time series forecasting. The source code is available at https://github.com/dxpython/KALFormer .