UMOT: A unified framework for long- and short-term association for multi-object tracking

Abstract

Multi-object tracking (MOT) is an important research direction in the field of computer vision, but it often leads to problems such as trajectory breakage and identity switches in complex scenarios due to the similarity of target appearance and long-term occlusions. Although the existing Transformer-based end-to-end methods have made some breakthroughs in short-term motion modelling, they are still deficient in long-term dependent information modelling and target recovery. To this end, this paper proposes a unified framework for long-term and short-term association, UMOT, which fundamentally alleviates the contradiction between motion prediction between neighboring frames and long-term trajectory recovery across frames. UMOT constructs short-term correlation using pre-trained YOLOX detector and MOTR-ConvNext network, and through the dynamically updated track queries and detect queries jointly to build the motion and appearance feature models of targets between adjacent frames to optimize the short-term motion prediction. Meanwhile, it effectively stores and smoothly updates the historical trajectory information by designing the Track Query Memory Module (TQMM) and associates the unmatched detections across frames by the Historical backtracking Module, so as to realize the restoration of the long-term lost targets and the identity preservation. Extensive experiments on DanceTrack and MOT17 datasets show that UMOT achieves significant improvement in HOTA, IDF1 and other metrics compared to existing state-of-the-art methods, which verifies its robustness and effectiveness in complex occlusion and long-term dependency scenarios.