K-M LLM-pro: Physics-guided cross-modal adaptation for fine-grained spatiotemporal trajectory classification

Abstract

Spatiotemporal trajectory classification is essential for intelligent perception systems but faces challenges including weak separability of dynamic features, representation collapse under limited samples, and heterogeneous conflicts in multimodal data. To address these issues, we propose K-M LLM-pro, a physics-guided cross-modal adaptation framework that integrates statistical mechanics with large language models (LLMs) to improve trajectory understanding. Our approach incorporates: (1) physics-informed prompt engineering based on Kramers-Moyal coefficients, embedding physical constraints via reproducing kernel Hilbert space projection; (2) a dynamic patching optimization mechanism combining variance maximization and Lyapunov stability criteria for unified modeling of heterogeneous trajectories; and (3) dual spatiotemporal adapters with a parameter-efficient expansion strategy, injecting domain knowledge while optimizing only 3.8% of new parameters. Experimental results on public datasets such as Geolife and AIS show that K-M LLM-pro outperforms state-of-the-art models in classification accuracy, demonstrating strong performance even in few-shot scenarios with only 1% of training data. To our knowledge, this is the first work to integrate K-M coefficients as interpretable statistical priors into LLMs, offering a lightweight and effective solution for modeling complex spatiotemporal dynamics.