Integrated decision-control for social robot autonomous navigation considering nonlinear dynamics model

Abstract

Reinforcement learning (RL) has demonstrated significant potential in social robot autonomous navigation, yet existing research lacks in-depth discussion on the feasibility of navigation strategies. Therefore, this paper proposes an Integrated Decision-Control Framework for Social Robot Autonomous Navigation (IDC-SRAN), which accounts for the nonlinearity of social robot model and ensures the feasibility of decision-control strategy. Initially, inverse reinforcement learning (IRL) is employed to tackle the challenge of designing pedestrian walking reward. Subsequently, the Four-Mecanum-Wheel Robot dynamic model is constructed to develop IDC-SRAN, resolving the issue of dynamics mismatch of RL system. The actions of IDC-SRAN are defined as additional torque, with actual torque and lateral/longitudinal velocities integrated into the state space. The feasibility of the decision-control strategy is ensured by constraining the range of actions. Furthermore, a critical challenge arises from the state delay caused by model transient characteristics, which complicates the articulation of nonlinear relationships between states and actions through IRL-based rewards. To mitigate this, a driving-force-guided reward is proposed. This reward guides the robot to explore more appropriate decision-control strategies by expected direction of driving force, thereby reducing non-optimal behaviors during transient phases. Experimental results demonstrate that IDC-SRAN achieves peak accelerations approximately 8.3% of baseline methods, significantly enhancing the feasibility of decision-control strategies. Simultaneously, the framework enables goal-oriented autonomous navigation through active torque modulation, attaining a task completion rate exceeding 90%. These outcomes further validate the intelligence and robustness of the proposed IDC-SRAN.