Deep reinforcement learning-based multi-lane mixed traffic ramp merging strategy

Abstract

Due to concentrated conflicts, on-ramp merging is an important scenario in the study of new hybrid traffic control. Current research mainly focuses on optimizing the vehicle passage sequence of ramp vehicles merging with mainline vehicles in single-lane scenarios, neglecting the coordination problem of vehicles in multiple mainline lanes. Therefore, an Improved Dueling Double DQN (D3QN) On-ramp Merging Strategy (IDS stands for the initials of Improved, D3QN, and Strategy) combined with a sine function is proposed, establishing a Vehicle Coordination System (VCS) to guide the merging of vehicles in multi-lane mainline traffic. This strategy uses the improved D3QN algorithm combined with the excellent smoothness of the sine function to evaluate driving safety, helping vehicles find suitable gaps in traffic flow. An action masking mechanism was deployed during the strategy exploration phase to prevent unsafe actions. The proposed VCS + IDS strategy was tested in SUMO simulations of on-ramp merging under different density of vehicle flow. Under a traffic flow of 1200 vehicles per lane per hour, the on-ramp merging completion rate of VCS + IDS reached 98.62%, and the task completion rate was 98.11%, which increased by 11.08% and 10.79% compared to traditional D3QN, respectively, validating the effectiveness of this method.