An auditory-visual cooperative perception method for honking vehicle localization

Abstract

Locating honking vehicles is crucial for controlling arbitrary honking and reducing environmental noise. However, traditional methods for honking vehicle localization, which utilize sound source localization technology, suffer from inaccuracies and limited detection range due to the multipath effects of sound propagation and environmental noise interference. To address these challenges, an auditory-visual cooperative perception (AVCP) method for honking vehicle localization is proposed, and a detailed workflow of this method is presented. In the AVCP method workflow, the Emphasized Channel Attention, Propagation, and Aggregation in Time-Delay Neural Network (ECAPA-TDNN) is used to recognize honking vehicle models from captured audio signals, as different vehicle models exhibit distinct horn sound characteristics. Subsequently, YOLO v9 is employed to detect vehicles and recognize their corresponding models in the images captured by the camera. Thus, among the vehicles detected and identified using YOLO v9, the honking vehicle is determined as the one whose model matches the vehicle model recognized by ECAPA-TDNN. Additionally, experiments with simulated and public datasets were conducted to evaluate the performance of the AVCP method for honking vehicle localization. The experimental results show that the AVCP method is less susceptible to environmental noise and can more accurately identify and locate vehicles from greater distances compared to traditional methods based on sound source localization technology.