A two-stage deep learning framework for lead instrument recognition in polyphonic music featuring Chinese instruments

Abstract

Rapid advancement of deep learning methodologies has considerably progressed the domain of automatic music analysis; however, several challenges remain in adapting these approaches to the intricate and multifaceted realm of Chinese traditional music. In particular, the task of discerning lead instruments within polyphonic textures, a fundamental element of Chinese ensemble performance, has not been sufficiently explored, and current models frequently fail to address the complex acoustic features and structural nuances present in real-world recordings. To bridge this gap, we propose a deep learning framework for Chinese instrument recognition employing a two-stage “separation-then-classification” strategy. Initially, a source separation module is deployed to extract individual instrument representations from mixed audio, which is subsequently followed by a multi-label classification network to identify the target instruments. Empirical results from a newly constructed dataset reveal marked enhancements in accuracy, precision, recall, and F1 score metrics, with especially notable improvements in distinguishing instruments similar in timbral characteristics, such as dizi and xiao. This study offers a novel approach to the recognition of polyphonic instruments within traditional music contexts, presenting a scalable and model-agnostic improvement strategy. It also laid the methodological and technological foundations for the preservation, analysis, and promotion of the cultural heritage of Chinese music through intelligent audio systems. The code is available at: https://github.com/CB389636/chic .