Knowledge distillation for named entity recognition in traditional chinese medicine

Abstract

Abstract Named entity recognition (NER) in traditional Chinese medicine (TCM) text is central to the structuring and intelligent application of TCM knowledge. TCM case texts are characterized by sparse entity distributions and limited domain-specific data. Existing methods often struggle to recognize low-frequency entities and show limited capacity for effective knowledge transfer. This paper proposes a TCM NER framework incorporating structured knowledge distillation. First, multi-source TCM corpora are constructed using natural language processing techniques. A teacher model is trained on high-density structured data, and its semantic knowledge is transferred to the student model through soft labels. The student model uses BERT for encoding, BiLSTM for sequential feature extraction, Transformer for global context modeling, and CRF for structured sequence decoding. Experimental results on the constructed StudentDataset show that the distilled TBTC model improved Precision by 8.01 percentage points, Recall by 5.31 percentage points, and F1-score by 6.67 percentage points over its non-distilled counterpart, achieving an overall F1-score of 77.01.