Zero-shot classification of ECG signals using CLIP-based models

Abstract

Abstract In this study, we present a comprehensive evaluation of Contrastive Language-Image Pre-training (CLIP) for zero-shot electrocardiogram (ECG) classification across multiple datasets and diagnostic classes. Traditional ECG classification models require significant labeled training data for every diagnostic class, limiting their adaptability to new or unseen classes. To address this limitation, we trained and evaluated 24 CLIP-based models, composed of one image encoder (CNN Base, CNN V2, CNN V3, RNN, ISIBrno) and one text encoder (BioBERT, Bio+ClinicalBERT, bert-case-uncased) on 27 seen classes on three datasets (PTB-XL, Ningbo, and Gerogia) and evaluated zero-shot classification performance on 11 unseen classes. We investigate the impact of training dataset size, encoder architectures, and pretraining effects on both in-distribution and out-of-distribution generalization performance. Our experiments included internal evaluation (Experiment A) and external validation on two independent datasets (SPH and CODE-15%, Experiment B). The top-performing models achieves a macro-averaged ROC-AUC of 0.70 for zero-shot out-of-distribution classification, 0.70 for zero-shot in-distribution classification, and 0.83 for out-of-distribution classification with classic training. These results demonstrate that CLIP-based models can meaningfully classify ECG conditions beyond the scope of their training, offering a flexible alternative to current ECG diagnostic systems where clinical needs are constantly evolving.