Overcoming surveillance gaps: Deep learning for accurate detection and chronicity classification of hospital-acquired pulmonary embolism

Abstract

Abstract Introduction: Hospital-acquired venous thromboembolism (HA-VTE) is one of the most preventable causes of in-hospital death. Accurate detection is essential for evaluating the effectiveness of thromboprophylaxis and guiding safety interventions. At our institution, radiology reports are unstructured, limiting the effectiveness of automated tools. Prior efforts using ICD codes and trigger-based systems were inaccurate, and manual chart review—though more reliable—proved too labor-intensive for sustained surveillance. Moreover, existing systems do not reliably distinguish between acute and chronic pulmonary embolism (PE), which is critical for identifying new, hospital-acquired events. These limitations drove the development of an AI-based solution to extract and classify VTE events directly from radiology reports. Methodology Study Setting We included all CT pulmonary angiography (CTPA) reports from January 2024 to June 2025 at a tertiary National Guard hospital in Saudi Arabia. Each report was independently reviewed by two hematologists to determine the presence and chronicity (acute vs. chronic) of PE. Discrepancies were resolved through consensus discussion. Model Development We developed a semi-supervised active learning paradigm for PE detection and acute/chronic classification in radiology reports using BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext as the backbone model. Initially, 1,934 labeled reports (approximately 10% positive) were used to fine-tune the transformer with a binary classification objective for PE detection. The dataset exhibited severe class imbalance, which we addressed using class-balanced loss weighting, AdamW optimization, and linear learning-rate scheduling with warmup. To overcome both limited labeled data and extreme class imbalance, we implemented a pool-based semi-supervised active learning approach: the fine-tuned model was applied to 5,000 unlabeled reports, strategically identifying potential positive cases. From these predictions, 745 high-confidence positive cases were manually reviewed and verified by clinical experts, then integrated into the training dataset. This resulted in a final curated dataset of 2,679 reports (35.27% positive), achieving a 387% increase in positive cases while improving class balance from 9:1 to 1.84:1 ratio. Results The final binary PE detection model achieved exceptional performance on the expanded dataset of 2,679 reports, with Accuracy = 99.47%, Precision = 98.65%, Recall = 100.00%, F1-score = 99.32%, and AUC = 99.99%. Only two false negative classifications were observed in binary detection. The acute/chronic PE classification model also showed strong generalization, achieving Accuracy = 96.48%, F1-score = 89.36%, Precision = 91.30%, Recall = 87.50% and AUC = 98.62% with only 5 misclassified cases. Discussion and Conclusion This study demonstrates the feasibility and effectiveness of a semi-supervised active learning framework for pulmonary embolism detection and chronicity classification from radiology reports. By combining transformer-based feature extraction with model-assisted case selection and expert validation, we developed a scalable strategy that addresses two fundamental challenges in clinical NLP: limited labeled data and class imbalance. The iterative training approach, beginning with a small, imbalanced dataset and enriched through targeted sampling from a large unlabeled pool, enabled a 387% increase in positive case representation (from 10% to 35.27%) while requiring manual review of only 14.9% of the unlabeled data. This substantially improved class balance and model performance with minimal annotation burden. To our knowledge, this is the first application of a semi-supervised active learning strategy in radiology report classification for PE, particularly one that captures both acute and chronic status. Compared to prior approaches relying on rule-based heuristics or fully supervised learning, our method achieved superior performance with far less labeled data. Our AI-based tool accurately detects hospital-acquired PE from unstructured radiology reports, addressing key limitations of existing surveillance methods. In addition to binary detection, it classifies the chronicity of PE, enabling more precise identification of hospital-acquired events. Future work will extend this approach to lower limb DVT, enable real-time clinical integration, and support prediction of hospital-acquired VTE to guide prevention strategies.