Artificial intelligence and image processing framework for automated airway invasion detection and residue classification from swallowing endoscopy

Abstract

Abstract Fiberoptic endoscopic evaluation of swallowing (FEES) is a widely used instrumental method for dysphagia, but its interpretation depends on lighting and evaluator experience. This study developed and validated a framework artificial intelligence (AI) and image processing to detect penetration, aspiration, and classify residues according to the Yale Pharyngeal Residue Severity Rating Scale (YPR-SRS). The system combines anatomical tracking of the epiglottis, arytenoids, and vocal folds with contrast adjustments, color-based segmentation, and enhancement filters, optimizing the visibility of clinical events. In validation with 60 videos, accuracy was of 0.90 for penetration and 0.87 for aspiration, with sensitivities of 1.00 and 0.90, specificities of 0.80 and 0.85, and Kappa values of 0.80 and 0.75. For residues in the pyriform sinuses, accuracy ranged from 0.91 to 0.95, sensitivity from 0.91 to 1.00, specificity from 0.83 to 1.00, and Kappa = 0.87. In the valleculae, accuracy ranged from 0.95 to 1.00, with sensitivity and specificity of 0.91–1.00 and Kappa = 0.91. These findings highlight its assistive potential for FEES analysis and clinical decision-making support in penetration, aspiration, and pharyngeal residue classification, a capability not previously reported.