BAASNet: boundary-aware deep learning for accurate polyp segmentation in colonoscopy

Abstract

Abstract Colorectal polyps are primarily detected through colonoscopy, which plays a central role in early cancer prevention. Precise polyp segmentation supports treatment planning and diagnostic accuracy by providing masks that encode clinically relevant structures. Recent advancements in deep learning have led to several polyp segmentation models. However, performance remains hindered by challenges such as image noise, complex textures, indistinct boundaries, and diverse polyp morphologies. The high cost and time burden of manual annotation underscore the need for automated segmentation systems. To overcome these limitations, BAASNet, a Boundary-Aware Attention-Based Segmentation framework, is introduced for polyp segmentation. A boundary-aware loss function is integrated to improve performance, particularly in delineating polyp edges. The method is evaluated on nine publicly available datasets spanning five imaging modalities, including two center-wise polyp detection benchmarks, demonstrating strong generalization capability. On PolypDB, the model attains a mean Dice similarity coefficient (mDSC) of at least $$89.60\%$$ across all five modalities. Across all evaluated benchmarks, the proposed model achieves an average absolute improvement of approximately $$3.3\%$$ in Dice. Gains vary by dataset, ranging from approximately $$0.7\%$$ to $$4.7\%$$ relative improvement over the best previous results. These results demonstrate BAASNet’s potential for robust, real-time clinical deployment in automated colonoscopy workflows.