A lightweight transformer framework for open set anomaly segmentation in smart city applications

Abstract

Abstract Open-set anomaly segmentation task in diverse infrastructure faces substantial challenges due to its computational overhead and accuracy measures. Although the existing transformer-based methods are efficient, that are limited in the factors of Computational efficiency and accuracy trade-offs. This paper presents LightMask, a lightweight transformer-based architecture designed for efficient, context-aware segmentation of anomalous regions in complex urban environments. The proposed framework has five key contributions: optimized EfficientNet-B0 backbone, adaptive inference mechanism, separable self-attention (SSA) with linear complexity, progressive multi-scale decoder with dynamic early termination, and boundary-aware contrastive loss for open-set anomaly segmentation tasks. LightMask focuses on the lightweight framework with computational efficiency first, while preserving the performance of anomaly detection. The evaluation results showcase that LightMask produces lower parameter count of 4.29 million (16.35 MB) ensures the lightweight structure and a computational efficiency with only 8.72 GFLOPs. For training and evaluation, the Cityscapes and RoadAnomaly datasets were used and the finding reveals the model robustness with 91.79% precision, 93% recall, 77.66% F1 score, 88.28% AUC-ROC, and a low false positive rate of 36.24% at 95% TPR. Based on these findings LightMask balances computational costs with robust anomaly detection capabilities.