LDMP-RENet: Reducing intra-class differences for metal surface defect few-shot semantic segmentation

Abstract

Given their fast generalization capability for unseen classes and segmentation ability at pixel scale, models based on few-shot segmentation perform well in solving data insufficiency problems during metal defect detection and in delineating refined objects under industrial scenarios. Extant researches fail to consider the inherent intra-class differences in data about metal surface defects, so that the models can hardly learn enough information from the support set for guiding the segmentation of query set. Specifically, it can be categorized into two types: the semantic intra-class difference induced by internal factors in metal samples and the distortion intra-class difference caused by external factors of surroundings. To address these differences, we introduce a Local Descriptor-based Multi-Prototype Reasoning and Excitation Network (LDMP-RENet) to learn the two-view guidance, i.e., the local information from the graph space and the global information from the feature space, and fuse them to segment precisely. Given the contribution of relational structure of graph space-embedded local features to the Semantic Difference obviation, a multi-prototype reasoning module is utilized to extract local descriptors-based prototypes and to assess relevance between local-view features in support-query set pairs. Meanwhile, since global information helps obviate Distortion Difference in observations, a multi-prototype excitation module is employed for capturing global-view relevance in the above pairs. Lastly, an information fusion module is employed to integrate the learned prototypes in both global and local views, thereby creating pixel-level masks. Thorough experiments are conducted on defect datasets, revealing the superiority of proposed network to extant benchmarks, which sets a new state-of-the-art.