SegCycle-SPADE: An end-to-end framework for semantic segmentation-based automated extraction and artistic reconstruction of traditional craft patterns using conditional GAN

Abstract

This paper addresses the challenge of traditional handicraft pattern extraction and generation, focusing on accurate segmentation and high-quality pattern reconstruction. We propose the SegCycle-SPADE model, combining SegFormer for semantic segmentation, CycleGAN for pattern generation, and SPADE for style transfer, to achieve a balance between segmentation accuracy, generation quality, and inference efficiency. Experiments on datasets such as Batik Nitik 960, Fashion-MNIST, and DeepFashion show that SegCycle-SPADE outperforms baseline models like U-Net and DeepLabV3+ with significant improvements in PA (88.6%), mIoU (78.1%), and Boundary F1 (73.8%). In terms of pattern generation, SegCycle-SPADE also achieves superior results in PSNR (27.8 dB), SSIM (0.89), and FID (34.2), outperforming Pix2Pix, Stable Diffusion, and other models. The model demonstrates its potential for the digital regeneration of traditional handicraft patterns, offering a robust solution for high-quality and efficient pattern generation, with substantial contributions to digital cultural heritage preservation and innovation.