A lightweight hybrid deep learning approach for fashion mnist classification with explainable attention visualization

Abstract

The classification of fashion images is an essential task in the e-commerce sector, where accurate categorization improves user experience and refines product discovery. Convolutional Neural Networks (CNNs) and Transformers have demonstrated strong performance in image classification tasks due to their ability to learn complex visual features. However, deep variants of these architectures, such as VGG-19, ResNet-50, Vision Transformer (ViT), and Swin Transformer, contain tens of millions of parameters, requiring high memory and powerful GPUs for training, which makes them less suitable for low-resource and edge device environments. To address these limitations, this research proposes a lightweight hybrid architecture, TinyCNN with Linear Self-Attention (LSA), optimized for resource-constrained settings. The proposed model contains fewer than half a million parameters and is trainable on a CPU, achieving a classification accuracy of 91.47% on the Fashion-MNIST dataset. In addition, multiple Explainable Artificial Intelligence (XAI) techniques are implemented, including Self-Attention visualization, Multi-Head Attention, Attention Flow, Attention Rollout, Fixed query position attention maps, Integrated Gradients, LIME, and SHAP, to provide visual interpretability of the model’s predictions and enhance transparency in its decision-making process.