CAFusion: A progressive ConvMixer network for context-aware infrared and visible image fusion

Abstract

Image fusion is a challenging task that aims to generate a composite image by combining information from diverse sources. While deep learning (DL) algorithms have achieved promising results, most rely on complex encoders or attention mechanisms, leading to high computational cost and potential information loss during one-step feature fusion. We introduce CAFusion, a DL framework for visible (VI) and infrared (IR) image fusion. In particular, we propose a context-aware ConvMixer block that uniquely integrates dilated convolutions for expanded receptive fields with depthwise separable convolutions for parameter efficiency. Unlike existing CNN or transformer-based modules, our block captures multi-scale contextual information without attention mechanisms, with computational efficiency. Additionally, we employ an attention-based intermodality multi-level progressive fusion strategy, ensuring an adaptive combination of multi-scale modality-specific features. A hierarchical multiscale decoder reconstructs the fused image by aggregating information across different levels, preserving low and high-level details. Comparative evaluations of benchmark datasets demonstrate that CAFusion outperforms recent transformer-based and SOTA DL-based approaches in fusion quality and computational efficiency. In particular, on the TNO benchmark dataset, CAFusion achieves a 0.769 score in the structural similarity index measure, a 2.07 percent increase as compared to the best competing method.