Top-down and bottom-up attention for joint pattern classification and reconstruction

Abstract

We introduce a recurrent inference framework for the Classification and Reconstruction of Overlapping Patterns (CROP) in mixtures formed by overlapping two patterns drawn from the same distribution. The framework alternates between bottom-up classification and top-down generative reconstruction within an iterative inference procedure. At each iteration, the method estimates the most likely class present in the mixture, reconstructs the corresponding signal using a conditional generative model, and applies a mask to isolate that component. This classification-guided reconstruction progressively separates the overlapping signals while also producing their class labels. The objective is therefore to iteratively separate and classify the overlapping patterns rather than perform general blind source separation. An important feature of the framework is that the generative model can be trained using only clean samples, without requiring paired mixed–clean training data. The iterative procedure implicitly implements a form of attention in which saliency- and priority-driven estimates guide the masking and reconstruction of individual patterns. Experimental results on mixtures of handwritten digits show that the proposed framework can successfully separate and classify overlapping patterns through this iterative classification–reconstruction process.