Virtual sample based techniques using deep features for SSPP face recognition in unconstrained environment

Abstract

As challenging as it is to use face recognition with a Single Sample Per Person, it becomes even more difficult when face recognition based on a single sample is performed in an unconstrained environment. The unconstrained environment is normally considered irregular in facial expressions, pose, occlusion, and illumination. This degree of difficulty increases as a result of the single sample and in the presence of occlusion. Extensive research has been done on face recognition under pose and expression changes. Comparatively, less research has been reported on the occlusion problem that occurs in facial images. Occlusion may alter the appearance of facial images and cause deterioration in recognition. A robust method is required to handle the occlusion in the face image to improve the recognition performance. This study aimed to implement an effective augmentation technique that improves the performance of the Single Sample Per Person face recognition system in unconstrained environments. Virtual samples were created to expand the sample size to address the problem of a single sample. A local region-based technique was proposed to deal with occlusion by creating virtual samples. A deep neural network-based model, FaceNet, was used to extract the features and a support vector machine was used for classification. The performance of the proposed approach was evaluated, demonstrating its superiority in handling occlusion compared to that of its state-of-the-art counterparts. The proposed method achieved significant accuracy improvements, specifically 94.83% for the occlusion with sunglasses and 98% for the occlusion with scarves in the AR dataset.