CNN-LSTM based emotion recognition using Chebyshev moment and K-fold validation with multi-library SVM

Abstract

Human emotions are not necessarily tends to produce right facial expressions as there is no well defined connection between them. Although, human emotions are spontaneous, their facial expressions depend a lot on their mental and psychological capacity to either hide it or show it explicitly. Over a decade, Machine Learning and Neural Networks methodologies are most widely used by the researchers to tackle these challenges, and to deliver an improved performance with accuracy. This paper focuses on analyzing the driver’s facial expressions to determine their mood or emotional state while driving to ensure their safety. we propose a hybrid CNN-LSTM model in which RESNET152 CNN is used along with Multi-Library Support Vector Machine for classification purposes. For the betterment of feature extraction, this study has considered Chebyshev moment which plays an important role as it has a repetition process to gain primary features and K-fold validation helps to evaluate the models performance in terms of both training, validation loss, training, and validation accuracy. This study performance was evaluated and compared with existing hybrid approaches like CNN-SVM and ANN-LSTM where the proposed model delivered better results than other models considered.