Hand Sign Detection for the hearing and speech Impaired using Convolutional Neural Network

Abstract

Learning and utilizing sign languages to communicate with deaf and mute persons has been easier because to developments in computer vision technology. The development of a global platform for communication in a variety of sign languages is an attractive area of research. In this research, we introduce a Deep Learning-based method that uses an image as input to recognize special sign in Indian Sign Language. When the user enters digits 0 through 9, the system can guess their signs. The Convolutional Neural Network's training time and storage needs are effectively decreased by using image processing to transform RGB data to grayscale images. The experiment's goal is to identify a less sophisticated deep learning architecture and image processing is used to implement the system on embedded single-board computers or mobile applications. The database is trained from scratch using small networks like LeNet-5 and AlexNet, as well as deeper networks like Vgg16 and MobileNet v2. Comparing the accuracy of recognition is covered in the study. A dropout layer that improved training and testing accuracy to 91.37% and 87.5%, respectively, is one of just 10 layers that comprise the final design that was selected.