MULTIMODAL EMOTION DETECTION USING MACHINE LEARNING TECHNIQUES AND DATA ANALYSIS

In this project we have implemented facial feature extraction and detection to detect 7 categories of emotions in a person, namely ‘happy', ‘sad', ‘surprise', ‘anger', ‘fear', ‘disgust' and ‘stress'. This is done by special type of Deep Learning technique known as Convolution Neural Network (CNN) framework to detect the emotions. We report accuracies of 91.8% in the FER-2013 dataset. These emotions are checked by using real time webcam emotion feed, which checks emotion frame-by-frame. Since age and gender also play a significant role in our general day-to-day social interactions hence, we have tried to bridge the gap between age and gender estimation methods and real time face recognition through deep convolution networks. We report accuracies of 93% in gender classification and about 50% in age classification though the IMDb-Wikipedia dataset. Further, we have developed a machine learning model to detect the emotions through speech. Accuracy of 98 % has been reported for speech classification using RAVDESS and SAVEE datasets. Along with this we have prepared a self-assessment test for emotion detection and later on for mental health being. We prepared a test for emotion detection in which percentage of dominating emotion among: ‘happy', ‘sad', ‘depression', ‘anxiety', ‘fear' and ‘anger' is calculated. Based on the score obtained, the person is asked to attend a Computer Adaptive test (CAT) to check percentage of ‘anger', ‘depression' or ‘anxiety', and a proper solution to improve their mental health is provided according to the score they got.