Multi-Modal Emotion Recognition Using Ensemble Deep Learning and DeepNet Model

Abstract

Human speech signals are a vital source for determining human emotions. Classic datasets of human speech signals can be explored for many applications in healthcare, gaming, and advertisement domains. Emotion recognition using Deep Learning (DL) algorithms can facilitate the assessment of well-informed decision-making. Existing DL algorithms have limitations in their predictive performance as they do not leverage feature engineering in their approach. In this work, we propose a novel DeepNet Base Audio Emotion Recognition Framework to achieve automation in the identification of human emotions by means of supervised learning to enhance predictive performance. Proposed DeepNet incorporates algorithms namely the DeepNet Base Audio Emotion Recognition Framework algorithm to efficiently identify features having discriminative capability and Recognition (DeepNet) algorithm which uses Multi-Layer Perceptron for automatic emotion recognition. DeepNet algorithms are trained on the RAVDESS dataset for empirical study which has a broad collection of distinct natural human emotions such as angry, neutral, disgust, fear, surprise, sad, and happiness. Further, a web-application is designed to recognize human emotion based on the saved DeepNet model. Results from the application suggest that the DeepNet algorithm outperformed standard algorithms such as Bi-LSTM and Ensemble Model-D tested on the same dataset. Bi-LSTM process the data forward and backwards using Bi-directional layers from input to output. Whereas the ensemble models built various assorted base model, but fails to interpret generalization type error, Bi-LSTM achieves an accuracy of 81.53%, while the Ensemble Model-D reaches 85.22%. The performance efficiency of the proposed DeepNet algorithm-based DeepNet model has 88.9% better predictive performance than the conventional
existing algorithm.