Figure 1 - uploaded by Charlyn Pushpa Latha
Content may be subject to copyright.
Source publication
Deep Learning is the recent machine learning technique that tries to model high level abstractions in data by using multiple processing layers with complex structures. It is also known as deep structured learning, hierarchical learning or deep machine learning. The term deep learning indicates the method used in training multi-layered neural networ...
Similar publications
Deep Learning is the recent machine learning technique that tries to model high level abstractions in data by using multiple processing layers with complex structures. It is also known as deep structured learning, hierarchical learning or deep machine learning. The term “deep learning" indicates the method used in training multi-layered neural netw...
Citations
... Researchers integrate the three steps of face location, face detection and feature extraction into an end-to-end emotion recognition system. Because of this, Pushpa and Priya [14] used Deep Boltzman Machine (DBM) and CNN for emotion analysis. Compared with traditional methods, deep learning methods greatly improved the ability of emotion analysis. ...
Humans express their emotions in a variety of ways, which inspires research on multimodal fusion-based emotion recognition that utilizes different modalities to achieve information complementation. However, extracting deep emotional features from different modalities and fusing them remain a challenging task. It is essential to exploit the advantages of different extraction and fusion approaches to capture the emotional information contained within and across modalities. In this paper, we present a novel multimodal emotion recognition framework called multimodal emotion recognition based on cascaded multichannel and hierarchical fusion (CMC-HF), where visual, speech, and text signals are simultaneously utilized as multimodal inputs. First, three cascaded channels based on deep learning technology perform feature extraction for the three modalities separately to enhance deeper information extraction ability within each modality and improve recognition performance. Second, an improved hierarchical fusion module is introduced to promote intermodality interactions of three modalities and further improve recognition and classification accuracy. Finally, to validate the effectiveness of the designed CMC-HF model, some experiments are conducted to evaluate two benchmark datasets, IEMOCAP and CMU-MOSI. The results show that we achieved an almost 2%∼3.2% increase in accuracy of the four classes for the IEMOCAP dataset as well as an improvement of 0.9%∼2.5% in the average class accuracy for the CMU-MOSI dataset when compared to the existing state-of-the-art methods. The ablation experimental results indicate that the cascaded feature extraction method and the hierarchical fusion method make a significant contribution to multimodal emotion recognition, suggesting that the three modalities contain deeper information interactions of both intermodality and intramodality. Hence, the proposed model has better overall performance and achieves higher recognition efficiency and better robustness.
... In addition, it can be applied to a variety of problems, for example, classification and segmentation. In the emotional researches, ML and DL have addressed various tasks, including face detection and tracking, speech recognition, voice-activity detection, and emotion classification from face and voice [39]. ...
Emotion plays an important role in our daily lives. Ever since the 19th century, experimental psychologists have attempted to understand and explain human emotion. Despite an extensive amount of research conducted by psychologists, anthropologists, and sociologists over the past 150 years, researchers still cannot agree on the definition of emotion itself and have continued to try and devise ways to measure emotional states. In this paper, we provide an overview of the most prominent theories in emotional psychology (dating from the late 19th century to the present day), as well as a summary of a number of studies which attempt to measure certain aspects of emotion. This paper is organized chronologically; first with an analysis of various uni-modal studies, followed by a review of multi-modal research. Our findings suggest that there is insufficient evidence to neither prove nor disprove the existence of coherent emotional expression, both within subjects and between subjects. Furthermore, the results seem to be heavily influenced by both experimental conditions as well as by the theoretical assumptions that underpin them.
... Automatic recognition of emotion has been researched actively because it has many useful applications for humancentric services and human-computer interactions. Various modalities such as facial expression, affective speech, and gesture have been explored for detecting emotion [1,2]. In addition, cerebral signals, in particular electroencephalography (EEG), have received much attention in recent years along with the noticeable development of sensing devices, which is expected to contain the comprehensive information of emotion [3]. ...
Emotion recognition based on electroencephalography (EEG) has received attention as a way to implement human-centric services. However, there is still much room for improvement, particularly in terms of the recognition accuracy. In this paper, we propose a novel deep learning approach using convolutional neural networks (CNNs) for EEG-based emotion recognition. In particular, we employ brain connectivity features that have not been used with deep learning models in previous studies, which can account for synchronous activations of different brain regions. In addition, we develop a method to effectively capture asymmetric brain activity patterns that are important for emotion recognition. Experimental results confirm the effectiveness of our approach.
... Facial expressions recognition (FER) has a wide spectrum of application potentials in human-computer interaction, cognitive psychology, computational neuroscience, and medical healthcare. In recent years, convolutional neural networks (CNN) have achieved many exciting results in artificial intelligent and pattern recognition and have been successfully used in facial expression recognition [1]. Jaiswal et al. [2] present a novel approach to facial action unit detection using a combination of Convolutional and Bidirectional Long Short-Term Memory Neural Networks (CNN-BLSTM), which jointly learns shape, appearance, and dynamics in a deep learning manner. ...
In the facial expression recognition task, a good-performing convolutional neural network (CNN) model trained on one dataset (source dataset) usually performs poorly on another dataset (target dataset). This is because the feature distribution of the same emotion varies in different datasets. To improve the cross-dataset accuracy of the CNN model, we introduce an unsupervised domain adaptation method, which is especially suitable for unlabelled small target dataset. In order to solve the problem of lack of samples from the target dataset, we train a generative adversarial network (GAN) on the target dataset and use the GAN generated samples to fine-tune the model pretrained on the source dataset. In the process of fine-tuning, we give the unlabelled GAN generated samples distributed pseudolabels dynamically according to the current prediction probabilities. Our method can be easily applied to any existing convolutional neural networks (CNN). We demonstrate the effectiveness of our method on four facial expression recognition datasets with two CNN structures and obtain inspiring results.
... It has many advantages when compared to traditional machine learning techniques. DL is able to detect complex interactions among features, capable to learn low- level features from minimally processed raw data and able to work with unlabeled data (Latha and Priya, 2016). ...
Intelligent machine translation systems have a remarkable importance in integrating people with disabilities in community. Arabic to Arabic sign language systems are limited. Deep Learning (DL) was successfully applied to problems related to music information retrieval, image recognition and text recognition, but its use in sign language recognition is rare. This paper introduces an automatic virtual translation system from Arabic language into Arabic Sign Language (ASL) via a popular DL architecture: The Recurrent Neural Network (RNN). The proposed system uses a deep neural network training-based system for ASL that convolves RNN and Graphical Processing Unit (GPU) parallel processors. The system is evaluated using both objective and subjective measures. Obtained results are towards reducing errors, speeding up avatar and expressing signs and facial expressions in a well-received manner by Deaf. The signing avatar is highly encouraged as a simulator for natural human signs.
Recent advancements in human-computer interaction research of Internet-of-Emotion have led to the possibility of emotional communication via the human-computer interface for a user with neuropsychiatric disorders or disabilities. There are several ways of recording psychophysiology data from humans, and in this paper, we focus on emotion detection using electroencephalogram (EEG). Various emotion extraction techniques can be used on the recorded EEG data to classify emotional states. Band energy (E), frequency band energy ratio (REE), the logarithm of the frequency band energy ratio (LREE), and differential entropy (DE) of band energy ratio is some emotion features that previously have been used to classify EEG data in various emotional states. Four different emotion features were analyzed in this paper, classifying EEG data associated with specific emotional states. The results showed that DE is the best choice in Wavelet Transform-Support Vector Machine (WT-SVM) model during the period of training an SVM classifier to be accurate over the whole data sets (16 subjects), and the whole accuracy up to 86.5%, while DE’s classification results are between 73.81% to 97.62%. This phenomenon shows that it is difficult to find features that are generally working well over each subject, and there is also the possibility that the pictures of the International Affective Picture System (IAPS) did not induce strong enough emotions on some subjects making it difficult to classify some emotional states. Based on the result, we only conclude that DE is the optimal choice for the WT-SVM model, and the individual factors will be considered as one of the influencing factors of the emotion classification system in future work.KeywordsPattern recognitionEmotion computingFeature extractionDE
