Figure - available from: Frontiers in Human Neuroscience
This content is subject to copyright.
Source publication
The research shows that subjective feelings of people, such as emotions and fatigue, can be objectively reflected by electroencephalography (EEG) physiological signals Thus, an evaluation method based on EEG, which is used to explore auditory brain cognition laws, is introduced in this study. The brain cognition laws are summarized by analyzing the...
Similar publications
Introduction
Establishing a driving fatigue monitoring system is of utmost importance as severe fatigue may lead to unimaginable consequences. Fatigue detection methods based on physiological information have the advantages of reliable and accurate. Among various physiological signals, EEG signals are considered to be the most direct and promising...
Citations
... The correlation between automotive sound quality and EEG signals has been initially explored [17,19,29,30]; however, there is little published data on the analysis of the correlation between EEG physiological signals and the physical acoustic and psychoacoustic parameters of sounds, and the studies on the construction of evaluation models from the above three aspects are much fewer. Equally, the studies on the effect of scene video for the evaluation results of sound quality have only been carried out in a small number of areas, where the existence of the effect is demonstrated, but the correction function which can objectively quantify the extent of effect has not been proposed. ...
... In the uniformity test, I 1,n is defined as the set of invalid participants for the n th semantic evaluation index without scene video, and I 2,n as the set of invalid participants with scene video: I 1,1 = [2,17,19,20,23] with the total number of 5, I 1,2 = [4,19,23,27] with the total number of 4, and I 1,3 = [2, 3,4,5,8,12,14,16,17,18,19,20,23,24,28,29] with the total number of 16; on the contrary, I 2,1 = [19] with the total number of 1, I 2,2 = [23,29] with the total number of 2, and I 2,1 = [10,13,18,19,21] with the total number of 5. It is noted from the above results that the number of invalid participants without scene stimulus is more than with scene stimulus, indicating that the subjective evaluation results are more consistent between the participants with the video scenes. ...
... In the uniformity test, I 1,n is defined as the set of invalid participants for the n th semantic evaluation index without scene video, and I 2,n as the set of invalid participants with scene video: I 1,1 = [2,17,19,20,23] with the total number of 5, I 1,2 = [4,19,23,27] with the total number of 4, and I 1,3 = [2, 3,4,5,8,12,14,16,17,18,19,20,23,24,28,29] with the total number of 16; on the contrary, I 2,1 = [19] with the total number of 1, I 2,2 = [23,29] with the total number of 2, and I 2,1 = [10,13,18,19,21] with the total number of 5. It is noted from the above results that the number of invalid participants without scene stimulus is more than with scene stimulus, indicating that the subjective evaluation results are more consistent between the participants with the video scenes. ...
The evaluation of automobile sound quality is an important research topic in the interior sound design of passenger car, and the accurate and effective evaluation methods are required for the determination of the acoustic targets in automobile development. However, there are some deficiencies in the existing evaluation studies of automobile sound quality. (1) Most of subjective evaluations only considered the auditory perception, which is easy to be achieved but does not fully reflect the impacts of sound on participants; (2) similarly, most of the existing subjective evaluations only considered the inherent properties of sounds, such as physical and psychoacoustic parameters, which make it difficult to reflect the complex relationship between the sound and the subjective perception of the evaluators; (3) the construction of evaluation models only from physical and psychoacoustic perspectives does not provide a comprehensive analysis of the real subjective emotions of the participants. Therefore, to alleviate the above flaws, the auditory and visual perceptions are combined to explore the inference of scene video on the evaluation of sound quality, and the EEG signal is introduced as a physiological acoustic index to evaluate the sound quality; simultaneously, an Elman neural network model is constructed to predict the powerful sound quality combined with the proposed indexes of physical acoustics, psychoacoustics, and physiological acoustics. The results show that evaluation results of sound quality combined with scene videos better reflect the subjective perceptions of participants. The proposed objective evaluation indexes of physical, psychoacoustic, and physiological acoustic contribute to mapping the subjective results of the powerful sound quality, and the constructed Elman model outperforms the traditional back propagation (BP) and support vector machine (SVM) models. The analysis method proposed in this paper can be better applied in the field of automotive sound design, providing a clear guideline for the evaluation and optimization of automotive sound quality in the future.
... Recent studies have suggested that EEG signals are a physiological indicator that contributes to reflecting individual variability and objectively mapping the intuitive perception of different evaluators for sound [8,9]. There are existing studies that have proved that EEG signals can be utilized to evaluate automobile sound quality [10,11]. In addition, different drivers have different demands for the sound quality of different automobiles, and the design of switching modes with multiple types of driving sounds helps to enhance the brand image and competitiveness of automobiles [12]. ...
The advancement of an intelligent automobile sound switching system has the potential to elevate the market standing of automotive products, with the pivotal prerequisite being the selection of automobile sounds based on the driver’s subjective perception. The subjective responses of diverse individuals to sounds can be objectively manifested through EEG signals. Therefore, EEG signals are employed herein to attain the recognition of automobile sounds. A subjective evaluation and EEG signal acquisition experiment are designed involving the stimulation of three distinct types of automobile sounds, namely comfort, power, and technology sounds, and a comprehensive database of EEG signals corresponding to these three sound qualities is established. Then, a specific transfer learning model based on a convolutional neural network (STL-CNN) is formulated, where the method of training the upper layer parameters with the fixed bottom weights is proposed to adaptively extract the EEG features related to automobile sounds. These improvements contribute to improving the generalization ability of the model and realizing the recognition of automobile sounds fused with EEG signals. The results of the comparison with traditional support vector machine (SVM) and convolutional neural network (CNN) models demonstrate that the accuracy of the test set of the STL-CNN model reaches 91.5%. Moreover, its comprehensive performance, coupled with the ability to adapt to individual differences, surpasses that of both SVM and CNN models. The demonstrated method in the recognition of automobile sounds based on EEG signals is of significance for the future implementation of switching driving sound modes fused with EEG signals.
... Priyasad et al. [40] proposed a new method to recognize emotion through unprocessed EEG signals, and it was found that, with the new method, the classification accuracy for arousal, valence and dominance could be higher than 88%. Xie et al. [41] presented an assessment approach based on EEG signals. They first determined the brain cognition laws by evaluating the EEG power topographic map under the effect of three types of automobile sound, i.e., the qualities of comfort, powerfulness, and acceleration, then classified the EEG features thus recognized as different automobile sounds, and finally employed the Kalman smoothing and minimal redundancy maximal relevance algorithm to modify the accuracy and reduce the amount of calculation. ...
People’s emotions play an important part in our daily life and can not only reflect psychological and physical states, but also play a vital role in people’s communication, cognition and decision-making. Variations in people’s emotions induced by external conditions are accompanied by variations in physiological signals that can be measured and identified. People’s psychological signals are mainly measured with electroencephalograms (EEGs), electrodermal activity (EDA), electrocardiograms (ECGs), electromyography (EMG), pulse waves, etc. EEG signals are a comprehensive embodiment of the operation of numerous neurons in the cerebral cortex and can immediately express brain activity. EDA measures the electrical features of skin through skin conductance response, skin potential, skin conductance level or skin potential response. ECG technology uses an electrocardiograph to record changes in electrical activity in each cardiac cycle of the heart from the body surface. EMG is a technique that uses electronic instruments to evaluate and record the electrical activity of muscles, which is usually referred to as myoelectric activity. EEG, EDA, ECG and EMG have been widely used to recognize and judge people’s emotions in various situations. Different physiological signals have their own characteristics and are suitable for different occasions. Therefore, a review of the research work and application of emotion recognition and judgment based on the four physiological signals mentioned above is offered. The content covers the technologies adopted, the objects of application and the effects achieved. Finally, the application scenarios for different physiological signals are compared, and issues for attention are explored to provide reference and a basis for further investigation.
... In the aspect of emotion recognition, electroencephalogram (EEG) has been paid more attention by researchers among many physiological signals. The analysis of EEG signals in the field of emotion recognition depends on data preprocessing, feature extraction, and feature classification (Xie et al., 2021). Many researchers use traditional machine learning or deep neural network to classify EEG signals by extracting the energy features of the delta, theta, alpha, beta, and gamma bands. ...
In emotion recognition based on physiological signals, collecting enough labeled data of a single subject for training is time-consuming and expensive. The physiological signals’ individual differences and the inherent noise will significantly affect emotion recognition accuracy. To overcome the difference in subject physiological signals, we propose a joint probability domain adaptation with the bi-projection matrix algorithm (JPDA-BPM). The bi-projection matrix method fully considers the source and target domain’s different feature distributions. It can better project the source and target domains into the feature space, thereby increasing the algorithm’s performance. We propose a substructure-based joint probability domain adaptation algorithm (SSJPDA) to overcome physiological signals’ noise effect. This method can avoid the shortcomings that the domain level matching is too rough and the sample level matching is susceptible to noise. In order to verify the effectiveness of the proposed transfer learning algorithm in emotion recognition based on physiological signals, we verified it on the database for emotion analysis using physiological signals (DEAP dataset). The experimental results show that the average recognition accuracy of the proposed SSJPDA-BPM algorithm in the multimodal fusion physiological data from the DEAP dataset is 63.6 and 64.4% in valence and arousal, respectively. Compared with joint probability domain adaptation (JPDA), the performance of valence and arousal recognition accuracy increased by 17.6 and 13.4%, respectively.
There is a bottleneck in the design of vehicle sound that the subjective perception of sound quality that combines multiple psychological factors fails to be accurately and objectively quantified. Therefore, EEG signals are introduced in this paper to investigate the evaluation and design method of vehicle acceleration sound with powerful sound quality. Firstly, the experiment of EEG acquisition and subjective evaluation under the stimulation of powerful vehicle sounds is conducted, respectively, then three physiological EEG features of PSD_β, PSD_γ and DE are constructed to evaluate the vehicle sounds based on the correlation analysis algorithms. Subsequently, the Adaptive Genetic Algorithm (AGA) is proposed to optimize the Elman model, where an intelligent model (AGA–Elman) is constructed to objectively predicate the perception of subjects for the vehicle sounds with powerful sound quality. The results demonstrate that the error of the constructed AGA–Elman model is only 2.88%, which outperforms than the traditional BP and Elman model; Finally, two vehicle acceleration sounds (Design1 and Design2) are designed based on the constructed AGA–Elman model from the perspective of order modulation and frequency modulation, which provide the acoustic theoretical guidance for the design of vehicle sound incorporating the EEG signals.
The evaluation of automobile sound quality is not only related to the inherent properties of the sound, but also to the psychological and physiological state of the evaluator, which makes the evaluation of sound quality become an interdisciplinary research. However, there are some deficiencies regarding the existing studies as follows: (1) it is imprecise enough to visualize the preference of evaluators for sound on the scale; (2) the evaluators without acoustic experience are prone to deviations in their evaluations, resulting in the universally and unapplicable results; (3) the automobile sound quality cannot be fully reflected only by physical and psychological parameters, and most of the existing evaluation models of sound quality are only characterized by shallow architectures. To alleviate the above flaws, a hybrid deep neural network (HDNN) is constructed to achieve the evaluation of automobile interior acceleration sound fused with physiological signals in this paper. An EEG test paradigm under the stimulation of automobile sound is meanwhile designed for a feasibility confirmation of the proposed method. In addition, the acquired EEG datum are respectively analyzed from two perspectives: the manual extraction of typical EEG features and the adaptive extraction of EEG features based on HDNN model. Furthermore, the performance of the proposed HDNN is also validated by comparing with conventional convolutional neural network (CNN) and long short time memory (LSTM). The results indicate that the adaptively extracted EEG features are superior to the manually extracted EEG features, and the proposed HDNN outperforms the CNN and LSTM in terms of accuracy, sensitivity as well as precision, and the optimal accuracy of HDNN is up to 88.1%. Thus, the constructed HDNN contributes to achieving accurate evaluation of automobile sound quality with EEG signal.
