ArticlePDF Available

Continuous Estimation of Stress Using Physiological Signals during a Car Race

Authors:

Abstract and Figures

Mental stress refers to the feeling of strain and anxiety caused by internal and/or external factors. Stress may have both positive and negative influences on cognitive performance. For example, small amounts of stress may improve concentration, athletic performance, and reaction speed. However, excessive stress may harm health, and disturb concentration and motor control. Furthermore , increase in short-term stress has a considerable effect on physiological processes such as heart rate and sweating. However, to our knowledge, so far no study has estimated mental stress through continuous monitoring of physiological signals under a situation with various stressors. Therefore, it remains unclear how different physiological signals correlate with each other and how they change continuously in response to the changes in stressors. Here, we measured heart rate variability, galvanic skin response, and activity of the masseter muscle in a professional racer during a real car race. Car racing is one of the most dangerous sports, and the competition with other racers and physical discomfort during the race induce great short-term stresses. We used factor analysis to examine the relation between the three types of physiological signals, and clarified the events associated with stress (e.g., acceleration , competing car in vision, overtaking). The results showed that the heart rate variability and galvanic skin response correlated with each other, and were associated with the event of competition, which brought great mental stress. In contrast, activity of the masseter muscle did not correlate with the other two physiological signals, but was associated with the events of acceleration or deceleration, which brought great physical discomfort. We concluded that heart rate variability and galvanic skin response reflect mental stress, which is associated with an internal desire to win and is triggered by changes in the external world (e.g., the appearance of competing cars). In contrast, masseter muscle activity reflects the endurance of body discomfort, which is not linked to any internal state. Our results indicated that internal and external stressors may be dissociated from each other, and may affect different or-How to cite this paper:
Content may be subject to copyright.
Psychology, 2017, 8, 978-986
http://www.scirp.org/journal/psych
ISSN Online: 2152-7199
ISSN Print: 2152-7180
DOI: 10.4236/psych.2017.87064 May 19, 2017
Continuous Estimation of Stress Using
Physiological Signals during a Car Race
Wen Wen1, Daisuke Tomoi1, Hiroshi Yamakawa1, Shunsuke Hamasaki1, Kaoru Takakusaki2,
Qi An1, Yusuke Tamura1, Atsushi Yamashita1, Hajime Asama1
1Department of Precision Engineering, University of Tokyo, Tokyo, Japan
2The Center for Brain Function and Medical Engineering, Asahikawa Medical College, Asahikawa, Japan
Abstract
Mental stress refers to the feeling of strain
and anxiety caused by internal
and/or external factors. Stress may have both positive and negative influences
on cognitive performance. For example, small amounts of stress may improve
concentration, athletic performance, and reaction speed. However, exces
sive
stress may harm health, and disturb concentration and motor control. Fu
r-
thermore, increase in short-term stress has a considerable effect on physi
o-
logical processes such as heart rate and sweating. However, to our knowledge,
so far no study has estima
ted mental stress through continuous monitoring of
physiological signals under a situation with various stressors. Therefore, it
remains unclear how different physiological signals correlate with each other
and how they change continuously in response to t
he changes in stressors.
Here, we measured heart rate variability, galvanic skin response, and activity
of the masseter muscle in a professional racer during a real car race. Car ra
c-
ing is one of the most dangerous sports, and the competition with other ra
cers
and physical discomfort during the race induce great short-
term stresses. We
used factor analysis to examine the relation between the three types of phys
i-
ological signals, and clarified the events associated with stress (e.g., acceler
a-
tion, competing
car in vision, overtaking). The results showed that the heart
rate variability and galvanic skin response correlated with each other, and
were associated with the event of competition, which brought great mental
stress. In contrast, activity of the massete
r muscle did not correlate with the
other two physiological signals, but was associated with the events of acceler
a-
tion or deceleration, which brought great physical discomfort. We concluded
that heart rate variability and galvanic skin response reflect me
ntal stress,
which is associated with an internal desire to win and is triggered by changes
in the external world (e.g., the appearance of competing cars). In contrast,
masseter muscle activity reflects the endurance of body discomfort, which is
not linked to any internal state. Our results indicated that internal and exte
r-
nal stressors may be dissociated from each other, and may affect different or-
How to cite this paper:
Wen, W.,
Tomoi,
D
., Yamakawa, H., Hamasaki, S., Takaku-
saki, K
., An, Q., Tamura, Y., Yamashita, A.,
&
Asama, H. (2017). Continuous Estima-
tion of Stress Using Physiological Signals
during a Car Race
.
Psychology, 8,
978-986.
https://doi.org/10.4236/psych.2017.87064
Received:
April 5, 2017
Accepted:
May 16, 2017
Published:
May 19, 2017
Copyright © 201
7 by authors and
Scientific
Research Publishing Inc.
This work is licensed under
the Creative
Commons Attribution International
License (CC BY
4.0).
http://creativecommons.org/licenses/by/4.0/
Open Access
W. Wen et al.
979
gans and physiological receptors.
Keywords
Stress, Physiological Signals, Heart Rate, Galvanic Skin Response,
Masseter Muscle Activity, Factor Analysis
1. Introduction
In psychology, stress refers to the mental state of feeling strain and anxiety, and
is usually caused by changes in the environment or internal desire. Small
amounts of stress could be helpful to improve concentration and athletic per-
formance. However, excessive or chronic stress usually causes depression, harms
health, and results in errors in decision-making or motor control. There are
many types of stressors, such as crises, threats, daily annoyances, and life events.
In the present study, we focused on short-term stress, which arises from rapid
environmental changes such as threat, cognitive load, and emotions.
Stress is a significant subjective experience that is difficult to estimate quanti-
tatively from a subjective perspective. An individual may be able to report
whether he/she is undergoing great or small stress, but is not able to quantify it,
or to compare the amount of stress between different stressors. However, pre-
vious studies have attempted to measure physiological reactions such as heart-
beat, sweating, muscle activities, and blood pressure in stressful situations as an
approach to detect or estimate the amount of stress (Hidaka, Yanagi, & Takada,
2004b; Hjortskov et al., 2004; Kurniawan, Maslov, & Pechenizkiy, 2013; Shi et al.,
2007; Thayer, Ahs, Fredrikson, Sollers, & Wager, 2012; Yemm, 1969). Previous
studies have revealed useful links between some physiological signals and stress.
However, none of these studies has examined physiological signals in a situation
with multiple and dynamically changing stressors.
In the present study, we focused on the situation of a real car race. We conti-
nuously measured three types of physiological signals in a professional driver
during a real race. Car racing is one of the most dangerous sports, and requires
continuous and rapid decisions during a relatively long duration. Both the com-
petition and physical discomfort could be stressors, and multiple stressors may
change rapidly. The present study aimed to clarify the relations between multiple
physiological signals (i.e. heartbeat, sweating, and masseter muscle activity) in
such a situation, and to elucidate the relation between physiological signals and
different stressors. Our present findings could not only provide useful estima-
tion of stress during the specific situation of car race but also contribute to the
understanding of the subjective experience of stress.
2. Methods
2.1. Participant and Experimental Environment
The present study focused on within-individual correlation in a highly specific
W. Wen et al.
980
situation. Therefore, we invited a professional race driver (male, age 27 years) to
take part in a real car race, and measured physiological signals during the race.
The study was approved by the ethics committee of the Faculty of Engineering at
the University of Tokyo, and written informed consent was obtained from the
participant prior to the experiment.
The experiment was performed during a practice session of a race, in which all
racers took part in order to get used to the racing course in advance of the actual
race. The racing condition was the same as the actual one, in which the racers
were free to overtake other cars. Figure 1 illustrates the map of the racing course
of the present study. The racing course contained 12 corners and two slopes, and
therefore required highly precise control of speed and direction.
2.2. Equipment
The present study measured the participant’s heart rate variability (HRV), gal-
vanic skin response (GSR), and muscle activity of the masseter (MAM). All three
physiobiological signals have been reported to be associated with stress (Hidaka
et al., 2004b; Hjortskov et al., 2004; Kurniawan et al., 2013; Shi et al., 2007;
Thayer et al., 2012; Yemm, 1969). However, previous studies used different sig-
nals in different stressful conditions, and the relation between these signals and
different stressors remains unknown.
The three physiobiological signals were measured using galvanic skin response
(GSR) sensors (DL-340, S&ME), heartbeat (electrocardiogram [ECG]) sensors
(DL-310, S&ME), and electromyography (EMG) sensors (DL-41, S&ME), and
were recorded in a data logger (DL-3100, S&ME), which was attached to the par-
ticipant’s waist. The sensors were attached to the participant’s skin using medi-
cal tape, and were referenced to the right wrist. Figure 2 illustrates the measur-
ing positions of all three signals.
2.3. Preprocessing of the Physiobiological Signals
All three physiobiological signals were recorded at a frequency of 1000 Hz. The
heart rate variability was recorded from ECG electrodes, which were placed at
Figure 1. Racing course used in the experiment.
W. Wen et al.
981
Figure 2. Positions of the sensors. GSR, galvanic skin response; EMG,
electromyogram (to measure muscle activities); ECG, electrocardiogram
(to measure heart rate).
the distal part of the sternum and at the sixth rib in the left axilla. An offline
band filter of 0.16 - 500 Hz was applied to the ECG signals. From the ECG, we
calculated the intervals between peaks (R-R intervals), standard deviation of the
R-R intervals (SDNN), and root mean squared successive differences of R-R in-
tervals (RMSSD). The ratio of SDNN to RMSSD was used as the index of HRV.
The GSR was recorded using GSR sensors placed at the left and right mastoids.
Most previous studies placed the GSR sensors on the hand or foot (e.g., Healey &
Picard, 2000; Nourbakhsh, Wang, Chen, & Calvo, 2012). However, in the real
car racing, this may disturb the driver’s sensation on hands and feet, and has the
risk of causing fatal mistakes. Therefore, we avoided these positions and chose
mastoids as the measuring site instead. Finally, the MAM was recorded from two
pairs of EMG sensors placed at the left and right masseters. Activities of facial
muscles are useful objective indices of emotion (Cacioppo, Petty, Losch, & Kim,
1986), and jaw muscles in particular are susceptible to mental stress (Hidaka,
Yanagi, & Takada, 2004a; Hidaka et al., 2004b). However, unlike HRV and GSR,
masseters can be activated by consciousness or habit; therefore, the role of the
MAM sensor as an anchor of stress may be different from the former two sen-
sors.
2.4. Procedure
Physiological signals were recorded during the practice session of a real race.
The participant raced over the entire racing course six laps without break. The
race was recorded with a video camera that was installed in the racing car and
recorded the view of the driver. The position of the participant’s racing car was
recorded using global positioning system (GPS) sensors attached to the car.
3. Results
According to the results of an interview with the participant, the first and second
W. Wen et al.
982
laps were considered as a warming up phase and were therefore excluded from
the analysis. Only the results from the third and sixth laps were included in the
analysis. We used factor analysis to examine the relations between the three
physiological signals. The analysis revealed two factors: the first factor 1) influ-
enced both HRV and GSR, while the second factor 2) influenced MAM. Table 1
summarizes the coefficients of the two factors. The factor coefficient shows the
extent to which a factor influences a variable.
In order to determine the events that were associated with the observed activi-
ties of the two factors, we abstracted two events from the video and GPS records
of the car: 1) competition and 2) acceleration/deceleration. The event of compe-
tition refers to the time when another car appeared in the visual field of the par-
ticipant (Figure 3). The event of acceleration/deceleration was calculated from
the GPS recordings. Figure 4 illustrates the plots of the factor scores in function
of time. The events of competition greatly overlaid with the high scores of factor
1 (i.e. HRV and GSR), while the events of acceleration/deceleration greatly over-
laid with high scores of factor 2 (i.e. MAM). Furthermore, a qualitative analysis
showed that 92% of the events of competition consisted of high scores of factor 1
(>2.5 standard deviation), but only 37% contained high scores of factor 2. In
contrast, 54% of the events of acceleration/deceleration contained high scores of
factor 1, while 63% contained high scores of factor 2. In summary, the results
showed that factor 1 influenced HRV and GSR and was associated with the event
of competition, while factor 2 influenced MAM and was associated with the
event of acceleration/deceleration.
Table 1. Coefficients of the two factors.
Factor 1 Factor 2
HRV 0.362 0.154
GSR 0.346 0.207
MAM 0.015 0.355
Figure 3. An example of a competition event.
W. Wen et al.
983
Figure 4. The scores of factor 1 (a) and 2 (b) plotted in function of time, and the overlay with the events of competition and acce-
leration/deceleration.
4. Discussion
The present study measured three different physiological signals, all of which are
considered to reflect mental stress, in a professional driver during a real car race.
We examined the relations between the three physiological signals using factor
analysis, and the association with significant events during the race. The results
showed that HRV and GSR were highly correlated with each other, and were as-
sociated with the events of competition. On the other hand, MAM was not cor-
related with the other two physiological signals, and was associated with the
W. Wen et al.
984
events of acceleration/deceleration. Our present findings suggested that, al-
though all the three physiological signals are somehow affected by stress, they
probably reflect different types of stressors. In particular, HRV and GSR are
probably closer to mental stress, which is mainly the result of internal state of
desire and triggered by changes in the external world. On the other hand, MAM
may be triggered by physical discomfort, which could also result in stress, but is
more bottom-up and does not require an internal state.
To our knowledge, the present study is the first that measured multiple physi-
ological signals from an extremely dangerous and stressful situation in real life.
Most previous studies examined stress under experimental conditions, and the
“real” stress may have not been accurately simulated. The stressors examined in
previous studies were more controlled and had less variability. For example,
cognitive load (Hjortskov et al., 2004; Lattimore & Maxwell, 2004; Shi et al.,
2007), unfriendly attitude (Hjortskov et al., 2004; Montirosso, Provenzi, Calci-
olari, Borgatti, & Group, 2011), and emotion (Kim, Bang, & Kim, 2004) are the
most common stressors used in experimental tasks. However, none of them pro-
vides continuous and diverse changes in stress, and therefore could not reveal
the relationship between different physiological signals during such changes.
During the car race, the driver was not only under high cognitive load but also
faced a life-threatening situation, and was inspired by the desire to win at every
moment. In addition, because of the high velocity and g-force during accelera-
tion/deceleration, the driver also suffered from stress resulting from physical
discomfort. Therefore, the situation of car racing provided an ideal environment
of multiple sources of stress and continuous changes, allowing us to examine the
continuous changes in physiological reactions that are associated with different
stressors.
The most important finding of our results was that stressors associated with
an internal state, such as the desire to win, and that are only linked to external
stimuli may result in different physiological reactions. During the race, accelera-
tion/deceleration brought discomfort to the body because of large g-force, and
probably resulted in a stressful situation. However, both HRV and GSR were not
sensitive to such condition, indicating that, although the participant might be
suffering from a physical discomfort, his mental stress associated with the inter-
val state did not change significantly. This does not necessarily mean that the
mental stresses resulted from internal and external states have to be dissociated.
Instead, in most occasions, they might be linked to each other, but learning (e.g.,
long-term exposure to a specific physical discomfort) may be able to separate
them, thus controlling mental stress from the bottom-up physical stimuli.
5. Conclusion
The present study showed that competition and physical discomfort may result
in different physiological reactions during a real car race. In particular, we found
that the event of competition triggered an increase in HRV and GSR, while the
acceleration/deceleration event triggered MAM. The results indicated that dif-
W. Wen et al.
985
ferent stressors, associated with internal or external states or stimuli, may result
in disparate physiological reactions.
Acknowledgements
This work was supported in part by JSPS KAKENHI (Grant Numbers 26120005).
References
Cacioppo, J. T., Petty, R. E., Losch, M. E., & Kim, H. S. (1986). Electromyographic Activ-
ity over Facial Muscle Regions Can Differentiate the Valence and Intensity of Affective
Reactions.
Journal of Personality and Social Psychology, 50,
260-268.
https://doi.org/10.1037/0022-3514.50.2.260
Healey, J., & Picard, R. (2000). SmartCar: Detecting Driver Stress.
Proceedings 15th In-
ternational Conference on Pattern Recognition. ICPR-2000, 4,
218-221.
https://doi.org/10.1109/ICPR.2000.902898
Hidaka, O., Yanagi, M., & Takada, K. (2004a). Changes in Masseteric Hemodynamics
Time-Related to Mental Stress
. Journal of Dental Research, 83,
185-190.
https://doi.org/10.1177/154405910408300220
Hidaka, O., Yanagi, M., & Takada, K. (2004b). Mental Stress-Induced Physiological
Changes in the Human Masseter Muscle.
Journal of Dental Research, 83,
227-231.
https://doi.org/10.1177/154405910408300308
Hjortskov, N., Rissén, D., Blangsted, A. K., Fallentin, N., Lundberg, U., & Søgaard, K.
(2004). The Effect of Mental Stress on Heart Rate Variability and Blood Pressure dur-
ing Computer Work.
European Journal of Applied Physiology, 92,
84-89.
https://doi.org/10.1007/s00421-004-1055-z
Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion Recognition System Using
Short-Term Monitoring of Physiological Signals.
Medical and Biological Engineering
and Computing, 42,
419-427. https://doi.org/10.1007/BF02344719
Kurniawan, H., Maslov, A. V., & Pechenizkiy, M. (2013). Stress Detection from Speech
and Galvanic Skin Response Signals.
Proceedings of CBMS 201326th IEEE Interna-
tional Symposium on Computer-Based Medical Systems,
209-214.
https://doi.org/10.1109/cbms.2013.6627790
Lattimore, P., & Maxwell, L. (2004). Cognitive Load, Stress, and Disinhibited Eating
. Eat-
ing Behaviors, 5,
315-324. https://doi.org/10.1016/j.eatbeh.2004.04.009
Montirosso, R., Provenzi, L., Calciolari, G., Borgatti, R., & Group, N.-A. S. (2011). Mea-
suring Maternal Stress and Perceived Support in 25 Italian NICUs.
Acta Paediatrica,
101,
136-142. https://doi.org/10.1111/j.1651-2227.2011.02440.x
Nourbakhsh, N., Wang, Y., Chen, F., & Calvo, R. A. (2012). Using Galvanic Skin Re-
sponse for Cognitive Load Measurement in Arithmetic and Reading Tasks.
Proceed-
ings of the 24th Conference on Australian Computer-Human Interaction OzCHI’12,
420-423. https://doi.org/10.1145/2414536.2414602
Shi, Y., Park, T., Ruiz, N., Taib, R., Choi, E. H. C., & Chen, F. (2007). Galvanic Skin Re-
sponse (GSR) as an Index of Cognitive Load.
CHI EA’07 CHI’07 Extended Abstracts
on Human Factors in Computing Systems,
2651-2656.
https://doi.org/10.1145/1240866.1241057
Thayer, J. F., Ahs, F., Fredrikson, M., Sollers, J. J., & Wager, T. D. (2012). A Me-
ta-Analysis of Heart Rate Variability and Neuroimaging Studies: Implications for Heart
Rate Variability as a Marker of Stress and Health.
Neuroscience and Biobehavioral Re-
views, 36,
747-756. https://doi.org/10.1016/j.neubiorev.2011.11.009
W. Wen et al.
986
Yemm, R. (1969). Variations in the Electrical Activity of the Human Masseter Muscle
Occurring in Association with Emotional Stress.
Archives of Oral Biology, 14,
873-878.
https://doi.org/10.1016/0003-9969(69)90265-9
Submit or recommend next manuscript to SCIRP and we will provide best
service for you:
Accepting pre-submission inquiries through Email, Facebook, LinkedIn, Twitter, etc.
A wide selection of journals (inclusive of 9 subjects, more than 200 journals)
Providing 24-hour high-quality service
User-friendly online submission system
Fair and swift peer-review system
Efficient typesetting and proofreading procedure
Display of the result of downloads and visits, as well as the number of cited articles
Maximum dissemination of your research work
Submit your manuscript at: http://papersubmission.scirp.org/
Or contact psych@scirp.org
... In total, 42 features were extracted for detecting three levels of stress. A short list of effective biosignal features from the ECG and GSR in the reviewed studies (tables 1 and 2) are HRV [56], HR [57], difference and phasic-tonic components of EDA signals [58], inter-beat-interval (IBI) [59], wavelet components [55], [60], non-biosignal features extracted from video (head movement and the mean level of the eye opening) [51], environmental data (light, darkness, fog) [50], and GPS position [61]. ...
... On the other hand, some studies confirmed that EMG is not good for stress identification. Wen et al. [61] applied short-term stress on the drivers in a real car race experiment to find out how the human body reacts to real stress. In the experiment, a combination of EMG (masseter muscle), HRV, and GSR signals was employed. ...
... Each biosignal has specific hallmark features for stress detection, and the hybrid methods combine them in one algorithm. Several studies successfully employed hybrid methods [61], [76], [84]. For example, Brookhuis et al. [76] investigated drivers' mental workload in driving simulators using the ECG and EEG signals. ...
Article
Full-text available
Heavy equipment operation is a responsible and difficult task causing mental workload on a human operator and exposing the operator to a range of harmful factors. Human factors and ergonomics in heavy equipment design have traditionally been focused on anthropometry and questionnaires. More advanced techniques involving biosignal measurements were not applied to heavy equipment, mainly due to the diversity of real working conditions that were hard to reproduce in a laboratory environment and that prevented ambulatory studies. Recent advances in wearable biosensors and real-time simulators produce the capability of using biosignals for improving the ergonomics of heavy equipment operation. The present paper reviews the use of biosignals in human factors and the ergonomics of heavy machines by focusing on stress detection for the last ten years. The aim of the paper is analyzing the previous implemented algorithms to find a set of biosignals and methods of stress identification that could be suitable for identifying stress in heavy equipment operators both in laboratory and ambulatory studies. The conclusion emphasizes successful stress identification methods and a combination of the algorithms from different studies that facilitate the use of heavy equipment operator’s applications. Also, feasible methods and directions for future research are considered.
... Studies have shown that high anxiety levels harm drivers' performance, increasing the likelihood of errors and accidents (Wilson et al., 2002). During the competition, anxiety is present in all motor races, as demonstrated in car race drivers, based on objective measures of heart rate variability and galvanic skin response (Wen et al., 2017). Furthermore, it is inversely related to psychological skills and performance success in stock car racing (Ebben & Gagnon, 2012). ...
... These findings in other competitive sports situations might be considered "normal." In motorsports, however, high-stress arousal during the race, as demonstrated via heart rate variability studies (i.e., Wen et al., 2017), could carry over into the recovery period after the race. However, our postrace data were obtained at various intervals within 20 min after the race, which might have prevented the detection of such carry-over effects. ...
Article
Motocross racing is a seldom-researched popular extreme sport. This field research aimed to investigate feeling states, perceived arousal, anxiety, and negative and positive affect in the anticipatory and recovery race periods and their relationship to expected and perceived performance. Twenty Motocross racers completed psychometric scales before and after a national championship race. Results revealed that objective performance was unrelated to psychological measures. Arousal, anxiety, and positive affect were lower after the race. Expected performance was unrelated to postrace measures. Still, perceived performance correlated significantly with the feeling state, anxiety, and positive affect after the race and the feeling state before the race. Furthermore, racers who performed as expected or better showed improved feeling states after the race compared with those who did worse than expected. The core affect of the latter group declined. This research on psychological states during Motocross races could motivate new initiatives for future studies.
... As mentioned in the previous Section, the ear is the intersection of multiple microsleep biomarker sources (e.g. the brain, the eyes, facial muscles, and sweat glands) and is also a natural harbor point where a wearable device could be worn. While recent works on ear-based biosensing have shown the feasibility of capturing individual biosignals (e.g., EEG [23], EOG [24], EEG/EMG [25], and EDA [26]) from the area around and behind the ears, monitoring microsleep-related biosignals with a wearable form-factor has not been explored before. Thus, it is unclear about (1) where are the best places for EEG, EOG, EMG and EDA sensors to achieve both wearability and sensing sensitivity, (2) what is the minimum number of required electrodes, and (3) what are the unique characteristics of BTE signals? ...
... However, the ability to detect microsleep from BTE sensors has not been evaluated before. Vital signs monitoring using wearable and mobile sensors has also been investigated in various studies such as breathing measurement [61], stress estimation (heart rate variability, galvanic skin response, and EMG) [26], and eating detection (IMU, microphone, and proximity) [62,63]. However, to the best of our knowledge, there are no existing works to detect microsleep from wearable BTE sensors accurately and reliably. ...
Article
Full-text available
Every year, the U.S. economy loses more than $411 billion because of work performance reduction, injuries, and traffic accidents caused by microsleep. To mitigate microsleeps consequences, an unobtrusive, reliable, and socially acceptable microsleep detection solution throughout the day, every day is required. Unfortunately, existing solutions do not meet these requirements. In this paper, we propose WAKE, a novel behind-the-ear wearable device for microsleep detection. By monitoring biosignals from the brain, eye movements, facial muscle contractions, and sweat gland activities from behind the user's ears, WAKE can detect microsleep with a high temporal resolution. We introduce a Three-fold Cascaded Amplifying (3CA) technique to tame the motion artifacts and environmental noises for capturing high fidelity signals. Through our prototyping, we show that WAKE can suppress motion and environmental noise in real-time by 9.74-19.47 dB while walking, driving, or staying in different environments ensuring that the biosignals are captured reliably. We evaluated WAKE using gold-standard devices on 19 sleep-deprived and narcoleptic subjects. The Leave-One-Subject-Out Cross-Validation results show the feasibility of WAKE in microsleep detection on an unseen subject with average precision and recall of 76% and 85%, respectively.
... As mentioned in the previous Section, the ear is the intersection of multiple microsleep biomarker sources (e.g. the brain, the eyes, facial muscles, and sweat glands) and is also a natural harbor point where a wearable device could be worn. While recent works [9,37,78,120] on ear-based biosensing have shown the feasibility of capturing biosignals from the area behind the ears, monitoring microsleep-related biosignals have not been done before. Thus, it is unclear about (1) where are the best places for EEG, EOG, EMG and EDA sensors to achieve wearability and sensing sensitivity, (2) what is the minimum number of required electrodes, and (3) what are the unique characteristics of BTE signals? ...
... However, the ability to detect microsleep from BTE has not been evaluated before. Vital signs monitoring using wearable and mobile sensors has also been investigated in various studies such as breathing measurement [75], tongue-teeth localization (EEG, EMG, and skin surface deformation) [78], stress estimation (heart rate variability (HRV), galvanic skin response, and EMG) [120], mental health management (EEG, hemoencephalography, and HRV) [42], and eating detection (IMU, microphone, and proximity sensor) [3,8]. However, to the best of our knowledge, there are no existing works to detect microsleep from wearable BTE sensors accurately and reliably. ...
... However, the central organization of skin conductance response (SCR) is not completely understood. 1 Nevertheless, GSR has been utilized widespread as noninvasive tool to study stress through sympatho-cholinergic stimulation. [2][3][4] The waveforms analyzed vary considerably in population thus is complicated to quantify the physiological response yet, has been widely utilized either for diagnosis of diseases along with other lab tests or related pain analysis. 5,6 In our previous study, GSR has been proved to be useful for detection of sympathetic arousal for cognitive load. ...
... However, the central organization of skin conductance response (SCR) is not completely understood. 1 Nevertheless, GSR has been utilized widespread as noninvasive tool to study stress through sympatho-cholinergic stimulation. [2][3][4] The waveforms analyzed vary considerably in population thus is complicated to quantify the physiological response yet, has been widely utilized either for diagnosis of diseases along with other lab tests or related pain analysis. 5,6 In our previous study, GSR has been proved to be useful for detection of sympathetic arousal for cognitive load. ...
Article
Full-text available
Background Several studies have found skin conductance a good indicator for detection of sympathetic response. But, valid and reliable tool for detection of sympathetic outflow in health and disease is still a quest. Thereby, comparison of superficial and, at core sympathetic effluence induced by deliberately supplied discrete external stimuli has been attempted in this study. Objective To assess the degree of sympathetic outflow for discrete cognitive and physical stimuli through perturbations in skin conductance and variations in heart rate in healthy adults. Method Quantitative and cross-sectional study was performed in 104 healthy subjects following random sampling method. Induction of sympathetic activity was realized by providing separate time bound cognitive exercises intervened with change in posture. Recordings to detect sympathetic responses at rest and, for supplied stimuli were made by electrocardiogram and galvanic skin response. Result Cognitive performance and postural change shifts baseline effluence and increases the sympathetic outflow significantly (p=0.000). There occurs no detectable rise in sympathetic effluence at the core (p=0.362) but, eventuate significantly appreciable sympathetic outflow to sweat glands in skin (p=0.000), when compared cognitive versus physical stimuli. Conclusion Sympathetic outflow induced by cognitive challenge and physical change in posture is readily assessable through sympathetic skin response yet core sympathetic effluence for latter stimuli is steady and unwavering. Differential effluence for sympathetic response called upon by discrete stimuli is operational for maintenance of steady state in healthy subjects.
... Few and relatively recent studies in the literature analyze the complexity of the EDA, although such signal was known to have a special ability to capture human affect and was particularly proven to reflect mental stress during real-world driving 30 and even during car races. 31 For instance, Visnovcova et al. 32 evaluated the EDA changes in response to two cognitive stressors: stroop color word and arithmetic tests based on a nonlinear analysis using indices that describe irregularity of the sympathetic nervous system. These studies concluded that the EDA nonlinear characteristics can provide relevant information about stress-related complex sympathetic regulation. ...
Article
This paper characterizes stress levels via a self-similarity analysis of the electro-dermal activity (EDA) collected in a real-world driving context. To characterize the EDA richness over scales, the fractional Brownian motion (FBM) process and its corresponding exponent H, estimated via a wavelet-based approach, are used. Specifically, an automatic scale range selection is proposed in order to detect the linearity in a log scale diagram. The procedure is applied to the EDA signals, from the open database drivedb, originally captured on the foot and the hand of the drivers during a real-world driving experiment, designed to evoke different levels of arousal and stress. The estimated Hurst exponent H offers a distinction in stress levels when driving in highway versus city, with a reference to restful state of minimal stress level. Specifically, the estimated H values tend to decrease when the driving environmental complexity increases. In addition, the estimated H values on the foot EDA signals allow a better characterization of the driving task than that of hand EDA. The self-similarity analysis was applied to various physiological signals in literature but not to the EDA so far, a signal which was found to correlate most with human affect. The proposed analysis could be useful in real-time monitoring of stress levels in urban driving spaces, among other applications.
Article
Full-text available
With the rising amount of data in the sports and health sectors, a plethora of applications using big data mining have become possible. Multiple frameworks have been proposed to mine, store, preprocess, and analyze physiological vitals data using artificial intelligence and machine learning algorithms. Comparatively, less research has been done to collect potentially high volume, high-quality ‘big data’ in an organized, time-synchronized, and holistic manner to solve similar problems in multiple fields. Although a large number of data collection devices exist in the form of sensors. They are either highly specialized, univariate and fragmented in nature or exist in a lab setting. The current study aims to propose artificial intelligence-based body sensor network framework (AIBSNF), a framework for strategic use of body sensor networks (BSN), which combines with real-time location system (RTLS) and wearable biosensors to collect multivariate, low noise, and high-fidelity data. This facilitates gathering of time-synchronized location and physiological vitals data, which allows artificial intelligence and machine learning (AI/ML)-based time series analysis. The study gives a brief overview of wearable sensor technology, RTLS, and provides use cases of AI/ML algorithms in the field of sensor fusion. The study also elaborates sample scenarios using a specific sensor network consisting of pressure sensors (insoles), accelerometers, gyroscopes, ECG, EMG, and RTLS position detectors for particular applications in the field of health care and sports. The AIBSNF may provide a solid blueprint for conducting research and development, forming a smooth end-to-end pipeline from data collection using BSN, RTLS and final stage analytics based on AI/ML algorithms.
Conference Paper
Big data captured through sensory-integrated devices present a unique opportunity for emerging applications in urban research towards understanding, quantifying, and promoting aspects of the urban environment based on the residents' needs. We examine the effect of various built environmental features of the urban environment on individuals emotional distress, as captured through self-assessment indices and physiological measures. Through statistical analysis, we study potential associations between signal-based electrodermal activity (EDA) measures and self-reports. We further propose a novel saliency detection mechanism for physiological signals, according to which prominence of an EDA segment is calculated based on its distance metrics from the remaining signal segments. Results indicate that the signal-based and self-reported indices of emotional distress, as depicted in relation to various built environmental features, are weakly but significantly correlated. Visual inspection of the saliency-based measures further suggests that the hypothesized "emotional distress hotspots" of the urban environment, such as discontinuous sidewalks, threatening noise, and housing with broken features, elicit distinct physiological responses for the majority of participants in the experiments. These suggest the feasibility of physiological measures for quantifying emotional distress caused by the built environment and provide a foundation for the development of signal-based indices of the human felt-sense in urban settings.
Conference Paper
Full-text available
Galvanic Skin Response (GSR) has recently attracted researchers' attention as a prospective physiological indicator of cognitive load and emotions. However, it has commonly been investigated through single or few measures and in one experimental scenario. In this research, aiming to perform a comprehensive study, we have assessed GSR data captured from two different experiments, one including text reading tasks and the other using arithmetic tasks, each imposing multiple cognitive load levels. We have examined temporal and spectral features of GSR against different task difficulty levels. ANOVA test was applied for the statistical evaluation. Obtained results show the strong significance of the explored features, especially the spectral ones, in cognitive workload measurement in the two studied experiments.
Conference Paper
Full-text available
Multimodal user interfaces (MMUI) allow users to control computers using speech and gesture, and have the potential to minimise users. experienced cognitive load, especially when performing complex tasks. In this paper, we describe our attempt to use a physiological measure, namely Galvanic Skin Response (GSR), to objectively evaluate users. stress and arousal levels while using unimodal and multimodal versions of the same interface. Preliminary results show that users. GSR readings significantly increase when task cognitive load level increases. Moreover, users. GSR readings are found to be lower when using a multimodal interface, instead of a unimodal interface. Cross-examination of GSR data with multimodal data annotation showed promising results in explaining the peaks in the GSR data, which are found to correlate with sub-task user events. This interesting result verifies that GSR can be used to serve as an objective indicator of user cognitive load level in real time, with a very fine granularity.
Article
Full-text available
Physiological measures have traditionally been viewed in social psychology as useful only in assessing general arousal and therefore as incapable of distinguishing between positive and negative affective states. This view is challenged in the present report. Sixteen subjects in a pilot study were exposed briefly to slides and tones that were mildly to moderately evocative of positive and negative affect. Facial electromyographic (EMG) activity differentiated both the valence and intensity of the affective reaction. Moreover, independent judges were unable to determine from viewing videotapes of the subjects' facial displays whether a positive or negative stimulus had been presented or whether a mildly or moderately intense stimulus had been presented. In the full experiment, 28 subjects briefly viewed slides of scenes that were mildly to moderately evocative of positive and negative affect. Again, EMG activity over the brow (corrugator supercilia), eye (orbicularis oculi), and cheek (zygomatic major) muscle regions differentiated the pleasantness and intensity of individuals' affective reactions to the visual stimuli even though visual inspection of the videotapes again indicated that expressions of emotion were not apparent. These results suggest that gradients of EMG activity over the muscles of facial expression can provide objective and continuous probes of affective processes that are too subtle or fleeting to evoke expressions observable under normal conditions of social interaction.
Article
Aims: To determine the validity and reliability of the Parental Stressor Scale: Neona-tal Intensive Care Unit (PSS: NICU) and the Nurse Parental Support Tool (NPST) for use with Italian parents; to investigate to which extent demographic variables and ⁄ or situational factors affect NICU-related maternal stress. Methods: Mothers (N = 156) of very preterm (VPT) infants from 25 Italian NICUs completed a socio-demographic form, the PSS: NICU and the NPST at discharge. Psychometric properties of both tools were evaluated.
Conference Paper
The problem of stress-management has been receiving an increasing attention in related research communities due to a wider recognition of potential problems caused by chronic stress and due to the recent developments of technologies providing non-intrusive ways of collecting continuously objective measurements to monitor person's stress level. Experimental studies have shown already that stress level can be judged based on the analysis of Galvanic Skin Response (GSR) and speech signals. In this paper we investigate how classification techniques can be used to automatically determine periods of acute stress relying on information contained in GSR and/or speech of a person.
Article
To determine the validity and reliability of the Parental Stressor Scale: Neonatal Intensive Care Unit (PSS: NICU) and the Nurse Parental Support Tool (NPST) for use with Italian parents; to investigate to which extent demographic variables and/or situational factors affect NICU-related maternal stress. Mothers (N = 156) of very preterm (VPT) infants from 25 Italian NICUs completed a socio-demographic form, the PSS: NICU and the NPST at discharge. Psychometric properties of both tools were evaluated. High internal consistency and split-half reliability were found for both measures. The multi-dimensional structure of the PSS:NICU was confirmed. Alteration in parental role emerged as the greatest source of NICU-related stress. Length of stay in NICU and familiar socio-economic status explained partial variance in the PSS: NICU scores. NPST score mitigates the stress because of the infant's appearance and behaviour, but not that related to the parental role alteration. PSS: NICU and NPST demonstrated adequate psychometric properties in a large sample of Italian mothers. The need for a psychologically informed support to NICU mothers is suggested.
Article
Human subjects were asked to perform a task while recordings were made of the electrical activity of the right masseter muscle. The task apparatus consisted of a series of lamps illuminated in random order, and a button panel. The subjects were asked to press the buttons to correspond with the lamps illuminated. Recordings made from the muscle during this task showed an increase in electrical activity. The magnitude of this increase was related to the number of errors made by the subject. Muscle activity with similar characteristics has been shown by earlier workers to occur elsewhere in the body. It is suggested that such psychogenic activity in the masticatory muscles could affect mandibular posture, and the findings reported are discussed in relation to the current hypothesis of the control of mandibular posture.
Article
Mental stress may cause a dissociation of sympathetic outflow to different regions. However, it remains unclear how the sympathetic outflow to jaw muscles is related to other sympathetic outflow under mental stress. The objective of this study was to clarify the temporal relationship between the finger sweat expulsion elicited by mental stress and the hemodynamic and electromyographic changes in the masseter muscle. Healthy adult female volunteers participated in this study. Masseteric hemodynamic changes were closely time-related to mental stress, showing a decrease in oxygen saturation of muscle blood around the onset of mental stress. In contrast, EMG activity of jaw-closing muscles was not time-related to mental stress. These results suggest that mental stress induces hemodynamic changes that are not associated with EMG activity in the masseter muscle of healthy adult females.
Article
A physiological signal-based emotion recognition system is reported. The system was developed to operate as a user-independent system, based on physiological signal databases obtained from multiple subjects. The input signals were electrocardiogram, skin temperature variation and electrodermal activity, all of which were acquired without much discomfort from the body surface, and can reflect the influence of emotion on the autonomic nervous system. The system consisted of preprocessing, feature extraction and pattern classification stages. Preprocessing and feature extraction methods were devised so that emotion-specific characteristics could be extracted from short-segment signals. Although the features were carefully extracted, their distribution formed a classification problem, with large overlap among clusters and large variance within clusters. A support vector machine was adopted as a pattern classifier to resolve this difficulty. Correct-classification ratios for 50 subjects were 78.4% and 61.8%, for the recognition of three and four categories, respectively.