Conference PaperPDF Available

Detecting Distraction in Drivers using Electroencephalogram (EEG) Signals

Authors:
S.Pradeep Kumar at Vels University
S.Pradeep Kumar
Jerritta Selvaraj at Vels University
Jerritta Selvaraj
R. Krishnakumar
R. Krishnakumar
R. Krishnakumar
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Arun Sahayadhas at Vels Institute of Science Technology and Advanced Studies
Arun Sahayadhas
  • Vels Institute of Science Technology and Advanced Studies
Download full-text PDF
Read full-text
Download citation

Figures

Figures - uploaded by S.Pradeep Kumar
Author content
All figure content in this area was uploaded by S.Pradeep Kumar
Content may be subject to copyright.
Content uploaded by S.Pradeep Kumar
Author content
All content in this area was uploaded by S.Pradeep Kumar on Dec 23, 2020
Content may be subject to copyright.
Detecting Distraction in Drivers using
Electroencephalogram (EEG) Signals
S. P radeep Kumar, Jerritta Selvaraj*, R. Krishnakumar, Arun Sahayadhas
Artific ial Inte llige nc e Re sea rch La b, Ve ls Institute of Sc ie nc e Te chnology and Advanced Studies, Chennai
E- ma il: jerritta.s e@ velsu niv.ac.in
Abstract
Driver distraction i s considered as major factors in most
of the traffic accidents. Driving errors may arise due to
the distraction of the drivers. The aim of this paper is to
analyze the EEG signals to detect distractive driving.
Data from 10 different subjects were obtained and
categorised into different frequency bands. Distractive
driving i s related with Theta band, so the Theta
fr eque n cy ban d we re dec om pos e d u si ng Dis crete Wa vele t
Transform (DWT) and 17 different features were
extracted. By enabling Principle Component Analysis
(PCA) the accuracy rate was found for Cognitive
Distraction and Visual Distraction using different
machine learning algorithms. K Nearest Neighbour
(KNN) performed well when compared to other machine
le arni ng algorithms with a bette r accuracy rate of 71.1%.
Keywords: Electroencephalogram, Distraction, Cognitive,
Visual, Theta band.
I. Introduction
Dis t rac t ed d riving is d riv ing wh ile p e rfor ming
secondary which leads to the chance of a motor vehicle
crash[1]. To understand the driver behaviour it is
significant to monitor their states [2]. A statistics
indicates that there were 34,439 fatal crashes happened
in US involved by 51,914 drivers, and 37,461 peoples
lost their life’s in the year 2016[3]. National Highway
Traffic Safety Administration (NHTSA) had taken a
necessary effort to save lives by prevent ing this
dangerous behaviour [4]. Driver’s inattention can be
categorized into Distraction and Fatigue in [5]. Five
types of driver distraction was considered in a study by
American Automobile Association Foundation for
Traffic Safety (AAAFTS) [6] namely attentive,
Distracted, Looked but did not see, sleepy and
unknown. In this paper, two categories of distraction
namely Cognitive Distraction, which is looked but did
not see and Visual Distraction is analysed using EEG
signals.
Generally, distraction of the drivers is detected using
Vehicular measures (pedal movement, braking, and
lane deviation). Behavioural measures (Face
Movement, Head
Movement, Eye Movement, and Mouth Movement). In
case of behavioural measures, and Physiological
measures such as Electrooculogram EOG),
Electrocardiogram (ECG), Electroencephalogram
(EEG) etc. The percentage of eye closure measures the
alert level of the drivers and the eye position was
classified as open and close using classifiers such as
support vector machine[7]. In[8] a stereo camera was
used to monitor the driver distraction in real time by
analysing the face gaze of a driver. The current driving
state of the driver was indicated using an android-based
Smartphone apps which receives physiological and
facial data via wireless sensor network[9]. The
alertness of the driver was continuously monitored and
determined using the visual analysis of eye behaviour
and head posture[10]. An innovative driver assistance
system was proposed to find the alert state of the
drivers which can measure the physiological signals
and eye-blinking activities in a single device[11].
In[12] the eye and head tracking data were applied
through advanced data processing methods. As an
alternative of focussing the individual parts of the body
by studying the upper body posture and motion as a
whole overcomes the limitations of general purpose
tracking algorithms[13].
II. Related Work s
One challenge in the study of distraction driving is the
wide range of distraction exposed to drivers. These
distraction can induced the drivers to perform the
secondary task such as using mobile, interacting with
passengers, seeing ad boards, listening music etc[14].
To build up a real time approach for cognitive
detection using driver’s eye blink and driver’s
performance data were applied to Support Vector
Machines (SVM) mode ls and the result indicates that
an accuracy rate of 81% was obtained for detecting the
driver distraction[15]. A new framework by map
viewing to predict the starting and ending time of a
distraction was executed in[16], and the overall
accuracy values were 81% and 70%, respectively for
24 s ubjects. Most researchers focussed on vehicular
Proceedings of the Fourth International Conference on Computing Methodologies and Communication (ICCMC 2020)
IEEE Xplore Part Number:CFP20K25-ART; ISBN:978-1-7281-4889-2
978-1-7281-4889-2/20/$31.00 ©2020 IEEE 635
2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC) 978-1-7281-4889-2/20/$31.00 ©2020 IEEE 10.1109/ICCMC48092.2020.ICCMC-000118
and behavioural based prediction for distraction
detection. So in this study we are focussing on EEG
based method to predict distraction.
III. Propose d Methodolog y
The proposed methodology using EEG for driver
distraction detection is shown in Figure 1. The EEG
data is acquired from different subjects are splitted into
sub bands. Theta band which is suitable for distraction
detection was pre-processed; features were extracted
and classified as Visual and Cognitive Distraction.
Figure 1: Proposed Methodology
A. Experimental Setup
A Virtual Reality based driving environment setup was
installed in AI Research Lab at VISTAS. A speed
dreams game was installed to ma ke a feel of real road
driving environment and the speed of the vehicle was
set at a maximum of 60 Km/hr. A 21 channel EEG
electrodes was used to obtain the data from 10 subjects
in 3 different timings.Each participant has to
participate the experiments in 3 different timings
(midnight (01.00 am to 2.00 a m); early mo rning (4.00
am to 5.00 am) and afternoon (2.00 p m to 4.00 pm).
The experiments were held for 90- 120 minutes. An
Infrared ( I R) ca mera was us ed t o capture the
behavioural measures of the drivers. The subjects will
be interrupted both visually (through messages) and
cognitively (by phone call) while driving and the
corresponding EEG data are recorded. The wall of the
experimental area was covered with black cloth to have
a feel of driving in night time. Figure 2 & Figure 3
represents the image taken during Cognitive
Dis traction & Visual Distraction
Figure2: Image taken during Cognitive Figure 3: Image taken during Visual
distraction distraction
Proceedings of the Fourth International Conference on Computing Methodologies and Communication (ICCMC 2020)
IEEE Xplore Part Number:CFP20K25-ART; ISBN:978-1-7281-4889-2
978-1-7281-4889-2/20/$31.00 ©2020 IEEE 636
IV. S i g nal Proc es s ing and Analys i s
A band pass filter was set at a range of 0.5 Hz to 49 Hz
to remove the noises and other artifacts like power line
interference; eye move ment etc from the raw EEG
data. The filtered EEG data were categorized into
different bands such as Delta (δ), Theta (θ), Alpha (α),
Beta (β), and Ga mma (γ). The distraction was related to
Theta band (8-13 Hz), so only the theta band was used
for fu rther proces s . A wav elet analys is for the EEG
was done and the Daubechies (DB4) fourth order
Discrete Wavelet Transform was used to decompose
the Theta band signals. The linear and nonlinear
features such as Mean, Median, Maximu m, Minimu m,
Standard Deviation, Kurtosis, Hurst, Root Mean
Square, Trim Mean, Harmonic Mean, Nan Mean,
Sample Entropy, Power, Energy, Mode and Variance
were extracted for the theta band. Principle Component
Analysis was enabled for features reduction. By using
the classification learner tool the classifier accuracy for
identifying the d istraction type (Cognitive and Visual)
was found using the following machine learning
alg orith ms su ch as SVM an d KNN.
V. Results and Discussion
A two class comparison between cognitive distraction
and visual distraction was made using SVM and KNN
algorith ms. Table1 shows the accuracy rate of 21 EEG
channels for both cognitive and visual distraction using
SVM and KNN.
Table 1: Accuracy rate of detecting distraction using SVM and KNN classifiers
Channels Classifiers
Visual Distraction
Cognitive
Distraction
Total
FP2
SVM
26
81.8
53.3
KNN
91.3
50
71.1
FP1
SVM
100
4
51.1
KNN
63.6
60.8
62.2
F7
SVM
100
4
53.3
KNN
56.5
45.4
51.1
F3
SVM
100
0
51.1
KNN
95.6
40.9
68.9
FZ
SVM
95.4
17.3
55.6
KNN
22.7
91.3
57.8
F4
SVM
100
4.5
53.3
KNN
100
9
55.6
F8
SVM
95.6
22.7
60
KNN
73.9
36.3
55.6
T3
SVM
40.9
78.2
60
KNN
95.4
8.6
51.1
C3
SVM
100
9
55.6
KNN
52.1
63.6
57.8
CZ
SVM
95.6
9
53.3
KNN
78.2
27.2
53.3
C4
SVM
100
4.3
51.1
KNN
72.7
34.7
53.3
T4
SVM
95.6
9
53.3
KNN
72.7
43.4
57.8
M2
SVM
91.3
18.1
55.6
KNN
13
100
55.6
M1
SVM
95.4
13
53.3
KNN
59
47.8
53.3
T5
SVM
9
100
55.6
KNN
54.5
52.1
53.3
P3
SVM
100
21.7
60
KNN
72.7
56.5
64.4
PZ
SVM
95.4
8.6
51.1
KNN
95.4
13
53.3
P4
SVM
100
8.6
53.3
KNN
59
56.5
57.8
T6
SVM
95.6
9
53.3
KNN
60.8
50
55.6
Proceedings of the Fourth International Conference on Computing Methodologies and Communication (ICCMC 2020)
IEEE Xplore Part Number:CFP20K25-ART; ISBN:978-1-7281-4889-2
978-1-7281-4889-2/20/$31.00 ©2020 IEEE 637
O1
SVM
4
100
51.1
KNN
100
0
51.1
O2
SVM
45.4
73.9
60
KNN
90.9
17.3
53.3
Figure 4 shows the Accuracy rate of dis traction using
SVM and KNN classifier. It is clear that KNN
Classifier performs well when compared to SVM. FP2
channel has the highest accuracy of 71.1% when
compared with all other channels.
Figure 4: Total Accuracy rate of distraction using SVM and KNN classifier
The cognitive and visual distraction predicting
percentage was also performed as shown in Figure 5
and Figure 6 respectively. Channels FP1, F3, F4, F7,
C3, C4, P3 P4 and O1 has the accuracy rate of 100%
for visual distraction and ChannelsM2, T5 and O1 have
the highest accuracy rate of 100% for predicting
cognitive distraction.
Figure 5: Comparison of Visual distraction using SVM and KNN classifier
0
20
40
60
80
FP2 FP1 F7 F3 FZ F4 F8 T3 C3 CZ C4 T4 M2 M1 T5 P3 PZ P4 T6 O1 O2
SVM KNN
91.3
63.6
56.5
95.6
22.7
100
73.9
95.4
52.1
78.2
72.7
72.7
13
59
54.5
72.7
95.4
59
60.8
100
90.9
0
20
40
60
80
100
120
FP2 FP1 F7 F3 FZ F4 F8 T3 C3 CZ C4 T4 M2 M1 T5 P3 PZ P4 T 6 O1 O2
Visual Distraction
SVM KN N
Proceedings of the Fourth International Conference on Computing Methodologies and Communication (ICCMC 2020)
IEEE Xplore Part Number:CFP20K25-ART; ISBN:978-1-7281-4889-2
978-1-7281-4889-2/20/$31.00 ©2020 IEEE 638
Figure 6: Comparison of Cognitive distraction using SVM and KNN classifier
In this study we imp lemented EEG based distraction
detection. Based on our result the channels C3, C4, P3,
P4, FP1, F7, F3, F4, and O1 has 100 % accuracy for
predicting the visual distraction. While the driver is
visually distracted he/she has to concentrate on the
secondary task also (messaging while driving). But
while in cognitive distraction the subjects view will not
be affected, as he/she can see the road while talking on
the phone. So the EEG signals will be more related to
visual distraction when compared to cognitive
distraction.
VI. Conc lus i on
In this paper, the results of both cognitive distraction
and visual Distraction are compared, and it is found
that channel FP2 has the highest accuracy rate of
71.1% fo r p redicting d is t rac t io n in d rivers . A n
accuracy rate of 100% was obtained for predicting
Visual distraction in 9 channels (FP1, F 3, F7, F4, C3,
C4, P3 P4 and O1). Similarly channels M2, T5 and O1
have the highest accuracy rate of 100% for predicting
cognitive distraction. In future, using channel reduction
techniques the channels can be reduced into a single
channel or two channel electrodes for providing a
compact model to alert the drivers from distraction.
Funding: This research was supported by Science &
Engineering Research Board (SERB),
[SERB/F/3759/2016-17, 2016] Depart ment of Science
and Technology (DST), Government of India.
References
[1]N. C. for I. P. and C. Centers for Disease Control and Prevention,
“M otor Vehicle Safety , ” 2 019. [On line]. Ava il abl e:
ht t p s :/ /www. c dc . go v /m ot or v eh ic l es af et y / di st ra ct e d_d r iv in g/ i n dex .ht
ml.
[2]G. A. P. C, F. García, A. De Escalera, and J. M. Armingol,
“Driver Monitoring Based on Low-Cost 3-D Sensors,” vol. 15, no. 4,
pp. 1855–1860, 2014.
[3]D. O. T. Hs, “Research Note Distract ed Driving 2016,” no. April
2018, pp. 1–6, 2016.
[4]NHTSA, “Distracted Driving,” National Highway Traffic Safety
Administration, 2 017 . [Onlin e] . Av a i la ble :
h t t p s:/ /www. n h ts a. gov / ris ky - driv i ng/ dis t ra ct ed- dr iv in g.
[5]Y. Dong, Z. Hu, K. Uchim ura, and N. Murayama, “Driver
Inattention Monitoring Sy stem for Intelligent Veh icles : A Review,”
vol. 12, no. 2, pp. 596–614, 2011.
[6]J. C. St utts, D. W. Reinfurt, L. Staplin, and E. A. Rodgman, “THE
ROLE OF DRIVER DISTRACTI ON,” n o. May, 2001.
[7] A. Dasgupt a , A. Geor ge, S. L. Happ y, and A. Rout r ay, “A Vision -
Based Sy stem for Monitoring t he Loss of Attention in Automotive
Drivers,” vol. 14, no. 4, pp. 1825–1838, 2013.
[8]P. Jiménez, L. M. Bergasa, J. Nuevo, N. Hernández, and I. G.
Daza, “ Gaze Fixation System for the Evaluat ion of Driver
Dist ractions Induced by IVIS,” vol. 13, no. 3, pp. 1167–1178, 2012.
[9]B. Lee and W. Chung, “Driver Alertness Monitoring Using Fusio n
of Facial Features and Bio -Signals,” vol. 12, no. 7, pp. 2416–24 22,
2012.
[10]R. O. Mbouna, S. G. Kong, and M. G. Chun, “Visual analysis of
eye state and head pose for driver alertness monitoring,” IE EE T ran s .
Intell. Transp. Syst., vol. 14, no. 3, pp. 1462–1469, 2013.
[11]Y. Sun, S. Member, and X. B. Yu, “An Innovative Nonintrusive
Driver Assistance System for Vital Signal Monitoring,” vol. 18, no.
6, pp. 1932–1939, 2014.
[12 ] C. Ahlstrom, T. Victor, C. Wege , and E. Ste inmet z , “Proce ssing
of Eye / Head-Tracking Data in Large-Scale Naturalistic Driving
Data Sets,” vol. 13, no. 2, pp. 553– 564, 2012.
[13]A. Kondyli, V. P. Sisiopiku, L. Zhao, and A. Barmpoutis,
“ Comp uter Assist ed Analysis o f Driv ers ’ Body Act ivit y Usin g a
Range Camera,” no. July, pp. 18–28, 2015.
[14 ]N . Li, S. Mem be r , C . Bus so , and S. M em ber, “Pr e di ct in g
Perceived Visual and Cognit ive Distractions of Drivers With
Multimodal Features,” vol. 16, no. 1, pp. 51–65, 2015.
[15]F. Tango and M. Bott a, “Real-Time Detection Syst em of Driver
Distr action Usin g Mac hin e Lear ning, ” vo l . 14, no . 2 , pp. 89 4 –9 05 ,
2013.
[16 ] S. Wan g, Y . Zh an g, C. W u, F. Da rv a s, W . A. Cha ov a l it won gse ,
and S. Member, “Online Prediction of Driver Distraction Based on
Brain Activity Patterns,” vol. 16, no. 1, pp. 136–150, 2015.
0
20
40
60
80
100
FP2 FP1 F7 F3 FZ F4 F8 T3 C3 CZ C4 T4 M2 M1 T5 P3 PZ P4 T6 O1 O2
81.8
4 4 0
17.3
4.5
22.7
78.2
9 9
4.3
9
18.1
13
100
21.7
8.6 8.6 9
100
73.9
50
60.8
45.4
40.9
91.3
9
36.3
8.6
63.6
27.2
34.7
43.4
100
47.8
52.1
56.5
13
56.5
50
0
17.3
SVM KN N
Proceedings of the Fourth International Conference on Computing Methodologies and Communication (ICCMC 2020)
IEEE Xplore Part Number:CFP20K25-ART; ISBN:978-1-7281-4889-2
978-1-7281-4889-2/20/$31.00 ©2020 IEEE 639
... Over recent years, different scientific works have been focused on distraction detection through the processing of neurophysiological data. In this regard, it has to be noted that also subjective and behavioral measurements were largely explored in the scientific literature for distraction evaluation [24][25][26]. However, such approaches are susceptible to bias and lack objectivity. ...
... Twenty-five (25) participants in possession of their driving license, with normal or corrected-to-normal vision, were recruited on a voluntary basis. The participants were selected in order to have a homogeneous sample in terms of age (age ranged from 18 to 69 years old, corresponding to an average of 26 years old), driving experiences, and cars that they normally used to drive. ...
A Novel EEG-Based Assessment of Distraction in Simulated Driving under Different Road and Traffic Conditions
Article
Full-text available
  • Feb 2024
  • BSRCCS
The drivers’ distraction plays a crucial role in road safety as it is one of the main impacting causes of road accidents. The phenomenon of distraction encompasses both psychological and environmental factors and, therefore, addressing the complex interplay contributing to human distraction in automotive is crucial for developing technologies and interventions for improving road safety. In scientific literature, different works were proposed for the distraction characterization in automotive, but there is still the lack of a univocal measure to assess the degree of distraction, nor a gold-standard tool that allows to “detect” eventual events, road traffic, and additional driving tasks that might contribute to the drivers’ distraction. Therefore, the present study aimed at developing an EEG-based “Distraction index” obtained by the combination of the driver’s mental workload and attention neurometrics and investigating and validating its reliability by analyzing together subjective and behavioral measures. A total of 25 licensed drivers were involved in this study, where they had to drive in two different scenarios, i.e., City and Highway, while different secondary tasks were alternatively proposed in addition to the main one to modulate the driver’s attentional demand. The statistical analysis demonstrated the reliability of the proposed EEG-based distraction index in identifying the drivers’ distraction when driving along different roads and traffic conditions (all p < 0.001). More importantly, the proposed index was demonstrated to be reliable in identifying which are the most impacting additional driving tasks on the drivers’ distraction (all p < 0.01).
View
... According to the latest study published by the World Health Organization (WHO) in 2020, about 1.3 million people died as a result of road accidents. Undoubtedly, it caused a direct impact on victims and families, but also a high cost to government tions and drowsiness in the last few years (Kumar et al., 2020). In this context, Brain-Computer Interfaces (BCIs) have been widely used to identify cognitive patterns that provide evidence of distracted driving to fulfill this purpose (Cheng et al., 2018). ...
... However, they detailed the difficulties in reducing noise and ensuring a reliable EEG signal. Finally, Kumar et al. (2020) generated some standard statistical measures in the research. A binary SVM was used for distraction detection while comparing the results with those obtained with the kNN algorithm. ...
SAFECAR: A Brain-Computer Interface and intelligent framework to detect drivers’ distractions
Article
Full-text available
  • May 2022
  • EXPERT SYST APPL
As recently reported by the World Health Organization (WHO), the high use of intelligent devices such as smartphones, multimedia systems, or billboards causes an increase in distraction and, consequently, fatal accidents while driving. The use of EEG-based Brain-Computer Interfaces (BCIs) has been proposed as a promising way to detect distractions. However, existing solutions are not well suited for driving scenarios. They do not consider complementary data sources, such as contextual data, nor guarantee realistic scenarios with real-time communications between components. This work proposes an automatic framework for detecting distractions using BCIs and a realistic driving simulator. The framework employs different supervised Machine Learning (ML)-based models on classifying the different types of distractions using Electroencephalography (EEG) and contextual driving data collected by car sensors, such as line crossings or objects detection. This framework has been evaluated using a driving scenario without distractions and a similar one where visual and cognitive distractions are generated for ten subjects. The proposed framework achieved 83.9% F1-score with a binary model and 73% with a multiclass model using EEG, improving 7% in binary classification and 8% in multi-class classification by incorporating contextual driving into the training dataset. Finally, the results were confirmed by a neurophysiological study, which revealed significantly higher voltage in selective attention and multitasking.
View
... Kumar et al. in their research used EEG to capture both visual and cognitive distraction. As a result of the measurements, the EEG was highly accurate at measuring both types of distraction [6]. ...
Possibilities of using commercial electronics to measure driver physiological functions in a vehicle simulator
Article
  • Sep 2023
The aim of this paper is to explore the potential of using inexpensive, commercial electronics such as a smartwatch or chest belt to measure human physiological functions while driving in vehicle simulator, both to improve safety and to use these devices in other experiments to facilitate their conduct and replace more complex laboratory equipment. In the first part of the paper, the signs of stress and cognitive load through physiological functions are researched, then the possibilities of measuring these signs are explored. This is followed by an experiment in which two commonly available devices are compared in terms of accuracy and usefulness for measurement during the experiment.
View
View
Driver Distraction from the EEG Perspective: A Review
Article
  • Jan 2023
A large proportion of car accidents are caused by distracted drivers. Thus, comprehensive analysis and understanding on driver distraction is essential for traffic safety improvement. Driver distraction can be revealed from their facial expression in images. However, this is easily affected by complex light distribution on faces or by low illumination during nighttime. Differently, drivers’ physiological signals like electroencephalography (EEG) have been convinced to be one of the most reliable and direct tools for driver distraction studies, either for deeper understanding on driver distraction or for effective detection of driver distraction. Therefore, this paper comprehensively reviews multiple aspects of driver distraction from the EEG perspective. Firstly, the research progress on distracted driving is reviewed from three aspects: the definition of distraction, the types of distraction and the main data sets of distracted driving. Secondly, the computer signal processing is summarized into four aspects: signal acquisition, signal pretreatment, EEG main frequency bands and EEG characteristics, and analyzed in turn. Thirdly, the variation trends of EEG frequency bands under different distraction types were analyzed and compared. Fourthly, the methods of feature extraction and detection of EEG in distracted driving are reviewed from the perspective of methodology. Finally, a new distraction detection method based on EEG integration with other physiological signals is summarized, and the future development trend and technical challenges are prospected.
View
Driver mental load identification model for adaptive urban road traffic scene
Article
Full-text available
  • Sep 2023
Objective At present, most research on driver mental load identification are based on a single driving scene. However, the driver mental load model established in a road traffic scene is difficult to adapt to the changes of the surrounding road environment during the actual driving process. We proposed a driver mental load identification model which adapts to urban road traffic scenarios. Methods The model includes driving scene discrimination sub model and driver load identification sub model, in which the driving scene discrimination sub model can quickly and accurately determine the road traffic scene. The driver load identification sub model selects the best feature subset and the best model algorithm in the scene based on the judgment of the driving scene classification sub model. Results The results show that the driving scene discrimination sub model using five vehicle features as feature subsets has best performance, The driver load identification sub model based on the best feature subset reduces the feature noise, and the recognition effect is better than the feature set using single source signal and all data. The best recognition algorithm in different scenarios tends to be consistent, and SVM algorithm is better than KNN algorithm. Conclusion The proposed driver mental load identification model can discriminate the driving scene quickly and accurately, and then identify the driver mental load. In this way, our model can be more suitable for actual driving and improve the effect of driver mental load identification.
View
Efficient Driver Anomaly Detection via Conditional Temporal Proposal and Classification Network
Article
  • Jan 2022
Detecting driver inattentive behaviors is crucial for driving safety in a driver monitoring system (DMS). Recent works treat driver distraction detection as a multiclass action recognition problem or a binary anomaly detection problem. The former approach aims to classify a fixed set of action classes. Although specific distraction classes can be predicted, this approach is inflexible to detect unknown driver anomalies. The latter approach mixes all distraction actions into one class: anomalous driving. Because the objective focuses on finding the difference between safe and distracted driving, this approach has better generalization in detecting unknown driver distractions. However, a detailed classification of the distraction is missing from the predictions, meaning that the downstream DMS can only treat all distractions with the same severity. In this work, we propose a two-phase anomaly proposal and classification framework [driver anomaly detection and classification network (DADCNet)] robust for open-set anomalies while maintaining high-level distraction understanding. DADCNet makes efficient allocation of multimodal and multiview inputs. The anomaly proposal network first utilizes a subset of the available modalities and views to suggest suspicious anomalous driving behavior. Then, the classification network employs more features to verify the anomaly proposal and classify the proposed distraction action. Through extensive experiments in two driver distraction datasets, our approach significantly reduces the total amount of computation during inference time while maintaining high anomaly detection sensitivity and robust performance in classifying common driver distractions.
View
Driver Distraction Detection Using Bidirectional Long Short-Term Network Based on Multiscale Entropy of EEG
Article
  • Oct 2022
Driver distraction diverting drivers’ attention to unrelated tasks and decreasing the ability to control vehicles, has aroused widespread concern about driving safety. Previous studies have found that driving performance decreases after distraction and have used vehicle behavioral features to detect distraction. But how brain activity changes while distraction remains unknown. Electroencephalography (EEG), a reliable indicator of brain activities has been widely employed in many fields. However, challenges still exist in mining the distraction information of EEG in realistic driving scenarios with uncertain information. In this paper, we propose a novel framework based on Multi-scale entropy (MSE) in a sliding window and Bidirectional Long Short-term Memory Network (BiLSTM) to explore the distraction information of EEG to detect driver distraction based on multi-modality signals in real traffic. Firstly, MSE with sliding window is implemented to extract the EEG features to determine the distraction position. Statistical analysis of vehicle behavioral data is then performed to validate driving performance indeed changes around distraction position. Finally, we use BiLSTM to detect driver distraction with MSE and other traditional features. Our results show that MSE notably decreases after distraction. Consistent with the result of MSE, driving performance significantly deviates from the normal state after distraction. Besides, BiLSTM performance of MSE outperforms other entropy-based methods and is better than behavioral features. Additionally, the accuracy is improved again after adding MSE feature to behavioral features with a 3% increasement. The proposed framework is useful for mining brain activity information and driver distraction detection applications in realistic driving scenarios.
View
Internet Video Traffic Classification with Convolutional Neural Networks
Conference Paper
Full-text available
  • Nov 2021
The presented article provides results on Deep Learning model application for internet traffic classification. The classification task is performed for video-traffic of two types: real-time streaming video and on-demand video record. Multiple Machine Learning methods have been used to solve the specified task, and this article presents results obtained by Convolutional Neural Networks method. The article describes data pre-processing and formatting, as well as used model parameters. The article is concluded with some of the obtained results presented in form of Tables and 3D surface plots. The conclusion finalizes the article.
View
Multi-Objective Genetic Algorithm for Optimizing an ELM-Based Driver Distraction Detection System
Article
  • Sep 2021
Driver Assistance Systems (DAS) have been progressively incorporated into commercial vehicles in recent years. All these systems are paving the way for the forthcoming autonomous vehicle which will become a reality in the near future. Existing systems are based on numerous electronic systems with advanced skills, high performances, and high degrees of adaptability and intelligence. As is to be expected, these cutting-edge features require, in most cases, the use of powerful computing platforms. However, the deployment of such platforms is not an easy task, since they have to be integrated in the vehicle where there exist important restrictions regarding size, power consumption and cost. In this sense, every smart proposal aimed at reducing the complexity of these systems without degrading performance, is always a valuable contribution in the field. In this work, we propose a methodology to reduce the dimensionality of a driver distraction recognition system. The methodology is based on a multi-objective genetic algorithm that looks for the minimum set of useful features collected during the driving task and also for the simplest recognition system. The recognition algorithm is an Extreme Learning Machine (ELM) whose simplicity and fast learning procedure make it especially suitable to be used by a Genetic Algorithm which needs to evaluate thousands of candidate solutions. The proposed methodology has been tested with a real-world database collected from different drivers performing an itinerary with an instrumented car. The results obtained validate the proposal as a method to reduce the complexity of a driver distraction recognition system.
View
Computer Assisted Analysis of Drivers' Body Activity Using a Range Camera
Article
Full-text available
  • Sep 2015
A significant amount of research has been involved with the development of advanced driver-assistance systems. Such systems typically include radars, laser or video sensors that detect the vehicle trajectory and warn for an imminent lane departure, or sense the front vehicle's speed and apply the brakes of the following vehicle to maintain safe distance headways (i.e., collision avoidance system). However, most of these systems rely on the subject vehicle and surrounding vehicles' position and do not explicitly consider the driver's actions during the driving task. In addition, safety research has focused on eye tracking as a means of capturing driver's attention, fatigue, or drowsiness; however, the body posture has not been investigated in depth. This paper presents a novel approach for studying the actual movements of drivers inside the vehicle, when performing specific maneuver types such as lane changing and merging. A pilot study was conducted along a freeway and arterial segment, where the 3D shapes of selected participants were constructed with the use of Microsoft Kinect range camera while merging and changing lanes. A 7-point human skeletal model was fit to the captured range data (depth frame sequences) using the proposed framework. The analysis of the captured 3D data showed that there are important differences between participants when performing similar driving maneuvers. The preliminary results of this pilot research set the basis for implementing the proposed methodological framework for conducting full-scale experiments with a variety of participants, and exploring differences due to driver behavior attributes, such as age, gender and driving experience.
View
An Innovative Nonintrusive Driver Assistance System for Vital Signal Monitoring
Article
Full-text available
  • Nov 2014
This paper describes an in-vehicle nonintrusive biopotential measurement system for driver health monitoring and fatigue detection. Previous research has found that the physiological signals including eye features, electrocardiography (ECG), electroencephalography (EEG) and their secondary parameters such as heart rate and HR variability are good indicators of health state as well as driver fatigue. A conventional biopotential measurement system requires the electrodes to be in contact with human body. This not only interferes with the driver operation, but also is not feasible for long-term monitoring purpose. The driver assistance system in this paper can remotely detect the biopotential signals with no physical contact with human skin. With delicate sensor and electronic design, ECG, EEG, and eye blinking can be measured. Experiments were conducted on a high fidelity driving simulator to validate the system performance. The system was found to be able to detect the ECG/EEG signals through cloth or hair with no contact with skin. Eye blinking activities can also be detected at a distance of 10 cm. Digital signal processing algorithms were developed to decimate the signal noise and extract the physiological features. The extracted features from the vital signals were further analyzed to assess the potential criterion for alertness and drowsiness determination.
View
Online Prediction of Driver Distraction Based on Brain Activity Patterns
Article
Full-text available
  • Jul 2014
This paper presents a new computational framework for early detection of driver distractions (map viewing) using brain activity measured by electroencephalographic (EEG) signals. Compared with most studies in the literature, which are mainly focused on the classification of distracted and nondistracted periods, this study proposes a new framework to prospectively predict the start and end of a distraction period, defined by map viewing. The proposed prediction algorithm was tested on a data set of continuous EEG signals recorded from 24 subjects. During the EEG recordings, the subjects were asked to drive from an initial position to a destination using a city map in a simulated driving environment. The overall accuracy values for the prediction of the start and the end of map viewing were 81% and 70%, respectively. The experimental results demonstrated that the proposed algorithm can predict the start and end of map viewing with relatively high accuracy and can be generalized to individual subjects. The outcome of this study has a high potential to improve the design of future intelligent navigation systems. Prediction of the start of map viewing can be used to provide route information based on a driver's needs and consequently avoid map-viewing activities. Prediction of the end of map viewing can be used to provide warnings for potential long map-viewing durations. Further development of the proposed framework and its applications in driver-distraction predictions are also discussed.
View
A Vision-Based System for Monitoring the Loss of Attention in Automotive Drivers
Article
Full-text available
  • Dec 2013
Onboard monitoring of the alertness level of an automotive driver has been challenging to research in transportation safety and management. In this paper, we propose a robust real-time embedded platform to monitor the loss of attention of the driver during day and night driving conditions. The percentage of eye closure has been used to indicate the alertness level. In this approach, the face is detected using Haar-like features and is tracked using a Kalman filter. The eyes are detected using principal component analysis during daytime and using the block local-binary-pattern features during nighttime. Finally, the eye state is classified as open or closed using support vector machines. In-plane and off-plane rotations of the driver's face have been compensated using affine transformation and perspective transformation, respectively. Compensation in illumination variation is carried out using bihistogram equalization. The algorithm has been cross-validated using brain signals and, finally, has been implemented on a single-board computer that has an Intel Atom processor with a 1.66-GHz clock, a random access memory of 1 GB, ×86 architecture, and a Windows-embedded XP operating system. The system is found to be robust under actual driving conditions.
View
Gaze Fixation System for the Evaluation of Driver Distractions Induced by IVIS
Article
Full-text available
  • Sep 2012
We present a method to monitor driver distraction based on a stereo camera to estimate the face pose and gaze of a driver in real time. A coarse eye direction is composed of face pose estimation to obtain the gaze and driver's fixation area in the scene, which is a parameter that gives much information about the distraction pattern of the driver. The system does not require any subject-specific calibration; it is robust to fast and wide head rotations and works under low-lighting conditions. The system provides some consistent statistics, which help psychologists to assess the driver distraction patterns under influence of different in-vehicle information systems (IVISs). These statistics are objective, as the drivers are not required to report their own distraction states. The proposed gaze fixation system has been tested on a set of challenging driving experiments directed by a team of psychologists in a naturalistic driving simulator. This simulator mimics conditions present in real driving, including weather changes, maneuvering, and distractions due to IVISs. Professional drivers participated in the tests.
View
Driver Alertness Monitoring Using Fusion of Facial Features and Bio-Signals
Article
Full-text available
  • Jul 2012
Driver drowsiness is among the leading causal factors in traffic accidents occurring worldwide. This paper describes a method to monitor driver safety by analyzing information related to fatigue using two distinct methods: eye movement monitoring and bio-signal processing. A monitoring system is designed in Android-based smartphone where it receives sensory data via wireless sensor network and further processes the data to indicate the current driving aptitude of the driver. It is critical that several sensors are integrated and synchronized for a more realistic evaluation of the driver behavior. The sensors applied include a video sensor to capture the driver image and a bio-signal sensor to gather the driver photoplethysmograph signal. A dynamic Bayesian network framework is used for the driver fatigue evaluation. A warning alarm is sounded if driver fatigue is believed to reach a defined threshold. The manifold testing of the system demonstrates the practical use of multiple features, particularly with discrete methods, and their fusion enables a more authentic and ample fatigue detection.
View
Driver Monitoring Based on Low-Cost 3-D Sensors
Article
  • Aug 2014
A solution for driver monitoring and event detection based on 3-D information from a range camera is presented. The system combines 2-D and 3-D techniques to provide head pose estimation and regions-of-interest identification. Based on the captured cloud of 3-D points from the sensor and analyzing the 2-D projection, the points corresponding to the head are determined and extracted for further analysis. Later, head pose estimation with three degrees of freedom (Euler angles) is estimated based on the iterative closest points algorithm. Finally, relevant regions of the face are identified and used for further analysis, e.g., event detection and behavior analysis. The resulting application is a 3-D driver monitoring system based on low-cost sensors. It represents an interesting tool for human factor research studies, allowing automatic study of specific factors and the detection of special event related to the driver, e.g., driver drowsiness, inattention, or head pose.
View
Visual Analysis of Eye State and Head Pose for Driver Alertness Monitoring
Article
  • Sep 2013
This paper presents visual analysis of eye state and head pose (HP) for continuous monitoring of alertness of a vehicle driver. Most existing approaches to visual detection of nonalert driving patterns rely either on eye closure or head nodding angles to determine the driver drowsiness or distraction level. The proposed scheme uses visual features such as eye index (EI), pupil activity (PA), and HP to extract critical information on nonalertness of a vehicle driver. EI determines if the eye is open, half closed, or closed from the ratio of pupil height and eye height. PA measures the rate of deviation of the pupil center from the eye center over a time period. HP finds the amount of the driver's head movements by counting the number of video segments that involve a large deviation of three Euler angles of HP, i.e., nodding, shaking, and tilting, from its normal driving position. HP provides useful information on the lack of attention, particularly when the driver's eyes are not visible due to occlusion caused by large head movements. A support vector machine (SVM) classifies a sequence of video segments into alert or nonalert driving events. Experimental results show that the proposed scheme offers high classification accuracy with acceptably low errors and false alarms for people of various ethnicity and gender in real road driving conditions.
View
Real-Time Detection System of Driver Distraction Using Machine Learning
Article
  • Jun 2013
There is accumulating evidence that driver distraction is a leading cause of vehicle crashes and incidents. In particular, increased use of so-called in-vehicle information systems (IVIS) and partially autonomous driving assistance systems (PADAS) have raised important and growing safety concerns. Thus, detecting the driver's state is of paramount importance, to adapt IVIS and PADAS accordingly, therefore avoiding or mitigating their possible negative effects. The purpose of this paper is to show a method for the nonintrusive and real-time detection of visual distraction, using vehicle dynamics data and without using the eye-tracker data as inputs to classifiers. Specifically, we present and compare different models that are based on well-known machine learning (ML) methods. Data for training the models were collected using a static driving simulator, with real human subjects performing a specific secondary task [i.e., a surrogate visual research task (SURT)] while driving. Different training methods, model characteristics, and feature selection criteria have been compared. Based on our results, using a support vector machine (SVM) has outperformed all the other ML methods, providing the highest classification rate for most of the subjects. Potential applications of this paper include the design of an adaptive IVIS and of a “smarter” PADAS.
View
Processing of Eye/Head-Tracking Data in Large-Scale Naturalistic Driving Data Sets
Article
  • Jun 2012
Driver distraction and driver inattention are frequently recognized as leading causes of crashes and incidents. Despite this fact, there are few methods available for the automatic detection of driver distraction. Eye tracking has come forward as the most promising detection technology, but the technique suffers from quality issues when used in the field over an extended period of time. Eye-tracking data acquired in the field clearly differs from what is acquired in a laboratory setting or a driving simulator, and algorithms that have been developed in these settings are often unable to operate on noisy field data. The aim of this paper is to develop algorithms for quality handling and signal enhancement of naturalistic eye- and head-tracking data within the setting of visual driver distraction. In particular, practical issues are highlighted. Developed algorithms are evaluated on large-scale field operational test data acquired in the Sweden–Michigan Field Operational Test (SeMiFOT) project, including data from 44 unique drivers and more than 10 000 trips from 13 eye-tracker-equipped vehicles. Results indicate that, by applying advanced data-processing methods, sensitivity and specificity of eyes-off-road glance detection can be increased by about 10%. In conclusion, postenhancement and quality handling is critical when analyzing large databases with naturalistic eye-tracking data. The presented algorithms provide the first holistic approach to accomplish this task.
View