Content uploaded by Hussein A. A. Al-Khamees
Author content
All content in this area was uploaded by Hussein A. A. Al-Khamees on Jun 20, 2022
Content may be subject to copyright.
Classifying the Human Activities
of Sensor Data Using Deep Neural
Network
Hussein A. A. Al-Khamees(B
), Nabeel Al-A’araji ,
and Eman S. Al-Shamery
Babylon University, Babylon - Hilla, Iraq
Hussein.alkhamees7@gmail.com,
{nhkaghed,emanalshamery}@itnet.uobabylon.edu.iq
Abstract. Today sensors represent one of the most important appli-
cations for generating data stream. This data has a number of unique
characteristics, including fast data access, huge volume, as well as the
most prominent feature, the concept drift. Machine learning in general
and deep learning technique in particular is among the predominant and
successful selections to classify the human activities. This is due to sev-
eral reasons such as results quality and processing time. The recognition
of human activities that produced from sensors considers is an effec-
tive and vital task in the healthcare field, meanwhile, it is an attractive
to researchers. This paper presents a DNN model to classify the human
activities of the HuGaDB sensor dataset by implementing multilayer per-
ceptron (MLP) structure. The current model achieved results, 91.7% of
accuracy, 92.5% precision, 92.0% recall, and 92.0% of F1-score, using a
tiny time. The model results were compared with the previous models
and it has proven its efficiency by outperforming those models.
Keywords: Deep neural network ·MultiLayer Perceptron (MLP) ·
Human activities classification ·Sensor data stream ·HuGaDB dataset
1 Introduction
Many real-world applications in different domains can generate a massive amount
of data; It is known as a data stream which has various unique properties that
traditional data do not have. Some of these characteristics are unlimited data
size, fast-access data from the source, limited memory, processing time and the
evolving in its nature this causes the concept drift [1]. Most traditional data algo-
rithms fail when dealing with a data stream, this is due to newly characteristics
of the data stream [2].
Machine learning is a sub-field of Artificial Intelligence (AI). In reality,
machine learning consists of many techniques that can be used on data stream
Supported by Babylon University.
c
Springer Nature Switzerland AG 2022
A. Bennour et al. (Eds.): ISPR 2022, CCIS 1589, pp. 107–118, 2022.
https://doi.org/10.1007/978-3-031-08277-1_9
108 H. A. A. Al-Khamees et al.
such as classification, clustering, regression, ..., etc. [3]. Despite these techniques,
neural networks (NN) are just as important as those techniques and that can
be also implemented on the data stream. Neural networks have two types, shal-
low or deep. Recently, deep learning techniques that use deep neural networks
(DNN) are a major area of interest and increasingly being applied [4]. Therefore,
DNN is applied in various fields such as healthcare [5].
Deep learning depends mainly on Artificial Neural Networks (ANN), which
are originally inspired by neurons in the human brain [6]. However, the DNNs
structure involves three layers (input, hidden and output), where every layer
has several neurons and the neuron numbers differ from a layer to another [4].
Multilayer Perceptron (MLP) is an important and widely used architectural type
of deep learning [7].
The recognition task of human activities that produced from sensors consid-
ers is an effective and vital task in the healthcare field. Indeed, the recognition
models are either wearable or external sensor-based models [8].
This paper presents a deep learning model based on MLP and the back-
propagation algorithm to train MLP for classifying the human activities. This
model consists of four hidden layers that able to implement the classification
task in a short period of time. For evaluating the proposed model, the HuGaBD
sensor dataset was used. More specifically, five sub-datasets of the main HuGaDB
dataset were selected for the current model.
The proposed model achieved results as follow, 91.7% of accuracy, 92.5% pre-
cision, 92.0% recall, and 92.0 % of F1-score, using a tiny time. Accordingly, this
model outperforms many previous models that used the same dataset (HuGaDB
dataset) to classify the human activities. Furthermore, our evaluation demon-
strates how this DNN model proved the enhancing of results by implementing it
into different numbers of both hidden layers and also neurons for every hidden
layer.
The current paper organizes as follows. Section 2discusses related works
which related to deep neural network that implemented on HuGaDB dataset.
Section 3explains the DNN structure. The methodology of the proposed model
is presented in Sect. 4. While dataset description is introduces in Sect. 5. Section
6illustrates the evaluation metrics and Sect. 7dedicates to the results of the
model and finally, the conclusion of the current paper summarizes.
2 Related Work
This section covers the studies based on NNs as a machine learning technique
that applied to the HuGaDB dataset to classify the human activities.
In [9], the authors presented model aims to classify different activities of the
human. The model depended on ANN to estimate many parameters such as
IMUs displacements, velocity, and angle. The study focuses on three body area
that are shin, thigh and waist that resulted in accurate results of the lower limbs
of the human body. Moreover, the proposed model aims to solve an important
issue, which is the contradictions that occur (while capturing the motion signal)
Classifying the Human Activities of Sensor Data 109
to the movements of body parts such as the hand and the leg. In general, the
model consists of two phases that are training and application. In the first phase,
the ANN is trained to estimate the received signals while in the second phase,
the ANN that was trained is implemented to estimate the signals related to the
lower extremities (during real time). This model achieved an accuracy of 88.0%.
Accordingto[10], the authors applied feature vector length reduction and
how it affects deep learning networks besides other techniques of machine learn-
ing. The key idea behind the model is to apply Long Short-term Memory (LSTM)
as a deep learning classifier to extract different high dimensional features. The
model has several phases which are data pre-processing, feature extraction,
feature selection, training and finally the testing phase. The proposed model
attained an accuracy of 91.1%.
B. Fang et al. [11] suggested a gait neural network (GNN) model which
depended on a temporal convolutional neural network. The model aims to predict
a human activity in the lower limbs. In general, the structure of the proposed
model consists of gait prediction and gait recognition where it focuses on the
gait data that received from the right leg. The accuracy achieved by the model
based on GNN is 79.24%, which is considered the highest accuracy among the
techniques used in the same study.
3 DNN Structure
The DNN structure consists of three layer types that are, input layer, hidden
layers and output layer. The data are received from the external source through
the input layer, therefore there isn’t any processing (computations). Most of the
processing steps that implemented in the hidden layers are nonlinear computa-
tions, whereas the processing in the output layer either linear or nonlinear [12].
The nonlinear transforming which starts from the input to the hidden layers till
the output layer, is called as the forward propagation.
The number of hidden layers and the number of neurons in each layer has an
effective effect on the final results of the deep neural network model. Therefore,
it must be carefully selected (after testing) [13].
Each layer contains several neurons, take into consideration that the neuron
number are differs from a layer to another. In a specific layer, every neuron
is connected to their counterparts in adjacent layers. This connection can be
indicated by weights which reflecting both strength and direction. Every neuron
can transform data through computation of weighted sum (of the output neurons
in past layer) and then passes it by a nonlinear function (activation functions)
for deriving the neuron outputs [14].
3.1 Multilayer Perceptron (MLP)
It’s a feed-forward neural network with multi hidden layers. MLP doesn’t require
any prior assumptions about the distribution of data. In MLP, the neurons are
connected by weights and also the signals of output that represented as a function
of the sum of the inputs to the neuron modified by an activation function [15].
110 H. A. A. Al-Khamees et al.
3.2 MLP Training
Usually, the training of the deep neural network is more difficult and complex
than the classic neural network [16].
The training of DNN contains many sequential steps for adjusting the weights
between the neurons in the network, in a similar way to the learning of the human
brain. But before the adjustment step, the model must initialize these weights.
This initialization is done randomly [17], where the resulting weights have the
ability to [18]:
– Maximize the relationship strength between network input and its output.
– Minimize a difference of the neural networks (such as an error) between a
specific task and its real target (i.e. between the network prediction and its
associated target). Usually, a neural network technique aims to minimize this
error value.
More specifically, the back propagation (BP) is the most successful and widely
used algorithm for MLP training [15]. BP repeatedly can analyze the errors and
optimize every value of weight depending on the errors that generated by the
next layer [18]. Accordingly, this algorithm was used in the current model.
To simplify the weight computation, suppose a neural network contains (m)
neuron, this neuron is driven by input vector Xn, where n indicates to the time
step of the iterative process contains the adjusting step of the input weights
w(mi). Therefore, each sample of data passes through the training step of a
DNN containing X(n)and its output denoting by d(n).
Then the processing step to X(n), of a neuron (m) is generating an output
which is referred by ym(n), and computed by:
ym(n)=f
j
i=1
x.w(mi)(1)
where f indicates to activation function. This output is compared with the target
output dm(n)which normally is given in a sample. The error em(n)can compute
by:
em(n)=(dm(n)−ym(n)) (2)
Because its capacity of the back propagation, it is a very appropriate method
to problems that don’t have any relation between the input and output [19].
4 Methodology
The proposed DNN model consists of:
1. Prepare the dataset that will be used in DNN model. In this model, HuGaDB
is used.
Classifying the Human Activities of Sensor Data 111
2. Apply the data pre-processing step by implementing an appropriate tech-
nique. Normalization is a major step in most problems. The normalization
technique has several methods, including the Min-max that applied to this
model. Mathematically, if there is a set of matching scores (Ms) where, s =
1, 2, ..., n, the normalized scores (Ms’) calculate as:
Ms=(Ms−min)/(max −min) (3)
3. Divide the dataset into training and testing data by applying a suitable tech-
nique. In this model, the cross validation is used, 80% as a training data and
20% as a testing data.
4. Determine the number of hidden layers that required to build the MLP model.
For further analysis, two and four hidden layers were applied.
5. Determine the number of neurons in each hidden layer. In the case of two
hidden layers, the number of neuron is set to (10, 10, 12) while in the case of
four hidden layers, the number of neuron is set to (10, 12, 14, 26, 30).
6. Determine the training algorithm. In this model, the back propagation (BP) is
used for training MLP. In addition to the number of hidden layers and neurons
for every layer, setting another parameters such as, (a) the weights that can
be computed according to equation (1); (b)the error based on equation (2);
and (c) the learning rate that set to 0.001.
7. Start the training phase using the training data (step 3) and parameters
(steps 4 and 5) by the back-propagation (BP) training method (step 6).
8. Start the testing phase by using the test data (step 3). However, in this phase,
the ability of the proposed model is tested if it has been trained to accurately
classify data samples.
9. After completing the training and testing phases, the evaluation step is imple-
mented, as it is the last step in this model. The model uses four different mea-
sures, namely, accuracy, precision, recall and F1-score to evaluate the current
model.
Figure 1shows the model methodology.
5 Data Set Description
Human Gait Database (HuGaDB) to activity recognition from six inertial sen-
sor networks was presented in 2017 [20], these sensors can be shown in Fig. 2(a).
HuGaDB dataset contains 12 behaviors actions which are: walking, sitting, sit-
ting down, sitting in a car, going up, going down, standing, standing up, up
by elevator, down by elevator, bicycling, and running. However, some of these
actions are displayed in Fig.2(b).
According to these behaviors actions, the dataset contains static and dynamic
activities. These several activities are implemented and recorded at various times
like recording the running behavior over about 20 min. Additionally, all the
behaviors actions are gathered by 18 participants.
Furthermore, the main HuGaDB dataset consists of 637 data files and all of
them has the same number of features that are 39 features. Also, all these files
112 H. A. A. Al-Khamees et al.
Fig. 1. Methodology of proposed DNN model.
contain the sentence (various) in their titles to indicate the various activities it
contains. This dataset is a publicly available1.
In the current model, five sub-datasets from the main HuGaDB dataset are
used therefore, it 10 of the 12 activities have been covered through this study.
The activities covered are all activities above except sitting in a car and bicycling.
These sub-datasets are:
1. HuGaDB-v2-various-01-01: consists of 2435 records and it has four classes
that are, ‘sitting’, ‘sitting-down’, ‘standing’, and ‘standing-up’. This dataset
denotes by DS1.
2. HuGaDB-v2-various-05-12: it has 4393 records and it has three classes that
are, ‘going-down’, ‘standing’, and ‘walking’. DS2 is the symbol of this dataset.
3. HuGaDB-v2-various-13-10: it contains 4850 records and it has three classes
that are, ‘down-by-elevator’, ‘standing’, and ‘up-by-elevator’. HuGaDB-v2-
various-13-10 has the symbol DS3.
4. HuGaDB-v2-various-14-05: this dataset has 2392 records. Two classes for this
dataset which are ‘running’ and ‘walking’ and denotes by DS4.
5. HuGaDB-v2-various-17-07: it consists of 2930 records and it has three classes
that are, ’going-up’, ‘standing’, and ‘walking’. DS5 is the symbol for this
dataset.
6 Evaluation Metrics
The performance of the proposed model is evaluated by four different measure-
ments that are [10]:
1https://www.kaggle.com/romanchereshnev/hugadb-human-gait-database.
Classifying the Human Activities of Sensor Data 113
1. Accuracy (refers to the ratio of all true cases divided by the overall dataset
cases).
Accuracy = TP + TN/(TP + TN + FP + FN)
2. Precision (determine the number of true cases predictions which really belong
to the true cases).
Precision = TP/(TP + FP)
3. Recall (determines the number of true cases predictions that implemented
over all true cases).
Recall = TP/(TP + FN)
4. F1-score (indicates the harmonic mean measure to both the precision and
recall).
F1-score = 2 ×(P r ecision ×Recall)/(P recision +Recall)
7 Results
The model implements with two hidden layers, four hidden layers respectively.
Figure 2shows the comparison of accuracy between the two implementations.
Fig. 2. The accuracy for all sub-datasets of HuGaDB dataset with two and four hidden
layers.
Furthermore, Table 1describes the measurements with two hidden layers,
while these measurements with four hidden layers, detail in Table2, and the
best results are highlighted in bold font. While Figs. 3and 4visualize these
measurement values.
114 H. A. A. Al-Khamees et al.
Table 1. The measurements of the model (two hidden layers).
Dataset name Accuracy Precision Recall F1-score
DS1 95.0 96.3 92.5 94.4
DS2 92.3 61.5 63.2 62.3
DS3 74.0 75.9 74.0 74.9
DS4 97.2 97.7 96.4 97.0
DS5 92.4 93.8 93.9 93.8
AVE 90.1 85.0 84.0 84.4
Table 2. The measurements of the model (four hidden layers).
Dataset name Accuracy Precision Recall F1-score
DS1 97.2 97.3 96.4 96.9
DS2 94.0 95.8 96.0 95.9
DS3 76.8 78.7 77.1 77.9
DS4 98.1 98.1 97.8 97.5
DS5 92.4 92.6 92.9 91.9
AVE 91.7 92.5 92.0 92.0
Fig. 3. The measurements of implementation with two hidden layers for every dataset.
Classifying the Human Activities of Sensor Data 115
Fig. 4. The measurements of implementation with four hidden layers for every dataset.
After all these implementations, we notice that the proposed model with
four hidden layers achieves higher accuracy (in all five sub-datasets) than its
counterpart when implementing with two hidden layers.
In the same context, it achieves the highest results in terms of other mea-
surements (precision, recall, and also F1-score) as shown in Table2and Fig. 4.
In fact, the overall accuracy of the proposed model is 91.7 %, which is supe-
rior to many other methods that implemented on the same dataset (HuGaDB
dataset). Table 3and Fig. 5indicate the comparison between the previous models
and our model that implemented for HuGaDB dataset.
Additionally, in term of processing time, the proposed DNN model needs
1.71 s to classify the first sub-dataset (DS1) and 1.66 s to the second sub-dataset
(DS2). While it needs 1.84 s to implement (DS3) and 1.93 s for (DS4). Finally,
it requires 1.85 s to do the classification of the last sub-datasets (DS5). Figure 6
indicates these time details.
Table 3. The accuracy comparisons between previous models and our model.
No Study, publication year Accuracy %
1 [9], 2018 88.0
2 [10], 2020 91.1
3 [11], 2020 79.2
4Our model 91.7
116 H. A. A. Al-Khamees et al.
Fig. 5. The accuracy comparisons between previous models and our model.
Fig. 6. The time needed to implement every dataset.
Classifying the Human Activities of Sensor Data 117
8 Conclusion
The past decade witnessed a prominent development in sensors to generate the
data stream in various fields includes the health field. In this field, the classifica-
tion of the patient’s activities has become the focus of many researchers because
it provides knowledge of the current state of a patient.
Deep Neural Networks (DNNs) are the latest and most preferred machine
learning techniques, especially when processing the data stream. DNN includes
many architectures, the Multi-Layer Perceptron (MLP) is a significant architec-
ture.
This paper presents a deep neural network model based MLP architecture to
classify the human activity during a tiny time. The proposed model was tested
by HuGaDB dataset and evaluated its performance by four measurements which
are accuracy, precision, recall and F1-score. The results proved the superiority
of the proposed model over the previous works, as it achieved an accuracy of
91.7 %, precision of 92.5 %, recall of 92.0% recall, and F1-score as 92.0%.
References
1. Al-Khamees, H.A.A., Al-A’ara ji, N., Al-Shamery, E.S.: Survey: clustering tech-
niques of data stream. In: 1st Babylon International Conference on Information
Technology and Science (BICITS), pp. 113–119. IEEE, Babil (2021). https://doi.
org/10.1109/BICITS51482.2021.9509923
2. Bahri, M., Bifet, A.: Incremental k-nearest neighbors using reservoir sampling
for data streams. In: Soares, C., Torgo, L. (eds.) DS 2021. LNCS (LNAI), vol.
12986, pp. 122–137. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-
88942-5 10
3. Al-Khamees, H.A.A., Al-A’araji, N., Al-Shamery, E.S.: Data stream clustering
using fuzzy-based evolving Cauchy algorithm. Int. J. Intell. Eng. Syst. 14(5), 348–
358 (2021). https://doi.org/10.22266/ijies2021.1031.31
4. Saikiaa, P., Baruaha, R.D., Singhb, S.K., Chaudhurib, P.K.: Artificial neural net-
works in the domain of reservoir characterization: a review from shallow to deep
models. Comput. Geosci. 135, 104357 (2020). https://doi.org/10.1016/j.cageo.
2019.104357
5. Al-Khamees, H.A.A., Al-Jwaid, W.R.H., Al-Shamery, E.S.: The impact of using
convolutional neural networks in COVID-19 tasks: a survey. Int. J. Comput. Digit.
Syst. 11(1), 189–197 (2022). https://doi.org/10.12785/ijcds/110194
6. Lee, J., Chang, C., Kao, T., Wang, J.: Age estimation using correlation-refined
features of convolutional neural network. J. Inf. Sci. Eng. 37(6), 1435–1448 (2021).
https://doi.org/10.6688/JISE.202111-37(6).0014
7. Zhang, S., Yao, L., Sun, A., Tay, Y.: Deep learning based recommender system:
a survey and new perspectives. ACM Comput. Surv. (CSUR) 52(1), 1–38 (2019).
https://doi.org/10.1145/3285029
8. Jansi, R., Amutha, R.: A novel chaotic map based compressive classification scheme
for human activity recognition using a tri-axial accelerometer. Multimed. Tools
Appl. 77(23), 31261–31280 (2018). https://doi.org/10.1007/s11042-018-6117-z
118 H. A. A. Al-Khamees et al.
9. Sun, Y., Yang, G., Lo, B.: An artificial neural network framework for lower limb
motion signal estimation with foot-mounted inertial sensors. In: 15th International
Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 132–
135. IEEE, Las Vegas (2018). https://doi.org/10.1109/BSN.2018.8329676
10. Kumari, G., Chakraborty, J., Nandy, A.: Effect of reduced dimensionality on deep
learning for human activity recognition. In: 11th International Conference on Com-
puting. Communication and Networking Technologies (ICCCNT), pp. 1–7. IEEE,
Kharagpur (2020). https://doi.org/10.1109/ICCCNT49239.2020.9225419
11. Fang, B., et al.: Gait neural network for human-exoskeleton interaction. Front.
Neurorobot. 14, 1–9 (2020). https://doi.org/10.3389/fnbot.2020.00058
12. Sarker, I.H.: Deep learning: a comprehensive overview on techniques, taxonomy,
applications and research directions. SN Comput. Sci. 2(6), 1–20 (2021). https://
doi.org/10.1007/s42979-021-00815-1
13. Madhiarasan, M., Deepa, S.N.: Comparative analysis on hidden neurons estimation
in multi layer perceptron neural networks for wind speed forecasting. Artif. Intell.
Rev. 23, 1–23 (2016). https://doi.org/10.1007/s10462-016-9506-6
14. Goli, P.: A new perceptually weighted cost function in deep neural network based
speech enhancement systems. Hear. Balance Commun. 17(3), 191–196 (2019).
https://doi.org/10.1080/21695717.2019.1603948
15. Gardner, M.W., Dorling, S.R.: Artificial neural networks (the multilayer percep-
tron) a review of applications in the atmospheric sciences. Atmos. Environ. 32(14–
15), 2627–2636 (1998). https://doi.org/10.1016/S1352-2310(97)00447-0
16. Xu, Z.-Q.J., Zhang, Y., Xiao, Y.: Training behavior of deep neural network in
frequency domain. In: Gedeon, T., Wong, K.W., Lee, M. (eds.) ICONIP 2019.
LNCS, vol. 11953, pp. 264–274. Springer, Cham (2019). https://doi.org/10.1007/
978-3-030-36708- 4 22
17. Larochelle, H., Bengio, Y., Louradour, J., Lamblin, P.: Exploring strategies for
training deep neural networks. J. Mach. Learn. Res. 10(1), 1–40 (2009)
18. Vieira, S., Pinaya, W.H.L., Garcia-Dias, R., Mechelli, A.: Machine Learning Meth-
ods and Applications to Brain Disorders, 1st edn. Academic Press, San Diego
(2019)
19. Nawi, N.M., Khan, A., Rehman, M.Z.: A new back-propagation neural network
optimized with Cuckoo search algorithm. In: Murgante, B., et al. (eds.) ICCSA
2013. LNCS, vol. 7971, pp. 413–426. Springer, Heidelberg (2013). https://doi.org/
10.1007/978-3-642-39637- 3 33
20. Chereshnev, R., Kert´esz-Farkas, A.: HuGaDB: human gait database for activity
recognition from wearable inertial sensor networks. In: van der Aalst, W.M.P.,
et al. (eds.) AIST 2017. LNCS, vol. 10716, pp. 131–141. Springer, Cham (2018).
https://doi.org/10.1007/978-3-319- 73013-4 12
