Pattern identification of different human joints for different human walking styles using inertial measurement unit (IMU) sensor

Abstract

A bipedal walking robot is a kind of humanoid robot. It is suppose to mimics human behavior and designed to perform human specific tasks. Currently, humanoid robots are not capable to walk like human being. To perform the walking task, in the current work, human gait data of six different walking styles named brisk walk, normal walk, very slow walk, medium walk, jogging and fast walk is collected through our configured IMU sensor and mobile-based accelerometers device. To capture the pattern for six different walking styles, data is extracted for hip, knee, ankle, shank, thigh and foot. A total six classes of walking activities are explored for clinical examination. The accelerometer is placed at center of the human body of 15 male and 10 female subjects. In the experimental setup, we have done exploratory analysis over the different gait capturing techniques, different gait features and different gait classification techniques. For the classification purpose, three state of art techniques are used as artificial neural network, extreme learning machine and deep neural network learning based CNN mode. The model classification accuracy is obtained as 87.4%, 88% and 92%, respectively. Here, WISDM activity data set is also used for verification purpose.

Full text

Vol.:(0123456789)
Artificial Intelligence Review
https://doi.org/10.1007/s10462-021-09979-x
1 3
Pattern identification ofdifferent human joints fordifferent
human walking styles using inertial measurement unit (IMU)
sensor
VijayBhaskarSemwal1 · NehaGaud2· PraveenLalwani3· VishwanathBijalwan4·
AbhayKumarAlok5
© The Author(s), under exclusive licence to Springer Nature B.V. 2021
Abstract
A bipedal walking robot is a kind of humanoid robot. It is suppose to mimics human
behavior and designed to perform human specific tasks. Currently, humanoid robots are
not capable to walk like human being. To perform the walking task, in the current work,
human gait data of six different walking styles named brisk walk, normal walk, very slow
walk, medium walk, jogging and fast walk is collected through our configured IMU sen-
sor and mobile-based accelerometers device. To capture the pattern for six different walk-
ing styles, data is extracted for hip, knee, ankle, shank, thigh and foot. A total six classes
of walking activities are explored for clinical examination. The accelerometer is placed at
center of the human body of 15 male and 10 female subjects. In the experimental setup,
we have done exploratory analysis over the different gait capturing techniques, different
gait features and different gait classification techniques. For the classification purpose,
three state of art techniques are used as artificial neural network, extreme learning machine
and deep neural network learning based CNN mode. The model classification accuracy is
obtained as 87.4%, 88% and 92%, respectively. Here, WISDM activity data set is also used
for verification purpose.
Keywords Clinical gait· Connectionist learning· Biometrics· Human activities
recognition· Motion analysis· Deep learning
1 Introduction
Gait assessment refers to the study of human locomotion (Semwal 2017; Semwal and
Nandi 2015), which plays a significant role in clinical examination for the identification
of gait abnormality for neurological disorder person (Semwal etal. 2015b; Sivakumar
etal. 2016). This can also be used in biometrics as it is unique and difficult to hide.
Tables1 and  2 show the important comparative study of gait biometrics (Bovi etal.
2011; Patil etal. 2019). The human gait can be utilized for wide number of applications
* Vijay Bhaskar Semwal
vsemwal@manit.ac.in
Extended author information available on the last page of the article
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
e.g., surveillance (Kusakunniran etal. 2009), forensics (Zhang etal. 2011), biometric
identification (Chaki etal. 2019), rehabilitation (Yang etal. 2017), clinical assessment
(Gallow and Heiderscheit 2016) and prosthesis development (Nandi etal. 2009). Real-
time tracking of human walking provide important information about individual’s gait
biometric through which one can investigate different clinical analysis for gait abnor-
mality (Semwal etal. 2018) and reconstruction of gait (Semwal and Nandi 2016). The
human gait is the coordination of two limbs with forward propulsion of Center of Mass
(CoM). The inherent nature of human gait is bipedal & biphasic i.e., swing & stance
phase (Semwal etal. 2013a, 2015c). The gait signatures are captured by changes in the
joints angle of hip, knee, ankle, shank, thigh and foot of left and right human legs (Raj
etal. 2018a).
1.1 Problem description andmotivation
Gait disorder is the most common clinical problem of stroke survivors patients; thus, it
is a major targeted area for the research of post-stroke rehabilitation. Human gait analy-
sis, study of human locomotion & push recovery (Semwal etal. 2013b), is very useful in
diagnosing the patients of neurological disorders, post-stroke hemiparetic patients, reha-
bilitation of patients, analysis of sports-person patterns, unique biometric identification and
other focused research areas. The popularly known human motion capture systems for gait
analysis are vision-based in which markers are positioned on the human body with multi-
ple Infrared (IR) sensors, thus, system is accurate but very cost expensive and should need
a specific controlled environment (Chan etal. 2018). Force plates are also used for gait
analysis but, these devices are very expensive and need specific clinical labs environment
also, it is used to measure the dynamics of lower limbs only (Weiss etal. 2012a).
Table 1 Different physiological based biometric characteristics (Patil etal. 2019)
Biometric traits Universality Distinctiveness Permanence Collectabilty Circumvention
Face High Medium Medium High Low
Iris High High High Medium Low
Palm print High High Medium Medium Medium
Fingerprint High High Medium Medium High
Retina High High High Low Low
Table 2 Different behavior based biometric characteristics (Patil etal. 2019)
Biometric traits Universality Distinctiveness Permanence Collectability Circumvention
Gait Low Medium Low High Low
Speech High Medium High Medium Medium
Signature High Medium Low Medium Low
Keystroke High Medium Low Medium Low
Device uses Low Medium Low High Low
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
1.2 Author’s contribution
The major contribution of this work is to analyze the six different walking activity using
gait analysis. To achieve the objective following tasks are performed.
• The gait data of 25 subjects using IMU based wearable sensors is captured. These sen-
sors are placed to various parts of the body and capture the real time data for different
walking activities.
• The mobile phone based accelerometer is used for walking activities tracking. The
9-degree IMU sensor consists of 3-degree of freedom (dof) accelerometer sensor (ax,
ay & az), 3-dof gyroscope sensor (gx, gy & gz) and 3-degree of magnetometer sensor
(mx, my & mz).
• In the experimentation, the 6-dof IMU sensor (IMU BWT61CL) is used for accelerom-
eter data. The walking trajectories of 6 joins, namely, Hip, Knee, Ankle, Foot, Shank
and Thigh for gait analysis is also being calculated (Gouwanda etal. 2016).
• Finally, the 6 features of raw data named subject-id, time stamp, ax, ay, az, and activi-
ties name is being provided as input for machine learning (ML) algorithm. The ANN
(Nandi etal. 2016), ELM (Semwal etal. 2019) and DNN (Semwal etal. 2017b) based
ML algorithm is being applied for classification.
From the obtained results, better accuracy is achieved as compared to previous work.
The walking pattern of different joints can be utilized for the assessment & recovering of
rehabilitation of individuals with abnormal gait. The muscular activity pattern of a healthy
subject is compared with rehabilitation walk. The proposed technique is very robust &
accurate in clinical analysis. We have reported 87.84%, 88% and 92% accuracy using ANN,
ELM and CNN model respectively for clinical walking data.
1.3 Organization ofresearch article
The remainder section of this paper is divided as follows. The second section is literature
review about gait analysis techniques and comparison of different biometric identification.
This selection also, presents the study of various gait capturing techniques and various gait
classification techniques. Section3, describes about data collection and prepossessing of
raw data. This section also, describes data set description and different human walking
activities recognition tasks. The fourth section is methodology and proposed algorithm.
The fifth section is about results and analysis of different joints angle analysis. This section
also, presents the classification of human gait using neural network, deep learning based
CNN model and ELM. The fifth and final section describes our conclusion and future
scope.
2 Literature review ofrelated work
There are three popularly known techniques to capture the gait data using Kinect sensor,
IMU sensor and force plate sensor based approach for gait analysis. The Kinect sensor
extract 3D virtual skelton data of human being and joint position information data is being
extracted from the stick model. This system analyzes the spatio-temporal features such as
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
gait cycle time, stride length, left & right steps length, and kinematic features like hip,
knee, ankle joints angle. The kinect based technique is further categorized into 2 parts,
model based approach and model free approach. The model based approach is view and
scale invariant but required a lot of parameters. Whereas, IMU & accelerometer sensor
based technique is noninvasive and less parameters. Table3 present the comparison of all
gait data collection techniques.
Hsu et al. (2018) proposed the multiple wearable sensors based technique to analyz-
ing and classifying the gait data of neurological disorders patients. They have collected
gait data for patience suffering from freezing of gait, multiple sclerosis, cerebral palsy and
stroke (Hsu etal. 2018). The seven sensor were placed on 20 subjects to collect features.
They applied many machine leaning for classification out of which Multi-layer Perceptron
(MLP) outperform others. Clinical gait assessment can also be utilized for early diagno-
sis of gait abnormality in neurological disorder subjects. Lau etal. (2009) applied support
vector machine (SVM) for analyzing of walking conditions of stroke survive patient with
dropped foot. In Patil etal. (2019), used different types of classification techniques to iden-
tify the patients suffering from neurological disorders and stroke patients. The performance
of the ELM classifier was found to be the best in terms of accuracy and performance time.
In Adil etal. (2016) proposed surface electromyography (sEMG) signal in order to recog-
nize drop foot of the stroke patients and provide a therapy as they needed for rehabilita-
tion. They used ELM classifier to classify the healthy and weak muscles of the human
leg and the performance was also compared with the SVM and ANN. The performance
of the ELM can be further increased by eliminating sensitivity of its hyper parameters by
considering a variable length optimization using particle swarm algorithm (Ekinci 2006).
This technique was also used to optimize the hidden layers neurons of ELM corresponding
to input weights and biases. Another approach was proposed by Guo etal. (2019) to iden-
tify gait disorders in Parkinson’s disease by using machine learning algorithms in 2019.
This model was used to classify four different types of gait abnormality. Another novel
approach was proposed by Mekruksavanich and Jitpattanakul (2019) where wearable sen-
sors on smartphones were used to collect data. The study was made to classify gait patterns
for three different activities like walking upstairs, walking downstairs and walking on floor
(Mekruksavanich and Jitpattanakul 2019) using different types of learning techniques.
Gait can also be used as a biometric characteristic which can find its application in vari-
ous fields like surveillance and forensics (Connor and Ross 2018; Semwal etal. 2017a; Chel-
lappa et al. 2007; Anusha and Jaidhar 2019). A number of techniques have been proposed
like utilizing mutual information obtained from a query and gallery sample. This method uses
the region of interests (ROI) extracted from gait energy image to perform classification and
identification. In another approach Li etal. (2019), proposed a method in which subject is
identified from extracting skeleton information obtained from single depth sensors. Another
application of gait analysis is in clinical and medical purposes where the patients suffering
Table 3 Different technique to acquire the human gait data
Data acquisition technique References
IMU sensor based Semwal etal. (2017b), Akdoğan and Yilmaz
(2014), Sivakumar etal. (2018)
Computer vision Chellappa etal. (2007), Anusha and Jaidhar (2019)
Model free approach using kinect Guo etal. (2019), Akdoğan and Yilmaz (2014)
Model based approach using kinect Bovi etal. (2011), Milovanovic (2008)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
from neurological disorders can analyze their gait abnormalities and can design personalized
treatment (Papavasileiou etal. 2017). Human gait classification is also an important area of
research interest where a number of techniques are implemented to study and analyze the gait
pattern for different human locomotion activities like walking, running, climbing, etc (Bovi
etal. 2011). Chen etal. (2018) proposed deep convolutional neural networks (DCNNs) based
on multistatic radar micro-Doppler signatures for gait classification. A new gait based gender
classification method based on kinect sensor was proposed by Ahmed and Sabir (2017) and
Blumrosen etal. (2016).
Bayat etal. (2014) proposed the mobile based accelrometer for data capturing of different
human activity (Bayat etal. 2014). Lockhart etal. (2011, 2012) presented smart phone based
sensor mining architecture for capturing different human activity walking, sitting, jogging,
stand up, stand down and running for total six activities. Later they have used neural network
and deep learning based CNN model for classification (Weiss and Lockhart 2012b; Lockhart
et al. 2011). The other previously used gait features are listed in Table4 and classification
approach are listed in Table5.
3 Preliminaries
In this section, we are providing the full 3- link manipulators Inverse kinematics (IK) solution:
In this solution, manipulator is of R–R–R type is considered, whose rotations are restricted
within the following ranges:
• Here, Eq.1 represents the joint angle range of different joints and Fig.1 shows a simple
structure of a planar 3-link robotic arm.
• The manipulator, in our case consists of three movable links that can move limited to a
plane only. The Human leg is considered as 3-link manipulator.
• The links of above manipulator are linked by rotational joints whose axis of rotation are
perpendicular to the planes of the joints.
• We will use some extra constraints in our model so that we can restrict our sample space in
first quadrant only. We have done so, because in most of the practical cases robot will have
to work only in the first quadrant.
3.1 Forward kinematics
The forward kinematic equations are given in Eqs.3 and 4. Where
xe
and
ye
are the cartesian
coordinate and
𝜃1
,
𝜃2
&
𝜃3
are joints coordinate.
--
(1)
𝜃1∈[
0, 𝜋
]
,𝜃
2∈ [−
𝜋,0
]
,𝜃
3∈ [−
𝜋
∕
2, 𝜋
∕
2
]
(2)
⎧
⎪
⎨
⎪
⎩
l1cos(𝜃1)+l2cos(𝜃1+𝜃2)>0
l1sin(𝜃1)+l2sin(𝜃1+𝜃2)>0
0<𝜃
1+𝜃2+𝜃3<𝜋∕2
.
(3)
xe=l1cos(𝜃1)+l2cos(𝜃1+𝜃2)+l3cos(𝜃1+𝜃2+𝜃3)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
Table 4 Different gait feature for data acquisition techniques
Different features category Name of features References
Time domain feature Mean absolute value root mean square,
wave from length, zero crossing, variance
Lau etal. (2009), Prakash etal.
(2018), Sabir etal. (2019)
Gait spatio-temporal features Gait cycle time, stride length, left & right s
teps length
Yang etal. (2017)
Kinematics features Hip, knee, ankle, foot, shank & thingh joints angle Semwal etal. (2013a)
Combined features (gait temporal and time
domain)
Average, SD, average absolute difference, average
resultant acceleration time between peaks, binned distribution
Semwal etal. (2015b), Dua etal.
(2021)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
Here, Eqs. 3 and 4 are not sufficient to locate the end effector completely within the
plane. Its orientation is also required. Therefore, position can be given by Eqs.3 and 4 and
the orientation is represented by Eq.5.
Hence, it is completely clear from aforementioned equations that they can be solved eas-
ily. Therefore, extra efforts is not required for forward kinematic solutions. Now, we will
try to get inverse kinematic solutions using the Eqs.3–5.
3.2 Analytical approach
We can find the inverse kinematics solution by simply solving the forward kinematics
equations for the joint angles. In this section, firstly, simple 2-link manipulator inverse kin-
ematics solution is derived and presented in Sect.3.2.1. Afterwards, using the 2-link solu-
tion, 3-link manipulators inverse kinematic solution is derived and presented in Sect.3.2.2.
3.2.1 Inverse kinematics solution for2‑link manipulator
(4)
ye=l1sin(𝜃1)+l2sin(𝜃1+𝜃2)+l3sin(𝜃1+𝜃2+𝜃3).
(5)
𝜃=𝜃1+𝜃2+𝜃3.
(6)
W=𝜓(𝛩)
Table 5 List of state of art machine learning algorithm for Gait identification
Classification technique References
Adaboost, decision tree, random forest Semwal etal. (2017b), Ahmed etal. (2018a, b), Chiu etal.
(2019)
SVM, ELM, MLP Semwal etal. (2017b), Huan etal. (2018b), Raj etal. (2018b),
Gupta etal. (2014, 2020)
Dynamic Bayesian, RBF-Kernal Milovanovic (2008), Akdoğan and Yilmaz (2014), Huan etal.
(2018a, 2019)
Self-organizing map Zhang etal. (2019), Semwal etal. (2015a)
CNN, RBM, DNN Semwal etal. (2017b), Chiu etal. (2019), Raj etal. (2019),
Mahfouf etal. (2018)
Fig. 1 Solution to 3-link problem
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
From Fig.1, we can derive the following equations:
But also,
𝜃2=𝜋−𝛼
Using tangent rule:
And also,
Hence,
Note Now, we can easily expand this concept for higher order complex problems.
3.2.2 Inverse kinematics solution for3‑link manipulator
Let’s consider solving 3-link manipulator problem using the results obtained from the
2-link manipulator.
In Fig. 1, positions
(wx,wy)
are estimated from simple trigonometric equations as
follows:
(7)
[
xe
y
e]
=
[
l1cos(𝜃1)+l2cos(𝜃1+𝜃2)
l
1
sin(𝜃
1
)+l
2
sin(𝜃
1
+𝜃
2
).
]
(8)
d2
=x
2
e
+y
2
e
=l
2
1
+l
2
2
−2l
1
l
2
cos(𝜃
2)
(9)
cos
(𝛼)= l
2
1+l
2
2−x
2
e−y
2
e
2l
1
l
2
(10)
cos
(𝜃2)=cos(𝜋−𝛼)=−cos(𝛼)= l
2
1+l
2
2−x
2
−y
2
2l
1
l
2
(11)
𝜃
2=





𝜋−cos−1

l2
1+l2
2−x2−y2
2l1l2

cos−1

l2
1+l2
2−x2−y2
2l1l2

.





(12)
tan
(𝜃1+𝛽)=
y
x
(13)
𝜃
1=tan−1
(y
x)
−
𝛽
(14)
𝛽
=tan−1
(
l2sin
(
𝜃2
)
l1+l2cos(𝜃2))
(15)
𝜃
1=tan−1

y
x

−tan−1

l2sin(𝜃2)
l
1
+l
2
cos(𝜃
2
)
.
(16)
wx=xe−l3cos(𝜃3)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
Now, the problem reduces like that of a simple 2-link manipulator. And we can do
this for any number of links.
The formulation of 3-link solutions by considering Fig.1 as follows:
From Eqs.3–5
And,
Thus, we can find the joint angles from the aforementioned equations.
4 Proposed methodology
The entire research work involves the two majors steps. Figure2 represents the detailed
flow chart of our working methodology. The first step involves following steps: data
acquisition through IMU sensor, pre-processing of data, feature extraction for human
activities recognition using different learning algorithm. The next working hypothesis
is joints angle calculation from accelerometer’s raw data (Raj etal. 2018b). We have
solved the inverse kinematics (Raj etal. 2019) to extract the joints angles value using
Algorithm1. The detailed solution of 2-link and 3-link manipulator’s inverse kinemat-
ics has been provided in Sect.3.
(17)
wy=ye−l3sin(𝜃3).
(18)
D2
=w
2
x
+w
2
y
=l
2
1+l
2
2−2l1l2cos(𝜃2
)
(19)
cos
(𝜃2)=(w
2
x
+w
2
y
−l
2
1
−l
2
2
)∕2l1l
2
(20)
sin
(𝜃2)=±
√
1−(cos(𝜃2))2
(21)
𝜃2=atan2(sin(𝜃2),cos(𝜃2)).
(22)
wx=(l1+l2cos(𝜃2))cos(𝜃1)−l2sin(𝜃1)sin(𝜃2)
(23)
wy=(l1+l2cos(𝜃2))sin(𝜃1)−l2cos(𝜃1)sin(𝜃2)
(24)
sin
(𝜃1)=(l1+l2cos(𝜃2))w
y
−l2sin(𝜃2)w
x
)∕
△
(25)
cos
(𝜃
1
)=(l
1
+l
2
cos(𝜃
2
))w
x
−l
2
sin(𝜃
2
)w
y
)∕
△
(26)
𝜃1=atan2(sin(𝜃1),cos(𝜃1))
(27)
𝜃3=𝜃e
−𝜃1−𝜃2.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
4.1 Data representation
4.1.1 Data acquisition andcalibration
Figure 3 represents the data collection technique of different subjects using IMU sensor
and mobile based accelerometer. Total 25 subjects was considered, out of which 15 were
male and 10 were female. We have also collected data for pregnant women. The sensor
was placed on 6 joints and CoM position of each subjects. It was asked to each subjects to
perform the 6 different walking styles. Figure4 shows the number of samples collected for
different walking styles. The data was collected on frequency 100Hz.
Fig. 2 Flow chart of detailed working
Fig. 3 Different subject(s) data collection using IMU sensor
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
4.1.2 Data set description
The data set is collected for 25 subjects using wearable accelerometer (6-Degree IMU sen-
sors) for six different walking activities named: brisk walk, normal walk, very slow walk,
medium walk, jogging and fast walk for clinical examination purpose. We have considered
the subject from different age group, gender and also collected data for pregnant woman
data. Each data file is having following column: time(s) stamp, activities name, acceler-
ation, angular velocity and joint angles. The details of all the columns are presented in
Table6.
4.2 Feature engineering
4.2.1 Feature extraction andprocessing
We have used 6 features for our data set. All the data was labeled into 6 different walking
style as given in Fig.4. We have used subject id, time stamp, activities name, ax, ay, az.
The data is processed by providing label to each samples.
4.2.2 Proposed algorithm tocalculate thejoint angles
Algorithm 1: describes the way how we have calculated joint angles value?. In Algo-
rithm1, L1, L2 and L3 represents thigh length, shank length and foot length respectively,
and current position coordinates of foot when placed on ground is represented by X and Y.
Fig. 4 Distribution of different
walking activities samples
Table 6 Description of data
Notation Description
Time (s) It shows the time stamp
ax (g), ay (g), az (g) Represents the acceleration about x, y and z axis
wx (deg/s), wy (deg/s), wz (deg/s) Show the angular velocity about x, y, & z axis
Angle X (deg), Angle Y (deg), Angle Z (deg) Various joint angle value in degree
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
4.2.3 Model fitting andclassification
This section present the parameters used by ELM, ANN & deep learning architecture.
The proposed architecture of neural network includes three hidden layer with 100 neu-
rons, 6 inputs neurons (represents Hip, Knee, Ankle, Thigh, Shank & Foot) and 6 out-
put neurons corresponds 6 walking activities.
Illustration of Table7: It represents the learning parameter used by ANN model.
The optimizer used by ANN model was ‘adam’ and cross entropy considered as loss
function for 50 epochs. The hidden layer activation function was ‘relu’ and output
layer activation function was ‘softmax’.
The next classifier was ELM and all the set of parameters used by ELM are reported
in Table8. Table9 shows the parameters used by CNN model.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
5 Results anddiscussions
To evaluate the performance of proposed methodology, experiments were conducted using
GPU machine with a Intel 2.50-GHz, i7 CPU and 32.0-GB RAM. The code was executed
on Google colab using GPU. The grid search was preformed to find the best set of param-
eters for each of the classifiers. Next, tenfold cross validation is performed to reduce vali-
dation error. Finally, all the classifiers i.e., ANN, ELM and DNN were implemented using
TensorFlow-Keras Python libraries.
5.1 Experiment outcome condition anddiscussion
First step for the experimentation is the data collection. In this process, IMU sensor was
placed at six different positions, namely, shank, thigh, hip, knee, ankle and foot for captur-
ing the six different activity.
The IMU sensor has given accelerometer value which is converted into joint
angle(degree). This objective is achieved using the inverse kinematics algorithm solution
which considers Center of Mass(CoM) at reference points. The detail description of inverse
kinematic solution is provided in Sect.3 (preliminaries).
Table 7 Set of parameter for
ANN Parameter Value
No. of hidden layer 3
Hidden layer neurons 100
Hidden layer activation function relu
Output layer activation function softmax
Solver ‘Lbfgs’
Alpha 0.0001
Momentum 0.9
Epoch 50
Table 8 Set of parameter for
ELM Parameter Value
Hidden neurons 100
Activation function ‘multiquadric’
Rbf-width 0.4
Table 9 Set of parameter for
DNN Parameter Value
No. of convolution layers 4
Activation function of hidden layers: ‘relu’
No. of output layer neuron 6
Activation function of output layer Soft max
Sub sampling Max pooling
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
Illustration of Fig.5: It shows the 6 joint angle trajectories for five different walks. In
Fig.5, hip, shank, knee, ankle, thigh, & foot joint trajectories are represented by (a), (b),
(c), (d) and (e) respectively. In this figure, jogging is excluded due to it required more
prepossessing.
5.2 Model validation andtraining accuracy
The classification of different walking activities were preformed using three different clas-
sifiers named ANN, ELM and Deep learning model CNN. The ANN has reported test-
ing accuracy 87.84% whereas ELM and DNN based model has reported 88% & 92%
respectively.
Illustration of Figs.6 and 7: In Fig.6, the obtained overall classification accuracy per-
centage using ANN, ELM and CNN is presented. In Fig.7, different testing accuracy per-
centage of each class output is provided.
5.3 Performance analysis anddiscussion
In this section, we have provided the obtained results of various performance matrix such
as accuracy, precision, recall, f-score and support value to classify the individual walking
activity.
Illustration of Tables10 and 11: it was observed from Table10, obtained accuracy of
CNN is better than remaining tested classifiers. In precision, ANN obtains the highest
value. In recall, ELM outperforms over others. Finally, in F1-score, ANN shows the supe-
rior performance. Table11 shows the different performance of different classifier as indi-
vidual and combined for classifying the different walking activities.
5.4 Data‑set validation
In the comparative analysis, the proposed data-set is compared with the existing data-set
WISDM in terms of classification accuracy. The classifier models, namely, ANN, ELM
and DNN are applied on both the data-set.
Illustration of Table12: From the obtained results, it was observed that the obtained
classification accuracy of WISDM data-set is 87% in ANN, 85% in ELM & 90% in DNN.
In the proposed data-set, 87%, 88% and 92% classification accuracy was achieved in ANN,
ELM and DNN. Hence, the proposed data-set shows the variety of data and also obtains the
better results in both ELM and DNN classifiers. It shows the validity of collected data-set
correctly. This data-set can be considered as a benchmark data-set for the future research.
6 Conclusion andfuture research direction
This section consists of conclusion of this research article and future research direction.
6.1 Conclusion
In this paper, walking problem of humanoid robot was addressed using the deep learn-
ing models, namely, ANN, ELM, and DNN. To achieve the objective, firstly, six different
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
Fig. 5 Different joints angle curves. a Hip joint, b shank joint, c knee joint, d ankle joint, e thigh joint, f
foot joint
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
walking activities were explored for clinical examination purpose, namely, (1) brisk walk,
(2) normal walk, (3) very slow walk, (4) medium walk, (5) jogging & (6) fast walk. From
the aforementioned activities, we have collected data for 25 subjects using 6-degree IMU
sensor and mobile-based accelerometers. In the next step, joint trajectories pattern of hip,
Fig. 6 Accuracy percentage of different classifier
Fig. 7 Different activity classification percentage
Table 10 Metrics showing
combined results Model Accuracy Precision Recall F1-Score Support
ANN 0.87 1.00 0.99 0.99 459
ELM 0.88 0.99 1.00 0.99 459
CNN 0.92 0.96 0.97 0.96 459
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
ankle, knee, shank, foot and thigh were compared for five walking activities. Finally, deep
learning models have been applied, namely, ANN, ELM and DNN for the classification of
different walking activities. From the obtained results, it was observed that 87.84%, 88%
and 92% accuracy was achieved using ANN, ELM and DNN respectively. Hence, DNN
is the most suitable deep learning model for the different walking activities classification.
This research work can be utilized for the generation of robot walking trajectories and can
be adopted for various applications such as clinical patient monitoring, surveillance, foren-
sic application. In addition, the collected data set for 25 subjects is made available publicly
for research purpose.
6.2 Future research direction
In the future, this work can be extended for more varieties of human activities and collec-
tion for better features. In addition, the computer vision based technique will be explored
in future.
Acknowledgements The author(s) would like to thank all the participants who have allowed us to capture
the data using a wearable device. Special thanks to Human motion capturing & analysis unit of MANIT
Bhopal and Motion Capturing Sensor laboratory of Institute of Technology Gopeshwar, Uttarakhand
for providing opportunity to collect data and providing the basic computing facility. The data set is also
Table 11 Metrics showing individual results
Model Walking style Slow walk Brisk walk Natural walk Medium walk Jogging Fast walk
ANN Accuracy 0.87 0.92 0.83 0.90 0.85 0.85
Precision 0.96 0.96 0.77 0.67 0.66 0.69
Recall 0.96 0.96 0.76 0.68 0.67 0.70
F1 0.96 0.96 0.77 0.67 0.66 0.69
Support 117 120 120 120 120 120
ELM Accuracy 0.88 0.92 0.81 0.91 0.86 0.85
Precision 0.89 0.95 0.78 0.69 0.71 0.75
Recall 0.88 0.94 0.79 0.68 0.72 0.76
F1 0.98 0.94 0.78 0.68 0.71 0.75
Support 117 120 120 120 120 120
CNN Accuracy 0.92 0.94 0.84 0.93 0.95 0.93
Precision 0.85 0.86 0.71 0.73 0.72 0.72
Recall 0.83 0.87 0.71 0.73 0.74 0.73
F1 0.83 0.86 0.71 0.73 0.72 0.72
Support 117 120 120 120 120 120
Table 12 Comparison with
WISDM & our data set Model WISDM (Kwapisz etal.
2011)
Collected dataset
ANN 0.87 0.87
ELM 0.85 0.88
DNN 0.90 0.92
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
available publicly for research purposes. One can download from here:Data- set Link. We also would like to
express our thanks to SERB, DST govt. of India for funding project under the schema of Early career award
(ECR), DST No: ECR/2018/000203 dated on 04/06/2019. We would also like to thank Uttarakhand Techni-
cal University, Dehradun for providing CRS Scheme project to our one research collaborator Mr. Vishwa-
nath Bijalwan, Assistant Professor, Institute of Technology, Gopeshwar.
Funding The work is supported by SERB, DST, Government of India to Dr. Vijay Bhaskar Semwal under
Early Career Award(ECR) with DST No: ECR/2018/000203 dated 04-June-2019.
Declarations
Conflict of interest The authors declare that they have no conflict of interest.
References
Adil S etal (2016) Extreme learning machine based sEMG for drop-foot after stroke detection. In: Sixth
international conference on information science and technology (ICIST) 2016, pp 18–22
Ahmed MH, Sabir AT (2017) Human gender classification based on gait features using kinect sensor. In:
2017 3rd IEEE international conference on cybernetics (CYBCON), pp 1–5
Ahmed MH, Sabir AT, Maghdid HS (2018a) Kinect-based human gait recognition using triangular gird
feature. In: 2018 1st international conference on advanced research in engineering sciences (ARES).
IEEE
Ahmed MH, Sabir AT, Maghdid HS (2018b) KHGI: kinect-based human Gait identification using statistical
Moments. In: 2018 1st international conference on advanced research in engineering sciences (ARES).
IEEE
Akdoğan KE, Yilmaz Y (2014) Gait measurement using radial basis function networks and comparison of
its performance with analytical methods. In: 2014 18th National Biomedical Engineering Meeting.
IEEE, pp 1–4
Anusha R, Jaidhar CD (2019) An approach to speed invariant gait analysis for human recognition using
mutual information. In: TENCON 2019—2019 IEEE Region 10 conference (TENCON), pp 1616-1621
Bayat A, Pomplun M, Tran DA (2014) A study on human activity recognition using accelerometer data
from smartphones. Proc Comput Sci 34:450–457
Blumrosen G, Miron Y, Intrator N, Plotnik M (2016) A real-time kinect signature-based patient home moni-
toring system. Sensors 16(11):1965
Bovi G etal (2011) A multiple-task gait analysis approach: kinematic, kinetic and EMG reference data for
healthy young and adult subjects. Gait Posture 33(1):6–13
Chaki J etal (2019) Pattern mining approaches used in sensor-based biometric recognition: a review. IEEE
Sens J 19(10):3569–3580
Chan B etal (2018) Marker-less stereo-vision human motion tracking using hybrid filter in unconstrained
environment. In: TENCON 2018—2018 IEEE Region 10 Conference. IEEE
Chellappa R, Roy-Chowdhury AK, Kale A (2007) Human identification using gait and face. In: 2007 IEEE
conference on computer vision and pattern recognition, Minneapolis, MN, pp 1–2
Chen Z, Li G, Fioranelli F, Griffiths H (2018) Personnel recognition and gait classification based on multi-
static micro-Doppler signatures using deep convolutional neural networks. IEEE Geosci Remote Sens
Lett 15(5):669–673
Chiu H, Lin C, Li TS (2019) Gait recognition using histogram of oriented gradient and self-organizing
feature map classification in variable walking speed. In: 2019 international conference on fuzzy theory
and its applications (iFUZZY), New Taipei City, Taiwan, pp 283–286
Connor P, Ross A (2018) Biometric recognition by gait: a survey of modalities and features. Comput Vis
Image Understand 167:1–27
Dua N, Singh SN, Semwal VB (2021) Multi-input CNN-GRU based human activity recognition using
wearable sensors. Computing. https:// doi. org/ 10. 1007/ s00607- 021- 00928-8
Ekinci M (2006) A new approach for human identification using gait recognition. In: International confer-
ence on computational science and its applications. Springer, Berlin
Gallow A, Heiderscheit B (2016) Clinical aspects of running gait analysis. In: Endurance sports medicine.
Springer, Cham, pp 201–213
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
Gouwanda D, Gopalai AA, Khoo BH (2016) A low cost alternative to monitor human gait temporal param-
eters—wearable wireless gyroscope. IEEE Sens J 16(24):9029–9035
Guo Y, Wu X, Shen L, Zhang Z, Zhang Y (2019) Method of gait disorders in Parkinson’s disease clas-
sification based on machine learning algorithms. In: 2019 IEEE 8th joint international information
technology and artificial intelligence conference (ITAIC). IEEE, pp 768–772
Gupta JP, Polytool D, Singh N, Semwal VB (2014) Analysis of gait pattern to recognize the human
activities. IJIMAI 2(7):7–16
Gupta A etal (2020) Multiple task human gait analysis and identification: ensemble learning approach.
In: Emotion and information processing. Springer, Berlin
Hsu W-C, Sugiarto T, Lin Y-J, Yang F-C, Lin Z-Y, Sun C-T, Hsu C-L, Chou K-N (2018) Multiple-wear-
able-sensor-based gait classification and analysis in patients with neurological disorders. Sensors
18:3397
Huan TT etal (2018a) Adaptive gait generation for humanoid robot using evolutionary neural model
optimized with modified differential evolution technique. Neurocomputing 320:112–120
Huan TT, Van Kien C, Anh HPH, Nam NT (2018b) Adaptive gait generation for humanoid robot using
evolutionary neural model optimized with modified differential evolution technique. Neurocomput-
ing 320:112–120
Huan TT, Anh HPH, Van Kien C (2019) Optimal nature-walking gait for humanoid robot using Jaya
optimization algorithm. Adv Mech Eng 11(11):1687814019888082
Kusakunniran W, Qiang W, Hongdong L etal (2009) Automatic gait recognition using weighted binary
pattern on video. In: Sixth IEEE international conference on advanced video and signal based sur-
veillance, pp 49–54
Kwapisz JR, Weiss GM, Moore SA (2011) Activity recognition using cell phone accelerometers. ACM
SigKDD Explor Newsl 12(2):74–82
Lau H-Y etal (2009) Support vector machine for classification of walking conditions of persons after
stroke with dropped foot. Hum Mov Sci 28(4):504–514
Li R etal (2019) Constraint-based optimized human skeleton extraction from single-depth camera. Sen-
sors 19(11):2604
Lockhart JW, Weiss GM, Xue JC, Gallagher ST, Grosner AB, Pulickal TT (2011) Design considerations
for the WISDM smart phone-based sensor mining architecture. In: Proceedings of the fifth interna-
tional workshop on knowledge discovery from sensor data (at KDD-11), San Diego, CA
Lockhart JW, Pulickal T, Weiss GM (2012) Applications of mobile activity recognition. In: Proceedings
of the 2012 ACM conference on ubiquitous computing
Mahfouf Z, Merouani HF, Bouchrika I, Harrati N (2018) Investigating the use of motion-based features
from optical flow for gait recognition. Neurocomputing 283:140–149
Mekruksavanich S, Jitpattanakul A (2019) Classification of gait pattern with wearable sensing data. In:
2019 Joint international conference on digital arts, media and technology with ECTI Northern sec-
tion conference on electrical, electronics, computer and telecommunications engineering (ECTI
DAMT-NCON). IEEE, pp 137–141
Milovanovic I (2008) Radial basis function (RBF) networks for improved gait analysis. In: 2008 9th
symposium on neural network applications in electrical engineering. IEEE, pp 129–132
Nandi GC etal (2009) Development of adaptive modular active leg (AMAL) using bipedal robotics tech-
nology. Robot Auton Syst 57(6–7):603–616
Nandi GC etal (2016) Modeling bipedal locomotion trajectories using hybrid automata. In: Region 10
Conference (TENCON), 2016 IEEE. IEEE
Papavasileiou I etal (2017) Classification of neurological gait disorders using multi-task feature learn-
ing. In: 2017 IEEE/ACM international conference on connected health: applications, systems and
engineering technologies (CHASE). IEEE
Patil P etal (2019) Clinical human gait classification: extreme learning machine approach. In: 2019 1st
international conference on advances in science, engineering and robotics technology (ICASERT).
IEEE
Prakash C, Kumar R, Mittal N (2018) Recent developments in human gait research: parameters, approaches,
applications, machine learning techniques, datasets and challenges. Artif Intell Rev 49(1):1–40
Raj M, Semwal VB, Nandi GC (2018a) Bidirectional association of joint angle trajectories for humanoid
locomotion: the restricted Boltzmann machine approach. Neural Comput Appl 30(6):1747–1755
Raj M, Semwal VB, Nandi GC (2018b) Hybrid model for passive locomotion control of a biped human-
oid: the artificial neural network approach. IJIMAI 5(1):40–46
Raj M, Semwal VB, Nandi GC (2019) Multiobjective optimized bipedal locomotion. Int J Mach Learn
Cybern 10(8):1997–2013
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

V.B.Semwal et al.
1 3
Sabir AT etal (2019) Gait-based gender classification using smartphone accelerometer sensor. In: 2019 5th
international conference on frontiers of signal processing (ICFSP). IEEE
Semwal VB (2017) Data driven computational model for bipedal walking and push recovery. arXiv preprint
arXiv: 1710. 06548
Semwal VB, Nandi GC (2015) Toward developing a computational model for bipedal push recovery—a
brief. IEEE Sens J 15(4):2021–2022
Semwal VB, Nandi GC (2016) Generation of joint trajectories using hybrid automate-based model: a rock-
ing block-based approach. IEEE Sens J 16(14):5805–5816
Semwal VB etal (2013a) Biped model based on human gait pattern parameters for sagittal plane movement.
In: International conference on control, automation, robotics and embedded systems (CARE). IEEE, p
2013
Semwal VB, Bhushan A, Nandi GC (2013b) Study of humanoid Push recovery based on experiments. In:
2013 international conference on control, automation, robotics and embedded systems (CARE). IEEE
Semwal VB etal (2015a) Biologically-inspired push recovery capable bipedal locomotion modeling through
hybrid automata. Robot Auton Syst 70:181–190
Semwal VB, Raj M, Nandi GC (2015b) Biometric gait identification based on a multilayer perceptron.
Robot Auton Syst 65:65–75
Semwal VB, Chakraborty P, Nandi GC (2015c) Less computationally intensive fuzzy logic (type-1)-based
controller for humanoid push recovery. Robot Auton Syst 63:122–135
Semwal VB etal (2017a) An optimized feature selection technique based on incremental feature analysis for
bio-metric gait data classification. Multimed Tools Appl 76(22):24457–24475
Semwal VB, Mondal K, Nandi GC (2017b) Robust and accurate feature selection for humanoid push recov-
ery and classification: deep learning approach. Neural Comput Appl 28(3):565–574
Semwal VB etal (2018) Design of vector field for different subphases of gait and regeneration of gait pat-
tern. IEEE Trans Autom Sci Eng 15(1):104–110
Semwal VB, Gaud N, Nandi GC (2019) Human gait state prediction using cellular automata and classifica-
tion using ELM. In: Machine intelligence and signal analysis. Springer, Singapore, pp 135–145
Sivakumar S etal (2016) ANN for gait estimations: a review on current trends and future applications. In:
2016 IEEE EMBS conference on biomedical engineering and sciences (IECBES). IEEE
Sivakumar S et al (2018) Estimation of joint angle from ground reaction force in human gait. In: 2018
IEEE-EMBS conference on biomedical engineering and sciences (IECBES). IEEE
Weiss GM, Lockhart J (2012a) The impact of personalization on smartphone-based activity recognition. In:
Workshops at the twenty-sixth AAAI conference on artificial intelligence
Weiss GM, Lockhart JW (2012b) The impact of personalization on smartphone-based activity recognition.
In: Proceedings of the AAAI-12 workshop on activity context representation: techniques and lan-
guages, Toronto
Yang X, She H, Lu H, Fukuda T, Shen Y (2017) State of the art: bipedal robots for lower limb rehabilitation
Zhang Z, Hu M, Wang Y (2011) A survey of advances in biometric gait recognition. In: Chinese conference
on biometric recognition. Springer, Berlin
Zhang J, Wu F, Hu W, Zhang Q, Xu W, Cheng J (2019) WiEnhance: towards data augmentation in human
activity recognition using WiFi signal. In: 2019 15th international conference on mobile ad-hoc and
sensor networks (MSN), Shenzhen, China, pp 309–314
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.
Authors and Aliations
VijayBhaskarSemwal1 · NehaGaud2· PraveenLalwani3· VishwanathBijalwan4·
AbhayKumarAlok5
Neha Gaud
gaud28neha@gmail.com
Praveen Lalwani
praveen.lalwani@vitbhopal.ac.in
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Pattern identification ofdifferent human joints fordifferent…
1 3
Vishwanath Bijalwan
vishwanath.bijalwan@itgopeshwar.ac.in
Abhay Kumar Alok
abhayalok@gmail.com
1 Maulana Azad Nation Institute ofTechnology Bhopal, Bhopal, M.P., India
2 SCSIT DAVV Indore, Indore, India
3 VIT Bhopal, Bhopal, M.P., India
4 Institute ofTechnology Gopeshwar, Gopeshwar, U.K., India
5 Indian Institute ofTechnology Patna, Patna, India
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center
GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers
and authorised users (“Users”), for small-scale personal, non-commercial use provided that all
copyright, trade and service marks and other proprietary notices are maintained. By accessing,
sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of
use (“Terms”). For these purposes, Springer Nature considers academic use (by researchers and
students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and
conditions, a relevant site licence or a personal subscription. These Terms will prevail over any
conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription (to
the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of
the Creative Commons license used will apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may
also use these personal data internally within ResearchGate and Springer Nature and as agreed share
it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not otherwise
disclose your personal data outside the ResearchGate or the Springer Nature group of companies
unless we have your permission as detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial
use, it is important to note that Users may not:
use such content for the purpose of providing other users with access on a regular or large scale
basis or as a means to circumvent access control;
use such content where to do so would be considered a criminal or statutory offence in any
jurisdiction, or gives rise to civil liability, or is otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association
unless explicitly agreed to by Springer Nature in writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a
systematic database of Springer Nature journal content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a
product or service that creates revenue, royalties, rent or income from our content or its inclusion as
part of a paid for service or for other commercial gain. Springer Nature journal content cannot be
used for inter-library loans and librarians may not upload Springer Nature journal content on a large
scale into their, or any other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not
obligated to publish any information or content on this website and may remove it or features or
functionality at our sole discretion, at any time with or without notice. Springer Nature may revoke
this licence to you at any time and remove access to any copies of the Springer Nature journal content
which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or
guarantees to Users, either express or implied with respect to the Springer nature journal content and
all parties disclaim and waive any implied warranties or warranties imposed by law, including
merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published
by Springer Nature that may be licensed from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a
regular basis or in any other manner not expressly permitted by these Terms, please contact Springer
Nature at
onlineservice@springernature.com