Real-time gait biometrics for surveillance applications: A review

Full text

Real-time Gait Biometrics for Surveillance Applications: A Review
Anubha Parashara,<,Apoorva Parasharb,Andrea F. Abatec,Rajveer Singh Shekhawatdand
Imad Ridae
aSchool of Computing & Information Technology, Manipal University Jaipur, Jaipur, Rajasthan, India
bConsultant, Emerging technology, Mahindra Integrated Business Solutions, Mumbai, India,
cUniversity of Salerno, Via Giovanni Paolo II, 132, Fisciano, Salerno 84084, Italy
dDean, Manipal University Jaipur, Rajasthan, India
eBMBI Laboratory, University of Technology of Compiègne, 60200, Compiègne, France
ARTICLE INFO
Keywords:
Gait Recognition
Biometrics
Deep Learning
Vision-based
Real-time Surveillance
ABSTRACT
Deep learning (DL) pipelines have evolved for over a decade now and are efficient at solving
many challenging problems of image and signal processing applications. Designing deep learn-
ing pipelines for a particular application requires a good understanding of deep learning and
various intermediate layers available. To develop a DL pipeline, one uses available dataset(s)
suitable for an application, and the pipeline is refined by iterating over intermediate layers. A
large amount of time and extensive thinking goes into these selections and validating the per-
formance of each configuration. Thus, it is hard to choose the correct and robust DL pipeline
that performs well on all relevant datasets. This review aims to aid researchers in understand-
ing different gait sensing technologies and provide foundational knowledge of the deep learning
concepts for faster solutions for a given problem. Gait recognition is more recent since it hasn’t
yet been used in a real-world situation. This article provides a comprehensive overview of gait
biometrics suited to real-time surveillance applications. All the important parameters of deep
learning pipelines are explained, along with their selection and implication for a given prob-
lem. Authors have reviewed important research articles recently on deep learning models and
how these perform across different application datasets. The benefits and drawbacks of the ap-
proaches are elucidated to help arrive at the optimized pipeline derived from a fusion of available
pipelines to achieve faster but accurate results for a given problem.
1. Introduction
The rise in popularity of automated identification over the past few years has increased the focus of professionals
working in computer vision and kinematics on gait recognition in real-time settings. This is because automated iden-
tification entries are becoming increasingly common. A person’s gait, which can be recognized from a great distance
and does not need cooperation from the target, is one of the most important biometric qualities humans possess. In
vision-based techniques, it is likely that videos from low-resolution cameras will be carried out. Other biometric iden-
tification methods (in figure 1) have absolutely no possibility of functioning correctly in such stated scenarios [76].
Thanks to this capability of gait, it is now possible to employ it in a real-time environment. Because gait patterns are
challenging to replicate and substantially more difficult to hide than facial characteristics, they are regarded as more
secure forms of biometric identification [77].
Deep learning has emerged as a promising method for recognizing humans through gait [80]. Starting gait recog-
nition with deep learning might be difficult since researchers don’t know which deep learning pipeline to use or what
results to anticipate. There are presently just a few review publications discussing deep learning techniques for gait
identification, including deep pipeline parameters. A few surveys on gait analysis have been conducted[70][71][72]
[73][74][75], but the majority of survey publications concentrated on model-free gait recognition systems, ignoring
model-based strategies. Despite the aforementioned benefits, gait identification real-time performance suffers because
of factors like gait characteristics analyzed from various deep learning architectures and datasets. Researchers have
focused on finding a method to develop a reliable gait recognition system in response to these problems [78][79]. This
survey article aims to analyze in-depth the most recent developments in gait recognition research.
<Corresponding authors
(A. Parashar); (A.F. Abate)
1 / 18
REVISED Manuscript (text Unmarked) Click here to view linked References

Figure 1: Various types of biometric approaches that are used for recognition
1.1. Contribution
The following are the important contributions of the paper.
1) The paper presented essential deep learning approaches in gait identification with a comprehensive emphasis on
the architecture of gait recognition.
2) Outline the methodologies that are used the majority of the time throughout the various deep learning pipelines
that have been reported, along with an explanation and assessment of these approaches.
3) The performance of gait recognition in a real-world setting is thoroughly investigated in this work.
4) The paper focuses on the future prospects that should be taken into account for the real-time use of gait recog-
nition.
1.2. Organization
The paper is organized as follows. The first section provides an outline of the topics, including motivation, contri-
bution, and organization. In Section 2, an examination of numerous factors reported in studies resulted in the develop-
ment of a deep learning pipeline for gait in terms of – data collection, data types, dataset, preprocessing, deep learning
techniques, feature extraction, feature reduction, regularisation, activation function, hyper-parameters, optimizer, loss
function, classification, and system configuration. Section 3 discusses and highlights the benefits and shortcomings of
pipeline components along with gait sensing technology. The accuracy and dataset utilized in research publications

compare major features of deep learning pipelines. Section 4 is the conclusion. We list the acronyms used in this
article in the abbreviations section.
An overview of deep learning techniques for covariate conditions in gait recognition is presented in this work.
A thorough examination of the recent literature published in reputable journals (Scopus or Web of Science), confer-
ences, and Google Scholar was carried out. "Gait recognition", "Sensor-based", "gait identification", "model-based
gait recognition", "pose-based", "gait-datasets", "gait surveillance", "biometrics", and other similar terms were used to
search for papers. The chosen search yielded roughly 1200 papers. A total of 200 unique publications related to gait
identification were chosen after analyzing the title, abstract, and keywords. After reading through the publications and
considering their relevance to this survey, 69 references were finalized, which focused on gait recognition. Figure 2
depicts a systematic review methodology adopted.
Figure 2: Systematic review methodology adopted for searching papers and categorizing them
2. Various Deep Learning Parameters to Recognizing Gait
Gait recognition is a challenging task in computer vision and deep learning. Here are some of the key parameters
that are commonly used in deep learning models for recognizing gait:
Convolutional Neural Network (CNN): CNN is a popular deep learning architecture used for gait recognition. It
consists of several convolutional layers that can extract meaningful features from the input image.
Recurrent Neural Network (RNN): RNN is another deep learning architecture that is used for gait recognition. It
is especially useful for handling sequential data, such as video frames.
Transfer Learning: Transfer learning is a technique where a pre-trained model is used as a starting point for a new
model. This technique can be used for gait recognition by fine-tuning a pre-trained model on a gait recognition dataset.
Data Augmentation: Data augmentation is a technique used to artificially increase the size of a dataset by creating
new training examples through various transformations such as rotation, flipping, or cropping.
Learning Rate: The learning rate is a hyperparameter that controls how much the model’s weights are updated
during training. A higher learning rate can help the model converge faster, but it may also cause the model to overshoot

the optimal weights.
Batch Size: The batch size is a hyperparameter that determines how many samples are used in each iteration of
training. A larger batch size can increase the training speed but can also lead to overfitting.
Dropout: Dropout is a regularization technique used to prevent overfitting. It randomly drops out a percentage of
the neurons in the network during training.
Activation Function: The activation function is a non-linear function applied to the output of each neuron in a
neural network. Common activation functions used in gait recognition models include ReLU and Sigmoid.
Loss Function: The loss function is a function that measures how well the model is performing on the training
data. Common loss functions used in gait recognition include binary cross-entropy and mean squared error.
Optimization Algorithm: The optimization algorithm is responsible for updating the model’s weights during train-
ing. Common optimization algorithms used in gait recognition include stochastic gradient descent (SGD), Adam, and
RMSprop.
2.1. Data Collection
The gathering of gait datasets using a variety of methods is the initial step of gait detection (sensing technologies).
The quality of the data used has a significant impact on how well the deep learning model performs and how accurate it
is. Figure 3 illustrates various gait sensing devices like displacement sensors, accelerometers, strain gauges, tiltmeters,
depth cameras, Microsoft Kinetic, lidar, and radar for gait collection. List of data collection techniques used in deep
learning-based gait papers are listed in table 1.
Sensors
Non wearable
Contactless
Wearable Body contact
Body Contact
F-scan
Wearable body
suit
Soft robotic
sensor
Wearable
pants
Piezoelectric
sensor
Wearable
accelerometer
EEG headset
Inertial
Measurement Unit
Wearable
gyroscope
AccelerometerGyroscope
Magnetometer
Force platform pressure
sensor
CCTV camera Depth
camera
Microsoft
kinetic
Dynamic
vision sensor
Event sense
camera
Radar micro-
doppler Radar Millimetre wave
radar sensor
RGB
Camera
Flash lidar
camera
Depth kinetic
sensor
Lidar(VLP16,
HDL64)
Figure 3: Different gait sensing devices through which gait dataset can be prepared
2.2. Data Types
Sensors may be used to collect data as inputs to the gait process. A deep learning pipeline or CNN receives a dataset
as input. Gait recognition systems are used to recognize individuals based on their walking patterns, which serve as
a representation of a person’s gait. They guess the identification of a probing sample from a gallery of samples the

system has collected. [44]. Low-resolution frames are captured from video surveillance using closed-circuit television
(CCTV) [30][31][43]. In a conventional preprocessing step, most of the existing works on RGB images will first
calculate silhouettes based on the images. RGB gait image silhouettes must be utilized, although extracting the gait
may cause color loss in any locations where the gait is identical to the backdrop color [39][40][41][45][46][84].
A Kinect sensor was employed in an indoor setting to capture gait and depth data. Using depth data to model gait
characteristics based on model-based data [24][37].The effectiveness of 2D posture motion analysis systems for 2D
posture gait analysis is well recognized. It does not need as much expertise as 3D systems, and the necessary technology
is readily accessible and affordable. Due to high processing costs, 3D efforts in monitoring and research have been
restricted so far [6].
For a single silhouette frame, a human skeleton was created. After that, the raw gait signature is produced by
recognizing skeletons in each frame of the video feed [47]. GEI is a large descriptor that comprises a vast quantity of
data, which diminishes the quality of distinguishing characteristics. It encompasses both linear and angular kinematics,
which explain the trajectories and angular positions of body segments as they move through time [2]. The gait cycle is
the time period or series of events or activities that occur during locomotion from the moment one foot makes contact
with the ground to the time the same foot makes contact with the ground again, and it includes the center of gravity’s
propulsion in the direction of motion [4][13]. Shapelets are subsequences of time series data that may distinguish
across classes [17]. Shapelets are the most emblematic of their kind. Grayscale R-D maps, which take grayscale input
pictures and map them[19]. List of data types used in deep learning-based gait papers are listed in table 1.
2.3. Preprocessing
After the input is taken in the gait pipeline, it is passed to the next gait pipeline stage for preprocessing and feature
selection. Table 2 shows the various techniques used by the researchers for data preprocessing and feature selection.
2.4. DL Techniques
To distinguish gait, a variety of deep learning approaches are applied. We provide a summary of deep learning
methods and succinct explanations of each strategy used to identify gait. CNN, or Convolutional Neural Networks,
were created to transform image input into parameters. List of deep learning techniques used in gait papers are listed in
table 3. They have proven to be so successful that they are now the solution of choice for any kind of prediction problem
that takes visual data as input [2][4][5][8][10][12][14][17][18][19][20][21][22][25][26][27][29][30][33]
[35][38][39][40][41][42][46]. A generative adversarial network (GAN) is based on "indirect" training through a
discriminator that is also dynamically updated. This means that the generator is trained to fool the discriminator rather
than lessen the distance to a certain image. This enables the model to learn without supervision [37]. The Encoder-
Decoder is a recurrent neural network created to deal with sequence-to-sequence (seq2seq) problems. Sequence-to-
sequence prediction tasks are challenging when the number of items in the input and output sequences differs [7]
[28]. Autoencoders are an unsupervised artificial neural network that learns efficient data coding. An autoencoder
trains the network to eliminate signal "noise," generally for dimensionality reduction, in order to learn a representation
(encoding) for a set of data, [45]. An artificial recurrent neural network with deep learning is designed for long short-
term memory. LSTM has feedback connections as opposed to traditional feedforward neural networks. It can manage
whole data sequences as well as individual data points [3][6][9][25][27][31][32][37][40][45][47]. A residual
neural network is a kind of artificial neural network based on cerebral cortex pyramidal cells. Skip connections, or
shortcuts, are used by residual neural networks to avoid specific layers [15][33][39].
A pose estimation network is a computer vision technique that predicts and tracks the position of a person or object.
This is performed by looking at a person’s or object’s posture and orientation. Skeleton in 2D Pose Estimate makes
advantage of GPU acceleration to deliver low-latency joint real-time object identification and high-accuracy 2D key
point pose estimate [24]. A recurrent neural network is a kind of artificial neural network in which node connections
form a directed graph over time. It is possible to display temporal dynamic behaviour as a result of this [3][23][27]
[31][44][47].The RCNN algorithm creates a set of boxes in the picture and examines them to determine whether
any of them contain any objects. RCNN uses selective search to extract these boxes from a picture [4]. LeNet-5 is a
large-scale image processing feedforward neural network containing artificial neurons that can respond to a fraction
of the cells in its coverage region [19]. High-order sequences may be stored, learned, inferred, and recalled using
Hierarchical Temporal Memory. HTM is different from most machine learning algorithms because it constantly learns
time-based patterns in unlabeled data. HTM is quiet and can hold a lot of things [43]. Radial basis function (RBF)
networks are a form of artificial neural network that is often employed for function approximation challenges [16].

A neural network is a collection of algorithms that attempts to detect underlying connections in a batch of data by
simulating how the human brain works [31]. In most circumstances, an artificial neural network (ANN) is employed
when something that occurred in the past is repeated nearly identically in the same manner [9]. A deep neural network
is a machine learning system that uses numerous layers of nodes to extract high-level functions from input data. It
involves transforming data into a more ethereal and imaginative element [1][13][36].

2.5. Feature Extraction
When dealing with massive amounts of raw data, feature extraction refers to the process of transforming raw data
into numerical characteristics that may be analysed while retaining the information in the original data set. A large
number of variables in these large data sets involve using a large number of computer resources to process them.
Table 3 provides a detailed overview of the various methods used in deep learning frameworks with different feature
extraction and representation techniques explored.

2.6. Hardware and Software Details
A tiny computer devoted to one specific activity is known as a graphics processing unit (GPU). It is distinct from
a CPU that does several tasks concurrently. GPUs have their own processors, motherboards, vRAMs, and a suitable
thermal design for cooling and ventilation. Gait recognition algorithms are performed on CPUs in 65% of the listed
publications, whereas GPUs are utilized in 35% of them. The platforms that authors have utilized to execute their gait
recognition algorithms are listed in Table 4.

3. Comparative Analysis of Most Adopted Deep Learning Approach
The most often used data types are shown in Figure 4 (a). The most often used deep learning methods are shown
in Figure 4 (b). shows the most popular deep learning approaches. CNN is the most adopted deep learning technique
followed by LSTM.

(a) Most adopted data types
(b) Most used deep learning techniques
Figure 4: Most adopted deep learning approaches
3.1. Potential Future Directions
There are open issues and challenges in surveillance application of gait analysis that have not been fully explored
or addressed:
1. The inability of DL models to be interpreted when used in conjunction with applications that take into account
variables
2. The restriction of computing resources and the risk of compromising the confidentiality of data when using a
variety of gait sensing methods to identify gait.
3. Various real-time gait datasets are missing due to which performance of deep learning pipeline cannot be
evaluated properly.

4. Wearable sensing devices have greater accuracy than non wearable devices but wearable devices are not feasible
every place and intervene with the subject during surveillance.
5. Various gait covariates need to be adressed like perturbation, speed, and other environment factors.
6. The majority of the efforts on gait recognition concentrate on observing a single person in the scenario while
they are conducted in controlled situations. However, situations that really occur in real life often demand solutions
that can withstand uncontrolled conditions in which more than one person is present.
7. Lots of gait recognition algorithms performs below average on few datasets (table 8, 9) due to wrong model
selection or less training.
8. Hyper-parameter tuning is an important concept and many research donot consider it.
9. Model-based techniques provide superior outcomes and seem to be well suited for identifying applications in
which the surrounding environment is subject to significant change. However, the quality of the data has a considerable
influence on the performance of the algorithm (noise sensitivity), and the process of fitting frequently needs a large
amount of processing resources to complete. It is possible that in the future, a potential avenue worth pursuing might
be the combining of model-based approaches with model-free techniques to provide a comprehensive approach to
gait-based recognition.
10. The combination of a number of different characteristics is also quite interesting. However, there are still con-
cerns that have not been solved, such as which characteristics should be picked, the best approach for fusion, and how
to adaptively choose the modalities. The procedures that are currently used often result in a reduction in recognition
accuracy because of redundant or noisy data. The multimodal fusion of a variety of distinguishing characteristics has
been an unique notion in the area, and numerous studies are now in the process of researching it further.
Before we wrap up our study, we would want to emphasise that gait-based recognition is a very new topic that is
fraught with difficulties and offers a wealth of possibilities that have not yet been fully explored. In these early years
of study, a number of approaches have been developed and are producing promising findings; nevertheless, they have
only been tested on datasets whose properties differ greatly from one another. The absence of a refefrence dataset
that can test the algorithms with a large range of conceivable situations has, up to this point, rendered it impossible
to provide an accurate comparison between different techniques [85] - [95]. We anticipate significant progress in the
years to come as a result of recent developments including the availability of massive datasets, improved sensors, and
end-to-end training methods such as deep learning.
4. Conclusion
While the gait recognition system is still in its early stages compared to other biometrics like fingerprint, face,
voice, and iris identification, its non-intrusiveness makes it more desirable than other approaches for many applica-
tions. However, running deep learning methods in real-time has made it impossible to utilize this biometric effectively
in any circumstance and is thus its limitation as applications move to the edge for privacy and security in real time
gait applications. In this study, a large number of deep learning pipeline parameters for recognizing gaits were deter-
mined. We evaluated a set of datasets suitable for training deep learning pipelines to handle a range of confounders
in addition to evaluating the methods’ accuracy. Furthermore, we analyzed the most current deep learning models
for each parameter and highlighted those that delivered remarkable results. There is a variety of possible advantages
and disadvantages associated with deep learning techniques, which have been thoroughly addressed. Deep learning
approaches have also been discussed in terms of potential advantages and limitations. We examined the deep learning
approaches to gait identification that has been most widely used. In order to identify the gait recognition gaps that need
to be addressed, comparisons of accuracy attained and datasets utilized are also summarized.

A. Abbreviations
The following abbreviations are used in this manuscript:
Abb. Full form Abb. Full form
DL Deep Learning DCNN Deep Convolutional Neural Network
BCE Binary Cross Entropy DNN Deep Neural Network
CCE Categorical Cross Entropy PReLU Parametric Rectification
SGD Stochastic Gradient Decent ReLU Rectified Linear Unit
MSE Mean Square Error LReLU Leaky Rectified Linear Unit
WD Weight Decay LSTM Long Short-Term Memory
LR Learning Rate FCL Focal Convolution Layer
M Momentum LDA Linear Discriminant Analysis
BP Back Propagation PCA Principal Component Analysis
SNE Stochastic Neighbor Embedding DCT Discrete Cosine Transform
SD Standard Deviation GAN Generative Adversarial Network
MSE Mean Square Error BS Background Subtraction
ED Euclidean Distance GEI Gait Energy Image
LRL Logistic Regression Loss OF Optical Flow
PSO Particle Swarm Optimization SSA Stacked Sparse Autoencoder
RMSP Root Mean Square Propagation SSM Silhouette Stereo Map
GA Genetic Algorithm CFA Canonical Feature Aggregation
CGI Chrono Gait Image PFA Pose Feature Aggregation
IMU Inertial Measurement Unit GMM Gaussian Mixture Model
BN Batch Normalization FCNN Fully Convolutional Neural Network
LRN Local Response Normalization HMM Hidden Markov Model
NAC Normalized Auto Correlation NDNN Non-Linear Deep Neural Network
SNR Spectral Norm Regularization COG Centre of Gravity
NN Nearest Neighbor RNN Recurrent Neural Network
KNN K Nearest Neighbor DRN Deep Recurrent Network
NC Nearest Centroid MB Model Based
SVM Support Vector Machine PB Pose Based
SNN Spiking Neural Networks MF Model Free
CNN Convolution Neural Network SNN Siamese Neural Network
MLP Multilayer Perceptron BCC Binary Cross Entropy
DCRNN Deep Convolutional and
Recurrent Neural Network ABGAN Alpha Blending Generative
Adversarial Networks

B. Data availability
My manuscript has no associated data.

References
[1] J. Hannink, T. Kautz, C. F. Pasluosta, K.-G. Gasmann, J. Klucken, and B. M. Eskofier, “Sensor-Based Gait Parameter Extraction With Deep
Convolutional Neural Networks,” IEEE J. Biomed. Heal. Informatics, vol. 21, no. 1, pp. 85–93, Jan. 2017, doi: 10.1109/JBHI.2016.2636456.
[2] O. Dehzangi, M. Taherisadr, and R. ChangalVala, “IMU-Based Gait Recognition Using Convolutional Neural Networks and Multi-Sensor
Fusion,” Sensors, vol. 17, no. 12, p. 2735, Nov. 2017, doi: 10.3390/s17122735.
[3] X. Zhang, L. Yao, C. Huang, T. Gu, Z. Yang, and Y. Liu, “DeepKey: A Multimodal Biometric Authentication System via Deep Decoding Gaits
and Brainwaves,” ACM Trans. Intell. Syst. Technol., vol. 11, no. 4, pp. 1–24, Jul. 2020, doi: 10.1145/3393619.
[4] G. Giorgi, F. Martinelli, A. Saracino, and M. Sheikhalishahi, “Walking Through the Deep: Gait Analysis for User Authentication Through
Deep Learning,” in IFIP International Conference on ICT Systems Security and Privacy Protection, 2018, pp. [.5ex]–76, doi: 10.1007/978-3-
319-99828-2_5.
[5] R. Delgado-Escano, F.M. Castro, J. R. Cozar, M. J. Mar in-Jimenez, and N. Guil, “An End-to-End Multi-Task and Fusion CNN for Inertial-Based
Gait Recognition,” IEEE Access, vol. 7, pp. 1897–1908, 2019, doi: 10.1109/ACCESS.2018.2886899.
[6] P. Kumar, S. Mukherjee, R. Saini, P. Kaushik, P. P. Roy, and D. P. Dogra, “Multimodal Gait Recognition With Inertial Sensor Data and Video
Using Evolutionary Algorithm,” IEEE Trans. Fuzzy Syst., vol. 27, no. 5, pp. 956–965, May 2019, doi: 10.1109/TFUZZ.2018.2870590.
[7] D. Kim, M. Kim, J. Kwon, Y.-L. Park, and S. Jo, “Semi-Supervised Gait Generation With Two Microfluidic Soft Sensors,” IEEE Robot. Autom.
Lett., vol. 4, no. 3, pp. 2501–2507, Jul. 2019, doi: 10.1109/LRA.2019.2907431.
[8] L. Chen, Y. Zhang, and L. Peng, “METIER: A Deep Multi-Task Learning Based Activity and User Recognition Model Using Wearable Sensors,”
Proc. ACM Interactive, Mobile, Wearable Ubiquitous Technol., vol. 4, no. 1, pp. 1–18, Mar. 2020, doi: 10.1145/3381012.
[9] S. Davarzani et al., “Closing the Wearable Gap—Part VI: Human Gait Recognition Using Deep Learning Methodologies,” Electronics, vol. 9,
no. 5, p. 796, May 2020, doi: 10.3390/electronics9050796.
[10] M. Gadaleta, L. Merelli, and M. Rossi, “Human authentication from ankle motion data using convolutional neural networks,” in 2016 IEEE
Statistical Signal Processing Workshop (SSP), Jun. 2016, pp. 1–5, doi: 10.1109/SSP.2016.7551815.
[11] V. Prabhu and J. Whaley, “Vulnerability of deep learning-based gait biometric recognition to adversarial perturbations,” CVPR 2017 CV-COPS
Work., vol. 1, no. 2, 2017, [Online]. Available: https://blog.unify.id/wp-content/uploads/2017/07/Paper.pdf.
[12] K.-T. Nguyen, T.-L. Vo-Tran, D.-T. Dinh, and M.-T. Tran, “Gait Recognition with Multi-region Size Convolutional Neural Network for Au-
thentication with Wearable Sensors,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and
Lecture Notes in Bioinformatics), vol. 10646 LNCS, 2017, pp. 197–212.
[13] G. Giorgi, F. Martinelli, A. Saracino, and M. Sheikhalishahi, “Try Walking in My Shoes, if You Can: Accurate Gait Recognition Through
Deep Learning,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics), vol. 10489 LNCS, 2017, pp. 384–395.
[14] Y. Zhao and S. Zhou, “Wearable Device-Based Gait Recognition Using Angle Embedded Gait Dynamic Images and a Convolutional Neural
Network,” Sensors, vol. 17, no. 3, p. 478, Feb. 2017, doi: 10.3390/s17030478.
[15] S. Abdulatif, F. Aziz, K. Armanious, B. Kleiner, B. Yang, and U. Schneider, “A Study of Human Body Characteristics Effect
on Micro-Doppler-Based Person Identification using Deep Learning,” Arxiv, no. November, pp. 1–6, Apr. 2018, [Online]. Available:
https://ieeexplore.ieee.org/document/8835652/.
[16] W. Zeng, J. Chen, C. Yuan, F. Liu, Q. Wang, and Y. Wang, “Accelerometer-based gait recognition via deterministic learning,” in 2018 Chinese
Control And Decision Conference (CCDC), Jun. 2018, no. September, pp. 6280–6285, doi: 10.1109/CCDC.2018.8408232.
[17] W. Yuan and L. Zhang, “Gait Classification and Identity Authentication Using CNN,” in 18th Asia Simulation Conference, AsiaSim 2018,
Kyoto, Japan, 2018, vol. 946, pp. 119–128, doi: 10.1007/978-981-13-2853-4_10.
[18] M. Gadaleta and M. Rossi, “IDNet: Smartphone-based gait recognition with convolutional neural networks,” Pattern Recognit., vol. 74, pp.
25–37, Feb. 2018, doi: 10.1016/j.patcog.2017.09.005.
[19] S. Abdulatif, Q. Wei, F. Aziz, B. Kleiner, and U. Schneider, “Micro-doppler based human-robot classification using ensemble and deep learning
approaches,” in 2018 IEEE Radar Conference (RadarConf18), Apr. 2018, pp. 1043–1048, doi: 10.1109/RADAR.2018.8378705.
[20] J. Gao, P. Gu, Q. Ren, J. Zhang, and X. Song, “Abnormal Gait Recognition Algorithm Based on LSTM-CNN Fusion Network,” IEEE Access,
vol. 7, pp. 163180–163190, 2019, doi: 10.1109/ACCESS.2019.2950254.
[21] D. Jung et al., “Deep Neural Network-Based Gait Classification Using Wearable Inertial Sensor Data,” in 2019 41st Annual International Con-
ference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jul. 2019, pp. 3624–3628, doi: 10.1109/EMBC.2019.8857872.ing
to Identity and Covariate Features,” IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020.
[22] S. Z. Gurbuz and M. G. Amin, “Radar-Based Human-Motion Recognition With Deep Learning: Promising Applications for Indoor Monitor-
ing,” IEEE Signal Process. Mag., vol. 36, no. 4, pp. 16–28, Jul. 2019, doi: 10.1109/MSP.2018.2890128.
[23] Fernandez-Lopez, Liu-Jimenez, Kiyokawa, and Wu, “Recurrent Neural Network for Inertial Gait User Recognition in Smartphones,” Sensors,
vol. 19, no. 18, p. 4054, Sep. 2019, doi: 10.3390/s19184054.
[24] N. Sadeghzadehyazdi, T. Batabyal, N. K. Dhar, B. O. Familoni, K. M. Iftekharuddin, and S. T. Acton, “GlidarCo: gait recognition by 3D skele-
ton estimation and biometric feature correction of flash lidar data,” pp. 1–11, May 2019, [Online]. Available: http://arxiv.org/abs/1905.07058.
[25] R. Delgado-Escaño, F. M. Castro, J. R. Cózar, M. J. Marín-Jiménez, N. Guil, and E. Casilari, “A cross-dataset deep learning-based classi-
fier for people fall detection and identification,” Comput. Methods Programs Biomed., vol. 184, no. December, p. 105265, Feb. 2020, doi:
10.1016/j.cmpb.2019.105265.
[26] L. Tran and D. Choi, “Data Augmentation for Inertial Sensor-Based Gait Deep Neural Network,” IEEE Access, vol. 8, pp. 12364–12378,
2020, doi: 10.1109/ACCESS.2020.2966142.
[27] Q. Zou, Y. Wang, Q. Wang, Y. Zhao, and Q. Li, “Deep Learning-Based Gait Recognition Using Smartphones in the Wild,” IEEE Trans. Inf.
Forensics Secur., vol. 15, pp. 1–1, 2020, doi: 10.1109/TIFS.2020.2985628.
[28] M.-C. Lee, Y. Huang, J. J.-C. Ying, C. Chen, and V. S. Tseng, “DeepIdentifier: A Deep Learning-Based Lightweight Approach for User
Identity Recognition,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in

Bioinformatics), vol. 11888 LNAI, no. March, Springer International Publishing, 2019, pp. 389–405.
[29] S. Kitic, G. Puy, P. Perez, and P. Gilberton, “Scattering features for multimodal gait recognition,” in 2017 IEEE Global Conference on Signal
and Information Processing (GlobalSIP), Nov. 2017, vol. 2018-Janua, pp. 843–847, doi: 10.1109/GlobalSIP.2017.8309079.
[30] Z. Meng et al., “Gait Recognition for Co-existing Multiple People Using Millimeter Wave Sensing,” in Proceedings of the AAAI Conference
on Artificial Intelligence, 2020, no. Vol 34 No 01, [Online]. Available: https://github.com/mmGait/people-gait.
[31] P. Zhao et al., “mID: Tracking and Identifying People with Millimeter Wave Radar,” in 2019 15th International Conference on Distributed
Computing in Sensor Systems (DCOSS), May 2019, pp. 33–40, doi: 10.1109/DCOSS.2019.00028.
[32] A. Turner and S. Hayes, “The Classification of Minor Gait Alterations Using Wearable Sensors and Deep Learning,” IEEE Trans. Biomed.
Eng., vol. 66, no. 11, pp. 3136–3145, Nov. 2019, doi: 10.1109/TBME.2019.2900863.
[33] P. Terrier, “Gait Recognition via Deep Learning of the Center-of-Pressure Trajectory,” Appl. Sci., vol. 10, no. 3, p. 774, Jan. 2020, doi:
10.3390/app10030774.
[34] O. Costilla-Reyes, R. Vera-Rodriguez, P. Scully, and K. B. Ozanyan, “Analysis of Spatio-Temporal Representations for Robust Footstep
Recognition with Deep Residual Neural Networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 2, pp. 285–296, Feb. 2019, doi:
10.1109/TPAMI.2018.2799847.
[35] B. Gálai and C. Benedek, “Gait Recognition with Compact Lidar Sensors,” in Proceedings of the 12th International Joint Conference on
Computer Vision, Imaging and Computer Graphics Theory and Applications, 2017, vol. 6, pp. 426–432, doi: 10.5220/0006124404260432.
[36] Y. Wang et al., “EV-Gait: Event-Based Robust Gait RecognitionUsing Dynamic Vision Sensors,” in 2019 IEEE/CVF Conference on Computer
Vision and Pattern Recognition (CVPR), Jun. 2019, vol. 2019-June, pp. 6351–6360, doi: 10.1109/CVPR.2019.00652.
[37] J. Luo and T. Tjahjadi, “Multi-Set Canonical Correlation Analysis for 3D Abnormal Gait Behaviour Recognition Based on Virtual Sample
Generation,” IEEE Access, vol. 8, pp. 32485–32501, 2020, doi: 10.1109/ACCESS.2020.2973898.
[38] P. Limcharoen, N. Khamsemanan, and C. Nattee, “View-Independent Gait Recognition Using Joint Replacement Coordinates (JRCs) and
Convolutional Neural Network,” IEEE Trans. Inf. Forensics Secur., vol. 15, pp. 3430–3442, 2020, doi: 10.1109/TIFS.2020.2985535.
[39] W. An, R. Liao, S. Yu, Y. Huang, and P. C. Yuen, “Improving Gait Recognition with 3D Pose Estimation,” Int. J. Comput. Sci. Inf. Secur.,
vol. 1, no. April 2019, pp. 137–147, 2018, doi: 10.1007/978-3-319-97909-0_15.
[40] R. Liao, S. Yu, W. An, and Y. Huang, “A model-based gait recognition method with body pose and human prior knowledge,” Pattern Recognit.,
vol. 98, p. 107069, Feb. 2020, doi: 10.1016/j.patcog.2019.107069.
[41] A. Sokolova and A. Konushin, “Pose-based deep gait recognition,” IET Biometrics, vol. 8, no. 2, pp. 134–143, Mar. 2019, doi: 10.1049/iet-
bmt.2018.5046.
[42] H. L. Tavares, J. B. C. Neto, J. P. Papa, D. Colombo, and A. N. Marana, “Tracking and Re-identification of People Using Soft-Biometrics,” in
2019 XV Workshop de Visão Computacional (WVC), Sep. 2019, no. March 2020, pp. 78–83, doi: 10.1109/WVC.2019.8876921.
[43] J. Luo and T. Tjahjadi, “Gait Recognition and Understanding Based on Hierarchical Temporal Memory Using 3D Gait Semantic Folding,”
Sensors, vol. 20, no. 6, p. 1646, Mar. 2020, doi: 10.3390/s20061646.
[44] C. Technology, “Multi-Level Feature Fusion for Robust Pose-Based Gait Recognition using RNN,” Int. J. Comput. Sci. Inf. Secur., vol. 18,
no. 1, pp. 20–31, 2020.
[45] W. Sheng and X. Li, “Siamese denoising autoencoders for joints trajectories reconstruction and robust gait recognition,” Neurocomputing,
vol. 395, pp. 86–94, Jun. 2020, doi: 10.1016/j.neucom.2020.01.098.
[46] N. Li, X. Zhao, and C. Ma, “A model-based Gait Recognition Method based on Gait Graph Convolutional Networks and Joints Relationship
Pyramid Mapping,” Arxiv, pp. 1–10, Apr. 2020, [Online]. Available: http://arxiv.org/abs/2005.08625.
[47] K. Jun, D.-W. Lee, K. Lee, S. Lee, and M. S. Kim, “Feature Extraction Using an RNN Autoencoder for Skeleton-Based Abnormal Gait
Recognition,” IEEE Access, vol. 8, pp. 19196–19207, 2020, doi: 10.1109/ACCESS.2020.2967845.
[48] Y. Makihara et al., “The OU-ISIR Gait Database Comprising the Treadmill Dataset,” IPSJ Trans. Comput. Vis. Appl., vol.4, pp. 53–62, 2012,
doi: 10.2197/ipsjtcva.4.53.
[49] Takemura, Noriko, et al. "Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition." IPSJ
Transactions on Computer Vision and Applications 10.1 (2018): 1-14"
[50] W. An et al., “Performance Evaluation of Model-based Gait on Multi-view Very Large Population Database with Pose Sequences,” IEEE
Trans. Biometrics, Behav. Identity Sci., vol. XX, no. Xx, pp. 1–1, 2020, doi: 10.1109/TBIOM.2020.3008862.
[51] Liang Wang, Tieniu Tan, Huazhong Ning, and Weiming Hu, “Silhouette analysis-based gait recognition for human identification,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. 25, no. 12, pp. 1505–1518, Dec. 2003, doi: 10.1109/TPAMI.2003.1251144.
[52] Shiqi Yu, Daoliang Tan, and Tieniu Tan, “A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait
Recognition,” in 18th International Conference on Pattern Recognition (ICPR’06), 2006, vol. 4, pp. 441–444, doi: 10.1109/ICPR.2006.67.
[53] M. Hofmann, J. Geiger, S. Bachmann, B. Schuller, and G. R. Institute, “The TUM gait from audio, image and depth (GAID) database,” J. Vis.
Commun. Image Represent., vol. 25, no. 1, pp. 195–206, 2014.
[54] R. Gross and J. Cohn, “The CMU Motion of Body (MoBo) Dataset,” Cmu-Ri-Tr-01-18, no. June, pp. 1–11, 2001.
[55] Y. Iwashita, R. Baba, K. Ogawara, and R. Kurazume, “Person Identification from Spatio-temporal 3D Gait,” in 2010 International Conference
on Emerging Security Technologies, Sep. 2010, pp. 30–35, doi: 10.1109/EST.2010.19.
[56] J. Bar th et al., “Stride Segmentation during Free Walk Movements Using Multi-Dimensional Subsequence Dynamic Time Warping on Inertial
Sensor Data,” Sensors, vol. 15, no. 3, pp. 6419–6440, Mar. 2015, doi: 10.3390/s150306419.
[57] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and Harnessing Adversarial Examples,” 3rd Int. Conf. Learn. Represent. ICLR 2015
- Conf. Track Proc., pp. 1–11, Dec. 2014, [Online]. Available: http://arxiv.org/abs/1412.6572.
[58] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and Harnessing Adversarial Examples,” 3rd Int. Conf. Learn. Represent. ICLR 2015
- Conf. Track Proc., pp. 1–11, Dec. 2014, [Online]. Available: http://arxiv.org/abs/1412.6572.
[59] Yuting Zhang, Gang Pan, Kui Jia, Minlong Lu, Yueming Wang, and Zhaohui Wu, “Accelerometer-BasedGait Recognition by Sparse Represen-
tation of Signature Points With Clusters,” IEEE Trans. Cybern., vol. 45, no. 9, pp. 1864–1875, Sep. 2015, doi: 10.1109/TCYB.2014.2361287.

[60] J. Frank, S. Mannor, J. Pineau, and D. Precup, “Time Series Analysis Using Geometric Template Matching,” IEEE Trans. Pattern Anal. Mach.
Intell., vol. 35, no. 3, pp. 740–754, Mar. 2013, doi: 10.1109/TPAMI.2012.121.
[61] S. Keshishzadeh, A. Fallah, and S. Rashidi, “Improved EEG based human authentication system on large dataset,” in 2016 24th Iranian
Conference on Electrical Engineering (ICEE), May 2016, pp. 1165–1169, doi: 10.1109/IranianCEE.2016.7585697.
[62] G. Gallego et al., “Event-based Vision: A Survey,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1, 2020, doi:
10.1109/TPAMI.2020.3008413.
[63] B. Kwolek, A. Michalczuk, T. Krzeszowski, A. Switonski, H. Josinski, and K. Wojciechowski, “Calibrated and synchronized multi-view
video and motion capture dataset for evaluation of gait recognition,” Multimed. Tools Appl., vol. 78, no. 22, pp. 32437–32465, Nov. 2019, doi:
10.1007/s11042-019-07945-y.
[64] R. Horaud, M. Hansard, G. Evangelidis, and C. Ménier, “An overview of depth cameras and range scanners based on time-of-flight technolo-
gies,” Mach. Vis. Appl., vol. 27, no. 7, pp. 1005–1020, Oct. 2016, doi: 10.1007/s00138-016-0784-4.
[65] A. Sucerquia, J. López, and J. Vargas-Bonilla, “SisFall: A Fall and Movement Dataset,” Sensors, vol. 17, no. 12, p. 198, Jan. 2017, doi:
10.3390/s17010198.
[66] D. Micucci, M. Mobilio, and P. Napoletano, “UniMiB SHAR: A Dataset for Human Activity Recognition Using Acceleration Data from
Smartphones,” Appl. Sci., vol. 7, no. 10, p. 1101, Oct. 2017, doi: 10.3390/app7101101.
[67] A. Özdemir, “An Analysis on Sensor Locations of the Human Body for Wearable Fall Detection Devices: Principles and Practice,” Sensors,
vol. 16, no. 8, p. 1161, Jul. 2016, doi: 10.3390/s16081161.
[68] T. Hoang, D. Choi, and T. Nguyen, “On the Instability of Sensor Orientation in Gait Verification on Mobile Phone,” in Proceedings of the
12th International Conference on Security and Cryptography, 2015, pp. 148–159, doi: 10.5220/0005572001480159.
[69] N. Khamsemanan, C. Nattee, and N. Jianwattanapaisarn, “Human Identification From Freestyle Walks Using Posture-Based Gait Feature,”
IEEE Trans. Inf. Forensics Secur., vol. 13, no. 1, pp. 119–128, Jan. 2018, doi: 10.1109/TIFS.2017.2738611.
[70] Connor, P., & Ross, A. (2018). Biometric recognition by gait: A survey of modalities and features. Computer vision and image understanding,
167, 1-27.
[71] Alharthi, A. S., Yunas, S. U., & Ozanyan, K. B. (2019). Deep learning for monitoring of human gait: A review. IEEE Sensors Journal, 19(21),
9575-9591.
[72] Topham, Luke K., et al. "Human Body Pose Estimation for Gait Identification: A Comprehensive Survey of Datasets and Models: A Com-
prehensive Survey of Datasets and Models." ACM Computing Surveys (2021).
[73] Prakash, C., Kumar, R., & Mittal, N. (2018). Recent developments in human gait research: parameters, approaches, applications, machine
learning techniques, datasets and challenges. Artificial Intelligence Review, 49(1), 1-40.
[74] M. Mao and Y. Song, "Gait Recognition Based on 3D Skeleton Data and Graph Convolutional Network," 2020 IEEE International Joint
Conference on Biometrics (IJCB), 2020, pp. 1-8, doi: 10.1109/IJCB48548.2020.9304916
[75] Singh, J. P., Jain, S., Arora, S., & Singh, U. P. (2021). A survey of behavioral biometric gait recognition: Current success and future perspec-
tives. Archives of Computational Methods in Engineering, 28(1), 107-148.
[76] Masupha, L., Zuva, T., Ngwira, S., & Esan, O. (2015, December). Face recognition techniques, their advantages, disadvantages and perfor-
mance evaluation. In 2015 International Conference on Computing, Communication and Security (ICCCS) (pp. 1-5). IEEE.
[77] Liu, S., & Silverman, M. (2001). A practical guide to biometric security technology. IT Professional, 3(1), 27-32.
[78] Zhu, Q., Yuan, N., Huang, J., Hao, X., & Zhang, D. (2019). Multi-modal AD classification via self-paced latent correlation analysis. Neuro-
computing, 355, 143-154.
[79] Zhang, Z., Tran, L., Yin, X., Atoum, Y., Liu, X., Wan, J., & Wang, N. (2019). Gait recognition via disentangled representation learning. In
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4710-4719).
[80] Zou, Q., Wang, Y., Wang, Q., Zhao, Y., & Li, Q. (2020). Deep learning-based gait recognition using smartphones in the wild. IEEE Transactions
on Information Forensics and Security, 15, 3197-3212.
[81] Zheng, J., Liu, X., Gu, X., Sun, Y., Gan, C., Zhang, J., ... & Yan, C. (2022). Gait Recognition in the Wild with Multi-hop Temporal Switch.
arXiv preprint arXiv:2209.00355.
[82] Khan, A., Javed, M. Y., Alhaisoni, M., Tariq, U., Kadry, S., Choi, J., & Nam, Y. (2022). Human Gait Recognition Using Deep Learning and
Improved Ant Colony Optimization. Comput. Mater. Cont, 70, 2113-2130.
[83] Giorgi, G., Saracino, A., & Martinelli, F. (2021). Using recurrent neural networks for continuous authentication through gait analysis. Pattern
Recognition Letters, 147, 157-163.
[84] Saber, S., Amin, K. M., & Adel Hammad, M. (2021). An Efficient Person Re-Identification Method Based on Deep Transfer Learning Tech-
niques. IJCI. International Journal of Computers and Information, 8(2), 94-99.
[85] Rida, Imad, Noor Almaadeed, and Somaya Almaadeed. "Robust gait recognition: a comprehensive survey." IET Biometrics 8.1 (2019): 14-28.
[86] Rida, Imad, Xudong Jiang, and Gian Luca Marcialis. "Human body part selection by group lasso of motion for model-free gait recognition."
IEEE Signal Processing Letters 23.1 (2015): 154-158.
[87] Rida, Imad, Somaya Almaadeed, and Ahmed Bouridane. "Gait recognition based on modified phase-only correlation." Signal, Image and
Video Processing 10 (2016): 463-470.
[88] Parashar, Anubha, et al. "Intra-class variations with deep learning-based gait analysis: a comprehensive survey of covariates and methods."
Neurocomputing (2022).
[89] Rida, Imad, et al. "Robust model-free gait recognition by statistical dependency feature selection and globality-locality preserving projections."
2016 39th International Conference on Telecommunications and Signal Processing (TSP). IEEE, 2016.
[90] Rida, Imad, Somaya Almaadeed, and Ahmed Bouridane. "Improved gait recognition based on gait energy images." 2014 26th International
Conference on Microelectronics (ICM). IEEE, 2014.
[91] Rida, Imad. "Towards human body-part learning for model-free gait recognition." arXiv preprint arXiv:1904.01620 (2019).
[92] Rida, Imad, et al. "Improved human gait recognition." Image Analysis and Processing—ICIAP 2015: 18th International Conference, Genoa,

Italy, September 7-11, 2015, Proceedings, Part II 18. Springer International Publishing, 2015.
[93] Rida, Imad, Somaya Al Maadeed, and Ahmed Bouridane. "Unsupervised feature selection method for improved human gait recognition."
2015 23rd European Signal Processing Conference (EUSIPCO). IEEE, 2015.
[94] Parashar, Anubha, et al. "Deep learning pipelines for recognition of gait biometrics with covariates: a comprehensive review." Artificial
Intelligence Review (2023): 1-65.
[95] Rida, Imad, et al. "Improved model-free gait recognition based on human body part." Biometric Security and Privacy: Opportunities &
Challenges in The Big Data Era (2017): 141-161.

Anubha Parashar is a Ph.D. candidate at Manipal University Jaipur, India. She received her M.Tech degree in Computer
Science and Engineering from VCE Rohtak, India, in 2016 and her B.Tech degree in Computer Science and Engineering
from PDMCE Bahadurgarh, India, in 2013. Her research interests include Gait Recognition, Biometrics, Deep Learning,
Computer Vision, Pattern Recognition, and IoT.
Apoorva Parashar is AI Consultant in Emerging technology, Mahindra Integrated Business Solutions, Mumbai. She has
done her M.Tech and B.Tech in Computer Science and Engineering from Maharshi Dayanand University, Rohtak, India.
She provides solutions in computer vision for the real-time processing of video analysis. Her research interests include
Computer Vision, Deep Learning, Medical Imaging, Biometrics, IoT, and Artificial Intelligence.
Andrea F. Abate currently serves as Full Professor with the University of Salerno from 2006, where he is team leader of
the Computer Graphics Laboratory. Dr. Abate is a member of the IEEE Haptics Technical Committee and a member of the
the International Association for Pattern Recognition. His current research interests include multibiometric systems, vir-
tual/augmented/mixed reality, haptics and human–computer interaction. He has authored many scientific papers published
in scientific journals and proceedings of refereed international conferences and co-edited one book. He currently serves as
Associate Editor for Pattern Recognition Letters and IEEE Access.
Rajveer S Shekhawat is currently the Dean and Director of SCIT, and Professor in the CSE department. He has been a
scientist for more than 20 years at the national research lab of Council of Scientific and Industrial Research (CSIR),Central
Electronics Engg Research Institute (CEERI, Pilani). He is recipient of prestigious fellowships by German Research bodies.
He received all the degrees from Birla Institute of Technology and Science (BITS) Pilani. PhD (Soft Computing) in 2002,
MS in 1994, B. Tech (EEE) in 1981, and M.Sc. (Hons) Physics. His research areas include Image Processing, Pattern
Recognition, Computer Vision, Deep Learning and IoT.
Imad Rida received the Ph.D. degree in computer science from Normandy University, MontSaint-Aignan, France, in 2017.
He is currently an Associate Professor with the University of Technology of Compiègne, Compiègne, France. His research
interests include machine learning, pattern recognition, and signal/ image processing.